Provisioning This Site

Published on: 2020-11-13

Introduction

This really culminates the articles "Making this site" (part 1, part 2 and part 3), surrounding the development of this blog. I've been terrible at getting round to doing this for a whole host of reasons, not least due to the onset of lockdown part two!

Ansible is a tool I've been using for years, almost as long as I've been doing devops style stuff over "plain old" software development. Alas, I've never been as involved as some in the development of it as it's moved from strength to strength. That said, one thing I've never seen a lot of is people exposing their ansible infrastructure or the methodologies behind it. I can, for sure, say that some of the influences for mine have come from implementations I've worked on for massive infrastructures, but I feel compelled to open mine up as an example people can use.

This too is true for this site, though the Lektor structure I have put together is very rudimentary from what I've seen around. There are much better examples, but it's a work in progress. Therefore this article focuses on the ansible setup I use to deploy the site, and from here on out it's just going to be an incremental process as the passage of time flows on!

The provisioning repository

The point of having a provisioning repository was to make it shareable. The repository is intended as a top level container that links together the full orchestration process via use of the Invoke tool. However, I've never quite finished automating this too well, but it does contain everything nicely.

Please do forgive how loosely everything is put together. If you follow instructions or inspect it, you should find some method to the madness, but this is very much just what was required for recreational usage...

Structure

We have the following:

ansible: the ansible code required to build everything with some generic-ish inventory, playbooks and role setups
ansible/environments: this should be a seperate repository designed to provide the var_files looked up from the playbooks (in my case this is very much a private repo!!!)
ansible/playbooks: you should know what these are if you're interested in ansible, very much the starting point of the documentation!!!
ansible/roles: this generally contains only my personally developed repositories, plus requirements from galaxy.
kickstarts: contains CentOS7 kickstarts for various scenarios.
tasks: defines all the actions Invoke can perform from the CLI
Some requirements and configuration files...

The idea is that you always invoke tasks from the this base directory. The only time you need to go anywhere else is to copy kickstarts to an http server (you only do that to begin with) and to edit stuff.

Each of the following sections goes through these areas of functionality in order...

Kickstarts

There were two originally drafted methods of deploying my online presence.

The first was originally to set up a libvirt host using an ansible role with a set of VMs. This used to make sense when I used a physical server in a data center, but it didn't work so well with a VPS since they didn't provide virtualisation extensions when I moved host (lesson learnt, but it's way cheaper.) In the end, I redeveloped the ansible to make both options available, though you should consider the vps approach the only mature one.

Invoke

This is not very mature, but I use it as an easy proxy for running ansible-playbook. Basically, I can run any playbook with some structure via the following:

invoke ansible -e <env> [-t <tag(s)>] playbook

The point here is that it automates vault inclusion and saves me the hassle of doing too much manipulating the commands I want to run. "Is it worth it" I hear you ask? Well, it certainly is if you use this same approach on massive infrastructures, as you can automate injection of parameters via invoke to drive the ansible inventories and var lookups, but this isn't a great example of that...

I also use invoke for vault editing, there's just some more efficient command line-fu you can get too.

With the release of Fabric v2 I thought it would be easy to migrate, but I vaguely remember it wasn't as trivial as I hoped. Therefore, I just install invoke on its own. Though I love Fabric, Invoke and Paramiko, the developments and integrations within the ecosystem over the recent years have been a little hard to keep track of!

Ansible

It really is just ansible-playbook initiating playbooks. However, it's worth noting that recently the focus under RedHat has seen the adoption of collections, which I'm yet to understand, as well as more complexity in the underlying system.

You'll notice at the end of playbooks I have the following snippet:

  vars_files:
    - ../environments/all.yml
    - ../environments/data.yml
    - "../environments/{{ env }}/all.yml"
    - "../environments/{{ env }}/data.yml"
    - "../environments/{{ env }}/vault.yml"

The reason I do this is to make very explicit the variable hierarchy that I'm working to. env is provided by the invoke command line invocation (eg. staging or production at the moment, I've never bothered getting round to making docker work for my personal development process, as I've no requirement for it just yet, but there is something in the pipeline that would justify the effort...)

This is probably going to annoy ansible pro's, but I find it very important. A lot of the time with things like cloud-init, there's no reason to have host_vars. I consider it bad practice myself, for example web servers should be set up when scaling out horizontally and the orchestration process should provide host level variables, not the ansible repo.

I have in the past investigated the variable precedence in ansible and find it irritating to recall, especially as I seem to remember it changing or extending, or otherwise not suiting my group-oriented approach. By using var_files you can easily overcome the need to research this and limit the hierarchy to a set number of levels.

This is one thing I liked from Puppets Hiera (I don't like Puppet because of the DSL and it's huge amount of changes over the years), which asked you to DEFINE the hierarchy. When a data hierarchy is driving the configuration managements effect on deployment, it needs to be easily understood and obviously specified. Also, in my opinion, working with people of varying amounts of experience with such systems, a limit on the amount of levels is no bad thing.

Hopefully that explains the design a little. The data drives the code, which largely (there is an exception for an LetsEncrypt ACME deployment I've not yet moved into a role) is functionally defined by roles, included from playbooks...

Simples! ;-)

Environments

This is a private repository for me, but lives in the location mentioned above: ansible/environments and is structured to meet the requirements of var_files in the playbooks. Though I won't reveal the setup I have, it's up to the user of this repo to define what's necessary based on the knowledge of technologies contained therein to define the data in a way that deploys they way they want.

That would be a very large article, so for the moment I'll just show you the structure of my repo so you know how to start, then you can define based on identifying key variables from the underlying roles.

./data.yml
./development
./development/all.yml
./host.yml
./production
./production/all.yml
./production/data.yml
./production/proxy.yml
./production/vault.yml
./production/web.yml
./staging
./staging/data.yml
./staging/web.yml
./staging/host.yml
./staging/proxy.yml
./staging/vault.yml
./staging/all.yml
./web.yml
./proxy.yml
./all.yml

Ignore the development environment in my case, I never did anything with it, as I said before! ;-)

Improvements I've never made

Here's a load of things I'd like to do, but never did:

I never created a kickstart hosting invoke task to save on having kickstarts deployed - never needed it yet, but easy to do
I intend to use docker for playing about with the web, data, proxy structure: but that was pre-VPS
I've never quite settled on whether include_role, include_tasks, import_role or import_tasks is the way to go. It's largely because I find tags EXTREMELY useful for rerunning tasks, such as renewing certificates, and imports don't work properly. If I'm honest I've just never solidly bothered to read the docs since they added all these in. Though I think things are working, who knows for some of it...
I really should get round to publishing the ISSO role I wrote, which seems to work well for me!!! I love ISSO, it offers comments without selling people out for leaving them...
I still haven't worked out how I feel about cookies, especially in relation to analytics. So at the moment this should be largely cookie free.
The blog section is the only part really completed so far: the projects need deriving from github, the music section doesn't exist yet and I want to offer a better way of accessing my Antarctic galleries without having to trawl through posts.
I do a lot in my job that I want to blog about, far removed from what this site is so far about. Give it time...
Probably should have an RSS feed or whatever the vibe is these days! ;-)
There are loads of things to consider, especially as I have other web applications to bring under here. The staging environment becomes far more complex as a result (it already is). By inference I should split the role requirements based on the ansible environment being deployed....

The ansible I write is completely idempotent (as far as I've ever realised), so you should be safe to run over and over and over and over again... provided you configure environments properly...

Conclusion

Since we're in the middle of lockdown number two, I'm happy I've concluded this piece on building the VPS out with Lektor. I really hope people find things useful, but also welcome comments about what information I've left out.

There are plenty more articles on the way, but hopefully this has got me caught up now things are working properly!!!

Table Of Contents