Automate deployment with Ansible

Ansible allows you to specify instructions in yaml format, called playbooks, that will be executed on remote hosts, specified in so called inventory files, and will generate reproducible configuration of the host. I won’t be going into many details of how Ansible works or it should be configured as you can consult the documentation and other online resources on it in case you are interested. Instead, I will offer you a gist of what it takes to provision and deploy a Django application with Ansible.

Provisioning

The provisioning step sets up the host and installs all required third party software that your application relies on. In this step you might want to install typical software like Git, Supervisor, Pyenv, Nginx, etc… You also want to set up a separate user for your app, separate from the one that you use to ssh on the node, to isolate the application from the rest of the system and not accidently allow it to perform actions with superuser privileges. I like to give the user the same name as my app to avoid any confusion.

The steps to achieve this have already been described in previous posts, but here is the summary:

Update package lists
Create a user and a group that will be the owner of all application related files and folders
Install git, which is required to clone our repository from GitHub. We could also pull it locally, and copy it over to the host, but this takes generally longer.
Install pyenv as described in this post

Here is how my provision.yml would look like

Now that you have set up your host, you can proceed to deploying your application to the remote host.

Deployment

During deployment you make sure that your application’s source code is transfered to the remote host and all third party software is configured in such a way that they are able to interact with each other and your application. As described in Deploy your app on a VM you have three options:

You clone your code locally and copy it to the remote host
You create a ssh key on your remote host and add it to your SSH keys in GitHub
You create a personal access token and clone your repo directly from the remote host I won’t go into more details with the second method since it is pretty straightforward. The other two have interesting implications though, that are worth taken a look at in more detail:
- How can you set the correct file permissions when copying files with Ansible?
- How can you *encrypt secrets like the the access token during playbook execution?

Copy files to the remote host with Ansible

The biggest issue I had when following this approach is setting the correct file permissions. For this you have to understand the concept between Ansible’s variables ansible_user, ansible_become, become and become_user.

ansible_user is the user with which you can log in to the remote host and you have set up as of Set up a VirtualBox VM
ansible_become can be set on inventory level, and tells Ansible that for this host, every action should be performed as if the variable become is set to “true”. Else, every action is perfomed as the ansible_user on the remote host
become tells Ansible that it should become a specific user, and this is root in case become_user is not specified
become_user specifies the specific user that Ansible should become, for the execution of the task, block, playbook or role. Obviously, this user should also be present on the remote host, which should have been set up during the provisioning step.

If this concept is clear, then you can easily copy your files from your local machine to the remote host in three steps (make sure to read the code comments):

Check out the desired version (branch, tag or commit hash) on your local machine
Create a directory on the remote host
Copy the files from your local host to that directory

Clone a private repository from GitHub

In the post Deploy your app on a VM I have described how to create an access token for GitHub, with which you can pull the code from private repositories. Like other secrets and passwords, you don’t want to keep them in plain text, especially not if you plan to push your ansible repo into some VCS. Ansible comes on installation with something called ansible-vault that lets you encrypt strings and contents of files, and decrypts it during playbook execution. To encrypt your access token for example, you could execute something like:

ansible-vault encrypt_string --vault-id @prompt "..." --name "vault_github_access_token"

Then, inside your playbook, you could have a task that checks out the repository like this one:

- name: Deploy app
  hosts:
    - vbox
  
  tasks:
    - name: Clone repo
      become: true
      become_user: "{{ app_user }}"
      ansible.builtin.git:
        repo: "https://{{ vault_github_access_token }}@github.com/<GitHub username>/<repo name>.git"
        version: "{{ version }}"
        dest: "/path/to/repo/"
    
    # ... to be continued

And you can execute the playbook with

ansible-playbook playbook.yml --ask-vault-password -e version=<hash|tag|branch>

Notice that –vault-id @prompt and –ask-vault-password are equivalent methods of telling ansible-vault that it should use the default vault-id. You can specify specify different vault-ids like this: --vault-id production@prompt.

Install requirements with pip and pyenv

Now that we have our code on the remote host, we need to set up the virtual environment for our application. This consists of following steps:

Get the commit hash of the currently checked out repository
Ensure the required python version is installed
Create a new virtual environment with a command equivalent to pyenv virtualenv <python version> <commit hash>
Install requirements with pip from a requirements.txt file

This is how these tasks would look like with Ansible:

- name: Deploy app
  hosts:
    - vbox

  tasks:

    # ... clone or copy repo
    
    - name: Get commit hash
      become: true
      become_user: "{{ APP_USER }}"
      ansible.builtin.command:
        cmd: "git rev-parse HEAD"
        chdir: "{{ app_repo_dir }}"
      register: git_hash
      changed_when: false

    - name: Install python version
      become: true
      become_user: "{{ APP_USER }}"
      ansible.builtin.command:
        cmd: "{{ pyenv_executable }} install {{ python_version }} --skip-existing"
      register: output
      changed_when: output.stderr != ""

    - name: Set facts
      ansible.builtin.set_fact:
        virtual_env: "{{ pyenv_root }}/versions/{{ python_version }}/envs/{{ git_hash.stdout }}"

    - name: Check if virtual environment already exists
      ansible.builtin.stat:
        path: "{{ virtual_env }}"
      register: stat_result

    - name: Create virtual environment
      become: true
      become_user: "{{ APP_USER }}"
      ansible.builtin.command:
        cmd: "{{ pyenv_executable }} virtualenv {{ python_version }} {{ git_hash.stdout }}"
      when: not stat_result.stat.exists
      changed_when: not stat_result.stat.exists

    - name: Install requirements
      become: true
      become_user: "{{ APP_USER }}"
      ansible.builtin.command:
        cmd: "{{ virtual_env }}/bin/pip install -r requirements/development.txt"
        chdir: "{{ app_repo_dir }}"
      changed_when: true
    
    # ... to be continued

Our code is on the remote host, the pyton environment is set up, now all that is left to do is configure Systemd and Gunicorn to run your application.

Configure Systemd and Gunicorn to serve your application

Now you need the following files inside the /srv/<app name>/ folder, as described in the previous post Deploy your app on a VM:

/srv/<app name>/
├── gunicorn.conf.py
├── run
└── .env

and a <app name>.service file inside the /etc/systemd/system/ directory that will run your application in the background with Systemd. To create those files on the target host with Ansible, I would suggest to use the ansible.builtin.template module. Check out the full example to see how it can be used.

Full example

Here is the full example of the playbook, that you can execute with

ansible-playbook playbook.yml --ask-vault-password -e version=<hash|tag|branch>

Your application should now be running on the IP address of your remote host, port 80.