Now that we have automated our deployment, it wouldn't be too hard to wire it with our code management setup. In this post, we will hook the Ansible scripts with our Git hosting setup so that a deployment gets triggered when you do a "git push". The idea is, deployment shouldn't be a chore, so that developers don't even think of it and only focus on the business logic of their application.
Also, if deployment is such a low activation energy task, you ship more frequently. Why ship frequently? because speed of iteration beats quality of iteration. There are a special class of tools which act as a hub between source code management and deployment, called continuous integration services. There are so many of them, but I picked up Gitlab CI for this exercise.
My criteria for choosing CI tool is it should be free as in free beer and free as in freedom. I should be able to look under the hood and make changes(though I don't do it usually), and if things break, I should be able to see what's happening behind the scenes. I was sort of tied between Jenkins and Gitlab CI, which shares the same traits. I went ahead with Gitlab CI as my code was hosted in Gitlab and adopting Gitlab CI was easier. Hint: If you are already using Gitlab, you're better off using Gitlab for the CI.
Also, there is no need to host and maintain a separate service if you are using Gitlab. The CI comes built in and you can create runner environments if your setup scales.
Gitlab CI overview
You check in the Gitlab CI configuration file as a part of your codebase. It is called
.gitlab-ci.yml at the top level directory.
This is a declarative YAML file which has information about how your CI process should execute.
Each Gitlab CI pipeline consists of one or more "jobs", with each job belonging to a "stage".
A stage is one of "build", "test" or "deploy". Multiple jobs belonging to the same stage are executed in parallel. Our CI pipeline only consits of the "deploy" stage,
stages: - deploy
with 2 jobs,
do_dev_deploy: stage: deploy . . . do_prod_deploy: stage: deploy
The job names are arbitrary, and the jobs run only if their condition is met, i.e. do a dev deploy only if code is pushed to
do_dev_deploy: stage: deploy image: python:3.6 only: - develop
Also, there is an
image directive which tells Gitlab CI to execute the job's tasks in a "python:3.6" container(and that's because we use Ansible).
Let's quickly walk through the deployment steps:
script: - pip install ansible # install ssh-agent - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' # run ssh-agent - mkdir -p ~/.ssh - eval $(ssh-agent -s) - ssh-add <(echo "$SSH_PRIVATE_KEY") - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config # deploy - ansible-playbook -i "$DEV_HOST," -u root ansible/playbook.yml --tags deploy --vault-password-file=./ansible/vault-env when: always
We install Ansible, and because Ansible requires an SSH key, we inject the key from an environment variable
$SSH_PRIVATE_KEY. Now, how can we securely inject this environment variable which holds a private ssh key into the CI pipeline? Gitlab allows you to do it by adding these variables via the UI, so that they are automatically inserted in every build.
The actual playbook execution also uses environment variables which are defined in the CI file itself.
variables: DEV_HOST: staging.example.com PROD_HOST: www.example.com
Notice that we only run the tasks in the playbook with the "deploy" tag. Also, we inject the Ansible Vault decryption password the same way as we did with the ssh private key, i.e. we add it as a CI variable. This is a secure and recommended practice.
Here's how your pipeline looks after successful execution.
Notice that the deployment job ends with a
when: always condition. This means that the job will execute irrespective of the result of the previous jobs. This can be removed to add a rule like "deploy only if the tests pass".
Where do all the jobs run for our pipelines? Gitlab provides a piece of infrastructure called runner. This is provided by Gitlab if you have hosted your code in Gitlab.com. You can also specify your own runners to augment your CI infrastructure.
There is more stuff we can do in Gitlab CI, which will be the scope of a later post. For current context, the Ansible playbook will suffice and does all the heavy lifting. Try doing a git push to your codebase and see the CD pipeline getting triggered automatically.
Where to go from here
Adding test steps in CI
You can add a
test stage and run all your tests, which will be executed before the deployment. You can fire a deployment only if all tests pass.
Adding multiple environments
You can tweak the Ansible scripts to create per-branch environments, also called review apps and manage their lifecycle using Gitlab CI and Ansible.
Traefik using docker swarm
The current Trafeik setup assumes a single machine running Docker. Your setup can quickly outgrow this if you are running lot of sites. To mitigate this, you can run your containers in a Docker swarm cluster and configure Traefik to this setup rather than a single node docker setup. More on this in a future post coming soon!
Adopting to a different CD pipeline
You can adopt this exact setup to a different CI/CD tool like Jenkins, Travis or Drone.
Pre baked images
You can inject the source code and composer dependencies and build a fresh docker image for every build(by building this on top of the existing PHP FPM image) during the
build stage of the CD pipeline and directly deploy this newly minted image during the deployment stage. I'll be writing a detailed post about this in the future.