Docker or: How I Learned To Stop Worrying And Love Deployment

By Martin Splitt posted October 31, 2014

Our development workflow

We're working with a number of feature branches in parallel. To be able to foster feedback and communication about the different features being worked on, we'd like to have a discoverable list of links to a running version of each feature branch. This could look something like this:

That picture shows what you get at, say, testing.example.com. There are three versions:

Each being deployed at testing.example.com/, for example testing.example.com/interior_caching takes you to a running version of that feature branch.


Ingredients: Docker, etcd and confd

At the core of the solution are three things:


The deployment workflow

The interaction between these three and our webserver will look like this: continuous deployment flow

Step by step this will happen:

  1. A continuous integration build finishes successfully
  2. A new docker image is created using the branch as its tag
    1. If the build belongs to a new branch a new key is created within etcd
    2. If the build belongs to an existing branch, the previous container is removed and the corresponding etcd key is updated
  3. confd picks up the changes in etcd and rewrites the HTML page and the web server configuration accordingly
  4. Our web application is (re-)started, properly proxying to the new containers

Putting it all together

JENKINS

We're using Jenkins as our continuous integration server and our Jenkins "post build" task is something like this:

Jenkins Archilogic

CONFD

For confd we'll have 4 files:

The first two files contain configuration to tell confd what to observe in etcd, which files to rewrite and what service to reload on changes. The last two files are the configuration file templates that are used to rewrite the actual configuration files.

CONFD Archilogic


Wrap up

With Jenkins, etcd and confd properly set up, we're now running a docker container for each feature branch, like this: docker-ps

And each container has its internal, virtual IP announced to etcd like this: etcd

("/web" is the etcd key that is observed by confd)

With this in place, we can share, test, exchange thoughts and collaborate on the different active feature branches and be certain that they work outside our development server environments.

We can, hypothetically, also use the created docker images to deploy the exact same codebase and configuration into production - but that's a future experiment.