Confd and Docker - Separating Config and Code for Containers

tl;dr bootstrap Docker containers with confd and environment variables to set their run-time configuration.

Config Management tools like Puppet and Chef have methods to separate infrastructure code from infrastructure config. Storing config data in Hiera or Databags allows modules and cookbooks to be easily re-used and shared between environments, teams and with the open-source community.

Can we apply similar patterns with Docker?

Perhaps the Dockerfile represents the infrastructure code. But how might we go about applying custom configuration to shared Docker Images?

environment variables - Docker makes it easy to set environment variables when starting containers. This might be sufficient as a source of config for some situations, but many applications and software are configured with files rather than just environment variables.

data volumes - Docker data volumes can be used to share files or directories between Docker hosts and containers. Although it’s typically used to enable containers to persist data or logs to the Docker host, it could equally be used to share configuration files from the host to a container. Unfortunately as you scale up the number of applications, environments and Docker hosts that you’re managing this approach becomes more burdensome. There are also some Docker environments where you may not have access to create and manage files on Docker hosts.

If only there was way to manage configuration files in containers in a manner that was as simple and straight forward as container environment variables…

Say hi to confd

confd is a tool that uses templates and data to render local configuration files.

The data can be sourced from a number of key-value databases like etcd, consul, dynamodb, redis or zookeeper in order to keep local files up to date with changing data. While storing config data in a network key-value database brings some benefits, it also introduces a level of complexity that not everyone is willing to adopt in a production environment.

If you want a simpler option, confd can also source its data from local environment variables. Let’s explore how confd and environment variables can be applied to configure Docker containers at run-time. A sample Docker project can be found here.

Here are the steps involved.

1. Install confd in a Docker image

Add the following lines to your Dockerfile to install the confd executable in your Docker image.

ADD https://github.com/kelseyhightower/confd/releases/download/v0.10.0/confd-0.10.0-linux-amd64 /usr/local/bin/confd
RUN chmod +x /usr/local/bin/confd

2. Create confd templates

confd templates are written using Go’s “text/template” package. The example below inserts the $proxy_domain and $backend_host environment variables into the generated file.

Store your confd templates in the same repo as your Dockerfile. Then simply add them to your Docker image by adding the following line to your Dockefile.

ADD files/confd /etc/confd

3. Create a container bootstrap script

Create a simple shell script to perform the following

validate required environment variables (exit if not set)
execute confd to create config files
launches the desired container process

Here’s an example.

#!/bin/bash
set -e  

# if $proxy_domain is not set, then default to $HOSTNAME
export proxy_domain=${proxy_domain:-$HOSTNAME}

# ensure the following environment variables are set. exit script and container if not set.
test $backend_host

/usr/local/bin/confd -onetime -backend env

echo "Starting Nginx"
exec /usr/sbin/nginx -c /etc/nginx/nginx.conf

Now add the container bootstrap script as the Command in your Dockerfile

CMD ["/start.sh"]

4. Start your container

Run the following command to start a container with desired environment variables to be used by confd

docker run -it -p 80:80 \
-e proxy_domain="awesome.app.local" \
-e backend_host="app01.domain.local" \
nginx-confd

2015-08-19T16:06:06Z 5b35bcced635 /usr/local/bin/confd[5]: INFO Backend set to env
2015-08-19T16:06:06Z 5b35bcced635 /usr/local/bin/confd[5]: INFO Starting confd
2015-08-19T16:06:06Z 5b35bcced635 /usr/local/bin/confd[5]: INFO Backend nodes set to
2015-08-19T16:06:06Z 5b35bcced635 /usr/local/bin/confd[5]: INFO Target config /etc/nginx/conf.d/proxy.conf out of sync
2015-08-19T16:06:06Z 5b35bcced635 /usr/local/bin/confd[5]: INFO Target config /etc/nginx/conf.d/proxy.conf has been updated
Starting Nginx

We now have nginx running in a container with configuration files created with data from environment variables.

More than strings

What if you want to update configuration files with more than simple strings of text. Perhaps you have a group of servers you wish to set in a file where the number of them is N.

1 solution is to use the split function in golang’s string package. Now a list of field separated objects can be set as a single environment variable, and the confd template can split it and iterate over each created slice.

Here’s an example.

Set an environment variable with a seperator like :

app_servers="myapp01.local:myapp02.local:myapp03.local:myapp04.local"

Create a confd template that splits the $app_servers environment variable based on the specified seperator :, and iterates over the slices in a for range loop.

The following is the file generated by confd

upstream backend_hosts {

    server myapp01.local;
    server myapp02.local;
    server myapp03.local;
    server myapp04.local;
}

We now have a flexible way to create customised configuration files within containers driven by simple old environment variables.

Michael Richardson