A Quick Guide to Building a Custom Docker Image for CI

Most of today’s popular CI services support running jobs in arbitrary Docker containers. In this post, I’ll describe why Docker is such a great fit for CI and how it can be made even better with custom images.

Why Docker

Docker is great for CI for several reasons:

• Speed: Containers spin up very quickly.
• Consistency: Docker images provide a consistent environment that changes on your schedule. If you need it, you can still spin up something old like ubuntu:14.04, even as the cloud host evolves its infrastructure to follow the market.
• Tailor-made: Images can be targeted to your specific needs (e.g. node:12 or mcr.microsoft.com/dotnet/core/sdk:3.1 or circleci/android:api-29)

This is a big improvement over running CI on general-purpose VMs or natively on machines you manage, but it gets even better.

Why Custom Docker

The idea behind a custom image is that you can take an existing image meant for your language ecosystem, and then apply specific changes for your project.

If your CI job configuration has accumulated a handful of setup steps, especially if they take a while to run and don’t change often, then you’re a good candidate for a custom image. Perhaps you need to install a particular version of a compiler or SDK, download a proprietary CLI tool, or precompile a big slow native dependency.

If you’ve never built a custom image before, follow along! It’s easier than I expected.

How

A Docker container is a running instance of a Docker image. A Docker image is defined in a Dockerfile. Let’s write one!

1. Begin by installing and starting Docker Desktop if you don’t already have it.
2. Choose an upstream image. If your CI job is already running in Docker, you probably want to start with its existing image. Otherwise, you can search Docker Hub for your language, tool, or Linux distribution of choice.
3. In an empty directory, create a text file called Dockerfile, and put your upstream image at the top behind a FROM instruction:

   
   FROM ubuntu:latest
   

4. Build it with docker build . -t my-image:

   > docker build . -t my-image
   Sending build context to Docker daemon  2.048kB
   Step 1/1 : FROM ubuntu:latest
    ---> 775349758637
   Successfully built 775349758637
   Successfully tagged my-image:latest
   

5. Shell into it with docker run -it my-image:

   > docker run -it my-image
   root@a27df4742348:/# cat /etc/lsb-release
   DISTRIB_ID=Ubuntu
   DISTRIB_RELEASE=18.04
   DISTRIB_CODENAME=bionic
   DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"
   

   This interactive shell is an ephemeral sandbox, so it’s a great place to try out the setup steps that you’d like to bake into your CI image. Once you know what commands you need, add them to the Dockerfile behind RUN instructions:

   
   FROM ubuntu:latest
   
   RUN apt-get update && apt-get install -y clang
   

   Rebuild it with the same docker build command until you’re satisfied with the results.

6. Push your new image to a registry. For Docker Hub, start with docker login, then tag your image according to your user or organization name:

   docker tag my-image:latest my-username/my-image:latest

   And then push it with:

   docker push my-username/my-image:latest

   (You can also use alternative registries like Amazon’s ECR or Microsoft’s Azure Container Registry.)

7. Finally, update your CI config to reference your new image, kick off a new build, and enjoy the improvement.

Conclusion

This approach works with CircleCI, GitHub Actions, GitLab CI, BitBucket Pipelines, Azure Pipelines, and probably others.

For concrete examples, I’ve had good outcomes using it to add Mono to a ruby-node image and to build an embedded project with Espressif’s Xtensa GCC, shaving minutes off project build times.

Could your project benefit from a custom image? Let me know in the comments!