Dockerize Your Development Environment

This post talks about hosting multiple development environments with IDE's on a single machine using Docker. Along the post we'll write a simple Dockerfile to host a Scala project and we'll also install an IDE into the docker container by sharing the X11 socket with the host machine. But first, a bit of history.

A bit of history

As a software developer, I'm jumping between many different software projects. Many of the projects I've worked on had different tech choices, from Java, Scala, Ruby, Python to NodeJs and even C. Before there were VMs, I was installing all the necessary tools to my local machine, under my default user. As you'd expect, my machine was becoming cluttered pretty quick. It got worse when I had to deal with different versions of the same tool for different projects. When the projects are on Ruby or Python there were solutions such as rbenv or virtualenv, but even then managing all those different versions was not easy. There were always some issues. Having all those projects under a single OS was a mess. A mess that cries for a better solution. That's when I discovered Vagrant, and soon enough I built a vagrant box for each project I had on my laptop. They were all running independent of each other and I was quite happy with the result.

My vagrant solution was using lxc under the hood, and it was as fast as the bare machine. It also shared the X11 socket, so that any application ran under my vagrant box would be able to use X11, thus draw graphics. So it seemed like a perfect solution, until I realized that now I am running out of disk space. Every vagrant box I ran, each project I work on had to install a fresh OS, libraries for the project and one or more IDEs. Even if I already have those tools and IDE's installed for another project (thus vagrant box), being independent, every vagrant box had to have them again. My disk was filling up with unnecessary copies of the same stuff. This had to change.

I knew Docker was using AUFS, and thanks to it, Docker does not copy duplicated FS blocks again, saving lots of precious space. So I replaced lxc with Docker (which also utilizes lxc under the hood) and got a flawless development machine. Now I would like to share it with you. Just to keep it short I won't talk about the Vagrant configurations, or the way I provision my projects using Ansible. From now on I'll just talk about Docker.

Let the games begin

For the sake of this blog post we will be working on a simple Scala project (I pick Typesafe scala-logging project for its small size), and we will install IntelliJ IDEA as an IDE. You may pick different a different project (possibly yours) and a different IDE.

In order to work on a new or an existing software project, we have to create a Docker container for it. A great way to create a Docker container is to write a Dockerfile for it.

Lets start with a simple one:

FROM ubuntu

RUN apt-get update
RUN apt-get install -y openjdk-7-jdk

Save it as a file named Dockerfile and create an image out of it by (do not forget the dot at the end),

docker build -t ubuntu_java .

Now you can run a container from this image by,

docker run --rm -it ubuntu_java /bin/bash

After all builds and installs are finished, you will be on the terminal of you container. You may want to take your time there, just exit once you are ready to continue.

Ok, let me briefly tell you what we did here. The Dockerfile you just created, tells Docker to fetch ubuntu image and run the commands in order. The commands install openjdk-7. As you build the docker image using docker build command, this Dockerfile is read, and a java 7 installed docker image based on the ubuntu is created. Then by the run command you started a container based on this newly created docker image.

Lets jump ahead and add other necessary packages for our project to run.

FROM ubuntu

RUN echo "deb http://dl.bintray.com/sbt/debian /" > /etc/apt/sources.list.d/sbt.list
RUN apt-get update
RUN apt-get install --force-yes -y openjdk-7-jdk git sbt

Nothing new here, moving on.

Now that we have everything we need to build our project lets fetch it and see if it builds ok.

FROM ubuntu

RUN echo "deb http://dl.bintray.com/sbt/debian /" > /etc/apt/sources.list.d/sbt.list
RUN apt-get update
RUN apt-get install --force-yes -y openjdk-7-jdk git sbt

RUN git clone https://github.com/typesafehub/scala-logging.git ~/scala-logging

Make sure you build docker image again.

docker build -t myproject .

And run a container.

docker run --rm -it myproject /bin/bash

if you cd ~/scala-logging and type sbt test, you should be able to build and test the project.

Ok now to summarize what we did so far, we wrote a Dockerfile to install our project’s dependencies such as java, and sbt, then we fetch the project from git and confirmed that all is good by running sbt test to compile the project. Note that, you might want to keep your project files on a separate Docker volume in order to separate data from applications, however to keep things short I am not getting into details here.

Great! Now we can build and test our project. But how can we install an IDE to continue developing it.

Where are my graphics?

If you are using an IDE with a GUI, such as Eclipse, NetBeans, IDEA or anything other than vim or emacs, you will need some more settings before fully utilizing your new Docker environment.

Lets start sharing our X11 socket, so that applications on your Docker environment can also draw on your host machines window system.

Although there are many ways to let a remote machine draw on local window system, such as forwarding through ssh, using network to pass X11 traffic, or sharing the X11 unix socket with XAuthority magic cookie, we will use the simplest and a bit hacky way. We will trick the X window system to think that we are the same user as the host machine and use the same X11 socket that the host system uses.

The first thing we need is the UID and GID of our host systems users. In general those IDs are 1000, but they can be different. So please check yours using the id command line utility. Once we got those ids open up your Dockerfile and add these lines on top, just after the FROM ubuntu line,

...

RUN apt-get install -y x11-apps

ENV USER=myuser UID=1000 GID=1000  # /etc/sudoers.d/${USER}
RUN chmod 0440 /etc/sudoers.d/${USER}

...

Ok, the first line we have is to install some X11 apps for diagnosing if everything works ok. The other lines are for creating a user with the same UID and GID as the user on your host machine. We will use this user later to trick your X11 window system. Please make sure you replace all three fields with your information.

The final Dockerfile should look like this,

FROM ubuntu

RUN apt-get install -y x11-apps

ENV USER=myuser UID=1000 GID=1000  # /etc/sudoers.d/${USER}
RUN chmod 0440 /etc/sudoers.d/${USER}

RUN echo "deb http://dl.bintray.com/sbt/debian /" > /etc/apt/sources.list.d/sbt.list
RUN apt-get update
RUN apt-get install --force-yes -y openjdk-7-jdk git sbt

RUN git clone https://github.com/typesafehub/scala-logging.git ~/scala-logging

Lets add a USER directive just before the git clone in the Dockerfile to switch our new user, before fetching our project. Make sure you use the same username as you used above.

USER myuser

Rebuilding the docker image and running it will let us use the window system on our host machine. However, this time, while we are running the container we need to share the X11 socket as well. Here is how,

Building the image as usual

docker build -t myproject .

Run it as follows,

docker run --rm -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -it myproject /bin/bash

So with -v parameter, you share the X11 socket found on your host machine with the docker container. X11 socket is found under /tmp/.X11-unix and you mount it to the same place in your Docker environment.

-e parameter sets the necessary DISPLAY environment variable. This is what the X reads to pick the display to draw on.

Now if you ran that command you can call xeyes to have some eyes on your host machine. Whoa they track your mouse cursor :blush:

The last step is to install our IDE. It should be obvious now. We will download the package and install. Here are the commands I used,

RUN sudo apt-get install -y wget
RUN wget "http://download.jetbrains.com/idea/ideaIC-14.0.3.tar.gz" -O ~/idea.tgz
RUN cd ~ && tar -xzvf ~/idea.tgz

Once you’re done. Build your docker image one final time, and run it.

You can then run idea.sh under your home and voila!. You’ll be able to use your IDE just as you were.

Conclusions

In this post I wrote about how to utilize docker to host your different software projects in the same machine. I also showed how to share the X window system between your host and guest machines, thus letting you run things like IDE’s.

To keep things short I didn’t mention about keeping project code/data in a separate Docker volume or using Vagrant to manage multiple Docker containers for complex projects. You may want to check out Docker and Vagrant sites for more info.

Keep us posted about your experiences on Docker...

Note: You may find the full Dockerfile here: Gist