In the past year, process isolation through the use of containers has exploded and you can find containers for almost anything these days. So why not creating a container to isolate your CherryPy application from the rest of the world?
I will not focus on the right and wrongs in undertaking such a task. This is not the point of this article. On the other hand, this article will guide you through the steps to create a base container image that will support creating per-project images that can be run in containers.
We will be using docker for this since it’s the hottest container technology out there. It doesn’t mean it’s the best, just that it’s the most popular which in turns means there is high demand for it. With that being said, once you have decided containers are a relevant feature to you, I encourage you to have a look at other technologies in that field to draw your own conclusion.
Docker uses various Linux kernel assets to isolate a process from the other running processes. In particular, it uses control groups to constraints the resources used by the process. Docker also makes the most of namespaces which create an access layer to resources such as network, mounted devices, etc.
Basically, when you use docker, you run an instance of an image and we call this a container. An image is mostly a mille-feuille of read-only layers that are eventually unified into one. When an image is run as a container, an extra read-write layer is added by docker so that you can make changes at runtime from within your container. Those changes are lost everytime you stop the running container unless you commit it into a new image.
So how to start up with docker?
Getting started
First of all, you must install docker. I will not spend much time here explaining how to go about it since the docker documentation does it very well already. However, iI encourage you to:
- install from the docker repository as it’s more up to date usually than official distribution repositories
- ensure you can run docker commands as a non-root user. This will make your daily usage of docker much easier
At the time of this writing, docker 1.4.1 is the latest version and this article was written using 1.3.3. Verify your version as follow:
$ docker version Client version: 1.3.3 Client API version: 1.15 Go version (client): go1.3.3 Git commit (client): d344625 OS/Arch (client): linux/amd64 Server version: 1.3.3 Server API version: 1.15 Go version (server): go1.3.3 Git commit (server): d344625
Docker command interface
Docker is an application often executed as a daemon. To interact with it you use the command line interface via the docker command. Simply run the following command to see them:
$ docker
Play a little with docker
Before we move on creating our docker image for a CherryPy application, lets play with docker.
The initial step is to pull an existing image. Indeed, you will likely not create your own OS image from scratch. Instead, you will use a public base image, available on the docker public registry. During the course of these articles, we will be using a Ubuntu base image. But everything would work the same wth Centos or something else.
$ docker pull ubuntu Pulling repository ubuntu 8eaa4ff06b53: Download complete 511136ea3c5a: Download complete 3b363fd9d7da: Download complete 607c5d1cca71: Download complete f62feddc05dc: Download complete Status: Downloaded newer image for ubuntu:latest
Easy right? The various downloads are those of the intermediary images that were generated by the Ubuntu image maintainers. Interestingly, this means you could start your image from any of those images.
Now that you have an image, you may wish to list all of them on your machine:
$ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE ubuntu latest 8eaa4ff06b53 10 days ago 188.3 MB
Notice that the intermediate images are not listed here. To see them:
$ docker images -a
Note that, in the previous call we didn’t specify any specific version for our docker image. You may wish to do so as follow:
$ docker image ubuntu:14.10 Pulling repository ubuntu bf49414948ac: Download complete 511136ea3c5a: Download complete a7cca9443999: Download complete dbbd544a49e2: Download complete 98b540cf0569: Download complete Status: Downloaded newer image for ubuntu:14.10
Let’s pull a centos image as well for the fun:
$ docker pull centos:7 Pulling repository centos 8efe422e6104: Download complete 511136ea3c5a: Download complete 5b12ef8fd570: Download complete Status: Image is up to date for centos:7
Let’s now run a container and play around with it:
$ docker run --rm --name playground -i -t centos:7 bash [root@7d5761d100e4 /]# ls bin dev etc home lib lib64 lost+found media mnt opt proc root run sbin selinux srv sys tmp usr var
In the previous command, we start a bash command executed within a container using the Centos image tagged 7. We name the container to make it easy to reference it afterwards. This is not compulsory but is quite handy in certain situations. We also tell docker that it can dispose of that container when we exit it. Otherwise, the container will remain.
[root@7d5761d100e4 /]# uname -a Linux 7d5761d100e4 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
This is interesting because it shows that, indeed, the container is executed in the host kernel which, in this instance, is my Ubuntu operating system.
Finally below, let’s see the network configuration:
[root@7d5761d100e4 /]# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 12: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff inet 172.17.0.3/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:3/64 scope link valid_lft forever preferred_lft forever
Note that the eth0 interface is attached to the bridge the docker daemon created on the host. The docker security scheme means that, by default, nothing can reached that interface from the outside. However the docker may contact the outside world. Docker has an extensive documentation regarding its networking architecture.
Note that you can see containers statuses as follow:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 25454ad13219 centos:7 "bash" 4 minutes ago Up 4 minutes playground
Exit the container:
[root@7d5761d100e4 /]# exit
Run again the command:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
As we can see the container is indeed gone. Let’s now rewind a little and do not tell docker to automatically remove the container when we exit it:
$ docker run --name playground -i -t centos:7 bash [root@5960e4445743 /]# exit
Let’s see if the container is there:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Nope. So what’s different? Well, try again to start a container using that same name:
$ docker run --name playground -i -t centos:7 bash 2015/01/11 16:09:53 Error response from daemon: Conflict, The name playground is already assigned to 5960e4445743. You have to delete (or rename) that container to be able to assign playground to a container again.
Ooops. The container is actually still there:
$ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5960e4445743 centos:7 "bash" About a minute ago Exited (0) 57 seconds ago
There you go. By default docker ps doesn’t show you the containers in the exit status. You have to remove the container manually using its identifier:
$ docker rm 5960e4445743
I will not go further with using docker as it’s all you really need to start up with
A word about tags
Technically speaking, versions do not actually exist in docker images. They are in fact tags. A tag is a simple label for an image at a given point.
Images are identified with a hash value. As with IP addresses, you are not expected to recall the hash of the images you wish to use. Docker provides a mechanism to tag images much like you would use domain names instead of IP address.
For instance, 14.10 above is actually a tag, not a version. Obviously, since tags are meant to be meaningful to human beings, it’s quite sensible for Linux distributions to be tagged following the version of the distributions.
You can easily create tags for any images as we will see later on.
Let’s talk about registries
Docker images are hosted and served by a registry. Often as it’s the case in our previous example, the registry used is the public docker registry available at : https://registry.hub.docker.com/
Whenever you pull an image from a registry, by default docker pulls from that registry. However, you may query a different registry as follow:
$ docker pull hostname:port/path/to/image:tag
Basically, you provide the address of your registry and a path at which the image can be located. It has a similar form to an URI without the scheme.
Note that, as of docker 1.3.1, if the registry isn’t served over HTTPS, the docker client will refuse to download the image. If you need to pull anyway, you must add the following parameter to the docker daemon when it starts up.
--insecure-registry hostname:port
Please refer to the official documentation to learn more about this.
A base Linux Python-ready container
Traditionnaly deploying CherryPy application has been done using a simple approach:
- Package your application into an archive
- Copy that archive onto a server
- Configure a database server
- Configure a reverse proxy such as nginx
- Start the Python process(es) to server your CherryPy application
That last operation is usually done by directly calling nohup python mymodule.py &. Alternatively, CherryPy comes with a handy script to run your application in a slightly more convenient fashion:
$ cherryd -d -c path/to/application/conf/server.conf -P path/to/application -i mymodule
This runs the Python module mymodule as a daemon using the given configuration file. If the -P flag isn’t provided, the module must be found in PYTHONPATH.
The idea is to create an image that will serve your application using cherryd. Let’s see how to setup an Ubuntu image to run your application.
$ docker run --name playground -i -t ubuntu:14.10 bash root@d91ec7935e33:/#
First we create a user which will not have the root permissions. This is a good attitude to follow:
root@d91ec7935e33:/# useradd -m -d /home/web web root@d91ec7935e33:/# mkdir /home/web/.venv
Next, we install a bunch of libraries that are required to deploy some common Python dependencies:
root@d91ec7935e33:/# apt-get update root@d91ec7935e33:/# apt-get upgrade -y root@d91ec7935e33:/# apt-get install -y libc6 libc6-dev libpython2.7-dev libpq-dev libexpat1-dev libffi-dev libssl-dev python2.7-dev python-pip root@d91ec7935e33:/# apt-get autoclean -y root@d91ec7935e33:/# apt-get autoremove -y
Then we create a virtual environment and install Python packages into it:
root@d91ec7935e33:/# pip install virtualenv root@d91ec7935e33:/# virtualenv -p python2.7 /home/web/.venv/default root@d91ec7935e33:/# source /home/web/.venv/default/bin/activate root@d91ec7935e33:/# pip install cython root@d91ec7935e33:/# pip install cherrypy==3.6.0 pyopenssl mako psycopg2 python-memcached sqlalchemy
These are common packages I use. Install whichever you require obviously.
As indicated by Tony in the comments, it is probably overkill to create a virtual environment in a container since, the whole point of a container is to isolate your process and its dependencies already. I’m so used to using virtual env that I automatically created one. You may skip these steps.
Those operations were performed as the root user, let’s make the web user those packages owner.
root@d91ec7935e33:/# chown -R web.web /home/web/.venv
Good. Let’s switch to that user now:
root@d91ec7935e33:/# sudo su web web@d91ec7935e33:/# cd /home/web
At this stage, we have a base image ready to support a CherryPy application. It might be interesting to tag that paricular container as a new image so that we can use it various contexts.
web@d91ec7935e33:/# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
675ad8e8752d ubuntu:14.10 "bash" 7 minutes ago Up 7 minutes 0.0.0.0:9090->8080/tcp playground
web@d91ec7935e33:/# docker commit -m "Base Ubuntu with Python env" 675ad8e8752d
web@d91ec7935e33:/# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
78bc8c5c2e3f 6 seconds ago 506 MB
web@d91ec7935e33:/# docker tag 78bc8c5c2e3f lawouach/ubuntu:pythonbase
web@d91ec7935e33:/# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
lawouach/ubuntu pythonbase 78bc8c5c2e3f 32 seconds ago 506 MB
We take the docker container and we commit it as a new image. We then tag the new created image to make it easy to reuse it later on.
Let’s see if it worked. Exit the container and start a new container from the new image.
$ docker run --name playground -i -t lawouach/ubuntu:pythonbase bash root@66c6b2a5bb08:/# sudo su web web@66c6b2a5bb08:/$ cd /home/web/ web@66c6b2a5bb08:~$ source .venv/default/bin/activate (default)web@66c6b2a5bb08:~$
Well. We are ready to play now.
Run a CherryPy application in a docker container
For the purpose of this article, here is our simple application:
import cherrypy
class Root(object):
@cherrypy.expose
def index(self):
return "Hello world"
cherrypy.config.update({'server.socket_host': '0.0.0.0'})
cherrypy.tree.mount(Root())
Two important points:
- You must make sure CherryPy listens on the eth0 interface so just make it listen on all the container interfaces. Otherwise, the CherryPy will listen only on 127.0.0.1 which won’t be reachable from outside the container.
- Do not start the CherryPy engine yourself, this is done by the cherryd command. You must simply ensure the application is mounted so that CherryPy can serve it.
Save this piece of code into your container under the module name: server.py. This could be any name, really. The module will be located in /home/web.
You can manually test the module:
$ docker run --name playground -p 9090:8080 -i -t lawouach/ubuntu:pythonbase bash (default)web@66c6b2a5bb08:~$ cherryd -d -P /home/web -i server (default)web@66c6b2a5bb08:~$ ip addr list eth0 | grep "inet " inet 172.17.0.11/16 scope global eth0
The second line tells us the IPv4 address of this container. Next point your browser to the following URL: http://localhost:9090/
“What is this magic?” I hear you say!
If you look at the command we use to start the container, we provide this bit: -p 9090:8080. This tells docker to map port 9090 on the host to port 8080 on the container alllowing for your application to be reached from the outside.
And voilà!
Make the process a little more developer friendly
In the previous section, we saved the application’s code into the container itself. During development, this may not be practical. One approach is to use a volume to share a directory between your host (where you work) and the container.
$ docker run --name playground -p 9090:8080 -v `pwd`/webapp:/home/web/webapp -i -t lawouach/ubuntu:pythonbase bash
You can then work on your application and the container will see those changes immediatly.
Automate things a bit
The previous steps have shown in details how to setup an image to run a CherryPy application. Docker provides a simple interface to automate the whole process: Dockerfile.
A Dockerfile is a simple text file containing all the steps to create an image and more. Let’s see it first hand:
FROM ubuntu:14.10 RUN useradd -m -d /home/web web && mkdir /home/web/.venv &&\ apt-get update && sudo apt-get upgrade -y && \ apt-get install -y libc6 libc6-dev libpython2.7-dev libpq-dev libexpat1-dev libffi-dev libssl-dev python2.7-dev python-pip && \ pip install virtualenv && \ virtualenv -p python2.7 /home/web/.venv/default && \ /home/web/.venv/default/bin/pip install cython && \ /home/web/.venv/default/bin/pip install cherrypy==3.6.0 pyopenssl mako psycopg2 python-memcached sqlalchemy && \ apt-get autoclean -y && \ apt-get autoremove -y && \ chown -R web.web /home/web/.venv USER web WORKDIR /home/web ENV PYTHONPATH /home/web/webapp COPY webapp /home/web/webapp ENTRYPOINT ["/home/web/.venv/default/bin/cherryd", "-i", "server"]
Create a directory and save the content above into a file named Dockerfile. Create a subdirectory called webapp and store your server.py module into it.
Now, build the image as follow:
$ docker build -t lawouach/mywebapp:latest .
Use whatever tag suits you. Then, you can run a container like this:
$ docker run -p 9090:8080 -i -t lawouach/mywebapp:latest
That’s it! A docker container running your CherryPy application.
In the next articles, I will explore various options to use docker in a web application context. Follow ups will also include an introduction to weave and coreos to clusterize your CherryPy application.
In the meantime, do enjoy.