What’s up Doc(ker)?

A recent post from Doug Davis “Docker: Managing the excitement” has some nice insights on Docker as a “relatively new” technology. Among the other Docker niceties that Doug points out, one that resonated with me is, and I quote “it takes seconds (if not less than a second, after the first time since things are cached) for it to complete”. Reading between the lines in some of my other posts, I’ve often battled the dark forces of virtual machine world, usually trying to squeeze every bit of extra juice out of my poor laptop. Then I tried Docker and well, suffice to say I need fewer coffee breaks.

My long-time colleague and friend Takehiko Amano has a post on “Immutable Infrastructure” where he talks about Docker in the context of OpenStack and with a bunch of my earlier posts focused on OpenStack, I decided to figure out how the three – OpenStack, Docker, and UCD+P might work together.

The first thing to note is that there are two ways to go about getting Docker to play with OpenStack: via Heat or via Nova and here are my notes on trying both out. Given that the initial OpenStack post on this topic  recommends that the Heat way is preferred I first set out to get my existing Heat engine on Icehouse working with Docker containers. Google produces any number of results on how to get the two to play nice but I spent many futile hours until I stumbled upon Docker plug-in for OpenStack Heat that I found the crucial piece to the puzzle. So many thanks to Lars.

1. HEAT + Docker


Looking at the example YAML in both Lars’ post and the Docker OpenStack post, the Docker host is a newly provisioned Nova Server into which Docker is installed on boot. But remember that I want things to happen in seconds or less, so even the minute or so of added time to get an additional VM up and Docker installed is too much:-) Ergo the first thing I did was install Docker directly into  the same VM that has the Heat engine installed by UCD+P so the containers get directly instantiated on the OpenStack host. Not a good thing but the need for speed…

One simple command got Docker installed and running.

$ sudo yum install docker-io

I also did

$ sudo usermod -a -G docker heat

so the user that runs the  Heat services can talk to Docker.

Next I cloned the OpenStack Heat GitHub repo and following Lars’ instructions edited “heat/contrib/heat_docker/heat_docker/resources/docker_container.py” to patch the Docker plugin.

Then I copied the plugin into /usr/lib/heat and  restarted the heat-engine service.

Running “heat resource-type-list” showed that the Docker resource type was successfully loaded.

$ heat resource-type-list | grep Docker
| DockerInc::Docker::Container       |

Time to try provisioning a simple sample HEAT template with a Docker container in it.

heat_template_version: 2013-05-23
description: Simple Hello World web server in Docker
    type: number
    description: host port to bind container port 80 to
    default: 20000

    type: DockerInc::Docker::Container
      hostname: helloworld
      image: "tutum/hello-world"
        - 80
        80: { get_param: bind_port }
    description: URL for Web Site
          "%port%": { get_param: bind_port }

When provisioned with “heat stack-create“, this pulls the “tutum/hello-world” image from the Docker registry  and runs it, mapping the containers port 80 to a user specified port on the host. Since I didn’t specify the docker_endpoint parameter for the container, it will by default use Docker on the host.

Though the first time it takes about a minute to pull the “tutum/hello-world” image, it takes about a second to get it up and running and serving up a static web page. I can see the image/container on the host via docker commands and heat shows the stack’s status, properties and outputs as specified in the template.

$ heat stack-create -f tutum_hello-world.yaml helloworld
| stack_name | stack_status       | creation_time        |
| helloworld | CREATE_IN_PROGRESS | 2014-09-11T00:06:07Z |
$ heat stack-list
| stack_name | stack_status    | creation_time        |
| helloworld | CREATE_COMPLETE | 2014-09-11T00:06:07Z |
$ heat stack-show helloworld
| Property             | Value                                                    
| capabilities         | []                                                       
| creation_time        | 2014-09-11T00:06:07Z                                     
| description          | Simple Hello World web server in Docker                  
| disable_rollback     | True                                                     
| id                   | e9176d6a-e43e-49fe-a58c-03cdb4c05130                     
| links                |
| notification_topics  | []                                                       
| outputs              | [                                                        
|                      |   {                                                      
|                      |     "output_value": "",       
|                      |     "description": "URL for Web Site",                   
|                      |     "output_key": "webURL"                               
|                      |   }                                                      
|                      | ]                                                        
| parameters           | {                                                        
|                      |   "bind_port": "20000",                                  
|                      |   "OS::stack_name": "helloworld",                        
|                      |   "OS::stack_id": "e9176d6a-e43e-49fe-a58c-03cdb4c05130" 
|                      | }                                                        
| stack_name           | helloworld                                               
| stack_status         | CREATE_COMPLETE                                          
| stack_status_reason  | Stack CREATE completed successfully                      
| template_description | Simple Hello World web server in Docker                  
| timeout_mins         | 60                                                       
| updated_time         | None                                                     
$ docker ps
CONTAINER ID        IMAGE                      COMMAND             CREATED             STATUS              PORTS                
a2bd223e9140        tutum/hello-world:latest   /run.sh             11 minutes ago      Up 11 minutes>80/tcp

And accessing the web page via :


Deleting the stack (heat stack-delete), also destroys the Docker container.

2. Nova + Docker

500px-Docker-under-the-hoodThe steps to set this up are in the OpenStack Docker Wiki and I followed them to install the plug-in into an all-in-one Devstack’ed Icehouse Ubuntu VM.

First the one step Docker install

$ curl -sSL https://get.docker.io/ubuntu/ | sudo sh

followed by

$ sudo usermod -a -G docker stack

and then install the driver

$ sudo pip install -e git+https://github.com/stackforge/nova-docker#egg=novadocker

Then I configure Nova:

$ cat << EOF >  /etc/nova/rootwrap.d/docker.filters
# nova-rootwrap command filters for setting up network in the docker driver
# This file should be owned by (and only-writeable by) the root user

# nova/virt/docker/driver.py: 'ln', '-sf', '/var/run/netns/.*'
ln: CommandFilter, /bin/ln, root

$ sed -i.orig '/compute_driver/c\compute_driver = novadocker.virt.docker.DockerDriver' /etc/nova/nova.conf

Next I configure Glance:

$ sed -i.orig '/container_formats/c\container_formats=ami,ari,aki,bare,ovf,ova,docker' /etc/glance/glance-api.conf

Finally I restart the Nova Compute and Glance API services and everything should be good to go.

With this setup the hypervisor being used is Docker so before being able to launch any instances I need to get the required Docker images into Glance. Rather than set up a local Docker registry that could store images in Glance, for this example I simply use “docker save” to get the image into Glance. So to get the “tutum/helloworld” image from the HEAT + Docker example above I need to:

$ docker pull tutum/hello-world


$ docker save "tutum/hello-world" | glance image-create --is-public=True --container-format=docker --disk-format=raw --name "tutum/hello-world"

Glance now has the Docker image that I can boot with Nova.

$ glance image-list
| Name                | Disk Format | Container Format | Size      | Status |
| tutum/hello-world   | raw         | docker           | 293746688 | active |
| Ubuntu-14.04.x86_64 | qcow2       | bare             | 255787520 | active |
 $ nova boot --image tutum/hello-world --flavor m1.tiny helloworld
| Property                             | Value                                                    |
| created                              | 2014-09-12T00:40:12Z                                     |
| flavor                               | m1.tiny (1)                                              |
| hostId                               |                                                          |
| id                                   | e1de8ccd-8be8-4311-81c5-09041d00b274                     |
| image                                | tutum/hello-world (28d1fc94-7cea-4ecb-8d30-1f31614bfa46) |
| name                                 | helloworld                                               |

It only takes a couple of seconds and the new “server” is up and running.

$ nova list
| ID                                   | Name       | Status | Task State | Power State | Networks                     |
| e1de8ccd-8be8-4311-81c5-09041d00b274 | helloworld | ACTIVE | -          | Running     | private=, |

And I can get to the web page using the floating ip


So there we have it: two different ways of using Docker with Openstack. I’m not sure which one is “better” but I prefer using the first (Heat + Docker) approach as it just seems more “natural”. Since UCD+P uses Heat templates, that leads to the question of how UCD+P works with each of the above approaches. The answer to that will come in a future post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s