A recent post from Doug Davis “Docker: Managing the excitement” has some nice insights on Docker as a “relatively new” technology. Among the other Docker niceties that Doug points out, one that resonated with me is, and I quote “it takes seconds (if not less than a second, after the first time since things are cached) for it to complete”. Reading between the lines in some of my other posts, I’ve often battled the dark forces of virtual machine world, usually trying to squeeze every bit of extra juice out of my poor laptop. Then I tried Docker and well, suffice to say I need fewer coffee breaks.
My long-time colleague and friend Takehiko Amano has a post on “Immutable Infrastructure” where he talks about Docker in the context of OpenStack and with a bunch of my earlier posts focused on OpenStack, I decided to figure out how the three – OpenStack, Docker, and UCD+P might work together.
The first thing to note is that there are two ways to go about getting Docker to play with OpenStack: via Heat or via Nova and here are my notes on trying both out. Given that the initial OpenStack post on this topic recommends that the Heat way is preferred I first set out to get my existing Heat engine on Icehouse working with Docker containers. Google produces any number of results on how to get the two to play nice but I spent many futile hours until I stumbled upon Docker plug-in for OpenStack Heat that I found the crucial piece to the puzzle. So many thanks to Lars.
1. HEAT + Docker
Looking at the example YAML in both Lars’ post and the Docker OpenStack post, the Docker host is a newly provisioned Nova Server into which Docker is installed on boot. But remember that I want things to happen in seconds or less, so even the minute or so of added time to get an additional VM up and Docker installed is too much:-) Ergo the first thing I did was install Docker directly into the same VM that has the Heat engine installed by UCD+P so the containers get directly instantiated on the OpenStack host. Not a good thing but the need for speed…
One simple command got Docker installed and running.
$ sudo yum install docker-io
I also did
$ sudo usermod -a -G docker heat
so the user that runs the Heat services can talk to Docker.
Next I cloned the OpenStack Heat GitHub repo and following Lars’ instructions edited “heat/contrib/heat_docker/heat_docker/resources/docker_container.py” to patch the Docker plugin.
Then I copied the plugin into /usr/lib/heat and restarted the heat-engine service.
Running “heat resource-type-list” showed that the Docker resource type was successfully loaded.
$ heat resource-type-list | grep Docker | DockerInc::Docker::Container |
Time to try provisioning a simple sample HEAT template with a Docker container in it.
heat_template_version: 2013-05-23
description: Simple Hello World web server in Docker
parameters:
bind_port:
type: number
description: host port to bind container port 80 to
default: 20000
resources:
hello_world_web:
type: DockerInc::Docker::Container
properties:
hostname: helloworld
image: "tutum/hello-world"
port_specs:
- 80
port_bindings:
80: { get_param: bind_port }
outputs:
webURL:
description: URL for Web Site
value:
str_replace:
template: http://192.168.27.101:%port%
params:
"%port%": { get_param: bind_port }
When provisioned with “heat stack-create“, this pulls the “tutum/hello-world” image from the Docker registry and runs it, mapping the containers port 80 to a user specified port on the host. Since I didn’t specify the docker_endpoint parameter for the container, it will by default use Docker on the host.
Though the first time it takes about a minute to pull the “tutum/hello-world” image, it takes about a second to get it up and running and serving up a static web page. I can see the image/container on the host via docker commands and heat shows the stack’s status, properties and outputs as specified in the template.
$ heat stack-create -f tutum_hello-world.yaml helloworld +------------+--------------------+----------------------+ | stack_name | stack_status | creation_time | +------------+--------------------+----------------------+ | helloworld | CREATE_IN_PROGRESS | 2014-09-11T00:06:07Z | +------------+--------------------+----------------------+ $ heat stack-list +------------+-----------------+----------------------+ | stack_name | stack_status | creation_time | +------------+-----------------+----------------------+ | helloworld | CREATE_COMPLETE | 2014-09-11T00:06:07Z | +------------+-----------------+----------------------+
$ heat stack-show helloworld
+----------------------+----------------------------------------------------------
| Property | Value
+----------------------+----------------------------------------------------------
| capabilities | []
| creation_time | 2014-09-11T00:06:07Z
| description | Simple Hello World web server in Docker
| disable_rollback | True
| id | e9176d6a-e43e-49fe-a58c-03cdb4c05130
| links | http://192.168.27.101:8004/v1/f87ddd64778e45eb8279cc8827f
| notification_topics | []
| outputs | [
| | {
| | "output_value": "http://192.168.27.101:20000",
| | "description": "URL for Web Site",
| | "output_key": "webURL"
| | }
| | ]
| parameters | {
| | "bind_port": "20000",
| | "OS::stack_name": "helloworld",
| | "OS::stack_id": "e9176d6a-e43e-49fe-a58c-03cdb4c05130"
| | }
| stack_name | helloworld
| stack_status | CREATE_COMPLETE
| stack_status_reason | Stack CREATE completed successfully
| template_description | Simple Hello World web server in Docker
| timeout_mins | 60
| updated_time | None
+----------------------+----------------------------------------------------------
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS a2bd223e9140 tutum/hello-world:latest /run.sh 11 minutes ago Up 11 minutes 0.0.0.0:20000->80/tcp
And accessing the web page via http://192.168.27.101:20000 :
Deleting the stack (heat stack-delete), also destroys the Docker container.
2. Nova + Docker
The steps to set this up are in the OpenStack Docker Wiki and I followed them to install the plug-in into an all-in-one Devstack’ed Icehouse Ubuntu VM.
First the one step Docker install
$ curl -sSL https://get.docker.io/ubuntu/ | sudo shfollowed by
$ sudo usermod -a -G docker stack
and then install the driver
$ sudo pip install -e git+https://github.com/stackforge/nova-docker#egg=novadocker
Then I configure Nova:
$ cat << EOF > /etc/nova/rootwrap.d/docker.filters # nova-rootwrap command filters for setting up network in the docker driver # This file should be owned by (and only-writeable by) the root user [Filters] # nova/virt/docker/driver.py: 'ln', '-sf', '/var/run/netns/.*' ln: CommandFilter, /bin/ln, root EOF $ sed -i.orig '/compute_driver/c\compute_driver = novadocker.virt.docker.DockerDriver' /etc/nova/nova.conf
Next I configure Glance:
$ sed -i.orig '/container_formats/c\container_formats=ami,ari,aki,bare,ovf,ova,docker' /etc/glance/glance-api.conf
Finally I restart the Nova Compute and Glance API services and everything should be good to go.
With this setup the hypervisor being used is Docker so before being able to launch any instances I need to get the required Docker images into Glance. Rather than set up a local Docker registry that could store images in Glance, for this example I simply use “docker save” to get the image into Glance. So to get the “tutum/helloworld” image from the HEAT + Docker example above I need to:
$ docker pull tutum/hello-world
then
$ docker save "tutum/hello-world" | glance image-create --is-public=True --container-format=docker --disk-format=raw --name "tutum/hello-world"
Glance now has the Docker image that I can boot with Nova.
$ glance image-list +---------------------+-------------+------------------+-----------+--------+ | Name | Disk Format | Container Format | Size | Status | +---------------------+-------------+------------------+-----------+--------+ | tutum/hello-world | raw | docker | 293746688 | active | | Ubuntu-14.04.x86_64 | qcow2 | bare | 255787520 | active | +---------------------+-------------+------------------+-----------+--------+
$ nova boot --image tutum/hello-world --flavor m1.tiny helloworld +--------------------------------------+----------------------------------------------------------+ | Property | Value | +--------------------------------------+----------------------------------------------------------+ ... | created | 2014-09-12T00:40:12Z | | flavor | m1.tiny (1) | | hostId | | | id | e1de8ccd-8be8-4311-81c5-09041d00b274 | | image | tutum/hello-world (28d1fc94-7cea-4ecb-8d30-1f31614bfa46) | ... | name | helloworld | ... +--------------------------------------+----------------------------------------------------------+
It only takes a couple of seconds and the new “server” is up and running.
$ nova list +--------------------------------------+------------+--------+------------+-------------+------------------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+------------+--------+------------+-------------+------------------------------+ | e1de8ccd-8be8-4311-81c5-09041d00b274 | helloworld | ACTIVE | - | Running | private=10.0.0.9, 172.24.4.6 | +--------------------------------------+------------+--------+------------+-------------+------------------------------+
And I can get to the web page using the floating ip http://172.24.4.6
So there we have it: two different ways of using Docker with Openstack. I’m not sure which one is “better” but I prefer using the first (Heat + Docker) approach as it just seems more “natural”. Since UCD+P uses Heat templates, that leads to the question of how UCD+P works with each of the above approaches. The answer to that will come in a future post.
