As I write this the first Docker convention is going on in San Francisco (sounds funny to write it was “docker convention” instead of “dockercon”…a convention sounds like a scene from Fear and Loathing in Las Vegas), and Docker has hit 1.0 and been declared production worthy.
Docker has had, for a few versions, the ability to use btrfs as it’s backing driver, instead of aufs.
Before installing docker I’m going to setup btfrs.
In this example I have a disk device called /dev/vdb to use with btrfs. I’ve removed some of the output from the commands.
$ sudo su
$ apt-get install btrfs-tools
$ mkfs.btrfs /dev/sdb
$ mkdir /var/lib/docker
$ mount /dev/sdb /var/lib/docker
$ mount | grep btrfs
/dev/sdb on /var/lib/docker type btrfs (rw)
Now that btrfs is installed and a device is formatted and mounted under /var/lib/docker we can install docker.
Next we need to install docker. It’s important that we also configure docker to startup with the “-s btrfs” option as well.
$ apt-get install docker.io
$ vi /etc/default/docker.io
We need to change the DOCKER_OPTS entry to look like this:
$ grep DOCKER_OPTS docker.io
# Use DOCKER_OPTS to modify the daemon startup options.
DOCKER_OPTS="-s btrfs"
Now restart docker.
$ service docker.io restart
We should see that docker is running with “-s btrfs”:
$ ps ax |grep [d]ocker
10088 ? Sl 0:02 /usr/bin/docker.io -d -s btrfs
If it’s not running with “-s btrfs” then ensure it’s set to do so in /etc/default/docker.io and has been restarted.
I’ve already created several containers and pulled images, actually just the base busybox image which is quite small, so if I run “btrfs subvolume list /var/lib/docker” I should see some subvolumes have been created.
vagrant@host1:~$ sudo btrfs subvolume list /var/lib/docker | head -5
ID 258 gen 11 top level 5 path btrfs/subvolumes/511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
ID 259 gen 16 top level 5 path btrfs/subvolumes/42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229
ID 260 gen 13 top level 5 path btrfs/subvolumes/c120b7cab0b0509fd4de20a57d0f5c17106f3451200dfbfd8c6ab1ccb9391938
ID 261 gen 13 top level 5 path btrfs/subvolumes/d200959a3e91d88e6da9a0ce458e3cdefd3a8a19f8f5e6a1e7f10f268aea5594
ID 262 gen 15 top level 5 path btrfs/subvolumes/120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16
And we can see the size of the btrfs file system.
$ btrfs filesystem df /var/lib/docker
Data, single: total=1.01GiB, used=337.56MiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=1.00GiB, used=2.72MiB
Metadata, single: total=8.00MiB, used=0.00
$
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 40G 1.2G 37G 4% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 745M 12K 745M 1% /dev
tmpfs 150M 368K 150M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 750M 0 750M 0% /run/shm
none 100M 0 100M 0% /run/user
vagrant 233G 185G 49G 80% /vagrant
/dev/sdb 20G 344M 18G 2% /var/lib/docker
$
$ sudo btrfs filesystem show /var/lib/docker
Label: none uuid: c8f11393-9268-475a-82de-cbd697ab3847
Total devices 1 FS bytes used 340.30MiB
devid 1 size 20.00GiB used 3.04GiB path /dev/sdb
Btrfs v3.12
Each image and container has a subvolume.
And, as far as I know at this point, that’s it–we’re now running docker with btrfs.
I’ve setup an Ansible playbook with a Vagrant file that will setup a virtual machine with docker and btrfs configured in the same way that I describe in this blog post. Vagrant will automatically provision the virtual machine using Ansible.
$ git clone https://github.com/ccollicutt/vagrant-docker-btrfs
$ cd vagrant-docker-btrfs
$ vagrant up --provider virtualbox
Bringing machine 'host1' up with 'virtualbox' provider...
==> host1: Importing base box 'trusty64'...
SNIP!
TASK: [add alias for vagrant user docker = docker.io] *************************
changed: [host1]
NOTIFIED: [restart docker] ****************************************************
changed: [host1]
PLAY RECAP ********************************************************************
host1 : ok=17 changed=16 unreachable=0 failed=0
$
Now that that is booted up and configured, we can login and check a couple things.
$ vagrant ssh
SNIP!
vagrant@host1:~$ sudo btrfs subvolume list /var/lib/docker
vagrant@host1:~$# none because we have no images or containers
vagrant@host1:~$ mount |grep btrfs
/dev/sdb on /var/lib/docker type btrfs (rw)
vagrant@host1:~$ ps ax |grep [d]ocker
10041 ? Sl 0:00 /usr/bin/docker.io -d -s btrfs
vagrant@host1:~$ docker pull busybox
Pulling repository busybox
a9eb17255234: Download complete
d200959a3e91: Download complete
fd5373b3d938: Download complete
37fca75d01ff: Download complete
511136ea3c5a: Download complete
42eed7f1bf2a: Download complete
f06b02872d52: Download complete
120e218dd395: Download complete
c120b7cab0b0: Download complete
1f5049b3536e: Download complete
vagrant@host1:~$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
busybox buildroot-2013.08.1 d200959a3e91 4 days ago 2.489 MB
busybox ubuntu-14.04 37fca75d01ff 4 days ago 5.609 MB
busybox ubuntu-12.04 fd5373b3d938 4 days ago 5.455 MB
busybox buildroot-2014.02 a9eb17255234 4 days ago 2.433 MB
busybox latest a9eb17255234 4 days ago 2.433 MB
vagrant@host1:~$# now we should have some subvolumes
vagrant@host1:~$ sudo btrfs subvolume list /var/lib/docker
ID 258 gen 12 top level 5 path btrfs/subvolumes/511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
ID 259 gen 17 top level 5 path btrfs/subvolumes/42eed7f1bf2ac3f1610c5e616d2ab1ee9c7290234240388d6297bc0f32c34229
ID 260 gen 14 top level 5 path btrfs/subvolumes/f06b02872d5253f5123284edcf49749b352400a1c5880b5ebf2864f5afddeb22
ID 261 gen 14 top level 5 path btrfs/subvolumes/37fca75d01ffc49df7b99aacdbcd4a0ebae39de299787b8f77bb5b6698414308
ID 262 gen 16 top level 5 path btrfs/subvolumes/120e218dd395ec314e7b6249f39d2853911b3d6def6ea164ae05722649f34b16
ID 263 gen 16 top level 5 path btrfs/subvolumes/a9eb172552348a9a49180694790b33a1097f546456d041b6e82e4d7716ddb721
ID 264 gen 19 top level 5 path btrfs/subvolumes/1f5049b3536eb73e7a660a672976ae9c19e8460bf57c2528f9c1e4b2c4bf309f
ID 265 gen 19 top level 5 path btrfs/subvolumes/c120b7cab0b0509fd4de20a57d0f5c17106f3451200dfbfd8c6ab1ccb9391938
ID 266 gen 19 top level 5 path btrfs/subvolumes/d200959a3e91d88e6da9a0ce458e3cdefd3a8a19f8f5e6a1e7f10f268aea5594
ID 267 gen 19 top level 5 path btrfs/subvolumes/fd5373b3d93820744a327e609ee86166e5984d7377987f0fde78daeaa345705d
I haven’t explored much more in terms of using btrfs with docker yet, but this is a good start.