Using macOS with Docker can be “interesting”. When I got started, I followed the useful advice at https://pilsniak.com/how-to-install-docker-on-mac-os-using-brew/. This approach appealed to me, especially the use of xhyve. Because sometimes I just make life difficult for myself.
Thus my initial setup was:
brew install docker docker-machine xhyve docker-machine-driver-xhyve f=/usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve sudo chown root:wheel $f; sudo chmod u+s $f # because yay, more setuid root binaries; it's written in Go, which is # something at least. docker-machine create default --driver xhyve --xhyve-experimental-nfs-share eval "$(docker-machine env default)" tmutil addexclusion $HOME/.docker ; sudo tmutil addexclusion -p $HOME/.docker
docker-machine create line will fail for releases of the
xhyve driver after
v0.3.3; you’ll need to add an area to be shared to the
end of the command-line; use
/Users to match previous behavior.)
Excluding the Docker VM area from regular backups is a useful addition. Note
that the usual docker-machine eval guidance from the maintainers is wrong:
it will result in exporting an empty environment variable named
Wrap the command-substitution in double-quotes to preserve newlines.
This is why decent tools emitting export commands for eval will insert
semi-colons at the end of each line, and avoid comments: to make life easier
for sloppy command-line usage and missing quotes.
This gives us a VM running the fairly minimal OS named
runs Docker and exposes that to Docker on the host macOS via some environment
variables pointing the client tool at the VM instead of a local socket.
The biggest problem I had was that my DNS server as seen by the VM kept getting reset to be the IP of my laptop on that virtual network, and my laptop’s DNS resolver was not set to be exposed over the network. I’m reluctant to do so, because I do attend conferences and use untrusted WiFi, but exposing the DNS server to only also listen to an arbitrary list of dynamically created virtual networks, but not to the WiFi, is a fragile pain I don’t want. So the laptop is not itself a suitable DNS server for the VM running boot2docker.
docker-machine-driver-xhyve driver’s DHCP server simply doesn’t
set a DNS field and ignores whatever’s in the config elsewhere, so the VMs end
up with the IP address of the server being used as the DNS server IP too.
In addition, the latest tagged version of that driver,
0.3.3, predates a fix
from August wherein that driver was also stomping over
bootlocal.sh on each
boot, removing the ability of users to work around the first issue. My
grateful thanks to Hugues Alary for committing the fix, which I found in git
when I went looking to figure out why
boot2local.sh changes weren’t
Really, the xhyve driver is neat, but not yet ready for prime time. In
fairness, it does call itself version
0.3.3. That’s pretty clear. So let’s
show how to stick with it.
docker-machine stop brew unlink docker-machine-driver-xhyve brew install --HEAD docker-machine-driver-xhyve f=/usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve sudo chown root:wheel $f; sudo chmod u+s $f sudo true # to get the password in the cache, because I/O is messed up through docker-machine docker-machine start # json: cannot unmarshal bool into Go struct field Driver.Virtio9p of type string
Ah, JSON config changes. I have Docker images I really don’t want to have to
reload. So instead of nuking the machine and recreating, we study commit
dff9c59d of the driver, bring up a temporary second machine to have a
template to compare against, and then:
C="$HOME/.docker/machine/machines/default/config.json" jq < "$C" > "$C.new" ' .Driver.Virtio9p = null | .Driver.NFSShares = [.Driver.Virtio9pFolder] | .Driver.NFSShare = false | .Driver.NFSSharesRoot = "/xhyve-nfsshares" | .Driver.Virtio9pRoot = "/xhyve-virtio9p" | del(.Driver.Virtio9pFolder) ' ls -ld "$C" "$C.new" chmod 0600 "$C.new" mv "$C.new" "$C"
That should get us a bootable VM.
At this point, we can once more boot boot2docker, and
bootlocal.sh won’t be
stomped over on each boot (but has cruft left in it).
sudo true docker-machine start docker-machine ssh sudo -Hi cd /var/lib/boot2docker cp /usr/share/udhcpc/default.script ./dhcp.script vi dhcp.script # near top, after other checks but before the 'case': [ -f /var/lib/boot2docker/dns-servers ] && dns="$(cat /var/lib/boot2docker/dns-servers)" vi dns-servers # whatever IPs you want, separated by whitespace; no comments allowed # for Google's Public DNS servers: 18.104.22.168 22.214.171.124 vi bootlocal.sh # everything in this script is now junk, left behind by old versions of # the xhyve driver, so bring it down to: #!/bin/sh sed -i -e 's,/sbin/udhcpc,& -s /var/lib/boot2docker/dhcp.script,' /etc/rc.d/dhcp.sh # and to trigger one run now, because we're too late ( DEVICE=eth0 /sbin/udhcpc -s /var/lib/boot2docker/dhcp.script -b -i $DEVICE -x hostname:$(/bin/hostname) -p /var/run/udhcpc.$DEVICE.pid ) exit; exit docker-machine stop sudo true docker-machine start
And lo, working DNS.
-The Grumpy Troll