| ZooKeeper Series | Docker Swarm Series |
Docker Swarm is native clustering for Docker. Before Docker Engine 1.12 (June 2016); it was a separate thing that turned the pool of docker hosts into a single virtual Docker host; and now since 1.12 its included and is part of Docker Engine and now called “Swarm Mode”. We can use Docker CLI to create a swarm, deploy application services to a swarm and manage its behavior.
For Swarm; we need to have multiple Docker Engines running on nodes; one or more node acts as Manager and then we add Workers into the Swarm. The quickest way to try it is to use docker-machine and setup multiple Docker Engines across different Hosts or Virtual Machines. I have three VMs running Docker Engine v1.13. For these VMs; I used RancherOS; the tiny Linux distro ideal to run Docker Engine. I added them into my environment using Docker-Machine. Please note; RancherOS and Racher are seperate products; Rancher OS is the Linux distro and Rancher is Swarm like container management product. Rancher also supports using Swarm as its underlying clustering engine along with Cattle; its own; and Kubernetes and Mesos. But for this post we will remain committed to using Swarm using the Docker CLI and tools!
- I used the docker-machine’ generic driver to add the remote Docker Engines
- RancherOS doesnt use Docker Engine 1.13 by default; but you can change the Docker Version using ros engine command; https://docs.rancher.com/os/configuration/switching-docker-versions has the details
- You can use docker-machine to create required VMs for Docker Engine; https://docs.docker.com/machine/get-started has the details
- docker-machine has notion of drivers; it uses Virtual Box by default; you can use HyperV driver to create VMs; there are few gotchas and things are not as seamless as with Virtual Box; http://weblogs.com.pk/khurram/archive/2016/06/04/docker-on-windows.aspx has the details
- You can use RacnherOS to create Docker Engine VM; http://docs.rancher.com/os/running-rancheros/workstation/docker-machine has the details
To create a Swarm; we choose one machine as the Manager, set the Docker environment for that machine; and run docker swarm init; it will initialize the Swarm environment on that machine, make it a manager and outputs the docker CLI command that we can run on the other machines with which we can add them as as workers
- If the machine has more than one IP address; use docker swarm init –advertise-addr <ManagerIP>; https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ for more information
- https://docs.docker.com/engine/swarm/manage-nodes to learn more about how to manage the nodes in a swarm
- https://docs.docker.com/engine/swarm/swarm-tutorial for step by step tutorial
Unlike Rancher; there is no GUI or web based interface to manage the Docker Swarm, but there are third party tools available; and mostly as containers that we can run on the underlying Docker Engines. Docker Swarm Visualizer is the popular one; Portainer is another!
In Docker 1.13 (January 2017); they have added a docker-compose support to the docker stack deploy command so that services can be deployed using a docker-compose.yml file directly. They have also introduced compose file v3 format that has new options like deploy related to deployment and running of the services in a Swarm, labels to specify the labels to the services
Lets make a v3 compose file for our ZooKeeper; sadly for such application; where one node needs to know about the other; and every node need its own configuration; we have to define service for each node. Once we have the compose file; we deploy “the stack” using docker stack deploy –compose-file yml-file NameOfStack; we defined the deployment constraints; and the manager will deploy zoo1 service (single node) on swarm1, zoo2 on swarm2 and zoo3 on swarm3 node automatically
- We can use, node.id, node.role (manager or worker) and node labels (node.labels.some-label) or engine labels (engine.labels.some-label) to define deployment constraints; https://github.com/docker/docker/pull/24397 has more information
- http://stackoverflow.com/questions/42062598/how-to-setup-zookeeper-cluster-on-docker-swarm; a related question on Stack Overflow!
We can list the services using docker service ls
Hopefully such workarounds will not be required once Swarm and Compose get more matured!