Unable to access the services in Docker swarm

Question

I am quite new to docker and I have been following the Nigel Poulton course lately from PluralSight. Now, I started following the video tutorial on swarm mode and did exactly told in the video.

I have 6 EC2 instances running. I logged into each of them and installed a docker. The engine version is 18.09.9 on every machine. Now, I initialized the swarm mode from machine 1 using this command:

sudo docker swarm init --advertise-addr 172.31.41.99:2377 --listen-addr 172.31.41.99:2377

Then I got the token for the workers and managers and logged into other machines and at last created 3 managers and 3 workers.

This is what my node looks like in the end:

ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
sbh3v850ko7h8f279366fxlt9 *   ip-172-31-41-99     Ready               Active              Leader              18.09.9
f7kqsy3yano3vxo8451mi9z7o     ip-172-31-83-172    Ready               Active              Reachable           18.09.9
jb0innx7l4q56kq4dk9phginw     ip-172-31-90-8      Ready               Active                                  18.09.9
xr3oz2x1wyl3a0bd83i3atjk4     ip-172-31-91-47     Ready               Active                                  18.09.9
ttug67rbhfai88wq2sbulo8dm     ip-172-31-94-197    Ready               Active              Reachable           18.09.9
kkun5zo93zxg4jksryvbeh3mt     ip-172-31-94-220    Ready               Active                                  18.09.9

Then I created service using the following command:

sudo docker service create --name psight1 -p 8080:8080 --replicas 5 nigelpoulton/pluralsight-docker-ci

now when I pick up the public DNS of my machine 1 and enter http://ec2-3-89-157-63.compute-1.amazonaws.com:8080 , I get an error saying that the site cannot be reached.

However, instead of using docker service, if I were to use the command:

docker run -d --name web -p 8080:8080 nigelpoulton/pluralsight-docker-ci

Then when I go to http://ec2-3-89-157-63.compute-1.amazonaws.com:8080 it works perfectly.

I see that there is an open issue ongoing in github which is something similar to this. Is it because of the same issue in docker 18.09.9 or am I missing something?

Answer 1

Have you created an overlay network and deployed the service to that network.

Something like docker network create -d my-net to create the overlay.

Then docker service create --network my-net... to deploy the service to the overlay.

I'm scratching my head as it's been a while, but an overlay network might be vital to the routing mesh that makes the service available via any node.

What I'm thinking is that you might be hitting a node in the cluster that doesn't have a service replica.

Another way to test would be to deploy with more replicas than nodes. This should ensure at least one on each node.

Alternatively, deploy a single replica, find out which node it's running on, connect to that node (browser or curl) and then try again hitting a node that isn't running the replica.

HTH. Also pinged you on Twitter.

@nigelpoulton