private registry image as docker swarm service

Question

First of all, to mention I checked multiple questions in Stackoverflow but the requirement was somewhat different or the solution offered was not working in my case, so thought of creating a new question.

1. I am using Local Registry and able to push/pull image from all worker nodes as well
2. Registry service is up and running in all nodes
3. Issue while creating service of local image that I already pushed to the docker. Issue: overall progress: 0 out of 1 tasks 1/1: ready [======================================> ] verify: Detected task failure

Steps I have done:

1. docker service create --name registry --publish 5000:5000 armbuild/registry (mine is raspi so used armbuild)

2. docker tag XYZImage localhost:5000/XYZImage -> Working Fine

3. docker push localhost:5000/XYZImage -> Working Fine

4. docker service create --name XYZService --replicas 2 localhost:5000/XYZImage --> Issue Note: Even I tried using IP address and adding that address to insecure registries in daemon.json file.

Any leads? or if I am missing something?

Answer 1

Each container or node is writing to its own volume (/var/lib/registry). You must use a distributed storage driver if you want to use replicas.

From the documentation :

The storage back-end you use determines whether you use a fully scaled service or a service with either only a single node or a node constraint.

If you use a distributed storage driver, such as Amazon S3, you can use a fully replicated service. Each worker can write to the storage back-end without causing write conflicts.

If you use a local bind mount or volume, each worker node writes to its own storage location, which means that each registry contains a different data set. You can solve this problem by using a single-replica service and a node constraint to ensure that only a single worker is writing to the bind mount.