docker push to nexus registry (behind proxy) ends with EOF

Question

I have tried a lot, but I can't find a solution to this problem.

I am running a nexus sonatype (3.21.1-01) docker image on a centos7 server behind a vthunder a10 proxy.
The docker login and pull works great but docker push fail with EOF after some retrying.

Here the interested routes:

• docker image port 8081 > my.server:8081
• docker image port 8443 > my.server:8443
• proxy.domain.local:443 > my.server:8081
• proxy.domain.local:8443 > my.server:8443
• I have created a docker repository in nexus which have the http connector exposed on 8443
• The proxy is exposed under ssl with self signed certificate
• The client's /etc/docker/daemon.json file contains the insecure registry options:
  "insecure-registries": ["proxy.domain.local:8443","proxy.domain.local"]

Here the situation:

1. If I try to push from the client an image of which all layers already exist on the remote server (but missing on nexus repository), it works.

2. If I try the same but adding some difference to the same image (such as a new LABEL), it fail in this way:
   

(9c27e219663c: Layer already exists 
Patch https://proxy.domain.local:8443/v2/test4/blobs/uploads/6862fe60-d63b-4942-bbb6-f403307e677a: EOF)

3. If I push directly from my.server machine, pointing to localhost:8443 it works.

4. If i push from the client machine an image with new layers it fail in this way after some retrying (the same behavior with smaller images):
   

docker push proxy.domain.local:8443/ara
The push refers to repository [proxy.domain.local:8443/ara]
edb7a4f74e22: Retrying in 8 seconds 
de421654540d: Retrying in 8 seconds
-------------
The push refers to repository [proxy.domain.local:8443/ara]
edb7a4f74e22: Pushing [==================================================>]  172.6MB/172.6MB
de421654540d: Pushing [==================================================>]  200.8MB/200.8MB
EOF

this is a summary of what happen in wireshark

the.client  my.server   HTTP    316 GET /v2/ HTTP/1.1 
...
my.server   the.client  HTTP    654 HTTP/1.1 401 Unauthorized  (application/json)
...
the.client  my.server   HTTP    442 HEAD /v2/alpine-test/blobs/sha256:95f5ecd24e438e09033c8e69ec136079f8774ab8284f1431f5433a829054b5e7 HTTP/
(asking to nexus if the image is already uploaded)
my.server   the.client  HTTP    493 HTTP/1.1 404 Not Found 
(it isn't)
the.client  my.server   HTTP    437 POST /v2/alpine-test/blobs/uploads/ HTTP/1.1 
(so it start to post the image)
my.server   the.client  HTTP    584 HTTP/1.1 202 Accepted 
...
the.client  my.server   HTTP    437 POST /v2/alpine-test/blobs/uploads/ HTTP/1.1 
...
my.server   the.client  HTTP    584 HTTP/1.1 202 Accepted
..
and so on with some FIN/ACK in the middle until the client stops to send it...

** on nexus server log there is absolutely no trace about this **

this is the nexus docker compose:

services:

  nexus:
    build:
      context: .
      args:
        DOCKER_GID: ${DOCKER_GID}
        NEXUS_UID: ${NEXUS_UID}
        NEXUS_GID: ${NEXUS_GID}
    restart: always
    environment:
      - NEXUS_UID_GID=${NEXUS_UID_GID}
      - HOSTNAME_DOCKER_NEXUS=${HOSTNAME_DOCKER_NEXUS}
    ports:
      - "8081:8081"
      - "8443:8443"
    user: ${NEXUS_UID_GID}
    hostname: ${HOSTNAME_DOCKER_NEXUS}
    volumes:
      - /var/nexus-data:/nexus-data
      - /etc/hosts:/etc/hosts
      - /var/run/docker.sock:/var/run/docker.sock

Can you help me?

I was thinking about a possibile nexus-docker-user permission issue on the local machine/docker binary permissions (if i try from localhost it works, yes, but the image is already stored on the system of course) - but I think it is not so probable.

I was thinking also about proxy configuration issue (more probable), but I don't know much about proxy.

Answer 1

[Workaround] Because I can not figure out the problem, I ended up with make proxy transparent and configuring nexus to serve directly in https throught it's jetty.xml, jetty.https and nexus.properties.

Serving https directly from jetty instead of let the proxy upgrade the connection solved the above problem.