How to create a docker image which depends on private go packages that are hosted within a subgroup in gitlab?

Question

I'm working on a go project which imports a private package. The private package repo is inside a subgroup in gitlab. I have my development environment set up using a ~/.netrc file and setting GOPRIVATE="gitlab.mycompany.io" and everything works fine.

However, running go mod download during the docker build always fails.

What is REALLY WIERD is that if I build a container which includes all steps before RUN go mod download I can run the container interactively and execute go mod download inside the container without issue.

Here is what my Dockerfile looks like:

FROM golang:latest AS builder

# Adds .netrc
ARG NETRC
RUN echo $NETRC > ~/.netrc
ENV GOPRIVATE="gitlab.mycompany.io/*"
ARG SSH_PRIVATE_KEY
RUN mkdir ~/.ssh && echo "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
RUN echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa && chmod 0600 ~/.ssh/id_rsa

WORKDIR /app
COPY . .

RUN go mod download && go mod verify
RUN GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -ldflags="-w -s"


# FROM scratch (...etc.)

I am running this with the following command, where NETRC and SSH_PRIVATE_KEY contain the cat output of those files.

docker build --no-cache --build-arg NETRC=$NETRC --build-arg SSH_PRIVATE_KEY=$SSH_PRIVATE_KEY -t test-downloader .

This Dockerfile fails with:

remote: 
        remote: ========================================================================
        remote: 
        remote: The project you were looking for could not be found.
        remote: 
        remote: ========================================================================
        remote: 
        fatal: Could not read from remote repository.

        Please make sure you have the correct access rights
        and the repository exists.

Can anyone help me understand:

1. Is there a difference between environments when docker runs a command as a build step and when I run a container interactively?
2. How to make this work? :)

Additional info:

1. The .netrc, .gitconfig, and ssh key look as they should inside the container.
2. I have tried authenticating with .git-credentials and git url (using insteadOf in git config). This fails with the same error and also fails when I run go mod download inside a container that has all preceding steps.
3. I have tried using all manner of replace directives in the go.mod file in order to point mod to the correct package, one such attempt was:

    ...
    gopkg.in/yaml.v2 v2.2.7
)

replace gitlab.mycompany.io/maingroup/subgroup/myapp v0.0.0 => gitlab.mycompany.io/maingroup/subgroup/myapp.git v0.0.0

Answer 1

Alternative solution that worked for me

Perhaps not the answer you are looking for, but a way that worked for me would be to not try to do any sort of authentication or credential stuff in your Dockerfile and go with vendoring the apps. I personally dislike having to have a vendor folder but this is better than having to deal with authentication in Docker.

You would run go mod tidy and go mod vendor outside the Docker command, commit that and then your Dockerfile would something like this (path and workdir might not be the correct ones, the relevant part is how to build):

FROM golang:1.13-alpine AS build-stage

WORKDIR /app

COPY . /app
RUN go build -mod=vendor

# Final Stage
FROM alpine

RUN mkdir /app
WORKDIR /app

# Copy only the binary so that your Docker image does not have the uncompiled code.
COPY --from=build-stage  /app/APP_NAME .

EXPOSE 3000

ENTRYPOINT ./APP_NAME

Cleaner solution that I haven't tried

Another solution is to figure out how to GOPROXY. I haven't tried that solution but it would be a cleaner way. You could use something like Athens: https://docs.gomods.io/configuration/authentication/