GitHub / private repository / SSH: go get/go mod tidy fail, while git clone is working

Question

I hope to find the answer to the question which is bothering me for the past couple of days - I have read a lot of threads, helpers, and publications, and do not seem to find a solution. I am a bit new with SSH.

The problem is as follows: I cannot get/install/refresh the modules belonging to my project, when using go get or go mod tidy commands, while git clone of the same repo via SSH in any of the terminals or via TortoiseGit are working fine, ie I can clone, but I cannot build.

getting a module fails the following way (personal data obfuscated):

    example.com/hello/PROJECTS/src/git.<orgname>.com/<reponame>/<subreponame>/tests/functional/resources tested by
    example.com/hello/PROJECTS/src/git.<orgname>.com/<reponame>/<subreponame>/tests/functional/resources.test imports
    git.<orgname>.com/<reponame>/<subreponame>/tests/functional/resources/vm_specific: module git.<orgname>.com/<reponame>/<subreponame>/tests/functional/resources/vm_specific: git ls-remote -q origin in C:\GO\PROJECTS\pkg\mod\cache\vcs\a5d519aeafc0ac08e2b20d7c8a6a8b2cea11cda88cb058cf2ebcc079d07b260a: exit status 128:
    debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> 'C:\\Users\\<username>/.ssh/known_hosts'
    debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> 'C:\\Users\\<username>/.ssh/known_hosts2'
    debug2: resolving "git.<orgname>.com" port <port>
    debug3: resolve_host: lookup git.<orgname>.com:<port>
    debug3: ssh_connect_direct: entering
    debug1: Connecting to git.<orgname>.com [10.XXX.X.XX] port <port>.
    debug1: Connection established.
    ...
    debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.9
    debug1: Remote protocol version 2.0, remote software version APACHE-SSHD-2.4.0
    debug1: compat_banner: no match: APACHE-SSHD-2.4.0
    debug2: fd 3 setting O_NONBLOCK
    debug1: Authenticating to git.<orgname>.com:<port> as 'git'
    ...
    debug1: Authentications that can continue: publickey
    ...
    debug3: send packet: type 50
    debug3: receive packet: type 52
    Authenticated to git.<orgname>.com ([10.XXX.X.XX]:<port>) using "publickey".
    ...
    debug1: Sending command: git-upload-pack '/scm/<reponame>/<subreponame>.git'
    debug2: channel 0: request exec confirm 1
    debug3: send packet: type 98
    debug2: channel_input_open_confirmation: channel 0: callback done
    debug2: channel 0: open confirm rwindow 2097152 rmax 32768
    debug3: receive packet: type 99
    debug2: channel_input_status_confirm: type 99 id 0
    debug2: exec request accepted on channel 0
    debug2: channel 0: rcvd ext data 106
    debug3: receive packet: type 96
    debug2: channel 0: rcvd eof
    debug2: channel 0: output open -> drain
    debug3: receive packet: type 98
    debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
    debug3: receive packet: type 97
    debug2: channel 0: rcvd close
    debug2: chan_shutdown_read: channel 0: (i0 o1 sock -1 wfd 4 efd 6 [write])
    debug2: channel 0: input open -> closed
    debug3: channel 0: will not send data after close
    debug2: channel 0: obuf_empty delayed efd 6/(106)
    debug2: channel 0: written 106 to efd 6
    Repository not found
    The requested repository does not exist, or you do not have permission to access it.
    debug3: channel 0: will not send data after close
    debug2: channel 0: obuf empty
    debug2: chan_shutdown_write: channel 0: (i3 o1 sock -1 wfd 5 efd 6 [write])
    debug2: channel 0: output drain -> closed
    fatal: Could not read from remote repository.

    Please make sure you have the correct access rights
    and the repository exists.
    debug2: channel 0: almost dead
    debug2: channel 0: gc: notify user
    debug2: channel 0: gc: user detached
    debug2: channel 0: send close
    debug3: send packet: type 97
    debug2: channel 0: is dead
    debug2: channel 0: garbage collecting
    debug1: channel 0: free: client-session, nchannels 1
    debug3: channel 0: status: The following connections are open:
      #0 client-session (t4 r0 i3/0 o3/0 e[write]/0 fd -1/-1/6 sock -1 cc -1 io 0x00/0x08)

    debug3: send packet: type 1
    Transferred: sent 3268, received 2200 bytes, in 0.2 seconds
    Bytes per second: sent 15169.6, received 10212.1
    debug1: Exit status 1

If a fuller log is needed I can upload it somewhere.

last rows of ssh -T(vvv set as default) git@git.<orgname>.com :

    debug3: send packet: type 50
    debug3: receive packet: type 52
    debug1: Authentication succeeded (publickey).
    Authenticated to git.<orgname>.com ([10.XXX.X.XX]:<port>).
    debug1: channel 0: new [client-session]
    debug3: ssh_session2_open: channel_new: 0
    debug2: channel 0: send open
    debug3: send packet: type 90
    debug1: Entering interactive session.
    debug1: pledge: filesystem full
    debug3: receive packet: type 91
    debug2: channel_input_open_confirmation: channel 0: callback start
    debug2: fd 3 setting TCP_NODELAY
    debug3: set_sock_tos: set socket 3 IP_TOS 0x08
    debug2: client_session2_setup: id 0
    debug2: channel 0: request shell confirm 1
    debug3: send packet: type 98
    debug2: channel_input_open_confirmation: channel 0: callback done
    debug2: channel 0: open confirm rwindow 2097152 rmax 32768
    debug3: receive packet: type 100
    debug2: channel_input_status_confirm: type 100 id 0
    shell request failed on channel 0

I am using a public key deriving from a key generated by ssh-keygen of OpenSSH - it is freshly generated and is added to my Git profile. git clone is thus working fine which is driving me insane already. I already have the most part of the modules which fail to be checked in the remote private repo, which is even crazier.

I dislike this part, which most probably means that I have issues with my certificate -

Authenticated to git..com ([10.XXX.X.XX]:) using "publickey".

and this part, as OpenSSH on AIX requests PKCS12 support by default when EFS is enabled and non-AIX systems may reject these PKCS12 requests.

 debug2: channel 0: obuf_empty delayed efd 6/(106) debug2: channel 0: written 106 to efd 6

See changes in my config below.

My environment:

• Win10 Enterprise
• Git version - 2.31.1.windows.1
• I usually clone via TortoiseGit 2.13.0.1
• SSH client being used within CMD/Powershell - OpenSSH_for_Windows_8.9, installed manually

go env: (personal data obfuscated)

    set GO111MODULE=
    set GOARCH=amd64
    set GOBIN=
    set GOCACHE=C:\Users\<username>\AppData\Local\go-build
    set GOENV=C:\Users\<username>\AppData\Roaming\go\env
    set GOEXE=.exe
    set GOEXPERIMENT=
    set GOFLAGS=
    set GOHOSTARCH=amd64
    set GOHOSTOS=windows
    set GOINSECURE=
    set GOMODCACHE=C:\Program Files\Go\pkg\mod
    set GONOPROXY=*.<orgname>.com;git.<orgname>.com/users/<username>
    set GONOSUMDB=*.<orgname>.com;git.<orgname>.com/users/<username>
    set GOOS=windows
    set GOPATH=C:\Program Files\Go
    set GOPRIVATE=*.<orgname>.com;git.<orgname>.com/users/<username>
    set GOPROXY=https://proxy.golang.org,direct
    set GOROOT=C:\Program Files\Go
    set GOSUMDB=sum.golang.org
    set GOTMPDIR=
    set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
    set GOVCS=
    set GOVERSION=go1.18.3
    set GCCGO=gccgo
    set GOAMD64=v1
    set AR=ar
    set CC=gcc
    set CXX=g++
    set CGO_ENABLED=1
    set GOMOD=NUL
    set GOWORK=
    set CGO_CFLAGS=-g -O2
    set CGO_CPPFLAGS=
    set CGO_CXXFLAGS=-g -O2
    set CGO_FFLAGS=-g -O2
    set CGO_LDFLAGS=-g -O2
    set PKG_CONFIG=pkg-config
    set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\USER~1.NAM\AppData\Local\Temp\go-build1143995380=/tmp/go-build -gno-record-gcc-switches

My SSH config:

    # GITHUB FOR SRC
        Host git.<orgname>.com
           ServerAliveInterval 600
           TCPKeepAlive yes
           IPQoS throughput
           AddKeysToAgent yes
           HostName git.<orgname>.com
           Port <port>
           User <username>
           PreferredAuthentications publickey
           IdentityFile C:\Users\<username>\.ssh\git_rsa
           #IdentityFile ~/.ssh/git_rsa
           PubkeyAcceptedKeyTypes +ssh-rsa
           #ForwardAgent no
           #AllowPKCS12KeystoreAutoOpen no
           LogLevel DEBUG3

• The certificate was previously added to the agent, but I have specified it here to be sure, and the correct certificate seems to be taken for authentication according to the log.

• It does not have a passphrase.

• I do not have totally unknown entries in my known-hosts , and the correct entry judging by a certificate timestamp is being used.

• I found that the PKCS12 requests can be disabled by AllowPKCS12KeystoreAutoOpen set to no , but it is being marked as a bad option -

   C:\GO\PROJECTS\src\git.<orgname>.com\<reponame>\<subreponame>>go get git.<orgname>.com/<name>/go-service/log go: module git.<orgname>.com/<name>/go-service/log: git ls-remote -q origin in C:\GO\PROJECTS\pkg\mod\cache\vcs\d0d607237eeba0d1c9d5ce996ed36c0f3746b2c8f94b538ace2d3f2a9476839e: exit status 128: C:\\Users\\<username>/.ssh/config: line 15: Bad configuration option: allowpkcs12keystoreautoopen C:\\Users\\<username>/.ssh/config: terminating, 1 bad configuration options

GIT configs:

• global:

   [user] name = <username> email = <usrname>@<orgname>.com [url "ssh://git@git.<orgname>.com:<port>/"] insteadOf = https://git.<orgname>.com

• system:

   [url "ssh://git@git.<orgname>.com:<port>/"] insteadOf = https://git.<orgname>.com/

git remote -v:

    origin  ssh://git@git.<orgname>.com:<port>/~<username>/<subreponame>.git (fetch)
    origin  ssh://git@git.<orgname>.com:<port>/~<username>/<subreponame>.git (push)

• the same I use for cloning,

repo config:

    [remote "origin"]
        url = ssh://git@git.<orgname>.com:<port>/~<username>/<subreponame>.git
        fetch = +refs/heads/*:refs/remotes/origin/*

I tried both ssh:// and git@git... and it did not help.

module config (set automatically), however:

    [remote "origin"]
        url = https://git.<orgname>.com/scm/<reponame>/<subreponame>.git
        fetch = +refs/heads/*:refs/remotes/origin/*

I am out of ideas at this moment.

Answer 1

I'd recommend not to use go get with private repositories. This command caches repositories at pkg.go.dev and thus exposes them to the world. I was so embarrassed to find our private development published openly there.

I deploy my private dependencies in the file system manually and specify the dependency in go.mod using replace:

require your.server/your/package 1.2.3

replace your.server/your/package 1.2.3 => /path/to/local/your/package

Answer 2

That was an issue with repository - it was not really made to work with dependencies, and was not meant to work with modules and module-related requests. C'est la vie, but at least I've learned a lot about SSH and GIT interaction.