Golang compilation cache from Docker

Question

I'm using the official golang alpine image to compile my source code (my host machine is a Mac), and I've noticed that even when mounting whole $GOPATH inside of the container it doesn't use cached data from previous builds. I checked that it creates it in the $GOPATH/pkg directory, but it does not affect the subsequent builds speed.

However, if I reuse the same container for several compilation, it does make use of some kind of cache, you can see the results in this experiment I did:

Using different containers, time remains around 28-30s in each build:

$ rm -r $GOPATH/pkg/linux_amd64
$ time docker run -v$GOPATH:/go -e CGO_ENABLED=0 golang:1.9-alpine3.6 go build -i github.com/myrepo/mypackage
...
0.02s user 0.08s system 0% cpu 30.914 total

$ time docker run -v$GOPATH:/go -e CGO_ENABLED=0 golang:1.9-alpine3.6 go build -i github.com/myrepo/mypackage
...
0.02s user 0.07s system 0% cpu 28.128 total

Reusing the same container, subsequent builds are much faster:

$ rm -r $GOPATH/pkg/linux_amd64    
$ docker run -d -v$GOPATH:/go -e CGO_ENABLED=0 golang:1.9-alpine3.6 tail -f /dev/null
bb4c08867bf2a28ad87facf00fa9dcf2800ad480fe1e66eb4d8a4947a6efec1d

$ time docker exec bb4c08867bf2 go build -i github.com/myrepo/mypackage
...
0.02s user 0.05s system 0% cpu 27.028 total

$ time docker exec bb4c08867bf2 go build -i github.com/myrepo/mypackage
0.02s user 0.06s system 0% cpu 7.409 total

Is Go using any kind of cache in some place outside of $GOPATH ?

Answer 1

To anyone who landed here from google search, i have found a working answer on a reddit post .

It basically says to map the /root/.cache/go-build to your host go build cache folder.

In my case, i am on windows and have a project that requires cross compilation with gcc, i had to spin up a linux container to build the binary to be deploy to a alpine container, and i map it to a data volume instead:

some-volume-name:/root/.cache/go-build

Answer 2

When you are building inside the golang container, it is using the directory $GOPATH/pkg inside this container . If you then start another golang container, it has an empty $GOPATH/pkg. However if you continue to use the same container (with exec), the $GOPATH/pkg is re-used.

rm -r $GOPATH/pkg/linux_amd64 will only remove this directory on your local machine. So this has no effect.

A possible alternative to re-using the same container could be

• to commit the container after the first build, or
• to mount $GOPATH/pkg as volume from your host machine or a data volume.

Answer 3

Use the -v flag to print which packages are getting compiled. This might be a better indicator than time spent.

I was able to produce the desired result by mounting the gopath as volume (as you have done, so it should work...). Please see below snippet. First time it compiles both packages, second time just the main.

Side note: one issue i've had with this approach is the volume dir will "overwrite" (ie shadow) anything already in the image at that dir, which is fine if you are using just the base golang alpine image since /go should be empty.

pkm$ tree
.
└── src
    └── github.com
        ├── org1
        │   └── mine
        │       └── main.go
        └── org2
            └── somelib
                └── lib.go

6 directories, 2 files
pkm$ docker run --rm -v $GOPATH:/go golang:1.9-alpine go build -i -v github.com/org1/mine
github.com/org2/somelib
github.com/org1/mine
pkm$ tree
.
├── mine
├── pkg
│   └── linux_amd64
│       └── github.com
│           └── org2
│               └── somelib.a
└── src
    └── github.com
        ├── org1
        │   └── mine
        │       └── main.go
        └── org2
            └── somelib
                └── lib.go

10 directories, 4 files
pkm$ docker run --rm -v $GOPATH:/go golang:1.9-alpine go build -i -v github.com/org1/mine
github.com/org1/mine
pkm$