Resolving 'cannot find package' error with vendor in go 1.7

Question

I have a project structure which looks like below:-

session-service
    _libs   //Contains all the external dependencies
    api
    constants
    exceptions
    idgen
    jsonDecoder
    log
    model
    monitor
    persistence
    redis
    routes
    src/bddtest/servicetest
    util

Content of _libs looks like below:-

github.com 
golang.org 
    x
        net
gopkg.in

My Makefile looks like below:-

.PHONY: deploy

LOGLEVEL ?= 1
CONFIGFILE ?= 2
GOFLAGS ?= $(GOFLAGS:)

PWD = $(shell pwd)
export GOPATH = $(shell echo $$GOPATH):$(PWD)/_libs:$(PWD)
export GOBIN = $(PWD)/bin
export GOROOT = $(shell echo $$GOROOT)

deploy: clean build install 

build:
    @rm -rf pkg/ 2>/dev/null
    @rm -rf _libs/pkg/ 2>/dev/null
    @go build $(GOFLAGS) ./...

install:
    @go install ./...

clean:
    @go clean $(GOFLAGS) -i ./...


## EOF

Everything is working fine. Now I am thinking of moving to vendor . So I renamed my _libs to vendor and modified my Makefile like below:-

export GOPATH = $(shell echo $$GOPATH):$(PWD)

But after this I started getting the following error:-

vendor/golang.org/x/net/html/charset/charset.go:20:2: cannot find package "golang.org/x/text/encoding" in any of:
        /Users/debraj/golang/src/b/m/session-service/vendor/golang.org/x/text/encoding (vendor tree)
        /usr/local/go/src/golang.org/x/text/encoding (from $GOROOT)
        /Users/debraj/golang/src/golang.org/x/text/encoding (from $GOPATH)
        /Users/debraj/golang/src/b/m/session-service/src/golang.org/x/text/encoding
    vendor/golang.org/x/net/html/charset/charset.go:21:2: cannot find package "golang.org/x/text/encoding/charmap" in any of:
        /Users/debraj/golang/src/b/m/session-service/vendor/golang.org/x/text/encoding/charmap (vendor tree)
        /usr/local/go/src/golang.org/x/text/encoding/charmap (from $GOROOT)
        /Users/debraj/golang/src/golang.org/x/text/encoding/charmap (from $GOPATH)
        /Users/debraj/golang/src/b/m/session-service/src/golang.org/x/text/encoding/charmap
    vendor/golang.org/x/net/html/charset/charset.go:22:2: cannot find package "golang.org/x/text/encoding/htmlindex" in any of:
        /Users/debraj/golang/src/b/m/session-service/vendor/golang.org/x/text/encoding/htmlindex (vendor tree)
        /usr/local/go/src/golang.org/x/text/encoding/htmlindex (from $GOROOT)
        /Users/debraj/golang/src/golang.org/x/text/encoding/htmlindex (from $GOPATH)
        /Users/debraj/golang/src/b/m/session-service/src/golang.org/x/text/encoding/htmlindex
    vendor/golang.org/x/net/html/charset/charset.go:23:2: cannot find package "golang.org/x/text/transform" in any of:
        /Users/debraj/golang/src/b/m/session-service/vendor/golang.org/x/text/transform (vendor tree)
        /usr/local/go/src/golang.org/x/text/transform (from $GOROOT)
        /Users/debraj/golang/src/golang.org/x/text/transform (from $GOPATH)
        /Users/debraj/golang/src/b/m/session-service/src/golang.org/x/text/transform
    vendor/golang.org/x/net/http2/h2i/h2i.go:38:2: cannot find package "golang.org/x/crypto/ssh/terminal" in any of:
        /Users/debraj/golang/src/b/m/session-service/vendor/golang.org/x/crypto/ssh/terminal (vendor tree)
        /usr/local/go/src/golang.org/x/crypto/ssh/terminal (from $GOROOT)
        /Users/debraj/golang/src/golang.org/x/crypto/ssh/terminal (from $GOPATH)
        /Users/debraj/golang/src/b/m/session-service/src/golang.org/x/crypto/ssh/terminal

Environment:-

• go version go1.7.3 darwin/amd64
• Mac OS X 10.11.6

Can someone let me know why I am getting the above errors with vendor but everything works fine with _libs ?

UPDATE

In my local the newlines in the output of $(go list ./... | grep -v /vendor/) was causing some problem. So to resolve this I had to modify the jimb 's solution a bit. I introduced a variable in Makefile PKG = $(shell go list ./... | grep -v /vendor/ | tr "\\n" " ") and then used that variable in go install & go build like @go build $(GOFLAGS) $(PKG)

Answer 1

The _libs directory starts with _ , and is ignored by the go tool. When you move the packages to vendor/ , the ./... wildcard now includes all packages in the vendor directory.

You should explicitly list the package you want to install, rather than rely on the ./... wildcard. If you still want the wildcard behavior, you can use go list ./... and filter any package containing a vendor/ directory in their path. Depending on your specific needs, this could be as simple as:

go install $(go list ./... | grep -v vendor/)