使用 Go Modules 时外部基准测试的最佳实践

Question

I have a Go repository, and within it I have some benchmarks (in a _test suffixed package). These benchmarks compare it to, among other things, some third party libraries. I am not using these libraries in my non-benchmark code.

I am now migrating my repo to go modules. I do not want those third party libraries in my go.mod since my library doesn't need them for normal usage, and I don't want to tie my module to those unnecessarily.

What is the recommended go-mod way to do this? My ideas:

• build tag on the benchmarks
• benchmarks to another repo
• module within my module

Answer 1

If someone wants to run your benchmark (for example, to check whether its stated results hold for their machine configuration), then they need to know what versions of dependencies those benchmarks were originally run with. The information needed to reproduce your test and benchmark results belongs in your go.mod file.

But note that “having a minimum version” is not the same as “importing”.

If a user builds your package but does not build and run its test, or if they build some other package within your module, then they will not need to download the source code for the benchmark dependency even if that dependency is included in your go.mod file.

(And the proposal in https://golang.org/issue/36460 doubles-down on that property: if implemented, that proposal would avoid loading dependencies of packages that are never imported, potentially pruning out large chunks of the dependency graph.)

So if you really don't want users to have to build the dependencies of your benchmark, put the benchmark in a separate package from the one that you expect your users to import.