profiling golang runtime.systemstack

Question

I have a networking app written in go. the business logic isn't important.
The problem is I'm struggling to utilize the full capabilites of my machine
and when I profile the applicaiton most of the time is spent in runtime stuff. It seems that ALOT of time is spent under runtime.systemstack
I don't understand what it means or how to increase performance.
I add the profiling of the related slow execution methods.
svg file

Answer 1

If you did cpu profiling ( go test -cpuprofile ), issue 10609 mentioned "runtime: cpu profile is not useful due to systemstack" ( fixed here , for Go 1.5 )

You can have a more interesting view with the recent Go 1.10 trace: see " Using Go 1.10 new trace features to debug an integration test "

go test -trace trace.out
go tool trace -http=localhost:8080 trace.out

See an example here . You will see clearly the inactivity period for each of your processors.