What is the cpu cores caused by GC in Golang 1.4.1

Question

I have recently come across a situation where one of our Golang app, consuming almost 30GB memory, will periodically eating all 24 cpu cores with nearly almost 100%. This will last maybe for more than 3 seconds. Our Golang version is 1.4.1 on linux 64-bit.

I have googled for some info. Here is my assumption:

• in our app, we use a data type []map[string]*list and the instance of this type will contain more than 250K keys.
• maybe the gc in golang 1.4.1 consuming more cpu time and stop the world. However, i can not find the parameter to configure the parallelism of gc goroutines(threads). And, does it have some relationship with the GOMAXPROCS parameter.

Answer 1

You've really gotta profile to figure a problem like this out.

That said you might be able to reduce the load you put on the garbage collector. Here are a couple of suggestions:

1. Linked lists result in a lot of small allocations. Have you considered using a []map[string][]whatever_you_are_storing ?
2. Are you adding and removing things from this giant map? Are the things you're adding and removing all basically the same? If so you may be able to use a sync.Pool .
3. Have you tried storing the item directly: map[string]list instead of map[string]*list . It will change the behavior of your program, but for a small struct it may make sense anyway.

Those are shots in the dark.