How to time and profile a *section* of a program in golang with `go test`?

Question

Question : How to time and profile a section of a program in golang with go test ?

Use Case : I have a concurrent bulk operation processing algorithm for B+tree. I am using go test to do the profiling and compare to other baseline algorithms(serialized version, pessimistic locking etc). For the test case setup, I will create a B+tree with 1M entries and create a list of 1M operations, then I start the actual test to BulkProcess these operations.

func TestInputTreeM1e6N1e6(*testing.T) {
    M := 1000000

    //Test Preparation 1: Setup the tree
    tree := NewTree(cmp)
    file1name := "InitalTree_10000000.txt"

    testF1, err := os.Open(file1name)
    if err != nil {
        panic(err)
    }
    defer testF1.Close()

    nums, _ := ReadInts(testF1)
    for i, num := range nums {
        if i == M {
            break
        }
        tree.Set(Int(num), Int(1))
    }


    fmt.Println("Tree initialized")

    //Test Prepration 2: Initialize operations
    N := 1000000 
    inputs := make([]SearchPair, N)

    file2name := "Operations_10000000.txt"
    testF2, err := os.Open(file2name)
    if err != nil {
        panic(err)
    }
    defer testF2.Close()

    var opt string
    var num int
    var input SearchPair
    for i:=0; i < N; i++{

        fmt.Fscanf(testF2, "%s %d", &opt, &num)

        switch opt {
        case "DEL":
            input = SearchPair{
                opt: DEL,   //Operation to be applied
                k: Int(num),    //k to be operated upon
                v: Int(0),
            }
        case "PUT":
            input = SearchPair{
                opt: PUT,   //Operation to be applied
                k: Int(num),    //k to be operated upon
                v: Int(-1),
            }
        case "SET":
            input = SearchPair{
                opt: SET,   //Operation to be applied
                k: Int(num),    //k to be operated upon
                v: Int(2),
            }
        }

        inputs[i] = input
    }

    //Start actual testing
    fmt.Println("Start processing...")
    tree.BulkProcess(inputs)
    fmt.Println("Done")

}

Problem : I'm using go test -v -cpuprofile cpu.out -memprofile mem.out to profile BulkProcess performance, then use go tool pprof --pdf ConcurrentBPlusTree.test mem.out > mgraph.pdf and go tool pprof --pdf ConcurrentBPlusTree.test cpu.out > cgraph.pdf to visualize the result.

Right now, go test seems to be profiling the entire test case including the tree building and operations building. The file I/O during test setup has been dominating the performance to the extend where I could not even see what's going on in my actual testing.

Also, go test have output lines like this

=== RUN   TestInputTreeM1e6N1e6
--- PASS: TestInputTreeM1e6N1e6 (63.36s)

Is it possible to configure the timer so that it only shows the time spent on bulk processing instead of the entire case? I know b *testing.B has API such as b.ResetTimer() and b.StopTimer() . Is there similar API for *testing.T so I can modify the timing behavior?

Specific Question

1. How can I use go test to profile only the last three lines of my current test case?
2. How can I use go test to time only the last three lines of my current test case?

Answer 1

If you don't want to profile the entire program, you can start and stop a CPUProfile yourself using the runtime/pprof package:

f, err := os.Create(profileFileName)
if err != nil {
    log.Fatal("could not create CPU profile: ", err)
}
if err := pprof.StartCPUProfile(f); err != nil {
    log.Fatal("could not start CPU profile: ", err)
}
defer pprof.StopCPUProfile()