Goroutines with sync.WaitGroup end before last wg.Done()

Question

I have an example code (you can find it on Go Playground ):

package main

import (
    "fmt"
    "sync"
    "time"
)

func main() {
    messages := make(chan int)
    var wg sync.WaitGroup
    var result []int

    // you can also add these one at 
    // a time if you need to 

    wg.Add(1)
    go func() {
        defer wg.Done()
        time.Sleep(time.Second * 1)
        messages <- 1
    }()
    wg.Add(1)
    go func() {
        defer wg.Done()
        time.Sleep(time.Second * 1)
        messages <- 2
    }() 
    wg.Add(1)
    go func() {
        defer wg.Done()
        time.Sleep(time.Second * 1)
        messages <- 3
    }()
    go func() {
        for i := range messages {
            fmt.Println(i)
        result = append(result, i)
        }

    }()

    wg.Wait()
    fmt.Println(result)
}

I got this output:

2
1
[2 1]

I think I know why is it happening, but I can't solve it. There is 3 item in the WaitGroup I mean the three goroutine, and the 4th groutine consume the data from the channel. When the last groutine say wg.Done() the program is over because of the wg.Wait() said every goroutine finished and the last goroutine result the 4th goroutine couldn't have consume, because the program ended. I tryed to add plus one with wg.Add(1) and the wg.Done() in the 4th function but in this case I got deadlock.

Answer 1

The last goroutine you spawn—that one intended to collect the results—is not waited by main() so wg.Wait() in there returns, main() quits and reaps the remainin goroutines. Supposedly just a single—collecting—goroutine remains by that time but it fails to update the slice.

Also note that due to the same reason you have a data race in your program: by the time main() reads the slice of results, it does not know whether it's safe to read it—that is, whether the writer is done writing to there.

An easy fix is to do add wg.Add(1) for that goroutine and defer wg.Done() in it, too.

A better solution is to close() the messages channel after wg.Wait() and before reading from the slice. This would make the collecting goroutine's range loop to terminate and this would also create a proper synchronization point between that goroutine and main() .

Answer 2

Closing channels is an idiomatic Go pattern for signalling and if you close a buffered channel, the consumer can read all the queued data then stop.

This code works correctly:

func main() {
    messages := make(chan int)
    var wg sync.WaitGroup
    var result []int

    // you can also add these one at
    // a time if you need to

    wg.Add(1)
    go func() {
        defer wg.Done()
        time.Sleep(time.Second * 1)
        messages <- 1
    }()
    wg.Add(1)
    go func() {
        defer wg.Done()
        time.Sleep(time.Second * 1)
        messages <- 2
    }()
    wg.Add(1)
    go func() {
        defer wg.Done()
        time.Sleep(time.Second * 1)
        messages <- 3
    }()

    // this goroutine added to signal end of data stream
    // by closing messages channel
    go func() {
        wg.Wait()
        close(messages)
    }()

    // if you need this to happen inside a go routine,
    // this channel is used for signalling end of the work,
    // also another sync.WaitGroup could be used, but for just one
    // goroutine, a single channel as a signal makes sense (there is no
    // groups)
    done := make(chan struct{})
    go func() {
        defer close(done)
        for i := range messages {
            fmt.Println(i)
            result = append(result, i)
        }
    }()

    <-done
    fmt.Println(result)
}

As you see we just added another goroutine that closes the messages channel, when all producers are done.

Answer 3

kostix 's answer was correct till they mentioned

An easy fix is to do add wg.Add(1) for that goroutine and defer wg.Done() in it, too.

That would cause your loop to never finish without the messages channel being closed! So the main goroutine would again finish before your last "collecting" goroutine finishes. You would also get an error for having a goroutine tied to your wg WaitGroup that will never send the Done() signal.

Then again when they mentioned

A better solution is to close() the messages channel after wg.Wait() and before reading from the slice

The placement they suggested will again give you the same error since you'll be waiting on the same WaitGroup wg . While your last "collecting" goroutine will keep looking for more messages in your messages channel and will never reach the deferred wg.Done()

Then Alex Yu 's comment fixed it by waiting before reading the results altogether, which is a good fix. But if you want your collecting goroutine to start right away and NOT wait for all previous goroutines (that write to messages channel) to finish before it starts reading from said channel, I would suggest the following...

Create a result WaitGroup, Add(1) before starting your last "collecting" goroutine, defer wgResult.Done() inside your last "collecting" goroutine, then at the end, between your wg.Wait() and your fmt.Println(result) , you should close(messages) and wgResult.Wait() .

This allows all your go routines to start as soon as possible and waiting only when needed on the writing goroutines as well as the reading one.

Here's a GoPlayground link with the suggested solution

https://play.golang.org/p/na0JS1HTwNP