golang: Make select statemet deterministic?

Question

Let's look carefully at the Ticker example code in Go's time package:

package main

import (
    "fmt"
    "time"
)

func main() {
    ticker := time.NewTicker(time.Second)
    defer ticker.Stop()
    done := make(chan bool)
    go func() {
        time.Sleep(10 * time.Second)
        done <- true
    }()
    for {
        select {
        case <-done:
            fmt.Println("Done!")
            return
        case t := <-ticker.C:
            fmt.Println("Current time: ", t)
        }
    }
}

With the interval adjusted to 1 second for convenience, after running the example enough times, we see an instance where the current time is never printed (or it would have only printed 9 times rather than 10):

Current time:  2020-06-10 12:23:51.189421219 -0700 PDT m=+1.000350341
Done!
Current time:  2020-06-10 12:23:52.193636682 -0700 PDT m=+1.000473686
Done!
Current time:  2020-06-10 12:23:53.199688564 -0700 PDT m=+1.000322824
Done!
Current time:  2020-06-10 12:23:54.204380186 -0700 PDT m=+1.000420293
Done!
Current time:  2020-06-10 12:23:55.21085129 -0700 PDT m=+1.000266810
Done!
Done!
Current time:  2020-06-10 12:23:57.220120615 -0700 PDT m=+1.000479431
Done!
Current time:  2020-06-10 12:23:58.226167159 -0700 PDT m=+1.000443199
Done!
Current time:  2020-06-10 12:23:59.231721969 -0700 PDT m=+1.000316117
Done!

When both the done and ticker.C channels are ready concurrently, we enter the realm of Go nondeterministic behavior:

A select blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready.

I understand Go's design rationale for why select is non-deterministic. It mostly boils down to a problem the language does not venture to solve because doing so is generally hard and may lead users to write unknowingly racy code, and thus prioritized select and exercise left to the reader.

Let's assume that, for whatever reason, I'd like to ensure all pending ticks are consumed prior to winding down the program and printing Done! . Is there a general transformation that can be applied to this simple example to make it deterministic?

I tried adding another signal channel:

func main() {
    ticker := time.NewTicker(time.Second)
    stop := make(chan bool)
    done := make(chan bool)
    tick := make(chan time.Time)
    go func() {
        time.Sleep(1 * time.Second)
        stop <- true
    }()
    go func() {
        for t := range tick {
            fmt.Println("Current time: ", t)
        }
        done <- true
    }()
    for {
        select {
        case <-stop:
            ticker.Stop()
            close(tick)
        case t := <-ticker.C:
            tick <- t
            break
        case <-done:
            fmt.Println("Done!")
            return
        }
    }
}

But it seems to preform worse...

Current time:  2020-06-10 13:23:20.489040642 -0700 PDT m=+1.000425216
Done!
Current time:  2020-06-10 13:23:21.495263288 -0700 PDT m=+1.000338902
Done!
Current time:  2020-06-10 13:23:22.501474055 -0700 PDT m=+1.000327127
Done!
Current time:  2020-06-10 13:23:23.503531868 -0700 PDT m=+1.000244398
Done!
Current time:  2020-06-10 13:23:24.510210786 -0700 PDT m=+1.000420955
Done!
Current time:  2020-06-10 13:23:25.516500359 -0700 PDT m=+1.000460986
Done!
Done!
Current time:  2020-06-10 13:23:27.527077433 -0700 PDT m=+1.000375330
Done!
Current time:  2020-06-10 13:23:28.533401667 -0700 PDT m=+1.000470273
Done!
panic: send on closed channel

goroutine 1 [running]:
main.main()
    /home/dcow/Desktop/ticker-go/main2.go:29 +0x22f
Current time:  2020-06-10 13:23:30.547554719 -0700 PDT m=+1.000399602
Done!
Current time:  2020-06-10 13:23:31.55416725 -0700 PDT m=+1.000443683
Done!
Current time:  2020-06-10 13:23:32.56041176 -0700 PDT m=+1.000436364
Done!
Done!
Current time:  2020-06-10 13:23:34.572550584 -0700 PDT m=+1.000445593
Done!
Current time:  2020-06-10 13:23:35.578672712 -0700 PDT m=+1.000357330
Done!
Done!
Current time:  2020-06-10 13:23:37.590984117 -0700 PDT m=+1.000447504
Done!

We can't guarantee that we won't receive the stop message at the same time as we receive the final tick, so we've just shuffled the problem around to something that panics when it behaves "incorrectly" (which is marginally better than doing so silently). If we nil ed the tick channel, we'd devolve to the original case. And we still have cases where no tick is printed at all likely because it's possible we close the timer before it ever has a chance to fire..

How about a ready channel?

func main() {
    ticker := time.NewTicker(time.Second)
    tick := make(chan time.Time)
    ready := make(chan bool, 1)
    stop := make(chan bool)
    done := make(chan bool)
    go func() {
        time.Sleep(1 * time.Second)
        <-ready
        stop <- true
    }()
    go func() {
        for t := range tick {
            fmt.Println("Current time: ", t)
        }
        done <- true
    }()
    for {
        select {
        case <-stop:
            ticker.Stop()
            close(tick)
        case t := <-ticker.C:
            select {
            case ready<-true:
                break
            default:
            }
            tick <- t
            break
        case <-done:
            fmt.Println("Done!")
            return
        }
    }
}

This seems to work. It's somewhat involved with the addition of 3 new channels and an additional go routine, but it hasn't failed thus far. Is this pattern idiomatic in go? Are there general form strategies for applying this type of transformation in scenarios where you want to prioritize one of the select cases? Most advice I've come across is related to sequential and nested selects which don't really solve the problem.

Alternatively, is there a way to say "give me the list of ready channels so I can pick the order in which I process them"?

Edit:

Adding some clarifying remarks: I'm not interested in preserving ordering of concurrent operations. I agree that's a silly endeavor. I simply want to be able know if a selection of channels are ready to be processed and provide my own logic dictating what to do when multiple channels are ready concurrently. I'm essentially interested in a Go analog to POSIX select . And/or I'm interested in literature describing or common knowledge surrounding a generalized "convert non-deterministic select to deterministic select in Go" pattern.

eg Do people use the heap package and deposit data into a priority queue and ultimately read from that? Is there an x/reflect style package that implements a prioritized select using unsafe? Is there some simple pattern like, "Convert all selects with a single channel that should take priority into a dual channel style and forward the "done" request the the producer which in turn should terminate and close their channel then block on a channel range loop (kinda like my working solution)? Actually, lock on a shared condition variable for reasons x, y. etc..

Answer 1

Unless the application has some known ordering between ready state of the ticker and done channels, it's impossible to ensure that the application processes the values from the channels in the order that the values are sent.

The application can ensure that values queued in ticker.C are received before a value from done by using nested select statements.

for {
    select {
    case t := <-ticker.C:
        fmt.Println("Current time: ", t)
    default:
        // ticker.C is not ready for commination, wait for both 
        // channels.
        select {
        case <-done:
            fmt.Println("Done!")
            return
        case t := <-ticker.C:
            fmt.Println("Current time: ", t)
        }
    }
}

If the done communication is executed before a ready <-ticker.C communication in the inner select, then the two channels entered the ready state at almost the same time. Unless there's a requirement not stated in the question, this shouldn't make a difference to the application.

The application can nest a third select to give receive on ticker.C one last opportunity to execute before the function returns. This approach gives priority to the ticker when the two channels enter the ready state at almost the same time. I mention this for completeness, not because I recommend it. As I said in the previous paragraph, the first snippet of code in this answer should be good enough.

for {
    select {
    case t := <-ticker.C:
        fmt.Println("Current time: ", t)
    default:
        // ticker.C is not ready for commination, wait for both
        // channels.
        select {
        case <-done:
            // Give communication on <-ticker.C one last
            // opportunity before exiting.
            select {
            case t := <-ticker.C:
                // Note that the ticker may have entered
                // the ready state just after the done channel
                // entered the state. 
                fmt.Println("Current time: ", t)
            default:
            }
            fmt.Println("Done!")
            return
        case t := <-ticker.C:
            fmt.Println("Current time: ", t)
        }
    }
}

Answer 2

If you need to pick one channel over another when both are enabled, then you can do a nested select. This will pick the high priority one over the low priority one if both channels are enabled at the beginning of select:

select {
  case <-highPriority:
     // Deal with it
  default:
     select {
       case <-lowPriority:
         // low priority channel
       default:
     }
}

If you have N channels with a priority ranking, then you can try selecting in a loop:

for _,channel:=range channels {
   select {
     case <-channel:
      //
     default:
   }
}

This of course will be an approximation of what you need because it'll miss channel state changes that happen while it is looping. But it will prioritize channels based on their state at the beginning of the for loop.

Then there is reflect.Select , but that will not prioritize.

Answer 3

Let's look carefully at the Ticker example code in Go's time package:

package main

import (
    "fmt"
    "time"
)

func main() {
    ticker := time.NewTicker(time.Second)
    defer ticker.Stop()
    done := make(chan bool)
    go func() {
        time.Sleep(10 * time.Second)
        done <- true
    }()
    for {
        select {
        case <-done:
            fmt.Println("Done!")
            return
        case t := <-ticker.C:
            fmt.Println("Current time: ", t)
        }
    }
}

With the interval adjusted to 1 second for convenience, after running the example enough times, we see an instance where the current time is never printed (or it would have only printed 9 times rather than 10):

Current time:  2020-06-10 12:23:51.189421219 -0700 PDT m=+1.000350341
Done!
Current time:  2020-06-10 12:23:52.193636682 -0700 PDT m=+1.000473686
Done!
Current time:  2020-06-10 12:23:53.199688564 -0700 PDT m=+1.000322824
Done!
Current time:  2020-06-10 12:23:54.204380186 -0700 PDT m=+1.000420293
Done!
Current time:  2020-06-10 12:23:55.21085129 -0700 PDT m=+1.000266810
Done!
Done!
Current time:  2020-06-10 12:23:57.220120615 -0700 PDT m=+1.000479431
Done!
Current time:  2020-06-10 12:23:58.226167159 -0700 PDT m=+1.000443199
Done!
Current time:  2020-06-10 12:23:59.231721969 -0700 PDT m=+1.000316117
Done!

When both the done and ticker.C channels are ready concurrently, we enter the realm of Go nondeterministic behavior:

A select blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready.

I understand Go's design rationale for why select is non-deterministic. It mostly boils down to a problem the language does not venture to solve because doing so is generally hard and may lead users to write unknowingly racy code, and thus prioritized select and exercise left to the reader.

Let's assume that, for whatever reason, I'd like to ensure all pending ticks are consumed prior to winding down the program and printing Done! . Is there a general transformation that can be applied to this simple example to make it deterministic?

I tried adding another signal channel:

func main() {
    ticker := time.NewTicker(time.Second)
    stop := make(chan bool)
    done := make(chan bool)
    tick := make(chan time.Time)
    go func() {
        time.Sleep(1 * time.Second)
        stop <- true
    }()
    go func() {
        for t := range tick {
            fmt.Println("Current time: ", t)
        }
        done <- true
    }()
    for {
        select {
        case <-stop:
            ticker.Stop()
            close(tick)
        case t := <-ticker.C:
            tick <- t
            break
        case <-done:
            fmt.Println("Done!")
            return
        }
    }
}

But it seems to preform worse...

Current time:  2020-06-10 13:23:20.489040642 -0700 PDT m=+1.000425216
Done!
Current time:  2020-06-10 13:23:21.495263288 -0700 PDT m=+1.000338902
Done!
Current time:  2020-06-10 13:23:22.501474055 -0700 PDT m=+1.000327127
Done!
Current time:  2020-06-10 13:23:23.503531868 -0700 PDT m=+1.000244398
Done!
Current time:  2020-06-10 13:23:24.510210786 -0700 PDT m=+1.000420955
Done!
Current time:  2020-06-10 13:23:25.516500359 -0700 PDT m=+1.000460986
Done!
Done!
Current time:  2020-06-10 13:23:27.527077433 -0700 PDT m=+1.000375330
Done!
Current time:  2020-06-10 13:23:28.533401667 -0700 PDT m=+1.000470273
Done!
panic: send on closed channel

goroutine 1 [running]:
main.main()
    /home/dcow/Desktop/ticker-go/main2.go:29 +0x22f
Current time:  2020-06-10 13:23:30.547554719 -0700 PDT m=+1.000399602
Done!
Current time:  2020-06-10 13:23:31.55416725 -0700 PDT m=+1.000443683
Done!
Current time:  2020-06-10 13:23:32.56041176 -0700 PDT m=+1.000436364
Done!
Done!
Current time:  2020-06-10 13:23:34.572550584 -0700 PDT m=+1.000445593
Done!
Current time:  2020-06-10 13:23:35.578672712 -0700 PDT m=+1.000357330
Done!
Done!
Current time:  2020-06-10 13:23:37.590984117 -0700 PDT m=+1.000447504
Done!

We can't guarantee that we won't receive the stop message at the same time as we receive the final tick, so we've just shuffled the problem around to something that panics when it behaves "incorrectly" (which is marginally better than doing so silently). If we nil ed the tick channel, we'd devolve to the original case. And we still have cases where no tick is printed at all likely because it's possible we close the timer before it ever has a chance to fire..

How about a ready channel?

func main() {
    ticker := time.NewTicker(time.Second)
    tick := make(chan time.Time)
    ready := make(chan bool, 1)
    stop := make(chan bool)
    done := make(chan bool)
    go func() {
        time.Sleep(1 * time.Second)
        <-ready
        stop <- true
    }()
    go func() {
        for t := range tick {
            fmt.Println("Current time: ", t)
        }
        done <- true
    }()
    for {
        select {
        case <-stop:
            ticker.Stop()
            close(tick)
        case t := <-ticker.C:
            select {
            case ready<-true:
                break
            default:
            }
            tick <- t
            break
        case <-done:
            fmt.Println("Done!")
            return
        }
    }
}

This seems to work. It's somewhat involved with the addition of 3 new channels and an additional go routine, but it hasn't failed thus far. Is this pattern idiomatic in go? Are there general form strategies for applying this type of transformation in scenarios where you want to prioritize one of the select cases? Most advice I've come across is related to sequential and nested selects which don't really solve the problem.

Alternatively, is there a way to say "give me the list of ready channels so I can pick the order in which I process them"?

Edit:

Adding some clarifying remarks: I'm not interested in preserving ordering of concurrent operations. I agree that's a silly endeavor. I simply want to be able know if a selection of channels are ready to be processed and provide my own logic dictating what to do when multiple channels are ready concurrently. I'm essentially interested in a Go analog to POSIX select . And/or I'm interested in literature describing or common knowledge surrounding a generalized "convert non-deterministic select to deterministic select in Go" pattern.

eg Do people use the heap package and deposit data into a priority queue and ultimately read from that? Is there an x/reflect style package that implements a prioritized select using unsafe? Is there some simple pattern like, "Convert all selects with a single channel that should take priority into a dual channel style and forward the "done" request the the producer which in turn should terminate and close their channel then block on a channel range loop (kinda like my working solution)? Actually, lock on a shared condition variable for reasons x, y. etc..