Coroutines in Python: Best Practices

Question

I am wondering what the best practices are for writing coroutines in Python 3. I am developing basic methods which should accept some input (using the .send() method), perform computations on this input, and then yield output.

The first approach I found is to essentially do the following:

def coroutine(func):
  data = yield
  while 1:
    data = yield func(data)

That seems to work, but the line in the loop is bending my mind. It appears to first yield a function, and then take input and perform the assignment after resuming. This is completely non-intuitive to me.

The other approach I'm looking at is:

def coroutine():
  while 1:
    data = yield
    [ do stuff with data here ... ]
    yield result

This code is much easier for me to understand, and it also lets me put code right into the generator instead of passing in a function. But it's annoying to use. Every actual call to the generator (like "gen.send(2)") has to be followed by a "gen.send(None)" to advance the generator to the next yield.

It seems to me like the problem here stems from the "yield" keyword being used for two different things: a return statement, and an input statement.

If possible I want an approach that lets me take input, do calculations on that input, and then yield output, without having to pass in functions and use one-liners as in the first approach, or having to send extraneous values as in the second approach. How can I do this?

---

Please note: In reality, I will be sending in multiple values. So the problems of having extraneous "g.send(None)" statements get worse.

Answer 1

You can do it as you did in your first example. You just have to "do stuff with data" inside the loop. Here is an example:

def coroutine():
  data = yield
  while True:
    print("I am doing stuff with data now")
    data = data * 2
    data = yield data

You can use it like this:

>>> co = coroutine()
>>> next(co)
>>> co.send(1)
I am doing stuff with data now
2
>>> co.send(88)
I am doing stuff with data now
176

You are correct that yield plays a dual role, both yielding a result out and accepting the value subsequently passed in via send . (Likewise, send plays a dual and complementary role, in that every send call returns the value that the generator yields.) Note the order there: when you have a yield expression, it first yields the value out, and then the value of the yield expression becomes whatever is sent in afterwards .

This may seem "backwards", but you can get it to be "forwards" by doing it in a loop, as you essentially already did. The idea is that you first yield some initial value (maybe a meaningless one). This is necessary because you can't use send before a value has been yielded (since there would be no yield expression to evaluate to the sent value). Then, every time you use yield , you are giving out the "current" value, while simultaneously accepting input to be used in computing the "next" value.

As I mentioned in a comment, it is not clear from your example why you're using generators at all. In many cases, you can achieve a similar effect just by writing a class that has its own methods for passing things in and getting things out, and if you write the class, you can make the API whatever you want. If you choose to use generators, you have to accept the dual input/output roles of send and yield . If you don't like that, don't use generators (or, if you need the suspended function-state they provide, you can use them but wrap them with a class that separates the sending from the yielding).

Answer 2

To add an important clarification to BrenBarn's answer: the sentence "when you have a yield expression, it first yields the value out, and then the value of the yield expression becomes whatever is sent in afterwards." isn't completely accurate and only happens in the example he gave because the same yield is used in a loop. What actually occurs is the yield assignment is made first (at the yield where the program had paused) and then execution continues to the next yield which returns its result.

When you use the send() method, it will make the assignment at the yield where execution was paused (but not return a result from THAT yield) and then continue up to the next yield at which point a value will be returned and execution will pause. This is demonstrated in the following graphic and example code. Below is a design pattern used for the modeling and verification of synchronous hardware systems, creating design components that can take up to M inputs and provide N outputs on every iteration and demonstrates the operation I describe well:

This code using Python 3.8 demonstrates/confirms the operation described above:

def GenFunc():
    x = 'a'
    in1 = yield x
    y = 'b'
    print(f"After first yield: {in1=}, {y=}")
    in2 = yield y
    z = 'c'
    print(f"After second yield: {in1=}, {in2=}")
    in3 = yield z
    print(f"After third yield: {in1=}, {in2=}, {in3=}")

Which executes as follows:

>>> mygen = GenFunc()
>>> next(mygen)
Out: 'a'
>>> mygen.send(25)
After first yield: in1=25, y='b'
Out: 'b'
>>> mygen.send(15)
After second yield: in1=25, in2=15
Out: 'c'
>>> mygen.send(45)
After third yield: in1=25, in2=15, in3=45
-----------------------------
StopInteration Error

And here is an additional example showing the same behavior with a single yield in a loop:

def GenFunc(n):
    x = 0
    while True:
    n += 1
    x = yield n,x 
    x += 1
    print(n,x)
    x += 1

Which executes as follows:

>>> mygen = GenFunc(10)
>>> next(mygen)
Out: (11, 0)
>>> mygen.send(5)
11 6
Out: (12, 7)