Unlike regular generators, which simply exit as soon as the input sequence is exhausted, coroutines generally employ infinite while loops, which means that they won't be torn down the normal way. That's why coroutines also support both close and throw methods, which will exit the function. The important thing here is not the closing but the possibility of adding a teardown method. Essentially, it is very comparable to how context wrappers function with an __enter__ and __exit__ method, but with coroutines in this case:
@coroutine
def simple_coroutine():
print('Setting up the coroutine') try:
while True: item = yield
print('Got item: %r' % item) except GeneratorExit: print('Normal exit') except Exception as e: print('Exception exit: %r' % e) raise finally: print('Any exit')
print('Creating simple coroutine') active_coroutine = simple_coroutine() print()
print('Sending spam')
print('Close the coroutine') active_coroutine.close() print()
print('Creating simple coroutine') active_coroutine = simple_coroutine() print()
print('Sending eggs')
active_coroutine.send('eggs') print()
print('Throwing runtime error')
active_coroutine.throw(RuntimeError, 'Oops...') print()
This generates the following output, which should be as expected—no strange behavior but simply two methods of exiting a coroutine:
# python3 H06.py
Creating simple coroutine Setting up the coroutine Sending spam
Got item: 'spam' Close the coroutine Normal exit
Any exit
Creating simple coroutine Setting up the coroutine Sending eggs
Got item: 'eggs' Throwing runtime error
Exception exit: RuntimeError('Oops...',) Any exit
Traceback (most recent call last): ... File ... in <module> active_coroutine.throw(RuntimeError, 'Oops...') File ... in simple_coroutine item = yield RuntimeError: Oops...
Bidirectional pipelines
In the previous paragraphs, we saw pipelines; they process the output sequentially and one-way. However, there are cases where this is simply not enough—times where you need a pipe that not only sends values to the next pipe but also receives information back from the sub-pipe. Instead of always having a single list that is processed, we can maintain the state of the generator between executions this way. So, let's start by converting the earlier pipelines to coroutines. First, the lines.txt file again:
spam eggs spam spam eggs eggs spam spam spam eggs eggs eggs
Now, the coroutine pipeline. The functions are the same as before but using coroutines instead:
>>> @coroutine
... def replace(search, replace): ... while True:
... item = yield
... print(item.replace(search, replace))
>>> spam_replace = replace('spam', 'bacon') >>> for line in open('lines.txt'):
bacon bacon eggs eggs
bacon bacon bacon eggs eggs eggs
Given this example, you might be wondering why we are now printing the value instead of yielding it. Well! We can, but remember that generators freeze until a value is yielded. Let's see what would happen if we simply yield the value instead of calling print. By default, you might be tempted to do this:
>>> @coroutine
... def replace(search, replace): ... while True:
... item = yield
... yield item.replace(search, replace)
>>> spam_replace = replace('spam', 'bacon') >>> spam_replace.send('spam')
'bacon'
>>> spam_replace.send('spam spam') >>> spam_replace.send('spam spam spam') 'bacon bacon bacon'
Half of the values have disappeared now, so the question is, "Where did they go?" Notice that the second yield isn't storing the results. That's where the values are disappearing. We need to store those as well:
>>> @coroutine
... def replace(search, replace): ... item = yield
... while True:
... item = yield item.replace(search, replace)
>>> spam_replace = replace('spam', 'bacon') >>> spam_replace.send('spam')
'bacon'
>>> spam_replace.send('spam spam') 'bacon bacon'
>>> spam_replace.send('spam spam spam') 'bacon bacon bacon'
But even this is far from optimal. We are essentially using coroutines to mimic the behavior of generators right now. Although it works, it's just a tad silly and not all that clear. Let's make a real pipeline this time where the coroutines send the data to the next coroutine (or coroutines) and actually show the power of coroutines by sending the results to multiple coroutines:
# Grep sends all matching items to the target >>> @coroutine
... def grep(target, pattern): ... while True:
... item = yield
... if pattern in item: ... target.send(item)
# Replace does a search and replace on the items and sends it to # the target once it's done
>>> @coroutine
... def replace(target, search, replace): ... while True:
... target.send((yield).replace(search, replace)) # Print will print the items using the provided formatstring >>> @coroutine
... def print_(formatstring): ... while True:
... print(formatstring % (yield)) # Tee multiplexes the items to multiple targets >>> @coroutine
... item = yield
... for target in targets: ... target.send(item)
# Because we wrap the results we need to work backwards from the # inner layer to the outer layer.
# First, create a printer for the items: >>> printer = print_('%s')
# Create replacers that send the output to the printer >>> replacer_spam = replace(printer, 'spam', 'bacon')
>>> replacer_eggs = replace(printer, 'spam spam', 'sausage') # Create a tee to send the input to both the spam and the eggs # replacers
>>> branch = tee(replacer_spam, replacer_eggs) # Send all items containing spam to the tee command >>> grepper = grep(branch, 'spam')
# Send the data to the grepper for all the processing >>> for line in open('lines.txt'):
... grepper.send(line.rstrip()) bacon
spam
bacon bacon sausage
bacon bacon bacon sausage spam
This makes the code much simpler and more readable, but more importantly, it shows how a single source can be split into multiple destinations. While this might not look too exciting, it most certainly is. If you look closely, you will see that the tee method splits the input into two different outputs, but both of those outputs write back to the same print_ instance. This means that it's possible to route your data along whichever way is convenient for you while still having it end up at the same endpoint with no effort whatsoever.
Regardless, the example is still not that useful, as these functions still don't use all of the coroutine's power. The most important feature, a consistent state, is not really used in this case.
The most important lesson to learn from these lines is that mixing generators and coroutines is not a good idea in most cases since it can have very strange side effects if used incorrectly. Even though both use the yield statement, they are significantly different creatures with different behavior. The next paragraph will show one of the few cases where mixing coroutines and generators can be useful.