Part 10: Poetry Transformed
Now we’re going to add some transformation logic to our poetry client, along the lines suggested in Part 9. But first, I have a shameful and humbling confession to make: I don’t know how to write the Byronification Engine. It is beyond my programming abilities. So instead, I’m going to implement a simpler transformation, the Cummingsifier. The Cummingsifier is an algorithm that takes a poem and returns a new poem like the original but written in the style of e.e. cummings. Here is the Cummingsifier algorithm in its entirety:
def cummingsify(poem) return poem.lower()
Unfortunately, this algorithm is so simple it never actually fails, so in client 5.0, located in twisted-client-5/get-poetry.py, we use a modified version of
cummingsify that randomly does one of the following:
- Return a cummingsified version of the poem.
- Raise a GibberishError.
- Raise a ValueError.
In this way we simulate a more complicated algorithm that sometimes fails in unexpected ways.
The only other changes in client 5.0 are in the
def poetry_main(): addresses = parse_args() from twisted.internet import reactor poems =  errors =  def try_to_cummingsify(poem): try: return cummingsify(poem) except GibberishError: raise except: print 'Cummingsify failed!' return poem def got_poem(poem): print poem poems.append(poem) def poem_failed(err): print >>sys.stderr, 'The poem download failed.' errors.append(err) def poem_done(_): if len(poems) + len(errors) == len(addresses): reactor.stop() for address in addresses: host, port = address d = get_poetry(host, port) d.addCallback(try_to_cummingsify) d.addCallbacks(got_poem, poem_failed) d.addBoth(poem_done) reactor.run()
So when the program downloads a poem from the server, it will either:
- Print the cummingsified (lower-cased) version of the poem.
- Print “Cummingsify failed!” followed by the original poem.
- Print “The poem download failed.”
Although we have retained the ability to download from multiple servers, when you are testing out client 5.0 it’s easier to just use a single server and run the program multiple times, until you see all three different outcomes. Also try running the client on a port with no server.
Let’s draw the callback/errback chain we create on each
Deferred we get back from
Note the pass-through errback that gets added by
addCallback. It passes whatever
Failure it receives onto the next errback (
poem_failed can handle failures from both
get_poetry (i.e., the deferred is fired with the
errback method) and the
Also note the hauntingly beautiful drop-shadow around the border of the deferred in Figure 19. It doesn’t signify anything other than me discovering how to do it in Inkscape. Expect more drop-shadows in the future.
Let’s analyze the different ways our deferred can fire. The case where we get a poem and the
cummingsify function works correctly is shown in Figure 20:
In this case no callback fails, so control flows down the callback line. Note that
None as its result, since
got_poem doesn’t actually
return a value. If we wanted subsequent callbacks to have access to the poem, we would modify
got_poem to return the poem explicitly.
Figure 21 shows the case where we get a poem, but
cummingsify raises a GibberishError:
try_to_cummingsify callback re-raises a
GibberishError, control switches to the errback line and
poem_failed is called with the exception as its argument (wrapped in a
Failure, of course).
poem_failed doesn’t raise an exception, or return a
Failure, after it is done control switches back to the callback line. If we want
poem_failed to handle the error completely, then returning
None is a reasonable behavior. On the other hand, if we wanted
poem_failed to take some action, but still propagate the error, we could change
poem_failed to return its
err argument and processing would continue down the errback line.
Note that in the current code neither
poem_failed ever fail themselves, so the
poem_done errback will never be called. But it’s safe to add it in any case and doing so represents an instance of “defensive” programming, as either
poem_failed might have bugs we don’t know about. Since the
addBoth method ensures that a particular function will run no matter how the deferred fires, using
addBoth is analogous to adding a
finally clause to a
Now examine the case where we download a poem and the
cummingsify function raises a
ValueError, displayed in Figure 22:
This is the same as figure 20, except
got_poem receives the original version of the poem instead of the transformed version. The switch happens entirely inside the
try_to_cummingsify callback, which traps the
ValueError with an ordinary
except statement and returns the original poem instead. The deferred object never sees that error at all.
Lastly, we show the case where we try to download a poem from a non-existent server in Figure 23:
None so afterwards control switches to the callback line.
In client 5.0 we are trapping exceptions from
cummingsify in our
try_to_cummingsify callback using an ordinary
except statement, rather than letting the deferred catch them first. There isn’t necessarily anything wrong with this strategy, but it’s instructive to see how we might do this differently.
Let’s suppose we wanted to let the deferred catch both
ValueError exceptions and send them to the errback line. To preserve the current behavior our subsequent errback needs to check to see if the error is a
ValueError and, if so, handle it by returning the original poem, so that control goes back to the callback line and the original poem gets printed out.
But there’s a problem: the errback wouldn’t get the original poem, it would get the
ValueError raised by the
cummingsify function. To let the errback handle the error, we need to arrange for it to receive the original poem.
One way to do that is to modify the
cummingsify function so the original poem is included in the exception. That’s what we’ve done in client 5.1, located in twisted-client-5/get-poetry-1.py. We changed the
ValueError exception into a custom
CannotCummingsify exception which takes the original poem as the first argument.
cummingsify were a real function in an external module, then it would probably be best to wrap it with another function that trapped any exception that wasn’t
GibberishError and raise a
CannotCummingsify exception instead. With this new setup, our
poetry_main function looks like this:
def poetry_main(): addresses = parse_args() from twisted.internet import reactor poems =  errors =  def cummingsify_failed(err): if err.check(CannotCummingsify): print 'Cummingsify failed!' return err.value.args return err def got_poem(poem): print poem poems.append(poem) def poem_failed(err): print >>sys.stderr, 'The poem download failed.' errors.append(err) def poem_done(_): if len(poems) + len(errors) == len(addresses): reactor.stop() for address in addresses: host, port = address d = get_poetry(host, port) d.addCallback(cummingsify) d.addErrback(cummingsify_failed) d.addCallbacks(got_poem, poem_failed) d.addBoth(poem_done)
def cummingsify_failed(err): if err.check(CannotCummingsify): print 'Cummingsify failed!' return err.value.args return err
We are using the
check method on
Failure objects to test whether the exception embedded in the
Failure is an instance of
CannotCummingsify. If so, we return the first argument to the exception (the original poem) and thus handle the error. Since the return value is not a
Failure, control returns to the callback line. Otherwise, we return the
Failure itself and send (re-raise) the error down the errback line. As you can see, the exception is available as the
value attribute on the
Figure 25 shows what happens when we get a
So when we are using a deferred, we can sometimes choose whether we want to use
except statements to handle exceptions, or let the deferred re-route errors to an errback.
In Part 10 we updated our poetry client to make use of the
Deferred‘s ability to route errors and results down the chain. Although the example was rather artificial, it did illustrate how control flow in a deferred switches back and forth between the callback and errback line depending on the result of each stage.
So now we know everything there is to know about deferreds, right? Not yet! We’re going to explore some more features of deferreds in a future Part. But first we’ll take a little detour and, in Part 11, implement a Twisted version of our poetry server.
- Figure 25 shows one of the four possible ways the deferreds in client 5.1 can fire. Draw the other three.
- Use the deferred simulator to simulate all possible firings for clients 5.0 and 5.1. To get you started, this simulator program can represent the case where the
try_to_cummingsifyfunction succeeds in client 5.0:
r poem p r None r None r None r None