This continues the introduction started here. You can find an index to the entire series here.
Introduction
In the last Part we learned a new way of structuring sequential asynchronous callbacks using a generator. Thus, including deferreds, we now have two techniques for chaining asynchronous operations together.
Sometimes, though, we want to run a group of asynchronous operations in “parallel”. Since Twisted is single-threaded they won’t really run concurrently, but the point is we want to use asynchronous I/O to work on a group of tasks as fast as possible. Our poetry clients, for example, download poems from multiple servers at the same time, rather than one server after another. That was the whole point of using Twisted for getting poetry, after all.
This continues the introduction started here. You can find an index to the entire series here.
Introduction
In this Part we’re going to return to the subject of callbacks. We’ll introduce another technique for writing callbacks in Twisted that uses generators. We’ll show how the technique works and contrast it with using “pure” Deferreds. Finally we’ll rewrite one of our poetry clients using this technique. But first let’s review how generators work so we can see why they are a candidate for creating callbacks.
This continues the introduction started here. You can find an index to the entire series here.
Introduction
The servers we have written so far have just run in a terminal window, with output going to the screen via print statements. This works alright for development, but it’s hardly a way to deploy services in production. A well-behaved production server ought to:
Run as a daemon process, unconnected with any terminal or user session. You don’t want a service to shut down just because the administrator logs out.
Send debugging and error output to a set of rotated log files, or to the syslog service.
Drop excessive privileges, e.g., switching to a lower-privileged user before running.
Record its pid in a file so that the administrator can easily send signals to the daemon.
We can get all of those features using the twistd script provided by Twisted. But first we’ll have to change our code a bit.
This continues the introduction started here. You can find an index to the entire series here.
Introduction
We’ve written a lot of code in our exploration of Twisted, but so far we’ve neglected to write something important — tests. And you may be wondering how you can test asynchronous code using a synchronous framework like the unittest package that comes with Python. The short answer is you can’t. As we’ve discovered, synchronous and asynchronous code do not mix, at least not readily.
This continues the introduction started here. You can find an index to the entire series here.
Introduction
In this part we’re going to learn another aspect of the Deferred class. To motivate the discussion, we’ll add one more server to our stable of poetry-related services. Suppose we have a large number of internal clients who want to get poetry from the same external server. But this external server is slow and already over-burdened by the insatiable demand for poetry across the Internet. We don’t want to contribute to that poor server’s problems by sending all our clients there too.
This continues the introduction started here. You can find an index to the entire series here.
Introduction
Recall poetry client 5.1 from Part 10.The client used a Deferred to manage a callback chain that included a call to a poetry transformation engine. In client 5.1, the engine was implemented as a synchronous function call implemented in the client itself.
Now we want to make a new client that uses the networked poetry transformation service we wrote in Part 12. But here’s the wrinkle: since the transformation service is accessed over the network, we’ll need to use asynchronous I/O. And that means our API for requesting a transformation will have to be asynchronous, too. In other words, the try_to_cummingsify callback is going to return a Deferred in our new client.
This continues the introduction started here. You can find an index to the entire series here.
One More Server
Alright, we’ve written one Twisted server so let’s write another, and then we’ll get back to learning some more about Deferreds.
In Parts 9 and 10 we introduced the idea of a poetry transformation engine. The one we eventually implemented, the cummingsifier, was so simple we had to add random exceptions to simulate a failure. But if the transformation engine was located on another server, providing a network “poetry transformation service”, then there is a much more realistic failure mode: the transformation server is down.
This continues the introduction started here. You can find an index to the entire series here.
A Twisted Poetry Server
Now that we’ve learned so much about writing clients with Twisted, let’s turn around and re-implement our poetry server with Twisted too. And thanks to the generality of Twisted’s abstractions, it turns out we’ve already learned almost everything we need to know. Take a look at our Twisted poetry server located in twisted-server-1/fastpoetry.py. It’s called fastpoetry because this server sends the poetry as fast as possible, without any delays at all. Note there’s significantly less code than in the client!
