Tested Poetry

Part 15: Tested Poetry

This continues the introduction started here. You can find an index to the entire series here.

Introduction

We’ve written a lot of code in our exploration of Twisted, but so far we’ve neglected to write something important — tests. And you may be wondering how you can test asynchronous code using a synchronous framework like the unittest package that comes with Python. The short answer is you can’t. As we’ve discovered, synchronous and asynchronous code do not mix, at least not readily.

Fortunately, Twisted includes its own testing framework called trial that does support testing asynchronous code (and you can use it to test synchronous code, too).

We’ll assume you are already familiar with the basic mechanics of unittest and similar testing frameworks, in which you create tests by defining a class with a specific parent class (usually called something like TestCase), and each method of that class starting with the word “test” is considered a single test. The framework takes care of discovering all the tests, running them one after the other with optional setUp and tearDown steps, and then reporting the results.

The Example

You will find some example tests located in tests/test_poetry.py. To ensure all our examples are self-contained (so you don’t need to worry about PYTHONPATH settings), we have copied all the necessary code into the test module. Normally, of course, you would just import the modules you wanted to test.

The example is testing both the poetry client and server, by using the client to fetch a poem from a test server. To provide a poetry server for testing, we implement the setUp method in our test case:

class PoetryTestCase(TestCase):

    def setUp(self):
        factory = PoetryServerFactory(TEST_POEM)
        from twisted.internet import reactor
        self.port = reactor.listenTCP(0, factory, interface="127.0.0.1")
        self.portnum = self.port.getHost().port

The setUp method makes a poetry server with a test poem, and listens on a random, open port. We save the port number so the actual tests can use it, if they need to. And, of course, we clean up the test server in tearDown when the test is done:

    def tearDown(self):
        port, self.port = self.port, None
        return port.stopListening()

That brings us to our first test, test_client, where we use get_poetry to retrieve the poem from the test server and verify it’s the poem we expected:

    def test_client(self):
        """The correct poem is returned by get_poetry."""
        d = get_poetry('127.0.0.1', self.portnum)

        def got_poem(poem):
            self.assertEquals(poem, TEST_POEM)

        d.addCallback(got_poem)

        return d

Notice that our test function is returning a deferred. Under trial, each test method runs as a callback. That means the reactor is running and we can perform asynchronous operations as part of the test. We just need to let the framework know that our test is asynchronous and we do that in the usual Twisted way — return a deferred.

The trial framework will wait until the deferred fires before calling the tearDown method, and will fail the test if the deferred fails (i.e., if the last callback/errback pair fails). It will also fail the test if our deferred takes too long to fire, two minutes by default. And that means if the test finished, we know our deferred fired, and therefore our callback fired and ran the assertEquals test method.

Our second test, test_failure, verifies that get_poetry fails in the appropriate way if we can’t connect to the server:

    def test_failure(self):
        """The correct failure is returned by get_poetry when
        connecting to a port with no server."""
        d = get_poetry('127.0.0.1', 0)
        return self.assertFailure(d, ConnectionRefusedError)

Here we attempt to connect to an invalid port and then use the trial-provided assertFailure method. This method is like the familiar assertRaises method but for asynchronous code. It returns a deferred that succeeds if the given deferred fails with the given exception, and fails otherwise.

You can run the tests yourself using the trial script like this:

trial tests/test_poetry.py

And you should see some output showing each test case and an OK telling you each test passed.

Discussion

Because trial is so similar to unittest when it comes to the basic API, it’s pretty easy to get started writing tests. Just return a deferred if your test uses asynchronous code, and trial will take care of the rest. You can also return a deferred from the setUp and tearDown methods, if those need to be asynchronous as well.

Any log messages from your tests will be collected in a file inside a directory called _trial_temp that trial will create automatically if it doesn’t exist. In addition to the errors printed to the screen, the log is a useful starting point when debugging failing tests.

Figure 33 shows a hypothetical test run in progress:

If you’ve used similar frameworks before, this should be a familiar model, except that all the test-related methods may return deferreds.

The trial framework is also a good illustration of how “going asynchronous” involves changes that cascade throughout the program. In order for a test (or any function or method) to be asynchronous, it must:

1. Not block and, usually,
2. return a deferred.

But that means that whatever calls that function must be willing to accept a deferred, and also not block (and thus likely return a deferred as well). And so it goes up and up. Thus, the need for a framework like trial which can handle asynchronous tests that return deferreds.

Summary

That’s it for our look at unit testing. If would like to see more examples of how to write unit tests for Twisted code, you need look no further than Twisted itself. The Twisted framework comes with a very large suite of unit tests, with new ones added in each release. Since these tests are scrutinized by Twisted experts during code reviews before being accepted into the codebase, they make excellent examples of how to test Twisted code the right way.

In Part 16 we will use a Twisted utility to turn our poetry server into a genuine daemon.

Suggested Exercises

1. Change one of the tests to make it fail and run trial again to see the output.
2. Read the online trial documentation.
3. Write tests for some of the other poetry services we have created in this series.
4. Explore some of the tests in Twisted.