An Introduction to Asynchronous Programming and Twisted

Part 15: Tested Poetry

This continues the introduction started here. You can find an index to the entire series here.

Introduction

We’ve written a lot of code in our exploration of Twisted, but so far we’ve neglected to write something important — tests. And you may be wondering how you can test asynchronous code using a synchronous framework like the unittest package that comes with Python. The short answer is you can’t. As we’ve discovered, synchronous and asynchronous code do not mix, at least not readily.

Fortunately, Twisted includes its own testing framework called trial that does support testing asynchronous code (and you can use it to test synchronous code, too).

We’ll assume you are already familiar with the basic mechanics of unittest and similar testing frameworks, in which you create tests by defining a class with a specific parent class (usually called something like TestCase), and each method of that class starting with the word “test” is considered a single test. The framework takes care of discovering all the tests, running them one after the other with optional setUp and tearDown steps, and then reporting the results.

The Example

You will find some example tests located in tests/test_poetry.py. To ensure all our examples are self-contained (so you don’t need to worry about PYTHONPATH settings), we have copied all the necessary code into the test module. Normally, of course, you would just import the modules you wanted to test.

The example is testing both the poetry client and server, by using the client to fetch a poem from a test server. To provide a poetry server for testing, we implement the setUp method in our test case:

class PoetryTestCase(TestCase):

    def setUp(self):
        factory = PoetryServerFactory(TEST_POEM)
        from twisted.internet import reactor
        self.port = reactor.listenTCP(0, factory, interface="127.0.0.1")
        self.portnum = self.port.getHost().port

The setUp method makes a poetry server with a test poem, and listens on a random, open port. We save the port number so the actual tests can use it, if they need to. And, of course, we clean up the test server in tearDown when the test is done:

    def tearDown(self):
        port, self.port = self.port, None
        return port.stopListening()

That brings us to our first test, test_client, where we use get_poetry to retrieve the poem from the test server and verify it’s the poem we expected:

    def test_client(self):
        """The correct poem is returned by get_poetry."""
        d = get_poetry('127.0.0.1', self.portnum)

        def got_poem(poem):
            self.assertEquals(poem, TEST_POEM)

        d.addCallback(got_poem)

        return d

Notice that our test function is returning a deferred. Under trial, each test method runs as a callback. That means the reactor is running and we can perform asynchronous operations as part of the test. We just need to let the framework know that our test is asynchronous and we do that in the usual Twisted way — return a deferred.

The trial framework will wait until the deferred fires before calling the tearDown method, and will fail the test if the deferred fails (i.e., if the last callback/errback pair fails). It will also fail the test if our deferred takes too long to fire, two minutes by default. And that means if the test finished, we know our deferred fired, and therefore our callback fired and ran the assertEquals test method.

Our second test, test_failure, verifies that get_poetry fails in the appropriate way if we can’t connect to the server:

    def test_failure(self):
        """The correct failure is returned by get_poetry when
        connecting to a port with no server."""
        d = get_poetry('127.0.0.1', 0)
        return self.assertFailure(d, ConnectionRefusedError)

Here we attempt to connect to an invalid port and then use the trial-provided assertFailure method. This method is like the familiar assertRaises method but for asynchronous code. It returns a deferred that succeeds if the given deferred fails with the given exception, and fails otherwise.

You can run the tests yourself using the trial script like this:

trial tests/test_poetry.py

And you should see some output showing each test case and an OK telling you each test passed.

Discussion

Because trial is so similar to unittest when it comes to the basic API, it’s pretty easy to get started writing tests. Just return a deferred if your test uses asynchronous code, and trial will take care of the rest. You can also return a deferred from the setUp and tearDown methods, if those need to be asynchronous as well.

Any log messages from your tests will be collected in a file inside a directory called _trial_temp that trial will create automatically if it doesn’t exist. In addition to the errors printed to the screen, the log is a useful starting point when debugging failing tests.

Figure 33 shows a hypothetical test run in progress:

Figure 33: a trial test in progress
Figure 33: a trial test in progress

If you’ve used similar frameworks before, this should be a familiar model, except that all the test-related methods may return deferreds.

The trial framework is also a good illustration of how “going asynchronous” involves changes that cascade throughout the program. In order for a test (or any function or method) to be asynchronous, it must:

  1. Not block and, usually,
  2. return a deferred.

But that means that whatever calls that function must be willing to accept a deferred, and also not block (and thus likely return a deferred as well). And so it goes up and up. Thus, the need for a framework like trial which can handle asynchronous tests that return deferreds.

Summary

That’s it for our look at unit testing. If would like to see more examples of how to write unit tests for Twisted code, you need look no further than Twisted itself. The Twisted framework comes with a very large suite of unit tests, with new ones added in each release. Since these tests are scrutinized by Twisted experts during code reviews before being accepted into the codebase, they make excellent examples of how to test Twisted code the right way.

In Part 16 we will use a Twisted utility to turn our poetry server into a genuine daemon.

Suggested Exercises

  1. Change one of the tests to make it fail and run trial again to see the output.
  2. Read the online trial documentation.
  3. Write tests for some of the other poetry services we have created in this series.
  4. Explore some of the tests in Twisted.

12 thoughts on “An Introduction to Asynchronous Programming and Twisted”

  1. Hello Dave,

    First of all, thank you so much for this amazing tutorial and for taking care of so many tiny but so important details. (and the challenging exercices ). It is such a huge work.

    I have a little issue on running the test ( on Windows 7) : the second test fails whatever I tried after 30sec by generating an OverFlowError (because port was not in the right range) :
    [FAIL]
    Traceback (most recent call last):
    File “C:\Python27\lib\site-packages\twisted\trial\unittest.py”, line 402, in _eb
    raise self.failureException(output)
    twisted.trial.unittest.FailTest:
    Expected: (,)
    Got:
    [Failure instance: Traceback (failure with no frames): : User timeout caused connection failure.
    ]
    I tried to change it by TimeoutError, but I still got the same error:
    [ERROR]
    Traceback (most recent call last):
    File “C:\Python27\lib\site-packages\twisted\internet\base.py”, line 793, in runUntilCurrent
    call.func(*call.args, **call.kw)
    File “C:\Python27\lib\site-packages\twisted\internet\tcp.py”, line 617, in resolveAddress
    self._setRealAddress(self.addr[0])
    File “C:\Python27\lib\site-packages\twisted\internet\tcp.py”, line 624, in _setRealAddress
    self.doConnect()
    File “C:\Python27\lib\site-packages\twisted\internet\tcp.py”, line 650, in doConnect
    connectResult = self.socket.connect_ex(self.realAddress)
    File “C:\Python27\lib\socket.py”, line 224, in meth
    return getattr(self._sock,name)(*args) exceptions.OverflowError: getsockaddrarg: port must be 0-65535.

    test_poetry.PoetryTestCase.test_failure

    I could not figure out what I did wrong…

    Helene

    1. Hi Helene, glad you like the series! I’m not sure you did anything wrong, I think perhaps that
      a different exception is raised on Windows than on Linux, and so the test_failure test is actually
      failing. I’ll check it out and see about fixing it up. I’ll probably need your help testing it out as I
      don’t have Windows at home.

      thanks,
      dave

    2. Hi Helene, turns out this wasn’t windows related at all. It may actually be a change
      in Python’s behavior. At any rate, I changed the test port to ‘0’, how does that work
      for you now?

  2. I don’t know if this matters, but I had to change the test_failure line to this. It wasn’t returning a ConnectionRefusedError

    return self.assertFailure(d, ConnectError)

  3. I am using Windows 7 64bit. I am running everything thru PyCharm IDE. I have a linux box I haven’t tried it on though. I can try it on linux tomorrow and report back just to see.

  4. Confirmed on Ubuntu; there seems to be a different response on Windows 7 64bit. Here is the response with a fresh clone of your git repo:

    twisted-intro/tests$ trial test_poetry.py
    test_poetry
    PoetryTestCase
    test_client … [OK]
    test_failure … [OK]

    ——————————————————————————-
    Ran 2 tests in 0.009s

    PASSED (successes=2)

Leave a Reply