Categories
Blather Programming Python Software

The End

Part 22: The End

This concludes the introduction started here. You can find an index to the entire series here.

All Done

Whew! Thank you for sticking with me. When I started this series I didn’t realize it was going to be this long, or take this much time to make. But I have enjoyed creating it and hope you have enjoyed reading it.

Now that I have finished, I will look into the possibility of generating a PDF format. No promises, though.

I would like to conclude with a few suggestions on how to continue your Twisted education.

Further Reading

First, I would recommend reading the Twisted online documentation. Although it is much-maligned, I think it’s better than it is often given credit for.

If you want to use Twisted for web programming, then Jean-Paul Calderone has a well-regarded series called “Twisted Web in 60 Seconds“. I suspect it will take a little longer than that to read, though.

There is also a Twisted Book, which I can’t say much about as I haven’t read it.

But more important than any of those, I think, is to read the Twisted source code. Since that code is written by people who know Twisted very well, it is an excellent source of examples for how do to things the “Twisted Way”.

Suggested Exercises

  1. Port a synchronous program you wrote to Twisted.
  2. Write a new Twisted program from scratch.
  3. Pick a bug from the Twisted bug database and fix it. Submit a patch to the Twisted developers. Don’t forget to read about the process for making your contribution.

The End, Really

Happy Hacking!

Figure 47: The End
Figure 47: The End

44 replies on “The End”

Dave,

This was an incredibly well-written and well-thought-out tutorial. This morning I decided that Twisted was the right tool for my job and I went about learning it and I came upon this series. 8 hours later, I’ve gone through each and every article and worked through all the examples; now I fully understand the different parts within Twisted, how they fit together, and the thought processes that went into designing them. It is truly rare that any library has documentation this clear and helpful, and you should be proud of your contribution.

I also feel lucky that out of all the times that I could have stumbled upon this, I happened to do so within 2 weeks of you finishing this epic series. Guess things just work out nicely sometimes 🙂

Thank you,
Jonathan

Dave,

After thinking about Twisted for a while, it seems to me like it’s very similar to other event-engine frameworks, especially those dealing with GUI’s. For example, a common pattern in Javascript is to subscribe to an onMouseOver() event, where you specify code that gets control when an event fires. I’m very familiar with this model, and if this were all Twisted did I would be happy, but not particularly impressed.

The real magic of Twisted, as opposed to an ordinary event framework, is that deferreds are very lightweight and easy to create/listen to. Especially when combined with the @inlineCallbacks decorator, this is the best effort I’ve seen to simplify asynchronous programming. I’ve always hated the proliferation of callbacks that you get with event engines, and having lots of anonymous methods was always a pain (also annoyingly impossible in python).

Anyway, thanks again for introducing me to the magic of Twisted.

–Jonathan

This is the best tutorial ever! You are like the teacher whom you remember for all your life because he taught you more than you were supposed to know. Taking Erlang, Haskell and poetry (I didn’t know of Cummings before) to Twisted and Async programming in general counts to 5 topics instead of 1. Brilliant!
Thank you very much for your hard work!

Great Twisted tutorial! Thank you so much for writing it. And thank you for explaining deferreds and deferred lists, those were pretty confusing. The connection you drew through Erlang and Haskell between asynchronous programming and lazy evaluation was new to me, I know some Haskell but that had not occurred to me.

That crack you made about the reactor loop picture back in chapter 8 just about killed me.

Thanks for this very well written and informative tutorial, Dave. After reading through the Twisted docs originally, my head was spinning and I didn’t really grasp the concepts behind writing Twisted applications. Your tutorial helped immensely. You explained so many important details that the Twisted docs didn’t even bother covering that seem to be crucial in understanding how to use the framework properly. Well done.

Thank you! I don’t have any plans at the moment, but the urge
to write it seized me at random, so who knows 🙂 Did you have
any specific topic in mind?

Well, I’m currently struggling to get started with a correct directory layout for my little project. You make some random remarks during the tutorial about separating things out and I have trouble seeing how that would actually work.

For example, I’m writing a protocol (or trying to, at least) that allows a client to request a range of datapoints from a server. What would be a good practise to actually keep code separated? Would I write just one Protocol class and make it work for both the client and the server? And would I differentiate between the two via the Factory? Or would it be better to write separate Protocols for client and server?

Also, maybe not really in the Twisted scope per se, but I’d be interested in working with authenticated connections, via x501 or something else? Would that be easy to do with Twisted?

Lastly, would the async mindset work for distributing some kind of work load (think genetical algorithmes or MapReduce or something) among several machines? Or would that just complicate things?

Lots of questions, but I’m a real newbie with Twisted and async programming (well, not entirely, since I actually read your entire tutorial!). Any more insights and explained code examples would be welcome, really. The Twisted code itself is pretty advanced, really, not really good examples to actually try to learn from, imho.

Thanks for the tutorial, that’s the only reason why I dived into this 🙂

I think it makes sense to have just one Protocol class most of the time,
since conceptually there is just one protocol that both ends are speaking.
And the server would use a Factory whereas clients can generally get away
without having one.

The key things to keep separate are the business logic that your service is
providing via the protocol and the protocol itself. So write a ‘DatapointService’
with a pure python API that python code running in the same process could use
normally, without making a network connection at all.

Then expose that service via one or more protocols, with code that lives outside
the datapoint service class.

For example, you could implement a REST endpoint using Twisted web that returns
that datapoints as JSON. Consider the ‘Web Server’ example on the front page of
the twistedmatrix.com site (second tab). Replace the render_GET implementation
with code that calls your datapoint service (with a normal python function call),
sets the content type to ‘application/json’, and returns the datapoints in a json
string.

Make sense?

Doing workload distribution is definitely a use-case for async, though if you just
want a solution, you might consider using something like Hadoop.

Hi Dave,
Your series rocks. I read it from the beginning to the end, and I learned a lot.

I really appreciate your work.
real.

I was new to Twisted; your 22 part blog is a valuable reference. Thanks!
Liked your rendition of the john hurt tune. Is that a steel-stringed acoustic guitar you’re playing?

Thanks for a great introduction to twisted. It helped me to port a synchronous program to twisted with relative ease. It has also inspired me to actually read the copy of real world Haskell sitting on my desk!

I’m coming back to Twisted after close to 10 years, and Your series is great as a refresher.
And You can write, You really can write 😉

Dave, this is an amazing work – this was the best introduction to asynchronous programming I could ever wish for. The path that you took us with you from the low level implementation to high level astractions help a lot to grasp the whole concept really well. Again, huge thanks for making this and making available to all.

So many years later this is still one of the best ways to learn twisted. If you still check this I just wanted to say thank you for your hard working making this. I found it incredibly helpful!

Hey Dave, this was an excellent coverage of Twisted. Thanks for explaining the concepts so well. I had a request for you, can you suggest any way, how to learn asyncio and converting an asynchronous program written in Twisted to Asyncio?

Sorry to bother you again, but I wanted to ask that how would one go about writing the error handling code, if one does not use Twisted’s inbuilt error callback mechanism i.e. errback ?

Leave a Reply to kornerrCancel reply

%d bloggers like this: