[Twisted-Python] What's everyone using Twisted for?

Matt Campbell mattcampbell at pobox.com
Wed Dec 12 01:37:15 EST 2001


I'm using Twisted in a custom server which I've written for ACB
Radio Interactive (http://interactive.acbradio.org/), or ACBRI for
short.  ACBRI is a non-commercial "Internet radio station" run by
blind and visually-impaired people (myself included), playing music
in a variety of styles.  We use the Shoutcast MP3-streaming system
(www.shoutcast.com), in which the broadcaster sends the encoded
audio to the server (over a TCP connection), and the server sends
that audio to all currently connected clients.

We wanted to provide a high-quality audio stream for users with
broadband Internet connections, while not preventing modem users
from listening.  One way to solve this problem is for the broadcaster
to simultaneously send two separate streams to the server (or
potentially two different servers), one encoded at a high bit rate
and one at a low bit rate.  But not everyone can do that, due to
lack of adequate CPU time or bandwidth or the limited functionality
of the Windows applications most of the broadcasters use.

So my idea was that the broadcaster should send only one stream to
the server, which should re-encode that stream (using an external
MP3 encoder like LAME) into a low-bandwidth stream for modem users
and make that one available on a different port.  Our server also
needed to be able to relay one or both of the streams to another
Shoutcast server in case we didn't have enough bandwidth on our
own machine to serve many listeners.  I knew of no server that did
these two things, so I decided to write my own.

I had written a Shoutcast-compatible server in Python a few months
earlier, but it used threads to handle multiple connections (using
the classes in the SocketServer module).  After reading Sam Rushing's
description of Medusa and his tutorial on the asyncore and asynchat
modules, I knew that I should drop multithreading in favor of a
single-threaded asynchronous approach.  So I figured I'd rewrite
my server sometime but never got around to it, and one thing that
hindered me was the lack of built-in support for delayed callbacks
in asyncore (for timeouts which are necessary in some cases).

It seemed that it would be too difficult to implement the new
requirements in my original multi-threaded server (which was probably
poorly designed in other ways as well), so I decided to throw it
out and start over.  At that point I took a look at Twisted and
chose to use it.  I think the two main features that made me decide
to use Twisted instead of asyncore+asynchat were built-in support
for delayed callbacks and the Process class (since I needed to
start the MP3 encoder and communicate with it through pipes).
Initializing a server and handling a new incoming connection are
also easier under the twisted.internet framework, and I thought
the telnet server was cool.

This leads me to what I don't like about Twisted.  After taking a
closer look at Medusa, I found that Medusa's "monitor" is better
if you don't use a GUI (and therefore can't use Twisted Manhole).
I guess I ought to write a command-line Manhole client using the
twisted.internet.stdio module.  I'd also strongly recommend moving
the higher-level components into separate packages.  For example.
I have no use for Twisted Reality or Words and don't currently use
Twisted Spread, though Twisted Web may come in handy in the future
(I currently only use the low-level twisted.protocols.http module).

I started working on this server in August.  ACBRI has now been
using it for a little over two months.  It is not released to the
public, at least not yet; if it's released, it will be under the
GPL.

Sorry if this is too long; feel free to condense and paraphrase
what I've said as much as you like.  I hope you found it interesting.
-- 
Matt Campbell
E-mail and MSN Messenger:  mattcampbell at pobox.com
Web site:  http://www.pobox.com/~mattcampbell/





More information about the Twisted-Python mailing list