[Twisted-Python] Considering Twisted for OfflineIMAP
jgoerzen at complete.org
Sat Jul 26 09:48:15 EDT 2003
I am the author of OfflineIMAP, a multi-threaded multi-account
bi-directional synchronization tool
(http://quux.org/devel/offlineimap) written in Python.
I am running up against several limitations of the current way I'm
doing things. They include:
1. Lack of thread groups in Python, meaning that if there was a
network error synchronizing account A, I'd also have to kill off
the entire application and the synchronizing threads for account B
-- I couldn't just kill of account A.
2. Various bugs in Python's threading implementation
3. imaplib.py being a stinking festering heap of crap leading to
unmaintainable code in everything that touches it
My first plan was to just rewrite imaplib, which wouldn't be too
hard. But someone on the OfflineIMAP list pointed me to Twisted,
which I observe already contains an IMAP library that looks nice, plus
a different paradigm that may spare me from the whole thread groups
My questions, then, are these:
1. How does Twisted interact with things like threads and connection
I currently let the user control the maximum number of connections
that are open to a given server, and whenever the code needs to
access an IMAP server, it grabs a connection from the appropriate
pool and uses it (using threading primitives to block if no
connection is available.) Can Twisted handle multiple connections
to multiple servers doing multiple things at once? Can it do
connection pools? If it needs threading to do these things, does
it play nice with it?
3. I notice the "Deferred" object and the benefits it provides when a
server response is expected to take some time. I assume that some
sort of internal threading is taking place here?
What if the server *request* takes some time -- say, uploading a
2MB e-mail over a dialup link. Are we still OK with handling that
in a non-blocking fashion?
4. The howto "book" alludes to pending improvements on the mail
infrastructure. Anything I should be aware of here?
5. Is there any sort of unified mail API in Twisted (like JavaMail)
that would present me with a single API to both IMAP and Maildir
repositories, or is that something I need to do on my own? (I've
already done it, so it's no big deal to do that again)
6. OfflineIMAP supports several different user interfaces (two written
with Tkinter, 1 using Curses, and three plain console ones.) I would
like to reimplement the Tkinter ones with wxPython, and am glad to
see that Twisted works well with this.
One concern, though, is that the users can supply a list of UIs to try
in the config file: for instance, Tk, Curses, Noninteractive. The
system will try each one in turn until it reaches one that works.
(For instance, Tk may not work if the user is not in X, and Curses
may not work if stdout is not a tty). It looks like this may be
problematic with Twisted since the UI selection always seems to be
known in advance with the import commands related to the reactor.
Any thoughts here?
-- John Goerzen
More information about the Twisted-Python