[Twisted-web] Having reactor run at the main thread make things harder for late adopters

[glyph at divmod.com]

On 03:08 pm, amcnabb at mcnabbs.org wrote:
>On Wed, Nov 05, 2008 at 03:11:08PM +1100, Andrew Bennetts wrote:
>>
>>The documentation could perhaps be clearer about this (that document 
>>implies,
>>but doesn't directly state, that there's only one reactor instance).
>
>I've always struggled with reactor being a singleton.  It makes it very
>difficult to use Twisted on a small scale in a larger existing project.
>Sometimes it would be really nice to stop a reactor and later start
>another one.  Unit testing is one of several situations where this can
>be a problem.
>
>I suppose this wouldn't be quite as severe of a problem if it were easy
>to restart a stopped reactor, but even then, it would be nice to be 
>able
>to have two threads that each have their own reactor.  But I guess I'm
>just ranting now. :)

I completely agree, architecturally speaking, for Twisted, that this is 
a good idea.  I already linked to 
<http://twistedmatrix.com/trac/ticket/3205> in this thread, which is a 
broad outline of my long-term plan to eliminate the reactor's singleton- 
ness.

On the other hand, the approach you're describing here, using Twisted 
"just a little", starting up a reactor and spinning it in some obscure 
corner of an application, is basically an architectural antipattern, 
which should be avoided as much as possible.  One of the hardest things 
to do on a modern software project is to keep a coherent high-level 
system design that makes enough sense to introduce new people to it. 
Concurrency, in particular, is a property which is basically impossible 
to introduce into a software system without a comprehensive rethinking 
of how the whole thing works.

I can definitely think of a few "larger projects" - names withheld to 
protect the guilty - whose designs have been influenced and 
rehabilitated by their decision to use Twisted.  I worry about making it 
easier to throw Twisted into a situation where it will make the overall 
design of a system into even more of an incomprehensible mess.  Just 
imagine trying to decipher a GUI project using Twisted for networking, 
where there are 3 or 4 different global reactors, each of which is used 
by a different window or dialog box, and you have to figure out which 
one you need based on what context your Twisted-using code is running 
in.

It's nice to be able to say "you can't do that, it's impossible", when 
someone asks how to set up something like that.