[Twisted-web] Having reactor run at the main thread make things harder for late adopters

Andy Fundinger Andy at newworldelectric.com
Thu Nov 6 14:41:27 EST 2008


I just read through this whole thread and don't understand why no one has
raised suggested running your "little bit of twisted" in another process.
That gives good separation, no multiple thread or singleton issues and
should be quite controllable and it breaks any GIL issues that might come up
as a freebie too.  Of course as you get past dipping your toes in it'll be
rather difficult, but so will anything else.

-Andy Fundinger

On Thu, Nov 6, 2008 at 1:46 PM, Andrew McNabb <amcnabb at mcnabbs.org> wrote:

> On Thu, Nov 06, 2008 at 02:55:37AM -0000, glyph at divmod.com wrote:
> >
> > This is almost certainly true, and as I've already said I am keenly
> > interested in de-globalizing the reactor myself; making it easier to
> > test things is a key reason for doing so.  However, we should be aware
> > of the risks and try to mitigate them.  It could well be that we could
> > come up with an API which is smooth as glass for the "good" use-cases we
> > have in mind here, and raises exceptions or emits helpful warnings when
> > used for the "bad" ones.  Maybe that's not possible, maybe we can only
> > get halfway there: but it would be nice to try.
>
> I think we really agree for the most part.
>
> >> - k threads that each have their own reactor
> >
> > This is a whole other, almost unrelated can of worms.  You can only use
> > Twisted from one thread at a time.  There are things which make
> > assumptions about non-reentrancy and mutual exclusion of global module-
> > level state.  You can find and fix every instance of this if you want,
> > but don't bundle it in with multiple reactors :).
>
> Now that you mention that, I have noticed that a few times when reading
> Twisted code.  I think that long-term, it's nice to remove global
> module-level state wherever possible, but in the meantime, just putting
> the above paragraph in the docs as a warning is probably sufficient.
>
>
> >> If I wanted to be constantly told "you can't do that, it's impossible",
> >> I wouldn't be a Python guy.  If I want to shoot myself in the foot,
> >> that's my problem.  :)
> >
> > Another bogus argument.  There are lots of things that are "impossible"
> > in Python.  For example, you "can't" change the values of a tuple, or
> > the contents of a string.  I mean, there's terrible stuff you can do
> > with ctypes, but if it breaks, you get to keep both pieces.  Similarly,
> > there are lots of things that already kind of work, by accident, if you
> > instantiate multiple reactors; you can kind of re-set a reactor if you
> > know exactly what to twiddle.
>
> Let me rephrase that, then.  I think that in Python, the goal is usually
> to create a flexible API and to document how it should be properly used
> rather than creating barriers to stop people from doing something just
> because it might be a bad idea in some contexts.
>
>
> > When I say it's nice to be able to say that something's impossible, that
> > doesn't mean that I don't want to support all the reasonable use-cases,
> > just that boundaries are a useful tool.  This is very much the Python
> > idiom - consider your options for indenting your code in strange ways,
> > for example :).
>
> Again, I think we agree for the most part.
>
> --
> Andrew McNabb
> http://www.mcnabbs.org/andrew/
> PGP Fingerprint: 8A17 B57C 6879 1863 DE55  8012 AB4D 6098 8826 6868
>
> _______________________________________________
> Twisted-web mailing list
> Twisted-web at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
>
>


-- =

Blog:  http://channel3b.wordpress.com
Second Life Name:  Ciemaar Flintoff

Watch out for the invisible man.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-web/attachments/20081106/1f=
6ceea9/attachment.htm


More information about the Twisted-web mailing list