[Twisted-Python] Advise for heavy concurrency

Alec Matusis matusis at yahoo.com
Sun Dec 2 00:39:30 MST 2007


PS. Sorry I forgot: 

> t.i.threads.deferToThread(f, clients) ?

I suppose my question also applies to 

twisted.internet.reactor.callInThread(f, clients)


> -----Original Message-----
> From: twisted-python-bounces at twistedmatrix.com [mailto:twisted-python-
> bounces at twistedmatrix.com] On Behalf Of Alec Matusis
> Sent: Saturday, December 01, 2007 11:27 PM
> To: 'Twisted general discussion'
> Subject: RE: [Twisted-Python] Advise for heavy concurrency
> 
> > Because it is not a supported use of the API.  Twisted APIs, unless
> > other-
> > wise documented, are not thread safe and can only be called from the
> > thread
> > in which the reactor is running.
> 
> Then I do not understand when it is safe to use
> 
> t.i.threads.deferToThread(f, clients) ?
> 
> I was thinking of calling this from the main reactor thread, where f is
> 
> def f(clients):
>     for client in clients:
>         client.protocol.transport.write('hello')
> 
> 
> This function would not even have a deferred attached to it.
> Is the point that it's not thread-safe to have Twisted API calls
> inside the function f?
> 
> > -----Original Message-----
> > From: twisted-python-bounces at twistedmatrix.com [mailto:twisted-
> python-
> > bounces at twistedmatrix.com] On Behalf Of Jean-Paul Calderone
> > Sent: Saturday, December 01, 2007 10:25 PM
> > To: Twisted general discussion
> > Subject: RE: [Twisted-Python] Advise for heavy concurrency
> >
> > On Sat, 1 Dec 2007 19:40:28 -0800, Alec Matusis <matusis at yahoo.com>
> > wrote:
> > >> > You cannot call any Twisted functions from a thread, with the
> > >> > exception
> > >>                                         ^^^^^^^^^^^^^ Er, I guess
> I
> > >> should have worded this more carefully. What I meant is: you
> cannot
> > >> call any Twisted functions from a different thread to the one the
> > >> reactor is running in, which includes any thread created with
> > >> deferToThread.
> > >>
> > >> > of callFromThread; so this would probably be pretty pointless.
> > >
> > >
> > >I am not sure I understood this: suppose my message sending loop
> > simply
> > >loops over client protocol instances and calls
> > >transport.write(some_message)
> > >
> > >Why cannot I call this loop in a thread from the main reactor loop
> > >to have it non-blocking in case this sending loop is giant (over
> 1000s
> > of clients)?
> >
> > Because it is not a supported use of the API.  Twisted APIs, unless
> > other-
> > wise documented, are not thread safe and can only be called from the
> > thread
> > in which the reactor is running.
> >
> > Jean-Paul
> >
> > _______________________________________________
> > Twisted-Python mailing list
> > Twisted-Python at twistedmatrix.com
> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python





More information about the Twisted-Python mailing list