[Twisted-Python] "Injecting" a callWhenRunning when the reactor is already running?

[glyph at divmod.com]

On Tue, 01 Aug 2006 12:09:17 -0500, "justind2 at ussonet.net" <justind2 at ussonet.net> wrote:
>Here is a script I've made and am playing with to try to understand how 
>things behave in Twisted. I'm using twistedsnmp but I think my problem here 
>is my understanding of how to use the reactor (or lack there of).

Yeah... I don't know anything about twistedsnmp (it's not actually done by twistedmatrix labs); but I can see a couple of problems.

>I feel like I'm misunderstanding some fundamentals, but I may just be hard- 
>headed.

Here are some fundamentals I think you might be missing:

 - Twisted is not thread safe.  Do not, e.g., call Deferred callbacks or create Deferreds in a thread, call them from methods like dataReceived or connectionLost.  (99% of your problem is here)
 - This goes for pretty much any Twisted-using code as well.  I'm not really sure, not being familiar with twistedsnmp, but I'd bet that AgentProxy is not designed for instantiation or initialization from a thread.
 - It looks like you want callFromThread, not callWhenRunning.
    - callWhenRunning: http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IReactorCore.html#twisted.internet.interfaces.IReactorCore.callWhenRunning
    - callFromThread: http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IReactorThreads.html#twisted.internet.interfaces.IReactorThreads.callFromThread
 - err.trap raises exceptions, so checking it in an 'if' is nonsensical.  Maybe you want err.check?
   - trap: http://twistedmatrix.com/documents/current/api/twisted.python.failure.Failure.html#twisted.python.failure.Failure.trap
   - check: http://twistedmatrix.com/documents/current/api/twisted.python.failure.Failure.html#twisted.python.failure.Failure.check
 - The return value of callWhenRunning is ignored; there's no sense to having return_def return a value.

Some stylistic stuff, too:

 - When posting examples to this mailing list, please reduce your example to a minimal runnable example
 - Messages posted here should also generally follow Twisted conventions, unless there is a specific need not to.  For example:
   - use twisted.python.log, not python's 'logging' module.
   - use reactor.callInThread and reactor.callFromThread, not Python's 'threading' or 'thread' modules.  These have defined interactions with the reactor, random other threads don't.

In summary: your "working" example works only by accident, and only sometimes.    I'd love to give you a simple answer, but you don't have a specific problem or misunderstanding; I think you've got entirely the wrong idea of how Twisted is running your code.

You probably want to back off from SNMP a little bit and familiarize yourself with more basic applications of the Twisted programming model before trying this again.  This tutorial might be a good place to get started:

http://twistedmatrix.com/projects/core/documentation/howto/tutorial/index.html

Of course, the O'Reilly book is a pretty good source of documentation too :).