[Twisted-Python] question about threading
glyph at divmod.com
glyph at divmod.com
Tue Oct 25 11:27:51 EDT 2005
On Tue, 25 Oct 2005 11:05:21 -0400, Phil Christensen <phil at bubblehouse.org> wrote:
Let me start from the other end of your message:
>the client object holds a reference to the protocol object, and sendCommand
>basically just executes:
> print "Sending " + message + " to " + str(self.protocol.source)
> self.protocol.transport.write(message + "\0")
The reactor is not thread safe, and thus this call to 'write' has undefined (and bad) behavior. You _MUST_ use the reactor thread APIs; they're not optional conveniences, they're the only way you can use threads with Twisted :)
>i need to spawn a thread
danger, danger will robinson. Here is where your trouble started :). Actually you *don't* need to spawn a thread, you need to spawn a process - twisted supports processes - and I can't guarantee that os.system will work properly from within a Twisted application. Handling of SIGCHILD has proven to be a sticky wicket in the past.
>i know there's a bunch of reactor methods that deal with threads, but i've
>never used them before, and i'm not sure which one will fix this issue."
There are really only 2: callInThread and callFromThread. There is a convenience API, twisted.internet.threads.deferToThread, which might be what you wanted, if what you wanted was in fact a thread. However, you want spawnProcess in any event. As I said, os.system may not work at _all_ from within Twisted, depending on your operating system.
>here's my thread subclass:
Don't subclass thread. Twisted implements its own threadpool; use callInThread with what used to be your 'run' function.
> def run(self):
> pres = event.getActivePresentation()
> t = datetime.datetime.now()
> base_name = 'sample-file-name'
> out_html = file('/tmp/' + base_name + '.html', 'w')
> # [snip snip snip]
> # write some html to the file
This bit could actually be threaded, if it's slow and blocking. My suggestion: If it's for a demo, just block.
> os.system('html2ps /tmp/' + base_name + '.html > /tmp/' + base_name
>+ '.ps') # > /tmp/' + base_name + '_ps.log')
Since reactor.spawnProcess might be a bit tedious for simply running this here, try this:
twisted.internet.utils.getProcessOutput('/usr/bin/html2ps', ['html2ps', ...).addCallback(keepGoing)
> os.system('mkdir data/transcripts/' + str(pres.id))
^ UGH! Why are you spawning another process here?? os.mkdir, please.
> os.system('ps2pdf /tmp/' + base_name + '.ps data/ transcripts/' +
>str(pres.id) + '/' + base_name + '.pdf > /tmp/' + base_name + '_pdf.log')
^ Another getProcessOutput here. Return the resultant Deferred from within your keepGoing callback...
> lock = threading.Lock()
^ Hooray, now you can forget about this garbage
> self.client.sendCommand('presentTranscript', ['/ transcripts/' +
>str(pres.id) + '/' + base_name + '.pdf'])
^ Do this in the final callback of the Deferred that you've just created.
I hope this helped.
More information about the Twisted-Python