[Twisted-Python] Twistd thread problem

orbitz at ezabel.com orbitz at ezabel.com
Wed May 18 13:57:52 EDT 2005


If your making a proxy you shouldn't a queue or worker threads.  Just 
connect to the destination and send your incoming data to it and it's 
outgoing data back to the client.  This can all be done in one thread.  
Even if you don't do it like this you also don't need a thread to send 
the data, sending smtp is fairly trivial you can do it by hand, or use 
the smtp client that comes with twisted (might as well learn sometime).

On May 16, 2005, at 7:50 PM, Jan Van Uytven wrote:

> Hi all,
>
> I coded an SMTP proxy that receives incoming e-mail requests and 
> pushes the received e-mail onto a Queue were it is processed by a 
> worker thread that finishes by sending the e-mail on to another SMTP 
> server.
>
> The server part of the proxy was written in Twisted, but the worker 
> thread uses smtplib to fire off the processed e-mails (I didn't know 
> how to use the twisted smtp client in a thread).
>
> When I run the application using twistd -noy <appname>, it works 
> great. When I try to daemonize it (twistd -oy <appname>) the server 
> part works, but the worker thread doesn't. It creates and then I 
> suspect it immediately dies, as no error is returned upon creation. A 
> debug of the queue reveals that the thread is not pulling items off 
> it. A debugging statement placed at the top of the thread never runs, 
> and the twistd log reports nothing wrong. I'm using 
> reactor.callInThread to start the worker function.
>
> I tried running twistd --spew -oy <appname> and captured the log it 
> produced. I did a filter for anything with 'thread' in it, this is the 
> result:
>
> 2005/05/16 09:15 PDT [-] function ? in 
> /usr/lib/python2.3/site-packages/twisted/python/threadpool.py, line 11
> 2005/05/16 09:15 PDT [-] function WorkerStop in 
> /usr/lib/python2.3/site-packages/twisted/python/threadpool.py, line 23
> 2005/05/16 09:15 PDT [-] function init in 
> /usr/lib/python2.3/site-packages/twisted/python/threadable.py, line 
> 219
> 2005/05/16 09:15 PDT [-] function ThreadPool in 
> /usr/lib/python2.3/site-packages/twisted/python/threadpool.py, line 31
> 2005/05/16 09:15 PDT [-] function ThreadSafeList in 
> /usr/lib/python2.3/site-packages/twisted/python/threadpool.py, line 
> 199
> 2005/05/16 09:15 PDT [-] function init in 
> /usr/lib/python2.3/site-packages/twisted/python/threadable.py, line 
> 219
> 2005/05/16 09:15 PDT [-] method __init__ of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method start of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method adjustPoolsize of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method _startSomeWorkers of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method callInThread of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method _startSomeWorkers of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method startAWorker of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] method __init__ of threading.Thread at 
> -1216568116
> 2005/05/16 09:15 PDT [-] method __init__ of threading.Thread at 
> -1216568116
> 2005/05/16 09:15 PDT [-] method _set_daemon of threading.Thread at 
> -1216568116
> 2005/05/16 09:15 PDT [-] function currentThread in 
> /usr/lib/python2.3/threading.py, line 669
> 2005/05/16 09:15 PDT [-] method isDaemon of threading._MainThread at 
> -1214935060
> 2005/05/16 09:15 PDT [-] function Condition in 
> /usr/lib/python2.3/threading.py, line 147
> 2005/05/16 09:15 PDT [-] method __init__ of threading._Condition at 
> -1216567604
> 2005/05/16 09:15 PDT [-] method __init__ of threading._Condition at 
> -1216567604
> 2005/05/16 09:15 PDT [-] method start of threading.Thread at 
> -1216568116
> 2005/05/16 09:15 PDT [-] method _note of threading.Thread at 
> -1216568116
> 2005/05/16 09:15 PDT [-] method run of threading.Thread at -1216568116
> 2005/05/16 09:15 PDT [-] method _worker of 
> twisted.python.threadpool.ThreadPool at -1216569236
> 2005/05/16 09:15 PDT [-] function currentThread in 
> /usr/lib/python2.3/threading.py, line 669
> 2005/05/16 09:15 PDT [-] function threadMailer in twisted-copyuser.py, 
> line 201
> 2005/05/16 09:15 PDT [-] method acquire of 
> twisted.python.threadable._XLock at -1216639284
> 2005/05/16 09:15 PDT [-] method release of 
> twisted.python.threadable._XLock at -1216639284
> 2005/05/16 09:15 PDT [-] method acquire of 
> twisted.python.threadable._XLock at -1216639284
> 2005/05/16 09:15 PDT [-] method release of 
> twisted.python.threadable._XLock at -1216639284
> 2005/05/16 09:15 PDT [-] method acquire of 
> twisted.python.threadable._XLock at -1216639284
> 2005/05/16 09:15 PDT [-] method release of 
> twisted.python.threadable._XLock at -1216639284
> 2005/05/16 09:15 PDT [-] function registerAsIOThread in 
> /usr/lib/python2.3/site-packages/twisted/python/threadable.py, line 
> 262
>
> Why would the worker thread act differently with twistd daemonized? 
> Any suggestions, advice, or hints would be welcome.
>
> Thanks,
>
> Jan
>
> -- 
> Jan Van Uytven
> Computer Systems & Network Engineer
> Convergent Media Network
> wyvern at crm3.com
>
> Only in silence the word,
> only in dark the light,
> only in dying life:
> Bright the hawk's flight on the empty sky.
>
> - Ursula K. LeGuin, 'A Wizard of Earthsea'
>
> *** LEGAL STUFF ***
> This electronic message transmission, including any attached files, 
> contains information which may be confidential and/or privileged.
> The information is intended to be for the SOLE use of the individual 
> or entity named above. If you are not the intended recipient, be aware 
> that any review, disclosure, copying, distribution or use of the 
> contents of this information is prohibited. This message is not to be 
> forwarded for any reason without prior consent of the sender.
> If you have received this electronic transmission in error, please 
> notify the sender immediately by telephone (250) 386-4266 or by a 
> 'reply to sender' only message and destroy all electronic and hard 
> copies of the communication, including attachments, without reading or 
> saving in any manner.
>
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>





More information about the Twisted-Python mailing list