[Twisted-Python] Real-world but self-contained example?

Orestis Markou orestis at orestis.gr
Sat Jun 18 02:56:51 EDT 2011


Hello,

I'm trying to come up with a real-world example of performant twisted code to show as the conclusion of the europython training I'll be giving this Monday.

After much help in the #twisted channel, I've written a simple "website monitoring service". I'm trying to keep everything as small and self-contained as possible. There are 3 files, one with a very simple http GET implementation, one with a 1000 sites (some might be NSFW-this is taken from the Alexa site rank) and a simple web service that uses a couple of DeferredQueues and cooperate to do the actual work.

However, I find that still there are issues - sometimes the output will block completely, signifying a blocked reactor (?). I don't know what else to try and fix, apart from setting up a local DNS server to cache the DNS queries. I'd appreciate either

a) someone looking at the code and pointing out possible bugs or ways it could be improved or
b) someone pointing out a proven, scaling real-world project that shows examples of doing things like this

Thanks,
Orestis


 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: httpget2.py
Type: text/x-python-script
Size: 1519 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20110618/0996bfaa/attachment-0003.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: monitor7.py
Type: text/x-python-script
Size: 3653 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20110618/0996bfaa/attachment-0004.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sites.py
Type: text/x-python-script
Size: 15151 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20110618/0996bfaa/attachment-0005.bin 


More information about the Twisted-Python mailing list