[Twisted-web] Twisted.web/Twisted.web2 and multiple threads
Phil Mayers
p.mayers at imperial.ac.uk
Fri Oct 12 07:37:57 EDT 2007
On Fri, 2007-10-12 at 00:34 +0200, Fabien Schwob wrote:
> Hello,
>
> I'm new to twisted and I would like to know if the following problems
> can be solved with Twisted.
>
> My goal is to create a daemon that will download RSS/Atom feeds, parse
> them and insert various information in the database. So my idea is to
> use twisted create a program that will have two "main" threads :
You don't use threads with twisted. Twisted is an async event-driven
framework.
The only use for threads in twisted is calling non-Twisted APIs that
block (and ideally release the python GIL). The most prominent instance
of this is the twisted.enterprise.adbapi wrapper around the python
dbapi.
>
> * the first on is an HTTP REST interface to allow the user to add feeds
> url in a queue. I think I will use twisted.web ou web2 here
>
> * the second thread launch several child thread that download feeds and
> parse the. I take feeds from the queue and parse them.
You don't do it that way (though possibly you might want to run the XML
parsing in a thread if the feeds are large).
I suggest you look at the Twisted examples to get a feel for running a
server and a client in the same thread.
>
> So the goal is to transform the http://dpaste.com/22198/ script into a
> daemon where I can add feed with a REST API. So, what is the base
> solutions to do that with twisted.
>
> Thanks
>
More information about the Twisted-web
mailing list