[Twisted-Python] etag and last-modified
lists at debpro.webcom.dk
Fri Nov 5 09:53:34 EST 2004
exarkun at divmod.com wrote:
>>Instead I'll let Feedparser do the download and use twisted for threads.
>>What is the maximum pool size I can use?
> Using threads to do this, there is no point using Twisted at all. Since Twisted is perfectly capable of downloading select web pages based on their headers, there's no reason to use threads.
But I need to download approx 150000 files several times every day from
approx 7000 servers. That's why I thought threading would be the solution.
Can I use Twisted for this?
More information about the Twisted-Python