[Twisted-Python] Using reactor to connect to multiple servers

Andrew Bennetts andrew-twisted at puzzling.org
Fri Oct 26 03:06:23 MDT 2007


Vijay Mathew wrote:
> Hi all,
> 
> I am in the process of writing a http client using twisted.
> I want it to connect to multiple http servers and download more than one page
> asynchronously, without using threads.
> How can I use the reactor object to connect to multiple servers and download
> data like this?

Just call getPage (or reactor.connectTCP or whatever) multiple times, without
waiting for the first one to complete.

E.g. here's a toy example:

    from twisted.internet import reactor
    from twisted.web.client import getPage

    def gotPageCallback(page):
        print "got a page!"
    
    getPage('http://example.com/page1').addCallback(gotPageCallback)
    getPage('http://example.com/page2').addCallback(gotPageCallback)

    reactor.run()

The key thing to realise is that Twisted APIs like getPage are asynchronous:
they return immediately, before the operation they perform has completed (or
even begun, sometimes).  So doing things like calling getPage twice in a row
like that means you are getting two pages at the same time.

-Andrew.





More information about the Twisted-Python mailing list