[Twisted-Python] Making twisted web client requests on running reactor

[Ameya Lokare]

Hi everyone,

I'm writing a client library that makes (potentially long running) HTTP
requests. Since the library will be used within non-Twisted code, I was
thinking of running the reactor in a separate thread. The library code
looks roughly like this:

def start():
    Thread(target=reactor.run, args=(False,)).start()

def shutdown():
    reactor.callFromThread(lambda x: reactor.stop(), None)

def make_request(url):
        agent = Agent(reactor)
        d = agent.request(
            'GET',
            url,
            Headers({
                'User-Agent': ['Twisted SSE Client'],
                'Cache-Control': ['no-cache'],
            }),
            None)
        d.addCallback(self.cbRequest)

Then I make requests from the application like so:

start()
make_request("http://localhost:12000")
....
make_request("http://localhost:12000")
...
shutdown()

However, it looks like the 'd' deferred never gets fired. From a little
playing around, it looks like this is because the reactor is started before
the call(s) to agent.request are made. All examples in the twisted docs
create the request before running the reactor. I'm sure I'm missing
something very obvious here. Also, is there a better way to design such a
library?

-- 
-Ameya
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20140222/c98efa6d/attachment.html>