[Twisted-Python] Making twisted web client requests on running reactor
exarkun at twistedmatrix.com
exarkun at twistedmatrix.com
Sun Feb 23 06:11:32 MST 2014
On 02:17 am, lokare.ameya at gmail.com wrote:
>Hi everyone,
>
>I'm writing a client library that makes (potentially long running) HTTP
>requests. Since the library will be used within non-Twisted code, I was
>thinking of running the reactor in a separate thread. The library code
>looks roughly like this:
>
>def start():
> Thread(target=reactor.run, args=(False,)).start()
>
>def shutdown():
> reactor.callFromThread(lambda x: reactor.stop(), None)
>
>def make_request(url):
> agent = Agent(reactor)
> d = agent.request(
> 'GET',
> url,
> Headers({
> 'User-Agent': ['Twisted SSE Client'],
> 'Cache-Control': ['no-cache'],
> }),
> None)
> d.addCallback(self.cbRequest)
>
>Then I make requests from the application like so:
>
>start()
>make_request("http://localhost:12000")
>....
>make_request("http://localhost:12000")
>...
>shutdown()
>
>However, it looks like the 'd' deferred never gets fired. From a little
>playing around, it looks like this is because the reactor is started
>before
>the call(s) to agent.request are made. All examples in the twisted docs
>create the request before running the reactor. I'm sure I'm missing
>something very obvious here. Also, is there a better way to design such
>a
>library?
It sounds like you want to use Crochet (which Hynek already linked you
to).
The reason your code doesn't work, though, is that Twisted APIs are not
thread safe. You may *only* call them in the thread the reactor is
running in.
Your `start` function runs the reactor in a new thread. Then the code
proceeds to try to use `Agent` from the original thread. This is
unsupported.
Jean-Paul
More information about the Twisted-Python
mailing list