[Twisted-Python] Reactor Spinning?

[Werner Thie]

Does it spin forever or does it clear up after some time?

I have occasional 'hickups' with heavy loads with responses taking 
several ten seconds for no apparent reason, the server then working 
again like nothing happened.

Werner

Moxie Marlinspike wrote:
> Hey Twisted folks,
> 
> I'm using the twisted.web framework for a high performance HTTP proxy
> server that very closely resembles the HTTP proxy server example that
> comes with the twisted package.
> 
> Under heavy load, I occasionally run into a problem where the reactor
> appears to start spinning on an "empty" write.  An strace of the process
> reveals these system calls:
> 
> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1
> write(81, "", 0)                        = 0
> gettimeofday({1272980323, 567613}, NULL) = 0
> gettimeofday({1272980323, 567696}, NULL) = 0
> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1
> write(81, "", 0)                        = 0
> gettimeofday({1272980323, 567988}, NULL) = 0
> gettimeofday({1272980323, 568071}, NULL) = 0
> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1
> write(81, "", 0)                        = 0
> gettimeofday({1272980323, 568501}, NULL) = 0
> gettimeofday({1272980323, 568613}, NULL) = 0
> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1
> write(81, "", 0)                        = 0
> gettimeofday({1272980323, 568978}, NULL) = 0
> gettimeofday({1272980323, 569084}, NULL) = 0
> epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 6) = 1
> write(81, "", 0)                        = 0
> 
> ...it seems to be waiting for write availability on a FD, trying to
> write an empty buffer, seeing that it wrote zero bytes, and so waiting
> for availability again.
> 
> I've confirmed that when this occurs, whatever FD it is spinning on does
> correspond with a network socket connected to the proxy.  I've seen this
> with both Twisted 8.2 and 9.0 running on Linux using both the epoll and
> select reactors.
> 
> If anyone has any thoughts about this problem or has seen it before, I'd
> appreciate any insight that anyone might have.
> 
> Thanks,
> 
> - moxie
>