[Twisted-Python] Reactor Spinning?

[Moxie Marlinspike]

Hey Twisted folks,

I'm using the twisted.web framework for a high performance HTTP proxy
server that very closely resembles the HTTP proxy server example that
comes with the twisted package.

Under heavy load, I occasionally run into a problem where the reactor
appears to start spinning on an "empty" write.  An strace of the process
reveals these system calls:

epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1
write(81, "", 0)                        = 0
gettimeofday({1272980323, 567613}, NULL) = 0
gettimeofday({1272980323, 567696}, NULL) = 0
epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 8) = 1
write(81, "", 0)                        = 0
gettimeofday({1272980323, 567988}, NULL) = 0
gettimeofday({1272980323, 568071}, NULL) = 0
epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1
write(81, "", 0)                        = 0
gettimeofday({1272980323, 568501}, NULL) = 0
gettimeofday({1272980323, 568613}, NULL) = 0
epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 7) = 1
write(81, "", 0)                        = 0
gettimeofday({1272980323, 568978}, NULL) = 0
gettimeofday({1272980323, 569084}, NULL) = 0
epoll_wait(3, {{EPOLLOUT, {u32=81, u64=13194703840475086929}}}, 269, 6) = 1
write(81, "", 0)                        = 0

...it seems to be waiting for write availability on a FD, trying to
write an empty buffer, seeing that it wrote zero bytes, and so waiting
for availability again.

I've confirmed that when this occurs, whatever FD it is spinning on does
correspond with a network socket connected to the proxy.  I've seen this
with both Twisted 8.2 and 9.0 running on Linux using both the epoll and
select reactors.

If anyone has any thoughts about this problem or has seen it before, I'd
appreciate any insight that anyone might have.

Thanks,

- moxie

-- 
http://www.thoughtcrime.org