<div dir="ltr"><div>Hi,</div><div><br></div><div>I am encountering a weird bug, where some TCP client connections get into a state where the server is able to read data sent from the client, but not able to send any data with transport.write().</div>
<div><br></div><div>With some help from the #twitsed IRC channel, I was able to gather the following information regarding the bug. While I'm still unable to provide steps to reporduce this bug, I am able to reliably find clients who are in this state. I am running 24 instances of the twisted server (Epoll reactor) running on Ubuntu, with a peak traffic of >130k users. At any instance, there are < 20 TCP connections stuck in this state.</div>
<div><br></div><div>Here is some information about the bug:</div><div><br></div><div>1. transport.write() does not send anything down the socket</div><div>2. transport.doWrite() will send all the data that has been buffered up, and then stop sending any new data.</div>
<div>3. transport.writeSomeData() will send data</div><div>3. reactor.getWriters() will return a list of transports that are all stuck in this state, and the writers will remain in this list.</div><div>4. Calling reactor.removeWriter(transport) will "unstuck" the transport and data gets streamed once again.</div>
<div>5. A small number of clients will receive data for a while, and return to this stuck state. Most return to normal once reactor.removeWriter() is called.</div><div>6. Based on the suggestion from IRC user _habnabit, I used strace after removing the writer, here is the output:</div>
<div><br></div><div><span class="" style="white-space:pre"> </span>(4:52:09 PM) thewrongboy: epoll_ctl(3, EPOLL_CTL_MOD, 6504, {EPOLLIN, {u32=6504, u64=22205092589476200}}) = 0</div><div><span class="" style="white-space:pre"> </span>(4:52:09 PM) thewrongboy: epoll_ctl(3, EPOLL_CTL_MOD, 6504, {EPOLLIN|EPOLLOUT, {u32=6504, u64=22205092589476200}}) = 0</div>
<div><span class="" style="white-space:pre"> </span>(4:52:09 PM) thewrongboy: epoll_ctl(3, EPOLL_CTL_MOD, 6504, {EPOLLIN, {u32=6504, u64=22205092589476200}}) = 0</div><div><span class="" style="white-space:pre"> </span>(4:52:09 PM) thewrongboy: epoll_ctl(3, EPOLL_CTL_MOD, 6504, {EPOLLIN|EPOLLOUT, {u32=6504, u64=22205092589476200}}) = 0</div>
<div><span class="" style="white-space:pre"> </span>(4:52:09 PM) thewrongboy: epoll_ctl(3, EPOLL_CTL_MOD, 6504, {EPOLLIN, {u32=6504, u64=22205092589476200}}) = 0</div><div><br></div><div>For now, I am using a LoopingCall to check and remove transports that are stuck in getWriters().</div>
<div><br></div><div>I am using Twisted 12.3.0 on Ubuntu 12.04 - 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux.</div><div><br></div><div>Has anyone else experienced this weird problem? I'd love to provide more information regarding this bug.</div>
</div>