[Twisted-Python] Degrading under load

[Yitzchak Gale]

On 3/9/06, Itamar Shtull-Trauring <itamar at itamarst.org> wrote:
> On Thu, 2006-03-09 at 23:13 +0200, Yitzchak Gale wrote:
>> When running listenTCP, how often does twisted
>> accept pending connections on the port?

> The server will... accept connections on each iteration of
> the event loop where the server socket is readable.

OK.

> If processing something when data is received takes a long time though
> this mean the event loop won't get control back and so accept()ing will
> be delayed.

Right. So in this scenario, performance under load would
depend on breaking up the higher-level processing steps
into small enough pieces.

>> And when twisted does accept pending connections,
>> does it accept ALL of them and queue them all for
>> processing, or just one at a time?

> IIRC most reactors will try to accept as many as possible,
> up to some limit in each iteration.

If I was going to do A, I would certainly want to check that
carefully. It's not like using select() or poll(). To get a
non-blocking accept(), for example, I think you need to
set some flag on the listen() call.

But right now it looks like I'll do B. It is still quite simple,
and looks more robust to me.

Thanks,
-Yitz