[Twisted-Python] integrating CompStrm//adding background processing

[Glyph Lefkowitz]

On Mon, 2004-05-17 at 08:08, Glyph Lefkowitz wrote:
> On Sun, 2004-05-16 at 22:43, Bill la Forge wrote:
> 
> > As for the timing, I'm counting the number of items I can pass on an
> > async pipe between two virtual processes running in the background of
> > the main thread. Details for running over asyncore are available here:
> > http://compstrm.sourceforge.net/timing.html
> 
> That code doesn't run - for starters, there is no variable 'd' in
> WriteMany.cs, endWrite does not appear to take an argument, I think
> readCount() is supposed to be ReadCount() in test(); it also doesn't
> appear to test the twcs module, but rather only the asyncore loop.  Can
> you package some easy-to-run tests with your next release?

Well, this was an intriguing performance problem, and one that likely
impacts my work, so I went ahead and fixed the tests.  Attached is a
modified copy of twcs.py from the may 16th distribution of compstrm on
sf.net, a twcsperf.py that tests it, and a patch to Twisted that may be
good to consider including.  This patch special-cases a 0 argument to
callLater to bypass the incredibly expensive gettimeofday syscall that
we end up making as a result.  I _think_ this is safe but I haven't run
the test suite on it yet.

On my machine, running the tests twice in a row:

glyph at kazekage:~/Desktop% python twcsperf.py
Pristine Twisted
7.85587286949e-05
12729.3302299 per second
callLater-Patched Twisted
8.00807499886e-05
12487.3955369 per second
glyph at kazekage:~/Desktop% python twcsperf.py
Pristine Twisted
7.8611869812e-05
12720.7252848 per second
callLater-Patched Twisted
7.80673313141e-05
12809.4554171 per second

YMMV, but I believe this effectively eliminates any performance
difference for your use case.  (Without the patch, I was getting more
like "8000 per second".)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: twcs.py
Type: text/x-python
Size: 6093 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20040517/83af88a9/attachment-0004.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: special-case-zero.patch
Type: text/x-patch
Size: 718 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20040517/83af88a9/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: twcsperf.py
Type: text/x-python
Size: 990 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20040517/83af88a9/attachment-0005.py>