[Twisted-Python] Slow data transfer with Twisted + socat + Windows

[Jason Heeris]

I'm writing an application with Twisted/GTK to manage a serial device,
for use on both Windows and Linux. Unfortunately I can't use serial
comms directly under Windows[1], so my workaround is to launch
socat[2] in a subprocess to relay data between a local TCP port and
the serial port. The socat command I use is:

    socat TCP4-LISTEN:31415 /dev/com4,raw,echo=0,b57600

I can then send and receive data over localhost:31415 same as I would
for a serial port.

The trouble is, a transaction that takes a minute under Linux (using
*either* direct serial comms or the same socat subprocess approach)
now takes more than 50 times as long under Windows.

I've attached the script I'm using to test, mainly so it's clear what
I'm doing — although it's useless unless you happen to have a serial
device using a protocol you can shoehorn into that script. I've also
attached the output where you can see the difference between different
reactors on different platforms. (In my app I use
reactor.spawnProcess(...), but I'm not doing that here.)

Basically, under Linux either approach takes 4 seconds; under Windows
the SelectReactor takes 30s, and the gtk2reactor takes 220s!

Profiling the "reactor.run()" call on the different platforms (both
for the select reactor) showed that (a) most time was spent in the
select.select call, and (b) the time spent in select was 8 times more
under Windows than Linux. I have no idea if this is useful information
or not.

I'm pretty sure it's not a socat-specific issue, because I get the
same problem if I use com2tcp[3]. It could be Cygwin related, but I
don't know how determine that for sure. Also, the fact that it runs
slower using the gtk2reactor (vs. selectreactor) suggests to me that
it could be my code, not theirs.

I found an old Cygwin thread[4] that claimed setting NODELAY on the
listening socket helped with a similar problem, but using

    > socat TCP4-LISTEN:31415,nodelay [etc]

...made no difference.

So I'm a bit stuck. Does anyone know how I can narrow it down further?
Has someone else noticed slow TCP behaviour under Windows? Or Cygwin
utilities? Have I made some glaringly obvious mistake in my script
there?

Any help appreciated. :)

Cheers,
Jason

[1] http://twistedmatrix.com/trac/ticket/4862
[2] http://www.dest-unreach.org/socat/
[3] http://com0com.sourceforge.net/
[4] http://www.mail-archive.com/cygwin@cygwin.com/msg66791.html
-------------- next part --------------
====
WINDOWS
====

>python socat_test.py
Using reactor: SelectReactor
Connection made!
Progress: 0.00
Progress: 0.10
Progress: 0.20
Progress: 0.30
Progress: 0.40
Progress: 0.50
Progress: 0.60
Progress: 0.70
Progress: 0.80
Progress: 0.90
Progress: 1.00
Time: 28.279000

>python socat_test.py
Using reactor: PortableGtkReactor
Connection made!
Progress: 0.00
Progress: 0.10
Progress: 0.20
Progress: 0.30
Progress: 0.40
Progress: 0.50
Progress: 0.60
Progress: 0.70
Progress: 0.80
Progress: 0.90
Progress: 1.00
Time: 222.143000

====
LINUX
====

$ python socat_test.py 
Using reactor: SelectReactor
Connection made!
Progress: 0.00
Progress: 0.10
Progress: 0.20
Progress: 0.30
Progress: 0.40
Progress: 0.50
Progress: 0.60
Progress: 0.70
Progress: 0.80
Progress: 0.90
Progress: 1.00
Time: 4.030676

$ python socat_test.py 
Using reactor: Gtk2Reactor
Connection made!
Progress: 0.00
Progress: 0.10
Progress: 0.20
Progress: 0.30
Progress: 0.40
Progress: 0.50
Progress: 0.60
Progress: 0.70
Progress: 0.80
Progress: 0.90
Progress: 1.00
Time: 4.032430
-------------- next part --------------
A non-text attachment was scrubbed...
Name: socat_test.py
Type: text/x-python
Size: 2642 bytes
Desc: not available
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20110308/6162edde/attachment.py