<HTML>
<HEAD>
<TITLE>Re: [Twisted-Python] Flow-control for Pipes / Unix Domains Sockets?</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>Hi Itamar,<BR>
<BR>
once again Twisted surprises me: it just works!<BR>
<BR>
I have tested a master/child combo over pipe (both Twisted) with a push producer on the sending leg.<BR>
<BR>
It can push around 70MB/s raw binary (16k write size) to the child. Load is evenly on 2 CPU cores each at 80% and stays roughly constant. Memory is flat for both.<BR>
<BR>
This is on some years old Macbook (kqueue reactor .. a little faster than select) - tomorrow I repeat on a beefy system/OS.<BR>
<BR>
I will do more systematic throughput and also latency measurements .. but this is already not bad at all.<BR>
<BR>
Moreso: I really am curious now how that works under the hood, since the pausing/resuming seems to be upper/lower watermark controlled .. it'll resume the producer before the consumer starves. As it should be;)<BR>
<BR>
Background: this is part of experiments in preparation for a multi-core capable Autobahn based message broker ..<BR>
<BR>
Takeaway: Next time I don't waste time on the internet reading half-baked posts, but just hack away;)<BR>
<BR>
/Tobias <BR>
<BR>
PS: The following is actually slower than above master/slave pair .. which I also didn't expect:<BR>
<BR>
tobias-obersteins-macbook-pro-2:masterchild oberstet$ time dd if=/dev/zero bs=1k count=1000000 | wc -c<BR>
1000000+0 records in<BR>
1000000+0 records out<BR>
1024000000 bytes transferred in 17.454178 secs (58667902 bytes/sec)<BR>
1024000000<BR>
<BR>
real 0m17.460s<BR>
user 0m18.389s<BR>
sys 0m7.729s<BR>
<BR>
<BR>
<BR>
On 22.10.13 02:00, "Itamar Turner-Trauring" <<a href="itamar@itamarst.org">itamar@itamarst.org</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'> <BR>
On 10/21/2013 04:48 PM, Tobias Oberstein wrote:<BR>
<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'> Flow-control for Pipes / Unix Domains Sockets? Hi,<BR>
<BR>
with TCP (either remote or loopback) I can have flow-control using the producer-consumer machinery that Twisted provides.<BR>
<BR>
Is that (flow-control / producer-consumer) also available (and practically usable/recommended) with:<BR>
<BR>
a) Unix Domain sockets<BR>
b) Pipes (spawnProcess)<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'> <BR>
I would expect transport.registerProducer to work with both.<BR>
<BR>
</SPAN></FONT></BLOCKQUOTE>
</BODY>
</HTML>