[Twisted-Python] protocol and transport question

Luca Politti luca at unipex.it
Thu Aug 23 09:11:18 EDT 2007


Jean-Paul Calderone ha scritto:
> On Tue, 21 Aug 2007 13:02:34 +0200, Luca Politti <luca at unipex.it> wrote:
>> Hi all,
>> I have a problem with a simple client/server communication: I have a
>> server that send the data through "transport" (my class inherit from
>> twisted.internet.protocol.Protocol), every 0.4 seconds. On the client
>> (that now, for test propose are on the same machine) some time I receive
>> two messages concatenated from the server like it send on the same time.
>> Other times, I receive it correctly (in different moments).
> 
> For message-oriented protocols (what you seem to be implementing), it
> is necessary to have some "framing" mechanism - a way to tell where a
> message ends and the next begins.  You can't rely on time to tell
> messages apart over TCP, since TCP makes few guarantees about when it
> will do things.
> 
>> I tried to debug this situations with print some messages, and I see
>> that twisted sometimes wait "a time" before send data through the
>> channel:
> 
> Bytes written to a TCP transport will be sent (almost) as soon as they
> can be.  They are not (in the current implementation) sent before the
> transport.write() call returns, but they will be sent the next time the
> reactor regains execution control.  Unless your application is blocking
> the reactor from running (ie, performing some long running task), this
> means the reactor will try to send your data at most a few milliseconds
> after your write call.  It is not necessarily the case that the send will
> succeed at that time, though.  In such a case, the reactor will buffer
> the data and try to send it again later.
> 
>>
>> on the server, on twisted.internet.tcp.Connection on writeSomeData I add:
>> print "CCCCCCCCCCCCCCCCC", repr(data), time.time()
>> try:
>>     # Limit length of buffer to try to send, because some OSes are too
>> ....
>>
>> on my code:
>> print "BBBBBBBBBBBBBBBBBBB", repr(msg), time.time()
>> self.transport.write(str(msg))
> 
> writeSomeData and transport.write don't necessarily have a one-to-one
> correspondence to each other, so this debug output might be a bit
> misleading.
> 
>> [snip]
>>
>> How solve it? Is there a method for say to twisted (transport) to not
>> wait to send the data? (flush the buffer?)
> 
> Since there's no buffering except when absolutely necessary, there's no
> way to flush.
> 
> So there are three things to watch out for:
> 
>  * You must use a framing mechanism in order to differentiate your
>    messages.  This might be as simple as having them all be the same
>    length, or it might mean including a length prefix (see the
>    NetstringReceiver or Int{8,16,32}StringReceiver protocols in
>    twisted.protocols.base for examples of this), or it might be
>    something more complex.
> 
>  * Don't block the reactor.  If you want to wait a while, use the
>    callLater method of the reactor, not time.sleep.  If you have
>    to call a function that will block for a long time before it
>    returns, find an asynchronous version instead, or use the reactor's
>    threadpool.
> 
>  * Don't call reactor methods from any thread except the one which
>    is running the reactor.  This will have unpredictable results and
>    generally be broken.
> 
> Jean-Paul
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Ok, Jean-Paul, thanks a lot. We haven't thought to this. We thought that 
every time we called transport.write, it sent the msg...
But we have solved the problem using your first suggest: the framing 
mechanism :). So now, just before send the msg, I append the length of 
the msg at the beginning of this. On the other side, when I receive the 
msg I cut the msg until the length of the msg. The rest is putted in a 
buffer and processed after.
The most important thing is that IT WORKS!!!! :))))

So we thank you very a lot...

Best reguards..

Luca

-- 
Unipex srl

email: luca at unipex.it
Tel:   0432 - 931511
Fax:   0432 - 931378




More information about the Twisted-Python mailing list