[Twisted-Python] Flush socket

Tobias Oberstein tobias.oberstein at tavendo.de
Fri Aug 12 09:08:06 EDT 2011

Thanks for the tip, I've gone that route as it was the simplest to realize. Since I had centralized all calls
to transport.write() in a wrapper in the code, I didn't had to implement an ITransport ..

It seems to "work" (modulo the cautions you iterated) .. and I no longer use reactor internals.

I had to introduce a layer of buffering though (which kicks in as soon as I do a "synch"/"chopped" send .. test prg below.

Thanks again, cheers

=== CLIENT ===

import binascii, time
from collections import deque
from twisted.internet import reactor, protocol
from twisted.internet.defer import inlineCallbacks, Deferred

class TricklingClientProtocol(protocol.Protocol):

   def __init__(self):
      self.send_queue = deque()
      self.triggered = False

   def _trigger(self):
      if not self.triggered:
         self.triggered = True

   def _send(self):
      if len(self.send_queue) > 0:
         e = self.send_queue.popleft()
         reactor.callLater(0.000001, self._send)
         self.triggered = False

   def send(self, data, sync = False, chopsize = None):
      if chopsize > 0:
         i = 0
         n = len(data)
         done = False
         while not done:
            j = i + chopsize
            if j >= n:
               done = True
               j = n
            i += chopsize
         #print "chopped send"
         if sync or len(self.send_queue) > 0:
            #print "synced send"
            #print "normal send"

   def connectionMade(self):

   def part1(self):
      LEN = 50
      self.send("123" * LEN)
      for i in xrange(0, LEN):
         self.send("456", sync = True)
      self.send("789" * LEN, chopsize = 1)
      self.send("123" * LEN)
      reactor.callLater(0.3, self.part2)

   def part2(self):
      self.send("xyz" * 5)
      self.send("abc" * 5, chopsize = 1)
      reactor.callLater(5, self.transport.loseConnection)

class TricklingClientFactory(protocol.ClientFactory):

   protocol = TricklingClientProtocol

   def clientConnectionFailed(self, connector, reason):

   def clientConnectionLost(self, connector, reason):

if __name__ == '__main__':
   factory = TricklingClientFactory()
   reactor.connectTCP("localhost", 9000, factory)

=== SERVER ====

import binascii
from twisted.internet import reactor, protocol

class TricklingServerProtocol(protocol.Protocol):

   def __init__(self):

   def connectionMade(self):
      print "client accepted"
      self.stats = {}

   def connectionLost(self, reason):
      print "client lost"
      for s in sorted(self.stats):
         print "%dx chop of length %d" % (self.stats[s], s)

   def dataReceived(self, data):
      l = len(data)
      self.stats[l] = self.stats.get(l, 0) + 1
      #print data

class TricklingServerFactory(protocol.ServerFactory):

   protocol = TricklingServerProtocol

   def __init__(self):

   def startFactory(self):

   def stopFactory(self):

if __name__ == '__main__':
   factory = TricklingServerFactory()
   reactor.listenTCP(9000, factory)

Von: twisted-python-bounces at twistedmatrix.com [mailto:twisted-python-bounces at twistedmatrix.com] Im Auftrag von Glyph Lefkowitz
Gesendet: Freitag, 12. August 2011 06:06
An: Twisted general discussion
Betreff: Re: [Twisted-Python] Flush socket

On Aug 11, 2011, at 7:43 PM, exarkun at twistedmatrix.com<mailto:exarkun at twistedmatrix.com> wrote:

This will always be a somewhat unreliable way to test a remote process's
handling of packetization, since there are still two TCP/IP stacks which
can mess around with the data in a variety of ways, but it's as good as
you can do if you want to use normal sockets for this testing.

This bears repeating.  It's really unreliable.  Really.  The sizes of buffers passed to send() and recv() bear only a coincidental resemblance to each other; one test setup may reproduce them reliably when the next will suddenly behave completely differently.  If you want even a reasonably reliable heuristic here, you need to send() and then introduce a delay.  You can do this without your own IWriteDescriptor implementation though; just implement an ITransport that does its write() by breaking things up and then calling the underlying write() with callLater()s in-between.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20110812/bb7102d0/attachment-0001.htm 

More information about the Twisted-Python mailing list