[Twisted-Python] Flush socket

[Tobias Oberstein]

Thanks for the tip, I've gone that route as it was the simplest to realize. Since I had centralized all calls
to transport.write() in a wrapper in the code, I didn't had to implement an ITransport ..

It seems to "work" (modulo the cautions you iterated) .. and I no longer use reactor internals.

I had to introduce a layer of buffering though (which kicks in as soon as I do a "synch"/"chopped" send .. test prg below.

Thanks again, cheers

=== CLIENT ===

import binascii, time
from collections import deque
from twisted.internet import reactor, protocol
from twisted.internet.defer import inlineCallbacks, Deferred


class TricklingClientProtocol(protocol.Protocol):

   def __init__(self):
      self.send_queue = deque()
      self.triggered = False

   def _trigger(self):
      if not self.triggered:
         self.triggered = True
         self._send()

   def _send(self):
      if len(self.send_queue) > 0:
         e = self.send_queue.popleft()
         self.transport.write(e)
         reactor.callLater(0.000001, self._send)
      else:
         self.triggered = False

   def send(self, data, sync = False, chopsize = None):
      if chopsize > 0:
         i = 0
         n = len(data)
         done = False
         while not done:
            j = i + chopsize
            if j >= n:
               done = True
               j = n
            self.send_queue.append(data[i:j])
            i += chopsize
         self._trigger()
         #print "chopped send"
      else:
         if sync or len(self.send_queue) > 0:
            self.send_queue.append(data)
            self._trigger()
            #print "synced send"
         else:
            self.transport.write(data)
            #print "normal send"

   def connectionMade(self):
      self.transport.setTcpNoDelay(True)
      self.part1()

   def part1(self):
      LEN = 50
      self.send("123" * LEN)
      for i in xrange(0, LEN):
         self.send("456", sync = True)
      self.send("789" * LEN, chopsize = 1)
      self.send("123" * LEN)
      reactor.callLater(0.3, self.part2)

   def part2(self):
      self.send("xyz" * 5)
      self.send("abc" * 5, chopsize = 1)
      reactor.callLater(5, self.transport.loseConnection)


class TricklingClientFactory(protocol.ClientFactory):

   protocol = TricklingClientProtocol

   def clientConnectionFailed(self, connector, reason):
     reactor.stop()

   def clientConnectionLost(self, connector, reason):
      reactor.stop()


if __name__ == '__main__':
   factory = TricklingClientFactory()
   reactor.connectTCP("localhost", 9000, factory)
   reactor.run()

=== SERVER ====

import binascii
from twisted.internet import reactor, protocol

class TricklingServerProtocol(protocol.Protocol):

   def __init__(self):
      pass

   def connectionMade(self):
      print "client accepted"
      self.transport.setTcpNoDelay(True)
      self.stats = {}

   def connectionLost(self, reason):
      print "client lost"
      for s in sorted(self.stats):
         print "%dx chop of length %d" % (self.stats[s], s)

   def dataReceived(self, data):
      l = len(data)
      self.stats[l] = self.stats.get(l, 0) + 1
      #print data


class TricklingServerFactory(protocol.ServerFactory):

   protocol = TricklingServerProtocol

   def __init__(self):
      pass

   def startFactory(self):
      pass

   def stopFactory(self):
      pass


if __name__ == '__main__':
   factory = TricklingServerFactory()
   reactor.listenTCP(9000, factory)
   reactor.run()


Von: twisted-python-bounces at twistedmatrix.com [mailto:twisted-python-bounces at twistedmatrix.com] Im Auftrag von Glyph Lefkowitz
Gesendet: Freitag, 12. August 2011 06:06
An: Twisted general discussion
Betreff: Re: [Twisted-Python] Flush socket

On Aug 11, 2011, at 7:43 PM, exarkun at twistedmatrix.com<mailto:exarkun at twistedmatrix.com> wrote:


This will always be a somewhat unreliable way to test a remote process's
handling of packetization, since there are still two TCP/IP stacks which
can mess around with the data in a variety of ways, but it's as good as
you can do if you want to use normal sockets for this testing.


This bears repeating.  It's really unreliable.  Really.  The sizes of buffers passed to send() and recv() bear only a coincidental resemblance to each other; one test setup may reproduce them reliably when the next will suddenly behave completely differently.  If you want even a reasonably reliable heuristic here, you need to send() and then introduce a delay.  You can do this without your own IWriteDescriptor implementation though; just implement an ITransport that does its write() by breaking things up and then calling the underlying write() with callLater()s in-between.

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20110812/bb7102d0/attachment.html>