[Twisted-Python] IPushProducer problem

[Jean-Paul Calderone]

On Tue, 13 Jan 2009 16:34:29 +0100, Gabriel Rossetti <gabriel.rossetti at arimaz.com> wrote:
>Hello everyone,
>
>I implemented a push-produce a while back and I though it works, but it 
>didn't. When the msgs where spaced out, it worked, but if several msgs were 
>sent one after the other, then things go bad (the msgs get mixed up)...
>
>I found this thread : http://www.twistedmatrix.com/pipermail/twisted-python 
>/2007-March/014983.html
>
>and I think I ave the same problem, my code looks a bit like his :
>
>    def pauseProducing(self):
>        """
>        Pause the producer
>        """
>        self.__paused = True
>
>    def resumeProducing(self):
>        """
>        Resume the producer
>        """
>        self.__paused = False
>
>        while(not self.__paused and self.__startLimit < self.__total):
>            data = self.__buf[self.__startLimit:self.__endLimit]
>            self.transport.write(data)
>            self.__startLimit = self.__endLimit
>            self.__endLimit += self.__burstSize
>            if(not self.__stream):
>                break
>
>        if(not self.__paused or self.__startLimit >= self.__total):
>            self.stopProducing()
>
>    def stopProducing(self):
>        """
>        Stop the producer
>        """
>        self.__paused = False
>        self.__startLimit = 0
>        self.__buf = None
>        self.__total = None
>        self.__endLimit = self.__burstSize
>
>and when I send a msg I use this :
>
>def sendMessage(self, msg):
>        if domish.IElement.providedBy(msg):
>            msg = msg.toXml()
>              if isinstance(msg, unicode):
>            msg = msg.encode('utf-8')
>                      self.__buf = msg
>        self.__total = len(msg)
>        if(self.__stream):
>            self.resumeProducing()
>
>I think what happens is that the code is re-entered, just like Jean-Paul 
>suggested in the other thread. The thing is I didn't understand the 
>solution, could somebody please explain it to me or suggest something else?

It's extremely difficult to say without seeing a complete example and without
knowing exactly how the code is failing.  However, one thing I do see is that
it is not safe to call your `sendMessage` a second time before all of the
data from the first call has been written.  Since you are using `__buf´ to
record the message to send, if you try to send another message, the first
buffer will be clobbered.  Could this be the problem you're having?

Perhaps consider appending to `__buf´ instead of overwriting it.  There are
probably lots of other areas where this code could improve too, but not
corrupting the internal state of the producer is a good start. :)

Jean-Paul