[Twisted-Python] Fwd: [Twisted-commits] r11032 - Made LineReceiver about 3x as fast, and clarified some of the docstrings.

[Jp Calderone]

Begin forwarded message:

> From: James Knight <foom at wolfwood.twistedmatrix.com>
> Date: Tue, 06 Jul 2004 13:53:09 -0600
> To: twisted-commits at twistedmatrix.com
> Subject: [Twisted-commits] r11032 - Made LineReceiver about 3x as fast,
	and clarified some of the docstrings.
> Reply-to: twisted-python at twistedmatrix.com
> 
> Author: foom
>Date: Tue Jul  6 13:53:08 2004
>New Revision: 11032
>
>Modified:
>   trunk/twisted/protocols/basic.py
>Log:
>Made LineReceiver about 3x as fast, and clarified some of the docstrings.
>
>
>
>Modified: trunk/twisted/protocols/basic.py
>==============================================================================
>--- trunk/twisted/protocols/basic.py	(original)
>+++ trunk/twisted/protocols/basic.py	Tue Jul  6 13:53:08 2004
>@@ -204,24 +204,31 @@
>         rawDataReceived, depending on mode.)
>         """
>         self.__buffer = self.__buffer+data
>+        lastoffset=0
>         while self.line_mode:
>-            try:
>-                line, self.__buffer = self.__buffer.split(self.delimiter, 1)
>-            except ValueError:
>+            offset=self.__buffer.find(self.delimiter, lastoffset)
>+            if offset == -1:
>+                self.__buffer=self.__buffer[lastoffset:]
>                 if len(self.__buffer) > self.MAX_LENGTH:
>-                    line, self.__buffer = self.__buffer, ''
>+                    line=self.__buffer
>+                    self.__buffer=''
>                     return self.lineLengthExceeded(line)
>                 break
>-            else:
>-                linelength = len(line)
>-                if linelength > self.MAX_LENGTH:
>-                    line, self.__buffer = self.__buffer, ''
>-                    return self.lineLengthExceeded(line)
>-                why = self.lineReceived(line)
>-                if why or self.transport and self.transport.disconnecting:
>-                    return why
>+
>+            line=self.__buffer[lastoffset:offset]
>+            lastoffset=offset+len(self.delimiter)
>+
>+            if len(line) > self.MAX_LENGTH:
>+                line=self.__buffer[lastoffset:]
>+                self.__buffer=''
>+                return self.lineLengthExceeded(line)
>+            why = self.lineReceived(line)
>+            if why or self.transport and self.transport.disconnecting:
>+                self.__buffer = self.__buffer[lastoffset:]
>+                return why
>         else:
>-            data, self.__buffer = self.__buffer, ''
>+            data=self.__buffer[lastoffset:]
>+            self.__buffer=''
>             if data:
>                 return self.rawDataReceived(data)
>

This changes dataReceived to call down into user code while the LineReceiver's instance attributes are in an inconsistent state.  This can lead to duplicate data delivery if dataReceived is called into from lineReceived.  Normally this sounds like a ridiculous thing to do, but notice the implementation of resumeProducing():

     def resumeProducing(self):
         self.paused = False
         self.dataReceived('')
         self.transport.resumeProducing()

Since resumeProducing is called immediately when a producer is registered, this means LineReceivers can no longer register themselves as producers for their transports from lineReceived, which is a much more reasonable thing to do.

Demonstration of the bug in action attached.

Jp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: linetest.py
Type: text/x-python
Size: 618 bytes
Desc: not available
URL: </pipermail/twisted-python/attachments/20050618/e47e28dd/attachment-0002.py>