[Twisted-Python] How to receive a big stream data?

[Pavel Bastov]

Hello Steven,

I met the similar problem in the project I'm currently developing.

Here is how I solved it: I delimit each message with a special char
(0x1f in my case) which I'm sure will not occur inside any message.

Every time a new chunk arrives (either complete message or a part of
it) I check if it is delimited with this char. If yes, then I process
it, if not, then I add this chunk up to the internal buffer.

This seems to work perfectly for me.

Here is a piece of code illustrating the concept (taken as is from the
working code):

class xooChatProtocol(Protocol):

    def __init__(self):
        self.buf = ''

    def dataReceived(self, data):

        commands = data.split('\x1f')
        if len(commands) > 0:
            i = 1
            num = len(commands)
            for command in commands:
                if i < num:
                    request = self.buf + command
                    self.requestReceived(request)
                    self.buf = ''
                else:
                    self.buf += command
                i += 1


Hope this helps.

-- 

Good luck,

Pavel Bastov
xooChat Team Leader and xooChat Evangelist
http://www.xoochat.com/