[Twisted-Python] CONNECTION_LOST not an integer (docstring error?)

[Michael McNeil Forbes]

On Oct 3, 2008, at 5:25 PM, glyph at divmod.com wrote:
>> I know that and am using callFromThread: The reason I am having  
>> bad behavior is because the socket is backing up.  Once I put in  
>> the  logic to throw out data, it works fine... but the lack of  
>> reactor  support concerns me somewhat.
>
> What do you mean by "backing up"? That doesn't really make sense.   
> When a transport "backs up" in Twisted, it just allocates a larger  
> and larger buffer to store the data that is queued to be sent.
I thought that the socket buffer was backing up, but now see that  
write() does buffer.  I incorrectly attributed some problems with  
data arriving out of order and/or getting lost with this... I will  
have to look deeper into what is causing these problems.  I am using  
a push producer now and the server seems to be sending things fine.   
As far as I can tell, however, the receiver is sometimes getting the  
data out of order when data comes in too fast on the client side.  I  
have to look into that and see what the cause is... maybe I am not  
being careful enough with the threads.

(On the client side, I want the user to have full access to a python  
interpreter with readline functionality etc. and the ability to plot  
things with matplotlib using Tkinter.  As far as I can see, I can't  
get readline functionality with a manhole or similar interface  
controlled by twisted, so again I have to run twisted in a separate  
thread.)

> And what is the "lack of reactor support" you're talking about?
Oh, the "lack of support" for writeSomeData by some reactors bothered  
me and prompted me to use write;-)

>> I don't want the computation to stop,  but think I could put an  
>> intermediate "push producer" that stops by  discarding the data.
>
> I don't *really* understand what you're trying to accomplish here;  
> it soudns to me like you actually want a pull producer that just  
> always sends the latest state in resumeProducing(), assuming it's  
> always changing.  Of course the "every 30 minutes" timeframe  
> confuses that somewhat.  However, using some kind of producer is  
> definitely the way to go.
The point here is that there are two forms of usage: one is when the  
computation is fast and the data is coming so fast that the client  
will have to drop some of it.  I agree that this should be more of a  
pull producer, but the other form of usage is where the computation  
takes about 30 minutes before new data is ready.  This should be  
implemented as a push producer.

>> Is there a simpler way to do this all with twisted without using  
>> threads? (The main goal is having an extremely minimal set of  
>> hooks for the people writing the computational code, and having  
>> the computational code run as fast as possible.)
>
> You could use subprocesses.  Valentino Volonghi is working on a  
> convenient process-pool for use with Twisted: <https:// 
> launchpad.net/ampoule/>.  I don't think that would actually help  
> you that much though, since you'd still need to have application-  
> specific buffering logic (it's a slightly unusual requirement to  
> throw away intermediary state depending on the buffer saturation).

Thanks for the suggestions and help.
Michael.