[Twisted-Python] Two main loops

[Jasper]

Jean-Paul Calderone wrote:
> On Mon, 12 Nov 2007 20:08:29 +0100, Nitro <nitro at dr-code.org> wrote:
>> Am 12.11.2007, 19:47 Uhr, schrieb Jasper <jasper at peak.org>:
>>> To get to the point, I actually do feel that the fault lies with  
>>> Twisted, or more specifically PB.  Code using PB wants to be as 
>>> unaware  of the details of network transmission as possible.  It 
>>> certainly  doesn't want to slice things into a bazillion chunks for 
>>> individual  transmission, and then puzzle them back together on the 
>>> other end.
>>>
>>> IMHO it makes more sense for PB to understand it shouldn't block, 
>>> and to chunk and schedule its serialization (and unserialization) in 
>>> the same manner as the TCP packets sent underneath.  I'm guessing 
>>> this would make for cleaner separation, and more general usefulness.
>>
>> I think this is a) not possible and b) not desirable.
>
> I think it's possible.  It may even be desirable, although whether it is
> worth the cost of implementing or not is a question worth considering.
>
[snip]
>
> PB's serializer could try very hard to avoid running for a long period of
> time without giving control back to the reactor.  Of course, someone 
> would
> have to implement this.  Whether or not that is worthwhile to implement,
> instead of doing what Glyph suggested (manually breaking up the work into
> smaller pieces) is a separate question.

I'm not sure it's worth the cost either, but I'm also not splitting up 
the data into chunks is easier.  Looking through jelly.py, I find the 
places I'd want to split data align pretty closely with the chunking 
already done during jellying; why duplicate this work?

That said, I'm not precisely sure how to keep track of progress between 
time slices, although I nebulously intuit it could be done via nested 
generators.  I'm sure I don't fully understand the implications of this, 
but I've been looking for a practical reason to fiddle with generators 
for a while anyway. ;-)

-Jasper