[Twisted-Python] Two main loops

glyph at divmod.com glyph at divmod.com
Sun Nov 11 17:27:58 EST 2007


On 09:06 pm, jasper at peak.org wrote:
>That depends.  When I send moderately sized objects over Twisted, this 
>approach causes my frame rate to stop while Twisted is busy, and worse 
>makes my GUI non-interactive.  For a game, this is generally 
>unacceptable...

You can't send "objects" over "Twisted".

Are you talking about a particular protocol in Twisted?  Serializing 
large objects through PB might take a long time.  In that case, it's not 
"Twisted" that is busy, but your program which is busy serializing that 
object.
>I'm not sure of a good/nice way to deal with this.  Slicing 
>transmissions into a sequence of smaller chunks seems like a fair pain 
>in the ass, but then again the alternatives are punting to another 
>thread/process. :-/

Actually, punting to another thread or process is still going to slow 
down your rendering, but in a much less deterministic way.  The OS 
scheduler is going to be interrupting the rendering rather than the 
long-running serialization function.  Breaking down the serialization 
work into smaller chunks will allow you to better control what's going 
on.

Sending the work of serializing a large object to another process won't 
work anyway: how are you going to get that large object to the other 
process to serialize in the first place, without serializing it? 
Sending it to another thread probably won't work either, since if the 
object is large it is likely shared (and therefore will involve locking 
large portions of the data structure for long periods of time) and 
Python's GIL will introduce deleterious effects for your framerate 
anyway.

Fundamentally what you are doing here is optimizing your program.  Your 
serialization is slow and it needs to be faster so that it does not 
impact your framerate.  This is a difficult problem, one which is made 
worse by the fact that Python does not have a particularly fast runtime. 
There's no quick answer: you don't need to restructure your main loop, 
you need to think about what your program is spending its time doing.




More information about the Twisted-Python mailing list