[Twisted-Python] Twisted and python-ogre

[Jean-Paul Calderone]

On Fri, 15 Feb 2008 15:01:21 +0100, Thomas Boucher <thomas.boucher at student.ecp.fr> wrote:
>Jean-Paul Calderone a écrit :
>>Chris Armstrong's response to the post you refer to is accurate.  Twisted
>>APIs are not threadsafe.  In order to call them from a thread other than
>>the one in which the reactor is running, it is absolutely required that you
>>use reactor.callFromThread.  This is quite simple.  Instead of:
>>
>>    proto.sendLine("line")
>>
>>Simple do:
>>
>>    reactor.callFromThread(proto.sendLine, "line")
>>
>>You will have to do this for every call you make to a Twisted API (except
>>for reactor.callFromThread, of course ;).
>
>I have just tried this method, and it perfectly works.
>The twisted reactor is run in another thread and the calls to Twisted are 
>managed by the callFromThread method. The main advantage I see to it 
>(excepting the fact that it works) is that it does not require changes in 
>the way the graphic rendering is done : you do not care about how python- 
>ogre runs to code the twisted part.
>>The best solution would be to really integrate the two loops.  I don't know
>>the details of OGRE though, so I can't tell you specifically how to do 
>>this.
>>It is tightly dependent on the details of the loops being integrated.
>
>What I can't see for the moment is how much better it would be to do like 
>the other ways delivered in the discussion thread (by Werner, Bernie an 
>Stefan). These ways consist in running Twisted first and in letting it 
>manage python-ogre (by doing the rendering frame by frame with coiteration, 
>or loopingcalls).
>Is this better than doing separate threads ?

There are arguments about efficiency.  Using threads has overhead which is
not present when not using threads.  These arguments are probably not very
convincing in this case, since you would only have two threads and their
overhead will be dwarfed by your rendering costs.  You should be aware that
reactor.callFromThread _does_ have some non-trivial overhead.  Using it for
each sendLine call _could_ have a noticable detrimental impact on your app's
performance.

There are arguments about simplicity.  It's harder to understand what a
multithreaded program is doing.  It's harder to debug a multithreaded
program.  It's very, very hard to unit test a multithreaded program.  This
is probably the compelling argument to try to avoid threads.

>>Because most other event loops are very bad at managing events. :)  If you
>>want reasonable network performance, you need a good network event loop to
>>be in control.
>
>Don't you think using basic sockets (easy to code in Python) to make the 
>network connection from a client to a server is more efficient (considering 
>the time spent to do it or the complexity of the code) and more easy to keep 
>in a project ?

That might be nice.  However, almost without exception, the network code I
read which does not use Twisted is buggy in the most trivial ways.  It is
rare for it to take more than a cursory glance to notice critical bugs
(the sort which will result in data being lost).  Of course, this doesn't
mean that all network code written without Twisted is fatally flawed, but
it does give me a reasonable basis to assume that it will have issues. :)

Twisted's APIs aren't just easier to use correctly than the BSD socket APIs,
they're also consistent across platforms.  Are you sure you know the BSD
APIs well enough to write some that is correct on Linux and on Windows?
(Let alone on OS X or FreeBSD)  Even if you are, would you rather spend your
time writing the error handling logic for each of these platforms or adding
features to your game?

Twisted may seems to be harder than the socket module.  It probably does
have a higher up-front cost.  However, in the long run, I think that you'll
save yourself a lot of time using it rather than using sockets directly.

You also get the advantages which the structure Twisted attempts to impose
on your application confers.  For example, protocol/transport separation
means that you can unit test your protocol implementation very easily.  And
that's assuming you even need to implement your own protocol, rather than
using one of the protocols Twisted already includes an implementation of.

You also leave open the possibility of easily taking advantage of some other
Twisted features which don't seem useful now but which you may benefit from
later in development.

Hope this helps,

Jean-Paul