[Twisted-Python] A Solution Re: [Stackless] Advice on Debugging Stackless/Twisted Problem

Andrew Francis andrewfr_ice at yahoo.com
Fri May 11 13:39:57 EDT 2007


Hello Colleagues:

Earlier I posted that I was having a problem involving
the loss of deferred calls in a programme that uses
Stackless and Twisted. I found the problem and wrote a
solution. I think it interesting.

It helped when I started using the Twisted logging
facility :-)

The immediate reason the deferred were "lost" was
because the main reactor loop stopped iterating.

What happened was:

1. In the example, Stackless treats Twisted as a
tasklet. 

see:
http://www.stackless.com/pipermail/stackless/2007-April/002448.html

2. When the Request Handler is created, it is running
as a *part* of the Twisted tasklet. 

3. Consequently when the Request Handler blocked on a
channel, this blocked the entire reactor! In turn, 
doIteration() (doSelect) cannot do its work. Twisted
grinds to a halt. 

4. When the last tasklet (TestTasklet) blocked, all
the tasklets became blocked. Consequently the
programme ended. 

The solution was two-fold.

1) To prevent the reactor from blocking, I run the
request handler in its own tasklet. This prevents the
reactor from blocking for indeterminate periods of
time.

2) I modified doSelect to call stackless.schedule().
If the reactor does not call stackless.schedule(),
other tasklets will not get an opportunity to run.

(When I have time, I will write a proper reactor. This
should be a little more than overriding the
doIteration() method of selectReactor). Or re-read the
Blocking demo : I don't quite understand how it works)

With this approach, I 

1) Eliminate a OS-Thread (both Stackless and Twisted
folks concur that this is wrong, wrong, wrong).

2) Can execute tasklets in parallel with tasklets that
are blocked waiting for network I/O like an incoming
HTTP request.

I think it is neat that the Twisted framework is
flexible enough to be contorted in this fashion.
Unfortunately this approach is *very* slow. I feel it 
is wastefully that a tasklet is created for every
incoming request. I have to work on this. 

~

I understand the subset of Stackless and Twisted users
is small. And the philosophies are different. And what
I am doing is a bit esoteric and application specific.
Hopefully others will be able to build upon this : I
don't see why programmers can't get the best of the
Stackless and Twisted worlds.

Cheers,
Andrew 








       
____________________________________________________________________________________Yahoo! oneSearch: Finally, mobile search 
that gives answers, not web links. 
http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Solution.py
Type: text/x-python
Size: 9599 bytes
Desc: 2700386180-Solution.py
Url : http://twistedmatrix.com/pipermail/twisted-python/attachments/20070511/7f2c3259/attachment.py 


More information about the Twisted-Python mailing list