[Twisted-Python] Request for ideas

Ken Whitesell kwhitesell at adelphia.net
Sat Dec 3 16:24:07 EST 2005

Warning - fairly long post follows for what I hope is a simple question 
that only needs (possibly) a short answer. (I'm not looking for any 
code, just one of a couple possible ideas.)

Environment: A LAN with a server and multiple PCs. Full control is 
available over all the PCs. I have the ability to run whatever I want on 
each - in this case it means Python 2.4 / Twisted 2.

Background / Use Case: Imagine a simultaneous silent auction. (More like 
an RFP-type situation.) Any of the PCs can put an item up for bid. Other 
PCs can respond (or ignore!) that item. There are two timeout 
requirements - an "acknowledgement" timeout, where the PC has a certain 
period of time to respond "I want to make a bid", and a response timeout 
that restricts the clients to how much time they have to make the bid. 
These timeouts can change depending on the item going up for bid.
(How and 'what' goes up for bid is irrelevant to my question.)

What I've done so far: I've created a class called "CommTrans" to manage 
the auction. Each time an item goes up for bid, an instance of CommTrans 
is created.

CommTrans creates two objects, one each to manage the acknowlegements 
and the other to manage the replies (AckMgr, RepMgr). The RepMgr is a 
Viewable that can be called by the client to submit its bid. (RepMgr is 
passed to the client in the callRemote call.)

CommTrans issues the callRemote, passing RepMgr to the client. Then it 
sets a function in the AckMgr as the callback.

The client receives the request, and decides whether or not to respond 
to the request. If yes, it issues a callLater on the function to 
determine the bid and submit it to the RepMgr. Either way, the original 
function returns either "Yes" or "No", triggering the callback on the 

Both the AckMgr and RepMgr issue callLater to schedule the timeout. If 
all possible machines reply before the timeout occurs, the timeout is 
cancelled. If the timeout occurs first, a flag is set to have the 
objects reject all further information from the clients. (It's simply 
ignored, nothing special occurs.)

QUESTION #1: Did I miss something in the Twisted API that would handle 
this type of situation? I don't think a Deferred list would work in this 
situation, because it's not going to fire until all deferrs complete - 
something which might not occur.

But this isn't the problem - the code that I have written for these 
functions all work.

The specific problem occurs when objects are reused. I've had situations 
where a client fails to reply for significantly longer than the timeout 
period. In that interval, the original CommTrans, AckMgr and RepMgr have 
been deleted and replaced by new instances. The client finally replies 
to the original request, and the reply is caught by those new objects!
(There is what I think is a very small, but non-zero probability that 
I'm misinterpreting what I'm seeing.)

 From what I can see in the source code, the internal ID is what's set 
by the builtin 'id' function. I'm printing these at key times and I can 
see where they're being duplicated. The only time I see this problem 
occur is when a new object has been created with the same id as a 
previous object.

So this leads me to QUESTION #2: Anyone have ideas on how to resolve 
this? (Note: I can't channel everything through a single CommTrans 
object, because multiple bids might be happening concurrently with 
overlapping timeouts.)

About the only thought I've had so far is to generate a GUID-like object 
for each CommTrans, and include that in every communication to and from 
the clients. The server would check each ack and each reply to ensure 
that the ack and reply are for the proper auction.

Sending another message to the client isn't going to be a good solution, 
because the client might be tied up doing other things - starving the 
reactor - so by the time the message is processed, it may be too late.

This also brings up QUESTION #3, although it hasn't happened yet: Server
does a callRemote to the client from a commTrans object. Before the 
remote function returns (client is _extremely_ busy) the commTrans 
object goes away. What will happen when the function exits? (I know, I 
ought to generate a test-case for this - I'm more wondering if anyone 
knows right off-hand.)

All thoughts, ideas, comments, recommendations are encouraged. Thanks to 
any/everyone who managed to get this far through my writing.

Finally, I'm most pleased to announce that my proposal to provide a 
"Getting Started with Twisted" tutorial at PyCon has been accepted! I 
know I can't turn people into Twisted gurus in 3 hours, but I hope I can 
get a few over that initial hurdle.


More information about the Twisted-Python mailing list