[Twisted-Python] Request for ideas
kwhitesell at adelphia.net
Sat Dec 3 16:24:07 EST 2005
Warning - fairly long post follows for what I hope is a simple question
that only needs (possibly) a short answer. (I'm not looking for any
code, just one of a couple possible ideas.)
Environment: A LAN with a server and multiple PCs. Full control is
available over all the PCs. I have the ability to run whatever I want on
each - in this case it means Python 2.4 / Twisted 2.
Background / Use Case: Imagine a simultaneous silent auction. (More like
an RFP-type situation.) Any of the PCs can put an item up for bid. Other
PCs can respond (or ignore!) that item. There are two timeout
requirements - an "acknowledgement" timeout, where the PC has a certain
period of time to respond "I want to make a bid", and a response timeout
that restricts the clients to how much time they have to make the bid.
These timeouts can change depending on the item going up for bid.
(How and 'what' goes up for bid is irrelevant to my question.)
What I've done so far: I've created a class called "CommTrans" to manage
the auction. Each time an item goes up for bid, an instance of CommTrans
CommTrans creates two objects, one each to manage the acknowlegements
and the other to manage the replies (AckMgr, RepMgr). The RepMgr is a
Viewable that can be called by the client to submit its bid. (RepMgr is
passed to the client in the callRemote call.)
CommTrans issues the callRemote, passing RepMgr to the client. Then it
sets a function in the AckMgr as the callback.
The client receives the request, and decides whether or not to respond
to the request. If yes, it issues a callLater on the function to
determine the bid and submit it to the RepMgr. Either way, the original
function returns either "Yes" or "No", triggering the callback on the
Both the AckMgr and RepMgr issue callLater to schedule the timeout. If
all possible machines reply before the timeout occurs, the timeout is
cancelled. If the timeout occurs first, a flag is set to have the
objects reject all further information from the clients. (It's simply
ignored, nothing special occurs.)
QUESTION #1: Did I miss something in the Twisted API that would handle
this type of situation? I don't think a Deferred list would work in this
situation, because it's not going to fire until all deferrs complete -
something which might not occur.
But this isn't the problem - the code that I have written for these
functions all work.
The specific problem occurs when objects are reused. I've had situations
where a client fails to reply for significantly longer than the timeout
period. In that interval, the original CommTrans, AckMgr and RepMgr have
been deleted and replaced by new instances. The client finally replies
to the original request, and the reply is caught by those new objects!
(There is what I think is a very small, but non-zero probability that
I'm misinterpreting what I'm seeing.)
From what I can see in the source code, the internal ID is what's set
by the builtin 'id' function. I'm printing these at key times and I can
see where they're being duplicated. The only time I see this problem
occur is when a new object has been created with the same id as a
So this leads me to QUESTION #2: Anyone have ideas on how to resolve
this? (Note: I can't channel everything through a single CommTrans
object, because multiple bids might be happening concurrently with
About the only thought I've had so far is to generate a GUID-like object
for each CommTrans, and include that in every communication to and from
the clients. The server would check each ack and each reply to ensure
that the ack and reply are for the proper auction.
Sending another message to the client isn't going to be a good solution,
because the client might be tied up doing other things - starving the
reactor - so by the time the message is processed, it may be too late.
This also brings up QUESTION #3, although it hasn't happened yet: Server
does a callRemote to the client from a commTrans object. Before the
remote function returns (client is _extremely_ busy) the commTrans
object goes away. What will happen when the function exits? (I know, I
ought to generate a test-case for this - I'm more wondering if anyone
knows right off-hand.)
All thoughts, ideas, comments, recommendations are encouraged. Thanks to
any/everyone who managed to get this far through my writing.
Finally, I'm most pleased to announce that my proposal to provide a
"Getting Started with Twisted" tutorial at PyCon has been accepted! I
know I can't turn people into Twisted gurus in 3 hours, but I hope I can
get a few over that initial hurdle.
More information about the Twisted-Python