[Twisted-Python] pb.Cacheable vs. offline queuing

Brian Warner warner at lothar.com
Sat Nov 5 21:55:53 EST 2005

> I've been looking at Cacheable, as the live updates between servers
> would be useful, but I have a few questions:
> How does it avoid collisions, if an attribute is changed on both ends
> at once?  Is any kind of 2PC involved, or similar?  I read that some
> part of reading/observing is atomic, but is modifying atomic as well?

Luckily (well, for us at least :), there is no concept of "simultaneous"
here. PB offers two distinct classes for cacheable use: pb.Cacheable on the
"sending" side, and pb.RemoteCache on the "receiving" side. Modification
messages flow in one direction only. As long as you don't trigger some kind
of reentrancy problem, each update message is basically atomic, but there is
no mechanism in place to send changes back in the other direction.

So if you want to allow both sides to change things, you will have to build
some sort of layer on top of pb.Cacheable, which is where you'd put in your
synchronization logic. You have to figure out when the "other" object is
created, find a way to wire it back to its partner, and make sure that
changes don't travel in endless loops between the two paired objects.

> the two servers (public and office) must both keep serving users, even if
> the connection between them is severed.

Be aware that trying to accomodate disconnected operation is like planting
kudzu in your flowerbed with a promise that you'll "keep an eye on it". Each
time you think you've figured out everything there is to know about
distributed synchronization, you discover a new academic paper explaining a
new horrible failure method that you're vulnerable to. The depths awaiting to
be plumbed know no bound. (the problem is basically equivalent to merging in
a distributed version control system, and look at how many of *those* we've
got running around :).

Worse yet, the same problems actually exist in always-connected operation,
it's just that you can trick yourself into thinking you can ignore them more
often. Unidirectional dataflow is at least four orders of magnitude easier to
deal with.

> any collisions would be dealt with manually by the office staff once the
> connection came back up.

Good. You might still have a chance to retain your sanity :). 

Yeah, Cacheable won't be enough for you. My hunch is that you'll be
implementing enough new code that you might as well not bother building it on
top of pb.Cacheable: start with a pb.Referenceable and a remote_acceptUpdate
method, an outboundUpdates() queue, and some setter methods to trigger
outbound update messages.

good luck,

More information about the Twisted-Python mailing list