[Twisted-Python] Allowing a type to be transmitted by jelly?

Brian Warner warner at lothar.com
Tue Dec 27 13:38:28 EST 2005

>>I am pretty sure newpb, which Brian Warner was threatening to merge any
>>day now, will support this much better than PB does.


In oldpb, object graphs are serialized through recursive calls to Jelly,
which has a big switch statement (in twisted.spread.jelly._Jellier.jelly)
that uses a different one-object serializer for each type. To teach oldpb how
to handle a new type, you have to modify this function, and the corresponding
unjellier, and then arrange for your Broker to use the new _Jellier classes
instead of the normal ones. Subclassing pb.Copyable does away with this, but
as you've noticed it is both slightly inefficent (particularly if you're
using __slots__ anyway) and requires extensive surgery or pre-marshalling to
use with third-party code that doesn't happen to inherit from pb.Copyable.

In newpb, object graphs are serialized through iterative calls to a
collection of Slicers. Each time a new object needs to be serialized, the
enclosing Slicers help decide (based upon both the type of the object and any
adapters that have been registered for it) which new Slicer to use. To teach
newpb how to serialize a new type, you register an adapter for it (e.g.
registerAdapter(BooleanSlicer, bool, ISlicer)). To teach newpb how to
unserialize the resulting stream, you define an Unslicer class (there is some
metaclass magic which auto-registers the Unslicer for you.. I'm still going
back and forth on the best way to do this).

The advantage of the adapter/registration scheme is that you don't have to
modify the inheritance set of the classes you want to serialize, so
third-party code is easy to accomodate without surgery or pre-marshalling.
Pre-marshalling is a serious nuisance, because to do it correctly means
looking inside every possible container for further instances of the type
that needs to be converted into something jelly can handle, which is just as
much work as the final serialization itself.

Take a look at branches/newpb-again, in the doc/pb directory, for some
examples and descriptions of how to use newpb. The code itself is in the
twisted/pb directory (unless we come up with a different name for it). It's
still in development: it was in the trunk for a few hours last week, but
hopefully I'll get the remaining test failures resolved and it will go back
into the trunk some time this week.


More information about the Twisted-Python mailing list