[Twisted-Python] Foolscap-0.1.1 released

[Brian Warner]

Robert Gravina <robert at gravina.com> writes:

> Wow, I had a read through the docs you made based on oldpb (http://
> twistedmatrix.com/~warner/Foolscap/copyable.html and http://
> twistedmatrix.com/~warner/Foolscap/using-pb.html) and think I finally
> understand what Foolscap is about :)

Thanks!

> - I like how you've simplified Referenceable/Copyable/Cacheable to just the
> one type - Referenceable - while making the whole thing more  powerful in the
> process.

Well, to be honest, this is more a result of not having gotten around to
implementing Cacheable (or really finishing work on Copyable) than an
explicit design decision. But yes, the fundamental operation is the
Slicer/Unslicer pair, and Referenceables are just the pass-by-reference form
of this (and Copyables are just the pass-by-copy form). I suspect that
Cacheable is best implemented with something fairly application-specific, and
so I'm inclined to gather some use cases before providing explicit support
within Foolscap (and possibly corralling people into a non-ideal solution).

> - Being able to pass RemoteReferences to a third party - very cool!

This is a big new feature, and makes things much more transparent. The actual
implementation is going to change at some point (to provide some better
ordering guarantees, and reduce the amount of traffic on the wire), but the
user-visible API will remain the same: just pass your object through a
callRemote() somehow.

> - Serialisers are a great idea! I often want to send classes down the wire
> without having to subclass them. Also, many python types can be  serialised
> now (e.g. datetime - I don't think these work with PB, so  I have to convert
> them to strings and back again... I do this with  Decimals too.... yuck!)

Yeah, the ability to register an ISlicer adapter is what makes this powerful.
You can register an adapter for some arbitrary 3rd-party class and it will
get used to serialize the instances, no matter how buried they might be
inside the object graph. I'm still looking for common patterns and useful
refactorings here, though. If you have examples of ways that this is helpful,
please feel free to post them. For example, I'm not sure if the helper
classes for Copyable are actually all that helpful, because I haven't yet hit
a personal need to do pass-by-copy with anything larger than a dict.

> -  You can write a  Slicer/Unslicer pair to get an object that is copied by
> value the first time it sent and then copied by reference  all later times. I
> *really* would like to be able to do that.

The ISlicer interface should be powerful enough to do that, although I should
mention that the definition of "first time" is a bit hazy. Look at
foolscap.referenceable.ReferenceableSlicer.sliceBody() for an example of how
it detects "first time" versus later times. There is a network optimization
that we could make if we didn't need to make this distinction (it would mean
the 'decref' message wouldn't need a response), but since I want to send
RemoteInterface names on that first time and not thereafter, I'm inclined to
retain the distinction.

The biggest thing I want to figure out how to add is an object that's
serialized somewhere in-between Referenceable and Copyable. My use-case is a
little music-player client/server app I wrote: the server has a list of Song
objects which it can give to the client for display to (and selection by) the
user, and the client sends back the Song that it wants to add to the
playlist. Each Song has a set of strings describing the artist name, album
name, song title, etc. For the purpose of referencing which song to play, I
want Song to be Referenceable. For the purpose of carrying those strings
along with the Song (so the client can display a list of song titles), I want
it to be Copyable. Ideally, the first time the Song was sent over the wire,
it would be serialized as a connection-local ID (aka 'clid') plus a set of
immutable attributes. The client-side SongCopy should have attributes that
can be read, and when it gets sent back to the server in a playlist, it
should be serialized with just the clid.

I figure this might be a useful enough pattern to warrant a base class or
some kind of support code within Foolscap (maybe a CopyableAndReferenceable
class?), but I haven't yet figured out how it ought to be implemented.

> Other parts of Foolscap I've yet to understand, but I just wanted to say I
> like where PB is heading.

Thanks! There's a lot of code that I've implemented and started to use, but
have not yet started to document (like eventual-send, or much of the
schema/constraint mechanics). And there more that I've written but haven't
figured out how to use yet (like promises), and even more that's still in the
design stages (like remote promises, promise-pipelining, and automatic
keepalives). It's been very educational thus far, and promises (no pun
intended) to be even more interesting in the next couple months.

cheers,
 -Brian