[Twisted-Python] Components

Glyph Lefkowitz glyph at twistedmatrix.com
Thu Feb 26 17:56:23 EST 2004

On Thu, 2004-02-26 at 14:59, Phillip J. Eby wrote:
> At 02:43 PM 2/26/04 -0500, Bob Ippolito wrote:
> >How about just migrating off of t.p.components and switching to 
> >PyProtocols?  The license is compatible, it has PEP-backing, it is a 
> >superset of what t.p.components does, has optional Pyrex acceleration, and 
> >is compatible with Interfaces from itself, t.p.components, and 
> >zope.components.  The default Interface implementation does not support 
> >the __call__ adaptation that we all know and love, but it is actually 
> >generic enough to allow it.
> Yep, just subclass InterfaceClass and add a __call__ method that calls back 
> to adapt() and you'd be all set.  One other benefit to PyProtocols is that 
> you can track what interfaces an *instance* supports, independent of what 
> its class supports.

Would you consider adding this to PyProtocols directly?  If we're going
to maintain a component system as part of Twisted, I would like to at
least get the benefit of having full control over the component system
within Twisted.  I don't want to have some people using PyProtocols and
others using PyProtocols+TwistedHacks.

There are a lot of features I'd add to Twisted's component system if I
had time, such as:

	- implicit context-dependent location of closest running t.a.service
services by interface
	- interface-based context (moshez's context trick)
	- automatic generation of interfaces from any class
	- IComponentized
	- context / interface based log separations

And of course, integrating foom's string-based components would be great
too.  There is a lot of friction even to add something like this to
Twisted.  I imagine that adding something like this to PyProtocols, with
potentially more projects out there depending on the exact specifics of
all its semantics, would be even worse.

The other alternative is to add a bunch of specific hacks to PyProtocols
that get loaded only when Twisted gets loaded, which could potentially
introduce compatibility problems with other PyProtocols-using code,
which would sort of invalidate the whole point of using a common
components system in the first place.

Then we have the issue of the PyProtocols dependency; dependency
management can be quite hairy on windows.

> The more controversial aspect, however, is transitive adaptation.  IIRC, 
> some of the core Twisted developers consider implicit transitive adaptation 
> to be dangerous and/or evil.  However, that could possibly also be dealt 
> with in a protocols.Interface subclass by disabling the parts that support 
> that.

Before veering off into this, I'd like to retract any comments about
performance.  I tried pyprotocols a while ago, on some simple cases, and
it was pretty slow compared to Twisted's dirt-simple code.  I don't
think I was using the C code path, but it was more than a 2-4x
difference at the time.

I will test again with more reasonable cases before saying anything more
specific.  I bet Twisted has gotten slower and PyProtocols has gotten
faster since then, though.

Parts of PyProtocols do strike me as dangerous, evil, and overcomplex,
though :)  In particular,


The idea of passing numeric priorities for different implementations has
always seemed deeply wrong to me.  I have worked with one or two systems
like this in the past (some MUD code in C++) where, inevitably, someone
will want to make the 'real' default adapter for interface X; then
someone else will want to make the 'really real' default adapter. 
Different developers will eventually keep trying to write comparison
methods that leapfrog each other backwards to get to the correct result
for last-most-from-greater-than-everything, which turns into a
bug-ridden mess (and it's never really clear who should be "winning"
this race to be the final overrider anyway).

More importantly I don't really understand if that's in fact what the
'depth' value is used for, because my eyes glaze over halfway through
the above web page :)  PyProtocols feels to me like it's gone out of
even the upper levels of abstraction that the Twisted team is used to
inhabiting, straight into the Zopeosphere... 4000 lines of code related
to components, whereas t.p.components has 300?  It worries me.

Maybe I'm alone in these concerns, though.  Does anyone else feel that
depending on PyProtocols would increase the learning curve for Twisted
even more?  Or is this common knowledge in the Python community that
could be leveraged to actually make the curve shallower?  I can
certainly learn to wrap my head around the whole thing if nobody else
has trouble with it :)

> There might be some other issues that could come up, but I'm definitely 
> willing to try to "adapt" to Twisted's needs in these areas, especially if 
> it means I could get rid of PyProtocols' wrapper code and wrapping tests 
> for Twisted's existing interface class.  :)

This is clearly something that we need to talk about more.  As many
silly disagreements about design as I can come up with, a common
components system would be beneficial to everyone involved.  Are you
coming to PyCon? :)

More information about the Twisted-Python mailing list