[Twisted-Python] Twisted Plugins - Implementation Discussion

Phil Christensen phil at bubblehouse.org
Thu Apr 7 00:06:07 EDT 2011


On Apr 6, 2011, at 8:35 PM, Stephen Thorne wrote:
> For your consideration, and (constructive) critcism, here is a twisted
> plugin that is nearly identical to 6 that I have running in  
> production:
[snip]
> serviceMaker = ExampleServiceMaker()
>
> From these 19 lines of code there are 4 things that are relevent:
> tapname = example
> description = 'Example Twistd Plugin'
> options = {'debug':True} if args == ['--debug'] else {'debug':False}
> service = examplepackage.examplemodule.make_service(**options)
>
> Only 2 of which are relevent for running a twistd daemon other than
> 'example'
>
> So the goal of my post to this mailing list is:
>
> * I would like glyph's goal of having less arbitary code executed at
>  twistd launch time to become a realisation,

Makes sense. My first inclination is to suggest creating metadata  
files that are found via pkg_resources.

> * I would like the process of creating a twisted plugin to be less  
> of a
>  cut+paste+fill-in-blanks hassle.

This doesn't bother me so much. To go back in time a bit:

> First, the reason i use twisted plugins is because they're the a way  
> to
> easily do sensible things with logging, daemonisation and interaction
> from init.d files.
>
> * The number of imports required to compose a plugin is annoying. 2
>  interfaces from two different packages, plus needing
>  zope.interface.implements.
>
> * I've never liked the twisted arg parser, I use it only grudingly, it
>  would be nice to be able to throw argv at my make_service call.

I've got no issues with t.p.usage, and tend to keep its definitions in  
the plugin class. But I agree you should be able to skip it, although  
I have a suspicion you probably already can by being sneaky.

> * The entire task of having this python plugin is to link up the
>  metadata with a more or less standard

Not sure if I get this part. I tend to think of plugins as the service  
bootstrap file, like an int main(). I would probably *not* be into the  
idea of passing sys.argv directly to a service, for example.

> * It's very easy to accidentally make your plugin load your package  
> for
>  every other twistd daemon running out of the same plugin cache.

Never had this happen, but I think I can see where it might. It would  
be solved by some kind of external metadata, though, right?

> * The practice of putting a module under twisted/plugins/ 
> $mymodulehere.py
>  upsets my equilibrium, the only reason I tolerate this kind of thing
>  is that I install my python code via rpms and I automatically get
>  installation, ownership and uninstallation done in a stable manner.

The only issue I've found with this is the issue of having to create  
plugin cache files. Apart from the spurious error messages that were  
(partially?) covered at the sprint recently, the benefit they provide  
is definitely negated by bad permissions. I've done some acrobatics  
inside my setup.py to make it work, but it involves different steps  
depending on whether you're installing or building a package.

Still, it seems like most of their necessity would be negated by using  
external metadata files. It seems to be the way most plugin systems  
end up going in some way or another.

Anyways, that's just my 2/100ths. I'm pretty happy with most of the  
plugin model, but I think there's a lot of room for improvement,  
particularly in the area of dropin.cache files.

-phil



More information about the Twisted-Python mailing list