[Twisted-Python] Twisted Plugins - Implementation Discussion

Stephen Thorne stephen at thorne.id.au
Wed Apr 6 20:35:18 EDT 2011


G'day,

So Glyph and I had a discussion about the architecture and
implementation of plugins on IRC this week, I raised some issues that
I've seen with implementing plugins in that discussion, and he said that
I should take the discussion to the list because IRC wasn't the right
place for it.

First of all, a quick discussion of the current plugin architecture, so
that we're on the same page.

'twistd' automatically imports python modules from under twisted/plugins
relative to sys.path[1], or it loads a cache of those plugins from
dropin.cache, or it loads a cache of those plugins from dropin.cache.

The reason it loads all of those plugins is so that 'tapname' and
'description' can be grabbed out of all of the serviceMaker attributes
of all those modules.

Then running 'twistd' shows a helpful list of commands, and 'twistd
$tapname' uses the correct serviceMaker to start whatever service is
specified.

Part of the discussion was about how to rewrite this in such a way that
no python code needs to be run in order to discover all the
tapname+description combinations that are available to twistd, this is
because of a perceived performance and sanity deficit in using 'twistd'.

In the course of the discussion I raised several things that I consider
annoyances in the twisted plugin system. I will repeat them here.

First, the reason i use twisted plugins is because they're the a way to
easily do sensible things with logging, daemonisation and interaction
from init.d files.

* The number of imports required to compose a plugin is annoying. 2
  interfaces from two different packages, plus needing
  zope.interface.implements.

* I've never liked the twisted arg parser, I use it only grudingly, it
  would be nice to be able to throw argv at my make_service call.

* The entire task of having this python plugin is to link up the
  metadata with a more or less standard

* It's very easy to accidentally make your plugin load your package for
  every other twistd daemon running out of the same plugin cache.

* The practice of putting a module under twisted/plugins/$mymodulehere.py
  upsets my equilibrium, the only reason I tolerate this kind of thing
  is that I install my python code via rpms and I automatically get
  installation, ownership and uninstallation done in a stable manner.

For your consideration, and (constructive) critcism, here is a twisted
plugin that is nearly identical to 6 that I have running in production:

from zope.interface import implements
from twisted.python import usage
from twisted.plugin import IPlugin
from twisted.application.service import IServiceMaker

class Options(usage.Options):
    optFlags = [['debug', 'd', 'Emit debug messages']]

class ExampleServiceMaker(object):
    implements(IServiceMaker, IPlugin)
    tapname = 'example'
    description = 'Example Twistd Plugin'
    options = Options

    def makeService(self, options):
        from examplepackage.examplemodule import make_service
        return make_service(debug=options['debug'])

serviceMaker = ExampleServiceMaker()



More information about the Twisted-Python mailing list