[Twisted-Python] Twisted Plugins - Implementation Discussion

Glyph Lefkowitz glyph at twistedmatrix.com
Thu Apr 7 14:20:32 EDT 2011


On Apr 7, 2011, at 8:14 AM, Itamar Turner-Trauring wrote:

> On Thu, 2011-04-07 at 02:08 -0400, Glyph Lefkowitz wrote:
> 
>> My idea for exposing it is that if you then do 'getPlugins(IPlugin2)',
>> you will get back an iterable of IPlugin2 providers, but not
>> necessarily instances of your classes: they could be cached plugins,
>> with cached results for metadata() and willProvide() - the latter
>> based on the list currently saved as the 'provided' attribute.  So a
>> loop like this to load a twistd plugin by name:
>> 
>> 
>>        def twistdPluginByTapname(name):
>>            for p2 in getPlugins(IPlugin2):
>>                if p2.willProvide(IServiceMaker) and
>>        p2.metadata()['tapname'] == name:
>>                    return p2.load()
>> 
>> 
>> ... would not actually load any plugins, but work entirely from the
>> cached metadata.
> 
> That's where the whole idea falls down for me. Evidence suggests (and
> you note this earlier) that caching doesn't work anywhere in the real
> world. My current Ubuntu install complains about a read-only cache every
> time I run lore (and I'm pretty sure there's nothing added to my
> PYTHONPATH other than installed system packages). Any design which
> assumes caching works appears to be useless in the real world.
> 
> So, the design has to *not* rely on caching working.

Here's an idea: let's make caching actually work :).  Prior experience indicates that with some small amount of dedication, it's possible to make a module in Twisted not be broken all the time.


As you observed that I already mentioned earlier in the thread, caching never works because post-installation hooks are such a pain, and you have to have special permissions to access the cache file.  So, separately from this, we could attempt a secondary cache read/write to a location much more likely to be writable by the user (something like ~/.local/var/cache/usr_lib_python2.6_site-packages.dropin.cache) read if the first one is out of date and written if writing the first one fails.

Also: we already rely on this behavior, so things are just as broken now for you.  For example, you'll end up loading the code for all twistd plugins and trial reporters when what you want are lore plugins.  This could also be fixed independently.

(To fix your particular installation right now, 'sudo twistd --help' or 'sudo lore' once.)

And, finally, as a separate consideration, we could make "cached metadata" mean "explicitly specified metadata" instead.  The important thing that I'm talking about doing first is making the system work exactly the same way that it does now, with one additional feature in the API which would allow us to make use of metadata that lives outside the Python code, using the existing mechanism for storing metadata that is currently not used.  For a first cut, we wouldn't even remove the ServiceMaker hack, just add the new feature to it so that we could do slightly less importing at startup.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20110407/85b0c50f/attachment.htm 


More information about the Twisted-Python mailing list