[Twisted-Python] Twisted Plugins - Implementation Discussion

[Tim Allen]

On Thu, Apr 07, 2011 at 12:54:45AM -0400, Glyph Lefkowitz wrote:
> If we invent our own file extension which has to be separately
> installed, we have to teach distutils, and setuptools, and distribute,
> and pip, and distutils2, and 'packaging' (as I'm sure that will
> eventually be incompatible with distutils2 for some silly reason), and
> easy_install, and dpkg, and rpm, and yum, and apt, and probably five
> other horrible Python packaging things that I don't even know about
> yet, how to deal with it.  So I am strongly in favor of keeping
> everything in .py files and just making a minor tweak to what's stored
> in dropin.cache (and perhaps allowing dropin.cache to be stored in
> some location more likely to be writable by individual users, in case
> the installation process doesn't update it).

My understanding was that .py files have to be installed into
twisted/plugins as binary blobs, not as ordinary Python modules, because
of special rules like "twisted/plugins must not be a Python package". If
distutils/setuptools/etc. can handle a binary blob with a ".py"
extension, I figured it could handle a binary blob with any other
extension.

If that's wrong, then yeah, I guess that would be a problem.

> >    - Each such file is loaded with Python's ConfigParser module.
> 
> The first rule of the Twisted cabal is of course "don't talk about the
> Twisted cabal", but the second and possibly even more important rule
> is "no '.ini' files".  I'd seriously much rather we use XML.  And you
> can ask Stephen how he feels about XML configuration files.  (Although
> I'd strongly recommend standing well clear of him when you do that,
> and making sure that no sharp or otherwise dangerous objects are
> within easy reach.)

Well, the nice thing about ConfigParser is that it's in the stdlib, and
people already know how to create them, and rolling
yet-another-config-file-format seems crazy in this day and age.

If you need a non-Turing-complete config language and rule out .ini and
XML, I'm not sure what's left. JSON, perhaps.

> I don't want a solution that is hard-coded to deal with the metadata
> that 'twistd' specifically needs, as Twisted plugins are already used
> for more than just twistd plugins, and I'd like them to be used for
> even more.

I've never actually come across anything that used Twisted plugins
besides twistd, so I'd forgotten they weren't twistd-specific.

> So as not to make this message too long, I'll defer a description my
> own preferred implementation strategy for a future post to this
> thread.

I'll look forward to it. :)

Tim.