[Twisted-Python] Multiple plugins in "twistd"

Wed May 20 20:40:10 MDT 2015

FYI, https://twistedmatrix.com/trac/ticket/7907#ticket created.

On Wed, May 20, 2015 at 7:31 PM Moshe Zadka <zadka.moshe at gmail.com> wrote:

> Hi glyph and exarkun,
>
> [1] The compose syntax is the least interesting part for me here, and
> seems to cause the most bike-shedding, so I think I will punt on that.
> Instead, I will write a function called "compose()" which can be used from
> .tac files or possibly other plug-ins, leaving the issue of the syntax for
> a different day.
> [2] It seems like the biggest inconsistency in the docstrings is that they
> mention the parent without having an attribute for it. (E.g., the docstring
> for setServiceParent *says* "parent attribute is set", but there's no
> "parent attribute" in the interface. Glyph, if you want to open a ticket
> with the issues you found, please assign it to me -- otherwise, more
> specificity is useful!
> [2a] Since the docstrings do refer to the parent attribute, and since I
> would be surprised if services just implemented the interface without
> inheriting from "Service", I wonder if people object *too* much if I "fix"
> it by documenting the parent attribute as part of the interface?
> [3] I think that with the compose() function available in .tac files,
> maybe needing .tac *with* subcommands is not quite as important? Maybe I'll
> leave that off of the initial sprint.
>
> So bottom line, current plans:
> [4] Implement compose() as a function that takes an iterable-of-iterables,
> treats them as command-line arguments for services, and builds a
> MultiService() with them all.
> [5] Document the parent attribute in IService (and possibly other issues,
> depending on Glyph opening a bug for [2]
> [6] Implement provides() the way I wanted to (now that parent is
> documented)
>
> Since it seems like compose-as-a-function is the least controversial bit
> here, I'll start with that while the other issues settle down.
>
> Moshe Z.
>
>
> On Wed, May 20, 2015 at 5:18 PM <exarkun at twistedmatrix.com> wrote:
>
>> On 07:29 pm, glyph at twistedmatrix.com wrote:
>> >
>> >>On May 20, 2015, at 04:13, exarkun at twistedmatrix.com wrote:
>> >>
>> >>On 19 May, 09:01 pm, tom.prince at ualberta.net wrote:
>> >>>Glyph <glyph at twistedmatrix.com> writes:
>> >>>>You could also find some other way to split the argument list but
>> >>>>"+" doesn't seem particularly obscure in this context to me.  (If
>> >>>>there's really a need to pass a literal "+" to a plugin we could add
>> >>>>an escaping syntax as well.)
>> >>>
>> >>>I think if we are adding syntax, then we should also add escaping at
>> >>>the
>> >>>same time.
>> >>>
>> >>>On a related note, when designing this kind of syntax, I think it is
>> >>>often valuable to explictly leave some of the space as an explict
>> >>>error,
>> >>>to leave freedom to extend the syntax in the future.
>> >>
>> >>I think this is 100% correct.  This is part of why I don't want the
>> >>syntax added to `Options`.  If it's a feature of, say, a "compose"
>> >>twistd plugin then you can always throw the whole "compose" twistd
>> >>plugin in the trash and start again.  That gives you quite a lot of
>> >>space for syntax changes. :)
>> >>
>> >>(And of course, not introducing a syntax at all leaves you even more
>> >>room... but talking this crowd out of inventing weird syntaxes is
>> >>probably an exercise in futility.)
>> >
>> >Much as I love weird syntaxes, using "+" as a separator for a 'compose'
>> >plugin seems about as straightforward as I could imagine such a thing
>> >working.  Do you have an alternate proposal that is less 'weird'?  I
>> >wouldn't promise to accept it but I wouldn't want to go with something
>> >unnecessarily weird just due to a failure of my imagination.
>>
>> It's possible this is a case of necessary weirdness - at least, if
>> you're dead-set on some kind of option-based CLI interface to this
>> functionality.
>>
>> Think about other CLI tools that try to offer this kind of composition.
>> Only two really come to mind for me and one of those is not exactly well
>> regarded for its novel syntax (hint - it rhymes with "schmestreamer).
>>
>> This is not to say that this kind of composition is a bad thing - or
>> even that no other widely-used software supports the kind of invoke-
>> several-things-of-stuff behavior that we're talking about providing
>> access to here.
>>
>> However, I do think it is the case that most of the world tries to solve
>> this problem without trying to force everything into a world-view framed
>> by GNU getopt().
>>
>> With that in mind, here are some alternatives:
>>
>>   (0) The null proposal
>>
>>     $ twistd compose \
>>         web --port 80 +
>>         manhole --auth foo + \
>>         logging --path /var/run
>>
>>   (1) Borrow `--` which already means "stop parsing options here".  It
>> makes sense if you think about sub-commands as existing on a stack and
>> `--` meaning "pop the stack".  Possibly not actually feasible since I
>> think `Options` already knows what `--` means (by virtue of actually
>> *using* getopt()) and uses it to separate options from positional
>> arguments (and therefore it's not an unambiguous separator - but I
>> haven't double-checked this).
>>
>>     $ twistd compose \
>>         web --port 80 -- \
>>         manhole --auth foo -- \
>>         logging --path /var/run
>>
>>   (2) Put things into strings which already have well-defined escape
>> syntax:
>>
>>     $ twistd compose \
>>         'web --port 80' \
>>         'manhole --auth foo' \
>>         'logging --path /var/run'
>>
>>   (3) Load things from a file (really just a variation on (2)):
>>
>>     $ cat myapp.compose
>>     web --port 80
>>     manhole --auth foo
>>     logging --path /var/run
>>     $ twistd compose myapp.compose
>>
>> Note these aren't really syntax ideas.  Apart from (1), they're ideas
>> for not needing a new syntax.  As soon as you actually invent a new
>> syntax you invite comments like "Well, maybe you should use `;` instead
>> of `+` because `;` is already kind of ingrained as a separator and
>> people are used to having to escape it" (which, see, I put in quotes,
>> because I'm totally not saying that, even if it is totally true) to
>> which you can only really reply that your gut agrees or your gut
>> disagrees and it's always a bummer when the best support you have for
>> some programming decision is some gut feeling you had.
>>
>> I might actually be somewhat attracted to (3) because:
>>
>>   * I don't really want to build up giant piles of configuration on my
>> command line and then rely on my shell history for being able to find it
>> again.  myapp.compose is something I could even put in my version
>> control system.
>>
>>   * Despite shell quoting and escaping rules being perfectly well
>> defined, I'd really much rather not have to deal with an automatic extra
>> layer of quotes.
>>
>> And a last point to consider about new syntaxes - what are the security
>> considerations of a command like:
>>
>>     $ twistd compose \
>>         web --port ${WEB_PORT} +
>>         manhole --auth foo + \
>>         logging --path /var/run
>>
>> I'll grant there's a bug there (the shell variable should be quoted -
>> but I only know one person who consistently gets that right).
>>
>> Regardless, I think any proposal would greatly benefit from a much
>> greater consideration of use-cases and examples than I've yet seen
>> discussed in this thread.
>>
>> Jean-Paul
>>
>> _______________________________________________
>> Twisted-Python mailing list
>> Twisted-Python at twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20150521/1828ba01/attachment-0002.html>