[Twisted-Python] Multiple plugins in "twistd"

Wed May 20 21:39:51 MDT 2015

FYIFYI --  Patch attached. Docs are missing, but I'm sure JP/Glyph will be
happy to tell me what I've done wrong otherwise :-P

On Wed, May 20, 2015 at 7:40 PM Moshe Zadka <zadka.moshe at gmail.com> wrote:

> FYI, https://twistedmatrix.com/trac/ticket/7907#ticket created.
>
> On Wed, May 20, 2015 at 7:31 PM Moshe Zadka <zadka.moshe at gmail.com> wrote:
>
>> Hi glyph and exarkun,
>>
>> [1] The compose syntax is the least interesting part for me here, and
>> seems to cause the most bike-shedding, so I think I will punt on that.
>> Instead, I will write a function called "compose()" which can be used from
>> .tac files or possibly other plug-ins, leaving the issue of the syntax for
>> a different day.
>> [2] It seems like the biggest inconsistency in the docstrings is that
>> they mention the parent without having an attribute for it. (E.g., the
>> docstring for setServiceParent *says* "parent attribute is set", but
>> there's no "parent attribute" in the interface. Glyph, if you want to open
>> a ticket with the issues you found, please assign it to me -- otherwise,
>> more specificity is useful!
>> [2a] Since the docstrings do refer to the parent attribute, and since I
>> would be surprised if services just implemented the interface without
>> inheriting from "Service", I wonder if people object *too* much if I "fix"
>> it by documenting the parent attribute as part of the interface?
>> [3] I think that with the compose() function available in .tac files,
>> maybe needing .tac *with* subcommands is not quite as important? Maybe I'll
>> leave that off of the initial sprint.
>>
>> So bottom line, current plans:
>> [4] Implement compose() as a function that takes an
>> iterable-of-iterables, treats them as command-line arguments for services,
>> and builds a MultiService() with them all.
>> [5] Document the parent attribute in IService (and possibly other issues,
>> depending on Glyph opening a bug for [2]
>> [6] Implement provides() the way I wanted to (now that parent is
>> documented)
>>
>> Since it seems like compose-as-a-function is the least controversial bit
>> here, I'll start with that while the other issues settle down.
>>
>> Moshe Z.
>>
>>
>> On Wed, May 20, 2015 at 5:18 PM <exarkun at twistedmatrix.com> wrote:
>>
>>> On 07:29 pm, glyph at twistedmatrix.com wrote:
>>> >
>>> >>On May 20, 2015, at 04:13, exarkun at twistedmatrix.com wrote:
>>> >>
>>> >>On 19 May, 09:01 pm, tom.prince at ualberta.net wrote:
>>> >>>Glyph <glyph at twistedmatrix.com> writes:
>>> >>>>You could also find some other way to split the argument list but
>>> >>>>"+" doesn't seem particularly obscure in this context to me.  (If
>>> >>>>there's really a need to pass a literal "+" to a plugin we could add
>>> >>>>an escaping syntax as well.)
>>> >>>
>>> >>>I think if we are adding syntax, then we should also add escaping at
>>> >>>the
>>> >>>same time.
>>> >>>
>>> >>>On a related note, when designing this kind of syntax, I think it is
>>> >>>often valuable to explictly leave some of the space as an explict
>>> >>>error,
>>> >>>to leave freedom to extend the syntax in the future.
>>> >>
>>> >>I think this is 100% correct.  This is part of why I don't want the
>>> >>syntax added to `Options`.  If it's a feature of, say, a "compose"
>>> >>twistd plugin then you can always throw the whole "compose" twistd
>>> >>plugin in the trash and start again.  That gives you quite a lot of
>>> >>space for syntax changes. :)
>>> >>
>>> >>(And of course, not introducing a syntax at all leaves you even more
>>> >>room... but talking this crowd out of inventing weird syntaxes is
>>> >>probably an exercise in futility.)
>>> >
>>> >Much as I love weird syntaxes, using "+" as a separator for a 'compose'
>>> >plugin seems about as straightforward as I could imagine such a thing
>>> >working.  Do you have an alternate proposal that is less 'weird'?  I
>>> >wouldn't promise to accept it but I wouldn't want to go with something
>>> >unnecessarily weird just due to a failure of my imagination.
>>>
>>> It's possible this is a case of necessary weirdness - at least, if
>>> you're dead-set on some kind of option-based CLI interface to this
>>> functionality.
>>>
>>> Think about other CLI tools that try to offer this kind of composition.
>>> Only two really come to mind for me and one of those is not exactly well
>>> regarded for its novel syntax (hint - it rhymes with "schmestreamer).
>>>
>>> This is not to say that this kind of composition is a bad thing - or
>>> even that no other widely-used software supports the kind of invoke-
>>> several-things-of-stuff behavior that we're talking about providing
>>> access to here.
>>>
>>> However, I do think it is the case that most of the world tries to solve
>>> this problem without trying to force everything into a world-view framed
>>> by GNU getopt().
>>>
>>> With that in mind, here are some alternatives:
>>>
>>>   (0) The null proposal
>>>
>>>     $ twistd compose \
>>>         web --port 80 +
>>>         manhole --auth foo + \
>>>         logging --path /var/run
>>>
>>>   (1) Borrow `--` which already means "stop parsing options here".  It
>>> makes sense if you think about sub-commands as existing on a stack and
>>> `--` meaning "pop the stack".  Possibly not actually feasible since I
>>> think `Options` already knows what `--` means (by virtue of actually
>>> *using* getopt()) and uses it to separate options from positional
>>> arguments (and therefore it's not an unambiguous separator - but I
>>> haven't double-checked this).
>>>
>>>     $ twistd compose \
>>>         web --port 80 -- \
>>>         manhole --auth foo -- \
>>>         logging --path /var/run
>>>
>>>   (2) Put things into strings which already have well-defined escape
>>> syntax:
>>>
>>>     $ twistd compose \
>>>         'web --port 80' \
>>>         'manhole --auth foo' \
>>>         'logging --path /var/run'
>>>
>>>   (3) Load things from a file (really just a variation on (2)):
>>>
>>>     $ cat myapp.compose
>>>     web --port 80
>>>     manhole --auth foo
>>>     logging --path /var/run
>>>     $ twistd compose myapp.compose
>>>
>>> Note these aren't really syntax ideas.  Apart from (1), they're ideas
>>> for not needing a new syntax.  As soon as you actually invent a new
>>> syntax you invite comments like "Well, maybe you should use `;` instead
>>> of `+` because `;` is already kind of ingrained as a separator and
>>> people are used to having to escape it" (which, see, I put in quotes,
>>> because I'm totally not saying that, even if it is totally true) to
>>> which you can only really reply that your gut agrees or your gut
>>> disagrees and it's always a bummer when the best support you have for
>>> some programming decision is some gut feeling you had.
>>>
>>> I might actually be somewhat attracted to (3) because:
>>>
>>>   * I don't really want to build up giant piles of configuration on my
>>> command line and then rely on my shell history for being able to find it
>>> again.  myapp.compose is something I could even put in my version
>>> control system.
>>>
>>>   * Despite shell quoting and escaping rules being perfectly well
>>> defined, I'd really much rather not have to deal with an automatic extra
>>> layer of quotes.
>>>
>>> And a last point to consider about new syntaxes - what are the security
>>> considerations of a command like:
>>>
>>>     $ twistd compose \
>>>         web --port ${WEB_PORT} +
>>>         manhole --auth foo + \
>>>         logging --path /var/run
>>>
>>> I'll grant there's a bug there (the shell variable should be quoted -
>>> but I only know one person who consistently gets that right).
>>>
>>> Regardless, I think any proposal would greatly benefit from a much
>>> greater consideration of use-cases and examples than I've yet seen
>>> discussed in this thread.
>>>
>>> Jean-Paul
>>>
>>> _______________________________________________
>>> Twisted-Python mailing list
>>> Twisted-Python at twistedmatrix.com
>>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20150521/b54f8dd5/attachment-0002.html>