[Twisted-Python] Multiple plugins in "twistd"

Moshe Zadka zadka.moshe at gmail.com
Wed May 20 20:31:41 MDT 2015


Hi glyph and exarkun,

[1] The compose syntax is the least interesting part for me here, and seems
to cause the most bike-shedding, so I think I will punt on that. Instead, I
will write a function called "compose()" which can be used from .tac files
or possibly other plug-ins, leaving the issue of the syntax for a different
day.
[2] It seems like the biggest inconsistency in the docstrings is that they
mention the parent without having an attribute for it. (E.g., the docstring
for setServiceParent *says* "parent attribute is set", but there's no
"parent attribute" in the interface. Glyph, if you want to open a ticket
with the issues you found, please assign it to me -- otherwise, more
specificity is useful!
[2a] Since the docstrings do refer to the parent attribute, and since I
would be surprised if services just implemented the interface without
inheriting from "Service", I wonder if people object *too* much if I "fix"
it by documenting the parent attribute as part of the interface?
[3] I think that with the compose() function available in .tac files, maybe
needing .tac *with* subcommands is not quite as important? Maybe I'll leave
that off of the initial sprint.

So bottom line, current plans:
[4] Implement compose() as a function that takes an iterable-of-iterables,
treats them as command-line arguments for services, and builds a
MultiService() with them all.
[5] Document the parent attribute in IService (and possibly other issues,
depending on Glyph opening a bug for [2]
[6] Implement provides() the way I wanted to (now that parent is documented)

Since it seems like compose-as-a-function is the least controversial bit
here, I'll start with that while the other issues settle down.

Moshe Z.


On Wed, May 20, 2015 at 5:18 PM <exarkun at twistedmatrix.com> wrote:

> On 07:29 pm, glyph at twistedmatrix.com wrote:
> >
> >>On May 20, 2015, at 04:13, exarkun at twistedmatrix.com wrote:
> >>
> >>On 19 May, 09:01 pm, tom.prince at ualberta.net wrote:
> >>>Glyph <glyph at twistedmatrix.com> writes:
> >>>>You could also find some other way to split the argument list but
> >>>>"+" doesn't seem particularly obscure in this context to me.  (If
> >>>>there's really a need to pass a literal "+" to a plugin we could add
> >>>>an escaping syntax as well.)
> >>>
> >>>I think if we are adding syntax, then we should also add escaping at
> >>>the
> >>>same time.
> >>>
> >>>On a related note, when designing this kind of syntax, I think it is
> >>>often valuable to explictly leave some of the space as an explict
> >>>error,
> >>>to leave freedom to extend the syntax in the future.
> >>
> >>I think this is 100% correct.  This is part of why I don't want the
> >>syntax added to `Options`.  If it's a feature of, say, a "compose"
> >>twistd plugin then you can always throw the whole "compose" twistd
> >>plugin in the trash and start again.  That gives you quite a lot of
> >>space for syntax changes. :)
> >>
> >>(And of course, not introducing a syntax at all leaves you even more
> >>room... but talking this crowd out of inventing weird syntaxes is
> >>probably an exercise in futility.)
> >
> >Much as I love weird syntaxes, using "+" as a separator for a 'compose'
> >plugin seems about as straightforward as I could imagine such a thing
> >working.  Do you have an alternate proposal that is less 'weird'?  I
> >wouldn't promise to accept it but I wouldn't want to go with something
> >unnecessarily weird just due to a failure of my imagination.
>
> It's possible this is a case of necessary weirdness - at least, if
> you're dead-set on some kind of option-based CLI interface to this
> functionality.
>
> Think about other CLI tools that try to offer this kind of composition.
> Only two really come to mind for me and one of those is not exactly well
> regarded for its novel syntax (hint - it rhymes with "schmestreamer).
>
> This is not to say that this kind of composition is a bad thing - or
> even that no other widely-used software supports the kind of invoke-
> several-things-of-stuff behavior that we're talking about providing
> access to here.
>
> However, I do think it is the case that most of the world tries to solve
> this problem without trying to force everything into a world-view framed
> by GNU getopt().
>
> With that in mind, here are some alternatives:
>
>   (0) The null proposal
>
>     $ twistd compose \
>         web --port 80 +
>         manhole --auth foo + \
>         logging --path /var/run
>
>   (1) Borrow `--` which already means "stop parsing options here".  It
> makes sense if you think about sub-commands as existing on a stack and
> `--` meaning "pop the stack".  Possibly not actually feasible since I
> think `Options` already knows what `--` means (by virtue of actually
> *using* getopt()) and uses it to separate options from positional
> arguments (and therefore it's not an unambiguous separator - but I
> haven't double-checked this).
>
>     $ twistd compose \
>         web --port 80 -- \
>         manhole --auth foo -- \
>         logging --path /var/run
>
>   (2) Put things into strings which already have well-defined escape
> syntax:
>
>     $ twistd compose \
>         'web --port 80' \
>         'manhole --auth foo' \
>         'logging --path /var/run'
>
>   (3) Load things from a file (really just a variation on (2)):
>
>     $ cat myapp.compose
>     web --port 80
>     manhole --auth foo
>     logging --path /var/run
>     $ twistd compose myapp.compose
>
> Note these aren't really syntax ideas.  Apart from (1), they're ideas
> for not needing a new syntax.  As soon as you actually invent a new
> syntax you invite comments like "Well, maybe you should use `;` instead
> of `+` because `;` is already kind of ingrained as a separator and
> people are used to having to escape it" (which, see, I put in quotes,
> because I'm totally not saying that, even if it is totally true) to
> which you can only really reply that your gut agrees or your gut
> disagrees and it's always a bummer when the best support you have for
> some programming decision is some gut feeling you had.
>
> I might actually be somewhat attracted to (3) because:
>
>   * I don't really want to build up giant piles of configuration on my
> command line and then rely on my shell history for being able to find it
> again.  myapp.compose is something I could even put in my version
> control system.
>
>   * Despite shell quoting and escaping rules being perfectly well
> defined, I'd really much rather not have to deal with an automatic extra
> layer of quotes.
>
> And a last point to consider about new syntaxes - what are the security
> considerations of a command like:
>
>     $ twistd compose \
>         web --port ${WEB_PORT} +
>         manhole --auth foo + \
>         logging --path /var/run
>
> I'll grant there's a bug there (the shell variable should be quoted -
> but I only know one person who consistently gets that right).
>
> Regardless, I think any proposal would greatly benefit from a much
> greater consideration of use-cases and examples than I've yet seen
> discussed in this thread.
>
> Jean-Paul
>
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20150521/cc7436b3/attachment-0002.html>


More information about the Twisted-Python mailing list