Zsh tab completion is a bit too slow to use

[Julian]

It would appear from a quick look that most of the time is likely spent spinning up Python and searching PYTHONPATH. For me on this (albeit a bit modest and old) machine, that's a full second or two of a wait time.

I dunno what a good solution is considering searching the PYTHONPATH is certainly a nice thing. Perhaps what'd be a nice compromise would be not doing that if there's a match in the cwd?

[Julian]

Ignore the issue description text, it's not trial specific. The slowness is from spinning up python to find twisted's install location.

[teratorn]

I'm a little bit skeptical that spinning up Python accounts for 1 or 2 seconds of CPU time even on a slow machine.

I've tested this quite a bit when originally developing the (VERY FAST, but deprecated) completion system (zshcomp.py).

The new system, a private module (twisted/python/_shellcomp.py) implements much more advanced real-time completion by generating zsh shell code on stdout and having your shell interpret that to generate completion options.

I've found starting a raw Python interpreter (repl) to be nominal compared to the cost of loading Twisted modules (all of the top level modules that implement t.p.usage.Options subclasses for twistd plugins) and templating a shell code generate completions for the discovered options is QUITE slow.

And actually parsing the Options and generating (templating) the zsh completion function. This IS quite slow I believe, and probably could be optimized somewhere.

HOPEFULLY, the (now extensive) unit-test coverage should help you not break everything, as you would be almost sure to do otherwise!

Doing these things every tab-press is a pain on slow machines. On my newer faster machines, I *can* use the completion system and not feel bad, but I definitely agree it is TOO slow!

I would suggest two approaches.

• Profile the _shellcomp code (simulating tab press by calling Options.parseOptions() with ["test_command", "--_shell-completion", "zsh:2"] as the arguments list. This could be a benchmark on speed.twistedmatrix.com even, and might even be representative of general python string munging tasks.

• Implement a caching strategy for the ZSH tab-completion functions and/or data.
  • There are two places where data may be cached:
    • On disk ($HOME or $TMP, I suppose)
    • In memory (in user's zsh shell session). On disk might just be good enough, since the OS disk cache will pick up the file after the first access, and should keep it in memory during an interactive zsh session.

Otherwise it is not difficult to export variables in to the user's shell. We can add code to do that to the shell-completion function that is generated, or directly in the stub function. The stub-function's code is NOT tested directly, I don't think, since it is a VERY thin wrapper over the Python code, and at the time I couldn't imagine how to test it easily and cleanly.

[teratorn]

(In [36615]) Branching to 'faster-tab-completion-5761'