[Twisted-Python] supporting start/stop/restart behavior

[Glyph Lefkowitz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wednesday, March 19, 2003, at 07:58 PM, Clark C. Evans wrote:

> ... what [Twisted] misses, IMHO, is a simple interface to stop and 
> restart.

You're right, but this is the wrong way to go about fixing it.

> Also, it lacks a nice way to be embedded within a python script so 
> that the __main__ == __name__ hack runs the file properly.

I disagree.  Despite the fact that I occasionally write quick scripts 
that work this way, it's for quick hacks only, and providing better 
support for it is a misleading band-aid on the deficiencies of 
multi-platform twistd functionality.

> And add these three lines of code to make it runnable...
>
>     if '__main__' == __name__:
>         from twisted.scripts.twistd import runDirect
>         runDirect(save=1)
>
> In this way, the following commands 'just work':
>
>     python test.py          # runs the app directly (no daemon)
>     python test.py start    # runs the app as a deamon
>     python test.py stop     # stops the app as a deamon
>     python test.py restart  # restarts the app as a deamon

... on UNIX.

But, what does "restart" mean if you're running on Jython?  On win32?  
What interaction does this imply with native OS services?

> And, even twistd options can be included...
>
>     python test.py --quiet start
>     python test.py --quiet

In that case, why not 'twistd -y test.py --quiet'?

>   1. Refactoring runApp to remove a hunk of code which
>      runs os.kill on a pid found in the given pidfile.
>      This is moved to another function, killApp, which
>      takes two arguments, config and signal.   From the
>      runApp the code then calls killApp(config, signal=0)

Currently, that code is there _only_ to determine, in a rather 
platform-specific way, whether the server should attempt to keep 
starting or not.  It's not a generalized stop-the-application.

>   2. Adding a new function, runDirect which does
>      several things:
>
>      b.  it adds --python <filename> where filename is the
>          name of the current python file (argv[0]), further,
>          it gives a direct option in the function arguments
>          to not save the tap file (saving a tap file is
>          not always useful, and as an argument it is hard
>          to miss); and

Saving a tap file is pretty much always useful.  It provides a common 
ground for introspection tools to look at what a server is doing.  I 
have gone over this in previous mails.

>      c.  if stop or restart are chosen, then this kills the
>          current process using the given pidfile (which defaults
>          to twistd.pid in the current directory) through
>          killApp(signal=SIGTERM)

Why put this into Twisted where we have to swaddle it in a layer of 
cross-platform compatibility rather than just letting the user use 
whatever tools are available on their system for interacting with 
Twisted?

>      This function could be broken into the start/stop/restart
>      behavior from the --python option; but I don't have this
>      requirement and people in the IRC list didn't seem to think
>      that either of these two functions are useful.
>
> Anyway, I'm posting to this list beacuse I think the way in
> which twistd is used isn't obvious... i.e. the primary way
> you'd use it with a python source file is an option buried
> among many others.

So you are trying to solve a documentation problem by adding 
functionality to a different place, necessitating more documentation? 
:-)

> I raised this change on the IRC list, and the primary argument
> against the above was that it is out-of-scope; in other words,
> one could write a shell script to do the same.   I think this
> is a bad argument beacuse _all_ of twistd could be done in
> a shell script.  For example, if you ask D. J. Bernstein, he'd
> say that none of this deamon / logging behavior needs to be
> in an application, and this is why he has his 'daemontools'.

D.J. Bernstein doesn't give a rip about portability outside of UNIX, so 
he makes the decision to write his framework for logging and 
daemonization in sh.  We write these frameworks in Python, but I agree 
with this point: none of this daemon/logging/startup/shutdown behavior 
needs to be in an application.

My view on functionality like this is that it is a platform-specific 
thing.  Even assuming that signals worked like you expect them to 
across all platforms that we are trying to support, there are other 
issues.  What about the cultural expectation of command lines?  On 
win32, the way Python programs are run isn't very natural.  On the 
command line, the location of the .exe is significant, there's no such 
thing as a shebang line, and users aren't accustomed to running 
interpreters on scripts.  They expect to have an icon to click on, 
usually a Service.  There are also differing cultural expectations of 
how you run stuff like this on MacOS.  Does the start/stop thing have a 
GUI?  A monitoring icon in the dock?

Now, we don't *currently* have functionality like that, but we hope to 
one day.  I am beginning to despair of ever actually achieving that 
goal, though, since nobody who understands it has time, and so many 
other programmers have habits that are opposed to this approach...

Supporting mechanisms in the framework that make configuration and 
introspection impossible without editing the start-up code for your 
particular application would make this eventual goal even more 
difficult than it already is.

Components written for twisted should be written in such a way that 
they can be loaded into existing applications without running a script. 
  The --python option to twistd is pushing about as far as we want to go 
in the direction of hard-coding custom functionality for a particular 
server process.

> So, the argument shouldn't be _can_ this be done externally,
> of course it can.  The question is does it make sence to
> standardize on particular ways of using twisted so that everyone
> doesn't have their own approach, to logging, etc.  I feel that
> this comes all the way down to Twisted having a simple way
> to start/stop/restart a server without having to rely upon
> external, non-Twisted scripts.

If you want to improve twistd, please feel free to suggest ways that 
stopping/restarting could be made easier or even more consistent across 
platforms.  For example, "killApp" might make sense if, on 
signals-challenged platforms, twistd automatically opened a 
localhost-only socket which allowed communication with the server.

> At the very least, I'd like the killApp refactor to be
> accepted, as I'd rather not have to duplicate killApp
> code within a private/sandbox copy of runDirect.

What functionality, exactly, is your platform's kill(1) missing that 
makes you want a Python version of it?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (Darwin)

iD8DBQE+eVMwvVGR4uSOE2wRAopTAJ4+JQiM0WrJYIAH2qPMCbil4bE1+gCeMRaF
X5qxxeDWi10E8IeWcuBkQE4=
=L2sp
-----END PGP SIGNATURE-----