Opened 10 years ago

Last modified 5 years ago

#4072 enhancement new

twistd autoreload feature during development

Reported by: miracle2k Owned by:
Priority: normal Milestone:
Component: core Keywords: twistd
Cc: Patrick Cloke Branch:
Author:

Description

I wonder if it would be feasible/possible to support a kind of "autoreload" in twistd, similar to how the development servers of various web frameworks do this (say, Django's runserver).

Change History (6)

comment:1 Changed 10 years ago by Jean-Paul Calderone

I took a look at Django's auto-reloader. One interesting thing I noticed about it is that it has no Django dependency. I think it should work equally well for any Python program - including twistd. Unfortunately things are never really that easy. It wants to run things in a thread for some reason, which upsets reactor.run(). It also doesn't work as a nice simple command line wrapper, you have to write something like this:

$ cat autoreload-twistd.py
#!/usr/bin/python
import autoreload
import twisted.scripts.twistd as t
import sys
autoreload.main(t.run)

Still, this is pretty close, I think. It would be interesting to come up with a completely agnostic reloader that could be used by Django and Twisted (and anyone else).

I feel like this should be a project developed outside Twisted to ensure that other projects are comfortable adopting it.

comment:2 Changed 10 years ago by miracle2k

Thanks, I didn't realize it was that straight forward.

I played around with it a bit now, and I get a "ValueError: signal only works in main thread" error; not sure if those were the thread-related reactor issues you've been referring to.

When the reactor is run in the main thread, you have the problem that a sys.exit() call is caught by it. There doesn't seem to be way to stop the reactor and cause twistd to exit with a specific code, which is how autoreload currently does things. Not sure if there is a good way to provide that option, or how useful it would be.

One option might be the script restarting itself once it detects a code change, i.e. first calling os.spawnve(), then reactor.stop(), but that seems hacky, and you need to rely in sys.argv not having changed, as well as the shutdown happening quickly enough so you don't have the new process requiring some still locked resource.

FWIW, what I am doing now is using a modified autoreload.py which runs the code_changed checker in a thread, and calls os._exit, rather than sys.exit (which won't work in a thread, of course). That means there's no clean shutdown, which I'm not really happy about, but it works well enough for me.

comment:3 Changed 10 years ago by Jean-Paul Calderone

There's a couple older ticket, #761 and #2182, for being able to specify the exit status of twistd somehow. If those were resolved, it sounds like you might be in pretty good shape.

comment:4 Changed 9 years ago by Jean-Paul Calderone

Keywords: twistd added

comment:5 Changed 9 years ago by <automation>

Owner: Glyph deleted

comment:6 Changed 5 years ago by Patrick Cloke

Cc: Patrick Cloke added
Note: See TracTickets for help on using tickets.