[Twisted-Python] Twisted superservers and setuid/setgid subservers

Andrew Bennetts andrew-twisted at puzzling.org
Sat Aug 31 06:02:46 EDT 2002

Here's some ideas I've been thinking about with aid of my housemate (who is

A feature of some servers is the ability to run chroot'd, so say you want to
be a WebDAV server for a site, when a user logs in, it forks, chroots to the
users home directory, and sets its uid/gid to that of the user, thereby
restricting the potential for security holes to damage the system.  (The
chroot is an optional paranoid extra if you are dropping uid/gid)

Consider twisted.web.distrib (I might have bits of this slightly wrong;
please correct me if so).  It currently does something like this, although
manually: each user has to start their own twistd, which only runs
twisted.web, and talks to the superserver with PB via a pipe.  This is good,
but not as general as it could be.  It's limited to twisted.web, and users
have to manually ensure that twistd is running.

I think it'd be nice to generalise this to *any* service that can run within
the context of different users, e.g. for FTP, WebDAV, whatever.  So there'd
be some automated way to tell a Twisted application "Ok, now serve this as a
particular user by passing it to a user-owned twistd".

What I'm envisaging is something like this:
  * The superserver receives a WidgetService request from user Bob
  * The superserver connects to the user's WidgetService twistd, starting it
    if necessary (more on this in a moment)
  * The user's WidgetService then does all the hard work :)

If the user doesn't have a currently running twistd, then you could do, say:
  * The superserver creates a pipe
  * The superserver (which is running as root (or something effectively
    root)) forks a child
  * The child does a reactor.crash() to stop it interfering with the
    parent's normal handling of events
  * The child closes all file descriptors in the reactor, except for the
    pipe.  It should probably start logging to a different log file as well.
    Also, things like DB-backed twisted.cred authenticators probably should
    be shutdown... hmm, this part is messy :(
  * The child then starts a fresh Application, which runs the user's
    WidgetService and starts processing for the superserver

Now that I think about it, it is probably cleaner to simply spawn a fresh
process rather than futzing around with fork, so just:
  * The superserver spawns a user twistd as a particular uid/gid, and talks
    to it over a PB pipe.


The nice thing about this model is that there is only 1 twistd running per
user (and you could get the user twistds to automatically shutdown if
inactive for, say, 10 minutes to free resources), which means your process
load is still mostly independent of the number of connections.  It'd be
*really* cool if a superserver listening on multiple protocols, e.g.
"WidgetService" and HTTP, could have only a *single* twistd per user,
regardless of number of protocols the superserver is dealing with.

It'd be really nice to be able to chroot as well, but I suspect python
doesn't work cleanly with chroot, due to file descriptors being left open to

So yeah, this is a long-winded way of saying "twisted.web.distrib is cool,
but should be general and more automatic", because I think this would
probably be useful.

In vaguely related news, I've nearly got a simple Twisted inetd replacement
written...  expect a checkin sometime this weekend.


More information about the Twisted-Python mailing list