[Twisted-Python] What to do when a service fails to start, also, deferred and startService

[Terry Jones]

I've another of my pesky beginner questions. Note that this subject is
somewhat covered in the thread started by Matt Goodall in Jan 2006:

  http://twistedmatrix.com/pipermail/twisted-python/2006-January/012380.html


I imagine that it must be common that people write services that don't just
simply launch things listening on sockets, but instead need to do a couple
of things, one after another, in order to get going and to be ready to
provide their service (or multiservice).

If you do need to write something like that, it seems the chances are
pretty high you're going to be calling code somewhere along the way that
returns a deferred. And because the twisted/application/service.py code
that calls startService doesn't handle deferreds being returned, this
creates a real problem. At least as far as I understand things - which, as
usual, may not be very far.

If nothing goes wrong with the deferreds that startService is creating (via
whatever its calling), then you'll probably get away with things even
though your service will not really be up until after the deferreds fire,
which can be some time after the code calling startService gets its
deferred back (and ignores it).

But if something does go wrong, you've got a failure propagating its way
down a errback chain, eventually (unless an errback switches you back to
the callback chain) popping out the end and causing the reactor to issue an
Unhandled Error message. So you can't indicate that the service has failed
to start by throwing, because the exception is going to pop harmlessly out
the end of the deferred chain as a generic unhandled error and will not
cause Twisted to know that the service couldn't start.

This all feels quite ironic :-) Twisted leads you coyly into the dark and
powerful world of working with and heavily depending on Deferreds. But
then, right when you expect it to be there for you, covering your back, it
throws up its hands as if to say "What!!? You expect me to deal with you
returning a Deferred? You gotta be kidding, sucker."

I could follow Moof's approach (last poster in the above thread), but that
seems to just pass the problem on to a higher level, where something else
is calling startService (or something earlier) and so on up until we reach
the topmost point at which something is not allowing/expecting a deferred
to come back. Should I track down and subclass all these things?  That
would seem cruel and unusual punishment to the faithful Deferred user,
having to go in and subclass core classes because they don't deal with
Deferreds.

I could do something dramatic, like call reactor.stop or sys.exit in my
errback chain, but those seem completely wrong. Apart from the (remote?)
possibility that something other than Twisted plugin code is trying to
start my service, it's also anachronistic because it will happen at some
unpredictable time after startService has gotten back (and ignored) the
deferred and Twisted has moved on (perhaps even to start other services).

Terry