[Twisted-Python] suggestions sought on multi-service application dealing with shared database issue

[Jonathan Vanasco]

I have a few ideas on how to handle this, but wanted to see if anyone else has dealt with this and has a recommended approach:

I have a twisted daemon that I will loosely describe as this:

	* an application defined in a `.tac` file that is an instance of `twisted.application.service.Application`
	* multiple user-defined services (IIRC 8 or 9) that are subclasses as twisted.application.internet.TimeService
	* the services are added to the application via `setServiceParent`
	* the reactor is started via `twisted.internet.reactor.run()`

I've run into an annoying issue a few times, and need to temporarily suspend all/most of the services from running...  an internal backup script occasionally breaks (on a new edge case every time we fix it) and ends up consuming 100% of disk space.  This cascades into database failures in my application.

I've got an external service monitoring disk-space and alerting me.  It's presently shutting down the twisted daemon when the error happens, but I'd like to keep twisted running and just adapt to the server conditions.

The two ideas I had so far:

1. Create a new service that monitors server conditions and can call stopService/startService on registered services as needed
2. Keep all the services running, but put a conditional check in the timer service's callable

has anyone preferred one of these, or something different, in the past?