[Twisted-Python] Graceful shutdown

Jonas Lindmark jonas.lindmark at gmail.com
Wed Aug 28 09:20:54 MDT 2013


I'm trying to implement graceful shutdown of a HTTP server and I am unsure
of the preferred way of implementing it.

I'm attempting to do this by adding a system event trigger for "before"
"shutdown" that should stop accepting new requests and wait for current
ones to finish.

My problem is that though my requests has fired their notifyFinish()
deferreds the data has not been written to the client. Calling
reactor.getWriters() shows me that there are active writers who are
probably writing/flushing data to the client.

What is the preferred way of waiting for the requests to finish and finish
writing the response to the client?

Here is example code for my own attempt:

from twisted.internet import reactor, defer
from twisted.application import internet, service
from twisted.web.server import Site, NOT_DONE_YET
from twisted.web.resource import Resource

class SlowResource(Resource):
    isLeaf = True
    waiting_requests = []

    def notify_no_more_waiting(self):
        if not self.waiting_requests:
            return defer.succeed(None)
        return defer.gatherResults(self.waiting_requests, consumeErrors=True) \
                .addBoth(lambda ign: None)

    def write_result(self, request):

    def render_GET(self, request):
        reactor.callLater(5, self.write_result, request)

        d = request.notifyFinish()
        d.addBoth(lambda ign: self.waiting_requests.remove(d))

        return NOT_DONE_YET

slow_resource = SlowResource()
site = Site(slow_resource)
application = service.Application("MyApp")
server = internet.TCPServer(8080, site)

def wait_for_writers():
    d = defer.Deferred()

    def check_writers():
        if len(reactor.getWriters()) > 0:
            reactor.callLater(0.1, check_writers)


    return d

def graceful_shutdown():
    yield server.stopService()
    yield slow_resource.notify_no_more_waiting()
    #yield wait_for_writers()

reactor.addSystemEventTrigger('before', 'shutdown', graceful_shutdown)

Uncommenting the yield in graceful_shutdown gives me my intended behaviour
but I don't like the solution since I have no idea if the writers there
have anything to do with writing the results. They could be any writer
writing anything I assume.

Running this code with twistd -ny graceful.tac and curling
http://localhost:8080 and then issuing a SIGINT to the twistd process
results in an curl: (52) Empty reply from server.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130828/1ddb9777/attachment-0001.html>

More information about the Twisted-Python mailing list