[Twisted-Python] Graceful shutdown

[Jonas Lindmark]

Hi!

I'm trying to implement graceful shutdown of a HTTP server and I am unsure
of the preferred way of implementing it.

I'm attempting to do this by adding a system event trigger for "before"
"shutdown" that should stop accepting new requests and wait for current
ones to finish.

My problem is that though my requests has fired their notifyFinish()
deferreds the data has not been written to the client. Calling
reactor.getWriters() shows me that there are active writers who are
probably writing/flushing data to the client.

What is the preferred way of waiting for the requests to finish and finish
writing the response to the client?

Here is example code for my own attempt:

from twisted.internet import reactor, defer
from twisted.application import internet, service
from twisted.web.server import Site, NOT_DONE_YET
from twisted.web.resource import Resource

class SlowResource(Resource):
    isLeaf = True
    waiting_requests = []


    def notify_no_more_waiting(self):
        if not self.waiting_requests:
            return defer.succeed(None)
        return defer.gatherResults(self.waiting_requests, consumeErrors=True) \
                .addBoth(lambda ign: None)

    def write_result(self, request):
        request.write('{}')
        request.finish()

    def render_GET(self, request):
        reactor.callLater(5, self.write_result, request)

        d = request.notifyFinish()
        self.waiting_requests.append(d)
        d.addBoth(lambda ign: self.waiting_requests.remove(d))

        return NOT_DONE_YET


slow_resource = SlowResource()
site = Site(slow_resource)
application = service.Application("MyApp")
server = internet.TCPServer(8080, site)
server.setServiceParent(application)

def wait_for_writers():
    d = defer.Deferred()

    def check_writers():
        if len(reactor.getWriters()) > 0:
            reactor.callLater(0.1, check_writers)
        else:
            d.callback(None)

    check_writers()

    return d

@defer.inlineCallbacks
def graceful_shutdown():
    yield server.stopService()
    yield slow_resource.notify_no_more_waiting()
    #yield wait_for_writers()


reactor.addSystemEventTrigger('before', 'shutdown', graceful_shutdown)


Uncommenting the yield in graceful_shutdown gives me my intended behaviour
but I don't like the solution since I have no idea if the writers there
have anything to do with writing the results. They could be any writer
writing anything I assume.

Running this code with twistd -ny graceful.tac and curling
http://localhost:8080 and then issuing a SIGINT to the twistd process
results in an curl: (52) Empty reply from server.

-- 
/Jonas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://twistedmatrix.com/pipermail/twisted-python/attachments/20130828/1ddb9777/attachment-0001.html>