[Twisted-web] Large File Upload

John Bresnahan bresnaha at mcs.anl.gov
Sun Apr 25 23:25:15 EDT 2010

I am creating a service using twisted web that allows clients to upload files using HTTP put.

By reading through issue 288 I understand that the problems I am seeing are have been seen before
and are outstanding issues.  I have a solution that seems to work for me (I have little concern for
POST thus my issue is mitigated) but I was hoping to get some community comment on it.

My objective is 1) to authorize clients *before* they upload a large file to temporary storage, and
2) decrease latency and increase throughput by eliminating the copy to temp files.

I have achieved this by subclassing http.HTTPChannel and overloading allHeadersReceived() in the
following way:

class MyHTTPChannel(http.HTTPChannel):

    def allHeadersReceived(self):

	... parse headers for access tokens

	if not access_granted:
             self.transport.write(<http error message>)

        if self._command != "PUT":

        req = self.requests[-1]
        req.content = <create my own file-like object>

Then I create a site object:

class MySite(server.Site):
    protocol = MyHTTPChannel

Thus far this seems to work nicely for me, but my knowledge of the twisted internals is limited.
Does this seem at all reasonable?



More information about the Twisted-web mailing list