[Twisted-web] Large File Upload

John Bresnahan bresnaha at mcs.anl.gov
Sun Apr 25 23:25:15 EDT 2010


I am creating a service using twisted web that allows clients to upload files using HTTP put.

By reading through issue 288 I understand that the problems I am seeing are have been seen before
and are outstanding issues.  I have a solution that seems to work for me (I have little concern for
POST thus my issue is mitigated) but I was hoping to get some community comment on it.

My objective is 1) to authorize clients *before* they upload a large file to temporary storage, and
2) decrease latency and increase throughput by eliminating the copy to temp files.

I have achieved this by subclassing http.HTTPChannel and overloading allHeadersReceived() in the
following way:

class MyHTTPChannel(http.HTTPChannel):

    def allHeadersReceived(self):
        http.HTTPChannel.allHeadersReceived(self)

	... parse headers for access tokens

	if not access_granted:
             self.transport.write(<http error message>)
             self.transport.loseConnection()

        if self._command != "PUT":
            return

        req = self.requests[-1]
        req.content = <create my own file-like object>

Then I create a site object:

class MySite(server.Site):
    protocol = MyHTTPChannel

Thus far this seems to work nicely for me, but my knowledge of the twisted internals is limited.
Does this seem at all reasonable?

Thanks,

John



More information about the Twisted-web mailing list