[Twisted-web] Large File Upload

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Mon Apr 26 12:32:51 EDT 2010

On 03:25 am, bresnaha at mcs.anl.gov wrote:
>I am creating a service using twisted web that allows clients to upload 
>files using HTTP put.
>By reading through issue 288 I understand that the problems I am seeing 
>are have been seen before
>and are outstanding issues.  I have a solution that seems to work for 
>me (I have little concern for
>POST thus my issue is mitigated) but I was hoping to get some community 
>comment on it.
>My objective is 1) to authorize clients *before* they upload a large 
>file to temporary storage, and
>2) decrease latency and increase throughput by eliminating the copy to 
>temp files.
>I have achieved this by subclassing http.HTTPChannel and overloading 
>allHeadersReceived() in the
>following way:
>class MyHTTPChannel(http.HTTPChannel):
>    def allHeadersReceived(self):
>        http.HTTPChannel.allHeadersReceived(self)
>         ... parse headers for access tokens
>         if not access_granted:
>             self.transport.write(<http error message>)
>             self.transport.loseConnection()
>        if self._command != "PUT":
>            return
>        req = self.requests[-1]
>        req.content = <create my own file-like object>
>Then I create a site object:
>class MySite(server.Site):
>    protocol = MyHTTPChannel
>Thus far this seems to work nicely for me, but my knowledge of the 
>twisted internals is limited.
>Does this seem at all reasonable?

Until #288 is resolved, this seems like as good an approach as any. 
Note that "_command" is a private attribute though, and I wouldn't feel 
bad changing it in a future version of Twisted.


More information about the Twisted-web mailing list