[Twisted-Python] FW: Large (GB) File Upload

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Sat Jul 28 12:55:41 MDT 2012


On 04:09 pm, bob.novas at shinkuro.com wrote:
>Hi - I'd like some guidance, please, on writing a TCPServer that can
>efficiently receive and process (to a database) large files received 
>from a
>browser client form (multipart/form-data).  I'd like to be able to 
>process
>the command and headers without waiting for allContentReceived, if 
>possible.
>In other words, I'd like to actually handle the incoming stream rather 
>than
>buffer it to a file or a string and then handle the Request when all 
>content
>is received.  Note that the server is used only by a local browser by a
>single user.  It is not used by a large user populace.
>
>Is this possible? Are there any examples? Any guidance would be 
>appreciated.
>This was the best  I could find -
>http://twistedmatrix.com/pipermail/twisted- 
>python/2007-July/015738.html, but
>it's pretty dated.

There are a few answers.

One is <http://twistedmatrix.com/trac/ticket/288>, an enhancement 
request for a nice, documented API for handling request bodies as they 
arrive.

Another is to override Request.handleContentChunk, which is called each 
time request body bytes are received (and decoded).

A third is to override Request.gotLength and initialize the `content` 
attribute differently somehow.  The default implementation of 
`handleContentChunk` just calls `self.content.write` with the content 
chunk.

Jean-Paul




More information about the Twisted-Python mailing list