[Twisted-Python] reading multipart/form-data headers

[Adi Roiban]

On 11 August 2016 at 21:52, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> On Aug 11, 2016, at 8:55 AM, Burak Arslan <burak.arslan at arskom.com.tr>
> wrote:
>
> Hello All,
>
> You can find a sample HTTP POST request using HTTP multipart/form-data at
> the end of this message.
>
> The server that handles this request is using twisted so I end up with a
> Request object. Is there a way I can extract the file name ("image008.jpg")
> from this stream? I'm looking at the source of cgi.parse_multipart() and it
> seems to be ignored.
>
> Sadly Twisted just calls into cgi.parse_multipart and so it is in fact
> ignored.  You might be able to re-parse the request body
> (request.content.seek(0); request.content.read()) with something like
> <https://docs.python.org/2.7/library/email.mime.html#email.mime.multipart.MIMEMultipart>
> or <https://github.com/mailgun/flanker> to extract more information about
> the MIME.
>
> It would definitely be better for Twisted to have more robust facilities for
> dealing with request inputs, particularly to be able to process large
> uploads as a stream rather than an individual message (and such an API for
> form post uploads should obviously include the content disposition
> filename).  See <https://twistedmatrix.com/trac/ticket/288> for more
> discussion :).
>
> Thanks for using Twisted, and sorry about this shortcoming.

I have some coding which is doing a best effort to parse the request
body in a streaming mode... but it is using a fork based on the code
submitted for this ticket http://twistedmatrix.com/trac/ticket/6928

diff for the fork https://github.com/chevah/twisted/compare/6928-http-100-accept

It relies on the fact that the request will call the
resource.headerReceive() before the actual body is consumed.

form handling code using this fork
https://gist.github.com/adiroiban/7f593d6d18113aae797ad081e07f4745

It uses werkzeug.http.parse_options_header for parsing the headers

If your POST requests are just a few bytes, you can just use
request.content.seek(0); request.content.read() as suggested by Glyph
and redirect the content to the MultiPartFormData protocol

For my project I need to handle files larger than 5GB, so I ended up
with the modified request/resource

Good luck!
-- 
Adi Roiban