[Twisted-web] t.w.client.downloadPage with large amount of postData

Giorgos Logiotatidis seadog at sealabs.net
Tue Oct 5 14:45:15 EDT 2010


Hello everyone,
I'm writing a client that is using t.w.client.downloadPage function to
download a large chunk of data into a file from a webserver. This is
working fine.

Now I want use the same function (or at least take advantage of the
"save to file" functionality) and be able to send along with my request
a large chunk of data, without storing that chunk into memory.

To clarify my need here is an example

(1) client -> POST <large data file> -> server
(2) server computes
(3) server -> <REPLY with large data file> -> client

the (3) step is done in a memory efficient way using
t.w.c.downloadPage(). Right now I execute the (1) step using the
"postData" argument of t.w.c.downloadPage after loading all data into
memory.

Is there a way I can avoid loading all data into memory?

Thank you in advance,
-Giorgos Logiotatidis






More information about the Twisted-web mailing list