[Twisted-Python] re: Congrats / a little newbie confusion
bob at redivi.com
Fri Apr 4 14:21:56 EST 2003
On Friday, Apr 4, 2003, at 14:08 America/New_York, Peter Hansen wrote:
> Jeffrey Mathews wrote:
>> As for Peter's question about the need for a CGI.FieldStorage type
>> api, it's
>> really only an issue when you're doing huge multipart mime posts
>> (e.g., file
>> uploads). The idea of passing around those results (potentially multi
>> megabyte) as strings is kind of frightening. That said, I have no
>> grasp on
>> what issues are raised when handling these things asynchronously (the
>> module is all synchronous).
> What is it about passing around multi-megabyte data as strings that
> you? The data is only needed temporarily, I would think, until it is
> to a file. Servers have gobs of memory, and massive file uploads are
> rare... or if they're not, surely the server can have adequate capacity
> to handle the load (memory-wise).
> Maybe I'm too naive, but unless I was expecting, say, greater than 10MB
> or 20MB files on a regular basis, I wouldn't think twice about just
> strings. Much simpler than trying to juggle temporary files and
> to them and removing them when appropriate etc.
> Is your concern a case of premature optimization?
It's not premature optimization, it's proper design for an async
framework. If for no other reason, large strings take non-trivial
amounts of time to process. For example, on my laptop (which has a gig
of ram): ('a' * 2**20).replace('a', 'b') takes about 0.35 seconds..
Twisted is not threaded, so that's a bad thing to do. Processing
chunks with file-like interfaces is scalable, large strings are
More information about the Twisted-Python