[Twisted-web] render_GET and memory consumption

Pedro I. Sanchez psanchez at fosstel.com
Tue Dec 21 14:02:35 EST 2010


On 10-12-21 12:47 PM, exarkun at twistedmatrix.com wrote:
> On 04:43 pm, psanchez at fosstel.com wrote:
>>
>> OK, I guess I'm being slow :-( Here's another version, same results.
>>
>> import os
>>from twisted.internet import reactor
>>from twisted.web.server import Site
>>from twisted.web.resource import Resource
>>
>> CHUNK_SIZE = 32*1024
>> data = os.urandom(10*1024*1024)
>> chunks = []
>>
>> def make_chunks():
>>      s = 0
>>      for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''):
>>          chunks.append(chunk)
>>          s = s + CHUNK_SIZE
>>
>> class TestPage(Resource):
>>       isLeaf = True
>>
>>       def render_GET(self, request):
>>           for chunk in chunks:
>>               request.write(chunk)
>>
>> make_chunks()
>> root = Resource()
>> root.putChild('test', TestPage())
>> reactor.listenTCP(8880, Site(root))
>> reactor.run()
>
> This version has flat memory usage on my system - 33MB all the way
> through.
> Jean-Paul
>
> _______________________________________________
> Twisted-web mailing list
> Twisted-web at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web

Jean-Paul,

Did you try this running the clients on a different machine than the one 
running the server? I find that if I run both the server and the clients 
(using httperf) on the same machine things behave differently. I can 
only see the memory consumption problem consistently when running the 
clients on a separate machine (which would be my real use case).

BTW, I'm running these test on Ubuntu 10.04, Python 2.6.5, twisted 
10.0.0-2ubuntu2.

-- 
Pedro




More information about the Twisted-web mailing list