[Twisted-web] render_GET and memory consumption
Pedro I. Sanchez
psanchez at fosstel.com
Tue Dec 21 00:19:21 EST 2010
Here's a demo HTTP server that returns 10 MB of random data each time a
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.resource import Resource
data = os.urandom(10*1024*1024)
isLeaf = True
def render_GET(self, request):
root = Resource()
Now, when I run N clients simultaneously from a different host I see
that the server's memory consumption increases by N*10 MB. I can't
reproduce this example when running the clients from the same host as
the server; the test goes so fast that I can't gather any useful data.
I run the test using the following httperf command on a different host
and looking at the Gnome system monitor in the server (top will do as well).
httperf --server 192.168.1.10 --port 8880 --uri /test \
--rate 10 --num-conn 10
When the server is idle memory consumption is 17.1 MB, but during the
test it jumps to 117.2 MB. My questions are then:
1. Given that 'data' is a global variable, eventually read-only as well,
why is it replicated for each request? And who is replicating it?
2, What would be the proper way to re-write this example so that there
is one and only one 'data' structure at any time?
More information about the Twisted-web