[Twisted-Python] Twisted webserver performance issues
Maarten ter Huurne
maarten at treewalker.org
Sun Nov 8 09:54:29 EST 2009
On Sunday 08 November 2009, James Y Knight wrote:
> When I last looked into the performance issues, I found that sometimes
> trac appears to block for long periods of time without releasing the
> GIL. That seems to be the core of the performance issues, currently.
> When it's responding normally, it's perfectly snappy. But, sometimes,
> it blocks for 10sec at a time.
How much memory does Trac use? Python garbage collection works in
generations, which leads to recently allocated objects to be checked for
garbage more often than older objects.
When collecting the oldest generation (2), we saw the Python VM freeze for
several minutes. This was on a process that had several GB of allocated
memory though. There was enough RAM to contain all data, but it's still
possible the OS (Linux) swapped out pages there were not used for a long
time to get more memory for caching.
We decided to work around the issue by disabling mark-and-sweep collection
and relying on reference counting only. We had to add some code to forcably
break reference cycles, but this was only needed for a small number cases.
More information about the Twisted-Python