[Twisted-Python] Twisted Web at 630k reqs/sec

[Louis D. Burr]

Hi Tobias,

> On Nov 10, 2015, at 4:54 PM, Tobias Oberstein <tobias.oberstein at tavendo.de> wrote:
> 
> Hi,
> 
> maybe it's of interest how far it's possible to drive Twisted Web on modern multi-core CPUs.
> 
> Here are 2 data points from measuring:
> 
> * 627990 HTTP requests/s at 360 us avg latency
> * 12.6 GB/s HTTP reply traffic
> 
> This is using 40 Xeon cores and serves HTTP over loopback TCP to 8 threads of wrk. The Twisted Web resource served was
> 
> https://github.com/crossbario/crossbarexamples/blob/master/benchmark/web/myresource.py
> 
> Details, figures and more results are here:
> 
> https://github.com/crossbario/crossbarexamples/tree/master/benchmark/web
> 

Those are some nice numbers, to be sure!  Twisted really benefits from running in a pypy vm.  I have read that pypy still doesn’t optimize old-style classes as efficiently as new-style classes; I wonder if there would be any meaningful change in the metrics if resource.Resource was a new-style class instead of an old-style class.

(Note that mixing in object, e.g., class Foo(resource.Resource, object), to produce a new-style class apparently makes pypy unhappy.  See http://pypy.org/performance.html for details.)

Am I correct in stating that SO_REUSEPORT only produces the desired balancing of requests between twisted processes on Linux?

Thanks,

L. Daniel Burr

> Cheers,
> /Tobias
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python