[Twisted-Python] Why deferToThread is so slow?

[Nagy, Attila]

Hi,

I have a thread safe synchronous library, which I would like to use in a 
threadpool using deferToThread.

Without using (deferTo)threads I get consistent 1-3 ms response times, 
with deferring to threadpool, I get 30-300, varying wildly.

I've tried to construct a test, which is of course not perfect, but 
shows some differences, which widen with the number of threads (while 
directly calling that dumb function remains the same).
I can see similar results with using the (database) library, but because 
it's an external dependency and performance gets worse with the number 
of queries (threads), I thought it would be best to leave that out of 
the picture, hence the dumb function.

# python /tmp/deft.py 1
deferToThread: avg 316.17 us, sync: avg 1.38 us, 228.71x increase
deferToThread: avg 312.92 us, sync: avg 1.38 us, 226.96x increase
deferToThread: avg 320.22 us, sync: avg 1.39 us, 230.37x increase
deferToThread: avg 317.33 us, sync: avg 1.35 us, 235.24x increase

# python /tmp/deft.py 8
deferToThread: avg 2542.90 us, sync: avg 1.37 us, 1854.14x increase
deferToThread: avg 2544.50 us, sync: avg 1.35 us, 1878.13x increase
deferToThread: avg 2544.47 us, sync: avg 1.36 us, 1864.52x increase
deferToThread: avg 2544.52 us, sync: avg 1.38 us, 1839.01x increase
deferToThread: avg 2544.92 us, sync: avg 1.36 us, 1871.81x increase
deferToThread: avg 2546.71 us, sync: avg 1.39 us, 1830.35x increase
deferToThread: avg 2552.38 us, sync: avg 1.35 us, 1893.17x increase
deferToThread: avg 2552.40 us, sync: avg 1.36 us, 1870.20x increase

# python /tmp/deft.py 16
deferToThread: avg 4745.76 us, sync: avg 1.26 us, 3770.11x increase
deferToThread: avg 4748.67 us, sync: avg 1.24 us, 3817.03x increase
deferToThread: avg 4749.81 us, sync: avg 1.26 us, 3756.39x increase
deferToThread: avg 4749.72 us, sync: avg 1.24 us, 3839.88x increase
deferToThread: avg 4749.87 us, sync: avg 1.28 us, 3709.99x increase
deferToThread: avg 4752.63 us, sync: avg 1.24 us, 3842.90x increase
deferToThread: avg 4752.53 us, sync: avg 1.23 us, 3866.08x increase
deferToThread: avg 4752.55 us, sync: avg 1.23 us, 3855.40x increase
deferToThread: avg 4754.03 us, sync: avg 1.29 us, 3678.09x increase
deferToThread: avg 4754.97 us, sync: avg 1.25 us, 3817.19x increase
deferToThread: avg 4755.45 us, sync: avg 1.32 us, 3593.28x increase
deferToThread: avg 4756.35 us, sync: avg 1.25 us, 3804.18x increase
deferToThread: avg 4756.19 us, sync: avg 1.29 us, 3687.73x increase
deferToThread: avg 4757.19 us, sync: avg 1.23 us, 3860.74x increase
deferToThread: avg 4758.02 us, sync: avg 1.24 us, 3824.33x increase
deferToThread: avg 4759.63 us, sync: avg 1.24 us, 3830.40x increase

On 16 threads it takes nearly 5 ms(!) to call a local function...

The test program is here:
https://gist.github.com/bra-fsn/1fd481b44590a939e849cb9073ba1a41

cpython 2.7.11, Twisted 16.2.0

BTW, I've run this on pypy 5.1.1 with even worse results:
https://mail.python.org/pipermail/pypy-dev/2016-June/014477.html