[Twisted-Python] Twisted tips for designing highly concurrent twisted REST API

[Tobias Oberstein]

Am 11.07.19 um 23:34 schrieb Sean DiZazzo:
> Klein and Crossbar.io seem relevant as well
> 
> https://crossbario.com/blog/Going-Asynchronous-from-Flask-to-Twisted-Klein/

yeah, klein is neat!

fwiw, this might also be of interest, as it allows to scale-up twisted 
web (and hence also klein) on multi-core (on linux)

https://github.com/crossbario/crossbar-examples/tree/master/benchmark/web

combining SO_REUSEPORT with Klein results in a concurrent, async 
(threadless) server parallelized via processes ..

> 
> On Thu, Jul 11, 2019 at 1:46 AM Scott, Barry <barry.scott at forcepoint.com 
> <mailto:barry.scott at forcepoint.com>> wrote:
> 
>     On Tuesday, 9 July 2019 22:04:11 BST Tom Most wrote:
> 
>     ...snip...
> 
>      > The reactor's own thread pool is really for DNS
>      > resolution.
> 
>     Is that still true in the default case? We are use the twisted code
>     that talks
>     to DNS servers as the threaded resolver adds too much latency.
> 
>      > You risk deadlocks in a system that
>      > ThreadPoolThreadPoolThreadPool
>      >
>      > 3. The specifics of what long_computation are also important. If
>     it doesn't
>      > release the GIL you won't get real parallelism (this is a Python
>     thing, not
>      > a Twisted thing). See this recent thread on the topic
>      >
>     <https://twistedmatrix.com/pipermail/twisted-python/2019-June/032371.html>.
> 
>     We pass out the computational work to other processes over
>     unix-domain-sockets
>     to avoid the GIL issues.
> 
>      >
>      > Though the mechanisms differ athis thread on the topicny of the
>     above would
>      > cause the response time to increase as you add load.
>      >
>      > Good luck,
>      > Tom
> 
>     Barry
> 
> 
> 
>     _______________________________________________
>     Twisted-Python mailing list
>     Twisted-Python at twistedmatrix.com
>     <mailto:Twisted-Python at twistedmatrix.com>
>     https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> 
> _______________________________________________
> Twisted-Python mailing list
> Twisted-Python at twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 


-- 

Tobias Oberstein - phone +49 176 2375 2055 - tobias.oberstein at crossbario.com
Crossbar.io GmbH - Waldstrasse 18 - 91054 Erlangen
HRB 15870 - Amtsgericht Fuerth - Geschäftsfuehrer/CEO - Tobias Oberstein

https://crossbar.io
https://crossbario.com