[Twisted-Python] Twisted Python vs. "Blocking" Python: Weird performance on small operations.
rlotun at gmail.com
Tue Oct 13 10:04:06 EDT 2009
I took a look at your code sample and got the async benchmark to run
with the following values:
*** Starting Asynchronous Benchmarks.
-> Asynchronous Benchmark (1 runs) Completed in 0.000181913375854 seconds.
-> Asynchronous Benchmark (10 runs) Completed in 0.000736951828003 seconds.
-> Asynchronous Benchmark (100 runs) Completed in 0.00641012191772 seconds.
-> Asynchronous Benchmark (1000 runs) Completed in 0.0741751194 seconds.
-> Asynchronous Benchmark (10000 runs) Completed in 0.675071001053 seconds.
-> Asynchronous Benchmark (100000 runs) Completed in 7.29738497734 seconds.
*** Asynchronous Benchmarks Completed in 8.16032314301 seconds.
Which, though still quite a bit slower than the synchronous version,
is still much better than the 40 sec. mark that you were experiencing.
My modified version simply returned defer.succeed from your aync
i.e. Instead of your initial example:
#Packs an integer, result is 4 bytes
return struct.pack("i", i)
d = defer.Deferred()
my version does:
return defer.succeed(struct.pack('i', anInteger))
A few things to note in general however:
1) Twisted shines for block I/O operations - i.e. networking. A
compute intesive process will not necessarily yield any gains in
performance by using Twisted since the Python GIL exists (a global
2) If you are doing computations that use a C module (unforunately
struct pre 2.6 I believe doesn't use a C module), there may be a
chance that the C module releases the GIL, allowing you to do those
computations in a thread. In this case you'd be better off using
deferToThread as suggested earlier.
3) There is some (usually minimal but it exists) overhead to using
Twisted. Instead of computing a bunch of stuff serially and returning
your answer as in your sync example, you're wrapping everything up in
deferreds and starting a reactor - it's definitely going to be a bit
slower than the pure synchronous version for this case.
Hope that makes sense.
mobile: +44 (0)7521 310 763
email: rlotun at gmail.com
work: reza at tweetdeck.com
More information about the Twisted-Python