Opened 8 years ago

Closed 8 years ago

#3704 enhancement closed fixed (fixed)

Create performance benchmark for Deferreds

Reported by: Thijs Triemstra Owned by:
Priority: normal Milestone:
Component: core Keywords: benchmark
Cc: Thijs Triemstra Branch: branches/defer-benchmarks-3704
branch-diff, diff-cov, branch-cov, buildbot
Author: exarkun, peaker, thijs

Description

James Knight has previously [source:sandbox/foom/cdefer implemented Deferred in Pyrex]. The reason this isn't included in Twisted is that we just have no reliable, accurate way to measure what the performance changes of this are. The same applies to cdefer.c in Deferred implementation in C.

It would useful to have a benchmark which makes it possible to determine the utility of a Pyrex or C implementation of Deferred. Without actually measuring, statements about how fast Pyrex, C or pure Python is are worthless.

Deriving something from the code which showed a 5% - 20% speedup might be a good way to start on such a benchmark. It's important to cover all uses of Deferreds here. It's also okay if some parts show a slowdown, as long as the overall improvement is enough. Of course, it's better if everything is faster. :)

(ticket description derived from exarkun's comment in #2245)

Attachments (2)

defer.py (3.9 KB) - added by Thijs Triemstra 8 years ago.
defer benchmark from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)
timer.py (440 bytes) - added by Thijs Triemstra 8 years ago.
timer benchmark utility from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)

Download all attachments as: .zip

Change History (12)

Changed 8 years ago by Thijs Triemstra

Attachment: defer.py added

defer benchmark from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)

Changed 8 years ago by Thijs Triemstra

Attachment: timer.py added

timer benchmark utility from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)

comment:1 Changed 8 years ago by Thijs Triemstra

Keywords: review added
Owner: Glyph deleted

I attached defer.py and timer.py from the latest [source:branches/cdefer-2245-4/doc/core/benchmarks/ branch] for ticket #2245, that run a twisted.internet.defer benchmark, with outputting:

$ python defer.py Disabled GC
twisted.internet.defer
instantiate () 0.205390930176
instantiateShootCallback () 0.780241012573
instantiateShootErrback () 0.100492954254
instantiateAddCallbacksNoResult (10,) 0.00270199775696
instantiateAddCallbacksNoResult (1000,) 0.263583898544
instantiateAddCallbacksNoResult (10000,) 3.02256608009
instantiateAddCallbacksBeforeResult (10,) 0.00721502304077
instantiateAddCallbacksBeforeResult (1000,) 0.712515115738
instantiateAddCallbacksBeforeResult (10000,) 13.2036309242
instantiateAddCallbacksAfterResult (10,) 0.00922584533691
instantiateAddCallbacksAfterResult (1000,) 0.903044939041
instantiateAddCallbacksAfterResult (10000,) 8.97325801849
pauseUnpause (10,) 0.00899577140808
pauseUnpause (1000,) 0.912666082382
pauseUnpause (10000,) 15.3666989803

I suggest adding defer.py (and updating timer.py in the [source:trunk/doc/core/benchmarks trunk]).

comment:2 Changed 8 years ago by Moshe Zadka

Branch: branches/cdefer-2245-4

Adding branch field

comment:3 Changed 8 years ago by Thijs Triemstra

Plans for a benchmark api are described in #3715

comment:4 Changed 8 years ago by Jean-Paul Calderone

Branch: branches/cdefer-2245-4

I think that branch field setting is erroneous.

comment:5 Changed 8 years ago by Jean-Paul Calderone

Author: exarkun
Branch: branches/defer-benchmarks-3704

(In [26577]) Branching to 'defer-benchmarks-3704'

comment:6 Changed 8 years ago by Thijs Triemstra

(In [26594]) Add copyright header refs #3704

comment:7 Changed 8 years ago by Thijs Triemstra

Author: exarkunexarkun, peaker, thijs

comment:8 Changed 8 years ago by Jean-Paul Calderone

Keywords: review removed
Owner: set to Jean-Paul Calderone

I made some trivial tweaks. I think this is good enough to merge. I'm sure we'll want to continue changing things as we get more benchmarking experience.

comment:9 Changed 8 years ago by Jean-Paul Calderone

Resolution: fixed
Status: newclosed

(In [26630]) Merge defer-benchmarks-3704

Author: Peaker, thijs, exarkun Reviewer: exarkun Fixes: #3704

Add a benchmark for Deferred performance.

comment:10 Changed 6 years ago by <automation>

Owner: Jean-Paul Calderone deleted
Note: See TracTickets for help on using tickets.