Opened 8 years ago

Closed 8 years ago

#3704 enhancement closed fixed (fixed)

Create performance benchmark for Deferreds

Reported by: thijs Owned by:
Priority: normal Milestone:
Component: core Keywords: benchmark
Cc: thijs Branch: branches/defer-benchmarks-3704
branch-diff, diff-cov, branch-cov, buildbot
Author: exarkun, peaker, thijs


James Knight has previously [source:sandbox/foom/cdefer implemented Deferred in Pyrex]. The reason this isn't included in Twisted is that we just have no reliable, accurate way to measure what the performance changes of this are. The same applies to cdefer.c in Deferred implementation in C.

It would useful to have a benchmark which makes it possible to determine the utility of a Pyrex or C implementation of Deferred. Without actually measuring, statements about how fast Pyrex, C or pure Python is are worthless.

Deriving something from the code which showed a 5% - 20% speedup might be a good way to start on such a benchmark. It's important to cover all uses of Deferreds here. It's also okay if some parts show a slowdown, as long as the overall improvement is enough. Of course, it's better if everything is faster. :)

(ticket description derived from exarkun's comment in #2245)

Attachments (2) (3.9 KB) - added by thijs 8 years ago.
defer benchmark from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245) (440 bytes) - added by thijs 8 years ago.
timer benchmark utility from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)

Download all attachments as: .zip

Change History (12)

Changed 8 years ago by thijs

defer benchmark from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)

Changed 8 years ago by thijs

timer benchmark utility from source:branches/cdefer-2245-4/doc/core/benchmarks (#2245)

comment:1 Changed 8 years ago by thijs

  • Keywords review added
  • Owner glyph deleted

I attached and from the latest [source:branches/cdefer-2245-4/doc/core/benchmarks/ branch] for ticket #2245, that run a twisted.internet.defer benchmark, with outputting:

$ python Disabled GC
instantiate () 0.205390930176
instantiateShootCallback () 0.780241012573
instantiateShootErrback () 0.100492954254
instantiateAddCallbacksNoResult (10,) 0.00270199775696
instantiateAddCallbacksNoResult (1000,) 0.263583898544
instantiateAddCallbacksNoResult (10000,) 3.02256608009
instantiateAddCallbacksBeforeResult (10,) 0.00721502304077
instantiateAddCallbacksBeforeResult (1000,) 0.712515115738
instantiateAddCallbacksBeforeResult (10000,) 13.2036309242
instantiateAddCallbacksAfterResult (10,) 0.00922584533691
instantiateAddCallbacksAfterResult (1000,) 0.903044939041
instantiateAddCallbacksAfterResult (10000,) 8.97325801849
pauseUnpause (10,) 0.00899577140808
pauseUnpause (1000,) 0.912666082382
pauseUnpause (10000,) 15.3666989803

I suggest adding (and updating in the [source:trunk/doc/core/benchmarks trunk]).

comment:2 Changed 8 years ago by moshez

  • Branch set to branches/cdefer-2245-4

Adding branch field

comment:3 Changed 8 years ago by thijs

Plans for a benchmark api are described in #3715

comment:4 Changed 8 years ago by exarkun

  • Branch branches/cdefer-2245-4 deleted

I think that branch field setting is erroneous.

comment:5 Changed 8 years ago by exarkun

  • Author set to exarkun
  • Branch set to branches/defer-benchmarks-3704

(In [26577]) Branching to 'defer-benchmarks-3704'

comment:6 Changed 8 years ago by thijs

(In [26594]) Add copyright header refs #3704

comment:7 Changed 8 years ago by thijs

  • Author changed from exarkun to exarkun, peaker, thijs

comment:8 Changed 8 years ago by exarkun

  • Keywords review removed
  • Owner set to exarkun

I made some trivial tweaks. I think this is good enough to merge. I'm sure we'll want to continue changing things as we get more benchmarking experience.

comment:9 Changed 8 years ago by exarkun

  • Resolution set to fixed
  • Status changed from new to closed

(In [26630]) Merge defer-benchmarks-3704

Author: Peaker, thijs, exarkun Reviewer: exarkun Fixes: #3704

Add a benchmark for Deferred performance.

comment:10 Changed 6 years ago by <automation>

  • Owner exarkun deleted
Note: See TracTickets for help on using tickets.