Opened 2 years ago

Last modified 17 months ago

#5628 enhancement new

Use collections.deque for the data structure in DeferredQueue

Reported by: Dev0n Owned by:
Priority: low Milestone:
Component: core Keywords: DeferredQueue deque
Cc: Branch:
Author: Launchpad Bug:

Description

This is really micro-optimising as I was assured by a few of the twisted devs in #twisted that DeferredQueue is hardly ever a bottleneck.

Testing list vs deque:

python -mtimeit -s'q=range(10)' 'q.append(11);  q.pop(0)'
1000000 loops, best of 3: 0.776 usec per loop

python -mtimeit -s'import collections; q=collections.deque(range(10))' 'q.append(11); q.popleft()'
1000000 loops, best of 3: 0.416 usec per loop

As you can see, the data set doesn't necessarily have to be big to have a _slight_ effect on performance.

I've attached a benchmarking script that I used to compare before and after the data structure change and all tests passed.

trial test_defer.py
...
    testQueue ...                                                          [OK]
    test_cancelQueueAfterGet ...                                           [OK]
    test_cancelQueueAfterSynchronousGet ...                                [OK]
...

Attachments (2)

defer-queue-deque.patch (1.2 KB) - added by Dev0n 2 years ago.
benchDeferredQueue.py (500 bytes) - added by Dev0n 2 years ago.

Download all attachments as: .zip

Change History (4)

Changed 2 years ago by Dev0n

Changed 2 years ago by Dev0n

comment:1 Changed 2 years ago by glyph

Your list-versus-dequeue comparison is a good start, but unfortunately it doesn't indicate anything relevant to DeferredQueue; you'd have to show that your changes actually have a positive impact on performance, not just that a data structure that you might use has an impact on performance of something unrelated.

If you want to do a performance-sensitive change, you should first add a benchmark that actually measures the performance of something that uses DeferredQueue so that we can see whether it has a good impact or not. Or does your test have an impact on one of the existing benchmarks?

Thanks for your contribution, though.

comment:2 Changed 17 months ago by exarkun

#6359 was a duplicate of this.

Note: See TracTickets for help on using tickets.