Opened 5 years ago

Last modified 4 years ago

#5628 enhancement new

Use collections.deque for the data structure in DeferredQueue

Reported by: Dev0n Owned by:
Priority: low Milestone:
Component: core Keywords: DeferredQueue deque
Cc: Branch:


This is really micro-optimising as I was assured by a few of the twisted devs in #twisted that DeferredQueue is hardly ever a bottleneck.

Testing list vs deque:

python -mtimeit -s'q=range(10)' 'q.append(11);  q.pop(0)'
1000000 loops, best of 3: 0.776 usec per loop

python -mtimeit -s'import collections; q=collections.deque(range(10))' 'q.append(11); q.popleft()'
1000000 loops, best of 3: 0.416 usec per loop

As you can see, the data set doesn't necessarily have to be big to have a _slight_ effect on performance.

I've attached a benchmarking script that I used to compare before and after the data structure change and all tests passed.

    testQueue ...                                                          [OK]
    test_cancelQueueAfterGet ...                                           [OK]
    test_cancelQueueAfterSynchronousGet ...                                [OK]

Attachments (2)

defer-queue-deque.patch (1.2 KB) - added by Dev0n 5 years ago. (500 bytes) - added by Dev0n 5 years ago.

Download all attachments as: .zip

Change History (4)

Changed 5 years ago by Dev0n

Attachment: defer-queue-deque.patch added

Changed 5 years ago by Dev0n

Attachment: added

comment:1 Changed 5 years ago by Glyph

Your list-versus-dequeue comparison is a good start, but unfortunately it doesn't indicate anything relevant to DeferredQueue; you'd have to show that your changes actually have a positive impact on performance, not just that a data structure that you might use has an impact on performance of something unrelated.

If you want to do a performance-sensitive change, you should first add a benchmark that actually measures the performance of something that uses DeferredQueue so that we can see whether it has a good impact or not. Or does your test have an impact on one of the existing benchmarks?

Thanks for your contribution, though.

comment:2 Changed 4 years ago by Jean-Paul Calderone

#6359 was a duplicate of this.

Note: See TracTickets for help on using tickets.