Benchmark list.append vs deque.append in Python
Measures and compares the performance of appending to a Python list versus a collections.deque using timeit.repeat, showing best and average timings.
Python code
23 lines"""Benchmark list.append vs collections.deque.append."""
import timeit
def bench(stmt, setup, repeat=5, number=1_000_000):
times = timeit.repeat(stmt, setup=setup, repeat=repeat, number=number)
return min(times), sum(times) / len(times)
if __name__ == "__main__":
number = 1_000_000
list_best, list_avg = bench(
"lst.append(1)",
"lst = []",
number=number,
)
deque_best, deque_avg = bench(
"dq.append(1)",
"from collections import deque; dq = deque()",
number=number,
)
print(f"list.append — best: {list_best:.4f}s, avg: {list_avg:.4f}s")
print(f"deque.append — best: {deque_best:.4f}s, avg: {deque_avg:.4f}s")
print(f"deque is {'faster' if deque_best < list_best else 'slower'} on best time")
Output
list.append — best: 0.0345s, avg: 0.0350s
deque.append — best: 0.0512s, avg: 0.0530s
deque is slower on best time
How it works
This script uses timeit.repeat to run each append statement multiple times, collecting several timings to account for system variability. The min() result represents the fastest run, which is the most reliable indicator of performance, while the average gives a sense of typical behavior. Appending to a list is generally faster than appending to a deque because list append has a highly optimized path for adding to the end, while deque append has some overhead for maintaining the doubly-linked list structure. This benchmark highlights the importance of measuring before optimizing; while deques excel at appending to both ends, for simple end-append operations, lists are often sufficient and faster.
Common mistakes
- Using `timeit.timeit` instead of `timeit.repeat` and not taking the minimum, which can overestimate times due to system noise
- Not isolating setup code, causing the list or deque to be re-created each run, which skews results
- Comparing operations with different data structures without considering their specific use cases (e.g., deque for both-end appends)
Variations
- Use `timeit.default_timer()` and manual loops for more control over the benchmark setup
- Benchmark other operations like `list.pop(0)` vs `deque.popleft()` to see where deque outperforms
Real-world use cases
- Deciding between a list and deque when implementing a queue for a multi-threaded task scheduler.
- Optimizing a high-frequency logging buffer where elements are appended and occasionally drained from the front.
- Validating performance assumptions when building Python libraries that need predictable append-speed for large datasets.
Sponsored
More from Concurrency & performance
- Build a Python Performance Profiler That Generates Readable Reports medium
- Graceful Shutdown Executor Context Manager in Python medium
- How to Build a Producer-Consumer Pattern with asyncio.Queue in Python medium
- How to Cancel an asyncio Task with Graceful Cleanup in Python medium
- How to Convert Data in Parallel with ThreadPoolExecutor in Python easy
- How to Demonstrate the GIL with Python Threads vs Processes medium
Keep learning
Related tutorials and quizzes for this topic.