How to Use a Bounded Buffer with threading.Condition in Python
Implement a thread-safe bounded buffer using threading.Condition and show a producer–consumer example with exact output.
Python code
48 linesimport threading
import time
import random
class BoundedBuffer:
def __init__(self, capacity):
self.capacity = capacity
self.buffer = []
self.condition = threading.Condition()
def put(self, item):
with self.condition:
while len(self.buffer) >= self.capacity:
self.condition.wait()
self.buffer.append(item)
print(f"Produced {item}, buffer={self.buffer}")
self.condition.notify_all()
def get(self):
with self.condition:
while not self.buffer:
self.condition.wait()
item = self.buffer.pop(0)
print(f"Consumed {item}, buffer={self.buffer}")
self.condition.notify_all()
return item
def producer(buffer, items):
for i in range(items):
buffer.put(i)
time.sleep(random.uniform(0.1, 0.3))
def consumer(buffer, items):
for _ in range(items):
buffer.get()
time.sleep(random.uniform(0.1, 0.3))
if __name__ == "__main__":
buffer = BoundedBuffer(capacity=3)
threads = [
threading.Thread(target=producer, args=(buffer, 5)),
threading.Thread(target=consumer, args=(buffer, 5))
]
for t in threads:
t.start()
for t in threads:
t.join()
print("All done")
Output
Produced 0, buffer=[0]
Consumed 0, buffer=[]
Produced 1, buffer=[1]
Consumed 1, buffer=[]
Produced 2, buffer=[2]
Produced 3, buffer=[2, 3]
Consumed 2, buffer=[3]
Produced 4, buffer=[3, 4]
Consumed 3, buffer=[4]
Consumed 4, buffer=[]
All done
How it works
The threading.Condition wraps a lock and provides wait() and notify_all() for synchronizing threads. In put, the while loop checks if the buffer is full and waits when it is; each wait releases the lock, lets other threads run, and re-acquires it on wake-up. get does the same for an empty buffer. After each change, notify_all() wakes every waiting thread so they can re-check the condition. Using with self.condition ensures the lock is always released even on exceptions.
Common mistakes
- Using `if` instead of `while` in the wait condition causes spurious wakeups or lost signals
- Forgetting to call `notify_all()` after adding or removing an item, leaving producers or consumers waiting forever
- Calling `wait()` without holding the condition lock, which raises a RuntimeError
Variations
- Use `queue.Queue(maxsize)` for a simpler, built-in bounded buffer with `put()` and `get()` methods
- Use `threading.Semaphore` with two semaphores to limit capacity and ensure availability
Real-world use cases
- Implementing a task queue where a fixed number of worker threads consume jobs while producers add them
- Batching messages into chunks in an ETL pipeline with a bounded in-memory staging area
- Throttling network requests by buffering outgoing calls in a size-limited shared queue
Sponsored
More from Concurrency & performance
- Benchmark list.append vs deque.append in Python medium
- Build a Python Performance Profiler That Generates Readable Reports medium
- Graceful Shutdown Executor Context Manager in Python medium
- How to Build a Producer-Consumer Pattern with asyncio.Queue in Python medium
- How to Cancel an asyncio Task with Graceful Cleanup in Python medium
- How to Convert Data in Parallel with ThreadPoolExecutor in Python easy
Keep learning
Related tutorials and quizzes for this topic.