Thread-Safe Producer Consumer Queue in Python

A producer-consumer pattern using thread-safe queue.Queue with two threads, demonstrating safe communication and synchronized task completion.

Easy Python 3.9+ Aug 9, 2026 Concurrency & performance 12 views 0 copies

Python code

39 lines
Python 3.9+
import queue
import threading
import time
import random


def producer(q, item_count):
    for i in range(item_count):
        item = random.randint(1, 100)
        q.put(item)
        print(f"Producer added: {item}")
        time.sleep(0.1)


def consumer(q):
    while True:
        try:
            item = q.get(timeout=2)
            print(f"Consumer got: {item}")
            q.task_done()
        except queue.Empty:
            print("Queue empty, consumer exiting.")
            break


if __name__ == "__main__":
    q = queue.Queue(maxsize=5)
    item_count = 5

    producer_thread = threading.Thread(target=producer, args=(q, item_count))
    consumer_thread = threading.Thread(target=consumer, args=(q,))

    producer_thread.start()
    consumer_thread.start()

    producer_thread.join()
    q.join()
    consumer_thread.join()
    print("All done.")

Output

stdout
Producer added: 42
Consumer got: 42
Producer added: 17
Consumer got: 17
Producer added: 93
Consumer got: 93
Producer added: 55
Consumer got: 55
Producer added: 82
Consumer got: 82
Queue empty, consumer exiting.
All done.

How it works

The queue.Queue class is inherently thread-safe, so producers and consumers can add and remove items without explicit locks. q.task_done() signals that an item has been processed, and q.join() blocks until all items are marked done. The consumer uses q.get(timeout=2) to prevent infinite blocking, exiting with queue.Empty when no new items arrive. Producer and consumer threads run concurrently, demonstrating safe coordination through the queue.

Common mistakes

  • Calling q.join() before all producers finish adding items
  • Forgetting to call q.task_done() after processing each item
  • Using a plain list instead of queue.Queue, which is not thread-safe

Variations

  1. Use a sentinel value like None to signal the consumer to stop, instead of relying on timeout.
  2. Switch to multiprocessing.Queue if you need parallelism across CPU cores.

Real-world use cases

  • Log processors that enqueue incoming log lines and have worker threads write them to disk.
  • Web scrapers that queue URLs for multiple fetcher threads to download pages concurrently.
  • Image thumbnail generators that push image paths to a queue for background processing workers.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Concurrency & performance

Related tutorials and quizzes for this topic.