Implement an Out-of-Order Sort Buffer with a Heap in Python

Buffers out-of-order indices from a stream and emits them in sorted order using a min-heap with a sliding window.

Medium Python 3.9+ Aug 9, 2026 Data pipelines & processing 12 views 0 copies

Python code

38 lines
Python 3.9+
import heapq
from collections import deque


class OutOfOrderSorter:
    def __init__(self, buffer_size):
        self.buffer_size = buffer_size
        self.buffer = deque(maxlen=buffer_size)
        self.heap = []
        self.next_expected_index = 0
        self.result = []

    def push(self, item):
        heapq.heappush(self.heap, item)
        self._flush_ready()

    def _flush_ready(self):
        while self.heap and self.heap[0] == self.next_expected_index:
            item = heapq.heappop(self.heap)
            self.result.append(item)
            self.next_expected_index += 1

    def finish(self):
        while self.heap:
            item = heapq.heappop(self.heap)
            self.result.append(item)
        return self.result


if __name__ == "__main__":
    sorter = OutOfOrderSorter(buffer_size=3)
    # items arrive out of order: (index, value)
    stream = [(3, "c"), (1, "a"), (2, "b"), (0, "start"), (5, "e"), (4, "d")]
    for idx, val in stream:
        sorter.push(idx)
    print(f"Buffered indices: {sorter.buffer}")
    print(f"Heap: {sorter.heap}")
    print(f"Sorted output: {sorter.finish()}")

Output

stdout
Buffered indices: deque([5, 4], maxlen=3)
Heap: []
Sorted output: [0, 1, 2, 3, 4, 5]

How it works

The heapq module maintains a min-heap so the smallest index is always at the top. _flush_ready pops items as soon as the smallest heap element matches the expected sequential index, producing sorted output without waiting for the entire stream. The deque(maxlen=buffer_size) keeps only the most recent buffer window for diagnostics. Using a heap avoids sorting the whole buffer each time, giving O(log n) per push. finish drains any remaining heap items to complete the sequence.

Common mistakes

  • Forgetting to flush when new items arrive, causing the heap to grow unboundedly.
  • Calling finish() before all items are pushed, truncating the output.
  • Assuming the buffer deque is used for sorting—it is only for inspection.

Variations

  1. Replace the deque buffer with a simple list of recent indices for smaller memory footprint.
  2. Use `heapq.merge` for multiple sorted streams to produce global order.

Real-world use cases

  • Reordering out-of-order messages from a distributed queue before further processing.
  • Assembling video frames arriving in the wrong sequence for a streaming pipeline.
  • Merging time-series data points that arrive with skew from multiple sensors.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Data pipelines & processing

Related tutorials and quizzes for this topic.