How to Mock Offset Commit Auto vs Manual in Python

Demonstrates a Kafka-style offset commit function with auto/manual modes and tests it using unittest.mock.patch.

Medium Python 3.9+ Aug 9, 2026 Streaming & messaging 15 views 0 copies

unittest mocking kafka offset-commit streaming

Python code

32 lines

Python 3.9+

from unittest.mock import Mock, patch

def commit_offsets(topic_partition_offsets, auto_commit=False):
    """Manually commit offsets or simulate auto-commit."""
    if auto_commit:
        print(f"Auto-committing offsets: {topic_partition_offsets}")
        return {"status": "auto_committed"}
    
    print(f"Manually committing offsets: {topic_partition_offsets}")
    return {"status": "manual_commit_success"}

# Mocking the consumer for testing
@patch("__main__.commit_offsets")
def test_manual_auto(mock_commit):
    mock_commit.return_value = {"status": "mocked_commit"}
    
    # Manual commit path
    result_manual = commit_offsets({"topic": {"partition": 0}}, auto_commit=False)
    print(f"Result (manual): {result_manual}")
    
    # Auto-commit path
    result_auto = commit_offsets({"topic": {"partition": 1}}, auto_commit=True)
    print(f"Result (auto): {result_auto}")

if __name__ == "__main__":
    # Real function call without mocking
    offsets = {"orders": {0: 42, 1: 10}}
    commit_offsets(offsets, auto_commit=False)
    commit_offsets(offsets, auto_commit=True)
    
    # Demonstrate mocking
    test_manual_auto()

Output

stdout

Manually committing offsets: {'orders': {0: 42, 1: 10}}
Auto-committing offsets: {'orders': {0: 42, 1: 10}}
Result (manual): mocked_commit
Result (auto): mocked_commit

How it works

The commit_offsets function simulates both manual and auto commit modes for Kafka partitions. When auto_commit is true, it prints and returns an auto-commit status; otherwise it returns a manual commit success. The @patch decorator from unittest.mock replaces the function with a mock for testing, allowing you to isolate and verify behavior without real side effects. The mock_commit.return_value sets what the mocked function returns, making it easy to test code paths that depend on the function result.

Common mistakes

Forgetting that `@patch` replaces the function globally within the test scope
Not setting `return_value` on the mock, leading to `MagicMock` objects instead of dicts
Patching the wrong path — use the module where the function is defined, not where it's imported

Variations

Use `with patch(...) as mock:` instead of the decorator for scoped mocking
Use `side_effect` to make the mock raise an exception or call a real function

Real-world use cases

Unit testing Kafka consumers by mocking offset commits without touching real brokers.
Simulating auto-commit vs manual-commit behavior in stream processing pipelines.
Verifying idempotent offset committing logic before deploying to production.

How to Mock Offset Commit Auto vs Manual in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Streaming & messaging

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Streaming & messaging

Keep learning

Tutorials

Quizzes