How to Use Mock Flip Mutation Testing in Python
Demonstrates how mutation testing tools flip Boolean literals (mock flip) in Python source to verify test suite effectiveness in catching logic changes.
Python code
38 linesimport random
# In mutation testing, a "mock flip" intentionally changes a Boolean
# constant to False (or True) to see if the test suite catches it.
# This is a common "constant mutation" applied to a source file's literals.
def is_even(n: int) -> bool:
"""Return True if n is even. Contains a Boolean literal used as a mock target."""
return n % 2 == 0 # The 'True' implicit result; we'll flip the literal below.
def apply_mock_flip(original: str) -> str:
"""Simulate a mutation tool flipping the first 'True' literal in code to 'False'."""
return original.replace("True", "False", 1)
def run_mock_flip_demo():
# Example source snippet (like a file read by a mutation testing tool)
source_snippet = """
if is_even(0):
print("Zero is even")
else:
print("Zero is odd")
"""
# Capture test output before mutation
print("=== Before mutation (expected: 'Zero is even') ===")
exec(compile(source_snippet, "<string>", "exec"))
# Mutate the code: flip the 'True' (implicit in condition) — but here we
# explicitly flip a Boolean constant to demonstrate the concept.
mutated_source = source_snippet.replace("if is_even(0):", "if is_even(0) and True:") # placeholder for demo
# In practice, the tool would modify the file itself; we show effect:
flipped = source_snippet.replace("if is_even(0):", "if is_even(0) and False:")
print("\n=== After mock flip (expected: 'Zero is odd') ===")
exec(compile(flipped, "<string>", "exec"))
if __name__ == "__main__":
run_mock_flip_demo()
Output
=== Before mutation (expected: 'Zero is even') ===
Zero is even
=== After mock flip (expected: 'Zero is odd') ===
Zero is odd
How it works
The is_even function returns a Boolean result from an equality check, which serves as the implicit literal that mutation tools target. apply_mock_flip shows how tools like mutmut or Cosmic Ray scan source strings and replace True with False using str.replace with a count limit of 1. The demo uses exec with compile to run mutated source in isolation, comparing outputs before and after the flip. A high-quality test suite should detect these mutations by failing on the changed behavior, indicating the tests are strong enough to catch subtle logic errors.
Common mistakes
- Forgetting that `str.replace` with `count=1` replaces only the first occurrence, which may not be the intended literal
- Confusing mutation testing with fuzzing — mutation flips code, fuzzing changes input data
- Assuming the `exec` approach works for all code; imports and global state can break isolated execution
Variations
- Use `mutmut` pip package for automated mutation testing on a full test suite
- Manually edit source files with `pathlib.Path.replace` instead of string manipulation for production-like testing
Real-world use cases
- Evaluating whether unit tests for a banking module catch a flipped `if active:` condition to False.
- Verifying a CI pipeline fails when a configuration flag in a deployment script is inverted.
- Auditing test coverage for a data validation function by flipping an `is_valid` Boolean guard.
Sponsored
More from Testing & modern typing
- Capture stdout and stderr with pytest capsys easy
- Characterization Test for Legacy Python Code medium
- Dataclass with Type Hints Fields in Python easy
- Dependency Injection in Python for Testability easy
- Design Data Helpers with Python TypedDict and Literal easy
- Fix and Test a Regression Bug in Python with Unit Tests easy
Keep learning
Related tutorials and quizzes for this topic.