How to Use Mock Flip Mutation Testing in Python

Demonstrates how mutation testing tools flip Boolean literals (mock flip) in Python source to verify test suite effectiveness in catching logic changes.

Medium Python 3.9+ Aug 9, 2026 Testing & modern typing 13 views 0 copies

Python code

38 lines
Python 3.9+
import random

# In mutation testing, a "mock flip" intentionally changes a Boolean
# constant to False (or True) to see if the test suite catches it.
# This is a common "constant mutation" applied to a source file's literals.

def is_even(n: int) -> bool:
    """Return True if n is even. Contains a Boolean literal used as a mock target."""
    return n % 2 == 0  # The 'True' implicit result; we'll flip the literal below.

def apply_mock_flip(original: str) -> str:
    """Simulate a mutation tool flipping the first 'True' literal in code to 'False'."""
    return original.replace("True", "False", 1)

def run_mock_flip_demo():
    # Example source snippet (like a file read by a mutation testing tool)
    source_snippet = """
    if is_even(0):
        print("Zero is even")
    else:
        print("Zero is odd")
    """
    
    # Capture test output before mutation
    print("=== Before mutation (expected: 'Zero is even') ===")
    exec(compile(source_snippet, "<string>", "exec"))
    
    # Mutate the code: flip the 'True' (implicit in condition) — but here we
    # explicitly flip a Boolean constant to demonstrate the concept.
    mutated_source = source_snippet.replace("if is_even(0):", "if is_even(0) and True:")  # placeholder for demo
    # In practice, the tool would modify the file itself; we show effect:
    flipped = source_snippet.replace("if is_even(0):", "if is_even(0) and False:")
    
    print("\n=== After mock flip (expected: 'Zero is odd') ===")
    exec(compile(flipped, "<string>", "exec"))

if __name__ == "__main__":
    run_mock_flip_demo()

Output

stdout
=== Before mutation (expected: 'Zero is even') ===
Zero is even

=== After mock flip (expected: 'Zero is odd') ===
Zero is odd

How it works

The is_even function returns a Boolean result from an equality check, which serves as the implicit literal that mutation tools target. apply_mock_flip shows how tools like mutmut or Cosmic Ray scan source strings and replace True with False using str.replace with a count limit of 1. The demo uses exec with compile to run mutated source in isolation, comparing outputs before and after the flip. A high-quality test suite should detect these mutations by failing on the changed behavior, indicating the tests are strong enough to catch subtle logic errors.

Common mistakes

  • Forgetting that `str.replace` with `count=1` replaces only the first occurrence, which may not be the intended literal
  • Confusing mutation testing with fuzzing — mutation flips code, fuzzing changes input data
  • Assuming the `exec` approach works for all code; imports and global state can break isolated execution

Variations

  1. Use `mutmut` pip package for automated mutation testing on a full test suite
  2. Manually edit source files with `pathlib.Path.replace` instead of string manipulation for production-like testing

Real-world use cases

  • Evaluating whether unit tests for a banking module catch a flipped `if active:` condition to False.
  • Verifying a CI pipeline fails when a configuration flag in a deployment script is inverted.
  • Auditing test coverage for a data validation function by flipping an `is_valid` Boolean guard.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from Testing & modern typing

Related tutorials and quizzes for this topic.