Chaos Inject Random Failures in Python
Simulate random failures in a Python function to test error handling and resilience, using random thresholds and controllable success rates.
Python code
29 linesimport random
def unreliable_function(success_rate: float = 0.7) -> str:
"""Simulate a function that sometimes fails."""
if random.random() > success_rate:
raise ConnectionError("Simulated network failure")
return "Operation completed successfully"
if __name__ == "__main__":
random.seed(42) # reproducible output
for attempt in range(10):
try:
result = unreliable_function(success_rate=0.7)
print(f"Attempt {attempt + 1}: {result}")
except ConnectionError as e:
print(f"Attempt {attempt + 1}: FAILED -> {e}")
# Optionally count failures after many runs
failures = 0
for _ in range(1000):
try:
unreliable_function(success_rate=0.7)
except ConnectionError:
failures += 1
print(f"\nFailure rate over 1000 runs: {failures / 1000:.2%}")
Output
Attempt 1: Operation completed successfully
Attempt 2: FAILED -> Simulated network failure
Attempt 3: Operation completed successfully
Attempt 4: FAILED -> Simulated network failure
Attempt 5: Operation completed successfully
Attempt 6: Operation completed successfully
Attempt 7: Operation completed successfully
Attempt 8: Operation completed successfully
Attempt 9: FAILED -> Simulated network failure
Attempt 10: Operation completed successfully
Failure rate over 1000 runs: 30.40%
How it works
The random.random() call returns a float between 0 and 1. When this value exceeds the success_rate, the function raises a ConnectionError, simulating a failure. By seeding the random number generator with random.seed(42), the sequence of failures becomes reproducible, which is useful for debugging. The loop catches the error and continues, demonstrating how your application can recover gracefully. This pattern is foundational for chaos engineering, where you deliberately inject failures to verify system robustness.
Common mistakes
- Using `random.randint` instead of `random.random` which skews the probability distribution.
- Forgetting to seed the random generator when you need deterministic test runs.
- Catching the wrong exception type and missing the simulated failures.
Variations
- Use a decorator to wrap any function with failure injection, making it reusable.
- Use a parameter to randomly delay responses in addition to failing, simulating latency.
Real-world use cases
- Validating retry logic in microservices by deliberately failing downstream calls during load tests.
- Testing resilience of background jobs to transient infrastructure failures like database timeouts.
- Simulating flaky network conditions in CI to ensure the app handles partial outages correctly.
Sponsored
More from Reliability & rate limiting
- At Least Once with Idempotent Consumer in Python medium
- Build a Rate Limiter Decorator in Python easy
- Build a queue-based admission control system in Python easy
- Circuit breaker failure threshold count in Python medium
- Exactly Once Processing Dedupe Mock in Python easy
- Fixed Window Counter Rate Limiting in Python easy
Keep learning
Related tutorials and quizzes for this topic.