Bonferroni Correction in Python
Applies the Bonferroni correction to a list of p-values to control the family-wise error rate when performing multiple comparisons.
Requires third-party packages — install first
pip install numpy
Python code
21 linesimport numpy as np
def bonferroni_correction(p_values, alpha=0.05):
"""Apply Bonferroni correction to a list of p-values."""
n = len(p_values)
corrected_alpha = alpha / n
significant = [p < corrected_alpha for p in p_values]
return corrected_alpha, significant
if __name__ == "__main__":
# Mock p-values from three comparisons
p_values = [0.01, 0.04, 0.07]
alpha = 0.05
corrected_alpha, significant = bonferroni_correction(p_values, alpha)
print(f"Number of comparisons: {len(p_values)}")
print(f"Original alpha: {alpha}")
print(f"Corrected alpha (Bonferroni): {corrected_alpha:.4f}")
print(f"P-values: {p_values}")
print(f"Significant after correction: {significant}")Output
Number of comparisons: 3
Original alpha: 0.05
Corrected alpha (Bonferroni): 0.0167
P-values: [0.01, 0.04, 0.07]
Significant after correction: [True, False, False]How it works
The Bonferroni correction divides the original significance level by the number of comparisons to account for the increased risk of false positives. In this example, with three comparisons, the corrected alpha is 0.0167, so only the p-value below that threshold is considered significant. This is a conservative approach that reduces Type I errors but may increase Type II errors.
Common mistakes
- Forgetting to account for multiple comparisons in A/B tests with many variants.
- Using the raw alpha level instead of the corrected one when interpreting results.
- Not considering less conservative alternatives like the Benjamini-Hochberg procedure.
Variations
- Use scipy.stats.false_discovery_control with method='bonferroni' for a standard implementation.
Real-world use cases
- Adjusting p-values when testing multiple variants in an A/B testing platform.
- Correcting for multiple hypotheses in feature selection or genomics research.
- Controlling error rates in clinical trial subgroup analyses.
Sponsored
More from A/B testing & experimentation
Keep learning
Related tutorials and quizzes for this topic.