How to Run a Fisher Exact Test in Python
Compute the two-sided Fisher exact test p-value for a 2x2 contingency table using pure Python and the math module.
Python code
49 linesfrom math import comb, factorial
from itertools import combinations
def hypergeometric_probability(a, b, c, d):
"""Probability of observing table [[a, b], [c, d]] under the null."""
row1 = a + b
row2 = c + d
col1 = a + c
col2 = b + d
total = row1 + row2
return (comb(row1, a) * comb(row2, c)) / comb(total, col1)
def fisher_exact_test(a, b, c, d):
"""Two-sided Fisher exact test p-value for 2x2 contingency table."""
observed = [a, b, c, d]
row1 = a + b
row2 = c + d
col1 = a + c
col2 = b + d
if min(row1, row2, col1, col2) < 0:
raise ValueError("Table entries must be non-negative")
if row1 == 0 or row2 == 0 or col1 == 0 or col2 == 0:
return 1.0 # Degenerate table
prob_observed = hypergeometric_probability(a, b, c, d)
p_value = 0.0
max_a = min(row1, col1)
min_a = max(0, col1 - row2)
for x in range(min_a, max_a + 1):
y = row1 - x
z = col1 - x
w = row2 - z
if y >= 0 and z >= 0 and w >= 0:
prob = hypergeometric_probability(x, y, z, w)
if prob <= prob_observed:
p_value += prob
return p_value
if __name__ == "__main__":
# Example: [[6, 2], [1, 5]]
table = [6, 2, 1, 5]
p = fisher_exact_test(*table)
print(f"Table: {table}")
print(f"Two-sided p-value: {p:.6f}")
Output
Table: [6, 2, 1, 5]
Two-sided p-value: 0.103226
How it works
The Fisher exact test calculates the probability of observing a table as extreme as the one given, under the null hypothesis of independence. This implementation uses the hypergeometric distribution to compute the probability of each possible table with the same row and column totals. It sums the probabilities of all tables with a probability less than or equal to the observed table to get the two-sided p-value. The comb function from math (available in Python 3.8+) efficiently computes binomial coefficients. This method is exact and does not rely on large-sample approximations, making it ideal for small sample sizes.
Common mistakes
- Using `factorial` instead of `comb` for marginal totals, which can overflow for large tables
- Forgetting to handle degenerate tables where a row or column sum is zero
- Assuming a one-sided test when a two-sided p-value is needed for typical A/B testing
- Not checking for negative input before computing probabilities
Variations
- Use `scipy.stats.fisher_exact` for a faster, optimized implementation with one- and two-sided options
- Implement a one-sided p-value by summing only tables where the odds ratio is in one direction
Real-world use cases
- Determining if a conversion rate difference between a control and treatment group is statistically significant in an A/B test with small sample sizes.
- Analyzing whether a rare adverse event occurs more frequently in a drug treatment group compared to a placebo in clinical trials.
- Checking for association between two categorical variables in a 2x2 table from survey data when the expected cell counts are below five.
Sponsored
More from A/B testing & experimentation
Keep learning
Related tutorials and quizzes for this topic.