How to Run a Fisher Exact Test in Python

Compute the two-sided Fisher exact test p-value for a 2x2 contingency table using pure Python and the math module.

Medium Python 3.8+ Aug 9, 2026 A/B testing & experimentation 18 views 0 copies

Python code

49 lines
Python 3.8+
from math import comb, factorial
from itertools import combinations


def hypergeometric_probability(a, b, c, d):
    """Probability of observing table [[a, b], [c, d]] under the null."""
    row1 = a + b
    row2 = c + d
    col1 = a + c
    col2 = b + d
    total = row1 + row2
    return (comb(row1, a) * comb(row2, c)) / comb(total, col1)


def fisher_exact_test(a, b, c, d):
    """Two-sided Fisher exact test p-value for 2x2 contingency table."""
    observed = [a, b, c, d]
    row1 = a + b
    row2 = c + d
    col1 = a + c
    col2 = b + d

    if min(row1, row2, col1, col2) < 0:
        raise ValueError("Table entries must be non-negative")
    if row1 == 0 or row2 == 0 or col1 == 0 or col2 == 0:
        return 1.0  # Degenerate table

    prob_observed = hypergeometric_probability(a, b, c, d)
    p_value = 0.0
    max_a = min(row1, col1)
    min_a = max(0, col1 - row2)

    for x in range(min_a, max_a + 1):
        y = row1 - x
        z = col1 - x
        w = row2 - z
        if y >= 0 and z >= 0 and w >= 0:
            prob = hypergeometric_probability(x, y, z, w)
            if prob <= prob_observed:
                p_value += prob
    return p_value


if __name__ == "__main__":
    # Example: [[6, 2], [1, 5]]
    table = [6, 2, 1, 5]
    p = fisher_exact_test(*table)
    print(f"Table: {table}")
    print(f"Two-sided p-value: {p:.6f}")

Output

stdout
Table: [6, 2, 1, 5]
Two-sided p-value: 0.103226

How it works

The Fisher exact test calculates the probability of observing a table as extreme as the one given, under the null hypothesis of independence. This implementation uses the hypergeometric distribution to compute the probability of each possible table with the same row and column totals. It sums the probabilities of all tables with a probability less than or equal to the observed table to get the two-sided p-value. The comb function from math (available in Python 3.8+) efficiently computes binomial coefficients. This method is exact and does not rely on large-sample approximations, making it ideal for small sample sizes.

Common mistakes

  • Using `factorial` instead of `comb` for marginal totals, which can overflow for large tables
  • Forgetting to handle degenerate tables where a row or column sum is zero
  • Assuming a one-sided test when a two-sided p-value is needed for typical A/B testing
  • Not checking for negative input before computing probabilities

Variations

  1. Use `scipy.stats.fisher_exact` for a faster, optimized implementation with one- and two-sided options
  2. Implement a one-sided p-value by summing only tables where the odds ratio is in one direction

Real-world use cases

  • Determining if a conversion rate difference between a control and treatment group is statistically significant in an A/B test with small sample sizes.
  • Analyzing whether a rare adverse event occurs more frequently in a drug treatment group compared to a placebo in clinical trials.
  • Checking for association between two categorical variables in a 2x2 table from survey data when the expected cell counts are below five.

Sponsored

Run this sample

Open the browser IDE to tweak the example and see results without installing anything.

Open editor

More from A/B testing & experimentation

Related tutorials and quizzes for this topic.