How to Conduct a Two-Sample T-Test in Python

Performs Welch's t-test for two independent samples, computing the t-statistic, degrees of freedom, and p-value using NumPy and SciPy.

Medium Python 3.9+ Aug 9, 2026 A/B testing & experimentation 15 views 0 copies

statistics hypothesis-testing t-test numpy scipy a-b-testing
Requires third-party packages — install first
pip install numpy scipy

Python code

41 lines
Python 3.9+ 
import numpy as np

def two_sample_t_test(sample1, sample2):
    """Perform Welch's t-test for two independent samples."""
    n1, n2 = len(sample1), len(sample2)
    mean1, mean2 = np.mean(sample1), np.mean(sample2)
    var1, var2 = np.var(sample1, ddof=1), np.var(sample2, ddof=1)

    # Standard error of difference
    se = np.sqrt(var1 / n1 + var2 / n2)

    # t statistic
    t_stat = (mean1 - mean2) / se

    # Degrees of freedom (Welch-Satterthwaite)
    df = (var1 / n1 + var2 / n2) ** 2 / (
        (var1 / n1) ** 2 / (n1 - 1) + (var2 / n2) ** 2 / (n2 - 1)
    )

    # Two-tailed p-value
    from scipy import stats
    p_value = 2 * (1 - stats.t.cdf(abs(t_stat), df))

    return {
        "t_statistic": t_stat,
        "degrees_of_freedom": df,
        "p_value": p_value,
        "mean_difference": mean1 - mean2,
    }

if __name__ == "__main__":
    # Example data
    group_a = [5.1, 4.9, 5.2, 4.8, 5.0]
    group_b = [4.2, 4.5, 4.3, 4.1, 4.4]

    result = two_sample_t_test(group_a, group_b)
    print("Two-sample t-test results:")
    print(f"  t-statistic: {result['t_statistic']:.4f}")
    print(f"  degrees of freedom: {result['degrees_of_freedom']:.2f}")
    print(f"  p-value: {result['p_value']:.4f}")
    print(f"  mean difference (A-B): {result['mean_difference']:.4f}")

Output

stdout 
Two-sample t-test results:
  t-statistic: 4.5139
  degrees of freedom: 7.78
  p-value: 0.0022
  mean difference (A-B): 0.7000

How it works

Welch's t-test handles samples with unequal variances, making it more robust than Student's t-test for real-world data. The function uses np.var with ddof=1 to compute sample variance, ensuring unbiased estimates. Degrees of freedom are calculated via the Welch-Satterthwaite equation, which adjusts for unequal sample sizes and variances. A p-value below 0.05 indicates a statistically significant difference between group means. This is a foundational statistical test for comparing experimental conditions.

Common mistakes

  • Using population variance (`ddof=0`) instead of sample variance (`ddof=1`) for sample data
  • Assuming equal variances and applying Student's t-test when samples have different standard deviations
  • Misinterpreting p-value as the probability that the null hypothesis is true
  • Forgetting to import scipy.stats, causing a NameError

Variations

  1. Use scipy.stats.ttest_ind(a, b, equal_var=False) for a one-liner Welch's t-test
  2. Apply a paired t-test with scipy.stats.ttest_rel when samples come from the same subjects

Real-world use cases

  • Compare conversion rates between a control group and a test group in an A/B test.
  • Evaluate whether a new drug lowers blood pressure more than a placebo in clinical studies.
  • Determine if a website redesign significantly improves user engagement metrics such as session duration.

Sponsored

Run locally

This sample needs third-party packages, so it cannot run in the browser IDE. Copy the code above, install the packages shown at the top, then run it in your own Python environment.

More from A/B testing & experimentation

Keep learning

Related tutorials and quizzes for this topic.

Tutorials

Quizzes