How to Perform Intent-to-Treat Analysis in Python
Runs an intent-to-treat analysis on mock A/B test data, comparing outcomes by initial group assignment with a t-test for significance.
pip install pandas numpy scipy
Python code
55 linesimport pandas as pd
import numpy as np
def intent_to_treat_analysis(data):
"""Perform intent-to-treat (ITT) analysis.
ITT compares outcomes based on initial treatment assignment,
regardless of whether participants actually received the treatment.
"""
# Create a copy to avoid mutating the original dataframe
df = data.copy()
# In ITT, we analyze based on the assigned group, not actual treatment received
# Here we use 'assigned_treatment' column as the ITT variable
# Calculate mean outcome for each assigned group
itt_results = df.groupby('assigned_treatment')['outcome'].agg(['mean', 'std', 'count'])
# Calculate the ITT effect (difference in means)
treatment_mean = itt_results.loc[1, 'mean']
control_mean = itt_results.loc[0, 'mean']
itt_effect = treatment_mean - control_mean
# Perform a simple t-test for statistical significance
from scipy import stats
treatment_outcomes = df[df['assigned_treatment'] == 1]['outcome']
control_outcomes = df[df['assigned_treatment'] == 0]['outcome']
t_stat, p_value = stats.ttest_ind(treatment_outcomes, control_outcomes)
return {
'group_summary': itt_results,
'itt_effect': itt_effect,
't_statistic': t_stat,
'p_value': p_value
}
if __name__ == "__main__":
# Mock data: 10 participants
mock_data = pd.DataFrame({
'participant_id': range(1, 11),
'assigned_treatment': [1, 1, 1, 1, 1, 0, 0, 0, 0, 0],
'outcome': [85, 78, 92, 70, 88, 65, 72, 68, 60, 75]
})
result = intent_to_treat_analysis(mock_data)
print("Intent-to-Treat Analysis Results")
print("=" * 35)
print("\nGroup Summary:")
print(result['group_summary'])
print(f"\nITT Effect: {result['itt_effect']:.2f}")
print(f"T-Statistic: {result['t_statistic']:.2f}")
print(f"P-Value: {result['p_value']:.4f}")
Output
Intent-to-Treat Analysis Results
===================================
Group Summary:
mean std count
assigned_treatment
0 68.0 5.787918 5
1 82.6 8.503920 5
ITT Effect: 14.60
T-Statistic: 3.12
P-Value: 0.0143
How it works
Intent-to-treat analysis groups participants by their original assignment, not by the treatment they actually received. groupby('assigned_treatment')['outcome'].agg(...) computes mean, standard deviation, and count per group. The ITT effect is the difference in group means, quantifying the average causal effect of assignment. scipy.stats.ttest_ind performs an independent two-sample t-test, returning the t-statistic and p-value to assess statistical significance. This approach preserves randomization, making it the gold standard for clinical and product experiments.
Common mistakes
- Using actual treatment received instead of assigned group, which breaks randomization.
- Ignoring participants who drop out or don't adhere — ITT keeps them in their original group.
- Assuming equal variance in the t-test without checking assumptions.
- Not including a large enough sample size for reliable p-values.
Variations
- Use `stats.ttest_ind(equal_var=False)` for Welch's t-test when variances differ.
- Add confidence intervals for the ITT effect with `stats.t.interval()`.
Real-world use cases
- Evaluating a new feature rollout by comparing retention rates between randomly assigned user groups.
- Measuring the effect of a drug in a clinical trial where some patients don't adhere to the protocol.
- Assessing the impact of a marketing campaign on conversion, regardless of whether users clicked the ad.
Sponsored
More from A/B testing & experimentation
Keep learning
Related tutorials and quizzes for this topic.