Descriptive Statistics in pandas

Raw numbers in a spreadsheet rarely tell a story — the average of a column is a single value, but it hides the spread, the outliers, and the shape of the data. When you're handed a new dataset, your first instinct is to run a quick summary: What's typical? How much variation is there? That's where pandas descriptive statistics come in. With just a few lines of code, you can compute the mean, median, standard deviation, and other key metrics that transform columns of raw values into actionable insights.

The problem this lesson solves

You've loaded a DataFrame, say sales records or customer feedback scores, but you're staring at thousands of rows. How do you quickly answer questions like: "What's the average order value?" or "Is the data heavily skewed?" Manually scanning rows is impractical and error-prone. Without a systematic way to summarize, you might miss outliers or misunderstand the distribution, leading to bad decisions.

The pandas library provides a suite of methods — describe(), mean(), median(), std(), and more — that compute descriptive statistics in seconds. These functions turn raw data into meaningful metrics, giving you a high-level view of central tendency, dispersion, and distribution shape. This lesson closes the gap between raw data and interpretable insights, a critical step in any data analysis workflow.

Core concept / mental model

Think of descriptive statistics as the vitamin label for your data. Just as a nutrition label summarizes the contents of a food package (serving size, calories, fat), descriptive statistics summarize a dataset's key characteristics: central tendency (where the data clusters), spread (how variable it is), and shape (symmetry and tails).

In pandas, these metrics are computed using vectorized operations — meaning they apply to whole columns efficiently, without writing explicit loops. The core methods you'll use daily are:

• mean() — the arithmetic average
• median() — the middle value when sorted
• mode() — the most frequent value
• std() — the standard deviation, measuring spread
• min() and max() — range endpoints
• quantile() — percentiles (e.g., the 25th percentile)
• describe() — a one-shot summary of multiple stats

These methods are column-wise by default, meaning they compute the statistic for each column independently. For example, df.mean() returns the mean of each numeric column as a Series.

Imagine a simple DataFrame of exam scores:

The center (mean) is 84.3, but the scores vary by about 6.1 points (standard deviation). That summary tells you more than any single row.

How it works step by step

Computing descriptive statistics in pandas follows a predictable pattern:

1. Import pandas and load your data into a DataFrame (from CSV, Excel, or a dictionary).
2. Select the relevant columns — often numeric ones, but you can also summarize categorical data using value_counts().
3. Call the statistic method on the DataFrame or a specific column (e.g., df['price'].mean()).
4. Inspect the results — often a Series or a DataFrame for describe().
5. Handle missing values — by default, methods skip NaN, but you can control this with the skipna parameter.
6. Interpret the results in context — is the mean higher than the median? That signals skewness.

For example, computing means and standard deviations across numeric columns is as simple as:

import pandas as pd

# Sample DataFrame
df = pd.DataFrame({
    'price': [10, 15, 20, 25, 30],
    'quantity': [2, 3, 5, 8, 13]
})

print(df.mean())
print(df.std())

Output:

price      20.0
quantity    6.2
dtype: float64

price      7.905694
quantity   4.324350
dtype: float64

Hands-on walkthrough

Let's put this into practice with a realistic dataset. We'll create a DataFrame of monthly sales and compute comprehensive descriptive statistics.

First, create your data:

import pandas as pd

# Sales data (in thousands) for 12 months
sales_data = {
    'month': ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun',
              'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'],
    'revenue': [12.5, 15.2, 13.8, 22.1, 19.4, 25.0,
                23.3, 18.7, 21.0, 27.5, 24.9, 30.2]
}
sales_df = pd.DataFrame(sales_data)
print(sales_df)

Output (first few rows):

   month  revenue
0    Jan     12.5
1    Feb     15.2
...

Now compute the key statistics:

# Basic statistics
mean_revenue = sales_df['revenue'].mean()
median_revenue = sales_df['revenue'].median()
std_dev = sales_df['revenue'].std()
min_revenue = sales_df['revenue'].min()
max_revenue = sales_df['revenue'].max()

print(f"Mean: {mean_revenue:.2f}")
print(f"Median: {median_revenue:.2f}")
print(f"Std Dev: {std_dev:.2f}")
print(f"Range: {min_revenue} - {max_revenue}")

Output:

Mean: 20.38
Median: 20.20
Std Dev: 5.47
Range: 12.5 - 30.2

But the fastest way to get a full summary is describe():

summary = sales_df['revenue'].describe()
print(summary)

Output:

count    12.000000
mean     20.383333
std       5.470765
min      12.500000
25%      16.550000
50%      20.200000
75%      24.450000
max      30.200000
Name: revenue, dtype: float64

Pro tip: Use include='all' in describe() to include non-numeric columns — it will show counts, unique values, and the top value for categorical columns.

Compare options / when to choose what

You have several methods at your disposal, but which one should you use? Here's a quick comparison:

For example, if your data has a few extreme outliers (like billionaire income), the median is a better measure of central tendency than the mean. If you need a single comprehensive report, describe() is your best friend.

Variations: - Use df.describe(percentiles=[0.1, 0.5, 0.9]) to custom percentiles. - Use df.agg(['mean', 'std', 'min', 'max']) to compute multiple statistics in one pass. - Use groupby() with these methods to compute statistics per category (e.g., average sales by region).

Troubleshooting & edge cases

Encountering errors or unexpected results is common. Here are the issues you'll likely face:

Problem 1: Missing values (NaN) Pandas methods skip NaN by default (skipna=True). But if you need to include them or want to know how many are missing, check with isna().sum().

# Count missing values in a column
print(df['revenue'].isna().sum())

Problem 2: Non-numeric data Calling mean() on a string column raises a TypeError. Filter to numeric columns first:

numeric_df = df.select_dtypes(include='number')
print(numeric_df.mean())

Problem 3: Empty or all-NaN columns If a column is empty after dropping missing values, mean() returns NaN. Check with empty before computing.

Problem 4: Mode() returns multiple values When multiple values tie, mode() returns a Series — remember to access it like a Series, not a scalar.

modes = df['category'].mode()
print(modes)

Problem 5: Interpretations skewed by outliers A high mean vs. low median signals outliers. Always combine mean() and median() to detect skewness.

What you learned & what's next

In this lesson, you've learned to compute descriptive statistics in pandas to summarize data quickly. You can now: - Use describe() for a comprehensive summary - Extract specific metrics like mean, median, and standard deviation - Choose the right metric for your data's characteristics - Handle common edge cases like missing or non-numeric data

These skills are foundational for any data analysis task. Next, you'll explore data aggregation and grouping, where you'll learn to compute these statistics per category (e.g., by product or region) using groupby(). That will unlock even deeper insights into your data. Keep practicing with your own datasets!