Aggregate with Named and Custom Functions

You've mastered groupby basics — splitting data into groups and applying built-in functions like sum() and mean(). But then reality hits: your manager asks for a report with five different statistics, each with a clear column name, and one metric that doesn't exist out of the box, like a weighted average or a range. Chaining agg() with lambdas and renaming columns gets messy fast. This lesson unlocks the aggregate with named and custom functions pattern in pandas — a clean, readable, and powerful way to compute multiple aggregations in one shot, using both pre-built functions and your own logic.

The problem this lesson solves

Raw agg() calls work, but they quickly become unwieldy:

• Anonymous columns: pandas names aggregated columns like 'Total', 'Mean', or worse, '<lambda>'. You then need extra rename() steps.
• Multiple aggregations: Applying the same function to many columns, or many functions to one column, produces a confusing MultiIndex column structure.
• Custom logic: Built-ins cover sum, mean, median, etc., but not domain-specific metrics like percent change or range. You end up with a jumble of apply() and temporary columns.

Without a structured approach, your code grows fragile. This lesson introduces two pandas features that solve these headaches: named aggregation (via agg() with a dictionary of column -> (name, function) pairs) and custom aggregation functions (user-defined functions passed to agg() or apply()).

Core concept / mental model

Think of groupby().agg() as a mini reporting engine. You feed it a spec — "for each group, give me these columns with these calculations" — and it returns a clean DataFrame.

Named aggregation lets you write that spec as a dictionary:

(
    df.groupby('group')
      .agg(total=('amount', 'sum'),  # "total" column = sum of 'amount'
           mean_price=('price', 'mean'))
)

Each key becomes the output column name; each tuple (column, function) says which input column and which function to use. No renaming afterward.

Custom functions are just regular Python functions (or lambdas) you define and pass anywhere a function name is expected:

def price_range(group_col):
    return group_col.max() - group_col.min()

# Later:
.agg(price_span=('price', price_range))

Pandas calls your function on a Series (the input column values for each group) and uses the return value as the aggregated result.

How it works step by step

1. Build your spec dictionary. Keys are new column names. Values are tuples (source_column, function).
2. Choose the function. You can use: - A string alias: 'sum', 'mean', 'count', 'median', 'min', 'max', 'std', etc. - A callable: built-in like sum or np.mean, or your own function/lambda.
3. Apply with .agg() on the grouped object. For each key in the spec, pandas: - Extracts the source column as a Series for each group, - Calls the function on that Series, - Stores the result under the new column name.
4. Result is a DataFrame with one row per group and one column per spec key. Group keys become the index (or columns if you use as_index=False).

You can mix named aggregations with custom functions in the same agg() call — the spec remains flat and readable.

Hands-on walkthrough

Let’s apply this to a real scenario: sales data for three regions, with daily amounts and prices.

import pandas as pd

sales = pd.DataFrame({
    'region': ['East', 'East', 'West', 'West', 'North', 'North'],
    'amount': [100, 250, 150, 200, 300, 120],
    'price': [10, 12, 8, 9, 11, 10],
})

# 1. Basic named aggregation
result = (
    sales.groupby('region')
         .agg(total_sales=('amount', 'sum'),
              avg_price=('price', 'mean'))
)
print(result)

Output:

        total_sales  avg_price
region
East            350       11.0
North           420       10.5
West            350        8.5

Now add a custom function to compute the price range and a weighted average (total value / total amount):

def price_range(s):
    return s.max() - s.min()

def weighted_avg(group):
    # group is a DataFrame; we compute sum(amount*price)/sum(amount)
    return (group['amount'] * group['price']).sum() / group['amount'].sum()

result2 = (
    sales.groupby('region')
         .agg(total_sales=('amount', 'sum'),
              price_span=('price', price_range),
              weighted_avg=('amount', weighted_avg))  # custom on the group
)
print(result2)

Output:

        total_sales  price_span  weighted_avg
region
East            350           2     10.571429
North           420           1     10.714286
West            350           1      8.428571

Notice: weighted_avg receives the whole group (a DataFrame), not just one column. For single-column functions, pass the column Series. For multi-column logic, use a function that takes the entire group.

Pro tip: Use lambdas for quick custom stats, but define a named function if you reuse it or it’s more than one line — readability wins.

Compare options / when to choose what

Rule of thumb: Start with named aggregations using built-ins. Add custom functions when you need a specific calculation that pandas doesn’t provide. Avoid mixing many apply() calls — named aggregation keeps everything in one clean agg().

Troubleshooting & edge cases

• KeyError on column names: Your spec tuple uses a column that doesn’t exist. Double-check spelling and case.
• TypeError: agg() got an unexpected keyword argument: Forgetting parentheses? In older pandas, you need agg(total=('amount', 'sum')) — not agg(total). The dictionary form always works.
• Custom function returning multiple values: Your function should return a scalar. If you return a Series, pandas will create a MultiIndex column — that’s usually messy. Instead, split into separate named aggregations.
• Lambda with multiple columns: Lambdas in named aggregation get only one Series (the source column). For multi-column logic, define a function that takes the whole group.
• Empty groups: If a group has no rows (e.g., after a filter), mean returns NaN, sum returns 0. Handle with min_count if needed.

What you learned & what's next

You now know how to aggregate with named and custom functions — building clean, single-agg() reports that mix built-in statistics with your own logic. You can:

• Use named aggregation to control output column names and functions.
• Write and apply custom aggregation functions for metrics like range, weighted averages, or custom formulas.
• Avoid messy rename() and MultiIndex headaches.

Next step: In the following lesson, you'll learn how to apply multiple aggregations at once using agg() with lists — perfect for computing ['sum', 'mean', 'count'] for every column in a single call. Named and custom functions you mastered here will compose naturally with that pattern.

Practice this skill: take any dataset, group by a category, and produce a summary with at least two named aggregations and one custom metric. You’re ready.