agg in pandas — practice examples (Bamboo Weekly)

Apply one or more aggregation functions at once, overall or per group.

agg is what you reach for when sum() alone is not enough — when you want the total and the count and the largest, each in its own column, computed in a single pass. It is also how you give those columns names you chose rather than names Pandas invented.

Official documentation: DataFrame.agg

The forms worth knowing

The one to learn first is named aggregation — each keyword becomes a column name, and each value is a (column, function) pair:

df.groupby('Region').agg(
    units=('Capacity (MW)', 'size'),
    total_mw=('Capacity (MW)', 'sum'),
    biggest=('Capacity (MW)', 'max'),
)

The older forms still work and you will meet them in other people's code:

df.groupby('Region').agg(['sum', 'mean'])          # every numeric column, both functions
df.groupby('Region').agg({'Capacity (MW)': 'sum'}) # per-column dict
df.agg({'Capacity (MW)': ['sum', 'max']})          # no groupby: whole-frame summary

Named aggregation is worth preferring because it produces flat, readable column names. The list and dict forms give you a MultiIndex on the columns, which then needs flattening before you can do anything else with it.

You can pass your own function too — agg(shortest=('Start year', lambda s: s.min())) — though a built-in string like 'min' runs in optimized code and is much faster.

A worked example, on real data

The Global Coal Plant Tracker lists every coal-fired generating unit on earth, one row per unit. Bamboo Weekly #64 used this dataset.

One question, four answers, one pass over the data:

import pandas as pd

url = ('https://www.bambooweekly.com/content/files/wp-content/uploads/2024/02/'
       'global-coal-plant-tracker-january-2024.xlsx')

(
    pd.read_excel(url, sheet_name='Units',
                  usecols=['Region', 'Capacity (MW)', 'Start year'])
    .groupby('Region')
    .agg(units=('Capacity (MW)', 'size'),
         total_mw=('Capacity (MW)', 'sum'),
         biggest=('Capacity (MW)', 'max'),
         oldest=('Start year', 'min'))
)

Which gives:

          units   total_mw  biggest  oldest
Region
Africa      431   149016.6   1980.0  1952.0
Americas   1466   457503.9   2000.0  1935.0
Asia      10129  4020051.4   6300.0  1939.0
Europe     1740   428795.9   3000.0  1927.0
Oceania     140    46937.0   2000.0  1958.0

Note that oldest aggregates a different column from the other three — named aggregation lets each output column choose its own input. Asia has more than five times the units of any other region, and the oldest plant still running anywhere dates to 1927.

Three mistakes people make

Reaching for apply when agg will do. .apply hands your function an entire sub-frame and runs in Python; .agg with a built-in name like 'sum' runs in optimized code. For standard reductions the difference is large and free.

Using the list or dict form and then fighting the column names. agg(['sum','mean']) returns a MultiIndex on the columns, so you end up writing df[('Capacity (MW)','sum')] or flattening by hand. Named aggregation avoids the problem rather than solving it.

Forgetting that size and count differ. size counts rows in the group; count counts non-missing values in a column. When there are gaps in your data these give different answers, and only one of them is the one you meant.

Watch it

Five Pandas "groupby" mistakes to avoid covers several aggregation traps, since agg and groupby almost always travel together.

Practice it

Work through an agg exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/agg/

Go deeper

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 60 Bamboo Weekly exercises that use agg on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.

pandas agg