Any cut point you like, not just the three that describe hands you.
What does a bad day look like? A mean cannot tell you, because a mean is designed to smooth bad days away. .quantile() is the method that answers questions about the tail: give it q=0.98 and it returns the value that 98 percent of your observations fall below.
It is also the general case of two methods you already use. .median() is .quantile(0.5), and the 25%, 50% and 75% rows of describe are .quantile([0.25, 0.5, 0.75]). Reach for quantile when you want a cut point those do not include.
Official documentation: DataFrame.quantile and Series.quantile.
The arguments that earn their keep
s.quantile(q=0.5, # one number, or a list of them
interpolation='linear')
q takes a scalar and hands back a scalar, or takes a list and hands back a Series indexed by the cut points. On a data frame, a list of q values gives you one row per quantile.
interpolation decides what to do when the cut point falls between two observations, which it usually does: 'linear' (the default, weighted between the neighbors), 'lower', 'higher', 'nearest' and 'midpoint'. Most of the time they agree closely enough not to matter. In the tail of a skewed column they do not, as below.
Note what is not in that signature: there is no skipna. .quantile() always drops missing values, and unlike .mean() and .std() it gives you no switch to say otherwise.
A worked example, on real data
The US air quality standard for fine particulates is written in two parts, and one of them is a percentile. The annual standard is a mean of 9.0 micrograms per cubic meter; the 24-hour standard is 35 micrograms per cubic meter at the 98th percentile, per the EPA's NAAQS table. A regulator who only had means could not have written the second rule.
Here is every daily PM2.5 reading the EPA collected in 2024, from its air data downloads:
import pandas as pd
url = 'https://aqs.epa.gov/aqsweb/airdata/daily_88101_2024.zip'
pm = (
pd.read_csv(url, low_memory=False,
usecols=['State Name', 'Local Site Name', 'Pollutant Standard',
'Date Local', 'Arithmetic Mean'])
.loc[lambda df_: df_['Pollutant Standard'] == 'PM25 24-hour 2012']
.rename(columns={'State Name': 'state', 'Local Site Name': 'site',
'Date Local': 'date', 'Arithmetic Mean': 'pm25'})
)
pm['pm25'].quantile([0.5, 0.9, 0.98, 0.999])
0.500 6.1
0.900 12.6
0.980 20.2
0.999 49.7
Name: pm25, dtype: float64
The mean of that column is 7.12. Half of all American monitor-days come in under 6.1, and one day in a thousand comes in over 49.7 — eight times the middle. No single average holds both of those facts; a list of q values shows the whole shape of the tail in four lines.
Now compute both halves of the standard, per monitoring site:
sites = (
pm
.groupby(['state', 'site'])['pm25']
.agg(days='count', mean='mean', p98=lambda s: s.quantile(0.98))
.loc[lambda df_: df_['days'] >= 300]
)
(
sites
.loc[lambda df_: (df_['mean'] < 9) & (df_['p98'] > 35)]
.sort_values('p98', ascending=False)
.round(1)
)
days mean p98
state site
Wyoming Sheridan - Police Sta. SLAM Site 351 7.3 51.9
California Big Bear 350 7.9 45.6
Oregon Oakridge - Willamette Activity Center (WAC) 353 8.3 41.4
North Dakota Ryder 366 7.1 39.0
Alaska A Street 691 8.7 38.3
North Dakota PAINTED CANYON 366 6.3 37.6
LOSTWOOD NWR 349 6.8 37.2
Lake Ilo 359 6.8 36.8
Oregon Klamath Falls - Peterson Elementary School (KFP) 732 8.1 36.5
North Dakota TRNP-NU 366 6.3 35.3
Oregon Lakeview - Center St & M St (LCM) 366 7.5 35.2
Eleven of the country's 848 well-monitored sites look clean on the annual number and fail the daily one. Sheridan, Wyoming averages 7.3 micrograms, well inside the limit, and its 98th percentile is 51.9 — half again over. These are places with clean air most of the year and a handful of genuinely dangerous days, and telling them apart is the entire reason the standard has two parts.
Three mistakes people make
Leaving interpolation at the default without checking. Big Bear, California has 350 readings, and the 98th percentile lands between two of them. Watch what happens:
bb = pm.loc[lambda df_: df_['site'] == 'Big Bear', 'pm25']
bb.quantile(0.98, interpolation='linear') # 45.64399999999978
bb.quantile(0.98, interpolation='lower') # 45.4
bb.quantile(0.98, interpolation='higher') # 57.6
bb.quantile(0.98, interpolation='nearest') # 45.4
bb.quantile(0.98, interpolation='midpoint') # 51.5
A 12-microgram spread, from one keyword. The top of that column reads 39.4, 45.4, 57.6, 67.3 and on up to 372.1, so the neighbors around the cut point are far apart and the rule you pick shows. When a percentile is going to be compared against a threshold, the interpolation rule is part of the answer and belongs next to the number.
Writing quantile(0.5) when you mean the median. They agree exactly — both return 6.1 here — because with q=0.5 and 'linear' they are the same computation. So pick the one that reads better. .median() says what you mean in a summary; .quantile(0.5) earns its place when that 0.5 sits in a list beside other cut points, or arrives in a variable.
Assuming the percentile covers the rows you think it does. Missing values are dropped, there is no skipna to stop that, and the output never says how many values survived — so count is a keyword you have to supply yourself. Neither the days='count' above nor the days >= 300 filter is decoration. Drop the filter and the frame holds 1,001 sites rather than 848, including Peach Springs, Arizona, which reported on exactly one day in 2024. Its 98th percentile comes back as 8.5: one reading wearing a percentile's clothes.
Where it shows up in Bamboo Weekly
Bamboo Weekly #56: Rent increases uses quantile to build bin edges rather than to report a statistic. The bins handed to pd.cut are [min, quantile(0.25), quantile(0.75), max], which sorts counties into small, medium and large by their own distribution rather than by round numbers somebody made up. Free to read.
Bamboo Weekly #35: Terrorism is quantile as a filter: df['2022 Rank'] < df['2022 Rank'].quantile(0.1) keeps the worst ten percent of countries. The post also flags the trap that the ranking runs backwards, so the tenth percentile — not the ninetieth — is the end you want. Free to read.
Bamboo Weekly #42: Plant hardiness keeps the ZIP codes whose minimum temperature rose past quantile(.9) between the 2012 and 2023 USDA maps, then counts them by state. Washington, DC tops the list with 120, which says more about how many ZIP codes DC has than about its climate. Free to read.
Bamboo Weekly #25: Entrepreneurship plays two cut points against each other — the bottom 30 percent on innovation and the top 30 percent on perceived opportunity — and intersects the resulting indexes. Thirteen countries appear in both. Free to read.
Practice it
Work through a quantile exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/quantile/
Go deeper
describe gives you the quartiles without asking, and takes a percentiles= argument when the defaults are wrong. median is the 50th percentile under a friendlier name, and mean is the statistic percentiles exist to correct. To slice a column into bands rather than read one cut point, that is cut and qcut; for each row's position in the distribution rather than the distribution's value at a position, that is rank.
More Pandas videos on Python and Pandas with Reuven Lerner.
Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.
Related methods
.median()— which is the same thing at q=0.5, with a name that reads better.cut()— when you want labelled bins rather than the cut points themselves