Skip to content
Bamboo Weekly

pandas set_index

Promote one or more columns to be the data frame's index.

The index is not just row labels — it is what .loc selects by, what joins align on, and what makes time-based slicing possible. Choosing the right index is often the difference between a fiddly query and a one-liner.

Official documentation: DataFrame.set_index

The forms worth knowing

# One column becomes the index
df.set_index('time')

# Several columns become a MultiIndex
df.set_index(['Country', 'Status'])

# Keep the column as a column too, rather than consuming it
df.set_index('Country', drop=False)

# Undo it -- the index becomes an ordinary column again
df.reset_index()

set_index takes column names, not pd.col expressions, because it is asking which column rather than computing a value.

A worked example, on real data

The USGS publishes every earthquake it records as CSV, through a public API with no key — the source Bamboo Weekly #3 worked with.

Earthquakes are events in time, so the timestamp is the natural index. Once it is, you can slice by date range directly:

import pandas as pd

url = ('https://earthquake.usgs.gov/fdsnws/event/1/query.csv'
       '?starttime=2024-01-01&endtime=2024-12-31&minmagnitude=6')

(
    pd.read_csv(url, usecols=['time', 'place', 'mag'], parse_dates=['time'])
    .set_index('time')
    .sort_index()
    .loc['2024-04-01':'2024-04-04']
)

Which gives:

                                                                place  mag
time
2024-04-02 09:54:08.569000+00:00  137 km ENE of Saipan, Northern M...  6.2
2024-04-02 23:58:12.173000+00:00      15 km S of Hualien City, Taiwan  7.4
2024-04-03 00:11:25.266000+00:00    15 km NNE of Hualien City, Taiwan  6.4
2024-04-04 03:16:30.313000+00:00        77 km E of Minami-Sōma, Japan  6.1

That .loc['2024-04-01':'2024-04-04'] is the payoff. Because the index is a DatetimeIndex, Pandas understands partial date strings and slices by them — no comparison operators, no boolean mask. You could equally ask for .loc['2024-04'] to get the whole month.

The sort_index() matters: label-based slicing on an unsorted index raises an error or returns surprising results. Sort once, then slice freely.

Three mistakes people make

Slicing a DatetimeIndex that is not sorted. Range selection assumes monotonic order. If you get a KeyError on a date range that clearly exists, chain .sort_index() first.

Losing the column you indexed on. By default set_index moves the column into the index, so it is no longer available as a column. Pass drop=False if you need it in both places.

Setting an index that is not unique, then being surprised by .loc. A non-unique index is legal, but .loc['x'] then returns every matching row rather than one — a frame instead of a series. That is often what you want with a MultiIndex, and rarely what you want otherwise.

Watch it

Pandas time series superpowers: Why datetime indexes matter covers exactly the payoff above, and Load your data with the right index in Pandas shows how to set it at load time instead.

Practice it

Work through a set_index exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/set-index/

Go deeper

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 106 Bamboo Weekly exercises that use set_index on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.