Skip to content
Bamboo Weekly

pandas rename

Give columns or index labels new names, using a dict or a function.

How much of your analysis code is spent typing out a column name that somebody else chose badly? Government agencies and statistical offices are wonderfully generous with their data and remarkably careless with their headers. You will get Time Period Start Date when you wanted start, SITE_LATITUDE when you wanted latitude, and a column whose name is an entire sentence explaining the methodology. Those names then show up in every expression you write and in every plot legend you show somebody else.

rename is the fix. You hand it a mapping — a dict from old name to new name — and it returns a new data frame with those labels replaced. Anything you do not mention is left alone, which is what separates it from the alternatives: you supply only the names you want changed. It works on the columns, on the index, on one level of a MultiIndex, and it accepts a function as easily as it accepts a dict.

Official documentation: DataFrame.rename

The arguments that earn their keep

df.rename(columns={'old': 'new'},   # a dict, or a function, for the columns
          index={'old': 'new'},     # the same, for the row labels
          level='Group',            # restrict to one level of a MultiIndex
          errors='raise')           # complain if a key does not exist

df.rename(str.lower, axis='columns')   # positional mapper plus axis=

There are two ways to say the same thing. Either you name the axis with the columns= or index= keyword, or you pass the mapper positionally and say which axis it applies to with axis=. They do exactly the same work; I prefer the keyword form for a dict, because columns={...} reads as a sentence.

The mapper itself can be a dict or any callable. A dict renames the labels it mentions and ignores the rest. A callable is applied to every label on that axis, and whatever it returns becomes the new label — so str.lower lowercases everything, and a lambda can strip whitespace, remove a prefix, or cut a long sentence down to something that fits in a legend.

A worked example, on real data

The CDC's long covid pulse survey, the data behind Bamboo Weekly #59, is a good specimen: a three-level row index, a column whose name is six words long, and indicator strings that no chart legend will ever survive.

import pandas as pd

url = 'https://data.cdc.gov/api/views/gsea-w83j/rows.csv?accessType=DOWNLOAD'

df = pd.read_csv(url,
                 usecols=['Phase', 'Group', 'Subgroup', 'Indicator',
                          'Time Period Start Date', 'Value'],
                 parse_dates=['Time Period Start Date'],
                 index_col=['Phase', 'Group', 'Subgroup'])

df.columns

Index(['Indicator', 'Time Period Start Date', 'Value'], dtype='str')

Two of those three names are worse than they need to be. A dict fixes them and leaves Indicator alone:

df = df.rename(columns={'Time Period Start Date': 'start',
                        'Value': 'pct'})
df.columns

Index(['Indicator', 'start', 'pct'], dtype='str')

When the change is mechanical rather than specific, pass a function instead and skip the dict entirely:

df.rename(str.lower, axis='columns').columns

Index(['indicator', 'start', 'pct'], dtype='str')

str.lower is an ordinary Python function, and rename calls it once per label. A lambda works the same way, which is how you deal with the headers that arrive with stray whitespace: df.rename(columns=lambda c: c.strip()).

Now the row labels. Pivot the age breakdown into a table and the index is a set of age bands, each with a redundant years on the end:

ages = (
    df
    .loc[df['Indicator'] ==
         'Ever experienced long COVID, as a percentage of all adults']
    .xs('By Age', level='Group')
    .reset_index()
    .pivot_table(index='Subgroup', columns='Phase', values='pct')
    .round(1)
)

ages.rename(index=lambda s: s.removesuffix(' years'))

Phase                3.1   3.5   3.6   3.7   3.8   3.9   4.0   4.1   4.2
Subgroup
18 - 29             16.4  16.4  15.6  16.0  16.7  16.4  19.2  19.0  18.2
30 - 39             16.8  16.6  16.7  16.0  16.7  17.1  19.9  20.5  18.8
40 - 49             17.8  17.8  17.4  18.1  17.7  18.8  20.2  21.6  21.2
50 - 59             15.9  15.9  15.8  16.1  16.9  17.1  19.4  19.7  21.4
60 - 69             11.9  11.1  11.2  12.3  12.6  13.2  15.0  16.0  15.1
70 - 79              8.7   8.1   7.7   8.3   9.3   9.2  11.1  11.1  11.1
80 years and above   8.3   5.8   6.9   7.6   8.9   8.3   9.6  10.9  11.1

Notice the last row. removesuffix did nothing to 80 years and above, and rename reported no problem, because a callable that returns a label unchanged is a legal way to say "leave this one alone." To shorten that row too, add a dict pass: .rename(index={'80 years and above': '80+'}).

Renaming one level of a MultiIndex

The original data frame has three index levels, and almost every label on the Group level begins with the same wasted word:

df.index.get_level_values('Group').unique()

Index(['National Estimate', 'By Age', 'By Sex', 'By Gender identity',
       'By Sexual orientation', 'By Race/Hispanic ethnicity', 'By Education',
       'By Disability status', 'By State'],
      dtype='str', name='Group')

Stripping the By off them makes the table read better. But index= on a MultiIndex applies to every level, and the Phase level holds floats:

df.rename(index=lambda s: s.removeprefix('By '))

AttributeError: 'float' object has no attribute 'removeprefix'

level= is the answer. Name the level and Pandas applies the mapper only there:

df.rename(index=lambda s: s.removeprefix('By '), level='Group')

Index(['National Estimate', 'Age', 'Sex', 'Gender identity',
       'Sexual orientation', 'Race/Hispanic ethnicity', 'Education',
       'Disability status', 'State'],
      dtype='str', name='Group')

level= takes a name or an integer position, and it works with a dict mapper just as well as with a function.

Four mistakes people make

A misspelled key does nothing, quietly. This is the one that costs the most time. Ask to rename a column that does not exist and Pandas shrugs:

df.rename(columns={'Values': 'percent'}).columns

Index(['Indicator', 'start', 'pct'], dtype='str')

No error, no warning, and a KeyError three cells later when you go looking for percent. The default is errors='ignore', which is useful when one standard mapping is being applied to several files with different columns, and a trap the rest of the time. When you know the label is there, say so:

df.rename(columns={'Values': 'percent'}, errors='raise')

KeyError: "['Values'] not found in axis"

Renaming the index labels when you meant the index name. These are different things, and the language does not help. The labels are the values in the index; the name is the title printed above them. rename changes labels, rename_axis changes names. So df.rename(index={'Subgroup': 'subgroup'}) matches nothing and leaves the index names as ['Phase', 'Group', 'Subgroup'], while df.rename_axis(index={'Subgroup': 'subgroup'}) gives you ['Phase', 'Group', 'subgroup']. The same applies sideways: df.rename_axis(columns='measure') names the column axis without touching a single column name.

Reaching for inplace=True. It still works, and it still returns None, which means the moment you use it your method chain ends. df.rename(columns={...}, inplace=True) cannot be followed by .plot.line(), so a chain that was building nicely has to be broken into statements and a variable. Under Pandas 3's copy-on-write rules it does not buy you the memory saving people imagine, either. Assign the result instead, or better, keep renaming as one link in the chain.

Renaming two columns to the same thing. rename does not check for collisions. Ask for a name another column already has and you get duplicate labels — Index(['Indicator', 'pct', 'pct']) — which is legal, and turns the next df['pct'] into a two-column data frame rather than a series.

rename, rename_axis, or set_axis?

Three methods, one area of overlap, and a lot of wasted afternoons. rename changes some labels, by mapping. rename_axis changes the name of an axis and never its labels. set_axis replaces every label at once, positionally:

ages.head(3).set_axis(['young', 'thirties', 'forties'], axis='index')

Phase      3.1   3.5   3.6   3.7   3.8   3.9   4.0   4.1   4.2
young     16.4  16.4  15.6  16.0  16.7  16.4  19.2  19.0  18.2
thirties  16.8  16.6  16.7  16.0  16.7  17.1  19.9  20.5  18.8
forties   17.8  17.8  17.4  18.1  17.7  18.8  20.2  21.6  21.2

set_axis demands the full list and counts it: give it the wrong number and you get ValueError: Length mismatch: Expected axis has 7 elements, new values have 2 elements. That strictness is the point. Use set_axis when you are replacing all the names anyway and want Pandas to catch a miscount, and rename when you are changing a few names out of many. There is a fuller treatment on the reset_index page, which covers set_axis alongside the other ways of moving labels around.

One more overlap worth knowing: on a series, rename does both jobs depending on what you pass it. A dict or a function renames the index labels; a plain string renames the series itself. s.rename('percent') changes Name: pct to Name: percent and leaves every label alone.

Where it shows up in Bamboo Weekly

More than two dozen Bamboo Weekly solutions call rename. Four worth studying, all free to read:

Bamboo Weekly #51: Academy Awards is the plainest possible case, and the most common one in practice: .rename(columns={'CanonicalCategory': 'Category'}), a single column, straight after read_csv, inside the chain that builds the data frame.

Bamboo Weekly #59: Long covid is where the callable form earns its keep. The CDC's indicator names are full sentences, so the solution stacks three rename calls with lambdas — removesuffix('long COVID, as a percentage of all adults'), then removesuffix('from '), then removesuffix(' years') — each trimming one more piece, until the chart labels are readable.

Bamboo Weekly #29: Auto accidents shows that the dict does not have to be typed out. The OECD data uses three-letter country codes, and the solution builds the mapping from another data frame with .rename(columns=country_series.to_dict()), turning USA and DEU into real country names for the chart.

Bamboo Weekly #46: Pedestrians renames a label that was never a string. After value_counts().unstack() the columns are integers, and .rename(columns={0: 'noped'}) turns the zero-pedestrian column into something you can name in a list of columns to plot. Keys match by value and by type, so 0 and '0' are different labels.

Practice it

Work through a .rename() exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/rename/

Go deeper

rename is usually one link in a longer chain, and its neighbors are worth knowing: reset_index for moving labels between the index and the columns, and for set_axis when you are replacing all the labels at once, set_index for the other direction, filter for selecting columns by name once you have fixed them, and drop for the ones you would rather not rename at all. The official docs for DataFrame.rename_axis and Series.rename cover the two corners that surprise people most.

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 29 Bamboo Weekly exercises that use rename on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.