Skip to content
Bamboo Weekly

pandas drop

Remove columns or rows by label.

Real spreadsheets arrive with columns you did not ask for — blank separators, repeated headers, notes the analyst left in cell J1. drop is how you get rid of them, and it is one of the first things you reach for when a file was built for humans rather than for Pandas.

Official documentation: DataFrame.drop

The forms worth knowing

# Remove columns by name
df.drop(columns=['Notes', 'Unnamed: 3'])

# Remove one column
df.drop(columns='Notes')

# Remove rows by index label
df.drop(index=[0, 1])

# Do not complain if a label is absent
df.drop(columns=['Notes'], errors='ignore')

drop takes column names, not pd.col expressions. And note the modern spelling: columns= and index= are clearer than the older axis=1 form, which forces the reader to remember which axis is which.

To remove rows by a condition rather than a label, use .loc with the condition negated — that is what .loc is for:

df.loc[~(pd.col('Status') == 'cancelled')]

A worked example, on real data

The New York Fed publishes its Household Debt and Credit report as a spreadsheet built for reading, not for parsing. Bamboo Weekly #54 worked with it.

Load one sheet and look at what arrives:

import pandas as pd

url = ('https://www.bambooweekly.com/content/files/medialibrary/interactives/'
       'householdcredit/data/xls/hhd_c_report_2023q4.xlsx')

df = pd.read_excel(url, sheet_name='Page 3 Data', nrows=8)
list(df.columns)

['Total Debt Balance and Its Composition', 'Unnamed: 1', 'Unnamed: 2',
 'Unnamed: 3', 'Unnamed: 4', 'Unnamed: 5']

Five of the six columns are Unnamed — Pandas' way of saying the spreadsheet had a merged title row and no real header there. Those columns are structure, not data.

drop wants actual labels, so build the list of names to remove:

(
    lambda d: d.drop(columns=[c for c in d.columns if str(c).startswith('Unnamed')])
)(pd.read_excel(url, sheet_name='Page 3 Data', nrows=8))

That is awkward inside a chain, because you need the column names before you can name them. For a pattern like this, filter is the cleaner tool — it selects columns by regular expression, and a negative lookahead keeps everything that does not match:

(
    pd.read_excel(url, sheet_name='Page 3 Data', nrows=8)
    .filter(regex='^(?!Unnamed)')
)

Either way you are left with:

['Total Debt Balance and Its Composition']

Note that drop(columns=...) does not accept a function — passing one raises KeyError: '[<function <lambda>>] not found in axis', because it treats your function as a label to look up. Labels only.

Three mistakes people make

Expecting drop to modify the frame. It returns a new one. df.drop(columns='x') on its own line accomplishes nothing — keep the result or chain onto it.

Using drop where .loc belongs. Removing rows by condition through drop means first computing the labels to remove, then dropping them: two steps where a negated .loc is one. Reserve drop for the case where you genuinely know the labels.

Being surprised by KeyError on a column that is not there. drop raises if a label is missing, which is usually right — a typo should be loud. When you are deliberately clearing columns that may or may not exist, pass errors='ignore'.

Watch it

Messy spreadsheets are a genre of their own: Cleaning messy Excel data with Pandas.

Practice it

Work through a drop exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/drop/

Go deeper

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 74 Bamboo Weekly exercises that use drop on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.