Skip to content
Bamboo Weekly

pd.col

Refer to a column before the data frame has a name — the piece that makes method chaining work in Pandas 3.

pd.col('mag') does not fetch anything. It is a deferred reference: a description of "the column called mag", which Pandas resolves later, against whatever data frame it is eventually handed. That indirection is what lets you write a filter inside a chain, where the intermediate frame exists but has no variable name to point at.

Official documentation: pandas.col

The problem it solves

Outside a chain, filtering is easy, because there is a df to refer to twice:

df[df['mag'] > 7]

Inside a chain there is no such name. The frame produced by read_csv and then modified by assign has never been assigned to anything:

(
    pd.read_csv(url)
    .assign(shallow=...)
    .loc[???]          # what do we call the frame at this point?
)

pd.col is the answer to ???. You describe the column, and Pandas supplies the frame:

(
    pd.read_csv(url)
    .assign(shallow=pd.col('depth') < 70)
    .loc[pd.col('mag') > 7]
)

Where pd.col works

Anywhere Pandas is expecting a value:

# Comparisons and arithmetic
df.loc[pd.col('mag') > 7]
df.assign(total=pd.col('a') + pd.col('b'))

# One column compared against another
df.loc[pd.col('b') > pd.col('a') * 5]

# Combined conditions -- parenthesize each side
df.loc[(pd.col('mag') > 7) & (pd.col('depth') < 50)]
df.loc[~(pd.col('mag') > 7)]

# Accessors
df.assign(year=pd.col('time').dt.year)
df.assign(shouty=pd.col('place').str.upper())

# The usual series methods
df.loc[pd.col('region').isin(['Asia', 'Europe'])]
df.loc[pd.col('mag').between(6, 7)]
df.assign(capped=pd.col('depth').clip(upper=100))
df.assign(filled=pd.col('mag').fillna(0))
df.assign(as_float=pd.col('mag').astype('float'))

It also works in .loc assignment, which is the correct cure for chained indexing:

df.loc[pd.col('mag') > 7, 'notable'] = True

And one genuinely handy trick: inside a single assign, pd.col can refer to a column created earlier in that same call, because assign evaluates its arguments in order:

df.assign(doubled=pd.col('a') * 2,
          plus_one=pd.col('doubled') + 1)

Where it does not work — and the rule

pd.col describes a value, not a name. Wherever pandas is asking which
column, it wants a plain string, and pd.col fails.

So: .loc and .assign take pd.col. groupby, sort_values and drop take
strings. Once you have that split, the rest is intuition — and each of the
errors below is that same rule, arriving in a different disguise.

One thing worth knowing before the errors: .loc is not actually required for
filtering. df[pd.col('mag') > 7] works exactly like df.loc[pd.col('mag') > 7],
because both hand pandas a value. However, I strongly recommend that you use df.loc, because it's just easier to remember and work with. Also, .loc is still required when you are assigning.

AttributeError: module 'pandas' has no attribute 'col'

import pandas as pd
pd.col('mag')
# AttributeError: module 'pandas' has no attribute 'col'

You are on pandas 2.x. pd.col was added in pandas 3.0, released 21 January
2026, and there is no backport.

import pandas as pd
pd.__version__      # if this starts with a 2, that is your answer

The fix is pip install --upgrade pandas (or uv add 'pandas>=3'). If you
cannot upgrade — a pinned production environment, a shared notebook server —
the lambda form does the same job and still works in both versions:

df.loc[lambda df_: df_['mag'] > 7]
df.assign(shallow=lambda df_: df_['depth'] < 70)

That is the syntax pd.col replaced, not a lesser alternative. See The lambda
form it replaces below.

ValueError: expr must be a string to be evaluated

df.query(pd.col('mag') > 7)
# ValueError: expr must be a string to be evaluated,
#             <class 'pandas.api.typing.Expression'> given

query is the one place this mistake is actively invited: the official
documentation for pandas.col lists DataFrame.query under See Also, so it
looks like the two are meant to go together. They are not.

query takes a string and parses it itself. It never sees a pd.col
object as anything but the wrong type:

df.query('mag > 7')             # query's own mini-language, as a string
df.loc[pd.col('mag') > 7]       # pd.col's job

Both filter. They are two separate mechanisms for deferring a reference to a
column, and pd.col is the one that composes with the rest of a chain — which
is usually why you wanted it.

TypeError: boolean value of an expression is ambiguous

df.groupby(pd.col('region'))
# TypeError: boolean value of an expression is ambiguous

This one is worth reading twice, because the message is unhelpful and you will
meet it. Nothing here is boolean. What has happened is that groupby received
an object it did not expect and tried to test it for truthiness, and
Expression refuses to answer that question.

groupby wants the name of a column:

df.groupby('region')

If you want to group by something that does not exist yet, build it first with
assign — where pd.col is welcome — and then group by its name:

(
    df
   .assign(shallow=pd.col('depth') < 70)
   .groupby('shallow')
   .size()
)

That two-step is the general shape of the fix whenever a name is wanted and you
only have a description.

KeyError: col('mag')

df.sort_values(pd.col('mag'))
# KeyError: col('mag')

A more honest message than the groupby one: pandas took the Expression,
used it as a dictionary key against your columns, and found no column literally
called col('mag').

sort_values wants a name:

df.sort_values('mag', ascending=False)

This is the error you are most likely to hit in a chain that is otherwise
correct, because sorting usually comes after several pd.col steps and the
hand keeps typing. It is normal for one chain to mix both forms — three
pd.col references and then a plain string for the sort.

TypeError: unhashable type: 'Expression'

df.drop(columns=pd.col('mag'))
# TypeError: unhashable type: 'Expression'

The same cause as the KeyError above, caught one step earlier. drop wants to
put the column name in a set, and an Expression cannot be hashed.

df.drop(columns='mag')
df.drop(columns=['mag', 'depth'])

The same applies anywhere else you pass column names as labels — rename,
set_index, usecols in read_csv. All of them want strings.

AttributeError: 'Series' object has no attribute 'columns'

s = df['mag']
s.loc[pd.col('mag') > 7]
# AttributeError: 'Series' object has no attribute 'columns'

pd.col works on a DataFrame. It never works on a Series. That is the whole
rule, and this single error is how every violation of it announces itself:

s.loc[pd.col('mag') > 7]              # AttributeError
s[pd.col('mag') > 7]                  # AttributeError
s.where(pd.col('mag') > 7)            # AttributeError
s.case_when([(pd.col('mag') > 7, 0)]) # AttributeError

Four different methods, one identical message — because the cause is the same
every time. pd.col('mag') means "the column called mag, in whatever frame is
being worked on." A Series is not a frame. It has no columns, so there is
nothing for the reference to resolve against, and pandas says so in the most
literal way available.

On a Series, use a lambda. It is not a downgrade; it is the right tool,
because a lambda receives the object itself rather than describing a column of
something larger:

s.loc[lambda x: x > 7]
s[lambda x: x > 7]
s.where(lambda x: x > 7)
s.case_when([(lambda x: x > 7, 0)])

So the working split is:

You have Use
A DataFrame — .loc, .assign, [...] pd.col
A Series a lambda

And note that this is about the object, not the method. case_when is a good
example: called on a bare Series it fails, but reached through assign — where
there is a frame — pd.col is welcome, in both the condition and the result:

df.assign(size=pd.col('mag').case_when([(pd.col('mag') > 7, 'major')]))
df.assign(x=pd.col('a').case_when([(pd.col('b') > 3, pd.col('b'))]))

Which is the rule one more time: pd.col needs a frame. Every error on this
page is that sentence, wearing a different hat.

A worked example, on real data

The USGS publishes every earthquake it records as CSV, through a public API with no key — the same source Bamboo Weekly #3 worked with. Let's find 2024's largest shallow quakes — shallow ones do far more damage for a given magnitude.

import pandas as pd

url = ('https://earthquake.usgs.gov/fdsnws/event/1/query.csv'
       '?starttime=2024-01-01&endtime=2024-12-31&minmagnitude=6')

(
    pd.read_csv(url, usecols=['time', 'place', 'mag', 'depth'], parse_dates=['time'])
    .assign(month=pd.col('time').dt.strftime('%B'),
            shallow=pd.col('depth') < 70,
            region=pd.col('place').str.split(', ').str[-1])
    .loc[pd.col('mag') >= 7]
    .loc[pd.col('shallow')]
    .sort_values('mag', ascending=False)
    [['month', 'region', 'mag', 'depth']]
    .head(5)
)

Which gives:

   month           region  mag  depth
 January Japan Earthquake  7.5 10.000
   April           Taiwan  7.4 40.000
December          Vanuatu  7.3 54.372
    June             Peru  7.2 24.000
  August Japan Earthquake  7.1 24.000

Four pd.col references, three of them creating columns that did not exist when the chain started — and sort_values taking a plain string, because it wants a name. That mixture is normal, and the rule above tells you which is which. (The scruffy Japan Earthquake region is the USGS data, not a bug: splitting a free-text place field is exactly the kind of real-world mess that toy datasets never teach you to handle.)

The lambda form it replaces

Before Pandas 3, the same deferral was expressed with a lambda, which .loc and .assign still accept:

df.loc[lambda df_: df_['mag'] > 7]

The df_ name was a convention meaning "the frame at this point in the chain". You will see this throughout the older Bamboo Weekly archive, and it continues to work — pd.col simply says the same thing with less ceremony.

The lambda still earns its place in one situation: when you need to operate on the frame as a whole, across every column, rather than on columns you can name. pd.col('a') + pd.col('b') is fine when there are two of them, but summing forty columns you would rather not enumerate still wants a lambda:

df.loc[lambda df_: df_.sum(axis=1) > 100]

For the lambda era in its own right, see Method chaining in Pandas: Cleaner queries with assign, loc, and lambda and Optimizing Pandas queries with method chaining.

Watch it instead

If you would rather see this in action: Cleaner Pandas Queries with pd.col — New in Pandas 3.

It is part of Getting ready for Pandas 3, a series of short videos on what breaks, what changes, and how to prepare for the upgrade.

Practice it

Work through a pd.col exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/pd-col/

Go deeper

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

Where you will use it

pd.col shows up wherever you filter or derive columns inside a chain. See it in context on Pandas loc and Pandas assign.

See it on real data

Below are the 34 Bamboo Weekly exercises that use pd.col on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.