Skip to content
Bamboo Weekly

pandas fillna

Fill in missing values — a constant, a forward/backward fill, or more.

Missing data is not a mistake in your pipeline; it is usually a fact about the
world. A sensor was offline. A country did not report that year. Nobody bought
anything that month. fillna is where you tell Pandas what that absence should
become — and the interesting part is that the right answer changes from column
to column.

Official documentation: DataFrame.fillna

What you can fill with

The argument can be a single value, a mapping of column names, or another
Series entirely:

df.fillna(0)                          # one value everywhere
df.fillna({'price': 0, 'note': ''})   # a different value per column
df.fillna(df.mean())                  # each column gets its own mean

For ordered data — time series, mostly — you usually want the neighboring
value rather than a constant:

df.ffill()          # carry the last known value forward
df.bfill()          # carry the next known value backward
df.ffill(limit=1)   # ...but only across a single gap

That limit is worth knowing about. Without it, one reading in January will
happily fill the rest of the year, and the resulting chart looks like data.

And when the values genuinely lie on a line between their neighbors:

df.interpolate()    # 1.0, NaN, NaN, 4.0  ->  1.0, 2.0, 3.0, 4.0

A worked example, on real data

The USGS publishes every earthquake it records, through a public API with no
key. Count the magnitude-5.5-and-above quakes by country and month for 2024:

import pandas as pd

url = ('https://earthquake.usgs.gov/fdsnws/event/1/query.csv'
       '?starttime=2024-01-01&endtime=2024-12-31&minmagnitude=5.5')

df = (pd.read_csv(url, usecols=['time', 'place', 'mag'], parse_dates=['time'])
        .assign(country=pd.col('place').str.split(', ').str[-1],
                month=pd.col('time').dt.month))

top = list(df['country'].value_counts().head(5).index)

grid = (df.loc[pd.col('country').isin(top)]
          .pivot_table(index='country', columns='month',
                       values='mag', aggfunc='count'))

month              1    2    3    4    5    6    7    8    9    10   11    12
country
Alaska            1.0  1.0  1.0  NaN  2.0  NaN  1.0  1.0  NaN  NaN  1.0  10.0
Indonesia         1.0  1.0  4.0  4.0  3.0  2.0  NaN  1.0  2.0  3.0  NaN   2.0
Japan             6.0  NaN  2.0  4.0  1.0  1.0  1.0  NaN  3.0  NaN  3.0   NaN
Papua New Guinea  NaN  1.0  3.0  1.0  5.0  1.0  1.0  NaN  4.0  1.0  2.0   NaN
Tonga             5.0  2.0  1.0  1.0  2.0  3.0  2.0  3.0  1.0  2.0  2.0   2.0

Thirteen missing cells — and here they really do mean zero. There was no
earthquake in Japan above 5.5 in February; the absence is the measurement.

grid.fillna(0).astype(int)

Two things to notice. The counts arrived as 1.0 and 10.0 rather than 1 and
10, because a column with a NaN in it cannot be an integer — NaN is a
float. Once the gaps are filled, astype(int) puts that right, and it is worth
doing: a table of counts with decimal points in it looks like a mistake.

The second thing is more uncomfortable, and it is the next section.

Filling before you aggregate usually does nothing

Look at the row totals before and after the fill:

grid.sum(axis=1)              # Alaska 18.0, Indonesia 23.0, Japan 21.0 ...
grid.fillna(0).sum(axis=1)    # Alaska 18,   Indonesia 23,   Japan 21   ...

Identical. sum, mean, count and friends skip missing values already, so
filling with zero first changed nothing except the dtype. A great deal of
fillna(0) in the wild is doing exactly this — costing a pass over the data to
produce the same answer.

Fill when the filled value is part of the output: a table someone will read, a
chart with no gaps in it, a file for a system that cannot represent NaN. Not
as a reflex before .sum().

When filling with zero is a lie

Change the grid from counting quakes to averaging their magnitude, and the same
fillna(0) becomes actively wrong:

                     1     2     3     4  ...
Alaska            5.73   NaN  5.66   NaN

Alaska's true average across the months it recorded anything is 5.775. Fill
the gaps with zero first, and it becomes 3.850 — a number that describes no
earthquake that has ever happened.

The test is whether zero is a plausible value for that column. For a count, it
is: no quakes is a real outcome. For a magnitude, an average, a temperature or a
price, zero is not a small value — it is a fictional one, and it will drag every
statistic you compute toward it.

Downcasting object dtype arrays on .fillna, .ffill, .bfill is deprecated

If you are on Pandas 2.x and filling an object column, you have met this:

FutureWarning: Downcasting object dtype arrays on .fillna, .ffill, .bfill is
deprecated and will change in a future version. Call result.infer_objects(copy=False)
instead. To opt-in to the future behavior, set
`pd.set_option('future.no_silent_downcasting', True)`

The old behavior was that Pandas would quietly narrow the dtype for you after
the fill:

s = pd.Series([True, False, np.nan], dtype=object)
s.fillna(False)
# Pandas 2.x -> dtype: bool     (with the warning above)
# Pandas 3.0 -> dtype: object   (no warning, no downcast)

The warning is gone in 3.0 because the behavior it warned about is gone. Your
column now stays object, and nothing tells you so.

That matters more than it sounds. Object columns are slow and large: summing
200,000 filled values took 0.097 seconds as object against effectively zero as
int64, and cost 7.2 MB of memory instead of 1.6 MB. Any code that checks
col.dtype == bool also quietly stops matching.

The fix is to say what you want:

s.fillna(False).astype(bool)      # you know the type
s.fillna(False).infer_objects()   # let Pandas work it out, explicitly

Both behave identically on 2.x and 3.x, which is the point — the version stops
mattering once you have said it out loud.

NDFrame.fillna() got an unexpected keyword argument 'method'

s.fillna(method='ffill')
# TypeError: NDFrame.fillna() got an unexpected keyword argument 'method'

method= was deprecated in Pandas 2.1 and removed in 3.0. It has its own
functions now:

s.ffill()      # was fillna(method='ffill')
s.bfill()      # was fillna(method='bfill')

This one catches people out because it is in a great deal of code written before
2023, and in most tutorials that have not been touched since. If you are working
through older material, this is the substitution to make as you go.

ChainedAssignmentError, and a fill that silently does nothing

df['a'].fillna(0, inplace=True)
# ChainedAssignmentError: A value is being set on a copy of a DataFrame or
# Series through chained assignment using an inplace method.

Worse than the error is what happens alongside it: the values are not
filled. df['a'] hands you a temporary object, inplace=True modifies that
temporary, and it is discarded. Under Copy-on-Write, which is the only mode in
Pandas 3, this can never work.

Either of these does work:

df.fillna({'a': 0}, inplace=True)   # inplace on the frame itself
df['a'] = df['a'].fillna(0)         # or assign the result back

The second is the one to reach for by default. inplace=True saves no memory
worth having and does not compose with method chaining, which is where the rest
of your Pandas code probably lives.

Practice it

Work through a fillna exercise, with instant feedback and no signup required:
practice.lernerpython.com/bamboo-weekly/fillna/

Go deeper

Bamboo Weekly is the practice. If you want the structured version — full courses
with downloadable Jupyter notebooks, plus live Pandas office hours when you get
stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 16 Bamboo Weekly exercises that use fillna on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.