to_numeric in pandas — practice examples (Bamboo Weekly)

Convert text to numbers, and find out how much of the text was never a number.

How many values in that column are not actually numbers? .astype(int) will not tell you. It meets the first one, raises, and stops — so you fix that value, run it again, and meet the second.

pd.to_numeric is a function rather than a method: hand it a Series, get a numeric Series back. Its reason to exist is one keyword argument, errors='coerce', which turns every unconvertible value into NaN instead of raising. Follow it with .isna().sum() and you have a count of the damage before you decide what to do about it.

Official documentation: pandas.to_numeric

The arguments that earn their keep

pd.to_numeric(series,
              errors='coerce',                 # unconvertible values become NaN
              downcast='integer',              # use the smallest dtype that fits
              dtype_backend='numpy_nullable')  # Int64 with <NA>, not float64 with NaN

Three arguments, and you will use the first one every time.

A worked example, on real data

NOAA's IBTrACS archive holds every North Atlantic tropical storm track since 1851, and it is the data set behind Bamboo Weekly #142. It has a quirk that ruins a naive load: line one is the column headers, and line two is the units.

import pandas as pd

url = ('https://www.ncei.noaa.gov/data/international-best-track-archive-'
       'for-climate-stewardship-ibtracs/v04r01/access/csv/ibtracs.NA.list.v04r01.csv')

cols = ['SID', 'SEASON', 'NAME', 'WMO_WIND']
df = pd.read_csv(url, usecols=cols)[cols]

df.head(3)

             SID SEASON     NAME WMO_WIND
0                  Year               kts
1  1851175N26270   1851  UNNAMED
2  1851175N26270   1851  UNNAMED

One row of text at the top, and every numeric column in the file is now a column of strings. Ask for integers and you learn exactly one thing:

df['WMO_WIND'].astype(int)

ValueError: invalid literal for int() with base 10: 'kts'

There is a bad value, and it says kts. Is it the only one? Are there ten? Seventy thousand? astype does not know and cannot say. Coerce instead:

wind = pd.to_numeric(df['WMO_WIND'], errors='coerce')

wind.dtype, wind.isna().sum(), len(wind)

(dtype('float64'), np.int64(71453), 127140)

Seventy-one thousand of the 127,140 values are not numbers. That is not one typo in a units row — that is most of the column. Now ask what they actually are:

df.loc[wind.isna(), 'WMO_WIND'].value_counts()

WMO_WIND
       71452
kts        1
Name: count, dtype: int64

One units cell, and 71,452 blanks. The column is not dirty; it is empty. Wind speed was simply not recorded for most of the nineteenth and twentieth centuries, so any average I compute covers fewer than half the rows. astype would have shown me the units cell and hidden all of that behind it.

With the count in hand, the analysis is honest:

(
    df
    .assign(wind=pd.col('WMO_WIND').pipe(pd.to_numeric, errors='coerce'),
            season=pd.col('SEASON').pipe(pd.to_numeric, errors='coerce'))
    .dropna(subset=['wind'])
    .groupby('season')['wind'].max()
    .tail(5)
)

season
2021.0    135.0
2022.0    140.0
2023.0    145.0
2024.0    155.0
2025.0    165.0
Name: wind, dtype: float64

errors='ignore' is gone

If you find errors='ignore' in an old notebook or an old answer online, it no longer works. It was removed in Pandas 3, and the replacement is a raise, not a warning:

pd.to_numeric(pd.Series(['10', 'kts']), errors='ignore')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    pd.to_numeric(pd.Series(['10', 'kts']), errors='ignore')
  File ".../pandas/core/tools/numeric.py", line 183, in to_numeric
    raise ValueError("invalid error value specified")
ValueError: invalid error value specified

That is a good change. errors='ignore' handed back the original text column when anything failed, so a pipeline could run to the end and produce a report built on strings. There are two settings now: 'raise' and 'coerce'. This applies to pd.to_numeric and pd.to_datetime, and not to astype, whose own errors= argument is alive and well.

Four mistakes people make

Reaching for astype on a column you have not inspected. .astype(float) is right when the text is clean and wrong the moment it is not. If the column came from a web page, a spreadsheet, or an API, coerce first and count.

Expecting integers back. NaN is a float, so a coerced integer column arrives as float64, and wind speeds print as 135.0. downcast='integer' will not save you either — it silently does nothing while a NaN is present. Pass dtype_backend='numpy_nullable' for an Int64 column that holds <NA> instead.

Passing a pd.col() expression directly. pd.to_numeric(pd.col('x'), errors='coerce') raises TypeError: boolean value of an expression is ambiguous, because the function inspects its argument before Pandas can resolve it. Use pd.col('x').pipe(pd.to_numeric, errors='coerce'), as above, or a lambda df_: inside assign.

Coercing and then not looking. errors='coerce' converts a data problem into missing data, which is only an improvement if you check how much you created. .isna().sum() costs one line and is the entire point of using this function.

Where it shows up in Bamboo Weekly

Two solutions call pd.to_numeric, which is fewer than the method deserves, but both are instructive.

Bamboo Weekly #173: IPOs is this page in miniature. FinnHub returns numberOfShares and totalSharesValue as text, and the solution converts both with lambda df_: pd.to_numeric(df_["numberOfShares"], errors="coerce"). I asked in the write-up why that rather than astype(float), and answered it: so that a bad value gives us NaN instead of killing the request.

Bamboo Weekly #154: University rankings uses the other argument, calling pd.to_numeric(..., downcast='float') across every float64 column to shrink the frame — the case where downcast= works, because the values are already numeric.

Bamboo Weekly #142: Hurricanes is the near miss, and the data above. That units row, I wrote at the time, "prevented me from being able to run astype on a number of columns." I solved it by reading the file twice, with skiprows=2 and names= — a fine solution, and one that errors='coerce' would have made unnecessary.

Practice it

Work through a to_numeric() exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/to-numeric/

Go deeper

astype is the page to read beside this one: it converts a whole frame at once, and it is the right tool the moment your data is clean. to_datetime is the same idea for dates, down to the same errors='coerce', and read_csv has the thousands= and na_values= arguments that stop some of this text from arriving as text at all.

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

.astype() — when you are confident every value converts and want it to raise if not
.isna() — which is how you count what errors='coerce' just turned into NaN

See it on real data

Below are the 2 Bamboo Weekly exercises that use to_numeric on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.

pandas to_numeric