Have you ever tried to turn a column of whole numbers into integers, and had Pandas refuse?
visitors = pd.Series([1250, None, 980], index=['Mon', 'Tue', 'Wed'], name='visitors')
visitors.astype(int)
IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer.Replace or remove non-finite values or cast to an integer typethat supports these values (e.g. 'Int64')
(Yes, "integer.Replace" and "typethat" are in the actual message. That's how Pandas 3.0 prints it.)
The short answer is that the column contains at least one NaN or infinity, and a standard NumPy integer has no way to store either. So you have two real choices: deal with the missing values, or leave the column as floats. More often than people expect, the second is the right one.
Why a column of whole numbers is float in the first place
Look at how visitors displayed before the conversion:
visitors
Mon 1250.0
Tue NaN
Wed 980.0
Name: visitors, dtype: float64
Nobody asked for 1250.0. A NumPy int64 is just 64 bits describing a whole number, and every one of those bit patterns is already a valid integer, so there is nothing left over to mean "missing." Floating-point numbers, by contrast, reserve special patterns for NaN and infinity. So when Pandas meets a missing value in otherwise-integer data, it promotes the whole column to float64. Without that missing value, the column would have been int64.
That's the real story behind the error. You're not converting floats to ints. You're asking Pandas to undo the promotion, while the missing value that caused it is still there.
Do you actually need integers?
Ask yourself why you want the conversion. Often the answer is that "1250.0" looks wrong. But the float column is doing everything you need:
visitors.mean()
np.float64(1115.0)
Aggregations skip NaN, just as you would want. And a float64 stores whole numbers exactly, all the way up to 2⁵³, which is about nine quadrillion. Unless you're counting something bigger than that, nothing is being rounded.
If the decimal point in the output is the problem, then fix the display, not the data. option_context changes how numbers print for just the code inside the with block:
with pd.option_context('display.float_format', '{:,.0f}'.format):
print(visitors)
Mon 1,250
Tue NaN
Wed 980
Name: visitors, dtype: float64
When you do want integers
If the column has to be integers, for example because another system insists on it, then decide what each missing value means, and act on that.
If it's missing because there was nothing to count, zero is the honest answer, and you can fill before converting:
visitors.fillna(0).astype(int)
Mon 1250
Tue 0
Wed 980
Name: visitors, dtype: int64
But be careful with this one. If Tuesday's number is missing because the counter was broken, filling with zero doesn't just fix a dtype. It invents a day with no visitors, and it drags down every average you calculate afterward.
If the rows are useless without a value, drop them instead: visitors.dropna().astype(int).
What about Int64?
The error message itself suggests 'Int64', with a capital I. That's one of Pandas' nullable extension types, and it will indeed accept the missing value. My advice is to stay away from it unless you have no other choice. Very few people use these types in practice, and PyArrow's nullable types (such as int64[pyarrow]) are still experimental.
There's also a bigger difference than the capital letter suggests. Both kinds of nullable type mark missing values with pd.NA, not NaN, and pd.NA behaves differently. Compare:
np.nan == np.nan
False
pd.NA == pd.NA
<NA>
A comparison with pd.NA returns pd.NA, and Python can't treat that as true or false:
i = visitors.astype('Int64')
if i['Tue'] > 1000:
print('busy day')
TypeError: boolean value of NA is ambiguous
So converting to Int64 doesn't just change how the numbers look. It changes how missing values behave everywhere downstream, and that's a lot to take on in exchange for removing a decimal point.
Where the infinity came from
The "or inf" part of the message usually means a division by zero happened somewhere upstream:
df = pd.DataFrame({'cases': [10, 0, 7], 'tests': [200, 0, 0]})
df['cases'] / df['tests']
0 0.05
1 NaN
2 inf
dtype: float64
Zero divided by zero gives NaN, and seven divided by zero gives inf. Decide what a rate with no tests means (usually "unknown"), and say so explicitly:
(df['cases'] / df['tests']).replace([np.inf, -np.inf], np.nan)
0 0.05
1 NaN
2 NaN
dtype: float64
A rate is a fraction anyway, so this is a column that should stay float.
Two surprises on the way
The first: when there are no missing values, astype(int) truncates without saying a word:
pd.Series([2.7, 1.2]).astype(int)
0 2
1 1
dtype: int64
That 2.7 became 2, not 3. If rounding is what you meant, say so with .round() first.
The second: the same underlying problem shows up with different wording depending on where you hit it. Asking read_csv for an integer column that has blanks gives you this:
import io
csv = io.StringIO('city,population\nOslo,709000\nLima,\n')
pd.read_csv(csv, dtype={'population': int})
ValueError: Integer column has NA values in column 1
Leave out the dtype, and read_csv loads the column as float64 with a NaN, which you can then fill or drop as above. Converting a column of numeric strings with a None in it produces yet another message, ValueError: cannot convert float NaN to integer. Different messages, one cause: somewhere in that column is a value that isn't a number.
Practice
See astype in Pandas for converting column types on real data, and to_numeric in Pandas for cleaning text columns before you convert them. Then try the to_numeric exercise.