Skip to content
Bamboo Weekly

pandas abs

Throw away the sign — on purpose, and in two situations where that is the whole answer.

How big was the move? Not up or down — just how big. Signed numbers answer that badly, because a rise of 2 and a fall of 2 average out to nothing, and "nothing" is a poor description of a wild week. .abs() replaces every value with its distance from zero, and that turns two awkward questions into one-liners.

It takes no arguments, and works on a Series, a whole data frame, timedeltas, and the nullable dtypes, where pd.NA stays pd.NA. Python's builtin abs(df) calls it too. A string column is the only thing that raises: with PyArrow installed the message is ArrowNotImplementedError: Function 'abs_checked' has no kernel matching input types (large_string).

Official documentation: Series.abs and DataFrame.abs.

Use one: how big, regardless of direction

Daily dollars per euro from FRED, 6,926 trading days since January 1999:

import pandas as pd

url = 'https://fred.stlouisfed.org/graph/fredgraph.csv?id=DEXUSEU'

euro = (
    pd.read_csv(url, parse_dates=['observation_date'], index_col='observation_date')
    ['DEXUSEU']
    .dropna()
)

euro.diff().mean()        # -3.3357400722021836e-06
euro.diff().abs().mean()  # 0.00499441155234657

Two numbers about the same 27 years. The first says the euro has not moved: every rise cancels a fall, and the mean daily change rounds to zero. The second says it moves half a cent every single day. Both are true, and only the second one describes what it is like to hold euros. Put .abs() before the aggregation and you get volatility; put it after, and you get the size of a net drift.

.abs() sits in that position after diff or pct_change whenever you want the size of a move rather than its direction — grouped by year, it is a volatility series. In front of nlargest it ranks by magnitude, so crashes and rallies compete on equal terms:

euro.diff().abs().nlargest(5)

observation_date
2009-03-19    0.0620
2008-12-17    0.0548
2008-12-19    0.0423
2008-10-29    0.0375
2009-01-05    0.0370
Name: DEXUSEU, dtype: float64

Every one of the five is the financial crisis. Two of them were the euro falling.

Use two: which row is nearest to a target

This is the idiom to memorize:

(series - target).abs().idxmin()

Subtract the target, drop the sign, ask for the label of the smallest remaining value. Distance from a number is a minimum problem, and .abs() is what makes it one. When was the euro closest to parity with the dollar?

(euro - 1).abs().nsmallest(5)

observation_date
2022-08-29    0.0000
2002-12-04    0.0001
2022-08-26    0.0002
2002-07-25    0.0003
2022-09-13    0.0003
Name: DEXUSEU, dtype: float64

August 29, 2022, dead on 1.0000. Drop the .nsmallest(5) for (euro - 1).abs().idxmin() and you get that Timestamp on its own. The same three steps find the nearest date or the observation closest to a threshold, and swapping .idxmin() for a comparison gives you everyone within a tolerance rather than only the winner.

Two mistakes people make

.mean().abs() when you meant .abs().mean(). The two numbers in the first code block above differ by a factor of 1,500. Absolute value does not commute with anything that averages, and the version that reads more naturally in English is usually the wrong one.

Dividing by .abs() to get a sign. s / s.abs() gives you +1 and -1, which is neat until a value is exactly zero and you get NaN rather than 0. That is the trick behind Bamboo Weekly #183: Hiring, which scores each month up or down and sums the result by year. np.sign handles the zero; the division does not.

Where it shows up in Bamboo Weekly

Only a handful of solutions use .abs() — but each one is a clean example of one of the two uses above.

Bamboo Weekly #175: Inflation runs .pct_change().abs().mean().nsmallest(10) across a table of national price indexes to find the countries whose inflation was steadiest, in either direction. Without the .abs(), a country that swung wildly and came back would win.

Bamboo Weekly #141: Argentina is the nearest-value idiom in tolerance form: .sub(1).abs() on the peso rate, then keep everything under 0.01, then .iloc[[0, -1]] for the first and last row where the peso sat within one percent of the dollar.

Bamboo Weekly #144: Museum heists calls .abs() on a .diff(axis='columns') between date columns, because a gap between two dates should not come out negative just because the columns were in the other order. Bamboo Weekly #163: Daylight saving time does the same for the length of each country's DST period.

Practice it

Work through an abs exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/abs/

Go deeper

idxmin is the other half of the nearest-value idiom, and idxmax is its mirror. .abs() almost always follows diff, pct_change or a plain subtraction, and leads into nlargest or nsmallest. To floor a column at zero rather than fold it, you want clip(lower=0) instead.

More Pandas videos on Python and Pandas with Reuven Lerner.

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 5 Bamboo Weekly exercises that use abs on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.