tail in pandas — practice examples (Bamboo Weekly)

Look at the last few rows — and make sure they really are the rows you think they are.

What is at the bottom of your data frame? head is the method everyone reaches for the moment a file finishes loading, and tail is its mirror: the last five rows instead of the first five. That part is easy. The harder question is whether those last five rows are the ones you actually want.

Because tail counts positions, not dates and not values. It returns whatever sits at the bottom of the frame right now. If your data arrived oldest-first, those rows are the most recent records and everyone goes home happy. If it arrived newest-first — and plenty of government and scientific APIs hand you data exactly that way — then tail quietly returns the oldest rows in the file, with no error and output that looks entirely reasonable.

Official documentation: DataFrame.tail, Series.tail and GroupBy.tail

One argument, and a trick hiding inside it

df.tail()                        # the last 5 rows; n defaults to 5
df.tail(3)                       # the last 3 rows
df.tail(-3)                      # everything EXCEPT the first 3 rows
s.tail(2)                        # the last 2 values of a series
df.groupby('country').tail(1)    # the last row within each group

There is only one parameter, n, and it defaults to 5. Ask for more rows than the object contains and you get all of them rather than an error, which is what makes tail safe to drop into a chain you are still building. Ask for tail(0) and you get an empty frame with the columns intact.

The line worth memorizing is the third one. A negative n means "skip this many from the front," so df.tail(-3) is every row except the first three — identical to df.iloc[3:], and much easier to read inside a method chain. Almost nobody knows this argument exists, and it is exactly right for spreadsheets that open with two rows of notes before the real data begins. head runs the same trick the other way: df.head(-4) drops the last four rows, which is how you lose a "Source: …" footer without counting the good rows first.

A worked example, on real data

The USGS publishes every earthquake it records, and its CSV endpoint is an honest place to meet this method, because it returns events newest-first:

import pandas as pd

url = ('https://earthquake.usgs.gov/fdsnws/event/1/query.csv'
       '?starttime=2024-01-01&endtime=2024-12-31&minmagnitude=5.5')

quakes = (
    pd.read_csv(url, usecols=['time', 'place', 'mag'], parse_dates=['time'])
    # a place reads '63 km W of Coquimbo, Chile', so the country is last
    .assign(time=pd.col('time').dt.floor('s'),
            country=pd.col('place').str.split(', ').str[-1]
                                   .str.removesuffix(' Earthquake'))
    .drop(columns='place')
)

quakes.tail()

                         time  mag country
339 2024-01-01 09:03:48+00:00  5.5   Japan
340 2024-01-01 07:56:47+00:00  5.6   Japan
341 2024-01-01 07:18:41+00:00  6.2   Japan
342 2024-01-01 07:10:09+00:00  7.5   Japan
343 2024-01-01 07:06:05+00:00  5.8   Japan

That is 344 quakes of magnitude 5.5 or greater in 2024, and the tail is New Year's Day — the Noto Peninsula sequence, magnitude 7.5 included. Interesting rows, but not the recent ones. One line says why:

quakes['time'].is_monotonic_decreasing

True

Sort first, and the tail means what you assumed:

quakes.sort_values('time').tail()

                       time  mag                      country
4 2024-12-28 08:34:06+00:00  5.6  southern Mid-Atlantic Ridge
3 2024-12-28 19:57:19+00:00  5.5                        Tonga
2 2024-12-30 02:56:16+00:00  5.5                  Philippines
1 2024-12-30 05:40:49+00:00  5.5                        Chile
0 2024-12-30 05:41:06+00:00  5.5                        Chile

Notice the index: 4, 3, 2, 1, 0. tail never renumbers, which is how you can see that these rows came from the top of the file.

Now the idiom that makes tail more than a convenience. Call it on a groupby object and you get the last n rows of every group, stacked back into one frame. With n=1 and a sort in front of it, that is the latest record per group:

(
    quakes
    .sort_values('time')
    .groupby('country')
    .tail(1)
    .nlargest(8, 'time')
)

                       time  mag                      country
0 2024-12-30 05:41:06+00:00  5.5                        Chile
2 2024-12-30 02:56:16+00:00  5.5                  Philippines
3 2024-12-28 19:57:19+00:00  5.5                        Tonga
4 2024-12-28 08:34:06+00:00  5.6  southern Mid-Atlantic Ridge
5 2024-12-27 12:47:37+00:00  6.8                Kuril Islands
6 2024-12-27 05:30:07+00:00  5.9  northern Mid-Atlantic Ridge
7 2024-12-26 21:02:25+00:00  5.8                 Japan region
8 2024-12-25 06:34:48+00:00  5.6                  Fiji region

Eighty-two regions, one row each, every one of them that region's most recent large quake of the year. Nothing is aggregated and no columns are lost: groupby(...).tail(n) selects rows rather than summarizing them, so unlike .max() it leaves you the whole record, not just the field you sorted on.

All of which rests on that sort_values call. groupby preserves the order of the rows it is given, so skip the sort and you get each region's last row as stored:

quakes.groupby('country').tail(1).nlargest(8, 'time')

                        time   mag                     country
19 2024-12-12 01:51:35+00:00  5.50           Wallis and Futuna
21 2024-12-09 23:08:31+00:00  5.71                      Nevada
34 2024-12-08 10:25:00+00:00  6.00               Kuril Islands
37 2024-12-05 18:44:21+00:00  7.00                  California
45 2024-11-20 14:43:51+00:00  5.70               Greenland Sea
53 2024-11-10 15:50:03+00:00  5.90                        Cuba
79 2024-10-08 00:35:35+00:00  5.70  southeast of Easter Island
86 2024-09-26 19:19:28+00:00  6.30  Mauritius - Reunion region

Same method, same groups, a completely different and completely wrong answer. Because this file is newest-first, "the last row per country" now means the oldest quake in each region: the Kuril Islands row has moved from December 27 back to December 8.

Negative n deserves a demonstration of its own. Take the six most recent quakes:

recent = quakes.sort_values('time').tail(6).reset_index(drop=True)
recent.tail(-2)

                       time  mag      country
2 2024-12-28 19:57:19+00:00  5.5        Tonga
3 2024-12-30 02:56:16+00:00  5.5  Philippines
4 2024-12-30 05:40:49+00:00  5.5        Chile
5 2024-12-30 05:41:06+00:00  5.5        Chile

Four rows, not two. tail(-2) dropped the first two and kept the rest, however many that turned out to be — the point being that you rarely know the frame's length when you write the line. Overshoot with recent.tail(-10) and you get an empty frame rather than an exception. This is the form I should have used in Bamboo Weekly #27, where I dropped two summary rows from an Excel extract with .tail(6) — correct, but only because I had counted what was left.

Finally, tail behaves the same way on a series, returning the last n values instead of the last n rows. Series are what aggregations and value_counts produce, so this comes up constantly:

quakes.groupby(quakes['time'].dt.month)['mag'].max().tail(3)

time
10    6.6
11    6.8
12    7.3
Name: mag, dtype: float64

The strongest quake in each of the last three months of 2024. That one is safe because grouping by month number sorts the index for you — but that is groupby's doing, not tail's, and it is worth knowing which of the two you are trusting.

Three mistakes people make

Assuming that the tail is the most recent data. This is the one that costs real time, because nothing goes wrong on screen: the frame comes back, the numbers look plausible, and the chart you build on it is describing last January. Any file you did not sort yourself deserves a sort_values in front of the tail, or at least a look at is_monotonic_increasing on the date column. Sorting costs a few milliseconds; the alternative is a silent error.

Reaching for .tail(1) when you meant .iloc[-1]. They sound like the same request and return different types. df.tail(1) is a one-row data frame; df.iloc[-1] is a series representing that row, with the column names as its index. One level down, s.tail(1) is a one-element series while s.iloc[-1] is the number itself. Format that one-element series and Pandas stops you: f"{s.tail(1):.1f}" raises TypeError: unsupported format string passed to Series.__format__. At least that failure is loud. The quiet one is arithmetic: recent['mag'].tail(1) - recent['mag'].head(1) gives two NaN values rather than a change, because Pandas aligns the two series by index label and labels 0 and 5 have nothing to match. If you want a scalar, ask for one — .iloc[-1] or .item().

Expecting groupby(...).tail() to respect a sort you never performed. groupby keeps rows in the order it received them, so "the last row per group" means the last one in the frame's current arrangement. That is a feature — you decide what "last" means by choosing what to sort on — but the responsibility is yours. If you want the newest record per group, sort by the timestamp first, every time.

Where it shows up in Bamboo Weekly

Bamboo Weekly #14: JOLTS is the pattern at its cleanest. After filtering the Bureau of Labor Statistics extract down to the national quit rate, the answer is .sort_values(['year', 'period']).tail(1) — sort on year and then on period, and the final row is the most recent month. The sort comes first, and it is doing the real work.

Bamboo Weekly #2: Egg prices asks whether egg prices have come down lately, and answers with df.groupby([df['Date'].dt.year, df['Date'].dt.month])['Low Price'].mean().tail(10). I follow it with the same line plus .sort_index() before the tail, because the first version assumes an ordering rather than establishing one.

Bamboo Weekly #90: Voter participation is where groupby(...).tail(1) earns its keep: .sort_values('Year').groupby('Country').tail(1) gives the most recent presidential election for every country, turnout figure included, ready for nlargest.

Bamboo Weekly #18: World population uses tail six times in one issue, always as .sort_values(...).tail(10) — the ten most populous countries, the ten with the highest birth rate — and chains it straight into diff.

Practice it

Work through a .tail() exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/tail/

Go deeper

Start with the other half of the pair: head is the same method pointed at the top of the frame, negative n included. From there, sort_values and sort_index are what make a tail meaningful, iloc is the general positional selector that tail specializes, nlargest answers the "biggest few" question people often ask tail to handle, and groupby is the whole story behind the per-group version.

More Pandas videos on Python and Pandas with Reuven Lerner.

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

.head() — when the rows worth checking are at the start of the file
.sort_index() — when the last rows are only last because nobody put them in order

See it on real data

Below are the 16 Bamboo Weekly exercises that use tail on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.

pandas tail