Skip to content
Bamboo Weekly

pandas str.strip

The six small cleanup methods you reach for before any of the big ones.

What they do

What is the first thing you do with a text column you did not create? Not a regular expression. First you find out what is actually in the column — how long the values are, what character they start with, whether somebody left a space on the end — and then you knock the obvious junk off, so that the rest of your chain has something clean to work with.

That is what these six methods are for, and it is why I am covering them together. Each does one small thing to every element of a series and hands back a new series. None takes a regular expression; none can be talked into doing anything clever. They are the first pass:

The heavy text methods — str.replace, str.split, str.contains and str.extract — come after. Reach for those first and you spend your afternoon debugging a pattern that was never going to match, because the value you were matching against had a trailing space.

Official documentation: Series.str.strip, Series.str.lower, Series.str.upper, Series.str.len, Series.str.get and Series.str.slice

The one argument that will bite you

Between them these six take four arguments, and three of the four behave exactly as you would guess. The fourth is .str.strip()'s, and it does not mean what almost everyone assumes it means.

Called with nothing, .str.strip() removes whitespace from both ends, and it means every kind of whitespace — spaces, tabs, newlines, and the non-breaking spaces that arrive whenever data has passed through a web page:

pd.Series(['\xa0 Equipment \n']).str.strip()

0    Equipment
dtype: str

Called with an argument, it removes any character in that argument, from either end, repeatedly, until it hits a character that is not in the set. The argument is a set of characters, not a substring. I will show you what that costs in a moment, but here is where the set behavior is exactly what you want:

pd.Series(['..Boston, MA Metro Division']).str.strip('.')

0    Boston, MA Metro Division
dtype: str

Two leading periods, both gone, from one call. Compare .str.removeprefix('.'), which removes one occurrence of a literal string and stops:

pd.Series(['..Boston, MA Metro Division']).str.removeprefix('.')

0    .Boston, MA Metro Division
dtype: str

.str.lstrip() and .str.rstrip() are the one-sided versions, with the same set semantics. Bamboo Weekly #159: State of the Union puts the set to good use: presidents who served twice arrive as Donald J. Trump - I and Donald J. Trump - II, and .str.strip(' -I') takes spaces, hyphens and capital I's off both ends in one pass, merging the two terms back into one name. That is a genuine set of three unrelated characters, and no suffix method could do it. Of the 55 str.strip calls in the Bamboo Weekly archive, 34 pass no argument at all; when I do pass one it is most often string.punctuation, which is a set of 32 characters and exactly the case .str.strip() was designed for.

That is the trick worth taking away from this page. Everything the character-set behavior does wrong when you hand it a suffix, it does right when you hand it a genuine set — and Python ships one:

import string

string.punctuation

'!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~'

Thirty-two characters, in one name. Pass that to .str.strip() and every combination of quotes, brackets, commas and stray periods comes off both ends of every value, without your having to anticipate which ones a particular file uses:

places = pd.Series(['"Chicago",', '(Houston)', '...Phoenix!', '#Seattle',
                    'St. Louis', '¿Bogotá?'])

places.str.strip(string.punctuation)

0      Chicago
1      Houston
2      Phoenix
3      Seattle
4    St. Louis
5      ¿Bogotá
dtype: str

Two things in that output are worth a second look. St. Louis keeps its period, because strip only ever works on the ends — an interior character is safe no matter what you pass. And ¿Bogotá? lost its question mark but kept the inverted one, because string.punctuation is ASCII only. If your data is not, you want str.strip('¿¡' + string.punctuation), or a regular expression.

.str.get(n) takes one index. .str.slice(start, stop, step) takes up to three, and means exactly what [start:stop:step] means in Python, negative indexes included. .str.len(), .str.lower() and .str.upper() take nothing at all.

A worked example, on real data

The US Census Bureau publishes population estimates for every metropolitan statistical area as a spreadsheet, and it is a fine specimen of a table built for human eyes rather than for Pandas:

import pandas as pd

url = ('https://www2.census.gov/programs-surveys/popest/tables/2020-2024/'
       'metro/totals/cbsa-met-est2024-pop.xlsx')

df = pd.read_excel(url, storage_options={'User-Agent': 'Mozilla/5.0'},
                   skiprows=4, header=None,
                   names=['area', 'base', '2020', '2021',
                          '2022', '2023', '2024'])

df['area'].head(6)

0                        United States
1    .In Metropolitan Statistical Area
2              .Abilene, TX Metro Area
3                .Akron, OH Metro Area
4               .Albany, GA Metro Area
5               .Albany, OR Metro Area
Name: area, dtype: str

441 rows, and already three problems. Names carry a leading period, which is how the Census indicates a sub-part of the row above it. Every metro area name ends in the same eleven characters, Metro Area, which tell me nothing. And row 1 is not a place at all — it is a subtotal.

Before touching any of that, I want to know what else is hiding in the column. .str.len() is how I ask:

df['area'].str.len().describe()

count    441.000000
mean      33.800454
std       32.979761
min        1.000000
25%       24.000000
50%       28.000000
75%       38.000000
max      618.000000
Name: area, dtype: float64

A maximum of 618 characters and a minimum of 1, in a column of place names. This is why .str.len() earns its place in the first pass — not because the lengths themselves are interesting, but because a length distribution finds malformed rows faster than reading the file does. Let me look at both ends:

df.loc[df['area'].str.len() > 60, 'area']

295    ..Montgomery County-Bucks County-Chester Count...
435    The Census Bureau has reviewed this data produ...
436    Note: The estimates are developed from a base ...
438    Annual Estimates of the Resident Population fo...
Name: area, dtype: str

df.loc[df['area'].str.len() == 1, 'area']

426    .
Name: area, dtype: str

Three of the long rows are the Census Bureau's footnotes, sitting in the same column as the place names because that is how a spreadsheet works. The one-character row is a spacer. Row 295 is genuine — a metro division with three county names in it — which is exactly the sort of thing you want to see before you write a rule that would have thrown it away.

.str.get(0) gives me the first character of every value, and counting those is the fastest structural summary of a column I know:

df['area'].str.get(0).value_counts()

area
.    433
S      2
U      1
P      1
T      1
N      1
A      1
R      1
Name: count, dtype: int64

433 rows start with a period and eight do not. The eight are the national total, the Puerto Rico total, and the six footnote lines. The leading period is the Census Bureau telling me which rows are data, and one call to .str.get(0) read that structure out of the file.

Now, the periods themselves. .str.strip('.') removes them from both ends, however many there are:

df['area'].str.strip('.').head(4)

0                       United States
1    In Metropolitan Statistical Area
2              Abilene, TX Metro Area
3                Akron, OH Metro Area
Name: area, dtype: str

And the Metro Area suffix? Here is the trap, on real values, and I want you to look closely at row 4:

df['area'].str.strip(' Metro Area').head(6)

0                   United States
1    .In Metropolitan Statistical
2                    .Abilene, TX
3                      .Akron, OH
4                      .Albany, G
5                     .Albany, OR
Name: area, dtype: str

Georgia is now G. .str.strip(' Metro Area') did not remove a suffix; it removed characters belonging to the set {' ', 'A', 'M', 'a', 'e', 'o', 'r', 't'} from both ends until it ran out. On .Albany, GA Metro Area it chewed backward through Area, then the space, then Metro, then the space, and then kept going into the A of GA — because A is in the set. Ohio and Oregon survived because H and R are not.

Nothing raised. Nothing warned. And this is not a one-row curiosity: 100 of the 393 metro areas in this file came out wrong, across nine states — CA, GA, IA, LA, MA, NM, PA, VA and WA. .Amherst Town-Northampton, MA Metro Area lost both letters of its state and the comma behind them, ending up as .Amherst Town-Northampton,. Had I joined on this column afterward, I would have been hunting the bug for an hour.

The fix is .str.removesuffix(), which removes one occurrence of a literal string from the end and leaves everything else alone:

df['area'].str.removesuffix(' Metro Area').head(6)

0                        United States
1    .In Metropolitan Statistical Area
2                         .Abilene, TX
3                           .Akron, OH
4                          .Albany, GA
5                          .Albany, OR
Name: area, dtype: str

Georgia is Georgia again. Notice too that row 1 was left untouched — the subtotal row ends in Area but not in Metro Area, and removesuffix is precise enough to know the difference. .str.removeprefix() is its mirror image at the front.

With the mess understood, the whole cleanup is one chain. Keep the metro-area rows, take the period off the front, take the suffix off the back, and pull the state code out of what is left with .str.slice(-2):

metros = (
    df
    .loc[lambda df_: df_['area'].str.endswith(' Metro Area')]
    .assign(area=lambda df_: (df_['area']
                              .str.strip('.')
                              .str.removesuffix(' Metro Area')),
            state=lambda df_: df_['area'].str.slice(-2))
)

metros[['area', 'state', '2024']].head()

                          area state      2024
2                  Abilene, TX    TX  184278.0
3                    Akron, OH    OH  702209.0
4                   Albany, GA    GA  145451.0
5                   Albany, OR    OR  132474.0
6  Albany-Schenectady-Troy, NY    NY  913485.0

393 rows of clean place names and a state column that did not exist a moment ago. The .str.slice(-2) is worth dwelling on: a negative start counts from the right, so it means "the last two characters" without my knowing or caring how long the name is. Albany-Schenectady-Troy, NY and Akron, OH are wildly different lengths and both give up their state code to the same call.

From here the analysis is ordinary Pandas. Which metro areas grew fastest since 2020?

(
    metros
    .assign(growth=lambda df_: df_['2024'] / df_['2020'] - 1)
    .nlargest(6, 'growth')
    .set_index('area')
    [['state', 'growth']]
    .round(3)
)

                                           state  growth
area
Wildwood-The Villages, FL                     FL   0.187
Myrtle Beach-Conway-North Myrtle Beach, SC    SC   0.169
Lakeland-Winter Haven, FL                     FL   0.168
St. George, UT                                UT   0.142
Ocala, FL                                     FL   0.136
Port St. Lucie, FL                            FL   0.135

Four of the six fastest-growing metro areas in the country are in Florida, and retirement destinations dominate the list.

That leaves .str.lower(). The Census is disciplined about capitalization, so nothing above needed it — but the moment I compare against something I typed myself, it does:

(metros['state'] == 'tx').sum()

0

(metros['state'].str.lower() == 'tx').sum()

25

Zero Texas metro areas, when there are 25. That is the whole case for .str.lower(): it is not about tidiness, it is about making two values comparable before you compare them. Lower both sides, or upper both sides, and the question you asked is the question that gets answered.

Six mistakes people make

Treating .str.strip()'s argument as a suffix. This is the big one, and the .Albany, G above is what it costs. The version that catches everybody is a column of domain names:

pd.Series(['microsoft.com', 'example.com', 'apple.com']).str.strip('.com')

0    icrosoft
1     example
2       apple
dtype: str

Two of the three are right, which is what makes this so dangerous. .str.strip took ., c, o and m off both ends, and microsoft happens to begin with an m. Test on the wrong three rows and you will ship it. If you mean a suffix, say .str.removesuffix(); if you mean a prefix, say .str.removeprefix(); use .str.strip() only when you genuinely mean a set of characters.

Calling .str twice where once would do. Every .str call is a separate pass over the column. Two slices are exactly equivalent to one:

(df['area'].str.slice(1).str.slice(0, -11) ==
 df['area'].str.slice(1, -11)).all()

True

On this column repeated to 882,000 rows, .str.slice(1).str.slice(0, -11) takes 41 ms and .str.slice(1, -11) takes 16 ms. I am not precious about this — .str.strip('.').str.removesuffix(' Metro Area') in the chain above is two passes and I left it that way, because the two steps mean different things and splitting them made the code readable. But when two consecutive calls are the same method, collapse them.

Calling .str on a column that is not text. The accessor checks the dtype and refuses:

df['2024'].str.len()

AttributeError: Can only use .str accessor with string values, not floating

That error is a gift. The dangerous case is an object column holding a mixture, because there the accessor works and silently discards everything that is not a string:

mixed = pd.Series(['.Akron, OH Metro Area', 41, None,
                   '.Albany, GA Metro Area'])

mixed.str.len()

0    21.0
1     NaN
2     NaN
3    22.0
dtype: float64

The integer 41 became NaN, exactly as the missing value did, and the two are now indistinguishable. If a column arrives as object and you expect text, call astype first and find out.

Expecting .str.len() to return integers. One missing value turns the whole column float64, which then breaks a comparison or a groupby. That, and using length as a validation tool rather than a measurement, are on the .str.len() page.

Forgetting that these mean something different after a split. On a column of strings .str.get(-1) is the last character and .str.len() is a count of characters; on a column of lists they are the last element and the number of elements. Same call, different question. Both cases are worked through on the .str.get() and .str.len() pages.

Splitting on the wrong delimiter and then not stripping. Split on ',' rather than ', ' and every piece after the first keeps a leading space, which is invisible in the printed output and fatal to any comparison:

df['area'].str.split(',').str.get(-1).head(4).tolist()

['United States', '.In Metropolitan Statistical Area', ' TX Metro Area', ' OH Metro Area']

df['area'].str.split(',').str.get(-1).str.strip().head(4).tolist()

['United States', '.In Metropolitan Statistical Area', 'TX Metro Area', 'OH Metro Area']

I used .tolist() deliberately: print those two as series and they look identical, because Pandas right-aligns the column and the leading space disappears into it. A bare .str.strip() after a split is cheap insurance, and bare is how str.strip is called in 34 of its 55 appearances in the archive.

One more thing worth knowing rather than a mistake: .str.title() looks like a polite way to standardize capitalization, and it is not safe on data holding abbreviations. metros['area'].str.title() turns Abilene, TX into Abilene, Tx. Use it where the values are English words, and .str.lower() where they are codes.

Does Pandas 3 change any of this?

In Pandas 3 a text column has the new str dtype, and when PyArrow is installed it is backed by PyArrow rather than by Python objects. You can check which you have with df['area'].array.dtype.storage, which reports pyarrow or python.

I ran every example on this page both ways. The results are identical, byte for byte: the same set semantics on .str.strip(), the same NaN from an out-of-range .str.get(), the same float64 from .str.len() when a missing value is present, the same AttributeError on a numeric column. For these six methods, PyArrow backing is an implementation detail and nothing more.

What did change in Pandas 3 is the dtype your text arrives as. A column read from a CSV or a spreadsheet is now str rather than object, which is why the AttributeError above is reliable and why the mixed-type trap has become something you have to construct on purpose rather than something you stumble into.

Where it shows up in Bamboo Weekly

44 Bamboo Weekly solutions use at least one of these six methods. str.strip leads at 19 solutions, then str.get at 14, str.len at 12, str.lower and str.slice at 10 apiece, and str.upper at exactly one. Four worth studying, all free to read:

Bamboo Weekly #48: Aviation accidents is the best demonstration on the site of .str.len() and .str.get() doing their non-string job. The cm_vehicles column holds a list of aircraft per accident, and df['cm_vehicles'].str.len().value_counts(normalize=True) shows that 98.5% of accidents involved exactly one airplane. I called that "a bit sneaky" at the time, and timed it: str.len took about 9 ms against 6.9 ms for apply(len), so the trick is elegant rather than fast. The same solution then uses .str.get(0) to take the first aircraft out of each list and .str.get('make') to pull a key out of each resulting dictionary — one method, three different meanings, in one chain. It is also the clearest .str.lower() case in the archive: value_counts across make and model reported CESSNA 172 with 547 accidents and Cessna 172 with 429 as two unrelated rows, and lowering the column merged them.

Bamboo Weekly #44: Global economics is the .str.slice page. OECD TIME values arrive as 2013 or 2013-Q2, and the solution uses .str.slice(0, 4).astype(int) >= 2010 to filter by year and .str.slice(2) to shorten the axis labels to two digits. It also contains the warning that goes with that second call: sort by a two-digit year across a century boundary and 1995 sorts after 2023 — "your own little Y2K problem on your desktop."

Bamboo Weekly #63: Ukraine aid is bare .str.strip() earning its keep. A column describing kinds of aid held 34 distinct values; .str.title() standardized the capitalization and brought that to 28, but two rows still read Equipment and two still read Weapons, identical to the naked eye. Adding .str.strip() to the chain brought the count down to 26. That is the whole method in one paragraph: invisible whitespace, visibly wrong counts.

Bamboo Weekly #74: UK elections uses .str.len() for the thing it is nominally for, on a question you would not otherwise be able to ask: do winning candidates have shorter names than the people they beat? The solution adds .str.len() across the first, middle and last name columns for both the winner and the defeated runner-up, compares the two totals, and runs value_counts on the result.

Practice it

Work through a .str.strip() exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/str-strip/

Go deeper

These six are the setup, not the analysis. Once the column is clean, the methods that do real work on text are str.contains for asking a yes-or-no question of every value, str.split for taking each string apart on a delimiter, and str.replace for rewriting part of every value. If your cleanup chain is heading toward a number, it almost certainly ends at astype; if it is heading toward an answer, it usually ends at value_counts.

Two neighbors are worth knowing as well. explode turns the lists that .str.get() indexes into one row per element, which is usually what you wanted. And read_excel is where columns like the one above come from — spreadsheets built for human eyes, with the footnotes sitting in column A.

More Pandas videos on Python and Pandas with Reuven Lerner.

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 19 Bamboo Weekly exercises that use str.strip on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.