Skip to content
Bamboo Weekly

pandas str.len

Not how long your values are. Which of them are the wrong length.

What it does

How do you find the broken rows in a column you did not create? You cannot read ten thousand values, and you cannot write a rule against a problem you have not seen yet. But most broken values announce themselves the same way: they are the wrong size. The postal code truncated to four digits, the ID that lost its leading zeros on the way through Excel, the short-answer field somebody pasted three paragraphs into.

.str.len() returns a new series holding the size of every element, and on its own that number is rarely interesting — I have never needed to know that NVIDIA CORP is eleven characters long. What I need is the row where the number is impossible. This is not a measurement tool, it is a validation tool, and the idiom is .str.len().value_counts().

Official documentation: Series.str.len

It takes no arguments

There is nothing to configure. The only thing worth knowing is what it counts, and that depends on what the column holds:

Same call, four meanings. The last one is the one that will cost you an afternoon, and I will come back to it.

A worked example, on real data

The SEC publishes the mapping from every registered company to its ticker symbol and exchange. It asks that you identify yourself in the User-Agent header, and returns 403 if you do not:

import pandas as pd

url = 'https://www.sec.gov/files/company_tickers_exchange.json'
ua = {'User-Agent': 'Your Name your@email.com'}

raw = pd.read_json(url, storage_options=ua, typ='series')
df = pd.DataFrame(raw['data'], columns=raw['fields'])

df.head()

       cik            name ticker exchange
0  1045810     NVIDIA CORP   NVDA   Nasdaq
1   320193      Apple Inc.   AAPL   Nasdaq
2  1652044   Alphabet Inc.  GOOGL   Nasdaq
3   789019  MICROSOFT CORP   MSFT   Nasdaq
4  1018724  AMAZON COM INC   AMZN   Nasdaq

10,403 rows the day I ran it; the SEC refreshes the file nightly, so your counts will be close rather than identical. Before I join this against anything, I want to know whether the ticker column is what I think it is:

df['ticker'].str.len().value_counts().sort_index()

ticker
1      21
2     247
3    2014
4    4989
5    2654
6     299
7     179
Name: count, dtype: int64

US ticker symbols run one to five characters. There are 478 rows here that are longer than that, and every one of them is a finding. Pull them out with .loc:

df.loc[pd.col('ticker').str.len() > 5, ['name', 'ticker']].head(5)

                                             name   ticker
6748                          CONSUMERS ENERGY CO   CMS-PB
6962             Brookfield Oaktree Holdings, LLC   OAK-PA
7032                   Seritage Growth Properties   SRG-PA
7040  InPoint Commercial Real Estate Income, Inc.   ICR-PA
7350      Kensington Capital Acquisition Corp. VI  KCAC-UN

Not common stock. Those are preferred series and unit lines, with the class appended after a hyphen. Joined against a price feed keyed on plain symbols, 478 rows would have failed to match in silence — and one line found them without my knowing in advance that share classes existed.

The check also tells me what rule to write next. Length was a screen, not the answer: BRK-B is five characters and looks exactly like GOOGL. Counting pieces rather than characters catches all of them:

df['ticker'].str.split('-').str.len().value_counts()

ticker
1    9861
2     542
Name: count, dtype: int64

542, not 478. Same method, one .str.split() earlier in the chain, and now it is counting elements in a list instead of characters in a string.

Now the identifier. A CIK is the SEC's company number, and EDGAR's file paths want it zero-padded to ten digits:

df['cik'].astype(str).str.len().value_counts().sort_index()

cik
4      58
5     509
6    1453
7    8383
Name: count, dtype: int64

Four lengths in a fixed-width identifier column. That is the shape of the leading-zero bug: the field was numeric somewhere upstream, the zeros went away, and nothing complained. A well-formed identifier column shows exactly one length, which is what makes this such a fast test — you are looking for a one-row answer, and anything else is the finding.

(
    df
    .assign(cik=pd.col('cik').astype(str).str.zfill(10))
    .head(3)
)

          cik           name ticker exchange
0  0001045810    NVIDIA CORP   NVDA   Nasdaq
1  0000320193     Apple Inc.   AAPL   Nasdaq
2  0001652044  Alphabet Inc.  GOOGL   Nasdaq

One length, 10,403 times. Now it joins.

Four mistakes people make

Letting one missing value turn the whole column into floats. exchange has 198 gaps, and .str.len() propagates each one rather than calling it zero:

df['exchange'].str.len().value_counts(dropna=False)

exchange
6.0    4365
4.0    3336
3.0    2504
NaN     198
Name: count, dtype: int64

Lengths of 6.0 and 4.0, because NaN is a float and one of them drags the whole series to float64. That is the right answer — an unknown string has an unknown length, not a length of zero — but it breaks the two things you were about to do next. astype(int) raises IntCastingNaNError, and a groupby on the lengths drops those rows without a word:

df.groupby(df['exchange'].str.len()).size().sum()

10205

198 companies short of 10,403, and no warning anywhere. Reach for .astype('Int64') — the nullable integer type, capital I — when you want whole numbers and want the gaps to survive.

Counting characters when you meant elements, or the reverse. A column of lists prints almost exactly like a column of strings, so the only sign you are on the wrong one is that the number is implausible:

df['name'].str.len().head(3)

0    11
1    10
2    13
Name: name, dtype: int64

df['name'].str.split().str.len().head(3)

0    2
1    2
2    2
Name: name, dtype: int64

Both are correct answers to different questions. Check the dtype of the series you are measuring: str means characters, object almost always means lists. On a series of dicts it counts keys — pd.read_json(..., typ='series') on the SEC's other ticker file gives 3 on every row, for cik_str, ticker and title, and str.get is what reaches into it by key.

Validating unstripped text. A trailing space is a character, and .str.len() counts it:

pd.Series(['NVDA ', 'AAPL', ' MSFT']).str.len()

0    5
1    4
2    5
dtype: int64

Two of those three now look like five-character tickers, so whatever rule you write next fires on the wrong rows in both directions. str.strip before you measure — and if the two counts differ, you have found a second problem.

Measuring characters when the limit is in bytes. Database columns, fixed-width files and most legacy systems budget bytes, and outside ASCII the two numbers part company:

pd.Series(['Nestlé S.A.', 'Ørsted A/S']).str.len()

0    11
1    10
dtype: int64

pd.Series(['Nestlé S.A.', 'Ørsted A/S']).str.encode('utf-8').str.len()

0    12
1    11
dtype: int64

Eleven characters, twelve bytes. If the receiving field is VARCHAR(11), your length check passed and your load will still fail.

Is it still slower than apply(len)?

In Bamboo Weekly #48 I called .str.len() on a column of lists "a bit sneaky," timed it, and admitted it was elegant rather than fast: about 9 ms against 6.9 ms for .apply(len). I re-ran that on Pandas 3.0.5 with PyArrow-backed strings, using the name column above repeated to 208,060 rows:

names = pd.concat([df['name']] * 20, ignore_index=True)

%timeit names.str.len()      # 2.2 ms
%timeit names.apply(len)     # 32.5 ms

A complete reversal — fifteen times faster, not slower. PyArrow computes the lengths in one pass over the string buffer and never builds a Python object. On lists nothing changed: 25.4 ms against 24.9 ms, because a list is still a Python object either way. The old warning is right for the case where I made it, and now wrong for text.

Where it shows up in Bamboo Weekly

12 Bamboo Weekly solutions use .str.len(), across 44 separate calls, and most of those are the .str.split().str.len() word count rather than a character count.

Bamboo Weekly #15: Eurovision is the simplest: df['lyrics'].str.split().str.len() gives the number of words in each song, plotted against the song's finishing place. The regression line comes out flat — wordier songs do not do better.

Bamboo Weekly #79: Cyber attacks is the filtering idiom you will reuse most: .loc[lambda s_: s_.str.len() >= 5] inside a split-explode-count chain, throwing away the short words before value_counts. That pattern appears in half the solutions on this list.

Bamboo Weekly #6: End of the humanities? is where I introduce the .str accessor with .str.len() as the example, on a column of degree names carrying footnote markers — a length anomaly of exactly the kind this page is about.

Practice it

Work through a str.len exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/str-len/

Go deeper

.str.len() travels with the rest of the small string methods — str.strip, str.lower, str.get and str.slice — which together make the first pass over a text column. str.split turns a character count into an element count, and explode is usually what follows it. Once the lengths have named the wrong rows, loc gets at them and value_counts counts what is left. And a length problem is often a type problem underneath, which is astype.

More Pandas videos on Python and Pandas with Reuven Lerner.

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 12 Bamboo Weekly exercises that use str.len on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.