Skip to content
Bamboo Weekly

pandas read_csv

Load a CSV file — from your disk or straight from a URL — into a data frame.

read_csv is the front door to nearly every Pandas project, and it is far more capable than most people use it for. Getting the arguments right at load time saves you from cleaning up afterwards: the wrong dtype, dates left as strings, or columns you never needed all cost you memory and effort later.

Official documentation: pandas.read_csv

The arguments that earn their keep

pd.read_csv(source,
            usecols=['a', 'b'],        # read only these columns
            parse_dates=['when'],      # real datetimes, not strings
            index_col='id',            # use this column as the index
            dtype={'zip': 'string'},   # stop pandas guessing
            na_values=['-', 'n/a'],    # what counts as missing
            nrows=1000)                # peek at a big file

source can be a path or a URL. Pandas fetches the URL for you, which means an example like the one below runs anywhere without downloading anything first.

I go through these in more depth in The six most important read_csv arguments in Pandas, and cover getting the index right on the way in with Load your data with the right index in Pandas.

A worked example, on real data

The USGS publishes every earthquake it records, as CSV, through a public API. No key, no signup — just a URL with a date range and a minimum magnitude. Bamboo Weekly #3 used this same source, and reached for parse_dates for exactly the reason below.

Let's find the five largest earthquakes of 2024:

import pandas as pd

url = ('https://earthquake.usgs.gov/fdsnws/event/1/query.csv'
       '?starttime=2024-01-01&endtime=2024-12-31&minmagnitude=6')

(
    pd.read_csv(url,
                usecols=['time', 'place', 'mag', 'depth'],
                parse_dates=['time'])
    .sort_values('mag', ascending=False)
    .head(5)
)

Which gives:

                            time   depth  mag                                  place
2024-01-01 07:10:09.476000+00:00  10.000  7.5  2024 Noto Peninsula, Japan Earthquake
2024-07-19 01:50:48.571000+00:00 127.291  7.4 41 km ESE of San Pedro de Atacama, ...
2024-04-02 23:58:12.173000+00:00  40.000  7.4        15 km S of Hualien City, Taiwan
2024-12-17 01:47:25.741000+00:00  54.372  7.3        24 km WNW of Port-Vila, Vanuatu
2024-06-28 05:36:36.902000+00:00  24.000  7.2            10 km WSW of Atiquipa, Peru

Two arguments did real work there. usecols meant we never read the dozen columns we did not want. parse_dates meant time arrived as a datetime, so it can be sorted, filtered by range, and grouped by month — none of which works on a string that merely looks like a date.

Three mistakes people make

Leaving dates as strings. Without parse_dates, a date column is just text. Sorting it puts "10 January" before "2 February", and .dt accessors are unavailable. Pass parse_dates at load time rather than converting afterwards.

Reading the whole file, then throwing most of it away. If you only need four columns out of forty, usecols reads only those — less memory, faster load. On a large file this is the difference between comfortable and impossible. The same goes for nrows when you are still exploring and just want to see the shape of things.

Letting Pandas guess dtypes on identifier columns. Zip codes, phone numbers and account IDs look numeric, so Pandas reads them as integers — and 02134 becomes 2134. Pass dtype={'zip': 'string'} for anything that is a label rather than a quantity, even when it is made of digits.

Practice it

Work through a read_csv exercise, with instant feedback and no signup required: practice.lernerpython.com/bamboo-weekly/read-csv/

Go deeper

Bamboo Weekly is the practice. If you want the structured version — full courses with downloadable Jupyter notebooks, plus live Pandas office hours when you get stuck — that is LernerPython+Data. A paid Bamboo Weekly subscription is included with it.

See it on real data

Below are the 96 Bamboo Weekly exercises that use read_csv on real-world data — try each one, then study the worked solution.

Part of the Pandas Methods Index. See also practice by skill.