> ## Content Index
> Fetch the complete content index at: https://www.bambooweekly.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Bamboo Weekly #68: Dangerously hot weather
- URL: https://www.bambooweekly.com/bw-68-dangerously-hot-weather/
- Published: 2024-06-18T09:55:35.000Z
- Updated: 2026-08-06T10:45:18.000Z
- Description: Get better with: Extracting from PDF files, multi-indexes, comprehensions, cleaning data, regular expressions, using "apply", plotting, and finding correlations.
- Author: Reuven M. Lerner
- Tags: pdf-extraction, multi-index, comprehensions, cleaning, regular-expressions, apply-function, plotting, correlations

Summer is starting, at least in the northern hemisphere. I keep hearing people say that it's surprisingly hot out. That's certainly true in Israel, where we expect hot weather during the summer (and often during the spring and fall, too), but the rest of the world is also experiencing unusually hot summers. Just this week, the New York Times reported that heat-related deaths are an increasingly big problem – for workers, for employers who want to keep them safe on the job, and the government, which wants to ensure a safe workplace. (You can read the article here: [https://www.nytimes.com/2024/05/25/climate/extreme-heat-biden-workplace.html?unlocked\_article\_code=1.vk0.ekJ8.Kg0h3dcMGz9d&smid=url-share](https://www.nytimes.com/2024/05/25/climate/extreme-heat-biden-workplace.html?unlocked%5Farticle%5Fcode=1.vk0.ekJ8.Kg0h3dcMGz9d&smid=url-share&ref=bambooweekly.com) )

The article cited a number of sources, one of which was from the National Weather Service ([https://www.weather.gov/](https://www.weather.gov/?ref=bambooweekly.com)), which publishes statistics about various weather-related disasters and hazards:

[https://www.weather.gov/hazstat/](https://www.weather.gov/hazstat/?ref=bambooweekly.com)

As we start to enjoy (or not!) warmer weather, I thought it might be interesting to dig into this data, to see if heat-related fatalities are really increasing -- and if so, by how much.

### Data and seven questions

On the National Weather Service's hazards list, there is a link to download the 80-year summary of all weather-related fatalities in the United States:

[https://www.weather.gov/media/hazstat/80years\_2023.pdf](https://www.weather.gov/media/hazstat/80years%5F2023.pdf?ref=bambooweekly.com)

As you can see from the file extension, it's a PDF file. You'll want to use the Tabula-py ([https://tabula-py.readthedocs.io/en/latest/](https://tabula-py.readthedocs.io/en/latest/?ref=bambooweekly.com)) package to read this into Pandas.

This week, I have seven tasks and questions for you to answer; I'll be back tomorrow with my solutions and explanations. The learning goals for this week include working with PDF files, nullable dtypes, plotting, and correlations.

The questions:

- Download the PDF file describing extreme weather incidents. Read the table into a data frame. We don't need the final "All Wx Fatalities" column. We also don't need the final three rows with summaries and totals. Ensure that both header rows are used for the header names. How much memory is being used? What dtypes are being used?
- Set all columns to be of type \`pd.Int16Dtype\` except for where \`pd.Float64Dtype\` or \`pd.StringDtype\` would be more appropriate. Remove any rows containing only NA values. Set "Year" to be the index. How much memory (if any) do you save by using these dtypes?
- What is the first year in which each measurement was taken?
- The New York Times story says that "heat kills more people each year than hurricanes, floods and tornadoes combined." Does this match the data we have? Show it both numerically and in a bar graph, comparing on an annual basis.
- What is the percentage year-over-year change in heat-related deaths? Do we see them increasing in the last few years?
- With which other weather-related fatality types do we see the strongest correlation with heat fatalities? Is that a strong correlation?
- How much, in millions of dollars, is the damage due to weather-related incidents? Create a line plot showing that amount. Can you think of what the peaks might represent?

I’ll be back tomorrow with my solutions.

Reuven