> ## Content Index
> Fetch the complete content index at: https://www.bambooweekly.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Bamboo Weekly #33: Fracking
- URL: https://www.bambooweekly.com/bw-33-fracking/
- Published: 2023-09-27T15:01:02.000Z
- Updated: 2026-08-06T10:45:47.000Z
- Description: Get practice with CSV files, multiple files, comprehensions, dates and times, formatting, missing data, plotting, and grouping.
- Author: Reuven M. Lerner
- Tags: csv, multiple-files, comprehensions, datetime, formatting, missing-data, plotting, grouping

However much we might want and need to move away from fossil fuels, our world continues to depend on them to a huge degree. Over the last few decades, a growing amount of oil and gas have come not from traditional oil rigs, but rather from a technique known as "fracking" ([https://en.wikipedia.org/wiki/Fracking](https://en.wikipedia.org/wiki/Fracking?ref=bambooweekly.com)). That technique, also known as "hydraulic fracturing," forces oil and gas to emerge from deep underground by injecting high-pressure fluid. The good news is that fracking has made it easier and cheaper to extract petroleum from the ground.

The bad news? There's actually quite a bit of it: First, cheaper prices for oil and gas encourage us to use them more, just the opposite of what we need to combat climate change. Second, there are numerous worries about the environmental issues associated with fracking, from polluted groundwater to earthquakes.

![](https://storage.ghost.io/c/06/ba/06ba0cc0-be6f-4de7-af2f-5c20165279b9/content/images/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https-3a-2f-2fsubstack-post-media.s3.amazonaws.com-2fpublic-2fimages-2fe084dfd3-ae3d-4796-91f6-18bdf6267a75_1024x1024.jpg)

Dall-E, in response to: A car (in its entirety) on the beach. A nozzle is filling the car not with gas, but with water from the ocean. .png

I had learned a bit about fracking several years ago from NPR's Planet Money podcast. In a series they did on oil, they discussed the history of fracking, and its good and bad sides: [https://www.npr.org/sections/money/2016/08/17/490375230/oil-3-how-fracking-changed-the-world](https://www.npr.org/sections/money/2016/08/17/490375230/oil-3-how-fracking-changed-the-world?ref=bambooweekly.com)

This week, [the New York Times reported](https://www.nytimes.com/interactive/2023/09/25/climate/fracking-oil-gas-wells-water.html?unlocked%5Farticle%5Fcode=zFnLnDWVyCJEtxBQzfNZVjRdUyKaabTnYWa%5FLpjLLhCP2F9n3d4FH2WMBoqol5mRScexzGwLhoRpDWBO7jcUehRt3vwU8-w%5F7JONRJkKU5g1reDVORf5wHK3WceY7Vzy1Uf-1KRb5YRihRGjaq%5FywzOsyDqZL7q4zLsSB0fQIu6rg%5FBIpgaQe0d3hgo2jO7sGUUr-ax%5Fv0z-rrqQAy7Xo61vn2BG4GLIwTjENxOsWa6Hrr646lYUCCiarPsiBUanmzCes5FMg6nt80B4t8keLp6Wk7xW4V8oGtA1dVqb1lLeJWv9yc1YTkfbV%5FDldCTm7eI8vXbvcKNYGQEA5PBcKSmUYJu2aCHikXB9vNeXV3%5F5CQ&smid=url-share&ref=bambooweekly.com) that there's a separate environmental issue with fracking, namely the huge amounts of groundwater that oil companies are using. This is especially a problem in the Western US, where there is a serious water shortage.

The NYT story said that they had used data from FracFocus ([https://fracfocus.org/)](https://fracfocus.org/%29?ref=bambooweekly.com), a registry of fracking sites that gets data from 27 states. I thought that this might be an interesting data set for us to look at this week. And indeed, that's what we'll do.

### Data and eleven questions

This week, we'll look at the data from FracFocus. The files are all in CSV format, and can be downloaded via this link:

```
    https://fracfocusdata.org/digitaldownload/FracFocusCSV.zip
```

The download page with full information about the data files is at https://fracfocus.org/data-download . Note that there is a data dictionary that comes with the files, in a document called \`readme.txt\`. That file seems to refer more to the SQL version of the data, rather than the CSV version, but is close enough to understand things.

I have eleven tasks and questions for you this week. The learning goals are: Working with multiple files, parsing and handling weird date formats, formatting numbers in Pandas, pivot tables, selecting columns based on another query, grouping, cleaning data, and looking into the differences between mean and median.

I'll be back tomorrow with my full answers to these questions, along with the Jupyter notebook that I used to solve them:

- Import the "FracFocusRegistry" CSV files into a single data frame. Include only the columns: JobStartDate, TotalBaseWaterVolume, StateName, CountyName, and FederalWell. Don't bother trying to parse the \`JobStartDate\` column into datetimes just yet. Let them be strings.
- Now turn \`JobStartDate\` into datetime dtypes. If the input date string cannot be parsed, then leave it as \`NaT\`.
- Tell Pandas to display numbers with up to 3 digits after the decimal point, and with commas before every three digits.
- Remove any rows that contain null values (\`NaN\` or \`NaT\`). How many rows did you remove?
- How many entries are there for each state? Given that there are 50 states in the US, how do you explain the number of results we get? How could you fix that?
- Create a line graph showing how many (total) gallons of water have been used, per month.
- Create a line graph showing how many (total) gallons of water have been used, per month, in each of the 10 most commonly mentioned states in this data set.
- Create a line graph showing the mean number gallons of water have been used, per month, in each of the 10 most commonly mentioned states in this data set. Do all locations seem to use the same amount of water?
- What five counties have used the most total water in the last two years?
- What are the mean and median amounts of water used for all of the wells in each county? Where do you see the greatest difference between mean and median – and what does that mean?
- Do federally owned wells use, on average, more water or less than commercially run wells?

Comments, questions, or anything else? Please leave a comment.

Meanwhile, I’ll be back tomorrow with my solutions.

Reuven