BW #37: Consumer finances

Announcement: PythonDAB 4

I’m excited to announce that the fourth cohort of PythonDAB, my Python Data Analytics Bootcamp, will be starting on November 8th. It’s an online, four-month introduction to Python, Git, NumPy, and Pandas with twice-weekly office hours and projects with real-world data.

For more information, go to https://PythonDAB.com, where you can read about the course and sign up for a free coaching session with me, where we’ll discuss your career and whether PythonDAB is a good fit for you.

I’m also holding a free webinar this coming Sunday, where I’ll present the course and answer questions. The webinar will be recorded, so if you can’t make it, you can always catch it afterwards. Register for the webinar at https://store.lerner.co.il/pythondab-4-webinar-october-29th-2023.

PythonDAB is my biggest and most ambitious course. It’s a great investment in your career and future, using all of the best educational techniques and technological resources that I know. I hope to see you at the webinar and/or in a coaching session!

Consumer finances

You probably know that the Federal Reserve, the equivalent of a central bank in the United States, sets monetary policy for the US, including interest rates. But in order to set policy, it also spend a lot of time researching the economy. Every three years, it produces a research report called the “Survey of Consumer Finances,” giving a snapshot of American households.

The latest edition of the SCF was just released. A 58-page document describing not just this year’s survey, but comparing it with previous surveys, is downloadable from /content/files/publications/files/scf23.pdf. I found it particularly interesting to compare pre-pandemic numbers with current ones.

They found all sorts of things in their data, including that net worth increased dramatically, and that a large number of American families are now invested in the stock market. Moreover, a huge number of families own businesses — so the entrepreneurship boom that we thought we saw during the pandemic was real.

This week, we'll look through a small part of this report, asking questions about the state of American families' finances past and present.

Data and seven questions:

The Survey of Consumer Finances is based on a massive amount of data. You can download that raw data from the SCF home page, at https://www.federalreserve.gov/econres/scfindex.htm . However, I can warn you that this data is very large, as well as hard to slog through and understand. Just the codebook that describes the variables and what they mean is 48,000 lines long — the rough equivalent of more than 700 pages of printed text.

A better bet is an Excel file with a number of sheets, each of which corresponds to a different table in the survey report, downloadable from /content/files/econres/files/scf2022_tables_public_nominal_historical.xlsx.

This file has more than enough data for us to play with, asking some interesting questions and exploring Pandas features.

And indeed, this week I have seven questions and tasks for you. The learning goals include working with Excel files (including rejiggering spreadsheets into useful data frames), multi-indexes, and plotting. As usual, I'll share the Jupyter notebook that I used to answer the questions with you tomorrow, along with my solutions and explanations.

This week's tasks:

• Create a data frame from the first two sheets (Table 1, 89-98 and Table 1, 01-22), where the index will be the descriptions (in column A) of rows 34-37, and the values will come from each year's values. The result will be a data frame with four rows and a multi-index columns, with the years on the outer level and "median" and "mean" in the inner level.
• Create a line plot with a separate line for each educational level, the x axis representing years, and the y axis showing median income in 2022 dollars. Which level of educational achievement appears to be improving its mean income level the most over the years?