Labor unions, and specifically US labor unions, have been in the news quite a bit lately. Whether it's UPS drivers (who reached an agreement), Hollywood writers (on strike for more than 140 days), Hollywood actors (on strike for more than 60 days), and auto workers (currently on strike for five days), we're hearing quite a bit about and from union members.
Unions aren't a panacea for stopping the abuse that so many companies and managers inflict upon their workers, but in the last century, they've helped to improve the conditions and wages for workers all over the world. A lot has changed in the last century, though. Heck, a lot has changed just in the last few years, thanks to technology and the pandemic.
Among them: Companies are earning record profits, people want (expect?) to work from home, multinationals are able to hire people outside of the US (who don't belong to a union), unemployment is at record lows, and AI might soon be able to replace many workers, including writers and actors. Any one of these issues would be enough to cause trouble for labor relations, and when you combine all of them, it's no wonder that employers and employees are feeling rather tense. Add to the mix next year's US presidential elections, and Joe Biden's long-standing pro-union sentiments, and it just gets better. (Or worse.)
I thus decided that it might be worth looking into some data about union membership in the US. Do union members earn more than non-members? If so, how much more, and is that true across different jobs?
Where could I go to get such data? The US Bureau of Labor Statistics (https://www.bls.gov/) is all about collecting that kind of information, analyzing it, and then disseminating it to the public. The thing is, the BLS data is a bit annoying to find, download, and then assemble. Fortunately, three labor economists (Barry T. Hirsch, David A. Macpherson, and William E. Even) have taken this data and put it into easy-to-read Excel documents at their site, UnionStats.com (https://unionstats.com). We'll be taking our data from there this week.
Data and 10 questions
Our data all comes from Unionstats.com. This site contains a large number of Excel spreadsheets summarizing and analyzing BLS data.
This week's learning goals include working with multiple Excel files, taking data from those files and manipulating them into useful data frames, comparing data across files, working with text, and plotting. I'll send you my complete solutions to these questions, along with the Jupyter notebook I used to solve them, tomorrow. Meanwhile, here are my 10 questions:
- First, let's look at union membership per industry. Read in the data for union membership across industries in 2022 from from http://unionstats.com/ind/xls/ind_2022.xlsx, on the page http://unionstats.com/ind/ind_index.html.
- Looking only at the rows containing top-level job categories (where all of the words are capitalized): Which five categories have the highest union membership? The lowest?
- Looking only at the rows containing specific job categories (where the words are a mix of capital and lowercase letters): Which five job categories have the highest union membership? The lowest?
- Retrieve the Excel files from 1983, 1993, 2003, 2013, and 2022. Create a data frame in which the rows are the years, and the columns are the top-level (all-caps) categories, showing the percentage of union membership.
- Create a line plot with the above data.
- Now let's find out how much more union members are paid: Create a data frame from http://unionstats.com/wages/xls/wages_all.xlsx .
- Why don't you need to remove the copyright notice (on line 54 of the Excel file) from your data frame?
- Create a line plot showing the adjusted union wage premium, with the years as the x axis. What can you say about the salary benefits of belonging to a union over the last few decades?
- Now create a new data frame, based on Excel files for specific demographic groups at http://unionstats.com/wages/wages_index.html : White male/female, Black male/female, and Hispanic male/female. The data frame's index should contain years, and the six columns should contain the adjusted union wage premium for each of these groups in each of these years.
- Create a line plot showing the adjusted union wage premium for these groups. Which groups seem to enjoy a larger premium from belonging to a union, and which have a smaller one?
Thoughts? Questions? Suggestions? Leave a comment!
I’ll be back tomorrow with my solutions.
Reuven