Following up on the histograms showing the age distributions of the US population over time that we discussed in class, here is a related set of histograms that were featured in the NYT Business section a couple months ago, as part of an article titled “Younger Turn for a Graying Nation“:
That was an installment of a weekly column in the NYT Business section titled “Off the Charts,” which discusses a graph and the underlying data every Saturday.
I hope you all got something out of studying statistics this past semester. I’ll continue to occasionally post relevant links here–as I tried to emphasize, data and statistics are becoming increasingly important is just about every field. Here is a recent New York Times article which makes that point, headlined “The United States of Metrics – NYTimes.com“.
An excerpt:
In the last few years, there has been a revolution so profound that it’s sometimes hard to miss its significance. We are awash in numbers. Data is everywhere. Old-fashioned things like words are in retreat; numbers are on the rise. Unquantifiable arenas like history, literature, religion and the arts are receding from public life, replaced by technology, statistics, science and math. Even the most elemental form of communication, the story, is being pushed aside by the list.
It goes on to outline a number of such areas: health, sports, social media, social science, “lifestyle.”
Here is an article on the (recently launched) data journalism website FiveThirtyEight.com, with more details on the example we discussed in class recently: the huge number of possible ways of filling out an NCAA tournament bracket, and hence the extremely small probability of anyone filling out a perfect bracket.
Via Buffett’s Billion Won’t Lead to a Perfect Bracket | FiveThirtyEight:
Before any games have tipped off, the NCAA tournament already has an almost surefire winner: Warren Buffett.
As you’ve likely heard, the billionaire investor is backing a prize of $1 billionto anyone who correctly picks the winner of every game in the men’s college basketball tournament. Announced on Jan. 21, the contest grabbed far more attention than any perfect bracket competition ever had before.
No forecast, even FiveThirtyEight’s, is a sure thing. But here’s one prediction we can make with almost complete certainty: Buffett won’t have to pay out. Mathematicians disagree on precisely how difficult it is to make a perfect bracket, but they agree the chance is so small that it’s not worth quibbling over.
The article subsequently discusses the 9.2 quintillion number–which we explained and calculated in class, as 2^63, since filling out a bracket consists of choosing 1 of the 2 teams to win in each of the 63 games. In fact, the article makes the connection with the chances of predicting 63 coin flips correctly, and goes on to discuss some more realistic estimates for the probability of filling out a perfect bracket:
Correctly predicting any one game in the tournament may be only a bit easier than calling a coin flip — not very good odds. Predicting all 63 games is nearly impossible.
Several mathematicians offered explanations for this difficulty, ranging from the abstract to the concrete. DePaul University mathematician Jeffrey Bergen, in a 2012 YouTube video that resurfaced after Buffett offered his billion, pointed out that predicting 63 coin flips correctly was a 1-in-9.2-quintillion proposition.1 That’s because the probability of getting 63 out of 63 right is the product of the probability of getting each one right, which for a coin flip is 50 percent.
Most games aren’t coin flips, though. A No. 1 seed has never lost to a No. 16, which makes those round-of-64 games about as close to sure things as big sporting events get. Make a few assumptions about your chances of calling each game right and you can get estimates. They’ll be very sensitive to the assumptions, though, because they’re being multiplied together 63 times.2
The challenge is so enormous that some scientists consulted for this article shrank from providing detailed calculations in their estimates of the probability of choosing a perfect bracket. Their guesses ranged from about 1 in 5 billion to 1 in 135 billion. In response to a journalist’s query, University of Minnesota biostatistician Bradley P. Carlin used 1 in 128 billion, which is “just the opinion of one expert I found online,” he said.
That expert is Bergen, who cites the estimate in his video but doesn’t explain it. At my request, he provided his assumptions: that the probability of calling a first-round game correctly ranges from 51 percent for the No. 8 vs. No. 9 game to 100 percent for the No. 1 vs. No. 16; and that second-round games can be called with 65 percent accuracy. The figures are 60 percent for Sweet Sixteen games and 50 percent for every game from the Elite Eight through the final.
“This is based on some simplifying assumptions and could be tweaked based on historical data, but is probably not unreasonable,” Bergen said. (He has since posted a new video explaining his calculation.)
It’s worth watching the two short videos by the DePaul mathematician Bergen cited above: the 2012 video, explaining the 2^63 = 9.2 quintillion number; and the video from last month, explaining some assumptions that lead to a 1-in-128 billion probability of filling out a perfect bracket
You can view the GoogleDoc spreadsheet we built in class on Monday, which calculates the mean and standard deviation for the quiz score data set (from the Day 3 and 5 Handouts), by clicking here.
You should be able to click through the cells and view the formulas that are used to do the calculations. If you have a Google account, you should be able to “Make a Copy” (under the File menu) which will save a copy of the spreadsheet to your GoogleDrive. Then you can edit it, and try to make a version that you can use for the homework exercises.
If you’d rather use Microsoft Excel, try downloading via this link, which should give you the same spreadsheet converted to an Excel file.
I strongly encourage you to spend some time understanding the basics of how to set up a spreadsheet like this, and to try to use spreadsheets on homework assignments to do your calculations. As I’ve said in class, if you do deal with data in the future, it will most likely be within the context of a spreadsheet.
Here are the scatter plots I showed in class on Monday (the graphs were also posted and discussed on this USA Today blog). You can read about scatter plots in Sec 2.2 of the text. We will return to paired data sets, scatter plots and regression towards the end of the course.
We discussed frequency distributions and histograms last week, and they will be central concepts in the course. Here is the example I showed in class from the New York Times’ website, which shows the age distributions of the US population over time, as well as a similar link from the Washington Post:
What do you notice about how the distributions evolve over time? Click thru to either the CalculatedRisk blog post on which this animation first appeared or to the WashingtonPost link to read some discussion.
Welcome to the OpenLab for my Spring 2014 sections of Math 1272, Introduction to Statistics. I will use this blog primarily to post examples illustrating the concepts that we will cover in the course.
If you’re a student in one of my sections for this course, you should create an OpenLab account (see the instructions here) and join the course group. That will allow you to participate in the discussion forums, and you will also get email notifications when I post here or in the discussion forums. In particular, I will use the discussions forums for posting announcements about homework, office hours, etc.
(To join the course group, once you have an OpenLab account and have logged in, click on the “Course Profile” link at the top left of this page (just above the picture).  Then click the “Join Now” button, which should appear just underneath course photo.)
The OpenLab is an open-source, digital platform designed to support teaching and learning at City Tech (New York City College of Technology), and to promote student and faculty engagement in the intellectual and social life of the college community.
