Numbers don't lie, but graphs can

By Daniel Kaplan

Last week’s front-page article on financial aid used a graphic to illustrate a sharp drop in the fraction of Macalester students on need-based aid. This graphic was based on incorrect numbers. But beyond this, it was misleading in a way made famous in Darrell Huff ‘s classic book, “How to lie with statistics.”I don’t want to reprint the original graphic since the numbers on which it was based were grossly wrong. But to show how the style of graphic is misleading I have made the equivalent using correct data for 2005 and 2007.

The graph is honest in the sense that the bars end in the right place; it’s a lie because the bars begin in the wrong place. A graphic like this conveys almost no information and considerable disinformation because the visual impression left by the graph is that the fraction of students on financial aid has dropped severely. If you look at the numbers – 70.1 percent in 2005 and 66.2 in 2007 – the change doesn’t seem nearly so big.

To decide whether the drop is big or small requires some basis for comparison, some context. One relevant context is the typical year-to-year fluctuations in financial aid. These arise just due to the unpredictable changes in which students choose to come to Macalester after being admitted. Every year, the college admits enough financially needy students to more than fill the entire entering class: the actual composition of the class is determined just as much by these admitted students’ decisions as by any admissions policy.

Another relevant context for interpreting the year-to-year change is the range displayed by other colleges. After all, why should the 2005 figure of 70.1 percent be taken as a benchmark. Perhaps Macalester should have been at 90 percent all the time. Or perhaps 60 percent is a pretty good rate.

The second graphic shows what fraction of students received need-based aid at each of Macalester’s peer schools. These data are for 2003 – the last year I have available for published data from those other schools. For Macalester, much more data is available publicly and I have added Macalester’s data for 2005 and 2007, the two years cited in The Mac Weekly article.

Seen in the context of our peer schools, the change from 2005 to 2007 is small. Macalester continues to provide much more financial aid than just about any of our peer schools.

Macalester has always had need-aware admissions for foreign students and for domestic transfer students. Three years ago, a college committee consisting of faculty, students, and staff unanimously recommended adopting a need-aware policy for about 10 percent of domestic first-year students. At the time, we anticipated a reduction from having about 70 percent of students on aid to having about 62 percent. The plan called for a four-year ramp to implement the policy fully. Now, two years into the four-year process, we are at about 66 percent – right on target.

Daniel Kaplan is the DeWitt Wallace Professor in the department of Mathematics and Computer Science. Contact him at [email protected]