When Statistics Blow Your Mind

I am an apologist for statistics. It's time for everyone to accept that. I have.

I love stats. I also fear stats. Statistics are easily manipulated – indeed, in this post-modern world one could argue that all statistics, by virtue of being measured, are manipulated. I'm sure that I'm not the first person to wonder about Heisenberg's Uncertainty Principle as it relates to social indicators. Concepts like "teaching to the test" and "gaming the statistics" exist because people attempt to achieve a statistical gain – in, let's say, student intelligence as measured by a test – at the expense of meaningful advancement. It's what happens when the numbers become more important than what they try to describe.

I think the power of statistics, though, is when they help us to understand a particular dimension of a difficult or complex problem. Hidden complicating variables can obscure a seemingly obvious connection between simple events. The challenge to consider all the complicating factors is a thrilling mental puzzle. Using 14^th Century England as an example – because, why not? – it would seem that the significant decline in exports witnessed in the middle of the century might be attributed to the outbreak of the Hundred Year's War. But with regression analysis, one can see that other factors include poor harvests from a changing climate and -- quite significantly -- over-taxation by the English Kings (Marriott, 1999, pg. 30).

Mostly what I love about statistics (and numbers and indicators and regressions and all of the rest of it) is that it is so interesting to find multi-dimensional connections between events or actions that seem, perhaps, intuitively related but are not proven to be related in that way. Then, with an intelligent model and some good data, suddenly everything becomes clear! It's that move -- from intuition to illustration – is where the magic is for me. It's where people have epiphanies that lead to brand-new research.

[Of course, when one suspects an intuitive link, and goes hunting for results, one will invariably suffer from confirmation bias and a myriad of other statistical ailments. Let us put those aside for another post.]

The reason I'm writing today [instead of studying for my macroeconomics midterm] is that I was recently reminded of Hans Rosling, a Swedish MD/MPH who does some pretty cool stuff at his site Gapminder. He speaks regularly to TED (the "Ideas Worth Spreading" people) about a number of global issues. In this particular video, he's discussing the relationship between HIV/AIDS prevalence and Income. I think it's a pretty good example of what happens when a compelling speaker finds persuasive data and develops a simple, elegant way to demonstrate an outcome. Essentially, this is the highest and best use of statistics, to my mind:

So carve out 10 minutes, and enjoy Hans Rosling.