Originally Published: December 30, 2009

When statistics lack meaning

It is 1:30 a.m. here in Lansing, Michigan, and the heat in my study is slowly seeping through the glass of a closed window. Obviously, I have not gone to bed. Tonight, the blame falls on a meaningless statistic.

Jonathan Fields tweeted this out a few hours ago. It was retweeted by influential blogger Chris Brogan and has made some early rounds in the Tweetosphere.

92% of resolutions fail because they’re largely outcome focused. This year, zoom in on the process, too.

Okay. I might be able to believe that 92% of resolutions fail, but Fields offers a solution and presents it as a pearl of retweetable wisdom.

I think it is bullshit.

Where does the number come from?

A few years ago, Steven Shapiro authored a self-help book instructing people how to live a “goal-free” life. To help promote his book, he sponsored a telephone survey of 1000+ people with assistance from the Opinion Research Corporation.

A press release about the survey claims, “Only 8% of Americans say they always achieve their New Year’s resolutions.” Flip it around and you will see where the 92% number came from.

Those are people, not resolutions

Fields’ tweet implies that an average person making 10 resolutions will fail to accomplish 9 of them. This is a misinterpretation. It actually suggests that if you grab 10 people at random, you will usually find that 9 do not always achieve their resolutions.

It is a subtle, but crucial difference.

More than 8% of people achieve resolutions

The cherry-picked data does not suggest that only 8% of people achieve their resoultions, just that 8% of people do not always achieve their resolutions. And that makes sense, because I might list 20 things I want to do in 2010 and if I fail to do one, then technically I am a member of the 92%.

Unfortunately, we are not privy to the other side of the coin. The discussion was biased to support the author’s goals.

Not all resolutions are created equal

When you write down (or think up) your resolutions for 2010, odds are that the first few on the list will be more important than those following it. Resolutions are not a commodity.

Because they have different values, it is possible that someone can fail most of their resolutions yet still feel a strong sense of accomplishment at the end of the year.

And what does it even mean to “zoom in” on the process anyway?

This line actually made me laugh out loud. Not only did Fields invent rationale for a statistic pulled from a press release, but he went a step further and prescribed a remedy. And not just any remedy, an abstract, buzz-wordy remedy guaranteed to make people salivate without knowing what the heck it even means.

In conclusion

We do not need hard data to see how dubious this statistic is. A handful of logic and a willingness to question credibility does just fine to debunk.