Two Minute Tips

How to check a source’s statistics

January 8, 2018

Share this article:

It’s up to reporters to discover whether statistics are valid or pulled out of thin air. (Image by Flickr user Donnell King)

You’ve probably noticed that PR departments and agencies often try to make their pitches seem like authentic news. To that end, many will toss in statistics and other numbers.

In marketing, this is called verisimilitude, or the appearance of being real. It’s why advertisements might use a precise number, instead of a rounded figure, to convey authenticity. A retailer that says it is selling a product for $16.13 less than a competitor’s price, rather than rounding that down to $16, taps into a belief that the more details you provide, the more truthful you are being.

Here are three steps to keep ahead of the masters of manipulation.

1. Ask for the study

Unless you know the source of the statistic, you can’t tell if it’s genuine or not. The company providing the statistic should be able to reference it or give you a link to the original study. If it can’t, you can likely disregard most everything else in the pitch or press release. If the source is a publication, you need to find out who or what provided the original data. Years ago I wrote an entire article on the false assumption in the world of tech media that 14 percent of all laptops were stolen. It turned out that not even the FBI or the insurance industry tracked that particular category of theft. The number was nonsense.

2. Look at the methodology and interpretation

Studies don’t matter unless they’re conducted in a scientific way. Always check the methodology, particularly how the study’s authors recruited respondents. I’ve seen many companies issue numbers regarding consumer behavior or preferences, only to learn that they had limited the poll to their own customers. Unless you are talking about their  customers, that’s not a valid sampling. Also, see if the way the number is presented is consistent with the way the study authors presented it. Taking one number out of context can make it appear to be something that it isn’t. For example, a statistic from a study could claim that the average person was willing to spend $1,000 on a new smartphone. But if the respondent demographics show the bulk of people were all high net worth individuals, you’d question whether the finding was valid in a more economically representational group.

3. Beware of old data

Although this isn’t always possible, you want fairly recent data. I try to keep to a one-year window. Society and business change constantly, and last year’s numbers may not reflect the current reality. In some cases, as with certain types of population surveys or government analyses (the U.S. census, for example, conducted every 10 years), the most recent information may be from data that is a few years old. If there aren’t better options, at least note the age of the survey.

The overall rule is to take time enough to look carefully. You don’t want to be spoon-fed verisimilitude chosen to sway your judgment.

More Like This...

Checking sources for ‘fake stories’

“Fake news” has become a thorn in the side of reporting. Yes, there are always stories that have failings in bias, bad research, or other problems. But the idea of

Double-check all the things people are sure they know

You’d think this would be a given, but too many business stories incorporate information that, while popularly believed, is simply wrong. If you don’t check statements that business experts, academics,

Forget the source’s fancy pedigree

You know the drill. You receive a response to an outstanding query, or maybe it’s something tepid coming in over the transom. The email lauds the potential source. Went to

Two Minute Tips

Sign up now.
Get one Tuesday.

Every Tuesday we send out a quick-read email with tips for business journalism.

Subscribers also get access to the Tip archive.

Get Two Minute Tips For Business Journalism Delivered To Your Email Every Tuesday

Two Minute Tips

Every Tuesday we send out a quick-read email with tips for business journalism. Sign up now and get one Tuesday.

Our New Look
The Reynolds Center for Business Journalism is starting 2023 with a new look that we hope better illustrates our core mission to provide accurate and authoritative resources about business journalism, in order to help both reporters and news consumers understand the importance of business news and to demystify the sometimes arcane topics it covers.
Businesses, markets, and economies move in cycles – ups and downs – which is why our new logo contains a “candlestick” chart representing increases as well as downturns, and serves as a reminder that volatility is an unavoidable attribute of modern life. But it’s also possible to prepare for volatility by being well informed, and informing the general public to help level the information playing field is the primary goal of business journalism. The Reynolds Center is committed to supporting that goal, which is why the candlestick pattern in our logo merges directly into the name of our founding sponsor, Donald W. Reynolds.
Our new logo comes with a shorter name. Business is borderless, and understanding the global links in supply chains, trade, and flows of funds and people is essential to make sense of our fast-paced, globalized world. So we’re dropping the word “National” from our name and will aim to provide content that is applicable to business news globally.
We hope you like the new look. Best wishes for 2023!