REGRET THE BUSINESS ERROR: Business journalists are probably more comfortable inside a factory or boardroom instead of a laboratory or operating room. But scientific and medical studies frequently impact the business world, which means you need to know when a study is presenting important new data, or bringing forward initial findings that may or may not matter.
For help with making sense of these studies, I spoke with Ivan Oransky, the executive editor of Reuters Health and an associate adjunct professor at New York University, where he teaches medical journalism. Along with Gary Schwitzer, the publisher of HealthNewsReview.org, he produced “Covering Medical Research: A Guide for Reporting on Studies,” which came out last year. (You get it free with a membership to the Association of Health Care Journalists, where Oransky currently serves as treasurer.)
Below are four tips from Oransky, who has an M.D. as well as being a journalist.
1. Get the full study and read it – “I think it’s journalistic malpractice to not have the full study in front of you when you’re reporting,” Oransky says. Read the introduction to get a sense of the overall study, and note all references to previous studies in the same field. The authors of those studies will make good outside sources. Oransky recommends examining any data tables to discover findings that aren’t laid out in other parts of the study. The discussion and results sections of any study are also key, and Oransky says it’s essential to read the limitations. “Even if you’re on a tight deadline, the limitations will often … put stuff in context for you,” he says.
2. Evaluate the type of study – “Was it done on humans or on animals or just in a petri dish?” Oransky says. “A lot of studies that may move markets and become a story are promising, but the reality is they are not going to make it to prime time or [FDA] approval, for example.” Other things to consider: was the study published in a reputable journal or presented at a major conference? “Or is it just a press release?” he says. Quality medical research often comes in the form of a late stage randomized controlled double blind trial (RCT), or a “meta analysis” that combines several RCTs and pools the data. If the material is labeled a case study or observational study, you’re dealing with a less rigorous form of investigation.
3. Ask dumb questions – If you lack experience dealing with scientific material, don’t be afraid to ask for definitions of jargon and scientific terms. This is no time to pretend you understand everything. Oransky says the science and medical industries are full of jargon that mask important details. “You’ll get off the phone and have a notebook full of gibberish and jargon,” he says. “You can’t be afraid of asking a dumb question.”
4. Beware of cause and effect – Be especially skeptical of studies that claim a direct link of X to a reduction/increase in Y. It’s rarely that simple, and this is where spin enters the equation. “One thing a lot of reporters make a mistake on is to ascribe cause and effect when you can’t really prove causal effect,” Oransky says. “You can’t say anything about causal effect unless you’re really properly controlling for every possible confounding factor.”