Reading about research results is something we all do a lot. But how do you know if what you are reading is the truth? It used to be that you could rely on source credibility – certain sources are more likely to tell you the truth because they have a reputation for getting things right. Source credibility used to be tied up with media names like The New York Times or the major TV networks. But now that Amazon and Netflix are making television and bloggers get press credentials, it’s harder to use that alone as a rule of thumb.
When you read about research, here’s a couple of things you can ask yourself:
Who is giving me this information, and why? This relates to source credibility, but is harder to judge now. Look at lots of things from this information source to see if you can detect patterns of bias (like reading back into the archives of a blog). It’s also good to look at the quality of the logic. If lots of things don’t make sense, it may be that the source isn’t qualified to evaluate the issue.
How was the research information collected? Most research studies that report with numbers rely on the notion of randomness – basically, did every person/thing affected have an equal opportunity to be chosen for the study. Ask yourself if the people or things evaluated relate to the question being asked. If you’re investigating how adults use cell phones, but you pass out a questionnaire in a college class, you didn’t do a good job because most types of adults had no chance of being surveyed.
Do the conclusions match the evidence? It’s usually good practice for a news report to focus on the big conclusion. Check if the conclusion drawn makes sense given what the data said. If you can’t tell, there’s a problem.
It’s common to see research in the form of polls. This article explains how people are chosen for a poll and what it means.