One thing that is challenging for me in my role as a public health nutritionist, is that people fairly frequently, say something strange and when I ask about it their response, is in essence, “It’s real; I read a study.”

Trying to weed out studies that you should believe and those that may be flaky is not always an easy job. You may want to consider giving yourself a few rules to go by, when deciding whether a study is flaky or not. Think about some of the following:

• One of the prime concepts when studying public health is the idea of the type of study. If a study is an observational study (also called an epidemiological study), it can identify associations (relationships, correlations). What’s driven home to public health students is that this type of study, does NOT PROVE anything. This type of study does not allow you to say, “X” causes “Y.” To prove something causes something else you need a well-designed clinical trial. So on this first idea, be sure to watch out for what type of study recommendations are based upon. Sometimes a clinical trial is not possible and the best we can hope for is just finding relationships, usually that type of study is done repeatedly, in order to strengthen the findings.
• Is a single study being cited? It’s rare that a single study overturns all the previous studies. Be very careful when you read something that is contrary to a plethora of evidence. Studies like that are sometimes published. It’s best to get the “sit tight” attitude; wait to see if some other studies can duplicate whatever was supposedly discovered.
• The credibility of the source is very important. For one thing, try to see if you can bump into a conflict of interest before thinking that a study is valid. You’d maybe be surprised with the number of questionable study outcomes that are paid for by the very product that is being studied. E.g. If the sugar industry/association is paying for a study, and the outcome is reported that sugar is really a health food; maybe wait to start piling on the sugar, until you read a study not supported by the sugar industry. In addition, if you’ve never heard of the journal or university, that may be a big clue that you should look into the study a bit. I really enjoy finding studies, in which the big public health universities, where qualified health experts and/or scientists are involved. Also, the elite public health journals are more likely to publish peer-reviewed, quality research.

By contrast, there are “predatory journals” that are designed to give anybody a quick publication, for a mere $50 or so. Anything can be published. There is no serious editorial work, peer reviews to find and correct errors, or rejection of unsound methods or unsupported conclusions. Jeffrey Beall at the University of Colorado created a list of such journals. You can use it to evaluate journals questionable as sources by using his on-line list. (Beware; it is really long!) A respected science journalist, John Bohannon (2013), wrote a terribly flawed fake article full of so many serious errors that any journal that uses even a single competent peer reviewer should have rejected it for publication as scientifically unacceptable. He submitted computer-generated variations of it in 2012 to 304 open-access journals. Well over half, 157, agreed to publish the one they received. [Note the citation is: Bohannen, J. (2013) can be found in Science 342 (Oct 4): 60-65]. This publish for profit is another influence corrupting the public’s perception of science. We as readers must learn to sort the wheat from the chaff, as good evidence is the only thing we should accept to back up conclusions, not opinions, politics, or the money of companies willing to generate “fake news” about their products.

• Becoming critical readers is essential. Tufts University recently reminded its readers to be careful with the wording reported in studies. Sometimes there are reports that jump from “a result suggests” to this “proves.” Or saying something “may help” to “it will help.” Reliable sources are very careful in selecting the accurate terms.
• One last issue to notice is who or what was the research done on? Were the people like you in age, sex, race, and lifestyle? Were the subjects animals or people? These studies may be important, but they may not apply to you.

By asking yourself a few questions, you can be more likely to avoid getting drawn into believing flaky research.