Bad Science
Quacks, Hacks, and Big Pharma Flacks
by Ben Goldacre
Ben Goldacre takes issue with individuals using pseudoscience to promote health related topics and items. His position is that there is absolutely nothing wrong with eating healthy, exercising, and avoiding things like excessive alcohol use. However, based on current information, only about 13% of treatments have good evidence to support them and another 21% are likely to be beneficial.
While Goldacre tackles the issue of pseudo-science, he explains it through story-telling by referencing celebrity doctor endorsements, pharmaceutical studies, famous court cases, and famous medical myths. This takes a what could have been a dry book on science and turned it into an enjoyable and eye-opening read.
Why should HR and Benefits people read this?
All of the companies that try to sell to us quote tons of different studies. Most of them are flawed studies, often sponsored by the company itself. This book explains how to tell what a good study is, how to interpret the findings, and how to not get sucked into common biases.
Main points from the book:
Why is there so much “Bad Science”?
He cites a study published in the March 2008 edition of the Journal of Cognitive Neuroscience that showed that people will believe fake explanations when technical or scientific words are included in the explanation. Similar research has shown that people tend to consider longer explanations as “better” than shorter explanations.
What is “Good Science”?
There are good studies and bad studies. Sometimes, studies are considered methodologically flawed if a study was poorly constructed and performed. The best studies include trials – for example, taking 200 people, dividing them into random groups, letting them go through a process, and measuring the individuals over the time in the study. In addition, there are also some important aspects that need to be in each trial:
Blinding: An important feature in a good trial is blinding, when the researchers don’t know which individual received the real pill and which one received a placebo.
Randomization: There are different methods of randomization and some lead to more fair methods than others. For example, calling a special phone number and getting assigned based on a computer randomization program. In studies, poorer methods of randomization led to overestimated treatment effects by 41%.
Once there are a significant number of trials on a given subject, a meta-analysis can be done. These collect all of the results from all of the trials on a given subject and mathematically review all of the results. The Cochrane Collaboration is one of the most widely known non-profit groups that produces summaries of the research on health care.
Common errors in interpretation
While a study itself might have been conducted appropriately, once it is published, there are several errors that often creep into the publicity around the study as the findings are picked up by the media.
Data is non-existent
Sometimes studies get quoted in the media that do not exist. Always check to see of the study is real before relying on it.
Observation or Intervention
Look at the study to see if the media is interpreting research incorrectly. Is the research an observational study or an intervention study? Was there one group being given something and then measured on it or was the study a view of lifestyle habits across groups?
Over-extrapolation
Was the study done on humans or on rats? The study being quoted needs to be reviewed to determine who or what it was conducted on and looking for any surrogate outcomes. A surrogate outcome is when a trial shows changes in a blood or biomarker but does not necessarily assess causality or link to an outcome.
Misrepresentation of Figures
The media sometimes quote the relative risk, the number that will be impacted above the naturally occurring number versus the absolute risk, the total number. For example, the media might report that a risk is 50% higher if you have xyz condition. In actuality, this could mean 2 extra people out of 1,000.
