The Pitfall of Evidence-Based Nutrition & Medicine

hierarchy-of-evidenceThese days, you don’t have to look far and wide to find information about nutrition, fitness, and medicine. There are millions of blog posts, news stories, books, and magazine articles covering these topics out there. Many of these are written by people – including nutritionists, doctors, scientists, and self-proclaimed fitness experts –  who use Randomized Controlled Trials (RCTs), meta-analyses, and other similar research studies to back up their claims and opinions. These authors are generally considered to be more trustworthy than those who don’t cite any scientific research. This is probably an accurate perception, as there’s no doubt that the aforementioned types of studies are a key source of credible information about health, medicine, and nutrition. That said, just because an article or book includes plenty of references to scientific papers doesn’t necessarily make it trustworthy…

The limitations of RCTs, meta-analyses, and systematic reviews

If you’re a regular reader of Darwinian-Medicine.com, you know that I use scientific research to guide my understanding of health, medicine, and fitness, and you’ve probably noticed that virtually every article of mine includes links to studies that support the statements I’m making. In other words, I believe in using a science-based approach to nutrition, health, and medicine. That said, I never rely on RCTs, meta-analyses, and similar studies as my sole source of information, and I try to always remember to think twice before I draw any conclusions from a study or scientific paper.

The scientific literature on health, nutrition, and medicine is in some ways analogous to an area of shallow water, in the sense that it’s filled with reefs, sharks, rocks, and other dangers in the form of confounding variables, mediators, undisclosed conflicts of interests, chance findings, type 1 and 2 errors, and biases. If you have never learned how to read a research study or done a course in statistics or research design, you may be unaware of these dangers, which lurk in virtually every research study. Scientists generally try to eliminate as many of these problems as possible, for example by carrying out multivariate analyses as a way of controlling for confounders, but they rarely manage to remove them all.

What a lot of people also seem to forget when they use intervention studies to make inferences about diet and health is that a study doesn’t present us with a full picture of the topic in question; rather, it only provides us with some of the answers we need to make a truly evidence-based conclusion. This often holds true even when we combine the results of dozens of trials.

Many of the most commonly used study types in nutrition and medicine, such as cohort studies and RCTs, investigate how an exposure (e.g., consumption of fish oil capsules) affects one or a couple of outcomes (e.g., blood levels of c-reactive protein and low-density lipoproteins). This information can certainly be useful; however, it is also obviously very limited in nature.

RCTs and other similar research studies usually fail to pick up subclinical adverse effects, tend to be of short duration, and typically tell us little to nothing about the long-term effects the exposure under investigation has on human health. Also, we have to remember that most studies only tell us about the relationship between a specific exposure and one or a couple of outcomes; they don’t tell us how an exposure affects our overall health or physical fitness condition. It may be that the exposure induces adverse effects that are not measured.

Before we jump to conclusions, we have to consider the biological mechanisms and evolutionary evidence

So, where does this leave us? Should we just delete PubMed from our list of bookmarks and abandon the scientific literature altogether? No, of course not, we just have to be smart about how we use scientific data.

RCTs, systematic reviews, and meta-analyses are considered to be on top of the hierarchy of evidence, and there’s little doubt that these types of studies provide us with important information about how we should eat to attain a fit, well-functioning body. That said, it is a recipe for disaster to base our understanding of diet and health solely on these types of studies. To really be able to say something concrete about how a food, medicine, exercise regimen, or nutritional product impacts our health, we also have to consider the biological mechanisms that are involved, and perhaps more importantly, we have to look at what clues evolution has left for us.

I don’t claim to have all the answers, but if there’s one thing I’ve learned, it’s that without a basic understanding of evolution and human biology, it’s impossible to make sense of nutrition and medicine. The scientific literature – with all its RCTs, observational data, and cohort studies – shouldn’t be our first stop in the search for good health, but rather the second or third stop. Before we enter into the depths of PubMed, we must establish an evolution-based framework, or else it is impossible to make sense of all the seemingly conflicting scientific data that are out there.

The first questions I always ask myself when I read a scientific paper are: Are the statements, results, and conclusions found in this paper supported by the anthropological/evolutionary evidence, and do they make sense from a biological and physiological perspective? If the answer to these questions is no, then I quickly move on.

Any idiot can do a search on PubMed and find a couple of RCTs and meta-analyses that seemingly support the case he wants to make. For example, I would have no problem finding dozens of studies that seem to show that a vegetarian diet is the optimal diet for humans. I could then go on to write an article on the benefits of vegetarianism; an article that a lot of people would find highly convincing. The statements and conclusions in the article would quickly fall apart, however, if the studies in question were deciphered more carefully and the evolutionary evidence and biological mechanisms were considered.

What studies don’t tell us

I know that all of this may seem foreign to people who are not familiar with research design, so I thought I would finish off with a couple of examples that should help illustrate my points.

First, let’s do one on probiotics, a topic that should be familiar to the readers of this site. Several RCTs have shown that probiotic supplements improve markers of inflammation and alleviate symptoms of irritable bowel syndrome (1, 2, 3). If we just look at the results of these studies, without considering any other pieces of evidence, we’ll likely conclude that probiotic supplements are highly beneficial.

However, if we first establish which mechanisms that are involved and consider the role probiotics have played in the human diet throughout our evolution, we’ll likely come to a different conclusion. Since it’s well established that commonly used probiotic strains such as Lactobacillus infantis and Bifidobacterium breve can stimulate our immune system, tighten intestinal junctions, and carry out some of the same functions as a healthy, established gut microbiota, it’s no surprise that studies show that probiotic supplements can improve markers of inflammation and alleviate symptoms of irritable bowel syndrome.

What we have to ask ourselves is: Are the probiotics inducing some permanent beneficial effects, or are they merely masking the symptoms of an unhealthy gut? May it be that they are actually doing more harm than good, in the sense that they impede the development of a normal, healthy gut microbiota (e.g., by producing bacteriocins)? The RCTs on probiotics don’t help much in terms of answering these questions. If we then go on to consider the issue from an evolutionary perspective, we’ll see that it’s very unnatural for a human being to ingest large numbers of just one or a couple of strains of microorganisms every day.

This is not to say that probiotic supplements are never useful. All I’m trying to say is that simply looking at the results and conclusions of the RCTs and meta-analyses in this area doesn’t get us very far. To really be able to make some headway we also have to bring in other pieces of evidence, and perhaps more importantly, we have to do some logical thinking.

Okay, let’s do one more before we wrap up; this time on low-carbohydrate diets. Many bloggers and diet authors make the case that low-carbohydrate diets are healthier than high-carbohydrate diets, particularly for those who are overweight and metabolically deranged. To support this statement, they bring up RCTs and meta-analyses that seem to support this notion (4, 5, 6).

If we just look at the results and conclusions of these studies, we may be led to believe that everyone should restrict their carbohydrate intake. However, if we dig a little deeper, we’ll likely see that the picture is a lot more nuanced than what it looked like at first sight. There are many other factors besides the difference in carbohydrate intake between the study groups that could explain the difference in health outcomes, including differences related to the glycemic index and satiety index score of the food and the intake of protein and fiber.

Researches usually try to control for confounding variables, but perhaps needless to say, there’s a limit to how much you can control for. A diet that contains 15% carbohydrate by calories and one that contains 50% carbohydrate by calories will differ in several respects, regardless of how well you try to match the intake of non-carbohydrate nutrients.

Again, this is not to say that I don’t like low-carbohydrate diets. Actually, as you know if you’ve been following my work, I think most people will benefit from eating less carbohydrate than what government-supported guidelines recommend. All I’m saying is that we can’t just look at the conclusions of the studies in this area and jump to conclusions. We have to be smarter than that.

Last words

Some may get the impression from reading this article that everything in nutrition, health, and medicine is extremely complicated, and that studies and research papers within these fields have so many flaws and limitations that there’s not really any point in even looking at them. This is not what I’m trying to say. Actually, I think nutrition and medicine are less complicated than most people make it out to be. As long as we use the evolutionary template to guide our understanding of the scientific literature, then we’re usually on safe ground.

It’s somewhat comical that recent scientific findings about glycemic index, human protein needs, gene expression, Paleolithic diets, and many other topics within health, nutrition, and medicine are viewed as revolutionary, novel discoveries. This wouldn’t have been the case if all nutritionists, health practitioners, and scientists used a conceptual framework based on evolutionary biology to guide their understanding of diet and health. Most of the time, nutritional and medical research doesn’t provide us with new and novel insights about food and health; rather, it just adds scientific support to the conclusions the anthropological sciences and evolutionary record have provided us with.

The two key takeaways from today’s article are:

  • Nothing in science makes sense except in the light of evolution.
  • We have to consider the evidence as a whole before we jump to conclusions.

Comments

  1. Well said…I’d like to point out that we’ll never get far until we go on with talking about carbs, proteins and fats. I’ll never be tired to say that it’ s about food. Carbs are not the peocessed stuff we are pushed to eat as we are forced to believe by the mainstream media driven by business. Food is much more and what people put in their mouth every day shouldn’t even be considered as “food”…the Kitavans carbs are far different from pasta and pizza

Leave a Reply

Follow

Get every new post delivered to your Inbox

Join other followers: