Recently, the Globe and Mail online edition published an article with this headline:

“Just cut calories – protein/carb/fat ratio doesn’t matter: study”.

The article led with this:

“There’s no shortage of diet plans promising to melt away the pounds by calling for strict proportions of protein, carbohydrate and fat.

But, according to a new study, it doesn’t matter where the calories come from. What matters most for shedding body fat is simply eating fewer calories – and sticking to your plan, be it high protein, low carb or low fat.”

Okay, that gets my attention and you can understand why. It flies in the face of everything I have learned about low-carb dieting since my epiphany over nine years ago. So, what was this new study and how did it arrive at this remarkable conclusion? The article goes on to say:

“Some, but not all, studies have demonstrated that high protein, low carbohydrate diets work better than others at losing fat and preserving muscle mass over the short term.”

Oops! The studies I am familiar with looked at high fat, low carbohydrate diets and yes, they are better at causing fat loss and preserving muscle mass. But, let’s let the author, Leslie Beck, the G&M nutritional writer, make her point:

“Participants were assigned to one of four diets: 1) low fat (20 per cent daily calories), average protein (15 per cent); 2) low fat (20 per cent), high protein (25 per cent); 3) high fat (40 per cent), average protein (15 per cent); or 4) high fat (40 per cent), high protein (25 per cent).”

Okay, right away we have a bit of a problem. The range of carbs goes from 65% to 35%, from fairly high to not terribly low in terms of the kind of diet others have studied and to which people like me adhere. On the other hand, if there were to be a benefit from carb restriction, one would expect to see at least some signs of it within this range. Let’s not be too deterred just yet. Leslie goes on:

“Each diet was low in saturated fat and cholesterol and high in fibre, included low glycemic carbohydrates and was designed to cut 750 calories a day. All participants were offered group and individual counselling sessions over two years.

Body fat and muscle mass was measured using CT (computed tomography) scanning after six months and two years of follow up.

At the six-month mark, participants had lost, on average, more than nine pounds of total body fat along with five pounds of lean muscle, but had regained some of this after two years. Fat loss or muscle loss did not differ between the four diet groups.”

Whoa! Look at that muscle loss. That’s not good. How in the heck did that happen?

“As well, the proportion of protein, carbohydrate or fat in the diet did not affect the amount of abdominal fat, visceral fat or liver fat that was lost during the study. After six months, participants shed about 40 per cent of visceral fat and 60 per cent of liver fat.

At the two-year follow-up, people were able to maintain a weight loss of more [than] nine pounds, including three pounds of abdominal fat.

The bottom line: The major factor for weight loss was adherence to a calorie-reduced diet, not the proportion of carbohydrate, protein or fat it contained. People who followed their diets better lost more weight and body fat than those who didn’t.”

Well. That’s pretty clear, isn’t it? How can you argue for any exceptional benefits for a carb-restricted diet in the face of such conclusive evidence to the contrary? All you need to do is reduce calories and, macronutrient proportions be damned, just stick to it and you will do as well as the next guy (or gal). Isn’t that what everyone has been saying all along? Now that it has been well and truly confirmed we can fold our low-carb tent and go home. This is clearly the position that Leslie takes in her article which, I am sure, has the defenders of the status quo dancing with glee everywhere.

So, why haven’t we seen the magic reduction in obesity rates one would expect with this prescription being so simple and clear and all?

Maybe we should have a closer look. The first thing we discover is that this “new” study is anything but. This is a recently published paper generated from data collected between October 2004 and December 2007. Those data are over four years old and the principle article from this study was published in the New England Journal of Medicine in February 2009, exactly three years ago. In that report we learned that the weight loss was the same regardless of the dietary macronutrient pattern. This more recent paper goes into the findings with respect to which fat depots were affected, ie liver, visceral or subcutaneous. Okay, fair enough, there is more to discuss than the simple weight loss and metabolic marker values of the first paper. This is done all the time. One dataset can lead to a number of publications over a period of years. The main point, though, is not new. It was published three years ago at which time Leslie did a similar article extolling the virtues of simple calorie restriction. Why the need to make the same point again now from the same dataset? Hmmm? Good question. Feeling a little defensive, are we? A little bit under siege, perhaps? Interestingly, although the recent Nature article condemning sugar, by Robert Lustig, got oodles of media coverage, not a mention was made of it in the G&M. I digress.

We still have the problem of the results themselves. How is it that fairly large variations in macronutrient content seem to have no effect on weight loss and other important factors? Let’s dig a little deeper, shall we?

Whenever a diet study is reported in the lay media, I always dig up the original article to see what actually happened. In this case, I did that back in 2009 and I did it again here. What I found is quite disturbing and can perhaps give some insight into why there is so much confusion about what might be the ideal dietary choice for weight loss.

The crux of the problem, as I see it, is the use of an “intention to treat” analysis. What this means is that the researchers design a trial where different subjects are assigned different treatments, in this case, different dietary patterns. During the course of the trial some subjects drop out for various reasons and others fail to comply with the study protocol to which they are assigned. That is not surprising, it happens all the time. The problem is that the prevailing wisdom, in terms of the best way to analyze a dataset that is messy because of these issues, is that everyone’s results at the end of the trial are analyzed as though they hadn’t dropped out and as though they had stuck to their assigned protocol. This kind of study with this kind of analysis is called an “effectiveness” trial.

There is a valid reason for doing this. If you want to find out how effective an intervention might be in the real world you have to factor in whether people will comply with it. You may discover the cure for cancer but it would turn out to be useless if nobody could comply with it. This is important information to know.

Problems with this approach arise, however, when efficaciousness conclusions are drawn from these types of effectiveness findings.  In the vernacular there is no difference between “efficaciousness” and “effectiveness”. However, in the scientific world there is a very important distinction. If I want to know if a drug or diet is efficacious, I need to apply it to somebody and observe what happens when they comply with taking the drug or following the diet. That is a very different approach than trying to figure out how well therapies might work at the population level. In a study of efficaciousness, one is measuring one thing: the biological effect of the intervention. In an effectiveness study, on the other hand, one is measuring the biological effect plus the mitigating factors of attrition and compliance. In an effectiveness study where the intervention is very potent but where compliance is poor, after an intention to treat analysis has been applied the results will indicate that the intervention was not particularly useful. This is exactly what happened with this diet trial.

In the most recent paper (but not so much in the earlier one, which is interesting in itself), the authors admit that by the 6 month point their subjects were not fully compliant with their prescribed diets. Here’s how they characterized that:

“Differences between macronutrient amounts at 6 mo were all in the directions predicted on the basis of macronutrient assignments, although target intakes were not fully achieved.”

In other words, people had only partly complied with the diets they had been prescribed. It gets worse at 24 months, however. By the end of the study the only difference between any of the groups was in the fat consumption between the highest and lowest assigned groups. And how big was that difference? About a teaspoon of fat per day!

Let me emphasize that. By the end of the study, everybody still in the study was eating exactly the same thing except for two groups whose only difference amounted to about as much fat as you would get from eating a single bite of cheddar cheese.

Hmmm. That’s disappointing, right? The researchers invested a lot of time, money and effort in conducting this study to find out whether different proportions of macronutrients would generate different results and their subjects didn’t comply with the diets they were prescribed. That’s a pretty big fail (as my eleven year old son would say). When things go so completely off the rails, how can one draw any meaningful conclusions? Common sense would suggest that you can’t. Ah, but we don’t rely on common sense when doing nutritional studies, you see. We rely on an intention to treat analysis. This allows the researchers to publish their results as though everyone did comply with their diets and draw conclusions about the merits of the prescribed diets even though nobody complied with those diets. Wow, that’s amazing, isn’t it? Talk about your Hail Mary Pass!

Of course, knowing this, why would you be surprised to learn that the results were the same for each of the diet groups? You wouldn’t. But, when the paper is published the authors still make claims about the efficaciousness of the different diets.

In the 2009 paper, the authors conclude:

“Reduced calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize.”

And in the more recent paper, the authors conclude:

“Participants lost more fat than lean mass after consumption of all diets, with no differences in changes in body composition, abdominal fat, or hepatic fat between assigned macronutrient amounts.”

Duh! They all ate the same thing. Why would you expect any differences? Oops, I let common sense intrude again, forgive me.

The authors of the 2009 study comment on compliance thusly:

“Despite the intensive behavioral counseling in our study, participants had difficulty achieving the goals for macronutrient intake of their assigned group.”

And:

“In addition, trials of low-carbohydrate diets have reported a very low incidence of urinary ketosis after 6 months suggesting it is futile to sustain a low intake of carbohydrates.”

As I have blogged earlier on the altercation I had with Dr Frank Sacks, lead author on this study, at the Metabolic Syndrome meeting in Barcelona in the spring of 2007, I won’t go into great detail here. Suffice to say that after he presented some data on the merits of high protein, I suggested it may actually have been reduced carb that generated the benefits. This provoked him to respond was an angry rant about how low-carb had no scientific merit whatsoever. Of course, he was still in the midst of running this study in which he purports to investigate the merits of carb-restricted diets. Am I the only one who senses a bias here that might have crept into the way the study was conducted? Naw, that couldn’t happen in the pristine world of nutritional research.

When I hear this nonsense about the inability of anyone to sustain a low-carb diet over the long haul, I think about how we approached smoking cessation. We knew that compliance rates with the various approaches to smoking cessation were poor. However, we also knew that it was the best approach for the prevention of tobacco-related illness and consequently, a great deal of effort was directed at addressing the compliance issues. Had we assumed the posture of current researchers who dismiss a low-carb diet as too difficult to sustain, we would never have achieved any success in diminishing the harm of tobacco. Fortunately, we have the data from case reports and from high-compliance studies by researchers like Jeff Volek and Eric Westman, which tell us that carbohydrate restriction is the best therapy for obesity, metabolic syndrome and type 2 diabetes.

We should really be focusing on the compliance issues at this point instead of constantly trying to defend against the smoke screen thrown up by biased researchers and their poorly done studies and a lazy media that is only too happy to report their misconstrued results as fact. A pox on all their houses!