You may have seen the article in Huffington Post by notorious Atkins-basher, Dr. Dean Ornish. He opens his irrational diatribe with this paragraph:

“A major study was just published in the Annals of Internal Medicine from Harvard. In approximately 85,000 women who were followed for 26 years and 45,000 men who were followed for 20 years, researchers found that all-cause mortality rates were increased in both men and women who were eating a low-carbohydrate Atkins diet based on animal protein.”

At the same site, another low-fat acolyte, Dr David Katz, is eager to pile on with the headline, “The Beef with Atkins”. He gives us a clue as to whether he has any preconceptions with this statement: “Do I think eating a high-meat, low-plant diet increases risk of death and disease? Hell ya!” and goes on to try to explain away the unexplainable.

From all this you might conclude that someone has done a study with a lot of people doing Atkins for a lot of years and that there is now proof that, compared to a control diet, Atkins led to higher death rates. If that’s what you thought, and that’s clearly what Ornish and others of his ilk, would like you to think, you would be completely wrong.

Where to begin. I suppose the first point to make here is that nobody in this study was doing Atkins. The lowest level of carbohydrate consumption reported was in the range of 180 g per day. That doesn’t resemble any version of Atkins that I am aware of.

Maybe, even more importantly, is the nature of the study itself. It is an observational study which means that no intervention was tested. What the researchers did was enlist a large number of people, in this case, nurses and other health professionals, and surveyed them with questionnaires over a period of years. Food recall questionnaires are highly unreliable (to get a sense of just how unreliable they are, have a look at this commentary by Chris Masterjohn – http://blog.cholesterol-and-health.com/2010/09/new-study-shows-that-lying-about-your.html). The reported results appear to be based on data gathered in 1986 and then extrapolated. A single questionnaire is used to try to determine what people ate over the previous year. How accurately can you tell me what you ate over the last month? Last week? You can see why this might be a tad unreliable. Still, I think you will be surprised just how wildly unreliable this kind of data is if you visit Chris Masterjohn’s site. Yet, it is the standard for these kinds of studies.

In this study, the food recall questionnaires considered “plausible” reported caloric intakes ranging from 500 to 3500 kcal per day. How many people get by on 500 kcal per day, do you think? Not too many. So, in the analysis, median or average numbers were chosen which still look, in all groups, to be significantly lower than what any American presently eats. The men reportedly ate about 2000 kcal per day while the women ate between 1600 and 1800. That should make for a pretty slim bunch of typical Americans! Interestingly, these absurd numbers for caloric intake are averaged while the analysis is done to determine the amounts of carbs, proteins, fats, etc in the various deciles. One wonders what would the data have looked like if they averaged the macronutrients and broke the caloric intake into deciles. The first decile would be 500 kcal per day, the next 800 kcal per day, the next 1100 kcal per day, etc etc. Do you see the problem? Of course it would be absurd to think that the first groups in the series could eat so few calories and actually live. That study would never get published. However, when these data are flipped around and the analysis is done the other way to determine macronutrient proportions by decile the study gets published. In what universe does that make any sense?

The next thing you need to know is that observational studies do not tell you anything about what causes what. They are useful for generating hypotheses which can then be tested using intervention trials to see if a hypothesis based on the original observations is valid. The reason that relying on the original observations is fraught with hazard is because, as I said, you cannot draw conclusions about causality, but also because of the high likelihood there will be unaccounted for factors that  explain the observation rather than the variables you hypothesized were responsible.

In this study, they divided the entire study population into deciles based on their reported consumption of carbohydrates. The percentage of calories from carbs ranged from about 37% to about 60% (for comparison, I eat about 5%). They then arranged two other decile groupings based on another comparison which looked at their consumption of animal sourced fats and proteins vs. vegetable sources. They found that the lowest carb group that ate the highest amount from animal sources had 23% higher all-cause mortality compared to those who ate the fewest calories from animal sources. Now you can see why the vegan zealots are very pumped about this and why their leaders are using every opportunity to hitch their anti-Atkins message to these findings, in spite of the fact that the study had nothing whatsoever to do with the Atkins diet or a vegan diet, for that matter.

So, 23% greater chance of dying – that sounds serious. Is it really? Let’s look a bit further. Apart from the problems inherent in determining causality, another issue that arises from the use of observational data of this sort is the size of the observed hazard ratio. Given all the potential problems with this kind of data, one wants to see a hazard ratio which is either below 0.5 or above 2.0. The hazard ratio here was 1.23, which falls significantly short of the mark.

The researchers also did a sensitivity analysis to determine how large a confounding variable they would have to have overlooked in order to void these findings and came up with 20%. In other words, a factor that could affect mortality by 20% would have to have been missed in arriving at these results. That sounds pretty big, right? How could they miss something so big?

Well, consider this. In an earlier study using questionnaires from this same group, it was determined that the use of hormone replacement therapy (HRT) conferred significant protection from cardiovascular disease. The hazard ratio was 0.39, well under the 0.5 point at which you should normally take notice. That sounds pretty good. Where can I sign up for some HRT?

Not so fast! A subsequent study which tested this observation using a large randomized control trial actually found that HRT increased the risk of CVD, by a factor of 1.29 (in a randomized controlled trial, a hazard ratio of 1.29 is considered to be significant). The unaccounted for factors in this case were five times as great as the 20% needed to void the carbohydrate mortality link in the current study. Five times as big and it was still overlooked! This is precisely why one should not draw conclusions from observational data until it is tested through properly designed intervention trials.

Okay, I will grant you that it’s not always possible to do the kinds trials needed to test every observation and that, by default, we may need to rely on the evidence we have in hand from observational studies. If that is the case, then we need to work within some parameters which will improve our chances of being right. In the case of smoking, for instance, the hazard ratios for harm when smokers are compared to non-smokers are north of 10. Remember that you want to see ratios above 2.0 before you get too concerned about observational data. Remember, also that the ratio reported here was 1.23.

Secondly, you want to see other things in the data. For instance, a dose-response effect would be supportive and a lack of it would suggest problems in the data. Dose-response means that as you increase the factor that you think is causing mortality, you want to see the mortality rates rise alongside those increases. In this study, there was no dose-response effect. Although the published study does not provide information for each decile, we can see what happened at the 1st, 5th and 10th decile. The high-vegetable group was actually eating more animal protein and fat and more saturated fat than the high-animal group at the 1st and 5th deciles. Only at the 10th decile are they eating less and, even then, not hugely less (30% vs. 45%).

Okay, while you are trying to get your head around that, consider that the reported hazard ratios for all-cause mortality at the 5th decile still favoured the high-vegetable group, the ones who are eating the most animal protein and fat at this point! Such an inconsistency in the data surely suggests there are other significant factors at work here. How this got by the reviewers and into print is a mystery to me.

What about the high-vegetable low-carbers? First of all, they are hardly low-fat vegans. They were eating about 30% of their calories from animal fat and protein. And, when you look at this group’s all-cause mortality rates, they have a hazard ratio of 0.80, which means 20% less mortality than people eating less animal fat and protein! How does that support the vegan or low-meat or low-fat agendas? It doesn’t. For the men, those who ate 38% of their calories as fat vs those who ate 24% had a 19% reduction in mortality. Let me emphasize that – the people in the high-vegetable group who ate more fat, more animal fat and more animal protein had a lower mortality rate. Why didn’t Ornish’s headline say: “Eating More Meat and Fat Reduces Your Risk of Dying”? This would be just as valid, in fact, more valid than the ridiculous headline suggesting that the diet had anything to do with Atkins and mortality.  This part of the study findings is not mentioned in either of the Huffington Post pieces or in any other mainstream media coverage that I have seen. You have to ask yourself “Why?”.

I could go on but there are other sites you can visit to get more informed criticism. I recommend Chris Masterjohn’s site mentioned above. Also, Denise Menger, a new voice of reason in the diet debates, can be found at http://rawfoodsos.com/ where her deft evisceration of “The China Study” is also worth a read. Another fairly recent and very funny commentator is Tom Naughton at his blog http://www.fathead-movie.com/. I recommend his movie, “Fat Head”, as well. Also, have a look at the usual reliable sources  such as Jimmy Moore’s www.livinlavidalowcarb.com and the Eades at www.proteinpower.com (although I haven’t seen a comment from Mike yet).

So, what can we learn from this? The study is so deeply flawed, it is hard to figure out whether there is anything here of importance. As I said at the beginning, all this type of study can do is generate hypotheses that need further testing. One intriguing hypothesis, of course, is that a higher fat, higher animal protein, lower carb diet high in vegetables seems to confer a better all-cause mortality rate. That would be the one I would like to see pursued. Funny how that sounds eerily similar to the diet recommended in the most recent Atkins book, too. Ornish and Co. are, of course, free to pursue their agenda, too, and I wish them luck. However, it cannot be a happy time for this crowd. As more evidence piles up demonstrating the benefits of low-carb over low-fat, they do seem to be getting more desperate.

I think what we are actually witnessing here is a paradigm in its death throes. This would explain the stunning perfidity of its proponents who appear willing to pull out all the stops in fighting their rear-guard action to save a belief system that is rapidly devolving into shambles. No Geneva convention here, folks. Flat out lies and misrepresentions are justified when defending the faith. And a lazy and, and possibly corrupt, media is all that is needed to keep the lies coming. Stay tuned. I am sure we will see more of the same.

ps – edited to correct error picked up by commenter