Post hoc ergo propter hoc

The Greeks and Romans recognized this fallacy in argument thousands of years ago. Because one event follows another event does not mean that the first event caused the second (post hoc ergo propter hoc). However, in contemporary medical news this logic is not understood. This faulty reasoning is the most common cause of false and misleading conclusions of research results that are presented as medical news.

This point was forcibly made to me in 1981 when I was a Senior Editor at JAMA, and I received a phone call from a friend of mine, Alvan Feinstein, a professor at Yale University. He discussed with me a publication in the New England Journal of Medicine by Brian MacMahon, a professor at Harvard University. The case-control study that MacMahon and his colleagues had done indicated that coffee drinking was associated with cancer of the pancreas (1). This conclusion had caused quite a stir both in the lay public and medical media. Commentators were going so far as to advocate that people should stop drinking coffee because of this risk.

Feinstein told me that he and his colleagues at Yale had analyzed the MacMahon study and found a number of biases that invalidated its conclusion. He criticized the use of a case-control study in studying such a prevalent practice as coffee drinking and its link to a specific cancer. He believed that the control population, even if some characteristics were well matched, was unlikely to be completely comparable in view of the dramatic differences among individuals. He believed that a more rigorous study design such as a randomized controlled trial or a cohort study would have been preferable. He also concluded that such a major conclusion as the causation of pancreatic cancer by coffee drinking should not have been brought into such public prominence based on a single, imperfect study. And the lack of a compelling scientific mechanism by which coffee could be oncogenic weakened such epidemiologic evidence.

I suggested that he write up his findings concerning possible biases in the MacMahon study and send them to me at JAMA for peer review and possible publication, which is what he did. Peer review and an analysis of the paper by an expert in statistics were favorable, and it was published in JAMA (2). There was some wrangling between the two groups of epidemiologists, but when another study of the possible coffee/pancreatic cancer association was published in the New England Journal, and no such link was found, most observers agreed that the findings of the MacMahon study were not valid. Unfortunately, there was no public coverage of the debate and its resolution, so most people continued to think that coffee drinking could cause pancreatic cancer. However, a survey indicated that in spite of the prevalent misconception, coffee consumption was not affected.

My ICC colleagues recently advised me that COPD patients are confused and disturbed by the many stories they see in lay media concerning the endless series of common behaviors that are reported to cause cancer or other adverse outcomes. In most cases these are case-control studies looking at multiple factors and finding small associations (2-fold or less increased risk), and these single studies are not supported or validated by additional investigations. Reporting such frightening results attracts an audience for media in the public sector, but these stories are almost never scientifically credible, and it does a disservice to patients. It has come to the point where I recommend that physicians tell their patients not to believe frightening medical news reports unless they can be verified. Ask the patients to identify the source of the report and then help them to research the study the report was based on to identify any errors or limitations so the patients can be correctly informed.

I recently saw an example of a questionable association reported in public and medical media that did come from a credible source: the University of Texas MD Anderson Cancer Center. They found that a diet with high glycemic index was associated with developing lung cancer, which is a very real concern to COPD patients (3). It was not surprising that many of them filled the COPD help websites with questions. Should they eliminate foods with a high glycemic index from their diets? Reviewing the details of the case-control study is instructive in how questionable information can become a major national source of medical misinformation.

The lead newscast stories and the front page headlines about the glycemic index study around the country said, “Carbs cause lung cancer”. Mehmet Oz, a physician who has often been criticized by medical experts and organizations for presenting false and misleading medical information on his television “health program” announced that “Carbs are like cigarettes. They cause lung cancer”.

This surprising information was taken (incorrectly) from the case-control study comparing lung cancer patients with healthy controls. The University of Texas researchers actually found no significant difference between lung cancer patients and controls concerning their dietary glycemic index—a measurement of how rapidly the carbohydrates they eat elevate blood glucose levels. However, the vast majority of lung cancer patients in the study were smokers—the most powerful risk factor for lung cancer—and although there was no significant difference between their diets’ glycemic index and those of the controls, the subgroup of non-smokers with lung cancer—10% of the total—were found to have about a 2-fold increase of dietary glycemic index over controls. The higher dietary glycemic index was also associated with those non-smoking patients having less than 12 years of education—a measure of educational and socio-economic status. In addition, the non-smokers had an increased likelihood of having a less common form of lung cancer—squamous cell carcinoma—than the smokers.

The study was a retrospective case-control study, which is notoriously subject to errors of recall of past dietary intake. Other limitations of the study were that it only included non-Hispanic white subjects, and it did not account for differences in factors such as diabetes, hypertension, and heart disease between study patients and controls. These limitations could certainly have biased the results.

The lung cancer association was just a single finding in a large series of metabolic comparisons and since this is the first suggestion that glycemic index and lung cancer are associated, it seems inappropriate that these compromised data should be presented as authoritative and actionable. Nevertheless, the authors suggested that on the basis of these findings people should avoid higher glycemic index foods such as bagels and white bread and eat lower glycemic index foods such as pumpernickel bread and pasta instead. Because of the participants in the study, these suggestions would only apply to the non-smokers who were non-Hispanic white patients, assuming that the study can be repeated.

The extensive national publicity for these preliminary findings and the inappropriate medical prescription by Mehmet Oz of his dietary choices based on the lung cancer threat were unfortunate events for medicine and for patients. We need to remember Hippocrates’ perspective on physicians drawing conclusions: “…experience is perilous and decision difficult.” The next time you see a study such as “Blueberries may boost memory in mild cognitive impairment (4)” you will know what questions to ask and how to help your patients find the truth.