The Fallacy of a Single Diagnosis

Abstract

Background

Diagnostic reasoning requires clinicians to think through complex uncertainties. We tested the possibility of a bias toward an available single diagnosis in uncertain cases.

Design

We developed 5 different surveys providing a succinct description of a hypothetical individual patient scenaric. Each scenario was formulated in 2 versions randomized to participants, with the versions differing only in whether an alternative diagnosis was present or absent. The 5 scenarios were designed as separate tests of robustness using diverse cases, including a cautious scenario, a risky scenario, a sophisticated scenario, a validation scenario, and a comparative scenario (each survey containing only 1 version of 1 scenario). Participants included community members (n = 1104) and health care professionals (n = 200) who judged the chances of COVID infection in an individual patient.

Results

The first scenario described a cautious patient and found a 47% reduction in the estimated odds of COVID when a flu diagnosis was present compared with absent (odds ratio = 0.53, 95% confidence interval 0.30 to 0.94, P = 0.003). The second scenario described a less cautious patient and found a 70% reduction in the estimated odds of COVID in the presence of a flu diagnosis (odds ratio = 0.30, 95% confidence interval 0.13 to 0.70, P < 0.001). The third was a more sophisticated scenario presented to medical professionals and found a 73% reduction in the estimated odds of COVID in the presence of a mononucleosis diagnosis (odds ratio = 0.27, 95% confidence interval 0.10 to 0.75, P < 0.001). Two further scenarios—avoiding mention of population norms—replicated the results.

Limitations

Brief hypothetical scenarios may overestimate the extent of bias in more complicated medical situations.

Conclusions

These results demonstrate that an available simple diagnosis can lead individuals toward premature closure and a failure to fully consider additional severe diseases.

Highlights

• Occum’s razor has been debated for centuries yet rarely subjected to experimental testing for evidence-based medicine.

• This article offers direct evidence that people favor an available simple diagnosis, thereby neglecting to consider additional serious diseases.

• The bias can lead individuals to mistakenly lower their judged likelihood of COVID or another disease when an alternate diagnosis is present.

• This misconception over the laws of probability appears in judgments by community members and by health care workers.

• The pitfall in reasoning extends to high-risk cases and is not easily attributed to information, incentives, or random chance.

Introduction

The principle of diagnostic parsimony has endured for centuries and is popularized by the term Occam’s razor. The general concept is that simple explanations ought to be preferred over more convoluted theories. ¹ Incorporating Occam’s razor into modern medicine, however, has led to dissenting perspectives. ² Hickam’s dictum is a counterargument that asserts a patient, especially in an era in which patients are older and increasingly frail, can have multiple diagnoses occurring together. ³ Such conflicting perspectives around diagnostic simplicity and complexity abound in daily medical practice and will not end soon. ⁴

The severe acute respiratory syndrome coronavirus 2 pandemic starting 2019 (termed COVID) provides a new opportunity to examine medical judgment under uncertainty because symptoms are ambiguous, test results can take time, a definitive diagnosis is eventually established, and outcomes can be serious. Moreover, the presence of one virus does not exclude the possibility that a second infection is present simultaneously. ⁵ Epidemiology suggests, furthermore, that co-infection from different pathogens is frequent because of common risk factors such as crowded living spaces, lapses in hand hygiene, social interactions, insufficient distancing, and other health determinants. ⁶

Undue deference to Occam’s razor can thus lead to oversimplification, premature diagnostic closure, and potentially faulty patient care. ⁷ Direct tests of Occam’s razor in judgment under uncertainty, however, rarely appear in evidence-based medicine (beyond editorial commentary and case reports).^8,9 Here, we tested whether judgments about COVID infection show a bias toward diagnostic parsimony. Specifically, we hypothesized that an available alternative explanation might lead individuals to underestimate the likelihood of a COVID diagnosis. A randomized test of this fallacy may help motivate greater awareness and more effective medical care.

Methods

Overview

We developed survey materials using methods adapted from behavioral decision science. This involved creating different scenarios that each provided a succinct description of an individual patient who had symptoms suggestive of COVID. When appropriate, each scenario also provided the population baseline norm (average risk) of COVID relevant to the setting. The purpose of each scenario was to elicit judgments about the relative likelihood of COVID in the hypothetical individual patient. The question wording was, “What are the chances that [the patient] has COVID?” The 5 response options were “definitely below the norm,” “somewhat below the norm,” “about the same as the norm,” “somewhat above the norm,” and “definitely above the norm.”

Each scenario was formulated in 2 different versions, defined the absent version and the present version. The absent version described an alternative infection (influenza, mononucleosis, strep throat) that had been considered, tested, and excluded (the patient did not have the alternative infection). The present version described the same alternative infection that had been considered, tested, and confirmed (the patient did have the alternative infection). Given the information about the absence or presence of the alternative infection, participants then judged the relative likelihood of COVID infection in the patient. In all other respects, the 2 versions were identical, involved only a single question, and were randomly assigned to participants.

Five other survey features were designed to minimize bias. First, each scenario was conceived based on direct clinical experience to make materials original, succinct, clear, meaningful, and relevant. Second, each survey was conducted during active waves of the COVID pandemic to check robustness, provide realistic population norms, and respect community priorities. Third, the scenarios described an individual who was young and had no comorbidities to reduce the need for medical expertise and minimize complexity. Fourth, each respondent was assigned only 1 version of 1 scenario to avoid carryover artifacts and survey fatigue. Fifth, no identifying demographic information on respondents was collected to maintain confidentiality and reduce respondent workload.

Participants

We surveyed different participant groups primarily from North America. The largest group was community participants recruited by the Qualtrics Survey platform (https://www.qualtrics.com/lp/survey-platform/). Another group was health care workers surveyed in person during a mandatory 15-min observation interval immediately after receiving a COVID vaccination. The overall intent was to elicit judgments from potential patients as well as from health care workers who were experienced, motivated, informed, and engaged. Surveys for community participants were conducted in September 2020 (second wave of the pandemic) and April 2021 (after the third wave had subsided); surveys for health care workers occurred in February 2021 (initial vaccine rollout).

All participants completed the survey at their own pace (with a planned time to completion of less than 2 min) and returned responses anonymously (with no formal debriefing). Each participant was blinded to the hypothesis, unaware of alternative versions, and given only a single scenario. Community participants were compensated at the standard internet platform rate ($15 per 1h, equal to $1 per 4 min), whereas health care workers received no incentives. The introductory script for healthcare workers was, “I am hoping you might complete a short survey. It is voluntary, intended for research, and needs no identifying data. Your involvement, results, or lack of participation will not affect your status here at Sunnybrook.

The study was approved by the Research Ethics Board of Princeton University and by the Sunnybrook Research Institute (file No. 3798). The approval included a waiver for signed consent, with consent expressed by checking the appropriate box on the survey platform or by willingness to complete the short paper survey. The community participants received the survey online, and the health care group received the survey on paper in person. Patients and the public were not involved in study design or reporting. At the time of the surveys, PCR testing was the primary method for diagnosing COVID in the community, with a typical 1- to 2 day delay for reporting. Community respondent demographics are summarized in the Appendix.

Statistical Analysis

The primary analysis compared responses in the 2 versions of each scenario, expressed as the proportion of respondents who judged the patient’s chance of COVID infection as below the norm. Statistical analyses used the chi-square test and the odds ratio as a consistent measure of effect size (analyses using the Mann-Whitney test to account for the full nonparametric distribution yielded similar results and are not reported). Sample size calculations were designed to provide an 80% power (beta = 0.20) for detecting an effect size of at least 15% (baseline 25% reduced to 10%). The analytic plan was specified in advance, and each scenario was considered an independent test. All p values were 2 tailed, and the same statistical tests were used for all scenarios.

Results

Cautious Scenario

The cautious scenario involved a man who followed public health COVID guidelines and maintained cautious behavior. The absent and present versions were identical except that the bold text (absent version) was replaced by the bold text in brackets (present version). The wording was as follows:

Mr. FR is a 25-year-old man who develops a runny nose, slight cough, and general fatigue. He has no thermometer and does not know his temperature. He has been very careful isolating, not socializing, and always with a mask. Still, he is worried about COVID and goes to be tested (results in 1–2 days). At his location, the norm is 3% of COVID tests are positive. FR is also tested for influenza; those results come back fast, and those results are negative (he does not have the flu) [positive (he does have the flu)]. What do you think the chances are FR has COVID?

A total of 199 individuals responded, of whom 101 and 98 received the absent and present versions, respectively. Consistent with the cautious behavior, many respondents judged the chances of COVID below the norm in this scenario. As hypothesized, however, more participants judged the chances of COVID below the norm in the present version, when the flu test was positive, than in the absent version, when the patient tested negative for the flu (49% v. 34%, P = 0.003). This discrepancy equaled a 47% reduction in the estimated odds of COVID in the presence of a flu diagnosis (odds ratio = 0.53, 95% confidence interval 0.30 to 0.94).

Risky Scenario

The risky scenario involved a man who failed to follow standard guidelines and exhibited risky behaviors. The absent and present versions were identical except that the bold text (absent version) was replaced by the bold text in brackets (present version):

Mr TR is a 25-year-old man who develops a runny nose, slight cough, and general fatigue. He has no thermometer and does not know his temperature. He has been socializing a lot recently, often indoor, not sufficiently distancing, and without a mask. He is worried about COVID and goes to be tested (results in 1–2 days). At his location, the norm is 3% of COVID tests are positive. TR is also tested for influenza, those results come back fast, and those results are negative (he does not have the flu) [positive (he does have the flu)]. What do you think the chances are TR has COVID?

A total of 207 individuals responded, of whom 104 and 103 received the absent and present versions, respectively. Consistent with the risky behaviors, relatively few respondents judged the chances of COVID as below the norm in this scenario. Importantly, more respondents judged the chances of COVID below the norm in the present version, when the patient tested positive for the flu, than in the absent version, when the flu test was negative (22% v. 8%, P < 0.001). This discrepancy equaled a 70% reduction in the estimated odds of COVID in the presence of a flu diagnosis (odds ratio = 0.30, 95% confidence interval 0.13 to 0.70).

Sophisticated Scenario

The sophisticated scenario was presented to health care workers and involved a woman who faced occupational risks. The absent and present versions were identical except that the bold text (absent version) was replaced by the bold text in brackets (present version):

SR is a 27-year-old ICU nurse who has not received a COVID vaccination yet. She now has a fever, fatigue, and sore throat. SR is tested for COVID and will receive results in 1–2 days. At her location, an average of 10% of COVID tests are positive. SR is also tested for mononucleosis, those results are fast, and those results are negative (she does not have mononucleosis) [positive (she does have mononucleosis)]. What are the chances SR has COVID?

A total of 200 individuals responded, of whom 100 each received the absent and present versions. Consistent with the occupational risks described, few respondents judged the chances of COVID below average in this scenario. That notwithstanding, more respondents judged the chances of COVID below average in the present version than in the absent version (17% v. 5%, P < 0.001). This discrepancy equaled a 73% reduction in the estimated odds of COVID in the presence of an alternative diagnosis (odds ratio = 0.27, 95% confidence interval 0.10 to 0.75).

Validation Scenario

A validation scenario provided nearly no descriptive features and directly tested the impact of the availability of alternative explanations on judgments. The absent and present versions were identical except that the bold text (absent version) was replaced by the bold text in brackets (present version):

Mr FR is a 25-year-old man who develops a runny nose, slight cough, and general fatigue. He has no thermometer and does not know his temperature. He is worried about COVID and goes to be tested (results in 1–2 days). FR is also tested for influenza, those results come back fast, and those results are negative (he does not have the flu) [positive (he does have the flu)]. In your view, do the flu results make it more or less likely that FR has COVID?

A total of 202 community participants responded, of whom 102 received the absent version and 100 received the present version. With a paucity of background information, judgments were largely influenced by the results of the flu test. Specifically, a minority of respondents judged a higher likelihood of COVID in the present version when the patient was confirmed to have the flu, whereas a majority judged a higher likelihood in the absent version when the flu test was negative (33% v. 84%, P < 0.001). This discrepancy was equal to a 90% reduction in the estimated odds of COVID when another diagnosis was present (odds ratio = 0.10, 95% confidence interval 0.05 to 0.19).

Comparative Scenario

A comparative scenario avoided any mention of population norms and simply compared between 2 contrasting individuals following an uncertain exposure. The absent and present versions were identical except that the bold text (absent version) was replaced by the bold text in brackets (present version):

MB and TA are close friends, 25 years old, who have just returned from a crowded outdoor concert. They are not vaccinated, they hear of others at the concert who tested positive for COVID, and they feel worried. MB has a mild headache, slight cough, and general fatigue. He goes to be tested (results pending), and in the meantime he is also tested for strep throat. Those results come back fast, and those results are negative (MB does not have strep throat) [positive (MB does have strep throat)]. TA is feeling tired and has not been tested. In your view, who is more likely to have COVID?

A total of 496 community participants responded, of whom 249 received the absent version and 247 received the present version. Given the symptoms, most respondents agreed the chances of COVID were greater for MB compared to TA. As anticipated, however, fewer participants judged the chances of COVID greater for MB in the present version, when he was diagnosed with strep throat, than in the absent version when the strep test was negative (51% v. 86%, P < 0.001). Despite substantial consensus that MB was more likely than TA to have COVID when he tested negative for strep, half the participants judged him less likely than TA to have COVID once diagnosed with strep. This discrepancy was equal to an 83% reduction in the estimated odds of COVID when another diagnosis was present (odds ratio = 0.17, 95% confidence interval 0.11 to 0.26).

Discussion

Overview

Through scenarios involving a spectrum of risk, we found that respondents judged the probability of COVID infection to be lower in the presence of another diagnosis compared with when an alternative diagnosis was absent. These judgments are problematic because a simple diagnosis (e.g., influenza) provides no protection against COVID infection. Co-occurrence of influenza and COVID is less likely when behaviors are cautious (scenario 1) than risky (scenario 2), but the chance of COVID does not decrease in the presence of another disease unless the two are negatively correlated. ¹⁰ Instead, our respondents consistently lowered their judged likelihood of COVID infection when an alternate diagnosis was present (Table 1). This bias might reduce a patient’s willingness to seek care and a clinician’s propensity to investigate a medical diagnosis.

Table 1.

Summary of Main Results ^a

Scenario	Chance of COVID below the Norm		Odds Ratio b (95% Confidence Interval)
	Alternative Diagnosis Present	Alternative Diagnosis Absent
Cautious	49% (48 / 98)	34% (34 / 101)	0.53 (0.30 to 0.94)
Risky	22% (23 / 103)	8% (8 / 104)	0.30 (0.13 to 0.70)
Sophisticated c	17% (17 / 100)	5% (5 / 100)	0.27 (0.10 to 0.75)
	Chance of COVID Relatively Lower		Odds Ratio a (95% Confidence Interval)
	Alternative Diagnosis Present	Alternative Diagnosis Absent
Validation	84% (84 / 100)	33% (34 / 102)	0.10 (0.05 to 0.19)
Comparative	49% (122 / 247)	14% (35 / 249)	0.17 (0.11 to 0.26)

^aDetailed distribution of responses shown in appendix.

^bBased on dichotomized distribution of responses.

^cScenario assigned to health care workers.

Selective Justifications

Diagnostic parsimony is sometimes appropriate. Some anatomical disorders are mutually exclusive; for example, a diagnosis of micrognathia tends to exclude a diagnosis of macrognathia. Similarly, many metabolic disturbances cannot coexist; for example, a diagnosis of Addison’s disease excludes a diagnosis of Cushing’s syndrome. Other problems are confined to different populations; for example, a diagnosis of uterine cancer is unlikely to occur with a diagnosis of prostate cancer. And some issues are segregated across time; for example, a diagnosis of ectopic pregnancy is unlikely to occur with a diagnosis of osteoporosis. In many other cases, Occam’s razor will appear correct simply due to chance, because of the low base-rate frequency of most individual diseases. ¹¹

Reliance on Occam’s razor is sometimes expedient. Heuristic thinking early in an epidemic may be justified by the lack of hard data for directly estimating disease probabilities. Reliance on simplicity is also encouraged by binary diagnostic algorithms and by billing systems that ask for a single diagnosis for a clinical encounter. Furthermore, exhaustive diagnostic testing can potentially yield adverse patient outcomes from iatrogenic harms, false positives, and wasted time. ¹² A plurality of diagnoses may also be hard to defend when a patient has minimal symptoms, since double-trouble should otherwise lead to worse severity. Medical care might also prove overwhelming if clinicians considered all the complexity occurring each day. ¹³

Fundamental Misconception

Many diseases, however, cluster together (e.g., hypertension and diabetes). A diagnosis of one of these diseases is no time to appeal to parsimony and neglect combined comorbidities. In the initial years of the pandemic (when COVID tests were in short supply), guidelines mistakenly suggested the presence of the flu reduced the likelihood of a COVID diagnosis and prioritized testing only when influenza was absent. ¹⁴ Subsequent microbiology evidence contradicted this assumption and found similar rates of SARS-CoV-2 infection in patients with and without other pathogens. ¹⁵ Of course, the converse is also relevant: the presence of a COVID diagnosis does not guarantee the absence of another serious disease (e.g., tuberculosis). ¹⁵

Our study highlights a misconception about the laws of probability. The logic of Venn diagrams implies that an intersection set tends to be smaller than a single set; however, the arithmetic of conditional probability is more complex (Figure 1). As an analogy, consider that a patient’s probability of having a fever exceeds the probability of fever and fatigue; however, a patient’s probability of fever with fatigue can exceed the probability of fever without fatigue. Similarly, a patient’s risk of co-infection is necessarily less than the risk of having 1 infection; however, the risk of co-infection may easily exceed the risk of having 1 infection exclusively. Fallible intuitions often reflect a failure to think of the inverse conditional probability.^16–18

Figure 1.

Venn diagram logic. Venn diagram of potential infections in patients with respiratory symptoms. The black rectangle depicts the full span of diagnostic possibilities. The left circle depicts patients with COVID infection, the right circle depicts patients with other pathogens, the middle sector depicts patients with coinfections, and the surrounding space depicts patients with no infections. In this example, the area of the gray middle sector exceeds the area of the black crescent, thereby suggesting the likelihood of co-infection exceeds the likelihood of having COVID exclusively.

Microbiology surveillance studies frequently document co-infections in COVID patients. Some case series show that more than 50% of patients with COVID have co-infections.^19–22 Case-control analyses show variable trends and almost none with a significant inverse association between other pathogens and COVID risk.^23–28 One study of symptomatic patients (N = 1207), for example, found lower COVID risk when another infection was present (23/315 = 7%) than when another infection was absent (92/892 = 10%); however, this inverse association was small (odds ratio = 0.70), attributable to chance (95% confidence interval: 0.43 to 1.12), and inconsistent with the magnitude of discrepancies observed in our study (median odds ratio = 0.27). ²⁹

Limitations and Extensions

Our study has limitations that merit mention. We tested different groups at different times, yet more replication is justified. Future research could also test whether the effect varies in different individuals. We examined diagnostic judgments in simple scenarios, whereas daily care is more complicated and challenging. We examined brief scenarios that allowed time for uninterrupted thought, whereas clinical decisions often involve distractions, uncertainties, and conflicting priorities. We examined one viral infection, whereas a bias toward simple diagnoses might extend more broadly. To be sure, our design renders the results difficult to attribute to a lack of education, sloppy responses, strategic ploys, fallible test results, atypical respondent samples, incentives, or chance.

Other pitfalls in diagnostic reasoning can reinforce an unduly parsimonious explanation. People tend to discount alternative explanations once a compelling account is available. ³⁰ In addition, a focusing illusion can arise wherein a particular explanation or a popular misconception appears overly compelling due to repeated consideration. ³¹ A related pitfall occurs when information originally pursued and then confirmed receives more attention than information available from the start. ³² Teamwork can also lead to groupthink, including a tendency to focus on points shared by all rather than unique insights known to a few. ³³ These and other factors may contribute to a failure to search for secondary diagnoses in patients who have notable acute or chronic diseases.^34,35

Summary

Our results suggest that an available simple diagnosis can lead to premature closure and a failure to fully consider additional serious diseases. Like other heuristics, this shortcut in reasoning is often effective but sometimes mistaken. The appeal to Occam’s razor is justified mostly when the alternative diagnosis is inversely correlated or relatively rare. ³⁶ In the case of COVID infection, overreliance on Occam’s razor can contribute to misplaced complacency, discourage the search for alternatives, and reduce attention to preventive health strategies. ³⁷ The tension between diagnostic parsimony and precision is not easy to resolve, will continue beyond the COVID pandemic, and is worth keeping in mind when diagnosing patients during the next disease outbreak.