The first casualty of an epidemic is evidence

Abstract

Background

The COVID‐19 has posed a wide range of urgent questions: about the disease, testing, immunity, treatments, and outcomes. Extreme situations, such as pandemics, call for exceptional measures. However, this threatens the production and application of evidence.

Methods

This article applies standard categories in epistemology to analyse the pandemic in terms of four kinds of uncertainty: Risk, Fundamental uncertainty, Ignorance, and Ambiguity.

Results

Mapping the uncertainties of the pandemic onto the four types of uncertainty directs evidence production towards specific tasks in order to address the challenges of the pandemic: Eliminating ambiguity, being alert to the unknown, and gathering data to estimate risks are crucial to preserve evidence and save lives.

Conclusion

In order to avoid fake facts and to provide sustainable solutions, we need to pay attention to the various kinds of uncertainty. Producing high‐quality evidence is the solution, not the problem.

BACKGROUND.

• The COVID‐19 has posed a wide range of urgent questions: about the disease, testing, immunity, treatments, and outcomes.

• Extreme situations, such as pandemics, call for exceptional measures.

• This threatens the production and application of evidence.

What this article adds

• This article directs evidence production towards four types of uncertainty in order to address the challenges of the pandemic.

• The four types of uncertainty are Risk, Fundamental uncertainty, Ignorance, and Ambiguity.

• Eliminating ambiguity, being alert to the unknown, and gathering data to estimate risks are crucial to preserve evidence and save lives.

We fight a ventured war against a virus, and the truth is claimed to be the first casualty of war. However, behind concealment, conspiracy theories, and lies ¹ lie uncertainty and lack of evidence. Hence, evidence is the first casualty of an epidemic.

We appear to be haunted by an invisible enemy, ² , ³ and despite months with careful monitoring, extended testing, experimental treatment, and 18 702 scientific articles in PubMed (June 4), uncertainty still prevails. Our urgent need for information makes us lower the bar for evidence and thereby increasing the chance of bias and bad decisions. ⁴ The serious situation has led to ethical exceptionalism, ⁵ for example, in terms of controlled human infection studies. Correspondingly, we are exposed to an epistemic exceptionalism. ⁴ For example, the extremely rapid and “opinion‐based” peer review ⁶ has resulted in a number of retractions of COVID‐19 articles. ⁷ Accordingly, we seem to be subject to an “epidemic of false claims and potentially harmful actions.” ⁶

In a situation with extensive uncertainty and an urgent need to act, understanding the character of uncertainty is key.

1. GETTING TO TERMS WITH UNCERTAINTY

The uncertainties of the COVID‐19 pandemic can be mapped onto four types: Risk, Fundamental uncertainty, Ignorance, and Ambiguity. With Risk we have known outcomes and we know their probability distributions. With Fundamental uncertainty we know the outcomes, but not the probability distribution. When being ignorant we know neither. Ambiguity arises when experts disagree over the framing of possible contexts, options, outcomes, benefits, or harms. ⁸

Table 1 gives an overview of some specific and crucial uncertainties in the COVID‐19 pandemic.

TABLE 1.

Four types of uncertainty classified according to outcomes and risks

Possibilities probability	Known outcome	Unknown outcome
Known probability	Risk Test accuracy (sensitivity, specificity, predictive values) 9 for the various tests in different contexts 10 Effects and side effects of new treatments Prevalence of disease	Ambiguity Unclear diagnostic criteria of COVID‐19 What are the appropriate tests? 10 What are the appropriate test procedures? How to verify tests? 11 How to define immunity?
Unknown probability	Fundamental (Knightian) Uncertainty Basic Reproduction Number (R) 12 Case Fatality Rate/Infection Fatality Rate Being infected Spreading the virus Treatment outcomes 13 Immunity Effectiveness of intensive care treatment 6	Ignorance Late‐stage consequences of COVID‐19 Treatment options 13 Mutation potential Obstacles to vaccine development and production

Note: Adapted from Reference 8.

Making decisions based on risks is not easy, but commonplace. The problem with COVID‐19 is that so many risks are unknown, as probability distributions are wanting. Moreover, decisions based on fundamental uncertainty tend to be speculative and potentially harmful. ⁶ Decisions based on ambiguity “are not just potentially misleading — they are a fundamental contradiction in terms.” ⁸ Therefore it is crucial to clarify definitions. Additionally, ignorance poses great challenges, as we do not know what we do not know—and hence where to search for solutions.

2. TAKING UNCERTAINTY SERIOUSLY

No doubt, scientists are working relentlessly to find answers to the many questions and solutions to the pandemic. But the desperate situation appears to excite desperate measures. ¹³ We are lead into what has been called a “once‐in‐a‐century evidence fiasco.” ¹⁴

In order to target our efforts to reduce uncertainty, we need to pay attention to the various kinds of uncertainty. Reducing risk ¹⁵ and fundamental uncertainty is in vain if ambiguity prevails. For example, increasing test accuracy (technically) has shown to be of little help when the sampling method or validation procedure is inappropriate. Correspondingly, great treatment efforts can be futile if we ignore important factors for preventing, diagnosing, or treating COVID‐19.

Hence, the tasks for scientists strongly depend on the kinds of uncertainty. Accordingly, our tasks are fourfold and mapped in Table 2.

TABLE 2.

Tasks for scientists corresponding to the four types of uncertainty

Possibilities probability	Known outcome	Unknown outcome
Known probability	Risk With known risk, the tasks are to: (a) reduce the negative consequences of specific outcomes and (b) to reduce their probability, for example, by vaccines, better diagnostics, prognostics, and treatment	Ambiguity Define diagnostic criteria of COVID‐19 Define appropriate tests (type) Define test procedures Define test verification Define immunity (in COVID‐19) Define treatment outcomes
Unknown probability	Fundamental (Knightian) Uncertainty To reduce Fundamental uncertainty to Risk by gathering and analysing data and estimating probabilities	Ignorance Be alert to and reveal unknown but important factors, and reduce Ignorance to Fundamental uncertainty

The first casualty of an epidemic is evidence. In extreme situations the imperative of action is strong. ¹⁶ This makes extreme measures tempting—including scientific and ethical shortcuts. ¹⁷ , ¹⁸ Rigorous evidential and ethical criteria appear to obstruct progress. However, producing high quality evidence is the solution to the pandemic, not the problem.

In order to avoid fake facts and to provide sustainable solutions science needs to pay attention to the various kinds of uncertainty. Eliminating ambiguity, being alert to the unknown, and gathering data to estimate risks are crucial to preserve evidence and save lives.

CONFLICT OF INTEREST

I certify that there is no actual or potential conflict of interest in relation to this manuscript, and there are no financial arrangements or arrangements with respect to the content of this comment with any companies or organizations.

AUTHOR CONTRIBUTIONS

I am the only author of this manuscript; The manuscript is sent exclusively to you, has not been previously published elsewhere, and is not currently under review elsewhere.

Hofmann B. The first casualty of an epidemic is evidence. J Eval Clin Pract. 2020;26:1344–1346. 10.1111/jep.13443