When the role of uncertainty is…uncertain

The SARS-CoV-2 pandemic has had a devastating effect on life expectancy and public health in the United States and globally (1). Since the virus’ recognized emergence in late 2019, it has killed tens of millions of people, including over 1.1 million in the United States (2). For most US states, excess mortality during the pandemic was up to twofold higher than reported COVID-19 deaths, suggesting that the true pandemic-attributable death toll is higher (3). The virus’ long-term health effects also recently led the Director-General of the World Health Organization to warn that around 10% of SARS-CoV-2 survivors would have care needs over the longer term, due to post-COVID medical conditions (4). That the impact of this pandemic could have been lessened through competent public health practice is demonstrated by the extraordinary heterogeneity in pandemic impacts across countries (5). While optimal management of a novel infectious disease is challenging, mathematical models of infectious diseases have been considered a useful tool to aid decision-making in the face of uncertainty (6); such models can capture the fundamental communicability of infection, as well as impacts of population immunity on epidemic growth. Prior to the SARS-CoV-2 pandemic, the US Centers for Disease Control and NIH invested in comparative “forecasting challenges” in which groups of modelers competed to see which models could provide the most accurate projections for different epidemic processes (7, 8). If these efforts were a dress rehearsal, the SARS-CoV-2 pandemic was “showtime” for many of the same groups. In this issue of PNAS, Shea et al. (9) provide an account of a multiteam pandemic modeling process in place in the United States during the pandemic and describe how Models of Infectious Disease Agent Study groups fared in projecting which back-to-work policies would be optimal during the early part of the pandemic (Fig. 1).

Fig. 1.

Pielke (10) proposed that science-informed policy decisions can be conceptualized as a two-dimensional problem, with difficulty of decisions driven by both scientific uncertainty and lack of value consensus. When there is value consensus and little uncertainty, decisions are easy; when there is neither, decisions are hard. In the presence of value consensus, uncertainty can be managed with the precautionary principle. During the SARS-CoV-2 pandemic, easy decisions were made more difficult through production of manufactured uncertainty, e.g., via scientific disinformation campaigns. Figure from ref. 11, used with permission.

As expected, modeling a novel disease process presented challenges to participating groups. They were presented a SARS-CoV-2 pandemic scenario, and data, for a mid-sized US county, which had adhered to nonpharmaceutical public health interventions through mid-May 2020, and were asked to model out reopening scenarios with a focus that was largely on health-related indicators (infections, deaths, hospitalizations, outbreak probabilities) as well as workplace closure days. Four competing strategies (ongoing closure, immediate opening, and two scenarios that linked reopening to epidemiological data or time since epidemic peak) were evaluated. Several key elements of SARS-CoV-2 epidemiology were unknown at the end of May 2020, when the exercise began, including airborne transmission (12), disease epidemiology in children (13), the durability of immunity (14), and the timeline for vaccine development, creating parameter uncertainty and structural uncertainty in models. Compounding these difficulties was what the authors’ term “linguistic uncertainty”: the fact that different groups interpreted challenge instructions differently, with resolution via discussions. But, the key result is this: Notwithstanding tremendous uncertainty from a variety of sources, all groups ranked the relative benefits of competing strategies similarly, notwithstanding substantial variation in the magnitude of benefit and uncertainty in projections from individual models. Modeling projections could also be validated with real-world data (9, 15).

At second glance, however, this consistency is puzzling, considering a heterogeneous response to the pandemic across US states (15), and indeed globally (3, 5). If basic information on disease dynamics early in the pandemic was sufficient to identify optimal control strategies, why did the United States, for all its resources, experience one of the highest pandemic death tolls per capita during the SARS-CoV-2 pandemic? It does not appear that the answer lies in scientific uncertainty that led decision makers to act incorrectly or carelessly.

When scientists discuss policy, or advise policy makers, they often focus heavily on the role of scientific uncertainty as the driver of policy decisions. If only we understood the process in question more clearly, we would be able to give optimal advice to decision makers. However, as Roger Pielke has pointed out (10), science-driven policy decisions require that we consider two domains, rather than one. While scientific uncertainty is important, policy decisions also reflect the values of those making the decisions. Pielke suggests that when there is value consensus, and little scientific uncertainty, decisions are easy. Important scientific uncertainty and disagreement on values combine to make decisions “hard” (10). Where there is lack of value consensus, some actors may find scientific uncertainty desirable as it may be, to use Pielke’s words, “a resource for...bargaining, negotiation, and compromise.” Values may be an even more important driver of policy decisions than scientific uncertainty; when there is value consensus, we can respond to scientific uncertainty with precaution. The findings of Shea et al. indicate that it was likely lack of value consensus, rather than scientific uncertainty, that drove the heterogeneity of the US pandemic response.

The findings of Shea et al. indicate that it was likely lack of values consensus, rather than scientific uncertainty, that drove the heterogeneity of the U.S. pandemic response. Coronavirus collection: Not eligible for waived OA fees.

The ethical framing of the pandemic response used by Shea et al. is a utilitarian one, as is typical in public health. An outcome is considered good if it creates the greatest good for the greatest number of individuals, at reasonable cost (16). However, different actors may have favored alternate value positions at the height of the pandemic, publicly or privately. Specific economic sectors may have been more concerned with disrupted operations or financial losses than with the overall health of the population. The so-called lockdowns early in the pandemic resulted in dramatic reductions in the price of oil (and, one presumes, the earnings of the oil and gas industry) (17); the use of technology to support “work from home” has created major challenges for the commercial real estate sector (18). It seems not unlikely that some sector lobbied politicians for policy positions more favorable to their commercial activities. As Jadad has noted, public health workforces in the United States and elsewhere are within government, and must be responsive to the demands of politicians “out of fear of committing career or financial suicide” (19). Viewed through this lens, the heavy funding of disinformation campaigns during the SARS-CoV-2 pandemic (20), and muddying of the epidemiological landscape through contraction, rather than expansion, of public health surveillance efforts and data sharing (21), may have (intentionally) generated scientific uncertainty where little important uncertainty was initially present, creating a “resource for negotiation” on policy.

Many scientists who have served as public health advocates during the pandemic have been admonished for “politicizing” public health. But, health (as a societal objective influenced by policy) and science itself (22) cannot be dissociated from politics. Indeed, the political geography of the United States has emerged as a powerful predictor of the pandemic’s impact over the past 3 y (15).

In summary, Shea et al. (9) have demonstrated that scientific and epidemiological uncertainties were not barriers to formulating optimal disease control policy early in a pandemic caused by a novel virus. I suggest that the greater barrier to good policy in the United States and elsewhere was lack of value consensus, at least behind closed doors. Given current pandemic threats on our horizon (23), perhaps we need to apply the same attention and study to issues of values, ethics, and equity during pandemics as we have successfully applied to questions of epidemiological uncertainty.