Learning from weather and climate science to prepare for a future pandemic

Established pandemic models have yielded mixed results to track and forecast the SARS-CoV-2 pandemic. To prepare for future outbreaks, the disease-modeling community can improve their modeling capabilities by learning from the methods and insights from another arena where accurate modeling is paramount: the weather and climate research field.

To prepare for future outbreaks, the disease-modeling community should draw on the methods and insights of the weather and climate research field. Image credit: Shutterstock/NASA Images.

We argue that these improvements fall into four categories: model development, international comparisons, data exchange, and risk communication. A proper quantification of uncertainties in observations and models—including model assumptions, tail risks, and appropriate communication using probabilistic, Bayesian-based approaches—did not receive enough attention during the pandemic. Standardized testing and international comparison of model results is routine in climate modeling. No equivalent currently exists for pandemic models. Sharing of data is urgently needed. The homogenized real-time international data exchange, as organized by the World Meteorological Organization (WMO) since the 1960s, can serve as a role model for a global (privacy-preserving) data exchange by the World Health Organization. Lastly, researchers can look to climate change and high-impact weather forecasting to glean lessons about risk communication and the role of science in decision-making, in order to avoid common pitfalls and guide communication. Each of the four improvements is detailed here.

Accept and Incorporate Uncertainty

The accuracy of a numerical weather forecast improves with better estimates of the current state of the atmosphere (1).^* One source of error here relates to the accuracy and type of measurement instrument used and low representativity of measurements. For example, station-based measurements are heavily biased towards land, while satellite coverage is more homogeneous, but has poor vertical resolution.

These issues have their analogies in SARS-CoV-2 measurements: Virus loads in sewage water only represent the catchment area; tests are prone to false-positive and false-negative errors; and testing strategies change over time. But even for the hypothetical case of a perfect estimate of the current state, the forecast will have errors because of model shortcomings.

In weather and climate models, shortcomings result from limited spatio-temporal resolution and the need to simplify the effects of unresolved processes, such as convection. Also, some relationships are not well understood—for example, the influence of clouds on the circulation. Modeling epidemics relies on similar simplifications and aggregations, where the transmission or the severity of disease may depend on age, sex, human behavior, environmental conditions, and individual pre-existing health conditions.

When combined, imperfect measurements and model shortcomings lead to a phenomenon called deterministic chaos (2), in which small uncertainties lead to forecasts that can develop in entirely different directions. Weather forecasters deal with deterministic chaos by running the models multiple times, with small changes in the start conditions. Such an ensemble approach produces a set of different plausible forecasts, which, in the case of weather forecasting, allows for a probabilistic weather forecast.

Ensemble forecasts have gained some ground in the field of pandemic predictions (3, 4). Importantly, uncertainties deduced from the ensemble spread could become an integral part of pandemic-related communication efforts, just as for daily weather reports. Ensemble forecasts can be tuned by using a mathematical technique called data assimilation, which attempts to estimate the most likely outcome and is commonly used to produce weather forecasts. Data assimilation has led to the development of efficient algorithms—for example, ensemble Kalman methods—that also suggest the most likely model parameters (5).

Ensemble data-assimilation methods can help to forecast disease outbreaks. In Fig. 1, we show the results of one such modeling exercise, used to assess the most likely state of the pandemic in fall 2020 in Switzerland. We used an extended dynamic SEIR (susceptible–exposed–infected–recovered) model, with age classes plus number of hospitalized and quarantined. Surveillance data came from three sources (6). Hospitalizations and fatalities were collected by the Federal Office of Public Health (BAG; ref. 7) and SARS-CoV-2 genetic data measured in wastewater collected by the Swiss Federal Institute of Aquatic Science and Technology (Eawag; ref. 8). The data assimilation used, an ensemble Kalman inversion scheme, constrains the future by past measurements in a process known as history matching and finds optimal SEIR model parameters. We also used reported infection numbers instead of wastewater and obtained comparable results; however, wastewater data are independent of the testing strategy.

Fig. 1.

Estimated hospitalizations due to COVID-19 until November 7, 2020 (orange line) and projection into the future until the end of November 2022 in Switzerland. Used surveillance data in the assimilation scheme are reported hospitalizations, fatalities, and infection numbers until November 7, 2020. Subsequently, three scenarios for the effective reproductive number R (0.9, 1.0, and 1.1) are projected into the future, either corresponding to uncertainty in the measurement of R or representing different intervention measures (labeled as reduced, continued, and strong measures, respectively). Confidence intervals are given for the middle scenario, based on the ensemble spread.

The model output includes the number of infections, hospitalizations, and fatalities, as well as the time-dependent reproduction number R (9). These results allow subsequent study projections of future infections, hospitalizations, and fatalities that depend on R. Scenarios reflect decisions of what containment measures are in place, how they collectively affect R, and uncertainties in R. Included are ensemble-based CIs, which enables real-time projections with uncertainty bounds.

The modeling system finds the best match between the future and the past and estimates SEIR model parameters (e.g., transmission rates). Compared with observations from the time, the model indicates that the true value of R in mid-November and December 2020 in Switzerland was between 0.9 and 1, corresponding roughly to the intermediate scenario.

There is one key difference between forecasting weather and disease. No perfect analogue exists for the spread of mutations in SARS-CoV-2 that lead to changes in transmission or disease. In the case of the spread of a new variant of a pathogen, routine recalibration of the pandemic model is necessary. But data assimilation is ideally suited to perform this recalibration based on observations that reflect the impact of the mutation on, for example, the incubation period.

Coordinate Model Development

Standardized test cases and scenarios to quantify model differences have been routine in climate projections for almost three decades, with dozens of models now participating in commonly coordinated model intercomparisons and sharing their results via the Coupled Model Intercomparison Project (10). Standardized setups and data structures also allow projections based on a combination of models, which are shown to often be more skillful than those based on single models in both seasonal forecast (11) and climate applications, and they allow one to more robustly assess uncertainties (12).

In the realm of climate science, global modeling efforts and model intercomparison projects are regularly organized by the World Climate Research Programme, to support the Intergovernmental Panel on Climate Change. The disease-modeling community needs a counterpart to check and improve pandemic forecasts. Over time, such efforts allow scientists to understand which of the inevitable simplifications in models matter most and which model structures perform best. While there is not a pandemic every year for repeated evaluation (just as, by definition, extreme weather events are rare), standardized testing of a model with test cases in different regions or countries relying on different strategies or interventions at different points in time during the COVID-19 pandemic would better guide model development and so provide more robust projections based on multiple models.

Global Data Exchange

The esteemed British meteorologist Sir Napier Shaw once said that: “Meteorology is necessarily a cooperative enterprise; one’s own measurements at one time must be viewed in relation to other measurements at the same or other times” (13).

The same applies to science in a pandemic and highlights the need for global standardized, routine, and immediate data exchange (14). While preprints of publications are immediately available, the pandemic has demonstrated severe limitations in how health agencies collect, document, and provide data that are critical to calibrate models and evaluate the effectiveness of interventions. Shaw’s plea for global data exchange in meteorology paved the way for the World Weather Watch Programme managed by the WMO, which was founded in 1950 and became a role model for long-term international cooperation (15, 16).

The international exchange of meteorological data has never been interrupted since, even at the height of the Cold War, and led to the so-called global observing system (17). WMO member states are typically represented through their national meteorological agencies, which maintain national standardized surveillance networks, operate numerical weather-prediction models, offer expert judgments, provide climate services, and issue warnings in close collaboration with public safety organizations.

Taking this as an exemplar, many of the scientific networks that formed ad hoc during the pandemic could integrate into national health agencies that could coordinate the data collection and operate their own operational forecasting models similar to national weather services. These health agencies would have a renewed, more ambitious, and internationally coordinated portfolio. Routinely testing disease agents in wastewater could become an important cornerstone in future pandemic preparation and response planning, comparable to the global network of WMO-standardized surface meteorological stations, which remain the backbone of modern weather forecasting.

Risk Communication

In terms of risk and uncertainty communication, the lessons learned from meteorology and climate change indicate a widespread misbelief that more frequent, and more accurate, scientific information automatically leads to more rational human decisions. Instead, we know that different communication concepts rooted in social science are appropriate for different situations (18).

Faced with a new threat, such as climate change, when uncertainty is large and decisions must be made rapidly, we lack the familiar heuristics from the past to decide, and we therefore must tolerate uncertainty. Clear communication of uncertainty builds trust. It is also not enough to be scientifically correct: In situations where fundamental values and worldviews are important, numbers, facts, and scenarios can only be a starting point of a societal debate on what to do (19). While the advice to seek shelter when a hurricane is coming may be relatively uncontroversial and unpolitical at first, even this example is not free from social biases and different risk perceptions (20, 21).

It is time to start building predictive systems for the next pandemic.

The pandemic has shown us that there is widespread disagreement in society about what measures are appropriate to ensure public health at the expense of limiting personal freedom. The same is true for individual scientists and expert bodies: Should they only provide numbers that describe the current situation? Should they provide predictions? Should they recommend policies? Often, scientists have multiple roles. Can they represent the views of an interdisciplinary body advising the government and advocate their own views on social media at the same time? A key question for us is how to protect scientists and the integrity of the scientific process when they are attacked by those who question established facts, while being open to discussion with those who acknowledge facts, but oppose specific measures, because positive and negative effects of a measure are perceived differently by different people.

Many of the above raised questions do not have unique and final answers. What we have learned from weather forecasting and climate change is that established procedures, clear mandates, trust between scientists, the public, and policymakers, paired with clear and transparent communication (22), are crucial to establish common ground. So are openly dealing with mistakes, an explicit discussion of uncertainties, tail risks, and the assumptions underlying future scenarios.

Framing the future as multiple “what if” scenarios, or as requirements needed to achieve a certain goal, can provide a basis for the political and public debate without compromising the independence of the scientific process.

Scientific information alone cannot and should not determine policies, but universal and immediate access to privacy-preserving data and carefully calibrated model projections with quantified uncertainties can ensure that the best scientific information is available to guide policy.

At times of intersecting crises, cross-disciplinary interaction is more important than ever. Weather and climate scientists are not wiser than experts in other fields, and the pandemic, like climate change, is not only a natural science problem. But a half-century of experience with uncertainties, forecasting in the spotlight, global data exchange, and public debates with billions of dollars at stake offer the opportunity for disease modelers to avoid some pitfalls and provide opportunities for mutual learning.