Public health response to COVID-19: the forecaster’s dilemma

As the health crisis is unfolding on the world stage, with the United States leading in both the number of infected and dead, we are witnessing sparce examples of success in controlling the epidemics and a solid daily dose of failures affecting almost every aspect of our lives. This is a price we are paying for systematic negligence of national and global surveillance, early prevention, and thoughtful preparedness planning needs. At the peak of pandemic, we are still debating the use of masks and school openings. The rush to produce a vaccine has sidelined conversation about rapid, frequent, and affordable mass testing.

In the midst of these ongoing challenges to control the spread of this virus, it is more valuable than ever to understand and communicate how an actionable and effective public health response could be developed through a systems-thinking approach. Systems thinking is a core skill in public health and helps policymakers build health programs and policies reflecting awareness of and preparedness for unintended consequences. This approach calls for treating all aspects of response as an evolving process involving various stakeholders and actors, with all their inter-relationships. A systems science approach classifies a system as simple, complicated, or complex. For a simple system, a well-defined question has one or more executable solutions. A complicated system requires posing a set of questions correctly for identifying a set of solutions; then such solutions must be continuously monitored to assure course corrections. For a complex system, not only are the questions difficult to get right, the solutions may be constantly in flux. Complex systems require in-depth learning, training and retraining, detecting emerging pattens at various scales, building near-term forecasts, and testing our thinking in real time. A public health response to a localized outbreak of a well understood infection or a multistate outbreak of emergent pathogens illustrates the contrast in behavior of a simple or complicated system, respectively. The pandemic calls for accepting complex system responses and behaviors with feedback loops, rapid change, and exponential escalations of challenges to establish a controlled environment. Responding to pandemic mistakes is costly, and time is short.

A few weeks ago, the 10th conference on complex systems organized by the New England Complex Systems Institute, USA [1] attracted over 200 participants from many research fields: mathematics, computer sciences, economics, epidemiology, systems biology, and social sciences, to name a few. In the new virtual environment, almost every presentation directly or indirectly touched the pandemic and its effect on what we know and what remains unknown. The overarching theme of the conference was the best use of next generation science to meet societal, organizational, and global challenges and discover the potential of compassionate science. My talk addressed the challenges of forecasting infectious outbreaks.

Forecasting methodology is under active development and improvement in many disciplines and practical applications. The remarkable advances in forecasting have been well recognized in meteorology for predicting local weather, overall trends, extreme events, and trajectories for rapidly evolving processes like storms and hurricanes. As the accuracy of prediction improves continuously, we tend to take forecasts seriously and follow the prescribed response plans of local or federal authorities. Yet, sometimes these forecasts and advisories are not precise. Sometimes, we were prepared for a severe storm and it was less than predicted. In this case we could think about the costs of unnecessary preparations. However, sometimes a storm is more severe than predicted and there is no preparation. In that instance, the costs may be more extreme from lack of preparation. In forecasting, these mismatches are called Type 1 and Type 2 errors.

In public health the issue of the mismatch is convoluted because it involves the proof of no appearance. For example, if as a forecaster, I predicted a strong flu season and you listened, got the needed vaccination, and the flu was severe, yet you did not get sick. Your reaction to this forecast of a strong flu season could be either ‘Thank you’, but it may be ‘I never get flu anyway.’ In another scenario, I predict ‘No flu this year’ and you ignored the vaccination invite from your doctor. But I was wrong, and you become very ill with the flu. The reaction to this forecast of low flu season rightfully should be critical. It is clear in the second example that there was a mismatch error when I predicted no flu season, and you became ill. However, in the first example it is more difficult to discern if the reason you did not become ill was due to an accurate forecast and proper preparation or a forecasting error. In the best of circumstances, the correct forecast and proper preparation should lead to no appearance of the disease. Unlike a meteorological forecast, when an infectious disease forecast is accurate and an outbreak is avoided, there is no proof that the ‘storm’ existed and was successfully ‘weathered.’

Proof of no appearance in public health means controlling an outbreak. The crisis is avoided and the only evidence to point to success is in the cost of preparation. This is the disease outbreak forecaster’s dilemma.

Though the price for forecasting errors on both sides is always uneven, the price that includes lives, health, wellbeing, and economic cost of being unprepared could be far higher and should always be minimized. Just as we consider the weather forecasts seriously, the precautionary actions and response plans from disease outbreak forecasts and public health professionals must also be recognized and duly considered by all who could be affected. This requires that trust and cooperation between public health authorities and the public be established and continuously maintained.

It is clear today that the cooperation between public health agencies, officials, and the public has become highly contentious and polarized. The forecaster’s dilemma is getting worse with incomplete data and misinformation, without knowing what could be known if measures, like mass control strategies, are universally applied. Knowing the unknown in case of COVID-19 requires collecting, sharing, and analyzing standardized, harmonized, and synchronized data across agencies. In no other way can we learn fast that asymptomatic people are transmitting infection, identify the super spreaders, and stop transmission to buy time to organize effective strategies. For infectious disease outbreaks, we could learn the unknown with the rapid, frequent, and affordable mass testing for both symptomatic and asymptomatic cases, robust surveillance systems, and contact tracing at the outbreak onset. The grassroot organizations are advocating for rapid tests and small businesses, like e25bio or Sherlock Biosciences in my hometown Cambridge, Massachusetts are working on developing testing platforms [2]. In other parts of the world, fast blood tests to check for coronavirus antibodies are now available without prescription offering some assurance and guidance to the public. Thus, the existing standards of the U.S. Food and Drug Administration for rapid testing have to prioritize high speed over high accuracy in order to implement mass COVID-19 surveillance.

We need this knowledge to reactivate an entire slew of seemingly small steps leading to sweeping behavior change by each member of our societies. Those steps include our decisions about

• whom we select to lead and fund the local and national preparedness efforts and ensure that public health workforce is well equipped and supported to detect and control emerging threats [3];

• how to keep the public well informed about health risks and health behaviors at individual and community levels and to enable the public to make decisions based on strong evidence and good science [4];

• how we train public health professionals to devise effective protective strategies using cutting edge techniques, theories, and concepts, like systems-thinking and behavioral change theories to advocate and communicate science-based health messages to all who should take proper actions.

I hope our readers will be inspired to read the new book on the COVID-19 Catastrophe [5] by Richard Horton—an avid medical writer and the Editor-in-Chief of The Lancet—and reflect on what should be our priorities in finding solutions to mitigate the long-term consequences of the pandemic moving forward. The thoughtful book review [6] by Phyllis Freeman and Anthony Robbins, Journal’s Editor Emerita, reminded me to check the last paragraph of the C.J. Peters' Virus Hunter 1998 book. It reads [7]:

I think it's essential that we be prepared. There's been a lot of discussion among experts and commentators, book and magazine journalists about how much of a threat these so-called emerging viruses represent. The current vogue is they're not as much of a large-scale, Andromeda Strain-type threat as some people have suggested. And they might be right. Maybe nothing’s going to happen. But there is something terrifying about the fact that nothing can stop the implacable evolution of these viruses as they test, through mindless mutation, ever more strategies to facilitate their survival, a survival that just represent disease and death for us humans. Maybe no deadly pandemic will occur. But I wouldn’t want to bet my life on it.

We cannot turn back the clock to use the best judgment and actions to blunt the force of this pandemic now, so it continues, and we have no choice but to deal with its long-term consequences by building resilient response and recovery systems.