The Challenge of Mortality Surveillance in Pandemics—Imperfect Perceptions of Reality

Outbreaks of emerging infectious diseases, especially those with pandemic potential, generate considerable demand for information that will aid in understanding the spread and severity of the infectious disease. This vital intelligence informs the dynamic risk assessment of the public health threat posed by the pathogen, which in turn influences public health advice and disease control measures.¹

Disease-specific mortality is one of the key indicators of disease severity and is accordingly a priority parameter gathered. The value of disease mortality data was apparent during the COVID-19 pandemic, during which many countries used their mortality surveillance systems to study and track the pandemic as it unfolded.^2,3 The resulting intelligence gathered enabled public health decisionmakers to understand the natural history of the disease, identify who and what settings were at risk, anticipate the timing needed to implement public health and social measures, and gauge those measures’ efficacy and the pandemic’s impact.

However, in most countries such endeavors involved herculean efforts to corral, link up, and analyze the various clinical, public health surveillance, and vital statistics data systems. The scale and pace required to reengineer these systems involved massive undertakings that were usually invisible to the public. In their article in this issue of AJPH (p. 1071), Khan et al. discuss the US Centers for Disease Control and Prevention’s (CDC’s) multiple system strategy for mortality surveillance during the COVID-19 pandemic. Their work is a timely and important record of the efforts taken and challenges faced and provides invaluable insights that can inform preparations for future pandemics during which the need to compile robust mortality surveillance data undoubtedly will arise.

Some critics may question the accuracy, completeness, and timeliness of the CDC’s strategy for mortality surveillance. However, it is important to recognize that all forms of surveillance have limitations. Surveillance data are frequently delayed, inaccurate, and subject to surveillance bias.⁴ Such data are a proxy of reality and, depending on the strength of surveillance systems used, may be incomplete. For example, studies from Europe in 2020 observed that, for every COVID-19 case reported, between 9 and 12 cases were missed by the surveillance systems.⁴

The true incidences of disease and disease mortality at any given time were difficult to determine with exact certainty because of variable coverage of COVID-19 testing and varying accuracy of clinical diagnoses. Public health–seeking behavior also affected the detection of cases. Even in death, which should be an objective and incontrovertible indicator, confirmation of COVID-19 as cause of death was not always possible, was frequently delayed, and was even missed.⁵ Consequently, reported COVID-19 numbers were only ever imperfect estimations. For those who were basing public health decisions on this information, it was akin to steering a car while looking through a cracked and distorted rearview mirror.

So how does the CDC’s approach compare with those used in other countries? International comparisons are difficult, as contexts and systems differ considerably between countries. Such comparisons are flawed because of the variation in case definitions used for COVID-19 between countries but also because of changes to case definitions and the emergence of new SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) variants over time.

Moreover, surveillance issues were especially challenging in countries with more fragmented or less well-developed surveillance systems.⁶ Health data and civil registration and vital statistics systems often exist independently, with limited interconnectivity or semantic consistency between them.⁷ Crucially, the robustness of surveillance systems and their data outputs were only ever as good as the clinical and laboratory reporting and surveillance data collection systems that formed the foundation of those surveillance systems. Where frontline clinicians have poor access to lab facilities, or lack the means or motivation to report, significant underdetection and underreporting of cases will occur.

Completeness of surveillance was also affected by public health–seeking and –reporting behaviors, which were in turn influenced by variable public awareness, social media, cultural norms and stigma, media misinformation and disinformation, reporting fatigue, and effects of at-home testing, as reported by Khan et al.

Also notable is the diversity of stakeholders interested in surveillance outputs. Traditionally, interest in these outputs has been narrowly thought of in terms of upward reporting to decisionmakers and politicians. However, during the COVID-19 pandemic, it also emerged that the public was interested in surveillance outputs, which may have influenced behavior.⁸ In the contemporary social media age, it must be recognized that members of the public are not passive receptacles of information but active consumers who choose who and what to believe and modify their behaviors accordingly. Going forward, we need to better understand how data consumerism operates and can be influenced for the good of public health.

Another emergent phenomenon during the recent pandemic was the plurality of lay interpretations, including by so-called armchair epidemiologists, that were widely shared through social media. These nonexpert analyses or interpretations of surveillance data were often inaccurate or taken out of context, highlighting a deficiency in public epidemiological literacy. Although public misinformation has always been a public health challenge, social media amplified its reach and speed of spread.⁹

It is also worth recognizing that politicians could and did politicize surveillance information to suit their political agendas, sometimes leading to misinformation or disinformation that undermined public trust.¹⁰ The diversity of opinions and interpretations of data does not help public confidence in official information, and discrepancies and changes in official reported mortality numbers may lead some to question the veracity of official figures.

Although there is the inherent danger of misinterpretation of mortality surveillance data, these data, however imperfect, have important utility, as highlighted earlier. The challenge is how to optimize the data’s utility. The CDC’s strategies of greater use of open source data, automated processes such as Web scraping, and synergies with third-party data aggregators are pragmatic approaches that worked. However, efforts elsewhere to integrate disease surveillance systems around the world have seen mixed results.⁷

Moreover, the pursuit for ever greater accuracy, coverage, and depth of detail of surveillance data comes with increasing cost. Similarly, attaining close to real-time data often necessitates a trade-off that relies on incomplete, provisional data.⁴ Consequently, the key questions that need to be answered are how much information is enough and what level of detail and accuracy can we afford? The latter will no doubt be guided by the risk posed by the pathogen but may be skewed by political appetite and public interest rather than by actual public health value. A pragmatic approach requires the consideration of the opportunity cost of investing precious public health resources in strengthening surveillance efforts versus the actual return on investment to guide public health decisions.

As we prepare for the next pandemic, further efforts are needed to strengthen and better link up surveillance systems. Beyond data system integration, we also need to consider functional integration, but this does not mean just database linkages.⁷ Triangulation of trends from various sources, as was done by the CDC and national public health agencies elsewhere, did help provide a more timely and complete picture.

In addition, there is considerable value in bringing in subject matter experts, surveillance epidemiologists, data scientists, public health professionals, and others to collectively add depth and context to provide data interpretations that best fit reality. This requires multidisciplinary and multisectoral collaboration at all levels from subnational to international.¹¹ Most importantly, we must not forget the role of surveillance—surveillance exists to guide public health action. If it does not do that, it is pointless.