Using Surveillance Data to Estimate Infectious Disease Burden: Opportunities and Challenges

Public health responses to infectious disease outbreaks rely on accurate counts of parameters that are often unavailable or even impossible to measure directly. Numbers of cases, timing of epidemic peaks, incubation times, transmissibility, vaccine efficacy, and numerous other parameters can be expensive (or impossible) to measure reliably. In developing infectious disease models capable of providing estimates of these quantities, modelers are often faced with recursive instances of the initial problem, as they review existing data to identify gaps and determine which needed quantities might be estimated and how. These estimations often require painstaking review of data from disparate sources and the eventual harmonization of those data. Given the expense of these efforts and the importance of their outputs, it is imperative that the public health community learns from this work, improves available data, and increases the reproducibility of modeling efforts.

ESTIMATING INFLUENZA HOSPITALIZATION BURDEN

These challenges are illustrated in the work of Couture et al. (p. 546) on the estimations of influenza hospitalization burden during 2022 to 2023 in the United States. Noting both the lack of robust state-level data on hospitalization owing to influenza and the limits that case ascertainment difficulties place on the accuracy of counts that are collected, Couture et al. adapted a modeling approach previously used for similar work on COVID-19.¹

To generate influenza estimates, they started with the best available data: FluSurv-Net (the Influenza Hospitalization Surveillance Network of the Centers for Disease Control and Prevention [CDC]) counts of hospitalizations in 13 states, adjusted for underreporting. They then used nine federal data sources together with data from more than 20 states to estimate covariates, including rates of positive influenza tests, percentages of deaths attributed to influenza, influenza vaccination rates, percentages of outpatient visits for influenza-like illness, and percentages of influenza patients among intensive care unit patients, along with demographic variables including racial/ethnic representation, rural–urban distributions, social vulnerabilities, and measurements of behavior risks. Couture et al. used these data to generate a Bayesian hierarchical model expressing state-level hospitalizations in terms of unobserved “true” hospitalization rates, which were in turn determined by covariates.¹ The resulting models stratified estimated hospitalization counts by age group, month, and state. Comparison of adjusted estimates for FluSurv-NET states to model outputs demonstrates the accuracy of the proposed approach.

APPLICATIONS AND LIMITATIONS OF THE MODEL

The contributions of this effort include both the immediate practical hospitalization estimates and numerous potential applications of the generalized approach. Beyond obvious extensions to case or death counts, one might imagine finer-grained examination of hospitalizations of different severities, possibly including those requiring intensive care unit admission or ventilator usage. Given appropriate data, these strategies might be adapted to other outbreaks or extended to address challenges with different data availability profiles.

Couture et al. provide a useful discussion of the potential pitfalls of using information from multiple, diverse sources to fill gaps in available data. Acknowledged limitations include the limited coverage of FluSurv-NET, the related difficulty of extrapolating to states not participating in FluSurv-NET, and gaps in data used to generate key covariates. Given known variations in influenza burden between states, these concerns could have a substantial impact on resulting estimates. Although broader participation in FluSurv-NET might reduce or eliminate this problem, expansion of surveillance efforts seems an unlikely prospect, thus ensuring the relevance of estimation efforts.

INNOVATIVE MODELING APPROACHES

Fortunately, the flexibility of the framework used by Couture et al. suggests approaches that might address these limitations and open new possibilities, As the authors acknowledge, their approach might be readily adapted to reflect new signals that may become available in the future, such as wastewater data surveillance, which might be seen as an attractive and inexpensive alternative to broad-based surveillance.² Electronic health record data on virology tests and admitting diagnoses have previously been used to inform hospitalization burden.³ More fine-grained insights on differing impacts across racial/ethnic groups might be achieved through integration of available data on influenza vaccination rates.⁴ Although any of these data sources might be added to the Bayesian hierarchical model, they all raise caveats of their own. For example, data on wastewater or vaccination rates might be hard to incorporate, particularly if those data came from a subset of states.

The approach used by Couture et al. is just one example of the recent explosion of innovation in infectious disease modeling research motivated largely (although not exclusively) by the COVID-19 pandemic. Recent efforts have explored the use of time series neural networks,⁵ novel sampling techniques designed to reduce bias,⁶ and Bayesian approaches for estimating latent variables.^7,8 These and numerous related advances present the potential of improved models capable of providing public health practitioners with more precise information about future outbreaks.

ENCOURAGING OPEN SCIENCE PRACTICES

To fully realize the utility of these and similar infectious disease modeling efforts, epidemiologists responsible for initial data collection and dissemination and modelers who use those data must work together to adopt modern data science techniques. On the data side, this means working toward compliance with the Findable, Accessible, Interoperable, and Reusable (FAIR) data vision of data sharing.⁹ FAIR strategies include the use of unique and persistent identifiers, rich and accessible metadata based on shared vocabularies and representations, clear descriptions of licenses and usage description, and detailed provenance describing the derivation of data sets. For modelers, these approaches would provide increased confidence that any of the multiple data sources used by Couture et al. to derive covariates would be readily available for future reanalyses.

Improved data practices are only one side of the story. Modelers should work to adopt open science practices, which have made huge impacts in bioinformatics, physics, and many other fields. Proposed reporting checklists aimed at supporting increased reproducibility of infectious disease modeling efforts specify extensive information, including the types of data used, how those data were processed, evaluation methods, the source code used to build the model, and other relevant details.^10,11 The modeling community might also adopt the expectation that modeling code be shared as open-source code built on open platforms and available on public code-sharing platforms as prerequisites for publication.

TOWARD GREATER TRANSPARENCY AND COLLABORATION

Increased data and code sharing for infectious disease modeling is complicated by organizational and privacy concerns. Many data sets cannot be shared easily, because of data use requirements or restrictive or ambiguous licenses that may not allow redistribution in altered forms. Similarly, code sharing may be limited by organizations that have not fully adopted open science principles. Recent federal initiatives, such as the CDC’s move toward increased data sharing¹² and the National Institutes of Health’s data-sharing policies, provide some encouragement for increased openness. In the meantime, imperfect transparency in published work can have immediate effects. For efforts such as the estimates of influenza hospitalization burden by Couture et al., this might mean sharing the code used to harmonize data sources and derive covariate matrices, even if the underlying data could not be shared.

BUILDING A SHARING AND ADAPTABILITY CULTURE

The effective use of modeling to inform responses to infectious disease outbreaks is likely to continue to require learning to quickly adapt existing methods to new conditions in the face of incomplete data. Couture et al. have developed a model that might be adapted to rise to this challenge. Establishing a culture of data and code sharing is an important step to help modelers learn from this and similarly instructive examples.