Skip to main content
JMIR Public Health and Surveillance logoLink to JMIR Public Health and Surveillance
. 2024 Apr 8;10:e54340. doi: 10.2196/54340

Responding to the Return of Influenza in the United States by Applying Centers for Disease Control and Prevention Surveillance, Analysis, and Modeling to Inform Understanding of Seasonal Influenza

Rebecca K Borchering 1,, Matthew Biggerstaff 1, Lynnette Brammer 1, Alicia Budd 1, Shikha Garg 1, Alicia M Fry 2, A Danielle Iuliano 1, Carrie Reed 1
Editor: Amaryllis Mavragani
Reviewed by: Eric Lau, Jennifer Olsen
PMCID: PMC11036179  PMID: 38587882

Abstract

We reviewed the tools that have been developed to characterize and communicate seasonal influenza activity in the United States. Here we focus on systematic surveillance and applied analytics, including seasonal burden and disease severity estimation, short-term forecasting, and longer-term modeling efforts. For each set of activities, we describe the challenges and opportunities that have arisen because of the COVID-19 pandemic. In conclusion, we highlight how collaboration and communication have been and will continue to be key components of reliable and actionable influenza monitoring, forecasting, and modeling activities.

Keywords: disease burden, modeling, seasonal influenza, surveillance

Context

The COVID-19 pandemic disrupted the typical timing, intensity, and duration of influenza virus activity in the United States and many parts of the world [1,2]. These disruptions are likely due in part to the adoption of nonpharmaceutical interventions, such as mask wearing and physical distancing, which were deployed to prevent SARS-CoV-2 transmission and likely also reduced transmission of influenza viruses and other respiratory viruses [1]. Reductions in the use of nonpharmaceutical interventions in the United States during the 2022-2023 influenza season compared to what was observed during the 2020-2021 and 2021-2022 seasons may have contributed to a return of influenza virus activity more comparable to levels observed during some prepandemic seasons, which had annual estimates of 9 million-41 million illnesses, 140,000-710,000 hospitalizations, and 12,000-52,000 deaths between 2010 and 2020 [3]. The interim estimated burden of influenza for the 2022-2023 influenza season indicated that between 27 and 54 million illnesses, 300,000 and 650,000 hospitalizations, and 19,000 and 58,000 deaths occurred that season [4]. The 2022 and ongoing 2023 influenza seasons observed in the Southern Hemisphere winter (approximately May to September) were characterized by the circulation of multiple seasonal influenza viruses (ie, influenza A/H3, A/H1, and B Victoria) [5]; atypical timing patterns, including early and late seasons; or increased intensity compared to recent seasons during the COVID-19 pandemic. However, changes in care-seeking behaviors, testing practices, and general reporting practices due to the pandemic complicate the interpretation of influenza surveillance data.

The experience of the Southern Hemisphere in 2022 and 2023 [5] and the atypically early and rapid start to the 2022-2023 influenza season in the United States [6-8] serve as reminders that seasonal influenza remains a major public health threat. Influenza burden during the 2022-23 season was elevated compared to activity observed in the United States during the first 2 years of the COVID-19 pandemic [4,9], and early and high rates of influenza-associated hospitalizations were observed, particularly in children [7,8,10]. Over the past decade, the Centers for Disease Control and Prevention’s (CDC’s) Influenza Division has worked internally and with external collaborators to develop, validate, and improve forecasting, mathematical modeling, and analytical tools that help translate US influenza surveillance data into action by forecasting short-term influenza activity and characterizing the burden and severity within and across influenza seasons. With the decreased influenza activity observed in the 2020-2021 season, atypical activity and timing in the US 2021-2022 and 2022-2023 seasons, and the Southern Hemisphere’s 2022 and 2023 seasons, continued circulation of SARS-CoV-2, and the return of other respiratory pathogens (eg, respiratory syncytial virus [RSV]) [1,11], we summarize these activities to highlight how these tools can help inform influenza prevention and control efforts during future seasons in the postpandemic respiratory virus landscape.

Data From Surveillance Systems Provide the Foundation for Influenza Situational Awareness and Risk Communication

Multiple surveillance systems provide complementary information to create a comprehensive picture of influenza activity in the United States [6,12]. These information streams are fundamental for situational awareness, risk communication, and subsequent modeling and analysis. Types of surveillance data collated and distributed by the CDC include influenza hospitalizations (hospitalization rates by age, sex, and race or ethnicity, as well as clinical characteristics and outcomes) [9,13]; influenza-related mortality (the National Center for Health Statistics Mortality Surveillance Data and pediatric influenza-associated mortality); virologic (eg, percent positivity, influenza virus type or subtype, and genomic characterization of influenza viruses, which can be used to assess the match of circulating strains to vaccine components); and syndromic outpatient illness [12]. Maintaining these data sources requires coordination within the CDC as well as with international organizations, vital statistics offices, state, and local public health partners in both clinical and laboratory settings, hospitals, and health care providers.

Weekly FluView [6] reports provide systematic data updates and interpretation from the many influenza surveillance systems. These reports, related data updates, and visualizations provide a valuable resource for the CDC, external partners, and the public to understand and interpret influenza activity. Regular and rapid dissemination of surveillance data that is easily accessible and interpretable plays an important role in risk communication and promotion of prevention and mitigation measures, such as vaccination, testing, antiviral use, and nonpharmaceutical interventions.

Given the cocirculation of multiple respiratory viruses with the potential to cause severe illness, interpreting influenza virus data alongside complementary surveillance data for other respiratory viruses provides additional context to the added burden of influenza on the population and the health care system. Symptom-based surveillance for influenza has traditionally been complicated, due in part to overlapping symptomologies. Accordingly, data from the US outpatient Influenza-like Illness Surveillance Network, where outpatient visits are monitored for influenza-like illness defined as fever plus cough or sore throat, can include reports due to infection with cocirculating respiratory pathogens that may be clinically indistinguishable from influenza, including COVID-19 and RSV. In addition, changes in health care–seeking behavior due to the COVID-19 pandemic, both in terms of provider choice and timing relative to illness onset and severity, will be important to consider when assessing surveillance data collected from health care settings. Additionally, the expansion of remote or telehealth options will complicate surveillance, particularly for laboratory-based surveillance. The Influenza Division will continue to leverage the strengths of different surveillance systems to provide a comprehensive understanding of seasonal influenza virus activity. In addition, in order to better understand the burden of influenza in relation to other respiratory viruses like RSV and SARS-CoV-2, the Influenza Division has cosupported the development of multipathogen respiratory virus surveillance networks and developed publicly facing summaries that compare acute respiratory illness emergency department visits for laboratory-confirmed influenza, RSV, and COVID-19 [14] and weekly in-season rates of laboratory-confirmed influenza, COVID-19, and RSV-associated hospitalizations by age, sex, and race or ethnicity [15]. Influenza surveillance and communication efforts will continue to adapt in the uncertain and changing landscape of continued respiratory virus cocirculation.

Estimating Influenza Disease Incidence Using Analytic Methods That Account for the Underdetection of Influenza Provides a More Complete Picture of the Burden and Severity of the Influenza Season

Not all people seek medical care for respiratory illnesses, and not all of those who do will be tested to confirm infection with a specific pathogen. Even when an individual seeks medical care and is tested, this may not occur during the time window of their illness when a collected sample has the potential to test positive (ie, the pathogen or antibodies may no longer be detectable). A variety of viral and bacterial respiratory illnesses are clinically indistinguishable from one another, complicating diagnosis based on symptoms or clinical findings alone. Laboratory testing for specific pathogens can be more or less common in certain care settings, for different age groups, and at differing levels of disease severity. To account for the incomplete detection of influenza virus infection from clinical testing in health care settings, models used to estimate the disease burden incorporate information on health care usage and laboratory testing for respiratory viruses [16]. Each year, the CDC uses mathematical and statistical models to estimate the broader disease burden from influenza in the US population; since 2018, this has been estimated weekly during the influenza season [17]. In the 2022-2023 season, the first preliminary in-season weekly influenza burden estimates were released in October [4], earlier than any other season due to the atypically high levels of influenza activity observed at that time of year.

The ongoing effects of the COVID-19 pandemic on health care usage and testing for respiratory viruses [18], including influenza, are unclear and will likely continue to evolve over time. Major changes to health care–seeking behaviors, including where patients are seeking care and the use of clinical versus at-home diagnostic testing, could influence trends observed in long-standing surveillance systems, impact ongoing efforts to forecast influenza activity, and introduce new biases into epidemiological and clinical studies that rely on the results of clinical testing. The CDC will continue to make estimates using the best available data, adapt to challenges presented by atypical seasonality, and assess developments in care-seeking behaviors and influenza testing [19]. Communicating in-season and end-of-season burden estimates through the CDC website and FluView reports will help promote public awareness and add to our understanding of influenza burden alongside COVID-19 burden. These ongoing activities will ensure that influenza burden estimation continues to be improved, using the best available statistical methods and data sources.

Collaborative Forecasting Efforts Hosted by the CDC Use Surveillance Data to Look Ahead and Provide Quantitative Forecasts for Short-Term Trends in Influenza Activity

Although these traditional surveillance systems and disease burden estimates are comprehensive, they measure influenza activity after it has occurred, which limits their utility to anticipate future trends and inform risk assessment, resource allocation, and health care preparedness. To address these limitations, the CDC has been collaborating with and supporting public health officials and researchers from academia, industry, and the government in open forecasting challenges since the 2013-2014 season [20]. This process has ensured that forecasting targets are relevant to public health, forecast data are openly available and communicated; and the evaluation of forecast performance across different targets, seasons, geographic locations, and methods is transparent [21-23]. This foundation helped lead to the rapid establishment of the COVID-19 forecast hub in early 2020 [24-26].

Building on 10 years of experience and working in collaboration with academic and industry partners, including 2 Centers of Excellence in Influenza Forecasting (the University of Massachusetts and Carnegie Mellon University), the CDC has regularly communicated short-term forecasts at the national, state, and territory level for the 2023-2024 influenza season for the weekly number of patients hospitalized with laboratory-confirmed influenza based on data from the National Healthcare Safety Network (NHSN) hospital admission system (previously known as the US Department of Health & Human Services (HHS) system HHS-Protect) [13,27]. This hospitalization-based forecast target was adopted during the 2021-2022 season when reporting of laboratory-confirmed influenza hospital admissions became mandatory in February 2022 at the state- and hospital facility- level and is different from the forecast target used before the COVID-19 pandemic, which was based on outpatient visits for influenza-like illness [20]. This shift was made because of changes in outpatient care seeking associated with the pandemic, the continued cocirculation of SARS-CoV-2 and RSV, which complicated the interpretation of influenza-like illness data, and the new availability of laboratory-confirmed influenza hospital admission data through NHSN for all 50 states. This centralized source of virus-specific hospitalization data may provide actionable and reliable information moving forward [28,29]. Experience from using the NHSN hospital admission indicator for influenza forecasting during the 2021-2022 and 2022-2023 influenza seasons [30], as well as during other forecasting and modeling efforts during the COVID-19 pandemic, indicates that the NHSN hospital admission data [13], which became available during the COVID-19 pandemic, are a robust source of timely information about the impact of influenza in US hospitals. Reported data include daily state-level counts of total hospitalized patients, previous day’s admissions, and total hospitalized intensive care unit patients each with a laboratory-confirmed influenza virus infection [13].

Insights from past forecasting efforts, including for the COVID-19 pandemic and past influenza seasons, can provide users of forecasts with valuable takeaways for interpreting influenza forecasts with appropriate levels of caution. First, forecasts are less reliable during periods of rapid change (eg, during periods of sharp increases in incidence or when activity is peaking). During these periods, forecasts are often unable to reliably predict changing trends, and interim evaluation of early season forecasts received during the periods of the 2022-2023 season when influenza activity was increasing rapidly or peaking indicated that more reported hospitalizations than expected fell outside of the forecast prediction intervals, indicating a lack of forecast reliability. Improvements in forecasting methodology are needed to ensure that periods of rapid change can be forecasted reliably. Due to the unreliability of forecasts observed in these periods, the Influenza Division did not start public communication of these forecasts until observed performance became more reliable (January 2023). To begin addressing this limitation, the FluSight challenge [27] piloted a set of new forecast targets for probabilities of observing categorical changes in hospital admission trends, such as increases and decreases above certain thresholds, at state, territory, and national levels and will continue the use of these targets in the 2023-2024 season. Second, ensemble models typically outperform individual model forecasts, making them a more robust choice for forecasts with dependably accurate performance [22-24,26]. Finally, forecast performance degrades as they predict farther into the future, as evidenced by the higher performance of 1-week ahead forecasts than 2-, 3-, and 4-week ahead forecasts [22], and the CDC may also pause communication of these longer-horizon forecasts in periods of rapid change. In addition to the challenges associated with rapid increases, atypical seasonality, and new data sources for forecasting present additional challenges for forecasting influenza during and following the COVID-19 pandemic since many forecasting models rely on historical data to inform predictions of seasonal trends. Ongoing discussions with academic and industry partners that complement collaborations with the Council for State and Territorial Epidemiologists, including the pilot activities mentioned above, will help ensure that forecasting and modeling results are shared, discussed, and interpreted in a timely manner.

Additional Modeling Approaches Are Needed to Increase Our Understanding of the Potential Severity and Timing of the Upcoming Influenza Seasons

Inherent uncertainties in epidemiological conditions and the effects of interventions over longer time horizons require different modeling frameworks than forecasting alone. Therefore, the CDC started collaborating with the Scenario Modeling Hub to use a multiple-model approach to consider potential courses of the 2022-2023 influenza season, including 4 scenarios covering higher and lower rates of vaccine impact and different levels of preexisting immunity [31-33]. The simultaneous consideration of multiple scenarios (sets of assumptions) allows modelers to consider what could happen under a variety of different circumstances. Results from the first round of the influenza Scenario Modeling Hub suggested in September 2022 that hospitalizations in the 2022-2023 influenza season would be higher than in recent seasons [32]. Early in the 2022-2023 season, projections for scenarios with lower population-level immunity demonstrated the plausibility of peak hospitalizations comparable to those of larger prepandemic influenza seasons, while projections for scenarios with higher vaccine effectiveness and coverage were accompanied by substantial reductions in peak size and cumulative burden of hospitalizations due to influenza [32]. Subsequent rounds of the Influenza Scenario Modeling Hub undertaken in November and December 2022 reinforced these findings that projected peak hospitalizations could be comparable to those of larger prepandemic influenza seasons in the most plausible scenarios [32]. As demonstrated by several completed rounds of the COVID-19 Scenario Modeling Hub [34], these longer-term projections provide a useful information source for intervention planning, preparedness, and decision-making [35-38]. By continuing to engage with the Scenario Modeling Hub in weekly meetings, the CDC will be able to provide insight on which questions are of most interest for public health practice and preparedness during the 2023-2024 influenza season to help guide scenario development as well as lend expertise on data sets that may enhance modeling efforts. Results from the Influenza Scenario Modeling Hub may be used to inform potential peak timing and intensity. Additionally, influenza modelers within the CDC will continue to engage with the broader infectious disease modeling community. Information on changing influenza epidemiology and surveillance systems will be regularly communicated across forecasting groups, modeling groups, and data curators. This will be achieved in part through coordinated efforts with partners in the Models of Infectious Disease Agent Study Network [39].

Conclusion

The COVID-19 pandemic has changed many things that could influence trends observed in long-standing influenza surveillance systems. These changes have impacted ongoing efforts to describe influenza disease burden and forecast influenza activity and introduced new biases into epidemiologic and clinical studies that rely on the results of clinical testing. However, the pandemic has also provided opportunities for innovation that could better inform and support influenza efforts moving forward, including novel data sources such as the NHSN hospital admission data set, the ability to compare the impact of multiple respiratory pathogens in different clinical settings, and increased engagement with the public and external partners. Prepandemic approaches used to inform influenza prevention and control efforts, such as surveillance, burden estimation, and modeling, will continue to be reevaluated to ensure their reliability. State-level data may be used in the future to provide more spatially resolved estimates of burden beyond the national burden estimates that are currently provided. The US CDC Influenza Division remains committed to these key data and analytic activities, including the simultaneous monitoring of syndromic influenza surveillance with complimentary data sources, as well as providing a central platform for collaboration and communication across disciplines and institutions to ensure that information to inform influenza prevention, situational awareness, and control remains robust and readily available.

Abbreviations

CDC

Centers for Disease Control and Prevention

HHS

Health & Human Services

NHSN

National Healthcare Safety Network

RSV

respiratory syncytial virus

Disclaimer

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

Footnotes

Conflicts of Interest: None declared.

References

  • 1.Olsen SJ, Winn AK, Budd AP, Prill MM, Steel J, Midgley CM, Kniss K, Burns E, Rowe T, Foust A, Jasso G, Merced-Morales A, Davis CT, Jang Y, Jones J, Daly P, Gubareva L, Barnes J, Kondor R, Sessions W, Smith C, Wentworth DE, Garg S, Havers FP, Fry AM, Hall AJ, Brammer L, Silk BJ. Changes in influenza and other respiratory virus activity during the COVID-19 pandemic—United States, 2020-2021. MMWR Morb Mortal Wkly Rep. 2021;70(29):1013–1019. doi: 10.15585/mmwr.mm7029a1. doi: 10.15585/mmwr.mm7029a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Merced-Morales A, Daly P, Abd Elal AI, Ajayi N, Annan E, Budd A, Barnes J, Colon A, Cummings CN, Iuliano AD, DaSilva J, Dempster N, Garg S, Gubareva L, Hawkins D, Howa A, Huang S, Kirby M, Kniss K, Kondor R, Liddell J, Moon S, Nguyen HT, O'Halloran A, Smith C, Stark T, Tastad K, Ujamaa D, Wentworth DE, Fry AM, Dugan VG, Brammer L. Influenza activity and composition of the 2022-23 influenza vaccine—United States, 2021-22 season. MMWR Morb Mortal Wkly Rep. 2022;71(29):913–919. doi: 10.15585/mmwr.mm7129a1. doi: 10.15585/mmwr.mm7129a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Past seasons estimated influenza disease burden. Centers for Disease Control and Prevention. [2023-08-11]. https://www.cdc.gov/flu/about/burden/past-seasons.html .
  • 4.2022-2023 U.S. Flu season: preliminary in-season burden estimates. Centers for Disease Control and Prevention. [2023-08-11]. https://www.cdc.gov/flu/about/burden/preliminary-in-season-estimates.htm .
  • 5.FluNet. Global influenza programme. World Health Organization. [2023-07-25]. https://www.who.int/tools/flunet .
  • 6.Weekly U.S. influenza surveillance report. Centers for Disease Control and Prevention. [2022-09-06]. https://www.cdc.gov/flu/weekly/index.htm .
  • 7.Early wave of flu brings early flu hospitalizations. Centers for Disease Control and Prevention. [2022-10-31]. https://www.cdc.gov/flu/spotlights/2022-2023/early-wave-hospitalizations.htm .
  • 8.Thomas CM, White EB, Kojima N, Fill MA, Hanna S, Jones TF, Newhouse CN, Orejuela K, Roth E, Winders S, Chandler DR, Grijalva CG, Schaffner W, Schmitz JE, DaSilva J, Kirby MK, Mellis AM, Rolfes MA, Sumner KM, Flannery B, Talbot HK, Dunn JR. Early and increased influenza activity among children—Tennessee, 2022-23 influenza season. MMWR Morb Mortal Wkly Rep. 2023;72(3):49–54. doi: 10.15585/mmwr.mm7203a1. doi: 10.15585/mmwr.mm7203a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Influenza hospitalization surveillance network (FluSurv-NET) Centers for Disease Control and Prevention. [2022-12-12]. https://www.cdc.gov/flu/weekly/influenza-hospitalization-surveillance.htm .
  • 10.FluView Interactive. Centers for Disease Control and Prevention. [2023-08-10]. https://www.cdc.gov/flu/weekly/fluviewinteractive.htm .
  • 11.Increased respiratory virus activity especially among children, early in the 2022-2023 fall and winter. Centers for Disease Control and Prevention. 2022. [2024-03-05]. https://emergency.cdc.gov/han/2022/han00479.asp .
  • 12.U.S. influenza surveillance: purpose and methods. Centers for Disease Control and Prevention. [2022-12-12]. https://www.cdc.gov/flu/weekly/overview.htm .
  • 13.COVID-19 reported patient impact and hospital capacity by state timeseries (RAW) HealthData.gov. [2022-12-12]. https://healthdata.gov/Hospital/COVID-19-Reported-Patient-Impact-and-Hospital-Capa/g62h-syeh .
  • 14.Respiratory virus laboratory emergency department network surveillance (RESP-LENS) Centers for Disease Control and Prevention. [2023-08-11]. https://www.cdc.gov/surveillance/resp-lens/dashboard.html .
  • 15.Respiratory virus hospitalization surveillance network (RESP-NET) Centers for Disease Control and Prevention. [2023-08-11]. https://www.cdc.gov/surveillance/resp-net/dashboard.html .
  • 16.Rolfes MA, Foppa IM, Garg S, Flannery B, Brammer L, Singleton JA, Burns E, Jernigan D, Olsen SJ, Bresee J, Reed C. Annual estimates of the burden of seasonal influenza in the United States: a tool for strengthening influenza surveillance and preparedness. Influenza Other Respir Viruses. 2018;12(1):132–137. doi: 10.1111/irv.12486. https://europepmc.org/abstract/MED/29446233 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Why CDC estimates the burden of season influenza in the U.S. Centers for Disease Control and Prevention. [2022-09-06]. https://www.cdc.gov/flu/about/burden/why-cdc-estimates.htm .
  • 18.Rader B, Gertz A, Iuliano AD, Gilmer M, Wronski L, Astley CM, Sewalk K, Varrelman TJ, Cohen J, Parikh R, Reese HE, Reed C, Brownstein JS. Use of at-home COVID-19 tests—United States, august 23, 2021-march 12, 2022. MMWR Morb Mortal Wkly Rep. 2022;71(13):489–494. doi: 10.15585/mmwr.mm7113e1. doi: 10.15585/mmwr.mm7113e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Reese H, Iuliano AD, Patel NN, Garg S, Kim L, Silk BJ, Hall AJ, Fry A, Reed C. Estimated incidence of coronavirus disease 2019 (COVID-19) illness and hospitalization-United States, february-september 2020. Clin Infect Dis. 2021;72(12):e1010–e1017. doi: 10.1093/cid/ciaa1780. https://europepmc.org/abstract/MED/33237993 .6000389 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Lutz CS, Huynh MP, Schroeder M, Anyatonwu S, Dahlgren FS, Danyluk G, Fernandez D, Greene SK, Kipshidze N, Liu L, Mgbere O, McHugh LA, Myers JF, Siniscalchi A, Sullivan AD, West N, Johansson MA, Biggerstaff M. Applying infectious disease forecasting to public health: a path forward using influenza forecasting examples. BMC Public Health. 2019;19(1):1659. doi: 10.1186/s12889-019-7966-8. https://bmcpublichealth.biomedcentral.com/articles/10.1186/s12889-019-7966-8 .10.1186/s12889-019-7966-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.McGowan CJ, Biggerstaff M, Johansson M, Apfeldorf KM, Ben-Nun M, Brooks L, Convertino M, Erraguntla M, Farrow DC, Freeze J, Ghosh S, Hyun S, Kandula S, Lega J, Liu Y, Michaud N, Morita H, Niemi J, Ramakrishnan N, Ray EL, Reich NG, Riley P, Shaman J, Tibshirani R, Vespignani A, Zhang Q, Reed C. Collaborative efforts to forecast seasonal influenza in the United States, 2015-2016. Sci Rep. 2019;9(1):683. doi: 10.1038/s41598-018-36361-9. doi: 10.1038/s41598-018-36361-9.10.1038/s41598-018-36361-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Reich NG, McGowan CJ, Yamana TK, Tushar A, Ray EL, Osthus D, Kandula S, Brooks LC, Crawford-Crudell W, Gibson GC, Moore E, Silva R, Biggerstaff M, Johansson MA, Rosenfeld R, Shaman J. Accuracy of real-time multi-model ensemble forecasts for seasonal influenza in the U.S. PLoS Comput Biol. 2019;15(11):e1007486. doi: 10.1371/journal.pcbi.1007486. https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007486 .PCOMPBIOL-D-19-01044 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Reich NG, Brooks LC, Fox SJ, Kandula S, McGowan CJ, Moore E, Osthus D, Ray EL, Tushar A, Yamana TK, Biggerstaff M, Johansson MA, Rosenfeld R, Shaman J. A collaborative multiyear, multimodel assessment of seasonal influenza forecasting in the United States. Proc Natl Acad Sci U S A. 2019;116(8):3146–3154. doi: 10.1073/pnas.1812594116. https://www.pnas.org/doi/full/10.1073/pnas.1812594116 .1812594116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Cramer EY, Ray EL, Lopez VK, Bracher J, Brennen A, Castro Rivadeneira AJ, Gerding A, Gneiting T, House KH, Huang Y, Jayawardena D, Kanji AH, Khandelwal A, Le K, Mühlemann A, Niemi J, Shah A, Stark A, Wang Y, Wattanachit N, Zorn MW, Gu Y, Jain S, Bannur N, Deva A, Kulkarni M, Merugu S, Raval A, Shingi S, Tiwari A, White J, Abernethy NF, Woody S, Dahan M, Fox S, Gaither K, Lachmann M, Meyers LA, Scott JG, Tec M, Srivastava A, George GE, Cegan JC, Dettwiller ID, England WP, Farthing MW, Hunter RH, Lafferty B, Linkov I, Mayo ML, Parno MD, Rowland MA, Trump BD, Zhang-James Y, Chen S, Faraone SV, Hess J, Morley CP, Salekin A, Wang D, Corsetti SM, Baer TM, Eisenberg MC, Falb K, Huang Y, Martin ET, McCauley E, Myers RL, Schwarz T, Sheldon D, Gibson GC, Yu R, Gao L, Ma Y, Wu D, Yan X, Jin X, Wang YX, Chen YQ, Guo L, Zhao Y, Gu Q, Chen J, Wang L, Xu P, Zhang W, Zou D, Biegel H, Lega J, McConnell S, Nagraj VP, Guertin SL, Hulme-Lowe C, Turner SD, Shi Y, Ban X, Walraven R, Hong QJ, Kong S, van de Walle A, Turtle JA, Ben-Nun M, Riley S, Riley P, Koyluoglu U, DesRoches D, Forli P, Hamory B, Kyriakides C, Leis H, Milliken J, Moloney M, Morgan J, Nirgudkar N, Ozcan G, Piwonka N, Ravi M, Schrader C, Shakhnovich E, Siegel D, Spatz R, Stiefeling C, Wilkinson B, Wong A, Cavany S, España G, Moore S, Oidtman R, Perkins A, Kraus D, Kraus A, Gao Z, Bian J, Cao W, Ferres JL, Li C, Liu TY, Xie X, Zhang S, Zheng S, Vespignani A, Chinazzi M, Davis JT, Mu K, Piontti APY, Xiong X, Zheng A, Baek J, Farias V, Georgescu A, Levi R, Sinha D, Wilde J, Perakis G, Bennouna MA, Nze-Ndong D, Singhvi D, Spantidakis I, Thayaparan L, Tsiourvas A, Sarker A, Jadbabaie A, Shah D, Penna ND, Celi LA, Sundar S, Wolfinger R, Osthus D, Castro L, Fairchild G, Michaud I, Karlen D, Kinsey M, Mullany LC, Rainwater-Lovett K, Shin L, Tallaksen K, Wilson S, Lee EC, Dent J, Grantz KH, Hill AL, Kaminsky J, Kaminsky K, Keegan LT, Lauer SA, Lemaitre JC, Lessler J, Meredith HR, Perez-Saez J, Shah S, Smith CP, Truelove SA, Wills J, Marshall M, Gardner L, Nixon K, Burant JC, Wang L, Gao L, Gu Z, Kim M, Li X, Wang G, Wang Y, Yu S, Reiner RC, Barber R, Gakidou E, Hay SI, Lim S, Murray C, Pigott D, Gurung HL, Baccam P, Stage SA, Suchoski BT, Prakash BA, Adhikari B, Cui J, Rodríguez A, Tabassum A, Xie J, Keskinocak P, Asplund J, Baxter A, Oruc BE, Serban N, Arik SO, Dusenberry M, Epshteyn A, Kanal E, Le LT, Li CL, Pfister T, Sava D, Sinha R, Tsai T, Yoder N, Yoon J, Zhang L, Abbott S, Bosse NI, Funk S, Hellewell J, Meakin SR, Sherratt K, Zhou M, Kalantari R, Yamana TK, Pei S, Shaman J, Li ML, Bertsimas D, Lami OS, Soni S, Bouardi HT, Ayer T, Adee M, Chhatwal J, Dalgic OO, Ladd MA, Linas BP, Mueller P, Xiao J, Wang Y, Wang Q, Xie S, Zeng D, Green A, Bien J, Brooks L, Hu AJ, Jahja M, McDonald D, Narasimhan B, Politsch C, Rajanala S, Rumack A, Simon N, Tibshirani RJ, Tibshirani R, Ventura V, Wasserman L, O'Dea EB, Drake JM, Pagano R, Tran QT, Ho LST, Huynh H, Walker JW, Slayton RB, Johansson MA, Biggerstaff M, Reich NG. Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the United States. Proc Natl Acad Sci U S A. 2022;119(15):e2113561119. doi: 10.1073/pnas.2113561119. https://www.pnas.org/doi/abs/10.1073/pnas.2113561119?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cramer EY, Huang Y, Wang Y, Ray EL, Cornell M, Bracher J, Brennen A, Rivadeneira AJC, Gerding A, House K, Jayawardena D, Kanji AH, Khandelwal A, Le K, Mody V, Mody V, Niemi J, Stark A, Shah A, Wattanchit N, Zorn MW, Reich NG. The United States COVID-19 forecast hub dataset. Sci Data. 2022;9(1):462. doi: 10.1038/s41597-022-01517-w. doi: 10.1038/s41597-022-01517-w.10.1038/s41597-022-01517-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Reich NG, Lessler J, Funk S, Viboud C, Vespignani A, Tibshirani RJ, Shea K, Schienle M, Runge MC, Rosenfeld R, Ray EL, Niehus R, Johnson HC, Johansson MA, Hochheiser H, Gardner L, Bracher J, Borchering RK, Biggerstaff M. Collaborative Hubs: making the most of predictive epidemic modeling. Am J Public Health. 2022;112(6):839–842. doi: 10.2105/AJPH.2022.306831. https://ajph.aphapublications.org/doi/full/10.2105/AJPH.2022.306831 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.FluSight: flu forecasting. Centers for Disease Control and Prevention. [2022-09-06]. https://www.cdc.gov/flu/weekly/flusight/index.html .
  • 28.Reich NG, Ray EL. Collaborative modeling key to improving outbreak response. Proc Natl Acad Sci U S A. 2022;119(14):e2200703119. doi: 10.1073/pnas.2200703119. https://www.pnas.org/doi/abs/10.1073/pnas.2200703119?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Fox SJ, Lachmann M, Tec M, Pasco R, Woody S, Du Z, Wang X, Ingle TA, Javan E, Dahan M, Gaither K, Escott ME, Adler SI, Johnston SC, Scott JG, Meyers LA. Real-time pandemic surveillance using hospital admissions and mobility data. Proc Natl Acad Sci U S A. 2022;119(7):e2111870119. doi: 10.1073/pnas.2111870119. https://www.pnas.org/doi/abs/10.1073/pnas.2111870119?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed .2111870119 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Mathis SM, Webber AE, León TM, Murray EL, Sun M, White LA, Brooks LC, Green A, Hu AJ, McDonald DJ, Rosenfeld R, Shemetov D, Tibshirani RJ, Kandula S, Pei S, Shaman J, Yaari R, Yamana TK, Agarwal P, Balusu S, Gururajan G, Kamarthi H, Prakash BA, Raman R, Rodríguez A, Zhao Z, Meiyappan A, Omar S, Baccam P, Gurung HL, Stage SA, Suchoski BT, Ajelli M, Kummer AG, Litvinova M, Ventura PC, Wadsworth S, Niemi J, Carcelen E, Hill AL, Jung SM, Lemaitre JC, Lessler J, Loo SL, McKee CD, Sato K, Smith C, Truelove S, McAndrew T, Ye W, Bosse N, Hlavacek WS, Lin YT, Mallela A, Chen Y, Lamm SM, Lee J, Posner RG, Perofsky AC, Viboud C, Clemente L, Lu F, Meyer AG, Santillana M, Chinazzi M, Davis JT, Mu K, Piontti APY, Vespignani A, Xiong X, Ben-Nun M, Riley P, Turtle J, Hulme-Lowe C, Jessa S, Nagraj VP, Turner SD, Williams D, Basu A, Drake JM, Fox SJ, Gibson GC, Suez E, Thommes EW, Cojocaru MG, Cramer EY, Gerding A, Stark A, Ray EL, Reich NG, Shandross L, Wattanachit N, Wang Y, Zorn MW, Al Aawar M, Srivastava A, Meyers LA, Adiga A, Hurt B, Kaur G, Lewis BL, Marathe M, Venkatramanan S, Butler P, Farabow A, Muralidhar N, Ramakrishnan N, Reed C, Biggerstaff M, Borchering RK. Evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations. medRxiv. Preprint posted online on December 11, 2023. 2023 doi: 10.1101/2023.12.08.23299726. https://europepmc.org/abstract/MED/38168429 .2023.12.08.23299726 [DOI] [Google Scholar]
  • 31.Flu Scenario Modeling Hub GitHub. [2023-08-11]. https://github.com/midas-network/flu-scenario-modeling-hub .
  • 32.Flu Scenario Modeling Hub. [2023-08-11]. https://fluscenariomodelinghub.org/
  • 33.Flu Scenario Modeling Hub Resources GitHub. [2023-08-11]. https://github.com/midas-network/flu-scenario-modeling-hub_resources .
  • 34.COVID-19 Scenario Modeling Hub GitHub. [2022-12-12]. https://github.com/midas-network/covid19-scenario-modeling-hub .
  • 35.Borchering RK, Viboud C, Howerton E, Smith CP, Truelove S, Runge MC, Reich NG, Contamin L, Levander J, Salerno J, van Panhuis W, Kinsey M, Tallaksen K, Obrecht RF, Asher L, Costello C, Kelbaugh M, Wilson S, Shin L, Gallagher ME, Mullany LC, Rainwater-Lovett K, Lemaitre JC, Dent J, Grantz KH, Kaminsky J, Lauer SA, Lee EC, Meredith HR, Perez-Saez J, Keegan LT, Karlen D, Chinazzi M, Davis JT, Mu K, Xiong X, Piontti APY, Vespignani A, Srivastava A, Porebski P, Venkatramanan S, Adiga A, Lewis B, Klahn B, Outten J, Schlitt J, Corbett P, Telionis PA, Wang L, Peddireddy AS, Hurt B, Chen J, Vullikanti A, Marathe M, Healy JM, Slayton RB, Biggerstaff M, Johansson MA, Shea K, Lessler J. Modeling of future COVID-19 cases, hospitalizations, and deaths, by vaccination rates and nonpharmaceutical intervention scenarios—United States, april-september 2021. MMWR Morb Mortal Wkly Rep. 2021;70(19):719–724. doi: 10.15585/mmwr.mm7019e3. doi: 10.15585/mmwr.mm7019e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Truelove S, Smith CP, Qin M, Mullany LC, Borchering RK, Lessler J, Shea K, Howerton E, Contamin L, Levander J, Kerr J, Hochheiser H, Kinsey M, Tallaksen K, Wilson S, Shin L, Rainwater-Lovett K, Lemaitre JC, Dent J, Kaminsky J, Lee EC, Perez-Saez J, Hill A, Karlen D, Chinazzi M, Davis JT, Mu K, Xiong X, Pastore y Piontti A, Vespignani A, Srivastava A, Porebski P, Venkatramanan S, Adiga A, Lewis B, Klahn B, Outten J, Orr M, Harrison G, Hurt B, Chen J, Vullikanti A, Marathe M, Hoops S, Bhattacharya P, Machi D, Chen S, Paul R, Janies D, Thill JC, Galanti M, Yamana TK, Pei S, Shaman JL, Healy JM, Slayton RB, Biggerstaff M, Johansson MA, Runge MC, Viboud C. Projected resurgence of COVID-19 in the United States in july-december 2021 resulting from the increased transmissibility of the Delta variant and faltering vaccination. medRxiv. Preprint posted online on September 2, 2021. 2022 doi: 10.7554/elife.73584.sa0. https://www.medrxiv.org/content/10.1101/2021.08.28.21262748v2 . [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Borchering RK, Mullany LC, Howerton E, Chinazzi M, Smith CP, Qin M, Reich NG, Contamin L, Levander J, Kerr J, Espino J, Hochheiser H, Lovett K, Kinsey M, Tallaksen K, Wilson S, Shin L, Lemaitre JC, Hulse JD, Kaminsky J, Lee EC, Hill AL, Davis JT, Mu K, Xiong X, Pastore Y Piontti A, Vespignani A, Srivastava A, Porebski P, Venkatramanan S, Adiga A, Lewis B, Klahn B, Outten J, Hurt B, Chen J, Mortveit H, Wilson A, Marathe M, Hoops S, Bhattacharya P, Machi D, Chen S, Paul R, Janies D, Thill JC, Galanti M, Yamana T, Pei S, Shaman J, España G, Cavany S, Moore S, Perkins A, Healy JM, Slayton RB, Johansson MA, Biggerstaff M, Shea K, Truelove SA, Runge MC, Viboud C, Lessler J. Impact of SARS-CoV-2 vaccination of children ages 5-11 years on COVID-19 disease burden and resilience to new variants in the United States, november 2021-march 2022: a multi-model study. Lancet Reg Health Am. 2023;17:100398. doi: 10.1016/j.lana.2022.100398. https://linkinghub.elsevier.com/retrieve/pii/S2667-193X(22)00215-0 .S2667-193X(22)00215-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Biggerstaff M, Slayton RB, Johansson MA, Butler JC. Improving pandemic response: employing mathematical modeling to confront coronavirus disease 2019. Clin Infect Dis. 2022;74(5):913–917. doi: 10.1093/cid/ciab673. https://europepmc.org/abstract/MED/34343282 .6338173 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Models of Infectious Disease Agent Study (MIDAS) [2022-12-12]. https://midasnetwork.us .

Articles from JMIR Public Health and Surveillance are provided here courtesy of JMIR Publications Inc.

RESOURCES