COVID-19 Scientific Publications From the Centers for Disease Control and Prevention, January 2020–January 2022

Abstract

Objective:

High-quality scientific evidence underpins public health decision making. The Centers for Disease Control and Prevention (CDC) agency provides scientific data, including during public health emergencies. To understand CDC’s contributions to COVID-19 science, we conducted a bibliometric evaluation of publications authored by CDC scientists from January 20, 2020, through January 20, 2022, by using a quality improvement approach (SQUIRE 2.0).

Methods:

We catalogued COVID-19 articles with ≥1 CDC-affiliated author published in a scientific journal and indexed in the World Health Organization’s COVID-19 database. We identified priority topic areas from the agency’s COVID-19 Public Health Science Agenda by using keyword scripts in EndNote and then assessed the impact of the published articles by using Scopus and Altmetric.

Results:

During the first 2 years of the agency’s pandemic response, CDC authors contributed to 1044 unique COVID-19 scientific publications in 208 journals. Publication topics included testing (n = 853, 82%); prevention strategies (n = 658, 63%); natural history, transmission, breakthrough infections, and reinfections (n = 587, 56%); vaccines (n = 567, 54%); health equity (n = 308, 30%); variants (n = 232, 22%); and post–COVID-19 conditions (n = 44, 4%). Publications were cited 40 427 times and received 81 921 news reports and 1 058 893 social media impressions. As the pandemic evolved, CDC adapted to address new scientific questions, including vaccine effectiveness, safety, and access; viral variants, including Delta and Omicron; and health equity.

Conclusion:

The agency’s COVID-19 Public Health Science Agenda helped guide impactful scientific activities. CDC continues to evaluate COVID-19 priority topic areas and contribute to development of new scientific work. CDC is committed to monitoring emerging issues and addressing gaps in evidence needed to improve health.

High-quality scientific evidence is critical to support public health decision making, particularly in dynamic situations in which key information may be unavailable or subject to a high degree of uncertainty. ¹ As part of the US Department of Health and Human Services, the Centers for Disease Control and Prevention (CDC) conducts science, provides information to protect the nation against health threats, and responds to public health emergencies when they arise.^2,3 Since the agency’s founding in 1946, more than 50 000 people have worked at CDC to prevent disease and support communities and citizens to do the same. ⁴ In the past 2 decades alone, CDC has served an important role in preventing, detecting, and responding to major disasters and disease outbreaks across the globe, including those caused by severe acute respiratory syndrome (SARS; 2003), influenza A/H1N1 (2009), Ebola virus (2014), and Zika virus (2016). ⁵

The COVID-19 pandemic has required a substantial response by public health authorities at all levels. On January 20, 2020, CDC activated its Emergency Operations Center to support public health partners in responding to an initial outbreak of cases of a novel coronavirus, later named SARS-CoV-2 (the virus that causes COVID-19).^6,7 As of January 19, 2022, a total of 68 671 563 COVID-19 cases and 856 288 COVID-19 deaths had been reported in the United States. ⁸ As of the same date, 75.3% of the total US population had received ≥1 dose of COVID-19 vaccine during the 13 months after initiation of the US COVID-19 vaccination program on December 14, 2020. ⁸

COVID-19 continues to be a formidable and evolving global public health challenge, and CDC has been a major source of COVID-19 data and health information used by public health personnel, medical personnel, and policy decision makers globally. In the first 2 years of CDC’s COVID-19 pandemic emergency response, more than 8000 CDC personnel contributed. ⁹ CDC websites were viewed 3.7 billion times, staff responded to 1.6 million telephone calls and email inquiries, and the agency produced nearly 10 000 guidance documents for business and the public. ⁹

Description of the Program Being Evaluated

To help guide development of an evidence base for the public health response to COVID-19, CDC identified COVID-19 public health science priority topic areas and questions that, if addressed, could provide measurable improvements in public health outcomes; the agency disseminated these topics and questions in the online CDC Public Health Science Agenda for COVID-19. ¹⁰ After priority topics were developed, in December 2020, CDC established a formalized process for regular review and revision to solicit and incorporate input from the CDC COVID-19 Response staff and subject matter experts, other CDC programs, and external public health partners. To reflect the changing landscape of the pandemic, priorities are revised regularly for timeliness, actionability, and impact. CDC developed and released regular updates to the agenda from February through March 2021, June through August 2021, and October 2021 through January 2022. The most recent of these updates involved input from more than 20 groups, including CDC COVID-19 Response groups (eg, Chief Health Equity Office; Chief Medical Office; Chief Science Office; Data, Analytics, and Visualization Task Force; Epidemiology and Surveillance Task Force; Expansion of Screening and Diagnostics Task Force; Global Migration Task Force; Southwest Border and Migrant Health Task Force; Health Systems and Worker Safety Task Force; International Task Force; Joint Information Center; Laboratory Task Force; Pregnancy and Infants Linked Outcomes Team; State, Tribal, Local, and Territorial Support Task Force; Vaccine Task Force; Response Associate Director for Science); other CDC programs (eg, Office of Science, Center for Forecasting and Outbreak Analytics); and external health partners (eg, Association of Public Health Laboratories, Association of State and Territorial Health Officials, Council of State and Territorial Epidemiologists, National Association of County and City Health Officials). CDC incorporated more than 400 comments on the priority questions from October through December 2021. As of January 5, 2022, the agency’s COVID-19 Public Health Science Agenda had listed 14 public health science priority questions and key activities across 7 topic areas. ¹⁰

Purpose of the Evaluation

To advance the understanding of COVID-19, CDC provides timely data and evidence from fundamental public health activities, including domestic and international epidemiologic investigations, laboratory-based pathogen detection, disease surveillance and research, and other scientific work. ¹⁰ One important mechanism by which findings from these activities are disseminated by CDC scientists is through articles published in the scientific literature. Subject matter experts and other CDC senior staff routinely review manuscripts authored by CDC scientists for scientific quality through a rigorous multilevel process before publication. ¹¹ We conducted an evaluation to understand CDC’s contributions to new COVID-19–related science during the first 2 years of the agency’s pandemic emergency response (January 20, 2020, through January 20, 2022), including recording the number of scientific publications, documenting which priority topic areas were addressed, and evaluating the scientific impact of these publications.

Methods

We conducted a bibliometric evaluation of scientific publications authored by ≥1 CDC scientist and the impact of these publications by using a quality improvement approach (SQUIRE 2.0). ¹²

We catalogued articles with ≥1 CDC-affiliated author published in a scientific journal and indexed by January 20, 2022, in the World Health Organization’s COVID-19 database, which included articles published online ahead of print in peer-reviewed journals but not articles posted to preprint servers. ¹³ For each article, we used keyword scripts (Box) developed by a CDC library scientist to assign tags in EndNote (Clarivate Analytics), which allowed us to identify any of the 7 public health science priority topic areas addressed in the full text; publications could address multiple topic areas. We developed a searchable interface that listed these articles and metadata ¹⁴ ; results are available to the public online. ¹⁵

Box.

Keyword scripts used to identify 7 priority topic areas in searches of any field or PDF of COVID-19 scientific publications with authors from the Centers for Disease Control and Prevention, January 20, 2020–January 20, 2022.

1. Health equity: disparit* OR inequit* OR equity OR inequalit* OR equality OR discrimination* OR prejudice* OR racism OR xenophobia* OR racial bias* OR equitable* 2. Vaccines: vaccin* OR breakthrough* OR post-vacc* OR fully vacc* OR pfizer OR moderna OR johnson & johnson OR j&j OR janssen OR comirnaty OR spikevax OR booster OR primary series OR third dose OR fourth dose 3. Variants: variant OR variants OR voc OR b.1.617 OR mutation or mutant* OR omicron OR nowcast OR wastewater surveill* 4. Prevention strategies: prevention strateg* OR mitigat* OR non-pharmaceutical intervention* OR nonpharmaceutical intervention* OR NPI OR NPIs OR social distanc* OR mandate OR mask OR masking OR masks 5. Testing: test OR tested OR testing OR tests OR PCR OR RT-PCR OR self-test 6. Transmission: transmiss* OR transmitted OR asymptom* OR after vacc* OR post vacc* OR fully vacc* OR previously infect* OR previous infect* OR reinfect* OR breakthrough* OR break through* OR natural histor* OR post-infect* OR post infect* 7. Post–COVID-19 conditions: sequela* OR long covid OR long-covid OR long haul* OR post covid OR post-covid OR PASC OR pasc) OR linger

We assessed scientific impact by using Scopus (Elsevier BV), which identified the number of citations in other published scientific literature. We also assessed impact by reviewing the volume of news media reports and the number of social media impressions and by evaluating Altmetric attention scores, which are composite measures of online activity derived by an automated algorithm.^16,17

This activity was reviewed by CDC and was conducted consistent with applicable federal law and CDC policy (eg, Protection of human subjects, 45 CFR part 46 [2018]; Institutional review board, 21 CFR part 56 [1981]; Public health and welfare, research and investigations generally, 42 USC §241[d] [2018]; Public information; agency rules, opinions, orders, records, and proceedings: records maintained on individuals, 5 USC §552a [2012]; and Paperwork Reduction Act, 44 USC §3501 et seq [1995]). The CDC Human Research Protections Office issued a nonresearch determination for this secondary data analysis, which did not collect data from individuals.

Results

From January 20, 2020, through January 20, 2022, CDC authors contributed to 1044 unique scientific publications addressing the 7 public health science priority topic areas in the agency’s COVID-19 Public Health Science Agenda (Table). Topics included testing (n = 853 publications, 82%); prevention strategies (n = 658 publications, 63%); natural history, transmission, breakthrough infections, and reinfections (n = 587 publications, 56%); vaccines (n = 567 publications, 54%); health equity (n = 308 publications, 30%); variants (n = 232 publications, 22%); and post–COVID-19 conditions (n = 44 publications, 4%). The total was >100% because some publications addressed multiple topic areas.

Table.

COVID-19 scientific publications (N = 1044) with authors from the Centers for Disease Control and Prevention (CDC) by priority topic area and questions in the CDC COVID-19 Public Health Science Agenda, January 20, 2020–January 20, 2022 ^a

Priority topic area	Questions	No. (%) of publications b
Health equity	1. How can the public health community effectively identify and address health inequities to protect populations disproportionately affected by COVID-19?	308 (30)
Vaccines	2. What are the effectiveness and duration of protection afforded by COVID-19 primary series and booster vaccines? 3. What interventions, programs, and communication approaches are most effective at increasing equitable COVID-19 vaccination access and coverage? 4. What are the risks and benefits associated with COVID-19 primary series and booster vaccines?	567 (54)
Variants	5. How can the public health community effectively and efficiently enhance surveillance for known and emerging SARS-CoV-2 variants? 6. How do SARS-CoV-2 variants affect diagnostics, vaccine effectiveness, clinical outcomes, transmissibility, and public health interventions?	232 (22)
Prevention strategies	7. What effective prevention strategies and nonpharmaceutical interventions should be prioritized to reduce transmission of SARS-CoV-2 in various populations and settings, including schools and workplaces, particularly where vaccination coverage is low? 8. When should SARS-CoV-2 prevention strategies and nonpharmaceutical interventions in various populations and settings be adjusted, for example, based on vaccination coverage, variant prevalence, community transmission, or transitioning into an endemic disease control phase?	658 (63)
Testing	9. How effective are at-home/self-testing, rapid diagnostic testing, point-of-care testing, routine screening, and serial testing strategies on reducing outbreaks, reducing disease burden, detecting potential surges, evaluating criteria for reopening, and detecting reintroduction of SARS-CoV-2 into low transmission settings?	853 (82)
Natural history, transmission, breakthrough infections, and reinfections	10. How does the public health community effectively and efficiently enhance surveillance for SARS-CoV-2 reinfections, breakthrough infections, vaccination, and various health outcomes? 11. What factors best inform SARS-CoV-2 transmission dynamics and predict surges of community-level infection? 12. What are reliable immune correlates of protection from SARS-CoV-2 infection and accurate ways to measure this?	587 (56)
Post–COVID-19 conditions	13. How does the public health community effectively conduct epidemiologic research on post–COVID-19 conditions, overall and in various populations and settings? 14. What short- and long-term impacts from the COVID-19 pandemic are of the greatest public health importance, and what are the best ways to address them?	44 (4)

^aAdapted from CDC. ¹⁰

^bTotals are >100% because some publications addressed multiple topic areas.

The 1044 CDC-authored COVID-19 publications appeared in 208 scientific journals, most frequently Morbidity and Mortality Weekly Report (MMWR; n = 339 publications, 32%), which is published by CDC.^18,19 Additional journals in which articles were published frequently included Clinical Infectious Diseases (n = 71 publications, 7%); Journal of the American Medical Association journals (n = 62 publications, 6%); Emerging Infectious Diseases, also published by CDC (n = 61 publications, 6%); and PLOS journals (n = 26 publications, 2%). Publications included those led by CDC researchers and those that had CDC scientists as collaborators: of the 1044 publications, 775 (74%) had a first or senior author affiliated with CDC, 683 (65%) had a CDC-affiliated first author, and 549 (53%) had a CDC-affiliated senior author.

CDC addressed changing scientific priorities over time (Figure). For example, testing was the most common topic during all months of the pandemic; publications on this topic increased from a mean of 21 per month from January through November 2020 to a mean of 48 per month from December 2020 through January 2022, as SARS-CoV-2 testing increased in the United States. ²⁰ Prevention strategies were the second most common publication topic area early in the pandemic; however, publications on the topic of vaccines increased over publications on prevention strategies a few months after the initial COVID-19 vaccine rollout in the United States in December 2020.^21,22 As new SARS-CoV-2 variants of concern emerged, including Delta in mid-2021 and Omicron in late 2021,^8,23 the number of CDC publications on variants increased to a peak monthly total of 27 publications in September 2021, addressing questions on variant surveillance and transmissibility. The number of CDC publications on health equity also increased during the 2-year period, from a mean of 6 per month from January through December 2020 to a mean of 18 per month from January 2021 through January 2022.

Figure.

COVID-19 scientific publications from the Centers for Disease Control and Prevention (N = 1044), by priority topic areas addressed, January 20, 2020–January 20, 2022. ^a COVID-19 vaccine approval, December 2020. ^b Emergence of Delta variant in the United States, July 2021. ^c Emergence of Omicron variant in the United States, November 2021.

Overall, CDC-authored publications were cited 40 427 times in the scientific literature during the 2-year period; these publications were the topic of 81 921 news media reports and 1 058 893 social media impressions and received a combined total Altmetric attention score of 920 763. On average, the 339 articles published in MMWR yielded a higher number of citations and other attention than the 705 articles published in all other journals combined. The mean number of citations per article was 55 for MMWR and 30 for all other journals, the mean number of news media reports per article was 179 for MMWR and 30 for all other journals, the mean number of social media impressions per article was 2017 for MMWR and 529 for all other journals, and the mean Altmetric attention score per article was 1865 for MMWR and 406 for all other journals. Per Altmetric, we found that information from CDC COVID-19 publications was included in more than 510 policy documents from ≥20 countries.

Lessons Learned

Since its founding, CDC has supported the development of high-quality science oriented to improving public health outcomes and supporting public health decision making.^{24 -26} Consistent with this long-standing commitment, scientific articles from CDC-affiliated authors have been published to inform public health policy and practice in the United States and internationally throughout the COVID-19 pandemic.

Our evaluation indicated that, as the pandemic evolved, CDC adapted to address new scientific questions to best inform public health policy and practice. For example, studies on COVID-19 prevention strategies were prominent early in the pandemic as US jurisdictions increasingly implemented such strategies, including physical distancing and the use of face masks in indoor public areas, to help reduce the spread of COVID-19. Similarly, after COVID-19 vaccines were made available in late 2020, ²¹ the number of studies on COVID-19 vaccines increased as researchers began to address issues of vaccine effectiveness, safety, duration of protection, booster doses, and equitable vaccination access and coverage. ²² The number of studies on COVID-19 vaccines peaked in May 2021 when the Emergency Use Authorization for the Pfizer-BioNTech COVID-19 vaccine was expanded to include adolescents aged 12 to 15 years. ²⁷ Also, the number of studies on COVID-19 viral variants increased in summer 2021 and continued through the end of 2021, coinciding with the emergence of various SARS-CoV-2 variants of concern in the United States, including Delta (in July) and Omicron (in November).^8,23 During this time, increases also occurred in the number of studies on natural history, transmission, breakthrough infections, and reinfections; research on these topic areas was important to inform initial understanding of the transmissibility and severity of these novel variants relative to earlier variants. Also, an increase after 2020 in publications addressing health equity may have reflected a societal shift toward addressing racism as an obstacle to health equity. ²⁸

We found that publications with CDC-affiliated authors were highly cited in the scientific literature, reflecting the utility of these studies in the scientific community. The 1044 articles published during the first 2 years of the pandemic received an average of nearly 40 citations each, or more than 50 citations per day across all published articles. In addition, these publications garnered substantial Altmetric attention scores, which reflect the extent to which the findings from the studies have engaged key audiences, including the general public. Publications with CDC-affiliated authors received an average Altmetric attention score per article of 1865 for MMWR and 406 for all other journals. These levels exceeded accepted standards for well-performing articles; for example, an Altmetric score >30 typically places an article in the top 5% of all research outputs scored. Because we found that CDC COVID-19 publications were included in more than 510 policy documents from ≥20 countries, we suggest that these studies were also important for global health practices.

This evaluation had 4 limitations. First, we contend that lag time can be highly variable between the date when a manuscript is accepted for publication and its publication date and subsequent indexing. Completed manuscripts recently submitted or accepted for publication but not yet published in a journal or indexed would not have been captured in our evaluation. Second, because measures of online interest can include positive or negative commentary, some articles could have received high attention scores for reasons other than their public health or scientific merit. Third, articles published in MMWR may receive particularly high attention scores within a discrete time frame because the publication process of this journal emphasizes timeliness, and reports are readily accessible free of charge to the public and the scientific community. Fourth, because preprints (ie, manuscripts posted online before peer review) were not included in this evaluation, the effects of the CDC scientific publications included here may have been underestimated.

CDC is committed to ensuring that every person has the opportunity to live a healthy life.^24,25 We learned several lessons from this evaluation: (1) the agency’s COVID-19 Public Health Science Agenda served as a framework to help guide scientific activities; (2) scientific manuscripts that were developed and published to address the public health science priority topic areas became highly impactful, garnering both scientific impact and high attention scores; and (3) CDC has demonstrated an ongoing commitment to monitoring emerging issues and to remaining nimble in addressing gaps in evidence needed to improve health outcomes. Ongoing CDC activities include internal and external review and revision of the agency’s COVID-19 Public Health Science Agenda as part of the COVID-19 emergency response. ¹⁰ CDC also continues to evaluate and report COVID-19 articles and priority topic areas as CDC-affiliated authors contribute to the development and publication of new scientific work. ¹⁵ Data-driven strategies are essential to reduce disparities and to improve the health outcomes of people disproportionately affected by COVID-19.