Abstract
Context
A highly skilled public health workforce is needed for responding to health threats, and that workforce must be able to communicate its scientific findings effectively.
Objective
We evaluated the scientific communication effectiveness of the Centers for Disease Control and Prevention’s (CDC’s) field-based Epidemic Intelligence Service officers (EISOs).
Design
A descriptive analysis of all scientific information products produced and submitted for institutional clearance by CDC’s field-based EISOs during 2009–2014.
Main Outcome Measure(s)
The number of abstracts, journal manuscripts, Morbidity and Mortality Weekly Reports (MMWRs), and other information products approved by CDC during 2009–2014; the number of those products published; and of those published, the number cited in the scientific literature.
Results
During 2009–2014, a total of 152 field-based EISOs produced 835 scientific information products, including 437 abstracts, 261 manuscripts, and 103 MMWRs. The majority of scientific information products submitted for clearance were abstracts (52.3%), and infectious diseases (75.3%) constituted the majority of topics. Among the 103 MMWRs and 261 manuscripts cleared, 88 (85%) and 199 (76%) were published, respectively, with the majority also infectious disease-related. The 199 published manuscripts were cited in the scientific literature 2415 times, and the 88 published MMWRs were cited 1249 times. Field-based EISOs published their work in 74 different peer-reviewed medical and public health journals, with 54% published in journals with impact factors of 1 to 5.
Conclusions
Field-based EISOs’ publications are a measurable marker that reflects proficiency in epidemiology, written communication, and professionalism, and those publications are a direct reflection of EISOs’ contribution to local and state health departments. Our study establishes a baseline for future evaluations of publication outcome of scientific information products by EISOs. Information released by EISOs provides health professionals with the scientific knowledge necessary for improving their practice and helps CDC achieve a broader societal, environmental, cultural, and economic impact.
Keywords: education, Epidemic Intelligence Service, journal impact factor, publishing
The capacity and readiness of the public health system to respond to threats is dependent on a knowledgeable, highly trained, and skilled public health workforce.1 A required competency of such a workforce is skill in oral and written communication, which includes conveying complex data and disseminating scientific information through multiple approaches (eg, manuscripts, presentations, newsletters, and social media) in audience-appropriate language.2,3 As the nation’s health protection agency, the Centers for Disease Control and Prevention (CDC) provides health information, messages, and recommendations that help protect populations against dangerous health threats. Integral with providing health information is another aspect of CDC’s mission—developing leaders and training the public health workforce, including the agency’s Epidemic Intelligence Service officers (EISOs), often known as disease detectives.
Established in 1951, the Epidemic Intelligence Service (EIS) Program is a unique 2-year postgraduate training program for health professionals interested in the practice of applied epidemiology.4 Approximately 80 health scientists are selected to participate in the EIS Program each year; in almost 65 years of program existence, more than 3000 EISOs have responded to requests for epidemiologic assistance within the United States and throughout the world. The EIS Program is modeled after a traditional medical residency program where much of the education occurs through hands-on assignments and mentoring;5 consequently, EISOs’ applied, competency-based training includes both classroom and on-the-job experience.
All EISOs are expected to attain prescribed competencies in professionalism, epidemiology, and scientific communication. Competencies are attained through fulfilling 11 basic core activities of learning (Box). As part of their training, EISOs are required to participate in and they often lead epidemiologic investigations, applied public health research, and evaluations of public health interventions or surveillance systems. More intensive epidemiologic-assistance investigations (Epi-Aids) are conducted in response to serious and urgent public health problems after CDC receives formal requests for rapid assistance from states, other federal agencies, international organizations, and other countries’ ministries of health.6 Findings from these investigations are often communicated by preparation and submission of conference abstracts, reports of urgent public health situations for CDC’s Morbidity and Mortality Weekly Report (MMWR), journal manuscripts, and oral or poster presentations for scientific audiences. All such information products are required to undergo rigorous scientific review before dissemination or submission to a publisher for consideration. At CDC, this process is referred to as clearance.7
BOX. Core Activities of Learning Required of the Centers for Disease Control and Prevention’s Epidemic Intelligence Service (EIS) Officers.
| Core Activities of Learning |
|---|
The EIS Program’s competency-based education incorporates prescribed Core Activities of Learning (CALs). All EIS officers must complete all 11 CALs, which enable them to develop proficiency as skilled epidemiologists who can effectively address public health challenges.
|
From Centers for Disease Control and Prevention.5
Although the majority of EISOs are assigned to positions at CDC headquarters’ locations, approximately 25% to 30% each year choose to complete their EIS training in state or local health departments or other public health organizations in the United States and its territories. These EISOs are referred to as field-based EISOs. Field assignments offer EISOs generalized experiences and mentorship with a nonfederal perspective and serve as an avenue for strengthening their assigned organization’s public health infrastructure by providing rapid, creative, and effective solutions to real-world public health problems.8,9 In contrast to the majority of EIS position at headquarters, field-based EISOs’ assignments tend to include diverse experiences across a broad array of public health topics. In addition, field-based EISOs constitute an important pipeline for epidemiologist working in state and local health departments.
To evaluate the effectiveness of field-based EISO training related to the core competency of scientific communication and the related core activities of learning, we conducted a study of the characteristics of scientific information products (eg, publications, abstracts, and CDC fact sheets) produced by field-based EISOs, timeliness of journal manuscripts and MMWRs, and the journal impact factor (IF)10 of published manuscripts.
Methods
We conducted a descriptive analysis of all scientific information products produced by field-based EISOs during January 1, 2009, to December 31, 2014, by assessing data captured in eClearance, CDC’s official electronic clearance system used for submission, review, and approval of information products. When scientific information products have completed CDC’s clearance process, they are considered cleared or approved to be disseminated (eg, submitted to a journal or presented at a conference). Scientific information products were categorized into 4 product types: abstracts, manuscripts, MMWRs, and other. The other category included book chapters, fact sheets, and all other science-related reports and material. We further categorized the products’ topic areas by infectious disease, chronic disease, environmental-related, injury-related, and other.
To identify the publication outcome of manuscripts and MMWRs, we accessed the National Library of Medicine’s database of biomedical literature (PubMed, available at http://www.ncbi.nlm.nih.gov/pubmed) to confirm publication and capture data regarding the publication date and journal. To assess the information product dissemination timeliness, the length of time between cleared status date and publication date was calculated; if the exact date of publication was unavailable, the 15th day of the specified month was used. To assess scientific information product dissemination timeliness for Epi-Aids conducted by field-based EISOs, we calculated the number of days from the Epi-Aid initiation date to publication date. Morbidity and Mortality Weekly Reports and manuscripts published before May 2015 were included in our study; if both the electronic publication and print dates were available, the earlier date was chosen.
To assess whether field-based EISOs were successful in publishing in their first-choice journal (ie, the intended journal specified by the author in eClearance) and to find the number of times each publication was cited in the scientific literature, we compared the publication journal with the intended journal and used Google Scholar (Google, Inc, Mountain View, California) and Scopus (Elsevier B.V., Amsterdam, the Netherlands) databases to search the number of times each publication was cited in the scientific literature. Publications were categorized by their respective journal IF, obtained from the 2014 Journal Citation Reports (2013 Journal Impact Factor Thomson Reuters, Rochester, New York).10 The IF of a journal is the average number of citations received per paper published in that journal during the 2 preceding years. Journal IFs were available only for journals that are indexed in the Journal Citation Reports for 2 or more years. All data were analyzed during January to March 2015 by using Microsoft Excel (Microsoft Corporation, Redmond, Washington) software. This project was reviewed by CDC for human subjects’ protection and was deemed to be nonresearch.
Results
During January 1, 2009, to December 31, 2014, a total of 152 field-based EISOs produced 835 scientific information products—approximately 140 per year (median, 143; range, 107–161), including 437 abstracts, 261 manuscripts, and 103 MMWRs (Table 1). As a result of their 2-year training, each field-based EISO produced an average of 4 abstracts (range, 1–9), 2 manuscripts (range, 1–6), and 1 MMWR (range, 0–3). The majority of scientific information products submitted for clearance were abstracts (52.3%); infectious diseases (75.3%) constituted the majority of topics. No difference occurred in the product type or variety of health topics submitted from year to year.
TABLE 1.
Scientific Products Authored by Field-Based Epidemic Intelligence Service Officers and Cleared by the Centers for Disease Control and Prevention, by Product Type—2009–2014a
| Abstracts | Manuscripts | MMWRs | Otherb | Total | |
|---|---|---|---|---|---|
| Health topic, no. (%) | |||||
| Infectious disease | 323 (73.9) | 195 (74.7) | 83 (80.6) | 28 (82.4) | 629 (75.3) |
| Chronic disease | 39 (8.9) | 21 (8.0) | 7 (6.8) | 2 (5.9) | 69 (8.3) |
| Environmental-related | 31 (7.1) | 14 (5.4) | 7 (6.8) | 1 (2.9) | 53 (6.3) |
| Injury-related | 22 (5.0) | 15 (5.7) | 5 (4.9) | 2 (5.9) | 44 (5.3) |
| Other subject matterc | 22 (5.0) | 16 (6.1) | 1 (1.0) | 1 (2.9) | 40 (4.8) |
| Total, no. (%) | 437 (52.3) | 261 (31.3) | 103 (12.3) | 34 (4.1) | 835 (100) |
Abbreviation: MMWR, Morbidity and Mortality Weekly Report.
Percentages might not total 100 because of rounding.
Other includes books, fact sheets, all other reports, interagency reports, educational materials, nonmanuscript epidemiologic assistance investigation-related reports, instructional science, lesson plans, training materials, presentations, and organizational media with science-related content.
Other subject matter includes emergency preparedness, public health workforce education and training, unexplained illnesses (eg, nodding syndrome, sudden infant death syndrome), and multiple topics (eg, patients with asthma hospitalized with influenza).
Among the 103 MMWRs and 261 manuscripts cleared during 2009 to 2014, 88 (85%) and 199 (76%) had been published, respectively. An average of 15 MMWRs and 33 manuscripts were published each year. The publication rate for information products cleared during the study period ranged between 75% in 2009 and 100% in 2012 for MMWRs, and between 23% in 2014 and 97% in 2009 for manuscripts. Reflecting the topics of scientific information products submitted for clearance, the majority of MMWRs (83.0%) and manuscripts (75.4%) published also were infectious disease-related. The median time from cleared status to publication was 57 days (mean, 76 days; range, 2–525 days) for MMWRs and 373 days (mean, 395 days; range, 57–1193 days) for manuscripts. Among the 88 MMWRs and 199 manuscripts published, 14 (15.9%) and 26 (13.1%), respectively, were based on Epi-Aids (Table 2). Among these, 71% of MMWRs were published within 1 year of the Epi-Aid event, and 56% of manuscripts were published within 2 years of the Epi-Aid event. The median time from the initiation of the Epi-Aids to publication was 207 days (mean, 284 days; range, 39–827 days) for MMWRs and 682 days (mean, 726 days; range, 336–1308 days) for manuscripts (Table 2).
TABLE 2.
Main Publications by the Centers for Disease Control and Prevention Field-Based Epidemic Intelligence Service Officers, by Product Type and Topic—PubMed, 2009–2014a
| MMWRs No. (%)(n = 88)b | Manuscripts No. (%)(n = 199)b | |
|---|---|---|
| Health topic | ||
| Infectious disease | 73 (83.0) | 150 (75.4) |
| Chronic disease | 6 (6.8) | 13 (6.5) |
| Environmental-related | 6 (6.8) | 10 (5.0) |
| Injury-related | 3 (3.4) | 13 (6.5) |
| Other subject matterc | 0 (0) | 13 (6.5) |
| Time from cleared status to publication, daysb | ||
| Mean | 76 | 395 |
| Median (range) | 57 (2–525) | 373 (57–1193) |
| Publications related to an Epi-Aid, no. (%) | 14 (15.9) | 26 (13.1) |
| Time from Epi-Aid initiation to publication, daysd | ||
| Mean | 284 | 726 |
| Median (range) | 207 (39–827) | 682 (336–1308) |
Abbreviations: Epi-Aid, epidemiologic assistance investigation; MMWR, Morbidity and Mortality Weekly Report.
Percentages might not total 100 because of rounding.
Total is based on published products.
Other subject matter includes emergency preparedness, public health workforce education and training, unexplained illnesses (eg, nodding syndrome, sudden infant death syndrome), and multiple topics (eg, patients with asthma hospitalized with influenza).
Numbers are based on the 14 MMWRs and 26 manuscripts related to Epi-Aids.
Field-based EISOs published 172 (86%) first-authored and 27 (14%) coauthored manuscripts; the 199 manuscripts published during the study period were cited 2415 times, approximately 12 times per manuscript (median, 5; range, 0–409), with 161 (81%) manuscripts cited at least once. The manuscript that was cited the most described the public health response to pandemic influenza A (H1N1) affecting a New York City school in 2009;11 it was cited 409 times. Among 59 manuscripts cited 10 or more times, as expected, the majority (80%) were related to infectious diseases. The majority of manuscripts published during 2014 (19 of 31; 61%) and 2015 (7 of 8; 88%) have not been cited yet. The 88 MMWRs published during the study period were cited 1249 times, approximately 14 times per MMWR (median, 6; range, 0–168). Four MMWRs were cited more than 100 times, all related to pandemic influenza A (pH1N1) (range, 121–168). The MMWR that was cited the most reported on an investigation of oseltamivir-resistant pH1N1 virus infection in 2 immunosuppressed patients in Seattle, Washington, 200912; it was cited 168 times.
Field-based EISOs published their work in 74 different peer-reviewed medical and public health journals and various state medical journals. Ninety-two (46%) of 199 manuscripts were published in the field-based EISOs’ first-choice journal. The majority of the manuscripts (54%) were published in journals with IFs of 1 to 5, which corresponds to the IF of the majority of public health journals; an additional 30 (15%) manuscripts were published in journals with IFs of 1 to less than 2; and 49 (25%) manuscripts were published in journals with IFs of 2 or more to 3. Three (1.5%) manuscripts were published in a journal with an IF of more than 20. Manuscripts were most commonly published in Clinical Infectious Diseases (10.1%; IF 9.4), Emerging Infectious Diseases (10.1%; IF 7.3), and Epidemiology & Infection (5.5%; IF 2.5), Journal of Public Health Management & Practice (4.5%; IF 0.84), Public Health Reports (3.5%; IF 1.6), and Infection Control & Hospital Epidemiology (3.5%; IF 3.9).
Discussion
Epidemic Intelligence Service officers are highly trained, highly motivated health professionals who, since 1951, have made a major impact on the nation’s health and have contributed substantially to public health literature for the scientific community.3,8 Our study provides evidence that, as a result of their work, field-based EISOs have contributed approximately 300 scientific publications since 2009. Although the vast majority of the publications focused on infectious diseases, reflecting the predominance of infectious disease-related field EIS assignments and acute nature of public health problems that frequently require their response, 25% or more of the publications targeted and advanced important areas associated with chronic diseases, environmental-related, or injury-related topics.
Scientific publications further the progress of public health science by reporting the latest public health research (eg, findings from recent outbreak investigations or meta-analyses), public health guidelines and recommendations for clinical practice, and evaluations of existing surveillance reporting systems or interventions. Although journal publication is highly regarded among both the scientific and academic communities, with a lag of approximately 2 years for manuscripts to be published after an Epi-Aid occurs, it might not be the optimal method for informing decision makers when urgent public health action is needed; instead, journal manuscripts are more appropriate for communicating new foundational knowledge and lessons learned, and while MMWRs are more timely mechanism, even they take approximately 6 months to be published. However, for guiding public health action, CDC and public health departments use alternative methods for rapidly disseminating urgently needed information, including public broadcasting, Web-based information and social media, and CDC’s Epi-X system.13 In addition, MMWR publishes urgent reports through their Early Release mechanism, which allows for a more rapid dissemination of information regarding ongoing public health emergencies. All of these outlets foster a coordinated response to emergent public health events and facilitate effective interventions and accurate communication without having to wait for more formal publication.
Scientific publications included in this study were cited approximately 2500 times, with the majority of the manuscripts published in journals with IFs of 1 to 5. The most common publication venues for field-based EISOs did not include PLoS, the most common journal in which CDC authors published during 2011–2013.14 Although publication in scientific journals is highly regarded in public health and can provide a metric by which to measure how widely a product is disseminated, that is not a direct measure of its actual public health impact. Rarely can a single study or publication effect change independently or provide information regarding how scientific findings were used to improve public health practice or policy.
To measure the public health impact of its science from across the agency, CDC has implemented a framework that links dissemination of agency science with influence on subsequent events or actions that ultimately lead to population health improvements.15 This method of tracing the influence from a public health activity to public health impact begins with a point of scientific significance (eg, a manuscript publication by an field-based EISO) and then identifies forward or backward links from the significant event. Health outcomes, the ultimate goal, are impacted by 5 layers of scientific influence: disseminating science (eg, scientific publications), creating awareness (eg, continuing education or electronic communication), catalyzing action (eg, technology creation or congressional hearings), effecting change (eg, building public health capacity or creation of registries and surveillance), and shaping the future (eg, continuous quality improvement and implementation of public health initiatives); these degrees of impact might not necessarily be chronological and can be used in any order. Centers for Disease Control and Prevention’s science impact framework has changed how we conduct performance evaluations—moving away from an emphasis on the number of publications and citations toward the real-world impact of CDC’s work. Through the science impact framework lens, CDC programs can envision how to increase their effectiveness in improving the population’s health.15 Our study is a first step to monitoring not only the impact of field-based EISOs’ single publications but also the broader impact of science primarily through citations by capturing qualitative data that measure the changes occurring as a result of the science published.
Our study has some main limitations. First, although we report on a substantial number of scientific information products contributed by EISOs, these numbers are an underestimate, given that the time between a journal’s acceptance of a manuscript and its publication ranges from months to years; hence, manuscripts pending publication (ie, those in press or awaiting acceptance decisions, particularly those cleared during 2014) were unaccounted for in our analysis underestimating the publication and citation success and skewing the average time to publication to shorter time frames. A future analysis would offer a more accurate reflection of the timeliness of publication and the influence in terms of number of citations. In addition, submissions as a result of investigations of the Ebola outbreak in 2014 were not included in this study as it followed a distinct clearance process, which underestimates the numbers of the publications for 2014. Second, our study was limited to field-based EISOs; because they represent only approximately 20% of each EIS class, further study is needed to identify scientific publications by all EISOs. Third, the impact of publications based on Epi-Aids published by all EISOs should be evaluated as well as the impact of those Epi-Aids on public health practice by using CDC’s science impact framework. Finally, we did not have information of the date of Epi-Aid completion and were not able to calculate the time from Epi-Aid completion to publication, which would have been a more accurate marker to measure the time it takes for an field-based EISO to publish their work based on Epi-Aids.
Addressing public health problems requires keeping up with the best available scientific evidence to guide practice. Field-based EISOs have positively affected public health practice in the United States and their publications are a measurable marker of productivity that reflects proficiency in epidemiology, written communication, and professionalism—core competencies of the EIS program,9 and are a direct reflection of their contribution to local and state health departments. Our study establishes a baseline for future evaluations of publication outcome of scientific information products by EISOs. Information released by EISOs and CDC provides health professionals with the scientific knowledge necessary for improving their practice, increases the public’s awareness of important public health topics, and helps CDC achieve a broader societal, environmental, cultural, and economic impact.
Acknowledgments
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. No financial disclosures were reported by the authors of this paper.
References
- 1.Lichtveld MY, Cioffi JP. Public health workforce development: progress, challenges, and opportunities. J Public Health Manag Pract. 2003;9(6):443–450. doi: 10.1097/00124784-200311000-00003. [DOI] [PubMed] [Google Scholar]
- 2.The Council on Linkages Between Academia and Public Health Practice. Core Competencies for Public Health Professionals. Washington, DC: Public Health Foundation; 2014. [Accessed May 1, 2015]. Competency 3: communication skills. http://www.phf.org/resourcestools/Documents/Core_Competencies_for_Public_Health_Professionals_2014June.pdf. [Google Scholar]
- 3.Birkhead GS, Davies J, Miner K, Lemmings J, Koo D. Developing Competencies for applied epidemiology: from process to product. Public Health Rep. 2008;123(suppl 1):67–76. doi: 10.1177/00333549081230S114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Centers for Disease Control and Prevention. Epidemic Intelligence Service. Atlanta, GA: US Department of Health & Human Services, CDC; 2015. [Accessed May 1, 2015]. http://www.cdc.gov/eis/about.html. [Google Scholar]
- 5.Centers for Disease Control and Prevention. Epidemic Intelligence Service: Service and Learning. Atlanta, GA: US Department of Health & Human Services, CDC; 2015. [Accessed May 1, 2015]. http://www.cdc.gov/eis/service-learning.html. [Google Scholar]
- 6.Thacker SB, Stroup DF, Sencer DJ. Epidemic assistance by the Centers for Disease Control and Prevention: role of the Epidemic Intelligence Service, 1946–2005. Am J Epidemiol. 2011;174(11 suppl):S4–S15. doi: 10.1093/aje/kwr307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Centers for Disease Control and Prevention (CDC)/Office of the Chief Science Officer. Clearance of Information Products Disseminated Outside CDC for Public use. Atlanta, GA: US Department of Health & Human Services, CDC; 2005. [Accessed May 1, 2015]. http://www.cdc.gov/maso/Policy/PublicUse.pdf. [Google Scholar]
- 8.Thacker SB, Dannenberg AL, Hamilton DH. Epidemic Intelligence Service of the Centers for Disease Control and Prevention: 50 years of training and service in applied epidemiology. Am J Epidemiol. 2001;154(11):985–992. doi: 10.1093/aje/154.11.985. [DOI] [PubMed] [Google Scholar]
- 9.Moolenaar RL, Thacker SB. Evaluation of field training in the Epidemic Intelligence Service—publications and job choices. Am J Prev Med. 2004;26(4):299–306. doi: 10.1016/j.amepre.2003.12.014. [DOI] [PubMed] [Google Scholar]
- 10.Thomason Reuters Web of Science. Journal Citation Reports®. Rochester, NY: Thomason Reuters; [Accessed May 1, 2015]. [undated]. http://thomsonreuters.com/journal-citation-reports/ [Google Scholar]
- 11.Lessler J, Reich NG, Cummings DA. Outbreak of 2009 pandemic influenza A (H1N1) at a New York City school. New Engl J Med. 2009;361(27):2628–2636. doi: 10.1056/NEJMoa0906089. [DOI] [PubMed] [Google Scholar]
- 12.Centers for Disease Control and Prevention. Oseltamivir-resistant novel influenza A (H1N1) virus infection in two immunosuppressed patients—Seattle, Washington, 2009. Morb Mortal Wkly Rep. 2009;58(32):893. [PubMed] [Google Scholar]
- 13.Centers for Disease Control and Prevention. Epi-X: The Epidemic Information Exchange. Atlanta, GA: US Department of Health & Human Services, CDC; [Accessed May 1, 2015]. [undated]. http://www.cdc.gov/epix/ [Google Scholar]
- 14.Iskander J, Bang G, Stupp E, Connick K, Gomez O, Gidudu J. Articles published and downloaded by public health scientists: analysis of data from the CDC public health library, 2011–2013. J Public Health Manag Pract. doi: 10.1097/PHH.0000000000000277. published online ahead of print June 4, 2015. [DOI] [PubMed] [Google Scholar]
- 15.Centers for Disease Control and Prevention. Advancing Excellence & Integrity of CDC Science. Atlanta, GA: US Department of Health & Human Services, CDC; 2014. [Accessed May 1, 2015]. http://www.cdc.gov/od/science/impact/ [Google Scholar]