Public reporting guidelines for outbreak data: Enabling accountability for effective outbreak response by developing standards for transparency and uniformity

Abstract

Objectives

There are few standards for what information about an infectious disease outbreak should be reported to the public and when. To address this problem, we undertook a consensus process to develop recommendations for what epidemiological information public health authorities should report to the public during an outbreak.

Study design

We conducted a Delphi study following the steps outlined in the ACcurate COnsensus Reporting Document (ACCORD) for health-related activities or research.

Methods

We assembled a steering committee of nine experts representing federal and state public health, academia, and international partners to develop a candidate list of reporting items. We then invited 45 experts, 35 of whom agreed to participate in a Delphi panel. Of those, 25 participated in voting in the first round, 25 in the second round, and 25 in the third round, demonstrating consistent engagement in the consensus-building process. The final stage of the Delphi process consisted of a hybrid consensus meeting to finalize the voting items.

Results

The Delphi process yielded nine core reporting items representing a minimum standard for public outbreak reporting: numbers of new confirmed cases, new hospital admissions, new deaths, cumulative confirmed cases, cumulative hospital admissions, and cumulative deaths, each reported weekly and at Administrative Level 1 (typically state or province), and stratified by sex, age group, and race/ethnicity.

Conclusions

This minimum reporting standard creates a strong framework for uniform sharing of outbreak information and promotes consistency of data between jurisdictions, enabling effective response by promoting access to information about an unfolding epidemic.

1. Introduction

Situation reports are the regular updates that jurisdictions issue to describe the latest developments in infectious disease outbreaks. Situation reports, as well as other forms of public reporting, serve as vital sources of information for public health professionals in other jurisdictions, elected leaders, researchers, media, and the public.^1,2 These reports often provide information that can guide the development and implementation of disease spread control measures.^1,2 During the early days of the COVID-19 outbreak, for example, the information published by each jurisdiction was critical for helping other communities increase situational awareness and better understand the nature and trajectory of the growing pandemic.^{3, 4, 5}

Existing frameworks, such as the International Health Regulations (IHR) (2005) and the European Center for Disease Prevention and Control's EpiPulse, contain mechanisms for information sharing, however these are primarily aimed at technical agencies and states.^6,7 Currently, no guidelines exist to assist jurisdictions in deciding what and when to report to the public.^8,9 This has resulted in substantial heterogeneity in the information available across jurisdictions for various outbreaks. More uniform reporting would allow more consistent and reliable information to be available to stakeholders, enhancing the ability of decision makers to prepare for and respond to outbreaks.

To address this gap in guidance, we undertook a consensus process to develop recommendations for what epidemiological information public health authorities should report during an outbreak, including the administrative level and frequency of reporting. The Outbreak Reporting Best Practices in Transparency (ORBIT) guidelines aim to set a minimum standard for publicly available information during an outbreak to augment data reporting consistency and transparency, with a focus on the United States. As part of this process, the research team also developed recommendations for types of outbreak situations where these guidelines would apply. However, we recognize the need for flexibility and professional judgment in these decisions.

As part of our efforts, we undertook a literature review to identify previous efforts to develop reporting guidelines. Although the research team identified several papers prioritizing epidemiological information, we did not find guidelines overlapping or conflicting with our project aim in the systematic review. To our knowledge, practitioners have only recently started developing standards for public reporting. The CORHA Principles and Practices for Healthcare Outbreak Response, for example, provide guidance to public health and healthcare practitioners on when and what to notify patients and the public when an outbreak occurs in a healthcare setting, including how to communicate information in a way that helps mitigate risk.¹⁰ Additional information about the companion systematic review is available at https://doi.org/10.1101/2024.05.22.24307752.

2. Methods

We followed the steps outlined in the ACcurate COnsensus Reporting Document (ACCORD) for health-related activities or research, as documented in the EQUATOR network database.¹¹ First, we assembled a steering committee of nine experts representing federal public health, state public health, academia, and international partners. Due to scheduling conflicts that precluded synchronous meetings, the principal investigator and research team met with steering committee members individually or in small groups to plan and finalize the project methodology.

The research team developed a preliminary list of candidate reporting items, which was informed by the literature review. Items proposed at this stage, for example, included confirmed, probable and suspected cases and deaths. We shared the draft list with steering committee members, who provided feedback regarding the relevance and wording of proposed items. Steering committee members also offered new candidate items for inclusion in the draft list and nominated individuals with relevant expertise to serve on the Delphi panel.

After finalizing the preliminary list of candidate items, the research team assembled a slate of 45 potential Delphi¹² panel members based on the recommendations of the steering committee and invited them to participate in the development of a consensus on reporting items. This panel included 45 experts from various jurisdictions, expertise and career stages, who were invited via email to participate in a four-phase Delphi panel. Of the 45 invitees, 35 agreed to participate (Table 1). Of the 35 participants, 23 were current or former public health practitioners with significant expertise leading public health outbreak response at the local, state, or national level (Table 2). The remaining 12 were global or academic experts in public health outbreak response.

Table 1.

Participant jurisdictions. This table provides a broad overview of the countries and regions represented in the Delphi panel. The United States jurisdiction was further broken down into U.S. Federal and the respective Health and Human Services (HSS) Regions. Of the 35 individuals that agreed to participate, 25 participated in each of the three rounds. The composition of participants that voted across the three rounds differed.

Jurisdictions		Number of individuals that agreed to participate
Australia		1
Hong Kong		1
South Africa		1
United States	HHS Region 1	2
	HHS Region 2	1
	HHS Region 3	1
	HHS Region 4	8
	HHS Region 5	2
	HHS Region 8	1
	HHS Region 9	1
	HHS Region 10	2
	U.S. Federal	4
United Kingdom		7
World Health Organization		1
Council of State and Territorial Epidemiologists		2
Total		35

Table 2.

Participant expertise. This table provides a broad overview of the range of expertise represented in the Delphi panel.

Expertise	Number of individuals that agreed to participate
Administrative Level 0 (Federal)	6
Administrative Level 1 (typically state or province)	9
Administrative Level 2 (Local)	6
Academic	11
International	1
National	2
Total	35

The research team sent each confirmed participant a spreadsheet with the candidate items. Participants were asked to assign each item on a scale of one to ten, with one being not important and ten being most important. Participants were instructed to vote only on the importance and practicality of each item, not the wording, and to remember that the list only represents the minimum set of reporting items. Of the 35 individuals that agreed to participate, 31 submitted a voting sheet and/or attended the consensus meeting.

After a one-month voting period, the research team calculated the mean score for each item. Items that received a score of at least eight were judged to have reached consensus, and therefore admitted to the final stage of candidacy. Similarly, items that received a mean score less than or equal to five were dropped. Items with a score between five and eight were sent to participants for a second round of voting. This process was repeated for a total of three rounds of voting (Fig. 1).

Fig. 1.

Delphi voting process. This figure provides an overview of the voting process at each Delphi stage.

Of the expert participants who agreed to participate in the project, 25 participated in voting in the first round. Twenty-five participants returned round two spreadsheets, and 25 returned round three spreadsheets, demonstrating consistent engagement in the consensus-building process. Across all three rounds, 30 unique individuals participated in voting.

Following the three rounds of emailed voting, a final consensus meeting was held to finalize the reporting items. All Delphi participants were invited to attend the hybrid meeting in January 2024. Fifteen participants attended in person and seven participated online. The meeting aimed to discuss the remaining items that asynchronous voting had not resolved. The agenda also included time for participants to suggest changes to the wording of each item and to develop justifications for why each reporting item is important.

Meeting participants also discussed the types of outbreaks where these recommendations would be applicable. To resolve uncertainty, they engaged in a 45-min discussion followed by a vote on the final wording of the recommendation. Items that received a score of at least eight were included in the final list. Therefore, the recommendations reflect the majority view, not the unanimous agreement of those present.

3. Results

This process resulted in a final list of nine reporting items to include in publicly available situation reports (Table 3):

• 1.The number of new confirmed cases registered during the previous reporting period. This item represents the number of new confirmed cases, as defined by the case definition (which should be published with the reporting items), reported to public health authorities since the last report. Although the diagnosis date is preferred, especially for epidemiological modeling and analytics, most jurisdictions will find that the reporting date is most feasible. Therefore, Delphi participants determine this to be an acceptable option and encourages readers to refer to Council of State and Territorial Epidemiologists (CSTE) guidance.¹³
• 2.The number of new hospital admissions related to the outbreak (among confirmed cases) registered during the previous reporting period. This item represents the number of new hospital admissions among confirmed cases, as defined by the case definition, reported to public health authorities since the last report.
• 3.The number of new deaths related to the outbreak (among confirmed cases) registered during the previous reporting period. This item represents the number of new deaths among confirmed cases, as defined by the case definition, reported to public health authorities since the last report.
• 4.The number of cumulative confirmed cases registered throughout each reporting period. This item represents the total number of confirmed cases, as defined by the case definition, reported to public health authorities since the start of the outbreak.
• 5.The number of cumulative hospital admissions related to the outbreak (among confirmed cases) registered throughout each reporting period. This item represents the total number of hospital admissions among confirmed cases, as defined by the case definition, reported to public health authorities since the start of the outbreak.
• 6.The number of cumulative deaths related to the outbreak (among confirmed cases) registered throughout each reporting period. This item represents the total number of deaths among confirmed cases, as defined by the case definition, reported to public health authorities since the start of the outbreak.
• 7.Establish weekly reporting. This item specifies that public health authorities should, at a minimum, report to the public weekly. This is a minimum recommendation; jurisdictions may exceed these standards at their discretion (e.g., report every day, every other day, etc.).
• 8.Establish Administrative Level 1 (typically state or province) reporting of the final reporting items. This item specifies that public health authorities should, at minimum, report these data to the public with state, province, or equivalent level geographic granularity.
• 9.Stratify reporting items by sex, age group, and race/ethnicity. This item specifies that public health authorities should stratify the reporting items by sex, age group, race/ethnicity, when doing so would not compromise privacy or confidentiality. Public health authorities should also use their discretion to prevent the misuse of stratified data and the potential for stigmatization. Where and when the data is available, public health authorities should refer to their jurisdiction's census bureau stratification for sex, age group, race/ethnicity.

Table 3.

Final reporting items. This table provides an overview of the final list of nine reporting items.

Item #	ORBIT Reporting Item
1	The number of new confirmed cases registered during the previous reporting period.
2	The number of new hospital admissions related to the outbreak (among confirmed cases) registered during the previous reporting period.
3	The number of new deaths related to the outbreak (among confirmed cases) registered during the previous reporting period.
4	The number of cumulative confirmed cases registered throughout each reporting period.
5	The number of cumulative hospital admissions related to the outbreak (among confirmed cases) registered throughout each reporting period.
6	The number of cumulative deaths related to the outbreak (among confirmed cases) registered throughout each reporting period.
7	Establish weekly reporting. This item specifies that public health authorities should, at a minimum, report to the public weekly.
8	Establish Administrative level 1 (typically state or province) reporting of the final reporting items.
9	Stratify reporting items by sex, age group, and race/ethnicity.

3.1. Deciding when to apply the ORBIT guidelines

Participants discussed and voted on the applicability of the ORBIT reporting guidelines. There was general agreement among participants that small, point-source outbreaks, such as an outbreak of foodborne illness in a restaurant, did not merit a public-facing situation report. On the other hand, participants expressed a reluctance to limit reporting to the most serious outbreaks, such as the novel coronavirus, in recognition that reliable reporting of more moderate outbreaks allows neighboring jurisdictions to take proactive preparedness and control measures based on early information.

Ultimately, participants reached consensus that an outbreak that necessitates reporting under the International Health Regulations (IHR) (2005) should trigger use of the reporting guidelines. Participants also recommend that for other outbreaks, the decision to trigger the reporting guidelines falls to in-country public health authorities.

Under Article 6 of the IHR (2005), State Parties must report events constituting potential public health emergencies of international concern (PHEIC) to the World Health Organization (WHO) if the situation of concern meets two of the following four criteria:⁶

• 1.Is the public health impact of the event serious?
• 2.Is the event unusual or unexpected?
• 3.Is there a significant risk of international spread?
• 4.Is there a significant risk of international travel or trade restrictions?

The WHO Director-General may then choose to declare a PHEIC based on advice from the IHR Emergency Committee, information provided by State Parties, scientific expert recommendations, and risks posed to human health, international disease spread and international travel.^6,14

Participants agreed that this scenario adequately captures emerging outbreaks of high consequence. The definition also provides sufficient room for in-country public health authorities to exercise their judgment on emerging outbreaks that have not reached an international level of concern under the IHR (2005) but may evolve to outbreaks of higher consequence or still pose a significant risk of jurisdictional spread.

3.2. Defining confirmed cases

Participants discussed the role of case definitions in the reporting guidelines extensively. They emphasized the need for nuance and flexibility in defining confirmed cases, depending on the outbreak stage. The ability to confirm cases often depends on the availability of laboratory tests. However, this may not be possible at the start of an outbreak with novel pathogens. For example, the first U.S. Centers for Disease Control and Prevention (CDC) diagnostic test kits for SARS-CoV-2 were approved by the U.S. Food and Drug Administration (FDA) on February 4, 2020, about two months after the WHO was notified of the first COVID-19 cases in China.^15,16

Relying solely on laboratory confirmation assumes that tests are accessible and equitably distributed, which is often not the case. Unequal access to testing across U.S. cities and in low- and middle-income countries demonstrates this issue. Therefore, probable cases are important indicators to capture outbreak trends when confirmed diagnoses are difficult.^{17, 18, 19}

To reflect this in the ORBIT guidelines, participants agreed that public health authorities should release information about probable or suspected cases at their discretion, considering outbreak timing, evolving case counts, and testing capacity. They also stressed the importance of coordination and transparency among federal, state and local authorities to ensure consistency across jurisdictions.

3.3. Implementation strategies and best practices

Implementation of these guidelines should not pose a significant burden on the resources available for response. Most of these data are routinely collected and will need minimal additional analytic effort to facilitate public reporting. However, if this type of data has not been previously reported, preparation in advance of an active outbreak to develop appropriate policies and procedures may be required. These policies and procedures should include careful review and consideration of any privacy concerns associated with the data, as well as the determination of an appropriate spokesperson and venue for reporting. Preparing both leaders and the public in advance of an outbreak regarding the importance of public reporting will help create an expectation of transparency that will facilitate the implementation of the guidelines.

It is also important to note that the consensus guidelines provide public health jurisdictions with a minimum standard for what information to include in their regular reporting to the public. Public health jurisdictions may, however, choose to report additional data at their discretion to further inform the public. Additional reporting recommendations discussed and generally supported amongst the participants include:

• 1.The median number of days from sample collection to the earliest known confirmed test result. This item represents an indicator for testing lag and has the potential to be subject to right truncation bias in an exponentially growing epidemic.²⁰
• 2.Administrative Level 2 (county or equivalent) reporting of the final reporting items. This item suggests that public health authorities should report these data to the public with county or equivalent level geographic granularity when permissible considering the protection of privacy and preservation of confidentiality. As the consensus guidelines represent a minimum viable data set, jurisdictions may choose to exceed the reporting standards at their discretion by reporting county or equivalent level data alongside state, province, or equivalent data.
• 3.When the data is available and considerations of privacy and confidentiality permit it, public health authorities should report the onset date, followed by the specimen collection date and the diagnosis date, as per CSTE guidance, when reporting the number of new confirmed cases registered during the previous reporting period.¹³

4. Discussion

This set of minimum recommended reporting items creates a framework for uniform sharing of epidemic information. If adopted, it will also enhance consistency of data between jurisdictions. The goal is to improve outbreak response by ensuring that the public, media, researchers, healthcare providers, and governments can access detailed information about an unfolding epidemic. Reporting needs may change over the course of an outbreak. Therefore, we encourage coordination among public health authorities to maintain uniformity and comparability of reporting. We also encourage the use of the ORBIT guidelines to complement, rather than to replace, efforts to communicate information to the public about disease etiology, investigative status, and public health interventions.

In future work, we will consider disease-specific iterations of these minimum reporting recommendations to increase reporting uniformity and usability. We will also consider how these recommendations can be applied and adapted to low resource settings domestically and internationally. Additionally, we intend to align our current recommendations with other relevant efforts, such as those advanced by the U.S. CDC's Data Modernization Initiative, the WHO's Global Health Observatory and other relevant data systems.²¹

Our guidelines were developed with input primarily from the United States and the United Kingdom, along with participation from the WHO. We invite colleagues from other regions to develop similar processes that reflect their local contexts.

There are some limitations associated with the ORBIT guidelines. While consensus methods can be subject to some group biases, expert group approaches are more reliable than individual expert opinion.²² In this case, participants contributing to the development process may also have been influenced by their experiences with recent outbreaks, including COVID-19 and mpox, which may not fully reflect the needs of future outbreaks.

Additionally, while collecting individual data, such as sex, age group, race/ethnicity, typically allows for more nuanced analysis than aggregate data, it also poses greater privacy and confidentiality risks. Race/ethnicity data, for instance, should be collected and communicated thoughtfully, with consideration for the protection of vulnerable groups. Public health authorities should seek to build meaningful and lasting relationships with their communities and leverage communication best practices, such as using plain language²³ and working with trusted messengers, to communicate effectively with the public.^24,25

In conclusion, these guidelines represent a minimum standard for what public health jurisdictions should include in their regular reporting to the public during an outbreak that meets IHR (2005) criteria or when in-country public health authorities have otherwise decided to trigger use of the guidelines. The standards detail reporting items (e.g., the number of new confirmed cases), the frequency, and administrative level of reporting. We hope these standards facilitate timely, standardized reporting for critical public health events in recognition that such events pose a risk to the public and other public health jurisdictions who may wish to develop response plans.

Ethical statement

This study did not qualify as human subjects research according to the Johns Hopkins Bloomberg School of Public Health Institutional Review Board (IRB00023808).

Funding

This work was supported by Open Philanthropy Project. Open Philanthropy did not play a role in the writing of the manuscript or the decision to submit it for publication.

M.U.G.K. acknowledges funding from The Rockefeller Foundation, Google.org, the Oxford Martin School Programmes in Pandemic Genomics & Digital Pandemic Preparedness, European Union's Horizon Europe programme projects MOOD (#874850) and E4Warning (#101086640), the John Fell Fund, a Branco Weiss Fellowship and Wellcome Trust grants 225288/Z/22/Z, 226052/Z/22/Z & 228186/Z/23/Z, United Kingdom Research and Innovation (#APP8583) and the Medical Research Foundation (MRF-RG-ICCH-2022-100069). The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the views of the European Commission or the other funders.

Declaration of competing interest

C. M. Brown: The author serves as an unpaid Executive Board member of CSTE.

C. A. Donnelly: All support for the present manuscript was provided by the UK National Institute for Health Research (NIHR), with payments made to the University of Oxford [HPRU in Emerging and Zoonotic Infections: NIHR200907]. The author has also received grants or contracts to Imperial College London from the MRC Centre for Global Infectious Disease Analysis [grant number MR/R015600/1]. The author has received royalties from Cambridge University Press for a book co-authored with Sir David Cox and from Chapman & Hall/CRC Press for a book co-authored with Neil Ferguson. Consulting fees were received from UK Defra via the UK Animal and Plant Health Agency for work on developing a surveillance system to report TB in cattle herds exposed to badger control in England. The author holds unpaid leadership roles as a trustee and council member of the Royal Society, Vice President for External Affairs of the Royal Statistical Society, and trustee of St Peter's College, Oxford. Additional activities include an unpaid position with the WHO R&D Blueprint, and payments from Science Foundation Ireland and the Singapore National Environment Agency for reviewing grant proposals and participating in the Dengue Expert Advisory Panel.

S. Hopkins: The author has received grants or contracts from NIHR, UKRI, and MRC.

J. Lessler: All support for the present manuscript was provided by a CDC center grant for the Center for Forecasting and Outbreak Analytics. The author has also received grants or contracts from NIH, CDC, and the Gates Foundation. The author provided expert testimony for Cohen Ziffer regarding SARS-CoV-2 transmission and received support for travel from NIH for a study section and from the Isaac Newton Institute to attend a program in Cambridge.

R. Lynfield: The author's work has been supported by grant funding from the CDC to the Minnesota Department of Health. The author received support for travel from CSTE, NFID, IDSA, and AAP for attendance at meetings including CSTE, NFID, IDWeek, and the AAP Committee on Infectious Diseases. The author serves as an Executive Officer for CSTE, was an Executive Officer for NFID, is on the Program Committee for IDWeek, and is an Associate Editor for the AAP Red Book. The author also received a fee for work as Associate Editor, which was donated to the Minnesota Department of Health.

J. M. McCaw: The author's institution received COVID-19 response funding from the Australian Government Department of Health for applied research on epidemic situational assessment using Australian outbreak data. The author also serves as an invited expert on Australia's Communicable Disease Network, the peak national body responsible for making recommendations to government on surveillance practices.

T. D. McPherson: The author supported the New York City Health Department's application for the Epidemiology and Laboratory Capacity Cooperative Agreement through the CDC, which provides funding for public health activities. Payment is made directly to the Health Department for work plans included in the funding application. The author also received support for travel from the Infectious Diseases Society of America to attend IDWeek as an invited speaker and from the Johns Hopkins Center for Health Security to participate in a consensus panel on minimum reporting requirements during outbreak investigations and public health emergencies.

Z. Moore: The author received support from Johns Hopkins University for travel to attend a consensus meeting to finalize the voting items.

S. Riley: The author has received an Investigator Award from the Wellcome Trust. The author is also seconded to the UK Health Security Agency as a Director General and serves as a member of the agency's executive committee.

R. Rosenfeld: The author's work has been supported by funding via CFA/InsightNet and the CDC Flu Division. The author also serves as a member of the Data and Surveillance Workgroup of the Advisory Committee to the Director of the CDC.

W. Schaffner: All support for the present manuscript was provided by the CDC Emerging Infections Program.

J. Shaffner: The author received sponsored travel, including airfare and lodging, for a meeting related to manuscript development.

R. Sturm: The author received support from Johns Hopkins University for travel to attend a consensus meeting for final voting.

D. Terashita: The author received support for travel to attend a consensus meeting to finalize the voting items.

R. E. Washington: The author holds a leadership role as Chair of the Big Cities Health Coalition.

C. M. Rivers: All support for the present manuscript was funded by the Open Philanthropy Foundation. The author's institution has also received funding from ARPA-H, CDC, Johns Hopkins, and the Open Philanthropy Foundation. In addition, the author has received personal payments for professional writing, including a trade book, newsletter, and essays in major media outlets.

All other authors have no interests to declare.

All the authors, except for Vanessa Grégoire, Alex W. Zhu, and Caitlin M. Rivers, participated in the Delphi study.

Institutions where the work was done

The authors conducted this work remotely from their respective institutions, with the exception of the final stage of the Delphi process, which was carried out as a hybrid consensus meeting to finalize the voting items.