Using after‐action reviews of outbreaks to enhance public health responses: lessons for COVID‐19

Learning from outbreak after‐action reviews can enhance our response to COVID‐19 and future public health threats

Every outbreak is a lesson in prevention. After‐action reviews (AARs) and intra‐action reviews of outbreaks play a similar role to clinical audits and reviews but in public health contexts. ¹ They promote blame‐free system learning to enhance responses. Australian health authorities have increasingly conducted outbreak AARs using a structured audit methodology since a Communicable Diseases Network Australia pilot in 1997. ² In 2009, we published a structured methodology that was incorporated into Australian guidelines for multijurisdictional foodborne outbreak investigations. ³

The World Health Organization (WHO) recommends that outbreak AARs and joint external evaluations be collated systematically to allow analysis and assessment of preparedness for future events, and is urging countries to conduct intra‐action reviews during the current coronavirus disease 2019 (COVID‐19) pandemic. ⁴ , ⁵

To identify lessons for our future response to the pandemic and routine and emerging infectious diseases, we assessed findings from 14 Australian outbreak reviews. The methods for the past reviews have previously been published; ³ in brief, a pre‐meeting survey identified participant concerns and defined the scope of each review. Potentially contentious issues were constructively reframed and discussions about critical response issues were facilitated by an experienced public health physician (CD). ³ Reviews of each outbreak took 3–4 hours, with typically 10 to 30 attendees. Each review concluded with a report containing a summary of findings and list of recommendations for enhancing future practice.

It is difficult to access and collate outbreak reviews because of variations in methods and documentation; our analysis therefore includes reviews conducted by a single outbreak review facilitator (CD). We extracted all of the recommendations from the 14 review reports then collated recommendations common to two or more outbreaks. We organised them under the major themes used in the review reports.

The 14 outbreak reviews included 13 AARs and one intra‐action review from 2005 to 2020 across scenarios including six respiratory and eight foodborne disease outbreaks conducted at local, state, and multijurisdictional or national level (Box 1).

Box 1. Outbreaks subject to after‐action review or intra‐action review.

Outbreak description	Participants	Year
Local response to COVID‐19*	Regional public health unit	2020
Community meningococcal meningitis outbreak	Regional public health unit, state health department	2016
Legionnaires’ disease outbreak	Multiple regional public health units and state health department	2016
Multi‐state foodborne hepatitis A outbreak	Multiple state health departments, OzFoodNet, state and national food authorities	2015
Hospital listeriosis outbreak	State health department, multiple regional public health units, state food authority, laboratory	2013
Multi‐state listeriosis outbreak	Multiple state health departments, OzFoodNet, state and national food authorities, federal health and agriculture departments	2012
Regional measles outbreak	Multiple regional public health units, state health department	2012
Pandemic influenza national laboratory response	Public health laboratory network, multiple state reference laboratories	2009
Multi‐state listeriosis outbreak	Multiple state health departments, OzFoodNet, state and national food authorities, state reference laboratories	2009
Regional salmonellosis outbreaks	Regional public health unit, state health department, state food authority	2009
Salmonellosis outbreak in a restaurant	Regional public health unit, state health department, state food authority	2008
Salmonellosis outbreak in an aged care facility	Regional public health unit, state health department, state food authority	2008
Prophylaxis of contacts of food handler with hepatitis A to prevent an outbreak	Regional public health unit	2006
Multiple influenza outbreaks in aged care facilities	Regional public health unit, federal health department	2005

^*The only intra‐action review.

Seven themes were commonly identified during these outbreak responses and are explored below (Box 2). Key recommendations from these reviews that addressed these themes are listed in Box 3.

Box 2. Category and number of themes identified across 14 outbreak reviews.

Area requiring improvement	Number of outbreaks in which problem was identified*
Communication	14
Information management and sharing	9
Clarification of roles	8
Surge capacity	5
Incident command system	4
Decision support/rapid risk assessment	3
Need for exercises	3

^*Many themes were identified across multiple reviews.

Box 3. Key outbreak response recommendations, based on past reviews with potential relevance to the ongoing COVID‐19 pandemic response.

Communication

• ■Health alert networks including primary care, emergency departments, intensive care, hospital infection control and hospital epidemiology (where it exists) should be developed. Approved self‐subscription could ensure email addresses are current.
• ■Social media should be activated to allow a rapid and proactive sharing of information and correction of misunderstandings.
• ■Situation reports that update all stakeholders on the evolving epidemiological narrative should be broadly disseminated.
• ■Stakeholder communication (internal and external) should be a standing agenda item for outbreak management meetings. This is particularly important for aged care facility outbreaks.

Information management and interagency access

• ■Information management and sharing could be enhanced through investment in a common national (interoperable) digital platform that is specifically built for outbreak investigation integrating outbreak cohort, case, contact and laboratory data.
• ■Legal and policy instruments should enable sharing of information across jurisdictions and between agencies within the same jurisdiction. Legal barriers to sharing information across jurisdictions and agencies should be removed. In particular, agreements for sharing of information between epidemiological, environmental inspection and laboratory arms of investigations should be developed.
• ■Alternatives to distributing information via email need to be explored.

Clarification of roles

• ■National protocols for leadership in outbreak investigations, both vertically and horizontally, should be developed. This is particularly important in foodborne and zoonotic outbreaks where multiple agencies may be involved.

Surge capacity

• ■Agencies should develop a culture that encourages surged responses based on forward prediction of overwhelm, not evidence of current overwhelm.
• ■Legal and human resource impediments to rapid on‐boarding of staff need to be adapted for emergencies.
• ■Stockpiles of laboratory and personal protective equipment are required for pandemics and other large outbreaks.
• ■Scalable call centres and help desk applications to respond to massive community information needs are necessary.

Incident command systems and surge

• ■Adoption of an incident command system (ICS) in outbreak responses may better clarify leadership and coordination responsibilities.
• ■ICSs need to be adapted and exercised across multiple sectors (eg, health, agriculture, food safety, hospitals) with regard to operational, planning, logistics and communications functions.
• ■An ICS needs to be activated early in outbreaks.
• ■There needs to be a tolerance for down‐time in outbreaks that wax and wane. Commitment to longer deployments needs to be institutionalised.
• ■ICS training needs to be developed for a wide range of agencies, with clear job descriptions for all ICS roles.
• ■Mechanisms and triggers for emergency funding for public health response should be identified.

Decision support and rapid risk assessment

• ■Capacity for rapid risk assessment and rapid literature review should be developed as part of outbreak response.
• ■Appropriate depth of epidemiological support is required for outbreak investigations.

Need for exercises

• ■Exercises that engage vertically through local, state and national public health agencies and horizontally across collaborating external agencies should be conducted and assessed at appropriate frequency.

Communication

“Poor communication” was the most commonly cited concern; reported in all 14 outbreak reviews (Box 2). Outbreaks often require new, enhanced or more rapid communication pathways. The most common communication issues were: inability to rapidly communicate new or changing surveillance case definitions and response protocols to clinicians across multiple settings; lack of agreement on the agency responsible for providing public comments and media releases on sensitive issues such as the case, hospitalisation or death counts; limited interactions between public health agencies and important partners until a crisis occurred; and clinical network fax streams (eg, for emergency department directors) replaced with less reliable and changing email addresses.

Agencies sometimes limited distribution of sensitive outbreak situation reports leading to dissatisfaction among outbreak response teams and collaborating agencies. Many agencies had limited internal portals for sharing sensitive information, thus undermining internal communication. It was recognised that social media was becoming an essential communication component during responses. Real time social media use was inhibited in some outbreaks by multiple approval levels required to release information and then respond interactively.

Communication across government agencies, health providers and private entities was often challenging and there was a need for a stakeholder communication role in complex outbreaks. Messaging to stakeholders should be a standard agenda item for outbreak control meetings. Communicating effectively with staff, residents and relatives in aged care facility outbreaks was identified as a priority.

Information management and data exchange

Information management and data exchange was the next most frequently identified issue; noted in nine outbreak reviews. While most public health agencies have mature software platforms for processing routine notifiable conditions, outbreaks require new systems to track cases and contacts in community and workplace settings and across a range of novel and routine laboratory tests, referral pathways and reporting systems. New systems were required to track cases and contacts through: incubation periods; daily symptom, quarantine and isolation status; release from monitoring; administration of treatments; and prophylactic therapies. Lacking a central shared database, multiple spreadsheets were often used to manage data, with potential duplication and data integrity issues. System limitations sometimes led to data being entered into a local database and then re‐entered into a separate spreadsheet to produce reports or for data exchange across jurisdictions. This presented challenges owing to agency‐specific password protection, dissimilar information technology platforms, and lack of legal precedent for data exchange across jurisdictions. Additionally, most current surveillance databases do not integrate with call centre contact tracing activity systems.

Important information was often disseminated via email which, in the absence of formal structured record management systems, became a repository of corporate knowledge and decision‐making records. Email overload was a concern of staff in large outbreaks. Version control of widely circulated and updated documents was challenging. Scarce epidemiological resources were often consumed in generating duplicative reports for multiple stakeholders.

Access to outbreak information improved over time; however, it was variable and sometimes relied on interpersonal relationships rather than established data exchange permissions. In several foodborne outbreaks, epidemiological, laboratory and food safety investigations were conducted with minimal, or delayed, sharing of data between agencies. This prevented an understanding of the epidemiological, laboratory and environmental outbreak interdependencies, which inhibited response and control.

Clarification of roles

There were multiple reviews for which, when the question was asked “who was in charge of this outbreak response?”, the reply was “that was unclear”. There was often an absence of an identified lead agency or investigator in multi‐agency investigations. This lack of clarity occurred between public health agency hierarchies and between different agencies; for example, food safety and public health. This was not due to competition between agencies, but rather an unrecognised need for a single lead investigator and team, despite each agency’s unique responsibilities. Lacking formal recognition, lead investigators struggled to access information and resources.

Surge capacity

Outbreaks often required enhanced capacity across epidemiological, environmental health, food, laboratory, logistics and communications personnel. In many outbreaks, resources were overwhelmed before staffing was surged. There was limited awareness of the early signs of being overwhelmed or forecasting to pre‐empt being overwhelmed. With routine staff under pressure, they were unable to specify the surge needs, or recruit and train surge staff.

Maintaining the engagement of surged staff seconded from external organisations during lulls in activity was often reported as challenging. Seconding organisations and seconded staff themselves were often intolerant of down‐time, with requests for return within days of declines in activity. One challenge to surge response was the lack of systems to share workloads and track workflows across different agencies or jurisdictions.

Surge capacity was sometimes limited by budgetary constraints. There were rarely mechanisms for identifying or triggering funding for emergency public health responses. Barriers to on‐boarding paid or volunteer staff delayed surge responses. Inability to surge led to outbreak investigation and control timelines being determined by available resources rather than by public health control objectives. Laboratories were often challenged by the need to surge staff, introduce and validate new testing and analytical platforms, and prioritise specimen handling and processing during large outbreaks.

Incident command systems

Military inspired incident command (or control) systems (ICSs) align structures and communication across disparate organisations, allowing rapid and controlled resource surging to meet outbreak control objectives. ⁶ Agencies had varied ICS training and application experience. An ICS was often implemented later than optimal, incompletely, or with inadequate training. This resulted in obstacles to span of control (number of staff each manager supervised), clarity of leadership, surge capacity, and considerable stress on personnel who went unrelieved for prolonged periods. In large and prolonged responses, team leaders were required to attend multiple committee meetings, which they indicated limited their effectiveness.

Rapid risk assessment and decision support

Outbreaks often involved uncertainty requiring rapid risk assessment. The virulence and transmissibility of pathogens may be unknown and exposure levels may be uncertain. Public health agencies need to make informed judgements regarding public health orders, venue closures, travel restrictions, product recalls, case isolation and quarantine of close contacts, and administration of prophylaxis. Identified delays in decision making highlighted the importance of rapid risk assessment capacity. Assessments should explain the level of uncertainty underlying public health actions and recommendations. ⁷ Public health resources were often fully focused on response and operations with little time, or trained staff, available to conduct formal risk assessments.

In outbreak investigations, junior epidemiologists were sometimes given significant responsibility with limited support, which resulted in delays and uncertainty in design, analysis and interpretation of complex analytical studies.

Need for outbreak exercises

While only three outbreak reviews formally identified outbreak exercises as important, exercises have the potential to improve performance across all identified themes. Exercises, involving local, state and national agencies, are required to address the array of issues identified. Very few outbreak responses benchmarked performance, and standards are required to assess performance of both outbreak practice and exercises.

Discussion

The outbreak reviews identified seven common themes for improvement and presaged challenges for Australian pandemic response. The major theme centred on communication between and within teams, across agencies, and between agencies and important stakeholders such as general practitioners, the public and laboratories. Both our review and the National Contact Tracing Review report ⁸ highlight the importance of surging epidemiological, laboratory and communications responses, cross‐jurisdictional information management and access, performance metrics and conducting exercises. ⁸ The recommendations hold lessons for the ongoing response to the COVID‐19 pandemic and future responses to emerging infectious diseases.

AARs are designed to celebrate good performance as well as identify opportunities for improvement. Australia has made substantial improvements in outbreak performance over the past two decades, including development of national guidelines for response. ⁹ The sharing of information and coordination between jurisdictions has improved significantly through the Communicable Diseases Network Australia, which was formed in 1989 with jurisdictional and institutional representatives to provide public health coordination and leadership, and support prevention and control of communicable diseases. ¹⁰ Laboratory capacity to perform timely whole genome sequencing and other subtyping has also increased through the national Public Health Laboratory Network. Electronic laboratory reporting has reduced reporting delays, and sequencing has improved responses. The unfiltered use of social media by agencies has enhanced community engagement. ¹¹ It is unclear whether recommendations from past AARs contributed to these improvements.

The AARs analysed here provided recommendations that, if enacted, could have enhanced our pandemic response. Australian health agencies should consider using these AAR recommendations as the basis for a framework to improve responses to future outbreaks. Failure to enact lessons from bushfire AARs led to the development of a database of AAR recommendations to inform practice, and the United States has a public health lessons learned database. ¹² , ¹³

In addition to AARs of real public health emergencies to support International Health Regulations implementation, the WHO recommends AARs of simulation exercises. ¹⁴ Poorly executed tabletop exercises can often produce a false sense of security. ¹⁵ A 4‐day pandemic influenza exercise in a public health unit conducted in 2008 permitted a rapidly surged response during the 2009 influenza pandemic, and participants noted that the pandemic was less stressful than the exercise. ¹⁶ Exercise frequency determines performance outcomes and should include local public health units, the frontline of outbreak responses, which are infrequently included in national exercises.

Performance standards are essential for enhancing outbreak response and provide indicators for evaluation in reviews and exercises. Indicators can include meta‐level domains of governance, ethics and strategy, down to practical operational response indicators. Many countries have developed performance indicators and metrics for public health response and capacity. ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ Examples of important operational response indicators for COVID‐19 are provided in Box 4.

Box 4. Performance indicators relevant to COVID‐19 pandemic response.

• ■Number of cases (and their contacts) able to be interviewed or contact traced per day per million population
• ■Proportion of unlinked (mystery) cases (24 hours, weekly, monthly)
• ■The number of hours from specimen collection to notifying all people of their results, with the target being fewer than 24 hours at the 90th percentile ⁸
• ■The number of hours from the patient’s specimen collection to notifying their close contacts that they must quarantine, with the target being fewer than 48 hours at the 90th percentile ⁸
• ■Positivity of laboratory tests by age group, gender, ethnicity, and socio‐economic status of postcode (weekly time series)
• ■Testing rates by distance from testing sites stratified by socio‐economic status
• ■Number of contact tracers per million population (to address heterogeneity in resources)

These reviews were a snapshot of outbreaks since 2005, guided by a single facilitator. While this raises the possibility of bias, similar themes were identified overseas in expert consultations on recommendations for outbreak response ¹⁹ and indicators for outbreak preparedness. ¹⁸ Many agencies conduct reviews; however, practice and reporting are inconsistent. The included reviews were consistently conducted and reported, with findings and recommendations made by the participants. Recommendations for a national framework and community of practice for AARs should reduce practice and reporting variability. There is a growing body of guidance in the conduct of AARs in public health to assist countries to build a community of practice. ⁴ , ²¹ , ²² , ²³

Future reviews should consider: higher level decision making; cognitive biases; variations in situational awareness; group decision making; staff wellbeing; equity issues related to gender, culturally and linguistically diverse people and First Nations peoples; ethics; and privacy. The role of political interference with disease control should also be considered, given its impact in other countries, particularly the US. ²⁴

In many outbreaks, the response was determined by routinely available resources rather than achievement of specific public health objectives. The COVID‐19 pandemic demonstrates that public health response should be reconceived as a combat agency response. Similar to fire and defence services, public health requires significant reserve capacity in peacetime to allow rapid expansion when called upon in an emergency. A pandemic of the scale of COVID‐19 takes years, perhaps decades, to prepare for optimally. The timing of herd immunity is uncertain and the pandemic may be extended by variant strains. Australia and other countries should review the recommendations from past reviews and urgently begin WHO recommended pandemic intra‐action reviews. ⁵ , ²⁵ Looking to the future, it is clear that rapid public health responses can prevent weeks or months of lockdown, with savings variably estimated at between $450 million and $4 billion per week. ²⁶ Emergency public health response capacity is clearly both a health and economic investment.

Competing interests

No relevant disclosures.