Evaluating simulations as preparation for health crises like CoVID-19: Insights on incorporating simulation exercises for effective response

Abstract

Today's health emergencies are increasingly complex due to factors such as globalization, urbanization and increased connectivity where people, goods and potential vectors of disease are constantly on the move. These factors amplify the threats to our health from infectious hazards, natural disasters, armed conflicts and other emergencies wherever they may occur. The current CoVID-19 pandemic has provided a clear demonstration of the fact that our ability to detect and predict the initial emergence of a novel human pathogen (for example, the spill-over of a virus from its animal reservoir to a human host), and our capacity to forecast the spread and transmission of the pathogen in human society remains limited. Improving ways in which we prepare will enable a more rapid and effective response and enable proactive preparations (including exercising) to respond to any novel emerging infectious disease outbreaks.

This study aims to explore the current state of pandemic preparedness exercising and provides an assessment of a number of case study exercises for health hazards against the key components of the WHO's Exercises for Pandemic Preparedness Plans (EPPP) framework in order to gauge their usefulness in preparation for pandemics. The paper also examines past crises involving large-scale epidemics and pandemics and whether simulations took place to test health security capacities either in advance of the crisis based on risk assessments, strategy and plans or after the crisis in order to be better prepared should a similar scenario arise in the future. Exercises for animal and human diseases have been included to provide a “one health” perspective [1,2]. This article then goes on to examine approaches to simulation exercises relevant to prepare for a health crisis involving a novel emergent pathogen like CoVID-19.

This article demonstrates that while simulations are useful as part of a preparedness strategy, the key is to ensure that lessons from these simulations are learned and the associated changes made as soon as possible following any simulation in order to ensure that simulations are effective in bringing about changes in practice that will improve pandemic preparedness. Furthermore, Artificial Intelligence (AI) technologies could also be applied in preparing communities for outbreak detection, surveillance and containment, and be a useful tool for providing immersive environments for simulation exercises for pandemic preparedness and associated interventions which may be particularly useful at the strategic level.

This article contributes to the limited literature in pandemic preparedness simulation exercising to deal with novel health crises, like CoVID-19. The analysis has also identified potential areas for further research or work on pandemic preparedness exercising.

1. Introduction

Today's health emergencies are increasingly complex. We live in a globalized, urbanized and connected world where people, goods and potential vectors of disease are constantly on the move. These movements amplify the threats to our health from infectious hazards, natural disasters, armed conflicts and other emergencies wherever they occur. As past outbreaks of Severe Acute Respiratory Syndrome (SARS), Middle East Respiratory Syndrome (MERS), Ebola and Nipah viruses, and pandemic influenza make clear, emergent infectious agents pose a continued and considerable threat to human health [3,4]. To effectively respond to these health threats, the various health actors need to understand the operational environment in responding to these crises, which can be provided via pandemic/epidemic preparedness exercising.

For instance, the UK has been preparing for a pandemic response and has focused on a pandemic that would lead to thousands of deaths. The first pandemic in recent history was the H1N1 pandemic which emerged in 2009 and turned out to be a relatively mild illness for most of those affected, though it must not be forgotten that for some people its effects were very serious. Sadly, 457 people died during the pandemic in the UK [5]. The NHS Operating Framework for Managing the Response to Pandemic Influenza explained that the 2009 H1N1 influenza pandemic has not altered the likelihood of a future pandemic, and the generally mild nature of the 2009 event must not be taken as an indicator of the severity of such future events [6]. From the global effects and ongoing ramifications of the CoVID-19 pandemic, it is clear that many countries did not heed the warning provided by the H1N1 incident to be better prepared to deal with emerging infectious diseases.

The current CoVID-19 pandemic has provided a clear demonstration of the fact that our ability to detect and predict the initial emergence of a novel human pathogen (for example, the spill-over of a virus from its animal reservoir to a human host), and our capacity to forecast the spread and transmission of the pathogen in human society remains limited [7]. Hence the relevance to be proactive in preparing (including exercising) to respond to any sudden infectious disease outbreaks.

The current COVID 19 pandemic while a mild disease in most people infected, has led to thousands of fatalities. So, how did the UK and other countries learn from previous disease outbreaks and prepare for this threat that is a contemporary health risk across the world? How can the way in which we prepare be improved to enable a more rapid escalation of the response to an emerging or re-emerging health threat such as a pandemic subsequently? These are some of the questions that this research seeks to answer, with a focus on epidemic/pandemic preparedness exercising.

Emergency exercises involving the health community are considered an important and integral part of emergency preparedness activities [8]. The use of simulation exercises to provide immersive environments to simulate a real world emergency is well recognised in disaster management for strengthening the preparedness and response of organisations to major emergencies, including health emergencies such as those involving communicable diseases [9,10]. Past crises have taught us that even the most qualified personnel require continued learning to respond safely and effectively to emerging and re-emerging threats. Therefore, improving ways in which we prepare will enable a more rapid and effective response [11,12].

There is a paucity of research on how health security indices and simulation exercises are used as part of preparedness and their effectiveness, especially in low to middle income countries [13]. The impact of exercise participation on real world emergency response and the type of exercises that are most effective in preparing healthcare workers (HCWs) for real-world emergency response also remain poorly researched [11]. Skryabina et al. [14] were able to look at the role of preparedness exercises by examining two previously planned health emergency preparedness exercises which were undertaken shortly before both the Manchester and London attacks in the UK in 2017. Although the sample size of respondents was small and under representative of some areas of the healthcare response, they found that respondents found the exercises useful in practicing their roles in a mass casualty incident. However, the exercises studied were sudden onset mass casualty scenarios and not rising tide scenarios, such as pandemics. Therefore, more research is required using pandemic scenarios to examine exercise effectiveness for pandemic preparedness. Furthermore, poorly designed, executed and unevaluated exercises may provide a false sense of security, eventually leading to poor performance when an actual emergency happens [15]. This underscores the relevance of standards for public health exercises and drills [12,16] that can be adapted for sudden onset events and rising tide scenarios, such as pandemics.

The aim of this paper is to explore the current status of pandemic preparedness exercising in countries around the world. The objectives are to: (1) gauge pandemic preparedness exercises against a recognised pandemic preparedness framework; (2) examine the state of pandemic preparedness through exercises; and (3) emphasise the relevance of pandemic preparedness exercises in tackling a novel health crisis like the COVID-19 pandemic. This would provide insights on how the COVID-19 pandemic can influence future approaches and methodologies to better prepare responders at the national, sub-national and local levels. Therefore, this article contributes to knowledge in health crises by exploring the current state of pandemic preparedness through the lens of exercises and/or simulations following contemporary standardised practices.

2. COVID-19 implications for health system preparedness

While noting that the coronavirus is here for the long haul, the World Health Organisation (WHO) Director-General, Dr. Tedros Adhanom Ghebreyesus, cautioned that the world has to be healthier, safer and better prepared to deal with the disease [17]. In response to the current COVID-19 outbreak, the WHO encouraged governments to immediately and collectively scale up the necessary national actions. The first phase of the coronavirus pandemic produced a global lock down in less than 100 days and has potentially led to an era of hyper-awareness regarding the threat posed by infectious diseases and their potential to develop into pandemics which will result in long term changes in our approach to policy investments, behaviours and attitudes [18]. A second phase of the pandemic has emerged as we conclude this article. The relaxation of restrictive measures and lockdowns has seen a steep rise in incidence with thousands of new cases and rise in the death toll across Europe. Many European countries have taken swift action (curfews, closed bars/restaurants, limited groups, new lockdowns etc.) to contain this phase of the pandemic [19]. Key lessons that have emerged from the CoVID-19 pandemic include issues around the speed of the global response, public trust and collective action, whether existing governance tools were used or ignored and whether co-operation at the national and international levels was effective in the early stages of the emerging pandemic [20]. The COVID-19 pandemic has also highlighted the need for scalable solutions, which can be accessed online by exercise facilitators and participants so that they can carry out their own real-world simulations and take the lessons learned from these simulations forwards to provide positive long-term impact on practices and on the response to real emergencies.

3. Brief review of exercises for epidemic/pandemic preparedness

3.1. Importance of simulation exercises

There are two major groups of emergency preparedness exercises used to test an organisation's preparedness. This includes operation-based exercises (to test procedures, plans, technologies, interagency cooperation, tools, command centres) such as command post/functional exercises, field-based exercises that replicate an emergency response, drills and specific skills (evacuation, triage, communication); and decision-based exercises such as desktop/tabletop exercises and seminar/workshop-based exercises [21]. While operation-based exercises are scenario-oriented and designed to replicate practical/realistic situations, decision-based exercises tend to make participants practice and/or get acquainted with roles, procedures and plans [11].

These exercises are important in public health training for several reasons. Health care professionals need to understand their role when dealing with a health crisis. Exercising also helps health workers to comprehend the importance of being proactive within their communities and health units in developing plans for health crises rather than being reactive to determine plausible courses of action after an incident. Furthermore, the collaborative environment between the different health actors and other stakeholders required to tackle a health crisis has to be fostered during exercise preparations for health crises [8]. Furthermore, most reported benefits from emergency preparedness exercises were in identifying gaps (in emergency plans, procedures, resources, communication); in sharing the lessons from emergency exercises; that only a few studies reported positive long-term impact of exercises on job practices and on real emergency response and the optimal frequency and type of health emergency preparedness exercises that would lead to increased effectiveness [11,14].

The Global Health Security (GHS) Index that was endorsed by 195 countries that make up the States Parties to the International Health Regulations (IHR) was aimed to spur measurable changes in national health security and improve international capability to address infectious disease outbreaks that can lead to international epidemics and pandemics [22]. With little evidence that the health security capacities of most countries would be functional in a crisis, they were recommended to test their health security capacities (including training for health crises) and publish after-action reviews, at least annually [23]. Whilst there is sparse evidence that countries have been complying with the IHR, some exercising in anticipation of public health crises has been documented.

Several public health agencies and their stakeholders have conducted training in public health exercises that utilise simulations and drills in preparation for health crises caused by pandemics/epidemics and other shocks like natural hazards [9,10,24]. These exercises are popular for intra- and inter-agency pandemic preparedness and they depict a hypothetical scenario where various aspects of health crises mitigation, preparedness, response and recovery are discussed under different crises settings [[25], [26], [27]].

3.2. World Health Organisation (WHO)'s training/exercises for health crises

Training responders for outbreaks and health emergencies is one of the key functions of the Health Emergencies Programme of the WHO. The WHO and European Centre for Disease Prevention and Control (ECDC) have published guidance and manuals on carrying out simulation exercises. These documents provide technical guidance to public health officials in developing full-scale field exercises and simulations for supporting preparedness and response to communicable diseases [21,28]. The IHR 2005 advises that countries test their risk and pandemic communication ability/capacity at least every two years [29].

As part of global health preparedness, the WHO has acknowledged the use of different types of simulation exercises. In some instances, this includes live role-play simulation exercises, in which responders are trained for disease outbreaks with a large training faculty and mentoring. The WHO also recognises that some elements of live simulation exercise can be partially replaced by online-supported formats to provide more content-aligned, structured and replicable exercises while also creating cost efficiencies and reaching larger audiences [30]. In fact, the WHO has developed several pandemic preparedness frameworks [12] and guidance on simulation exercises to validate pandemic preparedness plans. The use of simulation exercises to test pandemic preparedness plans are an integral part of WHO's pandemic preparedness frameworks. The WHO recommends that adjacent countries or countries with strong trade/tourism relations should consider carrying out collective exercises and that lessons from all exercises should be used to revise existing pandemic preparedness plans [12,28,29].

3.3. Pandemic preparedness plans/exercises in Asian countries

A review of pandemic preparedness plans in South East Asia revealed that the translation of strategic plans into operational plans is lacking particularly in countries with low resources. Hanvoravongchai et al. [13] examined the pre-pandemic simulation exercises carried out in six countries (Cambodia, Indonesia, Lao PDR, Taiwan and Thailand) before the start of H1N1/2009 and found that most were table-top style where relevant officers discussed and managed a hypothetical pandemic situation in a round-table manner. For example, Thailand had at least one table-top exercise at the central level and in each province. Viet Nam has conducted many simulations for outbreaks of avian influenza transmitting to humans to assess preparedness at national, provincial and district level as well as at airports and borders.

There were also a few regional (multi-country) table-top exercises coordinated by the WHO. Only Indonesia and Taiwan had full-scale exercises involving real field activities. Indonesia's full-scale exercise in Bali in April 2008 was the first of its kind in the world. Taiwan's full-scale exercise at its national airport focused on its response to the arrival by plane of a suspected H5N1 case. Most simulation exercises in all the countries focused on early containment but not on pandemic preparedness in later phases. The limitations identified in the exercises included lack of management and coordination between various players; preparation for mitigation efforts at more advanced stages of a pandemic was quite limited; and knowledge/skills for pandemic preparation at the local level were more limited than for central level staff. Although countries had identified various channels for risk communication to the public, only Taiwan had clear operational procedures to sustain service provision and resource mobilization in a pandemic [13].

3.4. Simulations for pandemic preparedness

Leaming et al. [31] investigated computer simulation as a tool for assessing decision-making in pandemic influenza response training by developing and testing a computer-based, interactive simulation of a hypothetical pandemic influenza outbreak. The simulation used integrated video and branching decision trees, built upon current planning assumptions; presented over 50 key decisions to 6 role-players on networked laptops in a conference area and played out an 8-week scenario, beginning with pre-incident decisions. Key factors were looked at including effective communications, working with external organisations, development of internal policies and procedures, maintaining supplies and services, technical infrastructure support, public relations and training. Pre- and post-simulation responses of participants were recorded with post-simulation participants indicating a greater likelihood of needing to improve their organisation in terms of communications, mass casualty incident planning and public information and training.

The research concluded that the use of a computer-simulation was effective in providing a facilitated environment for determining the perception of preparedness, evaluating general preparedness concepts and introduced participants to critical decisions involved in handling a regional pandemic influenza surge. The design and use of simulations like this eliminate geography as a limitation to education delivery and can be applied at operational, tactical and strategic levels across many different sectors in addition to health. Furthermore, it can also enhance the development of principles and governance tools that can support the development of an effective pandemic preparedness and response strategy [31].

The COVID-19 pandemic has also provided an opportunity to test in situ simulation in a clinical context using virtual application of clinical training for clinical procedures required during the pandemic. A video-recorded simulation with virtual distribution was used to aid in the development of protocols for personal protective equipment (PPE) and Intensive Care practises. This enabled staff input and education under the constraints and pressures of a global pandemic [32].

Li et al. [33] also looked at the challenges and possible solutions for simulation and technology-enhanced learning during the COVID-19 outbreak in China and argue that after identifying the need for training, simulation has become a powerful weapon fighting against the virus with respect to providing operational training for front line healthcare staff as it can not only ensure patient safety but also provide a safe learning and training environment for HCWs to develop practical skills to deal with COVID-19. They propose the development of simulation curricula to ensure better preparedness of HCWs for this and future pandemics. This approach could be incorporated into a pandemic preparedness framework such as the WHO's EPPP framework used in this article.

3.5. The role of Artificial Intelligence (AI) in preparing for pandemics

In addition to online training platforms and virtual reality (VR) environments created through computer simulations, discussed earlier, AI is a growing area of research for health emergency preparedness. Davies [34] examined AI's potential contributions and limitations in the prediction, treatment and containment of a global influenza outbreak. AI is defined in public health as “a machine-learning tool that can improve medical diagnostics and treatment decision processes based on digital data” [34]; p. 182). Advantages of AI include aiding global influenza surveillance platforms by improving the capacity of organisations to look for novel influenza outbreak strains in the right places, identify populations most likely to spread influenza, and produce real-time information about the disease's spread by monitoring social media communications to track outbreak events. However, the research also reported limitations to the capability of AI and argued that AI should not only be used in strengthening health systems but rather should be used as part of the investment solution. Arguably, AI could be used to improve our understanding of who should receive a vaccine and to assess the most effective methods for large-scale vaccination [34]. This technology, therefore, has potential as a decision-making tool for use in simulations at the strategic level.

4. Conceptual framework for pandemic preparedness

Several frameworks for pandemic preparedness exist albeit with varied applications and scope. Savoia et al. [35] articulate a conceptual framework for the Evaluation of Emergency Risk Communications (ERC). The framework is useful to guide public health communication assessment procedures and evaluations, which is just one aspect of pandemic-operational planning. The WHO's pandemic influenza preparedness (PIP) framework that was adopted in May 2011 governs the sharing of influenza virus specimens with the potential to cause human disease and access to vaccines. Underpinning the framework, is the WHO's Global Influenza Surveillance and Response System (GISRS), which requires states to share virus specimens and facilitate equal access to benefits [36]. A conceptual framework to measure systems' performance during emergency preparedness exercises, developed by Savoia et al. [37] explains the key components to consider when assessing public health emergency exercises. This framework describes how information/data generated from simulated exercises support the assessment/judgements of the state of preparedness of health emergency systems. Furthermore, Khan et al. [56] developed a conceptual framework of the essential elements for a resilient public health emergency preparedness (PHEP) system. This empirically derived and theoretically informed framework identifies key actions to enhance preparedness for emergencies and disasters. Although complex, the framework is resilience focused, adopts an adaptive system approach to preparedness for PHEP and is applicable to regional public health agencies. Whilst these frameworks are useful, their scope and breath are limited to be used as a tool to gauge the pandemic preparedness exercises for countries. The WHO's [12] novel pandemic preparedness framework discussed in the next section is more comprehensive, has a wider scope for pandemic readiness, is applicable to pandemic preparedness at the national level and fits the aims/objectives of this article.

4.1. Framework for Developing and Conducting Simulation Exercises to test and validate pandemic influenza preparedness plans

The WHO's novel framework for Developing and Conducting Simulation Exercises to Test and Validate Pandemic Preparedness Plans—referred to subsequently as Exercises for Pandemic Preparedness Plans (EPPP) [12] has been adopted to provide focus and orientation to this article [38]. The framework, which aims to support countries to test and update their national pandemic influenza preparedness plans, can be adapted to provide guidance on how to select, plan, conduct and evaluate simulation exercises specific to other epidemics/pandemics. Hence the framework can be used by any individual, organisation or agency involved in public health emergency planning or building pandemic preparedness capacity. The challenge in using this framework, however, is that it is heavily institutionally oriented. To successfully apply this framework, the implementing body, agency or organisation has to be abreast with previous WHO guidance documents to enhance pandemic plans and exercises. The EPPP framework is heavily informed by the WHO's simulation exercises and pandemic influenza risk management documents (and its supporting checklist)—the EPPP framework should be applied in consultation with these documents [12]. Therefore, agencies in charge of preparing for health crisis may find it challenging to comprehend and implement pandemic prepared exercises/plans due to the shear complexity of the documents to consult if the EPPP framework has to be implemented successfully as recommended by the WHO.

The EPPP framework consists of seven key components:(1) selecting the exercise, (2) planning the exercise, (3) developing the scenario, (4) describing the pandemic, (5) planning the evaluating, (6) staging the exercise and (7) after the exercise (see Fig. 1 ).

Fig. 1.

Framework for EPPP [12].

A succinct description of the seven key components that must occur systematically for each exercise or simulation is presented in Fig. 1 and a summary of the key points in each component presented in Table 1 .

Table 1.

Key points to note about the WHO's EPPP framework.

Components	Summary/Key Points to Note
Selecting the Exercise (1)	⁃Agree on a document that sets expectations for the exercise (e.g. a concept note) and circulate this to stakeholders. ⁃Set a small number of clear, well-defined objectives – less is often better. ⁃Clearly communicate the aims and objectives of the exercise to everyone involved. ⁃When choosing an exercise, consider the aim and objectives of the exercise; type of plan being exercised; and level of human and financial resources you can commit. ⁃Exercises can be extremely resource-intensive – do not underestimate the time, staff and funds you will need. ⁃Build organizational experience with basic exercises (e.g. a series of small exercises to test specific components of the plan) before moving to more complex ones.
Planning the Exercise (2)	⁃Good communication within the exercise management team is crucial; therefore, set up regular team meetings and collaboration mechanisms early on. ⁃Define target audiences for different exercise products and communications – exercises are also opportunities to increase multisector engagement in pandemic preparedness. ⁃Limit participation to staff directly involved in the plans being tested – having too many participants tends to reduce the quality of the exercise. ⁃Preparation is key to a successful exercise – make sure that participants, evaluators and team members are fully briefed on how the exercise will work and what their role is.
Developing the Scenario & Describing the Pandemic Variables (3)	⁃A good scenario will, support the aims and objectives of the exercise, generate discussion and/or action from players and be as realistic as possible. ⁃Be creative when delivering injects - realistic delivery methods can make the exercise more engaging and interesting for everyone involved. ⁃Use enough injects to keep participants challenged, but not so many that they feel overwhelmed and cannot meet the objectives of the exercise. ⁃Exercises are dynamic events - be prepared to adapt and create new injects depending on how the exercise is progressing.
Planning the Evaluation (4)	⁃Plan the evaluation from the start, in parallel with the exercise. ⁃Align evaluation questions to the exercise aim and objectives. Prioritize the issues and questions to be addressed and keep them to a manageable level. ⁃Assessment criteria should focus on the plan, response system or operations being tested, not the participants. ⁃Where possible choose an external lead evaluator, with the support of an internal evaluator, or a mixed team of internal and external evaluators. Ensure that evaluators are thoroughly trained and briefed on the exercise and evaluation process - an evaluation team that is fully prepared will deliver a more useful, insightful and fair evaluation.
Staging the Exercise (5)	⁃Check and test all equipment and materials at least the day before the exercise - particularly for the delivery of injects and communication within the exercise management team. ⁃Hold a pre-exercise briefing for the exercise management team and separate briefings for the facilitation and evaluation teams. If an exercise is run over multiple days, team briefings should be held at least once a day. ⁃Hold a pre-exercise briefing for participants and observers to go over the purpose of the exercise, the guidelines and the rules of play. ⁃Debriefing is a critical part of exercises and should be allocated enough time. For large exercises, a formal exercise debriefing should take half a day at minimum.
After the Exercise (6)	⁃Complete and circulate the exercise report as soon as possible to capture the momentum of the exercise - ideally within 1 month or less. ⁃Ensure that evaluation recommendations are useful, practical and relevant. ⁃Develop an action plan with key stakeholders for reviewing and improving the pandemic influenza preparedness plan, including specific improvement actions progress indicators, responsible person or agency and completion date for each improvement action.

Source: Authors, adapted from WHO [12].

The framework has been used to gauge examples of pandemic exercises to access their usefulness in preparing for a pandemic.

5. Methodology

As this study aims to explore the current state of pandemic preparedness exercising, a qualitative approach was adopted. The empirical focus is assessing exercises for health hazards in order to ascertain their usefulness in preparation for pandemics. Since research on training or preparation for health hazards and/or epidemics and pandemics is sparse, and have not received much attention, this research is also exploratory in nature. Hence, primary, and secondary information was invaluable for analysis [39,40].

Primary and secondary document search was undertaken during October and November 2020 to identify a variety of pandemic training, exercises or simulations that have been carried out in different parts of the world prior to and/or after health crises/epidemics to obtain data on the research enquiries. Due to COVID-19 restrictions, the document search was completed online to access digitized information from the various sources.

The search for all electronic sources of information was undertaken through random sampling to avoid bias. This was simply guided by the following key phrases in the title, abstract, introduction, or main body of the document: (healthcare system or health system) and (pandemic or epidemic or communicable diseases or health crisis or health emergency) and (preparedness or exercising or simulation or immersive environment). Random sampling, using the aforementioned search criteria was suitable to select exercises worldwide used to achieve objective 1 by assessing the exercises against the EPPP framework, and subsequently, objective 2 (to examine the state of pandemic preparedness).

Primary sources included administrative data on pandemic preparedness plans/programmes that could be found in technical reports, press releases or manuals of government institutions, regional administrations like the European Union, and international health organisations/agencies like the Centres of Disease Control and WHO. Secondary data was drawn from books and academic articles. These sources, which are appropriately referenced when used in the article served to elicit knowledge about the issues under investigation [40].

Documents from governments, health organisations and agencies were obtained from their websites. Search for academic articles utilised electronic databases like PubMed, EMBASE, Google Scholar and Global Health. The main focus was descriptive studies or case studies of pandemic exercises/simulations in the English-Language that have taken place within the last two decades from 2000 to 2020.

The selected documents were examined and scrutinised for various exercises/simulations via content analysis [41] to identify the key issues (exercise components) relevant to achieve the research objectives [42]. This helped to sieve out the key exercise activities/aspects ideal for epidemic/pandemic preparedness, which were gauged against the seven components of the EPPP framework to determine their validity or relevance for epidemic/pandemic or health crisis preparedness. This article builds on these to determine the state or preparedness of countries for outbreaks of infectious diseases in view of the COVID-19 pandemic. Hence, the article thus draws on the insights from the state of preparedness from different countries that have dealt with health crises to highlight trends that need emulation or improvement.

6. Case studies: country-level preparedness for epidemics/pandemics exercises

In order to inform the future use of simulations to enhance preparedness this paper will now examine past crises involving large scale epidemics and pandemics and will look at whether simulations took place to test health security capacities either in advance of the crisis based on risk assessments, strategy and plans or after the crisis in order to be better prepared should a similar scenario arise in the future. Exercises for animal and human diseases have been included to provide a “one health” perspective [1,2]. The relevance of these case study exercises have been assessed in table two and gauged against the EPPP framework.

6.1. Food and mouth disease (FMD) exercises

6.1.1. UK's plans for the FMD (2001)

The FMD epidemic of 2001 was one of the largest in history. The outbreak took governments and the farming industry by surprise [43]. However, the disease was eventually contained and prevented from becoming endemic. Nevertheless, the FMD outbreak unveiled several limitations in the UK government's response.

The UK's Ministry of Agriculture, Fisheries and Food had contingency plans to fight the FMD that met EU requirements but lacked scope. The plan had not been shared widely or vigorously rehearsed outside the State Veterinary Service [44]. As a result, there was a limited shared sense of ownership by stakeholders. The plans had failed to keep up with changing farming practices and was not seen as part of a wider process of disease prevention and risk management [43].

The major lessons from the UK FMD crisis included; to maintain vigilance through international, national and local surveillance and reconnaissance; to be prepared with comprehensive contingency plans; to build mutual trust and confidence through training and practice; to react with speed and certainty to an emergency or escalating crisis by applying well-rehearsed crisis management procedures; to explain policies, plans and practices by communicating with all interested parties comprehensively, clearly and consistently in a transparent and open way; to respect local knowledge and delegate decisions wherever possible, without losing sight of the national strategy to use data and information management systems that conform to recognised good practice in support of intelligence gathering and decision making [43].

6.1.2. FMD exercise in Nordic countries (2005)

Following the FMD outbreak, an EU Directive was put in place, which requires Member States to undertake real-time exercises to assess their FMD contingency plans twice within a five year period [45] or “two times during the five years period after an outbreak of a major epizootic disease has been effectively controlled and eradicated” (p.72). In conformity with the directive, several exercises took place in many European countries.

From September 19th-23rd 2005, an FMD simulation exercise involving the five Nordic countries (Denmark, Finland, Iceland, Norway and Sweden) took place [46]. This was a command post exercise (CPX) that dealt both with inter-Nordic activities and the activities of individual countries and was designed to test contingency plans and cross border communications and coordination. Geographical Information Systems (GIS) was used as a mapping tool during the exercise. The exercise also looked at management of the outbreak, particularly decision making and policy development; technical aspects covering disease surveillance; diagnostic capabilities and capacities; use of FMD airborne spread models; culling and disposal of animals; communication between countries and media communication; information transfer with regard to epidemiological data and use of GIS to monitor resources in relation to people and equipment [47].

6.1.3. Exercise Silver Birch (2010)

Exercise Silver Birch took place in 2010 to test the UK Governments’ plans for a medium to large scale FMD outbreak [48]. The exercise had over 600 participants from various UK agencies including senior officials from the UK administration and the Animal Health Agency. It included a field operational element, a tabletop exercise (TTX), simulated strategic meetings and exercise briefings. It concluded with a two-day live exercise involving participants from across the UK. The report from the exercise provides an evaluation of the exercise and records the lessons identified for further action; these include a number of lessons that were similar to those identified during the FMD outbreak in 2001, which were, to improve communications processes, to improve data sharing and to work with stakeholders to share and formalise plans for a large scale outbreak.

6.2. Pandemic influenza exercises

6.2.1. Exercise Winter Willow (2007)

The aim of the 2007 Exercise Winter Willow (EWW) was to check the UK's preparedness for an influenza pandemic and to integrate any lessons learned into the influenza pandemic plans and procedures [49]. EWW was delivered in two stages; Stage 1 (held on January 30, 2007) comprised a national TTX meeting that simulated UK pandemic alert level 2 while the second phase (February 16–21, 2007) involved a full scale national exercise to follow up on the decisions taken in stage 1. Stage 2 was designed to test the UK response at UK pandemic alert level 4. The exercise had over 5000 participants from various sectors including the UK government and business organisations. In developing the scenario, it was recognised that some weeks' notice was vital to developing the global situation and there would have been several high-level meetings prior to confirming the first case in the UK. The exercise explored the opportunities and challenges working between the established crisis response structures of the devolved administrations at all levels particularly within the health community to identify potential challenges during a UK influenza pandemic.

6.2.2. Malawi influenza plan (2009)

Malawi developed a national influenza plan in 2006 to reduce and/or manage the burden of an influenza outbreak. However, the country's Health Sector Strategic Plans does not have influenza control strategies [50]. The plan was activated in 2009 to respond to an influenza pandemic that hit the country [3]. The authorities implemented several public health initiatives: they strengthened influenza surveillance activities, pandemic training and education and enhanced communication strategies. However, the lack of simulations, influenza drills and exercises to test or activate the plan was problematic during the response to the pandemic. Consequently, the pandemic response was weak and did not match the activities of the contingency plan. This example underscores the relevance of good preparation to respond to influenza pandemics especially in effective communications, evidence-based decision making, adequate public health responses and a reliable surveillance system. Building these into simulation exercises is absolutely vital to effectively prepare for future pandemics [3].

6.2.3. Exercise Cygnus (2016)

Exercise Cygnus was an influenza pandemic exercise that took place in 2016 and tested some of the issues highlighted in the Hine report. This was a full-scale national simulation exercise carried out by the Department of Health and Social Care and its purpose was to estimate the impact of a hypothetical H2N2 influenza pandemic on the United Kingdom. The exercise showed that the pandemic would put the country's health system under great pressure due to a lack of resources. The Chief Medical Officer at the time (Dame Sally Davies) highlighted potential challenges including lack of enough ventilators, disposal of dead bodies and potential problems in the social care sector in the event of a pandemic [4].

As of April 2020, the full results of the exercise remained classified. Nevertheless, it was leaked and published in full by The Guardian [51]. It highlights the following points: (1) The UK's preparedness and response, in terms of its plans, policies and capability, is currently not sufficient to cope with the extreme demands of a severe pandemic that will have a nationwide impact across all sectors; (2) concerning the exercise, while each participating government body had its own bespoke plans, enabling a flexible and decentralised response, nobody in the centre had oversight over everyone else; (3) It will be much harder to shift resources between one another to react to unexpected rises and falls in demand for services such as social care beds.

Exercise Cygnus provided a realistic immersive environment and demonstrated that a pandemic would severely stretch the resources of the health system in the UK and therefore, in planning for future pandemics, we need to plan for an escalation in the response to novel viruses with the potential to cause a pandemic with a resultant increase in infections and a surge in demand across the health and social care sector. Exercise Cygnus highlighted the issue around lack of preparedness in the care home sector which was exacerbated by lack of co-ordination between the health care sector and the social care system, however, these lessons were not taken forward fully into preparedness planning [51].

6.2.4. Open WHO courses on pandemic influenza preparedness (from 2017)

Since 2017, the WHO developed and has been delivering a series of online courses for Pandemic/Epidemic Preparedness that has registered more than 3600 participants from nearly 100 different countries [16]. WHO's website “openwho.org” advertises the courses stating that they can be found on a web-based, interactive and knowledge-transfer platform aimed to improve the response to health emergencies. Since the courses became popular, the WHO has been offering introductory courses on the treats of specific diseases—pandemic/seasonal influenza; surveillance, impacts and developing strategies for combating seasonal or influenza pandemics and improving risk communication and public health interventions for an influenza pandemic. The training includes modules on simulation exercises for pandemics [52].

6.3. Coronavirus exercises

6.3.1. Event 201 exercise (2019)

While this is the first pandemic to be caused by a Coronavirus, there have been previous outbreaks of Coronavirus Severe Acute Respiratory Syndrome (CoV SARS). From November 2002 to July 2003, the severe acute respiratory syndrome (SARS) quickly spread from Foshan (Shunde district), Guangdong Province in the People's Republic of China to 33 other countries. There were 8447 cases, 21% occurring in HCWs, and 813 deaths (9.6% overall mortality) by the time SARS was contained in July 2003 [57]. Challenges faced by the health care institutions included closure of the ICU beds; loss of staff through quarantine and illness; emergent introduction of new, complex, and restrictive infection control procedures; rapid staff education, system planning, and maintaining morale. It was recommended that coordinated leadership, communication infrastructure, and systems be put in place to quickly expand and modify critical care services as this is essential to meeting the demands of a SARS outbreak [53]. These challenges were noted but the focus of pandemic preparedness since 2003 has been on influenza and only a limited number of simulations have looked at the emergence of a Coronavirus with the potential to cause a pandemic.

One of the most recent simulation exercises to test plans to respond to a coronavirus epidemic/pandemic was conducted on October 18, 2019 (known as Event 201) in New York, hosted by Johns Hopkins Centre for Health Security in partnership with the World Economic Forum and the Bill and Melinda Gates Foundation [54]. Event 201 was a pandemic TTX that simulated a series of plausible scenario-based discussions about the response to a hypothetical pandemic. The participants included policymakers, business leaders, and health officials from around the world. The pandemic simulation exercise was designed to assess how much work is needed to bolster the disaster response systems, especially in areas where public/private partnership will be necessary during the response to a severe pandemic. The format was a TTX with round table discussion based on injects provided through PowerPoint presentation and simulated media injects that described the worsening situation [54]. Event 201 involved a simulation exercise that put participants amid an uncontrolled coronavirus outbreak that was spreading rapidly out of South America to have a catastrophic effect worldwide (crippling trade and travel, sending the global economy into freefall, just like global ramifications of COVID-19). The exercise highlighted the need for urgent decision making early in a potential or worsening pandemic situation when information is sparse and unreliable.

7. Gauging the case study exercises against the WHO EPPP framework

These past pandemics and large epidemics and the simulation exercises used to test plans for these serve to demonstrate the need to effectively learn from these events and to incorporate this learning into preparedness planning, for example through the development of effective simulation exercises that bring together the health sector and other sectors that will need to be part of an escalated response as the pandemic becomes more wide scale. It would also be beneficial, as part of the preparedness process, to bring together learning from outbreaks of both animal; and human diseases that have the potential to cause pandemics in a “one health” approach to preparedness [1,2]. Simulations of epidemics of animal disease, in addition to epidemics and pandemics of human disease have been included in this analysis. A comparative analysis of the key aspects of the exercises has been done against the WHO EPPP framework to assess their suitability to mitigate a pandemic. A comparative analysis of the key aspects of the exercises has been done against the WHO EPPP framework to explore the current state of pandemic preparedness exercising and whether a framework approach can augment pandemic preparedness by enabling a closer examination of the reported lessons from these events.

8. Discussion

By gauging the case study exercises against the WHO EPPP framework (Table 2 ) it is clear that the national level TTX and field exercises carried out in Europe and America employed components of the EPPP framework or a similar framework/methodology to select, plan, deliver, evaluate and review the exercise. The use of such a framework assists in delivering real world exercise simulations that can inform pandemic preparedness. However, the focus tends to be on the exercise itself rather than developing an effective action plan following the exercise review/debrief element of the framework.

Table 2.

The case study exercises gauged against the WHO EPPP framework to determine their usefulness for pandemic preparedness.

Exercise, Disease or Plans & Date	Type of Exercise	Country	Assessment of Exercise in relation to WHO's EPPP Framework Components (Fig. 1 & Table 1) and Other Observations
•Simulation exercise (for FMD) •2005	•Functional •CPX	5 Nordic Countries (Denmark, Finland, Iceland, Norway and Sweden)	•All seven components of the WHO's EPPP framework was manifested. •The exercise facilitated testing of FMD contingency plans from the participating countries from scenarios jointly prepared by the Central Veterinary Administrations of the participating countries. A master events list was used to deliver the exercise injects which kept the exercise on track. An evaluation process was carried out which dealt both with inter-Nordic activities and the activities of individual countries •GIS was used during the exercise that enabled countries to share maps and improved situational awareness. This enhanced the administration of cross-border zones. •A key outcome of the exercise was that good communications between the 5 countries, including their veterinary administrations, is key to effective co-ordination and containment. •The exercises resulted in a series of recommendations for countries to take forward.
•Exercise Winter Willow •Stage 1 (January 30, 2007) •Stage 2 (February 16–23, 2007)	•Table-Top Exercise (Stage 1) •Full Scale Exercise (Stage 2)	United Kingdom (England, Wales, Northern Ireland)	•The components of the WHO's EPPP framework were manifested (considering both stages) •The aim was to check the UK's preparation for major disruptive challenges presented by an influenza pandemic. •As part of the after-action review, decisions taken in stage 1 were followed up in stage 2. •The exercise was designed to test UK's response to widespread cases of pandemic flu at local, regional and national levels. •The exercise tested the planning assumptions outlined in the draft UK National Framework for Responding to an Influenza Pandemic •Pandemic modelling was incorporated as part of the scenario development. •Key lessons from the exercise include that crisis management/coordination should be improved between the regional and national resilience structures; the need for better sharing of information between national and international bodies.
•Malawi Influenza Pandemic Plan •2009	•Influenza plans lacked simulation exercises	Malawi	•The influenza plans did not have simulation exercises, hence did not conform with WHO's EPPP framework. •The lack of influenza simulations/exercises in the plan led to a poor response to the 2009 Influenza outbreak. •The need to inculcate drills and/or exercises for influenza contingency plans cannot be overemphasised.
•Exercise Silver Birch •November 2010	•Full-scale TTX exercise •Field Operational exercise	United Kingdom (England, Wales, Northern Ireland)	•All seven components of the WHO's EPPP framework was manifested. •The key motive was to test the UK's contingency plans for an outbreak of FMD and how the devolved administrations would interact in the event of a disease outbreak that involves all four administrations. •The exercise scenario was predicated on disease spread through markets in England, Wales and Scotland. •The exercise involved simulated strategic meetings and briefings. •Key lessons from the exercise were: to improve communications processes, to improve data sharing and to work with stakeholders to share and formalise plans for a large scale FMD outbreak.
•Exercise Cygnus (Pandemic flu exercise) •October 2016	•Full scale exercise	United Kingdom (England, Wales, Northern Ireland)	•All components of the EPPP framework were used in preparing for, delivery, evaluation and after exercise actions. •The aim was to assess the UK's preparedness and response arrangements at local and national level for an influenza pandemic. •The full results of the exercise remain classified. Nevertheless, the leaked findings reveal that the UK's preparedness and response for an extreme pandemic are currently insufficient. •Recommendations were not fully implemented before the CoVID19 pandemic.
•Open WHO online training •2017 and ongoing	•Online web-based Exercises	Global	•Has training on Pandemic Simulation Exercises as a separate module that incorporates the WHO's EPPP components (see Section 5.1).
•Event 201 (Global Pandemic Exercise) •October 18, 2019	•Strategic TTX	US/Global	•All seven components of the WHO's EPPP framework was manifested in the exercise •The exercise was designed to help policymakers gain a fuller understanding of the urgent challenges they could face in a dynamic, real-world crisis. •The strategic exercise used a coronavirus pandemic as the scenario. •Recommendations from the exercise were not fully implemented before the CoVID19 pandemic as some were long term recommendations. •A key recommendation highlighted the need for urgent decision making early in a potential or worsening pandemic situation when information is sparse and unreliable.

Pandemic planning has historically focused on pandemic influenza and not on other potential causes. Likewise, exercises have tended to focus on the health system response rather than a wider systems approach. An effective action plan is an essential tool to take forward and implement any lessons learned from the exercise. For example, the lessons from Exercise Winter Willow resulted in the revision of the pandemic preparedness frameworks in the UK. Unfortunately, the lessons learned from Exercise Cygnus were not all implemented prior to the CoVID19 pandemic, possibly due to competing priorities at the time following the EU Exit referendum. These exercises demonstrate that there needs to be an essential linkage to key follow-up actions particularly in terms of action plan. This was incorporated strongly in all exercises in their design, delivery, and outcomes and even if they are achieved, pandemic planning is subject to external pressures and variables that affect their success. Both observations highlight that simulation exercises do not just need to highlight the need for adaptation, they also need to be integral and endemic in the design and linkage to the follow up and outcomes stemming from them. Hence exercises need to be seen not just as static instruments but should also be intrinsically linked to follow up and action plans with measurable outputs to improve preparedness and drive positive change.

The exercise, Event 201, identified a number of recommendations, however, there was insufficient time to implement these prior to the onset of the CoVID19 pandemic but did serve to act as a warning that more needed to be done in terms of pandemic preparedness at the strategic level. Event 201 demonstrated that robust public-private partnership (PPP) systems are necessary to provide a systems response approach to pandemic emergencies which need to be part of the preparedness and planning process and tested thoroughly through simulation exercises. This is in contrast to current approaches which have tended to focus on the health system response rather than a wider systems approach which, as we have seen in the current COVID-19 pandemic, is key to delivering the required capacity and capability.

Simulation and online learning enhanced by technology have already played a role in dealing with the COVID-19 outbreak in China, and its importance is likely to be increasingly recognised by more individuals and organisations in the health system. Simulations need to be designed that will address the lessons emerging from the COVID-19 pandemic that can inform and enable improved national and international co-operation, information sharing and practise. The technology used will need to enable the rapid development and deployment of training through real world simulations during a pandemic or other emergency for rapid education of decision makers at the policy/strategic level and practitioners alike.

Indeed, emphasis should be put on the desired outcome and the quality of learning from simulations with experience being shared across organisations to help improve understanding and enable better preparation regarding the spread of the COVID-19 outbreak and other potential pandemics [33]. The WHO has digitized parts of its simulation training and this is being further developed to provide online training including simulation exercises. The use of digital technologies such as this and the use of AI technologies can improve the learning experience for those involved. As reviewed earlier in this article, computer-simulation can be effective in providing a facilitated environment for determining the perception of preparedness for a mass casualty incident [31]. Nevertheless, it is imperative that simulations should follow closely the principles contained in the WHO's EPPP framework.

AI has been used here as an illustrative example of technologies that can be used to provide improved real-world simulations that allow strategic leaders and policy makers decision making opportunities in real time in a pandemic scenario. AI is reaching into every aspect of global health and it is, therefore, important to apply simulations that better replicate the real world by using AI to provide realistic immersive environments for testing preparedness planning and highlight any gaps. The use of emerging technologies such as AI will enable one simulation event to reach a wide audience, and if required to be used during a pandemic, will allow participants to respect social distancing, while at the same time providing a realistic educational and development tool which will also enable the rapid adoption of strategies and interventions in real time. Whilst this would be a significant step forward, the lessons from these simulations must be implemented and incorporated into future planning to ensure that we take full advantage of the technologies available.

The implementation of the key lessons emerging from COVID19 is essential to ensure that in future pandemics/epidemics governments are prepared, can detect and test all suspect cases; can prevent, suppress and interrupt transmission; provide safe and effective clinical capacity and care; share knowledge; and are able to develop and distribute new diagnostics, drugs and vaccines. These measures would contribute significantly to global health security and provide a more rapid response to an emerging infectious disease with the potential to cause a pandemic. In order to achieve this, countries need to put in place action plans at local and national levels and have a systematic way of tracking actions to ensure that they have been completed. Furthermore, we would argue that as expected, any framework for pandemic preparedness should take a systematic approach that include all of the elements of the WHO's EPPP framework and also take a systematic approach to learning from real emergencies and simulation exercises.

9. Conclusion

This article has examined approaches to simulation exercises relevant to prepare for health crisis within the context of the CoVID-19 influence. Against the backdrop of a paucity of research on the impact of exercising for epidemic/pandemic preparedness [11], the motive, in view of the dire ramifications of the CoVID-19 pandemic, was to examine the current state of pandemic preparedness exercising that is suitable to respond to a novel health crisis like CoVID-19.

In order to gauge the state of pandemic exercising preparedness, cases studies or pre- and/or post-exercising plans for epidemics/pandemics or public health crises around the world was examined. The case studies involved past exercises/simulations for infectious diseases preparedness like the FMD, Influenza, SARS and Ebola that took place in countries around the world. To determine the effectiveness of the exercises/training, the key components of the WHO's EPPP framework (Fig. 1, Table 1) were used to gauge their usefulness in preparing for pandemics.

Therefore, this study provides an evaluation of past pandemic exercises against the WHO EPPP framework and concludes that while the framework is useful in supporting countries in their testing and updating of national pandemic preparedness plans it could have a stronger emphasis on identifying lessons and translating these into action plans with measurable outputs to drive change. The importance of this is highlighted by the case studies examined in this study which provide evidence for having a more direct link between real world events and simulations in order to improve pandemic preparedness.

The summary, which is presented in Table 2, demonstrates that while simulations are useful as part of a preparedness strategy, the key is to ensure that lessons from these simulations are learned and the associated changes made as soon as possible following any simulation in order to ensure that simulations are effective in bringing about changes in practice that will improve pandemic preparedness. Indeed, the use of simulation exercises that can create real world immersive environments is an essential part of these preparedness activities. Furthermore, AI technologies could also be applied in preparing communities for outbreak detection, surveillance and containment and therefore useful as a tool for providing immersive environments for simulation exercises for pandemic preparedness and associated interventions and particularly useful at the strategic level. As part of a strengthened framework for pandemic preparedness a systematic and integrated approach that includes the key elements of a lessons learned system should be adopted to provide evidence and assurance that lessons identified are learned [55]. Further work is required in this area, including the use of technologies, such as AI, to deliver an integrated approach to training, simulation exercises and embedding learning and driving effective change.

The potential for a health crisis to severely stretch the resources of a health system was demonstrated by Exercise Cygnus. Therefore, in planning for future health crises, we need to plan for an escalation in the response to novel viruses with the potential to cause a pandemic with a resultant increase in infections and a surge in demand across the health and social care sector. More importantly, it is imperative to build such planning into pandemic preparedness simulation exercises that inculcate the components of the WHO EPPP framework in order to achieve effective response to novel health crisis like the CoVID-19 pandemic.

This article contributes to the limited literature in pandemic preparedness simulation exercising to deal with novel health crises, like CoVID-19. The analysis has also identified potential areas for further research or work on pandemic preparedness exercising. First, there is need to understand effective ways of achieving a systematic approach to monitor the benefits of real-world simulations and ensure that improved practices from lessons learned can be sustained over the long term. Second, it is imperative to understand which methods used for simulations are the most effective at taking forward the lessons learned in order to effectively influence policy, governance and practice. Third, more work is required to assess the use of computer simulation exercises developed within the framework of the WHO's EPPP framework or similar pandemic preparedness planning frameworks to assess utility in pandemic preparedness. Lastly, when developing exercise simulations in the future it is important to incorporate technological solutions to enhance the simulation exercise by providing immersive environments for the participants.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.