Recommendations for Modeling Disaster Responses in Public Health and Medicine: A Position Paper of The Society for Medical Decision Making

Abstract

Purpose

Mathematical and simulation models are increasingly used to plan for and evaluate health sector responses to disasters, yet no clear consensus exists regarding best practices for the design, conduct, and reporting of such models. We examined a large selection of published health sector disaster response models to generate a set of best practice guidelines for such models.

Methods

We reviewed a spectrum of published disaster response models addressing public health or healthcare delivery, focusing in particular on the type of disaster and response decisions considered, decision makers targeted, choice of outcomes evaluated, modeling methodology, and reporting format. We developed initial recommendations for best practices for creating and reporting such models and refined these guidelines after soliciting feedback from response modeling experts and from members of the Society for Medical Decision Making.

Results

We propose six recommendations for model construction and reporting, inspired by the most exemplary models: Health sector disaster response models should address real-world problems; be designed for maximum usability by response planners; strike the appropriate balance between simplicity and complexity; include appropriate outcomes, which extend beyond those considered in traditional cost-effectiveness analyses; and be designed to evaluate the many uncertainties inherent in disaster response. Finally, good model reporting is particularly critical for disaster response models.

Conclusions

Quantitative models are critical tools for planning effective health sector responses to disasters. The recommendations we propose can increase the applicability and interpretability of future models, thereby improving strategic, tactical, and operational aspects of preparedness planning and response.

1. BACKGROUND

Disasters such as the 2001 U.S. terrorist attacks, Hurricane Katrina, and the 2008 Chengdu earthquake have highlighted the need for improved public health and medical disaster response capabilities at local, regional, national, and international levels. Disasters are commonly defined as events whose consequences exceed the capability of local or regional medical and public health systems to provide necessary responses in a timely manner (Figure 1). We use the term “health sector disaster response” to encompass all activities of medical and public health systems affected by a disaster, including treatment of existing casualties and prevention and treatment of future casualties. Major unresolved issues remain at all levels of disaster response decision making, including long-range strategic planning, tactical response planning, and real-time operational support.^* For example, how should the logistical systems for response to various types of public health disasters be organized? How much hospital-based surge capacity might be needed for potential mass casualty events? What is the most efficient and effective way to rapidly dispense medications or vaccines to large numbers of individuals?

Figure 1. Health System Disaster Response Modeling.

This figure presents the target of health system disaster response modeling. We distinguish between unexpected events and disasters. An unexpected event (e.g., hurricane, earthquake) may not necessarily result in a disaster. A disaster (e.g., war, famine) may not necessarily be an unexpected event. Similarly, public health activities include a broad range of activities, including some (e.g., asthma prevention) that are not relevant to health sector disaster responses. Medical activities, likewise, include a broad range of activities (e.g., routine patient care) that are not relevant to disaster responses. The focus of this paper is models of public health and medical activities that address unexpected disasters.

Increasingly, simulation and mathematical modeling are being used to explore the capabilities, costs, and consequences of public health and medical disaster response plans. Members of the Society for Medical Decision Making and its journal have been active in this nascent field, publishing a variety of models including those that address the epidemiological and medical consequences of bioterrorism-related illness and pandemic influenza, logistical challenges during disaster responses, and the cost-effectiveness of specific disaster response strategies.^1–5 Recently, the U.S. Department of Health and Human Services and the Centers for Disease Control and Prevention created initiatives to support modeling projects of relevance to their missions. However, the field of health sector disaster response modeling lacks a coherent thematological or methodological organization. Such organization could assist modelers in understanding and addressing the requirements of end users, facilitate consistent and systematic reporting of model inputs and results, and promote the development of standardized methodological approaches -- just as the Report from the Panel on Cost Effectiveness in Health and Medicine provides guidance for the field of cost-effectiveness analysis.⁶

This paper describes the range of models that have been developed for public health and medical responses to disasters and proposes a set of recommendations for developing and reporting the results of these models—something that has been lacking in the literature. This paper addresses three questions: 1) What kinds of health sector disaster response models have been developed? We describe exemplary and representative articles from the literature on applied decision support modeling for health sector disaster response, paying particular attention to the ways in which these articles differ from classic cost-effectiveness studies. 2) What methodologies have been and might be used for such models? We discuss the use of a range of modeling techniques for a variety of decision problems. 3) What are best practices for modeling and reporting of health sector disaster responses? We provide recommendations that we hope will serve as guides for the development of models focused on public health and medical responses to disasters; for funders of relevant modeling efforts; and for journals interested in publishing the findings of these models.

2. METHODS

We sought to identify best practices in modeling health sector disaster responses and the reporting of such models. We focused on models that can support informed decision making about responses to public health disasters. We did not perform a systematic review; instead, we sought to identify a range of articles that used a variety of modeling methodologies to address a broad range of disasters and response decisions. The focus of our analysis was on modeling methodologies, model quality, and the reporting of model analysis and results—but not on the specific planning insights from the models. Thus, we do not present detailed results of analyses performed with the included models.

2.1. Literature Search

We searched for published models from relevant sources including MEDLINE, International Abstracts in Operations Research, content experts, and the authors of included models. We also searched the tables of contents of relevant journals in such diverse fields as economics, industrial engineering, operations research, nuclear technology, and risk analysis (e.g., Operations Research, Management Science, Risk Analysis, European Journal of Operational Research, Interfaces, Journal of Intelligent Transportation Systems, Nuclear Technology, Decision Analysis). From recent U.S. National Preparedness Guidelines,⁷ we identified a variety of disasters that could have a significant effect on public health.

We identified 567 potentially relevant articles, reviewed 120, and included 67 articles describing 55 models. The models we selected for inclusion are representative of the literature and are not intended to be exhaustive; however, they directly inform our recommendations. We included only articles that describe a quantitative model designed to support informed decision making about public health or medical responses to disasters. We excluded highly stylized mathematical models that have little immediate practical application (e.g., those focused on the mathematics of infectious disease progression), non-peer reviewed models and those presented in abstracts only, and articles not written in English.

From each included article, we collected information about the purpose of the model, type of disaster considered, aspect of response the model was designed to address, modeling approach, model setting, decision makers included in the model, decisions modeled, and model outputs. We also collected information about the quality of each model and its reporting based in part on established criteria for the evaluation of healthcare models (e.g., whether all relevant model inputs and their sources were described, whether all assumptions of the model were explained).⁶

2.2. Development of Recommendations

As a team, we developed criteria for characterizing high quality modeling and model reporting and then developed a set of recommendations addressing the design, conduct, and reporting of such models. During these detailed discussions, we drafted and then revised the recommendations in an iterative manner. There were no significant disagreements among the writing team regarding model classification or resulting recommendations.

2.3. Critical Review by Content Experts and Members of the Society for Medical Decision Making

We sought critical feedback from content experts and members of the Society for Medical Decision Making (SMDM) on drafts of this position paper. We presented our results at the 2008 SMDM Annual Meeting in Philadelphia and the 2008 INFORMS conference in Washington, DC, and posted a draft of this paper on the SMDM website for additional comments. The final paper incorporates these comments.

3. RESULTS

3.1. Response Models

Table 1 describes the included studies and outlines the purpose of each model, the disaster considered, the model design, and outputs. Included models address a variety of decision makers (e.g., first responders, hospital officials, planners, public health officials), geographic settings (e.g., local, regional, national), decisions modeled (e.g., dispensing, inventory/stockpiling, supply chain network design, prevention or mitigation of disaster effects, treatment, health care work force staffing, transportation), and outcomes evaluated (e.g., costs, morbidity, mortality, quality-adjusted life years, hospital utilization measures, logistical outcomes), and use a range of modeling methodologies. The quality of modeling methodology and model reporting was variable (Figure 2). The following sections provide an overview of response decisions modeled, outcomes evaluated, and modeling methodologies used in these selected papers.

Table 1.

Description of Selected Health Sector Disaster Response Models

Author (Reference)	Purpose of Model/Study	Modeling Methodology	Disaster Evaluated†	Decision Makers Considered*	Geographic Setting	Decisions Modeled§	Outcomes¶
Disease Outbreaks
Braithwaite, Fridsma, Roberts1	To assess the cost-effectiveness of pre-exposure anthrax vaccination vs. an emergency surveillance and response system	Dynamic compartmental model	Anthrax	P	Local and Regional	Rx, P	$, QALY, M
Hupert, Mushlin, Callahan3, 9	To determine staffing levels necessary to maintain throughput requirements for antibiotic dispensing centers in the aftermath of a bioterrorism attack	Simulation (Discrete event)	Anthrax	P, PH	Local	D, S	L
Lee and others34, 35	To construct and implement a real-time decision support system for planning antibiotic dispensing in response to a large-scale disease outbreak	Simulation, Optimization	Anthrax	P, PH	Local (Urban setting)	D, S	L
Brookmeyer, Johnson, Bollinger11, 79	To assess the optimum duration of antibiotic prophylaxis and evaluate varying prophylactic strategies (pre or post-exposure vaccination, antibiotic prophylaxis) for anthrax response	Competing risks, probability/mathematical model	Anthrax	P, PH	Local	Rx, P	M
Wein, Craft, and others66–68	To evaluate the effectiveness of several response strategies for anthrax (pre-exposure vaccination, achievable levels of distributed antibiotic prophylaxis, biosensor efficacy)	Multi-tiered mathematical model	Anthrax	P	Local (A large city)	I, ND, Rx, P	L, M
Fowler and others62	To assess the cost-effectiveness of vaccination (pre- or post-exposure) vs. post-exposure antibiotic prophylaxis response strategies for anthrax	Simulation (Decision analytic)	Anthrax	P	Local (A large US metropolitan area)	Rx, P	$, QALY, M
Schmitt and others63	To evaluate the cost-effectiveness of vaccination (pre- or post-exposure) vs. post-exposure antibiotic prophylaxis response strategies for anthrax response	Simulation (Markov model)	Anthrax	P	Regional (Attack via US Postal Service)	Rx, P	$, QALY, M
Zaric, Bravata, Brandeau, and others5, 41, 52	To evaluate the cost-effectiveness of alternative strategies for maintaining and dispensing antibiotic inventories (local vs. regional) and communication with the public during an anthrax response	Dynamic compartmental model	Anthrax	P	Local (A large US metropolitan area)	I, ND, P	$, QALY, M
Whitworth40	To evaluate plans for anthrax response (e.g., number and location of dispensing centers, dispensing strategies, staffing plans, and traffic-management plans)	Simulation (Discrete event)	Anthrax	P	Local	D, S	L
Medema and others60	To evaluate health and economic outcomes of interventions for pandemic influenza (e.g., increasing the vaccine supply through egg-based or cell culture, provision of antivirals)	Simulation	Pandemic influenza	P, O	National	Rx	$, L, M
van Genugten, Heijnen, Jager54	To estimate (using FluSurge4) hospitalizations and deaths in the Netherlands from pandemic influenza, as a function of response strategy (no intervention, vaccinate high-risk individuals, vaccinate all, treat symptomatic people with antiviral drugs)	Spreadsheet	Pandemic influenza	PH	National (Netherlands)	P, Rx	M, H
Longini and others56	To investigate the effectiveness of targeted use of antivirals to contain the first wave of an influenza pandemic in the United States (before a vaccine can be developed)	Simulation (Stochastic, discrete time, network of 2000 individuals)	Pandemic influenza	PH	“A typical American community”	P, Rx	O
Meltzer, Cox, Fukuda57	To estimate outcomes of pandemic influenza (illnesses, deaths, etc.) and the effects of potential vaccination strategies, and to determine how much should be spent each year to plan/prepare for mass vaccination	Simulation (Monte Carlo)	Pandemic influenza	P	National (US)	P	$, M, H
Eichner and others53	To evaluate the impact of three types of interventions on pandemic influenza outcomes: antivirals, social distancing, and contact reduction	Compartmental epidemic model (Deterministic)	Pandemic influenza	P	National, Regional, or Local	P	M, L, H
Wilson, Mansoor, Baker26	To estimate population health and economic impacts of the next influenza pandemic in New Zealand	Deterministic model	Pandemic influenza	P	National (New Zealand)	P	$, H
Zhang, Meltzer, Wortley4	To estimate the impact of pandemic influenza on hospital services	Spreadsheet	Pandemic influenza	H, P	Regional	Rx	H, M
Soberiaj and others25	To estimate the impact of pandemic influenza on hospital services at the William Beaumont Army Medical Center in El Paso, Texas	Spreadsheet	Pandemic influenza	H, P	Local	Rx	M
Siddiqui and Edmonds59	To evaluate the cost-effectiveness of antiviral stockpiling and near-patient testing (rapid diagnostic tests at point of care) for an influenza pandemic in the United Kingdom	Spreadsheet (Incorporates a decision tree; allows for probabilistic and other sensitivity analyses)	Pandemic influenza	P, PH	National (UK)	I, Rx	$, QALY
Balicer and others58	To evaluate the cost-benefit of three different strategies for the use of stockpiled antiviral drugs during an influenza pandemic: therapeutic; long-term pre-exposure prophylaxis (PrEP), and short-term PrEP	Spreadsheet	Pandemic influenza	P, PH	National	I, Rx	$
Germann and others36	To simulate pandemic influenza in the United States and evaluate the effect of potential mitigation strategies, including antivirals, vaccines, and modified social mobility (travel restrictions, school closures)	Simulation (Microsimulation of 281 million individuals in 2000-person subgroups)	Pandemic influenza	P	National (US)	P	M
Khazeni and others61	To estimate the cost-effectiveness of two control strategies for pandemic influenza: antiviral prophylaxis and prime-boost vaccination	Compartmental epidemic model (Deterministic)	Pandemic influenza	P, PH	Local (A large US metropolitan city)	P	$, QALY
Colizza and others80	To evaluate the effect of international travel restrictions and antiviral treatment on the worldwide spread of pandemic influenza	Compartmental epidemic model (Stochastic; linked models, one for each of 3100 cities/airports)	Pandemic influenza	P, PH	Global	P	M
Gupta, Moyer, and Stern64	To evaluate the cost-benefit of quarantine in controlling SARS	Mathematical (Deterministic)	SARS	PH	Local (Toronto)	P	$, M
Lloyd-Smith, Galvani, and Getz81	To evaluate the effects on SARS transmission within a hospital and a community of hospital-based contact precautions, quarantine, and isolation	Compartmental epidemic model (Stochastic)	SARS	H, PH	Local (Hospital and community)	P	M, O
Lipsitch and others82	To evaluate the effects on SARS transmission of quarantine and isolation measures	Compartmental epidemic model (Deterministic)	SARS	PH	Local	P	M, L, O
Massin and others55	To evaluate interventions for controlling a pneumonic plague outbreak: masks, quarantine, prophylaxis, travel restrictions	Compartmental epidemic model (Deterministic)	Plague	P	National, Regional	P	M
Kaplan, Craft, Wein83	To compare mass vaccination vs. ring vaccination for responding to a smallpox attack in a major US city	Compartmental epidemic model (deterministic)	Smallpox	PH	Regional	P	M
Meltzer and others70	To evaluate the amount of quarantine and vaccination (alone or in combination) that would be required to control a smallpox outbreak caused by bioterrorists, and to estimate the number of vaccine doses needed	Markov model (Spreadsheet)	Smallpox	P	Local	P	M
Miller, Randolph, Patterson84	To evaluate the effects on health and the healthcare system of strategies for responding to a smallpox attack, including vaccination (mass vaccination or ring vaccination), social distancing measures, and quarantine	Simulation (Discrete event, modeling individual people)	Smallpox	P	Local	P	M, H
Glasser and others71	To evaluate the effects of a variety of smallpox control strategies, including isolation of infectives, vaccination of healthcare workers, general vaccination, ring vaccination, and school closure	Compartmental epidemic model (Deterministic)	Smallpox	P	Local	P	M
Porco and others85	To evaluate the effects of contact tracing and ring vaccination in controlling smallpox	Simulation (Discrete event, network of households and workplaces/social groups)	Smallpox	P, PH	Local (A community with households, workplaces, social groups)	P	M
Riley and Ferguson86	To assess the efficacy of symptomatic case isolation, contact tracing with vaccination, and mass vaccination in controlling a smallpox outbreak	Simulation (Individual-based, incorporates spatial factors)	Smallpox	P	National (Great Britain)	P	M
Natural Disasters
Barbarosoğlu and Arda48	To determine the most efficient flow of relief supplies in a transportation network in the aftermath of a rapid-onset disaster	Optimization (Stochastic programming)	Earthquake	P, FR	Local (Urban setting)	ND, T	L
Balcik and Beamon39	To determine the number and location of global distribution centers for stockpiled relief items, as well as the quantity of those items to be maintained, in order to improve disaster response	Optimization (Linear and dynamic programming)	Earthquake	P	Global	ND, T	L
Fawcett and Oliveira15	To estimate the impact of health facility damage, rescue time, and out-of-region transportation on overall mortality from an earthquake	Simulation	Earthquake	H, P, PH, O	Local (Lisbon, Portugal)	T, Rx	L, M,O
Paul and others16	To estimate the transient patient surge at regional hospitals resulting from an earthquake	Simulation (Discrete event with regression-based parameters)	Earthquake	H, P, PH, O	Regional	ND, S, T, Rx	L, H
Regnier17	To determine, for specific locations in the United States, the relationship between hurricane track prediction accuracy and lead time for evacuations	Simulation (Markov model)	Hurricane	P	Local (Four U.S. coastal cities)	T	L, $
Özdamar, Ekinci, Küçükyazici47	To apply vehicle routing and multi-commodity network flow techniques to develop an algorithm for efficiently dispatching relief supplies to a community affected by a rapid-onset disaster	Optimization	General natural disaster	P	Regional	T	L
Manmade Disasters
Beamon and Kotleba18, 19	To develop inventory management models (order quantities and reorder points) to aid sustained humanitarian response to complex emergencies	Optimization (Simulation used to test model in a case study)	Conventional warfare	P	Regional	I	L
Papazoglou and Christou21	To determine the best short-term emergency response to a nuclear accident, considering the tradeoff between adverse health effects and costs	Optimization (Multiobjective)	Nuclear	P	Regional	P	M, $
Feng and Keller22	To evaluate different plans for distribution of potassium iodide after release of radioactive iodine caused by a nuclear accident or terrorism	Optimization (Multiobjective decision analysis)	Nuclear	P, PH	Regional	D, I, P	O
Dombroski and Fischbeck49, 50	To evaluate strategies (e.g., caring for patients at the bomb site vs. evacuation) for response to a “dirty bomb” (a conventional explosive wrapped in radioactive material)	Dispersion Model	Radiologic	FR, PH, O	Local	P, O	M, O
Georgopoulos and others51	To evaluate key parameters affecting the exposure of healthcare workers to hazardous materials from contaminated patients	Simulation	Chemical	H, P, FR	Local	P	M, O
Inoue, Yanagisawa, Kamae20	To determine how to increase patient survival rates after a large-scale disaster through improvements in triage and transport procedures	Simulation	Airport accident	FR, PH	Local (Urban airport)	T, O	M
Christie and Levary45	To develop a scenario planning tool for use in the event of a manmade rapid-onset disaster to effectively assign and transport patients for treatment	Simulation	Airplane crash in urban area	FR, PH	Local (Urban setting)	T, O	L
Hospital Planning
Levi and others27– 29, 53	To evaluate Israeli hospitals’ disaster capacity and plans, train decision makers, and assist in managing real situations by identifying bottlenecks and evaluating a variety of response strategies	Simulation	Mass casualty events (e.g., conventional warfare)	H	Hospital	S, Rx, P, O	L, M, H
Kanter30, 31	To evaluate tradeoffs in pediatric hospital strategies that involve altering the standard of care and increasing ICU surge capacity	Simulation	Mass casualty events	H	Hospital	O	H
Earnest and others32	To predict the number of available isolation beds	Autoregressive moving average model	SARS	H	Hospital	O	H
Hupert and others33	To estimate overcrowding of emergency departments due to adverse events from rapid mass prophylaxis campaigns	Spreadsheet model	Smallpox, Anthrax	PH	Hospital	P	M,H
Other Types of Models
Han and others46	To determine efficient route and destination assignments for public evacuation after a large-scale disaster	Simulation	Large-scale disaster requiring evacuation of large urban area	P	Local (Urban setting)	T	L
Narzisi and others73	To analyze hospital capacity, public health preparedness and response, and behavior of the public during a rapid-onset urban disaster	Simulation (Agent-based)	General disaster (distributed or point-source)	P	Local (Urban setting)	H	L
Dekle and others38	To apply facility location techniques to identify potential disaster recovery centers for a local planning authority	Optimization (Integer programming)	General large-scale disaster	P	Local (County)	ND, T	L
Balcik, Beamon, and Smilowitz42	To determine the allocation of relief supplies and scheduling and routing of vehicles for the “last mile” distribution of supplies in response to a disaster	Optimization (Mixed integer programming)	General disaster	P	Local	T, O	$, O
Barbarosoğlu, Özdamar, Çevik43	To evaluate operational routing and loading decisions for helicopter dispatch during the aftermath of a disaster	Optimization (Mixed integer programming)	General disaster	P	Local (Urban setting)	T	L
Jotshi, Gong, Batta44	To develop efficient emergency vehicle routing strategies in the aftermath of a major disaster using available real-time data	Optimization	General disaster	P	Local (Urban setting)	ND, T	H, L

^† Disaster evaluated: Some of the included models apply to general disasters but for illustrative purposes used data from specific disaster types. Here, we have categorized them according to the specific type of disaster considered.

^*Decision makers considered: FR=first responders; H=hospital officials; P=planners (e.g., military planners, national-level emergency response planners); PH=public health officials; O=others (e.g., vaccine manufacturers)

^§ Decisions modeled: D=dispensing; I=inventory/stockpiling; ND=supply chain network design; P=prevention or mitigation of the disaster effects (e.g., vaccination strategies, quarantine, isolation, prophylaxis); Rx=treatment; S=healthcare workforce staffing; T=transportation; O=others (e.g., financing, traffic management)

^¶Outcomes: $=costs; H=hospital utilization measures (e.g., bed capacity); L=logistical outcomes such as inventory levels or queue lengths; M=morbidity or mortality; QALY=quality-adjusted life years; O=others (e.g., probability of containment).

Figure 2. Model and Reporting Quality.

This figure presents the summary assessment of the key quality criteria applied to the included models and the reports describing them.

3.1.1. Response Decisions Modeled

We identified models that address the following broad categories of response decisions: treatment and control of disease outbreaks, response to natural disasters, response to manmade disasters, hospital planning, and disaster response logistics. These categories are not mutually exclusive (for example, a model examining response to a natural disaster could focus on logistical issues). Also, models that focus on treatment and control of disease outbreaks are, of necessity, significantly different from models that focus on responses to natural and manmade disasters; however, all of these disasters require public health and medical responses.

Treatment and control of disease outbreaks

A variety of models have addressed preparation for and response to noncommunicable diseases such as anthrax or communicable diseases such as pandemic influenza, bioterrorist smallpox, and SARS. These models assess the impact of a variety of mitigation strategies, including vaccination, prophylaxis and treatment with antimicrobials, social distancing measures (e.g., masks, school closures), quarantine, travel restrictions, and vaccine development. Several notable models of disease outbreaks have been validated with available data from relevant prior outbreaks in humans^{3, 8, 9} and animal experiments.^{10, 11} In one study, researchers from three different groups compared the results of their pandemic influenza simulation models.¹² They noted that, “the results need to be viewed more as helping structure thinking about pandemic planning, rather than being predictive of the precise effectiveness of different policies.”¹²

Responses to natural disasters

Despite the frequency and potentially catastrophic health consequences of natural disasters (e.g., in 2007, 406 natural disasters killed 17,000 people and affected another 201 million people¹³; and in 2004, the Boxing Day tsunami killed more than 200,000 people¹⁴), remarkably few published models have focused on the public health and medical responses to such events. Several of the included models focused on earthquakes,^{15, 16} including one that is geographically customizable and allows the user to estimate the impact of health facility damage, rescue time, and out-of-region transportation on overall mortality.¹⁵ One study of evacuation timing in response to the threat of a hurricane presented receiver operating characteristic curves for four coastal U.S. cities comparing the probability of correct landfall prediction against the probability of a false alarm.¹⁷

Response to manmade disasters

Relatively few models have addressed response to manmade disasters such as conventional warfare, radiologic and chemical accidents, acts of terrorism, and large-scale industrial accidents. Some manmade disasters such as war and famine have a slow onset, and may require sustained responses in which relief demands may have occasional spikes.^{18, 19} One model of famine in Sudan used data from the ongoing crisis there to determine the optimal reorder points and order quantities for relief supplies.¹⁸ The authors used simulation to evaluate the interdependence of critical response variables, and reported key insights that humanitarian agencies can use to improve disaster response.¹⁹ A study of a hypothetical airport accident used historical data and interviews with responders to simulate the relationships among transport time, triage classification, and patient survival rates.²⁰

Several models have evaluated public health responses to nuclear incidents.^{21, 22} In general, these models evaluate tradeoffs among different accident mitigation objectives (e.g., minimize adverse health effects, minimize costs). One study of competing plans for distribution of potassium iodide after the release of radioactive iodine from a nuclear accident or terrorism showed how the preferred distribution plan changes as a function of the weights associated with different objectives.²²

Hospital planning

Hospital planning models for public health disasters have estimated demand for health care, evaluated triage strategies, planned hospital capacity, and developed appropriate protocols for decontamination and the use of personal protective equipment. Some of these models have been made publicly available in easy-to-use spreadsheets for use by planners for estimating demand for both inpatient and outpatient hospital services,^{4, 23} and have been used to estimate response capability in a variety of specific settings.^24–26

Several models investigate hospital capacity during disasters and, importantly, evaluate potential response bottlenecks at key sites (e.g., emergency departments, operating rooms) and for key resources (e.g., specialized staff). ^27–33 One model has been used to evaluate a hospital’s disaster plan, train decision makers, and assist in managing real situations by identifying bottlenecks and evaluating a variety of response strategies.^27–29 One model evaluates tradeoffs in hospital strategies that involve altering the standard of care and increasing ICU surge capacity.^{30, 31} Another model estimates expected overcrowding of emergency departments due to adverse events from rapid vaccination campaigns.³³

Response logistics

Logistical decisions are integral to many aspects of disaster response. These include decisions about procurement, transport, stockpiling, and maintenance of needed supplies; mass vaccination, prophylaxis and treatment; transportation and evacuation; and assignment of response personnel.

For many disasters, needed supplies must be rapidly dispensed to large populations. Several models evaluate the location, design, and operation of mass dispensing centers and other supply facilities (e.g., staffing, facility layout).^{3, 9} One publicly available model has been widely utilized by public health officials to plan dispensing of antibiotics, vaccines, and other disaster supplies in response to anthrax bioterrorism.^{3, 9, 34, 35} Another model, a decision support system that evaluates facility layout and staffing allocations for emergency mass dispensing, was tested in a field exercise conducted in Georgia.^{34, 35}

Dispensing antimicrobials, vaccines, and other materials during a large-scale disaster response places enormous strain on local health sector infrastructure; conversely, the limits of that infrastructure may influence the course of the disaster. Thus, it is critical that models examining such scenarios evaluate the feasibility of modeled interventions. One pandemic influenza planning model considered the impact of vaccinating 10 million U.S. residents per week over the course of a multi-month pandemic response.³⁶ An unresolved issue is how or whether such vaccination would be feasible given the public health and medical infrastructure, vaccine supply chains, and other aspects of current response logistics in the U.S.

Other important logistical decisions relate to the design and management of the disaster response supply chain including, for example, the location of supply centers, disaster recovery centers, and dispensing sites.^37–39 Two notable models incorporate multiple components of the anthrax response supply chain: one considers local level components (e.g., dispensing points, dispensing processes, staffing and traffic management plans)⁴⁰; the other considers national, regional, and local components.^{5, 41} Integrated models such as these can help identify the critical elements needed for efficient performance of a disaster response supply chain.

A variety of routing and transportation problems arise in response logistics planning. Models have been developed to determine the allocation of relief supplies and scheduling and routing of vehicles for the “last mile” distribution of supplies in response to a disaster,⁴² evaluate operational routing and loading decisions for helicopter dispatch in the aftermath of a disaster,⁴³ optimize the transportation of triaged casualties to local hospitals for treatment,⁴⁴ determine efficient ways to transport patients to hospitals, and model the evacuation of a populated area in anticipation of a disaster.^{20, 45, 46} Some studies investigate alternative routing of supplies after disaster-related damage to local infrastructure (e.g., bridge destruction after earthquakes), notably using real-world data to inform model inputs.^{44, 47, 48} A critical factor that can affect disaster response logistics is the behavior of the public and first responders. Several included models have considered such behavioral factors: for example, radiologic exposure after a dirty bomb and timeliness of care to patients as a function of use of cumbersome personal protective equipment;^{49, 50} contamination risk after chemical terrorism as a function of the behavior of the public and first responders;⁵¹ and costs and benefits of alternative communication strategies during an anthrax response as a function of the behavior of the public in response to public health instructions.⁵²

3.1.2. Response Outcomes Evaluated

The included models considered a wide variety of outcomes. For example, in addition to disease incidence and prevalence,⁵³ the included models of infectious disease response considered outcomes such as hospitalizations,⁵⁴ outpatient visits,⁵³ deaths,^{54, 55} number of doses of drugs or vaccines required,⁵⁵ and chance that a pandemic is averted.⁵⁶ Hospital planning models typically evaluate the demand for various hospital services^{28, 29, 57} and patient waiting times,¹⁶ and often attempt to identify potential bottlenecks in the response.^{30, 31}

Some analyses include costs and attempt to measure the cost-effectiveness of various planning and response strategies. For example, researchers have evaluated the cost-effectiveness (or cost-benefit) of strategies for preparation for and response to influenza pandemics,^57–61 anthrax attacks,^{5, 52, 62, 63} and SARS.⁶⁴ One study estimated the financial burden of incorrect evacuation decisions in response to hurricanes.¹⁷ Comparing estimated health outcomes and costs associated with different disaster response strategies may promote the effective and efficient use of limited public health resources.⁶⁵

A number of the included response models did not consider intervention costs, sometimes as a result of direct proscription of such analyses by funding sources. In one analysis of the effectiveness of several response strategies for reducing inhalational anthrax mortality (e.g., pre-attack vaccination, antibiotic prophylaxis), the authors noted that their funders discouraged them from considering costs.^66–68 They concluded that they had “mixed success” influencing policymakers, partly because one of their key findings was perceived as “too expensive and logistically difficult to achieve.”⁶⁸

3.1.3. Modeling Methodologies

A variety of modeling methodologies have been applied to analyze disaster response decisions, with considerable variation in the type and complexity of mathematical and analytical techniques used. These include statistical analyses,^{32, 69} simple spreadsheet calculations,^{4, 54, 58} Markov models,^{17, 63, 70} epidemiological models,^{4, 53–55, 58, 61, 71, 72} supply chain and inventory management models,^{5, 18, 19, 41} facility location models,^{38, 39} and vehicle routing and network flow techniques.^{43, 44, 46–48} Some of the models are quite simple, and consider a relatively small number of entities (e.g., groups of individuals categorized by disease stage, or patients in a single emergency room),^{32, 55, 61, 71} whereas others simulate complex systems with thousands or even millions of individuals.^{36, 73}

Because of the inherent uncertainty and complexity of disasters and disaster response, many researchers have used computer simulation to model potential outcomes of a disaster under different scenarios: for example, to determine staffing requirements at dispensing centers³ and plan dispensing strategies^{34, 35} after a bioterror attack, to determine effective routes and assignments for evacuating a population,⁴⁶ and to evaluate strategies for transporting of victims of an airport disaster to hospitals.^{20, 45} One large-scale agent-based simulation models all individuals in a dynamically interacting urban population after a disaster such as that caused by biological, radiological, or chemical hazards.⁷³ Such a model can be used to investigate a variety of logistical and operational decisions, but must be interpreted in light of the large number of input assumptions required.

Some researchers have made extensive efforts when developing models to incorporate input from disaster response planners, and to build models that planners can use. For example, one spreadsheet model for planning mass dispensing and vaccination clinics was developed in collaboration with county health officials⁷⁴ and made available as a tool that emergency response planners can use to automatically generate spreadsheets customized for their own needs.^{75, 76}

3.2. Recommendations

Based on our evaluation of the included models, we propose six recommendations for best practices for modeling and reporting of public health and medical responses to disasters. Clearly, models for any purpose, disaster responses or otherwise, should be rigorous in their design, execution, and reporting. The leaders in the field of modeling health sector responses to disasters come from a wide array of disciplines (e.g., clinical medicine, public health, medical informatics, operations research). Typically, their analyses follow established guidelines for best practices in modeling and model reporting from their relevant disciplines (e.g., ^{6, 77}). The following recommendations identify specific additional features needed for modeling health sector responses to disasters that are not always included in other standards, and represent an effort to enunciate best practices for this new field of research and action.

Health sector disaster response models differ from many models in that they are designed to support ongoing planning endeavors. These range from long-term strategic decisions to immediate-term operational decisions. Thus, they might be used at one point for pre-event planning and later for decision making during an actual response. Moreover, in contrast to published reports of cost-effectiveness analyses that provide insights into a decision problem, disaster response models often need to be used for ongoing decision making by planners who must customize models for their own needs. To complicate matters, these planners typically do not have any particular modeling expertise—a fact that has critical implications for the design and reporting of such models.

Recommendation 1. Health sector disaster response models should be designed with stakeholder input to effectively address the spectrum of relevant real-world planning and response problems

A critical aspect of any disaster response model is that it be useful for addressing the practical planning problems faced by clinicians, first responders, and public health officials. Thus, such models should be able to realistically assess the health effects of disasters and the response capabilities of the public health and health care systems. To do so, these models should be designed to address realistic scenarios with inputs based on real-world data. Where data are not available, other related data from related disasters, drills and tabletop exercises, or expert opinion could be incorporated. For such models to be useful, their results should be validated to the extent possible—for example, by comparing model outputs to observed data from relevant disasters. Disaster response models should evaluate the feasibility (e.g., implementation logistics) of alternative response strategies proposed. We advocate the early involvement of key stakeholders as part of an iterative process to support the design of realistic and useful models.

Recommendation 2. Health sector disaster response models intended for use by response planners should include a user-friendly interface, capacity to customize model inputs to suit local needs, ability to quickly and easily perform sensitivity analyses, and ongoing user support

Some highly stylized mathematical models can provide critical insights into disaster response planning; however, for many disaster response planning endeavors, ongoing decision making by end users is needed. In this case, they must be designed so that they (or appropriate end-user versions of them) can be readily used by response planners. Health sector disaster response models can play an important role in ongoing decision making, in local settings by response professionals (e.g., public health officials, physicians, emergency responders, hospital management, military planners, vaccine and pharmaceutical manufacturers, among many others) who may not be analytically trained but have considerable ground-level expertise. Thus, the input data and assumptions of models need to be clear so that planners can evaluate the relevance of model results. Models must provide results quickly and allow users to easily perform sensitivity analyses on different response scenarios. Good models will also allow users to evaluate tradeoffs in strategies (e.g., evaluating the tradeoffs between investment in local stockpiles versus the development of local dispensing capacity, or downgrading the standard of care in hospitals versus enhancing hospital surge capacity for the critically ill). Finally, the usefulness of such models is greatly enhanced when they are publicly available for use by planners, who can customize them for their own situations.

Recommendation 3. Health sector disaster response models should strike the appropriate balance between simplicity and complexity so that they adequately represent real-world scenarios, but are still useable and interpretable by intended end-users

The appropriate level of complexity is difficult to define a priori; it is a function of the planning decision being addressed and the desired level of detail of the results. For example, models designed to consider dispensing center operations may need to provide detailed results, whereas models designed to evaluate broader strategic decisions such as comparisons of treatment options may reasonably provide less detailed results. Good models will be complex enough to provide credible insights into the planning problems they were designed to address while remaining as simple and tractable as possible, thus supporting the use of such models for decision making by planners.

Recommendation 4. Health sector disaster response models should include relevant outcomes which may extend beyond those typically considered in cost-effectiveness analyses; these may include timeliness and efficiency of response, resource utilization, evacuation timing, and/or measures related to behavior of responders and the public

Unlike traditional cost-effectiveness analyses, outcomes other than morbidity, mortality, and costs may also be important in the design or evaluation of a disaster response plan. Critical outcomes of disaster response may include resource utilization (e.g., hospital capacity usage, emergency department closure rates, and transportation network congestion) and other logistical measures (e.g., bottlenecks, evacuating timing); and measures related to behavior of responders and the public (e.g., patient adherence to prophylaxis and treatment, patient self-referral patterns). Unfortunately, few well-validated metrics exist for many of these key outcomes (this is an important area of active research). It may also be important to consider longer term outcomes such as the capacity for sustained response and long-term health effects (e.g., mental health effects) since plans for sustained responses may differ from those for rapid responses. Costs may not be particularly relevant for every decision that is analyzed. Modelers should carefully consider the appropriate outcome measures to include, as well as means for validating those metrics.

Recommendation 5. Health sector disaster response models should address the fundamental uncertainties in disaster scenarios including the likelihood and magnitude of an event, operational response capabilities, supply chain capacity and robustness, behavior of responders and the public, and counter-measure effectiveness

Public health disasters are rare events that may unfold in unpredictable ways. Thus, disaster response models must be designed to allow for comprehensive sensitivity analyses of uncertain model parameters and uncertain event scenarios. A notable feature of all preparedness activity is that plans must be made and paid for now in order to improve response to a wide range of possible future events, most of which will never occur. The cost-effectiveness of a disaster response plan is often critically dependent on the probability that the disaster occurs as well as its possible magnitude. One way to evaluate the effectiveness of a specific response plan under a variety of potential disaster conditions is with scenario planning.⁷⁸ Scenario planning can highlight critical aspects of disaster response plans (e.g., bottlenecks) and can assist in designing response plans with adequate capacity and redundancy. As appropriate, good response models will incorporate the uncertainties associated with human and behavioral factors. For example, it may be important to evaluate a variety of assumptions about the behavior of both the public and first responders during an event (e.g., willingness of first responders to come to work in a dangerous environment, selection of treatment facilities by the public, adherence to recommended treatment/prophylaxis, behaviors in response to evacuation orders).

Recommendation 6. Good reporting is particularly critical for health sector disaster response models. In this context, good reporting consists of, at a minimum: addressing the impetus for the study; defining critical assumptions; explaining modeling methodology; discussing key sensitivity analyses; making available public-use versions of the model; and identifying all relevant partners in model creation

Given the rarity of most disasters, the uncertainties involved in both the events and their response, and the considerable societal importance placed on effective disaster response, detailed reporting of modeling methodologies, assumptions, and data sources is essential. Good model reporting will enable planners to customize analyses for their own purposes and compare the results of different models. A flow chart describing the event sequence may be helpful for planners who are interested in implementing the model. Good reporting of disaster models also includes the presentation of sensitivity analyses that identify key uncertainties, and the development of policy statements around these results. Additionally, reports of disaster response models should include specific insights for planners, describe how the model could be applied to response scenarios other than those specifically considered, and identify model limitations and key needs for future data collection.

4. DISCUSSION

Recent large-scale disasters in the U.S. and internationally have underscored the need for effective and efficient public health and medical responses. Partly as a result of these events, the field of disaster response modeling has emerged. The growing literature of disaster response modeling includes a number of exemplary models which we present in this paper. This field differs in key ways from traditional cost-effectiveness and healthcare modeling, leading to specialized requirements for model design and reporting. These informed our six recommendations: disaster response models should 1) be designed to address real-world disaster response problems, 2) be made available (in some form) for use by planners, 3) strike an appropriate balance between computational complexity and usability, 4) evaluate relevant disaster response outcomes, which often extend beyond those considered in traditional cost-effectiveness analyses, 5) explore critical uncertainties, and 6) be presented in sufficient detail that their results can be fully interpreted.

These recommendations are intended to increase the usefulness of disaster response models. Some of these recommendations are similar to those in established standards for modeling in some areas. For example, in medical cost-effectiveness analysis, it is standard practice to explore the effects of critical uncertainties on the results of the analysis. However, this is not necessarily the case in other modeling disciplines such as logistics or operations research. Other recommendations, such as making models available for use by planners, are not widely included in any published guidelines for public health or medical modeling.

The nascent field of modeling public health responses to disasters is reminiscent of the state of medical cost-effectiveness modeling in the 1970s and 1980s: It is highly heterogeneous in terms of methodologic approaches, outcomes evaluated, and quality of presentation. This has led to different approaches in framing and analyzing the research questions. As a consequence, it is typically not possible to compare results across models—even across models that evaluate related problems (such as treatment strategies for inhalational anthrax). Just as standard guidelines have facilitated the comparison of medical cost-effectiveness models by encouraging modelers to adhere to a set of principles (e.g., the adoption of the societal perspective when appropriate, discounting both costs and benefits, etc.),⁶ we hope that our recommendations will similarly help to standardize approaches adopted by modelers of public health responses to disasters.

Quantitative models are essential to planning effective health sector responses to disasters. The recommendations we propose should increase the applicability and interpretability of future models, thereby improving strategic, tactical, and operational aspects of preparedness planning and response.