Systematic Review of Strategies to Manage and Allocate Scarce Resources during Mass Casualty Events

Abstract

Context:

Efficient management and allocation of scarce medical resources can improve outcomes for victims of Mass Casualty Events (MCEs). However, the effectiveness of specific strategies has never been systematically reviewed.

Objectives:

We analyzed published evidence on strategies to optimize the management and allocation of scarce resources across a wide range of MCE contexts and study designs.

Data Sources:

Our literature search included Medline, Scopus, Embase, Cumulative Index to Nursing and Allied Health Literature, Global Health, Web of Science®, and the Cochrane Database of Systematic Reviews, from 1990 through late 2011. We also searched the grey literature using the New York Academy of Medicine’s Grey Literature Report and key websites. We included both English and foreign language articles.

Study Selection:

We included studies that evaluated strategies employed in real-world MCEs or tested through drills, exercises, or computer simulations. We excluded studies that lacked a comparison group or did not report quantitative outcomes.

Data Extraction and Synthesis:

Data extraction, quality assessment, and strength of evidence ratings were conducted by a single researcher and reviewed by a second; discrepancies were reconciled by the two reviewers. Due to heterogeneity in outcome measures, we qualitatively synthesized findings within categories of strategies.

Results:

From 5,716 potentially relevant citations, 74 studies met inclusion criteria. Strategies included: reducing demand for healthcare services (18 studies), optimizing use of existing resources (50), augmenting existing resources (5), implementing crisis standards of care (5), and multiple categories (4). The evidence was sufficient to form conclusions on two strategies, although the strength of evidence was rated as low. First, as a strategy to reduce demand for healthcare services, Points of Dispensing (PODs) can be used to efficiently distribute biological countermeasures following a bioterror attack or influenza pandemic, and their organization influences speed of distribution. Second, as a strategy to optimize use of existing resources, commonly used field triage systems do not perform consistently during actual MCEs. The number of high-quality studies addressing other strategies was insufficient to support conclusions about their effectiveness because of differences in study context, comparison groups, and outcome measures.

Limitations:

Our literature search may have missed key resource management and allocation strategies due to their extreme heterogeneity. Inter-rater reliability was not assessed for quality assessments or strength of evidence ratings. Publication bias is likely given the large number of studies reporting positive findings.

Conclusions:

The current evidence base is inadequate to inform providers and policymakers about the most effective strategies for managing or allocating scarce resources during MCEs. Consensus on methodological standards that encompass a range of study designs is needed to guide future research and strengthen the evidence base. Evidentiary standards should be developed to promote consensus interpretations of the evidence supporting individual strategies.

Introduction

Mass casualty events (MCEs) generate large numbers of acutely ill or injured people who require immediate medical and/or mental healthcare.¹ MCEs may occur suddenly, as is typical of an earthquake, tornado, or bombing,² but they may also evolve gradually over hours, days or even weeks, as is typical of a hurricane, flood, epidemic, or a chemical attack.³

For the purpose of this review, we define an MCE as an event that produces demand for medical care that may outstrip the capacity of local or even regional healthcare systems to provide conventional standards of care to the affected population.⁴ Because MCEs typically occur with little or no warning, providers must have contingency plans to meet exceptional demands for care. If these measures prove to be inadequate, providers may need to re-allocate resources using a fair, equitable, and transparent process—a concept known as “crisis standards of care.”⁵

Many strategies have been suggested to optimize management and allocation of scarce resources during an MCE, including cancelling elective admissions, conserving and reusing resources, accessing resources from stockpiles, and implementing early Intensive Care Unit discharge protocols. These examples illustrate four broad categories of strategies that providers and policymakers might use to manage and allocate scarce resources during an MCE: (1) managing or reducing less urgent demand for healthcare services, (2) optimizing use of existing resources, (3) augmenting existing resources, and (4) implementing crisis standards of care (Table 1).

Table 1.

Conceptual Framework for the Review

	Resource Management Strategies under Conventional or Contingency Conditions			Implement Strategies Consistent with Crisis Standards of Care
	Manage/reduce less urgent demand for healthcare services	Maximize use of existing resources	Augment resources
Policy Makers	Implement community-based triage capabilities (e.g., phone, Web or SMS -based triage) Risk communication about when and where to seek treatment Ensure security of healthcare facilities Communicate triage and treatment guidelines to community-based providers	Provide legal protection for healthcare providers Provide situational awareness of resource needs and availability Expand scope of practice Relax regulatory obligations	Distribute supplies from stockpiles Exercise mutual aid agreements Request resources from federal government Open alternate care facilities Coordinate and distribute donated resources	Develop and disseminate administrative and clinical guidance regarding crisis standards of care Disseminate nationally-sanctioned activation criteria, clinical algorithms, and other decision support tools
Providers	Cancel elective admissions Triage non-urgent ED visits to off-site settings Use “reverse triage” to identify patients who can be discharged early, treated in a non-hospital setting, or moved to lower acuity level of care Use remote technologies to support home-based and off-site care	Conserve, reuse, adapt, and substitute resources Transfer patients to less affected hospitals Modify or extend staff shifts Activate stress management programs Reduce record-keeping requirements Re-purpose patient care and non-patient care areas	Access resources from provider stockpile Request supplemental resources from county or state, or federal governments Request resources from unaffected providers Recruit qualified personnel from outside labor force and/or unaffected areas	Use pre-established and pre-agreed upon decision tools to allocate critical resources in short supply (e.g., based on SOFA score) Implement less than definitive care (e.g., “damage control” surgery)

Source: The authors developed the conceptual framework drawing on reports published by the Institute of Medicine⁵ and the Agency for Healthcare Research and Quality.¹

The effectiveness of implementing specific strategies in each of these areas across a range of meaningful outcomes, particularly patients’ health, has not been systematically assessed. To help policymakers and providers identity the best options, the Office of the Assistant Secretary for Preparedness and Response (ASPR), U.S. Department of Health and Human Services, formally requested that the Agency for Healthcare Research and Quality (AHRQ) commission a systematic review of the evidence on the effectiveness of resource management and allocation strategies across a wide range of MCE contexts.

Methods

This evidence review was guided by a 2009 report issued by the Institute of Medicine (IOM) describing adaptive strategies for surge conditions arising from MCEs.⁵ We organized our work around the four broad categories of strategies mentioned above—a framework that accommodated the vast majority of strategies that were referenced in the IOM report. We sought to identify the best available evidence within each broad category. The full report with additional details on our methodology can be found elsewhere.⁶ The 27-item PRISMA Checklist guided the structure and the content of this report (See Appendix A).⁷

Data Sources

To identify relevant studies, we searched seven research databases, including PubMed, Scopus, Embase, Cumulative Index to Nursing and Allied Health Literature, Global Health, Web of Science®, and the Cochrane Database of Systematic Reviews, from 1990 through November 2011. To identify relevant books, we searched the National Library of Medicine’s online catalog. We supplemented these searches with a scan of the grey literature using the New York Academy of Medicine’s Grey Literature Report and websites of government agencies, provider organizations, and academic research centers. Appendix B contains a list of the literature search terms.

Study Selection

We considered both U.S. and international sources, including non-English language articles that met our two primary inclusion criteria. First, eligible studies had to prospectively assess a strategy during an actual event; document its impact in an after-action report; or test its effectiveness in an exercise, drill, or computer simulation. Second, to ensure studies met a minimum threshold of scientific rigor, they had to have compared an intervention with one or more alternative interventions, employ a control group, or use some other objective benchmark. Given the anticipated diversity of strategies, we considered a broad set of outcome measures, including health, cost, ethical, and legal outcomes, as well as process measures, such as triage accuracy rate and triage time. Because studies inconsistently reported the issuance of disaster declarations and often failed to fully characterize the extent of resource shortages, our review included both MCEs and quite likely some events that produced conditions of excessive surge that would not be classified as MCEs. Appendix C contains a list of excluded studies and the reasons for exclusion.

Data Extraction and Quality Assessment

Two researchers screened all titles. Abstracts and full-text articles underwent dual review. We resolved disagreement by consensus or when necessary, third-party reconciliation. Data were abstracted using the DistillerSR program. Core data elements included study design, geographic location, type of MCE, description of the strategy, outcomes, and facilitators and/or barriers to the implementation of each strategy.

After finding no suitable quality assessment tools in the peer-reviewed literature, we developed a 5-item scale that enabled comparisons of methodological quality across diverse types of studies. We combined key domains from three sources. From the National Registry of Evidence-based Programs and Practices (NREPP)⁸ quality assessment scale from the Substance Abuse and Mental Health Services Administration we used two items: (1) whether the strategy was implemented with fidelity (i.e., implemented consistently), and (2) whether the authors discussed potential confounders to the strategy’s effectiveness. The remaining three items represent core elements of two common frameworks commonly used to appraise the quality of qualitative research: those by Mays & Pope⁹ and Lincoln & Guba.¹⁰ These three items assessed whether or not: (1) the level of detail used to describe the resource allocation strategy was adequate, (2) data collection was systematic (and if so, whether it was retrospective or prospective), and (3) the authors assessed the generalizability of findings. Two reviewers reconciled all differences in scores for each quality item. For computer simulations, we included two items that assessed justification of model inputs and the robustness of sensitivity analyses (and eliminated the data collection item and implementation fidelity item). For systematic reviews, we used the 11-item AMSTAR instrument.¹¹

Data Synthesis and Analysis

Because of the breadth of topics we reviewed, we conducted systematic reviews within sub-categories of resource management/allocation strategies. We combined these results into a single report because our conclusions were largely similar across most categories. The data were not amenable to quantitative synthesis because abstracted studies rarely addressed similar strategies. Moreover, when multiple studies did assess a common strategy, they typically differed widely in their settings, comparison groups, and outcome measures. Accordingly, we summarized the results qualitatively, using the four broad categories from the conceptual framework shown in Table 1 and sub-categories when clusters of related strategies emerged. Wherever possible, we summarized the degree of consistency in the magnitude and direction of the most relevant outcomes. We also highlighted contextual and methodological differences that were relevant to the interpretation of results.

We graded the strength of evidence using the methodology commonly employed for systematic reviews commissioned by AHRQ.¹² It requires reviewers to consider four key domains: risk of bias, consistency, directness, and precision, then grade the overall strength of the evidence using a four-point scale (i.e., high, moderate, low, or insufficient). Grades were assigned by one reviewer and confirmed by a second, followed by discussion when conflicts arose. We rated the strength of evidence within the four domains for individual categories (or subcategories) of resource management/allocation strategies depending on the number of studies available. A single reviewer graded the strength of evidence for each domain within categories of strategies and overall. The ratings were then reviewed by a second researcher and differences were reconciled through discussion.

Results

Our search strategy identified 5,716 potentially relevant citations. After applying exclusion criteria, only 74 articles underwent data extraction (Figure 1). These 74 studies comprised three main types of analyses: 48 were intervention studies (i.e., involving human subjects), including 23 that evaluated the outcomes of drills and 25 that reported findings from actual MCEs (Table 2). Nearly half of the intervention studies (25 of 48) occurred in the United States. The other 23 took place elsewhere, particularly Europe (8) and Israel (6). The remaining 26 non-intervention studies included computer simulations (17), systematic reviews (2), validation analyses (5), and laboratory studies (2).

Figure 1.

Literature flow

Table 2.

Characteristics of Studies Included in the Review

	Number of studies (Percent of all studies)
All studies	74 (100)
Study type
Intervention studies*	48 (65)
Drills	23 (48)
Actual MCEs	25 (52)
Computer simulations	17 (23)
Systematic reviews	2 (3)
Validation analyses	5 (7)
Laboratory analyses	2 (3)
Study setting**
United States	25 (52)
Europe	8 (17)
Israel	6 (13)
Asia	3 (6)
Canada	2 (4)
Australia	1 (2)
Mexico	1 (2)
Rwanda	1 (2)
Haiti	1 (2)
Type of event
Pandemic influenza	13 (18)
Explosive	10 (14)
Multiple hazards	10 (14)
Anthrax	7 (9)
Natural disasters: earthquake	6 (8)
Natural disasters: hurricane	3 (4)
Nuclear/radiological events	3 (4)
Transportation accidents	3 (4)
Chemical events	3 (4)
Smallpox	2 (3)
SARS	1 (1)
Other	3 (4)
Unspecified events	10 (14)

^*We use the term “intervention study” to refer to any study that tested a strategy with human subjects, and includes both drills and actual MCE.

^**Among intervention studies only

Twenty-three studies focused on biological threats, including pandemic influenza (13), anthrax (7), smallpox (2), and SARS (1). Nine studies addressed natural disasters including 6 earthquakes (3 of which involved Hurricane Katrina). Ten reported outcomes following a terrorist attack with explosives. The remaining MCEs included: nuclear/radiological events (3), transportation accidents (3), chemical events (3), multiple hazards (10), unspecified events (10), and other types of MCEs (5).

Table 3 groups the studies according to the four broad categories of resource allocation strategies defined in our conceptual framework. In the sections that follow, we discuss only the studies relevant to distribution of “biological countermeasures” and “field triage,” the two subcategories for which the evidence was sufficient to form tentative conclusions about their effectiveness. We also summarize the available evidence on implementation of crisis standards of care—an area of particular interest to providers and policymakers. Appendix D contains detailed descriptions of each study.

Table 3.

Resource Allocation Strategies Included in the Systematic Review

Author	Study type	MCE type	Description of Strategies	Quality Score	Summary
Strategies to reduce or manage less-urgent demand for healthcare services
Allocation of biological counter-measures
Arora, 201039	Computer sim	Influenza	Optimize level of pre-allocation of antiviral stockpile and percentage allocated for prophylaxis and treatment for influenza	4/7	Non-comparable strategies  Three different biological threats studied  All studies were computer simulations  Insufficient evidence
McCaw, 200840	Computer sim	Influenza	Optimize the use of multiple drugs for prophylaxis and/or treatment	7/9
Wein, 200341	Computer sim	Anthrax	Implement prioritization policies, biosensor deployment, and/or deploy federal or military resources and volunteers	5/9
Bravata, 200642  Zaric, 200843	Computer sim Computer sim	Anthrax Anthrax	Increase local stockpiles, deploy stockpile to targeted areas, enhance detection, or increase dispensing capacity	7/9 3/9
Glasser, 201044  McVernon, 201045  Medlock, 200946	Computer sim Computer sim Computer sim	Influenza Influenza Influenza	Targeting vaccination to different populations	2/7 4/7 5/9
Zenihana, 201047	Computer sim	Smallpox	Mass vaccination, contact tracing, and school closure	3/7
Distribution of biological counter-measures
Ablah, 201013	Exercise	Anthrax	Centralized POD system vs. hybrid (“pull” and “push” approach)	6/8	Mail carrier strategy may dominate centralized model  Three large-scale exercises  Low strength of evidence
Koh, 200814	Exercise	Anthrax	Centralized POD system vs. “push” model utilizing U.S. Postal Service mail carriers	6/8
Lee, 200615	Exercise	Anthrax	POD operations supported by a decision-support software tool	4/8
Hupert, 200916	Computer sim	Anthrax	Use of dynamic staffing during POD implementation	2/7
Erwin, 200917	Actual MCE	Smallpox	Implementation of CDC mass vaccination protocol	4/8
Nonbiological counter-measures
Cahill, 200848	Computer sim	Influenza	Distribution of surgical masks or N95 respirators to the public	2/9	Each study evaluated a distinct strategy  Insufficient evidence
Schull, 200749	Actual MCE	SARS	Restrict non-urgent demand for hospital care	4/8
Savovia, 200950	Exercise	Influenza	Training public health officials in their legal authority to implement strategies to limit the spread of pandemics	4/7
Adini, 201051	Actual MCE	All-hazards	Automated central information distribution system for families	4/8
Strategies to optimize use of existing resources
Field Triage studies: Implementation Tests
Cohen, 199852	Actual MCE	Explosive	American College of Surgeons Committee on Trauma criteria	4/8	Few triage systems tested during real events;  Studies assessed different outcomes;  Few studies tested in real MCEs assessed mortality. Low strength of evidence
Cone, 200818	Exercise	Chemical	Combined trauma/CBRN-specific system	6/8
Rodriguez-Noriega, 201020	Actual MCE	Influenza	Influenza-Like Illness Scoring System	5/8
Aylwin, 200631	Actual MCE	Explosive	London transit bombings protocol	5/8
Kuniak, 200819	Exercise	Radiological	Radiation Injury Severity Classification	6/8
Cone, 200953 Lerner, 201054	Exercise Exercise	All-hazards Explosive	Sort- Assess- Lifesaving Interventions- Treatment/transport (SALT) protocol	5/8 5/8
Zoraster, 200721  Schenker, 200622	Actual MCE Exercise	Transportation accident Multiple	Simple Triage and Rapid Treatment (START)	4/6 6/8
Gutsch, 200655	Exercise	Unspecified	Modified START triage algorithm (mSTART)	4/4
Rehn, 201056	Exercise	Transportation accident	Interdisciplinary Emergency Service Cooperation Course (TAS Triage Method)	6/8
Cryer, 201057	Actual MCE	Transportation accident	Los Angeles Commuter Rail accident	5/8
Janousek, 199958	Exercise	Multiple	NATO triage classification system	3/7
Field Triage studies: Derivation/Validation Studies
Adeniji, 201159	Validation	Influenza	Simple Triage Scoring System (STSS)	3/6
Casagrande, 201160	Computer sim	Nuclear	Model of Resource and Time-based Triage	6/9
Cancio, 200861	Validation	Combat	Field Triage Score vs. Revised Trauma Score	4/6
Guest, 200962	Validation	Influenza	Christian et al. triage protocol	5/7
Navin, 200925  Sacco, 200727	Computer sim Computer sim	Unspecified Unspecified	Sacco Triage method	3/7 5/7
Nie, 201063	Actual MCE	Earthquake	Modified START (includes resuscitation category)	2/8
Beyersdorf, 199664	Actual MCE	Mass shooting	Revised Trauma Score vs. Pre-hospital Index vs. Washington state Pre-hospital Trauma Triage Procedure	2/6
Kilner, 201023	Sys Review	Multiple	Systematic review of triage systems	8/8
Field Triage studies: other
Hirshberg, 201065	Computer sim	Explosive	Role assignment during two-stage triage system	6/9
Romm, 201166	Laboratory Test	Radio/Nucl	Expedited biodosimetry method	5/5
Gao, 200767	Exercise	Unspecified	Electronic triage tags to monitor vital signs and transmit information to first responders	5/8
Load-sharing
Kanter, 200728  Epley, 200668  Leiba, 200669  Raiter, 200870  Xiong, 201071	Computer sim Actual MCE Actual MCE Actual MCE Computer sim	Unspecified Hurricane Explosive Explosive Earthquake	Existing regional systems to optimize allocation of patients to hospitals	3/9 4/8 2/8 3/8 2/7	Israeli home command system may not be generalizable to US  Three studies provide low-applicability evidence Insufficient evidence
Simon, 200172	Actual MCE	Explosive	Ad hoc incident command structures to optimize allocation of patients to hospitals	2/8
Wolf, 200973	Exercise	Unspecified	Load-sharing protocol for mass gatherings	8/8
Imaging
Beck-Razi, 200774  Sarkisian, 199175	Actual MCE Actual MCE	Explosive Earthquake	Focused assessment of sonography for trauma (FAST) for triage	6/8 4/8	Sonogram studies used distinct gold standards;  CT protocols tested in exercises only Insufficient evidence
Korner, 200676  Korner, 201177	Exercise Exercise	Unspecified Unspecified	Accelerated CT protocols	5/7 7/8
Medical Interventions
Gunal, 200478	Actual MCE	Earthquake	Medical interventions for the prevention of acute renal failure in crush victims	6/8	Studies addressed different topics Insufficient evidence
Vardi, 200479	Exercise	Chemical	Novel drug infusion devices	6/8
Space optimization
Satterthwaite, 201080  Van Cleve, 201181	Actual MCE Actual MCE	Explos/Trans accident Influenza	Reverse triage to create surge capacity	2/7 5/8	Only one of the two reverse triage protocols was truly tested Insufficient evidence
Scarfone, 201182	Actual MCE	Influenza	Conversion of lobbies, clinics, and other units to accommodate surge	2/8
Training
Hsu, 200483	Sys Review	Multiple	MCE response training (including drills, tabletop exercises, computer simulations)	7/10	Systematic review suggests drills are effective;  Most studies reported lessons learned;  Longer term effectiveness of training is unknown Insufficient evidence
Jarvis, 200984	Exercise	Unspecified	Game-based training	4/8
Andreatta, 201085  Vincent, 200886	Exercise Exercise	Explosive Unspecified	Virtual reality training	6/6 4/7
Vincent, 200987	Exercise	Explosive	Podcasts and multi-manikin simulations	3/5
Sanddal, 200488	Exercise	Explos/Trans accident	“JumpSTART” training session followed by drill	6/8
Other
Einav, 200989	Actual MCE	Explosive	Hospital-based case managers to ensure care coordination	3/8	Studies addressed different topics Insufficient evidence
Amlot, 201090	Exercise	CBRN	Multiple strategies to increase decontamination effectiveness	3/6
Loeb, 200937	Actual MCE	Influenza	Influenza prophylaxis for healthcare workers: surgical masks vs. N95 respirators	5/6
Strategies to augment existing resources
Temporary facilities					Three studies assessing temporary facilities shared same context: hurricane.  Potentially limited generalizability to other MCEs. Insufficient evidence
Eastman, 200791	Actual MCE	Hurricane	Alternate-site surge capacity facilities	4/7
Blackwell, 200792	Actual MCE	Hurricane	Deployment of mobile field hospital	3/5
Wein, 200341	Computer sim	Anthrax	Activating mobile provider units from other federal agencies to provide hospital surge capacity	5/9
Other
Arora, 201039	Computer sim	Influenza	Mutual aid agreements for the transfer of antivirals between counties	4/7
Corvino, 200693	Laboratory	Chemical	Conversion between formulations of nerve agents to augment supply	6/7
Strategies for implementing crisis standards of care
Damage-control surgery/care					Contexts include terrorist bombing, earthquake, civil war, and unspecified.  Limited measurement of patient outcomes  Insufficient evidence
Dhar, 200829  Labeeu, 199630	Actual MCE Actual MCE	Earthquake Combat	External fixation of fractures rather than definitive orthopedic care	5/8 1/6
Aylwin, 200631	Actual MCE	Explosive	Limited advanced on-scene interventions, delaying non-urgent CT scans, limited use of blood typing and cross-matching	5/8
Other
Merin, 201032	Actual MCE	Earthquake	Very early discharge from intensive care unit	1/6
Kanter, 200728	Computer sim	Unspecified	Essential interventions (unspecified) for pediatric MCE victims	3/9

Notes: Computer sim=computer simulation. The numerator of the quality score reflects the number of points achieved. The denominator reflects the total number of points for the applicable items.

Reduction of demand through rapid distribution of biological countermeasures.

Among the 18 studies that addressed strategies to reduce or manage demand for healthcare services, five examined the utility of “points of dispensing,” or PODs, to speed distribution of biological countermeasures. Three of the five studies used exercises to assess techniques for rapidly distributing medical countermeasures against anthrax. The first of these, an exercise conducted in Nassau County, NY, demonstrated that a traditional “centralized” POD system provided slightly faster throughput than a hybrid model. In the centralized POD model, individuals came to a fixed site to receive a medical countermeasure (a “pull” approach), whereas in the hybrid model, the “pull” approach was combined with a “push” strategy in which countermeasures were distributed to individuals at their work site.¹³

The second study compared a “pull” model to a different type of “push” model—one that used U.S. Postal Service mail carriers to deliver countermeasures to people’s homes.¹⁴ It found that using mail carriers served more people per hour per provider than did operating a fixed dispensing site. Although these two studies provide only limited evidence, both were relatively large-scale, high-quality exercises that were conducted in different geographic regions. Therefore, we judged them as providing highly applicable evidence.

The third study, a multi-county exercise in metropolitan Atlanta, documented that POD operations supported by a particular decision-support software tool were more efficient than traditional dispensing systems. However, the majority of findings from this study were qualitative in nature and therefore not comparable to the other results in this category.¹⁵ This same software tool was used to simulate different patient arrival patterns in another study of the effectiveness of PODs that concluded that a dynamic staffing model significantly reduces waiting times for individuals seeking countermeasures.¹⁶

The only POD study conducted during an actual MCE involved countermeasure distribution during a Hepatitis A outbreak in Eastern Tennessee. In this study, the community implemented a mass vaccination protocol originally developed by the CDC for anthrax and achieved benchmark levels of throughput.¹⁷

Although two exercise-based studies that compared different POD systems were judged to provide high-quality evidence, neither tested the approach during an actual event. Furthermore, only two of the five studies in this category were comparable, representing a very limited evidence base. Accordingly, the overall strength of evidence regarding the optimal approach to distributing biological countermeasures was rated as low.

Pre-hospital triage systems.

Among the 50 studies that evaluated strategies intended to optimize use of existing resources during an MCE, fully half focused on pre-hospital (also known as “field”) triage systems. Thirteen studies assessed the performance of responders who used these triage systems during a drill or an actual MCE, while 9 sought to validate new or existing triage systems through other design (e.g., medical record review). The other 3 examined issues related to triage, but were determined to be non-comparable. We concluded that the strength of evidence to favor use of any particular field triage system over others was low.

Implementation Tests of Triage Systems.

Thirteen studies of field triage systems examined their performance during actual (6) or simulated MCEs (7). Few of the triage tools assessed in these tests applied to pediatric victims, and only two studies addressed chemical, biological, radiological, or nuclear MCEs.^18,19 Most focused on adult populations involving victims of trauma and thus used quite similar triage protocols.

The reported accuracy of these systems ranged from 62 to 100 percent across the ten studies that included this information (Table 4). Studies reporting results from actual MCEs (two terrorist bombing events, including the London Transit bombings), suggest that over-triage rates may range from 33 to 64 percent. In another study, the use of “reverse” triage among patients with low scores on an influenza-like illness scale successfully increased surge capacity in a Mexican Emergency Department during the 2009 H1N1 pandemic.²⁰

Table 4.

Accuracy of Triage for Individual Triage Tools Reported in Ten Included Studies

Triage system	Study Design	Under-triage rate (%)	Over-triage rate (%)	Overall Triage accuracy (%)
ACS Committee on Trauma criteria52	MCE	1*	33	-
Influenza-like Illness Scoring system20	MCE	<1	-	-
London transit bombings triage method31	MCE	-	64	-
CBRN triage system18	Drill	11	2	-
Radiation Injury Severity Classification19	Drill	-	-	95
SALT54	Drill	10	6	83
SALT53	Drill	4	13	79
START22	Drill	-	-	70
START22	Drill	-	-	62**
mSTART55	Drill	3	5	-
TAS Triage method56	Drill	0	0	100

ACS=American College of Surgeons; SALT=Sort, Assess, Life-saving Interventions, Treatment/Support; Simple Triage and Rapid Treatment (START); mSTART=modified Simple Triage and Rapid Treatment; TAS=Interdisciplinary Emergency Service Cooperation Course; CBRN=Chemical/Biological/Radiological/Nuclear.

^*Rates moderately injured patients are reported. Under-triage rate for critical patients was 1% and for severely injured patients was 14%.

^**Accuracy of triage when clinical status was manipulated for 47 patients.

Note. Data from three systems were not amenable to synthesis.^21,57,58; 4 studies included prior training^18,53,54,56 and 1⁵³ included the use of aids which may have influenced triage accuracy rates.

The seven studies that used drills to evaluate triage systems reported higher rates of accuracy than studies examining the performance of these systems in actual MCEs. Two systems associated with higher levels of performance examined triage during chemical, biological, radiological or nuclear events¹⁸ or radiological events alone.¹⁹ However, only one of these systems was developed using data from patients experiencing acute radiation syndrome; the other was based on expert opinion. Four studies, including two that evaluated the accuracy of the Sort, Assess, Life-saving interventions, Treatment/Support (SALT) triage system, were immediately preceded by a formal training program, so it is unlikely that their results will accurately predict the performance of each system during an actual MCE.

The methodological shortcomings of drills may help to explain why several studies have found that field triage systems often face challenges when implemented in practice. For example, in a commuter rail incident, use of another well-known field triage tool, Simple Triage and Rapid Treatment (START), resulted in misallocation of patients between trauma centers and community hospitals due to provider confusion about the meaning of each triage category.²¹ In addition, one exercise found that the START triage system is not sensitive to serious non-trauma conditions, such as a myocardial infarction or an asthma attack, and may therefore under-triage victims with these conditions.²²

Non-implementation Studies Involving Triage Systems.

Nine studies reported data from validation analyses, but did not formally test one or more triage systems. One study was a systematic review comprising 11 articles on 8 triage systems.²³ Only one of the 11 studies used data collected from an actual MCE in which START was used to triage patients of a transportation incident and resulted in an over-triage rate of 53%.²⁴ The authors concluded that there is little evidence to favor one system over the others because of small sample sizes and low methodological quality of the studies, but the Sacco Triage Method (STM)^25–27 was described as “promising” because it takes the available capacity at receiving hospitals into consideration.

Adding eight additional validation studies of triage systems did not change the overall conclusion from the prior systematic review. Because the vast majority of these studies did not include evidence on how these triage systems performed under real or realistically simulated conditions, we judged the collection of studies to have very limited applicability. For example, the STM requires relatively advanced health information technology and a reliable communication system to develop and transmit triage instructions to providers. The feasibility of implementing such a system in an MCE is unclear.

Crisis Standards of Care.

Five studies evaluated resource allocation strategies that might be implemented under crisis standards of care. Three examined “damage-control surgery”—an approach to initial surgical treatment that strictly focuses on initial stabilization, with definitive repair deferred to a later time. The fourth study examined the usefulness of modified ICU admission and discharge policies. The fifth and final study in this subcategory used computer simulation to model the potential impact of restricting care to only “essential interventions,” but did not specify them.²⁸

Two of the damage-control surgery studies focused on orthopedic surgery, while the third examined trauma surgery more broadly. The first study reported that hospitals implementing “damage control” orthopedic surgery in the aftermath of an earthquake in Kashmir in 2008 expanded operating room capacity by 37 percent, with minimal impact on patient outcomes one year after the earthquake.²⁹ The second found that under battlefield conditions, early use of external fixation improves throughput but at the cost of a higher rate of complications, particularly surgical infections.³⁰ The third study noted that use of damage control surgery boosted surge capacity in the hospitals that received a sudden influx of complex trauma victims from the 2005 London transit bombings.³¹

The study that focused on ICU admission and discharge policies described the experience of the Israeli Defense Force field hospital, which cared for numerous victims of the 2010 Haiti earthquake.³² Faced with extraordinary demand, the hospital prioritized its ICU beds for patients who could be stabilized within 24 hours, and adopted a policy of accelerated discharge from intensive care. Although the authors concluded that these policies enabled them to treat a substantially larger number of victims than would otherwise have been possible, they did not systematically assess how the policies affected patient outcomes.

Limitations

To accommodate the vast and heterogeneous body of literature on this topic, we were obliged to make several methodological trade-offs. First, as the objective of our study was to conduct the first systematic review on the most effective resource management and allocation strategies during MCEs, we were unable to specify more precise research questions. For example, while we might have included only those studies that used a narrow set of outcome measures, such as survival, we erred on the side of being more inclusive. Second, because we were asked to identify resource allocation strategies across the full spectrum of preparedness and response options, we used a broad set of search terms to scan the literature. It is possible that our approach did not identify every important strategy or every key study for the strategies included in the review, because they were not specified as formal search terms. Third, in recognition that the evidence was likely to be sparse in many areas, we elected not to use exclusion criteria relating to study type. As a result, two existing systematic reviews were included in the review. Fourth, to accommodate a broad range of study types, we developed a quality assessment scale. While we did not calculate the inter-rater reliability of quality scores, the incidence of discrepant scores that required reconciliation between reviewers was frequent enough to suggest that the scale could be further refined. The scale should also undergo validity testing. Fourth, although the scope of our review was broad, it did not address every aspect of the management of MCEs, such as detailed clinical treatment and the technical considerations involved in inter-facility transport of critically ill and injured patients. Finally, although we did not conduct a formal assessment of publication bias, the majority of studies we found reported positive findings, suggesting that publication bias may be a factor

Discussion

In 2009, the IOM Committee on Guidance for Establishing Standards of Care for Use in Disaster Situations published a Letter Report recommending that healthcare providers, organizations, government officials, and the public approach the challenge of allocating scarce resources in a thoughtful and proactive way.⁵ Building on that report, the IOM recently described the core functions of stakeholders engaged in preparedness and response to MCEs, and enumerated specific tasks to help ensure that each stakeholder successfully fulfills these core functions.³³ To complement the IOM’s work, we undertook this systematic review to compile, for the first time, the best available evidence on strategies to manage and allocate scarce medical resources during MCEs. Our hope was that the resulting evidence base might be sufficient to improve existing disaster response plans and more closely align them within the IOM’s disaster response framework.^5,33

The need for better evidence is clear. It is only a matter of time before the United States will experience another natural or man-made disaster, a pandemic, or a terrorist attack that outstrips the capacity of our healthcare system to respond. When it occurs, healthcare workers and the policymakers that support them will be forced to make difficult decisions under highly demanding and generally quite public circumstances. Because resource allocation decisions are fraught with clinical, legal, and ethical implications, they should be grounded in rigorous evidence.

Unfortunately, our review found limited evidence to help providers and policymakers make such decisions. Within all but a few categories, individual strategies were evaluated by no more than three studies. Most of the studies we reviewed were prone to at least a moderate level of bias, and many had serious methodological flaws. Most relied on process measures (such as patient throughput) to assess their effectiveness rather than the intervention’s impact on patient health. The few that did report outcomes generally did so over a very short timeframe. Only one-third of the studies that met our generous inclusion criteria used data from actual MCEs. This is problematic, because simulations and drills may not adequately replicate the physical conditions and emotional stress of responding to an actual disaster. In all but two areas, the evidence was insufficient to draw conclusions about the effectiveness of specific resource management or allocation strategies.

By their nature, MCEs are both uncommon and unpredictable. Conducting health services research in such a context presents extraordinary challenges. The rarity of MCEs necessitates use of study designs—such as computer simulations, drills, and quasi-experimental studies involving actual MCEs, all of which are commonly regarded as producing inferior evidence to prospective, randomized trials. Computer simulations and staged drills can accommodate the design features of experiments, such as prospective recruitment of participants and random allocation of strategies, but they raise significant concerns about fidelity and validity. Consensus is urgently needed to establish acceptable methodological and evidentiary standards for future research in disaster preparedness and response, particularly regarding studies that use drills, computer simulations, and prospective evaluation of strategies implemented during an actual MCE. For example, experts might consider process measures sufficient to evaluate a field triage system, but outcome measures should be collected to assess the impact of an accelerated discharge policy.

Given the practical and ethical challenges of conducting prospective studies during MCEs, federal agencies should proactively work with experts in the field to create “research preparedness” – the capacity to rapidly generate useful empirical observations and conduct urgently-needed studies during unfolding MCEs.³⁴ Useful steps might include a standardized, “Utstein style” approach to documenting essential information about the immediate medical response to disasters and large scale acts of terrorism,³⁵ a more standardized approach to the selection of comparison groups, a thoughtful approach to the ethics of disaster research, including the potential for pre-approved protocols sanctioned by national institutional review boards,³⁶ and contingent grant awards to provide the necessary funding required to immediately deploy a research team to a future MCE. Such innovations could quickly build a reasonably rigorous evidence base to inform policy and practice.

Even if a research infrastructure can be established, one vexing question remains: How much evidence is sufficient to support conclusions about the effectiveness of a particular resource management or allocation strategy? For example, one high-quality study included in the review—not summarized above—reported the results of a policy restricting ambulatory and inpatient care of non-urgent cases among 32 hospitals in the greater Toronto area during the 2003 SARS epidemic.³⁷ Can a single study, no matter how compelling, constitute sufficient evidence to support conclusions about the value of a particular policy? In the absence of consensus evidence hierarchies or other rubrics specific to MCEs, providers and policymakers may interpret evidence differently and therefore reach different conclusions. Developing evidentiary standards through consensus among relevant experts could promote swift adoption of effective strategies into disaster planning.

Given the near certainty that large scale MCEs will continue to occur, and the high cost and relative scarcity of healthcare services on a typical day, it is deeply concerning how little high-quality evidence is available to help policymakers, healthcare providers, and the public determine the best course of action during MCEs. Although we identified findings that support the utility of a few interventions, the overall body of evidence is far from definitive. Because MCEs do not lend themselves to gold-standard randomized controlled trials, non-experimental study designs must be employed. However, the absence of methodological and evidentiary standards, the challenges of conducting ethical research during an MCE, and competing funding priorities at the federal, state, and local levels, create formidable barriers to advancing the research in this area. Nevertheless, we believe our findings can serve as a key starting point for disaster researchers, first responders, and program evaluators across the nation.

In summary, the current evidence base appears inadequate to inform providers and policymakers about the most effective strategies for managing or allocating scarce resources during MCEs. Very limited evidence suggests that Points of Dispensing (PODs) can be used to efficiently distribute biological countermeasures following a bioterror attack or influenza pandemic, and different models may vary in their effectiveness. Some evidence also suggests that commonly used field triage systems do not perform consistently during actual MCEs. Few strategies were evaluated in more than two studies, and heterogeneity in outcome measures and context prevented quantitative synthesis of results. Improving the evidence base will require a diverse group of policy makers, funders, and non-governmental experts to come together to craft a well-organized and coordinated program of applied research³⁸ that adheres to appropriate ethical and methodological standards.