A Validation Study of Five Triage Systems Using Data from The 2005 Graniteville, SC Chlorine Spill

Introduction

Effective response to mass casualty incidents, which encompass a broad spectrum of threats and hazards, represents one of the greatest challenges to our nation’s emergency response infrastructure.¹ Natural, unintended (technological) or deliberate catastrophic events (all hazards events) will necessitate the effective and timely management of mass casualties.^2–3Chemical incidents involving irritant chemicals such as chlorine, pose a significant threat to life and require rapid assessment. In the aftermath of a disaster such as a large chemical spill, first responders have only seconds to evaluate the condition of a victim before moving to the next. Casualties generated by such disasters can overwhelm healthcare capabilities, jeopardizing the lives of victims and healthcare providers alike.¹ To mitigate the ‘surge’ of casualties into a healthcare facility after a mass casualty event, hospitals and emergency responders use triage to assess patients and prioritize care with the goal of saving as many lives as possible. However, the proposed national guideline for existing mass casualty triage does not take fully into account all hazards events⁴ that include chemical incidents requiring decontamination and toxidrome (a group of symptoms or a specific syndrome associated with exposure to a specific poison or agent^5,6) assessment.

This study evaluated five frequently used triage systems:1) Simple Triage and Rapid Treatment (START),⁷ 2) JumpSTART,⁸ 3) Sort, Assess, Lifesaving Interventions, Treatment/Transport (SALT)⁹, and 4)Chemical/Biological/Radiologic/Nuclear Mass Casualty Triage System (CBRN),⁶ and, 5) Emergency Severity Index (ESI).¹⁰ Table 1 describes each of these systems.

Table 1.

The Five Triage Systems Used for Triage Simulation Studies, Abstracted Hospital Record Data; Graniteville, SC (2005)

TRIGE SYSTEM	DESCRIPTION
START7 Field Triage System	Uses ability to walk, respirations, circulation, and mental status for individuals ≥ 8 years of age
JumpSTART Pediatric Triage Algorithm8 Field Triage System	Uses same assessments as START but designed for individuals < 8 years of age and includes a more comprehensive neurological assessment.
SALT9 Field Triage System	Uses ability to walk, respirations, pulse, mental status, available resources and injuries to assess individuals ≥ 8 years of age
CBRN6 Field Triage System	Uses the ability to walk, respirations and is the only system that additionally assess contamination and toxidromea symptoms related to chemical, biological radiological, or nuclear incidents. to assess individuals > 8 years of age
ESI10 ED-based System	Uses patient acuity, pulse, respiration, SpO2b and number of resources needed for all ages.

^aToxidrome refers to a group of symptoms or a specific syndrome associated with exposure to a specific poison or agent^54–55

^bSpO₂: Arterial oxygen saturation measured by pulse oximetey

Mass casualty research is not suitable for randomized, controlled, experimental exposure studies. Therefore, current mass casualty research designs and evaluation strategies usually are anecdotal and much of the data reported has little external validity.^11–14 The ability to collect accurate, timely and valid data at the time of an incident is difficult. Because data is often missing or biased it is impossible to test the utility of a specific model. There is no gold standard for measuring the efficacy of triage.^1,15 Strategies are needed to study the accuracy and efficacy^16,17 of initial triage from actual mass casualty datasets.

Lack of outcomes-based research results in uncertainty about the efficacy of any of the triage systems on patient outcomes.¹⁸ This research builds on the work of Lerner et al.,⁴ Kirk and Deaton,¹⁹ Jenkins et al.,¹⁷ Wenck et al.,²⁰ and Van Sickle et al.¹⁶ in identifying the most relevant and appropriate data needed for initial triage of mass causalities using an all hazards approach. For this study initial triage assessments made by first responders, Emergency Department (ED) personnel, and primary care providers were used so that a priority code for treatment and/or transport could be determined.

Background

On January 6, 2005, a freight train carrying tanker cars of liquid chlorine was inadvertently switched onto an industrial spur in the center of Graniteville where it crashed into a parked locomotive. The train derailed and heavily damaged chlorine tankers ruptured, immediately releasing approximately 60 tons of chlorine.^5,21 Within minutes, the dense chlorine gas infiltrated the surrounding town with a population of over 7,000. Nine people died, 71 were hospitalized, 840 were treated as outpatients, and 220 experienced immediate health problems including skin, eye, nose and throat injuries.^16,21 The South Carolina Department of Health and Environmental Control (SC DHEC) and epidemiologists from the Centers for Disease Control and Prevention (CDC) collected data at the scene during the first weeks of the incidents.

Purpose

The purpose of this research was to identify and validate triage systems appropriate for use in chemical events using the Graniteville data. Research questions: do five currently used triage systems correlate with the level of care needed after a chlorine chemical mass- casualty event? What, if any, additional information would improve accurate triage classifications during chemical incidents?

Methods

Design

This paper presents the results of the secondary data analysis used to study the efficacy of five triage systems, using data from a real chemical mass casualty event. The Division of Acute Disease Epidemiology and the regional and county offices of the SC DHEC and the CDC collaborated to design and conduct public health interventions^22,23 and surveillance²⁰ to better characterize clinically the chlorine exposed victims. Abstracted ED and medical records data collect by these two agencies were linked with the other public health datasets of the chlorine disaster victims within the South Carolina Office of Research and Statistics (SC ORS). These data provided the research dataset used in this study.

Since none of the databases included data regarding actual triage categories assigned during the chlorine incident, a statistical logic model was created for each triage system to generate ‘extrapolated’ injury severity triage outcome classifications (triage category) for each triage system. Logic for each of the five triage decision trees or algorithms was coded in Statistical Analysis System (SAS 9.2)²⁴ software to create a statistical logic model for each triage system.

Wenck et al.,²⁰ used nine injury severity categories to classify patients who received medical care following the 2005 chlorine exposure: 1) deceased, 2) hospitalized >3 nights with ventilator and/or ICU support, 3) hospitalized >3 nights, 4) hospitalized 1–2 nights, 5) multiple ED visits during acute period, 6) ED visit with substantial respiratory symptoms, 7) ED with mild symptoms, 8) ED presentation due to exposure only and 9) non-emergent medical office visit. These actual injury severity categories were used to retrospectively identify patients who actually needed treatment immediately and those who did not. For analysis, categories 1–4 were classified as experiencing a severe medical outcome and assigned an “immediate “extrapolated injury severity status for triage; categories 5–7 were classified as having a less severe medical outcome and assigned a “delayed” extrapolated injury severity status for triage; and categories 8 and 9 were assigned a “minimal” extrapolated injury severity status for triage. For the purpose of assessing accuracy of the five triage systems, the nine actual injury severity category measures, as determined by Wenck et al.,²⁰ were considered as the standard to which extrapolated injury severity triage outcome categories from the five triage systems were compared.

Sample

Data were abstracted by public health officials from 155 ED records of patients who were either admitted to the hospital or seen more than once in the ED of the local medical center.¹⁶ The public health authority made the decision as to which patient records were abstracted based on the assumption that patients who were treated multiple times in the ED would have been hospitalized in a non-surge scenario. In addition to these 155 patients, another 257 patients whose medical records had not been abstracted yet were included in the sample because other linked public health datasets, that included data from these same patients, had sufficient data detailing their initial clinical presentation.

Collected data on the victims were entered into six datasets.^16,20,23,25 After these linked data were de-identified and data use agreements were approved by all parties, the data were provided to the research team by the SC ORS for use in this study. The study was approved by the Institutional Review Boards of the University of South Carolina and the SC DHEC prior to initiation. Every member of the research team reviewed the de-identified datasets for accuracy; duplicate records were eliminated. The datasets were then merged to generate a research dataset to create code for five triage systems. These data were used to determine whether the triage categories from the five triage systems correlate with patient outcomes and to explore what additional information would improve predictions of accurate triage classifications during chemical incidents.

Measures

Actual patient outcome data were used to determine whether extrapolated triage scores from the five triage systems correlated and/or agreed with the actual patient injury severity developed by Wenck et al.²⁰ Triage categories were extrapolated based on the statistical logic model created for each triage system.

Extrapolated Injury Severity Triage Outcome Category

The biophysical response to acute chlorine exposure is uniform and very characteristic despite route of exposure and individual susceptibility. This permits the analysis of linkages between injury severity and triage systems for an acute chlorine disaster.^26,27, Select variables that characterize severity of injury outcome following a chlorine exposure were used in the development of a statistical logic model to study the effectiveness of the five triage systems in extrapolating injury severity category outcomes. Variables (triage data elements) included physiological measurements, toxidrome sign/symptoms, resources, SpO₂, and injury severity categories. Triage systems that are often studied typically assess physiologic parameters and resource constraints to determine their accuracy in identifying patients with critical injury. However, few studies have demonstrated strong agreement between triage systems using actual patient outcomes.¹⁸

Each triage system data element was operationalized to select or extrapolate the correct value from the clinical data available. For example, the clinical records did not have a data entry field for “doesn’t obey commands” which is a key component of the START/JumpSTART triage assessment. Therefore, it was determined that the patient did obey commands if all other data elements which required obedience (e.g., able to give a medical history) were completed. Many data measures required for completion of the triage systems were readily available, like respiratory rate or blood pressure at presentation. Such data elements were culled from the clinical data and used in the triage system analysis. Because this study focused on triage efficacy, the initial clinical presentation data were exclusively used to assess triage effecacy and the later clinical data collected during the remainder of the hospitalization or treatment within the ED were simply used contextually or to determine the actual severity of injury.⁵ This methodology was described by Culley et al.⁵

Pulse oximetry was previously found useful in assessing the health of surge patients.¹⁶ However, due to the surge of patients to the ED at the time of the chlorine incident, that medical resource was not completed for all patients.⁵

Data Analysis

Descriptive statistics were used to describe selected variables such as demographics and patient outcomes (actual and extrapolated). The research database was analyzed using measures of central tendency for descriptive variables and patient outcomes. Spearman correlation was used to compare actual injury severity (from the Wenck²⁰ study) with the extrapolated injury severity triage outcome classifications and SpO₂. Kappa statistics were calculated to examine the level of agreement between extrapolated injury severity triage outcome classifications and actual injury severity.

Results

The final merged data set consisted of 631 abstracted records. Actual outcome sample size, and therefore the number of rows with populated columns for each triage model, varied from 110 to 412 depending on the data available from the records. The SC DHEC datasets include information on patients hospitalized in nine hospitals (n=71), examined during the first few days after the accident (n=631), or those who died (n=9). The data included patients who ranged in age from 4 months to 85 years, were 58.9% male, 59.9% white, 35.2% black, 2.5% Hispanic and other, 2.4%.

Spearman correlations between the extrapolated injury severity triage outcome classifications and the actual injury severity outcomes range 0.38 to 0.64, (p<0.0001). The Spearman correlations between actual injury severity and triage systems were: CBRN (r=.64), SALT (r=.44), START (r=.55), ESI (r=.59), and ED SpO₂ (r=.38). The Spearman correlations between ED SpO₂ and CBRN were: (r=.52), SALT (r=.60), START (r=.60), and ESI (r=.71). There was slight to fair agreement between the extrapolated injury severity triage outcome classifications and the actual injury severity outcomes. Simple Kappa statistics for ESI, CBRN, START/JumpSTART, SALT were: 0.36 (95% CI .28, .43), 0.15 (95% CI .09, .20), 0.14 (95% CI .11, .18), and 0.13 (95% CI .09, .16), respectively, and the weighted Kappa statistics were: 0.42 (95% CI .34, .49), 0.30 (95% CI .24, .36), 0.25 (95% CI .19, .31), and 0.23 (95% CI .17, .29), respectively.

Table 2 summarizes the results of the extrapolated injury severity outcome categories compared to the actual injury severity outcomes. The data showed an overestimate of the victims classified as Levels l and 2 using the ESI triage system and an underestimate of red (immediate) victims by the START/JumpSTART system. Victims classified as T1 using the CBRN system showed similar triage misclassifications. Victims classified as yellow (delayed) were either over-or under-triaged or not classified by the START/JumpSTART and SALT systems. All of the triage systems except ESI dramatically overestimated the green-minor, Level 5 or T3 category by 68–75%. All of the victims in this study were seen in the ED of the local hospital. ESI, designed for ED triage, determines “how many different resources are needed” as a variable to differentiate among triage categories. All victims in this study used at least one resource (e.g., intravenous line, X-ray, oxygen, etc.) when seen in the ED. Level 5 in the ESI system specifies no resources needed; therefore, none of the victims were classified as Level 5 or green by this system. This accounts for the large number of victims classified as Level 3 and 4 indicating that at least one resource was needed.

Table 2.

Frequency and % of Victims’ Actual Injury Severity Outcomes as Categorized Using Extrapolated Injury Severity Triage Outcomes, Graniteville, SC (2005)

ACTUAL INJURY SEVERITY OUTCOMES	EXTRAPOLATED INJURY SEVERITY TRIAGE OUTCOME CATEGORIES
n (%) 631 (100)	START/JumpSTART All ages n (%)	ESI All Ages n (%)	CBRN Adults n (%)	SALT Adults n (%)
Deceasedb 8 (1.27)	Black – Deceased 8 (1.27)	Level 0 – Dead 8 (1.27)	T4 8 (1.27)	White – Dead Black – Expectant 8 (1.27)
Hospitalized ≥ 3 nights with ICU/Ventilatorc Hospitalized ≥ 3 nights Hospitalized 1–2 nights 72 (11.41)	Red – Immediate 38 (6.02)	Levels 1 & 2 131 (20.76)	T1a 87 (13.79)	Red – Immediate 78 (12.36)
ED multiple visitsd ED with substantial respiratory symptoms ED with mild symptoms 257 (40.73)	Yellow – Delayed 0 (0)	Level 3 & 4 459 (72.74)	T2 30 (4.75)	Yellow – Delayed 1 (0.16)
ED without symptomse Physician office visit 75 (11.89)	Green – Minor 546 (86.53)	Level 5 0 (0)	T3 506 (80.19)	Green - Minimal Normal 544 (86.21)
Missing data 219 (34.7)	Not Classified 39 (6.18)	Not Classified 33 (5.23)	T3 506 (80.19)	Not Classified 0 (0)

^aAll values are significant (P<0.001) except for the CBRN T1 value

Outcomes Categories were collapsed into four categories for analysis:

^b.Deceased: 8 declared dead at the scene and not triaged; 1 taken to the ED and died at the hospital

^c.ICU/hospitalization of any length of stay

^d.ED with substantial or mild symptoms

^e.ED without symptoms for exposure and non-emergency medical office visit

Arterial Oxygen Saturation (SpO₂)

Results of this study indicated that 23.6% (n=27) of the victims assessed had a SpO₂ <90%, 7.3% (n=8) had SpO₂ 90–91%, and 68.2% (n=75) had SpO₂ > 92% (Table 3). Analyses of the SpO₂ values for ESI compared to the actual injury severity outcomes indicate similar values for all triage categories.

Table 3.

SpO₂ Correlated with Actual Injury Severity Categories and the ESI Triage System Extrapolated Injury Severity Triage Outcome Categories; Graniteville SC (2005)

Actual Injury Severity Categories	n = 110 Actual SpO2 n (%)			ESI Categories	n = 112 ESI Extrapolated Injury Severity Triage Outcome Categories with SpO2 n (%)
	45–89%	90– 91%	≥92%		45–89%	90– 91%	≥92%
Deceaseda	1 (0.91)	0 (0)	0 (0)	Level 0 – Dead	a	a	a
Hospitalized > 3 nights with ICU/Ventilatorb Hospitalized > 3 nights Hospitalized 1–2 nights	24 (21.82)	7 (6.36)	36 (32.73)	Levels 1 & 2 – Red	27 (24.11)	8 (7.14)	40 (35.41)
ED multiple visits ED with substantial respiratory symptomsc ED with mild symptomsc	2 (1.82)	1 (0.91)	39 (35.45)	Level 3 & 4 – Yellow	0 (0)	0 (0)	37 (33.04)
ED without symptomsd Physician office visitd	0 (0)	0 (0)	0 (0)	Level 5 – Green	0 (0)	0 (0)	0 (0)

Outcomes Categories were collapsed into four categories for analysis:

^aDeceased: 8 declared dead at the scene and not triaged; 1 taken to the ED and died at the hospital

^bICU/hospitalization of any length of stay

^cED with substantial/mild symptoms

^dED for exposure and non-emergency medical office visit

Discussion

The Graniteville chlorine disaster presented an ideal opportunity to test the accuracy of common triage systems. Anecdotal information and personal communication strongly suggests that triage was used in the 9 hospitals which received patients from the Graniteville chlorine disaster.^28,29 However, no data on the actual triage approaches were available within any of the databases used. Ambulance run data were not available in the datasets used for this analysis primarily because most patients self-transported.²⁰ Therefore, this study does not use any data that specified a triage category nor make any inferences on the efficacy of the actual triage provided to the Graniteville chlorine spill disaster victims.

Hospitals rapidly ascertained the limitations of the available triage system and used other approaches, such as SpO₂ and their clinical judgment, to triage patients.^19,28,29 Data from the poison control center clinical toxicologist’s consultations suggest that a methodology was rapidly developed and used in the hospitals based on a toxic syndrome.^19.29 Given the lack of data on the actual triage approaches used in the Graniteville disaster, this study extrapolated how Graniteville disaster patients might have been triaged had the clinicians exclusively used each of the five triage systems. Inferences are based only on the extrapolated injury severity triage outcome categories and their correlation and agreement with the actual injury severity score within the Graniteville disaster population.

Many chemicals such as chlorine and phosgene have a latency period from exposure to onset of symptoms, that may result in the rapid deterioration of a patient’s condition after initial triage.^6,30Victims of chlorine exposure may experience difficulty breathing or shortness of breath immediately if high concentrations of chlorine gas are inhaled, or signs/symptoms may be delayed if concentrations are low.¹⁶ Children may be more vulnerable to the effects of chlorine.³¹ A review of the literature indicates that none of these triage systems has been evaluated for sensitivity and specificity under conditions that include chemicals and their delayed effects and a variety of criteria and parameters, including physiological "cut-off" values;^32–34 this was supported by data from this study.

The triage systems studied showed only modest effecacy in establishing priorities for the treatment of victims exposed to chlorine. ESI, the only triage system studied that uses SpO₂ as a physiological measure, performed better than the other triage systems (r=.71). The CBRN system specifically addresses toxidrome symptoms related to chemicals such as chlorine. Incorrectly performed triage can underestimate the need of critically injured patients for immediate care, resulting in preventable deaths or deformities (undertriage) or overestimating the extent of minor injuries, resulting in mortality or disability of patients with more severe injuries (overtriage).¹ Analysis of data from the CBRN system showed that 80% of the victims studied were classified with an extrapolated injury severity triage outcome category as T3 (minor), or under triaged, compared to only 12% classified as minor from the actual injury severity category. The extrapolated injury severity triage outcome category for T1 (immediate) was 14% (over triaged) and T2 5% (under triaged) compared to actual injury severity categories T1 of 11% and T2 41%.

Results showed good sensitivity for ESI and outcome (.86) and very poor sensitivity for CBRN (.03), START/JumpSTART (0), and SALT (.03) with outcome. The specificity of each triage system with outcomes was 0.44, 0.64, 0.75, and 0.88 for ESI, CBRN, START/JumpSTART, and SALT, respectively. The positive predictive value was 0.27 for ESI with outcome and close to 0 for the other triage systems with outcome. Such accuracy could lead to critical misdiagnoses and faulty treatment decisions had the currently available triage systems been used exclusively.

Limitations

Two specific issues influenced the outcomes of this study. First, during chemical incidents the time from exposure to triage assessment is a critical determinant of an accurate triage classification. During this disaster, victims were admitted to the ED over more than an 8 hour time frame. Chemicals such as chlorine and phosgene have a latency period. Since the data abstracted included only the initial SPO₂ measurement upon arrival in the ED, this may have influenced the extrapolated severity outcome. Four of the triage systems studied are intended for field triage (START/JumpSTART, CBRN and SALT) and the ESI system is used in hospital EDs. Although all five of these systems have similar characteristics they are implemented differently in the field than in a hospital setting. The influences of a latency period in sign and symptom evolution in both of these settings may be different and warrants further investigation. Second, due to the surge of large numbers of patients, many patients did not receive an SPO₂ assessment. Accuracy and completeness of charting also influence the statistical analysis.

Implications for Emergency Nurses

Mitchell et al.,³⁵ stress the need for triage competency training of ED nurses for chemical, incidents. SpO₂ measured by pulse oximetry, one of the best predictors for triage classification for inhalation chemical hazards, should be incorporated into the competency training of ED nurses. The study illustrates the importance of SpO₂ in the initial assessment of victims of chlorine exposure when other assessments do not prove to be reliable indicators of the severity of the health effects from a chemical exposure. SpO₂ offers a relatively inexpensive, simple and reliable means to monitor respiratory function in a wide variety of settings. An electronic clip-on sensing device is typically attached to the tip of the finger. SpO₂ readings were recorded for some of the victims seen in the ED as a result of the 2005 chlorine leak. Published research on these data indicate that SpO₂ was predictive of acute medical outcomes.^16,20 Van Sickle et al.,¹⁶ showed that the most frequently reported symptoms among the patients (i.e., coughing and burning eyes) were not sufficient to distinguish mild from severe injury and that SpO₂ provided early indications of injury outcome severity as assessed by the duration of hospitalization and need for intensive care support. The Van Sickle, et al.¹⁶ study further showed that patients who were hypoxic on room air (SpO₂< 90%) were hospitalized three times as long as those who were not hypoxic (p<.001) and that SpO₂ provided early indications of outcome severity. SpO₂ is a very easy and quick approach to understanding the physiological functioning and latency effects of victims of chemical disasters and should be the first priority assessment used in EDs.

Conclusions

This study is unique in that the dataset include children in sufficient numbers for analysis. The most important finding is that SpO₂ was very predictive of actual injury severity in a respiratory irritant chemical event and should be a part of a mass casualty protocol for any chemical spill. The extrapolated injury severity triage outcome categories from the five triage systems did not agree with the actual injury severity categories.

SpO₂ provides early indications of outcome severity in incidents involving irritant chemical exposures such as chlorine. Additional research is needed to identify and validate the most sensitive clinical measures for triaging victims of toxic inhalation disasters. Research is needed to: 1) explore the effect of SpO₂ as an assessment variable for triage involving irritant chemical exposures such as chlorine; and (2) explore additional information in an all hazards mass casualty triage system that could sufficiently improve accurate triage classifications for initial treatment involving a chemical incident.³⁵