Performance evaluation and validation of the animal trauma triage (ATT) score and modified Glasgow coma scale (mGCS) with suggested category adjustment in dogs - a VetCOT Registry study

Abstract

Objective:

To examine the animal trauma triage (ATT) and modified Glasgow coma scare (mGCS) scores as predictors of mortality outcome (death or euthanasia) in injured dogs.

Design:

Observational cohort study conducted September 2013 to March 2015 with follow-up until death or hospital discharge

Setting:

9 veterinary hospitals including private referral and veterinary teaching hospitals

Animals:

Consecutive sample of 3,599 dogs with complete data entries recruited into the Veterinary Committee on Trauma (VetCOT) patient registry.

Interventions:

None

Measurements and Main Results:

We compared the predictive power (area under receiver operating characteristic AUROC) and calibration of the ATT and mGCS scores to their components. Overall mortality risk was 7.3% (n=264). Incidence of head trauma was 9.5% (n=341). The ATT score showed a linear relationship with mortality risk. Discriminatory performance of the ATT score was excellent with AUROC=0.92 (95% CI 0.91-0.94), pseudo R²=0.42. Each ATT score increase of 1 point was associated with an increase in mortality odds of 2.07 (95% CI=1.94-2.21 P<0.001). The ‘Eye/Muscle/Integument’ category of the ATT showed poor discrimination (AUROC=0.55). When this component together with the skeletal and cardiac components were omitted from calculation of the overall score, there was no loss in discriminatory capacity (AUROC=0.92 vs 0.91), P=0.09) compared with the full score. The mGCS showed good performance overall, but performance improved when restricted to head trauma patients (AUROC=0.84, 95% CI=0.79-0.90, n=341 vs 0.82 95% CI=0.79-0.85, n=3599). The motor component of the mGCS showed the best predictive performance (AUROC=0.79 vs 0.66/0.69), however the full score performed better than the motor component alone (P=0.002). When assessment was restricted to patients with head injury (n=341), the ATT score still performed better than the mGCS (AUROC=0.90 vs 0.84, P=0.04).

Conclusions:

In external validation on a large, multi-center dataset, the ATT score showed excellent discrimination and calibration, however a more parsimonious score calculated on only the perfusion, respiratory, and neurological categories showed equivalent performance.

Introduction

Illness severity scoring is a methodology developed to characterize disease severity for individuals or populations from a set of objective data. Illness severity is typically expressed as an integer score or as a mortality risk probability. While use of illness severity scores is widespread in the human medical field, fewer scores have been developed for use in veterinary medicine^1–5. Trauma scores are applied to patients presenting with acute trauma and are typically diagnosis-independent and based on observations made on patient presentation. Some of these observations may be trauma specific^1–3. Trauma scores have multiple applications including assistance with patient triage, performance benchmarking of hospitals and physicians, guidance in the use of hospital resources, and to characterize patient populations for clinical research.^5–7 Trauma scores are particularly relevant in veterinary medicine due to the high incidence of trauma related injury, with polytrauma accounting for up to 72.3% of trauma patients.⁸

To maximize efficacy, trauma scores must have predictive accuracy, which requires that the scores are validated and updated as new information becomes available.^9,10 Score validation is typically accomplished by benchmarking the original score against a contemporary population, and using regression analysis to adjust or reweight the score calculation. The animal trauma triage (ATT) score is a veterinary illness severity score that numerically classifies the degree of trauma in an attempt to quantify mortality risk probability.¹ The ATT score is based on a 0-3 scale (0 being slight or no injury, 3 indicating severe injury) with assessment of 6 independent components (perfusion, cardiac, respiratory, eye/muscle/skin, skeletal, and neurologic) that contribute equally to the overall predictive score.¹ (Appendix 1) The ATT has been widely utilized in veterinary medicine, both clinically and in clinical research settings.^11–16 However, despite its utility as a benchmark veterinary trauma score, the ATT has received relatively limited prospective validation¹⁶ and has not been updated in the last 20 years.

The Glasgow Coma scale (GCS) is an illness severity score originally described in the 1970s for people with traumatic brain injury.^17,18 Its use in human medicine relies on a patient’s eye, motor, and verbal responses.^19,20 This scale has been modified for veterinary use (Appendix 2) and the subsequent Modified Glasgow Coma Scale (mGCS) has been evaluated for its prognostic ability in head trauma cases.^2,21,22 The mGCS evaluates 3 components of neurologic function namely motor activity, brain stem reflexes, and level of consciousness. To date, the mGCS has not been evaluated for its survival predictive ability against a large data set, nor have the individual components been assessed.

Therefore, the purposes of this study were to evaluate the discriminatory performance of the ATT and the mGCS against a large trauma data set; to assess the individual components that make up the composite scores; and to determine if reweighting or eliminating any of the components affected score performance.

Materials and methods

The American College of Veterinary Emergency and Critical Care Veterinary Committee on Trauma (ACVECC-VetCOT) established the Trauma Registry as an international repository of trauma data for canine and feline patients and began collecting data in 2013. At the time of this analysis, the registry contained patient information from 9 veterinary hospitals located in North America representing Level I and II trauma centers in both private referral practices and veterinary teaching hospitals. All hospitals had both ICU and non-ICU areas where patients were hospitalized. The database includes information on signalment, type of trauma, outcome at time of discharge, and modified Glasgow coma score (mGCS) and Animal Trauma Triage score (ATT) recorded within 6 hours of admission (Appendices 1 and 2). The mGCS and ATT subscores for each scoring category were available, but not the base physiologic data resulting in the assignment of each subscore. The mGCS consists of 1-6 score summed across 3 categories for a total score range of 3 to 18, with a lower score reflecting greater abnormality. The ATT score consists of a 0-3 score summed across 6 categories for a total score range of 0-18, with a higher score reflecting greater abnormality. Cases were collected between September 2013 and March 2015, and included all cases presenting to the trauma centers as inpatients or outpatients that had history and clinical signs consistent with traumatic injury as assessed by the primary clinician.

Statistical methods

Descriptive data was assessed for normality using the Shapiro–Wilk test. Parametric data are summarized as means (± standard deviation, SD), while non-parametric data are summarized as median (inter-quartile range, IQR) and parametric and non-parametric hypothesis tests used as applicable. Continuous data were compared with Student’s t-test, or the Mann–Whitney U test. Data were characterized as hierarchical in structure, with nesting of patients within hospitals. Violation of the independence assumption was controlled for through use of mixed-effect logistic regression models with random intercepts at the hospital level. The log-likelihood was estimated using adaptive Gaussian quadrature, with 7 integration points. The number of integration points was assessed using the criteria of <0.01% change in coefficients with a doubling of integration points to indicate sufficiency. Postestimation model checking was performed using examination of Pearson and deviance residuals, together with dispersion parameters.²³

We examined the individual predictor subscores of the mGCS and ATT for availability in the database and assessed score linearity with respect to survival at discharge. Survival models were constructed using the subscores of the mGCS and ATT individually and as simple sums. Clustering on hospital was controlled for using random intercept logistic regression models. Survival models were evaluated for discrimination using the area-under the receiver operating characteristic curve (AUROC) and the overall percentage of variability explained by the model was evaluated by calculation of the pseudo R². Model calibration was assessed with the Hosmer–Lemeshow statistic, and Akaike’s information criteria. Statistical significance of differences between AUROC values were calculated using the non-parametric method of Hanley and McNeil.²⁴ All statistical calculations were performed using commercial software.^a

Results

Population characteristics

The VetCOT dataset used in this analysis included a total of 3,616 dogs with outcome information. The overall mortality for the dataset was 7.6% (n=274; 43 died, 231 euthanized). For 9 dogs, neither the ATT or mGCS were recorded, while the mGCS alone was unrecorded for 6 dogs, and the ATT score alone was unrecorded for 2 dogs. Where one or more scores were unrecorded, mortality was 58.8%, which was higher than in the overall population (P<0.001) (n=17; 1 died, 9 euthanized). The mGCS and ATT scores were available concurrently for 3,599 dogs, and the remainder of this analysis is restricted to these animals. For the 3,599 dogs analyzed, median age was 4 years (IQR 1.5-7.5). Male/female split was 54% (n=1910)/46% (n=1669); (n=20 unknown) of which 76% of female dogs were neutered (n=1268) and 68% of male dogs were neutered (n=1294). Median bodyweight was 12.8 kg (IQR 5.8-26.6). 24.1% of dogs (n=866) were hospitalized in the ICU. 9.5% (n=341) of dogs were suspected to have suffered head injury as a component of their trauma. Where recorded, median time from trauma to admission was 8.3 hours (IQR 2.0-19.6, n=3,453). Median time from admission to discharge or death was 3.1 hours (IQR 2.6-24, n=3,598). Mortality risk in the 3,599 patients with both scores was 7.3% (n=264) with 84% of deaths (n=222) occurring by euthanasia. Where euthanasias were performed, 22% (n=49) were recorded as being predominantly financially driven.

Score performance

ATT score

The relationship between ATT score and mortality risk is shown in Figure 1. Thirty-seven percent of dogs received an ATT score of 1, while ATT scores of 0 and 2 were also frequent, with approximately 18% of observations contained in each group. ATT scores of >10 were rare, with <1% of observations in any group. No animals had an ATT score of 17. The ATT score showed good overall linearity with respect to survival, with the exception of a score of 11 (n=7), which showed a lower mortality risk (57%) than a score of 8, 9 or 10. Overall, each ATT score increase of 1 point was associated with an increase in mortality odds of 2.07 (95% CI=1.94-2.21 P<0.001).

Figure 1.

Graph depicting the association between mortaity risk and animal trauma triage score

The discrimination performance of the full ATT score was excellent, with AUROC=0.92 (95% CI=0.91–0.94), and the model showed good calibration. The ability of the ATT to predict survival was also evaluated with models based on its individual component subscores (perfusion (p), cardiac (c), respiratory (r), eye/muscle/integument (e), skeletal (s), neurological (n)) to determine the most predictive components of the score. The results are shown in Table 1.

Table 1.

Predictive performance of the Animal Trauma Triage (ATT) score and components

Model	AUROC (95% CI)	Pseudo R2	AIC*
ATT score (full)	0.92 (0.91-0.94)	0.42	1100.082
ATT subscore (perfusion)	0.79 (0.76-0.82)	0.22	1484.601
ATT subscore (cardiac)	0.72 (0.68-0.75)	0.14	1628.61
ATT subscore (respiratory)	0.78 (0.75-0.81)	0.21	1504.09
ATT subscore (eye/muscle/integ)	0.55 (0.51-0.59)	0.01	1867.816
ATT subscore (skeletal)	0.71 (0.68-0.75)	0.13	1655.275
ATT subscore (neurological)	0.82 (0.79-0.85)	0.29	1347.17

^*Akaike Information Criteria

The least predictive component was the eye/muscle/integument subscore. The three most predictive components were the neurological, perfusion, and respiratory subscores. Starting with the least predictive, subscores were sequentially eliminated from the model, and the discrimination of the resulting more parsimonious model compared to the full ATT score until a significant loss of discrimination occurred. Results are shown in Table 2. Elimination of the eye/muscle/integument subscore from the overall ATT score resulted in no loss of performance and the absolute value of the AUROC increased. Further refinement of the model by sequential elimination of subscores found no difference in performance between the current 6 category ATT score and a more parsimonious model (ATTnpr) containing only the neuro, perfusion, and respiratory categories (AUROCs of 0.92 vs 0.91, P=0.09).

Table 2.

Predictive performance of the full Animal Trauma Triage (ATT) score and with sequential subtraction of subcategories. AUROC – area under receiver operating characteristic.

Model	AUROC (95% CI)	Comparison test of sub-model AUROCs to full model- P value
ATT score (full)	0.92 (0.91-0.94)
ATT score – (e/m/i)	0.93 (0.91-0.94)	0.50
ATT score- (e/m/i+s)	0.91 (0.89-0.93)	0.10
ATT score- (e/m/i +s+c)	0.91 (0.89-0.93)	0.09
ATT score- (e/m/i +s+c+r)	0.89 (0.86-0.91)	<0.01

e/m/i, eye/muscle/integument; s, skeletal; c, cardiac; r, respiratory

Modified GCS score

The relationship between mGCS and mortality risk with the numbers of dogs on each category of mGCS score is shown in Figure 2. The mGCS was originally constructed for use in evaluating dogs suffering from neurological impairment following head trauma, however in this population the mGCS was calculated for all dogs, and was recorded as abnormal (<18) in 380 dogs (10.6%) not suspected to have suffered specific head injury.

Figure 2.

Graph depicting the association between mortality risk and modified Glasgow Coma Scale.

Eighty-five percent of mGCS scores were recorded as 18 (normal). Categories of mGCS <13 contained between 2 and 14 animals per group. The mGCS showed fair overall linearity with respect to survival, with the exception of a score of 10 (n=7), which showed a lower mortality risk (14.3%) than all other scores except 18. Each mGCS score decrease of 1 point was associated with an increase in mortality odds of 2.07 (95% CI=1.90-2.27 p<0.001).

The discrimination performance of the full mGCS on the general trauma population was good, with AUROC=0.82 (95% CI=0.79-0.85). The ability of the mGCS to predict survival was also evaluated with models based on its individual component subscores (motor (m), brain stem reflexes (b), level of consciousness (c)) to determine the most predictive components of the score. The results are shown in Table 3.

Table 3.

Predictive performance of the modified Glasgow Coma (mGCS) score and components

Model	AUROC (95% CI)	Pseudo R2	AIC*
mGCS (full)	0.82 (0.79-0.85)	0.25	1427.094
mGCS subscore (motor)	0.79 (0.76-0.82)	0.24	1443.969
mGCS subscore (brain stem reflexes)	0.66 (0.63-0.69)	0.13	1645.076
mGCS subscore (level of consciousness)	0.69 (0.66-0.72)	0.16	1591.967

^*Akaike Information Criteria

The most predictive component was the motor subscore. Starting with the least predictive, subscores were sequentially eliminated from the model, and the discrimination of the resulting more parsimonious model compared to the full mGCS until a significant loss of discrimination occurred. Results are shown in Table 4. Elimination of the brain stem reflexes subcategory from the overall mGCS resulted in no detectable loss of performance however the absolute value of the AUROC decreased slightly from 0.82 to 0.81.

Table 4.

Predictive performance of the full modified Glasgow Coma Scale (mGCS) and with sequential subtraction of subcategories. AUROC - area under receiver operating characteristic

Model	AUROC (95% CI)	Comparison test of sub-model AUROCs to full model- P value
mGCS (full)	0.82(0.79-0.85)
mGCS– (b)	0.81 (0.78-0.84)	0.11
mGCS- (b+c)	0.79 (0.76-0.82)	<0.01

b, brainstem; c, consciousness

Head trauma population

When the population was restricted to dogs presenting with head injury (n=341), the AUROC of the mGCS increased compared with the larger population (AUROC=0.84, 95% CI=0.79-0.90) however this difference was not significant (P=0.12).

Modified GCS and ATT score performance comparisons

When evaluated on the overall population (n=3,599), ATT score discrimination was better than the mGCS (AUROC 0.92 (95% CI 0.91-0.94) vs 0.82 (95% CI 0.79-0.85), p<0.001). When evaluated on a population restricted to dogs with head injury (n=341), ATT score performance was still better than the mGCS (AUROC=0.90 (95% CI 0.86-0.93) vs 0.84 (0.79-0.90) P=0.04). The ROC curves for the two populations are shown in Figures 3 and 4.

Figure 3.

Area Under the Receiver Operator Curve characteristics for animal trauma triage (ATT) and modified Glasgow Coma Scale (mGCS).

Figure 4.

Area Under the Receiver Operator Curve characteristics for animal trauma triage (ATT) and modified Glasgow Coma Scale (mGCS), head trauma population.

Discussion

The current study validates the predictive performance of the ATT and mGCS scores against a large multicenter trauma dataset. The predictive performance, discrimination, and calibration of the ATT was excellent. The full score AUROC of 0.92 (95% CI 0.91-0.94) compares favorably with the most predictive scores in either the human or veterinary literature, and is similar to that previously reported in a smaller study (0.91). ^16,25–27

In the original description of the ATT score, the authors reported a 2.3-2.6 times decrease in survival odds for each 1 point increase in ATT. The current study showed a 2.07 times decrease in survival odds for each 1 point increase in ATT. The difference in these odds ratios may reflect the performance improvement in the clinical treatment of trauma in the last 20 years. Each subcategory of the ATT was not equally predictive of survival. Omitting the eye/muscle/integument subcategory of the ATT score increased the predictive value of the overall score (AUROC=0.93) and would likely make the ATT faster to calculate in a clinical setting. Pursuing the theme of efficiency further, a reduced ATT score calculated on respiratory, neurological and perfusion subcategories alone was found to be equivalent in predictive power to the full score, and would presumably be less labor intensive to calculate.

The mGCS discriminatory performance was also good with an AUROC of 0.82 when applied to the entire trauma population. This suggested that the mGCS was able to offer reasonable discriminatory performance even in the absence of a specific history or injury pattern consistent with head trauma. This may be because many polytrauma patients have occult head injury that is challenging to recognize in a veterinary setting, or because the mGCS is acting as a proxy variable for a systemic shock state that compromises motor function and level of consciousness. When the population was restricted to patients with known history or physical exam consistent with head trauma, the performance of the mGCS increased (AUROC=0.84), however this falls well below the discriminatory capacity (0.908-0.946) of the same test for traumatic brain injuries in people.²⁰ The cause of this difference remains unclear, however increased accuracy in scoring mentation and brainstem reflex categories may be appreciated in human patients due to the ability of the patient to verbally communicate changes in mental status. The brain stem reflex category was the least predictive component of the mGCS in this population, a finding that duplicates a study assessing the various categories that make up the human GCS.²⁸ An unexpected finding of the current study was that the ATT outperformed the mGCS in a population restricted to known head trauma cases (AUROC of 0.9 vs 0.84 respectively). As the ATT provides a more global assessment of the patient’s trauma pathology, it may more accurately predict outcomes of cases where head trauma is a single component of a wider polytrauma. This may be particularly true in a veterinary setting where the majority of deaths occur by euthanasia, and the decision to euthanize may be influenced by the financial implications of multiple injuries.

This study has several limitations. Firstly the case data from which the score subcategories were calculated could not be reviewed, and this precluded any attempt to redefine the calculation of subcategory scores to improve score performance. Secondly use of the VetCOT registry carried all the advantages but potentially also the disadvantages of a large multicenter dataset, with limited opportunity to quality control data entry. While the dataset was closely scrutinized and all obvious errors removed, data entry inaccuracy if significant could limit the validity of our results. Finally, there was the risk that euthanasia bias caused inflation of our assessments of score performance. If the clinical team were in any way using the calculated scores to influence their recommendations to owners regarding euthanasia decisions, this would artificially improve score discrimination.

In conclusion, the ATT provides excellent predictive performance for objectively describing severity of illness in a trauma population, however we recommend omitting the eye/ muscle/ integument category from score calculation, resulting in a 0-15 instead of a 0-18 score. If major constraints with resources available for data collection are present, then a score limited to the neurological, perfusion and respiratory categories (ATT_npr, 0-9 score) will still provide a good objective assessment tool. In the context of head trauma, the ATT score outperforms the mGCS in predictive strength.

Abbreviation list:

ATT

animal trauma triage

AUROC

area under receiver operating characteristic

mGCS

modified Glasgow coma scale

VetCOT

Veterinary Committee on Trauma

Appendix 1. The Animal Trauma Triage Scoring System

Grade	Perfusion	Cardiac	Respiratory	Eye/Muscle/Integument	Skeletal	Neurological
0	mm pink & moist CRT ~ 2 sec Rectal temp 37.8 °C (100 °F) Femoral pulses strong or bounding	HR: Dog: 60-140 Cat: 120 -200 Normal sinus rhythm	Regular resp rate with no stridor No abdominal component to resp	Abrasion, laceration: none or partial thickness Eye: no fluorescein uptake	Weight bearing in 3 or 4 limbs, no palpable fracture or joint laxity	Central: conscious, alert →sl dull; interest in surroundings Periph: normal spinal reflexes; purposeful movement and nociception in all limbs
1	Mm hyperemic or pale pink; mm tacky CRT 0 -2 sec Rectal temp 37.8 °C (100 °F) Femoral pulses fair	HR: Dog: 141-180 Cat: 201 -260 Normal sinus rhythm or VPCs < 20/min	Mildly incr resp rate & effort, ± some abdominal component Mildly incr upper airway sounds	Abrasion, laceration: full thickness, no deep tissue involvement Eye: corneal laceration/ulcer, not perforated	Closed appendicular/rib fx or any mandibular fx Single joint laxity/luxation incl. sacroiliac joint Pelvic fx with unilateral intact SI-ilium-acetab Single limb open/closed fx at or below carpus/tarsus	Central: conscious but dull, depressed, withdrawn Periph: abnormal spinal reflexes with purposeful movement and nociception intact in all 4 limbs
2	Mm v pale pink & v tacky CRT 2-3 sec Rectal temp < 37.8 °C (100 °F) Detectable but poor femoral pulses	HR: Dog: > 180 Cat: > 260 Consistent arrhythmia	Mod incr resp effort with abdmon component, elbow abduction Moderately incr upper airway sounds	Abrasion, laceration: full thickness, deep tissue involvement, and arteries, nerves, muscles intact Eye: corneal perforation, punctured globe or proptosis	Multiple Grade 1 conditions (see above) Single long bone open fx above carpus/tarsus with cortical bone preserved Non-mandibular skull fx	Central unconscious but responds to noxious stimuli Periph: Absent purposeful movement with intact nociception in 2 or more limbs or nociception absent only in 1 limb Decr anal and/or tail tone
3	Mm gray, blue, or white CRT > 3 sec Rectal temp < 37.8 °C (100 °F) Femoral pulse not detected	HR: Dog: < 60 Cat < 120 Erratic arrhythmia	Marked resp effort or gasping/agonal resp or irregularly timed effort Little or no detectable air passage	Penetration to thoracic/abd cavity Abrasion, laceration: full thickness, deep tissue involvement, and artery, nerve, or muscle compromised	Vertebral body fracture/luxation except coccygeal Multiple long bone open fx above carpus/tarsus Single long bone open fx above tarsus/carpus with loss of cortical bone	Central: nonresponsive to all stimuli; refractory seizures Periph: Absent nociception in 2 or more limbs; absent tail or perianal nociception

Appendix 2. Modified Glasgow Coma Scale

Motor activity	Score
Normal gait, normal spinal reflex	6
Hemiparesis, tetraparesis, or decerebrate activity	5
Recumbent, intermittent extensor rigidity	4
Recumbent, constant extensor rigidity	3
Recumbent, constant extensor rigidity with opisthotonus	2
Recumbent, hypotonia of muscles, depressed or absent spinal reflexes	1
Brain Stem
Normal pupillary light reflexes and oculocephalic reflexes	6
Slow pupillary light reflexes and normal to reduced oculocephalic reflexes	5
Bilateral unresponsive miosis with normal to reduced oculocephalic reflexes	4
Pinpoint pupils with reduced to absent oculocephalic reflexes	3
Unilateral, unresponsive mydriasis with reduced to absent oculocephalic reflexes	2
Bilateral, unresponsive mydriasis with reduced to absent oculocephalic reflexes	1
Level of consciousness
Occasional periods of alertness and responsive to environment	6
Depression or delirium, capable of responding but response may be inappropriate	5
Semicomatose, responsive to visual stimuli	4
Semicomatose, responsive to auditory stimuli	3
Semicomatose, responsive only to repeated noxious stimuli	2
Comatose, unresponsive to repeated noxious stimuli	1