Performance evaluation and validation of the animal trauma triage (ATT) score and modified glasgow coma scale (mGCS) in injured cats: A Veterinary Committee on Trauma (VetCOT) registry study

Abstract

Objectives:

To examine the Animal Traume Triage (ATT) and modified Glasgow Coma Scale (mGCS) scores as predictors of mortality in injured cats.

Design:

Observational cohort study conducted September 2013 to March 2015

Setting:

9 Level I and II veterinary trauma centers

Animals:

Consecutive sample of 711 cats reported to the Veterinary Committee on Trauma (VetCOT) trauma registry

Interventions:

None

Measurements and main results:

We compared the predictive power (area under receiver operating characteristic curve; AUROC) and calibration of the ATT and mGCS scores to their components. Overall mortality risk was 16.5% (95% confidence interval [CI]=13.9–19.4). Head trauma prevalence was 11.8% (n=84). The ATT score showed a linear relationship with mortality risk. Discriminatory performance of the ATT score was excellent (AUROC=0.87 [95% CI 0.84–0.90)]). Each ATT score increase of 1 point was associated with an increase in mortality odds of 1.78 (95% CI=1.61–1.97, P<0.001). The eye/muscle/integument category of the ATT showed the lowest discrimination (AUROC=0.60). When this component, skeletal, and cardiac components were omitted from score calculation, there was no loss in discriminatory capacity compared with the full score (AUROC=0.86 vs 0.87, respectively, P=0.66). The mGCS showed fair performance overall for prediction of mortality, but the point estimate of performance improved when restricted to head trauma patients (AUROC=0.75, 95% CI=0.70–0.80 vs AUROC=0.80, 95% CI=0.70–0.90). The motor component of the mGCS showed the best predictive performance (AUROC=0.71); however, the full score performed better than the motor component alone (P=0.004). When assessment was restricted to patients with head injury (n=84), there was no difference in performance between the ATT and mGCS scores (AUROC=0.82 vs 0.80, P=0.67).

Conclusion:

On a large, multi-center dataset of feline trauma patients, the ATT score showed excellent discrimination and calibration for predicting mortality, however an abbreviated score calculated from the perfusion, respiratory, and neurologic categories showed equivalent performance.

Introduction

Traumatic injuries are a common cause of ICU admission and mortality in cats.^1–4 According to the 2012 U.S. Pet Ownership and Demographics Sourcebook, there were an estimated 74 million cats living in households in the United States.⁵ From September 2013 until March 2015, cats made up 17% of the 4336 animals reported to the VetCOT trauma registry. Trauma was the 3^rd most common reason for ICU admission in a Canadian feline population, with an associated mortality rate of 19.2%.⁶ In a survey of 1,325 cats in Taiwan, trauma was the 5^th most common cause of death or reason for euthanasia.⁷ Trauma scores assist in the characterization of patient populations for clinical research and may have application in patient triage and hospital or clinician performance benchmarking.^8–10 Trauma scores facilitate objective assessment of feline trauma patients, and may improve the quality of clinical research and ultimately outcomes in this population.

The use of illness severity scores and trauma scoring in the human medical field is widespread; however, fewer scores have been developed and validated for veterinary patients, particularly feline patients. The Animal Trauma Triage (ATT)¹¹ and modified Glasgow Coma Scale (mGCS)¹² scores are trauma specific illness severity scores which allow objective quantification of injury severity. The scores provide a numeric value for each patient which correlates with a probability of mortality.

The ATT score was developed in 1994 using a small mixed canine and feline trauma population.¹¹ This score is based on a 0–3 scale (0 being slight or no injury, 3 indicating severe injury). The score assesses 6 body system categories (perfusion, cardiac, respiratory, eye/muscle/integument, skeletal, and neurologic) that contribute equally to the overall 0–18 predictive score.¹¹ The ATT score is widely used⁸ in veterinary medicine and has been applied to cats,¹³ but has never undergone external validation on a large feline population.

The Glasgow Coma Scale (GCS) was originally described for use in people with traumatic brain injury in the 1970s.¹⁴ Because this scale relies on the assessment of a patient’s eye, motor, and verbal responses, it was modified for application to veterinary patients. The modified GCS (mGCS) was developed for dogs in 2001 and is based on a 1–6 scale (6 being normal, 1 being severely abnormal) across 3 categories (motor, brainstem reflexes, and level of consciousness).¹² This score has been evaluated in canine head trauma cases but has not been benchmarked against a feline data set, nor have its components been assessed individually.¹²

Despite widespread use of the ATT and mGCS in feline patients, neither of these scores has been validated using a large population of cats. Validating these scores for feline use would allow researchers to confidently apply the scores to cats. The purposes of this study were to evaluate the discriminatory performance of the ATT and mGCS as mortality predictors using a large feline trauma data set and to determine if reweighting or eliminating any of the score components could improve score performance in this population. Our hypothesis was that the ATT and mGCS could be validated for feline use.

Materials and Methods

The American College of Veterinary Emergency and Critical Care Veterinary Committee on Trauma (ACVECC-VetCOT) established a trauma registry for canine and feline patients in 2013. At the time of this analysis, the registry contained patient information submitted from 9 veterinary hospitals located in North America representing Level I and II trauma centers in both private referral practices and veterinary teaching hospitals. All hospitals had both intensive care units (ICU) and non-ICU areas where patients were hospitalized. For each patient, the database contains information on signalment, type of trauma, outcome, and mGCS and ATT scores recorded within 6 hours of admission. The mGCS and ATT sub-scores for each scoring category were available for review, but not the specific physiologic data resulting in the assignment of each subscore. The mGCS has a total possible score ranging from 3 to 18, with a lower score reflecting greater abnormality. The ATT score results in a possible total score range from 0–18, with a higher score reflecting greater abnormality. Data was collected for all cats and dogs presenting to the trauma centers as inpatients or outpatients between September 2013 and March 2015 that had history and clinical signs consistent with traumatic injury.

Statistical methods

All statistical calculations were performed using commercial software.^a. Descriptive data was assessed for normality using the Shapiro–Wilk test. Parametric data are summarized as mean (± standard deviation [SD]), while non-parametric data are summarized as medians (inter-quartile range [IQR]). Parametric and non-parametric hypothesis tests were used as applicable. Continuous data were compared using either a Student’s t-test or Mann–Whitney-U test, as appropriate for the data distribution. Data were characterized as hierarchical in structure, with nesting of patients within hospitals. Violation of the independence assumption was controlled for through use of mixed-effect logistic regression models with random intercepts at the hospital level. The log-likelihood was estimated using adaptive Gaussian quadrature, with 7 integration points. The number of integration points was assessed using the criteria of <0.01% change in coefficients with a doubling of integration points to indicate sufficiency. Postestimation model checking was performed using examination of Pearson and deviance residuals, together with dispersion parameters. We examined the individual predictor sub-scores of the mGCS and ATT for availability in the database and assessed score linearity with respect to survival at discharge. Survival models were constructed using the sub-scores of the mGCS and ATT individually and as simple sums. Clustering on hospital was controlled for using random intercept logistic regression models. Survival models were evaluated for discrimination using the area-under the receiver operating characteristic curve (AUROC), and the overall percentage of variability explained by the model was evaluated by calculation of the pseudo R². Model calibration was assessed with the Hosmer–Lemeshow statistic and Akaike’s information criteria. Statistical significance of differences between AUROC values were calculated using the non-parametric method of Hanley and McNeil.¹⁵

Results

Population characteristics

The VetCOT dataset used in this analysis contained a total of 720 cats and 3616 dogs. Of these, outcome information together with concurrent mGCS and ATT scores were available for 711 cats and 3616 dogs. The overall feline mortality was 16.5% (95% confidence interval [CI]=13.9–19.4, n=117). This was substantially higher than dogs in the same population (mortality 7.3%, 95% CI=6.5–8.2, P<0.001) Ninety percent (n=105) of feline deaths occurred by euthanasia. Where euthanasia was performed, 20% (n=21) were recorded as being predominantly financially driven. The median ATT score was lower (P=0.02) in euthanized cats (median 5, IQR=4) than in euthanized dogs (median=6, IQR=3).

The median age of cats in this dataset was 4 y (IQR=6.6). Four hundred twenty cats (59%) were male and 273 (38%) were female with the sex of 18 cats unknown. Two hundred twelve (78%) of female cats were neutered and 347 (83%) male cats were neutered. The median bodyweight of cats in the dataset was 4.4 kg (IQR=2). One hundred seventy five (24.6%) cats were hospitalized in an ICU. Eighty-four (11.8%) cats were suspected to have suffered head injury as a component of their trauma. Where recorded (n=523), median time from trauma to admission was 14 h (IQR=17.3). The median time from admission to discharge or death was 3.7 h (IQR=25.4).

ATT score performance

The relationship between ATT score and mortality risk is shown in Figure 1. Twenty-six percent of cats received an ATT score of 1, 17% received a score of 2, and 13% received a score of 3. Only 9 cats received an ATT score >10, and no animals had ATT scores of 15, 16 or 17. The ATT score showed good linearity with respect to survival, with the exception of scores of 6 (n=32, mortality risk=28.1%) and 9 (n=11, mortality risk=63.6%) which showed lower mortality risks than scores of 5 and 8, respectively. Each ATT score increase of 1 point was associated with an increase in mortality odds of 1.78 (95% CI=1.61–1.98, P<0.001).

Figure 1:

Graph showing the association between mortality risk and Animal Trauma Triage (ATT) score in a population of 711 injured cats derived from the VetCOT trauma registry.

The discrimination performance of the full ATT score was excellent, with AUROC=0.87 (95% CI 0.84–0.90) and pseudo R²=0.30. The model showed reasonable calibration (P=0.22). The percentage of all cats correctly classified to their survival outcome was 87.2%, with 66 of 117 deaths accurately predicted by the model. An ATT score of 6 corresponded to a mortality probability of 0.41, while an ATT score of 7 corresponded to a mortality probability of 0.56. The predictive characteristics of varying scores are shown in Table 1.

Table 1.

Predictive performance of the Animal Trauma Triage (ATT) score when applied to 711 cats following traumatic injury. Data was obtained from the VetCOT trauma registry. ATT= Animal Trauma Triage; AUROC= area under the receiver operatoing characteristic curve; AIC=Akaike information criteria; OR=odds ratio

Trauma severity group (n)	AUROC	Correctly classified	Sensitivity	Specificity	No. of non-survivals predicted by the ATT	Actual no. of non-survivals
Full population (n=711)	0.87	87.2	39.32	96.63	66	117
ATT>0 (n=625)	0.85	85.44	39.32	96.06	66	117
ATT>1 (n=438)	0.78	79.9	40.35	93.83	66	114
ATT>2 (n=310)	0.74	74.52	43.8	90.2	66	105
ATT>3 (n=207)	0.68	68.6	50.5	82.76	66	91
ATT>4 (n=148)	0.68	67.6	62.16	72.97	66	74
ATT>5 (n=98)	0.74	70.41	83.64	53.49	66	55
ATT>6 (n=66)	0.64	69.7	100.0	0	66	46

The ability of the ATT to predict survival was also evaluated with models based on its individual component sub-scores (perfusion, cardiac, respiratory, eye/muscle/integument, skeletal, neurologic) to determine the most predictive components of the score. The results are shown in Table 2. The least predictive component was the eye/muscle/integument sub-score. The three most predictive components were the perfusion, neurologic, and respiratory sub-scores. Starting with the least predictive, sub-scores were sequentially eliminated from the model, and the discrimination of the resulting model compared to the full ATT score until a statistically significant loss of discrimination occurred. Sequential elimination of sub-scores found no difference in performance between the 6-category ATT score and an abbreviated model (ATTnpr) containing only the neurologic, perfusion, and respiratory categories (AUROCs of 0.87 vs 0.86, respectively, P=0.66; Table 3).

Table 2.

Predictive performance of the Animal Trauma Triage (ATT) score applied to 711 cats following traumatic injury. Data was obtained from the VetCOT trauma registry. ATT= Animal Trauma Triage; AUROC= area under the receiver operating characteristic curve; AIC=Akaike information criteria; OR=odds ratio

Model	AUROC (95% CI)	Pseudo R2	AIC	OR, 95% CI and P value
ATT score (full)	0.87 (0.84–0.90)	0.30	449.96	1.78 (1.61–1.97) P<0.001
ATT sub-score (perfusion)	0.76 (0.71–0.81)	0.18	524.19	3.51 (2.76–4.45) P<0.001
ATT sub-score (neurologic)	0.76 (0.71–0.80)	0.18	527.22	3.89 (2.96–5.12) P<0.001
ATT sub-score (respiratory)	0.72 (0.67–0.77)	0.13	557.91	3.50 (2.64–4.64) P<0.001
ATT sub-score (skeletal)	0.63 (0.58–0.68)	0.05	610.30	1.88 (1.50–2.36) P<0.001
ATT sub-score (cardiac)	0.62 (0.57–0.67)	0.06	601.58	2.43 (1.84–3.23) P<0.001
ATT sub-score (eye/muscle/integ)	0.60 (0.54–0.66)	0.04	615.76	1.65 (1.36–2.02) P<0.001

Table 3.

Predictive performance of the full Animal Trauma Triage (ATT) score and with sequential subtraction of subcategories in a population of 711 cats following trauma. Data was obtained from the VetCOT trauma registry. ATT = Animal Trauma Triage; AUROC= area under the receiver operating characteristic curve; e/m/i=eye, muscle, integument sub-score; c=cardiac sub-score; s=skeletal sub-score; r=respiratory sub-score

Model	AUROC (95% CI)	Comparison test P value of sub-model AUROCs to full model
ATT score (full)	0.87 (0.84–0.90)	--
ATT score – (e/m/i)	0.85 (0.81–0.88)	P = 0.09
ATT score- (e/m/i+c)	0.85 (0.82–0.89)	P= 0.36
ATT score- (e/m/i +c+s)	0.86 (0.82–0.90)	P = 0.66
ATT score- (e/m/i +s+c+r)	0.83 (0.79–0.87)	P= 0.03

mGCS score performance

The relationship between mGCS and mortality risk in this population of cats is shown in Figure 2. The mGCS was calculated for all cats, regardless of the presence of head trauma, and was recorded as abnormal (<18) in 66 cats (9.3%) not suspected to have experienced specific head injury. Seventy-one percent of cats had mGCS scores of 18,corresponding to a normal exam. mGCS scores <15 were represented by between 0 and 9 cats per score. The mGCS showed overall fair linearity with respect to survival. Each mGCS score decrease of 1 point was associated with an increase in mortality odds of 2.06 (95% CI=1.71–2.48, P<0.001).

Figure 2.

Graph showing the association between mortality risk and modified Glasgow Coma Scale (mGCS) in a population of 711 injured cats derived from the VetCOT trauma registry.

The discrimination performance of the full mGCS on the general trauma population (n=711) was fair, with AUROC=0.75 (95% CI=0.70–0.80). The ability of the mGCS to predict survival was also evaluated with models based on its individual component sub-scores (motor, brainstem reflexes, level of consciousness; Table 4). The most predictive component was the motor sub-score. Starting with the least predictive, sub-scores were sequentially eliminated from the model, and the discrimination of the resulting models compared to the full mGCS until a statistically significant loss of discrimination occurred. Elimination of the brainstem reflexes sub-category from the overall mGCS resulted in no detectable loss of performance (P=0.24); however, the absolute value of the AUROC decreased from 0.75 to 0.74 (Table 5).

Table 4.

Predictive performance of the modified Glasgow Coma Scale (mGCS) and components in a population of 711 cats following trauma. Data was obtained from the VetCOT trauma registry. AUROC= area under the receiver operating characteristic curve; mGCS = modified Glasgow Coma Scale; e/m/i=eye, muscle, integument sub-score; c=cardiac sub-score; s=skeletal sub-score; r=respiratory sub-score

Model	AUROC (95% CI)	Pseudo R2	AIC	OR and P value
mGCS(full)	0.75 (0.70–0.80)	0.18	519.29	2.06 (1.72–2.47) P<0.001
mGCS motor	0.71 (0.66–0.76)	0.15	542	4.77 (3.21–7.10) P<0.001
mGCS cons	0.67 (0.63–0.72)	0.13	560.04	4.15 (2.72–6.35) P<0.001
mGCS brain	0.62 (0.57–0.66)	0.09	583.92	2.64 (1.92–3.64) P<0.001

Table 5.

Predictive performance of the full modified Glasgow Coma Scale (mGCS) score and with sequential subtraction of subcategories in a population of 711 cats following trauma. Data was obtained from the VetCOT trauma registry. AUROC= area under the receiver operating characteristic curve; mGCS = modified Glasgow Coma Scale; mGCS-brain= brain stem reflexes category; mGCScons=level of consciousness category

Model	AUROC (95% CI)	Comparison test P value of sub-model AUROCs to full model
mGCS score (full)	0.75 (0.70–0.80)	--
mGCS-brain	0.74 (0.69–0.79)	P = 0.24
mGCS- (brain+cons)	0.71 (0.66–0.76)	P = 0.004

When the population was restricted to cats with head injury (n=84) the point estimate AUROC of the mGCS increased compared to the AUROC when applied to the general population (AUROC=0.80, 95% CI=0.70–0.90 vs AUROC=0.75, 95% CI=0.70–0.80, respectively).

When evaluated on the general population, ATT score discrimination for prediction of survival was better than the mGCS (AUROC=0.87 [95% CI=0.84–0.90] vs AUROC=0.75 [95% CI=0.70–0.80], respectively, P<0.001). When evaluated on the population restricted to cats with head injury, there was no difference in the discriminatory capacity of the 2 scores (AUROC=0.82 [95% CI=0.73–0.92] vs AUROC=0.80 [95% CI=0.70–0.90], respectively, P=0.67). The ROC curves for the ATT and mGCS scores on the general population are shown in Figure 3.

Figure 3.

Area under the receiver operating characteristic curve for Animal Trauma Triage (ATT) and modified Glasgow Coma Scake (mGCS) in a population of 711 injured cats derived from the VetCOT trauma registry

Discussion

This study validated the predictive performance of the ATT and mGCS scores using a large multicenter feline trauma dataset. The overall predictive performance, discrimination, and calibration of the ATT score was excellent with an AUROC for the full score of 0.87 (95% CI 0.84–0.90). Each increase in 1 point of the ATT score resulted in a 1.78 times decrease in survival odds. This is less than reported in the original description of the ATT score in a mixed population of cats and dogs, where a 1 point increase resulted in a 2.3–2.6 times decreased likelihood of survival.¹¹ This difference may be attributable to improvements in clinical care of trauma patients over the last 20 years, or to the characteristics of a mixed canine and feline population.

Each subcategory of the ATT was not equally predictive of survival. The perfusion, neurologic, and respiratory subcategories were the best predictors. When the eye/muscle/integument, cardiac, and skeletal subcategories were removed, there was no significant loss of ATT score predictive ability. Thus a reduced score calculated on the perfusion, neurologic, and respiratory subcategories alone will have equivalent predictive power and may be less labor intensive to calculate. This finding may facilitate use of the score in future trauma research, and may result in wider usage of the ATT for evaluating feline trauma victims.

The mGCS had fair discriminatory performance overall when applied to the entire feline trauma population. This indicates that the mGCS can offer reasonable discriminatory performance regardless of a specific history of, or injuries consistent with, head trauma. The mGCS may serve in this context as a proxy variable for a systemic shock state in patients where motor function and level of consciousness are compromised by perfusion abnormalities. It is also possible that a proportion of polytrauma patients have occult head trauma which was not specifically identified. In the veterinary setting, this may reflect difficulties with screening trauma patients for head injury by means of verbal questioning of owners. When the study population was restricted to patients with known history or physical exam consistent with head trauma, the mGCS performance increased (AUROC 0.80). Despite this improvement, the mGCS falls well below the discriminatory capacity (AUROC 0.908–0.946) of the same test for traumatic brain injuries in people.¹⁶ This may be related to the increased accuracy in scoring the mentation and brainstem reflex categories in human patients due to verbal communication. When the mGCS categories were tested, the motor category was found to be the most predictive and the brainstem reflex category was found to be the least predictive for survival. However, the predictive ability of the score was not improved when the brainstem reflex or level of consciousness categories were removed. There was no significant difference in discriminatory capacity between the ATT and mGCS when restricted to head trauma patients (AUROC 0.82 vs AUROC 0.80, respectively), although this study was underpowered to specifically compare one score against another in this population.

Several limitations of this study exist. Use of the VetCOT registry, while advantageous due to its large multicenter dataset, carries the disadvantage of limited opportunity for data quality control. The raw case data from which the subcategories were calculated was not available for review. This precluded any attempt to redefine the calculation of subcategory scores in order to attempt to improve score performance. Relatively few cats received higher ATT scores and relatively few cats were assigned very low mGCS scores, which also limited the power of this analysis. Additionally, as with any retrospective veterinary study, there is the risk that euthanasia bias caused inflation of the assessment of score performance, especially if medical providers were allowing the calculated scores to influence their recommendations to owners.

The ATT score provided excellent predictive performance for survival in this large population of feline trauma patients. Omission of the eye/muscle/integument, cardiac, and skeletal subcategories does not reduce ATT score discriminatory capacity in injured cats, and may facilitate ease of use for trauma researchers and clinicians.

Abbreviations:

ATT

animal trauma triage

ATT(npr)

abbreviated ATT, containing only data from the perfusion, neurologic and respiratory categories

AUROC

area under the receiver operating characteristic curve

CI

confidence interval

IQR

interquartile range

mGCS

modified Glasgow coma scale

SD

standard deviation