Outcome prediction model for severe traumatic brain injury

Abstract

Aim

Treatment of severe traumatic brain injury is aided by better prediction of outcomes. The purpose of the present study was to develop and validate a prediction model using retrospective analysis of prospectively collected clinical data from two tertiary critical care medical centers in Japan.

Methods

Data were collected from 253 patients with a Glasgow Coma Scale score of <9. Within 24 h of their admission, 15 factors possibly related to outcome were evaluated. The dataset was randomly split into training and validation datasets using the repeated random subsampling method. A logistic regression model was fitted to the training dataset and predictive accuracy was assessed using the validation data.

Results

The best model included the variables age, pupillary light reflex, extensive subarachnoid hemorrhage, intracranial pressure, and midline shift. The estimated area under the curve for the model development data was 0.957, with a 95% confidence interval of 0.926–0.987, and that for validation data was 0.947, with a 95% confidence interval of 0.909–0.980.

Conclusion

Our predictive model was shown to have high predictive value. It will be useful for review of treatment, family counseling, and efficient allocation of resources for patients with severe traumatic brain injury.

Introduction

Severe traumatic brain injury (TBI) is a major cause of death and severe disability after trauma.1, 2 Outcome prediction is useful for family counseling, evaluation of treatment effectiveness, efficient use of limited medical resources, and efficient design and conduct of randomized clinical trials. Although numerous investigators have proposed prognostic models for TBI, a mathematical model with high predictive value has not yet been established.1, 2, 3, 4, 5, 6 Previously, we developed a prediction model with high predictive value using multivariate logistic regression.7 However, this model was limited in that it was derived from a single institute and was not subjected to validation analysis. The purpose of the present study was to develop an improved prediction model using data from two tertiary critical care medical centers and to examine its predictive value using a sophisticated statistical method.

Methods

Patients

Three hundred and ten TBI patients with a Glasgow Coma Scale (GCS) score of <9 within 12 h after injury were registered in our database during the period June 1997–October 2009. Of these 310 patients, 57 were excluded from the study due to: >2 h transport time to the critical care centers (n = 17); alcohol‐induced coma (n = 14); circulation collapse due to hemorrhagic shock (n = 11); neurologically dead upon admission (n = 10); and no intracranial pressure (ICP) monitor indwelling due to minor injury (n = 5). Thus, the study group comprised 253 patients from two tertiary critical care medical centers. This study was carried out according to the principles of the Declaration of Helsinki and approved by the institutional review board at Osaka University Hospital (approval number: 12220). The board waived the need for informed consent because this was a retrospective study using clinical data.

Initial management

All patients received the same initial standardized treatment protocol, which included appropriate resuscitation and stabilization in accordance with the Advanced Trauma Life Support Guidelines8 and immediate neurologic evaluation (GCS score, pupil size and reactivity, and neurologic deficits). Patients were examined by computed tomography (CT) as soon after stabilization as possible. According to CT findings, patients were taken either to the operating room for surgical evacuation of significant space‐occupying lesions or directly to the intensive care unit. Computed tomography scanning was repeated 6 and 24 h after injury, as well as immediately after surgery, or when a patient's level of consciousness deteriorated. An ICP monitor was inserted in each patient. A ventricular catheter was used if available; otherwise, a parenchymal or subdural monitoring system was used. Arterial blood samples were taken on admission for blood gas analysis and cell counts.

Treatment of intracranial hypertension

Cerebrospinal fluid drainage was used as a first option for ICP elevation. If cerebrospinal fluid drainage was ineffective in controlling ICP or was not available, barbiturate therapy and then mild hypothermia was provided according to a published protocol.9 Craniectomy was not carried out routinely for refractory intracranial hypertension that was not controllable with mild hypothermia and barbiturate therapy, although it was considered when requested by a patient's family.

Outcome

In all cases, the outcome was assessed prospectively at 6 months after injury according to the Glasgow Outcome Scale.10 Good recovery and moderate disability were considered to be favorable outcomes. Severe disability, persistent vegetative state, and death were considered unfavorable.

Statistical analyses

The present study involved retrospective analysis of prospectively collected clinical data. We considered a total of 15 candidate prognostic variables possibly related to outcome: age, sex, GCS score, pupillary light reflex (LR), ICP, cerebral perfusion pressure (CPP), base deficit, body temperature, PaO₂ (hypoxia), PaCO₂ (hypocapnia or hypercapnia), hypotension, and three features on the CT scan (absence of the cisterns, midline shift, extensive traumatic subarachnoid hemorrhage [Ext‐SAH]) (Table 1). Age, GCS score, LR, body temperature, and systolic blood pressure were recorded on admission. Base deficit, hypoxia, and hypocapnia or hypercapnia were also evaluated on admission by arterial blood gas analysis. Both ICP and CPP were recorded immediately after patients were admitted to the intensive care unit. We calculated CPP by subtracting ICP from mean arterial pressure. The CT features were identified on scans obtained within 24 h of injury, and the most serious findings were recorded. Cistern absence was defined as absence of the basal cisterns including the suprasellar cisterns, ambient cisterns, and quadrigeminal cisterns. Extensive traumatic subarachnoid hemorrhage was defined as the presence of a high density area both in the basal cisterns and over the convexity on the CT scan. Midline shift was measured at the level of the septum pellucidum.

Table 1.

Grades and definitions of variables related to outcomes in 253 patients with severe traumatic brain injury

Variable	Grades and definition	Variable	Grades and definition
Age	Years	Hypoxia	PaO2 <60 mmHg
Sex	Male or female	Hypocapnia	PaCO2 <35 mmHg
GCS score	3–15	Hypercapnia	PaCO2 >45 mmHg
LR	1: Bi/unilaterally present 0: Bilaterally absent	Absent cisterns	0: Basal cistern present 1: Basal cistern absent
		Hypotension	Systolic blood pressure <90 mmHg
ICP	mmHg	Midline shift	mm
CPP	mmHg	Ext‐SAH	0: Absent 1: Present
BD	mEq/L	BT	°C

BD, base deficit; BT, body temperature; CPP, cerebral perfusion pressure; ext‐SAH, extensive subarachnoid hemorrhage; GCS, Glasgow Coma Scale; ICP, intracranial pressure; LR, pupillary light reflex.

Statistical analyses were carried out using SAS version 9.1.3. for Windows (SAS Institute, Cary, NC, USA) and in‐house validated Fortran programs. The repeated random subsampling method was used to build a substantially predictable and robust model to an independent dataset. The dataset was first randomly split into training and validation datasets. Then, for each such split, a logistic regression model was fitted to the training dataset and predictive accuracy was assessed using the validation data. To assess predictive accuracy, positive predictive value, negative predictive value, total predictive value, and sensitivity and specificity were calculated. In addition to the receiver operating characteristic (ROC) curve, the area under the ROC curve (AUC) and its 95% confidence interval (CI) were calculated. These are commonly used as a summary measure of a model's predictive accuracy. The subset of variables with the largest value of AUC was selected as the most predictive.

Results

Patient outcomes at 6 months after injury are shown in Table 2. Seventy‐six percent of 253 patients had an unfavorable outcome, and 24% had a favorable outcome. The results of univariate analysis with logistic regression are shown in Table 3. Age, GCS score, LR, ICP, CPP, hypercapnea, cistern absence, midline shift, and Ext‐SAH were significantly associated with outcome (P < 0.05). The variables selected by repeated random subsampling validation were age, Ext‐SAH, LR, ICP, and midline shift. The results of the repeated random subsampling validation analysis are shown in Table 4. The odds ratios for each these variables were 1.072, 11.95, 0.157, 1.043, and 1.087, respectively.

Table 2.

Glasgow Outcome Scale in 253 patients 6 months after severe traumatic brain injury

Glasgow Outcome Scale	n (%)
Death	115 (45.5)
Persistent vegetative state	40 (15.8)
Severe disability	37 (14.6)
Moderate disability	26 (10.3)
Good recovery	35 (13.8)

Table 3.

Univariate analysis of variables related to outcomes in 253 patients with severe traumatic brain injury

Variable	Median (range)	Percent	OR (95%CI)	P‐value
Age	49 (0–91)	—	1.062 (1.042–1.079)	0.000
Sex (male)	—	67	0.581 (0.303–1.117)	0.103
GCS score	6 (3–15)	—	0.850 (0.760–0.950)	0.004
LR (present)	—	57	0.139 (0.063–0.308)	0.000
ICP	18 (0–147	—	1.060 (1.034–1.087)	0.000
BD	2 (−9–21)	—	1.004 (0.932–1.082)	0.912
CPP	64 (0–120)	—	0.962 (0.948–0.975)	0.000
BT	36 (29–40)	—	0.965 (0.760–1.225)	0.769
Hypoxia	—	9	0.874 (0.328–2.328)	0.788
Hypocapnia	—	40	1.739 (0.934–3.239)	0.081
Hypercapnia	—	15	0.469 (0.225–0.979)	0.044
Hypotension	—	8	3.052 (0.687–13.546)	0.142
Absent cisterns	—	17	17.32 (2.331–128.6)	0.005
Midline shift	5 (0–29)	—	1.144 (1.075–1.217)	0.000
Ext‐SAH	—	70	11.94 (6.122–23.31)	0.000

—, not applicable. BD, base deficit; BT, body temperature; CI, confidence interval; CPP, cerebral perfusion pressure; Ext‐SAH: extensive subarachnoid hemorrhage; GCS, Glasgow Coma Scale; ICP, intracranial pressure; LR, pupillary light reflex; OR, odds ratio.

Table 4.

Results of repeated random subsampling validation analysis of factors possibly related to outcome in patients with severe traumatic brain injury (n = 253)

Covariate	Coeff(β)	SE(β)	P value	OR	95%CI
Age	0.069	0.019	0.000	1.072	1.051–1.093
Ext‐SAH	2.481	0.902	0.000	11.95	4.849–29.45
LR	−1.852	0.975	0.000	0.157	0.059–0.416
ICP	0.042	0.020	0.000	1.043	1.023–1.064
Midline shift	0.084	0.068	0.016	1.087	1.016–1.163

CI, confidence interval; Coeff(β), coefficient; Ext‐SAH, extensive subarachnoid hemorrhage; ICP, intracranial pressure; LR, pupillary light reflex; OR, odds ratio; SE(β), standard error of coefficient.

According to these results, a statistical model for outcome prediction was developed as: Pu = exp (B)/1 + exp (B), where Pu is the probability of an unfavorable outcome, and exp (B) is the exponential function of B, where B = 0.069 ∗ age (years) + 0.042 ∗ ICP (mmHg) + 0.084 ∗ midline shift (mm) + 2.481 ∗ ext‐SAH (1 or 0) − 1.852 ∗ LR (1 or 0) − 3.098.

Both the model development data and validation data had considerably high AUC values. The AUC for model development data was estimated to be 0.957, with a 95% CI of 0.926–0.987, and the AUC for validation data was 0.947, with a 95% CI of 0.909–0.980.

The probability of an unfavorable outcome for all the patients (n = 253) with favorable and unfavorable outcomes is shown in Figure 1. If the cut‐off value was imposed at 0.517, the positive predictive value of this model was 94.7% (180/190), negative predictive value was 81.0% (51/63), and total predictive value was 91.3% (231/253). Sensitivity was 93.8% (180/192), and specificity was 83.6% (51/61). The ROC curve is shown in Figure 2. The AUC for the model was 0.956, with a 95% CI of 0.925–0.987.

Figure 1.

Probability of an unfavorable outcome (Pu) in 253 patients with severe traumatic brain injury. Each circle represents one patient.

Figure 2.

Receiver operating characteristic curve of our predictive model for all patients with severe traumatic brain injury (n = 253).

discussion

Our previous model was limited in that it was derived from a single institute and was not subjected to validation analysis. Therefore, in the present study, we developed an improved prediction model using data from two tertiary critical care medical centers and examined its predictive value using a sophisticated statistical method. Five indicators (age, Ext‐SAH, LR, ICP, midline shift) were shown to be powerful prognostic indicators with a high predictive value. These findings are consistent with, and build upon, our previous study model, which had the same selected variables, although coefficients are a little different (age, 0.069 versus 0.11; Ext‐SAH, 2.481 versus 3.44; LR, 1.852 versus 4.09; ICP, 0.042 versus 0.12; midline shift, 0.084 versus 0.16).

Recently, two prediction models have been developed based on large datasets. One is published by the Medical Research Council (MRC) corticosteroid randomization after significant head injury (CRASH) trial,3 and the other is by the International Mission for Prognosis and Analysis of Clinical Trials in traumatic brain injury (IMPACT) project.4 The MRC CRASH study included patients with mild to severe traumatic brain injury (GCS ≤14), used variables assessed within a few hours of admission, and developed a prediction model of unfavorable outcome at 6 months after injury. External validation for unfavorable outcome at 6 months in high income countries showed an AUC of 0.77. In the model derived from the IMPACT project, patients with moderate to severe traumatic brain injury (GCS ≤12) were included, admission characteristics were used for model development, and unfavorable outcome at 3 or 6 months was predicted. The authors reported that external validation confirmed the discriminative ability of the model to be adequate (AUC 0.80).

Due to the fact that many factors influence prognosis, it has been difficult to develop an all‐round predictive model for all degrees of traumatic brain injury. In the present study, we focused on severe traumatic brain injury defined as GCS <9 within 12 h of injury. We also excluded several patient categories: those who had more than 2 h transport time to the centers; an alcohol‐induced coma; circulation collapse due to hemorrhagic shock; and those who were neurologically dead upon admission.

Another feature of our model was that predictions were made 24 h after injury. There are some reports of outcome prediction using patient characteristics on admission or variables within a few hours of injury.4, 5 However, those models were not strongly predictive. This may be because there are considerable numbers of patients who deteriorate rapidly after admission. In 20 cases (7.9%) out of 253 patients in the present study, GCS on admission was ≥9, but decreased thereafter (data not shown). Therefore, to improve the predictive value, we selected possible prognostic indicators within 24 h after injury.

The model is likely to provide a more accurate prediction of outcomes as more patients were included in the study and from multiple centers. However, with an increased number of patients and participating institutes it may be more difficult to collect precise and detailed parameters. Although we collected data from only two institutes in this study, we were able to collect such data as ICP, CPP, CT findings, and blood gases because both centers had the same level of facilities and shared the same treatment algorithm. One way of developing the best prediction model may be to choose robust indicators based on a small but well‐examined study, and then use these to calibrate coefficients in a large cohort study.

Limitations

There are several limitations to the present study. First, differences in emergency medical services (EMS) might have influenced the results of the study. For example, in Japan, EMS is not permitted to intubate the trachea except in cases of cardiopulmonary arrest. Although this is unlikely to affect the utility of the five predictive indicators (age, Ext‐SAH, LR, ICP, midline shift), the model may have to be adjusted for different EMS procedures.

Second, even if the prognostic value is high, it is still difficult to make a decision to continue or withdraw the treatment of a patient with severe traumatic brain injury. In our model, 22 patients out of 253 were not correctly predicted. The discrimination of severe and moderate disability was especially difficult.11 The Pu values of 13 (59%) out of the 22 patients were between 0.40 and 0.80 (data not shown). This may highlight a limitation of using a dichotomous classification such as favorable and unfavorable outcome.

Finally, although we carried out cross‐validation using a repeated random subsampling method, external validation based on a completely different dataset is yet to be performed. There is also the possibility that our model was over fitting because the sample size was relatively small and the treatment algorithm was identical in the two institutes. However, erroneous omission of predictors with weaker effect may occur in small samples such as this, due to a lack of power. To examine the true quality of the present model, a validation study in a large cohort will be required.

conclusion

We developed and validated a predictive model of severe traumatic brain injury. Five indicators—age, Ext‐SAH, LR, ICP, and midline shift—were detected, consistent with our previous study. As this new model has a high predictive value, further validation studies in a large patient cohort are warranted.

Conflict of interest

None.