Abstract
Traumatic brain injury (TBI) is a complex injury that has a multi-faceted recovery process. The current “gold standard” for classifying severity of TBI symptoms is the Glasgow Outcome Scale (GOSE), a crude measure of overall dysfunction after TBI. Exploratory factor analysis performed on TRACK-TBI Pilot (N = 297) identified candidate multi-variate outcome measures of neuropsychological impairment and cognitive speed and flexibility at 6 months post-TBI that were confirmed in data from the COBRIT study (N = 645) using confirmatory factor analysis. These new outcome measures were used as the dependent variables in an ordinal logistic regression model, using common data elements (CDE) collected in the emergency department as independent variables, including basic demographics, socioeconomic status, medical history, and measures of blood alcohol and blood pressure. We directly compared these prediction models with the GOSE as the 6-month outcome variable and found that in both the TRACK-TBI pilot and COBRIT studies, both neuropsychiatric complications (approx. 36.0% and 22.3% variance explained) and cognitive speed and flexibility (approx. 33.9% and 24.5% variance explained) were better explained by the prediction model, compared with GOSE (approx. 19.9% and 14.4% variance explained), respectively. While differences in overall distributions of impairment between TRACK-TBI pilot and COBRIT exist and should be explored further for applications of these prediction models, we think these multi-variate end-points more accurately characterize patients' functioning at six-months post-TBI. A multi-variate assessment of end-points seems especially important for characterizing TBI outcomes in cases where gross impairment, such as those measured by the GOSE, may be less evident.
Keywords: cognitive function, human studies, neuropsychology, outcome measures, traumatic brain injury
Introduction
Traumatic brain injury (TBI) is a major cause of death and disability in the United States. In 2014, there were approximately 2.53 million emergency department (ED) visits, 288,000 hospitalizations, and 56,800 deaths related to TBI.1 The TBI pathogenesis is a complex process resulting from primary and secondary injuries that lead to temporary or permanent neurological damage. The primary injury is directly related to the external impact of the brain. The secondary injury consists of a molecular, chemical, and inflammatory cascade that results in further brain damage and occurs after the primary injury, but the temporal window in which occurs ranges from minutes to days.2 Although TBI is a leading cause of death and disability, there is no data-driven outcome measure to quantify the magnitude of the TBI severity.
The Glasgow Outcome Scale (GOS) was developed to approximate the severity of TBI injury to support communications among clinicians and researchers.3 The original GOS had five ordered categories: Death, Vegetative State, Severe Disability, Moderate Disability, and Good Recovery. It was argued, however, that these five outcome categories did not capture the wide range of mental and physical handicaps a patient can have after TBI. The Glasgow Outcome Scale Extended Scale (GOSE) was intended to bridge these pockets of ambiguity by splitting each of the Severe Disability, Moderate Disability, and Good Recovery into upper and lower categories.4 Even still, the GOSE is an eight-point ordinal scale that is used to quantify the complexity of dysfunction after a TBI.
A TBI, however, affects multiple biological systems that are difficult to quantify using a single-dimensional metric.5 There are other measures that try to capture this complexity by looking at cognitive function or psychological status, such as processing speed or symptoms of depression and anxiety; however, as univariate measures, they are only capturing one dimension of this pathology. Others have examined multiple bivariate associations among outcome measures to understand the intersections of these domains,6 but these approaches are not able to capture underlying dimensions of variance that account for pairwise associations among scales. Multi-dimensional outcome measures comprised of several measurement domains could more accurately represent the relationships between these domains.
To accurately make a diagnosis and properly treat patients 6 months after their TBI event, clinicians involved in high-stakes clinical treatment trials of TBI need as much detail as possible regarding the patients' health outcomes. The GOS attempted to categorize the outcome into five categories, and the GOSE improved on this measure by splitting each of the Severe Disability, Moderate Disability, and Good Recovery into upper and lower categories.4 Although categorical classification schemes have benefits in interpretability and ease of use, the drawbacks in the subjectivity and ambiguity intrinsic to the rigidity of such schemes far outweigh the benefits. For instance, we cannot glean any information about individual recovery from measures of GOSE.
A previous study attempted to justify the use of multi-variate statistical methods for analyzing TBI outcome by considering multiple domains consisting of relevant outcome measures in their exploratory data analysis. The authors concluded that analyses of outcome measures across the functional domains will provide more comprehensive results pertaining to individual patient recovery after TBI.4
We draw inspiration from the conclusions of several investigations exploring the relationship between TBI patient outcomes and clinical predictors. One approach identified reproducible subclasses of TBI that correlate with patient outcomes.7 A significant amount of unique information about patient recovery was discovered after creating groups of patients considering the heterogeneity of TBI and comparing the results of their TBI outcomes, such as GOSE, psychological status, and neuropsychological impairment. Other approaches found that while clinical predictors collected in the ED, such as education, pre-injury psychiatric disorders, and previous TBI, were strong predictors of functional outcome at 6 months after mild traumatic brain injury (mTBI),8 computed tomographic (CT) characteristics were not predictive of mTBI.9
The present study built on these conclusions to create a multi-variate outcome measure that is more data-driven and interpretable than the GOSE. We hypothesized that our new outcome measure would capture the heterogeneous nature of TBI, and that we could generate a more accurate clinical prediction model for these new outcomes using data collected from the patients' initial ED visits.
We tested our hypothesis on the TRACK-TBI Pilot and Citicoline Brain Injury Treatment Trial (COBRIT) datasets retrieved from the Federal Interagency Traumatic Brain Injury Research (FITBIR) repository and used in two previous analyses of TBI outcomes.7,8 The FITBIR is a bioinformatics platform created to share existing TBI datasets and facilitate collaboration between laboratories conducting TBI research. Given the high dimensional data in both datasets, we approached the problem from a multi-variate perspective.
The current study aimed to test the accuracy of the following models in predicting TBI severity: (1) a model with GOSE as the dependent outcome variable with the ED clinical predictors and (2) models with multi-variate outcomes as the dependent variable with the ED clinical predictors. We compared the goodness of fit in each of the prediction models. We hypothesized that the multi-variate outcomes would be more accurately predicted than the univariate GOSE outcomes alone. This finding would lend support for the use of multi-variate outcomes as more precise assessments of patient outcomes post-TBI.
Methods
Datasets mined from FITBIR
The TRACK-TBI Pilot and COBRIT datasets were mined from FITBIR, and have been used in two previous studies assessing prediction of TBI outcomes based on the GOSE. These datasets included many overlapping data elements collected both during neurocritical care immediately after TBI, as well as outcomes collected six months post-TBI. The datasets were processed according to the PRISMA diagram in Figure 1.
FIG. 1.
PRISMA diagram of data preparation and analysis. Color image is available online.
The TRACK-TBI multicenter observational pilot study was designed to assess the implementation of the TBI Common Data Elements (TBI-CDEs) to support data sharing in TBI research (FITBIR-STUDY0000246). A total of 650 patients who received CT scans in the ED within 24 h of injury were enrolled at three level I trauma centers and one rehabilitation center.10,11 We analyzed TRACK-TBI Pilot data between April 2010 and May 2011 in which 599 patients with acute TBI were enrolled. As displayed in Figure 1, we excluded 27 patients who were under 18 years old and who did not complete outcome assessments at six-months post-injury, resulting in an analysis sample of 297 individuals.
The COBRIT is a phase 3 double-blind randomized clinical trial examining efficiency of citicoline compared with placebo for improving outcomes in TBI patients (FITBIR-STUDY0000240).12,13 The dataset consists of 1213 patients at eight level I trauma centers in the United States and was used to replicate the multi-variate outcomes identified in the TRACK-TBI Pilot and assess the predictive abilities of the ED clinical variables in a more severe TBI cohort. Only participants who completed the outcome assessments at six months post-TBI were included, resulting in an analysis sample of N = 645 (Fig. 1).
We include syntax used to pre-process the data, perform multiple imputation of missing clinical predictors, generate descriptive statistics, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA) (Supplementary Data 1), as well as a data dictionary to accompany the R code (Supplementary Data 2), code to generate the graphical results (Supplementary Data 3), and SPSS syntax for regression models (Supplementary Data 4). We provide more detailed descriptions of the statistical analyses used in the Technical Appendix.
TBI outcomes and ED clinical predictors
The reference TBI outcome is the GOSE. The other TBI outcomes consist of variables from the following domains: neuropsychological impairment, psychological status, TBI-related symptoms, and perceived health-related quality of life. Variables from the physical function domain, such as functional independence, were not included in the analysis because of an abundance of missing values. From the neuropsychological impairment domain, we used the completion time measurements from parts A and B of the Trail Making Test (TMT)14 and the Wechsler Adult Intelligence Scale-Fourth Edition Processing Speed Index (WAIS-IV PSI).15,16
From the psychological status domain, we used the three subcategories of the Brief Symptom Inventory (BSI-18),17,18 which includes anxiety, depression, somatization, and global severity scores. Last, we used scores from Satisfaction With Life Scale (SWLS)19,20 in the perceived health-related quality of life domain. All outcomes were assessed at six months post-injury.
The ED clinical predictors consist of gender, age, employment status, education status, marriage status, previous TBI injury, injury severity, blood alcohol level, systolic/diastolic blood pressure, intravenous saline application, and medical history of type social, neurological, and psychiatric. These predictors parallel those included in previous studies of the GOSE using the same data7,8; we aimed to compare their performance predicting the GOSE versus the multi-variate outcomes derived in the present study.
Dimension reduction layout
Dimension reduction techniques were used to identify multi-variate TBI outcome domains that could explain shared variance across the measures of neuropsychological, psychological status, TBI symptoms, and health-related quality of life measures. The multi-variate outcome domains were first derived using EFA on the TRACK-TBI Pilot measures, and then were cross-validated using CFA in the COBRIT dataset.
EFA
This analysis using principal components extraction was used to identify underlying dimensions of variance across the TBI outcome domains in the TRACK-TBI Pilot. Principal components extraction uses eigenvalue decomposition of the observed-measure correlation matrix to extract orthogonal sources of variance that account for the interrelations among the observed variables.21 The components associated with the largest proportions of variance in the data were selected for interpretation as multi-variate outcome factors. The criteria for selecting components included a minimum cutoff of an eigenvalue of 1.0 and observed disjunctions in the scree plot of eigenvalues indicating a plateau in the additional variance explained through the inclusion of additional factors.
CFA
This analysis was used to assess the replicability of the multi-variate outcome dimensions identified in the TRACK-TBI Pilot dataset in the COBRIT sample. The CFA assumes the covariation structure among a set of variables can be described as a linear combination of latent variables or unobserved variables called factors. The CFA estimates the relations between user-specified factors and the observed measures via maximum likelihood procedures. As such, we specified the relations between the factors and observed measures in COBRIT based on the results of the EFA in the TRACK-TBI Pilot.
The fit and loadings associated with the CFA model in the COBRIT sample were used to assess the replicability of the most salient multi-variate outcome dimensions identified in the TRACK-TBI Pilot. Because the CFA model does not attempt to explain the entirety of the variance-covariance matrix as with EFA, model residuals and the correspondence between the model-estimated and observed covariance matrices could be examined to detect misfit of the model in the replication dataset. The comparative fit index (CFI), root mean squared error of approximation (RMSEA), and squared root mean residual (SRMR) were used to determine the extent of misfit of the TRACK-TBI Pilot multi-variate outcome dimensions in the COBRIT sample.22,23
Prediction model layout
We compare the performance of proportional odds logistic regression models that regress either the GOSE or the multi-variate outcomes onto the ED predictors. In addition, we identify predictors that emerge as significant across models as well as those that vary among the models with different outcomes. Using this information, we attempt to characterize the performance of the ED predictors in relation to participants' prognosis as described by either the simpler GOSE characterization or by the data-driven, multi-variate outcomes across the TRACK-TBI Pilot and COBRIT samples.
We entered an identical set of predictors into the regressions in each sample, but allowed the weighting of predictors (i.e., the regression coefficients) to vary across models given differences in TBI severities and patient characteristics. We therefore ran models regressing the GOSE and multi-variate outcomes onto the same set of predictors in each sample, allowing the predictors to be uniquely weighted according to the samples and outcomes.
Proportional odds logistic regression
The proportional odds logistic regression was used to assess the relative influence and overall explanatory ability of the clinical predictor variables across samples and outcomes.24 Before running the proportional odds logistic regressions, the outcomes data were pre-processed to maximize comparability of the models and outcomes across the TRACK-TBI Pilot and COBRIT samples.
Discretization was used to transform the continuous distributions of the multi-variable outcome measures into six-category, ordinal scales for comparison with the GOSE severity rankings. To achieve this discretization, each multi-variable outcome was cut at percentiles corresponding to -2, -1, 0, 1, and 2 standard deviations of an underlying Gaussian distribution. This transformation increased the comparability of models predicting the GOSE and multi-variate outcomes such that the estimation procedures were equivalent and the resulting parameter estimates had the same interpretations. For instance, the exponentiated regression coefficients from either the GOSE or the multi-variate outcome models would similarly reflect the increased odds of scoring in a higher category across the category thresholds given a one-unit change in the predictors.
In addition to discretizing the outcome variables, the predictor variables were pre-processed according to their variable types. Binary categorical variables (i.e., gender, employment, psychiatric history, drug and alcohol abuse history, TBI history, and whether participants received intravenous saline in the ED) were converted to dummy-codes such that a one-unit change in the variable was indicative of the difference between categories. Two dummy variables were computed for participant education: one variable indicated whether participants attended college and another indicated whether participants did not complete high school, such that high school completion was the reference category.
In addition, three dummy variables were computed for participant marital status: one variable indicated whether or not participants were married, the second indicated whether participants were divorced, and the third indicated whether participants had been widowed, such that having never been married served as the reference category. Finally, an additional binary variable was included to account for zero-inflation in the blood alcohol levels predictor (1 = participant had zero blood alcohol, 0 = participant did not have zero blood alcohol) to account for unique factors associated with having zero blood-alcohol. Imputation via the “mice” package in R was used to handle missing values on the predictor variables.25
Results
Sample characteristics
Table 1 presents the descriptive statistics for all study variables and comparisons across the TRACK-TBI Pilot and COBRIT samples. The samples differed significantly on the majority of ED predictor variables with the exception of gender, and differed significantly on all TBI outcome variables except satisfaction with life and two of the BSI scales. Overall, COBRIT participants tended to show somewhat greater severity in the clinical measures, although this finding was not consistent across all scales.
Table 1.
Clinical Outcomes and Predictors
| Demographic and clinical variables | TRACK full sample (N = 586) n (%) or M(SD) |
COBRIT full sample (N = 1213) n (%)or M(SD) |
Comparison Mann-Whitney U Test χ2 test of homogeneity |
p |
|---|---|---|---|---|
| Gender | χ2(1) = 1.7773 | 0.183 | ||
| Female | 167 (28.5%) | 310 (25.5%) | ||
| Male | 419 (71.5%) | 903 (74.5%) | ||
| Age | 43.3 (18.4) | 39.8 (15.8) | W = 389127 | 0.001 |
| SWLS | 21.4 (7.8) | 22.4 (7.6) | W = 120192 | 0.051 |
| TMT Part A | 35.4 (16.9) | 34.0 (18.8) | W = 112526 | 0.006 |
| TMT Part B | 89.6 (62.8) | 85.9 (60.5) | W = 107433 | 0.060 |
| WAIS PSI | 99.3 (15.7) | 94.8 (17.9) | W = 115505 | < 0.001 |
| BSI 18 anxiety score | 52.7 (11.4) | 52.7 (12.2) | W = 127326 | 0.791 |
| BSI 18 depression Score | 53.2 (11.2) | 54.9 (11.5) | W = 117521 | 0.021 |
| BSI 18 total score GSI | 54.7 (11.4) | 56.29 (12.6) | W = 120832 | 0.109 |
| BSI 18 somatic score | 54.9 (10.7) | 55.4 (11.1) | W = 124536 | 0.399 |
| Employment | χ2(1) = 14.477 | < 0.001 | ||
| Unemployed | 237 (40.4%) | 384 (31.6%) | ||
| Employed | 335 (57.1%) | 816 (67.2%) | ||
| Education | χ2(2) = 7.0199 | 0.030 | ||
| Below high school | 63 (10.7%) | 163 (13.4%) | ||
| High school graduate | 285 (48.6%) | 546 (45.0%) | ||
| College | 199 (33.9%) | 485 (39.9%) | ||
| Marital status | χ2(3) = 18.511 | < 0.001 | ||
| Single | 295 (50.3%) | 548 (45.1%) | ||
| Married | 188 (32.0%) | 460 (37.9%) | ||
| Divorced/Separated | 55 (9.3%) | 174 (14.3%) | ||
| Widowed | 27 (4.6%) | 30 (2.4%) | ||
| Previous TBI | χ2(1) = 109.76 | < 0.001 | ||
| Yes | 115 (19.6%) | 6 (0.4%) | ||
| No | 471 (80.3%) | 582 (47.9%) | ||
| Blood alcohol level | 89.6 (117.1) | 74.3 (110.6) | W = 156279 | .172 |
| Systolic blood pressure | 140.8 (26.8) | 135 (24.1) | W = 389381 | < 0.001 |
| Diastolic blood pressure | 82.5 (18.9) | 75.6 (17.7) | W = 354792 | < 0.001 |
| IV Saline | χ2(1) = 84.975 | < 0.001 | ||
| Yes | 2 (0.3%) | 187 (15.4%) | ||
| No | 583 (99.4%) | 1026 (84.5%) | ||
| Alcohol history | χ2(1) = 538.64 | < 0.001 | ||
| Yes | 308 (52.5%) | 63 (5.1%) | ||
| No | 278(47.4%) | 1150 (94.8%) | ||
| Drug history | χ2(1) = 113.67 | < 0.001 | ||
| Yes | 128 (21.8%) | 63 (5.1%) | ||
| No | 458 (78.1%) | 1150 (94.8%) | ||
| Psychiatric history | χ2(1) = 125.54 | < 0.001 | ||
| Yes | 172 (29.3%) | 107 (8.8%) | ||
| No | 414 (70.6%) | 1106 (91.1%) | ||
| GOSE | χ2(7) = 740.39 | < 0.001 | ||
| Death/Vegetative | 29 (4.9%) | 10 (0.8%) | ||
| Lower/Upper severe | 28 (4.7%) | 108 (8.9%) | ||
| Lower/Upper moderate | 113 (19.2%) | 309 (25.5%) | ||
| Lower/Upper good recovery | 234 (39.9%) | 403 (33.2%) | ||
| GCS | χ2(12) = 2969.2 | < 0.001 | ||
| Minor | 479 (81.7%) | 549 (45.2%) | ||
| Moderate | 28 (4.7%) | 122 (10%) | ||
| Severe | 42 (7.1%) | 542 (44.6%) | ||
| CT | χ2(1) = 378.34 | < 0.001 | ||
| Yes | 290 (49.5%) | 1022 (84.2%) | ||
| No | 296 (50.5%) | 191 (15.7%) |
Bold values indicate statistical significance.
SD, standard deviation; SWLS, Satisfaction with Life Scale; TMT, Trail Making Test; WAIS PSI, Wechsler Adult Intelligence Scale Processing Speed Index; BSI, Brief Symptom Inventory; GSI, Global Severity Index; TBI, traumatic brain injury; IV, intravenous; GOSE, Glasgow Outcome Scale Extended; GCS, Glasgow Coma Scale; CT, computed tomography.
EFA
Before running the EFAs on the full TRACK-TBI Pilot sample, we randomly split the data into two subsets and performed a Kolmogorov-Smirnov test for each pair of variables to assess univariate consistency across the sample. Results suggested that there were no differences in the empirical cumulative distribution functions of the variables, indicating the subsets of the data showed comparable univariate distributions on each measure. We then used eigenvalue decomposition to extract the salient dimensions of variance across the TBI outcome measures. The component loadings and variance explained by each component were similar across splits of the dataset; therefore, the EFA was applied to the full study sample.
The EFA on the TBI outcome measures revealed three salient dimensions of variance accounting for 48.7% and 25.7%, and 8.4% of the variance in the TRACK-TBI Pilot scales. The first dimension showed notable associations with all outcome domains, with the strongest loadings from the SWLS and all BSI-18 subscales. As such, the first component appeared to reflect global neuropsychological impairment. The second dimension was most strongly associated with the TMT and WAIS-Processing Speed scales, suggesting this domain appeared to be characterized by cognitive speed and flexibility. The third dimension was primarily associated with the SWLS and appeared to reflect variability in life satisfaction that was not explained by the primary neuropsychological impairment dimension. The loadings associated with each of these dimensions are shown in the left panel of Table 2.
Table 2.
Exploratory Factor Analysis Loadings and Confirmatory Factor Analysis Results
| |
EFA (TRACK-TBI Data) |
CFA (COBRIT) |
||||
|---|---|---|---|---|---|---|
| Outcome | Factor 1 | Factor 2 | Factor 3 | Factor 1 | Factor 2 | Residual |
| SWLS | 0.66 | 0.16 | 0.69 | -0.58* | 0.66* | |
| TMT Part A | -0.41 | 0.78 | -0.05 | 0.20* | 0.89* | 0.17* |
| TMT Part B | -0.4 | 0.8 | 0.07 | 0.23* | 0.82* | 0.28* |
| WAIS PSI | 0.37 | -0.73 | 0.01 | -0.31* | -0.70* | 0.42* |
| BSI 18 Total score anxiety | -0.81 | -0.3 | 0.22 | 0.86* | 0.26* | |
| BSI 18 Total score depression | -0.88 | -0.21 | -0.1 | 0.86* | 0.26* | |
| BSI 18 Total score GSI | -0.94 | -0.26 | 0.13 | 0.99* | 0.02 | |
| BSI 18 Total score somatic | -0.8 | -0.14 | 0.3 | 0.77* | 0.41* | |
EFA, exploratory factor analysis; CFA, confirmatory factor analysis; SWLS, Satisfaction with Life Scale; TMT, Trail Making Test; WAIS PSI, Wechsler Adult Intelligence Scale Processing Speed Index; BSI, Brief Symptom Inventory; GSI, Global Severity Index. Bold indicates factor loading greater than 0.40 for positive loadings, and less than −0.40 for negative loadings. All CFA loadings and residual variances are standardized. *CFA loading is statistically significant at p < 0.001.
CFA results
Based on the EFA in the TRACK-TBI Pilot sample, a CFA was used to assess the stability of the neuropsychological impairment and cognitive speed and flexibility domains in the COBRIT sample. Consistent with the EFA results, the neuropsychological impairment factor included loadings explaining variance in all outcome measures. The cognitive speed and flexibility domain were in turn represented as an orthogonal factor with loadings explaining variance in the TMT and WAIS-Processing Speed measures. These factors were modeled as orthogonal dimensions to replicate the EFA dimension reduction approach. The satisfaction with life domain identified in the TRACK-TBI Pilot sample was not represented as a factor in this analysis, as it appeared to represent unique variability in the SWLS independent of the other multi-variate dimensions.
The results of our two-factor model indicated adequate model fit based on standard cutoffs χ2[17]2 = 91.255, p < 0.001, RMSEA = 0.082, CFI = 0.98, SRMR = 0.041). The factor loadings and residual variances estimated in the COBRIT sample are presented in the right-hand panel of Table 2.
Multi-variate outcome derivation
The results of the EFAs and CFAs suggested consistency in the first two multi-variate dimensions and in a third, orthogonal satisfaction with life domain. Therefore, we computed scores for the first three dimensions corresponding to neuropsychological impairment, cognitive speed and flexibility, and satisfaction with life in each sample using linear combinations of the items weighted by their component loadings. Positive neuropsychological impairment scores represented greater distress and cognitive impairment, while negative neuropsychological impairment scores reflected lower distress and greater cognitive functioning. Similarly, higher cognitive speed and flexibility scores represented lower speed and accuracy in cognitive assessment tasks, while lower cognitive speed and flexibility scores represented greater speed and accuracy in the cognitive assessments. The satisfaction with life dimension was mostly weighted by the SWLS in both samples, with higher scores representing greater satisfaction with life.
To examine the relations between the multi-variate outcomes and GOSE categories, participants' scores on the joint distributions of neuropsychological impairment, cognitive speed and flexibility, and satisfaction with life were plotted and colored by GOSE severity grouping in both samples. Figure 2 displays the relations among these domains and GOSE severity. In both samples, there appeared to be some differentiation among the GOSE categories based on neuropsychological impairment scores, yet minimal differentiation on either cognitive speed and flexibility and satisfaction with life scores. Altogether, despite the various severity categories represented by the GOSE, it did not align especially well with overall neuropsychological outcomes and hardly differentiated performance on measures of cognitive speed and flexibility or perceived life satisfaction.
FIG. 2.
Pair plot of TRACK-TBI Pilot and COBRIT Multivariate Outcome Scores Colored by Glasgow Outcome Scale Extended (GOSE). The GOSE functional categories were most strongly differentiated by scores on the neuropsychological impairment factor. The GOSE scores were minimally differentiated by scores on the other factors. The COBRIT sample showed greater differentiation of the lowest-functioning GOSE categories (3,4) from higher-functioning categories (5–8) on both multi-variate outcomes and the SWLS. Color image is available online.
Proportional odds logistic regression results
Proportional odds logistic regression analyses were used to assess the capacity of the ED predictors to explain variance in each multi-variate outcome dimension compared with GOSE categories. The satisfaction with life dimension was omitted from these analyses, because it represented a narrower outcome dimension largely explained by the SWLS. Table 3 shows the results of regressions predicting scores in the top GOSE categories, Table 4 shows the results of regressions predicting neuropsychological impairment scaled into six ordinal categories based on proportions observed in an underlying Gaussian distribution, and Table 5 shows the results of regressions predicting cognitive speed and flexibility also scaled into six ordinal categories based on an underlying Gaussian process.
Table 3.
Ordinal Logistic Regressions Predicting Glasgow Outcome Scale Extended Category in Each Study Sample
| Sample |
Track TBI (Nagelkerke R2 = 0.199) |
COBRIT (Nagelkerke R2 = 0.144) |
||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Thresholds | Threshold | SE | Wald | p | Threshold | SE | Wald | p | ||||
| 3:4 | -4.682 | 0.993 | 22.217 | < 0.001 | -4.048 | 0.541 | 56.078 | <0 .001 | ||||
| ≤4:5 | -3.814 | 0.893 | 18.234 | < 0.001 | -3.281 | 0.524 | 39.239 | < 0.001 | ||||
| ≤5:6 | -1.75 | 0.82 | 4.557 | 0.033 | -1.829 | 0.508 | 12.945 | < 0.001 | ||||
| ≤6:7 | -0.498 | 0.811 | 0.377 | 0.539 | -0.805 | 0.504 | 2.55 | 0.110 | ||||
| ≤7:8 | 1.017 | 0.813 | 1.562 | 0.211 | 0.326 | 0.504 | 0.417 | 0.518 | ||||
| Coefficients | Estimate | SE | Wald | p | Estimate | SE | Wald | p | ||||
| Age | -0.013 | 0.008 | 2.491 | 0.115 | 0.002 | 0.006 | 0.094 | 0.759 | ||||
| Gender | -0.629 | 0.259 | 5.893 | 0.015 | -0.317 | 0.168 | 3.547 | 0.060 | ||||
| Any psych. history | -0.183 | 0.263 | 0.486 | 0.486 | 0.221 | 0.252 | 0.769 | 0.381 | ||||
| Alcohol history | -0.022 | 0.254 | 0.008 | 0.930 | -2.143 | 0.29 | 54.67 | < 0.001 | ||||
| Drug History | 0.225 | 0.302 | 0.554 | 0.457 | 0.000 | . | . | . | ||||
| Blood alcohol | -0.003 | 0.002 | 2.31 | 0.129 | 0.001 | 0.001 | 1.163 | 0.281 | ||||
| Zero blood alc. | -0.39 | 0.401 | 0.945 | 0.331 | 0.103 | 0.223 | 0.213 | 0.644 | ||||
| Systolic BP | 0.008 | 0.005 | 2.399 | 0.121 | -0.005 | 0.006 | 0.744 | 0.388 | ||||
| Diastolic BP | -0.004 | 0.008 | 0.334 | 0.563 | -0.002 | 0.004 | 0.147 | 0.701 | ||||
| Employment | 0.047 | 0.259 | 0.033 | 0.855 | -0.135 | 0.158 | 0.724 | 0.395 | ||||
| No high school | -0.548 | 0.421 | 1.695 | 0.193 | -0.546 | 0.239 | 5.21 | 0.022 | ||||
| College degree | 1.426 | 0.266 | 28.798 | < 0.001 | 0.321 | 0.157 | 4.175 | 0.041 | ||||
| Married | 0.215 | 0.287 | 0.559 | 0.455 | -0.205 | 0.209 | 0.964 | 0.326 | ||||
| Divorced | 0.382 | 0.435 | 0.773 | 0.379 | -0.661 | 0.253 | 6.843 | 0.009 | ||||
| Widowed | 0.184 | 0.754 | 0.06 | 0.807 | -1.044 | 0.578 | 3.266 | 0.071 | ||||
| Previous TBI | -0.363 | 0.291 | 1.558 | 0.212 | 0.072 | 0.44 | 0.027 | 0.870 | ||||
| IV Saline | -2.951 | 1.945 | 2.301 | 0.129 | -0.467 | 0.23 | 4.111 | 0.043 | ||||
TBI, traumatic brain injury; SE, standard error; BP, blood pressure; IV, intravenous.
Note. Gender coded as 1, female, 2, male. Employment coded as 1, employed, 0, unemployed. Drug history was removed from the COBRIT sample because it was exactly collinear with alcohol history.
Table 4.
Ordinal Logistic Regressions Predicting Neuropsychological Impairment Category in Each Study Sample
| Sample |
Track TBI (Nagelkerke R2 = 0.360) |
COBRIT (Nagelkerke R2 = 0.223) |
||||||
|---|---|---|---|---|---|---|---|---|
| Thresholds | Threshold | SE | Wald | p | Threshold | SE | Wald | p |
| 1:2 | -6.872 | 1.006 | 46.709 | < 0.001 | -5.557 | 0.6 | 85.685 | < 0.001 |
| ≤2:3 | -4.161 | 0.866 | 23.095 | < 0.001 | -3.229 | 0.536 | 36.271 | < 0.001 |
| ≤3:4 | -1.859 | 0.835 | 4.960 | 0.026 | -1.222 | 0.521 | 5.500 | 0.019 |
| ≤4:5 | 0.203 | 0.832 | 0.060 | 0.807 | 0.697 | 0.522 | 1.788 | 0.181 |
| ≤5:6 | 2.475 | 0.908 | 7.423 | < 0.001 | 2.908 | 0.576 | 25.479 | < 0.001 |
| Coefficients | Estimate | SE | Wald | p | Estimate | SE | Wald | p |
| Age | -0.034 | 0.009 | 15.966 | < 0.001 | -0.021 | 0.007 | 10.555 | 0.002 |
| Gender | -0.191 | 0.261 | 0.534 | 0.465 | 0.328 | 0.174 | 3.554 | 0.054 |
| Any psych. history | -0.933 | 0.274 | 11.62 | 0.001 | -0.589 | 0.26 | 5.156 | 0.016 |
| Alcohol history | -0.340 | 0.257 | 1.748 | 0.186 | -1.766 | 0.295 | 35.87 | < 0.001 |
| Drug history | -0.072 | 0.308 | 0.055 | 0.814 | 0.000 | . | . | . |
| Blood alcohol | 0.000 | 0.002 | 0.015 | 0.901 | 0.001 | 0.001 | 0.059 | 0.557 |
| Zero blood alc. | -0.123 | 0.404 | 0.092 | 0.761 | 0.354 | 0.231 | 2.357 | 0.144 |
| Systolic BP | -0.005 | 0.005 | 0.979 | 0.322 | -0.007 | 0.006 | 1.457 | 0.347 |
| Diastolic BP | 0.004 | 0.008 | 0.236 | 0.627 | -0.004 | 0.004 | 0.982 | 0.111 |
| Employment | 0.390 | 0.265 | 2.161 | 0.142 | 0.328 | 0.163 | 4.054 | 0.036 |
| No high school | -1.303 | 0.452 | 8.306 | 0.004 | -0.590 | 0.249 | 5.636 | 0.003 |
| College degree | 1.216 | 0.266 | 20.951 | < 0.001 | 0.906 | 0.165 | 30.164 | < 0.001 |
| Married | 0.448 | 0.29 | 2.39 | 0.122 | 0.212 | 0.215 | 0.972 | 0.358 |
| Divorced | -0.156 | 0.448 | 0.122 | 0.727 | -0.685 | 0.261 | 6.857 | 0.005 |
| Widowed | -0.777 | 0.796 | 0.953 | 0.329 | -0.245 | 0.599 | 0.167 | 0.700 |
| Previous TBI | -0.950 | 0.301 | 9.954 | 0.002 | -0.047 | 0.450 | 0.011 | 0.725 |
| IV Saline | -0.790 | 1.983 | 0.159 | 0.690 | 0.000 | 0.237 | 0.000 | 0.896 |
TBI, traumatic brain injury; SE, standard error; BP, blood pressure; IV, intravenous.
Note. Gender coded as 1, female, 2, male. Employment coded as 1, employed, 0, unemployed. Gaussian proportions were used to determine category cutoffs for neuropsychological impairment. Drug history was removed from the COBRIT sample because it was exactly collinear with alcohol history.
Table 5.
Ordinal Logistic Regressions Predicting Cognitive Speed and Flexibility Category in Each Study Sample
| Sample |
Track TBI (Nagelkerke R2 = 0.339) |
COBRIT (Nagelkerke R2 = 0.245) |
||||||
|---|---|---|---|---|---|---|---|---|
| Thresholds | Threshold | SE | Wald | p | Threshold | SE | Wald | p |
| 1:2 | -7.749 | 1.011 | 58.695 | <.001 | -6.162 | 0.615 | 100.235 | <.001 |
| ≤2:3 | -5.147 | 0.883 | 33.944 | <.001 | -3.744 | 0.543 | 47.609 | <.001 |
| ≤3:4 | -2.922 | 0.839 | 12.119 | <.001 | -1.689 | 0.524 | 10.397 | <.001 |
| ≤4:5 | -0.872 | 0.829 | 1.106 | 0.293 | 0.243 | 0.523 | 0.215 | 0.643 |
| ≤5:6 | 1.417 | 0.906 | 2.443 | 0.118 | 2.457 | 0.577 | 18.145 | <.001 |
| Coefficients | Estimate | S.E | Wald | p | Estimate | SE | Wald | p |
| Age | -0.047 | 0.009 | 28.996 | < 0.001 | -0.036 | 0.007 | 28.822 | < 0.001 |
| Gender | 0.206 | 0.261 | 0.623 | 0.430 | 0.125 | 0.174 | 0.52 | 0.471 |
| Any psych. history | 0.061 | 0.267 | 0.053 | 0.818 | 0.576 | 0.259 | 4.937 | 0.026 |
| Alcohol history | -0.061 | 0.256 | 0.056 | 0.813 | -2.197 | 0.303 | 52.596 | < 0.001 |
| Drug history | 0.362 | 0.307 | 1.394 | 0.238 | 0.000 | . | . | . |
| Blood alcohol | -0.002 | 0.002 | 1.017 | 0.313 | 0.001 | 0.001 | 0.008 | 0.930 |
| Zero blood alc. | -0.335 | 0.403 | 0.692 | 0.406 | 0.222 | 0.231 | 0.924 | 0.336 |
| Systolic BP | -0.013 | 0.005 | 6.04 | 0.014 | 0.003 | 0.006 | 0.221 | 0.638 |
| Diastolic BP | 0.008 | 0.008 | 1.003 | 0.317 | -0.008 | 0.004 | 3.404 | 0.065 |
| Employment | 0.460 | 0.264 | 3.031 | 0.082 | 0.229 | 0.163 | 1.967 | 0.161 |
| No high school | -1.773 | 0.452 | 15.366 | < 0.001 | -0.696 | 0.25 | 7.781 | 0.005 |
| College degree | 1.011 | 0.262 | 14.88 | < 0.001 | 0.861 | 0.165 | 27.275 | < 0.001 |
| Married | -0.051 | 0.288 | 0.032 | 0.858 | 0.251 | 0.216 | 1.355 | 0.244 |
| Divorced | -0.149 | 0.445 | 0.112 | 0.738 | -0.224 | 0.261 | 0.735 | 0.391 |
| Widowed | -0.697 | 0.794 | 0.77 | 0.380 | -0.493 | 0.601 | 0.672 | 0.412 |
| Previous TBI | -0.573 | 0.296 | 3.739 | 0.053 | -0.497 | 0.451 | 1.215 | 0.270 |
| IV Saline | -0.475 | 1.971 | 0.058 | 0.810 | -0.361 | 0.238 | 2.296 | 0.130 |
TBI, traumatic brain injury; SE, standard error; BP, blood pressure; IV, intravenous.
Note. Gender coded as 1, female, 2, male. Employment coded as 1, employed, 0, unemployed. Gaussian proportions were used to determine category cutoffs for cognitive speed and flexibility. Drug history was removed from the COBRIT sample because it was exactly collinear with alcohol history.
Altogether, approximations of variance explained were lower for the GOSE in both samples and higher for each multi-variate outcome. Across outcomes, the approximate variance explained (Nagelkerke R2) was larger in the TRACK-TBI Pilot sample compared with the COBRIT sample. This may be explained by potentially greater homogeneity in the smaller TRACK-TBI Pilot sample versus in the larger COBRIT sample, or by potential differences in the predictors across samples. For instance, drug and alcohol abuse history were assessed at 180 days post-injury in COBRIT, versus at baseline in TRACK-TBI Pilot, and covaried identically in COBRIT such that drug history was excluded from the models.
Across samples and outcomes, the education variables were the most consistent, significant predictors of participants' outcome scores. Odds ratios associated with completing a college degree compared with completing high school ranged from 2.7 to 4.2 across the outcomes in the TRACK-TBI Pilot sample and from 1.4 to 2.5 across outcomes in the COBRIT sample. Conversely, the odds ratios associated with not completing high school compared with completing high school ranged from 0.2 to 0.6 across outcomes in TRACK-TBI Pilot and from 0.5 to 0.6 in COBRIT. As such, across samples and approaches to measuring functioning post-TBI, education variables showed the greatest unique explanatory ability.
Additional predictors were commonly significant across samples within each multi-variate outcome domain. Age was a significant predictor of both neuropsychological impairment and cognitive speed and flexibility in both TRACK-TBI Pilot and COBRIT samples. For each year increase in age, participants were 3.3% less likely to score above each neuropsychological impairment category threshold in TRACK-TBI Pilot and 2.1% less likely to score above each neuropsychological impairment category threshold in COBRIT. Similarly, a one-year increase in age was associated with 4.6% lower odds of scoring above each cognitive speed and flexibility category threshold in Track TBI Pilot and a 3.4% lower odds of scoring above each cognitive speed and flexibility category threshold in COBRIT.
In addition, psychiatric history was statistically significantly associated with neuropsychological impairment in both samples. Participants endorsing any psychiatric history were 60.7% less likely to score above each neuropsychological impairment category threshold in Track-TBI Pilot and 44.5% less likely to score above each neuropsychological impairment category threshold in COBRIT. Surprisingly, college education was the only commonly significant predictor of participants' GOSE scores across samples.
Other predictors emerged uniquely for specific samples and outcome domains. In the COBRIT sample, having a history of alcohol abuse significantly predicted all outcome scores, and having been divorced compared with being single (i.e., never married) was significantly predictive of GOSE and neuropsychological impairment scores. In the TRACK-TBI Pilot sample, gender emerged as a unique significant predictor of GOSE scores, previous TBI was a uniquely significant predictor of neuropsychological impairment, and systolic blood pressure was a uniquely significant predictor of cognitive speed and flexibility.
Discussion
The EFA procedures applied to TBI outcome measures revealed the presence of more complex and data-driven outcome measures compared with the GOSE. In the two-dimensional space, the factor scores displayed a positive relationship between neuropsychological impairment and cognitive speed and flexibility. A third dimension was also identified that reflected differences in the satisfaction with life of the patients, which was independent of neuropsychological impairment and cognitive speed and flexibility.
The domains derived from our factor analysis provided a multi-dimensional representation of patients' functioning that was more nuanced than the global ratings provided by the GOSE. Unlike categorical schemes like the GOSE, these domains represented linear combinations of multiple outcome measures that may be more sensitive to individual variability. The lack of discernible clusters of factor scores in three-dimensional space when the scores are color coded by GOSE severity provides further evidence that the GOSE may fail to capture individual nuances in recovery. These results demonstrate important limitations to coining a single outcome measure to be the “gold standard.”
Multi-variate methods recently have become more popular in neurotrauma, and the new discoveries made in the field continue to support the use of these methods. Techniques for pattern detection, such as factor analysis, principal components analysis (PCA), and sparse hierarchical clustering (SHC) can produce data-driven results and have expanded in their estimation procedures and capacities for handling non-normal and non-scalar data, which means these methods are applicable to a breadth of problems.
In a previous study, PCA was used successfully to detect syndromic patterns related to the spinal cord injury (SCI) of rats, which showed that the multi-variate approach provided a more complete analysis of the SCI syndrome compared with considering outcome measures individually.26 In another study, SHC was used to detect similarities in the TBI subclasses, “mild,” “moderate,” and “severe,” between TRACK-TBI Pilot and COBRIT, which demonstrated the ability of the multi-variate approach to detect the heterogeneous nature of TBI, especially because this was a cross-study analysis of two different populations.5 It is unlikely that complex injuries like TBI and SCI can be explained by a single outcome measure; thus, as more advanced multi-variate methods are developed, it is important to take advantage of these approaches to better address research objectives.
The use of multi-variate outcomes such as those derived here could provide important information on a patient's outcome status. Multi-variate composite scores give a more nuanced perspective on a patient's post-TBI status than a singular measure such as the GOSE, and may be useful in characterizing changes in a patient's functioning at various end-points or stages of treatment. Simultaneously, the specific measurements used to create multi-variate outcomes can provide a detailed picture of a patient's functioning in distinct domains that help tailor individualized treatment plans. Consequently, just as individual items of the GOSE may be useful to interpret in conjunction with the total GOSE score, the components of any multi-variate outcome may be interpreted usefully while using the composite score as a clinical benchmark.
Predictive modeling methods using the factor scores may also be useful for patient prognosis and making recommendations for patient-specific treatment. In a previous study, clinical variables at baseline were predictive of mTBI at six months. After performing bootstrap validation, only 14% of the variance in the outcome was explained by the predictors in their linear regression model.6 They suggested that larger datasets, more granular variables, and objective biomarkers are needed to improve their model before making predictions in clinical practice.
Taking these points into consideration, our prediction models found that the ED predictors were able to explain greater variability in the multi-variate outcomes than the GOSE, suggesting a multi-variate perspective may be used to capture more effectively patient prognosis in a more granular way. When predicting GOSE severity categories, our predictive models performed similarly to previous studies; conversely, 70% to 150% greater variance was explained in the multi-variate outcomes (based on the Nagelkerke R2) using the same set of predictors. These findings illustrate the potential nuances in recovery that the GOSE alone may fail to capture, even when multi-variate measures are rescaled to match its ordinal metric.
In addition, our predictive models suggested important prognostic factors could be missed when the GOSE is used as the sole outcome indicator. Specifically, education was the only commonly significant prognostic predictor across the GOSE and multi-variate outcome domains in both study samples. Participant age and prior psychiatric history, however, also emerged as significant predictors of the multi-variate outcomes assessed in each sample. Consistent with these findings, psychiatric history previously has been shown to affect TBI recovery,27 and age previously has been implicated in long-term post-TBI cognitive functioning.28,29 Our results indicated insensitivity of the GOSE versus our multi-variate outcomes to this potential prognostic variability.
It is noteworthy that our predictive models explained greater variance in all outcomes in the Track-TBI Pilot sample versus the COBRIT sample. The Track-TBI Pilot included greater representation of patients classified as having mTBI, whereas COBRIT patients had more clinical severity as measured by commonly used TBI outcome metrics (e.g., the GOSE, a positive CT, or the Glasgow Coma Scale). Given this, our multi-variate outcomes may perform better as clinical end-points for patients with mTBI or limited evidence of more gross functional impairment.
It is possible that outcomes among patients with milder TBI are better assessed via psychiatric and cognitive processing measures, compared with questions about assistance and performance in daily activities as measured by the GOSE. Even in COBRIT, however, these outcomes were better explained by the ED clinical predictors than GOSE outcomes. Lacking a robust benchmark for TBI severity aside from gross neurological damage, it is therefore difficult to assess to what extent the prediction of our multi-variate outcomes was impacted by this factor. Our results would benefit from further validation using prospective predictive models wherein pre-morbid functioning and decline post-TBI could be evaluated as a benchmark for clinical severity. Such predictive analyses may also be used to determine the most appropriate set of predictors to use across varying levels of clinical severity so defined.
In future studies, multi-variate end-points should be designed to predict better which patients may decline after discharge and require closer monitoring for neuropsychiatric and cognitive decline. The direct focus of the modeling in the present study was to predict the outcome at the six-month time point post-TBI. The results, however, cannot be used to make conclusions about patient decline after hospital discharge, because the validity of the model depends on its predictions being interpreted as measurements of a patient's neuropsychiatric and cognitive function at a certain point in time. In a previous study of the COBRIT sample, latent class mixed models were used to identify dramatically divergent trajectories in good recovery, moderate disability, and severe disability groups from the one-month time point to the six-month time point.30 The results from similar studies will inspire clinical trial designs that capture the heterogeneity in recovery after TBI.
Although multi-variate methods have many benefits, it is important to understand fully the limitations of these approaches in generalizing across populations. Although we replicated the findings of our factor analysis in two distinct samples, we were only able to use approximately half of the observations to compute the loadings and scores because of missing values. There may be unmeasured mechanisms affecting participants with missing values that could limit the capacity to generalize the multi-variate outcome factor structure and scores to such individuals. Similarly, the variance in these outcomes explained by the ED predictors may differ for individuals with complete and incomplete data.
One hypothetical mechanism affecting missingness may be clinical severity, because persons with more severe impairment may have been unable to complete certain assessment tasks (e.g., tracing and task-shifting in the context of the Trail Making Test). This aligns with our finding that the variance explained in the multi-variate outcomes tended to be lower in the COBRIT sample.
The present study identified and replicated two multi-variate outcome domains across the TRACK-TBI Pilot and COBRIT post-TBI samples in an attempt to show improvement on the existing GOSE outcome classification system. Our results suggested a neuropsychological impairment dimension accounting for combined cognitive and psychiatric distress symptoms, and a cognitive speed and flexibility dimension accounting for diminished cognitive performance, may better capture variability in TBI outcomes. Further, our predictive analyses suggested these outcomes were more sensitive to potential prognostic predictors of post-TBI functioning than the GOSE. Taken together, our findings highlight the importance of investigating explanatory factors associated with interrelated outcome dimensions, such that can more effectively describe the interrelated biological, cognitive, and affective systems impacted by TBI.
Supplementary Material
Acknowledgments
Data and/or research tools used in the preparation of this manuscript were obtained and analyzed from the controlled access datasets distributed from the DOD- and NIH-supported Federal Interagency Traumatic Brain Injury Research (FITBIR) Informatics Systems. FITBIR is a collaborative biomedical informatics system created by the Department of Defense and the National Institutes of Health to provide a national resource to support and accelerate research in TBI. Dataset identifier(s) included the TRACK-TBI pilot (FITBIR-STUDY0000246, Clinical Trial ID Number: NCT01565551, Grant/Project ID Number(s): 1RC2NS069409-01, 3RC2NS069409-02S1, 5RC2NS069409-02, Principal Investigator: Geoffrey Manley, MD, PhD) and COBRIT (FITBIR-STUDY0000240, Clinical Trial ID Number: NCT00545662; Grant/Project ID Number(s): 1R03NS088021-01, 5M01RR000065-47 (6183), 5M01RR000065-46 (7161), 5M01RR000065-48 (6406), Principal Investigator: Ross Zafonte, DO). This manuscript reflects the views of the authors and may not reflect the opinions or views of the DOD, NIH, or of the Submitters submitting original data to FITBIR Informatics System. We'd also like to thank Adam R. Ferguson, PhD for fruitful discussions about statistical models.
Funding Information
This project was supported by NIMH Grant/Project R01MH116156-01A1, Principal Investigator: Jessica Nielson, PhD.
Author Disclosure Statement
No competing financial interests exist.
Supplementary Material
References
- 1. CDC Injury Center. (2019). TBI-related Emergency Department Visits, Hospitalizations, and Deaths (EDHDs). Available at: www.cdc.gov/traumaticbraininjury/data/tbi-edhd.html
- 2. Galgano, M., Toshkezi, G., Qiu, X., Russell, T., Chin, L., and Zhao, L.R. (2017). Traumatic brain injury: current treatment strategies and future endeavors. Cell Transplant. 26, 1118–1130 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Teasdale, G.M., Pettigrew, L.E., Wilson, J.T., Murray, G., and Jennett, B. (1998). Analyzing outcome of treatment of severe head injury: a review and update on advancing the use of the Glasgow Outcome Scale. J. Neurotrauma 15, 587–597 [DOI] [PubMed] [Google Scholar]
- 4. Weir, J., Steyerberg, E.W., Butcher, I., Lu, J., Lingsma, H.F., McHugh, G.S., Roozenbeek, B., Maas, A.I., and Murray, G.D. (2012). Does the extended Glasgow Outcome Scale add value to the conventional Glasgow Outcome Scale? J. Neurotrauma 29, 53–58 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Polinder, S., Cnossen, M.C., Real, R.G., Covic, A., Gorbunova, A., Voormolen, D.C., Master, C.L., Haagsma, J.A., Diaz-Arrastia, R., and von Steinbuechel, N. (2018). A multidimensional approach to post-concussion symptoms in mild traumatic brain injury. Front. Neurol. 9, 1113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Nelson, L.D., Ranson, J., Ferguson, A.R., Giacino, J., Okonkwo, D.O., Valadka, A., Manley, G., and McCrea, M. (2017). Validating multidimensional outcome assessment using the TBI common data elements: an analysis of the TRACK-TBI Pilot sample. J. Neurotrauma 34, 3158–3172 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Si, B., Dumkrieger, G., Wu, T., Zafonte, R., Dodick, D.W., Schwedt, T.J., and Li, J. (2018). A cross-study analysis for reproducible sub-classification of traumatic brain injury. Front. Neurol. 9, 606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Cnossen, M.C., Winkler, E.A., Yue, J.K., Okonkwo, D.O., Valadka, A., Steyerberg, E.W., Lingsma, H., and Manley, G.T. (2017). Development of a prediction model for post-concussive symptoms following mild traumatic brain injury: A TRACK-TBI pilot study. J. Neurotrauma 34, 2396–2409 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Jacobs, B., Beems, T., Stulemeijer, M., van Vugt, A.B., van der Vliet, T.M., Borm, G.F., and Vos, P.E. (2010). Outcome prediction in mild traumatic brain injury: age and clinical variables are stronger predictors than CT abnormalities. J. Neurotrauma 27, 655–668 [DOI] [PubMed] [Google Scholar]
- 10. Yue, J.K., Vassar, M.J., Lingsma, H.F., Cooper, S.R., Okonkwo, D.O., Valadka, A.B., Gordon, W.A., Maas, A.I., Mukherjee, P., Yuh, E.L., Puccio, A.M., Schnyer, D.M., Manley, G.T., and TRACK-TBI Investigators. (2013). Transforming research and clinical knowledge in traumatic brain injury pilot: multicenter implementation of the common data elements for traumatic brain injury. J. Neurotrauma 30, 1831–1844 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACI-TBI) Pilot. (2019). University of California San Francisco: BRICS System. Available at: www.fitbir.nih.gov/study_profile/246. PLEASE VERIFY [Google Scholar]
- 12. Zafonte, R.D., Bagiella, E., Ansel, B.M., Novack, T.A., Friedewald, W.T., Hesdorffer, D.C., Timmons, S.D., Jallo, J., Eisenberg, H., Hart, T., Ricker, J.H., Diaz-Arrastia, R., Merchant, R.E., Temkin, N.R., Melton, S., and Dikmen, S.S. (2012). Effect of citicoline on functional and cognitive status among patients with traumatic brain injury: Citicoline Brain Injury Treatment Trial (COBRIT). JAMA 308, 1993–2000 [DOI] [PubMed] [Google Scholar]
- 13. Adding Legacy Clinical Data to the Federal Interagency Traumatic Brain Injury: Citicoline Brain Injury Treatment Trial (COBRIT) (2014). Spaulding Rehabilitation Hospital and Harvard Medical School: BRICS System. Available at: www.fitbir.nih.gov/study_profile/240. PLEASE VERIFY
- 14. Giovagnoli, A.R., Del Pesce, M., Mascheroni, S., Simoncelli, M., Laiacona, M., and Capitani, E. (1996). Trail making test: normative values from 287 normal adult controls. Ital. J. Neurol. Sci. 17, 305–309 [DOI] [PubMed] [Google Scholar]
- 15. Weiss, L.G., Saklofske, D.H., Coalson, D.L., and Raiford, S.E. (2010). CHAPTER 3 - Theoretical, Empirical and Clinical Foundations of the WAIS-IV Index Scores., in: L.G. Weiss, D.H. Saklofske, D.L. Coalson, and S.E. Raiford, (eds). WAIS-IV Clinical Use and Interpretation. Academic Press, San Diego, pps. 61–94 [Google Scholar]
- 16. Cullum, C.M., and Larrabee, G.J. (2010). CHAPTER 6 - WAIS-IV Use in Neuropsychological Assessment, in: L.G. Weiss, D.H. Saklofske, D.L. Coalson, and S.E. Raiford (eds). WAIS-IV Clinical Use and Interpretation. Academic Press, pps. San Diego, 167–187 [Google Scholar]
- 17. Boulet, J., and Boss, M.W. (1991). Reliability and validity of the Brief Symptom Inventory. Psychological Assessment 3, 433–437 [Google Scholar]
- 18. Derogatis, L.R., and Savitz, K.L. (1999). The SCL-90-R, Brief Symptom Inventory, and Matching Clinical Rating Scales, in M. E. Maruish (ed.), The Use of Psychological Testing for Treatment Planning and Outcomes Assessment, 2nd ed. Mahwah, NJ: Lawrence Erlbaum Associates Publishers, pps. 679–724 [Google Scholar]
- 19. Diener, E., Emmons, R.A., Larsen, R.J., and Griffin, S. (1985). The Satisfaction With Life Scale. J. Pers. Assess. 49, 71–75 [DOI] [PubMed] [Google Scholar]
- 20. Pavot, W., and Diener, E. (2009). Review of the Satisfaction With Life Scale, in: E. Diener (ed). Assessing Well-Being: The Collected Works of Ed Diener. Dordrecht: Springer Netherlands, pps. 101–117 [Google Scholar]
- 21. Smith, L.I. (2002). A Tutorial on Principal Components Analysis. Dunedin, New Zealand: University of Otago Library. Available at: ourarchive.otago.ac.nz [Google Scholar]
- 22. Hu, L., and Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Modeling 6, 1–55 [Google Scholar]
- 23. Maydeu-Olivares, A., Shi, D., and Rosseel, Y. (2018). Assessing fit in structural equation models: a Monte-Carlo evaluation of RMSEA versus SRMR confidence intervals and tests of close fit. Struct. Equ. Modeling 25, 389–402 [Google Scholar]
- 24. Harrell, F.E. (2015). Ordinal logistic regression, in: F.E. Harrell, Jr. (ed). Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Springer International Publishing, Cham, pps. 311–325 [Google Scholar]
- 25. van Buuren S, Groothuis-Oudshoorn K (2011). mice: Multivariate imputation by chained equations in R. J. Stat. Software, 45, 1-67 [Google Scholar]
- 26. Ferguson, A.R., Irvine, K.A., Gensel, J.C., Nielson, J.L., Lin, A., Ly, J., Segal, M.R., Ratan, R.R., Bresnahan, J.C., and Beattie, M.S. (2013). Derivation of multivariate syndromic outcome metrics for consistent testing across multiple models of cervical spinal cord injury in rats. PLoS One 8, e59712. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Brandel, M.G., Hirshman, B.R., McCutcheon, B.A., Tringale, K., Carroll, K., Richtand, N.M., Perry, W., Chen, C.C., and Carter, B.S. (2017). The association between psychiatric comorbidities and outcomes for inpatients with traumatic brain injury. J. Neurotrauma 34, 1005–1016 [DOI] [PubMed] [Google Scholar]
- 28. Yue, J.K., Levin, H.S., Suen, C.G., Morrissey, M.R., Runyon, S.J., Winkler, E.A., Puffer, R.C., Deng, H., Robinson, C.K., Rick, J.W., Phelps, R.R., Sharma, S., Taylor, S.R., Vassar, M.J., Crossen, M.C., Lingsma, H.F., Gardner, R.C., Temkin, N.R., Barber, J., Dikmen, S.S., Yuh, E.L., Mukherjee, P., Stein, M.B., Cage, T.A., Valadka, A.B., Okonkwo, D.O., and Manley, G.T., and TRACK-TBI Investigators. (2019). Age and sex-mediated differences in six-month outcomes after mild traumatic brain injury in young adults: a TRACK-TBI study. Neurol. Res. 41, 609–623 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Li, W., Risacher, S.L., McAllister, T.W., Saykin, A.J., and Alzheimer's Disease Neuroimaging Initiative. (2017). Age at injury is associated with the long-term cognitive outcome of traumatic brain injuries. Alzheimers Dement. 6, 196–200 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Gardner, R.C., Cheng, J., Ferguson, A.R., Boylan, R., Boscardin, J., Zafonte, R.D., Manley, G.T., and Citicoline Brain Injury Treatment Trial Investigators. (2019). Divergent six month functional recovery trajectories and predictors after traumatic brain injury: novel insights from the Citicoline Brain Injury Treatment Trial Study. J. Neurotrauma 36, 2521–2532 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.