Response to Foks et al. (doi: 10.1089/neu.2018.5979): Why Our Long-Term Functional Prognosis Tools are a Valuable Contribution to the Traumatic Brain Injury Outcome Literature

Dear Editor:

We thank Dr. Foks and colleagues¹ for their careful read of the methodology and statistics used in our manuscript and submitting their remarks. However, we believe that their criticism does not take into account the specific question that we sought to address. This may be due to a lack of intimacy with the specific patient population under study and/or a lack of familiarity with the prognostic questions faced by rehabilitation clinicians in caring for these individuals.

The rationale for our study was the need for clinically useful and relevant long-term prognostic models for persons with severe traumatic brain injury (TBI) who survive urgent care, emerge from coma, and progress enough to enter an intensive rehabilitation program. The Corticosteroid Randomisation After Significant Head Injury (CRASH) and International Mission on Prognosis and Analysis of Clinical Trials (IMPACT) models advocated by Dr. Foks and colleagues focus on injury variables at initial trauma presentation.^2,3 While these models address a broader range of TBI patient population, they are best at identifying who may or may not survive their acute course and may or may not emerge from coma. They assume, but do not provide any evidence, that such early prediction models using initial neurologic status will translate into accurate long-term functional prediction for the group that survives and advances to inpatient rehabilitation. By their very design, the CRASH and IMPACT models cannot consider downstream predictors such as post-traumatic amnesia (PTA) duration, shown repeatedly as the best injury severity predictor at the rehabilitation stage.^4–6 Another crucial limitation of the CRASH and IMPACT models is that they only predict out to 6 months post-injury, which is well under the timeframe for maximal functional recovery after severe TBI and does not meet the needs of loved ones and other stakeholders desiring longer term prognosis.^7,8

One criticism of Dr. Foks and colleagues was our exclusion of death and vegetative outcomes, which occur at very low frequency in our cohort. However, the prediction of future catastrophic events leading to death or relapse into vegetative state was not the focus of this study. We encourage Dr. Foks and colleagues, as well as interested readers, to review prior studies conducted in the Traumatic Brain Injury Model Systems cohort that address this research question.^9–12 In our vast experience caring for patients in this population, the prediction of such future events is rarely posed by their loved ones or other stakeholders, such as disability payers. Having just survived one catastrophic event and seeing encouraging progress, they want to know what functionality their loved one will eventually return to and in what timeframe. Regardless, we have provided the information on death and vegetative outcomes in the supplementary online tables.⁸ Further, among the few patients with dead or vegetative outcomes, most (76%, 65%, and 58% at Years 1, 2, and 5, respectively) have high PTA durations and would end up in the terminal node with the poorest outcomes such that their inclusion would not add significant information to the model.

Dr. Foks and colleagues also criticize our general statistical approach—decision tree modeling—as not being of adequate methodological quality. However, as stated in the manuscript, our goal was not simply to perform a mathematical exercise; instead, we were willing to accept marginal reduction in prediction accuracy to achieve a set of models that clinicians would actually use. Logistic regression models, which they endorse, are less transparent, as users cannot visualize the hierarchy and permutations of the model. Additionally, they are less usable in the face of missing lower-order variables. Further, regression models lend themselves to overfitting when many variables are used, leading to small model bias, but high variance and low generalizability. Classification trees allow us to present the results in a way that is easy to interpret while concurrently offering us a framework to reduce the set of variables to the most salient predictors of outcomes.¹³

We agree that our restriction to only include patients that did not have missing Glasgow Outcome Scale outcomes at each of the 1-, 2-, and 5-year follow-up periods could have biased the results and commented on this in the original article discussion section.⁷ However, we did use nodewise imputation within our classification tree algorithm as a strategy to include patients with missing predictors, as stated in the statistical methods portion of the manuscript.

Regarding our method of model validation, we agree that validation in a truly external dataset is ideal. Our method of splitting our internal data into separate building and validation test datasets was simply the first stage. We had planned to test the accuracy in an external dataset and this work is currently in progress.

In summary, our approach was to apply statistics and develop highly usable prognosticate tools for TBI patients who have survived their acute course and are able to interact with their environment. We are confident that stakeholders including patients, their families, rehabilitation professionals, and payers will find this work addresses their needs more effectively than previously published models.