Abstract
The Neurological Outcome Scale for Traumatic Brain Injury (NOS-TBI) is a measure assessing neurological functioning in patients with TBI. We hypothesized that the NOS-TBI would exhibit adequate concurrent and predictive validity and demonstrate more sensitivity to change, compared with other well-established outcome measures. We analyzed data from the National Acute Brain Injury Study: Hypothermia-II clinical trial. Participants were 16–45 years of age with severe TBI assessed at 1, 3, 6, and 12 months postinjury. For analysis of criterion-related validity (concurrent and predictive), Spearman's rank-order correlations were calculated between the NOS-TBI and the Glasgow Outcome Scale (GOS), GOS-Extended (GOS-E), Disability Rating Scale (DRS), and Neurobehavioral Rating Scale-Revised (NRS-R). Concurrent validity was demonstrated through significant correlations between the NOS-TBI and GOS, GOS-E, DRS, and NRS-R measured contemporaneously at 3, 6, and 12 months postinjury (all p<0.0013). For prediction analyses, the multiplicity-adjusted p value using the false discovery rate was <0.015. The 1-month NOS-TBI score was a significant predictor of outcome in the GOS, GOS-E, and DRS at 3 and 6 months postinjury (all p<0.015). The 3-month NOS-TBI significantly predicted GOS, GOS-E, DRS, and NRS-R outcomes at 6 and 12 months postinjury (all p<0.0015). Sensitivity to change was analyzed using Wilcoxon's signed rank-sum test of subsamples demonstrating no change in the GOS or GOS-E between 3 and 6 months. The NOS-TBI demonstrated higher sensitivity to change, compared with the GOS (p<0.038) and GOS-E (p<0.016). In summary, the NOS-TBI demonstrated adequate concurrent and predictive validity as well as sensitivity to change, compared with gold-standard outcome measures. The NOS-TBI may enhance prediction of outcome in clinical practice and measurement of outcome in TBI research.
Key words: assessment tools, neuropsychology, outcome measures, recovery, traumatic brain injury
Introduction
Clinical trials in traumatic brain injury (TBI) frequently rely on a single global outcome measure to determine the efficacy of an interventional procedure or pharmacological agent. Historically, this has involved the use of the Glasgow Outcome Scale (GOS)1 or the GOS-Extended (GOS-E).2 Although an assessment of neurophysical functioning was intended to supplement GOS ratings, a standard reliable measure of this domain did not emerge. This gap in the literature led, in part, to the design of the Neurological Outcome Scale for Traumatic Brain Injury (NOS-TBI). Details of the development and psychometric properties of the NOS-TBI have been presented previously.3–5 Test items include assessment of motor and sensory functions, visual fields, facial paresis, and limb ataxia to assess gross neurological functioning in addition to items with specific applicability to patients with TBI, such as pupillary response, olfaction, and gait ataxia. The NOS-TBI was included as a recommended measure in the National Institute of Neurological Disorders and Stroke (NINDS) and Federal Interagency Common Data Elements (CDE) Initiative for TBI6 and is currently a supplemental measure in version 2.0 of these recommendations (http://www.commondataelements.ninds.nih.gov), which has merged recommended measures for adults6 and children7 with TBI. The NOS-TBI complements other measures in the CDE for TBI because it is one of very few measures that address neurological functioning per se and was specifically modified for patients with TBI. The NOS-TBI requires less time, training, and cost to administer and is potentially useful in a larger range of injury chronicity than the Funtional Independence Measure.
Since the measure's introduction, interest has increased regarding its use as a supplementary measure of outcome in clinical trials of TBI. Though the NOS-TBI has been validated in patients with TBI undergoing rehabilitation up to 18 months postinjury, this measure's performance in a clinical trial for TBI is unknown. To address this, the NOS-TBI was included as part of the assessment battery for patients enrolled in the National Acute Brain injury Study: Hypothermia-II (NABIS:H-II) trial. Additionally, criterion-related validity had not been explored in previous studies of the NOS-TBI, so concurrent and predictive validity were investigated along with the measure's sensitivity to change over time periods comparable to TBI clinical trials. The NOS-TBI was also compared with other commonly used early predictors of outcome to investigate its relative usefulness in gauging outcome.
Methods
Sample characteristics
The NABIS:H-II was a multi-center randomized trial of the effect of induction of acute moderate hypothermia in patients with severe brain injury who received either early cooling to 33°C maintained for 48 h or treatment at normothermia. Inclusion criteria included age 16–45 years, nonpenetrating brain injury, and failure to obey simple commands (i.e., Glasgow Coma Scale [GCS]8 score of 3–8). Exclusions included GCS score of 3 with nonreactive pupils, GCS score of 7–8 with normal brain computed tomography (CT) scan, inability to measure an accurate GCS score, Abbreviated Injury Severity score of 4 or greater for organs other than the brain, systolic blood pressure less than 110 mmHg, diastolic blood pressure less than 60 mmHg, persistent hypoxia (O2 saturation <94%), or a positive pregnancy test. Injury and demographic data are presented in Tables 1 and 2. More details regarding NABIS:H-II can be found in Clifton and colleagues.9 For the present study, an additional criterion of survival to at least the 3-month postinjury endpoint was included.
Table 1.
Categorical Variables: Demographics and Injury Characteristicsa
| Variables | n | % |
|---|---|---|
| Gender | ||
| Male | 79 | 84 |
| Female | 15 | 16 |
| Mechanism of injury | ||
| Assault | 4 | 4.7 |
| Auto-pedestrian | 7 | 8.1 |
| Fall | 13 | 15.1 |
| MVC | 55 | 54.7 |
| Motorcycle crash | 8 | 9.3 |
| Other | 7 | 8.1 |
| Race | ||
| European American | 80 | 85.1 |
| African American | 9 | 9.6 |
| Asian | 2 | 2.1 |
| Other | 3 | 3.2 |
| Ethnicity | ||
| Hispanic | 19 | 21.4 |
| Non-Hispanic | 74 | 77.5 |
| Unknown | 1 | 1.1 |
| Primary Language | ||
| English | 91 | 96.3 |
| Spanish | 3 | 3.7 |
n=94.
MVC, motor vehicle crash.
Table 2.
Continuous Variables: Demographics and Injury Characteristicsa
| Variable | Mean | SD | Median |
|---|---|---|---|
| Age at injury, years | 29.1 | 10.1 | 25.4 |
| Education, years | 12.2 | 2.4 | 12 |
| Trauma triage GCS score | 5.3 | 1.6 | 6 |
| Lesion volume (cm3)b | 38.8 | 32.1 | 31.5 |
| NOS-TBI (1 month, n=21) | 10.5 | 8.8 | 8 |
| NOS-TBI (3 months, n=73) | 6.8 | 11.2 | 3 |
| NOS-TBI (6 months, n=61) | 5.1 | 9.3 | 2 |
| NOS-TBI (12 months, n=54) | 5.7 | 11.9 | 2 |
n=94.
Lesion volume is the sum of the volume of epidural and subdural hematomas and intraparenchymal hemorrhages in cm3 as measured on the initial computed tomography scan.
SD, standard deviation; GCS, Glasgow Coma Scale; NOS-TBI, Neurological Outcome Scale for Traumatic Brain Injury.
Measures
Instruments included in the neuropsychological assessment battery of the NABIS:H-II clinical trial were selected based on the proceedings of the Houston Conference,10 in which measures were selected based on demonstrated acceptable psychometric properties of reliability and validity, brevity, and feasibility of administration in patients with sub- and postacute TBI. The GOS1,11 is a single-item rating scale summarizing patient outcome in five categories: good recovery; moderate disability; severe disability; vegetative state; or death. The GOS-E2 is an expanded version of the GOS that subdivides good recovery, moderate disability, and severe disability into upper and lower levels for a total of eight categories. The Disability Rating Scale (DRS) is a frequently used, well-validated rating scale of disability following TBI measuring a patient's level of function from coma to community reentry12–16 that has demonstrated predictive validity.14,17 The Neurobehavioral Rating Scale-Revised (NRS-R)18–20 is a revision of the frequently used NRS21 that rates cognitive, affective, and behavioral dysfunction on a 29-item scale.
Other measures employed in this study include the Marshall CT classification scale,22,23 as modified by Bigler and colleagues,24 which categorizes CT scans into seven groups: four levels of diffuse injury (I–IV); evacuated mass lesion (V); nonevacuated mass lesion (VI); and an added level specifically to include brainstem lesions (VII). Lesion volume was also included in this study as a separate variable, which was defined as the sum of the volumes of epi- and subdural hematomas, and intraparenchymal hemorrhages in cm3, as determined from initial or preoperative CT scans as measured by a neuroradiologist. Finally, the GCS score at trauma triage was used.
Procedure
Patients were enrolled into NABIS:H-II using exception from informed consent (EFIC), unless a legally authorized representative was immediately able to provide consent. The trial's protocol and use of EFIC were approved by the institutional review boards of each participating center. Participants were given the choice to decline or continue participation in the trial (i.e., reconsenting under their own authority) once they emerged from post-traumatic amnesia and were determined to be cognitively able to give informed consent. As part of the design of the clinical trial, most participants were assessed at 1, 3, 6, and 12 months postinjury. Because of the multiple patient groups included in the conduct of this study (detailed above), not all participants received all measures at each time point.
Statistical analysis
All analyses were conducted using SAS software for Windows (Version 9.3; SAS Institute Inc., Cary, NC). Statistical significance was defined as p value≤0.05 for all analyses, unless otherwise specified. Distributions of the dependent variables were analyzed for violations of normality; Spearman's rank-order correlations were performed as review of the NOS-TBI data revealed non-normal distributions. Data were screened for entry errors, and necessary corrections were made. No participant had missing data for any item on the NOS-TBI.
Results
Concurrent validity
Concurrent validity is a form of criterion-related validity in which an instrument correlates substantially with other validated measures of a given theoretically related construct.25 Concurrent validity was determined through Spearman's rank-order correlation of the NOS-TBI total score with the GOS, GOS-E, DRS, and NRS-R (Table 3). The NOS-TBI demonstrated good-to-excellent concurrent validity with these outcome measures at 3, 6, and 12 months postinjury (correlations ranged from 0.44 to 0.70; all p≤0.0013).
Table 3.
Concurrent Validity Between NOS-TBI and Standard Outcome Measures Assessed Contemporaneously
| |
NOS-TBI study endpoints |
||
|---|---|---|---|
| Outcome measures | 3 months | 6 months | 12 months |
| GOS | r=0.53, p<0.0001 (n=73) | r=0.62, p<0.0001 (n=61) | r=0.52, p<0.0001 (n=54) |
| GOS-E | r=0.59, p<0.0001 (n=73) | r=0.70, p<0.0001 (n=61) | r=0.62, p<0.0001 (n=54) |
| DRS | r=0.67, p<0.0001 (n=73) | r=0.70, p<0.0001 (n=61) | r=0.57, p<0.0001 (n=54) |
| NRS-R | r=0.51, p<0.0001 (n=57) | r=0.52, p<0.0001 (n=55) | r=0.44, p=0.0013 (n=50) |
NOS-TBI, Neurological Outcome Scale for Traumatic Brain Injury; GOS, Glasgow Outcome Scale; GOS-E, GOS-Extended; DRS, Disability Rating Scale; NRS-R, Neurobehavioral Rating Scale-Revised.
Predictive validity
Another type of criterion-related validity, predictive validity, was determined through Spearman's rank-order correlations of the NOS-TBI total score at both 1 and 3 months postinjury with the GOS, GOS-E, DRS, and NRS-R assessed at 3, 6, and 12 months postinjury (Table 4). Given the number of correlations produced, the false discovery rate (FDR)26 was used to control the type I error rate for multiple comparisons; the multiplicity-adjusted p value, therefore, was determined as ≤0.015. The NOS-TBI at 1 month postinjury was a significant predictor of outcome as measured by the GOS, GOS-E, and DRS assessed at 3 and 6 months postinjury (all p≤0.015). The NOS-TBI at 3 months was a very robust predictor of outcome, as measured by the GOS, GOS-E, DRS, and NRS-R, assessed at 6 and 12 months postinjury (all p≤0.0015). This is not surprising because recovery typically slows or stabilizes by this point, and patients less frequently advance from one GOS or GOS-E classification to another.
Table 4.
Outcome Prediction of Participants Surviving ≥3 Months
| |
Outcome measures and study endpoints |
||
|---|---|---|---|
| Predictor | 3 Months | 6 Months | 12 Months |
| GCS | NOS-TBI | ||
| r=−0.28, p=0.024 (n=66) | r=−0.15, NS (n=61) | r=−0.23, p=0.09 (n=54) | |
| GOS | |||
| r=−0.14, NS (n=78) | r=−0.20, NS (n=77) | r=−0.20, NS (n=62) | |
| GOS-E | |||
| r=−0.20, p=0.077 (n=78) | r=−0.20, p=.078 (n=77) | r=−0.16, NS (n=62) | |
| DRS | |||
| r=−0.26, p=0.023 (n=75) | r=−0.22, p=.058 (n=74) | r=−0.17, NS (n=61) | |
| NRS-R | |||
| r=−0.08, NS (n=58) | r=−0.14, NS (n=57) | r=−0.14, NS (n=50) | |
| Marshall CT classificationa | NOS-TBI | ||
| r=0.13, NS (n=65) | r=0.28, p=0.028 (n=60) | r=0.20, NS (n=53) | |
| GOS | |||
| r=0.27, p=0.0199 (n=77) | r=0.20, NS (n=76) | r=0.37, p=0.0036 (n=61) | |
| GOS-E | |||
| r=0.31, p=0.00063 (n=77) | r=0.23, NS (n=76) | r=0.33, p=0.01 (n=61) | |
| DRS | |||
| r=0.37, p=0.0011 (n=74) | r=0.25, p=0.033 (n=73) | r=0.37, p=0.0037 (n=60) | |
| NRS-R | |||
| r=0.22, NS (n=57) | r=0.39, p=0.0028 (n=56) | r=0.34, p=0.0178 (n=49) | |
| Lesion volumeb | NOS-TBI | ||
| r=0.03, NS (n=30) | r=0.18, NS (n=23) | r=0.00, NS (n=20) | |
| GOS | |||
| r=−0.04, NS (n=35) | r=0.04, NS (n=35) | r=0.10, NS (n=23) | |
| GOS-E | |||
| r=0.08, NS (n=35) | r=0.15, NS (n=35) | r=0.19, NS (n=23) | |
| DRS | |||
| r=0.24, NS (n=34) | r=0.28, NS (n=33) | r=0.05, NS (n=23) | |
| NRS-R | |||
| r=0.54, p=0.0043 (n=26) | r=0.37, NS (n=24) | r=0.28, NS (n=18) | |
| NOS-TBI at 1 month | GOS | ||
| r=0.66, p=0.001 (n=21) | r=0.54, p=0.014 (n=20) | r=0.39, NS (n=12) | |
| GOS-E | |||
| r=0.60, p=0.0011 (n=21) | r=0.53, p=0.014 (n=20) | r=0.54, p=0.07 (n=12) | |
| DRS | |||
| r=0.56, p=0.0089 (n=21) | r=0.54, p=0.015 (n=20) | r=0.34, NS (n=12) | |
| NRS-R | |||
| r=0.12, NS (n=12) | r=0.18, NS (n=10) | r=0.38, NS (n=11) | |
| NOS-TBI at 3 months | GOS | ||
| — | r=0.59, p<0.0001 (n=69) | r=0.58, p<0.0001 (n=55) | |
| GOS-E | |||
| — | r=0.66, p<0.0001 (n=69) | r=0.54, p<0.0001 (n=55) | |
| DRS | |||
| — | r=0.66, p<0.0001 (n=69) | r=0.63, p<0.0001 (n=55) | |
| — | NRS-R | ||
| r=0.43, p=0.0015 (n=52) | r=0.51, p=0.0003 (n=46) | ||
For all correlations in this table, exact p values are presented and p values≥0.10 are reported as not significant (NS). Using the false discovery rate procedure described by Benjamni and Hochberg26 for the families of correlations in this table, a p value≤0.015 is considered significant. GCS refers to the Glasgow Coma Scale score at time of trauma triage.
This is the Bigler version of the Marshall CT classification scheme.
Lesion volume is the sum of the volume of epidural and subdural hematomas, and intraparenchymal hemorrhages in cm3 as measured on the initial CT scan.
NOS-TBI, Neurological Outcome Scale for Traumatic Brain Injury; GOS, Glasgow Outcome Scale; GOS-E, GOS-Extended; DRS, Disability Rating Scale; NRS-R, Neurobehavioral Rating Scale-Revised; CT, computed tomography.
These results contrast with the weaker predictive ability of the acute GCS score (measured at trauma triage), which failed to correlate significantly (p≥0.015) with any outcome measure assessed at 3, 6, or 12 months postinjury. Using the FDR, the Bigler-modified Marshall CT classification significantly predicted DRS scores at 3 and 12 months postinjury, but was only predictive of the GOS at 12 months postinjury and the GOS-E at 3 and 12 months postinjury. The modified Marshall CT classification was also a significant predictor of NRS-R performance at 6 months. Lesion volume prediction (in cm3) was significant for NRS-R performance at 3 months postinjury (p=0.0043), but was nonsignificant for all other measures at all other endpoints.
Overall, the acute GCS and initial CT-derived lesion volumes were found to be weak predictors of outcome assessed at 3, 6, and 12 months postinjury. In contrast, the Bigler-modified Marshall CT classification scores and the 1-month NOS-TBI were found to be much stronger predictors of outcome at these same endpoints.
Outcome prediction
Analyses were conducted to demonstrate the relative strength of commonly used acute injury variables (e.g., GCS, lesion volume, and so on) to predict outcome at standard time points postinjury, in comparison to the NOS-TBI. The 1-month postinjury endpoint was selected to administer the NOS-TBI because there is no variability in these scores until patients emerge from coma. In our sample of patients, 82.5% had emerged from coma by this time. In this way, the predictive ability of the acute injury measures and the NOS-TBI are more equitably matched.
Sensitivity to change
As expected, the NOS-TBI scores at each assessment endpoint demonstrated a non-normal distribution. Using a procedure similar to that of McCauley and colleagues,20 Wilcoxon's signed-rank test (the nonparametric equivalent of paired t-tests) was used to determine differences in the NOS-TBI score for participants whose GOS score did not change from 3 to 6 months. Results indicated that the NOS-TBI score (n=38; 3 months, mean [M]=6.39±9.21; 6 months, M=4.97±8.70) changed significantly (S=−94; p=0.038). For those whose GOS score did not change from 6 to 12 months postinjury (n=31; NOS-TBI 6 months, M=3.23±3.5; NOS-TBI 12 months, M=3.81±5.96), the difference was not significant (S=5.5; p=0.85). For those patients whose GOS-E did not change between 3 and 6 months postinjury, the NOS-TBI score (n=28; M=7.93±10.28; 6 months, M=5.89±9.96) changed significantly (S=−81.5; p=0.016). For those whose GOS-E score did not change from 6 to 12 months (n=23; NOS-TBI 6 months, M=4.0±3.67; NOS-TBI 12 months, M=7.74±15.09), the difference was not significant (S=10.5; p=0.60). These results suggest that although the GOS and GOS-E did not reflect changes in the participant's level of recovery from 3 to 6 months, the NOS-TBI was sensitive to changes in neurological functioning over this period.
Discussion
In this study, specific indices of criterion-related validity and sensitivity to change of the NOS-TBI were evaluated in a sample of participants with severe TBI (sTBI) enrolled in the NABIS:H-II clinical trial. Validation in such a sample is critical to determine the measure's psychometric properties when it is intended to be used as a secondary outcome measure supplementing standard outcome indices, such as the GOS and GOS-E. Concurrent validity was robustly demonstrated through significant Spearman's correlations between the NOS-TBI and the GOS, GOS-E, DRS, and NRS-R measured simultaneously at 3, 6, and 12 months postinjury (all p≤0.0013). Further, correlations with each outcome measure remained fairly similar at each endpoint, indicating a degree of stability in the relation across the course of recovery. Although assessing different aspects of functioning than the GOS, DRS, and NRS-R, the correspondence between the NOS-TBI and the standard global measures of outcome subsequent to TBI is strong. This suggests that, as asserted by Jennett and Bond, neurophysical functioning is an important part of global outcome assessment and provides nonredundant, complimentary information to gold-standard measures. Although preliminary, this extends previous findings3–5 supporting the ability of the NOS-TBI to fulfill this role as a standard measure of neurophysical functioning.
Predictive validity was demonstrated between the 1-month NOS-TBI and the GOS, GOS-E, and DRS at 3 and 6 months postinjury. Although prediction of 12-month outcome did not achieve statistical significance, this may have been attributable, at least in part, to the attenuation of the sample size at that endpoint. Of note is that prediction of the 12-month GOS-E demonstrated a trend, and the magnitude of the correlation was similar to that of correlations with the same measure at 3 and 6 months postinjury; this suggests that with a larger sample, significant prediction of 12-month outcome also may be possible. Similarly, the 1-month NOS-TBI did not predict performance on the NRS-R at any endpoint. Although the NRS-R primarily assesses cognitive, emotional, and behavioral functioning—not neurological functioning per se—the lack of correlation with the NOS-TBI was likely a result of the very small sample of participants who had these measures at the required endpoints. Prediction of outcome at 6 and 12 months was substantial using the 3-month NOS-TBI, further substantiating the view that sample size, not content, may have largely explained the lack of relation between the 1-month NOS-TBI and the sub- and postacute NRS-R scores.
Overall, the NOS-TBI has shown adequate concurrent and predictive validity in a clinical trial for the treatment of sTBI. Further, greater sensitivity to change during postacute recovery was found, when compared with the GOS and GOS-E. These results support and extend previous studies indicating that the NOS-TBI is a valid and reliable tool to assess neurological functioning in patients with a range of TBI severity from acute to postacute endpoints.
This study suffers from several limitations. The sample size was restricted for some correlations, thereby making comparisons across time difficult. This was the result of changes in the trial such that the NOS-TBI was added to the assessment procedures partway through. The NOS-TBI is currently being used as a secondary outcome measure in a large treatment trial for ?moderate and severe TBI, which will include a larger sample of patients to provide an opportunity to replicate the current preliminary findings.
Few, if any, outcome measures commonly applied in TBI are appropriate for use across the complete spectrum of injury severity and acuity.6 The NOS-TBI was designed to address this gap. Additionally, instruments that address functional outcome in comatose or near-comatose patients are sparse, and the NOS-TBI scoring convention accommodates assessment of these patients who form a substantial percentage of clinical trial participants in the acute postinjury period. We acknowledge that the measurement of recovery differs at the lower end of the functional spectrum (i.e., floor effects) and in the acute phase as a patient regains consciousness. The NOS-TBI would be most useful if used to evaluate patients once they have emerged from coma. Otherwise, emergence from coma would produce nonlinear decreases in scores as patients are automatically assigned the greatest level of impairment while still in coma. We also acknowledge that this measure has undetermined sensitivity to change in patients at the mild end of the TBI spectrum, where few, if any, neurological deficits will be appreciated. Additional investigations of the use of this measure in other TBI patient populations and assessment intervals are warranted and are underway. Despite the limitations of this study, the NOS-TBI represents a promising measure for the assessment of neurological consequences of TBI.
Acknowledgments
The authors extend their gratitude to the participants and their families whose cooperation and patience helped make this study possible. This study was supported, in part, by grant NS 43353 from the National Institutes of Health (NIH)/NINDS (Guy L. Clifton, principal investigator). The information in this article and the article itself has never been published either electronically or in print. None of the authors has any financial or other relationship(s) that could be construed as a conflict of interest with respect to the content of this manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NINDS or the NIH.
Author Disclosure Statement
No competing financial interests exist.
References
- 1.Jennett B. Bond M. Assessment of outcome after severe brain damage. Lancet. 1975;1:480–484. doi: 10.1016/s0140-6736(75)92830-5. [DOI] [PubMed] [Google Scholar]
- 2.Wilson J.T. Pettigrew L.E. Teasdale G.M. Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use. J. Neurotrauma. 1998;15:573–585. doi: 10.1089/neu.1998.15.573. [DOI] [PubMed] [Google Scholar]
- 3.McCauley S.R. Wilde E.A. Kelly T.M. Weyand A.M. Yallampalli R. Waldron E.J. Pedroza C. Schnelle K.P. Boake C. Levin H.S. Moretti P. The Neurological Outcome Scale for Traumatic Brain Injury (NOS-TBI): II. Reliability and convergent validity. J. Neurotrauma. 2010;27:991–997. doi: 10.1089/neu.2009.1195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wilde E.A. McCauley S.R. Kelly T.M. Weyand A.M. Pedroza C. Levin H.S. Clifton G.L. Schnelle K.P. Shah M.V. Moretti P. The Neurological Outcome Scale for Traumatic Brain Injury (NOS-TBI): I. Construct validity. J. Neurotrauma. 2010;27:983–989. doi: 10.1089/neu.2009.1194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wilde E.A. McCauley S.R. Kelly T.M. Levin H.S. Pedroza C. Clifton G.L. Robertson C.S. Valadka A.B. Moretti P. Feasibility of the Neurological Outcome Scale for Traumatic Brain Injury (NOS-TBI) in adults. J. Neurotrauma. 2010;27:975–981. doi: 10.1089/neu.2009.1193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Wilde E.A. Whiteneck G.G. Bogner J. Bushnik T. Cifu D.X. Dikmen S. French L. Giacino J.T. Hart T. Malec J.F. Millis S.R. Novack T.A. Sherer M. Tulsky D.S. Vanderploeg R.D. von Steinbuechel N. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch. Phys. Med. Rehabil. 2010;91:1650–1660. doi: 10.1016/j.apmr.2010.06.033. [DOI] [PubMed] [Google Scholar]
- 7.McCauley S.R. Wilde E.A. Anderson V.A. Bedell G. Beers S.R. Campbell T.F. Chapman S.B. Ewing-Cobbs L. Gerring J.P. Gioia G.A. Levin H.S. Michaud L.J. Prasad M.R. Swaine B.R. Turkstra L.S. Wade S.L. Yeates K.O. Pediatric TBI Outcomes Workgroup. Recommendations for the use of common outcome measures in pediatric traumatic brain injury research. J. Neurotrauma. 2012;29:678–705. doi: 10.1089/neu.2011.1838. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Teasdale G. Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet. 1974;2:81–84. doi: 10.1016/s0140-6736(74)91639-0. [DOI] [PubMed] [Google Scholar]
- 9.Clifton G.L. Valadka A. Zygun D. Coffey C.S. Drever P. Fourwinds S. Janis L.S. Wilde E. Taylor P. Harshman K. Conley A. Puccio A. Levin H.S. McCauley S.R. Bucholz R.D. Smith K.R. Schmidt J.H. Scott J.N. Yonas H. Okonkwo D.O. Very early hypothermia induction in patients with severe brain injury (the National Acute Brain Injury Study: Hypothermia II): a randomised trial. Lancet Neurol. 2011;10:131–139. doi: 10.1016/S1474-4422(10)70300-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Clifton G.L. Hayes R.L. Levin H.S. Michel M.E. Choi S.C. Outcome measures for clinical trials involving traumatically brain-injured patients: report of a conference. Neurosurgery. 1992;31:975–978. doi: 10.1227/00006123-199211000-00028. [DOI] [PubMed] [Google Scholar]
- 11.Jennett B. Snoek J. Bond M.R. Brooks N. Disability after severe head injury: observations on the use of the Glasgow Outcome Scale. J. Neurol Neurosurg. Psychiatry. 1981;44:285–293. doi: 10.1136/jnnp.44.4.285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rappaport M. Hall K.M. Hopkins K. Belleza T. Cope D.N. Disability rating scale for severe head trauma: coma to community. Arch. Phys. Med. Rehabil. 1982;63:118–123. [PubMed] [Google Scholar]
- 13.Hall K. Cope D.N. Rappaport M. Glasgow Outcome Scale and Disability Rating Scale: comparative usefulness in following recovery in traumatic head injury. Arch. Phys. Med. Rehabil. 1985;66:35–37. [PubMed] [Google Scholar]
- 14.Fleming J.M. Maas F. Prognosis of rehabilitation outcome in head injury using the Disability Rating Scale. Arch. Phys. Med. Rehabil. 1994;75:156–163. [PubMed] [Google Scholar]
- 15.Gouvier W.D. Blanton P.D. LaPorte K.K. Nepomuceno C. Reliability and validity of the Disability Rating Scale and the Levels of Cognitive Functioning Scale in monitoring recovery from severe head injury. Arch. Phys. Med. Rehabil. 1987;68:94–97. [PubMed] [Google Scholar]
- 16.Hall K.M. Bushnik T. Lakisic-Kazazic B. Wright J. Cantagallo A. Assessing traumatic brain injury outcome measures for long-term follow-up of community-based individuals. Arch. Phys. Med. Rehabil. 2001;82:367–374. doi: 10.1053/apmr.2001.21525. [DOI] [PubMed] [Google Scholar]
- 17.McCauley S.R. Hannay H.J. Swank P.R. Use of the Disability Rating Scale recovery curve as a predictor of psychosocial outcome following closed-head injury. J. Int. Neuropsychol. Soc. 2001;7:457–467. doi: 10.1017/s1355617701744025. [DOI] [PubMed] [Google Scholar]
- 18.Vanier M. Mazaux J.M. Lambert J. Dassa C. Levin H.S. Assessment of neuropsychologic impairments after head injury: interrater reliability and factorial and criterion validity of the Neurobehavioral Rating Scale-Revised. Arch. Phys. Med. Rehabil. 2000;81:796–806. doi: 10.1016/s0003-9993(00)90114-x. [DOI] [PubMed] [Google Scholar]
- 19.Soury S. Mazaux J.M. Lambert J. De Seze M. Joseph P.A. Lozes-Boudillon S. McCauley S. Vanier M. Levin H.S. The neurobehavioral rating scale-revised: assessment of concurrent validity. Ann. Readapt. Med. Phys. 2005;48:61–70. doi: 10.1016/j.annrmp.2004.08.007. [DOI] [PubMed] [Google Scholar]
- 20.McCauley S.R. Levin H.S. Vanier M. Mazaux J.M. Boake C. Goldfader P.R. Rockers D. Butters M. Kareken D.A. Lambert J. Clifton G.L. The neurobehavioural rating scale-revised: sensitivity and validity in closed head injury assessment. J. Neurol. Neurosurg. Psychiatry. 2001;71:643–651. doi: 10.1136/jnnp.71.5.643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Levin H.S. High W.M. Goethe K.E. Sisson R.A. Overall J.E. Rhoades H.M. Eisenberg H.M. Kalisky Z. Gary H.E. The neurobehavioural rating scale: assessment of the behavioural sequelae of head injury by the clinician. J. Neurol. Neurosurg. Psychiatry. 1987;50:183–193. doi: 10.1136/jnnp.50.2.183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Marshall L.F. Marshall S.B. Klauber M.R. Van Berkum. Clark M. Eisenberg H. Jane J.A. Luerssen T.G. Marmarou A. Foulkes M.A. The diagnosis of head injury requires a classification based on computed axial tomography. J. Neurotrauma. 1992;9(Suppl. 1):S287–S292. [PubMed] [Google Scholar]
- 23.Marshall L.F. Marshall S.B. Klauber M.R. van Berkum Clark M. Eisenberg H.M. Jane J.A. Luerssen T.G. Marmarou A. Foulkes M.A. A new classification of head injury based on computerized tomography. J. Neurosurg. 1991;75:S14. [Google Scholar]
- 24.Bigler E.D. Ryser D.K. Gandhi P. Kimball J. Wilde E.A. Day-of-injury computerized tomography, rehabilitation status, and development of cerebral atrophy in persons with traumatic brain injury. Am. J. Phys. Med. Rehabil. 2006;85:793–806. doi: 10.1097/01.phm.0000237873.26250.e1. [DOI] [PubMed] [Google Scholar]
- 25.Anastasi A. Urbina S. Psychological Testing. 7th. Prentice Hall, Inc.; Upper Saddle River, NJ: 1997. [Google Scholar]
- 26.Benjamini Y. Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Statist. Soc. B. 1995;57:289–300. [Google Scholar]