Dichotomous “Good Outcome” Indicates Mobility More Than Cognitive or Social Quality of Life

Abstract

Objective

Worthwhile interventions for intracerebral hemorrhage (ICH) or subarachnoid hemorrhage (SAH) generally hinge on whether they improve the odds of “good outcome.” While good outcome is correlated with mobility, correlations with other domains of health-related quality of life (HRQoL), such as cognitive function (CF) and social functioning, are not well described. We tested the hypothesis that good outcome is more closely associated with mobility than other domains.

Design

We defined “good outcome” as 0 through 3 (independent ambulation or better) vs. 4 through 5 (dependent) on the modified Rankin Scale (mRS) at one, three and 12 months. We simultaneously assessed the mRS and HRQoL using web-based computer adaptive testing in the domains of mobility, CF (executive function and general concerns), and satisfaction with social roles and activities (SRA). We compared the area under the curve (AUC) between different HRQoL domains.

Setting

Neurological intensive care unit with web-based follow-up

Measurement and Main Results

We longitudinally followed 114 survivors with data at one month, 62 patients at three months, and 58 patients at 12 months. At one month, AUC was highest for mobility (0.957, 95% CI 0.904 – 0.98), higher than CF - general concerns (0.819, 95%CI 0.715-0.888, P=0.003 compared to mobility), satisfaction with SRA (0.85, 95%CI 0.753-0.911, P=0.01 compared to mobility) and CF - executive function (0.879, 95%CI 0.782-0.935, P=0.058 compared to mobility). Optimal specificity and sensitivity for ROC analysis were approximately 1.5 SD below the US population mean.

Conclusions

HRQoL assessments reliably distinguished between good and poor outcome as determined by the mRS. “Good outcome” indicated HRQoL about 1.5 SD below the US population mean. Associations were weaker for CF and social function than mobility.

Introduction

The pivotal outcome in most clinical research is a dichotomized assessment of “good” vs. “poor” outcome. For patients with intracerebral hemorrhage (ICH) and subarachnoid hemorrhage (SAH), good outcome typically means a modified Rankin Scale (mRS) of 0 through 3, able to independently ambulate or better, a de facto measure of mobility, inclusive of all levels of cognitive and social function for ambulatory patients. Mobility is an important domain of health-related quality of life (HRQoL), but not the only one. Survivors of ICH and SAH may have impairments in cognitive function (CF, such as keeping track of appointments, managing financial affairs) (1), social functioning and other domains. Cognition and social function are implied in the mRS with key questions regarding social function and ability to work. Previous investigations have used questionnaires such as the 136-question Sickness Impact Profile; (1) while comprehensive, the time needed may be prohibitive.

Recognizing the importance of accurate assessment of HRQoL, the NIH supported the development of the Patient Reported Outcomes Measurement Information System (PROMIS) and Neuro-QOL (2). Despite their introduction (3) there are few data in survivors of ICH or SAH. Particular advantages include web-based assessment with computer adaptive testing, where each response affects the next question asked. We tested the hypothesis that good outcome would be higher for the domain of mobility than other specific domains of HRQoL.

Materials and Methods

Patients

We prospectively enrolled consecutive patients from January 2011 through January 2014. All patients had a diagnosis of spontaneous ICH or SAH confirmed by a board-certified neurologist with head computed tomography (CT). Patients with trauma, hemorrhagic conversion of ischemic stroke, or structural lesions (e.g., tumor) were excluded. We approached patients or a legally authorized representative during the index hospitalization and asked for written consent to track identifiers and obtain outcomes, a preferred telephone number and email addresses. The study was approved by the Northwestern University Institutional Review Board. We recorded the medical history, severity of injury including the NIH Stroke Scale (NIHSS), and demographics.

mRS assessment

The mRS is a validated scale from 0 (no symptoms) to 6 (death). A single interviewer (MB) obtained the mRS by validated interview at one, three and 12 months. (4) mRS Scores were not given to respondents. For patients no longer in the hospital, the mRS was assessed by telephone interview, a commonly used method validated by others. (1, 5-8) We defined “good outcome” as independence, mRS 0 through 3 versus 4 through 5, typical for outcome studies of patients with ICH (9) or SAH. (“Good outcome” after acute ischemic stroke is usually more favorable, mRS 0 or 1 vs. worse. (10))

HRQoL assessment

Our methods for obtaining HRQoL with Neuro-QOL have been previously described.(11) When Neuro-QOL became available for research use in January 2011, we obtained HRQoL at one and three months, and follow-up at 12 months starting in late May, 2011. Coincident with the mRS assessment we sent an email with a link to complete the HRQoL assessment, the usual method. Respondents could also answer HRQoL questions over the telephone with study staff (MB) performing proxy entry, recording answers on behalf of a patient or family member. We administered computer adaptive banks (12) in the following Neuro-QOL instruments: lower extremity function (mobility), CF – executive function (managing finances and household affairs), CF – general concerns (clarity of thinking, train of thought), and satisfaction with social roles and activities (SRA, ability to get work done, be with family). Computer adaptive testing algorithms ask questions at the predicted level of HRQoL until further data are unlikely to alter the estimate. Results are expressed in T scores, continuous numbers where the general US population scores 50 ± 10. Further information on the algorithms, underlying iterative response theory and detailed information about these and other available instruments is available at www.nihpromis.org and www.neuroqol.org.

Statistical Analysis

We tested the hypothesis that, in analysis of receiver operating characteristic (ROC) curves, the area under the curve (AUC) would be different between domains of HRQoL. We assessed this by comparing the difference in AUC between curves and the corresponding Z-value. The maximal sensitivity and specificity were ascertained from the ROC analytic tables. We compared normally distributed variables (e.g. T Scores between the groups of good vs. poor outcome) with t-tests while non-normally distributed variables (NIH Stroke Scale, NIHSS, length of stay, etc.) were compared with the U statistic. Statistical analysis was performed with NCSS v. 9 (NCSS, LLC, Kaysville, UT, www.ncss.com).A Neuro-QoL statistician who was not involved in the acquisition of data (JLB) directed and reviewed the statistical analysis.

Results

Patient population

The demographics of the patients assessed at one month are shown in Table 1. (Data for survivors assessed at three and 12 months were similar.) We excluded 23 patients for whom we could not obtain HRQoL data, 23 patients for whom we could not obtain the mRS, 11 patients who died after hospital discharge, and 3 who were lost to follow-up and had neither mRS nor HRQoL data available. The sample was typical of patients with SAH and ICH in terms of age and historical rates of hypertension and diabetes. Compared to patients with good outcome, patients with poor outcome were older, had more comorbidities, had more severe neurologic injury on admit and had a longer length of stay in the intensive care unit and hospital.

Table 1.

Demographics of the patients assessed at one month; data were similar for the subset of patients who survived to three and 12 months. Patients with poor outcome were more likely to have ICH rather than SAH, be older, had greater severity of injury, longer length of stay and lower health related quality of life T Score.

Variable	Good Outcome	Poor Outcome	P
N	76	38	-
Patients with ICH	26 (34)	25 (66)	0.001
Modified Rankin Scale	2 [1 – 2]	4 [4 – 5]	<0.00001
Age	53.4 ± 14.5	64.4 ± 14.3	0.0002
Ethnicity, Caucasian	57 (75)	26 (68)	0.4
Black	10 (13)	9 (23)
Asian	8 (11)	3 (8)
Other	1 (1)
Women	42 (55)	18 (47)	0.4
Glasgow Coma Scale on admit	15 [15 – 15]	12 [8 – 14]	<0.00001
NIH Stroke Scale on admit	1 [0 – 1]	13 [8 – 19]	<0.00001
NIH Stroke Scale at 14 days	0 [0 – 1]	14.5 [12 – 19]	<0.00001
Length of stay in the ICU	7.5 [3.3 – 9.6]	9.0 [7.5 – 14.2]	0.02
Length of stay in the hospital	10.6 [9.3 – 12.3]	16.1 [13.3 – 24.2]	0.00008
History of hypertension	32 (42)	24 (63)	0.03
History of coronary artery disease	4 (5)	6 (16)	0.06
History of Diabetes	9 (11)	7 (18)	0.3
Respondent			<0.00001
Patient	39 (51)	3 (8)
Caregiver	8 (11)	19 (50)
Study Staff	29 (38)	16 (42)
Discharge Disposition			<0.00001
Home	56 (77)	3 (8)
Rehabilitation	13 (18)	18 (49)
Nursing facility	3 (4)	8 (22)
Acute care	1 (1)	7 (19)
Other	3 (4)	1 (2)
Mobility T Score	44.5 ± 9.5	22.5 ± 9.8	<0.00001
CF – General Concerns T Score	44.7 ± 10.8	31.1 ± 10.9	<0.00001
CF – Executive function T Score	43.6 ± 11.7	23.4 ± 11.7	<0.00001
Satisfaction with SRA T Score	45.6 ± 7.3	36.7 ± 5.5	<0.00001

Data are N (%), mean ± SD, or median [Q1 – Q3] as appropriate. The Glasgow Coma Scale is score from 3 (unresponsive) to 15 (alert and oriented). The NIH Stroke Scale is scores from 0 (normal) to 42 (worse possible score), with a score of 8 indicating a moderately severe deficit. T Scores are normalized to the US general population at 50 ± 10.

CF, applied cognition; SRA, social roles and activities

Each instrument required a modest number of questions to estimate the T Score, and this number of questions varied slightly. The computer adaptive test administered median 4 questions for CF – general concerns and satisfaction with SRA, median 5 questions for CF – executive function, and median 6 questions for mobility (P<0.00001). Patients with poor outcome had worse HRQoL T Scores (Table 1), varying from 8.9 points (0.9 SD) in the domain of satisfaction with social roles and activities to 22 points (2.2 SD) in the domain of mobility.

ROC curves are shown in the Figure. “Good outcome” generally indicated T scores approximately 1.5 SD below the US population mean, the value corresponding to the highest sensitivity and specificity for mRS 0 through 3 versus 4 through 5, e.g. the point on the ROC curve closest to the upper left corner of the curve. The area under the curve, optimal T score cutoff for distinguishing “good” vs. “poor” outcome and P value for comparison with mobility is shown in Table 2.

Figure.

ROC Curves of specific domains of health related quality of life (HRQoL) vs. dichotomous “good outcome,” defined as independence, modified Rankin Scale (mRS) 0 – 3 vs. 4 – 5 at one month follow-up. AUC was greatest for mobility (dashed line, 0.957), and less for satisfaction with SRA (dotted line, 0.850) and CF – general concerns (solid line, 0.819). CF – executive function was similar to CF – general concerns, and not shown for clarity of the graphic.

CF, cognitive function; SRA, satisfaction with social roles and activities

Table 2.

Comparison of the empirical area under the curve for good outcome (modified Rankin Scale of 0 – 3, independent for ambulation or better) vs. poor outcome (modified Rankin Scale 4 or 5) at one month follow-up in 114 patients, stratified by specific domains of health related quality of life measured with Neuro-QOL. Mobility T Scores were most closely associated with good outcome, although all were (P<0.0001). Results were similar for follow-up at three months and 12 months. A mobility T score of 35.8 had 92% sensitivity and 87% specificity for good outcome. Receiver operating characteristic curves are shown in the Figure.

Domain	Sample content	Area under the curve	95% CI	P for comparison with mobility	Optimal T Score Cutoff (sensitivity, specificity)
Mobility	walking on uneven surfaces or stairs	0.957	0.904 – 0.981	reference	35.8 (0.92, -.87)
CF – Executive function	managing finances and household affairs	0.879	0.782 – 0.935	0.058	34.6 (0.82, 0.84)
Satisfaction with SRA	ability to get work done, be with family	0.850	0.753 – 0.911	0.01	40.5 (0.79, 0.78)
CF – General concerns	clarity of thinking, keeping train of thought	0.819	0.715 – 0.888	0.003	35.7 (0.85, 0.61)

CF, applied cognition; SRA, satisfaction with social roles and activities

Results were similar when the time of outcome assessment was three months or 12 months, or whether “good outcome” was defined at 0 through 2 (moderate disability) vs. 3 through 5, rather than 0 through 3 vs. 4 through 5.

Discussion

We found that continuous measures of domain-specific HRQoL distinguished between patients with good and poor outcome. These data add to the literature by using web-based computer adaptive testing and formal analysis of differences in AUC. AUC was different between specific domains, demonstrating that good outcome reflected mobility more than CF or social roles and activities, both of which are important facets of HRQoL.

These data underscore the utility of the mRS as a global measure correlated with multiple domains of HRQoL, particularly mobility.(13) Patients who had good outcome had higher T Scores in the domains of CF and satisfaction with SRA than patients with poor outcome. We used a typical definition of them RS to define good outcome. As clinical outcomes scales are highly correlated with each other, (14) our results are likely to generalize to other outcome scales. Similarly, several NIH PROMIS instruments are highly correlated with Neuro-QOL, so one is likely to find similar results using the NIH PROMIS physical function instrument as opposed to the Neuro-QOL mobility instrument. Cross-walks between Neuro-QOL, PROMIS and other validated outcome assessments are available at www.PROsettastone.org. More domains are available than the ones we assessed and both SAH and ICH impair multiple domains of HRQoL in survivors. Future research might select other specific domains of particular interest for a given research hypothesis. NIH PROMIS and Neuro-QOL instruments are periodically updated, and scores from new instruments can be cross-walked to those from older instruments. This is part of the original mission of Neuro-QOL and PROMIS, crucial to obviate allegiance to outdated instruments for the sake of consistency over time.

Transitions from “good” to “poor” outcome were associated with large decreases in T Scores. The ability to compare continuous numeric T Scores rather than dichotomous categories may reduce type II error, particularly in domains that are not as well measured by the mRS and similar ordinal summary scores. A meaningful difference is generally regarded as 0.5 SD, (15) or 5 points on the T Score for the measures we describe here. In addition to a research setting, this might be helpful to assess the clinical status of patients, or changes over time as we have previously described. (11)

Different domains of HRQoL may give complementary assessments of outcome. Previous investigations using multi-domain HRQoL assessments have also noted that good outcome is primarily defined by physical function, (16) although the AUC was not compared. It's important to be able to walk to the grocery store (mobility), but also important to be able to remember what to buy and pay the bill (CF), distinctions difficult to capture with a summary score.

We assessed HRQoL using web-based computer adaptive testing. This permits an algorithm to select questions likely to be most informative for a specific respondent during the assessment, and avoids asking repeated questions that may be inappropriate. Once it is known a patient is non-ambulatory, subsequent questions about walking do not improve the estimate of ability; adaptive tests query the respondent about easier tasks, such as sitting on the edge of a bed. Each domain was assessed in a median of only four to six questions. Unreliable computer adaptive tests would have been reflected in a high number of questions needed to estimate the T Score for each domain and bias results toward the null hypothesis, not the results we found.

The designation of good outcome has a ceiling effect, discounting potential deficits in physical, social and cognitive HRQoL. (17) Indeed, the optimal discrimination point of T Scores for “good outcome” in ROC analysis was approximately 1.5 SD (15 points on the T-score) below the US population mean. Continuous T Scores allow one to compare the mean (or median) T Score, a statistically more powerful comparison than comparing the odds of good outcome,(15) potentially decreasing the number of patients needed for clinical research and allowing one to examine differences within the category of good outcome. One might assess Neuro-QOL or NIH PROMIS outcomes and compare T scores between groups for a primary endpoint, while other specific domains might be pre-defined secondary endpoints. High mortality might attenuate the advantages of measuring HRQoL as a primary endpoint, but mitigating this has been previously described. (18) Neuro-QOL and PROMIS may complement the mRS, and be of particular interest in patients with at least some mobility.

We focused on domains that could be reliably assessed by the patient or a proxy. Neuro-QOL was validated for proxy report as part of its development and we have previously noted that correcting for proxy report yields similar results,(13) particularly with regards to CF.(11) Other HRQoL scales have also been validated for proxy report.(19, 20) A potential limitation is that we did not perform specific neurocognitive testing to elucidate the cognitive impairments that underlie lower T Scores for CF, and this is an opportunity for future research, specifically with standard assessments such as the NIH Toolbox.(21) We did not perform follow-up in person after hospital discharge, although the mRS has been validated for telephone assessment and vis-a-vis the in-person NIHSS.(14)

In this study we focused on SAH and ICH, although our results are likely to apply to other critical illnesses. For example, acute respiratory distress syndrome impairs function years after illness, even for independent patients. (22) The advantages of Neuro-QOL and NIH PROMIS are also likely to apply.

In sum, we found that web-based computer adaptive testing for HRQoL reliably distinguished between good vs. poor outcome. The AUC was greater for mobility than CF or satisfaction with SRA, underscoring that much clinical research necessarily focuses on mobility even when other domains may be of equivalent or greater interest. Measuring domain-specific HRQoL may provide an opportunity to make domain-specific improvements in outcome, such as improving CF alone, even if an outcome summary score would not change. Choosing continuous, domain-specific measures may allow for more patient-centered research.