Evaluating individual change with the Quality of Life in Neurological Disorders (Neuro-QoL) Short Forms

Abstract

Objective

To provide a clinically useful means of interpreting change for individual patients on the Neurological Quality of Life (Neuro-QoL) adult short forms (SFs) by applying a Classical Test Theory concept for interpreting individual change.

Design

Secondary analysis of existing data.

Setting

Community.

Participants

Persons with neurological conditions including stroke, epilepsy, amyotrophic lateral sclerosis, multiple sclerosis, and Parkinson’s disease residing in community settings.

Interventions

Not applicable.

Main Outcome Measures

Neuro-QOL SFs for Applied Cognition-General Concerns, Applied Cognition-Executive Function, Applied Cognition-Combined, Ability to Participate in Social Roles and Activities, Satisfaction with Social Roles and Activities, Positive Affect and Well-Being, Depression, Stigma, Upper Extremity Function (Fine Motor, Activities of Daily Living), Lower Extremity Function (Mobility), Anxiety, Sleep Disturbance, Fatigue, and Emotional and Behavioral Dyscontrol.

Methods

We estimated conditional minimal detectable change (cMDC) indices from the pooled standard errors (SEs) adjusted for a 95% confidence interval (CI) using the average of the SEs for any given pair of scores multiplied by the z-score, or [(SE_Score1 +SE_Score2)/2 · 1.96].

Results

The cMDC indices are generally smallest in the mid-range of all scales, ranging from 3.6 to 11.2 T-score points, and higher on the outer quartiles ranging from 3.7 to 21.6 T-score points. The lowest mid-range cMDCs were for Satisfaction with Social Roles and Activities (3.6–4.7 T-score points) and the largest was for Sleep Disturbance (9.4–11.2 T-score points).

Conclusions

Change indices can help clinicians and investigators identify differences for individual patients or subjects that are large enough to motivate treatment change. cMDCs can reduce misclassification of magnitudes of change that are near the margins of error across the range of the Neuro-QoL SFs.

Background

Evaluating change across time is critical to decision making in rehabilitation research and clinical practice. However, evaluating clinical change implies an ability to measure change at the individual patient level,^1–3 rather than at the group-level, as is typical in most research. In the assessment of individuals, Liang differentiated between sensitivity to change, defined as measured change regardless of its clinical meaningfulness, and responsiveness, defined as measured change that is clinically meaningful.¹ In this framework, sensitivity to change describes distribution-based indices such as minimal detectable change (MDC), smallest detectable change, and minimal real change,² while responsiveness describes externally anchored indices like minimal clinically important difference (MCID) and clinically relevant change.² In brief, distribution-based indices describe the smallest change that exceeds an instrument’s measurement error, while anchor-based indices represent the smallest change perceived as meaningful or relevant by patients or clinicians.⁵ However, this framework is not ubiquitous throughout healthcare, and some disciplines consider distribution-based and anchor-based indices to be different forms of responsiveness.²

Reporting of individual change indices for has improved the utility of many classic instruments for use with different patient populations, as demonstrated though instrument reviews available from the Rehabilitation Measures Database.⁴. Often a single change index value is reported, which assumes that measurement error is consistent across all possible scores for the instrument. Yet, statistically, instruments tend to be more reliable (discriminate better) near the midrange of the scale, and less so at either extreme. Accordingly, the margin of error may vary depending upon where scores fall along the range of the scale. To illustrate this, Stratford and colleagues described a conditional MDC (cMDC) index for the Roland Morris Questionnaire for back pain related disability.⁵ While the cMDC improves sensitivity to change b, it has only been reported instruments that are scored as the sum of their items, which are ordinal in nature. Newer instruments based on Item-Response Theory (IRT) provide measurement with interval properties and item- or score-level reliability (standard errors (SEs) are reported for each item and for each score across a scale range), presumably making them more sensitive to change.

The Neurological Quality of Life (Neuro-QoL) Measurement System⁶ was developed using IRT methods primarily for research applications, but the instruments may be ideal for clinical use.

Neuro-QoL instruments have large item banks with wide scale ranges which can be administered by computer adaptive test or short form (SF), and by use of internet-based survey platforms such as Assessment Center.⁷ However, interpreting change on a Neuro-QoL instrument is based on a reported SE for a pair of scores, which does not account for the overlap of the margins of error between the scores, or for a different confidence interval (CI). We sought to develop change indices for the Neuro-QoL instruments that are analogous to the cMDC, based on the assumption that the SE is analogous to the standard error of measurement (both provide a margin of error for a point score), which is the basis for calculating MDC () on classic instruments. This paper reports these indices for 14 Neuro-QoL SFs.

Methods

The Neuro-QoL Measurement System (http://www.neuroqol.org/Pages/default.aspx) comprise a set of self-report instruments of HRQOL for adults and children with neurological disorders, which are available as item banks for computer adaptive tests and as fixed-length SFs. We report on the 14 adult SFs, including Applied Cognition-General Concerns, Applied Cognition-Executive Function, Applied Cognition-Combined, Ability to Participate in Social Roles and Activities, Satisfaction with Social Roles and Activities, Positive Affect and Well-Being, Depression, Stigma, Upper Extremity Function (Fine Motor, Activities of Daily Living), Lower Extremity Function (Mobility), Anxiety, Sleep Disturbance, Fatigue, and Emotional and Behavioral Dyscontrol.

We estimated cMDC from the average of the SEs for any given pair of scores multiplied by the z-score for a given CI (in this case 1.96 for a 95% CI identified as a subscript), which gives [((SE_Score1 +SE_Score2)/2) · 1.96] using Microsoft Excel 2010.^a Change scores were determined for each pair of T-scores on each of 14 Neuro-QoL SFs (13 Version-1 SFs plus the composite Version-2 Applied Cognition SF) and compared to the estimate for the cMDC₍₉₅₎ for that pair of scores. Change scores that do not meet the MDC₍₉₅₎ are those that do not exceed the pooled adjusted margin of error. All data were obtained from the publicly available Neuro-QoL User’s Manual (http://www.neuroqol.org/Resources/Neuro-QoLReports-Manuals/Pages/default.aspx),⁸ which details the instrument design and sampling procedures.

Results

The cMDC values for the middle half and end quartiles the scale ranges are summarized for the 14 Adult Neuro-QoL short forms (table 1), and the index table for the Anxiety SF is provided as an example (figure 1). Index tables which include interactive cMDC calculators are provided for all 14 adult Neuro-QoL SFs in an Excel workbook (online supplement). The raw scores, T-scores, and the associated SEs increase across the top and down the left side of the table. To interpret change between two scores for an individual using the table, find the corresponding raw scores across the top (Time 1) and down the left side (Time 2) and locate the intersecting cell. If the cell is highlighted (grey in figure 1 or red in the online supplement) the scores probably do not represent a detectable change. Scores above the highlighted zone represent an increase on the construct (i.e., worsening Anxiety in figure 1), and scores below the zone represent a decrease. To use the calculator, enter the Time 1 and Time 2 scores and the resulting change score and an interpretation will be returned.

Table 1.

Summary of conditional minimal detectable change indices for individual assessment of difference scores at the lower quartile, middle half, and upper quartile of Neuro-QoL short form scale ranges

Neuro-QoL Short Form Domain and Description	Scale Range*
	Lower Quartile	Middle Half	Upper Quartile
Applied Cognition-General Concerns:	5.0 – 9.4	5.0 – 6.1	5.3 – 17.2
Applied Cognition-Executive Function:	6.4 – 7.5	6.1 – 7.8	6.4 – 18.6
Cognitive Function – Combined: Perceived difficulties in cognitive abilities, or in the application of such abilities to everyday tasks	7.5 – 11.9	7.2 – 8.9	7.5 – 15.8
Ability to Participate in Social Roles and Activities: Involvement in social roles, activities and responsibilities (e.g., work, family, friends and leisure)	3.9 – 10.5	3.9 – 4.4	3.9 – 16.1
Satisfaction with Social Roles and Activities: Satisfaction with involvement in one’s usual social roles, activities and responsibilities, including work, family, friends and leisure	3.7 – 11.4	3.6 – 4.7	3.7 – 15.8
Anxiety: Unpleasant thoughts/feelings related to fear, helplessness, worry and hyper-arousal	5.0 – 11.2	4.7 – 5.4	4.9 – 7.8
Upper Extremity Function (Fine Motor, Activities of Daily Living): One’s ability to carry out various activities involving digital, manual and reach-related functions, ranging from fine motor to self-care (activities of daily living)	5.8 – 6.9	5.5 – 9.3	6.0 – 21.6
Lower Extremity Function (Mobility): One’s ability to carry out various activities involving the trunk region and increasing degrees of bodily movement, ambulation, balance or endurance	6.0 – 8.3	5.7 – 7.5	6.2 – 17.7
Positive Affect and Well-Being: Aspects of a person’s life that relate to a sense of well-being, life satisfaction or an overall sense of purpose and meaning	3.9 – 10.8	3.9 – 4.2	3.9 – 12.5
Emotional and Behavioral Dyscontrol: A set of disease and/or treatment related manifestations including disinhibition, emotional lability, irritability, impatience, and impulsiveness	6.7 – 13.6	6.7 – 7.1	6.7 – 9.1
Depression: Experience of loss and feelings of hopelessness, negative mood, decrease in positive affect, information-processing deficits, negative views of the self, and negative social cognition	4.2 – 14.4	4.2 – 4.6	4.2 – 10.3
Stigma: Perceptions of self and publically enacted negativity, prejudice and discrimination as a result disease-related manifestations	5.4 – 16.1	5.3 – 6.4	5.5 – 9.7
Fatigue: Sensations ranging from tiredness to an overwhelming, debilitating and sustained sense of exhaustion that decreases one’s capacity for physical, functional, social and mental activities	4.7 – 12.2	4.7 – 5.1	5.0 – 11.3
Sleep Disturbance: Perceptions of sleep quality, depth, and restoration associated with sleep; perceived difficulties with getting to sleep or staying asleep; and perceptions of the adequacy of and satisfaction with sleep	10.0 – 16.4	9.4 – 11.2	9.7 – 10.8

^*The ranges of conditional minimal detectable change (cMDC) values are provided for the lower quartile (lowest 8 scores, middle half (middle 17 scores), and upper quartile (highest 8 scores) of the scale range for all Neuro-QoL adult short forms except for Positive Affect and Wellbeing which correspond to the lowest 9, middle 19 scores, and highest 9 scores.

Figure 1.

Conditional minimal detectable change (cMDC) index table for assessment of individual difference scores on the Neuro-QoL Anxiety adult short form. To determine whether a difference between two scores represents a statistically detectable change on a 95% confidence interval, locate the raw or T-score from the first time point along the top margin of the table, and the score for the second time point along the left margin. The intersecting cell provides the cMDC for that pair of scores; if the cell is in the grey zone that runs diagonally down from the top-left corner, the difference score is less than the cMDC and probably does not represent a true change for an individual on the Neuro-QoL Anxiety short form.

With a few exceptions, the cMDCs are smallest in the mid-range of the scales (table 1). The lowest mid-range cMDC was for Satisfaction with Social Roles and Activities and the largest was for Sleep Disturbance. The Upper Extremity Function and Applied Cognition – Executive Function SFs showed low cMDCs (6.1–6.4 and 6.7–7.1 T-score points respectively) at the low end of the scale range due to small SE values.

Discussion

We aimed to facilitate clinical application of the Neuro-QoL SFs by developing distribution-based change indices at the individual level. Our method was similar to that reported for classic instruments.⁵. Instruments based on IRT have an advantage over classic instruments in that SE is estimated for each item or for each score across their scale ranges, which allows more granular reporting cMDC indices. We have not seen this concept applied to an IRT instrument, but have found single MCD indices which assume error is constant across a scale range reported IRT-based instruments.^{9, 10}

The cMDC values aid one to reliably consider whether an observed change for an individual exceeds measurement error. These values should not be applied to group-level changes which may actually be smaller in magnitude than the cMDC. These change indices will benefit clinicians by facilitating care decisions based on whether or not patients have experienced true change. Clinicians can be confident that the change is not attributable to measurement error when an individual’s change score exceeds an MDC value. However, applying a single MDC across an entire scale range may result in misclassification Variability in SEs, and erroneous inferences of change at scale ends (or of no change in the mid-range) could affect decisions to continue or discontinue care.

Limitations

A limitation of this study is that the cMDC values do not indicate whether a measured change is clinically important. The Neuro-QoL T-scores are interpretable using a population-based norm for each domain represented by a score of 50 with a SD of 10 points on an interval scale.^{6, 8} Thus, importance may be inferred by the magnitude of change relative to the norm, once one is confident that the scores represent change that exceeds measurement error. Future efforts could develop anchor-based MCID indices for Neuro-QoL instruments and develop cMDC tables and calculators for other IRT-based instruments.

Conclusions

Change indices can help clinicians and investigators identify differences that are large enough to motivate treatment change. cMDCs may reduce misclassification of magnitudes of change that are near the margins of error across the range of the Neuro-QoL SFs.

Highlights.

• Clinicians benefit from instruments on which change can be measured for individual patients

• The Neuro-QoL Measurement System covers domains of physical, mental, and social functioning for persons with a variety of neurological conditions

• Measurement error varies across scale ranges

• Conditional minimal detectable change (cMDC) indices improve sensitivity to measure change

• We provide cMDC tables to facilitate individual change measurement for 14 Neuro-QoL adult short forms

List of Abbreviations

CI

confidence interval

cMDC

conditional minimal detectable change

HRQOL

Health-related quality of life

IRT

Item-Response Theory

Neuro-QOL

Quality of Life in Neurological Disorders

SE

standard error (IRT)

SF

short form