Quantifying the Value of Stroke Disability Outcomes: WHO Global Burden of Disease Project Disability Weights for Each Level of the Modified Rankin Scale

Abstract

Background

The modified Rankin Scale (mRS) categorizes poststroke disability among 7 broad, ordinal grades, but the interval distances between these levels are spaced along the disability spectrum has not been previously investigated.

Methods

We used the person trade-off procedure developed by the World Health Organization Global Burden of Disease Project (WHO-GBDP) to generate disability weights (DWs) ranging from from 0 [normal] to 1 [dead] for each 7 mRS grade. The ratings of an international, 9 member panel of stroke experts were combined by a modified Delphi process.

Results

DWs (95% confidence interval) were 0 for mRS 0, 0.046 (0.004–0.088) for mRS 1, 0.212 (0.175–0.250) for mRS 2, 0.331 (0.292–0.371) for mRS 3, 0.652 (0.625–0.678) for mRS 4, 0.944 (0.873–1.015) for mRS 5, and 1.0 for mRS 6. DWs of adjacent mRS levels were significantly different (p<0.001 for all). Coefficient of variations showed high degree of consensus for DWs among panel members. DWs placed each of the five intermediate mRS states in different disability class levels of the WHO-GBDP anchor conditions, and identified natural clusters to employ when reducing the mRS to fewer categories.

Conclusion

Formal DW assignment confirms that the mRS is an ordered, but unequally spaced scale. The availability of DWs for each mRS level now permits direct comparison of each poststroke outcome state with the outcomes of hundreds of other diseases in the WHO-GBDP and the expression of stroke burden in different populations using the uniform metric of disability-adjusted life years lost.

Introduction

Residual disability among stroke survivors spans a wide functional range. A variety of ordinal scales have been developed to classify patients along this spectrum, arraying individuals among a few broad, ordered levels. The most comprehensive and widely employed ordinal poststroke global disability scale is the modified Rankin Scale (mRS), which assigns stroke survivors among seven levels, including the extremes of normal and dead and five intermediate degrees of disability.

However, while ordinal scales are substantially more informative than binary outcome measures, they are not as sensitive to change as continuous outcome measures. Moreover, unlike interval scales, in which the distance between each scale step is uniform, ordinal scales array patients among unevenly spaced steps along the outcome range. Without knowing how far apart each step of an ordinal scale is along the disability spectrum, it is difficult to assess and compare therapies that exert differential beneficial and/or harmful effects at various scale state transitions.

Several methods have been developed to quantify how far apart on a health outcome dimension are the ranked levels of an ordinal scale. Most studies have employed patient preference techniques, including standard gamble, time trade-off, and visual analogue scale methods. These techniques elicit utilities - how much value patients or healthy individuals place on specified outcomes,^1–3 and permit analysis of outcomes in terms of quality-adjusted life years (QALYs) gained or lost. However, when ratings are required for multiple, rather than a single, outcome states, these methods have the substantial weakness that patients and healthy individuals have limited direct experience of different diseases and disease intensities. Patients experience disease in depth, but not in breadth.

Employing as raters physicians and allied health professionals, who have direct encounters with multiple diseases and disease severities, overcomes the unfamiliarity problem of patient-based quality weighting techniques. Healthcare professionals observe disease both in breadth and in depth, albeit externally.^4,5 Accordingly, tasked with deriving health outcome state ratings for all human diseases, the World Health Organization Global Burden of Disease Project (WHO-GBDP) developed the person trade-off (PTO) method.⁶ In the PTO framework, health professionals judge health states from a broad public health viewpoint that ensures equity across health states.⁷ This technique yields disability weights (DWs) for each disease, permitting analysis of treatment and policy outcomes in terms of disability-adjusted life years (DALYs) gained or lost. The WHO’s PTO approach requires raters to undertake interpersonal comparisons of utility for individuals with different disease states providing greater content validity to the resulting relative weights.^6,8 In addition, the approach minimizes the confounders arising from adjustment to disease bias and lack of knowledge of a health state.^6,9 The WHO’s DALY analyses have clarified understanding of the relative contribution of individual diseases to the total burden of disease in developed and underdeveloped regions and worldwide.⁶

The PTO method can facilitate comparison of health outcomes not only across different diseases, but also across varying severities of one disease.¹⁰ Consequently, the PTO method is well-suited to assign quantified DWs for each of the ordinal levels of poststroke outcome scales.

Methods

We convened an international expert panel of stroke neurologists (7), a rehabilitation neurologist (1), and a double-boarded emergency physician and neurologist (1) from the US (4), South Korea (3), mainland China (1), and Taiwan (1).

DWs range from 0 (normal health without disability) to 1.0 (dead or as bad as being dead). DWs for mRS 0 and 6 were assigned 0 and 1.0, respectively. To determine the DWs for the five intermediate mRS categories, the panel members each individually performed the WHO-GBD PTO1 exercise,⁶ in which they were asked to assume they were allocating health system resources and had to decide what number would make them indifferent between the choices of extending the lives of 1,000 healthy persons for 1 year versus extending lives of N persons with the mRS health state for 1 year. As a reference framework during this task, the panel members were provided with the WHO-GBDP framework table, which displays seven disability classes and 22 anchoring example conditions (Table 1).⁶

Table 1.

WHO Global Burden of Disease Project: Disability Classes and Person Trade-Off numbers of Disabled Individuals of 22 Indicator Conditions⁶

Disability class	PTO number of disabled individuals	Indicator conditions
1	1000 – 1020	Vitiligo on face; weight for height < 2 standard deviations
2	1021 – 1136	Watery diarrhea; severe sore throat; severe anemia
3	1137 – 1316	Radius fracture in a stiff cast; infertility; erectile dysfunction; rheumatoid arthritis; angina
4	1317 – 1562	Below-the-knee amputation; complete deafness
5	1563 – 2000	Rectovaginal fistula; mild mental retardation; Down syndrome
6	2001 – 3333	Unipolar major depression; complete blindness; paraplegia
7	3334 – infinity	Continuous psychosis; dementia; quadriplegia

PTO, person trade-off.

In the first round of the modified Delphi process,¹¹ each expert individually provided ratings for the 5 intermediate mRS levels. The panel then met as a group and discussed each mRS level in turn. At the start of each level discussion, each panelist received a report showing: 1) the group’s first round mean and median PTO and disability class values for that mRS level, 2) the PTO and disability class values given by all respondents (anonymized) in the first round, and 3) the panelist’s own individual PTO rating in the first round. The group discussed the most appropriate PTO value for each of the 5 mRS states, with the moderator emphasizing full participation of all panel members, respect for all opinions, and continuation of discussion until distinctive thematic considerations were no longer elicited. Each panel member then again, independently and confidentially, assigned a PTO value for the discussed mRS state. The coefficient of variation (CV) was calculated to determine the need for additional Delphi rounds, with CV ≤ 0.5 indicating achievement of substantial consensus.¹²

PTO values were converted to DWs by the formula: DW=1–1000/PTO. As PTO values are exponentially distributed, their central tendency was assessed by calculating geometric mean and 95% confidence interval. A few experts assigned mRS 5 state a PTO value of infinity (equal to death). In these instances, the extreme PTO value of 1,000,000,000 was substituted to permit calculation of geometric means. DWs are linearly distributed and were described by linear means and 95% confidence intervals.

Normalization to existing WHO-GBDP rating for poststroke disability

The WHO-GBDP has generated DWs for acute stroke (0.920) and chronic poststroke disability (0.266).¹³ Although these ratings appropriately distinguish between acute and chronic poststroke health states, they fail to distinguish among chronic poststroke states of varying severity. The multispecialty WHO panel was well-suited to compare disability across diverse organ-specific conditions, but not to make fine distinctions among different stroke outcomes. In contrast, the PTO ratings of our stroke expert panel may be expected to be most reliable in delineating the relative value of different stroke outcomes compared with each other, and less precise in determining the relative position of the disease category of stroke compared with hundreds of other, especially non-neurologic, diseases. To compare the DW ratings of the five intermediate mRS ranks generated by our stroke expert panel with the unitary chronic poststroke DW rating of WHO-GBDP, we generated a composite DW for all stroke survivors from the DWs of individual mRS levels, using frequency-weighted averaging. For each mRS level, the panel-assigned DW was multiplied by the frequency of occurrence of that level among stroke survivors. To make the panel’s DWs for individual mRS levels fully consistent with the WHO-GBDP framework, an adjustment formula was derived to normalize the DWs for each mRS level to the WHO-GBDP unitary DW for all stroke survivors (supplemental Mathematical Appendix, available online at http://stroke.ahajournals.org).

Statistical Analyses

The Mann-Whitney test was used to compare the DWs of adjacent mRS states and DWs between the US and the Asian panel members. Wilcoxon signed rank test was used to compare first and second round DW CVs.

Results

The mean DWs and geometric mean PTO values yielded by the panel members’ PTO ratings are shown in Table 2. All five intermediate disability states were assigned DW values consonant with their rank order on the mRS. DWs of adjacent mRS states were significantly different for all levels (p<0.0001). PTO values placed each of the five intermediate mRS states in different disability class levels of the WHO-GBDP anchor conditions, with mRS 1 assigned to WHO disability class (WHO-DC) 1, mRS 2 to WHO-DC 3, mRS 3 to WHO-DC 4, mRS 4 to WHO-DC 6, and mRS 5 to WHO-DC 7.

Table 2.

Person Trade-Off and Disability Weight for each modified Rankin Scale status

	mRS 1	mRS 2	mRS 3	mRS 4	mRS 5
PTO number
Geometric mean	1050	1272	1500	2881	349968
Geometric 95% CI	1003–1099	1211–1335	1415–1590	2676–3102	8950–1.37×107
Disability weight
Mean	0.046	0.212	0.331	0.652	0.944
95% CI	0.004–0.088	0.175–0.250	0.292–0.371	0.625–0.678	0.873–1.015
Adjusted disability weight	0.053	0.228	0.353	0.691	0.998

PTO, person trade-off; mRS, modified Rankin Scale; CI, confidence interval.

Adjusted disability weights were derived by the formula: DW_Xadj = 1.052×DW_Xsp + 0.005, where DW_Xadj is the adjusted disabililty weight for mRS level X, DW_Xsp is the stroke expert panel generated disability weight for mRS level X, 1.052 is the adjustment factor, and 0.005 is the y-axis interception.

For all mRS states, 95% confidence intervals of DWs were narrow, less than 0.15 (Table 2). In the first round, the CV values for the DWs were less than 0.5 for all mRS ranks except for mRS 1 (Table 3). Inspection of individual rater values (supplemental Table A, available online at http://stroke.ahajournals.org) shows actual good agreement for DWs for mRS 1. The extremely low value range of the DWs of mRS 1 magnifies the relative value of small absolute differences. The CVs decreased in the second Delphi round compared to those in the first round (p=0.043). As the overall CVs showed high degree of consensus, further rounds were not necessary. Panel members from the US and Asia showed no difference in DW ratings (p>0.2 for all DWs).

Table 3.

Coefficient of variations of disability weight for each modified Rankin Scale status in the first and second round

Round	mRS 1	mRS 2	mRS 3	mRS 4	mRS 5
1	1.40	0.41	0.25	0.13	0.11
2	1.19	0.23	0.16	0.05	0.10

mRS, modified Rankin Scale.

P=0.043 for comparison of the coefficient of variations of disability weights between the first and the second Delphi round (Wilcoxon signed rank test)

The distance between one step mRS increments on the DW scale varied, confirming that the mRS is not an interval scale with equally spaced steps, but rather a rank order scale with unequally spaced steps (Figure 1). The closest mRS levels were mRS 0 and mRS 1, separated by 0.046 units; furthest apart were mRS 3 and mRS 4, separated by 0.321 units, a 7-fold difference. Modified Rankin Scale steps clustered among 4 groups along the DW scale: no disability, with or without symptoms – mRS 0–1; mild to moderate disability – mRS 2–3; severe disability – mRS 4; and extreme disability or death – mRS 5–6.

Figure 1.

Disability weights (means and 95% CIs) for each level of modified Rankin Scale

Medline review identified six studies in broad populations measuring the frequency of each mRS level among stroke survivors, 2 studies each for ischemic stroke (IS), intracerebral hemorrhage (ICH), and subarachnoid hemorrhage (SAH) (supplemental References, available online at http://stroke.ahajournals.org). Since each study was performed among not fully comparable race-ethnic groups, the mRS frequencies for each stroke subtype were derived by averaging at the study level, rather than by weighting for sample size. Similarly Medline review identified 2 population-based studies reporting the incidence of different stroke subtypes, one from the Cincinnati/Northern Kentucky region in the US and one from the Changsha region in China.^14,15 The frequency of ICH was substantially higher among the Asian than the North American population. Based on the global perspective of the WHO-GBDP, these frequencies were averaged at the study level. This yielded incidence frequencies of stroke subtypes of IS – 64.3%, ICH – 29.8%, and SAH – 2.1%. The relative frequency of each stroke subtype among all stroke survivors was derived by correcting for differential stroke subtype 30-day case-fatality rates (10.2% for IS, 37.6% for ICH, and 31.3% for SAH)¹⁴ to yield frequencies of each stroke subtype among stroke survivors of 74.3% IS, 23.9% ICH, and 1.9% SAH. The frequency-weighted average DWs for survivors of stroke subtypes were 0.223 for IS, 0.329 for ICH, and 0.209 for SAH (Table 4). The frequency-weighted average DW for all stroke survivors was 0.248, which was close to chronic poststroke DW of the WHO-GBDP multispecialty panel (0.266).¹³ Adjusted individual mRS DWs that normalize the stroke expert panel DWs to the WHO-GBDP global DW are shown in Table 2, and are close to the unadjusted values.

Table 4.

Frequencies of modified Rankin Scale Outcomes Among Stroke Survivors and Resulting Frequency-Weighted Disability Weights for Stroke

	Ischemic Stroke			Intracerebral Hemorrhage			Subarachnoid Hemorrhage			All Stroke*
	German stroke Data Bank (n=2455)	COMPASS (n=1142)	Ischemic Stroke Combined	FAST placebo (n=217)	OXVASC (n=14)	ICH Combined	ISAT (n=1969)	JSSRS (n=367)	SAH Combined
mRS0	23.3	30.2	26.7	5.0	7.2	6.1	17.6	44.9	31.2	21.9
mRS1	30.2	24.2	27.2	20.0	21.6	20.8	29.6	21.8	25.7	25.7
mRS2	17.4	16.0	16.7	18.8	28.7	23.7	26.8	7.7	17.2	18.4
mRS3	9.3	13.2	11.2	23.8	21.6	22.7	15.0	6.4	10.7	14.0
mRS4	14.0	9.3	11.6	27.5	14.4	20.9	4.1	7.7	5.9	13.7
mRS5	5.8	7.3	6.6	5.0	7.2	6.1	6.8	11.5	9.2	6.5
Composite DW	0.228	0.218	0.223	0.354	0.304	0.329	0.211	0.207	0.209	0.248

mRS, modified Rankin Scale; DW, disability weight

^*Frequencies of mRS outcomes among all stroke survivors were adjusted for the frequency of each stroke subtype. Numbers are percentile except for DWs.

References of the individual studies are listed in the Supplemental References, available online at http://stroke.ahajournals.org.

Discussion

This study is the first to derive DWs for each level of the mRS of poststroke global disability. The internal validity of the study is supported by the low CVs among the raters and the derived DWs conforming to the original rank order of the mRS. The external validity is supported by consistency with 1) two prior DW studies of broader poststroke categories, 2) a survey study of physician intuitions regarding differences in mRS levels, 3) patient-derived quality of life (QOL) utility ratings of poststroke states, and 4) concurrent validity studies of complementary functional measures associated with different mRS levels.

Our results concur with and extend two prior studies of DWs for poststroke health states.^10,13 The composite DW for all stroke survivors yielded by frequency-weighting individual DWs for each mRS level in the current study (0.248) is quite close to the unitary DW for the chronic poststroke state derived by the WHO-GBDP multispecialty panel (0.266).¹³ As a result, calibrating the fine-grained state distinctions made by stroke experts to the broad global state assessment made by a multispeciality panel required minimal adjustment (Table 2).

A Dutch study used PTO to derive DWs for three categories of poststroke impairment: severe, moderate, and mild.¹⁰ The DW of 0.94 for mRS level 5 in this study is consistent with the DW of 0.8–1.0 for severe poststroke impairment in the Dutch study, and the DW of 0.65 for mRS level 4 in the present study comports with the Dutch DW of 0.5–0.65 for moderate poststroke impairment. In contrast, the DWs of 0.05, 0.21, and 0.33 for mRS levels 1, 2 and 3 in the current study are generally lower than the Dutch DW for mild poststroke impairment of 0.3–0.4. This may reflect differences in the specialty composition of the expert panels, as the general practitioners in the Dutch study may have less familiarity and experience with mild poststroke impairments than the stroke physicians in our study. Cultural difference regarding equity, an important consideration in determining PTO values, might also account for the differences in the DWs ratings. However, as our panelists were composed of multinational experts (although none of European countries), the cultural difference might not be an important determinant factor. The current study goes beyond prior DW investigations to provide specific DWs for a more granular series of poststroke states, yielding valuations with greater sensitivity to change over the full stroke outcome range.

The findings of the current study largely conform to a prior, informal questionnaire study delineating stroke physician valuation of individual mRS levels.¹⁶ The survey found stroke physicians value health state transitions across all the mRS levels, with the frequent exception of transitions from extreme disability (mRS 5) to death (mRS 6), which are often collapsed into a single worst outcome category.¹⁷ Using PTO methodology, the current study formally demonstrated a monotonic, ordered separation of DWs for all mRS levels, but with the smallest adjusted DW separation between mRS 5 and 6.

No prior study has derived patient preference QOL utility ratings for each mRS level. However, many studies of broader poststroke categories have been undertaken, typically delineating separate utilities for two (minor and major) or three (minor, moderate, and major stroke) poststroke states. To compare these utility values with disability weights, it is important to recognize that QOL utility analyses assign a score of 0 for being as bad as dead and 1 for being as good as normal, and thus yield inverse values compared with DWs. To permit direct comparison, QOL utility values may be converted to quality weights (QWs) by the formula: QW=1 utility. A systematic review of all studies deriving QOL utility ratings by patient time tradeoff (TTO) preferences for poststroke states found that, to minor stroke, stroke survivors assigned a QW of 0.28 and nonstroke patients with vascular risk factors a QW of 0.45; while to major stroke, stroke survivors assigned a QW of 0.59 and nonstroke patients with vascular risk factors a QW of 0.74.¹⁸ The DWs in this study for the mRS levels comprising minor stroke (mRS 1, 2, possibly 3) and major stroke (4, 5, and possibly 3) are roughly consonant with the QWs previously derived using patient TTO techniques.

It is noteworthy that the physician-derived DWs correspond closely with the stroke patient-derived QWs, in contrast with the QWs derived from nonstroke patients, which are substantially more severe. It is a common finding that patients rate their disability less than non-patients.¹⁸ This pattern has been attributed to several factors, including adaptation to disease by affected patients (“adjustment to disease bias”), and non-patients’ fear of unaccustomed conditions (“unfamiliarity with disease bias”). We hypothesize that physician-derived ratings resemble affected patient-derived ratings because physicians are knowledgeable regarding the disease state and have observed the substantial capacities of their patients to adjust to physical challenge.

This study’s findings of distinctive, separate DWs for each mRS level also correlates with prior studies of the clinical and economic course of patients at different mRS grades.^19,20 The proportion of the first 3 poststroke months spent at home decreases as mRS level increases, longterm life expectancy decreases as mRS level increases, and cost of care increases as mRS level increases from 0–5.

That adjacent mRS levels are unequally spaced along the disability spectrum has important implications for interpreting the results of stroke treatment in clinical trials and in health policy models. In the past, when evaluating socioeconomic consequences of trial results and policy changes, an equal-interval distance between mRS levels had to be assumed.²¹ Now, actual DW value distances can be employed, enabling more accurate assessment of the human impact of stroke treatments and public health policies.

Sometimes it is desirable to reduce the full seven level mRS to fewer categories to simplify analysis. Hitherto, the assortment of the seven mRS levels into fewer categories was performed in an intuitive, somewhat arbitrary manner. Supraordinate category formation may now be guided by the natural clustering of the formally derived, quantitative DWs observed in this study. For example, thrombolysis trialists have reduced the mRS to four categories: mRS 0–1, mRS 2–3, mRS 4–5, and mRS 6.²² However, the clustering of DWs derived in this study indicates that a more natural tetrachotomization of the Rankin Scale, combining mRS levels with similar DWs, would be mRS 0–1, mRS 2–3, mRS 4, and mRS 5–6. Another approach to tetrachotomization was taken by the CLOTBUST trialists, who reduced the mRS to: 0–1, 2, 3–5, and 6.²³ The DWs here derived suggest the CLOTBUST approach yields a distorted tetrachotomy, with the single mRS 3–5 category spanning 70% of the disability spectrum, and the remaining 3 categories subdividing only the remaining 30%. This might be useful for a clinical trial aimed at detecting only transitions to, from, and among a range of good outcomes, but will miss important changes in outcome in the moderate to severe range. An additional approach in stroke trial analysis is to reduce the mRS to 6 levels, rather than 7, by collapsing mRS 5 and mRS 6 into a single worst outcome category.¹⁷ This study supports this hexachotomization approach, indicating little difference along the disability dimension among mRS 5 and 6 outcomes.

This study has limitations. The expert panel included stroke physicians from North America and Asia, but not other regions of the world. The panel included representatives of three specialties/subspecialties (Stroke Neurology, Rehabilitation Neurology, Emergency Medicine) but not additional specialties involved in stroke care, nor nurses or allied health personnel. Some panelists who rated mRS 5 equivalent to death may have actually considered extreme poststroke disability to be worse than death, but the PTO task does not permit scoring an outcome state as worse than death.

Stroke is the only disease that is both a leading cause of death and a leading cause of disability worldwide. While stroke death rates are inherently easy to enumerate, the total burden of poststroke disability has been challenging to quantify. The availability of DWs for each mRS level now permits direct comparison of each poststroke outcome state with the outcomes of hundreds of other diseases in the WHO-GBDP and the expression of stroke burden in different populations using the uniform metric of DALYs lost. Since DALYs are a continuous outcome measure, stroke treatments and health policy effects can now be examined not only with dichotomized and ordinal statistical methods, but also with the often more powerful methods appropriate to continuous data.