Reliability of the modified Rankin Scale applied by telephone

Abstract

We aimed to evaluate the reliability of the modified Rankin Scale applied telephonically compared with face-to-face assessment in clinically stable hospitalized patients with acute stroke. One hundred and thirty-one patients were interviewed twice by 2 certified nurses (unstructured interview). Half of the patients were randomized to be interviewed by telephone followed by the face-to-face assessment, and half in the reverse order. The median value of the modified Rankin Scale score was 4 (first to third interquartile range 3–5) by telephone as well as by face-to-face assessment (P=0.8). The weighted kappa between the two methods was 0.82 (95% confidence interval: 0.77–0.88). Sensitivity of the telephone assessment was lower for scores 2 and 3 (17% and 46%, respectively) than for the other scores (range 67–90%). Telephone assessment of stroke disability with the modified Rankin Scale is reliable in comparison to direct face-to-face assessment.

Introduction

The modified Rankin Scale (mRS) is a measure of global disability that has been widely used to assess outcome after stroke. The scale consists of six grades from 0 (no symptoms) to 5 (severe disability); 6 indicates death.¹ Few studies have evaluated telephone assessment of the mRS.^2–4 These studies mostly showed high agreement between telephone and face-to-face (f-to-f) assessment. We aimed to evaluate the reliability of the mRS applied by telephone compared with f-to-f assessment in clinically stable patients with stroke.

Materials and Methods

We assessed 157 hospitalized patients with stroke consecutively admitted to the Department of Neurology, Maggiore della Carità Hospital of Novara (Italy) during a 9-month period. This is a first referral hospital in Northern Italy. The study was approved by the hospital Ethics Committee. Inclusion criteria were: diagnosis of stroke, age 18 years or over, clinical stability, and written informed consent. When possible, informed consent was obtained from the patient, or otherwise from the caregivers. Clinical stability was defined as no worsening of the National Institute of Health Stroke Scale for three consecutive days. Each patient was interviewed twice by 2 certified nurses with identical training in the use of stroke scales. Half of the patients were randomized to receive the telephone interview followed by the f-to-f interview, and half were evaluated in the reverse order. Interviews were unstructured. The sequence of the interviewers and of the modalities was randomized; the randomization list was concealed, and the 2 nurses were blind to each other's scores. The 2 interviews were administered with a 2-day interval; this was regarded as long enough to ensure that the first responses would be forgotten and short enough to ensure that the clinical condition would not change. The caregivers of patients unable to be interviewed by telephone were interviewed as proxy respondents. Telephone calls were made from a room outside the ward to the telephone in the patient's room.

Data were analyzed with SAS software.⁵ Weighted kappa (wK) statistics were used to evaluate agreement (PROC FREQ). Wilcoxon's signed-rank test and Wilcoxon's rank-sum tests were used were appropriate.

Results

Nineteen patients died before reaching clinical stability. Seven [mean age 76.6 years; standard deviation (SD) 9.9] refused to give their consent. We investigated 131 hospitalized patients with a mean age of 73.9 years (SD 13.3). Nineteen patients were diagnosed with hemorraghic stroke and 112 ischemic. The mean interval from stroke onset to scoring was 7.5 days (SD 5.2). An interview with caregivers was needed for 45 patients (34.4%).

The median value of the mRS score was 4 (first to third interquartile range, 3 to 5) by telephone and 4 (first to third interquartile range, 3 to 5) for f-to-f assessment (P=0.8). There was no significant difference (P=0.68) in median scores of the telephone mRS when this modality came first (n=68; median score 4; interquartile range, 3 to 5) or second (n=63; median score 4; interquartile range, 3 to 5). There was no statistical difference in the median score of the f-to-f mRS assessment (P=0.98) when this modality came first (n=63; median score 4; interquartile range, 3 to 5) or second (n=68; median score 4; interquartile range, 3 to 5). The frequency distribution of the scores assigned in telephone and f-to-f interviews is reported in Table 1. The mRS score attributed by the two methods was the same for 97 patients (74.0%). In 16 cases, telephone rating was lower than f-to-f, and in 18 cases higher. A difference of one level was observed in 31 cases and of 2 levels in 3 cases. The proportion of agreements differed according to mRS score: 67% for score 0 (2 of 3 patients), 82% for score 1 (18 of 22), 17% for score 2 (1 of 6), 46% for score 3 (5 of 11), 72% for score 4 (36 of 50), and 90% for score 5 (35 of 39). There was excellent agreement between the two methods: wK was 0.82 (95% CI: 0.77–0.88) for all patients. The wK for the 86 self-respondent was 0.81 (0.74–0.87), whereas it was 0.63 (0.43–0.84) in the 45 patients interviewed with the caregivers.

Table 1. Modified Rankin Scale: cross-tabulation of scores assessed by telephone and face-to-face interview.

Telephone assessment	Face-to-face assessment						Total
	0	1	2	3	4	5
0	2	1	0	0	0	0	3
1	1	18	4	1	0	0	24
2	0	2	1	1	1	0	5
3	0	1	1	5	4	0	11
4	0	0	0	4	36	4	44
5	0	0	0	0	9	35	44
Total	3	22	6	11	50	39	131

Discussion

This study shows that telephone assessment of stroke disability with the mRS is reliable in comparison to f-to-f assessment.

Most patients received the same score by the two methods and there was excellent agreement between the two methods, as in three other studies that found a range of wK from 0.71 to 0.82.^2–4 A fourth study (published only as a letter) found less agreement (k=0.30–0.38); however, the fact that the interviewers had different professional backgrounds could partly explain the disagreement.⁴

We applied the mRS to hospitalized consecutive patients with a wide severity range. The sensitivity of telephone mRS was lowest in the mid-range of the scale (mRS=2 and 3), where it correctly rated less than 50% of patients. This was also evident in the studies by Newcommon and Candelise et al.^2,4 Although it could be related to our small frequencies and the difficulty of estimating ability to live independently and to walk without assistance when one is still hospitalized, the lowest sensitivity of telephone assessment for mRS scores 2 and 3 should be considered in clinical trials and field stroke studies using this interview method.

The major limit of this study is that our setting (acute hospitalization) is not the optimal setting for using mRS, and it might not be possible to immediately transfer our results to interviews obtained in a home setting, where the telephone mRS is most often used.