Oswestry Disability Index: Is Telephone Administration Valid?

Abstract

Background:

The Oswestry Disability Index (ODI) is among the most widely used patient reported outcome measures for the assessment of spinal conditions. Traditionally, the ODI has been administered in outpatient clinics on a face-to-face basis, which can be expensive and time consuming. Furthermore, the percentage of patients lost to clinical follow-up is high, particularly after 2-5 years. Thus, telephonic administration of the ODI, if valid, could be a convenient way of capturing patient outcomes and increasing follow-up rates. The objective of this study was to validate telephonic administration of the ODI compared to face-to-face administration.

Methods:

A convenience sample of individuals with and without back pain in an academic medical center were recruited for this study. Face-to-face administration of the ODI was completed and retested 24 hours later via phone. Test-retest reliability was determined by calculating the intraclass correlation coefficient.

Results:

22 individuals completed the ODI questionnaire face-to-face, then via telephone 24 hours later. There was a mean 2% (± 3) intra-rater ODI score difference (range: 0% to 12%). The intraclass correlation coefficient overall was 0.98 (95% CI: 0.96, 0.99, p<0.001) with a range of 0.95 to 1.0, revealing near-perfect test-retest reliability.

Conclusions:

Administration of the ODI questionnaire over the phone has excellent test-retest reliability when compared to face-to-face administration. Telephone administration is a convenient and reliable option for obtaining follow-up outcomes data.

Clinical Relevance:

Telephonic administration of the ODI is scientifically valid, and should be an accepted method of data collection for state-level and national-level outcomes projects.

Introduction

The measurement of disability is a critical factor in caring for patients with spinal conditions. Although multiple measurement tools exist, the Oswestry Disability Index (ODI) is among the most widely used.¹ Traditionally, the ODI was collected as a patient-administered questionnaire during face-to-face clinic visits.¹ However, administration in clinic requires time and clinical resources. Furthermore, patients may be unable or unwilling to return to clinic, and the percentage lost to clinical follow-up is high, particularly by 2-5 years.^2,3 Thus, telephonic methods of administering ODI data forms would be helpful both in terms of convenience for the patient and provider, and also in increasing follow-up participation rates. Although telephone administration of the ODI has been reported,⁴ it has not been extensively validated.

Recently, increased emphasis has been placed on public reporting of patient-outcomes data.⁵ The goal of these initiatives is to facilitate practice-based learning, transparency, and shared decision making for patients. However, there exist both inducements for success and penalties for poor results, and physicians and hospitals risk financial loss if they fail to comply.⁵ Specifically in Minnesota, the Minnesota Community Measurement organization (MNCM) has begun collecting and publicly releasing patient-reported outcome measures (PROM) from health care centers across the state.⁶ Furthermore, they provide a data collection guide which outlines acceptable forms of administration for various PROM tools, including the ODI.⁷ In recent editions, telephonic administration was deemed not acceptable due to a lack of validation.⁷

Thus, further validation of the ODI administered via telephone is needed. Successful validation of this tool should be useful to promote its utility for payors and governmental organizations, and should provide evidence for a more convenient method of administering the survey for clinicians.

Methods

Patient Selection

A convenience sample of 22 individuals with and without back pain in an academic medical center was recruited to participate in this study. Convenience sampling is a type of non-probability sampling that involves the sample being drawn from the population close at hand, and the only inclusion criteria is a willingness to participate. Patients with symptomatic back pain (10 of 22) were recruited from the senior author’s clinic during a scheduled visit for a spinal condition. The remaining study subjects (12 of 22) were asymptomatic support staff who worked in the clinic and agreed to participate. Mean age was 48.6 years (standard deviation ± 14.7 years) and most subjects were women (73%). Participants were asked to complete the ODI questionnaire (ODI v 2.1a) in standard fashion.1 Twenty-four hours later, the test was re-administered via telephone by a single individual. The ODI questionnaire asks patients about symptoms they are experiencing “now.” Thus, the 24-hour interval was selected in order to remain as close as possible to the observation period for assessing those current symptoms.

Administration, Scoring, and Data Analysis

Both telephone and in-person administration were performed in accordance with the MAPI Research Trust administration guidelines.8 Test-retest reliability and intra-class correlation coefficient (ICC) were calculated using SPSS Statistics Desktop, version 20.0.0 (IBM Corporation 2011, Armonk NY, USA).

Results

Table 1 reports the patient demographic results in this study. The mean scaled score for both the written and telephone versions was 25% (range 0-70% for written and 0-76% for telephone). There was a mean 2% (± 3) intrarater ODI score difference (range 0% to 12%) (Figure 1). The intra-class correlation coefficient overall was 0.98 (95% CI: 0.96, 0.99, p<0.001) with a range of 0.95 to 1.0, revealing near-perfect test-retest reliability.

Table 1.

Patient Demographics

Sample Size	n = 22
Female gender	16 (73%)
Scheduled clinic visit	10 (45%)
Age	48.6 ± 14.7 years

Figure 1.

Oswestry Disability Index scores obtained via written and telephone administration.

Discussion

Maintaining clinical follow-up rates beyond 6-12 months is particularly difficult, and even well conducted clinical trials with substantial administrative support frequently report less than 50% follow-up rates beyond 5 years.³ Thus, physicians need methods of collecting patient outcome data that do not require face-to-face clinical visits, and telephonic administration holds substantial promise in that regard. However, given that many of the original PROM measures were designed to be conducted in-person, administration over the telephone must be validated before the method gains widespread acceptance.⁷

In this study, we found very high reliability for telephonic administration of the ODI with an ICC of 0.98, suggesting near perfect test-retest reliability. In the original publication of the ODI by Fairbank et al., reliability was assessed with repeated written administrations 24 hours apart (n=22, r = 0.99).⁸ With an ICC of 0.98, our study suggests that telephonic administration is just as effective.

Two primary advantages exist for administration of the ODI via telephone. First, it should be easier to obtain clinical follow-up if telephonic administration is an accepted method of data collection. In a study evaluating the performance characteristics of the verbal QuickDash, follow-up could not be obtained for 17% of patients for the written survey, but that number dropped to 0% for the verbal method.⁹ Second, telephonic administration is more convenient for patients. In a study by Bokshan et al., which investigated the validity of spine based PROMs, the majority of patients (57%) preferred the telephone over in-person administration (29%, with 14% expressing no preference).⁴

A limitation to this study is that all of the patients had the ODI administered face-to-face prior to the telephone attempt, so they were familiar with the written format. De novo telephone testing may not have the same reproducibility. Other studies have used a wider interval between the written and telephone versions of the survey to try and minimize this recall bias.⁴ However, the ODI specifically asks patients about symptoms they are experiencing “now.” Thus, we felt that a narrow window between the two survey administrations was most appropriate. Furthermore, telephone administration, while more convenient for the patient, may increase the burden on the physician’s office staff.

Conclusions

Overall, telephonic administration is a well-recognized and validated method for multiple other PROMs across multiple medical disciplines,^10,11 as well as for other PROMs in spine.4 Our data adds to this bulk of literature supporting this method of data collection. Telephonic administration of the ODI is scientifically valid, and should be an accepted method of data collection for state-level and national-level outcomes projects.