Validation in the Cross-Cultural Adaptation of the Korean Version of the Oswestry Disability Index

Abstract

Disability questionnaires are used for clinical assessment, outcome measurement, and research methodology. Any disability measurement must be adapted culturally for comparability of data, when the patients, who are measured, use different languages. This study aimed to conduct cross-cultural adaptation in translating the original (English) version of the Oswestry Disability Index (ODI) into Korean, and then to assess the reliability of the Korean versions of the Oswestry Disability Index (KODI). We used methodology to obtain semantic, idiomatic, experimental, and conceptual equivalences for the process of cross-cultural adaptation. The KODI were tested in 116 patients with chronic low back pain. The internal consistency and reliability for the KODI reached 0.9168 (Cronbach's alpha). The test-retest reliability was assessed with 32 patients (who were not included in the assessment of Cronbach's alpha) over a time interval of 4 days. Test-retest correlation reliability was 0.9332. The entire process and the results of this study were reported to the developer (Dr. Fairbank JC), who appraised the KODI. There is little evidence of differential item functioning in KODI. The results suggest that the KODI is internally consistent and reliable. Therefore, the KODI can be recommended as a low back pain assessment tool in Korea.

INTRODUCTION

Outcome measurements of patients undergoing treatment for spinal disorders are essential. Outcome measurements on the treatment of spinal disorders should include an assessment of the benefits as well as the risks of complications that are associated with new spinal treatments. Orthopedic evaluation methods for the outcome measurements, such as radiographic evaluations or other technical studies, are less significantly related with outcomes and are most relevant with respect to functional status and symptoms (1, 2). For this reason, patient-oriented measurement instruments have been developed, which measure the quality of life for the patients. The measurement instruments consist of questionnaires, most of which are administered by the patients themselves.

Society has been changing as a result of the increase in multicultural populations. Diversity is present among many cultures. Since languages are highly associated with cultures, it is important to address cross-cultural assessment in health outcome measures. When an assessment tool is translated into a different language, the latter language, which includes cultural differences, may express the experience of low back pain differently from that expressed in the original version of the assessment tool. The importance of outcome measurements has been recognized among spinal surgeons in Korea over the last decade. However, the most published instruments for outcome measurements are in English, and theses instruments needed to be translated into Korean, considering that Korean cultural backgrounds and language are different from those of English speaking countries.

The Oswestry Disability Index (ODI) measures the extent to which a patient's functional level is limited by low back pain. The aim of this study was to assess the validation and reliability of the cross-cultural adaptation of the Korean version of the Oswestry Disability Index (Version 2.0) (3). The problems affecting cross-cultural outcome measurements included problems of illiteracy, concepts, and language. The authors used the process of cross-cultural adaptation, suggested by Beaton et al. (4) and Guillemin et al. (5), in order to obtain semantic, idiomatic, experimental, and conceptual equivalencies in translation. We utilized these processes to validate our adaptation of the Korean version of the Oswestry Disability Index (KODI).

MATERIALS AND METHODS

Korean population has a homogenous language and there is no significant variation in spoken and written Korean, although slight regional accents exist. For the current study, we used the standard Korean language (designated by the Korean Government) for translation. Four independent translators, who were bilingual (in both Korean and English at a native speaker level), were utilized: two for the forward translation and two for the backward translation. They made the independent translations without knowledge of the purpose of this study.

To validate the translation of the original version (English) of the ODI into Korean, the following processes were performed: forward translation, synthesis of the translation, backward translation, expert committee review, pre-test of the pre-final version, reliability, validation, and submission of the final Korean version ODI to the developer.

First, two independent translators translated the English (original) version of ODI into Korean, literally. The translators were unaware of the purpose of this study.

Two translators and three orthopedic surgeons (who specialized in the treatment of back pain and were familiar with the ODI scales) compared the two Korean translated ODIs (completed previously by two other independent translators) and formed a consensus. During this process, we found that the meaning of the word "heavy", from question #3, that was used to describe lifting, needed to be clarified for several patients. For example, when the patients answered question #3, which dealt with lifting weight, several patients asked for the exact amount of kilograms that were necessary to be considered heavy; and we encouraged the Korean patients to evaluate whether or not the objects were heavy to them, based on their daily life experiences.

With the Korean translation version, two independent translators, who did not have any medical information about the patient's backgrounds about the ODIs, made the back translations. The back translators were bilingual (in both Korean and English) and were unaware of the original English version and the application of the questionnaires. Two Korean translated ODIs were translated back into English independently. The two independent back translators were in agreement with the back-translation in the final version, except for the words "travel" and "journey" in question #10. There was no difference between these two words in Korean.

The committee consisted of two translators, two back translators, three orthopedic spinal surgeons, a statistics specialist, a Korean linguistics and literature specialist, and a English linguistics and literature specialist. The committee reviewed the forward translation, the backward translation, and the original version of the ODI. To resolve discrepancies between the forward and the backward translations, the committee decided to repeat the translation and then back translation process simultaneously. In order to adjust for inappropriately translated items, the committee ensured that the introductions and explanations about the ODI questionnaires were translated correctly to replicate the measurement results between the Korean version ODI and the original (English version) ODI. With these processes, idiomatic and conceptual equivalences between the Korean version and the English version of the ODI scales were obtained.

For pre-test, thirty-two randomly selected patients with low back pain completed the KODI. The 32 patients included 14 men and 18 women (average age, 47.91±16.81 yr; age range, 18-87 yr). The questionnaires were administered in the waiting room of the outpatient clinic, before the physicians evaluated the patients. The patients were asked to document any problems during the questionnaire administration. Although a few patients hesitated to answer question #8, which included references to their sexual lives, they all eventually answered the question. Korean people, in general, are much more reluctant about expressing their sexual lives openly (even in a questionnaire) than are people in Western societies. On the basis of the results of the pre-test, the committee developed the final version of the KODI (refer to the Appendix).

One hundred sixteen Korean patients with chronic low back pain participated in the study. These patients were selected randomly from three different orthopedic clinics (Ajou Medical Center, Ewha Medical Center, and Severance Hospital). Among them, 44 patients were men and 72 were women. All the participants were examined carefully and the diagnoses were confirmed by clinical and radiographic examinations. Each patient completed the KODI. The average age of the patients was 47.78±15.64 yr and the age range was between 20 and 73 yr.

For the reliability, data were analyzed using the SAS program (Version 8.1). The descriptive statistics were used for the demographic analysis. To measure of internal consistency we adapted Cronbach's alpha coefficients, which is the most common form of reliability. Cronbach's alpha coefficients were computed in order to measure the internal consistency of the KODI, which included 10 questions. The internal consistency reliability is an index that represents the relationship among the items in a test. Cronbach's alpha coefficients range from 0 to 1. The confidence interval of Cronbach's alpha coefficients was calculated with Bootstrap method.

Test-retest reliability, which is administering the same test to the same subjects at two points in time, is used to measure of stability over time. In order to measure how consistently the examinees responded to the KODI, the test-retest reliability was assessed, which is the measure of stability. For the test-retest reliability, having some of the patients take the KODI twice in separate situations, allowing a sufficient time interval to minimize a memory or practice effect (6), completed this. Thirty-two patients were asked to answer the KODI twice, and were given a time interval of 4 days between the tests. During the 4-day interval, none of the patients had any interventional treatment.

When we computed both Cronbach's alpha coefficients and the test-retest correlation reliability of the KODI, we first computed the reliability with all the items; and then we deliberately excluded question #8 because we several patients hesitated to answer the question.

The validation of a test refers to the extent to which the test measures the attributes of the respondents in a population that the test is supposed to measure, such as back pain and physical function. In order to increase the validation of the KODI, we adopted four procedures: forward translation, synthesis of the translation, backward translation, and expert committee review.

The entire process of the validation, cross-cultural adaptation, and reliability tests of the ODI were reported to the developer (Dr. Jeremy Fairbank); and the KODI was submitted then to the developer.

RESULTS

Table 1 shows the mean scores of the Oswestry Disability Index. The internal consistency reliability (Cronbach's alpha coefficients) for the KODI reached 0.9168. Test-retest correlation reliability was 0.9332 as illustrated in Table 2, 3 shows that the internal consistency reliability for the Korean translation of the ODI, excluding question #8, reached a Cronbach's alpha coefficient of 0.9041.

Table 1.

Internal consistency reliability testing for the Korea translation of the Oswestry Disability Index

CI, Confidence Interval.

Table 2.

Test-retest correlation reliability for the Korean translation of the Oswestry Disability Index

CI, Confidence Interval.

Table 3.

Internal consistency reliability testing for the Korean translation of the Oswestry Disability Index (except for question 8, which was omitted)

CI, Confidence Interval.

DISCUSSION

Disease specific questionnaires that assess disability and measure the outcomes of applied treatments are necessary for patients with low back pain in order to improve clinical treatment and research. The questionnaires that measure the low back pain patient's perspectives with respect to symptoms and function have been developed mainly in the English-speaking scientific community. In Korea, there has been few reliable and valid assessment tools on low back pain (7), and little research has been performed about assessing low back pain.

An increasing number of recent publications reflect the growing importance and interest in the process of cross-cultural adaptation when the English version of a measurement instrument (e.g., the ODI) is translated into a different language (e.g., Korean). The development of disease-specific questionnaires and the analysis of symptoms that are shared by specific pathologies are not complicated; whereas the cross-cultural adaptation of questionnaires originally designed for English speaking countries requires a complex analysis in both psychological and cultural aspects. This cross-cultural adaptation is required to measure the same phenomena in different cultures (8). The cross-cultural adaptation of the foreign language version of measurement instruments is a prerequisite for the investigation of cross-cultural differences. The cross-cultural adaptation described by Guillemin et al. (5) in 1993, with respect to non-specific questionnaires, is a challenging process and more than just a mere linguistic translation.

Subjective complaints about the symptoms of low back pain were investigated by means of specific questions that described the personal complaints of the patients, without any fundamental interference from cultural influences. However, the appropriate use of the original (English) ODI questionnaires depended on how well and appropriately they were adapted to a different language and culture (e.g., Korean). Guyatt et al. (9) suggested that patients should be given access to their pre-treatment questionnaires prior to completing their follow-up questionnaires.

The culturally equivalent outcome measurements allowed researchers to perform low back pain studies reliably in culturally different countries. In order to maintain the original meanings of the questionnaires in the different countries was a time-consuming, but unavoidable, process. The current study included the validation and translation procedures of how the original (English) version of the ODI could be utilized as a low back pain outcome measurement for Korean patients, since the ODI is one of the most widely used low back pain outcome measurements. The ODI Version 2.0 was translated into Korean, because Dr. Fairbank, who is one of two ODI authors, recommended the use of version 2.0 in Fairbank and Pynsent's study of the ODI (3).

An inappropriate translation process may lead to incorrectly translated questionnaires, which may not contain the equivalent meanings of the original version (5, 10). In order to avoid the distortion of the translated (from English into Korean) questionnaires, a rigorous adaptation process was required (3, 11): specifically, independent forward and backward translators were hired to prepare the KODI. A committee for this rigorous adaptation process was arranged, which consisted of forward translators, backward translators, three orthopedic spinal surgeons, specialists of statistics, Korean linguistics and literature, and English linguistics and literature. The expertise of each committee member was utilized to finalize the adapted KODI. This process ensured that the translation procedure for the ODI was satisfactory and valid; and since the KODI was developed through such a rigorous adaptation process, it was a valid measurement of outcomes for Korean patients with low back pain.

Most of the translation processes were completed without any serious difficulties. There was minor difficulty in obtaining an answer for question #8 about the sexual activities of patients; because revealing personal information about one's sexual life is culturally forbidden in most Asian societies. As a results of these cultural differences, omission of the question regarding the sexual activity, may limit the comparability of the ODI measurements between different countries. Therefore, the question regarding sexual life was not omitted in the KODI in order to maximize its equivalence with the original version of the ODI.

Alpha coefficient, as introduced in Cronbach (12), is a frequently used statistic in empirical research involving with various items. Cronbach's alpha coefficient estimates the reliability of scale by determining the internal consistency of the test or the average correlation of items within the test (13). Cronbach's alpha coefficient ranges in value from 0 to 1 and may be used to describe the reliability of factors extracted from dichotomous and multi-point formatted questionnaires or scales. The higher the score, the more reliable the generated scale is. Nunnally has indicated 0.7 to be an acceptable reliability coefficient but lower thresholds are sometimes used in the literature (14). To ensure the internal consistency of the KODI, Cronbach's alpha coefficients were calculated from the 10 questions answered by 116 patients with low back pain. The internal consistency reliability of the KODI (Cronbach's alpha coefficient=0.9168) was similar to those reported by other authors who used the original version of the ODI (3).

Test-retest reliability indicates the extent to which the same results are obtained on repeated administrations of a given instrument when no change is expected. In the study of ODI, patients with chronic low back pain were tested twice at a 24-hr interval or even shorter period (15, 16). This may include a memory effect. If the test-retest interval is extended to 5 days, the correlation of scores decreases and, if retested after a week, the correlation decreases at the level of low reliability (17, 18). The disadvantage of increasing the time interval is that natural symptom fluctuation may also be an influence. In this study, 4-day time interval was chosen because, compared with shorter time intervals, it minimizes the possible memory effect and provides a more realistic view of the degree of score change that may occur for non-specific reasons. In addition, we investigated test-retest reliability, because we were interested in how consistently the examinees responded to the KODI at different times. To calculate the correlation of test-retest reliability, we utilized 32 patients, who were not included in computing the internal consistency reliability; and the patients used for the test-retest reliability were an independent sample from the total sample population (n=116) that was used for the correlation of test-retest reliability. The correlation of test-retest reliability was 0.9332.

The results of this study suggested that these translated questionnaires were assumed to be both reliable (Cronbach's alpha coefficient=0.9168 and test-retest correlation reliability=0.9332) and valid (by using the rigorous adaptation process). The KODI demonstrated psychometric properties of reliability tests that were similar to those presented in the original reports (11, 15, 16). This satisfactory psychometric property of the KODI suggested that the KODI was as reliable as the original (English) version of the ODI, and it was expected to be as reliable and valid a measurement tool in assessing the status (after treatment) of Korean patients with low back pain. In addition, the KODI will be a useful tool for future research on low back patients in Korea, since there has been no such measurement instrument for evaluating the conditions of these patients after treatment. The fully adapted Korean version of the ODI is provided in the appendix.

The results of this study demonstrated that the Korean version of the Oswestry Disability Index was cross-culturally adapted. The Korean version of the Oswestry Disability Index is a valid and reliable instrument for the assessment of Korean patients with low-back pain.