Translation, Cross-Cultural Adaptation, Validity, and Reliability of the Indonesian Version of the IKDC Subjective Knee Form

Abstract

Background:

No questionnaire is currently available for use in patients with anterior cruciate ligament (ACL) injuries in an Indonesian population. The most-used questionnaire in clinical research for these patients is the International Knee Documentation Committee (IKDC) Subjective Knee Form, as its psychometric properties are considered to be excellent.

Purpose:

To translate the IKDC into Indonesian and assess its validity for use in Indonesian-speaking patients with ACL injuries.

Study Design:

Cohort study (diagnosis); Level of evidence, 2.

Methods:

After a forward-and-backward translation procedure and cross-cultural adaptation, the validity and reliability of the questionnaire were investigated. The responses of ACL injury patients on 3 questionnaires, the Indonesian-IKDC (I-IKDC), 36-Item Short Form Health Survey, and Kujala Anterior Knee Pain Scale, were compared. Following consensus-based standards for the selection of health measurement instruments guidelines, construct validity, test-retest reliability, internal consistency, floor and ceiling effects, and measurement error were determined. The Bland-Altman method was used to explore absolute agreement.

Results:

Of 253 ACL injury patients, 106 (42%) responded to the invitation. Construct validity was considered good, as all predefined hypotheses on correlations between the I-IKDC and other scores were confirmed. Reliability proved excellent, with a high test-retest correlation (intraclass correlation coefficient = 0.99). Bland-Altman analyses showed no systematic bias between test and retest. Internal consistency was good (Cronbach α = .90). There were no floor or ceiling effects. Standard error of measurement was 2.1, and the minimal detectable change was 5.8 at the individual level and 0.7 at the group level.

Conclusion:

The I-IKDC, as developed, appeared to be a good evaluation instrument for Indonesian patients with ACL injuries.

The overall age- and sex-adjusted annual incidence of anterior cruciate ligament (ACL) tears is 74.6 per 100,000 person-years.²² Although exact data are lacking about this incidence in Indonesia, the number of ACL implants used in ACL surgery increased by 42% in 2019 as compared with 2018 in Indonesia (1575 implants in 2018 vs 2236 in 2019, based on data from the 3 main ACL-implant companies in Indonesia; R. Deviandri, MD, unpublished data, 2021). The goal in treating ACL injuries is to regain knee function, preserve long-term knee-related health, return to the activities of daily living, and return to work and sporting activities.^21,23,24 One way to improve management of ACL injuries is to evaluate treatment outcomes from a patient’s perspective. Therefore, reliable and valid patient-reported outcome measures (PROMs) are needed.

In 1987, specialists of the AOSSM and the European Society of Sports Traumatology, Knee Surgery and Arthroscopy compiled the International Knee Documentation Committee (IKDC) Subjective Knee Form to develop a standardized international knee documentation system to evaluate the outcome of randomized controlled trials in patients with knee injuries. In 1998, the IKDC was revised, and that version was published in 2001.¹¹ Tanner et al²⁵ reported that the IKDC is the questionnaire that contains the most important questions on symptoms and disabilities of patients with ACL, meniscal, and degenerative knee pathology. Since then, the IKDC has been used widely and translated into many languages in different cultural settings, but a version for Indonesian-speaking patients has yet to be developed and validated.

The aim of this study was to translate and adapt cross-culturally the English version of the IKDC Subjective Knee Form into an Indonesian version (I-IKDC) and to analyze the validity and reliability of this questionnaire for individuals with ACL injuries. We hypothesized that the I-IKDC would be both valid and reliable for use in the Indonesian-speaking ACL population.

Methods

The study protocol was approved by an institutional review board, and patients provided informed consent before participating in this study. The translation procedure was approved by the AOSSM scientific board and the developers of the IKDC.

Translation Procedure

The I-IKDC was developed according to the forward-and-backward translation guidelines originally proposed by Guillemin et al⁹ and later revised by Beaton et al.¹ This procedure consists of 6 stages: initial translation, translation synthesis, back translation, committee review, pretesting, and submission and appraisal of all written reports to the committee. Initially, 2 independent Indonesian individuals with a good command of the English language completed a conceptual and literal translation of the IKDC into Indonesian (T1 and T2). Next a synthesized version (T12) was produced, based on the comments of the 2 initial translations, and back-translated into English by 2 independent professional translators (BT1 and BT2) to check for inconsistencies with the original English version. The expert committee, consisting of 3 sports medicine orthopaedic surgeons, 1 methodological expert, and 1 translator, also reviewed this Indonesian translation. After being reviewed by the committee, the questionnaire was edited to its prefinal version by one of the authors (R.D.). This prefinal translated version was then distributed to 10 patients with ACL injuries. One of the researchers (R.D.) documented any difficulties that patients experienced while completing the questionnaire. The documentations were reviewed and used to modify the questionnaire into the final version of the I-IKDC.

Patients and Procedure

Between April and July 2020, a total of 253 randomly selected patients were invited by email to participate in the validity and reliability study. All participants were Indonesian patients who had been treated for ACL rupture with ACL reconstruction or structured rehabilitation between January 2015 and March 2020 at a hospital in Indonesia and had provided informed consent to participate. Next, patients were sent information and 2 sets of questionnaires (parts A and B) via email. Part A was sent in the first week and included the I-IKDC and 2 complementary questionnaires, the Medical Outcomes Study 36-Item Short Form Health Survey (SF-36) and the Kujala Anterior Knee Pain Scale. To explore test-retest reliability, participants were asked to complete the I-IKDC again after 1 week (part B). Patients were instructed to fill out each set of questionnaires and reply immediately to send them back. To determine whether health status and knee function remained indeed stable between the completion of parts A and B, patients were explicitly asked at the start of the part B questionnaire, “Has your status changed since filling out the initial questionnaire?” The 3 possible responses were (1) no; (2) yes, the problem changed for the better; and (3) yes, the problem changed for the worse. Only patients indicating no change in their knee function were included in the test-retest analysis; patients who returned both questionnaires on the same day or more than 1 month apart were excluded.

Patient-Reported Outcome Measures

The IKDC is a region-specific PROM. It includes 18 questions and was designed to measure symptoms, function, and sports activity in patients with a variety of knee conditions, including ligamental and meniscal injuries, articular cartilage lesions, and patellofemoral pain. The form is scored by summing the scores for the individual items and then transforming the score to a scale that ranges from 0 to 100, with 100 indicating the absence of symptoms and higher levels of functioning.¹

The Kujala Anterior Knee Pain Scale is a disease-specific outcome measure of anterior knee pain that documents responses to questions about 6 activities specifically associated with anterior knee pain: whether there is pain while walking, stairs, squatting, running, jumping, or during prolonged sitting with the knee in flexion; whether there is limping, swelling, or subluxation of the patella; and whether there is a need for a walking aid. There are also questions about amount of atrophy in the quadriceps muscle, flexion deficiency, and pain. The Kujala scale consists of 13 questions, with a total score ranging from 0 to 100 and the highest value indicating the best score.¹⁴

The SF-36 is a generic score used to establish a health profile. It consists of 8 scaled scores, where each subscale is transformed directly into a scale from 0 to 100 to identify the patient’s physical and mental state: physical functioning (PF), physical role functioning (PR), bodily pain (BP), general health perceptions (GH), vitality (VT), social function (SF), emotional role functioning (ER), and mental health (MH). In addition, the sum of the PF, PR, BP, and GH subscales generates a physical component summary score (PCS), and the sum of the VT, SF, ER, and MH subscales generates a mental component summary score (MCS). Standardized scores range from 0 to 100, with higher scores indicating better health status.²⁷

Validity

Validity is the extent to which a score measures what it is supposed to measure: that is, whether it has the intended interpretation. The construct validity of the I-IKDC was assessed by determining the correlation with the Kujala score plus physical and mental domains of the SF-36.

According to the consensus-based standards for the selection of health measurement instruments (COSMIN) guidelines,¹⁸ predefined hypotheses were formulated about the magnitude of the associations between the I-IKDC and subscales of the SF-36 and Kujala score. As both I-IKDC and Kujala scores were knee scales and more specific for abnormalities of the knee, we hypothesized that the I-IKDC would correlate better with the Kujala score than the SF-36. The correlation between the Turkish version of the IKDC and the Kujala score was 0.89.⁸ Based on the correlations found in that study, correlations of 0.8 or higher between the I-IKDC and the Kujala score were hypothesized. Irrgang et al¹¹ showed a correlation of 0.66 between the original IKDC and the PCS subscale of SF-36 and 0.16 between the original IKDC and the MCS subscale of SF-36. Metsavaht et al¹⁷ showed a correlation of 0.51 between the Brazilian Portuguese version of the IKDC and MCS subscale of SF-36. Based on this, it was hypothesized that the correlations of 0.6 or higher between the I-IKDC and the PCS of SF-36 and those SF-36 subscales that measure BP and physical function (PF, PR). Further, it was hypothesized that the correlation between the I-IKDC and the MCS of SF-36 would be lower than 0.6 as well as between the I-IKDC and all SF-36 subscales that measure general health and social and mental function (GH, VT, SF, ER, MH). The IKDC was developed to measure PF rather than social and/or emotional aspects; therefore, lower correlations were expected between the I-IKDC and the MCS than between the I-IKDC and the PCS. Construct validity can be considered good when at least 75% of the hypotheses are confirmed.²⁶

Floor and Ceiling Effects

The occurrence of floor and ceiling effects was assessed. These effects are considered to be present if more than 15% of respondents achieve the lowest or highest possible score.²⁶

Reliability

Reliability represents the extent to which individuals can be distinguished from each other despite measurement errors. Following the COSMIN guidelines,¹⁸ reliability was assessed in terms of internal consistency, test-retest reliability, and measurement error. Internal consistency refers to the extent to which subscales of a questionnaire are related, test-retest reliability concerns the extent to which patients’ scores are the same for repeated measurements, and measurement error is a measure of systematic error of a patient’s score that is not caused by actual changes in the measured construct. The Bland-Altman method was used to explore absolute agreement, which reflects the amount of agreement in repeated measurements.²

Statistical Analysis

The characteristics of the study population and scores on the questionnaires are described using means and standard deviations or frequencies and percentages. To determine construct validity, as variables showed normal distribution, the Pearson correlation coefficient (r) was calculated between the scores on the I-IKDC and the other questionnaires. To determine internal consistency, Cronbach α was calculated.⁵ Values between .70 and .95 are considered to indicate good internal consistency.²⁶

The intraclass correlation coefficient (ICC) between test and retest I-IKDC scores was calculated to assess test-retest reliability.¹⁸ Values <0.50, 0.50 to 0.75, 0.76 to 0.90, and >0.90 are indicative of poor, moderate, good, and excellent reliability, respectively.¹³ Standard error of measurement (SEM) and minimal detectable change (MDC) were calculated to assess measurement error. SEM was calculated by multiplying the pooled standard deviation by Ö(1 – r), where r is the ICC.⁶ The MDC at the individual level (MDC_ind) was calculated using the formula 1.96 × SEM × Ö2 and at the group level (MDC_grp) by dividing MDC_ind by Ön.²⁶

Absolute reliability was assessed using Bland-Altman plots: When 0 was in the 95% CI of the mean difference between the first and second administration of the IKDC, no systematic bias was present. The 95% limits of agreement (LOA) were calculated with the formula mean difference ± 1.96 × SD_diff, where SD_diff is the standard deviation of the mean difference between the first and second administration of the IKDC.² Statistical analyses were performed using SPSS Statistics Version 26.0 (IBM), with level of significance set at 5%.

Results

Translation and Cross-Cultural Adaptation

Some cultural adaptations were made during the translation procedure. The word “skiing” was changed to “badminton” in questions 1, 5, 7, and 8, as the former is not a typical sport in Indonesia. The word “squatting” was changed to “squat while prayer,” which is quite common among the Indonesian population.

Patient Characteristics

Of the 253 patients, 106 (42%) responded to the invitation; 75 (70.8%) of these 106 patients filled in and returned 2 complete sets of questionnaires. The remaining 31 patients (29.2%) returned only part A. No patients reported a better or worse function of the operated knee when they filled out the second questionnaire. None of the patients were excluded because of missing data. Data of 106 patients were used for validity analyses and internal consistency, and data of 75 patients were used for determining test-retest reliability. Most of the patients were nonathlete men with a mean age of 30.5 ± 9.2 years. Mean scores of the first and second assessments of the I-IKDC were 66.4 ± 18.8 and 66.8 ± 19.1, respectively. Demographic patient characteristics are described in detail in Table 1. Scores from the PROMs can be found in Table 2.

Table 1.

Demographic Patient Characteristics (N = 106)^a

Characteristic	Mean ± SD or n (%)
Age (y)	30.5 ± 9.2
Sex
Male	75 (70.8)
Female	31 (29.2)
Affected side
Right	60 (56.6)
Left	46 (43.4)
Both	0
Diagnosis
Isolated ACL injury	103 (97.2)
Multiligamentous injury	3 (2.8)
Activity level
Athlete	14 (13.2)
Nonathlete	92 (86.8)
Activity type at injury
ADL	11 (10.3)
Sport	73 (68.9)
Traffic-related accident	20 (18.9)
Work	2 (1.9)

^aACL, anterior cruciate ligament; ADL, activities of daily living.

Table 2.

Patient-Reported Outcome Measures^a

Measure	Mean ± SD
I-IKDC
Part A	66.4 ± 18.8
Part B	66.8 ± 19.1
Kujala Anterior Knee Pain Scale	81.6 ± 14.9
SF-36
Physical functioning	68.5 ± 26.6
Physical role functioning	35.8 ± 41.9
Bodily pain	72.1 ± 24.2
General health perceptions	67.5 ± 18.3
Vitality	65.1 ± 20.6
Social function	68.6 ± 21.1
Emotional role functioning	46.5 ± 45.5
Mental health	69.9 ± 22.2
PCS	43.6 ± 9.7
MCS	35.2 ± 16.6

^aI-IKDC, Indonesian version of the International Knee Documentation Committee; MCS, mental component summary score; PCS, physical component summary score; SF-36, 36-Item Short Form Health Survey.

Validity

All predefined hypotheses on the magnitude of associations between the I-IKDC and either the Kujala score or the SF-36 were confirmed. The I-IKDC score showed strong correlations with the Kujala score (r = 0.85). Correlation coefficients higher than 0.6 were found between the I-IKDC and SF-36 PCS, PF, PR, and BP subscales (r = 0.66, 0.63, 0.61, and 0.69, respectively). Correlation coefficients 0.6 or lower were found between the I-IKDC and SF-36 MCS, VT, SF, ER, MH, and GH subscales (r = 0.59, 0.53, 0.42, 0.52, 0.58, and 0.52, respectively) (Table 3). The I-IKDC was related more strongly to the Kujala score than to the PCS score and more strongly to PCS than to MCS (Table 3). There were no floor or ceiling effects.

Table 3.

Pearson Correlation Coefficients Between I-IKDC, Kujala, and SF-36 Subscales^a

Correlation	Hypothesized r	Found r
I-IKDC and Kujala	>0.8	0.85
I-IKDC and SF-36 physical functioning subscale	>0.6	0.63
I-IKDC and SF-36 physical role functioning subscale	>0.6	0.61
I-IKDC and SF-36 bodily pain subscale	>0.6	0.69
I-IKDC and SF-36 PCS subscale	>0.6	0.66
I-IKDC and SF-36 general health perceptions subscale	≤0.6	0.52
I-IKDC and SF-36 vitality subscale	≤0.6	0.53
I-IKDC and SF-36 social function subscale	≤0.6	0.42
I-IKDC and SF-36 emotional role functioning subscale	≤0.6	0.52
I-IKDC and SF-36 mental health subscale	≤0.6	0.58
I-IKDC and SF-36 MCS	≤0.6	0.59
I-IKDC and Kujala correlation stronger than between I-IKDC and PCS		0.85:0.66
I-IKDC and PCS correlation stronger than between I-IKDC and MCS		0.66:0.59

^aValues are presented as Pearson correlation coefficient. Values with a colon present a comparison between the 2 categories. I-IKDC, Indonesian translation of International Knee Documentation Committee; MCS, mental component summary score; PCS, physical component summary score; SF-36, 36-Item Short Form Health Survey.

Internal Consistency, Test-Retest Reliability, and Measurement Error

The Cronbach α was .90, indicating good internal consistency. The ICC had a value of 0.99 (P < .001), and the 95% CI ranged from 0.989 to 0.996. The SEM, MDC_ind, and MDC_grp were determined to be 2.1, 5.8, and 0.7, respectively. The Bland-Altman approach revealed a mean difference of the 2 I-IKDC assessments of –0.3 (95% CI, –0.8 to 0.2; 95% LOA, –37.7 to 37.2) (Figure 1). No systematic bias was present because the value of 0 was in the 95% CI of the mean difference between the test and retest scores.

Figure 1.

Bland-Altman plot of the mean difference between the first and second measurements of the Indonesian translation of International Knee Documentation Committee (I-IKDC). The open circles represent data points, the solid line represents the mean difference, the dotted lines represent the 95% CIs, and the dashed lines represent the limits of agreement.

Discussion

From this study, we were able to confirm all predefined hypotheses on correlations between the I-IKDC and other scores, implicating good construct validity: An ICC value of 0.99 implicated good test-retest reliability; a Cronbach α of .90 implicated good internal consistency; and no floor or ceiling effects were found. In addition, the SEM was 2.1, and the MDC was 5.8 at the individual level and 0.7 at the group level. Based on these results, the I-IKDC can be considered a valid and reliable questionnaire for use in patients after ACL injury.

The construct validity of the I-IKDC was considered good, with all predefined hypotheses confirmed. As hypothesized, the I-IKDC score showed a strong correlation (>0.6) with the PCS and the PF subscale of the SF-36 (r = 0.66 and 0.63, respectively). These findings lend evidence for the validity of the I-IKDC as a measure of knee-specific symptoms and limitations in function and sports activity. Our results are in line with those of research in other languages, as in the original English version and the Dutch, Italian, Brazilian, and Chinese versions (Table 4).^{7,10,11,17,20} The I-IKDC also showed, as hypothesized, a correlation lower than 0.6 to the MCS and SF subscale of the SF-36, which is weaker than that to its physical domains. The same result was observed in the validity study of the original English version and of the Brazilian version of the IKDC.^11,17 We also used the Kujala score to evaluate construct validity, and, as hypothesized, the I-IKDC score showed a strong correlation with the Kujala score (r = 0.85), similar to the results found in the Turkish version (r = 0.89).³ Other questionnaires could have been used to test construct validity, such as the Lysholm score and the Western Ontario and McMaster Universities Osteoarthritis Index, which were used in the Brazilian Portuguese IKDC translation, or the Oxford 12 Questionnaire, which was used in the Dutch IKDC translation. However, as no validated Indonesian translations are available for these questionnaires, we could not use them to test the construct validity of the I-IKDC.

Table 4.

Correlations Between Different Versions of the IKDC and the SF-36 Subscales^a

	IKDC Version
SF-36 Subscale	Indonesian	English11	Dutch10	Italian20	Thai16	Brazilian17	Chinese7	Korean12	Turkish3
Physical functioning	0.63	0.63	0.71	0.67	0.75	0.75	0.64	0.66	0.69
Physical role functioning	0.61	0.47	0.55	0.56	0.37	0.54	0.50	0.49	0.53
Bodily pain	0.69	0.64	0.69	0.75	0.76	0.63	0.64	0.30	0.47
General health perception	0.52	0.30	0.41	0.26	0.21	0.54	0.50	0.11	0.32
Vitality	0.53	0.39	0.40	0.36	0.29	0.46	0.44	0.15	0.24
Social function	0.46	0.47	0.42	0.58	0.22	0.43	0.41	0.48	0.40
Emotional role functioning	0.52	0.26	0.30	0.44	0.34	0.50	0.24	0.30	0.22
Mental health	0.58	0.25	0.21	0.65	0.29	0.40	0.41	0.15	0.13
PCS	0.66	0.66	—	0.60	0.63	0.79	—	—	0.70
MCS	0.59	0.16	—	0.40	0.37	0.51	—	—	0.05

^aDashes indicate that there is no value of those categories from the previous literature. IKDC, International Knee Documentation Committee; MCS, mental component summary score; PCS, physical component summary score; SF-36, 36-Item Short Form Health Survey.

No floor or ceiling effects were observed. Theoretically, a ceiling effect would have occurred, especially in the longer term after treatment of an ACL injury, but this was not the case in the original version of the IKDC or the Dutch, Brazilian, or Turkish versions.^3,10,11,19 Hence, this effect was also expected to be absent in the Indonesian version of this form.

Internal consistency of I-IKDC was good, with a Cronbach α of .90. Higher values are not desirable, as this may indicate a redundancy of questionnaire items. Good internal consistency also was shown in the original version (0.92) and the Italian (0.91), Dutch (0.92), Thai (0.92), and Korean (0.91) versions.^{10–12,16,20}

Test-retest reliability of the I-IKDC was found to be excellent (ICC = 0.99). This is comparable with the original English version (ICC = 0.95) and the Dutch (ICC = 0.96), Thai (ICC = 0.92), Brazilian (ICC = 0.99), and Turkish (ICC = 0.91) versions (Table 5).^{3,10,11,16,17} The Bland-Altman plots showed adequate agreement, reporting only a small difference between the mean scores of the first and second assessments of the I-IKDC. No systematic bias was present because the value of zero was within the 95% CI of the mean difference between the test and retest score of the I-IKDC.

Table 5.

Psychometric Properties of the Different Versions of the IKDC^a

Study	Language Version	Test-Retest Reliability, ICC	Cronbach α
Irrgang et al11	English	0.95	.92
Padua et al20	Italian	0.9	.91
Haverkamp et al10	Dutch	0.96	.92
Crawford et al4	English	0.95	.77
Lertwanich et al16	Thai	0.92	.92
Greco et al8	English	0.91	.93
Metsavaht et al17	Brazilian Portuguese	0.99	.92
Fu and Chan7	Chinese	0.87	.97
Kim et al12	Korean	0.94	.91
Çelik et al3	Turkish	0.91	.89
Present study	Indonesian	0.99	.90

^aICC, intraclass correlation coefficient.

For the I-IKDC, the SEM was 2.1, MDC_ind was 5.8, and MDC_grp 0.7. These values are comparable with those presented in the original English version (SEM, 4.6) as well as the Brazilian (SEM, 2.4; MDC, 6.7) and Turkish (SEM, 6.0; MDC, 16.4) versions.^3,11,17 Low MDC values at the group level indicate that the I-IKDC can be used for group comparisons, as only low values are needed to detect change. However, only values higher than the SEM can be distinguished from the measurement error; therefore, to detect a statistically significant change in scores on the I-IKDC, the difference should be higher than the SEM. To distinguish from a measurement error and confirm that a real change occurred, the difference between 2 measurements should be greater than the MDC_ind value. It is therefore questionable whether the I-IKDC is an appropriate tool for monitoring individual patients over time.

The current study has some limitations. The response rate was not high (42%), yet the total number of participants can be considered sufficient, because according to the general recommendations for comparing measurement properties, at least 100 patients are needed to study validity and 50 patients to study test-retest reliability.²⁶ Another potential limitation is that we compared the I-IKDC scores with the SF-36 and only 1 other disease-specific scoring system (Kujala score). However, as discussed, no other validated PROM in the Indonesian language is available to evaluate knee pain or function in addition to these scores.

Next to the psychometric properties that have been examined in this study, future research should investigate the minimal important change (MIC) as a measure of responsiveness of the I-IKDC. The MIC should be determined to assess whether a found difference is also clinically important as perceived by the patient.⁹ The MIC of the English version of the IKDC was reported to be 10.9.¹⁹ For the German version, an MIC of 6.8 is reported, which is smaller than the MDC value.¹⁵ In addition, effect sizes are reported: 1.13 in the English version,⁸ 2.09 in the Turkish version,³ and 1.04 in the German version.¹⁵ These findings are, however, not transferable to the Indonesian version, and hence, the MIC of the I-IKDC should be determined in future research. Again, these results cannot be transferred to the I-IKDC. Knowledge about responsiveness and capability of the I-IKDC to detect change over time is needed for the I-IKDC to be used in longitudinal research and in practice to follow Indonesian-speaking patients over time.

Conclusion

The I-IKDC can be considered to be both valid and reliable; therefore, it is applicable for patients with an ACL injury in the Indonesian population as a PROM. Now that this questionnaire is available, we can measure symptoms, function, and sports activity in the Indonesian ACL population. More research is needed regarding MIC and responsiveness.