Translation and Validation of the German New Knee Society Scoring System

Abstract

Background

In 2011 the Knee Society Score (KSS) was revised to include patient expectations, satisfaction, and physical activities as patient-reported outcomes. Since the new KSS has become a widely used method to evaluate patient status after TKA, we sought to translate and validate it for German-speaking populations.

Questions/purposes

After translation of the new KSS into German using established guidelines, we sought to test the new German version for (1) validity; (2) responsiveness; and (3) reliability.

Methods

The new KSS form was translated and adapted according to the available guidelines. The final version was used to validate the German version of the new KSS (GNKSS) in 133 patients undergoing TKA, of which 100 patients were included in the study as per inclusion criteria. Patients completed the GNKSS form along with the German WOMAC and the German SF-36 scores preoperatively and at the 2-year postoperative followup. Construct validity was tested by comparing domain scores of the GNKSS with domain scores of the German WOMAC and the SF-36. Responsiveness was evaluated by comparing pre- and postoperative scores in all questionnaires in all patients using standardized response means. To evaluate reliability, every second patient (n = 50) in the whole group was asked to complete the GNKSS form a second time 1 week after their 2-year followup; 39 patients responded. This sample group was considered representative after testing the difference among age, sex, body mass index, operation side, preoperative or postoperative GNKSS, and WOMAC scores with the original group. Intraclass correlation coefficients (ICCs) were used to assess reliability and Cronbach’s α was an indicator of internal consistency of each domain score.

Results

Construct validity was excellent pre- and postoperatively between the GNKSS and the WOMAC for domains including symptoms, satisfaction, total functional score, and total score and activity subdomains, except the expectation domain and advanced and discretionary subdomains of the GNKSS and the stiffness domain of WOMAC. The expectation domain showed either no significant correlation or only weak correlations with the domains of WOMAC pre- as well as postoperatively (r ranging between -0.19 and -0.34). Correlation of the function section of the GNKSS as well as the physical function and role-physical domains of the SF-36 pre- and postoperatively were moderate to strong, respectively, with statistically significant (p < 0.001) r values of 0.49 and 0.48 preoperatively and 0.73 and 0.65 postoperatively. Correlation of the symptom section of the GNKSS and bodily pain domain of the SF-36 was also strong pre- and postoperatively. Regarding responsiveness, all domains of the GNKSS showed large changes except the expectation domain. The symptom and functional sections of the GNKSS showed higher responsiveness than the corresponding pain and function domains of the WOMAC and bodily pain and physical function domains of the SF-36. Also, the total score changes were larger for the GNKSS compared with the WOMAC. No floor or ceiling effect was observed. Reliability was excellent with ICCs of 0.83 to 0.97 as an indicator of test-retest reliability and Cronbach’s α values of 0.78 to 0.85 preoperatively and 0.92 to 0.94 postoperatively as an indicator of internal consistency for all domains and subdomains.

Conclusions

The GNKSS is a valid, responsive, reliable, and consistent outcome measurement tool that may be used to evaluate the outcome of TKA.

Level of Evidence

Level II, diagnostic study.

Introduction

The Knee Society Score (KSS) has been widely accepted and used worldwide after its initial introduction in 1989 [11]. There have been concerns regarding its reliability and responsiveness, however, and contemporary patients may have expectations that differ from those of nearly 30 years ago [22]. The increased number of younger patients undergoing TKA and the projected growth of TKAs have also contributed to the need for an updated scoring system that reflects enhanced functional and recreational activities after TKA [20]. Consequently, a new KSS was developed in 2011 that included a new patient-reported outcome section that also measures satisfaction, functional activities, and expectations. The new KSS has been validated in terms of reliability and consistency [20].

The increase in TKAs in Germany mirrors that of the United States [33]. To compare international outcomes after TKA, accepted outcome scoring instruments locally adapted to each language and culture are necessary. For this purpose, the new KSS has been translated and validated in various languages including Dutch, Japan, Chinese, and Korean in previous studies [10, 13, 16, 31]. However, a validated German-language version of the new KSS does not exist.

The purpose of this study therefore was to establish a validated German version of the new KSS for German-speaking individuals undergoing TKA by translating the new KSS into German and testing it in terms of (1) construct validity; (2) responsiveness; and (3) reliability.

Patients and Methods

Translation

The translation and adaptation of the score were completed according to previously published guidelines [2]. Two independent native German-speaking translators (RB, VM) translated the new KSS from English to German. One translator was aware of the process and the other unaware. Both versions were evaluated and merged into a single translation draft. This draft was then backtranslated by two other independent translators (WF, MEK). Finally, a review committee evaluated the translations and established a prefinal version of the scoring tool. This version was tested on 30 patients with osteoarthritis to identify comprehension issues or problems associated with completing the questionnaire. We made some minor modifications, and the final version was established (Appendices, Supplemental Digital Content 1 and Supplemental Digital Content 2).

Study Design

We obtained ethical approval from the ethical committee of the state of Brandenburg, Germany, before starting the study. A total of 133 patients were identified who had undergone primary TKA from the first quarter of 2014 to the third quarter of 2015. Inclusion criteria were patients undergoing primary TKA who provided their consent, could complete the questionnaire without assistance, and were willing to complete the questionnaire at followup visits. Patients were excluded if they underwent revision TKA during the followup period (n = 1), had surgery on the contralateral side (n = 6), had accompanying hip or lumbar pain (n = 8), or did not complete all three questionnaires sufficiently as required for score calculations (n = 18) as per user manuals for each questionnaire as explained subsequently. The resulting 100 patients, 60 of whom were women, were included in the study (Table 1). We determined the number of patients and methods to be used in this validation study based on prior reports [2, 29]. The well-defined adaptation guidelines from Beaton et al. [2] suggest a sample size of 30 to 40 patients for pretesting a new scoring tool, but does not suggest a minimum group size for validation. However, Terwee et al. [29] recommend at least 100 patients for internal consistency, and 50 patients are needed to effectively evaluate floor and ceiling effects and to evaluate construct validity in validation studies. Still, a prior meta-analysis indicated a lack of clear guidance and consensus about sample size determination [1]. Finally, because the new KSS is designed to assess patients before and after TKA, it has two separate versions for each application with totally different questions in the expectation section. Therefore, separate assessments of both forms for construct validity were necessary. However, some prior translation and validation studies of the new KSS included either postoperative patients or mixed groups of patients from preoperative and postoperative periods by stating the sum of both groups as the sample size group of their study [10, 16, 31]. In the current study, the same sample group of patients was included pre- and postoperatively to prevent any related responder issues and to validate both versions to make them available at the same time.

Table 1.

Demographic data of the sample group population (n = 100)

All patients were evaluated preoperatively and underwent primary TKAs at the University Hospital of Brandenburg Medical School Department of Orthopaedics and Traumatology and were then reevaluated at the 2-year followup point. The mean followup time interval was 24 months (range, 22-26 months). The mean age and body mass index (BMI) at the time of preoperative evaluation were 72 (± 9) kg/m² and 31 (± 5) kg/m², respectively (Table 1).

All patients were asked to complete the three questionnaires (German New KSS [GNKSS], WOMAC, SF-36) pre- and postoperatively at the 2-year followup. The patients completed all the forms except the objective portion of the new KSS without any assistance. A senior orthopaedic resident (MEK) under the supervision of the senior attending (RB) completed the clinical and radiologic examinations on all patients. Radiographs were used to document implant position and to exclude component loosening. The WOMAC scores were converted into a 100-point scale before statistical analyses were made, as mentioned in a previous study [26]. Scores from the German SF-36 were included as eight individual single-domain scores with the mental and physical summary scores on a 100% scale as previously recommended [15, 28].

Every second one of the randomly listed patients (n = 50) by the data recording program were asked to repeat the questionnaire 1 week after the 2-year followup. Patients did not receive any treatment of any kind in this timeframe to test reliability of the GNKSS as recommended in the literature [31]. Patients were asked to send these forms per mail to the hospital. The symptom section of these forms was specifically designed to let patients fill out only the 10-point scales but not the scoring boxes, because this part of the questionnaire should be filled out or calculated by the physician as the developers intended [30]. Thirty-nine patients out of 50 sent back the completed forms by mail to the hospital. For a Type I error rate, Van der Straeten et al. [31] recommended a sample size of 38 for a power of 0.90 and an α of 0.05, but in their mixed population, groups from pre- and postoperative periods were used without a clear definition of the sample configurations. However, the sample group in this study included patients only from the postoperative period, which creates a more uniform sample group with likely more powerful results.

The sample group was compared with the original group in terms of age, gender, BMI, and preoperative and postoperative GNKSS total and functional scores. The chi-square test, Student’s t-test, and Mann-Whitney U tests were performed as required for evaluating the differences between the groups. No significant differences for gender (p = 0.912), age (p = 0.592), BMI (p = 0.421), preoperative GNKSS total (p = 0.454) and functional (p = 0.590) scores, or postoperative GNKSS total (p = 0.150) and functional (p = 0.153) scores were observed. The sampling group was considered representative of the original group according to these findings.

All collected clinical data were then entered into a computerized database (Microsoft Access®, Microsoft Office 2013; Microsoft, Redmond, WA, USA).

Statistical Analysis

Construct Validity

Construct validity shows to what degree a test measures what it is intended for. Because the GNKSS was intended to reflect patient status before and after primary TKA, a comparison with already existing and validated scoring tools was deemed necessary. Therefore, we analyzed the construct validity using the German WOMAC and the German SF-36, because there is no accepted reference method to reflect the status of patients before and after TKA and because these tests have been validated in previous studies [8, 27]. A comparison among domain scores of all three tools was made pre- and postoperatively in all 100 patients. Using Spearman’s coefficient, we computed correlations. These correlations were hypothesized to be either less converging or divergent for mental domains, including expectations, and strongly converging for physical domains as seen in previous studies [20, 31]. The strength of converging correlation was considered weak, moderate, or strong for coefficient values of 0.35, 0.35 to 0.5, and > 0.5, respectively.

Responsiveness

Responsiveness is the ability of a test to show differences after a defined treatment method. This means differences in pre- and postoperative scores of the GNKSS should reflect the improvements after primary TKA. Greater differences between scores before and after a treatment method would show greater ability of the tool to reflect changes when compared with other scoring tools. To assess responsiveness, correlations of the results of related domains among all tests were evaluated preoperatively and postoperatively using standardized response means (SRM). SRM values were calculated as the mean difference between preoperative and 24-month postoperative scores divided by the SD of the score, whereby the 95% confidence intervals (CIs) were calculated with a jackknife procedure. The aim was to prove the ability of the outcome measure to reflect the effect of TKA. SRM values were graded in means of change as small (values of 0.2-0.5), moderate (0.5-0.8), and large (> 0.8). We thought that SRM values for each domain of the GNKSS except expectations would be > 0.8 because TKA would be expected to provide better functional results and decrease pain associated with osteoarthritis.

A scoring tool is expected to reflect patients’ status in outcome with a normal distribution of scores. In case of an accumulation of scores toward the maximal or minimal zone, it is called a ceiling or floor effect, which shows the limitations of the scoring tool to differentiate among patients’ status in outcome. These effects were examined by analysis of distribution of scores to show the ability of the GNKSS to differentiate improvements and further prove its responsiveness.

Reliability

Test-retest reliability tests the stability and consistency of a scoring tool over a time period. Patients complete the forms a second time after a certain timeframe without receiving any additional interim treatment. A timeframe of 1 week was chosen for the evaluation of test-retest reliability because other investigators [18] have suggested that a timeframe ranging from 2 days to 2 weeks seems to be a reasonable compromise between avoiding changes in a patient’s condition and preventing recall bias. Test-retest reliability intraclass correlation coefficients (ICCs) were assessed with a 95% CI and evaluated internal consistency was evaluated with Cronbach’s α coefficients, which show the ability to maintain coherence of the different components of the scale. Variance components calculated by a random-effects analysis of variance were used to calculate ICCs. Reproducibility was accepted as excellent for an ICC value > 0.8. Internal consistency was evaluated as fair (0.7 α values), good (0.8 α values), or excellent (0.9 α values).

Other Considerations

When a missing answer was detected, the following process was observed: For the GNKSS, dummy values equal to the average of all of the other items in the same domain were entered. If the patient indicated fewer activities than required in the discretionary activities section, a mean score was inserted for the missing item. Patient responses of “I never do this” were rated as 0 point. All of these steps were in compliance with suggestions made in the user manual from the developers [30]. For the WOMAC, we followed a similar method that was also recommended in the WOMAC user guide [3]. For the SF-36, the missing item percentage of the related domain was relevant, so when the patient answered more than half of the questions in the respective domain, the missing item was replaced with an average score derived from answers to other questions of the same domain, as recommended [32]. As an overview, 10.6% missing items were detected in the German SF-36, 3% in the German WOMAC, and 1% in the GNKSS, very similar to percentages previously published [31]. If more than the allowed missing items for each test according to relevant guidelines were detected, or two or more domains or whole tests were missing, those patients were excluded.

The results were analyzed using SAS, Version 9.4, software (SAS Institute, Cary, NC, USA).

Statistical analyses were performed by a certified statistician (TK). All the scores included mean and SD values and p values of < 0.05 were considered statistically significant.

Results

Validity

Convergent validity was used to assess the construct validity of the GNKSS, which shows the theoretical correlations between two scoring tools. Correlation of corresponding scores from the GNKSS, the WOMAC, and the SF-36 was strong, suggesting that the GNKSS is able to reflect patient status similar to these already validated tests. WOMAC scores correlated negatively with the GNKSS in reciprocal nature, that is, higher scores on the WOMAC reflected worse clinical outcome, whereas they reflected better clinical outcomes in the new KSS and vice versa. Construct validity, as indicated by Spearman coefficients, was strong among all subdomains of the GNKSS and WOMAC pre- and postoperatively with values between -0.51 (p < 0.001) and -0.82 (p < 0.001); the only exception was the expectation domain of the new KSS and stiffness domain of the WOMAC, where the correlations showed weak converging results. This was the result of the fact that these domains of the scoring tools did not correspond with any other domains directly, that is, the WOMAC does not include questions regarding patient expectations and the KSS does not evaluate knee stiffness like in the WOMAC, so a high correlation ratio would not be expected. Moreover, all the domains and the four subdomains of the function section of the GNKSS, except the expectation section, as a result of same reasons, correlated moderately or strongly as expected with the German WOMAC total score (Table 2).

Table 2.

Construct validity between the German New Knee Society Score (GNKSS) and the German WOMAC

Correlations between the physical domains of the SF-36 and the GNKSS such as bodily pain with symptoms and physical function and physical role with activity domains of the GNKSS were moderate to strong in a converging manner preoperatively and postoperatively, as anticipated a priori, suggesting that the GNKSS also performed similarly with the formerly validated SF-36 in corresponding domains. However, a strong correlation among all domains was not expected, because the SF-36 is a general health-related quality-of-life assessment tool and the GNKSS differs from it because it aims primarily to reflect patient status regarding primary TKA. Coefficient values ranged from 0.48 to 0.73, proving the moderate-to-strong correlations. Furthermore, the correlation of the symptom section of the GNKSS and bodily pain domain of the SF-36 was strong pre- and postoperatively. Mental domains such as vitality and emotional role functioning showed diverging or weakly converging results with all the domains and subdomains of the GNKSS, because these domains do not directly correspond to any domain of the GNKSS as explained previously. These results were in line with the a priori hypothesis that mental components would not show strong correlations with the domains of the GNKSS (Table 3).

Table 3.

Construct validity between the German New Knee Society Score (GNKSS) and the German WOMAC

Responsiveness

Responsiveness evaluation, as demonstrated by SRMs, showed that all GNKSS domains had large changes, proving the superior ability of the GNKSS to reflect improvements in outcome after primary TKA with SRM values ranging between 1.65 and 2.36, except for the expectation domain, as hypothesized a priori. The symptom section along with the functional subdomains also showed large changes, which were greater than the corresponding pain and function domains of the WOMAC and the physical function and bodily pain domains of the SF-36, which indicates the GNKSS is more responsive in corresponding domains, which are the pain and functional domains. Total score changes were reflected with SRM values of 1.87 for the GNKSS and -1.49 for the WOMAC, which demonstrated that overall responsiveness of the GNKSS was also larger than that of the WOMAC (Table 4), which means that the total GNKSS score is also a more sensitive parameter of showing improvements after primary TKA when compared with the WOMAC total score.

Table 4.

Responsiveness of the German New Knee Society Score (GNKSS) compared with the WOMAC and SF-36

Analysis of distribution of scores at the second year followup showed that there were no floor or ceiling effects.

Reliability

Regarding reliability, all domains of the GNKSS showed excellent results for all domains both in terms of test-retest reliability represented by ICC values between 0.82 and 0.97 and internal consistency represented by Cronbach’s α values between 0.78 and 0.85 preoperatively and 0.92 and 0.94 postoperatively (Table 5).

Table 5.

Reliability measurements of the German New Knee Society Score (GNKSS)

Discussion

The new KSS has been widely adopted worldwide, and several translation and validation studies have been published in other languages. Our purpose was to produce a German version of the new KSS using well-defined guidelines to confirm that it has valid measurement properties [2, 13, 29]. Our results indicate that the GNKSS proposed in this study is a valid, responsive, reliable, and consistent outcome tool to be used in German-speaking populations to evaluate the pre- and post-TKA status of patients.

The current study has several limitations. First, like in all adaptation and validation studies, patients had to complete three separate scoring tools simultaneously, which may have resulted in missing or invalid responses as a result of an increase in responders’ burden. The developer of the KSS recognized this and has published a short version [17, 23]. This version should also be validated in a future study for German-speaking populations. Nonetheless, missing item percentages in the current study were similar to prior studies with 10.6% missing items in the German SF-36, 3% in the German WOMAC, and 1% in the GNKSS [31]. Second, only one center was involved in the study, although it is the only tertiary university hospital in its federal state of Brandenburg (population 2.5 million) and its patients reflect rural and urban populations. Theoretically, sample configurations should be conducted with recruitment from different areas and states of a country to decrease the risk of bias related to demographic and cultural factors. However, the results obtained in this current study, which are comparable with other well-designed validation studies, as well as the development study of the new KSS, make it less questionable whether the sample configuration was adequately conducted [13, 20, 31]. The proportion of women (60% [60 of 100]) reflects the published demographic features of German patients undergoing knee arthroplasty [33], and it also matches exactly the gender distributions in the development study of the new KSS [20]. Therefore, an additional analysis regarding the gender composition of our study was not deemed necessary.

The current study proved that the GNKSS has good construct validity by evaluating the convergent validity with the already validated German WOMAC and SF-36. Overall correlation of GNKSS and WOMAC total scores along with corresponding pain and total functional score demonstrated strong correlations, which were very similar to the findings by Kim et al. [13] and Van der Straeten et al. [31]. Correlation of the expectation domain of the new KSS on the other hand showed weak insignificant convergent correlations, whereas Kim et al. [13] found weak divergent correlations, where half of their results were also statistically insignificant. Other investigators either did not publish their data or did not use the WOMAC in evaluating their versions [10, 31]. These results were in line with our a priori hypothesis, because the expectation section of the KSS has no corresponding domain in the WOMAC.

Correlation of the pain and total activity scores of the KSS with the bodily pain and physical function domains of the SF-36 was strong and moderate for the preoperative group and strong for both domains for the postoperative group. Only two other studies in the literature published their comparative correlation results [10, 13]. Their study designs included either only pre- [13] or postoperative [10] patients. Our results regarding pre- as well as postoperative groups’ pain domains showed strong correlations, whereas other authors reported either weak [13] or moderate [10] correlations. The seemingly divergent correlation result in the study of Hamamoto et al. [10] is likely the result of a calculation error we explain subsequently. Activity score correlation was on the other hand moderate for the pre- and strong for the postoperative group in our study, whereas it was strong [13] for pre- and moderate [10] for the postoperative group in other studies. There may be several explanations for these results. Variable correlations between the same domains in pre- and postoperative groups were also observed and reported in the development of the new KSS study [20]. Because the SF-36 is a general health-related quality-of-life assessment tool and the GNKSS was developed to reflect patient status regarding primary TKA, strong correlations were not expected. Moreover, the only available comparative data are published in Asian countries, where their authors explained the variable results as related to cultural differences [13].

Our results also showed that the GNKSS was very responsive. The most responsive domain of the GNKSS was the symptom section; the total functional score and the total score also showed large changes. Both GNKSS and Korean versions [13] showed very similar large changes in symptom-oriented domains of all scoring tools: the symptom domain in the KSS, pain domain in the WOMAC, and bodily pain domain in the SF-36 as well as function-oriented domains; and total functional score of the KSS, function domain of the WOMAC, and physical function domains of the SF-36 (Supplemental Table, Supplemental Digital Content 3). As expected, the GNKSS proved to be more responsive to changes after TKA compared with the WOMAC and the SF-36. The new KSS was developed to reflect changes in patient status in relation to TKA, whereas the WOMAC was developed to highlight the status of patients with osteoarthritis without being specifically responsive to treatment, and the SF-36 was developed as a monitoring tool of overall well-being of patients [4, 7, 20].

The GNKSS also demonstrated excellent reliability with overall higher ICC scores compared with other validation studies of the new KSS, especially in satisfaction, total functional activity, and total score results [13, 31] (Supplemental Table, Supplemental Digital Content 4). Our results were excellent in all domains, whereas other studies also showed excellent results except in two domains in the Korean [13] and three domains in the Dutch version [31]. There may be several explanations for the excellent reliability seen here. First, in the current study, test-retest evaluation was made after a mean of 24 months postoperatively, whereas other investigators did it either preoperatively or 12 months postoperatively. Hence, the time interval could be a factor in the slightly different results for the ICCs. Second, cultural and linguistic factors could have affected test-retest evaluations, because we noted that previous German validation studies of other patient-reported outcome measures commonly stated higher scores for ICC in test-retest evaluations when compared with initial scoring tools, although they used similar time intervals for retesting [5, 6, 9, 19, 21, 25]. Cronbach’s α values were calculated for both pre- and postoperative scores. Postoperative results were higher with α values between 0.92 and 0.94 compared with 0.80 to 0.88 preoperatively. These results were interpreted as good and excellent, respectively. In previously published adaptation and translations studies of the new KSS, either pre- or postoperative versions or mixed groups were examined [10, 13, 31]. Nevertheless, our findings are comparable to these results as absolute numbers (Table 6). No discussion of the difference between pre- and postoperative results was noted in review of prior validation studies, but the development study of the new KSS also showed higher α values in every domain and subdomain except the advanced activities subdomain postoperatively [20]. Overall, our results concerning the Cronbach’s α coefficients were higher than in the development study and in other adaptation and validation studies (Table 6). Results from both of these measurement properties proved that the GNKSS is reproducible.

Table 6.

Comparison of internal consistency results as Cronbach’s α values

We observed some common issues while analyzing other validation studies regarding the new KSS that may prove helpful to others considering such studies. During pretesting we realized that how patients’ responses to symptoms section were noted and scored was prone to produce calculation errors. This section includes two 10-level scale pain questions and responses, which should not be carried to the score box directly adjacent to the scales. Not subtracting the patient’s recorded answer from the maximum score of 10 results in disproportionate scores in this section, which causes higher (that is, better) scores although the patient reports more severe pain resulting from the reciprocal nature of the pain scale and symptom section score. Some prior published validation studies revealed that this possible error may have produced disproportionate results, as can be seen in the French and Japanese versions of the new KSS [10, 12]. Several validation studies have not shared their results either in absolute numbers or in percentages or direction of changes for domains of the new KSS; they have shared only the statistical results of comparisons made with other scoring tools [16, 24, 31]. To prevent the aforementioned error, we contacted the developers and updated this section by marking the pain scale section as “to be calculated by the patient” and the scoring box as “to be calculated by the physician.”

When evaluating the results from the scoring tools, we highlighted correlations of corresponding domains of the used scoring tools; we also analyzed total scores from the WOMAC and the new KSS in relation to each other. Because there is no total score calculation in the SF-36 and summary component scores should not be analyzed on their own, we did not analyze physical and mental summary scores in this manner. We did include them in the study for further observational and comparable value, although a number of validation studies used either only summary scores or made separate analyses using them [14, 15, 28].

The GNKSS is a valid, responsive, reliable, and consistent outcome measurement tool to be used in German populations to evaluate preoperative and postoperative TKA status, including patients’ symptoms, expectations, satisfaction, and physical activities. Future studies sampling other German-speaking populations may increase the external validity of the GNKSS.