Reliability of the "American Knee Society Score" (AKSS)

Abstract

Objective

To analyze the reproducibility of the "American Knee Society Score" (AKSS) scale, and determine its measurement, in order to make it useful for the evaluation of patients with osteoarthritis or who have undergone total knee arthroplasty.

Methods

In the first interview, the AKSS was applied along with the SF-36 and WOMAC (examiner 1). After thirty minutes the same patients answered only the AKSS. After a two week break, a third additional interview with AKSS was applied (examiner 2).

Results

We selected 58 patients with a mean age of 67.4 years. In the analysis of reproducibility, by ICC, there was strong inter-examiner and intra-examiner correlation for two AKSS components. In the individual items analysis there was good correlation for "Pain", "Range of Motion", "Flexion contracture" and all items of the AKSS Function component. Validation through the Pearson coefficient showed good correlation between AKSS "Pain," WOMAC "pain" and SF-36 "Pain domain", and good correlation between the AKSS and SF-36 "Functional Capacity domain".

Conclusion

The AKSS adapted to Brazilian culture is useful and reliable for the evaluation of individuals with osteoarthritis or those who have undergone TKA.

INTRODUCTION

Osteoarthritis is one of the most frequent musculoskeletal disorders, responsible for the recurring work incapacity of the individuals affected. Consequently, total knee arthroplasty (TKA), a procedure designed to repair this damaged joint, is among one of the most common orthopedic procedures in the world, and its results are constantly quantified in scientific studies with the objective of evaluating the results of this intervention. Nowadays, the criteria for evaluation of musculoskeletal disorders are being constantly modified, which demonstrates a concern among health professionals, both with discovering whether a particular treatment has obtained the expected result, and with ascertaining its impact on the quality of life, functionality and satisfaction of patients. Standardized evaluation scales and questionnaires are necessary for the quantification and reproduction of such results, thus avoiding misinterpretations and conclusions with limited reliability.^1-4 These measurement instruments, based on scoring systems, are applied so as to assess the benefits achieved by the surgical procedure^2,5, which can be either specific or generic in nature; however, we suggest the inclusion of both in the evaluation of the individual, since this combination represents the best method of analysis of the surgical results of patients.^6,7 Among the generic instruments found in literature, we can cite the SF-36 questionnaire (The Medical Outcomes Study 36-item Short-Form Health Survey), created in 1992, and used worldwide, whose objective is to evaluate the health and quality of life of the individual in a generic manner, without taking into account a specific disease, age or treatment group.^8,9 The specific questionnaires used the most often for osteoarthritis and after total knee arthroplasty include WOMAC, the Oxford Knee Score and others; they address parameters such as pain, mobility, gait and level of difficulty in the performance of daily activities.^9,10 However, this population of patients is generally elderly, and can present comorbidities that may affect the individuals' levels of mobility and function, regardless of the state of the knee joint, making it a major challenge for any scale to objectively evaluate knee function independent of the patient's general functional capacity.^11-13 In 1989,¹⁴ the "American Knee Society" group published an examiner-dependent clinical evaluation system known as the "American Knee Society Score" (AKSS) scale, divided into two components. The first assesses the knee clinically through the physical examination (Clinical AKSS - "Knee Score"), and the second assesses the individual's functionality (Functional AKSS - "Function Score"), while both attain a total of 100 points each. The objective of this separation was to make the scoring of the Clinical AKSS independent on the Functional AKSS, not being influenced by variables such as comorbidities and advanced age. The Clinical AKSS evaluates pain, in a total of 50 points, stability, 25 points, and range of motion, 25 points. The maximum score of 100 points is reached when there is no pain, with good alignment of the knee in extension, and at least 125° of range of motion, without any anteroposterior or mediolateral instability. Deductions are made for flexion contracture, loss of extension and poor alignment. The Function AKSS evaluates the walking distance, totaling 50 points, and the act of climbing and descending stairs, 50 points. The maximum score of 100 points is attributed to the individual capable of walking unlimited distances without walking aids, and of climbing and descending stairs normally. Deductions are made for the use of canes, crutches or walking frame. The system for selecting patients evaluated by the AKSS scale is provided through categories: A- unilateral or bilateral (opposite knee replaced successfully), B- unilateral, other knee symptomatic and C- polyarthritis or clinical disease.^15-17 The AKSS is currently the scale of choice in the United Kingdom for evaluation between the pre and postoperative results of TKA.^16,17 The aim of this study was to analyze the reproducibility of the "American Knee Society Score" (AKSS) scale, in comparison to the SF-36 and WOMAC questionnaires, with the purpose of detecting correlation between these instruments, as well as the reliability of the AKSS for evaluating individuals suffering from osteoarthritis or who have undergone TKA.

METHODS

This study was approved by the Research Ethics Committee of Universidade Federal de São Paulo -Escola Paulista de Medicina.

Fifty-eight patients from the Knee Group Outpatient Clinic of the Department of Orthopedics and Traumatology of Universidade Federal de São Paulo/Escola Paulista de Medicina were selected randomly between January and November 2009. Adult Brazilian individuals of both gender, with a radiological and clinical diagnosis of knee osteoarthritis or who had undergone total knee arthroplasty more than 6 months prior to the start of the study and had not undergone a change of medication or any other procedure, in a period of less than 15 days from the start of the study, were included in the sample. Individuals with neurological pathologies and/or cognitive alterations, with a history of knee joint infection or other disorders in the joints of the lower limbs were excluded from this analysis.

Measurements

The evaluation of reproducibility was performed through the application of the AKSS to 58 individuals who fulfilled the predetermined inclusion criteria. The appraisers were trained and used standardized physical examination methods in order to minimize inter-examiner variability. All the patients were evaluated in the same position, using the same physical evaluation techniques. The participants were selected at random. In the first evaluation (A1), the AKSS scale was applied together with the WOMAC and SF-36 questionnaires, by examiner 1 (E1). All the patients were reevaluated after thirty minutes (A2) and again after an interval of two weeks (A3) (a sufficiently long time to guarantee that the patients do not remember their answers in the first interview and sufficiently short to guarantee that no significant change occurs in the patient's condition), whereas both evaluations were carried out by examiner 2 (E2). The SF-36 questionnaire includes 36 items that are combined in 8 domains: functional capacity, physical aspects, pain, general state of health, vitality, social aspects, emotional aspects and mental health. The score from 0 to 100 is produced independently by each domain. This instrument was translated into and validated for the Portuguese language.⁷ The WOMAC is a specific questionnaire for knee osteoarthritis (OA), self-administered and with 3 domains: pain (5 questions), joint stiffness (2 questions) and degrees of difficulty in activities of daily living (17 questions). Each one of the questions is graded on a scale from 0 to 4 points.¹⁰

STATISTICAL ANALYSIS

The statistical analysis of the demographic data was conducted in a descriptive manner. The reproducibility of the two components of the AKSS was evaluated using the intraclass correlation coefficient (ICC), with a confidence interval of 95%, in order to quantify the inter-observer and intra-observer analyses for the total scores of the two components of the scale and the correlation between each item separately. An ICC greater than or equal to 0.70 (p>0.50) (with a sample larger than or equal to 50 individuals) receives positive evaluation. Cronbach's coefficient was used to demonstrate the internal consistency of the analyzed scale (values between 0.70 and 0.95 are accepted as positive).¹⁹ The AKSS is derived from an algorithm and contains positive and negative scores, while it is inappropriate to test the internal consistency in the individual correlation of each item.¹⁷ The instrument's validity was assessed by the correlation between the "Pain" item of the Clinical AKSS and the Functional AKSS with the corresponding domains of the WOMAC and SF-36 questionnaires, using Pearson's linear correlation coefficient, with a confidence interval of 95% (>0.50, 0.35-0.50, and <0.35, are considered strong, moderate and weak, respectively).²⁰ The level of significance α equal to 5% was used in all the conclusions obtained through the inferential analyses

RESULTS

Characteristics of the patients

A total of 58 Brazilian individuals was recruited (23 with osteoarthritis and 35 submitted to TKA) to test the reproducibility of the AKSS scale. The average age was 67.4 years (ranging from 46 to 85, standard deviation of 8.9 years). The female sex (41/58) (68.4%) and involvement of the right lower limb (39/58) (67.2%) predominated in this sample. The two scale components were evaluated separately.

Reproducibility

In the analysis of the inter-examiner (correlation between A1 and A2) and intra-examiner (correlation between A2 and A3) reproducibility, according to the estimates of the intraclass correlation coefficient (ICC), strong correlation was observed for the total scores of the two components: Clinical AKSS and Functional AKSS (Clinical AKSS 0.87 and 0.89 / Function AKSS 0.89 and 0.81, respectively). The analysis of internal consistency (Cronbach's Coefficient) was also considered strong for both inter and intra-observer correlations. (Table 1)

Table 1.

Reproducibility of the AKSS scale - analysis of the inter and intra-observer correlation for the total scores of the two components (Clinical AKSS and Functional AKSS).

	ICCa	ICb(95%)	p	Cronbach's Alpha
Inter-observer Clinical AKSS (A1 e A2)	0.87	[0.79 ; 0.92]	<0.001	0.93
Intra-observer Clinical AKSS (A2 and A3)	0.80	[0.68 ; 0,87]	<0.001	0.89
Inter-observer Functional AKSS (A1 and A2)	0,89	[0.83 ; 0.93]	<0.001	0.94
Intra-observer Functional AKSS (A2 and A3)	0,81	[0.70 ; 0.88]	<0.001	0.89

a intraclass correlation coefficient / b interval with 95% of confidence.

The analysis of inter and intra-observer reproducibility of the individual items of the Clinical AKSS component, by the ICC, demonstrated good correlation in the items "Pain" (0.88 and 0.84), "Range of Motion" (0.92 and 0.88) and "Flexion Contracture" (0.70 and 0.70). There was poor correlation in the items "Mediolateral Stability" (0.58 and 0.51), and "Extension deficit" (0.56 and 0.53), and no correlation in the items "Anteroposterior stability" (0.20 and 0.25) and "Alignment" (0.25 and 0.21). As regards the Functional AKSS component, there was good correlation in all items. (Table 2)

Table 2.

Reproducibility of the AKSS scale - analysis of the inter and intra-observer correlation of each item of the two components (Clinical AKSS and Function AKSS).

	inter-observer ICC	Intra-observer ICC
Clinical AKSS
Pain	0.88 (p<0.001)	0.84 (p<0.001)
Range of Motion	0.92 (p<0.001)	0.88 (p<0.001)
Anteroposterior stability	0.20 (p<0.061)*	0.25 (p<0.025)*
Mediolateral stability	0.58 (p<0.001)	0.51 (p<0.001)
Flexion contracture	0.70 (p<0.001)	0.70 (p<0.001)
Extension deficit	0.56 (p<0.001)	0.53 (p<0.001)
Alignment	0.25 (p<0.024)*	0.21 (p<0.049)*
Functional AKSS
Distance walked	0.80 (p<0.001)	0.79 (p<0.001)
Stairs	0.72 (p<0.001)	0.71 (p<0.001)
Use of walking aids	0.83 (p<0.001)	0.71 (p<0.001)

CC -Intraclass correlation coefficient with confidence interval of 95%.

The analysis between the Clinical AKSS and the Functional AKSS and the WOMAC and SF-36 questionnaires was carried out by calculating the correlation between the items of each component, respectively: "Pain" of the Clinical AKSS with the "Pain" domains of the WOMAC and SF-36; and Functional AKSS (total score) with the "Function" domains of the WOMAC and Functional Capacity of the SF-36, analyzed by Pearson's linear correlation coefficient (r>0.70 and p< 0.005). There was good correlation between the Clinical AKSS "Pain" and the WOMAC "Pain" domains (r = 0.69), and between the Clinical AKSS "Pain" and the SF-36 "Pain" (r = 0.50). (Table 3) There was moderate correlation between the Functional AKSS and the WOMAC "Function", and good correlation between the Functional AKSS and the Functional Capacity domain of the SF-36. (Table 4). As there is no validated scale to measure the patient's clinical aspects after total knee arthroplasty, such as range of motion, alignment and stability, it was not possible to evaluate the validity of this part of the Clinical AKSS component.

Table 3.

Estimates of the Pearson's linear correlation coefficients between the "Pain" domains of the Clinical AKSS, WOMAC and SF-36.

	WOMAC Pain	SF-36 Pain
Clinical AKSS Pain	0.69	0.50
	p <0.001	p <0.001

Table 4.

Estimates of Person's linear correlation coefficients between the Functional AKSS and the "Function" domains of WOMAC and SF-36.

	WOMAC Function	SF-36 Functional Capacity
Functional KSS	0.36	0.56
	p < 0.001	p < 0.001

DISCUSSION

The purpose of this study was to analyze the reproducibility of the "American Knee Society Score (AKSS)"scale, in order to determine its reliability in the reproduction between different examiners (inter-examiner analysis), as well as the examination repeated by the same examiner (intra-examiner analysis). Contrary to many instruments used to evaluate post-TKA results, the AKSS scale has been the target of some validation studies.¹⁰ The variety of evaluation systems existing in literature hinders the comparison between the functional results of patients who have undergone TKA and the merits of surgery. The AKSS scale was proposed as a means of resolving the problem that arises when the deterioration of the patient's general health or the presence of comorbidities influence their functional state, even though the knee joint conditions are satisfactory after surgery.^11,16 Pollard et al.⁷ conducted a study that analyzed the instruments used to evaluate the state of health of individuals with knee osteoarthritis according to the WHO International Classification of Functioning, Disability and Health - ICF, which proposes a model that defines three main outcomes, based on consensus: lesion, functional limitation and restriction of activities. Thirteen questionnaires were evaluated and the authors noted that only the AKSS presents the items lesion and functional limitation separately and with measurement subscales, as defined by the ICF model, yet some authors criticize the AKSS, arguing that it has limited validation and shows low inter-examiner correlation between items. According to Gioe et al.¹³, the AKSS has been criticized as presenting biases such as the selection of patients by categories, confusing scale of pain with poor discriminatory capacity, exclusion of psychological factors in the evaluation and considerable variability in intra and inter-examiner reproducibility analyses. Liow et al.¹¹ verified that the examiner's experience influences the reliability of the AKSS, as it is an examiner-dependent instrument. In the present study, the AKSS scale was tested and evaluated by a group of 10 orthopedists. We discovered that the vast majority of these orthopedists reported that the items of the Clinical AKSS component, which refer to the physical examination, are considered subjective at the time of the evaluation, and can obtain different results according to the examiner's experience. Accordingly, the examiners of this study (two orthopedists) were trained and used a standardized method to gather data for the physical examination, in order to minimize the inter-observer variability. All the patients were evaluated in the same position and the same measurement techniques were used upon the physical examination. The observations of the "trained evaluators" are considered necessary when the reliability of a scoring system is evaluated. A weak inter-examiner concordance analysis can assuredly be attributed to the limited reliability of the instrument, and not to the examiners' heterogeneity. The results of this study demonstrated that the inter and intra-analysis reproducibility were strong for the total score of the two components of the AKSS scale. The results of internal consistency (Cronbach's coefficient) were strong for the inter and intra-observer correlation analyses in relation to the total score of both AKSS components; on the other hand, considerable inter and intra-examiner variations were observed in the analysis of the individual items of the Clinical AKSS component: there was good correlation for the items "Pain", "Range of Motion" and "Flexion contracture"; weak correlation for both items "Mediolateral stability" and "Extension deficit", and no correlation for the items "Anteroposterior stability" and "Alignment". In the Functional AKSS, there was good correlation for all the items analyzed individually. The items were analyzed separately according to the consensus of the "American Knee Society" group. There are no data available in literature indicating which criteria were used to develop the scale, which makes the validity of the AKSS questionable. Lingard et al.¹⁷ demonstrated poor correlation between the items of the Clinical AKSS, suggesting that a good score in one part of the scale may not reflect a good score in another, making final interpretation difficult. For example, a score of 80 points in the Clinical AKSS may be given to a patient without symptoms of pain, with range of motion from 0° to 25° of knee flexion, normal alignment, and without signs of joint instability, or to a patient who presents slight or occasional pain when climbing or descending stairs, 0° to 130° of knee flexion, normal alignment, and without signs of joint instability. These individuals clearly obtained considerably different results. Bach et al.⁶ demonstrated strong inter-observer correlation for the variables "Range of motion", "Flexion contracture" and "Extension deficit". All the variables were measured in the sagittal plane, using the simple goniometer. The lateral malleolus of fibula, lateral femoral condyle and greater trochanter were reference marks for guidance and measurement of the physical examination, performed in the same manner in this study. Low inter-observer correlation was also observed in the analysis of the "Alignment" item. A possible explanation considered by the authors is the difficulty of measurement, which was also observed by the authors of this study, since according to the AKSS, knee alignment should only be measured with the use of the goniometer. A line was required to measure the line from the center of the femoral head to the center of the patella up to the ankle. ^5,12

As the AKSS is calculated using a clinical scoring algorithm that includes both positive and negative items, statistically it is inappropriate to test the internal consistency of these values. In comparison, the WOMAC and SF-36 questionnaires are easier to interpret, since there is high internal consistency already proven scientifically and a strong correlation between items. Therefore, a patient with a score of 50 points in the "Pain" item of the WOMAC can be interpreted as an individual who presents, on average, moderate pain during activities. Likewise, a patient with 50 points in the "Functional Capacity" domain of SF-36 can have, on an average, a low level of limitation in the majority of activities.¹⁷ The construction of validity indicates whether the instrument correlates with other measurements or attributes that have an established relationship with the domains of interest. In analyzing constructive and discriminative criterion validity of the AKSS, we opted to compare it with other similar knee evaluation instruments, such as the WOMAC questionnaire and the generic quality of life questionnaire SF-36, yet as there is no scale for the clinical evaluation of the knee after TKA, it was not possible to conduct the analysis of comparison with the items of the Clinical AKSS component. The validity of the "Pain" item of the Clinical AKSS and of the Functional AKSS was established by the conclusion that they presented slight correlation with the analogous domains of the WOMAC and SF-36 questionnaires, since there is no gold standard evaluation instrument for TKA. There was strong correlation between the "Pain" items of the Clinical AKSS and WOMAC "Pain" (r = 0.69) and the "Pain" domain of SF-36 (r = 0.50). The better correlation between the AKSS and WOMAC than between the AKSS and SF-36 was expected, since the AKSS was created to be applied to patients with osteoarthritis or submitted to TKA, and the WOMAC questionnaire was specifically designed to evaluate patients with hip or knee osteoarthritis, disease of common basis in all patients of our sample, and the "Pain" domain of the SF-36 is a subjective evaluation of pain without specifying the affected site. The same results were demonstrated by the few similar studies found in literature.^4,17

As regards the Functional AKSS there was strong correlation with the "Functional Capacity" domain of SF-36 (r = 0.56) and slight correlation with the WOMAC "Function" (r = 0.36). A reason for this finding may be the difference between these items, in the Functional AKSS they are only related to the distance walked, capacity to climb and descend stairs and use of walking aids, while in the SF-36 half of the points are dedicated to the same activities, while the WOMAC evaluates other skills of the individual in addition to those presented.

However, this study presents some limitations that should be considered. The small sample size is not representative of the whole population of Brazilian patients with TKA. Although the questionnaires for evaluation of patients who have undergone TKA present certain limitations (the joint stability test is an example), they represent an important part of the armamentarium of professionals interested in the long-term results of the replaced joint.⁵ Orthopedic surgeons and health professionals should agree on a uniform method for evaluating the results of TKA.

CONCLUSION

The AKSS ("American Knee Society Score") scale is useful and reliable for evaluating individuals with osteoarthritis or submitted to TKA, demonstrating good measurements of psychometric properties. However, in the absence of AKSS validation studies, our results showed that the evaluations of the individual items of the Clinical AKSS component need further consideration, being performed by trained examiners, using standardized physical examination techniques, in order to minimize the possibility of biases.