The Knee Society Short Form Reduces Respondent Burden in the Assessment of Patient-reported Outcomes

Abstract

Background

The patient’s own evaluation of function and satisfaction is a fundamental component of assessing outcomes after total knee arthroplasty (TKA). The new Knee Society Knee Score was introduced in 2012 and has been shown to be a valid and reliable instrument for measuring the outcome of TKA. This score combines an objective, physician-derived component and a patient-reported component to characterize the expectations, satisfaction, and functional activities of diverse lifestyles of contemporary patients undergoing TKA. However, in the routine clinical setting, the administration and scoring of outcome measures is often resource-intensive, as the expenditure of time and budget for outcome measurement increase with the length and complexity of the instrument used, and so a short-form assessment can help to reduce the burden the assessment of outcomes.

Questions/purposes

The purposes of this study were (1) to develop a short-form version of the new Knee Society Knee Score; (2) to validate the short form against the full Knee Society Knee Score; and (3) to evaluate the responsiveness to treatment (TKA) of the new Knee Society short-form assessment.

Methods

To develop the short form, data from the sample of 497 patients recruited during validation of the original long form the new Knee Society Knee Score were used. The multicenter study was approved by the institutional review boards at 15 participating medical institutions within the United States and Canada. An analytic item reduction approach was applied simultaneously but separately to preoperative and postoperative patient-reported data to select a subset of items from the original form that had good measurement properties and closely reflected the scores obtained using the original form.

Results

Expectations and satisfaction were reflected by a single item in the newly developed short form compared with a total of five satisfaction and three expectation items in the long form. The functional activities subscale was reduced from 17 to six items. An excellent correlation was demonstrated between function scores derived from the functional activities subscale of the original long-form score (17 items) and the six-item short form (r = 0.97; p < 0.01). The sample mean difference between the two scores was less than 4 points with a SD of 6.7 points. The short form was capable of discriminating clinically different groups of patients before and after TKA with virtually the same estimated effect size as the original functional activities subscale of the new Knee Society Knee Score.

Conclusions

The Knee Society Knee Score long form is still recommended for research studies and for more sensitive measurement of the outcomes of individual patients. However, for general clinical use with large patient populations, the short form is expected to improve the rate of patient completion while also being easier to administer. In this study, we found the short-form version of the Knee Society Knee Score to be practical, valid, reliable, and responsive for assessing the functional outcome of TKA.

Electronic supplementary material

The online version of this article (doi:10.1007/s11999-015-4370-2) contains supplementary material, which is available to authorized users.

Introduction

After TKA, the patient’s perspective of improvement and satisfaction is fundamental to assessing the outcome of the procedure. The new Knee Society Knee Score was introduced in 2012 as a validated scoring system that combines an objective physician-derived component and a patient-reported component characterizing the expectations, satisfaction, and diverse lifestyles and activities of contemporary patients undergoing TKA [8, 12]. The new Knee Society Knee Score is broadly applicable across gender, age, activity level, and implant type. It is a highly responsive outcome measuring tool that may be applied in both the clinical and research settings to elucidate the profound variability of activity levels, function, and satisfaction after knee arthroplasty. However, in the routine clinical setting, the administration and scoring of outcome instruments are often resource-intensive with the expenditure of time and budget increasing in proportion to the length and complexity of the instrument itself. On the other hand, physicians are increasingly being held accountable for tracking outcomes as payers tie reimbursement to quality, cost, and outcomes metrics. This realization has prompted the development of shortened versions of many outcome instruments to reduce the financial and administrative burden of utilization while retaining, as much as possible, the reliability, validity, and responsiveness of the original [5, 14, 15].

In the case of the new Knee Society Knee Score, clinical experience has shown that a tradeoff exists among the length of the instrument, its utility in routine clinical use, and the incidence of questionnaires with incomplete responses. This speaks to the need for a shorter version of this instrument to facilitate inclusion of a greater proportion of patients undergoing knee arthroplasty at lower expense than has been possible in the past.

In this article, we (1) describe the development of a shortened version of the Knee Society Knee Score to facilitate patient-based assessment of outcomes in routine clinical practice (Fig. 1); (2) validate the short form against the full Knee Society Knee Score; and (3) evaluate the responsiveness to treatment (TKA) of the new Knee Society short-form assessment.

Fig. 1.

The Knee Society Knee Score Short Form is easier for the patient to complete. Reprinted with permission from The Knee Society, Copyright 2015 by The Knee Society.

Materials and Methods

Sample

To develop the short form, data were obtained from the sample of 497 patients recruited during validation of the original long form of the Knee Society Knee Score as part of a multicenter institutional review board-approved study at 15 medical institutions within the United States and Canada [8]. After consent was obtained, each patient completed a self-administered prototype version of the long-form instrument after orthopaedic consultations. Two hundred fifty-four patients (96 men, 158 women) were scheduled to undergo TKA on a date within 6 months of data collection and had not undergone a previous TKA, and 243 postoperative patients (108 men, 135 women) had undergone TKA at a minimum of 12 months (average, 25 months) before data collection. The average age of the study patients was 66 years for the preoperative group and 67 years for the postoperative group. Demographic characteristics such as age, gender, height, and weight were collected as part of the patient survey. Each study participant completed the prototype instrument in addition to two instruments that had undergone psychometric testing previously, the Knee Injury and Osteoarthritis Score (KOOS) and the SF-12 [7, 9, 10, 15].

Measures

The long-form Knee Society Knee Score contains the subscales of objective knee indicators (four items), symptoms (three items), satisfaction (five items), expectations (three items), and functional activities (19 questions forming 17 measurement items). The long-form questionnaire contains 19 questions; two of them form a single measurement item (Can you walk without any aids; and if no, which aids do you use?). One question (Do you use these aids because of your knees?) is not used in scoring of the long form. Therefore, item reduction was performed on the set of 17 items. The functional activity score is further divided into several components: walking and standing (five questions forming three measurement items), standard activities (six items), advanced activities (five items), and discretionary activities (three items).

Analytical Approach

The approach described subsequently was applied in parallel to preoperative and postoperative patient-reported data to select a subset of items from the original form that had good measurement properties and closely reflected the scores obtained using the original form. Based on the review by the subject matter experts, the three-item symptom subscale of the original form was kept in the short form. The five-item satisfaction subscale of the original form was reduced to a single item, which was chosen as most representative of the satisfaction subscale score based on the largest correlation with the sum of the responses to the five items forming the original satisfaction score.

Seventeen items from the functional activities subscale of the long form were first subjected to factor analyses to identify a one-dimensional subset of items. Mplus software (Muthen and Muthen, Los Angeles, CA, USA) was used to implement factor analyses with categorical indicators separately to preoperative and postoperative data to identify a common unidimensional subset of items. Root mean square error of approximation (RMSEA) and the left confidence limit of the 90% confidence interval for the RMSEA of < 0.05 indicate excellent fit.

The one-dimensional subset was then analyzed using the the Samejima graded response model that includes a discrimination parameter and location parameters for the response categories of each item. The Samejima model was fit in IRT PRO software, Version 2.1 (Scientific Software International, Skokie, IL, USA). In the parameterization used, excellent coverage is indicated by location parameters ranging from −3 to 3. Items with similar location parameters were considered for removal from the short form. In deciding which of these items should be retained, a larger discrimination parameter was preferred. When discrimination parameters were similar, subject matter expertise was used to select items to be retained in the short form. We note that the initial factor analysis of 17 items produced more than a single factor. Items not fitting into a single factor were removed through an iterative process using preoperative and postoperative samples. Nearly every time an item was removed, the factor structure changed. A single-factor solution was sought to have good factor analytic properties and to be acceptable to subject matter experts, and so this was the approach used.

After the candidate set of items was selected using the process described previously, the internal consistency reliability of the set of functional activities items was evaluated using Cronbach’s α. Correlation coefficients of the short-form scores with the scores from the long form, KOOS and SF-12 were computed. Responsiveness of the short form to the effect of treatment (TKA) was evaluated by comparing pre- and postoperative samples using t-tests and estimating the effect sizes expressed as Cohen’s d, the difference between means divided by the SD [1].

Results

Creation of the Short-form Knee Society Score

We were able to create a short-form assessment of satisfaction consisting of one item, satisfaction while performing light household activities, that correlated well with the five-item satisfaction subscale scores of the Knee Society Knee Score long form (r = 0.71 and 0.81 for pre- and postoperative, respectively, p < 0.01). The p value for differences between the pre- and postoperative satisfaction scores, as reflected by the single item, was < 0.01 with the effect size of 3.24. In other words, satisfaction with new knee function while performing light household activities was higher by more than 3 SDs after TKA compared with before TKA. This effect size was not statistically different from that obtained using the original five-item satisfaction subscale, namely 4.11. Based on these results, this item was deemed to be an excellent reflection of the original set of five satisfaction items and so was retained in the short-form version.

The process of the development of the short form occurred in several stages (Fig. 2). Initially, a set of eight items forming a unidimensional subscale of functional activities was selected from the original 17 items based on factor analysis. The item response theory (IRT) analysis further identified two items that could be removed (Appendix 1 [Supplemental materials are available with the online version of CORR^®.]). The remaining set of six items had excellent coverage of the underlying construct of functional activities: the item location parameters covered the range from −5.13 to 3.46 in the preoperative sample and from −3.03 to 3.21 in the postoperative sample. The wide range of item location parameters ensures that a wide range of function is captured from low levels of function to demanding functional activities.

Fig. 2.

The progression of stages in the development of the Knee Society Knee Score Short Form is shown.

Validation of the Short-form Score Against the Long-form Knee Society Score

The final set of six functional items was sufficiently unidimensional based on the RMSEA (Table 1). Cronbach’s α for the six-item short form of functional activities was very good: 0.78 in the preoperative sample and 0.82 in the postoperative sample. The correlations with the original subscale scores of the new Knee Society Knee Score all were near or exceeded 0.7 indicating high correlations (Table 1). The correlations with the total functional activities score from the original form were 0.92 in the preoperative sample and 0.93 in the postoperative sample, indicating excellent reflection of the original form by the short form. The correlations of the short-form score with the SF-12 and KOOS scores were very close to those obtained during validation of the long form [8]. The function scores calculated using the new Knee Society Knee Score (17 items) and the six-item short form showed strong linear association (r = 0.97; p < 0.01 for pre- and postoperative samples combined) (Fig. 3). The short-form score differed from the original score by a sample mean of less than 4 points with a SD of the difference of 6.7 points (Fig. 3).

Table 1.

Summary of the properties of the functional activities short form

Property of functional activities short form	Preoperative sample	Postoperative sample
RMSEA	0.093	0.034
RMSEA LCL	0.057	< 0.001
Cronbach’s alpha	0.78	0.82
Correlation with the long-form walking and standing	0.63	0.75
Correlation with the long-form standard activities	0.85	0.85
Correlation with the long-form original advanced activities	0.79	0.80
Correlation with the long-form original discretionary activities	0.71	0.68
Correlation with the long-form original functional activities score	0.92	0.93
Correlation with the SF-12 physical component score	0.47	0.61
Correlation with the SF-12 mental component score	0.28	0.15
Correlation with KOOS pain	0.70	0.62
Correlation with KOOS symptoms	0.34	0.33
Correlation with KOOS ADL	0.71	0.72
Correlation with KOOS sport/recreation	0.44	0.61
Correlation with KOOS quality of life	0.57	0.64

RMSEA = root mean square error of approximation; LCL = left confidence limit; KOOS = Knee Injury and Osteoarthritis Score; ADL = activities of daily living.

Fig. 3.

The regression of values of the patient function score is derived from the Knee Society Long Form function score (17 items) versus the Short Form function score (six items). DF = degrees of freedom; MSE = mean square error.

Responsiveness of the Short-form Scale to Treatment (TKA)

We found the short-form Knee Society score to be responsive to the effects of treatment (TKA) in that it could clinically discriminate across different groups of patients before and after TKA. Using the original total functional activities score, the difference between pre- and postoperative corresponded to the effect size of 2.31 (p < 0.01). When functional activities were measured with the six-item short form, the difference between pre- and postoperative samples corresponded to the effect size of 2.19 (p < 0.01).

Discussion

TKA has been shown to be a durable and predictable procedure in the treatment of knee arthritis as measured by survivorship analysis and long-term clinical studies. However, it has become increasingly important to measure patient satisfaction and functional activity after TKA. The Patient Protection and Affordable Care Act of 2010 created the Patient Centered Outcomes Research Institute that uses outcome assessment as a key component of comparative effectiveness research to determine best practices in medicine [4]. TKAs are increasingly being scrutinized by payers for cost-effectiveness and quality, and patient outcome measures and functional outcomes are important metrics for monitoring interventions and procedures. We therefore developed and validated the short form of the Knee Society Knee Score long form as a reliable and responsive patient-reporting instrument for assessing the outcome of TKA in clinical practice.

This study had several limitations. First the sample and data collected during the development and validation of the long form were used to derive the short form. If only items of the short form were administered, the respondent burden would have been less and some of the questions would have been preceded or followed by different questions, which could have influenced patient responses. Second, full evaluation of the validity was performed for the long form, and evidence of validity of the short form came from the high correlations of the short- and long-form scores. Third, floor and ceiling effects were not specifically evaluated. Defining floor and ceiling effects of functional activities in the present population remains a direction for future work, which would be facilitated by the availability of the short form. Finally, future work may include imputation rules for missing data in the short form, although a substantially smaller number of items would likely result in less missing data for the short form compared with the long form.

The Knee Society Knee Score long form was originally developed to increase the scope and relevance of the original Knee Society Knee Score by including items addressing patient satisfaction and fulfillment of patients’ expectations without the ceiling effects seen in older instruments when administered to younger and more active patients. The new knee score also queried the ability of each patient to participate in a broad range of activities encountered in daily living, exercise, recreation, sporting activities, and those activities of greatest personal importance as well as measuring patient satisfaction and expectations. The new score was developed according to rigorous psychometric principles using preliminary patient interviews, an initial survey of 100 post-TKA patients, and, finally, a large cohort of nearly 500 preoperative and postoperative patients recruited from surgical centers throughout the United States and Canada. Analysis of the validation process reported elsewhere demonstrated its reliability, validity (face, content, and construct), and its responsiveness to clinical change [8].

The focus of the new Knee Society Knee Score was to include sufficient range and diversity of items that knee-related outcomes could be reliably measured in the diverse spectrum of patients undergoing treatment of knee disorders without differential effects resulting from the age or gender of the patient. To make this feasible, the instrument consisted of 34 items in five domains. The resulting instrument succeeds in providing a comprehensive description of the facets of each patient’s knee outcome, which is particularly useful in tracking changes experienced by individual patients over time. However, this sensitivity and accommodation of patient diversity comes at a cost. First, a large number of items is required to ensure that patients of all lifestyles and physical abilities are accommodated, especially if the instruments are to assess the extent to which patients are able to perform voluntary, discretionary activities in addition to the activities of daily living.

Many instruments now address this challenge through the use of computerized adaptive testing strategies, in which items are selected by a computer algorithm from a large item pool based on each successive response. However, this approach requires prevalidation of a large set of items, typically through testing many hundreds (or thousands) of patients. It also restricts use of the instrument to settings with ready access to a computer and/or the Internet. In generating the short-form version of the Knee Society Knee Score, responses to items on the long form were condensed to include the minimum set of items required to reflect the variation captured by the longer instrument. Using the sample available for analysis, it was possible to achieve this replication after condensing the original satisfaction and expectation subscales to just one satisfaction item and the original 17 function items to only six. A 100-point short-form Knee Society Knee Score (Table  2) derived from these items was found to be highly correlated with the original long-form score with a 95% confidence interval of less than ± 15 points for predicting the long-form score from the responses to the short-form version.

Table 2.

The Knee Society Knee Score Short Form has six activities with a maximum score of 100

Activity	Maximum point score
How long can you walk	20
Walking on an uneven surface	15
Climbing or descending stairs	15
Getting up from low couch or chair without arms	15
Running	20
Discretionary activity	15
Maximum total	100

A second consideration is the increased incidence of incomplete responses to items relating to activities that are not universal or are physically demanding [6, 13, 16]. By definition, discretionary or demanding activities will be performed by only a segment of the respondent pool, unlike the activities of daily living, which are performed by almost everyone. The result is that some patients will respond that they “do not do this activity,” not because of inability, but because of lifestyle. This leads to discrepancies between different patient populations in terms of participation rates, potentially lessening the value of the outcomes measurement. This deficiency was a criticism of the Knee Society Knee Score long form with a significant proportion of patients failing to complete items relating to the ability to carry heavy loads (eg, shopping bags) and items requiring the respondent to identify three knee activities they consider personally important [2, 3]. Because patients who do not respond to followup surveys, or do not complete the survey, tend to report significantly poorer outcomes than do responders [11], this potential source of response bias should be considered in evaluating the results of patient-reported outcome measures. Missing values from various patient-reported outcome measures occur in as many as 5% to 30% of cases, depending on various factors such as the patient population, the length of the instrument, and the “questionnaire burden” imposed on each patient [11]. Because many statistical techniques used for analysis of outcomes data require complete data sets and analyses that incorporate missing data are often based on nontestable assumptions, instrument developers (eg, the SF-12) impute missing values using different statistical methods as a result of the inherent correlation between responses to items [6, 13, 16]. However, as these instruments become shorter through selection of independent items, imputation becomes less reliable. Moreover, our desire to attain 100% response rates in very large series of patients requires not only that a short form be valid, but that completion be as universal as possible.

We found the short-form version of the Knee Society Knee Score to be a practical, valid, reliable, and responsive patient-reporting tool for assessing both patient function and satisfaction after TKA. The short form provides a brief measure for three domains of the long form and can be used for monitoring patient outcome after TKA in research studies or clinical practice. The Knee Society Knee Score long form is still recommended for research studies and for more sensitive measurement of the outcomes of individual patients. However, for general clinical use with large patient populations, the short form is expected to improve the rate of patient completion while also being easier to administer.

Electronic supplementary material

Below is the link to the electronic supplementary material.