Confounding pain and function: the WOMAC's failure to accurately predict lower extremity function

Abstract

Background

Investigations have revealed the Western Ontario and McMaster Universities Osteoarthritis Index’s (WOMAC) inability to provide distinct assessments of pain and function. The Lower Extremity Functional Scale (LEFS) has not displayed this deficiency. Our purposes were to investigate further the WOMAC physical function's (WOMAC-PF) ability to accurately assess lower extremity mobility in patients undergoing total knee arthroplasty (TKA) and to establish a relationship between pre- and post-TKA WOMAC-PF and LEFS scores that accounts for the apparent bias WOMAC pain scores impose on WOMAC-PF scores.

Methods

WOMAC, LEFS, and Timed-up-and-go measures were administered before TKA and 4 days, 6 weeks, and 3 months after TKA. To evaluate the WOMAC-PF and LEFS ability to provide a distinct assessment of pain and function, a paired t-test compared pre-TKA and 4 days after TKA values. Generalized estimating equation (GEE) analysis assessed the relationship between pre- and post-TKA values: dependent variable WOMAC-PF scores; independent variables LEFS scores, and measurement occasions.

Results

Timed-up-and-go and LEFS demonstrated a reduction in lower extremity function (P < .001); pain decreased (P < .001); and there was no significant change in WOMAC-PF scores (P = .61). GEE analysis revealed a linear relationship between WOMAC-PF and LEFS with similar slope coefficients for all four occasions. The relationship between WOMAC-PF and LEFS scores was virtually identical for the postarthroplasty assessment occasions.

Conclusions

Our findings support previous investigations that showed the WOMAC-PF’s inability to provide a valid assessment in change in function. The GEE analysis coefficients can be used to convert LEFS scores to WOMAC-PF scores that adjust for the bias between pre- and post-TKA assessments.

Introduction

Pain and decreased lower extremity function—defined as the ability to move around [1]—are two important determinants of knee arthroplasty in persons with osteoarthritis of the knee. Recognizing their importance, Outcome Measures in Rheumatoid Arthritis Clinical Trials III identified pain and function as two of four core outcomes to be assessed independently in phase III trials [2]. Further support for the independent assessment of pain and function is found in the Outcome Measures in Rheumatoid Arthritis Clinical Trials-OsteoArthritis Research Society International responder criteria that recommend moderate improvement in two of the following three categories: (1) pain, (2) function, and (3) patient’s global rating [3]. The Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) [4], both in its original and embedded form in the Knee injury and Osteoarthritis Outcome Score [5], is one of the most widely used patient-reported outcome measures applicable to patients with osteoarthritis of the knee undergoing arthroplasty. However, a number of studies have consistently demonstrated the WOMAC’s inability to distinguish between pain and function [6], [7], [8], [9], and in an evidence-based era, the continued prominence of this measure is puzzling.

There are several possible explanations for this practice. First, we suspect that conformity is an influential determinant of the WOMAC’s continued notoriety. For example, the WOMAC is often included in joint registries, and this practice facilitates the comparison of patients’ outcomes over time as interventions evolve. A second explanation is a lack of awareness of both the inability of the measure to distinguish between pain and function and the consequence of this deficiency. For example, studies supporting the validity of the WOMAC’s ability to assess function and change in function [10], [11], [12], [13], [14] frequently, if not uniformly, do not cite studies that challenge the measure’s ability to accurately assess function. Also, study designs supporting the WOMAC’s ability to detect valid change in pain and function after arthroplasty typically select reassessment intervals (ie, >2 months after arthroplasty) where pain and function are expected to display similar change trajectories, thus masking the measure’s ability to provide a unique and accurate assessment of each [10], [11], [12], [13], [14]. A consequence of not being able to distinguish pain and function is that a clinician may form an inaccurate impression of a patient’s status, particularly as it applies to mobility. For example, Parent et al [8] reported a 20-point improvement in WOMAC physical function (WOMAC-PF) scores when measured 2 months after arthroplasty compared with a 39-meter decrease in 6-minute walk distance and similar gait speed and stair ascent times compared with preoperative values. Calatayud et al [15] reported slower Timed-up-and-go (TUG) and stair test times but a significant improvement (ie, ≥9 points) in WOMAC-PF scores 1 month after arthroplasty compared with preoperative values. Stratford et al [16] have found similar WOMAC-PF scores before and 16 days after arthroplasty assessments; however, time to complete performance measures (ie, timed stair test, TUG test, and a self-paced walk test) increased more than double. To determine the extent to which WOMAC pain and function subscales provide distinct assessments, a study design that provides noticeably different change trajectories for pain and function is required.

One design would be to take advantage of the natural or clinical history after total joint arthroplasty (TKA) [17]. For example, a number of studies have reported a significant increase in performance measure times and a reduction in pain when early postarthroplasty values are compared with preoperative values [16], [18], [19], [20], [21]. However, investigations of the WOMAC-PF and its embedded version in the Knee injury and Osteoarthritis Outcome Score (ADL scale) have shown an inability to detect significant deterioration in mobility over this period [15], [16], [17], [21], [22]. In contrast to the WOMAC-PF’s limited ability to provide an accurate representation of lower extremity mobility, the Lower Extremity Functional Scale (LEFS) has been able to detect significant reductions in the ability of patients to move around in the early weeks after arthroplasty [16], [17].

Our purposes were to contribute further information concerning the WOMAC-PF’s ability to accurately assess lower extremity mobility in patients undergoing TKA,and to determine whether a relationship between pre- and post-TKA WOMAC-PF and LEFS scores could be established that takes into account the apparent bias WOMAC pain scores impose on the interpretation of WOMAC-PF scores. The intent of the latter goal was to assess the feasibility of converting LEFS scores to WOMAC-PF scores accounting for the limited ability of WOMAC-PF scores to accurately represent mobility distinct from pain.

Material and methods

Participants

Participants were those who took part in a randomized clinical trial which was approved by the Sunnybrook Health Sciences Centre (Toronto, Canada) research ethics board that examined the effect of perioperative gabapentin on patients undergoing TKA [23]. Patients were eligible for the trial if they were between the ages of 18 and 75 years; had an American Society of Anesthesiologists score of I, II, or III; and provided written informed consent. Patients were ineligible if they had a known allergy to medications being used, a history of drug or alcohol abuse, a history of being on chronic pain medications, rheumatoid arthritis, a psychiatric disorder, a history of diabetes with impaired renal function, a body mass index > 40, or were unable or unwilling to use a patient-controlled analgesia pump [23].

Study design

The original randomized clinical trial’s purpose was “to examine whether, in the context of preoperative spinal anesthesia, femoral and sciatic nerve blocks, and celecoxib coadministration, a 4-day perioperative regimen of gabapentin vs placebo improves knee function on performance and self-reported measures of physical function, and movement evoked pain on postoperative day 4 and at 6 weeks and 3 months after surgery” [23]. Given there was no between-group difference in performance or self-reported outcomes, the present study viewed the entire sample as a single group that was assessed at four fixed occasions (before arthroplasty, at 4 days, at 6 weeks, and at 3 months). The TUG test was applied to obtain a performance-based assessment of change in the patients’ abilities to move around before and 4 days after arthroplasty. Similarly, WOMAC-PF and LEFS change scores taken before and 4 days after arthroplasty were compared. The extent to which a consistent relationship existed between WOMAC-PF and LEFS scores was assessed by comparing the association between their scores at each of the four measurement occasions.

Outcome measures

Timed-up-and-go

The TUG is an OsteoArthritis Research Society International-recommended performance test [24], [25]. Patients start seated in a chair and are required to rise, walk 3-meters, turn, return to the chair, and sit down. The outcome is the time to perform the test in seconds. The minimal detectable within-patient change is approximately 2.5 seconds [26].

Western Ontario and McMaster Universities Osteoarthritis Index (LK3.1 version)

The WOMAC is a patient-reported measure conceived for persons with osteoarthritis of the lower extremity. It has subsequently been used frequently to assess patients before and after arthroplasty [1].

The pain subscale consists of 5 items each scored 0 to 4. Total scores can vary from 0 to 20, with lower scores representing lower pain levels. The minimal within-patient change is approximately 4 points [27]. The physical function subscale consists of 17 items each scored 0 to 4. Total scores can vary from 0 to 68, with lower scores representing higher levels of functional status. The minimal detectable within-patient change is 9 points [28].

Lower Extremity Functional Scale

The LEFS is a patient-reported measure of lower extremity functional status [29]. It was designed to be applicable to persons with a spectrum of lower extremity problems and ability levels. Validation studies have supported its use in a variety of populations including patients with sports injuries, stroke, osteoarthritis, and hip or knee arthroplasty [28], [30], [31], [32], [33], [34], [35]. The LEFS consists of 20 items with each scored 0 to 4. Total scores can vary from 0 to 80, with higher scores representing higher levels of functional status. The minimal detectable within-patient change is approximately 9 points [29].

Statistical analysis

We performed a secondary analysis of data obtained from a clinical trial reported previously [23]. Our sample size was one of convenience and dictated by the clinical trial. Descriptive statistics were summarized as frequency counts for categorical data, mean and standard deviation, or median and first and third quartiles if the data were skewed. To address our first purpose, we applied a paired t-test or Wilcoxon’s Signed Rank test to test for a difference between prearthroplasty and 4-day postarthroplasty measurements. We applied a generalized estimating equation (GEE) analysis for longitudinal data to address our second purpose which examined the relationship between WOMAC-PF and LEFS scores across occasions. The dependent variable was WOMAC-PF scores, and the independent variables were LEFS scores and occasions at 4 levels (before arthroplasty, after arthroplasty at 4 days, at 6 weeks, and at 3 months). Our model-building approach was as follows: (1) establish the relationship between WOMAC-PF and LEFS scores (eg, linear or polynomial); (2) test for a LEFS-by-occasion interaction; and (3) identify the most appropriate covariance structure. After the final model was established, we evaluated the stability of the relationship by testing whether the occasion-specific regression lines were coincident, parallel but not coincident, or not parallel. Analyses were performed in STATA v15.1 (StataCorp, College Station, TX), and an effect was considered statistically significant at P < .05 (95% confidence interval excluded zero).

Results

Participants

The sample size for this study consisted of 176 patients with an equal distribution of males and females. The sample’s mean age (standard deviation) and body mass index were 62.9 (6.8) kg and 31.7 (5.4) kg/m², respectively. Further details concerning the sample can be found in the previously reported clinical trial [23].

Change assessment

Table 1 provides a summary of the measures' scores at each of the four occasions. A comparison of prearthroplasty and 4-day postarthroplasty scores revealed a significant increase in TUG times ($\text{mean}\text{difference}\overline{\text{d}}=29.0\text{s};95\text{\%}\text{CI}:24.2-33.9;$P < .001) and decrease in LEFS scores ($\overline{\text{d}}=12.3;95\text{\%}\text{CI}:8.6-16.0;$P < .001), both representative of a reduction in lower extremity functional status. A comparison of WOMAC pain scores showed a decrease in pain ($\overline{\text{d}}=1.5;95\text{\%}\text{CI}:0.8-2.1;$P < .001) and no appreciable change in WOMAC-PF scores ($\overline{\text{d}}=0.7;95\text{\%}\text{CI}:1.9-3.3;$P = .61). Given the sample sizes were different for the LEFS and WOMAC-PF (Table 1), we recalculated the change estimates for the identical sample and obtained a similar result (WOMAC-PF: $\overline{\text{d}}=1.1;95\text{\%}\text{CI}:2.1-4.3;$ P = .50; LEFS: $\overline{\text{d}}=11.9;95\text{\%}\text{CI}:7.8-16.0;$P < .001).

Table 1.

Occasion-specific summary statistics.

Measure	Measurement occasion
	Preoperative mean (SD), n	Day 4 mean (SD), n	6 weeks mean (SD), n	3 months mean (SD), n
TUG (s)	12.4 (4.6), 176 11 (9, 13)a	41.3 (28.9), 137 32 (22, 49)	13.7 (7.2), 159 11 (9, 15)	10.5 (5.2), 150 9 (8, 11)
WOMAC pain/20	9.8 (3.1), 172	8.4 (3.1), 153	6.8 (3.2), 160	3.8 (3.3), 161
WOMAC-PF/68	33.5 (11.0), 172	34.5 (9.4), 95	22.0 (12.2), 153	14.4 (11.3), 157
LEFS/80	29.1 (11.4), 172	17.5 (11.4), 58	35.6 (16.2), 136	48.4 (14.3), 148

SD, standard deviation.

^aMedian (first and third quartiles).

Relationship between WOMAC-PF and LEFS

Table 2 summarizes the GEE results that applied an exchangeable covariance structure. This analysis revealed a linear relationship between WOMAC-PF and LEFS scores. There was no evidence of a LEFS-by-occasion interaction (P = .76); all regression lines were parallel. However, a significant difference was noted among occasions (P < .001), and a contrast analysis showed that all postarthroplasty occasion coefficients differed from the prearthroplasty coefficient by approximately 7 WOMAC-PF points. A further contrast analysis revealed that the three postarthroplasty coefficients did not differ (P = .90); these three regression lines were coincident. Given there was no difference among the three postarthroplasty regression coefficients, we dichotomized the occasion variable to prearthroplasty and postarthroplasty and reran the GEE analysis as described previously. These results are also reported in Table 2 and used to generate Figure 1. In addition to showing the relationship between prearthroplasty and postarthroplasty WOMAC-PF and LEFS scores, the figure also includes the 95% confidence bands (shaded area around each line). The confidence bands convey the likely location of the WOMAC-PF population's mean score for a given LEFS score.

Table 2.

Generalized estimating equation coefficients.

Source	4 Occasions		Dichotomized occasions
	Coefficients (95% CI)		Coefficients (95% CI)
LEFS	−0.61 (−0.66, −0.56)	LEFS	−0.61 (−0.65, −0.57)
Occasion		Occasion
Day 4	−7.14 (−9.52, −4.75)	Postoperative	−6.81 (−8.14, −5.47)
6 Weeks	−6.62 (−8.19, −5.05)
3 Months	−6.86 (−8.67, −5.05)
Constant	51.17 (49.32, 53.01)	Constant	51.13 (49.48, 52.78)

CI, confidence interval.

Figure 1.

Pre–Post arthroplasty regression lines with 95% confidence bands.

Discussion

A test or measure is useful to the extent that it allows valid inferences to be drawn from its measured values. The purpose of the current manuscript was to contribute information concerning the WOMAC-PF’s ability to accurately assess lower extremity mobility as defined by the WOMAC-PF and to determine whether a relationship between pre- and post-TKA WOMAC-PF and LEFS scores could be established that takes into account the apparent bias WOMAC pain scores impose on the interpretation of WOMAC-PF scores. When assessed 4 days after arthroplasty, we found that WOMAC-PF scores did not detect deterioration in lower extremity mobility compared with prearthroplasty scores. In contrast, we found strong evidence of decreased mobility based on a marked increase in TUG times and a significant reduction in LEFS scores. To be clear, we were not interested in what happens specifically between pre- and 4-day post-TKA, but rather applied these time points to expose a deficiency which cannot be disentangled when pain and function have similar change trajectories. With respect to our second purpose, we found a linear relationship between WOMAC-PF and LEFS with a postarthroplasty WOMAC-PF score being approximately 7 points less than the prearthroplasty score for a given LEFS value. The relationship between WOMAC-PF and LEFS scores was virtually identical for the three postarthroplasty assessment occasions.

Our finding that the WOMAC-PF was not capable of detecting deterioration in function in the early days after arthroplasty is consistent with the results of previous investigations [15], [16], [21], [22]. It is likely that the compromised ability of the WOMAC-PF to accurately represent a change in mobility is not a function of its items alone but rather of the structure of the measure and the similarity of pain and function item content. Approximately one-half of the WOMAC-PF items address activities similar to those included in the WOMAC pain subscale, which patients encounter first when completing the measure. Given the similarity of item phrasing and content, we suspect that responses to pain items bias responses to similar function items. Support for this premise is found in a previous study that examined the ability of two subsets of WOMAC-PF items to address change using a study design similar to that of the current investigation. Patients were assessed before arthroplasty and within 16 days after arthroplasty [36]. WOMAC-PF items were divided into 8 items with content similar to that of pain items (ie, descending and ascending stairs, rising from sitting, standing, walking on a flat surface, rising from bed, lying in bed) and 8 items dissimilar to pain items' content (bending to the floor, getting in or out of a car, going shopping, putting on your socks or shoes, getting in or out of the bath, getting on or off the toilet, performing heavy domestic duties, performing light domestic duties). Three performance measures (TUG, stair test, self-paced walk) were also applied. Consistent with the results from the performance measures, the sum of the dissimilar 8 items demonstrated a significant decrease in function after arthroplasty (ie, higher WOMAC scores), whereas the sum of the similar 8 items showed a modest improvement in function [36].

The longitudinal analysis revealed similar slope coefficients for all four measurement occasions, suggesting a stable relationship between WOMAC-PF and LEFS scores. However, a systematic difference of approximately 7 WOMAC-PF points was noted between prearthroplasty and postarthroplasty values for a given LEFS score. The interpretation of the occasion-specific regression coefficients reported in Table 2 (ie, −7.14, −6.62, −6.86) is that the bias identified between prearthroplasty and 4-day postarthroplasty values is, likely owing to the influence of pain, the same at 6 weeks and 3 months. Thus, although it was not possible to disentangle a change in pain from a change in function at 6 weeks and 3 months because both are expected to improve, the extent to which a bias exists was consistent with the 4-day assessment. We are unaware of previous investigations that have modeled the relationship between WOMAC-PF and LEFS scores in a context similar to our study. A consistent slope coefficient and bias exists between prearthroplasty and postarthroplasty WOMAC-PF and LEFS scores, allows for a simple conversion of LEFS scores to WOMAC-PF scores (ie, WOMAC-PF = 51.1 – 0.6 [LEFS] – 6.8 [1 if after arthroplasty]). Also, given the narrow width of the confidence bands, our results suggest the conversion of LEFS scores to mean WOMAC-PF scores can be done with a high level of confidence.

The following vignette is offered to illustrate how information from Table 2, and the figure can be applied. A future investigator conducts a longitudinal study of patients undergoing TKA and applies the LEFS as the only patient-reported outcome measure of function. LEFS mean scores preoperatively, at 6 weeks, and at 3 months, were 30, 35, and 50, respectively. These values convert to WOMAC-PF scores of 33, 23, and 14, respectively, which can be compared to mean values reported in historical publications and joint registries.

A limitation of our study is that it represents a secondary analysis of data obtained from a clinical trial. As such, LEFS measurements were not mandated on day 4, and this resulted in an imbalance in measure-specific sample sizes. With respect to the study component that examined the measures’ abilities to detect deterioration between prearthroplasty and 4-day postarthroplasty, we supplemented our analysis with one that contained identical samples (n = 58). Our results showed that the WOMAC-PF was unable to detect deterioration and that the upper 95% confidence limit on the change estimate did not include the value of a clinically important change which has been estimated to be 9 points [28]. In contrast, the LEFS was able to detect a deterioration in function, and the point estimate of change exceeded the clinically important change value of 9 points (lower 95% confidence limit was approximately 8) [29]. The interpretation of these confidence limits is that despite the sample size, there is strong evidence to suggest that the WOMAC-PF cannot detect an important deterioration in function when WOMAC pain improves; however, the LEFS can detect worsening. The smaller LEFS sample size at day 4 affected the longitudinal data analysis such that a wider slope coefficient confidence interval was noted for this occasion than the other postarthroplasty measurement occasions (Table 2). Also, our results do not discern whether the prearthroplasty WOMAC-PF score is inflated owing to pain or the postarthroplasty score “optimistically” improved because of a reduction in pain.

Conclusions

In summary, our results relating to the WOMAC-PF’s inability to provide a distinct assessment of a patient’s ability to move around is consistent with previous reports, and this deficiency may lead to invalid inferences being drawn from a score or change score. Our findings also showed a consistent relationship (ie, slope coefficient) between WOMAC-PF and LEFS scores that was biased by approximately 7 WOMAC-PF points (ie, occasion coefficient) after arthroplasty. This relationship allows a transformation of LEFS scores to WOMAC-PF scores that accounts for the bias between pre- and post-TKA assessments. Given this is the first study to investigate the relationship between WOMAC-PF and LEFS scores before and after arthroplasty, further investigations are necessary to support our findings. Finally, it is essential to stress that measurement properties are context specific. Accordingly, future investigations are required to determine whether our findings are generalizable to other surgeries such as hip replacement.