Minimal Clinically Important Differences of PROMIS PF in Ankle Fracture Patients

Abstract

Background:

Evaluating the minimal clinically important differences (MCIDs) in patient-reported outcome scores is essential for use of clinical outcomes data. The purpose of the current study was to calculate MCID of Patient Reported Outcome Information System Physical Function (PROMIS PF) scores for ankle fracture patients.

Methods:

All patients who underwent operative fixation for ankle fractures at a single level 1 trauma center were identified by Current Procedural Terminology code. PROMIS PF scores were collected. Patients had to complete an anchor question at 2 time points postoperatively to be included in this study. Anchor-based and distribution-based MCIDs were calculated

Results:

A total of 331 patients were included in the distribution-based analysis, and 195 patients were included in the anchor-based analysis. Mean age was 45.3 years (SD 17.5), and 59.4% of participants were female. MCID for PROMIS PF scores was 5.05 in the distribution-based method and 5.43 in the anchor-based method.

Conclusion:

This study identified MCID values based on 2 time points postoperatively for PROMIS PF scores in the ankle fracture population. Both methods of MCID calculation resulted in equivalent MCIDs. This can be used to identify patients outside the normal preoperative and postoperative norms and may help to make clinically relevant practice decisions.

Level of Evidence:

Level I, diagnostic study, testing of previously developed diagnostic measure on consecutive patients with reference standard applied.

Introduction

Patient-reported outcomes have gained traction in both research and clinical settings to evaluate orthopedic surgical treatment efficacy from a patient-centered perspective. The Patient Reported Outcome Measurement Information System (PROMIS) was designed by the National Institutes of Health (NIH) to measure outcome scores in various health domains.^2,3 PROMIS instruments are designed to result in a mean score of 50 and an SD of 10 in the general population.¹⁰ PROMIS has been found to have adequate precision and coverage in several orthopedic populations, but the contribution to clinical decision making is unclear. Minimal clinically important difference (MCID), the change below which there is no clinical significance to the patient, is a method that can link the score to a clinical decision.¹²

PROMIS physical function (PF) MCID scores have been reported for several orthopedic conditions.^1,5,7 The MCIDs for PROMIS domains in orthopaedic conditions range from 2 to 8.^6,8 The purpose of this study was to determine PROMIS PF MCID values for patients with acute, operatively treated rotational ankle fractures.

Materials and Methods

Patients treated at a single tertiary level 1 trauma center between January 2014 and December 2020 were identified using Current Procedural Terminology (CPT) codes for ankle fractures (CPT 27766, 27769, 27792, 27814, 27822, and 27823). A total of 1228 unique patients were identified. Demographic information including age on admission, sex, BMI, smoking status, American Society of Anesthesiologists class, and Charleston Comorbidity Index were collected. Patients without PROMIS v1.2 CAT Physical Function (PF) scores (600 patients, nonresponder group) were excluded, resulting in 628 patients. Patients were required to have scores at 2 time points postoperatively defined as a PROMIS PF score <7 weeks postoperatively, as well as a follow-up score within >7 weeks and 18 months from original surgery. Postoperative scores are typically from the first outpatient visit at 2 weeks after surgery. After excluding patients without 2 time points, 331 patients were available for analysis. Of the 331 patients, 195 had recorded responses to our anchor question. Distribution-based and anchor-based methods of calculating MCIDs were used.

Statistical Analysis

Descriptive statistics were used to summarize patient demographic characteristics in the study cohort. We also compared the PROMIS score at both the first postoperative visit and the follow-up period. The distributions of PROMIS scores at both time points were summarized and compared by the response in the anchor question. The anchor-based and distribution-based MCIDs were calculated as follows below.

Anchor-based MCID.

The question of interest is as follows: “Compared to your first evaluation, how would you describe your physical function level now?” The responses were split into Improvement and No Improvement groups. Improvement group includes the responses “slightly improved,” “improved,” and “much improved.” No Improvement group includes “no change,” “slightly worse,” “worse,” and “much worse.” The MCID was defined by the average score difference between these 2 groups.

Distribution-based MCID.

The distribution based MCID was calculated using a half SD of the change in the PROMIS PF score. The half SD method is a common distribution-based MCID definition in the literature.^9,11

All these statistical analyses were performed using statistical software R 4.1.0 (R Development Core Team, Vienna, Austria).

Results

A total of 331 patients met inclusion criteria for operative ankle fractures with sufficient PROMIS PF scores at postoperative baseline and follow-up. One hundred thirty-six of these patients did not have a recorded response to the anchor question and were therefore excluded from our anchor-based MCID analysis. Three hundred thirty-one patients were included in the distribution-based MCID analysis. One hundred ninety-five patients were included in the Anchor-based MCID analysis (Figure 1).

Figure 1.

Consort diagram.

Patient Characteristics

For responders, 59.4% of the patients were female, with a mean age of 45.3 (SD 17.5). The mean BMI was 28 (SD 8.5). Most patients never smoked (57.6%) and had mild systemic disease (47.6%) (Table 1). The average time from surgery to the postoperative baseline assessment was 2.5 weeks (SD 1.7, range 0–6.9). The average time from surgery to follow-up assessment was 28 weeks (median 12.14, range 7–78). There was overall improvement in the PROMIS PF score from postoperative baseline (28.17) to follow-up (39.89) periods (Table 2). The nonresponder group had a greater percentage of male patients, was older, and had greater American Society of Anesthesiologists grade and Charleston Comorbidity Index scores than the responder group (Table 3).

Table 1.

Patient Characteristics by First Follow-up Anchor Responses.

Characteristics	Overall	Improvement	No Improvement	P Valuea
No. of patients	195	163	32
Female, %	61.5	60.1	68.8	.429
Age, mean (SD)	44.9 (17.8)	43.1 (17.1)	54 (18.8)	.003
BMI, mean	27.1	32.1	30.7	.892
Smoker, %	12.8	14.1	6.2	.440
ASA class 3, %	23.6	20.2	40.6	.115
CCI score, mean (SD)	1.2 (1.9)	1.1 (1.7)	2 (2.5)	.055

Abbreviations: ASA, American Society of Anesthesiologists; BMI, body mass index; CCI, Charlson Comorbidity Index.

^aP values were calculated using the Wilcoxon rank sum test for continuous variables and Fisher exact test for categorical variables. Boldface indicates significance.

Table 2.

Comparison in Scores by Postoperative Baseline and Follow-up in Distribution Group.

	Scores Included in Calculation	Postoperative Baseline	Follow-up	Change	P Valuea
Overall	1197	28.17 (6.8)	39.89 (8.94)	11.72	<.001

^aP value was calculated based on the Wilcoxon signed rank test. Boldface indicates significance.

Table 3.

Demographic Comparison between Ankle Fracture Patients with and without PROMIS Physical Function Scores.

Demographic Variable	Responder	Nonresponder	P Value
Number of patients	628	600
Female, %	59.4	45.8	<.001
Average age, mean (SD)	45.3 (17.5)	40.6 (16.6)	<.001
Never smoker, %	58.6	56.5	.06
ASA class 1, %	27.4	31.5	.015
CCI score, mean (SD)	1.3 (2.1)	0.6 (1.5)	<.001

Abbreviations: ASA, American Society of Anesthesiologists; CCI, Charlson Comorbidity Index; PROMIS, Patient Reported Outcome Information System

Anchor-Based MCID

There were 195 patients with 435 score observations in the Anchor-based MCID analysis; 163 (84%) improved and 32 (16%) did not improve. There was significantly more improvement in the score in the Improvement group (11.8) compared to the No Improvement group (6.3) when using the anchor-based method (P < .001) (Table 4). The Anchor-based MCID was 5.43 with an SD of 13.41. In the improvement group, 20.2% of patients had severe systemic disease compared with 40.6% in the No Improvement group (P = .115). CCI was also higher in the No Improvement group (2 vs 1.1 in the Improvement group) (P = .055).

Table 4.

Comparison in Scores by Postoperative Baseline and Follow-up in Anchor Group.

	Overall	Improvement	No Improvement	P Valuea
No. of scores	435	356	79
Postoperative baseline score, mean (SD)	28.47 (7.1)	28.64 (7.06)	27.68 (7.25)	.046
Follow-up score, mean (SD)	39.26 (9.2)	40.42 (8.9)	34.02 (8.87)	<.001
Score change, mean (SD)	10.79	11.78 (10.07)	6.34 (8.86)	<.001

^aP values were calculated based on the Wilcoxon signed rank test. Boldface indicates significance.

Distribution-Based MCID.

There were 331 patients available for the distribution-based analysis using 1197 patient score observations. Using the formula MCID = 0.5 × SD, the distribution-based MCID was 5.05. The mean postoperative early time point score in this cohort was 28.17, with an SD of 6.8, and the mean follow-up time point score was 38.89 with an SD of 8.9 (P < .001).

Discussion

This study identifies the MCID for PROMIS PF in the setting of operative ankle fractures as 5.05 (distribution based) or 5.43 (anchor based). The MCID calculation methods result in essentially identical MCIDs for the PROMIS PF in this setting. The similarities in the MCID regardless of the calculation method lend confidence to the accuracy of these results. Ultimately, use of multiple approaches to identify a potential range is the optimal solution.¹¹ These MCID values can be used to design an appropriately powered clinical trial to detect a clinically important difference in PROMIS PF scores between ankle fracture patients .

Gausden et al⁴ evaluated PROMs in unstable ankle fractures comparing PROMIS PF, PROMIS LE (lower extremity), FAOS (Foot and Ankle Outcome Score), and OMAS (Olerud and Molander Ankle Score), reporting a distribution-based MCID of 4.3 for the PROMIS PF. However, the current study includes more than twice the number of patients as Gausden et al and is unique in using both distribution-based and anchor-based methods of calculation. Hung et al recently published regarding the MCIDs of PROMIS and Foot and Ankle Ability Measure in all foot and ankle patients experiencing foot and ankle treatment and concluding MCIDs between 3 and 30 depending on the method of calculation used. PROMIS PF MCID was 7.8 (7.7) for >6-month follow-up from baseline in their anchor-based analysis. This is likely higher than our results because of a more broad population. As they did not define an intervention, they performed their analysis based on the first presentation to a clinic.

Limitations of this study include a specific study population and inherent weaknesses in generalizing this finding beyond a single institution’s treatment of traumatic ankle fractures, the lack of preinjury scores, the high number of nonresponders, and variable follow-up time points. This complicates efforts to evaluate patient’s return of function. This study was based on outcomes from a single level 1 referral center, with a patient population that does not represent the entirety of the United States. The scope of injury in an ankle fracture encompasses all forms of operative fixation, open and closed fractures, and soft tissue injury. When using the anchor-based MCID calculations, there is inherent patient recall bias that can impact whether a patient believes she or he has improved as compared to the previous visit. However, because both the distribution MCID and anchor-based MCID are similar, the impact of this bias appears minimal in the current study.

In conclusion, as PROMs continue to be incorporated into clinical and research practices, MCIDs must be well documented and understood. Given that our patient group has been focused to include only operatively treated rotational ankle fractures, surgeons should have confidence with a PROMIS PF MCID of 5 for this patient population, whereas clinical conclusions should not be drawn in the setting of operative ankle fractures for PROMIS PF differences less than 5.