The Minimal Clinically Important Difference for PROMIS Physical Function in Patients With Thumb Carpometacarpal Arthritis

Abstract

Background: This study was performed to determine the minimal clinically important difference (MCID) of Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) computer adaptive test (CAT) for patients with thumb carpometacarpal (CMC) arthritis. Methods: This study retrospectively analyzed data from 152 adults receiving surgical and nonsurgical care for unilateral thumb CMC arthritis at a single institution between January 2016 and January 2018. Patients completed PROMIS PF v1.2/2.0 CAT at each visit. At follow-up, patients also completed two 6-item anchor questions assessing the degree of perceived improvement. Statistical testing analyzed the ability of the clinical anchor to discriminate levels of improvement. An anchor-based MCID estimate was calculated as the mean PROMIS PF change score in the mild improvement group. The anchor-based MCID value was examined for the influence of patient age, initial and final PROMIS scores, and follow-up interval. A distribution-based MCID value was calculated incorporating the standard error of measurement and effect size. Results: The change in PROMIS PF scores was significantly different between encounters where patients reported no change, mild improvement, and much improvement. The anchor-based MCID estimate for PROMIS PF was 3.9 (95% confidence interval, 3.3-4.7). Individual MCID values were weakly correlated with the final absolute PROMIS PF score but did not correlate with patient age, time between visits, or the initial absolute PROMIS PF score. The distribution-based MCID value was 3.5 (95% confidence interval, 3.1-3.9). Conclusions: The estimated range of MCID values for PROMIS PF is 3.5 to 3.9 points in patients treated for thumb CMC arthritis.

Introduction

Patient satisfaction has increasingly become an indicator of quality of care in hand surgery. Accordingly, a growing number of patient-reported outcome (PRO) instruments have been introduced to assess treatment effects.^1-3 In the hand and upper extremity, commonly administered PROs include the Disabilities of the Arm, Shoulder and Hand (DASH), the QuickDASH (a subset of DASH), the Patient-Rated Wrist Evaluation (PRWE), the American Shoulder and Elbow Surgeons (ASES) score, and the Simple Shoulder Test (SST) questionnaires. Although these scales are valid and reliable measures of upper extremity disability,^2-4 there are limitations inherent to each of these static surveys. Lengthy questionnaires can lead to high responder burden that compromises the accuracy and completeness of survey material.³ Moreover, floor and ceiling effects may fail to capture the full range of patient experiences.⁴

Recognizing the barriers to adoption of traditional fixed-length scales, the National Institutes of Health designed the Patient-Reported Outcomes Measurement Information System (PROMIS) to improve the efficiency and precision of PRO measures. The PROMIS consists of item banks that draw from a variety of health domains, including Physical Function (PF) and Pain Interference (PI). The key advantage of PROMIS is the use of computer adaptive testing (CAT), a mode of test delivery based on item response theory (IRT) which can increase measurement accuracy and reproducibility while reducing responder burden.^5-7 Preliminary studies of the PROMIS Physical PF CAT in patient populations with upper extremity conditions have shown favorable results compared with conventional metrics, including the DASH,^5,6 the QuickDASH,⁸ and the ASES.⁹ Furthermore, the PROMIS instrument has consistently exhibited minimal floor and ceiling effects while requiring substantially less patient assessment time (1 minute of PROMIS PF vs 4.4 minutes of DASH).^5-8

To judge the magnitude of treatment effect using a patient-centered measure such as PROMIS, it is necessary to have an understanding of the patient’s perspective of change. The minimal clinically important difference (MCID) was defined by Jaeschke et al¹⁰ as the smallest change in an outcome that is deemed worthwhile by a patient. Thus, the MCID establishes a therapeutic threshold beyond which a change in score would be considered clinically meaningful. In addition, the MCID can be used in a priori powering assessments to determine the appropriate sample size for research studies.^11,12 The MCID values have been established for multiple outcome measures in hand and upper extremity surgery, including the DASH,^13,14 the QuickDASH,^13,15 the Michigan Hand Outcomes Questionnaire (MHQ),^16,17 the ASES score,^18,19 and the SST.^18,19

Several methodologies have been described for determining MCID values. There is no single consensus method, but anchor-based and distribution-based methods are frequently used. An anchor-based approach relates a change in score to an external reference or clinical “anchor” (ie, “mildly worse,” “no change,” or “mildly better”).¹¹ A commonly used anchor-based approach is the anchor mean change method that calculates the MCID as the mean change in outcome score corresponding to a mild degree of improvement. This method has been previously used to derive MCIDs of the DASH,¹³ the QuickDASH,¹³ the PRWE,¹³ the ASES,¹⁹ and the SST.¹⁹ Distribution-based methods use the standard error of measurement (SEM) and effect size of the outcome score of interest to calculate the MCID based on the statistical distribution of scores on an instrument of interest.^16,20

Although MCID values have been reported for a number of PRO measures in hand and upper extremity surgery, MCID estimates for the PROMIS PF score are still being defined. The purpose of this investigation is to estimate the MCID for the PROMIS PF score in patients with thumb carpometacarpal (CMC) arthritis through both anchor-based and distribution-based approaches.

Methods

Institutional review board approval was obtained to retrospectively analyze data from our institution’s musculoskeletal outcomes registry. All demographic data were retrieved from patients older than 18 years of age seen at the outpatient offices of 1 of 7 hand surgeons at a single tertiary institution between January 2016 and January 2018. Patients were identified through an electronic medical record search for International Classification of Diseases, Tenth Revision diagnosis codes for thumb CMC arthritis (codes M18*). The following inclusion criteria had to be met to be included in this study: (1) a diagnosis of a unilateral thumb CMC arthritis treated either operatively or nonoperatively; and (2) at least1 follow-up visit at our institution. All diagnoses of arthritis were confirmed radiographically. Patients presenting with bilateral conditions or seeking care for multiple other symptomatic arthritides in the hand or upper extremity were excluded. In addition, patients scoring ≥70 on the initial PROMIS PF CAT were excluded due to their inability to demonstrate improvement over 3 points on the CAT as the effective ceiling on the CAT is 73.

Patients were given a handheld tablet at their initial patient visit and asked to complete the PROMIS PF CAT (v1.2/v2.0). The PROMIS PF CATs were repeated at each subsequent encounter after the initial visit. The PROMIS raw scores are translated into t scores, with a mean of 50 and standard deviation (SD) of 10 representing the U.S. general population.²¹ Scores range from 0 to 100 on all PROMIS measures, with higher scores representing more of a given domain (eg, higher score denotes more function). No relevant scoring change occurred between versions of the PROMIS PF CAT, and therefore, all scores were treated as comparable.²² At each follow-up, patients also completed two 6-item anchor questions that independently assessed the degree of perceived improvement from their initial evaluation and from their most recent visit (Table 1). This anchor question was adapted from that used in MCID studies by Juniper et al²³ and Tubach et al²⁴ for other common outcome measures.

Table 1.

Clinical Anchor Questions.

1. Since your first visit to the doctor who you are seeing today, has your condition(s) treated by this doctor changed?  a. Much worse  b. Mildly worse  c. No change  d. Mildly better  e. Much better  f. I am seeing the Doctor for a new problem 2. Since your most recent visit to the doctor who you are seeing today, has your condition(s) treated by this doctor changed?  g. Much worse  h. Mildly worse  i. No change  j. Mildly better  k. Much better  l. I am seeing the Doctor for a new problem

Statistical Analysis

Descriptive statistics described demographic characteristics of patients and their treatments. Patients contributed data to the study from a maximum of 1 follow-up visit to prevent a minority of patients from disproportionately contributing data. In particular, the follow-up visit that was chosen was the first visit in which the patient reported no change, mild improvement, or much improvement. Analysis of variance testing with post hoc Tukey B subset analysis and Fisher least significant difference pairwise comparisons evaluated whether the mean change in PROMIS PF scores was significantly different between encounters where patients reported no change, mild improvement, and much improvement in their anchor-based responses since their prior visits. The anchor-based MCID estimate was then calculated as the mean PF change score in the mild improvement group. Bivariate statistical testing was used to identify variables influencing the anchor-based MCID value. For patients indicating mild improvement, Spearman correlation coefficients were computed to quantify the association between the anchor-based MCID value and patient age, the initial PROMIS PF score, the final PROMIS PF score, and the amount of time (days) between visits.

A distribution-based method for estimating the MCID was also performed incorporating both the SEM and effect size. The SEM $(\mathrm{SEM}=\mathrm{SD}\sqrt{1-r})$ is calculated using between-person SD and Cronbach α as a lower-bound estimate of scale reliability (r). Using IRT, the SEM for the PROMIS PF CAT has been interpreted as the minimum detectable change (MDC), which represents the smallest change in an outcome measure that exceeds measurement error.²⁵ To have a meaningful value, the MCID must be greater than or equal to the MDC.²⁵ Based on SEM data from more than 2500 patients seeing our hand surgeons, the MDC of the PROMIS PF CAT in our hand clinic is 2.3. An effect size range of 0.2 to 0.8 was used to select plausible mean PF change scores for patients in the minimal improvement group. This effect size filter represents the distribution-based calculation consistent with the methods by Yost et al.²⁵

Based on prior investigations of MCID estimation using the PROMIS instrument, we conducted an a priori power analysis for sample size determination with a conservatively estimated MCID of 2 points on the PROMIS PF scale.^21,25 To achieve α = .05 and β = .8 with an SD of 10 and an effect size of 0.2 for the PROMIS PF score, we need to collect 45 data points representing no change and 45 data points representing mild improvement. Data were collected over a 2-year period to meet our minimum sample size requirements.

Results

In total, 152 patients were included in this analysis (Table 2). Most patients were white (n = 149, 98%) and women (n = 129, 85%). The mean age of the patients was 63 years (range = 35-82 years). The mean initial absolute PROMIS PF score was 41 (SD = 8, range = 24-59). All patients had a first follow-up visit at a median of 63 days (interquartile range [IQR], 42-125). A total of 70 patients (46%) reported mild improvement. Of these, 35 patients were treated nonoperatively during the study period, whereas the remaining 35 patients were treated with surgery (Table 3). For the surgically treated patients, the median time from surgery to the time at which they first reported mild improvement was 68 days (IQR, 28-79).

Table 2.

Patient Demographics.

Demographic Number (%)
Total number of patients	152
Sex, No. (%)
Female	129 (85)
Male	23 (15)
Race, No. (%)
White	149 (98)
African American	3 (2)

Table 3.

Treatments for Patients Reporting Mild Improvement.

Patient Breakdown Number
Total number of patients	70
Treatment
Nonoperative	35
Operative	35
LRTI	32
Thumb CMC fusion	3

Note. LRTI = ligament reconstruction and tendon interposition; CMC = carpometacarpal.

The change in PROMIS PF scores was significantly different (P < .01) between encounters where patients reported no change, mild improvement, and much improvement since their prior visits (Table 4). Using an anchor-based approach, the mean change in the PROMIS PF score for the mild improvement group was 3.9 with a 95% confidence interval of 3.3 to 4.7. This point estimate for the MCID exceeded the MDC level of instrument error. The mean changes in the PROMIS PF score for patients reporting mild improvement were also calculated for the nonsurgically treated and surgically treated groups.

Table 4.

Mean Change in PROMIS Physical Function According to Clinical Anchor Response.

Clinical anchor response	Δ physical function	95% confidence interval
No change (n = 48)	−1.3	−2.3 to −0.3
Mild improvement (n = 70)	3.9a	3.3 to 4.7
Much improvement (n = 34)	7.1	5.4 to 8.7

Note. PROMIS = Patient-Reported Outcomes Measurement Information System.

^aMinimal clinically important difference estimate.

For patients with mild improvement, the degree of change in their PROMIS PF score was not correlated with patient age (r = 0.09), the time between visits (r = −0.11), or the initial absolute PROMIS PF score (r = −0.01) (Table 5). The magnitude of change in PROMIS PF for the mild improvement group was weakly correlated with the final absolute PROMIS PF score (r = 0.35). There was also some variation based on treatment as the MCIDs for the nonsurgically treated and surgically treated groups were 3.1 (95% confidence interval, 2.6-3.6) and 4.8 (95% confidence interval, 4.0-5.6), respectively.

Table 5.

Spearman Correlation Coefficients Between Patient Variables and MCID.

Spearman correlations	MCID
Initial absolute PROMIS PF score	r = −0.01
Final absolute PROMIS PF score	r = 0.35
Time between visits	r = −0.11
Patient age	r = 0.09

Note. MCID = minimal clinically important difference; PROMIS = Patient-Reported Outcomes Measurement Information System; PF = Physical Function.

A distribution-based MCID estimate was computed using the MDC threshold of 2.3. After the effect size parameters of 0.2 to 0.8 were applied to data from patients reporting mild improvement, the estimated MCID value was 3.5 with a 95% confidence interval of 3.1 to 3.9. This point estimate for the MCID also exceeded the MDC level of instrument error.

Discussion

This study found a change of 3.5 to 3.9 points on PROMIS PF scores to constitute a meaningful change for patients with thumb CMC arthritis. This estimated range is similar to what has been reported for PROMIS assessments from other patient populations. Sandvall et al²⁶ used both anchor- and distribution-based approaches to estimate the MCID for PROMIS PF scores in patients treated nonoperatively for distal radius fractures. They reported a PROMIS PF MCID of 3.6 to 4.6 points. Using a distribution-based method, Ho et al²⁷ calculated an MCID of 4.2 for PROMIS PF scores for foot and ankle patients treated with surgery. Chen et al²⁸ estimated MCIDs for PROMIS PF and PI scores in patients who underwent primary anatomical total shoulder arthroplasty. Using a distribution-based approach, they reported MCIDs of 4.0 and 3.2 for PROMIS PF and PI, respectively. We believe that our data add to the literature by contributing to the goal of having a critical mass of publications indicating a consistent PROMIS MCID range for patients seeking upper extremity care so that researchers can confidently generalize it to the multitude of conditions treated by the hand surgeon.

Anchor- and distribution-based methods are the most common methods used for determining the MCID. There is lack of consensus regarding the optimal method for determining the MCID, and therefore, some authors have recommended using both approaches to calculate a range of MCID estimates from which an overall MCID value can then be triangulated.^16,20 An anchor-based approach relates a change in score to an external reference and has the advantage of incorporating the patient perception of a clinically meaningful change. We believe that basing the determination of meaningful change on patient report is ideal as it should ensure that an MCID estimate truly indicated change that patients are appreciating. Using an anchor-based approach, the MCID estimates have been determined for the DASH (10 points), QuickDASH (14 points), and PRWE (14 points).¹³ With similar methods in Swedish patients, 10 points was estimated to be the threshold for clinically important change on the DASH.²⁹ A distribution-based method, on the contrary, uses the SEM and effect sizes to estimate important differences. Depending on the specific approach used to calculate the MCID, one may expect comparable but nonidentical estimates of the MCID. Both methods were therefore used in this study to provide a reasonable range of MCID values.

The MCID values are most helpful for interpretation of group-level data as opposed to guiding individual patient decision-making. When analyzing individual responses, we observe significant variation between patients in the scores they associate with minimal improvement. As such, MCID estimates have less use in identifying which patients are likely to improve. Instead, MCID values are valid for group-level applications. Researchers use MCID values to determine whether cohorts demonstrate clinically relevant improvement. The MCID values are also frequently referenced as a proxy for meaningful differences between groups of patients. In this manner, MCID thresholds are used by researchers when conducting comparative clinical studies to assess whether differential outcomes between treatment groups are clinically relevant. Finally, MCID estimates are incorporated into sample size calculations when designing appropriately powered studies.

Multiple PRO measures could be used to study patients with thumb CMC arthritis. Similar to the DASH and MHQ, PROMIS is not disease-specific. The PROMIS PF is a generalized assessment of musculoskeletal function. Admittedly, this type of assessment could miss some of the morbidity specific to the thumb. However, as PROMIS PF has strongly correlated with the DASH and ASES, it appears to capture functional change associated with upper extremity conditions.^5,6 It is important to understand the performance characteristics of PROMIS PF as it is positioned to potentially serve as a consensus outcome measure. This single measure could assess treatment value to multiple patient populations, from those with heart conditions to neurologic conditions or extremity arthritis. To our knowledge, PROMIS is the first PRO measure applied across entire departments and even across entire medical centers.³⁰ Even among common conditions treated by hand surgeons, meta-analyses are limited by disparate outcomes.^31,32 In our practice, PROMIS has become our exclusive PRO secondary to the reduced patient burden (eg, typical CAT of 4-8 questions) and electronic delivery with automated scoring available in the electronic medical record at the point of care.

This study has several limitations. First, the clinical anchor question used in this study is a simplified version as opposed to an exact duplication of that administered in multiple prior studies, which has produced consistent results across different diagnoses.^13,16,18,19 We reduced the number of possible answers as we, and other researchers, have otherwise combined answers into composite groups of no change, minimal change, and substantial change, which obviated the need for more possible answers. Similarly, the PROMIS instrument is still an emerging PRO measure in hand surgery and is evolving. However, PROMIS PF’s updates now consist of subtle algorithm changes that have not affected final scores, and thus we expect our conclusions to be durable as opposed to a value that will become quickly obsolete. Second, as patients are asked to provide a retrospective perception of clinical change, their judgments may be prone to recall bias.¹¹ Third, we used the PROMIS PF CAT as the main PRO measure rather than the PROMIS Upper Extremity Function CAT. At the time when these patients were evaluated, our institution was not routinely collecting the Upper Extremity Function CAT. As the Upper Extremity CAT continues to be modified in an effort to reduce its ceiling effects, we expect that hand surgeons may use both PROMIS PF and Upper Extremity Function scores. Finally, it is possible that patients treated surgically require slightly more change in function before it is appreciated. However, the magnitude of difference in our patients was relatively small across treatments, and we did not have enough surgical patients to make definitive statements in this regard.

In summary, we found an improvement of 3.5 to 3.9 points on the PROMIS PF CAT to reflect a clinically relevant change at the group level for patients with thumb CMC arthritis. Both anchor- and distribution-based methods produced similar values of our MCID estimate. This is consistent with MCID values that have been reported from other patients with musculoskeletal conditions.^26-28 Ultimately, defining the MCID in this manner will help guide interpretation of group-level responses and inform comparisons between different treatments.