The PROMIS CAT Demonstrates Responsiveness in Patients After ACL Reconstruction Across Numerous Health Domains

Abstract

Background:

The Patient-Reported Outcomes Measurement Information System (PROMIS) has emerged as a dynamic, efficient, and validated patient-reported outcome measure in the field of orthopaedics. However, the responsiveness, which is defined as the ability to detect changes in scores over time, of PROMIS computer adaptive tests (CATs) after anterior cruciate ligament reconstruction (ACLR) has not been well documented.

Purpose:

To investigate the responsiveness up to 1 year postoperatively of multiple PROMIS CAT domains in patients undergoing ACLR.

Study Design:

Cohort study (diagnosis); Level of evidence, 3.

Methods:

All patients who underwent ACLR by 1 of 2 fellowship-trained sports medicine orthopaedic surgeons, with preoperative and at least 6 months postoperative visits, were included in this study. PROMIS CAT physical function (PF), pain interference (PI), and depression (D) scores from each visit were collected and analyzed. Preoperative patient-centric factors, including demographic factors and meniscal pathology, were analyzed for associations with improvements in PROMIS scores.

Results:

A total of 100 patients (62 male patients and 38 female patients; mean age, 27.6 ± 11.8 years) with an average follow-up of 338.5 ± 137.5 days were included in this study. Preoperative PF, PI, and D scores improved significantly from 38.5 ± 7.3, 60.3 ± 7.0, and 47.9 ± 9.1, respectively, to 53.6 ± 10.3, 48.1 ± 8.5, and 41.0 ± 9.9, respectively (P < .001 for each domain). Lower preoperative PF scores, PI scores, and a lower body mass index (BMI) were predictive for greater improvements in PF. Higher preoperative PI scores and a lower BMI were predictors for greater improvements in PI. Meniscal pathology was not predictive of improvement in PROMIS scores.

Conclusion:

PROMIS CAT assessments of PF, PI, and D demonstrated responsiveness in patients undergoing ACLR up to 1 year. Worse preoperative PROMIS scores and a lower BMI were predictive of greater improvements in PROMIS scores.

Approximately 200,000 anterior cruciate ligament (ACL) reconstructions (ACLR) are performed in the United States each year, costing an estimated $2 billion.^15,21,28 ACLR is used to treat an ACL rupture manifesting as functional instability of the affected knee. In turn, a key factor for a successful ACLR is knee stability testing. To determine the success of the ACLR, a physician can utilize patient-reported outcome measures (PROMs), knee stability and range of motion testing, and a visual analog scale for pain and satisfaction. PROMs are advantageous because they assess the subjective experience of the patient across physical, mental, and social domains, granting the physician unique insight into the patient’s health.¹

There are dozens of PROMs utilized for lower extremity diseases, each varying in their scope and usefulness. For ACL injury, it is recommended to use knee-specific PROMs, such as the International Knee Documentation Committee or the Knee injury and Osteoarthritis Outcome Score (KOOS), each of which have varying performance, responsiveness, reliability, and validity.³⁰ Unfortunately, these differences compound with inconsistent utilization and make comparisons between studies difficult and unreliable.²⁰

In an effort to standardize and expedite reporting in PROMs, the National Institutes of Health created the Patient-Reported Outcomes Measurement Information System (PROMIS), with assessments covering numerous health domains. Since its creation in 2003, PROMIS has emerged as an efficient and reliable PROM and has been widely adopted in orthopaedics for both clinical and research endeavors.^7,8,10,11,17 PROMIS has been validated against previously established PROMs in patients with knee, foot and ankle, spine, and upper extremity conditions.^‡ For example, PROMIS scores have been correlated with “legacy” measures such as the American Shoulder and Elbow Surgeons and Simple Shoulder Test scores.¹⁷ Moreover, the computer adaptive test (CAT) versions of PROMIS questionnaires, which utilize item response theory to sequentially select the most appropriate questions in response to a patient’s prior answers,³ have shown favorable psychometric properties when compared with traditional legacy PROMs^{2,9,13,16,18,29} and the static short-form versions of PROMIS domains.⁵

Improvement after ACLR has been demonstrated using PROMIS CAT forms in a few studies^13,22,26; however, the responsiveness—or ability to detect changes in outcome scores over a designated period of time—of multiple PROMIS CAT domains in patients undergoing ACLR has not been well documented. Chen et al⁴ demonstrated the responsiveness of multiple PROMIS CAT domains in the early postoperative period (average of 4.5 months) after ACLR; however, the responsiveness of PROMIS has not been demonstrated out to 1 year postoperatively.

The purpose of this study was to investigate the responsiveness of multiple PROMIS CAT domains up to 1 year after surgery in patients who underwent ACLR. The secondary purpose was to identify preoperative patient-centric factors, including preoperative PROMIS scores, meniscal pathology, body mass index (BMI), race, sex, age, and smoking status, that are associated with improvement in PROMIS scores over time. We hypothesized that measures of physical function, pain interference, and depression would all significantly improve after ACLR and that worse preoperative PROMIS scores would be associated with greater improvement in PROMIS scores over time.

Methods

Study Design and Participants

This study was approved by our institutional review board before data collection. Consecutive patients who underwent ACLR by 1 of 2 fellowship-trained sports medicine orthopaedic surgeons (E.C.M. and V.M.) between July 11, 2017, and October 30, 2018, were identified using Current Procedural Terminology code 29888 and were retrospectively included in this study. Our inclusion criteria were ACLR patients who completed both preoperative and minimum 6-month postoperative PROMIS CAT assessments. Exclusion criteria included patients who could not complete PROMIS CAT forms owing to a language barrier (forms are administered in English), as well as those with incomplete PROMIS CAT data; 34 patients were excluded (Figure 1). Chart review was conducted on each patient meeting the inclusion criteria to identify any preoperative meniscal pathology; the review was based on his or her preoperative magnetic resonance imaging (MRI) report and BMI.

Figure 1.

Flow diagram of inclusion criteria. ACL, anterior cruciate ligament; PROMIS, Patient-Reported Outcomes Measurement Information System.

Instruments

In our ambulatory sports medicine orthopaedic clinic, all lower extremity patients complete the PROMIS physical function CAT V 2.0 (PROMIS-PF), PROMIS pain interference CAT V 1.1 (PROMIS-PI), and PROMIS depression CAT V 1.0 (PROMIS-D) at each clinic visit. Demographic data are also collected using a standard intake form provided at the first clinic visit. All demographic and PROMIS forms were emailed or administered on a tablet computer using Research Electronic Data Capture (REDCap), an online, Health Insurance Portability and Accountability Act--compliant data management and collection instrument maintained by Vanderbilt University (Nashville, Tennessee, USA).¹⁴ All PROMIS instruments are calibrated against a reference population and have a mean t score of 50 and a standard deviation of 10, with higher scores indicating more of the health domain in question. For example, a high score on the pain and depression forms indicates more pain and depressive symptoms, while a high score on the physical function form denotes greater functional ability.²³

Statistical Analysis

Our primary outcome of interest was the responsiveness of PROMIS scores from preoperative to the longest postoperative timepoint (mean, 338.5 ± 137.5 days) for each included patient. Patient characteristics and PROMIS score summaries are displayed using descriptive statistics. The mean pre- and postoperative PROMIS scores were compared using a paired-samples t test. A 1-way analysis of variance (ANOVA) was used to compare differences among the preoperative and several postoperative timepoints for each PROMIS domain. Additionally, a linear regression analysis was conducted to examine the relationship between time from injury to surgery and preoperative PROMIS scores. A general linear model univariate analysis of covariance was used to identify independent predictors of improvement for PROMIS-PF and PROMIS-PI scores. The dependent variable was set to the change in PROMIS-PF or PROMIS-PI scores from the preoperative to the postoperative clinic visit. Age and BMI were set as covariates, and meniscus pathology, race, sex, and smoking status were included as fixed factors. The beta estimate, standard error, and P value are listed from the parameter estimates. The beta estimate values denote the change in PROMIS physical function or pain interference improvement for every 1-unit increase in the independent variable. Finally, an ANOVA and least square differences post hoc test was used to compare the mean change in PROMIS-PF and PROMIS-PI based on their respective preoperative scores. All analyses were performed using SPSS software (Version 25.0; IBM), and P < .05 was considered statistically significant.

Results

In total, this study included 100 patients with an average follow-up of nearly 1 year after ACLR. Complete patient characteristics can be found in Table 1.

TABLE 1.

Patient Characteristics^a

Variable	Value
Age, y	27.6 ± 11.8 (14-57)
BMI, kg/m2	26.1 ± 5.2 (18.7-41.6)
Sex
Male	62 (62)
Female	38 (38)
Meniscal pathology
Yes	67 (67)
No	33 (33)
Race
White	61 (61)
Black	13 (13)
Asian	7 (7)
Other/Unknown	19 (19)
Smoking status
Never	85 (85)
Former	9 (9)
Current	2 (2)
Unknown	4 (4)
Clinic visit (days)
Preoperative	22.5 ± 18.4 (0.7-91.6)
Postoperative	338.5 ± 137.5 (171.5-634.7)

^aData are reported as n (%) or mean ± SD (range). BMI, body mass index.

Baseline PROMIS CAT scores were 38.5 ± 7.3, 60.3 ± 7.0, and 47.9 ± 9.1 for PROMIS-PF, PROMIS-PI, and PROMIS-D, respectively. Postoperatively, these scores all significantly improved, to 53.6 ± 10.3, 48.1 ± 8.5, and 41.0 ± 9.9, respectively (P < .001 for each domain) (Table 2). Additionally, PROMIS CAT scores demonstrated significant improvements over many other postoperative timepoints (P < .001 for each domain) (Figure 2).

TABLE 2.

Summary of PROMIS Scores^a

PROMIS Domain	Preoperative	Postoperative	Mean Change	P Value
Physical function	38.5 ± 7.3	53.6 ± 10.3	15.1 ± 11.7	<.001
Pain interference	60.3 ± 7.0	48.1 ± 8.5	–12.3 ± 8.7	<.001
Depression	47.9 ± 9.1	41.0 ± 9.9	–6.8 ± 10.1	<.001

^aData are reported as mean ± SD. PROMIS, Patient-Reported Outcomes Measurement Information System.

Figure 2.

Patient-Reported Outcomes Measurement Information System (PROMIS) scores change over time after anterior cruciate ligament reconstruction. Patients showed improvement over time after surgery in all 3 health domains. Error bars indicate ± 2 SE. *Days before (for preoperative) or after surgery (mean ± SE). CAT, computer adaptive test.

Preoperative data that best predicted change in PROMIS-PF scores included preoperative PROMIS-PF scores, preoperative PROMIS-PI scores, and BMI (Table 3). Lower preoperative PROMIS-PF scores (less function), PROMIS-PI scores (less pain), and BMIs were predictive of greater improvements in PROMIS-PF. Additionally, patients with lower preoperative PROMIS-PF scores improved to a greater degree than those with higher preoperative PROMIS-PF scores (P < .001) (Figure 3). Age, preoperative PROMIS-D scores, preoperative meniscal pathology, race, sex, and smoking status did not predict greater improvements in PROMIS-PF after up to 1 year. There was no significant relationship on linear regression analysis (R ² = 0.0021-0.024) between the time from injury to surgery and preoperative PROMIS scores.

TABLE 3.

Independent Predictors of Improvement in PROMIS Physical Function Scores^a

	Beta Estimateb	SE	P Value
Preoperative PROMIS score
Physical function	-1.26	0.20	<.001
Pain interference	-0.67	0.20	.002
Depression	0.08	0.12	.549
Age, y	-0.10	0.10	.309
BMI, kg/m2	-0.58	0.22	.013
Meniscal pathology
Yes	-1.94	2.19	.377
No	Reference
Race
White	-0.88	2.67	.743
Black	-1.04	3.56	.771
Asian	-0.26	4.41	.954
Other/Unknown	Reference
Sex
Male	0.37	2.11	.860
Female	Reference
Smoking status
Never	1.73	5.20	.740
Current	6.34	8.51	.458
Former	0.54	6.09	.930
Unknown	Reference

^aBMI, body mass index; PROMIS, Patient-Reported Outcomes Measurement Information System. Bolded values indicate statistical significance at P < .05.

^bBeta estimate values denote the change in PROMIS physical function improvement for every 1-unit increase in the independent variable.

Figure 3.

Improvement in Patient-Reported Outcomes Measurement Information System (PROMIS) physical function scores from pre- to postoperative. Values that share the same lowercased letter are not significantly different from each other. Error bars indicate ±2 SE.

Preoperative data that best predicted change in PROMIS-PI scores included preoperative PROMIS-PI scores and BMI (Table 4). A greater preoperative PROMIS-PI score and a lower BMI were predictors of greater improvements in PROMIS-PI. Additionally, patients with higher preoperative PROMIS-PI scores improved more greatly in PROMIS-PI than those with lower preoperative PROMIS-PI scores (P<0.001) (Figure 4). No other factors were found to predict improvements in PROMIS-PI.

TABLE 4.

Independent Predictors of Improvement in PROMIS Pain Interference Scores^a

	Beta Estimateb	SE	P Value
Preoperative PROMIS score
Physical function	-0.21	0.16	.202
Pain interference	0.44	0.17	.011
Depression	0.04	0.10	.737
Age	-0.06	0.08	.455
BMI	-0.47	0.18	.011
Meniscal pathology
Yes	-0.43	1.82	.815
No	Reference
Race
White	-2.07	2.21	.352
Black	-0.84	2.96	.778
Asian	0.15	3.66	.967
Other/Unknown	Reference
Sex
Male	-0.59	1.75	.739
Female	Reference
Smoking status
Never	-0.15	4.32	.972
Current	5.68	7.07	.424
Former	-3.45	5.06	.498
Unknown	Reference

^aBMI, body mass index; PROMIS, Patient-Reported Outcomes Measurement Information System. Bolded values indicate statistical significance at P < .05.

^bBeta estimate values denote the change in PROMIS physical function improvement for every 1-unit increase in the independent variable.

Figure 4.

Improvement in Patient-Reported Outcomes Measurement Information System (PROMIS) pain interference scores from pre- to postoperative. Values that share the same lowercased letter are not significantly different from each other. Error bars indicate ±2 SE.

Discussion

The results of our study demonstrated statistically significant improvements found across all PROMIS CAT domains at approximately 12 months postoperatively. Additionally, we found that preoperative PROMIS scores and BMI are significant predictors of mean improvement in PROMIS scores after ACLR.

PROMIS CAT forms have been established as valid and efficient outcome instruments for patients with knee injuries.^{12,13,22,24,26} These studies validated the PROMIS-PF CAT against legacy PROMs and demonstrated favorable psychometric properties; however, an assessment of the responsiveness for PROMIS-PF, PROMIS-PI, and PROMIS-D had yet to be completed in an ACLR surgical cohort. The results of this study demonstrate that PROMIS-PF, PROMIS-PI, and PROMIS-D are sensitive and responsive measures for describing patient-reported outcomes after ACLR. These findings further support the adoption of PROMIS CAT assessments for use in ACLR patients for both clinical and research purposes. A major strength of this study was its focus on a single surgical patient cohort over preoperative and several postoperative timepoints.

We found that patients with more preoperative pain and lower function improve to a greater degree than those with less pain and more function, in each respective domain. Our findings echo those of a recent study by Chen et al,⁴ in which the authors found that patients with worse preoperative pain and function were more likely to achieve a minimal clinically important difference after ACLR. However, it must be taken into account that patients with lower preoperative baseline function scores tend to have a greater increase in postoperative scores, due to the fact that there is a larger margin for improvement. Intuitively this makes sense, because a patient with very limited function can improve to a greater extent after an ACLR, as compared with a patient with only minimal impairment. Additionally, the relationship between time of injury and surgery and preoperative scores must be considered, as function generally improves with reduction of swelling. However, our data did not show any correlation on linear regression analysis between preoperative scores and time from injury to surgery. While further research is required to establish more widely representative data, these findings suggest that preoperative PROMIS scores may have use in making evidence-based treatment decisions and could potentially play a role in setting appropriate patient expectations.

Additionally, our study investigated the impact of patient-centric factors on improvement in PROMIS scores. Prior studies have identified certain factors, including meniscal injury and obesity, as predictive of decreased postoperative outcomes after ACLR.^19,27 Obesity has also been associated with increased readmission rates after ACLR.⁶ Our study demonstrated that a lower BMI was predictive of greater improvements in both PROMIS-PF and PROMIS-PI. Taken together, these findings suggest that patients with a higher BMI have worse postoperative outcomes and do not benefit from ACLR to the same degree as those with a lower BMI, highlighting the need for prospective studies investigating impact of weight loss on PROMs after ACLR. In our study, evidence of meniscal pathology on preoperative MRI predicted lower postoperative function scores, which is in agreement with prior studies.^19,27 However, it was not found to be associated with greater improvement in outcomes. This could be because longer follow-up is necessary to detect differences in outcomes as well as other possible confounding factors that are related to meniscal damage. Chronic damage could possibly lead to early arthritis, which could have an impact on preoperative and postoperative scores.

There are several notable limitations to this study. First, it was conducted within a single health care system, and all study patients were able to communicate in English. Therefore, results may not be applicable to a variety of other patient populations. Another limitation is that patients may have been restricted in their ability to navigate the electronic forms because of limited computer and/or cognitive abilities. Furthermore, this study did not correlate PROMIS scores with other lower extremity legacy PROMs, such as the KOOS, Marx knee activity rating scale, or EuroQol-5 Dimension questionnaire. However, studies have already validated PROMIS-PF in patients with knee injuries.^{12,13,22,24,26}

Conclusion

The PROMIS CAT domains of physical function, pain interference, and depression demonstrated responsiveness—significant improvements over time—in patients after ACLR. Additionally, we found that preoperative PROMIS scores and BMI were significant predictors of mean improvement in PROMIS scores after ACLR, such that lower preoperative function, more preoperative pain, and lower BMI predicted greater improvements in PROMIS-PF and PROMIS-PI. Clinicians can rely on PROMIS CAT assessments to measure patient-reported improvements after ACLR for up to 1 year in length, and they should consider adopting the PROMIS platform into their routine practice.