Patients Require Less Time to Complete Preoperative Patient-Reported Outcomes Measurement Information System (PROMIS) Than Legacy Patient-Reported Outcome Measures

Abstract

Purpose

To analyze time to completion of preoperative legacy patient-reported outcomes (PROs) and more recent computer adaptive Patient-Reported Outcomes Measurement Information System (PROMIS) questionnaires in patients with symptomatic femoroacetabular impingement syndrome undergoing primary hip arthroscopy.

Methods

A retrospective analysis was conducted on patients undergoing hip arthroscopy by a single fellowship-trained hip arthroscopist. Inclusion criteria were patients undergoing primary arthroscopic hip surgery and completion of at least 1 legacy PRO or PROMIS questionnaire at the preoperative time point. Exclusion criteria were history of contralateral or ipsilateral hip surgery, non–English-speaking patients, patients who completed PROs by phone or by paper form, and patients who did not complete preoperative PROs. Legacy PROs included modified Harris Hip Score (mHHS), Hip Outcome Score (HOS), International Hip Outcome Tool (iHOT-12), and Hip Pain Visual Analog Scale (VAS-Pain). PROMIS PROs included Physical Function (PROMIS-PF), Pain Interference (PROMIS-PI), and Depression (PROMIS-D). Only preoperative PROs were included in the analysis. Completion time was calculated using the questionnaire start and stop time reported by the survey collecting software. The median and interquartile range of each PRO were reported for analysis of central tendency and statistical dispersion, respectively.

Results

A total of 1,901 patients and 269 patients were included in the legacy and PROMIS groups, respectively. The median time required for completion of each PRO in (minutes: seconds) format was as follows: mHHS (1:29), HOS (3:58), iHOT (2:11), VAS-Pain (0:32), PROMIS-PF (0:46), PROMIS-PI (0:37), and PROMIS-D (0:43). The interquartile range of the middle 50% of respondents was as follows; mHHS (0:58), HOS (2:46), iHOT (1:22), VAS-Pain (0:28), PROMIS-PI (0:19), PROMIS-D (0:29), and PROMIS-PF (0:20).

Conclusions

This study supports that preoperative PROMIS forms require less time to complete than preoperative legacy PROs and are not significantly influenced by age, race, or workers compensation status.

Level of Evidence

Level IV, case series.

The drive to improve efficiency in patient-reported outcome (PRO) collection and delivery stems from a will to improve compliance and accuracy of information. Legacy PROs including Hip Outcome Score – Activities of Daily Living (HOS-ADL),¹ Hip Outcome Score – Sports Subscale (HOS-SS),² modified Harris Hip Score (mHHS),³ International Hip Outcome Tool (iHOT-12),⁴ and visual analog scale for pain (VAS Pain) traditionally have been used to evaluate preoperative hip function and postoperative clinical outcomes in patients undergoing hip arthroscopy. In 2004, The National Institutes of Health developed the Patient-Reported Outcomes Measurement Information System (PROMIS) with the goal of providing a singular, standardized outcomes measure that can be applied to a wide variety of conditions. To alleviate complications associated with multiple, noncomparable PROs, PROMIS has added integrated response theory (IRT) with computerized adaptive testing (CAT), which has led to the development of a single, validated, generalizable system for assessing PROs.^5,6 With the addition of these measures, PROMIS has reduced redundant questions while addressing multiple modalities, therefore reducing the number of responses required to provide an outcome score assessment in comparison with legacy PROs, while providing accurate health state estimations across specialties.^7,8 PROMIS also has been shown to be an effective and efficient tool for reporting patient outcomes in various orthopaedic surgery procedures9, 10, 11 and has strongly correlated with common legacy PROs.^12,13

It is well established within the literature that hip arthroscopy is an effective treatment for femoroacetabular impingement syndrome (FAIS) and leads to improvement in PROs over time.14, 15, 16, 17, 18, 19, 20, 21 A series of legacy PROs often are used to evaluate patient functional status and pain but are not without limitations. PROs often are administered in the physician’s office before appointments, which can delay care and lead to inefficiencies. Excessive and redundant questions can cause questionnaire burnout and lead to inaccurate or faulty data, limiting its clinical and research value. In addition, burdensome PROs may limit willingness to complete forms following surgery. Recently, PROMIS CAT forms have been used for measuring functional status in the hip arthroscopy population and have shown to have excellent correlation with legacy hip PROs, establishing itself as a useful adjunct and possible alternative to traditional outcome measures in patients with FAIS.22, 23, 24 PROMIS PROs have demonstrated improved average time to completion (TTC) in patients undergoing cervical spine²⁵ and upper extremity²⁶ surgery; however, much remains unknown about its effect on the efficiency of questionnaire delivery in patients undergoing hip arthroscopy.

The purpose of this study is to analyze time to completion of preoperative legacy PROs and more recent computer adaptive PROMIS questionnaires in patients with symptomatic FAIS undergoing primary hip arthroscopy. We hypothesized that PROMIS PROs would require less TTC when compared with legacy PROs, and that PROMIS PROs would display less variability in TTC across patients undergoing hip arthroscopy.

Methods

Patient Selection

After institutional review board approval, a prospectively collected database of patients undergoing hip arthroscopy for FAIS by a single surgeon between November 2013 and July 2019 was retrospectively analyzed. Inclusion criteria were patients undergoing primary arthroscopic hip surgery and completion of at least 1 legacy PRO or PROMIS questionnaire at the preoperative time point. Exclusion criteria were history of contralateral or ipsilateral hip surgery, non–English-speaking patients, patients who completed PROs by phone or by paper form, and patients who did not complete at least 1 preoperative legacy PRO or at least 1 PROMIS questionnaire. Postoperative PROs were not included in this study. Legacy PROs included mHHS, HOS, iHOT-12, and VAS-Pain, whereas PROMIS questionnaires included Physical Function (PROMIS-PF), version 2.0, Pain Interference (PROMIS-PI), version 1.1, and Depression (PROMIS-D), version 1.0.

Data Collection

Legacy PROs were collected between November 2013 and December 2017, and PROMIS scores were collected from their introduction at our institution in January 2018 to July 2019. All surveys collected were administered before the surgical date at the initial visit using a table device (iPad tablet; Apple, Cupertino, CA). All PROs were electronically collected and stored on the OBERD data collection platform (Universal Research Solutions, Columbia, MO). The outcomes of interest were the TTC each individual PRO. The raw data export from OBERD contains a start and stop time stamp for PROs completed by each patient. The completion time was determined by calculating the difference between the questionnaire start and stop time reported by the survey collecting software.²⁷ Patient demographics also were collected, including age, sex, race, ethnicity, and workers compensation status.

Statistical Analysis

The Shapiro–Wilk test was used to determine normality and box plots were used to identify outliers. All PRO datasets were non-normal, right skewed, and contained outliers. As such, the interquartile range (IQR) method was used to identify outliers, with limits placed at 1.5 IQR from the first and third quartile. Cases outside of these limits were removed from subsequent analysis.²⁸ The median and IQR of each PRO were reported for analysis of central tendency and statistical dispersion, respectively. In addition, the number of questions required to complete each PRO was determined.

All data were inspected before analysis to determine whether parametric statistical analysis assumptions were met. The Shapiro–Wilk test was used to determine normality. The Levene test was performed to assess for equality of variances. For continuous variables with 2 groups, independent sample t-test or Mann–Whitney U test was used. For continuous variables with more than 2 groups, analysis of variance or Kruskal–Wallis test were used. Categorical variables were performed to determine significant differences in categorical variables. All statistical testing was performed in SPSS, version 26 (IBM Corp., Armonk, NY).

Results

Time to Completion

A total of 1,901 patients and 269 patients completed at least 1 legacy PRO and at least 1 PROMIS subscore at the preoperative time point, respectively. The percentage of patients completing each preoperative PRO is as follows: mHHS (83.2%), HOS (79.0%), iHOT-12 (84.6%), VAS for pain (76.2%), PROMIS-PI (89.2%), PROMIS-D (91.4%), and PROMIS-PF (91.4%). The IQR method was used before subsequent analysis, identifying a greater percentage of outliers in the legacy PROs in comparison with PROMIS PROs (Table 1). The average number of questions required to complete PROMIS PROs was less than all legacy PROs, with the exception being VAS-Pain (Table 2). The median TTC (minutes: seconds) of each PRO was as follows: mHHS (1:29), HOS (3:58), iHOT (2:11), VAS-Pain (0:32), PROMIS-PI (0:37), PROMIS-D (0:43), and PROMIS-PF (0:46) (Fig 1). The IQR of the middle 50% of respondents was as follows: mHHS (0:58), HOS (2:46), iHOT (1:22), VAS-Pain (0:28), PROMIS-PI (0:19), PROMIS-D (0:29), and PROMIS-PF (0:20).

Table 1.

Percent of Cases Within Outlier Limits After IQR Outlier Removal

Legacy
mHHS	85.7%
HOS	88.3%
iHOT-12	86.8%
Pain	80.4%
PROMIS
PI	88.8%
D	91.0%
PF	90.6%

D, PROMIS Depression; HOS, Hip Outcome Score; iHOT-12, international Hip Outcome Tool; IQR, interquartile range; mHHS, modified Harris Hip Score; Pain, Hip Pain visual analog scale; PF, PROMIS Physical Function; PI, PROMIS Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System.

Table 2.

Number of Questions Required to Complete Legacy and PROMIS PROs

Legacy
mHHS	8
HOS	31
iHOT-12	12
Pain	1
PROMIS
PI	4.1 ± 0.7
D	5.6 ± 2.8
PF	4.0 ± 0.2

D, PROMIS Depression; HOS, Hip Outcome Score; iHOT-12, international Hip Outcome Tool; mHHS, modified Harris Hip Score; Pain, Hip Pain visual analog scale; PF, PROMIS Physical Function; PI, PROMIS Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System.

Fig 1.

Completion time to completion of legacy and novel patient-reported outcomes (in minutes). The box plots depict the minimum, first quartile, median, third quartile, and maximum completion time. (D, PROMIS Depression; HOS, Hip Outcome Score; iHOT, international Hip Outcome Tool; mHHS, modified Harris Hip Score; Pain, Hip Pain visual analog scale; PF, PROMIS Physical Function; PI, PROMIS Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System.)

Demographics: Legacy Versus PROMIS Groups

A χ² analysis of the demographics of patients completing legacy and PROMIS PROs indicated similar group composition in all patient demographics except sex. Patients completing PROMIS PROs was 77.2% female, whereas legacy PROs contained 71.2% female subjects (Table 3).

Table 3.

Demographics of Patients’ Complete Legacy and PROMIS PROs

	Legacy (n = 1901)	PROMIS (n = 269)	P Value
Age, y
<20	23.7%	26.0%	.195
20-30	22.4%	25.7%
30-40	23.8%	25.3%
40-50	16.6%	13.0%
>50	13.7%	10.0%
Sex
Male	28.8%	22.8%	.040∗
Female	71.2%	77.2%
Race
White	93.4%	92.7%	.332
Black	3.5%	2.7%
Asian	2.0%	2.3%
Other	1.1%	2.3%
Ethnicity
Not Hispanic or Latino	92.5%	93.0%	0.765
Hispanic or Latino	7.5%	7.0%
Workers compensation	2.8%	2.8%	0.994

D, PROMIS Depression; HOS, Hip Outcome Score; iHOT-12, international Hip Outcome Tool; IQR, interquartile range; mHHS, modified Harris Hip Score; Pain, hip pain visual analog scale; PF, PROMIS Physical Function; PI, PROMIS Pain Interference; PROMIS, Patient-Reported Outcomes Measurement Information System.

^∗Indicates statistically significant (P < .05) between groups.

Demographic: TTC

Analysis of completion time stratified by age indicated significant differences in TTC between age groups for all legacy PROs (P < .05; Table 4). Planned post hoc comparison was performed on each individual legacy PRO comparing TTC between 2 individual age groups at a time. All legacy PROs at each age group had at least one significant difference in TTC on post hoc testing. The Shapiro–Wilk test for PRO completion times indicated non-normal distribution. As such, Spearman rank correlation analysis was performed between age and TTC for each PRO. Significant positive correlations were found between age and time to completion in all legacy PROs (Table 4). No significant correlations were found between age and time to completion in PROMIS PROs.

Table 4.

Preoperative PRO Completion Time (Minutes: Seconds) by Demographic

	Legacy (n = 1901)				PROMIS (n = 269)
	mHHS	HOS	iHOT-12	Pain	PI	D	PF
Age, y
<20	01:44	04:44	02:27	00:55	00:44	00:51	00:52
20-30	01:38	04:23	02:29	00:46	00:37	00:44	00:44
30-40	01:41	04:35	02:28	00:50	00:40	00:50	00:52
40-50	01:48	04:55	02:34	00:49	00:38	00:48	00:50
>50	01:59	04:58	02:53	01:02	00:39	00:53	00:50
Correlation coefficient	0.106	0.063	0.101	0.077	–0.062	0.060	0.007
Sex
Male	01:44	04:26†	02:32	00:51	00:37	00:48	00:48
Female	01:44	04:45†	02:32	00:52	00:41	00:49	00:49
Race
White	01:43	04:34	02:30	00:50	00:40	00:48	00:49
Black	01:59	06:29	03:18	01:09	00:35	00:54	00:41
Asian	01:40	04:42	02:29	00:44	00:37	00:45	00:58
Other	02:01	05:25	02:17	01:33	00:42	01:16	00:52
Ethnicity
Hispanic or Latino	02:00∗	05:11∗	02:46	00:49	00:44	00:54	01:01∗
Not Hispanic or Latino	01:42∗	04:38∗	02:33	00:49	00:39	00:48	00:48∗
Workers compensation
Yes	02:30∗	06:10∗	03:18∗	01:15∗	00:48	00:47	00:55
No	01:42∗	04:40∗	02:28∗	00:49∗	00:38	00:47	00:48

NOTE. Values are presented as means unless otherwise noted.

D, PROMIS Depression; HOS, Hip Outcome Score; iHOT-12, international Hip Outcome Tool; IQR, interquartile range; mHHS, modified Harris Hip Score; Pain, hip pain visual analog scale; PF, PROMIS Physical Function; PI, PROMIS Pain Interference; PRO, patient-reported outcome; PROMIS, Patient-Reported Outcomes Measurement Information System.

^∗Indicates time to completion that are involved in a statistically significant outcomes (P < .05) between groups. Statistically significant difference in completion time compared by patient age and race are not depicted in the table.

An analysis stratified by race indicated significant differences in TTC between groups in HOS and iHOT-12, and VAS for pain (P < .001 for all; Table 4). Planned post hoc comparison of HOS, iHOT-12, and VAS for pain indicated that African American patients had a greater TTC in comparison with White and Asian patients (P < .01). Mann–Whitney U analysis indicated that patients of Hispanic or Latino ethnicity required a greater amount of time to complete mHHS, HOS, PROMIS-PF (P < .05) than non-Hispanic patients (Table 4). Mann–Whitney U analysis of sex indicated that female subjects had a greater TTC for HOS (P = .012) in comparison with their male counterparts (Table 4). Patients receiving workers compensation had greater TTC for mHHS, HOS, iHOT-12, and VAS for pain (P < .01; Table 4).

Discussion

In this study, we found that PROMIS PROs required less TTC, demonstrated reduced variability in TTC, necessitated fewer questions to complete, and displayed less variability across patient demographics than legacy PROs. This study demonstrated that the majority of patients undergoing hip arthroscopy are able to complete PROMIS-specific questionnaires in less than 1 minute, which is consistent with other patient populations.29, 30, 31, 32 In comparison, the median TTC of individual legacy PROs ranged from 30 seconds in VAS-Pain to nearly 4 minutes for the HOS questionnaire. In addition, PROMIS PROs required fewer total questions to complete than legacy PROs.

It is well-established within the literature that waiting time is negatively correlated with patient satisfaction in the outpatient orthopaedic setting33, 34, 35, 36 and that a reduction in wait time leads to improved patient satisfaction.³⁷ PROs have provided clinicians with useful, quantitative data to determine the efficacy of various treatments within orthopaedic surgery and beyond.38, 39, 40 To ensure accuracy of information, patient compliance and reduction of questionnaire burnout, PRO measures must be short, concise, and simple to answer. The implementation of a shorter, quicker, and more predictable PRO system could potentially lead to more accurate patient appointment slots, shorter waiting times, improved patient satisfaction, and ultimately improved accuracy of patient outcome information in busy orthopedic practices.

Previous studies have evaluated the TTC of PROMIS CAT in multiple contexts.25, 26, 27 Kadri et al.²⁷ evaluated older cohort (age: 47 years; race: 65% White, 19.4% African American) undergoing cervical spine surgery and reported a mean TTC of PI (1.05 minutes), PF (0.74 minutes), and D (0.57 minutes). Interestingly, examination of race demonstrated the longest and shortest TTC in Asian and White patients, respectively. In addition, Hispanic patients demonstrated the shortest TTC in comparison with non-Hispanic patients.²⁷ Iyer et al.²⁵ evaluated an older cohort (age: 57 years; race: 92% White) undergoing cervical spine surgery and reported a median TTC of PI (29.5 seconds) and PF (37.0 seconds). In addition, Tyser et al.²⁶ evaluated patients undergoing upper-extremity surgery (age: 42 years; race: 92% White) and reported an average PF of 57 seconds. Similarly, in a study consisting of 2,952 operative and nonoperative patients (age: 51.0 years), Gulledge et al.¹⁰ reported a mean TTC and number of questions necessary to complete PROMIS-UE, PROMIS-PF, and PROMIS-PI ranging from 45.3 to 54.4 seconds and 4.1 to 4.9 questions, whereas PROMIS-D ranged from 20.9 to 38.6 seconds for all groups and required 6.2 to 6.7 questions to complete. These results are similar to the findings of the present study; however, it is important to note that patient age, race, and the reporting of central tendency may influence the reported average TTC in completion of PROMIS forms.

The findings in the present study support that PROMIS PROs have less variability in TTC than legacy PROs. The IQR, defined as the difference in TTC for the 25th and 75th percentile, for completion of legacy PROs ranged from 28 seconds for VAS-Pain, which is a single question survey, to just less than 3 minutes for HOS. The IQR for each of the PROMIS forms was less than 30 seconds. These results are supported by previous studies evaluating the variability in TTC for PROMIS forms.^29,41 In addition, the IQR of patients completing PROMIS forms was less than 30 seconds, and there was no difference in TTC based on age, sex, race, or workers compensation status. Significant differences were seen in TTC for legacy PROs based on age, sex, race, ethnicity and workers compensation status. Patients older than 50 years of age completed each of the legacy PROs slower than their younger counterparts, which is consistent with previous studies using electronic software for data collection.^27,42 African Americans completed legacy PROs slower than that of White and Asian ethnicity, female subjects took longer to complete HOS than male subjects, and patients with workers compensation injuries took significantly longer to complete mHHS, HOS, iHOT-12, and VAS for pain forms. These results differ from that of a recent study by Kadri et al.,²⁷ which found that PROMIS forms were completed fastest by young patients and those of Hispanic ethnicity.

Limitations

The present study has limitations that are worth noting. Time required to complete each of the PROMIS and legacy PROs is intrinsically right skewed and further skewed, in part, due to a number of patients failing to complete the questionnaires in one sitting, without interruption. For this reason, reporting the median TTC and the IQR provides a more accurate representation of average time to complete the PROMIS and legacy PROs in this population.⁴³ In addition, our institution uses multiple PRO collection modalities, including OBERD (Columbia, MO), phone calls, and written forms. Patient who completed the questionnaires over the phone or on paper may be less comfortable with technology, leading to our study potentially misrepresenting the true population. Moreover, we did not collect the order in which each PRO was completed. If questionnaire burnout influences response rate, patients may complete later questionnaires faster (with less care) or possibly slower due to lack of motivation or attention. In addition, while most questionnaires were completed at the time of clinic visit, some patients completed surveys at another time outside the clinic but before surgery. Finally, our current study was performed at a single tertiary care institution and may not be representative of the entire hip arthroscopy population. Future studies are warranted to determine the utility of PROMIS forms in assessing clinical outcomes in patients undergoing hip arthroscopy. Specifically, the relationship between preoperative and postoperative PRO completion time and achievement of clinically significant outcome may be of value.

Conclusions

This study supports that preoperative PROMIS forms require less time to complete than preoperative legacy PROs and are not significantly influenced by age, race, or workers compensation status.