Performance of the Patient-Reported Outcome Measurement Information System in Patients With Patellofemoral Instability

Abstract

Background:

The Patient-Reported Outcome Measurement Information System (PROMIS) was developed to improve patient-reported outcome measures (PROMs) and administration through a computer adaptive test (CAT). The PROMIS physical function-CAT (PF-CAT) has not been investigated in patients with patellofemoral instability (PFI).

Purpose/Hypothesis:

The purpose of this study was to evaluate the construct validity of the PROMIS PF-CAT with previously validated tools for measuring PROMs in patients with a diagnosis of PFI. We hypothesized that the PF-CAT will have the strongest correlations with other PROMs that evaluated PF as well as moderate correlations with PROMs that measured other health domains.

Study Design:

Cohort study (diagnosis); Level of evidence, 2.

Methods:

Patients enrolled in this study who underwent operative intervention for PFI completed the following evaluations preoperatively: PROMIS PF-CAT, 36-Item Short Form Health Survey (SF-36), Knee injury and Osteoarthritis Outcome Score (KOOS), EuroQol-5 dimensions (EQ-5D), and Kujala Anterior Knee Pain Scale (AKPS). Correlation coefficients and the percentage of patients achieving the highest and lowest possible outcome score of each instrument were calculated to assess floor and ceiling effects. Statistical significance was defined as P < .05.

Results:

In total, 91 participants (63.7% females; mean age, 20.1 ± 7.2 years) completed the questionnaires. PF-CAT had the lowest number of questions (4.3 ± 1.1). The strongest correlations were between the PF-CAT and SF-36 PF subscale (r = 0.78; P < .01), AKPS (r = 0.68; P < .01), and KOOS Activities of Daily Living subscale (r = 0.68; P < .01). Correlation was moderate between the PF-CAT and the KOOS subscales of Sports/Recreation (r = 0.58; P < .01), Quality of Life (r = 0.53; P < .01), and Symptoms (r = 0.47; P < .01). The PROMIS PF-CAT demonstrated no floor or ceiling effects.

Conclusion:

In patients with PFI, construct validity of the PROMIS PF-CAT was supported by strong correlations demonstrated between the PF-CAT and PROMs evaluating PF and moderate correlations with those assessing other health domains. Our results demonstrated a low respondent burden and no floor or ceiling effects associated with the PROMIS PF-CAT. The PROMIS PF-CAT may be considered a beneficial alternative to previously established PF PROMs for preoperative evaluation of patients with PFI.

Patients with patellofemoral instability (PFI) commonly include adolescent and young adult populations. PFI is a common condition in adolescents and young adults, and it presents a need for a valid patient-reported outcome measure (PROM) instrument to longitudinally follow patients with operative PFI pathology. Assessing high-functioning patients with a PROM becomes challenging because patients may report the highest possible score.²⁶ A validated PROM should correspond to changes in instrument scores in comparison with a reference PROM (construct validity) and should have few floor or ceiling effects to effectively differentiate outcomes.⁴

The National Institutes of Health developed the Patient-Reported Outcome Measurement Information System (PROMIS) in 2004 to assess an array of health domains, including physical health, mental health, and social health.²⁷ Fatigue, pain intensity, pain interference, physical function (PF), and sleep disturbance are the profile domains included in the physical health category. The PROMIS computer adaptive tests (CATs) are adaptive tests that utilize item response theory to assess responses to individual questions and the relationships between questions in a given health domain.^1,20,33 The PROMIS CATs were developed utilizing multidimensional correlations (shared set of questions for traits measured) with a stopping feature when the standard error of the mean reaches a variation from the mean of 0.3.²⁹ The item response theory allows for interdependencies among traits to be assessed by PROMIS through the selection of test items to maximize participant information on correlated traits, which allows for a substantial reduction in the number of questions administered. PROMIS CATs may offer advantages compared with legacy PROMs because of the computer adaptive design, which minimizes the completion time and reduces overall survey burden.^{4,12,13,17,18,20,23}

Previously validated PROMs for knee pathology include the 36-Item Short Form Health Survey (SF-36), Knee injury and Osteoarthritis Outcome Score (KOOS), Western Ontario and McMaster Universities Osteoarthritis Index, Kujala Anterior Knee Pain Scale (AKPS), and EuroQol-5 dimensions (EQ-5D). However, the ability of these PROMs to capture incremental changes in PF assessed in high-functioning, younger patients is limited.^3,21,36 Ceiling effects have been observed in prior studies^5,19,30 utilizing legacy instruments in patients with knee conditions. Further, the legacy PROMs utilized for patients with a diagnosis of PFI typically require a fixed number of questions answered by all respondents, which greatly increases the survey burden, questionnaire fatigue, and overall survey administration time.²⁰ PROMIS PF-CAT has been evaluated in patients with other knee pathologies, including anterior cruciate ligament reconstruction (ACLR) and meniscal repair, which is why it was chosen in this study for patients with PFI.^‡

The purpose of the present study was to administer the PROMIS PF-CAT in patients with a diagnosis of PFI indicated for operative intervention with medial patellofemoral ligament reconstruction, Fulkerson osteotomy, or a combination of the 2 procedures. We hypothesized that (1) the PROMIS PF-CAT would show the greatest convergent validity with legacy instruments focusing on PF and modest convergent validity with PROMs assessing general health (GH) and quality of life (QOL) measures and (2) the PROMIS PF-CAT would have fewer floor and ceiling effects and pose a lower survey burden compared with legacy instruments.

Methods

This study was approved by the institutional review board at the participating institution and deemed Health Insurance Portability and Accountability Act compliant. A sample size estimate was performed utilizing SAS Statistical software (Version 9.4; SAS Institute). We determined that a sample size of 46 participants would provide 80% power, at an alpha level of 0.05, to detect at least a moderate correlation (0.4) between PROM instruments.^15,16,36 A total of 91 patients with a diagnosis of PFI indicated for operative intervention were enrolled in the patellofemoral registry, and all were eligible for inclusion in the current study. Six fellowship-trained sports medicine orthopaedic surgeons made the diagnosis of PFI based on injury presentation and clinical imaging. Characteristic variables included age, body mass index (BMI), sex, and smoking status. These characteristic and patient-specific variables were collected to control for confounding variables across the patients included in the analyses. All 91 patients completed the PROMIS PF-CAT, the SF-36, the KOOS, Marx Activity Rating Scale (Marx), and the EQ-5D. Half of the study participants completed the PROMIS PF-CAT first and the other half of the study participants completed the PROMIS PF-CAT last via random assignment. In total, 45 of 91 patients completed the Kujala AKPS because of the addition of the survey later in the study. The addition of the Kujala questionnaire was also randomized to the first or latter portion of the PROMs to evenly distribute the possible effect of survey fatigue among the questionnaires.

Descriptive statistics included frequency distributions and estimation of summary measures. Construct validity was tested by assessing the correlation between the PROMIS PF-CAT and legacy PROMs that measured (1) PF (convergent validity: SF-36 PF subscale and the KOOS Activities of Daily Living (ADL) and Sports/Recreation subscales) and (2) other health domains (Marx, SF-36 GH, EQ-5D, and KOOS subscales of Pain, QOL, and Symptoms). The distributions of continuous variables were evaluated using the Shapiro-Wilk test and through evaluation of histograms. Pearson or Spearman correlation coefficients were used to describe the relationships between PROMs. Correlation was defined as excellent (0.7), excellent-good (0.61-0.7), good (0.31-0.6), or poor (0.2-0.3).^7,35 Convergent validity, as indicated by strong correlation coefficients, was expected between PROMIS PF-CAT and instruments more specific for function, whereas divergent validity was expected with instruments measuring other health domains, as this was found to be true in other patient populations.³⁶ Ceiling and floor effects were evaluated by determining the proportion of participants who achieved the highest and lowest possible scores on each PROM, with the order dependent on the specific PROM. Floor and ceiling effects were considered present if more than 15% of participants scored the lowest or highest possible total PROM score.³⁹

Results

Of the 91 patients, 58 were females (63.7%) with a mean age of 20.1 ± 7.2 years and BMI of 26.9 ± 7.2 kg/m². Within the cohort, 7 (7.7%) patients were current smokers, 3 (3.3%) were former smokers, 76 (83.5%) were nonsmokers, and 5 (5.5%) did not report a smoking status. Participant characteristics can be found in Table 1. Participants completed a median of 4 questions (range, 4-10) utilizing the PROMIS PF-CAT (Table 2).

Table 1.

Characteristics of the Study Cohort (N = 91)^a

Variable
Age, mean ± SD, y	20.1 ± 7.2
BMI, mean ± SD, kg/m2	26.9 ± 7.2
Sex, n (% female)	58 (63.7)
Smoking status
Current	7 (7.7)
Former	3 (3.3)
Never	76 (83.5)
Not reported	5 (5.5)
Insurance type
Private	61 (67.0)
Medicaid	27 (29.7)
Other	3 (3.3)

^aBMI, body mass index.

Table 2.

Description of PROM^a

PROM Instrument	No. of Items	Health Domains Assessed	PROM Instrument	Number of Items	Health Domains Assessed
PROMIS PF-CAT	4-10	PF	PROMIS PF-CAT	4-10	PF
SF-36	36	Includes subscales of PF, pain, general health, vitality, mental health, and social functioning	EQ-5D	6	QOL
SF-36 PF	10	PF	AKPS	13	Symptoms and function
KOOS	43	Includes subscales of symptoms, ADL, Sports/Recreation, and QOL	Marx	4	Activity
KOOS ADL	17	Function in daily living	SF-36
KOOS Sports/Recreation	5	Function in sport and recreation	PF	10
WOMAC	Calculated from KOOS	Pain and stiffness	Physical limitations	4
EQ-5D	6	Overall function	Emotional limitations	3
AKPS	13	Symptoms and function	Energy	4
Marx	4	Activity	Emotional well-being	5
			Social functioning	2
			General health	5
			Pain	2
			KOOS
			Pain	9
			Symptoms	7
			ADL	17
			Sports/Recreation	5
			QOL	4

^aADL, Activities of Daily Living; AKPS, Anterior Knee Pain Scale; CAT, computer adaptive test; EQ-5D, EuroQol-5 dimensions; KOOS, Knee injury and Osteoarthritis Outcome Score; Marx, Marx Activity Rating Scale; PF, physical function; PROM, patient-reported outcome measure; PROMIS, Patient-Reported Outcome Measurement Information System; QOL, Quality of Life; SF-36, 36-Item Short Form Health Survey; WOMAC, Western Ontario and McMaster Universities Osteoarthritis Index.

The PROMIS PF-CAT demonstrated excellent correlation with the SF-36 PF subscale (r = 0.78; P < .01); excellent-good correlation with Kujala (r = 0.68; P < .01), EQ-5D (r = 0.60; P < .01), and the KOOS subscales of ADL (r = 0.68; P < .01) and Pain (r = 0.62; P < .01); and good correlation with KOOS subscales of Sports/Recreation (r = 0.58; P < .01), QOL (r = 0.53; P < .01), and Symptoms (r = 0.47; P < .01). The results of these analyses are represented in Table 3. The PROMIS PF-CAT demonstrated no floor or ceiling effects, with zero patients achieving the lowest or highest possible outcome score. By definition, none of the other instruments demonstrated floor or ceiling effects (>15% of patients having the lowest or highest possible outcome score), but these instruments did have a percentage of patients achieving the lowest or highest possible outcome score (Table 4).

Table 3.

Spearman Correlation Coefficients of PROMIS PF-CAT Versus Traditional Instruments With Those Expected to Have Convergent and Divergent Validity According to Health Domains Assessed^a

Instrument	Spearman Correlation	P
Convergent validity
SF-36 PF	0.78	<.01
KOOS ADL	0.68	<.01
KOOS Sports/Recreation	0.58	<.01
AKPS	0.68	<.01
Divergent validity
Marx	0.13	.23
SF-36 GH	0.12	.26
KOOS Pain	0.62	<.01
KOOS QOL	0.53	<.01
KOOS Symptoms	0.47	<.01
EQ-5D	0.60	<.01

^aADL, Activities of Daily Living; AKPS, Anterior Knee Pain Scale; CAT, computer adaptive test; EQ-5D, EuroQol 5 dimensions; GH, general health; KOOS, Knee injury and Osteoarthritis Outcome Score; Marx, Marx Activity Rating Scale; PF, physical function; PROMIS, Patient-Reported Outcome Measurement Information System; QOL, Quality of Life; SF-36, 36-Item Short Form Health Survey.

Table 4.

Floor and Ceiling Effects of PROMIS PF-CAT and Legacy PROMs^a

Instrument	Number of Items	Ceiling	Floor
PROMIS PF-CAT	4.3 ± 1.1	0	0
SF-36 PF	10	3 (3.3%)	1 (1.1%)
KOOS ADL	17	8 (8.8%)	0
KOOS Sports/Recreation	5	4 (4.4%)	8 (8.8%)

^aADL, Activities of Daily Living; CAT, computer adaptive test; KOOS, Knee injury and Osteoarthritis Outcome Score; PF, physical function; PROM, patient-reported outcome measure; PROMIS, Patient-Reported Outcome Measurement Information System; SF-36, 36-Item Short Form Health Survey.

Discussion

This study evaluated the performance of the PROMIS PF-CAT in patients with PFI preoperatively. The results demonstrated strong construct validity in comparison with legacy PROMs, no floor or ceiling effects, and low respondent burden overall. These findings suggest that the PROMIS PF-CAT may be valid for use in patients with a diagnosis of PFI.

Patient-reported outcome instruments tailored to specific patient populations are clinically beneficial to assess subjective, patient-perceived outcomes in a standardized manner. Although previous instruments have been validated in orthopaedic patients with knee pathology, these prior tools are often lengthy and cumbersome to administer.^15,16,34 The PROMIS PF-CAT allows for a more streamlined administration process for PROM data collection in comparison with previously validated tools while reducing the overall survey burden.²⁰

We hypothesized that the PROMIS PF-CAT would demonstrate the strongest convergent validity with previously validated tools that assess PF. Legacy PROMs that have been used to evaluate patients with knee pathology include instruments that focus on PF (SF-36 PF, KOOS ADL, KOOS Sports/Recreation, and AKPS), whereas others explore additional health domains (Marx, SF-36 GH, KOOS Pain, KOOS QOL, KOOS Symptoms, and EQ-5D). Previous studies have indicated that measures of the PROMIS PF-CAT correlated most strongly with instruments assessing PF,^10,16,36 with modest convergent correlations reported with PROMs measuring GH and QOL measures in the orthopaedic population.^11,15 In the current study, the PROMIS PF-CAT had slightly higher correlation coefficients with instruments of PF and pain, such as the SF-36 PF and KOOS ADL. Consistent with the results of previous studies, the PROMIS PF-CAT demonstrated excellent correlation with validated legacy PROMs that assessed measures of PF in other patient populations with knee pathology.^15,16 Moderate correlations were also found between the PROMIS PF-CAT and instruments expected to have more divergent validity based on health domain assessed. For example, there was good-excellent correlation between the PROMIS PF-CAT and KOOS Pain. This is likely because of the interplay of the health domains, for example, as one may expect pain to affect PF. These findings suggest that the PROMIS PF-CAT may be a valid alternative to traditionally employed PROM instruments when applied to adolescent and young adult populations with a diagnosis of PFI.

The PROMIS PF-CAT demonstrated no floor or ceiling effects, defined as 15% of patients who achieved the lowest or highest possible score, respectively, among a young patient population with PFI. Ceiling effects may indicate that higher functioning patients were not identified by a PROM adequately¹² and are of particular concern in young and active populations. Previous studies have detected ceiling effects using the PROMIS Upper Extremity instrument in young patients with shoulder instability.² The PROMIS PF-CAT, however, has been shown to demonstrate no floor or ceiling effects in patients undergoing ACLR or meniscal surgery.^15,36 In the present study, we found no ceiling effects when utilizing the PROMIS PF-CAT for patients with PFI; yet, the patients in this cohort were indicated for surgery and may therefore have more severe symptoms and functional limitations. Ceiling effects may be present, however, following recovery and rehabilitation after operative intervention for PFI, which will be the focus of a future study in the postoperative period. The absence of ceiling effects in our study may suggest that the question bank contains sufficient numbers of questions to differentiate PROM scores preoperatively.^12,14,28,38

The use of CAT with instruments such as the PROMIS PF-CAT allows for fewer questions and greater precision than traditional PROMs.^12,20 Previous orthopaedic studies have demonstrated the superiority of the PROMIS PF-CAT compared with traditional PROMs in terms of efficient administration time and reduced survey burden.^2,8,9,37 When utilizing the PROMIS PF-CAT in the current study, patients answered a mean 4.3 ± 1.1 questions. This represents a reduction in question burden when compared with traditionally employed PROMs for knee pathology (Table 2). A decreased number of questions leads to decreased PROM administration time and likely improves completion rates and patient satisfaction when filling out a PROM.^12,22,25

There are several limitations to the current study. As part of the survey administration, patients were tasked with completing multiple PROM instruments sequentially. Questionnaire fatigue and survey burden may have developed. Future studies assessing use of the PROMIS PF-CAT with fewer survey instruments may help reduce this bias. However, previous literature has demonstrated that the order in which patients complete the PROMIS survey does not influence PROM results.² In the current study, approximately half of the patients filled out PROMIS questions first and half completed the PROMIS instrument last. The Kujala AKPS was not instituted until later in the study, which affected the number of participants who completed the survey. The data were nonetheless included, as there were a sufficient number of patients according to our power analysis. This study was designed to capture PROMs at a single time point among a prospective cohort of patients and did not assess responsiveness to changes in patient condition over time. Therefore, our results may not apply to postoperative patients who may be more prone to ceiling effects, warranting further investigation. The patients assessed by the present study included a predominantly Caucasian population from a single, large academic institution in the Midwest, which may limit the external generalizability of our results.

Conclusion

The PROMIS PF-CAT demonstrated excellent-to-good correlations with traditional PROM instruments used for knee pathology as well as the SF-36 PF subscale in patients undergoing operative intervention for PFI. The PROMIS PF-CAT may capture measures of PF especially well because of strong correlations with other PROM tools assessing PF and moderate correlations with tools exploring other health domains. Additionally, the PROMIS PF-CAT has demonstrated the ability to detect differences in PROM scores for high-functioning patient populations, with the results of the present study demonstrating that the PROMIS PF-CAT may be applied to these populations with zero floor and ceiling effects. Therefore, we suggest that the PROMIS PF-CAT is an efficient PROM to evaluate adolescent and young adult patient populations with a diagnosis of PFI.