PROMIS Versus Legacy Patient-Reported Outcome Measures for Sports Medicine Patients Undergoing Arthroscopic Knee, Shoulder, and Hip Interventions: A Systematic Review

Abstract

Background

The Patient-Reported Outcomes Measurement Information System (PROMIS®)^ǂ was designed to monitor the global wellbeing of patients, with the Physical Function Computer-Adaptive Test (PF-CAT) component focused specifically on functional outcome. PROMIS aims for increased item-bank accuracy, lower administrative burden, and decreased floor and ceiling effects compared to legacy patient-reported outcome measures (PROMs). Our primary research outcomes focused on sports medicine surgical populations, which may skew younger or have wide-ranging functional statuses. Specifically, for this population, we questioned if PROMIS PF-CAT was equal to legacy PROMs in (1) construct validity and (2) convergent/divergent validities; and superior to legacy PROMs with respect to (3) survey burden and (4) floor and ceiling effects.

Methods

Searches were performed in April 2019 in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, utilizing PubMed, Cochrane Central, and Embase databases for Level I-III evidence. This resulted in 541 records, yielding 12 studies for inclusion. PROM data was available for patients undergoing arthroscopic orthopaedic procedures of the knee, shoulder, and hip. Measures of construct validity, convergent/divergent validity, survey burden, and floor/ceiling effects were evaluated for PROMIS PF-CAT versus legacy PROMs.

Results

PROMIS PF-CAT demonstrated excellent or excellent-good correlation with legacy PROMS for physical function and quality of life for patients undergoing arthroscopic interventions of the knee, shoulder, and hip. Compared to legacy PROM instruments, PROMIS PF-CAT demonstrated the lowest overall survey burden and had the lowest overall number of floor or ceiling effects across participants.

Conclusion

PROMIS PF-CAT is an accurate, efficient evaluation tool for sports medicine surgical patients. PROMIS PF-CAT strongly correlates with legacy physical function PROMs while having a lower test burden and less incidence of floor and ceiling effects. PROMIS PF-CAT may be an optimal alternative for traditional physical function PROMs in sports medicine patients undergoing arthroscopic procedures. Further studies are required to extend the generalizability of these findings to patients during postoperative timepoints after shoulder and hip interventions

Level of Evidence: III

Introduction

The National Institutes of Health developed the standardized Patient-Reported Outcomes Measurement Information System (PROMIS®)ǂ to monitor physical, social, and mental wellbeing of patients.^1,2 In the age of value-based care and shared decision-making between patients and surgeons, Patient-Reported Outcome Measures (PROMs) are often employed to assess patient-specific variables. A specific sub-analysis in the PROMIS group, PROMIS Physical Function Computer-Adaptive Test (PF-CAT), has been determined to effectively assess the same information as legacy PROMs, while requiring fewer questions on average and having less statistical interference for low and high-scoring patients. PROMIS PF-CAT has been deemed advantageous for use in general orthopaedic patients, both pre- and postoperatively.^2,3

Any widely-utilized PROM should demonstrate high content validity, reliability, and item responsiveness across survey participants.^1,2,4 Previous literature has demonstrated that PROMIS PF-CAT correlates strongly with previously validated PROMs for patients undergoing arthroscopic procedures of the knee, shoulder, and hip. PROMIS PF-CAT test/re-test reliability is also equivalent to or exceeding that of legacy measures.^4-6 In addition, PROMIS PF-CAT has demonstrated a reduced survey burden (number of questions and/or time for completion) as compared to legacy PROM instruments. For patient populations which tend to score on the low or high end of legacy measures, PROMIS PF-CAT has the advantage of the lowest overall floor and ceiling effects (attaining the bottom or top 15% scores, respectively) compared to legacy PROMs.^7-10 Increased item-bank accuracy, lower administrative burden, and decreased floor and ceiling effects make PROMIS PF-CAT an attractive alternative for orthopaedic sports medicine practices. In busy sports medicine practices PROMIS is both efficient and adaptive, and accommodates a population which tends to skew younger, functionally high-level, and therefore prone to ceiling effects on non-adaptive PROMs.¹¹

The purpose of this systematic review was to assess the utility of PROMIS-PF-CAT for patients undergoing arthroscopic sports medicine interventions of the knee, shoulder, and hip. Specifically, for this population, we questioned if PROMIS PF-CAT was equal to legacy PROMs in (1) construct validity and (2) convergent/ divergent validities; and superior to legacy PROMs with respect to (3) survey burden and (4) floor and ceiling effects. We hypothesized that PROMIS would correlate strongly with legacy sports medicine PROMs of the knee, shoulder, and hip; while demonstrating decreased end-range statistical errors and overall lower survey burden compared to the legacy PROMs.

Methods

A systematic review was performed in April 2019 in accordance with the 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (PRISMA) utilizing PubMed, Cochrane Central, and Embase databases. Eligible Level I, II, or III studies published on or before April 2019 included patients undergoing operative arthroscopic sports medicine interventions of the knee, shoulder, or hip at the preoperative and/or postoperative timepoints and utilization of PROMIS PF-CAT as a patient-reported outcome measure. PubMed MeSH terms for the patient-reported outcomes search component included:

“(((((((patient-reported outcomes measurement information system) OR patient-reported outcomes measurement information systems) OR patient reported outcomes measurement information system) OR patient reported outcomes measurement information systems) OR PROMIS)) AND (physical function OR physical functions OR PF OR pf))”

MeSH terms for the patient population/target intervention included:

“((((((((((((shoulder joint) OR knee joint) OR hip joint) OR acromioclavicular joint) OR knee ligament) OR meniscus) OR rotator cuff) OR anterior cruciate ligament) OR posterior cruciate ligament) OR collateral ligaments) OR cartilage, articular) AND (((((((((shoulder injury) OR knee injury) OR ligament injury) OR impingement syndromes, shoulder) OR hip injuries) OR patellofemoral syndrome) OR cartilage disease) OR bankart lesion) OR musculoskeletal injury)) AND (((endoscopy) OR minimally invasive surgical procedures) OR arthroscopy)”

Search components were then combined, with the English language filter being the only pre-established search restriction utilized. Search terms were then adapted for use in the other databases. Full search strategies are available upon request.

The search was conducted under the guidance of a trained librarian specializing in orthopaedic medicine. In total, 541 unique records resulted after removal of duplicate records, which was facilitated by software screening of the records supplemented by manual review. Of the 541 unique records, 515 articles were excluded for not meeting the necessary inclusion requirements and/or for aspects qualifying as exclusion criteria (Figures 1-2). Of the 26 full-text articles reviewed by 2 research assistants under the guidance of 2 fellowship trained sports medicine surgeons; 12 articles satisfied the inclusion criteria (Figures 1-2) and were included for quantitative and qualitative analyses (Table 1).

Figure 1.

Included and Excluded Studies of PROMIS PF-CAT for Sports Medicine Patient Populations.

Figure 2.

Inclusion and Exclusion Criteria for Review Literature.

Table 1.

Patient Demographic Data and Individual Study Inclusion/Exclusion Criteria, by Joint

	Author	Year	Level of Evidence	Number of Patients (% male)	Mean Age (Range) or ±Std (years)	±Mean Std; kg/BMI m3 or Weight (lbs)	Condition and/or Procedure Type	Inclusion Criteria	Exclusion Criteria
Shoulder	Anthony et al.7	2017	II	82 (61.0%)	54 (22-72)	31.7±7.4	Rotator cuff repair	Preoperative diagnosis of RCa pathology	Incomplete survey data
	Anthony et al.11	2017	II	70 (74.3%)	27 (12-54)	27±5.1	Surgical management of shoulder instability	Scheduled for operative intervention	Incomplete survey data
	Patterson et al.24	2018	II	164 (51.8%)	58 (18-79)	30±6.2	Rotator cuff repair	Primary rotator cuff repair, preoperative PROMsb	Revision rotator cuff repair, younger than age 18, incomplete survey responses
Knee	Bernholt et al.20	2018	III	75 (50.7%)	53.3 (22.2-72.7)	30.6±6.1	Arthroscopic partial meniscectomy	Had preoperative PRO- MIS scores	Additional procedures performed
	Chen et al.21	2018	III	233 (50.6%)	26.5±12.5	Not provided	ACLc reconstruction	Primary reconstruction, pre- and post-operative PROMIS scores	None
	Hancock et al.8	2017	III	107 (34.6%)	37.7±16.0	29.6±6.3	Arthroscopic meniscal surgery	Scheduled for meniscal repair, meniscectomy, or debridement	Major simultaneous procedures or grade 4 OA
	Hancock et al.9	2018	III	100 (45%)	26 (11-57)	27.2±6.2	ACL reconstruction	Indicated for operative management of ACL injury	Significant simultaneous operations
	Meredith et al.22	2018	III	383 (55.4%)	40.2±13.6 (opioid), 39.0±16.7 (no opioid)	30±6.7 (opioid), 29.1±5.6 (non-opioid)	Knee surgery w/wo preoperative opioid use	Over 12 years of age, English speaking, not incarcerated or a ward of the state	No data on preoperative opioid use
	Papuga et al.23	2014	III	106 (46.2%)	29.5±13.3	167.7 lb	ACL reconstruction	Age 13+, scheduled for BTBd autograft ACL reconstruction	Disorders of balance, gait degeneration, or neuromuscular junction; dementia, depression, or cognitive impairment
	Scott et al.11	2018	II	111*	24±9.3	26.3±4.7	ACL reconstruction	Primary or revision reconstruction, age 14+, written English proficiency	Bilateral ACL reconstruction
Hip	Kollmorgen et al.25	2019	III	125 (18.4%)	38.1 (18-70)	26.6±5.01	Arthroscopic hip procedures	Age 18-80, completion of CAT and legacy scores	Repeat encounters, non-English speaking, no CAT data
	Sheean et al.26	2017	III	42 (71.4%)	32.2±6.4	186 lb (con- trol), 198.7 lb (FAI)	FAIe	Unilateral hip pain, age 18-50, positive impingement test, relief after IA anesthetic injections	Radiographic evidence of OAf; history of femoral head pathology or hip dysplasia
	Nwachukwu et al.34**		II	196 (23.4%)	32.8±12.6	25.7±5.6	FAIe	Clinical and radiographic findings of symptomatic FAI, failure of conservative management	Prior surgery on hip; non-FAI indications, concomitant procedures; hip OA, dysplasia, or congenital disorders

Table 1: ^aRC:Rotator cuff; ^bPRO: Patient-Reported Outcome Measure; ^cACL: Anterior cruciate ligament; ^dPatellar bone-tendon-bone graft; ^eFemroacetabular impingement; ^fOsteoarthritis; *sex data unavailable; study used for discussion purposes only, not meta-analysis

Measures of PROMIS PF-CAT construct validity, convergent validity, and divergent validity were evaluated as compared to legacy PROMs. Based on prior literature detailing research of the PROMIS PF-CAT instrument^7,11,12 the correlation cut-off (r=0.4) was used to differentiate between convergent correlations (r>0.4, p<0.05) and divergent correlations (r<0.4; p>0.05), respectively. Convergent validity was defined as moderate-strong correlation coefficients between survey instruments (r>0.4, p<0.05). Divergent validity was defined as weak or no correlation between survey instruments (r<0.4; p>0.05). Survey burden, floor effects, and ceiling effects were additional factors compared between PROMIS PF-CAT and legacy PROMs. When possible, data was stratified by anatomic location (knee, shoulder, hip) for additional analyses. The overall quality of included evidence was evaluated by two research assistants, utilizing the Modified Coleman Methodology, under the supervision of two fellowship trained orthopaedic surgeons.

Legacy Sports Medicine PROMs for Knee, Shoulder, and Hip

Legacy PROMs for the knee in the present study included the Knee Injury and Osteoarthritis (KOOS) battery [Activities of Daily Living (ADL), Quality of Life (QOL), Symptoms, Sports Participation (Sports), and Pain] and the International Knee Documentation Committee score (IKDC).¹³ KOOS, depending on which tests in the battery are utilized, assesses functional, work-related, social, and emotional dimensions of knee symptoms.¹³ IKDC assesses the functional status of the knee.¹³

Legacy PROMs for the shoulder included the Western Ontario Shoulder Instability Index (WOSI), Western Ontario Rotator Cuff Index (WORC), American Shoulder and Elbow Surgeons Score (ASES), and the SST (Simple Shoulder Test).^14,15 WOSI, WORC, and ASES assess the functional status of the shoulder and the impact of shoulder symptoms on work, social, and emotional aspects of patient lives.¹⁴ SST assesses only functional status and work status.¹⁵

Legacy PROMs for the hip included the modified Harris Hip Score (mHHS), the International Hip Outcome Tool (iHOT-12), and the Hip Outcome Score (HOS) battery [Sports-Specific (SS) and Activities of Daily Living (ADL)].¹⁶ The HOS battery and iHOT assess functional status; and work-related, social, and emotional concerns related to hip symptoms.¹⁶ The mHHS focuses on the physical function of the hip.¹⁶

Additional legacy sports medicine PROMs assessed across patients undergoing operative interventions of the knee, shoulder, and hip included the EuroQol Five Dimensions test (EQ-5D), Veterans Rand 6 Domain (VR-6D), Marx Activity Rating Scale (Marx) and the Short Form Survey (SF-36) test battery [Physical Function (PF), General Health (GH), and Pain subscales].^17-19 VR-6D assesses physical and emotional status and SF-36 assesses physical, emotional, and work-related ability to function.^18,19 EQ-5D assesses physical, emotional, social, and work-related aspects of health.¹⁸ Marx assesses overall physical function.¹⁷

Results

Study Population Demographics and Characteristics

The present review included 12 studies of patients undergoing arthroscopic interventions of the knee (7),^8-10,20-23 shoulder (3),^7,11,24 and hip (2)^25,26 (Table 6). In total, PROM data was available for assessment from 1105 knee arthroscopy patients, 210 shoulder arthroscopy patients, and 167 hip arthroscopy patients. Demographic variables for the arthroscopic populations were investigated in groups, stratified by joint undergoing surgery. Knee patients were 51.9% male (547 patients), with a mean age of 34.2 years (range: 11-72 years), and a mean BMI of 28.7±5.9 kg/m² (data not available for 2/7 studies). Shoulder patients were 67.1% male (102 patients), with a mean age of 46.3 years (range: 12-79 years), and a BMI of 29.4±6.3 kg/m²). Hip patients were 31.7% male (53 patients), with a mean age of 32 years (range: 18-70 years), and a mean BMI of 26.6±5.0kg/m² (Table 1). The notably wide patient age ranges were helpful in assessing what limitations, if any, were present in PROMIS PF-CAT adaptations to avoid floor or ceiling effects.

Table 6.

Study Outcome Measures utilizing PROMIS PF-CAT for Arthroscopic Interventions of the Shoulder, Knee, and Hip

	Author	Year	Condition Procedure and/Type or	Outcome Measure	Results
Shoulder	Anthony et al.1	2017	Rotator cuff repair	Correlationb of PROMIS PF CATa with traditional shoulder and upper extremity PROMsc	PROMIS PF CAT had excellent correlation with SF-36 PFa; excellent-good correlations with EQ-5Da, WORCa; no floor effects; 1 ceiling effect.
	Anthony et al.2	2017	Surgical management of shoulder instability	Correlationb of PROMIS PF CAT with traditional shoulder and upper extremity PROMs	PROMIS PF CAT had excellent correlation with SF-36 PF; excellent-good correlation with ASESa, no ceiling effects.
	Patterson et al.24	2018	Rotator cuff repair	Correlationd of PROMIS PF CAT with ASES and SSTa	PROMIS PF-CAT had weak correlations with ASES and moderate correlations with SST.
Knee	Bernholt et al.20	2018	Arthroscopic partial meniscectomy	Ability of PROMIS to detect early variations in postoperative outcomes	Preoperative PROMIS PF-CAT scores were significantly lower than the general population. At 6 weeks postoperatively, significant improvements in PF scores were seen.
	Chen et al.21	2018	ACL (anterior cruciate ligament) reconstruction	Changes in PROMIS scores postoperatively; predictive ability of preoperative PROMIS scores to identify patients who will not achieve MCIDe	Patients on average had significant improvement in PROMIS PF-CAT measures.
	Hancock et al.8	2017	Arthroscopic meniscal surgery	Correlation of PROMIS PF-CAT with legacy PROM instruments	PROMIS PF-CAT had high correlations with SF-36 PF, KOOS Sporta; high-moderate correlations with KOOS QOLa, EQ-5D. Average number of PROMIS questions (4). No floor or ceiling effects.
	Hancock et al.9	2018	ACL reconstruction	Correlation of PROMIS PF-CAT with current measures of knee PROMs, PROMIS test burden, floor/ceiling effectsf	PROMIS PF-CAT demonstrated high correlations with SF-36 PF, EQ-5D, KOOS ADLa, and KOOS Sport. Mean number of questions (4.2). No floor or ceiling effects.
	Meredith et al.22	2018	Knee surgery with, without preoperative opioid use	Correlation of clinical variables and PROMs with opioid use. (significance at p<0.05)	Preoperative opioid use was associated with worse scores using PROMIS and legacy PROM instruments.
	Papuga et al.23	2014	ACL reconstruction	Time to administer PROMIS, correlation of PROMIS and legacy PROMs, ability of PROMIS to identify patients likely to have poor outcomes	PROMIS took less time to administer and detected functional changes better than IKDCa. Patients with poor outcomes had significantly lower PROMIS scores.
	Scott et al.10	2018	ACL reconstruction	Correlationb between PROMIS PF CAT and legacy PROMs	PROMIS PF-CAT had excellent, excellent-good, or good correlations with SF-36 PF, KOOS-ADL, KOOS-Sport, KOOS-QOL, and EQ-5D preoperatively and postoperatively. Minimal floor and ceiling effects, mean 4 questions.
Hip	Kollmorgen et al.25	2019	Arthroscopic hip procedures	Correlationg of PROMIS with legacy measures, PROMIS floor and ceiling effects	PROMIS PF-CAT had strong correlations with mHHSa, iHOT-12a, HOS-SSa, HOS-ADLa, and VR-6Da. Ceiling effects were not demonstrated for any PROMIS measures.
	Sheean et al.26	2017	FAI	PROMIS score differences between FAI (femoroacetabular impingement) patients and controls, association of PROMs and PF measuresh	Significant differences were found for legacy PROMs, physical performance, and PROMIS measures between the FAI and control group.
	Nwachukwu et al.34	2019	FAI	Correlation of PROMIS with legacy PROMs for hip arthroscopy patients undergoing operative intervention for FAI	PROMIS PF-CAT demonstrated good-excellent correlations with legacy hip-specific instruments as well as hip-related quality of life measures

Table 6 Key

¹Anthony et al. “Preoperative Performance of the Patient-Reported Outcomes Measurement Information System in Patients With Rotator Cuff Pathology”

²Anthony et al. “Performance of PROMIS Instruments in Patients With Shoulder Instability” ________________________________________

^asee key below for PROM acronyms

^bExcellent >0.7, excellent-good 0.61-0.7, good 0.4-0.6, poor 0.2-0.3; significance at p<0.05

^cPROMs= Patient-Reported Outcome Measures

^dVery strong 0.90-1.00, strong 0.70-0.89, moderate 0.50-0.69, weak 0.30-0.49, none 0.00-0.29; significance at p<0.05

^eMCID= minimal clinically important difference (significance at p<0.05)

^fHigh 0.7+, high-moderate 0.61-0.69, moderate 0.4-0.6, moderate-weak 0.31-0.39, weak 0.3-; significance at p<0.05

^gStrong >0.7, mod-strong 0.5-0.7, weak 0.3-0.5, negligible 0-0.3; significance at p<0.05

^hHigh r>0.5, moderate r=0.3-0.5, low r<0.3; significance at p<0.05

__________________________________________

^aPatient-Report Outcome Measures (PROM) Instruments

  PROMIS PF (CAT): Patient-Reported Outcomes Measurement Information System, Physical Function (Computer-Adaptive Testing)

  SF-36 (PF, GH, PAIN): Short Form Survey (Physical Function, General Health, Pain)

  EQ-5D: EuroQol Five Dimensions

  WOSI: Western Ontario Shoulder Instability Index

  WORC: Western Ontario Rotator Cuff Index

  ASES: American Shoulder and Elbow Surgeons Score

  MARX/MARS: Marx Activity Rating Scale

  SST: Simple Shoulder Test

  KOOS (ADL, QOL, Symptoms, Sport, Pain): Knee Injury and Osteoarthritis Outcome Score (Activities of Daily Living, Quality of Life,

  Symptoms, Sports Participation, Pain)

  IKDC: International Knee Documentation Committee

  mHHS: Modified Harris Hip Score

  iHOT-12: International Hip Outcome Tool

  HOS(-SS, -ADL): Hip Outcome Score (Sports-Specific, Activities of Daily Living)

  VR-6D: Veterans Rand 6 Domain

The majority of the included studies (9/12, 75.0%) directly compared PROMIS PF-CAT with legacy PROMs assessing physical function, such as ASES (shoulder),¹ KOOS (knee),¹⁹ and mHHS (hip)^26,41 (Table 2).^{1,2,19,20,26,33,39,41} PROMs assessing general measures of health were also compared to PROMIS PF-CAT.^1,2,19,20 The remaining 3 studies examined the ability of PROMIS PF-CAT to detect additional measures of health and function.^4,11,28,32 Construct validity, convergent validity, divergent validity, survey burden, floor effects, and ceiling effects were evaluated for PROMIS PF-CAT in comparison to the legacy PROMs (Tables 3-5).^{1,2,19,20,26,32,39}

Table 2.

Correlation of PROMIS PF-CAT with Legacy PROMs, Stratified by Joint

Legacy PROM Instrument	Knee [Pearson/ Spearman Correlation Coefficient]	Shoulder [Pearson/ Spearman Correlation Coefficient]	Hip [Pearson/ Spearman Correlation Coefficient]
SF-36 PF	0.80±0.04	74.5±3.54
KOOS Sport	0.72±0.04
IKDC	0.90*
MARX	0.05±0.04**	0.26±0.04**
SF-36 GH	0.18±0.08**	0.5*
SF-36 Pain	0.56±0.06
EQ-5D	0.68±0.05	0.60±0.04
KOOS Symptoms	0.56±0.02
KOOS Pain	0.59±0.01
KOOS QoL	0.55±0.07
ASES		0.55±0.12
WORC		0.61*
WOSI		0.49*
mHHS			0.71*
iHOT-12			0.76*
HOOS ADL			0.87*
HOOS SS			0.81*
VR-6
HOS Sports
HOS ADL
VAS

Table 2. *Average value not available; **one or more of the individual p-values>0.05.

Table 3.

Convergent and Divergent Validities of PROMIS PF-CAT vs. Legacy PROMs, by Joint

Convergent Correlations						Divergent Correlations
	Study Author		Convergence Demonstrated [r>0.4; p<0.05]	Spearman Correlation Coefficient	p-value		Divergence Demonstrated [r<0.3; p>0.05]	Spearman Correlation Coefficient	p-value
Shoulder	Anthony et al.1ǂ					MARX	No	0.34	<0.01
						SF-36 GH	No	0.5	<0.01
	Anthony et al.2¥	ASES	Yes	0.67	<0.01	MARX	Yes	0.18	0.14
		WOSI	Yes	0.49	<0.01
		SF-36 PF	Yes	0.72	<0.01
	Shoulder Averages					MARX	Yes	0.26	0.075
Knee	Hancock et al.8	KOOS ADL	Yes	0.6	<0.01	MARX	Yes	0.05	0.59
		KOOS SPORT	Yes	0.76	<0.01	SF-36 GH	Yes	0.27	<0.01
		SF-36 PF	Yes	0.82	<0.01	SF-36 PAIN	No	0.6	<0.01
						KOOS SYM	No	0.57	<0.01
						KOOS PAIN	No	0.6	<0.01
						KOOS QOL	No	0.63	<0.01
						EQ-5D	No	0.62	<0.01
	Hancock et al.9	SF-36 PF	Yes	0.82	<0.01	SF-36 GH	Yes	0.12	0.22
		KOOS SPORT	Yes	0.7	<0.01	SF-36 PAIN	No	0.51	<0.01
		KOOS ADL	Yes	0.74	<0.01	KOOS SYM	No	0.54	<0.01
						KOOS PAIN	No	0.58	<0.01
						KOOS QOL	No	0.49	<0.01
						EQ-5D	No	0.70	<0.01
						MARX	Yes	0.08	0.46
	Papuga et al.23	IKDC	Yes	0.90*	<0.01
	Knee Averages	KOOS ADL	Yes	0.67	<0.01	MARX	Yes	0.07	0.53
		KOOS SPORT	Yes	0.73	<0.01	SF-36 GH	Yes	0.20	0.12
		SF-36 PF	Yes	0.82	<0.01	SF-36 PAIN	No	0.56	<0.01
						KOOS SYM	No	0.55	<0.01
						KOOS PAIN	No	0.59	<0.01
						KOOS QOL	No	0.56	<0.01
						EQ-5D	Yes	0.35	0.24

*Pearson correlation coefficient used by Papuga et al. only

^1ǂAnthony et. al “Preoperative Performance of the Patient-Reported Outcomes Measurement Information System in Patients With Rotator Cuff Pathology”

^2¥Anthony et. al “Performance of PROMIS Instruments in Patients With Shoulder Instability”

Table 5.

Question Burden for PROMIS PF-CAT vs. Legacy PROMs

PROM Instrument ǂ	Questions	Health Domains Assessed
PROMIS PF-CAT	4-12	Physical Function
SF-36	36	General Health, includes Physical Health (10 questions)
EQ-5D	6	Overall Function
VR-6D	12	Physical and emotional function
WOSI	21	Shoulder function, work ability, emotional and social impact
WORC	21	Shoulder function, work ability, emotional and social impact
ASES	11	Shoulder function, work ability, emotional and social impact
SST	12	Shoulder function and work ability
Marx	4	Knee Activity Rating Scale
IKDC	10	Knee function
KOOS	43	Knee Pain, Symptoms, Function, and Quality of Life
mHHS	8	Hip function
iHOT-12	12	Hip function, work ability, emotional and social impact
HOS	31	Hip function, work ability, emotional and social impact

^ǂTable 5 Key

• PROMIS PF (CAT): Patient-Reported Outcomes Measurement Information System, Physical Function (Computer-Adaptive Testing)
• SF-36 (PF, GH, PAIN): Short Form Survey (Physical Function, General Health, Pain)
• EQ-5D: EuroQol Five Dimensions
• WOSI: Western Ontario Shoulder Instability Index
• WORC: Western Ontario Rotator Cuff Index
• ASES: American Shoulder and Elbow Surgeons Score
• MARX/MARS: Marx Activity Rating Scale
• SST: Simple Shoulder Test
• KOOS (ADL, QOL, Symptoms, Sport, Pain): Knee Injury and Osteoarthritis Outcome Score (Activities of Daily Living, Quality of Life, Symptoms, Sports Participation, Pain)
• IKDC: International Knee Documentation Committee
• mHHS: Modified Harris Hip Score
• iHOT-12: International Hip Outcome Tool
• HOS(-SS, -ADL): Hip Outcome Score (Sports-Specific, Activities of Daily Living)
• VR-6D: Veterans Rand 6 Domain

PROMIS PF-CAT versus Legacy Patient-Reported Outcome Measures

Survey Validity in the Context of Varying Joint Procedures

In total, 4 of the 7 knee arthroscopy studies (57.1%) compared PROMIS PF-CAT to legacy knee PROMs (Table 2).^8-10,23 PROMIS PF-CAT demonstrated excellent correlation with SF-36 PF (r=0.80, p<0.01) and IKDC (r=0.90, p<0.01)³²; and excellent-good correlations with the KOOS sub-scales ADL (r=0.66, p<0.01) and Sports (r=0.67, p<0.01).^8-10 Moderate convergent validity with PROMIS PF CAT was present for SF-36 Pain (r=0.56, p<0.01) and KOOS subscales of Symptoms (r=0.55, p<0.01), Pain (r=0.59, p<0.01), and Quality of Life (r=0.56, p<0.01).^8,9 Divergent validity was demonstrated for Marx (r=0.07, p=0.53), SF-36 GH (r=0.20, p=0.12), and EQ-5D (r=0.35, p=0.24) (Table 3).^8,9

PROMIS PF-CAT was compared with legacy shoulder PROMs in all three included studies of patients undergoing shoulder arthroscopy (100%) (Table 2).^7,11,24 PROMIS PF-CAT demonstrated excellent correlation with SF-36 PF (r=0.74, p<0.01) and good correlation with ASES (r=0.51, p<0.01).^7,11,24 PROMIS PF-CAT demonstrated convergent validity with ASES (r=0.67, p<0.01), SF-36 PF (r=0.72, p<0.01), and WOSI (r=0.49, p<0.01).¹¹ Divergent validity was present for Marx (r=0.26, p=0.08) (Table 3).^7,11

Of the two included studies assessing hip arthroscopy patients, only Kollmorgen et al.²⁷ compared PROMIS PF-CAT to legacy hip PROMs (50%) (Table 2). PROMIS PF-CAT demonstrated excellent correlations with both mHHS (r=0.71, p<0.05) and iHOT-12 (r=0.76, p<0.05). Sheean et al. did not directly report correlation coefficients in their study of hip arthroscopy patients.⁴¹ Due to limited data, convergence and divergence values were unable to be reported for arthroscopic interventions of the hip.

Survey Burden, Floor Effects, and Ceiling Effects

In the studies of arthroscopic knee interventions, data from completed survey instruments was available from the preoperative timepoint for 590/1105 (53.4%) patients. Additionally, survey data was available for 515/1105 (46.6%) knee arthroscopy patients from both the preoperative and postoperative timepoints. Of the patients undergoing arthroscopic shoulder interventions, survey data was available for 210/210 (100%) of participants from the preoperative timepoint. Similarly, survey data for 364/364 (100%) of hip arthroscopy patients was available from the preoperative timepoint. It is noted that postoperative survey data in literature qualifying for this review is currently only available for arthroscopic procedures of the knee.

The mean number of PROMIS PF-CAT questions completed by patients undergoing arthroscopic intervention was stratified by joint (Table 4). To acquire all necessary outcome measures data, patients undergoing knee arthroscopy had to answer an average of 4.2±0.2 questions. Shoulder arthroscopy patients answered an average of 4.5±1.5 questions, and hip arthroscopy patients averaged slightly less, answering 4.1 questions (no standard deviation available).^7-11,20-26 The survey burden for each legacy PROM is provided in Table 5 for comparison. With the exception of the Marx scale for the knee (4 questions), PROMIS PF-CAT required fewer questions than all other legacy sports medicine PROMs, which ranged as high as 43 questions (KOOS). PROMIS PF-CAT also had floor and ceiling effects reported for at least one study in each subgroup of knee, shoulder, and hip; all reported effects being minimal (1.6%) to none (0%) (Table 4).

Table 4.

PROMIS PF-CAT Administration Data: Survey Burden, Floor Effects, and Ceiling Effects by Joint

	Study Author	Average Questions ±std	Floor Effects (%)	Ceiling Effects (%)
Shoulder	Anthony et al.7	4.3±1.2	0	1 (1.2%)
	Anthony et al.11	4.6±1.8	0	0
	Patterson et al.24	Not provided	Not provided	Not provided
Shoulder Averages ±std		4.5±1.5	0	(0.6%±0.9%)
Knee	Bernholt et al.21	Not provided	Not provided	Not provided
	Chen et al.23	Not provided	Not provided	Not provided
	Hancock et al.8	4.4±1.4	0	0
	Hancock et al.9	4.2±0.9	0	0
	Meredith et al.22	Not provided	Not provided	Not provided
	Papuga et al.23	Not provided	Not provided	Not provided
	Scott et al.10	4.0±1.6	0	0
Knee Averages ±Std		4.2±0.2	0	0
Hip	Sheean et al.26	Not provided	Not provided	Not provided
	Kollmorgen et al.25	4.1	Not provided	0
Hip Averages*		4.1	Not provided	0
Total		4.3±0.2	Not Provided	0

*Data not available for multi-study calculation of averages

Study Evidence Quality

For the included studies, the mean consensus Modified Coleman Methodology Score (MCMS) was 54 (range: 32-50). Scores were influenced by the nature of validation studies examining PROM instruments; most studies did not directly assess traditional surgical outcome measures, and smaller sample sizes were used. These factors lowered the overall MCMS scores. Consensus by the authors when comparing independently determined scores was that certain MCMS measures of study quality—including type of study, clearly defined endpoints, and demographic data of patients—were adequate. Despite lower scores for other MCMS criterion, such as outcome measures and radiographic measurements, the average value of included studies was “Good” per MCMS standards.

Discussion

In the current study, excellent-good correlations were found between PROMIS PF-CAT and legacy PROMs for sports medicine patient populations undergoing arthroscopic interventions of the knee, shoulder, and hip. This demonstrates the flexibility of PROMIS PF-CAT to independently collect outcome measures data which previously required utilization of multiple, joint-specific legacy PROMs. It also suggests a high construct validity for PROMIS PF-CAT, a vital element signaling that it does measure the intended endpoint(s), presuming validity of the legacy PROMs to which it is compared.^9,43 PROMIS PF-CAT was able to measure these endpoints while demonstrating the lowest overall survey burden, and minimizing floor and ceiling effects—efficiently adapting across variable populations.

The current study also showed that PROMIS PF-CAT had the strongest convergent correlations with PROM instruments assessing physical function and quality of life measures; while demonstrating the strongest divergent correlations with PROMs assessing additional health domains. PROMIS PF-CAT correlating well with legacy PROMs measuring joint function is consistent with the design of PF-CAT to focus on patient physical function. The strong divergent correlations with measures of general health, in turn, imply that PROMIS PF-CAT is able to differentiate a specific measured attribute (physical function). Essentially, the convergence and divergence data shows that PROMIS PF-CAT is capable of differentiating between measures of function, pain, and quality of life; and it is closely tailored to assess physical function rather than measures of general health.^28-31

Our findings differed from expectations regarding a frequently hypothesized divergent correlation in previous literature between PROMIS PF-CAT and PROMs assessing the general health of the patient. These results may partially be explained by the overlap in questions and attributes assessed by PROMIS PF-CAT; there are likely inter-domain overlaps between PROMIS PF-CAT and legacy general health PROMs such as KOOS Symptoms and EQ-5D, as reflected by some results reported in our review. However, by pooling convergence and divergence data from multiple studies, we were able to confirm that PROMIS PF-CAT does indeed have divergent correlation with more general measures of health, which do not traditionally overlap with joint health assessment themes.

Our results suggest that this single PROM instrument with a low overall survey burden may serve as a multi-domain alternative to employing numerous, lengthier legacy PROMs which individually assess one or two domains. PROMIS PF-CAT may provide the most utility for assessing physical function in multiple axial or central sites (shoulder, knee, hip).^21,25,26 This is particularly important for patient populations aiming to return to preinjury functional levels and/or return to sports; applicable to a large proportion of patients seen in orthopedic sports medicine practices.^32,33 Caution must be taken when extending generalizability to the postoperative timepoint due to a lack of postoperative correlations available for assessment, specifically for shoulder and hip arthroscopic procedures. Further investigations postoperatively are therefore warranted given the encouraging performance of PROMIS in the preoperative setting.^21,25,26

Interestingly, a study by Nwachuckwu et al.³⁴ that was published after the study inclusion date demonstrated excellent to good correlation of PROMIS PF-CAT with legacy hip PROMs in 197 preoperative hips, including mHHS, HOS, and iHOT-12. This data supports and reinforces preoperative hip findings in the current study. Nwachukwu et al. also recently demonstrated that postoperative knee patient PROMIS PF-CAT scores demonstrated stronger legacy PROM correlations postoperatively versus preoperatively, without exhibiting any floor or ceiling effects.³⁵ Scott et al. similarly demonstrated strong postoperative correlations of PROMIS PF-CAT up to 2 years postoperatively for patients undergoing ACL reconstruction.¹⁰ These more recent study findings continue to highlight the efficiency and effectiveness of PROMIS for both pre- and postoperative knee arthroscopy patients.

It is the unique design of PROMIS PF-CAT which allows for these dramatically reduced survey burdens, floor effects, and ceiling effects. It is constructed around the critical components of item response theory (IRT) and computer adaptive testing (CAT).³⁶ IRT uses item banks of validated questions, many drawn from legacy PROMs, to address specific domains such as physical function.^1,37 CAT then tailors each follow-up question based on the user’s previous answer.^36,37 This combination decreases the number of questions needed to obtain focused information as compared with legacy PROMs.^36,37 In the present study, survey burden for PROMIS PF-CAT was dramatically lower than the static survey burden of legacy PROMs. Despite recent studies combining machine learning with legacy PROMs to decrease survey burden, the PROMIS PF-CAT survey burden remains significantly lower. Additionally, it is notable that the “learning enhanced” legacy PROMs remain static instruments and require additional validation.³⁸

Floor effects and ceiling effects, respectively, have been defined as the lowest and highest 15% of scores possible on a survey instrument.^39,40 Young, active sports medicine patients may be especially prone to ceiling effects due to higher functional status that legacy PROMs may not capture.^9,11 The authors are not aware of floor or ceiling effects with PROMIS PF-CAT use, aside from ceiling effects of PROMIS Upper Extremity Version 1.0, which has subsequently been updated to version 2.0 with minimal ceiling effects. A recent systematic review and meta-regression analysis by Gulledge et al.⁴¹ noted 0 floor and ceiling effects of PROMIS PF-CAT across 43 recent studies (published through 2018) of shoulder and knee patients, not limited to arthroscopic procedures. Nwachukwu et al. demonstrated that preoperatively, the PROMIS PF-CAT in a study of 197 hips did not demonstrate any floor or ceiling effects, suggesting that the PROMIS PF instrument has broad applicability for hip arthroscopy patients.

No floor or ceiling effects were noted for legacy PROMs in the current study specifically for patients undergoing arthroscopic shoulder interventions, indicating that these effects may not have been detected or may not have been fully assessed by these studies. Notably, PROMIS PF-CAT was only administered at the preoperative timepoint for patients undergoing arthroscopic shoulder and hip interventions in the included studies of the present review. Potential postoperative ceiling effects would not have been captured as patients continued their postoperative rehabilitation protocols, strengthening, and return to function/sport, warranting further investigation at postoperative timepoints.

Limitations

Due to the heterogeneity in study protocols, not all endpoints were assessed in each individual analysis, introducing the possibility of analysis bias. Additionally, heterogeneity in the patient populations prevented pooled estimates. Most included studies assessed number of question for survey completion rather than completion time; we adjusted our outcome analyses and results reporting accordingly. In the present review, PROMIS PF-CAT data was available only at the preoperative timepoint for patients undergoing shoulder or hip arthroscopy, limiting the generalizability of these findings. As more literature on the subject becomes available, further quantitative analyses are warranted to assess pooled estimates of important PROM measures such as convergent vs. divergent validity, and floor or ceiling effects. The findings of the present review may require revisiting to expand the conclusions and broaden the applicability of the findings.

Conclusion

PROMIS PF-CAT is an accurate, efficient evaluation tool for sports medicine surgical patients. PROMIS PF-CAT strongly correlates with legacy physical function PROMs while having a lower test burden and less incidence of floor and ceiling effects. PROMIS PF-CAT may be an optimal alternative for traditional physical function PROMs in sports medicine patients undergoing arthroscopic procedures. Further studies are required to extend the generalizability of these findings to patients during postoperative timepoints after shoulder and hip interventions.