Variability of PROMIS Scores Across Hand Conditions

Abstract

Purpose:

This study aimed to determine whether PROMIS Physical Function and Pain Interference scores varied at presentation for specialty care by non-trauma hand condition. The secondary aim was to compare PROMIS scores to a reference standard, the QuickDASH, regarding the magnitude and direction of score differentials between diagnoses.

Methods:

PROMIS Physical Function and Pain Interference scores were analyzed from 1471 consecutive new adult patient clinic visits at a tertiary orthopaedic hand clinic presenting with one of 5 non-trauma hand conditions. A 5-point difference on PROMIS assessments was presumed to be clinically relevant. A random sample of 30 QuickDASH scores from each diagnostic group was evaluated for score differentials between groups. We also measured the correlation between PROMIS and QuickDASH scores.

Results:

Patients with carpal tunnel syndrome and thumb basal joint arthritis reported worse Physical Function and more Pain Interference, while those with Dupuytren contractures and ganglion cysts reported less pain and better function. For both domains, patients with trigger fingers averaged PROMIS scores between the other groups. Similar differences were observed in QuickDASH scores as patients with carpal tunnel syndrome and thumb arthritis reported clinically worse upper extremity function than patients with ganglion cysts and Dupuytren contracture. A strong correlation was seen between QuickDASH scores with both PROMIS Physical Function scores and Pain Interference scores.

Conclusions:

PROMIS is sufficiently able to capture differences in self-reported function and pain interference between patients with different hand conditions. PROMIS Physical Function demonstrates construct validity when evaluated against a reference of the QuickDASH across non-trauma hand conditions.

INTRODUCTION

With increasing emphasis on patient reported outcomes in healthcare, the National Institute of Health created a comprehensive health outcomes system that can be utilized across all fields of healthcare^1–3. This effort led to the development of the Patient-Reported Outcome Measurement Information System (PROMIS), a computerized adaptive testing (CAT) system that includes modules addressing physical, mental, and social health^4–7. It has been validated in multiple patient populations, with the ultimate goal of being applicable across all of healthcare^7–11.

In populations with upper extremity conditions, general measures of physical function tend to be less specific and responsive than anatomically-specific measures^12–14. For example, the Levine-Katz carpal tunnel questionnaire was more sensitive to change in symptoms and function in patients with carpal tunnel syndrome than more generic questionnaires or even physical function measurements¹². The patient-rated wrist evaluation (PRWE) and the Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire were both more responsive than the Short-Form 36 in patients with a distal radius fracture¹³. However, both PROMIS Physical Function and Pain Interference scores have demonstrated strong correlations with legacy instruments as well as comparable responsiveness to change when used in hand and upper extremity patient populations^15–20. At the same time, PROMIS modules also reduce respondent burden by requiring less time to complete and fewer questions than legacy measures^21–23.

While PROMIS Physical Function has demonstrated utility when researching populations with upper extremity conditions, its ideal use involves clinic-wide administration to all patients with varying conditions. If delivered universally as part of routine care, the scores can quantify changes in patients’ perceived function for treating physicians and scores collected at the point of care can be analyzed later to investigate treatment outcomes while avoiding potential recall bias. In this type of heterogeneous population, there are potential differences in pain interference and functional limitations between diagnoses. At this time, it is unclear if PROMIS scores are sufficiently sensitive to capture these differences. Therefore, this study aimed to test the construct validity of PROMIS Physical Function and Pain Interference scores by examining for score variation according to the symptomatic diagnosis and comparing scores to a reference standard of the Quick Disabilities of the Arm, Shoulder, and Hand (QuickDASH).

METHODS

This cross-sectional evaluation analyzed 1491 consecutive new patient outpatient clinic visits of adult patients presenting to a tertiary orthopaedic hand clinic between 7/1/2015 and 11/30/2016 with one of five non-trauma hand conditions. The study was approved by our Institutional Review Board with a waiver of written consent for the use of data collected during routine clinical care. Visits coded with ICD-10 codes for one of the following conditions were included: carpal tunnel syndrome, Dupuytren contracture, trigger finger, thumb carpometacarpal arthritis, ganglion cyst (Appendix 1). Ganglion cysts predominately originated from the wrist (carpus n=148, interphalangeal joint n=46, flexor tendon sheath=21).

All patients were given a tablet computer (iPad mini, Apple, Cupertino, CA) preloaded with the PROMIS Physical Function (v1.2) and Pain Interference (v1.1) CATs (Computer Adaptive Tests) at check-in. Upon completion, PROMIS scores automatically loaded into the patients’ electronic health record. Visits with at least one valid PROMIS score were included in analysis (n=1471, 98.7%) (Figure 1).

Figure 1.

Flowchart of inclusion criteria.

PROMIS assessments were developed by using item response theory (IRT) to narrow a candidate item bank that was tested in the general population²⁴. Questions are then pulled from this item bank in a computerized adaptive testing (CAT) format so that each patient will answer 4–12 questions. All PROMIS modules are normalized to a population mean score of 50 with a standard deviation of 10 points where a higher score represents more of that construct (i.e. a higher Physical Function score represents more, or better, functionality, while a higher Pain Interference score represents more, or worse, pain interference)²⁴. A minimal clinically important difference (MCID) for change in clinical status using PROMIS scores of nearly 5 points has been suggested in other patient populations but not specifically in the conditions studied here ^7,25,26. We approached this study considering this 5-point change (Effect Size 0.5) as a reasonable proxy for a clinically relevant difference between groups.

All new patients also completed the paper version of the QuickDASH as a part of their registration packet. The QuickDASH was developed from the Disabilities of the Arm, Shoulder, and Hand Questionnaire²⁷. The QuickDASH is a validated outcome measure which consists of 11 questions, each of which has five Likert-scale responses where 1 indicates the least and 5 the worst function or pain. A difference of 14 points has been determined to be clinically relevant²⁸.

One-way ANOVA with Tukey post-hoc analysis was utilized to assess for age differences between diagnostic groups, while a chi-square analysis with Bonferroni correction assessed differences in sex and race between diagnostic groups.

Identical ANOVA testing compared the effect of the independent variable diagnostic group on PROMIS Physical Function and Pain Interference scores.

To account for patient demographic factors, a generalized linear model (ANCOVA) assessed for differences in marginal means of PROMIS scores between diagnostic groups. Modeling accounted for race and sex (categorical factors) as well as age (continuous covariate). This modeling was performed for both PROMIS Physical Function and Pain-Interference.

As QuickDASH scores were only collected from a subgroup of the study participants, we confirmed the necessary sample size a priori for a one-way ANOVA to determine how many Quick-DASH scores would be required to obtain a power of 0.9 and a two-sided alpha of 0.05 to look for an effect size of 0.4 in the setting of five groups and an assumed standard deviation of 24^18,28. This resulted in a total of 100 patients. To protect against increased data variance, scores from 150 patients (30 per group) were analyzed. Systematic random sampling was performed within each diagnostic group to select 30 patients per group. Patients were randomly selected and then charts were manually reviewed for scorable QuickDASH questionnaires. One researcher collected QuickDASH data with a second researcher verifying a random subset of 60 patients to ensure data accuracy.

Nonparametric Kruskal-Wallis Chi-square analyses with post-hoc comparisons using the Mann-Whitney U tested for differences in QuickDASH scores between diagnostic groups as descriptive and graphical examination of the data showed a non-normal distribution. Spearman’s correlation tested the association between QuickDASH scores and PROMIS Physical Function and Pain Interference scores. Correlation coefficients were interpreted as follows: 0.00–0.19 very weak, 0.20–0.39 weak, 0.40–0.59 moderate, 0.60–0.79 strong, 0.80–1.00 very strong²⁹.

RESULTS

Data from 1471 patient visits were included for analysis (Table 1). Patients with carpal tunnel syndrome and ganglion cysts were younger on average than other patients. Dupuytren patients and those with thumb base arthritis were more likely to be Caucasian than other patients, and Dupuytren patients were predominantly male.

Table 1.

Demographic information of study population.

	Overall n=1471	CTS n=465	Thumb CMC OA n=258	Trigger Finger n=405	Dupuytren’s n=128	Ganglion Cyst n=215
Mean Age (SD)	57.2 (14.3)	55.6*(14.7)	60.8 (9.3)	60.8 (12.6)	61.9 (11.6)	46.9*(17.0)
Females (%)	63.0%	65.6%	66.3%	63.7%	33.4%	69.3%
Caucasian (%)	85.2%	79.5%	96.0%*	79.7%	100.0%	85.8%

PROMIS Physical Function and Pain Interference scores varied significantly (p<0.05 and clinically relevant >5 points) between diagnostic groups (Table 2). Physical Function and Pain Interference scores were worst among patients with carpal tunnel syndrome and thumb arthritis, while the least pain and best functionality were reported by those with ganglion cysts or Dupuytren contracture.

Table 2.

PROMIS scores according to diagnosis^†

	Physical Function (SD)	Pain Interference (SD)
CTS (n=465)	43.1 a (9.3)	61.2 a (7.5)
Thumb CMC OA (n=258)	45.9 b (8.1)	60.0 a,b (6.0)
Trigger Finger (n=405)	47.0 b (9.9)	58.4 b (7.0)
Ganglion Cyst (n=215)	50.3 c (8.5)	54.6 c (8.2)
Dupuytren’s (n=128)	51.1 c (10.0)	52.2 d (9.8)

^†Homogenous subsets (p>0.05) represented by alphabetic subscripts

After accounting for demographic characteristics (i.e., age, sex, race), differences in PROMIS scores persisted (Figure 2, Appendices 2–3). Patients with carpal tunnel syndrome and thumb arthritis reported worse physical function than all other groups, while those with Dupuytren contracture and ganglion cysts reported the least pain. In this analysis, race and age each remained significant variables that were associated with mean PROMIS Physical Function and Pain Interference scores. Advancing age and African-American race were associated with worse reported function and greater reported pain.

Figure 2.

Average QuickDASH and PROMIS Physical Function scores with 95% confidence intervals by diagnosis corrected for race, sex, age.

Similar differences persisted in QuickDASH scores as patients with carpal tunnel syndrome and thumb arthritis reported worse upper extremity function than patients with ganglion cysts and Dupuytren contracture (Figure 2).

Overall, QuickDASH scores were strongly correlated with both PROMIS Physical Function scores (r= −0.66) and Pain Interference scores (r= 0.79).

DISCUSSION

While PROMIS Physical Function and Pain Interference modules are not disease-specific, our data support their construct validity. Scores on these assessments varied with the condition treated in a direction and magnitude similar to the QuickDASH. . Patients with carpal tunnel syndrome reported the worst physical function, and those with thumb arthritis or carpal tunnel reported significantly worse pain interference than patients with Dupuytren contracture or ganglion cysts, even when accounting for differences in age, race, and sex.

The relatively good function experienced by patients with Dupuytren contracture as compared to those with thumb base arthritis, carpal tunnel syndrome, or trigger finger is consistent with prior studies³⁰. Among 262 patients with non-trauma hand conditions, patients with hand arthritis reported worse function on the QuickDASH than patients with carpal tunnel syndrome, trigger finger, and Dupuytren contracture, with the Dupuytren population reporting significantly lower scores than all other groups. This contrasts with Sorensen et al’s baseline QuickDASH scores from a group of 102 patients with hand osteoarthritis, nerve compression syndromes, or tendonitis. No statistically or clinically relevant differences in scores were found, however specific diagnoses were not elucidated²⁸.

Using PROMIS to assess patient outcomes offers the advantages of minimizing patient burden, optimizing testing through computer adaptive testing, and delivering immediate scoring in the electronic health record. The literature has previously described correlations between PROMIS and QuickDASH scores. Overbeek et al found a correlation of r= −0.55 between the QuickDASH and PROMIS Physical Function and r=0.74 between QuickDASH and PROMIS Pain Interference when analyzing scores from 93 consecutive patients presenting to an upper extremity clinic¹⁸. Other studies have found correlations with the DASH and PROMIS Physical Function ranging from −0.68 to −0.82 in patients with proximal humerus or rotator cuff pathologies^21,22. The strength of correlation between PROMIS and QuickDASH scores across studies has consistently indicated a moderate to strong correlation with minor variation based on the diagnosis studied. Adding further evidence of construct validity, our analysis determined that PROMIS Physical Function/Pain Interference and the QuickDASH all produced similar relative rankings of diagnoses according to their health impact.

An upper extremity specific measure of physical function, the Upper Extremity – Physical Function CAT, has been developed for PROMIS. Scores from the Upper Extremity CAT have been shown to correlate as well as those from the PROMIS Physical Function CAT with the QuickDASH and DASH^16,17. However, the Upper Extremity CAT has a ceiling effect and currently cannot discriminate between higher levels of function, making it difficult to solely utilize the Upper Extremity CAT as a measure of function^{16,17,31–33}. Notably, the PROMIS Upper Extremity CAT continues to be modified as new versions are released. However, these limitations have persisted, albeit to lesser degrees, with the latest Upper Extremity CAT. We acknowledge that the PROMIS Upper Extremity CAT may eventually supplant the PROMIS Physical Function CAT in hand practices, but until that time our data indicate that the Physical Function CAT can discriminate among hand conditions.

Accounting for diagnosis and sex, patient age and race were significant predictors of PROMIS Physical Function and Pain Interference scores. However, due to the lack of details about symptom severity and acuity, it is impossible to determine whether these differences are due to worse perception of similar symptoms, or whether individuals in this population presented with more severe disease processes. Similarly, the unavailability of disease history as well as the patients’ socioeconomic status makes interpreting differences in average PROMIS scores by racial groups difficult. Although this study was not designed to explain why PROMIS score differences may occur between patients according to demographic data, it was necessary to control for these variables during analysis to account for predictable demographic differences between diagnostic groups (e.g. the predominance of Caucasian males within the Dupuytren population).

Our study has several limitations. The cross-sectional design does not allow longitudinal data collection, which is necessary to determine if the PROMIS measures demonstrate responsiveness to treatment when compared to the QuickDASH. Second, the large number of patients studied allows appreciation of general patterns, but does not account for variability in the acuity, severity, or comorbid diseases in these patients. Finally, we only compared PROMIS scores to the QuickDASH. It is possible that the choice of an alternative reference standard (e.g., Michigan Hand Questionnaire) could have affected our results. However, we have no reason to suspect that the overall patterns of differences would change.

In conclusion, we have found that PROMIS captures differences in self-reported function and pain interference between patients with different hand conditions. Additionally, these differences correlate with those captured by the QuickDASH, a well-validated and extensively cited measure of upper extremity impairment. The consistency of our findings when using PROMIS scores and the QuickDASH suggests that these differences are more likely true differences attributable to specific diagnoses as opposed to a finding specific to a single patient-reported outcome measure. Our data suggest that PROMIS Physical Function is well suited for widespread delivery in the upper extremity practice.

Clinical Relevance Statement:

The use of PROMIS is expanding but as PROMIS is not disease specific, assessment of its construct validity is necessary for hand conditions.

Appendix 1. International Classification of Diseases Codes.

Carpal Tunnel	G56.00	G56.11
Syndrome	G56.01	G56.12
	G56.02
Thumb CMC Arthritis	M18.0	M18.31
	M18.11	M18.32
	M18.12	M18.9
Trigger Finger	M65.30	M65.332
	M65.311	M65.339
	M65.312	M65.341
	M65.319	M65.342
	M65.321	M65.349
	M65.322	M65.351
	M65.329	M65.352
	M65.331	M65.359
Ganglion Cyst	M67.40	M67.441
	M67.431	M67.442
	M67.432	M67.449
	M67.439
Dupuytren’s Contracture	M72.0

Appendix 2. Between-subject effect of independent variables on PROMIS Physical Function in ANCOVA analysis.

Variable	p-value
Diagnosis Group	<0.05
Age	<0.05
Sex	0.14
Race	<0.05

Appendix 3. Between-subject effect of independent variables on PROMIS Pain Interference in ANCOVA analysis.

Variable	p-value
Diagnosis Group	<0.05
Age	<0.05
Sex	0.13
Race	<0.05