Performance of Pediatric PROMIS CATs in Children With Upper Extremity Fractures

Abstract

Background: This study was designed to quantify the performance of the pediatric Patient-Reported Outcome Measurement Information System (PROMIS) when delivered as part of routine care to children with upper extremity (UE) fractures. Methods: This cross-sectional study analyzed 964 new pediatric patients presenting with an UE fracture. All patients completed PROMIS computer adaptive tests for pain interference, peer relationships, UE function, and mobility domains at clinic registration. PROMIS was completed by parent-proxy (n = 418) for 5- to 7-year-olds and self-reported by 8- to 10-year-olds (n = 546). PROMIS score distributions were defined, and Pearson correlations assessed the interrelation between PROMIS domains. Student’s t tests compared mean PROMIS scores between parent-proxy and self-completion groups. Results: UE scores indicated the greatest average impairment of all PROMIS domains. However, 13% of patients reached the UE score ceiling indicating maximal UE function. UE scores and mobility scores had a strong positive correlation while UE scores had a moderate negative correlation with pain interference. In all patients, peer relationships were, at most, very weakly correlated with any other PROMIS domain. After grouping by fracture type, parent-proxy completion estimated worse UE function, more pain interference, and worse peer relationship. Conclusions: Pediatric PROMIS UE function scores capture impairment from UE fractures but do have a strong positive correlation with pediatric PROMIS Mobility, which assesses lower extremity function. Among children with UE fractures, parent-proxy completion of pediatric PROMIS appears associated with worse scores on most PROMIS domains.

Introduction

Patient-reported outcome measures (PROMs) have become increasingly emphasized in orthopedic surgery. They are used in clinical care, research, and are now being considered as a factor influencing reimbursement.⁵ In pediatrics, examples of commonly used, validated PROMs include the Pediatric Outcomes Data Collection Instrument (PODCI), Child Health Questionnaire (CHQ), DISABKIDS Chronic Generic Measure (DCGM), KINDL-R, and Pediatric Quality of Life Inventory (PedsQL). All of these PROMs contain physical function domains and no single consensus measure has emerged.¹³ Recently, the Patient-Reported Outcome Measurement Information System (PROMIS) was developed by the National Institutes of Health (NIH) as a means of standardizing patient-reported outcome scores and measuring health-related quality of life.⁴ PROMIS is not disease specific, which improves its applicability and positions it as a possible universal outcome measure.

PROMIS assessments are provided in a computer adaptive tests (CAT) format that leverages item response theory to improve survey performance. The pediatric PROMIS Physical Function assessment is comprised of individual mobility and upper extremity (UE) assessments that produce separate component scores.^4,9,14,20 In the pediatric population, a self-report version can be completed by the child (8-17 years old) or a parent-proxy version can be completed by a parent or caregiver (for children 5-7 years old).²⁴ PROMIS scores are normalized to a mean score of 50, standard deviation of 10, with a theoretical range of 0 to 100.³ A higher score corresponds to a greater amount of the domain being measured. For example, a higher physical function score corresponds to better physical function, whereas a higher depression score corresponds to greater depression.

Although the adult PROMIS assessments have been extensively studied, there is a paucity of literature regarding the performance of pediatric PROMIS assessments and even less information regarding pediatric PROMIS scores in the context of orthopedic surgery.^14,15 Currently, PROMIS has been incorporated into health care clinics for research purposes to further understand the utility of PROMIS scores.^17,18 Nonetheless, the performance of pediatric PROMIS assessments following UE injuries remains uncertain. There is a need for additional study to describe and analyze the current performance of PROMIS assessments in a pediatric orthopedic population. This could potentially serve to further refine PROMIS so that it can become a tool confidently utilized to inform clinical care.

This study was designed to assess the performance of pediatric PROMIS Physical Function (mobility, UE), pain interference, and peer relationship components in patients with UE fractures. The aims for this study were to determine the degree to which these 3 PROMIS domains are correlated and to identify any ceiling/floor effects in this population. We also aimed to test the null hypothesis that there would be no difference in score distributions between patient-reported and parent-proxy PROMIS domain scores.

Materials and Methods

This cross-sectional study analyzed data from 964 pediatric (5-10 years old) patient visits to the offices of a tertiary orthopedic center. New patients who presented with UE fractures between June 1, 2016, and June 1, 2017, were eligible for inclusion. Our Institutional Review Board deemed this study exempt from full review as only de-identified data were used in this study.

Our department administers PROMIS as part of routine clinical care at all office visits. All patients were provided a computer tablet (iPad mini, Apple, Cupertino, California) at check-in that automatically delivered the following PROMIS CATs: Physical Function-Upper Extremity v1.0, Physical Function-Mobility v1.0, PROMIS Pain Interference-v1.0, and PROMIS Peer Relationships-v1.0. Resulting PROMIS scores were automatically uploaded into the patient’s electronic health record upon assessment completion. Consistent with the recommended administration of PROMIS pediatric CATs, for patients aged 5 to 7 years, the parent/guardian was instructed to complete the PROMIS parent-proxy CATs. Patients aged 8 to 10 years were instructed to complete the PROMIS pediatric self-report CATs on their own.

Our institution’s electronic medical record and administrative databases were queried for age, sex, race, provider visited, International Classification of Disease, Tenth Revision (ICD-10) codes, and PROMIS domain scores (pain interference, UE function, peer relationships, and mobility) at the time of the initial visit. The PROMIS domains comprise specific groups of questions that inquire about a given health domain. For example, the pediatric UE function assessment includes statements such as, “I could open a jar by myself,” and “I could pull open heavy doors.” Each statement then allows a Likert style answer with options ranging from “with no trouble” to “not able to do so.” The type of primary fracture was defined by ICD-10 code. The following ICD-10 codes were used to define each fracture group: humeral shaft (S42.2, S42.3), distal humerus (S42.4), proximal forearm (S52.0, S52.1, S52.2, S52.3), distal forearm (S52.5, S52.6), unspecified forearm (S52.9), wrist/hand (S62.0, S62.1, S62.2, S62.3, S62.5, S62.6, S62.9).

Statistical Analysis

Patients were grouped according to PROMIS being completed by parent-proxy (n = 418) or self-report (n = 546). Univariate descriptive statistics characterized PROMIS scores and the frequency of patients with each UE fracture type. For each group, Pearson correlations assessed the degree of interrelation between PROMIS domains.

The percentage of patients reaching the ceiling and floor scores were calculated as the percentage of patients reaching the highest (ceiling) and lowest (floor) scores within each PROMIS domain. These percentages defined the floor and ceiling effects for each domain. Correlation coefficients (r) were interpreted as recommended by Evans: 0.00-0.19 very weak, 0.20-0.39 weak, 0.40-0.59 moderate, 0.60-0.79 strong, 0.80-1.00 very strong.¹²

To assess the impact of parent-proxy completion versus self-report on absolute PROMIS scores, student’s t tests compared mean PROMIS scores between the groups stratified by fracture type. The unspecified forearm and humeral shaft fracture groups were excluded from the subgroup analysis as they contained too few patients to draw reasonable conclusions.

Results

After applying the inclusion criteria, 964 patients contributed data that were analyzed (Table 1). For the self-report surveys, UE function scores indicated the greatest impairment of all PROMIS domains (mean: 33, SD: ±11) (Table 2). However, at presentation, 8.3% reached the ceiling UE score indicating maximal UE function. There were also ceiling effects in mobility (15%) and peer relationship (17%) scores and a floor effect in pain interference (7%). UE scores showed a strong positive correlation with mobility (r = 0.62) (Figure 1), and a moderate negative correlation with pain interference (r = −0.58) (Table 3). Pain interference and mobility scores also had a moderate negative correlation (r = −0.43). Peer relationships, at most, showed a very weak positive correlation with any other PROMIS domain (r < 0.15).

Table 1.

Cohort Demographics for Total Study Population.

	N (%)
	Age: 5-7 years	Age: 8-10 years
Sex
Female	181 (43.3)	265 (48.5)
Male	237 (56.7)	281 (51.5)
Race
White/Caucasian	329 (78.7)	441 (80.8)
Black/African American	74 (17.7)	84 (15.4)
Other	15 (3.6)	21 (3.9)
Upper extremity fracture type
Humeral shaft	17 (4.1)	21 (3.9)
Distal humerus	62 (14.8)	37 (6.8)
Proximal forearm	107 (25.6)	92 (16.9)
Unspecified forearm	17 (4.1)	18 (3.3)
Distal forearm	151 (36.1)	221 (40.5)
Wrist and hand	64 (15.3)	156 (28.6)
PROMIS surveys completed
Upper extremity	407 (97.4)	529 (96.9)
Mobility	418 (100)	534 (97.8)
Peer relationships	409 (97.8)	527 (96.5)
Pain interference	411 (98.3)	532 (97.4)

Note. PROMIS = Patient-Reported Outcome Measurement Information System.

Table 2.

PROMIS Scores According to Patient Group.

PROMIS domain	Mean (SD)		Range		Floor effect		Ceiling effect
	Parent-proxy	Self-administered	Parent-proxy	Self-administered	Parent-proxy	Self-administered	Parent-proxy	Self-administered
Upper extremity	30 (10)	33 (11)	14-56	14-57	5.20%	1.32%	4.90%	8.30%
Mobility	45 (9)	44.9 (9)	22-60	23-62	0.20%	0.01%	14.60%	14.8%
Peer relationship	50 (10)	52.5 (10)	15-66	17-66	1.20%	0.57%	1.70%	16.70%
Pain interference	54 (8)	48.7 (8)	22-78	32-74	0.20%	6.80%	0.50%	0.38%

Note. PROMIS = Patient-Reported Outcome Measurement Information System.

Figure 1.

Correlation of mobility and upper extremity scores in 8- to 10-year-old total study population.

Table 3.

Correlations Between PROMIS Scores in 8- to 10-Year-Olds (r Values) in Total Study Population.

	Mobility	Upper extremity	Pain interference	Peer relationships
Mobility	1
Upper extremity	0.619	1
Pain interference	−0.428	−0.582	1
Peer relationships	0.146	−0.009	−0.094	1

Note. PROMIS = Patient-Reported Outcome Measurement Information System.

For the parent-proxy survey group, UE function scores indicated the greatest impairment of all PROMIS domains (mean: 30, SD: ±10) (Table 2). At presentation, 4.9% reached the ceiling UE score indicating maximal UE function and 5.2% reached the floor UE score indicating minimal UE function. UE scores showed a strong positive correlation with mobility (r = 0.64) (Figure 2), and a moderate negative correlation with pain interference (r = −0.48) (Table 4). Pain interference also had a moderate negative correlation with mobility (r = −0.41). Peer relationships, at most, showed a very weak positive correlation with any other PROMIS domain (r < 0.12).

Figure 2.

Correlation of mobility and upper extremity scores in 5- to 7-year-old total study population.

Table 4.

Correlations Between PROMIS Scores in 5- to 7-Year-Olds (r Values) in Total Study Population.

	Mobility	Upper extremity	Pain interference	Peer relationships
Mobility	1	0.608	0.608	0.608
Upper extremity	0.639	1	0.608	0.608
Pain interference	−0.409	−0.480	1	0.608
Peer relationships	0.118	−0.097	0.015	1

Note. PROMIS = Patient-Reported Outcome Measurement Information System.

Comparing the two groups defined by administration mode, parent-proxy completion estimated worse UE function (−3.7 points, P < .01), more pain interference (5.1 points, P < .01), and worse peer relationships (−2.5 points, P < .01) but similar mobility scores compared with self-report completion (Table 5).

Table 5.

Test for Significance Between 5 to 7 and 8 to 10 PROMIS Scores in the Study Population.^a

PROMIS domain	Distal humerus		Proximal forearm		Distal forearm		Wrist and hand		Overall
	P value	Mean difference (95% CI)	P value	Mean difference (95% CI)	P value	Mean difference (95% CI)	P value	Mean difference (95% CI)	P value	Mean difference (95% CI)
Mobility	0.239	2.39 (−1.62 to 6.40)	0.292	−1.34 (−3.85 to 1.16)	0.884	−0.14 (−2.05 to 1.77)	0.508	−0.81 (−3.24 to 1.61)	0.75	−0.19 (−1.33 to 0.96)
Pain interference	<0.001	−6.65 (−10.21 to −3.08)	0.009	−3.26 (−5.70 to −0.82)	<0.001	−5.58 (−7.38 to −3.79)	<0.001	−6.38 (−8.50 to −4.25)	<0.001	−5.12 (−6.19 to −4.05)
Peer relationships	0.368	2.21 (−2.25 to 6.01)	0.045	2.90 (0.06 to 5.74)	0.043	2.22 (0.07 to 4.38)	0.121	2.45 (−0.65 to 5.55)	<0.001	2.49 (1.16 to 3.82)
Upper extremity	0.001	7.54 (3.38 to 11.71)	0.055	2.73 (−0.6 to 5.53)	0.005	3.31 (1.00 to 5.62)	0.125	2.28 (−0.64 to 5.20)	<0.001	3.74 (2.38 to 5.11)

Note. PROMIS = Patient-Reported Outcome Measurement Information System; CI = confidence interval.

^aA positive mean difference indicates a higher domain score in the 5- to 7-year-old study population.

Discussion

PROMs are increasingly utilized in the adult orthopedic population as important clinical information.^10,22 The NIH developed PROMIS CATs as a standardized measurement of patient-reported outcomes across diseases and specialties.^7,8 In orthopedics, PROMIS CATs have been compared to validated legacy PROM instruments and performed favorably across many adult orthopedic conditions.^2,23 However, in the pediatric orthopedic population, PROMs such as the PODCI are routinely used and PROMIS CATs are slowly being incorporated alongside them.^18,25

Few studies have evaluated pediatric PROMIS assessments in the orthopedic population. Wall et al reported on the long-term outcomes of Huber Opposition Transfer for augmenting hypoplastic thumb.²⁵ In that study, PODCI and PROMIS scores were collected. Wall et al utilized PROMIS CATs to evaluate a specific pediatric orthopedic population similar to how our group analyzed pediatric orthopedic patients presenting with an UE fracture. In this small series (8 self-report PROMIS, 7 parent-proxy PROMIS administrations), their data indicate a similar finding to our study in that parent-proxy administration resulted in poorer perceived physical function but similar pain levels. Our data substantiate this finding and complements Wall’s early data in 15 cases with a considerably larger population of over 950 patients. The current project expands our understanding of pediatric PROMIS assessments by analyzing the degree of correlation between PROMIS assessments in this population.

In adults, the PROMIS physical function item bank is a unidimensional construct, assessing overall physical function. The pediatric PROMIS Physical Function item bank is split into two separate assessments, mobility and UE function. Mobility captures lower extremity function while UE represents UE function. During preliminary pediatric PROMIS testing, these two item banks were kept separate because they were not highly correlated.¹¹ Contrary to this presumed independence, our data indicate a strong correlation between the UE and mobility scores in both parent-proxy and self-report surveys. We cannot definitively explain this concordance. This correlation may be due to an overlap between movements described in the questions of both item banks such that neither is as extremity specific as intended. For example, within the mobility assessment questions include, “I could keep up when I played with other kids,” “I could bend over to pick something up,” and “I could get up from the floor by myself.” These tasks, although intended to measure lower extremity function, likely capture more global function impacted by UE injury. Alternatively, the strong correlation may be secondary to studying a population of children with UE fractures. A painful, immobilized UE may limit the patient’s overall activity level despite any lower extremity injury.

The peer relationships score is used as a proxy for depressive symptoms and isolation in our orthopedic practice. Interestingly, we found that peer relationships are not related to UE, mobility, or pain interference scores after an UE fracture. This is in contrast to the adult population, where depression and social deprivation are correlated with UE physical function.^6,16,19 This lack of correlation in the pediatric population may be due to the acute trauma that was studied. The acuteness of these patient’s injury may not make the children feel “different” for a long period of time and subsequently not cause them to associate the injury with their identity. In this case, the child has a temporary condition that will resolve and is unlikely to change their interactions with their peers. For this reason, we were not surprised by the presence of nearly 1 in 5 patients scoring at the ceiling of the peer relationship survey indicating no issues with social interactions.

The parent-proxy item banks should be completed by the parents or caregivers of children ages 5 to 7 years.²⁴ In our study, parent-proxy surveys indicated more pain interference, less UE function, and worse peer relationships for their child than the self-administrated group. These differences were both statistically significant and clinically relevant as they approached and surpassed the suggested Minimal Clinically Important Difference (MCID) (3-5 points) values for these assessments.^1,21 Parent-proxy and self-administered surveys produced similar mobility scores. Prior research found that parental responses and child self-reported PROMIS answers were more congruent on objective scales such as physical function as compared to scales of internal experiences (eg, energy, pain interference).²³ In contrast to the work by Varni et al, which queried children and parents from pediatric clinics, we have found greater average differences in scores between those surveys completed by parents and those by the children themselves. That prior study also found greater agreement between children and adult answers when the children had chronic pain, which was postulated to have allowed the adults to better understand the children’s feelings over time. In our patient population with acute fractures, it is possible that children suffering an acute trauma creates a novel experience for the child and the minimal time for the parent to observe the impact of the trauma could contribute to the increased discrepancy in scores that we have identified. As the mode of administration in our clinical practice is dictated by patient age, it is possible that the differences in scoring could be influenced by the age of the patient groups. We have attempted to minimize this possibility by analyzing a narrow age window of 5- to 7-year-olds against 8- to 10-year-olds. Also, given the consistency of differences found when examining each fracture location, we do believe that the differential scoring represents a systematic bias associated with the mode of completion.

The pediatric PROMIS UE function score appropriately appeared to be the PROMIS scale most affected by an UE fracture. However, there appeared to be a substantial ceiling effect. This ceiling effect has been documented to affect 7% to 11% of adult patients with UE conditions.² This ceiling effect is a relevant concern as those indicating maximal function at initial presentation will be unable to demonstrate any value of treatment delivered. We are not aware of a consensus tolerance for ceiling effects among orthopedic outcome measures but are concerned when this affects nearly 1 in 10 patients. In adults, the ceiling effect has been linked to a lack of highly demanding UE questions. This is likely compounded when delivering similar questions to children who are remarkably adaptable and often find ways to continue the majority of activities despite an UE cast.

Our study has several limitations. Inherent to our cross-sectional design, we are unable to determine how the scores of these patient-reported health assessments collected at presentation impact treatment outcomes. Second, patients were grouped by the anatomic location of fracture sites without further delineation of fracture severity. To minimize potential bias from differential percentages of fracture types impacting on the overall mean PROMIS score difference between groups, we analyzed each subgroup to assure similar PROMIS score patterns across all fracture groups. Finally, all patients and parents were specifically instructed on who should complete the PROMIS survey when handed the mini-tablet computer but we did not monitor these patients and parents as they completed the PROMIS surveys. Therefore, there was still the potential for parents or other siblings to have completed the PROMIS CATs on behalf of patients aged 8 to 10 years or for the patient aged 5 to 7 years to fill out the PROMIS CATs on behalf of the parent.

This study presents an initial experience with pediatric PROMIS in patients with UE fractures. As anticipated, UE function scores were the most impacted by the fractures but a ceiling effect was present. Our data also indicate that the PROMIS UE and mobility components of physical function are strongly correlated and this may warrant potential survey refinement to increase their independence. Finally, among children with UE fractures, parent-proxy completion of pediatric PROMIS magnifies perceived physical impairment, peer relationships, and estimated pain.