Applying PROMIS to Assess Upper Extremity Function Among Children with Congenital Hand Differences

Abstract

Purpose

Few studies have evaluated self-assessment tools among children with congenital hand differences. We compared three upper extremity disability instruments with the Patient Reported Outcomes Measurement Information System (PROMIS®) Pediatric Upper Extremity Item Bank.

Methods

Thirty-three children (ages 6–17) with congenital hand differences completed the Pediatric Outcomes Data Collection Instrument (PODCI), the Michigan Hand Outcomes Questionnaire (MHQ), the Disability of the Arm, Shoulder, and Hand (DASH), and the PROMIS® Upper Extremity short form (SF) and computerized adaptive test (CAT). Hand function was also assessed using grip and pinch strength and the Nine-Hole Peg Test. We used Spearman correlation coefficients to determine construct validity, and examined feasibility by comparing completion time, reading level, need for assistance, and patient preference among the instruments.

Results

PROMIS® demonstrated good construct validity. SF and CAT versions of PROMIS® were highly correlated with DASH scores (r > 0.80, p<0.001) and all PODCI domains except sports (r>0.70, p<0.001). Correlations with the MHQ were moderate (r> 0.40, p < 0.05). PROMIS® SF and CAT scores also correlated with grip strength (r≥0.60, p<0.001) and pinch strength (r>0.50, p<0.001). Compared to the other questionnaires, PROMIS® was much more feasible. It took the least time to complete, fewer children required assistance, and it is written at a lower, more age-appropriate reading level than the MHQ and DASH.

Conclusions

PROMIS® is highly correlated with both functional tests and traditional hand function questionnaires. Our results provide encouraging evidence that PROMIS® may be an efficient, feasible option to capture hand function among children with congenital hand differences.

Level of Evidence: III

Congenital hand differences represent approximately 10% of congenital anomalies, and affect 0.2% of all live births.^1–8 To date, there is no commonly accepted measure of self-reported hand function among these children. Although some aspects of hand function can easily be obtained by functional assessment, such as grip or pinch strength, these measures do not always reflect disability. For example, muscle strength and joint motion do not account for the effect of arm disability on activities of daily living, and the ability of children to adapt over time.⁹ Among adults, many instruments exist that capture self-reported upper extremity function with excellent reliability, sensitivity, and accuracy.^10,11 However, similar tools for pediatric upper limb conditions are much less common, and the influence of surgical reconstruction on self-reported upper extremity function is not well understood.

In 2004, the National Institutes of Health (NIH) initiated a collaborative effort to measure aspects of health-related quality of life (HRQOL) using common data elements, or questions that are applicable across a variety of conditions. The Patient Reported Outcomes Measurement Information System (PROMIS®) can capture many aspects of HRQOL, such as pain, physical function, and with less responder burden and greater discriminative ability compared with traditional instruments.¹² Among children, PROMIS® item banks have been developed to capture self-reported anger, depression, peer relationships, and physical function. More recently, PROMIS® item banks regarding upper extremity function have been constructed, and validated among children with cancer-related diagnoses and inflammatory arthropathies.^13,14 The PROMIS® Upper Extremity Item Bank contains 29 items and has previously undergone extensive testing in large diverse groups of children in order to establish unidimensionality. However, it has not yet been applied to children with severe upper extremity disability, including congenital hand differences.^15–18

Identifying efficient and accurate measures of self-reported upper extremity function among children with congenital hand differences could provide an opportunity to rigorously compare outcomes by surgical treatment and longitudinally assess upper extremity adaptation and disability. Thus, the purpose of this study is to examine the construct validity and feasibility of the PROMIS® Pediatric Upper Extremity Item Bank among children with congenital hand differences in comparison with functional assessment measures and traditional extremity disability assessment tools. We hypothesize that the PROMIS® Upper Extremity Item Bank will provide a more accurate assessment of self-reported hand function among children with congenital hand anomalies compared with existing measures that can be easily administered in a clinical setting.

Methods

Study Population and Data Collection

We identified 40 children with congenital hand differences who were cared for at a tertiary care institution. Children were eligible if they were diagnosed with a congenital hand difference, were between ages 6 and 17, were able to read and speak English, and had no previous history of cognitive impairment. Children were excluded if they were unable to complete data collection methods (n=7). Participants completed four measures of self-reported upper extremity function and underwent functional assessment of upper limb strength and dexterity. The Institutional Review Board approved all aspects of this study; parents provided informed consent and children assent prior to enrolling in this study.

Measures: Self-reported Outcomes

PROMIS® Pediatric Upper Extremity Item Bank

We administered the pediatric PROMIS® Upper Extremity Item Bank to all participants as both a paper and pencil short form (Pediatric Physical Function – Upper Extremity Short Form 8a, Appendix 1) as well as a web-based computer adaptive test (CAT).^6,19,20 Parents and children completed instruments separately. Items specified a 7-day recall period, and included 5-point answer choices, with higher scores indicating better functioning. For example, the item “In the past 7 days, I could put toothpaste on my toothbrush by myself,” was followed by five answer choices: With no trouble, With a little trouble, With some trouble, With a lot of trouble, or Not able to do. We tabulated raw scores for the short form, which we then translated to normalized T-scores (mean=50, standard deviation=10) using the conversion charts available through the Assessment Center website.²¹ Children completed the PROMIS® CAT on touchpad devices using PROMIS® Assessment Center software.²² The upper extremity and peer relationship scales were combined into a single test that included a minimum of 5 and maximum of 12 items per domain. The Assessment Center software collected all CAT data and provided normalized T-scores based on item response theory (mean=50, SD=10; higher scores indicate better function).

The Michigan Hand Outcomes Questionnaire

The Michigan Hand Outcomes Questionnaire (MHQ) is a 37-item instrument that measures upper-extremity disability along six domains: function, activities of daily living, pain, work ability, appearance, and satisfaction.^23,24 It has been widely used for both acute and chronic upper extremity conditions.^25–29 In this study, we examined only 5 domain scores, and omitted work ability, as the majority of children did not have employment outside of school. Items specify a 7-day recall period, with 5-point response choices. Domains are scored as the sum of scores across each item in the domain, and the normalized for a range of 0 (poorest function) to 100 (best possible function).

The Disabilities of the Arm, Shoulder and Hand

The Disabilities of the Arm, Shoulder and Hand questionnaire (DASH), is a 30-item instrument that measures disability and symptomatology from upper extremity conditions, with additional modules available for work, sports, and performing arts.^30,31 Like PROMIS® and the MHQ, the DASH also uses 7-day recall periods with 5-point response choices. Score responses are summed, and then normalized for a range of 0 (best possible function) to 100 (poorest function).

Pediatric Outcomes Data Collection Instrument

Children completed the Pediatric Outcomes Data Collection Instrument (PODCI), which collects self-reported disability and function, with a specific module assessing upper extremity function using 8 items. Response scores are summed, and normalized to a score of 0 (worst possible function) to 100 (best possible function). The PODCI has been previously validated for use among children with cerebral palsy, and has been shown to be reliable among children with congenital and traumatic amputations.³² The PODCI is only validated in children ages 10 and above. Therefore, participants 9 years of age and younger did not complete the PODCI.

Measures: Functional Assessment

In addition to legacy measures, patients completed three functional assessments for each hand: grip strength, key pinch strength, and dexterity. Grip and key pinch gauges were used (Jamar® Hydraulic Hand Dynamometer) to measure strength in kilograms. Dexterity was assessed using the Nine-Hole Peg Test, a timed examination of how quickly patients can individually place and remove pegs in a standardized construct, and was repeated four times to generate an average score.³³

Data analysis

Construct validity

Descriptive statistics were calculated for all patients to describe the distribution of age, gender, ethnicity, bilateral involvement, type of congenital hand difference, and associated conditions. Construct validity describes the ability of an instrument to predict an outcome measured by an alternate instrument. To measure the construct validity of the PROMIS® Pediatric Upper Extremity item bank, we administered three “legacy” self-report instruments of upper extremity function to children with congenital hand differences: the MHQ, DASH, and PODCI. Additionally, children completed three functional assessments of upper extremity disability: grip strength, pinch strength, and dexterity. To assess construct validity, we calculated Spearman’s correlation coefficients for total and domain scores of all surveys and functional tests. Because some instruments are scored individually for each hand, we performed correlation analyses using data for participants affected hands. To quantitatively assess content validity, we examined the presence of floor and ceiling effects for all instruments, which are quantitative indicators of limited content validity. Floor and ceiling effects were defined as greater than 15% of participants achieving the worst and best possible scores, respectively.³⁴ For the PROMIS® CAT Upper Extremity scale we defined the “best” and “worst” scores as those falling more than two standard deviations above or below the normal value.

Feasibility

Feasibility refers to responder and administrative burden of a particular instrument.³⁵ In this study, we specifically examined the following feasibility parameters: instrument completion time, instrument reading level, need for assistance associated with each patient-reported instrument, and patient preference regarding the instruments. Assessment Center software captures completion times of the PROMIS® Pediatric Upper Extremity instrument administered as a CAT, including overall and individual item times. Study coordinators also timed children as they read and filled out paper-based surveys using a stopwatch. Timing was continuous throughout survey completion, including any questions that participants’ had, to obtain comparable times to the CAT system. Using Mann Whitney U tests, we compared the completion times between all surveys. We determined the reading level of each instrument by copying all text from instructions, items, and responses into Microsoft® Word 2010 and obtaining Readability Statistics. We report the Flesch-Kincaid Grade Level, which assigns a grade level in United States schools to the text. Participants’ need for assistance in reading surveys was recorded by the research assistants administering them.

Results

In our cohort, the average age was 11.4 years (Table 1). Thirty-four percent of children had bilateral hand involvement, and 58% had additional congenital conditions, most commonly lower limb anomalies (25%) and VACTERL association (13%). In general, children reported good upper extremity function (Table 2). Mean MHQ scores indicated high satisfaction (77.9), followed by function (72.7), ADLs (68.8), and appearance (67.1). The mean DASH score was 13.1. All PODCI scores were high, with the highest mean scores for mobility (98.8), upper extremity function (94.2), and global functioning (94.3). PROMIS® scores fell below the norm (50) for the CAT (42.3) and SF (42.8).

Table 1.

Patient Demographic Data

Characteristic	Sample n (%)
Age (years), (M, SD)	11.4±3.9
<8	5 (16%)
8–17	27 (85%)
Sex
Male	19 (59%)
Female	13 (41%)
Race
White	25 (78%)
Asian	4 (13%)
Other	3 (9%)
Hand Involvement
Bilateral	11 (34%)
Unilateral	22 (66%)
Primary Diagnosis
Duplicated thumb	8 (25%)
Hypoplastic or absent thumb	6 (19%)
Syndactyly: simple	6 (19%)
Amniotic band syndrome	5 (16%)
Radial deficiency	2 (6%)
Syndactyly: complex	2 (6%)
Polydactyly	1 (3%)
Ulnar hypoplasia	1 (3%)
Cleft hand	1 (3%)

Table 2.

Average values for self-reported outcomes and functional assessment among children with congenital hand anomalies

Survey	Survey Domain	Mean	SD
MHQ	Overall function	72.7	28.3
	ADL	68.8	32.4
	Pain	15.7	19.5
	Hand appearance	67.1	29.5
	Satisfaction	77.9	26.3
PROMIS®	Upper extremity CAT	42.3	13.9
	Upper extremity short form	42.8	12.3
DASH		13.1	12.3
PODCI	Upper extremity function	94.2	8.2
	Mobility	98.8	4.3
	Sports	91.9	11.1
	Pain	92.4	17.1
	Happiness	88.9	15.5
	Global function	94.3	9.3

MHQ=Michigan hand outcomes questionnaire; ADL=activities of daily living; DASH=Disabilities of the arm, shoulder, and hand questionnaire, PROMIS®=Patient reported outcomes measurement information system; CAT=computer adaptive test

PROMIS® was moderately to highly correlated with all other questionnaires (Table 3). Both versions of PROMIS® exhibited high correlations with DASH scores (r > 0.80, p<0.001) and with all PODCI domains except mobility (r >0.70, p < 0.001). Correlations with MHQ domains were moderate (r > 0.40, p < 0.05). Grip strength and pinch strength were more closely correlated with the PROMIS® CAT and SF scores (r≥0.60, p<0.05) than with scores from the MHQ or DASH (Table 4). However, PROMIS® correlated more weakly with the Nine-Hole Peg Test (r <0.40, p<0.05) compared with the MHQ and DASH.

Table 3.

Correlation Between PROMIS® Pediatric Upper Extremity Item Bank and Other Surveys

Survey	Domain	PROMIS® CAT	PROMIS® Short Form
MHQ	Overall function	0.46*	0.5*†
	Satisfaction	0.41*	0.47*
	Hand appearance	0.17	0.32*
	ADL	0.49†	0.47*
	Pain	−0.46*	−0.43*
DASH		−0.87†	−0.84†
PODCI	Upper extremity function	0.89†	0.85†
	Mobility	0.63*	0.62*
	Sports	0.76†	0.77†
	Pain	0.71†	0.7†
	Happiness	0.4	0.43†
	Global function	0.8†	0.79†

MHQ=Michigan hand outcomes questionnaire; ADL=activities of daily living; DASH=Disabilities of the arm, shoulder, and hand questionnaire, PROMIS®=Patient reported outcomes measurement information system; CAT=computer adaptive test; PODCI = Pediatric outcomes data collection instrument

^*P value <0.05

^†P value<0.001

Table 4.

Correlation Between Each Survey and Objective Functional Measures

Survey	Domain	Grip Strength	Pinch Strength	9-hole Peg Test
MHQ	Overall function	0.37*	0.37*	−0.15
	Satisfaction	0.19	0.23	−0.08
	Hand appearance	0.31	0.19	−0.03
	ADL	0.42*	0.46*	−0.53†
	Pain	−0.22	−0.07	−0.09
DASH		−0.47*	−0.43*	0.38*
PROMIS®	Upper extremity CAT	0.60†	0.52†	−0.36*
	Upper extremity short form	0.63†	0.55†	−0.26
PODCI	Upper extremity function	0.41	0.34	−0.19
	Mobility	0.07	0.10	0.14
	Sports	0.36	0.21	−0.21
	Pain	0.16	0.13	0.04
	Happiness	0.09	0.16	−0.12
	Global function	0.28	0.21	−0.07

MHQ=Michigan hand outcomes questionnaire; ADL=activities of daily living; DASH=Disabilities of the arm, shoulder, and hand questionnaire, PROMIS®=Patient reported outcomes measurement information system; CAT=computer adaptive test

We measured the floor and ceiling effects of each instrument in order to understand their sensitivity among children with congenital hand differences. Ceiling effects were present for all MHQ domains, the DASH, and the PROMIS® SF measuring upper extremity function. However, neither PROMIS® instrument demonstrated ceiling effects. We did not find any floor effects.

In this study, the PROMIS® CAT and SF were more feasible for children to complete than the DASH or MHQ (Table 5). PROMIS® SF and CAT required significantly less time (mean=2.4 minutes) to complete than other measures (p<0.001), and fewer children needed assistance to fill out PROMIS®. The PROMIS® CAT had the lowest reading level (Flesch-Kincaid Grade Level=1.5) followed by the SF (Flesch-Kincaid Grade Level=1.9). The MHQ and DASH had substantially higher reading levels of 6.1 and 6.2, respectively.

Table 5.

Feasibility of Patient-Reported Outcomes Instruments

Survey	No. Items	Mean completion time (min)	Flesch-Kincaid Grade Level	Proportion needing assistance to complete
MHQ	71*	12.2	6.1	41%
DASH	34**	6.7	6.2	38%
PODCI	83	7.0	3.8	17%
PROMIS® Upper extremity short form	8	2.4	1.9	25%
PROMIS® Upper extremity CAT	Range: 5–12	2.4	1.5‡	25%

^*MHQ contains 62 items plus 9 demographic questions

^**DASH includes 30 items plus 4 additional sports/performing arts items

^‡Readability of item banks

Discussion

In this sample of children with congenital hand differences, PROMIS® demonstrates moderate correlations with functional assessment measures and moderate to high correlations with self-reported hand function, providing support for its validity in this population. Many children, particularly younger children, required assistance to complete all instruments, but were able to do so much more quickly with PROMIS® compared with other self-reported assessment tools. When PROMIS® was administered as a CAT, ceiling effects disappeared, and the majority of children preferred this mode of survey administration compared with traditional pen-and-paper completion. Although more research is needed to understand the sensitivity, responsiveness, and reliability by pediatric hand condition, our results provide encouraging evidence that PROMIS® may be an efficient and feasible option to capture self-reported hand function at the bedside.

Traditionally, objective, functional assessment has been used to measure outcomes following reconstruction for congenital hand differences. Objective measures are simple to obtain during clinical visits and can be easily followed over time. More complex tools, such as the Nine-Hole Peg Test or the Jebsen Taylor Test can assesses difficulty with activities of daily living.³⁷ Although objective, functional measures capture important elements of recovery, they fall short in several critical ways. First, objective functional testing is subject to observer variation. Second, such measures may only provide a sensitive assessment of one facet of functioning, and cannot capture all aspects of disability, such as pain and satisfaction. Children are often able to compensate for objective deficits; pain, joint instability, and exercise tolerance are more predictive of disability and general health than clinical or radiologic joint appearance for many musculoskeletal conditions. Third, objective measures may be sensitive for a specific aspect of function, but do not evaluate the impact of disability on everyday functioning. For example, a child may be able to grip an object (and would receive a high score on such a measure), but cannot participate in extracurricular activities, such as sports or music. Finally, the majority of these tests have been developed for use among adult patients, and few have been consistently used among children.

Among adults, many tools have been developed to capture self-reported hand function.¹⁰ Although no reference standard has emerged, common aspects of all instruments include the ability to complete activities or daily living, vocation and recreation without pain or disability. However, for individuals with acute or chronic upper extremity conditions, these instruments can be cumbersome and difficult to complete due to hand disability, leading to missing items and poor response rates.^31,38,39 Furthermore, surveys may lack sensitivity across all conditions, or contain irrelevant items. Instruments based on item-response theory can overcome these limitations by relying on the specificity of each survey item to estimate disability, requiring fewer items and less time with equivalent or superior accuracy. ^12,40 Recently, Döring et al. have examined the validity of PROMIS® adult item banks, including the upper extremity CAT, against other upper extremity self-assessment tools among 84 adult patients presenting with common hand conditions in an orthopedic hand clinic. Similar to our findings, PROMIS® upper extremity CAT demonstrated strong correlation against traditional measures, required less time for completion, and did not exhibit floor or ceiling effects.⁴¹ PROMIS integrates normative data into its scoring scheme, allowing scores to be compared against normal controls, as well as to children with other conditions or with the same condition. Such information is useful to clinicians and parents who are discussing the prognosis and expectations for children with congenital hand anomalies, and provides an efficient barometer that is easily interpreted. These findings, combined with the deep investment by the NIH toward developing these simple and elegant instruments, suggest that PROMIS® will be the common denominator in health assessment going forward.⁴²

Our study has several notable limitations. Although we gathered data from children representing a range of congenital hand conditions across several age groups, our sample size was small, and our findings may not be representative of all children born with congenital hand differences. Additionally, we surveyed only children who underwent reconstruction for congenital hand differences, and we cannot comment on the degree to which PROMIS® and other self-reported outcomes tools can be applied among children managed nonoperatively. Additionally, our study used a cross-sectional in design, and thus, we cannot examine other important psychometric properties of these instruments, including reliability and responsiveness. Although we do not have data drawn from children without congenital hand differences from our institution, we can compare our data with published norms for several assessment tools, such as the PODCI, PROMIS®, and functional measures. Population norms among children for the DASH and MHQ have not been established, and their appropriateness in this age group is uncertain.

Nonetheless, our findings represent an important first step to understanding the applications and limitations of self-assessment tools to capture children’s hand function. Currently, population-based evidence regarding the treatment effectiveness of reconstruction for congenital hand differences is lacking. The reasons for this are multifactorial, but are likely related to not only the infrequency of cases and the technical variation and nuances of reconstructive procedures, but also a lack of appropriate self-assessment tools that can be efficiently administered in a clinical setting. To date, few studies have systematically evaluated assessment tools among children born with upper limb disability. Current pediatric measures, such as the PODCI, lack specific aspects of hand function, are lengthy, and can be cumbersome for children with hand conditions to complete. Instruments designed for adults, such as the MHQ and DASH, contain greater detail regarding hand-related tasks, but also include elements less relevant to children, such as occupational disability. In our cohort, the PROMIS® Upper Extremity Item Bank captures disability using fewer items with equivalent sensitivity and less responder burden. Previous studies examining PROMIS® item banks among children with other chronic conditions, such as asthma, nephrotic syndrome, and cancer, demonstrate strong correlations between CAT and short form scores of the same domain.^36,42,43 However, the role of PROMIS® to gather self-reported health status among children ages 7 and younger has not been widely explored, and in our cohort, children 7 and younger were generally unable to complete these measures independently. Future efforts could be directed towards refining these existing scales and items specifically for patients with congenital hand differences in order to more precisely capture function and discern more granular detail amongst groups.

In summary, the PROMIS® Upper Extremity Item Bank is a feasible and valid measurement tool that can be efficiently used to capture upper extremity disability among children with congenital hand differences. Compared to traditional questionnaires and functional assessment tools, PROMIS® is simple to administer and efficient for children and their families to complete, and offers an innovative solution for capturing self-reported outcomes following reconstruction for congenital hand differences.