Validity and responsiveness of the Patient-Reported Outcomes Measurement Information System in Children with Ulcerative Colitis

Abstract

Objectives:

Patient-reported outcome (PRO) measures allow children to directly report on their health and well-being. We assessed the construct validity and responsiveness of the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Pediatric measures in children and adolescents with ulcerative colitis (UC).

Methods:

Through the Inflammatory Bowel Disease Partners Kids & Teens’ internet-based cohort, children with UC reported symptoms related to disease activity [Pediatric Ulcerative Colitis Activity Index (PUCAI)], IMPACT-III health-related quality of life measure, and five PROMIS Pediatric measures (anxiety, depressive symptoms, pain interference, fatigue, and peer relationships). We included participants aged 9–17 years and conducted cross sectional and longitudinal, mixed-linear regression analyses to examine the extent to which PROMIS Pediatric scores are associated with and respond to changes in PUCAI and IMPACT-III.

Results:

We evaluated 91 participants with UC (mean age 13 years, 57% female). Better PROMIS Pediatric scores were associated with lower disease activity, in both cross sectional and longitudinal analyses. For a change from moderate/severe to remission, observed effect estimates were −5.1 points for anxiety, −5.0 for depressive symptoms, −14.7 for pain interference, −13.7 for fatigue, and 5.3 for peer relationships (p <0.05 for all domains). Better PROMIS Pediatric scores were associated with improved IMPACT-III scores (p values <0.01), and changes in scores were moderately correlated with changes in IMPACT-III over time (adjusted p values <0.01).

Conclusions:

This study provides evidence for the construct validity and longitudinal responsiveness of the PROMIS Pediatric measures in pediatric UC patients, thus supporting their use in clinical research and patient care.

Introduction

Patient-reported outcome (PRO) measures capture how patients function and feel by obtaining data directly from patients themselves. PROs can capture outcomes that matter most to patients, and thus serve as valuable endpoints in both research and clinical care.^{1, 2}

PRO measures may be particularly important in pediatric ulcerative colitis (UC), a chronic, relapsing-remitting gastrointestinal disease that causes significant symptom burden, disruptions to daily life, and psychological comorbidity throughout and following childhood.^3–5 Gaining a better understanding of the physical, social and emotional impact of UC on children and adolescents will allow providers to improve patient outcomes in both disease activity and overall quality of life.

The Patient-Reported Outcomes Measurement Information System^® (PROMIS^®) includes a set of high-quality PRO measures developed with funding from the National Institutes of Health (NIH) and measures physical, emotional, and social health. It has strong evidence of reliability and validity in a variety of chronic conditions, such as cancer, nephrotic syndrome, and sickle cell disease.^{6, 7} For children ages 8 to 17 years, the PROMIS Pediatric measures are designed as self-report measures.⁷

The reliability, validity, and responsiveness of the PROMIS Pediatric measures have previously been demonstrated in pediatric Crohn’s disease, along with a variety of chronic conditions ranging from cancer to asthma.^7–15 While the PROMIS measures have been validated for use in adults with UC, the PROMIS Pediatric measures have not previously been evaluated in a pediatric UC population.¹⁶

We evaluated how the PROMIS Pediatric measures relate to disease activity and disease-specific health-related quality of life (HRQOL) in children and adolescents with UC in a cross-sectional analysis, and analyzed the responsiveness of the PROMIS Pediatric measures to changes in disease activity and HRQOL over time. We hypothesized that worse symptoms and poorer peer relationships (captured by PROMIS Pediatric measures) would be associated with higher disease activity and lower HRQOL, and that PROMIS pediatric measures would be responsive to change in disease activity and HRQOL over time.

Methods

IBD Partners

IBD Partners is a web-based cohort of adults with IBD that was launched in 2011, the methodology of which has been described elsewhere.¹⁷ IBD Partners Kids & Teens, established in 2013, invites patients with self-reported IBD and their parents to complete bi-annual surveys on health behaviors, treatments, disease activity, and other health outcomes. Patients age 9 to 17 complete self-report forms, while parents report outcomes for children younger than 9 years of age. For the present study, we included children age 9–17 with self-reported UC who completed PROMIS Pediatric measures and a disease activity measure (PUCAI) between 8/2/2013 and 5/15/2019. Children included in the study self-reported all symptoms and functioning-related questions. Demographic and clinical/disease information was obtained via parent report for all study participants.

Measures

PROMIS Pediatric measures included domains of anxiety, depressive symptoms, pain interference (a measure of the impact of pain on various aspects of life), fatigue, and peer relationships.^{1, 17, 18} For each PROMIS Pediatric measure, participants completed surveys either by four-item fixed short forms (surveys prior to 5/7/2017) or Computerized Adaptive Testing (CAT) (surveys after 5/7/2017). CAT is a flexible testing methodology that enables the answer to one question to inform the choice of the most informative following question, resulting in more precision for similar respondent burden.¹⁴ Prior research indicates measurement equivalence between CAT and fixed length measures.¹⁴ Consequently, the two assessment mechanisms were treated equivalently in the present study. PROMIS Pediatric measures are calibrated with a T-score metric with the mean of the original calibration population set to 50, and the standard deviation (SD) in the calibration population set to 10.^{6, 7} Higher PROMIS Pediatric scores represent more of the domain being measured, such that higher scores indicate worse symptom burden or better peer relationships.⁷ The minimally important difference (MID), or the difference in scores that is detectable and clinically relevant, is estimated to be three points for PROMIS Pediatric measures.^{1, 19}

We used a self-report version of the Pediatric Ulcerative Colitis Activity Index (PUCAI) to measure disease activity and the IMPACT-III as a measure of HRQOL.^20–23 The PUCAI includes questions regarding abdominal pain, rectal bleeding, stool consistency, number of stools in 24 hours, nocturnal stools, and activity level. The PUCAI is scored from 0–85 points, with 0–9 points indicating remission, 10–34 points indicating mild disease, 35–64 points indicating moderate disease, and ≥65 points indicating severe disease.^{21, 23} The IMPACT-III includes 35 HRQOL-related items in six domains (social functioning, emotional functioning, bowel symptoms, systemic symptoms, body image, and treatment interventions). The IMPACT-III is scored from 35–175, with higher scores indicating better overall HRQOL.^{20, 22}

We obtained the following demographic and clinical information from the cohort: age, sex, race, ethnicity, parental education level, disease characteristics, state of residence, current and historical medical therapy, and presence of smokers in the household.

Statistical analysis

We computed summary statistics for participant’s demographic characteristics and for all measures included in the study.

Relationship between PROMIS Pediatric measures and disease activity

We evaluated the association between PROMIS Pediatric measures and PUCAI using a mixed linear regression model. A random intercept was added to each model to account for clustering of individual participants who completed multiple surveys. We additionally adjusted for time, age, sex, race, ethnicity, and parental education level in our models.

For each of the five PROMIS Pediatric domains, we used the mixed linear model to estimate average T-scores and the standard errors, with associated 95% confidence intervals, for participants stratified by the following categories of disease activity: remission (PUCAI 0–9), mild (PUCAI 10–34), and moderate/severe (PUCAI ≥35).^{21, 23} We used the Type 3 F test to assess significance of association between the disease activity and the PROMIS measures.

To evaluate responsiveness of PROMIS Pediatric domains to changes in PUCAI, we used the mixed linear models to estimate predicted change in PROMIS Pediatric measures corresponding to a change in PUCAI disease severity categories, from moderate/severe to mild, from mild to remission, from moderate/severe to remission and the converse for each dyad.

Relationship between PROMIS Pediatric measures and HRQOL

We also evaluated the association between PROMIS Pediatric measures and IMPACT-III using mixed linear models as described above. For each PROMIS domain, we calculated average T-scores for participants, stratified by quartile of IMPACT-III. Quartiles of IMPACT-III scores were assigned using baseline (i.e. initial survey) data for all participants. We reported the standard error of the means, with associated 95% confidence intervals, and the fixed effect of IMPACT-III significance test p values.

To evaluate responsiveness of PROMIS Pediatric domains to changes in HRQOL, we used the mixed linear models to estimate predicted change in PROMIS Pediatric measures corresponding to a change in IMPACT-III, analyzed as a continuous variable. We additionally performed unadjusted analyses by calculating within-patient change from last consecutive survey and used Pearson’s correlation coefficient to evaluate the linear relationship between change in IMPACT-III and change in each of the PROMIS Pediatric measures.

Ethical considerations

All data were prepared and analyzed using SAS v 9.3 (SAS Institute, Cary, North Carolina). The IBD Partners Kids & Teens protocol and the PEPR Coordinating Center were reviewed and approved by the Institutional Review Boards (IRB) of the University of North Carolina at Chapel Hill and Duke University School of Medicine, respectively.

Results

Characteristics of Study Population

Our study sample included 91 participants with UC from 30 states across the US. The mean age was 13.3 years (SD 3.0 years) and mean duration since IBD diagnosis was 3.5 years (SD 3.1 years) (Table, Supplemental Digital Content 1 http://links.lww.com/MPG/C277); 56.7% were female and 10.0% were of Hispanic ethnicity. The sample included 2.2% African American participants, 1.1% Asian, 5.6% multiracial, and 87.8% White. Disease activity (by PUCAI) was remission for 42.4% of participants, mild for 37.8%, moderate for 15.6%, and severe for 4.4% at the time of baseline survey. Most participants had ≥1 parent who completed at least some college (81.2% for father; 90.1% for mother). The 43 participants who completed more than one survey had similar demographic characteristics to the entire cohort (Table, Supplemental Digital Content 1 http://links.lww.com/MPG/C277).

Relationship between PROMIS Pediatric Measures and Disease Activity

Adjusted PROMIS Pediatric scores were worse among patients with more severe disease activity, as measured by the PUCAI (Table 1), with a significant trend of worsening PROMIS Pediatric scores across rank ordered categories of disease activity (p < 0.01 for all PROMIS Pediatric measures). Therefore, patients who reported worse disease activity also reported worse anxiety, depressive symptoms, pain interference, fatigue, and peer relationships.

Table 1.

Adjusted PROMIS^® Pediatric mean scores^** by ulcerative colitis disease activity^*

PROMIS Pediatric Domain**	Pediatric Ulcerative Colitis Disease Activity Index (PUCAI) Category*; mean (95% CI; SE)			p value (for all PUCAI categories)
	Remission	Mild	Moderate/Severe
Anxiety	52.1 (45.8 to 58.4; 3.2)	56.0 (49.7 to 62.2; 3.1)	57.2 (50.3 to 64.1; 3.5)	<0.0001
Depressive symptoms	48.5 (42.8 to 54.2; 2.9)	49.3 (43.7 to 54.9; 2.8)	53.5 (47.4 to 59.7; 3.1)	<0.0001
Pain interference	41.0 (35.3 to 46.6; 2.9)	45.8 (40.3 to 51.4; 2.8)	55.7 (49.4 to 62.1; 3.2)	<0.0001
Fatigue	52.1 (45.8 to 58.3; 3.1)	56.4 (50.3 to 62.6; 3.1)	65.7 (58.9 to 72.6; 3.4)	<0.0001
Peer relationships	48.2 (42.0 to 54.3; 3.1)	46.8 (40.8 to 52.9; 3.1)	42.9 (36.4 to 49.4; 3.1)	<0.0001

^*Measured by Pediatric Ulcerative Colitis Disease Activity Index (PUCAI). Disease activity categories as follows: Remission = PUCAI score <10; Mild = PUCAI score 10–34; Moderate/Severe = PUCAI score 35–85

^**Lower scores in anxiety, depression, pain interference, and fatigue indicate improvements in health and/or functioning. A higher score in peer relationships indicates improved functioning.

All PROMIS Pediatric measures responded to changes in PUCAI, indicating improved physical, emotional, and social health corresponding to improved disease activity and the converse. For a change from moderate/severe to remission, observed effect estimates were -5.1 points for anxiety, -5.0 for depressive symptoms, -14.7 for pain interference, -13.7 for fatigue, and 5.3 for peer relationships (p<0.05 for all domains) (Figure 1; Table, Supplemental Digital Content 2 http://links.lww.com/MPG/C278).

Figure 1.

Predicted mean change in PROMIS^® Pediatric domains according to change in ulcerative colitis disease activity* as measured by the Pediatric Ulcerative Colitis Activity Index (PUCAI)**

*Disease activity categories as follows: Remission = PUCAI score <10; Mild = PUCAI score 10–34; Moderate/Severe = PUCAI score 35–85.

**The sign of change for peer relationships was inversed for this figure, such that a higher score indicates worsened peer relationships. For all other domains, higher scores indicate more of the domain being measured. The threshold used to indicate change in disease activity was a change in PUCAI category.

Relationship between PROMIS Pediatric Measures and HRQOL

Adjusted PROMIS Pediatric scores were worse among patients who reported worse HRQOL, as indicated by lower IMPACT-III scores (Table 2). For all PROMIS Pediatric domains, there was statistically significant trend of better PROMIS Pediatric scores across higher rank-ordered quartiles of IMPACT-III (p < 0.0001 for all PROMIS Pediatric measures). Therefore, patients who reported better HRQOL also reported better anxiety, depressive symptoms, pain interference, fatigue, and peer relationships.

Table 2.

Adjusted PROMIS^® Pediatric mean scores^** by health-related quality of life category^*

PROMIS Pediatric Domain**	HRQOL* Quartile***; mean (95% CI; SE)				p value (for all quartiles)
	1	2	3	4
Anxiety	60.9 (55.0 to 66.7; 2.9)	59.6 (53.6 to 65.7; 3.1)	52.3 (46.4 to 58.2; 2.9)	47.0 (41.1 to 52.8; 2.9)	<0.0001
Depressive symptoms	58.1 (53.0 to 63.2; 2.6)	51.3 (46.1 to 56.5; 2.6)	46.8 (41.8 to 51.9; 2.6)	42.3 (37.2 to 47.4; 2.6)	<0.0001
Pain interference	53.0 (47.0 to 58.9; 3.0)	47.6 (41.3 to 53.9; 3.2)	43.6 (37.6 to 49.6; 3.0)	38.8 (32.8 to 44.7; 3.0)	<0.0001
Fatigue	66.8 (61.7 to 71.9; 2.6)	60.2 (54.8 to 65.6; 2.7)	53.0 (47.9 to 58.2; 2.6)	45.1 (40.0 to 50.2; 2.6)	<0.0001
Peer relationships	42.7 (37.2 to 48.3; 2.8)	44.1 (38.4 to 49.8; 2.9)	49.5 (43.9 to 55.0; 2.8)	53.2 (47.6 to 58.7; 2.8)	<0.0001

^*Measured by the IMPACT-III, a 35-item health-related quality of life (HRQOL) measure. Possible scores range from 35–175, with higher scores indicating better functioning.

^**Lower scores in anxiety, depression, pain interference, and fatigue indicate improvements in health and/or functioning. A higher score in peer relationships indicates improved functioning.

^***Increasing quartile indicates better health-related quality of life.

Changes in PROMIS Pediatric scores were moderately correlated with changes in IMPACT-III, such that decreasing IMPACT-III correlated with worsening symptoms or functioning (adjusted p values <0.01 for all domains). Unadjusted Pearson correlation coefficients were also statistically significant for all domains (r = − 0.47 for anxiety, r = −0.61 for depressive symptoms, r = −0.43 for pain interference, r = −0.65 for fatigue, and r = 0.24 for peer relationships [p <0.05 for all domains]) (Figure 2).

Figure 2.

Mean change in PROMIS^® Pediatric scores according to change in IMPACT-III:

*IMPACT-III is a 35 item health-related quality of life (HRQOL) measure. Possible scores range from 35–175, with higher scores indicating better functioning.

^#The sign of change for peer relationships was inversed for this figure, such that a higher score indicates worsened peer relationships. For all other domains, higher scores indicate more of the domain being measured.

Discussion

PRO measures such as the PROMIS Pediatric measures allow a child with a health condition to directly report on their health and well-being. PRO measures that consistently assess a child’s functional status and respond to changes in functioning over time can serve as powerful tools for both clinicians and researchers. They allow providers to better understand a patient’s functioning and tailor treatment appropriately. By enabling providers to better identify patients in need of more urgent services, these tools help providers better allocate resources to their patients. In the research arena, they can act as valuable endpoints in clinical, observational, comparative effectiveness, and health services research.^{1, 2}

We performed cross-sectional and longitudinal evaluations of the PROMIS Pediatric measures in a sample of children and adolescents with UC. PROMIS Pediatric scores were significantly associated with self-reported PUCAI score, a validated UC disease activity index.^{21, 23} PROMIS Pediatric scores responded to changes in disease activity over time, worsening in response to worsening disease activity and the converse. PROMIS Pediatric scores also correlated closely with HRQOL, as measured by IMPACT-III. PROMIS Pediatric scores tracked closely with IMPACT-III scores over time, worsening in response to worsening IMPACT-III and the converse. Overall, these data provide evidence for the construct validity of the PROMIS Pediatric measures in the pediatric UC population.

To our knowledge, this is the first study to evaluate the use of PROMIS Pediatric measures in children and adolescents with UC. In an adult UC cohort, Kappelman et al previously showed that PROMIS measures are associated with disease activity and HRQOL, and that changes in disease activity over time track with changes in PROMIS measures.¹⁶ IsHak et al found the PROMIS measures were highly correlated with HRQOL in a tertiary IBD clinic that included adult UC and Crohn’s patients.²⁴ PROMIS Pediatric measures have previously been shown to be both valid and responsive to changes in disease status and HRQOL in pediatric Crohn’s disease.^{1, 18} Beyond Crohn’s disease, the PROMIS Pediatric measures have been validated in children with a wide range of chronic conditions, including cancer, nephrotic syndrome, sickle cell disease, asthma, and obesity.^7–15 In cohorts of children with cancer, nephrotic syndrome, and sickle cell disease, respectively, Reeve et al found that the PROMIS Pediatric scores responded to changes in disease status, with worsened scores correlating to disease events and improved scores with disease remission.¹² Together, there is strong evidence for the validity of the PROMIS Pediatric measures to be used across a broad range of health conditions. This will allow us to evaluate the comparative impact of disease on the lives of children and adolescents.

The strengths of this study include the geographically diverse study population and the longitudinal study design. The direct-to-patient nature of the web-based cohort enabled us to include patients from a variety of practice settings, as opposed to most pediatric IBD studies conducted at tertiary referral centers.

Limitations of the study include the volunteer nature of the cohort, which may limit generalizability. The majority of participants were white and from households with parents who completed at least some college. Furthermore, since IBD Partners Kids & Teens participants complete surveys online only, the cohort was limited to computer literate individuals with internet and computer access. Future studies including children with more diverse backgrounds are needed to confirm our findings. In addition, since the IBD Partners Kids & Teens cohort collects data from surveys only, we relied on self-report symptom-based disease activity indices instead of endoscopic endpoints. Future clinical trials would be useful to evaluate the relationship between PROMIS Pediatric domains and histologically confirmed disease activity. While the inclusion of patients with self-reported instead of physician-validated UC could introduce bias into our study, a validation study in the adult IBD partners cohort demonstrated high reliability of IBD status self-report.²⁵

This study provides evidence for the construct validity and longitudinal responsiveness of the PROMIS Pediatric measures with respect to disease status and HRQOL in pediatric UC patients. These findings support the use of PROMIS Pediatric in clinical research involving children and adolescents with UC.

What is Known

• Patient-reported outcomes capture how patients function and feel can serve as valuable endpoints in research and clinical care.

• The validity and responsiveness of the Patient-Reported Outcomes Measurement Information System^® (PROMIS^®) Pediatric measures has been previously demonstrated in pediatric Crohn’s disease and other chronic conditions.

What is New

• This study provides evidence for the construct validity and longitudinal responsiveness of the PROMIS Pediatric measures in pediatric ulcerative colitis patients.

• This study supports the use of the PROMIS Pediatric measures in children with ulcerative colitis for clinical research and patient care.