Skip to main content
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2015 Feb 13;40(6):591–601. doi: 10.1093/jpepsy/jsv005

Interpretability of the PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales in Pediatric Patients With Functional and Organic Gastrointestinal Diseases

James W Varni 1,2,, Cristiane B Bendo 3, Robert J Shulman 4, Mariella M Self 5,6, Samuel Nurko 7, James P Franciosi 8, Miguel Saps 9, Shehzad Saeed 8, George M Zacur 8, Chelsea Vaughan Dark 10, John F Pohl 11; for the Pediatric Quality of Life Inventory™ Gastrointestinal Symptoms Module Testing Study Consortium
PMCID: PMC4469917  PMID: 25682210

Abstract

Objective The present study investigates the clinical interpretability of the Pediatric Quality of Life Inventory™ (PedsQL™) Gastrointestinal Symptoms Scales and Worry Scales in pediatric patients with functional gastrointestinal disorders or organic gastrointestinal diseases in comparison with healthy controls. Methods The PedsQL™ Gastrointestinal Scales were completed by 587 patients with gastrointestinal disorders/diseases and 685 parents, and 513 healthy children and 337 parents. Minimal important difference (MID) scores were derived from the standard error of measurement (SEM). Cut-points were derived based on one and two standard deviations (SDs) from the healthy reference means. Results The percentages of patients below the scales’ cut-points were significantly greater than the healthy controls (most p values ≤ .001). Scale scores 2 SDs from the healthy reference means were within the range of scores for pediatric patients with a gastrointestinal disorder. MID values were generated using the SEM. Conclusions The findings support the clinical interpretability of the new PedsQL™ Gastrointestinal Symptoms Scales and Worry Scales.

Keywords: gastrointestinal symptoms, minimal important difference; patient-reported outcomes, PedsQL.

Introduction

Patient-centered outcomes and comparative effectiveness research is increasingly integrating patients’ perspectives regarding their health and well-being using patient-reported outcomes (PROs; Calvert et al., 2013; Desai et al., 2014; Reeve et al., 2013). PROs, including health-related quality of life (HRQOL) and symptom-specific measurement instruments, are assuming a greater role in assessing the impact of pediatric diseases and treatments from the perspective of pediatric patients with functional gastrointestinal (GI) disorders (FGIDs, e.g., chronic constipation, functional abdominal pain, irritable bowel syndrome, functional dyspepsia) and organic GI diseases (e.g., Crohn’s disease, ulcerative colitis, gastroesophageal reflux disease) (Hartman et al., 2014; Kunz, Hommel, & Greenley, 2010; Marcus et al., 2009; Ryan et al., 2013; Varni, Bendo, Denham, et al., 2015; Varni, Bendo, Nurko, et al., 2015; Varni et al., 2014; Varni, Lane, et al., 2006; Youssef, Langseder, Verga, Mones, & Rosh, 2005; Youssef, Murphy, Langseder, & Rosh, 2006). Particularly with the advent of the Food and Drug Administration (FDA) guidelines to industry regarding PROs (FDA, 2009), there has been greater emphasis on the integration of PROs with clinical and biological data in the evaluation of treatment efficacy for gastroenterology clinical trials (FDA, 2012; Mohammad et al., 2014; Williet, Sandborn, & Peyrin–Biroulet, 2014).

This paradigm shift toward PROs in clinical trials has provided the opportunity to include the perspective of pediatric patients using PROs as efficacy outcomes (Mohammad et al., 2014; Williet et al., 2014). Generic HRQOL measures provide a common metric on which to compare interventions both within and across patient groups. While generic HRQOL measures enable comparisons across patient populations and facilitate benchmarking with healthy population norms (Varni, Bendo, Nurko, et al., 2015), GI symptom-specific measures are essential to understanding symptoms most relevant for patients across multiple FGIDs and organic GI diseases (FDA, 2012; Varni, Kay, Limbers, Franciosi, & Pohl, 2012).

Interpreting PROs scores is an important requirement in the application of these measurement instruments in clinical trials, epidemiological research, and clinical practice (McLeod, Coon, Martin, Fehnel, & Hays, 2011; Schünemann, Akl, & Guyatt, 2006; Wyrwich, Norquist, Lenderking, & Acaster, 2013). The International Society for Quality of Life Research (ISOQOL) has recommended minimum standards for PROs in patient-centered outcomes and comparative effectiveness research, including the interpretability of scores (Reeve et al., 2013). Similar to the recommendations from the FDA (2009), these ISOQOL standards include “documentation of the conceptual and measurement model; evidence of reliability, validity (content validity, construct validity, responsiveness), interpretability of scores; quality translation, and acceptable patient and investigator burden” (Reeve et al., 2013, p. 1889). Meeting these standards is an iterative process, in which a new measurement instrument ideally demonstrates supportive evidence over time documenting achievement of these criteria.

The initial measurement properties of the new Pediatric Quality of Life Inventory™ (PedsQL™) Gastrointestinal Symptoms Module have been demonstrated (Varni et al., 2014). The PedsQL™ Gastrointestinal Symptoms Module was developed to address a significant gap in the pediatric literature. There was not previously available an empirically derived multidimensional GI symptom-specific instrument that measured GI symptoms across multiple FGIDs and organic GI diseases using patient self-reports for ages 5–18 years and parent proxy-reports for ages 2–18 years. The PedsQL™ Gastrointestinal Symptoms Module consists of 10 Gastrointestinal Symptoms Scales, two Gastrointestinal Worry Scales, a Medicines Scale, and a Communication Scale. The Gastrointestinal Symptoms Scales and the Gastrointestinal Worry Scales were designed to measure GI symptoms and worry across pediatric populations (Varni, Bendo, Denham, et al., 2015; Varni et al., 2014).

Consistent with recommendations from the FDA and the PRO field (FDA, 2009; Reeve et al., 2013), documentation of the conceptual and measurement model (Varni, Bendo, Denham, et al., 2015; Varni et al., 2014), and empirical evidence supporting content validity (Varni et al., 2012), reliability, and construct validity have been demonstrated (Varni, Bendo, Denham, et al., 2015; Varni et al., 2014). However, the interpretability of scale scores has not been explicitly addressed. As delineated by the ISOQOL task force (Reeve et al., 2013), the interpretability of scores includes the meaning of high and low scores, comparing scores in populations known to be healthy versus populations known to have a specific disease (known-groups validity, benchmarking with a reference or normative group), and the delineation of a minimal important difference (MID) in scores between groups and/or changes over time.

In prior research with the PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales, known-groups validity was demonstrated by comparing patients with FGIDs or organic GI diseases with an age, gender, and race/ethnicity-matched healthy sample (Varni, Bendo, Denham, et al., 2015). Consistent with expectations, pediatric patients manifested significantly worse scores on the PedsQL™ Gastrointestinal Symptoms and Worry Scales in comparison with the healthy controls with generally large effect sizes (ES), supporting the initial clinical interpretability of these scale scores. Nevertheless, these initial findings are necessary but not sufficient in clarifying the interpretation of the scores. Additional evidence in support of the clinical interpretability of these scale scores may be derived from the examination of MID scores (McLeod et al., 2011; Schünemann et al., 2006; Wyrwich et al., 2013), and the meaning of high and low scores through providing clinically relevant cut-points or deviations from a healthy reference sample (Carle, Blumberg, Moore, & Mbwana, 2011).

The MID, previously known as the minimal clinically important difference (Jaeschke, Singer, & Guyatt, 1989), has been defined as “the smallest difference in score in the outcome of interest that informed patients or informed proxies perceive as important, either beneficial or harmful, and that would lead the patient or clinician to consider a change in the management” (Schünemann et al., 2006, p. 1). In this definition, “informed proxies” are considered only if “informed patients” cannot or prefer not to make decisions about the management of their health condition (Schünemann et al., 2006). In the context of pediatric disease management, the perspectives of both pediatric patients and their parents are essential (Eiser & Varni, 2013). In the literature, both anchor-based and distribution-based methods have been used in calculating the MID for interpreting PRO scores (Wyrwich et al., 2013).

Anchor-based methods investigate the association between a PRO instrument and a similar concept measured or anchored to changes in an independent measure such as patient ratings of change, clinician ratings of change, and clinical variables (e.g., glycated hemoglobin test), that are considered to have an intuitive meaning as methods to interpret changes in the PRO instrument (Wyrwich et al., 2013). The most widely referenced anchor-based method is the original approach proposed by Jaeschke et al. (1989), which involved asking patients to rate how much overall change they experienced over time on the anchor concept. This retrospective approach to overall change, while used in adult patients, may be difficult for pediatric patients because it requires remembering the initial symptoms’ frequency or intensity, and then mentally calculating differences between current symptoms and past symptoms. Clinician’s global rating of change uses a similar approach (Wyrwich et al., 2013). However, proxy ratings of PRO concepts have been widely recognized to be only partially congruent with patient perspectives (Eiser & Varni, 2013). Direct clinical anchors are also often used, such as change in joint tenderness in patients with arthritis (Wyrwich et al., 2013). However, as succinctly summarized by McHorney (2002), “QOL scores correlate modestly at best with clinical outcomes. This finding suggests that clinical and human function are relatively independent. It does not imply that one or the other is inherently superior or correct. They simply measure different things, and using both will likely yield more information than any set alone” (p. III-58). Thus, the anchor-based method by itself is potentially insufficient in determining the MID values of a PRO instrument. An emerging perspective is that both anchor-based and distribution-based methods inform the interpretability of PRO scores (McLeod et al., 2011).

Calculations of the standard error of measurement (SEM) and effect size (ES) are the most widely used distribution-based methods, reflecting a change score difference relative to a standardized measure of variability (Wyrwich et al., 2013). ES for differences in means have been traditionally designated as small (0.20), medium (0.50), and large (0.80) in magnitude based on Cohen’s original recommendations (Cohen, 1988), with an ES of 0.50 suggested as an MID (Norman, Sloan, & Wyrwich, 2003). More recently, an ES approximating a small ES (0.20) has also been proposed, rather than the medium ES of 0.50 (Fayers & Hays, 2014). However, neither ES values have been widely adopted as methods for determining the MID (Wyrwich et al., 2013). In contrast, the SEM has been more widely accepted as a distribution-based method reflective of an MID (Wyrwich, Tierney, & Wolinsky, 1999). As articulated by Hilliard et al. (2013, p. 1892), “The SEM estimates the variation in scores due to the measurement precision in the scale and assumes that a change in scores smaller than the value of the SEM likely results from measurement error rather than a meaningful increase or decrease in the value of the construct being measured”. The SEM has been linked to the MID, in which 1 SEM has demonstrated a strong correspondence to anchor-based individual change thresholds for a number of PRO measures (Wyrwich et al., 2013), and is considered the smallest clinically meaningful change in a PRO instrument that can be detected (Crosby, Kolotkin, & Williams, 2003; Hilliard et al., 2013). For the reasons above, we used the 1 SEM distribution-based method in the current study in determining MID scores.

Cut-point scores facilitate the clinical interpretability of scale scores in addressing the ISOQOL recommendation regarding the meaning of high and low scores (Reeve et al., 2013). Cut-points create a categorical indicator by designating a point on a continuous measure that divides patient scores into categorical variables that ideally provide intuitive interpretation of the proportion or percentages of patients who scored above or below a clinically relevant indicator (Carle et al., 2011). Although there are different approaches to determining clinical cut-points, cut-point scores that delineate categorical indicators as greater than or equal to 1 or 2 standard deviations (SDs) from the mean of a healthy reference population have been used to designate an “at risk” status for a clinical disorder (≥1 SD from the healthy reference mean), and clinically significant impairment (≥2 SD from the healthy reference mean) in standardized measurement instruments, including the widely used Behavior Assessment System for Children, second edition (Reynolds & Kamphaus, 2004), and the PedsQL™ 4.0 Generic Core Scales (Varni, Burwinkle, Seid, & Skarr, 2003), and is the method selected for the current study.

To address the clinical interpretability of the new PedsQL™ Gastrointestinal Symptoms and Worry Scales, the primary objective of the present study was to investigate MID scores using the 1 SEM distribution-based method, and cut-points ≥1 and ≥2 SD from the healthy reference means as further evidence of the clinical interpretability of scores using the database from the PedsQL™ Gastrointestinal Symptoms Module field test (Varni, Bendo, Denham, et al., 2015; Varni et al., 2014). We expected that the PedsQL™ Gastrointestinal Symptoms and Worry Scales scores for patients with FGIDs or patients with organic GI diseases would demonstrate a significantly larger percentage of scores ≥1 SD and ≥2 SD from the healthy reference means in comparison with the healthy controls based on previous findings in which known-groups validity was demonstrated (Varni, Bendo, Denham, et al., 2015). We expected that the percentage of patients ≥1 SD and ≥2 SD from the healthy reference means would be generally larger for patients with FGIDs than patients with organic GI diseases based on previous findings with these new scales (Varni, Bendo, Denham, et al., 2015), and previous findings with the PedsQL™ 4.0 Generic Core Scales, in which patients with FGIDs manifested greater impaired generic HRQOL in comparison with pediatric patients with organic GI diseases (Varni, Bendo, Nurko, et al., 2015).

Methods

Pediatric Patients and Settings

Pediatric patients aged 5–18 years and parents of pediatric patients aged 2–18 years with physician-diagnosed GI disorders using ICD-9-CM Diagnosis Codes and/or Rome III criteria for FGIDs for seven GI diagnostic groups including both functional (chronic constipation, functional abdominal pain, irritable bowel syndrome, and functional dyspepsia) and organic (Crohn’s disease, ulcerative colitis, and gastroesophageal reflux disease) diseases were recruited from nine pediatric tertiary care GI clinical sites across the United States for the PedsQL™ Gastrointestinal Symptoms Module field test study (Varni et al., 2014). The diagnosis of an FGID or an organic GI disease was made by each of the site investigators, who were board-certified pediatric gastroenterologists. Diagnoses were based on current Rome III diagnostic criteria for FGIDs (Rasquin et al., 2006) and organic GI diseases (North American Society for Pediatric Gastroenterology, Hepatology, and Nutrition and/or European Society for Pediatric Gastroenterology, Hepatology, and Nutrition guidelines/consensus statements/reports).

Data collection for the PedsQL™ Gastrointestinal Symptoms Module field test study took place between March 2011 and November 2013 (Varni et al., 2014). The current study reports statistical analyses of the data from the existing field test study database not previously conducted (Varni, Bendo, Denham, et al., 2015; Varni, Bendo, Nurko, et al., 2015; Varni et al., 2014). Specifically, the calculations of MID scores and clinical cut-points have not been previously reported, and are the focus of the current set of analyses. A total of 689 families (587 children aged 5–18 years and 685 parents of children aged 2–18 years) participated in the field test study. The average age of the 318 boys (46.2%) and 371 girls (53.8%) was 11.43 years (SD =  4.58). Table I contains the participants’ characteristics for the GI group.

Table I.

Demographic Characteristics of Pediatric Patients With Gastrointestinal Disorders/Diseases and Healthy Controls

Characteristics Patient families n (%) or mean (SD) Healthy families n (%) or mean (SD)
Total number 689 552
Age 11.43 (4.58) 11.43 (4.33)
Gender
 Male 318 (46.2%) 255 (46.2%)
 Female 371 (53.8%) 297 (53.8%)
Race/ethnicity
 White non-Hispanic 517 (75.0%) 415 (75.2%)
 Hispanic 68 (9.9%) 55 (10.0%)
 Black non-Hispanic 63 (9.1%) 50 (9.1%)
 Asian/Pacific Islander 13 (1.9 %) 10 (1.8 %)
 Native American 1 (0.1%) 1 (0.2%)
 Other 27 (3.9%) 21 (3.8%)
Parent education mothers
 Less than high school graduate 40 (5.8%) 42 (7.6%)
 High school graduate 88 (12.8%) 85 (15.4%)
 Some college or certification course 180 (26.1%) 124 (22.5%)
 College graduate 228 (33.1%) 193 (35.0%)
 Graduate or professional degree 122 (17.7%) 108 (19.6%)
 Missing 31 (4.5%) 0 (0%)
Parent education fathers
 Less than high school graduate 59 (8.6%) 42 (7.6%)
 High school graduate 111 (16.1%) 99 (17.9%)
 Some college or certification course 141 (20.5%) 134 (24.3%)
 College graduate 170 (24.7%) 164 (29.7%)
 Graduate or professional degree 118 (17.1%) 113 (20.5%)
 Missing 90 (13.1%) 0 (0%)

Note. No significant differences were found between patient families in comparison with healthy families.

Healthy Controls Reference Sample

The PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales-matched healthy controls data were previously derived from the existing field test study database (Varni, Bendo, Denham, et al., 2015). These data were collected by the scientific research group at YouGov® (www.yougov.com), an Internet polling firm based in Palo Alto, CA, which has been used by a number of National Institutes of Health-funded Patient Reported Outcomes Measurement System (PROMIS®) investigators (Bjorner et al., 2014; Chung et al., 2014; Liu et al., 2010). YouGov® maintains a large panel of respondents who have agreed to participate in online surveys. YouGov® was contracted to select participants from among their panel that age, gender, and race/ethnicity-matched the age, gender, and race/ethnicity of the GI sample. Web-based data collection for the Internet panel survey took place between July 2013 and September 2013, and thus, the majority of the GI sample had already been accrued by the time the Internet panel survey was conducted.

In addition to completing the PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales, parents completed the PedsQL™ Family Information Form, which included a question on whether their child had a chronic health condition (Varni, Seid, & Kurtin, 2001). Specifically, parents answered a question on the presence of a chronic health condition (“In the past 6 months, has your child had a chronic health condition?”) defined as a physical or mental health condition that had lasted or was expected to last at least 6 months and interfered with the child’s activities as previously used in PedsQL™ studies (Varni, Burwinkle, & Seid, 2006; Varni et al., 2001, 2003). Families in which parents self-reported that their child had a chronic health condition were not included in the matched healthy sample, consistent with previous PedsQL™ studies (Alonso et al., 2010; Pohl et al., 2012). From the total sample of 792 families who participated in the Internet survey, an age, gender, and race/ethnicity sample of 552 families with healthy children was derived to match the age, gender, and race/ethnicity of the final GI sample as previously reported (Varni, Bendo, Denham, et al., 2015). The 552 families included 513 children aged 5–18 years who completed the PedsQL™ Gastrointestinal Symptoms and Worry Scales and 337 parents of children aged 2–18 years who completed the PedsQL™. The average age of the 255 boys (46.2%) and 297 girls (53.8%) was 11.43 years (SD = 4.33). Table I contains the participants’ characteristics for the healthy controls group.

Procedures

Written parental informed consent and child assent (when age appropriate) were obtained for these data during the field test study (Varni et al., 2014). The research protocol for the field test study was approved by the institutional review board at each participating institution. Following initial identification by medical staff, eligible families were notified about the field test study, which varied across the nine sites, and included mailed recruitment letters, telephone contact, or in-person contact during outpatient clinic appointments. Data were collected across the nine sites by graduate and undergraduate students, nurses, research assistants, and clinical research coordinators following the online PedsQL™ administration guidelines (www.pedsql.org). Questionnaire administration for the GI sample was primarily conducted during clinic visits after the completion of the informed consent and assent forms. For the healthy controls reference sample, parental informed consent and child assent (when age appropriate) were obtained through the Web-based panel survey protocol before questionnaire administration.

PedsQL™ Gastrointestinal Symptoms Module Development

The PedsQL™ Gastrointestinal Symptoms Module items and scales were developed through a literature review of the relevant research, national consultation with pediatric gastroenterologists, and focus interviews, cognitive interviews, and pretesting protocols with pediatric patients and their parents (Varni et al., 2012). The PedsQL™ Gastrointestinal Symptoms Module includes 74 items incorporated into 14 individual scales (Varni et al., 2014). The PedsQL™ Gastrointestinal Symptoms Module consists of 10 Gastrointestinal Symptoms Scales, two Gastrointestinal Worry Scales, a Medicines Scale, and a Communication Scale. Only the PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales from the PedsQL™ Gastrointestinal Symptoms Module are included in the present study because the Medicines and Communication Scales are not relevant to healthy populations.

PedsQL™ Gastrointestinal Symptoms Scales

The PedsQL™ Gastrointestinal Symptoms Scales encompass 10 individual scales: (1) Stomach Pain and Hurt Scale (6 items), (2) Stomach Discomfort When Eating Scale (5 items), (3) Food and Drink Limits Scale (6 items), (4) Trouble Swallowing Scale (3 items), (5) Heartburn and Reflux Scale (4 items), (6) Nausea and Vomiting Scale (4 items), (7) Gas and Bloating Scale (7 items), (8) Constipation Scale (14 items), (9) Blood in Poop Scale (2 items), and (10) Diarrhea Scale (7 items). The format, instructions, Likert response scale, and scoring method for the PedsQL™ Gastrointestinal Symptoms Scales are identical to the PedsQL™ 4.0 Generic Core Scales (Varni et al., 2001), with higher scores indicating better GI-specific HRQOL and hence lower symptoms (Varni et al., 2014).

The Scales are composed of parallel child self-report and parent proxy-report formats for children aged 5–18 years, and a parent proxy-report format for children aged 2–4 years. Child self-report forms are specific for ages 5–7, 8–12, and 13–18 years. Parent proxy-report forms are specific for children aged 2–4 (toddler), 5–7 (young child), 8–12 (child), and 13–18 (adolescent) years, and assess parents’ perceptions of their child’s GI-specific symptoms. The items for each of the forms are essentially identical, differing in developmentally appropriate language, or first- or third-person tense. The instructions ask how much of a problem each item has been during the past 1 month. A 5-point response scale is used across child and adolescent self-report for ages 8–18 years and parent proxy-report (0 = never a problem; 1 = almost never a problem; 2 = sometimes a problem; 3 = often a problem; 4 = almost always a problem). To further increase the ease of use for the young child self-report (aged 5–7 years), the response scale is reworded and simplified to a 3-point scale (0 = not at all a problem; 2 = sometimes a problem; 4 = a lot of a problem), and uses a faces scale adapted from the Pediatric Pain Questionnaire (Varni, Thompson, & Hanson, 1987).

Items are reverse-scored and linearly transformed to a 0–100 scale (0 = 100, 1 = 75, 2 = 50, 3 = 25, 4 = 0), so that lower scores demonstrate more (worse) GI symptoms and hence lower (worse) GI-specific HRQOL. Scale Scores are computed as the sum of the items divided by the number of items answered (this accounts for missing data). If >50% of the items in the scale are missing, the Scale Score is not computed (Fairclough, 2002). This accounts for the differences in sample sizes for scales reported in the tables. Although there are other strategies for imputing missing values, this computation is consistent with the previous PedsQL™ peer-reviewed publications as well as other well-established HRQOL measures (Fairclough, 2002; Varni & Limbers, 2009). To create the PedsQL™ Gastrointestinal Symptoms Scales Total Score (58 items), the mean is computed as the sum of the items divided by the number of items answered in the 10 PedsQL™ Gastrointestinal Symptoms Scales.

PedsQL™ Gastrointestinal Worry Scales

The PedsQL™ Gastrointestinal Worry Scales encompass two individual scales: (1) Worry About Going Poop Scale (5 items) and (2) Worry About Stomach Aches Scale (2 items). The format, instructions, Likert response scale, and scoring method for the PedsQL™ Gastrointestinal Worry Scales are identical to the PedsQL™ 4.0 Generic Core Scales (Varni et al., 2001), with higher scores indicating better GI-specific HRQOL and hence lower worry (Varni et al., 2014).

PedsQL™ Family Information Form

Parents completed the PedsQL™ Family Information Form, which contains demographic information including the child’s date of birth, gender, race/ethnicity, and parental education information (Varni et al., 2001).

Statistical Analysis

Cronbach's coefficient alpha was used to determine scale internal consistency reliability (Cronbach, 1951). Scales with internal consistency reliabilities of ≥0.70 are recommended for comparing patient groups, while an internal consistency reliability criterion of 0.90 is recommended for analyzing individual patient scores (Nunnally & Bernstein, 1994).

The MID was calculated using the SEM. The SEM is derived by multiplying the SD by the square root of 1-alpha (Cronbach’s alpha reliability coefficient) (Wyrwich et al., 1999).

Cut-points were determined by examining scale scores ≥1 SD and ≥2 SD below the healthy reference means (Carle et al., 2011; Reynolds & Kamphaus, 2004). Once the cut-points were determined, the percentages of patients and healthy controls who scored ≥1 SD and ≥2 SD below the healthy reference means were calculated. The percentages of patients who scored lower than a dichotomous threshold are consistent with recommendations for interpreting PRO scores in the literature, and are similar in meaning to the “proportions” of patients who achieve a designated dichotomized value (Schünemann et al., 2006). Pearson’s chi-square or Fisher’s exact (for values <5) tests were used to test for differences in proportions when expressed as percentages between patients in comparison with the healthy controls and to each other. Bonferroni familywise correction for multiple comparisons = 0.05/13 was used for child self-report and parent proxy-report. Statistical analyses were conducted using SPSS.

Results

Demographic Characteristics: Comparisons Between Groups

Descriptive characteristics of patients with GI disorders/diseases as a group and healthy controls are shown in Table I. Independent samples t tests and chi-square analyses were used to determine any group differences in age, gender, race/ethnicity, and parental education. There were no significant differences between the combined sample of pediatric patients with GI disorders/diseases and the healthy controls for age (t(1,239) = 0.00, p > .05), gender (χ2 (1) = 0.00, p > .05), race/ethnicity (χ2 (5) = 0.05, p > .05), mothers’ education (χ2 (4) = 4.93, p > .05), or fathers’ education (χ2 (4) = 2.03, p > .05). The FGID group (M = 9.86, SD = 4.61) was significantly younger than the organic GI group (M = 13.26, SD = 3.77; t(683) = −10.45, p = .000) and was composed of significantly more females (61% vs. 45%; χ2 (1) = 18.92, p = .000). There were no race/ethnicity differences between the FGID group and the organic GI group.

Internal Consistency Reliability

Cronbach’s alpha internal consistency reliability coefficients all exceeded the minimum reliability standard of 0.70 required for group comparisons (Tables II and III). The Gastrointestinal Symptoms Scales Total Score exceeded the reliability criterion of 0.90 recommended for analyzing individual patient scores, as did a number of the individual scales, including patient self-reported Symptoms Total Score, Stomach Pain and Hurt Scale, Stomach Discomfort When Eating Scale, Food and Drink Limits Scale, Gas and Bloating Scale, and the Constipation Scale (Table II).

Table II.

PedsQL™ Gastrointestinal Symptoms Scales and Worry Scales Minimal Important Difference (MID) Scores and Cut-Points for Pediatric Patients With Gastrointestinal Disorders/Diseases and Healthy Sample Cut-Points

Gastrointestinal Symptoms Scales and Worry Scales Items Gastrointestinal disorders/diseases
Healthy controls
α Mean (SD) MID SEMa (95% CI) % Cut-pointb ≥ 1 SD (95% CI) % Cut-pointb≥ 2 SD (95% CI) α Mean (SD) Cut-pointc ≥ 1 SD % Cut-pointb ≥1 SD (95% CI) Cut-pointc ≥ 2 SD % Cut-pointb ≥ 2 SD (95% CI)
Child self-report N = 587 N = 513
 Symptoms total scored 58 .96 72.5 (17.0) 3.40 (1.93–4.87) 51.9 (47.86– 55.94) 27.7 (24.08– 31.32) .98 88.6 (12.9) 75.7 15.8* (12.64– 18.96) 62.8 4.5* (2.71–6.29)
 Stomach pain and hurt 6 .92 58.0 (25.5) 7.21 (5.12–9.30) 58.6 (54.62–62.58) 27.8 (24.18–31.42) .90 81.1 (17.8) 63.3 19.7* (16.26–23.14) 45.5 2.9* (1.45–4.35)
 Stomach discomfort when eating 5 .90 74.0 (25.7) 8.13 (5.92–10.34) 39.7 (35.74–43.66) 23.6 (20.16–27.04) .90 89.6 (16.2) 73.4 14.8* (11.73–17.87) 57.2 6.6* (4.45–8.75)
 Food and drink limits 6 .90 68.6 (27.0) 8.54 (6.28–10.80) 51.3 (47.26–55.34) 30.6 (26.87–34.33) .91 89.7 (17.0) 72.7 14.6* (11.54–17.66) 55.7 7.2* (4.96–9.44)
 Trouble swallowing 3 .81 91.1 (16.1) 7.02 (4.95–9.09) 25.2 (21.69–28.71) 11.5 (8.92–14.08) .88 95.6 (10.9) 84.7 13.8 (10.82–16.78) 73.8 3.7 (2.07–5.33)
 Heartburn and reflux 4 .72 78.8 (20.0) 10.58 (8.09–13.07) 43.5 (39.49–47.51) 15.9 (12.94–18.86) .78 90.6 (14.3) 76.3 16.6* (13.38–19.82) 62.0 5.3 (3.36–7.24)
 Nausea and vomiting 4 .85 79.7 (22.5) 8.71 (6.43–10.99) 39.2 (35.25–43.15) 17.8 (14.71–20.89) .88 91.6 (14.7) 76.9 16.2* (13.01–19.39) 62.2 5.1 (3.20–7.00)
 Gas and bloating 7 .90 64.3 (24.6) 7.78 (5.61–9.95) 43.5 (39.49–47.51) 23.1 (19.69–26.51) .93 83.3 (20.1) 63.2 16.6* (13.38–19.82) 43.1 5.1* (3.20–7.00)
 Constipation 14 .94 71.1 (23.5) 5.76 (3.88–7.64) 42.6 (38.6–46.6) 18.7 (15.55–21.85) .96 86.9 (17.6) 69.3 17.0* (13.75–20.25) 51.7 4.3* (2.54–6.06)
 Blood in poop 2 .89 85.9 (23.6) 7.83 (5.66–10.0) 29.9 (26.2–33.6) 18.5 (15.36–21.64) .87 96.3 (12.0) 84.3 9.2* (6.70–11.70) 72.3 4.3* (2.54–6.06)
 Diarrhea 7 .89 78.5 (22.7) 7.53 (5.40–9.66) 47.7 (43.66–51.74) 27.9 (24.27–31.53) .91 94.3 (11.5) 82.8 13.5* (10.54–16.46) 71.3 4.9* (3.03–6.77)
 Worry about going poop 5 .86 78.1 (25.4) 9.50 (7.13–11.87) 42.9 38.90–46.90) 25.6 (22.07–29.13) .85 94.2 (12.4) 81.8 12.5* (9.64–15.36) 69.4 4.5* (2.71–6.29)
 Worry about stomach aches 2 .84 60.5 (32.8) 13.12 (10.39–15.85) 54.0 (49.97–58.03) 47.3 (43.26–51.34) .77 91.2 (16.4) 74.8 10.9* (8.20–13.60) 58.4 7.2* (4.96–9.44)
Parent proxy-report N = 685 N = 337
 Symptoms total scored 58 .97 70.0 (17.1) 2.96 (1.69–4.23) 63.0 (59.38–66.62) 36.4 (32.80–40.00) .98 90.0 (12.7) 77.3 16.0* (12.09–19.91) 64.6 6.8* (4.11–9.49)
 Stomach pain and hurt 6 .95 54.2 (25.8) 5.77 (4.02–7.52) 60.3 (56.64–63.96) 35.5 (31.92–39.08) .94 80.7 (19.2) 61.5 19.0* (14.81–23.19) 42.3 4.5* (2.29–6.71)
 Stomach discomfort when eating 5 .93 66.0 (26.8) 7.09 (5.17–9.01) 53.7 (49.97–57.43) 30.0 (26.57–33.43) .94 88.6 (17.7) 70.9 17.2* (13.17–21.23) 53.2 6.2* (3.63–8.77)
 Food and drink limits 6 .95 68.2 (29.5) 6.60 (4.74–8.46) 56.3 (52.59–60.01) 35.2 (31.62–38.78) .92 91.0 (15.6) 75.4 21.1* (16.74–25.46) 59.8 7.7* (4.85–10.55)
 Trouble swallowing 3 .89 92.2 (15.3) 5.08 (3.44–6.72) 22.6 (19.47–25.73) 10.0 (7.75– 12.25) .93 96.5 (11.3) 85.2 9.5 (6.37–12.63) 73.9 3.9 (1.83–5.97)
 Heartburn and reflux 4 .81 80.8 (20.8) 9.07 (6.92–11.22) 37.6 (33.97–41.23) 23.6 (20.42–26.78) .80 93.3 (13.0) 80.3 12.5* (8.97–16.03) 67.3 5.3* (2.91–7.69)
 Nausea and vomiting 4 .93 78.3 (24.9) 6.59 (4.73–8.45) 42.5 (38.80–46.20) 24.7 (21.47–27.93) .92 92.1 (15.2) 76.9 15.7* (11.82–19.58) 61.7 5.6* (3.15–8.05)
 Gas and bloating 7 .93 62.9 (25.3) 6.69 (4.82–8.56) 57.9 (54.20–61.60) 28.0 (24.64–31.36) .94 86.9 (18.9) 68.0 16.9* (12.90–20.90) 49.1 5.9* (3.38–8.42)
 Constipation 14 .95 66.5 (26.0) 5.81 (4.06–7.56) 55.2 (51.48–58.92) 37.1 (33.48–40.72) .96 89.3 (16.0) 73.3 16.9* (12.90–20.90) 57.3 6.2* (3.63–8.77)
 Blood in poop 2 .94 84.5 (24.8) 6.07 (4.28–7.86) 32.0 (28.51–35.49) 20.3 (17.29–23.31) .92 96.3 (12.7) 83.6 8.4* (5.44–11.36) 70.9 4.2* (2.06–6.34)
 Diarrhea 7 .90 77.4 (22.6) 7.15 (5.22–9.08) 51.8 (48.06–55.54) 37.4 (33.78–41.02) .92 94.8 (11.3) 83.5 12.8* (9.23–16.37) 72.2 5.9* (3.38–8.42)
 Worry about going poop 5 .90 75.7 (26.0) 8.22 (6.16–10.28) 48.4 (44.66–52.14) 35.5 (31.92–39.08) .90 95.4 (11.9) 83.5 8.3* (5.35–11.25) 71.6 5.0* (2.67–7.33)
 Worry about stomach aches 2 .86 60.0 (32.0) 11.97 (9.54–14.40) 67.7 (64.2–71.2) 46.8 (43.06–50.54) .79 92.8 (16.1) 76.7 14.5* (10.74–18.26) 60.6 5.9* (3.38–8.42)

Note. Lower scores demonstrate more (worse) gastrointestinal symptoms and hence lower (worse) gastrointestinal-specific HRQOL.

*p < .001 for group differences. Bonferroni familywise correction for multiple comparisons, p = .0038.

aThe Standard Error of Measurement (SEM) was derived by multiplying the SD by the square root of 1-α. PedsQL™ scores in the column represent the transformed value of 1 SEM. For example, a 3.40 change in the Gastrointestinal Symptoms Total Score for patient self-report on the 0–100 scale represents an MID.

bDemonstrates the percentage of scores ≥1 or ≥2 SD below the healthy reference means. For example, 51.9% of patients self-reported Gastrointestinal Symptoms Total Scores ≥1 SD below the healthy reference mean. Significance levels indicate differences between patients versus healthy controls.

cScores ≥1 or ≥2 SD below the healthy reference means.

dSummary Score of the 10 Gastrointestinal Symptoms Scales scores.

α = Cronbach’s alpha internal consistency reliability; SD = standard deviation; MID = minimal important difference; CI = confidence interval.

Table III.

PedsQL™ Gastrointestinal Symptoms Scales and Worry Scales Minimal Important Difference (MID) Scores and Cut-Points for Pediatric Patients With Functional or Organic Gastrointestinal Diseases

Gastrointestinal Symptoms Scales and Worry Scales Items Functional gastrointestinal disorders
Organic gastrointestinal diseases
α Mean (SD) MID SEMa (95% CI) % Cut-pointb≥ 1 SD (95% CI) % Cut-pointb≥ 2 SD (95% CI) α Mean (SD) MID SEMa (95% CI) % Cut-pointb ≥1 SD (95% CI) % Cut-pointb ≥2 SD (95% CI)
Child self-report N = 282 N = 299
 Symptoms total scorec 58 .96 67.5 (17.8) 3.56 (1.40–5.72) 64.9 (59.33–70.47) 38.7 (33.02–44.38) .96 77.2 (14.7) 2.94 (1.03–4.85) 39.5* (33.96–45.04) 17.7* (13.37–22.03)
 Stomach pain and hurt 6 .91 47.8 (25.8) 7.74 (4.62–10.86) 72.6 (67.39–77.81) 44.1 (38.31–49.89) .91 67.3 (21.1) 6.63 (3.81–9.45) 46.0* (40.35–51.65) 12.8* (9.01–16.59)
 Stomach discomfort when eating 5 .90 66.8 (28.5) 9.01 (5.67–12.35) 52.7 (46.87–58.53) 32.3 (26.84–37.76) .87 80.7 (20.9) 7.54 (4.55–10.53) 27.5* (22.44–32.56) 15.8 11.67–19.93)
 Food and drink limits 6 .91 65.1 (29.3) 8.79 (5.49–12.09) 52.9 (47.07–58.73) 36.1 (30.49–41.71) .88 72.1 (24.1) 8.35 (5.21–11.49) 49.7 (44.03–55.37) 25.2 (20.28–30.12)
 Trouble swallowing 3 .85 87.4 (19.5) 7.55 (4.47–10.63) 33.3 (27.80–38.80) 18.4 (13.88–22.92) .64 94.6 (11.1) 6.66 (3.83–9.49) 17.4 (13.10–21.70) 5.0 (2.53–7.47)
 Heartburn and reflux 4 .74 74.3 (22.5) 11.48 (7.76–15.20) 51.8 (45.97–57.63) 23.8 (18.83–28.77) .63 82.9 (16.4) 9.98 (6.58–13.38) 35.8 (30.37–41.23) 8.7 (5.51–11.89)
 Nausea and vomiting 4 .82 73.1 (23.9) 10.14 (6.62–13.66) 51.4 (45.57–57.23) 26.6 (21.44–31.76) .87 85.6 (19.2) 6.92 (4.04–9.80) 28.1* (23.01–33.19) 9.7* (6.35–13.05)
 Gas and bloating 7 .91 60.4 (26.8) 8.04 (4.87–11.21) 50.0 (44.16–55.84) 29.8 (24.46–35.14) .88 68.0 (21.7) 7.52 (4.53–10.51) 37.2 (31.72–42.68) 16.6 (12.38–20.82)
 Constipation 14 .95 65.4 (25.5) 5.70 (2.99–8.41) 51.6 (45.77–57.43) 25.8 (20.69–30.91) .94 76.4 (20.2) 4.95 (2.49–7.41) 33.8 (28.44–39.16) 12.2 (8.49–15.91)
 Blood in Poop 2 .89 89.3 (22.3) 7.40 (4.34–10.46) 21.9 (17.07–26.73) 12.9 (8.99–16.81) .89 82.6 (24.5) 8.13 (5.03–11.23) 37.5 (32.01–42.99) 24.0 (19.16–28.84)
 Diarrhea 7 .87 75.2 (23.3) 8.40 (5.16–11.64) 53.2 (47.38–59.02) 32.0 (26.56–37.44) .91 81.3 (21.8) 6.54 (3.74–9.34) 42.6 (37.00–48.20) 24.3 (19.44–29.16)
 Worry about going poop 5 .86 72.7 (28.5) 10.66 (7.06–14.26) 51.6 (45.77–57.43) 33.8 (28.28–39.32) .85 83.0 (21.1) 8.17 (5.07–11.27) 34.9 (29.50–40.30) 18.1 (13.74–22.46)
 Worry about stomach aches 2 .84 51.7 (34.2) 13.68 (9.67–17.69) 64.8 (59.23–70.37) 61.2 (55.51–66.89) .80 68.5 (29.1) 13.01 (9.20–16.82) 44.3 (38.67–49.93) 34.6* (29.21–39.99)
Parent proxy-report N = 363 N = 312
 Symptoms total scorec 58 .96 66.3 (16.6) 3.32 (1.48–5.16) 72.2 (67.59–76.81) 45.2 (40.08–50.32) .97 74.4 (16.8) 2.91 (1.04–4.78) 52.6 (47.06–58.14) 26.3 (21.41–31.19)
 Stomach pain and hurt 6 .94 47.3 (25.0) 6.12 (3.65–8.59) 70.2 (65.49–74.91) 47.5 (42.36–52.64) .95 62.0 (24.4) 5.46 (2.94–7.98) 48.9* (43.35–54.45) 21.9* (17.31–26.49)
 Stomach discomfort when eating 5 .91 61.9 (26.9) 8.07 (5.27–10.87) 61.2 (56.19–66.21) 34.6 (29.71–39.49) .95 70.8 (26.0) 5.81 (3.21–8.41) 45.0 (39.48–50.52) 24.8 (20.01–29.59)
 Food and drink limits 6 .95 68.3 (30.3) 6.78 (4.19–9.37) 56.2 (51.10–61.30) 35.5 (30.58–40.42) .95 68.4 (28.4) 6.35 (3.64–9.06) 55.8 (50.29–61.31) 34.3 (29.03–39.57)
 Trouble swallowing 3 .90 91.0 (16.9) 5.34 (3.03–7.65) 24.7 (20.26–29.14) 11.8 (8.48–15.12) .86 93.5 (13.3) 4.98 (2.57–7.39) 20.4 (15.93–24.87) 8.0 (4.99–11.01)
 Heartburn and reflux 4 .82 80.0 (22.1) 9.37 (6.37–12.37) 38.5 (33.49–43.51) 26.1 (21.58–30.62) .79 81.8 (19.2) 8.80 (5.66–11.94) 36.7 (31.35–42.05) 20.8 (16.30–25.30)
 Nausea and vomiting 4 .92 76.0 (26.1) 7.38 (4.69–10.07) 46.4 (41.27–51.53) 28.0 (23.38–32.62) .93 80.8 (23.1) 6.11 (3.45–8.77) 38.3 (32.91–43.69) 21.1 (16.57–25.63)
 Gas and bloating 7 .92 58.5 (25.2) 7.13 (4.48–9.78) 66.0 (61.13–70.87) 35.9 (30.97–40.83) .93 67.8 (24.4) 6.46 (3.73–9.19) 48.7 (43.15–54.25) 18.7 (14.37–23.03)
 Constipation 14 .96 58.3 (26.9) 5.38 (3.06–7.70) 67.7 (62.89–72.51) 49.6 (44.46–54.74) .95 76.1 (21.2) 4.74 (2.38–7.10) 41.1* (35.64–46.56) 22.7* (18.05–27.35)
 Blood in poop 2 .93 88.9 (22.4) 5.93 (3.50–8.36) 21.9 (17.65–26.15) 13.9 (10.34–17.46) .93 79.6 (26.3) 6.96 (4.14–9.78) 43.5* (38.00–49.00) 27.4 (22.45–32.35)
 Diarrhea 7 .87 76.7 (22.5) 8.11 (5.30–10.92) 53.1 (47.97–58.23) 38.6 (33.59–43.61) .94 78.2 (22.8) 5.58 (3.03–8.13) 50.3 (44.75–55.85) 36.1 (30.77–41.43)
 Worry about going poop 5 .89 71.8 (27.8) 9.22 (6.24–12.20) 55.8 (50.69–60.91) 42.0 (36.92–47.08) .91 80.2 (23.0) 6.90 (4.09–9.71) 39.9 (34.47–45.33) 28.3 (23.30–33.30)
 Worry about stomach aches 2 .84 54.1 (33.5) 13.40 (9.90–16.90) 73.0 (68.43–77.57) 53.2 (48.07–58.33) .86 66.6 (28.7) 10.74 (7.30–14.18) 61.7 (56.31–67.09) 39.9 (34.47–45.33)

Note. Lower scores demonstrate more (worse) gastrointestinal symptoms and hence lower (worse) gastrointestinal-specific HRQOL.

*p < .001 for group differences. Bonferroni familywise correction for multiple comparisons, p = .0038.

aThe Standard Error of Measurement (SEM) was derived by multiplying the SD by the square root of 1-α. PedsQL™ scores in the column represent the transformed value of 1 SEM. For example, a 2.94 change in the Gastrointestinal Symptoms Total Score for patient self-report on the 0–100 scale represents a MID for organic gastrointestinal diseases.

bDemonstrates the percentage of scores ≥1 or ≥2 SD below the healthy reference means in Table II. For example, 64.9% of patients with functional gastrointestinal diseases self-reported Gastrointestinal Symptoms Total Scores ≥1 SD below the healthy reference mean. Significance levels indicate differences between patients with functional disorders versus organic disease.

cSummary Score of the 10 Gastrointestinal Symptoms Scales scores.

α = Cronbach’s alpha internal consistency reliability; SD = standard deviation; MID = minimal important difference; CI = confidence interval.

MID Scores

Table II shows the MID scores for pediatric patients with GI disorders/diseases as a group. Table III shows the MID scores separately for patients with FGIDs or organic GI diseases.

These MID values provide information on the clinical interpretability of the scales. For example, in Table II, a patient self-reported score that changed ≥5.76 on the Constipation Scale is a numerical value indicating the smallest clinically meaningful change in this scale that can be detected specifically for patients with GI disorders/diseases as a group. The other MID values in Table II can be similarly interpreted. In Table III, for example, a patient self-reported score that changed ≥7.74 on the Stomach Pain and Hurt Scale for the FGIDs group is a numerical value indicating the smallest clinically meaningful change in this scale that can be detected specifically for patients with FGIDs as a group. The other MID values in Table III can be similarly interpreted for the FGID and organic GI groups.

Cut-Point Scores

Table II shows the ≥1 SD and ≥2 SD cut-points for the PedsQL™ Gastrointestinal Symptoms and Worry Scales based on the healthy reference means in the columns labeled “Cut Point ≥1 SD” or “Cut Point ≥2 SD” under the “Healthy Controls” heading. Using these cut-points, the percentage of patients and healthy controls who scored below these cut-points was computed as shown in the columns labeled “% Cut Point ≥1 SD” or “% Cut Point ≥2 SD” under the headings “Gastrointestinal Disorders/Diseases” and “Healthy Controls” in Table II, and “Functional Gastrointestinal Disorders” and “Organic Gastrointestinal Diseases” in Table III.

In Table II, all of the percentages were significantly higher for patients with GI disorders/diseases as a combined group in comparison with the healthy controls (p .001), except for the Trouble Swallowing Scale for patient self-report and parent proxy-report, and the patient self-reported Heartburn and Reflux Scale and Nausea and Vomiting Scale scores for ≥2 SD from the healthy controls reference means after Bonferroni familywise correction for multiple comparisons (p = .0038). These significance levels are shown in the columns labeled “% Cut Point ≥1 SD” or “% Cut Point ≥2 SD” under the “Healthy Controls” heading in Table II.

As shown in Table III, patients with FGIDs demonstrated higher percentages of scale scores ≥1 or ≥2 SDs from the healthy controls reference cut-points in comparison with patients with organic GI diseases for the patient self-reported Symptoms Total Score, Stomach Pain and Hurt Scale, Stomach Discomfort When Eating Scale (for ≥1 SD only), Nausea and Vomiting Scale, and the Worry About Stomach Aches Scale (for ≥2 SD only) after Bonferroni familywise correction for multiple comparisons (p = .0038). These significance levels are shown in the columns labeled “% Cut Point ≥1 SD” or “% Cut Point ≥2 SD” under the “Organic Gastrointestinal Diseases” heading in Table III. Parent proxy-report demonstrated significant differences between patients with FGIDs in comparison with patients with organic GI diseases for the Stomach Pain and Hurt Scale, Constipation Scale, and the Blood in Poop Scale (for ≥1 SD only) in Table III after Bonferroni familywise correction for multiple comparisons (p = .0038).

Discussion

The findings support the clinical interpretability of the new PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales in pediatric patients with GI disorders/diseases as a group and patients with FGIDs and patients with organic GI diseases separately by providing MID values for each of the scale scores and cut-points for ≥1 and ≥2 SD below the healthy controls reference means. The MID scores in the tables represent the smallest clinically meaningful change in the individual scales that can be detected for these patient groups, and represent meaningful variation in the measured construct (latent variable) that is likely not a function of measurement error (Hilliard et al., 2013). Thus, the MID represents a single value for each scale (with 95% confidence intervals), separately for patient self-report and parent proxy-report, that indicates the magnitude of change in scale scores that is detectable by the patient and parent as a clinically meaningful difference in the construct being measured by the individual scale, and provides an important reference point that can be used in clinical research and practice (Hilliard et al., 2013). Taken together with the findings on known-groups validity (Varni, Bendo, Denham, et al., 2015), these data address the recommended minimum standards for clinical interpretability of PRO scale scores (Reeve et al., 2013).

The cut-point scores derived from the healthy controls reference sample may be useful in benchmarking. PedsQL™ Gastrointestinal Symptoms and Worry Scales scores ≥2 SD from the healthy reference means were all within the range of scores for patients with a GI disorder/disease. The results demonstrated that pediatric patients with FGIDs and organic GI diseases as a group manifested significantly greater percentages of symptoms and worry scores below the cut-points than the healthy controls. The results also showed that pediatric patients with FGIDs as a group generally manifested greater percentages of symptoms and worry scores below the cut-points than pediatric patients with organic GI diseases. Because GI symptoms are relatively ubiquitous in pediatric populations (Chitkara, Rawat, & Talley, 2005; Saps et al., 2009; van den Berg, Benninga, & Di Lorenzo, 2006), future research should investigate whether these new scales have potential application not only in disease-specific groups, but also at the population health level, in identifying children with persistent GI symptoms that may warrant further evaluation and intervention.

Each of the individual PedsQL™ Gastrointestinal Symptoms and Worry Scales can be used as stand-alone scales targeting the specific symptoms pertinent to a particular pediatric FGID or organic GI disease given the requirements of a clinical trial or practice need, as well as subgroup differences across scales. This use of individual scales has precedence from other PedsQL™ Modules (Varni et al., 2011). Each of the individual scales is brief and easy to score. Selecting scales that are most relevant given the objectives and hypotheses of a clinical trial addresses the recommended standards for “acceptable patient and investigator burden” (Reeve et al., 2013, p. 1889) and the Consolidated Standards of Reporting Trials PRO extension (Calvert et al., 2013).

The present study has several strengths, including the relatively large overall sample size for both the GI patients and healthy samples, the broad age range of participants, and the nationwide representation of the participants. Limitations include the lack of information on the demographic characteristics and number of families who were approached and chose not to participate in the study, and a primarily Caucasian sample and high level of parental education, which may limit generalizability. Further, not all possible GI disorders were included (e.g., celiac disease). Due to sample size limitations for the individual age-groups, we were not able to provide MID and cut-points for specific ages. Future research with larger sample sizes for the individual age-groups will be needed to further validate the findings of the current analyses. The FGID group was younger than the organic GI group and was composed of more females. These sample differences are consistent with the broader GI literature, in which females are reported to experience greater functional symptoms, particularly abdominal pain and somatic complaints without identifiable organic cause (Chitkara et al., 2005; Saps et al., 2009). The older age of the organic GI group may reflect the fact that patients with Crohn’s disease, which comprised the largest group of patients with organic GI disease in the current study, are typically diagnosed during adolescence and hence would be expected to be older than the FGID group (Kim & Ferry, 2004), who are often initially diagnosed at younger ages (Chitkara et al., 2005; Saps et al., 2009; van den Berg et al., 2006). Thus, the age and gender characteristics of the sample in this regard may resemble population parameters for these characteristics. We did not adjust for these age and gender differences in our comparative analyses between patients with FGIDs and patients with organic GI diseases because the unadjusted scores are required for reporting MID scale scores, which will serve as reference values for future investigations in determining sample size requirements and clinically meaningful change using these new scales.

We used the widely used SEM distribution-based method in determining the MID given the data available from the field test study database (Varni et al., 2014). As stated by Wyrwich et al. (2013), “these distribution-based methods for PRO interpretation provide an alternative to anchor-based methods when an appropriate anchor is not available” (p. 480). Nevertheless, future studies with these new scales should investigate anchor-based methods such as clinical variables and disease-specific indices. For example, for patients with organic GI diseases such as inflammatory bowel disease, disease activity indices such as the Pediatric Crohn’s Disease Activity Index (Hyams et al., 2005) and the Pediatric Ulcerative Colitis Activity Index (Turner et al., 2007) or other potential anchor-based methods such as laboratory-based measures of inflammation should be considered. Future research will need to include these indices in the validation of these new scales in patients with organic GI diseases. However, the diagnosis of an FGID based on Rome III criteria specifically requires that an FGID diagnosis is made only in patients in whom there is no evidence of organic disease (Rasquin et al., 2006). Therefore, identifying clinical parameters for anchor-based methods may be more challenging in pediatric patients with FGIDs. Additionally, it might be anticipated that the MID calculated for individual diagnostic groups (e.g., Crohn’s disease, irritable bowel syndrome) will be more precise in the determination of MID values most relevant for a given diagnosis. Calculating the cut-point percentages for individual scales for specific diagnostic groups and age-groups will further contribute to our understanding of the clinical interpretability of these new scales.

Finally, the methods contained within this study have direct relevance to pediatric psychology research and practice. Pediatric psychologists are typically strong advocates for hearing the voices of the children in matters of their health and well-being in pediatric settings (Varni et al., 2005), including the youngest children feasible (Varni, Limbers, & Burwinkle, 2007a). As such, developing standardized PRO measurement instruments that reflect the patient’s perspective, and which include meaningful clinical indicators of change and the identification of levels of functioning indicative of at-risk status and clinical impairment, are well within the purview of pediatric psychology research and practice (Hilliard et al., 2013; Ryan et al., 2013; Varni, Limbers, & Burwinkle, 2007b).

Funding

No funding was specifically designated for the PedsQL™ Gastrointestinal Symptoms Module field test study data collection effort or manuscript preparation. Item development for the PedsQL™ Gastrointestinal Symptoms Module was previously supported by Takeda Pharmaceuticals North America, Inc. Data collection for the healthy controls sample was supported by intramural funding from the Texas A&M University Foundation.

Conflicts of interest: Dr. J.W.V holds the copyright and the trademark for the PedsQL™ and receives financial compensation from the Mapi Research Trust, which is a nonprofit research institute that charges distribution fees to for-profit companies that use the Pediatric Quality of Life Inventory™. Dr. J.W.V received investigator-initiated funding from Takeda Pharmaceuticals North America, Inc. (Deerfield, IL) for the previous item generation qualitative methods study. Dr. J.F.P. received investigator-initiated funding from Takeda Pharmaceuticals North America, Inc. (Deerfield, IL), for the previous item generation qualitative methods study. Drs. J.W.V and J.F.P. did not receive funding from Takeda Pharmaceuticals North America, Inc., for the current quantitative methods field test study. Dr. J.F.P. has received the following funding: INSPPIRE to Study Acute Recurrent and Chronic Pancreatitis is Children, Grant # 10987759, National Institutes of Health (NIH), National Institute of Diabetes and Digestive and Kidney Diseases. Dr. J.F.P. is on the speaker’s bureau for Medical Education Resources. The other authors report no competing interests related to this study.

Appendix

Pediatric Quality of Life Inventory™ (PedsQL™) Gastrointestinal Symptoms Module Testing Study Consortium

The Pediatric Quality of Life Inventory™ (PedsQL™) Gastrointestinal Symptoms Module Testing Study Consortium sites include a Network and Statistical Center at the Center for Health Systems & Design, Colleges of Architecture and Medicine, Texas A&M University, College Station, TX (PI: J.W.V, PhD), and nine primary research data collection sites: Division of Pediatric Gastroenterology, Nationwide Children's Hospital, Ohio State University School of Medicine, Columbus, OH (PI: Jolanda Denham, MD); Department of Pediatrics, Baylor College of Medicine, Children’s Nutrition Research Center, Texas Children’s Hospital, Houston, TX (PIs: R.J.S., MD, and M.M.S., PhD); Division of Gastroenterology, Hepatology and Nutrition, Children's Hospital Colorado, Aurora, CO (PI: Deborah A. Neigut, MD); Center for Motility and Functional Gastrointestinal Disorders, Boston Children's Hospital, Harvard Medical School, Boston, MA (PI: S.N., MD); Division of Pediatric Gastroenterology, Children’s Medical Center of Dallas, University of Texas Southwestern Medical School, Dallas, TX (PI: Ashish S. Patel, MD); Division of Gastroenterology, Hepatology and Nutrition, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH (PIs: J.P.F., MD, S.S., MD, and G.M.Z., MD); Division of Gastroenterology, Hepatology and Nutrition, Lurie Children's Hospital of Chicago, Northwestern University Feinberg School of Medicine, Chicago, IL (PI: M.S., MD); Division of Pediatric Gastroenterology, Hepatology and Nutrition, Goryeb Children’s Hospital, Morristown Medical Center, Morristown, NJ (PI: Barbara Verga, MD); Department of Pediatric Gastroenterology, Primary Children’s Hospital, University of Utah, Salt Lake City, UT (PI: J.F.P., MD).

References

  1. Alonso E. M., Limbers C. A., Neighbors K., Martz K., Bucuvalas J. C., Webb T., Varni J. W. (2010). Cross-sectional analysis of health-related quality of life in pediatric liver transplant recipients. Journal of Pediatrics, 156, 270–276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bjorner J. B., Rose M., Gandek B., Stone A. A., Junghaenel D. U., Ware J. E. (2014). Difference in method of administration did not significantly impact item response: An IRT-based analysis from the Patient-Reported Outcomes Measurement Information System (PROMIS) initiative. Quality of Life Research, 23, 217–227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Calvert M., Blazeby J., Altman D. G., Revicki D. A., Moher D., Brundage M. D. (2013). Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension. JAMA, 309, 814–822. [DOI] [PubMed] [Google Scholar]
  4. Carle A. C., Blumberg S. J., Moore K. A., Mbwana K. (2011). Advanced psychometric methods for developing and evaluating cut-point-based indicators. Child Indicators Research, 4, 101–126. [Google Scholar]
  5. Chitkara D. K., Rawat D. J., Talley N. J. (2005). The epidemiology of childhood recurrent abdominal pain in Western countries: A systematic review. American Journal of Gastroenterology, 100, 1868–1875. [DOI] [PubMed] [Google Scholar]
  6. Chung H., Kim J., Cook K. F., Askew R. L., Revicki D. A., Amtmann D. (2014). Testing measurement invariance of the patient-reported outcomes measurement information system pain behaviors score between the US general population sample and a sample of individuals with chronic pain. Quality of Life Research, 23, 239–244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cohen J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. [Google Scholar]
  8. Cronbach L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. [Google Scholar]
  9. Crosby R. D., Kolotkin R. L., Williams G. R. (2003). Defining clinically meaningful change in health-related quality of life. Journal of Clinical Epidemiology, 56, 395–407. [DOI] [PubMed] [Google Scholar]
  10. Desai A. D., Zhou C., Stanford S., Haaland W., Varni J. W., Mangione-Smith R. M. (2014). Validity and responsiveness of the Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Core Scales in the pediatric inpatient setting. JAMA Pediatrics, 168, 1114–1121. [DOI] [PubMed] [Google Scholar]
  11. Eiser C., Varni J. W. (2013). Health related quality of life and symptom reporting: Similarities and differences between children and their parents. European Journal of Pediatrics, 172, 1299–1304. [DOI] [PubMed] [Google Scholar]
  12. Fairclough D.L. (2002). Design and analysis of quality of life studies in clinical trials: Interdisciplinary statistics. New York, NY: Chapman & Hall/CRC. [Google Scholar]
  13. Fayers P. M., Hays R. D. (2014). Don’t middle your MIDs: Regression to the mean shrinks estimates of minimally important differences. Quality of Life Research, 23, 1–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. FDA. (2009). Guidance for Industry: Patient-reported outcome measures: Use in medical product development to support labeling claims . Rockville, MD: Food and Drug Administration, U.S. Department of Health and Human Services. [Google Scholar]
  15. FDA. (2012, September 19). Gastroenterology regulatory endpoints and the advancement of therapeutics (GREAT Workshop). College Park, MD: FDA. [Google Scholar]
  16. Hartman E. E., Pawaskar M., Williams V., McLeod L., Dubois D., Benninga M. A., Joseph A. (2014). Psychometric properties of the PedsQL Generic Core Scales for children with functional constipation in the Netherlands. Journal of Pediatric Gastroenterology & Nutrition, 59, 739–747. [DOI] [PubMed] [Google Scholar]
  17. Hilliard M. E., Lawrence J. M., Modi A. C., Anderson A., Crume T., Dolan L. M., Hood K. K. (2013). Identification of minimal clinically important difference scores of the Pediatric Quality of Life Inventory in children, adolescents, and young adults with Type 1 and Type 2 diabetes. Diabetes Care, 36, 1891–1897. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hyams J., Markowitz J., Otley A., Rosh J., Mack D., Bousvaros A., Griffiths A. (2005). Evaluation of the pediatric Crohn disease activity index: A prospective multicenter experience. Journal of Pediatric Gastroenterology and Nutrition, 41, 416–421. [DOI] [PubMed] [Google Scholar]
  19. Jaeschke R., Singer J., Guyatt G. H. (1989). Measurement of health status: Ascertaining the minimal clinically important difference. Controlled Clinical Trials, 10, 407–415. [DOI] [PubMed] [Google Scholar]
  20. Kim S. C., Ferry G. D. (2004). Inflammatory bowel diseases in pediatric and adolescent patients: Clinical, therapeutic, and psychosocial considerations. Gastroenterology, 126, 1550–1560. [DOI] [PubMed] [Google Scholar]
  21. Kunz J. H., Hommel K. A., Greenley R. N. (2010). Health-related quality of life of youth with inflammatory bowel disease: A comparison with published data using the PedsQL 4.0 Generic Core Scales. Inflammatory Bowel Diseases, 16, 939–946. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Liu H., Cella D., Gershon R., Shen J., Morales L. S., Riley W., Hays R. D. (2010). Representativeness of the patient-reported outcomes measurement information system Internet panel. Journal of Clinical Epidemiology, 63, 1169–1178. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Marcus S. B., Strople J. A., Neighbors K., Weissberg-Benchell J., Nelson S. P., Limbers C. A., Varni J. W., Alonso E.M. (2009). Fatigue and health-related quality of life in pediatric inflammatory bowel disease. Clinical Gastroenterology and Hepatology, 7, 554–561. [DOI] [PubMed] [Google Scholar]
  24. McHorney C. A. (2002). The potential clinical value of quality of life information. Medical Care, 40(Suppl), III-56–III-62. [DOI] [PubMed] [Google Scholar]
  25. McLeod L. D., Coon C. D., Martin S. A., Fehnel S. E., Hays R. D. (2011). Interpreting patient-reported outcome results: US FDA guidance and emerging methods. Expert Review of Pharmacoeconomics and Outcomes Research, 11, 163–169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Mohammad S., Di Lorenzo C., Youssef N. N., Miranda A., Nurko S., Hyman P., Saps M. (2014). Assessment of abdominal pain through global outcomes and recent FDA recommendations in children: Are we ready for change? Journal of Pediatric Gastroenterology and Nutrition, 58, 46–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Norman G. R., Sloan J. A., Wyrwich K. W. (2003). Interpretation of changes in health-related quality of life: The remarkable universality of half a standard deviation. Medical Care, 41, 582–592. [DOI] [PubMed] [Google Scholar]
  28. Nunnally J. C., Bernstein I. R. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill. [Google Scholar]
  29. Pohl J. F., Limbers C. A., Kay M. T., Harman A., Rollins M., Varni J. W. (2012). Health-related quality of life in pediatric patients with long-standing pancreatitis. Journal of Pediatric Gastroenterology and Nutrition, 54, 657–663. [DOI] [PubMed] [Google Scholar]
  30. Rasquin A., Di Lorenzo C., Forbes D., Guiraldes E., Hyams J. S., Staiano A., Walker L. S. (2006). Childhood functional gastrointestinal disorders: Child/adolescent. Gastroenterology, 130, 1527–1537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Reeve B. B., Wyrwich K. W., Wu A. W., Velikova G., Terwee C. B., Snyder C. F., Schwartz C., Revicki D. A., Moinpour C. M., McLeod L. D., Lyons J. C., Lenderking W. R., Hinds P. S., Hays R. D., Greenhalgh J., Gershon R., Feeny D., Fayers P. M., Cella D., Brundage M., Ahmed S., Aaronson N. K., Butt Z. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22, 1889–1905. [DOI] [PubMed] [Google Scholar]
  32. Reynolds C. R., Kamphaus R. W. (2004). Behavior assessment system for children (2nd ed.). Circle Pines, MN: American Guidance Service. [Google Scholar]
  33. Ryan J. L., Mellon M. W., Junger K. W., Hente E. A., Denson L. A., Saeed S., Hommel K. A. (2013). The clinical utility of health-related quality of life screening in a pediatric inflammatory bowel disease clinic. Inflammatory Bowel Diseases, 19, 2666–2672. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Saps M., Sesphadri R., Sztainberg M., Schaffer G., Marshall B. M., Di Lorenzo C. (2009). A prospective school-based study of abdominal pain and other common somatic complaints in children. Journal of Pediatrics, 154, 322–326. [DOI] [PubMed] [Google Scholar]
  35. Schünemann H. J., Akl E. A., Guyatt G. H. (2006). Interpreting the results of patient reported outcome measures in clinical trials: The clinician's perspective. Health and Quality of Life Outcomes, 4, 62. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Turner D., Otley A. R., Macks D., Hyams J., De Bruijne J., Uusoue K., Walters T. D., Zachos M., Mamula P., Beaton D. E., Steinhart A. H., Griffiths A. M. (2007). Development, validation, and evaluation of a Pediatric Ulcerative Colitis Activity Index: A prospective multicenter study. Gastroenterology, 133, 423–432. [DOI] [PubMed] [Google Scholar]
  37. van den Berg M. M., Benninga M. A., Di Lorenzo C. (2006). Epidemiology of childhood constipation: A systematic review. American Journal of Gastroenterology, 101, 2401–2409. [DOI] [PubMed] [Google Scholar]
  38. Varni J. W., Bendo C. B., Denham J., Shulman R. J., Self M. M., Neigut D. A., Nurko S., Patel A. S., Franciosi J. P., Saps M., Verga B., Smith A., Yeckes A., Heinz N., Langseder A., Saeed S., Zacur G. M., Pohl J. F. (2014). PedsQL™ Gastrointestinal Symptoms Module: Feasibility, reliability, and validity. Journal of Pediatric Gastroenterology and Nutrition, 59, 347–355. [DOI] [PubMed] [Google Scholar]
  39. Varni J. W., Bendo C. B., Denham J., Shulman R. J., Self M. M., Neigut D. A., Nurko S., Patel A. S., Franciosi J. P., Saps M., Yeckes A., Langseder A., Saeed S., Pohl J. F. (2015). PedsQL™ Gastrointestinal Symptoms Scales and Gastrointestinal Worry Scales in pediatric patients with functional and organic gastrointestinal diseases in comparison to healthy controls. Quality of Life Research, 24, 363–378. [DOI] [PubMed] [Google Scholar]
  40. Varni J. W., Bendo C. B., Nurko S., Shulman R. J., Self M. M., Franciosi J. P., Saps M., Pohl J. F. (2015). Health-related quality of life in pediatric patients with functional and organic gastrointestinal diseases. Journal of Pediatrics, 166, 85–90. [DOI] [PubMed] [Google Scholar]
  41. Varni J. W., Burwinkle T. M., Seid M. (2006). The PedsQL™ 4.0 as a school population health measure: Feasibility, reliability, and validity. Quality of Life Research, 15, 203–215. [DOI] [PubMed] [Google Scholar]
  42. Varni J. W., Burwinkle T. M., Seid M., Skarr D. (2003). The PedsQL™ 4.0 as a pediatric population health measure: Feasibility, reliability, and validity. Ambulatory Pediatrics, 3, 329–341. [DOI] [PubMed] [Google Scholar]
  43. Varni J. W., Burwinkle T. M., Sherman S. A., Hanna K., Berrin S. J., Malcarne V. L., Chambers H. G. (2005). Health-related quality of life of children and adolescents with cerebral palsy: Hearing the voices of the children. Developmental Medicine and Child Neurology, 47, 592–597. [PubMed] [Google Scholar]
  44. Varni J. W., Kay M. T., Limbers C. A., Franciosi J. P., Pohl J. F. (2012). PedsQL™ Gastrointestinal Symptoms Module item development: Qualitative methods. Journal of Pediatric Gastroenterology and Nutrition, 54, 664–671. [DOI] [PubMed] [Google Scholar]
  45. Varni J. W., Lane M., Burwinkle T., Fontaine E., Youssef N., Schwimmer J., Pardee P. E., Pohl J. F., Easley D. (2006). Health-related quality of life in pediatric patients with irritable bowel syndrome: A comparative analysis. Journal of Developmental and Behavioral Pediatrics, 27, 451–458. [DOI] [PubMed] [Google Scholar]
  46. Varni J. W., Limbers C. A. (2009). The Pediatric Quality of Life Inventory™: Measuring pediatric health-related quality of life from the perspective of children and their parents. Pediatric Clinics of North America, 56, 843–863. [DOI] [PubMed] [Google Scholar]
  47. Varni J. W., Limbers C. A., Burwinkle T. M. (2007a). How young can children reliably and validly self-report their health-related quality of life?: An analysis of 8,591 children across age subgroups with the PedsQL™ 4.0 Generic Core Scales. Health and Quality of Life Outcomes, 5(1), 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Varni J. W., Limbers C. A., Burwinkle T. M. (2007b). Literature review: Health-related quality of life measurement in pediatric oncology: Hearing the voices of the children. Journal of Pediatric Psychology, 32, 1151–1163. [DOI] [PubMed] [Google Scholar]
  49. Varni J. W., Limbers C. A., Sorensen L. G., Neighbors K., Martz K., Bucuvalas J. C., Alonso E. M. (2011). PedsQL™ Cognitive Functioning Scale in pediatric liver transplant recipients: Feasibility, reliability and validity. Quality of Life Research, 20, 913–921. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Varni J. W., Seid M., Kurtin P. S. (2001). PedsQL™ 4.0: Reliability and validity of the Pediatric Quality of Life Inventory™ Version 4.0 Generic Core Scales in healthy and patient populations. Medical Care, 39, 800–812. [DOI] [PubMed] [Google Scholar]
  51. Varni J. W., Thompson K. L., Hanson V. (1987). The Varni/Thompson Pediatric Pain Questionnaire: I Chronic musculoskeletal pain in juvenile rheumatoid arthritis. Pain, 28, 27–38. [DOI] [PubMed] [Google Scholar]
  52. Williet N., Sandborn W. J., Peyrin–Biroulet L. (2014). Patient-reported outcomes as primary end points in clinical trials of inflammatory bowel disease. Clinical Gastroenterology and Hepatology, 12, 1246–1256. [DOI] [PubMed] [Google Scholar]
  53. Wyrwich K. W., Norquist J. M., Lenderking W. R., Acaster S. (2013). Methods for interpreting change over time in patient-reported outcome measures. Quality of Life Research, 22, 475–483. [DOI] [PubMed] [Google Scholar]
  54. Wyrwich K. W., Tierney W. M., Wolinsky F. D. (1999). Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. Journal of Clinical Epidemiology, 52, 861–873. [DOI] [PubMed] [Google Scholar]
  55. Youssef N. N., Langseder A. L., Verga B. J., Mones R. L., Rosh J. R. (2005). Chronic childhood constipation is associated with impaired quality of life: A case-controlled study. Journal of Pediatric Gastroenterology and Nutrition , 41, 56–60. [DOI] [PubMed] [Google Scholar]
  56. Youssef N. N., Murphy T. G., Langseder A. L., Rosh J. R. (2006). Quality of life for children with functional abdominal pain: A comparison study of patients' and parents' perceptions. Pediatrics, 117, 54–59. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES