Abstract
Objective
To examine the clinical validity of the Patient Reported Outcome Measurement Information System (PROMIS) Pediatric Sleep Disturbance (SD) and Sleep-Related Impairment (SRI) short forms.
Methods
Youth (8–17 years) from clinical populations with known SDs (sleep clinic n = 126, autism n = 276, asthma n = 82, asthma + eczema n = 68) and the general population (n = 902) completed the PROMIS Pediatric SD and SRI 8-item short forms, along with established measures of sleep (Children’s Report of Sleep Patterns, Sleep Habits Survey), PROMIS Pediatric Fatigue, and parent-reported clinical indicators (does child have sleep problem, use melatonin, use prescription sleep medication).
Results
Confirmatory factor analyses demonstrated factorial invariance for all clinical groups. Significant differences between the general population and clinical groups were found for SD and SRI (medium to large effect sizes). Convergent validity was demonstrated through separate hierarchical regression models that showed significant associations between parent-reported clinical indicators and SD and SRI, above and beyond clinical group, as well as moderate to strong correlations between the PROMIS sleep measures and both established measures of sleep and fatigue.
Conclusions
The PROMIS Pediatric SD and SRI short forms provide clinicians and researchers a brief, accurate, and valid way to measure patient-reported sleep outcomes in pediatric populations.
Keywords: chronic illness, developmental disorder, person-reported outcome, sleep disturbance, sleepiness
Sleep problems are common in children with chronic medical conditions or neurodevelopmental disorders, affecting up to 80% of these youth (Lewandowski, Ward, & Palermo, 2011; Singh & Zimmerman, 2015). There are a number of factors that may contribute to increased sleep problems in these populations. For example, during a hospitalization sleep schedules are highly disrupted due to noise, alarming machines, vital sign checks, and early morning rounds (Meltzer, Davis, & Mindell, 2012; Papaconstantinou, Hodnett, & Stremler, 2018). Disease symptoms, including pain and itching, also impact sleep, making it more difficult for children to fall asleep, as well as contributing to increased and prolonged night wakings (Allen, Graef, Ehrentraut, Tynes, & Crabtree, 2016; Boozalis, Grossberg, Puttgen, Cohen, & Kwatra, 2018). Children with autism and attention-deficit hyperactivity disorder (ADHD) are known to have difficulties with self-soothing and regulating at bedtime, as well as a potentially delayed melatonin onset or reduced melatonin, prolonging sleep onset latency (Singh & Zimmerman, 2015; Souders et al., 2017). Finally, youth with chronic illness may also be at increased risk for physiological sleep disorders, with studies finding higher rates of OSA in youth with asthma (Khassawneh, Tsai, & Meltzer, 2019), higher rates of restless legs syndrome in children with ADHD (Cortese et al., 2005), and increased periodic limb movements and abnormal sleep architecture in youth with juvenile fibromyalgia (Tayag-Kier et al., 2000).
While there are over 300 pediatric sleep questionnaires, identified weakness across measures include reliance on parent report and limitations in development methodology, including the lack of factor analytic methods or use of item-response theory (IRT) and the lack of direct input from youth and attention to content validity (e.g., meaningfulness and understandability of the items; Bevans et al., 2019; Forrest et al., 2018; Lewandowski, Toliver-Sokol, & Palermo, 2011; Spruyt & Gozal, 2011). In addition, although a number of pediatric sleep measures have been commonly used in populations of youth with chronic illnesses or neurodevelopmental disorders (Darwish & Abdel-Nabi, 2016; Jarasvaraparn et al., 2019; Lawless, Turner, LeFave, Koinis-Mitchell, & Fedele, 2018; Martin et al., 2018; Vandeleur et al., 2017), the clinical validity of subjective sleep measures in different populations has rarely been considered. Although sleep is a universal phenomenon, it is important to consider potential measurement differences that may result from pediatric chronic illness or neurodevelopmental disorders.
To address limitations in pediatric subjective sleep measures, the recently published Patient Reported Outcome Measurement Information System (PROMIS) Pediatric Sleep Disturbance (SD) and Sleep-Related Impairment (SRI) measures were developed (Bevans et al., 2019; Forrest et al., 2018). The PROMIS program was started by the National Institutes of Health to create brief, accurate, and valid patient-reported measures of health to be used by clinicians and researchers (Cella et al., 2007). PROMIS item-banks measure health domains (e.g., pain, fatigue) that are universal experiences and not illness specific. Over 40 adult and 20 pediatric (both child-report and parent-proxy versions) PROMIS item-banks are freely available for use in for both clinical and behavioral research studies, large-scale epidemiology surveys, and clinical practice (www.healthmeasures.net).
The PROMIS Pediatric SD and SRI item banks were developed using the rigorous PROMIS methodology. Specifically: (a) conceptual specification of the domain; (b) item pool construction; (c) content validation; (d) survey administration to a sample representative of the full range of severity for the domain; (e) psychometric evaluation using classical test and IRT approaches (including exploratory and confirmatory factor analysis); and (f) clinical evaluation of the final products (HealthMeasures, 2017). Our previous papers have clearly described the development and validation of these two sleep health item banks (Bevans et al., 2019; Forrest et al., 2018). However, these studies included a general population sample, thus questions remain about the validity of these new measures for use in clinical populations. Because youth with chronic illnesses and neurodevelopmental disorders are at increased risk for sleep problems, it is important to ensure subjective sleep measures are valid in different populations.
Thus, the purpose of the current study is to examine the construct validity and clinical utility of the PROMIS Pediatric SD and SRI short forms in different pediatric populations known to have sleep problems: sleep disorders, autism, and asthma (with and without eczema), compared to a general population sample of children and adolescents. We hypothesized that (a) SDs and SRI would be greater in the clinical populations compared to the general population sample; (b) parent-reported clinical indicators of poor sleep (i.e., sleep problem, use of melatonin for sleep, and use of prescription sleep medication) would be related to both self-reported SDs and SRI above and beyond clinical condition; and (c) convergent validity would be demonstrated with existing self- and parent- report measures of pediatric sleep and fatigue.
Methods
Participants
Study participants were 1,473 children ages 8–17 years (mean = 12.2, SD = 2.8) recruited from either a national online panel, outpatient clinics, or patient registries (each group described below). The full sample was 56.8% male, 78.3% White, 10.6% Black, 2.7% Asian, and 8.4% Multi-racial/Other, with 17.7% Hispanic. Table I provides demographic information for the full sample by clinical group. The clinical populations in this study were selected because of known differences in sleep compared to the general population.
Table I.
Demographic Characteristics by Clinical Group
| General population sample (n = 902) | Sleep clinic (n = 126) | Autism (n = 276) | Asthma (n = 82) | Asthma + Eczema (n = 68) | |
|---|---|---|---|---|---|
| Mean Age (SD) | 12.2 (2.8) | 11.8 (2.8) | 12.6 (2.8) | 11.9 (2.7) | 11.9 (2.6) |
| Gender | |||||
| % Male (n) | 51.3 (463) | 43.7 (55) | 78.6 (217) | 58.5 (48) | 58.8 (40) |
| % Female (n) | 48.7 (439) | 56.3 (71) | 21.4 (59) | 41.5 (34) | 41.2 (28) |
| Child race | |||||
| %White (n) | 79.7 (715) | 48.0 (59) | 86.8 (236) | 83.5 (66) | 72.3 (47) |
| % Black (n) | 8.9 (80) | 40.7 (50) | 4.8 (13) | 2.5 (2) | 12.3 (8) |
| % Asian (n) | 3.6 (32) | 3.3 (4) | 0.4 (1) | 2.5 (2) | 0 (0) |
| % Other/multi (n) | 7.8 (70) | 8.1 (10) | 8.1 (22) | 11.4 (9) | 15.4 (10) |
| Child ethnicity | |||||
| % Non-Hispanic | 80.2 (723) | 89.7 (113) | 89.9 (248) | 78.0 (64) | 72.1 (49) |
| % Hispanic | 19.8 (179) | 10.3 (13) | 10.1 (28) | 22.0 (18) | 27.9 (19) |
| Clinical indicators | |||||
| % Sleep problem (n) | 8.1 (73) | 88.1 (111) | 82.6 (238) | 57.3 (47) | 69.1 (47) |
| % Melatonin (n) | 6.2 (56) | 19.8 (25) | 40.6 (112) | 7.3 (6) | 11.8 (8) |
| % Sleep med (n) | 1.0 (9) | 6.3 (8) | 14.5 (40) | 3.7 (3) | 4.4 (4) |
General Population Sample
The general population sample (n = 902) was recruited from the GfK Knowledge Panel, an existing online panel of participants in the United States. As described elsewhere, sampling weights were used to render a final sample representative of the U.S. population of children in the 2015 Current Population Survey (Forrest et al., 2018).
Sleep Center
The Sleep Center sample (n = 126) was recruited from children seen in the Children’s Hospital of Philadelphia Sleep (CHOP) Sleep Center for the diagnosis and management of sleep disorders. One week prior to clinic appointments or overnight polysomnography, schedules were reviewed for eligible participants. Families were either recruited during their child’s scheduled appointment (with surveys complete using an iPad) or via an email sent from the study team. Because of their presentation to a sleep center for treatment, it was expected these youth would have greater severity of SDs and SRI compared to the general population.
Autism
Children with autism (n = 276) were recruited from two different online registries. autismMatch is an online registry run by the Center for Autism Research at the Children’s Hospital of Philadelphia. Coordinators of the autismMatch online panel identified potentially eligible families, and sent a total of two recruitment emails to the parent listed in the registry. The Interactive Autism Network (IAN) is an online national and international registry for families of children with autism. Potentially eligible families were identified by IAN personnel, and the CHOP study team contacted parents by email. There were no significant differences between the autismMatch (n = 96) and IAN (n = 180) samples in terms of child mean age (12.4 ± 2.8 vs. 12.6 ± 2.7 years), gender (82.3% vs. 76.7% male), race (84.2% vs. 88.1% white), or ethnicity (10.4% vs. 10.0% Hispanic). Further, no significant differences were found between groups on either the PROMIS Pediatric SD or SRI short forms, thus the groups were combined into a single autism group. Because up to 86% of children with autism have sleep issues (Robinson-Shelton & Malow, 2016; Souders et al., 2017), it was expected that these youth would have greater SDs and SRI compared to the general population.
Asthma (With and Without Eczema)
Patients seen in the outpatient clinics at National Jewish Health are provided the option to be contacted about future research studies, with those that agree entered in the NJH research database. Potential eligible participants were identified through the NJH research database by the NJH study team. Parents were sent up to two recruitment emails. The study sample included 82 youth with asthma, and 68 youth with asthma + eczema. Studies have demonstrated poorer sleep quality in youth with asthma, both with and without eczema (Koinis-Mitchell, Craig, Esteban, & Klein, 2012), thus we expected these youth to have more SDs and SRI compared to the general population sample.
Materials
PROMIS Pediatric SD and SRI 8-Item Short Forms
The 8-item PROMIS Pediatric SD short form assesses difficulties with sleep onset, sleep continuity, and sleep quality. The 8-item PROMIS Pediatric SRI short form assesses daytime sleepiness, sleep offset, and the impact of sleepiness on cognitive functioning, affect and behaviors, and daily activities. For both scales, raw to T-score conversions were established based on a large general population sample (Forrest et al., 2018). The general population mean is 50, the standard deviation is 10, with higher scores reflecting greater severity of SDs or SRIs. Concept elicitation with children, parents, and pediatric sleep experts, a theoretical concept base, and cognitive interviews of items established content validity (Bevans et al., 2019). Exploratory and confirmatory factor analyses confirmed a two-factor (SD and SRI) factor structure. Both measures were shown to be reliable (Cronbach’s alpha SD = 0.91, SR = 0.88), with high levels of precision across the full range of the latent variable. Convergent validity was demonstrated through correlations with existing measures of pediatric sleep health (Bevans et al., 2019; Forrest et al., 2018).
Children’s Sleep Habits Questionnaire
The Children’s Sleep Habits Questionnaire (CSHQ) was developed to assess parent-reported clinical symptoms of sleep problems. For this study the 9-item daytime sleepiness scale captured parent-reported sleep awakening time, difficulties waking in the morning, and daytime sleepiness (Owens, Spirito, & McGuinn, 2000). The CSHQ was completed by the autism, asthma, and asthma + eczema groups. Correlations for the sleep clinic group have been previously reported (Forrest et al., 2018).
Sleep Habits Survey
The School Sleep Habits Survey (SHS) is a self-reported measure of sleep patterns, including sleep habits and daytime sleepiness (Wolfson & Carskadon, 1998). For this study, the Sleep/Wake Problem Behavior Scale was used to assess daytime sleepiness, difficulty waking, and sleep quality. The SHS was completed by the autism, asthma, and asthma + eczema groups. Correlations for the sleep clinic group have been previously reported (Forrest et al., 2018).
PROMIS Pediatric Fatigue Measure
A 4-item subset of the PROMIS Pediatric Fatigue Short Form 10a items was used to asses fatigue. These items assessed both general fatigue, as well as fatigue impact. Raw scores are converted to T-scores (mean = 50, SD = 10), with higher scores reflecting greater fatigue (Lai et al., 2013). Content validity was determined using PROMIS standards, with reliability of 0.80 (Lai et al., 2013). The PROMIS Pediatric Fatigue was completed by all clinical groups. Correlations for the sleep clinic group have been previously reported (Forrest et al., 2018).
Parent-Reported Clinical Validation Items
Parents completed three items used for clinical validation: (a) do you think your child has a sleep problem?; (b) does your child currently take melatonin to help him/her sleep?; (c) does your child currently take any other medication to help him/her sleep? Parents provided the name of the “other medication.” The most common prescription medications were alpha-agonists (n = 26) and antihistamines (n = 15). It was expected that children with parent-reported sleep problems, were currently taking melatonin, or were currently taking a prescription medications for sleep would have greater child-reported SD and greater child-reported SRI.
Procedure
Parents/guardians of potential participants received an email explaining the study and providing a link to the survey. Parents first completed eligibility questions (child age 8–17 years; child able to read English; no child intellectual or developmental delay that would prevent completing questions about their own sleep). Once eligibility was determined, parents provided informed consent and completed questions about their child’s sleep. When complete, parents asked their children to complete surveys about their own sleep. Prior to answering questions, children completed an informed assent form. The study protocol was approved by the institutional review boards at Children’s Hospital of Philadelphia (CHOP IRB protocol numbers 15-012503 and 16-013083) and National Jewish Health (NJH IRB protocol number HS-3005).
To test construct validity and clinical utility of the PROMIS Pediatric SD and SRI short forms we (a) used confirmatory factor analysis (CFA) to ensure factor invariance across clinical groups, (b) compared sleep outcomes across clinical groups to examine validity for known group differences, and (c) tested convergent validity with parent-reported clinical indicators of poor sleep, as well as associations with both self- and parent-reported legacy sleep measures.
Data Analysis
We evaluated whether measurement of SD and SRI varied between the community sample and each clinical sample by fitting a series of increasingly restrictive multigroup CFA models to the data (Kline, 2015; van der Schoot, Lugtig, & How, 2012). Models were fit using the Lavaan package in R (Rosseel, 2012) and tested for multigroup invariance using semTools (Jorgensen, Pornprasertmanit, Schoemann, & Rosseel, 2018). Fit was considered adequate if CFI values were ≥ 0.90, and good if they were ≥ 0.95. Root Mean Square Error of Approximation (RMSEA) values ≤ 0.10 were considered marginal and ≤ 0.08 adequate (Browne & Cudeck, 1993; Hu & Bentler, 1999; van der Schoot et al., 2012). The first multigroup CFA models tested for configural invariance by allowing all factor loadings and item intercepts to freely vary for each group. We interpreted adequate multigroup model fit to indicate that the overall SD and SRI factor structure holds up similarly across subgroups. Next, we evaluated weak/metric invariance by testing for differences in fit between the baseline CFA models (with freely estimated loadings and intercepts) and models in which factor loadings are constrained to be equivalent across groups. We used ΔCFI < 0.01 as an indicator of measurement invariance, because this metric is independent of model complexity and sample size (Cheung & Rensvold, 2002; Meade, Johnson, & Braddy, 2008). Finally, we evaluated strong/scalar invariance by testing for differences in fit between the weak/metric invariance models and models in which item intercepts were constrained to be equivalent across groups. Again, we considered ΔCFI < 0.01 to indicate measurement invariance. Evidence of scalar invariance substantiates multi-group comparisons of factor means (e.g., t-tests or Analysis of Variance [ANOVA]), because it demonstrates that any statistically significant differences in group means are not due to variation in scale properties between groups.
ANOVA was conducted to compare the clinical groups’ scores on the SD and SRI short forms with the general population sample, with effect sizes (ES; Cohen’s d, 0.2 small ES, 0.5 medium ES, 0.8 large ES) used for post hoc group comparisons. Due to a violation of the homogeneity of variance assumption, the Welch’s statistic is reported. In addition, because of demographic differences between groups, Analysis of Covariance (ANCOVA) was conducted, controlling for participant age, gender, and race. Significance was set at p ≤ .01 because of multiple comparisons.
Two hierarchical regressions were used to examine the relationship between parent-reported clinical indicators and the PROMIS Pediatric SD and SRI short form T-scores. Clinical group was entered in the first level, parent-reported sleep problem in the second level, melatonin use in the third level, and prescription medication use in the fourth level. Significance was set at p ≤ .01.
Finally, Pearson correlation was used to assess associations between child-reported SD and SRI and other measures of sleep and fatigue by clinical group (<0.3 weak, 0.3 to < 0.7 moderate, ≥ 0.7 strong).
Results
Factor Structure
As shown in the Supplementary Table, the CFA model with unconstrainted factor loadings and intercepts adequately fit the data for all participants combined and for each subsample separately. In multigroup models (Table II), adequate fit of the configural models show that the overall SD and SRI factor structure is similar for the community sample and each clinical subsample. Small differences (ΔCFI < 0.01) in the fit of configural versus weak/metric and weak/metric versus strong/scalar models demonstrate between-group equivalence in factor loadings and intercepts, respectively.
Table II.
Factor Invariance Statistics by Group Compared to the General Population Sample (n = 902)
| Model 1: Configural | Model 2: Weak/metric | Model 3: Strong/scalar | |
|---|---|---|---|
| Sleep clinic (n = 126) | |||
| Chi-square | 1017.9 | 1074.3 | 1145.5 |
| Df | 206 | 220 | 234 |
| CFI | 0.912 | 0.908 | 0.902 |
| RMSEA | 0.096 | 0.095 | 0.095 |
| Δ Chi-square (df), p-value | N/A | 56.38 (14), p < .0000 | 71.20 (14), p < .0000 |
| Δ CFI | N/A | 0.005 | 0.006 |
| Δ RMSEA | N/A | 0.001 | 0 |
| Autism (n = 277) | |||
| Chi-square | 1144.6 | 1227.6 | 1283.4 |
| Df | 206 | 220 | 234 |
| CFI | 0.916 | 0.91 | 0.907 |
| RMSEA | 0.095 | 0.095 | 0.094 |
| Δ Chi-square (df), p-value | N/A | 82.92 (14), p < .0001 | 55.88 (14), p < .0001 |
| Δ CFI | N/A | 0.006 | 0.004 |
| Δ RMSEA | N/A | 0 | 0.001 |
| Asthma/AD (n = 150) | |||
| Chi-square | 957.26 | 989.84 | 1019.66 |
| Df | 206 | 220 | 234 |
| CFI | 0.92 | 0.918 | 0.916 |
| RMSEA | 0.091 | 0.089 | 0.087 |
| Δ Chi-square (df), p-value | N/A | 32.58 (14), p = .0033 | 29.82 (14), 0 = 0.0081 |
| Δ CFI | N/A | 0.002 | 0.002 |
| Δ RMSEA | N/A | 0.002 | 0.002 |
Comparison of Sleep in Clinical Groups and General Population Sample
Mean T-scores, percentiles, and ESs for each group and for both analyses are seen in Table III. A significant difference was found between groups for SD for both the Welch’s ANOVA, F(4, 241.0) = 60.9, p < .001, and ANCOVA, F(4, 1,428) = 66.0, p < .001. Compared to the general population sample, significantly greater SDs were reported by youth with autism or youth seen in sleep clinic (large ES), as well as youth with asthma + eczema (medium ES). A significant difference was also found between groups for SRI for both the Welch’s ANOVA, F(4, 240.8) = 26.6, p < .001, and ANCOVA, F(4, 1,250) = 30.2, p < .001, with significantly more daytime sleepiness and impairment reported by youth with autism, seen in sleep clinic, or with asthma + eczema (medium to large ES).
Table III.
Mean T-Scores, Percentiles, and Effect Sizes (Compared to General Population Sample) for the PROMIS Pediatric Sleep Short Forms by Clinical Group
| General population sample (n = 902) | Sleep clinic (n = 126) | Autism (n = 276) | Asthma (n = 82) | Eczema + Asthma (n = 68) | |
|---|---|---|---|---|---|
| Unadjusted ANOVA | |||||
| Sleep Disturbance | |||||
| Mean T (SD) | 49.9 (9.1) | 59.2 (8.8) | 58.4 (10.1) | 52.7 (8.2) | 55.0 (7.9) |
| Percentile | 50 | 82 | 79 | 62 | 70 |
| Cohen’s d effect size | – | 1.04 | 0.88 | 0.32 | 0.60 |
| Sleep-Related Impairment | |||||
| Mean T (SD) | 49.8 (9.0) | 57.3 (10.2) | 55.5 (10.5) | 51.6 (9.6) | 53.5 (8.2) |
| Percentile | 50 | 76 | 73 | 58 | 66 |
| Cohen’s d effect size | – | 0.78 | 0.58 | 0.19 | 0.43 |
| ANCOVA (estimated means controlling for age, gender, and race) | |||||
| Sleep Disturbance | |||||
| Mean T (SD) | 49.8 (9.2) | 59.0 (9.2) | 58.7 (9.4) | 53.1 (9.2) | 55.4 (9.2) |
| Percentile | 50 | 82 | 82 | 62 | 70 |
| Cohen’s d effect size | – | 1.00 | 0.96 | 0.35 | 0.61 |
| Sleep-Related Impairment | |||||
| Mean T (SD) | 49.8 (9.4) | 57.3 (9.4) | 55.9 (9.6) | 51.9 (9.4) | 53.7 (9.4) |
| Percentile | 50 | 76 | 73 | 58 | 66 |
| Cohen’s d effect size | – | 0.80 | 0.62 | 0.23 | 0.42 |
Sleep Outcomes and Parent-Reported Clinical Indicators
Multiple regression statistics are found in Table IV. For SD, clinical group (Model 1) contributed significantly to the regression model, F(4, 1,502) = 60.79, p < .001, accounting for 13.7% of the variation. Adding parent-reported sleep problem (Model 2) explained an additional 14.2% of the variation in SD, with a significant change in R2, F(5, 1,501) = 117.74, p < .001. Melatonin use (Model 3) explained an additional 1.7% of the variance, F(6, 1,500) = 106.56, p < .001, while prescription sleep medication use (Model 4) explained an additional 0.6% of the variance, F(7, 1,499) = 94.07, p < .001. When all independent variables were included in the final model, clinical groups were no longer associated with SD. While all three parent-reported clinical indicator items were significantly associated with SD, parent-reported sleep problem was the most important predictor of child-reported SD. Together all variables accounted for 30% of the variance in SD.
Table IV.
Summary of Hierarchical Regression Analysis for Variables Predicting PROMIS Pediatric Sleep Disturbance
| Model 1 |
Model 2 |
Model 3 |
Model 4 |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Β | SE Β | β | Β | SE Β | β | Β | SE Β | β | Β | SE Β | β | |
|
Sleep Disturbance
| ||||||||||||
| Sleep | 8.99 | 0.87 | .25*** | 0.83 | 0.93 | 0.02 | 0.94 | 0.92 | 0.03 | 0.92 | 0.91 | 0.03 |
| Autism | 8.23 | 0.63 | .32*** | 0.65 | 0.72 | 0.03 | −0.14 | 0.73 | −0.01 | −0.48 | 0.73 | −0.02 |
| Asthma | 2.53 | 1.06 | .06* | −2.35 | 1.01 | −.05* | −1.98 | 1.00 | −.05* | −1.98 | 1.00 | −.05* |
| Eczema + Asthma | 4.86 | 1.16 | .10*** | −1.28 | 1.12 | −0.03 | −0.99 | 1.10 | −0.02 | −0.99 | 1.1 | −0.02 |
| Sleep Problem | 10.67 | 0.62 | .52*** | 9.79 | 0.63 | .47*** | 9.56 | 0.63 | .46*** | |||
| Melatonin | 4.15 | 0.69 | .15*** | 3.99 | 0.68 | .14*** | ||||||
| Prescription medication | 4.09 | 1.11 | .08*** | |||||||||
| R2 | 0.14 | 0.28 | 0.30 | 0.30 | ||||||||
| Sleep-Related Impairment | ||||||||||||
| Sleep | 7.26 | 0.91 | .22*** | 1.69 | 1.02 | 0.05 | 1.73 | 1.06 | 0.05 | 1.73 | 1.01 | 0.05 |
| Autism | 5.41 | 0.67 | .22*** | −0.24 | 0.80 | −0.01 | −0.12 | 0.86 | −0.01 | −0.49 | 0.81 | −0.02 |
| Asthma | 1.50 | 1.10 | 0.04 | −1.80 | 1.10 | −0.04 | −1.63 | 1.12 | −0.04 | −1.62 | 1.10 | −0.04 |
| Eczema + Asthma | 3.44 | 1.20 | .08** | −.73 | 1.22 | −0.02 | −0.61 | 1.24 | −0.01 | −0.58 | 1.21 | −0.01 |
| Sleep problem | 7.40 | 0.69 | .37*** | 7.02 | 0.77 | .35*** | 6.72 | 0.71 | .33*** | |||
| Melatonin | 1.87 | 0.77 | .07* | 1.68 | 0.76 | 0.06 | ||||||
| Prescription medication | 4.91 | 1.21 | .11*** | |||||||||
| R2 | 0.07 | 0.15 | 0.15 | 0.16 | ||||||||
p ≤ .05; **p ≤ .01; ***p ≤ .001.
For SRI, clinical group (Model 1) contributed significantly to the regression model, F(4, 1,324) = 27.53, p < .001, accounting for 7.4% of the variation. Adding parent-reported sleep problem (Model 2) explained an additional 7.3% of the variation in SRI, with a significant change in R2, F(5, 1,323) = 46.77, p < .001. Melatonin use (Model 3) explained an additional 0.4% of the variance, F(6, 1,322) = 40.11, p < .001, while prescription sleep medication use (Model 4) explained an additional 1.0% of the variance, F(7, 1,321) = 37.15, p < .001. When all independent variables were included in the final model, clinical groups were no longer associated with SRI. While all three parent-reported clinical indicator items were significantly associated with SRI, parent-reported sleep problem was the most important predictor of child-reported SRI. Together all variables accounted for 16% of the variance in SRI.
Association with Child- and Parent-Report Legacy Measures
Pearson correlations are shown in Table V. Parent-reported sleepiness on the CSHQ was moderately associated with the SD and SRI short forms for both youth with autism and youth with asthma + eczema. Further, parent-reported sleepiness on the CSHQ and SRI were moderately correlated for youth with asthma. Child-reported sleep/wake behaviors on the SHS were moderately to strongly correlated with both the SD and SRI for autism, asthma, and asthma + eczema groups. Finally, the child-reported PROMIS Pediatric Fatigue was moderately associated with SD across clinical groups, with strong correlations between Fatigue and SRI for all groups.
Table V.
Correlations of the Child-Reported Sleep Disturbance and Sleep-Related Impairment Short Forms with Other Measures (all p < .001 Unless Otherwise Noted)
| Autism (n = 276) | Asthma (n = 82) | Asthma + eczema (n = 68) | |
|---|---|---|---|
| Sleep Disturbance | |||
| Parent-reported CSHQ | 0.38 | .26∗ | 0.42 |
| Child-reported SHS | 0.56 | 0.57 | 0.58 |
| Child-reported Fatigue | 0.50 | 0.59 | 0.44 |
| Sleep-Related Impairment | |||
| Parent-reported CSHQ | 0.37 | 0.39 | 0.40 |
| Child-reported SHS | 0.64 | 0.77 | 0.67 |
| Child-reported Fatigue | 0.80 | 0.80 | 0.75 |
Note. CSHQ = Children’s Sleep Habits Questionnaire; SHS = Sleep Habits Survey.
p = .018, ∗∗p = .11, ∗∗∗p = .007.
Discussion
Study results demonstrate the validity and clinical utility for the PROMIS Pediatric SD and SRI measures across different chronic medical illness and developmental disorders. Both of the PROMIS Pediatric Sleep short forms showed factorial invariance, suggesting that in these clinical populations the measures perform similar to the general population of youth. As hypothesized, child-reported SD and SRI scores were greater in the clinical populations, with the greatest impairment in the sleep clinic and autism groups. Higher scores among sleep clinic patients is not surprising, due to the nature of their clinical complaints, while sleep problems among youth with autism are well established (Singh & Zimmerman, 2015; Souders et al., 2017).
Our hypothesis that convergent validity would be demonstrated with parent-reported clinical indicators of poor sleep (i.e., parent-reported sleep problem, use of melatonin for sleep, and use of prescription sleep medication) was also supported, with these indicators related to both SD and SRI, above and beyond the clinical group differences. The strongest of these variables was the parent report that the child had a sleep problem. While this single question may provide a quick way to determine whether further assessment is necessary, alone it does not clearly define what that “sleep problem” may be. Further, as children get older, parents are not always aware of sleep disturbances, and unless there are visible signs of impairment, parents may also be less aware of daytime sleepiness and the subjective feeling of poor sleep quality (Brimeyer et al., 2016; Meltzer et al., 2013). Similarly, our hypothesis that convergent validity would be demonstrated through significant correlations of the SD and SRI short forms with both child- and parent-report legacy measures of sleep/wake problems, daytime sleepiness, and fatigue was supported. Thus, if there is a clinical concern for a “sleep problem,” or if a research study wants to capture either sleep disturbances or daytime sleepiness, the PROMIS Pediatric SD and SRI items provides clinicians and researchers a precise and valid way of capturing more information about the nature of the sleep problem (e.g., nocturnal sleep disturbance, daytime sleepiness) among youth with chronic illness or neurodevelopmental disorders.
Compared to existing measures, the PROMIS Pediatric SD and SRI short forms have a number of advantages for researchers and clinicians, including the rigorous development process, the focus on the child’s subjective experience of sleep and daytime functioning (vs. relying on parental report), and the ability to compare results to a representative general population sample. With a 7-day recall period these measures can also be used for repeated assessments to track changes following interventions that may impact sleep (e.g., behavioral sleep training, new medication to reduce itching and thus improve sleep). In addition, because there is significant overlap between these measures and the adult versions of the PROMIS SD and SRI (Buysse et al., 2010; Forrest et al., 2018), these measures can be utilized in longitudinal studies across the lifespan. Finally, with the integration of patient-reported outcomes into electronic health records, the PROMIS Pediatric SD and SRI could provide an easy way for clinicians to quickly identify SDs or daytime sleepiness that may require intervention, as well as to monitor sleep across development.
This study has both a number of strengths, but also limitations that provide guidance for future research in this area. First, a primary strength of this study is the comparison of different clinical pediatric populations to a general national population of youth. However, there may be limitations to the representativeness of the national sample (Forrest et al., 2018), and more research is needed to validate these measures in other populations of youth with known sleep issues (e.g., type 1 diabetes, epilepsy, and ADHD). Another strength of this study was the use of both child-report and parent-report measures. However, it is possible that some of the convergent validity may be due to the child reporting on both the PROMIS and other measures. In addition, we were unable to examine divergent/discriminant validity in this study, thus future studies should also include additional child-and parent-report questionnaires that would demonstrate divergent/discriminative validity, as well as objective data that captures sleep patterns and disturbances for multiple days (i.e., actigraphy). Finally, as this study was limited by the cross-sectional examination of sleep, further research is needed to examine the responsiveness of these new measures to clinical change over time.
The PROMIS Pediatric SD and SRI short forms provide brief (4-item and 8-item forms, and computerized adaptive tests are available), accurate, and valid assessments of sleep in youth with chronic medical illnesses and neurodevelopmental disorders. These measures address many of the shortcomings of the existing sleep questionnaires. For clinicians and researchers, the PROMIS Pediatric SD and SRI short forms provide a useful way to capture patient-reported sleep outcomes in youth ages 8–17 years.
Funding
This work was funded by the Patient-Centered Outcomes Research Institute (ME-1403-12211; PI Forrest) and the National Institutes of Health (R01 HL119441; PI Meltzer).
References
- Allen J. M., Graef D. M., Ehrentraut J. H., Tynes B. L., Crabtree V. M. (2016). Sleep and pain in pediatric illness: A conceptual review. CNS Neuroscience & Therapeutics, 22, 880–893. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bevans K. B., Meltzer L. J., De La Motte A., Kratchman A., Viel D., Forrest C.B. (2019). Qualitative development and content validation of the PROMIS Pediatric Sleep Health items. Behavioral Sleep Medicine, 17, 657–671. [DOI] [PubMed] [Google Scholar]
- Boozalis E., Grossberg A. L., Puttgen K. B., Cohen B. A., Kwatra S. G. (2018). Itching at night: A review on reducing nocturnal pruritus in children. Pediatric Dermatology, 35, 560–565. [DOI] [PubMed] [Google Scholar]
- Brimeyer C., Adams L., Zhu L., Srivastava D. K., Wise M., Hudson M. M., Crabtree V. M. (2016). Sleep complaints in survivors of pediatric brain tumors. Supportive Care in Cancer, 24, 23–31. [DOI] [PubMed] [Google Scholar]
- Browne M. W., Cudeck R. (1993). Alternative ways of assessing model fit In Bollen K. A., Long J. S. (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage. [Google Scholar]
- Buysse D. J., Yu L., Moul D. E., Germain A., Stover A., Dodds N. E., Pilkonis P.A. (2010). Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments. Sleep, 33, 781–792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cella D., Yount S., Rothrock N., Gershon R., Cook K., Reeve B., Rose M. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cheung G. W., Rensvold R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 255. [Google Scholar]
- Cortese S., Konofal E., Lecendreux M., Arnulf I., Mouren M. C., Darra F., Bernardina B. D. (2005). Restless legs syndrome and Attention-Deficit/Hyperactivity Disorder: A review of the literature. Sleep, 28, 1007–1013. [DOI] [PubMed] [Google Scholar]
- Darwish A. H., Abdel-Nabi H. (2016). Sleep disorders in children with chronic kidney disease. International Journal of Pediatrics and Adolescent Medicine, 3, 112–118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Forrest C. B., Meltzer L. J., Marcus C. L., De La Motte A., Kratchman A., Buysse D. J., Bevans K.B. (2018). Development and validation of the PROMIS Pediatric Sleep Disturbance and Sleep-Related Impairment item banks. Sleep, 41, zsy054.. [DOI] [PubMed] [Google Scholar]
- HealthMeasures. (2017). Measure development & research Retrieved from http://www.healthmeasures.net/explore-measurementsystems/promis/measure-development-research
- Hu L.‐t., Bentler P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. [Google Scholar]
- Jarasvaraparn C., Zlomke K., Vann N. C., Wang B., Crissinger K. D., Gremse D. A. (2019). The relationship between sleep disturbance and disease activity in pediatric patients with inflammatory bowel disease. Journal of Pediatric Gastroenterology and Nutrition, 68, 237–243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jorgensen T. D., Pornprasertmanit S., Schoemann A. M., Rosseel Y. (2018). Useful tools for structural equation modeling. R package version 0.5-1 Retrieved from https://CRAN.R-project.org/package=semTools
- Khassawneh B., Tsai S. C., Meltzer L. J. (2019). Polysomnographic characteristics of adolescents with asthma and low risk for sleep-disordered breathing. Sleep and Breathing, 23, 943–951. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kline R. B. (2015). Principles and practice of structural equation modeling. New York, NY: Guilford Press. [Google Scholar]
- Koinis-Mitchell D., Craig T., Esteban C. A., Klein R. B. (2012). Sleep and allergic disease: A summary of the literature and future directions for research. Journal of Allergy and Clinical Immunology, 130, 1275–1281. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lai J. S., Stucky B. D., Thissen D., Varni J. W., Dewitt E. M., Irwin D. E., DeWalt D. A. (2013). Development and psychometric properties of the PROMIS((R)) pediatric fatigue item banks. Quality of Life Research, 22, 2417–2427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lawless C., Turner E. M., LeFave E., Koinis-Mitchell D., Fedele D. A. (2018). Sleep hygiene in adolescents with asthma. Journal of Asthma, 1–9. doi: 10.1080/02770903.2018.1553049. [DOI] [PubMed] [Google Scholar]
- Lewandowski A. S., Toliver-Sokol M., Palermo T. M. (2011). Evidence-based review of subjective pediatric sleep measures. Journal of Pediatric Psychology, 36, 780–793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewandowski A. S., Ward T. M., Palermo T. M. (2011). Sleep problems in children and adolescents with common medical conditions. Pediatric Clinics of North America, 58, 699–713. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martin C. A., Hiscock H., Rinehart N., Heussler H. S., Hyde C., Fuller-Tyszkiewicz M., Sciberras E. (2018). Associations between sleep hygiene and sleep problems in adolescents with ADHD: A cross-sectional study. Journal of Attention Disorders, 108705471876251. doi:10.1177/1087054718762513 [DOI] [PubMed] [Google Scholar]
- Meade A. W., Johnson E. C., Braddy P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568–592. [DOI] [PubMed] [Google Scholar]
- Meltzer L. J., Avis K. T., Biggs S., Reynolds A. C., Crabtree V. M., Bevans K. B. (2013). The Children's Report of Sleep Patterns (CRSP): A self-report measure of sleep for school-aged children. Journal of Clinical Sleep Medicine, 9, 235–245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Meltzer L. J., Davis K. F., Mindell J. A. (2012). Patient and parent sleep in a children's hospital. Pediatric Nursing, 38, 64–71. [PubMed] [Google Scholar]
- Owens J. A., Spirito A., McGuinn M. (2000). The Children's Sleep Habits Questionnaire (CSHQ): Psychometric properties of a survey instrument for school-aged children. Sleep, 23, 1–1051. [PubMed] [Google Scholar]
- Papaconstantinou E. A., Hodnett E., Stremler R. (2018). A behavioral-educational intervention to promote pediatric sleep during hospitalization: A pilot randomized controlled trial. Behavioral Sleep Medicine, 16, 356–370. [DOI] [PubMed] [Google Scholar]
- Robinson-Shelton A., Malow B. A. (2016). Sleep disturbances in neurodevelopmental disorders. Current Psychiatry Reports, 18, 6.. [DOI] [PubMed] [Google Scholar]
- Rosseel Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1–36. [Google Scholar]
- Singh K., Zimmerman A. W. (2015). Sleep in autism spectrum disorder and attention deficit hyperactivity disorder. Seminars in Pediatric Neurology, 22, 113–125. [DOI] [PubMed] [Google Scholar]
- Souders M. C., Zavodny S., Eriksen W., Sinko R., Connell J., Kerns C., Into-Martin J. (2017). Sleep in children with autism spectrum disorder. Current Psychiatry Reports, 19, 34.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spruyt K., Gozal D. (2011). Pediatric sleep questionnaires as diagnostic or epidemiological tools: A review of currently available instruments. Sleep Medicine Review, 15, 19–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tayag-Kier C. E., Keenan G. F., Scalzi L. V., Schultz B., Elliott J., Zhao R.H., Arens R. (2000). Sleep and periodic limb movement in sleep in juvenile fibromyalgia. Pediatrics, 106, e70–e70. [DOI] [PubMed] [Google Scholar]
- van der Schoot R., Lugtig P., How J. (2012). A checklist for testing measurment invariance. European Journal of Developmental Psychology, 9, 486–492. [Google Scholar]
- Vandeleur M., Walter L. M., Armstrong D. S., Robinson P., Nixon G. M., Horne R. S. C. (2017). How well do children with cystic fibrosis sleep? An actigraphic and questionnaire-based study. Journal of Pediatrics, 182, 170–176. [DOI] [PubMed] [Google Scholar]
- Wolfson A. R., Carskadon M. A. (1998). Sleep schedules and daytime functioning in adolescents. Child Development, 69, 875–887. [PubMed] [Google Scholar]