Sleep and Sleep Disturbance in Children: Reliability and Validity of the Dutch Version of the Child Sleep Habits Questionnaire

Abstract

Study Objectives:

The Child Sleep Habits Questionnaire (CSHQ) was developed in the US for measuring medical and behavioral sleep disorders in school-aged children. This study was conducted to assess the reliability and structural validity of the Dutch version of the CSHQ.

Design:

Population-based study

Setting:

Questionnaires (n = 2385) were distributed to children in primary schools and daycare centers to be completed by the parent/guardian. An identical second questionnaire was distributed for test-retest and interobserver reliability, which were assessed using intraclass correlation, and compared with published data. Internal consistency was assessed by Cronbach α (per subscale). Validity was analyzed by confirmatory and exploratory factor analysis.

Participants:

School-aged children.

Interventions:

None.

Measurements & Results:

The questionnaire was returned by 1502 (63%) parents, 47% returned the questionnaire for test-retest, and 32% for interobserver reliability. Test-retest reliability was moderate to good, ranging from 0.47 to 0.93. Interobserver reliability was moderate to good, ranging from 0.53 to 0.87, with the exception of Sleep duration. Cronbach α ranged from 0.47 to 0.68. In confirmatory factor analysis the domain structure of the original American CSHQ could not be confirmed. Exploratory factor analysis suggested a 4-factor structure rather than the original 8 domains.

Conclusions:

The CSHQ seems to have an adequate reliability and moderate internal consistency in a Dutch population with different sociocultural characteristics than the US population in which it was devised. Factor analysis suggests that translation, cultural background, or subscales of the original instrument may affect the performance of the CSHQ.

Citation:

Waumans RC; Terwee CB; Van den Berg G; Knol DL; Van Litsenburg RRL; Gemke RJBJ. Sleep and sleep disturbance in children: reliability and validity of the Dutch version of the Child Sleep Habits Questionnaire. SLEEP 2010;33(6):841-845.

SLEEP PROBLEMS OCCUR AT ALL AGES AND MAY HAVE A SUBSTANTIAL IMPACT ON EVERYDAY FUNCTIONING. ACCORDING TO PREVIOUS STUDIES, 20% TO 40%^1–4 of children experience sleep problems. Sleep plays an important role in normal development and everyday functioning,⁵ and sleep problems can lead to significant morbidity, such as behavior problems,^6–7 depressive symptoms,⁸ more anxiety in adulthood,⁹ impaired neurobehavior functioning,¹⁰ poorer child-parent and peer relations, and hyperactivity.¹¹ Because of these potential serious effects, valid assessment of sleep habits in school-aged children is essential.

Many sleep studies in children have used different instruments, hampering a direct comparison of results. Also, psychometric properties (reliability, validity) of these instruments have not always been reported. To examine sleep in Dutch children, an internationally used and valid pediatric sleep instrument, the Child Sleep Habits Questionnaire (CSHQ), was translated. The CSHQ was developed in the US and has been used in several other countries, e.g., China and Italy.^2,12 Its main goal is to identify sleep problems and the need for further diagnostic investigation. The CSHQ was designed as a screening tool for sleep difficulties in children aged 4 to 10 years, based on the International Classification of Sleep Disorders pediatric diagnoses.¹³ Acceptable to good reliability and validity have been demonstrated in American and Chinese populations.^13–14

The aim of the present study was to translate the CSHQ into Dutch and to assess its reliability and structural validity in a large community-based sample of school-aged children, which differs in sociocultural background compared with the populations in which the CSHQ has been used.

METHODS

The CSHQ

The CSHQ is a 33-item questionnaire to be completed by a parent or guardian. It allows for a total score and 8 subscales or domains; Bedtime resistance (6 items), Sleep-onset delay (1 item), Sleep duration (3 items), Sleep anxiety (4 items), Night wakening (3 items), Parasomnias (7 items), Sleep disordered breathing (3 items), and Daytime sleepiness (8 items), see Table 1. Three additional questions gather information about evening bedtime, morning wake-up time, and total sleep duration. Parents are asked to report their child's average sleep behavior during the last typical week. Items are rated on a 3-point scale; usually (5 to 7 times per week), sometimes (2 to 4 times per week), and rarely (0 to 1 time per week). Parents can also indicate whether a particular sleep item is perceived as a problem by circling yes, no, or not applicable for each item. A higher CSHQ score indicates more sleep problems.

Table 1.

Subscales of the Child Sleep Habits Questionnaire¹³

Bedtime resistance	Sleep anxiety
Item 1 - Goes to bed at same time	Item 5 - Needs parent in room to sleep
Item 3 - Falls asleep in own bed	Item 7 - Afraid of sleeping in the dark
Item 4 - Falls asleep in other's bed	Item 8 - Afraid of sleeping alone
Item 5 - Needs parent in room to sleep	Item 21 - Trouble sleeping away
Item 6 - Struggles at bedtime
Item 8 - Afraid of sleeping alone
Sleep duration	Sleep disordered breathing
Item 9 - Sleeps too little	Item 18 - Snores loudly
Item 10 - Sleeps the right amount	Item 19 - Stops breathing
Item 11 - Sleeps same amount each day	Item 20 - Snorts and gasps
Parasomnias	Daytime sleepiness
Item 12 - Wets the bed at night	Item 26 - Wakes by himself
Item 13 - Talks during sleep	Item 27 - Wakes up in negative mood
Item 14 - Restless and moves a lot	Item 28 - Others wake child
Item 15 - Sleepwalks	Item 29 - Hard time getting out of bed
Item 17 - Grinds teeth during sleep	Item 30 - Takes long time to be alert
Item 22 - Awakens screaming, sweating	Item 31 - Seems tired
Item 23 - Alarmed by a scary dream	Item 32 - Watching TV
	Item 33 - Riding in car
Night wakenings	Sleep onset delay
Item 16 - Moves to other's bed in night	Item 2 - Falls asleep in 20 minutes
Item 24 - Awakes once during night
Item 25 - Awakes more than once

Translation

The CSHQ was translated to Dutch with permission of the author, based on international guidelines.¹⁵ Meticulous attention was paid to the translation process using well-accepted guidelines (e.g., by the MAPI institute that functions as a clearing house of official translations of similar instruments). Forward translations were made independently by 2 native Dutch and fluent English-speaking persons. Following consensus discussions, a single Dutch version was obtained for backward translation by an independent bilingual Dutch-English speaker. A small number of discrepancies with the original version were solved by consensus discussion with all 3 translators.

Participants and Procedures

Eleven urban and suburban primary schools and daycare centers participated in this study, which was conducted from December 2006 to April 2007. The CSHQ was distributed at the schools, together with written information on the study, a brief survey on demographic variables, and a stamped return envelope to be taken home by the children for their parents. The demographic survey included questions on age and sex of the child, health status and medication, cosleeping, family structure, and parental age, sex, and highest education. To increase the response rate, the study was announced in school magazines.

To assess test-retest reliability, a second CSHQ, to be completed by the same parent, was distributed to 159 children 2 weeks later. For assessment of interobserver reliability, 2 copies of the CSHQ were simultaneously distributed to 175 children to be completed independently by each parent. The aim was to gather 5% of total returned questionnaires for test-retest measurements and 5% for interobserver reliability.

The CSHQ was devised for children 4 to 10 years of age.¹³ To investigate whether the CSHQ could also be applied to an older age range, initially all children attending primary school, i.e., up to 12 years of age, were included.

Analysis

Analyses were done using the Statistical Program for Social Sciences (SPSS) for Windows, version 15. A P value of < 0.05 was considered significant.

The returned CSHQ questionnaires were checked for completeness and were excluded from analysis if more than 20% of data were missing. For the remaining missing values, the subject's overall mean of the other items in the same subscale were imputed. Imputation was necessary for subscale analysis in 0% to 8% of children. Internal consistency of the original subscales was assessed with Cronbach α coefficients. Test-retest and interobserver reliability were assessed using intraclass correlation coefficients (ICC), using a 2-way random-effects model. Values for Cronbach α between 0.70 and 0.90 and values for ICC above 0.70 are generally considered adequate.^16–17

Factor analysis was used to assess whether the questionnaire could be divided into the same dimensions as the original American CSHQ. Exploratory and confirmatory factor analyses for ordered categorical items were performed in Mplus using the method of weighted least squares with mean and variance adjustment (WLSMV). Since few questions were answered with rarely, the response categories rarely and sometimes were aggregated, leading to a dichotomized response (1. frequently, 2. sometimes/rarely). First, confirmatory factor analysis was performed on the total sample. A model was specified based on the factor structure of the original CSHQ.¹³ Because this model did not fit well, the dataset was randomly split in 2 parts: a training set and a validation set. Exploratory factor analysis was performed on the training set. Based on the exploratory analysis, a confirmatory factor model was built in which each item loaded on precisely 1 factor. Based on the modification indexes and the residual correlations, we further adjusted the model until a model was found that fit the data. This model was then tested in a confirmatory factor analysis on the validation set.The Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and the Root Mean Square Error of Approximation (RMSEA) were used as measures for model fit. A CFI and TLI of > 0.95 and a RMSEA of < 0.05 were considered as adequate fit. For a moderate fit, values > 0.90 and < 0.08 were used.

RESULTS

Preliminary Analyses

Preliminary analyses to investigate whether the CSHQ could also be applied to children older than 10 years of age showed that the results for 10- to 12-year-olds had inadequate validity and reliability compared with the 4- to 10-year-old age group. Cronbach α values were 0.57 (Bedtime resistance), 0.68 (Sleep duration), 0.53 (Sleep anxiety), 0.45 (Night wakenings), 0.48 (Parasomnias), 0.37 (Sleep disordered breathing). and 0.68 (Daytime sleepiness). Test-retest ICC correlations were 0.36 (Bedtime resistance), 0.92 (Sleep onset delay), 0.57 (Sleep duration), 0.36 (Sleep anxiety), 0.75 (Night wakenings), 0.60 (Parasomnias), 0.73 (Sleep disordered breathing), and 0.78 (Daytime sleepiness). There were not enough data for interobserver ICC analysis in this subgroup. Therefore, the results presented here will focus on the age group 4 to 10 years.

Basic Characteristics

In total, 2385 questionnaires were distributed, of which 1502 (63%) were returned. Test-retest and interobserver response rates were 47% and 32%, respectively. Since the study was limited to children aged 4 to 10 years, 1145 questionnaires were available for internal-consistency analysis. Final test-retest and interobserver analyses were performed on 34 and 44 children. Of the included children, 52% were boys; mean age was 7.5 years (SD 2.0). Most questionnaires were filled in by the mother (89%). Seven percent of the children used medication, and 5% had comorbidities, mostly attention-deficit/hyperactivity disorder, physical problems, or autism. Although sleep difficulties are more common in children with comorbidities,¹⁸ analyses were performed on the whole dataset to ensure applicability to a general population.

Internal Consistency

For the original 8 subscales, Cronbach α ranged from 0.47 to 0.68. Apart from Sleep disordered breathing (0.47), most αs were moderate (Bedtime resistance 0.68, Sleep duration 0.63, Sleep anxiety 0.54, Night wakenings 0.62, Parasomnias 0.54, Daytime sleepiness 0.67). See Table 2.

Table 2.

Cronbach α and ICC values per subscale in subjects aged 4-10 years

Subscale	Cronbach α (n)		Test-retest (n, % complete)		Interobserver (n)
	Dutcha	American13	Dutcha	American13	Dutch
Bedtime resistance	0.68 (1059)	0.70 (441)	0.83 (30, 97%)	0.68 (56)	0.87 (37, 97%)
Sleep onset delay	NA	NA	0.65 (34, 100%)	0.62 (60)	0.53 (40, 100%)
Sleep duration	0.63 (1106)	0.69 (459)	0.47 (31, 100%)	0.40 (60)	0.03 (36, 100%)
Sleep anxiety	0.54 (1030)	0.63 (432)	0.93 (29, 100%)	0.79 (56)	0.71 (36, 100%)
Night wakenings	0.62 (1008)	0.54 (437)	0.79 (29, 100%)	0.63 (56)	0.73 (32, 100%)
Parasomnias	0.54 (1014)	0.36 (425)	0.73 (29, 97%)	0.62 (57)	0.63 (36, 92%)
Sleep disordered breathing	0.47 (1037)	0.51 (439)	0.72 (29, 100%)	0.69 (58)	0.84 (34, 100%)
Daytime sleepiness	0.67 (1018)	0.65 (437)	0.79 (29, 97%)	0.65 (56)	0.54 (36, 100%)

^aDutch sample: Cronbach α total n = 1145, test-retest total n = 34, interobserver total n = 44. ICC refers to intraclass correlation coefficients; n: the number of questionnaires included for analysis; % complete: percentage of completely filled in questionnaires before imputation of missing values (as explained in Methods section). NA: not applicable, subscale consists of 1 item.

Test-retest Reliability

Based on the inclusion criteria of fewer than 20% missing values, fewer than 6 (15%) questionnaires were excluded from subscale analysis. Fewer than 2 (3%) of included questionnaires were partially completed and therefore imputed. Test-retest ICC correlations of the original 8 subscales ranged from 0.47 to 0.93, see Table 2. Apart from the low ICC value for Sleep duration (0.47), all other subscales values were moderate (Sleep onset delay 0.65) to good (Bedtime resistance 0.83, Sleep anxiety 0.93, Night wakenings 0.79, Parasomnias 0.73, Sleep disordered breathing 0.72, Daytime sleepiness 0.79).

Interobserver Reliability

Based on the inclusion criteria of fewer than 20% missing values, 4 to 12 (9%-27%) of questionnaires—depending on the subscale—were excluded from subscale analysis. Zero to 3 (0%-8%, again differing per subscale) of included questionnaires were partially completed and therefore imputed. As shown in Table 2, the original subscale ICCs ranged from 0.53 to 0.87, with the exception of Sleep duration (0.03). Subscale ICC scores for interobserver reliability were moderate (Sleep onset delay 0.53, Parasomnias 0.63, Daytime sleepiness 0.54) to good (Bedtime resistance 0.87, Sleep anxiety 0.71, Night wakenings 0.73, Sleep disordered breathing 0.84).

Factor Analysis

The original 8-factor model as proposed in the original CSHQ did not fit in the total sample of children aged 4 to 10 years. CFI was 0.825, TLI was 0.852, and RMSEA was 0.053. An exploratory factor analysis of data in the training set revealed 4 factors, based upon the scree plot. Furthermore, factor solutions with more than 4 factors were not interpretable. The model could be improved by allowing items 9, 25, and 31 to load on 2 factors and by deleting items 3, 18, 24 and 26.

The final confirmatory model in the training set had a CFI of 0.917, TLI was 0.930, and RMSEA was 0.035. Improvement of the fit indexes could be obtained by deleting more items, but the gain per deleted item was very small. We were therefore satisfied with a moderately fitting model. This model was subsequently tested in the validation set. The fit was again moderate: CFI was 0.900, TLI 0.913, and RMSEA 0.041. The modification indexes did not indicate any further improvements.

DISCUSSION

This study investigated the psychometric properties of the CSHQ in a Dutch population with different sociocultural characteristics than the population in which the CSHQ was developed. The CSHQ was originally devised as a screening tool. Based on our results, the Dutch version seems to be an acceptable instrument to identify sleep patterns and disturbances in school-aged children.

The results regarding internal consistency are in accordance with those from the American study (Cronbach α ranging from 0.36-0.70)¹³ and the Chinese study (Cronbach α ranging from 0.42-0.69).¹⁴ Although the Cronbach α values compare well with those of the original version, most were suboptimal according to Nunnally and Streiner.^16–17 This may be because the original subscales of the CSHQ were based on clinical entities instead of statistical analyses. Our findings on test-retest reliability (0.47-0.93) were at least comparable with American findings (0.40-0.79),¹³ since we used ICC, which is based on stricter assumptions than the Pearson correlations used in the American study. The Chinese study by Li et al. also used ICC to assess test-retest reliability and found comparable ICCs, ranging from 0.60 to 0.88 for the subscales.¹⁴ Interobserver reliability was not assessed in the American study. In the Chinese study, higher ICCs between mother and father were found: 0.89 for the overall questionnaire, ranging from 0.83 to 0.92 for the subscales.¹⁴ The differences in interobserver reliability could possibly be explained by cultural variances, such as more frequent cosleeping (48%²) in Chinese families, making parents more aware of their child's sleep, and the Chinese 1-child policy, which may lead to more focus on the child and better knowledge of the child's habits by both parents. This might explain a more uniform parental evaluation of sleep. Also, although parents in this study were explicitly instructed to complete the questionnaire independently of each other, the Chinese methodology might have been different, leading to different results. Because the study results were published in Chinese only, we did not have full access and were unable to interpret these differences.

In the interobserver analysis, the Sleep duration subscale had a very low ICC score (0.03). This subscale contains the following items: Sleeps too little (item 9), Sleeps the right amount (item 10), and Sleeps same amount each day (item 11). The low ICC value might be the result of a difference of opinion between parents about what the right amount of sleep is. In that case, it would simultaneously affect the answers to item 9 and item 10, leading to lower interobserver correlation values.

The preliminary analyses of children aged 10 to 12 years did not demonstrate sufficient validity and reliability for this group, suggesting that the CSHQ is not appropriate for use in this age group.

Exploratory and confirmatory factor analyses seem to suggest that the original 8-factor structure of the questionnaire, as proposed in the original CSHQ, does not fit the Dutch dataset. As far as we know, factor analysis was not performed on the American data. The factor analysis of the Chinese study showed a 3-factor structure, the 3 factors representing Bedtime behavior problems, Sleep disturbance, Sleep duration and Daytime sleepiness.¹⁴ Our results suggest that the Dutch version of the CSHQ would gain statistical strength by rearranging the items in a 4-factor structure and by omitting items 3, 18, 24, and 26. Items 9, 25, and 31 loaded on 2 factors. The eventual model fit reasonably: CFI was 0.900, TLI 0.913, and RMSEA 0.041. An explanation for the differences in factor structure could be that most children in our dataset had no (serious) sleep disorders. In the study by Owens et al., the Cronbach α coefficients of the entire scale were higher for the clinical sample (0.78) than for the community sample (0.68).¹³ The questionnaire might be a more suitable screening tool for a clinical group. This could explain the scarcity of the answers rarely and sometimes and the need for dichotomization, as explained in the methods section. It would be interesting to perform a factor analysis in a Dutch clinical group in a future study. Another explanation might be that the subscales of the CSHQ are statistically suboptimal, because they are based on clinical entities instead of statistical analyses.¹³ To investigate to what extent the classification of CSHQ subscales might be improved based on statistical results, it would be interesting to eliminate language and cultural influences by performing factor analysis in the American population using the original CSHQ.

Our study has several limitations. The response rate in our study was 63%. Response rates in previous studies are either not reported or vary widely (47%¹³; 65%¹⁹; 80%¹²; 92%²).The low return rate may have caused bias, although it has been argued that there is little empiric support for bias as a direct result of low response rates.²⁰ An overestimation as well as an underestimation of sleep difficulties is possible because the likelihood of participation increases if sleep problems are present, but participation was better in higher socioeconomic classes in which children are known to experience fewer difficulties sleeping.^21–22 Therefore, there does not seem to be a reason to presume a systematic bias. Also, since the study was limited to children aged 4 to 10 years, the number of questionnaires available for test-retest and interobserver reliability was smaller than intended. Finally, it is possible that translation or cultural background has affected the performance of the CSHQ. Although meticulous attention was paid to the translation process, we cannot exclude nor confirm that differences found by factor analysis may (in part) be due to the translation and cultural differences.

CONCLUSION

The translated version of the CSHQ seems to have an adequate reliability and moderate internal consistency in a population with different sociocultural characteristics than the American population in which the CSHQ was developed and validated. Using factor analysis, the proposed 8-domain structure of the original American CSHQ could not be confirmed in the Dutch dataset. We found a reasonably fitting 4-factor structure. Differences with the originally proposed factor structure could be due to translation, cultural differences, or lack of fit of the proposed clinic-based factor structure in the original data.

DISCLOSURE STATEMENT

This was not an industry supported study. The authors have indicated no financial conflicts of interest.

ABBREVIATIONS

CSHQ

Child Sleep Habits Questionnaire

ICC

intraclass correlation

WLSMV

Weighted Least Squares with Mean and Variance adjustment

CFI

Comparative Fit Index

TLI

Tucker-Lewis Index

RMSEA

Root Mean Square Error of Approximation