Sensitivity of the child behaviour checklist sleep items and convergent validity with the Sleep Disorders Scale for Children in a paediatric ADHD sample

Abstract

Objective

The Child Behavior Checklist (CBCL) is a commonly used measure of child and adolescent functioning, which includes seven items that can be aggregated to provide a purportedly valid measure of sleep functioning. The objective of this study was to examine the convergent validity of the CBCL in a paediatric ADHD population and to evaluate the sensitivity of the instrument when benchmarked against the Sleep Disorders Scale for Children (SDSC).

Methods

The parents of 215 individuals (ages 6–17 years, 86% male) completed the CBCL and SDSC as part of a battery of measured administered as part of a specialised ADHD service located in Perth, Western Australia. All participants had a diagnosis of ADHD confirmed by a paediatrician or psychiatrist prior to attending the service.

Results

The CBCL Sleep Composite Scale was strongly correlated with the SDSC, but reported below adequate internal reliability. Receiver Operating Characteristic (ROC) suggests that a cut-off score of 4 may have good diagnostic accuracy compared to SDSC.

Conclusions

The CBCL Sleep Composite Scale may be reasonable to use if no purpose-developed sleep screening tool is available. The CBCL sleep items demonstrated good convergent validity, however, did not otherwise demonstrate acceptable psychometric properties that would endorse its use in an ADHD sample. The development of a specific measure of sleep in children with ADHD children is recommended.

Highlights

• •The CBCL is strongly correlated with the SDSC in an ADHD sample.
• •The CBCL may be suitable to screen for sleep difficulties in ADHD children.
• •CBCL sleep items were poorly correlated with each other.
• •The CBCL may be able to accurately identify ADHD children with sleep difficulties.
• •There is no gold standard measure of sleep for children with ADHD.

1. Introduction

The Child Behavior Checklist (CBCL) for ages 6–18 years assesses the mental health and social functioning of children and adolescents [1]. This measure is well-validated and is one of the most common measures of psychopathology used by clinicians working in the paediatric setting [2]. A strength of the measure is the ability to obtain reliable and valid estimates of the client's functioning across a wide variety of domains (ie, ten empirically-based scales and six DSM-oriented scales) using 118 items each rated on three-point scale. The CBCL is routinely used to assess the functioning of children with neurodevelopmental disorders and can aid in the identification of co-occurring behaviour problems [3]. This includes children with Attention Deficit Hyperactivity Disorder (ADHD) – the most common childhood neurodevelopmental disorder – that is thought to affect 5–6% of all children [4].

The 118 items that form the CBCL have been shown to asses domains of functioning that exceed those domains specified in the original scoring and administration procedures of the measure. These documented processes often involve the grouping of specific items in a novel way that is based on shared similarities or descriptions and/or implementing modified scoring procedures [5,6]. One domain of functioning that may be able to be assessed using selected CBCL items is difficulties with sleep [7]. Several studies have used the CBCL to estimate a child's sleep difficulties [[7], [8], [9]]. The six items routinely used are: Nightmares (item 47), Overtired without good reason (item 54), sleeps less than most kids (Item 76), sleeps more than most kids during the day and/or at night (item 77), talks or walks in sleep (item 92), trouble sleeping (item 100). A seventh item – wets the bed (item 108) – is also included in some studies. These individual items each represent unique aspects of the sleep experience. Additionally, scores can be combined to yield an estimate of ‘overall’ sleep difficulties [7], as is common in other multidimensional screening measures of sleep difficulties [10]. These findings highlight the potential additional utility of the CBCL as a multidimensional measure of functioning – parents may be able to provide clinicians with a valid estimate of a child's overall sleep difficulties without having to take additional time to complete different measures. This may also help to address the underreporting of sleep difficulties among Australian children [11].

Validation of the CBCL as a screening measure for sleep difficulties was performed in a sample of 383 youth (aged 6–18 years; 96% of whom met diagnostic criteria for a sleep disorder) attending a paediatric sleep clinic for assessment of potential sleep disorders [7]. Becker et al. [7] had parents and children complete multiple measures of sleep-related functioning in order to examine convergence of the CBCL with purpose-built, previously-established screening instruments. These instruments were the Children's Sleep Habit Questionnaire (CSHQ) [12], the Sleep Disorders Inventory for Students (SDIS) [13], and the Adolescent Sleep-Wake Scale (ASWS; adolescent participants only) [14]. Findings revealed moderate correlations between CBCL ‘Sleep Composite’ scores and CSHQ total scores (r = 0.55), SDIS scores (r = 0.47), and ASWS scores (r = −0.39). Weak-to-strong correlations with Individual CBCL items and the subscales of these existing instruments were also reported (see Ref. [7]. Conclusions were that the measure had acceptable convergent validity with existing measures of sleep, though the CBCL may only be preferable in settings where clinicians are seeking retrospective assessment in samples where no measure of sleep was administered, or where measures of sleep cannot be introduced into assessment protocols [7]. Whilst these findings were initially examined in a youth sample seeking assessment for potential sleep disorders, the utility of the CBCL as a measure of sleep has not been explored in an ADHD context. This is a worthwhile area of investigation, as the CBCL is more commonly administered as part of routine ADHD assessment compared to sleep screening instruments.

Problems with sleep are frequently observed in children diagnosed with ADHD [15]. Up to 75% of ADHD children experience some difficulties relating to sleep [16]. This may include disruptions to the sleep–wake cycle, side-effects of stimulant medication, or behavioural issues resulting in challenging behaviours around bedtime (eg, bedtime refusal). Furthermore, emerging research has highlighted that the effective management of sleep difficulties in ADHD children can result in improved functioning and better outcomes for children [17]. Screening for sleep difficulties in children with ADHD has been recommended as part of routine initial assessment [18]. However, there remains no ‘gold standard’ screening measure of sleep for children and adolescents. This issue is especially pertinent for children with ADHD as many measures have been developed and validated in non-ADHD samples that may impact the accuracy of estimated sleep difficulties in ADHD children [19]. Existing sleep screening instruments are poorly validated in children with ADHD [7]. It is not clear whether these instruments can accurately assess sleep and related-issues in this unique clinical population. The validation of existing measures of sleep in the ADHD setting is of clinical significance as the misdiagnosis of sleep difficulties can adversely impact the functioning and wellbeing of the child. Failing to recognise sleep difficulties can perpetuate dysregulation in children with ADHD.

Parents of children with ADHD are frequently engaged in the process of ADHD diagnosis and management. It is also not uncommon to see the CBCL administered as part of initial intake to understand the functioning of the child with ADHD. Correspondingly, the number of studies that describe the administration of the CBCL in an ADHD population far exceeds the number of studies that describe the administration of sleep-specific measures in this same population. The widespread implementation of the CBCL has led to suggestion that this measure may be a viable instrument to screen for sleep difficulties that may already be collected as part of clinician's protocol for ADHD management, or in longitudinal studies where specific sleep screening instruments were not administered [7]. Evaluating the convergent validity of this measure against popular alternatives will help to inform whether a purpose-built sleep screening tool should be added into routine ADHD assessment, or if the existing CBCL offers similar benefits without the additional burden of additional assessment placed on consumers. A similar process has been undertaken in other clinical groups [7], but not yet in children with ADHD.

The objective of the present study was to validate the CBCL sleep items in a paediatric ADHD population to determine whether this routinely-administered instrument could serve as a suitable sleep screening instrument (and potentially reduce the burden of assessments completed by parents in the ADHD setting) or whether additional purpose-developed instruments, such as the Sleep Disorders Scale for Children [10], should be used instead. The three specific aims are to (a) evaluate the underlying factor structure and internal reliability of the CBCL sleep items in the ADHD population, (b) examine the convergent validity of the CBCL sleep items with the SDSC, and (c) provide preliminary evaluation of the sensitivity and the specificity of the CBCL sleep items using the clinically-significant and not clinically-significant categories of the SDSC.

2. Method

2.1. Participants

This sample comprised of 215 children and adolescents aged 6–17 years (185 males, 30 females; M = 10.21 years, SD = 2.70 years). All participants were recruited between 2014 and 2018 via a government-operated specialised ADHD assessment service located in Perth, Western Australia. This service is a referral-based service comprised of a multidisciplinary team providing assessment in the areas of clinical psychology, neuropsychology, occupational therapy, psychiatry, social work, and speech pathology. Families who access the service provide informed consent for assessment data to be used for research purposes at initial intake – the present study used all available participant data. All clients have a primary diagnosis of ADHD given by a paediatrician or psychiatrist prior to referral that is confirmed at intake. Clients are ineligible to access this service if ADHD is a secondary diagnosis to another diagnostic disorder (eg, Intellectual Disability, Autism Spectrum Disorder). Clients are often diagnosed as ADHD combined subtype (approximately 84%), followed by inattentive-only (approximately 14%), then hyperactive-only (approximately 2%). Based on the scoring procedure of the SDSC specified by Bruni et al. [10], 47.91% of the current sample (n = 103) were in the ‘clinically significant’ range for sleep difficulties.

2.2. Materials

The Child Behavior Checklist [1]. The CBCL is a 118-item guardian-rated checklist used to assess psychopathology of children aged 6–18 years. The guardian (typically the child's parent) responds to each item using a three-point scale ranging from 0 (Not True [as far as you know]) to 2 (Very True or Often True). The CBCL provides an estimate of many areas of a child's functioning over the previous six months. The present study extracts the seven items pertaining to sleep and related behaviours, as previously described by Becker et al. [7]. Standardised scores are not provided as this subscale is outside the scope of the original measure. Raw scores are summed to provide an overall CBCL Sleep Composite subscale ranging from 0 to 14, with higher scores indicating a higher level of sleep-related dysfunction. Internal reliability for this subscale in the current study was poor (α = 0.47). The validity of this subscale has been previously supported in a paediatric sleep-disorder sample by Becker et al. [7].

The Sleep Disorders Scale for Children [10]. The SDSC is a 26-item guardian-rated measure used to assess sleep difficulties in children over the previous six months. A five-point scale is used to record responses to each item. The first two items include unique scale points. Item 1 measures the child's average hours of sleep, from 1 (9 to 11 h) to 5 (less than 5 h) and Item 2 measures the average time to fall asleep from 1 (less than 15 min) to 5 (more than 60 min). The remaining items are rated from 1 (Never) to 5 (Always [Daily]). The SDSC is comprised of six individual sleep subscales which can also be combined to yield a total overall score. The measure also includes standardised scores based on data collected as part of the original development of the measure [10]. Scores can also be converted into t-scores and used to categorise children in the ‘normal’ range (t-score < 50), ‘borderline’ range (t-score 50–70) or ‘clinically significant’ range (t-score > 70). Two studies have yielded mixed findings regarding the validity of the measure in the neurodevelopmental population [18,19]. However, in the absence of a ‘gold standard’ screening instrument, the SDSC remains popular among clinicians seeking to assess sleep difficulties among clinical groups, including in children with ADHD. Internal reliability for the overall score in the current study was α = 0.87, with subscale scores ranging from α = 0.60 to α = 0.83.

2.3. Procedure

Approval to use this information was obtained from the Child Adolescent Health Service (CAHS) ethics committee. The information used in the current study is included as part of the battery of measures routinely completed by all clients who access the service. These measures are completed at initial intake into the service. No additional procedures beyond those included as part of routine clinical practice were required for the present study. Participating families provide consent for aggregated and de-identified data to be used for research and quality assurance purposes at intake to the ADHD service.

3. Results

3.1. Descriptive statistics and correlations

Descriptive statistics and correlations for measurement variables are presented in Table 1. Each of the SDSC subscales (and SDSC total) were significantly correlated with the seven-item total of the CBCL Sleep Composite items used in previous research. These correlations ranged from weak positive (r = 0.17) to strong positive (r = 0.66) by conventional standards [20]. The strongest correlation examined was between the total scores of each instrument – scores on the total SDSC and total CBCL Sleep Composite scale are strongly and positive correlated (r = 0.66, p < 0.001), suggesting that scores on one measure were positively associated with scores on the other. Individual CBCL items had variably sized correlations with SDSC subscales, ranging from weak negative (r = −0.15) to strong positive (r = 0.70).

Table 1.

Descriptive statistics, parametric (lower quadrant), and non-parametric (upper quadrant) correlations between measurement variables (N = 215).

	Descriptives			SDSC							CBCL
	M	SD	Range	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
SDSC
1. DIMS	18.41	6.93	7–35	–	0.31∗∗	0.33∗∗	0.39∗∗	0.45∗∗	0.09	0.84∗∗	0.17∗	0.07	0.57∗∗	-0.10	0.20∗∗	0.71∗∗	0.05	0.59∗∗
2. SBD	4.43	2.04	2–14	0.25∗∗	–	0.27∗∗	0.29∗∗	0.15∗	0.22∗∗	0.46∗∗	0.05	0.17∗	0.15∗	0.07	0.07	0.21∗∗	0.04	0.23∗∗
3. DOA	4.42	2.32	0–15	0.34∗∗	0.15∗	–	0.42∗∗	0.22∗∗	0.15∗	0.51∗∗	0.47∗∗	0.13	0.21∗∗	-0.14∗	0.39∗∗	0.20∗∗	0.03	0.36∗∗
4. SWTD	11.45	4.34	4–26	0.43∗∗	0.25∗∗	0.45∗∗	–	0.23∗∗	0.24∗∗	0.68∗∗	0.30∗∗	0.09	0.23∗∗	-0.02	0.48∗∗	0.27∗∗	0.15∗	0.44∗∗
5. DOES	9.34	3.96	4–21	0.46∗∗	0.11	0.20∗∗	0.27∗∗	–	0.15∗	0.63∗∗	0.14∗	0.36∗∗	0.20∗∗	0.24∗∗	0.15∗	0.33∗∗	0.07	0.46∗∗
6. SH	3.67	2.27	0–10	0.14∗	0.25∗∗	0.15∗	0.32∗∗	0.15∗	–	0.33∗∗	0.10	0.18∗∗	0.00	0.05	0.06	0.06	0.06	0.15∗
7. SDSC Total CBCL	51.73	14.60	27–104	0.84∗∗	0.42∗∗	0.55∗∗	0.73∗∗	0.64∗∗	0.41∗∗	–	0.29∗∗	0.24∗∗	0.48∗∗	0.01	0.34∗∗	0.60∗∗	0.11	0.68∗∗
8. Nightmares	0.59	0.68	0–2	0.20∗∗	0.07	0.53∗∗	0.33∗∗	0.15∗	0.10	0.34∗∗	–	0.08	0.17∗	-0.14∗	0.31∗∗	0.14∗	0.01	0.44∗∗
9. Overtired	0.46	0.67	0–2	0.05	0.17∗	0.08	0.08	0.34∗∗	0.14∗	0.20∗∗	0.05	–	0.06	0.23∗∗	0.02	0.11	0.03	0.43∗∗
10. Sleeps less	0.90	0.90	0–2	0.57∗∗	0.13	0.20∗∗	0.26∗∗	0.22∗∗	0.07	0.47∗∗	0.19∗∗	0.03	–	-0.18∗	0.11	0.64∗∗	0.11	0.68∗∗
11. Sleeps more	0.23	0.55	0–2	-0.08	0.05	-0.15∗	-0.02	0.22∗∗	-0.01	0.00	-0.14∗	0.31∗∗	0.18∗∗	–	0.06	-0.06	-0.07	0.16∗
12. Sleep Talks/Walks	0.70	0.87	0–2	0.23∗∗	0.02	0.49∗∗	0.46∗∗	0.16∗	0.08	0.38∗∗	0.36∗∗	0.02	0.10	0.07	–	0.13	0.04	0.45∗∗
13. Trouble Sleeping	1.06	0.87	0–2	0.70∗∗	0.18∗∗	0.19∗∗	0.29∗∗	0.33∗∗	0.10	0.58∗∗	0.16∗	0.10	0.64∗∗	-0.08	0.14∗	–	0.09	0.71∗∗
14. Wets Bed	0.33	0.65	0–2	0.09	0.06	0.04	0.14∗	0.08	0.10	0.13	0.00	0.05	0.1	-0.05	0.05	0.10	–	0.32∗∗
15. CBCL Total	4.07	2.49	0–11	0.59∗∗	0.21∗∗	0.42∗∗	0.47∗∗	0.44∗∗	0.17∗	0.66∗∗	0.48∗∗	0.42∗∗	0.66∗∗	0.18∗∗	0.50∗∗	0.70∗∗	0.36∗∗	–

Note. ∗p < 0.05. ∗∗p < 0.001.

3.2. Exploratory factor analysis and internal reliability of the CBCL Sleep Composite

Principal axis factoring using a promax rotation was used to explore the underlying factor structure of the CBCL Sleep Composite subscale. Initial extraction using the Kaiser-criterion (Eigenvalues ≥ 1.00 are retained as factors) revealed that three factors were extracted. Each factor was driven by two items each, with the seventh item “Wets the bed” failing to have substantial item loadings onto any factor. These factors explained a combined 41.75% of the variance in the current sample data. The pattern matrix of item loadings is displayed in Table 2. Factor 1 explained approximately half of this variance in the current sample data – this factor also perhaps best represented sleep difficulties relating to initiating and maintaining sleep (including two items: (“Sleeps less than most kids during the day and/or at night” and “Trouble sleeping”). The second extracted factor included two items relating to indicators of hyper-somnolence. The last factor included two items relating to disordered sleep arousal, namely nightmares and sleep talking/walking. The names of these factors were further supported by correlations identified in Table 1. For example, the CBCL items in Factor 1 were most strongly correlated with the Disorders Initiating and Maintaining Sleep subscale of the SDSC, demonstrating convergent validity at the item-subscale level (see Table 2).

Table 2.

Item Loadings for a Promax-Rotated Factor Matrix of the CBCL Sleep Composite Subscale using the Kaiser-Criterion (N = 215).

CBCL Item	Factor 1a	Factor 2b	Factor 3c	r with Corresponding SDSC Factor
76. Sleeps less than most kids during the day and/or at night	0.81			0.57∗∗d
100. Trouble sleeping	0.80			0.70∗∗d
108. Wets the bed	–	–	–	–
77. Sleeps more than most kids during the day and/or at night		0.81		0.22∗∗e
54. Overtired without good reason		0.40		0.34∗∗e
47. Nightmares			0.70	0.53∗∗f
92. Talks or walks in sleep			0.52	0.49∗∗f
% of Variance	21.30%	11.64%	8.81%
Cronbach's α	0.78	0.47	0.52

Note. Item loadings < 0.30 are suppressed.

∗∗p < 0.001.

^aFactor 1 = Initiating and Maintaining Sleep.

^bFactor 2 = Hyper-somnolence.

^cFactor 3 = Disordered Sleep Arousal.

^dCorresponding SDSC Subscale = Disorders Initiating and Maintaining Sleep.

^eCorresponding SDSC Subscale = Disorders of Excessive Somnolence.

^fCorresponding SDSC Subscale = Disorders of Arousal.

Previous studies have combined responses on the CBCL Sleep Composite subscale to yield an indication of ‘overall’ sleep difficulties. Reliability analysis using Cronbach's alpha was used to identify if this previously documented process would be supported using the current data. The initial Cronbach's alpha statistic was 0.47. This was substantially below the typically acceptable lower limit of α ≥ 0.70. This finding suggests that responses to these specific-items are poorly correlated and therefore not reflective of the same underlying construct (ie, ‘overall’ sleep difficulties). Levels of internal reliability did not substantially improve with the removal of any item, including ‘Wets the bed’ (item 108) which did not load strongly onto any of three identified factors (α = 0.48). However, this item was retained as internal reliability statistics did not improve with the removal of this item, and to remain consistent with past studies that include this item.

Table 2 presents internal reliability statistics for each of the three factors. The first factor demonstrated acceptable internal reliability whilst the remaining two factors demonstrated below adequate levels of internal reliability. However, findings should be interpreted with caution as subscales are conventionally recommended to include three or more items to reduce estimation problems [21]. Analyses of the current data did not yield clear direction for how CBCL Sleep Composite subscale items should be combined to provide ‘overall’ scores. It is for this reason that the item that did not load onto any of the three identified factors was retained for subsequent analyses. The internal factor structure of the CBCL sleep items presents as an unreliable construct regardless of the factor analysis or rotation method used or items included or excluded. Despite the below-adequate internal reliability reported, an ‘overall’ score was still calculated to remain consistent with previous research using the CBCL Sleep Composite subscale items.

3.3. Receiver Operating Characteristic (ROC) analysis

ROC analysis was performed in order to evaluate the overall utility of the CBCL at classifying ADHD children with clinically significant sleep difficulties and to evaluate possible efficacy of a clinically useful ‘cut-off’ score should the CBCL Sleep Composite subscale score be used to classify children with clinically significant sleep difficulties. Clinical significant sleep difficulties were operationalised according the SDSC scoring protocol (t-scores > 70 indicative of clinical significant difficulties).

The area under the ROC curve (AUC) was 0.83, 95% CI [0.77, 0.88], and is depicted in Fig. 1. This highlights that the CBCL Sleep Composite subscale score has good utility in being able to identify children with clinically significant sleep difficulties as quantified using the SDSC.

Fig. 1.

Receiver Operating Characteristic (ROC) Curve depicting the sensitivity and specificity of the Child Behavior Checklist seven-item Sleep Subscale in identifying children experiencing clinically-significant sleep difficulties as scored on the Sleep Disorders Scale for Children (SDSC). Area Under Curve (AUC) = 0.83 (95% CI [0.77, 0.88]).

3.4. Discriminative accuracy

Additional analyses were performed to examine whether the CBCL Sleep Composite subscale could be dichotomized to yield dichotomous indication of sleep difficulty risk (ie, ‘at-risk’ and ‘not at-risk’. Scores on the SDSC were first used to dichotomize the current sample into those that were reported to experience clinically-significant (ie, t-score > 70) and non-clinically significant (ie, t-score ≤ 70) sleep difficulties. In the absence of a ‘gold standard’ screening instrument for sleep difficulties, this SDSC dichotomy was used as a measure to benchmark the potential diagnostic test evaluation of different ‘cut-off’ scores on the CBCL Sleep Composite subscale. Scores on the CBCL Sleep Composite Subscale range from 0 to 14, providing a number of ‘cut-points’ at which a child may be classified as at-risk for clinically significant sleep difficulties. The most suitable ‘cut-point’ was derived using the well-established assessment of diagnostic test accuracy metrics [22]. Specifically, we prioritised cut-scores that maximised overall test accuracy. Test sensitivity was prioritised over test specificity. This allows for more children with clinically-significant sleep difficulties to be identified, but at the expense of an inflated rate of ‘false-positive’ cases (ie, those children who are mistakenly identified as being in the clinically-significant range for sleep difficulties). A screening measure that prioritises measure sensitivity will ensure greater likelihood that those who need treatment will be identified, at the concession of an inflated rate of false positives.

The results of the ROC curve analysis provided the sensitivity and 1-specificity values for several cut-off scores. Four potential cut-off scores were identified as being potentially suitable based on these metrics. These were CBCL Sleep Composite subscale scores of 2, 3, 4 and 5. Follow-up metrics were calculated using the MedCalc Diagnostic Test Evaluation Calculator [23]. These results are presented in Table 3. Based on the various test evaluation statistics, the ‘cut-off’ score of equal to or greater than 4 was identified as being the most suitable. This ‘cut-off’ had higher accuracy statistics than the other three measures, and reported adequate specificity characteristics whilst maintaining good levels of test specificity. The result of which is a test that provides a balance of higher accuracy and minimises false-negative results.

Table 3.

Diagnostic Test Evaluation of the Child Behavior Checklist (CBCL) Sleep Composite Subscale Scores using Various Cut-Off Scores to Identify Children with ‘Clinically Significant’ Sleep Difficulties as Measured by the Sleep Disorders Scale for Children (SDSC) (N = 215).

Statistic	Description	CBCL Sleep Composite Subscale ‘Cut-Off’ Score
		2	3	4	5
Sensitivity (95% CI)	Probability that the CBCL Sleep Composite subscale score will identify a child ‘at-risk’ of sleep difficulties when they are in the ‘clinically significant’ range on the SDSC.	95.13 (89.03%–98.41%)	92.23% (85.27%–96.59%)	87.38% (79.38%–93.11%)	67.96% (58.04%–76.825)
Specificity (95% CI)	Probability that the CBCL Sleep Composite subscale score will identify a child ‘not at-risk’ of sleep difficulties when they are in the ‘not clinically significant’ range on the SDSC.	33.04% (24.44%–42.56%)	49.11% (39.54%–58.73%)	66.96% (57.44%–75.56%)	81.25% (72.78%–88.00%)
Positive Predictive Value Sensitivity (95% CI)	Probability that the child is in the ‘clinically significant’ range on the SDSC when the CBCL Sleep Composite subscale identifies them as ‘at-risk’.	56.65% (53.25%–59.98%)	62.50% (57.94%–66.85%)	70.87% (64.91%–76.18%)	76.92% (68.92%–83.36%)
Negative Predictive Value (95% CI)	Probability that the child is in the ‘not clinically significant’ range on the SDSC when the CBCL Sleep Composite subscale identifies them as ‘not at-risk’.	88.10% (75.15%–94.77%)	87.30% (77.49%–93.21%)	85.23% (77.35%–90.70%)	73.39% (67.25%–78.74%)
Accuracy (95% CI)	Overall probability that the CBCL Sleep Composite subscale will correctly classify the child according to the SDSC categories.	62.79% (55.96%–69.27%)	69.77% (63.15%–75.83%)	76.74% (70.52%–82.22%)	74.88% (68.53%–80.53%)
Diagnostic Odds Ratio	The positive likelihood ratio divided by the negative likelihood ratio. Values exceeding 10.00 indicate test suitability.	9.47	11.31	13.89	9.28
True Positive N	Children ‘at-risk’ on the SDSC Sleep Composite subscale who are also in the ‘clinically significant’ range on the SDSC.	98	95	90	70
True Negative N	Children ‘not at-risk’ on the SDSC Sleep Composite subscale who are also in the ‘not clinically significant’ range on the SDSC	37	55	75	91
False Positive N	Children ‘at-risk’ on the SDSC Sleep Composite subscale who are in the ‘not clinically significant’ range on the SDSC	75	57	37	21
False Negative N	Children ‘not at-risk’ on the SDSC Sleep Composite subscale who are actually in the ‘clinically significant’ range on the SDSC	5	8	13	33

4. Discussion

The aim of this study was to provide a validation of the Sleep Composite Subscale of the CBCL in a sample of children with ADHD. The utility of this measure has been previously identified in non-ADHD samples [[7], [8], [9]]. However, the expression of sleep difficulties in paediatric ADHD samples has highlighted the need for validation studies that can justify the implementation of specific sleep screening instruments in this unique clinical population [19]. Specific objectives that allow the aim of this study to be met were to: (a) examine the underlying factor structure and internal reliability of the CBCL Sleep Composite subscale, (b) assess the convergent validity of the CBCL Sleep Composite subscale against the widely-used SDSC, and (c) report on the ability of the CBCL Sleep Composite subscale to categorise children ‘at-risk’ and ‘not at-risk’ of clinically significant sleep difficulties. These results help to address the need for sleep screening instruments validated for use in the ADHD setting.

The first critical finding in this study was the strong positive correlation between the CBCL Sleep Composite subscale and the SDSC. This correlation is comparable to correlations previously examined between the CBCL Sleep Composite subscale and three other sleep screening instruments: the CSHQ, the SDIS, and the ASWS. Becker et al. [7] found effect sizes whereby the CBCL accounted for 30.25%, 22.09%, and 15.21% of the variability in these instruments, respectively. The present effect size estimate was slightly higher, with 43.56% of the variability in children's SDSC scores being predicted by CBCL Sleep Composite subscale scores. Our findings add to existing literature that suggests the CBCL Sleep Composite subscale score has good convergent validity with existing sleep instruments.

Some limitations regarding the use of the CBCL as a measure of sleep were also provided given the substantial proportion of variability remaining unaccounted for by this instrument. Specifically, the efficacy of the CBCL Sleep Composite subscale may be best utilised when purpose-built measures of sleep cannot be implemented [7]. For example, when working with already-collected data or when purpose-built sleep screening instruments cannot be introduced for practical reasons. The present results may warrant a similar conclusion – CBCL Sleep Composite scores are highly correlated with the SDSC, but do not completely explain variability in these scores. The currently available evidence limits the extent to which the CBCL Sleep Composite subscale can be considered an equal alternative to purpose-built sleep screening instruments, and instead be best thought of as a reasonable alternative when logistical reasons prohibit other instruments being implemented.

The CBCL Sleep Composite Subscale contains seven items that relate to different dimensions of sleep yet have been previously totalled to provide an approximate indication of a child's ‘overall’ sleep difficulties [7]. Our study provided partial support for this, given the strong correlation between SDSC total scores and CBCL Sleep Composite subscale scores. Unlike these previous studies that had treated the CBCL Sleep Composite subscale scores as a single score, the present study provided a preliminary test of the underlying factor structure of this seven-item scale. Findings revealed that this instrument may actually assess three dimensions of sleep. Based on the correlations between CBCL items and the SDSC subscales, and the content in each of these CBCL items, these three dimensions bare resemblance to three of the SDSC subscales: Disorders Initiating and Maintaining Sleep, Disorders of Excessive Somnolence and Disorders of Arousal. Comparable to the SDSC, the CBCL Sleep Composite subscale currently takes different dimensions of sleep and then aggregates responses across items into a single score. However, unlike the SDSC, the CBCL Sleep Composite subscale reported well below adequate levels of internal reliability. This suggests that the aggregation of CBCL Sleep Subscale scores into a single overall score may overlook specific areas of sleep difficulty. However, calculating three factor scores may not be suitable given that each factor only contains two items [21]. It is therefore recommended that additional consideration be given to the underlying factor structure of the CBCL in future research, should specific aspects of sleep be of clinical interest.

4.1. Cut-off scores

Any effective screening instrument should be able to identify individuals with (or at-risk of) a specified outcome of interest with accuracy. Previous studies using the CBCL Sleep Composite subscale have used this measure as a continuous indicator of sleep difficulties, with higher scores indicative of greater disruption. This continuous measure of functioning has also been correlated with other continuous measures of sleep difficulties to establish convergent validity [7]. The present findings provide additional preliminary evidence that suggests the CBCL may be used to classify the risk of sleep difficulties in children with ADHD, when using the SDSC ‘clinically significant’ and ‘not-clinically significant’ categories as an outcome measure. It is however to emphasise the exploratory nature of this research as the SDSC has been used as a substitute in the absence of an established gold standard measure of sleep for children with ADHD – there is preliminary validation evidence for the SDSC in children with ADHD [19]. Our results identified that a ‘cut-off’ score equal to or greater than 4 yielded optimal test evaluation metrics based on conventional metrics used to assess diagnostic test accuracy. Using this cut-off score of 4 resulted in the CBCL Sleep Composite subscale being able to correctly identify children in the ‘clinically significant’ and ‘not-clinically significant’ categories of the SDSC 76.74% of the time – this was the highest accuracy of any of the other cut-points. Further, this cut-off score yielded good sensitivity statistics without substantially compromising measure specificity.

Correctly identifying children at-risk of sleep difficulties (ie sensitivity) was prioritised over the ability to correctly identifying children not at-risk of sleep difficulties (ie, specificity) due to the clinical application of brief screening instruments. This is due to the impact of unrecognised problems on activities of daily living and ADHD symptomology. Having a screening measure that is high in sensitivity will increase the likelihood that at-risk children will be correctly identified, that can then be verified by a more comprehensive diagnostic sleep assessment using more comprehensive measures (eg, polysomnography). This would then allow children and families to be directed to appropriate intervention and management strategies. The trade-off of measures high in sensitivity tends to be a greater rate of false-positive cases (ie, children who are flagged at ‘at-risk’ of sleep difficulties despite the absence of any sleep disorder). However, any false-positives classified by the screening instrument should be correctly classified by the more comprehensive ‘gold-standard’ measure. Our findings suggest that the CBCL Sleep Composite subscale could be routinely collected at intake and used to identify any children at possible risk of sleep difficulties.

However, these initial findings should be considered preliminary and interpreted with caution. These findings are yet to be validated in other samples, and should also consider the diagnostic accuracy of the CBCL Sleep Composite scale items against established ‘gold-standard’ instruments, once a gold standard measure has been established. The present research employed the SDSC as a substitute reference measure due to preliminary evidence demonstrating that the measure may be suitable as an estimate of total sleep difficulties in an ADHD sample [19]. Further work to establish a gold standard measure is critical in enabling accurate identification of sleep difficulties in children with ADHD and to validate proposed screening instruments.

4.2. Strengths and limitations

The current study is the first to evaluate the CBCL Sleep Composite subscale in a large sample of children with ADHD. This research also forms part of a small body of emerging literature that validates the suitability of existing sleep screening instruments for use in ADHD populations. Research in this area is vital, as the growing demand for clinicians to identify and incorporate the management of sleep difficulties as part of best-practice ADHD management drives the need for efficient, valid, and reliable measures of sleep. The good convergent validity of the seven-item CBCL Sleep Composite Subscale observed in our study is comparable to what has been previously identified in non-ADHD samples [7]. These findings provide preliminary support to suggest that the CBCL Sleep Composite subscale items can be used for similar purposes in an ADHD context. In line with Becker et al. [7], this instrument is particularly well-suited to longitudinal studies making use of historical data and when other sleep screening measures cannot be introduced into settings where the CBCL is already administered.

There are several limitations that must be considered when interpreting the study findings. First, the absence of a control group means that it cannot be stated with certainty that children with ADHD score higher on the CBCL Sleep Composite subscale compared to typically developing children – this would be expected considering the much higher prevalence of sleep difficulties in the ADHD population [16]. Second, sample size restricted the ability to stratify our sample by ADHD subtype or comorbidity – we do not currently know whether sleep difficulties are more common in ADHD subtypes, or when specific comorbidities are present. Certain comorbidities co-occurring with ADHD (eg, anxiety, autism spectrum disorder) may lead to greater disruption to sleep as these disorders independently impact sleep difficulties [[24], [25], [26]]. Common comorbidities present in this population were motor problems, emotional problems and learning difficulties and largely consistent with the other paediatric ADHD samples [27] – however these comorbidities were secondary to the primary ADHD diagnosis. Future research could quantify the severity of these comorbidities to establish unique impact on sleep. Third, all children were actively engaged in ADHD management with their treating physician external to the specialised assessment service where the present data was obtained. As a result, information about course of medical management and compliance was not available for use in research. Consequently, the potential confounding impact of medication on children's sleep cannot be established in the current research. Finally, the SDSC is only one of several screening instruments used to assess sleep difficulties in children. It is not yet clear whether similar results would be obtained should another sleep screening instrument, such as the Children's Sleep Habits Questionnaire [12] was used in place of the SDSC. However, in the absence of a ‘gold standard’ screening instrument, we used the SDSC – one of the most commonly used screening instruments for sleep difficulties - as a benchmark for convergent validity and to also assess the discriminative accuracy of the CBCL Sleep Composite subscale. This measure has recently been validated for use in an ADHD population – the preliminary findings suggest that total scores on the SDSC generated in an ADHD population may be appropriate for use as an indication of overall sleep problems [19]. However, further work in establishing a gold standard measure of sleep for children with ADHD is required.

5. Conclusion

The prevalence and impact of sleep difficulties in children with ADHD has led to recommendations that sleep difficulties are screened as part of routine ADHD assessment. The absence of a ‘gold standard’ sleep screening instrument has resulted in a variety of parent-rated measures of their child's sleep difficulties. These measures are seldom validated in the paediatric ADHD population. The Sleep Composite subscale derived from seven-items within the CBCL is used as a measure of sleep difficulties in non-ADHD populations. Given that the CBCL is often administered as part of routine ADHD assessment, the scale is well-positioned to yield an estimate of a child's sleep difficulties. Reliability for the seven-item scale was well below conventionally appropriate levels which reflect the multidimensional nature of these seven-items in this scale. However, the current results indicated good convergent validity between total scores on the CBCL Sleep Composite subscale and the SDSC – this pattern is consistent with previous findings in the non-ADHD setting [7]. Based on these findings, scores on the CBCL Sleep Composite subscale demonstrates levels of convergent validity with other paediatric sleep instruments in a paediatric ADHD sample. This instrument may be useful when purpose-built paediatric sleep instruments cannot be implemented or in retrospective research designs.

CRediT author statement

Vincent O. Mancini: Conceptualization, Methodology, Formal analysis, Investigation, Data Curation, Writing - Original Draft, Writing – Review and Editing, Project administration.

Benjamin T.D. Pearcy: Conceptualization, Methodology, Formal analysis, Investigation, Data Curation, Writing - Original Draft, Writing – Review and Editing, Project administration.

Conflict of interest

The following are the supplementary data related to this article: