Validity of Sluggish Cognitive Tempo in South America: An Initial Examination Using Mother and Teacher Ratings of Chilean Children

Abstract

Objective.

Our objective was to examine the validity of sluggish cognitive tempo (SCT) and attention-deficit/hyperactivity disorder (ADHD) inattention (IN) symptoms in children from Chile.

Method.

Mothers and teachers rated SCT, ADHD-IN, ADHD-hyperactivity/impulsivity (HI), oppositional defiant disorder (ODD), anxiety, depression, academic impairment, social impairment, and peer rejection (teachers only) in 652 Chilean children (55% boys) ages 6–14.

Results.

For both mother and teacher ratings, the eight SCT symptoms and nine ADHD-IN symptoms showed substantial loadings on their respective factors (convergent validity) along with loadings close to zero on the alternative factor (discriminant validity). ADHD-IN showed a uniquely stronger relationship than SCT with ADHD-HI and ODD whereas SCT showed a uniquely stronger relationship than ADHD-IN with anxiety and depression. Although ADHD-IN uniquely predicted academic impairment and social difficulties, SCT did not.

Conclusions.

This study provides the first evidence for the validity of SCT among children outside of North America or Western Europe.

Sluggish cognitive tempo (SCT) is a symptom dimension characterized by inconsistent alertness and slow thinking/slow behavior (Becker, 2013). Although the construct has been of interest for three decades (Becker, Marshall, & McBurnett, 2014), most studies used only a few SCT items serendipitously included in commonly used behavior-rating scales. Thus, a clear need existed for the development of measures of SCT and only recently have traditional psychometric procedures been used to create such measures (Barkley, 2013; Becker et al., 2015; Lee et al., 2014; Penny et al., 2009).

In addition to assessing symptoms of ADHD, oppositional defiant disorder (ODD), anxiety, and depression, the Child and Adolescent Disruptive Behavior Inventory (CADBI; Burns et al., 2014) was adapted to include a module assessing children’s SCT symptoms. In the initial validation, Lee et al. (2014) identified eight SCT symptoms with convergent validity (high loadings on the SCT factor) and discriminant validity from ADHD-IN (low loading on the ADHD-IN factor) with parent and teacher rating of children from the United States. These eight SCT symptoms were daydreams, alertness fluctuates, absent-minded, loses train of thought, easily confused, looks drowsy, thinking is slow, and behavior is slow. The SCT dimension also predicted symptom and impairment correlates even after controlling the ADHD-IN dimension, thus supporting the external validity of SCT (Lee et al., 2014).

Three subsequent studies have examined the CADBI SCT scale with ratings by mothers, fathers, primary teachers, and secondary teachers with children from Spain as the children progressed through the first, second, and third grades (i.e., a separate validity study at the completion of each grade, Bernad et al., 2014; Burns et al., 2013; Servera et al., 2015). Although these studies replicated and extended the unique correlates of SCT relative to ADHD-IN, only five SCT symptoms showed convergent and discriminant validity with ADHD-IN for ratings by mothers and fathers (loses train of thought, easily confused, looks drowsy, thinking is slow, and behavior is slow; the SCT dimension was defined by these five). For primary and secondary teachers, only three SCT symptoms showed convergent and discriminant validity (looks drowsy, thinking is slow, and behavior is slow; the SCT dimension was defined by these three). The children from Spain, however, comprised a narrow age range, which may have contributed to the different number of SCT symptoms with validity between the samples drawn from the United States and Spain (see Cortés et al., 2014). Thus, additional studies are needed with a broader age range and other cultural contexts to better understand the validity of the SCT symptoms. Moreover, all SCT studies conducted to date have been conducted in North America or Western Europe, making the examination of the SCT construct in other cultural contexts a research priority.

In an attempt to resolve the differences between the United States and Spain studies, this study tested the validity of the CADBI SCT measure with a much broader age range of children from Chile. The first objective was to determine if the eight SCT and nine ADHD-IN symptoms have convergent and discriminant validity. If supported, this finding would be the first time since Lee and colleagues’ (2014) initial validation study that all eight SCT symptoms showed convergent and discriminant validity. The second objective was to determine the validity of a comprehensive measurement model (SCT, ADHD-IN, ADHD-HI, ODD, anxiety, depression, academic impairment, social impairment, and peer rejection). The model was expected to demonstrate a good fit. In addition, while ADHD-IN was expected to have larger factor correlations than SCT with ADHD-HI and ODD, ADHD-IN and SCT were expected to have similar factor correlations with anxiety, depression, academic impairment, social impairment, and peer rejection. The third objective was to determine the unique correlates of SCT and ADHD-IN (i.e., SCT’s relationships with the outcomes after controlling for ADHD-IN and ADHD-IN’s relationships with the outcomes after controlling for SCT). While both SCT and ADHD-IN were expected to have significant unique relationships with anxiety, depression, academic impairment, social impairment, and peer rejection, ADHD-IN was expected to have a significantly stronger unique relationship than SCT with ADHD-HI and ODD. Support for these hypotheses would help resolve the differences between the earlier studies and expand the validity research on SCT beyond North America and Western Europe into Chile.

Method

Participants and Procedures

The participants were the mothers and teachers of 652 first to eighth grade children (55.4% boys) ages 6–14 years (M_age = 9.64, SD = 1.77) from 20 (6 rural and 14 urban) schools in the Maule Region of Chile). A total of 7,461 first to eight grade children were enrolled in these 20 schools with approximately 5% excluded from the potential sample due to having an official school diagnosis (e.g., developmental disabilities, ADHD, learning and disruptive behavior disorders). Nine percent of the mothers of the potential sample volunteered also giving permission for their child’s teacher to complete the measures (all teachers given permission participated). Each teacher (88 total teachers) rated an average of 7.41 children (SD = 6.77). The socioeconomic status of most of the inhabitants of the region is low to medium with 5% of the individuals having no formal education, 49% have a primary education, and only 6% have college degrees. The IRB of the Catholic University of Maule approved this research.

Measures

Child and Adolescent Disruptive Behavior Inventory (CADBI, Burns et al., 2014).

Parents and teachers completed their respective versions of the CADBI. The CADBI measures SCT (eight symptoms), ADHD-IN (nine DSM-5 symptoms), ADHD-HI (nine DSM-5 symptoms), ODD toward adults (e.g., argues with adults; eight DSM-5 symptoms with adults the target), ODD toward siblings/peers (e.g., argues with siblings/peers; eight items that modify the DSM-5 symptoms to relationships with siblings/peers), anxiety (six symptoms), depression (six symptoms), academic impairment (four items: completion of homework, reading skills, arithmetic skills, and writing skills), and social impairment (four items: quality of interactions with parents [teachers at school]; quality of interactions with other adults than parents (grandparents, babysitters, family friends [other adults at school], quality of interactions with brothers and sisters [quality of interactions with peers in the classroom], and quality of interactions with other children in the home and community [peers outside of the classroom at school]). The Spanish and English versions of the scale are available from the authors.

The symptoms were rated on a 6-point frequency of occurrence scale (i.e., almost never [never or about once per month], seldom [about once per week], sometimes [several times per week], often [about once per day], very often [several times per day], and almost always [many times per day]). The four academic and four social impairment items were rated on a 7-point scale (severe difficulty, moderate difficulty, slight difficulty, average performance [average interactions] for grade level, slightly above average, moderately above average, and excellent performance [excellent interactions] for grade level). The academic and social impairment items were reversed keyed so higher scores represent higher impairment. The two ODD scales were combined into a single scale. Earlier studies support the reliability and validity of the scores (Burns et al., 2013; Bernad et al., 2014; Lee et al., 2014; Servera et al., 2015).

Dishion Social Acceptance Scale (DSAS, Dishion, 1990).

The DSAS is a three-item teacher rating scale that assesses a child’s peer rejection. Teachers rated the proportion of classmates who “dislike,” “like,” and “ignore” the target child on a 5-point scale (very few [less than 25%]; some [25 to 49%]; about half [50%]; many [51 to 75%]; and almost all [greater than 75%]). The three items were used to define a peer rejection factor (the “like” item was reverse keyed). This is a well-validated measure with scores being associated with peer sociometric nominations (Dishion, 1990) and impairment in children with ADHD (Becker et al., 2013).

Analytic Strategy

The item ratings were treated as ordered-categories (i.e., the Mplus WLSMV estimator, Version 7.3, Muthén & Muthén, 1998-2012). All analyses took into account the children were nested within teachers (Type = complex option in Mplus). The fit of the models was evaluated with comparative fit index (CFI, study criterion ≥ .95), Tucker-Lewis Index (TLI, study criterion ≥ .95), and root mean square error of approximation (RMSEA, study criterion ≤ .05). The Mplus model constraint procedure was used to determine if factor correlations and standardized partial regression coefficients differed significantly.

Results

Convergent and Discriminant Validity of SCT and ADHD-IN Symptoms

A two-factor model was applied to the eight SCT and nine ADHD-IN symptoms (i.e., the analysis was restricted to two factors with cross-loadings allowed). The eight SCT and nine ADHD-IN symptoms showed substantial loadings on their respective factors (mothers: SCT—M = .87, SD = .04; ADHD-IN—M = .86, SD = .10; teachers: SCT—M = .86, SD = .06; ADHD-IN—M = .88, SD = .06) in conjunction with low loadings on the other factor (mothers: SCT symptoms on ADHD-IN factor—M = .02, SD = .06; ADHD-IN symptoms on SCT factor—M = .02, SD = .13; teachers: SCT symptoms on ADHD-IN factor—M = .07, SD = .10; ADHD-IN symptoms on SCT factor—M = .02, SD = .12,). These results provided the justification to examine the correlates of the SCT and ADHD-IN symptom dimensions.

Measurement Model

Global fit.

The confirmatory factor analytic model resulted in a good fit for the eight-factor model for mothers, the nine-factor model for teachers, and the eight-factor model for mothers and teachers simultaneously (i.e., mothers: χ² (1349) = 2100, p < .001, CFI = .974, TLI = .973, and RMSEA = .029 [90% CI: .027, .032]; teachers: χ² (1448) = 2395, p < .001, CFI = .972, TLI = .970, and RMSEA = .032 [90% CI: .029, .034]; mothers and teachers: χ² (5444) = 6782, p < .001, CFI = .969, TLI = .968, and RMSEA = .019 [90% CI: .018, .021]).

Reliability coefficients.

The reliability coefficients (measures of true score variance for each scale similar to coefficient alpha, see Brown, 2015, pp. 305–321) for mothers (teachers) for SCT, ADHD-IN, ADHD-HI, ODD, anxiety, depression, academic impairment, and social impairment were .95 (.95), .95 (.96), .94 (.95), .97 (.97), .87 (.87), .90 (.90), .93 (.96), and .95 (.96), respectively. The reliability coefficient for three-item peer rejection measure (completed by teachers only) was .83.

Mother-teacher factor correlations.

The mother-teacher factor correlation for SCT was .76, .73 for ADHD-IN, .73 for ADHD-HI, .73 for anxiety, .69 for depression, .51 for ODD, .56 for social impairment, and .47 for academic impairment.

SCT and ADHD-IN factor correlations.

For mothers’ ratings, the SCT and ADHD-IN factor correlation was .79 (SE = .02) with the same value and SE for teachers. For mothers’ ratings, the ADHD-IN and ADHD-HI factor correlation was .84 (SE = .02) with the value for teachers being .74 (SE = .02). The discriminant validity of SCT with ADHD-IN was similar to ADHD-IN with ADHD-HI.

SCT and ADHD-IN factor correlations with outcomes.

Higher scores on SCT and ADHD-IN were associated with significantly higher scores (ps < .01) on the ADHD-HI, ODD, anxiety, academic impairment, social impairment, and peer rejection. ADHD-IN showed a significantly (ps < .05) larger correlation than SCT with ADHD-HI, ODD, academic impairment, social impairment, and peer rejection while SCT showed a significantly (ps < .05) larger correlation than ADHD-IN with anxiety and depression. Table 2 shows these factor correlations.

Table 2.

Structural Regression of ADHD-HI, ODD, Anxiety, Depression, Academic Impairment, Social Impairment, and Peer Rejection Factors on Sluggish Cognitive Tempo and ADHD-IN Factors

	ADHD-HI		ODD		ANX		DEP		AI		SI		PR
Predictors	β	SE	β	SE	β	SE	β	SE	β	SE	β	SE	β	SE
Parents
SCT	.22**	.05	.23**	.06	.56**	.08	.58**	.05	−.15ns	.08	−.07ns	.07	------------
ADHD-IN	.66**	.04	.48**	.05	.18*	.06	.30**	.04	.70**	.06	.46**	.06	------------
Teachers
SCT	.27**	.05	.18*	.06	.45**	.09	.62**	.06	−.07ns	.08	−.22*	.09	−.06ns	.08
ADHD-IN	.53**	.04	.40**	.06	.03ns	.08	.18*	.05	.70**	.06	.53**	.07	.56**	.08

Note. SCT = sluggish cognitive tempo; ODD = oppositional defiant disorder toward adults and peers; ADHD-IN = attention-deficit/ hyperactivity disorder-inattention; ADHD-HI = attention-deficit hyperactivity disorder-hyperactivity/impulsivity; DEP = depression; AI = academic impairment; SI = social impairment; PR = peer rejection. Only teachers completed the peer rejection measure.

^*p < .05.

^**p < .001.

ns= non-significant.

Unique Effects of SCT and ADHD-IN on Outcomes

A structural regression analysis was used to determine the unique effects of SCT and ADHD-IN on ADHD-HI, ODD, anxiety, depression, academic impairment, social impairment, and peer rejection (i.e., the ability of SCT to predict the outcomes after controlling for ADHD-IN and the ability of ADHD-IN to predict the outcomes after controlling for SCT). Table 3 shows these partial standardized regression coefficients.

Unique effects of SCT and ADHD-IN on ADHD-HI and ODD.

Higher scores on ADHD-IN and SCT uniquely predicted significantly higher scores on ADHD-HI and ODD (ps < .001). ADHD-IN, however, had significantly stronger unique relationship than SCT with both ADHD-HI and ODD (ps < .05). The same results occurred for mothers and teachers.

Unique effects of SCT and ADHD-IN on depression and anxiety.

For mothers and teachers, higher scores on SCT and ADHD-IN both uniquely predicted significantly (ps < .05) higher scores on depression with SCT’s unique relationship with depression being significantly stronger than ADHD-IN’s unique relationship with depression (ps < .05). For mothers and teachers, higher scores on SCT uniquely predicted higher scores on anxiety while ADHD-IN only predicted uniquely higher anxiety scores for mothers. SCT’s unique relationship with anxiety was also significantly stronger than ADHD-IN’s unique relationship with anxiety (ps < .05) for mothers and teachers.

Unique effects of SCT and ADHD-IN on academic impairment.

For mothers and teachers, while higher scores on ADHD-IN uniquely predicted significantly (ps < .001) higher scores on academic impairment, SCT was not significantly related to academic impairment after controlling for ADHD-IN.

Unique effects of SCT and ADHD-IN on social impairment and peer rejection.

For mothers and teachers, higher scores on ADHD-IN uniquely predicted significantly (ps < .001) higher scores on social impairment while mothers’ ratings of SCT were not significantly related to social impairment when controlling for ADHD-IN. For teachers, higher scores on SCT uniquely predicted lower levels of social impairment after controlling for ADHD-IN (p < .05). Finally, for teacher ratings of peer rejection, higher scores on ADHD-IN uniquely predicted significantly higher scores on peer rejection (p < .001) while SCT was not uniquely related to peer rejection.

Sex effects.

The structural regression analysis was repeated for boys and girls separately. The results from these four analyses yielded the same conclusions with only two exceptions: for mothers’ ratings of boys and teachers’ ratings of girls, SCT was not significantly related to ODD after controlling for ADHD-IN.

Discussion

In the initial validation study examining the SCT module of the CADBI, Lee et al. (2014) identified eight SCT symptoms demonstrating both convergent validity (substantial loadings on the SCT factor) and discriminant validity (low loadings on the ADHD-IN factor) with parent and teacher ratings of United States children. The eight SCT symptoms were daydreams, alertness fluctuates, absent-minded, loses train of thought, easily confused, looks drowsy, thinking is slow, and behavior is slow. This SCT dimension also showed unique correlates relative to the ADHD-IN dimension. Although subsequent research examining the CADBI SCT measure in children from Spain identified the same unique correlates of SCT and ADHD-IN (Bernad et al., 2014; Burns et al., 2013; Servera et al., 2015), only five of eight SCT symptoms showed convergent and discriminant validity for mother and father ratings and only three for primary and second teacher ratings (i.e., the SCT dimension was defined by 5 symptoms for parents and 3 symptoms for teachers). The possible reasons for the failure of all eight SCT symptoms to show validity with the Spanish children were the narrow age range of the children along with possible problems with the translation of the SCT symptoms into Spanish. The purpose of the current study was to simultaneously evaluate both of these possibilities, while also being the first study to examine then internal and external validity of SCT outside of North America or Western Europe.

In the current study, using mother and teacher ratings of a broad age range of children (first to eighth grade) from the Spanish-speaking country of Chile, all eight SCT symptoms showed strong convergent validity as well as discriminant validity with the ADHD-IN dimension. The same results occurred for the ADHD-IN symptoms. Importantly, results were consistent across both mother and teacher ratings. This is the first study since the initial Lee et al. (2014) study where all eight SCT symptoms showed convergent and discriminant validity. Thus, although the current study will need to be repeated with a broad age range of children from Spain, the narrow age range of the children from Spain (as opposed to differences in the Spanish translation version of the CADBI) appears most likely responsible for the inconsistent SCT symptom validity results between the United States and Spain.

The current study with the Chilean children also replicated some of the external correlates of SCT relative to ADHD-IN. As in the earlier studies in the United States and Spain, as well as other studies using different measures of SCT (Becker, Luebbe, Fite, Stoppelbein, & Greening, 2014; McBurnett et al., 2014; Penny et al., 2009), SCT’s unique relationship with ADHD-HI and ODD was weaker than ADHD-IN’s unique relationship with ADHD-HI and ODD. Also in line with previous studies (Becker et al., 2015; Bernad et al., 2014; Burns et al., 2013; Lee et al., 2014; Penny et al., 2009; McBurnett et al,. 2014; Servera et al., 2015), higher levels of SCT predicted higher levels of anxiety and depression even after controlling for ADHD-IN. The main difference from earlier studies examining the CADBI was that SCT did not predict academic or social impairment after controlling for ADHD-IN in the current study for either mothers or teachers. The reason for this result, which is inconsistent from all earlier studies examining the CABDI SCT module, is not immediately clear. One possibility is that students who had an official school diagnosis (e.g., ADHD, learning disabilities, disruptive behavior disorders) were not eligible for participation in this study, which may have led to children with the most severe academic and social impairments being excluded from the study and subsequent analyses. While this exclusionary criterion was initially used in order to identify a more clearly “non-clinical” sample, in hindsight the exclusion of children with school diagnoses was a limitation. However, more research is needed to evaluate this possibility, as well as to further examine the extent to which SCT symptoms are associated with functional impairments in Chilean children.

In conclusion, it was encouraging to find that all eight SCT symptoms showed excellent convergent validity as well as discriminant validity with the ADHD-IN dimension using both mother and teacher ratings of Chilean children. Moreover, this is the first study to examine the internal and external validity outside of North America or Western Europe, with almost all studies to date conducted in children from the United States or Spain. Thus, our findings are important for not only demonstrating the internal and external validity in a large sample of children from Chile but also in being a first step in evaluating whether the SCT construct has cross-cultural validity. This is an important area for future research, and it remains important to examine SCT in additional cultural contexts.

Table 1.

Correlations (Standard Errors) of Sluggish Cognitive Tempo and ADHD-IN Factors with ADHD-HI, ODD, Anxiety, Depression, Academic Impairment, Social Impairment, and Peer Rejection Factors

	ADHD-HI	ODD	ANX	DEP	AI	SI	PR
Parents
SCT	.74 (.03)	.60 (.03)	.70 (.04)	.81 (.02)	.41 (.05)	.29 (.05)	-----
ADHD-IN	.84 (.02)	.66 (.03)	.62 (.03)	.75 (.02)	.59 (.03)	.41 (.04)	-----
Teachers
SCT	.69 (.03)	.49 (.04)	.47 (.06)	.76 (.03)	.49 (.04)	.20 (.06)	.37 (.05)
ADHD-IN	.74 (.02)	.54 (.04)	.38 (.06)	.66 (.03)	.65 (.02)	.35 (.05)	.51 (.05)

Note. All correlations were significant at p < .01. SCT = sluggish cognitive tempo; ADHD-IN = attention-deficit/hyperactivity disorder-inattention; ADHD-HI = hyperactivity/impulsivity; ODD-A = oppositional defiant disorder toward adults; ODD-P = oppositional defiant disorder toward peers; DEP = depression; AI = academic impairment; SI = social impairment; PR = peer rejection. Only teachers completed the peer rejection measure.