How consistent is sluggish cognitive tempo across occasions, sources, and settings? Evidence from latent state-trait modeling

Abstract

Research has yet to determine how much of the true score variance in sluggish cognitive tempo (SCT) symptom ratings is consistent across occasions, sources, and settings versus specific to occasion, source, and setting (i.e., a trait or state-like construct). Our first objective was to determine the amount of variance in SCT ratings that was consistent (trait consistency) across three occasions of measurement over 12-months versus specific to the occasion (occasion-specificity) with ratings by mothers, fathers, primary teachers, and secondary teachers of 811 Spanish children. Our second objective was then to determine the convergent validity for trait consistency and occasion-specificity variance components within and across settings. SCT ratings reflected mostly trait consistency for mothers, fathers, and primary teachers (less so for secondary teachers) with the convergent validity for trait consistency also being strong for mothers with fathers and for primary teachers with secondary teachers. Across home and school, however, convergent validity for trait consistency was low and even lower for occasion-specificity. Although SCT symptoms showed similar levels of trait consistency across occasions and convergent validity within settings as ADHD symptoms in a prior study, SCT symptoms had slightly weaker convergent validity for trait consistency across settings relative to ADHD symptoms.

Sluggish cognitive tempo (SCT) is characterized by inconsistent alertness, slow thinking/behavior, and drowsiness. In terms of internal validity, a recent meta-analysis identified SCT symptoms that loaded higher on the SCT factor than the attention-deficit/hyperactivity disorder-inattention (ADHD-IN) factor with preliminary evidence that SCT is also distinct from anxiety, depression, and daytime sleepiness (Becker, Leopold, et al., 2016). In terms of external validity, this meta-analysis found higher levels of SCT predicted higher levels of anxiety, depression, academic impairment, and social impairment even after controlling for ADHD-IN.

SCT also has trait-like qualities (high test-retest correlations and moderate to high inter-rater correlations). In terms of test-retest correlations, SCT showed 1- to 12-month stability coefficients from .80 to .92 (Becker, Leopold, et al., 2016, Table 4; Khadka, Burns, & Becker, 2016), two-year stability coefficients of .60 for mothers’, .65 for fathers’, and .45 for teachers’ ratings (different teachers across occasions; Bernad, Servera, Becker, & Burns, 2016), and a ten-year stability coefficient of .43 for mothers’ ratings (Leopold et al., 2016). In terms of inter-rater correlations, correlations for raters within the same setting (mothers with fathers; teachers with teachers) ranged from .71 to .80 (Becker, Leopold, et al., 2016, Table 4; Burns, Becker, Servera, Bernad, & García, 2016) with average home-to-school inter-rater correlations being .54 (Becker, Leopold, et al., 2016, Table 4). The test-retest and inter-rater correlations for SCT are also similar to those for ADHD-IN (Willcutt et al., 2012).

Table 4.

Average Shared Trait Consistency, Unique Trait Consistency, Shared Occasion-Specificity, and Unique Occasion-Specificity for a Multisource Latent State Trait Analysis on the Sluggish Cognitive Tempo Symptoms

Source	Shared Trait Consistency	Unique Trait Consistency	Shared Occasion- Specificity	Unique Occasion- Specificity
Fathers	.80 (.78, .81)	.20 (.19, .22)	.37 (.32, .44)	.63 (0.56, .68)
Primary Teachers	.29 (.27, .31)	.71 (.69, .73)	.00 (.00, .00)	1.00 (1.00, 1.00)
Secondary Teachers	.22 (.20, .24)	.78 (.76, .80)	.00 (.00, .01)	1.00 (0.99, 1.00)

Note. Values indicate the average across parcels with the range in parentheses. Mothers were used as the reference source.

Utility of Latent State-Trait Measurement Models

Although there is some indication that family and other environmental stressors are associated with SCT (Becker, Leopold, et al., 2016; Becker, Withrow et al., 2016), test-retest and inter-rater correlations suggest that SCT is a more trait-like (i.e., consistent across occasions, sources, and settings) than state-like (i.e., specific to the occasion, source or setting) construct. Latent state-trait (LST) measurement models, however, can better test if SCT is a more trait- or state-like construct. We now describe LST models.

Single source LST models

A single source LST model allows a direct test of the consistency across measurement occasions question by decomposing the observed score variance in SCT symptom ratings into true score variance and error variance, and then further decomposing the true score variance into trait (trait consistency [consistency across occasions]) and state residual (occasion-specificity) variance (Steyer, Geiser, & Fiege, 2012). Trait variance represents variance that is person-specific and independent of occasion and/or the person-occasion interaction. In contrast, occasion-specific variance represents occasion-specific influences and/or person-occasion interactions.

If trait variance (consistency across occasions) in SCT symptom ratings is higher than occasion-specific variance for various raters, such findings would indicate that SCT symptoms are more trait- than state-like across occasions of measurement for each source. If occasion-specific variance is larger than trait variance, then such would indicate that SCT symptoms represented more of an occasion-specific state for the individual sources. Whether SCT is more state- (occasion-specific) or trait-like (consistent across occasions) has important theoretical and clinical implications. If SCT is more trait-like, it will be important for future research to examine linkages with individual differences that are generally stable, such as temperament and personality, as well as the familial aggregation and heritability of SCT. If SCT is more state-like, it will be important to assess and understand situational stressors that may contribute to the expression of SCT.

To summarize, the single source LST model can determine if SCT ratings are more trait- or state-like across multiple occasions of measurement for an individual source. Figure 1 shows this LST model. Table S1 in the supplemental materials shows the equations and coefficients for the single source LST model.

Figure 1.

Single source latent state-trait model with parcel-specific trait factors measured at three occasions. This model was applied to each of the four sources separately. T = trait factor; O = state (occasion-specific) residual factor. This figure was adapted with permission from Litson et al. (2016).

Multiple source LST models

Although moderate to high inter-rater correlations among different sources for SCT ratings indicate convergent validity, it is not known if such consistency reflects convergent validity at the trait level (i.e., shared trait variance), convergent validity at the occasion-specific level (i.e., shared occasion-specific variance) or a combination of both (a single occasion of measurement is not able to separate these two sources of true score variance). A multiple source LST measurement model allows an answer to this question by determining the amount of trait (trait consistency) and occasion-specific variance that is either shared or not shared between a reference source (e.g., mothers) and non-reference sources (e.g., fathers, primary teachers, and secondary teachers; Courvoisier, Nussbeck, Eid, Geiser, & Cole, 2008). Figure 2 shows our multiple source LST model and Table S1 in the supplemental materials the equations and coefficients for this model. We now describe the variance components of this model.

Figure 2.

Multiple source latent state-trait model with parcel-specific trait factors. In order to reduce the complexity of the figure, only three sources (mothers, fathers, and primary teachers) and two occasions are shown whereas the actual model involved four sources (mothers, fathers, primary teachers, and secondary teachers) and three occasions. T = reference trait factors that are defined by mother reports and are parcel-specific across sources; TS = residual trait factors that pertain to the non-reference sources and are source- and parcel-specific; O = reference state residual factors that are defined by mother reports and are parcel-specific across sources; OS = state residual factors that pertain to the non-reference sources and are source-specific across parcels. This figure was adapted with permission from Litson et al. (2016).

The first variance component is shared trait consistency, which is the proportion of trait variance in SCT ratings that a non-reference source shares with the reference source (i.e., convergent validity for trait variance [trait consistency]). The second is unique trait variance, which is the proportion of trait variance in SCT ratings that a non-reference source does not share with the reference source. Unique trait variance represents the degree of source specificity for trait aspects of SCT ratings (discrepancy in trait variance). The third is shared occasion-specificity, which is the proportion of occasion-specific variance in SCT ratings that a non-reference source shares with the reference source (the convergent validity for the occasion-specific aspects of SCT). The fourth is unique occasion-specificity. This component represents occasion-specific variance in SCT ratings that is unique to a non-reference source, thus not shared with the reference source (the degree of source-specificity with regard to the state aspects of the ratings).

The multiple source LST model also allows one to determine convergent validity of residual (source-specific) trait variance among the non-references sources (i.e., fathers with primary and secondary teachers; primary teachers with secondary teachers). Residual trait variance is trait variance for non-reference sources (fathers, primary teachers, and secondary teachers) that is independent of trait variance for the reference source (mothers) and is reflected in trait residual factors (TS correlations in Figure 2). Correlations between these factors allow one to determine the convergent validity between fathers and teachers as well as primary teachers with secondary teachers after partialing out what these three sources share with mothers, thus providing additional information on whether the non-reference sources share a common perspective that deviates from the reference source (mothers). Similar analyses can be applied with the source-specific occasion-specific variance for the non-reference sources (OS correlations in Figure 2).

If non-reference sources share a high proportion of trait variance (trait consistency) in SCT ratings with the reference source, then such findings would indicate convergent validity of trait variance within (i.e., fathers with mothers) and across (i.e., primary and secondary teachers with mothers) settings. If non-reference sources share a high proportion of occasion-specific variance in SCT ratings with the reference source, then such would provide support for convergent validity of the occasion-specific variance within and across settings. The lack of shared trait or occasion-specific variance between non-reference sources and the reference source would indicate a lack of convergent validity for both trait and occasion-specific variance (i.e., consistent [trait] variance was source specific and occasion-specific variance was source specific at a specific occasion).

To summarize, the multiple source LST model provides a novel way to study informant agreements and disagreements for trait and occasion-specific variance in symptom ratings as well as to identify correlates of the shared trait consistency and unique trait consistency (see also De Los Reyes, 2013; De Los Reyes, Alfano, Lau, Augenstein, & Borelli, 2016; De Los Reyes & Ohannessian, 2016; De Los Reyes et al., 2015). We elaborate more on these features of the multiple source LST model in the Discussion.

Trait and Occasion-Specific Variance in ADHD-IN, ADHD-HI, and ODD Symptoms

Two recent studies applied single and multiple source LST models to ADHD-IN, ADHD-hyperactivity/impulsivity (HI), and oppositional defiant disorder (ODD) symptom ratings in Spanish children (Litson, Geiser, Burns, & Servera, 2016; Preszler, Burns, Litson, Geiser, & Servera, 2016). Each study involved the same four sources (mothers, fathers, primary teachers, and secondary teachers) rating Spanish children in the first and second grades. ADHD-IN, ADHD-HI, and ODD symptom ratings contained a moderate (53%) to substantial (78%) amount of trait variance for the four sources. The results also showed strong convergent validity for trait variance for ADHD-IN, ADHD-HI, and ODD ratings within settings (68% to 80%) along with low (ODD: approximately 32%) to moderate (ADHD-IN and HI: approximately 50%) across-setting convergent validity for trait variance. It is currently unknown how SCT symptom ratings will compare to these results on ADHD and ODD symptom ratings. It is especially important to determine if SCT symptom ratings are as trait-like as the ADHD-IN symptom ratings (i.e., Do SCT symptom ratings represent a trait as much as ADHD-IN symptom ratings?). Given the analyses on the SCT symptoms involved the same four sources and Spanish children, a direct comparison of the earlier results on ADHD-IN symptoms to SCT symptoms was possible.

Study Objectives

This study used ratings of SCT symptoms by mothers, fathers, primary teachers, and secondary teachers of Spanish children in the first and second grades to address two objectives. The first objective was to determine the proportion of trait variance (trait consistency) and occasion-specific variance (occasion-specificity) in SCT ratings for mothers, fathers, primary teachers, and secondary teachers. This objective required that a single source LST model be fit to the ratings for each source separately (Figure 1). It was predicted that SCT symptoms would show larger amount of trait (consistency over occasions) than occasion-specific variance for each source (approximately 70% trait variance if the findings are similar to the ADHD-IN symptom ratings, Litson et al., 2016). This outcome would indicate that the SCT symptoms were more trait- than state-like for each source over the three occasions of measurement for the one-year interval.

The second objective was to determine the proportion of trait and occasion-specific variance in SCT ratings that fathers, primary teachers, and secondary teachers either shared or did not share with mothers. This objective required that a multiple source LST model be fit to the ratings of the four sources simultaneously (Figure 2). Eight predictions were made for this analysis. The first prediction was that fathers would share a large amount of trait variance with mothers (approximately 80% if similar to ADHD-IN results). The second prediction was that fathers would share a much smaller amount of occasion-specific variance with mothers (approximately 40% if similar to ADHD-IN results). The third prediction was that primary and secondary teachers would only share a moderate amount of trait variance with mothers (approximately 50% if similar to ADHD-IN results) while the fourth prediction was that primary and secondary teachers would share no occasion-specific variance with mothers (approximately 0% if similar to ADHD-IN results).

The fifth and sixth predictions dealt with correlations among residual trait factors for fathers, primary teachers, and secondary teachers (residual trait correlations indicate whether there is convergent validity for residual trait variance among father, primary teacher, and secondary teacher reports after accounting for trait variance that these sources share with mother reports.). The fifth prediction was that fathers would share approximately 0% residual trait variance with primary and secondary teachers whereas the sixth prediction was that primary and secondary teachers would share substantial residual trait variance with each other (approximately 80% if similar to ADHD-IN results).

The seventh and eighth predictions dealt with the correlations among the residual occasion-specific factors for fathers, primary teachers, and secondary teachers. Residual occasion-specific correlations indicate whether there is convergent validity for the residual occasion-specific variance among father, primary teacher, and secondary teacher reports after accounting for variance that these sources share with mother reports. The seventh prediction was that fathers would share approximately 0% residual occasion-specific variance with primary and secondary teachers, and the eighth prediction was that primary and secondary teachers would share a very small amount of residual occasion-specific variance with each other. Support for these eight predictions would indicate strong convergent validity of trait variance within the home and school settings, moderate convergent validity of trait variance across the two settings, low convergent validity for occasion-specific variance within settings, and zero convergent validity for occasion-specific variance across settings.

This is the first study to apply single and multiple source LST models to SCT ratings. The findings will indicate if SCT symptoms reflect a more trait- or state-like construct for mothers, fathers, primary teachers, and secondary teachers as well as the degree of convergent validity of trait and occasion-specific variance within and across settings. These results will also us to determine if SCT symptoms are as trait-like as ADHD-IN symptoms by a comparison to the earlier results for the ADHD-IN symptoms, an important question given the recent interest in whether SCT represents a unique attention disorder (Barkley, 2016; Becker, Leopold, et al., 2016).

Method

Participants and Procedures

Forty-three of 46 elementary schools on Majorca (Spain) volunteered with 22 of these 43 schools randomly selected to participate (resources on Majorca only allowed for data collection at 22 schools) with eight additional schools recruited from Madrid (eight were asked and eight agreed to participate). The eight schools were recruited to increase sample size. Potential participants were mothers, fathers, primary teachers (children’s main classroom teacher), and secondary teachers (children’s teachers of special subjects such as English, Catalan language, music, visual arts, and physical education) of 1,045 first grade children. The study involved 811 unique children (54% boys) across the three assessments (i.e., [1] spring semester of the first grade, [2] six-weeks after the first assessment, and [3] spring semester of the second grade). There were 758 unique children at the first assessment, 746 at the second assessment, and 718 at the third assessment. Children had the same primary teachers for first and second grades (first and second grades were together in the same room) with each child being rated by the same primary teacher across the three assessments. Secondary teachers were the same for the first and second assessments yet could have been different for the third assessment. A cover letter that explained the purpose was given to parents, and, with parental written consent, a similar cover letter was given to teachers. Teachers also provided written consent. The IRB approved the protocol.

For the first grade assessment, 723 mothers and 603 fathers returned the measures with the numbers for the second assessment being 667 mothers and 584 fathers and for the third assessment 604 mothers and 540 fathers. At the first assessment, 61 primary teachers and 49 secondary teachers from 28 of 30 schools participated with primary teachers rating an average of 11.76 (SD = 5.09, n = 743) and secondary teachers an average of 9.02 (SD = 6.58, n = 574) children. For the second assessment, 58 primary teachers and 51 secondary teachers participated with primary teachers rating an average of 10.93 (SD = 5.27, n = 679) and secondary teachers an average of 9.93 (SD = 6.45, n = 623) children. For the third assessment, teachers from 29 of 30 schools participated with 62 primary teachers rating an average of 11.19 (SD = 4.43, n = 701) and 59 secondary teachers an average of 10.45 (SD = 5.18, n = 664) children. At the first assessment, the average age of the children was seven years with little variation. Although ethnicity was not collected for individual children, school-level data indicated that approximately 90% of the children were Caucasian and 10% were North African. SES information was not available. Although the ADHD symptom ratings were not the focus of the current study, 7.5% of the sample (range = 6% to 9% across the four sources) showed ADHD-IN and/or ADHD-HI symptoms in the clinical range (i.e., six of nine symptoms). A rating of “often” or higher was used to consider an ADHD symptom as qualitatively present. Given these procedures do not include other ADHD diagnostic criteria, children meeting criteria for ADHD would less than these percentages.

Measure

Child and Adolescent Disruptive Behavior Inventory (CADBI; Burns et al., 2014)

The study used the eight SCT items from the CADBI. Table 1 shows the eight SCT symptoms. SCT symptoms were rated on a 6-point scale (almost never [never or about once per month], seldom [about once per week], sometimes [several times per week], often [about once per day], very often [several times per day], and almost always [many times per day]). Mothers and fathers were asked to make their ratings independently on the children’s behavior in home and community and not consider reports of behavior at school. Primary teachers and secondary teachers were instructed to make their ratings independently on school behavior. Earlier studies support the reliability and validity of scores on the SCT scale with ratings by mothers, fathers, and teachers from Chile, Korea, Nepal, Spain, and United States (Belmar, Servera, Becker, & Burns, 2015; Burns et al., 2016; Khadka et al., 2016; Lee, Burns, Snell, & McBurnett, 2014; Lee, Burns, & Becker, 2016, in press). Table 2 shows the descriptive information for the SCT scale for this study.

Table 1.

Parcel Assignment of Sluggish Cognitive Tempo Symptoms

Parcel 1 Loses train of thought during homework or home activities such as chores (e.g., forgets what he or she was about to say, forgets what he or she was looking for, loses his or her place when working on an activity) Easily confused during homework or home activities such as chores (e.g., gets mixed up trying to follow the instructions for an activity, becomes confused when the directions for an activity are a little complicated) Looks drowsy during homework or home activities such as chores even when he or she has had a good night sleep (e.g., seems sleepy and tired, yawns) (i.e., Drowsiness is NOT due to sleep problems at night)

Parcel 2 Daydreams during homework or home activities such as chores (e.g., stares off into space during activities, lost in his or her own thoughts during activities rather than paying attention to the activity) Absent-minded during homework or home activities such as chores (e.g., puts something down during activities and then immediately forgets where he or she puts it; becomes so absorbed in one thought that he or she does not attend to the current activity) Behavior is slow during homework or home activities such as chores (e.g., moves at a “snail’s pace,” behavior is as “slow as molasses,” last to finish the activity, slow at routine activities)

Parcel 3 Alertness changes from moment to moment during homework or home activities such as chores (e.g., spaces or zones out during activities, brief lapses of attention during activities) Thinking seems slow during homework or home activities as chores (e.g., takes longer than you expect to respond to your questions even when you have his or her attention, slow to make decisions or choices)

Note. The eight SCT items on the teacher rating scale were the same except classroom activities was substituted for during homework or home activities.

Table 2.

Descriptive Information for Sluggish Cognitive Tempo Scale

	n	α	M	SD	Range
	Occasion 1
Mothers	723	.87	0.90	0.79	0.00, 4.75
Fathers	603	.87	0.84	0.74	0.00, 4.50
Primary Teachers	743	.95	1.02	1.12	0.00, 5.00
Secondary Teachers	574	.95	0.85	1.06	0.00, 5.00
	Occasion 2
Mothers	667	.89	0.86	0.75	0.00, 4.00
Fathers	584	.89	0.85	0.75	0.00, 3.75
Primary Teachers	679	.95	0.90	1.02	0.00, 4.63
Secondary Teachers	623	.95	0.89	1.09	0.00, 5.00
	Occasion 3
Mothers	604	.88	0.89	0.77	0.00, 3.63
Fathers	540	.88	0.84	0.74	0.00, 4.38
Primary Teachers	701	.94	0.84	1.03	0.00, 5.00
Secondary Teachers	664	.94	0.83	1.00	0.00, 5.00

Note. The range for the scale was from 0.00 to 5.00. α = Cronbach’s alpha.

Analyses

Analytic strategy

For item-level analyses, items were treated as ordered categories, using the robust weighted least squares estimator (WLSMV estimator, Mplus version 7.4). For the parcel-level analyses, the study used robust maximum likelihood estimation (MLR estimator). The MLR estimator uses all the available information for the analyses and also accounts for any non-normality. Parcels were used in the more complex LST models to reduce the number of indicators (i.e., 36 indictors rather than 96). Measurement models were evaluated with comparative fit index (CFI, ideal study criterion ≥ .95), the Tucker-Lewis Index (TLI, ideal study criterion ≥ .95), and root mean square error of approximation (RMSEA, ideal study criterion ≤ .05). Decrease in CFI was used to evaluate the invariance analyses for like-item loadings and thresholds as well as like-item parcels and intercepts. If the decrease in CFI was ≤ .01, then the invariance constraints imposed at the step were assumed to be tenable (Little, 2013, chap. 5). Given children were clustered within classes, the Mplus type = complex option was used to correct the standard errors.

Preliminary analyses on individual items to justify creation of parcels

A multiple indicator (eight SCT symptoms) by multiple occasion (three assessments) by multiple source (mothers, fathers, primary teachers, and secondary teachers) confirmatory factor analysis (CFA) was used to evaluate the invariance of like-item loadings and thresholds across sources and occasions. If constraints on like-symptom loadings and thresholds hold across the three occasions and four sources, then this outcome yields one set of eight unstandardized loadings rather 12 sets of eight unstandardized loadings. The single set of eight SCT symptom factor loadings was then used to create parcels for LST analyses. As noted earlier, the use of parcels reduces the number of indicators in the LST models from 96 to 36, thus making these complex analyses possible.

The SCT items were assigned to parcels in this manner. The eight unstandardized loadings were first ranked from highest to lowest. These rank ordered loadings were then arranged into parcels using a procedure recommended by Little (2013), which involves assigning the two items with highest and lowest loadings to parcel one, the two items with second highest and lowest loadings to parcel two, and the two items with third highest and lowest loadings to parcel three. This process was repeated until all items were assigned to three parcels. This procedure increases likelihood of homogeneous parcels (for justification on the use of parcels, see Little, Rhemtulla, Gibson, & Schoemann, 2013; Rhemtulla, 2016).

Psychometric properties of SCT parcels

A multiple indicator (three SCT parcels) by multiple occasion (three assessments) by multiple source (mothers, fathers, primary teachers, and secondary teachers) CFA was used to determine the invariance of like-parcel loadings and intercepts across sources and occasions. Invariance of like-indicator loadings and intercepts would establish measurement invariance of SCT across occasions and sources at the parcel level, thus providing justification for the multiple source LST analysis with parcels.

Single source latent state trait analyses

A single source LST analysis with parcel-specific trait factors was used to determine the proportion of trait (trait consistency) and state residual variance (occasion-specificity) for each source separately. The Mplus code for this model is available from the corresponding author.

Multiple source latent state trait analyses

A multiple source LST analysis was used to determine the proportion of trait consistency, unique trait consistency, shared occasion-specificity, and unique occasion-specificity for the three comparison sources (fathers, primary teachers, and secondary teachers) relative to mothers. This analysis used mothers as the reference source due to the use of mothers as the reference source in the earlier studies (Litson et al., 2016; Preszler et al., 2016). The Mplus code for this model is available from the corresponding author.

Results

Measurement Models

Item level CFA

The item level invariance analysis indicated a close fit for the four sources and three occasions model, χ² (4302) = 7181, CFI = .975, TLI = .974, and RMSEA = .029 (.028, .031). The constraints on like-item loadings and thresholds across the four sources and three occasions did not result in a decrement in the CFI value greater than .01, χ² (4799) = 7909, CFI = .973, TLI = .975, and RMSEA = .028 (.027, .029). The single set of eight unstandardized item-factor loadings from the final model was used to assign the eight SCT items to the three parcels in the manner described earlier. Table 1 shows the items in each parcel.

Parcel level CFA

The parcel level invariance analysis indicated a close fit for the four sources and three occasions model, χ² (438) = 583, CFI = .993, TLI = .990, and RMSEA = .020 (.016, .024). The model with constraints on like-parcel loadings and the model with constraints on like-parcel loadings and intercepts also resulted in close fits along with no decrements in CFI greater than .01, χ² (460) = 636, CFI = .991, TLI = .988, and RMSEA = .022 (.01, .024), χ² (482) = 681, CFI = .990, TLI = .987, and RMSEA = .023 (.019, .026), respectively.

Latent factor means and correlations

Mothers, fathers, and secondary teachers did not show a significant change in factor means across the three occasions (ps > .26). Primary teachers, however, showed a statistically significant decrease in factor means across occasions (ps < .05, latent Cohen d values from .06 to .14). Strong measurement invariance across sources and occasions along with stable means for three of the four sources and only trivial mean decreases for the fourth source indicate LST analyses were appropriate (Geiser et al., 2015). Table S2 shows the SCT factor correlation within (.65 to .81) and across settings (.33 to .47) and occasions (.53 to .86).

Single Source Latent State-Trait Analyses

The four single source LST analyses resulted in close fit (CFIs ≥ .993, TLIs ≥ .992, and RMSEAs ≤ .037). Table 2 shows the estimated trait variance (trait consistency), occasion-specific variance, and reliability coefficients for the four analyses. SCT ratings for mothers, fathers, and secondary teachers contained substantially more trait variance (Ms = 75%, 77%, and 70%, respectively) than occasion-specific variance (Ms = 25%, 23%, and 30%, respectively) with the ratings for secondary teachers containing slightly more trait than occasion-specific variance (Ms = 54% and 46%, respectively).

Multiple Source Latent State-Trait Analyses

The multiple source LST model resulted in a close fit, χ² (561) = 819, CFI = .987, TLI = .986, RMSEA = .024 (.020, .027). Table 3 shows the shared trait consistency, unique trait consistency, shared occasion-specificity, and unique occasion-specificity for fathers, primary teachers, and secondary teachers with mothers. In terms of shared and unique trait variance, fathers shared a high proportion trait consistency with mothers (M = 80%) with a corresponding low amount of unique trait consistency (M = 20%). This indicates strong convergent validity for trait variance within the home. In contrast, primary and secondary teachers shared a low proportion of trait consistency with mothers (Ms = 29% and 22%, respectively) with a correspondingly high proportion of unique trait consistency (Ms = 71% and 78%, respectively). These results indicate low convergent validity for trait variance for primary and secondary teachers with mothers. For the occasion-specific variance (shared occasion-specificity and unique occasion-specificity), fathers, primary teachers, and secondary teachers all showed high unique occasion-specificity (Ms = 72%, 100%, and 100%, respectively) relative to shared occasion-specificity with mothers (Ms = 37%, 0%, and 0%, respectively).

Table 3.

Average Trait Consistency, Occasion-Specificity, and Reliability Estimates from Single Source Latent State-Trait Analyses on the Sluggish Cognitive Tempo Symptoms

Source	Trait Consistency	Occasion-Specificity	Reliability
Mothers	.75 (.72, .81)	.25 (.19, .28)	.80 (.76, .88)
Fathers	.77 (.68, .83)	.23 (.17, .32)	.81 (.76, .88)
Primary Teachers	.70 (.66, .75)	.30 (.25, .34)	.92 (.88, .95)
Secondary Teachers	.54 (.51, .60)	.46 (.40, .49)	.92 (.88, .96)

Note. Entries indicate averages across parcels with ranges in parentheses.

The residual trait factor correlations among fathers, primary teachers, and secondary teachers provide additional information on whether the trait-like aspects of SCT ratings showed convergent validity within and across settings. The residual trait correlations indicate whether father and teacher reports share trait variance that these reports do not share with mothers. Primary and secondary teachers shared a high proportion of residual trait variance (M = 75%) whereas fathers did not share a meaningful amount of residual trait variance with primary and secondary teachers (Ms = 1%). These results indicate strong convergent validity within settings (between primary and secondary teachers) but non-existent convergent validity across settings (fathers with primary and secondary teachers) for residual trait aspects of SCT.

The occasion-specific residual factor correlations for fathers, primary teachers, and secondary teachers indicate relationships among occasion-specific residual factor scores pertaining to non-reference sources after occasion-specific variance that non-reference sources share with mother reports have been partialled out. The average amount of shared occasion-specific residual variance for fathers with teachers was less than 1% with the average amount for primary with secondary teachers being 15% across the three occasions. These results indicate no shared occasion-specific residual variance for fathers with primary and secondary teachers and only a small amount for primary with secondary teachers.

Discussion

This is the first study to determine the amount of trait (consistency across occasions) and occasion-specific variance in SCT symptoms, a question examined with Spanish children with four different sources (mothers, fathers, primary teachers, secondary teachers) over three measurement occasions. SCT symptoms showed a substantial amount of trait variance relative to occasion-specific variance for the four sources, especially mothers, fathers, and primary teachers. These findings provide the strongest evidence to date of the trait-like nature of SCT in children (Barkley, 2016; Becker, Leopold et al., 2016).

This study also determined the amount of trait and occasion-specific variance in SCT symptoms that fathers, primary teachers, and secondary teachers either shared or did not share with mothers as well as with each other. This question deals with the convergent validity and the source-specificity of the trait and occasion-specific variance within and across settings (i.e., agreement and disagreement for trait and occasion-specific variance). Strong convergent validity occurred for trait variance within settings (mothers with fathers, primary with secondary teachers) and weak convergent validity for trait variance across settings with no convergent validity for occasion-specific variance across settings. How do these results for SCT symptoms compare to the results for ADHD and ODD symptoms? We now turn to these questions.

Trait and Occasion-Specific Variance in SCT, ADHD-IN, ADHD-HI, and ODD Symptoms

Given earlier studies used the same four sources, children, and one-year interval to determine trait and occasion-specific variance in ADHD and ODD symptom ratings (Litson et al., 2016; Preszler et al., 2016), it was possible to compare the trait-like nature of ADHD and ODD to SCT symptoms. SCT symptoms contained similar amounts of trait variance (consistency across occasions) as the ADHD-IN (69% vs. 69%, averages for four sources), ADHD-HI (69% vs. 69%), and ODD symptoms (69% vs. 63%). SCT symptoms were thus as trait-like as ADHD-IN, ADHD-HI, and ODD symptoms for the four sources across the three measurement occasions.

Convergent validity of trait variance in SCT, ADHD-IN, ADHD-HI, and ODD ratings across the four sources was more complex. The amount of trait variance that fathers shared with mothers for SCT, ADHD-IN, and ADHD-HI symptom ratings was substantial and identical (80%, 79%, and 80%, respectively) with these values slightly larger than trait variance fathers shared with mothers for ODD (68%). Convergent validity, however, varied substantially across home and school. Primary and secondary teachers shared 46% to 55% of their trait variance with mothers for ADHD-IN symptoms, 42% to 55% for ADHD-HI symptoms, 31% to 32% for ODD symptoms, and 22% to 29% for SCT symptoms. Trait variance in SCT ratings showed less consistency across home and school than trait variance in ADHD symptom ratings. This across setting difference in trait consistency in SCT and ADHD symptoms, especially ADHD-IN symptoms, requires additional study. However, it should also be noted that the average convergent validity of the residual trait variance for primary with secondary teachers was 75%, 75%, and 75% for SCT, ADHD-IN, and ADHD-HI, respectively, thus indicating strong and consistent convergent validity for the residual trait variance within the school setting for the three symptom dimensions.

These three sets of findings—(1) SCT symptoms showed the same level of trait consistency across occasions as ADHD-IN symptoms, (2) SCT symptoms showed the same level of convergent validity for trait consistency within home and school settings as ADHD-IN symptoms, and (3) SCT symptom ratings showed less convergent validity for trait consistency across home and school settings than ADHD-IN symptoms—have important implications. Taken together, these findings provide strong support for the consistency of SCT symptoms across occasions, suggesting that SCT symptoms in children are not likely to be transient. This is important to note given research linking SCT to functional impairment (Becker, Leopold, et al., 2016). In addition, our findings suggest that is especially important that research on SCT obtain information within both the home and school settings. Although SCT symptoms are invariant across parent and teacher ratings (Burns et al., 2016), it is nonetheless possible that specific features of SCT are more clearly observed in certain contexts and, when observed, especially linked to impairment. For example, Burns et al. (2016) found teacher-rated SCT was a clearer predictor of parent-rated impairment than vice versa. It may be that teachers are optimal raters when assessing SCT. This is not to say that parents (and even youth themselves) are not good raters of SCT, but rather that it is especially important to gather teacher ratings of SCT whenever possible. Of course, gathering both parent and teacher ratings of ADHD is also recommended (American Academy of Pediatrics, 2011), but findings from the present study indicate that SCT symptoms may be more context-specific than ADHD symptoms. Additional research supporting this possibility would point to the need to better uncover the unique context-dependent aspect(s) of SCT that may be useful for informing etiological models as well as intervention efforts.

Use of Multiple Source LST Models to Study Informant Discrepancies

Understanding the extent to which different informants agree or disagree in their perspectives of psychopathology and behavior is an important research area (Achenbach, 2016; Burns et al., 2008; De Los Reyes, 2013; De Los Reyes, Alfano, Lau, Augenstein, & Borelli, 2016; De Los Reyes & Ohannessian, 2006; De Los Reyes et al., 2015). One focus of informant discrepancy research is to quantify the level of consistency (convergence) and inconsistency (discrepancy) for multiple sources within and across settings, and a second focus is to evaluate the extent to which discrepancies are theoretically and clinically meaningfully as opposed to representing bias or measurement error (De Los Reyes, 2013). Our study focused on the first issue of whether there is convergence or discrepancy within and across settings for parent and teacher ratings of SCT. The multiple source LST model applied in this study allows for a more fine-grained analysis of informant discrepancies than previously applied methods by considering both trait (temporally stable) and state (more fluctuating) aspects of behavior and examining informant discrepancies for both aspects separately. With a minimum of two sources and two occasions of measurement, it is possible to determine the amount of trait variance (consistency across occasions) that is either shared (convergence or shared trait consistency) or not shared (discrepant or unique trait consistency) among sources. We found a high degree of convergence within home and school contexts, but not between home and school contexts. It will be important for future research to evaluate why cross-setting discrepancies exist in ratings of SCT, as well as how such discrepancies relate to functioning. It is possible that the different contexts in which parents and teachers observe children completing academic work and engaging in social relationships may contribute to not only differences in ratings of SCT, but also in the magnitude to which SCT is associated with impairment. The multiple source LST model provides a useful approach to these and other questions that can advance the understanding of agreements and disagreements among multiple sources in relation to SCT and other psychopathology symptom dimensions.

Limitations and Future Directions

Limitations include the one-year longitudinal span and narrow age range of participants in this study. It would also be useful to include self-ratings of SCT symptoms by children and adolescents in order to determine the convergent validity for trait variance of self-ratings with trait variance in ratings by parents and teachers. Likewise, in this study a unidimensional measure of SCT was used, but there is some evidence that SCT may have a hierarchical structure with lower-order factors of the higher-order, general factor (Smith et al., 2016). It is possible that SCT subfactors differ in their etiologies and the extent to which they are more state-like or trait-like, and this too is another area for future research. Finally, it would be important to investigate these questions in clinical samples of children and adolescents, particularly since SCT symptom ratings may have more occasion-specific variance in youth who are experiencing acute stressors (Becker, Withrow et al., 2016).

Conclusions

Single and multiple source LST models were used to determine the amount of trait and occasion-specific variance in SCT symptom ratings for mothers, fathers, primary teachers, and secondary teachers. SCT was strongly trait-like across time and within settings but showed low consistency across settings, suggesting that SCT may be a more setting-specific trait than ADHD-IN. These findings indicate that SCT symptoms should not be dismissed as transient or inconsequential; rather, these findings point to the importance of developing and evaluating clinical interventions that aim to modify the presence and developmental trajectory of SCT. This is particularly important given a growing body of research linking SCT to a range of functional impairment outcomes, both concurrently and over time (Becker, Leopold, et al., 2016).