Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Nov 1.
Published in final edited form as: Psychol Assess. 2020 Jul 30;32(11):1047–1056. doi: 10.1037/pas0000946

Assessing Sluggish Cognitive Tempo and ADHD Inattention in Elementary Students: Empirical Differentiation, Invariance across Sex and Grade, and Measurement Precision

Stephen P Becker 1,2, Kandace W Mossing 1, Allison K Zoromski 1,2, Aaron J Vaughn 1,2, Jeffery N Epstein 1,2, Leanne Tamm 1,2, G Leonard Burns 3
PMCID: PMC8019062  NIHMSID: NIHMS1685427  PMID: 32730075

Abstract

To advance the research examining the sluggish cognitive tempo (SCT) construct, a key priority has been to develop assessment tools that are reliable and valid. The current study builds upon existing work by conducting the most thorough psychometric evaluation to date of the teacher-reported Child and Adolescent Behavior Inventory (CABI) SCT and attention-deficit/hyperactivity disorder inattention (ADHD-IN) modules in a large sample of elementary students. Participants were 7,613 students (grades 2–5; 50.3% boys) attending 24 elementary schools in three school districts. Teachers (N = 398) provided ratings of SCT, ADHD-IN, academic impairment, and social impairment. An a priori two-factor model with cross-loadings found the SCT items to demonstrate excellent structural validity with ADHD-IN items. The measurement properties of the SCT and ADHD-IN constructs were also invariant across sex and grade. SCT and ADHD-IN were both uniquely associated with academic and social impairment. Graded response item response theory analysis indicated that the SCT and ADHD-IN scales provided a high level of information and precision. The current study replicates and extends previous research and provides the strongest psychometric evidence to date of teacher-rated SCT using the CABI. The teacher-report CABI may be especially useful in the school-based screening of SCT and ADHD-IN.

Keywords: assessment, attention-deficit/hyperactivity disorder, impairment, measure, sluggish cognitive tempo, teachers

Sluggish cognitive tempo (SCT) refers to a set of behavioral symptoms characterized by mental confusion, excessive daydreaming, lethargy, staring, and slowed behavior/thinking (Becker, Leopold, et al., 2016). It is clear that SCT symptoms are distinct from attention-deficit/hyperactivity disorder inattentive (ADHD-IN) symptoms, and a growing body of research also links SCT symptoms to academic, social, and emotional impairments (for reviews, see Barkley, 2014; Becker & Barkley, 2018; Becker, Leopold, et al., 2016). Accordingly, it has been suggested that SCT may be a construct of transdiagnostic importance (Becker & Willcutt, 2019), informative for understanding ADHD heterogeneity (Nigg, Karalunas, Feczko, & Fair, 2020), or a psychiatric disorder in its own right (Barkley, 2014). For SCT-related research to advance, a key priority has been to develop assessment tools that are reliable and valid. A number of SCT-specific rating scales have been developed over the past decade, including the SCT module of the Child and Adolescent Behavior Inventory (CABI; Burns, Lee, Servera, McBurnett, & Becker, 2015). The CABI SCT module items were selected to reflect the SCT items shown to be optimal for distinguishing SCT from ADHD-IN in a meta-analysis (Becker, Leopold, et al., 2016), in addition to items reflecting mental confusion with growing empirical support as part of the SCT construct (McBurnett et al., 2014). To date, the psychometric properties of the teacher version of the CABI SCT module have been examined in two studies (Becker, Burns, Schmitt, Epstein, & Tamm, 2019; Sáez, Servera, Becker, & Burns, 2019). The current study builds upon this work by conducting the most thorough psychometric evaluation to date of the teacher-reported CABI SCT scale in a large sample of elementary students.

The two previous studies provide initial support for the teacher-report CABI SCT scale as distinct from ADHD-IN symptoms and uniquely associated with functional impairment. One study was conducted in Spain and included 196 teachers who provided ratings on 1,773 children in 3rd–6th grades (Sáez et al., 2019). The second study was conducted in the United States and included 67 teachers who provided ratings on 1,349 children in 2nd–5th grades (Becker et al., 2019). These studies examined 16 SCT items and 9 ADHD-IN items on the CABI, and both studies found 15 SCT items demonstrated excellent structural validity (substantial loadings on the SCT factor and low loadings on the ADHD-IN factor); in both studies, one SCT item related to motivation failed to demonstrate structural validity (higher loadings on the ADHD-IN than SCT factor). In addition, both studies found SCT symptoms to be uniquely associated with teacher ratings of academic and social impairment (Becker et al., 2019; Sáez et al., 2019). Becker et al. (Becker et al., 2019) also found that the 15 SCT items represented a unidimensional construct that was invariant across sex. The first goal of the present study was to replicate these previous studies by examining the factor structure, dimensionality, and invariance across sex of the teacher-report CABI SCT module.

The present study also extends existing research in two important ways. First, invariance across grade is examined. Establishing invariance across elementary grades using the teacher-report CABI is important since (1) most studies of SCT have been conducted in school-aged children (Becker, 2017; Becker, Leopold, et al., 2016), (2) schools are often at the front-line in mental health screening (Siceloff, Bradley, & Flory, 2017; Weist, Rubin, Moore, Adelsheim, & Wrobel, 2007), and (3) children are often first referred for further evaluation for attentional, emotional, and behavioral difficulties while in elementary school (Lloyd, Kauffman, Landrum, & Roe, 1991; Severson, Walker, Hope-Doolittle, Kratochwill, & Gresham, 2007). Establishing invariance across grade for teacher ratings of SCT ensures that clinicians and school mental health professionals can have confidence that the underlying construct is the same for students in different grades, and will similarly help ensure that teacher assessments that are completed as students progress through school are comparable across different grades. The previous studies examining the teacher-report CABI SCT module had insufficient sample sizes for examining invariance across specific grades. In addition, the longitudinal studies that have demonstrated the temporal invariance of SCT (Burns, Becker, Geiser, Leopold, & Willcutt, 2019; Dvorsky, Becker, Tamm, & Willoughby, 2019; Leopold et al., 2016) used short, non-optimal measures of SCT and had far smaller samples than the current study. The current study builds upon these studies by being the first to examine whether SCT symptoms are invariant across grade when using the teacher-report CABI SCT scale in a large sample of elementary students.

The second way the current study extends previous research is by examining measurement precision. Whereas factor structure and invariance are based in classical test theory, measurement precision draws from item response theory (IRT) and refers to the degree to which a scale is effective for discriminating among individuals at critical segments along a construct’s dimension (Edelen & Reeve, 2007). As such, measurement precision is especially important when evaluating assessment tools that may be used for screening purposes. When conducting school-based screening for constructs such as SCT and ADHD-IN, it is crucial for a scale to have adequate precision across the clinically at-risk and elevated range of the constructs. However, no SCT measures have been examined for their measurement precision, and the current study does so with teacher ratings collected via school-wide screening.

Study Objectives

We previously reported on the teacher-report CABI SCT and ADHD-IN scales in an initial cohort of over 1,300 students (Becker, Burns, Schmitt, Epstein, & Tamm, 2019). Building from and extending this work, the current study used school-wide screening across six cohorts (including the initial cohort reported previously in Becker, Burns et al., 2019) to conduct the most rigorous psychometric evaluation yet of the teacher-report CABI SCT scale. Using teacher ratings of SCT and ADHD-IN in a sample of over 7,600 elementary students, the current study had five objectives:

  • 1

    Examine the structural validity of SCT and ADHD-IN symptoms. The first objective was to determine the structural validity of the 15 SCT and 9 ADHD-IN symptoms. Consistent with previous findings using the parent and teacher versions of the CABI (Becker et al., 2019; Burns & Becker, 2019; Sáez et al., 2019), SCT symptoms were predicted to have substantial loading on the SCT factor and much higher loadings on the SCT factor than the ADHD-IN factor. Similar structural validity results were expected for the ADHD-IN symptoms. The SCT and ADHD-IN symptoms were also expected to show structural validity for boys and girls separately.

  • 2

    Examine the dimensionality of SCT. The second objective was to determine if the SCT symptoms reflect a single factor or multiple factors. It was expected that the 15 items on the SCT module would be best represented as unidimensional (Becker et al., 2019).

  • 3

    Establish invariance of SCT and ADHD-IN symptoms across sex and grade. It was expected that the 15 SCT symptoms would show invariance across boys and girls. Consistent with previous research (Sáez et al., 2019), although boys were expected to have a significantly higher SCT factor mean than girls, the effect size was expected to be small (d < .20). The 15 SCT symptoms were also expected to show invariance across second to fifth grades with no differences in the SCT factor means across the grades.

The ADHD-IN symptoms were also expected to show invariance across boys and girls. Boys, however, were expected to have a moderately larger ADHD-IN factor mean than girls (d > .40). Measurement invariance was also expected across the four grades for the ADHD-IN symptoms with no significant differences in the ADHD-IN factor means.

  • 4

    Examine unique relations of SCT and ADHD-IN with social and academic impairment. Based on previous research (see Becker & Barkley, 2018), higher SCT scores were expected to be associated with significantly higher levels of social and academic impairment even after controlling for ADHD-IN scores, child sex, grade, and cohort. Higher ADHD-IN scores were also expected to have significant unique associations with social and academic impairment. Such results would indicate that SCT and ADHD-IN have significant associations with social and academic impairment independent of the other construct and control variables.

  • 5

    Evaluate the precision of SCT and ADHD-IN scores. Although the analyses associated with the above objectives provide a strong evaluation of the psychometric properties of the scores on the SCT scale (as well as the ADHD-IN scale), additional information on the ability of the SCT scale to measure SCT occurs with the application of the graded response IRT model to SCT symptom ratings. More specifically, test information and standard error functions from IRT analyses provide data on the information and precision of the total scores on the SCT scale across the SCT trait range. We expected the SCT and ADHD-IN scales to provide a high level of information and precision from slightly below their respective trait means to two standard deviations above their respective trait means. Such an outcome would provide additional support for the SCT and ADHD-IN scales for research and clinical screening purposes.

Methods

Participants

Participants included 7,613 students (grades 2 through 5) attending 24 elementary schools in three school districts in the Midwestern United States. Data were provided by 398 teachers (100 2nd grade teachers, 100 3rd grade teachers, 94 4th grade teachers, and 104 5th grade teachers). As expected in a community sample, there was an equivalent percentage of girls (49.7%) and boys (50.3%) children. With regards to grade, 23.2% of the sample were in the 2nd grade (50.3% girls), 24.3% of the sample were in 3rd grade (47.3% girls), 26.6% of the sample were in 4th grade (51.6% girls), and 25.9% of the sample were in 5th grade (49.6% girls). Limited demographic information was available for individual students. However, the racial/ethnic distribution in each school district is approximately 85% non-Hispanic White and 25–55% of students in the districts receive free or reduced lunch.

Procedures

Data was collected as part of a larger study, which was approved by Cincinnati Children’s Hospital Medical Center Institutional Review Board and the school districts where data were collected. After obtaining permission from each school’s principal, opt-out letters were sent to the 10,275 families of students in the 2nd–5th grades in the 24 elementary schools participating in this study. The letter stated that the child’s school was partnering in a research study to learn more about student behavior in the classroom and its impact on academic, social, and emotional functioning. Families were informed that if they did not opt out of the study, their child’s teacher would complete a rating scale assessing their child’s concentration as part of the screening for a broader study. The letter explicitly stated that families were not obligated to participate in the screening process, and included instructions to opt-out, with multiple ways of opting-out available (including e-mailing study staff or returning the letter to school with an opt-out box checked). Parents of 551 children opted out of the study. Additionally, 255 non-English speaking families were excluded as a part of eligibility criteria for the full study. Teachers were also provided a written overview of the study and given the option of not participating; 56 of the 454 2nd–5th grade teachers, teaching 1,245 students, declined to participate. Remaining teachers completed a survey via Research Electronic Data Capture (REDCap), which is a secure, web-based application designed to support data capture for research studies (Harris et al., 2009). The survey contained a list of all students listed on the school’s classroom roster at the start of the school year whose parents did not opt out of the study. Four initial questions asked teachers to only complete the survey for each student if the student (1) was still in the classroom, (2) did not have significant visual, hearing, or speech impairments that were not corrected with assistive devices (e.g., glasses, hearing aids), (3) was not known to be on the autism spectrum, and (4) did not spend the majority of the day outside of the general classroom (e.g. student has special needs that prevent inclusion in general education) which further reduced the sample by 418 children. Teachers then completed the SCT, ADHD-IN, and impairment items on the CABI (described below). Fourteen teachers did not complete surveys for all of the students in their classrooms, further reducing the sample size by 193 students. This resulted in the final sample size of 7,613 participants. Each teacher rated an average of 19.13 (SD = 4.86) children (range = 1–29 students). Teachers were compensated $5 for each survey they completed.

Measure

Child and Adolescent Behavior Inventory (CABI) (Burns et al., 2015).

The teachers completed the SCT (15 symptoms), ADHD-IN (9 symptoms), social impairment (2 items, quality of interactions with teachers and peers), and academic impairment (4 items, completion of homework/classwork, reading skills, arithmetic skills, and writing skills) modules of the CABI – Teacher Version. As noted above, the SCT module was initially comprised of 16 items, but one item related to motivation failed to show discriminant validity in previous studies (Becker et al., 2019; Sáez et al., 2019) and was therefore not used in this study. The CABI also includes modules assessing other psychopathology dimensions (e.g., ADHD hyperactivity-impulsivity, oppositional defiant disorder, anxiety, depression), but due to length and teacher burden considerations we did not use the full CABI in this study. The SCT and ADHD-IN symptoms were rated on a 6-point scale for the past month (almost never [never or about once per month], seldom [about once per week], sometimes [several times per week], often [about once per day], very often [several times per day], and almost always [many times per day]). The social and academic impairment items were rated on a 7-point scale (i.e., severe difficulty, moderate difficulty, slight difficulty, average performance [average interactions] for grade level, slightly above average, moderately above average, and excellent performance [excellent interactions] for grade level). The academic and social impairment items were reverse keyed, so higher scores represent higher impairment. Cronbach’s alpha for scores on the SCT, ADHD-IN, social impairment, and academic impairment scales in the current study were .97, .97, .93, and .93, respectively. Alphas for boys (girls) were .97 (.97), .96 (.97), .93 (.92), and .94 (.93) for the scores from the four scales, respectively. Table 1 provides descriptive statistics for boys and girls on the four CABI modules. The wording of the 15 SCT items are provided in Table 2, and the full CABI measure is available from the corresponding author.

Table 1.

Measure (Manifest Variables) Descriptive Statistics

Girls (n = 3787) Boys (n = 3826)
M SD SE Range M SD SE Range d (95% CI)
Sluggish Cognitive Tempo 0.73a 1.01 0.02 0 to 5 0.95b 1.15 0.02 0 to 5 .20 (.15, .24)
ADHD-Inattention 0.79a 1.13 0.02 0 to 5 1.28b 1.41 0.02 0 to 5 .39 (.34, .43)
Social Impairment 2.15a 1.37 0.02 0 to 6 2.49b 1.39 0.02 0 to 6 .25 (.20, .29)
Academic Impairment 2.73a 1.51 0.02 0 to 6 2.97b 1.51 0.02 0 to 6 .15 (.11, .20)
Open in a new tab

Note. Sluggish cognitive tempo and ADHD-inattention were rated on a 0 to 5 scale whereas social and academic impairment were rated on a 0 to 6 scale. Row means with different superscripts differ a p < .001. Higher scores represent greater symptom occurrence and higher levels of impairment. ADHD = attention-deficit hyperactivity disorder; d = Cohen’s d for manifest variable means (boys scores were higher than girls).

Table 2.

Standardized Primary and Secondary Factor Loadings (SEs)

SCT Factor ADHD-IN Factor
Sluggish Cognitive Tempo Symptoms
1. Behavior is slow (e.g., sluggish) 0.92 (.01)* −0.03 (.01)ns
2. Lost in a fog 0.94 (.01)* 0.02 (.01)ns
3. Stares blankly into space 0.94 (.01)* 0.01 (.01)ns
4. Drowsy or sleepy (yawns) during the day 1.08 (.01)* −0.23 (.02)*
5. Daydreams 0.87 (.01)* 0.06 (.01)*
6. Loses train of thought 0.78 (.01)* 0.18 (.01)*
7. Low level of activity (e.g., underactive) 1.05 (.01)* −0.21 (.02)*
8. Gets lost in own thoughts 0.81 (.01)* 0.12 (.01)*
9. Easily tired or fatigued 1.13 (.02)* −0.27 (.02)*
10. Forgets what was going to say 0.79 (.01)* 0.13 (.02)*
11. Easily confused 0.73 (.01)* 0.23 (.02)*
12. Spaces or zones out 0.81 (.01)* 0.16 (.01)*
13. Gets mixed up 0.70 (.01)* 0.27 (.02)*
14. Thinking is slow 0.72 (.01)* 0.22 (.02)*
15. Difficulty expressing thoughts (e.g., gets tongue-tied) 0.65 (.02)* 0.24 (.02)*
ADHD-Inattention Symptoms
1. Close attention 0.17 (.01)* 0.78 (.01)*
2. Sustaining attention 0.08 (.01)* 0.89 (.01)*
3. Listen 0.20 (.02)* 0.72 (.02)*
4. Follow through 0.10 (.02)* 0.87 (.01)*
5. Organizational skills −0.04 (.02)ns 0.98 (.02)*
6. Avoids tasks requiring sustained effort 0.10 (.02)* 0.83 (.02)*
7. Loses things −0.06 (.02)ns 0.99 (.02)*
8. Easily distracted 0.00 (.00)ns 0.93 (.01)*
9. Forgetful 0.23 (.02)* 0.73 (.01)*
Open in a new tab

Note. N = 7613. Each symptom was rated on a 6-point scale. The complete wording of the ADHD-IN items is available from the corresponding author. Partial standardized regression coefficients can be larger than one with cross-loadings and correlated factors. ADHD-IN = attention-deficit/hyperactivity disorder inattention. SCT = sluggish cognitive tempo.

*

p < .001;

ns = non-significant (p > .001).

Analytic Approach

Estimation.

The Mplus statistical software (Version 8.4) (Muthén & Muthén, 1998–2019) was used for factor, invariance, and regression analyses. The items were treated as categorical indicators with the use of the robust weighted least squares estimator (WLSMV).

Clustering.

Given the children were clustered within teachers, the Mplus type = complex option was used to take into account the clustering to obtain adjusted parameter standard errors and chi-square tests of model fit.

Criteria for model fit.

Global model fit was evaluated with the comparative fit index (CFI, acceptable fit ≥.90 and close fit ≥.95) and the standardized root-mean-square residual (SRMR, acceptable fit ≤.08 and close fit ≤.05) (Little, 2013). Localized ill fit was also evaluated with an inspection of the residual correlational matrix (i.e., the number of residuals greater than .10, see Kline, 2016 for a discussion of the inspection of the residual matrix to assess localized ill fit).

Structural validity of SCT and ADHD-IN symptoms.

An a priori two-factor model with cross-loadings was first applied to the SCT and ADHD-IN symptoms (Asparouhov & Muthén, 2009). These analyses allowed the SCT symptoms to cross-load on the ADHD-IN factor and the ADHD-IN symptoms to cross-load on the SCT factor. These analyses determined the structural validity of the SCT and ADHD-IN symptoms (i.e., each symptom should have a substantial loading on its primary factor and a low loading on the secondary factor). Figure 1 shows this factor analytic model.

Figure 1.

Figure 1.

Open in a new tab

Factor Analytic Model. All 15 SCT symptoms and all nine ADHD-IN symptoms were included in the model (only the first and last SCT and ADHD-IN symptoms are shown in the figure). The model was a priori restricted to two factors. The thinner paths in the model represent the cross-loadings. The items were treated as categorical indicators. The five thresholds and one latent response variable underlying each item are not shown in the model.

Dimensionality of SCT.

An exploratory factor analysis was applied to the 15 SCT symptoms to determine if the SCT dimension was better viewed as a unidimensional or multidimensional construct (i.e., daydreaming, mental confusion, and under-arousal dimensions).

Invariance of SCT across child sex and grade.

Invariance analyses were used to determine the invariance of SCT like-symptom loadings and like-symptom thresholds across girls and boys. The Mplus difference test was used to determine if the measurement model with constraints on like-symptom loadings and thresholds resulted in a significant decrement in fit relative to the model without the constraints on like-symptom loadings and thresholds. Alpha level was set at p < .001 for these tests given the large sample size (Little, 2013). If there was measurement invariance for the SCT symptoms, then the invariance of SCT factor means and variances were evaluated across boys and girls. These same procedures were used to evaluate the invariance of the 15 SCT symptoms across the second, third, fourth, and fifth grades. Although our primary interest was invariance of the SCT symptoms, we also used the same procedures to evaluate the invariance of the ADHD-IN symptoms to ensure that both symptom sets were invariant across sex and grade.

Unique associations of SCT and ADHD-IN factors with social and academic impairment.

A structural regression model (Asparouhov & Muthén, 2009) was used to determine the unique associations of the SCT and ADHD-IN factors with social and academic impairment (i.e., the social and academic impairment factors were regressed on the SCT and ADHD-IN factors with SCT symptoms allowed to cross-load on ADHD-IN factor and ADHD-IN symptoms allowed to cross-load on the SCT factor). This model provides a more accurate estimate of the partial regression coefficients when the cross-loadings on the predictor factors are not zero (Asparouhov & Muthén, 2009). Cross-loadings, however, were not allowed on the outcomes, the academic and social impairment factors. This analysis controlled for the effect of the child sex, grade (second to fifth), and cohort (six cohorts) on the predictors and outcomes. Cohort was included as a covariate to account for any effects of school district or semester of assessment (fall vs. spring). Figure 2 shows this model.

Figure 2.

Figure 2.

Open in a new tab

Structural Regression Model. All 15 SCT symptoms and all nine ADHD-IN symptoms were included in the model (only the first and last SCT and ADHD-IN symptoms are shown in the figure). Thresholds and latent response variables are not shown.

Item response theory analysis.

Stata version 16.0 was used to apply the graded response IRT analysis to the SCT and ADHD-IN symptom ratings. For the SCT (and ADHD-IN) scale to be useful for clinical screening purposes, SCT (and ADHD-IN) scores should provide a high level of information and precision (low standard error) from approximately 1.50 to 2.00 standard deviation above the trait mean (1.50 standard deviations above the mean is usually considered the start of the clinical range on rating scales measures).1 These analyses also took into account that children were clustered within teachers.

Results

Missing Information

The covariance coverage was greater than 99% for all variances and covariances. Thus, there was little missing information.

Model Fit for the 15 SCT and 9 ADHD-IN Symptoms

The SCT and ADHD-IN two-factor model provided a close fit, χ2(229) = 12,200, p < .001, CFI = .972 and SRMR = .039. An inspection of the 276 correlational residuals in the residual matrix found only three greater than absolute value .10 (i.e., −.102, to −.111). These results indicate there was no major localized ill fit (Kline, 2016). The average loading of the 15 SCT symptoms on the SCT factor was .86 (SD = .15, range = 0.65 to 1.13) and .06 (SD = .18, range = −.27 to .27) on the ADHD-IN factor. For the ADHD-IN symptoms, the average loading of these symptoms on the ADHD-IN factor was .86 (SD = .10, range = .72 to .99) and .09 (SD = .10, range = −.06 to .23) on the SCT factor. Each of the SCT and ADHD-IN symptoms thus showed excellent structural validity. The correlation between the SCT and ADHD-IN factors was .80 (SE = .01). Table 2 shows the loadings for the total sample. Similar results occurred for boys and girls separately (close fit of the model, no major localized ill-fit, and convergent and discriminant validity of SCT and ADHD-IN symptoms). Table S1 shows the loadings separately for boys and girls.

Dimensionality of the SCT Construct

An exploratory factor analysis was applied to the 15 SCT symptoms to determine if there were strong multiple SCT factors. The first three eigenvalues were 12.22, 0.75, and 0.51. Although an inspection of the three-factor solution provided some evidence for daydreaming, mental confusion, and under-arousal SCT factors (see Table S2), the correlation between daydreaming and under-arousal factors was too high to support a distinction between these two factors (r = .87, SE = .01). The two-factor solution yielded weak (or no) discriminant validity for seven of the 15 SCT items. The eigenvalues along with the factor correlations and factor loadings suggest a single SCT factor was best with the teacher ratings for this sample.

Invariance of SCT

SCT invariance across sex.

A one factor model applied to the 15 SCT symptoms yielded a close fit for boys and girls with no major localized ill-fit, χ2 (180) = 9992, p < .001, CFI = .976 and SRMR = .044. The model with constraints on like-symptom loadings and like-symptom thresholds also yielded a close fit, χ2(253) = 10030, p < .001, CFI = .976 and SRMR = .045. The model with constraints on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrease in fit relative to the model without the constraints, χ2 (73) = 105, p = .01. These findings support the measurement invariance of the 15 SCT symptoms across girls and boys. Boys had a significantly (p < .001) higher SCT factor mean than girls with the effect size being small, latent d value = .18, SE = .05. Boys and girls did not differ on their variability on the SCT factor, χ2 (74) = 90, p = .11, with the reliability coefficient (omega) for girls and boys being .99 for both.2

SCT invariance across grades.

A single factor model applied to the 15 SCT symptoms across the second through fifth grades yielded a close fit with no major localized ill fit, χ2 (360) = 7550, p < .001, CFI = .977 and SRMR = .045. The model with constraints on like-symptom loadings and like-symptom thresholds also yielded a close fit, χ2(579) = 7564, p < .001, CFI = .977 and SRMR = .045. This model with constraints on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrease in fit relative to the model without the constraints, χ2 (219) = 264, p = .02. Constraints on the SCT factor mean across the four grades (four SCT factor means held equal) with constraints still on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrement in fit, χ2 (222) = 248, p = .06. In addition, constraints on the SCT factor variances across the four grades with constraints still on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrement in fit, χ2 (222) = 256, p = .06. The SCT symptoms thus showed invariance of like-symptom loadings and thresholds across the four grades with no significant differences in the factor means and variances across grades.

Invariance of ADHD-IN

ADHD-IN invariance across sex.

A factor analysis applied to nine ADHD-IN symptoms yielded a close fit for boys and girls with no localized ill-fit, χ2 (54) = 3036, p < .001, CFI = .992 and SRMR = .019. The model with constraints on like-symptom loadings and like-symptom thresholds also resulted in a close fit with no localized ill-fit, χ2 (97) = 2642, p < .001, CFI = .993 and SRMR = .020. The model with constraints on like-symptom loadings and thresholds, however, resulted in a significant decrement in fit relative to the model without the constraints, χ2 (43) = 96, p < .001. An inspection of the modification indices for thresholds indicated that girls had significantly higher thresholds than boys for the ADHD-IN symptom shows poor organizational skills. The release of the constraints on these five thresholds for this symptom as well as the constraint on the loading on this symptom across boys and girls yielded a model with no significant decrement in fit, χ2 (38) = 49, p = .10. Eight of the nine ADHD-IN symptoms thus showed invariance of like-symptom loadings and thresholds across boys and girls. Although boys had a significant higher ADHD-IN factor mean than girls, latent d = .40, SE = .05, p < .001, boys and girls did not differ significantly in their variability on the ADHD-IN factor, χ2 (39) = 41, p = .37. The reliability coefficient (omega) for boys and girls was .98 for both.

ADHD-IN invariance across grades.

A factor analysis applied to the nine ADHD-IN symptoms across the second through fifth grades yielded a close fit with no localized ill fit, χ2 (108) = 2339, p < .001, CFI = .992 and SRMR = .019. The model with constraints on like-symptom loadings and like-symptom thresholds yielded a close fit with no localized ill fit as well, χ2(237) = 2107, p < .001, CFI = .994 and SRMR = .020. In addition, the model with constraints on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrease in fit relative to the model without the constraints, χ2 (129) = 184, p = .001. Constraints on the ADHD-IN factor mean across the four grades (IN factor mean held equal) with constraints still on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrement in fit, χ2 (132) = 169, p = .02. In addition, constraints on the ADHD-IN factor variances across the four grades with constraints still on like-symptom loadings and thresholds did not result in a significant (alpha criterion of p < .001) decrement in fit, χ2 (132) = 181, p = .003. The ADHD-IN items thus showed invariance of like-item loadings and thresholds across the four grades with the factor means and factor variances not differing significantly across grades as well.

SCT and ADHD-IN in Relation to Social and Academic Impairment

Correlation analyses.

A SCT, ADHD-IN, social impairment, and academic impairment four-factor model yielded close fit, χ2(377) = 18471, p < .001, CFI = .962 and SRMR = .039. The factor correlations for SCT and ADHD-IN with academic impairment were .67 (SE = .01) and .74 (SE = .01), respectively, with this difference being statistically significant (p < .001). The correlations of SCT and ADHD-IN with social impairment were .52 (SE = .02) and .58 (SE = .01) with this difference also being statistically significant (p < .001).

Structural regression analyses.

Higher scores on the SCT factor were associated with significantly higher scores on the social impairment factor (β = .17, SE = .03, p < .001) and the academic impairment factor (β = .19, SE = .02, p < .001) even after controlling for the ADHD-IN factor, child cohort, sex, and grade. Higher scores on the ADHD-IN factor were also still associated with higher scores on the social impairment (β = .44, SE = .03, p < .001) and academic impairment (β = .60, SE = .02, p < .001) factors after controlling for the SCT factor, child cohort, sex, and grade. The β values are partial standardized regression coefficients.

Graded Response IRT Analyses

Figure 3 shows the test information and standard error functions for the total scores on the SCT and ADHD-IN scales. Both scales provided a high level of information and precision (low standard error) from slightly below their respective trait means to approximately 2.00 standard deviations above their respective trait means. Tables S3 and S4 show the item parameter estimates (discrimination and difficulty values for the SCT and ADHD-IN items). Figures S1 and S2 show the test information and standard error functions for boys and girls for SCT and ADHD-IN scores. The results for boys and girls separately were similar as for the total sample. Tables S5 to S8 show the item parameter estimates for the SCT and ADHD-IN items for boys and girls.

Figure 3.

Figure 3.

Open in a new tab

Test information and standard error functions for total scores on the sluggish cognitive tempo and ADHD-inattention scales (n = 7613). The test information function (TIF) curve is the high curve in the middle of the figure (y-axis on left) while the standard error curve is the low curve in the middle of the figure (y-axis on right). The TIF indicates how well the total scores on the SCT and ADHD-inattention scales measure their respective latent traits at various levels of the latent trait. Higher levels of information indicates better precision of measurement along the latent trait. Both scales provide a high level of information in the clinical screening range (i.e., 1.50 to 2.00 standard deviations above the trait mean).

Discussion

For research investigating the SCT construct to advance, reliable and valid assessment tools must be developed and evaluated. The Child and Adolescent Behavior Inventory (CABI) is unique in that it was directly informed by meta-analytic findings of SCT items that are distinct from ADHD-IN items (Becker, Leopold, et al., 2016). Using a sample of over 7,600 elementary students, the current study provides the strongest psychometric support yet for the teacher-report version of the CABI SCT scale.

The first aim of the current study was to use our large sample of teacher ratings to replicate previous research using the teacher-report CABI SCT and ADHD-IN scales (Becker et al., 2019; Sáez et al., 2019). Replicating psychometric properties soon after a measure has been developed allows for greater confidence in the measure before it is more widely used. In this study, we found clear support for an SCT factor that was empirically distinct, yet strongly correlated with, the ADHD-IN factor. The size of the sample and the school-wide screening procedures used bolsters confidence in the 15 SCT items as well-suited for assessing the SCT construct in the school setting. In addition, the SCT and ADHD-IN constructs both demonstrated invariance across sex and across grade in the present study, demonstrating that the underlying SCT construct is similar for boys and girls across 2nd–5th grades.

In addition, findings from this study indicate that SCT is best represented as a unidimensional construct. Whether or not SCT is best conceptualized as unidimensional or multidimensional remains unclear. In examining the face validity of the SCT items on the CABI scale, there does appear to be possible groupings of daydreaming, mental confusion, and under-arousal items, yet analyses in the present study suggested a single factor was best for the teacher ratings of SCT using this measure. Further, it is important to note that two studies that reported multiple subdimensions of SCT when using teacher ratings found only one “pure” SCT factor when ADHD symptoms were examined simultaneously with the SCT items (Jacobson et al., 2012; Penny, Waschbusch, Klein, Corkum, & Eskes, 2009). Thus, with few exceptions (McBurnett et al., 2014), the body of research conducted to date indicates that SCT may be best conceptualized as unidimensional, at least when using teacher ratings. It is important to further examine the dimensionality of SCT with other informants (e.g., parents, self-report), analytic methods (e.g., network analysis), and perhaps by examining a larger SCT item set with additional items capturing the daydreaming, mental confusion, and under-arousal aspects of SCT.

A unique feature of the present study was our examination of measurement precision using graded response IRT analyses. Findings from the present study indicate that the teacher-report CABI SCT and ADHD-IN scales provide a high degree of information and precision. In particular, the scales demonstrate precision from slightly below the trait mean to approximately two standard deviations above the trait mean. These findings indicate that these scales are effective for measuring SCT and ADHD-IN across the at-risk and clinically elevated ranges – precisely the levels desired for screening and identification purposes.

The demonstration of measurement precision is an important step before determining the potential benefit of including SCT in school-based prevention and intervention efforts. Universal screening is frequently conducted in schools, and teachers often make referrals for educational and mental health evaluations (Severson et al., 2007; Weist et al., 2007). SCT is not routinely assessed in school settings and the utility of including SCT in such screening and referral processes has not been examined. It is likewise unknown what school services and accommodations are most frequently given to students with elevated SCT, or whether including SCT in education referral and assessment processes is useful for informing intervention and accommodation decisions. For instance, although extended time is a very common accommodation given to children with ADHD and behavior challenges, the evidence supporting extended time is weak (Harrison, Bunford, Evans, & Owens, 2013); it has been suggested that children displaying SCT symptoms may be especially likely to benefit from extended time if slow work speed is contributing to academic impairment more so than distractibility (Becker, Ciesielski, et al., 2016).

In terms of intervention, a study of students in the same grades as the current study (2nd–5th grades) found some indication that lower parent-reported SCT symptoms are associated with higher rates of positive response to behavioral treatment for ADHD (Owens, Hinshaw, McBurnett, & Pfiffner, 2018). No study has examined teacher-reported SCT as either a predictor or outcome of school-based interventions. Although our study cannot directly speak to the possible clinical importance of including SCT in school-based assessment and intervention efforts, the findings demonstrating the measurement precision of the teacher-report SCT scale, coupled with findings supporting the scale’s reliability and validity (Sáez et al., 2019; Servera, Saez, Burns, & Becker, 2018), indicate that the scale should be useful for such efforts.

Strengths, Limitations, and Future Directions

Strengths of this study include the large sample size, school-wide screening procedures that reduce ascertainment bias, and rigorous analytic plan incorporating aspects of both classical test theory and item response theory. Still, several limitations are important to note. First, the measures included were limited to a brief teacher survey. It will be important for future studies to further evaluate the validity of the teacher-report CABI SCT and ADHD-IN scales with other informants (e.g., parents, children), other methods (e.g., academic achievement testing, specific learning disorder diagnoses), and a broader range of adjustment domains (e.g., homework/organization, emotion regulation, friendship). It also remains important to examine SCT in relation to cognitive functioning, including intelligence. Meta-analyses have found small-to-medium effect size associations between SCT and ADHD-IN dimensions with lower intelligence (weighted rs = −.24 and −.31, respectively) (Becker, Leopold, et al., 2016; Willcutt et al., 2012), with SCT having a weaker correlation than ADHD-IN with intelligence (Becker, Burns, Leopold, Olson, & Willcutt, 2018). The external correlates of SCT are largely unchanged whether or not intelligence is covaried in analyses (Becker et al., 2018; Willcutt et al., 2014), and it would be worthwhile for studies to confirm this when using a school-based sample as well as to examine whether children with low intellectual functioning have a different profile of SCT symptomatology. Other samples, such as children with mental health diagnoses (e.g., ADHD, autism), learning disorders, or sleep problems, may also have distinct SCT-related profiles that should be examined. Second, and relatedly, the cross-sectional nature of this study precludes drawing any temporal conclusions, and the predictive validity of the teacher-report CABI SCT and ADHD-IN scales remain unexamined. We were also unable to examine whether the CABI identifies children with clinically-elevated SCT as indicated by other SCT measures (e.g., clinical interview; parent-report scale). Third, the current study included children who were attending three school districts in the Midwest United States, in which the majority of the students are non-Hispanic White. Examining teacher ratings of SCT in a nationally representative sample of children is needed, particularly as this would allow for representative norms to be established. Finally, only children in 2nd-5th grades were included in this study, and there remains a need for research examining teacher-reported SCT in younger children and adolescents.

Conclusions

This study used teacher ratings of over 7,600 children in 2nd–5th grades to examine psychometric properties of the CABI SCT and ADHD-IN modules. The SCT items demonstrated excellent structural validity with ADHD-IN items. In addition, the measurement properties of the SCT and ADHD-IN constructs were invariant across sex and grade, and both SCT and ADHD-IN were uniquely associated with academic and social impairment. Finally, graded response IRT analyses indicated that the SCT and ADHD-IN scales provided a high level of information and precision. The current study replicates and extends previous research and provides the strongest psychometric evidence to date of teacher-rated SCT using the CABI. The teacher-report CABI may be especially useful in the school-based screening of SCT and ADHD-IN.

Supplementary Material

Supplemental Materials

Public Significance Statement:

For research investigating the sluggish cognitive tempo (SCT) construct to advance, reliable and valid assessment tools must be developed and evaluated. Using a sample of over 7,600 elementary students, the current study provides the strongest psychometric support yet for the teacher-report version of the Child and Adolescent Behavior Inventory (CABI) SCT scale.

Acknowledgements:

We are grateful to the school districts, principals, and teachers who participated in this study. This research was supported by award number R305A160064 (PI: Becker) from the Institute of Education Sciences (IES), U.S. Department of Education. Stephen Becker is supported by award number K23MH108603 from the National Institute of Mental Health (NIMH). The use of REDCap in this project was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health (NIH), under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the IES or the NIH.

Footnotes

Conflict of interests: The authors declare no potential conflicts of interest with respect to the research, authorship, or publication of this article.

1

Given the wording of the SCT items which describes a symptom (e.g., daydreams) and the lowest rating anchor indicates almost no occurrence for the past month, we did not expect the SCT (or ADHD-IN scores) to measure low levels of their respective traits (−0.50 to −2.00 below their trait means) with much information and precision.

2

With categorical indicators and the WLSMV estimator, omega represents proportion of true score variance in a composite of the theoretical continuous latent response variables underlying categorical manifest item scores.

References

  1. Asparouhov T, & Muthén B (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397–438. doi: 10.1080/10705510903008204 [DOI] [Google Scholar]
  2. Barkley RA (2014). Sluggish cognitive tempo (concentration deficit disorder?): current status, future directions, and a plea to change the name. Journal of Abnormal Child Psychology, 42, 117–125. doi: 10.1007/s10802-013-9824-y [DOI] [PubMed] [Google Scholar]
  3. Becker SP (2017). “For some reason I find it hard to work quickly”: Introduction to the Special Issue on sluggish cognitive tempo. Journal of Attention Disorders, 21, 615–622. doi: 10.1177/1087054717692882 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Becker SP, & Barkley RA (2018). Sluggish cognitive tempo. In Banaschewski T, Coghill D, & Zuddas A (Eds.), Oxford textbook of attention deficit hyperactivity disorder (pp. 147–153). Oxford, England: Oxford University Press. [Google Scholar]
  5. Becker SP, Burns GL, Leopold DR, Olson RK, & Willcutt EG (2018). Differential impact of trait sluggish cognitive tempo and ADHD inattention in early childhood on adolescent functioning. Journal of Child Psychology and Psychiatry, 59, 1094–1104. doi: 10.1111/jcpp.12946 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Becker SP, Burns GL, Schmitt AP, Epstein JN, & Tamm L (2019). Toward establishing a standard symptom set for assessing sluggish cognitive tempo in children: Evidence from teacher ratings in a community sample. Assessment, 26, 1128–1141. doi: 10.1177/1073191117715732 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Becker SP, Ciesielski HA, Rood JE, Froehlich TE, Garner AA, Tamm L, & Epstein JN (2016). Uncovering a clinical portrait of sluggish cognitive tempo within an evaluation for attention-deficit/hyperactivity disorder: A case study. Clinical Child Psychology and Psychiatry, 21, 81–94. doi: 10.1177/1359104514554312 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Becker SP, Leopold DR, Burns GL, Jarrett MA, Langberg JM, Marshall SA, … Willcutt EG (2016). The internal, external, and diagnostic validity of sluggish cognitive tempo: A meta-analysis and critical review. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 163–178. doi: 10.1016/j.jaac.2015.12.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Becker SP, & Willcutt EG (2019). Advancing the study of sluggish cognitive tempo via DSM, RDoC, and hierarchical models of psychopathology. European Child and Adolescent Psychiatry, 28, 603–613. doi: 10.1007/s00787-018-1136-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Burns GL, & Becker SP (2019). Sluggish cognitive tempo and ADHD symptoms in a nationally representative sample of U.S. children: Differentiation using categorical and dimensional approaches. Journal of Clinical Child and Adolescent Psychology. Advance online publication. doi: 10.1080/15374416.2019.1678165 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Burns GL, Becker SP, Geiser C, Leopold DR, & Willcutt EG (2019). Are sluggish cognitive tempo, ADHD, and oppositional defiant disorder trait- or state-like constructs from prekindergarten to fourth grade? Journal of Clinical Child and Adolescent Psychology. Advance online publication. doi: 10.1080/15374416.2019.1567348 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Burns GL, Lee S, Servera M, McBurnett K, & Becker SP (2015). Child and Adolescent Behavior Inventory - Teacher Version 1.0. Pullman, WA: Author. [Google Scholar]
  13. Dvorsky MR, Becker SP, Tamm L, & Willoughby MT (2019). Testing the longitudinal structure and change in sluggish cognitive tempo and inattentive behaviors from early through middle childhood. Assessment. Advance online publication. doi: 10.1177/1073191119872247 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Edelen MO, & Reeve BB (2007). Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement. Qual Life Res, 16 Suppl 1, 5–18. doi: 10.1007/s11136-007-9198-0 [DOI] [PubMed] [Google Scholar]
  15. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform, 42, 377–381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Harrison JR, Bunford N, Evans SW, & Owens JS (2013). Educational Accommodations for Students With Behavioral Challenges: A Systematic Review of the Literature. Review of Educational Research, 83, 551–597. doi: 10.3102/0034654313497517 [DOI] [Google Scholar]
  17. Jacobson LA, Murphy-Bowman SC, Pritchard AE, Tart-Zelvin A, Zabel TA, & Mahone EM (2012). Factor structure of a sluggish cognitive tempo scale in clinically-referred children. Journal of Abnormal Child Psychology, 40, 1327–1337. doi: 10.1007/s10802-012-9643-6 [DOI] [PubMed] [Google Scholar]
  18. Kline RB (2016). Principles and practice of structural equation modeling (4th ed.). New York: Guilford. [Google Scholar]
  19. Leopold DR, Christopher ME, Burns GL, Becker SP, Olson RK, & Willcutt EG (2016). Attention‐deficit/hyperactivity disorder and sluggish cognitive tempo throughout childhood: Temporal invariance and stability from preschool through ninth grade. Journal of Child Psychology and Psychiatry, 57, 1066–1074. doi: 10.1111/jcpp.12505 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Little TD (2013). Longitudinal structural equation modeling. New York: Guilford. [Google Scholar]
  21. Lloyd JW, Kauffman JM, Landrum TJ, & Roe DL (1991). Why do teachers refer pupils for special education? An analysis of referral records. Exceptionality, 2, 115–126. [Google Scholar]
  22. McBurnett K, Villodas M, Burns GL, Hinshaw SP, Beaulieu A, & Pfiffner LJ (2014). Structure and validity of sluggish cognitive tempo using an expanded item pool in children with attention-deficit/hyperactivity disorder. Journal of Abnormal Child Psychology, 42, 37–48. doi: 10.1007/s10802-013-9801-5 [DOI] [PubMed] [Google Scholar]
  23. Muthén LK, & Muthén BO (1998–2019). Mplus user’s guide (Eigth ed.). Los Angeles, CA: Muthén & Muthén. [Google Scholar]
  24. Nigg JT, Karalunas SL, Feczko E, & Fair DA (2020). Toward a revised nosology for ADHD heterogeneity. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. doi: 10.1016/j.bpsc.2020.02.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Owens EB, Hinshaw SP, McBurnett K, & Pfiffner L (2018). Predictors of response to behavioral treatments among children with ADHD-inattentive type. Journal of Clinical Child and Adolescent Psychology, 47, S219–S232. doi: 10.1080/15374416.2016.1228461 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Penny AM, Waschbusch DA, Klein RM, Corkum P, & Eskes G (2009). Developing a measure of sluggish cognitive tempo for children: content validity, factor structure, and reliability. Psychological Assessment, 21, 380–389. doi: 10.1037/a0016600 [DOI] [PubMed] [Google Scholar]
  27. Sáez B, Servera M, Becker SP, & Burns GL (2019). Optimal items for assessing sluggish cognitive tempo in children across mother, father, and teacher ratings. Journal of Clinical Child and Adolescent Psychology, 48, 1–15. doi: 10.1080/15374416.2017.1416619 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Servera M, Saez B, Burns GL, & Becker SP (2018). Clinical differentiation of sluggish cognitive tempo and attention-deficit/hyperactivity disorder in children. Journal of Abnormal Psychology, 127, 818–829. doi: 10.1037/abn0000375 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Severson HH, Walker HM, Hope-Doolittle J, Kratochwill TR, & Gresham FM (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193–223. doi: 10.1016/j.jsp.2006.11.003 [DOI] [Google Scholar]
  30. Siceloff ER, Bradley WJ, & Flory K (2017). Universal behavioral/emotional health screening in schools: Overview and feasibility. Rep Emot Behav Disord Youth, 17, 32–38. [PMC free article] [PubMed] [Google Scholar]
  31. Weist MD, Rubin M, Moore E, Adelsheim S, & Wrobel G (2007). Mental health screening in schools. Journal of School Health, 77, 53–58. doi: 10.1111/j.1746-1561.2007.00167.x [DOI] [PubMed] [Google Scholar]
  32. Willcutt EG, Chhabildas N, Kinnear M, DeFries JC, Olson RK, Leopold DR, … Pennington BF (2014). The internal and external validity of sluggish cognitive tempo and its relation with DSM-IV ADHD. Journal of Abnormal Child Psychology, 42, 21–35. doi: 10.1007/s10802-013-9800-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Willcutt EG, Nigg JT, Pennington BF, Solanto MV, Rohde LA, Tannock R, … Lahey BB (2012). Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. Journal of Abnormal Psychology, 121, 991–1010. doi: 10.1037/a0027347 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Materials
NIHMS1685427-supplement-Supplemental_Materials.docx (355.6KB, docx)

RESOURCES