Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 1.
Published in final edited form as: J Abnorm Child Psychol. 2020 Jul;48(7):881–894. doi: 10.1007/s10802-019-00608-4

Application of the Bifactor S – 1 Model to Multisource Ratings of ADHD/ODD Symptoms:An Appropriate Bifactor Model for Symptom Ratings

G Leonard Burns 1, Christian Geiser 2, Mateu Servera 3, Stephen P Becker 4, Theodore P Beauchaine 5
PMCID: PMC8017439  NIHMSID: NIHMS1685443  PMID: 31834589

Abstract

The symmetrical bifactor model is often applied to attention-deficit/hyperactivity disorder (ADHD)-hyperactive/impulsive (HI), ADHD-inattentive (IN), and oppositional defiant disorder (ODD) symptoms, but this model frequently yields anomalous or inadmissible results. An alternative model, the bifactor S – 1 model, is more appropriate for examining the hierarchical structure of ADHD/ODD symptoms. Both models were applied to ADHD-HI, ADHD-IN, and ODD symptom ratings by mothers, fathers, and teachers for 2,142 Spanish children (49.49% girls; ages 8–13 years). The symmetrical bifactor model yielded the typical anomalous loadings, with a weakly defined ADHD-HI specific factor and difficult to interpret associations of general and specific factors with correlates. In contrast, the bifactor S – 1 model with ADHD-HI symptoms as general reference factor produced clearly interpretable results. For mothers and fathers, slightly more than 50% of true score variance in ADHD-IN and ODD symptoms represented specific residual variance not shared with the general ADHD-HI reference factor. For teachers, approximately 69% and 39% of true score variance in ADHD-IN and ODD symptoms, respectively, represented specific residual variance not shared with the general ADHD-HI reference factor. The general ADHD-HI reference factor and specific ADHD-IN and ODD residual factors showed convergent and discriminant validity across sources, along with unique associations with peer rejection, social impairment, and academic impairment factors. The bifactor S – 1 model also yielded results consistent with predictions from trait-impulsivity theory of ADHD/ODD development. Researchers should use the bifactor S – 1 model rather than the symmetrical bifactor model if hypotheses involve the latent hierarchical structure of ADHD/ODD symptoms.

Keywords: attention-deficit/hyperactivity disorder, bifactor model, bifactor S – 1 model, disruptive behavior disorders, oppositional defiant disorder, p factor, trait impulsivity theory


Bifactor statistical models (Holzinger & Swineford, 1957; Reise, 2012) are frequently used to study the hierarchical nature of psychological constructs. As an example, bifactor analyses of attention-deficit/hyperactivity disorder (ADHD)-inattentive (IN), ADHD-hyperactive/impulsive (HI), and oppositional defiant disorder (ODD) symptoms have become increasingly popular in recent years (Arias, Ponce, & Núñez, 2016; see their Table 1 for 22 such studies; see also Rodenacker, Hautmann, Görtz-Dorten & Döpfner, 2016, 2017, 2018). In these studies, the bifactor model was considered to offer a better conceptualization of the structure of ADHD/ODD symptoms than the first-order ADHD-IN, ADHD-HI, and ODD three-factor model or other alternative models.

Table 1.

Standardized Factor Loadings (Standard Errors) of ADHD and ODD Symptoms on General Reference HI and Specific Residual IN and ODD Factors

Mothers Fathers Teachers
Symptoms General Reference HI Factor Srf-IN Srf-ODD General Reference HI Factor Srf-IN Sp-ODD General Reference HI Factor Srf-IN Srf-ODD
ADHD-HI
Fidgets/squirms .87 (.01) .85 (.01) .93 (.01)
Leaves seat .90 (.01) .91 (.01) .94 (.01)
Runs/climbs .86 (.01) .88 (.01) .92 (.01)
Loud/noisy .81 (.01) .84 (.01) .93 (.01)
Driven/on the go .89 (.01) .88 (.01) .91 (.01)
Talks too much .74 (.01) .76 (.01) .85 (.01)
Blurts .84 (.01) .83 (.01) .96 (.01)
Does not wait turn .87 (.01) .88 (.01) .97 (.01)
Interrupts/intrudes .84 (.01) .83 (.01) .93 (.01)
ADHD-IN
Close attention .54 (.02) .64 (.01) .54 (.02) .64 (.02) .56 (.03) .75 (.02)
Sustain attention .59 (.02) .71 (.01) .61 (.02) .69 (.02) .59 (.02) .77 (.02)
Listen .64 (.02) .56 (.02) .66 (.02) .56 (.02) .60 (.02) .66 (.02)
Follow through .61 (.02) .64 (.02) .64 (.02) .61 (.02) .60 (.02) .74 (.02)
Organization .55 (.02) .70 (.02) .56 (.02) .67 (.02) .56 (.02) .77 (.02)
Avoids tasks .55 (.02) .69 (.02) .60 (.02) .66 (.02) .57 (.02) .73 (.02)
Loses things .54 (.02) .57 (.02) .54 (.02) .54 (.02) .60 (.02) .67 (.02)
Easily distracted .62 (.02) .68 (.01) .64 (.02) .65 (.02) .63 (.02) .71 (.02)
Forgetful .54 (.02) .70 (.02) .60 (.02) .64 (.02) .58 (.02) .75 (.02)
ODD
Argues .55 (.02) .60 (.02) .59 (.02) .54 (.02) .71 (.02) .54 (.02)
Loses temper .61 (.02) .61 (.02) .60 (.02) .63 (.02) .72 (.02) .57 (.02)
Defies/refuses .63 (.02) .57 (.02) .64 (.02) .55 (.02) .77 (.02) .52 (.02)
Annoys others .61 (.02) .53 (.02) .63 (.02) .50 (.02) .79 (.02) .45 (.02)
Blames others .59 (.02) .57 (.02) .56 (.02) .57 (.02) .73 (.02) .55 (.02)
Annoyed by others .56 (.02) .60 (.02) .53 (.02) .62 (.02) .66 (.02) .64 (.02)
Angry/resentful .51 (.02) .70 (.02) .50 (.03) .71 (.02) .66 (.02) .70 (.02)
Spiteful/vindictive .47 (.03) .66 (.02) .46 (.03) .63 (.03) .68 (.02) .65 (.02)

Note. ADHD = attention-deficit/hyperactivity disorder. HI = hyperactivity-impulsivity. IN = inattention. ODD = oppositional defiant disorder.

Srf = specific residual factor. All loadings significant at p < .001.

Despite popularity of the bifactor approach, it is not without problems (see e.g., Watts, Poore, & Waldman, 2019). As pointed out by Eid, Geiser, Koch, and Heene (2017) and Heinrich, Zagorscak, Eid, and Knaevelsrud (2018), the symmetrical bifactor model often yields anomalous or inadmissible results in empirical applications. Such results include zero, close-to-zero, or negative factor variances and/or a number of zero, small, non-significant, and/or negative factor loadings.

In this paper, we show that these issues are also common in symmetrical bifactor models of ADHD and ADHD/ODD symptom ratings. We provide a psychometric explanation for why these problems occur and show how they can be avoided by specifying an alternative and psychometrically sounder bifactor model, the bifactor S – 1 model. We illustrate the bifactor S – 1 model in an application to ADHD/ODD symptom ratings and show that it provides more meaningful results than the symmetrical bifactor approach. Below we first review the symmetrical bifactor model, and outline problems that often occur when it is applied to ADHD/ODD symptoms ratings.

Symmetrical Bifactor Model

Application of the symmetrical bifactor model to ADHD/ODD symptoms involves specifying: (1) a general disruptive behavior factor (g factor; all symptoms load on this factor); (2) a specific ADHD-HI factor (ADHD-HI symptoms load on this factor); (3) a specific ADHD-IN factor (ADHD-IN symptoms load on this factor); and (4) a specific ODD factor (ODD symptoms load on this factor). The general factor is (typically) not allowed to correlate with any of the specific factors (for an exception, see Martel, Gremillion, Roberts, von Eye, & Nigg, 2010). The general factor therefore accounts for common variance that is shared across ADHD-HI, ADHD-IN, and ODD symptom ratings, whereas specific ADHD-HI, ADHD-IN, and ODD factors account for specific variance in respective symptom domains, independent of the general factor. In most applications, correlations between different specific factors are constrained to zero. This is because common variance that is shared across symptom facets should be reflected by the general factor in the model, not by correlations among specific factors. Figure 1A illustrates application of the symmetrical bifactor model to ADHD-HI, ADHD-IN, and ODD symptom ratings.

Figure 1.

Figure 1.

Schematic representations of the symmetrical and S – 1 bifactor models of ADHD-HI, ADHD-IN, and OD symptoms (only the first and last symptom are shown for each of the three symptom sets to reduce the complexity of the figure). HI = hyperactivity-impulsivity; IN = inattention; OD = oppositional defiant disorder; DB = disruptive behavior; GF = general factor; SF = specific factor; GRF = general reference factor; SRF = specific residual factor.

The symmetrical bifactor model is appealing for several reasons. For example, it is possible to determine whether ADHD and ODD symptom ratings contain a meaningful amount of specific true score variance, independent of a general disruptive behavior factor. In addition, it is possible to determine whether specific ADHD-HI, ADHD-IN, and ODD factors have associations with external correlates independent of a general disruptive behavior factor. Finally, it is possible to determine whether the g factor has associations with external correlates independent of the three specific factors. Answers to such questions can offer unique insights into the internal structure and external correlates of ADHD/ODD symptoms.

Anomalous and Inadmissible Results with the Symmetrical Bifactor Model

Although use of the symmetrical bifactor model is popular, this model frequently yields anomalous and even inadmissible parameter estimates in empirical applications (Eid et al., 2017; Heinrich et al., 2018). This includes unexpected or abnormal patterns of factor loadings whereby many (or all) take on values that are close to zero, negative, and/or non-significant. In addition, one or more specific factors often have variance estimates that are close to zero, negative, and/or non-significant. By definition, variances cannot be negative. Negative variance estimates have no meaningful interpretation and are seen as inadmissible parameter estimates.

Factor variances of zero and patterns of many small loadings suggest that the factor in question does not exist. Such results contradict the basic idea of the symmetrical bifactor model—to separate general, specific, and unique/error variance for all symptom groups. Negative factor loadings are unexpected when all symptoms have positive correlations. A negative loading indicates that the symptom correlates negatively with the factor that is supposed to represent this symptom—a result contradictory to and unexpected for the symptom.

To examine the severity of this problem, Eid et al. (2017) reviewed 82 published bifactor applications in different areas of psychology (including clinical psychology) and found that about 61% of applications (n = 50) yielded anomalous results. In our own review of recent articles examining the latent structure of ADHD and ADHD/ODD symptoms using the bifactor approach, we examined bifactor applications in all of the 22 studies listed in Arias et al. (2016, their Table 1), including two of our own (Burns et al., 2014a; Lee et al., 2016), as well as three more recent studies (Rodenacker et al., 2016, 2017, 2018). Many of the studies reported multiple bifactor applications across different types of raters or different bifactor models applied to the same rater.

Across 49 applications of the symmetrical bifactor approach to ADHD or ADHD/ODD symptoms, 34 (69.4%) contained at least two standardized factor loadings that were either < |.2|, negative, or not significantly different from zero. Most anomalous loading patterns occurred for specific factors—in particular specific factors representing hyperactivity/impulsivity (HI) or just hyperactivity (HY) symptoms (see discussion below). One additional study (DuPaul et al., 2015) reported “weak” specific HI factors in two bifactor applications without providing exact loadings. Another study (Ullebø et al., 2012) found an inadmissible solution and subsequently dropped the specific factor for HY symptoms to obtain a proper solution. In total, 37 out of 49 bifactor applications (75.5%) contained at least one issue. Interestingly, the most prevalent issue was a set of unexpectedly small (and/or negative) loadings of HI or HY symptoms on their specific factors, or HI or HY specific factors that vanished completely (31 studies; 63.3%).

Why is it a problem when specific factors vanish in a bifactor analysis, and/or when specific symptoms do not have significant loadings on their specific factors, indicating that these factors may be weak or non-existent? First, when a specific factor is zero (as shown by an estimated variance that is zero or close to zero and/or by zero or close-to-zero estimated factor loadings), this implies that a group of symptoms (e.g., HY or HI) has no specific variance (i.e., residual variance independent of general factor). Instead, the shared variance of the symptom group in question now defines the general factor—particularly when general factor loadings of the symptoms with negligible specific factor loadings are strong (e.g., .7 or larger). This compromises the intended interpretation of the general factor as a factor common to all symptom groups. Under these circumstances, the general factor becomes the factor of symptoms for which no specific factor can be identified (i.e., if the ADHD-HI specific factor vanishes, the general factor becomes an ADHD-HI factor; Eid et al., 2017).

As a result, the general factor can no longer be interpreted as a general factor with meaning defined by the entire set of symptoms. Moreover, meaning of the general factor depends on which specific factor vanishes—which is a purely data-driven result that depends on the empirical pattern of intercorrelations among first-order factors. Depending on which specific constructs are included in a study, different symptom facets might drop their specific factor, resulting in different meanings of the general factor across studies. Our review of ADHD and ADHD/ODD studies confirms that different specific factors disappeared in different studies (i.e., sometimes for IN, other times for HI or HY), resulting in different meanings of the general factor across these symmetrical bifactor studies of ADHD symptoms. Of importance, when correlations of the general factor with external variables are considered, different conclusions might be drawn depending on which specific factor is non-substantial in a particular application (see e.g., Eid, Krumm, Koch, & Schulze, 2018). Thus, results may not be comparable across studies.

We now describe conditions under which use of the symmetrical bifactor model are appropriate from a psychometric perspective. We then explain why these conditions are not usually met in applications to ADHD/ODD symptom ratings and other clinical constructs—leading to the problems described above. Subsequently, we discuss an alternative approach that is more suitable for symptom ratings in psychopathology research.

Appropriate Use of the Symmetrical Bifactor Model with Interchanageable Factors

Eid et al. (2017) examined the symmetric bifactor approach from a psychometric (measurement-theoretical) perspective. According to Eid et al. (2017), the symmetrical bifactor model implies that all facets (e.g., symptoms such as ADHD-HI, ADHD-IN, and ODD) are randomly selected from a universe of facets. An example of random facets are randomly selected students from the universe of students who rate their teachers’ lectures. Random selection of facets from a pool of equivalent facets implies interchangeability of facets such that a researcher could, in theory, simply replace one facet by another from the same set without losing or changing critical information or content. As a consequence, random facets follow a multilevel sampling structure, which is in accordance with the mathematical structure of the symmetrical bifactor approach (Eid et al., 2017).

Unfortunately, the implicit interchangeability assumption of the symmetrical bifactor model is violated in most applications to symptom ratings. Symptoms (across facets) cannot typically be viewed as randomly sampled from a universe of exchangeable symptoms. Instead, symptoms are typically “fixed factors” rather than “random factors”—they differ structurally and are non-interchangeable. As an example, consider inattention, hyperactivity, and impulsivity facets. These three facets are structurally different and cannot be replaced by other equivalent symptom facets because each facet contributes uniquely to coverage of the overall ADHD construct. One facet cannot be deleted or replaced without losing critical information and construct coverage. If one structurally different facet is replaced by another (or if one or more facets are dropped), loadings on and meaning of the general factor in the symmetrical bifactor approach changes. This is because the general factor is only invariant across randomly selected (interchangeable) facets. In typical applications in clinical psychology, this is not the case. Instead, researchers select specific (non-random) facets (symptoms) based on theory and/or specific research questions.

Geiser, Bishop, and Lockhart (2015) conducted a simulation study and found that the symmetrical bifactor model is more likely to result in typically seen anomalous loading patterns when it is applied to data generated from structurally different (as opposed to interchangeable) facets. Eid et al. (2017) showed that a more appropriate statistical model for non-interchangeable (fixed, structurally different) facets such as symptom ratings (which are more typical in psychology than exchangeable facets) is a revised bifactor approach that a priori specifies one specific factor less than factors considered (the so-called bifactor S – 1 model, Eid et al., 2017). The bifactor S – 1 model is described next, along with application of the model to ADHD/ODD symptom ratings.

Bifactor S – 1 Model with Structurally Different Factors

The bifactor S – 1 model allows facets to differ structurally, and statistically contrasts S – 1 facets against a reference facet. The first step requires selection of one of the first-order factors—for example, ADHD-HI, ADHD-IN, or ODD—as a general reference factor. Symptom ratings pertaining to the reference domain are used as indicators of the general reference factor, and there is no specific factor for these symptoms. For example, if ADHD-HI symptoms are selected as the reference domain, then the general reference factor reflects common true score variance underlying the nine ADHD-HI symptoms, with no specific factor for ADHD-HI symptoms. This procedure results in an unambiguous and psychometrically sound a priori definition of the general factor (i.e., ADHD-HI symptoms corrected for random measurement error). Moreover, meaning of the general reference factor does not change, regardless of which other facets or symptoms are included in the model, as long as the same symptom facet is chosen as general reference factor.

With ADHD-HI symptoms used to define the general reference factor, ADHD-IN and ODD symptoms are contrasted statistically against ADHD-HI symptoms. Specific factors in the S – 1 approach are defined as regression residual factors in a latent regression on the general reference factor. As a result, the specific factors represent true score variance in non-reference symptom facets that is not shared with the general reference factor. The two specific residual factors are thus not allowed to correlate with the general reference factor but may correlate with each other (Eid et al., 2017). This is a partial correlation that indicates shared specific variance in the two non-reference factors that is not shared with the general reference factor. Figure 1B shows the bifactor S – 1 model for ADHD and ODD symptoms with ADHD-HI symptoms used to define the general reference factor.1

Trait Impulsivity Theory and Selection of the General Reference Factor

As alluded to above, an important issue when using the bifactor S – 1 model concerns selection of symptoms to define the general reference factor. Selection of the general reference factor should follow from theory (Eid et al., 2017; Heinrich et al. 2018) or be based on ease of interpretation (e.g., the most outstanding, severe, or otherwise special symptom group may be chosen as reference). In the current study, trait impulsivity theory of ADHD/ODD development provided the justification to define the general reference factor with ADHD-HI symptoms (see e.g., Beauchaine, 2015; Beauchaine & McNulty, 2013; Beauchaine, Zisner, & Sauder, 2017). Although full articulation of trait impulsivity theory is beyond the scope of this article, we briefly describe reasons why this theory supports using ADHD-HI as reference symptom group. We then describe theoretical predictions for application of the bifactor S – 1 model to ADHD/ODD symptoms.

Trait impulsivity theory postulates two different etiological pathways in the development of ADHD. One pathway to ADHD implicates low tonic and low phasic responding in primarily one neural system—the mesolimbic reward system (“bottom-up” pathway)—in temperamental impulsivity—expressed by ADHD-HI symptoms very early in life (Gatzke-Kopp et al., 2009). For this highly heritable pathway, ADHD-IN symptoms arise secondarily to hyperactivity and impulsivity. This neurodevelopmental vulnerability, which gives rise to both the ADHD-HI and ADHD combined presentations, increases the likelihood of developing ODD and conduct disorder (CD) in high-risk environments characterized by coercive family processes, deviant peer group affiliations, neighborhood violence and criminality, and other adversities (Beauchaine & McNulty, 2013; Beauchaine et al., 2017). A second pathway to ADHD involves fronto-parietal and cerebellar dysfunction (Diamond, 2005; Fair et al., 2013), resulting in direct (as opposed to secondary) development of inattention, including ADHD-IN symptoms. For this presentation, likelihood of progression to comorbid ODD, CD, and eventual criminality is low (see Beauchaine et al., 2017).

Trait-impulsivity theory suggests that ADHD-HI symptoms should be used to define the general reference factor in the bifactor S – 1 model because ADHD-HI symptoms are often detected before ADHD-IN symptoms, and typically develop prior to ODD symptoms. Trait-impulsivity theory, however, still predicts that specific ADHD-IN and specific ODD residual factors should contain a substantial amount of true score variance that is independent of the general ADHD-HI reference factor given that development of ADHD-IN and ODD symptoms requires additional etiological processes. This rationale provided the basis to define the general reference factor with ADHD-HI symptoms and to expect specific ADHD-IN and ODD residual factors to contain a substantial amount of true score variance, above-and-beyond the general ADHD-HI reference factor.

Objectives

The study involved five objectives, with predictions associated with each:

Objective 1.

Our first objective was to apply the symmetrical bifactor model to ADHD-HI, ADHD-IN, and ODD symptom ratings by mothers, fathers, and teachers. Each analysis was expected to yield an anomalous solution, results consistent with other studies using this model. The specific ADHD-HI factor was expected to either partially or completely vanish. Such an occurrence, as described earlier, turns the general factor into an ADHD-HI factor, with the specific ADHD-HI factor becoming meaningless, thus making interpretations of associations of the general and specific factors with external correlates problematic. To demonstrate this problem, we examined associations of the general and specific factors from the symmetrical bifactor model with academic impairment, social impairment, and peer rejection. The purpose of Objective 1 was to illustrate problems with use of the symmetrical bifactor model.

Objective 2.

Our second objective was to apply the bifactor S – 1 model to ADHD-HI, ADHD-IN, and ODD symptom ratings by all three sources. The general reference factor was defined by ADHD-HI symptoms, with specific ADHD-IN and ODD residual factors defined by ADHD-IN and ODD symptoms, respectively. It was expected that each analysis would yield an admissible solution. More specifically, ADHD-HI symptoms were predicted to show substantial loadings (> .80) on the general reference factor. Approximately 50% of true score variance in ADHD-IN and ODD symptoms was also predicted to be independent of the general ADHD-HI reference factor. These outcomes would be consistent with predictions by trait impulsivity theory of ADHD/ODD development.

Objective 3.

Our third objective was to evaluate convergent and discriminant validity of the general reference factor and the two specific residual factors across mothers, fathers, and teachers. Like-factor correlations were expected to be substantial (> .70) for mothers with fathers. For mothers/fathers with teachers, like-factor correlations for the general reference factor and the specific ADHD-IN residual factor were expected to be moderate (> .40), with the correlation for the specific ODD residual factor across mothers/fathers to teachers being small (.25). Like-factor correlations were also expected to be significantly larger than different-factor correlations across sources, thus providing support for the discriminant validity of the three factors.

Objective 4:

Our fourth objective was to evaluate invariance of like-symptom loadings and thresholds for the bifactor S – 1 model across sources. Invariance of like-symptom loadings and thresholds was expected across mothers and fathers. We also expected no significant differences on the general ADHD-HI reference factor mean across mothers and fathers (residual factor means are zero). Invariance of loadings and thresholds across parents and teachers was less certain.

Objective 5:

Our fifth objective was to determine if the general reference factor and the specific ADHD-IN and ODD residual factors had unique associations with academic impairment, social impairment, and peer-rejection factors. The specific ADHD-IN residual factor was expected to have a stronger unique association than the general reference and specific ODD residual factors with the academic impairment factor. In contrast, the general reference and specific ODD residual factors were expected to have stronger unique associations than the specific ADHD-IN residual factor with social impairment and peer-rejection factors. Such a pattern would demonstrate unique external validity for the general and two specific residual factors.

Summary

Objective 1 demonstrates problems associated with application of the symmetrical bifactor model to ADHD/ODD symptom ratings. Objectives 2–5 then demonstrate resolution of these problems through application of the bifactor S – 1 model to symptom ratings. To the best of our knowledge, this represents the first application of the bifactor S – 1 model to ADHD/ODD symptoms.

Method

Participants and Procedures

Participants were mothers, fathers, and teachers of 2,142 third- to sixth-grade children (50.51% boys, Mage = 10.30, SD = 1.21) from 32 randomly selected Balearic Islands schools. An informed consent form was provided to mothers and fathers and with parental written approval a similar informed consent form was given to teachers. Mothers, fathers, and teachers provided written informed consent for their participation. Mothers and fathers provided ratings on 1,649 and 1,358 children, respectively (1,777 unique children were rated by parents). A total of 196 teachers provided ratings on 1,773 children with each teacher rating an average of 10.93 (SD = 6.05) children. Approximately 85% of the children lived with both parents and approximately 13% with a single parent (approximately 2% missing). Additional details about sample characteristics can be found elsewhere (Sáez, Servera, Becker, & Burns, 2019; Servera, Sáez, Burns, & Becker, 2018). The Research Ethics Committee of the University of the Balearic Islands approved the protocol.

Measures

Child and Adolescent Behavior Inventory (CABI).

Mothers, fathers, and teachers completed the CABI (Burns, Lee, Servera, McBurnett, & Becker, 2015a, 2015b). The ADHD-IN (nine symptoms), ADHD-HI (nine symptoms), ODD (eight symptoms), social impairment (four items for parents—quality of interactions with parents, other adults, siblings, and peers; two items for teachers—quality of interactions with adults and peers at school), and academic impairment (five items: quality of homework/classwork, reading skills, arithmetic skills, writing skills, and global academic skills) scales were used in this study. Parents and teachers were instructed to base their ratings on the past month. Mothers and fathers were also told to make their ratings independently. Symptoms were rated on a 6-point scale (0=almost never [never or about once per month], 1=seldom [about once per week], 2=sometimes [several times per week], 3=often [about once per day], 4=very often [several times per day], and 5=almost always [many times per day]. A 7-point scale was used for academic and social impairment items (0=severe difficulty, 1=moderate difficulty, 2=slight difficulty, 3=average performance [average interactions] for grade level, 4=slightly above average, 5=moderately above average, and 6=excellent performance [excellent interactions] for grade level). Academic impairment and social impairment items were reverse keyed so that higher scores indicated greater academic and social impairment.

Earlier studies support reliability (internal consistency, test-retest, inter-rater) and validity of scale scores (e.g., Becker, Burns, Schmitt, Epstein, & Tamm, 2019; Burns & Becker, 2019; Burns, Servera, Bernard, Carrillo, & Geiser 2014b). In the current study, Cronbach’s alpha (α) values for mothers (fathers, teachers) for ADHD-IN, ADHD-HI, ODD, social impairment, and academic impairment scores were .95 (.95, .97), .93 (.93, .96), .92 (.91, .95), .90 (.91, .87), and .94 (.95, .97). Inter-rater factor correlations for ADHD-IN, ADHD-HI, ODD, social impairment, and academic impairment for mothers with fathers were .83 (SE = .01), .79 (SE = .02), .75 (SE = .02), .70 (SE = .02), and .87 (SE = .01), respectively, mothers with teachers .55 (SE = .02), .40 (SE = .03), .30 (SE = .03), .18 (SE = .03), and .72 (SE = .02), respectively, and fathers with teachers .54 (SE = .03), .44 (SE = 03), .30 (SE = 04), .14 (SE = 03), and .69 (SE = .02), respectively. Supplemental Table S1 provides descriptive information on the scales.

Dishion Social Acceptance Scale (DSAS).

Two items from the DSAS (Dishion, 1990) were used to create the measure of social rejection in the classroom. Teachers rated the proportion of classmates who “like” and “dislike” the child on a 5-point scale (very few [less than 25%]; some [25 to 49%]; about half [50%]; many [51% to 75%]; and almost all [greater than 75%]). These two items were used to index peer rejection (the “like” item reversed) (α = .85).

Analytic Approach

Estimation and clustering.

Analyses used Mplus statistical software (version 8.3) with items treated as categorical indicators (WLSMV estimator, delta parameterization). The Mplus type=complex option was used to correct standard errors and fit statistics for non-independence of observations (children nested within teachers). The WLSMV estimator uses a pairwise approach to missing information. Covariance coverage was greater than 99.70% for all variances and covariances for each source separately.

Criteria for model fit.

Model fit was evaluated with comparative fit index (CFI; acceptable fit ≥ .90 and close fit ≥ .95), standardized root mean square residual (SRMR; acceptable fit ≤ .08, close fit ≤ .05), and root-mean-square error of approximation (RMSEA; acceptable fit ≤ .08 and close fit ≤ .05). The alpha level for significance tests was .001 given the large size of the sample.

Model identification.

The ADHD-HI symptom has difficulty waiting turn was used to set the metric of the general reference factor, the ADHD-IN symptom has difficulty keeping attention focused during tasks the specific ADHD-IN residual factor, and the ODD symptom appears angry or resentful toward others the specific ODD residual factor. These symptoms had the strongest associations with their respective factors.

Results

Symmetrical Bifactor Model

Although the symmetrical bifactor model produced appropriate fit (CFIs > .98, RMSEAs < .06, and SRMRs < .04), it yielded anomalous results for all three sources. For mother ratings, 3 of 9 standardized loadings on the specific HI factor were negative (range: –.17 to –.11, ps < .001). One additional specific HI standardized factor loading was approximately zero (.02, p = .42). These four HI items were fidgets/squirms, leaves seat, runs/climbs, and driven/on the go). For fathers, 5 of 9 HI specific factor loadings were negative (range: –.31 to –.05) with 4 of 5 negative loadings being significant (ps < .001). Of importance, the HI items that showed negative loadings for fathers were different from the items that had shown anomalous values for mothers (i.e., the items were loud/noisy, talks too much, blurts, difficulty awaiting turn, and interrupts/intrudes for fathers). For teachers, 4 of 9 HI items showed standardized specific HI factor loadings smaller than .20 (range: .06 to .19). Two of these item loadings were not significant (ps > .06). Here, 3 of the 4 problematic teacher items were identical to those that yielded anomalous results for mothers.

Taken together, these results are similar to problems noted in our earlier description of results from most applications of the symmetrical bifactor model to ADHD symptoms. Consistent with earlier results, a set of anomalous specific ADHD-HI factor loadings occurred for each rating source, casting doubt on the integrity and interpretation of the specific factor for ADHD-HI symptoms. In the current study, the specific ADHD-HI factor for mothers and teachers became mostly a specific ADHD-HY factor, whereas the specific ADHD-HI factor for fathers became mostly a specific ADHD-IM factor. More generally, for all three sources, the specific ADHD-HI factor contained 4 to 5 symptoms with negative or close to zero loadings, resulting partial collapse of the specific ADHD-HI factor (i.e., little true score variance in the specific ADHD-HI factor independent of the now mostly general ADHD-HI factor).

Partial collapse of the specific ADHD-HI factor, different meanings of the specific ADHD-HI factor for mothers/teachers versus fathers, and a new meaning for the general factor (i.e., a general ADHD-HI factor mostly) make it difficult/impossible to interpret associations of the general and specific factors with external correlates (see also Eid et al., 2018). To illustrate these problems, correlates of social impairment, peer-rejection, and academic impairment were regressed on the general factor and three specific factors. As expected, the specific ADHD-HI factor had non-significant (essentially zero) associations with the three outcomes for the three sources with one exception (for teachers higher scores on the specific ADHD-HI factor were uniquely associated with significantly (p < .001) lower levels of academic impairment, an anomalous outcome). More importantly, unclear definitions of the general and three specific factors from the symmetrical bifactor model make their associations with such external correlates impossible to interpret. Supplemental Table S2 shows these results.

Bifactor S – 1 Model

Model fit.

The model resulted in acceptable to close fit for each source, mothers: χ2 (281) = 2,387, p < .001, CFI = .97, RMSEA = .067 (.065, .070), and SRMR = .04; fathers: χ2( 281) = 1,871, p < .001, CFI = .98, RMSEA = .065 (.062, 067), and SRMR = .04; and teachers: χ2 (281) = 2,053, p < .001, CFI = .99, RMSEA = .060 (.057, .062), and SRMR = .04. Inspection of the 325 standardized residuals for each source did not indicate any major localized ill-fit.

Correlations between the specific factors within each source.

Correlations between the specific ADHD-IN and specific ODD residual factors were .30 (SE = .03), .29 (SE = .03), and .20 (SE = .03, ps < .001) for mothers, fathers, and teachers, respectively. This shows that after variance shared with the general ADHD-HI reference factor was removed, the ADHD-IN and ODD factors had little residual true score variance in common (less than 10%).

General reference and specific residual factors loadings.

Table 1 shows standardized loadings of all symptoms on general reference and two specific residual factors. All factor loadings were positive, substantial in magnitude, and highly significant (all ps < .001). Specific factor loadings showed a fairly homogenous pattern across symptoms for each source. For mothers, fathers, and teachers, the ADHD-HI symptoms had substantial (.74 to .97) loadings on the general ADHD-HI reference factor, indicating that these symptoms measured the general reference factor with substantial reliability. For mothers and fathers, the ADHD-IN and ODD symptoms had similar and moderately high loadings on the general ADHD-HI reference factor and their respective residual factors. For teachers, ADHD-IN symptoms had lower loadings (M = .59, SD = .02) on the general ADHD-HI reference factor than on the specific ADHD-IN residual factor (M = .73, SD = .04). ODD symptoms loaded more highly on the general reference factor than on their specific ODD residual factor for teachers (M = .72, SD = .05 and M = .58, SD = .08, respectively). This indicates that based on teacher reports, IN symptoms showed more specificity with respect to HI compared to ODD symptoms. This was not the case for mother and father reports for which IN and ODD symptoms on average loaded about equally highly on the HI reference factor.

Consistency and specificity of ADHD-IN and ODD symptoms.

Table 2 shows consistency and specificity values for ADHD-IN and ODD symptoms. Consistency represents the amount of true score variance in ADHD-IN and ODD symptoms accounted for by the general ADHD-HI reference factor, whereas specificity represents the amount of true score variance in ADHD-IN and ODD symptoms not shared with the general ADHD-HI reference factor.

Table 2.

Consistency and Specificity Values for the ADHD-Inattention and Oppositional Defiant Disorder Symptoms

Mothers Fathers Teachers
Symptoms Consistency Specificity Consistency Specificity Consistency Specificity
ADHD-inattention
 Close attention .41 .59 .42 .58 .36 .64
 Sustaining attention .40 .60 .44 .56 .37 .63
 Listen .56 .44 .58 .42 .45 .55
 Follow through .47 .53 .52 .48 .39 .61
 Organization .39 .61 .42 .58 .35 .65
 Avoids tasks .39 .61 .45 .55 .37 .63
 Loses things .47 .53 .50 .50 .45 .55
 Easily distracted .45 .55 .49 .51 .44 .56
 Forgetful .38 .62 .46 .54 .37 .63
ODD
 Argues .45 .55 .54 .46 .64 .36
 Loses temper .50 .50 .48 .52 .61 .39
 Defies/refuses .55 .45 .58 .42 .69 .31
 Annoys others .57 .43 .61 .39 .75 .25
 Blames others .52 .48 .50 .50 .64 .36
 Annoyed by others .46 .54 .42 .58 .52 .48
 Angry/resentful .35 .65 .33 .67 .47 .53
 Spiteful/vindictive .34 .66 .35 .65 .52 .48

Note. Consistency represents the proportion of true-score variance in the non-reference domain symptom (ADHD-IN or ODD symptoms) that is shared with the general factor (ADHD-HI reference domain). Specificity represents the proportion of true-score variance in the non-reference domain symptom (ADHD-IN or ODD symptoms) that is not shared with the general factor (ADHD-HI reference domain). ADHD = attention-deficit/hyperactivity disorder. HI = hyperactivity-impulsivity. IN = inattention. ODD = oppositional defiant disorder.

For mothers, average consistency and specificity values for ADHD-IN symptoms were .44 (SD = .06) and .56 (SD = .06), respectively, with average values for fathers being .47 (SD = .05) and .53 (SD = .05), respectively. For ODD symptoms, average consistency and specificity values for mothers were .47 (SD = .09) and .53 (SD = .09), respectively, with average values for fathers being .48 (SD = .10) and .52 (SD = .10), respectively. Thus, on average, slightly more than 50% of the true score variance in ADHD-IN and ODD symptoms was not associated with the general ADHD-HI reference factor for mothers and fathers.

For teachers, average consistency and specificity values for ADHD-IN symptoms were .39 (SD = .04) and .61 (SD = .04), respectively, whereas for ODD symptoms, average consistency and specificity values were .61 (SD = .10) and .39 (SD = .10). For teachers, more true score variance in ADHD-IN symptoms was thus associated with the specific ADHD-IN residual factor, whereas for ODD symptoms, more true score variance was associated with the general reference factor.

Convergent and Discriminant Validity of General Reference and Specific Residual Factors

The bifactor S – 1 model yielded a close fit when applied simultaneously to mother, father, and teacher symptom ratings, χ2 (2,818) = 5,634, p < .001, CFI = .99, RMSEA = .022 (.021, .023), and SRMR = .04. Inspection of the 3,081 standardized residuals did not indicate any major localized ill-fit.

Table 3 shows factor variances and correlations from this model. The general reference and specific residual factors each had significant (ps < .001) variance estimates for each source. For mothers with fathers, convergent correlations of the general reference, specific ADHD-IN residual, and specific ODD residual factors were .79 (SE = .02), .78 (SE = .02), and .70 (SE = .02, ps < .001), respectively, thus indicating strong convergent validity. Convergent correlations were also significantly larger than discriminant correlations for mothers and fathers, ps < .001. For mothers with teachers, convergent correlations for the general reference, specific ADHD-IN residual, and specific ODD residual factors were .41 (SE = .03), .54 (SE = 03), and .25 (SE = .04), respectively, with respective values for fathers with teachers being .44 (SE = .03), .49 (SE = .04), and .23 (SE = .04, ps < .001). These convergent correlations were also significantly (ps < .001) larger than the discriminant correlations across mothers/fathers to teachers.

Table 3.

Convergent and Discriminant Validity Correlations (SE) of General Reference HI and Specific Residual Factors across Mother, Father, and Teacher Ratings With Factor Variances on the Main Diagonal

Mother General Reference HI Mother Srf-IN Mother Srf-ODD Father General Reference HI Father Srf-IN Father Srf-ODD Teacher General Reference HI Teacher Srf-IN Teacher Srf-ODD
Mother General Reference HI .77 (.02)*
Mother Srf-IN .50 (.02)*
Mother Srf-ODD .30 (.03)* .49 (.03)*
Father General Reference HI .79 (.02)* .08 (.02)* .01 (.03)ns .76 (.02)*
Father Srf-IN .02 (.02)ns .78 (.02)* .27 (.03)* .46 (.02)*
Father Srf-ODD .00 (.02)ns .20 (.03)* .70 (.02)* .29 (.03)* .51 (.04)*
Teacher General Reference HI .41 (.03)* .02 (.03)ns −.05 (.04)ns .44 (.03)* .04 (.03)ns −.05 (.04)ns .94 (.01)*
Teacher Srf-IN .12 (.03)* .54 (.03)* .09 (.04)ns .10 (.03)ns .49 (.04)* .06 (.04)ns .59 (.03)*
Teacher Srf-ODD −.02 (.04)ns .05 (.03)ns .25 (.04)xref* −.03 (.04)ns .05 (.04)ns .23 (.04)* .20 (.03)* .48 (.03)*

Note. Convergent factor correlations are in bold. The general and specific factor correlations are zero by definition within each source. IN = inattention; ODD = oppositional deviant disorder; srf = specific residual factor. Values in parentheses are standard errors.

*

p < .001; ns = non-significant

Invariance of the Bifactor S – 1 Model across Mother and Father Ratings

The baseline model provided a close fit across mothers and fathers, χ2 (1,203) = 3,912, p < .001, CFI = .98, RMSEA = .036 (.034, .037), and SRMR = .04. The model with constraints on like item loadings and thresholds also yielded a close fit with no decrement in fit indices, χ2 (1,346) = 3,882, p < .001, CFI = .98, RMSEA = .033 (.031, .034), and SRMR = .04. The χ2 different test (Mplus difftest) did not meet our alpha (.001) level, χ2 (143) = 185, p = .01. Mothers and fathers also did not differ significantly on the general factor mean (latent d difference value = -0.01, SE = .03). Residual factors have a mean of zero, thus comparisons are not meaningful. Given that loadings for teachers were usually substantially higher than for parents, invariance of the model across parents and teachers was not evaluated.2

Association of General Reference and Specific Residual Factors with Impairment

Table 4 shows partial standardized regression coefficients from regressions of social impairment, academic impairment, and peer-rejection (teachers only) factors on the general reference and specific residual factors for each source. These three structural regression models provided close fit, with no major localized ill-fit. It is important to emphasize that the general reference factor was a priori defined by ADHD-HI symptoms, with specific ADHD-IN and ODD factors thus being residual factors in relation to the general ADHD-HI reference factor. This procedure provides clear definitions of the factors, allowing meaningful interpretation of their associations with external correlates. Such was not the case with the symmetrical bifactor model.

Table 4.

Standardized Regression Coefficients for the Associations of General Reference HI and Specific Residual ADHD-IN and Specific ODD Factors from the Bifactor S – 1 Model with Social and Academic Impairment Factors

Social Impairment Academic Impairme Peer Rejection
β (SE) β (SE) β (SE)
Mother Ratings
General Reference HI 0.28 (.02)* 0.34 (.02)** ----
Specific Residual IN 0.10 (.03)* 0.63 (.02)** ----
Specific Residual ODD 0.37 (.02)* -0.07 (.02)ns ----
Father Ratings
General Reference HI 0.29 (.03)* 0.37 (.03)* ----
Specific Residual IN 0.11 (.03)* 0.61 (.02)* ----
Specific Residual ODD 0.32 (.03)* -0.10 (.03)* ----
Teacher Ratings
General Reference HI 0.35 (.03)* 0.31 (.03)* 0.49 (.03)*
Specific Residual IN 0.36 (.03)* 0.78 (.02)* 0.26 (.02)*
Specific Residual ODD 0.36 (.03)* -0.03 (.02)ns 0.52 (.02)*

Note. HI = ADHD-hypoactivity/impulsivity; IN = ADHD-inattention; ODD = oppositional defiant disorder; ns = non-significant.

*

p < .001.

For mothers and fathers, higher scores on the general reference factor and the two specific residual factors were associated with significantly (ps < .001) higher scores on the social and academic impairment factors with one exception (specific ODD residual did not have a significant unique association with academic impairment for mothers). The unique association of the general reference and specific ODD residual factors with social impairment were approximately 3–4 times larger than unique association of the specific ADHD-IN residual factor with social impairment—mothers: ß = .28 (SE = .02) and ß = .37 (SE = .02) versus ß = .10 (SE = .03); fathers: ß = .29 (SE = .03) and ß = .32 (SE = .03) versus ß = .11 (SE = .03). In contrast, the unique association of the specific ADHD-IN residual factor with the academic impairment factor was approximately twice as large as the unique association of the general reference factor with academic impairment—mothers: ß = .63 (SE = .02) versus ß = .34 (SE = .02); fathers: ß = .61 (SE = .02) versus ß = .37 (SE = .02) and approximately six times as large as the unique association of the specific ODD residual factor with academic impairment—mothers: ß = .63 (SE = .02) versus ß = -.07 (SE = .02); fathers: ß = .61 (SE = .02) versus ß = -.10 (SE = .03).

For teachers, higher scores on the general reference and two specific residual factors were uniquely associated with significantly (ps < .001) higher scores on the social impairment, peer rejection, and academic impairment factors with one exception (the specific ODD residual did not have a significant unique association with academic impairment). Although the general and two specific factors had approximately equal unique associations with social impairment for teachers (ßs = .35 to .36), the general and specific ODD factors had unique associations with peer rejection that were approximately twice as large as the specific IN factor—ß = .49 (SE = .03) and ß = .52 (SE = .02) versus ß = .26 (SE = .02), respectively. In contrast, the specific ADHD-IN factor had a unique association with the academic impairment factor that was over twice as large as the general factor (ß = .78 [SE = .02] vs. ß = .31 [SE = .03]) and approximately eight times as large as the specific ODD factor (ß = .78 [SE = .02] vs. ß = -.03 [SE = .02]).

Across source unique effects.

Next, all seven impairment factors were regressed on the general reference and two specific residual factors for all three sources simultaneously. Only two across-source unique effects met our alpha level (.001). Higher levels on the specific ADHD-IN residual factor for teachers uniquely predicted higher scores on the academic impairment factor for mothers and fathers, ß = .37 (SE = .03) and ß = .34 (SE = .04), respectively.

Discussion

The symmetrical bifactor model is often used to examine the structure of ADHD and ODD symptoms. Without exception, all 22 studies reviewed by Arias et al. (2016) as well as others (e.g., Rodenacker et al., 2016, 2017, 2018) concluded that the symmetrical bifactor model was a better model than first-order factor models. Use of the symmetrical bifactor model with ADHD/ODD symptom ratings, however, frequently yields anomalous or inadmissible solutions such as unexpected loading patterns or negative factor variances. More than 75% of applications to ADHD and ADHD/ODD symptoms that we reviewed showed such problems, with the most frequent issue being small or negative factor loadings, in particular for specific factors pertaining to either ADHD-HI or ADHD-HY.

When specific factors have symptom loadings that are close to zero or negative, the meaning of general and specific factors is usually compromised. For example, if the ADHD-HI specific factor collapses or is weakly defined due to many small loadings, the general factor becomes equal to the specific HI factor and can no longer be interpreted as a general disruptive behavior factor underlying all the symptoms. In addition, in our literature review, we frequently observed anomalous loadings for the symptoms on the ADHD-HI specific factor (i.e., either ADHD-HY or ADHD-IM symptom loadings were non-substantial or negative on the HI specific factor).

In our own applications of the symmetrical bifactor model presented in this paper, anomalous ADHD-HI factor loadings were found for all three sources (mothers, fathers, and teachers), with several symptoms showing close-to-zero and/or negative ADHD-HI specific factor loadings for each source. Although the ADHD-HI specific factor was consistently affected across raters, different items showed anomalous loadings across raters. For mothers and teachers, symptoms representing the hyperactivity component of ADHD-HI were affected whereas for fathers, symptoms representing the impulsivity component of ADHD-HI showed small or negative loadings. As a result, the intended meaning of the general and specific factors in the symmetrical bifactor model did not match the assumed meaning within symmetrical bifactor model.

Not only did the general factor become specific to ADHD-HI symptoms, its meaning also varied across sources. For mothers and teachers, the general factor was most properly interpreted as the hyperactivity facet of ADHD-HI, whereas for fathers the general factor represented mostly the impulsivity facet of ADHD-HI. As a result of such variability in how the general factor is empirically defined across different applications of the symmetrical bifactor model, problems may occur when relations between the general factor and external variables are examined (Eid et al., 2018).

When the general factor in the symmetrical bifactor model has the definitional problems described in the above paragraph and is correlated with external variables, different results may be found across studies (or across raters within the same study, Eid et al, 2018). These differences may be simply due to the fact that the general factor has different meanings across raters and/or studies (e.g., ADHD-HY for mothers and teachers versus ADHD-IM for fathers as in our application). It is also possible that the general factor shows the same associations with external correlates for different sources, even when the general factor has a different meaning for each source (e.g., an ADHD-HY general factor for mothers and an ADHD-IM general factors for fathers).

The likely reason for anomalous and/or inadmissible solutions is that the symmetrical bifactor model implies a measurement design with randomly sampled interchangeable facets, whereas ADHD/ODD symptoms represent structurally different fixed facets (Eid et al., 2017; Geiser et al., 2015; Heinrich et al., 2018). The bifactor S – 1 model solves problems associated with the symmetrical bifactor model because it takes the fixed, structurally different nature of facets into account. Rather than defining a general factor as a common factor across all facets, the S – 1 approach contrasts different symptom groups against a reference symptom group. The general factor is defined by reference symptoms, with no specific factor for these symptoms. Specific factors are clearly defined as residual factors in reference to the general factor, and are allowed to correlate with all other specific factors. The general and specific factors thus have an a priori defined meaning that does not differ across sources and that does not change when other facets are added or deleted from the model.

We used ADHD-HI symptoms to define the general reference factor, with two specific residual factors comprising ADHD-IN and ODD symptoms. Specific ADHD-IN and ODD symptoms were expected to have a substantial amount of true score variance independent of the general ADHD-HI reference factor (ADHD-HI symptoms). Trait impulsivity theory of ADHD/ODD development guided our decision to define the general reference factor with ADHD-HI symptoms, and the prediction that the two specific residual factors would contain a substantial amount of true score variance independent of the general reference factor. Our decision to use ADHD-HI as reference facet was further backed up by our literature review, which revealed that specific factors for ADHD-HI (or HY) were those that most frequently showed anomalous factor loadings in previous applications of symmetrical bifactor models.

The bifactor S – 1 model yielded admissible and clearly interpretable solutions with no unexpected loadings for mothers, fathers, or teachers. ADHD-HI symptoms had high loadings on the general reference factor, with specific ADHD-IN and ODD residual factors containing a substantial amount of true score variance independent of the general ADHD-HI reference factor. The general ADHD-HI reference and specific ADHD-IN and ODD residual factors also showed convergent and discriminant validity across all three sources. Finally, the bifactor S – 1 model showed invariance of like-symptom loadings and thresholds as well as no difference on the general ADHD-HI factor means across mothers and fathers. Symptom loadings, however, were higher for teachers than parents.

Although the above results are important for establishing construct validity, factors also need to have unique correlates. The specific ADHD-IN residual factor had a much larger unique association than the general ADHD-HI reference factor and specific ODD residual factor with the academic impairment factor. In contrast, the general ADHD-HI reference factor and specific ODD residual factor had much larger unique associations than the specific ADHD-IN residual factor with the social impairment factor for mothers and fathers and the peer rejection factor for teachers. For teachers, however, all three factors had similar unique associations with social impairment, perhaps given the importance of ADHD-IN symptoms for classroom adjustment. Finally, the specific ADHD-IN residual factor for teachers predicted mother and father perception of academic impairment, even after controlling for all other factors. The general ADHD-HI reference factor and specific ADHD-IN and ODD residual factors thus showed theoretically consistent correlates.

Implications of the Bifactor S – 1 Model for ADHD/ODD Research

Explanation of ADHD-IN, ADHD-HI, and ODD symptom dimensions results from identification of common and unique correlates of the symptom dimensions (e.g., causes, risk factors, associated features, prognosis, and responses to treatments). Countless studies have identified correlates associated with the ADHD-IN, ADHD-HI, and ODD symptom dimensions (factors) (e.g., Beauchaine et al. 2017; McDonough-Caplan and Beauchaine 2018; Willcutt et al. 2012). This research provides much of our current understanding of ADHD and ODD.

The bifactor S – 1 model offers an alternative approach to examine correlates of ADHD-HI, ADHD-IN, and ODD symptoms. The bifactor S – 1 model, however, requires a priori specification of the general reference factor. Selection of the general reference factor should ideally follow from theoretical models of ADHD/ODD symptom development. In the current study, trait impulsivity theory of ADHD/ODD development provided justification to define the general reference factor with ADHD-HI symptoms (e.g., Beauchaine et al., 2017). This justification was based on etiological processes associated with ADHD-HI, ADHD-IN, and ODD symptom development, whereby hyperactivity-impulsivity represents a highly heritable, temperamental predisposition to development of both inattentive and oppositional symptoms among affected children. Purely inattentive ADHD is specified by a different etiological pathway that does not portend heightened risk for ODD. Our application of the bifactor S – 1 model accommodates both pathways by allowing IN and ODD symptoms to simultaneously relate to the HI general factor while being represented by independent factors. This interpretation predicts that specific ADHD-IN and specific ODD residual factors should contain substantial true score variance, independent of the general ADHD-HI reference factor. Our findings are consistent with these predictions. It should be noted, however, that structural models of symptoms rarely provide strong tests of etiology given the complexity of contributing factors across genetic, neural, hormonal, and environmental levels of analysis (e.g., Beauchaine & Constantino, 2017). Nevertheless, consistency with predictions derived from etiological theory is important.

The bifactor S – 1 model also allowed for unambiguous evaluation of predictions about correlates of general and specific residual factors. In the current study, these correlates were limited to social impairment, peer rejection, and academic impairment. However, a much broader range of correlates could be selected on the basis of theory with the bifactor S – 1 model allowing a psychometrically sound test of the predictions. For all these reasons, the bifactor S – 1 model offers a new approach to study the correlates of the ADHD-HI, ADHD-IN, and ODD symptoms dimensions relative to the examination of the correlates of the three first-order factors.

Summary

Primary foci of our study were to (1) demonstrate the problems associated with the application of the symmetrical bifactor model to ADHD/ODD symptoms, (2) demonstrate the absence of such problems and the clarity of interpretations with the application of the bifactor S – 1 model to ADHD/ODD symptoms, (3) evaluate certain predictions from trait impulsivity theory of ADHD/ODD development; and (4) indicate the overall the usefulness of the bifactor S – 1 model to examine the internal and external structure of ADHD/ODD symptoms. The bifactor S – 1 model is also a better model than the symmetrical bifactor model for development and evaluation of p models of psychopathology (e.g., Caspi et al., 2014). We encourage researchers to use the bifactor S – 1 model in research on the hierarchical nature of psychopathology.

Supplementary Material

Supplemental Tables

Acknowledgments

This research was supported by two grants from the Ministry of Economy and Competitiveness of Spanish Government under award numbers PSI2014-52605-R and PSI2017-82550-R (AEI/FEDER, UE), and a predoctoral fellowship co-financed by MINECO (Spanish Government) and the European Social Fund (BES-2015-075142). Stephen Becker is supported by award number K23MH108603 from the National Institute of Mental Health (NIMH). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health (NIH) or the Spanish Government. We thank Cristina Trias for assistance with the study.

Footnotes

Conflict of Interest: The authors of the current study have no conflicts of interest.

Ethical Approval: The research protocol was approved by the Research Ethics Committee (Institutional Review Board [IRB]) of the University of the Balearic Islands.

Informed Consent: Mothers, fathers, and teachers provided written informed consent.

1

Several studies first applied the symmetrical bifactor model to ADHD-HI, ADHD-IN, and ODD symptoms and, with anomalous results (i.e., a specific ADHD-HI factor with no factor variance and negative loadings), then switched to a model referred to as an incomplete bifactor model (i.e., no specific factor for hyperactivity symptoms but with specific factors for ADHD-IN, ADHD-impulsivity and ODD symptoms, e.g., Rodenacker et al., 2017, 2018; Ullebø et al., 2012). These studies therefore switched from the fully symmetrical to an incomplete bifactor model without awareness of the interpretational differences between the fully symmetrical and incomplete models (i.e., in these studies the general factor became a general ADHD-hyperactivity reference factor with specific ADHD-IN, ADHD-impulsivity and ODD residual factors in the incomplete bifactor model).

2

Invariance of like-symptom loadings and like-symptom thresholds occurred for ADHD-HI, ADHD-IN, and ODD symptoms across boys and girls within each source. Boys had significantly (ps < .001) higher factor means than girls on the general ADHD-HI reference factor for mothers, fathers, and teachers (i.e., latent d values: M = .30, SE = .07, M = .40, SE = .07, and M = .58, SE = .10, respectively).

Contributor Information

G. Leonard Burns, Washington State University.

Christian Geiser, Utah State University.

Mateu Servera, University of the Balearic Islands.

Stephen P. Becker, Cincinnati Children’s Hospital Medical Center and University of Cincinnati College of Medicine

Theodore P. Beauchaine, The Ohio State University

References

  1. Arias VB, Ponce FP, & Núñez DE (2018). Bifactor models of attention-deficit/hyperactivity disorder (ADHD): An evaluation of three necessary but underused psychometric indexes. Assessment, 25, 885–897. doi: 10.1177/1073191116679260 [DOI] [PubMed] [Google Scholar]
  2. Beauchaine TP (2015). Future directions in emotion dysregulation and youth psychopathology. Journal of Clinical Child and Adolescent Psychology, 44, 875–896. doi.org/10.1080/15374416.2015.1038827 [DOI] [PubMed] [Google Scholar]
  3. Beauchaine TP, & Constantino JN (2017). Redefining the endophenotype concept to accommodate transdiagnostic vulnerabilities and etiological complexity. Biomarkers in Medicine, 11, 769–780. doi: 10.221/bmm-2017-0002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Beauchaine TP, & McNulty T (2013). Comorbidities and continuities as ontogenic processes: Toward a developmental spectrum model of externalizing psychopathology. Development and Psychopathology, 25, 1505–1528. doi.org/10.1017/S0954579413000746 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Beauchaine TP, Zisner A, & Sauder CL (2017). Trait impulsivity and the externalizing spectrum. Annual Review of Clinical Psychology, 13, 343–368. doi: 10.1146/annurev-clinpsy-021815-093253 [DOI] [PubMed] [Google Scholar]
  6. Becker SP, Burns GL, Schmitt AP, Epstein JN, & Tamm L (2019). Toward establishing a standard symptom set for assessing sluggish cognitive tempo in children: Evidence from teacher ratings in a community sample. Assessment, 26, 1128–1141. doi.org/10.1177/1073191117715732 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Burns GL, & Becker SP (2019). Sluggish cognitive tempo and ADHD symptoms in a nationally representative sample of U.S. children: Differentiation using categorical and dimensional approaches. Journal of Clinical Child and Adolescent Psychology. Advance online publication. doi: 10.1080/15374416.2019.1678165 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Burns GL, Lee S, Servera M, McBurnett K, & Becker SP (2015a). Child and Adolescent Behavior Inventory–Parent Version 1.0. Pullman, WA: Authors. [Google Scholar]
  9. Burns GL, Lee S, Servera M, McBurnett K, & Becker SP (2015b). Child and Adolescent Behavior Inventory–Teacher Version 1.0. Pullman, WA: Authors. [Google Scholar]
  10. Burns GL, Moura MA, Beauchaine TP, & McBurnett K (2014a). Bifactor latent structure of ADHD/ODD symptoms: Predictions of dual-pathway/trait-impulsivity etiological models of ADHD. Journal of Child Psychology and Psychiatry, 55, 393–401. doi: 10.1111/jcpp.12165 [DOI] [PubMed] [Google Scholar]
  11. Burns GL, Servera M, Bernard M, Carrillo JM, & Geiser C (2014b). Ratings of ADHD symptoms and academic impairment by mothers, fathers, teachers and aides: Construct validity within and across settings as well as occasions. Psychological Assessment, 26, 1247–1258. doi: 10.1037/pas0000008 [DOI] [PubMed] [Google Scholar]
  12. Caspi A, Houts RM, Belsky DW, Goldman-Mellor SJ, Harrington H, Israel S, … & Moffitt TE. (2014). The p Factor: One general psychopathology factor in the structure of psychiatric disorders. Clinical Psychological Science, 2, 119–137. doi: 10.1177/2167702613497473 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Diamond A (2005). Attention deficit disorder (ADHD without hyperactivity): A neurological and behavioral distinct disorder from ADHD (with hyperactivity). Development and Psychopathology, 17, 807–825. doi: 10.1017/S0954579405050388 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Dishion TJ (1990). The peer context of troublesome child and adolescent behavior. In Leone PE (Ed.), Understanding troubled and troubling youth: Multiple perspectives (pp. 128–153). Thousand Oaks, CA: Sage. [Google Scholar]
  15. DuPaul GJ, Reid R, Anastopoulos AD, Lambert MC, Watkins MW, & Power TJ (2015). Parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms: Factor structure and normative data. Psychological Assessment, 28, 214–225. 10.1037/pas0000166 [DOI] [PubMed] [Google Scholar]
  16. Eid M, Geiser C, Koch T, & Heene M (2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22, 541–562. doi: 10.1037/met0000083 [DOI] [PubMed] [Google Scholar]
  17. Eid M, Krumm S, Koch T, & Schulze J (2018). Bifactor models for predicting criteria by general and specific factors: Problems of nonidentifiability and alternative solutions. Journal of Intelligence. Advance online publication. doi: 10.3390/jintelligence6030042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fair DA, Nigg JT, Iyer S, Bathula D, Mills KL, Dosenbach NUF, … Milham MP. (2013). Distinct neural signatures detected for ADHD subtypes after controlling for micro-movements in resting state functional connectivity MRI data. Frontiers in Systems Neuroscience, 6, 80. doi: 10.3389/fnsys.2012.00080 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gatzke-Kopp LM, Beauchaine TP, Shannon KE, Chipman J, Fleming AP, Crowell SE, … & Aylward E. (2009). Neurological correlates of reward responding in adolescents with conduct disorder and/or attention-deficit/hyperactivity disorder. Journal of Abnormal Psychology, 118, 203–213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Geiser C, Bishop J, & Lockhart G (2015). Collapsing factors in multitrait-multimethod models: Examining consequences of a mismatch between measurement design and model. Frontiers in Psychology, 6:946. doi: 10.3389/fpsyg.2015.00946 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Heinrich M, Zagorscak P, Eid M, & Knaevelsrud C (2018). Giving G a meaning: An application of the bifactor-(S-1) approach to realize a more symptom-oriented modeling of the Beck Depression Inventory–II. Assessment. Advance online publication. doi: 10.1177/1073191118803738.2016.1144192 [DOI] [PubMed] [Google Scholar]
  22. Holzinger K, & Swineford F (1937). The bi-factor method. Psychometrika, 2, 41–54. 10.1007/BF02287965 [DOI] [Google Scholar]
  23. Lee SY, Burns GL, Beauchaine T, & Becker SP (2016). Bifactor latent structure of ADHD/ODD symptoms and first-order latent structure of sluggish cognitive tempo symptoms. Psychological Assessment, 28, 917–928. doi.org/10.1037/pas0000232 [DOI] [PubMed] [Google Scholar]
  24. Martel MM, Gremillion M, Roberts B, von Eye A, & Nigg JT (2010). The structure of childhood disruptive behaviors. Psychological Assessment, 22, 816–826. 10.1037/a0020975 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Gatzke-Kopp LM, Beauchaine TP, Shannon KE, Chipman J, Fleming AP, Crowell SE, … & Aylward E. (2009). Neurological correlates of reward responding in adolescents with conduct disorder and/or attention-deficit/hyperactivity disorder. Journal of Abnormal Psychology, 118, 203–213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Reise SP (2012). The rediscovery of the bifactor measurement models. Multivariate Behavioral Research, 47, 667–696. 10.1080/00273171.2012.715555 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Rodenacker K, Hautmann C, Görtz-Dorten A & Döpfner M (2018). Evidence for the trait-impulsivity etiological models in a clinical sample: Bifactor structure and its relation to impairment and environmental risk. Journal of Abnormal Child Psychology, 46, 659–669. doi: 10.1007s/10802-017-0329-y. [DOI] [PubMed] [Google Scholar]
  28. Rodenacker K, Hautmann C, Görtz-Dorten A & Döpfner M (2017). The factor structure of ADHD—different models, analyses, and informants in a bifactor framework. Journal of Psychopathology and Behavioral Assessment, 39, 92–102. doi: 10.1007/s10862-016-9565-7 [DOI] [Google Scholar]
  29. Rodenacker K, Hautmann C, Görtz-Dorten A & Döpfner M (2016). Bifactor models show a superior fit: Examination of the factorial validity of parent-reported and self-reported symptoms of attention-deficit/hyperactivity disorders in children and adolescents. Psychopathology, 49, 31–39. doi: 10.1159/000442295. [DOI] [PubMed] [Google Scholar]
  30. Sáez B, Servera M, Becker SP, & Burns GL (2019). Optimal items for assessing sluggish cognitive tempo in children across mother, father, and teacher ratings. Journal of Clinical Child and Adolescent Psychology, 48, 825–839. doi: 10.1080/15374416.2017.1416619 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Servera M, Sáez B, Burns GL, & Becker SP (2018). Clinical differentiation of sluggish cognitive tempo and attention-deficit/hyperactivity disorder in children. Journal of Abnormal Psychology, 127, 818–829. doi: 10.1037/abn0000375 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Ullebø AK, Breivik K, Gillberg C, Lundervold AJ, & Posserud MB (2012). The factor structure of ADHD in a general population of primary school children. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 53, 927–936. doi: 10.1111/j.1469-7610.2012.02549.x [DOI] [PubMed] [Google Scholar]
  33. Watts AL, Poore HE, & Waldman ID (2019). Risker tests of the validity of the bifactor model of psychopathology. Clinical Psychological Science. Advance online publication. doi: 10.1177/2167702619855035 [DOI] [Google Scholar]
  34. Willcutt EG, Nigg JT, Pennington BF, Solanto MV, Rohde LA, Tannock R, … Lahey BB (2012). Journal of Abnormal Psychology, 121, 991–1010.doi: org/10.1037/a0027347 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Tables

RESOURCES