Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Oct 1.
Published in final edited form as: J Adolesc. 2015 Aug 14;44:182–190. doi: 10.1016/j.adolescence.2015.07.003

DISRUPTIVE BEHAVIOR DISORDERS AND INDICATORS OF DISINHIBITION IN ADOLESCENTS: THE BRIEF-SR, ANTI-SACCADE TASK, AND D-KEFS COLOR-WORD INTERFERENCE TEST

E C Long a,c,1,*, J Hill a, B Luna b, B Verhulst c, D B Clark b
PMCID: PMC4632655  NIHMSID: NIHMS708949  PMID: 26277405

Abstract

Disinhibition contributes to the development of disruptive behavior disorders (DBD) in adolescents. Self-reports and behavioral tasks are commonly used to assess disinhibition, each with their unique strengths and limitations. Accordingly, it is important to identify which measure, or combination thereof, is the most effective in predicting DBD symptoms. This study assessed the relationship between DBD (symptoms of ADHD/ODD/CD) and two behavioral disinhibition tasks: the anti-saccade task and the D-KEFS color-word interference test), as well as a self-report measure (the BRIEF-SR). The results indicated that the BRIEF-Inhibit scale accounted for the majority of the variance in the DBD sum score. The anti-saccade task and color-word interference test were also significantly associated with an increase in the number of DBD symptoms endorsed. These behavioral tasks accounted for 9% additional variance than the self-report alone. Therefore, combining self-report measures with behavioral disinhibition tasks may provide the most thorough assessment of adolescent DBD.

Keywords: Disinhibition, measurement, self-report, laboratory behavioral tasks, disruptive behavior disorders

Introduction

Disinhibition refers to difficulties with behavioral, cognitive, and emotional regulation (Tarter et al., 2003). Classically, it has been defined as “human behavior that has been interpreted as arising from lessened controls on response inclinations” (Gorenstein & Newman, 1980, p. 302). Accordingly, impulsivity is an essential component of disinhibition, as problems with impulsivity stem from difficulties in inhibitory processing (Logan, Schachar, & Tannock, 1997). There is substantial evidence demonstrating that disinhibition contributes to the development of disruptive behavior disorders (DBD) in adolescents, as defined by symptoms of attention deficit hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), and conduct disorder (CD; Clark, Chung, Thatcher, Pajtek, & Long, 2012; Dougherty et al., 2003; Habeych, Folan, Luna, & Tarter, 2006). Furthermore, instruments used to measure disinhibition, such as self-report measures and laboratory-based behavioral tasks, are widely used in both clinical and research settings, but the empirical evidence regarding the relationships among these instruments and their relationship to DBD remain somewhat unclear.

Self-report measures of disinhibition, such as the Behavioral Rating Inventory of Executive Function, Self-Report version (BRIEF-SR) and the Barratt Impulsivity Scale require participants to complete a battery of items that assess different constructs of disinhibition. Alternatively, laboratory-based behavioral tasks, such as the anti-saccade task and Stroop task, also provide effective assessments of disinhibition in both nonclinical populations (Enticott, Ogloff, & Bradshaw, 2006) and clinical populations (Holmes et al., 2010). These measures of disinhibition may provide valid measures of different underlying etiological factors.

Self-report and behavioral measures each have specific advantages and limitations. While self-report measures may be biased by faulty memory and subject to social desirability pressures, they are easy to administer and interpret, making them very popular and accessible. Alternatively, behavioral tasks, such as the behavioral disinhibition tasks used in the present study, are not subject to the same social desirability and recall biases as self-report measures, because they infer impulsivity from observed behavior. Further, behavioral inhibition tasks are sensitive to state-dependent impulsivity and may be more adaptable to repeated administration when compared to self-report measures (Dougherty et al., 2003). However, these tasks are typically administered in a controlled environment, which diminishes ecological validity (Enticott et al, 2006), and may produce practice effects with repeated administrations (i.e., better performance after more trials due to increased familiarity with the task; Falleti, Maruff, Collie, & Darby, 2006; Heiman, 2002).

Centrally, these behavioral measures are effective predictors of clinically diagnosed DBD. Response disinhibition and working memory deficits have been shown to discriminate between children with and without ADHD on a variety of behavioral tasks (Berger, Alyagon, Hadaya, Atzaba-Poria, & Auerbach, 2013; Dolan & Lennox, 2013; Holmes et al., 2010). Importantly, the deficiencies found in children with ODD and CD were not due to the comorbid diagnosis of ADHD (Saarinen, Fontell, Vuontela, Carlson, & Aronen, 2014). Finally, four separate meta-analyses were conducted examining the relationship between a variety of executive functioning behavioral tasks (including disinhibition tasks such as the Stroop task and go/no go task) and externalizing behavior problems in preschool children (i.e., a diagnosis or symptoms of ADHD and ODD/CD). Results showed that executive function deficits, in particular difficulty with inhibition, were associated with externalizing behavior problems (Schoemaker, Mulder, Dekovic, & Matthys, 2013). Taken together, these findings imply that behavioral measures may compliment the information provided by DBD diagnosis. In the current study, we specifically use the anti-saccade task and the D-KEFS color-word interference task because previous research has suggested that these measures are reliable predictors of DBD in adolescent populations, details of which are provided below.

Self-Report Measures of Disinhibition

Self-report measures of disinhibition have been designed to assess overall severity as well as hypothesized distinct dimensions (e.g., impaired inhibitory control, impulsive decision-making; Ivanov, Schulz, London, & Newcorn, 2008). The BRIEF-SR is a self-report instrument that is designed to measure impulsivity across the domains of cognition, behavior, and mood along with other executive functioning dimensions. More specifically, the Inhibit scale from the BRIEF assesses disinhibition. Scores on the BRIEF are correlated with symptoms of DBD in adolescents (Clark et al., 2012; Mahone et al., 2002) and are predictive of ADHD subtypes (Isquith & Gioia, 2000; McCandless & Laughlin, 2007; Reddy, Hale, & Brodzinsky, 2011).

Behavioral Disinhibition Tasks

Behavioral disinhibition tasks measure an individual’s ability to inhibit a reflexive, pre-potent response (i.e., a response they are ready to give). Individuals with higher levels of disinhibition have more difficulty inhibiting these automatic responses. One of the most established disinhibition tasks is the Stroop task (Stroop, 1935). Based upon the classic Stroop task, the Delis-Kaplan Executive Function System (D-KEFS) color-word interference test measures the ability to read the name of a color rather than naming the ink color in which the word is printed (Delis, Kaplan, & Kramer, 2001a). This test, along with three other tests from the D-KEFS (trail making, verbal fluency, and tower tests) has identified executive dysfunction in children with ADHD using summary scores, but not contrast scores (Wodka et al., 2008a) or process scores (Wodka et al., 2008b). Slower response times and higher error rates have also been found in children with ADHD compared to controls (Bledsoe, Semrud-Clikeman, & Pliszka, 2010; Holmes et al., 2010). Finally, response inhibition, as measured by condition three on the color-word interference test, has been shown to be more elevated in boys with ADHD than girls (O’Brien, Dowell, Mostofksy, Denckla, & Mahone, 2010), in unaffected siblings of children with ADHD compared to controls (Nickolas & Nigg, 2014), and to predict future working memory deficits (Tillman, Brocki, Sorensen, & Lundervold, 2013).

More recently, the anti-saccade (Hallet, 1978) task has emerged as a popular measure of response inhibition (Geier & Luna, 2009; Munoz & Everling, 2004). This task requires the participant to suppress reflexive eye movements, also termed pre-potent saccades, and make a voluntary eye movement to the location opposite of a presented stimulus (Geier & Luna, 2009). Response inhibition, as measured by the anti-saccade task, is significantly associated with disinhibition, more so than other executive function tasks (working memory and set-shifting; Young et al., 2009). Additionally, adolescents with ADHD show impairment on this task in terms of accuracy and response time (Carr, Henderson, & Nigg, 2010; Hanisch, Radach, Holtkamp, Herpertz-Dahlmann, & Konrad, 2005; Karatekin, 2006; Loe, Feldman, Yasui, & Luna, 2009). Slower reaction times are also related to ADHD symptom counts, suggesting a relationship between inhibition deficits and ADHD severity (Van der Stigchel et al., 2007).

There has been a substantial amount of research examining the ability of self-reports and behavioral disinhibition tasks to assess disinhibition and its association with DBD separately. However, studies that have examined the associations between all of these measures together have yielded inconsistent results and have narrowly focused on ADHD. For example, one study that compared behavioral measures such as the stop task with the parent and teacher ratings of the BRIEF found that behavioral measures predicted ADHD status when examined separately from the BRIEF ratings (Toplak, Bucciarelli, Jain, & Tannock, 2008). When the BRIEF ratings were added into the analyses, only the BRIEF ratings emerged as significant predictors of ADHD. The BRIEF ratings also contributed to the bulk of unique variance, whereas the performance-based measures only accounted for a small amount. Because there was little overlap in the amount of variance explained between the BRIEF ratings and performance-based measures, the authors suggested that performance-based measures supplement assessment of ADHD. Another study found the BRIEF was associated with behavioral disruption and impairment, but found no significant correlations between BRIEF ratings and behavioral disinhibition tasks (McAuley, Chen, Goos, Schachar, & Crosbie, 2010). The finding that self-report ratings and behavioral tasks are not correlated is consistent with some previous research (Barkley & Murphey, 2011; Vriezen & Pigott, 2002), but inconsistent with other research that has found modest correlations (Toplak et al., 2008; Enticott et al., 2006). Because of these inconsistent findings, further examination is needed to better understand whether behavioral measures and self-report measures are redundant or can be complementary by capturing unique variance.

Study Aims

The primary aim of the present study is to find the best predictor, or set of predictors of adolescent psychopathology, as measured by DBD symptom counts, using the BRIEF-Inhibit scale, the anti-saccade task, and the D-KEFS color-word interference test. An additional aim is to assess whether these measures are correlated, which may suggest they are redundant. Our specific hypotheses are:

Hypothesis I: We expect that the measures will be significantly but modestly correlated. As all of the variables are expected to predict DBD symptoms, we expect a modest relationship between them. However, as we expect the behavioral measures to uniquely account for variation in DBD symptoms, we do not expect these relationships to be very large.

Hypothesis II: We hypothesize that self-report and behavioral measures will be significant predictors of DBD symptoms. Specifically, while we expect the self-report to account for the greatest bulk of variance in DBD symptoms, we expect the behavioral measures to account for additional variance above and beyond what is captured by the self-report measure. Accordingly, all of the measures will contribute to our understanding of DBD.

Method

Participants

The present investigation utilized data from a study that investigated relationships among adolescent brain development, executive functioning, and substance use disorder risk factors (such as DBD). One hundred and twenty-one adolescent participants between the ages of 12 and 15 (M = 14.09, SD = 1.15) and their primary caretakers were recruited using random digit dialing procedures. Fifty-eight of the participants were males (48%) and 63 were female (52%). Caretakers were compensated $50.00 and adolescent participants were compensated $160.00. Written informed assent was obtained from all adolescent participants and written informed consent was obtained from the adolescents’ accompanying primary caretakers. The University of Pittsburgh Institutional Review Board approved all study procedures.

Procedure

Adolescent participants arrived with and their primary caretaker, who underwent a clinical interview with a trained assessor that consisted of demographic information as well as the Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS) to report on their adolescent’s behavior and assess for Axis I disorders. In addition, adolescent participants completed a clinical interview that consisted of demographic information, the K-SADS, several self-report questionnaires, and a battery of neuropsychological tests. Finally, participants were escorted to the Magnetic Resonance Research Center at the University of Pittsburgh Medical Center Presbyterian Hospital where they underwent an fMRI, during which they completed the anti-saccade task. All together, the caretakers and adolescents spent a full 8-hour day completing the protocol. The present study focuses on information from the K-SADS, BRIEF-Inhibit scale, the anti-saccade task, and the D-KEFS color-word interference test.

Materials

Kiddie-Schedule for Affective Disorders and Schizophrenia (K-SADS)

Our construct of disruptive behavior disorders included symptoms of CD, ODD, and ADHD, as determined by the adolescent version of the K-SADS (Kaufman et al., 1997). When determining whether the adolescent met criteria for a symptom, we considered information from both the parental report and adolescent report. For symptom items and diagnoses, consensus was reached in a diagnostic conference meeting between the interviewer and a clinically experienced faculty psychiatrist or clinical psychologist using the best-estimate method (Kosten & Rounsaville, 1992; Leckman et al., 1982). In general, when adolescent and parent reports were inconsistent, more weight was placed on the parental report for symptoms with objective referents, such as functional deficits in ADHD, and more weight was placed on the adolescent report for symptoms with subjective characteristics, such as anhedonia in major depressive disorder. Using this type of multi-informant approach yields a more reliable and valid consensus measure (Kraemer et al., 2003). While ADHD, ODD, and CD have both common and specific features, all of these diagnoses involve difficulties with inhibition. Furthermore, in twin studies with children and adolescents, a single genetic dimension has been found to best represent DBD characteristics (Hicks, Krueger, Iacono, McGue, & Patrick, 2004; Tuvblad, Zheng, Raine, & Baker, 2009; Young, Rhee, Stallings, Corley, & Hewitt, 2006). We therefore computed a sum score of DBD symptom counts as the outcome of interest in this study (M = 3.11; SD = 4.88).

The K-SADS is a semi-structured interview designed to assess DSM-IV Axis I disorders. These clinical diagnostic interviews were conducted by trained assessors and then discussed in a consensus conference with the primary investigator (DBC) to determine diagnoses. The procedures were identical to a previous study, where inter-rater reliability was achieved by conducting joint interviews until greater than 90% agreement was reached with an experienced assessor (Clark et al., 1997). Inter-rater reliability by the K statistic for DBD diagnoses was 0.88.

Behavior Rating Inventory of Executive Function – Self Report Version (BRIEF-SR)

The BRIEF is an 80-item self-report behavior rating scale that measures adolescents’ views of their difficulties with regulation of cognitive, behavioral, and mood domains (Guy, Isquith, & Gioia, 2004) and takes approximately 20 minutes for the participant to complete. Examples of items include, “I have a short attention span,” “I have trouble sitting still,” and “I get upset easily.” The full scale has eight clinical sub-scales: Shift, Task Completion, Inhibit, Emotional Control, Working Memory, Plan/Organize, Organization of Materials, and Monitor. The indexes and clinical scales yield T scores, with T scores above 65 being considered clinically significant and T scores 60 – 64 warranting clinical interpretation. The variable of interest in the current study was the T scores from Inhibit scale because it assesses inhibitory control and impulsivity.

This scale includes 13 items that reflect disinhibited behavior (Guy et al., 2004). Example items include “I have trouble sitting still,” “I act too wild or ‘out of control,’” and “I don’t think of consequences before acting.” BRIEF – Inhibit has been shown to be significantly correlated with the Externalizing scale of the Child Behavior Checklist, which reflects DBD, as well as related subscales including attention problems, delinquent behavior, and aggressive behavior (Guy et al., 2004).

Anti-Saccade Task

The anti-saccade task measures disinhibition by assessing participants’ ability to inhibit automatic responses (Munoz & Everling, 2004). It was administered as part of an fMRI protocol. During the task, participants viewed a computer screen via an angled adjustable mirror. They were first instructed to stare at a white plus sign on the computer screen, which served as a fixation point. A preparation period of 1.5 seconds followed where the white plus sign changed to the color red, which indicated that a stimulus was about to appear. While the plus sign was still red, but before the stimulus appeared, participants were presented with either a neutral cue, in which blue pound signs appeared around the plus sign, or a reward cue, in which green dollar signs appeared around the plus sign. In the reward trials, they were told they could win extra money if they performed accurately on the trial (although all participants were given the same amount of money, regardless of performance on the trial). Finally, a yellow dot appeared on one side of the screen and the participant was instructed to look at the exact opposite location of the stimulus.

The inter-trial fixation period varied among intervals of 1.5, 3 or 4.5 seconds, which were equally distributed. There were 14 reward cues and 14 neutral cues, each presented for 1.5 seconds over a 5-minute session, yielding a total of 28 trials per each session. Each participant underwent 4 5-minute sessions resulting in a grand total of 112 trials for each participant. The preparation and response were also presented for 1.5 seconds (see Figure 1; Geier & Luna, 2009). The total duration of this task was 20 minutes.

Figure 1.

Figure 1

Open in a new tab

Design of the anti-saccade task.

The participants’ eye positions with response times were recorded using a long-range optics eye-tracking system. This system utilizes pupil-corneal reflection. The data were then scored using ILAB software by a trained research assistant. Correct movements with no interruption in response time were coded as a 1,1; incorrect movements with no interruption in response time were coded as a 2,2; refining movements made after a correct response were coded as a 3,3; and corrective movements made following an incorrect movement were coded as a 4,4. Trials were dropped if there were too many blinks, the participant seemed confused, or the data were otherwise unclear. The percentage of correct responses across all trials for each participant was the only score used in these analyses.

Delis-Kaplan Executive Function System (D-KEFS) Color-Word Interference Test

The D-KEFS color-word interference test is a variation of the classic Stroop procedure and measures the executive function task of inhibition (Delis et al., 2001a). This test had four conditions. First, participants viewed a series of color patches that they verbally name (Condition 1). Second, participants viewed a series of color words, displayed in black ink, which they were asked to read aloud (Condition 2). These first two conditions provide a baseline measure of reading performance. Third, participants were presented with color words displayed in different color ink (i.e., the word “green” written in red ink; Condition 3). They were required to inhibit the “natural” verbal responses of reading the name of a color to instead name the ink color in which the word is printed. This is the traditional Stroop task and a measure of inhibition. In the final condition, participants were again presented with color words displayed in different ink. This time, however, they were asked to switch back and forth between naming the ink colors and reading the words (Condition 4). Therefore, some of the time they were performing the traditional Stroop task (i.e., naming the ink colors) and the rest of the time they were doing the reverse (i.e., reading the word). This condition evaluated both inhibition and cognitive flexibility. The task took approximately two minutes to administer each of the four sections, for a total of eight minutes.

Three types of scores can be computed for the color-word interference test: scaled scores based on completion times, contrast scaled scores, and raw scores based on errors (Delis, Kaplan, & Kramer, 2001b). Scores based on completion times consist of the number of seconds that the participant takes to complete each condition. The raw scores are then converted to scales for 16 different age groups. Contrast scores are calculated by subtracting the completion time scaled score for Condition 1 and/or Condition 2 from the completion time scaled score for Condition 3 and/or Condition 4. Thus, performance on Conditions 1 and/or 2 are parceled out from performance on Conditions 3 and/or 4, which takes into account impairments in the fundamental skills of basic naming and reading. Finally, scores based on errors included both corrected and uncorrected responses. An error is classified as corrected if the participant initially made an error but self-corrected it immediately after making the error. Otherwise, it was counted as an uncorrected error. Scores of interest for these analyses were the scaled completion time inhibition score (from Condition 3), the contrast score, number of uncorrected errors made on Condition 3, and number of corrected errors made on Condition 3.

Statistical Analyses

The first hypothesis was tested by estimating Pearson correlations among the measures of disinhibition: the BRIEF-Inhibit scale, percentage of correct responses on the anti-saccade task, and four scores from the color-word interference test (i.e., the scaled completion time inhibition score, the contrast score, uncorrected errors on Condition 3, and corrected errors on Condition 3). To test the second hypothesis, ordinary least squares (OLS) linear and Poisson regression analyses were conducted using the measures of disinhibition as predictors of the sum of DBD symptoms. Percentages and scaled scores for some of the measures were used rather than raw scores for ease of interpretability. Using percentages and scaled scores puts the measures on a theoretically similar metric, allowing for comparability. Because the distribution of the sum score of DBD symptoms was highly positively skewed, the OLS linear regression results are paired with the Poisson regression results, which accounts for the skew by following the Poisson distribution.

Results

To test the first hypothesis, Pearson correlations were estimated to test for redundancy across the measures of disinhibition. The results showed that the anti-saccade task and the color-word interference test scores were not significantly correlated with the BRIEF-Inhibit scale (Table 1), although there was a modest negative correlation between the number of uncorrected errors on the color-word interference test and percentage of correct responses on the anti-saccade task that was of marginal significance (r = −.179; p = .050). Therefore, any relationship between the behavioral measures and the BRIEF can be interpreted as statistically independent. However, there were a number of significant correlations between the four scores within the color-word interference test (i.e., the scaled completion time inhibition score, the contrast score, uncorrected errors, and corrected errors). These significant correlations are likely a function of the fact that these scores are part of the same measure.

Table 1.

Pearson correlations of DBD measures

1. 2. 3. 4. 5. 6.
1. BRIEF-Inhibit Scale 1.00
2. Anti-saccade1 −.019 1.00
3. CW Scaled Completion
Time2 		−.006 .164 1.00
4. CW Contrast Score2 −.075 .004 .270** 1.00
5. CW Uncorrected Errors2 −.005 −.179* −.287** −.044 1.00
6. CW Corrected Errors2 .075 −.027 −.456** −.215* .051 1.00
Open in a new tab
1

Anti-saccade = Percentage of correct responses on the anti-saccade task;

2

CW = D-KEFS Color-Word Interference Test.

**

p < 0.01.

*

p ≤ 0.05.

OLS linear and Poisson regression models were then used to examine the relationship between DBD and the BRIEF-Inhibit scale, anti-saccade task, and the color-word interference test scores are presented in Table 2. As expected, higher scores on the BRIEF-Inhibit scale (indicating more difficulty with inhibition) were associated with significantly higher levels of DBD symptoms endorsed in both models (βols= .216; pols < .001; βpoisson= .055; ppoisson < .001). Lower percentages of correct responses on the anti-saccade task were also associated with significantly higher DBD sum scores, albeit marginally significant in the linear regression model (βols= −.032; pols = .055; βpoisson= −.009; ppoisson < .001). Finally, the scaled completion time inhibition score was significant only in the Poisson regression model (βols= −.080; pols = .583; βpoisson= −.053; ppoisson = .008). Likewise, both the number of uncorrected (βols= .462; pols = .071; βpoisson= .138; ppoisson < .001) and corrected errors (βols= .355; pols = .103; βpoisson= .098; ppoisson < .001) made on Condition 3 was significantly associated with an increase in DBD symptoms only in the Poisson regression model. Because the Poisson model corrects for our skewed data, we believe this model yields the most accurate estimates. The contrast score was not a significant predictor in either model. Accordingly, in line with Hypothesis 1, the BRIEF scale clearly predicts DBD symptoms, but the anti-saccade task and the color-word interference tasks also predict DBD symptoms, above and beyond the self-report measure.

Table 2.

Predictors of DBD

Linear Regression Poisson Regression
β
(se) 		p β
(se) 		p
BRIEF-Inhibit Scale .216
(.034)		 <.001 .055
(.004)		 <.001
Anti-saccade1 −.032
(.017)		 .055 −.009
(.002)		 <.001
CW Scaled Completion Time2 −.080
(.146)		 .583 −.053
(.019)		 .008
CW Contrast Score2 .042
(.156)		 .788 −.014
(.020)		 .478
CW Uncorrected Errors2 .462
(.254)		 .071 .138
(.029)		 <.001
CW Corrected Errors2 .355
(.216)		 .103 .098
(.027)		 <.001
R2 .345
Akaike Information Criteria
(AIC) 		662.29
Open in a new tab
1

Anti-saccade = Percentage of correct responses on the anti-saccade task;

2

CW = D-KEFS Color-Word Interference Test

When all variables were included in the OLS linear regression model, the R2 was 0.345, indicating that 34.5% of the variance in the DBD sum score was explained by the predictors. To determine how much additional variance is accounted for by the behavioral tasks above and beyond that accounted for by the BRIEF-Inhibit scale alone, an additional OLS linear regression analysis was conducted with only the BRIEF-Inhibit scale. This model showed that the BRIEF-Inhibit scale accounted for 25.4% of the variance. As such, the behavioral measures accounted for an additional 9% of the variance, suggesting that the addition of both behavioral tasks accounts for more variance than the use of self-report alone.

Discussion

The present study found that both the self-report and behavioral measures are important predictors of DBD symptoms. Further, while the BRIEF-Inhibit scale accounted for approximately one-fourth of the variance in DBD symptoms, the anti-saccade task and color-word interference test accounted for an additional 9% of the variance, suggesting that behavioral measures used in conjunction with self-report provides the most thorough assessment of disinhibition and more accurately predicts adolescent psychopathology. The fact that the behavioral measures were not correlated with the BRIEF-Inhibit scale, however, implies the behavioral measures are capturing unique information above and beyond the information captured by the BRIEF-Inhibit scale. This is reinforced by the finding that inclusion of the behavioral measures increased the amount of variance explained.

The analyses were conducted at the T-score level for the BRIEF to allow us to compare the different measures. Using standardized scores places the measures on a theoretically similar metric. In addition, T-scores are normed against children of a similar age, therefore providing information that is relative to the children of similar ages in the standardization sample (Guy et al., 2004).

Consistent with previous research (Carr, Henderson, & Nigg, 2010; Hanisch, Radach, Holtkamp, Herpertz-Dahlmann, & Konrad, 2005; Karatekin, 2006; Loe, Feldman, Yasui, & Luna, 2009; Van der Stigchel et al., 2007), we showed that poor performance on the anti-saccade task is associated with symptoms of DBD. However, the majority of previous research focused on ADHD symptoms. Therefore, our results extend the use of the measure to other DBD symptoms and accounts for additional unique variance in the construct of DBD, which facilitates greater understanding of the underlying processes contributing to DBD symptoms.

Also consistent with previous research (Bledsoe et al., 2010; Holmes et al., 2010; Wodka et al., 2008a), the results from the Poisson regression showed that slower completion times and more errors (both corrected and uncorrected) on Condition 3 (Inhibition) of the color-word interference test were associated with a significant increase in the number of DBD symptoms. However, the contrast score was not a significant predictor of DBD symptoms. One potential interpretation of this finding is that the completion time score, which is the primary method for analyzing performance, assesses general impairment in fundamental inhibition skills (Delis et al., 2001b). The contrast score can be used as an alternative and assesses whether the participant exhibits more impairment than expected on the higher-level task than the baseline measures. Thus, as Wodka et al. (2008a) suggests, completion time scores may more closely capture the underlying executive dysfunction that distinguishes the participants with DBD from those without.

One potential criticism of the use of multiple measures is that all of the measures may be assessing the same construct and conflicts with parsimony. Because these measures were not correlated, however, they are not redundant measures and instead offer unique etiological insights. Additionally, because self-report and behavioral measures have distinct and complimentary strengths and limitations (Enticott et al., 2006), clinicians and researchers may be able to obtain the most thorough assessment of adolescent DBD and capture subclinical presentations of DBD when a multimodal assessment approach is utilized. These findings add to the growing body of literature suggesting that this approach can provide a more accurate understanding of the processes underlying the construct of disinhibition (Denckla, 2002; Dougherty, Mathias, Marsh, Jagar, 2005; Toplak et al., 2008).

Another potential criticism is that the BRIEF-Inhibit scale and behavioral measures together only captured 34.5% of the variance, leaving 65.5% of the variance unexplained. Other possible predictors of disinhibition that may account for the rest of the variance include additional scales on the BRIEF, (e.g., the Shift scale, the Working Memory scale, and the Plan/Organize scale; Toplak et al., 2008), additional tasks from the D-KEFS (e.g., trail making, verbal fluency, and tower tests; Wodka et al., 2008a), and other disinhibition tasks such as the go/no go task (Schoemaker et al., 2013). Outside of measurement predictors, genetic risks also contribute to disinhibition, with heritability estimates of 59% (Young et al., 2009), which were not examined in the current study.

In addition to disinhibition, there are other risk factors that are associated with the development of DBD problems, but were not examined in the present study. Some of the most consistently replicated risk factors include harsh parenting, neighborhood deprivation, difficult temperament, and callous/unemotional traits (Barker & Maughan, 2009; Byrd, Loeber, & Pardini, 2009; Campbell, Shaw, & Gilliom, 2000; Frick & Morris, 2004; Loeber, Burke, & Pardini, 2009). There is also a substantial genetic influence, with genetic risks accounting for approximately 57% of the variance (Tuvblad et al., 2008).

These results should be considered in the context of several other limitations. First, there were a limited number of participants who endorsed DBD symptoms. While we did not have a sufficient number of participants to explore possible differences among ADHD, ODD and CD, the DBD construct is theoretically related and has been shown to share a common genetic liability (Hicks et al., 2004; Tuvblad et al., 2009; Young et al., 2006). We feel that the current analyses provide a proof of principle for the use of behavioral measures in assessing the construct of DBD, but future research should attempt to ascertain a larger sample to conduct a more nuanced analysis of each of the individual diagnoses. Another limitation is that DBD symptoms were assigned based on the K-SADS, which relies on the adolescent and parent to provide accurate information. However, the behavioral measures may be particularly valuable in cases where the participant’s responses may be biased because they are behavioral indicators of the underlying trait and therefore not subject to the social desirability bias found in self-reported information.

Conclusion

The present study supports the use of the BRIEF-Inhibit scale in conjunction with behavioral disinhibition measures, such as the anti-saccade task and color-word interference task. The set of these measures provided a more thorough assessment of adolescent psychopathology than any of the measures individually. Our results add to the literature informing the processes that underlie the emergence of DBD symptoms in adolescents.

Acknowledgments

This research was supported by Grant DA05605, R21AA017312; PA-HEAL SPH00010; U01AA021690.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Barker ED, Maughan B. Differentiating early-onset persistent versus childhood-limited conduct problem youth. American Journal of Psychiatry. 2009;166:900–908. doi: 10.1176/appi.ajp.2009.08121770. [DOI] [PubMed] [Google Scholar]
  2. Barkley RA, Murphy KR. The nature of executive function (EF) deficits in daily life activities in adults with ADHD and their relationship to performance on EF tests. Journal of Psychopathology and Behavioral Assessment. 2011;33(2):137–158. [Google Scholar]
  3. Berger A, Alyagon U, Hadaya H, Atzaba-Poria N, Auerbach JG. Response inhibition in preschoolers at familial risk for attention deficit hyperactivity disorder: A behavioral and electrophysiological stop-signal study. Child Development. 2013;84(5):1616–1632. doi: 10.1111/cdev.12072. [DOI] [PubMed] [Google Scholar]
  4. Bledsoe JC, Semrud-Clikeman M, Pliszka SR. Response inhibition and academic abilities in typically developing children with attention-deficit-hyperactivity disorder-combined subtype. Archives of Clinical Neuropsychology. 2010;25:671–679. doi: 10.1093/arclin/acq048. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Byrd AL, Loeber R, Pardini DA. Understanding desisting and persisting forms of delinquency: The unique contributions of disruptive behavior disorders and interpersonal callousness. Journal of Child Psychology and Psychiatry. 2012;53(4):371–380. doi: 10.1111/j.1469-7610.2011.02504.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Campbell SB, Shaw DS, Gilliom M. Early externalizing behavior problems: toddlers, preschoolers at risk for later maladjustment. Developmental Psychopathology. 2000;12:467–488. doi: 10.1017/s0954579400003114. [DOI] [PubMed] [Google Scholar]
  7. Carr L, Henderson J, Nigg JT. Cognitive control and attentional selection in adolescents with ADHD versus ADD. Journal of Clinical Child & Adolescent Psychology. 2010;39:726–740. doi: 10.1080/15374416.2010.517168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Clark DB, Chung T, Thatcher DL, Pajtek S, Long EC. Psychological dysregulation, white matter disorganization, and substance use disorders in adolescence. Addiction. 2012;107:206–214. doi: 10.1111/j.1360-0443.2011.03566.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Clark DB, Moss H, Kirisci L, Mezzich A, Miles R, Ott P. Psychopathology in preadolescent sons of fathers with substance abuse disorders. American Academy of Child & Adolescent Psychiatry. 1997;36:495–502. doi: 10.1097/00004583-199704000-00012. [DOI] [PubMed] [Google Scholar]
  10. Delis DC, Kaplan E, Kramer JH. Delis-Kaplan Executive Function System. San Antonio, TX: The Psychological Corporation; 2001a. [Google Scholar]
  11. Delis DC, Kaplan E, Kramer JH. Delis-Kaplan Executive Function System (D-KEFS) examiner’s manual. San Antonio, TX: The Psychological Corporation; 2001b. [Google Scholar]
  12. Denckla MB. The Behavior Rating Inventory of Executive Function: Commentary. Child Neuropsychology: A Journal on Normal and Abnormal Development in Childhood and Adolescence. 2002;8:304–306. doi: 10.1076/chin.8.4.304.13512. [DOI] [PubMed] [Google Scholar]
  13. Dolan M, Lennox M. Cool and hot executive function in conduct-disordered adolescents with and without co-morbid attention deficit hyperactivity disorder: Relationships with externalizing behaviours. Psychological Medicine. 2013;43:2427–2436. doi: 10.1017/S0033291712003078. [DOI] [PubMed] [Google Scholar]
  14. Dougherty DM, Bjork JM, Harper AR, Marsh DM, Moeller FG, Mathias CW, Swann AC. Behavioral impulsivity paradigms: A comparison in hospitalized adolescents with disruptive behavior disorders. The Journal of Child Psychology and Psychiatry. 2003;44:1145–1157. doi: 10.1111/1469-7610.00197. [DOI] [PubMed] [Google Scholar]
  15. Dougherty DD, Mathias CW, Marsh DM, Jagar AA. Laboratory behavioral measures of impulsivity. Behavior Research Methods. 2005;37:82–90. doi: 10.3758/bf03206401. [DOI] [PubMed] [Google Scholar]
  16. Enticott PG, Ogloff JRP, Bradshaw JL. Associations between laboratory measures of executive inhibitory control and self-reported impulsivity. Personality and Individual Differences. 2006;41:285–294. [Google Scholar]
  17. Falleti MG, Maruff P, Collie A, Darby DG. Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one-week, and one-month test-retest intervals. Journal of Clinical and Experimental Neuropsychology. 2006;28(7):1095–1112. doi: 10.1080/13803390500205718. [DOI] [PubMed] [Google Scholar]
  18. Frick PJ, Morris AS. Temperament and developmental pathways to conduct problems. Journal of Clinical Child and Adolescent Psychology. 2004;33:54–68. doi: 10.1207/S15374424JCCP3301_6. [DOI] [PubMed] [Google Scholar]
  19. Geier C, Luna B. The maturation of incentive processing and cognitive control. Pharmacology, Biochemistry and Behavior. 2009;93:212–221. doi: 10.1016/j.pbb.2009.01.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gorenstein EE, Newman JP. Disinhibitory psychopathology: A new perspective and a model for research. Psychological Review. 1980;87:301–315. [PubMed] [Google Scholar]
  21. Guy SC, Isquith PK, Gioia GA. Behavior Rating Inventory of Executive Function – Self Report version. Lutz, FL: Psychological Assessment Resources; 2004. [Google Scholar]
  22. Habeych ME, Folan MM, Luna B, Tarter RE. Impaired oculomotor response inhibition in children of alcoholics: The role of attention deficit hyperactivity disorder. Drug and Alcohol Dependence. 2006;82:11–17. doi: 10.1016/j.drugalcdep.2005.07.012. [DOI] [PubMed] [Google Scholar]
  23. Hallet PE. Primary and secondary saccades to goals defined by instructions. Vision Research. 1978;18:1279–1296. doi: 10.1016/0042-6989(78)90218-3. [DOI] [PubMed] [Google Scholar]
  24. Hanisch C, Radach R, Holtkamp K, Herpertz-Dahlmann B, Konrad K. Oculomotor inhibition in children with and without attention-deficit hyperactivity disorder (ADHD) Journal of Neural Transmission. 2005;113:671–684. doi: 10.1007/s00702-005-0344-y. [DOI] [PubMed] [Google Scholar]
  25. Heiman GW. Research Methods in Psychology. 3rd Edition. Boston & New York: Houghton Mifflin Company; 2002. [Google Scholar]
  26. Hicks BM, Krueger R, Iacono WG, McGue M, Patrick CJ. Family transmission and heritability of externalizing disorders. Archives of General Psychiatry. 2004;61:922–928. doi: 10.1001/archpsyc.61.9.922. [DOI] [PubMed] [Google Scholar]
  27. Holmes J, Gathercole SE, Place M, Alloway TP, Elliott JG, Hilton KA. The diagnostic utility of executive function assessments in the identification of ADHD in children. Child and Adolescent Mental Health. 2010;15:37–43. doi: 10.1111/j.1475-3588.2009.00536.x. [DOI] [PubMed] [Google Scholar]
  28. Isquith PK, Gioia GA. BRIEF predictions of ADHD: Clinical utility of the Behavior Rating Inventory of Executive Function for detecting ADHD subtypes in children. Archives of Clinical Neuropsychology. 2000;15:653–850. Abstract retrieved from Abstracts/Archives of Clinical Neuropsychology. [Google Scholar]
  29. Ivanov I, Schulz KP, London ED, Newcorn JH. Inhibitory control deficits in childhood and risk for substance use disorders: A review. The American Journal of Drug and Alcohol Abuse. 2008;34:239–258. doi: 10.1080/00952990802013334. [DOI] [PubMed] [Google Scholar]
  30. Karatekin C. Improving antisaccade performance in adolescents with attention-deficit/hyperactivity disorder (ADHD) Experimental Brain Research. 2006;174:324–341. doi: 10.1007/s00221-006-0467-x. [DOI] [PubMed] [Google Scholar]
  31. Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, Ryan M. Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): Initial reliability and validity data. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36:980–988. doi: 10.1097/00004583-199707000-00021. [DOI] [PubMed] [Google Scholar]
  32. Kosten TA, Rounsaville BJ. Sensitivity of psychiatric diagnoses based on the best estimate procedure. American Journal of Psychiatry. 1992;149:1225–1227. doi: 10.1176/ajp.149.9.1225. [DOI] [PubMed] [Google Scholar]
  33. Kraemer HC, Measelle JR, Ablow JC, Essex MJ, Boyce WT, Kupfer DJ. A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. American Journal of Psychiatry. 2003;160:1566–1577. doi: 10.1176/appi.ajp.160.9.1566. [DOI] [PubMed] [Google Scholar]
  34. Leckman JF, Sholomska D, Thompson WD, Belanger A, Weissman MM. Best estimate lifetime psychiatric diagnosis: A methodological study. Archives of General Psychiatry. 1982;39:879–883. doi: 10.1001/archpsyc.1982.04290080001001. [DOI] [PubMed] [Google Scholar]
  35. Loe IM, Feldman HM, Yasui E, Luna B. Oculomotor performance identifies cognitive deficits in attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 2009;48:431–440. doi: 10.1097/CHI.0b013e31819996da. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Loeber R, Burke JD, Pardini DA. Development and etiology of disruptive and delinquent behavior. Annual Review of Clinical Psychology. 2009;5:291–310. doi: 10.1146/annurev.clinpsy.032408.153631. [DOI] [PubMed] [Google Scholar]
  37. Logan GD, Schachar RJ, Tannock R. Impulsivity and inhibitory control. Psychological Science. 1997;8:60–64. [Google Scholar]
  38. Mahone EM, Cirino PT, Cutting LE, Cerrone PM, Hagelthorn KM, Hiemenz JR, Denckla MB. Validity of the behavior rating inventory of executive function in children with ADHD and/or Tourette syndrome. Archives of Clinical Neuropsychology. 2002;17(7):643–662. [PubMed] [Google Scholar]
  39. McAuley T, Chen S, Goos L, Schachar R, Crosbie J. Is the behavior rating inventory of executive function more strongly associated with measures of impairment or executive function? Journal of the International Neuropsychological Society. 2010;16(03):495–505. doi: 10.1017/S1355617710000093. [DOI] [PubMed] [Google Scholar]
  40. McCandless S, O’Laughlin L. The clinical utility of the Behavior Rating Inventory of Executive Function (BRIEF) in the diagnosis of ADHD. Journal of Attention Disorders. 2007;10(4):381–389. doi: 10.1177/1087054706292115. [DOI] [PubMed] [Google Scholar]
  41. Munoz DP, Everling S. Look away: the anti-saccade task and the voluntary control of eye movement. Nature Reviews Neuroscience. 2004;5(3):218–228. doi: 10.1038/nrn1345. [DOI] [PubMed] [Google Scholar]
  42. Nikolas MA, Nigg JT. Moderators of Neuropsychological Mechanism in Attention-Deficit Hyperactivity Disorder. Journal of Abnormal Child Psychology. 2014:1–11. doi: 10.1007/s10802-014-9904-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. O’Brien JW, Dowell LR, Mostofsky SH, Denckla MB, Mahone EM. Neuropsychological profile of executive function in girls with attention-deficit/hyperactivity disorder. Archives of Clinical Neuropsychology. 2010;25(7):656–670. doi: 10.1093/arclin/acq050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Reddy LA, Hale JB, Brodzinsky LK. Discriminant validity of the Behavior Rating Inventory of Executive Function Parent Form for children with attention-deficit/hyperactivity disorder. School Psychology Quarterly. 2011;26(1):45. [Google Scholar]
  45. Robins E, Guze SB. Establishment of diagnostic validity in psychiatric illness: Its application to schizophrenia. The American Journal of Psychiatry. 1970;126:983–987. doi: 10.1176/ajp.126.7.983. [DOI] [PubMed] [Google Scholar]
  46. Saarinen S, Fontell T, Vuontela V, Carlson S, Aronen ET. Visuospatial Working Memory in 7-to 12-Year-Old Children with Disruptive Behavior Disorders. Child Psychiatry & Human Development. 2014:1–10. doi: 10.1007/s10578-014-0449-3. [DOI] [PubMed] [Google Scholar]
  47. Schoemaker K, Mulder H, Deković M, Matthys W. Executive functions in preschool children with externalizing behavior problems: a meta-analysis. Journal of abnormal child psychology. 2013;41(3):457–471. doi: 10.1007/s10802-012-9684-x. [DOI] [PubMed] [Google Scholar]
  48. Stroop JR. Studies of interference in serial verbal reactions. Journal of Experimental Psychology. 1935;18:643–662. [Google Scholar]
  49. Tarter RE, Kirisci L, Mezzich A, Cornelius JR, Pajer K, Vanyukov M, Clark DB. Neurobehavioral disinhibition in childhood predicts early age at onset of substance use disorder. The American Journal of Psychiatry. 2003;160:1078–1085. doi: 10.1176/appi.ajp.160.6.1078. [DOI] [PubMed] [Google Scholar]
  50. Tillman C, Brocki KC, Sørensen L, Lundervold AJ. A longitudinal examination of the developmental executive function hierarchy in children with externalizing behavior problems. Journal of Attention Disorders. Advance online publication. 2013 doi: 10.1177/1087054713488439. [DOI] [PubMed] [Google Scholar]
  51. Toplak ME, Bucciarelli SM, Jain U, Tannock R. Executive functions: performance-based measures and the behavior rating inventory of executive function (BRIEF) in adolescents with attention deficit/hyperactivity disorder (ADHD) Child Neuropsychology. 2008;15(1):53–72. doi: 10.1080/09297040802070929. [DOI] [PubMed] [Google Scholar]
  52. Tuvblad C, Zheng M, Raine A, Baker LA. A common genetic factor explains the covariation among ADHD, ODD, and CD symptoms in 9–10 year old boys and girls. Journal of Abnormal Child Psychology. 2009;37:153–167. doi: 10.1007/s10802-008-9278-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Van der Stigchel S, Rommelse NNJ, Deijen JB, Geldof CJA, Witlox J, Oosterlaan J, Theeuwes J. Oculomotor capture in ADHD. Cognitive Neuropsychology. 2007;24(5):535–549. doi: 10.1080/02643290701523546. [DOI] [PubMed] [Google Scholar]
  54. Vriezen ER, Pigott SE. The relationship between parental report on the BRIEF and performance-based tasks of executive function in children with moderate to severe traumatic brain injury. Child Neuropsychology. 2002;8:296–303. doi: 10.1076/chin.8.4.296.13505. [DOI] [PubMed] [Google Scholar]
  55. Wodka EL, Loftis C, Mostofsky SH, Prahme C, Larson J, Denckla MB, Mahone EM. Prediction of ADHD in boys and girls using the D-KEFS. Archives of Clinical Neuropsychology. 2008a;23:283–293. doi: 10.1016/j.acn.2007.12.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Wodka EL, Mostofsky SH, Prahme C, Gidley Larson JC, Loftis C, Denckla MB, Mahone EM. Process examination of executive function in ADHD: Sex and subtype effects. The Clinical Neuropsychologist. 2008b;22(5):826–841. doi: 10.1080/13854040701563583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Young SE, Rhee SH, Stallings MC, Corley RP, Hewitt JK. Genetic and environmental vulnerabilities underlying adolescent substance use and problem use: general or specific? Behavior Genetics. 2006;36:603–615. doi: 10.1007/s10519-006-9066-7. [DOI] [PubMed] [Google Scholar]
  58. Young SE, Friedman NP, Miyake A, Willcutt EG, Corley RP, Haberstick BC, Hewitt JK. Behavioral disinhibition: Liability for externalizing spectrum disorders and its genetic and environmental relation to response inhibition across adolescence. Journal of Abnormal Psychology. 2009;118(1):117. doi: 10.1037/a0014657. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES