The conduct and oppositional defiant disorder scales (CODDS) for disruptive behaviour disorders

Abstract

This study evaluates the clinical validity of a five-minute instrument, the Conduct and Oppositional Defiant Disorder Scales (CODDS), for assessing oppositional defiant disorder (ODD) and conduct disorder (CD). Children (N = 428) aged 11–12 years and their caregiver were administered the NIMH DISC-IV (Diagnostic Interview Schedule for Children), the CODDS, and validity measures. A second sample (N = 671) was utilized to develop a brief measure of limited prosocial emotions based on DSM 5. Receiver operating characteristic (ROC) curves documented good sensitivity and specificity for CODDS scales in predicting DISC-IV clinical diagnoses of ODD (85%, 72% respectively) and CD (85%, 88%) diagnoses. Baseline CODDS provided added value over and above baseline clinical DISC- diagnoses in predicting future DISC ODD and CD diagnoses 12 months later, as well as in predicting social and school functioning. Study 2 further established psychometric properties of the CODDS, with brief measures of CODDS limited prosocial emotions (LPE) having a good fit to the hypothesized DSM 5 four-factor structure of LPE. Findings indicates that the CODDS has utility as a five-minute proxy for diagnoses of ODD and CD in clinical research and potentially practice where time and resources are limited.

1. Introduction

Conducting diagnostic assessments of conduct disorder (CD) and oppositional defiant disorder (ODD) using DSM clinical criteria can be challenging, entailing significant research time, resources, and clinical skills. One clinical assessment of these childhood conditions which in part addresses these challenges consists of the NIMH DISC-IV (National Institute of Mental Health Diagnostic Interview for Children - Shaffer et al., 2000). This clinical interview is largely (although not exclusively) structured and can be administered by non-clinicians, and has been used with a wide range of child clinical disorders (Dumas et al., press) and in countries throughout the world (Erskine et al., press). Impairments in six different domains of social, educational and other important areas of functioning are evaluated for presence, severity, and frequency (Shaffer et al., 2000). A particular strength of the DISC is that, compared to other well-established diagnostic instruments which include the Children’s Assessment Schedule (Hodges et al., 1982), the Diagnostic Interview for Children and Adolescents (Reich, 2000), and the Child and Adolescent Psychiatric Assessment (Angold and Costello, 2000), it is stringent in very closely following Diagnostic and Statistical Manual (DSM) criteria (Shaffer et al., 2000).

Despite these significant strengths, clinical diagnostic instruments like the DISC-IV are not without limitations. The CD/ODD modules of the DISC-IV have a total of 63 stem questions, and symptoms are further probed with a potential 240 further questions (167 for CD, 73 for ODD), involving a significant amount of time. An additional Introductory / Demographic module that includes a recall exercise is required to set up questions for certain symptoms and to provide a time-line to aid memory (Fisher et al., 2006). Interviewers need to be trained. The DISC manual warns that while two hand-scoring methods can be used, “Both are complex, time-consuming, and error-prone” (Fisher et al., 2006). While computer-scoring is available, it is somewhat complex and in itself requires 2–3 days of training (Shaffer et al., 2000). As such, a brief and easy-to-use indicator of ODD/CD diagnoses could have some utility both in research and clinical settings.

In this context, the present study aim to assess the capacity of a brief instrument – the Conduct and Oppositional Defiant Disorder Scales (CODDS) - to reliably and validly screen for diagnoses of ODD and CD in child/adolescent samples and to provide a brief measure of limited prosocial emotions (LPE). The overarching goal of the study is to establish construct validity for the instrument - criterion validity, predictive validity, convergent validity, discriminant validity, incremental validity, content validity, and face validity – alongside internal reliability, test-retest reliability, and stability over time. In addition to adequate internal and test-retest reliability, it was anticipated that these scales would provide reasonable convergence with clinical diagnostic indicators of CD and ODD based on the NIMH DISC-IV (Shaffer et al., 2000). Moderate stability over one year was expected, together with relatively robust relationships with measures of antisocial and aggressive behaviour (Achenbach and Rescorla, 2001; Buss and Warren, 2000; Frick et al., 2000; Raine et al., 2006). Negative associations were anticipated with school and social functioning (Achenbach and Rescorla, 2001). Finally, its utility in supplementing clinical diagnostic measures of ODD/CD to predict future ODD/CD was evaluated (American Psychiatric Association, 2013). These issues were examined with an in-person caregiver and child community sample (Study 1), while an on-line sample was used to provide further normative data, model a brief limited prosocial emotions specifier for conduct disorder based on the four dimensions of LPE outlined in DSM 5, and allow testing for replicability of findings (Study 2).

2. Methods

2.1. Participants

2.1.1. Study 1.

This is based on data collected in a prior study (Liu et al., 2013; Richmond et al., 2013). Briefly, 454 male and female community-dwelling children aged 11 to 12 years together with their caregiver were recruited into a baseline assessment. In addition, 291 of these 428 went on to participate in a treatment study (Raine et al., 2016) and were assessed at 3, 6, and 12 months following baseline (0 months). This reduced sample was utilized for predictive and incremental validity (see below). Ethnic composition was as follows: Caucasian (12.4%), African-American (79.9%), Latino (0.7%), Asian (0.5%), Native American (0.2%), Multiracial (5.1%), Other (0.9%), with 0.2% declining to answer. Ethical approval was obtained from the University of Pennsylvania and also the Philadelphia Department of Health.

2.1.2. Study 2.

Caregivers with a child aged 8–16 was recruited online using Amazon’s Mechanical Turk (MTurk). Several steps were taken to maximize data quality and enhance validity. First, respondents had to have been experienced survey-takers (completion of above 5000 HITS – Human Intelligence Tasks). Second, they had to have an approval rating of at least 95%. Third, participants were further screened using validity check items from the NEO Personality Inventory-Revised (NEO PI-R) (Costa and Mccrae, 1992). Fourth, participants were excluded if they started the survey but did not complete it, if they answered all questions in an impossibly short time period, and if they provided stereotyped responding. This resulted in 57 exclusions from an initial sample of 708 (8.1%), resulting in a final sample of 651 (mean age 40.98 years; 61.9% female; 74.8% White, 13.1% Asian; 6.1% Black, 3.4% Hispanic/Latino, 0.6% American-Indian). Caregivers reported on themselves and also their child (mean age 14.10 years, range 8–16 years; 51.3% female; 69.7% white, 13.7% Asian, 6.9% black, 5.7% Hispanic/Latino, 0.2% Native Hawaiian or Pacific Islander). Ethical approval was obtained from the IRB at the University of Pennsylvania.

2.2. Conduct and oppositional defiant disorder scales (CODDS)

A breakdown of all measures used in both studies, broken down by respondent, is given in Supplement Table S5. The CODDS is a 23-item instrument allowing the caregiver to report-on-child and the child to self-report, and is modelled on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM 5) (American Psychiatric Association, 2013), with each item mirroring each of the eight oppositional defiant disorder criteria and the 15 conduct disorder criteria. Full details of item creation, subscales scores, and interpretation of scores are given in the Supplement. Briefly, each item is assessed on a 3-point scale (never, sometimes, often), with items summed to yield conduct disorder and oppositional defiant disorder scores. In Studies 1 and 2 the caregiver reported on their child, while the children self-reported in Study 1. Caregiver report-on-child and child self-report versions of the CODDS are provided in the Appendix and are freely available for research purposes.

2.3. Limited prosocial emotions specifier (CODDS-LPE)

In Study 2 a CODDS-LPE instrument was developed given that DSM 5 contains a specifier for CD. The initial pool for this caregiver report-on-child instrument consisted of 26 items which were generated from description of the four domains of prosocial behaviors specified in DSM 5 (lacking remorse / guilt; callous / lack of empathy; unconcerned about performance; shallow / deficient affect). Items were written to match, as reasonably as possible, the descriptors in DSM 5. We aimed to assess each of the four domains using three items, resulting in a short but reliable 12-item instrument. The 26 items were reduced to a final set of 12 using a face validity approach, examination of skew and kurtosis, and the subsets option in the OMEGA macro (Hayes and Coutts, 2020) which for each of the four hypothesized subscales identifies the items which maximize internal reliability. The final set of 12 items was then subjected to confirmatory factor analysis using Mplus (Version 8.4). The scale is provided in the Appendix and is freely available for research purposes. Low scores indicate a lack of prosocial emotions.

2.4. NIMH diagnostic interview schedule for children (NIMH DISC-IV)

The NIMH DISC-IV is a structured, computer-based diagnostic instrument designed for non-clinicians to diagnose DSM-IV psychiatric disorders in children (Shaffer et al., 2000). Caregivers in Study 1 were interviewed about their child by research assistants trained on DISC administration to derive diagnoses of CD and ODD. While the DISC was modelled on DSM-IV criteria, it also assesses DSM 5 criteria which were unchanged from DSM-IV to DSM 5 for both ODD and CD. Good diagnostic reliability and scale reliability have been reported (Shaffer et al., 2000). Good internal reliability in the current study was found for CD (α = 0.82) and ODD (α = 0.84).

2.5. Antisocial / aggressive behaviour and other validity measures

2.5.1. Aggression Questionnaire (Study 1).

The self-report Buss-Perry Aggression Questionnaire (Buss and Warren, 2000) was administered to children, producing sub-scale scores for physical aggression (α = 0.73), verbal aggression (α = 0.68), indirect aggression (α = 0.64), anger (α = 0.61), and hostility (α = 0.70). Acceptable reliability and validity have been documented (Buss and Warren, 2000).

2.5.2. Reactive-Proactive Aggression Questionnaire (RPQ – Studies 1 and 2).

Only children completed the self-report RPQ (Raine et al., 2006) in Study 1, yielding scales of reactive (α = 0.81), proactive (α = 0.80), and total aggression (α = 0.87). Reliability and validity have been extensively documented (Baker et al., 2008; Fossati et al., 2009).

2.5.3. Antisocial Process Screening Device (APSD – Study 1).

The APSD (Frick et al., 2000), assesses caregiver report-on-child and child-reported psychopathic-like traits yielding three subscales assessing callous-unemotional traits (α = 0.65), narcissism (α = 0.79), and impulsivity (α = 0.68) in addition to a total score (α = 0.85). In Study 2, caregivers reported on themselves and their child on the callous-unemotional subscale only. The APSD has been extensively validated and has adequate reliability (Frick et al., 2000), with evidence for convergent and discriminant validity for self-reports in adults (Beier et al., 2014).

2.5.4. Child behaviour Checklist (CBCL) and Youth Self Report (YSR) (Study 1).

The CBCL (caregiver report-on-child) and the YSR (child self-report) are extensively-used psychometric instruments with high reliability/validity in many countries (Achenbach and Rescorla, 2001). They both yield subscale scores for rule-breaking (α = 0.96) and aggression (α = 0.95), from caregiver and child reports, as well as DSM-orientated scales for conduct disorder and oppositional defiant disorder. Reliability, validity, and cross-cultural generalizability have been documented in over 100 societies (Achenbach and Rescorla, 2001; Rescorla et al., 2012).

2.5.5. Inventory of Callous-Unemotional Traits (ICU-10 - Study 2).

Caregivers rated their child on the 10-item version of the ICU (Ray et al., 2016). This shorter measure has been found to have comparable reliability and validity compared to the original ICU (Frick et al., 2014), with α in this study of = 0.91.

2.6. Social and school functioning

This was assessed in Study 1 using the Social and School Competence scales of the CBCL (Achenbach and Rescorla, 2001). These caregiver report-on-child scales assess different aspects of social functioning (e.g. number of friends, engagement in social activities / clubs, quality of relationships with siblings, caregivers, and other children) and school functioning (performance in multiple academic subjects). Construction and validation of these scales are provided in Achenbach and Rescorla (2001). In Study 2, caregivers reported whether their child had been sent to the school principal or expelled.

2.7. Statistical analyses

A receiver operating characteristic curve (ROC) analysis was conducted to test the CODDS clinical ability to discriminate between those diagnosed with ODD and CD from those without these diagnoses. Values of 0.50 to 0.70 are taken as poor discrimination, 0.70 to 0.80 as acceptable, 0.80 to 0.90 as excellent, and > 0.90 as outstanding (Hosmer et al., 2013). Values of 0.80 and above have been taken as indicating adequate diagnostic discrimination to be used in making medical decisions on intervention (Hosmer and Lemeshow, 2005). Data are available for cross-checking findings upon reasonable request.

To assess the ability of CODDS and DISC diagnostic measures to predict diagnoses 12 months later, binary logistic regression was run using the Wald χ² test and the Nagelkerke R² statistic as an effect size. Multiple regression was used to analyse relationships with social and school functioning as dependant variables, and CODDS/DISC measures as independent variables using forced entry, with standardized beta coefficients. Linear regression was used to evaluate the relative contributions of the caregiver DISC-IV and CODDS scores on social and school functioning. Confirmatory factor analysis was conducted on CODDS limited prosocial emotions with Mplus (Version 8.4) using the WLSMV estimator for ordinal data (Muthén and Muthén, 2019). Fit was evaluated using the χ2 test, the Comparative Fit Index (CFI), the Tucker-Lewis Index (TLI), and the Root Mean Square Error of Approximation (RMSEA) (Hu and Bentler, 1998)). McDonald’s omega (ω) (McDonald, 1999) was used to calculate internal reliability.

3. Results

STUDY 1

3.1. Internal reliability and sub-scale intercorrelations for CODDS

Internal reliabilities (McDonald’s ω) for caregiver report-on-child CODDS scales were as follows: ODD (0.90), CD (0.82), total (0.91). Reliabilities for child self-reports were: ODD (0.83), CD (0.84), total (0.89). ODD and CD subscales were significantly correlated for caregiver-reports (r = 0.59, p < .001) and child reports (r = 0.65, p < .001). See also the online Supplement for Chronbach’s alpha.

3.2. Receiver operating characteristic (ROC) curve analyses: criterion validity of CODDS

CODDS scales were used as predictor test variables of DISC diagnoses in ROC analyses using diagnostic criteria (present / absent) for CD and ODD as state variables. For the caregiver report-on-child CD CODDS scale, the area under the curve was 0.90 (p < .0001, d = 1.81) indicating strong prediction (Fig. 1). A CODDS CD raw score of 4.5 results in maximal prediction, with sensitivity (true positive) of 84.8% and specificity (true negative) of 88.1%. For the caregiver ODD CODD scale, the area under the curve (Fig. 1) was .86 (p < .0001, d = 1.53), with a raw score of 8.5 having 85.3% sensitivity and 72.3% specificity.

Fig. 1.

Receiver-Operator Curve (ROC) Analyses showing sensitivity and specificity for CODDS (A) CD and (B) ODD scales predicting their respective DISC CD/ODD diagnoses from Sample 1.

3.3. Stability of CODDS

Correlations between 0, 3, 6, and 12-month reports of the CODDS scales (all significant, p < .0001) are given in Table S1. For caregiver report-on-child CODDS scales, one-year stability was relatively good for total scores (0.64), CD (0.61) and ODD (0.55). As expected, the average of 3-month (r = 0.71) and 6-month (r = 0.70) lagged correlations were higher. Somewhat lower but still moderate one-year stability was observed for child self-reports (r = 0.40), with higher 3-month (r = 0.58) and 6-month (r = 0.59) stability.

3.4. Differential relationships with CBCL DSM scales: discriminant validity of CODDS

Caregiver report-on-child CODDS scales were differentially related to DSM measures of ODD and CD derived from the CBCL. Controlling for CODDS CD, CODDS ODD was more associated with DSM-orientated ODD (r = 0.59) than with DSM-orientated CD (r = 0.29), z = 7.76, p < .00001. Similarly, controlling for CODDS ODD, CODDS CD was more associated with DSM-orientated CD (r = 0.69) than with DSM-orientated ODD (r = 0.30), z = 11.34, p < .00001.

Parallel findings were observed for child self-report CODDS scales in relation to ODD/CD DSM measures derived from the YSR. Controlling for CODDS CD, CODDS ODD was more associated with DSM-orientated ODD (r = 0.54) than with DSM-orientated CD (r = 0.29), z = 7.31, p < .0001. Similarly, controlling for CODDS ODD, CODDS CD was more associated with DSM-orientated CD (r = 0.47) than with DSM-orientated ODD (r = 0.11), z = 10.01, p < .0001.

3.5. Relationships with other measures of antisocial / aggressive behaviour: construct validity of the CODDS

Caregiver report-on-child CODDS scores correlated with other caregiver-reported antisocial measures (r = 0.57 to 0.80), documenting convergent validity (Table 1). Callous-unemotional (CU) traits was an exception where correlations were markedly smaller (mean r = 0.32) (Table 1). Similar findings were observed with child-report CODDS scores (r = 0.36 to 0.80 – see Table 2), again with the exception of CU traits (mean r = 0.17).

Table 1.

Correlations between caregiver report-on-child Conduct and Oppositional Defiant Disorder Scales (CODDS) and other antisocial / aggressive behaviour measures from Sample 1. CBCL = Child behaviour Checklist; APSD – Antisocial Process Screening Device.

		Caregiver-Report
		CODDS
	ODD	CD	Total
	N = 437	N = 435	N = 433
Oppositional defiant disorder (CBCL)	.725***	.601***	.752***
Conduct Disorder (CBCL)	.612***	.802***	.764***
Callous-Unemotional (APSD)	.264***	.373***	.345***
Impulsivity (APSD)	.631**	.568**	.674**
Narcissism (APSD)	.651**	.632**	.717**
Total APSD	.662**	.681**	.747**
Rule-Breaking (CBCL)	.589**	.721**	.715**
Aggression (CBCL)	.706**	.712**	.791**

^***p < .001, two-tailed.

^**p < .01, two-tailed.

Table 2.

Correlations between child-reported Conduct and Oppositional Defiant Disorder Scales (CODDS) and other antisocial / aggressive behaviour measures from Sample 1. ODD = Oppositional Defiant Disorder, CD = Conduct Disorder. RPQ = Reactive-Proactive Aggression Scale. CBCL = Child Bhevaior Checklist. AQ = Aggression Questionnaire. YSR = Youth Self-Report.

		Child Self-Report
		CODDS
Antisocial / Aggression Scales	ODD	CD	Total
	N = 439	N = 438	N = 437
Oppositional defiant disorder (YSR)	.69***	.50***	.67***
Conduct Disorder (CBCL)	.60***	.66***	.69***
Proactive Aggression (RPQ)	.62***	.80***	.77***
Reactive Aggression (RPQ)	.72**	.57***	.72***
Physical Aggression (AQ)	.54***	.51***	.58***
Verbal Aggression (AQ)	.57***	.43***	.56***
Anger (AQ)	.56***	.45***	.56***
Hostility (AQ)	.51***	.36***	.49***
Indirect Aggression (AQ)	.53***	.43***	.53***
Total Aggression (AQ)	.69***	.56***	.69***
Callous-Unemotional (APSD)	.14**	.19***	.18**
Narcissism (APSD)	.61***	.58***	.66***
Impulsivity (APSD)	.64***	.50***	.63***
Total (APSD)	.67***	.61***	.70***
Rule-Breaking (YSR)	.54***	.59***	.62***
Aggression (YSR)	.71***	.61***	.73***

^**p < .01, two-tailed

^***p < .001, two-tailed.

Regarding discriminant validity, a significant distinction between ODD and CD scores was observed for reactive and proactive aggression (see Table 2). For child self-reports, proactive aggression was significantly more associated with CD than ODD (p < .0001), while reactive aggression was more associated with ODD (p < .0001).

3.6. Baseline CODDS and one-year clinical diagnoses of ODD and CD: predictive validity

A DISC diagnosis of ODD at 12 months was predicted by both baseline DISC ODD diagnosis (Wald χ² = 43.43, R² = 0.31, p < .0001) and also baseline caregiver report-on-child CODDS ODD scores (Wald χ² = 30.96, R² = 0.29, p < .0001). CODDS ODD scores at baseline also predicted ODD diagnosis at 12 months over and above baseline DISC ODD diagnosis, (Wald χ² = 11.83, p < .001), raising variance explained from 31% to 38%.

A DISC diagnosis of CD at 12 months was predicted by both baseline DISC CD diagnosis (Wald χ² = 13.89, R² = 0.15, p < .0001) and baseline caregiver report-on-child CODDS CD scores (Wald χ² = 23.10, R² = 0.34, p < .0001). CODDS ODD scores at baseline also predicted ODD diagnosis at 12 months over and above baseline DISC ODD diagnosis, (Wald χ² = 13.71, p < .001), raising variance explained from 15% to 32%. The reverse was not the case; DISC CD diagnosis at baseline did not significantly add to prediction of 12-month CD diagnosis after accounting for baseline the CODDS CD measure (Wald χ² = 00.39, p = .53).

3.7. Relationships between CODDS and social / school functioning

3.7.1. Baseline.

Caregiver report-on-child CODDS ODD score was a significant predictor of social functioning (beta = −0.25, t = −3.91, p < .0001), but the DISC was not (beta = 0.05, t = 0.82, p = .41). Similarly, significant effects were observed for the CODDS CD (beta = −0.20, t = −2.98, p = .003) but not the DISC CD (beta = 0.01, t = 0.16, p = .88).

Regarding school functioning, CODDS ODD scores yielded significant effects (beta = −0.24, t = −3.74, p = .0001) but DISC ODD did not (beta = −0.03, t = −0.49, p = .63). Similarly, significant effects were observed for the CODDS CD (beta = −0.33, t = −4.91, p = .0001) but not DISC CD (beta = 0.01, t = 0.21, p = .84).

3.7.2. 12 Months Outcome.

Regarding social functioning, for caregiver report-on-child ODD scores, significant effects were observed for the CODDS (beta = −0.16, t = −2.10, p = .037) but not the DISC (beta = 0.10, t = 1.34, p = .18). For CD, effects were non-significant for both the CODDS (beta = −0.10, t = −1.23, p = .21) and the DISC (beta = −0.08, t = −0.97, p = .33).

For school functioning, for ODD scores, significant effects were observed for the CODDS (beta = −0.19, t = −2.61, p = .01) but not the DISC (beta = −0.08, t = −1.34, p = .28). For CD, significant effects were observed for the CODDS (beta = −0.24, t = −3.15, p = .002) but not the DISC (beta = −0.08, t = −1.10, p = .31).

STUDY 2

Study 2 was conducted during Covid-19 using an on-line sample and aimed at both providing further validation data for CODDS and also for modelling a brief prosocial specifier for conduct disorder (CODDS-LPE) for the CODDS. Caregivers reported on both their child and themselves, with the focus below being on findings on caregivers’ report-on-child. All means and SDs are given in Table S2.

3.8. CODDS

3.8.1. Internal Reliability and Sub-Scale Inter-Correlations.

Internal reliabilities (ω) and inter-correlations for the MTurk sample closely correspond to those obtained in the in-person sample. Internal reliabilities for caregiver report-on-child CODDS scales were as follows: ODD (0.85), CD (0.86), total (0.87). Reliabilities for caregiver self-reports were very similar as follows: ODD (0.83), CD (0.81), total (0.79). ODD and CD subscales were significantly correlated for caregivers’ report-on-child (r = 0.46, p < .001).

3.8.2. Construct Validity Measures.

All associations were significant (p < .001) and in the predicted direction (see Table S3). As in Study 1, there was a double-dissociation for aggression subtype, with reactive aggression being more strongly related to ODD than CD (p <.001), while proactive aggression was more strongly related to CD than ODD (p < .001). Both ODD and CD were associated with having school behaviour problems (p < .001), being sent to the school principal (p < .001), and being expelled from school (p < .013) – see Table S4.

3.9. Limited prosocial emotions (CODDS-LPE)

3.9.1. Initial Item Selection, Internal Reliability, and Sub-Scale Inter-correlations.

Items were initially grouped under their respective putative factor. For each factor, the three items which produced the most reliable subscale were selected using an OMEGA macro (Hayes and Coutts, 2020), producing a final scale compromised of 12 items. Internal reliabilities (ω) for the four subscales and total scores for caregivers reporting-on-child are as follows: factor 1 (0.80), factor 2 (0.79), factor 3 (0.85), factor 4 (0.77), total score (0.92). Subscales significantly intercorrelated (p < .001) between 0.59 and 0.72. This putative selection was then subjected to confirmatory factor analysis for caregiver report-on-child.

3.9.2. Factor Structure.

The second-order model specifies one higher second-order factor (total score for limited prosocial emotions) and four subordinate first-order factors (Fig. 2). Fit indices were as follows: χ² = 193.44, p < .001; RMSEA = 0.066 (CI = 0.057–0.076); CFI = 0.97; TLI = 0.96; SRMR = 0.031; indicating a good fit.

Fig. 2.

Factor and item loadings from the second-order hierarchical confirmatory factor analysis of caregiver report-on-child Limited Prosocial Emotions scale (CODDS-LPE) from Sample 2. Three individual items load on each of the four DSM lower-order factors which load on one higher-order general factor of limited prosocial emotions. Boxes refer to item numbers (see Appendix).

Fit indices for the one-factor model for children were as follows: χ² = 464.11, p < .001; RMSEA = 0.11 (CI = 0.099–0.117); CFI = 0.87; TLI = 0.85; SRMR = 0.053; indicating a poorer fit compared to the second-order model. The second-order model produced a significant better fit than a one-factor model (Δχ² = 270.67, df = l, p < .001).

3.9.3. Construct Validity.

Regarding convergent validity, total CODDS-LPE was significantly associated with ICU-10 callous-unemotional traits (r = 0.71, p < .001). Associations between the CODDS-LPE scales and other antisocial behaviour measures (aggression, callous-unemotional traits, ODD, CD, school behaviour problems) are shown in Table 3. All correlations were significant (p < .001), and in the expected direction, ranging from 0.29 to 0.71. Analyses on dichotomous variables indicated that being sent to the school principal (t = 6.49, p < .001), and being expelled from school (t = 2.87, p < .004) were both associated with higher LPE scores (see Table S5).

Table 3.

Construct validity for the Limited Prosocial Emotions Scale from Sample 2 (N = 651). CODDS = Conduct and Opposiitonal Disorder scales; RPQ = Reactive-Proactive Aggression Questionnaire; ICU = Inventory for Callous-Unemotional Traits; LPE= Limited Prosocial Emotions.

Caregiver Report-on-Child	Factor 1 Guilt-Remorse	Factor 2 Callous-Lacks Empathy	Factor 3 Performance	Factor 4 Shallow Affect	Total LPE
ODD (CODDS)	.45***	.44***	.36***	.40***	.48***
CD (CODDS)	.35***	.32***	.30***	.32***	.37***
Total (CODDS)	.47***	.45***	.39***	.42***	.50***
Trouble in School	.32***	.29***	.36***	.29***	.37***
Proactive Aggression (RPQ)	.35***	.31***	.28***	.31***	.36***
Reactive Aggression (RPQ)	.36***	.36***	.32***	.32***	.40***
Total Aggression (RPQ)	.40***	.38***	.34***	.35***	.43***
Callous-Unemotional (ICU-10)	.65***	.62***	.62***	.56***	.71***

^**p < .01, two-tailed.

^***p < .001, two-tailed.

4. Discussion

This investigation assessed whether a brief and easy-to-use instrument – the Conduct and Oppositional Defiant Disorder Scales (CODDS) – could reliably and validly screen for DSM diagnoses of ODD and CD. In Study 1, ROC analyses documented good sensitivity and specificity in predicting ODD and CD clinical diagnoses made with the DISC-IV interview, importantly documenting criterion validity. ROC analyses also showed that CODDS predicted to clinical diagnoses 12 months later at a level equal to or better than the DISC-IV, establishing predictive validity. Good internal reliability was established alongside three-month test-retest reliability and one-year stability. Discriminant validity was established, while expected associations with other measures of antisocial behaviour documented convergent validity. Mapping items directly to DSM 5 criteria yielded content and face validity. Study 2 supported the construct validity of the instrument and provided initial validation of a measure of limited prosocial emotions, with factor analytic support for the four dimensions of LPE proposed in DSM 5. Findings support the validity of a brief measure of DSM ODD and CD, and suggest potential utility in community and clinical research settings.

Probably the most salient finding concerns the validity of CODDS in predicting clinic interview-based measures of DSM diagnoses of ODD and CD using the DISC. At baseline for CD, sensitivity and specificity were 84.8% and 88.1% respectively, with figures of 85.3% and 72.3% for ODD, indicating relatively high true positives and low false negatives. Area-under-the-curve (AOC) values were 0.90 (CD) and 0.86 (ODD), values which convert to effects sizes (Cohen’s d) of 1.28 and 1.08 respectively. Bearing in mind that AOC values approaching 0.80 are taken in medical research to reach criterion for decision-making on interventions (Hosmer and Lemeshow, 2005), these initial findings indicate the utility of CODDS as proxy measures for clinical ODD and CD diagnoses. Furthermore, both predictive and incremental validity was documented for CODDS in predicting to diagnoses 12 months later. CODDS at baseline also predicted variance in one-year CD and ODD diagnoses over and above DISC-IV baseline diagnoses, whereas the reverse was not the case. Similar incremental validity was established for social and school functioning, with the CODDS outperforming the DISC-IV. Importantly, this level of convergence with diagnostic measures with high AOC values and superior prediction to social/educational functioning has not been documented in other community measures of externalizing behaviour. Overall, findings provide relatively strong clinical validation of CODDS and also utility in predicting future DSM diagnoses.

Validity for CODDS documented in Study 1 was supported by findings from a demographically different community sample (Study 2) where construct validity and internal reliability was confirmed. This sample also provided initial validity for CODDS Limited Prosocial Emotions scales, a specifier for CD. Confirmatory factor analysis provided to our knowledge the first confirmation of the four sub-components of LPE outlined in DSM 5, with a good fit observed. Another substantive finding was that proactive aggression was more related to CD than ODD, while reactive aggression was more associated with ODD than CD. To our knowledge, this double dissociation has not been previously documented. Proactive aggression is viewed as an instrumental, planned, and goal-directed form of aggression (Scarpa et al., 2010), a view consistent with components of DSM CD involving using aggression for gain, forced sexual activity, theft, and lying to obtain goods. In contrast, reactive aggression is viewed as a more impulsive and emotion-laden form of aggression (Scarpa et al., 2010) which may be more consistent with components of ODD dealing with angry/irritable mood. Importantly, this double-dissociation observed in Study 1 was replicated in Study 2, and as such appears robust. These findings both support the validity of the distinction between reactive and proactive aggression, and also document discriminant validity for the CODDS ODD/CD scales.

Limitations should be recognized. First, findings in both studies are based on community participants and further work is required to document validity within clinical samples. Second, while there is an 8-year age-range in Study 2, Study 1 participants constituted a much narrower age-range, and future studies with wider age-ranges would be advantageous. Third, while we provide clinical cut-offs to be used for proxy diagnoses of ODD and CD using CODDS, these need to be re- evaluated in other populations. Fourth, while the CODDS had high AOC values, predicted to future clinical diagnoses of CD and ODD, and outperformed the DISC-IV in predicting social and school functioning, cut-off scores outlined here cannot be taken as equivalent to a clinical diagnosis. Fifth, findings from Study 2 were collected using MTurk due to Covid-19 shutdown. There is significant evidence that MTurk data is better in quality than data collected in the laboratory, but it comes with significant limitations if validity indicators are not employed (Chmielewski et al., 2020). Although we employed these validity indicators, further replication and extension of Study 2 findings to laboratory populations would be desirable.

In conclusion, this study documents initial evidence for the validity of the CODDS, providing a simple and brief measure of DSM diagnoses of CD and ODD in community samples, with evidence for convergent, discriminant, criterion, incremental, content, predictive, face, and construct validity alongside internal reliability, test-retest reliability, and stability. Together with the associated CODDS-LPE scales, this brief instrument could have utility as an easy-to-use screen for diagnoses of ODD and CD in both cross-sectional and longitudinal research where time and resources are limited, and potentially also in clinical practice.

Appendix

CONDUCT AND OPPOSITIONAL DEFIANT DISORDER SCALES (CODDS): Parent report-on-child

Instructions to Parent

There are times when nearly all our children have done things they should not have done. Answer the following questions about your child by putting a circle around either 0 (never), 1 (sometimes: once), or 2 (often: two or more times). Make sure you answer all the items.

	NEVER	SOMETIMES	OFTEN
How often in the past year has your child....
1. lost their temper	0	1	2
2. argued with adults	0	1	2
3. refused to follow requests or rules	0	1	2
4. deliberately annoyed people	0	1	2
5. blamed others for their own mistakes or bad behavior	0	1	2
6. been touchy or easily annoyed	0	1	2
7. been angry or resentful	0	1	2
8. been spiteful or mean	0	1	2
9. How often have any of things above caused problems at home or at school, with people or doing school work?	0	1	2
10. bullied or threatened someone	0	1	2
11. started a physical fight	0	1	2
12. used a weapon to harm someone	0	1	2
13. been physically cruel to someone	0	1	2
14. been physically cruel to an animal	0	1	2
15. stolen or grabbed things from someone	0	1	2
16. forced someone into a sexual activity	0	1	2
17. started a fire to damage things	0	1	2
18. destroyed people’s things	0	1	2
19. broken into a house, building, or car	0	1	2
20. lied to get things or favors, or avoid doing something	0	1	2
21. stolen things or shoplifted	0	1	2
22. stayed out at night without permission	0	1	2
23. run away from home for a while	0	1	2
24. stayed off school without permission	0	1	2
25. How often have any of these things caused problems at home or school, with people or school work.	0	1	2
26. Did your child start doing any of these things (items 10–24) before age 10?		NO	YES

LIMITED PROSOCIAL EMOTIONS: Parent report-on-child

Please answer how true or not true the following statements are about your child by putting a circle around either 0 (true), 1 (partly true), or 2 (not true). Make sure you answer all the items.

My child:	TRUE	PARTLY TRUE	NOT TRUE
1. is concerned about what happens if they break rules	0	1	2
2. is thoughtful to other people	0	1	2
3. shares their feelings with others	0	1	2
4. is concerned about performing poorly in school and in other activities	0	1	2
5. expresses emotions that are consistent with their actions	0	1	2
6. is regretful when they have broken a rule	0	1	2
7. makes a lot of effort to perform well	0	1	2
8. is concerned about the feelings of others	0	1	2
9. is genuine and sincere in their feelings for others	0	1	2
10. tries hard to do well at school	0	1	2
11. is less concerned about themselves and more concerned about others	0	1	2
12. apologizes after hurting someone	0	1	2

CONDUCT AND OPPOSITIONAL DEFIANT DISORDER SCALES (CODDS): child self-report

Instructions to Child/Adolescent

There are times when nearly all children and adolescents have done things they should not have done. Answer the following questions by putting a circle around either 0 (never), 1 (sometimes: once), or 2 (often: two or more times). Make sure you answer all the items.

	NEVER	SOMETIMES	OFTEN
How often in the past year have you ....
1. lost your temper	0	1	2
2. argued with adults	0	1	2
3. refused to follow requests or rules	0	1	2
4. deliberately annoyed people	0	1	2
5. blamed others for your own mistakes or bad behavior	0	1	2
6. been touchy or easily annoyed	0	1	2
7. been angry or resentful	0	1	2
8. been spiteful or mean	0	1	2
9. How often have any of things above caused problems at home or at school, with people or doing school work?	0	1	2
10. bullied or threatened someone	0	1	2
11. started a physical fight	0	1	2
12. used a weapon to harm someone	0	1	2
13. been physically cruel to someone	0	1	2
14. been physically cruel to an animal	0	1	2
15. stolen or grabbed things from someone	0	1	2
16. forced someone into a sexual activity	0	1	2
17. started a fire to damage things	0	1	2
18. destroyed people’s things	0	1	2
19. broken into a house, building, or car	0	1	2
20. lied to get things or favors, or avoid doing something	0	1	2
21. stolen things or shoplifted	0	1	2
22. stayed out at night without permission	0	1	2
23. run away from home for a while	0	1	2
24. stayed off school without permission	0	1	2
25. How often have any of these things caused problems at home or school, with people or school work.	0	1	2
26. Did you start doing any of these things (items 10–24) before age 10?		NO	YES

LIMITED PROSOCIAL EMOTIONS (CODDS-LPE): child self-report

Please answer how true or not true the following statements are about yourself by putting a circle around either 0 (true), 1 (partly true), or 2 (not true). Make sure you answer all the items.

	TRUE	PARTLY TRUE	NOT TRUE
1. I am concerned about what happens if I break rules	0	1	2
2. I’m thoughtful to other people	0	1	2
3. I share my feelings with others	0	1	2
4. I’m concerned about performing poorly at school and in other activities	0	1	2
5. I express emotions that are consistent with my actions	0	1	2
6. I’m regretful when I have broken a rule	0	1	2
7. I make a lot of effort to perform well	0	1	2
8. I’m concerned about the feelings of others	0	1	2
9. I’m genuine and sincere in my feelings for others	0	1	2
10. I try hard to do well at school	0	1	2
11. I am less concerned about myself and more concerned about others	0	1	2
12. I apologize after hurting someone	0	1	2

Scoring Instructions

CD and ODD Scales (CODDS)

Scores (0, 1 or 2) for oppositional defiant disorder (ODD - items 1 to 8) and conduct disorder (CD – items 10 to 24) are summated to create total scores for ODD and CD. ODD and CD total scores are added to create a total antisocial behaviour score. This scoring applies to both child and adult versions. High scores indicate high CD / ODD.

Limited Prosocial Emotions (CODDS-LPE)

Sum the following items to create factor scores and a total score:

Factor 1 (remorse/guilt): 1, 6, 12

Factor 2 (callous/lacks empathy): 8, 11, 2

Factor 3 (concern about performance): 4, 10, 7

Factor 4 (shallow/deficient affect): 3, 5, 9

Total (Limited Prosocial Emotions): Sum all 12 items

Low scores indicate reduced prosocial emotions.