A brief validated screen to identify boys and girls at risk for early marijuana use

Abstract

To guide recruitment, the ABCD Study requires a method for identifying children at high risk for early-onset substance use that may be utilized during the recruitment process. This study was undertaken to inform the development of a brief screen for identifying youths’ risk of early-onset substance use and other adverse outcomes. To be acceptable by participants in this context, consideration of potential items was limited to child characteristics previously determined to be potentially pertinent and parental cigarette smoking. To focus the analyses on a single target substance use outcome pertinent to the stated goals of the ABCD Study, early-onset marijuana use was selected. Utilizing data collected prior to the initiation of the ABCD Study, four longitudinal data sets were used in nine secondary data analyses to test, replicate and validate a brief screening assessment for boys and girls to identify those at risk for early-onset marijuana use by ages 14–15. The combination of child externalizing problems reported by the parent (4 items: destroys things belonging to his/her family or others; disobedience at school; lying or cheating; steals outside the home) and parent smoking (1 item) proved to be the optimal screen. This was largely replicated across the four data sets. Indicators of predictive efficiency were modest in magnitude and statistically significant in 8 out of the 9 analyses. The results informed the screen’s optimal threshold for identifying children at risk for early-onset marijuana use. The addition of child internalizing problems did not improve these predictions. Further analyses showed the predictive utility of the screen for several other substance use outcomes at ages 15 to 18, including alcohol and nicotine use. The results support the use of a short screening assessment to identify youth at risk for early-onset substance use in the ABCD Study and other research.

1. Introduction

To guide recruitment, the Adolescent Brain Cognitive Development (ABCD) Study required a method for identifying children at high risk for early-onset substance use that may be utilized during the recruitment process (Garavan et al., 2018this issue). In this context, childhood risk refers to characteristics identified at ages 9 or 10 years that predict adverse outcomes in adolescence, and “high risk” refers to a categorical classification of some children as having increased risk compared to others. The construction of a brief measure for childhood substance use risk involves the identification of characteristics that predict early-onset substance use in mid to late adolescence. The identification and evaluation of optimal items for a brief childhood measure to serve as a high-risk screener ideally involves data from several large prospective studies with assessments initiated prior to the typical age of onset of substance use. To inform ABCD Study recruitment, secondary analyses are needed with datasets collected prior to ABCD Study initiation. In this context, a set of analyses with available data focused on a specific substance use outcome was determined to be most likely to be informative and feasible.

While other substance use outcomes are also important, early-onset marijuana use is a relevant target (Malmberg et al., 2012; Volkow et al., 2018this issue). Marijuana is the most commonly used illicit drug by adolescents, and regular marijuana use identifies youth likely to develop cannabis use disorder (Chung et al., 2003). In these secondary data analyses, the definition of early-onset marijuana use was defined by the initiation of regular use as indicated in the available datasets (Table 1). The studies contributing datasets were the Center for Education and Drug Abuse Research (CEDAR), the Pittsburgh Youth Study (PYS), the Pittsburgh Girls Study (PGS), and the Michigan Longitudinal Study (MLS). In the studies contributing data to the secondary analyses described here, the definitions of regular marijuana use differed by sample due to measurement variations. The variations in the definitions of regular marijuana use were as follows: (a} a pattern of monthly use (CEDAR: Clark et al., 2005); (b) five or more use occasions in the past year (PYS, PGS: Loeber et al., 2016) and; (3) six or more occasions in the past year (MLS: Zucker et al., 2000). By efficiently identifying children at high risk for early-onset marijuana use (by ages 14–15), a brief and effective measure of childhood risk measure could be utilized as a screen to identify high risk children in prevention research, primary medical care, and mental health clinic settings. The present analyses were specifically undertaken to inform the development a childhood high risk screen for use in the ABCD Study (www.abcdstudy.org; Volkow et al., 2018 this issue). The ABCD Study is the National Institute of Healths’ large-scale prospective population study of the biological and environmental factors that influence young people’s ability to successfully navigate adolescence. The study has a special emphasis on the risk and protective factors that influence marijuana and other substance use, and subsequent health problems including substance use disorders.

Table 1.

Average age at assessments and prevalence of regular marijuana use at age 14–16.

Study	Average Age at Time 1	Average Age at Follow-up	Outcome	Prevalence (%)
CEDAR (boys)	11.4	14.7	Monthly marijuana use, current	16.2
CEDAR (girls)	11.6	14.8	Monthly marijuana use, current	10.9
PYS (boys)	10.2	16.0	Used marijuana ≥ 5x past year	15.0
PGS (girls)	9.6	14.5	Used marijuana ≥ 5x past year	6.2
MLS (boys)	10.6	14.7	Marijuana use ≥ 6x in the past year	7.7
MLS (girls)	10.6	14.6	Marijuana use ≥ 6x in the past year	5.5

Utilizing data from previously conducted studies, the present study was thus undertaken to develop and establish the efficiency of a short measure (i.e., screen) to identify youth at high risk for early-onset marijuana use with optimal features for use in the ABCD Study (Garavan et al., 2018, this issue). To achieve this goal, the risk level of a potential participant needs to be determined at the time of recruitment and prior to their scheduling for the extensive ABCD Study assessment protocol. Consequently, the optimal ABCD Study high risk screen has several characteristics: (1) extreme brevity, including less than ten items; (2) lack of sensitive items that may raise confidentiality concerns at this early stage of considering participation; (3) consistency with prior research. These characteristics were taken into consideration in the analyses that follow.

Historically, studies focusing on mental disorders such as schizophrenia, alcohol use disorder, and major depressive disorder, have used positive family history as a risk marker (Clark and Cornelius, 2004; Goodwin et al., 1973, Weissman et al., 1987). Family history has been demonstrated to identify children at high risk of later substance use disorders in many prospective studies (Tarter et al., 1999; Chassin et al., 1991; Clark and Winters, 2002). However, a detailed family history may involve the parent(s) being asked to disclose their own socially undesirable, embarrassing or, in some cases, illegal behavior. There have been alternative strategies to acquire this information, such as the use of publicly available records of drunk driving or other drug offenses, or the use of hospital records to identify parental diagnosis (Kelly et al., 2001). Obtaining such records would not be feasible in the initial recruitment phase of the ABCD Study. Regardless of the method for obtaining this information, requesting this information at the point of introducing the ABCD Study raises the real possibility that the parent(s) will decline study involvement.

Few longitudinal studies have formulated and tested measures for identifying high risk children likely to exhibit early-onset marijuana use. There have been several approaches developed for predicting substance use disorders, but relatively few have targeted the adolescent developmental period. One of the risk measures developed to identify high risk children is the SUD Transmissible Liability Index (TLI) developed by Vanyukov, Tarter, Clark and colleagues (e.g., Vanyukov et al., 2003a,b; 2009), using longitudinal data from the CEDAR study. Although the TLI is sophisticated in its development, it is long (42 items), uses different portions of existing instruments, and is under copyright. In addition, the TLI did not focus on the age 15 outcome of marijuana use, and the publications did not use Receiver Operating Characteristic ((Hanley and McNeil, 1982, 1983) Area Under the Curve (AUC) analyses to determine an optimal threshold score. Another screening instrument, the DSM Guided Cannabis Screen (Alexander and Leung, 2011) has unknown predictive value because it was constructed using cross-sectional data from a small clinical sample aged 14–59. Therefore, the current study fills a significant gap in the empirical literature.

This report describes the process and results of secondary data analyses to prospectively identify a brief screening measure applicable to age 9–10-year-old children that would predict early-onset marijuana use in the 5–7 years following the initial screening measurement. To acquire data useful for developing this screening measure, we needed to identify population-based prospective studies which (a) began assessments in late childhood, (b) had been continued at least through ages 14–17, (c) included marijuana use variables at both age periods, (d) measured domains previously identified in the literature as predictive of adolescent substance use disorder outcomes, and (e) had a sufficient number of measures in these domains that were shared across these studies so that screening validation could be replicated across different demographic groups (e.g., males and females).

1.1. Present study objectives

The objectives of these secondary data analyses were as follows: (1) To develop a brief screener for 9–10-year-old boys and girls to predict early-onset marijuana and other substance use in mid adolescence with demonstrated predictive utility across four longitudinal data sets; (2)To dichotomize the outcome variable, which will reduce shrinkage (lower computational values upon replication), improve replicability and practical utility.; (3) To replicate findings across construction and validation samples (explained below). The advantage of this dual analysis approach is that we could construct a screener that considers shrinkage (meaning lower predictive efficiency when the screen is applied to a new, validation sample) that typically happens between construction of a screener and subsequent validation in another sample. In summary, the objective was to develop a brief and feasible approach to the identification of children at increased risk (i.e., high risk) for early-onset (i.e., adolescent) marijuana use that may inform the ABCD Study recruitment procedures.

1.2. Study populations and methods

To ascertain replication of results, we used four existing longitudinal data sets (i.e., data collected in other projects prior to the initiation of the ABCD Study). These data sets were utilized to build construction and validation samples for each sex (when N’s allowed), resulting in nine independent analyses. The four longitudinal data sets were from the CEDAR, PYS, PGS, and MLS (Table 1 for average ages at Time 1 and at follow-up). Where possible, we used both parent and child as informants, which is particularly important for externalizing behavior that is concealing in nature (e.g., theft), because parents usually have less knowledge of the behavior compared to the child.

1.3. Center for Education and Drug Abuse Research (CEDAR) data set

(Vanyukov et al., 2003a,b; 2009). The sample consisted of 506 boys and 202 girls with follow-up data. At the initial assessment (i.e., Time 1 or T1), 77% of the boys were White, and 23% Non-White (including African American, Biracial, and Hispanic), and 71% of the girls were White, and 29% Non-White (including African American, Biracial, and Hispanic). Sample selection: The sample was partially a high-risk sample, in that 44% of the youth had a father with a substance use disorder involving an illicit drug, who were recruited from addiction treatment programs, social service agencies, newspaper and radio advertisements, public service announcements, and random digit telephone calls. In the sample used here, participants’ ages at T1 ranged between 9.4–13.4 (mean age was 11.4 years; s.d. = 0.9). Only parent reports were available for the screener candidate variables. The CEDAR data set also contained auxiliary outcomes of ‘ever used marijuana’, and ‘DSM-IV diagnosis of cannabis abuse or dependence,’ all by age 15. In addition, the study had the following alcohol-related outcomes: ‘ever consumed a full standard drink’, ‘monthly alcohol use’, and ‘DSM-IV diagnosis of alcohol abuse or dependence’, all by age 15. Attrition was 8.5%.

1.4. Pittsburgh Youth Study (PYS) dataset

(Loeber et al.,1998). The focus was on the youngest cohort in this data set. The sample consisted of 452 boys who at T1 for the current analysis were ages 9–10 years old. At the T1 assessment, the cohort consisted of 41% White, 58% African American and 1% other race. Sample selection: The sample was recruited from public schools in Pittsburgh. With one exception, all public schools participated. Parent and son were contacted in the family home and 85% of the contacted families agreed to participate. The outcome of interest was marijuana use at 5 or more times per year around age 14. Compared to participation at T1, attrition was 10%.

1.5. Pittsburgh Girls Study (PGS) dataset

(Hipwell et al., 2002; Loeber et al., 2016). All 4 age cohorts (ages 5, 6, 7 and 8) were used for the analyses, consisting of 2177 girls. At the initial assessment, the cohorts consisted of 41% White, 53% African American, and 6% other race. Sample selection: A community sample was drawn from the city of Pittsburgh (see details in Hipwell et al., 2002); 85% of those contacted agreed to participate in the study. Screening data available for the current study were measurements at ages 9–10. Where possible, we used parent and child as informants. The outcome of interest was marijuana use at 5 or more times per year at age 14. Attrition was 11%.

1.6. Michigan Longitudinal Study (MLS) data set

(Zucker et al., 1996, 2000). The overall sample consisted of 882 boys and 368 girls. At the initiation of the study, 81.3% of the boys were White, and 18.7% Non-White (including African American, Native American, Biracial, and Hispanic), and 74.7% of the girls were White, 25.3% Non-White (including African American, Biracial, and Hispanic). Sample selection: Families were ascertained through two methods. The first involved recruitment through all district courts of fathers living in the area convicted for drunk driving with a biological son between the ages of 3 and 5 years old (N = 186). Fathers were also required to be living with the boy and his biological mother. The second group were required to have the same family composition, but were ascertained through the same neighborhoods as the court-recruited families. Door to door canvassing was carried out to recruit two subgroups: (1) families where neither parent met a lifetime substance use disorder diagnosis (designated as controls N = 143); (2) families where father met criteria for an alcohol use disorder (N = 138) but were not involved with the court. In addition to the original 3-5-year-old son and his biological parents, a female sibling within the range of 3–11, when present, was also recruited. If other siblings (both male and female) in the 3-11-year age range were also present in the home, they were recruited as well. Assessment at T1 for this study (MLS Wave 3): average ages: 10.55 for boys and 10.61 for girls. Where possible, we used both parent and child as informants, which is particularly important for the externalizing behaviors that are concealing in nature (e.g., theft), because parents often are not aware of this type of child behavior. The outcome of interest was child self-report of marijuana use at about age 14. Attrition was 10%.

2. Analyses

2.1. Predictor items for the screener

The potential items for analyses were identified by examining prior research (e.g., Clark and Winters, 2002), prior analyses with the available datasets, particularly the extensive analyses with CEDAR data (e.g., Vanyukov et al., 2009), identifying pertinent items available in the four longitudinal projects used in these secondary analyses, and deliberations on the acceptability of areas of inquiry for potential participants during the recruitment process. Based on these considerations, the constructs represented by the pool of items to be considered included child externalizing behaviors, child internalizing behaviors, and parent tobacco smoking. Child externalizing behaviors. In the case of the ABCD Study design, we are projecting from ages 9–10, when marijuana use typically is minimal and not a viable risk item for screening purposes. Therefore, for candidate items on child externalizing behaviors, we considered non-substance use characteristics that other studies have found to predict early-onset substance use in mid adolescence, particularly child externalizing behaviors (e.g., Vanyukov et al., 2009). Potential externalizing behaviors considered were vandalism, lying, and disobedience at school (see Table 2). Child internalizing behaviors. In addition, we examined whether selected internalizing behaviors augmented predictions. After examining potential internalizing items’ correlations with both the tentative screener (i.e., based on other screening variables) and with the outcome variable, we initially focused on the following items (see Table 2): (1) unhappy, sad or depressed; (2) too fearful or anxious; (3) secretive or keep things to oneself; (4) self-conscious or easily embarrassed. After considering which internalizing items correlated with the externalizing screener at that point, we finally focused on: (1) unhappy, sad or depressed; (2) too fearful or anxious (parents were the informants). Parent smoking. For candidate items on parent behaviors, parent smoking (Flay et al., 1998; Clark and Cornelius, 2004) was also considered a viable candidate. This candidate item for the screener (see Table 2) was available in the 4 study data sets.

Table 2.

Intermediate items set examined for the screener, and their comparability across the four studies.

Predictor items examined for the screener	CEDAR	Michigan	PYS	PGS
Child externalizing behavior
Destroys things belonging to his/her family or others	P	P	PC
Broke family possession				P
-broke school goods				P
-broke other goods				P
Lying or cheating	P	P	PC
Lied to get things				PC
Disobedient at school	P	P	PC
Sent home from school				PC
Lying or cheating	P	P	PC
Lied to get things				PC
Steals outside the home	P	P	PC
Steals things				PC
Child internalizing behavior
Unhappy, sad or depressed	P		P	C
Too fearful or anxious	P		P	C
Parent behavior
How frequently have you (parent) smoked cigarettes during the past 30 days?	P	P	P
Did you (parent) smoke any cigarettes in the past year?	P			P

Note: Parental smoking was dichotomized (yes/no) to be comparable over studies. In the PGS, three items (broke family possession, school goods, other goods were collapsed to reflect the item “destroys things belonging to his/her family or others”. C = Child as informant; P = Parent as informant.

2.2. Outcomes

The predicted outcome was marijuana use by ages 14–15 (before 15th birthday) with a frequency that indicated greater than experimental use (i.e., “regular use”). The available outcome categories varied across the studies (see Table 1), including monthly use in CEDAR, use at five times or more in the past year in the PYS and PGS, and 6 or more times during the past year in the MLS. The presence of marijuana use at or above these thresholds for the depicted ages defined “early-onset marijuana use” in these secondary analyses.

The evaluations of individual items and their combinations in relations to early-onset marijuana use were undertaken with Receiver Operating Characteristic (ROC) statistics (Hanley and McNeil, 1982, 1983). This approach is typically used in evaluating screening for diseases, with several examples in the prior literature focusing on substance use frequency in relation to adolescent substance use disorders (e.g., Chung et al., 2012; Clark et al., 2016).

Using ROC statistics, the evaluation of the prediction power of a screen is usually based on a 2 by 2 table, as illustrated in Fig. 1. The quality of a screen is indicated by four parameters: Sensitivity and specificity, which refer to True Positives/(True Positives + False Negatives), and True Negatives/(True Negatives + False Positives), respectively, and Positive predictive value and negative predictive value, which refer to True Positives/True Positives + False Positives and True Negatives/True Negatives + False Negatives, respectively. Area Under the Curve (AUC) analyses were used to establish (a) whether the prediction is better than chance; and (b) what the optimal cut-off is to minimize false negative and false positive errors. AUC can range from 0.5 (chance) to 1.0 (perfect), when sensitivity and specificity are considered equally important. In practice, AUC tends to be lower than 1.0, meaning that one cannot correctly classify all future marijuana users (True Positives) or correctly classify all future non-marijuana users (True Negatives). The general rule is that the higher the sensitivity, the lower the specificity. Lowering the cut-off score can increase sensitivity, but with the consequence that there will be more false positives.

Fig. 1.

The basics of a two by two prediction table in terms of correct and incorrect prediction.

Where sample sizes from study sites were sufficient, we created two subgroups, labeled “construction” and “validation” samples, using a randomization method, the SPSS random variable generation function. This partitioning of the samples was done to avoid idiosyncratic findings. Sufficient sample sizes were available to take this approach for CEDAR boys, PYS (boys), and PGS (girls), but not for CEDAR girls, MLS boys, or MLS girls. To support scale construction yet allow for validation in these limited samples, weightings were applied so that there were more subjects assigned to the construction subsample than to the validation subsample. To divide each of these samples into two groups, In summary, to advance the screener, we undertook 9 sets of analyses: (1) CEDAR, boys, construction sample (N = 304); (2) CEDAR boys, validation sample (N = 202); (3) CEDAR, girls (N = 202); (4) PYS boys, construction sample (N = 235); (5) PYS boys, validation sample (N = 217); (6) PGS girls’ construction sample (N = 1315); (7) PGS girls, validation sample (N = 862); (8) MLS boys (N = 509); (9) MLS girls (N = 343).

2.2.1. Optimizing screening items

We searched for equivalent predictor items of interest in each dataset. This is very important because we needed construct convergence among the four longitudinal datasets. We used prorating in cases where there were missing items (no more than 15% missing data) so that we would maximize the numbers of participants. Note that sample sizes varied somewhat due to missing cases for each analysis. In the PYS dataset, we combined parent and child information on child predictor variables to obtain a best estimate of the child behavior. For example, a behavior was counted when either the parent or child reported the behavior. Item scores were recoded as “Yes” or “No” where necessary to make them uniform across studies. For example, the Child Behavior Checklist [CBCL] has response options of 0, 1 or 2 (see details in Table 2) then item scores were recoded as Yes or No). We undertook separate analyses for each gender. We first determined which items were predictive of the outcome. We next summed significant items into an index, examined AUC, and computed sensitivity, specificity, and positive predictive power for the summary screening score. If the variance accounted for by these indicators proved too low, we repeated the procedure for “new items”. In the final analyses, three of the studies used CBCL items (Achenbach, 1978), and one study (PGS) used data based on self-reported antisocial behavior (child), MFQ (Mood and Feelings Questionnaire, child; (Messer et al., 1995), and the Child Symptom Inventory (Gadow and Sprafkin, 1994, parent version). The items from the CBCL, the MFQ, and the CSI were highly comparable (Table 2).

2.2.2. Data reduction

To reduce the number of potential screening items, we intercorrelated the predictor variables in the three Pittsburgh datasets (PYS, PGS, and CEDAR). The strategy was to only accept items with relationships in the positive direction (i.e., a risk that predicted higher substance use rate).

3. Results

Table 1 summarizes the average ages at baseline and follow-up assessments, the thresholds used for determining early-onset marijuana use, and the percentage of youth who had regularly used marijuana by ages 14–16 years. The prevalence of early-onset marijuana use by ages 14–15 varied from 5.5% to 16.2%. Not surprisingly, more youth from the higher-risk samples (e.g., CEDAR) compared to population-based samples (e.g., MLS) reported early-onset marijuana use, and boys reported higher levels of early-onset marijuana use than girls.

3.1. Data reduction

A series of analyses were undertaken to identify items most pertinent for a brief risk indicator. The intercorrelation results of the predictor items showed that some items were significantly negatively correlated with the outcome variable, and other items correlated with the outcome non-significantly across all three datasets. This reduced the number of viable items in the Pittsburgh datasets to 14. The Michigan group derived their own scale of 9 items (analysis results available from the authors). In brief, a procedure very similar to that described here for the three Pittsburgh datasets was used. We intercorrelated available predictor variables that overlapped with those originally identified across externalizing, hyperactivity/impulsivity, internalizing, and temperament items (i.e., initial template of 25 selected TLI items) with the outcome variable. This method was used to reduce the item pool, based on predictive accuracy. An intermediate, reduced set of items considered in the subsequent analyses are presented in Table 2.

In prediction analyses with early-onset marijuana use as the outcome, the most predictive items were three externalizing characteristics: destroys things belonging to his/her family or others (CBCL item 21), steals outside the home (CBCL item 82), and lying or cheating (CBCL item 43). We used these items to construct a preliminary 3-item scale. We the determined whether we could improve upon this 3-item scale by adding each of the intermediate sets of individual items (Table 2). The greatest improvement was seen with the addition of “disobedient at school” (CBCL 23). The addition of parent smoking (i.e., present if either parent regularly smokes cigarettes) also improved the prediction results in all data sets. Table 3 compares the performance of this “5-item Screener” (i.e., 4 child externalizing items and parental smoking) across both the construction and validation datasets. Considerable replication of results was shown across the nine analyses. Eight of the nine AUCs showed statistically significant findings. The AUCs were moderately strong, ranging from 0.59 to 0.74. In most instances, the AUC reduction from construction to validation analyses was low in magnitude (e.g., CEDAR boys reduced from 0.73 to 0.66, and PGS girls reduced from 0.67 to 0.65). However, in the case of the PYS boys, the validation sample result was a non-significant finding. Table 3 shows that across all analyses, almost all the results held equally well for boys and girls (Table 3).

Table 3.

Summary of the 5-item Screener (child: destroy-disobey-lie-steal; +parental smoking) predicting regular marijuana use.

Data set	AUC	Sensitivity, Specificity	Optimal Screening Score	Range of score
CEDAR (boys’ construction)	.761***	.65, .82	1.5	0 to 7
CEDAR (boys’ validation)	.689***	.91, .47	0.5	0 to 7
CEDAR (girls)	.773***	.88, .52	0.5	0 to 5
PYS (boys, construction)	.622*	.72, .47	1.5	0 to 7
PYS (boys, validation)	.594 ns	.76, .41	1.5	0 to 6
PGS (girls, construction)	.681***	.59, .70	2.3	0 to 10
PGS (girls, validation)	.646***	.70, .57	1.6	0 to 9
MLS (boys)	.621*	.77, .65	0.5	0 to 7
MLS (girls)	.707*	.78, .52	0.5	0 to 5

^*p < .05; ^**p < .01; ^***p < .001.

3.2. Optimal cut off score

The proposed cut-off score of the 5-item Screener was selected based on balancing sensitivity and specificity. The resulting categorization of youth at high risk for early-onset marijuana use was based on the best possible sensitivity and specificity according to the performed AUC analyses. Table 3 reports the sensitivity and specificity across the nine analyses, which varied somewhat across the data sets. The results show that the optimal cut-off score in 4 out of the 9 analyses is 1.5 or 1.6, and .5 in another set of 4 analyses. Thus, a score of 1 or 2 on the screener optimally identified children at high risk for early-onset marijuana use. Table 4 shows the more detailed results of the analyses with information about sensitivity, specificity, PPV, NPV, and overall accuracy for 5-item Screener scores. We interpret these results as indicating that a threshold or “cut off” score of 2 or more would provide acceptable results.

Table 4.

Results of the ROC analyses for the 5-item Screener predicting regular marijuana use.

Screen cut-off score	Sensitivity	Specificity	PPV	NPV	Overall accuracy (%)	Percentage of the population at or above screen cut-off score
CEDAR Boys	N = 465
1+	0.897	0.460	0.25	0.96	53	60.00
2+	0.603	0.762	0.34	0.90	74	29.89
3+	0.218	0.884	0.27	0.85	77	13.33
4+	0.128	0.948	0.33	0.84	81	6.45
5+	0.013	0.979	0.11	0.83	82	1.94
CEDAR Girls	N = 180
1+	0.882	0.521	0.16	0.98	56	51.67
2+	0.471	0.920	0.38	0.94	88	11.67
3+	0.118	0.957	0.22	0.91	88	5.00
4+	0.000	0.988	0.00	0.90	89	1.11
5+	0.000	0.988	0.00	0.90	89	1.11
PYS Boys	N = 431
1+	0.951	0.181	0.16	0.96	29	83.76
2+	0.738	0.441	0.18	0.91	48	58.47
3+	0.426	0.695	0.19	0.88	66	32.25
4+	0.180	0.859	0.17	0.86	76	14.62
5+	0.098	0.951	0.25	0.86	83	5.57
PGS Girls	N = 1543
1+	0.889	0.231	0.07	0.97	27	77.58
2+	0.722	0.549	0.09	0.97	56	46.66
3+	0.478	0.756	0.11	0.96	74	25.73
4+	0.300	0.884	0.14	0.95	85	12.64
5+	0.133	0.947	0.13	0.95	90	5.77
MLS Boys	N = 467
1+	0.765	0.346	0.08	0.95	38	66.17
2+	0.559	0.695	0.13	0.95	69	32.33
3+	0.265	0.857	0.13	0.94	81	15.20
4+	0.088	0.947	0.12	0.93	88	5.57
5+	0.029	0.977	0.09	0.93	91	2.36
MLS Girls	N = 206
1+	0.778	0.482	0.06	0.98	50	52.68
2+	0.444	0.909	0.18	0.97	89	10.68
3+	0.222	0.939	0.14	0.96	91	6.80
4+	0.111	0.980	0.20	0.96	94	2.43
5+	0.000	0.995	0.00	0.96	95	0.49

To further address whether internalizing items would enhance the accuracy of the 5-item Screener for predicting early-onset marijuana use, results with a scale including the 5-item Screener and two added internalizing items (i.e., sad, anxious: see Table 2) are presented (Table 5). A comparison between Table 3, Table 5 shows that the addition of internalizing behaviors did not systematically improve the AUC analysis results. The additional items resulted in marginally lower AUCs in six out of the nine analyses. Thus, the addition of internalizing items failed to improve the 5-item Screener.

Table 5.

Summary of the 7-item Screener (child: destroy-disobey-lie-steal--sad-anxious; + parent smoking) predicting regular marijuana use for construction and validation analyses.

Data set	AUC	Sensitivity, Specificity	Optimal Screening Score	Range of score
CEDAR (boys, construction)	.730***	.65, .76	1.5	0–11
CEDAR (boys, validation)	.660***	.54, .66	1.5	0–8
CEDAR (girls)	.744***	.59, .85	1.5	0–5
PYS (boys, construction)	.633*	.54, .65	2.2	0–8
PYS (boys, validation)	.601 n.s.	.59, .60	2.5	0–10
PGS (girls, construction)	# .673***	.63, .62	3.2	0–14
PGS (girls, validation)	# .653***	.55, .74	3.8	0–13
MLS (boys)	.590 n.s.	.82, .71	0.5	0–10
MLS (girls)	.685 n.s.	.78, .61	0.5	0 –6

#only one internalizing item, “felt miserable” added.

^*p < .05; ^**p < .01; ^***p < .001.

3.3. Other substance use outcomes

Using the 5-item Screener, subsequent analyses were conducted to investigate the prediction of other substance use outcomes. Specifically, we investigated the extent to which the screener predicted alcohol use and illicit drug use (i.e., other than marijuana use) at age 15 and older (Table 6). Because of measurement limitations, the analyses only focused on the CEDAR and the MLS data sets. The 5-item Screener applied to the CEDAR data set (Table 6) significantly predicted monthly alcohol use by the 15th birthday, illicit drug use (i.e., other than cannabis) by the 15th birthday, and the onset of a DSM-IV defined substance use disorder involving an illicit drug by the 18th birthday.

Table 6.

5-item Screener (four child externalizing behaviors and parental smoking) predicting other substance use outcomes (CEDAR data).

	Boys construction		Boys validation		all Girls
Outcome	AUC	p	AUC	p	AUC	p
AUD by 15th birthday	.611	n.s.	.665	n.s.	.525	n.s.
Monthly alcohol by 15th birthday	.603	*	.620	*	.665	*
Tried an illicit drug by 15th birthday	.743	***	.646	**	.788	***
Illicit drug SUD by 18th birthday	.658	***	.651	**	.707	**

Abbreviations: AUDDSM-IV defined alcohol use disorder diagnosis; illicit drugillegal substance of abuse other than cannabis; illicit drug SUDDSM-IV defined substance use disorder involving illegal drug of abuse other than marijuana.

^*p < .05; ^**p < .01; ^***p < .001.

We also examined two longer-term outcomes at an average age of 16.6 in the MLS data set. The 5-item Screener significantly predicted frequency of cigarette use in the past month (boys: AUC: 0.60, p < .05; girls: AUC: 0.62, p < .05), and frequency of problems associated with drinking in the past year (boys: AUC: 0.57, p < .05; girls: AUC: 0.61, p < .05). Thus, these results held for both and girls.

Lastly, we examined in the CEDAR data set whether the inclusion of two internalizing items (child sad, anxious) improved the prediction when added to the 5-item Screener on the longer-term outcomes. The results show that the inclusion of internalizing items did not improve predictions when added to the 5-item Screener and, instead, slightly reduced the AUC (Table 7).

Table 7.

7-item screener (four child externalizing behaviors, 2 internalizing items, and parental smoking) predicting other substance use outcomes (CEDAR data).

	Boys construction		Boys validation		all Girls
Outcome	AUC	p	AUC	p	AUC	p
Alcohol diagnosis by 15th birthday	.597	n.s.	.643	n.s.	.580	n.s.
Alcohol monthly use by 15th birthday	.605	*	.583	n.s.	.665	*
Tried an illicit drug by 15th birthday	.703	***	.633	**	.759	***
Illicit drug disorder by 18th birthday	.645	**	.643	**	.695	**

^*p < .05; ^**p < .01; ^***p < .001.

4. Conclusion

The results show that the combination of child externalizing items (4 items: destroys things belonging to his/her family or others; disobedience at school; lying or cheating; steals outside the home) and parent tobacco smoking (1 item) was the optimal risk screen to predict early-onset marijuana use by age 14–16. The results across the nine analyses using four longitudinal data sets were similar for boys and girls. Construction and validation analyses in three samples showed the reduction in AUC to be small. Therefore, we recommend the same screen for boys and girls. For boys and girls samples, internalizing behaviors did not strengthen the predictive utility of the screen for regular marijuana use in mid-adolescence. The inclusion or exclusion of these items in a future screen is not expected to alter the screen’s efficiency for this outcomes in mid adolescence. However, this does not preclude the possibility that internalizing behaviors become indicative of later substance use, use of other substance types, or mental disorders involving negative affect. For example, Hussong et al., (2016) have noted that a continuity model for the relationship of negative affectivity to substance use prior to adulthood.

4.1. Strengths and limitations

Among the present paper’s strengths are longitudinal data from four independent longitudinal studies, measurement of screening items in late childhood/early adolescence, multiple informants, the use of construction and validation samples, and substantial replication of the screener across data sets for boys and girls across 9 sets of analyses. The resultant screener was much shorter than the TLI developed by Vanyukov, Tarter, Clark and colleagues (e.g., Vanyukov et al., 2009). Our analyses had several other strengths. Most of the items included in the analyses were from a standard instrument (CBCL, parent report), complemented by the report of parent smoking. We undertook construction and validation analyses in three samples (CEDAR boys, PYS boys, PGS girls), which mostly showed that the reduction in variance accounted for was minimal, only one of the results [the PYS] was non-significant. The screener also predicted other substance use outcomes at ages 15–18, including alcohol and nicotine use. The screener applied to both boys and girls, which increases its applicability in the recruitment of participants for future studies.

Several limitations apply to the present report. Only one study reported here had both parent and child as informants on child externalizing behaviors. The significant results using only the parent as an informant suggests that a screener can be based just on parent responses. The current results balanced sensitivity and specificity to select risk score thresholds for future applications. The four longitudinal studies are restricted to local rather than national samples and were executed during different historical time periods. In addition, two of the samples had an overrepresentation of youth with parents with substance use disorders (i.e., CEDAR and MLS). Further, assessments (predictors and outcomes) varied slightly among the studies. The magnitude of the AUC, although statically significant in 8 out of the 9 analyses, was modest. While the performance of the 5-item Screener was adequate for the application intended here, the approach was not designed to be applicable to guiding preventive or clinical interventions. The outcome at age 15 was selected based on the potential application of this risk screen to the ABCD Study. Longer-term outcomes were available from the CEDAR and the MLS studies and in the future could be obtained from the other two studies. Analyses by race have not been undertaken. Parent substance use other than cigarette smoking was thought to be imprudent for a screener that may be collected during recruitment. We elected to exclude from consideration items that might elicit confidentiality concerns from potential participants. However, we acknowledge that other risk characteristics have been identified (e.g., Clark and Winters, 2002) and alternative or additional information may have improved prediction results. Furthermore, while these analyses focused on the outcome of early-onset marijuana use, other adverse outcomes are pertinent for the ABCD Study.Informed by these analyses, the ABCD Risk Indicator was developed (Garavan et al., 2018 this issue). As supported by these analyses, an “Externalizing Risk Profile” was included, with the four child externalizing behaviors and parental smoking, and a score of two or higher as the high-risk threshold. In addition, two negative affect risk components were also included which were not based on these analyses. “NA Component 1″ included three items (i.e., CBCL 50: too fearful or anxious; CBCL 103: unhappy, sad, or depressed; CBCL 56: physical problems without known medical cause) and “NA Component 2″ included two items (i.e., slow at making friends; finds life to be stressful). The ABCD High Risk Indicator is considered “higher risk” when either the Externalizing Risk Profile is two or higher, or both the NA Component scores are one or higher. The rationale for including these components was intended to broaden target outcomes to include mental disorders and is described elsewhere (i.e., Garavan et al., 2018 this issue).Since mental disorders involving negative affect, such as Major Depressive Disorder, are more common in girls than in boys in the adolescent developmental period (Mojtabai et al., 2016), the inclusion of these items may be particularly useful for the identification of high risk girls.

Future work on this and other risk screeners for 9–10-year-old youth’s risk for substance use should focus on long-term outcomes, including regular marijuana and other heavy substance use, as well as substance use disorders. The data from the ABCD study, with its very large sample across 21 sites, will be ideal for this purpose.

Conflict of Interest

None.