Prevention of Substance Use in Children/Adolescents with Mental Disorders: A Systematic Review

Abstract

Objective:

We conducted a systematic review to answer the question: Among youth ≤18 years of age with a mental disorder, does substance use prevention compared to no prevention result in reduced rates of substance use/abuse/disorder (SUD)? The review was requested by the Ontario Ministry of Health and Long-term Care through the Canadian Institutes for Health Research Evidence on Tap program.

Methods:

A four-step search process was used: Search 1 and 2: Randomized controlled trials (RCTs) that evaluated a SUD prevention intervention in individuals with a mental disorder who were: 1) ≤18 years; or, 2) any age. Search 3: Observational studies of an intervention to prevent SUD in those with mental disorder. Search 4: RCTs that evaluated a SUD primary prevention skills-based intervention in high-risk youth ≤18 years.

Results:

Searches 1 and 2: one RCT conducted in youth was found; Search 3: two observational studies were found. All three studies reported statistically significant reductions in substance use. Search 4: five RCTs were found with mixed results. Methodological weaknesses including inadequate study power may explain the results.

Conclusions:

Little is known about effective interventions to prevent SUD in youth with a mental disorder. Effective SUD primary prevention programs exist and should be evaluated in this high-risk group.

Introduction

Longitudinal studies show that youth under 18 years of age with mental disorders are at high-risk for developing substance use disorder (SUD) in adolescence and adulthood (Armstrong & Costello, 2002; Schwartz, Garland, Harrison, & Waddell, 2007; Costello, Mustillo, Erkanli, Keeler, & Angold, 2003; Hodgins et al., 2007; Mutale, 2003; Adair, 2009). A major review published in 2002 found that 50–80% of adults with SUD have at least one mental disorder, 50% of adolescents with conduct disorder or antisocial personality disorder meet criteria for drug dependence, and as many as 45% of adolescents with mood disorders were abusing drugs, followed by 35% of those with anxiety disorders (Armstrong & Costello, 2002). Studies of adolescents in substance use treatment programs also show the majority have co-occurring mental disorders (Chan, Dennis, & Funk, 2008).

Children and adolescents with both a mental disorder and substance abuse disorder (i.e., concurrent disorder) are difficult to serve and place a huge burden on individuals, families, and health systems. Each disorder exacerbates the other and together often lead to family dysfunction, delinquency, and reduced life opportunities (Faggiano et al., 2008). In addition, many of these youth have poor treatment compliance and poor outcomes, both in the short- and long-term (Couwenburgh et al., 2006; Schwartz et al., 2007; Adair, 2009).

Secondary prevention of SUD (excluding “tobacco only” prevention strategies) in youth with mental disorders was identified by the Ontario Ministry of Health and Long-term Care (the Ministry) as a policy and programming priority during a Canadian Institutes of Health (CIHR) Evidence on Tap (EonT) project (Bennett et al., 2010). The Ministry requested advice through the EonT program because: i) as many as 14% of youth in their jurisdiction may have a mental disorder (Waddell, Offord, Shepherd, Hua, & McEwan, 2002); and, ii) the benefit to individuals, the healthcare system and society associated with reducing the risk of concurrent disorder is potentially large. Accordingly, we conducted the systematic review reported below.

Methods

We used The Cochrane Collaboration (2009) systematic review methods as follows.

Research Question: Among youth ≤18 years of age with a mental disorder (population), does substance use prevention (intervention) compared to no prevention (comparison) result in reduced rates of SUD (outcome)?

Search Strategy: Our research librarian created a search strategy (available from the authors) designed to identify randomized controlled trials (RCTs) relevant to our research question. Medline, Embase, PsycINFO, CINAHL, and the Cochrane Collaboration databases were searched using variations of the following keywords: prevention, intervention, diagnosis, substance use/abuse, and mental disorders and included studies up to October, 2010.

Inclusion Criteria: All studies that met the following criteria were eligible: a) RCT; b) evaluated a substance use prevention intervention in youth ≤18 years of age with a pre-existing mental disorder; c) published in English; d) outcomes included substance use, abuse and/or disorder; and, e) clinic- or population-based sample.

Search 1 Results

Screening of the search results revealed only one RCT, so three additional searches were conducted in order to provide the Ministry with the best available information and promising leads.

Searches 2, 3, and 4

Search 2: The research question and second search were identical to the first search except that the population included all ages.

Search 3: The research question and third search were identical to the second search except that observational studies (non-randomized studies with or without a control group) were searched for rather than RCTs.

Search 4: Given the low yield of searches 1–3, a fourth search focused on SUD primary prevention strategies evaluated in children/adolescents who were deemed to be at increased risk for SUD. The research question was: Among youth ≤18 years of age deemed to be at high risk for SUD (population), does a skills-based substance use prevention (intervention) compared to no prevention, or non-skills-based prevention (comparison) result in reduced rates of SUD (outcome). Since the population of interest to the Ministry (i.e., youth with a mental disorder) is at high risk for developing SUD, SUD prevention programs shown to be effective in other high-risk groups can inform the selection of promising strategies worthy of further evaluation in youth with a mental disorder. Skills-based substance use primary prevention programs were the focus because a recent systematic review (Faggiano et al., 2008) concluded they are the most effective. Skills-based programs aim to improve students’ abilities in general life skills as well as refusal and safety skills (Faggiano et al., 2008).

Search Strategy: Our research librarian used variations of the following keywords to search the databases noted in Search 1 (search available from authors): RCTs, prevention, intervention, substance use/abuse and included studies up to February, 2011.

Inclusion Criteria: a) RCT; b) evaluated a skills-based substance use/abuse primary prevention intervention for alcohol and illicit substance use in youth ≤18 years of age; c) English language; d) outcomes included substance use, abuse and/or disorder; e) baseline high-risk status determined through individual screening; and, f) conducted in Canada, United States, Australia, New Zealand, or the United Kingdom (to optimize applicability to a Canadian setting).

Screening and Assessment of Eligibility

Authors Bennett, Salvo and Chen screened all titles/abstracts for relevance. Authors Salvo and Chen used the inclusion criteria to review the full text of all remaining studies and determine study eligibility. Kappa values were calculated for inter-rater reliability (Salvo and Chen) and disagreements were discussed with a third party (Bennett). Reference lists for all studies deemed eligible were searched for additional studies.

Study Level Data Extraction

A standardized form was used to record: i) study design characteristics; ii) measures used to assess the presence of mental disorder and substance use/abuse; iii) odds ratios (OR) and/or p-values for substance use/abuse behaviours; and, iv) Cochrane Risk of Bias/GRADE (Grading of Recommendations Assessment, Development, and Evaluation) criteria for assessing individual studies (see section 2.8). Salvo and Chen extracted data from the studies that met the inclusion criteria. Disagreements were resolved through review of original study reports and discussion.

Risk of Bias Assessment

Cochrane/GRADE Risk of Bias guidelines were used to assess all included RCTs (Balshem et al., 2011; Guyatt et al., 2011; The Cochrane Collaboration 2009; Higgins et al., 2011; see Table 3). An additional criterion was added that assessed whether program fidelity was measured (Dumas, Lynch, Laughlin, Smith, & Prinz, 2001) which is critical to interpreting trials reporting low or no effects (Borrelli et al., 2005).

Table 3.

Risk of bias assessment of RCTs

GRADE Criterion*	Study number
	1	4	5	6	7	8
1. Allocation concealment?	✕	✕	✕	✕	✕	✕
2. Blinding?	✕	✕	✕	✕	✓	✕
3. Power analysis?	✕	✕	✕	✕	✕	✓
4. Use of intention to treat analysis?	✕	✕	✕	✓	✓	✓
5. Use of validated outcome measure?	✓	✓	✓	✕	✓	✓
6. Loss to follow-up ≤ 20%	✕	✕	✕	✓	✓	✕
7. Controlled for clustering effects?	N/A	N/A	✓	N/A	N/A	N/A
8. Assessed program fidelity?	✕	✕	✓	✓	✕	✕

^*When studies failed to report it is assumed that the method was not employed

N/A: Not applicable

Results

Search Results

Figure 1 shows the PRISMA (preferred reporting items for systematic reviews and meta-analyses) flow diagram (Moher, Liberati, Tetzlaff, Altman, & The PRISMA Group, 2009). The combined results of the first three search strategies resulted in 2311 articles; 48 remained after title and abstract screening; following full text review, three studies met our inclusion criteria. Search 4 yielded 932 articles; 31 remained after title and abstract screening; after full text review, five studies met our inclusion criteria. The inter-rater reliability for all searches was 0.92.

Figure 1.

PRISMA Diagram: Search 1–3 (a), Search 4 (b)

Included Study Characteristics

Searches 1–3: Three studies were identified (See Table 1a for summary of study characteristics/intervention content). One RCT evaluated a program for the prevention of SUD in a clinical sample of children with disruptive behaviour disorder (DBD) (Zonneyville-Bender, Matthys, Van De Wiel, & Lochman, 2007). Children were assessed for substance use five years after initiation of the program using the Centre for Substance Abuse Prevention Youth Survey’s “use” items, which were dichotomized into “use” or “no use”. The second trial was a longitudinal cohort study in which adolescents with attention-deficit hyperactivity disorder (ADHD) who received pharmacotherapy were compared with: a) non-medicated adolescents with ADHD; and, b) a group of healthy controls for the presence of SUD (Biederman, Wilens, Mick, Spencer, & Faraone, 1999). The authors hypothesized that the ADHD medication may reduce the risk of developing a SUD. The third study was a naturalistic observation of female adolescents who were deemed to be at “ultra high risk” of developing psychosis or were in the early stages of psychosis (Bucci et al., 2010).

Table 1a.

Characteristics of included studies (Search 1–3)

Study (number)	Study type	Sample size	Age	Population	Intervention	Comparison	Follow-up (years)	Attrition rate (%)	Outcome measure
Zonneyville-Bender MJS, et al., 2007 (1)	RCT	77	8–13	DBD; child psychiatric clinics & MH centres	UCPP; CBT + behavioural parent intervention	Care as usual & healthy controls	5	21	CSAP Youth survey (past month use)
Biederman J et al., 1999 (2)	Cohort study	260	15–17 (males)	75 with diagnosed ADHD, 137 control subjects; originally families were chosen from both psychiatric and non-psychiatric settings	ADHD medication	ADHD no medication & control group	4	19	DSM-III-R substance use disorder
Bucci S et al., 2010 (3)	Naturalistic evaluation	58	12–27	At “ultra high risk” or early psychosis; either non-users or current users of cannabis	Non-users: brief advice, +/− reinforcement users: intense therapy (MI and CBT) (more regular users received longer program duration)	None	1	43	OTI (past 28 days)

ADHD: Attention Deficit Hyperactivity Disorder; CBT: Cognitive Behavioural Therapy; DBD: Disruptive Behaviour Disorder; DSM-III-R: Diagnostic and Statistical Manual v3 Revised; MI: Motivational Interviewing; OTI: Opiate Treatment Index; UCPP: Utrecht Coping Power Program

Search 4: Five skills-based substance use prevention programs for high-risk children/adolescents were identified (See Table 1b for summary of study characteristics/intervention content). In these studies, a variety of high-risk groups were identified by screening participants for risk factors for substance use/abuse (see Table 1b for specific screening criteria). All five studies utilized a self-report outcome measure for substance use. Two trials employed a study-designed measure and three used an existing, validated instrument. Only two trials validated the self-report measures with biological tests, such as breath carbon mon-oxide or urine samples. The median follow up length is one year (3 months – 2.5 years).

Table 1b:

Characteristics of included studies (Search 4)

Study (number)	Program	Sample size	Age	Population	Intervention description	Comparison	Follow-up	Attrition rate (%)	Outcome measure
Palinkas LA, et al., 1996 (4)	Project PALS (positive adolescent life skills)	293	14–19 (females)	At risk for drug use based on POSIT	Cognitive behavioural training + Facts of life	Facts of life only (education intervention)	0.25	24	PEI for drug use; urine toxicology (past 3 month use)
Griffin KW, et al., 2003 (5)	Life skills training	758	12	High social/academic risk	Cognitive-behavioural & life skills	Normal prevention curriculum	1	37	Self report; CO breath samples (composite scores of frequency/quantity of use)
Pantin H et al., 2009 (6)	Familias Unidas	213	14 (mean)	Hispanic families with parent-reported behaviour problems on the RBPC	Parenting skills	Community control	2.5	15	Self report (past 30 day use)
Walton M, et al., 2010 (7)	SafERteens	726	14–18	Past-year aggression and alcohol consumption	Brochure + 35 minute skills-based intervention (computer or therapist delivered)	Control group with brochure	0.25 & 0.5	14 (0.5 years)	AUDIT-C; (binary)
Conrod PJ et al., 2010 (8)	Preventure trial	732	11–13	≥1 SD on subscale of SURPS	Coping skills intervention	No intervention	2	47	RBQ (past 6 month use)

AUDIT-C: Alcohol Use Disorders Identification Test-Consumption; CO: Carbon monoxide; PEI: Personal Experiences Inventory; POSIT: Problem-Oriented Screening Instrument for Teenagers; RBPC: Revised Behaviour Problem Checklist; RBQ: Reckless Behaviour Questionnaire; SD: Standard deviation; SURPS: Substance Use Risk Profile Scale

Program Impact on SUD

Searches 1–3 (see Table 2a): The sole RCT found the intervention to be statistically significant for preventing marijuana use when compared with usual care. A non-significant result was found when comparing the intervention group to healthy controls, which suggests that risk for use among those receiving the intervention is reduced to that of the untreated, healthy control group (Zonneyville-Bender et al., 2007). The longitudinal cohort study found that adolescent males who were not pharmacologically treated for ADHD had a significantly higher risk of alcohol use disorder and SUD in general than those who were on ADHD medication (Biederman et al., 1999). Although no statistical differences were found for marijuana, cocaine, or hallucinogen use disorders Table 2a illustrates that large effects were found which may have significant clinical implications. The wide confidence intervals associated with these effects indicate the study’s lack of power to detect a statistically significant difference. These results, however, should be taken in light of the fact that in order to control for age as a confounder, the ADHD group was significantly smaller than the healthy controls as well as the fact that further follow-up may show a different result as the subjects age beyond 17 years old. The naturalistic observation found statistically significant reductions in both cannabis and polydrug use, however, less than 25% of the original sample was followed up to one year so the outcome in 75% of the sample is unknown (Bucci et al., 2010).

Table 2a.

Secondary prevention study results

Substance	Study number	Substance use
		OR (95% CI)	Other
Marijuana	1	0.28*1
	1	3.6*2
	1	1.003
	2	3.1 (0.8–12.5)4
	2	0.42 (0.11–1.7)5
	3		↓ (p<0.01)
Cocaine	2	7.5 (0.3–163.4)4
	2	0.2 (0.02–2.1)5
Alcohol	2	5.8 (1.7–19.3)*4
	2	0.16 (0.05–0.57)*5
Hallucinogen	2	1.0 (0.1–9.3)4
	2	0.76 (0.12–5.0)5
Illicit/hard drugs	-	-	-
Polydrug use	3		↓ (p<0.01)
Any substance	2	6.3 (1.8–21.4)*4
	2	0.015 (0.04–0.6)*5

^*Statistically significant result

¹UCPP versus care as usual

²Care as usual versus controls

³UCPP versus controls

⁴ADHD medication (−) versus controls

⁵ADHD medication (+) versus ADHD medication

(−) OR = Odds Ratio; CI = Confidence interval

Search 4 (see Table 2b): The results of the skills-based primary prevention programs were mixed. Three programs measured marijuana use but none found a statistically significant difference (Palinkas, Atkins, Miller, & Ferreira, 1996; Griffin, Botvin, Nichols, & Doyle, 2003; Conrod, Castellanos-Ryan, & Strang, 2010). Alcohol use prevention was studied in three trials, with one finding a statistically significant effect (p=0.008) (Griffin et al., 2003). Program results were more pronounced in the hard drug classes including cocaine use and inhalant use, however, fewer studies included these drug classes as an outcome. One study evaluated cocaine use and reported a statistically significant result with an OR of 0.20 (0.1–0.5) (Conrod et al., 2010). One study looked at illicit drug use but did not find a statistically significant difference (Palinkas et al., 1996). Another study assessed the effect of an intervention on inhalant use prevention and reported a statistical difference (p=0.043) (Griffin et al., 2003). Three of the five programs evaluated the effects on any substance use, two of which had statistically significant effects (Griffin et al., 2003; Pantin et al., 2009); one reported an OR of 0.65 (no confidence interval was provided) and the other showed significantly less use in the intervention group with a p-value of 0.004. Among the results in Table 2b that did not reach statistical significance, there were several that met criteria for clinical significance with respect to magnitude (i.e., 20% or greater increase/decrease in odds (Guyatt, Juniper, Walters, Griffith, & Goldstein, 1998)). Lack of adequate power may explain these results, especially when taking into account the high attrition rates (Table 1b).

Table 2b.

Results of ‘high risk’ group RCTs

Substance	Study number	Substance use
		OR (95% CI)	Other
Marijuana	4	1.4 (0.7–2.8)
	5		↓ (p=0.126)
	8	0.7 (0.5–1.0)
Cocaine	8	0.2 (0.1–0.5)*
Alcohol	4	0.9 (0.5–1.6)
	5		↓ (p=0.008)*
	7	0.87 (0.56–1.35)1
	7	0.78 (0.51–1.20)2
Illicit/hard drugs	4	0.8 (0.4–1.9)
Inhalants	5		↓ (p=0.043)*
Any substance	4	1.3 (0.7–2.4)
	5		↓ (p=0.004)*
	6	0.65*

^*Statistically significant result

¹Computer delivered versus control

²Therapist delivered versus control

OR = Odds Ratio; CI = Confidence Interval

Risk of Bias

Table 3 shows the Risk of Bias assessment. The two non-RCTs are excluded for simplicity (the cohort study did not utilize a control group with similar baseline risk to the intervention group and the naturalistic observational study did not have a control group, indicating significant Risk of Bias). All of the RCTs were lacking in several key areas. None of the studies discussed allocation concealment to the randomization process and only one study identified blinding participants prior to intervention initiation (Walton et al., 2010). Although it is impossible to blind participants due to the nature of the interventions, randomization concealment as well as blinding data analysts is possible. Only one study reported performing a sample size calculation in order to ensure adequate power (Conrod et al., 2010). Five of the studies employed a validated method of assessing the outcome (Zonneyville-Bender et al., 2007; Palinkas et al., 1996; Griffin et al., 2003; Walton et al., 2010; Conrod et al., 2010), however, self-report measures were used in every study and are subject to recall bias. Only two studies used biological testing to assess the level of bias associated with the self-report measure (Palinkas et al., 1996; Griffin et al., 2003). Four of the six studies had a loss to follow-up of greater than 80% (Zonneyville-Bender et al., 2007; Palinkas et al., 1996; Griffin et al., 2003; Conrod et al., 2010), which could result in biased estimates of effect size, particularly when loss is unequal between study groups. This was evaluated in two out of three studies where attrition was greater than 20%, both reported no differences between intervention and control groups (Griffin et al, 2003; Palinkas et al., 1996). The magnitude of loss to follow-up observed in these studies also indicates the challenge in performing longitudinal research in high-risk populations. Finally, two of the trials incorporated a fidelity measure—one program reported successful implementation (Pantin et al., 2009) while the other reported a 48% overlap between what was developed and what was delivered (Griffin et al., 2003).

Discussion

SUDs are prevalent among children and youth with mental disorder with estimates ranging from 20–70% (Couwenburgh et al., 2006). Effective secondary prevention of SUDs in youth ≤18 years of age could reduce the occurrence of concurrent disorder and the associated burden on individuals, families, and health systems. Our systematic review revealed that only three studies of strategies to prevent SUD in individuals with existing mental disorders are available, with only one conducted in youth under 18 years. It is encouraging that this RCT showed promising results for skills-based interventions.

When we broadened our search to include primary prevention programs evaluated in other high-risk groups (because these studies can inform the selection and evaluation of promising secondary prevention strategies for youth with a mental disorder), we found five RCTs. Overall the results are promising, but mixed. The prevention of illicit drug use in these trials was more successful than that of the other substances (e.g. marijuana, alcohol). Programs that incorporated the development of coping skills and drug resistance skills as well as the development of important personal and social skills in the early adolescent age group (11–13 years) showed the most promise with respect to preventing substance use (Conrod et al., 2010; Griffin et al., 2002).

Our Risk of Bias review revealed significant methodological weaknesses that need to be remedied in future research. Blind outcome assessment was not reported in any trial. Given this is possible for measures that are not completed by study participants, future trials would be significantly strengthened by including blinding, particularly for biological measures of substance use. High attrition rates were also observed, ranging from 14% to 47%. This is a challenging problem, but increased vigilance by investigators could reduce loss to a lower level than that reported. Only one trial addressed study power. This is critical to the rigor of further RCTs and our ability to interpret negative results. For example, approximately half of the primary prevention studies reported between study group differences that were clinically meaningful in size, but not statistically significant suggesting that inadequate power may be responsible for the mixed results. Finally, only two of the trials included fidelity measures to confirm that the intervention was administered as intended and to assist with the interpretation of negative results.

The strengths of this review are twofold. First, we address a question of great relevance to the mental health of children and adolescents that appears to be neglected. Our review brings the field into focus, provides guidance to health system decision-makers and practitioners regarding the current state of knowledge, suggests promising leads and sets the stage for further research. Second, we use transparent systematic review methods, including the multi-step search strategy used to identify pertinent literature and a Risk of Bias review that revealed significant methodological weaknesses in the available trials.

Possible limitations of our review center on the small number of RCTs available in our target population. In addition, the focus in Search 4 on skills-based primary prevention programs could have resulted in missed studies that might alter our conclusions. However, given the fact that other recent systematic reviews conclude that affective or knowledge-based programs are only marginally effective for SUD prevention in populations without a mental disorder (Faggiano et al., 2008), this risk seems negligible.

In summary, the paucity of RCTs relevant to our review question identifies a research gap that needs urgent attention. Search 4 findings provide direction by identifying primary prevention programs that may be relevant to youth with a mental disorder and should be considered for further evaluation in methodologically rigorous RCTs. Health system decision-makers and practitioners can aid in moving this agenda forward by contributing guidance on how to: i) harmonize efficacy research intervention programs with system and service design issues and the characteristics of routine delivery settings; and, ii) successfully engage youth. Researchers can provide leadership in the development of research protocols that address the methodological problems that characterize the available research. Together we can work towards the generation of better evidence that health service system decision-makers and practitioners can use to improve the outcomes of children/adolescents with mental disorder.