Table 2.
Strategies to improve mental health of children and adolescents: summary table
| Strategy, study designs, N | Target condition and ages of youth | Comparisons | Component of the strategy | Major findings | Strength of evidence from results | Reasons for strength of evidence |
|---|---|---|---|---|---|---|
| Training therapists to implement an EBP Beidas et al. [50] Cluster RCT, 115 therapists |
Anxiety Ages 8–17 years |
Augmented active learning vs. routine professional training workshop | Educational meetings or materials | No differences between arms for practitioner satisfaction with approach, protocol adherence, or practitioner skill | Low for no benefit for practitioner satisfaction, adherence, and skill | Low risk of bias, small sample size, imprecise results |
| Computerized routine training vs. routine professional training workshop | Educational meetings or materials | No differences between arms for practitioner protocol adherence or program model fidelity, or skill; computerized training group practitioners less satisfied than routine training group practitioners | Low for no benefit for practitioner satisfaction, adherence, and skill | Low risk of bias, small sample size, imprecise results | ||
| Feedback of patient symptoms to practitioners Bickman et al. [13] Cluster RCT, N of clinicians unclear, 340 youth, 144 clinicians, 383 caregivers |
General mental health problem (children who receive home-based mental health treatment) Mean age = 15 years |
Weekly and cumulative 90-day feedback vs. cumulative 90-day feedback only on patient symptoms and functioning to practitioners | Audit and feedback | Two thirds of practitioners did not view Web module | Insufficient for practitioner adherence | High study limitations, unknown precision for adherence |
| Membership in the weekly feedback group increased the rate of decline in functional severity scale by 0.01 (range: 1 to 5, higher scores indicate greater severity) | Low for benefit for functional severity | High study limitations, precise results for symptoms | ||||
| Feedback of patient treatment progress (symptoms and functioning) and process (e.g., therapeutic alliance) to practitioners Bickman et al. [61] Randomized block RCT, N of clinicians unclear, 257 youth, 2 clinics (one urban, one rural) at a single agency 21 clinicians, 255 caregivers |
General mental health problem (children who receive mental health treatment from a community mental health clinic) New patients aged 11–18 |
Session-by-session feedback vs. cumulative 6-month feedback to clinicians | Audit and feedback | No significant differences in percentage of sessions held or percentage of clinicians, youth, or caregivers who completed the questionnaire required at each visit | Insufficient for patient engagement, for practitioner adherence/program model fidelity, and system uptake | High study limitations, unknown precision for each intermediate outcome. |
| No patient-reported, caregiver-reported, or clinician-reported differences in symptoms or functioning of youth associated with intervention group in either clinic except feedback effects only seen in clinician ratings from one clinic (beta feedback*slope = −0.01, p = 0.045) | Low for no benefit for symptom severity | High study limitations, precise results for symptoms | ||||
| Computer decision support for guidelines Carroll et al. [46] Cluster RCT, 84 patients |
General mental health problem (children who receive home-based mental health treatment) Mean age = 15 years |
Computer decision support plus electronic health record (EHR) that included diagnosis and treatment guidelines vs. computer decision support plus EHR only | Educational meetings or materials Patient-reported data Reminders Quality monitoring |
Practitioner adherence improved through uptake of guidelines for diagnostic assessment (aOR, 8.0; 95% CI, 1.6 to 40.6); more reporting of 3 of 4 symptom domains at diagnosis | Low for benefit for practitioner adherence and program model fidelity | Medium study limitations, imprecise results with small number of events, large magnitude of effect |
| No statistically significant differences on practitioner adherence through reassessment of symptoms at 3 months, adjustment of medications, and mental health referral | Insufficient for practitioner adherence (reassessment of symptoms) at 3 months, adjustment of medications, and referral | Medium study limitations, imprecise results (CIs cross the line of no difference) | ||||
| Visit to a mental health specialist calculated OR 2.195; 95% CI, 0.909 to 5.303; p = 0.081; reported p value in study = 0.054 | Insufficient for service utilization | Medium study limitations, imprecise results (CIs cross the line of no difference) | ||||
| Internet portal to provide access to practice guidelines Epstein et al. [56] Cluster RCT, 746 patients |
Attention deficit hyperactivity disorder (ADHD) Ages 6 to 12 years |
Internet portal providing practitioner access to practice guidelines vs. wait-list control | Educational meetings or materials Patient-reported data Audit and feedback Reminders Quality monitoring |
Strategy appeared to improve 4 of 5 examined outcomes that measured practitioner protocol adherence and program model fidelity outcomes (mean change in proportion of patients who received targeted, evidence-based ADHD care outcomes between groups ranged from 16.6 to −50), but estimates were very imprecise, with large CIs | Low for benefit for practitioner protocol adherence and program model fidelity | Medium study limitations, imprecise (wide CIs) |
| Collaborative consultation treatment service to implement quality measures Epstein et al. [47] Cluster RCT, 38 practitioners, 144 patients |
ADHD Mean age = 7 years |
Collaborative consultation treatment service to promote the use of titration trials and periodic monitoring during medication management vs. control | Audit and feedback Multidisciplinary team |
Practitioner adherence/ fidelity as measured by use of titration trials β = −0.283; SE, 0.09; p < 0.01 and by use of medication monitoring trials: p = NS, details NR |
Insufficient for practitioner adherence and fidelity | High study limitations, imprecise results (small sample size) |
| Lower odds with overlapping confidence intervals of practitioner citing obstacles to implementation of EBP in 6 of 8 measures (2 reached statistical significance) | Insufficient for practitioner competence/ skills | High study limitations, imprecise results (small sample size) | ||||
| F score for decrease in combined parent and teacher ratings of ADHD symptoms for group x time interaction: F 2, 144 = 0.44, p = 0.65 | Insufficient for patient change in mental health symptoms | High study limitations, imprecise results (small sample size) | ||||
| Paying practitioners to implement an EBP Garner et al. [53] Cluster RCT, 105 therapists, 986 patients |
Substance use disorders Mean age = 16 years |
Paying practitioners for performance in successfully delivering an EBP intervention vs. implementation as usual | Provider incentives | Therapists in the P4P group were over twice as likely to demonstrate implementation competence compared with IAU therapists (Event Rate Ratio, 2.24; 95% CI, 1.12 to 4.48) | Moderate for benefit for practitioner competence | Medium study limitations, precise results |
| Patients in the P4P condition were more than 5 times as likely to meet target implementation standards (i.e., to receive specific numbers of treatment procedures and sessions) than IAU patients (OR, 5.19; 95% CI, 1.53 to 17.62) | Low for benefit for practitioner adherence and program fidelity | Medium study limitations, imprecise results (wide CIs) | ||||
| No statistically significant differences between groups OR, 0.68; 95% CI, 0.35 to 1.33 | Low for no benefit for patient change in mental health symptoms | Medium study limitations, precise results | ||||
| Program to improve organizational climate and culture Glisson et al. [14]a Two-stage RCT, 596 youth, 257 therapists |
Externalizing behaviors (youth referred to juvenile court with behavioral or psychiatric symptoms that require intervention) Ages 9–17 years |
Program to improve organizational climate and culture vs. usual care | Educational meetings or materials Educational outreach visits Provider satisfaction initiative Audit and feedback |
Details NR but does not demonstrate improvements in any measure of adherence by strategy group for any ARC vs. no ARC comparison | Low for no benefit for practitioner adherence | Medium study limitations, precise results |
| Difference in out-of-home placements and child behavior problem scores at 18 months between ARC-only and usual-care groups did not meet statistical significance (p = 0.05). | Low for no benefit for patient change in mental health symptoms at 18 months | Medium study limitations, precise results (small sample size), CIs likely overlap | ||||
| Program to improve organizational climate and culture Glisson et al. [51, 60] Cluster RCT 352 caregivers of youth ages 5–18 in 18 programs |
General mental health problems Ages 8–24 years |
Program to improve organizational climate and culture vs. usual care | Educational meetings or materials Educational outreach visits Provider satisfaction initiative Audit and feedback |
Trends toward improvement in all domains; nonoverlapping CI for some domains showing significant improvements (p < 0.05) for ARC group vs. usual care | Low for benefit for practitioner satisfaction | Medium study limitations, imprecise results (small study sample) |
| Lower problem behavior scores for youth in the ARC group compared with those in the control group during first 6 months of follow-up (following 18-month organizational implementation), effect size = 0.29 | Low for benefit for patient change in mental health symptoms | Medium study limitations, imprecise results (small study sample) | ||||
| Nurse training to implement an EBP Gully et al. [55] Interrupted time series in Study 1, 172 parents or caregivers; RCT in Study 2, 51 parents or caregivers |
General mental health symptoms (children suspected of abuse during forensic medical examinations) Ages 2–17 years |
Protocol to train nurses to educate parents about EBPs vs. typical services | Educational meetings or materials Educational outreach visits Patient-reported data |
Strategy improved parent ratings of access to care (mean difference between groups ranged from 0.08 to 2.1 points in Study 1 and 0.6 to 1.9 in Study 2) (scale = 1–5) | Low for benefit for patient access to care | High risk of bias, consistent, direct, precise results |
| Improved parent ratings of satisfaction of care by a mean of 0.4 in Study 1 and 0.9 in Study 2 (scale = 1–5) | Low for benefit for patient satisfaction | High risk of bias, consistent, direct, precise results | ||||
| Improved parent ratings of treatment engagement by a mean of 0.9 in Study 1 and 2.5 in Study 2 (scale = 1–5) |
Low for benefit for treatment engagement | High risk of bias, consistent, direct, precise results | ||||
| Improved parent ratings of therapeutic alliance by a mean of 0.4 in Study 1 and 0.9 in Study 2 (scale = 1–5) |
Low for benefit for therapeutic alliance | High risk of bias, consistent, direct, precise results | ||||
| Intensive quality assurance to implement an EBP Henggeler et al. [54] Controlled clinical trial, 30 practitioners, N of caregiver and patient reports and monthly data points NR |
Substance use disorders (adolescents with marijuana abuse) Ages 12–17 years |
Intensive Quality Assurance (IQA) system vs. workshop only to implement an EBP intervention | Quality monitoring | Study does not provide sufficient detail to judge magnitude of effect on practitioner adherence to cognitive behavioral therapy and monitoring techniques | Insufficient for practitioner adherence and fidelity | High study limitations, imprecise results |
| Training through workshop and resources to implement an EBP Henggeler et al. [59] Cluster RCT; 161 therapists |
Substance use disorders Ages 12–17 years |
Workshop and resources (WSR) vs. WSR and computer-assisted training (WSR + CAT) to implement an EBP intervention | Educational meetings or materials | No statistically significant difference between groups for use, knowledge, and adherence | Insufficient for additional benefit of WSR + CAT vs. WSR comparison group for practitioner use, knowledge, and adherence | Medium study limitations, imprecise, small sample sizes, cannot determine whether CIs cross line of no difference |
| WSR vs. WSR + CAT and supervisory support (WSR + CAT + SS) to implement an EBP intervention |
Educational meetings or materials Educational outreach visits |
No statistically significant difference between groups for use, knowledge, and adherence | Insufficient for additional benefit of WSR + CAT + SS vs. WSR comparison group on practitioner use, knowledge, and adherence competence/skills | Medium study limitations, imprecise, small sample sizes, cannot determine if CIs cross line of no difference | ||
| Professional training to identify and refer cases Lester et al. [48] Cluster RCT; 110 practices, 179 patients |
Psychosis (adolescents and adults with first-episode psychosis) Ages 14–30 years |
Professional training to identify and refer cases vs. usual care | Educational meetings or materials Educational outreach visits |
Relative risk (RR) of referral to early intervention after first contact: 1.20, 95% CI, 0.74 to 1.95, p = 0.48 | Insufficient for patient access to care | High study limitations, imprecise results |
| No statistically significant differences between groups in changes in patient mental health status | Insufficient for patient change in mental health symptoms | High study limitations, imprecise results | ||||
| Patients in the professional training group averaged 223.8 fewer days for time from the first decision to seek care to the point of referral to an early intervention service than patients in the control group | Low for benefit for service utilization | High study limitations, imprecise results | ||||
| No adverse events were reported, no significant between-group differences for false-positive referral rates from primary care | Insufficient for patient harms | High study limitations, unknown precision | ||||
| Professional training plus feedback Lochman et al. [57] Cluster RCT, 511 patients |
Externalizing behaviors (children at risk for aggressive behaviors) Ages: third-grade students |
Professional training plus feedback (CP-TF) to implement an EBP intervention vs. control | Educational meetings or materials Audit and feedback |
Students in CP-TF group had fewer behavioral problems as rated by teachers (beta = −0.41, SE = 0.16, p = 0.01) than controls but no significant difference in teacher ratings or parent ratings | Low for no benefit for changes in mental health status | Medium study limitations, precise results |
| Students in CP-TF group had fewer minor assaults (e.g., hitting or threatening to hit a parent, school staff, or student) as reported by the child (beta = −0.25, SE = 0.12, p = 0.03) and social/academic competence as reported by the teacher (beta = 0.35, SE = 0.13, p = 0.01) compared with controls | Low for benefit for change in socialization skills and behaviors | Medium study limitations, precise results | ||||
| Professional training only to implement an EBP intervention (CF-BT) vs. control | Educational meetings or materials | No significant difference in behavioral problems as rated by teachers or parents or student-reported assaults between CP-BT and control groups | Low for no benefit for changes in mental health status | Medium study limitations, precise results | ||
| No significant differences in social/ academic competence as reported by the teacher, nor were any significant differences found between groups on social skills as rated by parents. | Low for no benefit for change in socialization skills and behaviors | Medium study limitations, precise results | ||||
| Medication monitoring therapy Ronsley et al., 2012 [49] Interrupted time series Health care practitioners for 2376 patients |
Psychosis Ages <19 years (mean age = 11) |
Patient medication monitoring training program for practitioners vs. usual care | Educational meetings or materials Educational outreach visits Reminders |
38.3% of patients had a metabolic monitoring and documentation tool (MMT) in the charts after program implementation; drop in the prevalence of second-generation antipsychotic prescribing from 15.4% in the pre-metabolic monitoring training program (MMTP) period to 6.4% in the post-MMTP period (p < 0.001) | Low for benefit for practitioner adherence | High study limitations, precise outcomes |
| Increased metabolic monitoring over time (level of change varied by type of monitoring) | Low for benefit for patient service utilization | High study limitations, precise outcomes | ||||
| Staffing models to implement an EBP to screen, conduct a brief intervention, and refer adolescents with substance use to treatment from primary care settings Sterling et al. [58] Cluster RCT, 47 pediatricians with 1871 eligible patients |
Varied conditions among children attending a pediatric primary care office Ages 12–18 |
Pediatrician only vs. embedded behavioral health care practitioner (BHCP) implementation of an EBP | Multidisciplinary teams | No significant differences in substance use assessment between study arms (aOR, 0.93; 95% CI, 0.72 to 1.21); patients in the embedded BHCP group more likely than those in the pediatrician-only group to receive brief intervention (aOR = 1.74, 95% CI, 1.31 to 2.31); patients in the BHCP group less likely to receive a referral to a specialist than patients in the primary-careb only group (aOR = 0.58, 95% CI, 0.43 to 0.78) | Low for no benefit for practitioner adherence (2 of 3 adherence outcomes were statistically significant) | Medium study limitations, unable to assess precision |
| Co-location of a behavioral health EBP parenting program in primary care to help children with externalizing behavioral problems Wildman et al. [52] Controlled clinical trial, 4 pediatric practices, 20,917 children with primary care visit |
Externalizing behavior problems Ages 2–12 years |
Colocation of a behavioral health EBP parenting program in primary care vs. enhanced referral to a behavioral health EBP parenting program in a location external to the practice. | Changing the scope of benefits | OR for attending first EBP visit, 3.10; 95% CI, 1.63 to 5.89 | Low for benefit for patient access to care | High study limitations, precise results |
| No improvement in mean number of sessions attended (calculated mean difference: −1.01; 95% CI, −2.60 to 0.58) | Insufficient for patient service utilization | High study limitations, precise results | ||||
| Implementation of a school-based cognitive-behavioral group EBP Warner et al. [62] Stratified RCT 138 youth, 7 master’s level school counselors, 5 doctoral-level psychologists |
Social anxiety disorder. Adolescents in grades 9–11 from three suburban public high schools identified via school-wide screening, parent telephone screening, and clinical diagnostic evaluation with no other mental disorders of equal or greater severity. |
Implementation by a school counselor vs. by a psychologist | Changing provider | No significant differences in implementation adherence or competence. | Insufficient for practitioner adherence or competence | High study limitations, unknown precision for each intermediate outcome |
| No significant differences between groups for any of the severity or functioning scales at post-treatment or follow-up with the exception of 3 posttreatment outcomes (treatment response, treatment remission and social anxiety severity as rated by parents) where youth in the school counselors group did not do as well as those in the psychologist group when noninferiority was tested | Insufficient for patient change in mental health status | High study limitations, unknown precision for each intermediate outcome |
aFour study groups were examined: ARC + MST, ARC only, MST only, and usual care. Comparisons were ARC only vs. usual care or any ARC (combined ARC + MST and ARC only) vs. no ARC (combined MST and usual care), as noted
bFewer referrals seen as improvement because this outcome indicates that the practitioner was able to give brief intervention without referral to behavioral health specialists
ADHD attention deficit hyperactivity disorder, aOR adjusted odds ratio, ARC Availability, Responsiveness, and Continuity, CBT cognitive behavioral therapy, CI confidence interval, CP-TF Coping Power training plus feedback, EBP evidence-based practice, EHR electronic health record, IAU implementation as usual, IQA Intensive Quality Assurance, MMT metabolic monitoring program, MMTP metabolic monitoring training program, MST multisystemic therapy, N number, NR not reported, NS not significant, OR odds ratio, p probability, P4P pay for performance, RCT randomized controlled trial, RR relative risk, SE standard error, WSR workshop plus resources, WSR + CAT workshop plus resources plus computer-assisted training, WSR + CAT + SS workshop plus resources plus computer-assisted training plus supervisory support