The effectiveness of the Stop Now and Plan (SNAP) Program for boys at risk for violence and delinquency

Abstract

Among the available treatments for disruptive behavior problems, a need remains for additional service options to reduce antisocial behavior and prevent further development along delinquent and violent pathways. The Stop Now and Plan (SNAP) Program is an intervention for antisocial behavior among boys between 6 and 11. This paper describes a randomized controlled treatment effectiveness study of SNAP versus standard behavioral health services. The treatment program was delivered to youth with aggressive, rule-breaking or antisocial behavior in excess of clinical criterion levels. Outcomes were measured at 3, 9 and 15 months from baseline. Youth in the SNAP condition showed significantly greater reduction in aggression, conduct problems and overall externalizing behavior, as well as counts of Oppositional Defiant Disorder and Attention Deficit Hyperactivity Disorder symptoms. Additional benefits for SNAP were observed on measures of depression and anxiety. Further analyses indicated that the SNAP program was more effective among those with a higher severity of initial behavioral problems. At one year follow-up, treatment benefits for SNAP were maintained on some outcome measures (Aggression, ADHD and ODD, depression and anxiety) but not others. Although overall juvenile justice system contact was not significantly different, youth in SNAP had significantly fewer charges against them relative to those standard services. The SNAP Program, when contrasted with standard services alone, was associated with greater, clinically meaningful, reductions in targeted behaviors. It may be particularly effective for youth with more severe behavioral problems, and may result in improvements in internalizing problems as well.

Antisocial behaviors occurring prior to age 12 are particularly indicative of a risk for a chronic and increasing course of antisocial behavior over time (Tolan & Thomas, 1995). Early risk indicators are often evident in childhood, and for some youth develop into more severe problems as children grow (e.g. Burke, Waldman, & Lahey, 2010; Loeber & Hay, 1997). Interrupting these pathways is often difficult, and these youth are prone to poor outcomes, including contact with juvenile justice (e.g. Tolan & Gorman-Smith, 1998).

Evidence-based interventions to reduce antisocial behavior exist. Comprehensive reviews (Chorpita et al., 2011; Eyberg, Nelson, & Boggs, 2008) identify approximately two dozen intervention models with at least some empirical support. Parent management training (PMT; Patterson, Chamberlain, & Reid, 1982) is among the most empirically well-supported (Chorpita et al., 2011; Eyberg, Nelson, & Boggs, 2008) and is applicable broadly across ages. It focuses primarily on improving parenting skills and using behavioral principles to shape children’s behavior. Other treatment models include components targeted directly at youth, such as Problem Solving Skills Training (PSST; Kazdin, 2003), and there is support for the combination of PMT and PSST (Kazdin, Esveldt-Dawson, French, & Unis, 1987). Additional evidence-based programs include Parent-Child Interaction Therapy (PCIT; Brinkmeyer & Eyberg, 2003) and Multisystemic Therapy (MST; Henggeler & Lee, 2003).

Despite the fact that these interventions exist, two related problems remain regarding the ability to reduce serious violence in adolescence. First, preventing the development of violence and serious antisocial behavior in adolescence is preferable. This is true in terms of reducing the societal costs that accrue over the life course of those with early onset of antisocial behavior, and in terms of the potential for interrupting transitions from early antisocial behavior to chronic and serious adolescent and adult offending (Loeber, Farrington, Stouthamer-Loeber, & White, 2008). Secondly, most youth with significant behavioral problems never become involved in mental health services and instead often end up involved in the juvenile justice system (Burke, Mulvey, Schubert, & Garbin, in press; Stouthamer-Loeber & Loeber, 2002). Even when children show aggression that should serve as an “action sign” for treatment, these children and their caregivers rarely become engaged in appropriate services (Jensen et al., 2011).

As noted by Offord, Lipman, & Duku (2001), successful prevention programs have two prerequisites: the ability to identify high risk groups and the ability to deliver an effective prevention program to this group. On the first, evidence supports the ability to identify a group of children at high risk due to both the severity and diversity of their behavior in late childhood. For example, Offord et al. (2001) reported that children showing aggression in late childhood were 7 times more likely to persist in early adolescence. Further, prediction was enhanced by considering high severity along several dimensions of antisocial behavior (Offord, et al., 2001).

On the second prerequisite, that of delivering an effective prevention program, several challenges remain. The aforementioned evidence-based programs are not universally available. Gaps in the applicability of those treatment models across development means that not all available programs are equally appropriate for all children. Some are more appropriate for early to middle childhood, such as PCIT (Brinkmeyer & Eyberg, 2003) or the Incredible Years program (Webster-Stratton & Reid, 2003). MST (Henggeler & Lee, 2003) on the other hand is designed for adolescents with severe behavioral problems (e.g. Olsson, 2010).

Thus, the prevention of serious violence and antisocial behavior is often inhibited by problems with early identification and problems with linkages to appropriate services. If early red flags denoting risk for serious violence or aggression do not lead parents and children to engage in services, progression along notably undesirable developmental pathways may proceed unchecked. The SNAP (Stop Now and Plan) Program (Augimeri, Farrington, Koegl & Day, 2007) was developed with the specific intention of reducing the gap between identification and service engagement for at-risk youth in late childhood in the community.

SNAP is a manualized multi-component program aimed at reducing antisocial behavior among children ages 6 to 11. Implementation of the program includes efforts to work with local police, juvenile justice, school and other community agencies to encourage the recognition of early risk for serious violence and delinquency and to provide a common referral target when such problems are seen. It was designed in particular for youth who come into contact with the police, but who are yet under the age of criminal responsibility. Eligibility for SNAP is based on either a referral from the police or on a clinically significant score on one of four behaviorally-oriented Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001) subscales. Where parents, teachers, child service workers or others coming into contact with a child have concerns about antisocial or aggressive behavior, they are encouraged to make a referral to SNAP.

The SNAP Program model was initially developed in the 1970’s and was formalized in 1985. It was created to incorporate sound theoretical approaches, such as social skills training, problem solving, self-control and anger management strategies, cognitive self-instruction, family management skills training, and parent training. SNAP also incorporated aspects of the Oregon Social Learning Center’s (OSLC) Social Interactional Family Therapy (Patterson, Forgatch, & DeGarmo, 2010). For details regarding the SNAP program and the history of its development, see Augimeri, Walsh, Levene, Sewell, & Rajca (2014) and Augimeri, Walsh, & Slater (2011).

SNAP includes several distinct components. During the first 12 weeks, separate parent and child group-based modules are provided on a weekly basis. Children are taught cognitive and behavioral skills and are given structured practice experiences to apply these skills to specific circumstances. Topics include stealing, coping with anger, and managing group pressure. Each group session includes structured elements of role-play, problem solving and peer feedback to evaluate alternative solutions and the likelihood that a particular solution will lead to improved or to poorer outcomes. Parents are led in educational content, and discuss with other parents their use of parenting strategies and their efforts at coping with their own emotional reactions. Subsequent to the group treatment component, children are provided with an array of SNAP components based on their individual needs. Among these components are SNAP family counseling, booster sessions, academic tutoring, school advocacy, and mentoring.

Improvement associated with SNAP has been demonstrated in several investigations, including in comparison to a delayed treatment or wait-list control groups (Augimeri, Farrington, Koegl & Day, 2007; Lipman et al., 2008). A study of a variation of SNAP (Augimeri, Jiang, Koegl & Carey, 2006) found differential effects based on initial severity. For low-delinquency boys, both the standard and enhanced versions of the intervention led to equivalent improvement in delinquent behavior. For those with moderate or severe levels of delinquency, the enhanced intervention was associated with greater improvement than the standard program.

There have been no large scale randomized comparison-controlled studies of outcomes associated with SNAP, and much of the existing literature on SNAP has been generated by the original development team. The present study was an effectiveness study of SNAP, conducted independently from the SNAP originators. Further, local SNAP providers were authorized providers of SNAP and participated in ongoing assessment of their fidelity to the model, but otherwise operated independently themselves. As a result, this project represents an evaluation not only of the effectiveness of the program, but also of its transportability. This evaluation employed an intent-to-treat, randomized controlled design. Youth were randomly assigned to receive SNAP or standard services in the community and were retained in the analyses regardless of degree to which they subsequently participated in services.

The hypotheses for the present study were: 1) Youth participating in the SNAP Program would show greater reduction in aggressive and conduct problem behaviors in comparison to youth in the standard service condition; 2) The effects for the SNAP group treatment component would hold after controlling for differences in demographic factors and measures of intelligence and for measures of additional services used (including pharmacotherapy); 3) The effects of the group treatment relative to the standard service condition would be similar for youth with more severe initial problems in contrast to youth with lower severity at enrollment.

Method

Sample

Participants

A total of 252 boys participated in the study. Girls were not enrolled since only the SNAP boys’ model of the program was available locally. Families were disproportionately of lower income; 50% of the sample had a household income below $15,000, whereas 14% reported an income above $33,201. This was likely due to the study eligibility requirements and to the higher level of contextual risk factors for youth with high-level behavioral problems. By parental report, 76% identified as African American, 13% as White, and 10% using more than one racial category. The mean age of the boys was 8.5 (sd = 1.8) years. The average IQ was 91.6 (sd=12.5), and 14.7% (n=38) had parent-reported police contact due to the youth’s behavior. The CONSORT diagram in Figure 1 details the flow of participants into the study, through random assignment and through follow up assessments.

Figure 1.

Flowchart of participant enrollment and study participation.

Recruitment

Participants were recruited from new referrals to each of the two SNAP program providing agencies in the area. Parents may have been referred to the SNAP service by police or teachers, or may have been made aware of the service through local advertisement. Parents calling about the SNAP program were informed of the study. Those expressing interest were told that study participation would involve a random chance of participating in SNAP or standard community services as usual (STND). After being given basic study information, approximately 30% of parents declined further study contact. The most common reasons for doing so were an unwillingness to be randomly assigned to treatment or already being involved in other behavioral health services for the child at the time. Interested parents were contacted by research team staff, who provided more details about would be required of families during their participation in the study. Some parents declined participation at that point due to the intensity of the expectations for both parent and youth participation in treatment.

Eligibility

Participants had to have an IQ greater than 70, and a qualifying behavioral score via parent report (CBCL) or teacher report (Teacher Report Form, Achenbach & Rescorla, 2001) of Aggressive Behavior (T score greater than or equal to 70), Rule Breaking (70), DSM Conduct Problems (70) or Externalizing Behavior (T score greater than or equal to 64).

SNAP services are of somewhat greater intensity than typical individual outpatient services. We thus endeavored to involve, in the STND condition, as many higher intensity community services as possible. Wraparound services (wrap) were considered to reflect the highest intensity service in the area since it involves multiple team members providing often 10 or more service hours per week. Because wrap is typically paid for via medical assistance (MA), study participants were required to have, or to be eligible for, MA.

Study participants could not already be engaged in SNAP or wrap, but were not excluded if they were engaged in less intensive services or pharmacotherapy. Participants were informed that if after random assignment to STND, they decided to engage in SNAP, or if they engaged in wrap services after assignment to SNAP they would not be retained in the study. As an intent-to-treat study, once assigned to condition, participants remained in the study, regardless of their actual level of participation in the treatment to which they were assigned.

Random assignment

Upon signing consent and meeting eligibility requirements, participants were randomly assigned to study condition. Randomization was performed by the study investigators independently of the treatment providers using a random number generating computer program. Table 1 illustrates the equivalence between groups on demographic factors, IQ and baseline behavioral problems. No significant differences were found.

Table 1.

Demographic and Baseline Measures by Treatment Groups

	SNAP	Standard Service
N	130	122
Age	8.9 (1.9)	9.0 (1.8)
IQ	91.87 (13.1)	91.88 (13.7)
African American	86.2% (n = 112)	87.7% (n=107)
Income	Between $10,000 and $14,999	Between $15,000 and $19,999
Police Contact	13.8% (n=18)	16.4% (n=20)
	Baseline Behavioral Measures
CBCL Aggressive Behavior	79.1 (sd = 9.6)	79.3 (sd = 9.5)
CBCL Rule Breaking	72.4 (sd = 6.1)	72.3 (sd = 7.2)
CBCL DSM Conduct Problems	76.0 (sd = 6.8)	76.8 (sd = 7.4)
CBCL Externalizing	75.3 (sd = 4.9)	75.4 (sd = 5.2)
CBCL Internalizing	63.4 (sd =9.2)	64.3 (sd = 8.9)
CBCL Withdrawn-Depressed	64.1 (sd = 9.4)	65.4 (sd = 10.4)
CBCL Anxious-Depressed	62.7 (sd =8.6)	62.9 (sd = 9.1)
CBCL Somatic Complaints	58.2 (sd =7.7)	59.7 (sd =8.3)
CSI-4 ADHD symptoms	7.77 (sd = 5.6)	8.26 (sd = 5.2)
CSI-4 ODD symptoms	3.68 (sd = 2.7)	4.16 (sd = 2.6)
CSI-4 CD symptoms†	3.17 (sd = 2.6)	3.83 (sd = 2.4)
CSI-4 Depression symptoms	1.99 (sd = 2.6)	2.27 (sd = 2.8)
CSI-4 Separation Anxiety symptoms	0.65 (sd = 1.13)	0.46 (sd = 0.78)
CSI-4 General Anxiety symptoms	0.84 (sd = 1.25)	0.91 (sd = 1.33)

Note: No tests of significant differences between groups were significant at p = .05. ODD = oppositional defiant disorder, ADHD = attention deficit hyperactivity disorder, CD = conduct disorder, DSM = diagnostic and statistical manual, CBCL = child behavior checklist, CSI = child symptom inventory.

^†p < .10.

In the case of siblings seeking services, since the parent component of SNAP is essential, it would not have been possible to separate siblings across groups while preventing any possible effect of the SNAP parenting component on the STND group sibling. We thus randomized parents to group, rather than siblings and did not exclude siblings from participation. Of the total sample, 82.9% (n=175) were non-siblings, and 17.1% (n=77) were siblings, in 36 sibling clusters. There were 41 siblings in SNAP and 36 siblings in STND; the difference was not significant (χ² = .12, p = .72). Analyses accounted for nested observations among siblings.

SNAP treatment

Participants assigned to SNAP services were referred to the closest providing agency. Treatment fidelity was monitored by staff of the Child Development Institute in Toronto. Fidelity checks also included local observation from study team members. Fidelity was rated on the proper presentation of treatment content within the scheduled activities for each of six treatment segments of between 10 to 25 minutes each. Between 5 to 12 elements were rated for fidelity across each segment, and during preparation prior to each session. Adherence to specific SNAP treatment protocols was at least 92% or greater.

Standard services

Participants in STND received assistance from project staff in their efforts to engage in services, including initial referrals for wrap. As often occurs, parents experienced a number of barriers to treatment (e.g. appointment availability, travel issues, time commitments, parental motivation). Despite the severity of behavioral problems shown by these boys, clinical evaluations conducted by providers in the community did not always result in recommendations for wrap. Other service options included typical individual outpatient services provided by a psychiatrist, psychologist or social worker, group treatment, or other mental health services provided at community clinics or in private practices in the region.

Data collection

Interviews were administered using a laptop computer by trained research interviewers. All training and oversight of the interview and data collection activities for this study was conducted by research project staff independent of SNAP service providers. Participants were compensated for participating. Interviews were usually conducted in family homes, although office interviews and alternate locations were employed at family request. All study procedures were approved and monitored by the local Institutional Review Board.

Measures

The CBCL (Achenbach & Rescorla, 2001) was used at each wave. Test-retest reliabilities and internal consistency alphas (Achenbach & Rescorla, 2001) are given respectively in parentheses for each subscale. The primary outcomes were: Total Externalizing (EXT, .92, .94), Aggression (AGG, .90, .94), Rule Breaking (RB; .91, .85) and Conduct Problems (CP; .93, .91). These subscales are not independent of one another; EXT includes the items within the AGG and RB subscales. CP draws items from across subscales as a proxy for DSM CD, although it excludes some CD symptoms and includes other non-CD behaviors. The present analyses also examine CBCL Total Internalizing (INT; .91, .90), Withdrawn-Depressed (WD; .89, .80), Anxious-Depressed (AD; .82, .84) and Somatic Complaints (SC; .92, .78).

The Child Symptom Inventory-4: Parent Checklist (CSI-4; Gadow & Sprafkin, 1994) is a parent self-administered symptom checklist. Test-retest reliabilities and internal consistency alphas (Sprafkin, Gadow, Salisbury, Schneider & Loney (2002) are given respectively in parentheses for symptom count scores for each subscale. The present analyses examined symptom counts of ADHD (.70, .87), CD (.53, .73), ODD (.78, .86), depression (.68, .63), separation anxiety (SEP; .57, .69) and general anxiety (GEN; .53, .69).

The Earlscourt Family Information Form (Earlscourt Child and Family Centre, 2001) elicits information concerning ethnicity, marital status, SES, and parent education. The Kaufmann Brief Intelligence Test – 2 was administered at baseline to obtain an estimate of IQ. Internal consistency for the full composite score ranged from .90 to .93 across 6 to 11 year olds, and test-retest correlations for 4 to 12 year olds was .88 (Kaufman & Kaufman, 2004).

The Child and Adolescent Services Assessment (CASA; Ascher et al., 1996) queries a wide range of potential sources of help for behavioral or emotional problems, from inpatient facilities to family members or friends. Parents are asked whether help was sought from a given source in the child’s lifetime and during the past three months in particular, and whether medication was prescribed. Test-retest reliability kappas ranged across types of service settings, from .47 for non-mental health professional services to 1.0 for juvenile justice services. The intraclass correlation coefficient (ICC) for the total number of service settings was .76. A check of the validity of the measure found an 84% consistency in parent report of any service use against mental health center records.

Official criminal records were obtained from the county department of juvenile probation. The age of criminal responsibility in Pennsylvania is 10. There were 150 youth (80 SNAP and 66 Standard Service) who would have been at the age of criminal responsibility by the point that records were most recently requested. Records were not obtained where parents opted out of providing consent for this aspect of the data collection.

Statistical Analyses

Consistent with the intent-to-treat design of the study, participants were retained in the analysis regardless of their level of service participation. Because SNAP included group services, whereas STND consisted of individualized treatment, observations were partially nested (Bauer, Sterba, & Hallfors, 2008). The analytic strategy had to account for the possibility that SNAP scores may have been correlated due to unique aspects of their group experience, distinct from youth in other groups or in individual treatment. To account for this, a multilevel mixed modeling strategy was used (Bauer, et al., 2008; Rabe-Hesketh & Skrondal, 2008; Roberts & Roberts, 2005). Nesting within groups was modeled; youth in STND were treated as their own treatment group. To minimize confusion between treatment group participation and treatment condition, we will use the term block to refer to treatment group clustering. A random effect for the slope of treatment condition at the block level was modeled, with ICC estimates describing the degree to which clustering at each level contributes to the model.

Youth were also nested in sibling clusters, and observations by wave were nested within individuals. The distribution of the outcomes was modeled as appropriate (e.g. normal versus Poisson for symptom counts). Analyses were conducted using Stata (StataCorp, 2009). The analyses exclude observations at baseline, since the groups were randomized to be equivalent at that time. The analyses test group differences at the 3, 9 and 15 month follow-up time points, and time was coded as 0, 3, 9 and 15. Effect size estimates were determined by standardizing the outcome for continuous variables, which yields a value equivalent to Cohen’s d when the predictor is dichotomous (Lipsey & Wilson, 2001). For count variables, the exponentiated beta value (or the incidence rate ratio (IRR)) for the treatment parameter was used. The IRR indicates the relative proportional difference in the outcome for each increasing unit of the predictor.

Missing data

Multilevel modeling is flexible regarding missing data, making it well-suited for intent-to-treat analyses. We used maximum likelihood estimators, which provide advantages in handling missing data (Allison, 2012). We also tested models of AGG generated through multiple imputation (StataCorp, 2009) and found no notable differences. As a result, we did not use multiple imputation for the results presented here.

Results

SNAP service use

Of those assigned to the SNAP group, during the first 3 month period, children attended an average of 6.25 (sd = 4.3) of the 12 child sessions, and parents attended an average of 5.02 (sd = 4.2) of the 12 parent sessions. Of the 130 children assigned to SNAP, there were 30 children (23.1%) who attended no child SNAP groups, 37 parents (28.5%) who attended no parent SNAP groups, and 29 parent-child dyads in which neither attended any groups (22.3%). As this evaluation was initiated as an intent-to-treat study, all participants were retained in the analyses after randomization, regardless of their level of service use.

Subsequent to participating in the group treatment component of SNAP, youth received individual SNAP components as determined by protocol. Of all families in SNAP, 70.0% received at least one component; the maximum number of different components used was 4. The most common was Individualized Family Intervention, which was provided to 68 participants (52% of the SNAP group), who received an average of 4.12 (sd = 6.9) sessions. Individual Befriending was provided to 64 participants (49%), who received a mean of 2.5 (sd = 4.2) units. School Advocacy was used by 23 (18%) participants, who received between 0 and 11 units (mean = .43, sd = 1.35). Other SNAP components used by fewer than 10% of the SNAP families included academic tutoring, crisis counseling, and Leaders in Training.

Standard services group

Over the course of the study, 53% of those in the STND group had engaged in services through wrap, specialty behavioral health or school-based behavioral or emotional services. Regarding wrap specifically, by the three month follow up, 16 of the 122 assigned to the standard service condition (13.1%) had been engaged in wrap. They reported using a mean of 7.9 units of wrap (sd = 10.65), ranging from 0 to 31 units of service. Over the course of the project, the rate of engagement in wrap increased such that by the final assessment, 43 standard service participants (35%) had participated in wrap services.

The use of lower intensity professional mental health services (e.g. outpatient mental health, pharmacotherapy services) was equivalent for those in SNAP versus STND. At baseline, 17% of youth in each condition were involved in other behavioral health services, and during waves 2 through 4, 23% of youth in each condition were involved in any such services. Regarding school-based services for behavioral or emotional problems, more youth in STND than SNAP were involved in such services at baseline (40% vs. 21%). Over waves 2 through 4, 36% of youth in SNAP and 31% of youth in STND were involved in school-based services.

Prescribed medication use

A relatively low number of youth received medication at each wave. Only 12 youth in SNAP and 7 in STND reported medication use at baseline. Over time, more youth in STND were prescribed medications (10 at wave 2, 17 at wave 3 and 12 at wave 4), whereas 8 SNAP youth were reportedly prescribed medications at waves 2 and 3, and 6 at wave 4. Prescribed medications were almost exclusively stimulant medications for ADHD.

Preliminary Tests of Partial Nesting at the Treatment Condition Level

Multilevel mixed models were used to test for partial nesting within treatment block. Random slope coefficients for treatment condition were specified at the block level, and random intercepts were specified at the level of sibling cluster and of individual within time. Fixed effects for treatment condition, wave and age were modeled as predictors of outcomes at waves 2 through 4. For each outcome, the ICC for the block level was small (AGG ICC = .006, 95% confidence interval (CI) = .000 to .065; RB ICC = .005, CI = .000 to .073; CP ICC = .012, CI = .000 to .100; EXT ICC = .007, CI = .000 to .075). Since these values were not significantly different from 0, there was no evidence for systematic association among observations within treatment blocks. Thus, the rest of the analyses excluded treatment block as a level in the model.

Primary Behavioral Outcomes

Of primary interest for this study were the effects of treatment condition on the CBCL scales of AGG, RB, CP and EXT. Outcomes at waves 2 through 4 were tested using multilevel mixed models, with random intercepts for sibling cluster and for individuals, and random slope for time. Fixed effects in the models included treatment condition and month. In addition, fixed effects for the covariates of race, age, income, IQ score, and police contact were tested. A process of stepwise removal within covariates was employed, while treatment group and month were retained in all models. Table 2 shows the full model for AGG. The fixed effect of treatment group indicates that those in SNAP showed significantly lower AGG scores at waves 2 through 4 (B = −3.33, se = 1.31, p = .011). The covariate IQ was retained in the final model, with higher IQ being associated with lower AGG scores. The effect size, Cohen’s d, determined using the coefficient for treatment condition when predicting standardized AGG scores, was .29.

Table 2.

Mixed Regression Model Predicting Aggressive Behavior at Waves 2 through 4

Outcome	B	se	p	95% CI
Fixed Effects
SNAP Treatment Condition	−3. 33	1.31	.011	−5.92	−0.75
Month	−.24	.05	<.001	−0.35	−0.13
IQ	−.13	0.05	.004	−0.22	−.04
Constant	85.87	4.33	<.001	77.39	94.36

Random Effects
	estimate	se	95% CI
Sibling Cluster, intercept	6.10	1.04	4.37	8.52
Individual, intercept	5.83	1.03	4.12	8.25
Time, slope	0.24	.11	0.09	0.60

Note: IQ = KBIT full scale IQ estimate; CI = confidence interval. Parameter values indicate outcomes for SNAP in contrast to the standard service group. Beta coefficients are unstandardized beta values.

A similar strategy was used for RB, CP and EXT. The parameter values for treatment condition (Table 3) indicate significantly lower scores in SNAP after baseline on CP and EXT, but not on RB. Covariate parameters are not shown due to space limitations. It is worth noting that across these latter three outcomes, higher IQ scores were consistently associated with lower behavioral problems, and a history of police contact with higher behavioral problems. Figure 2 shows the observed mean scores on four behavioral outcomes by treatment group.

Table 3.

Fixed Effects of SNAP Treatment in Separate Models Predicting Outcomes at Waves 2 through 4

CBCL Outcomes	B	se	p	95% CI		Cohen’s d
CBCL Rule Breaking	−1.09	.99	0.270	−3.03	0.84	.13
CBCL Conduct Problems	−2.38	1.09	0.030	−4.51	−0.24	.25
CBCL Externalizing	−2.57	0.98	0.008	−4.49	0.66	.31
CBCL Internalizing Total	−3.12	1.25	0.010	−5.57	−0.66	.29
CBCL Withdrawn-Depressed	−2.71	1.16	0.020	−4.99	−0.42	.27
CBCL Anxious-Depressed	−2.51	0.86	0.003	−4.18	−0.83	.30
CBCL Somatic Complaints	−0.40	0.76	0.600	−1.88	1.09	.06

Symptom Count Outcomes	B	se	p	95% CI		IRR e.s.
CSI – ADHD	−0.67	0.11	<0.001	−0.88	−0.46	48%
CSI – ODD	−0.33	0.12	0.006	−0.57	−0.09	28%
CSI –Conduct Disorder	−0.22	0.13	0.080	−0.47	0.03	20%
CSI – Depression	−0.47	0.20	0.022	−0.86	−0.07	37%
CSI – Separation Anxiety	−0.77	.024	0.001	−1.23	−0.31	54%
CSI – General Anxiety	0.13	0.22	0.565	−0.31	0.56	13%

Note: CBCL = Child Behavior Checklist; CSI = Child Symptom Inventory; CI = confidence interval; ADHD = attention deficit hyperactivity disorder; ODD = oppositional defiant disorder; IRR e.s. = Incidence rate ratio based effect size. Parameter values indicate outcomes for SNAP in contrast to the standard service group. Beta coefficients are unstandardized beta values. Parameter values reflect the inclusion of significant covariates in a given model. Cohen’s d was determined as the treatment coefficient from each model when the outcome variable was standardized. Incidence rate ratio based effect sizes are the estimated proportional reduction in symptoms for the SNAP group relative to the standard service group based on the exponentiated beta value.

Figure 2.

Change over time for SNAP versus Standard Service participants on primary measures of antisocial behavior.

Outcomes for Youth with Higher, More Diverse Behavioral Problems

Initial concerns were expressed locally that program would not be sufficient to meet the needs of youth with higher severity of behavioral problems. In order to test this, and to see which youth may be most appropriate for SNAP, we identified youth with clinical level scores on all three of RB, AGG, and CP at enrollment. There were 85 (65.4%) such youth in the SNAP group and 84 (68.9%) in the standard service group. For youth with severe behavioral problems at baseline, the effects of SNAP were markedly more pronounced. Compared to STND, those in SNAP showed significantly lower AGG (B = −4.85, se = 1.53, p = .002), RB (B = −2.96, se = 2.82, p = .005), CP (B = −4.96, se = 1.18, p < .001) and EXT (B = −4.12, se = 1.01, p < .001).

Other Behavioral Concerns

Diagnostic symptom counts

Controlling for significant covariates, the SNAP group showed lower ODD and ADHD symptom counts, with differences in CD symptom count falling short of the criterion for significance (see Table 3). When considering the effect of SNAP for only the aforementioned youth with initial severe behavioral problems, the significant benefit for SNAP was further increased for ODD and ADHD. The effects for CD for SNAP were particularly improved. For these youth, SNAP was associated with a significant reduction in CD symptoms over waves 2 through 4 (B = −.42, se = .11, p < .001), compared to STND, rather than the trend-level difference for the full sample.

Controlling for Other Specialty Behavioral Health Services and Medication Use

For the primary behavioral outcomes, we examined the effects of controlling for the use of other services (excluding SNAP or wrap) and medications at waves 2 through 4 as reported by parents on the CASA. There were no meaningful changes on the effect of treatment condition.

Mood, Anxiety and Other Outcomes

In addition to the behavioral outcomes of primary interest, the CBCL and CSI included measures of affect and other outcomes. Significant treatment condition differences, favoring SNAP, were found for the CBCL subscales of INT, WD and AD, but not for SC (see Table 3). Significant differences were also found on the CSI measures of depression and SEP symptoms, but not for GEN (see Table 3). In none of these models did the inclusion of covariates of income, race, age, IQ, history of police contact or medication or specialty service use meaningfully alter the effect of treatment condition.

Differences over the Follow Up Period

Figure 2 suggests that the bulk of the change in the SNAP group relative to the STND group occurred over the initial three month period, during which those in the SNAP group were participating in parent and child group sessions. In each model above, we tested the interaction between treatment condition and month over waves 2 through 4. No significant effects were observed. No quadratic terms for month, nor interactions between quadratic terms and treatment condition were evident in any models. These results suggesting that the treatment group differences generally hold across the three month to one year follow up.

To further probe the maintenance of treatment effects over time, we examined models including outcomes at only the six month and one year follow up points. For most outcomes, no substantial differences were observed. However, two CBCL outcomes were reduced to non-significance: CP (B = −2.00, se = 1.24, p = .10) and EC (B = 1.73, se = 1.13, p = .13). Also, the two anxiety symptom count constructs were reduced to non-significance: separation anxiety (B = −0.47, se = .37, p = .20) and general anxiety (B = −0.19, se = .26, p = .48).

Post-hoc Service Comparisons

SNAP versus wraparound

The intent-to-treat analyses provide important information about treatment effectiveness. It is nevertheless of interest to look at a direct comparison of treatment for youth involved in wrap versus SNAP, and for all youth involved in any behavioral services. There were 42 youth (34%) in STND who were engaged in wrap during the course of the study, and 111 (85%) in the SNAP condition who received any SNAP services during the study. When the analyses are restricted to only these youth, the differences between SNAP and STND were not significant on AGG (B = −3.32, se = 1.89, p = .08), CP (B = −0.54, se = 1.6, p = .74), and EXT: (B = −1.48, se = 1.46, p = .31). Youth in the SNAP condition did still show significantly lower ADHD (B = −0.20, se = .09, p = .03) and depression (B = −0.94, se = .16, p < .001) symptom counts than youth who participated in wrap, and showed significantly lower CBCL INT (B = −3.97, se = 1.72, p = .02) and AD (B = −3.26, se = 1.19, p = .006) as well.

Effects among users of any services

When restricting the data to study participants who used any services (SNAP, WRAP, specialty behavioral health or school-based behavioral or emotional services), 85% of those in SNAP and 53% in STND were included. Among the subset of any service users, those in the SNAP condition had significantly better outcomes on AGG (B = −4.65, se = 1.44, p = .001), CP (B = −3.21, se = 1.24, p = .01), and EC (B = −3.71, se = 1.09, p = .001), and were marginal for RB (B = −1.97, se = 1.14, p = .08). These positive effects held up over the 9 and 15 month follow up waves for AGG (B = −3.28, se = 1.64, p = .045) and CP (B = −3.21, se = 1.24, p = .01), and were marginal for EC (B = −2.36, se = 1.27, p = .06). Further, significant differences favoring SNAP over STND were found at the 3 month and across the 9 and 15 month follow-up for the CBCL outcomes of WD, IC, AD, and for the CSI measures of ODD, ADHD, CD, and depression. Further details are available upon request.

Official Criminal Records

Of the 150 youth over the age of criminal responsibility by the end of the study, 25 had contact with the county juvenile probation department after baseline. Their number of charges ranged from 1 to 7, and included contempt, failure to pay fines, terroristic threats, robbery and aggravated assault with injury. In STND, 15 (22.4%) had any charges, compared to 10 (12.2%) of those in SNAP, a difference that did not reach statistical significance (χ² = 2.74, p = .09). Nearly all of these youth had received some service for emotional or behavioral difficulties during the study period (9 of 10 in SNAP and 14 of 15 in STND). Those in STND had significantly more (b = 0.97, se = .49, 95% confidence interval = .01 – 1.93; IRR = 2.6) charges against them (mean = 0.61, sd = 1.47) than those in SNAP (mean = 0.23, sd = 0.85).

Discussion

This is the first randomized effectiveness trial of SNAP that allows for comparisons to alternative treatments. The comparison to alternative treatments in this study provides important information about the relative utility of SNAP within the larger array of services in a community. Youth in the SNAP condition showed lower aggression, conduct problems, and externalizing behavior than STND. These effects held after controlling for demographic factors, history of police contact and IQ. Contrary to the study hypothesis, no effect of the number of SNAP sessions was evident. Further, no evidence of any effect for partial nesting was found. It may be that the manualized, structured nature of the SNAP intervention reduced the degree to which one treatment group might have differed from another, but this is not clear in the present analyses.

On the primary behavioral outcomes of interest, significant group differences favoring SNAP were evident for three of the four CBCL behavioral subscales. For these constructs, at 3 months, youth in the SNAP group on average had T-scores below 70, meaning that in general they had already moved out of the clinical range. Youth in STND did not fall out of the clinical range on CP until the one year follow up, and remained at or above clinical cut-off on AGG throughout the follow-up period. Additionally, those in SNAP showed significantly lower ADHD and ODD across follow up assessments.

A question for the study was whether the program would be effective for severe behavioral problems. Among the youth who showed severe behavioral problems on rule breaking, aggressive behavior and conduct problem behavior, greater effects for SNAP in comparison to STND were found. Additionally, among these youth, the differences for rule breaking behavior were significant, in contrast to the full sample. Similar results were found for symptom counts of CD, ODD and ADHD. Thus, in contrast to concerns that the program would be of insufficient intensity for severe behavioral problems shown by youth, the program was in fact more effective in reducing the problems experienced by youth with more severe problems.

Variability in other behavioral outcomes

Although ADHD symptoms are not among the specific criteria for entry into the program, behavioral control is a key focus of treatment. Reductions in ADHD symptoms suggest that youth in the SNAP program are improving in their ability to interrupt impulsive behavioral processes and to exercise greater behavioral control in areas other than manifest aggression and conduct problems. The improvements in ADHD symptoms were not explained by medication usage among participants. The effectiveness of the program in improving behavioral self-control and reducing impulsivity is of course beneficial in and of itself. It may also enhance the degree to which the program helps to interrupt longer term developmental processes leading to antisocial behavioral problems later in adolescence.

Mood, anxiety and other outcomes

Significant differences were also found on several internalizing problems. These were evident on both CBCL scale scores and CSI measured symptom counts. SNAP, although targeted to achieve change in disruptive behavior, may have somewhat more global positive effects. It is possible that these effects flow from treatment components, or they may be secondary to improvements in other areas or to changes in family functioning. Further investigation to test putative mechanisms of treatment is needed.

Improvement at follow up

It was apparent that the greatest degree of improvement among SNAP youth relative to those in STND occurred over the first three months, during the time that youth and parents in SNAP were participating in group treatment activities. For the most part, significant differences held over follow up. Aggressive behavior, ODD symptoms, ADHD symptoms, depression symptom count, and the overall Internalizing and the Anxious-Depressed subscales showed persisting lower scores for youth in SNAP.

However, when the analyses excluded participants who participated in no behavioral or emotional services of any type, outcomes for those in SNAP were superior to those in STND across most outcomes, including behavioral and affective symptoms, and were observed through follow up. When further limited to only those who received wrap versus those who participated in SNAP, differences were more limited. Although these results are beyond the primary intent-to-treat analyses of the study, they do suggest that SNAP services are associated with a variety of superior outcomes through follow up over typical services, and achieve similar outcomes to a specialized program that is of higher intensity.

SNAP was originally designed for youth at risk for juvenile justice system involvement who were below the age of criminal responsibility. In the present study, slightly more than half of the sample was old enough to potentially have contact with juvenile probation by the end of the study. Although preliminary, the initial results suggest that SNAP may be more effective than STND at reducing juvenile justice involvement. In particular in the present data, fewer youth were involved with juvenile probation (albeit a trend-level, and not significant, difference), and significantly fewer charges were alleged against youth in SNAP in comparison to STND.

SNAP’s use of group treatment for youth at risk for delinquency might raise concerns about iatrogenic effects, or deviancy training. Such concerns are not supported by the present results. Other researchers have suggested that concerns regarding deviancy training have been overstated (e.g. Handwerk, Field & Friman, 2000; Weiss et al., 2005). Whether the absence of deviancy training here is due to a generally low level of actual risk for such processes or to specific factors regarding SNAP is not clear.

Service use

The observed differences between groups on behavioral outcomes were not explained by medication use or use of other services for behavioral problems. Apart from differences in SNAP and wrap, no differences in the number of services or units of service used were observed between groups. Service use in this sample was reflective of typical service use in this population, including the lack of participation in services by some participants despite expressing an interest and need for the services initially.

Also consistent with the typical nature of help-seeking for behavioral problems was the difficulty parents had engaging in wrap. Among those who were ultimately successful in doing so, only 37% were engaged in wrap within three months of initiating these efforts. This was true even with the additional assistance that research staff attempted to provide in connecting families with providers. One of the benefits of SNAP is that it provides a manualized and transportable service and may enhance the access to services for families in the community. This is particularly true when SNAP is implemented as a referral program for youth identified by contact with the police or for concerns of antisocial behavior.

Prevention

These findings suggest that SNAP, relative to other behavioral health interventions, is more effective at achieving initial reductions in high risk behaviors. Whether such longer term outcomes are indeed reduced cannot be addressed with this data. However, other ample evidence does support prevention as the most cost effective approach to reducing serious antisocial behavior in adolescence (e.g. Loeber et al., 2008). This may be particularly true where risk indicators are observed in late childhood (Loeber et al., 2008; Offord et al., 2001). The present results support SNAP as a potential means to interrupt such development where markers of risk are evident in middle to late childhood. SNAP has the particular advantage of being a manualized and structured program that can be successfully transported to a variety of locales and delivered with fidelity (Augimeri, Walsh & Slater, 2011). Its incorporation of a group treatment component may provide additional appeal for communities seeking to efficiently implement services for this particular population. From a policy and prevention standpoint, the SNAP model – both in terms of content and implementation - addresses two key elements of prevention. First, it provides intervention content needed to interrupt antisocial processes. Secondly, its design and conceptual underpinnings link early identification of problems with available services. It is intended to be a community resource for children in contact with the police who may be under the age of criminal responsibility and who might otherwise fall through the cracks in a community’s network of prevention and intervention services. In general, policy efforts should be directed towards greater recognition of the early demonstration of behavioral risk factors and greater general knowledge in the community regarding appropriate service to which children should be referred.

Limitations

This was an effectiveness study, and was conducted using an intent-to-treat standard. It was thus limited by real-world and common challenges to families seeking services and to providers of those services; some participants in each group ultimately participated in no services whatsoever. The nature of engagement in wrap services in the community resulted in greater difficulty and greater effort required on the part of families to participate in this service, which markedly reduced the ability to compare SNAP directly with wrap services.

Apart from the specific service types influenced by the study design itself (SNAP and wrap), participants in either condition tended to avail themselves of various other services in an equivalent fashion. The results can thus be interpreted as the effectiveness of participating in SNAP in the broader context of typical service use for behavioral problems.

An additional limitation of the study is the fact that roughly half of those approached for participation declined to participate. It is possible that self-selection based on a willingness to participate in a study, including the random assignment to treatment condition, may limit the generalizability of the findings. Additionally, the inclusion of only boys is a limitation. There are clear needs for additional, empirically validated intervention programs for antisocial behavior in girls. As the SNAP for Girls model becomes more widely adopted, efforts to evaluate that treatment model for girls against alternative treatments should be undertaken.

Finally, we did not include a number of other potentially important outcomes or explanatory covariates, such as changes in parenting practices and family functioning. Future research should focus on describing such changes both as valuable intervention outcomes in their own right as well as potential mechanisms of other outcomes.