Being “SMART” about Adolescent Conduct Problems Prevention: Executing a SMART Pilot Study in a Juvenile Diversion Agency

Abstract

OBJECTIVE

The development of adaptive treatment strategies (ATS) represents the next step in innovating conduct problems prevention programs within a juvenile diversion context. Towards this goal, we present the theoretical rationale, associated methods, and anticipated challenges for a feasibility pilot study in preparation for implementing a full-scale SMART (i.e., sequential, multiple assignment, randomized trial) for conduct problems prevention. The role of a SMART design in constructing ATS is presented.

METHOD

The SMART feasibility pilot study includes a sample of 100 youth (13–17 years of age) identified by law enforcement as early stage offenders and referred for pre-court juvenile diversion programming. Prior data on the sample population detail a high level of ethnic diversity and approximately equal representations of both genders. Within the SMART, youth and their families are first randomly assigned to one of two different brief-type evidence-based prevention programs, featuring parent-focused behavioral management or youth-focused strengths-building components. Youth who do not respond sufficiently to brief first-stage programming will be randomly assigned a second time to either an extended parent- or youth-focused second-stage programming. Measures of proximal intervention response and measures of potential candidate tailoring variables for developing ATS within this sample are detailed.

RESULTS

Results of the described pilot study will include information regarding feasibility and acceptability of the SMART design. This information will be used to refine a subsequent full-scale SMART.

CONCLUSIONS

The use of a SMART to develop ATS for prevention will increase the efficiency and effectiveness of prevention programing for youth with developing conduct problems.

Overview

The present article describes a pilot study that is currently underway to assess feasibility concerns that will inform implementation of a subsequent full-scale SMART (i.e., sequential, multiple assignment, randomized trial). The research is aimed at a sample of predominantly first-time juvenile offenders referred for pre-court diversion programming. These youth are considered to be at varying levels of risk for reoffending and possible escalation to more serious conduct problems (CP) including the illicit use and abuse of substances. Because of this heterogeneity in risk, the likelihood is that these youth will vary considerably in their therapeutic response to conventional diversion programs. When derived through a SMART, adaptive treatment strategies (ATS) provide a vehicle for addressing this response variability by individualizing intervention options via decision rules that specify how the composition and/or intensity of an intervention should be adjusted at critical decision points over time based on the individual’s early response (i.e., primary tailoring variable). In addition, SMART can test whether client risk characteristics (i.e., secondary tailoring variables) might be useful in tailoring the initial intervention options to the client or tailoring second-stage intervention options if there is a non-response to a first-stage intervention (Almirall, Compton, Gunlicks-Stoessel, Duan, & Murphy, 2012).

SMART designs are used to operationalize ATS with the goal to optimize client outcomes while reducing the burden and cost of delivering fixed-type interventions where “one size fits all”. To date, SMART designs have been employed to develop ATS for the treatment of serious and chronic disorders where variable response, unwanted side-effects, frequent relapse, and poor adherence to first-line treatment options are common outcomes (see Lavori, Dawson, & Rush, 2000).

The program of research described herein represents a seminal effort to employ a SMART in a prevention context¹. The overarching goal is to reduce the risk of first-time offenders developing serious CP and becoming chronic offenders. A three-phase process will be used to apply SMART technology to construct ATS that optimize outcomes for CP prevention. The present article describes the initial pilot phase which addresses questions about the feasibility of conducting a full-scale SMART in a real-world community setting. Key questions address (1) community buy-in and practitioner ability to deliver sequenced interventions with fidelity, (2) recruitment and retention of research participants, (3) estimates of participant n/responder rates and compliance with the various intervention and assessment protocols, and (4) identification of pre-intervention characteristics that may be used as secondary tailoring variables to determine ‘who’ benefits most from which ATS. After establishing feasibility, a second phase fully powered SMART will be used to validate ATS (i.e., demonstrate which intervention types work best as stand-alone interventions and which work best together in a sequence to optimize long-term outcomes). Adjustments in intervention types during the trial will be based on whether a healthy change in a youth’s risk trajectory can be detected (i.e., early response indicator). Another goal of the full scale SMART is to identify which candidate secondary tailoring variables are associated with which ATS. The third and final phase involves a two group confirmatory trial in which the intervention model with developed ATS is compared to an appropriate alternative such as an existing diversion program in which all clients receive the same composition and dosage.

Theoretical Rationale

Need for ATS for Juvenile Diversion Services

Pre-court diversion programs are intended to keep juvenile offenders out of the criminal justice system from the outset and thus reduce pejorative labeling and minimize opportunities for affiliation with more serious offending peers. In most cases following initial contact with law enforcement, these youth are diverted from judicial processing and referred for programing from a contracted external agency. Across agencies there is considerable variability in the type of diversion programs that are offered, with some simply cautioning youth (i.e., issue warnings of punitive consequences), others requiring community service and still others providing brief therapeutic interventions. A recent meta-analysis of 73 diversion programs serving primarily low-risk youth (Wilson & Hoge, 2013) showed that recidivism rates were significantly lower for diversion programs (average 31.5%) compared to conventional judicial processing (average 41.3% ). Interestingly, for low risk offenders, caution-type programs were equally effective as therapeutic programs. The study did not present data on the characteristics of recidivists or suggest what types of services would be beneficial for youth at moderate or higher levels of risk. However, there is belief that the effectiveness of diversion programs can be enhanced with the use of evidence-based therapeutic programs particularly if they are adjusted to the youth’s level of risk (Wilson & Hoge, 2013). Our own data of 50 diversion cases showed that nearly 60% had conduct problem rating scores in the at-risk or clinical range. About 40% had both conduct and attentional problems. Findings such as these have led investigators to suggest that diversion agencies should assess the risks and needs of youth entering the system and assign different types of programing based on the youths’ level of risk (Hoge, 2008; Hoge & Andrews, 2010).

Adaptive Intervention Framework

An emerging innovation in intervention science is the adaptive intervention model (August, Gewirtz, & Realmuto, 2010; Collins, Murphy, & Bierman, 2004). Adaptive interventions resemble ‘real world’ clinical practice in that different variations of an intervention are assigned to different clients or assigned within clients over time based on the needs of the clients as judged by the practitioner . However, as demand increases for evidence-based practices, adaptive interventions are desired that utilize empirically-derived decision rules to guide treatment rather than subjective clinical judgment or client preference. In one type of empirically-based adaptive approach, different types, compositions or dosages of an intervention are assigned based on decision rules that link individuals’ personal characteristics that are expected to moderate the effect of the treatment (i.e., tailoring variables). Common types of tailoring variables include individual, family of context characteristics representing risk or protective factors that influence responsivity to various types of intensities of intervention (Collins et al., 2004). One example is the personality-targeted intervention approach described by Conrad and colleagues (Conrad, Castellanos & Mackie, 2008). In this approach participants scoring high on personality risk subscales such as negative thinking, anxiety sensitivity, impulsivity and sensation seeking are matched to brief, cognitive-behavioral interventions that feature education specific to their personality profile including associated problematic coping behaviors. In a series of randomized trials this adaptive-type targeted approach was found to be effective in reducing alcohol drinking quantity, drinking frequency and drinking problems in high school and college age drinkers (e.g., Conrad, Stewart, Comeau, & MacClean, 2006; Conrad, Castellanos-Ryan, & Strang, 2010).

Adaptive Intervention Strategies (ATS)

A more dynamic type of adaptive model makes adjustments to the intervention during the course of its implementation (time-varying). This approach has been defined as adaptive treatment strategies (ATS) (Lavori et al., 2000; Murphy, Oslin, Rush, & Zhu, 2007). ATS operationalize treatment via decision rules that recommend whether, when, and how treatment should change (Almirall et al., 2012). The recommendations are based on client characteristics collected during treatment such as “has the client exhibited a significant reduction in primary symptoms?” or “has the client adhered to the treatment prescriptions?” When a client displays insufficient response, the treatment plan is re-adjusted by increasing dosage or switching to a different type of intervention. In addition, decisions about sequencing of treatment options can be further enhanced with the use of empirically-derived secondary tailoring variables that differentiate responders from non-responders. For example, the Fast Track conduct problems prevention program was time-varying with dosage of the family support and reading tutorial components adjusted up to three times a year (Conduct Problems Prevention Research Group, 2002; 2004). Time-varying adaptive interventions appear to be particularly useful for the treatment of chronic disorders such as depression (Lavori et al., 2000), Attention-Deficit Hyperactivity Disorder (Pelham et al., 2005), and alcohol and drug dependence (Kranzler & McKay, 2012).

Chronic disorders present complex challenges to treatment providers as a consequence of several factors including (1) the waxing and waning of symptoms over time, (2) variability among and within individuals in treatment response, (3) the emergence of co-occurring disorders, and (4) client burden associated with participation in long-term treatment (Murphy et al., 2007). To address these factors, interventionists often rely on sequential decision making such that adjustments to an intervention can be made when a client fails to achieve a desired outcome or relapses at some point in time. The sequential approach as embodied in stepped care models also addresses cost and burden issues. For example, lower cost/burden interventions can be tried first with those insufficiently helped stepped up to more intensive and costly interventions. Alternatively, a high intensity/cost intervention may be initially offered and those showing positive response stepped down to a lower intensity/cost intervention (Haaga, 2000; Jaycox, Kataoka, Stein, Wong, & Langley, 2005). As such, sequential stepped care allows practitioners to manage scarce resources efficiently so as to deliver effective programs to as many people as possible.

SMART Technology

Construction of high quality ATS can be achieved with an innovative type of clinical research design referred to as a sequential multiple assignment randomization trial [SMART] (Murphy, Lynch, Oslin, McKay, & TenHave, 2007; Murphy et al., 2007; Lavori & Dawson, 2008). A SMART design is implemented in multiple stages (see Lei, Nahum-Shani, Lynch, Oslin & Murphy, 2012). Each stage represents a key time point where a decision must be made as to whether or not to adjust the intervention with clients randomized at each stage to one of several intervention options. Sequenced randomizations ensure that at each decision point, the groups of participants assigned to each of the treatment options are balanced in terms of participant characteristics. Possible intervention options include different types of behavioral and pharmacological interventions, different intervention foci (parent or youth), different modes of delivery (group or individual), different dosage levels, and different approaches to enhance engagement and adherence to the intervention. The goal is to operationalize decision rules that form the basis for assigning the best overall intervention strategy to each individual (i.e., type of intervention component, dose level, or sequence of components) based on baseline and time-varying characteristics and outcomes collected during the trial such as symptom levels, side effects and adherence. Depending on the number of decision points and the number of intervention options to consider at each decision point, multiple ATS can be embedded in any one SMART. Set up in this way, a SMART allows investigators to answer key tactical questions in personalization treatment research, such as, “What is the best first-stage intervention option”, “Which second-stage intervention option is best for youth who do not show adequate response to first-stage intervention options”, and “Which sequence of intervention options yields the greatest outcomes.” SMART can also be used to construct more deeply personalized ATS by examining client characteristics that moderate intervention effects. Such secondary tailoring variables can be assessed prior to the intervention in the form of individual values on variables such as demographic characteristics, genomic information, and personal and psychosocial risk factors that are expected to moderate the effect of the treatment. When then used in an ATS developed through a SMART, these secondary tailoring variables can be “matched” to specific intervention components targeting those variables. These latter data help answer the question “What works for whom?”

While other intervention designs could have been considered to address some of these questions, a SMART was selected for several reasons. In contrast to other research designs such as an RCT or even non-responder trials, SMART allows us to evaluate a greater number of different intervention components and sequences in order to create a more comprehensively tailored approach. Moreover, by evaluating intervention sequences, SMART allows for an examination of ‘downstream’ or synergistic effects whereby an initial intervention component enables a client to benefit more substantially from subsequent intervention components (Murphy et al., 2007). By evaluating different first-stage, second-stage, and overall sequences of intervention, a SMART allows us to most systematically identify which approach may be most effective in addressing the diverse needs and risk factors of this population.

The Juvenile Offender SMART Pilot Study

Overview of the Pilot Study

This SMART pilot study is being conducted as a collaborative effort between a community youth-serving agency (St. Paul Youth Services: SPYS) and investigators from a university-based prevention science research team. The agency’s client base is ethnically diverse and resides in urban neighborhoods where high rates of poverty, criminal activity, and violence are prevalent. The agency serves early stage juvenile offenders who have been cited by law enforcement for various status and misdemeanor offenses including shoplifting, vandalism, disorderly conduct, underage drug use, and assault but have not yet established a pattern of serious antisocial behavior or have been formally adjudicated. SMART technology is being used to deliver and test various types, dosages, and sequences of interventions that may lead to the development of high quality ATS. Figure 1 illustrates a model of the various phases of the present research program of research that will lead to the construction of ATS for CP prevention. This approach is similar to other proposed multiphase intervention development frameworks (e.g., Collins et al., 2011).

Figure 1.

The multiple stages of ATS development utilizing SMART

Phase 1: Pilot SMART

The primary goal of the phase 1 pilot study is to conduct feasibility research that will inform and guide implementation of a subsequent phase 2 full-scale SMART. Feasibility will be assessed by determining whether a community agency serving pre-court juvenile diversion youth can successfully implement the fundamental elements of a SMART. This pilot includes the following specific aims:

• To determine whether agency counselors will (a) master the principles and content for each of the intervention options, (b) adhere to the randomized intervention sequences assigned to each client, (c) implement the various first- and second-stage interventions options with fidelity, and (d) express satisfaction with the intervention options and their utility with the client population.

• To document (a) rate of recruitment into the SMART from a population of youth and parent(s) referred to a pre-court juvenile diversion agency, (b) rate of participant compliance with a real-time randomization procedure that will be employed to assign participants to various intervention sequences, (c) rate of participation and quality of engagement of youth and parent(s) in their assigned interventions, and (d) participant satisfaction with their assigned interventions.

• To determine the number of subjects that will need to be enrolled in the full-scale SMART by obtaining estimates of (a) trial attrition at the first and second stages of the intervention trial, and (b) response rates following first-stage intervention options.

• To explore the potential utility of theoretically-informed secondary tailoring variables (i.e., pre-intervention youth and family risk characteristics) that can inform the subsequent full-scale SMART study. This will include (a) examining inter-subject variability in measures of candidate tailoring variables, (b) refining administration and scoring procedures for s measures of these tailoring variables, and c) conducting preliminary exploratory moderation analyses of the impact of these tailoring variables on the relation between ATS and response rates.

Stage 2: Full-Scale SMART

After demonstration of the feasibility and acceptability of the SMART design in a pilot study, SMART may be further refined and implemented as a full-scale SMART. A full-scale SMART will include measurement of candidate tailoring variables identified in the pilot SMART to have satisfactory reliability and variability within the targeted population. Furthermore, using preliminary data from the pilot, decision rules and procedures may be refined in order to create optimal response rates and manage attrition. Most importantly, a full-scale SMART will utilize a large enough sample to sufficiently power statistical tests identifying optimal ATS embedded within the SMART as well as identifying effective secondary tailoring variables.

Stage 3: Validation of ATS

Upon completion of a full-scale SMART, outcomes for the multiple embedded ATS may be compared to identify optimal ATS. The final ATS that are developed through a SMART no longer involve randomization, but rather utilize decision rules to guide treatment options at each decision point. A SMART allows for the development of ATS, but does not compare ATS to a non-adaptive approach (i.e., standard agency program). Therefore, once optimal ATS have been identified from the full-scale SMART, it is necessary to conduct a final randomized trial comparing the identified ATS to a standard fixed intervention. This final confirmatory stage will allow researchers to demonstrate that an ATS improves upon established fixed intervention approaches through either improving outcomes or demonstrating similar outcomes with improved efficiency of delivery.

Selection of Intervention Options for SMART

Treatment research aimed at juvenile offenders has benefited from a developmental life course (DLC) orientation (Farrington, 2005; Thornberry, 2004). DLC theory identifies personal and contextual characteristics associated with intra-individual differences in risk trajectories as they unfold over time. By tracing the progression of offending over time, one can identify risk factors linked to the onset, escalation and diversification of offending as well as protective factors associated with desistance from offending. A key issue for preventive interventions based on DLC theory is promotion of personal strengths and building of human capital in order to redirect offenders away from a delinquent lifestyle toward more conventional goals. Core concepts embedded in this approach include motivation to change, personal agency to purposely follow a path toward change, as well as skill sets and prosocial attitudes to bring about change (Guerra, Williams, Tolan, & Modecki, 2008). A focus on motivation incentivizes youth to move away from a preferred life of crime. Personal agency refers to the active participation of a person in creating their own lives by making a conscious choice to pursue prosocial goals. The skills needed to help youth make positive change include self-regulation and decision-making.

Experts involved in the study of juvenile offending have recommended evidence-based interventions that feature structured cognitive and behavioral techniques (Guerra, Kim, & Boxer, 2008; Lipsey, 2009). There is overwhelming evidence across developmental periods to suggest that youth problem-solving skills training and/or parent behavioral management skills training are effective intervention options in the prevention and treatment of CP (Kazdin, 2010) and that a combined treatment approach is more effective than either modality alone (Kazdin, Siegel, & Bass, 1992). However, one or both of these options do not work with everyone and many clients drop out of treatment (Kazdin & Wassell, 1999), and some who drop out likely do so because of so-called sudden treatment gains (Kazdin, 2011). It stands to reason that a SMART aimed at CP prevention would examine various sequences of the youth- and parent-focused intervention models.

We selected two evidence-based skills training programs, one focused on youth and the other for parents. The Teen Intervene program (TI; Winters & Leitten, 2007) embodies the core treatments elements recommended by DLC theory, i.e., motivational enhancement, prosocial goal-setting and training in reflective decision-making and social problem-solving with the goal of choosing attitudes and behaviors that are healthier alternatives to antisocial behaviors. The Everyday Parenting program (EP; Dishion, Stormshak, & Kavanagh, 2011) addresses three broad areas of parent/family skills-building: (1) behavioral support to promote adaptive adolescent behaviors, (2) limit setting to reduce maladaptive/high risk adolescent behaviors, and (3) family interaction skills to facilitate parent-adolescent communication and problem-solving. For the current study, both programs have been modified to be delivered in ‘brief’ and ‘extended’ formats resulting in models of lower and higher program intensity. Brief models that include motivational interviewing-based concepts may be especially effective as a first-stage intervention option, particularly for youth at low risk for escalations in conduct problems (Jensen et al., 2011; Dishion & Stormshak, 2007). Extended models are best suited for youth at moderate to high risk for CP and include more intensive skills training (rehearsal, role plays, homework) that may be particularly effective as second-stage options.

Based on the logic presented above, the present SMART pilot includes first- and second-stage program options (see Figure 2). At stage one, youth are randomized to either TI-Brief or EP-Brief. Responders to either first-stage option are stepped down and monitored over time for maintenance of intervention effects. Non-responders to either first-stage program are stepped up and randomized to second-stage intervention options: either (1) continuation of the first-stage option with increased dosage (TI-Extended or EP-Extended), or (2) switched to the alternative extended intervention option. With these interventions options, our proposed full-scale SMART will be able to address two critical questions for developing a high quality adaptive intervention to prevent reoffending and escalation to more serious conduct problems, “Which intervention should be offered initially?” and, “Which intervention options should be offered to youth whose risk status or trajectory is not sufficiently impacted during the first-stage intervention (i.e., non-responders)?” In this research, proximal response is measured by a comprehensive index of risk that encapsulates ratings of problem behaviors and impaired functioning and quality of peer affiliations (see below). Based on number of stages and number of intervention options, one or more ATS can be embedded within a SMART. Our SMART has the following four ATS embedded within it:

• Youth Only ATS: Beginning with TI-Brief, youth exhibiting positive response to initial TI-Brief are stepped down to monitoring; youth exhibiting non-response are stepped up to TI-Extended (i.e., this is a youth continuing ATS).

• Youth then Parent ATS: Beginning with TI-Brief, youth exhibiting positive response to initial TI-Brief are stepped down to monitoring; youth exhibiting non-response are stepped up to EP-Extended (i.e., this is a youth then switch to parent ATS).

• Parent Only ATS: Beginning with EP-Brief, youth exhibiting positive response to initial EP-Brief are stepped down to monitoring; youth exhibiting non-response are stepped up to EP-Extended (i.e., this is a parent continuing ATS).

• Parent then Youth ATS: Beginning with EP-Brief, youth exhibiting positive response to initial EP-Brief are stepped down to monitoring; youth exhibiting non-response are stepped up to TI-Extended (i.e., this is a parent then switch to youth ATS).

These four embedded ATS are stated as decision rules which will be examined for effects on CP prevention-related outcomes in a subsequent full-scale SMART. The full-scale SMART will also allow comparisons of the relative effectiveness of the four ATS.

Figure 2.

SMART design for adaptive intervention strategies in conduct problems prevention.

Methods

Subject Recruitment and Eligibility Criteria

The target population is youth (13–17 years of age) identified by law enforcement as early stage offenders and referred for pre-court juvenile diversion programing. These youth vary widely in risk for reoffending and development of serious CP in later adolescence and possibly antisocial personality disorder (ASP) during the adult years. Some may be only one-time offenders who are at low risk for reoffending and serious CP and ASP development. Others may become chronic offenders and are consequently at heightened risk for developing serious and chronic CP and ASP. The pilot study plan is to recruit a minimum of 100 youth participants. Youth are currently being recruited from consecutive referrals that will continue over a 24-month period. In 2010, participants who were enrolled in SPYS diversion programming represented diverse ethnic backgrounds, including 45% African-American, 25% Caucasian, 17% Asian-American or Pacific Islander, 10% Latino, and 0.5% American Indian/Alaska Native. Youth in the program were approximately equally distributed by gender (48% female). Youth are referred to the SPYS diversion program either by a police officer (68% in 2010) or the Ramsey County attorney’s office (32% in 2010) following an arrest for a crime. In 2010, 48% of youth were referred to the program after a charge of shoplifting, 14% for disorderly conduct, 10% for possession or use of alcohol, 6% for curfew violations, 5% for possession or use of drugs, 5% for criminal damage to property, and 5% for theft. Approximately 11% of these offenses occurred in a school setting.

Upon presenting to SPYS after referral, youth are invited to enroll in the SMART research study. Referred youth who are deemed to have a serious DSM-5 psychiatric disorder (American Psychiatric Association, 2013) requiring specialized mental health treatment (e.g., psychotic disorders, bipolar disorders, etc.) are ineligible to participate in the research study. Youth who are arrested for using illicit substances will be included in the research with the stipulation that they are not assessed to have a substance use disorder. SPYS retains counselors who speak English and Spanish. Hence, youth with a primary language other than English or Spanish are excluded from the research.

Assessment of the Proximal Intervention Response

Critical to the execution of a SMART is the selection of a proximal response indicator (i.e., primary tailoring variable). The response indicator is used to adaptively determine a subsequent treatment option. Preferably, the indicator is a construct informed by theory, prior research or convention, possesses excellent reliability and validity (Lipsey, 1990), is sensitive to short-term treatment change including both improvement and deterioration (Lambert & Hill, 1994), and is brief enough to be administered with minimal burden, is easily scored, and is cost effective. SMART designs aimed at the treatment of chronic disorders typically use indicators such as reduction in symptoms and/or functional impairments as well as abstinence from substance use. For example, in an adaptive treatment study of children with ADHD using a SMART design (Pelham et al., 2005), response to behavioral and pharmacotherapy interventions was determined on the basis of indicators of school-rated behavioral symptoms of ADHD and ratings of functional impairments after eight weeks of first-line treatment. The investigators determined that 78% of children receiving behavioral treatment and 77% of children receiving medication were non-responders, who were then randomized to second-line treatments.

In comparison to treatment trials, response indicators in prevention trials are not readily apparent. Individuals enrolled in prevention trials are either asymptomatic (e.g., universal approach) or at the early stages of problem development (e.g., indicated approach). They are of interest because they are considered to be at heightened risk for negative outcomes. Our approach is considered preventative because the youth referred for diversion services are considered to be at elevated risk for reoffending and development of more serious CPs. Morever, the vast majority of these youth have had no prior involvement with the juvenile justice system or and had no more than one previous contact with diversion.

To determine treatment responder status, we opted to use a multidimensional, multi-informant risk assessment tool that included measures of 1) current conduct problems, 2) functional impairment in multiple domains, and 3) deviant peer affiliations. Current conduct problems will be assessed using the Conduct Problems Scale from the parent-report Behavioral Assessment System for Children-2 (BASC-2: Reynolds & Kamphaus, 2004). Respondents rate the frequency of 9 behaviors (e.g., “Disobeys,” “Steals,” “Lies to get out of trouble,”). From a large normative sample, Reynolds and Kamphaus (2004) found evidence for excellent reliability of this scale in adolescent samples (coefficient alpha =.87). Deviant peer affiliations will be assessed using parent and youth report on the Friendship Scale (Child and Family Center, 2013 a, b). The scale includes 8 items relevant to deviant peer affiliations (e.g., “How many of your child’s friends/your friends misbehaved or broke the rules?). The Friendship Scale has been used with large high-risk community samples and an earlier version of the scale demonstrated acceptable reliability (youth: α = 0.69; parent: α = 0.66) and convergent validity between youth and parent reporters (r = 0.34, p < 0.05) (Gardner, Dishion, & Connell, 2008). Functional impairment will be assessed through the practitioner-completed Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 2000). The CAFAS is widely used to assess the needs of youth in mental health, child welfare, juvenile corrections, education, and community-based systems of care and for evaluating program outcomes. The psychometric properties of the CAFAS are well established with research showing evidence of reliability, concurrent validity, predictive validity, and sensitivity to change (Hodges, 2005). We will use four CAFAS domains pertaining to the youth (School, Home, Community/Delinquency, and Behavior Towards Others). The items in these four domains assess functioning in multiple areas relevant to youth conduct problems, including school behavior, academic achievement, truancy, parental limit setting, parent-child communication, deviant peer involvement, drug use, anger management, and involvement in positive activities.

To be flagged as at risk and in need of additional services (i.e., a non-responder) following completion of the first-stage intervention, youth would need to show evidence of any of the following: (1) conduct behaviors in the at-risk range or higher on the BASC −2 (i.e., a T-score of 60 or above), or (2) impaired functioning as rated by practitioners in the School, Home, Community/Delinquency, or Behavior Towards Others domains on the CAFAS, or (3) elevated exposure to deviant peer influences on the Friendship scale (i.e., a T-score of 60 or above on either scale).

The operational definition of response in this SMART is based on the notion of whether or not risk is present after the first stage brief intervention based on our four response indicators. A conservative approach is utilized such that a responder must be in the subclinical range on all four indicators after first stage interventions to minimize classification of ‘false responders’ who might later relapse. It is important to keep in mind that this criterion does not reflect a significant change in risk as assessed from pre- to post-intervention because it is not administered before the first stage intervention. Rather, if the response index is above the cut-off threshold, the youth is labeled a ‘non-responder’ in that the risk is still evident in spite of the first stage intervention. In other words, she/he remains at elevated risk for serious CP even after the brief intervention. This approach was selected in part due to the heterogeneity of sample and the associated difficulty in creating a uniform decision rule documenting “progress” in youth presenting with a wide range of initial levels of risk and current symptoms.

First-Stage ‘Brief’ Interventions (3 Sessions)

Teen Intervene-Brief (TI-B)

TI-B is a youth-focused program originally developed for adolescents with illicit use and abuse of substances (Winters, 2004; Winters & Leitten, 2007). The program subsequently has been expanded to address other types of conduct problems. The theoretical basis of TI is grounded in motivational interviewing, cognitive-behavioral therapy, and self-change programs (Breslin, Li, Sdao-Jarvie, Tupker, & Ittig-Deland, 2002; Miller & Rollnick, 2002). TI strives to boost the youth’s problem recognition and interest in change by raising awareness of the problem, placing responsibility for change with the youth, negotiating realistic goals, and redirecting youth on a healthy developmental course. In this study, the youth-focused TI-B will consist of three 60-minute meetings delivered by a counselor using a motivational interviewing (MI) style. The content of sessions focuses on (1) motivation to change, (2) resisting negative peer influence and seeking out positive peer-group support, (3) inhibiting impulsive behaviors and making decisions that anticipate potential consequences, and (4) setting goals to help enhance success at school and with family and friends.

While originally validated in preventing adolescent substance use, the TI program was selected to target risk for progressions in conduct problems in the present study for a number of reasons, including (1) the high level of comorbidity between youth substance use/abuse and conduct problems and the associated shared risk factors (Loeber & Keenan, 1994), (2) the focus on intervention targets that are relevant for both substance use and conduct problems (i.e., impulsive decision-making, emotional dysregulation, affiliation with deviant peers, and lack of bonding to prosocial institutions), (3) TI was validated with adolescents in the same age range as the present diversion sample, and (4) many diversion programs report that use of illicit substances is the major offense that brings youth to diversion (Patrick & Marsh, 2005).

Everyday Parenting-Brief (EP-B)

The EP curriculum is adapted from the EcoFIT family intervention model (Dishion & Stormshak, 2007; Dishion et al., 2011). EP-B addresses three broad areas of parent/family skills building presented in a modular format. The first module, Positive Behavior Support, is devoted to positive behavior support such as monitoring and tracking daily adolescent behavior, reinforcing positive behavior, making effective requests, and using a written behavior-change plan. The second module, Healthy Limit Setting, focuses on limit setting including specifying clear rules and expectations, responding to inappropriate behaviors with small, realistic, non-abusive and effective consequences, and monitoring the teen. The third module, Communication and Problem-Solving, is geared toward communication and problem-solving skills involving parent–teen communication and discussion of pertinent adolescent topics such peers, dating, and alcohol/drugs. Parent skills are enhanced through didactic presentations, practitioner modeling, role-playing, and practice exercises.

In this study, EP-B will be delivered over three 60-minute sessions with a counselor guiding the parent through one of three program modules. In EP-B, the sessions are delivered to the parent(s) only. Counselors use motivational interviewing methods and a structured ‘Family Feedback’ form to organize and tailor parents’ concerns about the teen and family in order to determine an initial focus that fits one of the three EP modules. Then the counselor helps the parent set goals, prioritize material to work on within the selected module, and pledge effort. These skills are taught in relationship to dealing with typical problem situations, including household chores, improving parent-youth relationships, monitoring troublesome friends, improving school performance, reducing sibling fighting, sharing family space, complying with curfew, avoiding places where other teens are using drugs, and complying with parental requests.

Second-Stage ‘Extended’ Intervention (5 Sessions)

Teen Intervene-Extended (TI-E)

TI-E sessions seek to decrease impulsive decision-making via problem-solving strategies. Incorporated in these sessions are elements of reflective problem-solving steps, coping skills, assertiveness training, refusal skills (resistance to peer pressure) and goal-setting. For example, problem-solving steps include (a) stop and focus in order to inhibit reactions, (b) think ahead to anticipate longer term negative consequences to counteract the immediate perception of reward; (c) brainstorm and generate alterative solutions; (d) plan ways to resist negative peer influences; and (e) identify trusted adults who can support prosocial activities. Youth who receive TI-E following EP-B will receive the three sessions of TI-B along with two additional sessions focused on practice and application of decision-making skills. Youth who receive TI-E after TI-B will also work on decision-making skills and be exposed to additional content to address communication, conflict resolution, and anger management, and negotiation along with a focus on practice and application of earlier learned skills. TI-E utilizes experiential, exercise-based learning. Each session integrates a variety of activities and coaching modalities which include: discussion, modeling, worksheets, role playing, skill reinforcement, and feedback, as well as practice assignments. These activities are guided by MI techniques to actively engage participation of the youth. At the completion of each session, the parent/guardian is briefly informed of the content of the session and given handouts with suggestions on how to support youth skills development.

Everyday Parenting-Extended (EP-E)

In EP-E, some sessions are parent(s) only and some include parent(s) and youth. During sessions where the youth is involved parent(s) are prompted to teach or review goals with the youth. EP-E focuses on the remaining two EP modules. As was the format for EP-B, a collaborative process is used between parent and counselor to review current concerns and to organize and tailor the EP modules. New modules are then delivered and worked on until mastery is achieved. Skills are again taught using presentation, discussion, and behavioral rehearsal. Much more emphasis is place on goal-setting/attainment methods and homework/practice opportunities to reach these goals. Parents who receive EP-E after TI-B will receive the three sessions of EP-B along with two additional sessions focused on the EP parenting module that is most salient for that family. Whenever possible, the Communication and Problem-Solving module is a focus of EP-Extended at some point because the teen is also involved.

SMART Design

Figure 2 displays the protocol for the SMART. One-hundred (N=100) children will be enrolled in the pilot study. Sample size is not based on detecting an initial effect size between first- or second-stage treatments (or embedded ATS) with which to power the full-scale SMART because this is not the recommended practice for pilot studies (Kraemer, Mintz, Noda, Tinklenberg, & Yesavage, 2006). Rather, consistent with the goals of this pilot, sample size is based on the feasibility aims which are needed prior to carrying out a successful full-scale SMART: (1) to estimate study attrition at each programming stage, and (2) to estimate the overall response rate to first-stage program options.

Random Assignment

Participants are the unit of randomization. Each participant is randomized at the onset of the trial to one of two first-stage intervention options. Participants designated as responders are stepped down and monitored over time for progress. Non-responders are stepped up and randomized a second time to second-stage intervention options. Real time randomization will minimize confounding and allow for balancing of second stage options based on response to first-stage options (Almirall et al., 2012). This pilot study will provide an opportunity to explore how the real time (sequential randomization) procedure affects participant retention. This information will inform us how to stratify the sequential randomizations in the subsequent full-scale SMART. For example, we might stratify the second randomization by adherence to first-stage interventions to ensure that adherent participants and non-adherent participants are balanced in both second stage intervention options.

Secondary Tailoring Variables

The heterogeneity in this population suggests that different intervention options may be needed to optimize prevention outcomes for youth with different risk profiles. Thus, the choice of intervention options also may be tailored on the basis of individuals’ pre-intervention risk characteristics typically referred to as secondary tailoring variables. Secondary tailoring variables are derived from measures or indicators that predict or moderate response to the first-stage interventions. Ideally, these tailoring variables would also represent key mechanisms that underlie CD and can be shown to be malleable. Furthermore, these tailoring variables need to be translated into valid and reliable measures that have practical utility in the context of clinical decision making.

Within the context of a SMART pilot, it is not possible to reliably identify secondary tailoring variables that predict treatment response. An appropriately powered full-scale SMART is necessary to evaluate the utility of such variables. However, within the context of the pilot, we may identify candidate secondary tailoring variables that yield individual variability and display strong psychometric properties within the study population and establish feasibility of delivery. While lacking adequate power, exploratory analyses can also be used to identify promising candidate tailoring variables that may predict treatment response in the full-scale SMART.

A number of individual and environmental risk factors will be examined as potential secondary tailoring variables in the present pilot study and further refined for use in the subsequent full-scale SMART study. Inherent in the derivation of secondary tailoring variables is a focus on individual variation in etiological factors rather than symptom profiles. Within the context of CP prevention, this means evaluating multiple levels of risk factors associated with multiform developmental pathways to CP. We believe that developing a ‘risk profile’ of specific etiological factors for each youth will best allow us to predict response to CP prevention programing.

Neurocognitive factors such as executive functioning (EF) appear to have promise as potential secondary tailoring variables as they have been consistently implicated in the etiology of conduct problems in childhood and adolescence (Olson, Schilling, & Bates, 1999; Séguin & Zelazo, 2005). However, EF deficits are not universal among those exhibiting externalizing disorders, suggesting that EF deficits may be associated with some but not all etiologic pathways (Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). We will utilize a variety of performance-based tasks evaluating different aspects of youth EF, including “cool” aspects such as response inhibition and cognitive control and “hot” aspects such as risky decision making and delay discounting. Callous and unemotional (CU) traits are believed to be another distinguishing feature of differing developmental pathways to CD (Frick, 2012). Youth at risk for conduct problems are also likely to display heterogeneity in contextual risk factors. Those contextual risk factors most relevant to CP development include maladaptive family functioning (e.g., coercive interactions, poor parental monitoring), family stress and instability, and parental psychopathology (Dodge, Coie, & Lynam, 2006; Lanza & Drabick, 2011; Miller, Loeber, & Hipwell, 2009)

Challenges Confronted in the SMART Pilot Study

Almirall et al. (2012) present compelling reasons for conducting pilot research prior to rolling out a full-scale SMART. These reasons primarily address feasibility and acceptability questions such as “Can the proposed sequences of interventions be delivered properly?”, “Can the staff deliver the interventions with fidelity?”, and “Will participants accept their assigned intervention options?” In this penultimate section of the article, we describe preliminary work that we have conducted to address a number of anticipated challenges. Many of these challenges are not unique to a SMART pilot study and may be confronted when implementing other forms of adaptive treatment and also in establishing community-research partnerships. Our preliminary efforts involved the following: (1) regularly scheduled brainstorming sessions between research investigators and agency staff to discuss anticipated challenges in the implementation of the SMART,( 2) education and training of agency staff in administration of the interventions and training of research staff in the administration of various protocol measures, and (3) recruitment of a small sample (n=50) of clients to run through the protocol and provide feedback on various planned procedures. Below we describe four challenges that arose during this preliminary phase.

Establishing Agency Buy-In

Executing a SMART in a community agency presents numerous challenges for both agency care systems and research investigators. Agency care systems are political and dynamic, and thus, organization changes and crises should be anticipated (e.g., changes in administrative leadership, loss of program space, conflicts with existing programs, changing priorities, and power struggles between agency staff and research team). Failure to anticipate any of these potential points of conflict can inadvertently sabotage successful implementation of an intervention trial thus nullifying any of the study findings. For the present trial, a collaborative partnership was established between the agency and research team approximately 18 months in advance of the start of the pilot trial. A steering committee was assembled to prepare policy guidelines, discuss anticipated barriers, and resolve emergent crises. Meetings were scheduled on a monthly basis. Earlier meetings centered on (1) creating trust and respect between partners, (2) establishing a policy of shared ownership of the research protocol among research investigators, agency staff, and community stakeholders, (3) reconciling differences in the organizational cultures (research versus service) that could place strain on both teams in the implementation of evidence-based programs, and (4) organizing study operations such as intake procedures, consent materials, administration of research measures, assignment of counselors to interventions, and counselor work schedules. Later meetings addressed (1) procedures to maintain confidentiality and safety of research participations, (2) strategies for recruitment, participation, and retention of participants, (3) appropriateness of research measures from practical, logistical and cultural perspectives, and (4) education and training for agency implementers. Finally, it was considered useful to roll out a “pre-pilot” protocol with a small sample of participants. The pre-pilot would provide opportunities to refine recruitment and consent procedures, address technical difficulties in measure administration, provide counselors practice, coaching and correction in the implementation of the experimental interventions, and develop and refine fidelity assessment tools.

Identification of the Primary Tailoring Variable (Intervention Response Indicator)

In adaptive interventions, the primary tailoring variable is typically some definition of an early response to treatment. In prevention trials, the likely candidate for intervention response is a change in the youth’s risk trajectory. As noted above, there are manifold risk factors aligned on diverse developmental paths that predispose to CP development but fewer risk factors that operate to maintain or escalate CPs. Guided by the literature, and following lengthy discussions between investigators and practitioners, as well as examination of several assessment tools that were administered to a group of pre-pilot study clients, consensus was reached on a multidimensional, multi-informant index of risk for future offending (see Assessment of the Proximal Intervention Response above). This risk index was thought to capture problematic levels of conduct problems and impaired adaptive functioning as well as continuing affiliation with deviant peers. The current pilot study will be used to determine whether this response index (1) is feasible for use by agency staff, (2) is acceptable to informants and efficient for administration and scoring by investigators and staff, (3) is reliable and valid (concurrent and predictive), and (4) yields rates of responders and non-responders suitable to randomize participants to second stage interventions. Based on preliminary data, changes may be made in the composition of the index or in the criterion for response determination. Because the aim of the present study was to prevent reoffending and establishment of a pattern of serious CP, we selected an early response indicator that incorporated risk characteristics that have been shown to be predictive of our to-be-prevented outcome.

Handling Missing Response Data

A critical issue in developing any ATS is how to manage missing response data for the purpose of assigning subsequent intervention options (this problem is different from management of missing outcome data in intent-to-treat trials). This problem is further exacerbated by the likelihood that clients will vary in the circumstances related to their missing response data. For example, among those with missing response data there will be those who (1) complete stage 1 intervention but fail to complete forms necessary to assess intervention response (e.g., parent or youth are forgetful or uncooperative), (2) drop out of the program before completion of stage 1 intervention for reasons unrelated to a poor response (e.g., parent or youth cannot secure reliable transportation to attend sessions or cannot find child care), or (3) drop out of the program for reasons related to poor response (e.g., parent experiencing significant stress, youth exhibiting oppositional and noncompliant behavior). Given these possibilities, a critical issue is how to classify such clients. Whatever the definition, it needs to be set prior to the start of the trial. Almirall et al. (2012) suggested various methods to operationalize missing response data. In one method ‘missingness’ would be made part of the definition of early response. For example, all participants with missing response data would be classified as non-responders and then randomized to a stage 2 intervention if and when they re-entered the trial (‘nonresponding until proven responding’). As noted earlier in the paper, there is a subgroup of offenders who drop out of treatment because of so-called sudden treatment gains (Kazdin, 2011). Thus, we might expect a larger subgroup of early responders in an at-risk sample. Therefore, designating all participants with missing response data as non-responders would likely produce a high proportion of false positives (i.e., youth referred for extended programming who would not benefit from additional services). An alternative option would be to use only the counselor CAFAS ratings when youth and parent data are unavailable. This approach would rely exclusively on subjective judgment of a counselor. Last, we could classify participants failing to complete response forms as attriters, and reserve the category of non-responders only for those clients with response data who are rated via our risk assessment tool as non-responders. The drawback to this approach is that some youth with incomplete response measures but continued high risk will not be referred for additional services. However, families who are unable or unwilling to complete response measures after first-stage programming seem unlikely to attend second-stage programming.

Clearly, an important goal of the present pilot study will be to derive a final definition of response status for the subsequent full-scale SMART. To assist in this goal, we will make a substantial effort to contact those participants with missing response data and assist in collecting the data. If we still cannot collect these data, we will determine the various reasons for noncompliance. The pilot study, however, will need a working definition of stage 1 response status in order to make stage 2 assignments. After carefully weighing all options, we selected the approach best suited for a pilot feasibility study which is to classify as responders only those for whom response data are available. Those without such data will be classified as attriters from the research protocol. A protocol has also been established to reengage those participants in intervention programing, even if they no longer active research participants. This will involve a sequence of attempts to contact and reengage families through multiple means, including telephone, mail, and through a home visit. We will evaluate this approach via the pilot study to later inform and possibly make adjustments for the subsequent full-scale SMART.

Intervention Spillover and Fidelity Procedures

As described above, the planned SMART will involve delivery of two interventions, TI and EP in both brief (stage 1) and extended (stage 2) formats. All practitioners will be trained to administer both programs, TI and EP. While this approach minimizes counselor effects, it leaves open the possibility of program spillover effects with subsequent therapeutic equalization. To minimize this threat to internal validity it was considered imperative to set in place rigorous fidelity oversight procedures. Four procedures were developed. First, all practitioners receive up-front training from the research staff who have either developed the programs or are certified in administering the programs. Second, using the preliminary client sample, an intervention supervisor shadows the delivery of both programs by each practitioner. This provides an opportunity to observe unintended examples of spillover followed by coaching and correction. Third, it was agreed that counselors would complete logs following each session detailing the methods and content covered. This would continuously reinforce and remind practitioners about program parameters and would provide the intervention supervisors with additional data to detect any unintended contamination. Last, counselors attend regularly scheduled supervisory meetings. Audiotapes are recorded of all intervention sessions. A random sample will be evaluated to determine the success of the oversight procedures.

Summary

Youth referred for diversion represent a group at variable levels of risk for reoffending and escalating to more serious CPs. Targeted preventive interventions hold promise as they focus on reducing risk factors associated with reoffending and escalation and promoting protective factors associated with reclaiming a healthy developmental trajectory. Adaptive treatment strategies (ATS) have emerged as a new approach that may be particularly useful for delivering interventions to diversion youth. ATS individualize treatment via decision rules that recommend whether, when, and how treatment should be delivered and subsequently adjusted to optimize outcomes. As such, they work most effectively as a sequence of intervention options delivered over time. The construction of ATS can be achieved using a SMART design. In a SMART, participants move through multiple stages of intervention and at each stage they are randomized to two or more intervention options. The goal is to operationalize ATS that form the basis for assigning the best overall intervention sequence to each individual (i.e., type of intervention sequence, component, and dosage level) based on information collected during the trial such as a client’s response (primary tailoring variable) or secondary tailoring variables related to client characteristics.

The implementation of ATS for prevention and the application of SMART to develop them present considerable challenges to both research investigators and community practitioners who serve as intervention counselors. This article describes a SMART pilot study that is currently being conducted as a collaborative effort between a university-based research team and a community youth-serving agency that provides services for a juvenile diversion population. This pilot study will help address implementation challenges in preparation for conducting a full-scale SMART. This program of research seeks to build ATS as represented by sequences of evidence-informed interventions for the prevention of CPs. When delivered at full scale, the SMART will help researchers address three important decisions: 1. “Which intervention to provide first?” 2. “Which intervention to provide to youth who fail to respond to the initial intervention”, and 3. “Who responds best to the various intervention sequences?” By addressing these important questions, it is anticipated that ATS will increase the efficiency and effectiveness of prevention programming for youth at risk for CP progression.