Statewide Adoption and Initial Implementation of Contingency Management for Substance Abusing Adolescents

Abstract

Four hundred and thirty-two public sector therapists attended a workshop in contingency management and were interviewed monthly for the following 6 months to assess their adoption and initial implementation of contingency management to treat substance abusing adolescent clients. Results showed that 58% of the practitioners (n = 131) with at least one substance abusing adolescent client (n = 225) adopted contingency management. Rates of adoption varied with therapist service sector (mental health versus substance abuse), educational background, professional experience, and attitudes toward treatment manuals and evidence-based practices. Competing clinical priorities and client resistance were most often reported as barriers to adopting contingency management, whereas unfavorable attitudes toward and difficulty in implementing contingency management were rarely cited as barriers. The fidelity of initial contingency management implementation among adopters was predicted by organizational characteristics as well as by several demographic, professional experience, attitudinal, and service sector characteristics. Overall, the findings support the amenability of public sector practitioners to adopt evidence-based practices and suggest that the predictors of adoption and initial implementation are complex and multifaceted.

This article examines the adoption and initial implementation phases of a statewide effort to narrow the gap between science and practice in the treatment of substance abuse disorders in adolescents. The public health importance of transporting evidence-based substance abuse services to community-based treatment settings has been described in several reviews (e.g., Brown, 2000; Institute of Medicine, 1998; Marinelli-Casey, Domier, & Rawson, 2002) and is a federal research (Compton et al., 2005) and services (Substance Abuse and Mental Health Services Administration, 2007) priority. Yet, as noted in these reviews and elsewhere (e.g., Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; McCarty et al., 2007), the vast majority of substance abuse treatment services are not evidence-based, and relatively little research has been conducted examining the adoption and implementation of evidence-based practices.

Diffusion theory (Rogers, 2003) provides a valuable perspective on the diffusion of innovations such as evidence-based treatments, and this perspective has been applied by behavioral health services researchers (e.g., Henderson, MacKay, & Peterson-Badali, 2006; Mihalic & Irwin, 2003). The diffusion of innovation is viewed as occurring through four stages - dissemination (e.g., obtaining information or training about the innovation), adoption (e.g., deciding to use the innovation), implementation (e.g., initial or exploratory use of the innovation), and maintenance (e.g., consistent use of the innovation). With regard to the transport of substance abuse treatment innovations in particular, Simpson (2002) has articulated a highly similar conceptual framework, with special attention to the role that organizational characteristics play at each of the four stages in the diffusion process.

As suggested previously, this study builds on the dissemination phase (Henggeler et al., 2007) of a diffusion study by examining the subsequent adoption and initial implementation of an evidence-based substance abuse treatment (i.e., contingency management, CM) by public sector practitioners across South Carolina. CM is one of the most extensively supported treatments of substance abuse for both adults (e.g., Higgins, Silverman, & Heil, 2008; Petry, 2000) and adolescents (e.g., Azrin et al., 1996; Henggeler et al., 2006). In the initial phase of this study, a large heterogeneous sample of community-based therapists who treated adolescents in the state mental health and substance abuse service sectors (N = 543) was recruited to participate in the project and offered an opportunity to attend a 1-day workshop in CM. Eighty percent (N = 432) of these practitioners attended one of the 26 CM workshops offered across the state, reflecting widespread interest in the use of CM by practitioners in both the substance abuse and mental health service sectors. Analyses showed that workshop attendance was not predicted by practitioner demographic characteristics, professional backgrounds, or attitudes toward evidence-based practices. Consistent with Simpson's (2002) model for transferring research to practice, however, high organizational motivation and readiness to change was a significant predictor of workshop attendance. Workshop attendance reflects the first stage of the diffusion process articulated by Rogers (2003) - dissemination of information/exposure to training.

It should be noted, however, that the general consensus in the field (e.g., Fixsen et al., 2005; Institute of Medicine, 1998) is that workshops have little impact on practitioner behavior. Likewise, findings from the few studies of training using experimental designs (e.g., Miller, Yahne, Moyers, Martinez, & Pirritano, 2004; Sholomskas et al., 2005) have shown that intensive and sustained training efforts are more successful at increasing the adoption and improving the implementation of research-based interventions. Nevertheless, the findings from these same studies also clearly show that workshops can impact therapist behavior even at follow-up (also see Miller & Mount, 2001 for an excellent uncontrolled study demonstrating the effect of a workshop on therapist behavior). Thus, for empirical as well as practical (i.e., providing training for hundreds of public sector employees across the state) reasons, a 1-day workshop was the training experience provided in this study. Our assumption was that the workshop had a reasonable chance of impacting therapist behavior, at least to a modest degree.

The present article examines the subsequent adoption and initial implementation phases of the diffusion process. Specifically, workshop attendees were provided access to CM resources (i.e., manuals, drug testing kits, incentives for use with clients) and contacted monthly for 6 months following workshop attendance to assess their use of CM in treating adolescent substance abuse. In consideration of the underlying purpose to the study - to narrow the gap between science and practice - the key outcomes of interest were the numbers of practitioners who attempted to use CM (adoption phase) and the fidelity of such initial implementation (implementation phase). In addition, based on an emerging literature on the transport of evidence-based mental health and substance abuse practices, demographic, professional background, attitudinal, and organizational predictors of adoption and initial implementation were also examined.

Several investigators have described the multifaceted and multidimensional nature of the adoption and implementation processes in the area of children's services (Fixsen et al., 2005; Henderson et al., 2006; Mihalic & Irwin, 2003; Schoenwald & Hoagwood, 2001). Across reviews in the emerging field of implementation science, the general consensus is that pertinent predictors of adoption and implementation pertain to the nature of the intervention itself (i.e., some interventions are more amenable to adoption than are others; Henggeler, Lee, & Burns, 2002), the characteristics (e.g., age, gender) and professional backgrounds (e.g., degrees, experience) of potential adopters, their attitudes toward evidence-based practices (e.g., Godley, White, Diamond, Passetti, & Titus, 2001), and organizational constructs such as motivation to change and climate (Roman & Johnson, 2002; Simpson, 2002). In the present study, the selected intervention was held constant (i.e., CM was the only innovative intervention available for adoption and implementation), but participant backgrounds and attitudes and organizational features were measured and varied widely as might be expected in light of the nature of the sample (i.e., public sector practitioners across 44 provider agencies).

In summary, the adoption and initial implementation of CM were examined among a large sample of public sector mental health and substance abuse practitioners who treat adolescents and had attended a 1-day workshop in CM. Predictors of adoption and initial implementation were examined at the demographic, professional background, attitudinal, and organizational levels.

Method

Design

The primary purpose of the study was to examine the adoption and initial implementation of CM by public sector practitioners who attended a 1-day workshop in CM during the first phase of the project. As such, these therapists were surveyed monthly for 6 consecutive months following workshop attendance to determine the presence of substance-abusing adolescents on their caseloads, the adoption of CM with such adolescents, the nature of any barriers to CM adoption, and the fidelity of CM implementation if adopted.

Participants and Recruitment

As described in more detail by Henggeler et al. (2007), the state of South Carolina is served by 33 Department of Alcohol and Other Drug Abuse Services publicly funded provider organizations and 17 Department of Mental Health community mental health centers. Eighty-eight percent of these agencies agreed to participate in the study, and the research team visited each of the participating sites to recruit therapists that treated adolescents for some portion of their caseloads. The research team introduced and described all aspects of the project to prospective participants, emphasizing the voluntary nature of research participation and that participation would have no impact on their job performance evaluation. After answering any questions that the eligible practitioners (i.e., those treating adolescents) might have, the practitioners were consented to the study per approval from the University and State institutional review boards. Immediately following informed consent, the consenting practitioners were administered several self-report questionnaires that took about 45 minutes to complete. Practitioners who completed the questionnaires were reimbursed $20 in gift certificates for their time. Five hundred and forty-three practitioners consented to participate and completed the aforementioned questionnaires, 432 (80%) of these subsequently attended one of the 26 CM workshops that were offered across the state at convenient times and locations, and 430 (99%) of workshop attendees completed at least one research interview.

The Workshop

The CM workshop addressed the following content areas: an overview of the theoretical rationale and empirical support for CM procedures in treating substance abuse, functional analysis of youth drug use, objective monitoring of drug use, contingency contracting, and self management planning with drug avoidance skills. Chapters in the training manual (Cunningham et al., 2004) are each structured in the same way and are designed to provide therapists with a quick overview of treatment goals for each component, a checklist of tasks to accomplish in each session, worksheets and materials to use for each task, a trouble shooting section, and suggestions for implementing when caregivers are not present. The CM workshop followed this same structure and was designed to give therapists experience using the checklists and worksheets in the manual. For each CM component, there was didactic instruction, trainer role-play, and then dyadic role-plays for each therapist to practice implementing CM. These procedures allowed the trainers to provide hands-on training to each therapist during the workshop. Moreover, the format of the workshop was consistent with the research literature on effective training for the types of practitioners participating in this project (e.g., Daniels & Walter, 2002).

Procedures

Following workshop attendance, a researcher phoned each practitioner monthly for 6 consecutive months. During that call, the therapist was asked if he or she was treating a substance-abusing adolescent. If the therapist reported that he or she was treating a substance-abusing adolescent, the researcher asked the therapist to report whether he or she was using CM with up to three of these adolescents. Regardless of whether the therapist was using CM with a particular substance-abusing client, the CM Therapist Adherence Measure (CM-TAM; Chapman, Sheidow, Henggeler, Halliday-Boykins, & Cunningham, in press) was administered for up to three substance-abusing adolescent clients chosen by the therapist. If the therapist was not using CM with one of these substance-abusing clients, then the therapist was asked to complete the barriers to CM adoption survey. On average, the workshop attendees responded to 5.9 of the 6 monthly calls that were made following workshop participation. Thus, complete follow-up data are available for almost all of the participants. Practitioners received a $10 gift certificate for each completed phone interview or $100 in gift certificates if all six interviews were completed.

Dependent Measures

The primary measures of interest were whether the therapists reported use of CM (adoption) with a substance abusing adolescent client and the fidelity of CM implementation.

CM adoption

During each of the six monthly phone interviews, the research assistant asked the therapist if he or she treated any adolescents with substance use disorders during the past 30 days. If the therapist answered “no,” the interview was considered complete and was discontinued. If the therapist answered “yes,” he or she was asked whether CM was used to treat any of these clients, and this information was recorded. Thus, the fundamental measure of CM adoption was based on therapist reports of such.

CM implementation

Therapist reports of their CM implementation were assessed with the 9-item CM-TAM, which has been developed and validated across three substantive research projects (Chapman et al., in press) that included 3,629 CM-TAM administrations across 696 respondents (i.e., youths, caregivers, and therapists). The CM-TAM measures the level of adherence to both the cognitive behavioral techniques and monitoring techniques utilized in CM. Cognitive-behavioral aspects are measured with five items on a four-point (0, 1, 2, 3) scale assessing identification of drug use triggers and development of drug refusal and avoidance skills. Four items on a three-point (0, 1, 2) scale assess monitoring aspects of CM, including drug testing, rewarding, and consequenting. Cognitive behavioral and monitoring scores were computed as average ratings on the corresponding items. As described by Chapman et al. (in press), the cognitive behavioral and monitoring scales are highly reliable, and their discriminant and predictive validity have been supported in two other independent projects (Henggeler et al., 2006; Henggeler, Sheidow, Cunningham, Donohue, & Ford, in press). For example, findings from Henggeler et al. (in press) indicated that therapist reported adherence (i.e., CM-TAM scores) was significantly associated with reductions in biological and self-report measures of marijuana use among the substance abusing youths being treated by these therapists. Together, these findings support the validity of the therapist reported CM-TAM as a measure of the fidelity of CM implementation.

Predictor Measures

Consistent with the literature on the adoption and implementation of evidence-based practices (Fixsen et al., 2005), predictors of CM adoption and initial implementation indexed several pertinent aspects of the substance abuse treatment context.

Practitioner demographics, professional experience, and service sector

The Personnel Data Inventory (Schoenwald, 1998) was used to collect demographic data (i.e., age, gender, and minority status), information regarding therapists' professional background and experience (i.e., highest degree, addiction certification, discipline [education, psychology, social work], years of experience, and years in present position), and caseload characteristics (i.e., caseload size, proportions of youths in caseload, and proportion of substance abusing clients). In addition, practitioner service sector was assessed (i.e., mental health versus substance abuse sector).

Practitioner attitudes

Therapist attitudes toward treatment manuals, evidence-based treatments, and the usefulness of several types of interventions were assessed. Practitioner attitudes toward the use of treatment manuals were assessed using a modified version of a questionnaire developed and validated by Addis and Krasnow (2000) for doctoral-level psychologists. The wording of several of the original items was revised to increase suitability for administration to master's- and bachelor's-level public sector practitioners with various training backgrounds. In addition, two items with similar content were combined into a single item. According to Addis and Krasnow, principal-components analyses of the original 17 items, rated on 5-point Likert scales (1 = strongly disagree, 5 = strongly agree), suggest a two-factor structure: Positive Outcomes and Negative Process. Positive Outcomes items reflect practitioner perceptions that treatment manuals are valuable in guiding clinicians toward favorable outcomes with their clients (e.g., “Following a treatment manual helps get better outcomes.”). Negative Process items characterize treatment manuals as having a dehumanizing effect on the therapeutic process and emphasizing technique at the expense of relationship skills (e.g., “Using a treatment manual keeps a therapist from using his or her intuition.”). These attitudes have been shown to vary in predictable directions with practitioner theoretical orientations and work settings. Internal consistency estimates were computed for the present modified Positive Outcomes and Negative Process scales, accounting for the nesting of individual item responses within therapists and therapists within agencies according to the random-effects measurement model detailed by Raudenbush, Rowan, and Kang (1991). The estimated internal consistency of the Positive Outcomes and Negative Process scales within therapists was .67 and .88, respectively.

The Evidence-Based Practice Attitude Scale (EBPAS; Aarons, 2004) was used to assess general attitudes toward the adoption of evidence-based practices. The EBPAS is comprised of 15 items that are scored on 5-point Likert scales (1 = not at all and 5 = to a very great extent). Four theoretically derived subscales are included (i.e., appeal, requirements, openness, divergence) that assess: the intuitive appeal of evidence-based practices for adoption, extent of practitioner adoption if required by authorities, practitioner openness to new interventions, and practitioner perceptions of evidence-based practices as less relevant than clinical experience (i.e., divergence). The subscales have demonstrated strong internal consistency, and scores have varied with important practitioner and organizational characteristics (Aarons, 2004; 2005).

Finally, items from the Child and Adolescent Psychotherapy Survey (Kazdin, Siegel, & Bass, 1990) were used to measure practitioners' perceptions of the usefulness of three clinical approaches (i.e., cognitive behavior therapy, behavior therapy, and family therapy) that are widely recognized by practitioners across the service sectors. Respondents rated the usefulness of these approaches on 5-point Likert scales (1 = never useful and 5 = almost always or always useful).

Organizational characteristics

The Organizational Readiness for Change instrument - Program Staff Version (ORC-S; Lehman, Greener, & Simpson, 2002) is based on Simpson's (2002) aforementioned framework for transferring research to practice. The ORC includes 129 Likert-type items that comprise scales measuring motivational readiness (i.e., perceived need and pressure for change, immediate training needs), adequacy of resources (e.g., offices, staffing, training, computer access, e-communications), staff attributes (e.g., growth, efficacy, influence, adaptability), organizational climate (e.g., clarity of mission, cohesion, autonomy, communication, stress, change), and training exposure and utilization. The ORC has been widely used in the area of substance abuse services research, and the reliability and validity of its scales are supported by several studies published in a recent issue of the Journal of Substance Abuse Treatment (Simpson & Flynn, 2007). The ORC scales were developed for use primarily at the organizational level, and for the present study, analyses of within-organization interrater agreement provided justification for agency-level aggregation with r_wg values ranging from .82 to .90, and ICCs of .15, .33, .07, .17, and .18 for motivational readiness, adequacy of resources, staff attributes, organizational climate, and training exposure and utilization, respectively. Thus, practitioner responses for the ORC scales were aggregated within each of the 44 participating agencies.

Survey of Barriers to CM Adoption

If, during the monthly call, the therapist reported that he or she had treated a substance-abusing adolescent, but did not use CM with that client, the research assistant administered the Barriers to Contingency Management Adoption survey regarding that client (Halliday-Boykins, Chapman, Rowland, Armstrong, & Schoenwald, 2005). This survey was developed by the investigators to tap barriers to CM adoption and implementation that have been identified in the literature (Kirby, Amass, & McLellan, 1999; Petry & Simcic, 2002), and the survey was piloted with approximately 12 therapists who treated adolescent substance abuse and did not participate in the present study. The final iteration of the survey included 40 items that tapped system/agency level barriers (e.g., “I can not bill for contingency management.”), philosophical barriers consistent with negative attitudes toward evidence based treatments (e.g., “I do not like to use manuals in treatment.” “The contingency management philosophy does not fit with my philosophy for treating adolescent substance abuse.”), implementation difficulty (e.g., “Contingency management is too hard.”), client related barriers (e.g., “The parents of my substance abusing youth clients would not agree to contingency management.”), and competing clinical priorities (e.g., “My substance abusing youth clients have too many problems to participate in contingency management.”).

Data Analyses

As implied by the research procedures, an essential feature of the present data is the nesting of a maximum of three reports of CM use within each of six interviews that are nested within practitioners who are nested within agencies. This data structure can be accurately modeled using random-effects regression models (RRMs), where dependency in outcome variance is partitioned among each of the levels of nesting. In each of the RRMs described below, the model was limited to a three-level data structure due to (1) hypothesis-specific reductions in sample and cluster size and (2) software limitations restricting the models to a maximum of three levels of nesting. These models can readily accommodate both continuous and dichotomous outcomes as well as missing data and variability in the size of the nesting “clusters” (Raudenbush & Bryk, 2002). HLM software (Version 6.02; Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2004) was used to perform all analyses. Specification of random effects was based on the likelihood ratio test for models with continuous outcomes and the Wald test of models with dichotomous outcomes (Singer & Willett, 2003). Due to the relatively small number of agencies, asymptotic standard errors, rather than robust standard errors, were used for the computation of test statistics (Maas & Hox, 2004).

Model covariates were entered at the associated level of measurement, with the exception of the ORC domain scores that, as described previously, were aggregated at the agency level. Covariates with meaningful zero values were uncentered; otherwise, covariates were centered around their grand mean value (Kreft, de Leeuw, & Aiken, 1995). Model covariates were entered in hierarchical groups according to the nature of their theoretical domain, with historical and attitudinal domains entered before organizational domains. Thus, the order of entry was: practitioner demographics, professional experience, and service sector; practitioner attitudes; and organizational characteristics. Of note, due to a non-trivial proportion of “No Opinion” responses, models evaluating practitioners' perceptions of the usefulness of cognitive behavior therapy, behavior therapy, and family therapy as predictors of CM adoption and implementation fidelity were conducted separately. The specific RRMs for each the CM adoption and CM implementation outcomes are described in the corresponding sections of the Results.

Results

Participant Characteristics

Demographically, 81% of the 430 workshop attendees who completed at least one follow-up interview were female; 42% were African-American, 56% white non-Hispanic, 1% Hispanic, and 1% other; and their average age was 40 years. Professionally, 16% had bachelor's degrees, 82% had master's degrees, and 1% had doctorates. On average, these practitioners had more than 7 years of professional experience, and 26% were certified addictions counselors.

CM Adoption

CM adoption was evaluated according to a three-level RRM where 1 to 6 research interviews with 1 or more substance abusing youth clients per therapist (Level-1, a total of 852 interviews) were nested within the 225 therapists (Level-2) who reported having at least one substance abusing adolescent client, and who were nested within 44 agencies (Level-3). The outcome, as described previously, was a dichotomous indicator of whether the therapist reported any use of CM during a given interview. Because therapists reported on one, two, or three substance-abusing youth clients at each interview, the model was adjusted for the number of opportunities for CM adoption at each interview (i.e., the number of substance abusing youth clients). This resulted in a three-level Binomial Trial model using a logit-link function and Penalized Quasi-Likelihood estimation where the population-average results represented the average log-odds of CM adoption per substance-abusing client across interviews (Raudenbush & Bryk, 2002). A Level-1 linear polynomial term representing the interview number tested for change over time in the log-odds of CM adoption. However, models testing the predictors of CM adoption omitted the polynomial term, thereby focusing on predictors of therapists' average log-odds of adoption (i.e., averaging across interviews). The resulting coefficients were converted from log-odds units to odds ratios and probabilities according to exp(coefficient) and 1/(1 + exp{coefficient}), respectively.

The principle indicator of adoption was based on whether therapists with substance-abusing adolescent clients attempted to implement CM as recorded during the six monthly calls that were made following workshop participation. During at least one of the calls, 225 of the therapists reported treating a substance-abusing adolescent during the past 30 days. Of these, 131 practitioners reported the use of CM with at least one of their substance abusing adolescent clients during the follow-up period. Thus, 58% of the practitioners who identified the opportunity to adopt CM (i.e., they had at least one substance abusing adolescent client) decided to do so. However, recalling that therapists could report on up to three substance abusing adolescent clients per phone interview, the three-level RRM detailed above was estimated to adjust for the number of opportunities for CM adoption. For a therapist reporting on 1, 2, or 3 substance-abusing clients during an interview, the average probability of any CM adoption was 25%, 44%, or 58%, respectively. Subsequent models maintained this adjustment for the number of opportunities for CM adoption per interview.

Longitudinal Changes in Rate of Adoption

Assuming that practitioner experience with CM is positive, practitioners should be expected to increase their use of CM throughout the 6-months post workshop. Results of the three-level RRM testing for linear change over time in the log-odds of CM adoption indicated that there was a significant increase in the log-odds of adoption across the six interviews, π₁ = 0.090, SE = 0.033, T (850) = 2.71, p = .007. Specifically, the predicted probabilities of CM adoption for one substance abusing client at interviews 1 through 6 was .21, .22, .24, .26, .28, and .29, respectively. On average, the therapists actually reported on two substance-abusing clients per interview, resulting in predicted probabilities of CM adoption at interviews 1 through 6 of .38, .40, .42, .45, .47, and .50, respectively.

Predictors of CM Adoption

Results from analyses of the predictors of CM adoption are shown in Table 1. The results focus on the final RRM that includes all covariates in the model. However, in order to evaluate the independent effect of each predictor on CM adoption, the table also presents the unstandardized regression coefficient for each predictor based on a separate RRM with no other predictors in the model. The dependent measure in the RRMs was practitioner use of CM at each interview, controlling for the number of substance abusing adolescent clients reported during the respective interview.

Table 1.

Separate and Combined RRMs Evaluating Predictors of Contingency Management Adoption

	Separate RRMsa	RRM with All Predictors
Parameter	Coeff.	Coeff.	SE	OR	OR 95% CI
Note. The T ratio test statistic for each parameter (omitted) was computed as Coefficient / SE. Unless otherwise noted, the dfs for the significance tests for the π, β, and γ coefficients were 37, 202, and 37, respectively.
Intercept (π0)	−1.11***	−0.54	0.37	0.58	0.28 to 1.22
Practitioner demographics, professional experience, & service sector
Age (β01)	0.01	0.01	0.01	1.01	0.98 to 1.03
Gender (0 = male, β02)	0.05	−0.24	0.23	0.79	0.50 to 1.25
Minority (0 = caucasian, β03)	0.35	0.34	0.21	1.41	0.93 to 2.14
Degree type (0 = bachelor's or less, β04)	0.46*	0.27*	0.25	1.31	0.80 to 2.13
Addiction certification (0 = no, β05)	−0.65**	−0.28	0.24	0.75	0.47 to 1.21
Discipline: education (0 = no, β06)	0.28	0.70	0.29	2.02	1.14 to 3.59
Discipline: psychology (0 = no, β07)	−0.26	−0.10	0.31	0.90	0.49 to 1.65
Discipline: social work (0 = no, β08)	0.21	0.25	0.21	1.28	0.85 to 1.93
Years in field of counseling (β09)	−0.01	−0.01	0.02	1.00	0.97 to 1.03
Years at present job (β010)	0.01	0.04*	0.02	1.05	1.00 to 1.09
Size of caseload (β011)	−0.01	−0.07	0.07	0.93	0.82 to 1.07
Proportion youth clients (β012)	0.10	−0.02	0.08	0.98	0.83 to 1.15
Proportion youth sub abuse clients (β013)	−0.17*	−0.01	0.08	1.00	0.85 to 1.17
Service sector (0 = mental health, γ001)	−1.16***	−1.36**	0.36	0.26	0.12 to 0.54
Practitioner attitudes
Negative process (β014)	−0.07	0.05	0.16	1.05	0.76 to 1.45
Positive outcomes (β015)	0.39	0.47*	0.20	1.60	1.09 to 2.37
Requirements (β016)	0.16	0.23*	0.10	1.26	1.02 to 1.55
Appeal (β017)	0.10	−0.07	0.17	0.94	0.67 to 1.32
Openness (β018)	0.11	−0.01	0.16	0.99	0.72 to 1.37
Divergence (β019)	0.10	0.04	0.14	1.05	0.80 to 1.37
Cognitive behavior therapyb (β01)	0.17
Behavior therapyb (β01)	0.30*
Family therapyb (β01)	0.02
Organizational characteristics
Motivation for change (γ002)	−0.06	0.01	0.42	1.01	0.43 to 2.38
Resources (γ003)	−0.59	0.11	0.48	1.12	0.43 to 2.95
Staff attributes (γ004)	−0.31	−0.84	1.05	0.43	0.05 to 3.63
Organizational climate (γ005)	−0.18	0.86	0.63	2.37	0.66 to 8.53
Training (γ006)	−1.05	0.23	0.50	1.26	0.46 to 3.48

^aThis column provides the unstandardized regression coefficient for each predictor with no other predictors in the Binomial Trial RRM.

^bDue to a non-trivial proportion of missing data on this predictor (i.e., “No Opinion” responses), it was evaluated in the separate model only. The dfs for Cognitive Behavior Therapy, Behavior Therapy, and Family Therapy were 210, 206, and 211, respectively.

^*p<.05

^**p<.01

^***p<.001

Practitioner demographics, professional experience, and service sector

Significantly increased probability of CM use was found for practitioners in the mental health sector and by therapists with more education and more years in their present position. CM use was not associated with participant demographic characteristics or educational discipline (i.e., social work, psychology, education).

Practitioner attitudes

Analyses of the three practitioner attitude instruments yielded three significant predictors of therapist CM adoption. Significantly increased probability of CM use was found for practitioners who held more favorable attitudes toward behavior therapy interventions (Kazdin et al., 1990) and the Positive Outcomes that could be facilitated by treatment manuals (Addis & Krasnow, 2000). In addition, increased probability of CM use was associated positively with therapists' indications that they would agree to adopt an innovation if mandated by agency leadership (Aarons, 2004).

Organizational characteristics

CM adoption was not predicted by the organizational level measures.

Barriers to CM Adoption

The barriers to CM adoption were assessed in those cases when the therapist indicated that a substance-abusing adolescent was not being treated with CM. In general, the most frequently cited barriers to CM adoption pertained to competing clinical priorities and lack of youth and family engagement in treating substance abuse. Regarding the former, therapists essentially contended that the treatment of adolescent substance abuse was a low priority in light of the many other competing clinical problems presented by the youth and his or her family (cited by 74% of therapists) and their busy schedules (cited by 58%). In addition, therapist reports that the substance-abusing clients did not view their substance use as a problem (77%) and would not agree to use contingency management (61%) were cited frequently as barriers to CM adoption. On the other hand, the least frequently cited barriers to CM adoption pertained to philosophical incompatibility and difficulty implementing CM. For example, relatively few practitioners reported dislike of vouchers (27%) or manuals (27%) as barriers to CM adoption. Likewise, various difficulties in applying CM (30% to 40%) were not often cited as barriers to adoption. Agency level barriers varied widely in their range of endorsement. On the high end, 57% of practitioners reported that they referred substance abusing youth clients to other treatment providers for substance abuse treatment. On the low end, relatively few therapists reported billing (28%) or lack of agency support (28%) as barriers to CM adoption.

Level of CM Implementation

Importantly, the level of CM implementation fidelity for therapists in the present study can be evaluated by contrasting their CM-TAM scores with those of counterparts in two recent independent CM studies (Henggeler et al., 2006; Henggeler et al., in press). Thus, the average therapist reports of CM cognitive behavioral and monitoring adherence in the present study were compared with their counterparts in the four-condition (i.e., family court, drug court, drug court with MST, drug court with MST and CM) juvenile drug court study (Henggeler et al., 2006) and two-condition (workshop in CM only, intensive CM training) CM transportability study (Henggeler et al., in press) after rescaling the scores across studies to range from 0 to 1. Supporting the strength of CM implementation by therapists in the present study, their average cognitive behavioral scores were significantly higher than those of two of the drug court study conditions (i.e., family court, drug court with MST) and both of the CM transportability study conditions, -0.28 ≤ βs ≤ -0.16, SEs < .08, Ts (674) ≥ -2.40, ps ≤ .02. Likewise, the average monitoring scores for therapists in the present study were significantly higher than those of counterparts in the family court condition in the juvenile drug court study β₀₁ = -0.13, SE = .05, T (674) = -2.81, p = .01. The present therapists, however, had significantly lower monitoring scores than counterparts in the three drug court conditions (which require weekly drug urine screens as part of participation in juvenile drug court) in the juvenile drug court study and both conditions in the CM Transportability study, 0.11 ≤ βs ≤ 0.25, SEs < .05, Ts (674) ≥ 2.57, ps ≤ .01.

A further test of the degree of implementation can be determined by evaluating differences in the level of CM implementation between substance abusing youth who received CM in the present study and counterparts who did not. Because therapists completed the CM-TAM for 1 to 3 substance-abusing youth clients (i.e., both CM and non-CM cases) at each of six interviews, there was a maximum of 18 CM-TAM reports per therapist. Thus, outcome was modeled according to a three-level RRM with 1 to 18 substance abusing youth clients per therapist (Level-1, a total of 1,829 reports on clients) nested within 225 therapists (Level-2) nested within 44 agencies (Level-3). A Level-1 dichotomous indicator was included to differentiate CM from non-CM reports, testing for differences in the average level of cognitive-behavioral and monitoring adherence between the two types of reports. Results showed that both cognitive behavioral and monitoring scores were significantly higher in CM cases with substance abusing adolescents than in corresponding cases where CM was not used, π₁= 0.25, SE = 0.06, T (224) = 4.13, p < .001, and π₁ = 0.40, SE = 0.04, T (224) = 9.31, p < .001, respectively.. In sum, these data indicate a nontrivial level of CM implementation by therapists in the present study.

Longitudinal Changes in Fidelity of CM Implementation

Therapists who implement CM with their substance-abusing clients might be expected to improve their fidelity with increasing experience. Thus, to test whether cognitive behavioral and monitoring adherence scores changed over time for CM cases, non-CM cases were removed, and a three-level RRM with 1 to 6 interviews per therapist (Level-1, a total of 448 interviews) nested within 131 therapists (Level-2) nested within 40 agencies (Level-3) was performed. The outcomes were computed as the average cognitive behavioral and average monitoring score for all CM cases at a given interview occasion, and a linear polynomial term representing the interview number was entered at Level-1 for cognitive-behavioral and monitoring adherence. Results showed that adherence to both the cognitive behavioral and monitoring components increased significantly over the six interviews, π₁ = 0.066, SE = 0.016, T (130) = 4.23, p < .001 and π₁ = 0.055, SE = 0.013, T (130) = 4.19, p < .001, respectively. The average cognitive behavioral score (Range = 0-3) at the first interview was 1.98, and the predicted average by the final interview was 2.31. Similarly, the average monitoring score (Range = 0-2) at the first interview was 0.88, and the predicted average by the final interview was 1.16.

Predictors of the Fidelity of CM Implementation

RRMs were used to evaluate the predictors of therapist adherence to the CM implementation protocol (Cunningham et al., 2004) for cases in which CM was used. Analyses were conducted separately for the cognitive behavioral and monitoring components of the CM-TAM. For these models, the non-CM cases were removed, leaving CM cases nested within participants (N = 131) who were nested within agencies. Specifically, a three-level RRM with 1 to 18 CM cases (Level-1, a total of 841 reports on clients) nested within 131 therapists (Level-2) nested within 40 agencies (Level-3) was estimated. The model intercept represented the average cognitive behavioral and monitoring score for CM cases across therapists and agencies, and as described previously, model covariates were entered at the associated level of measurement. Each model was performed using Restricted Maximum Likelihood estimation. The 95% confidence interval for each of the resulting coefficients was computed as (coefficient ± 1.96 × SE). Additionally, the percentage reduction in total model variance associated with the inclusion of each group of predictors relative to the previous model was computed as a pseudo-R² statistic as described by Snijders and Bosker (1999, p. 102). Table 2 presents the results on the final RRM that included all covariates in the model as well as the unstandardized regression coefficient for each predictor based on a separate RRM with no other predictors in the model.

Table 2.

Separate and Combined RRMs Evaluating Predictors of Cognitive Behavioral and Monitoring Adherence to Contingency Management

	Cognitive Behavioral Adherencea				Monitoring Adherenceb
	Separate RRMsc	RRM with All Predictors			Separate RRMsc	RRM with All Predictors
Parameter	Coeff.	Coeff.	SE	Coefficient 95% CI	Coeff.	Coeff.	SE	Coefficient 95% CI
Note. The T ratio test statistic for each parameter (omitted) was computed as Coefficient / SE. Unless otherwise noted, the DFs for the significance tests for the π, β, and γ coefficients were 33, 110, and 33, respectively.
Intercept (π0)	2.14***	1.96***	0.25	1.47 to 2.45	1.02***	1.13***	0.15	0.84 to 1.42
Practitioner demographics, professional experience, & service sector
Age (β01)	−0.01	−0.02*	0.01	−0.04 to 0.00	−0.01	−0.01*	0.01	−0.03 to 0.01
Gender (0 = male, β02)	0.01	−0.21	0.16	−0.52 to 0.10	−0.17	−0.33**	0.10	−0.53 to−0.13
Minority (caucasian = 0, β03)	−0.07	0.07	0.15	−0.22 to 0.36	−0.06	−0.05	0.09	−0.23 to 0.13
Degree type (0 = bachelor or less, β04)	−0.06	0.05	0.18	−0.30 to 0.40	−0.02	0.08	0.11	−0.14 to 0.30
Addiction certification (0 = no, β05)	0.25	0.51**	0.18	0.16 to 0.86	0.18*	0.29**	0.11	0.07 to 0.51
Discipline: education (0 = no, β06)	0.16	0.29	0.19	−0.08 to 0.66	−0.03	0.12	0.11	−0.10 to 0.34
Discipline: psychology (0 = no, β07)	0.09	0.11	0.21	−0.30 to 0.52	0.06	−0.02	0.13	−0.27 to 0.23
Discipline: social work (0 = no, β08)	0.02	0.17	0.14	−0.10 to 0.44	0.14	0.25**	0.08	0.09 to 0.41
Years in the field of counseling (β09)	0.01	0.01	0.01	−0.01 to 0.03	−0.01	−0.01	0.01	−0.03 to 0.01
Years present job (β010)	0.01	−0.01	0.02	−0.04 to 0.04	0.01	0.01	0.01	−0.01 to 0.03
Caseload size (β011)	0.11*	0.12**	0.04	0.04 to 0.20	−0.01	−0.02	0.03	−0.08 to 0.04
Proportion youth clients (β012)	0.01	0.12*	0.06	0.00 to 0.24	−0.05	−0.02	0.03	−0.08 to 0.04
Proportion youth sub abuse clients (β013)	0.06	−0.04	0.06	−0.16 to 0.08	0.07*	0.02	0.04	−0.06 to 0.10
Service sector (0 = mental health, γ001)	0.21	−0.01	0.26	−0.52 to 0.50	0.21*	−0.18	0.16	−0.49 to 0.13
Practitioner attitudes
Negative process (β014)	−0.18	−0.23*	0.11	−0.45 to−0.01	−0.10	−0.11	0.07	−0.25 to 0.03
Positive outcomes (β015)	0.12	0.16	0.15	−0.13 to 0.45	0.01	−0.05	0.09	−0.23 to 0.13
Requirements (β016)	−0.07	0.06	0.07	−0.08 to 0.20	0.34	0.05	0.04	−0.03 to 0.13
Appeal (β017)	−0.14	−0.19	0.12	−0.43 to 0.05	0.03	−0.01	0.07	−0.15 to 0.13
Openness (β018)	0.10	0.07	0.11	−0.15 to 0.29	0.04	0.05	0.07	−0.09 to 0.19
Divergence (β019)	−0.05	0.02	0.09	−0.16 to 0.20	−0.06	−0.02	0.05	−0.12 to 0.08
Cognitive behavior therapyd (β01)	0.02				−0.01
Behavior therapyd (β01)	0.11				−0.01
Family therapyd (β01)	0.09				−0.01
Organizational characteristics
Motivation for change (γ002)	0.54	0.70*	0.31	0.09 to 1.31	0.19	0.25	0.19	−0.12 to 0.62
Resources (γ003)	0.32	0.43	0.32	−0.20 to 1.06	0.27*	0.10	0.19	−0.27 to 0.47
Staff attributes (γ004)	0.33	−1.32	0.75	−2.79 to 0.15	0.50	−0.40	0.44	−1.26 to 0.46
Organizational climate (γ005)	0.23	0.47	0.47	−0.45 to 1.39	0.21	0.16	0.28	−0.39 to 0.71
Training (γ006)	0.42	0.40	0.35	−0.29 to 1.09	0.45*	0.43*	0.21	0.02 to 0.84

^aRange = 0-3.

^bRange = 0-2.

^cThis column provides the unstandardized regression coefficient for each predictor with no other predictors in the RRM.

^dDue to a non-trivial proportion of missing data on this predictor (i.e., “No Opinion” responses), it was evaluated in the separate model only. The dfs for Cognitive Behavior Therapy, Behavior Therapy, and Family Therapy were 122, 120, and 122, respectively.

^*p<.05.

^**p<.01.

^***p<.001.

Practitioner demographics, professional experience, and service sector.

Several significant predictors of the fidelity of CM implementation were observed. The fidelity of the cognitive behavioral component of CM was greater for therapists certified in addictions counseling, therapists with larger caseloads, therapists with a larger proportion of youths on their caseloads, and younger practitioners. The fidelity of the monitoring component of CM was significantly higher for therapists certified in addictions counseling, male therapists, practitioners with degrees in social work, and younger therapists. Collectively, practitioner demographics, professional experience, and service sector explained 10% and 9% of the total model variance in cognitive behavioral and monitoring components of adherence, respectively.

Practitioner attitudes

In only one instance did therapist attitudes toward treatment manuals (Addis & Krasnow, 2000), evidence-based practices (Aarons, 2004), or specific intervention models (Kazdin et al., 1990) predict the fidelity of CM implementation. Practitioners who did not view treatment manuals as having a dehumanizing effect on the therapeutic process had greater fidelity to the cognitive behavioral component of CM. Collectively, practitioner attitudes explained an additional 9% and 0% of the model variance in cognitive behavioral and monitoring components of adherence, respectively, relative to the previous model. In separate models, practitioners' ratings of the usefulness of cognitive behavior therapy, behavior therapy, and family therapy explained less than 2% and 1% of the model variance, respectively, relative to the previous model.

Organizational characteristics

Two significant organizational-level predictors of the fidelity of CM implementation were observed. Higher organizational motivational readiness to change was associated with greater fidelity to the cognitive behavioral components of CM, and greater organizational training exposure and utilization was associated with increased fidelity to the monitoring components. Collectively, organizational characteristics explained an additional 6% and 10% of the model variance in cognitive behavioral and monitoring components of adherence, respectively, relative to the previous model.

Discussion

In an effort to better understand and bridge the science-service gap, the primary purposes of this study were to examine the adoption of an evidence-based treatment of adolescent substance abuse among a large sample of public sector practitioners, examine the key predictors of such adoption, identify barriers to adoption, and evaluate the quality of initial implementation and the key predictors of such.

In the context of a modest training experience (i.e., attendance at a 1-day CM workshop) and access to the resources needed to implement CM (e.g., manuals, client incentives, drug test kits), the majority of the workshop attendees (58%) attempted to use CM if they identified at least one substance abusing adolescent client. Although a lower percentage of mental health sector therapists than substance abuse sector therapists (38% versus 85%) reported having a substance abusing adolescent client, mental health sector therapists with such clients were more likely to adopt (defined by at least one attempted use) CM than were their substance abuse sector counterparts (65% versus 52%). Moreover, the longitudinal analyses showed that, across service sectors, the rate of CM adoption increased steadily throughout the 6-months post workshop. These findings suggest that the public sector therapists participating in this project, especially those in the mental health sector, were generally amendable to adopting this evidence-based treatment for substance abuse.

The final RRM (see Table 1), which included all covariates in the model (i.e., controlled for practitioner demographics, professional experience, service sector, and attitudes; and organizational characteristics), helped to clarify the influences on CM adoption. The findings showed that CM adoption was greater among therapists who were more educated and more experienced in their present position - suggesting that such therapists are better able to judge the utility of an innovative evidence-based intervention such as CM. Findings also showed that therapists who were less likely to already have the tools needed to treat adolescent substance abusers (i.e., practitioners in mental health system) and were favorably inclined toward the type of tool that was offered (i.e., favorable attitudes toward behavior therapy, view that treatment manuals help achieve better client outcomes) were more likely to adopt CM. Moreover, therapists who held more cooperative attitudes toward being required by agency leadership to adopt an innovation were more likely to have used CM. In sum, the findings suggest that intervention adoption is more likely to occur among relatively experienced and well educated practitioners who are treating a problem that pertains to their clients, but for which they do not have ready expertise; and when the intervention is compatible (Rogers, 2003) with their attitudes toward the type of intervention that is being adopted and such adoption is supported by agency leadership. These variables reflect a near ideal confluence of perceived need, availability of compatible interventions, and willingness to be a team player.

Findings from the survey of barriers to CM adoption most likely reflect the clinical environments experienced by many public sector practitioners. Among the most frequently cited reasons for not using CM pertained to the complex array of problems presented by the substance abusing youths and their families and the clients' reluctance to use CM. Adolescent clients and their families in public sector treatment settings often present multiple behavioral, psychiatric, educational, and social difficulties (Institute of Medicine, 1998; U.S. Public Health Services, 1999); and the lack of enthusiasm that many substance abusing clients have for substance abuse treatment is well established (Stark, 1992). Such challenges to implementing CM are realistic and are likely best addressed through increased clinical support (e.g., reduced caseload size, increased supervisory support) and training (e.g., in clinical engagement) for practitioners. Belying the common view that practitioner attitudes are barriers to CM adoption (McCarty et al., 2007; Petry & Simcic, 2002), practitioner philosophical or implementation difficulties with the various components of CM were least often cited as barriers to CM adoption. This finding is consistent with the aforementioned amenability of the practitioners in this study to adopt CM. In a similar vein, agency regulations were not often cited as barriers to CM adoption, except in cases where substance-abusing clients were routinely referred to other provider organizations. Lastly, and unfortunately, barriers to CM adoption for therapists who did adopt CM were not examined in this study. Otherwise, strategies for overcoming the aforementioned barriers identified by the non-adopters might have been revealed.

Predictors of the fidelity of the cognitive-behavioral and monitoring aspects of the initial implementation of CM were assessed for the 131 practitioners who adopted CM for at least one of their substance abusing adolescent clients. The final RRM analyses, which included all covariates in the model, showed that the fidelity of the cognitive-behavioral aspects of CM were predicted by working in an organization with high motivational readiness for change, being certified in addictions counseling, having high caseloads and a high percentage of youth clients, being of younger age, and not holding negative attitudes toward treatment manuals. The organizational finding provides further support for Simpson's (2002) conceptual framework for transferring research to practice. Simpson emphasized that organizational motivational readiness is critical for moving to the adoption and implementation stages of technology transfer. Interestingly, as noted previously in the adoption analyses, mental health sector therapists were more likely than their substance abuse sector counterparts (especially certified addictions counselors, see Table 1, coefficient for separate RRM) to try CM. Yet, the present analyses showed that certified addictions counselors had relatively high cognitive behavioral fidelity scores when they did use CM. This finding likely reflects their greater experience in providing the types of cognitive-behavioral interventions that are part of CM (e.g., drug refusal skill training). Similarly, higher fidelity scores for practitioners with larger caseloads and greater percentages of youth clients likely reflect their greater use of cognitive behavioral interventions in group therapy settings, in comparison with, for example, family therapy or psychodynamic interventions. Finally, in support of the importance of practitioner attitudes in the implementation of evidence-based practices (Godley et al., 2001), it seems likely that younger, less experienced therapists and those who do not view treatment manuals as interfering with the therapeutic process are more likely to follow such manuals.

Regarding fidelity to monitoring aspects of CM, controlling for all the types of covariates in the model, high fidelity was predicted by male therapist gender, certification in addiction counseling, social work degree, younger therapist age, and increased exposure to training in the therapist's provider organization. A central component of monitoring is the collection of drug urine screens, and male therapists might be more amenable and comfortable in collecting such screens from the adolescent males who comprise the majority of substance abusing clients. By virtue of their community-based training, social workers are likely more comfortable than traditionally trained counselors in implementing the relatively intrusive CM monitoring techniques. Similarly, the professional training received by certified addiction counselors and by practitioners in organizations that provide more training exposure and utilization likely accounts for their higher fidelity scores for CM monitoring. Again, although certified addictions counselors were less likely to adopt CM, they implemented CM with greater fidelity when it was adopted. Lastly, younger, less experienced therapists might be more receptive to implementing monitoring techniques. Together, these findings showed that CM implementation fidelity was predicted across demographic, experiential, attitudinal, and organizational levels of interest.

Finally, and importantly, several findings suggest that the practitioners were gaining increased expertise in the use of CM and that their level of initial implementation fidelity was not inconsequential. The longitudinal analyses showed that average fidelity scores for both cognitive behavioral and monitoring components of CM increased throughout the 6-month follow-up, as the practitioners gained more experience in using CM. In addition, as described more extensively in the Results section, the therapists' average fidelity scores for the cognitive behavioral aspects of CM were at a par with clinicians who have had much more intensive CM training, and their average fidelity scores for the monitoring aspects of CM were higher than those of substance abuse counselors not trained in CM. Such finding are encouraging in light of the established relationship between CM fidelity scores and youth outcomes in previous studies (Chapman et al., in press).

Clinical and Policy Implications

The findings have several implications for bridging the gap between science and practice in the treatment of adolescent substance abuse. First and most importantly, the majority of the public sector practitioners who reported having at least one substance abusing adolescent client also reported the adoption of CM with at least one of those clients. This suggests relatively widespread amenability to the use of a particular evidence-based treatment that has often been thought to conflict with the philosophies and values of treatment providers (Petry & Simcic, 2002). Stakeholders who are advancing the evidence-based practice movement should view such findings optimistically. Second, mental health sector practitioners were especially amenable (i.e., 65%) to adopting CM if they had a substance-abusing adolescent client, though only 38% of mental health practitioners reported having such a client. In light of the limited availability of evidence-based substance abuse treatment for adolescents (e.g., Institute of Medicine, 1998; McLellan, Carise, & Kleber, 2003), this findings suggests that the mental health sector holds the potential to greatly increase the availability of evidence-based substance abuse services to youths. Third, consistent with the perspectives of recent reviewers (e.g., Fixsen et al., 2005; Schoenwald & Hoagwood, 2001), the findings suggest that different types of variables are of greater or lesser significance at different phases of the diffusion process. For example, service sector and therapist attitudes were significant predictors of CM adoption, but organizational variables were not. Clearly, the field of implementation science is only in its beginning stages in understanding the multifaceted drivers of the diffusion process.

Limitations

The study includes several limitations pertaining primarily to external and internal validity. Although the vast majority of public sector substance abuse and mental health practitioners across the state participated in this project, generalization is limited by the fact that private practitioners and those working in institutional settings were not recruited to participate. Likewise and second, the findings cannot be fully generalized to other evidence-based practices, especially those practices that differ substantively from CM (e.g., relationally oriented family-based treatments). Third, adoption was operationalized as at least one attempted use of CM with a substance-abusing client. A more conservative definition (e.g., use of CM for 4 consecutive months) would result in lower rates of adoption and might evidence different predictors of such. Fourth, although care was taken to describe findings that support the validity of the key implementation measure (i.e., therapist reports on the CM-TAM), multiple perspectives of CM implementation were not obtained for cost and logistical reasons (e.g., high number of participating sites and practitioners, need to consent and interview hundreds of youths and families across the state). Fifth, there was no way to reliably track whether therapists were reporting on the same or different youth over the course of the six assessments. The consequence of this was a reduced ability to accurately partition outcome variance between youth and therapists. Sixth, sustainability of CM implementation, the fourth stage of the diffusion of innovation (Rogers, 2003), was not examined. Planned sustainability research, however, is currently being conducted.

Conclusion

The present findings in conjunction with other recent research (McCarty et al., 2007) suggest that public sector practitioners are more receptive to the adoption and initial implementation of evidence-based substance abuse treatments than is commonly assumed. However, as emphasized by reviewers (Fixsen et al., 2005; Schoenwald & Hoagwood, 2001) and further supported by this study, the predictors of innovation adoption and initial implementation are complex and multifaceted. Moreover, little is known about the conditions that support the fidelity of implementation of evidence based practices or the sustainability of these innovations. Although research in these issues is costly and challenging to conduct, such work is critical for narrowing the gap between science and service.