Description of a Standardized Treatment Center That Utilizes Evidence-Based Clinic Operations to Facilitate Implementation of an Evidence-Based Treatment

Abstract

Developers of evidence-based therapies are enhancing methods of teaching therapists to implement “best practices” with integrity. However, there is a relative dearth of information available as to clinic operations and related contextual factors necessary to sustain successful implementation of these treatments. This article describes various evidence-based administrative strategies and methods utilized by clinic staff to effectively implement a comprehensive evidence-based treatment for substance abuse (i.e., Family Behavior Therapy). The basic structure of the clinic, standardized behavioral methods associated with its day-to-day operations, and maintenance of treatment integrity are delineated. Infrastructural systems are underscored, including clinical record keeping, quality assurance, and staff management.

Evidence-based therapies (EBTs) in mental health settings are becoming increasingly prevalent (American Psychological Association, 2006). Indeed, funding institutions have made it clear to community-based mental health agencies that financial support is contingent on implementation of EBTs (Rapp et al., 2005). While developers of EBTs have been able to teach therapists to implement these interventions with considerable treatment integrity in efficacy trials, similar precision has not occurred in community service providers (Morgenstern et al., 2001). Successful translation of EBTs into community based clinics is dependent on establishment of procedures that facilitate integration of EBTs into the existing service provision structure, and support ongoing implementation and monitoring of evidence based services. Indeed, little is known about clinic procedures and related contextual factors that facilitate implementation of EBTs with integrity, particularly standardized strategies that are evidence-based (Miller, Zweben, & Johnson, 2005). As reported by McLellan and Meyers (2004), the administrative and organizational infrastructures of substance abuse programs, in particular, are often “fragile and unstable,” leading others to recently develop and evaluate methods to assist in clinic management of evidence-based treatments (Pritchard, Harrell, DiazGranados, & Guzman, 2008). Some of these methods include optimum supervision and training formats, techniques to enhance programmatic decisions, and ongoing training in effective strategies for administrators (Rousseau & McCarthy, 2007). These strategies are sometimes integrated within evidence-based treatment programs, but are rarely described in detailed format that is readily available (Pritchard et al., 2008).

Therefore, this article describes a “real-world” integration of evidence-based clinic methodologies used to support Family Behavior Therapy (FBT) in substance abusers. FBT is a comprehensive EBT that has demonstrated long-term positive outcomes in both adult and adolescent substance abuse (Azrin, Acierno et al., 1996; Azrin, Donohue, Besalel et al., 1994; Azrin, Donohue, Teichner et al., 2001; Azrin, McMahon et al., 1994; Donohue & Azrin, 2002; Donohue et al., 1998). FBT has evolved into 20 standardized interventions to address a wide array of problems, including depression, HIV prevention, conduct disorders, unemployment, family discord, and child maltreatment. There is a formalized program orientation, and methods of establishing behavioral goals and contingency contracting to support completion of goals, treatment plans, and assurance of basic necessities. Adult and adolescent behavioral treatments include self-control, stimulus control, communication skills training, job getting skills training, and financial management. There is also a child management skills training program that is specific for substance abusing parents. There are interventions that can be concurrently implemented with children of substance abusing parents to increase their perceived reinforcement value of children, thus acting to decrease risk of child maltreatment. Family-based treatments are focused on teaching family members to express appreciation and compliment one another. Interventions are also available to engage family members in safety tours of their home to remove hazards, enhance cleanliness, and cultivate an environment that is stimulating and developmentally appropriate for children. Interventions are implemented in therapy offices, or patient homes, and patients determine the order and extent to which therapies are emphasized. FBT was the first comprehensive behavioral therapy to demonstrate positive outcomes in adolescent substance abusers utilizing controlled methodology (see National Institute on Drug Abuse, 1998). Among evidence-based practices in the National Registry of Evidence-based Programs and Practices (NREPP-SAMHSA, 2008), FBT is rated among the top programs in its ability to be disseminated to community mental health settings. Consistent with this dissemination emphasis, this article, 1) delineates the basic structure of a clinic for which FBT is the chief intervention, and 2) describes standardized programmatic methods associated with the maintenance of treatment integrity and program integration.

Description of Clinic

Achievement Center (AC) is a research-based clinic involved in the continued development and dissemination of FBT. It was originated to develop EBTs for use in community contexts, and as such its membership is focused in the development of systematic methods to improve clinic efficiency. Its physical layout resembles those found in community-based treatment agencies. There is a small waiting room, 5 therapy rooms/offices (each equipped with an Internet capable computer, desk, and phone), a conference room with a supply cabinet, 2 chalkboards, table and chairs to accommodate up to 25 individuals, a locked filing room to store client confidential records and program forms, a fax machine printer, and conference phone to assist in off-site supervision and training of program therapists. However, there are also some unique features specific to EBT adoption, such as a personalized electronic library of controlled treatment outcome studies involving substance abuse disorders.

Clinical Governance

Implementation of program policies and standards of care is enhanced by clearly specified and well-organized governance structures (Ansell, 2002). Therefore, a hierarchy denoting various positions and corresponding responsibilities at the AC clinic is clearly specified. Although each of these positions is filled by a person who is trained to conduct some aspect of research, the positions are otherwise relatively consistent with community-based treatment programs. For instance, there is a doctoral level Program Director and Masters level Program Coordinator, while other administrative positions are filled at the bachelor level. Consultants are involved as needed to assist in collaborative auditing (Buetow & Roland, 1999). The remaining staff members are at the bachelor’s level, and hold Clinical and Research Assistant positions under the mentorship of the aforementioned administrators. All administrators and a few of the assistant positions maintain clinical caseloads ranging from 2 to 7 clients. The clinicians are not licensed, although they are supervised by a licensed psychologist. Persons providing direct services are students enrolled in undergraduate and graduate programs, and members in the community. A ubiquitous administrative principle is that every member in the clinic has a position title that is associated with behaviorally defined responsibilities and work products essential to the successful implementation of FBT and linked to specific training requirements and competencies. Although staff members have specific responsibilities, they are routinely cross-trained to facilitate teamwork (Marks, Sabella, Burke, & Zaccaro, 2002) and provide continuity of service provision when absenteeism occurs.

Recruitment of Program Staff and Volunteers

A positive and productive workplace environment is fostered through membership retention, and enhanced by goals that are consistent throughout the organization (Edwin, 2005). Therefore, membership is initiated with a formal application process to identify individuals with similar interests, such as desire to learn FBT, and past experiences demonstrating interest in learning evidence-based treatments. Utilizing up-to-date Internet technology (Jones & Dages, 2003), community and student volunteers are actively recruited almost exclusively from the AC Web site (http://www.unlv.edu/centers/achievement/) and staff referrals to assist in managing clinic responsibilities (Rapp et al., 2005). An online application form is completed, which includes prompts to record clinical and research-related experiences, motivation for applying to work in the center, personal strengths and interests, and how the applicant might assist the organization. The applications are screened by the Clinic Coordinator, and those accepted are interviewed utilizing a semi-structured interview to enhance reliability, objectivity, and appropriate placement within the organization (Garcia & Kleiner, 2001).

Orientation of New Members

New members are formally oriented to the program, which assists in their acceptance of program goals and values, as well a promoting affective commitment (Klein & Weaver, 2000). Orientation also facilitates better understanding of program policies and procedures, and increased productivity, retention rates, and integration of new members into established programs (Edwin, 2005). A “survival packet” (Heyroth, 2003) is provided that includes a flow chart depicting staff hierarchy and staff responsibilities, major program policies and procedures, and protocols depicting proper lines of communication (e.g., solicit administrative input to assist in settling disagreements but only after attempts to resolve conflicts with the perpetrating other have failed). A program-introduction is provided that includes the AC’s history, mission, and program related benefits reported by former members in exit interviews (e.g., paying for members’ professional conference registration and travel related expenses, opportunities to disseminate program research and practices, training opportunities). The initial orientation interview ends with a tour of the facility, and the new member is assigned to a well-suited project to decrease turnover and improve productivity (Garcia & Kleiner, 2001).

Mentorship Model

Mentorship programs that match senior staff members to new members with similar interests enhance organizational development and employee retention (Edwin, 2005; Morzinski & Fisher, 1996). Therefore, a mentorship model is implemented in the AC, whereby senior staff members who receive no formal training in mentoring are encouraged to regularly meet with their mentees. During these meetings the mentor and mentee discuss program-related adjustments and career ambitions and interests, and are encouraged to review the “roles and responsibilities” section of the orientation manual, which acts to facilitate team interaction (Marks et al., 2002). Although anecdotal, the mentoring component appears to have enhanced work production and skills enhancement, permitting new members to confidentially discuss difficulties with key experienced personnel who are invested in the program. As mentees become sufficiently familiar with the culture of AC, the mentorship responsibilities are gradually withdrawn, and mentees are provided an opportunity to be mentors (Cannon-Bowers, Salas, Blickensderfer, & Bowers, 1998).

Scheduled Shifts

Work hours are collaboratively determined, and work schedules are publicly posted on a large bulletin board in the conference room and on an internet calendar to assist in coordination of meetings and events. When members are unable to maintain scheduled hours, they are responsible for informing their immediate supervisor, and posting how they can be contacted during the missed shift, and the expected date and time of their return. All members are expected to make up missed hours, and as reviewed during orientation, scheduled meetings and work shifts are monitored and reflected in consideration for employment positions and raises in salary. Clinicians are also encouraged to find their own substitutes for scheduled therapy sessions, which is made possible because FBT is a prescribed intervention and often implemented in pairs (see below).

Program Assignments and Accountability

Orientation involves structured assistance in task-related initiatives and assignments (Heyroth, 2003). For example, new members are taught to answer telephone calls professionally, employ templates to address professional letters and e-mails, request office supplies, structure professional presentations, etc. The protocols required to conduct these responsibilities are recorded in the previously mentioned survival packet and role-played between the new member and person conducting the orientation. Following the orientation, each new member is assigned specific tasks to complete by their Coordinator. Since it often happens that members expeditiously complete all assigned work, senior members insert generic assignments into a Program Assignment Folder (PAF), which can be subsequently performed by any member in the AC. New members are taught to review this list of assignments when they initiate their shifts, and prioritize their shift responsibilities accordingly. There is a section in the PAF to record completed assignments, providing formal opportunities to publicly acknowledge efforts of those who assertively completed assignments, thus increasing obligation and urgency to complete these tasks (Guion & Bolton, 2006).

A Program Management Folder (PMF) is provided to all new members as an organizational tool (Heyroth, 2003). This folder includes a Personal Assignment Worksheet that members complete when they are given assignments in program meetings. To assist in accountability, this worksheet includes a section to record personal assignments, and assignments that are made to others during meetings, including due dates. The PMF includes a fill-in-the-blank template that outlines agenda items for program meetings (e.g., record date of meeting, state duration of meeting and assure everyone can attend full meeting, assign minute taker, provide brief summary of scheduled meeting topics, solicit additional topics, estimate times for each meeting topic). Finally, the PMF includes a directory of member schedules and their contact information. Also relevant to accountability, all e-mails regarding program assignments and changes in program policies and protocols specify a date and time in which members must indicate they have received the correspondence.

Training and Evaluation of Basic Competencies

An orientation quiz is implemented to ensure new members are familiar with the orientation packet, and know how to apply its information. Once orientation is complete, the Orientation Coordinator sends a welcoming e-mail to the program list-serve informing current members of the new member and encourages others to do the same. The new member is required to complete a training program in the Protection of Human Research Subjects offered by the Collaborative IRB Training Initiative (http://www.cititraining.org) because on-going studies are performed in the clinic. This on-line course was developed by investigators at the University of Miami to assist individuals in establishing research competencies specific to ethical and legal research issues.

Members who are interested in assisting therapists with child management during FBT sessions, or who are interested in becoming therapists, are assigned to read FBT treatment manuals and subsequently pass a competency quiz that includes multiple choice, fill-in-the-blank, and short essay questions about the FBT interventions. Community and student volunteers who appear to be promising are invited to observe FBT supervision and training meetings, and are occasionally invited to participate in role-plays that often occur during training and supervision. These role-plays are essentially “tryouts,” and it is expected that they will anecdotally evidence basic clinical skills such as warmth, empathy, and maturity. However, it is primarily important that they demonstrate good protocol adherence during these role-plays, as we have found clinical skills are relatively easy to teach when protocol adherence is good. Moreover, we have found inexperienced therapists (i.e., those with a bachelor’s degree) who have not acquired basic clinical skill sets rarely veer away from intervention protocol (i.e., resort to non-FBT strategies) when they experience difficult clients. Of course, this provides the supervisor systematic opportunities to teach them clinical skills that are likely to enhance the effective delivery of FBT. However, bachelor level therapists also provide a check to assure FBT interventions are relatively easy to learn, and therefore have excellent external validity.

Development of Position Roles and Responsibilities

To identify roles and responsibilities unique to each position within the AC, program coordinators were initially assisted by the Director in recording their major clinic responsibilities (e.g., data management, intake assessments, preparation for therapy sessions) into step-by-step protocol checklists that indicate specific methods of implementation. Including staff members in developing their job description protocols assists in bringing about a sense of pride and program ownership, which in turn decreases staff turnover common in these settings (Edwin, 2005). Accuracy and utility of these initial protocols were tested by providing the protocols to new staff members who were instructed to complete the job responsibilities based only upon the written instructions. When difficulties arose, the written protocols were revised and subsequently tested with other staff members who were unfamiliar with the respective tasks until they were able to consistently follow the instructions with a high degree of adherence. While requiring significant effort at the outset, developing these job description protocol checklists proved invaluable in training new staff to perform their tasks, made cross training easier, permitted monitoring systems to be developed, allowed colleagues to assume missed shifts during absences of key personnel, and enhanced overall clinic efficiency (Miller, Sorensen, Selzer, & Brigham, 2006).

Posting Functional Clinic Protocols Throughout the Clinic

To assure effective utilization and compliance with office tasks, step-by-step instructions are placed in functionally obvious locations throughout the clinic. For instance, instructions regarding how to send a fax are printed on the fax machine, instructions relevant to professionally answering the telephone or retrieving messages from the answering machine are affixed to each telephone, and forms for ordering office supplies are conveniently located in the supply cabinet. There are reminder prompts to lock all confidential filing cabinets on each of the drawers, and prompts to remind members to lock the main door of the clinic if they are the last person to leave for the evening. These messages are also summarized in the orientation survival packet, and are particularly important for new members.

Systems of Communication

Member e-mail addresses are organized into a list-serve to permit efficient intra-group communication, foster group identification, maintain coordination across divisions, and enhance adherence to group norms (Kerr & Tindale, 2004). The subject line of each e-mail indicates when messages are specific to particular members, or specific teams, so these e-mails may be quickly ignored by members for whom they are not relevant. This form of communication is particularly helpful for Coordinators, as their roles interact with individuals across multiple systems. An organizational “shared drive” is also employed which program administrators and senior members can exclusively access from their homes or individual office computers. The shared drive contains program forms, clinic protocols, assessment measures, databases, and treatment manuals. This shared drive is organized into in a hierarchical format according to distinct conceptually-driven categories (e.g., treatment manuals, client charting forms). When revisions are made to files on the shared drive, Coordinators are responsible for tracking these revisions, but only for those files over which they have direct oversight.

Activities to Enhance Leadership and Cohesion Among Members

Team performance and personal satisfaction in organizations is improved by clearly defining roles and assigning specific responsibilities to each team member (Tubre & Collins, 2000). Therefore, based on interest, seniority, experience, and availability, all members are assigned a task responsibility that requires administrative oversight, such as maintaining the snack bar, soliciting donations, maintaining contact information relevant to community resources, data management, protocol adherence, obtaining and managing outcome study articles, updating schedules, and so on. This method has permitted the clinic to recruit a large number of volunteers who are able to fulfill vital roles with minimal supervision.

Cohesion is also enhanced by participating in social activities with team members, including informally eating lunch together, attending social events together after hours, participating in community events, and sharing hotel rooms with appropriately matched peers while attending professional conferences. Recognizing that positive affirmations have been shown to enhance work ethic and strength in relationships (Guion & Bolton, 2006), once every month or two members are prompted to take turns telling the group things they admired or respected about other members during the program meeting. Along these lines, the “Catching My Co-Worker” worksheet is periodically distributed to all members during a program meeting, and the members are instructed to take turns disseminating the information to each other. Similar statements about members are distributed via the clinic listserve. Also, members are descriptively praised for bringing up ideas during group brainstorming, regardless of experience level (Kerr & Tindale, 2004).

Structured Meetings

Highly structured meetings that employ evidence-based strategies encourage group identity and cohesion, sense of accomplishment, motivation and involvement in projects (Guion & Bolton, 2006). Meetings also provide opportunities for members to demonstrate their creativity and communication skills (Esquivel & Kleiner, 1996). Therefore, there is a weekly one-hour program meeting that includes all members in the organization to discuss issues effecting program policy. Prior to every program meeting, an agenda is distributed by the meeting head to facilitate ample time for members to prepare for discussions and offer informed comment (Guion & Bolton, 2006). Of course, program meetings include assignment of a staff member to summarize meeting content, and this assignment occurs among Coordinators according to a rotating schedule so no one person is taxed. Minutes follow a standardized format, including the start and end time of meeting, a list of members present and absent, general announcements, and a list of project assignments to be completed prior to the next meeting. Additionally, minutes ensure reviewed information is recorded for those members who are unable to attend, and provides documentation for assignments to be reviewed in subsequent meetings (Esquivel & Kleiner, 1996).

Employment of agendas assists in maintaining efficiency in meetings (Guion & Bolton, 2006). Therefore, the program meeting is led by the Clinic Coordinator, who initiates the meeting by stating its purpose and reporting each agenda item, including estimated times for each item. The Coordinator proceeds to review each of the respective agenda items in order of importance. After agenda items are reviewed, each administrator provides a progress report for the week, including their completion of program tasks, and assignments for the next week and solicitation of comments from their immediate team members. The program director and coordinators aid team participation by asking questions, soliciting ideas to enhance program efficiency (Cawley, Keeping, & Levy, 1998), and lead discussion and progressive decision-making by summarizing decisions, resolving conflicts, managing discussions, and providing validation and support (Esquivel & Kleiner, 1996). Indeed, leaders that show strong consideration of members input, encourage greater trust in the leader as well as increased cooperation and diligence in program decisions (Korsgaard, Schweiger, & Sapienza, 1995). Administrators also schedule weekly clinical and research meetings to discuss FBT related projects, usually lasting between 30 and 60 minutes. These meetings resemble the program meetings in structure, but generally involve more brainstorming of solutions to day-to-day problems.

Quality Assurance for the Participation of Members

It has been shown that high accountability results in better accomplishment of goals, and higher levels of attentiveness (Frink & Ferris, 1998). Therefore, accountability is examined at all organizational levels of the clinic to facilitate effective performance management (Ansell, 2002). At the overall program performance level, the productivity of each member of the organization is monitored, including shift attendance and promptness, completion of program assignments (late/missed), and response to important e-mail announcements and other important communication (all e-mail correspondence includes a time-to-respond deadline). Each week, program administrators provide information about nonadministrative staff productivity to the Clinic Coordinator; the Program Director and Clinic Coordinator provide similar information about the administrators. Once a month, the Director and Clinic Coordinator review this chart for members who appear to be evidencing difficulties accomplishing these tasks; members are scheduled for appointments with their supervisors to generate solutions to the respective problem behaviors. Administrators emphasize positive behaviors first, and then report viable reasons the undesired behavior may have uncharacteristically occurred. When difficulties persist, the problem behavior is addressed in an administrative meeting. And if solutions cannot be implemented effectively, the member is asked to consider reapplying to the program at a later time when distractions are no longer present. Exemplary adherence to program requirements is also brought to the attention of team members during weekly program meetings, as discussion of individual member accountability with team members has been shown to enhance work performance (Frink & Ferris, 1998). In addition to serving as an impression management tool among colleagues, accountability records can also serve as an evaluation mechanism for team members to improve their performance and obtain future advancement opportunities.

Recruitment of Clients

Effective client recruitment is essential to implement FBT, and is facilitated by ongoing communication with referral sources, which is extremely important (Del Boca & Darkes, 2007). Therefore, most coordinators are cross-trained to implement standardized recruitment protocols with referral agents (e.g., Court, State and County Family Services) that are designed to assist in maintaining effective relationships. They are trained to speak with caseworkers about the program in unsolicited telephone calls, meet with judges about the program during court hearings, and conduct scheduled formal presentations during unit meetings at referral agencies. To assist in consistently reporting FBT-related information to referral agents, there are protocols that are utilized to prompt descriptions of FBT and the treatment process, list benefits of participation in FBT (including its success in controlled trials) and explain how to refer clients utilizing standardized referral forms. Recruitment meetings usually last 20 minutes and are performed at the referral agency during their previously scheduled program meetings. A professional brochure is distributed that highlights important aspects of the referral process, and the importance of making referrals to evidence-based programming, such as FBT. The referral forms are structured such that the criteria for program appropriateness are listed at the top of the form, permitting the referral agent to quickly determine if the referral is likely to be accepted or not prior to completing other required demographic and contact information. Staff presenters are also encouraged to bring food and drinks to these meetings to assist in improving attendance. Other recruitment strategies include the provision of brief pro-bono workshops for the referral agencies. These workshops are usually abbreviated versions of the FBT interventions, and are designed to assist in accomplishing training goals of the referral agencies. For instance, if the referral agency is a school, FBT interventions are modified to accomplish school-related goals, such as behavioral management for teachers or communication skills training for students. Caseworkers might receive a workshop that is relevant to helping them determine if potential referral agents are utilizing evidence-based treatments.

In/exclusionary criteria used to determine program eligibility varies across programs but is relatively unrestricted, usually requiring an adult significant other willing to participate in treatment with the identified client, recent drug use, a documented incident of child maltreatment by a child protective service agency (if child maltreatment is concurrently addressed), and exclusion of referrals in which the primary presenting problem is domestic violence or sexual abuse.

Referrals are screened in a telephone call to validate the referral agent’s information utilizing a structured clinical interview. This helps to decrease their time making the referral, and personalizes the referral process. The initial assessment session is scheduled to occur within 48 hours of the initial telephone screen to enlist clients when their motivation is relatively high (Lefforge, Donohue, & Strada, 2007). Clients and their participating family members are also sometimes provided inexpensive boxed lunches and cell phones to assist in maintaining motivation to attend sessions, and enhance communication between sessions.

Management and Quality Assurance of FBT Sessions

FBT intervention sessions are initially scheduled by the Enlistment and Retention Coordinator. Although only one therapist is scheduled for outpatient sessions, two therapists are scheduled to attend home-based sessions. This is done for a number of reasons, most notably safety. Safety is further enhanced by requiring at least one therapist to carry a cell phone that is left on during sessions, scheduling sessions prior to dark, and informing someone when the session is initiated and completed. When sessions are home-based, a map is utilized in the scheduling process that includes the client’s address. Whenever possible, inexperienced and experienced therapists are matched to assist inexperienced therapists in acquiring the specific skills of new treatments, provide on-site reinforcement and support (Marks et al., 2002; Miller et al., 2006), and enhance protocol adherence. However, when cases are expected to be particularly difficult, experienced therapists are matched together. Sessions are recorded in a schedule book to prevent double bookings, and assistance is provided in rescheduling clients and notification of therapists when clients cancel. Although not the norm, a therapist may substitute for another therapist. This is made possible because FBT follows agendas and specified treatment protocols. Comments from therapists suggest novelty of therapists may act to enhance motivation in family members to demonstrate their new skills to secondary therapists, and this method demonstrates consistency in treatment implementation across therapists.

Prior to conducting the initial pretreatment assessment session with clients, a member of the assessment team assembles a client folder that includes necessary consent and assent forms, a demographic questionnaire, psychometrically-validated questionnaires relevant to the presenting problem, and standardized progress notes. Therapists also utilize protocols that depict specific tasks they are responsible for accomplishing prior to each session. For example, they prepare session agendas, and assure they have all required assessment or intervention protocol checklists and materials (e.g., worksheets, questionnaires). For FBT intervention sessions only, therapists make sure they have tape recorders with charged batteries to audiotape sessions so protocol adherence can be assessed (assessment clinicians do not tape record sessions to decrease the likelihood of response reactivity). When FBT is home-based, therapists are taught to review their session agendas during car rides to client homes, and complete session case progress reports during their travel to the clinic after sessions.

Clinical records are legal documents; therefore, therapists are careful to assure these records are accurate. However, it is often the case that therapists are unaware or lack sufficient familiarity with NIDHA record-keeping standards. Indeed, issues associated with poor record keeping (e.g., using black ink, crossing out an error with one line and initialing) often become apparent when quality assurance methods are incorporated with new therapists (Brown, Topp, & Ross, 2003). The clinic has, therefore, created a reference binder in which all cabinets, drawers, and contents are labeled numerically, providing exact locations and content descriptions of all forms utilized in the clinic. This binder is utilized during orientation for new members to become familiar with these forms, and their respective standardized filing system. Additionally, in teaching therapists to understand the management of client records, a blank client chart is provided to all therapists. The chart is then systematically reviewed and the charting procedures are explained. Oral quizzes are utilized to assure therapists know how to complete charting procedures. The chart includes a table of contents, a log of all contacts, consent and assent forms, and consent forms to release and retrieve information. Standardized progress notes include information that is often required by most State licensing boards and agencies, such as duration and type of contact, session number, and the persons present during the assessment and therapy sessions. However, these progress notes are designed to be feasible and focused on FBT. For instance, they include “fill-in-the-blank” prompts for ratings of client compliance with each intervention. Also, administered assessment measures or interventions are prescriptively listed and endorsed by circling the ones that were implemented.

Maintenance of the Integrity of FBT Assessments

After the intake assessment session is conducted, a member of the quality assurance team reviews the assessment folder utilizing a standardized checklist to assure all forms are properly located in the chart, and determine accuracy in chart management procedures (e.g., signatures, all items are endorsed, progress notes are completed, date and id# on each page). Thereafter, members of the quality assurance team regularly review client charts once a month according to standardized checklists that depict specific recording requirements (e.g., dates of progress notes correspond with log of contacts, progress notes are signed). For each participant, this worksheet includes a spreadsheet with columns representing the dates of each review and the rows representing various processing errors, including missing signatures or dates, sloppy writing, missing forms, missing items, missing progress notes. Each review takes less than 5 minutes to complete, and FBT therapists are allowed 1 week to correct errors. Frequently occurring errors are reported in clinical supervision meetings. Advantages of these standardized quality assurance procedures include establishing clear standards of record-keeping, monitoring progress over time, improving performance (Bond, Evans, Salyers, Williams, & Kim, 2000), ensuring accountability, and cost effectiveness (Buetow & Roland, 1999).

FBT Training

To enhance clinical expertise while maintaining sufficient adherence to FBT protocol, intensive training opportunities are provided on an ongoing basis (Bartholomew, Joe, Rowan-Szal, & Simpson, 2007). Specifically, therapists are asked to become familiar with an FBT treatment manual prior to attending an initial 5-day workshop (Kluger & DeNisi, 1996). Each treatment manual includes a rationale underscoring the need and conceptual reasoning for the intervention, an overview of the intervention procedures, specific steps involved in the administration of each intervention, sample vignettes of therapists implementing intervention protocol, probable solutions to potential problems, and protocol checklists that may be utilized to prompt therapist actions during sessions (“cheat sheets”). During the initial workshop, FBT interventions are taught in a conceptually-driven sequence. For example, a program orientation is offered first, followed by behavioral goal setting, treatment planning, assurance of basic necessities (e.g., not being able to pay bills, risk of domestic violence or child maltreatment) and skill-based interventions aimed at preventing substance use, HIV risk behaviors, child maltreatment, and other related problems. For each intervention, the trainer first models the intervention within the context of a first session, with therapists sequentially taking turns portraying the role of a client, each for about 5 minutes. Therapists are instructed to portray real life situations, including noncompliance to particular protocols. Rather than asking questions during role-playing, therapists are encouraged to “act out” difficult scenarios to maintain flow and contextual appreciation of their clinical management (Rowan-Szal, Greener, Joe, & Simpson, 2007). The trainer then portrays the role of a relatively compliant client while the participants take turns portraying the therapist role in relatively easy scenarios, each for about 5 minutes. Therapists are not permitted to ask questions during the role-play interactions. Rather, they are instructed to work out mistakes as if they are in a real session. Therapists are provided very specific feedback from the trainer and other participants after their respective performances, thus therapists learn from the observation of others. Between role-plays, the supervisor points out clinical skills (e.g., empathy, reinforcement) that were effectively implemented, and suggests strategies to enhance their utilization, when warranted. Upon completion of the initial session format for the intervention, the trainer and therapists each model the intervention as it should be in future sessions (on-going sessions), and in difficult situations. After the first FBT intervention is role-played, the next FBT intervention is role-played, and so on.

Approximately 6 months after initial training in FBT, a 2 to 3 day booster training is performed. This training format is similar to the first, except the trainer does not model each of the interventions. Rather, participants take turns in the therapist’s role (5 minute blocks) while the trainer pretends to be a client, providing feedback between role-plays. The trainer increases difficulty commensurate with therapist experience level, thus, permitting inexperienced therapists to view the reactions of senior therapists in difficult scenarios while still being able to practice FBT at an appropriate level of difficulty. Of course, experienced therapists are able to sharpen their skills at an advanced level utilizing this approach.

Non-FBT Specific Clinical Training

A travel budget has been established to support key personnel to attend workshops at psychology conferences that are focused in child maltreatment, substance abuse, and related problem areas. There is a strong bias to attend workshops of other EBTs, such as Multi-Systemic Therapy, the Community Reinforcement Approach, and Project 12-Ways. Although support is limited, key personnel are usually reimbursed for hotel accommodations and some meals to these events, whereas less senior members are provided less substantial, albeit appreciated, travel stipends. Professionals in the community are also invited to perform pro bono workshops at the AC in their areas of expertise (e.g., child maltreatment reporting); and nationally recognized professionals sometimes accept invitations to conduct teleconference trainings with AC members.

Standardized Supervision

Although live supervision is often impractical, it is a preferred supervisory technique (Romans, Boswell, Carlozzi, & Ferguson, 1995). Therefore, FBT supervisors sometimes attend sessions, particularly when therapists are inexperienced or when sessions are expected to be difficult. Supervision is scheduled to occur weekly for 90 minutes with a licensed supervisor. All program therapists (usually up to 7 therapists), and volunteers who sometimes assist therapists with child management during sessions, attend supervision (Del Boca & Darkes, 2007). Therapists bring audiotape recordings of the sessions they have conducted since last supervision, and the chart records of each of their clients. The supervisor brings a standardized form to monitor cases, and guide supervision. Systematically assessing, reviewing, and brainstorming methods of preventing adverse events and factors that may lead to adverse events have been shown to substantially reduce harm to clients and significantly decrease clinic liability (Ansell, 2002). Therefore, supervision is initiated with an inquiry of any adverse events occurring in treatment sessions during the previous week (e.g., suicide risk, suspected child maltreatment, domestic violence). Of course, when adverse events are identified they are discussed, including safety plans and appropriate consultation with others.

The Enlistment and Retention Coordinator provides a status report of participants who are being assessed for program appropriateness or scheduled for their initial assessment. New referrals are assigned to therapists, and a report is provided regarding the extent to which enlistment and retention telephone contacts were made to clients since last supervision. Along these lines, two therapists are responsible for calling clients to assist in maintaining motivation for therapy, brainstorming solutions to problems that interfere with session attendance, and completion of therapy assignments, and assisting clients in obtaining needed services (e.g., representation in court, communicating effectively with caseworkers). These individuals utilize protocol checklists to prompt them in performing these calls with integrity. Therapists are informed when clients disclose problems during these telephone calls, and solutions to these problems are brainstormed during supervision, whenever necessary.

Supervision meetings focus on various aspects of treatment implementation, including therapist style, utilizing role-playing to teach therapeutic skills, providing descriptive feedback on adherence rating forms, and descriptively praising successful efforts to implement treatment (Carroll et al., 2002). When trainees are learning to implement evidence-based treatments, it is essential to measure adherence to treatment implementation (Madson, Campbell, Barrett, Brondino, & Melchert, 2005). Therefore, therapists are provided feedback regarding the quality and extent to which they provided therapy with integrity during the past week according to reviews of their session audiotapes by other therapists. Specifically, the Coordinator of Protocol Adherence directs individuals who reviewed audio-tapes during the previous week to utilize a protocol checklist to guide them in mentioning outstanding strengths relevant to therapeutic style, and methods of accomplishing treatment adherence. Therapists who conducted the session are then prompted to mention their own strengths, including methods of enhancing protocol adherence in the future. When difficulties in protocol adherence occur, solutions are brainstormed, and if judged helpful by consensus, the treatment protocol instructions are revised. Notably, active involvement of therapists in the process of adjusting evidence-based interventions to accommodate cultural aspects of the treatment program has been shown to encourage a sense of ownership, acceptability of interventions, and enhances creativity and initiative in treatment planning (Power et al., 2005).

In reviewing cases, therapists report how many cases they are assigned to treat (i.e., assigned caseload), and how many cases they conducted during the past week. Problem solving is initiated to assist therapists in generating solutions to nonattendance. The supervisor is required to decide, with input from therapists, which cases are reviewed first. Priority is given to inexperienced therapists (Morgan & Sprenkle, 2007), and cases that are complicated or dangerous for children. In disseminating case feedback, the therapist follows a case review protocol checklist. If the case was seen during the previous week for the first time, the therapist discloses basic family demographic information, the reason for referral, number of children in the home and their ages, diagnostic information, including drug use history, strengths and areas of growth, a brief conceptualization of how presenting problems developed and are maintained, and a tentative treatment plan. Each case presentation lasts about 15 minutes. The session audiotapes from each case are then reviewed at random areas for about 5 minutes. When cases have been presented previously, therapists present something idiosyncratic about the client’s family to jar the supervisor’s memory, the reason for referral, session number, approximate percentage of scheduled sessions attended, family members present in the session, methods of incorporating other family members in the future, treatment modules completed, solicitation of solutions to assist in problems that may have occurred during the session, and plans for next session. The supervisor then initiates a random review of the session tape, and potentially reviews clinical records for quality assurance (optional). Each case review lasts approximately 10 minutes. Finally, when clients were not seen, the therapist indicates family support available to bring about session attendance, treatment modules completed up to missed session, notable problems that may have influenced nonattendance, and plans to increase future attendance. The aforementioned format encourages therapists to be independent, and improves their self-efficacy.

The supervisor utilizes a standardized logbook to record the extent to which each case is reviewed (i.e., incidents assessed, case discussed, session audio-tape reviewed during supervision, therapeutic assignments, completion of past assignments). Supervisors end by placing the session audiotapes in a locked filing cabinet for assessment of protocol adherence. In study contexts, at least 10% of these tapes are randomly reviewed for protocol adherence by blind raters. However, in practice settings, these tapes are ideally reviewed by other therapists during the following week (as indicated previously). Of course, the supervisor indicates who was assigned to review tapes for protocol adherence in the aforementioned logbook to assist in accountability. Therapists have a greater likelihood of being assigned to review tapes when they have limited experience implementing FBT, they are experiencing difficulties maintaining treatment integrity, or their cases have demonstrated poor attendance during the previous week.

Treatment Integrity

It is recommended that clinical programs comprehensively assess treatment integrity (Power et al., 2005). Indeed, clinical programs that utilize standardized manuals and evaluate treatment integrity are consistently rated better than those programs that do not (Moyer, Finney, & Swearingen, 2002). However, when protocol adherence is assessed, it is done so in a limited and questionable manner (Power et al., 2005). There is some support to suggest examination of audio or videotaping of therapy sessions facilitates the most accurate means of assessing therapists’ adherence to therapeutic protocols (Del Boca & Darkes, 2007). Of course, intervention integrity is important to assess because it provides an objective measure of the efficacy of treatment (Power et al., 2005). If treatments have been shown to be effective in highly controlled trials, their replication in community settings should result in successful outcomes if they are implemented with integrity.

FBT is particularly amenable to evaluation of treatment integrity because all FBT components are specified in concrete easy to understand instructional steps that are sequential and easily monitored objectively. Indeed, the therapist uses the protocol instructions during therapy as a guide, with implementation of one step acting to prompt the next step, and so on. Adherence can subsequently be assessed by comparing (correlating) the therapist’s scores (the percentage of therapy steps completed for each intervention) with the scores of an independent rater. In this way, the percentage of instructional steps performed, as measured by the therapist conducting the session, serves as a measure of treatment integrity. And the relationship (correlation) of scores between the independent rater and the therapist conducting the session serves as a measure of reliability. Ten percent of all sessions are randomly selected for reliability checks, making this procedure feasible, particularly since the independent raters are typically other therapists who are attempting to learn FBT.

Therapist Ratings of Client Participation

Attendance data and measures of treatment involvement are carefully examined, as these factors have been shown to be predictive of outcome (Del Boca & Darkes, 2007). For instance, when the clinic was first established, compliance of clients was relatively poor. When the issue was initially reviewed in supervision, it was believed this finding was primarily due to the inexperience of FBT therapists. However, a few of the cases demonstrated marked improvement in their compliance to interventions subsequent to the therapists disclosing their ratings of the extent to which the clients actively participated during each session (7 = extremely compliant, 1 = extremely noncompliant). Later, these ratings became more sophisticated to include conduct in session, homework completion, and community/significant other support. The ratings appeared to be particularly effective when clients were permitted to be involved in the determination of ratings, the clients were provided explicit feedback regarding methods of improving future scores, and clients were informed referral agents might find the ratings to be useful in supporting their participation with third parties (e.g., Court).

Database for Assessment and Protocol Adherence

All databases are maintained utilizing the Statistical Package for the Social Sciences (SPSS) for windows. A Data Management Coordinator and 3 part-time data management specialists are responsible for effectively managing two databases, one is relevant to program evaluation, consisting of basic demographic and clinical outcome data. In practice settings, it is usually enough to monitor pre-treatment and post-treatment outcomes, whereas in research studies, follow-up data is also recorded (i.e., 4 months post-treatment). The 2nd database is relevant to the assessment of protocol adherence (therapists submit their completed session protocol checklists each session). All data are entered into the database in teams of two. One specialist is a designated typist who enters the data into the computer database, and the other reads the data to the typist to enter into the respective database. The data reader provides each bit of information; the typist observes the entered data on the screen, and consequently repeats the bit of information back so the reader can proceed to the next bit of information after assuring the information was recorded properly.

In recording adherence data, all clinicians provide their completed protocol checklists for each session during the previous week to a Data Management Coordinator. Ten percent of the session audio-tapes are randomly chosen for review. Selected tapes are distributed to a trained volunteer or therapist (see Supervision above) for integrity review. Individuals who are selected to perform integrity reviews are “blind” to therapist responses to the protocol checklists. For each FBT session, the clinician name, reviewer name, date of the session, date of protocol adherence review, the treatment administered, whether the treatment was an initial session or future session, therapist rating of client’s participation (7-point Likert scale), client rating of intervention helpfulness (7-point Likert scale), the percentage of therapy steps reported by the clinician and the individual conducting the integrity review are noted. This database permits quick assessment of interventions that need to be reviewed extensively in training to enhance protocol adherence, and permits a subjective assessment of the extent to which clients find interventions to be helpful. It is also possible to assess the extent to which therapists require training (i.e., less training time is necessary for therapists with greater adherence scores). Similarly, the measurement of program outcomes is useful to assess program effectiveness and identify therapists who may be in need of additional supervision. When an assessment is conducted, the information is summarized in a data chart by the Assessment Coordinator, and a protocol checklist provides the data entry team with a list of assessments conducted, and the order in which to enter them into the database. There is a location on the form for the Data Management Specialist and Assessment Team member to initial and date when they have completed entry for a respective assessment script, and the data is randomly checked by the quality assurance team to ensure accurate data entry.

Concluding Remarks

With few exceptions (i.e., Ronith, Grabowski, Rhoades, & McLellan, 1993), model mental health clinics have rarely been delineated in peer reviewed journals, particularly within the context of EBT adoption. In the prior sections we delineated organizational and structural procedures in a model substance abuse clinic specializing in FBT. The reviewed methods were based on empirical findings from studies involving organizational management. Most of the methods reviewed appear to have considerable potential to generalize to community treatment programs emphasizing evidence based approaches, such as FBT. Indeed, the topic areas are almost exclusively relevant to the efficient management of treatment delivery, and most strategies could be either directly or indirectly applied.

It can be argued that some of the methods reviewed in this article appear to have been adopted to accommodate the unique aspects of AC, and are not relevant to other treatment centers focused on the provision of community-based care. For instance, volunteer programs are certainly underutilized in community-based treatment centers, perhaps because these administrations may lack resources to train volunteers, are concerned with increased liability, or evidence difficulties recruiting student volunteers because programs of higher education are not in close proximity to the clinic. Although these reasons are valid points, AC demonstrates the utility of volunteer programs, and offers a set of evidence-based strategies that may be implemented to assist in their successful and feasible implementation (e.g., relatively low number of work hours, letters of recommendation relevant to advancement, awards). Of course, the cost of adopting novel evidence-based clinic procedures may be relatively high initially, and the adoption of these methods may require flexibility, training, and adaptation to some extent. Relevant to the aforementioned example, if schools of higher education are absent, nonstudent community volunteers may be exclusively recruited into clinic contexts (e.g., Strauss, 2005), and once a volunteer component is established, its initial start-up expenditures will likely be recuperated.

It can also be argued that some of the evidence-based procedures reviewed in this article are not relevant to the adoption of EBTs. Along this vein, recruitment of clients is often omitted in training programs that are specific to EBT adoption. However, one of the greatest difficulties we have encountered when providing training in FBT to community agents is that they are unfamiliar with basic recruitment strategies, resulting in poor support of their program and interfering with potential in vivo training opportunities after their initial EBT workshops. Thus, the recruitment strategies that were reviewed may be quite useful in the adoption of EBTs. We are concerned that in our recent anecdotal review of EBT training programs listed in NREPP (SAMHSA, 2008), very few of the training programs appeared to include standardized clinic charts and filing systems to assist in reducing paperwork requirements. Of course, the introduction of such systems inevitably improves clinic efficiency and permits greater time to adopt the respective EBT. Time efficient treatment integrity and quality assurance systems, similar to those utilized in FBT treatment sessions that were underscored in this article, need to be emphasized in the adoption of EBTs. Indeed, integrity of clinic operations is essential to the successful implementation of EBTs, and the guidelines presented in this article may assist in developing and maintaining a clinic capable of EBT delivery.

Lastly, when resources are limited, funding is often solicited from Government agencies (e.g., Substance Abuse and Mental Services Administration) to support successful adoption of EBTs. Most grant writers now request support for training and supervision that is directly relevant to treatment provision. However, they rarely request support for clinic operations, which is an infrastructural problem that may limit eventual adoption of EBTs. Therefore, this article may be utilized to appreciate the contextual factors that are necessary in EBT adoption. Moreover, it is hoped this article will spur developers of EBTs to disseminate their methods of managing clinics that have successfully incorporated EBTs.