Some Tools for Carrying Out a Proposed Process for Supervising Experience Hours for Aspiring Board Certified Behavior Analysts^®

Abstract

While task clarification, goal setting, feedback, and behavioral skills training (BST) are well-supported methods for performance improvement, there is no standardized approach to supervising aspiring Board Certified Behavior Analysts® (BCBAs®) that specifies how such practices should be used within a comprehensive supervision system, namely for supervising those who are still accruing experience hours for the purpose of becoming board certified. This article outlines a systematic approach to BCBA supervision and provides a set of tools that supervisors can use to ensure that they are engaging in empirically based supervision practices.

Electronic supplementary material

The online version of this article (doi:10.1007/s40617-017-0186-8) contains supplementary material, which is available to authorized users.

The Behavior Analyst Certification Board® (BACB®) Supervisor Training Curriculum Outline specifies that BCBAs trained in supervision techniques should be able to describe the purpose and important features of supervision, use behavioral skills training, including the use of effective performance feedback to teach targeted skills, evaluate the effectiveness of their supervision, and proactively engage in professional development activities to improve supervisory performance (BACB, 2012b). Each of these repertoires is essential for providing adequate supervision. We believe that of equal importance is a supervisor’s ability to conduct supervision within a process that accounts for the contingencies acting upon his environment and the environment of his supervisees. Insight from the work that has been done in organizational behavior management (OBM) can help us design effective supervision systems.

Behavior analysts should be accustomed to using a behavior-analytic technology to produce behavior change programs for clients. However, many may not be as accustomed to using the same technology to train and supervise individuals seeking certification, especially those who have limited practical experience. Organizational behavior management is a sub-field of applied behavior analysis that focuses specifically on organizational problems (Bucklin, Alvero, Dickinson, Austin, & Jackson, 2000). Because the same empirical approach that is followed in clinically focused behavior analysis is used in organizational behavior management, much of the research findings from the OBM literature can be used to aid in the development of applied behavior analysis (ABA) practices, including the supervision of others.

For example, it is clear that ensuring that consequences are contingent on the target performance improves that particular target performance. Komaki (1986) set out to, “specifically identify and empirically determine what constitutes effective supervisory behavior” (p. 270). The author found that effective managers spent significantly more time monitoring performance and used work sampling as a specific form of performance monitoring more often than marginally effective supervisors. Komaki also found that there was no difference between the groups regarding the number of positive, negative, or neutral consequences between effective and marginally effective managers. In other words, “effective managers were no more likely than the marginal managers to provide positive consequences, and they were no less likely to provide negative consequences” (pp. 275–6). The author concluded that because effective managers spent more time monitoring, their consequences were more likely to be contingent on, rather than independent of, employees’ performance.

Task clarification, goal setting, and feedback are common practices that have been used in a variety of settings to improve on-the-job performance. Tittelbach, DeAngelis, Sturmey, and Alvero (2007) used a task clarification, goal setting, and feedback intervention to improve punctuality, greeting, and correct front-desk behavior of student advisors in a university counseling center. Loewy and Bailey (2007) used graphic feedback, goal setting, and manager praise to improve customer service (i.e., greeting, eye contact, and smiling) in two locations of a home improvement store. Gil and Carter (2016) used graphic feedback, later combined with goal setting and performance feedback to improve data collection adherence of direct care staff in a large residential treatment facility. It is reasonable to expect the application of these practices to the supervision of aspiring BCBAs to result in supervisees’ performance improvement in key behavior-analytic skills.

In 2016, Behavior Analysis in Practice released a special section on supervision. Prior to its publication, few articles had been published on recommended practices for the supervision of aspiring BCBAs. The special section includes articles on ethical considerations in supervision (Sellers, Alai-Rosales, & MacDonald, 2016), recommendations for conducting group supervision (Valentino, LeBlanc, & Sellers, 2016), recommendations for addressing barriers to supervision (Sellers, LeBlanc, & Valentino, 2016), an examination of the relationship between supervision hours, supervisor credentials, years of experience, and supervisor caseload on ABA treatment outcomes (Dixon et al., 2016), and recommended practice guidelines and approaches to supervision (Sellers, Valentino, & LeBlanc, 2016; Hartley, Courtney, Rosswurm, & LaMarca, 2016; Turner, Fischer, & Luiselli, 2016).

Interestingly, though the supervision model presented by Turner et al. (2016) closely resembles the process that will be presented in this paper, the two systems were established by two separate groups in two separate locations. The current paper further enhances the literature base on supervision by emphasizing that supervision is a process with a beginning, middle, and end and by providing templates to assist supervisors in carrying out that process.

Knowing the general flow of a supervisory relationship from beginning to end helps supervisors to keep their documentation organized and plan for supervisees’ training and assessment needs. This is especially important given that supervision may often be a secondary job for BCBAs, resulting in time constraints and competing job responsibilities (Garza, Peterson, McGee, Jackson, & Malott, manuscript in preparation).

In a survey to assess the needs of individuals supervising experience hours for aspiring BCBAs, 18% of participants indicated that lack of available information or materials was a barrier to using assessment in supervision. Furthermore, 30% of participants indicated that time or competing contingencies were a barrier to completing assessments in supervision. When asked to indicate barriers to the use of behavioral skills training in supervision practices, 54% of participants indicated that lack of time was a barrier, and 48% of participants indicated that there were too many skills to teach. The majority of participants also indicated that additional materials would be useful for teaching most items in sections I and II of the BACB Task List (Garza et al., manuscript in preparation). In response to these expressed needs, the purpose of this paper is to present a concise process for supervising aspiring BCBAs and to provide materials to assist supervisors in providing high-quality supervision. Supervision for BCBA experience hours can be conceptualized as a process that occurs in five phases: establishing a supervisory relationship, skills assessment, training, ongoing performance monitoring, and ending the supervisory relationship. The steps involved in each of these phases will be discussed below. Those phases that are already discussed at length in the existing literature on supervision will be covered in less detail than those for which the existing literature does not address how to complete the phase in the context of BCBA supervision. In addition, strategies for ongoing professional development will be discussed.

Establishing a Supervisory Relationship

Individuals may receive supervision as part of a university practicum, from a supervisor within the organization at which he or she works, or from a BCBA who offers contracted supervision services. Regardless of the arrangement of supervision, no experience hours can be accrued before both the supervisor and supervisee sign a contract outlining the responsibilities of each party. The BACB Experience Standards outline the specific items that must be included in the contract (BACB, 2016), and the BACB® website offers several sample contracts to use as a template. We recommend using these templates as a guide and modifying the contract to describe the specific assessments and experience opportunities that the supervisee can expect to encounter within the organization in which experience hours will be accrued. We also recommend using universities’ or other organizations’ legal or contract departments as a resource for reviewing contracts for proper legalese whenever such resources are available.

In our professional experience, we have heard supervisors express concern that they could get locked into a supervisory relationship with a supervisee who does not make adequate progress or who engages in unprofessional behavior. In their Experience Standards, the BACB specifies that contracts should, “delineate the consequences should the parties not adhere to their responsibilities (including proper termination of the relationship)” (2016, p. 3). If there are circumstances under which a supervisor would be unwilling to continue supervising an individual, those circumstances should be clearly outlined in the supervision contract in order to protect both parties.

During an initial meeting, the supervisor should describe the purpose of supervision as described in the BACB’s Supervisor Training Curriculum Outline and go over the supervision contract with the supervisee. We recommend reading over the terms of the contract aloud during the meeting, pausing after each section to allow for questions, and having the supervisee sign his or her initials after each section of the agreement. This protects both parties by ensuring that there is a legitimate and understood agreement between the supervisor and supervisee. Once both parties sign the contract, the supervisory relationship has been officially established.

Sellers, Valentino, and LeBlanc (2016) provide excellent rationale and guidelines for effective establishment of the supervisor–supervisee relationship. We recommend reviewing their practice guidelines for more detailed information on creating supervision contracts, setting clear expectations for the supervisee, specifically setting expectations regarding feedback in the supervision process, and creating a “committed and positive” supervisory relationship (p. 276). Turner et al. (2016) also discuss strategies for establishing a supervisory relationship that results in collaborative and ethical practices. In a manner similar to Sellers et al. (2016), Turner et al. (2016) emphasize the importance of setting clear performance expectations and clearly defining the supervisory relationship.

Skills Assessment

After establishing the supervisory relationship, a supervisor must determine which performances and corresponding Task List (BACB, 2012a) items the supervisee currently possesses and which will need to be developed over the course of supervision. This determination is made through a performance/skills assessment. In assessment, the supervisor is either observing performance or reviewing products of performance without providing prompts, coaching, or instructional feedback. These components of training and performance management could bias the results of the assessment. Assessment should be done before beginning the training process because it helps the supervisor pinpoint which skills the supervisee can successfully practice independently. This allows the supervisor to focus training efforts on skills the supervisee has not yet mastered. In our discussion of assessment, we will first present a method for supervisors to define the scope of their assessment, followed by a method for conducting the assessment, and, finally, a method for using the results of the assessment.

Job Model

Job models can take different forms and be called by different names (e.g., role/responsibility matrix) but typically include the accomplishments or performance responsibilities associated with the job. Additional information, such as tasks required to complete the performance, measures of performance, goals or standards for performance, inputs to performance, recipients of outputs of performance, and how feedback on performance will be provided, are sometimes also included in the job model (Gilbert, 1996; Rummler & Brache, 2012).

Because training (discussed in the next section) can be a time-consuming process, we recommend conducting an analysis of the position supervisees will hold in the organization. This analysis helps supervisors identify which skills the supervisee will actually practice throughout the supervisory relationship. If the organization cannot provide specific training on a particular skill, it is categorized as a low priority for assessment and training. If the skill is practiced regularly within the organization, it is a high priority for assessment and, if needed, training. This is not to say that the skills not practiced within the organization are not important to the field of behavior analysis or would not be worthwhile for the supervisee to learn. However, it is best to match the training setting to the environment in which the performance will occur naturally (Kazbour, McGee, Mooney, Masica, & Brinkerhoff, 2013; Mager, 1997). If there are no natural practice opportunities within the organization, it is best to leave the training to a site in which those opportunities are available.

Job models can be conceptualized as a sort of task clarification, which is a widely supported performance improvement intervention in OBM (Cunningham & Austin, 2007; Gravina, VanWagner, & Austin, 2008; Durgin, Mahoney, Cox, Weetjens, & Poling, 2014; Slowiak, 2014). Job models can be organized in different ways depending on their specific utility within the organization. In the context of supervision for aspiring BCBAs, it is appropriate to examine which Task List (BACB, 2012a) items are practiced when a supervisee engages in each given performance that is part of her position within the organization.

Appendix 1 is an example of a job model for a behavior consulting organization. On the top of the form, there is a space to indicate the name of the position (e.g., graduate therapist), the supervisor (e.g., KS), and the job purpose (e.g., to provide clinical services to adults and children with developmental disabilities and develop professional skills for providing effective and ethical treatment). Specifying the purpose of the job helps establish the scope of the position and distinguish the position from others within the organization. This decreases the likelihood of ambiguity as to whose responsibility it is to engage in a given responsibility. Job responsibilities or performances are listed in the left column of the table, and the corresponding BACB Task List items are listed in the right column. For example, a graduate therapist in the behavior consulting organization might conduct a functional behavior assessment, but they might not be responsible for conducting discrete trial training.

We identified 11 different tasks from the Task List that someone might engage in to complete a functional behavior assessment: G-01 “review records and available data at the outset of the case,” G-02 “consider biological/medical variables that may be affecting the client,” G-03 “conduct a preliminary assessment of the client in order to identify the referral problem,” G-04 “explain behavioral concepts using nontechnical language,” G-06 “provide behavior-analytic services in collaboration with others who support and/or provide services to one’s clients,” G-07 “practice within one’s limits of professional competence in applied behavior analysis, and obtain consultation, supervision, and training, or make referrals as necessary,” G-08 “identify and make environmental changes that reduce the need for behavior analysis services,” I-01 “define behavior in observable and measurable terms,” I-02 “define environmental variables in observable and measurable terms,” I-03 “design and implement individualized behavioral assessment procedures,” and I-07 “design and conduct preference assessments to identify putative reinforcers” (BACB, 2012a, pp. 6, 7). Note that job performances could be more narrowly defined if it is likely that supervisees within the organization will complete components of the larger performance independently. For example, functional assessment interview, descriptive assessment, and functional analysis can be listed as separate performances on the job model if they are likely to be practiced as discrete performances that involve the use of unique sets of skills from the Task List.

Once all of the job responsibilities and corresponding Task List items are identified for the job position, a list of all of the Task List items that can be practiced while the supervisee is in that position can be compiled. The Task List items for which there are no natural practice opportunities can be better assessed and trained in another position, perhaps in another organization. While all of the skills on the Task List are relevant to behavior analysis as a field, not all of the skills are relevant to every job position. For example, in organizations that do not provide group instruction to clients, there may be few or no opportunities to use Direct Instruction (item F-03 on the Task List). There also may not be an opportunity to use a full range of experimental designs (Section I.B. on the Task List) within many organizations. Although these skills are valuable, eliminating them from a training program allows the supervisor to focus on teaching the skills that are immediately relevant for the supervisee’s role in the organization. This, in turn, eliminates some competition between training supervisees and the supervisor’s other job responsibilities. Once all of Task List items that can be practiced within a supervisee’s job role have been identified, supervisors can conduct a focused assessment to determine training needs.

Assessment of Skills

Assessing supervisees’ ability to perform the Task List items identified as relevant to the supervisee’s job position can also decrease the time supervisees spend on training. Assessment helps determine which skills must be trained and which can be practiced without training, thereby preventing unnecessary training and premature independent practice. Furthermore, initial assessment allows supervisors to obtain a baseline measure of their supervisees’ performance, which allows them to evaluate their own supervisory effectiveness (Kazdin, 1982). There are different methods to assess supervisees’ ability to perform job responsibilities. We recommend the following sequence of assessment: self-assessment, oral and written quizzes, and validation of self-assessment through review of permanent products and observation.

Turner et al. (2016) discuss the use of initial baseline assessment, including the use of direct observation; use of informal assessment such as interviews, conversation, and portfolio review; review of course syllabi; discussion of progress with previous supervisors; and review of the Task List. Here, we present multiple steps of baseline assessment and discuss a tool that can be used to organize assessment data for ongoing use in the supervision process.

Self-assessment involves having the supervisee rate her skill level for each job responsibility and/or task list skill. While behavior analysts might be hesitant to use self-assessment methods in their supervision practices, asking a supervisee to rate her ability to perform a skill has some benefits. Sellers et al. (2016) recommend having supervisees self-evaluate their ability to perform job responsibilities, noting that such an evaluation might help supervisors target their training efforts appropriately. If a supervisee indicates that she has no or limited experience with a task or that she would need coaching to perform the task, she may actually not have the skills to perform that particular task. In this situation, her self-assessment would indicate a training need. However, it is also possible that her rating does not match her actual skill level (i.e., the supervisee might actually perform the task at a mastery level). Even in this situation, it would be worthwhile to provide some training so that the supervisee can learn to tact her own correct performance and better recognize when she has made an error. Turner et al. (2016) point out that the level of rapport between a supervisor and supervisee may affect a supervisee’s tendency to seek her supervisor’s guidance over the guidance of others. Providing training when a supervisee indicates that her ability to perform a skill is below mastery levels could improve rapport and increase the frequency with which a supervisee asks for help when needed.

On the other hand, if a supervisee indicates that she could perform the task independently or even coach others on the performance, her performance could indicate otherwise (i.e., her performance could be well below mastery levels). Parsons, Rollyson, and Reid (2012) distinguish between performance skills and verbal skills in their discussion of the behavioral skills training (BST) model (discussed below). Just as it is recommended that training differ depending on the nature of the skills to be trained, we recommend different methods for verifying supervisee’s self-reports depending on the nature of the skills being assessed. Verification of self-assessment can be done via quizzes and interviews for knowledge-based or verbally based skills (e.g., Section III: Foundational Knowledge of the Task List; skills that involve explaining concepts or treatment programs and skills that involve clinical decision-making) and review of permanent products and direct observation for performance-based skills. If the supervisee has a portfolio of work she has done in the past, evaluating the items in the portfolio allows supervisors to assess skill levels without scheduling an observation. In fact, we recommend encouraging supervisors to keep a record of work products for this reason. For items that cannot be assessed via quizzes, interviews, and portfolio review, role play or in vivo, direct observations can be used.

The authors have developed a tool for organizing assessment data. Readers may access this tool by emailing the first author. The tool presents all items of the BACB Fourth Edition Task List and dropdown menus with self-assessment options for each Task List item. The supervisee can self-assess their ability to perform each skill by selecting an item from the dropdown menu. Once the supervisee completes the self-assessment, the assessment results populate other tabs in the spreadsheet, which correspond to each section of the Task List. For each Task List item, there is a space for the self-assessment standing (which is automatically populated from the self-assessment), a broad training goal (train, generalize, monitor, or lead/train others), validation method (portfolio review, natural observation, alternative training method, or N/A), and status (validated, in training, or monitoring). There is also a space for the supervisor to make comments regarding the supervisee’s progress on each Task List item.

Goal Setting

A variety of studies have examined the effects of goal setting on performance. Goal setting has been used successfully in interventions to improve the performance of soccer players (Brobst & Ward, 2002), to increase safe methods of passing sharp instruments in a hospital operating room (Cunningham & Austin, 2007), to increase cashiers’ checking of identification in a grocery store (Downing & Geller, 2012), and to decrease office workers’ bouts of prolonged sitting (Green, Sigurdsson, & Wilder, 2016). Additionally, Turner et al. (2016) discuss goal setting as one method for teaching and promoting skills within the supervisory relationship. Here, we introduce steps supervisors can take to set appropriate goals for their supervisees.

The results of skills assessment can be used to set long-term and short-term training goals for the supervisee. Long-term goals can be reviewed each quarter or semester, and short-term goals can be reviewed each supervisory period. Other time frames can be used as well, depending on the nature of the supervisory relationship. Goal setting interventions often involve the use of baseline performance data to set intervention goals that are achievable and that, if met, would result in meaningful performance improvement (e.g., Downing & Geller, 2012; Green et al., 2016). Additionally, there is some evidence that setting goals based on an individual’s ability results in more pronounced performance improvements over goals that are assigned to an entire group, especially for low performers (Jeffrey, Schulz, & Webb, 2012).

To set appropriate long-term goals, supervisors should consider the Task List items that are included in the job model, the supervisee’s overall progress on mastering those Task List items, and the anticipated needs of the organization during the review period. For example, if the supervisee has observed her supervisor collecting interobserver agreement (IOA) data, but has not yet had the opportunity to practice independently, long-term goals for the quarter might include collecting integrity data independently with 80% or better (IOA) with her supervisor. This goal could take one or several sessions to achieve. If the supervisee has never observed the process of writing a behavior support plan, one long-term goal for the quarter might be to have the supervisee take part in the treatment planning process and read the resulting behavior support plan.

Once goals are determined for the quarter or semester, short-term goals can be established with long-term goals in mind. To set appropriate short-term goals, supervisors should consider the immediate needs of the organization as well as their supervisee’s ability to meet those needs. For example, if the organization is serving a client who needs a functional behavior assessment, the supervisee’s demonstrated abilities related to functional behavior assessment will determine how much independence the supervisee should be allowed when performing the task. If the supervisee indicated on her self-assessment that she has no or limited experience in FBA, training goals for the supervisory period might include the supervisee’s observation of the planning process. If the supervisee has successfully completed a functional behavior assessment, training goals for the supervisory period might include the supervisee writing the protocol without guidance from the supervisor. The supervisor could then evaluate the protocol and provide positive and corrective feedback. A form for setting goals and delivering performance feedback is available in Appendix 5. See the section on “Ongoing Performance Monitoring” below for a more detailed description of the Feedback and Goal Setting Form.

Training

The Supervisor Training Curriculum Outline from the BACB lists “Behavioral skills training of supervisee” as an important feature of supervision (2012, p. 2). In addition, the BACB Experience Standards lists behavioral skills training as a component of “effective behavior-analytic supervision” (2016, p. 2). Therefore, our discussion of training is based on the behavioral skills training (BST) model.

Behavioral skills training is an approach to training that includes five major components: rationale and instruction, modeling, opportunity to practice, feedback, and repetition of practice and feedback until the learner achieves mastery (Parsons et al., 2012). Some models also include assessment of generalization to other clients, settings, or behaviors (BACB, 2012b, p. 3).

To measure the correctness of a performance, the correct performance must be operationally defined (Cooper, Heron, & Heward, 2007). Before implementing BST, one must define the target performance and create the necessary supporting materials. We recommend beginning with a task analysis. A task analysis breaks a performance down into its component steps. Behavior analysts often use task analyses to break down target client performances into smaller steps and then use chaining or shaping to improve client performance of the target. In training, task analyses are used in the same way, but the focus is on supervisee performance rather than client performance. See Appendix 2 for an example of a task analysis template. Our template includes a place to indicate the performance to be taught, the triggering event (i.e., the stimulus that indicates it is time to begin the performance), and the ending event (i.e., permanent products or changes in the environment that define the termination of the performance). Identifying the triggering event and the ending event helps to define the scope of the performance, which later helps to define the scope of the corresponding training. The template also has a place to indicate the tasks involved in the performance. For each task, there is a table to note the materials that are needed to perform the task and the designated location for those materials. Additionally, the template has a place to indicate any steps, sub-steps, or decisions involved in the task. Some performances may have only one task, while others might have multiple tasks, steps, sub-steps, and decisions.

Task analyses make teaching job performances easier because the target skills are broken into smaller, more manageable, sequential steps (Mager, 1997). Once the performance is broken into observable and measureable sub-steps, the task analysis can be used to create a data sheet. The data sheet can be used to guide an objective observation of the target skill. Data sheets built from a task analysis are more likely to capture the behaviors critical to successful performance. As supervisors build a portfolio of task analyses and data sheets related to the BACB Task List, they can be used to guide initial skills assessments as well.

After completing a task analysis, it is helpful to create a training storyboard to facilitate the training process (Bunch, 1991). A storyboard is a training design document that prompts the trainer to identify the rationale for the performance, to develop instructions for the trainee, to determine how the performance will be modeled and what he or she will say while modeling the performance, to determine what practice opportunities will look like for the trainee, to determine how feedback will be delivered to the trainee, and to determine how the trainee’s performance will be evaluated to determine whether mastery has been achieved. In other words, the storyboard helps ensure that trainers are, in fact, designing BST with each of its components and not some other form of training, which may or may not be evidence-based. Appendix 3 is a storyboard template that supervisors can use to develop their trainings.

Once the training storyboard is developed, all materials to be used in the training must also be developed or gathered. Examples of training materials include written instructions, video-based models, practice activities, written tests, and materials that are necessary for running protocols. Once the performance to be trained has been analyzed, the training has been designed, and the training materials have been developed, supervisors can implement BST according to the steps outlined in the storyboard.

Ongoing Performance Monitoring

Behavior analysis is not only a science of learning in the sense of skill acquisition. In fact, a significant portion of the field is dedicated to understanding the variables that increase, decrease, or maintain behavior after it is established (Skinner, 1950). While the BACB Experience Standards state that, “the supervisee’s primary focus should be acquiring new behavior-analytic skills related to the BACB Third Edition Task List or the BACB Fourth Edition Task List as appropriate,” supervisees also must maintain satisfactory performance after new skills are established (2016, p. 5).

The Experience Standards require supervisors to observe their supervisees in the natural environment at least once each supervisory period (2016, p. 2). As Komaki, Zlotnick, and Jensen (1986) point out, “only managers who systematically monitor and collect data are likely to be knowledgeable about subordinates’ performance” (p. 261). After training is complete for a given performance, ongoing monitoring serves to continue the development of the supervisee’s skills and to assess for generalization to new clients, targets, and settings. Ongoing performance monitoring involves regularly conducting formal and informal assessments of supervisee’s skills, updating goals according to assessment results, and delivering performance feedback.

Because we have already reviewed the process of conducting assessments and establishing goals, this section will focus on the delivery of performance feedback. Feedback is information about performance that allows a person to change that performance in the future (Daniels & Bailey, 2014). Effective feedback contains specific information; pertains to performances under the control of the performer; is individualized; is self-monitored when possible, if not self-monitored, is delivered by a person in charge; is focused on improvement; is easily understood; is graphed; is used in conjunction with reinforcement; and is provided as often as feasible (Alvero, Bucklin, & Austin, 2001; Balcazar, Hopkins, & Suarez, 1985-86).

Delivering corrective feedback deserves additional discussion, as it can easily be done inappropriately or ineffectively. For example, anger is a motivating operation that might increase the value of the reinforcing effects of yelling or other aggressive behaviors. Therefore, feedback should not be delivered when angry. Delivering corrective feedback in public could be perceived by others in the organization as unfair, which might increase the reinforcing effectiveness of behaviors that are inconsistent with the instructions given by the supervisor. Feedback should therefore be given privately. Supervisors should also avoid threatening consequences that they cannot deliver, so that their feedback does not lose effectiveness in managing their supervisees’ performance (Daniels & Bailey, 2014). To deliver corrective feedback, supervisors must describe the ineffective performance and the desired performance, provide rationale and instructions for the desired performance, model the correct performance, provide opportunities for the supervisee to practice the desired performance, provide feedback again, and follow-up as necessary (BACB, 2012b, p. 4). Note that this is the same process that is followed for behavioral skills training.

In their Supervisor Training Curriculum Outline, the BACB states that supervisors should be able to describe several different methods of delivering performance feedback: vocal, written, modeled, video, graphic, self-monitored, formal, and informal (2012c, pp. 4–5). Appendix 4 is a table that gives a description of each of these feedback methods and situations in which each method might be appropriate. Appendix 5 is a Supervision Feedback and Goal Setting Form that supervisors can use to guide their written and vocal feedback and establish goals for the next supervisory period. The form prompts the supervisor to identify the job performances that were practiced during the supervisory period, their supervisee’s current (validated) level of self-assessment, and goals for the upcoming supervisory period (or the next time the performance will be completed). The form also prompts supervisors to comment on their supervisee’s professional behavior during the supervisory period and provides space for the supervisor to write general comments. In addition, the form prompts supervisors to select several areas of focus for the next supervisory period. The Supervision Feedback and Goal Setting form can be used in conjunction with the tool for organizing assessment data that we described in the assessment section. Whereas assessment data may be collected only several times throughout the supervisory relationship, the Supervision Feedback and Goal Setting form can be used to document progress every supervisory period, allowing for broad analyses of performance change as well as more molecular analyses.

Ending the Supervisory Relationship

There are several steps supervisors should follow to appropriately conclude a supervisory relationship: evaluation of the supervisee’s overall skill development, evaluation of the supervisor’s supervision skills, and completion of documentation required by the BACB. See Turner et al. (2016) for additional considerations for unplanned terminations of supervisory relationships.

Evaluation of the Supervisee’s Overall Skill Development

As discussed earlier, evaluation of supervisees’ performance is, ideally, an ongoing process throughout supervision. However, to remain consistent with the “analytic” dimension of applied behavior analysis, the supervisee’s performance should be evaluated before, during, and after supervision to provide a measure of the supervisee’s overall progress and the impact of supervision (Baer, Wolf, & Risley, 1968; Turner et al., 2016). Quizzes on the Foundational Knowledge section of the Task List, self-assessment measures, direct observation, and reviews of permanent products—the same assessment tools and methods that are used to conduct an initial assessment of the supervisee’s skills—can be used to obtain an overall measure of the supervisee’s progress throughout the process of supervision (see the “Assessment of Skills” section above for a more detailed discussion of these tools). While the initial assessment gives a baseline from which training and performance goals can be established, the final assessment gives a post-supervision evaluation. Comparing the pre-supervision performance with post-supervision performance can provide a measure of the supervisee’s progress and growth over the course of supervision, which allows the supervisor to evaluate his or her own effectiveness.

During the final supervision meeting, the supervisor should discuss the results of the supervision evaluation. Pre- and post-supervision graphs of supervisee performance, summary statements from pre- and post-assessments, the supervisee’s updated portfolio, and written recommendations for which skills to focus on improving through subsequent supervision experiences should guide discussion during the final supervision meeting.

Evaluation of the Supervisor’s Effectiveness

Comparing pre- and post-assessments of supervisees’ self-assessments and work products not only provides a picture of the supervisees’ progress; it also provides a rough measure of the supervisor’s effectiveness. In addition to evaluating the supervisee’s progress, supervisors can ask supervisees to complete an evaluation of their supervisory effectiveness. Consistent with the recommendations of Turner et al. (2016), we also recommend that supervisors self-evaluate their supervisory effectiveness. Additionally, we recommend using the BACB (2012b) Supervisor Training Curriculum Outline to guide such an evaluation, as this document outlines what the BACB considers to be essential features of supervision. Supervisors can self-evaluate their supervisory effectiveness using the same method and compare their self-evaluations to their supervisees’ evaluations to help them set appropriate goals for professional development. When possible, it is a good idea to have peers or mentors observe and evaluate supervisory performance to obtain a direct measure of supervisory behavior.

Professional Development Planning

Planning to improve supervision practices takes time and effort. Appendix 6 is a template based on the literature on performance management (e.g., Malott, 2003, 2004), task clarification (e.g., Slowiak, 2014; Wilson, Boni, & Hogg, 1997; Slowiak, Madden, & Mathews, 2006), goal setting (e.g., Slowiak et al., 2006; Anderson, Crowell, Doman, & Howard, 1988), and reinforcement (e.g., Skinner, 1999/1961; Skinner, 1999/1968) that supervisors can use to establish goals and deadlines, determine the actions required to achieve the goals, and create a reinforcement plan to follow when those goals have been met. We hope that supervisors will use this tool in their own practices to add to their implementation of effective supervision practices.

Conclusion

BCBA supervision is a process concerned with teaching new behavior-analytic skills and maintaining those skills after they have been mastered. As a field that has developed effective techniques for increasing, decreasing, and maintaining behavior, it makes sense that we would use the same technology to teach and maintain the skills of those in training to become professional behavior analysts and to improve the effectiveness of supervision practices. This paper has proposed a general process to follow while supervising aspiring BCBAs as well as materials to guide the implementation of that process. We encourage supervisors to use these materials and to share training protocols with others with the goal of maintaining the integrity of behavior analysis and keeping an emphasis on data-based decision-making.

Compliance with Ethical Standards

This project was funded by the Michigan Department of Health and Human Services (Grant no. 20152526).

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.