Towards a Competency-Based, Ethical, and Socially Valid Approach to the Supervision of Applied Behavior Analytic Trainees

Abstract

Competency-based supervision of trainees has recently come to the forefront of behavior analytic practice; however, there are minimal data to support the effectiveness of various supervision practices on trainee outcomes. Accordingly, this paper is intended to spark further discussion and research activity regarding the supervision of those seeking to become Board Certified Behavior Analysts (BCBA). We present a practice model and considerations for supervising applied behavior analytic trainees consistent with the Behavior Analyst Certification Board (BACB) Supervisor Training Curriculum Outline (Behavior Analyst Certification Board, 2012b), the Professional and Ethical Compliance Code for Behavior Analysts (Behavior Analyst Certification Board, 2014), and extant literature from behavior analysis and related fields. Inherent to the current model is a focus on bi-directional feedback and collaboration between the supervisor and trainee to frequently evaluate the acceptability of the procedures, process, outcomes, and effectiveness of supervision. We present a Supervision Monitoring and Evaluation Form consistent with the current model and discuss the assumed importance of objective and subjective self-assessment of supervisor competence to the ultimate advancement of the practice of applied behavior analysis.

Electronic supplementary material

The online version of this article (doi:10.1007/s40617-016-0121-4) contains supplementary material, which is available to authorized users.

Supervision is a crucial activity of most Board Certified Behavior Analysts (BCBA) and is especially important given the growing number of applied behavior analytic trainees and the expanding demand for applied behavior analysis (ABA) services worldwide. It is generally assumed that quality supervision will contribute to the development of effective and ethical practitioners, protection of clients, and advances in the field. Initial research also suggests that the intensity of supervision provided by applied behavior analysts is positively related to clinically significant improvements in client performance (Eikeseth, Hayward, Gale, Gitlesen, & Eldevek, 2009). However, until recently, there were minimal standards on how to provide a valuable supervision experience for trainees. In 2011, the Behavior Analyst Certification Board (BACB) initiated a Supervision Task Force to develop definitive supervision guidelines and training standards for the field (Behavior Analyst Certification Board, 2011). Resulting from this task force were minimal competencies required of those providing supervision to individuals pursuing a BACB certification or practicing as Board Certified Assistant Behavior Analysts (BCaBA) or Registered Behavior Technicians (RBT; Behavior Analyst Certification Board, 2012b). Relevant issues were designing a supervision contract, implementing behavioral skills training (BST), and delivering frequent performance feedback.

Underlying this competency-based approach is the assumption that, rather than “osmosis” (Falender & Shafranske, 2012, p. 133), supervision is an independent area of practice and requires acquisition, refinement, and maintenance of skills; although strong clinical and research skills are essential to being a competent supervisor, these skills alone do not automatically equate to strong supervisory skills. Attention is now being given to the development of supervisor competencies and identification of effective components of supervision, a process mirrored by several other helping professions, such as social work (Kraemer-Tebes et al., 2011), psychology (American Psychological Association, 2015; Falender et al., 2004), and medicine (Frank et al., 2010).

The purpose of this paper is to present a practice model that integrates evidence-based training and supervision practices with the complex ethical and practical considerations regarding supervision. Because there are currently limited data on the effectiveness of supervision and specific supervision practices in developing well-trained BCBAs, BCaBAs, and RBTs, the current model is guided by the supervisory experiences of the authors, extant literature from behavior analysis and related fields, the BACB Supervision Training Curriculum Outline (Behavior Analyst Certification Board, 2012b), and the newly developed Professional and Ethical Compliance Code for Behavior Analysts (Behavior Analyst Certification Board, 2014), which now more comprehensively reflects the professional responsibilities of supervisors. Some of these recommendations may be time-consuming and go beyond the requirements of the BACB; however, these points of consideration on the application of existing supervisory and ethical guidelines may enhance supervisory practice. Note that we have adopted the term “trainee” throughout to refer to BCBA and BCaBA candidates, as well as practicing BCaBAs and RBTs as they require ongoing supervision and cannot practice independently.

Within this framework, we review the many competencies required of supervisors including fluency with the ethical guidelines regarding supervision, effective training techniques across basic and complex skill areas (e.g., clinical decision-making and professionalism), and performance feedback methods to support acquisition and maintenance of skills. Our premise is that the implementation of an evidence-based approach within a positive, collaborative, and individualized environment fosters a well-rounded and effective trainee. We also provide examples and considerations for how to incorporate these competency areas into a practice repertoire in which supervisor behavior, in addition to trainee behavior, is frequently evaluated and modified based on trainee feedback and progress. Focus is placed on the use of social validity measures to aid in the ongoing adjustment of supervisory behavior and the ongoing self-assessment of the supervisor.

Practice Model for Behavior Analytic Supervision

Set the Occasion for Collaborative and Ethical Supervision

A strong and collaborative supervisory relationship is widely acknowledged as contributing to the quality and effectiveness of clinical supervision across several other disciplines (e.g., Bernard & Goodyear, 2014; Bordin, 1983; Falender et al., 2004). Although notoriously complex and difficult to objectively define and measure, the supervisory relationship is typically defined in two parts. In behavior analytic terms, the first part can be described as the degree to which a supervisor and a trainee are mutual discriminative stimuli for generalized reinforcement (i.e., “rapport” as discussed by Carr and colleagues, 1997, p. 111). Carr and colleagues (1997; Magito, McLaughlin, & Carr, 2005) further discuss rapport as a setting event and demonstrate that the development of rapport is a key aspect to approaching skill development. Although their discussion is specific to individuals with developmental disabilities, the basic behavioral principles behind their observations have clear implications for the supervisor-trainee relationship.

As an example, the relative rapport between a supervisor and trainee may impact the trainee’s approach behavior towards the supervisor. If a supervisor does not function as a cue for reinforcement (or the supervisor functions as a cue for punishment), it is likely that the trainee will avoid the supervisor and instead seek out others, including non-behavior analysts, for guidance and feedback. While the specific behaviors required to develop rapport within the context of the supervisory relationship have not yet been systematically identified, a few face valid examples that may assist supervisors in establishing themselves as reinforcers include being approachable (e.g., visibility and nonverbal behaviors) and responsive (e.g., to emails), using more positive statements relative to negative (e.g., 4:1 ratio; Daniels & Bailey, 2014), and demonstrating undivided attention towards the trainee during meetings. Importantly, these behaviors certainly do not ensure that the supervisor will produce a competent trainee, and other variables are also likely to impact the degree to which the trainee approaches the supervisor with relevant issues and questions (e.g., supervisor competency).

The second component of the supervisory relationship can be described as the degree to which a supervisor and trainee generate mutually agreed upon expectations and goals for supervision and engage in behavior consistent with those expectations and goals. The supervision contract, a requirement of BACB approved supervision (Behavior Analyst Certification Board, 2015), can be used to guide the development of the supervisory relationship and the terms of supervision (see Guideline 5.05; Behavior Analyst Certification Board, 2014). The contract fosters an initial and collaborative discussion between a supervisor and trainee with regard to the nature of the professional relationship and the expectations and responsibilities of each member. Supervisors can use this discussion to model shared expectations and commitment to the supervision process.

Self-report data from related fields suggests that a poor supervisory relationship is related to a trainee’s decreased willingness to disclose relevant clinical information (Mehr, Ladany, & Caskie, 2015) and follow through with a supervisor’s recommendations (Ladany, Hill, Corbett, & Nutt, 1996), as well as increased likelihood of burnout (Gibson, Grey, & Hastings, 2009). However, to what extent the quality of the supervisory relationship influences the outcomes of supervision is unknown and should be a focus of future research (see an extended discussion below). Until then, we hypothesize that the development of a technically skilled, ethical, and professional behavior analyst may potentially suffer without the presence of a strong supervisory relationship. Thus, an important first, and ongoing, step in the supervision process may be to build a strong supervisory relationship. As part of, or in addition to the supervision contract, we suggest supervisors participate in the following discussions with trainees to further establish mutual expectations for the supervision experience and ensure consideration of relevant ethical guidelines in an effort to avoid potential harm to trainees and the professional supervisory relationship.

Define the Relationship

The first step in any professional relationship is to define basic and fundamental expectations between participating individuals. While there are clear examples of inappropriate supervisory relationships that often do not necessitate a lengthy initial discussion (e.g., sexual relationships; see Guideline 1.07; Behavior Analyst Certification Board, 2014), subtle aspects of the supervisory relationship may be unclear to the trainee and should be clarified. The supervisory relationship can be confusing as the supervisor may serve as a teacher, mentor, evaluator, and facilitator of self-assessment and growth, all of which are overlapping roles and imply a slightly different style of professional interaction and power differential. For example, a supervisor is commonly defined as someone who oversees certain activities and holds evaluative power over the trainee, while a mentor is often described as a trusted advisor and confidant. Further, the everyday practice of behavior analysis can be stressful, often physically and mentally exhausting, making initial discussion of a supportive relationship beneficial.

While all these roles are supervisory in nature and holding these roles concurrently with a trainee does not necessarily constitute a multiple relationship as defined by the BACB (Behavior Analyst Certification Board, 2014), supervisors may find themselves differentially allocating their time across several of these roles depending on the current context of supervision and the trainee’s experiences. It may be beneficial to discuss these potentially changing roles at the outset of supervision and to engage in self-assessment and monitoring, as well as peer consultation, to ensure the role fits the current needs of the trainee while still within the parameters of a professional relationship.

Further, trainees will already have negative and/or positive experiences working with people in these various roles and may have preconceived expectations about the nature of the current supervisory relationship. For example, trainees with past supervisors who did not encourage collaboration may be reticent to present contrasting ideas or raise concerns with their new supervisors. It may be fruitful for supervisors to ask trainees about their definition of supervision and their previous experiences being supervised (whether or not in behavior analysis) and to discuss how their definition and experiences align with those of the supervisor and the most recent version of the BACB Experience Standards (i.e., Behavior Analyst Certification Board, 2015). Also important is the verbal recognition of and sensitivity to the fact that the supervisory relationship can be impacted by beliefs, values, and interpersonal biases shaped by previous experiences. Discussing such issues may reduce the likelihood of dissimilar views and expectations regarding the supervision process.

Paying for supervision further complicates the role of the supervisor and trainee. Many individuals seeking a BACB credential do not have access to local or “in-house” BCBAs who are approved to provide supervision. This challenge requires those seeking supervision to identify approved BCBA supervisors in their community or even across great distances (through videoconferencing). When seeking supervision through this format, supervisors typically charge an hourly rate, which is determined by the supervisor and payment for supervision services must be discussed with the trainee at the onset of the supervisory relationship. Since trainees pay for services in this arrangement, the supervisor and trainee must consider any potential negative impact payment may have on the professional supervisory relationship.

Performance Expectations

It is important to clearly define, outline, and agree upon the performance expectations and responsibilities of both the trainee and supervisor. Dual responsibilities foster a give-and-take relationship and allow for shared accountability. To ensure a clear understanding of the expectations of the trainee and supervisor according to the BACB, we recommend reviewing the BACB Experience Standards (Behavior Analyst Certification Board, 2015) at the outset of supervision, even if the trainee has already accrued supervision hours with another supervisor. It is also important to identify any idiosyncratic expectations of the trainee and/or supervisor that, if not discussed, may place stress on the supervisory relationship. Such expectations may be that both parties must contact each other at least 24 h in advance to reschedule or that both parties will respond to email correspondence within 48 h.

The degree to which either a trainee or supervisor can consistently complete the agreed-upon responsibilities is often impacted by an individual’s time and competing commitments. Conducting an analysis of time for both parties is one method for outlining reasonable responsibilities. With regard to the supervisor’s time, consider a supervisor supervising one trainee who is accruing 30 h/week of Supervised Independent Fieldwork. Per the Experience Standards (Behavior Analyst Certification Board, 2015), this equates to 3 h of supervision for every 2-week supervisory period (i.e., 5 % of total accrued hours); however, 3 h of supervision almost always requires more than 3 h of time. Often overlooked in the initial calculation of time is (a) travel time to and from the trainee’s workplace for direct observations (i.e., if different from the supervisor’s workplace and if audio-video recording or videoconferencing is not viable), (b) travel time to and from supervision meetings (i.e., if videoconferencing is not an option), (c) time to search for appropriate articles and resources for the trainee, and (d) time to read (or re-read) those articles. Additional time is required in responding to trainee emails, reviewing trainee permanent products, giving detailed feedback during the supervision meeting, completing performance evaluations, documenting various aspects of the supervision process (e.g., remediation), and seeking peer feedback, and/or consultation with regard to the supervisory behaviors of the supervisor. Taken together, these activities can quickly increase those 3 h every 2 weeks to 6 h or more every 2 weeks! We encourage supervisors and trainees to complete this activity together to establish appropriate workloads and expectations for both parties (see Guideline 5.02; Behavior Analyst Certification Board, 2014).

Evaluation of Performance Expectations

One of the main responsibilities of supervisors is to consistently evaluate and document trainee performance. This matter is often an area of trainee distress and an area without strict guidelines, so it is crucial to gain initial agreement about the behaviors to be evaluated (e.g., technical, professional, and ethical), the evaluation process, and the consequences of poor performance evaluations. Agreeing on the terms of a poor evaluation of the trainee beforehand may make execution of the set consequences more straightforward and perhaps have fewer negative side effects. Include in the guidelines clear, objective and measureable circumstances for all consequences. For example, at what point will a remedial plan be developed (e.g., after 3 consistent supervisory periods of unsatisfactory performance), what will the remedial plan consist of (e.g., an additional individual meeting with the supervisor, extra readings), and what is the consequence of performing poorly during, or not completing, the remediation (e.g., supervisor terminating the relationship)? Lastly, if the relationship is ultimately terminated, it will be important to pre-determine what, if any, hours the supervisor will approve.

Supervisors are highly encouraged to seek feedback from trainees concerning their performance as a supervisor and their ability to follow through with the agreed upon expectations. We recommend communicating the value of this approach to the trainee at the outset of supervision to further create a sense of shared accountability with respect to obtaining the goals of the supervisory relationship. Obtaining candid information from trainees can be difficult (Mehr, Ladany, & Caskie, 2010), and supervisors must consider the inherent power differential; however, a strong supervisory relationship between the supervisor and trainee and an expectation of the reciprocal nature of feedback is hypothesized to counteract this limitation.

Confidentiality

The BACB Professional and Ethical Compliance Code (Behavior Analyst Certification Board, 2014) states that behavior analysts have a primary obligation to protect the confidentiality of the individuals they supervise (Guideline 2.06). Confidential information in a supervisory relationship predominantly includes performance evaluations, such as written reviews or ratings on evaluation forms. It may be beneficial to initially communicate to trainees that this information will be kept out of public sight and only discussed with others that have a clear, professional reason for being concerned with such matters, such as a trainee’s on-sight supervisor. It may also be beneficial to let trainees know that individual corrective feedback will be given in private and not during group supervision sessions.

Given the sometimes close, professional relationships that can develop through supervision, the trainee may disclose personal information germane to his or her behavior analytic work. As an example, a trainee may acknowledge an interpersonal conflict with a professional colleague. Direct conversation about the confidentiality of such disclosures may increase the likelihood that a trainee approaches their supervisor confidently and without fear of disclosure so that the issue may be discussed and an appropriate solution determined.

Remote Supervision

With the advent of contemporary videoconferencing technology, behavior analytic supervision can occur remotely by interacting through live two-way audio-video communication across the Internet or cellular networks. Supervision through videoconferencing allows supervisors to engage in supervision sessions with trainees without having to be in the same physical location, which results in reduced travel time, increased savings associated with travel costs, and the potential to provide supervision for individuals in rural or hard to reach areas who may not have access to locally qualified behavior analysts.

Despite these benefits, supervision conducted through videoconferencing also poses many potential challenges. These challenges include breaches of client confidentiality, technical difficulties, and lack of physical presence of a supervisor during high-risk situations (Florell, 2016). Supervision conducted via videoconferencing must ensure client confidentiality by facilitating sessions through HIPAA and FERPA compliant videoconferencing software (e.g., Adobe Connect, Vidyo, VSee); however, due to perpetual changes to videoconferencing software, supervisors should thoroughly scrutinize potential software before using it during remote supervision. Readers are also encouraged to review Cavalari, Gillis, Kruser, and Romanczyk (2015) for an extended discussion regarding digital sharing and storage of audio-visual recordings.

Regarding technical difficulties, supervisors and trainees should be familiar with the videoconferencing software that is used, ensure that they have the required hardware for high-quality videoconferencing (e.g., 720p or higher resolution webcam, high bandwidth Internet connection, computer/laptop/tablet with fast processing speed), and become familiar with the evidence-based practice guidelines for telemental health services (cf. Grady et al., 2011). To guarantee support for trainees during high-risk situations, the supervisor should develop a plan with the trainee so that an on-site supervisor is available or the supervisor can be reached by phone (Abbass et al., 2011; Panos, Panos, Cox, Roby, & Matheson, 2002).

Termination of the Relationship

Just as expected with clients, the parameters of planned or unplanned termination of the supervisory relationship should be discussed at the outset of supervision (Guideline 2.15; Behavior Analyst Certification Board, 2014). The BACB encourages trainees to accrue experience hours with different supervisors. Accordingly, an initial conversation about the number of supervisors trainees have already worked with and the number of experience hours trainees have accrued is warranted to determine a mutually agreed upon duration for the current supervisory relationship. Additionally, if the trainee is transferring to another supervisor, we encourage the previous supervisor to speak with the new supervisor about the strengths and weaknesses of the trainee and current goals. This dialogue should be initiated with approval from the trainee. In the event of an unplanned termination such as the supervisor changing employers, development of a contingency plan will ensure the trainee continues to receive supervision.

Baseline Assessment of Trainee Skills

Although there are no standard procedures to initially assess trainee skills, we can incorporate strategies from the field of behavior analysis to guide our evaluation. We suggest the following methods as strategies to evaluate trainee baseline performance across various skills: behavioral observation, informal assessment, review of course syllabi, discussion with previous supervisors, and review of the BACB Fourth Edition Task List (Behavior Analyst Certification Board, 2012a). Using these indirect and direct methods, the supervisor and trainee are able to collaboratively develop goals that are objective, measureable, and obtainable (Cooper, Heron, & Heward, 2007; Daniels & Bailey, 2014). We will discuss each of these strategies and their contribution to assessing trainee skills.

Behavioral observation, the basis of all behavior analytic work, requires a behavior analyst to operationally define behaviors of interest and subsequently record the occurrence of those behaviors. In the context of supervision, a supervisor should observe and record specific trainee behavior such as the extent to which an intervention or behavioral support plan is implemented with integrity (Gresham, Gansle, & Noell, 1993), verbal behavior of a trainee as they interact with staff, colleagues, clients, or parents, and reliability of trainee behavioral observation through assessment of inter-observer agreement (Cooper et al., 2007).

Informal (formative) assessment, although not behavioral, is useful to gather information from a trainee. This type of assessment is frequently used in education (Dunn & Mulvenon, 2009) and is achieved through conversations with trainees about their previous experiences, completion of checklists, and review of writing samples or portfolios. Such assessment should combine records review, semi-structured interview, and discussion about the areas of practice as a behavior analyst. The informal assessment should be used to gather information about past experience but also specific areas of strength and weakness. Additionally, as an ongoing skills assessment, supervisors should ask questions of the trainees to evaluate their knowledge of behavioral procedures (e.g., “What were the four test conditions of the Iwata, Dorsey, Slifer, Bauman, and Richman (1994) functional analysis [FA]?”). Depending on trainees’ understanding of the FA literature and their prior implementation history, supervisors should ask follow-up questions (e.g., “What are pros and cons regarding the many modifications to the original—Iwata et al., 1994—FA?”). These types of questions allow supervisors to gauge trainees’ skills and trainees’ responses provide a baseline assessment of their ability to think critically and discuss behavior analytic theory broadly or more acutely.

Review of course syllabi with the trainee is quite useful when assessing skills. In a collaborative format within supervision sessions, the supervisor and trainee may benefit from discussing the topics covered in the trainee’s BACB-approved coursework. Independent of the supervision session, the supervisor is encouraged to review the syllabi to ensure that relevant readings have been assigned—this action provides the opportunity to introduce readings to the trainee that might not have been assigned. The benefit of assigning readings within the supervision context is, as a supervisor, you can work individually with your trainee and critically analyze and discuss the behavior analytic literature. Subsequently, reviewing articles is assumed to teach critical thinking that is essential for applied work and preparation for the upcoming BACB exam.

Discussion with previous supervisors is a valuable way to assess a trainee’s skill set; however, the availability and ease of these options is variable depending on the practicum site. For example, if trainees receive their experience hours at a large human service provider with multiple schools within the organization, trainees might change schools or classrooms. In this situation, the supervisors might change as well, leaving the opportunity for discourse between the prior and current supervisor.

Finally, it is recommended that the supervisor and trainee jointly review the BACB Fourth Edition Task List (Behavior Analyst Certification Board, 2012a) and use the document as a checklist of areas the trainee has already mastered and those that are still in need of instruction and practice.

Methods to Teach and Promote Skills

Acknowledging that trainees have varying levels of skills, supervisors can expect at some point that they will need to directly and systematically teach new skills. Importantly, the supervisor must attempt to match teaching techniques and performance expectations to the trainee’s current performance. For example, a novice trainee might not be fluent with all the different approaches to conducting preference assessments. In this case, the supervisor could indirectly discuss methods and provide relevant literature for the trainee to read about the topic and improve his or her knowledge. For the trainee who is familiar with the literature on preference assessments but has not actually conducted such an assessment with a client, the supervisor could model correct procedures. Lastly, for the trainee who has mastered designing and implementing preference assessments, the supervisor could speak with the trainee about possible modifications to implementing preference assessments and/or ask that the trainee uses behavioral approaches to teach the skill to another staff member or parent.

In each of the cases described above, the supervisor engages in different behaviors and those behaviors are appropriate to the trainee’s skill set. Taking the time to comprehensively evaluate the current skill level of a trainee and set appropriate behavioral goals and expectations may allow for a more positive and successful supervisory experience for the trainee. As well, this strategy allows supervisors to provide ample amounts of positive feedback to trainees. If supervisors are unaware of trainees’ current skill set, supervisors might provide a type and/or level of supervision that is inappropriate, and run the risk of confusing or overwhelming trainees, which in turn may potentially harm a client (see Guideline 5.03; Behavior Analyst Certification Board, 2014).

Within this individualized framework, it is expected that supervisors use behavioral skills training (BST) to develop novel skills among trainees (Behavior Analyst Certification Board, 2012b). BST is ubiquitous in the behavior analytic literature (Rosales, Stone, & Rehfeldt, 2009) and shown to be effective for skill acquisition in typically developing children (Himle, Miltenberger, Flessner, & Gatheridge, 2004; Jones, Kazdin, & Haney, 1981; Yeaton & Bailey, 1978), children with autism (Gunby & Rapp, 2014), teachers and paraprofessionals (Lavie & Sturmey, 2002; Sarokoff & Sturmey, 2004), and parents (Himle & Wright, 2014; Seiverling, Williams, Sturmey, & Hart, 2012; Shayne & Miltenberger, 2013). In the context of supervision, BST involves providing a rationale to the trainee about the skills to be acquired, systematic instruction of skills to the trainee, modeling the previously taught skills to the trainee, rehearsal of the skills by the trainee (i.e., either in vivo or role-play), and supervisor positive and corrective feedback to the trainee about his or her performance of the skills (Behavior Analyst Certification Board, 2012b; Parsons, Rollyson, & Reid, 2012). Despite the effectiveness of BST, to our knowledge, there are no behavioral studies that have evaluated BST to teach skills (i.e., especially relatively complex skills) to individuals pursuing a BACB certification.

It is also expected that supervisors model technical, professional, and ethical behavior (Behavior Analyst Certification Board, 2012b). The behavior of the supervisor is at the foundation of competency-based supervision. Ideally, a trainee looks up to the supervisor to learn how to behave in a variety of situations. As a supervisor, modeling technical behavior is important for many skills, such as the language we use to discuss behavior analysis and for the work we conduct with individuals. A supervisor should demonstrate mastery of the formal terminology of ABA while being competent with colloquial speech when conversing with a layperson. Supervisors should teach their trainees both the formal and informal ways of discussing terminology, while explaining the limitations of using jargon with their constituents.

In addition to the technical language, supervisors should also model technical competency when working with individuals. Throughout the supervision process, the supervisor might need to “jump in” with the trainee and demonstrate how to implement a certain procedure with fidelity. As such, the supervisor must stay current on their clinical practice. Similar to professional athletes, a technically sound supervisor should be able to model technical skills and “make it look easy.” Not only should supervisors model technical skills, they should also model professional and ethical behavior.

Professional and ethical behavior is guided in part by the BACB Professional and Ethical Compliance Code (Behavior Analyst Certification Board, 2014). The BACB ethical code, like other ethical codes (e.g., American Psychological Association), allows for interpretation and professional judgment. As a supervisor, it is important not to engage in egregious behavior including sexual relationships with a trainee (or a client/family member), accept gifts, fabricate data, or plagiarize. The previously described behaviors are clearer in the ethical code; however, there are other more subtle professional and ethical behaviors that face fewer punishers for the supervisor, but that have the potential to negatively impact the development of a trainee. Common examples of those behaviors include, answering phone calls, emails, and text messages during supervision (i.e., not providing undivided attention), being tardy and unprepared for meetings, as well as not making time available outside of supervision for emergencies, staying relevant with the literature, protecting the confidentiality of supervision, and modeling effective professional behavior, interpersonal skills and collaboration with clients, constituents and other service providers.

Developing Case Conceptualization, Problem Solving, and Decision-Making Repertoires

Using effective teaching practices, supervisors should establish fluency in trainees’ ability to behaviorally conceptualize a case, as this is a critical component of ABA (Behavior Analyst Certification Board, 2012b). The ability to effectively conceptualize a case is influenced by mastery of behavior analytic theory and content, experience applying theory in practice, and the ability/willingness to engage in critical thinking. For trainees, all of the previously described components are developing with varying levels across areas. The integration of these skills requires practice and the process of mastering case conceptualization is thought to be enhanced by supervision that includes active and critical conversations by supervisors. As mentioned earlier, engaging trainees in critical conversation is a great way for trainees to gain competency in behavior analysis; likewise, it also allows trainees to carefully consider aspects of an individual’s behavior including, but not limited to, motivating operations, discriminative stimuli, consequences, and hypothesized function. As the supervision process progresses, the trainee should be more autonomous when conceptualizing cases and the role of the supervisor should be to confirm the trainee’s conclusion, while providing feedback as necessary.

Supervisors should also focus on developing problem solving and decision-making abilities among trainees (Behavior Analyst Certification Board, 2012b). The practice of ABA is rooted in a problem-solving framework. Behavior analysts deal with a variety of problems and their ability to effectively manage those problems largely dictates their professional success. Nezu, Nezu, and D’Zurilla (2013) defined problem solving as “…a self-directed process by which individuals attempt to identify, discover, and/or develop adaptive coping solutions for problems” (p. 8). Problem solving involves identification of a problem, the formation of potential alternatives to solve the problem, a decision to choose one solution over others, and evaluation of the chosen solution. In supervision, the supervisor is responsible for fostering problem-solving skills in the trainee.

Like any behavior, problem solving can be taught using BST. The supervisor should discuss and model each of the steps of problem solving with the trainee (as needed), and particular focus should be on the decision-making step. In our experience, this step is often difficult for trainees to make because they are concerned that they will make the “wrong” choice and subsequently “mess something up.” Although making a contraindicated choice is a concern, the supervisor’s duty is to ensure that the trainee’s choice does not cause harm. Ultimately, the supervisor should shape independent problem solving as competency in problem solving and decision-making is a critical component to the independent practice of a behavior analyst.

Considerations in the Delivery of Performance Feedback

It is widely acknowledged that performance feedback is key to the development, maintenance, and enhancement of all skills (e.g., Daniels & Bailey, 2014; Mortenson & Witt, 1998). The effectiveness of frequent, genuine, positive, corrective, and specific feedback based on direct observations of behavior is commonly accepted. As such, the delivery of timely and effective feedback (i.e., feedback that improves behavior) is required of all behavior analytic supervisors (Behavior Analyst Certification Board, 2012b; Guideline 5.06; Behavior Analyst Certification Board, 2014). Feedback can be given in a variety of modalities (e.g., verbal, written, graphic, informal) and has been shown to be most effective when combined with review of data, goal setting, and modifications to antecedents and consequences (Alvero, Bucklin, & Austin, 2001; Balacazar, Hopkins, & Suarez, 1985; DiGennaro, Martens, & Kleinmann, 2007; Sanetti, Luiselli, & Handler, 2007). Due to the importance of feedback to the supervision process, there are several variables to be considered, including but not limited to the individualization of feedback, ratio of positive to corrective feedback, use of rationales, delivery of corrective feedback regarding interpersonal or subjective behavior, documentation of feedback, and a trainee’s current skill level.

The BACB Supervision Experience Form includes a sample written evaluation of trainee performance across broad domains such as timeliness, professionalism, and acquisition of target behavior analytic skills. However, the form may be modified to better fit the individual needs of the trainee (but see the BACB Experience Standards [Behavior Analyst Certification Board, 2015] for the required components of this form). For example, items can be added (e.g., trainee-specific goals) and removed (e.g., items that are consistently rated as satisfactory) or the rating scale can be adjusted to better depict the breadth of trainee performance in any one area (e.g., 5-point rating scale). Individualizing the form collaboratively with the trainee at the outset of, and throughout, the supervision process may increase the relevancy of the feedback for the trainee, demonstrate the supervisor’s commitment to supervision, and allow the supervisor to focus feedback on the trainee’s current goals.

Feedback provided with a high positive to negative ratio is recommended within the performance management literature (e.g., Daniels & Bailey, 2014). Maintaining a supervisor’s value as a positive reinforcer may be enhanced by such a practice. We also suggest that positive feedback may be enhanced by a rationale (i.e., a core component of both BST and corrective feedback, Behavior Analyst Certification Board, 2012b) that intimately links trainees’ behavior to their short- and/or long-term goals. For example, “Very impressive performance today with Johnny. Even though it was a stressful situation, you quickly jumped in to help the other staff implement the reactive procedures of his behavior plan. The great thing was that all the other staff saw you keep your cool and implement the intervention as written, which is important as you’re trying to establish yourself as a role model in the classroom. The more you can get in there and model, just like you did today, the more others will start to look to you as a leader.”

Equally as necessary as positive feedback is the delivery of corrective feedback, which requires empathy, detailed review of incorrect and correct performance, modeling, and practice (Behavior Analyst Certification Board, 2012b). When giving corrective feedback, it is often recommended to remain objective (e.g., do not correct when in a bad mood; Daniels & Bailey, 2014), do so privately, and keep in mind the trainee’s context prior to giving feedback (i.e., empathy). As a point of emphasis, supervisors are encouraged to be mindful of “favorites” (i.e., potential side effect of mutual conditioned reinforcers) due to personal characteristics as these biases have the potential to overshadow difficulties of or mistakes made by the trainee.

While provision of corrective feedback for common behavior analytic activities such as graphing or implementing a preference assessment is relatively straightforward, delivering corrective feedback for personal or more subjective performance areas that are negatively impacting a trainee’s work can be significantly more challenging. These areas could be a trainee’s poor social interaction abilities, insensitivity to others, and inflexibility. Self-report data collected from researchers in related fields suggests that supervisors often withhold corrective feedback regarding negative reactions to subjective aspects of a trainee’s performance or give higher ratings in fear of harming the supervisory relationship (Hoffman, Hill, Holmes, & Freitas, 2005; Gonsalvez & Freestone, 2007).

As supervisors, we are the gatekeepers of our field; failing to raise important feedback can have negative implications for clients and the advancement of the profession. When supervisors are faced with having to provide relatively more subjective or difficult feedback, they are encouraged to seek peer consultation and discuss potential approaches and practice delivering corrective feedback prior to meeting with the trainee. We recommend objectively documenting all relevant variables to the delivery of corrective feedback including dates/times, details of events, feedback-specific areas of difficulty, and objective responses and reactions from the trainee.

Lastly, tailoring the parameters in which supervisors provide feedback should be considered as trainees progress through different skill areas. Novice trainees, or more advanced trainees learning a new skill, often require a higher rate of performance feedback. As trainees begin to develop skills in each area, the ultimate goal is to have them accurately and reliably evaluate their own ability to demonstrate each skill. Asking trainees questions such as, “What did you do correctly?” or “What would you do differently next time” may help them reflect and think critically about their performance. One method for aiding this process is to have trainees view audio-visual recordings of themselves performing activities such as leading a parent or staff training and implementing conditions of a functional analysis. Following observation of videos, the supervisor and trainee are able to discuss performance objectively. Another method that may promote this process is to have trainees evaluate their written work (e.g., FBA or BIP) relative to a rubric and compare their results to those of the supervisor.

Evaluation of the Process and Outcomes of Supervision

Behavior analytic supervisors should systematically evaluate the outcomes of their own supervision practices (Guideline 5.07, Behavior Analyst Certification Board, 2014, 2012b) using both objective and subjective measures. Direct and repeated measurement of trainees’ behavior is required to objectively evaluate the outcomes of supervision. Commensurate with the dimensions of ABA (Baer, Wolf, & Risley, 1968), the supervisor should define the skills to be acquired, collect data before, during and after skills training, and use these data to provide feedback to the trainee. For example, using a procedural fidelity form, supervisors can monitor and graph trainees’ performance implementing skill acquisition programs, behavior reduction plans, and behavioral skills training. Direct measures of clearly defined, professional behaviors (e.g., timeliness) can also assist in delivering feedback and evaluating outcomes. Additionally, behavior analysts are encouraged to publish the results in peer-reviewed journals to demonstrate the effectiveness of behavioral approaches within the context of supervision and the relevant skills.

The review of permanent products also serves to objectively evaluate a trainee’s performance. Permanent products are the tangible results of behavior and are commonly used as a data recording method. They can be collected on academic tasks (e.g., completion of mathematics problems), problem behavior (e.g., count of holes in a wall), and social skills (count of phone numbers obtained) to name a few. In the case of supervision, materials developed by the trainee are the most obvious permanent products. Those materials include, for example, written behavior intervention plans, data sheets, skill-acquisition plans, assessment reports, graphs, and relevant goals and objectives in a student’s IEP. Moreover, there are other permanent products that should be reviewed such as notes/letters to parents and email correspondence with teachers/staff. When utilizing a rubric and clear criteria, all of these documents allow a supervisor to objectively evaluate a trainee’s growing competency regarding the quality of written materials that are critical for constituents that behavior analysts serve.

Complimentary to objective measures, social validity measures (Wolf, 1978) can be used to evaluate socially significant changes in the behavior of a trainee’s clients and satisfaction with the performance of the trainee by parents and other professionals. Social validity measures can also be used to assess the trainee’s satisfaction with the goals, procedures, and outcomes of the supervision process. Besides the weekly (or biweekly) trainee rating forms we described earlier, supervisors are encouraged to obtain this type of feedback from their trainees. Although not mandated by the BACB, and while such measures may not be appropriate means to evaluate the effectiveness of the supervision process with regard to skill development and client and staff performance outcomes, they can establish more acceptable and mutually agreed-upon supervision practices and serve as a tool for monitoring the supervisor-trainee relationship. The objective is having the trainee identify the methods that are most acceptable, and allow supervisors to modify their approach during the trainee’s experience (if needed) and enhance the supervision process overall. The supervisor may disagree with the feedback provided by the trainee; however, this activity opens a dialogue about what is and is not working and ideally, ensures that the supervisor and trainee speak openly to each other. Then, a plan can be created to alter the behavior of the trainee and/or the supervisor. To reiterate, the foundation for this type of feedback is grounded in the performance feedback literature and has clear applications for BCBA supervision.

To assist with this type of evaluation, we developed a Supervision Monitoring and Evaluation Form as a project during training at a doctoral internship program in clinical psychology. The internship site had a strong behavioral orientation and employed/trained behavior analysts. During monthly meetings with the Director of Training, the internship class discussed strengths and weaknesses of supervisors, commented about their previous experiences in supervision, and read literature on competency-based supervision from multiple fields. After aggregating all of the information, the group developed the Supervision Monitoring and Evaluation Form (see appendix).

After developing the form, the clinical interns piloted implementation with their supervisors several times throughout the year. This implementation not only gave the trainees (i.e., behavioral psychology interns) the opportunity to “test-drive” the measure, but also allowed them to provide feedback to their supervisors. Supervisors, in fact, were uniformly positive about the process of evaluation and readily embraced the feedback from their trainees. We suspect that this positive outcome was due, in part, to having supervisors participate in developing the Supervision Monitoring and Evaluation Form at the earliest stages and being receptive to having interns objectively evaluate supervision practices. Again, it is the reciprocity between trainees and supervisors that is hypothesized to engender success and mutual satisfaction.

Continuing Education in Supervision

The above sections highlight the many responsibilities and considerations that are present to behavior analytic supervisors and the degree to which supervision of those pursuing a BACB certification can be complex, time consuming, and often requires skills beyond the research and practice skills taught in the formal education of behavior analysts. In recognition of this fact and in addition to maintaining competency in behavior analysis, the BACB now requires behavior analysts to receive ongoing education in supervision (Behavior Analyst Certification Board, 2015), such as effective supervision practices, ethical considerations regarding supervision, and the standards of BACB supervision.

While supervisors can obtain formal continuing education pertaining to supervision through conference presentations, workshops, and online seminars, supervisors should also seek out their own supervision (Behavior Analyst Certification Board, 2012b). The current model highlights the importance of reciprocal feedback and collaboration between the trainee and supervisor, but it is also important to develop this type of relationship among supervisors to encourage mentorship and consultation in an effort to maintain and enhance supervisor competence. In addition to consultation, to the extent possible, it is important to have other competent supervisors observe your supervision activities and provide feedback based on the objective criteria we outlined previously.

Social validity measures may further be used as a self-monitoring tool by providing a means for supervisors to continually evaluate their performance as a supervisor, which provides an excellent model of continual self-assessment for trainees. Placing focus on self-assessment will enable supervisors to be aware of their strengths and weaknesses in supervision practices as well as clinical and research activities. Using this self-assessment approach, competency-based supervision brings accountability to the practice of behavior analysis and sets the stage for quality supervision. A first next step is to conduct an evaluation of the impact of self-monitoring and evaluation on supervisor behavior and the supervisory relationship, and in turn the impact on trainee skill development.

Concluding Remarks: Initiating a Research Agenda in Behavior Analytic Supervision

Competency-based models allow for systematic measurement of both supervisor and trainee behavior throughout the entire supervision process, which makes these approaches appropriate for applied research. As there is a dearth of research in this area, the specific direction of and topics that future research may take are vast; however, there we present a few considerations for research in this area. First, it will be essential to objectively define and task analyze core competencies of trainees and supervisors, especially those falling within the realm of professional and ethical behaviors (e.g., specific skills denoting professionalism, display of empathic behavior), so that we can make objective conclusions about an individual’s competency. It will also be necessary to identify the validity and value of various outcome measures such as social validity, BACB exam pass rate, trainee observable behavior in contrived and naturalistic settings, and client behavior change.

Within this framework, future research should focus on the identification of effective methods for training generalizable and maintainable skills to trainees. While the training of basic procedural skills (e.g., implementing functional analysis conditions) may be a first step, methods for training the complex skills required of behavior analysts are needed (e.g., interviewing skills, problem solving, responding to ethical dilemmas). For example, what are the considerations for implementing BST for these skills? How many exemplars are needed to obtain competency and generalization? What are effective methods of prompting (e.g., video modeling) and teaching for generalization?

Further, there is a tentative hypothesis that the quality of the supervisory relationship as defined by the presence of rapport and mutually agreed upon expectations will impact the supervision experience, notwithstanding the need for more thorough empirical validation. For example, does the supervisory relationship actually lead to quicker acquisition of skills and/or a higher number of skill competencies obtained by the trainee? In order to answer these and many other questions, we must first objectively define the supervisory relationship. While we presented an initial definition of the relationship within this paper, objective indicators of such a relationship must be described, measured, and incorporated into supervision practices. Thus, attempts to measure “good” rapport have focused on the presence of smiling and laughing and the absence of problem behavior (Magito et al., 2005) but other measures appear relevant. Trust, for example, can be observed as someone consistently following through on verbal statements. “Likeability,” another researched construct, translates to a person who has a positive attitude, appears happy, speaks in a pleasant tone, and helps others achieve their goals (Sanders, 2006). Given the focus of the supervisory relationship in all other models of supervision across disciplines, it would be advantageous for behavior analysts to systematically evaluate the impact of the relationship on trainee outcomes and disseminate to the larger supervision community. Taken together, comprehensive considerations of the supervisory guidelines, ethical code and establishing a research agenda in behavior analytic supervision will ideally lead to an effective, ethical, and professional group of rapidly growing practicing behavior analysts.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Compliance with Ethical Standards

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interest

The authors declare that they have no conflict of interest.

Funding

Funding was not obtained for the creation of this manuscript.