The Apprentice Model 2.0: Enhancement of the Apprentice Model

Abstract

There are not enough board certified behavior analysts (BCBAs) with specialization in the treatment of individuals with autism to meet the growing patient demand for applied behavior analysis (ABA) treatment. In addition, there are no streamlined, organization-led fieldwork supervision systems to aid in the professional development of Trainees to meet this need. This article describes enhancements made to the Apprentice Model (Hartley et al., 2016), emphasizing the mutual benefit to the Apprentice, the Supervising BCBA, the organization for which they are both employed, and perhaps most important, the field of behavior analysis. Within the organization where the Apprentice Model has been adopted, 88% of Apprentices who took the exam, since the 2022 modifications were applied, passed on their first attempt despite attending universities with 2021 reported pass rates ranging from 34%–60% (Behavior Analysts Certification Board [BACB], 2023). In addition, through the 6-year history of the adoption of the Apprentice Model, 70% of the organization’s BCBAs who supervise a caseload of patients were prior Apprentices. Structuring a wide-scale supervision model within an organization creates opportunities for the Trainee to experience high-quality tailored fieldwork, it allows the Supervising BCBA to provide supervision under a well-organized and structured framework, and it permits the organization to employ high-quality, newly certified BCBAs to begin meeting patient treatment needs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40617-023-00799-9.

The profession of behavior analysis would benefit from more highly skilled board certified behavior analysts (BCBAs) trained to provide treatment to the growing number of individuals affected with a diagnosis of autism spectrum disorder (ASD). A strategy for producing more highly skilled BCBAs is through the organizational adoption of a mutually beneficial training and supervisory system designed for the benefit of the field at large, the organization for which the Trainee is employed, the Trainee, and their BCBA supervisor.

There has been an overwhelming increase in the need for BCBAs. In fact, a 2021 study conducted by Burning Glass Technologies and published by the Behavior Analyst Certification Board (BACB), found that demand for behavior analysts holding a BCBA or BCBA-D credential has increased by 4,209% since 2010 (BACB, 2021). The increased demand for BCBAs is largely due to a combination of three factors: (1) an increased prevalence of ASD; (2) the growing cost associated with a diagnosis of ASD; and (3) state mandates requiring ABA coverage. In 2021, according to the Centers for Disease Control and Prevention (CDC), 1 in 44 children are identified with ASD in the United States (CDC, 2022), representing a 100% increase in just the last decade. Blaxill (2021) prompted an “urgent focus on prevention strategies” to combat the influx in treatment needed for individuals with ASD, and as a result, the expanding cost of providing treatment for these individuals. In 2020, the cost of caring for individuals with ASD was estimated at $223 billion (about $690 per person in the United States) annually and projected to balloon to $5.54 trillion annually by 2060, likely due to the gradually increasing prevalence year after year. Because of the expanding need for treatment, it is encouraging that in 2019, all 50 of the United States required coverage of applied behavior analysis (ABA; National Conference of State Legislatures, 2021).

Due to the sky-rocketing demand for BCBAs, the profession of behavior analysis must continue producing more BCBAs each year. Although, simply more BCBAs is not sufficient, nor does it meet the needs of this call to action. Rather, the need is two-fold: (1) the development of more BCBAs; and (2) the establishment of a highly qualified skill set within those BCBAs. The development of more practicing BCBAs with treatment implementation expertise and a focus on positive outcomes is a viable solution to the surplus of need for meaningful autism care. The BACB reported in January 2023 that there were 59,976 people certified with the BCBA credential worldwide. This figure is promising because it is an additional 5,753 BCBAs in 2 years from 2021 to the first quarter of 2023; with 2021 showing the largest annual increase in new BCBA certificants since the BACB’s inception in 1998. It is favorable for the field of behavior analysis to see more professionals gain certification each year, thus beginning to better meet the need of the first call to action stated previously (the development of more BCBAs). The second call to action (the establishment of a highly qualified skillset within those BCBAs) is an objective that requires innovative training models such as The Apprentice Model outlined by Hartley et al. (2016).

Hartley et al. (2016) described the Apprentice Model as “a model wherein its Trainees become experts with the autism population by shadowing and purposefully learning the BCBA role in autism-specific programming by applying behavior analytic principles to skill acceleration and behavior deceleration objectives.” The intent of the Apprentice Model is to provide a framework that Supervising BCBAs and ABA organizations can adopt, to systematize their training processes to prepare Trainees for future successful oversight of a caseload of patients when they later become a BCBA. This is accomplished through purposeful mentoring, educating, and training by a BCBA to a Trainee (Apprentice) who is specifically assigned to them, so the two form a team working together with the primary objective of obtaining meaningful patient outcomes. To qualify for the Apprentice position, one must be enrolled in graduate coursework in pursuit of becoming a BCBA (therefore, all Apprentices have a minimum of a bachelor's degree). Both internal and external candidates are considered.

The BCBA/Apprentice pairing becomes a mutually beneficial relationship wherein the Apprentice obtains high-quality supervision and guidance (with BACB required practicum hours typically taking 2 years to accumulate), whereas the BCBA obtains additional highly skilled support for their patients through tasks completed by their Apprentice such as caseload management, staff and caregiver training, and completion of acquisition and deceleration assessments. This model allows the Supervising BCBA to spend more time face-to-face with patients because the Apprentice can complete indirect, nonpatient facing activities necessary for positive patient outcomes, whereas the Supervising BCBA shapes the Apprentice’s case conceptualization, problem-solving, and decision-making repertoires and models technical, professional, and ethical behavior. The intent of the Apprentice Model is to provide high-quality services to patients (a benefit to the patients and their supervising BCBA), in a cost-effective manner through increased BCBA caseload size (a benefit to the organization), while providing maximal learning for the Trainee (a benefit to the Apprentice). This model is a promising approach for ABA organizations because it helps ensure a level of expertise and competency with the autism population in need of treatment, and it allows for a more seamless transition to success and proficiency as a newly certified BCBA.

The purpose of this article is to describe refinements and enhancements that have been made to The Apprentice Model since its original inception in 2016 within the organization where the model was first pioneered (see Table 1). The implemented enhancements have been applied with the precedent that an Apprentice is “one who is learning by practical experience under skilled workers (i.e., BCBAs) a trade, act, or calling (i.e., supervision and oversight of ABA therapy”; Merriam-Webster.com). The enhancements are intended to improve the supervisor’s experience and subsequently increase the Trainee’s preparation for entering the field. It is conceptualized that there are three primary profiles of a supervisor who provides supervision to Trainees: (1) a professor within a university setting who provides supervision to their students; (2) an independent BCBA who offers supervisory oversight to Trainees through either in-person or remote support; and (3) a BCBA employed within an ABA organization who provides supervision to Trainees (who are also employed within the ABA organization). This article is most applicable to supervisors in the third supervision category, although elements of the model are also likely beneficial to the other two categories as well.

Table 1.

Summary of the elements in the Original Apprentice Model (2016) and the Enhancements to the Apprentice Model (2022)

2016 Apprentice Model Elements	2022 Apprentice Model Enhancements
• Development of an Apprentice position • Formation of BCBA/Apprentice pairings • Establishment of strategies to provide Trainees with experience in caseload management, staff and caregiver training, and acquisition and behavior deceleration assessments • Increase of face time by a BCBA with patients on their caseload	• Apprentice Model Curriculum • Group Meetings • Apprentice Performance Scorecard • Bidirectional Performance Appraisal

Enhancements to the 2016 Apprentice Model

The enhancements to the Apprentice Model were prompted by the understanding that a true apprentice model requires a lot of attention from the “Master” (Supervising BCBA; Collins et al., 1991), and therefore, the success of a Supervising BCBA is hinged on the ability to access, and use streamlined supervisory resources while simultaneously individualizing supervision practices to provide the time and attention needed by their Apprentice. In addition, the growing demand for qualified and highly skilled BCBAs has necessitated enhancements to the Apprentice Model to continually improve a system that can generate skilled BCBAs. The following section will provide a description of each enhancement with a review of the supporting literature, followed by a description of how a BCBA supervisor could incorporate the enhancement into their supervisory practice, and finally a discussion of how the enhancement was incorporated into the supervision system within the organization. Within the sections “Supporting Literature” and “Implementation by Supervisors within an Organization,” the term “Trainee” will be used to fit within the broad scope for the way in which the BACB refers to people pursuing board certification as a behavior analyst. Then, in the sections titled “The Apprentice Model Implementation,” the term “Apprentice” will be used rather than “Trainee” because “Apprentice” is the title used in practice with the implementation of The Apprentice Model. However, the terms “Trainee” and “Apprentice” are interchangeable, and both describe a person who is actively working towards meeting the criteria to become a board certified behavior analyst.

Apprentice Curriculum

Supporting Literature

Supervisors are tasked with identifying opportunities for their Trainees to practice and develop a wide range of skills. The BACB publishes a task list which includes the knowledge and skills that a Trainee must demonstrate to pass the BCBA examination. This task list serves as a foundation for designing the supervisory experience. However, the goal of supervision should not be the development of minimal competence to pass an exam, but rather a fluent understanding of the principles of behavior analysis and the procedures within the applied domain to permit conceptually sound practice. To this end, Sellers et al. (2016a) provided recommendations to supervisors creating their supervision structure. They recommended the development of objectives and competencies based on the current BACB task list and ongoing discussion about ethical dilemmas supported by the current ethics code. They further highlighted that the competencies should contain both knowledge and performance-based evaluations and should be shared with the Trainee at the outset of the experience. The supervisor ought to develop evaluation criteria and objectively evaluate their Trainee’s performance. If the Trainee does not meet the mastery criteria, additional support can be provided. Helvey (2021) also stated that supervisors need to establish and maintain a plan for structured supervision content and competence evaluation.

Practicing behavior analysts have many responsibilities that affect their ability to independently create an enriched supervisory experience. Behavior analysts who responded to a survey about supervisory practices reported that they engaged in practices more frequently when they had access to concrete guidelines and support (Hajiaghamohseni et al., 2021). To this end, several curricula have been developed to support the supervision experience. Kazemi et al. (2019) orient their guidance towards the Trainee who is receiving supervision and provide numerous competency evaluation checklists that can be used by the supervisor to assess competency. Britton and Cicoria (2019) guide the supervisor through material they can prepare for their Trainees (e.g., a list of recommended videos to develop) and provide scoring sheets to evaluate the implementation of various skills. LeBlanc et al. (2020) provide activities, agendas, and questions aimed at establishing a compassionate supervisory experience that extends into mentorship. Although impressive in both depth and breadth, a BCBA employed within an ABA organization who provides supervision to Trainees may have difficulty incorporating these resources into their supervision. These supervisors would likely benefit from a curriculum developed from the available activities at their site so they can teach in the context of service delivery.

Applied behavior analysis is continually advancing based on changes in the practice domain (e.g., consumer values), theoretical developments, and workplace demands. This necessitates a flexible curriculum that is responsive to the job market. An option is to create the curriculum from competencies identified through analyzed job skills (Joshi et al., 2021). A job analysis is a systematic empirical procedure for collecting and analyzing information about a job, usually using literature reviews, interviews, and surveys of experts’ opinions (Brannick & Levine, 2002). This process is adopted outside of behavior analysis as evidenced by the work of Gazit et al. (2022) who used job task analysis to refine the training provided to surgeons. This process identified the most important competencies for success as a surgeon and then designed training around those activities.

The BACB offers several resources that can be used to conduct a job analysis. For example, the BCBA Handbook (BACB, 2022a, 2022b) provides a list of acceptable activities that should be the primary focus of the fieldwork experience. This list can help a supervisor identify what they do as a regular part of their job that is considered an appropriate activity for fieldwork hours. The 5^th edition Task List (BACB, 2017) was developed by Subject Matter Experts specifically tasked with identifying the minimal competencies one must demonstrate to enter the field. The BACB’s Ethics Code for Behavior Analysts (BACB, 2020) provides an overview of the core principles certificants must adhere to and codes that guide ethical practice across five different domains. These resources together create an excellent foundation for developing a job analysis for a behavior analyst in general, but they are not specific to practice within any specified domain. Supervisors overseeing fieldwork for a Trainee within the same organization will need to identify the primary job duties that are specific to their practice, then identify how to incorporate the BACB resources into these activities. Applying the Task List and Ethics Code in the context of clinical practice moves supervision beyond simply identifying what a principle or procedure is, to determining when one is clinically indicated, demonstrating correct implementation, training others, and determining the effectiveness of the intervention through data analysis. It is not enough for Trainees to be able to define a term or state what is contained within an ethics code. They need to be able to ethically apply the principles of behavior analysis in the context of service delivery.

Implementation by Supervisors within an Organization

Although it may seem daunting for an individual supervisor to create a curriculum to capture the learning opportunities present in their work, the activities in which practicing BCBAs engage frequently encompass many of the items on the task list and can encourage discussion around the Ethics Code. To start, a behavior analyst considering becoming a supervisor should conduct a job analysis of their current position. It is likely that a quick review of daily expectations will reveal a variety of activities that fall within the BACB’s description of an appropriate activity. Next, the BCBA should identify repeated activities they frequently engage in, such as developing treatment plans, training front-line staff, analyzing data, writing programs, and conducting parent support sessions. By developing competencies around these repeated tasks, the BCBA is maximizing practice opportunities the Trainee can engage in without having to carve out time away from caseload management. Once the BCBA has a list of commonly repeated activities, they should identify the task list items they can incorporate into each activity. For example, the task lists within the Personnel Supervision and Management section can be targeted when training staff, therefore, the supervisor can set performance expectations and competency criteria for each item. This will allow the BCBA Supervisor to pinpoint errors made by the Trainee during staff training and provide behavior-specific feedback because the BCBA Supervisor knows exactly what they are looking for within this interaction. Building the experience from the BCBA’s job duties is likely to increase the value of the supervisory experience for the supervisor because supervision now enhances their work. With performance improvement, the Trainee becomes a well-trained colleague who contributes to caseload management thereby reducing the response effort for the supervisor. In this way, the supervisory relationship is mutually beneficial.

The Apprentice Model Implementation

An Apprentice curriculum was developed to increase consistency in training opportunities and standardize how competency was evaluated across supervisors. The curriculum was developed from an experience checklist modeled on the “job analysis” work of Crowell et al. (2011) wherein a particular job is analyzed to understand the specific tasks and responsibilities involved to perform successfully. The experience checklist includes expectations of BCBAs within the organization, therefore, completion of the checklist by Apprentices should lead to a successful transition into a BCBA position. The Apprentice curriculum was then built from this experience checklist. As recommended by Sellers et al. (2016c) the curriculum assesses both knowledge and performance of skills from the experience checklist. Groupings of relevant BACB task list items are interspersed throughout 42 competencies sequenced based on complexity (see Appendix 1). Each competency includes (1) a knowledge assessment related to the 5^th edition Task List; (2) a discussion question related to the Ethics Code for Behavior Analysts; (3) a performance assessment; (4) relevant terms and definitions; and (5) recommended readings. Also, the 5th edition Task List items and ethics codes covered within the competency are specified to ensure that the Apprentice is receiving both appropriate depth and breadth of the necessary content (see Appendix 2).

The Apprentices within the organization attend eight different universities, all of which require Applied Behavior Analysis, 3^rd edition, by Cooper et al. (2020). Therefore, this text was used as the primary resource for the definitions and knowledge questions across the competencies. Additional articles and textbooks were used to supplement this text as necessary. For example, an article by Wong et al. (2021) was incorporated into the competency on identifying mastery criteria for a written program. All Apprentices are also university students, and as such, they have access to their institution’s online libraries. Searching and reviewing literature are critical professional skills that must be established prior to independent practice.

Each week, the Supervising BCBA and their Apprentice meet for approximately 1 hr so the BCBA Supervisor can evaluate their Apprentice’s mastery of a competency within the curriculum. The apprentice must submit a written knowledge assessment and discussion answers to their supervisor before scheduling this evaluation. For optimal success of the BCBA Supervisor during these meetings, the supervisor is provided with a teacher edition of the curriculum with answer keys and sample graphs to ensure standardization across all BCBA/Apprentice pairings. Supervisors are likely to reflect both the positive and negative behaviors modeled for them by their own supervisors (Sellers et al., 2016a, 2016b); therefore, it was designed for the apprentice curriculum to include standardized answers and ethical scenarios to ensure all BCBA Supervisors within the organization base their evaluation on the current standards. The Apprentice cannot use notes during the evaluation as competency is evaluated on oral responses and independent work. The goal of the curriculum is to prepare Apprentices for a successful transition into the BCBA role. This requires vocal communication with patients, staff, caregivers, and other colleagues; therefore, it is critical for them to have the opportunity to vocally respond to challenging questions.

The direct assessment portion of the curriculum describes how performance will be scored and includes scoring rubrics for the supervisor. The Apprentice is expected to arrange the conditions to be assessed during the direct portion of the evaluation. This includes identifying relevant patients, staff, parents, or programs that necessitate the identified skill. By placing the responsibility for progression through the curriculum on the Apprentice, professional skills of time-management, organization, and meeting deadlines are established.

The numerous responsibilities placed on BCBAs may unintentionally lead to supervision practices that lack breadth and depth of training or lead to hours spent outside of work preparing for supervision. A standardized supervision curriculum addresses both concerns while also maintaining ambitious standards for the supervision provided throughout the organization. The supervisor’s experience is a critical variable to their apprentice’s experience. A newer supervisor may be fluent with terminology but may struggle with synthesizing the principles and technologies of behavior analysis in case management. A more experienced supervisor may excel in caseload management and collaboration but find it challenging to assess experimental designs or specific technologies (e.g., implementing functional analyses) because they do not use them frequently. However, both new and seasoned supervisors must prepare their Apprentice adequately. The availability of a curriculum is a viable solution to aid in these challenges.

Group Meetings

Supporting Literature

Supervisors are responsible for their Trainee’s full scope of practice (BACB, 2020). Behavior analysts’ scope of practice extends beyond the literature and requires cusp skills necessary for ongoing learning. Valentino et al. (2016) identified that well-designed group supervision may be more beneficial than individual supervision when developing observational learning, empathy, and public speaking and presenting skills. These skills are more easily contrived and shaped within a group setting. For example, consider the full range of professional behaviors that are evoked from the presenter and audience during a case review. Both parties are responsible for case conceptualization, technical precision, and active participation. The presenter can practice public speaking skills, including pacing of content, appropriate voice volume and intonation, and soliciting and managing ongoing discussion. The audience can learn how to address similar clinical cases and practice providing feedback. These ongoing interactions allow Trainees to experience the effect their behavior has on others and may help to develop empathy (i.e., contextually appropriate responses to a display of affect; Schrandt et al., 2009). Valentino et al. (2016) compares group supervision to graduate school training in that a cohort of individuals are brought together and may come to support each other when the fieldwork experience ends. This professional network leads to shared resources and opportunities and serves as an ongoing source of mentorship and support.

The supervisor also benefits from providing supervision to a group of Trainees. Many lessons and discussions are beneficial for several Trainees. The supervisor not only saves precious time by presenting content once, but they also receive feedback from different listeners allowing them to edit their teaching to increase clarity. In addition, discussion is likely to occur thereby permitting the supervisor to model appropriate and positive feedback and reinforce appropriate and professional peer feedback skills.

Valentino et al. (2016) provide resources, specific strategies, and an overall structure for the successful implementation of group supervision. This structure may prove useful for a supervising BCBA planning to begin providing group supervision or striving to revamp an existing group supervision experience. Despite this clear guidance, in a survey of 317 supervisors in 2021, Hajiaghamohseni et al. (2021) found that group supervision was infrequently used. This may be due to the upfront cost of time and the development of materials in addition to a lack of resources. For example, supervisors within a service organization are likely to oversee a caseload of patients, therefore the focus of the supervision they provide is pulled from their patients’ clinical needs. Although this provides depth of experience for their Trainee, it might not provide sufficient breadth. Creating training or discussion points, or even a simple agenda, is outside the scope of patient care; however, LeBlanc and Nosik (2019) made recommendations on establishing structured meeting that could be helpful when organizing group supervision. It may require someone external to the individual supervisor to structure and lead group supervision. Dubuque and Dubuque (2018) described a practical training system developed within a university with external fieldwork training sites. The sites provided individual supervision whereas group supervision was directed by university faculty. By dividing these responsibilities, the Trainee benefited from the depth of training from individual supervision and the breadth of experience from group supervision. In addition, this created manageable workloads for both supervisors.

Implementation by Supervisors within an Organization

Group supervision can account for no more than half of the supervision time within a supervisory period and is limited to 10 or fewer Trainees (BACB, 2022a, 2022b). The size of the organization affects the structure of group supervision. However, Trainees with similar experiences should be split into small groups and led by the same person repeatedly. The small group format allows the supervisor the opportunity to build rapport with the Trainees, establish clear expectations, and build a culture within the group. Membership within a group could be determined by service setting (e.g., group of Trainees working with similarly aged patients), course completion (e.g., group consists of Trainees enrolled in similar courses), or geographic location (e.g., all Trainees from the same clinic). The group supervisor should have extensive training and experience with supervision and should have open time to devote to supervision. The group should meet at a consistent time and follow a consistent schedule to promote attendance and participation. The supervisor should develop the agenda and open and close meetings on time. The agenda should be structured to promote active participation with guidance and feedback provided by the supervisor. For example, if the supervisor has an idea they would like to share, they should state it in a way that promotes discussion (i.e., “Who has experience using demand fading that might help move this conversation forward?”) as opposed to a directive (i.e., “You should use demand fading”). The supervisor should consider that they are always modeling professionalism during group supervision and behave accordingly. This includes being punctual, staying actively engaged (e.g., nodding, asking questions), responding appropriately to feedback, and staying calm under pressure. For example, if technology does not work, the supervisor should use this as an opportunity to model problem solving, asking for help, and engaging in teamwork to resolve the problem. If they are asked something they do not know, they can model saying, “I don’t know. Let me look into that and get back to you.”

The training a Trainee receives varies across universities and instructors and is not always timed according to the clinical demands of the caseload. Group meetings, however, align the BACB’s 5^th edition Task List with content relevant to the population served by the organization. With increased attendance to distance learning through remote online graduate programs, some Trainees may not have access to the same collegial environments for establishing professional relationships compared to on-campus programs. Holding group meetings within an organization supports the Trainees’ clinical development and increases their professional network.

The Apprentice Model Implementation

Each Apprentice is assigned to a single BCBA supervisor responsible for ongoing individual supervision. The group leader, also a BCBA, is in a leadership position within the organization that does not oversee a caseload of patients, thus permitting the bandwidth to develop content and provide ongoing support to numerous trainees. All the Apprentices across the organization attend group meetings, thus this time is counted towards unrestricted experience and not group supervision as the number of Trainees exceeds what is permitted by the BACB (BACB, 2022a, 2022b). The group meets virtually on the Microsoft Teams platform every week for 30 min outside of operating clinical hours. This permits most Apprentices to attend and stay fully focused on the content. The overarching goal of these meetings is to bridge the gap from university study, which the Apprentices receive in their coursework, to clinical practice by discussing ABA principles and procedures specifically in the context of ASD treatment. There are three types of meeting agendas that rotate weekly.

Didactic Training

The first meeting agenda involves a presentation delivered by the group leader that reviews the content to support a specific competency. These presentations occur every other week, are 20–30 min in duration, and are recorded. This recording is beneficial for Apprentices unable to attend, but the primary benefit of the recording is to automate this portion of the group meetings in the future and eliminate this type of meeting from the agenda. Once content is recorded, it can be used to develop a personalized system of instruction whereby Apprentices can access the content prior to preparing their written competency for individual supervision. This will allow them to progress through the curriculum as it pertains to their caseload (i.e., they can complete the parent training competency prior to conducting a parent meeting). This will permit group time to be fully dedicated to more interactive activities, such as case reviews.

Case Reviews

To maximize participation within the dedicated 30-min window, the case presentations are structured as “rose” and “thorn” graph presentations and are scheduled every other week. The person called on to present a rose graph has about 5 min to share any graph depicting progress. Next, a rose discussant provides feedback aimed at praising and extending the application of the intervention. For example, they may describe research they have read on similar interventions or discuss how they applied a similar intervention within their practice. The audience (other Apprentices) benefits from this presentation because they can learn how a particular skill was addressed and consider similar interventions for their own patients. It is common to have Apprentices share resources outside of group based on what was presented within the discussions. The rose graph is always presented first and tends to evoke active participation that builds momentum towards the more difficult thorn presentation. The thorn graph depicts lack of patient progress. The presenter has about 10 min to share their graph. Then, the thorn discussant provides critiques with rationale or asks questions intended to move the conversation forward. For example, rather than asking, “Can your patient dress themselves?” they would be encouraged to ask, “I am curious if you have assessed your patient’s ability to complete the dressing chain? To isolate the focus of the intervention on manding for the next piece of clothing, it would be important to know that the chain is mastered so we can assume the next item reinforces completion of the previous step.” Although the questions are similar, the latter provides context that prompts the presenter to look into the fluency of dressing and consideration around the controlling variables of a mand. The first question can be answered with a yes/no and does not provide any insight as to why the question is being asked; and therefore, does not move the presenter forward in their conceptualization. The thorn discussant may also solicit feedback and guidance from the other apprentices. Assigning a discussant for each presentation increases the participation from the apprentices and reduces the group leader’s participation to prompting and shaping Apprentice behavior. After the group meeting, the thorn presenter must email their supervisor and copy the group leader with a summary of the group’s guidance. By being copied on the email, the group leader can observe whether the Apprentice understood the recommendations provided and can help the BCBA/Apprentice team address the clinical issue identified. This assistance may come in the form of literature searches, client observations, or data analysis. In addition, this email ensures that the individual supervisor benefits from their Apprentice’s participation in the group meeting.

Thorn Reviews

To create accountability for implementing the feedback obtained within the case review, the last three thorn presenters represent the same graph during a thorn review meeting. This meeting is scheduled every other month, or after eight group meetings. During a thorn review, the presenter discusses what modifications they have made, the rationale for these modifications, and provides an analysis of the effect the modification had on the patient’s behavior. The Apprentices discuss how they communicated with their supervisor to advocate for the modification and any barriers that prevented successful implementing of the recommendations (e.g., absences, other clinical priorities). Thorn reviews allow the Apprentices to evaluate the effects of their recommendations and also engage in observational learning of their peers as described by LeBlanc et al. (2020).

Using a structured and repeating agenda for the weekly group meeting promotes efficient use of time for all participants. The group leader can prepare didactic training in advance because the curriculum is prearranged. The Apprentices know what is expected of them if they are assigned to present or discuss a rose or a thorn graph. Thorn presenters know that they must attend to the feedback provided because they will be required to send a summary email to their supervisor and copy the group leader. In addition, they will be required to show the same graph within the next 2 months, so timely decision making is required. If Apprentices provide their peers with recommendations that are not evidence-based or otherwise not sound, then the group leader intervenes to provide feedback and uses this as an additional learning opportunity.

Apprentice Performance Scorecard

Supporting Literature

A performance scorecard is a graphical representation of an individual employee’s progress made over time as it relates to the key features of success in their role and how it integrates with the overall mission of the organization. Minimum and maximum expectations are set for each component of an employee’s performance benchmarks (Abernathy, 2014; Bucklin et al., 2022). Support for the establishment of performance scorecard systems has been evidenced through behavior analytic literature over the years, with findings indicating benefit in fostering a spirit of collaboration, providing objective data on trends in employee performance, and evoking evaluation of ongoing strategies for performance improvement (Abernathy, 1996, 2014). In addition, performance scorecards have been shown to be effective outside the field of behavior analysis, specifically with medical doctors, in improving quality of services and patient outcomes (Maggard-Gibbons, 2014; Rankin et al., 2020). Because high-quality services and optimal patient outcomes are the primary objectives of ABA service provision, it is paramount that BCBAs providing supervision to Trainees consider the implementation of performance scorecard systems into their supervisory practice.

Recommendations for Use of the Apprentice Model Implementation by Supervisors within an Organization

Collaborative and objective performance monitoring systems are typically sought after as it relates to a supervisory relationship, therefore, striving for the adoption of a performance scorecard system for Trainees is recommended as part of best practice supervision. The components of a performance scorecard system are beneficial within the scope of a supervisory relationship because it creates alignment and clarity of expectations, that in turn, allow the Trainee to best understand what is expected of them and then work towards achievement of those expectations. The more clarity and objectivity a supervisor has when providing feedback to a Trainee, the more apt the Trainee is to fully understand the behavioral variables that require modification to achieve the desired accomplishments.

The Apprentice Model Implementation

An Apprentice performance scorecard was developed and designed to capture the key performance indicators (KPIs) of the Apprentice position, with monthly performance review. Key Performance Indicators “represent a set of measures focusing on those aspects of organizational performance that are the most critical for the current and future success of the organization” (Parmenter, 2015, p. 26). Because each person within the organization helps to accomplish the mission, it is crucial that every employee is aware of their KPIs and the corresponding applicability to their success, their patient’s success, and the success of the organization. The performance scorecard provides monthly graphic feedback to the Apprentice on how well they are meeting performance goals aligned with the organization’s values and strategic objectives. Monthly feedback allows for frequent collaboration between the BCBA Supervisor and the Apprentice on making timely modifications to enhance the performance of the Apprentice. The contingencies outlined through the Apprentice’s scorecard ensure that the Apprentice and their Supervising BCBA remain focused on the same goals throughout the month. A physical print-out of the Apprentices’ scorecard is given to their supervisor to occasion relevant feedback. These scorecards include a monthly progress graph for each KPI and a graph depicting past data for the supervisor to provide further analysis of level, trend, and variability.

There are several considerable benefits of a performance scorecard system for the Apprentice position, which include data-driven monthly performance feedback delivered directly by the Apprentice’s supervisor (the Supervising BCBA), an opportunity for a monthly monetary incentive based on quality performance, and exposure to setting realistic goals and establishing an effective time management repertoire which are critical skills that will be utilized later as a BCBA.

The performance indicators identified for the Apprentice position are the quantity of oversight provided to their assigned patients by their Supervising BCBA, the quality of their patient treatment plan writing, the number of project milestones completed (as assigned by their Supervising BCBA), the number of competencies completed, patient progress, the patient’s program quality, and the provision of family treatment guidance (see Table 2). As an example, additional detail on the implementation of the project milestones KPI includes the following process: project milestones are set collaboratively between the BCBA supervisor and the Apprentice prior to the start of the month and include projects the Apprentice is expected to complete during the upcoming month. These projects vary, but may include staff training and oversight, behavior assessments, and writing treatment plans. If the Apprentice did not meet the minimum expectation of 90% of project milestones met within a month, the BCBA may suggest time management strategies or assist the Apprentice in prioritizing tasks and describe the process by which they establish these priorities (Sellers et al., 2016a, 2016b). On the other hand, if the KPI of patient progress fell below the minimum expectation of 75% of identified treatment objectives met within the month, the discussion between the BCBA and the Apprentice would center around data analysis and modifications to the patient’s programming. In both scenarios, the Apprentice would learn to use data to analyze their effectiveness and make data-based clinical decisions. All performance indicators meet the criteria of meaningful fieldwork experience and provide Apprentices with exposure to the responsibilities they will encounter in the future as a BCBA.

Table 2.

Key performance indicators of the apprentice position

Key Performance Indicator	Description
BCBA oversight to patients	Quantity of BCBA oversight to patients on a shared caseload
Treatment plan quality	Quality of treatment plan writing based on a score generated from an objective scoring rubric
Project milestone completion	Percentage of milestones completed per agreement with the Supervising BCBA at the beginning of the month
Apprentice curriculum competency completion	Number of Apprentice curriculum competencies mastered
Patient progress	Timeliness of clinical decisions made
Patient program quality	Inclusion of identified variables for audited patient skill acquisition programs
Family treatment guidance	Quantity of family treatment guidance scheduled

Bidirectional Performance Appraisal

Supporting Literature

The assessment of work performance via bidirectional performance appraisal has been researched over hundreds of years. It is a formal process between a supervisor and their Trainee by which employees are evaluated along a given set of dimensions, then provided feedback based on the evaluation, in order to influence or improve individual performance (DeNisi & Murphy, 2017). The appraisal process allows both the supervisor and the Trainee to obtain information that better equips them to establish individual development objectives that can be continually reviewed and monitored through the supervisory process. Support for integration of bidirectional appraisal processes has been evidenced through behavior analytic literature in recent years. At the core of a good appraisal process is the delivery of, and the receipt of, feedback. Sellers et al. (2016b) described the feedback process, indicating the importance of identifying areas that meet expectations and areas that do not meet expectations. Through the process of reviewing performance, and subsequently delivering feedback, the requirements for success of the Trainee becomes more apparent for the supervisor as noted by Ninci et al. (2021)via the use of a “Trainee Needs Assessment.” When Trainee needs are formally assessed (via an appraisal process or a tool such as the “Trainee Needs Assessment”), the supervisor becomes more knowledgeable regarding modifications that can be made to fieldwork activities. This is further supported by Garza et al. (2018) and Komaki et al. (1986) who pointed out that supervisors who systematically monitor performance via formal and informal assessments, collect data, update Trainee goals, and deliver performance feedback are likely to be knowledgeable about their Trainee’s performance.

In 2019, Sellers et al. disseminated a survey to get feedback from BCBAs providing supervision in the hopes of assembling information about current supervisory practices and barriers. As a result of responses from 284 BCBA participants, the top areas of success and need were presented. One of the areas for improvement indicated a need for setting clear expectations for receiving feedback, evaluating the supervisory relationship, assessing and teaching skills in professionalism, and recruiting feedback about one’s own supervisory practices. These survey results solidified content found in the literature indicating the importance of formal bidirectional processes to assess supervisor and Trainee performance. It is now apparent that Supervising BCBAs in practical settings who supervise the next generation of BCBAs have agreed on the significance of performance appraisal systems.

Recommendations for Use of the Apprentice Model Implementation by Supervisors within an Organization

Due to the firm literature base highlighting the importance of formalized performance appraisal systems between a BCBA and their Trainee, the question for BCBAs then becomes, “What is the process for developing a system like this?” A Supervising BCBA is advised to first consider the priority areas they would like feedback on from their Trainee. Then, because this is a bidirectional process, the BCBA is encouraged to ask their Trainee what they would like feedback on from them (as their supervisor). Based on this information, the organizational needs, the patient population, and the goals of the supervisory relationship, questionnaires can be developed that highlight specific behaviors the Supervising BCBA and Trainee can engage in to have a strong relationship and meet supervisory goals.

The Apprentice Model Implementation

The organization uses a bidirectional appraisal process, between the Supervising BCBA and their Apprentice, for monthly feedback delivery. The process allows the BCBA Supervisor to provide feedback to their Apprentice, and for the Apprentice to provide feedback to their supervisor. The BCBA Supervisor evaluates their Apprentice’s performance across three areas: (1) administrative skills; (2) clinical skills; and (3) supervision/feedback (see Appendix 3). The Apprentice evaluates their supervisor’s performance with respect to the following four areas: (1) clinical skills; (2) supervision/feedback; (3) professionalism; and (4) administrative duties (see Appendix 4). The BCBA and Apprentice independently complete the questions in each section of the appraisal. Once the appraisals are complete, they come together live to share their responses with each other with the supervisor always providing feedback to the Apprentice first. The feedback obtained through this appraisal generates a score that is then captured on both the BCBA and Apprentice’s monthly performance scorecard. In this way, the performance appraisal can be thought of as an output (rather than an input) of the performance management process which does not begin or end with assessment, but rather represents a process of continuous performance data collection, facilitating developmental efforts throughout the year (Gravina & Siers, 2011). To optimize the output generated from the monthly appraisal process, appraisal data are aggregated across BCBAs and Apprentices to determine areas that require additional training, supports, or performance management interventions (i.e., job aides, task rearrangement, goal adjustment). As a result of those findings, additional organization-led training is provided to both BCBAs and Apprentices each month.

Discussion

The intent of the Apprentice Model is to establish mutual benefit for the Trainee (Apprentice), Supervising BCBA, and the organization for which they are both employed. Four enhancements to the original 2016 Apprentice Model publication were described (Apprentice curriculum, group meetings, Apprentice performance scorecard, and bidirectional performance appraisal). The enhancements include a summary of the literature that support their effectiveness in the context of a supervision model, recommendations regarding how a practicing BCBA can use each enhancement in their own supervisory practices, and details outlining the ways the enhancements were implemented within the organization that originally established the Apprentice Model. This model is best suited for a supervisory relationship in which the supervisor and the Trainee are employed by the same organization. A summary of how all three parties benefit from the four Apprentice Model enhancements is described below.

Benefit to the Apprentice

The Apprentice obtains advantages from the Apprentice Model in many ways. The curriculum ensures that they demonstrate competence through both knowledge and performance-based evaluations across a variety of common activities related to the specific practice domain in which they intend to focus. Via group meetings, the Apprentice can learn how behavioral concepts and principles are applied within the context of ASD service delivery and practice presenting data and providing feedback. Through a performance scorecard system, the Apprentice learns to use data to analyze their effectiveness and make data-based clinical decisions. The bidirectional appraisal process allows the Apprentice to generate experience providing feedback to one’s supervisor, which can be a difficult and intimidating experience. The enhancements allow the Apprentice to contact the contingencies that establish the professional skills of preparedness, fluency with behavior analytic content, and mastery of skills needed by a practicing BCBA.

Benefit to the BCBA Supervisor

A BCBA Supervisor benefits from each Apprentice Model enhancement as well. The curriculum structures the supervision content and connects the BACB resources to activities common within the practice setting. The answer key and scoring rubrics reduce the time required for the Supervisor to prepare for the Apprentice’s evaluation, thereby permitting more time for detailed experience around the unique needs of the caseload they are both managing. Group meetings help distribute the weight of supervision across another supervisor and several apprentices. These meetings eliminate the individual supervisor’s need to teach the knowledge portion of the competency so they can more appropriately focus on their Apprentice’s performance. The individual supervisor may also benefit from the recommendations provided on a program presented within a case review. The Apprentice performance scorecard system is beneficial to supervisors because it establishes a monthly cadence for frequent performance feedback delivery, representing best practice (Sleiman et al., 2020). Lastly, the bidirectional performance appraisal creates opportunities for critical feedback to be provided to both the supervisor and Apprentice, which can be challenging and uncomfortable for many people. The bidirectional appraisal process provides supervisors with frequent practice of feedback delivery and acceptance skills, with the intent of becoming more fluent and comfortable over time.

Benefit to the Organization

The organization for which the Apprentice and Supervising BCBA are employed, should also benefit from the Apprentice Model. Most ABA organizations’ revenue is generated primarily through direct patient service provision. Within this context, BCBAs are a provider that must spend much of their time with patients to ensure dynamic treatment while also maintaining the financial health of the organization through reimbursement for services rendered. Because of this, a BCBA’s time must be guarded and primarily focused on patient care. By utilizing a standardized curriculum, BCBAs are saved the time that would have otherwise been needed to create supervision content independently. Instead, BCBAs can provide quality supervision without compromising their primary role of patient-focused care, which allows for optimal BCBA utilization and financial stability for the organization. Group meetings allow the organization to allocate training resources to the Apprentices without sacrificing face-to-face patient care. As a result, group meetings are an efficient way to introduce relevant training content to Apprentices who will then apply that content within clinical practices that benefit the organization’s patients. The performance scorecard equips the organization with a monthly synopsis of how well each position is executing their KPIs, which has an impact on the organization’s ability to achieve its mission. Finally, the bidirectional performance appraisal system is beneficial to the organization because allocating resources to develop supervisory skills allows the organization to meet its clinical mission of developing exceptionally trained BCBAs who have the skill set to make a profound impact on best outcomes for patients, and meaningful life changes for their families.

Limitations

Despite the numerous benefits of the Apprentice Model, as previously described, there are also some limitations to consider. Given this supervision model is specific to a particular organization and was implemented with the intention of Apprentices remaining with the organization when they become certified as a BCBA, there may be limits in how well-prepared the Apprentice will be if they make a transition to a different organization upon becoming certified. Although the Apprentice curriculum was built from the BACB’s resources (BACB 2017, 2020), there was an intentional emphasis on the skills necessary for success based on the job requirements of a BCBA in the organization. Although the role of a BCBA who supervises a caseload of patients with autism is likely to look similar from organization to organization, there are certainly agency differences that were not factored into the curriculum development. An additional limitation is that because 70% of the organization’s BCBAs were trained through the Apprentice Model in the past 6 years (largely due to an increased number of total BCBAs within the organization), there may be a lack of experience among the Supervising BCBAs who are now tasked with providing oversight to Apprentices. However, this is a wide-spread challenge faced by the field of behavior analysis rather than something unique to this model. The curriculum guides the supervisors within the organization to provide a consistent quality of supervision across the Apprentices they oversee. Although this preserves the response effort necessary to prepare for supervision, it could minimize some desirable variability related to the Trainee’s individual needs. However, supervisors are encouraged to systematize core competencies to ensure progression through the vast content required for entry into the profession. This evaluation leads to the identification of targeted deficits where the supervisor can then individualize their guidance. In this way, supervisors can address the breadth of content required to enter the profession while providing individualized depth when indicated. The Apprentice Model has been systematically implemented in a single organization; therefore, further research and examination of implementation in other organizations of various sizes, with different patient populations, would be beneficial to assess generalization. Lastly, there are limitations in only using Apprentice’s BCBA exam pass rate as a primary indicator of the Model’s success. A significant contribution to the supervision literature would be other data supporting the use of similar models. Future researchers may wish to collect data on different outcomes (other than BCBA-exam pass rate) to determine applicability and success.

Conclusion

Since the writing of the 2016 article introducing The Apprentice Model, additional enhancements have been made to further develop this supervision model as a resource for BCBAs to utilize with the intent of increased efficiency and effectiveness for the purpose of providing high-quality oversight. This resource-heavy supervision model serves as an aid for Supervising BCBAs who then have more time to focus on supervisory practices rather than on finding and developing resources for those they supervise. In the absence of this kind of supervision model, a Supervising BCBA may not have the bandwidth for such high-level oversight and training due to the competing contingencies and responsibilities of overseeing the treatment delivery of a caseload of patients.

The development of standardized supervision practices that can be replicated and disseminated across organizations is crucial for sustaining a high expectation of exceptional quality for practicing behavior analysts. The four enhancements highlighted within this manuscript included: (1) a supervision curriculum; (2) group meetings; (3) an apprentice performance scorecard; and (4) a bidirectional appraisal process. Each of these four areas was built with the objective of mutual benefit for all parties involved.

Systems like the Apprentice Model are becoming crucial to the success of the field of behavior analysis to ensure a future BCBA’s training is conducted with integrity and with the Trainee’s future responsibilities in mind. The Apprentice Model’s structure allows BCBAs to function as “skilled workers” who foster development of the next generation without sacrificing the integrity of their responsibilities to their patients. BCBAs supervising Trainees are urged to consider adoption of the outlined supervisory strategies to contribute to the favorable clinical development for future BCBAs and meaningful outcomes of the patients they serve.

Data Availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Declarations

Conflicts with Interest

We have no conflict of interest to disclose.