A Survey of Staff Training and Performance Management Practices: An Update

Abstract

The field of behavior analysis has experienced marked growth in the number of credentialed professionals over the last decade. This growth may have implications for the quality of staff training, performance management, and supervision practices provided in human service settings. The purpose of this survey was to extend DiGennaro Reed and Henley (2015) by surveying credentialed and aspiring behavior analytic professionals on the staff training, performance management, and supervision practices available at their current place of employment. Three main differences were observed in relation to the findings of DiGennaro Reed and Henley. The current findings indicate notable changes in the demographic characteristics of survey respondents. In addition, we observed modest increases in the use of best practices for initial and ongoing training and performance management. Results also indicate several areas of concern regarding the provision of supervisory skills training.

• Results revealed improvements in the percentage of respondents who received initial or preservice training compared to DiGennaro Reed and Henley (2015). However, employers primarily rely on instructions and modeling to train their employees.

• Findings revealed greater reliance on asynchronous and synchronous online training modalities compared to DiGennaro Reed and Henley (2015).

• Results revealed a slight shift in the percentage of respondents who reported receiving ongoing training compared to DiGennaro Reed and Henley (2015). In the present study, BCaBAs and RBTs generally received ongoing training; however, fewer BCBAs reported receiving ongoing training.

• The reported use of performance management practices improved compared to DiGennaro Reed and Henley (2015). That is, a higher percentage of respondents reported being observed at work while carrying out their job responsibilities.

• Of the respondents who supervise staff, less than half of them reported receiving supervisory skills training. And only half of those respondents reported that their training prepared them to supervise others.

The field of behavior analysis has experienced marked growth in the number of credentialed professionals over the last decade (Behavior Analyst Certification Board [BACB], n.d.). Between 2015 and 2021, the numbers of board certified behavior analysts (BCBA1) and board certified assistant behavior analysts (BCaBA) have doubled, whereas the number of registered behavior technicians (RBT) has grown sevenfold (BACB, n.d.). This rapid growth has produced 172,162 credentialed professionals in 2021, an increase of nearly 135,000 professionals since 2015. This growth is also correlated with an increase in the demand for behavior analysts (BACB, 2021c).

The BACB specifies standards for each credential type. For example, aspirants seeking a BCBA credential are required to complete a graduate degree, coursework in specific behavior analytic content, and supervised practical experience in behavior analysis. These entry-level standards represent the minimum competency necessary for certification (Leaf et al., 2017; Kazemi & Shapiro, 2017). Passing a multiple-choice examination containing questions related to content specified on the relevant task list or test content outline (BACB, 2017; BACB, 2021b) in no way guarantees a certified professional demonstrates expert-level performance and has all the skills necessary to succeed professionally. As such, there is a need for ongoing training, professional development, and supervision of credentialed professionals (Parsons et al., 2012; Sellers et al., 2016a).

To determine the extent to which recommended staff training and performance management practices are adopted within organizations employing behavior analysts, DiGennaro Reed and Henley (2015) surveyed individuals who were certified or seeking certification by the BACB. Their findings revealed adoption of less-than-optimal staff training and performance management practices. Approximately 45% of respondents reported they did not receive initial training and 29% of respondents indicated they did not have access to ongoing training. Further, 60% of respondents indicated that their performance was not observed while working. Despite supervisory responsibilities, 66% of respondents reported they did not receive training on effective supervision practices.

Since the collection and publication of these findings, service delivery standards and the field of applied behavior analysis have experienced several changes. The RBT credential became available in 2014 and has subsequently grown since its inception. RBTs are primarily responsible for providing behavior analytic services directly to clients under the supervision of a BCBA. To become a credentialed RBT, an individual must successfully complete a 40-hr training course, pass a competency-based assessment in which candidates perform relevant skills (e.g., conduct preference assessments, graph data, implement teaching procedures), and pass a written examination (BACB, 2021a). Although the BACB requires supervisors to supervise each RBT on their caseload for a minimum of 5% of their monthly service hours (BACB, 2021a), the full range of training and performance management practices offered to RBTs is unknown. For example, at the time DiGennaro Reed and Henley (2015) administered their survey, supervision could include observations with detailed, meaningful feedback and ongoing support (e.g., coaching, professional development) in targeted areas of relative weakness or skills prioritized by the supervisee until competency is achieved. On the other hand, supervision could involve meeting the minimum requirement with brief observations, vague feedback, and meetings where supervisors are distracted or unprepared. Both types of experiences would have technically met the supervision requirements.2 However, it is likely supervisee skillsets would differ under these two approaches. Because the quality of RBT services directly impacts client outcomes (Leaf et al., 2017, 2021), ensuring the development of relevant skills through training and best-practice supervision is an ethical and professional obligation. Unfortunately, the extent to which organizations establish high-quality RBT training, performance management, and supervision practices is unknown.

The presence and quality of staff training is not only a concern for the RBT workforce. Best-practice staff training and performance management should be in place for certificants at all levels. That is, there should be systems in place for all certificants, regardless of certification, to receive training from their employer to successfully carry out their job responsibilities prior to working independently with clients or supervisees. What is equally important, once initial training is complete, employers should ensure ongoing training, performance management, and supervisory support are in place for all credentialed employees.3

Given the influx of RBTs requiring ongoing supervision and the requirement for aspiring BCBAs or BCaBAs to learn how to provide high-quality supervision, it is critical that current supervisors adopt effective supervision practices (BACB, 2017). Recommended practices for individual (Sellers et al., 2016b) and group (Valentino et al., 2016) supervision, and ways to detect and addresses barriers to successful supervision (Sellers et al., 2016a) have been outlined in the literature. Unfortunately, Sellers et al. (2019) reported that some BCBAs who responded to a survey were not aware of the BACB requirements for supervision. Moreover, respondents indicated they did not have the skills or time to effectively engage in supervisory practices.

Due to the exponential growth and demand for BACB certificants, it is critical that effective training, performance management, and supervision are provided. Previous survey data reveal that organizations that employ BCBAs often do not provide initial or ongoing training and support (DiGennaro Reed & Henley, 2015). Moreover, supervisors have reported gaps in their supervisory repertoires (Sellers et al., 2019), which is concerning given the rapid growth in RBTs needing supervision. Recent changes to the supervisory requirements require aspiring BCBAs to learn effective supervision strategies by consulting with a supervisor for their first-year post-certification (BACB, 2018). These collective changes and recent survey data provide a rationale for extending DiGennaro Reed and Henley (2015). Thus, we sought to assess current staff training, performance management, and supervision practices that organizations adopt by surveying credentialed and aspiring behavior analytic professionals.

Method

Participants

Participants were individuals currently certified (i.e., BCBA, BCaBA, RBT) or seeking certification from the BACB who responded to an invitation to complete an anonymous online survey. The survey was distributed by the BACB as well as posted on 14 behavior analytic social media sites (e.g., Students of ABA Facebook page). The number of individuals who received the invitation is unknown given the dissemination modalities (e.g., participants can opt out of receiving emails from the BACB). Thus, we were unable to calculate a response rate. Like DiGennaro Reed and Henley (2015), we required respondents to complete a minimum of 30% of survey items for their data to be included in the analyses. A total of 629 individuals opened the survey link and agreed to participate. Fifty-one respondents did not meet the inclusionary criterion, yielding a total of 578 participants.

Survey Instrumentation

The survey was created using Qualtrics, an experience management software. The settings on Qualtrics permitted respondents to complete the survey one time. The survey consisted of four sections and closely replicated the survey described in DiGennaro Reed and Henley (2015) with two major exceptions. We omitted survey questions on the use of incentives and added questions about RBT supervision. The first section asked respondents to provide demographic information in 10 questions including: (1) gender; (2) age; (3) race; (4) highest degree obtained; (5) degree area of study (e.g., applied behavior analysis, special education); (6) certification held (e.g., BCBA, BCaBA); (7) number of years certified; (8) primary place of employment (e.g. public school, client homes); (9) length of employment with current employer; and (10) primary job classification (e.g., administrator, consultant).

The second section of the survey contained seven questions about the respondents’ initial training (i.e., initial orientation or training provided upon hire) at their current place of employment. The questions included the (1) availability of an onboarding orientation or training upon hire and before working independently; (2) length of the training time; (3) training modality (e.g., live face-to-face, synchronous online, asynchronous online); (4) training format (e.g., group, individual); (5) training practices used (e.g., written instructions, lecture training, modeling, practice, feedback); (6) follow-up questions about ongoing training practices reported in (5); and (7) respondents opinion about the extent to which initial orientation and training prepared them to successfully complete their job responsibilities.

The third section of the survey asked respondents five questions about the ongoing training they receive at their current place of employment. Questions in this section included the (1) availability of ongoing training after a respondent started working; (2) types of training practices used (e.g., workshops, feedback, observation); (3) frequency of ongoing training; (4) follow-up questions about the ongoing training practices reported in (3) (e.g., training modality); and (5) respondents’ view on the relevance of the training topics to their daily job responsibilities.

The fourth section of the survey asked respondents four questions about their supervision practices and the training they received to supervise staff. Thus, only respondents who indicated they worked as a supervisor completed this section of the survey. The questions in this section included the (1) number of staff members they supervise; (2) type of staff supervised (e.g., RBT, BCaBA); (3) availability of training about effective supervision practices; (4) follow-up questions about the supervision practices they reported (e.g., written instructions, lecture); (5) whether the training prepared them to supervise others; (6) if their employer had a policy to monitor the quality of supervision; (7) whether opportunities to provide feedback were provided; and (8) modalities to provide feedback.

Procedure

Before conducting the survey, we obtained approval from the university’s Human Research Protection Program (study #0014010). A link to the survey was sent via email to BACB certificants who previously elected to receive email solicitations from the BACB and 14 behavior analytic Facebook pages (Behavior Analysis in Practice, Teaching Behavior Analysis, Applied Behavior Analysis, Students of ABA, Behavior Analysts for Autism, Confessions of a Behavior Analyst, Uncomfortable BCBA, ABA Business Builders, ABA Idea Share, ABA for Adults with Severe Challenging Behavior, BCBAs Working with Adults, #dobetter Professional Development Movement, Behaviorist Network, and ABA Business Collaborative). Recruitment and data collection occurred from December 5, 2019 through January 7, 2020.

Results

Respondent Demographics

Five hundred seventy-eight individuals completed at least 30% of the survey. Most of the sample identified as female (n = 493, 85.3%), were white (n = 497, 86.3%), and had a master’s degree (n = 359, 62.1%). The average age of respondents was 36.2 years (range: 19–76 years). The most reported degree area of study was applied behavior analysis (n = 294, 51%), followed by special education (n = 73, 12.7%) and general psychology (n = 58, 10.1%). Approximately 55% (n = 321) of respondents held a BCBA credential, followed by RBT (n = 131, 22.7%) and BCBA-D (n = 66, 11.4%). Of the individuals who reported having a credential from the BACB, a majority reported receiving their certification within the last 6 years (n = 390, 73.9%). When asked about their current place of employment, respondents indicated working in a wide range of settings. The most reported settings included private center-based programs (n = 176, 30.5%), client homes (n = 102, 17.7%), and public schools (n = 83, 14.4%). Regarding length of employment, most respondents reported working at their current place of employment for fewer than 6 years (n = 453, 78.5%). Clinician/clinical supervisor (n = 166, 28.7%), direct care staff (n = 132, 22.8%), and administrators/manager/director (n = 82, 14.2%) comprised two thirds of the professionals who responded to the survey. Table 1 provides more details about participant demographics.

Table 1.

Demographic Information

	n	%
Gender
Female	493	85.29
Male	77	13.32
Transgender female	0	0
Transgender male	0	0
Nonbinary/gender variant/gender fluid	2	.35
Other/not listed	1	.17
Prefer not to answer	5	.87
Age
M	36.25
Range	19-76
Of Hispanic, Latino, or Spanish origin
Yes	48	8.30
No	523	90.48
Prefer not to answer	7	1.21
Race (check all that apply)
Arab/West Asian	5	.87
Black or African American	22	3.82
Chinese	9	1.56
Filipino	8	1.39
Japanese	3	.52
Korean	0	0
Middle Eastern	4	.69
Native American or Alaskan Native	10	1.74
Native Hawaiian or Pacific Islander	0	0
Mixed race	20	3.45
South Asian	6	1.04
White	497	86.28
Other	8	1.39
Prefer not to answer	17	2.30
Highest degree obtained
High School Diploma/GED	19	3.29
Associates	13	2.25
Baccalaureate	98	16.96
Masters	359	62.11
Doctorate	89	15.40
Degree area of study
Applied behavior analysis	294	50.95
Experimental analysis of behavior	4	.69
Organizational behavior management	6	1.04
Behavioral psychology	8	1.39
Clinical psychology	22	3.81
Developmental psychology	8	1.39
Educational psychology	5	.87
Experimental psychology	3	.52
Industrial/organizational psychology	0	0
School Psychology	13	2.25
Psychology (general)	58	10.05
Law	0	0
Counseling	12	2.08
Education	27	4.68
Special education	73	12.65
Speech pathology	9	1.56
Social work	6	1.04
Other	29	5.03
Certification held
RBT	131	22.66
BCaBA	11	1.90
BCBA	321	55.54
BCBA-D	66	11.42
Seeking certification	38	6.57
Not seeking certification	11	1.90
Number of years certified
Less than 1 year	108	20.45
1–3 years	148	28.03
4–6 years	134	25.38
7–10 years	66	12.50
11–15 years	44	8.33
More than 15 years	28	5.30
Primary place of employment
Public school	83	14.38
Private school/center-based program	176	30.50
Hospital/medical center	18	3.12
Community mental health center	22	3.81
Residential setting overseen by agency	19	3.29
Client homes	102	17.68
Senior living center	0	0
University/college	44	7.63
Consulting (public agency or private firm)	36	6.24
Private practice/self-employed	38	6.59
Other	39	6.76
Length of employment with current employer
Less than 1 year	173	29.98
1–3 years	179	31.02
4–6 years	101	17.50
7–10 years	66	11.44
11–15 years	28	4.85
More than 15 years	30	5.20
Primary job classification
Administrator/manager/director	82	14.19
Consultant	71	12.28
Clinician/clinical supervisor	166	28.72
Direct care staff	132	22.84
Student	8	1.38
Psychologist/therapist	24	4.15
Researcher/research scientist	20	3.46
Social worker	1	.17
Speech/language pathologist	2	.35
School teacher	19	3.29
Staff trainer	12	2.08
Physical or occupational therapist	1	.17
Other	40	6.92

There was a shift in the respondent demographics from DiGennaro Reed and Henley (2015). This shift is likely due to the introduction of the RBT certification since the administration of the previous survey. In the current survey, a lower percentage of respondents reported having a BCBA credential and a master’s degree, and one of the top three job titles was direct care staff.

Initial or Preservice Training

Approximately two thirds of respondents indicated they received an initial orientation or preservice training before they began to work independently at their current place of employment (n = 383, 66.3%). An analysis of the data4 revealed that 59%, 92%, and 89% of BCBAs, BCaBAs, and RBTs, respectively, received initial or preservice training. The length of training time ranged from less than 1 day (n = 51, 13.4%) to more than 2 weeks (n = 67, 17.5%). Most respondents received training lasting between 1 and 10 days (n = 249, 65.2%) with the most frequently reported training duration lasting between 1 and 3 days (n = 109, 28.5%). Training for BCaBAs and RBTs was reported for longer durations than BCBAs. Training for BCBAs was most frequently reported to last from 1 to 3 days. BCaBAs most frequently indicated that training lasted from 6 to 10 days. RBTs most frequently reported that training lasted 1 to 3 days and more than 2 weeks, which reveals wide variability in the length of training for this credential.

Most respondents indicated the initial orientation or training was conducted live, face-to-face (n = 359, 94.0%) and in a group format (n = 219, 57.2%). With respect to the training methods used during their initial orientation or training, respondents selected all the items that applied; thus, the sum of the items is greater than the total number of participants who completed the survey. The most frequently endorsed training method was verbal instruction about how to perform a skill (n = 316, 82.7%). The second most frequently endorsed item was written instructions about how to perform a skill, constituting 66.2% of responses (n = 253). Practice with actual clients (n = 155, 40.6%) and written or oral quizzes (n = 141, 36.9%) were the training methods reported with relatively lower frequencies. Survey items also asked respondents to specify what component(s) of their initial orientation or training required them to reach a mastery criterion. Most often there was a performance criterion for written and oral quizzes (n = 112, 85.5%). Approximately half of the respondents who reported the use of a mastery criterion indicated a mastery criterion for role-play or rehearsal with an individual other than client (n = 93, 52.5%) and practice with actual clients (n = 77, 51.3%). In addition, slightly more than half of the respondents indicated they received performance feedback as part of training (n = 207, 54.2%). Of the respondents who endorsed receiving performance feedback, a majority indicated it was delivered verbally (n = 149, 77.2%) throughout training (n = 173, 89.8%). Overall, 56% of respondents (n = 203) reported that their initial orientation or training prepared them to successfully complete their job responsibilities. An analysis of the data revealed that 53%, 50%, and 70% of BCBAs, BCaBAs, and RBTs, respectively, reported that their initial training prepared them for their job responsibilities. Table 2 summarizes these results.

Table 2.

Initial or Preservice Training

	n	%
Availability of training
Yes	383	66.26
No, and I should have received training	89	15.40
No, but training was not necessary	106	18.34
Length of training
Less than 1 day	51	13.35
1–3 days	109	28.53
4–5 days	53	13.87
6–10 days	87	22.77
More than 2 weeks	67	17.54
Other	15	3.93
Training modality used
Live face-to-face (in person)	359	94.00
Live but via technology (video conference)	46	12.04
Online (no interaction with another person)	27	7.07
Online (with interaction with another person)	111	29.06
Group training format
Yes	219	57.18
No	164	42.82
Training practices used (select all that apply)
Written instructions about how to perform a skill	253	66.23
Verbal instructions about how to perform a skill	316	82.72
Lecture/Didactic training	222	58.11
Modeling (Trainer demonstrates skills to be performed)	241	63.09
Practice in a role-play or rehearsal with individual other than client	188	49.21
Practice with actual clients	155	40.57
Performance feedback	207	54.19
Praise	185	48.43
Interactive discussion	208	54.45
Shadow current employees while they completed job tasks	221	57.85
Written or oral quizzes	141	36.91
Mastery criterion for practice with an individual other than client
Yes	93	52.54
No	84	47.46
Mastery criterion for practice with actual clients
Yes	77	51.33
No	73	48.67
Mastery criterion for written or oral quizzes
Yes	112	85.50
No	19	14.50
Characteristics of performance feedback (select all that apply)
It was provided one time only at the completion of training	17	8.80
It was delivered throughout training	173	89.84
I received written feedback	70	36.27
I received verbal feedback	149	77.20
I received feedback displayed on a graph	9	4.66
Other	2	1.04
Training prepared me to complete my job duties
Yes	203	55.92
No	52	14.233
Somewhat	99	27.27
Other	9	2.48

The present findings regarding initial or ongoing training practices differ from those reported in DiGennaro Reed and Henley (2015). Current results reveal increases in the percentage of respondents who reported certain features of training, which may reflect improvements in some instances. We observed an increase in the percentage of respondents who reported receiving initial orientation or training, participating in role-play or rehearsal, and meeting a mastery criterion for practice with actual clients. Results also showed an increase in the percentage of respondents who received synchronous online initial orientation or training, which could have both positive and negative impacts on training outcomes depending on the quality of the online instruction. Unfortunately, in the present survey there was a reduction in the percentage of respondents who indicated initial orientation or training successfully prepared them to complete their job.

Ongoing or in-Service Training

Four hundred forty-three (80%) respondents indicated their current place of employment offered ongoing training after they started working. An analysis of the data revealed that 77%, 91%, and 89% of BCBAs, BCaBAs, and RBTs, respectively, received ongoing or in-service training. When asked about the training methods used during their ongoing or in-service training, respondents selected all items that applied; thus, the sum of the items endorsed is greater than the total number of participants who completed the survey. The most frequently endorsed practices were performance feedback (n = 304, 69.7%), workshops or lectures offered at the respondents’ current place of employment (n = 292, 67%), and in-person observations by supervisors during work hours (n = 292, 67%). Respondents indicated ongoing or in-service workshops or lectures offered at their place of employment were most often available monthly (n = 124, 43.1%) and delivered in person (n = 290, 89.2%). With respect to the training content, approximately two thirds of respondents (n = 295, 68.3%) reported the training topics were directly relevant to their daily job responsibilities. Respondents reported supervisory observations most often took place weekly (n = 104, 36.6%). An analysis of the data revealed that 62.7%, 2.4%, and 34.9% of BCBAs, BCaBAs, and RBTs, respectively, receive weekly feedback. When asked about the format of performance feedback, 92.5% of respondents indicated they received feedback verbally (n = 260) and by a supervisor (n = 244, 86.8%), which was consistent across certification types. More information is in Table 3.

Table 3.

Ongoing or In-Service Training

	n	%
Availability of training
Yes	443	79.96
No	111	20.04
Training practices used (select all that apply)
Workshops/lectures offered at your place of employment	292	67.00
Workshops/lectures offered outside your place of employment	157	36.01
Supervisor or trainer observes you working	292	66.97
Performance feedback	304	69.72
Conference attendance	190	43.58
Online trainings or webinars	214	49.08
Other	29	6.65
Frequency of workshops/lectures offered at place of employment
One time	6	2.08
Weekly	18	6.25
Monthly	124	43.06
Quarterly	67	23.26
Twice yearly	25	8.68
Annually	18	6.25
Other	30	10.42
Frequency of workshops/lectures offered outside place of employment
One time	6	3.87
Weekly	1	.65
Monthly	16	10.32
Quarterly	31	20.00
Twice yearly	25	16.13
Annually	39	25.16
Other	37	23.87
Training modality used (select all that apply)
Live face-to-face (in person)	290	89.23
Live but via technology (video conference)	111	34.15
Online (no interaction with another person)	58	17.85
Online (with interaction with another person)	64	19.69
The training topics were directly relevant to my daily job
Yes	295	68.29
No	10	2.31
Sometimes	127	29.40
Frequency of supervisor observations
Daily	17	5.99
Weekly	104	36.62
Monthly	73	25.70
Quarterly	31	10.92
Twice yearly	8	2.82
Annually	11	3.87
Never	6	2.11
Other	34	11.97
Format of performance feedback (select all that apply)
Written feedback	205	73.00
Verbal feedback	260	92.53
Graphical feedback	28	10.00
From my supervisor	244	86.83
From my co-workers	125	44.48
From the family of clients	99	35.23
Daily	45	16.01
Weekly	91	32.38
Monthly	81	28.83
Other	22	7.83

The present findings about ongoing or in-service training differ from those reported in DiGennaro Reed and Henley (2015). Current results reveal an increase in the percentage of respondents who received ongoing or in-service training compared to the previous survey. In addition, the results reveal an increase in the delivery of ongoing trainings via remote synchronous and asynchronous platforms. Fortunately, there was an increase in the percentage of respondents who indicated ongoing training was relevant to their daily job.

Supervisory Training

Sixty-one percent of respondents (n = 318) indicated they were responsible for supervising other staff. Most respondents reported supervising fewer than 15 staff (n = 264, 84.1%). Most often supervisors reported supervising RBTs (n = 134, 42.8%) or noncertified direct support professionals (n = 123, 39.3%). When asked about the training they received on effective supervision practices, most respondents reported that their current place of employment did not provide this training (n = 178, 56.9%). Of the respondents who reported receiving training on how to supervise staff, nearly 96% reported the supervisory training successfully or at least somewhat prepared them to supervise others. Nearly half of the respondents reported their current place of employment had a policy in place to monitor the quality of supervision provided (n = 195, 46.3%). In addition, most respondents, regardless of credential type, had the ability to provide feedback on the quality of supervision (n = 273, 64.9%) with the most commonly reported modality being supervisor solicitation of feedback (n = 194, 71.6%). Table 4 summarizes these data.

Table 4.

Supervisory Training

	n	%
Responsible for supervising staff
Yes	318	60.92
No	204	39.08
Number of staff supervised
1–5	125	39.81
6–10	97	30.89
11–15	42	13.38
16–20	13	4.14
21–30	12	3.82
31–40	1	.32
41–50	3	.96
51–75	1	.32
76–200	2	.64
Other	16	5.10
Type of staff supervised
RBT	134	42.81
BCaBA	6	1.92
BCBA	48	15.34
BCBA-D	2	.64
Noncertified direct support professionals	123	39.30
Number of RBTs supervised at any one time
1	51	38.64
2–3	31	23.48
4–6	23	17.42
7–10	15	11.36
11–15	9	6.82
16–20	1	.76
More than 20	2	1.52
Frequency of supervisor observations of supervisor performance
Daily	3	2.26
Weekly	16	12.03
Monthly	18	13.53
Quarterly	15	11.28
Twice yearly	7	5.26
Annually	1	.75
Never	59	44.36
Other	14	10.53
Availability of training
Yes	135	43.13
No	178	56.87
Type of training practices used (select all that apply)
Written instructions about how to perform a skill	83	63.85
Verbal instructions about how to perform a skill	98	75.38
Lecture/didactic training	70	53.85
Modeling (trainer demonstrates skills to be performed)	85	65.38
Practice in a role-play or rehearsal situation	75	57.69
Performance feedback	86	66.15
Praise	74	56.92
Interactive discussion	78	60.00
Shadow current employees while they completed job tasks	54	41.54
Written or oral quizzes	24	18.46
Training prepared respondents to supervise others
Yes	68	52.71
Somewhat	50	38.76
Not at all	5	3.88
Other	6	4.65
Employer had policy to monitor quality of supervision provided
Yes	195	46.32
No	195	46.32
Other	31	7.36
Opportunities to provide feedback on quality of supervision and support
Yes	273	64.85
No	148	35.15
Modalities to provide feedback on supervision (select all that apply)
Anonymous feedback form	117	43.17
Supervisor solicitation	194	71.59
Written feedback	131	48.34
Other	19	7.01

Discussion

The purpose of the current study was to extend DiGennaro Reed and Henley (2015) by querying certified and aspiring behavior analytic professionals about the staff training, performance management, and supervision practices adopted in their current organizations. Since 2015, the field has experienced marked growth in the numbers of credentialed professionals, including the addition of the RBT credential. In this time there has also been numerous advancements regarding recommended staff training, performance management, and supervision practices. Thus, an updated summary of these practices in settings employing board-certified staff is warranted.

The demographics of respondents differed from DiGennaro Reed and Henley (2015) given the introduction of the RBT credential in recent years. The current respondents were predominately white women with a background in applied behavior analysis credentialed as BCBAs or RBTs. It should be noted that the current respondents were relatively junior in terms of how long they had been certified and employed at their current place of employment. Nearly three quarters of respondents became certified in the past 6 years with 60% of respondents indicating they have been employed by their current place of employment for fewer than 3 years. These data are in line with the field’s growth in the past 5 years.

Results revealed improvements in the percentage of respondents who received initial or preservice training compared to DiGennaro Reed and Henley (2015). This increase may also be partly attributable to the introduction of the RBT credential. A high percentage of BCaBAs and RBTs received initial training (92% and 89%, respectively); whereas relatively fewer BCBAs received initial training (i.e., 59%). When BCBAs did receive training, it was reported to have occurred for very short durations (i.e., 1–3 days). This finding is problematic as all credentials, including the BCBA, represent an entry-level standard. Despite having a graduate education, BCBAs are professionals who have demonstrated a minimum level of competency on an examination (Carr & Nosik, 2017). Moreover, many new BCBAs will assume oversight of clinical or educational programs and supervision of trainees thus requiring a higher level of competency than that assessed on a multiple-choice exam. Our position is that initial training at the BCBA level is necessary, could incorporate advanced topics with strategic review and overlap of more basic concepts of our science and practice, and should include the use of evidence-based training procedures. At a minimum, we recommend new BCBAs undergo a performance-based assessment upon hire to demonstrate competency on skills necessary for their position and identify any areas of weakness that require additional training.

A promising finding is that training is being offered to a high percentage of BCaBAs and RBTs; training provided to RBTs is particularly important given the recent inception of the RBT credential and that requirements for the credential are less stringent. Unfortunately, an equal percentage of respondents (26.5%) with the RBT credential reported training lasting between 1 and 3 days or more than 2 weeks. Training lasting between 1 and 3 days would be insufficient to prepare RBTs for working directly with consumers. One way to tackle this issue is for funders to establish a minimum standard of initial and ongoing training that aligns with best practices, which could be linked to funding or reimbursement. Another option is for funders to require organizations to seek accreditation by respected accreditation bodies, such as the Commission on Accreditation of Rehabilitation Facilities. These bodies establish and uphold standards for training that are required for accreditation.

Although results revealed improvements in the percentage of respondents who indicated receiving initial training, employers are still primarily relying on instructions and modeling to train their employees. When practice opportunities were presented, employers created role-play scenarios or included actual clients. Parsons et al. (2012) discussed the importance of trainees practicing and receiving feedback on the skills taught in training prior to working independently in the work environment, as this practice may increase staff acceptance of the training. It is critical for employers to create opportunities for new employees to practice skills taught and receive feedback during initial training. It is unlikely that new employees will implement a procedure with high levels of integrity in the natural environment if they did not demonstrate a high level of performance during training. It is possible that integrity may decrease in the natural environment, even if performance in training was high; therefore, it is critical for employers to create these opportunities to prepare new employees for their job responsibilities.

One consistent finding between the current results and those of DiGennaro Reed and Henley (2015) is the reliance on peer “shadow” training. More than half of the participants in both surveys reported shadowing a peer employee performing on-the-job duties as part of their initial training. This finding is interesting given the limited research or practice guidelines for implementing this training approach. Peer training could serve as a form of in-vivo modeling that may potentially increase generalization of the skills to the target environment post-training. However, its efficacy is currently unclear, and it may be the case that organizations would be better served to reallocate training resources to empirically supported training methods (e.g., role-play with feedback). If organizations choose to implement peer training, we suggest using a structured approach in which organizations: (1) provide research-supported training to peer trainers (e.g., Blackman et al., 2022; Erath et al., 2020, 2021); (2) standardize the training process to ensure trainees observe relevant events and situations; (3) collect data to ensure the program is having a positive impact on new-staff performance; and (4) revise training procedures based on these data.

Findings revealed greater reliance on asynchronous and synchronous online training modalities compared to DiGennaro Reed and Henley (2015). However, the data show that these online training modalities supplement face-to-face training, as nearly all respondents indicated receiving face-to-face training. Recent research has summarized effective training components to maintain the quality of remote training while decreasing trainer presence (e.g., Erath & DiGennaro Reed, 2020; Marano et al., 2020). A review by Erath and DiGennaro Reed (2020) revealed that antecedent training components resulted in mastery-level performance for some staff, but coaching may be required for all to reach criterion. In the end, the antecedent training components may increase training efficiency with respect to trainer time, but do not eliminate the need for expert trainer time entirely.

Research has also evaluated the use of videoconferencing software to train staff various skills (e.g., Fischer et al., 2016; Neely et al., 2019). Neely et al. (2019) revealed that behavior analytic procedures can be effectively taught via videoconferencing software. Thus, antecedent training strategies could be beneficial for employers to adopt. This approach may allow for face-to-face or synchronous training time to be spent having employees practice the skills taught through online training and receive feedback on their performance, thereby enhancing training quality. In addition, more research is needed on best practices for online training, such as the degree to which training incorporates active participation components or approaches to maximize retention and knowledge of training content. When adopting online training, organizations should also consider the function and goals of the training, which could vary from an initial online primer to a standalone, self-paced training module.

Despite some advancements in the adoption of best-practice initial training procedures, the results revealed that more respondents indicated their initial training did not or only somewhat prepared them for their job responsibilities. It is critical for employers to determine the extent to which their initial training prepares employees for their job responsibilities, and to be able to adjust those practices as needed. Ongoing performance management practices may circumvent issues that arise when employees do not have the skills to carry out their job responsibilities, but it is better for employers to be proactive and provide support prior to these issues arising. When possible, we encourage employers to observe trainees in the real work environment and provide feedback as part of the initial training to ensure individuals can perform all relevant skills in the target setting.

The unscripted nature of in-vivo experiences makes it unlikely that trainers will be able to observe the myriad unique situations in which employees will be expected to perform the full range of newly trained skills. Thus, we also recommend employers adopt a mechanism for new employees to provide honest feedback on their initial training experience. It may be especially useful to collect post-training feedback after employees have been working independently and had the opportunity to implement the new skills with real clients in the absence of a trainer. Many new employees experience an overconfidence bias in which they overestimate their skills (LeBlanc, Sellers et al., 2020a). It is possible that collecting this feedback immediately following training, prior to working independently, may not be an accurate predictor of performance—although this immediate feedback certainly has utility. Information about which skills, or aspects of a skill, employees found most difficult to implement in the post-training work environment would provide employers with areas in which initial training may require modification. Employers may also wish to collect information from supervisors about the feedback they most frequently provide to new employees following training. This information may assist employers in continuing to improve the initial training they offer.

Results revealed a slight shift in the percentage of respondents who reported receiving ongoing training compared to DiGennaro Reed and Henley (2015). Our analysis revealed that, in general, BCaBAs and RBTs received ongoing training; however, relatively fewer BCBAs received ongoing training. An explanation as to why BCaBAs and RBTs receive ongoing training is that the BACB requires ongoing supervision for BCaBAs and RBTs. The finding that BCBAs do not regularly receive ongoing training, coupled with the initial training data for BCBAs, is concerning.

Plantiveau et al. (2018) reported that a majority of their survey respondents (i.e., 67%), newly certified BCBAs, reported experiencing burnout and little job satisfaction. Their data analyses revealed that a lack of frequent supervision and social support (e.g., attending frequent and relevant training opportunities) contributed to this finding. Many recently certified BCBAs find it challenging to adjust to the increased complexity and demands of their new roles—positions which often call for skills that were not required for certification (e.g., professional and interpersonal repertoires; LeBlanc, Sellers et al., 2020a). In the absence of ongoing training and support, individuals are likely to experience an imbalance between the skills needed for, and the demands of, their position—prolonged exposure to which has been linked to burnout in related fields (Plantiveau et al., 2018). Ongoing training can mitigate this imbalance by providing individuals with the skills needed to carry out their job responsibilities effectively and efficiently, likely leading to higher rates of reinforcement. Several discussions have occurred regarding the skills that should be taught to BCBAs after they are certified. These topics include compassionate care (LeBlanc, Taylor et al., 2020b) and cultural humility (Beaulieu et al., 2019), among others, as these skills are beneficial for supervisors to learn and are not currently required in the Association for Behavior Analysis International’s tiered model of education (see https://www.abainternational.org/higher-education/tiered-model-of-education.aspx) or to be a verified course sequence (https://www.abainternational.org/higher-education/verified-course-sequence.aspx).

Training also provides an important opportunity for social support through interactions with other employees. These opportunities may be especially beneficial for BCBAs who work in one-on-one settings with few opportunities to interact. However, given the increase in remote trainings we observed, more research is needed on the relation between training modality and the extent to which such training provides opportunities for social support.

Respondents reported an increase in the number of ongoing training opportunities that were hosted remotely, specifically via webinars. Like that of initial training, online training seemed to supplement face-to-face training. These results are consistent with Hajiaghamosheni et al. (2021), as 60% of their respondents indicated receiving training via conferences and through achieving continuing education credits. Given the increasingly digital world and the success of teaching various skills via videoconferencing software (e.g., Fischer et al., 2016; Neely et al., 2019), it is not surprising that more learning opportunities are being provided at a distance. To our knowledge, there has yet to be a study to evaluate the effects of a conference or continuing education presentation on attendee subsequent performance. Therefore, it is unknown to what extent the ongoing training opportunities, conducted via webinars, affected respondents’ subsequent performance. Employers should have systems in place to track the impact of their ongoing training to ensure the trainings themselves change the behavior of attendees.

The reported use of performance management practices improved compared to DiGennaro Reed and Henley (2015). That is, a higher percentage of respondents reported being observed at work while carrying out their job responsibilities. It is possible that employee performance from the training environment does not generalize to the work environment. If there is a lack of generalization and systems in place to observe employee performance and provide feedback, then employees may not feel supported or may make errors, and client outcomes may be negatively affected. With routine observations in place, employers can proactively identify and mitigate employee performance problems. Novak et al. (2019) outlined feasible ways for employers to adopt evidence-based performance management practices. The authors provided an observation protocol that supervisors could adapt for their observations. In addition, benefits of using a pay for performance system (i.e., employees are compensated for their work rather than time spent working) and investing in employee continuing education are discussed.

The current survey results revealed a decrease in the percentage of respondents who are responsible for supervising staff compared to DiGennaro Reed and Henley (2015). Given that the present results incorporate RBTs, this finding is unsurprising. Of the respondents who supervise staff, less than half of them reported receiving supervisory skills training and only half of those respondents reported that their training prepared them to supervise others. This finding is problematic as BCaBAs and RBTs require ongoing supervision and it appears supervisors lack the training to effectively supervise.

Recommendations for Practice

Aspiring behavior analysts are now required to learn how to provide effective supervision as part of the requirements to become certified (BACB, 2017).5 This change does not solve the problem that nearly 55,000 BCBAs are currently credentialed and may lack this important skill. One way to address this issue is for employers to adopt effective supervision training for current supervisors. Numerous resources have been published that employers can incorporate into supervisory training. For example, Reid et al. (2012) published a manual that provides evidenced-based suggestions and protocols for supervisors to improve employee performance. LeBlanc, Sellers et al. (2020a) published a book on how to build and maintain meaningful relationships with supervisees. Their book provides self-reflection exercises for supervisors on strategies to be better supervisors. In a special issue on supervision in Behavior Analysis in Practice, Valentino et al. (2016) outlined the benefits of group supervision and provided a structure for how supervisors can carry out this practice. In the same special issue, Sellers et al. (2016b) provided recommendations for providing effective individual supervision.

Another practice we recommend is for employers to evaluate the effects of the supervision practices their supervisors employ. At present, half of our respondents’ places of employment do not have a policy in place to monitor the quality of their supervisor’s supervision practices, despite this being a requirement for behavior analysts (BACB, 2021b). Sellers et al. (2016a) provide recommendations for supervisors to address gaps and barriers they have identified. For example, if a supervisee has difficulty accepting feedback, the authors suggest identifying the function, reviewing strategies for effective feedback, and outlining expectations for receiving feedback. These recommendations should be addressed individually and potentially systemically across the organization.

Our current results revealed that supervisors have adopted a variety of feedback modalities to receive feedback on their supervisory performance (e.g., written feedback, informal solicitation). Employers should develop and implement policies to ensure supervisors have these practices in place and that supervisees are comfortable providing honest feedback to their supervisor. Bidirectional feedback (i.e., supervisor to supervisee and supervisee to supervisor; LeBlanc et al., 2020a) will aid in improvements in the performance of everyone at the organization and assist in creating a culture where ongoing feedback is viewed as supportive and delivered in the best interests of those who receive it.

Limitations

This survey is not without limitations that should be addressed in future research. As with any survey research, the results were obtained through self-report. There is no way to determine whether the responses obtained were in accordance with the practices that are in place at the organizations in which the respondents work. Further, it is possible that some respondents were not familiar with the practices at their current place of employment given their tenure or experience with different training opportunities. For example, an RBT may not have been aware of the ongoing training opportunities that were in place if they had only been employed by the organization for 3 weeks. It is unknown the extent to which this situation may have influenced the results. In addition, it is possible that multiple respondents from the same organization completed the survey, which may have skewed the results. That is, if 50 respondents were from an organization that provides initial, ongoing, and supervisory training, and has performance management practices in place then the current results may look better than what is truly occurring in practice. The results specific to the training and performance management of BCaBAs should also be viewed with some caution given the relatively low number of respondents holding this certification (i.e., n = 11). Finally, we do not have performance data for any of the respondents. Some respondents may have indicated that their training successfully prepared them to complete their job responsibilities, but they routinely make errors.

Data Availability

The full dataset is freely available on Open Science Framework (https://osf.io/f4zds/) for readers who are interested.

Declarations

Conflicts of Interest

We have no known conflicts of interest.

Ethical Approval

The study was approved by the University IRB.

Informed Consent

All participants provided informed consent.