A Survey of Staff Training and Performance Management Practices: the Good, the Bad, and the Ugly

Abstract

The costs for delivering services to individuals with autism and other disabilities total more than $137 billion annually and grow exponentially. Given this figure, service-delivery organizations are under pressure to ensure staff are well-prepared to deliver services through the provision of training. Providing effective staff training and performance management is also necessary for the delivery of evidence-based practice and is an ethical obligation for Behavior Analyst Certification Board®, Inc. (BACB®) certificants. The purpose of the present study was to document the various types of staff and supervisory training and performance management procedures offered to BACB® certificants and aspirants working in applied settings.

The quality of services provided to consumers is a function of the behavior of staff delivering said services which, in turn, is influenced by the quality and consistency of training and performance management practices adopted by organizations. Organizations risk a host of negative outcomes when they offer ineffective staff training programs and little follow-up support. Research has shown that staff–consumer interactions (Finn and Sturmey 2009), opportunities for teaching (Schepis et al. 2001), consumer quality of life (Jahr 1998), and other important outcomes (e.g., consumer engagement, Szczech 2008) may be compromised when staff receive poor training and insufficient performance management practices. These findings underscore the importance of ensuring direct-care staff are well-trained to accurately and consistently implement the teaching and behavioral intervention plans for consumers (DiGennaro Reed et al. 2013). Given that behavior analysts may work in management or supervisory positions and commonly oversee direct-care staff or guide the development of teaching procedures and behavioral interventions, ongoing training and professional development is also necessary for upper-level employees. Thus, the provision of empirically supported training directly relevant to the daily job responsibilities of employees working at all levels of an organization is valuable. Behavior analysts working in a supervisory capacity could participate in a training program that addresses effective supervision practices (e.g., Reid et al. 2012). Organizations might offer monetary incentives or bonuses for directors, behavior analysts, and direct-care staff if they meet or exceed a minimum performance criterion on consecutive monthly observations. Ideally, organizations make available and require employees to participate in pre-service and in-service training, ongoing professional development, and performance management programs.

The Behavior Analyst Certification Board®, Inc. (hereafter, BACB®) offers guidance on this issue to behavior analysts with board certification in their Guidelines for Responsible Conduct for Behavior Analysts. Guideline 5.0 The Behavior Analyst as Teacher and/or Supervisor specifies the delivery of training and teaching. This particular guideline does not detail particular training practices and content but establishes an ethical obligation to design training programs that entail proper supervised experiences, competently designed training and supervisory activities, and training programs that meet their goals (Bailey and Burch 2011). Moreover, Guideline 1.0 Responsible Conduct of a Behavior Analyst requires behavior analysts to “rely on scientifically and professionally derived knowledge” in one’s professional activities, which may be interpreted to mean behavior analysts involved in staff training or supervision rely on empirically supported or evidence-based training and performance management practices.

Performance- and competency-based training (PCBT) is an evidence-based procedure that includes training sessions in a group format followed by on-the-job training (Parsons et al. 2012; Reid et al. 2011). The group training aspect incorporates a six-step protocol including the following components: (1) describe the target skill, (2) provide written instruction, (3) demonstrate the target skill (i.e., modeling), (4) trainee practice of the target skill, (5) feedback delivered to the trainee about his or her performance during practice, and (6) repetition of steps 4 and 5 until mastery is achieved (Parsons et al. 2012). Because this protocol requires trainers and trainees to perform the target skill with some competency (i.e., meet a mastery criterion), this procedure is both performance- and competency-based. On-the-job training is necessary due to the analogue nature of group training; that is, PCBT group training involves simulated practice removed from the real-world complexities of delivering services. As a result, organizations are strongly encouraged to invest resources in on-the-job training to ensure trainees’ skills generalize from the training environment to the delivery of services in the applied setting. Parsons et al. (2012) recommend trainers observe trainees delivering services and then provide feedback about their performance until trainees successfully perform the target skill in the applied environment. Organizations that provide both PCBT group training and on-the-job training will still need to allocate resources to ongoing staff support, professional development, and performance management to ensure skills maintain over time and staff acquire the skills required to serve diverse clients with varying needs (Daniels 1994; Parsons et al. 2012; Reid et al. 2012).

Behavioral skills training is similar to PCBT and involves instructions, modeling, rehearsal, and feedback (Sarokoff and Sturmey 2004). This package has been used to effectively teach staff to implement mand training (Nigro-Bruzzi and Sturmey 2010), three phases of the picture exchange communication system (Rosales et al. 2009), a paired-stimulus preference assessment (Lavie and Sturmey 2002), behavioral support plans (DiGennaro et al. 2010), and other teaching procedures (e.g., Palmen et al. 2010). Some researchers have measured the collateral effects of behavioral skills staff training packages on consumer outcomes (e.g., Nigro-Bruzzi and Sturmey 2010); in general, the results document effective staff training produces improvements for both trainees and consumers.

Despite the robust evidence in support of PCBT or behavioral skills training, there is much to learn about the most effective and efficient ways to implement the various components of the training package. For example, research has only begun to address how to maximize the effects of written instruction (e.g., Graff and Karsten 2012) and modeling (e.g., Catania et al. 2009; Moore and Fisher 2007), the need for rehearsal (e.g., Ward-Horner and Sturmey 2012), or the type and frequency of performance feedback (e.g., Pence et al. 2014; Wood et al. 2007). Perhaps more importantly, the extent to which organizations adopt empirically supported training and performance management practices for use with staff is unknown. To develop a better understanding of training and performance management practices, the purpose of the present study was to inquire about the various types of staff and supervisory training procedures offered to BACB® certificants and aspirants working in applied settings. The published literature includes summaries of myriad procedures, with some clearly more effective than others. Identifying which of these practices are adopted by organizations and experienced by staff will inform ways to address the research-to-practice gap and possibly stimulate research to tackle the real-world needs of practitioners.

Method

Participants

Participants in this study were primarily individuals currently certified (i.e., BCBA-D®, BCBA®, BCaBA®) or seeking certification from the BACB® who responded to an invitation to complete an online anonymous survey. The survey was distributed by the BACB®, behavior analytic social media sites (e.g., ABAI official Facebook® page), and a professional listserv (i.e., teaching behavior analysis). The number of individuals who received the invitation is unknown because certificants can opt out of receiving email solicitations from the BACB® and we were unable to collect data regarding the number of views on the social media sites. As a result, we did not calculate a response rate. Four hundred individuals opened the survey link all of whom agreed to participate. We excluded respondents from analyses if they responded to fewer than 30 % of survey items. Eighteen respondents met this exclusionary criterion. Thus, 95 % (n = 382) of the individuals who consented to participate completed more than 30 % of survey items and were included in the study.

Instrumentation

To assess training provided to respondents working in service-delivery settings, we developed a survey consisting of five sections posted online by Qualtrics. We arranged the settings on Qualtrics so respondents could only complete the survey one time. The first section asked participants to provide demographic information including the following: (a) gender; (b) age; (c) highest degree obtained; (d) degree area of study (e.g., applied behavior analysis, special education); (e) certification held (e.g., BCBA®, BCaBA®); (f) number of years certified as a behavior analyst; (g) primary place of employment (e.g., private center-based program, hospital/medical center, residential setting); (h) length of employment with current employer; and (i) primary job classification (e.g., clinician, direct-care staff, consultant).

The next section of the survey contained questions regarding pre-service training (i.e., initial orientation or training provided upon hire) at the respondents’ current place of employment. These included (a) availability of orientation or training upon hire and before working independently; (b) length of time (range of 1 day to 2+ weeks); (c) delivery format (e.g., face-to-face, online); (d) format (group or individual); (e) training practices used (e.g., written instructions, modeling, feedback); (f) follow-up questions about training practices reported in (e) (e.g., mastery criterion, type and frequency of feedback); and (g) respondents’ views on the extent to which the initial orientation or training prepared them to successfully complete their job responsibilities.

The third section of the survey asked respondents about ongoing training they receive at their current place of employment. Items included (a) availability of ongoing training after respondents started working; (b) training practices (e.g., workshops/lectures, feedback, conference attendance); (c) how often the ongoing training occurs; (d) follow-up questions about ongoing training practices reported in (c) (e.g., format of training, frequency of supervisory observations, frequency of feedback); and (e) respondents’ views on the relevance of the training topics to the job they perform daily.

The fourth section of the survey asked respondents about the use of incentives at their current place of employment. Items included (a) availability of monetary or non-monetary incentives or bonuses; (b) if receipt of incentives or bonuses are contingent on performance; (c) how often the incentives are delivered; and (d) an opportunity to describe the incentives.

If respondents indicated they are a supervisor, the final section of the survey asked them about the training they received to supervise staff. Items included (a) the number of staff members they supervise; (b) availability of training about effective supervision practices; (c) an opportunity to describe the training practices made available pending the response provided in (b); and (d) respondents’ views on the extent to which the training successfully prepared them to supervise others.

Procedure

Before conducting the survey, we obtained approval from the governing Institutional Review Board. We sent the survey link via electronic mail to BACB® certificants who previously elected to receive email solicitations from the BACB® and one behavior analytic listserv (i.e., teaching behavior analysis). In addition, we posted the invitation to seven behavior analytic social media sites (i.e., Facebook® pages of The Analysis of Verbal Behavior, Behavior Analysis in Practice, The Behavior Analyst, Association for Behavior Analysis International, Applied Behavior Analysis, Kansas Association for Behavior Analysis, and Students of Applied Behavior Analysis). Recruitment and data collection occurred in March 2014.

Results

Respondent Demographics

Three hundred and eighty-two individuals completed at least 30 % of the survey. The sample was primarily female (82.72 %) with a mean age of 37.87 (range, 23 to 75 years). A majority of respondents had master’s (76.70 %) or doctoral (16.75 %) degrees. The most commonly reported degree area of study was applied behavior analysis (n = 182, 47.64 %). Nearly three quarters of respondents identified themselves as having a BCBA® (n = 285, 74.61 %). Of individuals certified by the BACB®, a large majority reported receiving their certification in the last 5 years (n = 226, 62.78 %). Five individuals reported they were not seeking certification; they were employed as a behavioral consultant (n = 2), direct-care staff (n = 1), administrator of a university-based program (n = 1), and a staff trainer at a center-based program (n = 1). When asked about their current place of employment, respondents indicated working in a wide range of settings. Several of the most commonly reported settings include private center-based programs (n = 88, 23.04 %), public schools (n = 84, 21.99 %), and client homes (n = 72, 18.85 %). Nearly two thirds of respondents reported working at their current place of employment for fewer than 5 years (n = 262, 68.59 %). Consultants (28.27 %), administrators/managers (16.23 %), and clinicians (15.71 %) comprised more than half of the professionals who responded to the survey. Table 1 summarizes the demographic, education, and employment data in more detail.

Table 1.

Demographic information

	n	%
Gender
Female	316	82.72
Male	66	17.28
Age
M	37.87
Range	23-75
Highest degree obtained
Baccalaureate	25	6.55
Master’s degree	293	76.70
Doctorate	64	16.75
Degree area of study
Applied behavior analysis	182	47.64
Behavioral psychology	8	2.09
Clinical psychology	17	4.45
Counseling	11	2.88
Developmental psychology	5	1.31
Education	15	3.93
Educational psychology	9	2.36
Experimental analysis of behavior	6	1.57
Industrial/organizational psychology	2	0.52
Organizational behavior management	3	0.79
School psychology	14	3.67
Social work	4	1.05
Special education	69	18.06
Speech pathology	10	2.62
Other	27	7.07
Certification held
BCaBA	22	5.76
BCBA	285	74.61
BCBA-D	53	13.87
Not seeking certification	5	1.31
Seeking certification	17	4.45
Number of years certified as a behavior analyst
0 to 5	226	62.78
6 to 10	85	22.25
11 to 15	35	9.16
More than 15	14	3.67
Primary place of employment
Client homes	72	18.85
Consulting firm	30	7.85
Hospital/medical center (in-patient or out-patient)	17	4.45
Private center-based program	88	23.04
Public school	84	21.99
Residential setting overseen by an agency	23	6.02
University/college	25	6.55
Other	43	11.26
Length of employment with current employer
0–5 years	262	68.59
6–10 years	56	17.02
11–15 years	34	8.90
More than 15 years	30	7.85
Primary job classification
Administrator/manager	62	16.23
Clinician	60	15.71
Consultant	108	28.27
Direct-care staff	15	3.93
Psychologist/therapist	23	6.02
Researcher/research scientist	14	3.67
School teacher	15	3.93
Speech/language pathologist	2	0.52
Staff trainer	15	3.93
Student	10	2.62
Other	58	15.18

Initial or Pre-Service Training

Approximately half of the respondents indicated that upon hire at their current place of employment, they received an initial orientation or training before working independently (n = 209, 54.71 %). The lengths of time ranged from less than 1 day (13.40 %) to more than 2 weeks (18.66 %). Training lasting up to 3 (n = 57; 27.27 %) and 5 (n = 43; 20.57 %) days was reported with the highest frequency. A majority of the initial training/orientation was conducted live, face-to-face (n = 205, 98.09 %), and in a group (n = 135, 64.59 %) format. When asked about the training methods used during their initial orientation or training, respondents selected all items that applied; thus, the sum of percentages is greater than 100 %. The most commonly endorsed training method was verbal instruction about how to perform a skill (n = 170, 81.34 %). The second most frequently endorsed item was written instruction about how to perform a skill, constituting 67.94 % of responses (n = 142). Practice in a role-play or rehearsal situation (n = 74, 35.41 %) and practice with actual clients (n = 75, 35.89 %) were training methods reported with the lowest frequency, and fewer than half of the respondents reported that a mastery criterion was required for these training methods. Ninety-eight respondents (46.89 %) indicated they received performance feedback during training. Of respondents who endorsed receiving performance feedback, a majority indicated it was delivered verbally (n = 74, 75.51 %) throughout training (n = 84, 85.71 %). Overall, 63 % of respondents reported the initial orientation prepared them to complete their job responsibilities successfully. Table 2 depicts these data in more detail.

Table 2.

Pre-service training

	n	%
Availability of orientation or training upon hire and before working independently
Yes	209	54.71
No	173	45.29
Length of initial orientation/training
Less than one day	28	13.40
1-3 days	57	27.27
4-5 days	43	20.57
6-10 days	39	18.66
More than two work weeks	39	18.66
Other (please specify)	2	0.96
Ongoing	2
Did not specify	1	0.48
Format of initial orientation or training
Live face-to-face (in person)	205	98.09
Live but via technology (video conference)	7	3.35
Online (no interaction with another person)	26	12.44
Online (with interaction with another person)	8	3.83
Initial orientation or training conducted in a group format
Yes	135	64.59
No	74	35.41
Training practices used
Written instructions about how to perform a skill	142	67.94
Verbal instructions about how to perform a skill	170	81.34
Lecture/Didactic training	122	58.37
Modeling (Trainer demonstrates skills to be performed)	122	58.37
Practice in a role-play or rehearsal situation with an individual other than a client	74	35.41
Practice with actual clients	75	35.89
Performance feedback	98	46.89
Interactive discussion	139	66.51
Shadow current employees while they completed job tasks	115	55.02
Written or oral quizzes	91	43.54
Did not specify	1	0.48
Mastery criterion for role-play or rehearsal	74
Yes	35	47.30
No	38	51.35
Did not specify	1	1.35
Mastery criterion for practice with actual clients	75
Yes	30	40.00
No	45	60.00
Mastery criterion for written or oral quizzes	91
Yes	80	87.91
No	10	10.99
Did not specify	1	1.10
Characteristics of performance feedback	98
It was provided one time only at the completion of training	6	6.12
It was delivered throughout training	84	85.71
I received written feedback	34	34.69
I received verbal feedback	74	75.51
Did not specify	9	9.18
Initial orientation prepared staff for job responsibilities	209
Yes	132	63.16
No	51	24.40
Somewhat	11	5.26
Other	12	5.74
Did not specify	3	1.44

Ongoing or In-Service Training

Two hundred and sixty-two (71 %) respondents indicated their current place of employment offers ongoing training after they started working. The most frequently endorsed practice was workshops/lectures offered at the respondents’ current place of employment (n = 182, 69.47 %). Respondents indicated workshops/lectures were most often available monthly (n = 58, 31.87 %) or quarterly (n = 53, 29.12 %) and delivered in person (n = 159, 87.36 %). Performance feedback was the second most frequently endorsed ongoing training method (n = 173, 66.03 %). When asked about the format of performance feedback, a majority of respondents indicated they receive feedback once per month or less and that it is most commonly delivered verbally (89.02 %) by a supervisor (79.19 %). Respondents also reported they receive ongoing training by attending conferences (n = 169, 64.50 %). Being observed by a supervisor or trainer was reported with relatively lower frequency (n = 148, 56.49 %). Respondents reported supervisory observations most often take place weekly (n = 35, 23.65 %) or monthly (n = 41, 27.70 %). Slightly more than half of the respondents (n = 146, 55.73 %) indicated the training topics are directly relevant to the job they perform daily, and approximately one-third of respondents reported the topics are only sometimes relevant (n = 102, 38.93 %). Table 3 summarizes these data.

Table 3.

In-service training

	n	%
Availability of ongoing training
Yes	262	71.00
No	107	29.00
Practices used during ongoing training
Workshops/Lectures offered at your place of employment	182	69.47
Supervisor or trainer observes you working	148	56.49
Performance feedback	173	66.03
Conference attendance	169	64.50
Other	46	17.56
Frequency of workshops/lectures
Weekly	8	4.40
Monthly	58	31.87
Quarterly	53	29.12
Twice yearly	22	12.09
Annually	24	13.19
Other	16	8.79
Did not specify	1	0.55
Format of workshops or lectures
Live but via technology (video conference)	12	6.59
Live face-to-face (in person)	159	87.36
Online (no interaction with another person)	6	3.30
Online (with interaction with another person)	4	2.20
Did not specify	1	0.55
Frequency of supervisor or trainer observations
Annually	12	8.11
Daily	14	9.46
Monthly	41	27.70
Weekly	35	23.65
Twice yearly	12	8.11
Quarterly	16	10.81
Other	16	10.81
Did not specify	2	1.35
Format of performance feedback
I receive written feedback	111	64.16
I receive verbal feedback	154	89.02
I receive feedback displayed on a graph	13	7.51
I receive feedback from my supervisor	137	79.19
I receive feedback from my co-workers	94	54.34
I receive feedback from the family of clients	79	42.20
I receive feedback daily	26	15.03
I receive feedback weekly	54	31.21
I receive feedback monthly	50	28.90
Did not specify	5	2.89
Direct relevance of training topics
Yes	146	55.73
No	9	3.44
Sometimes	102	38.93
Did not specify	5	1.91

Incentives

Approximately 25 % of respondents indicated their current place of employment offers monetary or non-monetary incentives or bonuses (n = 97). A majority of these respondents reported the incentives are delivered contingent on desired work performance (n = 69, 71.13 %). When asked how frequently incentives are delivered, respondents endorsed annually with the highest frequency (n = 39, 40.21 %) and daily with the lowest frequency (n = 1, 1.03 %). Table 4 summarizes these data.

Table 4.

Incentives

	n	%
Availability of monetary or non-monetary incentives or bonuses
Yes	97	26.72
No	266	73.28
Incentives or bonuses are contingent on desired work performance
Yes	69	71.13
No	28	28.87
Frequency of incentives
Daily	1	1.03
Weekly	2	2.06
Monthly	10	10.31
Quarterly	16	16.50
Twice yearly	15	16.46
Annually	39	40.21
Other	14	14.43

Supervisory Training

Seventy-five percent of respondents indicated they are responsible for supervising other staff (n = 273). Respondents reported they supervise from one to 200 staff; most (n = 200, 73.26 %) supervise fewer than 15 staff members. When asked about the training they received, a majority of respondents reported their current place of employment did not provide training about effective supervision practices (n = 181, 66.30 %). Of the respondents who received training to supervise staff, only four indicated they did not feel their supervisory training prepared them to supervise others successfully (4.40 %). Table 5 depicts these data in more detail.

Table 5.

Supervisory training

	n	%
Responsible for supervising staff
Yes	273	75.62
No	88	24.38
The number of staff supervised
1–5	95	34.80
6–10	71	26.01
10–15	34	12.45
16–20	24	8.79
21–30	18	6.59
31–50	16	5.86
51–75	6	2.20
76–200	5	1.83
Other	3	1.10
Did not specify	1	0.37
Availability of training about effective supervision practices
Yes	91	33.33
No	181	66.30
Did not specify	1	0.37
Supervisory training prepared staff for successfully supervising others
Yes	41	45.06
Somewhat	38	41.46
Not at all	4	4.40
Other	7	7.69
Did not specify	1	1.03

Discussion

The individual and societal costs for autism treatment total $137 billion annually and grow exponentially (Autism Speaks, n.d.); however, this figure does not include direct and indirect costs of behavioral treatment for individuals with disabilities other than autism and for whom behavioral treatment is effective. Given these staggering collective costs, it is understandable for stakeholders to expect staff will be well-prepared to deliver effective services through the provision of training. Moreover, providing effective staff training and performance management is necessary for the delivery of evidence-based practice (Detrich 2008) and is an ethical obligation for BACB® certificants. The purpose of the present study was to document the various types of staff and supervisory training procedures offered to BACB® certificants and aspirants working in applied settings. We also inquired about various performance management practices. Our findings provide a glimpse into the training and support practices experienced by staff working in applied settings.

Only a slight majority of respondents indicated they received pre-service training before working independently. The percentage of respondents who did not receive training is alarming and introduces a host of potentially negative outcomes. Staff who are ill-equipped to deliver behavioral services are more likely to make errors during teaching or implementation of behavior support plans. Should errors arise, staff not only prevent consumer progress toward identified goals and objectives (e.g., Gresham et al. 1993; Dib and Sturmey 2007) but may actually harm consumers (e.g., DiGennaro Reed and Reed 2014). Insufficient or nonexistent staff training and low-quality treatment may compromise a consumer’s right to effective behavioral treatment (Van Houten et al. 1988) and are inconsistent with the BACB®’s Guidelines for Responsible Conduct for Behavior Analysts. Although the latter issue may only be relevant for certificants who are ethically obligated to behave in ways consistent with the guidelines, organizations that offer behavior analytic services would be well-served to ensure the services and organizational processes follow best practices at that time. Omitting initial training or orientation upon hire does not represent best practices. These findings may partially be an artifact of the setting in which respondents work. Unfortunately, we are unable to determine the extent to which the setting may have contributed to this finding. For example, behavior analysts working as independent consultants or starting their own businesses are less likely to participate in an initial training because they may be the only employee or the behavior analyst functions as the trainer for other employees. We suspect that at least a proportion of respondents did not participate in an initial training because of these reasons.

Respondents who participated in pre-service training indicated they experienced a variety of training procedures. At first glance, these findings appear as though they are consistent with best practices (e.g., van Oorsouw et al. 2009). However, training primarily consisted of verbal and written instruction as well as interactive discussion. Respondents reported important components of PCBT, or behavioral skills training, occurred with less frequency. Although approximately 60 % of the respondents indicated a trainer modeled desired performance, fewer than half of the respondents received performance feedback during initial training. Few participated in practice sessions with a confederate or client; when it was available, a majority of respondents were not required to meet a mastery criterion.

Unfortunately, research has shown verbal and written instructions do not produce desired changes in performance (Fixsen et al. 2005) and is less acceptable to staff (Sexton et al. 1996). The finding that a large majority of respondents did not receive the full package of PCBT, or behavioral skills training, before delivering services is concerning given the research supporting its effectiveness and the lack of research supporting the effectiveness of verbal and written instruction when used alone. Perhaps the lack of evidence-based training practices may explain the finding that over one-third of respondents reported they were not prepared to successfully complete their job responsibilities. We were pleased to learn that when performance feedback was provided during pre-service training, it was delivered verbally throughout the training period, which is consistent with best practices (Alvero et al. 2001).

Nearly three quarters of the respondents reported their current place of employment offers ongoing, in-service training primarily in the form of monthly or quarterly face-to-face workshops. This percentage is higher than the percentage of respondents who indicated they received initial training, which suggests that organizations are allocating more resources toward in-service than pre-service training at least for the current sample. Approximately half of the respondents who received ongoing training indicated definitively that the topics are directly relevant to their daily job responsibilities. Employers appear to offer regularly scheduled professional development workshops for a large majority of our respondents. Ongoing staff training can be cost-prohibitive for non-profit agencies, but providing on-site workshops on relevant topics in a group format may be a way to facilitate professional development in a cost-sensitive manner. It also appears employers and/or employees are devoting time and money for conference attendance, which helps staff remain abreast of recent research and current professional standards.

Despite these beneficial practices, the frequency with which other performance management techniques are used is low. For example, only 40 % of respondents indicated their supervisor or another trainer observes them working, which suggests that a majority of respondents are not being observed while completing work tasks. One explanation for this finding is that a majority of our respondents had master’s or doctoral degrees and engage in work tasks that do not easily lend themselves to direct observation (e.g., work as a consultant or an administrator). Respondents may not have supervisors (e.g., self-employed, senior management) or work in settings that make it challenging to schedule supervisory observations (e.g., client’s home, residential setting). Because many of our respondents also reported they supervise other staff, it might be the case that these respondents regularly observe their subordinates who may not have received or responded to the survey invitation. Unfortunately, our survey did not inquire about the frequency of supervision provided by respondents. When supervisory observations are conducted, they appear to occur with a consistent frequency for at least half of the respondents who are observed (daily, weekly, and monthly). Fortunately, over 60 % of respondents receive ongoing performance feedback primarily from their supervisor but also from co-workers and the families of clients. Feedback is delivered in both verbal and written formats at regular frequencies (daily, weekly, and monthly).

Approximately three quarters of our respondents supervise staff. Our findings revealed a majority of supervisors oversee 15 or fewer staff indicating that “supervision loads” are manageable. Unfortunately, most of these individuals were not trained on effective supervision practices, which suggests that organizations are not adequately preparing their supervisors for this important responsibility. Only a handful of respondents indicated they were not at all prepared to supervise after receiving training. A large number of respondents reported they were somewhat prepared to function as a supervisor, which may be a function of the type of training they received. Unfortunately, we did not inquire about the training type or duration. Decades of research provides evidence that the behavior of supervisors influences the performance of staff (e.g., Parsons et al. 2012). If supervisors are not trained to perform crucial supervisory behaviors, staff performance may suffer and service quality could deteriorate. Moreover, employees who are not prepared to supervise others are more likely to experience burnout and work-related stress (e.g., Blache et al. 2011) increasing the likelihood of staff and supervisor turnover. In a recent survey, employees reported their relationship with their immediate supervisor was an important aspect of job satisfaction (Ray et al. 2013). Supervisors who lack relevant supervisory skills (e.g., providing corrective feedback in a respectful manner or at all) risk contributing to staff dissatisfaction and higher rates of staff turnover. The collective costs of staff turnover, recruiting and hiring new staff, and providing training are high and greater than the costs of preparing supervisors to function effectively. These data were collected before the new BACB® supervision standards that require BCBAs® who supervise BACB® aspirants to complete an 8-h training on effective supervision practices. Perhaps, the 8-h training will help facilitate effective supervision practices with BACB® aspirants as well as other supervisor–subordinate relationships.

These findings reveal some organizations are providing opportunities for employees to earn a variety of monetary and non-monetary incentives. Participants predominantly reported the availability of monetary incentives. Non-monetary incentives included time off, breakfast with select staff, preferred shifts, food, conference attendance, or other training. More importantly, the incentives are generally provided contingent on desired work performance. The frequency with which incentives were delivered varied from daily to annually; annual incentives occurred the most often. Although more frequent delivery of incentives is preferable, we were pleased to learn that many respondents may earn performance-based incentives within a given year.

Implications for Applied Settings

The present findings have important implications for individuals working in applied settings. First, it appears best practices with respect to implementing effective pre-service training are not uniformly adopted by organizations. Best practice training includes instruction, modeling, rehearsal, and feedback until mastery is demonstrated before staff work with clients. Perhaps a reliance on face-to-face verbal and written instruction is a function of the costs associated with other components of behavioral skills training procedures, namely, rehearsal and feedback. To address this issue, we recommend employers adopt a video-based training package containing didactic instruction, voice-over narration, video models of correct performance, and guided notes. Adopting video-based training allows the trainer to allocate face-to-face time to creating rehearsal opportunities and delivering feedback. There will be upfront costs associated with developing the training package, but the return on investment may be high. Organizations will need to consider how best to incorporate effective instructional practices (e.g., multiple exemplar training, developing guided notes). We also recommend organizations analyze the barriers to developing and implementing a best-practice training program and identifying cost-sensitive organizational systems to support doing so. One solution might involve the adoption of pyramidal training. This peer training approach has been used to teach a variety of interventionist behaviors including correct implementation of functional analysis (Pence et al. 2014), teaching procedures (Neef 1995), and preference assessments (Pence et al. 2012). If a pyramidal approach is adopted, we advocate direct teaching on effective training practices as well as close monitoring of peer trainer behaviors to ensure integrity and adherence to organizational policies and procedures.

Our findings also suggest performance management procedures are lacking. Best practices include regularly scheduled observations and feedback after working with clients. Not surprisingly, this aspect of training is expensive for non-profit or educational settings but is arguably one of the most, if not the most, important component. If these practices are not being regularly adopted by supervisors, we encourage organizations to determine if (a) expectations to complete these tasks are clearly communicated to supervisors, and (b) supervisors have been sufficiently trained to perform these tasks. An evidence-based supervisor training curriculum is available from the American Association on Intellectual and Developmental Disabilities for a low price of $195 (Green et al. 2010). The curriculum comes with PowerPoint slides, a trainer manual, activities, and quizzes making it ready-to-use with minimal cost and relatively few resources. This training will require personnel time but is otherwise an affordable solution. Another option is to purchase an easy-to-read and affordable book by the same authors of the curriculum (e.g., Reid et al. 2012). We also recommend supervisors receive frequent feedback from their supervisors about the extent to which they are delivering empirically supported supervision. If these techniques are unsuccessful, organizations may be required to conduct an organization-level functional assessment to identify barriers to the implementation of effective supervision practices. Adopting or adapting the Performance Diagnostic Checklist (Austin 2000) or the Performance Diagnostic Checklist-Human Services (Carr et al. 2013) could yield useful information to guide the development of organizational systems to facilitate desired practices.

Finally, we encourage the use of incentives with several caveats. Non-profit organizations or educational settings may face financial difficulties if monetary incentives are not budgeted carefully. The use of low- or no-cost items may help mediate budgetary restrictions. Because supervisors often err when making predictions about employee preferences for rewards (e.g., Wilder et al. 2011), we recommend employees provide input about the types of incentives used. Additionally, any incentive program should ensure the incentives are contingent on desired work behavior and available on a more frequent basis. We advocate that organizations apply behavior analytic principles when positively reinforcing the behavior of employees. The use of a performance-based pay system, such as Profit-Indexed Performance Pay (Abernathy 2011), is an alternative that allows organizations to provide incentives based on performance and current financial climate (i.e., profit). Thus, it addresses both contingency and magnitude. The amount of a monetary incentive is a function of four variables: performance, salary, an incentive basis calculated as a percentage of salary, and profit thereby transitioning from a traditional pay system (e.g., clock in and clock out, billable hours) to a performance-based pay system.

Limitations and Future Research

This study has a number of limitations worth noting. First, our sample contained a relatively higher proportion of respondents with graduate degrees suggesting it may not represent the experiences of direct-care staff many of whom do not seek board certification. We also did not inquire about respondents’ state of residence; thus, we do not know if we have a geographically representative sample. Next, this survey did not measure workplace performance. It is possible respondents performed at acceptable levels despite a lack of training. Respondents may also demonstrate less-acceptable performance despite receiving best-practice training. We are unable to determine the extent to which the reported training practices impacted workplace behavior. Finally, respondents may not have reported training procedures that were used because they were unfamiliar with the terminology or simply did not remember. These limitations should be addressed in future research.

Take-Home Points

• Employers are not consistently adopting best-practice pre-service or in-service training.

• To address this issue, employers could develop a video-based training package containing didactic instruction, voice-over narration, video models of correct performance, and guided notes. This package allows the trainer to allocate face-to-face time to creating rehearsal opportunities and delivering feedback.

• Another option is to make use of pyramidal or peer training. Close monitoring of peer trainer behaviors will be necessary to ensure integrity and adherence to organizational policies and procedures.

• Supervisors should be trained to deliver effective supervision. Two affordable options include (a) an evidence-based supervisor training curriculum available from the American Association on Intellectual and Developmental Disabilities (Green et al. 2010) and (b) an easy-to-read book by the same authors of the curriculum (e.g., Reid et al. 2012). Moreover, supervisors would benefit from frequent feedback from their supervisors about their adherence to best-practice supervision procedures.

• Employers are encouraged to assess employee preference for incentives, make them available contingent on desired work behavior, and apply behavior analytic principles when positively reinforcing the behavior of employees.