Further Evaluation of a Brief, Intensive Teacher-Training Model

Abstract

The purpose of this study was to further evaluate the outcomes of a model program that was designed to train current teachers of children with autism. Nine certified special education teachers participating in an intensive 5-day summer training program were taught a relatively large number of specific skills in two areas (preference assessment and direct teaching). The teachers met the mastery criteria for all of the skills during the summer training. Follow-up observations up to 6 months after training suggested that the skills generalized to their classrooms and were maintained for most teachers with brief feedback only.

Most teachers receive relatively little formal instruction in evidence-based practices for children with autism (National Research Council, 2001). Although the voluminous training literature has shown that parents and teachers can learn to implement behavioral interventions with a high degree of integrity (for reviews, see Noell, Duhon, Gatti, & Connell, 2002; Reid & Green, 1990), the limited time that is available for teachers to participate in continuing education is one key barrier to disseminating research findings.

Lerman, Vorndran, Addison, and Kuhn (2004) reported the outcomes of a brief summer training program for public school teachers of children with autism. Initial training in the areas of reinforcer identification, direct teaching, and incidental teaching was conducted across 5 days via lectures, discussion, and role play with the experimenters. Results showed that the 5 participants mastered a relatively large number of specific skills in a brief period of time and that the skills were maintained for several weeks following the termination of feedback.

Nonetheless, the study was limited in a number of respects. First, baseline data on teacher behavior were collected during role-play sessions with the experimenters rather than during teaching sessions with children, making it difficult to directly compare pre- and posttraining performance. Second, some skills acquired during role-play sessions did not immediately generalize to teaching sessions with children. Third, data were not collected in the teachers' classrooms following the summer training. Finally, a relatively labor-intensive data-collection system was used to monitor teacher performance. The success with which school administrators and personnel could implement this complex model is questionable. Efforts to transfer research findings on autism into public schools would benefit from further evaluation of efficient teacher-preparation models. In this study, we report the outcomes of a training model based on that described by Lerman et al. (2004). The original training was modified to increase the ease, efficiency, and effectiveness of the program, as well as to address some of the other limitations described above.

Method

Participants and Settings

A total of 18 certified special education public school teachers participated in three separate training groups (6 teachers from each of three school districts). They were selected by their school districts to participate after responding to an announcement about the training. All participants taught children with developmental disabilities and received stipends from their districts for attending. Of the 18 teachers, 9 were available for the pretraining and posttraining observations in their regular classrooms. The participants, aged 33 to 55 years, had been teaching from 1 to 14 years. (Contact the first author for further information about the participants.) All instruction took place in classrooms at schools selected by the three districts. The rooms contained desks, tables, chairs, and materials necessary to conduct the training.

Sixteen students who had been diagnosed with developmental disabilities and who were attending summer school at the three training sites participated. Their ages ranged from 3 years to 18 years, and basic learning skills varied from poor (i.e., did not imitate or follow simple instructions) to good (i.e., followed multiple, complex instructions). Nearly all of the children engaged in problem behavior during instruction (e.g., aggression).

Response Measurement and Interobserver Agreement

The dependent variable was the percentage of skill components performed correctly by the teacher in two main areas: preference assessment and direct teaching. More than one approach was taught in each area (i.e., three different ways to assess preferences, three different ways to fade prompts) so that the teachers could choose from among multiple procedures when working with students in their classrooms. The components scored for each skill, as shown in Table 1, were drawn from the literature on preference assessment (e.g., DeLeon & Iwata, 1996) and direct teaching (see Cooper, Heron, & Heward, 2007). (Contact the first author for a complete description of scoring criteria.) Skill components were evaluated on a session-by-session basis. Each skill component was scored as correct, incorrect, or not applicable (no opportunity) based on the teacher's performance during the entire session. Thus, a component was scored as correct only if it was performed correctly on each opportunity during the session. Each session consisted of a predetermined number of trials (see Table 1). The total number of correctly performed components was divided by the total possible components and multiplied by 100% for each session.

Table 1.

Procedural Components Evaluated and Session Lengths for Training on Preference Assessments and Direct Teaching

Preference assessment (4 trials for single stimulus and multiple stimulus; 6 trials for paired choice)

1. Presented item(s) correctly (based on format used).

2. Waited at least 5 s for student response.

3. Permitted 20-s access to item selected (or consumption, if food).

4. Immediately removed items(s) not selected (paired or multiple formats).

5. Blocked selection of multiple items (paired or multiple formats).

6. Encouraged sampling if no item selected; item(s) re-presented.

7. Collected data appropriately.

Direct teaching (10 trials)

1. Secured and organized materials prior to teaching.

2. Delivered instructions only when child attending.

3. Delivered clear, concise, and consistent instructions that specified expected response.

4. Used appropriate and consistent prompting strategy for no or incorrect responses.

The same prompting strategy was used correctly on each trial (based on prompting technique); successive prompts spaced by at least 5 s but no more than 15 s; each prompt more intrusive than previous one.

5. Delivered praise, token, food, drink, or tangible item within 5 s for correct responses.

6. Paired highly preferred tangible reinforcers with praise.

7. Delivered varied reinforcers (i.e., different praise statements, verbal and physical attention, or two types of tangible reinforcers).

8. Delivered praise or tangible item at least once for attending or on-task behavior.

9. Withheld reinforcement until desired behavior occurred. Reinforcement never given when child was off task.

10. Removed distractions immediately. If child began to interact with materials unrelated to the instruction, these materials were immediately removed.

11. Managed problem behavior appropriately. Said nothing about the behavior and delivered the most intrusive prompt possible within 5 s. New trial began within 5 s. No reinforcement delivered.

12. Inserted brief interval between instructional trials. Interval no more than 10 s; if tangible item delivered, interval from 10 s to 30 s.

13. Collected data appropriately. At least 80% of trials scored.

A second observer independently collected data during at least 30% of the sessions for each teacher and skill. For each skill component, the observers' records were compared to determine agreement on the occurrence of a correct response, an incorrect response, or no opportunity for a response. The number of agreements was divided by the total number of agreements plus disagreements, and the quotient was multiplied by 100% to obtain the percentage of interobserver agreement for each session. Mean interobserver agreement was 90% (range, 61% to 100%).

Procedure

The effects of the instruction on teacher behavior were evaluated in a multiple baseline across teachers design. Baselines were conducted concurrently. S1, S3, S4, S5, and S6 participated in the first training sessions; S2 and S4 participated in the second training sessions; and S7, S8, and S9 participated in the third training sessions.

Baseline

Observations were conducted in the teacher's classroom approximately 4 weeks prior to the end of the school year. The teacher was asked to select any student for the observations and to show how he or she (a) identified reinforcers for the student and (b) taught a skill to the student.

Training

Training on the 1st day and from 12:30 p.m. to 3:30 p.m. on the remaining 4 days consisted of lectures, discussion, and role play. A variety of topics were covered during this portion of the training (e.g., basic principles of learning, managing problem behavior). The afternoon training sessions focused on the specific skills that would be evaluated when the teachers worked with the students during the following morning sessions. The in-class training, which consisted of modeling and practice with feedback, was conducted from 8:30 a.m. to 11:30 a.m. on 4 consecutive days. The practice sessions focused individually on single-stimulus, paired-choice, and multiple-stimulus-without-replacement preference assessments and direct teaching using least-to-most prompting, most-to-least prompting, and delayed prompting. These six skill areas were taught in a sequential order. The criterion for successful mastery of each skill area was two consecutive sessions with 100% accuracy (or, for some teachers, one session at 100% for each preference assessment). However, the number of practice sessions received by each teacher depended on other factors, including time constraints and teacher requests. Teachers were paired with three different students each morning.

Follow-Up (postinstruction Observations)

Two to 3 months after the conclusion of the summer training, each teacher was observed while working with a student in his or her own classroom. The teacher was asked to conduct a preference assessment and to work with a student on a new skill during each observation. The teacher selected the student, task, materials, and format. Each teacher was observed on three separate occasions with approximately 4 weeks between each visit. One teacher received only two of the three scheduled observations because she resigned from her teaching position for reasons unrelated to participation in this study. Another teacher declined additional follow-up observations after receiving the first one because she said that she was too busy. Immediately following each observation, the experimenter provided brief feedback to the teacher by describing the skills components that had been performed correctly and incorrectly.

Results and Discussion

All but 1 of the teachers reported indirect methods for identifying reinforcers during baseline and showed rapid mastery of the three preference-assessment formats during training (Figure 1). Furthermore, performance generalized and was maintained at or near 100% accuracy when the teachers returned to their classrooms in the fall, with the exception of S1, who reported that she did not conduct preference assessments because her current students “work well with praise.” Teachers showed varying levels of accuracy with the direct teaching skills during baseline (Figure 2). All of the teachers met the mastery criterion when using at least one of the three prompting techniques; 4 teachers met the criterion with all prompting techniques. Six of the teachers performed with at least 80% accuracy during the first follow-up observation in the fall and generally maintained this level of performance across subsequent follow-up sessions (with the exception of S9, who received no further follow-up). Of the 3 remaining teachers (S1, S5, and S8), only 1 exceeded 80% correct at the second follow-up session.

Figure 1.

Percentage of correct procedural components for the single-stimulus, paired-choice, and multiple-stimulus preference assessments during baseline (BL), training, and follow-up for each teacher.

Figure 2.

Percentage of correct procedural components for direct teaching across the three prompting strategies during baseline (BL), training, and follow-up for each teacher.

These findings indicate that brief, intensive training designed to promote generalization across students and stimuli was moderately successful for public school teachers with varied backgrounds, training, and experience. The brief training was conducted outside the context of the teacher's own classroom and with a substantial delay until the start of the school year. Performance was generally maintained with brief feedback from 2 to 6 months following the conclusion of the training. A relatively conservative but economical measurement system also was used. Results of treatment acceptability surveys completed by the participants after the training indicated that they liked the procedures taught and felt that they were appropriate for use in their classrooms (survey and results are available from the first author).

Limitations of this analysis include the circumscribed nature of the follow-up observations and the varying number of practice sessions across teachers. In addition, data on maintenance were limited to the assessment format and prompting technique selected by the teacher during follow-up. The impact of the training on student behavior also was not evaluated. Further research is needed on the long-term outcomes of this brief training model, including those related to student performance, and on the application of this model to other skills (e.g., behavioral assessment).