Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Jun 17.
Published in final edited form as: J Early Interv. 2010 Sep;32(4):286–301. doi: 10.1177/1053815110382973

The Effects of Consultation on Individualized Education Program Outcomes for Young Children With Autism: The Collaborative Model for Promoting Competence and Success

Lisa A Ruble 1, Nancy J Dalrymple 2, John H McGrew 3
PMCID: PMC3117578  NIHMSID: NIHMS298174  PMID: 21691449

Abstract

The effects of a teacher consultation intervention were examined—namely, the collaborative model for promoting competence and success (COMPASS), which was designed to improve objectives of individualized education programs for children with autism. The intervention consists of an initial parent–teacher consultation, followed by four teacher consultations across the school year. Thirty-five teachers and a randomly selected child with autism (M age = 6.1 years) from each classroom participated. Compared to the nonintervention teacher–child dyads, the intervention teacher–child dyads showed improvements in individualized education program objectives, with a large effect size (d = 1.51).

Keywords: teacher consultation, autism, teacher training, individualized education programs, goal attainment scaling

Children with autism present complex instructional challenges for teachers (Scheuermann, Webber, Boutot, & Goodwin, 2003). The core impairments of autism—communication, social interaction and understanding, and restricted and narrowed interests (American Psychiatric Association, 2004)—influence areas of development and learning (Rogers & Vismara, 2008). To be effective, teachers should target critical developmental areas related to autism using evidence-based practices (National Research Council, 2001). In addition, effective teachers should possess the professional skills to work with other teachers, therapists, and parents to develop individualized education programs (IEPs) that include plans to generalize skills beyond the initial educational circumstances (Ingersoll & Dvortcsak, 2006; Koegel, Bimbela, & Schreibman, 1996). To date, researchers have not often studied the knowledge, training, and skills of special education teachers of children with autism; hence, it is likely that many teachers may not be well prepared to meet the specialized needs of young children with autism (National Research Council, 2001; Scheuermann et al., 2003).

One method to begin to address the training needs of teachers may be to examine teacher consultation. Consultation has the potential for affecting many teachers and, by extension, an even larger number of children. Researchers who have reviewed the teacher consultation literature have generally found teacher consultation effective (e.g., Busse, Kratochwill, T. R., & Elliott,, 1995; Medway & Updyke, 1985; Sheridan, Welch, & Orme, 1996). Models of teacher consultation have included behavioral (e.g., Kratochwill & Bergan, 1990; Noell et al., 2005), conjoint behavioral (e.g., Freer & Watson, 1999; McDougal, Nastasi, & Chafouleas, 2005; Sheridan & Steck, 1995; Sheridan, Clarke, Knoche, & Edwards, 2006; Sheridan, Eagle, Cowan, & Mickelson, 2001; Sladeczek, Elliott, Kratochwill, Robertson Mjaanes, & Stoiber, 2001; Wilkinson, 2005), collaborative (e.g., Denton, Hasbrouck, & Sekaquaptewa, 2003; Erchul, Hughes, Meyers, Hickman, & Braden, 1992; Givens Ogle, Christ, & Idol, 1991; Ray, Skinner, & Watson, 1999; Yocom & Staebler, 1996), and responsive systems consultation (e.g., Denton et al., 2003; Hughes, Hasbrouck, Serdahl, Heidgerken, & McHaney, 2001). Moreover, several problem areas have been successfully addressed using consultation approaches, including student achievement (e.g., Givens Ogle et al., 1991; Theodore et al., 2009), student disruptive behavior (e.g., Denton et al., 2003; McDougal et al., 2005; Ray et al., 1999; Sheridan et al., 2001; Sladeczek et al., 2001; Wilkinson, 2005), teacher behavior (e.g., Cossairt, Vance Hall, & Hopkins, 1973; Meyers, Freidman, & Gaughan, 1975; Noell et al., 2005; Sparks, 1988; White & Fine, 1976), and parent–teacher relationships (Sheridan et al., 2006; Sheridan & Kratochwill, 2007).

To be effective as a professional support for teachers in providing intervention for autism, consultants should employ a conceptual framework for assessing and identifying children’s needs and recommending interventions to address those needs that are firmly grounded in the intervention literature. For example, researchers have provided demonstrations of effective interventions for children with autism (Boyd, Odom, Humphreys, & Sam, 2010; Dawson & Osterling, 1997; Hurth, Shaw, Izeman, Whaley, & Rogers, 1999; National Research Council, 2001; Strain, Wolery, & Izeman, 1998). Although interventionists have not empirically identified specific causal mechanisms that account for positive changes in children with autism (cf. Kasari, 2002); several researchers have identified common recommendations: (a) Interventions should be implemented at young ages; (b) interventions should be individualized for children and their families; (c) interventions should be systematic and include progress monitoring; (d) interventions should focus on child engagement and foster children’s initiative and adaptation to transitions; (e) curricula should be developmentally based programming in the areas of imitation, communication, play, and socialization; (f) interventions should encourage families’ involvement and generalization of skills to other circumstances; and (g) interventions for children’s problem behaviors should be based on identifying the function of those responses while teaching appropriate replacement behaviors (Dawson & Osterling, 1997; Hurth et al., 1999; National Research Council, 2001; Strain et al., 1998). An important component of an intervention process for children with educational disabilities has been the generation of IEP goals and objectives (Office of Special Education and Rehabilitation Services, 2000). Given the significant developmental and learning needs of children with autism, their IEPs should also be based on recommended practices in autism (National Research Council, 2001).

The Collaborative Model for Promoting Competence and Success

The purpose of our study was to examine the effects of a collaborative teacher consultation and training model referred to as COMPASS—the collaborative model for promoting competence and success. COMPASS has several features described in other teacher consultation approaches—conjoint behavior consultation (Sheridan, Kratochwill, & Elliott, 1990) and instructional consultation (Rosenfield, 1987), for example—and focused on promoting (a) collaboration between school personnel and parents or caregivers when initially generating interventions, (b) linkage between assessment information and program plan development, (c) prevention of problem behaviors by placing emphasis on acquisition of functional skills and accompanying environmental supports, and (d) development of teaching strategies only after objectives are identified. COMPASS consultation was designed as a program-planning framework that addressed children’s individualized needs for environmental supports and intervention by their teachers. Influenced by mental health consultation (cf. Caplan, Caplan, & Erchul, 1994), COMPASS consultation enhances the likelihood for the generation of individualized teaching objectives and teaching plans. The COMPASS manual is available from the first author.

With COMPASS, we targeted three goal areas that have been identified as being critical for children with autism: social skills, communication, and independence. Moreover, in a recent study, Ruble, McGrew, Dalrymple, and Jung (2010) found that these three areas appeared to be overlooked in educational planning for some children with autism. We expected our consultation efforts to generate IEPs consistent with best practices in intervention with children with autism (cf. Individuals with Disabilities Education Act, 2004; National Research Council, 2001). Specifically, we anticipated that the resultant IEPs would be specific to the children; include social, communication, and independence objectives; and have clearly specified objectives that were observable and measurable. Our primary research question was as follows: Do teacher–child dyads who participate in the COMPASS consultation have better IEP goal attainment for targeted objectives than that of teacher–child dyads in nonintervention classrooms? We expected that improved IEP outcomes in the targeted areas would be associated with teachers’ fidelity of implementation of the COMPASS-generated teaching plans.

Method

Participants

Participants included 35 special education teachers, a randomly selected child with autism from each class (n = 35), and the selected child’s parents or caregivers (n = 35). To obtain teachers’ involvement, school systems were contacted at the district level, and special education directors were asked to participate. Once district-level permission was obtained, the school administrators provided the names of teachers of children with autism, and those teachers were contacted directly.

Participating teachers

Ninety-four percent of the participating teachers were women (n = 33). Teachers had an average class size of 12.8 children (SD = 7.9). Teachers’ mean number of years teaching was 11.0 (SD = 7.6), and their mean number of years teaching children with autism was 7.7 (SD = 7.6). Sixteen teachers had bachelor’s degrees; the remainder had master’s degrees. With the exception of one teacher who had an emergency certification, all teachers were certified. Ninety-four percent (n = 33) of the teachers reported that they had participated in some formal autism training, such as courses, supervised field experiences, workshops, and in-service trainings. Sixteen teacher–child dyads were recruited from urban schools in population centers of 300,000 or more; the remaining 19 teacher–child dyads represented rural or small city schools. Hence, we enrolled a sample of convenience, with solicited teachers either accepting or rejecting participation in our study.

Participating children with autism

Children qualified for the study if they received special education services and were designated by Individuals with Disabilities Education Act under the category of autism. They also had to meet the Diagnostic and Statistical Manual’s definition of autistic disorder (American Psychiatric Association, 2004). We employed one of two screening assessments before children’s enrollment—one for those children under 4 years old, the Modified Checklist for Autism in Toddlers (Robins, Fein, Barton, & Green, 2001); the other for those 4 years old and older, the Social Communication Questionnaire (Rutter, Bailey, & Lord, 2004). The Modified Checklist for Autism in Toddlers is a 23-item questionnaire used to quickly screen for the likelihood of a diagnosis of autism in children with a mental age of less than 2 years. The internal consistency (alpha) of the checklist is .85 (Robins et al., 2001). The Social Communication Questionnaire is composed of 40 yes/no questions assessing communication skills and social functioning in children who may have an autism spectrum disorder. The internal consistency (alpha) of the questionnaire ranges from .84 to .93 across age groups. We confirmed children’s diagnoses of autism with two additional measures. The Autism Diagnostic Observation Schedule–Generic is a semi-structured student interaction assessment tool (Lord et al., 2000), and the Autism Diagnostic Interview–Revised is a semistructured caregiver interview (Lord, Rutter, & Le Couteur, 1994). Each instrument has its own scoring algorithm for diagnosis based on Diagnostic and Statistical Manual criteria and provides cutoff scores in the domains of social reciprocity, language and communication, and restricted and repetitive behaviors. For the Autism Diagnostic Observation Schedule–Generic, interrater reliability using mean exact agreement for Modules 1 and 2 is 91.5% and 89%, respectively; the internal consistency (alpha) ranges from .91 to .94; and the test–retest reliability is .82 for the social–communication domain. For the Autism Diagnostic Interview–Revised, interrater reliability using percentage agreement ranges from .88 to .96 for all algorithm items; the internal consistency (alpha) ranges from .95 for social to .84 for communication and .69 for restricted and repetitive behaviors; and reliability over time indicated exact agreement that exceeds 83% for all but six items (Lord et al., 1994). Children enrolled in our study had to meet the diagnostic criteria for autism spectrum disorder for both scales to be included in the investigation.

If more than one child with autism was in the teacher’s classroom, the initials of all eligible children were collected, and a student was randomly selected for recruitment. After a child was randomly chosen, the teacher asked the child’s parent or caregiver for permission to be contacted by the researchers. If the parent or caregiver refused to participate, another child was randomly selected. Between August 2005 and July 2007, 79 teacher–child dyads were screened, with 4 not meeting eligibility requirements, 15 declining to participate, and 21 refusing for other reasons. Teachers and parents or caregivers provided informed consent to participate. The ages of the children with autism ranged between 3 and 8 years, with a mean of 6.1 years (SD = 1.7). The primary placement for educational services was as follows: special education for 15 children, general education for 8 children, inclusive preschool for 8 children, and segregated preschool for 4 children. Eighty-three percent of the children were male; 74% were White, 23% were Black, and 3% were biracial. Twenty-eight percent of families’ household income was less than $24,999; 36% fell between $25,000 and $49,999; and 36% were above $50,000. Eight caregivers did not provide their incomes.

Study Measures

Unless otherwise indicated, all teacher measures were given at the beginning of school and at the end of school year (i.e., Time 1 and Time 2, respectively). Most child measures, which were administered at the beginning of the study, were used to verify children’s autism and developmental status and to assess any group differences after random assignment to intervention or nonintervention teacher–child dyads.

Goal attainment scaling

Because the main outcome measure of the consultation intervention was progress toward IEP objectives, we employed an alternative, nonstandardized assessment system. Specifically, children’s progress on their three targeted IEP objectives, for both intervention and nonintervention dyads, was observed and rated with goal attainment scaling (GAS; Cytrynbaum, Ginath, Birdwell, & Brandt, 1979; Oren & Ogletree, 2000). GAS is an alternative evaluation technique for developing individualized, multivariable, scaled descriptions for outcome measures, and it has been used in several studies of consultation effectiveness (e.g., Sheridan et al., 2001; Sheridan et al., 2006; Sladeczek et al., 2001). GAS allows evaluators to assess process and outcome goals; it provides a link between intervention objectives and outcomes; and it is especially well suited for measuring outcomes when objectives are individualized, as with IEPs (Oren & Ogletree, 2000).

Before group assignment and as part of children’s initial school year evaluation, we collected their IEP goals and objectives that represented each of our three primary domains of interest for children with autism: a social objective, a communication objective, and an independence objective. If an objective addressing one of the three domains could not be identified within an IEP, then an appropriate behavioral objective was substituted for coding. To verify the accuracy of the classification of IEP objectives into learning domains, a two-step process was applied. First, all the IEP objectives were written on 3 × 5 cards and categorized into one of the domains by two independent raters. Second, the raters compared their results and reconciled differences. Raters agreed on the categorization of over 90% of the objectives after Step 1.

Before children were enrolled in the investigation, we developed a GAS template as a behavioral progress–monitoring form. For each objective, specific behavioral descriptors were developed delineating observed estimates of degrees of progress toward the objective. A 5-point scale was used: −2 = child’s present levels of performance, −1 = progress, 0 = expected level of outcome, 1 = somewhat more than expected, 2 = much more than expected. Thus, a score of zero represented improvement consistent with the actual description of the written IEP objective. All scores were standardized and converted to T scores (M = 50, SD = 10) using the Kiresuk–Sherman formula (Kiresuk, Smith, & Cardillo, 1994). Because many IEP goals and objectives lack adequate descriptions of criteria for success, we masked the effects of the COMPASS consultation from independent observers. To do this, we modified the objectives for the nonintervention dyads’ GAS forms to be observable and measurable.

We based all GAS ratings on direct observations rather than teachers’ reported ratings. Teachers were instructed to demonstrate for the GAS observer each of the three-targeted teaching objectives during a teaching and learning episode lasting about 20 minutes. For the nonintervention group, the initial GAS ratings were used as the baseline score at the beginning of the school year. For the COMPASS consultation group, the GAS ratings collected during the first teacher coaching consultation were used as the baseline score. At the end of the school year, we were unable to obtain final observations for one intervention dyad and two nonintervention dyads’, and those dyads’ GAS ratings were excluded from our final analysis.

Interrater agreement on GAS measures

We videotaped teacher–child learning episodes for subsequent interrater agreement analysis. The first author served as the primary observer and, later during the study, trained another researcher to serve as a primary observer. Primary observers coded videotapes for both baseline and postintervention. To assess interrater agreement, the first author trained secondary observers who were naïve to the assignment of teacher–child dyads to intervention and nonintervention groups. The secondary observers did not participate in any of the consultation and teacher coaching activities. For a random selection of 20% of the videotapes of the teacher–child dyads, a primary observer and a secondary observer independently scored the GAS measures for each of the three targeted objectives. Based on intraclass correlations, the interrater reliability of the observers’ ratings was .73 at baseline and .99 at the final assessment.

COMPASS Process Measures

We developed several process measures for use during the study. Specifically, we developed a non-COMPASS services measure, a COMPASS fidelity checklist, an adherence-to-teaching-plans measure, a COMPASS exposure measure, an IEP quality assessment, a COMPASS satisfaction survey, and a coaching feedback measure. These process measures are described briefly below and are available from the first author.

Non-COMPASS services measure

Children with autism may use a number of services outside the school systems (Ruble & McGrew, 2007). To ensure that the groups were similar in number of services used and number of service hours received during the intervention, at the end of the school year parents and caregivers completed a questionnaire on services received outside the school during the study, such as speech therapy, occupational therapy, physical therapy, individual therapy, applied behavior analysis, and medication or other biological therapies.

COMPASS fidelity checklist

To evaluate the extent to which critical aspects of the consultation were implemented, we developed a 25-item close-ended (yes/no) COMPASS fidelity checklist, which we administered to participants immediately following the consultation. The internal consistency was acceptable for the teacher and parent samples (alpha = .96 and .95, respectively).

Adherence-to-teaching-plans measure

Two consultants, the first and second authors, completed a teacher adherence rating immediately following consultations. The single-item adherence impression was rated with a 5-point Likert-type scale to assess an estimate of the degree to which the teacher was following the teaching plan recommendations for the year (1 = not at all or 0%, 2 = about 25%, 3 = about 50%, 4 = about 75%, 5 = very much or 100%). To assess interrater agreement, raters independently rated the adherence item immediately following 80% of the coaching sessions. The interrater agreement was .90 kappa.

COMPASS exposure measure

At the end of the study, we administered a COMPASS exposure measure to the teachers in the comparison group to assess whether they were exposed to the COMPASS principles. The exposure measure consisted of four items measuring knowledge of the COMPASS framework and changes in behavior based on this knowledge during the school year. Items were rated using a 5-point Likert-type scale (1 = strongly disagree, 5 = strongly agree). Higher scores indicate greater knowledge of and behavior consistent with COMPASS principles.

IEP quality assessment

Initial IEPs for all children were collected before randomization of teacher–child dyads. The IEPs for the intervention teacher–child dyads were collected after being updated following the COMPASS consult; IEPs were updated only for the objectives targeted by the COMPASS consultation. Across intervention and nonintervention groups, the IEPs averaged 3.9 goals and 14.8 objectives. We adapted a targeted IEP quality indicator assessment to provide a qualitative measure of IEPs. The indicator assessment was developed using standards from the Individuals with Disabilities Education Act (2004) and best practices from the National Research Council (2001; Ruble et al., 2010). For the current study, we employed the Targeted IEP Quality Indicator Scale, which includes six items explicitly related to COMPASS consultation. The items for the scale were rated with a 3-point Likert-type scale (0 = no/not at all, 1 = somewhat, 2 = yes/clearly evident). For the targeted IEP quality measure, three items focused on (a) the degree to which the objective was measurable in behavioral terms, (b) whether the conditions under which the behavior is expected to occur were well specified, and (c) whether the criterion for objective acquisition was explicitly described. The remaining three items assessed the degree to which communication, social, and independence objectives were present on the IEPs. An overall mean score was calculated for a maximum mean score of 2.0, and the mean item score across the three objectives was used. To assess interrater agreement for both IEP quality measures, a second coder rated 20% of the IEPs and an intraclass correlation of .79 was obtained. To help ensure objective ratings, the primary rater for IEP quality was not involved in the consultation or coaching sessions.

COMPASS satisfaction survey

To assess teacher satisfaction with the COMPASS consultation, a 25-item COMPASS satisfaction survey was developed and administered to teachers. Respondents used a 4-point Likert-type scale (1 = strongly disagree, 4 = strongly agree) to rate each item. Sample items include “I felt involved during the consultation and able to express my views,” “The consultant’s communication skills were effective,” and “The consultant was knowledgeable about autism.” Internal consistency (alpha) was .92 for teachers.

Coaching feedback measure

To assess the helpfulness of the COMPASS consultation, a 10-item coaching feedback form was administered to the teachers at the end of the school year. Each question was rated on a 4-point Likert-type scale (1 = strongly disagree, 4 = strongly agree). Eight items measured positive aspects of the coaching sessions (e.g., “Coaching supported you in helping the child reach his IEP objectives”; “in implementing strategies to reach three targeted objectives”; “in documenting progress”), and two items measured potential negative aspects of coaching (i.e., “How much did the coaching cause you stress?” and “How much did it interfere with your work?”). After the two negative items were recoded, internal consistency (alpha) for the total scale was .92. We administered the COMPASS satisfaction survey and coaching feedback measure only to the intervention teachers.

Research Design

We employed a randomized, single-blind, control group design for comparing intervention and nonintervention teacher–child dyads. In schools with more than one participating teacher, we controlled for the potential confounding of within–school site differences; that is, we used a stratified randomization procedure, with participating teachers randomized in pairs within schools. Seventeen teacher–child dyads were assigned to the nonintervention group and 18 to the intervention protocol. Hence, the nonintervention teacher–child dyads represented special education services at the participating schools. Nonintervention teacher–child dyads received no intervention from COMPASS personnel and were seen only at the beginning of the school year for baseline assessment and at the end of the year for follow-up evaluation.

Data Analysis

We performed descriptive and statistical analyses with SPSS 17.0. Following random assignment and to examine potential prestudy differences between the intervention and nonintervention teacher–child dyads, we collected and analyzed several child measures and two teacher measures. Specifically, we assessed children’s language with the Oral and Written Language Scales (Carrow-Woolfolk, 1995), their cognitive abilities with the Differential Abilities Scales (Elliott, 1990), their severity of autism with the Childhood Autism Rating Scale (Schopler, Reichler, DeVellis, & Daly, 1980), their adaptive behaviors with the Vineland Adaptive Behavior Scale (Sparrow, Balla, Cicchetti, & Doll, 1984), and their social skills with the Behavior Assessment System for Children–Second Edition (Reynolds & Kamphaus, 2004). The two teacher variables we compared were the number of children teachers taught in their class and the number of years they were in the teaching profession. We compared these measures with independent t tests. We also collected information on family income and interventions received outside school, which was obtained retrospectively at the end of the school year. We compared these measures for intervention and nonintervention groups with the Mann–Whitney U test.

For our primary research question concerning whether teacher–child dyads who participated in the COMPASS consultation demonstrated better goal attainment outcomes than those of nonintervention teacher–child dyads, we employed independent t tests. In addition, to assess the association of some process measures with GAS, we employed correlational analyses.

COMPASS Consultation Intervention

Initial consultation meeting

During the initial consultations, the first and second authors, who have 20 years experience in the field of autism and who have conducted school-based consultations, met with teachers and parents. The initial consultations consisted of one 2.5- to 3.0-hour meeting with parents and teachers within the first 1.5 months of the start of school. Before initial consultations, parents and teachers completed COMPASS consultation assessment forms, which we collected and consolidated into reports to review during the consultations. Consultations included the following steps: First, the general background for the COMPASS consultation was explained, including (a) the purpose of the meeting, (b) the role of consultants as facilitators instead of experts, (c) COMPASS and the philosophy that child outcomes are related to high-quality planning and implementation of teaching plans to achieve well-specified objectives, and (d) an overview of recommended practices in the field of autism. Second, teachers and parents’ concerns were identified with the information they provided and were based on the COMPASS consultation assessment process reports for children. Three goals related to the targeted areas were identified and prioritized. Concerns were then translated into specific IEP objectives, which were monitored throughout the school year. For each skill, detailed descriptions of the children’s present level of performance were generated, as well as a description of the desired levels of performance of the skill. These two criteria were then used to help develop the GAS that was used for outcome measurements. We wrote targeted skills in behavioral terms that were objective and measurable and consistent with recommended practices in writing educational goals and objectives (Office of Special Education and Rehabilitative Services, 2000).

Following development of objectives in the three targeted skill areas, consultants worked with teachers to develop teaching plans for those objectives. During the development of the teaching plans, team members focused on identifying environmental challenges and specific strategies to promote children’s skill acquisition. Examples included acquisition of knowledge about teaching well-specified and important social skills or developing and implementing visual schedules to enhance children’s classroom participation. During the last step of the initial consultations, we had parents and teachers report on how well the consultation adhered to the fidelity of COMPASS, as well as their satisfaction with the consultation. Following initial consultations, participating teachers and consultants met within 2 weeks to update the IEPs and to integrate the teaching objectives identified during the those consultations.

Teacher coaching

After consultations, we provided teachers with four 1.5-hour coaching and consultation visits (hereafter referred to as coaching visits). We scheduled coaching visits approximately every 6 weeks, with two in the fall and two in the spring. Consultants followed a written protocol of activities during each coaching visit: (a) directly observing and videotaping teacher–child dyads interactions focusing on the three targeted objectives, (b) coding the children’s levels of progress using the GAS, and (c) conducting teacher interviews following our written protocol. During the coaching visits, consultants provided feedback to teachers and, if necessary, modeled instructional behaviors or helped teachers adapt materials and activities. After consultations, we wrote 2- to 3-page summary reports that included descriptions of observations, information from teacher interviews, progress reports using GAS forms, and recommendations to be followed before the next coaching visits. We provided parents or caregivers and teachers with reports and the accompanying GAS forms within a week of school coaching visits.

Results

We found no significant differences between intervention and nonintervention teacher–child dyads at baseline for child or teacher variables (see Table 1). Similarly, with Mann–Whitney U test, we did not find between-group family income differences (U = 64.5, p = .11).

Table 1.

Between-Group Comparisons of Child and Teacher Characteristics

Characteristics Nonintervention
Intervention
t (df) p
M (SD) M (SD)
Children
 Age 5.98 (1.5) 6.18 (1.9) −0.34 (33) .74
 Childhood Autism Rating Scale 41.43 (8.2) 36.38 (9.9) 1.55 (30) .13
 Differential Abilities Scalea,b 39.47 (18.4) 53.78 (27.1) −1.81 (33) .08
 Oral and Written Language Scalesa,b 41.13 (19.0) 51.56 (17.2) −1.68 (32) .10
 Vineland Adaptive Behavior Scales (Teacher Report)a,b 62.29 (9.2) 64.88 (16.7) −0.56 (32) .58
 Behavior Assessment System for Children–2 (Teacher)a,c 59.53 (8.5) 59.83 (7.0) −0.11 (31) .91
Teachers
 Number of children taughtd 8.85 (11.5) 4.56 (6.1) 1.29 (27) .21
 Total years working with children with autisme 8.27 (8.3) 5.34 (5.5) 1.16 (29) .25
Open in a new tab
a

Standard score.

b

Based on externalizing composite.

c

t score.

d

Refers to total number throughout teaching career.

e

Refers to total number of students with autism taught.

Goal Attainment Scale Comparisons

Our primary research question was as follows: Do teacher–child dyads who participate in the COMPASS consultation have better IEP goal attainment for targeted objectives than that of teacher–child dyads in nonintervention classrooms? As we expected, the COMPASS intervention teachers had significantly higher goal attainment change scores (M = 31.9, SD = 15.1) compared to those of the nonintervention teachers (M = 12.47, SD = 11.1), t(30) = −4.1 p = .000, d = 1.5. The results based on the overall mean GAS raw scores were also similar and significantly higher for the COMPASS participants, t(27) = −2.6, p = .02, d = 1.0.

COMPASS Process Measures

Non-COMPASS services, COMPASS fidelity, teacher adherence, and nonintervention teacher exposure to COMPASS

We found no statistically significant differences between intervention and nonintervention groups’ mean rank for number (U = 90, p = .50) and hours of services (U = 98, p = .16). With respect to our COMPASS fidelity measure, intervention teachers reported that 96% of the COMPASS components were implemented by the consultants during their school visits. In addition, on a 5-point scale (with a rating of 5 being the best adherence), the average ratings of the teachers’ adherence to our consultation recommendations were 3.0 (SD = 2.0) for the first consultation, 3.5 (SD = 1.3) for the second, 4.0 (SD = 1.2) for the third, and 4.1 (SD = 1.2) for the fourth. Ratings of teachers’ adherence to our coaching recommendations positively correlated with GAS change scores from the start of the year until the end of school (r = .589, p = .013). Finally, at the end of the school year, on a 5-point scale (with 1 indicating no exposure to COMPASS), the nonintervention teachers’ mean score was 1.1 (SD = 0.20).

IEP quality comparisons

We anticipated that IEPs in the intervention teacher–child dyads would be of higher quality than in the nonintervention dyads within the areas targeted and expected to change because of the COMPASS consultation. As expected, after consultation, targeted IEP quality scores were higher in the intervention group (M = 1.41, SD = 0.24) compared to the nonintervention group (M = 1.05, SD = 0.58), t(28; equal variances not assumed) = −2.6, p = .02, d = .81.

Teacher satisfaction and coaching feedback

With respect to the COMPASS satisfaction survey, intervention teachers reported a mean satisfaction score of 3.7 (SD = 0.24) on a 4-point scale (with 4 being most satisfied). Nevertheless, they were less satisfied with the coaching aspects of COMPASS, averaging 3.2 (SD = 0.70) on a 4-point scale (with 4 being most satisfied) for the eight items related to helpful aspects of the coaching. With respect to the two unhelpful aspects of coaching items intervention, teachers’ mean score was 1.9 (SD = 0.74), indicating disagreement with the items.

Discussion

During the last several decades, researchers have identified effective educational procedures for young children with autism, but the dissemination and widespread application of those interventions in community-based settings remains an especially elusive goal (e.g., Boyd et al., 2010; Odom et al., 2003; Odom, Rogers, McDougle, Hume, & McGee, 2007; Rogers & Vismara, 2008). Moreover, given the recent and dramatic increases in the numbers of children with autism spectrum disorders (Safran, 2008) and the compelling need for special education teachers with specific training in autism, effective models for professional development and support for well-trained teachers working with children with autism continue to be evident (National Research Council, 2001; Odom et al., 2007)

This study was our systematic effort to evaluate one type of professional development with teachers of young children with autism—namely, consultation and coaching. Although educators have employed other collaborative consultation models (e.g., Brown, Horn, Heiser, & Odom, 1996; Peck, Killen, & Baumgart, 1989; for review, see File & Kontos, 1992), we chose to focus on and carefully evaluate our collaborative consultation efforts with GAS to assess the proximal effects with randomly assigned teacher–child dyads that included children with autism.

Our primary result with GAS and several accompanying findings with process measures are noteworthy. With respect to the primary dependent measure for GAS—children’s progress on three targeted IEP objectives—COMPASS produced clear and robust differences between intervention and nonintervention teacher–child dyads. Given the study procedures employed, we believe that our basic finding is promising evidence that well-targeted and collaborative consultation such as COMPASS may assist practitioners in providing high-quality services to young children with autism in community-based educational settings.

In addition to our primary finding with GAS, the results of our process measures are promising. Specifically, our measure of fidelity showed that consultants were able to implement COMPASS procedures with teachers in a well-prescribed manner. Similarly, our adherence measure indicated that the teachers typically followed collaborative consultation procedures and that their adherence improved with consultations across the school year. With respect to IEP quality, the IEPs of teachers who participated in COMPASS consultations were of higher quality than those of nonintervention teachers as indicated by best practice recommendations (cf. Individuals with Disabilities Education Act, 2004; National Research Council, 2001). Importantly, teachers reported satisfaction with COMPASS and its consultation and coaching components. Moreover, teacher satisfaction with COMPASS consultation and coaching has clear implications for higher practitioner acceptability and future use of procedures and interventions (cf. Eckert & Hintze, 2000; West, Brown, Grego, & Johnson, 2008).

Strengths and Weaknesses of Study

Similar to many educational studies, our investigation has both strengths and weaknesses. With respect to strengths, we carefully measured our primary variable of interest, GAS, across the school year. In addition, we assessed several important process measures that related to our primary measure—such as consultant fidelity to the model, teacher adherence to consultation, the qualitative nature of children’s IEPs, and teacher satisfaction with various components of COMPASS. Given the prestudy random assignment of teacher–child dyads within schools to COMPASS intervention and nonintervention comparison groups and the lack of between-group differences on important child assessment measures at the start of the year, we have confidence in our primary finding that COMPASS resulted in better outcomes in children’s three targeted objectives. Moreover, our end-of-the-year assessment of non-COMPASS services indicated that differences outside of school did not account for our findings.

Similar to many educational investigations, our study has some weaknesses that limit and qualify the findings. For example, our sampling of teachers and children with autism represents a sample of convenience, with solicited teachers and parents choosing to enroll or not enroll in our study protocol. This limitation is common for investigators in community-based settings and makes generalizations beyond the study sample difficult. Similarly, owing to resource constraints, our study was performed with a relatively modest sample of dyads, settings, targeted IEP objectives, and interventionists (i.e., two consultants with 20 years experience), and we lost three teacher–child dyads by the end of the school year. These factors also represent limitations to the generalizability of our primary finding. Nevertheless, the essence of scientific-based practices should be systematic replication, and we understand the clear and compelling need for the replication of COMPASS and similar consultation approaches with other children, teachers, IEP objectives, and consultants in other educational settings. Only then can researchers increase their confidence in consultation and coaching models for improving children’s proximal educational outcomes. Moreover, researchers in future large-scale studies and systematic lines of inquiry may be able to take professional development models to scale and better determine mediators and moderators of those types of interventions (for a description of a deployment-based model, see Weisz, Jensen, & McLeod, 2004).

Conclusions

A clear and compelling need exists for better quality IEPs for children with autism (cf. Ruble et al., 2010; Scheuermann et al., 2003). What continues to be much less clear is how best to improve children’s educational programs with enhanced professional development and supportive technical assistance that results in measurable and meaningful child outcomes. We designed our study to begin to address issues about effective and supportive professional development by assessing the effects COMPASS with teachers and the children with autism in their classrooms. Our preliminary results are promising and support the potential for collaborative consultation and coaching to make a difference in educational programming for young children with autism.

Acknowledgments

This research was supported by Grant No. R34MH073071 from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. We are grateful to the teachers, families, and children who generously donated their time and effort and to the special education directors and principals for allowing their teachers to participate. We want to acknowledge April Young for assistance with data collection, Melissa Wheatley for data coding and entry, and Grace Mathai and Alex Nounopolous for conducting the final evaluations and data coding.

Contributor Information

Lisa A. Ruble, University of Kentucky, Lexington

Nancy J. Dalrymple, Bloomington, Indiana

John H. McGrew, Indiana University–Purdue University at Indianapolis

References

  1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: American Psychiatric Association; 2004. text rev. [Google Scholar]
  2. Boyd BA, Odom SL, Humphreys BP, Sam AM. Infants and toddlers with autism spectrum disorder: Early identification and early intervention. Journal of Early Intervention. 2010;32(2):75–98. [Google Scholar]
  3. Brown WH, Horn EM, Heiser JG, Odom SL. Project BLEND: An inclusive model of early intervention services. Journal of Early Intervention. 1996;20:364–375. [Google Scholar]
  4. Busse RT, Kratochwill TR, Elliott SN. Meta-analysis for single-case consultation outcomes: Applications to research and practice. Journal of School Psychology. 1995;33(4):269–285. [Google Scholar]
  5. Caplan G, Caplan RB, Erchul WP. Caplanian mental health consultation: Historical background and current status. Consulting Psychology Journal: Practice and Research. 1994;46(4):2–12. [Google Scholar]
  6. Carrow-Woolfolk E. OWLS: Oral and Written Language Scales. Circle Pines, MN: American Guidance Service; 1995. [Google Scholar]
  7. Cossairt A, Vance Hall R, Hopkins BL. The effects of experimenter’s instructions, feedback, and praise on teacher praise and student attending behavior. Journal of Applied Behavior Analysis. 1973;6:89–100. doi: 10.1901/jaba.1973.6-89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cytrynbaum S, Ginath Y, Birdwell J, Brandt L. Goal attainment scaling: A critical review. Evaluation Quarterly. 1979;3(1):5–40. [Google Scholar]
  9. Dawson G, Osterling J. The effectiveness of early intervention. Baltimore: Brookes; 1997. [Google Scholar]
  10. Denton CA, Hasbrouck JE, Sekaquaptewa S. The consulting teacher: A descriptive study in responsive systems consultation. Journal of Educational and Psychological Consultation. 2003;14:41–73. [Google Scholar]
  11. Eckert TL, Hintze JM. Behavioral conceptions and applications of acceptability: Issues related to service delivery and research methodology. School Psychology Quarterly. 2000;15(2):123–148. [Google Scholar]
  12. Elliott CD. Differential Ability Scales. New York: Harcourt Brace Jovanovich; 1990. [Google Scholar]
  13. Erchul WP, Hughes JN, Meyers J, Hickman JA, Braden JP. Dyadic agreement concerning the consultation process and its relationship to outcome. Journal of Educational & Psychological Consultation. 1992;3(2):119. [Google Scholar]
  14. File N, Kontos S. Indirect service delivery through consultation: Review and implications for early intervention. Journal of Early Intervention. 1992;16:221–233. [Google Scholar]
  15. Freer P, Watson TS. A comparison of parent and teacher acceptability ratings of behavioral and conjoint behavioral consultation. School Psychology Review. 1999;28(4):672–684. [Google Scholar]
  16. Givens Ogle L, Christ BA, Idol L. Collaborative consultation: The San Juan Unified School District project. Journal of Educational and Psychological Consultation. 1991;2(3):267–284. [Google Scholar]
  17. Hughes JN, Hasbrouck JE, Serdahl E, Heidgerken A, McHaney L. Responsive systems consultation: A preliminary evaluation of implementation and outcomes. Journal of educational and Psychological Consultation. 2001;12(3):179–201. [Google Scholar]
  18. Hurth J, Shaw E, Izeman SG, Whaley K, Rogers SJ. Areas of agreement about effective practices among programs serving young children with autism spectrum disorders. Infants and Young Children. 1999;12(2):17–26. [Google Scholar]
  19. Individuals with Disabilities Education Act. Building the legacy: IDEA 2004. 2004 Retrieved from http://idea.ed.gov/
  20. Ingersoll B, Dvortcsak A. Including parent training in the early childhood special education curriculum for children with autism spectrum disorders. Topics in Early Childhood Special Education. 2006;26(3):179–187. [Google Scholar]
  21. Kasari C. Assessing change in early intervention programs for children with autism. Journal of Autism and Developmental Disorders. 2002;32(5):447–461. doi: 10.1023/a:1020546006971. [DOI] [PubMed] [Google Scholar]
  22. Kiresuk TJ, Smith A, Cardillo JE. Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ: Erlbaum; 1994. [Google Scholar]
  23. Koegel RL, Bimbela A, Schreibman L. Collateral effects of parent training on family interactions. Journal of Autism and Developmental Disorders. 1996;26(3):347–359. doi: 10.1007/BF02172479. [DOI] [PubMed] [Google Scholar]
  24. Kratochwill TR, Bergan JR. Behavioral consultation in applied settings: An individual guide. New York: Plenum; 1990. [Google Scholar]
  25. Lord C, Risi S, Lambrecht L, Cook EH, Jr, Leventhal BL, DiLavore PC, et al. The Autism Diagnostic Observation Schedule–Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders. 2000;30(3):205–223. [PubMed] [Google Scholar]
  26. Lord C, Rutter M, Le Couteur A. Autism Diagnostic Interview–Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal of Autism and Developmental Disorders. 1994;24(5):659–685. doi: 10.1007/BF02172145. [DOI] [PubMed] [Google Scholar]
  27. McDougal JL, Nastasi BK, Chafouleas SM. Bringing research into practice to intervene with young behaviorally challenging students in public school settings: Evaluation of the behavior consultation team (BCT) project. Psychology in the Schools. 2005;42(5):537–551. [Google Scholar]
  28. Medway FJ, Updyke JF. Meta-analysis of consultation outcome studies. American Journal of Community Psychology. 1985;13(5):489–505. doi: 10.1007/BF00923263. [DOI] [PubMed] [Google Scholar]
  29. Meyers J, Freidman MP, Gaughan EJ. The effects of consultee-centered consultation on teacher behavior. Psychology in the Schools. 1975;23:288–295. [Google Scholar]
  30. National Research Council. Educating children with autism. Washington, DC: National Academy Press; 2001. [Google Scholar]
  31. Noell GH, Witt JC, Slider NJ, Connell JE, Gatti SL, Williams KL, et al. Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review. 2005;34(1):87–106. [Google Scholar]
  32. Odom SL, Brown WH, Frey T, Karasu N, Smith-Carter L, Strain P. Evidence-based practices for young children with autism: Evidence from single-subject research design. Focus on Autism. 2003;18:176–181. [Google Scholar]
  33. Odom SL, Rogers S, McDougle CJ, Hume K, McGee G. Early intervention for children with autism spectrum disorder. In: Odom S, Horner R, Snell M, Blacher J, editors. Handbook of developmental disabilities. New York: Guilford Press; 2007. pp. 199–223. [Google Scholar]
  34. Office of Special Education and Rehabilitative Services. Guide to the individualized education program. Washington, DC: U.S. Department of Education; 2000. [Google Scholar]
  35. Oren T, Ogletree BT. Program evaluation in classrooms for students with autism: Student outcomes and program processes. Focus on Autism and Other Developmental Disabilities. 2000;15(3):170–175. [Google Scholar]
  36. Peck CA, Killen CC, Baumgart D. Increasing implementation of special education instruction in mainstream preschools: Direct and generalized effects of nondirective consultation. Journal of Applied Behavior Analysis. 1989;22:197–210. doi: 10.1901/jaba.1989.22-197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Ray KP, Skinner CH, Watson TS. Transferring stimulus control via momentum to increase compliance in a student with autism: A demonstration of collaborative consultation. School Psychology Review. 1999;28:622–628. [Google Scholar]
  38. Reynolds CR, Kamphaus RW. Behavior Assessment System for Children. 2. Circle Pines, MN: American Guidance Service; 2004. BASC-2. [Google Scholar]
  39. Robins DL, Fein D, Barton ML, Green JA. The modified checklist for autism in toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disorders. 2001;31(2):131–144. doi: 10.1023/a:1010738829569. [DOI] [PubMed] [Google Scholar]
  40. Rogers SJ, Vismara LA. Evidence-based comprehensive treatments for early autism. Journal of Clinical Child and Adolescent Psychology. 2008;37(1):8–38. doi: 10.1080/15374410701817808. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Rosenfield SA. Instructional consultation. Hillsdale, NJ: Erlbaum; 1987. [Google Scholar]
  42. Ruble LA, McGrew J. Community services outcomes in autism. Research in Autism Spectrum Disorders. 2007;1:306–372. [Google Scholar]
  43. Ruble LA, McGrew J, Dalrymple N, Jung L. Examining the quality of IEPs for young children with autism. Journal of Autism and Developmental Disorders. 2010 doi: 10.1007/s10803-010-1003-1. [advance online publication] [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Rutter M, Bailey A, Lord C. Social Communication Questionnaire. Los Angeles: Western Psychological Services; 2004. [Google Scholar]
  45. Safran SP. Why youngsters with autistic spectrum disorders remain underrepresented in special education. Remedial and Special Education. 2008;29(2):90–95. [Google Scholar]
  46. Scheuermann B, Webber J, Boutot EA, Goodwin M. Problems with personnel preparation in autism spectrum disorders. Focus on Autism and Other Developmental Disabilities. 2003;18(3):197–206. [Google Scholar]
  47. Schopler E, Reichler RJ, DeVellis RF, Daly K. Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS) Journal of Autism and Developmental Disorders. 1980;10(1):91–103. doi: 10.1007/BF02408436. [DOI] [PubMed] [Google Scholar]
  48. Sheridan SM, Steck M. Acceptability of conjoint behavioral consultation: A national survey of school psychologists. School Psychology Review. 1995;24(4):633–647. [Google Scholar]
  49. Sheridan SM, Clarke BL, Knoche LL, Edwards CP. The effects of conjoint behavioral consultation in early childhood settings. Early Education and Development. 2006;17(4):593–617. [Google Scholar]
  50. Sheridan SM, Eagle JW, Cowan RJ, Mickelson W. The effects of conjoint behavioral consultation results of a 4-year investigation. Journal of School Psychology. 2001;39(5):361–385. [Google Scholar]
  51. Sheridan SM, Kratochwill TR. Conjoint behavioral consultation: Promoting family–school connections and interventions. 2. New York: Springer; 2007. [Google Scholar]
  52. Sheridan SM, Kratochwill TR, Elliott SN. Behavioral consultation with parents and teachers: Delivering treatment for socially withdrawn children at home and school. School Psychology Review. 1990;19(1):33–52. [Google Scholar]
  53. Sheridan SM, Welch M, Orme SF. Is consultation effective? A review of outcome research. Remedial and Special Education. 1996;17(6):341–354. [Google Scholar]
  54. Sladeczek IE, Elliott SN, Kratochwill TR, Robertson Mjaanes S, Stoiber KC. Application of goal attainment scaling to a conjoint behavioral consultation case. Journal of Educational and Psychological Consultation. 2001;12(1):45–58. [Google Scholar]
  55. Sparks GM. Teachers’ attitudes toward change and subsequent improvements in classroom teaching. Journal of Educational Psychology. 1988;80(1):111–117. [Google Scholar]
  56. Sparrow SS, Balla DA, Cicchetti DV, Doll EA. Vineland adaptive behavior scales: Interview edition, expanded form manual. Circle Pines, MN: American Guidance Service; 1984. [Google Scholar]
  57. Strain PS, Wolery M, Izeman S. Considerations for administrators in the design of service options for young children with autism and their families. Young Exceptional Children. 1998 winter;:8–16. [Google Scholar]
  58. Theodore LA, Dioguardi RJ, Hughes TL, Aloiso D, Carlo M, Eccles D. A class-wide intervention for improving homework performance. Journal of Educational and Psychological Consultation. 2009;19:275–299. [Google Scholar]
  59. Weisz JR, Jensen AL, McLeod BD. Development and dissemination of child and adolescent psychotherapies: Milestones, methods, and a new deployment-focused model. In: Hibbs ED, Jensen PS, editors. Psychosocial treatments for child and adolescent disorders: Empirically-based approaches. 2. Washington, DC: American Psychological Association; 2004. pp. 9–39. [Google Scholar]
  60. West TN, Brown WH, Grego JM, Johnson R. Practitioners’ judgments of peer interaction interventions: A survey of DEC members. Journal of Early Intervention. 2008;30(1):36–54. [Google Scholar]
  61. White PL, Fine MJ. The effects of three school psychological consultation modes on selected teacher and pupil outcomes. Psychology in the Schools. 1976;13:414–420. [Google Scholar]
  62. Wilkinson LA. Supporting the inclusion of a student with Asperger’s syndrome: A case study using conjoint behavioral consultation and self-management. Educational Psychology in Practice. 2005;21:307–326. [Google Scholar]
  63. Yocom DJ, Staebler B. The impact of collaborative consultation on special education referral accuracy. Journal of Educational and Psychological Consultation. 1996;7:179–192. [Google Scholar]

RESOURCES