Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2007 May 14.
Published in final edited form as: Calif School Psychol. 2006;11:7–19. doi: 10.1007/bf03341111

The Role of the Autism Diagnostic Observation Schedule in the Assessment of Autism Spectrum Disorders in School and Community Settings

Natacha Akshoomoff 1, Christina Corsello 1, Heather Schmidt 1
PMCID: PMC1868476  NIHMSID: NIHMS19646  PMID: 17502922

Abstract

Autism diagnostic practices among school and clinical psychologists, particularly those using the Autism Diagnostic Observation Schedule (ADOS), were examined using national survey results (N = 132). School and clinical psychologists were similar in following the Best Practice Guidelines for screening, diagnosis and assessment, School psychologists were more likely to include a school or home observation and teacher report than clinical psychologists but evaluated significantly fewer children with autism spectrum disorders per year compared to clinical psychologists. School psychologists who were ADOS users were more likely to consider themselves autism experts and include a review of records than ADOS non-users. Perceived advantages of the ADOS included its strength in capturing ASD-specific behaviors and the standardized structure provided for observation, while diagnostic discrimination and required resources were the most commonly identified disadvantages.

The “autism spectrum disorders” (ASD) include the DSM-IV diagnoses of Autistic Disorder, Asperger’s Disorder, and Pervasive Developmental Disorder-Not Otherwise Specified (Lord & Bailey, 2002). According to DSM-IV criteria (American Psychiatric Association, 2000), Autistic Disorder is characterized by difficulties in the areas of communication, reciprocal social interaction, and restricted/repetitive behaviors and interests, with an onset prior to three years of age. While children with Asperger’s Disorder also have significant difficulties in reciprocal social interaction and exhibit restricted/repetitive behaviors and interests, they do not have cognitive delays or significant delays in early language development, and exhibit less severe communication problems. Children with Pervasive Developmental Disorder-Not Otherwise Specified (PDD-NOS) exhibit symptoms associated with Autistic Disorder but do not meet the full diagnostic criteria.

The prevalence of ASD has been estimated to be approximately 60 per 10,000 (Fombonne, 2003). Better identification, broader categorization, and the growth of available services have contributed to the increased number of children being identified with ASD and requiring specific educational interventions (Akshoomoff & Stahmer, 2006; Croen, Grether, Hoogstrate, & Selvin, 2002; Fombonne, 2003). While ASD is relatively rare compared to other developmental disorders, it is more common than previously thought and quite significant in terms of impact and resource needs.

With the introduction of IDEA in 1990 (P.L. 101–476), the “Autism” category and its definition was added to the previous list of disability categories and definitions. School psychologists and other school professionals have thus been placed in the role of identifying and assessing children for special education eligibility under this educational disability category. However, there are significant differences in how states define and assess children for eligibility (Conderman & Katsiyannis, 1996). The administrative prevalence of children in the “Autism” special education category varies across states, across school districts within a state, and appears to be significantly associated with education-related spending as well as availability of health-related resources (Mandell & Palmer, 2005; Palmer, Blanchard, Jean, & Mandell, 2005). Although it appears to be more common for children with Autistic Disorder to be identified during the preschool period, many children are first identified by their local school system, not their local health care system (Yeargin-Allsopp et al., 2003). Given that epidemiological studies frequently utilize special education administrative data to estimate the national prevalence of autism (Mandell & Palmer, 2005; Newschaffer, Falb, & Gurney, 2005; Shattuck, 2006), making the practices followed by school psychologists in the classification of ASD of particular interest to epidemiologists and policy makers.

There is a general consensus that appropriately targeted early intervention improves outcome for children with ASD (National Research Council, 2001). Increased diagnostic accuracy is therefore expected to have a greater impact on the success of individualized early intervention and education programs (National Research Council, 2001). It is recommended that a diagnostic evaluation for ASD should include a formal multidisciplinary evaluation of social behavior, language and nonverbal communication, adaptive behavior, motor skills, atypical behaviors, and cognitive status by a team of professionals experienced with ASD (Charman & Baird, 2002; Filipek et al., 2000; National Research Council, 2001; Shriver, Allen, & Matthews, 1999). With regard to specific diagnostic information, it is recommended that the diagnostic process include measures of parental report, child observation and interaction, and the use of clinical judgment (Filipek et al., 1999). The large number of behaviors that define ASD and the variability seen between individual children, even within the same diagnostic category, increases the likelihood that inaccurate decisions about diagnosis and classification may be made by those with less training and experience.

In 2002, the California Department of Developmental Services published the “Best Practice Guidelines for Screening, Diagnosis, and Assessment of Autistic Spectrum Disorders.” These guidelines were developed under the direction of the California State Legislature, which passed a bill in 2001 acknowledging the need for the use of diagnostic tools and methods to ensure consistent and accurate diagnosis of ASD throughout California. It is not clear how these guidelines have been adopted across the state at this point. For example, while specialty clinics, regional centers, and some school districts may have “resident ASD experts” who fit the recommendations regarding training and experience outlined in the Best Practice Guidelines, it is not known what proportion of children identified as eligible for school and state funded services are diagnosed by those with the recommended expertise. Data regarding the methods and accuracy of diagnostic and functional assessment for children with ASD being served by school and public agencies are lacking. Due to the limited availability of standardized tools, diagnostic interviews and observations are typically unstructured, and the format can vary according to theoretical orientation and training of the psychologist. The use of standardized diagnostic instruments among professionals and the relative advantages of these instruments are not known.

A number of standardized autism diagnostic instruments that rely on parent or teacher report are available, including the Gilliam Autism Rating Scale (GARS; Gilliam, 1995). These measures are appealing because they require limited training and time to administer. However, recent studies have reported that the GARS may miss as many as 58% of children with Autistic Disorder (Lecavalier, 2005; South et al., 2002). The Childhood Autism Rating Scale (CARS: Schopler, Reichler, & Rochen Renner. 1988) is a diagnostic measure that involves an experienced professional rating the child’s behavior after direct observation. Scoring this measure requires some training. However, the CARS is based on DSM-III-R criteria and tends to miss children diagnosed with PDD-NOS and over-identify children with mental retardation (Perry, Condillac, Freeman, Dunn-Geier, & Belair, 2005; Pilowsky, Yirmiya, Shulman, & Dover, 1998).

The Autism Diagnostic Observation Schedule (ADOS) is one of the few standardized diagnostic measures that involves scoring direct observations of the child’s interactions and that accounts for the developmental level and age of the child. The ADOS is recommended in several Best Practice Guidelines as an appropriate standardized diagnostic observation tool (California Department of Developmental Services, 2002; Filipek et al., 1999, 2000; National Research Council, 2001). It includes a standardized administration of interactive activities introduced by the examiner, designed to elicit social interactions, communication and repetitive behaviors for the purpose of diagnosing an ASD (Lord et al., 2000; Lord, Rutter, DiLavore, & Risi, 2001). The measure takes 30 to 60 minutes to administer and consists of four different modules for use with individuals of different developmental and language levels. Activities vary based on the language level and chronological age of the child. For example, Modules 1 and 2, which are designed for use with children with a language level of less than 48 months, include playing with bubbles, the release of an inflated balloon, and a pretend birthday party. Modules 3 and 4, which are designed for older children, adolescents, and adults who have the ability to use complex sentences and talk about things that are not immediately present, include questions about emotions and relationships as well as retelling a story from a book and demonstrating a routine activity. For each task, a hierarchy of “presses” or social structures is provided. During the first administration of a task a child is able to take as much initiative as possible; if this does not occur, the examiner gradually makes the tasks more specific and increasingly structures the situation to observe the child’s response (Lord & Risi, 2000; Lord et al., 2001) The ADOS is standardized in terms of the materials used, the activities presented, the examiner’s introduction of activities, the hierarchical sequence of social presses provided by the examiner, and the way behaviors are coded or scored. Following the administration of the ADOS, behaviors are coded using a 0- to 3-point coding system, with a 0 indicating that the behavior is not abnormal in the way specified in the coding description and a 3 indicating that a behavior is abnormal and interferes in some way with the child's functioning.

ADOS classifications are based on specific coded behaviors that are included in a scoring algorithm using the DSM-IV diagnostic criteria, resulting in a Communication score, a Reciprocal Social Interaction score, and a Total score (a sum of the Communication and Reciprocal Social Interactions scores). Scores are compared with an algorithm cut-off score for Autism or the more broadly defined ASD in each of these areas. If the child’s score meets or exceeds cut-offs in all three areas, they are considered to meet criteria for that classification on the measure. The authors reported good inter-rater reliability estimates on the Communication, Reciprocal Social Interaction, Total, and Stereotyped Behaviors and Restricted Interests domains, with intraclass correlations ranging from .82 to .93 (Lord et al., 2001). Test-retest reliability was also good, with intraclass correlations ranging from .73 to .82 on the Communication and Reciprocal Social Interaction domains, and .59 to .86 on the Stereotyped Behaviors and Restricted Interests domain. Published validity studies also suggest good predictive validity, with sensitivities ranging from 90% to 97%, and specificities ranging from 87% to 94% for autism/ASD versus other clinical diagnoses (Lord et al., 2001).

The ADOS has been widely used in research and academic centers for approximately 15 years to classify children with an ASD diagnosis for research studies and to assist in making clinical diagnoses. A five-day training workshop has been developed to train those using the ADOS in research studies to administer the ADOS and establish reliability in scoring, defined as 80% agreement or better on the protocol and algorithm. Supporting information from a developmental history, additional observational information or a detailed parent interview are needed for a comprehensive diagnosis (Lord et al., 2000, 2001). The approach used in the majority of research studies of ASD includes administration of the ADOS and the Autism Diagnostic Interview-Revised (ADI-R) by a trained professional, with the clinician using a best estimate approach to diagnosis. The ADI-R is a semi-structured interview that is conducted with the parents or primary caregiver and is designed to elicit the full range of information needed to verify a diagnosis of autistic disorder or an ASD diagnosis (Lord, Rutter, & Le Couteur, 1994; Rutter, Le Couteur, & Lord, 2003). Diagnostic decisions are based on algorithm items. Both the ADI-R and the ADOS operationally define current DSM-IV criteria, and quantify separately the three domains that define ASD: social reciprocity, communication, and restricted, repetitive behaviors and interests. This can be very helpful in increasing parents’ understanding of their children’s disabilities and setting goals.

Since the ADOS became commercially available through Western Psychological Services (WPS) in 2001, it has become more familiar to practitioners and purchased widely for use within school and community settings. The authors of the ADOS indicate that it should be used by experienced clinicians who have received appropriate training (Lord et al., 2001). In an effort to meet these criteria, WPS offers a two-day workshop, which includes live demonstration of two ADOS administrations, a demonstration videotape, and discussion of coding and administration issues. For individuals who are unable to attend the workshop or want additional training in ADOS administration and coding, WPS also sells a guidebook and training videotapes developed for professionals working in clinical settings.

While interest in the ADOS has increased, little is known about the use of the ADOS in community and school settings, practitioners’ opinions about the ADOS, and how they follow the training guidelines outlined by the test developers. These issues were examined in the present survey study. Two comparisons were of primary interest: (a) differences in ASD diagnostic practices and ADOS use among school and clinical psychologists who use the ADOS, and (b) differences in ASD diagnostic practices among school psychologists who do and do not use the ADOS.

METHOD

Participants

The participants were 44 clinical psychologists who reported that they use the ADOS and 88 school psychologists (44 ADOS users, 44 ADOS non-users). The majority of the respondents were female (77.3%) and white (87%). A chi square analysis revealed there were significantly more doctoral degrees among the clinical psychologists (100%) than the school psychologists (29.5%), χ2(1, N = 132) = 58.46, p < .001. The majority of the respondents were from California (52%). Many respondents did not provide contact information (32%). Twenty-one other states were represented, with the most surveys received from Texas (4%), Illinois (4%), Georgia (3%), Indiana (3%), and North Carolina (3%).

Pilot Survey

In collaboration with Bobbie Kohrt (former President of the California Association of School Psychologists (CASP)), a brief pilot survey was posted to the CASP members’ website in October 2003. Results obtained at the end of November 2003 were presented in the January 2004 CASP Today. The results of the CASP survey were used to develop additional questions to include a broader array of topics and to allow for more specific responses beyond the yes/no format.

Survey Development and Description

The survey was initially administered in the form of a telephone interview. Based on the participants’ responses, a small number of questions were modified to allow for more specific responses. The final written survey consisted of over 50 questions printed on 15 pages. Some suggestions from Dillman’s Total Design Method of mail and telephone surveying (Bernard, 2002) were utilized in an effort to improve the response rate. For example, the surveys were professionally printed on light green paper, a cover letter was included, and the recommended formatting techniques were used. It was estimated that the survey would take approximately 30 to 45 minutes to complete. A subset of the questions had restricted responses, while a number of more open-ended questions were included to allow for further qualitative analyses. Six restricted questions pertained to the respondent’s current position and training. In another section, a list of over 20 tools and standardized tests were provided and respondents were asked to indicate how often they administer these instruments (never, sometimes, most of the time, or always). In the analyses, “most of the time” and “always” were collapsed into one category and “sometimes” and “never” were collapsed into a second category. A series of 20 questions regarding training and experience, current work placement, and the diagnostic process were included with space provided for written responses.

Respondents were also asked if they used the ADOS and if they had received training on the ADOS. Those who responded yes were asked to complete an additional set of 16 questions, all of which required a written response. These questions focused on the nature and quality of training received, specific questions regarding opinions about the ADOS, and how the ADOS is used in the respondent’s placement. An identical section and list of questions were included regarding the use of the ADI-R.

Recruitment Procedures

Initial surveys were completed in the form of a telephone interview with 6 clinical psychologists and 11 school psychologists, recruited through local contacts. The majority of the written surveys were obtained through a mass mailing in collaboration with WPS (N = 84). Survey packets were mailed using a confidential list of individuals who had purchased the ADOS or ADI-R from WPS or who had attended an ADOS training workshop sponsored by WPS. A cover letter indicated that individuals interested in completing the survey should submit a signed consent form along with the completed survey in the stamped self-addressed envelope. Those individuals who agreed to further contact were asked to complete a contact information sheet. Respondents were informed that their responses would be kept confidential. The surveys were mailed out in August 2005. Responses received through December 2005 were included in the present study. In addition, 10 participants were attendees at local professional presentations on autism who were asked to volunteer to complete the survey, and 21 school psychologists volunteered to complete the survey prior to participating in a two-day ADOS workshop conducted by the first author. Of the 44 school psychologists who reported that they did not use the ADOS, 21 were from this group, 6 were recruited in response to the WPS mailing, 8 were telephone interviews, and 9 volunteered to complete the survey after attending local professional presentations on autism.

RESULTS

Assessment Practices Among School and Clinical Psychologists Who Use the ADOS

In considering the components included in an ASD assessment “always” or “most of the time,” 2 × 2 chi square analyses revealed there were no significant differences between clinical (N = 44) and school psychologists (N = 44) in the inclusion of a parent interview, χ2(1, N = 88) = 1.01, p = .32, a developmental history, χ2(1, N = 88) = 1.01, p = .32, or review of records, χ2(1, N = 82) = 1.95, p = .16 (see Figure 1). However, school psychologists were significantly more likely than clinical psychologists to indicate that they include a teacher questionnaire, χ2(1, N = 87) = 5.44, p = .02, interview with the teacher, χ2(1, N = 88) = 13.41, p < .001, or school or home observation, χ2 (1, N= 88) = 45.52, p < .001 (see Figure 1).

Figure 1.

Figure 1

Open in a new tab

Percentage of Survey Respondents in Each Group Rating Each Assessment Component as “Most of the Time” or “Always” Used

The majority of participants in both groups of ADOS users reported that they use standardized diagnostic measures in their assessment always or most of the time (clinical = 96%; school = 93%), A chi square analysis revealed no group differences, χ2(1, N = 88) = .35, p = .56. There were no differences between the groups in their use of the most frequently administered diagnostic measures: the ADOS, χ2(1, N =88)=.10, p = .75;theCARS, χ2(1, N = 88) = 1.14, p = .29;ortheGARS, χ2(1, N = 88) = 2,44, p = .12 (see Figure 2).

Figure 2.

Figure 2

Open in a new tab

Percentage of Survey Respondents in Each Group Rating Each Diagnostic Measure as “Most of the Time” or “Always” Used

In terms of experience, a t-test revealed that there was not a significant difference between the school (M = 10.56, SD = 7.55) and clinical psychologists (M = 11.23, SD = 7.81) in years worked in a setting where ASD assessments were conducted, t (86) = .41, p = .68. The number of initial evaluations conducted per year was compared between clinical and school psychologists. Two clinical psychologists reported completing more than 500 ASD assessments per year and were considered to be significant outliers, and therefore dropped from this analysis. Clinical psychologists reported conducting significantly more evaluations per year (M = 64.0, SD = 74.83) than school psychologists (M = 27.8, SD = 42.9), t (84) = 2.77, p < .007, Cohen’s d = .59). The percentage of clinical psychologists who completed 30 or more initial ASD assessments per year (50%) was higher than the school psychologists (only 15.9%).

Assessment Practices Among School Psychologists

The assessment practices of the 44 school psychologists who are ADOS users and the 44 school psychologists who are not ADOS users were compared. Among the ADOS users, 63.6% considered themselves ASD specialists whereas only 11.4% of the ADOS non-users considered themselves ASD specialists. A t-test revealed there was not a significant difference in terms of years worked in a setting where ASD assessments were conducted between the ADOS users (M =10.56, SD = 7.55) and the ADOS non-users (M = 9.53, SD = 8.69), t (86) = .59, p = .55. There was no significant difference between these two groups in terms of how many initial ASD evaluations were conducted per year (ADOS users: M = 27.80, SD = 42.9; ADOS non-users: M = 17.28, SD = 29.40; t (86) = 1.34, p = .18), with 88% of each group conducting 36 or fewer initial evaluations per year.

When considering components included in an ASD assessment, 100% of the ADOS users and ADOS non-users reported that they include a parent interview and a developmental history. Both groups were as likely to include an interview with the teacher, χ2(1, N = 87) = 1.04, p = .31, or a school or home observation, χ2(1, N = 85) = 2.2, p < .14). ADOS users were significantly more likely than ADOS non-users to include a review of records, χ2(1, N = 79) = 4.32, p = .04; see Figure 1.

The majority of both groups indicated that they included standardized measures in their assessment most of the time or always (ADOS users = 95%; ADOS non-users = 88%) and there were no differences between the groups, χ2(1, N = 87) = 1.47, p = .23. There were no significant differences between the groups in their use of the most frequently administered diagnostic measures: CARS, χ2(1, N = 86) =1.85, p = .17, or the GARS, χ2(1, N = 84) =1.17, p = .28. ADOS users were significantly more likely than ADOS non-users to use the ADI-R, χ2(1, N = 72) = 8.16, p = .004. Very few respondents reported using the PDD-ST, but this was significantly more common among ADOS non-users than ADOS users, χ2(1, N = 76) = 6.61, p = .01.

Training Among ADOS Users

Of the 88 clinical and school psychologists who indicated that they used the ADOS, the majority had attended a clinical training (clinical psychologists = 81.4%; school psychologists = 81.8%) or watched training videotapes (clinical psychologists= 88.6%; school psychologists = 90.9%). Only 16% of the sample reported that they had only watched the WPS training videotapes and only 2% reported that they had not attended a workshop or watched training videotapes. Very few of the respondents had attended research training (clinical psychologists = 22.7%; school psychologists = 6.8%). Only 7 of the 13 respondents who indicated that they had attended research training indicated that they had met the research reliability criteria on the ADOS. Almost all of the 13 respondents who attended a research training worked in a setting that also conducted research (N = 8), or trained students (N = 4).

Perceived Advantages and Disadvantages of the ADOS

The 88 survey respondents who use the ADOS were asked to answer questions about the advantages and disadvantages of the ADOS. Within this group, 92% (N = 81) listed one (N = 40) or more than one (N = 41) advantage of using the ADOS. This resulted in a total of 131 advantages that were grouped into 7 categories: resources, scoring/reliability, diagnostic discrimination, test materials/items/organization, capturing behaviors of ASD, administration, and other benefits. Chi-square analyses (p’s > .05) indicated that there were no significant differences between the number of school and clinical psychologists indicating advantages in any category, with the exception of diagnostic discrimination, χ2(1, N = 88) = 3.94, p = .05. For this reason the groups were collapsed.

The most common responses fell in the categories of capturing behaviors of ASD and administration. Of the 46 respondents (57%) listing advantages associated with capturing behaviors of ASD, 33% indicated that it captured ASD behaviors generally, 29% specifically indicated that it captured social and communication behaviors, and 20% indicated that it was a good measure for capturing behaviors that are difficult to observe or probe for in other situations. Examples of respondents’ responses to this question included: “It gets at the reciprocal social communication data and the ability to understand social nuances better than any test I’ve used before,” “Shows strengths and weakness of the child’s social interactions, pragmatic, and communication skills,” and “Captures social behavior.” Others indicated that the ADOS elicits behaviors that might not otherwise be observed. For example: “The ADOS does a nice job of pulling out characteristics of ASD that you wouldn’t necessarily see in a routine battery of tests,” “The ADOS provides specific presses in several ways to elicit specific behaviors that I may not otherwise remember to probe,” Of the 45 respondents (56%) that listed an advantage in the administration category, 60% specifically listed the standardization or the structure the measure provides, 20% listed the presses designed to elicit ASD behaviors, 16% listed the direct observation of the child, and 8% listed that the children enjoyed the ADOS. Responses to this question included: “Gives specific probes/presses to elicit responses which are often delayed/impaired in children with autism,” “It provides stimuli/presses which tend to bring out some of those behaviors associated with ASD that may not be obvious (or observed) under the more structured circumstances of a cognitive or educational assessment.”

Among ADOS users, 80% (N = 70) listed one (N = 43) or more (N = 27) disadvantages of using the ADOS, resulting in a total of 100 disadvantages that were grouped into 6 categories: resources, scoring/reliability, diagnostic discrimination, test materials/items/organization, scope of evaluation, and administration. As with advantages, chi square analyses did not indicate any significant differences between the number of school and clinical psychologists indicating disadvantages in any category (p’s > .05). For this reason the groups were collapsed.

The most common responses fell into the diagnostic discrimination (41%) and resources (37%) categories. Within the diagnostic discrimination category, the majority of respondents (59%) indicated that the ADOS tends to over classify other diagnostic groups as ASD. The second most commonly cited disadvantage was that the evaluator tends to rely on the tool for a diagnosis (28%). Approximately 10% of this group was concerned that the measure does not discriminate well within ASD subgroups. Of those that indicated resources as a disadvantage, almost all (96%) indicated time of administration as a disadvantage. Interestingly, 58% of this group indicated that they spend more than 13 hours on an evaluation. Some participants also indicated that expense (19%) was a disadvantage.

DISCUSSION

Among these select groups of clinical and school psychologists, ASD assessment practices were quite similar. The majority of the psychologists reported that they typically include a parent interview and a developmental history in their assessment. The most commonly endorsed standardized diagnostic measures were also similar for both groups. School psychologists were more likely to report that they include teacher input and a school or home observation, probably due to greater accessibility to teachers and students within the school setting. It is also possible that clinical psychologists may be more likely to evaluate very young children who are not yet in a school setting. For those children who are in a school setting, it is important to obtain information from the teacher, particularly if a school visit is not practical or not covered by insurance companies. Clinical psychologists reported that they evaluate more children with ASD for an initial evaluation per year than school psychologists. This may reflect relative differences in assessment practices across professions, or a bias in the survey respondents who indicated they are ADOS users.

The assessment practices were also fairly similar between the two groups of school psychologists, regardless of whether they are ADOS users. There was a tendency for those who used the ADOS to be more likely to review records, otherwise there were no differences in the components included in an assessment. The school psychologists who reported using the ADOS were more likely to consider themselves ASD experts, but they did not necessarily complete more initial ASD assessments per year than the ADOS non-users. It is, therefore, unclear what criteria the participants used in response to this question. Additional information is needed to determine if school districts utilize any specific criteria when selecting school psychologists to attend ADOS training and begin using the ADOS routinely for diagnostic evaluations. The nature of the ADOS requires routine use to remain efficient in administration and coding and therefore those practitioners who have limited opportunities to assess children with ASD may not necessarily be the best candidates to learn the ADOS and be identified as an ASD expert.

It is possible that the results of this study are not necessarily representative of school and clinical psychologists who conduct diagnostic evaluations for ASD. The majority of participants were recruited from the WPS mailing list, which represents those individuals who had purchased ADOS or ADI-R materials and/or had attended an ADOS clinical workshop. These individuals also volunteered to complete a rather lengthy written survey, which may indicate greater than expected experience with ASD, enthusiasm for the ADOS, or confidence in diagnostic expertise. A small portion (12%) of the data was obtained in the form of a telephone interview rather than a written survey. Although the responses obtained over the telephone did not appear qualitatively different from those obtained in the survey format, the results from this study may have been slightly different if all participants had agreed to a telephone interview and the data had been obtained in that format. Additional research with a broader sample of school and clinical psychologists is needed to gain an understanding of diagnostic practices in school and community settings. Independent validation of practice is also needed, in the form of record review and diagnostic confirmation of cases. For example, a recent examination of diagnostic records from 115 children diagnosed with ASD through the Metropolitan Developmental Disabilities Surveillance Program of the CDC found that professionals assigning an initial ASD diagnosis used a standardized diagnostic tool in only 30% of evaluations (Wiggins, Baio, & Rice, 2006). This stands in stark contrast to the practices reported in the present study.

The majority of ADOS users attended a clinical training on the measure. Very few respondents from either group attended the research training. The majority of respondents who had attended the research training were most likely involved in training and research as well as clinical work because they indicated that they worked in a setting that was actively involved in research and/or training of graduate students and post-doctoral fellows. At this point, little is known about the accuracy of administration and coding of those who have only attended a clinical training, which is currently the training recommendation for those using the measure clinically. This is an important issue, particularly given that the ADOS is commercially available but the two-day clinical workshop training is limited in availability and most practitioners and administrators assume attendance of a two-day clinical workshop will be adequate preparation for implementation of the ADOS. We are currently examining this more directly in a series of ADOS training studies in school and community settings.

The ADOS is one of the few standardized measures administered to a child by a trained professional that allows for direct observation of behaviors characteristic of an ASD. Of those who use the ADOS as part of their assessments, many consider the administration and the ability to capture behaviors of ASD to be advantages of the measure. The most commonly reported disadvantages of the measure was the tendency to over classify children who have other clinical disorders, the cost of the test kit, and the time it takes to administer the measure. As with any clinical assessment, the pros and cons and priorities must be considered when making a decision about which measures to include. Certainly, if time and money are issues, it is less expensive and takes less time to have parents complete a rating scale or questionnaire. Most test kits are fairly expensive but offer the advantage of providing the clinician with a way of directly assessing development and behavior. In contrast, questionnaires do not typically allow the clinician to directly observe and rate the child’s behavior, a frequently reported advantage of the ADOS. In addition, many questionnaires have resulted in low sensitivity or a tendency to miss children who have a clinical diagnosis of ASD. While diagnostic classification is also a concern for the ADOS, it is important to remember that this is a limitation of any standardized diagnostic measure. Any measure will result in missing some children with an ASD and including some children who do not have an ASD. Ultimately, the clinician makes a diagnosis, incorporating all relevant information from the assessment, including, but not limited to scores on a standardized measure. The goal is to choose a standardized measure that minimizes classification errors and is practical to use, given the training and time necessary to use it properly.

As expected, those who used the ADOS were also more likely to use the ADI-R, although use of the ADI-R was fairly rare and use of the ADI-R outside of research settings appears to be very limited. Additional research is needed to develop a parent interview or parent report measure that provides the reliability and validity of the ADI-R with a shorter administration time and training requirements.

Additional studies are needed to examine how best to assess the issue of “expertise” among clinicians who evaluate children for a possible ASD diagnosis, particularly because adequate experience and training are important components of the Best Practice Guidelines (California Department of Developmental Services, 2002) and for proper administration of the ADOS (Lord et al., 2001). It is important for school districts to provide training for all school psychologists on the Best Practice Guidelines for screening, diagnosis and assessment of ASD. School districts may also consider implementing common protocols for the assessment of ASD. Although there are no data currently available on the advantages of “resident ASD experts” within school districts, our experience indicates this can be helpful for proper identification as well as program planning for a variety of students with ASD. It is also our experience that there is better coordination of services and improved parental satisfaction when school districts have close connections with ASD experts and service providers in the community. It is important for clinical and school psychologists to provide parents with information about the potential differences between the DSM-IV categories of ASD and the “autism” educational disability category. More research is needed on how practitioners interpret the various diagnostic criteria and the impact of different practices and level of expertise on classification and service utilization.

Footnotes

Supported by NIMH K23MH071796. We thank Barbara Deggelman and Melissa Connolly for assistance with data coding, and Drs. Larry Palinkas, Catherine Lord, Amy Esler, and John Landsverk for their support of this project. Dr. Christopher Gruber assisted in survey distribution through Western Psychological Services. Bobbie Kohrt shared the CASP survey results.

References

  1. Akshoomoff NA, Stahmer A. Early intervention programs and policies for children with autistic spectrum disorders. In: Fitzgerald HE, Lester BM, Zuckerman B, editors. The crisis in youth mental health: Critical issues and effective programs. 1: Childhood Disorders. Westport, CT: Praeger; 2006. pp. 109–131. [Google Scholar]
  2. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. Washington, DC: author; 2000. text revision. [Google Scholar]
  3. Bernard HR. Research methods in anthropology. 3. Walnut Creek, CA: Alta Mira Press; 2002. [Google Scholar]
  4. California Department of Developmental Services. Autistic Spectrum Disorders: Best practice guidelines for screening, diagnosis and assessment. Sacramento, CA: author; 2002. [Google Scholar]
  5. Charman T, Baird G. Practitioner review: Diagnosis of autism spectrum disorder in 2- and 3-year-old children. Journal of Child Psychology & Psychiatry. 2002;43(3):289–305. doi: 10.1111/1469-7610.00022. [DOI] [PubMed] [Google Scholar]
  6. Conderman G, Katsiyannis A. State practices in serving individuals with autism. Focus on Autism & Other Developmental Disabilities. 1996;11:29–36. [Google Scholar]
  7. Croen LA, Grether JK, Hoogstrate J, Selvin S. The changing prevalence of autism in California. Journal of Autism and Developmental Disorders. 2002;32(3):207–215. doi: 10.1023/a:1015453830880. [DOI] [PubMed] [Google Scholar]
  8. Filipek PA, Accardo PJ, Ashwal S, Baranek GT, Cook EH, Jr, Dawson G, et al. Practice parameter: screening and diagnosis of autism: Report of the Quality Standards Subcommittee of the American Academy of Neurology and the Child Neurology Society. Neurology. 2000;55(4):468–479. doi: 10.1212/wnl.55.4.468. [DOI] [PubMed] [Google Scholar]
  9. Filipek PA, Accardo PJ, Baranek GT, Cook EH, Jr, Dawson G, Gordon B, et al. The screening and diagnosis of autistic spectrum disorders. Journal of Autism and Developmental Disorders. 1999;29(6):439–484. doi: 10.1023/a:1021943802493. [DOI] [PubMed] [Google Scholar]
  10. Fombonne E. The prevalence of autism. Journal of the American Medical Association. 2003;289(1):87–89. doi: 10.1001/jama.289.1.87. [DOI] [PubMed] [Google Scholar]
  11. Gilliam JE. Gilliam Autism Rating Scales. Austin, TX: PRO-ED; 1995. [Google Scholar]
  12. Lecavalier L. An evaluation of the Gilliam Autism Rating Scale. Journal of Autism and Developmental Disorders. 2005;35:795–805. doi: 10.1007/s10803-005-0025-6. [DOI] [PubMed] [Google Scholar]
  13. Lord C. Follow-up of two-year-olds referred for possible autism. Journal of Child Psychology and Psychiatry. 1995;36(8):1365–1382. doi: 10.1111/j.1469-7610.1995.tb01669.x. [DOI] [PubMed] [Google Scholar]
  14. Lord C, Bailey A. Autism Spectrum Disorders. In: Rutter M, Taylor E, editors. Child and adolescent psychiatry: Modern approaches. 4. Oxford: Blackwell Publications; 2002. pp. 636–663. [Google Scholar]
  15. Lord C, Risi S. Diagnosis of autism spectrum disorders in young children. In: Wetherby A, Prizant B, editors. Autism spectrum disorders: A transactional developmental perspective. Baltimore: Paul H. Brookes; 2000. pp. 167–190. [Google Scholar]
  16. Lord C, Risi S, Lambrecht L, Cook EH, Jr, Leventhal BL, DiLavore PC, et al. The Autism Diagnostic Observation Schedule—Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders. 2000;30(3):205–223. [PubMed] [Google Scholar]
  17. Lord C, Rutter M, DiLavore PC, Risi S. Autism Diagnostic Observation Schedule. Los Angeles: Western Psychological Services; 2001. [Google Scholar]
  18. Mandell DS, Palmer R. Differences among states in the identification of autistic spectrum disorders. Archives of Pediatric and Adolescent Medicine. 2005;159:266–269. doi: 10.1001/archpedi.159.3.266. [DOI] [PubMed] [Google Scholar]
  19. National Research Council. Educating children with autism. Washington, DC: National Academy Press; 2001. [Google Scholar]
  20. Newschaffer CJ, Falb MD, Gurney JG. National autism prevalence trends from United States special education data. Pediatrics. 2005;115:277–282. doi: 10.1542/peds.2004-1958. [DOI] [PubMed] [Google Scholar]
  21. Palmer RF, Blanchard S, Jean CR, Mandell DS. School district resources and identification of children with autistic disorder. American Journal of Public Health. 2005;95(1):125–130. doi: 10.2105/AJPH.2003.023077. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Perry A, Condillac R, Freeman N, Dunn-Geier J, Belair J. Multi-site study of the Childhood Autism Rating Scale (CARS) in five clinical groups of young children. Journal of Autism and Developmental Disorders. 2005;35:625–634. doi: 10.1007/s10803-005-0006-9. [DOI] [PubMed] [Google Scholar]
  23. Pilowsky T, Yirmiya N, Shulman C, Dover R. The Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: Differences between diagnostic systems and comparison between genders. Journal of Autism and Developmental Disorders. 1998;28:143–151. doi: 10.1023/a:1026092632466. [DOI] [PubMed] [Google Scholar]
  24. Rutter M, Le Couteur A, Lord C. ADI-R. Autism Diagnostic Interview-Revised, WPS Edition. Los Angeles: Western Psychological Services; 2003. [Google Scholar]
  25. Schopler E, Reichler RJ, Rochen Renner B. The Childhood Autism Rating Scale. Los Angeles: Western Psychological Services; 1988. [Google Scholar]
  26. Shattuck PT. The contribution of diagnostic substitution to the growing administrative prevalence of autism in US special education. Pediatrics. 2006;117:1028–1037. doi: 10.1542/peds.2005-1516. [DOI] [PubMed] [Google Scholar]
  27. Shriver MD, Allen KD, Matthews JR. Effective assessment of the shared and unique characteristics of children with autism. School Psychology Review. 1999;28(4):538–558. [Google Scholar]
  28. South M, Williams BJ, McMahon WM, Owley T, Filipek PA, Shernoff E, et al. Utility of the Gilliam Autism Rating Scale in research and clinical populations. Journal of Autism and Developmental Disorders. 2002;32(6):593–599. doi: 10.1023/a:1021211232023. [DOI] [PubMed] [Google Scholar]
  29. Wiggins LD, Baio J, Rice C. Examination of the time between first evaluation and first autism spectrum diagnosis in a population-based sample. Journal of Developmental and Behavioral Pediatrics. 2006;27:79–87. doi: 10.1097/00004703-200604002-00005. [DOI] [PubMed] [Google Scholar]
  30. Yeargin-Allsopp M, Rice C, Karapurkar T, Doernberg N, Boyle C, Murphy C. Prevalence of autism in a US metropolitan area. Journal of the American Medical Association. 2003;289:49–55. doi: 10.1001/jama.289.1.49. [DOI] [PubMed] [Google Scholar]

RESOURCES