Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jul 1.
Published in final edited form as: Autism. 2017 Apr 1;21(5):564–572. doi: 10.1177/1362361317698938

The diagnosis of autism and autism spectrum disorder in Low and Middle Income Countries: Experience from Jamaica

Maureen Samms-Vaughan 1,*, Mohammad Hossein Rahbar 2, Aisha S Dickerson 3, Katherine Loveland 4, Manouchehr Hessabi 3, Deborah Pearson 4, Jan Bressler 2, Sydonnie Shakespeare-Pellington 1, Megan L Grove 2, Charlene Coore-Desai 1, Jody Reece 1, Eric Boerwinkle 2
PMCID: PMC5935133  NIHMSID: NIHMS959480  PMID: 28367671

Abstract

The administration requirements of the ADOS and the ADI-R, widely used in High Income Countries (HIC), make them less feasible for diagnosis of autism spectrum disorder (ASD) in Low and Middle Income Countries (LMIC). The flexible administration requirements of the Childhood Autism Rating Scale (CARS) have resulted in its use in both HICs and LMICs. This study examines the agreement between assessments using the CARS with those using the ADOS or ADOS-2 and ADI-R in Jamaica. Children aged 2-8 years (n=149), diagnosed with autism by an experienced clinician using the CARS were re-evaluated using the ADOS and ADI-R. The proportion diagnosed with ASD using the ADOS, ADOS-2, and ADI-R was determined and mean domain scores compared using ANOVA. The mean age was 64.4 (SD=21.6) months; the male:female ratio was 6:1. The diagnostic agreement of the CARS with the ADOS and ADOS-2 was 100.0% and 98.7%, respectively. Agreement with the ADI-R was 94.6%. Domain scores were highest for children with more severe symptoms (p<0.01). Despite a high level of agreement of the CARS with the ADOS, ADOS-2, and ADI-R, the CARS should be evaluated further with a broader range of ASD symptomatology, and by clinicians with varying experience before recommendation for use in LMICs.

Keywords: Autism spectrum disorder, ADI-R, ADOS, CARS, Jamaica

1. Introduction

Autism Spectrum Disorder (ASD) is a lifelong neurodevelopmental disorder manifesting in early life, characterized by impairments in social communication and social interaction; and restrictive, repetitive patterns of behavior, interests or activities, as indicated in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (American Psychiatric Association 2013). The DSM-5 definition of Autism Spectrum Disorder encompasses and replaces DSM-IV-TR definitions of autistic disorder or autism, Asperger’s disorder and Pervasive Developmental Disorder-Not Otherwise Specified (PDD-NOS) (American Psychiatric Association 2000). ASD is a serious public health concern with a high prevalence and major familial and societal economic impact; currently 1 in 68 children is affected in the USA (Christensen et al. 2016). This growing problem has resulted in intense interest in the development of valid and reliable instruments to facilitate early diagnosis and treatment.

In 2002, two assessment instruments were recommended by the National Institutes of Health (NIH) autism working group: the Autism Diagnostic Interview-Revised (ADI-R) (Lord et al. 1997) and the Autism Diagnostic Observation Schedule (ADOS) (Lord et al. 2000). The ADI-R is a 93 item, standardized, semi-structured interview administered to parents, which generates scores in four areas: A (Social Impairment), B (Communication for Verbal or Non-Verbal subjects), C (Restricted, Repetitive, and Stereotyped Patterns of Behavior) and D (Abnormality of Development Evident at or Before 36 Months) (Lord, et al. 1997). The ADOS is a semi structured observational assessment comprising activities that elicit behaviors consistent with autism. The ADOS is scored using an algorithm based on two domains, Social Interaction and Communication (Lord et al. 2000). The ADOS has four modules, the module chosen for administration is based on the verbal level of the individual being assessed. The 2012 ADOS-2 revision includes a new Toddler module for children 12-30 months, updated administration guidelines, a new algorithm and new domains of Social Affect (Communication and Reciprocal Social Interaction) and Restricted and Repetitive Behaviors (Lord et al. 2012).

ADOS and ADOS-2 administration and scoring result in classification in one of three categories – Autism, or the milder autism spectrum classification, are assigned when an individual’s scores meet or exceed threshold cut-points in each domain as well as a summation of the two domain scores; non-spectrum is assigned when these criteria are not met (Lord et al. 2000). ADI-R administration and scoring result in classification in one of two categories: autism or non-autism (Lord et al. 1997). A classification of autism requires meeting or exceeding threshold cut-points in all domains.

The ADOS and ADI-R require substantial time and costs for training and certification, as well as administration. Administration times are 40-60 minutes and 90 to 150 minutes for the ADOS and ADI-R, respectively (Lord et al. 2012; Rutter et al. 2003). Test kits and booklets for administration to 100 children for either instrument cost over US$2,000. These instruments are recognized as the gold standards to facilitate diagnosis of ASD in High Income Countries (HICs).

Early diagnosis and treatment of ASD are also of concern to professionals in Low and Middle Income Countries (LMICs), particularly because there are limited resources to address the long term disability that can occur without early intervention. Access to diagnostic instruments has been identified as a contributing factor to the diagnostic delay of two to three years that occurs in LMICs (Samms-Vaughan 2014). The ADOS and ADI-R may not be feasible assessment instruments in LMIC because of high administration costs, few trained specialists, and large numbers of children awaiting assessment on long waiting lists. Nevertheless, it is important that ASD be diagnosed accurately in LMICs so that children with and without ASD can receive appropriate intervention.

The original Childhood Autism Rating Scale (CARS) (Schopler et al. 1980) is a diagnostic behavioral assessment scale with good psychometric properties (Schopler et al. 1988), for use in children 2 years and above. The CARS second edition (CARS 2) was developed to more accurately identify persons with higher cognitive functioning and has two separate forms, one for persons with estimated IQs equal to and below 79, and one for those with IQs equal to or above 80 (Schopler et al. 2010).

Both the CARS and CARS-2 have 15 domains: relating to people; imitation; emotional response; body use; object use; adaptation to change; visual response; listening response; taste, smell, and touch response; fear or nervousness; verbal communication; non-verbal communication; activity level; level and consistency of intellectual response and general impressions (Chlebowski et al. 2010). Historical reports and direct observations of children’s behavior are used to rate each domain on a scale from 1 to 4, with 1 indicating the behavior of typically developing children and 4 indicating the greatest impairment associated with autism (Schopler et al. 1980). A CARS total score of ≥ 30 is consistent with a DSM-IV diagnosis of autism; with 30 to 36.5 rated mild to moderate autism and ≥37 rated severe autism. Though the PDD-NOS classification was not included when the original CARS was developed, subsequent research identified the cutoff score to distinguish autism from the milder PDD-NOS to be 32 for 2 year olds and 30 for 4 year olds, while that to distinguish autism and PDD-NOS from other developmental disorders and typical development was 25 for both ages (Chlebowski et al. 2010).

The CARS can be administered by professionals with prior licensing, certification and experience in assessment of children and ASD; its administration and scoring time is 5 to 10 minutes (Schopler et al. 2010) following a typical history taking exercise. Test kits and booklets for administration to 100 children cost less than US$350.

The CARS has therefore become a tool of choice for both HICs and LMICs, including Saudi Arabia (Blaurock-Busch et al. 2012), Australia (Williams et al. 2013), Indonesia (Winarni et al. 2013), the Republic of Korea (Yim et al. 2013), and the USA (Geier et al. 2012). Recently the CARS-2 (Schopler et al. 2010) was chosen for use in a low resource setting in Tanzania (Harrison et al. 2014).

Previous studies have evaluated agreement between the CARS and the ADOS and ADI-R. In a study of 83 individuals 2-34 years old in Israel, agreement between the CARS and the ADI-R was 85.7% (Pilowsky et al. 1998). In a clinical sample of children 22-114 months in Iceland, the CARS identified 36 of 54 children with autism (66.7%), while the recommended ADI-R three behavior domain cutoff identified only 18 of the 36 (Saemundsen et al. 2003). Observed agreement between the CARS and ADI-R was 66.7% when the threshold was met or exceeded for all three behavior domains, but was 83.3% and 94.4% when the threshold was met or exceeded for any two or any one of the behavior domains, respectively. Children diagnosed by both the CARS and the ADI-R had more severe behavior symptoms and cognitive deficits (Saemundsen et al. 2003). In a clinical sample of young children 18-32 months in the USA, there was significant agreement between ADOS classifications and clinical judgment (κ = 0.593, p < 0.001), and between ADOS and CARS classifications (κ = 0.619, p < 0.001), but not between ADI-R classifications and clinical judgment (κ = 0.153, p=0.176) or between ADI-R and CARS classifications (κ = 0.095, p = 0.486) (Ventola et al. 2006). It was suggested that very young children may not readily manifest the repetitive and stereotyped behaviors that form one of the three behavior domains required for an autism classification by the ADI-R (Ventola et al. 2006). These studies have been primarily conducted in HICs.

The ease of administration of the CARS also resulted in its use at Jamaica’s main university-based referral center for the diagnosis of ASD; however, it has never been evaluated for its diagnostic accuracy or feasibility in Jamaica. The language of the ADI-R could potentially have terms that are culturally insensitive or difficult for some Jamaican parents to understand, and the specific social situations selected for use in the ADOS may not be culturally relevant. The purpose of this paper is to determine the accuracy and feasibility of the CARS for diagnosis of ASD in Jamaica, a LMIC, by examining the agreement between diagnostic classifications made by the CARS and those made by the ADI-R, ADOS, and ADOS-2. As this study was the first to objectively evaluate these instruments in Jamaica, and was not designed for the evaluation of cultural adaptation in Jamaica, only language changes were made, where culturally relevant, to ensure that the concepts being examined were understood by the Jamaican population. No changes were made to ADOS/ADOS-2 administration procedures. Translation and cultural adaptation of instruments often build on the experience obtained through prior use in a population, and involve data collection and analytical procedures specific to validation (Smith et al. 2016).

2. Methods

The Epidemiological Research on Autism in Jamaica (ERAJ) project investigated whether environmental exposures to heavy metals play a role in ASD through an age- and sex-matched case-control study that enrolled children between December 2009 and May 2012. The criteria for enrolment in ERAJ as a case were: Jamaican birth, 2-8 years of age at enrollment, and a clinical diagnosis of autism. The cases were recruited from the University of the West Indies’ (UWI) Jamaica Autism Database (JAD), and comprised a clinical sample of over 400 referred children, who were assessed as having autism by a developmental pediatrician with over 20 years’ experience, using the CARS. Some 261 children from the JAD met criteria for the ERAJ study, and were invited to participate. The children reported on in this paper were those identified as cases in the ERAJ study. Details regarding recruitment and screening have been reported (Rahbar et al. 2012; Rahbar et al. 2013). All participating parents provided written informed consent. The study was approved by the Institutional Review Boards of the University of Texas Health Science Center at Houston and the UWI.

The ADOS and the ADI-R were administered by a trained senior psychologist. Similar to the study by Ventola et al (2006), the psychologist was not blinded, and was aware of the experimental study being undertaken (Ventola et al. 2006). Because the ADOS test items have not changed with the ADOS-2, ADOS administration (modules 1 or 2) was used to score the algorithms for both the ADOS and ADOS-2 (Lord et al. 2000). Established cutoff points were utilized for the ADOS, ADOS-2 and ADI-R (Lord et al. 1997). Total ADI-R scores were calculated by summation of all four areas scored; previous researchers have used summed domain scores for correlation analysis (Pilowsky et al. 1998). The ADOS and ADOS-2 domain scores were also summed.

2.1. Statistical Analysis

Of 261 eligible children, 149 agreed to participate in the ERAJ study, resulting in a refusal rate of 42.9%. Refusals were primarily due to the blood samples required by ERAJ, the length of time required to complete assessments (4-5 hours), and the absence of an incentive as children already had a diagnosis. Descriptive analyses were conducted to assess the demographic characteristics of the 149 participating children. CARS scores were available for 143 children; clinical records were not located for 6 children. Demographic characteristics for the sample refusing participation were also determined.

We assessed the proportion of children for whom there was diagnostic agreement using the ADOS, ADOS-2 and ADI-R. Correlation coefficients were determined for total CARS and ADOS, ADOS-2 and ADI-R scores. Using Analysis of Variance (ANOVA), mean differences in CARS, ADI-R and ADOS scores of those children who were classified as having autism vs. no autism by the ADI-R and autism, autism spectrum or non-spectrum by the ADOS, and ADOS-2 algorithms were determined. All statistical tests were performed at a 5% level of significance using SAS version 9.4 (SAS Institute Inc. 2013).

3. Results

Of 149 children, 128 (85.9%) were male with a male: female ratio of approximately 6:1; 92.6% of children were Afro-Caribbean. Children’s ages ranged from 25 to 115 months with a mean of 64.4 (SD=21.6) months. Further demographic information on participants is given in Table 1. Respondents providing historical information that facilitated CARS diagnosis were 47.5% mothers, 6.8% fathers and 42.4% both parents. Respondents for the ADI-R were 77.2% mothers, 6.7% fathers and 13.4% both parents, with other relatives being the additional responders. The mean total CARS score was 36.9 (SD=3.7), with a median of 36.5 and a range of 30 to 48.5. Seventy-five children (52.4%) were classified as having mild to moderate autism with CARS scores of 30 to 36.5; 68 children (47.6%) had CARS scores of ≥ 37 and were classified as having severe autism. The mean age of the sample refusing participation was 68.4 (SD=19.2) months, the male: female ratio was 5:1, and the mean total CARS score, was 35.7 (SD=4.2).

Table 1.

Characteristics of children and their parents (N=149)

Variables Categories ASD Case
No. (%)
Child’s sex Male 128 (85.9)
Female 21 (14.1)
Age of child
(months)		 24 ≤ age < 47 43 (28.9)
48 ≤ age < 72 58 (38.9)
72 ≤ age < 107 48 (32.2)
Maternal educationa Up to high school 74 (49.7)
Beyond high school 75 (50.3)
Paternal educationa Up to high school 50 (34.2)
Beyond high school 96 (65.8)
Open in a new tab
a

at child’s birth

The mean time between CARS and ADOS/ADI-R assessments was 17.4 (SD 20.1) months with a range of 0 to 77 months. All 149 children (100%) assessed as having autism by the CARS were also assessed as having autism or autism spectrum by the ADOS, 98.0 % (146/149) were assessed as having autism or autism spectrum by the ADOS-2 and 94.6% (141/149) were assessed as having autism by the ADI-R. We found significant (p<0.005) but moderate correlation coefficients between the CARS and ADOS (r=0.34), ADOS-2 (r=0.42), and ADI-R total scores (r=0.35). The correlation coefficient was highest between the CARS and ADOS-2.

Of the ten children for whom there was no diagnostic agreement between the CARS and the ADI-R or ADOS-2, seven did not reach the threshold in at least two domains of the ADI-R, two did not reach the threshold for the ADOS-2, and one did not reach the threshold for both the ADOS-2 and ADI-R. The ADI-R domains that were most frequently below threshold were social impairment (n=8) and restrictive, repetitive behaviors (n=8). Four children did not reach threshold for verbal communication. Restrictive, repetitive, and stereotyped behavior was the only ADOS-2 domain below threshold.

All 141/149 (94.6%) children classified with autism by the ADI-R, were also classified as having autism or autism spectrum by the ADOS. On the ADOS-2, 139/141 (98.6%) were assessed as having autism or autism spectrum, with 133/139 (95.7%) having autism. Table 2 shows the ADOS, ADOS-2, and ADI-R classification of children diagnosed with autism on the CARS.

Table 2.

ADOS, ADOS-2 and ADI-R classification of children diagnosed with autism on the CARS (n = 149)

ADOS ADI-R
Autism
No Autism
Total
N % N %
ADOS
Autism 125 83.9 3 2.0 128
Autism Spectrum 16 10.7 5 3.4 21
Non-Spectrum 0 0 0 0 0
ADOS-2
Autism 133 89.3 6 4.0 139
Autism Spectrum 6 4.0 1 0.7 7
Non-Spectrum 2 1.3 1 0.7 3
Total 141 94.6 8 5.4 149
Open in a new tab

Mean scores for ADI-R, ADOS and ADOS-2 domains by ADOS and ADOS-2 classifications are displayed in Table 3. There were significant differences in mean ADI-R domain scores between children classified with autism and those classified with autism spectrum by the ADOS (Table 3). Using the ADOS-2 classification, significance was attained in the Social Impairment (p<0.01) and Non-Verbal Communication (p<0.01) ADI-R domains. Total ADI-R score, ADOS, and ADOS-2 domain scores were significantly higher for children classified as having autism by the ADOS, compared to those classified as having autism spectrum (p<0.01), and non-spectrum (p<0.01).

Table 3.

Comparison of ADI-R, ADOS and CARS mean total and domain scores by ADOS and ADOS-2 classification

Instruments & Domains ADOS
ADOS-2
Autism
(n=128)
Autism Spectrum
(n=21)
p Autism
(n=139)
Autism Spectrum
(n=7)
Non-Spectrum
(n=3)
p
M SD M SD M SD M SD M SD
ADI-R Domains
 Social Impairment 18.96 5.16 11.71 4.41 <0.01 18.29 5.51 15.00 5.26 8.67 3.05 <0.01
 Communication (Verbal) 5.02 7.47 11.48 4.54 <0.01 5.55 7.46 11.29 6.75 11.00 1.00 0.07
 Communication (Nonverbal) 8.15 6.10 0.33 1.53 <0.01 7.47 6.25 1.57 4.16 0 0 <0.01*
 Restrictive/Repetitive/Stereotyped Behavior 3.38 1.52 1.95 1.96 <0.01 3.22 1.61 2.71 1.38 2.33 4.04 0.50
 Abnormality Evident ≤ 36 mths. 4.39 0.81 3.76 1.14 <0.01 4.34 0.88 3.86 0.38 3.67 1.53 0.17
 ADIR Total 39.89 7.88 29.24 7.87 <0.01 38.86 8.49 34.43 9.64 25.67 4.73 0.02
ADOS Domains
 Communication 5.94 1.53 3.62 0.80 <0.01 5.71 1.64 4.57 1.72 3.67 0.58 0.02
 Social Interaction (SI) 10.06 1.94 5.86 1.06 <0.01 9.76 2.14 5.86 1.35 4.67 0.58 <0.01
 Communication + SI Total 15.98 2.49 9.48 1.08 <0.01 15.45 2.98 10.43 2.70 8.33 0.58 <0.01
 Play 2.79 1.07 1.14 0.73 <0.01 2.65 1.14 1.57 1.13 0.67 0.58 <0.01
 Stereotyped Behaviors/Restricted Interests 2.92 1.41 1.24 0.94 <0.01 2.81 1.43 1.43 0.79 0 0 <0.01*
ADOS-2 Domains
 Social Affect (SA) 14.95 2.56 8.71 1.79 <0.01 14.53 2.87 8.29 2.21 6.67 0.58 <0.01
 Restricted/Repetitive Behavior (RRB) 3.74 1.68 1.43 1.03 <0.01 3.58 1.74 1.57 0.98 0.33 0.57 <0.01
 SA + RB Total 18.70 3.42 10.14 2.52 <0.01 18.10 3.87 9.86 3.18 7.00 0 <0.01*
CARS Domains Autism
(n=123) 		No Autism
(n=20) 		p Autism
(n=134) 		Autism Spectrum
(n=6) 		Non-Spectrum
(n=3) 		p
M SD M SD M SD M SD M SD
 Relating to People 2.75 0.55 2.65 0.36 0.43 2.74 0.53 2.67 0.61 2.67 0.29 0.92
 Emotional Response 2.61 0.50 2.28 0.70 <0.01 2.58 0.57 2.33 0.75 2.33 0.58 0.50
 Imitation 2.75 0.61 2.45 0.76 <0.05 2.72 0.62 2.58 0.74 2.50 1.32 0.76
 Body Use 2.65 0.58 2.28 0.70 <0.01 2.63 0.57 2.58 0.49 1.00 0.00 <0.01
 Object Use 2.32 0.82 2.08 0.91 0.22 2.29 0.84 2.67 0.41 1.33 0.58 0.07
 Adaptation to Change 2.04 0.86 1.70 0.95 0.11 2.02 0.88 1.83 0.82 1.17 2.89 0.23
 Listening Response 2.23 0.70 2.15 0.71 0.63 2.24 0.69 1.58 0.66 2.50 0.50 0.06
 Taste/Smell/Touch Response 2.30 0.74 2.14 0.81 0.40 2.26 0.76 2.58 0.58 2.27 0.25 0.58
 Visual Response 2.80 0.43 2.65 0.61 0.17 2.80 0.45 2.33 0.61 3.00 0.00 <0.04
 Fear/Nervousness 1.62 0.79 1.62 0.76 0.97 1.62 0.78 1.67 0.88 1.50 0.87 0.96
 Verbal Communication 3.17 0.34 3.00 0.28 0.04 3.15 0.34 3.00 0.00 3.17 2.89 0.56
 Activity Level 2.33 0.68 2.52 0.57 0.22 2.36 0.67 2.42 0.49 2.00 1.00 0.64
 Non-verbal Communication 2.37 0.58 1.95 0.48 <0.01 2.34 0.58 1.83 0.26 1.83 0.29 <0.04
 Level/Consistency of Intellectual Response 2.40 0.42 3.52 4.36 <0.01 2.55 0.42 2.50 0.32 2.50 0.00 1.00
 General Impression 3.00 0.45 2.78 0.38 <0.05 2.97 0.45 2.75 0.27 3.00 0.00 0.48
 Total CARS Score 37.52 3.61 34.75 3.42 0.004 37.06 3.65 35.33 3.72 33.17 2.56 0.11
Open in a new tab
*

Since standard deviation in the Non-Spectrum (n=3) group is equal to zero, we calculated the p-value by comparing the means in the other two groups (i.e., Autism and Autism Spectrum groups)

Similarly, mean scores for ADI-R, ADOS, and ADOS-2 domains by ADI-R classifications of children having autism or no autism are presented in Table 4. On the ADI-R, mean domain scores were significantly higher for children classified with autism compared to those without autism with respect to social impairment (p<0.01), nonverbal communication (p=0.02), and repetitive behaviors (p<0.01). ADOS and ADOS-2 mean domain scores were higher for children classified as having autism by the ADI-R, compared to those who were not so classified. Most, but not all mean domain score differences achieved statistical significance. Mean total ADI-R, ADOS, and ADOS-2 scores were significantly higher for children classified as having autism by the ADI-R. (p<0.01).

Table 4.

Comparison of mean total and domain ADI-R, ADOS and ADOS-2 scores by ADI-R classification (n=149)

ADI-R
p
Autism
(n=141)
No Autism
(n=8)
M SD M SD
ADI-R Domains
 Social Impairment 18.55 5.16 7.25 1.67 <0.01
 Communication (Verbal) 5.96 7.62 5.25 4.33 0.79
 Communication (Non-Verbal) 7.34 6.31 1.88 3.48 0.02
 Restrictive/Repetitive/Stereotyped Behavior 3.29 1.62 1.13 0.64 <0.01
 Abnormality Evident ≤ 36 months 4.32 0.88 4.00 0.92 0.32
 ADIR Total 39.46 7.60 19.50 3.38 <0.01
ADOS Domains
 Communication 5.69 1.66 4.25 1.04 0.02
 Social Interaction (SI) 9.58 2.30 7.50 2.51 0.02
 Communication + SI Total 15.26 3.17 11.75 3.28 <0.01
 Play 2.60 1.17 1.88 0.99 0.09
 Stereotyped Behaviors/Restricted Interests 2.74 1.48 1.63 0.92 0.03
ADOS-2Domain
 Social Affect (SA) 14.21 3.22 11.63 3.62 0.03
 Restricted/Repetitive Behavior 3.51 1.79 1.75 1.04 <0.01
 SA + RB Total 17.72 4.36 13.38 4.37 <0.01
Open in a new tab

4. Discussion

All Jamaican children who were classified as having autism using the CARS were also classified as having autism or autism spectrum using the ADOS. Using the ADOS-2 algorithm, 98.0% were assessed as having autism or autism spectrum and using the ADI-R, 94.6% were assessed as having autism. As the CARS did not include the older DSM-IV category of PDD-NOS (considered to reflect the milder end of the autism spectrum) as a diagnostic category, our sample did not include children with milder symptoms. However, a strength of the Jamaican sample is that there is fairly equal representation of age categories (2-3, 4-5, 6-8 years), socioeconomic status as measured by levels of maternal education (up to high school, beyond high school), and autism severity as defined by the CARS (mild to moderate, severe).

Agreement of the CARS with other instruments, rather than concordance, was evaluated; assessment of concordance typically includes children with diagnoses of non-spectrum, autism spectrum and autism as on the ADOS/ADOS2. Sample selection likely explains the high agreement between the ADOS and ADI-R (94.6%), and between the ADOS-2 and the ADI-R (93.9%). In a clinical sample of 5-8 year old children with intellectual disability in the Netherlands, the agreement between the ADOS and the ADI-R was 83.4% for children with autism and 81.0% for ASD (PDD) (de Bildt et al. 2004).

Despite sample limitations, some of our findings follow patterns observed in other studies. More children were classified with autism or autism spectrum by the ADOS than with autism by the ADI-R. The study by Ventola et al. (2006) documented earlier indicated that the ADOS, CARS and clinical judgement were in agreement with each other for toddlers, but were not in agreement with the ADI-R (Ventola et al. 2006). Both the CARS cutoff of 30 and stringent ADI-R criteria (meeting threshold in all three behavior domains) allow only a diagnosis of autism, and exclude some children who would be classified as the milder autism spectrum or the previously used classification of PDD-NOS (de Bildt et al. 2004). The difference in agreement may also be explained by the ADI-R relying on parents’ recollection of behaviors of children, while the CARS integrates history and direct observation.

We also found that the ADI-R, ADOS, and ADOS-2 mean scores for many domains were higher for children classified as having autism or autism spectrum vs. no autism. These results are consistent with those reported by Saemundsen et al. (2003) who showed that ADI-R mean domain and total scores were higher for those classified as having autism than not having autism using the CARS and ADI-R (Saemundsen et al. 2003).

There are some specific features to the administration of the CARS, ADOS and ADI-R in Jamaica that may have impacted our results. In this study, the two domains of the ADI-R most frequently below threshold for diagnosis were the age and social interaction criteria. Notably, the majority of children not meeting the age threshold (80%) were more than 5 years old, and 60% were in the age range of just below 6 years to 8 years at ADI-R administration. The ADI-R has a specific focus on behavior at 4-5 years; a greater proportion of these items contribute to the social interaction domain. Jamaican parents may not have accurately recalled the age of first symptoms or social interaction symptoms at 4-5 years, given the age at which the ADI-R was administered. This finding has practical implications for the use of the ADI-R in Jamaica for older children, especially given the recognized delay between parental concern and diagnosis (Samms-Vaughan 2014).

Recent analysis from the Study to Explore Early Development (SEED) found that ADI-R and ADOS diagnoses had 77% agreement (438/584). SEED ASD criteria, which included ADOS criteria in association with more relaxed ADI-R criteria, increased the agreement to 86% (500/584), but diagnosed 62 fewer children than clinical judgement (Wiggins et al. 2015). The relaxed criteria excluded the age domain, and required either the social or communication domain to be met along with sub-threshold scores in the other domain or the behavior domain. Application of the SEED criteria would have resulted in 3 of the 10 children below threshold being classified as ASD.

Many children whose original autism diagnosis on the CARS was not in agreement with the ADOS and ADI-R had higher language functioning. ADOS and ADI-R questions display several language and cultural nuances specific to the United States or other developed countries. Research has shown reduced diagnosis of ASD in certain racial and ethnic groups (Mandell et al. 2009). Researchers have suggested methods for more culturally sensitive assessment for detection of ASD in racial and ethnic groups that experience disparities (El-Ghoroury and Krackow 2012). In our experience, the language and social situations included in the ADI-R sometimes had to be adjusted to be more culturally relevant for Jamaicans. For example, the additional questions enquiring about change in personal routine include switching from one pair of mittens to another or from winter to summer clothing. In the ADOS, Jamaican children of lower socioeconomic status (SES) who may have limited experience with birthday parties may have been adversely scored in that section. The CARS is less structured and more flexible and does not require children to be placed in specific social situations, but integrates history and observation of the child’s typical social experiences.

According to Robertson, the ADOS and ADI-R measure different manifestations of ASD symptomatology using differing methods; therefore, these instruments should be used together (Robertson et al. 1999). The recommendation for both instruments to be used may be problematic in Jamaica. Apart from cost, the time for administration of both may be a disincentive to parents; administration time was one of the reasons for reduced participation in the ERAJ study. Moreover, instruments are meant to facilitate diagnosis, but expert clinical observation is considered more detailed and thorough. This was supported by the SEED study, which reported that diagnostic instruments alone cannot replace informed clinical judgement in diagnosing ASD (Wiggins et al. 2015).

This study indicates that a classification of autism using the CARS, administered by an experienced professional, is highly substantiated by the ADOS and the ADI-R. In limited resource settings, as occurs in LMICs, there are typically few diagnostic centers operated by the few highly trained and experienced clinicians present in a country. This lack of resources precludes access to diagnosis, early intervention and treatment for the majority of children. Use of the CARS with its shorter training and administration time, lower costs, and fewer cultural concerns, may allow more professionals across LMICs to be trained in the diagnosis of ASD, and may facilitate early diagnosis and treatment.

Our study had some limitations. First, our sample was limited to children assessed with autism by the CARS, and did not include the full spectrum of symptomatology; findings and conclusions are therefore restricted to this group. Specifically those with milder symptomatology, who may have had CARS scores of 25-29.5, were not included, and the usefulness of the CARS for diagnosis of this group is unknown. The absence of a non-ASD comparison group also precluded determination of specificity and concordance across instruments. Additionally, the CARS was administered by a single highly experienced professional. Instrument-related limitations included the absence of scientifically, culturally-adapted CARS, ADOS and ADI-R instruments for the population. Culturally adapted instruments might have had different norms and cut-off points and influenced diagnostic classifications and the agreement reported (Norbury and Sparks 2013). Finally, there was a prolonged time between administration of the CARS and other instruments (mean of 17 months); responses to interventions received could have impacted scoring on other instruments. However, the CARS classification has been shown to be stable over time (Kleinman et al. 2008), and as with many LMICs, few children receive comprehensive intervention services in Jamaica.

6. Conclusions

In this study, we have shown that for a very high proportion of children, a diagnosis of autism on the CARS was in agreement with assessment by the ADOS and ADI-R. This provides preliminary support for the use of this easily administered, cost- and time-efficient instrument in LMIC settings. The use of the CARS by clinicians of varying levels of experience with ASD and with a broader range of ASD symptomatology, particularly milder symptoms, should be further evaluated before recommendations for its use in LMICs are made. If the ADOS and ADI-R are to be used in LMIC, culturally valid adjustments should be considered.

References

  1. AMERICAN PSYCHIATRIC ASSOCIATION. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Text Revision (DSM-IV-TR) Washington, DC: American Psychiatric Publishing, Inc; 2000. [Google Scholar]
  2. AMERICAN PSYCHIATRIC ASSOCIATION. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. Arlington, VA: American Psychiatric Publishing; 2013. [Google Scholar]
  3. BLAUROCK-BUSCH E, AMIN OR, DESSOKI HH, RABAH T. Toxic Metals and Essential Elements in Hair and Severity of Symptoms among Children with Autism. Maedica (Buchar) 2012;7(1):38–48. [PMC free article] [PubMed] [Google Scholar]
  4. CHLEBOWSKI C, GREEN JA, BARTON ML, FEIN D. Using the childhood autism rating scale to diagnose autism spectrum disorders. J Autism Dev Disord. 2010;40(7):787–799. doi: 10.1007/s10803-009-0926-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. CHRISTENSEN DL, BAIO J, VAN NAARDEN BK, BILDER D, CHARLES J, CONSTANTINO JN, DANIELS J, DURKIN MS, FITZGERALD RT, KURZIUS-SPENCER M, LEE LC, PETTYGROVE S, ROBINSON C, SCHULZ E, WELLS C, WINGATE MS, ZAHORODNY W, YEARGIN-ALLSOPP M, CENTERS FOR DISEASE CONTROL AND PREVENTION (CDC) Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years–Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2012. MMWR Surveill Summ. 2016;65(3):1–23. doi: 10.15585/mmwr.ss6503a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. DE BILDT A, SYTEMA S, KETELAARS C, KRAIJER D, MULDER E, VOLKMAR F, MINDERAA R. Interrelationship Between Autism Diagnostic Observation Schedule-Generic (ADOS-G), Autism Diagnostic Interview-Revised (ADI-R), and the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) Classification in Children and Adolescents with Mental Retardation. Journal of Autism and Developmental Disorders. 2004;34(2):129–137. doi: 10.1023/b:jadd.0000022604.22374.5f. [DOI] [PubMed] [Google Scholar]
  7. EL-GHOROURY NH, KRACKOW E. Enhancing the identification of autism spectrum disorders via a model of culturally sensitive childhood assessment. Professional Psychology: Research and Practice. 2012;43(3):249–255. [Google Scholar]
  8. GEIER DA, KERN JK, KING PG, SYKES LK, GEIER MR. Hair toxic metal concentrations and autism spectrum disorder severity in young children. Int J Environ Res Public Health. 2012;9(12):4486–4497. doi: 10.3390/ijerph9124486. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. HARRISON AJ, ZIMAK EH, SHEINKOPF SJ, MANJI KP, MORROW EM. Observation-centered approach to ASD assessment in Tanzania. Intellect Dev Disabil. 2014;52(5):330–347. doi: 10.1352/1934-9556-52.5.330. [DOI] [PubMed] [Google Scholar]
  10. KLEINMAN JM, ROBINS DL, VENTOLA PE, PANDEY J, BOORSTEIN HC, ESSER EL, WILSON LB, ROSENTHAL MA, SUTERA S, VERBALIS AD, BARTON M, HODGSON S, GREEN J, DUMONT-MATHIEU T, VOLKMAR F, CHAWARSKA K, KLIN A, FEIN D. The modified checklist for autism in toddlers: a follow-up study investigating the early detection of autism spectrum disorders. J Autism Dev Disord. 2008;38(5):827–839. doi: 10.1007/s10803-007-0450-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. LORD C, PICKLES A, MCLENNAN J, RUTTER M, BREGMAN J, FOLSTEIN S, FOMBONNE E, LEBOYER M, MINSHEW N. Diagnosing autism: analyses of data from the Autism Diagnostic Interview. J Autism Dev Disord. 1997;27(5):501–517. doi: 10.1023/a:1025873925661. [DOI] [PubMed] [Google Scholar]
  12. LORD C, RISI S, LAMBRECHT L, COOK EH, JR, LEVENTHAL BL, DILAVORE PC, PICKLES A, RUTTER M. The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord. 2000;30(3):205–223. [PubMed] [Google Scholar]
  13. LORD C, RUTTER M, DILAVORE PC, RISI S, GOTHAM K, BISHOP DV, LUYSTER RJ, GUTHRIE W. Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) 2012. [Google Scholar]
  14. MANDELL DS, WIGGINS LD, CARPENTER LA, DANIELS J, DIGUISEPPI C, DURKIN MS, GIARELLI E, MORRIER MJ, NICHOLAS JS, PINTO-MARTIN JA, SHATTUCK PT, THOMAS KC, YEARGIN-ALLSOPP M, KIRBY RS. Racial/ethnic disparities in the identification of children with autism spectrum disorders. Am J Public Health. 2009;99(3):493–498. doi: 10.2105/AJPH.2007.131243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. NORBURY CF, SPARKS A. Difference or disorder? Cultural issues in understanding neurodevelopmental disorders. Dev Psychol. 2013;49(1):45–58. doi: 10.1037/a0027446. [DOI] [PubMed] [Google Scholar]
  16. PILOWSKY T, YIRMIYA N, SHULMAN C, DOVER R. The Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: Differences Between Diagnostic Systems and Comparison Between Genders. Journal of Autism and Developmental Disorders. 1998;28(2):143–151. doi: 10.1023/a:1026092632466. [DOI] [PubMed] [Google Scholar]
  17. RAHBAR MH, SAMMS-VAUGHAN M, ARDJOMAND-HESSABI M, LOVELAND KA, DICKERSON AS, CHEN Z, BRESSLER J, SHAKESPEARE-PELLINGTON S, GROVE ML, BLOOM K, WIRTH J, PEARSON DA, BOERWINKLE E. The role of drinking water sources, consumption of vegetables and seafood in relation to blood arsenic concentrations of Jamaican children with and without Autism Spectrum Disorders. Sci Total Environ. 2012;433:362–370. doi: 10.1016/j.scitotenv.2012.06.085. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. RAHBAR MH, SAMMS-VAUGHAN M, LOVELAND KA, ARDJOMAND-HESSABI M, CHEN Z, BRESSLER J, SHAKESPEARE-PELLINGTON S, GROVE ML, BLOOM K, PEARSON DA, LALOR GC, BOERWINKLE E. Seafood Consumption and Blood Mercury Concentrations in Jamaican Children With and Without Autism Spectrum Disorders. Neurotox Res. 2013;23(1):22–38. doi: 10.1007/s12640-012-9321-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. ROBERTSON JM, TANGUAY PE, L’ECUYER S, SIMS A, WALTRIP C. Domains of social communication handicap in autism spectrum disorder. J Am Acad Child Adolesc Psychiatry. 1999;38(6):738–745. doi: 10.1097/00004583-199906000-00022. [DOI] [PubMed] [Google Scholar]
  20. RUTTER M, LE CA, LORD C. Autism Diagnostic Interview-Revised (ADI-R) Los Angeles, CA: Western Psychological Services; 2003. [Google Scholar]
  21. SAEMUNDSEN E, MAGNUSSON P, SMARI J, SIGURDARDOTTIR S. Autism Diagnostic Interview-Revised and the Childhood Autism Rating Scale: Convergence and Discrepancy in Diagnosing Autism. Journal of Autism and Developmental Disorders. 2003;33(3):319–328. doi: 10.1023/a:1024410702242. [DOI] [PubMed] [Google Scholar]
  22. SAMMS-VAUGHAN ME. The status of early identification and early intervention in autism spectrum disorders in lower- and middle-income countries. Int J Speech Lang Pathol. 2014;16(1):30–35. doi: 10.3109/17549507.2013.866271. [DOI] [PubMed] [Google Scholar]
  23. SAS INSTITUTE INC. SAS® 9.4. Cary, NC: SAS Institute Inc; 2013. [Google Scholar]
  24. SCHOPLER E, REICHLER RJ, ROCHEN RENNER B. The Childhood Autism Rating Scale. Los Angeles: Western Psychological Services; 1988. [Google Scholar]
  25. SCHOPLER E, VAN BOURGONDIEN M, WELLMAN J, LOVE S. Childhood Autism Rating Scale, Second edition (CARS2): Manual. Los Angeles: Western Psychological Services; 2010. [Google Scholar]
  26. SCHOPLER E, REICHLER R, DEVELLIS R, DALY K. Toward objective classification of childhood autism: Childhood Autism Rating Scale (CARS) Journal of Autism and Developmental Disorders. 1980;10(1):91–103. doi: 10.1007/BF02408436. [DOI] [PubMed] [Google Scholar]
  27. SMITH L, MALCOLM-SMITH S, DE VRIES PJ. Translation and cultural appropriateness of the Autism Diagnostic Observation Schedule-2 in Afrikaans. Autism. 2016 doi: 10.1177/1362361316648469. [Epub ahead of print] [DOI] [PubMed] [Google Scholar]
  28. VENTOLA P, KLEINMAN J, PANDEY J, BARTON M, ALLEN S, GREEN J, ROBINS D, FEIN D. Agreement Among Four Diagnostic Instruments for Autism Spectrum Disorders in Toddlers. Journal of Autism and Developmental Disorders. 2006;36(7):839–847. doi: 10.1007/s10803-006-0128-8. [DOI] [PubMed] [Google Scholar]
  29. WIGGINS LD, REYNOLDS A, RICE CE, MOODY EJ, BERNAL P, BLASKEY L, ROSENBERG SA, LEE LC, LEVY SE. Using standardized diagnostic instruments to classify children with autism in the study to explore early development. J Autism Dev Disord. 2015;45(5):1271–1280. doi: 10.1007/s10803-014-2287-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. WILLIAMS K, PERKINS D, WHEELER D, HAYEN A, BAYL V. Can questions about social interaction correctly identify preschool aged children with autism? J Paediatr Child Health. 2013;49(2):E167–E174. doi: 10.1111/jpc.12092. [DOI] [PubMed] [Google Scholar]
  31. WINARNI TI, UTARI A, MUNDHOFIR FE, HAGERMAN RJ, FARADZ SM. Fragile X Syndrome: Clinical, Cytogenetics and Molecular Screening among Autism Spectrum Disorder Children in Indonesia. Clin Genet. 2013;84(6):577–580. doi: 10.1111/cge.12095. [DOI] [PubMed] [Google Scholar]
  32. YIM SV, KIM SK, PARK HJ, JEON HS, JO BC, KANG WS, LEE SM, KIM JW, CHUNG JH. Assessment of the correlation between TIMP4 SNPs and schizophrenia and autism spectrum disorders. Mol Med Rep. 2013;7(2):489–494. doi: 10.3892/mmr.2012.1221. [DOI] [PubMed] [Google Scholar]

RESOURCES