Assessment of Severity of Autism Using the Indian Scale for Assessment of Autism

Abstract

Background:

The Indian Scale for Assessment of Autism (ISAA) was developed to assess the severity of autism among Indian cases.

Aims:

The present study evaluated the ISAA in relation to the Childhood Autism Rating Scale (CARS) and the Developmental Disability- Children Global Assessment Scale (DD-CGAS).

Materials and Methods:

Indian children with ICD 10 diagnoses of Autistic disorder (AD, n = 50), Intellectual Disability (ID, n = 50), Attention Deficit Hyperactivity Disorder (ADHD, n = 26), other psychiatric disorders (PD-N=25) and control children without psychiatric disorders (n = 65) were evaluated using the ISAA, DD-CGAS and the CARS (total n = 216).

Statistical Analyses:

In addition to descriptive statistics and correlation, analysis of variance (ANOVA) was used to assess whether the ISAA scores were significantly different across diagnostic groups.

Results:

Total ISAA scores were significantly higher among children diagnosed with autistic disorder compared to four other diagnostic groups. Total ISAA scores were significantly correlated with CARS scores and DD-CGAS scores. Groups sub-divided on the basis of recommended ISAA cutoff points of severity showed significant differences in CARS scores.

Conclusion:

The ISAA can thus be used to assess severity of AD among Indian children.

INTRODUCTION

It is estimated that there are approximately 1.7-2 million children with autistic disorder (AD) in India, thus AD is a significant public health concern.[1] In order to facilitate clinical care, it is helpful to estimate the level of severity of AD among individual cases. A number of standardised instruments have been developed to diagnose and assess the severity of autism, such as the Childhood Autism Rating Scale (CARS)[2], the Gilliam Autism Rating Scale (GARS)[3], the Autism Diagnostic Observation Schedule (ADOS)[4] and the Autism Disorder Inventory — Revised (ADI-R).[5] These scales can be used across different diagnostic classifications such as the International Classification of Diseases, ICD 10[6] and Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV).[7] They were designed and tested in developed countries and often require licensing fees if used in the clinical setting. In developing countries, instruments that are adapted to local social situations and are free for use would be beneficial.

Following recommendations from the National Trust of India (www.thenationaltrust.in), the Government of India directed the Ministry of Health and Family Welfare and Ministry of Social Justice and Empowerment to develop a scale that would be free for use and that could assess severity of disability. Such a scale was necessary as there was no official Indian yardstick to assess children with autism to enable their placement in special schools and to enable issuance of Disability Certificates under the Persons with Disability (PWD) Act. Accordingly, the Indian Scale for Assessment of Autism (ISAA)[8] was developed. An initial draft consisting of 437 items was produced by collaborators from several national institutions in consultation with experts. The list was reduced to 40 questions (six domains, with a 5 point scoring system) after analysis of redundancy, sensitivity, relevance and specificity through pilot studies. Clinical research associates from ten centers across India were next trained to use the ISAA and the Childhood Autism Rating Scale (CARS) during a two day workshop. During field tests, both the ISAA and the CARS were administered to three groups of children at each center in a specified time frame (autism, intellectual disability and other psychiatric disabilities, and children without psychiatric diagnoses, total N = 1247). Total scores as well as individual domain scores of the ISAA were analyzed in relation to CARS scores. The total ISAA scores differed significantly across the diagnostic groups following multivariate comparisons[8] manual of the ISAA attached). The criterion test validity of ISAA was significant in comparison with the CARS (r = 0.765, P < 0.001). Internal consistency and reliability (Cronbach's coefficient alpha) were significant and were comparable to CARS (Cronbach's alpha 0.932 P < 0.001). Each ISAA item was highly correlated with the total score and could discriminate between AD and the other diagnostic groups. Inter-rater reliability (r > 0.83) as well as test-retest reliability after three months were satisfactory in a sub-sample (r > 0.89). A cut off score of 70 showed high and balanced sensitivity and specificity between autism and the group without psychiatric diagnosis, as well as between autism and the MR group. Receiver Operator Curve (ROC) analysis confirmed the discriminant ability of ISAA (Area under the curve, AUC = 0.931, SE = 0.009 using the cut off score of 70). The total score on the ISAA is therefore used to estimate the level of severity of symptoms. The ISAA is now freely available through the National Trust of India (http://www.thenationaltrust.co.in/nt/index.php?option=com_content&task=view&id=30&Itemid=130). It was used in a child guidance clinic and was thought to be convenient for assessing severity, even among children with intellectual disability.[9] The present study extended these analyses. To enable additional evaluation, the ISAA was compared with the CARS[10,11,12] and the Developmental Disability-Children's Global Assessment Scale (DD-CGAS)[13] among children diagnosed with autism and children with or without selected psychiatric disorders.

MATERIALS AND METHODS

The study was approved by the Ethics Committee, Dr. RML Hospital, New Delhi. Written informed consent was obtained from parents of all participating children.

Participants

Children visiting the outpatients’ were evaluated in detail and assigned clinical diagnoses based on ICD 10. Potential participants were referred to the investigators by their therapists after the therapists discussed the study with the parents. The following groups were included: Autistic Disorder (AD); Intellectual Disability (ID); Attention Deficit Hyperactivity Disorder (ADHD); Other Psychiatric disorders (PD) and children without psychiatric disorders (controls). The AD group included children with varied levels of IQ scores. The ‘other psychiatric disorders’ (PD) group included Conduct disorder, Schizophrenia, Bipolar disorder, Depression, Oppositional Defiant Disorder (ODD) and Selective Mutism. Children with multiple psychiatric diagnoses were excluded.

Recruitment sites

Children were recruited from three sites, namely Postgraduate Institute of Medical Education and Research-Dr. Ram Manohar Lohia Hospital (PGIMER-RMLH), Delhi (all groups), the National Institute for Mentally Handicapped (NIMH) (ID group), and GB Pant Hospital, New Delhi (other diagnostic groups). Children without psychiatric disorders were recruited from the same residential neighborhoods as the children with AD or from neighborhoods with similar socio-economic status. Approximately 40% of the AD group had participated in the initial validation studies of the ISAA described in the introduction above.

Rating scales

Indian Scale for Assessment of Autism (ISAA)[8]: The ISAA is a 40 item scale divided into six domains- Social Relationship and Reciprocity (9 questions); Emotional Responsiveness (5 questions); Speech — Language and Communication (9 questions); Behavior Patterns (7 questions); Sensory Aspects (6 questions) and Cognitive Component (4 questions). The scores for the each item of ISAA range from 1-5, depending on the intensity, frequency and duration of a particular behavior with the following anchors: score 1 = Rarely (up to 20%), score 2 = Sometimes (21-40%), score 3 = Frequently (41-60%), score 4 = Mostly (61-80%), and score 5 = Always (81%-100%). Scoring is based on information from parents and observation of the child following guidelines from the Manual of the ISAA. In the speech- language and communication domain the child should be rated 5 if he/she never developed speech or communication. Total ISAA scores range from 40-200. The lowest score represents no symptoms or symptoms which were present only rarely, and the maximum score indicates the most severe presentation of AD. The following categories are recommended; mild AD: 70-107, moderate AD: 108-153, severe AD: 153.

CARS[2]: The CARS can reliably differentiate among children with mild-to-moderate and severe autism and can distinguish autism from mental retardation.[14] It is applicable to children of all age groups and incorporates 15 items. Scores are based on direct observation[11], and items are scored from 1-4. A score of 1 indicates age appropriate behavior and a score of 4 indicates severely abnormal behavior. All items contribute equally to the total score that varies from 15 to 60. The CARS was selected in the present study as it was the scale originally used to validate ISAA, and also because it is widely used in India.

Children's Global Assessment Scale (DD-CGAS): The Children's Global Assessment Scale (CGAS) is a modification of the Global Assessment Scale (GAS) for adults.[15] Scores range from 1 to 100, where 1 represents the most impaired functioning while 100 indicates superior functioning. The rating reflects typical functioning of the child during a particular time period and is a global rating based on all available sources of information and across all domains of functioning (self care, communication, social behavior, and academic functioning).

Statistical analysis

In addition to descriptive statistics, analysis of variance (ANOVA) was used to assess whether the ISAA scores were significantly different across diagnostic groups. Pearson's correlation was used to calculate correlation coefficient (r) between scales, ISAA. The Statistical Package for Social Sciences (SPSS) version 16.0 was used for all analyses. Power estimates were based on G-power software.[16]

RESULTS

Demographic features: [Table 1]. Male children were over represented across all groups. Using post-hoc Bonferroni corrected analysis, the mean age of the children were significantly different among all three groups (F = 15.506, P = 0.0001). The parental income and education for children with AD was significantly higher than the other groups (F = 8.989, p = 0.000001; education F = 18.22, p = 7.3 × 10^-13).

Table 1.

Demographic and clinical features of the sample

Clinical features: Total scores for the ISAA were significantly different across groups (F = 313.5, P-value < 0.0001, ANOVA). The post hoc Bonferroni test indicated that the mean scores for the Autism group were significantly higher than all the other groups [Table 1].

Correlations between the ISAA total scores and ISAA component scores were evaluated in relation to the CARS and the DD-CGAS scores among the children with AD [Table 2]. Significant correlations were observed between the ISAA total scores and the CARS scores (r = 0.730, P < 0.001), between the ISAA total score and DD-CGAS score (r = –0.625, P < 0.001) and between the CARS and DD-CGAS score (r= –0.85, P < 0.001) (the negative correlations are consistent with the scoring schemes). Each of the six domains of the ISAA was significantly correlated with CARS total score except the cognitive component domain. Out of the six ISAA domain scores, three domains were significantly correlated with DD-CGAS scores, namely social relationship, emotional responsiveness and behavior patterns.

Table 2.

Correlations of ISAA with CARS and DD-CGAS among children with AD

Based on ISAA recommendations, the participants were divided into children with severe AD (ISAA scores >153) moderate AD (ISAA scores: 106-153), mild (ISAA scores: 70-106) and comparison groups without AD (ISAA scores <70). Mean CARS scores were significantly different between these groups (ANOVA, F = 561.312, df 3,218, P = 0.0001, Figure 1).

Figure 1.

Comparison of CARS scores among participants classified by ISAA scores

Power analysis: The sample has over 80% power to detect 0.1 small effect sizes for group-wise differences and 72% power to detect 0.3 medium effect size differences across groups.

DISCUSSION

Awareness about disability certification for ID and AD is relatively low in India.[17] There is also a dearth of tools to assess the severity of AD.[18] The ISAA was designed to assess the severity of functional impairment in AD to facilitate disability evaluation and certification in Indian children. It was field tested across India among children between the ages 3-20 years. ISAA scores are based on functionally relevant domains, facilitating a more fine-grained analysis than other rating scales that provide overall scores. The present study suggests that ISAA total scores discriminate between AD and four other diagnostic groups, with AD children receiving the highest scores. ISAA total scores were significantly correlated with individual ISAA domains. Groups sub-divided on the basis of recommended ISAA cutoff points of severity showed significant differences in CARS scores, further supporting its use for assessing the severity of specific domains of autism among Indian children.

There was substantial correlation between the CARS and ISAA total scores (r = 0.73), but the cutoff points for the CARS may differ for Indian children compared to those recommended originally for children in the USA or Europe.[19] In the present study, the average CARS score for the Autism group was 45, which is much higher than the suggested cutoff of 30 for children with AD and 27 for adolescents and adults with AD.[14] The CARS is reported as a total score. Items concerning special cognitive or savant abilities are not included in the CARS, unlike the ISAA. Regardless, the CARS score was significantly correlated with all the individual domains of the ISAA with the exception of the cognitive component (consisting of variable attention, delays in response, unusual memory or savant ability). The CARS has no items describing these symptoms.

The DD-CGAS is a clinician rated instrument designed to estimate global functioning among children. The score indicates the typical functioning ability of the child during a particular period of time. We found moderate correlation between total ISAA and DD-CGAS score (r = –0.625, the negative correlation reflects the different scoring systems in these scores). The relatively low correlation is consistent with modest correlations reported previously between the DD-CGAS and other published rating scales[20,21]; e.g., the Vineland Adaptive Behavior Scale,[22] the Stanford-Binet Intelligence Scale[23], the Autism Diagnostic Interview–Revised[5], Assessment of Basic Language and Learning Skills.[24] The DD-CGAS provides an overall estimate of function; it does not rate individual domains as is the case with the ISAA. The modest correlations may also be due to different sources of information and time frames used in these instruments. The ISAA scores are based on clinical observations, as well as information gathered from parents. The scoring time frame for the DD-CGAS is the past month. The scoring of DD-CGAS is based on parents report and clinical observation. Parents might report some symptoms as crucial while providing information that may not be directly observable at the time of interview (e.g., peer relationships).[25] Three domains of ISAA, namely the speech language communication, sensory aspects and cognitive components were not significantly correlated with the DD-CGAS total scores. Many of the participants with AD were non-verbal, and ratings for sensory aspects and cognitive domains may not have a strong impact on rating of global functioning provided by the DD-CGAS. DD-CGAS items emphasise functional ability and self-help skills of the child rather than the cognitive components. Consistent with this notion,[20] reported that the Communication Domain–Verbal, and Repetitive Behavior Domain of ADI-R did not correlate significantly with DD-CGAS scores.

The mean ISAA score in the ADHD group was higher than the ID group, as well as children with other psychiatric disorders and the group without a psychiatric diagnosis. The children in the ADHD group received relatively high scores for hyperactivity, aggressive behaviour, temper-tantrums, self-injurious behaviour, insensitive to pain and poor peer relations. Scores on other domains such as social relationships and reciprocity or emotional responsiveness were much lower than the ratings among the children with AD. Therefore, children with comorbid ADHD and AD[26] may yield ISAA ratings that require careful interpretation.

There are some shortcomings in the study. The correlational analyses were restricted to the AD group as the functional ratings with the CARS and the ISAA are most relevant to the AD group. Males were over-represented in our sample, consistent with the higher prevalence of male children with AD, ADHD and other psychiatric disorders.[27] The preponderance of males may also partially reflect referral bias. Although attempts were made to recruit children with similar age distributions across the diagnostic groups, there were marked differences [Table 1], possibly reflecting differential awareness of these disorders in India. The socio-economic status of the parents in the autism group was higher than the other three diagnostic groups. Though others have noted differences in prevalence of AD across socio-economic groups[28], the patterns observed here likely reflect differences in institutions that referred potential participants, some of which were fee for service facilities. As some of the participants in the present study were also included in the initial validation study of the ISAA, this report cannot be considered to be a replication. It should be noted that the ISAA was not intended to be a diagnostic test. Therefore, it should not replace conventional clinical diagnostic evaluations or structured evaluations.

In conclusion, the ISAA is a reliable scale for assessing the severity of autism in Indian children across a wide age range. It can usefully complement standard diagnostic assessment schedules. Additional studies are indicated to examine whether the ISAA scores change over time and whether they are sensitive to effective therapies. The ISAA needs to be tested further in children below the age of 3 years. Further studies are also required to see whether the ISAA evaluation can help to differentiate between high functioning and low functioning children with autism and comorbid psychiatric disorders, including intellectual disability. The English version of the ISAA has been translated into several Indian languages and validation of these versions is needed.[29]