Protocol for Development of the Indian Autism Screening Questionnaire: The Screening Version of the Indian Scale for Assessment of Autism

Abstract

Introduction:

Autism is included as a certifiable disability in the Indian Rights of Persons with Disability Act, 2016. The Indian Scale for Assessment of Autism (ISAA), developed by the Government of India and mandated for certifying disability, is a detailed instrument that needs trained mental health experts and takes time to administer. The current project was planned to develop a simple, easy to use screening tool based on the ISAA to identify possible cases in the community.

Methods:

The project is planned in three phases. During the first phase, data collected during the development of the ISAA (N = 433/436 children with autism) will be used to identify questions answered as frequently, mostly, and always. During the second phase, the psychometric properties of the screening tool based on these items will be evaluated among research participants recruited from hospitals and special schools (n = 100). In the third phase, the screening questionnaire will be administered in the community (n = 500).

Results:

The most frequently answered questions will be selected for inclusion in the proposed screening tool. The number of items in the screening tool will be kept as few as possible, with yes or no responses

Discussion:

Indian Autism Screening Questionnaire (IASQ) will be tested as a screening version of ISAA, which can be used by community health workers, teachers, or school counselors. The IASQ will not provide a diagnosis of autism. A positive screening result should be followed by a thorough assessment by a trained specialist. Analyzing the psychometric properties of the test can help ensure cost-effective screening of the community to identify autism.

Key Messages:

The screening tool will be developed in three phases. It will be short and easy to administer in the community. Psychometric properties of the screening tool will be evaluated among research participants. In the third phase, the tool will be administered in the community.

Autism (or autistic disorder [AD]) is a serious public health burden in India with an estimated 1.7–2 million children said to be affected with autism spectrum disorder (ASD).¹ Population-based studies in Asian countries since 2000 reported prevalence of 13.9 per 10,000 individuals² but population-based surveys in India possibly used incorrect methods.³ Earlier samples from psychiatric outpatient departments described varying predictions of the prevalence of autism, ranging from 2.9% to 62.5%.^4–7 The prevalence was estimated to be 61.9/10,000 worldwide in 2012² and may or may not be rising over the world⁸ and in India. Since 2017, autism has been included as a disability, certifiable for government assistance, in the Rights of Persons with Disability Act. The Indian Scale for Assessment of Autism (ISAA) was developed by the Government of India for measuring autism and is mandated for evaluating degree of disability. It is a detailed 40-item tool that can be used for certification and follow-up, but cannot be used for screening in population studies.⁹ Hence, it is important to develop a simple, easy to use screening tool to identify possible cases in the community.

Screening tools are brief measures that characterize children who are in danger for atypical development from those who are not. Early identification through screening reduces delays and encourages early intervention to reduce disability and burden,¹⁰ thus improving outcome. The purpose is to identify a child at high risk for the disorder, based on some criterion such as physician observation, family history, parent report, and assessment using a screening tool.

There are two types of screening contexts. Level 1 includes screening assessments applicable for general pediatric populace, whereas level 2 screening measures are developed for use in a part of the populace which could be “at risk” for the disorder. Level 1 screening tools should be low cost and short as they are used for not “at-risk” children. Level 2 screening measures can need more time or competence to administer, since children in the level 2 sample have a greater possibility of having the disorder. For identification of ASD children in the population, level 1 screening should be carried out by primary care givers.^11–15 The entire effort should result in immediate referral for an accurate diagnosis and multidisciplinary intervention.¹⁶ Screening enables general physicians and other healthcare workers to evaluate possible cases in the population that need further clinical consideration. Efficient screening should be low cost as to time, cost, and healthcare budget, and effective as to increasing to the maximum sensitivity and specificity.

The primary parameters of a screening instrument are as follows: (a) sensitivity, proportion of persons with disorder with positive screen result; (b) specificity, proportion of persons with disorder with negative screen result; (c) positive predictive value (PPV), proportion of individuals having positive screen result who actually present with the disorder. Sensitivity is needed to be high in order that the screen misses very few cases (avoiding falsely convincing parents and professionals). Specificity is required to be high so that few cases without the disorder are screen positive (avoiding falsely disturbing parents and costly referral for in-depth assessments). Glascoe predicted that 70%–80% sensitivity and specificity for developmental screening tests are sufficient, reflecting the nature and intricacy of measuring the continuous course of child development.^17,18

Development of a shorter, simpler screening version of ISAA as a level I screening tool, which can be used by primary healthcare providers, can help in early detection and initiate intervention to prevent disability and maximize functioning. The present research project proposes to develop a brief screening version of the ISAA and test it for general use in appropriate population of children.

Objectives

The current project is planned to develop a simple, easy to use screening tool based on the ISAA to identify possible cases in the community; to develop and validate the screening version of ISAA; and to test the sensitivity of the screening tool against the “gold standard”: full version of ISAA.

Methods and Materials

The study will be divided into three phases. The design of the study is illustrated in the flowchart (Figure 1).

Figure 1. Flowchart of the Design of the Study.

Phase 1: Short listing of Questions for the Screening Version

The ISAA is a 40-item scale that comprises six domains: Social Relationship and Reciprocity (9 questions); Emotional Responsiveness (5 questions); Speech-Language and Communication (9 questions); Behavior Patterns (7 questions); Sensory Aspects (6 questions); and Cognitive Component (4 questions). The score for each item of ISAA is from 1 to 5, based on the intensity, frequency, and duration of a particular behavior with the following basic ideas: score 1 = rarely (up to 20%), score 2 = sometimes (21%–40%), score 3 = frequently (41%–60%), score 4 = mostly (61%–80%), and score 5 = always (81%–100%). Scoring is based on information from parents and observation of the child. Total ISAA scores range from 40 to 200. The lowest score means that there are no symptoms or symptoms that are present only rarely, and the maximum score represents the most severe presentation of AD. The following categories are recommended: mild AD, 70–107; moderate AD, 108–153; severe AD, >153. ISAA requires 45–60 minutes to administer, and training is also required. Using data from the original ISAA field study where 436 children with autism were assessed from 10 centers all over the country, items that are present frequently or above will be selected (approximately 10–12). Each question of screening version will have a yes/no response. The questions will be discussed with experts in the field.

Face validity: This is the degree to which a test appears to measure what it claims to be measuring. Whether the screening version can be generalized or not will be evaluated with the help of experts in the field.

Phase 2: Obtaining the Sensitivity and PPV of the Screening Version

Sensitivity is the probability that an individual with the disease is screened positive. PPV is the odds that subjects with a positive screening test surely have the disease. PPV predicts how effective the screening version is. How many of test positives are true positives, and if this number is higher (as close to 100 as possible), then it recommends that this new version is doing as good as “gold standard,” i.e., ISAA. To check the sensitivity and PPV of the screening version, initial sample size needs to be 100. This calculation is based on the feasibility as well as on the rule of thumb for scale development that at least 10 respondents per item should be tested, both with and without the disorder. Therefore, for a 10-item scale, we will need to administer the scale on at least 100 participants. It will depend on total questions selected; however, we aim to try and make the screening instrument of not more than 10 questions. All participants presenting at Dept. of Psychiatry fulfilling the inclusion criteria will be referred to research department and those consenting to participate will be enrolled (Table 1).

Table 1.

Criteria for Participation in Phase 2

Inclusion criteria	Exclusion criteria
Inclusion criteria (Group 1: Autism disorder) ·  Age range: 3-18 years ·  Autism, diagnosed by the Clinician based on DSM IV* criteria ·  Primary diagnosis: autism ·  Consent of the parent	Exclusion criteria (Group 1) ·  Any co-morbid psychiatric or neurological disorder
Inclusion criteria (Group 2: Any psychiatric disorder) ·  Clinician diagnosed on DSM IV* criteria ·  Age range: 3-18 years Consent of the parent	Exclusion criteria (Group 2) ·  Any co-morbid neurological disorder

*Diagnostic and Statistical Manual of Mental Disorders, 4th Edition, American Psychiatric Association.

All children identified in phase 2 will be referred to Dept. of Psychology, Dr. Ram Manohar Lohia Hospital and will be tested for intellectual disability and diagnosed clinically. This sample will be divided into two groups. The first group will consist of children diagnosed with AD (n = 50) and second, with any other psychiatric disorder (n = 50). All the diagnoses will be made by qualified experienced psychologists, with at least M. Phil. in Clinical Psychology. After getting written informed consent from the parents and assent from the child, all 100 children will be evaluated on the screening version as well as on ISAA. Sensitivity analysis will be carried out. If the sensitivity is found to be less than 100%, the shortlisted questions will be revised. Data collection will be continued with the revised version. The shortlisted questions will be revised until the sensitivity approaches as near 100% as practical. PPV and cut-off for the screening version will be calculated after completion of data collection.

Test–retest reliability: In order to measure test–retest reliability in phase 1, retesting of 30% of sample from autism group will be conducted after three months by the same rater.

Inter-rater reliability: This method measures the external consistency of a test. This refers to the degree to which various raters arrive at consistent estimates of the same behavior. In order to assess inter-rater reliability, 30% of phase 1 sample will be evaluated by two raters.

Phase 3: Larger Sample from the Community

Once the cut-off for the screening version is decided and expected sensitivity of the tool has been determined, a larger sample (n = 500) will be recruited from the community, and from different government and private schools in and around Delhi. For the school sample, initially school authority will be contacted and after obtaining their permission, parents of the children will be contacted. We will send letters to the parents requesting participation in the study or permission to contact them on phone to explain the research in detail. If they agree to participate, written informed consent will be obtained from the parents and assent from the child. Interview of the parents will take place either in the school or at their residence according to the willingness of the respondents.

Information about family of the child will be incorporated in the sociodemographic data sheet, which includes information about parents, e.g. their age, education, occupation, monthly income, any medical/psychiatric history in the family, birth order of the child, etc.

All the children will be evaluated on the Indian Autism Screening Questionnaire (IASQ)—the screening version of ISAA. Those who score above cut-off along with a demographically matched sample of those scoring below cut-off will be assessed in detail on ISAA. After the completion of phase 3 data collection, sensitivity analysis will be done again. Finally, we will analyze the data for PPV. Plan over three years is illustrated in Table 2.

Table 2.

Plan of Conduct of Study over 3 Years

Time Line	Activity	Time Frame	Duration
1st Year	Ethics Committee approval Training of Project of Staff	Month 1–3	3 months
	Short listing of questions	Month 4–5	5 months
	Phase 1 data collection and sensitivity analysis in hospital sample	Month 6–12	6 months
2nd Year	Data collection from community	Month 13 to 30	1.5 Year
3rd Year	Data Analysis and Writing up of manuscripts	Month 31 to 36	Last 6 months

Data Capture

All data will be captured in real time using a dedicated online portal to be developed by Data Management Unit of Indian Council of Medical Research (ICMR). The hard copy of anonymized data will be kept under lock and key in a secure place.

Statistical Analysis

Phase 1: frequency and distribution analysis. To shortlist the questions for screening version, frequency distribution of all 40 questions form the original ISAA dataset will be analyzed. Shortlisted items will be discussed with experts. Phase 2: sensitivity and PPV. To check the sensitivity of the screening version against the gold standard, i.e., ISAA and DSM IV, data will be analyzed. Test–retest reliability, inter-rater reliability, and analysis for construct validity will be carried out in this stage. Phase 3: confirmation of sensitivity, specificity, and PPV. To confirm the sensitivity and PPV of screening version, a larger community sample will be recruited.

Results

A total of 433/436 children with autism from the original ISAA development study, whose complete data were available, will be used to identify possible screening items.

Discussion

Screening is the first important step in the diagnostic process and can help reduce the burden of healthcare providers by reducing need for prolonged evaluation of every person using detailed, time consuming, and costly formal diagnostic instruments. The paucity of trained personnel is also a barrier, as well as the perceived increase in numbers of children with autism.⁸ This study describes the protocol of a project to develop a simple screening tool that can be used by community level workers to screen children for symptoms of autism. The proposed IASQ is a screening version of ISAA, which can be used by community health workers, teachers, or school counselors. The IASQ is not designed to provide a diagnosis of autism. A positive screening result should be followed by a thorough assessment by a trained specialist. A screening tool should have robust psychometric features to back its accuracy and be culturally and linguistically applicable. While phase 1 will identify the questions, phases 2 and 3 that will recruit a prospective, larger sample will further ascertain psychometric properties of the tool and help understand the predictive validity of the study.

In India, prevalence of ASD was not studied with correct methods till 2015.¹⁹ The diagnosis is generally missed due to general lack of awareness and lack of indigenous tools. Indian researchers have developed some tools. A recently published Indian screening instrument is the Chandigarh Autism Screening Instrument (CASI),²⁰ which is a 45-item scale with sensitivity 89.16%, specificity 89.13%, PPV 67.89%, and negative predictive value 96.96%. Its shorter version is the 4-item CASI-Brief, with sensitivity 73.49% and specificity 90.68% at a cut-off score of 2. Kishore et al. developed an Autism Scale for infants through a survey. The scale has specificity of 0.90 and sensitivity of 0.88²¹; however, the scale has 40 items and difficult to be used as screening instrument.

The ISAA was designed as a detailed instrument to evaluate presence and severity of symptoms. ISAA was tested and validated in 10 centers all over India with a sample size of 1124 participants aged 3–18 years including 436 diagnosed as autism. ISAA requires 45–60 minutes, as well as training, to administer. Considering its large sample size and rigorous methodology, utilizing its most frequently scored items to design a shorter screening instrument seems feasible and acceptable, and would require less time. Considering the relative rarity of autism prevalence (0.139% in Asia),² population-based surveys would not be cost-effective. However, health workers may not be familiar with symptoms of autism. Hence, a brief instrument may not serve the entire purpose unless some explanations, descriptions, or supplements are provided as required. The ISAA illustrative training video is available free of charge on YouTube (https://www.youtube.com/watch?v=kz-rddmCEQQ) and will be included in the database being developed as part of the ICMR Task Force Capacity Building projects of which this is a part. In addition, supplementary explanatory questions (developed by workers at National Institutes for Mentally Handicapped for ISAA, and who routinely evaluate autism) will be included. The IASQ-ISAA database on the ICMR server, developed and tested as part of this program, will provide facilities for storing anonymous clinical data. However, experts who use it will need to register. Data they enter will be stored at the ICMR servers with full privacy protection and can be downloaded at their site of entry. Thus, we hope to build an anonymous virtual database of persons evaluated for autism from all over the country. We hope that this national resource can then be utilized by experts, policy makers, and researchers.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This protocol paper describes the methodology of a research project funded under ‘Capacity Building Task Force for Mental Health Research in India’ funded by the Indian Council of Medical Research vide file number 5/4-4/151/M/2017/NCD-1. International mentors were funded by the training program 'Cross Fertilized Research Training for New Investigators in India and Egypt' funded by FIC, NIH (No. D43 TW009114).