Abstract
The Social Responsiveness Scale (SRS) has been validated in high income countries, but not yet in low and middle-income countries. We aimed to assess the reliability of the SRS in a community sample and its validity to discriminate between children with and without autism spectrum disorder (ASD) in Vietnam. We used a 3-phase study: piloting the translated SRS, reliability testing, and validation of the SRS in 158 Vietnamese caretakers and their children (ages 4–9 years). We examined reliability, validity and sensitivity and specificity to ASD diagnosis. We applied receiver operator characteristic (ROC) analysis to determine optimal cutoff scores discriminating the children with ASD from those without ASD. We also assessed the performance of the SRS short form. We found that reliability was good with high internal consistency (0.88–0.89) and test–retest reliability (0.82–0.83), sensitivity (93%), and specificity (98%) for identification of children with ASD. The ROC curves were similar for total raw score and total T-score, with the area under the curve (AUC) values reaching 0.98 and the optimal cutoff of 62 for raw scores and 60 for T-scores. The SRS short form also performed well in distinguishing children with ASD from children without ASD, with high AUC (0.98), sensitivity (89%), and specificity (98%) when using a raw score of 15 as a cutoff. In conclusion, the translated and culturally adapted SRS shows good reliability, validity, and sensitivity for identification of children with ASD in Vietnam. Both SRS long and short forms performed adequately to discriminate between children with and without ASD.
Keywords: Social Responsiveness Scale, autism spectrum disorder, Vietnam, low- and middle-income countries, validity, reliability
LAY SUMMARY
Middle-income countries often lack validated tools to evaluate autism symptoms. The Social Responsiveness Scale (SRS) translated to Vietnamese was reliable and performed well to distinguish between children with and without autism spectrum disorder in Vietnam. The Vietnamese SRS, and translations of the tool to other languages with this methodology, may be useful in pediatric practice, potentially allowing providers to make more appropriate referrals for diagnostic evaluations and identify children for intervention to help them fulfill their developmental potential.
INTRODUCTION
Neurodevelopmental disorders (NDDs), including autism spectrum disorder (ASD), are a growing concern worldwide. As many as 1:10 children born in the United States are diagnosed with a NDD (Landrigan et al. 2012), including 1:59 with ASD (Baio et al. 2018). An estimated 250 million children (43%) under age 5 years in low- and middle-income countries (LMIC) are at risk of not reaching their developmental potential (Black et al. 2017). Among them, many may suffer from ASD but have not yet been diagnosed. The diagnosis of ASD in LMIC is a particular challenge due to serval factors: lack of trained providers to perform gold-standard clinical diagnostic evaluations, stigma toward children with developmental difficulties, lack of awareness of relevant signs and symptoms, and complicated and time-consuming diagnostic instruments requiring extensive training (Constantino et al. 2003).
The Social Responsiveness Scale (SRS) was developed to address the growing demand for a quantitative measure of autism traits in children and adolescents, (Constantino and Gruber 2005). The SRS probes reciprocal social behavior and social-communicative abilities, including both specific items related to ASD and non-specific items that are frequently observed symptoms among individuals with ASD (Bolte et al. 2008; Grzadzinski et al. 2011). The SRS has well-established psychometric properties in US samples and has been shown to reflect the dimensional ASD phenotype in the general population (Constantino and Todd 2003) and in high-risk sibling samples (Constantino et al. 2006; Constantino et al. 2013; Constantino et al. 2010). The SRS has also been used in several analyses of risk factors and the quantitative ASD phenotype (Braun et al. 2014; Krumm et al. 2013; Wallace et al. 2012).
The SRS has been validated in studies in high income countries outside the US, which demonstrate cross-cultural validity (Bolte et al. 2008; Wang et al. 2012). For example, a study in 1,436 German children and adolescents showed that the SRS had high internal consistency (0.91–0.97), test–retest reliability (0.84–0.97), interrater reliability (0.76 and 0.95), and convergent validity with other diagnostic tools for ASD (Bolte et al. 2008). Other studies in Germany (Bolte et al. 2011), Taiwan (Wang et al. 2012), and Britain (Charman et al. 2007) showed similar sensitivity and specificity of the SRS for diagnosing ASD compared to studies conducted in the US. While there is ample evidence for cross-cultural validity of the SRS in high income countries, there is very limited evidence for the validity of the SRS in LMIC. The objectives of this study were to 1) Translate of SRS from English into Vietnamese and pilot testing the SRS; 2) assess the reliability of the SRS in a community sample; and 3) examine its validity to discriminate between children with and without ASD in Thai Nguyen province in northeast Vietnam.
METHODS
Study design and Participants
Participants in the study were the parents and other primary caretakers of children aged 4–9 years in Thai Nguyen, a province in northeast Vietnam. Caretakers of a total of 158 children were consented and participated in the study in 3 phases: 1) Phase I: 30 caregivers of children without ASD participated in piloting the translated SRS; 2) Phase II: 30 additional caregivers of children without ASD participated in reliability testing the SRS; and 3) phase III: 98 caregivers (49 of children without ASD and 49 of children who had been previously diagnosed with ASD by a clinician) participated in the validation study (Supporting information Figure 1). The caregivers of children with ASD were recruited from autism centers in Thai Nguyen city and caregivers of community-matched control children without ASD were recruited from communities within 20 km of the city.
Procedures
Phase I: Translation of SRS from English into Vietnamese and pilot testing the SRS
We translated and adapted the SRS following the International Test Commission (ITC) guidelines (International Test Commission 2017) for translating and adapting tests. First, we translated, back-translated, and reconciled the SRS instructions and items using a modified Functional Assessment of Chronic Illness Therapy translation methodology (Eremenco et al. 2005) consisting of 1) one forward translation by a native speaker of the target language; 2) back-translation by an independent translator; and 3) independent comparison of source and back-translated versions to identify discrepancies and harmonize. We further refined the translation to ensure functional rather than literal equivalence (ITC guideline Test Development TD-2) using the procedure described below.
This first version was piloted with 30 community mothers with children aged 4–8 years using an interview format. We administered the SRS by interview in every phase of the study. ITC guideline Pre-Condition-3 is to “minimize the influence of any cultural and linguistic differences that are irrelevant to the intended uses of the test in the population of interest,” including any administration procedures that may introduce method bias. Caregivers in the study area had widely varying literacy levels, therefore using a questionnaire format could introduce method bias between respondents with high versus low literacy.
ITC guideline TD-3 is to “provide evidence that the test instructions and item content have similar meaning for all intended populations.” We gathered this evidence and refined the translations to ensure functional equivalence by asking comprehension questions after each item. For example, if the mother answered “sometimes” to the item “seems much more fidgety in social situations than when alone” then she would be asked “tell me an example of when the child showed that behavior.” Based on the transcribed responses, three researchers independently rated whether the caregiver understood the intended meaning of the question. These data were reviewed by the investigators to suggest modifications to the translations for items with low participant understanding. These modifications were made before the second pilot phase.
Phase II: Reliability testing the SRS
We interviewed 30 additional community mothers twice, with a test-retest interval of one week, in order to evaluate test-retest reliability. A relatively short retest interval of one week is typical in health research and minimizes the chance that the true score would change in the intervening time period (Polit 2014). In this phase, we again assessed item comprehension in the same way as described above. The transcribed responses were then again rated by three independent researchers. These data were compared with the previous phase to assess whether there were improvements in the levels of understanding the questions.
Phase III: Validation study
Finally, we interviewed mothers of 56 children who had been diagnosed with ASD and 57 community children who did not have ASD. This sample size was powered to detect 90% sensitivity and specificity with 8% precision. The children with ASD were diagnosed by a trained developmental pediatrician and psychologist after evaluation with the Denver Developmental Screening Test (Frankenburg and Dodds 1967) and Modified Checklist for Autism in Toddlers (MCHAT) (Robins et al. 2001). After conducting independent assessments, the pediatrician and psychologist discussed each case and came to a consensus diagnostic determination based on the DSM-IV-TR criteria for ASD (American Psychiatric Association 1994). DSM-IV criteria were used because this is the diagnostic standard in Vietnam; the transition to DSM-5 has not yet taken place (Ha et al. 2017).
Among the children diagnosed with ASD, the majority (81%) were diagnosed between 25–36 months of age. An additional 4% were diagnosed before 24 months of age, and 15% were diagnosed after 36 months of age. Common reasons for initial referral included language delays, social communication concerns (diminished response to name, reduced eye contact, reduced interest in or difficulty interacting with other children), and non-specific concerns such as excessive tantrums or regulatory problems (e.g., difficulty eating or sleeping). Children with ASD were significantly younger and a higher percentage were male compared to children without ASD. Therefore, we excluded 8 children without ASD and 7 children with ASD in order to create balanced groups for analysis.
Measures
The SRS questionnaire was administered by caregiver interview. The SRS is a 65-item rating scale that measures autism traits over the previous 6 months in children and adolescents aged 4–18 years. Each item is scaled from 0 (never true) to 3 (almost always true), generating a total score ranging from 0 to 195. The average time to complete SRS is typically around 15–20 minutes. We transformed total raw scores into T-scores in order to provide the standard score relative to a normative group of US children. We also calculated raw scores for five subscales: Social Awareness, Social Cognition, Social Communication, Social Motivation, and Autistic Mannerisms. Finally, we created a SRS short form raw score using a sub-set of 16 items (Supporting information Table 1), which were recommended previously (Sturm et al. 2017) to increase efficiency and reduce field worker and participant burden while still ensuring adequate reliability and content coverage.
Other information related to children’s parents was also collected, including age, marital status, education, occupation. For children, we collected information on age, sex, and child’s current grade in school.
Statistical analysis
Background characteristics of the study sample were examined using descriptive analyses. Bivariate analyses were used to compare differences between characteristics of children with and without ASD and their caregivers using Student’s t-tests for continuous variables and Chi-square tests for categorical variables. For each of the three raters, we calculated the percentage of children rated to understand each item. We then averaged these percentages across the three raters to estimate the percentage of participants that understood each item. The agreement between raters was high (ICC > 0.80). The correlation between children’s age and total SRS scores and subscale scores were examined by Pearson product moment correlations, and the equality of two independent correlations (among children with and without ASD) was tested using likelihood-ratio test.
Test-retest reliability by item was assessed by 1) percent agreement in item score between the first and second interview and 2) Cohen’s Kappa coefficient with an unweighted ordinal approach (McHugh 2012). Kappa scores >0.75 are indicative of excellent reliability, 0.40–0.75 indicate fair to good reliability, and <0.4 indicate poor reliability.
Overall reliability was assessed by: 1) computing internal consistency (Cronbach’s alpha) of all 65 items; 2) comparing the difference in mean total and sub-scale scores on the repeated measures between two visits; and 3) estimating intraclass correlations (ICCs) for test-retest reliability of the total and sub-scale scores. ICCs were calculated based on a single-rating, absolute-agreement, 2-way mixed-effects model (Shrout and Fleiss 1979; Koo and Li 2016); this model is recommended for evaluating test-retest reliability because repeated measurements cannot be regarded as randomized samples (Shrout and Fleiss 1979). ICC values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.90 indicate good reliability, and values greater than 0.90 indicate excellent reliability. All analyses were conducted for total scores and the five subscale raw scores of Social Awareness, Social Cognition, Social Communication, Social Motivation, and Autistic Mannerisms.
To assess validity, we first compared the SRS scores between the children with and without ASD using Student’s t-tests. We then used receiver operating characteristic (ROC) curves to examine the ability of the SRS to predict diagnostic classification (i.e., ASD vs. non-ASD) for each of the cutoffs. The area under the curve (AUC) is expressed as a percent, with 100% indicating perfect prediction and 50% indicating chance. Based on ROC analyses, we determined optimal cutoff points for ASD screening and estimated the indicator’s sensitivity, specificity, true positive, true negative, false positive, false negative, and positive and negative predictive values. We used these characteristics to assess the performance of the SRS at these cutoff points to correctly identify children with ASD. We also estimated the above indicators of performance for the standard cutoff of a total raw score of 70 or a total T-score of 60 based on US norms (Constantino 2012).
Finally, to assess the performance of the SRS short form, we estimated Pearson product moment correlations for the total score and five subscale scores of the long and short forms. We also performed the ROC analyses for the raw scores based on the short form (Sturm et al. 2017). All analyses were conducted using Stata 14.2 (Statacorp, College Station, TX). Statistical significance was set at 5% and all tests were 2-sided.
RESULTS
Background characteristics of the samples analyzed are presented in Table 1. The mean age of mothers was 33 years; approximately 80% had completed high school or higher. Around two thirds of mothers were office staff or service workers. Similar levels of education and occupation were observed for fathers. The mean age of the children assessed ranged from 5–6 years across phases. There was no difference in age and gender among children with or without ASD.
Table 1:
Comparison of sociodemographic data of the full and sub-samples selected from the study population
| Variables | Full sample (n=158) | 1st phase (n=30) | 2nd phase (n=30) | 3rd phase (n=98) | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Children without ASD (n=49) | Children with ASD (n=49) | P-value1 | |||||||||
| n | Mean ± SD (min, max)/Percent | n | Mean ± SD (min, max)/Percent | n | Mean ± SD (min, max)/Percent | n | Mean ± SD (min, max)/Percent | n | Mean ± SD (min, max)/Percent | ||
| Mother characteristics | |||||||||||
| Mother’s age (y) | 158 | 33.5 ± 5.0 (23, 46) | 30 | 32.5 ± 5.2 (23, 43) | 30 | 33.0 ± 4.3 (27, 42) | 49 | 34.9 ± 4.8 (24, 46) | 49 | 33.0 ± 5.4 (24, 44) | 0.077 |
| Marital status | |||||||||||
| Married | 151 | 95.6 | 28 | 93.3 | 29 | 96.7 | 48 | 98.0 | 46 | 93.9 | 0.309 |
| Widowed/divorced/separated | 7 | 4.4 | 2 | 6.7 | 1 | 3.3 | 1 | 2.0 | 3 | 6.1 | |
| Mother’s education | |||||||||||
| Primary school and lower | 3 | 1.9 | 1 | 3.3 | 0 | 0 | 1 | 2.0 | 1 | 2.0 | 0.300 |
| Junior high school | 27 | 17.1 | 10 | 33.3 | 0 | 0 | 5 | 10.2 | 12 | 24.5 | |
| High school | 31 | 19.6 | 6 | 20.0 | 1 | 3.3 | 13 | 26.5 | 11 | 22.5 | |
| College and higher | 97 | 61.4 | 13 | 43.3 | 29 | 96.7 | 30 | 61.2 | 25 | 51.0 | |
| Mother’s occupation | |||||||||||
| Farmer | 1 | 0.6 | 0 | 0.0 | 0 | 0 | 0 | 0 | 1 | 2.0 | 0.989 |
| Worker | 27 | 17.1 | 12 | 40.0 | 0 | 0 | 7 | 14.3 | 8 | 16.3 | |
| Officer/staff | 31 | 19.6 | 5 | 16.7 | 26 | 86.7 | 14 | 28.6 | 12 | 24.5 | |
| Business/services | 70 | 44.3 | 11 | 36.7 | 3 | 10.0 | 16 | 32.7 | 17 | 34.7 | |
| Freelance/unskilled worker | 9 | 5.7 | 1 | 3.3 | 1 | 3.3 | 2 | 4.1 | 3 | 6.1 | |
| Housewife/unemployed | 15 | 9.5 | 1 | 3.3 | 0 | 0 | 7 | 14.3 | 6 | 12.2 | |
| Other | 5 | 3.2 | 0 | 0 | 3 | 6.2 | 2 | 4.1 | |||
| Father characteristics | |||||||||||
| Father’s education | |||||||||||
| Primary school and lower | 5 | 3.3 | 2 | 7.1 | 0 | 0 | 2 | 4.2 | 1 | 2.2 | 0.453 |
| Junior high school | 27 | 17.9 | 10 | 35.7 | 0 | 0 | 7 | 14.6 | 10 | 21.7 | |
| High school | 34 | 22.5 | 6 | 21.4 | 2 | 6.9 | 11 | 22.9 | 15 | 32.6 | |
| College and higher | 85 | 56.3 | 10 | 35.7 | 27 | 93.1 | 28 | 58.3 | 20 | 43.5 | |
| Father’s occupation | |||||||||||
| Farmer | 24 | 15.9 | 11 | 39.3 | 0 | 0 | 6 | 12.5 | 7 | 15.2 | 0.829 |
| Worker | 38 | 25.2 | 6 | 21.4 | 2 | 6.9 | 15 | 31.3 | 15 | 32.6 | |
| Officer/staff | 52 | 34.4 | 5 | 17.9 | 21 | 72.4 | 14 | 29.2 | 12 | 26.1 | |
| Business/services | 12 | 8.0 | 0 | 0 | 4 | 13.8 | 4 | 8.3 | 4 | 8.7 | |
| Freelance/unskilled worker | 17 | 11.3 | 1 | 3.6 | 1 | 3.5 | 9 | 18.8 | 6 | 13.0 | |
| Other | 8 | 5.3 | 5 | 17.9 | 1 | 3.5 | 0 | 0 | 2 | 4.4 | |
| Child characteristics | |||||||||||
| Male | 101 | 63.9 | 17 | 56.7 | 18 | 60.0 | 29 | 59.2 | 37 | 75.5 | 0.085 |
| Child age (y) | 158 | 5.8 ± 1.5 (3.2, 10.2) | 30 | 6.2 ± 0.8 (4.6, 7.5) | 30 | 5.2 ± 1.1 (3.3, 7.0) | 49 | 6.0 ± 1.5 (3.2, 8.8) | 49 | 5.7 ± 1.8 (3.3, 10.2) | 0.415 |
| Child education level | |||||||||||
| Kindergarten | 90 | 57.0 | 11 | 36.7 | 23 | 76.7 | 21 | 42.9 | 35 | 71.4 | <0.001 |
| Grade 1 | 40 | 25.3 | 11 | 36.7 | 7 | 23.3 | 10 | 20.4 | 12 | 24.5 | |
| Grade 2 | 23 | 14.6 | 8 | 26.7 | 0 | 0 | 14 | 28.6 | 1 | 2.0 | |
| Grade 3 | 4 | 2.5 | 0 | 0 | 0 | 0 | 3 | 6.1 | 1 | 2.0 | |
| Grade 4 | 1 | 0.6 | 0 | 0 | 0 | 0 | 1 | 2.0 | 0 | 0 | |
| Relationship with the child | |||||||||||
| Mother | 139 | 88.0 | 18 | 60.0 | 30 | 100 | 48 | 98.0 | 43 | 87.8 | 0.071 |
| Father | 6 | 3.8 | 3 | 10.0 | 0 | 0 | 1 | 2.0 | 2 | 4.1 | |
| Grandparent/relatives | 13 | 8.2 | 9 | 30.0 | 0 | 0 | 0 | 0 | 4 | 8.2 | |
p-values tested for differences between characteristics of children with and without ASD
ADS: Autism spectrum disorder; SD: Standard deviation.
Comprehension
Among the 65 SRS items that were piloted in phase I, 42 of them had excellent understanding (>80% mothers), 14 had moderate understanding (50–80%), and 9 had poor understanding (<50%) (Figure 1A). After modifying the translations of items with poor understanding, the levels of understanding significantly improved between phases I and II, in which 60 items had excellent understanding, 4 items had moderate understanding, and only 1 item had poor understanding. The average percentage of participants who understood each item across all items improved from 78 to 92%, and improved for all subdomains (Figure 1B).
Figure 1:
Level of understanding Social Responsiveness Scale items and subdomains in phase I and II
A. Overall Social Responsiveness Scale
B. Subdomain Social Responsiveness Scale
Reliability
In phase II, the test-retest reliability for all items was high (Table 2). Based on kappa coefficients, 14 items (22%) showed excellent reliability (kappa > 0.75), 40 items (62%) showed good reliability (0.4 to 0.75), and 11 items (17%) showed poor reliability (< 0.4). However, all items with low kappa scores showed high percentage agreement between the first and second interview (> 85%). The low kappa scores for these items were due to a lack of variation in caregiver responses. The kappa statistic is known to underestimate agreement in the case of rare events (Viera and Garrett 2005).
Table 2:
Test-retest reliability by item on the Vietnamese version of the SRS-2 (n=30)
| Question | Percentage agreement between the first and second interview | Kappa score1 | Question | Percentage agreement between the first and second interview | Kappa score1 |
|---|---|---|---|---|---|
| Q1 | 90.0 | 0.85 | Q34 | 86.7 | 0.68 |
| Q2 | 93.3 | 0.46 | Q35 | 93.3 | 0.68 |
| Q3 | 66.7 | 0.45 | Q36 | 93.3 | 0.79 |
| Q4 | 86.7 | 0.80 | Q37 | 96.7 | 0.81 |
| Q5 | 80.0 | 0.72 | Q38 | 73.3 | 0.48 |
| Q6 | 93.3 | 0.64 | Q39 | 86.7 | 0.38 |
| Q7 | 73.3 | 0.55 | Q40 | 83.3 | 0.64 |
| Q8 | 93.3 | 0.63 | Q41 | 83.3 | 0.50 |
| Q9 | 86.7 | 0.66 | Q42 | 73.3 | 0.47 |
| Q10 | 86.7 | 0.73 | Q43 | 76.7 | 0.64 |
| Q11 | 70.0 | 0.51 | Q44 | 86.7 | 0.58 |
| Q12 | 70.0 | 0.44 | Q45 | 86.7 | 0.77 |
| Q13 | 90.0 | 0.68 | Q46 | 90.0 | 0.58 |
| Q14 | 93.3 | 0.47 | Q47 | 93.3 | 0.47 |
| Q15 | 73.3 | 0.47 | Q48 | 70.0 | 0.53 |
| Q16 | 96.7 | 0.00 | Q49 | 96.7 | 0.87 |
| Q17 | 83.3 | 0.74 | Q50 | 96.7 | 0.65 |
| Q18 | 96.7 | 0.65 | Q51 | 86.7 | 0.15 |
| Q19 | 86.7 | 0.76 | Q52 | 76.7 | 0.64 |
| Q20 | 96.7 | 0.00 | Q53 | 87.5 | 0.46 |
| Q21 | 86.7 | 0.72 | Q54 | 90.3 | −0.03 |
| Q22 | 96.7 | 0.89 | Q55 | 80.0 | 0.68 |
| Q23 | 100 | 1.00 | Q56 | 76.7 | 0.56 |
| Q24 | 93.3 | 0.78 | Q57 | 96.7 | 0.00 |
| Q25 | 76.7 | 0.62 | Q58 | 86.7 | 0.38 |
| Q26 | 83.3 | 0.75 | Q59 | 100 | 1.00 |
| Q27 | 83.3 | 0.56 | Q60 | 96.7 | 0.00 |
| Q28 | 83.3 | 0.61 | Q61 | 93.3 | 0.74 |
| Q29 | 100 | 1.00 | Q62 | 86.7 | 0.29 |
| Q30 | 90.0 | 0.67 | Q63 | 96.7 | 0.00 |
| Q31 | 86.7 | 0.70 | Q64 | 86.7 | 0.30 |
| Q32 | 90.0 | 0.67 | Q65 | 100 | 1.00 |
| Q33 | 83.3 | 0.49 | |||
Lower kappa scores with corresponding high % agreements are due to a lack of variation in caregiver responses
SRS: Social responsiveness scale, Q: Question.
The total raw scores and T-scores were generally stable between the first and the second visits (Table 3). Except for the Social Awareness subscale, which was slightly higher in the first visit, all other subscale scores were also similar between the two visits. The ICC values were 0.82 and 0.83 for total raw scores and T-scores, respectively, indicating good reliability. The ICC values were higher for two subscales of Social Motivation (0.92–0.93) and Autistic Mannerism (0.88), but lower for other subscales (ranged 0.63–0.76).
Table 3:
Comparison of total scores and sub-scores for children in Phase II: test-retest reliability
| Variables | 1st visit | 2nd visit | ICC |
|---|---|---|---|
| Total scores | |||
| Raw scores, mean ± SD (min, max) | 28.8 ± 12.5 (13, 69) | 27.5 ± 11.1 (11, 66) | 0.82 |
| T scores, mean ± SD (min, max) | 44.9 ± 4.8 (39, 60) | 44.5 ± 4.3 (38, 59) | 0.83 |
| Correlation with age (P-value) | −0.18 (0.341) | −0.21 (0.269) | |
| Cronbach alpha | 0.89 | 0.88 | |
| Sub-domains | |||
| Social awareness | |||
| Raw scores, mean ± SD (min, max) | 5.3 ± 2.3 (2, 12) | 4.7 ± 1.6 (1, 9) | 0.64 |
| T scores, mean ± SD (min, max) | 43.8 ± 6.7 (34, 63) | 41.9 ± 4.6 (31, 54) | 0.64 |
| Correlation with age (P-value) | −0.19 (0.313) | −0.25 (0.185) | |
| Cronbach alpha | 0.53 | 0.37 | |
| Social cognition | |||
| Raw scores, mean ± SD (min, max) | 7.3 ± 2.8 (2, 12) | 7.8 ± 3.3 (1, 15) | 0.76 |
| T scores, mean ± SD (min, max) | 47.8 ± 5.0 (38, 56) | 48.6 ± 5.7 (37, 61) | 0.76 |
| Correlation with age (P-value) | −0.49 (0.006) | −0.40 (0.031) | |
| Cronbach alpha | 0.33 | 0.57 | |
| Social communication | |||
| Raw scores, mean ± SD (min, max) | 7.6 ± 4.7 (2, 26) | 7.1 ± 3.6 (2, 19) | 0.65 |
| T scores, mean ± SD (min, max) | 43.5 ± 4.9 (37, 63) | 43.1 ± 3.7 (37, 56) | 0.63 |
| Correlation with age (P-value) | −0.16 (0.405) | −0.10 (0.601) | |
| Cronbach alpha | 0.79 | 0.68 | |
| Social motivation | |||
| Raw scores, mean ± SD (min, max) | 5.9 ± 2.9 (1, 12) | 5.6 ± 3.0 (1, 12) | 0.93 |
| T scores, mean ± SD (min, max) | 48.7 ± 5.6 (39, 60) | 48.1 ± 5.7 (39, 60) | 0.92 |
| Correlation with age (P-value) | −0.06 (0.734) | −0.11 (0.547) | |
| Cronbach alpha | 0.23 | 0.61 | |
| Autistic mannerisms | |||
| Raw scores, mean ± SD (min, max) | 2.6 ± 3.6 (0, 15) | 2.3 ± 3.0 (0, 11) | 0.88 |
| T scores, mean ± SD (min, max) | 45.3 ± 7.3 (40, 70) | 44.6 ± 6.1 (40, 62) | 0.88 |
| Correlation with age (P-value) | 0.14 (0.477) | 0.02 (0.927) | |
| Cronbach alpha | 0.83 | 0.76 | |
P-value to test difference between 1st visit and 2nd visit
ICC: Intraclass correlations; SD: Standard deviation.
Internal consistency, measured by Cronbach’s alpha, was high and consistent across rounds (ranged 0.88–0.89) (Table 3). Alphas for the subscale scores varied, with higher alphas for Social Communication (range = 0.68–0.79) and Autistic Mannerisms (range = 0.76–0.83), and lower alphas for other subscales (ranging from 0.23–0.61 across subscales and rounds). Similarly, good overall internal consistencies were observed among children with ASD (alpha = 0.90) and those without ASD (alpha=0.89) (Table 4).
Table 4:
Comparison of total scores and sub-scores for children with ASD and community-matched controls without ASD during phase III
| Variables | Non-ASD | ASD | P-values1 |
|---|---|---|---|
| Total scores | (n=49) | (n=49) | |
| Raw scores, mean ± SD (min, max) | 30.4 ± 14.0 (10, 85) | 86.8 ± 20.9 (28, 132) | <0.0001 |
| T scores, mean ± SD (min, max) | 48.6 ± 5.2 (41, 67) | 70.3 ± 8.8 (45, 89) | <0.0001 |
| Correlation with age (P-value) | −0.22 (0.121) | 0.39 (0.006) | 0.002 |
| Cronbach alpha | 0.89 | 0.90 | |
| Sub-domains | |||
| Social awareness | |||
| Raw scores, mean ± SD (min, max) | 5.4 ± 2.4 (0, 14) | 11.0 ± 2.9 (4, 16) | <0.0001 |
| T scores, mean ± SD (min, max) | 48.6 ± 7.8 (32, 76) | 64.7 ± 10.4 (40, 82) | <0.0001 |
| Correlation with age (P-value) | 0.11 (0.444) | 0.43 (0.002) | 0.088 |
| Cronbach alpha | 0.45 | 0.52 | |
| Social cognition | |||
| Raw scores, mean ± SD (min, max) | 7.1 ± 3.6 (1, 16) | 15.9 ± 4.1 (6, 23) | <0.0001 |
| T scores, mean ± SD (min, max) | 51.1 ± 6.1 (40, 63) | 66.5 ± 6.8 (50, 81) | <0.0001 |
| Correlation with age (P-value) | −0.42 (0.003) | 0.01 (0.945) | 0.026 |
| Cronbach alpha | 0.61 | 0.64 | |
| Social communication | |||
| Raw scores, mean ± SD (min, max) | 8.3 ± 5.8 (0, 36) | 29.8 ± 8.9 (8, 49) | <0.0001 |
| T scores, mean ± SD (min, max) | 47.2 ± 5.9 (38, 74) | 70.1 ± 10.1 (44, 90) | <0.0001 |
| Correlation with age (P-value) | −0.14 (0.352) | 0.42 (0.003) | 0.005 |
| Cronbach alpha | 0.82 | 0.81 | |
| Social motivation | |||
| Raw scores, mean ± SD (min, max) | 6.8 ± 3.6 (1, 19) | 15.6 ± 4.5 (3, 24) | <0.0001 |
| T scores, mean ± SD (min, max) | 51.7 ± 7.0 (40, 74) | 69.4 ± 9.6 (43, 87) | <0.0001 |
| Correlation with age (P-value) | −0.21 (0.138) | 0.21 (0.152) | 0.036 |
| Cronbach alpha | 0.63 | 0.59 | |
| Autistic mannerisms | |||
| Raw scores, mean ± SD (min, max) | 2.9 ± 2.2 (0, 9) | 14.4 ± 5.5 (3, 26) | <0.0001 |
| T scores, mean ± SD (min, max) | 46.6 ± 4.2 (41, 60) | 67.8 ± 10.4 (46, 90) | <0.0001 |
| Correlation with age (P-value) | −0.16 (0.261) | 0.381 (0.007) | 0.006 |
| Cronbach alpha | 0.42 | 0.68 | |
P-value to test difference between children with and without ASD
ADS: Autism spectrum disorder; SD: Standard deviation.
Discriminant validity
When comparing SRS scores for children with ASD with the sample of community-matched children without ASD during phase III, we found that, generally, total SRS raw scores and T-scores successfully distinguished children with ASD from those without ASD (Table 4). The mean total SRS raw score for the ASD group was 87 ± 21, compared to 30 ± 14 for the non-ASD group (p<0.001). Similar differences between children with and without ASD were observed for the five subdomain scores. Among the children without ASD, scores decreased as age increased (correlation =−0.22); in contrast, among children with ASD, scores increased as age increased (correlation= 0.39). These correlations were significantly different between the two groups (p=0.002)
Performance of SRS long form
To assess the ability of the SRS to predict diagnostic classification of ASD, we performed ROC analyses, using both raw scores and T-scores (Supporting information Table 2 and 3). The AUC was high for both the raw scores (0.98, 95% CI: 0.96, 1.00) and T-scores (0.97, 95% CI: 0.95 1.00) (Figure 2). These values differed significantly from the null value (0.5). We found that the optimal cutoffs for screening were a total raw score of 62 (sensitivity = 96%, specificity = 98%, false positive =1%) or a total T-score of 60 (sensitivity = 90% and specificity = 98%, false positive =1%). These cutoffs also yielded the highest true positive and lowest false negative values. The standard cutoff of 70 (for total raw scores) for screening in the general population yielded lower sensitivity (76%) and similar specificity (98%) (Table 5).
Figure 2:
Receiver operating characteristics (ROC) curve of the Social Responsiveness Scale for ASD vs. non-ASD children
A. Raw scores from the long form
B. T scores from the long form
C. Raw scores from the short form
ADS: Autism spectrum disorder
Table 5:
Comparison of scores for children with ASD and community-matched controls without ASD during phase III
| Variables | Sensitivity | Specificity | True Positives n (%) | True Negatives n (%) | False Positives n (%) | False Negatives n (%) | Positive Predictive Value | Negative Predictive Value |
|---|---|---|---|---|---|---|---|---|
| SRS long form1 | ||||||||
| Raw scores (cut off 62) | 0.96 | 0.98 | 47 (48.0) | 48 (49.0) | 1 (1.0) | 2 (2.0) | 0.98 | 0.96 |
| Raw scores (cut off 70) | 0.76 | 0.98 | 37 (37.8) | 48 (49.0) | 1 (1.0) | 12 (12.2) | 0.97 | 0.80 |
| T scores (cut off 60) | 0.90 | 0.98 | 44 (44.9) | 48 (49.0) | 1 (1.0) | 5 (5.1) | 0.98 | 0.91 |
| SRS short form2 | ||||||||
| Raw scores (cut off 15) | 0.90 | 0.98 | 44 (44.9) | 48 (49.0) | 1 (1.0) | 5 (5.1) | 0.98 | 0.91 |
Based on standard recommendation, SRS total raw score cutpoint value of 70 was used to determine which children screened positive for ASD. SRS T-score cutpoint value of 60 was used to determine which children screened positive for ASD
SRS Total raw score cutpoint value of 15 based on short form with 16 questions
ADS: Autism spectrum disorder; CI: Confidence interval; SRS: Social responsiveness scale.
Performance of SRS short form
Total scores from the short from were highly correlated with those from the long form (correlation=0.97) (Supporting information Table 4). Except for the Social Cognition subscale, which showed low correlation (0.37) between the short and long forms, the other subscales were highly correlated between the two forms (0.75–0.94). Results from ROC analyses showed high AUC for the short form total scores (0.98, 95% CI: 0.97, 1.00) (Figure 2), and identified the cutoff of 15 with high sensitivity (90%) and high specificity (98%) to distinguish children with ASD from children without ASD (Table 5 and Supporting information Table 5).
DISCUSSION
After an iterative process of translation, piloting, and revising the scale, the SRS-Vietnamese version administered using a caregiver interview format showed excellent reliability with high internal consistency (Cronbach’s Alpha 0.88–0.89) and test–retest reliability (ICC 0.82–0.83). Furthermore, the SRS-Vietnamese version also demonstrated high sensitivity in identifying children with ASD, with a lower raw cutoff compared to the standard cutoff (62 vs. 70) and similar T-score cut off of 60.
This is the first study to translate and pilot the SRS for use in LMIC, providing proof-of-principle for measuring ASD symptoms/traits in developing nations using standardized tools. Our findings demonstrate the reliability and validity of the SRS and indicate that the SRS is a useful tool for measuring ASD symptoms in Vietnam, and likely also in other LMIC. However, further evidence is needed examining the reliability and validity of diagnostic and developmental assessments when translated and transferred from a high-income country to a LMIC. A review of 114 publications reporting the use of early child development assessments in LMIC found that many of the studies did not report any information on validity (Semrud-Clikeman et al. 2017). Cross-cultural adaptations of other ASD screeners in countries outside the US have shown widely varying adherence to recommended adaptation guidelines. A systematic review identified 21 published studies reporting adaptations of ASD screening tools in 17 samples of children across cultures and countries. All were in high-income or upper-middle income countries, except one study in 10 Arabic-speaking countries, including 3 LMICs. The authors found that many of the studies did not adhere to recommended cultural adaptation guidelines (Soto et al. 2015). Another systematic review found that out of 28 studies that used or adapted ASD screeners in LMICs, only four reported any cultural adaptation of the screening instrument (Stewart and Lee 2017). Future studies should report how translations and adaptations were conducted and adhere to recommended guidelines.
Our findings also suggest the potential utility of the translated SRS as a screening tool in Vietnam. Many studies use translated screening tools in a new context without developing context-specific cutoff scores. Our finding that a raw score cutoff of 62 rather than the standard cutoff of 70 was the optimal cutoff to screen for ASD on the SRS Vietnamese version suggests that context-specific cutoff scores should be developed. Findings from a systematic review showed that, among studies that reported sensitivity and specificity of translated ASD screening tools in LMIC, a range of cutoff points were used for the same tools across studies (Stewart and Lee 2017). Applying the standard cutoff to a translated tool in a new context cannot be assumed to result in the same psychometric properties as the original tool.
Despite several rounds of revising the translated SRS items, one item remained poorly understood, with less than 50% understanding, and four items with moderate understanding (50–80%). The item with lowest understanding was the first item in the scale (“Seems much more fidgety in social situations than when alone”). Most of the examples given by caregivers were related to the child fidgeting and not being able to sit still, rather than how much the child fidgets or appears uncomfortable in social situations as compared to being alone. This is a somewhat difficult concept to convey. In an LMIC setting, it may be useful to modify the item order to start with straightforward items that are easily understood and move on to more complex concepts after caregivers have become familiar with the interview procedure and types of questions. However, despite the moderate to low understanding on a small subset of items, the overall scale showed high reliability, sensitivity, and specificity.
To address the concern of the influence of key developmental characteristics on SRS scores (Hus et al. 2013), the SRS short form with 16 items was developed and validated. A previous study showed that this short form was highly reliable and free of bias from gender, age, expressive language, behavior problems, and nonverbal IQ influence (Sturm et al. 2017). In our study, the SRS short from was highly correlated to the original SRS and performed well in distinguishing ASD from non-ASD children, with high AUC (0.98), sensitivity (91%), and specificity (96%) when using the cutoff of 15; thus the SRS short form could be used in settings where time is limited and to reduce respondent burden.
Our findings showed a positive correlation between total SRS scores with age (scores increased as age increased) among children with ASD, but a negative correlation between SRS scores and age among children without ASD. Children with ASD in Vietnam and other LIMCs, where adequate intervention services and support are not always available (Van Cong et al. 2015), may become more symptomatic as they get older. Prior studies in large general population samples have documented relative stability of ASD symptoms over time (Haraguchi et al. 2018; Robinson et al. 2011); it remains unknown how cultural variations in standards of appropriate child behavior might impact parental perceptions and SRS ratings.
This study had some limitations that deserve consideration. First, the sample size is small with limited age range from 4–9 years. It will be important to replicate these findings in larger samples with a wider age range as intended for SRS (up to age 18 years) and to consider whether Vietnam-specific norms should be developed. Second, as described above, participants may not have understood the intent of each question, and/or some of the questions may have been less appropriate for this population than the population for which they were originally developed. Additionally, we administered the SRS questionnaire by caregiver interview format which is different from a standard administration of the measure, due to a high range education levels (from first grade to college degree) among caregivers. Compared to other LMICs, caregivers’ education in Vietnam is higher (reflecting the general trends of education in Vietnam which improved substantially in recent years) (Dang and Glewwe 2017), thus they may be more likely to be aware of developmental concerns in their children and seek out evaluation. Finally, the children with ASD in this sample were not diagnosed using gold-standard procedures (e.g., use of outdated DSM-IV diagnostic criteria) and diagnostic tools (e.g., the Autism Diagnostic Observation Scale). Applying the same standards is difficult in Vietnam as in other LMICs where the health resources, expertise, and support services for ASD are still very limited (Ha et al. 2017), and the use of up-to-date diagnostic criteria lags behind that of high-income countries.
Despite these limitations, this study provides initial support for the use of a translated version of the SRS in rural Vietnam. Gold-standard diagnostic tools are not available in many LMICs nor are large numbers of ASD specialists, making the validation of reliable screening measures more important. Research has shown that approximately 45% of children are missed when utilizing developmental surveillance/clinical judgment alone (Aylward 2009), in the absence of a standardized screening measure. The availability of an autism-specific standardized screening measure in LMICs, like the SRS-Vietnamese translation, is therefore critical. Such measures can be administered in general pediatric practice and may allow providers to make more appropriate referrals for comprehensive diagnostic evaluations with the potential to more effectively, efficiently, and appropriately utilize the limited resources (e.g., specialists, services systems) available in LMICs.
CONCLUSION
Access to validated screening measures is limited in LMICs. The translated and culturally adapted Vietnamese SRS shows good reliability, validity, and sensitivity for identification of children with ASD in Vietnam. Both SRS long and short forms performed adequately in screening for children with ASD. Future studies should examine the utility of culture-specific norms.
Supplementary Material
Supporting information Figure 1: The flow of participants
Supporting information Table 1: List of items for SRS short form
Supporting information Table 2: Accuracy of indicators for classifying autism children using total raw scores
Supporting information Table 3: Accuracy of indicators for classifying autism children using total T-scores
Supporting information Table 4: Correlation between Subscales for the Long-Form and Short-Form
Supporting information Table 5: Accuracy of indicators for classifying autism children using total raw scores for short form
Acknowledgements
This publication is based on research funded by the UC Davis Clinical and Translational Science Center, Medical Investigation of Neurodevelopmental Disorders (MIND) Institute IDDRC, the Department of Pediatrics Rosa B. Sherman Research Fund, and the Center for Precision Medicine and Data Sciences. We gratefully acknowledge data collection by Thai Nguyen Central General Hospital.
Footnotes
ETHICAL APPROVAL
Approval for this study was obtained from the institutional review board at the University of California Davis and the Thai Nguyen Central General Hospital. All parents/caregivers were provided with detailed information about the study in writing and verbally at recruitment. Written informed consent was obtained from all caregivers.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards
Conflict of Interest Statement: PHN, MEO, DTKL, and ELP have no conflicts of interest. RJS has received lodging for The Baby Siblings Research Consortium Meeting; travel and lodging for NIH Study section reviews and invited talks at the University of Sherbrooke, Sherbrooke, Québec, Canada; the University of California Santa Cruz. Santa Cruz, California (Lodging); Epigenomics 2016, Puerto Rico (Lodging); Neurotoxicity Society & International Neurotoxicology Association, Florianópolis, Brazil; RISE 2017 Second International Meeting on Environmental Health in Strasbourg. Strasbourg, France. RJS also received Autism Speaks grant funding to develop an online autism environmental questionnaire. MM has received lodging for the Baby Siblings Research Consortium Meeting, travel and lodging for the American Psychological Association annual convention, and travel and lodging for invited talks at the HELP Group Summit.
REFERENCES
- American Psychiatric Association (1994). Diagnostic Criteria From DSM-IV. Washington DC: American Psychiatric Press Inc. [Google Scholar]
- Aylward GP (2009). Developmental screening and assessment: what are we thinking? J Dev Behav Pediatr, 30(2), 169–173, doi: 10.1097/DBP.0b013e31819f1c3e. [DOI] [PubMed] [Google Scholar]
- Baio J, Wiggins L, Christensen DL, Maenner MJ, Daniels J, Warren Z, et al. (2018). Prevalence of Autism Spectrum Disorder Among Children Aged 8 Years - Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2014. MMWR Surveill Summ, 67(6), 1–23, doi: 10.15585/mmwr.ss6706a1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Black MM, Walker SP, Fernald LC, Andersen CT, DiGirolamo AM, Lu C, et al. (2017). Early childhood development coming of age: science through the life course. Lancet, 389(10064), 77–90, doi: 10.1016/S0140-6736(16)31389-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bolte S, Poustka F, & Constantino JN (2008). Assessing autistic traits: cross-cultural validation of the social responsiveness scale (SRS). Autism Res, 1(6), 354–363, doi: 10.1002/aur.49. [DOI] [PubMed] [Google Scholar]
- Bolte S, Westerwald E, Holtmann M, Freitag C, & Poustka F (2011). Autistic traits and autism spectrum disorders: the clinical validity of two measures presuming a continuum of social communication skills. J Autism Dev Disord, 41(1), 66–72, doi: 10.1007/s10803-010-1024-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Braun JM, Froehlich T, Kalkbrenner A, Pfeiffer CM, Fazili Z, Yolton K, et al. (2014). Brief report: are autistic-behaviors in children related to prenatal vitamin use and maternal whole blood folate concentrations? J Autism Dev Disord, 44(10), 2602–2607, doi: 10.1007/s10803-014-2114-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Charman T, Baird G, Simonoff E, Loucas T, Chandler S, Meldrum D, et al. (2007). Efficacy of three screening instruments in the identification of autistic-spectrum disorders. Br J Psychiatry, 191, 554–559, doi: 10.1192/bjp.bp.107.040196. [DOI] [PubMed] [Google Scholar]
- Constantino JN (2012). (SRS™−2) Social Responsiveness Scale™, Second Edition.
- Constantino JN, Davis SA, Todd RD, Schindler MK, Gross MM, Brophy SL, et al. (2003). Validation of a brief quantitative measure of autistic traits: comparison of the social responsiveness scale with the autism diagnostic interview-revised. J Autism Dev Disord, 33(4), 427–433. [DOI] [PubMed] [Google Scholar]
- Constantino JN, & Gruber CP (2005). Social Responsiveness Scale. Los Angeles: Western Psychological Services. [Google Scholar]
- Constantino JN, Lajonchere C, Lutz M, Gray T, Abbacchi A, McKenna K, et al. (2006). Autistic social impairment in the siblings of children with pervasive developmental disorders. Am J Psychiatry, 163(2), 294–296, doi: 10.1176/appi.ajp.163.2.294. [DOI] [PubMed] [Google Scholar]
- Constantino JN, & Todd RD (2003). Autistic traits in the general population: a twin study. Arch Gen Psychiatry, 60(5), 524–530, doi: 10.1001/archpsyc.60.5.524. [DOI] [PubMed] [Google Scholar]
- Constantino JN, Todorov A, Hilton C, Law P, Zhang Y, Molloy E, et al. (2013). Autism recurrence in half siblings: strong support for genetic mechanisms of transmission in ASD. Mol Psychiatry, 18(2), 137–138, doi: 10.1038/mp.2012.9. [DOI] [PubMed] [Google Scholar]
- Constantino JN, Zhang Y, Frazier T, Abbacchi AM, & Law P (2010). Sibling recurrence and the genetic epidemiology of autism. Am J Psychiatry, 167(11), 1349–1356, doi: 10.1176/appi.ajp.2010.09101470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dang H-AH, & Glewwe PW (2017). Well Begun, but Aiming Higher: A Review of Vietnam’s Education Trends in the Past 20 Years and Emerging Challenges. RISE-WP-17/017 December 2017 https://www.riseprogramme.org/sites/www.riseprogramme.org/files/publications/RISE_WP-017_Dang%20%26%20Glewwe_0.pdf. [DOI] [PMC free article] [PubMed]
- Eremenco SL, Cella D, & Arnold BJ (2005). A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof, 28(2), 212–232, doi: 10.1177/0163278705275342. [DOI] [PubMed] [Google Scholar]
- Frankenburg WK, & Dodds JB (1967). The Denver developmental screening test. J Pediatr, 71(2), 181–191. [DOI] [PubMed] [Google Scholar]
- Grzadzinski R, Di Martino A, Brady E, Mairena MA, O’Neale M, Petkova E, et al. (2011). Examining autistic traits in children with ADHD: does the autism spectrum extend to ADHD? J Autism Dev Disord, 41(9), 1178–1191, doi: 10.1007/s10803-010-1135-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ha VS, Whittaker A, & Rodger S (2017). Assessment and Diagnosis of Autism Spectrum Disorder in Hanoi, Vietnam. J Child Fam Stud, 26, 1334–1344. [Google Scholar]
- Haraguchi H, Stickley A, Saito A, Takahashi H, & Kamio Y (2018). Stability of Autistic Traits from 5 to 8 Years of Age Among Children in the General Population. J Autism Dev Disord, doi: 10.1007/s10803-018-3770-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hus V, Bishop S, Gotham K, Huerta M, & Lord C (2013). Factors influencing scores on the social responsiveness scale. J Child Psychol Psychiatry, 54(2), 216–224, doi: 10.1111/j.1469-7610.2012.02589.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- International Test Commission (2017). The ITC Guidelines for Translating and Adapting Tests (Second edition). [www.InTestCo.org].
- Koo TK, & Li MY (2016). A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine, 15(2), 155–163, doi: 10.1016/j.jcm.2016.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krumm N, O’Roak BJ, Karakoc E, Mohajeri K, Nelson B, Vives L, et al. (2013). Transmission disequilibrium of small CNVs in simplex autism. Am J Hum Genet, 93(4), 595–606, doi: 10.1016/j.ajhg.2013.07.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Landrigan PJ, Lambertini L, & Birnbaum LS (2012). A research strategy to discover the environmental causes of autism and neurodevelopmental disabilities. Environ Health Perspect, 120(7), a258–260, doi: 10.1289/ehp.1104285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McHugh ML (2012). Interrater reliability: the kappa statistic. Biochem Med (Zagreb), 22(3), 276–282. [PMC free article] [PubMed] [Google Scholar]
- Polit DF (2014). Getting serious about test-retest reliability: a critique of retest research and some recommendations. Qual Life Res, 23(6), 1713–1720, doi: 10.1007/s11136-014-0632-9. [DOI] [PubMed] [Google Scholar]
- Robins DL, Fein D, Barton ML, & Green JA (2001). The Modified Checklist for Autism in Toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord, 31(2), 131–144. [DOI] [PubMed] [Google Scholar]
- Robinson EB, Munir K, Munafo MR, Hughes M, McCormick MC, & Koenen KC (2011). Stability of autistic traits in the general population: further evidence for a continuum of impairment. J Am Acad Child Adolesc Psychiatry, 50(4), 376–384, doi: 10.1016/j.jaac.2011.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Semrud-Clikeman M, Romero RAA, Prado EL, Shapiro EG, Bangirana P, & John CC (2017). Selecting measures for the neurodevelopmental assessment of children in low- and middle-income countries. Child Neuropsychol, 23(7), 761–802, doi: 10.1080/09297049.2016.1216536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shrout PE, & Fleiss JL (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428, doi: 10.1037/0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]
- Soto S, Linas K, Jacobstein D, Biel M, Migdal T, & Anthony BJ (2015). A review of cultural adaptations of screening tools for autism spectrum disorders. Autism, 19(6), 646–661, doi: 10.1177/1362361314541012. [DOI] [PubMed] [Google Scholar]
- Stewart LA, & Lee LC (2017). Screening for autism spectrum disorder in low- and middle-income countries: A systematic review. Autism, 21(5), 527–539, doi: 10.1177/1362361316677025. [DOI] [PubMed] [Google Scholar]
- Sturm A, Kuhfeld M, Kasari C, & McCracken JT (2017). Development and validation of an item response theory-based Social Responsiveness Scale short form. J Child Psychol Psychiatry, 58(9), 1053–1061, doi: 10.1111/jcpp.12731. [DOI] [PubMed] [Google Scholar]
- Van Cong T, Weiss B, Toan KN, Le Thu TT, Trang NT, Hoa NT, et al. (2015). Early identification and intervention services for children with autism in Vietnam. Health Psychol Rep, 3(3), 191–200, doi: 10.5114/hpr.2015.53125. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Viera AJ, & Garrett JM (2005). Understanding interobserver agreement: the kappa statistic. Fam Med, 37(5), 360–363. [PubMed] [Google Scholar]
- Wallace GL, Shaw P, Lee NR, Clasen LS, Raznahan A, Lenroot RK, et al. (2012). Distinct cortical correlates of autistic versus antisocial traits in a longitudinal sample of typically developing youth. J Neurosci, 32(14), 4856–4860, doi: 10.1523/JNEUROSCI.6214-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang J, Lee LC, Chen YS, & Hsu JW (2012). Assessing autistic traits in a Taiwan preschool population: cross-cultural validation of the Social Responsiveness Scale (SRS). J Autism Dev Disord, 42(11), 2450–2459, doi: 10.1007/s10803-012-1499-7. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supporting information Figure 1: The flow of participants
Supporting information Table 1: List of items for SRS short form
Supporting information Table 2: Accuracy of indicators for classifying autism children using total raw scores
Supporting information Table 3: Accuracy of indicators for classifying autism children using total T-scores
Supporting information Table 4: Correlation between Subscales for the Long-Form and Short-Form
Supporting information Table 5: Accuracy of indicators for classifying autism children using total raw scores for short form