Abstract
Objective
To develop a clinically valid interactive level 2 screening assessment for autism spectrum disorders (ASD) in toddlers that is brief, easily administered, and scored by clinicians.
Study design
We describe the development, training, standardization, and validation of the Rapid Interactive Screening Test for Autism in Toddlers (RITA-T) with ASD-specific diagnostic instruments. The RITA-T can be administered and scored in 10 minutes. We studied the validity of the RITA-T to distinguish between toddlers with ASD from toddlers with developmental delay (DD)/non-ASD in an early childhood clinic. We also evaluated the test’s performance in toddlers with no developmental concerns. We identified a cutoff score based on sensitivity, specificity, and positive predictive value of the RITA-T that best differentiates between ASD and DD/non-ASD.
Results
A total of 61 toddlers were enrolled. RITA-T scores were correlated with ASD-specific diagnostic tools (r = 0.79; P < .01) and ASD clinical diagnoses (r = 0.77; P < .01). Mean scores were significantly different in subjects with ASD, those with DD/non-ASD, and those with no developmental concerns (20.8 vs 13 vs 10.6, respectively; P < .0001). At a cutoff score of >14 , the RITA-T had a sensitivity of 1.00, specificity of 0.84, and positive predictive value of 0.88 for identifying ASD risk in a high-risk group.
Conclusion
The RITA-T is a promising new level 2 interactive screening tool for improving the early identification of ASD in toddlers in general pediatric and early intervention settings and allowing access to treatment.
Autism spectrum disorders (ASD) are characterized by deficits in social communication skills and restricted, repetitive behaviors or interests.1 Clinical signs sometimes are evident by as early as age 12 months.2 Early identification leads to earlier and more effective interventions, which often can significantly reduce the severity of the disorder.3 Unfortunately, the average age of ASD diagnosis in the US is around 4 years4; thus, there is an urgent public health need for more efficient early identification.
The American Academy of Pediatrics recommends universal (level 1) ASD screening at age 18 and 24 months.5 Level 1 tests, such as the Modified Checklist for Autism in Toddlers (M-CHAT),6 have by design high rates of false-positive results. The positive predictive value (PPV) for ASD of the M-CHAT revised with follow-up7 is 0.54, compared with 0.99 for children with developmental delay (DD). This discrepancy leads to overreferrals for ASD evaluations, delays in treatment, increased parental anxiety, and burdens on scarce resources. A second stage or a level 2 assessment after a positive initial screen provides confirmation of ASD-specific risk.8
There are currently 2 interactive level 2 ASD screening tests: the Systematic Observation for Red Flags (SORF)9 and the Screening Tool for Autism in Two-Year-Olds (STAT)10 The SORF requires videotaping and does not easily fit into the clinical flow. The STAT takes 20 minutes to administer. It has excellent sensitivity (0.92) and specificity (0.85) in identifying ASD between age 2 years and 3 years, but may miss the diagnosis in toddlers younger than 2 years of age, and its psychometric properties are weaker in those children.11
Here we report results of a new interactive, level 2 ASD screening test, the Rapid Interactive Screening Test for Autism in Toddlers (RITA-T). This brief interactive test (5–10 minutes to administer and score) can be administered after a positive level 1 test to identify those toddlers with ASD risk. We believe that use of the RITA-T could greatly expand the availability of effective level 2 ASD screening measures, which would lead to earlier diagnosis of ASD and access to treatment. In the present study, we aimed to assess the validity and study the discriminative properties of the RITA-T in differentiating toddlers with true ASD risk from toddlers with other forms of DD/non-ASD.
Methods
The RITA-T consists of 9 interactive activities (“items”) designed to evaluate developmental constructs known to represent early signs of ASD in toddlers aged 18–36 months: joint attention,12 social awareness, reaction to emotions,2 awareness of human agency, and some fundamental cognitive skills. Each of the 9 activities (Table I) probes for 1 or more constructs based on observations of specific behaviors “triggered” by the activity and scored as “yes” (= 0; expected behavior observed) or “no” (= 1; expected behavior not observed). The sum determines the score for each of the 9 activities. The RITA-T total score is computed by adding all 9 scores, with a maximum score of 30. Higher scores reflect greater atypicality. The tool can be administered and scored within 10 minutes. Four items are timed.
Table I.
RITA-T: items, constructs measured, materials, and scoring range
Items | Constructs | Materials | Administration | Score |
---|---|---|---|---|
*A, Blocked exploration of a toy (TL: 11 s) | SA; JA; HA | Toy phone | Child explores toy. Examiner blocks it, 3 times. Observe EC and latency to EC for 11 s. | 0–4 for EC; time to EC; or giving up |
B, Object tease | SA; JA; HA | Toy phone | Examiner pretends to give toy to child then pulls back, 3 times. Observe EC to examiner or parent. | 0–2 for EC to parent, examiner, or both |
*C, Blocked vision (TL: 11 s) | SA; JA; HA | Toy and opaque screen | Child explores toy; examiner blocks toy from behind the child using a screen for 11 s. Observe EC and JA | 0–3 for EC; time to EC |
D, Object permanence | Cognition; JA | “Magic” cup and ball | Examiner shows ball in magic cup to child, then makes it disappear, 3 times. Observe surprise; JA to examiner and parent. | 0–3 for reaction of surprise; EC to parent and/or examiner |
E, Color constancy | Cognition; JA | “Magic” scarf | Examiner shows double-sided magic scarf on one side initially then changes color abruptly. Observe surprise; JA to examiner or parent. | 0–2 for reaction of surprise; JA to parent or examiner |
F, *Object vs face (TL: 15 s) | SA | Pictures of train and baby | Examiner presents a foam circle with pictures of a baby on 1 side and a train on the other side to the child for 5 s each side. Observe picture preference for 5 s. | 0–2 for preference for baby picture (0), train picture (1), or no interest at all (2) |
G, Rapid JA | JA | Ceiling light | Examiner calls child suddenly and points at ceiling light. Observe JA. | 0–1 for JA |
H*, Sad face, still face (TL: 10 s each) | SA | Caregiver | Caregiver is asked to pretend to cry. Observe distress, EC, proximity seeking, or no interest for 10 s. Then caregiver is asked to assume a neutral expression. Observe same for 10 s. | 0–4 each; score of 0–1 for each reaction observed. |
I, Recognition | Cognition; SR | Marker mirror | Examiner marks a red dot on the child’s forehead with a removable, nonallergenic marker. Examiner holds small mirror to child. Observe reaction to recognizing dot and taking it away. | 0–2 for recognizing the red dot and attempting to remove it |
EC, eye contact; HA, human agency; JA, joint attention; SA, social awareness; SR, self recognition; TL, time limit.
For each score, 0 represents the skill being expressed, and higher scores represent the skill not expressed.
Timed test.
Several items in the RITA-T probe for the response to and initiation of joint attention. We evaluate the toddler’s response to the parent’s “feigned neutral and sad” emotions. Toddlers usually respond with distress to neutral maternal faces,13 whereas toddlers with ASD may react with less distress. We also probe for “self-recognition,” a well-known social cognition skill that emerges between ages 18 and 24 months14 but is reportedly disordered in children with autism.15 Other items probe for a construct that we term “awareness of human agency.”16 We can see this when we manually interrupt toddlers during their explorations of objects. We query which draws their attention, the “thwarting hand” or the person doing the thwarting. The items come from our own personal, clinical interactions (eg, blocking activities). The RITA-T also includes cognitive tests based on “naive physics” (eg, “object permanence,” “color and shape constancy”). We measure the toddler’s awareness of what is “possible” and what is “impossible,” along with the reaction of surprise when “impossible” events occur.
We developed and tested a 3-hour training module with the Early Childhood Clinic (ECC) team at the hospital in which this study was based. The team included 8 clinicians along with research assistants (RAs) at BA and MA levels. Training consisted of observation of 3 videotaped administrations of the RITA-T, group discussions, and individual scoring of 3 other videotaped assessments. The Cohen κ statistic17 was calculated for each rater and varied between 0.7 and 1, indicating good to excellent agreement.
Measures
The Autism Diagnostic Observation Schedule-Generic (ADOS-G)18 is a norm-referenced, semistructured evaluation of the core deficits in ASD. Four modules are available based on level of expressive language (EL). An algorithm with cutoff scores defines autistic disorder (AD), ASD, and non-ASD. Toddlers in the present study received module1. Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV19 and DSM-5 criteria checklists are ASD diagnostic criteria established by the American Psychiatric Association. The M-CHAT is a level 1 ASD screening questionnaire for toddlers aged 18–36 months completed by caregivers. The previous version was used in this study and consisted of 23 yes/no questions. A toddler failed the screen if 2 critical items or 3 or more criteria were failed, and confirmed by a follow-up interview.
The Mullen Scales of Early Learning (MSEL)20 is a normative measure of development from birth to 68 months in visual reception (VR), receptive language (RL), EL, and fine motor skills. Each subscale yields a T-score (mean ± SD, 50 ± 10). The fine motor scale was not administered in this study, so an “early learning composite” was not computed.
Study Design
Once the hospital’s Institutional Review Board approved the study design, we randomly recruited 2 experimental groups: a group with no developmental concerns raised (our control group) and a group of “clinically referred” toddlers with expressed concerns of either DD or ASD. The first group, which we termed the “no concerns raised” (NCR) group, comprised 18- to 36-month-old children selected at random between March 2012 and May 2013 from general and pediatric gastroenterology clinics that happened to be near the ECC. Exclusion criteria included developmental concerns endorsed by parents or a current acute illness. The RA checked the schedules on days when she was available and solicited participation from the parents of those eligible. Parents completed the M-CHAT, and the RA administered the RITA-T. No other tests were administered.
The clinically referred toddlers ranged in age from 18 to 36 months and had been referred to the ECC owing to concerns about DD or ASD. They were enrolled at random from the clinic between June 2012 and May 2013. Toddlers presenting with known genetic syndromes associated with ASD or with active seizures were excluded.
A neurodevelopmental disabilities pediatrician, 2 developmental behavioral pediatricians, and a speech and language pathologist staffed the ECC. All had extensive experience with evaluating ASD in toddlers and administering the ADOS-G. On the day of the visit, the RA obtained informed consent from the families who agreed to participate. Parents then completed the M-CHAT. The RA administered the RITA-T at the beginning of the 2-hour evaluation, and was blinded to referral concerns. Clinical evaluations included a detailed developmental and medical history, observation of play and behavior, and directed testing. Clinicians were blinded to the results of the RITA-T and the M-CHAT. They administered the ADOS-G when clinically indicated, the MSEL subscales, and sometimes other tests (eg, Childhood Autism Rating Scale, Cognitive Adaptive Test/Clinical Linguistic Auditory Milestone Scale) to inform their assessment. The VR served as a marker of cognitive developmental level.21 The developmental behavioral pediatricians and the neurodevelopmental disabilities pediatrician completed the DSM-IV and DSM-5 checklists for all toddlers at the visit. The DSM-IV checklist has been reported to accurately inform clinical diagnoses.22 Final clinical diagnoses of ASD or DD/non-ASD were then provided by these board-certified and experienced clinicians based on the totality of their assessments (history, observation, and all testing measures).
Statistical Analyses
The toddlers were divided into 3 groups: ASD, DD/non-ASD, and NCR. Mean total scores of the RITA-T and the M-CHAT failed items were compared across the 3 groups, and the mean MSEL scores and the mean total number of criteria endorsed on the DSM-IV and DSM-5 checklists were compared in the ASD and DD/non-ASD groups using 1-way ANOVA.23 Pearson correlations between RITA-T total scores and DSM-IV and DSM-5 total criteria endorsed, ADOS-G scores, and chronological age of toddlers in the sample were computed. Sensitivity, specificity, PPV, and negative predictive value (NPV) of the RITA-T score vs the final diagnostic decision were computed for each observed total score (range, 8–27). An ideal cutoff score was identified for our sample (Table II).
Table II.
Preliminary psychometrics of the RITA-T at different cutoff scores based on DSM-5 diagnoses in this sample
RITA-T total score | Sensitivity | Specificity | PPV | NPV |
---|---|---|---|---|
8 | 1 | 0.05 | 0.56 | 1 |
9 | 1 | 0.11 | 0.58 | 1 |
10 | 1 | 0.16 | 0.59 | 1 |
11 | 1 | 0.26 | 0.62 | 1 |
12 | 1 | 0.37 | 0.66 | 1 |
13 | 1 | 0.53 | 0.72 | 1 |
14 | 1 | 0.84 | 0.88 | 1 |
15 | 0.96 | 0.84 | 0.88 | 0.94 |
16 | 0.83 | 0.89 | 0.90 | 0.81 |
17 | 0.74 | 0.95 | 0.94 | 0.75 |
18 | 0.65 | 1 | 1 | 0.70 |
19 | 0.61 | 1 | 1 | 0.68 |
21 | 0.48 | 1 | 1 | 0.61 |
22 | 0.35 | 1 | 1 | 0.56 |
23 | 0.30 | 1 | 1 | 0.54 |
24 | 0.13 | 1 | 1 | 0.49 |
25 | 0.09 | 1 | 1 | 0.48 |
26 | 0.04 | 1 | 1 | 0.46 |
27 | 0 | 1 | 1 | 0.45 |
No child with ASD scored 14 or less on the RITA-T.
Results
Seventy-four toddlers were assessed for eligibility, and 6 declined participation; thus, a total of 68 toddlers were enrolled. Of these, 61 continued in the study, including 42 with developmental concerns (23 who ultimately received an ASD diagnosis and 19 who received a DD/non-ASD diagnosis) and 19 who were NCR. Clinicians completed the DSM-IV and DSM-5 ASD criteria checklists for all 42 toddlers with developmental concerns. At the time of the study, the DSM-5 ASD criteria had not been published; the DSM-IV criteria were the benchmark for diagnosis, and were previously shown to inform diagnosis.22 Thus, both checklists were used; however, only the DSM-5 results were used to calculate the cutoff scores of the RITA-T, because those criteria are the current benchmark.
Clinicians decided whether or not to administer the ADOS-G based on the toddler’s developmental history, presenting concerns and behaviors, and the DSM criteria met. The ADOS-G module 1 was administered to all 23 toddlers who were later diagnosed with ASD and to 7 others who were later diagnosed with DD/non-ASD. The MSEL subtests (RL, EL, and VR) and/or other tests were used to inform clinical diagnoses; the choice of testing depended on the clinical staff and the availability of the speech and language pathologist. The MSEL subtests were administered to 20 of 23 toddlers in the ASD group and to 17 of 19 toddlers in the DD/non-ASD group.
Mean Differences among the 3 Groups
Table III presents demographic characteristics and test results for the 3 groups. The mean RITA-T score differed significantly among the 3 groups. The mean total score of the RITA-T was significantly higher in the ASD group than in the DD/non-ASD group (20.8 ± 3.6 [range, 15–27] vs 13 ± 2.5 [range, 8–18]; P < .0001). The mean RITA-T score also differed significantly between the ASD and the NCR groups (20.8 ± 3.6 [range, 15–27] vs 10.9 ± 2.12 [range, 7–14]; P < .0001), with no overlap of scores (ie, no NCR subject had a score higher than the lowest score of an ASD subject). Importantly, the RITA-T total scores of the NCR group and the DD/non-ASD group were not significantly different from each other. The Figure displays a scatterplot of RITA-T scores from the 3 study groups. It is interesting to note that even in the NCR group, the lowest RITA-T total score was 8.
Table III.
Psychometric test results in patient subgroups
Variables | ASD (n = 23) | DD/non-ASD (n = 19) | NCR (n = 19) | P value |
---|---|---|---|---|
Demographics | ||||
Female sex, n (%) | 1 (4) | 7 (37) | 12 (63) | <.001 |
Age, mo, mean (SD) | 27.3 (5.7) | 29.3 (5.5) | 21.7 (6.5) | .001 |
Race n (%) | .07 | |||
White | 11 (47.8) | 15 (79) | 8 (42.1) | |
Hispanic | 8 (34.7) | 3 (15.8) | 4 (21) | |
Other* | 4 (17.3) | 1 (5.2) | 7 (36.8) | |
Income, USD, n (%) | NS | |||
>50 000 | 7 (30.4) | 8 (42.1) | 6 (31.5) | |
<50 000 | 16 (69.5) | 11 (58) | 13 (68.4) | |
Test scores, mean (SD) | ||||
RITA-T, total score | 20.8 (3.6) | 13 (2.5) | 10.9 (2.12) | <.0001 |
M-CHAT, total items failed | 8.7 (4.9) | 4 (3.6) | 1.3 (1.6) | <.0001 |
M-CHAT, critical items failed | 2.87 (2.3) | 1.42 (1.6) | 0.11 (0.31) | <.0001 |
DSM IV, total hits | 8.96(1.99) | 0.89 (1.15) | NA | <.0001 |
DSM-5, total hits | 6.04 (0.88) | 0.75 (1.15) | NA | <.0001 |
MSEL RL, T score | 29.7 (12.3) (n = 20) | 33.8 (15.2) (n = 17) | NA | NS |
MSEL EL, T score | 28.4 (13.2) (n = 20) | 29.6 (13.4) (n = 17) | NA | NS |
MSEL VR, T score | 32.8 (10.7) (n = 14) | 40 (14.65) (n = 17) | NA | NS |
NS, not significant; NA, not available.
P values for group differences are based on ANOVA for continuous variables and the χ2 test for categorical variables. P < .05 indicates significance. P = NS if >.05.
Other: Asian and African American.
Figure.
Scatterplot of RITA-T Total score by diagnosis (Dx) group: ASD, DD/non-ASD, and NCR.
The mean DSM-IV and DSM-5 total criteria endorsed differed significantly between the ASD and the DD/non-ASD groups, as did the number of M-CHAT total and critical items failed. We looked at both the total items and the critical items, because this was based on the previous version of the M-CHAT. The mean T-scores of the MSEL subscales were not significantly different in the ASD and DD/non-ASD groups.
Relationships among the Different Measures
The RITA-T total score was correlated with the total number of DSM-IV and DSM-5 criteria endorsed (r = 0.78 and 0.76, respectively; n = 42; P < .0001). The RITA-T total score was also correlated with the ADOS-G total score (r = 0.79; n = 25; P < .001). We examined the difference in the average total RITA-T score of the AD group (21.3 ± 3.4) and the ASD group (16.4 ± 3.6) based on ADOS-G classification. The mean RITA-T total score differed significantly between the AD and ASD groups (P < .001). The RITA-T total score did not correlate with chronological age across all samples. The final clinical diagnoses provided were highly related to both DSM-IV and DSM-5 checklist diagnostic assignment (Yates χ2 = 29.7; P < .000001).
Assessment of the Diagnostic Accuracy of the RITA-T in This Sample
To assess candidate cutoff total scores for the RITA-T in this particular sample, we calculated sensitivity, specificity, PPV, and NPV for each obtained cutoff score between 8 and 27 for the RITA-T compared with the final clinical diagnosis (Table II). For this analysis, we tested each obtained RITA-T total score against the final diagnosis of that particular total score and determined the number of false-positive, false-negative, and true-positive diagnoses were observed. In this sample, a cutoff score of 15 (ie, a RITA-T total score of ≥15) provided the best balance of sensitivity, specificity, and PPV (sensitivity, 1; specificity, 0.84; PPV, 0.88). At this cutoff score, the NPV was 1, meaning that of the 35 toddlers with a RITA-T score of ≤14, none ultimately received a diagnosis of ASD.
Discussion
We report initial findings for the RITA-T, a new interactive level 2 ASD screening test for toddlers that we have developed. We studied its ability to differentiate between toddlers with DDs and/or ASD and toddlers for whom no developmental concerns had been raised. Overall, our results show that a RITA-T score of ≥15 can discriminate between toddlers with ASD and those with non-ASD DDs at age 18–36 months with greater sensitivity and specificity than current level 2 ASD screening tools.
There is an important need for a psychometrically valid and interactive level 2 ASD screening test that is both easily learned and readily administered by clinicians in busy clinical settings and early intervention programs. Currently, the best level 1 ASD screening test available (the M-CHAT) has a false-positive rate of 46% for ASD. Thus high rate results in overreferrals to developmental evaluation centers and long waiting lists, both of which delay early therapeutic intervention for those who truly are on the autism spectrum. The 2 currently available interactive level 2 screening tests, the SORF and the STAT, require significant training to administer and can take up to 30 minutes to perform, making their integration in clinical settings challenging. In addition, the STAT is better able to identify AD than ASD, and although it has strong psychometric properties for children aged 24–36 months, these properties are weaker in toddlers under age 24 months.
We have demonstrated that the training for reliable administration and scoring of the RITA-T is easily accomplished with approximately 3 hours of targeted training with a small group of clinicians. The RITA-T can be administered and scored within 10 minutes.
Several key findings warrant further discussion. The RITA-T differentiated extremely well between toddlers with ASD and those with DD/non-ASD; RITA-T scores were significantly higher in the ASD group than in the DD/non-ASD group (mean score, 20.8 vs 13). The RITA-T also differentiated well between toddlers with ASD and those with no apparent developmental concerns. More importantly, there was no significant overlap between the mean RITA-T scores of the DD/non-ASD and NCR groups; this finding is significant, because the DD/non-ASD and NCR groups are likely to have significantly different global measures of development, such as the MSEL VR, EL, and RL. (We expect that the NCR group likely would have scored in the normal range on the MSEL.) Thus, the RITA-T evaluates ASD-specific skills, such as social communication and social referencing, and not language delay or DDs. Toddlers with no apparent DD and toddlers with a DD or language delay but with intact social communication skills score comparably on the RITA-T.
Another key finding of this study is that the RITA-T was closely correlated with established diagnostic measures of ASD. The RITA-T correlated positively with the ADOS-G total score and with its diagnostic assignment (ie, AD vs ASD). The RITA-T total score was correlated with the DSM-IV and the DSM-5 mean total checklist items completed by clinicians who were blinded to the RITA-T test results (r = 0.78 and 0.76, respectively).
Experienced clinicians and clinical teams provided the clinical diagnoses based on developmental history, clinical observations, direct testing, and DSM-IV and DSM-5 criteria checklists. We corrected for the fact that the ADOS-G was not administered to all 42 toddlers referred for developmental or ASD concerns by using both the DSM-IV and the DSM-5 diagnostic criteria, which are more generalizable to clinical settings.
The mean chronological age of the enrolled toddlers was significantly higher in the ASD and DD/non-ASD groups compared with the NCR group (27.3 ± 5.7 months vs 29.3 ± 5.5 months vs 21.7 ± 6.5 months, respectively). Although this difference was not intentional, the reality is that toddlers with DDs and ASD concerns are more developmentally equivalent to younger toddlers matched for chronological age. Furthermore, the ASD and the DD/non-ASD groups were developmentally comparable based on scores on the MSEL subscales. Thus, the significant differences obtained with the mean RITA-T total scores are not explained by DDs or language delays. The RITA-T total score was not correlated with age in any of the 3 study groups, further supporting its validity.
It is also interesting to note that even in toddlers with no developmental concerns raised, the lowest RITA-T score obtained was 7. It is difficult to score a “perfect 0” with the current scoring algorithm, likely because the scoring of some items demands the joint attention of both the parent and the examiner. In addition, the self-recognition activity requires an average cognitive level of approximately 18 months. Although we do not report these data here, it will be important to evaluate the relative contributions of the individual items to the final score. There was a greater proportion of males in the ASD samples relative to the non-ASD samples. The extent to which this plays a role in the differences observed here is not known; however, the higher prevalence of ASD in males compared with females is well known,4 further validating the representativeness of our ASD sample.
In our sample of toddlers aged 18–36 months, a RITA-T cutoff score of ≥15 had strong psychometric properties for differentiating ASD from DD/non-ASD based on final diagnoses. Raising the cutoff score to 17–18 would increase specificity to perfection, but at a cost to sensitivity. Because we are proposing the RITA-T as a screening instrument and not as a diagnostic instrument, it is more important to have higher sensitivity (100%) than specificity (84%). In addition, these preliminary results are appealing for a level 2 ASD screening test in toddlers, but they need to be replicated in an independent referral sample. Toddlers were enrolled from a referral sample, which is appropriate because level 2 screening tests are usually administered to toddlers with developmental concerns. The setting in which the RITA-T will be most useful include early intervention centers where a delay has already been recognized, a pediatrician’s office after a child fails the M-CHAT, or settings in which a suspicion has been identified by the pediatrician, another clinician, or a parent. An important key finding that was not an original goal for this study was how well the RITA-T fits into the clinical flow of this ECC.
This study has several limitations, including the small number of toddlers enrolled. Although small samples have been reported previously for psychometric studies of screening tools in toddlers with ASD,10,11 larger samples are needed to replicate children with different clinical presentations (eg, Down syndrome) and in different ethnic and cultural groups. It also will be necessary to explore the RITA-T’s validity in younger toddlers (age 12–18 months) with developmental concerns, and to replicate the preliminary psychometric properties reported here. This effort is currently underway. In addition, we relied on clinical diagnoses in addition to objective measures to assess the participants’ performance on the RITA-T. Board-certified and experienced clinicians provided clinical diagnoses relying on history, observation, and testing. Our study design did not allow us to have 2 independent clinicians evaluating the same toddler and verifying their clinical diagnoses for consensus. The ADOS-G was administered mainly to those toddlers later diagnosed with ASD; however, the clinical decision was based on history, DSM criteria checklist, observation of play, and interactions between the clinician and the toddler. We also recognize that the administration and training of the RITA-T needs to be replicated with primary care pediatricians and other clinicians (eg, nurses, early intervention providers); this is currently underway as well.
In conclusion, these initial results for the RITA-T, a new interactive ASD level 2 screening tool, demonstrate strong psychometric properties with respect to the identification of ASD in 18- to 36-month-old toddlers identified as at risk for or with DDs. Its administration and scoring can be accomplished within only 10 minutes, and the training for scoring reliability can be achieved in as little as 3 hours. An additional advantage over the only currently available level 2 screening test, the STAT, is that we plan to have the RITA-T and its training in the public domain (ie, at no cost to clinicians). Although its properties require further validation in larger samples, the RITA-T provides a useful additional screening step in a 2-level screening model after developmental concerns are identified and before a full ASD developmental evaluation is completed. This will result in more appropriate referrals, decreased clinical evaluation waiting times, and earlier access to appropriate treatment.
Acknowledgments
Funded by CVS charitable fund (2009; 2011), Susan Saltonstall Pediatric Department (2012), and Tufts Clinical and Translational Sciences Institute (2010-UL1 RR025752). The authors plan to have the test they describe be available in the public domain when it is ready and will not have any financial revenues from it. The authors declare no conflicts of interest.
We thank Marvin Natowicz, MD (received funding for research and as a consultant from the Autism Research Institute), for his critical and detailed review of the manuscript and his helpful comments and suggestions; Jean-Francois Lemay, MD, Deborah Fein, PhD, and Linda Beatty for their comments and revisions of an earlier version of the manuscript; Steve Gullans, PhD, for his thoughtful review of the final version of the manuscript; Susan Mangan, MS, Eric Stern, BS, and Lauren Brodsky, BA, for their work on this study as Research Assistants/Coordinators; the Tufts Floating Children’s Hospital Early Childhood Clinic team (Karen Miller, MD, Nicola Smith, MD, Kathleen Reilly, CCC-SLP, Krishna Banerjee, MD, Sheryl Levy, MD, Naomi Steiner, MD, Christina Sakai, MD, Carmina Erdei, MD); Jason Nelson, MPH, for completing statistical analyses; Susan K. Parsons, MD, MRP, for advice and consultation.
Glossary
- AD
Autistic disorder
- NCR
No concerns raised
- ADOS-G
Autism Diagnostic Observation Schedule-Generic
- NPV
Negative predictive value
- PPV
Positive predictive value
- ASD
Autism spectrum disorders
- RA
Research assistant
- DD
Developmental delay
- RITA-T
Rapid Interactive Screening Test for Autism in Toddlers
- DSM
Diagnostic and Statistical Manual of Mental Disorders
- RL
Receptive language
- ECC
Early Childhood Clinic
- SORF
Systematic Observation for Red Flags
- EL
Expressive language
- M-CHAT
Modified Checklist for Autism in Toddlers
- STAT
Screening Tool for Autism in Two-Year-Olds
- MSEL
Mullen Scales of Early Learning
- VR
Visual reception
References
- 1.American Psychiatric Association. Autism spectrum disorders In: Diagnostic and statistical manual of mental disorders. 5th ed Arlington (VA): American Psychiatric Association; 2013. [Google Scholar]
- 2.Ozonoff S, Iosif AM, Baguio F, Cook IC, Hill MM, Hutman T, et al. A prospective study of the emergence of early behavioral signs of autism. J Am Acad Child Adolesc Psychiatry 2010;49:256–66. [PMC free article] [PubMed] [Google Scholar]
- 3.Howlin P, Magiati I, Charman T. Systematic review of early intensive behavioral interventions for children with autism. Am J Intellect Dev Disabil 2009;114:23–41. [DOI] [PubMed] [Google Scholar]
- 4.Centers for Disease Control and Prevention. Prevalence of autism spectrum disorder among children aged 8 years: autism and developmental disabilities monitoring network, 11 sites, United States, 2010 MMWR Surveill Summ 2014;63:1–21. [PubMed] [Google Scholar]
- 5.Johnson CP, Myers SM , American Academy of Pediatrics Council on Children with Disabilities. Identification and evaluation of children with autism spectrum disorders. Pediatrics 2007;120:1183–215. [DOI] [PubMed] [Google Scholar]
- 6.Robins DL, Dumont-Mathieu TM. Early screening for autism spectrum disorders: update on the Modified Checklist for Autism in Toddlers and other measures. J Dev Behav Pediatr 2006;27(2 Suppl):S111–9. [DOI] [PubMed] [Google Scholar]
- 7.Robins DL, Casagrande K, Barton M, Chen CA, Dumont-Mathieu T, Fein D. Validation of the Modified Checklist for Autism in Toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics 2014; 133:37–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Oosterling IJ, Wensing M, Swinkels SH, van der Gaag RJ, Visser JC, Woudenberg T, et al. Advancing early detection of autism spectrum disorder by applying an integrated two-stage screening approach. J Child Psychol Psychiatry 2010;51:250–8. [DOI] [PubMed] [Google Scholar]
- 9.Wetherby A, Wood J, Allen L, Cleary J, Dickinson H, Lord C. Early indicators of autism spectrum disorders in the second year of life. J Autism Dev Disord 2004;34:473–93. [DOI] [PubMed] [Google Scholar]
- 10.Stone WL, Coonrod EE, Turner LM, Pozdol SL. Psychometric properties of the STAT for early autism screening. J Autism Dev Disord 2006;34: 691–701. [DOI] [PubMed] [Google Scholar]
- 11.Stone WL, McMahon CR, Henderson LM. Use of the Screening Tool for Autism in Two-Year-Olds (STAT) for children under 24 months: an exploratory study. Autism 2008;12:557–73. [DOI] [PubMed] [Google Scholar]
- 12.Rozga A, Hutman T, Young GS, Rogers S, Ozonoff S, Dapretto M, et al. Behavioral profiles of affected and unaffected siblings of children with autism: contribution of measures of mother–infant interaction and nonverbal communication. J Autism Dev Disord 2011;41: 287–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Moore GA. Developmental change and individual differences in infant affective behaviors in the still-face paradigm and relations with infant attachment and toddler problem behaviors. Diss Abstr Int B Sci Eng 2001;61:4995. [Google Scholar]
- 14.Brooks-Gunn J, Lewis M. The development of early visual recognition. Dev Rev 1984;4:215–39. [Google Scholar]
- 15.Reddy V, Williams E, Costantini C, Lan B. Engaging with the self: mirror behaviour in autism, Down syndrome and typical development. Autism 2010;14(5):531–54. [DOI] [PubMed] [Google Scholar]
- 16.Meltzoff AN. Understanding the intentions of others: re-enactment of intended acts by 18-month-old children. Dev Psychol 1995;31:838–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Cohen JA. Coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46. [Google Scholar]
- 18.Lord C, Rutter M, DiLavore P, Risi S. Autism Diagnostic Observation Schedule (ADOS) manual. Los Angeles (CA): Western Psychological Services; 2001. [Google Scholar]
- 19.American Psychiatric Association. Pervasive developmental disorders In: Diagnostic and statistical manual of mental disorders, fourth edition text revision (DSM-IV-TR). Washington (DC): American Psychiatric Association; 2000. [Google Scholar]
- 20.Mullen EM. Mullen Scales of Early Learning. Circle Pines (MN): American Guidance Services; 1995. [Google Scholar]
- 21.Caudle SE, Katzenstein JM, Oghalai JS, Caudle DD. Nonverbal cognitive development in children with cochlear implants: relationship between the Mullen Scales of Early Learning and later performance on the Leiter International Performance Scales-Revised. Assessment 2014;2:119–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Klin A, Lang J, Cicchetti DV, Volkmar FR. Interrater reliability of clinical diagnosis and DSM-IV criteria for autistic disorder: results of the DSM-IV autism field trial. J Autism Dev Disord 2000;30:163–7. [DOI] [PubMed] [Google Scholar]
- 23.Snedecor GW, Cochran WG. Statistical methods. 8th ed Ames (IA): Iowa State University Press; 1984. [Google Scholar]