Using the M-CHAT-R/F to Identify Developmental Concerns in a High-Risk 18-Month-Old Sibling Sample

Abstract

Objective

Given the high autism spectrum disorder (ASD) recurrence risk in younger siblings, it is important to identify early ASD markers within this high risk population. Although there is increasing evidence that the Modified Checklist for Autism in Toddlers-Revised, with Follow-Up Interview can identify many low-risk children during the second year of life, there has yet to be a study of how the M-CHAT-R/F functions in a high-risk sibling population at very young ages.

Methods

As part of a larger population-based study, we screened 74 infant siblings with the Modified Checklist for Autism in Toddlers-Revised, with Follow-Up Interview at 18 months and assessed diagnoses between the ages of 18–43 months.

Results

The M-CHAT-R/F had highest positive predictive value for identifying children at risk of any developmental concern (i.e., ASD, language delay). Overall, 33% of siblings who presented for follow-up evaluations received ASD diagnoses, with an additional 22% showing other developmental concerns.

Conclusion

Failing the M-CHAT-R/F at 18 months of age raises significant concern that a child will show some degree of developmental difference or delay over time. These findings highlight the need for close developmental monitoring of this high-risk sample.

There is growing evidence that accurate, stable diagnosis of autism spectrum disorder (ASD) is possible within community settings during the second year of life^1,2 and that very young children with ASD receiving early behavioral intervention services demonstrate substantial gains in functioning.^3,4,5 As such, the American Academy of Pediatrics (AAP) recommends screening all children at 18- and 24-month well child visits and at any time a parent expresses concern.⁶ The AAP also recommends screening children if they have an older sibling with ASD, given that best estimates of ASD recurrence in sibling populations are substantially higher (varying between 7 and 19%)^7,8 than its estimated population prevalence.

Although all children later diagnosed with ASD show core deficits in social-communication and patterns of play, interest, and activities, they can present with many different symptom profiles that vary between children and over time.^9,10,11,12 These complex issues of presentation and trajectory can make it difficult to isolate ASD-specific developmental concerns, especially within the context of screening in brief pediatric visits. This is especially true for high risk subgroups, such as younger siblings of children with ASD, who are at elevated risk of developmental concerns such as speech delay¹³ as well as a broader array of other neurodevelopmental and behavioral concerns.¹⁴

Collectively, the existing data regarding the most widely used screening instruments in pediatric practice (i.e., the Modified Checklist for Autism in Toddlers; M-CHAT) and the recently published M-CHAT-Revised with Follow-up (M-CHAT-R/F), suggest that their standardized use can identify many low-risk children with ASD during the second year of life.^15,16 There is also evidence that use of self-report screening instruments is feasible within well-child settings.^15,16,17 Although in many respects this represents tremendous progress towards early detection, there has yet to be a study of how the M-CHAT-R/F functions in a high-risk sibling population at very young ages. Given the complex trajectories of these younger siblings,¹⁸ in combination with the potential of reporting biases on early self-report screening instruments (i.e. over or under reporting concerning behavior),^19,20 it is particularly important to understand the psychometrics of early ASD screening instruments for parents who already have an older child with ASD.

In the current work, we examined the utility of the newly released Modified Checklist for Autism in Toddlers-Revised, with Follow-Up Interview (M-CHAT-R/F),¹⁶ in identifying siblings of children with ASD at 18 months who would later receive ASD diagnoses. We also examined the sensitivity, specificity, and positive and negative predictive values of the M-CHAT-R/F in discriminating between three different diagnostic classifications: 1) ASD/ASD risk (ASD/R), 2) Other Developmental Concerns (OD), and 3) Typical Development (TD). These data were collected as part of a larger prospective study examining high- and low-risk infants recruited across a variety of time points (between 18–36 months of age) and geographical regions across the United States [details redacted for blind review]. This work represents preliminary findings from the high-risk subset of participants recruited from one of these project sites, with a specific focus on attempts to follow screen negative participants as well as screen positives at earliest possible point of entry (i.e., approximately 18 months of age).

METHODS

Participants

Participants were recruited from existing research registries, outpatient clinics, and community referral as part of an ongoing multi-site longitudinal study.. All participants had older siblings diagnosed with ASD as confirmed by a comprehensive report from a qualified health provider and a parent-completed Social Communication Questionnaire (SCQ).²¹ Inclusion criteria were: (1) chronological age between 16 and 21 months at time of screening and (2) an older full or half sibling with a diagnosis of ASD between the ages of 30 months and 16 years. Exclusion criteria included: (1) adoption, (2) diagnosis or physical signs of serious medical, known genetic, or neurological conditions (e.g., encephalitis, seizure disorder, Fragile X Syndrome), (3) sensory or motor impairments so severe as to preclude testing, (4) birth weight <4 pounds and/or gestational age < 33 weeks, and (5) families who spoke English to the child less than 50% of the time.

Measures

Screening

All participants were screened for autism risk using the Modified Checklist for Autism in Toddlers – Revised, with Follow-Up Interview (M-CHAT-R/F). The M-CHAT-R/F involves a two-step screening process. For the first step, parents complete a 20-item paper-and-pencil questionnaire to indicate the child’s current skills and behaviors using a yes/no format. If 3–7 items are failed, parents then receive a follow-up interview designed to collect more specific information about the failed items. Children who continue to fail two or more interview questions, or children who failed 8 or more items on the paper-and-pencil screener, are designated as being at sufficient risk to warrant further evaluation, i.e., screening failures. In this study, follow-up interviews were conducted by trained research assistants immediately after M-CHAT-R screeners were scored as failing.

Developmental Evaluation

Developmental evaluations comprised a diagnostic clinical interview based on DSM-IV-TR criteria, a cognitive assessment (Mullen Scales of Early Learning, MSEL),²² an adaptive behavior assessment (Vineland Adaptive Behavior Scales-Second Edition, VABS-II),²³ and the Autism Diagnostic Observation Schedule (ADOS).²⁴ Diagnoses were made by licensed clinicians who had obtained research reliability in ADOS administration and scoring.

Three diagnostic categories were used: Typical Development (TD), Other Developmental Concern (e.g., speech-language delay, intellectual disability; OD), and either a diagnosis of ASD or a designation of “at risk” (ASD/R). Risk designation (AR) was given when the clinician was concerned about ASD symptoms, but the child’s profile was not deemed clear enough for diagnosis due to age, significant developmental and diagnostic complexity, or other factors.

Procedure

As seen in Figure 1, 74 siblings (63% male) were screened with the M-CHAT-R at approximately 18 months of age (range: 16 months – 21 months). Initially, only children who failed the M-CHAT-R/F (n = 30) were invited for “18-month” diagnostic evaluation. Two of these participants were lost to follow-up and 5 were evaluated at later time-points, yielding a total of 23 participants (14 male) who failed the M-CHAT-R/F and completed 18-month evaluations. Approximately two years into the project, recruitment was expanded to include children who passed the M-CHAT-R/F. This sample included newly recruited 18-month-old siblings as well as those who had previously passed the M-CHAT-R/F at 18 months, and were recontacted. This procedure provided an additional 18 participants (50% male) with 18-month evaluation data, yielding a total of 41 children (58% male) with both screening and evaluation information at around 18 months.

Figure 1.

Participant recruitment, retention, and diagnostic outcome timeline.

Follow-up data were available for 22 participants (17 who passed, the M-CHAT-R/F, 5 who failed) who did not participate in 18-month evaluations but returned for diagnostic visits at 24- or 36-month time points. This yielded a total of 63 participants who were screened at 18 months with the M-CHAT-R/F, although the timing of their diagnostic evaluations was staggered between an average of 18-, 24-, and 36-months. Of the original 74 participants, 11 were completely lost to follow-up (2 who failed the M-CHAT-R/F and 9 who passed).

RESULTS

Results are presented first for only those children who received 18-month evaluations, and then for children who received later follow-up evaluations.

18-month Evaluations

Forty-one siblings completed diagnostic evaluations at the 18-month time point. See Table 1 for MSEL and VABS-II scores.

Table 1.

MSEL and VABS-II scores by 18-month diagnostic category.

Diagnosis	MSEL	VABS-II ABC
TD	101.33	98.33
ASD/R	72.09	75.55
OD	76.00	83.17

Note. ASD/R = Autism Spectrum Disorder or At Risk for Autism Spectrum Disorder. OD = Other Developmental Concern. TD = Typical Development. MSEL = Mullen Scales of Early Learning, Early Learning Composite. VABS-II ABC = Vineland Adaptive Behavior Scales – Second Edition, Adaptive Behavior Composite.

As seen in Table 2, 23 siblings failed the M-CHAT-R/F. Of those who failed, 78% had ASD/R (ASD = 13, AR =5), 17% had other developmental concerns (n = 4) and only one participant was typically developing.

Table 2.

Participant diagnostic breakdown (n) at 18-month evaluation based upon passing/failing the M-CHAT-R/F

Diagnosis	Pass M-CHAT-R/F	Fail M-CHAT-R/F
TD	11	1
ASD/R	5	18
OD	2	4

Note. ASD/R = Autism Spectrum Disorder or At Risk for Autism Spectrum Disorder. OD = Other Developmental Concern. TD = Typical Development.

To determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the M-CHAT-R/F, we compared diagnostic groups in two ways (see Table 3). We first examined the extent to which screening at 18 months identified siblings with ASD/R diagnoses versus other developmental concerns (OD) and typical development (TD). Using these categories, the M-CHAT-R/F had sensitivity of 78.26%, specificity of 72.22%, PPV of 78.26%, and NPV of 72.22%. We next examined its ability to differentiate between any developmental concerns (ASD/R + OD) and typical development (TD). This yielded a sensitivity of 75.86%, specificity of 91.67%, PPV of 95.65%, and NPV of 61.11%.

Table 3.

M-CHAT-R/F Sensitivity, Specificity, PPV, and NPV by diagnostic group: 18-month evaluations (n = 41)

Group Comparison	Sensitivity	Specificity	PPV	NPV
ASD/R vs. OD + TD	78.26%	72.22%	78.26%	72.22%
ASD/R + OD vs. TD	75.86%	91.67%	95.65%	61.11%

Note. ASD/R = Autism Spectrum Disorder or At Risk for Autism Spectrum Disorder. OD = Other Developmental Concern. TD = Typical Development.

Later/Follow-Up Evaluations

Follow-up data from evaluations conducted after 18 months are available for a total of 36 participants (n = 22 with first evaluation at 24/36 months: n = 14 with initial evaluation at 18 months plus later reevaluations). To further clarify, this sample comprises 22 children who did not complete 18-month evaluations but were evaluated as part of this or other protocols between 24 and 43 months, and 14 children who also received evaluations at 18 months. It is important to note that none of the children who were typically developing (n =12) at their 18 month evaluation returned for follow-up. Of the 14 who did return (6/13 ASD, 3/6 OD, 5/5 AR), the stability of their diagnoses was as follows: Two children originally designated as OD were diagnosed with ASD. One child changed from OD (language disorder) to TD. Of the 11 children diagnosed as ASD/R at 18 months, all children diagnosed with ASD (n = 6) retained that diagnosis. Two diagnosed as AR were diagnosed with ASD, one remained AR, and two switched to Other Developmental Concerns.

Additional data are available for 22 participants (5 who failed the M-CHAT-R/F, 17 who passed) who were not evaluated at 18 months but returned at a later time point. Of the five who failed the M-CHAT-R/F, three were TD, one was diagnosed with OD (language delay) and one was diagnosed with ASD. Of those who passed, 12 were TD, two were OD and three were diagnosed with ASD.

In sum, we have information on 63 of the 74 children originally screened with the M-CHAT-R/F around age 18 months. Final diagnoses based upon screening pass/fail status are available in Table 4. For the purpose of this table, children who completed only one evaluation but never came back retained their original diagnosis.

Table 4.

Diagnostic breakdown at final evaluation (any time point) based upon pass/fail the M-CHAT-R/F at 18 months

Diagnosis	Pass M-CHAT-R/F	Fail M-CHAT-R/F
TD	23	5
ASD/R	6	21
OD	6	2

Note. ASD/R = Autism Spectrum Disorder or At Risk for Autism Spectrum Disorder. OD = Other Developmental Concern. TD = Typical Development.

Based upon screening with the M-CHAT-R/F at age 18 months, 70% of participants who failed and almost a fifth of those who passed eventually received ASD/R diagnoses. Seven percent of those who failed and 17% of those who passed received other diagnoses indicative of developmental concern. Performance characteristics are presented in Table 5.

Table 5.

M-CHAT-R/F Sensitivity, Specificity, PPV, and NPV by diagnostic group: final outcome evaluations (n = 63)

Group Comparison	Sensitivity	Specificity	PPV	NPV
ASD/R vs. OD + TD	77.78%	80.56%	75.00%	82.86%
ASD/R + OD vs. TD	65.71%	82.14%	82.14%	65.71%

Note. ASD/R = Autism Spectrum Disorder or At Risk for Autism Spectrum Disorder. OD = Other Developmental Concern. TD = Typical Development.

DISCUSSION

We examined the utility of an 18-month screening with the M-CHAT-R/F for detecting ASD risk in a high-risk population of infant siblings. Screening identified the majority of siblings who would later receive diagnoses of developmental concerns warranting referrals to intervention services, including ASD/R, global developmental delays, and language delays. Forty percent of the siblings screened with the M-CHAT-R/F at 18 months failed, the majority of those who came in for evaluations were diagnosed with ASD/R. However, over a third of siblings who passed the M-CHAT-R/F at 18 months and came in for an evaluation also received clinical diagnoses of developmental concerns (3 ASD, 3 AR, 6 OD). This finding underscores the need for close developmental surveillance for autism as well as language and cognitive delays within this high-risk population.

Within our very young, high-risk sample, screening with the M-CHAT-R/F at age 18 months picked up broader developmental concerns in addition to ASD. Our preliminary results mirror Chlebowski et al.’s finding using the original M-CHAT/F in a large population-based sample.¹⁵ These findings suggest that 18-month-olds who fail the M-CHAT-R/F should be monitored not only for possible autism but also for broader developmental concerns or delays that would benefit from intervention services. Very few of the siblings within our study who failed the M-CHAT-R/F went on to have typical early development. It is important to note that the high risk status of this targeted research sample likely contributed to the high PPV we found relative to existing work in community samples. Additionally, ASD diagnoses remained quite stable within our sample. None of the children receiving ASD or At Risk diagnoses at their 18 month evaluation had typical developmental outcomes when seen at later follow-up evaluations. It is unknown how these numbers were influenced by the children who never completed evaluations or returned for follow-up at additional time-points.

Overall, 18 of the 30 infant siblings who failed the M-CHAT-R/F at 18 months eventually received an ASD diagnosis. Three more were designated “at risk,” with two others showing other developmental concerns. Only five siblings who failed the M-CHAT-R/F at 18 months showed typical development at later time points, although the diagnostic status of the two additional participants lost to follow-up is unknown. This finding suggests that within a high-risk sibling population, failing the M-CHAT-R/F at 18 months of age raises significant concern that a child will show some degree of developmental difference or delay over time. Additionally, 12 of the 35 siblings who passed the M-CHAT-R/F at 18 months and came in for an evaluation showed some level of developmental concern (including 3 with ASD). That is, a large portion of the children whose parents reported typical development within this time frame went on to receive diagnoses. Even if the 9 children lost to follow-up were typically developing, this still constitutes 27% of children who passed the M-CHAT-R/F showing later developmental concerns. In sum, of the 63 infant siblings evaluated, 21 (1 in 3) were diagnosed with ASD before their fourth birthdays.

This mirrors work by Ozonoff et al. (2015) who prospectively evaluated high-risk siblings from 18–36 months and found high rates of stability in 18-month ASD diagnoses, with much lower stability of 18-month non-ASD diagnoses.²⁵ That is, many siblings not diagnosed with ASD at 18 months were diagnosed at 36 months of age. As the authors note, even full diagnostic evaluations at 18 months of age may not identify many high-risk children who go on to receive ASD diagnoses,²⁵ perhaps due to the variability in symptom profile that was documented by Chawarska and others.¹⁸ Therefore, as highlighted in the current American Academy of Pediatrics recommendations,⁶ sibling risk status clearly warrants intensive monitoring by pediatricians and other members of children’s care teams. The structure of that monitoring and its implementation within-practice, however, will likely vary by provider, region, and service systems.

Future work should also examine whether the use of enhanced screening tools within practice, such as the level 2 screeners provided by AAP guidelines,⁶ enhances provider ability to identify siblings who will later receive ASD diagnoses, as has been demonstrated in small community samples.²⁶ Even if so, a lack of clearly delineated referral pathways subsequent to identification may make providers less likely to screen for ASD at all, as may difficulties fitting screening processes into billing structures and clinic schedules.²⁷ In their review of factors contributing to age of diagnosis in tertiary care settings, Bickel et al. identified several child- and family-related variables that made children more likely to receive a diagnosis before age 3, which included children’s cognitive and language abilities as well as having an older sibling with a diagnosis.²⁸ As explored in Germani et al.’s study of parental responses to sensory profiles²⁹ and Rowberry et al.’s examination of parental reported concerns on screeners for siblings,³⁰ it may be helpful for future studies to examine further the unique population of parents of children with ASD and how they report sibling concerns to providers, given their enhanced knowledge and daily experience of the autism spectrum.

Again, it should be noted that these findings reflect only a subset of participants screened using the M-CHAT-R/F. Pending analyses from the larger study of high- and low-risk children from across the United States will provide additional information regarding the M-CHAT-R/F’s PPV when screening toddlers for ASD within the broader pediatric setting and how it compares to screening properties when applied to high-risk children. Further, we fully acknowledge that these psychometric values are calculated based on the clinical referral sample that constitutes the current study, not population based screening of infant siblings, and that designations such as “ASD Risk” warrant more explicit definition and exploration. Additional work is clearly needed to yield more accurate estimates of these psychometric properties.

In summary, preliminary results from our high-risk subsample suggest that the M-CHAT-R/F continues to pick up broader developmental concerns in addition to ASD-related vulnerabilities. This finding underscores the need for ongoing provider training, not only in talking to families about developmental benchmarks, but also in recommending appropriate referral pathways for following-up on questions of ASD or other developmental disabilities – especially for the younger siblings of children on the autism spectrum. Our findings indicate that many younger siblings of children with ASD who pass initial screening go on to have ASD or other developmental concerns. Additionally, these early diagnoses were highly stable within our sample; of those children who completed follow-up evaluations, all siblings diagnosed with ASD at 18 months retained that diagnosis at follow-up. Educating families about sibling risk status and providing comprehensive developmental monitoring over time seems especially warranted in light of this and other work indicating increased risk of developmental delays within this high risk population.^7,8