The impact of using standardized autism screening on referral to specialist evaluation for young children on the autism spectrum – a cluster-randomized controlled trial

Abstract

Objective.

We tested whether the implementation of standardized, high-fidelity screening for autism during routine well-child check-ups results in (a) increasing the number of children with suspected autism referred to diagnostic evaluation, (b) lowering the age at which they are referred, and (c) facilitating autism diagnosis for children across a more diverse range of demographic backgrounds and clinical presentations, including those with subtle manifestations.

Method.

As part of a multi-site cluster randomized trial, pediatric practices were randomly assigned to an experimental condition, involving training and supervision in the universal, standardized, high-fidelity implementation of the Modified Checklist for Autism in Toddlers, Revised with Follow-Up (M-CHAT-R/F), or a usual care condition. Children in both conditions identified at high likelihood of autism during well-child visits were referred to a diagnostic evaluation conducted by clinicians naive to referral source.

Results.

Children referred to the diagnostic evaluation from the practices in the experimental condition were more numerous (n=186) and younger (mean age = 20.65 m) than those referred from the practices in the usual care condition (n=39, mean age = 23.58 m). Children referred by experimental practices who received an autism diagnosis had milder clinical presentations across measures of cognitive, language, adaptive, and social-communication functioning, compared to those referred from usual care practices. Demographic characteristics were similar across groups.

Conclusion.

Standardized, high-fidelity implementation of autism screening during pediatric well-child visits facilitates the identification of children with high autism likelihood at a younger age, including those presenting with more subtle clinical manifestations.

Promoting Positive Outcomes for Individuals With ASD: Linking Early Detection, Treatment, and Long-term Outcomes; ClinicalTrials.gov; NCT03333629

One in 36 children in the United States is diagnosed with autism.¹Although the presentation and onset patterns of autism are variable, in many cases the core diagnostic features of autism can be identified during toddlerhood.² Early diagnosis is critical to facilitate access to services and supports. A growing body of research suggests that the provision of autism-specific early interventions based on developmentally appropriate behavioral/educational strategies has the potential to mitigate several disabling manifestations frequently associated with autism, such as difficulties in social engagement and verbal and non-verbal communication, as well as cognitive and adaptive delays.³ Interventions appear to be more beneficial when delivered earlier in life.^4,5,6,7

Nevertheless, up to 70% of children on the autism spectrum experience delay in access to evidence-based intervention.^5,8 Delays in diagnosis and intervention are more likely to occur for children whose manifestations of autism are more subtle,^9,10 and for children from underprivileged communities, reflecting the poorer quality of care experienced by families from minority and/or low-income backgrounds.¹¹ For example, in a recent US study examining medical records for 709 preschoolers evaluated for autism, early intervention services were received by only half of the sample, with decreased odds of early intervention access for children identified as Black and those who were older when developmental concerns were first identified.¹¹

Earlier and more equitable access to diagnosis and intervention might be facilitated by routinely screening young children for autism during pediatric well-child healthcare visits.^12,13 The most commonly used autism-specific screening tool is the Modified Checklist for Autism in Toddlers (M-CHAT)¹⁴ and its revision, the M-CHAT, Revised with Follow-up (M-CHAT-R/F),¹⁵ which consist of 20 questions completed by caregivers, and targeted follow-up for responses consistent with a potential autism diagnosis. A meta-analysis of 50 M-CHAT(-R/F) studies reported pooled sensitivity of 0.83 and specificity of 0.94, supporting its utility in identifying children at a high likelihood of autism during pediatric visits.¹⁶ This, in turn, has the potential to facilitate earlier referral to a diagnostic evaluation, and earlier access to autism-specific intervention. Additionally, standardized screening has the potential to mitigate the phenomenon of delayed referral and diagnosis for children from underprivileged backgrounds by counteracting human biases in the identification of children in need of autism-specific diagnostic and intervention services.

Further, standardized screening may facilitate access to diagnostic and intervention services for children whose manifestations of autism are less overt, and thus more difficult to identify using only clinical judgement. For example, recent research documented that children on the autism spectrum whose language, intellectual, or adaptive functioning is not noticeably delayed are diagnosed later than those with more obvious impairments, suggesting that the absence of an obvious intellectual or language disability leads parents and professionals to overlook more subtle manifestations of autism.¹⁷ The use of screening tools such as the M-CHAT-R/F may counteract these phenomena by establishing universal and standardized procedures for examining early manifestations of autism in the general population.

The full realization of this potential, however, is hampered by inconsistent adherence to screening guidelines in community practice. Indeed, although the American Academy of Pediatrics recommends routine autism screening at 18 and 24 months,¹⁸ screening tools are often not used as prescribed with frequent deviations including selective administration to children presenting noticeable red flags, selective referrals among screen positive children, and incomplete administration.^16,19,20 These deviations decrease the screener’s ability to detect children with high likelihood of autism and might contribute to delay in access to services for those whose manifestations of autism are less obvious. Rigorous research in this area, however, is limited. Further, there is limited knowledge on the long-term outcomes of children who screen positive and those who screen negative, as well as inconsistent findings on the performance of the M-CHAT-R/F, including both specificity and sensitivity, in different settings (e.g., primary care vs tertiary clinics), samples (e.g., elevated likelihood vs general pediatric population), and based on the case confirmation timing and methods (i.e., diagnostic evaluation for toddlers vs. review of medical records when children are older).^{12,16,21,22,23} In addition, some studies have suggested that the M-CHAT may identify children with severe developmental delays rather than autism specifically, thus being potentially less effective for those with milder or more subtle presentations of autism.²⁴ Additional limitations and controversies in this field include the potential burden of universal screening on chronically under-resourced service systems, and the potential harms of classification errors, particularly for the substantial proportion of children who pass the screening despite being on the autism spectrum (false negative cases).^25,26,27,28

Furthermore, although in many areas of medical practice the adoption of screening procedures has been supported by evidence from randomized controlled trials (RCT), no research has yet used an RCT to examine the impact of the standardized use of the M-CHAT-R/F on referrals, diagnosis, and outcomes for children on the autism spectrum. This gap led the US Preventive Services Task Force to conclude that evidence for impact of autism screening in the general pediatric population is insufficient to recommend universal screening as a standard of care.²¹

The Connecting the Dots study²⁹ was designed to address this gap through a cluster randomized trial examining the impact of universal, standardized, high-fidelity screening for autism in toddlers, versus usual care, on short- and long-term outcomes. Pediatric practices were randomly assigned to either an experimental condition, which involved training and supervision in the implementation of universal, standardized, high-fidelity screening for autism using the M-CHAT-R/F paired with ongoing surveillance, or a usual care condition, which included surveillance and, at the discretion of the practices, screening (although with no support for correct administration). To ensure that in practices allocated to the experimental condition the administration of the M-CHAT-R/F was standardized (i.e., delivered in the same way to all patients) and high-fidelity (i.e., errorless use of all screening and referral procedures), a novel web-based administration system designed to prevent errors and deviations was deployed. To ensure that the administration was universal rather than selected, training followed by audits to assess compliance was implemented. Children identified as showing a high likelihood of autism through any means were then referred for a diagnostic evaluation by the study team and, if meeting criteria for autism, were offered the Early Start Denver Model (ESDM³⁰), an evidence-based naturalistic developmental behavioral program, for a 12-month period.

While analysis of the trial’s planned primary outcomes (gains in response to the ESDM) is underway and will be reported elsewhere, the current study reports the baseline characteristics of children who were identified by practices in the experimental and usual care conditions as having a high likelihood for autism and referred for a diagnostic evaluation. This allows us to address the following research questions and hypotheses:

1. Are the proportions of children referred to diagnostic evaluation different between practices assigned to the experimental condition – universal, standardized, high-fidelity screening – compared to those from the usual care group? Based on previous literature, we hypothesized a higher number of referrals from practices in the experimental condition.

2. Does the age at which children are referred to diagnostic evaluation differ between those referred from practices assigned to the experimental condition compared to those from the usual care group? We hypothesized that children referred from practices in the experimental condition would be younger than those in the usual care condition.

3. Do demographic features differ between children referred from practices assigned to the experimental condition, compared to those from the usual care condition? We hypothesized that children (including those receiving an autism diagnosis) referred from practices assigned to the experimental condition would include participants from a more diverse range of demographic backgrounds, including those from underprivileged backgrounds across indicators of race/ethnicity, socioeconomic status, and parental education, compared to those referred from the usual care condition.

4. Does the degree of cognitive, adaptive, and social-communication impairment differ between children referred from practices assigned to the experimental condition, compared to those from the usual care condition? We hypothesized that children (including those receiving an autism diagnosis) referred from practices assigned to the experimental condition would have a lower degree of impairment overall than those in the usual care condition, by virtue of including participants across a more diverse range of impairments, including those with subtler clinical manifestations.

METHOD

Study setting

The trial was conducted across three university sites: Drexel University (Philadelphia, PA), University of Connecticut (Storrs, CT), and the MIND Institute at University of California, Davis (Sacramento, CA). Pediatric practices offering primary care to toddlers and located within a one-hour drive from each site were recruited to participate. Diagnostic testing took place in university clinics.

Study design and procedure

The Connecting the Dots study was designed as a cluster-randomized controlled trial in which participating pediatric practices were clustered within university site and randomized to either (1) administer universal, standardized, high-fidelity autism screening during 18-month well-child visits using the M-CHAT-R/F coupled with developmental surveillance, or (2) detect children at high likelihood of autism via usual practices. Children in both groups for whom concerns about possible autism were identified any time between their 12- and 48- months well-child visits were referred to a diagnostic evaluation. The study was approved by the IRB at Drexel University (protocol: 1607004653) and was registered with ClinicalTrials.gov (NCT03333629). As the outcomes for these baseline analyses were not included in the trial pre-registration, our examination of the stated research questions should be characterized as exploratory.

The flowchart of the trial is illustrated in Figure 1.

Figure 1 –

CONSORT Flow Diagram for the study activities

Recruitment and eligibility

Recruitment was based on a two-tiered approach, whereby pediatric practices were recruited to participate in the study, and clinicians recruited eligible primary care patients into the study. Practices were eligible if primary care clinicians were not already using standardized, universal, high-fidelity screening to detect autism in toddlers. This included using a validated screener, such as the 2-stage M-CHAT-R/F at fidelity, and referring all screen positive children for evaluation. Of 41 interested practices, 40 were eligible (see Figure 1). Children were eligible for study enrollment if they met all of the following criteria: (a) regularly seen in enrolled pediatric practice since age 21 months old or younger, (b) legal guardian fluent in English or Spanish, (c) not suffering from a severe motor or sensory impairment that would preclude standardized testing, (d) did not have an autism diagnosis prior to enrollment in the study, and (e) date of birth between July 30, 2016 and November 15, 2018.

Practices in both groups were offered the opportunity to refer children for autism concerns beginning with their 12-month well-child visit until their 48-month well-child visit. This corresponded to an age range of 12.00 to 50.99 months, to account for late 48-month well-child visits. Within this time-window, in the experimental practices the M-CHAT-R/F was administered during 18- month well-child visits, which corresponded to children between ages 16.00–21.99 months. Practices were also encouraged, but not required, to re-administer the M-CHAT-R/F during the 24-month well-child visit.

For usual care practices, children were enrolled when a clinician indicated an autism concern for an eligible child between their 12-month and 48-month well-child visit. As with experimental practices, this corresponded to an age range of 12.00 to 50.99 months. Therefore, the time-window for referring children based on possible autism was the same in both groups.

Randomization.

Randomization was stratified by university site and practice size (small practices estimated fewer than 20 18-month visits per month and large practices estimated 20 or more 18-month visits per month) and performed by the study Data Coordinating Center using computer-generated random numbers.

Training and Monitoring of Screening.

Staff in the practices randomized to the experimental condition, including physicians, nurse-practitioners, nurses, and other office staff, received an in-person training designed to promote the universal, standardized, high-fidelity administration of the M-CHAT R/F screening during toddler check-ups. The training was modeled on the Plan-Do-Study-Act Cycle, an established method for behavioral change in organizations,³¹ and included the following components: background, significance of detecting autism in primary care, barriers and facilitators of early detection, and practical training to use the web-based screening system developed for the study. Trainings incorporated video examples and discussion to encourage active participation. Pediatric practices were then audited randomly on a 3-month cycle to compare the number of target well-visits they were billing for with the number of children screened. Those who were screening fewer than 80% of children seen in the practice were retrained in study procedures and were audited the following month. If the proportion of children screened remained below 80%, additional discussions occurred to address the ongoing barriers to universal screening. Notably, even after retraining, only one third of the practices achieved universal screening (i.e., screened 80% or more of the eligible children). Practices randomized to the “usual care” condition received an in-person visit from study personnel, who presented generic information about autism to control for the impact of the in-person visit from the study personnel, and instructions on how to refer high-likelihood autism cases to the study team.

Screening and referrals procedures – Experimental group.

Practices randomized to the experimental condition were trained to administer the M-CHAT-R/F to eligible children during their 18-month well-child visits using a web-based electronic data capture system developed for the study to ensure that the screening was standardized and administered at fidelity. The electronic data capture system was designed to eliminate errors documented frequently in the administration of the M-CHAT-R/F in the community, such as administering some but not all items, changing wording of items, changing parent answers, scoring errors, and failing to administer the Follow-Up items when initial M-CHAT-R/F scores were in the moderate likelihood range. Additionally, results of screening were immediately available to both the primary care practice (after they logged their autism concerns) and the study team to ensure standardized invitations for evaluation for all screen positive children.

Experimental participants had to enroll between the date of practice launch and May 31, 2019; end dates were extended to October 31, 2019 for University of California Davis practices, and November 15, 2019 for UConn practices to achieve target enrollment. Children could be referred to diagnostic evaluation for possible autism based on the M-CHAT-R/F or clinical judgment anytime between the 12 and the 48-month well-child visits. Parents/legal guardians completed the M-CHAT-R/F and clinicians indicated whether they had concerns about a possible autism diagnosis in the web-based system, before viewing screening results. Clinicians could enter an updated concern for children in the study cohort either based on the M-CHAT-R/F results or based on clinical judgement in future visits, until referrals closed on March 31, 2022.

Referrals procedures – Usual care group.

As with the experimental practices, the cohort of usual care children was determined based on children’s age during the study period. Clinicians could refer children anytime between the 12- and 48-month well-child visits. Usual care clinicians based their concerns on surveillance, and they could use the M-CHAT or other screening tools at their discretion, although no training or supervision was provided on how to use them correctly. Clinicians indicated to the study team whether they had concerns about possible autism via the online portal, or via fax.

COVID-19-related disruptions and adaptations.

In response to COVID-19 delays in well-child care visits, we extended the maximum age for referrals for both groups from 50.99 to 57.99 months in January 2021. This adjustment applied throughout the remainder referral period ending in March 2022. Only two children, both in the experimental group, were referred for evaluation after reaching 51 months of age. Additionally, we originally planned universal use of the M-CHAT-R/F at 48 months for both the usual care and experimental practices to identify all children on the autism spectrum not referred earlier. This plan was discontinued due to the major disruptions to primary care caused by the pandemic.

Diagnostic evaluation.

In both groups, children identified as having a high likelihood of autism were invited to receive a comprehensive, no-cost evaluation which included standardized diagnostic and developmental/behavioral assessments. The evaluation was designed to (a) determine whether the child met criteria for autism and (b) provide a baseline characterization against which to assess gains for participants who enrolled in the ESDM program offered as part of the study. Assessments were conducted by study clinicians who were blinded as to whether children were referred by experimental or usual care practices, and blinded to the reason for referral (screen result or surveillance).

All measures (see below) were administered by trained personnel supervised by licensed psychologists. Clinical best estimate diagnosis was based on all available information. Evaluations were intended to occur within two months from the date of referral, and intervention was scheduled to start within two months of diagnosis. However, due to disruptions in staffing and COVID-19-related restrictions, the evaluation occurred on average 4.25 months (SD = 6.79) after referral, with no difference between the groups (p = .86). The period between referral and evaluation was longer than two months for 57 children, which corresponds to 42.5% of all evaluated children. Additionally, there was an average delay of 5.04 months (SD = 6.37) between evaluation and ESDM start, with no difference between the groups (p = .47). The time between diagnosis and ESDM was longer than two months for 42 children, representing 61.8% of all treated children. For 17 children whose delay between diagnosis and ESDM exceeded six months, the evaluation was repeated in closer proximity to ESDM start to provide a more accurate estimate of their pre-ESDM clinical presentation. For these 17 children we are reporting data from the ‘repeated’ baseline (i.e., the evaluation closer to the ESDM start) in the Results section. The same analyses using the original baseline for these 17 participants are reported in the Supplemental Materials (Table S1 and S2, available online).

Measures

Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2³²).

The ADOS-2 is a semi-structured, standardized assessment of autism symptomatology across communication, reciprocal social interaction abilities, play, and restricted and repetitive behaviors, and yields diagnostic classifications or concern classification scores. The Toddler Module or Module 1 was administered by a research reliable clinician.

Mullen Scales of Early Learning (MSEL³³).

The MSEL is a normed measure of cognitive development for young children ages 0–68 months. Scores from fine motor, visual reception, receptive language, and expressive language create the Early Learning Composite (ELC). To avoid floor effects, developmental quotient scores (DQ; Age Equivalent scores/chronological age × 100) were created for each domain.³⁴ Non-Verbal DQ (NVDQ) averaged Visual Reception and Fine Motor DQs, and Verbal DQ (VDQ) averaged Receptive Language and Expressive Language DQs.

Vineland Adaptive Behavior Scale—3^rd Edition (VABS-3³⁵).

The VABS-3 is a semi-structured parent interview that measures adaptive functioning in four domains: communication, daily living skills, socialization, and motor. Domain scores are combined to create the Adaptive Behavior Composite score.

Participants

Forty practices meeting eligibility criteria were recruited into the study. Five practices dropped out before the launch meeting, leaving 35 practices that began the study; of these, 31 of them referred children for diagnostic evaluation (see Figure 1). Characteristics of the participating practices are reported in Table 1. Across experimental practices, 1933 children were enrolled; of these, 1744 screened negative and had no autism concerns. Although most children in the experimental practices were enrolled following the administration of the M-CHAT-R/F during the 18-month well-child visit (i.e., between 16.00–21.99 months), three children were referred before 16 months of age based on clinician concern, and fourteen were referred during well-child visits after 21.99 months (having screened negative at 18 months). After excluding three ineligible children, 186 were referred for diagnostic evaluation based on positive screen and/or clinician concern. Across usual care practices, after excluding three ineligible children, 39 were referred based on autism concerns (Figure 1).

Table 1.

Characteristics of practices that enrolled children in the Connecting the Dots trial.

	Experimental N= 19	Usual Care N= 16
Location/Site	Drexel = 9; UConn = 6; UC Davis = 4	Drexel = 5; UConn = 5; UC Davis = 6
Sizea	52.63% Small, 47.37% Large	43.75% Small, 56.25% Large
Pool of potential participantsb	2,977	3,119
Total participants referred to evaluation	186	39

^aPractice size based on estimated number of 18-month visits per month, provided by office at enrollment. Small = patient volume 20 or less; Large = patient volume more than 20.

^bTo calculate the pool of potential participants in each condition, we aggregated the total number of visits per audit, multiplied by the respective number of months of enrollment in the study after deducting a three-week lag period. Each practice underwent a varying number of audits. For six practices lacking audit data, we utilized the number of estimated 18-month visits reported at study enrollment. Only practices that referred any children were included (17 experimental, 14 usual care). One additional experimental practice was excluded from the calculation due to an extensive lag between the launch date and the date of the first 18-month screen, as well as a negligible number of children screened (n=10) during the active enrollment months.

Analytic Plan

To ensure the appropriateness of our statistical approach, the tests employed were selected following assessments of the assumptions of normality and equal variance. We first calculated the proportion of referred children from the potential participant pool for each practice (see Table 1), and then averaged these proportions across the number of practices in each condition, to avoid undue weight for practices that enrolled more children. We then tested whether the proportion of children referred to evaluation due to suspected autism out of the eligible patient population differed between experimental and usual care practices using the Wilcoxon Rank Sum test, a non-parametric alternative to the two-sample t-test. Subsequently, we employed a two-sample test of proportions with a continuity correction to assess differences between experimental and usual care practices in (a) the proportion of referred children who completed their diagnostic evaluation, (b) the proportion of evaluated children who received an autism diagnosis, and (c) the proportion of diagnosed children who enrolled in the ESDM program.

To test the hypothesis that children referred from experimental practices would be younger than those in the usual care condition, the chronological age of participants from the two groups at the time of referral was compared using the Wilcoxon Rank Sum test. Finally, to test the hypotheses that children diagnosed with autism referred from experimental practices would differ in terms of demographic backgrounds and clinical presentation compared to those in the usual care practices, the characteristics of the two groups were compared at the time of diagnosis across indicators of race/ethnicity and socioeconomic status (maternal education and income), and cognitive, adaptive, and social-communication functioning (as measured via the MSEL, VABS-3 and ADOS-2, respectively). For continuous variables, we utilized Welch’s t-test, which accounts for unequal variances, whereas for categorical variables Fisher’s exact test was employed.

RESULTS

Group differences for children referred to diagnostic evaluation

A total of 225 children were referred to diagnostic evaluation based on identified likelihood of autism. Of these, 186 children were referred from experimental practices and 39 children from usual care practices, indicating that experimental practices referred 7.38% (SD = 5.44%) of their eligible patients, which was significantly higher than the usual care practices (1.49%, SD = 0.90%; p < .001). Consistent with our hypothesis, children referred to diagnostic evaluation by experimental practices were significantly younger (mean age = 20.65 months, SD = 7.04) than those referred from usual care practices (mean age = 23.58 months, SD=10.30), p = .036.

Of the children identified as having a high likelihood of autism, 98 children from experimental practices (53% of those referred) and 36 children from usual care practices (92% of those referred) attended the diagnostic evaluation. This unexpected result indicates that significantly fewer families referred from experimental practices elected to attend the diagnostic evaluation compared to those referred from usual care practices (p < .001). Table 2 reports demographic and clinical characteristics at the time of the diagnostic evaluation.

Table 2.

Demographic and clinical characteristics for children who completed the diagnostic evaluation by group.

	Usual Care N= 36	Experimental N= 98	Effect size (95% CI)a	p valueb
Chronological Age (months) at time of evaluation – Mean (SD)	27.14 (10.69)	23.67 (8.43)	0.20 (−0.02, 0.40)	.074
Sex (% Male)	63.89	70.41	0.75 (0.31, 1.83)	.530
Race				.296
Asian %	8.33	12.24	1.07 (0.23, 6.82)	>.99
Black %	30.56	14.29	0.34 (0.11, 1.06)	.059
More Than One Race %	13.89	15.31	0.80 (0.22, 3.40)	.758
Not Reported %	13.89	12.24	0.64 (0.17, 2.8)	.518
White %	33.33	45.92	reference
Ethnicity			1.94 (0.77, 4.8)	.124
Hispanic / Latino %	36.11	22.45
Not Hispanic / Latino %	63.89	77.55
Maternal Education				.255
< HS Diploma or HS Diploma / GED %	36.11	24.49	reference
Technical/Trade School or Some College %	27.78	23.47	1.24 (0.41, 3.86)	.800
College Degree or more%	36.11	52.04	2.11 (0.77, 5.81)	.155
Income Tierc				.033
Lower	58.33	36.73	reference
Middle	19.44	35.71	2.89 (1.02, 9.09)	.041
Upper	5.56	17.35	4.87 (0.99, 47.66)	.043
Not reported	16.67	10.20	0.97 (0.27, 3.75)	>.99
MSEL scores – Mean (SD)
Visual Reception DQ	73.61 (22.47)	77.22 (22.90)	−0.16 (−0.54,0.23)	.424
Fine Motor DQ	79.76 (23.25)	82.59 (18.34)	−0.13 (−0.54, 0.27)	.469
Rec. Language DQ	49.21 (25.90)	58.15 (28.69)	−0.33 (−0.71, 0.06)	.108
Expr. Language DQ	51.16 (24.69)	60.81 (24.00)	−0.40 (−0.79, 0.00)	.045
NonVerbal DQ	76.69 (21.68)	79.90 (19.29)	−0.16 (−0.55, 0.24)	.415
Verbal DQ	50.19 (23.42)	59.48 (24.36)	−0.39 (−0.78, 0.00)	.053
ELC	64.46 (17.03)	69.02 (17.06)	−0.27 (−0.66, 0.12)	.177
VABS-3 scores – Mean (SD)
Communication SS	56.29 (24.55)	64.53 (22.04)	−0.35 (−0.76, 0.06)	.073
Daily Living Skills SS	71.51 (18.44)	75.90 (15.65)	−0.26 (−0.66, 0.15)	.182
Socialization SS	70.43 (14.86)	76.88 (14.23)	−0.44 (−0.84, −0.04)	.026
Motor Skills SS	81.85 (13.45)	83.29 (11.86)	−0.01 (−0.42, 0.41)	.972
ABC SS	66.50 (16.09)	71.56 (13.12)	−0.34 (−0.76, 0.07)	.074
ADOS-2 scores – Mean (SD)
Calibrated Severity Tot.	6.92 (3.03)	6.51 (2.71)	0.15 (−0.31, 0.60)	.504
Calibrated Social Affect	7.23 (3.25)	6.30 (2.86)	0.30 (−0.16, 0.76)	.161
Calibrated RBBs	5.92 (2.92)	6.17 (3.11)	−0.08 (−0.51, 0.35)	.717

Note - SD=Standard Deviation. SS=Standard Score. DQ=Developmental Quotient. ELC= Early Learning Composite. ABC = Adaptive Behavior Composite.

^aAs measurements of effect size we utilized the Rank Biserial Correlation for non-normally distributed continuous variables (e.g., age), odds ratios for categorical variables, and Cohen’s d for normally distributed continuous variables. For categorical variables with multiple response options (i.e., race, maternal education, and income), we computed odds ratios within 2 × 2 contingency tables. In this approach, we designated specific response options as the reference categories: “white” was used for race, “<HS diploma or HS diploma / GED” for maternal education, and “lower” for the income tier.

^bWe employed the Wilcoxon Rank Sum test for continuous variables not meeting the normality assumption (e.g., age), and the Welch’s t test for other continuous variables, accounting for unequal variances. Fisher’s exact test was used for categorical variables.

^cParticipants with an annual household income ranging from $0 to $49,999, or those who exclusively reported their monthly household income as 0 to $3,000 or more, were categorized as lower income. Participants with an annual household income between $50,000 and $119,999 were categorized as middle income, and those with an annual household income of 120,000 or more were classified as upper income.

Contrary to our hypothesis, demographic characteristics were comparable between groups, except for family income level, which was higher for the experimental than the usual care group (Table 2).

As hypothesized, the clinical presentation of participants referred by experimental practices was significantly milder than for those referred by usual care practices at the time of evaluation across language (as measured via the MSEL Expressive Language subscale) and social adaptive functioning (as measured via the VABS-3 Socialization subscale), with no differences in non-verbal cognition or autism symptoms as measured via the MSEL and the ADOS-2 (Table 2).

Group differences for referred children who received an autism diagnosis

The percent of children who received a diagnosis of autism did not differ by group: Of the 98 participants referred by experimental practices, a diagnosis of autism was confirmed for 72 children (74%); of the 36 participants referred by usual care practices who completed the diagnostic evaluations, 28 (77%) received a diagnosis of autism (group comparison p = .776).

We then assessed whether clinical and demographic characteristics differed between children referred by experimental versus usual care practices in the subset of participants for whom a diagnosis of autism was established. As reported in Table 3, contrary to our hypothesis there were no group differences in demographic characteristics, except for participants’ age, which was younger in the experimental group. Clinical presentation, as hypothesized, was milder in the experimental versus the usual care group in overall cognitive functioning (MSEL ELC), verbal cognition (MSEL VDQ), social and overall adaptive behavior (VABS-3 Socialization and ABC), and social symptoms (ADOS-2 Calibrated Social Affect; see Table 3).

Table 3.

Demographic and clinical characteristics for children who received a diagnosis of autism by group.

	Usual Care N= 28	Experimental N= 72	Effect size (95% CI)a	p valueb
Chronological Age (months) at time of evaluation – Mean (SD)	27.94 (10.75)	23.58 (7.90)	0.26 (0.01, 0.47)	.047
Sex (% Male)	63.89	72.22		.472
Race				.403
Asian %	10.71	15.28	1.17 (0.23, 8.00)	>.99
Black %	32.14	13.89	0.36 (0.09, 1.36)	.130
More Than One Race %	14.29	16.67	0.96 (0.21, 5.14)	>.99
Not Reported %	10.71	15.28	1.17 (0.23, 8.00)	>.99
White %	32.14	38.89	reference
Ethnicity				.315
Hispanic / Latino %	32.14	22.22
Not Hispanic / Latino %	67.86	77.78
Maternal Education				.414
< HS Diploma or HS Diploma / GED %	39.29	26.39	reference
Technical/Trade School or Some College %	21.43	20.83	1.44 (0.38, 5.91)	.763
College Degree or more%	39.29	52.78	1.98 (0.65, 6.11)	.202
Income Tierc				.138
Lower	57.14	34.72	reference
Middle	21.43	34.72	2.63 (0.81, 9.62)	.120
Upper	7.14	19.44	4.38 (0.83, 44.81)	.064
Not reported	14.29	11.11	1.27 (0.28, 6.77)	>.99
MSEL scores – Mean (SD)
Visual Reception DQ	67.73 (20.51)	73.60 (22.65)	−0.27 (−0.71, 0.17)	.234
Fine Motor DQ	73.59 (21.30)	80.61 (18.53)	−0.35 (−0.81, 0.11)	.111
Rec. Language DQ	40.90 (21.32)	54.05 (29.10)	−0.52 (−0.93, −0.09)	.035
Expr. Language DQ	42.53 (18.46)	57.69 (24.92)	−0.69 (−1.12, −0.26)	.005
NonVerbal DQ	70.66 (19.65)	77.10 (19.26)	−0.33 (−0.78, 0.12)	.144
Verbal DQ	41.72 (17.56)	55.87 (25.02)	−0.65 (−1.08, −0.23)	.008
ELC	58.48 (10.11)	66.66 (15.96)	−0.61 (−1.02, −0.20)	.015
VABS-3 scores – Mean (SD)
Communication SS	48.27 (20.47)	60.11 (22.20)	−0.55 (−1.01, −0.09)	.021
Daily Living Skills SS	68.26 (15.25)	76.45 (14.12)	−0.56 (−1.02, −0.09)	.015
Socialization SS	65.44 (10.31)	73.78 (13.56)	−0.69 (−1.13, −0.25)	.005
Motor Skills SS	83.38 (15.73)	83.48 (12.70)	−0.11 (−0.57, 0.35)	.612
ABC	61.38 (10.38)	69.28 (12.53)	−0.69 (−1.14, −0.23)	.006
ADOS-2 scores – Mean (SD)
Calibrated Severity Tot.	8.42 (1.57)	7.77 (1.73)	0.40 (−0.12, 0.90)	.144
Calibrated Social Affect	8.84 (1.77)	7.48 (2.15)	0.69 (0.17, 1.19)	.014
Calibrated RBBs	6.63 (2.59)	7.30 (2.28)	−0.27 (−0.80, 0.26)	.282

Note - SD=Standard Deviation. SS=Standard Score. DQ=Developmental Quotient. ELC= Early Learning Composite. ABC = Adaptive Behavior Composite.

^aAs measurements of effect size we utilized the Rank Biserial Correlation for non-normally distributed continuous variables (e.g., age), odds ratios for categorical variables, and Cohen’s d for normally distributed continuous variables. For categorical variables with multiple response options (i.e., race, maternal education, and income), we computed odds ratios within 2 × 2 contingency tables. In this approach, we designated specific response options as the reference categories: “white” was used for race, “<HS diploma or HS diploma / GED” for maternal education, and “lower” for the income tier.

^bWe employed the Wilcoxon Rank Sum test for continuous variables not meeting the normality assumption (e.g., age), and the Welch’s t test for other continuous variables, accounting for unequal variances. Fisher’s exact test was used for categorical variables.

^cParticipants with an annual household income ranging from $0 to $49,999, or those who have exclusively reported their monthly household income as 0 to $3,000 or more, were categorized as lower income. Participants with an annual household income between $50,000 and $119,999 were categorized as middle income, and those with an annual household income of 120,000 or more were classified as upper income.

Of the 82 participants diagnosed with autism referred by experimental practices, families of 49 children (60%) enrolled in the ESDM program offered as part of study participation. Of the 28 participants referred by usual care practices who received an autism diagnosis, 19 (68%) enrolled in ESDM. This difference was not statistically significant (p = .592). Demographic characteristics and clinical presentation for this subset of participants are reported in the Supplemental Materials (Table S3, available online).

DISCUSSION

In this study we report initial results from the ‘Connecting the Dots’ randomized trial. Randomization occurred at the pediatric practice level; experimental practices were provided with training and an online system to facilitate standardized, universal, high-fidelity toddler autism screening using the M-CHAT-R/F, coupled with ongoing developmental surveillance. Usual care practices engaged in surveillance and often also, as documented in eligibility interviews, used screening that deviated from universal, standardized, or high-fidelity administration. The current report focuses on the characteristics of children who were referred for a diagnostic evaluation, and among the subset who received an autism diagnosis.

Pediatric practices randomized to the experimental condition referred substantially more children than usual care practices, after accounting for patient volumes across groups. In both groups, most of the children who attended evaluation received an autism diagnosis. Children diagnosed with autism referred by the experimental practices were younger and had milder clinical features, including less severe social symptoms, and higher scores in verbal cognitive and adaptive functioning measures compared to those referred from usual care practices. Considering that many of the children on the autism spectrum in usual care were not detected during the study period, the age difference for autism diagnosis between groups is likely to be even greater.

These results are aligned with the notion that administration of standardized, high-fidelity screening for autism via the M-CHAT-R/F has an added value, compared to clinical surveillance alone (or surveillance coupled with non-standardized or low-fidelity screening), in the identification of young children at a high likelihood of autism. Our findings are also consistent with previous literature demonstrating that implementation of the M-CHAT-R/F facilitates earlier identification of children in need of diagnostic evaluation,^14,23 although to our knowledge this is the first study in which the standardized implementation of the M-CHAT-R/F was experimentally manipulated using an RCT design. Unexpectedly, however, only approximately half of the families referred by the experimental practices agreed to attend the diagnostic evaluation, whereas this step was completed by more than 90% of those referred by usual care practices. It is possible that clinicians in usual care practices expressed more urgency to families about having their child attend the diagnostic evaluation, perhaps due to increased confidence in their clinical judgement about children’s likelihood of autism (as opposed to ‘trusting’ the screening results). Thus, deploying screening as intended is not sufficient to move the needle on earlier diagnosis of autism; standardized, high-fidelity screening must be coupled with primary care clinicians’ urging families to pursue referral for diagnostic evaluation. It is also possible that other factors related to parental perception of screening results and/or child characteristics contributed to the lower proportion of families electing to receive the diagnostic evaluation despite screening positive on the M-CHAT-R/F, pointing to the need for future research on the decision-making processes underlying actions taken by caregivers following positive screening for autism in toddlers. Nonetheless, among those families from experimental practices who did attend the evaluation, autism diagnosis was established at a younger age compared to those from usual care.

Additionally, our findings were aligned with the hypothesis that children referred from experimental practices would have a lower degree of impairment than those in the usual care condition, by virtue of including participants with a wider range of clinical manifestations, including subtle delays. The hypothesis was based on the notion that when children’s presentation is less obviously atypical, parents and professionals might adopt a “wait and see” approach,¹⁷ or fail to recognize autism symptoms until children grow older and their impairments become more obvious. Our findings suggest that deploying standardized, high-fidelity screening might counteract this phenomenon, as reflected in the inclusion of children presenting more subtle manifestations of impairment in the experimental condition. Therefore, although establishing the long-term impact of our experimental manipulation on intervention outcomes needs to await longitudinal data from the ESDM component of the trial, current findings support the utility of early screening for autism across at least three dimensions. These include (a) the identification of more children on the autism spectrum younger than 4 years old, (b) the earlier identification/referral of children at high likelihood of autism, and (c) the identification of children on the autism spectrum presenting with less obvious clinical manifestations, who nevertheless are in need of targeted, intensive early intervention. Contrary to our hypothesis, however, children referred from experimental practices did not differ in terms of demographic characteristics, suggesting that in this sample, implementation of the M-CHAT-R/F did not have added value in identifying children at higher likelihood of autism from underprivileged backgrounds. However, given other findings of disparities in access to early identification of autism,³⁶ this should be explored in future research.

Our study has several limitations. First, COVID-19 disrupted several aspects of this trial. Although all experimental screening was completed prior to the pandemic, our plan to conduct universal screening at 48 months to identify all children with autism not referred earlier was impossible with the changes to primary care. This likely reduced the number of children identified at later ages in both groups. Although we originally planned to provide the diagnostic assessment to all children referred by the participating practices within two months, there was an unplanned lag between the time of referral and the time of diagnosis, which was due to the impact of COVID-19 restrictions and staffing issues. The restrictions associated with COVID-19 might also have had an impact on the decisions of participants to attend the diagnostic evaluation, thus affecting our results.

It is also important to acknowledge that the unique context of the study, which included the opportunity to receive a diagnostic evaluation as well as ESDM intervention funded through the study, may have influenced both screening rates and evaluation attendance rates, potentially limiting the generalizability of the findings. However, it is plausible that these factors impacted participants in the experimental and usual care conditions similarly. Similarly, it cannot be ruled out that unmeasured factors related to cultural backgrounds and practices (both for families and clinicians) could have impacted the screening results. Another limitation in the study design is that the universal administration of the M-CHAT-R/F in the experimental practices could not be ‘enforced’, resulting in suboptimal adherence to study protocol for many of these practices. Specifically, although we were able to control standardization and fidelity of screening using the web-based system, the universal application of screening (i.e., screening all eligible patients rather than selected ones) was less than anticipated, which may have reduced the differences between groups. Our results suggest that the inclination to selectively administer the M-CHAT-R/F to certain families rather than deploy universal screening is difficult to counteract, even in the context of a study in which primary care clinicians are explicitly trained and reminded to do so. Future research should aim to discern whether there are systematic differences in to whom the M-CHAT-R/F is administered. Relatedly, future research should systematically examine whether clinicians express more urgency when communicating to families that their child should attend the diagnostic evaluation when this recommendation is based on their clinical judgement versus ‘trusting’ the autism screening results. Further, it cannot be excluded that receiving a negative screen might discourage families and clinicians to seek evaluation if autism concerns arise later – although current guidelines emphasize ongoing surveillance in conjunction with repeated screening to maximize detection.¹⁸

Additionally, as the current study focused on a low-likelihood pediatric population, it is unclear whether findings can be generalized to other settings and high-likelihood populations, such as children referred to tertiary clinics or to early intervention services. Finally, children who screened negative were not administered a diagnostic evaluation, and were not followed up diagnostically to examine whether they met criteria for autism at a later point or received a diagnosis outside of the context of the study. Therefore the true number of children on the autism spectrum in the two groups cannot be firmly established, given the possibility of false negative cases (i.e., screen-negative cases who would have met criteria for an autism diagnosis), although this is partially mitigated by the opportunity to repeat the screening³⁷ in the experimental practices and the opportunity for clinicians in both groups to refer for autism concerns up to 50 months. However, it is possible that results of the study would have differed if all false negative cases were examined and factored in the analyses. Future research should follow-up on children who screen negative to ensure they do not meet criteria for autism (see^22,27,38 for relevant research using this approach) and address the other stated limitations in the study.

Despite these challenges, this is, to our knowledge, the first study in which the impact of toddler autism screening is examined through an RCT design. Our results indicate the utility of the M-CHAT-R/F in increasing the number and lowering the age at which children showing early signs of autism are referred to a diagnostic evaluation, and in detecting children across a broader range of manifestations within the autism spectrum, including those with less obvious clinical features. Should these findings be substantiated by future RCTs, policies and practices supporting the use of universal, standardized, high fidelity screening may offer a valuable strategy to improve timing of access to diagnostic and intervention services in the autistic population.