Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: Autism. 2020 Sep 14;25(1):288–299. doi: 10.1177/1362361320957461

Improving Autism and Developmental Screening and Referral in U.S. Primary Care Practices Serving Latinos

Katharine E Zuckerman 1,2, Alison E Chavez 1, Laura Wilson 1, Katie Unger 3, Colleen Reuland 3, Katrina Ramsey 2, Margaret King 4, Julie Scholz 5, Eric Fombonne 6
PMCID: PMC7854844  NIHMSID: NIHMS1622522  PMID: 32921144

Abstract

Improving ASD screening and referral in primary care may reduce ASD disparities for Latino children. The REAL-START intervention aimed to increase primary care provider adherence to ASD and developmental screening guidelines, and to increase Early Intervention (EI) referral for children at developmental risk in primary care clinics serving Latinos. This quasi-experimental study enrolled 6 Oregon primary care clinics. Clinic staff attended one initial and three follow-up trainings. Trainings addressed screening, billing, referral, and follow-up issues specific to Latinos. Clinic leaders met with a quality improvement facilitator to review performance. Medical record review measured screening and referral at 18- and 24-month well-child visits at baseline and 3, 6, 9, and 12 months. State EI database queries assessed EI eligibility. Overall, 2,224 well-child visits were assessed (39% Latino). Clinics improved rates of ASD screening from 70% to 94% and general developmental screening from 62% to 95%. Adherence to screening guidelines increased from 46% to 91%. Proportion of children referred to EI was unchanged, but total referrals increased and age-range of referred children broadened. Time to EI evaluation was slightly shorter among screening-age children. REAL-START may improve screening and referral for ASD and developmental delay in Latino communities.

Lay Abstract

Latino children experience delays in access to diagnosis and treatment of autism spectrum disorder (ASD). Primary care based screening of all children for ASD and referring them for services may reduce racial/ethnic differences and improve care. REAL-START, a year-long screening intervention, was effective in increasing screening for ASD and general developmental delays, increasing therapy referrals, and shortening time for developmental assessment in primary care clinics with Latino patients

Early identification and treatment of autism spectrum disorder (ASD) is of public health importance: ASD is associated with lifetime disability, adverse family impacts (Kogan et al., 2008), and high costs (Buescher, Cidav, Knapp, & Mandell, 2014). Early intervention for children with ASD has been shown to improve child outcomes (Warren et al., 2011). Unfortunately, racial/ethnic disparities exist in early identification and treatment of autism spectrum disorder (ASD): compared to non-Latino white children, Latino children are evaluated for and diagnosed with ASD less often (Maenner, 2020), use fewer overall (Zuckerman et al., 2017) and fewer evidence-based services (Magana, Lopez, Aguinaga, & Morton, 2013), and have more unmet service needs (Iland, Weiner, & Murawski; Zuckerman et al., 2017). Improving ASD screening and referral may be one way to reduce disparities (Fein, 2016; Pierce, Courchesne, & Bacon, 2016).

Multiple health care and educational system factors impede access to ASD care for Latinos and other minorities. For instance, multiple visits and specialists are typically needed to make an ASD diagnosis, which may disadvantage families with fewer resources or lower health literacy (Goin-Kochel, Mackintosh, & Myers, 2006). Also, in the U.S., a medical diagnosis does not necessarily grant eligibility for therapy services in the educational setting such as through Individuals with Disabilities Education Act Part C Early Intervention (EI) (Zuckerman, Mattox, Sinche, Blaschke, & Bethell, 2014). As a result, families often must pursue parallel processes for medical and educational services, which may be particularly challenging for Latino families, who may have limited English proficiency or less understanding of the process (Zuckerman et al., 2017). Finally, presence, consistency or quality of health insurance (Karpur, Lello, Frazier, Dixon, & Shih, 2019) and geographic disparities in care (Kalkbrenner et al., 2011) pose systemic barriers in access to ASD care for minority populations.

Primary care providers (PCPs) play an important role in helping families overcome barriers to care. This may be particularly important for minority families: >90% of U.S. children attend primary care (U.S. Department of Health and Human Services, 2014), and PCPs may be the only developmental professional a child interacts with prior to school enrollment, particularly for families with fewer resources (National Center for Education Statistics). However, PCP bias, misinformation, and lack of training may impede access to ASD services, particularly among minorities such as Latinos: PCPs report low comfort with assessment of ASD risk overall (Carbone et al., 2013) and even less comfort assessing ASD risk in Latinos, particularly when families speak Spanish (Zuckerman et al., 2013). Latino parents of children with ASD are more likely to report their health care provider did not listen to them (Magana, Parish, Rose, Timberlake, & Swaine, 2012), which may impact how providers act on parent concerns.

Parent and family factors may also affect access to ASD care: one study showed that Latino parents report fewer autism-specific concerns than white parents as measured by a standard diagnostic tool, even when their child has high ASD risk (Vanegas, Magana, Morales, & McNamara, 2016) which may make diagnosis less accurate. Additionally, multiple studies show that Latino families may know less about ASD and the ASD diagnostic process than other families, and may have less trust in pediatric providers in the area of autism (Zuckerman, Chavez, Lindly, Regalado Murillo, & Reeder, 2018; Zuckerman et al., 2017; Zuckerman, Sinche, et al., 2014).

Improving screening, risk evaluation, and referral in primary care may facilitate access to ASD care for Latino children. However, few evidence based interventions exist for ASD risk evaluation in primary care, particularly for Latinos. American Academy of Pediatrics (AAP) guidelines recommend ASD screening (Hyman, Levy, & Myers, 2020) and general developmental screening (Lipkin & Macias, 2020) at routine visits; however, many U.S. children do not receive this screening in primary care (Arunyanart, A.; Ukritchon, S.; Imjaijitt, W.; Northrup, V.; Weitzman C., 2012; Radecki, Sand-Loud, O’Connor, Sharp, & Olson, 2011). Additionally, lower screening rates have been observed in some studies of children from Spanish-speaking families (Hirai, Kogan, Kandasamy, Reuland, & Bethell, 2018; Zuckerman et al., 2013). Thus, improvements in screening might aid earlier identification. However, improving screening is not enough: working with primary care providers regarding how to talk about screening with families, where and how to refer families with positive screening test results, and how to connect families with intervention resources is also necessary to improve access to ASD care (Hyman et al., 2020; Locke et al., 2020).

Several prior studies have addressed developmental and/or ASD screening in underserved primary care settings. Morelli and colleagues (Morelli et al., 2014) implemented an intervention promoting screening with the Ages and Stages Questionnaire (ASQ)-2 (Bricker et al., 1999) and the Modified Checklist for Autism in Toddlers (M-CHAT) (Robins, Fein, & Barton, 1999) in four clinics serving 1,397 children in Philadelphia. Over 18 months, 84% of parents completed at least one developmental screen and 9.2% were referred to EI. Similarly, Windham (Windham et al., 2014) implemented ASQ and M-CHAT screening in 1,965 children in two California clinics serving primarily Latinos. A “clinic specialist” administered screens; 84% of children received both screeners during the study. However, these studies had a number of limitations.

These limitations included lack of information on rates of screening prior to program intervention, and no analysis of the effects of the intervention on EI referral rates, which is part of the purpose of screening. In addition, neither study collected data on whether outcomes differed by family language among Latinos, even though language is known to be an important modifier of service use in children with developmental disabilities (Zuckerman et al., 2017).

Thus, it remains unclear which PCP training strategies are effective and sustainable in improving screening, referral, and eligibility outcomes for Latino children. To address this issue, we conducted a prospective quasi-experimental study in primary care clinics serving communities with Latino children, Risk Evaluation of Autism in Latinos – Screening tools and Referral Training with Practice Facilitation (REAL-START). Study goals were to improve identification and management of ASD by (1) improving autism and general developmental screening rates, (2) improving billing for screening, and (3) improving EI referral of children with positive screening tests. We hypothesized that the intervention would be associated with increased ASD and general developmental screening rates, increased billing for screening, and increased referral of children with positive screens to EI.

Methods

Enrollment

The study was conducted in six Oregon primary care clinics from 2016 to 2018, and was approved by Oregon Health & Science University Institutional Review Board. To meet initial eligibility criteria, a clinic needed to have >50% Medicaid and >25% Latino patients. We included both clinics that were not currently conducting routine autism and/or developmental screening and clinics that were currently screening. We included pediatric and family medicine clinics provided the latter had >30% child patients. Clinics were recruited through outreach by the study team and the state AAP chapter and through coordination with Medicaid Coordinated Care Organizations. 16 practices likely meeting inclusion criteria were successfully contacted; 6 ultimately enrolled. In addition to study-related supports, practices received Continuing Medical Education Credits, American Board of Pediatrics Part IV Maintenance of Certification Credits, and $1,000. Study enrollment was staggered over time. All PCPs and other practice staff participating in the study received informed consent materials at the initial training session; all staff chose to participate in the study and contribute their patients’ data.

Intervention Design

Intervention design was informed by focus groups with Latino parents of children with and without ASD and a survey of PCPs (Zuckerman et al, 2014a; Zuckerman et al, 2014b; Zuckerman et al, 2014c). In addition, the study’s Community Advisory Group met periodically throughout the study and advised on recruitment, materials, and overall direction. The group consisted of Latino and non-Latino parents of children with developmental conditions (including ASD) or typical development, health care providers (several primary care physicians, a physician assistant, a developmental pediatrician, and two medical assistants) and an educational provider (a speech-language pathologist working with Developmental Disabilities Services).

The intervention followed an academic detailing approach (Soumerai & Avorn, 1990), which is a peer-to-peer, on-site, educational outreach strategy to disseminate evidence-based practice in a targeted area. Academic detailing, which was adapted from pharmaceutical detailing (Agency for Healthcare Research and Quality & National Center for Excellence in Primary Care Research), has been used previously for developmental screening (Honigfeld, Chandhok, & Spiegelman, 2012) and other areas of clinical pediatrics (Cloutier & Wakefield, 2011; Schechter, Bernstein, Zempsky, Bright, & Willard, 2010). It also incorporated principles of quality improvement (e.g., rapid-cycle quality improvement, Plan-Do-Study-Act [PDSA] cycles) and clinic-level practice facilitation (Baskerville, Liddy, & Hogg, 2012), led by a master’s level quality specialist with expertise in implementation of developmental screening in primary care. Sample PDSA cycles included improving family completion of the ASQ-3, examining and addressing root causes of billing discrepancies, and streamlining referral processes.

The overall study structure is shown in Figure 1. Each clinic participated for 12 months; the intervention involved all practice staff, including providers, front office staff, medical assistants, nurses, and referral coordinators. Participants attended four on-site catered lunch meetings. All meetings were led in-person or via webinar by a general pediatrician with expertise in developmental and ASD screening. Informational pamphlets were provided for staff and patients regarding ASD screening and follow-up. The first 90-120 minute educational session addressed administration and scoring of the ASQ–3 (Squires, 2012) and M-CHAT – Revised with Follow-up (R/F) ( Robins et al., 2014) including use of the M-CHAT-R/F follow up interview, according to AAP guidelines in 2016. These guidelines suggested general developmental screening (i.e., ASQ-3) at 9, 18, and 24 or 30 months (Bright Futures: Guidelines for health supervision of infants, children, and adolescents, 2017), and ASD screening (i.e., M-CHAT-R/F) at 24 and 30 months.1 Both screening tools were made available in English and Spanish to enrolled practices immediately after the first session. Information was also provided on the EI referral process, billing for screening tools, and developmental screening in families with Limited English Proficiency, low literacy, and low health literacy. A parent of a child with ASD discussed his/her experience, and a community panel of EI, Developmental Disabilities Services, and county public health nursing representatives provided resources. Three 60-minute follow-up sessions occurred at 3,6, and 9 months after the initial intervention. At each follow-up session, site-level results for the three primary outcomes were reviewed and supplemental content was provided. The first session supplemental content included language development of children in bilingual households, and access to autism diagnostic and therapeutic services. The second session addressed culturally-responsive communication of screening tool results using a video-based exercise, and the third session addressed ASD management in the medical home.

Figure 1.

Figure 1.

Open in a new tab

REAL-START intervention and data collection structure

Between sessions, practice leaders held 30-minute conference calls with a master’s level practice facilitator with comprehensive knowledge to support primary care practices in implementation and follow-up aligned with primary study measures. The practice facilitator reviewed site- and provider-level data regarding the primary study outcomes, helped clinic leaders implement specific plans to improve performance, and reported progress to the study team. She identified variation in data findings and addressed gaps through implementation of specific Plan-Do-Study-Act cycles.

Measures

There were three primary outcome measures. These outcomes were considered on a clinic- and provider-level only; that is, screening results on individual children were not tracked over time.

The first primary outcome was the proportion of 18- and 24-month well visits screened for autism and general developmental delay with M-CHAT-R/F and ASQ-3 per AAP guidelines in 2016. Screening was considered complete if the provider documented the screening tool used and a score or a score interpretation, as per Medicaid billing guidelines. We did not collect data on item-level responses to the ASQ-3 or M-CHAT-R/F.

Completion of each screen was considered separately, and a composite indicator (both screens administered; complete adherence to AAP guidelines) was created. From these data, the screening proportion was calculated as the number of 18- and 24-month well visits with no screening, one screening test (either ASQ-3 or M-CHAT-R/F), or two screening tests (ASQ-3 and M-CHAT-R/F) and divided by the total number of 18- and 24-month well visits.

The second primary outcome measure was the proportion of 18- and 24-month visits that were billed for developmental and ASD screening with current procedural terminology (CPT) code 96110. Billing for screening is important in the US because it enhances financial sustainability of the activity (Earls & Hay, 2006; Pinto-Martin, Dunkle, Earls, Fliedner, & Landes, 2005). The billing proportion was calculated as the number of 18- and 24-month well visits in which 96110 was billed once (i.e., for ASQ-3 or M-CHAT-R/F) or twice (i.e. for both the ASQ-3 and the M-CHAT-R/F), divided by the total number of 18- and 24-month well visits.

The third primary outcome measure was the proportion of children with positive screens referred to EI. A positive screen was defined as an initial M-CHAT-R/F score of 3 or greater (or 2 or greater after follow-up interview), and/or an ASQ-3 result with one positive (“black zone”; >2 standard deviations below mean for age) or two borderline (“grey zone”; 1-2 standard deviations below mean for age) scores. The ASQ “positive screen” parameter is in alignment with AAP Bright Futures recommendations (Bright Futures: Guidelines for health supervision of infants, children, and adolescents, 2017). It also roughly aligns with eligibility assessment in Oregon EI: that is, generally speaking a child will be found eligible for EI in Oregon if he/she is below 2 standard deviations in any particular area or below 1.5 standard deviations in at least two areas. Compared to other states, Oregon’s EI program is considered to be moderate in terms of its EI eligibility (McManus, McCormick, Acevedo-Garcia, Ganz, & Hauser-Cram, 2009).

An EI referral was defined as present when there was a provider or staff note in the EMR indicating a referral was placed and/or a copy of an EI referral form scanned into the EMR. The proportion of visits with positive screens referred to EI was calculated as the number of 18- and 24-month visits with positive screening test results who were referred to EI, divided by the total number of 18- and 24-month visits with positive screening test results.

All primary outcome measures were collected via medical chart review, using a standardized form, by a non-clinical staff at each clinic who was trained by the study team. Ethnicity and household language were also extracted from EMR demographic fields. Data were extracted 8-12 weeks prior to the intervention (baseline period) and every 8-12 weeks thereafter (periods 1-4). Each chart review period took the staff member about one hour; no specific compensation was given to the staff member other than the general compensation given to the clinic. The research team prepared both site- and provider-level data reports for each primary outcome at each practice facilitation call.

The study had four secondary outcomes, each of which was obtained via tracking referrals from each practice to Oregon EI via an existing statewide referral form (Oregon Department of Education, 2017). We followed all referrals in the EI age range (0-3 years), not only those that resulted from screening visits. Secondary outcomes included time from referral to initial EI evaluation, time from referral to eligibility among those found eligible, proportion of evaluations resulting in eligibility, and child age at EI referral in the overall sample, and among those children ultimately found eligible. We tracked all EI referrals during the study period, and each child’s service use in EI for six months after referral. EI data as well as child and family demographics were directly extracted from the state EI database by a study team member. All outcomes were based on standard EI database fields with minimal missing data.

Data Analysis

In the initial phase of analysis, we reviewed clinic-level descriptive statistics by time period in table and graphic forms. We then used logistic regression models to estimate changes over the four intervention periods compared to the baseline (pre-intervention) period, controlling for clinic as a fixed effect. When modeling screening, we included a random intercept for provider to account for clustering; this was not feasible in the follow-up analyses of children with positive screens due to smaller numbers. Because providers might screen differently at 18 or 24 months of age, we included screening age as a binary variable. To study effects in specific ethnicity/language groups, we included terms for these three groups: Latinos with Spanish primary language, Latinos with English primary language, and non-Latinos.

Primary Outcomes

For the first primary outcome (screenings performed), we estimated the change from baseline in percent of well-child visits that had the ASQ-3, M-CHAT-R/F, or both screens. The logistic regression model, described above, included time as a set of indicator variables for intervention periods 1 through 4, with each coefficient representing the change in log-odds of screening compared to baseline. For interpretability, we translated these estimates to absolute differences in screening percentages using predicted probabilities from the logistic regression model (Kleinman & Norton, 2009). We used a similar analytic strategy for the second primary outcome, billing for screening. For the third primary outcome, we calculated both the overall number of referrals per study period and the proportion of screening tests with positive results referred to EI per study period.

Secondary Outcomes

For time (in days) from referral to (a) initial evaluation in EI and to (b) eligibility among those ultimately found eligible, we used Kaplan-Meier curves and log-rank tests at baseline and in periods 1-4. Outcomes were assessed among children of “screening age,” i.e., an age which 18- or 24-month developmental or autism screening might occur, defined as 15-28 months to account for the fact that some children attend well checks early or late. We also looked at outcomes among children of any age (i.e., the full range of 0-3 years). We limited analyses to children who ultimately received evaluations, as no dates were recorded when families declined or were unavailable. For proportion of evaluations found eligible for EI, we used descriptive statistics and Chi-square tests. For child age at evaluation, we used descriptive statistics and nonparametric tests to compare distributions, medians and interquartile ranges for the baseline period compared to periods 1-4, for all children, and also only for children ultimately found eligible.

Results

Sample

Four pediatrics and two family medicine clinics participated. Of those clinics, two had a clinic policy of conducting screening at AAP-recommended periodicity, but were not screening all patients and were also using an older version of the M-CHAT. One practice was not screening at all. The other three practices were screening either at non-standard intervals, or not screening for both ASD and general developmental delays (Table 1). There was no practice-level attrition during the study. In these practices, 47 PCPs participated. There was no provider attrition specifically from the study; however, 1 provider left her position, and 2 new providers were added midway and contributed partial data.

Table 1.

Characteristics of participating clinics

Type of clinic N or median % or range
 Pediatrics 4 67%
 Family Medicine 2 33%
County urbanicity
 Metro, >1 million population 2 33%
 Metro, 250K-1 million population 3 50%
 Metro, <250K population 1 17%
Number of providers in clinic
 Median, range 6 2 - 18
Number of patients seen in study, per clinic
 Median, range 329.5 78 - 946
Percent Latino in clinic, screening visits
 Median, range 31% 10 - 85%
Percent of referred children with Medicaid insurance type
 Median, range 73% 20 - 81%
Baseline screening practices
 Conducting developmental screening AND autism screening at AAP-recommended periodicity 2 33%
 Conducting developmental screening AND autism screening at other periodicity 2 33%
 Conducting developmental screening only at any periodicity 1 17%
 No screening 1 17%
Open in a new tab

Initially, 2,357 18- and 24-month visit records (1,157 18-month and 1,200 24-month visits) were reviewed. Of these, 134 records were excluded because the child was previously identified as having a developmental disability, missing language or ethnicity information, or no provider name recorded, resulting in a final analytic sample of 2,224 records. Although the overall percentage of Latino children in the study was relatively high (39%), two clinics had unexpectedly lower rates of Latino children of screening age (10% and 18% respectively) than predicted from baseline data. 20% (n=436) of families spoke Spanish as a primary language. EI referral data were followed for 381 children, of whom 216 were new referrals for “screening age” (15-28 month) children. Among children referred to EI, 47% were Latino and 30% spoke Spanish as a primary language. 63% were Medicaid-insured, and 37% were female. Median age at EI referral was 19 months.

Primary Outcomes

Screening Rates

Figure 2 shows change in proportion of 18- and 24-month visits with screening performed by study period (2A), and difference from baseline by study period (2B), with adjustment for visit type (18- vs 24-month), ethnicity/language group, and site. Rates for each screening (M-CHAT-R/F or ASQ-3) and both screenings (i.e., adherence to AAP guidelines) are shown. Overall, there was a significant increase in M-CHAT-R/F and ASQ-3 screening individually and both screenings together: baseline rate of complete guideline adherence was 46% of well visits, which increased to 91% by Period 4 (p<0.001). ASQ-3 screening increased from 62% to 95% (p<0.001), and M-CHAT-R/F screening increased from 70% to 94% (p=<0.001). There were no differences in screening rates by child ethnicity/language group.

Figure 2.

Figure 2.

Open in a new tab

Screening and billing for screening as proportion of 18- and 24-month well-child visits (n=2,224 visits)

Site-specific outcomes are shown in the Supplemental Figure. As the figure shows, the clinics with the lowest baseline screening rates showed the biggest improvements in screening rates; however, overall all clinics improved over the course of the intervention. One clinic decreased its screening proportion in Period 4, likely due to significant turnover of front office staff.

Billing for Screening

Figure 2 also shows billing for developmental screening over time (2C), and difference from baseline in the proportion of well visits billed for developmental screening by study period (2D). The proportion of well-child checks billed at least once (billed 96110 for ASD or developmental screening) and billed twice (billed for both screenings) is shown. Overall, billing rates for both screenings increased from 45% at baseline to 86% by Period 4 (p<0.001), adjusting for site and variables as described above. 99% of billing codes were associated with a documented ASD or developmental screen. There were no differences in billing rates by child ethnicity/language group.

Referral to EI

Figure 3 shows the proportion of children with positive screens (n=378) who were referred to EI, overall and by site. Although total number of EI referrals increased, the proportion of positive screens referred to EI did not change. The increase referral number was mainly attributable to a large increase at one site which was not previously screening and an increase in referrals for children outside of the context of 18- and 24-month visits. The referred population was 38% Latino and 21% Spanish primary language, with no difference in number or rate of referrals by ethnicity/language.

Figure 3.

Figure 3.

Open in a new tab

Proportion of at-risk screens referred to Early Intervention services (n=378 children in 6 practices)

Secondary Outcomes

Time to EI Evaluation

The increased number of referrals was not associated with delays in time from referral to evaluation in EI, either among all children who were ultimately evaluated in EI (n = 86; Figure 4A) or children of “screening age” only (n = 50; Figure 4B) in the baseline versus Period 4. 15 (38%) children in the baseline period and 21 (28%; p=0.28) in Period 4 were not evaluated, because the family declined (n=16) could not be located (n=19) or moved out of state (n=1). Median time to evaluation for all ages was 40 days at baseline and 34 days in Period 4, and for “screening age” children, 42 at baseline and 36 days in Period 4. Time to evaluation in the baseline versus Period 4 was equivalent for all children (p= 0.08) but showed evidence of improvement for screening-age children (p= .049) by the end of the intervention. Overall, 71% of children in Period 4 and 68% of children in the baseline group completed the evaluation in 45 days, which is the U.S.’s federally mandated time period under IDEA (p= 0.9), with no differences found by ethnicity/language.

Figure 4.

Figure 4.

Open in a new tab

Time from receipt of referral to evaluation (days) in the Early Intervention program, study baseline vs period 4

*P-value from log-rank test for baseline vs Period 4.

EI Evaluations Found Eligible for Services

Overall, 34% of children referred to EI, 38% of screening-age children referred, and 52% of all those who completed the initial evaluation were found eligible for EI. The proportion of children found eligible fluctuated during the study, though this may be partly attributed to small numbers, but final proportions were consistent with baseline (e.g.16/40 referred at baseline [0.40] vs 30/76 in period 4 [.39]; p= 0.5 for all children across all time periods; p=0.6 for screening age children).

Child Age at EI Evaluation

There was no change in age at initial referral in either the sample of all children, or in the sample of children who ultimately qualified for EI. However, the range of ages of children referred increased immediately after the initial intervention and this persisted throughout the intervention (Figure 5).

Figure 5.

Figure 5.

Open in a new tab

Child age at EI referral, by study period and EI eligibility status 6 months post referral

Note: each point represents one child; + = child found eligible for EI; o = child found ineligible for EI

Discussion

This study describes a multi-site ASD and developmental screening intervention implemented in primary care practices serving Latinos. The intervention was successful at raising ASD and developmental screening rates over a year’s time. Results suggest that low rates of ASD and developmental screening in practices serving Latinos may be modifiable via intervention. The intervention was also successful in increasing billing for developmental screening, which is important because improving financial compensation may be one way to for practices to sustain screening (Earls & Hay, 2006; Pinto-Martin et al., 2005). Effect sizes were highest in clinics with lowest baseline screening; however, even clinics with relatively high baseline screening showed improvement, which suggests the intervention could be effective in primary care clinics generally.

However, a screening program is only effective when screening is linked with changes in management of children with positive screens (Lipkin & Macias, 2020). In this respect, study results were mixed. Although the total number of referrals to EI increased, the proportion of children with positive screens who were referred to EI was unchanged. Additionally, though there was no change in EI eligibility rates during of the intervention, less than half of children referred to EI were eligible throughout the study. As Sheldrick and colleagues (2016) have noted, the relationship between the rate of positive screening tests, rates of PCP referral to EI, and EI eligibility rates may be complex. Providers may disregard a positive screening test if it is inconsistent with their overall clinical impression. They may also have concerns that increasing referrals may increase EI evaluations that do not result in eligibility, creating unnecessary worry or hassle for families. If providers feel that a large proportion of referrals do not result in eligibility, they may raise their referral thresholds regardless of guidance from experts or the screening tool developers (Sheldrick et al., 2016). Ultimately, study results suggest that it may be easier to change providers’ screening behaviors than their referral thresholds. More research is needed regarding both acceptability of EI evaluation to families of children who do not become eligible, and also the optimal rate of EI referral after positive developmental screens.

Though the proportion of children referred to EI did not change over time, the absolute number of referrals to EI did increase, mainly due to increased screening in a practice with lower baseline screening and increased referrals of children outside of the context of developmental screening. The age range of referrals to EI also broadened over the course of the intervention. These findings suggest this screening intervention had benefits to children who are not directly participating in the screening process, perhaps because the intervention cued providers to consider developmental status more frequently in all early childhood encounters.

The intervention unexpectedly was also associated with somewhat shorter time to EI evaluation among children of screening age. This was surprising as the intervention did not directly intervene on any part of the EI evaluation process. However, it is possible that study messaging around effective communication with families about screening tool results may have improved family EI engagement. Alternatively, this observed change could be due to environmental factors unrelated to the project (e.g., improved efficiency in EI system over time). Though the absolute change in time to evaluation (six days) was small, this change could be relevant on a systemic level.

In contrast to other research by our team (Zuckerman et al., 2013) and others (Arunyanart et al., 2012; Hirai et al., 2018) but consistent with parent-reported studies of developmental screening (Bethell, Reuland, Schor, Abrahms, & Halfon, 2011), we did not find different rates of developmental screening by race/ethnicity/language groups. This was true both at baseline and during the intervention. The intervention appeared to benefit screening and billing for non-Latinos, even though its explicit focus was on Latinos. We speculate that the majority of the training (e.g., how to conduct screening, how to talk about screening with families) would be relevant regardless of family ethnicity. Taken in context of existing literature, our findings also suggest that disparities in ASD screening for Latinos may more likely exist between practices (i.e., many practices serving Latinos are not routinely screening) rather than within them (i.e., practices selectively screen non-Latino white patients); however, more research is needed in this area. Restricting our study to only practices that were not screening might have better illuminated disparities issues; however, most Oregon practices were conducting ASD or developmental screening, at least sometimes, so generalizability would have been limited. It also would be helpful to compare this study’s findings by race/ethnicity/language, to other practices in Oregon or nationwide. This study provides hopeful evidence that by universalizing a strong ASD and developmental screening program, disparities might be reduced.

This study had limitations. First, the study took place in Oregon, where screening rates are high (Hirai et al., 2018). Findings might differ in areas with lower screening rates. In addition, we emphasized general developmental screening at 18- and 24-months, which was in compliance with AAP guidelines at the time for those visits, since 2006 recommendations allowed for screening at 18 and 24 or 30 months (Council on Children With, Section on Developmental Behavioral, Bright Futures Steering, & Medical Home Initiatives for Children With Special Needs Project Advisory, 2006). Recommending general developmental screening at 30 months (and not 24 months) was not feasible at the time of our study, because few practices were conducting 30-month visits. However, more recent guidelines have recommended screening at 30 months and not 24 months (Lipkin & Macias, 2020), and 30-month visits have become more routine. It is possible that results would differ if general developmental screening and autism screening were performed at separate visits (i.e. 30 and 24 months, respectively). Non-clinical practice staff collected practice-level data, which could have led to errors, biased reporting or a positivity bias. We relied on EMRs for race, ethnicity, and language data, which may have been inaccurate. In addition, to keep the chart review aspects of this study manageable for clinic staff, we did not collect child sex, race/ethnicity or language for non-Latinos, insurance type, or family literacy or health literacy data on screened children’s families, which might have been a mediator of screening rates. We also could not collect information about the use of the M-CHAT-R/F follow-up interview, mainly because it is not consistently documented in the EMR. However, use of the follow-up interview, which was not done at baseline at any site, and which was part of the training, may have altered referral rates—in particular it may have lowered the rate of “false positive” referrals; we were unable to measure whether this was a factor. Two practices had lower proportions of children who were Latino and on Medicaid than was initially projected (and lower than the study’s initial inclusion criteria), which may have reduced statistical power. Due to the study’s quasi-experimental design, outside factors may have impacted study outcomes. In particular, Oregon Medicaid financially incentivizes general developmental screening, and general developmental screening is tied to state Patient-Centered Primary Care Home metrics, so practices may have been especially motivated to improve developmental screening based on factors unrelated to this project. However, these state-level efforts do not directly explain the increase in autism screening we observed. We are unable to connect developmental screening test results and EI referral rates on a child-level, since identifiable data were collected only on referred children. We also could not track referrals that were not made on the state referral form; however, since use of the referral form was part of the training we suspect that we captured the vast majority. Finally, we cannot yet report on longterm outcomes of this intervention, and it is quite possible that changes in screening and billing are not durable. This was particularly evident for one clinic in our study, in which changes in office staffing led to decreases in screening.

Despite these limitations, to our knowledge, this is the first multi-site intervention focusing on autism and developmental screening in Latino populations. It was also one of the only developmental screening interventions of any type to consider EI outcomes. The intervention was effective in increasing screening rates as well as the number and age-range of referrals to EI. This is particularly important considering that the intervention was relatively low-intensity (4-6 hour time commitment total per clinic, and no changes to clinic personnel). In this sense, it may be a model that could be replicated in other states and/or adapted to other populations underserved in ASD or other developmental disabilities. In addition, the intervention could be considered in other formats, such as exclusively online. More research is also needed to understand how primary-care-based interventions such as this one impact longterm child developmental outcomes.

Supplementary Material

1

Acknowledgments

This project was funded by Autism Speaks Early Access to Care Grant 8932. Dr. Zuckerman’s effort was also supported by 1K23MH095828 from the National Institute of Mental Health. Statistical support was provided by the Oregon Clinical and Translational Research Institute (National Center for Advancing Translational Sciences of the National Institutes of Health UL1TR0002369). We acknowledge all of the community clinics who generously provided their time and energy to this project as well as the study’s Community Advisory Board for its input and guidance. We acknowledge Mauricio Gomez for help with data processing and Paulina Larenas for help with Spanish translations.

Footnotes

1

Note that in early 2020 the AAP general developmental screening guidelines changed to screening at 9, 18, and 30 months; there was no change to autism screening guidelines (Lipkin & Macias, 2020)

The authors have no conflicts of interest to declare.

References Cited

  1. Agency for Healthcare Research and Quality, & National Center for Excellence in Primary Care Research. (May, 2013). Prace facilitation handbooik: Module 10. Academic detailing as a quality improvement tool. Retrieved from https://www.ahrq.gov/ncepcr/tools/pf-handbook/mod10.html
  2. Arunyanart W, Fenick A, Ukritchon S, Imjaijitt W, Northrup V, & Weitzman C (2012). Developmental and autism screening: A survey across six states. Infants & Young Children, 25(3), 175–187. doi: 10.1097/IYC.0b013e31825a5a42 [DOI] [Google Scholar]
  3. Baskerville NB, Liddy C, & Hogg W (2012). Systematic review and meta-analysis of practice facilitation within primary care settings. The Annals of Family Medicine, 10(1), 63–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bethell C, Reuland C, Schor E, Abrahms M, & Halfon N (2011). Rates of parent-centered developmental screening: disparities and links to services access. Pediatrics, 128(1), 146–155. doi: 10.1542/peds.2010-0424; 10.1542/peds.2010-0424 [DOI] [PubMed] [Google Scholar]
  5. Bricker D, Squires J, Mounts L, Potter L, Nickel R, Twombly E, & Farrell J (1999). Ages & Stages Questionnaires: A parent-completed child-monitoring system. Baltimore, MD: Paul H. Brookes Publishing Co. [Google Scholar]
  6. Bright Futures: Guidelines for health supervision of infants, children, and adolescents. (2017). (Fourth ed.). Elk Grove Village, IL: American Academy of Pediatrics. [Google Scholar]
  7. Buescher AV, Cidav Z, Knapp M, & Mandell DS (2014). Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatrics, 168(8), 721–728. doi: 10.1001/jamapediatrics.2014.210 [doi] [DOI] [PubMed] [Google Scholar]
  8. Carbone PS, Murphy NA, Norlin C, Azor V, Sheng X, & Young PC (2013). Parent and pediatrician perspectives regarding the primary care of children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(4), 964–972. doi: 10.1007/s10803-012-1640-7 [doi] [DOI] [PubMed] [Google Scholar]
  9. Cloutier MM, & Wakefield DB (2011). Translation of a pediatric asthma-management program into a community in Connecticut. Pediatrics, 127(1), 11–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Council on Children With, D., Section on Developmental Behavioral Pediatrics Bright Futures Steering Committee & Medical Home Initiatives for Children With Special Needs Project Advisory Committee (2006). Identifying infants and young children with developmental disorders in the Medical Home: An algorithm for developmental surveillance and screening. Pediatrics, 118(1), 405–420. doi: 10.1542/peds.2006-1231 [DOI] [PubMed] [Google Scholar]
  11. Earls MF, & Hay SS (2006). Setting the stage for success: Implementation of developmental and behavioral screening and surveillance in primary care practice--the North Carolina Assuring Better Child Health and Development (ABCD) Project. Pediatrics, 118(1), e183–188. [DOI] [PubMed] [Google Scholar]
  12. Fein D (2016). Commentary on USPSTF Final Statement on Universal Screening for Autism. Journal of Developmental and Behavioral Pediatrics, 37(7), 573–578. doi: 10.1097/dbp.0000000000000345 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Goin-Kochel RP, Mackintosh VH, & Myers BJ (2006). How many doctors does it take to make an autism spectrum diagnosis? Autism, 10(5), 439–451. doi: 10.1177/1362361306066601 [DOI] [PubMed] [Google Scholar]
  14. Hirai AH, Kogan MD, Kandasamy V, Reuland C, & Bethell C (2018). Prevalence and variation of developmental screening and surveillance in early childhood. JAMA Pediatrics, 172(9), 857–866. doi: 10.1001/jamapediatrics.2018.1524 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Honigfeld L, Chandhok L, & Spiegelman K (2012). Engaging pediatricians in developmental screening: the effectiveness of academic detailing. Journal of Autism and Developmental Disorders, 42(6), 1175–1182. doi: 10.1007/s10803-011-1344-4 [DOI] [PubMed] [Google Scholar]
  16. Hyman SL, Levy SE, & Myers SM (2020). Identification, evaluation, and management of children with autism spectrum disorder. Pediatrics, 145(1), e20193447. doi: 10.1542/peds.2019-3447 [DOI] [PubMed] [Google Scholar]
  17. Iland ED, Weiner I, & Murawski WW Obstacles faced by Latina mothers of children with autism. Californian Journal of Health Promotion, 10(1), 1-12-11-12. [Google Scholar]
  18. Kalkbrenner AE, Daniels JL, Emch M, Morrissey J, Poole C, & Chen J-C (2011). Geographic access to health services and diagnosis with an autism spectrum disorder. Annals of Epidemiology, 21(4), 304–310. doi: 10.1016/j.annepidem.2010.11.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Karpur A, Lello A, Frazier T, Dixon PJ, & Shih AJ (2019). Health disparities among children with autism spectrum disorders: Analysis of the National Survey of Children’s Health 2016. Journal of Autism and Developmental Disorders, 49(4), 1652–1664. doi: 10.1007/s10803-018-3862-9 [DOI] [PubMed] [Google Scholar]
  20. Kleinman LC, & Norton EC (2009). Wha’s the Risk? A simple approach for estimating adjusted risk measures from nonlinear models including logistic regression. Health services research, 44(1), 288–302. doi: 10.1111/j.1475-6773.2008.00900.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kogan MD, Strickland BB, Blumberg SJ, Singh GK, Perrin JM, & van Dyck PC (2008). A national profile of the health care experiences and family impact of autism spectrum disorder among children in the United States, 2005-2006. Pediatrics, 122(6), e1149–1158. doi: 10.1542/peds.2008-1057; 10.1542/peds.2008-1057 [DOI] [PubMed] [Google Scholar]
  22. Lipkin PH, & Macias MM (2020). Promoting optimal development: Identifying infants and young children with developmental disorders through developmental surveillance and screening. Pediatrics, 145(1), e20193449. doi: 10.1542/peds.2019-3449 [DOI] [PubMed] [Google Scholar]
  23. Locke J, Ibanez LV, Posner E, Frederick L, Carpentier P, & Stone WL (2020). Parent perceptions about communicating with providers regarding early autism concerns. Pediatrics, 145(Supplement 1), S72–S80. doi: 10.1542/peds.2019-1895J [DOI] [PubMed] [Google Scholar]
  24. Maenner MJ (2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2016. Morbidity and Mortality Weekly Report Surveillance Summaries, 69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Magaña S, Lopez K, Aguinaga A, & Morton H (2013). Access to diagnosis and treatment services among Latino children with autism spectrum disorders. Intellect Dev Disabil, 51(3), 141–153. doi: 10.1352/1934-9556-51.3.141 [DOI] [PubMed] [Google Scholar]
  26. Magaña S, Parish SL, Rose RA, Timberlake M, & Swaine JG (2012). Racial and ethnic disparities in quality of health care among children with autism and other developmental disabilities. Intellect Dev Disabil, 50(4), 287–299. doi: 10.1352/1934-9556-50.4.287 [DOI] [PubMed] [Google Scholar]
  27. McManus B, McCormick MC, Acevedo-Garcia D, Ganz M, & Hauser-Cram P (2009). The effect of state early intervention eligibility policy on participation among a cohort of young CSHCN. Pediatrics, 124(Supplement 4), S368–S374. doi: 10.1542/peds.2009-1255G [DOI] [PubMed] [Google Scholar]
  28. Morelli DL, Pati S, Butler A, Blum NJ, Gerdes M, Pinto-Martin J, & Guevara JP (2014). Challenges to implementation of developmental screening in urban primary care: A mixed methods study. BMC Pediatrics, 14(1), 16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. National Center for Education Statistics. (May, 2016). The condition of education: Preschool enrollment. Retrieved from http://nces.ed.gov/programs/coe/indicator_cfa.asp [Google Scholar]
  30. Oregon Department of Education. (2017). Early Intervention/Early Childhood Special Education (EI/ECSE) Referral form for providers birth to age 5. Retrieved from https://www.oregon.gov/ode/students-and-family/SpecialEducation/earlyintervention/Documents/eiecsereferralformfillable.pdf
  31. Pierce K, Courchesne E, & Bacon E (2016). To screen or not to screen universally for autism is not the question: Why the Task Force got it wrong. Journal of Pediatrics, 176, 182–194. doi: 10.1016/j.jpeds.2016.06.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Pinto-Martin JA, Dunkle M, Earls M, Fliedner D, & Landes C (2005). Developmental stages of developmental screening: steps to implementation of a successful program. American Journal of Public Health, 95(11), 1928–1932. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Radecki L, Sand-Loud N, O’Connor KG, Sharp S, & Olson LM (2011). Trends in the use of standardized tools for developmental screening in early childhood: 2002–2009. Pediatrics, 128(1), 14–19. doi: 10.1542/peds.2010-2180 [DOI] [PubMed] [Google Scholar]
  34. Robins D, Fein D, & Barton M (1999). The Modified Checklist for Autism in Toddlers (M-CHAT). Storrs, CT: University of Conneticut. [Google Scholar]
  35. Robins DL, Casagrande K, Barton M, Chen C-MA, Dumont-Mathieu T, & Fein D (2014). Validation of the Modified Checklist for Autism in Toddlers, Revised with Follow-up (M-CHAT-R/F). Pediatrics, 133(1), 37–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Schechter NL, Bernstein BA, Zempsky WT, Bright NS, & Willard AK (2010). Educational outreach to reduce immunization pain in office settings. Pediatrics, 126(6), e1514–e1521. [DOI] [PubMed] [Google Scholar]
  37. Sheldrick RC, Breuer DJ, Hassan R, Chan K, Polk DE, & Benneyan J (2016). A system dynamics model of clinical decision thresholds for the detection of developmental-behavioral disorders. Implementation Science, 11(1), 156. doi: 10.1186/s13012-016-0517-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Soumerai SB, & Avorn J (1990). Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA, 263(4), 549–556. [PubMed] [Google Scholar]
  39. Squires J (2012). Ages and Stages Questionnaire (Third ed.). Baltimore, MD: Brookes Publishing Co. [Google Scholar]
  40. U.S. Department of Health and Human Services, H. R. S. A., Maternal and Child Health Bureau. (2014). The Health and Well-Being of Children: A Portrait of States and the Nation, 2011-2012. . Retrieved from Rockville, Maryland: [Google Scholar]
  41. Vanegas SB, Magana S, Morales M, & McNamara E (2016). Clinical Validity of the ADI-R in a US-Based Latino Population. J Autism Dev Disord, 46(5), 1623–1635. doi: 10.1007/s10803-015-2690-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Warren Z, Veenstra-VanderWeele J, Stone W, Bruzek J, Nahmias AS, & Foss-Feig JH (2011). [AHRQ Comparative effectiveness review #26: Therapies for children with autism spectrum disorders]. Web Page. [PubMed]
  43. Windham GC, Smith KS, Rosen N, Anderson MC, Grether JK, Coolman RB, & Harris S (2014). Autism and developmental screening in a public, primary care setting primarily serving Hispanics: Challenges and results. Journal of Autism and Developmental Disorders, 44(7), 1621–1632. doi: 10.1007/s10803-014-2032-y [DOI] [PubMed] [Google Scholar]
  44. Zuckerman KE, Chavez AE, Lindly OL, Regalado Murillo C, & Reeder JA (2018). Disparities in familiarity with developmental disabilities among low-income parents. Academic pediatrics, 18(8), 944–951. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Zuckerman KE, Lindly O, Reyes N, Chavez AE, Macias K, Smith KN, & Reynolds A (2017). Disparities in diagnosis and treatment of autism in Latino and non-Latino white families. Pediatrics, 139(5), 1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Zuckerman KE, Mattox K, Baghaee A, Batbayar O, Donelan K, & Bethell C (2013). Pediatrician identification of Latino children at risk for autism spectrum disorder. Pediatrics, 132(3), 445–453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Zuckerman KE, Mattox KM, Sinche BK, Blaschke GS, & Bethell C (2014). Racial, ethnic, and language disparities in early childhood developmental/behavioral evaluations: A narrative review. Clinical Pediatrics (Phila.), 53(7), 619–631. doi: 10.1177/0009922813501378 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Zuckerman KE, Sinche B, Mejia A, Cobian M, Becker T, & Nicolaidis C (2014). Latino parents’ perspectives on barriers to autism diagnosis. Academic pediatrics, 14(3), 301–308. doi: 10.1016/j.acap.2013.12.004 [doi] [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Zuckerman KE, Sinche B, Cobian M, Cervantes M, Mejia A, Becker T, & Nicolaidis C (2014). Conceptualization of autism in the Latino community and its relationship with early diagnosis. Journal of developmental and behavioral pediatrics, 35(8), 522–532. doi: 10.1097/DBP.0000000000000091 [doi] [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1622522-supplement-1.pdf (65.2KB, pdf)

RESOURCES