Effectiveness of the Extension for Community Health Outcomes Model as Applied to Primary Care for Autism

Key Points

Question

Is the Extension for Community Health Outcomes (ECHO) telementoring model effective in improving clinical practice among primary care clinicians caring for children with autism?

Findings

In this large-scale, multisite partial stepped-wedge study of the ECHO model as applied to autism, significant changes in autism screening or treatment of comorbidities were not demonstrated. However, primary care clinicians demonstrated significant improvements in knowledge and self-efficacy immediately following and 3 months after completing the ECHO program.

Meaning

The results provide support for the ECHO model in improving clinician knowledge and confidence in caring for patients with autism in primary care practices, but measurable changes in practice were not observed.

Abstract

Importance

The Extension for Community Health Outcomes (ECHO) model is a widely adopted technology-based model for training primary care physicians and practitioners (PCPs) to care for patients with complex conditions. Despite its popularity, to our knowledge, direct effects of ECHO on clinical practice have not been tested in a large-scale study.

Objective

To test the effectiveness of the ECHO model as applied to primary care for autism and whether it resulted in improved clinical practice, knowledge, and self-efficacy regarding autism screening and comorbidity management.

Design, Setting, and Participants

Primary care physicians and practitioners were recruited to participate in a 6-month ECHO Autism program delivered by 1 of 10 academic medical center sites. A sequential, staggered rollout of ECHO Autism was delivered to 5 cohorts of participants (15 per site; 2 sites per cohort). Sites were randomized after recruitment to cohort/start time. Cohorts launched every 3 months. The ECHO Autism program used videoconferencing technology to connect community-based PCPs with interdisciplinary expert teams at academic medical centers. There were 148 participants (PCPs [family practice physicians, pediatricians, nurse practitioners, and physician assistants] providing outpatient services to underserved children) studied between December 2016 and November 2018.

Interventions

The 6-month ECHO Autism program included twelve 2-hour sessions connecting PCP participants with an interdisciplinary expert team. Sessions included didactics, case-based learning, guided practice, and discussion.

Main Outcomes and Measures

Coprimary outcomes were autism screening practices and comorbidity management (assessed by medical record review). Secondary outcomes were knowledge (assessed by direct testing) and self-efficacy (assessed by self-report survey). Assessments were conducted at baseline, mid-ECHO, post-ECHO, and follow-up (3 months after ECHO).

Results

Ten sites were randomized to 1 of 5 cohorts. Participants were 82% female (n = 108), 76% white (n = 100), and 6% Hispanic or Latino (n = 8); the median age was 46 years (interquartile range, 37-55 years). Significant changes in autism screening and treatment of comorbidities in children with autism were not observed. Participants demonstrated significant improvements in knowledge (9%; 95% CI, 4-13; P < .001) and self-efficacy (29%; 95% CI, 25-32; P < .001).

Conclusions and Relevance

The ECHO model was developed to increase access to high-quality health care for underserved patients with complex conditions. Study results provide support for the model in improving clinician knowledge and confidence but little support for achieving practice change.

Trial Registration

ClinicalTrials.gov Identifier: NCT03677089

This randomized clinical trial tests the effectiveness of the Extension for Community Health Outcomes model as applied to primary care for autism and whether it resulted in improved clinical practice, knowledge, and self-efficacy regarding autism screening and comorbidity management.

Introduction

Despite rapid advancements in medical knowledge, patients with complex conditions from rural and underserved areas face significant barriers in accessing specialty care.¹ The Extension for Community Health Outcomes (ECHO) model was developed to address this gap by infusing specialty knowledge into local primary care practices.² Through ECHO, groups of community-based primary care physicians and practitioners (PCPs) connect with a team of experts using secure videoconferencing and receive best-practice knowledge through didactics, case-based learning, comanagement, and a virtual community of practice.^3,4 Since 2003, the framework has been adopted by more than 500 programs globally. Originally developed to improve care of hepatitis C, the model has now been applied to more than 100 complex conditions (ranging from diabetes to addiction).^5,6 Despite widespread adoption of the model, evidence of effectiveness remains preliminary. Evidence to date derives from tests of single-project ECHO clinics rather than from more robust multisite designs and primarily from self-reported outcomes rather than direct measures of practice change, as was extensively summarized for Congress.⁵ More rigorous tests of the model are needed to assess its effectiveness in improving the quality of health care for patients with complex conditions. Autism represents one such increasingly common and complex condition.

Autism is a neurodevelopmental disorder associated with high rates of medical and psychiatric comorbidities.^7,8,9 Increased autism prevalence over the past 2 decades¹⁰ has resulted in health care demands that greatly exceed the capacity of specialty centers.¹¹ Children with autism experience greater unmet health care needs, higher health care costs, and worse access to medical homes and specialty care than children with other special health care needs.^{12,13,14,15,16} Children with autism from underserved populations face disproportionate health care barriers owing to clinician shortages, high costs of services, and language or cultural barriers.^17,18,19 Poor access to care results in diagnostic and intervention delays, unmanaged comorbid conditions, poor health, and reduced quality of life for children with autism.^20,21,22

Community-based PCPs are ideally positioned to address these disparities. Primary care, by definition, consists of “integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community.”²³ Unfortunately, many PCPs lack training in autism and feel unprepared to provide best-practice care.^24,25 Although the American Academy of Pediatrics issued clear recommendations for developmental surveillance and screening for autism,^26,27 physician compliance has been strikingly poor.^28,29,30,31 Many PCPs lack knowledge about autism^20,28,32 and lack confidence in managing common co-occurring conditions in children with autism.³³ Equipping PCPs with the skills to provide effective community-based screening and management of autism has the potential to enhance outcomes for children with autism.

A 2017 pilot study^34,35 was conducted to examine the application of the ECHO model to enhance primary care for autism. The study consisted of a small sample of 14 PCPs who attended 12 biweekly ECHO Autism clinics focused on best-practice medical care for children with autism. The results were promising, showing significant improvements in participant self-efficacy and self-reported practice change. However, objective measures of improvement were not collected.³⁴ Thus, a more rigorous examination of the model is needed.

This study was conducted to test the effectiveness of ECHO Autism in a large and geographically diverse randomized sample using objective measures of improvement in clinical practice, while delivering a potentially important educational intervention to all participants. The specific hypotheses were that participating in ECHO Autism would result in significant improvements among PCPs in (1) clinical practice, (2) knowledge, and (3) self-efficacy regarding autism screening and management of comorbidities.

Methods

Study Design

Ten academic medical centers (eAppendix in Supplement 1) participating in the Autism Treatment Network and the Autism Intervention Research Network on Physical Health served as individual ECHO Autism clinic sites (each training a target sample of 15 PCP participants). A sequential, staggered rollout of the ECHO Autism program was studied in 5 cohorts of participants (2 sites every 3 months) between December 2016 and November 2018. Sites were randomized to startup order after recruitment into study. This design allowed us to distribute limited resources to assess the effect of the ECHO Autism program across a broad range of sites. Assessments were completed at 4 points separated by 3-month intervals: Baseline/pre-ECHO, mid-ECHO, post-ECHO, and a post-ECHO follow-up to assess persistence of effect. Figure 1 shows a CONSORT diagram of participation in the study. Figure 2 describes the timeline for data collection. The study was registered at ClinicalTrials.gov (NCT03677089), and the formal trial protocols are in Supplement 2.

Figure 1. CONSORT Flow Diagram.

PCP indicates primary care physician or practitioner; ECHO, Extension for Community Health Outcomes; WCV, well-child visit.

Figure 2. Planned Data Collection Times Across Cohorts.

ECHO indicates Extension for Community Health Outcomes; T, time.

Participants

Each of the 10 ECHO Autism sites (9 US and 1 Canadian site) planned to recruit a sample of 15 PCPs from their respective geographic region. Inclusion criteria were current practice as a PCP (ie, family practice physician, pediatrician, nurse practitioner, or physician assistant), providing outpatient services to children, and providing care to underserved populations (at least 50% of patients were underserved). Only 1 PCP from a single-practice location was allowed to participate in the study. Recruitment methods included professional distribution lists, professional meetings, and face-to-face and word-of-mouth recruitment. Participants received no-cost continuing medical education credit for attendance at each ECHO Autism clinic, and modest financial incentives were provided for completion of study measures at each assessment point. Informed consent followed procedures dictated by the institutional review board at the specific site; generally verbal consent was considered adequate because the study involved no more than minimal risk. The study was approved by the institutional review board at each site and at the Network Coordinating Center.

ECHO Autism Implementation and Procedures

ECHO Autism expert teams were formed at each of the 10 participating sites, consisting of at least 5 autism specialists (physician/autism medical specialist, psychologist, family resource specialist, dietician, and parent expert). One site included an additional Family Resource Specialist because participants were practicing in 2 different states. Teams were trained to fidelity in the ECHO model and in the ECHO Autism curriculum.³⁴ Training included in-person training and orientation at the ECHO Institute, regular learning network videoconference meetings, attendance and observation of at least 3 ECHO Autism clinics facilitated by the leadership team, and facilitation of at least 2 mock clinics with live coaching and feedback prior to clinic launch.

Each ECHO autism clinic site provided twice-monthly 2-hour ECHO Autism clinics during a 6-month period. Clinics were conducted using high-quality secure Health Insurance Portability and Accountability Act–compliant videoconferencing technology (Zoom), enabling screen-sharing (for document viewing) and real-time interactions among participants and ECHO Autism expert team members. All clinics followed the established ECHO Autism curriculum, which included a standard set of didactic presentation slides and content, case presentation forms, and materials.³⁴ Each clinic consisted of a brief 15-minute didactic presentation, 2 PCP-generated case presentations, and collaborative discussion among experts and participants. Didactic content focused on guidelines and algorithms for evidence-based best-practice medical care for children with autism, with particular emphasis on autism screening and identification and on medical management of comorbid conditions^{27,36,37,38,39,40,41} (didactic materials are available on request). The intent of the program was for participants to develop new clinical skills through guided practice and collaborative case-based learning while maintaining responsibility for care of their patients. Implementation fidelity was assessed by study team members at 2 randomly selected sessions per site using a form developed by the University of New Mexico Project ECHO Team to assess fidelity; adequate fidelity was defined as “agree” or “strongly agree” for at least 80% of the relevant items.

Unconstrained random allocation was generated in SAS (SAS Institute Inc) by the senior study statistician after sites were enrolled by principal investigator. Sites were allocated to a cluster based on the random allocation.

Some operational problems occurred during the study. Owing to scheduling issues, 2 sites started 1 month after the scheduled start of their cohort. This was ignored for the analysis. Because Canadian guidelines for screening for autism were not consistent with US guidelines, autism screening for the Canadian site was excluded from the primary analysis. Full details of all operational problems are provided in section 12 of the statistical analysis plan in Supplement 3.

Outcome Measures

Clinical practice/behavior was assessed through medical record review conducted by study staff in each PCP practice. Appropriate autism screening was assessed at 18-month and 24-month well-child visits for the 30 days prior to the scheduled date of medical record review. Medical record reviews were done at baseline (before the start of ECHO, but on rare occasions after the first but before the second ECHO session), at mid-ECHO (between the sixth and seventh ECHO session), post-ECHO (within 30 days after the end of the ECHO program), and for long-term effect (between 3 and 4 months after the end of the ECHO program). Appropriate autism screening was defined as administering an autism screening tool at 18 and 24 months, as recommended by the American Academy of Pediatrics²⁷ for the 9 US sites (all other measures included both US and Canadian sites). Treatment of 4 common autism comorbidities (constipation, sleep disturbance, behavior problems, and anxiety) was assessed for all visits for children with autism seen within the 60 days prior to the scheduled date of medical record review. We considered any treatment addressing a recognized problem (from a predefined list of appropriate treatment options for each condition, which are shown in eTable 1 in Supplement 1) as appropriately addressing the problem. Secondary outcome measures (knowledge, self-efficacy, and barriers to care) are described further in the trial protocol in Supplement 2. Self-reported demographic information (including age, sex, and race/ethnicity) was collected to characterize the sample (Table 1).

Table 1. Demographics for mITT Sample.

Variable	No. (%)
	Overall	Cohort 1	Cohort 2	Cohort 3	Cohort 4	Cohort 5
PCP participants, No.	132	27	33	29	28	15
Age, median (IQR), y	46 (37-55)	43 (36-55)	49 (45-60)	45 (37-53)	46 (36-55.5)	39 (36-50)
Sex
Female	108 (82)	21 (78)	26 (79)	23 (79)	23 (82)	15 (100)
Race
White	100 (76)	21 (78)	23 (70)	23 (79)	20 (71)	13 (87)
Black or African American	7 (5)	2 (7)	2 (6)	2 (7)	1 (4)	0
Asian	21 (16)	3 (11)	8 (24)	3 (10)	6 (21)	1 (7)
Other/multiracial	4 (3)	1 (4)	0	1 (3)	1 (4)	1 (7)
Ethnicity
Hispanic or Latino	8 (6)	2 (7)	0	2 (7)	4 (14)	0
Specialty
General pediatrician	100 (76)	18 (67)	30 (91)	23 (79)	20 (71)	9 (60)
Internal medicine–pediatrics physician	6 (5)	3 (11)	1 (3)	1 (3)	1 (4)	0
Family medicine physician	7 (5)	1 (4)	1 (3)	2 (7)	1 (4)	2 (13)
Nurse practitioner	15 (11)	5 (19)	1 (3)	2 (7)	3 (11)	4 (27)
Physician assistant	2 (2)	0	0	1 (3)	1 (4)	0
Other	2 (2)	0	0	0	2 (7)	0
Years in practice, median (IQR)	13.5 (6-22.5)	13 (2.5-23)	19 (11-25)	12 (7-17)	13.5 (5.5-25.5)	7 (5-15)
Children seen at practice per y, median (IQR)	3000 (1302-4000)	3500 (2079-4500)	3000 (1000-4000)	2880 (1000-3600)	2600 (2000-4200)	1504 (700-3700)
Additional training in autism	62 (47)	18 (67)	13 (39)	11 (38)	14 (50)	6 (40)
Type of additional traininga
Content within a graduate course (not autism-specific)	13 (21)	6 (33)	2 (15)	1 (9)	1 (7)	3 (50)
Specific graduate course on autism	5 (8)	1 (6)	1 (8)	2 (18)	0	1 (17)
Workshop	23 (37)	5 (28)	4 (31)	4 (36)	7 (50)	3 (50)
Conference presentation	17 (27)	4 (22)	3 (23)	2 (18)	6 (43)	2 (33)
Continuing medical education seminar	30 (48)	9 (50)	7 (54)	6 (55)	6 (43)	2 (33)
Webinar	6 (10)	0	1 (8)	4 (36)	0 (0)	1 (17)
Residency rotation during residency	23 (37)	5 (28)	7 (54)	3 (27)	6 (43)	2 (33)
Medical school rotation	7 (11)	3 (17)	2 (15)	1 (9)	1 (7)	0
Practice setting
Academic medical center	21 (16)	7 (26)	2 (6)	3 (10)	5 (18)	4 (27)
Federally qualified health center	13 (10)	3 (11)	1 (3)	1 (3)	2 (7)	6 (40)
Multiple settings	10 (8)	0	4 (12)	4 (14)	2 (7)	0
Solo practice	20 (15)	1 (4)	12 (36)	2 (7)	5 (18)	0
Private group practice	50 (38)	10 (37)	11 (33)	13 (45)	14 (50)	2 (13)
Other	18 (14)	6 (22)	3 (9)	6 (21)	0	3 (20)

Abbreviations: IQR, interquartile range; mITT, modified intent-to-treat; PCP, primary care physician or practitioner.

^aPercentages are of those with any additional training in autism.

Power Calculations

Given the complexity of the proposed analysis, power calculations were based on simulations. The data generation process allowed for random effects for site, PCP within site, and nominal period. There was no time trend in the data, although a potential time trend as a fixed effect was included in the model. Simulations were done for 10 randomly selected seeds (from several different websites and different random number tables) at 1000 simulations per seed. The data-generating process allowed for approximately a 50% intraclass correlation for the PCP within-group effect, reflecting the possibility that the effect of ECHO would be correlated within each site, even with good fidelity to the intervention program. Simulations allowed for varying numbers of patients per PCP practice.

If PCPs on average had 15 patients with well-child visits seen in the past month, we would have more than 90% power to detect an increase from 50% to 60% in autism screening (α = .025; all tests were 2-sided). If PCPs on average had seen 5 patients with autism in the last 60 days, we would have more than 90% power to detect an increase from 50% to 65% in appropriate comorbidity management (α = .025; all tests were 2-sided).

Statistical Methods

Data are summarized as median (interquartile range [IQR]) or counts. The primary analysis used a generalized linear mixed model with data from baseline, mid-ECHO, post-ECHO, and 9-month follow-up to assess program effect, with fixed effects for period (time since start of study) and time in study (categorical time since PCP participant entered study) including site-specific random intercept for each PCP participant. We accounted for repeated measures within PCP participants with unstructured covariance. This is a modified version of the standard model for a stepped-wedge design put forward by Hussey and Hughes.⁴² Model estimates for baseline, month 3, month 6, and month 9 are presented are presented with the primary analysis test of change over 6 months of study. Additional results test changes from month 6 to month 9 to assess any regression after ECHO is completed. Results are reported for a modified intent-to-treat population that includes all participants who consented and provided basic demographics at baseline. Although we initially planned an analysis restricted to participants who more closely followed protocol, we agreed with a reviewer to present mITT analyses because this was better suited to the study design; results were almost identical between the 2 analyses. The SAP called for baseline data to be imputed if more than 5% of data were missing for a primary outcome. Because less than 5% of baseline data were missing, no baseline data were imputed. Missing follow-up data had the baseline carried forward as a conservative estimate (5 PCPs had autism screening follow-up information imputed, and 15 PCPs had follow-up information imputed for the proportion of comorbidities addressed outcome). Because we had 2 coprimary end points, we used P < .025 to determine statistical significance for proportion screened appropriately and proportion of comorbidities addressed appropriately at 6 months; otherwise P < .05 was considered statistically significant. All tests were 2-sided.

Role of the ECHO Institute

The study team received training and consultation from ECHO Institute members on ECHO implementation. Members of the ECHO Institute were not involved in study design, analyses, interpretation of results, or decision to publish.

Results

Population, Participation, and Implementation

A total of 148 participants were enrolled across the 10 ECHO Autism clinic sites, ranging from 11 to 22 per site. See Table 1 for demographic data by recruitment cohort. Of the 129 participants (87%) who attended at least 1 session, the median attendance was 10 of 12 sessions (interquartile range [IQR], 8-11). Regarding implementation, all sessions assessed during random fidelity checks demonstrated fidelity to the ECHO model. Median fidelity at individual clinics was 96% (range, 82%-100%).

Some demographic information was provided by 132 participants (89%), and baseline medical record review was completed for 118 participants (80%). The number of medical records available for review at each point varied across participants and visit type. For the baseline medical record review window, participants had a median of 5 well-child visits (IQR, 2-9) reviewed for autism screening (18-month or 24-month visits) and 3 visits (IQR, 1-6) with children with autism. Of the 118 participants having a medical record review attempted at baseline, 25 had no relevant well-child visits within the 30-day review period, 18 had no visits with a child with autism within the 60-day review period, and 24 provided a medical record for a child with autism but did not identify any of the listed problems. Baseline outcome measures by cohort are presented in eTable 2 in Supplement 1.

Efficacy of ECHO Autism

Model-based estimates of the outcomes are presented in Table 2, while effect estimates and statistical significance are shown in Table 3. Our primary analysis models estimated a small 6-month increase in autism screening rates from 65% (95% CI, 40%-84%) to 71% (95% CI, 52%-84%) (P = .63) and a small 6-month decrease in mean percentage of comorbidities addressed from 81% (95% CI, 67%-90%) to 77% (95% CI, 66%-86%) (P = .59). There was substantial improvement in knowledge (9% increase; 95% CI, 4%-13%; P < .001) and overall self-efficacy (29% increase; 95% CI, 25%-32%; P < .001) and a substantial decrease in the number of barriers identified by participants (2.2 decrease; 95% CI, −2.7 to −1.7; P < .001) (Table 3).

Table 2. Model Estimates at Each Time.

Outcome	Mean (95% CI)
	Month 0	Month 3	Month 6	Month 9
Primary outcomes
Proportion of children screened appropriately for autism	0.65 (0.40-0.84)	0.69 (0.51-0.83)	0.71 (0.52-0.84)	0.63 (0.37-0.84)
Proportion of comorbid medical problems appropriately addressed in children with autism	0.81 (0.67-0.90)	0.76 (0.65-0.85)	0.77 (0.66-0.86)	0.83 (0.70-0.91)
Secondary outcomes
Proportion of children screened appropriately for general development	0.90 (0.55-0.98)	0.88 (0.64-0.97)	0.88 (0.64-0.97)	0.90 (0.58-0.98)
Proportion of comorbid medical conditions identified for each child with autisma	0.14 (0.09-0.22)	0.16 (0.12-0.22)	0.16 (0.12-0.22)	0.18 (0.12-0.26)
Knowledge Score, %	56 (52-61)	62 (58-65)	65 (62-68)	66 (61-70)
Self-efficacy Total Scoreb	42 (38-45)	61 (58-63)	70 (68-73)	71 (68-74)
Total perceived barriers to care (of 10 possible)	4.8 (4.3-5.3)	3.7 (3.3-4.1)	2.6 (2.2-3.0)	2.7 (2.2-3.1)

^aAmong 4 prespecified conditions (anxiety, behavioral problems, constipation, and sleep disorders).

^bRanging from 0 to 100, with higher scores meaning higher perceived self-efficacy.

Table 3. Model-Based Estimated Treatment Effects.

Outcome	Change at 6 mo		Persistence Between 6 and 9 mo
	Treatment Effect, OR (95% CI)	P Value	Difference, OR (95% CI)	P Value
Primary outcomes
Proportion of children screened appropriately for autism	1.29 (0.46 to 3.57)	.63	0.72 (0.42 to 1.23)	.23
Proportion of comorbid medical problems appropriately addressed in children with autism	0.80 (0.35 to 1.82)	.59	1.40 (0.82 to 2.40)	.22
Secondary outcomes
Proportion of children screened appropriately for general development	0.80 (0.13 to 5.04)	.81	1.18 (0.44 to 3.16)	.73
Proportion of comorbid medical conditions identified for each child with autisma	1.16 (0.69 to 1.94)	.57	1.11 (0.82 to 1.50)	.49
Knowledge Score, Δ (95% CI)	8.54 (4.09 to 12.98)	<.001	0.62 (−2.24 to 3.48)	.67
Self-efficacy Total Score, Δ (95% CI)b	29 (25 to 32)	<.001	1 (−1 to 4)	.29
Total perceived barriers to care (of 10 possible), Δ (95% CI)	−2.20 (−2.74 to −1.67)	<.001	0.02 (−0.37 to 0.41)	.92

Abbreviation: OR, odds ratio.

^aAmong 4 prespecified conditions (anxiety, behavioral problems, constipation, and sleep disorders).

^bRanging from 0 to 100, with higher scores meaning higher perceived self-efficacy.

Long-term Effects of ECHO Autism

The significant effects seen in knowledge, overall self-efficacy, and barriers identified did not subside by month 9, and there were no significant differences observed from the 6-month post-ECHO visit to the 9-month follow-up visit (Table 3).

Discussion

This study was designed to evaluate the effectiveness of a widely adopted technology-based continuing education model (project ECHO) in improving direct clinical care. In what is to our knowledge the first large-scale prospective multisite study of this model, we directly tested the effects of ECHO as applied to primary care for autism. Contrary to predictions, participants did not demonstrate statistically significant improvements in either autism screening or in treatment of comorbidities in children with autism after participation in the 6-month ECHO autism program. However, consistent with previous studies,⁵ participants did demonstrate significant increases in both knowledge and self-efficacy as well as significant reductions in perceived barriers to caring for children with autism in their practices.

The ECHO model was developed to increase access to high-quality health care for underserved patients with complex conditions. The results of this study do not provide strong support for the model in achieving direct practice change among PCPs, at least as it relates to the autism screening and comorbidity management. However, marked improvements in clinician knowledge and confidence in treating children with autism were observed. Although less direct than the specific practice behaviors we assessed through medical record review, these outcomes may ultimately enhance overall care and patient-clinician interactions for a population at high risk for poor health care experiences and outcomes.^{11,12,13,14,15}

Strengths and Limitations

This study benefitted from a large, multisite, and geographically diverse sample of PCPs. Additional strengths included randomization at the level of cohort, staggered implementation of the program across cohorts, and tests of effectiveness under baseline, intervention, and follow-up conditions. There are also several limitations that should be noted. Participants in this study had some motivation to improve their care of children with autism and may not be representative of the larger population of PCPs. However, their screening rates at baseline were generally consistent with 2016/2017 clinician-reported estimates,^43,44 suggesting that they did not differ substantially from their peers with regard to screening practices prior to participation in the program.

The ECHO Autism curriculum was delivered for a predefined period (6 months) for the purposes of the research study. However, typical ECHO implementation is ongoing, with no clearly defined stopping point. We also prohibited multiple participants from the same practice to minimize confounds; however, traditional ECHO models encourage participation from multiple clinicians from within a single practice. We also observed a relatively high rate of dropout, which may indicate issues with feasibility or acceptability of the model for some practicing PCPs.

Finally, our measure of comorbidity management may have been too simplistic. We were unable to assess whether PCPs were actively screening for each condition or whether they were noted only when concerns were raised by caregivers. As such, we were unable to determine the extent to which comorbid conditions were accurately screened and identified. Because the medical record review process did not allow for determination of best treatment approach for an individual patient (because this would have required thorough clinical review of each case), our measure simply noted whether the PCP administered 1 or more potentially appropriate treatments when a comorbid condition was observed. In future studies, a more fine-grained analysis of treatment strategies would be more helpful in determining whether a clinician followed a best-practice treatment approach or algorithm.

Conclusions

In conclusion, the results of this large-scale study provide some support for the effectiveness of the ECHO model in improving primary care knowledge and self-efficacy regarding treatment of children with autism but little evidence of practice change in specific screening and management domains. Future research is needed to further evaluate the cost, benefit, and effectiveness of ECHO. A more comprehensive assessment of screening and management practices, patient-clinician interactions, and patient outcomes would be particularly informative in future studies.

Supplement 1.

eAppendix. ECHO Autism Collaborative Sites

eTable 1. Chart Review Form: Comorbidity Management

eTable 2. Outcome Variables at Baseline by Study Cohort

Supplement 2.

Trial Protocol.

Supplement 3.

Statistical Analysis Plan.

Supplement 4.

Data Sharing Statement.