Training & Sustaining: Training and learning collaborative outcomes across a statewide network for early autism diagnosis

Abstract

Objective:

To describe the development of a primary care professional (PCP) autism diagnosis training model and to report on outcomes related to PCP training and sustained engagement in a longitudinal learning collaborative.

Method:

We developed ADAPT (Accelerating the Diagnosis of Autism with Primary care Training), a training program to prepare PCPs to develop independent competency in evaluation of autism in children ages 14-48 months. ADAPT includes didactic and case-based modules and practice-based coaching delivered by an expert diagnostic specialist; following training, PCPs participate in a longitudinal learning collaborative. Aligned with competency-based medical education standards, measures of autism evaluation knowledge and diagnostic competency are collected.

Results:

From 2021-2023, 13 PCPs completed ADAPT didactic and practicum training to reach competency in independent autism evaluation. Clinicians demonstrated significant improvement in total autism knowledge following didactic training (p=.02). Scoring agreement on an autism observational assessment tool between clinicians and expert diagnosticians improved over case observations and practicum evaluations. Similarly, PCPs demonstrated improved evaluation competence, moving on average from Advanced Beginner to Competent Performer as rated by expert diagnosticians. Following training, PCPs attended 57% of monthly learning collaborative sessions.

Conclusion:

Training PCPs to deliver autism evaluations for young children as part of tiered community-based models of care is a promising solution to address access and waitlist challenges. ADAPT is an intensive, standardized PCP training model which results in achievement of independent competency and sustained engagement in autism evaluation. Effectiveness-implementation studies are needed to ensure scalability and sustainability of training models.

Young children often wait months to years to access autism diagnostic evaluations. ¹ Diagnostic delays lead to delays in engagement in early developmental interventions. This is a critical problem because earlier engagement in intervention confers developmental advantage² and can alleviate family stress³ and reduce long-term care costs. ^4,5 A primary contributor to this diagnostic bottleneck is the lack of available specialists with expertise in autism diagnosis^6,7 and use of resource-intensive evaluation models. ⁸ With over 2% of children diagnosed with autism, ⁹ it is imperative to find solutions to this problem. ⁸

Given the heterogeneity of the autism phenotype, ¹⁰ no one model of autism evaluation is likely to be universally suitable. Many young children referred for evaluation do not need comprehensive evaluation by a specialist to accurately rule in an autism diagnosis. ¹¹ This evidence, coupled with the American Academy of Pediatrics’ endorsement of general pediatricians in making a diagnosis of autism, ¹² has allowed the field to challenge the ubiquitous application of comprehensive, expert-driven evaluation models. ⁸ There is now mounting evidence for tiered community-based approaches that build capacity of primary care clinicians to conduct autism diagnostic evaluations for young children. ¹³

The goal of tiered models is for primary care professionals (PCP) to rule in/out autism for children with straightforward clinical presentations and to refer children with more complex presentations to diagnostic specialists. ¹³ Methods for clinician initial training and ongoing support across the small number of studies on existing models vary. In a review by Guan and colleagues, ¹⁴ most studies of primary-care based autism diagnostic models include didactic training and case based learning as well as mentorship in the form of interprofessional learning collaboratives. Overall, PCP training models have demonstrated improved clinician knowledge, confidence, practice change, and diagnostic accuracy. ^14,15

Tiered community-based autism diagnostic models delivered in the primary care setting are one promising solution to the autism diagnosis bottleneck. To date, however, studies have provided limited details on the process of PCP training and sustaining these systems of care. The Early Autism Evaluation (EAE) Hub system¹⁶ is a statewide network of community PCPs who receive specialty training to provide streamlined autism diagnostic evaluations for young children, ages 14-48 months, within the primary care setting. ¹⁶ The purpose of this paper is to 1) describe the development our PCP training model called ADAPT: Accelerating the Diagnosis of Autism with Primary care Training, and 2) to report on ADAPT outcomes related to PCP training and sustained engagement in a longitudinal learning collaborative.

Methods

Setting

The EAE Hub system, initiated in 2012, is a statewide network of community PCPs trained to provide streamlined autism diagnostic evaluations for children ages 14-48 months. ¹⁶ EAE Hub sites are embedded within primary care sites which stretch across the state and range from large health systems to private practices. EAE Hub PCPs serve as regional consultants and receive referrals from local PCPs for evaluation of children at increased likelihood for autism based on surveillance and/or developmental screening. The standard clinical evaluation protocol employed by EAE Hub PCPs includes administration or review of standard screening tools (i.e., ASQ-3 and MCHAT-R/F), a DSM-5-focused diagnostic interview and medical history, physical examination, and administration of the Screening Tool for Autism in Toddlers and Young Children (STAT; ^17,18 which has demonstrated good psychometric properties across several studies^18–22). Although the STAT, when administered as a stand-alone assessment tool is considered a Level 2 screening or triage measure, it is integrated within the EAE Hub clinical evaluation protocol to inform autism diagnosis. The EAE Hub PCP synthesizes all assessment data to inform autism diagnostic decision making. A clinical report with diagnosis and next-step recommendations, including information on community and statewide interventions and resources for autistic children and those with developmental disabilities, is disseminated to the child’s family and referring medical home PCP. When autism diagnostic outcome cannot be determined (e.g., due to complex clinical presentation), children are referred on to autism diagnostic specialists.

Since 2012, 83 clinicians (i.e., physicians, advanced practice providers, psychologists, and others) have received training as part of broad EAE Hub training initiatives. Forty-two clinicians have gone on to launch EAE Hubs within their primary care practices. Quality indicator data is collected from every evaluation conducted as part of the EAE Hub system for utilization as individual and system-level analysis reports. EAE Hub PCPs have provided diagnostic evaluations to approximately 5000 young children in Indiana. ¹⁶ In 2022, the mean age at evaluation was 29 months with median evaluation wait time of 73 days. Children evaluated were referred by PCPs practicing in highly diverse regions with 60% of referrals from medically underserved areas and over 72% of children served by Medicaid insurance. In a rigorous study of EAE Hub accuracy,²⁴ we found high rates of diagnostic agreement between EAE Hub PCPs and expert specialist evaluation (i.e., 82% agreement; 82% sensitivity; 82% specificity). In the 14% of false negative or missed autism cases, nearly 70% of those cases were flagged for referral for specialty evaluation.

Further, we recently demonstrated moderate agreement and solid psychometric characteristics of the STAT when used by EAE Hub PCPs and compared against expert specialist evaluation (i.e., STAT classification matched expert best-estimate diagnosis in 78% of cases; sensitivity: 81%, specificity:72%).²⁵

EAE Hub Training Model & Performance Measures

The initial EAE Hub training model was developed in collaboration with [BLINDED] as an expansion of the STAT-MD model. ²⁶ This training model involved an individualized, on-site multi-day curriculum with didactic education and clinical practicum training (i.e., in-vivo practice and supervision on autism diagnostic evaluation of volunteer children) on the standard clinical evaluation protocol. To address the cost- and time-intensive nature of the original approach (i.e., where a team of autism specialists and quality improvement experts traveled to each EAE Hub site to conduct the training), we revised our training model to what we now call ADAPT.

ADAPT was designed in accordance with methods of competency-based medical education^27–29 and can be delivered in synchronous, asynchronous (i.e., delivered through Canvas, ³⁰ an electronic educational platform), and hybrid formats. Two phases of training estimated to take PCPs approximately 25-30 hours to complete include 1) didactic and case-based modules, and 2) clinical evaluation practicum. Aligned with competency-based medical education standards, ^31,32 we collect measures of autism evaluation knowledge and/or progression toward competency at each training phase to assess readiness for independent conduct of autism evaluations (see Table 1). Following training, PCPs engage in a longitudinal learning collaborative as part of the EAE Hub system.

Table 1.

EAE Hub Training Phases, Performance Measures, Administration Timepoints & Pass Criteria

Training Phase		Performance Measure	Timepoint	Pass Criteria
Phase 1	Didactics	Clinician Demographic Questionnaire: Brief survey measuring clinician demographics, years in practice, practice setting, and baseline perceived autism diagnostic knowledge and competency.	Baseline	N/A
		Autism Knowledge Questionnaire (AKQ): 45-item multiple choice questionnaire designed to measure knowledge about diagnosis, evaluation, and care management in young children with autism; a General autism knowledge total score and six subdomain scores (autism screening, diagnostic consultation, diagnostic formulation, communicating feedback; clinical recommendations, and cultural competency) are generated.	Pre- and post-didactic training	N/A
Phase 2	Case-based Observation (CBO)	Assessment Scoring Agreement. Item-level scores for the autism structured observational assessment tool (e.g., STAT) are submitted by PCPs for case observations. Percent agreement with faculty trainer consensus scores is calculated.	Ongoing throughout CBO	Scores of ≥ 80% agreement on 2 consecutive observations
	Case-based Practicum (CBP)	Assessment Scoring Agreement. Item-level scores for the autism structured observational assessment tool (e.g., STAT) are submitted by PCPs for practicum evaluations. Percent agreement with faculty trainer consensus scores is calculated.	Ongoing throughout CBP	Scores of ≥ 80% agreement on 2 consecutive observations
		Evaluation Competency Assessment. 17-item entrustment scale (5-point Likert scale with ratings from Novice – Exemplar;1 score of 3 reflects “Advanced Beginner” rating) detailing key competencies of autism diagnostic evaluation in young children; ratings are completed by trainer for case-based practicum evaluations		Ratings of Advanced Beginner or above (≥3) on all competencies for 2 consecutive observations

¹Competency Assessment Scale anchors: Novice: key components are absent or inaccurate; Advancing Beginner: key components are present but rudimentary; Advanced Beginner: most components performed in an efficient and accurate manner; Competent Performer: All components performed in an efficient and accurate manner; Exemplar: all components performed in an instinctive, thorough, and efficient manner.

Phase 1: Didactic Training

Phase 1 Didactic training is offered to a variety of PCPs and other clinical participants (hereafter clinicians who engage in Phase 1 are referred to as “participants”) across our pediatric training programs, children’s hospital clinics, and state partnerships for pediatric health. Not all participants who engage in didactic training intend to complete Phase 2 and join the EAE Hub system. Instead, they seek out this experience to expand their general knowledge about autism diagnosis and care management.

Didactic training involves education on best-practice procedures for diagnostic evaluation of autism covered over 15 hours of either live (i.e., 2-day) or self-paced asynchronous video modules. Content areas include general autism knowledge (prevalence, DSM-5 autism symptoms, symptom emergence and developmental course, and comorbidities), screening and assessment tools, conducting a diagnostic evaluation (including conducting a clinical interview and introduction to administration and scoring of the STAT), diagnostic decision-making, communicating a diagnosis, evidence-based autism interventions, and cultural considerations in diagnosis. Participants complete a demographic questionnaire at training intake and the Autism Knowledge Questionnaire (AKQ;³³ previously employed in evaluation of resident autism training³⁴) at pre- and post-didactic training. The AKQ is a 45-item multiple choice (with four response options) questionnaire designed to measure knowledge about diagnosis, evaluation, and care management in young autistic children. AKQ items were developed by a team of clinicians and researchers at Vanderbilt University Medical Center and Indiana University School of Medicine with expertise in primary and specialty care autism evaluation; items were iteratively refined based upon expert feedback from an independent group of experts from across the United States until consensus was reached on each item and response. An AKQ total knowledge score (range: 0-45) and seven subdomain scores (general autism knowledge: range 0-5; autism screening: range 0-4, diagnostic consultation: range 0-6, diagnostic formulation: range 0-6, communicating feedback: range 0-4; clinical recommendations: range 0-10, and cultural competency range: 0-10) are generated.

Phase 2: Case-based Practicum

Phase 2 of training, designed for PCPs (hereafter clinicians who engage in Phase 2 are referred to as “PCPs”) who intend to obtain competency in independent evaluation of autism in young children, involves case-based observation and practicum. Approximately 10-15 hours of practicum training is completed via live (i.e., synchronous with expert trainer) or recorded (i.e., asynchronous observations via recordings in Canvas; practicum conducted via submission of recorded evaluations to a HIPAA-secure shared folder for trainer review) format.

Case-based observation.

Case-based observation provides PCPs with an experiential introduction to competencies in diagnostic evaluation. In either live or recorded format, PCPs progress through observations of expert trainers conducting the STAT (paired with feedback on scoring accuracy) and then full diagnostic evaluations on a range of children representing diversity in age (14-48 months), sex, family sociocultural/educational background, and autism diagnostic outcome. PCPs are required to achieve STAT scoring agreement of 80% or greater on two consecutive cases (based on well-accepted scoring reliability guidelines for gold standard autism diagnostic tools, such as the Autism Diagnostic Observation Schedule, Second Edition³⁵) and observe at least one full diagnostic evaluation before moving on to the case-based practicum.

Case-based practicum.

Case-based practicum supports PCPs in achieving independent competency in autism diagnostic evaluation for young children. In either live or recorded format, PCPs progress through administration of the STAT (paired with feedback on administration and scoring accuracy) with volunteer children with developmental delay or autism until they achieve STAT scoring agreement of ≥ 80% on two consecutive cases. PCPs then progress through conduct of full diagnostic evaluations with children referred for autism evaluation. Trainers provide practice-based coaching³⁶ and written qualitative and quantitative performance feedback via use of the Evaluation Competency Assessment. The Evaluation Competency Assessment is an entrustability scale ³² with ratings made across five performance anchors from Novice through Exemplar (see Table 1 for definitions of anchors). PCPs are required to achieve ratings of “Advanced Beginner” or above on all items of the Evaluation Competency Assessment prior to moving on to independent evaluations.

EAE Hub Learning Collaborative

Following training completion, EAE Hub PCPs and their teams (i.e., EAE Hub support staff such as a nurse or medical assistant) engage in a monthly 40-minute virtual longitudinal learning collaborative (held via Zoom). The learning collaborative is offered twice monthly with repeated content to maximize opportunities for busy PCPs and their teams to participate. Each call is structured with the following topics: 1) update on EAE Hub data collection, 2) general announcements (e.g., educational opportunities, patient opportunities for community engagement/research, autism service updates, etc.), 3) focused topical presentation, and 4) discussion. The focused presentation involves topics such as review/discussion of challenging diagnostic cases, review of EAE Hub quality indicator data, evidence updates in autism research and services, and state and community resources. Data is collected on monthly learning collaborative attendance (i.e., by EAE Hub site and clinician).

Aligned with the learning health system approach, ³⁷ we utilize our EAE Hub quality indicator data in order to improve care in our system. New clinicians receive a standard report on individual as compared to system aggregate quality indicators (i.e., number of evaluations, rate of autism diagnosis, rate of referrals for secondary diagnostic consults) following their first quarter of evaluations. All clinicians receive an annual report with data on individualized compared to system aggregate quality indicators. Clinicians are coached to evaluate their own data to identify areas for improvement and system-wide data is used to identify needs for new quality improvement initiatives.

Procedure and Analysis Plan

University Institutional Review Board approval was obtained for this study. Data was collected either through paper records or electronic submission via institutional Microsoft Teams secure file sharing or REDCap. Data capture tools were developed and implemented throughout the study period; as such, complete data is not available for every learner. Data was analyzed using SPSS (IBM SPSS Statistics, Version 29, Armonk, NY: IBM Corp). Descriptive statistics were calculated for clinician demographics and performance measures. Change in autism knowledge (based on AKQ scores) from pre- to post-training was analyzed using paired samples t-tests.

Results

From 2021-2023, 53 training participants participated in EAE Hub ³⁵ Phase 1 didactic training. Of those, 25% (n=13) entered Phase 2 practicum training with the intent to launch an EAE Hub. Table 2 details demographics of participants who completed EAE Training Phase 1 and 2, respectively.

Table 2.

Demographic characteristics of EAE Hub training participants (2021-2023)

	Completed Phase 1 (N=53)	Completed Phase 1 &2 (N=13)
1Credential
MD/DO	29 (55)	11 (85)
NP/PA	12 (23)	2 (15)
Other	12 (23)	0 (0)
Age, years
< 40	17 (32)	4 (31)
40-50	8 (15)	3 (23)
51-60	2 (4)	1 (8)
> 60	5 (9)	3 (23)
Unknown	21 (40)	2 (15)
Sex
Female	43 (81)	10 (77)
Male	10 (19)	3 (23)
2Race/ethnicity
Asian	2 (4)	1 (8)
Black or African American	5 (9)	1 (8)
Hispanic/Latine	4 (8)	0 (0)
Native Hawaiian or Other Pacific Islander	0 (0)	0 (0)
Non-Latine White	44 (83)	11 (85)
Practice Type
Private independent group practice	4 (8)	1 (8)
Health system group practice	25 (47)	8 (62)
Academic medical center	11 (21)	1 (8)
Community health center	7 (13)	3 (23)
Other	6 (11)	0 (0)
Years in practice (post residency)
Mean, SD	12 (12)	13 (10)
Range	0-45	0-33
2Prior Training in Autism Care
Residency	16 (30)	7 (54)
Post-residency clinical training	6 (11)	1 (8)
Workshop/Conference	23 (43)	8 (62)
Other	10 (19)	1 (8)

N (%) based on available data.

¹Other Credential category includes doctoral level non-licensed faculty, psychologist, speech language pathologist, Board Certified Behavior Analyst.

²Participants could select more than one response. Prior training in autism care represents the format of pre- or post- residency training in any aspect of autism care management.

Phase 1: Didactic Training

Seventeen participants completed the Autism Knowledge Questionnaire (AKQ) at pre- and post-didactic training. Participants demonstrated significant improvement in total autism knowledge following participation in didactic training, t(16)= −2.55; p=.02; see Table 3). Significant gains across the domains of clinical recommendations, t(16)= −3.17; p=.006, and cultural competency, t(16)= −2.34; p=.03, were also demonstrated.

Table 3.

EAE Hub Training Performance Measures

		Mean (SD)
Phase 1	Autism Knowledge Questionnaire (n=17)	Pre-training	Post-Training	p
	Total score	36.06 (5.46)	39.24 (3.89)	.02
	General Autism Knowledge	4.29 (.59)	4.24 (.66)	.79
	Autism Screening	3.00 (.94)	3.29 (.85)	.26
	Diagnostic Consultation	5.24 (.97)	5.65 (.61)	.11
	Diagnostic Formulation	3.71 (.96)	4.24 (1.15)	.07
	Communicating Feedback	3.53 (.72)	3.53 (.80)	1.00
	Clinical Recommendations	8.06 (1.56)	9.18 (.95)	.006
	Cultural Competency	8.24 (1.60)	9.12 (.99)	.03
Phase 2 (CBO)	Percent STAT Observation Scoring Agreement, mean % (SD)
	Case 1 (N=13)	81 (22)
	Case 2 (N=13)	87 (7)
	Case 3 (N=13)	86 (9)
	Case 4 (N=8)	87 (12)
Phase 2 (CBP)	Percent STAT Practicum Scoring Agreement, mean % (SD)
	Case 1 (N=13)	88 (9)
	Case 2 (N=13)	89 (16)
	Case 3 (N=11)	96 (8)
	Total Evaluation Competency Assessment Score, mean (SD)
	Case 1 (N=10)	3.40 (.59)
	Case 2 (N=7)	3.68 (.41)

CBO: case-based observation; CBP: case-based practicum; STAT: Screening Tool for Autism in Toddlers and Young Children

Phase 2: Case-based Practicum

Case-based Observation

PCPs completed a mean of 3.9 (SD: 0.9) CBO cases; 96% of CBOs were completed in recorded format (recorded n=49; live n=2). Based upon review of descriptive data, STAT scoring agreement between learner and trainer improved across observations (see Table 3; Case 1: mean: 81%, SD: 22; Case 2: mean: 87%, SD: 7; Case 3: mean: 86%, SD: 9; Case 4: mean: 87%, SD: 12). Eighty five percent (n=11 of 13) of PCPs met specified performance criteria (i.e., ≥ 80% scoring agreement on two consecutive STAT observations) on the first two observations.

Case-based Practicum

PCPs completed a mean of 3.4 (SD:1.1) STAT CBP cases; 75% of CBPs were completed in recorded format (recorded n=35; live n=12). Based on descriptive data, baseline performance STAT scoring agreement was high (see Table 3; Case 1: mean: 88%, SD: 9; Case 2: mean: 89%, SD: 16; Case 3: mean: 96%, SD: 8). Seventy-seven percent (n=10 of 13) of PCPs met specified performance criteria (i.e., ≥ 80% agreement on two consecutive STAT practicum cases) on the first two cases.

Evaluation Competency Assessment data was submitted for 77% (n=10 of 13) PCPs who entered Phase 2 (mean per learner: 1.8; SD: .75); 85% of case-based observations were completed in recorded format (recorded n=17; live n=3). Review of descriptive data suggests that mean competency ratings across practicum evaluations improved over time (see Table 3; Case 1: mean: 3.4,0 SD: .59; Case 2: mean: 3.68, SD: .41). Sixty percent (n=6 of 10) of PCPs met specified performance criteria (i.e., Advanced Beginner ratings on Evaluation Competency Assessment) on the first case.

EAE Hub Learning Collaborative

From January 2021 – September 2023, each EAE Hub (i.e., at least one PCP representative) attended a mean of 76% (SD: 14; Range: 42-100) of learning collaborative sessions (see Figure 1). Individual EAE Hub clinicians attended a mean of 57% (SD: 15; Range: 27-100) of sessions.

Figure 1.

EAE Learning Collaborative Attendance by EAE Hub and EAE Hub PCP (2021-2023). Run chart details monthly attendance and yearly average by EAE Hub and EAE Clinician.

Discussion

Our goal in the current study was to report on outcomes related to clinician training and sustained engagement in a longitudinal learning collaborative across the EAE Hub system, a statewide network of community PCPs who provide streamlined diagnostic evaluations for young children, ages 14-48 months, at increased likelihood for autism. ¹⁶ Our training model, ADAPT, includes two phases of initial training, including didactic education and case-based practicum, followed by engagement in a longitudinal learning collaborative. Overall, we found that PCPs who engage in intensive standardized training following the ADAPT model demonstrate increases in autism knowledge and diagnostic competency, including reliable scoring of an observational assessment tool.

We offer our didactic education modules to diverse clinicians (i.e., trainees and residents, PCPs, psychologists, SLPs, etc.) across broad institutional and statewide networks. Following approximately 15 hours of training, we found significant increases in total autism knowledge, as well as specific knowledge increases related to making clinical recommendations and cultural considerations in autism assessment. Improved knowledge in AKQ general autism, screening, consultation, and diagnostic feedback domains were not demonstrated; this finding is likely in part due to high scores at baseline in some knowledge domains, potentially reflecting that these are solid areas of competency for PCPs. Alternatively, it is possible that the AKQ does not adequately measure the competencies taught in our didactic training, or that the training effect was not substantial enough to result in a change in AKQ scores. Nevertheless, our findings align with an emerging body of research suggesting that focused autism training can improve clinician knowledge. ^14,38 Our ultimate objective is that clinicians who engage in ADAPT didactic training are better equipped to serve young children with autism and related disabilities in their practices.

About 25% of didactic training participants intended to engage in independent autism evaluation in the primary care setting. Although our sample size is small, all EAE Hub PCPs who entered Phase 2 (i.e., practicum training) completed training. PCPs increased accuracy in STAT scoring over repeated observation cases, with most (i.e., 85%) PCPs meeting the specific pass criterion on the first two cases. PCP’s demonstrated high baseline scoring accuracy during practicum performance, with most PCPs also meeting performance criteria on the first two STAT practicum cases. PCPs demonstrated improved overall diagnostic competence (i.e., demonstrating observable skills²⁷ that reflect the components necessary for conduct of a best-practice autism diagnostic evaluation, versus just coming to an accurate diagnosis when compared against expert judgement), moving on average from Advanced Beginner to Competent Performer across two practicum evaluations. To our knowledge, while others have reported on self-efficacy and diagnostic accuracy, our study is the first to assess competency in PCP diagnostic training. ¹³

We believe that the “secret sauce” of the EAE Hub system is what happens after training. Over half of EAE Hub PCPs attend learning collaborative sessions each month, suggesting that there is value in the relational nature of the network and the sharing of information and ongoing skill development that benefits not only their work in the EAE Hub system, but also their general pediatric practice.

Challenges & Future Directions

While our ADAPT model improves upon the original on-site, individualized EAE Hub training model in terms of reducing travel time, it still requires a significant investment of time by pediatricians, expert trainers, and program staff. Given shortages of pediatric specialists, ⁶ training models that rely on autism experts may have limited potential for scalability. There is growing evidence that self-guided medical education approaches may be feasible and effective³⁹ and these approaches should be explored. However, use of remote technology governed and supported by our academic institution has posed challenges for useability with diverse clinicians from varied institutions across our state. Innovative learning management systems now allow integration of didactic and HIPAA-compliant practicum training (i.e., practice-based coaching) components. While these options are costly, they would likely reduce barriers for training across institutions and systems.

Multiple forces continue to impact our ability to sustain the EAE Hub system. PCPs and their teams work within time and productivity pressures which can limit their availability to participate in intensive training, as well as embed autism evaluation within their practice. To address these restrictions, trainings, ongoing support, and diagnostic methods must be efficient and feasible. Changes in what insurers deem valid for a medical diagnosis of autism continue to require the attention and advocacy of our leadership team. Continuing to fund our leadership team to support ongoing training and evolution of our methodology is a challenge as demands for dissemination of our model grow. In sum, reaching into primary care to build capacity for autism evaluation requires attention to the long-term sustainability of models to ensure ongoing standardization and support.

Limitations

A primary limitation of this study is the small sample size, rendering statistical analysis over time impossible for some performance measures. Second, given that we built data capture infrastructure as we expanded our training, we did not have data on all participants for all measures. As such, we cannot be sure that findings are representative of all participants and further replication is needed with a larger sample and more rigorous data collection. Finally, due to the innovative nature of this quality improvement effort, existing validated outcome measures were not available. As such, our expert team which included collaborators from academic centers across the US, developed and refined performance measures for use in our study of PCP training. Until these measures are tested with larger sample sizes and undergo rigorous psychometric evaluation, our findings should be interpreted with caution.

Conclusions

Training PCPs to deliver autism evaluations for very young children as part of tiered community-based models of care is a promising solution to address access and waitlist challenges. ADAPT, developed and implemented in the EAE Hub system, is an intensive standardized approach to training PCPs to reach independent competency in autism evaluation which results in improvements in autism knowledge and evaluation skills. Future research must focus on replication and head-to-head comparison of PCP training programs to understand the key elements necessary to achieve competency and diagnostic accuracy, and how those elements may differ depending on the PCP and practice characteristics. Further, studies of implementation and dissemination potential must be conducted to understand how to scale and sustain these models of training and care for optimal reach and impact.

Abbreviations:

ADAPT

Accelerating the Diagnosis of Autism with Primary care Training

CBO

Case-based Observation

CBP

Case-based Practicum

EAE

Early Autism Evaluation

PCP

primary care professional

STAT

Screening Tool for Autism in Toddlers and Young Children