Enhancing Developmental Behavioral Pediatric Rotations by Teaching Pediatric Residents How to Evaluate Autism in Primary Care

Abstract

Most pediatric residency programs do not provide adequate practical experiences for residents in autism-related care, with a specific dearth of training in diagnostic processes. Without this training, residents enter the primary care workforce with uncertainty surrounding implementation of standard-of-care recommendations for children with autism. Our curriculum was designed to improve upon previously passive shadowing experiences within our developmental behavioral pediatric rotation by integrating hands-on training in within-practice diagnostic identification and care coordination. In addition to multiple web-based learning activities (including a tutorial teaching administration of an interactive assessment tool), residents were required to actively participate in autism evaluations under supervision of an attending provider. Data were collected to assess feasibility of curricular changes as well as increased comfort level across recommended practice behaviors. Regarding feasibility, 95% of residents completed training components and pre/post ratings indicated significant increases in comfort in identifying symptoms/risk for autism, providing feedback about diagnostic decision, and effectively connecting families with services. Outcomes suggest feasibility of model and significantly increased resident comfort level across a range of recommended practice behaviors. This project reflects a first step in advancing incorporation of autism training into pediatric residency programs.

LAY ABSTRACT

Most physician preparation programs do not provide enough practical experiences in autism-related care. This is especially true for how to assess for and diagnose autism. Without this training, many pediatricians are not well prepared to implement appropriate care for children with autism and their families. We designed a curriculum to improve training for medical residents that involved explicit hands-on training in diagnostic identification and care coordination for toddlers at risk for autism. We collected data to assess whether our enhanced curriculum led to increased comfort level across recommended practice behaviors. Almost all the residents were able to complete the training within their rotation and our surveys indicated significant increases in residents feeling more comfortable identifying symptoms of autism, providing feedback about diagnostic decisions, and effectively connecting families with services. A significant majority of residents considered it appropriate or very appropriate for children to receive a diagnosis solely from a primary care provider. Our results suggest feasibility of the enhanced model and this project reflects the first step in advancing incorporation of autism training into pediatric residency programs.

Early accurate identification of autism spectrum disorder (ASD) represents a pressing public health and clinical care challenge. Screening for ASD is recommended at 18 and 24 months of age (Johnson & Myers, 2007), making pediatricians most families’ first point of contact for discussions about developmental risk. Despite large-scale screening initiatives, the average age of diagnosis remains over four years of age (Maenner et al., 2020). These diagnostic delays, which disproportionately affect children from underserved communities, contribute to healthcare disparities that restrict access to meaningful intervention services (Maenner et al, 2020; Stahmer & Mandell, 2007). Despite being on the frontlines of developmental assessment, most pediatric primary care providers (PCPs) report receiving inadequate training and having low levels of competence in recognizing and managing ASD (Carbone et al., 2010; Golnick et al, 2009). The standard of care continues to involve PCPs flagging children at risk and referring on for comprehensive evaluation that often presents with geographic, financial, and time-related barriers (Wiggins et al., 2006).

Past research shows a majority of children with clear symptoms can be accurately recognized by community PCPs (McDonnell et al., 2019; Swanson et al., 2014; Warren et al., 2009). For these children, waiting months for a multi-hour evaluation—often in unfamiliar settings with unfamiliar people—may not represent the most desirable or ethical standard of care (Constantino et al., 2020; Wiggins et al., 2006). This is especially salient given recent workforce surveys of developmental-behavioral pediatric (DBP) providers, suggesting that not only is there a need for more training, but workforce shortages will dramatically worsen within the next 3–5 years (Bridgemohan et al., 2018). Thus, medical training programs must provide expanded training in diagnostic assessment in order to give all children with ASD timely access to resources and intervention.

When asked why they do not regularly implement practices to identify children with ASD within their clinics, PCPs often report insufficient training on how to select and use appropriate tools to differentiate symptoms, how to provide meaningful feedback to families, and how to best connect families to appropriate services (Golnick et al., 2009; Carbone et al., 2010). Residency programs are required to provide four weeks of concentrated educational experiences in DBP; however, more pressing responsibilities or discretionary activities often supplant resident commitments during these rotations. Thus, most DBP rotations provide insufficient time and motivation for residents to learn practical skills related to DBP in general, let alone ASD-specific care (Broder-Fingert et al., 2014; Ghaderi et al., 2019). Most programs provide only brief didactics and training on parent-completed screening measures, without specific, hands-on training in the evaluation, diagnosis, and management of young children with ASD—a stark contrast to most other active components of residency programs. The purpose of this study was to evaluate a model “hands-on” service system intervention for resident training in ASD.

METHODS

A competency-based curriculum for ASD-related care was developed to enhance traditional shadowing-only approaches by providing active, supervised learning experiences designed to facilitate within-practice diagnostic identification and care coordination. Between March 2018 and December 2019, all Categorical Pediatrics and Med-Peds residents were eligible to participate. All residents scheduled for the 2-week DBP rotation were expected to complete the curriculum, but at the time of the study, completion of all components was not mandatory as requirements were still based on simple exposure and passive participation. No other critical learning components of the rotation were altered, given that ASD-related care is only one part of this diverse experience. The intervention was deliberately designed for implementation within the existing scheduling and training structure, imposing an active learning model on top of time previously spent in observation.

The first components of the curriculum occurred on the first administrative day of the rotation, prior to any clinical visits. It consisted of three online learning activities, completion of which was tracked via embedded online completion metrics: (1) a brief e-learning module on the fundamentals of early detection of ASD, (2) a module on parent-completed screening measures; and (3) the Screening Tool for Autism in Toddlers (STAT) training tutorial (Stone et al., 2000). The e-learning module included training on the etiology and early warning signs of ASD, how to evaluate and communicate concerns within the primary care setting, recommended referrals and treatments, and ASD-specific anticipatory guidance including common co-occurring behavioral/medical conditions. The STAT is a play-based standardized assessment of early social communication skills. The tool was designed to be completed in under 20 min and elicit key social-communication behaviors. Validation data for the STAT suggests strong correlations with other standard comprehensive assessment tools for ASD (Stone et al., 2004). The web-based tutorial uses video examples and practice tests to teach providers how to administer and score the STAT with young children and provide feedback to parents. Residents also received training on differential diagnosis in young children including careful consideration of additional complex medical/social histories (e.g., trauma). Overlap in symptoms with ASD as well as common screening practices for other diagnoses and recommended referral protocols were taught through regularly scheduled didactics as well as proctored evaluations with young children.

Residents were matched with a provider completing ASD evaluations, either in the specialist DBP clinic or embedded within their continuity clinic. All evaluations were for children < 48 months of age and were conducted and supervised by licensed clinical psychologists or a DBP. These evaluations encompassed observation and in vivo training on 1) history review, 2) a guided clinical interview, 3) STAT administration and structured observation utilizing a checklist of ASD symptoms in toddlers, and 5) provision of diagnostic feedback and recommendations to families. These evaluations were implemented as a streamlined diagnostic triage program which allowed a clinician to promptly rule in/out or defer ASD diagnosis based upon further evaluation. As with all other aspects of medical care that residents provide, the judgment of the supervising clinician ultimately dictated the level of resident involvement.

In addition to objective measures of completion to determine feasibility, acceptability and comfort-level ratings were elicited as part of a pre-rotation and immediate post-rotation REDCap survey. All survey items were designed by having expert clinicians provide feedback on similar survey items used in previous research (Broder-Fingert et al., 2014). On both the pre- and post-rotation surveys, residents provided Likert ratings regarding their comfort with a range of practice behaviors associated with ASD care in toddlers (see Table 2). Specifically, residents were asked to rate their perceived comfort level with identifying and distinguishing symptoms, diagnosing/referring, and providing both primary care and follow-up care to families of children with ASD. Questions were also incorporated regarding 1) resident perceptions of the importance of ASD-related training, 2) the appropriateness of PCPs taking a more independent role in identification and diagnosis, and 3) barriers to implementation.

Table 2.

Resident ratings on comfort with ASD-related competency areas

	Median Scores		Change in Comfort Level on Likert Scale					Standardized Test Statistic
	PRE	POST	−1.00	0	+1.00	+2.00	+3.00 or more
Identifying symptoms/risk of ASD in children < 3	4.00	5.00	3.2%	25.8%	40.3%	21%	9.7%	5.759**
Distinguishing ASD v other developmental concerns	4.00	5.00	7.9%	27%	33.3%	20.6%	11.1%	5.301**
Making a formal diagnosis in children < 3	2.00	4.00	3.2%	15.9%	22.2%	31.7%	27%	6.255**
Communicating with families/providing feedback about ASD diagnosis	3.00	5.00	6.3%	15.9%	22.2%	31.7%	27%	5.990**
Providing primary care to children with ASD	4.00	5.00	8.1%	16.1%	50%	9.7%	16.1%	5.669**
Providing general recommendations/referrals for ASD-specific services	4.00	5.00	1.6%	31.7%	36.5%	12.7%	17.5%	5.690**
Connecting families to school-based services	3.00	4.00	4.8%	30.2%	28.6%	15.9%	20.6%	5.478**
Connecting families to early intervention services	5.00	5.00	7.9%	33.3%	31.7%	14.3%	12.7%	4.968**
Connecting families to speech language therapy	4.00	5.00	6.3%	38.1%	33.3%	9.5%	12.7%	4.877**
Connecting families to behavioral support services	4.00	4.00	6.3%	38.1%	33.3%	17.5%	12.7%	5.303**
Connecting families to mental health services	4.00	5.00	7.9%	38.1%	28.6%	15.9%	9.5%	4.734**

Median Likert ratings for each variable are presented. 1.00 = very uncomfortable, 2.00 = uncomfortable, 3.00 = mildly uncomfortable, 4.00 = mildly comfortable, 5.00 = comfortable, 6.00 = very comfortable

Analyses examined change from pre- to post-training for residents. For binary outcome variables (e.g., yes/no), McNemar’s tests were used to test for change. For ordinal variables (e.g., Likert scales), related-samples Wilcoxon signed rank tests were used.

RESULTS

Out of the 92 residents to go through the rotation during this time frame, a total of 63 residents completed both pre and post surveys (68% response rate; see Table 1). Online metrics show that 95% of residents completed new training requirements and 87% of residents had performed a STAT within their rotation. Further, 42% of residents who had not issued a diagnosis of ASD at pre-test had issued a first-time diagnosis following training.

Table 1.

Characteristics of Residents Completing Surveys

	n	%
Total	63
Female	48	76
Male	15	24
Training Program
Categorical Pediatrics	51	81
Med-Peds	7	11
Neurology	5	8
Intended Career Plans
Primary Care	19	30
Subspecialty	35	56
Undetermined	9	14
Age	Median	Range
	28.6	26–33

There was statistically significant increase in comfort level within every area of ASD-related competency following training (Table 2). Residents reported significant increases in their ability to identify symptoms of ASD in young children and distinguish between ASD and related developmental concerns, with every resident feeling at least ‘Mildly Comfortable’ with differential diagnosis at post-test. Eighty-one percent of residents reported an increase in their comfort with making a formal ASD diagnosis in young children (when appropriate and necessary), with 59% of the residents who increased in comfort increasing by more than two points on the Likert scale. Further, 76% of residents reported increased comfort in their ability to provide primary care to children with ASD, with 29% of residents reporting feeling ‘Very Comfortable’ providing care to these children at post-test. There was also a significant increase in resident comfort communicating with families and providing ASD-related feedback and recommendations to various services.

There was a significant increase in residents’ perceptions as to how appropriate it is for a child to receive a diagnosis from their PCP alone (Wilcoxon signed rank test = 4.636, p <.001). At post-test, 79.4% of residents considered it appropriate or very appropriate to receive a diagnosis solely from a PCP, an increase from 39.7% at pre-test. There was no significant change in residents’ perceptions regarding the importance of ASD-related training (Wilcoxon signed rank test = 1.732, p = .083) or the importance of being able to independently diagnose ASD (p = .289), but this may be explained by the large percentage of residents who perceived these as important at pre-test.

Regarding barriers to practice change, 76% of residents initially identified lack of training as a substantial barrier as compared with 32% at post-test, representing a significant increase from pre-test (p =.000). Free responses suggested overall satisfaction with the hands-on, interactive nature of the training curriculum.

DISCUSSION

The primary goal of this training model was to enhance future PCP capacity for recognizing early symptoms of ASD and diagnosing when necessary. Initial data suggest this competency-building intervention has a significant impact both on resident comfort in providing various aspects of ASD-related care and in their perceptions of how appropriate it is for PCPs to assume this role. This lends support for supplying residents with foundational skills and resources, including active training on use of specific tools and clear pathways to service delivery to increase ownership over ASD-related care and uptake of these tools in future community practice settings.

Expectations for the reach of this model include residents receiving foundational knowledge and initial focused practice with differentiating and diagnosing ASD. Given the brevity of the training, it is unclear how this training will translate to future practice behavior; however, it is promising to see increased perceptions that ASD-related diagnosis and care is within the purview of PCPs and that these skills will be an expectation for their future role in primary care. The hope is that with this foundational knowledge, understanding of ownership, and familiarity with interactive tools, residents perceive that they can use more than parent-completed screening measures, they can provide diagnostic information to caregivers, and they can request more appropriate, hands-on, supervised training opportunities during their residency and continuity clinics.

This study has some important limitations. First, although based on prior research and designed using expert knowledge and feedback, the survey has not undergone validity testing. Further, although these data show important increases in perceived comfort and competence in providing ASD-related services, the current study design did not allow us to examine changes in resident practice behavior. Future research will utilize more objective markers, including expert observation, and audits of appointment notes to assess referral/care coordination behaviors. This will increase the ability to evaluate the curriculum regarding initial impact on core competencies, sustained use, and practice behavior change, as well as barriers to uptake and change. Second, this study was conducted at a single large academic medical center, and significant findings may not translate fully to other settings. However, past research has shown similar limitations in ASD training and practice behaviors across academic and community pediatric settings. Because this was a service system intervention nested within an existing medical training model, the curriculum was designed to be minimally disruptive or resource intensive. Completion data provide initial support for the feasibility, sustainability, and transportability of this curriculum when implemented within a resident training DBP rotation. Future research will explore the possibility of integrating the curriculum into established training frameworks in other diverse medical settings.

From a broader perspective, when thinking of utility beyond the training context, the goal is for PCPs to apply this training in the community to diagnose and efficiently triage children with clear ASD symptoms to appropriate services. The current screen-and-refer system of care is not adequately addressing the growing public health problem that ASD presents. The average age of diagnosis remains unchanged and barriers to care continue to be present and disproportionately affect those from underserved and underrepresented populations. There is a pressing need for a new paradigm of service delivery that will increase and expedite access to ASD-related care. By increasing pediatricians’ competency in recognizing, diagnosing, and communicating with families about ASD, there is an opportunity to improve upon both the efficiency and experience of service delivery for both providers and families.