A National Fetal Alcohol Spectrum Disorders Learning Collaborative for Pediatric Care Teams

Abstract

Objectives

To assess the impact of the SBIRT And FASD Education, Support and Treatment (SAFEST) Choice Learning Collaborative on US pediatric health care teams' knowledge, confidence, and clinical practices related to the identification and management of children with possible or diagnosed fetal alcohol spectrum disorders (FASD).

Study design

The SAFEST Choice program included 2 year-long 10-session virtual learning collaboratives based on the Project ECHO (Extension for Community Healthcare Outcomes) model, which included participant case presentations highlighting clinical challenges. Using a pre-/posttest evaluation design, program outcomes included changes in participant FASD-related knowledge, confidence, and practice. Participant case clinical challenges were documented and analyzed for common themes.

Results

The program enrolled 105 participants from 23 clinics in 8 states. Clinics attended a median of 9 of 10 ECHO sessions, whereas individual participants attended a median of 5 of 10. Participants were mostly physicians, nurse practitioners, and nurses. Upon program completion, participants reported increased FASD-related knowledge and significant increases in confidence and self-reported practices in screening for prenatal alcohol exposure and counseling families about, and managing patients with, prenatal alcohol exposure and FASD. Participant case clinical challenge themes included when to consider an FASD diagnosis, managing the care of patients with FASD, and addressing FASD-related stigma and bias.

Conclusions

The SAFEST Choice program successfully trained pediatric health care teams in 2 year-long FASD learning collaboratives with high levels of participation. Participants had increased self-reported knowledge, confidence, and practice change related to FASD identification and care. This program offers a promising educational model to improve the care of individuals with possible or diagnosed FASD.

Fetal alcohol spectrum disorders (FASD) are a range of conditions caused by prenatal alcohol exposure (PAE) that include physical, neurobehavioral, and cognitive challenges. FASD is permanent and the most common preventable cause of intellectual and developmental disabilities in the US.¹ FASD includes a spectrum of conditions including fetal alcohol syndrome; partial fetal alcohol syndrome, alcohol-related neurodevelopmental disorder,² and alcohol-related birth defects.¹ Although the true prevalence of FASD is uncertain, one frequently cited study found the prevalence among first graders in 4 US communities to be from 1.5% to 5.0%,³ with 1 community having a prevalence as high as 8.3%.⁴ FASD occurs in every socioeconomic demographic but is disproportionately represented among international adoptions,⁵ child welfare and foster care populations, correctional populations, and some Indigenous communities, in which high prevalence rates have been documented as the result of complex historical and systemic factors.^6,7

Individuals with an FASD may have difficulties with attention, learning and memory, language development, motor abilities, planning, and social skills,⁸ which may result in adverse life outcomes including problems in school, legal troubles, inappropriate sexual behaviors, mental illness, and substance use.⁹ There are evidence-informed treatments for managing the effects of FASD, and early diagnosis and treatment is associated with decreased risk of adverse outcomes.¹⁰

Clinical teams that care for children and adolescents play an important role in identifying and providing care for individuals with possible or diagnosed FASD as well as preventing future alcohol-exposed pregnancies. However, a history of PAE is not routinely obtained.¹¹ Care teams commonly underappreciate the prevalence of FASD and lack the knowledge, confidence, and skills to screen for PAE; recognize the signs and symptoms associated with FASD; and provide appropriate care for these individuals and their families.^12,13

The high prevalence of FASD, its serious consequences, and inadequate health professional awareness and expertise underscore the critical need for training. The SBIRT And FASD Education, Support and Treatment (SAFEST) Choice Learning Collaborative was developed to address this need. The purpose of this study is to describe the program and report on its feasibility and impact on participants' knowledge, competence, and self-reported practice.

Methods

Program Description

The SAFEST Choice Learning Collaborative was a collaboration between Boston Medical Center in Boston, Massachusetts, and Proof Alliance in St Paul, Minnesota. The program trained pediatric health care teams in the early identification and care management of individuals with possible or diagnosed FASD.¹⁴ The training used the Project ECHO (Extension for Community Healthcare Outcomes) model to facilitate a virtual learning community with expert and peer mentoring and support. Project ECHO is a medical education model developed at the University of New Mexico. The model involves a partnership between academic medical centers and rural community clinics and helps providers learn best practices to manage complex, common, and chronic diseases in underserved areas.¹⁵

SAFEST Choice enrolled interprofessional health care teams, including physicians, advanced practice providers, nurses, behavioral health professionals, social workers, and community health workers in pediatrics or family medicine practices in the Northeast (Massachusetts, Maine, Rhode Island) and the Upper Midwest (Minnesota, North Dakota, South Dakota, Michigan, Iowa) of the US. These regions were selected because of their high prevalence of binge and heavy drinking among women in 2020, according to the Centers for Disease Control and Prevention's Behavioral Risk Factor Surveillance System tool,¹⁶ and the SAFEST Choice team's pre-existing relationships with relevant organizations in these areas. We focused on enrolling community health centers as well as clinical practices that served rural and underserved and Native American populations.

This program included 2 year-long learning collaborative cohorts, one from June 2021 to April 2022 and the second from May 2022 to April 2023. After the first cohort, we added 15 minutes to each session to allow for more discussion of the content and to encourage more participant engagement. Clinics received stipends (up to $3250) to help offset the cost of participation. Free continuing education credits for physicians, nurses, and social workers were offered for each session.

Program Curriculum

The virtual learning collaborative began with an hour-long presentation covering foundational information on FASD, followed by 10 monthly ECHO sessions over the course of a year. Each ECHO session (60 minutes for cohort 1; 75 minutes for cohort 2) included a 10-15-minute faculty-led lecture, followed by a participant case presentation which highlighted clinical challenges.

Faculty-Led Lecture

The 10-15-minute faculty-led lecture covered core FASD-related competencies including screening for PAE, diagnosing FASD, addressing FASD stigma and biases, implementing evidence-informed FASD interventions, and identifying community resources for patients and their families. The curriculum was developed by program faculty and an advisory board with expertise in FASD, developmental and behavioral pediatrics, addiction medicine, primary care, education, and living experience. The lectures were recorded and made available to participants after each session. In cohort 2, although the core content did not change, the additional 15 minutes allowed for more time for discussion and additional teaching modalities, such as interviews with individuals with living experience, trigger tapes (videos showing clinical scenarios to generate discussion), and interactive teleconference features such as polls and breakout groups.

ECHO Participant Case Presentations

During each session, clinics presented patient cases which excluded protected health information. Case presentations started with the clinical challenge(s), followed by a patient history including birth, medical and mental health history, prenatal substance exposures, social and education history, current behavioral concerns, and the current treatment plan. After the case presentation, participants and faculty asked clarifying questions and made recommendations to address the clinical challenge(s). At the end of the session, a faculty member summarized the recommendations, which were transcribed and shared with participants.

Program Enrollment

Recruitment materials were developed and shared with state and national networks, state primary care associations,¹⁷ and relevant departments within state agencies from the priority regions of the Northeast and the Upper Midwest. Multiple national professional organizations, including the American Academy of Pediatrics, American College of Obstetricians and Gynecologists, National Association of Community Health Centers, and FASD United, disseminated program information via email. Interested clinics were encouraged to enroll interprofessional teams of multiple people and were given an onboarding form that outlined program expectations and confirmed the clinic's commitment to participation.

Program Evaluation

The evaluation design for this program was based on the Moore's expanded framework for educational outcomes.¹⁸ This hierarchical model outlines different levels of outcomes, starting from participation and progressing to community health. We targeted levels 1-5: participation, satisfaction, knowledge, competence, and performance (self-reported practice change). This study did not measure levels 6 and 7 (patient and community-level outcomes, respectively). Participation was measured by recording individual and clinic attendance at each ECHO session. Those who attended at least one ECHO session were included in the analytical sample. We measured participant satisfaction, knowledge, competence (intent to change, confidence, self-efficacy), and performance using immediate post-ECHO session questionnaires and pre- and postprogram surveys. Because the curricular content of cohorts 1 and 2 was the same, and we only added some extra engagement activities for cohort 2, we analyzed and reported both cohorts combined.

We were unable to find a validated survey that adequately captured the specific outcomes of this unique program. We therefore developed a custom survey designed by our faculty with expertise in evaluation design, continuing education, and FASD. This allowed us to create a survey that measured the program's specific goals and objectives. We verified face validity (clarity, relevance, completeness, understandability, and appropriateness) of the surveys by pilot testing it on content, evaluation, and education experts, as well as nonexperts, including those with similar background to the intended participant population. We did not report postprogram follow-up outcomes because the program ran over 12 months, giving participants adequate time to implement changes between the beginning and end of the program.

The post-ECHO session questionnaires were administered at the end of each ECHO session. These were initially sent through an anonymous link in Qualtrics but we changed the format to Zoom polling to increase response rates. The preprogram surveys were administered before the first ECHO session and the postprogram surveys were administered after the tenth ECHO session. The surveys were programmed in Qualtrics and took 10-15 minutes to complete. They were emailed to participants via anonymous link and matched across time points using the last 4 digits of participant cell phone numbers. Incentives in the form of $50 e-gift cards were sent for every completed pre- and postprogram survey. The study was approved as exempt by the Boston University Medical Campus Institutional Review Board (no. H-41125).

All of the clinical challenges presented during the participant case presentations for the 2 cohorts (34 questions from 20 ECHO sessions total) were analyzed for common themes using simple categorization with inductive coding. All clinical challenges were compiled and reviewed by a member of the program team who assigned codes describing relevant text (eg, “lack of resources,” “patient communication,” “when to pursue a diagnosis”). The codes were then sorted and grouped into broader categories or themes. Quality assurance of the coding process was performed by an additional team member.

Statistical Analysis

Participant and clinic characteristics were described through counts and percentages. Because different numbers of participants completed the pre- and postprogram surveys, and our ability to match participants across time points was limited, resulting in a much smaller sample size, we treated the pre- and postprogram survey samples as 2 independent samples to retain a larger analytic sample and maximize statistical power. To assess potential selection bias from this analytic choice, we compared baseline characteristics and preprogram survey question responses between matched and unmatched participants. Pre- and postprogram survey questions on participation, satisfaction, knowledge, competence, and performance (Likert scales) were described by means and SDs and medians and IQRs. These outcomes were assessed for normality using Shapiro-Wilks tests and visual inspection of histograms; owing to their skewed nature, pre- and postprogram results were compared using Wilcoxon rank-sum tests. As a sensitivity analysis, outcomes in the limited matched sample were analyzed using individual change scores (subtracting each individual's preprogram value from their postprogram value); change scores were assessed for normality as described previously, and mean change scores were then assessed via one-sample t tests, and the results compared with the primary analysis results. Effect sizes were reported for hypothesis tests to aid interpretation. For Wilcoxon rank-sum tests we calculated r = Z/√N, where N is the total number of observations included in the comparison; for paired/one-sample t tests we calculated the Cohen d_z (mean difference divided by the SD of the paired differences). We calculated the Cronbach coefficient alpha for the 5 confidence items, and the 5 practice items, to assess internal consistency. A 2-sided P value of .05 was considered statistically significant. All analysis was conducted using SAS/STAT, version 9.4, of the SAS System for Microsoft Windows (SAS Institute, Inc).

Results

A total of 105 individuals from 23 clinics in 8 states participated in the SAFEST Choice Learning Collaborative. Participation was defined as attending at least 1 ECHO session. Across all 20 ECHO sessions (10 per cohort), an average of 67% of attendees completed the post-ECHO session questionnaires. Preprogram surveys were completed by 81 participants, whereas 53 completed the postprogram survey. Of these, 31 surveys matched across time points, representing 30% of the total sample; 50 preprogram surveys had no matching postprogram surveys; 22 postprogram surveys had no matching preprogram surveys; and 2 individuals did not complete either a pre- or postprogram survey. Of the 81 participants who completed a preprogram survey, there were some differences in characteristics between the matched participants and the unmatched participants. Differences included the proportion who were physicians (45.2% matched, 20.4% unmatched); in practice 21 or more years (45.2% matched, 22.4% unmatched); and, among physicians, had a medical specialty of pediatrics (71.4% matched, 20% unmatched). Despite these differences, there were no differences between matched and unmatched on preprogram values of any of the confidence or practice change items.

Participation

On the basis of responses in the presurvey, most participants were physicians (29.6%), nurse practitioners (18.5%), and nurses (17.3%). Of the physicians, 50% were pediatricians, 33% were in family medicine, and 17% were in psychiatry. More than one-half of the participants were in practice for longer than 10 years. Of clinics enrolled, 65.2% served urban populations and 34.8% served rural populations. Overall, 30.4% served Native American populations (Table I). The median number of ECHO sessions attended per participant per cohort was 5 of 10 (IQR 2-7), whereas the median number of sessions attended per clinic was 9 of 10 (IQR 8-10). Median ECHO session attendance was 25 participants (IQR 20.6-28.9) and 10 clinics (IQR 9.5-10.4). In both cohorts there was a decrease in participant attendance per ECHO session over the 12 months (median number of participants in sessions 1-5 was 28.5 (IQR 25-34), and in sessions 6-10 was 21.5 (IQR 16-25.5). On average, 67% of participants claimed continuing education credit after attending a session.

Table I.

Participant characteristics

Characteristics	Total
Participant level	n = 81∗
Role, No. (%)
Physician	24 (30.0)
Nurse practitioner	15 (18.8)
Nurse	13 (16.3)
Behavioral health provider	9 (11.3)
Other†	19 (23.8)
Years in practice, No. (%)
5 years or fewer	26 (32.5)
6-20 years	29 (36.3)
21+ years	25 (31.3)
Clinic level	n = 23
Geographic area/population served, No. (%)
Urban	15 (65.2)
Rural	8 (34.8)
Native American	7 (30.4)
Clinic type, No. (%)
Primary care	21 (91.3)
Community-based clinic	15 (65.2)
Indian Health Service clinic	5 (21.7)
Hospital-based clinic	1 (4.3)
Specialty clinic	2 (8.7)

^∗Participants who completed 1 ECHO session and completed the preprogram survey.

^†“Other” responses: administrator, medical support staff, pharmacist, community health worker, pediatric physical therapist, health advocate, recovery coach, sonographer, therapist, clinician.

Satisfaction

Participant satisfaction was high, with 98.6% agreeing that the ECHO didactic sessions were well organized and easy to follow. In keeping with the goals of the ECHO model of learning, there were high levels of agreement that participants learned from their peers (mean [SD] 5.3 [1.0]; median [IQR] 6.0 [5.0-6.0], on a 6-point Likert scale from 1 = strongly disagree to 6 = strongly agree) and felt a sense of community within the group of ECHO participants (mean [SD] 3.2 [0.8]; median [IQR] 3.0 [3.0-4.0], on a 4-point Likert scale from 1 = none at all to 4 = a lot). Participants affirmed that their clinic could benefit from the knowledge learned (mean [SD] 3.5 [0.7]; median [IQR] 4.0 [3.0-4.0], on a 4-point Likert scale from 1 = none at all to 4 = a lot).

Knowledge

All participants agreed (48.9% strongly agreed, 36.2% agreed, 14.9% somewhat agreed) that the program increased their knowledge about the hazards of PAE and the options for screening for alcohol use during pregnancy, with 82.6% stating they planned to use the knowledge learned in the training to change patient care (56.5% a lot, 26.1% a moderate amount).

Competence (Intent to Change, Confidence, Self-Efficacy)

Of those who completed the post-ECHO session questionnaires, more than 90% reported that the lecture (96%) and the case presentations (91%) would change their practice. The planned practice changes included (1) increased screening for and documentation of PAE; (2) increased consideration of the diagnosis of FASD; (3) improved patient management (Individual Education Plans, improved coordination across care teams, connecting families to resources, providing support to caregivers); and (4) better communication skills (eg, using nonstigmatizing language). Most participants (86.1%) planned to use the skills learned in the training to change patient care (51.2% a lot, 34.9% a moderate amount).

There were significant increases in participant confidence (Table II) in screening for PAE, counseling families about PAE and FASD, and caring for patients with possible or diagnosed FASD. Effect sizes ranged from 0.37 to 0.52, indicating moderate-to-large effects. The skills with the lowest baseline confidence (counseling families about FASD, managing the care of patients with possible FASD, and making appropriate referrals for patients with FASD) had the largest improvements. Participants agreed or strongly agreed that the ECHO sessions increased their self-efficacy in screening for PAE (82.5%), counseling families about PAE (82.9%) and FASD (82.9%), and making referrals for patients with an FASD (76.7%). The five confidence items exhibited excellent internal consistency (Cronbach coefficient alpha = 0.91).

Table II.

Participant confidence change—primary analysis (unmatched samples); Wilcoxon rank-sum tests

Questions	Response	Pre-	Post-	Wilcoxon W	Z	P value	Effect size (r)
I am confident in my ability to…∗
…screen for PAE	No. Mean (SD) Median (IQR)	69 3.4 (1.4) 4.0 (2.0, 5.0)	43 4.4 (0.6) 4.0 (4.0, 5.0)	3055.0	3.92	<.0001	0.37
…counsel families about PAE	No. Mean (SD) Median (IQR)	70 3.3 (1.3) 4.0 (2.0, 4.0)	45 4.3 (0.8) 4.0 (4.0, 5.0)	3305.5	4.15	<.0001	0.39
…counsel families about FASD	No. Mean (SD) Median (IQR)	72 3.0 (1.2) 3.0 (2.0, 4.0)	45 4.2 (0.8) 4.0 (4.0, 5.0)	3558.5	5.23	<.0001	0.48
…make appropriate referrals for patients with FASD	No. Mean (SD) Median (IQR)	69 3.0 (1.2) 3.0 (2.0, 4.0)	43 4.3 (0.8) 4.0 (4.0, 5.0)	3263.0	5.17	<.0001	0.49
…manage the care of patients with possible FASD	No. Mean (SD) Median (IQR)	70 2.8 (1.2) 3.0 (2.0, 4.0)	43 4.1 (0.6) 4.0 (4.0, 5.0)	3348.0	5.54	<.0001	0.52

W is the Wilcoxon rank-sum test statistic (sum of ranks, including tie adjustments), Z is the standardized test statistic with continuity correction, and the 2-sided P value tests group differences. Effect size r is calculated as Z divided by the square root of the total sample size (N) and reflects the magnitude of group differences, with values closer to 1 indicating stronger effects.

^∗Likert scale 1, not at all confident; 2, not very confident; 3, neither; 4, fairly confident; 5, very confident.

Performance (Self-Reported Practice Change)

There were increases in all self-reported practice changes, including providing patients and families with FASD education and resources, assessing for FASD and coordinating care, and documenting PAE screening results (Table III). Effect sizes varied from small to large; the majority were small-to-moderate, and one reached a large magnitude (r ≈ 0.52). While the practices of assessing for FASD using appropriate tools and tests and coordinating care for a child with possible or diagnosed FASD showed the lowest baseline rates, they showed the greatest and statistically significant increases. The practice with the highest baseline rate and lowest increase was documenting results of PAE screening in the patient chart. Nearly 80% of participants either agreed (48.7%) or strongly agreed (30.8%) that the ECHO sessions increased their use of appropriate methods of PAE screening and intervention. The five practice items also exhibited excellent internal consistency (Cronbach coefficient alpha = 0.90).

Table III.

Participant self-reported practice change—primary analysis (unmatched samples); Wilcoxon rank-sum tests

Questions	Response	Pre-	Post-	Wilcoxon W	Z	P value	Effect size (r)
I share resources and provide education about PAE to my patients and/or their families∗	No. Mean (SD) Median (IQR)	66 2.8 (1.4) 3.0 (2.0, 4.0)	40 3.5 (1.3) 3.5 (3.0, 5.0)	2544.0	2.69	.0072	0.26
I share resources and provide education about FASD	No. Mean (SD) Median (IQR)	67 2.6 (1.3) 2.0 (2.0, 4.0)	41 3.3 (1.2) 3.0 (3.0, 4.0)	2672.0	2.83	.0046	0.27
I assess for FASD using appropriate tools and tests	No. Mean (SD) Median (IQR)	62 1.9 (1.1) 2.0 (1.0, 2.0)	39 3.4 (1.3) 4.0 (2.0, 4.0)	2714.0	5.22	<.0001	0.52
I coordinate care for a child with possible or diagnosed FASD	No. Mean (SD) Median (IQR)	63 2.5 (1.3) 2.0 (1.0, 3.0)	42 3.5 (1.2) 4.0 (3.0, 4.0)	2811.5	3.92	<.0001	0.38
I document the results of PAE screening in the patient's chart	No. Mean (SD) Median (IQR)	52 3.1 (1.5) 3.0 (2.0, 5.0)	14 3.9 (1.2) 4.0 (3.0, 5.0)	579.0	1.77	.0770	0.22

W is the Wilcoxon rank-sum test statistic (sum of ranks, including tie adjustments), Z is the standardized test statistic with continuity correction, and the 2-sided P value tests group differences. Effect size r is calculated as Z divided by the square root of the total sample size (N) and reflects the magnitude of group differences, with values closer to 1 indicating stronger effects.

^∗Likert scale: 1, never; 2, rarely; 3, sometimes; 4, often; 5, always.

Sensitivity Analyses

Sensitivity analyses limited to the matched subset showed similar results. There were significant improvements in participant confidence, with effect sizes ranged from 0.98 to 1.63, indicating large to very large within-subject effects. There were also improvements across all practice change items, with all but one significant; effect sizes ranged from 0.44 to 1.39, with most in the medium to very large range.

Case Presentation Clinical Challenge Themes

There were 4 common themes to the case presentations: (1) the benefits of pursuing an FASD diagnosis, (2) how to discuss a possible or confirmed FASD diagnosis with a family, (3) how to manage and support the care of a child with an FASD, and (4) how to minimize and address stigma and bias associated with PAE and FASD (Table IV).

Table IV.

Participant case presentation themes

Themes	Example
1. Understanding the potential benefits of pursing an FASD diagnosis	“For a child with a history of PAE with significant behavioral/learning difficulties who is diagnosed with Autism and ADHD, what is the therapeutic benefit of making an FASD diagnosis?”
2. How to have a conversation with a family about a potential FASD diagnosis	“How to best introduce and have conversation with family about referral for FASD evaluation? How to tell them about what to expect for an FASD diagnostic evaluation?”
3. How to support or manage care for a child with a possible or confirmed FASD diagnosis	“How to assist this child and family while awaiting formal diagnosis? How to manage this case with a lack of resources?”
4. How to address and minimize stigma and bias associated with PAE and FASD	“How to ensure that the PAE history is communicated to the child's care team with as little bias against baby and mom as possible?”

Discussion

The SAFEST Choice program successfully trained interprofessional pediatric health care teams throughout the upper Midwest and Northeast US, in 2 year-long Project ECHO FASD learning collaboratives. Although individual participant numbers decreased slightly over time, there were high rates of clinic participation and retention. The participants were highly satisfied with the program, including Project ECHO-related outcomes, such as learning from peers and feeling a sense of community within the program. Confidence, self-efficacy, and self-reported practices increased significantly on topics including screening for PAE, counseling families about PAE and FASD, and making referrals and coordinating care for patients with FASD. In addition, at the end of each session, participants reported high levels of intent to change their practice based on what they learned that day.

This study adds to the literature on the benefits of health care team FASD education. A study that examined the benefits of providing health professionals in Western Australia with educational resources about prevention of PAE and FASD found improvements in knowledge, attitudes, and practice concerning FASD and alcohol consumption in pregnancy.¹⁹ A survey examining the experiences of Australian and New Zealand health and education professionals who had attended FASD-specific trainings found improvements in self-reported FASD-related practices.²⁰ Our study also adds to the literature of training programs specifically using the Project ECHO model targeted at educating pediatric care teams on common conditions affecting pediatric populations. Project ECHO is established as an effective way for academic health centers to train care teams in resource constrained settings on complex health issues.²¹ The ECHO model uses an “all teach, all learn” approach where participants engage virtually with experts and their peers and learn best practices through case-based discussion in a supportive environment. An ECHO for increasing FASD diagnostic capacity²² demonstrated the feasibility of the model in teaching FASD diagnosis techniques to 19 nurse practitioners. An evaluation of an ECHO on caring for patients with autism spectrum disorders²³ reported increases in participants' ability and confidence to care for patients with autism spectrum disorders.

With the high prevalence of FASD, and inadequate health professional training resulting in insufficient FASD-informed care, there remains a need for FASD education targeted at health care teams who care for children and adolescent patients. This evaluation of the SAFEST Choice program supports the Project ECHO model as a feasible and effective approach for training pediatric health care teams in FASD prevention, identification, and care.

Our evaluation has several limitations. Because our ability to match participant data across time points was limited, we treated the pre- and postprogram survey samples as 2 independent samples to maximize the available analytic sample, which can inflate standard errors and limit the statistical power to detect significant differences. However, sensitivity analyses on the more limited matched sample, using paired sample t tests, were conducted and corroborated the main results. We recognize the potential for non-response and selection bias: the matched sample had greater proportions of physicians, more with ≥21 years in practice, and a larger proportion of pediatric specialists. Importantly, there were no observable or statistically significant differences between matched and unmatched participants on baseline (preprogram) measures of confidence and practice (all P > .20). Because baseline outcomes were similar, differences between matched and unmatched participants are unlikely to explain our findings. We therefore chose to present analyses on the full sample as our primary analysis, with analyses on the matched subset as a sensitivity check.

The self-reported outcomes in our study have the potential for self-assessment bias. We originally collected objective knowledge questions, but our questions resulted in a high number of correct responses at pretest. This left little room for knowledge improvement at posttest. So instead, we used a subjective measure asking participants to attribute knowledge acquisition (related to the hazards of alcohol exposure and the options for screening for alcohol use during pregnancy) to the program. Another opportunity for collecting objective measures could have been with practice change, through chart review. However, we determined that the chart review would have been too burdensome for the learners and would have been a deterrent for enrolling in the learning collaborative. We did ask an intent to change question at the end of each ECHO session that showed high levels of specific plans to change practice; intent to change has been shown to predict actual change in practice.²⁴ Moreover, we interacted with these participants over a 12-month period and had many opportunities to hear anecdotal reports of practice change.

Another limitation of self-reported outcomes is social desirability bias, which would skew the data in the direction of change. We tried to mitigate this bias by having the surveys anonymized and working with a separate evaluation team to administer the surveys. Also, there was a potential for nonresponse bias, as the results were from a sample of participants who responded to the surveys as opposed to the overall pool of participants. This potential bias could have skewed the results in a positive direction. Finally, as with any longitudinal educational program, participants received varying levels of exposure. In our analysis, we included anyone who attended at least one of ten ECHO sessions. Despite including participants with minimal exposure, we still showed positive participant outcomes.

There were lessons learned in this study that can be applied to future FASD training programs. Although we did not systematically collect information on why some participants dropped out of the program, there were informally reported reasons including job turnover and new clinical responsibilities. A condensed program over 5-6 months rather than 12 months, with ECHO sessions every 2 weeks instead of monthly, could potentially offset these participant retention challenges. Originally, enrollment focused on primary care teams; however, we received substantial interest from and enrolled specialty clinics (eg, developmental behavioral pediatrics), highlighting the need for FASD education beyond primary care. In the future, we plan to stratify participant outcome data based on type of practice (eg, generalist vs specialist) to understand the effectiveness of the curriculum in different learners. Finally, although it would be ideal to collect patient and public health-level outcomes (Moore's levels 6-7), requiring participants to collect and report clinical and community level data is time-consuming and would make participant and clinic recruitment more challenging. In fact, many participants and their clinic leadership were reassured that there were no data-reporting requirements to join this learning collaborative. Developing creative ways to collect clinical FASD-related outcomes following a training program without overly burdening the participants or their clinic is needed.²⁵

We believe our findings are generalizable to primary care teams beyond those in our target regions of the Upper Midwest and Northeast states because we trained a wide variety of clinics from rural, urban, suburban, and Native American communities with differing access to resources. However, generalizability should be confirmed through future trainings in other settings within the US, as well as in other countries.

In summary, the SAFEST Choice Learning Collaborative was able to enroll, engage, and retain a geographically and professionally diverse group of pediatric providers into a longitudinal FASD learning program using the project ECHO model. The program successfully improved participants' self-reported knowledge, confidence, and practice in identification and care management of children and adolescents with possible or diagnosed FASD.

CRediT authorship contribution statement

Daniel P. Alford: Writing – review & editing, Writing – original draft, Supervision, Methodology, Funding acquisition, Conceptualization. Jacqueline S. German: Writing – review & editing, Writing – original draft, Project administration, Methodology, Funding acquisition, Conceptualization. Nicole Kitten: Writing – review & editing, Writing – original draft, Project administration, Conceptualization. Kendra Gludt: Writing – review & editing, Methodology, Funding acquisition, Conceptualization. Sara Messelt: Writing – review & editing, Funding acquisition, Conceptualization. Ilana Hardesty: Writing – review & editing, Funding acquisition, Conceptualization. Jacey A. Greece: Writing – review & editing, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. Candice Bangham: Writing – review & editing, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. Amy Harlowe: Writing – review & editing, Conceptualization. Michael R. Winter: Writing – review & editing, Project administration, Formal analysis. Martha S. Fermín: Writing – review & editing, Supervision, Conceptualization. Vincent C. Smith: Writing – review & editing, Methodology, Funding acquisition, Conceptualization.

Declaration of Competing Interest

All phases of this study were supported by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS), award no. UT9MC39477, as part of a financial assistance award totaling $3,873,242.00 with 100% funded by HRSA/HHS. The contents are those of the author(s) and do not necessarily represent the official views of, nor an endorsement by, HRSA/HHS, or the US Government. The authors have no disclosures or conflicts of interest to report.