Conducting a Community Engagement Studio to Adapt Enhanced Milieu Teaching

Abstract

Purpose:

Barriers to implementing evidence-based practices occur at various levels. Stakeholder input is required to identify challenges specific to clinical practice settings, client populations, and service delivery approaches. The purpose of this project was to solicit feedback from stakeholders on the telepractice service delivery and implementation strategies proposed for a future study of enhanced milieu teaching (EMT) in rural counties.

Method:

A Community Engagement Studio was conducted with 11 caregivers of children with language delays living in rural counties. Caregivers and the researchers discussed early intervention service delivery for children with language delays in rural Oregon and the proposed telepractice EMT procedures. Researchers gathered feedback on three intervention components: session frequency and schedule, implementation strategies to encourage caregivers' use of EMT, and performance feedback techniques to teach caregivers.

Results:

Findings from the Community Engagement Studio led to four primary modifications to the telepractice EMT study protocol. The principal investigator increased available days and times for intervention sessions and added text-message reminders for parents. A survey was also added for caregivers to identify their preferences for additional implementation strategies (e.g., tip sheets, checklist, e-mailed session summaries) and graphic representations of performance feedback (e.g., bar graph, radial graph, mountain climber infographic).

Conclusion:

Community Engagement Studios are a promising method for increasing community engagement in clinical research and soliciting stakeholder feedback on evidence-based intervention adaptations.

Supplemental Material:

https://doi.org/10.23641/asha.17774819

Applying evidence-based interventions in routine clinical practice is challenging for communication sciences and disorders professionals (Douglas & Burshnic, 2019; Olswang & Prelock, 2015; Zipoli & Kennedy, 2005). Although many interventions are effective, researchers know less about adapting these approaches for local contexts such as clinical practice settings, client populations, and service delivery models (B. J. Cunningham & Cardy, 2020; Douglas et al., 2015; Olswang & Goldstein, 2017). Implementation scientists investigate the processes and methods applied to promote wide-scale adoption of evidence-based interventions in routine clinical practice (Bauer et al., 2015; Eccles et al., 2009; Eccles & Mittman, 2006; Glasgow et al., 2012). These scientists have prioritized stakeholder involvement in the research process because successful implementation of evidence-based interventions depends on stakeholder engagement (Damschroder et al., 2009; Fixsen et al., 2013). Necessary stakeholders in language intervention research often include clinicians providing the intervention, individuals with communication disorders, caregivers and family members, and administrators involved in service delivery. This article introduces the Community Engagement Studio (CES; Joosten et al., 2015) model for soliciting stakeholder input on clinical research. Findings from a CES conducted to inform a future feasibility study of enhanced milieu teaching (EMT; Kaiser & Hampton, 2016), a caregiver-implemented language intervention, are reported to demonstrate the CES model.

Preparing for Language Intervention Research

Implementation barriers occur at multiple levels, and stakeholder input is required to identify challenges specific to the local context (Glasgow et al., 2012; Powell et al., 2012; Proctor et al., 2011, 2013). Formatively identifying implementation barriers before applying an evidence-based intervention in routine clinical practice increases the likelihood of successful adoption and sustained use of an intervention (Damschroder et al., 2009; Hammer & Sawyer, 2016). Engaging with stakeholders early in the research process ensures their expertise is integrated into the research program; the approach is tailored for the local context; and the activities are acceptable, appropriate, feasible, and sustainable for those involved in clinical implementation (Greenhalgh et al., 2004; Klingner et al., 2013; Proctor et al., 2011, 2013).

Engagement with stakeholders is critical when systematically adapting interventions for a new population, clinical setting, or service delivery model (Card et al., 2011; Damschroder et al., 2009; Escoffery et al., 2018; Fixsen et al., 2013). However, few researchers have expertise in engaging stakeholders in the research, implementation, and dissemination processes (Bayne et al., 2020). In addition, researchers frequently have insufficient institutional support to engage with community partners (Fixsen et al., 2013). This lack of training and institutional support for community-engaged research methods contributes to the research-to-practice gap and difficulty bringing proven interventions to scale (Olswang & Goldstein, 2017).

Over the last decade, there has been a call for increased stakeholder engagement to improve the implementation and dissemination of evidence-based practices (Cook et al., 2013; Goldstein et al., 2014; Klingner et al., 2013; Smith et al., 2013). In response, communication sciences and disorders researchers have prioritized assessing the social validity (Wolf, 1978) of a language intervention's goals, procedures, and outcomes (Barton & Meadan, 2019; Cycyk & Huerta, 2020; D'Agostino et al., 2019; Larson et al., 2020; Lindo & Elleman, 2010; Olswang & Goldstein, 2017). Multiple methods exist for obtaining stakeholder input on language interventions, including surveys (e.g., B. J. Cunningham & Cardy, 2020; Peredo et al., 2018), focus groups (e.g., Binger et al., 2008; Cycyk & Huerta, 2020), interviews (e.g., Broder-Fingert et al., 2019; Feuerstein et al., 2018), and convening meetings (e.g., Odom et al., 2013). Although language intervention researchers have primarily conducted summative social validity assessments after delivering an intervention, researchers have argued for the importance of conducting social validity measures before, during, and after implementing an intervention (Goldstein, 2016; Olswang & Goldstein, 2017). Additionally, when adapting interventions for new contexts, service delivery models, and populations, it is essential to collaborate with community stakeholders early in the adaptation process before conducting large-scale evaluations (Escoffery et al., 2018). One important goal of these early collaborations is to anticipate implementation barriers and evaluate implementation strategies proposed for the study. Although researchers could adopt many approaches to accomplish this goal, one innovative approach, the CES (Joosten et al., 2015, 2018), is pertinent for this task.

CESs

The Meharry–Vanderbilt Community Engaged Research Core developed the CES model summarized in Figure 1 (Israel et al., 2018). A CES is a structured, consultative model for engaging patients and community members in research by facilitating a balanced discussion between researchers and stakeholders focused on evaluating specific research components such as the recruitment approach, survey instruments, instructional materials, and implementation and dissemination strategies (Byrne et al., 2012; Joosten et al., 2015, 2018). The CES goal is to help researchers receive rapid feedback from patients and community members with lived experience of a particular condition or membership in a specific community to adapt their proposals to their stakeholders' needs before initiating a project (Coulter et al., 2019; Joosten et al., 2015; Nielson et al., 2020). Thus, the CES model is advantageous for researchers with an evidence-based practice that they want to adapt or disseminate widely but need insight from community stakeholders on specific project components before initiating their study.

Figure 1.

Seven steps of the Community Engagement Studio (CES).

A team including a neutral facilitator, a community navigator, and staff to coordinate logistics conduct the CES. All team members have expertise in community engagement and have no affiliation with the research. The CES facilitator has skills in leading discussions with diverse participants; they prepare the researcher and stakeholders for the CES presentation and discussion, guide their conversation, and summarize the CES discussion in a written report. A neutral party's presence helps mitigate power differentials between stakeholders and researchers, reducing the stakeholders' hesitancy to contribute ideas during the discussion (Novak et al., 2020). The community navigator invites community stakeholders representing the population of interest to participate in the CES as research consultants. In the CES model, stakeholders are viewed as experts because they possess valuable firsthand knowledge, which the researcher integrates to improve the study (Joosten et al., 2015). Therefore, following CES methods and terminology, stakeholders will be referred to as community experts. The term caregiver experts will refer to participants in the CES on EMT throughout this article.

The CES model addresses several common barriers to community engagement. First, the model includes researcher instruction and coaching on community engagement principles and communicating with nonacademic audiences, addressing researchers' lack of training in convening community members (Bayne et al., 2020). Second, CESs foster an equitable balance of power between researchers and community experts by including a neutral CES facilitator and inviting community experts to be research consultants (Joosten et al., 2018; Michener et al., 2012; Nielson et al., 2020; Olswang & Prelock, 2015). Third, the CES team reduces institutional barriers to support community engagement by advocating for community-engaged research while developing trusting relationships with community groups and members (Israel et al., 2018; Schmittdiel et al., 2010). Although the goals of a CES align closely with the goals of other community engagement models, conducting a CES is different than conducting general focus groups or interviews with community members in nine key areas outlined in Table 1. CESs provide researchers a quick way to gain community input on specific project components (e.g., patient portal, social media strategies for dissemination). Finally, although researchers can use the CES model to inform a qualitative study, they are not intended as a primary qualitative analysis method (Israel et al., 2018). Instead, researchers use the CES as a community engagement approach, and the primary goal is to obtain immediate feedback from your population of interest on specific research components before initiating a study.

Table 1.

Comparing Community Engagement Studios (CESs) to traditional focus groups.

Key areas of comparison	Community Engagement Studio	Traditional focus groups and interviews
Primary goal	Guide the development, implementation, or dissemination of research Answer specific questions about the feasibility of research components (e.g., recruitment approach, instruments) Increase relevance of approach for communities Community engagement	Qualitative data collection and analysis Answer specific research questions about a phenomenon or population of interest
Preparation
Researcher	IRB approval (not required) Receives coaching on engaging with community experts and adapting research presentation for a nonacademic audience (4 hr) Researcher and CES staff co-construct discussion guide to achieve researcher goals.	IRB approval Writes IRB-approved script to guide focus group or interview
Participants	Orientation to research process and CES expectations for community experts (30 min)	No formal orientation
Community navigator role	Neutral party with expertise in community organizing and engagement Identifies, recruits, and helps prepare community experts Manages logistics Shares researcher's changes to study as a result of community expert feedback	Not an official role Research team member recruits and consent participants and manages logistics
Facilitator role	Neutral party Expertise in community-engaged research methods and leading discussions with diverse participants Uses strategies to balance power Adheres to CES fidelity guidelines	Research team member Expertise in research methodology Adheres to IRB-approved script
Participant role	Research consultant Community expert with relevant lived experiences	Research subjects Meet specified eligibility criteria
Approach to discussion	Bidirectional Flexible conversation guide with question prompts Diversions are appropriate if relevant.	Unidirectional Scripted questions to guide discussion Little to no diversions allowed
Summarizing findings	CES staff transcribes quotes and organizes comments and recommendations into a summary report for researchers. CES staff may help researchers interpret findings and adapt approach.	Researchers transcribe and code responses. Researchers apply qualitative methods to code participant responses to identify relevant themes.
Project duration	Typical time span of 2–5 months Rapid feedback cycle	Typical time span > 6 months Multistage, recursive, feedback cycle

Note. IRB = institutional review board.

CESs have been used in health services research to enhance participant recruitment and adapt study procedures to serve the target population better (J. Cunningham et al., 2015; Johnson et al., 2015; Nielson et al., 2020). For example, researchers adapted participant recruitment strategies after conducting a CES, leading to meeting recruitment goals ahead of schedule (Johnson et al., 2015) and a reduced attrition rate compared to similar studies (Nielson et al., 2020). Other researchers used feedback from the CES to identify and eliminate culturally insensitive survey items (J. Cunningham et al., 2015).

Researchers report high satisfaction with the CES model (Joosten et al., 2015, 2018). For example, Joosten et al. (2018) conducted surveys and interviews with researchers who completed a CES. Survey findings revealed that the CES led to most researchers reporting they understood community needs and research participation barriers better (61.8%, n = 21) and had a greater appreciation of patient/community input on research (76.5%, n = 26). During interviews, researchers highlighted three major themes regarding the benefits and importance of the CES model: engaging with community experts early in the research process; researchers need coaching before interacting with community experts; and neutral facilitators have a critical role in balancing power and encouraging bidirectional, candid discussions (Joosten et al., 2018).

EMT

The purpose of this project was to conduct a CES and utilize the community expert feedback to adapt EMT before pilot-testing these adaptations in a randomized controlled trial among toddlers with language impairments and their caregivers living in rural communities. EMT is an evidence-based, naturalistic, early language intervention for children aged 18–60 months with language impairments (Kaiser & Hampton, 2016; Kaiser & Roberts, 2013; Roberts & Kaiser, 2015). Therapists, teachers, and caregivers can provide EMT (McLeod et al., 2019; Roberts & Kaiser, 2015). Caregiver-implemented EMT involves a therapist teaching a caregiver to use specific language support strategies with their child. Therapists provide instruction to caregivers during twenty-four 50-min intervention sessions held twice per week. During intervention sessions, caregivers are coached to use EMT language strategies during toy play and typical home routines such as getting dressed, reading books, playing music, and doing chores. Therapists adhere to a treatment manual and use a research-based coaching framework, Teach-Model-Coach-Review (Roberts et al., 2014).

We chose to adapt EMT for implementation in rural communities because (a) it has robust positive effects on caregiver use of language support strategies and child language outcomes, (b) it has a treatment manual with procedural fidelity measures for therapists and caregivers, (c) it uses an empirically supported caregiver coaching framework, (d) it has been adapted successfully for children with various types of communication impairments, and (e) it has been modified for hybrid in-person and telepractice service delivery (Hampton et al., 2019, 2020; Kaiser & Roberts, 2013; Peredo et al., 2018; Quinn et al., 2021; Roberts & Kaiser, 2015; Scherer et al., 2020; Wright et al., 2013). Exploring early language intervention delivery adaptations for rural communities is important because rural children experience health disparities and service access barriers (Peltz et al., 2016; Robinson, Bitsko, et al., 2017; Robinson, Holbrook, et al., 2017; Skinner & Slifkin, 2007). One in five children in rural areas has developmental disabilities, and the prevalence of developmental disabilities is significantly higher in rural communities (Zablotsky et al., 2019). Rural children are also less likely to receive early intervention and special education services than their urban peers (Zablotsky et al., 2019). Consequently, there is a critical need to investigate innovative service delivery models such as hybrid and telepractice approaches with the potential to mitigate service access barriers and reduce health disparities in rural communities (Meadan et al., 2016; Quinn et al., 2021).

We conducted a CES to obtain specific feedback on three components of the proposed telehealth EMT study protocol: intervention dosage (session frequency and schedule), implementation strategies to increase treatment adherence (use of EMT strategies outside of therapy sessions), and performance feedback methods (visualizing caregivers' accuracy and frequency of EMT strategy use). Four questions guided the CES: (Q1) What are caregivers of young children with language disorders experiences with early intervention service delivery in rural Oregon? (Q2) What are caregivers' preferences regarding an EMT session frequency and schedule? (Q3) What are caregivers' preferences regarding implementation strategies to encourage parents' use of EMT language support strategies? (Q4) What are caregivers' preferences for receiving performance feedback?

Method

Language intervention researchers and staff from the Community Outreach, Research, and Engagement at Oregon Health & Science University (OHSU) partnered to develop a CES focused on telepractice EMT. Community Outreach, Research, and Engagement staff support community‑academic partnerships, gather community input for academic research, and increase community-based research that is responsive to the pressing health concerns of residents in Oregon.

CES Team

The CES team included two community engagement specialists who organized, directed, and summarized the CES; they were unaffiliated with the research project. A CES facilitator (K.B.; senior author) supervised all aspects of the CES. She was a senior manager with 6 years of experience leading community engagement projects, trained in CES methodology, and directed OHSU's CES pilot program. A CES community navigator led recruitment efforts, developed and vetted materials, managed logistics, and took notes. She was a community research liaison who lives and works in a rural county in Oregon. OHSU community research liaisons work primarily in rural communities alongside community health care workers and organizations to support research and community and academic collaborations in their regions.

Language Intervention Researchers

Three language intervention researchers contributed to the CES: the principal investigator (PI; first author: E.D.Q.), a speech-language pathology clinical fellow (second author: K.C.), and a research assistant (third author: K.K.). The PI was a certified speech-language pathologist with 11 years of experience providing interventions to children with language disorders. She led the Telepractice Enhanced Milieu Teaching Project (IRB No. 00019997). The clinical fellow had a master's degree in speech-language pathology and was enrolled in the OHSU Leadership Education in Neurodevelopmental and Related Disabilities program. The research assistant was a graduate student with 10 years of experience in early childhood education. The clinical fellow and research assistant analyzed data from the CES and identified themes from these data sources.

Caregiver Experts

A panel of 11 caregiver experts reflective of rural Oregon demographics participated in the CES. The target sample size recommended in the CES model is eight to 12 community experts to ensure various experiences to draw from while maintaining a conversational structure (Israel et al., 2018). Caregiver expert inclusion criteria were as follows: (a) role as a parent, custodial caregiver, or legal guardian, per self-report to CES facilitator; (b) child with a language impairment per medical record or parent report to CES facilitator; and (c) residence in a rural Oregon county defined as an area of land at least 10 miles from the centroid of a population of 40,000 or more persons (Oregon Office of Rural Health, 2020). All caregiver experts had personal experience of parenting a child with a language delay, which, according to CES principles, establishes them as “experts” (Israel et al., 2018).

Caregiver experts self-reported their racial and ethnic information. Ten women and one man participated. Eight caregiver experts (72%) identified as White, not Hispanic or Latinx, and three participants (27%) did not disclose their race/ethnicity. Demographic data from rural Oregon counties indicate the population in each racial and ethnic group is as follows: 83.9% White; 1.5% American Indian/Alaska Native; 1.3% Asian; 2.8%, 9.8% Hispanic/Latinx; 2.8% with two or more races; and less than 1% Black/African American, Native Hawaiian/Pacific Islander, or some other race (Oregon Office of Rural Health, 2020). The majority of caregiver experts (73%) were between the ages of 25 and 40 years, whereas 27% were aged 40–65 years. Insurance status was 55% on the state-sponsored Oregon Health Plan, 36% were insured by their private employer, and 9% preferred not to disclose. Per results from the Oregon Health Authority (2021), approximately one quarter of all insured Oregonians receive their insurance from the Oregon Health Plan, the state-sponsored Medicaid program. However, of the Oregonians on the Oregon Health Plan, 60.5% come from rural areas, despite only 16% of the total population living in rural areas. Three rural Oregon regions were represented by the caregiver experts: Columbia Gorge (n = 3), North Coast (n = 6), and Central Oregon (n = 2).

Setting

The CES team initially planned to conduct the telepractice EMT CES in person. However, due to the public health impacts of COVID-19, the CES team adapted the procedures for an online CES using videoconferencing software.

Materials

Zoom Client for Meetings software and the Zoom mobile app were used for videoconferencing. Microsoft PowerPoint was used to share the research presentation and discussion questions. The research presentation slideshow included videos and images illustrating EMT intervention procedures, equipment, and materials (see Supplemental Material S2). These slides included four short (45–60 s) video examples of the therapist teaching a caregiver EMT strategies, modeling EMT strategies, coaching the caregiver to use EMT strategies, and reviewing the caregiver's performance during the session while reflecting on how to use the EMT strategies throughout the day. The presentation slides also included images of the telepractice equipment given to families, EMT instructional materials, implementation strategies to increase EMT treatment adherence, and data visualizations to provide performance feedback. Telepractice equipment provided to families included a Kubi telepresence robot (remote-controlled iPad stand), a Bluetooth headset, and an iPad with a Zoom mobile app. Instructional materials included the EMT handbook and tip sheets describing EMT strategies. Images of eight implementation strategies (resources) were shown, including (a) an EMT video library, (b) a web portal, (c) weekly text message tips on using EMT strategies, (d) daily text message tips, (e) prompts for self-reflection, (f) an online file-sharing application (Box) to upload videos for therapist feedback between sessions, (g) placemats labeled with times for using EMT strategies during mealtimes, and (h) downloadable tip sheets. Data visualizations included a bar graph, a radial graph, a mountain climber infographic, and a walking path infographic. These graphs visualized caregivers' fidelity to the EMT strategies (e.g., changes in accuracy and frequency of EMT strategy use) and children's communication growth (e.g., changes in the number of different words, communication rate).

Design and Procedure

A CES was conducted to develop adaptations to EMT telepractice delivery in rural communities. It followed the seven-step process outlined by Joosten et al. (2015, 2018): (a) consultation, (b) creation of a caregiver expert panel, (c) researcher coaching, (d) caregiver expert orientation, (e) CES research presentation and discussion, (f) caregiver expert compensation, and (g) post–CES-presentation summary. The CES facilitator used a fidelity checklist to ensure adherence to all CES procedures (see Supplemental Material S1).

Step 1: Consultation

The CES team and the PI had an initial 60-min online consultation in January 2020. During the consultation, the CES facilitator provided an overview of the CES model and answered the PI's questions. Then, the PI described EMT, shared background on routine early intervention service delivery and telepractice, and specified the type of feedback she hoped to solicit from the caregiver expert panel. Next, the PI and the CES facilitator developed two CES goals: (a) to discuss early intervention service delivery for children with language delays in rural Oregon and (b) to gather feedback on the proposed adaptations to EMT for telepractice service delivery in rural communities. Specifically, the PI wanted the caregiver experts to evaluate three intervention components: (a) session frequency and schedule (e.g., weekly vs. twice-weekly sessions), (b) implementation strategies to increase treatment adherence outside of therapy sessions, and (c) performance feedback strategies (e.g., visualizing caregiver progress in graphs). Last, the PI defined the population of interest and caregiver expert eligibility criteria.

Step 2: Creation of Caregiver Expert Panel

The CES community navigator led a two-tiered purposive sampling approach to recruit parents or caregivers of young children with language impairments from rural counties in Oregon. First, she met online with two community-based organizations that serve families in rural areas receiving early intervention. She collaborated with the organizations to develop an appropriate recruitment approach. The community organizations posted flyers in their office locations and social media platforms and sent e-mails announcing the CES. Second, the CES community navigator held an online meeting for four OHSU community liaisons (not directly involved in the CES) to identify other community organizations interacting with rural families. The CES community navigator disseminated flyers, e-mails, and social media posts to OHSU community liaisons to share broadly in rural counties. OHSU community liaisons shared the CES opportunity through e-mail and social media until the CES team met their recruitment goals. Recruitment occurred over 6 weeks.

Step 3: Researcher Coaching

The CES facilitator and the PI held two 60-min coaching meetings. During these meetings, the CES facilitator taught the PI core community-engaged research principles and coached the PI on communicating to the caregiver expert panel in plain language. For example, “How can we say this to ensure all of our caregiver experts understand?” The CES facilitator also helped the researcher clarify her discussion questions to meet the goals of the CES. For example, “Let's make this a more open-ended question so we can hear about how this might work with different types of families.” The CES facilitator provided written and verbal feedback to ensure the presentation and discussion questions led to a meaningful discussion and inclusion of various caregiver expert perspectives. Across the coaching meetings, the PI and the CES facilitator created the CES Discussion Outline and CES Presentation Slideshow (see Supplemental Material S2).

Step 4: Caregiver Expert Orientation

One week before the CES meeting, caregiver experts met with the CES facilitator for an individual orientation. During this 30-min orientation, the CES facilitator confirmed the caregiver experts met the inclusion criteria, provided a job description that summarized their responsibilities, and taught them to use Zoom videoconferencing (as needed). After describing the caregiver expert role, the CES facilitator provided a summary of the telepractice EMT study and an outline of the CES activities. The orientation provided the CES facilitator an opportunity to build rapport with caregiver experts, assess comfort sharing information in a group setting, gain confidence that their participation and input are valuable to the PI, and practice videoconferencing before the CES presentation (Joosten et al., 2015).

Step 5: CES Research Presentation and Discussion

The CES was conducted in November 2020 and lasted 2 hr. First, the CES facilitator introduced the roles of each participant (CES facilitator, CES community navigator, PI, and caregiver experts). Next, the PI gave a brief research presentation that summarized routine delivery of early intervention in rural communities and telepractice service delivery models and explained the proposed telepractice EMT study.

During the research presentation, the PI showed four video examples of a therapist providing EMT via telepractice. Each video clip depicted a major activity completed by the therapist during an EMT session: teaching, modeling, coaching, and reviewing. The first video depicted a therapist teaching a caregiver an EMT language support strategy, responding to all communication. In the second video, the therapist used video modeling to highlight EMT language support strategies for the caregiver. The third video depicted the therapist coaching the caregiver to use EMT language strategies during 10 min of play with their child. In the fourth video, the therapist reviewed the caregivers' EMT strategy use during the session and encouraged the caregiver to reflect on their performance. After sharing the videos, the PI presented images of the telepractice equipment provided to families (e.g., Kubi, Bluetooth headset), instructional materials, and the eight implementation strategies to increase treatment adherence (e.g., daily/weekly text message tips, EMT video library). Then, the PI presented example graphs (e.g., bar, radial) and infographics (e.g., mountain climber, walking path) depicting the caregiver's progress using EMT strategies and children's communication growth.

Discussion questions were interspersed within the presentation, summarized in Table 2. The CES facilitator led the discussion, which began with open-ended questions and included additional prompts to clarify or elaborate caregiver expert responses. There were a total of five discussion rounds to encourage caregiver experts to share their opinions and ask questions.

Table 2.

Discussion questions and prompts to caregiver experts.

Topic area	Discussion question or prompt	Additional prompts
Early intervention (EI)	1. What are your experiences with EI?	What prompted you to access services? What did this look like for your family? If you haven't received services, what would you hope for? Has COVID-19 been a factor?
Telehealth service delivery	2. What is your initial impression of using telehealth for EIs?	Has COVID-19 been a factor? What are the challenges you may have? What are the benefits?
Presentation and video examples of enhanced milieu teaching	3. What are your thoughts on what the researcher described and what you just watched from the video?	How does it differ from what you are receiving now? Can you picture yourself and your family in this style of intervention? Why? Why not?
Frequency and duration of treatment	4. Best outcomes occur when we work with families 1×–2× per week. Do you think this would work for your family or community?	What are the challenges you may have?
Implementation strategies to support treatment adherence	5. The researcher described eight different resources to support families in using the intervention strategies outside of therapy sessions. Are there any that stand out to you as something you feel you would use and work well for you and your family?	Any that you'd never use? Why?
Receiving and giving performance feedback to families	6. Feedback to parents is essential to success of this intervention. How do you and your family like to receive feedback?	Do any of the methods the researcher described appeal to you? Why? Why not?

At the end of the discussion, the CES facilitator explained to the caregiver experts that the research team would use their input to adapt the telepractice EMT study procedures. After completing the CES, caregiver experts were sent a questionnaire to assess their satisfaction.

Step 6: Caregiver Expert Compensation

Caregiver experts received compensation, a $50 Amazon gift card, for participating in the CES. The CES community navigator confirmed that each caregiver expert received compensation.

Step 7: Post–CES-Presentation Summary

In accordance with the CES model, the CES facilitator and community navigator collated each of the caregiver expert's responses recorded verbatim by the CES community navigator in their field notes. Two weeks after the CES, the CES facilitator and community navigator provided a summary report to the investigator including participant demographics, CES team field notes including direct quotes recorded during the CES presentation and discussion, community expert satisfaction ratings, and CES team recommendations. The report highlighted specific recommendations made by the caregivers organized by discussion question and was developed following principles of community engagement and program evaluation (Clinical and Translational Science Awards Consortium Community Engagement Key Function Committee Task Force, 2011) and the CES fidelity guidelines (Israel et al., 2018). Formal quantitative, qualitative, or mixed methods approaches were not used by the CES facilitator and navigator.

Analysis of Data Sources and Selection of Themes

After the CES was conducted, the PI led the analysis of data sources from the CES. Data sources included PI field notes collected during the CES presentation and discussion, CES team field notes collected during the CES presentation and discussion, demographic information, caregiver expert satisfaction ratings, and CES team recommendations. Each researcher (PI, clinical fellow, research assistant) reviewed the following data sources independently and followed an inductive coding process to identify themes from the available data sources. Each researcher noted and recorded possible themes in Microsoft Excel and selected quotations to exemplify these themes. After the initial coding, the researchers met to discuss the themes they identified and quotation selections. The investigator led two coding meetings focused on gaining consensus on themes and prioritizing aspects of the intervention to adapt in the future pilot study. Reliability was not calculated when examining themes across caregivers.

Results

Experiences With Early Intervention

Caregiver experts indicated that their children were referred to early intervention between the ages of 12 months and 3 years. Half of the families were prompted by their pediatrician, and the other half pursued early intervention services on their own or did not mention a physician referral. Five caregiver experts noted their child(ren) received interventions from multiple providers, including speech-language pathologists, occupational therapists, and physical therapists. Caregiver experts described mixed experiences with early intervention services ranging from those who felt it positively impacted their child's development and were supported by therapists to those who felt frustrated when their family was denied services or felt the intervention did not fit their child's needs. Two caregiver experts mentioned that the early intervention provider's preparedness to teach children with an autism spectrum disorder negatively impacted their family's engagement. These caregiver experts reported that they had more difficulty participating in telepractice language interventions, which they attributed in part to the therapist's inexperience providing intervention to toddlers with an autism spectrum disorder. Table 3 summarizes the caregiver experts' experiences with early language interventions.

Table 3.

Caregiver experts' experiences with early intervention (EI).

Identified themes	Caregiver expert feedback
Question 1: What are your experiences with EI?
Benefits	“It was nice to talk to professionals and get help and learn how to help him.”
	“Doing [therapy] at home with the child now, and have seen the difference it makes.”
	“My experience has been really good. The speech therapist came to the house once a week. It's a small town and I knew the speech therapist when she was little so that's fun to have her work with [my child].”
Challenges of EI delivery	“It takes the EI specialist at least 30 minutes to reach our home, we live in a very rural place.”
	“Finally got a referral to EI at the ESD office in [rural county] office. Moved on and got a new person that wasn't very helpful. Didn't have any in house—preferred it that way—didn't like people poking nose around the home situation.”
Question 2: What is your initial impression of using telehealth for EI?
Benefits	“I really liked that I was the one learning to work with [my child] and get pointers.”
	“Allows parents to continue working on [strategies] seven days a week.”
	“Feels like more organic play with children and you are able to do it when you aren't in therapy.”
Challenges to telehealth	“Initially [my child] ignored the SLP on the zoom screen and played with his toys instead. The SLP adapted her teaching style to fit his preferences and he grew more actively engaged with her.”
	“I wish that [we] could get another camera so that the SLP can see what he is pointing to or looking at to more accurately gauge his understanding.”
	“Several children are at home doing online school on top of services for the child in EI. It is hard to coordinate all of the school times and appointments.”

Note. All statements in the caregiver expert feedback column are direct quotes. ESD = Educational Service District; SLP = speech-language pathologist.

Service access barriers mentioned included travel and COVID-19 restrictions. Four caregiver experts noted travel distance for therapists presented challenges to service delivery. Eight caregiver experts said COVID-19 travel restrictions led to hardships and changed their early intervention services, reducing the frequency and duration of sessions. Caregiver experts in rural areas reported early intervention providers often have long travel times to home visits, and communication with these providers can decrease over time. One caregiver expert noted their child's individualized family service plan services had transitioned to “a call once a month with the speech therapist” and “occupational therapy dropped off equipment” with “one e-mail every few months,” despite what the family felt are significant developmental delays. This caregiver expert reached out to her child's pediatrician for a referral to private services, which took 3 months to start.

The sudden transition from in-person sessions to telepractice sessions due to COVID-19 was difficult for many families. One caregiver expert reported that when early intervention services started as telepractice, her family spent “most of the visit chasing [her son] down” and she “wasn't feeling like a lot of progress was happening.” Another caregiver expert mentioned, “One-on-one was going well, so they introduced speech group through a local preschool. This was the best thing that came out of speech therapy. But it went away with COVID. We started Zoom, but it was difficult for the therapist to get him to sit still and pay attention. Now we are doing things like logging onto different apps to read books together.” Access to and the cost of Internet service were a significant barrier to one caregiver expert whose school district and early intervention system did not provide Internet service to families. She said, “I can't afford to use all my minutes on early intervention because I have three kids who need it for school. Instead, the therapist calls me on the phone and gives me ideas. But yeah, it would be great if they could provide it [internet service].”

Reactions to Telepractice EMT

Caregiver experts thought the therapist, child, and family interactions were positive after viewing telepractice EMT video clips. Table 4 summarizes caregiver experts' initial perceptions of the intervention. Most caregiver experts (n = 10) reported that the parent coaching and parents' active involvement in the interactions were essential aspects of EMT. One caregiver expert noted it was “100% different from what we have received where we are off to the side, and the therapist does all the work.” Another caregiver expert enjoyed the “real-time” coaching. She said, “[Telepractice EMT] felt more organic because new people in the home can cause children to behave differently.”

Table 4.

Caregiver expert feedback on enhanced milieu teaching.

Identified themes	Caregiver expert feedback
Question 3: What are your thoughts on what the researcher described and what you watched on the video?
Benefits	“I think this would benefit him more than what we are currently doing.” “This looks similar to another parent coaching intervention we have worked with and seen progress using.” “I haven't felt part of my child's therapy before. He plays with toys in-person that we do not have at home so I cannot continue to work on those same problems.” “Allows the parent to continue working on these things seven days a week, which is what we want. With this model, it lends itself to further discussion and questions on what to do next, so [Investigator] would be able to coach on what to do later.” “I really loved this approach to therapy. I am legally blind and can't drive.”
Challenges	“If felt kind of like a test. If someone was watching from the outside, I feel like I could be a kid in school getting told I was wrong.” “I like the feedback but I have a hard time imagining my child and me staying in front of a still camera.” “Kids are getting too much screen time and are too wound up—but they need to be on the computer [for school, services].”

Note. All statements in the caregiver expert feedback column are direct quotes.

Several caregiver experts (n = 3) expressed that rapport building with the family was essential to positive outcomes with both the child and the parent. One caregiver expert noted, “You [PI] show not only a love for the child but a mutual respect with the parents. Your words of encouragement work for both. Personally, I get afraid I'm screwing up as parent, but to hear the therapist celebrating the parent is so helpful.” This caregiver expert also reported having a spouse with anxiety and that having infrequent treatment sessions has acted as a barrier to rapport building with specialists in the past.

Caregiver experts anticipated various telepractice delivery challenges. One caregiver expert was worried about adding more “screen time” for their family. Four caregiver experts had concerns about telepractice equipment (e.g., camera position, Internet connectivity, sound). Three caregiver experts were concerned about their children being “too distracted” by the therapist's camera and voice, keeping their child engaged in play and in frame of the camera. One caregiver expert thought she might have difficulty limiting distractions of other children or people in the home during sessions while managing the language strategies, observing the child's behavior, and receiving coaching simultaneously. When the therapist mentioned that parents would receive an optional Bluetooth headset and an iPad stand remotely controlled by the therapist (Kubi telepresence robot), eight caregiver experts agreed that these tools would improve feasibility. However, they felt parents needed practice with these tools before beginning therapy.

Preferences for Session Schedule and Frequency

Two treatment sessions per week was the frequency preferred by most in the caregiver expert panel (n = 7). Four caregiver experts agreed twice a week would work, and three agreed but clarified that it depended on the family situation and timing of the intervention sessions. One caregiver expert noted, “With one, you tend to forget so twice a week is a good reinforcer of what you learned.” COVID-19 had a significant impact on families' availability and scheduling preferences. One caregiver expert commented some families have been “forced to work from home and have more flexibility with their work schedules.” Another caregiver expert explained she would “find it difficult to schedule especially when her other children had appointments with specialists or online school.” Caregiver experts had two primary suggestions regarding session frequency and scheduling. First, offer late afternoon, early evening, and weekend appointments to accommodate different work schedules. Second, give families the option to schedule therapy times on a week-to-week basis so that families could choose the times that work best for a specific week's schedule. Table 5 outlines the caregiver experts' comments regarding session frequency and scheduling.

Table 5.

Caregiver expert feedback on enhanced milieu teaching frequency and session schedule.

Identified themes	Caregiver expert feedback
Question 4: Would 1-hr sessions 1×–2× per week work for your family?
Benefits	“Two is a great number per week.” “Two is a good number. One, you tend to forget. Two is a good reinforcer of what you learned.” “Two is reasonable. Many parents are forced to work from home and have more flexibility to work with schedules. Not sure how they would do it if they were not working from home. One time is not enough because they often forget.” “Having evening/weekend times would be really helpful for my family.”
Challenges	“It might be possible to do two a week, but it will be hard to fit in with school, visits with bio parents, and work. It would have to be a group effort.” “Could do twice if weekends/evenings available. We have appointments with EI as it is during the day and it's like pulling teeth but with both kids involved and everything else going on.” “Twice a week. Speech sticks better (they see their therapist twice a week). Have the blessing to be a ‘stay at home mom’ and can do that. Do wonder about the working parents and how they would make it work. Is it possible to do later times when parents come home?” “Work full time and husband did work full time before COVID. Because of anxiety needs someone else there as a support. Meet during lunch time and on weekends.” “I do wonder about the working parents and how they would make it work.”

Note. All statements in the caregiver expert feedback column are direct quotes. EI = early intervention.

Preferences for Implementation Strategies to Increase Treatment Adherence

Most caregiver experts (n = 9) wanted to be sent weekly text message reminders to use specific language strategies during the week in between therapy sessions. Still, there was no consensus on how often the therapist should send text messages. Individualization was important to caregiver experts; they wanted to select multiple implementation strategies from the list of potential treatment adherence options (see Table 6). Several caregiver experts (n = 3) said that visual aids (e.g., downloadable tip sheets, placemats with suggestions on ways to practice the EMT strategies) would help them remember to use the strategies. One expert reported, “I like the placemats and any kind of visual aid.” Another said, “I like the idea of the parent taking a video and sending in to the therapist [for feedback].” Options such as e-mailed performance feedback, EMT video library, and tip sheets, which caregivers could share with friends and family members, had high approval among caregiver experts. One caregiver expert noted, “A lot of times, only one parent can participate,” and written feedback could help include the other parent in the strategies. Caregiver experts had two additional suggestions for implementation strategies: (a) set a separate time to talk directly to the clinician, ask questions based on the feedback they received, and review the goals for their family and (b) use existing social networks, such as Instagram, Facebook, and WhatsApp, to disseminate information to parents. None of the caregiver experts preferred questions promoting self-reflection.

Table 6.

Caregiver expert feedback on implementation strategies to increase treatment adherence.

Identified themes	Community expert feedback
Question 5: The researcher described eight different resources to support families in using the intervention strategies outside of therapy sessions. Are there any that stand out to you as something you feel you would use and work well for you and your family?
Online communication with the therapist (text message tips, uploading videos)	“I like the idea of daily text tips.”
	“I would like to be able to send voice message to send to the therapist.”
	“Uploading video or audio clip—I have already done this for my speech therapist.”
	“I like the idea of the parent taking a video and sending in to the therapist.”
Visual aids (tip sheets, placemats)	“Placemat is great.”
	“Lots of visual aids. Velcro to different spots in the house because my son can't vocalize. But I can't print at my house to my therapist will laminate and mail them to us. That works for us.”
Online website with resources (video library and web portal)	“A lot of those resources seem to be good for a platform like Instagram. Being able to save videos, get daily tips, and even direct messaging.”
	“Most helpful is the web portal where you get to interact with other parents. Also where you can upload a video to get suggestions on how to diffuse the situations. Least helpful is the snack time place mats with tips…. I like self-directed learning options that would help you, that are individualized for what you need.”
	“A video library for ideas of what worked with other families would be helpful for specific behaviors. There's a lot I'm not sure how to manage just on my own.”
Interest in combining multiple strategies	“I really like the uploading a video or audio clip. I also really like the placemats and any kind of visual aid. We have had some success with the laminated picture cards and ‘first/then’ boards.”
	“I would like a video of other sessions and text messages.”
	“I really like the moment to moment feedback along with e-mail feedback.”
	“Videos of other sessions are helpful and text messages. Cute idea for placemats.”
	“Uploading video or audio clip, have already done this for speech therapist. Liked daily text message tips.”

Note. All statements in the caregiver expert feedback column are direct quotes.

Preferences for Receiving Feedback From the Therapist

Five caregiver experts indicated an interest in charts and data visualization (e.g., bar graphs, radial graphs, or infographics) because they “like seeing the progress for what it is.” However, other caregiver experts expressed concern that data visualization would only capture a “snapshot” of their child's skills based on that particular day. For those who advocated for written feedback (n = 3), caregiver experts noted it would help as an additional reminder of what to practice between visits and would help explain the strategies to other caregivers working with their child.

As with caregiver expert preferences for additional intervention supports, individualization was stressed by the caregiver experts (n = 3), noting that successful feedback might look different for each family. Several (n = 6) caregiver experts additionally noted that rapport with the clinician was essential to successful feedback. They said that when receiving feedback from interventionists in the past, it felt as though they were “screwing up” as a parent, making it hard for them to stay engaged with the treatment. Table 7 summarizes caregiver experts' comments on performance feedback.

Table 7.

Caregiver expert input on performance feedback.

Themes	Caregiver expert feedback
Question 6: How do you like to receive feedback?
Written	“Written feedback lets parent stay in the moment and not feel they have to write everything down.” “I like written feedback and going over the written feedback on a Zoom meeting with specific examples of what has gone right, and what to further work on.” “Some weeks my child does well and other weeks he doesn't. It's hard to see that on a graph.” “A lot of times, only one parent can participate and written feedback could help include the other parent in the strategies.”
Graphic	“I like seeing the progress for what it is, I'm totally down for the bar graph. I understand some families might not like it, so maybe the mountain climber idea.” “I really like the graph—I'm a visual learner.”

Note. All statements in the caregiver expert feedback column are direct quotes.

Caregiver Expert Satisfaction With the CES

Figure 2 summarizes the caregiver experts' satisfaction with the CES process. Of the 11 caregiver experts who participated in the CES, eight completed the post-CES evaluation survey. When asked if the CES was worth the caregiver expert's time and if the caregiver expert was satisfied with the CES, 100% (n = 8) of the surveyed caregiver experts reported they “strongly agreed.” Most caregiver experts (n = 6) “strongly agreed” their feedback improved the project. When asked if the PI's presentation gave the caregiver expert enough information to provide appropriate feedback, most (n = 6) “strongly agreed.” All caregiver experts “agreed” or “strongly agreed” that the CES included relevant participants for the presented project, that there was sufficient time to participate in the discussion, and that they would participate again.

Figure 2.

Community expert evaluation survey results. CES = Community Engagement Studio.

Caregiver experts were also asked what they felt they contributed to the research project. Half of the respondents (n = 4) responded they provided feedback on the appropriateness of the project for the community, some (n = 3) reported they offered feedback on the feasibility of the project in the community, and others (n = 2) indicated they provided ideas on how to use the results of the project to benefit the community. Other responses included caregiver experts providing ideas on recruiting research participants (n = 1), increasing the PI's sensitivity to the community (n = 1), and increasing the PI's understanding of the community (n = 1).

Modifications to Telepractice EMT

Following the CES, the research team met to discuss potential adaptations to the intervention procedures. Caregiver experts' suggestions during the CES led to the following decisions for the feasibility trial. First, caregiver experts' scheduling preferences led to the addition of 5:00 and 6:00 p.m. appointments on weekdays and 10:00 a.m. to 6:00 p.m. on Saturdays, with options to select a reoccurring appointment time or appointment times that vary by week. The PI set the intervention frequency at twice per week. Second, caregiver experts' preferences for implementation strategies resulted in the therapist sending weekly text message check-ins with reminders to use the target EMT language support strategies. Caregivers will also be able to select from a menu of optional implementation strategies at study entry to customize the implementation strategies for each family. These options include EMT checklist, placemats with tips for using EMT strategies during mealtimes, and a weekly e-mail summarizing parent and child performance. Third, caregiver experts' concerns about the potential for being discouraged by “regression” when data were visualized session by session in a bar graph encouraged the investigators to add a mountain climber infographic and radial bar graph options to visualize the caregivers' and children's performance. Caregivers in the proposed study will select their preference (bar graph, mountain climber, radial progress bar) for visualizing performance so the therapist can provide data-based performance feedback throughout the intervention. Finally, caregiver experts mentioned the lack of telehealth equipment was a potential barrier for some families. As a result, the research team will communicate clearly to participants that materials are available to support each family's participation in telepractice. EMT therapists will provide telepractice equipment (iPad with a Zoom app, Kubi, Bluetooth headset) and will communicate that Internet service will be provided via wireless hot spots as needed.

Discussion

This is the first study in the discipline of communication sciences and disorders to use the CES model. The current study highlights how CES methods can empower caregiver experts to share their perspectives. These lived experiences can help clinical researchers identify which aspects of the intervention need modifications to be relevant, feasible, and sustainable for a particular local context. In this study, we were interested in caregivers' experiences receiving early intervention for their children with language impairments in rural communities. Our goal was to obtain project-specific feedback on the service delivery approach and implementation strategies proposed for a future study of telepractice EMT. Using the CES model, we identified parents' preferences and made four primary changes to our protocol due to their feedback: First, we increased available days and times for intervention by adding evening and Saturday appointments. Second, we expanded our communication with parents by adding weekly text messages to encourage EMT language support strategy use. Third, we provided participant choice in implementation strategies by creating a survey for parents to complete at study entry, which defined the available options. Parents could select as many options as they preferred and modify their selections throughout the study. Fourth, we provided participant choice in the data visualization of parents' performance for the target strategies following each intervention session. Parents selected whether they preferred viewing their performance on a bar graph, mountain climber infographic, or radial graph using the survey mentioned above.

The CES identified several critical issues related to telepractice. First, caregiver experts experienced an abrupt transition to telepractice service delivery due to the public health impacts of COVID-19 with mixed results. Some families had neutral or positive experiences with telepractice, whereas others had very negative experiences or lacked access to telepractice. Second, all caregiver experts reported frustration with telepractice technology or the therapist's ability to deliver the intervention at a distance. These experiences contribute to an emerging body of research on telepractice parent-implemented language interventions (McDuffie et al., 2013, 2015; Meadan et al., 2016; Quinn et al., 2021). Third, caregiver experts' comments highlight the need for systematic pre-intervention training in using telepractice equipment or creating a checklist of procedures to help parents prepare for a telepractice visit. These practices have been implemented in clinical research (e.g., McDuffie et al., 2013, 2015; Meadan et al., 2016) but not widely adopted in routine clinical practice. Fourth, caregiver experts experienced difficulty with their personal technologies, and some lacked Internet connectivity, which highlights the need for early intervention programs to provide families with technology equipment (mobile devices, Bluetooth headsets) and Internet access to ensure equitable access to telepractice service delivery. Last, parents' initial experiences with telepractice during COVID-19 may influence their “buy-in” for future telepractice or hybrid in-person and telepractice interventions. The relationship between end user buy-in and wide-scale adoption is well established (Klingner et al., 2013). Thus, exploring parents' perceptions of telepractice delivery will be beneficial prior to applying a telepractice language intervention in clinical practice or a clinical research study.

There were several advantages to using a CES model to seek community engagement with research planning. During the PI instruction and coaching, the PI gained valuable knowledge and skills in community-engaged research methods, mainly in communicating with rural families. Through interactions with the OHSU community liaisons, the language intervention researchers forged connections with community groups, which would help recruit participants for the future study of EMT in rural communities (e.g., pediatricians' offices, early childhood programs). Using a novel approach to community engagement allowed our team to assess caregiver preferences for critical intervention components, such as session frequency and schedule, implementation strategies to increase treatment adherence, and performance feedback strategies, efficiently with less need for research staff support. The neutral facilitator played an essential role in ensuring the conversation was bidirectional, consistent with prior work. All parties expressed high satisfaction with the CES model.

Limitations

One potential limitation of this CES model is sampling bias. Specifically, there is potential for self-selection bias. Individuals with reliable Internet access and interest in technology may be more likely to take part in a CES on telepractice language interventions. Initially, we planned on holding the CES in person, but we had to switch to videoconferencing due to COVID-19. Although there were options to connect via phone line and participants who indicated interest initially were recontacted, there is a potential for this to influence our findings more positively toward a telepractice intervention. Second, few participants who self-identified as culturally, racially, ethnically, and linguistically diverse participated in the CES. Although we used targeted recruitment strategies (e.g., coordinating with community liaisons, reaching out to community organizations serving diverse families) to increase representation among racially and ethnically diverse communities, these approaches were unsuccessful, and we will pursue alternatives in our intervention study. Finally, results from our study may not represent the experiences of rural parents who have limited English proficiency as well as parents from diverse racial, ethnic, and cultural groups including Black, Indigenous, Latinx, Asian, and other persons of color. Underrepresentation from these groups is important to highlight because a family's multiple cultural identities may impact their experiences accessing services due to implicit racial biases and systemic racism (FitzGerald & Hurst, 2017; Flores, 2010; Raphael & Oyeku, 2020; Trent et al., 2019). These are critical barriers to service access to explore using CES and other community engagement strategies (focus groups, convening meetings) and qualitative research methodology. Our findings should be considered in light of these limitations.

Even though qualitative methods are not traditionally applied in CES or required to conduct a CES successfully, a second potential limitation is that we did not use rigorous qualitative data collection and analysis methods in the design and conduct of the CES. The goals of qualitative research, namely, to understand the experiences of individuals and their environments, were beyond the scope and aims of this project. The primary purpose of the telehealth EMT CES was to obtain stakeholder feedback on specific elements of a research study for rural children with language impairments and their families. Therefore, the CES was not designed to develop a deep understanding of rural families' experiences or identify themes that would have broad applications to other language interventions or generalize to other populations. Accordingly, the findings presented and conclusions drawn should be viewed with this limitation in mind.

Implications and Future Directions

Stakeholder engagement is essential to the implementation and dissemination of language intervention research (Campbell & Douglas, 2017; Feuerstein et al., 2017). Despite the promise of this approach to gain valuable insights from stakeholders, researchers and clinicians may lack access to community engagement specialists. Few specialists may have training to conduct CESs, potentially limiting the feasibility and adoption of this approach. Additionally, researchers who have conducted CESs often had organizational support from a university, including staff with expertise in community engagement and funding to support this work (Coulter et al., 2019; Joosten et al., 2015, 2018; Nielson et al., 2020). Those without administrative support from a university may have difficulty implementing CESs.

Importantly, resources are readily available to support the implementation of the CES model. The Meharry–Vanderbilt Community Engaged Research Core holds an annual summit where participants receive training in the CES method at no cost. A CES toolkit created by the Meharry–Vanderbilt Community Engaged Research Core describes the CES approach in detail and is downloadable from their website (https://victr.vumc.org/community-engagement-studio/). Clinical and Translational Science Awards programs are a consortium of 60 medical research institutions engaged in health services research to reduce the time it takes for scientific discoveries to become treatments, practices, or policies to improve health. Researchers interested in conducting a CES may want to reach out to their local Clinical and Translational Science Awards program. These programs often have community engagement specialists and expertise in conducting CESs. The Patient-Centered Outcomes Research Institute website has an entire section devoted to community engagement, including free training on community engagement methods, which researchers may utilize when planning a CES, as well as traditional focus groups and interviews.

Clinicians who are interested in conducting a CES can reach out to the ASHA Clinicians and Researchers Collaborating program or ASHA Special Interest Groups to identify professionals with expertise in community engagement who may be able to help develop a CES. Although identifying a neutral facilitator may present a challenge, it is an essential aspect of the CES model (Israel et al., 2018; Joosten et al., 2015). Professionals such as social workers, community health workers, and staff from patient or community advocacy organizations (e.g., United Cerebral Palsy Foundation) often have expertise in community engagement and participatory action research. They may be willing to facilitate the CES and help recruit community experts. Although the CES model is new to the discipline of communication sciences and disorders, the focus on community engagement can advance the adoption of implementation science research methods and in hybrid effectiveness–implementation studies especially.

Conclusions

The CES method is an approach to community-engaged research that allows researchers to learn from the lived experiences of community (caregiver) experts and solicit feedback on the acceptability, appropriateness, feasibility, and sustainability of evidence-based practices. The CES conducted on telepractice EMT led to changes in the future study protocol, including increased flexibility in schedules, communication from the research team, and parent choice regarding the implementation strategies.

Funding Statement

The contents of this article were supported in part by the NW Center of Excellence & K12 in Patient-Centered Learning Health Systems Science Award provided by the Agency for Healthcare Research and Quality and Oregon Health & Science University (K12HS026370) and by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant Award UL1TR002369. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the National Institutes of Health.