Hybrid Telepractice Delivery of Enhanced Milieu Teaching: Effects on Caregiver Implementation and Child Communication

Abstract

Purpose

This study investigated the effects of enhanced milieu teaching (EMT) on caregiver implementation of language support strategies and child communication skills using a hybrid telepractice service delivery model.

Method

Four caregivers and children with language delays aged 18–27 months participated in a multiple baseline across behaviors single-case research design. The therapist provided EMT to each caregiver–child dyad using a hybrid telepractice service delivery model with approximately 40% in-person and 60% telepractice sessions. Caregivers were taught to use five EMT language support strategies: matched turns, target talk, expansions, time delays, and milieu teaching episodes. Caregiver instruction followed the Teach–Model–Coach–Review approach. Caregiver outcomes were two measures of EMT implementation fidelity, accuracy, and frequency of EMT strategy use. Accuracy was measured by the percentage of spoken turns in which caregivers used each strategy correctly. Frequency was measured by the number of spoken turns in which caregivers used each strategy correctly. Child outcomes were number of communication acts, weighted count of communication acts, and number of different words.

Results

There was a functional relation between the intervention and the accuracy of EMT strategy use for all four dyads, and the frequency of strategy use for three dyads. Caregiver use of EMT strategies maintained for 6 weeks post-intervention. After caregivers learned EMT strategies, gradual increases in the number of communication acts, weighted count of communication acts, and number of different words occurred for three children.

Conclusion

Results demonstrate the preliminary efficacy of using a hybrid telepractice service delivery model to teach caregivers EMT language support strategies.

Supplemental Material

https://doi.org/10.23641/asha.14977605

Early intervention encompasses the services and supports available for infants and young children with developmental delays and disabilities and their families to advance their development (Individuals with Disabilities Education Act, 2004). However, early intervention is underutilized nationally, with pronounced service gaps in rural areas (Cason et al., 2012; McManus et al., 2020). In addition, there are marked racial and economic disparities in the receipt of early intervention services (Feinberg et al., 2011). Nearly 12% of children under 36 months in the United States experience a communication disorder, yet less than 2% receive early intervention (Rosenberg et al., 2008, 2013; U.S. Department of Education, 2019). Bridging these service gaps is important. Young children with language impairments are at increased risk for learning and reading disabilities that can persist into adulthood and negatively impact future social, educational, and employment outcomes (Justice et al., 2009).

Telepractice is one service delivery model with potential to mitigate service gaps facing children with disabilities and their families, especially for families living in rural areas and in communities experiencing health disparities. It involves the provision of clinical or educational consultation, intervention, and/or assessment via communication technology (American Speech-Language-Hearing Association, 2016). Telepractice enhances service delivery by improving access to treatments, increasing the availability of clinicians, and reducing costs (Baggett et al., 2010; Meadan & Daczewitz, 2015; Molini-Avejonas et al., 2015).

Hybrid telepractice involves providing services through telepractice and in-person delivery. It blends the benefits of in-person and telepractice sessions (Ryer & Poll, 2020; Valentine, 2014). This approach is often used when some direct, in-person contact between families and therapists is preferred, needed for relationship building, diagnosis, or intensive training. Hybrid telepractice is especially useful when delivering interventions that involve coaching. During in-person sessions, there are opportunities for instruction, modeling, and physical guidance while telepractice sessions allow minimally obtrusive coaching and guided practice to independence when acquiring new skills (Baharav & Reiser, 2010).

Caregiver-Implemented Language Interventions

Caregiver-implemented language interventions are an empirically supported approach to early intervention, which involve a therapist teaching a caregiver to use language support strategies in order to improve the child's language skills (Carter et al., 2011; Roberts & Kaiser, 2011, 2015; Romski et al., 2010). Therapists can effectively coach caregivers to use language support strategies (e.g., responsiveness, expansions), improve the quantity and quality of their linguistic input (e.g., lexical diversity, modeling specific language targets), and adapt their everyday activities to provide greater opportunities for child communication (Roberts & Kaiser, 2011, 2015; Salisbury et al., 2018). Two meta-analyses of caregiver-implemented language interventions have indicated that teaching caregivers has significant positive effects on caregiver responsiveness, language modeling, language strategy use, and child skills including communication rate, social communication, receptive language, and expressive language (Heidlage et al., 2020; Roberts & Kaiser, 2011). The studies evaluated in these meta-analyses implemented caregiver instruction in person. None of these studies used a telepractice or hybrid service delivery model (Heidlage et al., 2020; Roberts & Kaiser, 2011).

Delivering caregiver-implemented interventions through telepractice is a promising service delivery model for early intervention (Bosivert et al., 2010; Hall & Bierman, 2015; Knutsen et al., 2016). Caregiver-implemented interventions are well suited for telepractice because they occur in natural environments and incorporate coaching strategies that can be feasibly adopted for telepractice (McDuffie et al., 2013, 2016; Meadan et al., 2016). Telepractice delivery also focuses the caregiver's role as a teacher and the therapist's role as a coach (Salisbury et al., 2018; Woods et al., 2011).

A recent review of single-case design studies that provided caregiver-implemented interventions through telepractice or hybrid models (Quinn et al., 2017) found that several early language interventions improved caregivers' implementation of language support strategies. The Telehealth Early Start Denver Model (Vismara et al., 2012, 2013), Reciprocal Imitation Training (Wainer & Ingersoll, 2013, 2015), Naturalistic Parent-Implemented Intervention (McDuffie et al., 2013, 2016), and the Internet-Based Parent-Implemented Communication Intervention (Meadan et al., 2016) had strong positive effects on caregiver language strategy use. Across studies, caregivers were taught a range of language support strategies including environmental arrangement, language modeling, mand-modeling, time delays, follow-in comments, expansions, prompting, reinforcement, imitation, and linguistic mapping. In these studies, therapists provided caregiver education through workshops, online modules, and coaching (McDuffie et al., 2013; Meadan et al., 2016; Vismara et al., 2012; Wainer & Ingersoll, 2013). These studies provide preliminary empirical support for caregiver-implemented language interventions provided through telepractice and hybrid service delivery models (Quinn et al., 2017).

Additional research is needed to build on this emerging evidence and to explore telepractice and hybrid service delivery models that can be implemented in clinical practice to reduce early intervention service gaps. In the absence of strong evidence, educational and public health organizations may not make substantive and enduring investments in hybrid and telepractice early intervention programs. Without continued research, the therapeutic benefits of hybrid and telepractice early language interventions will remain unrealized, especially for children and families with disproportionate access to early intervention (Baggett et al., 2010; Hall & Bierman, 2015). In summary, there is a critical need to adapt and test existing evidence-based interventions for telepractice and hybrid telepractice service delivery.

Enhanced Milieu Teaching

Enhanced milieu teaching (EMT) is an evidence-based naturalistic, caregiver-implemented intervention for children 18–60 months with language impairments and neurodevelopmental disabilities (Kaiser & Roberts, 2013; Roberts & Kaiser, 2015). Caregivers participate in one or two weekly intervention sessions during which they learn language support strategies (e.g., matched turns, target talk, expansions, time delays, and milieu teaching episodes). Caregivers are coached to use these strategies during toy play and typical home routines including book reading, dressing, and mealtimes (Kaiser & Hampton, 2016). Therapists follow a treatment manual and use a research-based caregiver instruction approach called Teach–Model–Coach–Review (Roberts et al., 2014) to deliver EMT to families (Kaiser et al., 2016) The treatment manual includes clinician procedural fidelity measures and specific criteria for caregiver implementation fidelity.

The Teach–Model–Coach–Review approach uses principles of adult learning to teach caregivers (Dunst & Trivette, 2009; Trivette et al., 2010). The teach component involves an introduction to the EMT strategy. The model component involves an illustration of the EMT strategy through therapist modeling or video modeling. The coach component involves an iterative process of caregiver practice with performance feedback. The review component involves the caregiver and therapist evaluating the session, reflecting on strengths and weaknesses, and problem solving to develop a plan of action for the next session (Roberts et al., 2014). This approach has been applied in several studies to increase caregivers' use of EMT strategies (e.g., Hampton, Kaiser, & Fuller, 2020; Peredo et al., 2018; Roberts & Kaiser, 2015).

EMT has significant positive effects on caregiver responsiveness and language strategy use, child expressive vocabulary, and the frequency of child communication acts when delivered to children and their families in person (Kaiser & Hampton, 2016; Roberts & Kaiser, 2015). A child communication act is defined as a signaling behavior, which is socially directed toward a communication partner. In-person delivery of EMT is effective for increasing the expressive communication skills of children with a variety of etiologies including children with cleft lip and palate (Scherer et al., 2020), Down syndrome (Wright et al., 2013), intellectual and developmental disabilities (Kaiser & Roberts, 2013), language delays (Roberts & Kaiser, 2015), and English- and Spanish-speaking children with language delays (Peredo et al., 2018).

Study Purpose and Research Questions

The current study addresses the critical need for evidence-based telehealth interventions by providing EMT (Kaiser & Hampton, 2016) to caregivers of toddlers with a variety of language impairments using a hybrid telepractice service delivery model. This study expands research on EMT and other caregiver-implemented language interventions in five important ways. It is the first study to adapt the EMT procedures, including the Teach–Model–Coach–Review approach for hybrid telepractice delivery. Utilizing an empirically supported intervention to test telepractice adaptations minimizes the risk that null or negative results are due to an ineffective intervention program. A hybrid telepractice model was selected because it combines the benefits of in-person instruction, modeling, and coaching (Friedman & Woods, 2015; Kaiser & Roberts, 2013), with the efficiency and flexibility of telepractice service delivery (Baggett et al., 2010; Cason et al., 2012; Knutsen et al., 2016).

Second, therapists provided caregiver education and coaching on EMT strategies using the Teach–Model–Coach–Review approach (Roberts et al., 2014). Coaching caregivers with a research-based approach allows the investigators to focus efforts on developing coaching adaptations for telepractice. This minimizes the risk that null or negative results are due to ineffective caregiver education practices.

Third, the telepractice intervention was provided at a higher dosage than previous studies (e.g., Vismara et al., 2012, 2013; Wainer & Ingersoll, 2013). The investigators adhered to the recommended treatment intensity for EMT in-person delivery, twice per week, over 4 months (e.g., Peredo et al., 2018; Roberts & Kaiser, 2015; Roberts et al., 2014). This minimizes the risk that null or negative results are due to inadequate treatment intensity.

Fourth, participating caregiver–child dyads included children with language impairments associated with various etiologies. The efficacy of EMT has been demonstrated in six randomized control trials with children with language impairments and various neurodevelopmental disabilities (Hampton, Kaiser, & Fuller, 2020; Kaiser & Roberts, 2013; Kasari et al., 2014; Roberts & Kaiser, 2015; Scherer et al., 2020). The investigators included children who have a range of etiologies to represent the population receiving early intervention services in the United States. Exploring effects with a range of children is critical because early intervention programs need access to evidence-based practices that can be implemented system wide (Romano & Schnurr, 2020).

Fifth, key stakeholders evaluated the acceptability and feasibility (social validity) of the intervention goals, procedures, and outcomes. Exploring the perceived acceptability and feasibility of an intervention is important because these factors can influence successful implementation (Lindo & Elleman, 2010; Strain et al., 2012).

The purpose of this study was to examine the preliminary efficacy of providing EMT using a hybrid telepractice service delivery model. Effects of the intervention on caregivers' implementation of five EMT language support strategies, children's expressive communication skills, and the social validity of the procedures were examined across seven research questions. The first three research questions addressed caregiver implementation fidelity: (a) Is there a functional relation between a hybrid telepractice delivery of EMT and the accuracy of caregiver EMT strategy use? (b) Is there a functional relation between a hybrid telepractice delivery of EMT and the frequency of caregiver EMT strategy use? (c) Do caregivers continue to use each EMT strategy at levels consistent with the performance criterion several weeks after completion of the intervention? Three additional questions addressed child communication skills: (d) Are increases in caregiver use of EMT strategies associated with increases in the number of child communication acts? (e) Are increases in caregiver use of EMT strategies associated with increases in the weighted count of child communication acts? (f) Are increases in caregiver use of EMT strategies associated with increases in the child number of different words? The final research question addressed social validity: (g) Did stakeholders judge the intervention goals to be socially important, procedures socially acceptable, and outcomes socially significant?

Method

Participants

Children and caregivers were recruited through Oregon Health and Science University clinics and early intervention service districts in metropolitan Portland and Willamette Valley regions in Oregon. Research was approved by the Vanderbilt University Institutional Review Board (171282). Inclusion criteria for caregivers were (a) age ≥ 18 years, (b) relationship to the child participant as a parent or custodial guardian, and (c) ability to speak and understand English. Child inclusion criteria were (a) age between 18 and 36 months; (b) sufficient visual perceptual and processing skills to participate in EMT, defined as age equivalence scores of at least 18 months on the Visual Reception subscale of the Mullen Scales of Early Learning (Mullen, 1995); (b) an expressive–receptive language delay, defined as a total score ≥ 1.5 SDs below the mean on the Preschool Language Scales–Fifth Edition (PLS-5; Zimmerman et al., 2011); and (c) hearing and vision skills within normal limits (or corrected to be within normal limits), confirmed by medical record or parent report. Child exclusion criteria were (a) results indicating an increased likelihood of an autism spectrum disorder (ASD), defined as scores of 8 or higher on the Modified Checklist for Autism in Toddlers–Revised With Follow-Up (Robins et al., 2009); (b) medical or educational diagnosis of ASD, confirmed by medical record or caregiver report; (c) mild to severe hearing loss (e.g., 25–90 dB HL) confirmed by medical record or caregiver report; and (d) moderate to severe visual impairment (e.g., 20/70–20/200 visual acuity) confirmed by medical record or caregiver report. Children with a diagnosis or increased likelihood of ASD were excluded because children with ASD may present with behavioral and learning characteristics, which may be best addressed by naturalistic and developmental behavioral interventions designed specifically for children with ASD and their parents (Hampton, Kaiser, & Fuller, 2020; Schreibman et al., 2015).

Five caregiver–child dyads participated: (a) Jameson with his mother Jessica, (b) Ira with his mother Elena, (c) August with his mother Alyssa, (d) Ambyr with her grandmother Terry, and (e) Nora with her mother Sydney. One dyad (August and Alyssa) left the study after the second baseline session because Alyssa was returning to work full time and initiating speech-language therapy for August with a private provider. This dyad was excluded from the analyses.

Family demographic characteristics are shown in Table 1. Participating children ranged in age from 18 to 27 months (M = 23.3 months, SD = 3.5 months) at study onset. All children had language skills significantly below age expectations. Ira had a primary language impairment. The other three children had language impairments secondary to other diagnoses, confirmed by parent report and their medical history. Jameson and August had unspecified developmental delays, and Ambyr and Nora had Down syndrome. Child developmental skills are summarized in Table 2.

Table 1.

Child and family demographics.

Child and family demographics	Participants
Child	Jameson	Ira	Ambyr	Nora
Sex	Male	Male	Female	Female
Age in months a	25	19	22	18
Etiology or Diagnosis	Premature birth, extremely low birth weight, developmental delay	Language impairment	Trisomy 21	Trisomy 21
Race	White	White	White	White
Ethnicity	Not Hispanic or Latinx	Not Hispanic or Latinx	Not Hispanic or Latinx	Not Hispanic or Latinx
Primary language spoken at home	English	English	English	English
People living at home	4	5	3	4
Annual family income	$40,000–$50,000	Above $100,000	—	$40,000–$50,000
Income-to-needs ratio b	2.01	> 3.42	—	2.01
Distance from research center	32 miles	12 miles	89 miles	68 miles
County designation	Rural	Urban	Rural	Rural
Receiving early intervention (Part C) services	Yes	Yes	Yes	Yes
Receiving speech and language therapy from a speech-language pathologist	No	Yes	No	No
Caregiver	Jessica	Elena	Terry	Sydney
Relationship to child	Mother	Mother	Grandmother	Mother
Age in years a	33	38	63	31
Employment outside the home	None	None	None/retired	Part-time
Occupation	Educator	Caregiver	Caregiver	Athletic trainer
Education level	College degree	Graduate degree	College degree	College degree
Experience with mobile digital devices	Yes	Yes	Yes	Yes
Experience with video-teleconferencing	Yes	Yes	Yes	Yes
Reliable wireless Internet access at home	Yes	Yes	Yes	Yes

^a Age at study onset.

^b Estimated from 2017 Census Bureau Measures of Poverty.

Table 2.

Child developmental skills.

Measures	Jameson	Ira	Ambyr	Nora
PLS-5	M (95% CI)	M (95% CI)	M (95% CI)	M (95% CI)
Auditory ComprehensionSS	66 [62, 74]	81 [75, 89]	66 [62, 74]	73 [68, 81]
Expressive CommunicationSS	74 [68, 86]	72 [67, 81]	75 [70, 84]	66 [61, 76]
Total ScoreSS	69 [64, 77]	74 [69, 82]	69 [64, 77]	68 [64, 75]
Mullen Scales of Early Learning
Visual ReceptionT	40	54	39	38
Fine MotorT	32	64	20	44
Receptive LanguageT	20	36	26	26
Expressive LanguageT	36	27	30	33
Early Learning CompositeSS	67	91	61	72
MacArthur–Bates CDI Words and Gestures Vocabulary Checklist	(95% CI)	(95% CI)	(95% CI)	(95% CI)
Total Words Understood	58 [55, 61]	86 [83, 89]	213 [210, 216]	143 [140, 146]
Total Words Produced	22 [19, 25]	8 [5, 11]	39 [36, 42]	3 [0, 6]
Total Verbs Understood	10	9	38	21
Total Verbs Produced	0	0	4	0
Structured Language Sample
Rate (Communication Acts/Min)	0.97	1.10	0.46	0.62
NDW	7	0	0	3
M-CHAT R/F Score M-CHAT R/F Risk Description	6 Medium risk	0 Low	4 Medium risk	3 Medium risk
Child Behavioral Checklist
Internalizing BehaviorT	71	43	51	45
Externalizing BehaviorT	58	43	55	54
Total Problem BehaviorT	68	38	52	38
Structured Play Assessment
Most Frequent Play Level	Indiscriminate actions	General combinations	Discriminate actions	Indiscriminate actions
Highest Play Level	Take apart combinations	Specific combinations/conventional attributes	Take apart combinations	Take apart combinations

Note. For the Child Behavioral Checklist, T = T score, T scores of less than 60 are considered typical and are within age expectations. PLS-5 = Preschool Language Scales–Fifth Edition; CI = confidence interval; SS = standard score with a mean of 100 and a standard deviation of 15; For the Mullen, T scores have a mean of 50 and a standard deviation of 10; CDI = Communicative Development Inventories; M-CHAT R/F = Modified Checklist for Autism in Toddlers–Revised With Follow-Up; NDW = Number of different words.

The first author, a certified speech-language pathologist, was the therapist and conducted all evaluation, baseline, intervention, and maintenance sessions. She had 11 years of experience working with children with disabilities in a clinical setting, 8 years of experience practicing speech-language pathology, and 6 years of experience researching interventions for children with language impairments. She was trained on delivering caregiver-implemented EMT using the Teach–Model–Coach–Review approach at the Vanderbilt University Kid Talk Lab.

Setting

All sessions occurred in families' homes. During in-person sessions, the therapist provided instruction and coaching, seated with the caregiver and child in the family's living room or kitchen. During telepractice sessions, the therapist provided instruction and coaching from a private office.

Across dyads and conditions (baseline, intervention, maintenance) the therapist conducted approximately 40% of sessions in person (M = 39.37%, SD = 1.74%) and 60% of sessions via video-teleconferencing (M = 60.63%, SD = 1.74%). The ratio of in-person versus telepractice sessions (40/60) was selected a priori based on EMT treatment intensity discussions with content experts and on prior hybrid telepractice caregiver-implemented language intervention research. In McDuffie et al. (2013, 2016), all four parent education sessions and four intervention sessions (33% of sessions) were provided in person. In the current study, the intervention was planned to introduce each strategy in person and then a sequence of two telepractice sessions followed by one in-person session. However, families' situations (e.g., illness, vacation schedule) occasionally did not allow the delivery following this precise sequencing. If an in-person session was cancelled, the therapist followed the planned sequence to the greatest extent possible. When caregivers requested, in-person cancellations were made up as telepractice sessions to prevent prolonged therapy gaps. The overall percentage of in-person and telepractice sessions was consistent; only the sequencing of the two types of sessions changed. Cancellations did not impact in-person and telepractice sessions differentially.

Materials

Several technologies were used to collect and analyze data. A Sony Handycam HDR-CX580V camera and Sunpak 2001 UT tripod were used to video-record in-person sessions. Zoom Client for Meetings software was used to video-record telepractice sessions. The System of Analysis of Language Transcripts (Miller & Iglesias, 2012) was used to transcribe caregiver and child utterances and code the dependent variables. Study data were managed using Research Electronic Data Capture, a secure web-based application (Harris et al., 2009).

The therapist provided families with a brief treatment manual to support adherence to the intervention. It included a series of tip sheets, written in plain language to help caregivers use EMT strategies at home during daily activities. These tip sheets were from the existing EMT suite of materials (Roberts & Kaiser, 2015; Roberts et al., 2014).

The therapist used multiple technologies to provide EMT through telepractice including a Dell Computer Workstation with Internet access, Zoom Client for Meetings software, Kubi Connect software, Logitech webcam, and a Blue Snowflake portable high-fidelity USB microphone. Caregivers received an iPad Mini with the Zoom Mobile app installed and a Kubi telepresence robot. Caregivers used the iPad Mini and Zoom Mobile app to video-teleconference with the therapist. The Kubi robot is a remote-controlled iPad stand that can pan 300° and tilt 90°. It held the iPad Mini and was remotely controlled by the therapist during telepractice sessions. This reduced the need for caregivers to adjust the iPad Mini during telepractice sessions, which allowed the caregivers to focus on practicing EMT strategies.

Descriptive Measures

Families participated in an assessment battery to establish eligibility and describe the characteristics of participants in the study. Assessments included a demographic questionnaire, Mullen Scales of Early Learning (Mullen, 1995), MacArthur–Bates Communicative Development Inventories (MCDI): Words and Gestures (Fenson et al., 2007), PLS-5 (Zimmerman et al., 2011), a structured language sample, Child Behavior Checklist ages 1.5–5 years (Achenbach & Rescorla, 2000), and the Structured Play Assessment (Ungerer & Sigman, 1981). A detailed summary of assessment procedures is in Supplemental Material S1.1, pp. 2–4. Three of these assessments were also conducted 6 weeks post intervention: MCDI, PLS-5, and the structured language sample.

Response Definitions for Caregiver Behaviors

Caregiver outcomes were two measures of EMT implementation fidelity. The primary dependent variable was accuracy of caregiver EMT strategy use. It was measured by calculating the percentage of spoken turns in which the caregiver used each strategy correctly (correct turns / correct + incorrect turns × 100%). The secondary dependent variable was the frequency of caregiver EMT strategy use. It was measured by counting the number of spoken turns in which the caregiver used each EMT strategy correctly. Percentage and count metrics were used to capture the two important aspects of EMT implementation fidelity precision (accuracy) and dose (frequency) of the EMT strategies (Kaiser & Hampton, 2016).

During a 10-min observation of a caregiver–child interaction (described below), coders observed the caregiver's implementation of five EMT strategies (a) matched turns, (b) target talk, (c) expansions, (d) time delays, and (e) milieu teaching episodes. Implementation of each strategy was assessed independently. Operational definitions and performance criterion for each EMT strategies are summarized below.

Matched Turns

Matched turns were taught to encourage caregivers to adopt a responsive interaction style. A matched turn was defined as a responsive utterance contingent on the child's communicative act or play behavior that occurred immediately after the child's turn (≤ 2 s). For example, a matched turn occurred when a child handed the caregiver a toy car and the caregiver labeled the toy, “Car!” Caregiver turns were coded as incorrect or unmatched when they (a) followed a child turn after a pause of > 2 s, (b) were utterances spoken independent of child communication, or (c) were unrelated to the child's communication or content focus. The performance criterion during training was that ≥ 75% of caregiver responses correctly implemented matched turns.

Target Talk

Target talk was taught to encourage caregivers to model language at the child's target mean length of utterance. All children had a target utterance length of one content word because they expressed fewer than 20 different words during the language sample at study entry. Content words included nouns, verbs, prepositions, and requesting words (e.g., more, done).

Target talk was defined as a caregiver spoken turn containing one content word, or two to three content words when in response to a child verbal communication act. Child verbal communication acts contained at least one intelligible word or recognizable manual sign. Target talk primarily involved caregivers stating single content words such as, “blanket,” although caregivers were also taught to use linguistic expansions (described below) in response to a child verbal communication act. Therefore, when responding to a child verbal communication act, the caregiver could use target talk correctly in two ways: (a) saying a single content word or (b) saying two or three content words. For example, when responding to a child's statement, “cookie” the caregiver could respond correctly with a single word (e.g., “eat”), or with two to three content words (e.g., “Daddy is eating cookies”). Articles, pronouns, and verb tense were used to model grammatically correct statements. Caregiver turns were coded as incorrect or not target talk when they used statements that were above target (e.g., > 1 content word in response to a child vocalization or gesture), below target (e.g., animal sounds, sound effects), behavioral commands (e.g., Stop that!), or questions (e.g., Baby?). The performance criterion was ≥ 50% of caregiver turns correctly implemented target talk.

Expansions

Expansions were taught to encourage caregivers to provide grammatically and/or semantically complex models for the children. An expansion was defined as a caregiver spoken response that immediately followed a child verbal communication act (≤ 2 s) and included all or part of the child's communication act with an addition of one to two content words, or replaced a word in the child's communication act to make the statement grammatically correct. For example, an expansion occurred when a child said, “drink” and the caregiver responded, “The cat is drinking.” Expansions were well formed, short utterances, but not telegraphic. Articles, pronouns, and verb tense were included. Caregiver turns were coded as incorrect, or not expansions when the child initiated a verbal communication act and the caregiver (a) did not respond within 2 s, (b) responded without adding content words, (c) responded but added more than two content words, (d) responded but added other words (e.g., “wow”), or (e) expanded using a grammatically incorrect utterance (e.g., “Truck go!”). The performance criterion was that ≥ 40% of caregiver turns, which followed a child verbal communication, act correctly implemented expansions (Roberts et al., 2014).

Time Delays

Time delays were taught as a simple method for caregivers to induce child communication. A time delay was defined as a naturalistic, nonverbal procedure for eliciting child communication. It involved a pause and expectant look that indicated to the child that a communicative response was required. After the child communicated, the caregiver provided access to the object, performed the action of the child's interest, or prompted specific language using milieu teaching (Kaiser & Hampton, 2016).

Four time delay strategies were taught including nonverbal choice, pause in routine, assistance, and inadequate portions. Nonverbal choices involved the caregiver holding up two objects that a child might want and waiting with an expectant look until the child communicated. For example, the caregiver held up a toy spoon and blanket and waited for the child to select one of the toys. Pause-in-routines occurred after two or three predictable repetitions in a play routine when the caregiver paused with an expectant look until the child communicated. For example, the caregiver put three dolls on the bus, while modeling “on.” Then, for the fourth doll, the caregiver paused, holding the doll in the child's line of sight while waiting for the child to communicate. Opportunities for assistance involved providing a child with an activity that they needed assistance to gain access to or operate and waiting until the child communicated. For example, while playing with a toy pizza, the caregiver placed the remaining pepperoni on the plate, handed the child a closed container of pizza toppings, and waited for the child to communicate. Inadequate portions involved the caregiver presenting a small amount of something that the child wanted such as a goldfish cracker, and waiting for the child to communicate. Unlike milieu teaching episodes (described below), time delays did not result in a series of prompts to further elicit the child response. The goal was to reinforce child attempts to communicate while keeping the demand for communication natural but relatively low.

All time delay strategies were scored on a 3-point scale, with considerations for the caregiver's adherence to the strategy steps and the child's interest. The score represents the quality and accuracy of the time delay. For the purposes of this study, to be considered correct, time delays required a score of TD3, outstanding performance. Supplemental Material S1.2, p. 5, provides detailed scoring procedures for the time delay strategies. Time delays were executed correctly if the caregiver (a) started naturally so that it did not disrupt play; (b) waited for a clear request from the child before giving the object/action; (c) gave the requested object/action, at the end of the time delay; (d) labeled the action using a single content word within 1 s of giving it; and (e) for nonverbal choices, presented two distinctly different objects (e.g., apple and boy). Objects were considered distinctly different if they had a different label or function. Objects that differed by size or color were not distinctly different. If the caregiver missed any of these steps, the time delay was scored as incorrect. The performance criterion was that ≥ 80% of time delays were implemented correctly (followed all steps). It was expected that caregivers would use 1–10 correct time delays per session (Roberts et al., 2014).

Milieu Teaching Episodes

Milieu teaching episodes were taught to encourage caregivers to prompt specific communication targets during a highly motivating everyday communication context. A milieu teaching episode was defined as a prompting procedure that supports communication by either directing the child to make a response or asking the child a question. It used a least to most prompting procedure, which started with the least supportive prompt (e.g., expectant pause) and sequentially introduced more supportive prompts such as open questions (e.g., “What do you want?”), followed by model prompts (e.g., “Say open.”). Milieu teaching episodes used the child's interest in an object, action, or person, to teach specific communication targets. Two types of milieu teaching episodes were taught: model prompts (e.g., “Say open.”) and choice prompts (e.g., “Baby or blocks?”).

In the current study, caregivers applied milieu teaching when a child initiated a nonverbal request (e.g., vocalization, gesture, triadic eye gaze). All children had single-word response targets. When a child initiated a nonverbal request (e.g., pointed to a juice cup), the caregiver followed a model prompt or choice prompting sequence to guide practice communicating the target word (e.g., “juice”). Milieu teaching episodes required caregivers to prompt twice at each level of the prompting sequence if a child did not respond with the target word. For example, if a child requested a toy car by pointing to it, the caregiver would ask an open question (e.g., “What do you want?”). Unless the child responded correctly (e.g., “Car!”), the caregiver would continue the milieu episode by giving a model prompt (e.g., “Say car.”) or a choice prompt (e.g., “Car or boy?”). If the child responded with the target word (e.g., “Car!”), the caregiver would give the child the toy car and expand (e.g., “Let's drive the car”). However, if after two prompts at each level the child did not respond with the target word, the caregiver would model the target again (e.g., “car”) and hand the child the toy.

All milieu teaching episodes were scored on a 3-point scale, with considerations for the caregiver's adherence to the prompt sequence and the child's interest. For the purposes of this study, milieu teaching episodes were executed correctly if they were scored as an MT3: outstanding performance. Supplemental Material S1.2, p. 6, provides detailed scoring procedures. Milieu teaching episodes were considered correct when the adult (a) began the episode naturally and in response to a nonverbal child request, (b) prompted a language target using one content word, (c) prompted the same word throughout the episode, (d) used the correct prompting sequence, (e) gave the child adequate time to respond, (f) stopped prompting when the child lost interest or said the target word, (g) gave the child the requested object at the end of the episode, and (h) labeled the object/action with the prompted language target. If the adult missed any of these steps, the episode was coded as incorrect. The performance criterion was that ≥ 80% of milieu teaching episodes were implemented correctly (followed all steps). It was expected that caregivers would use one to five correct milieu teaching episodes per session (Roberts et al., 2014).

Response Definitions for Child Behaviors

All child communication skills were secondary dependent variables and considered exploratory because the study design did not allow experimental control for these behaviors (Ledford & Gast, 2018). Three behaviors were observed: number of child communication acts, weighted count of communication acts, and number of different words.

Number of Child Communication Acts

Each communication act was classified as a potentially communicative vocalization, conventional gesture, manually signed word, or spoken word. A potentially communicative vocalization was defined as a child vocalization that was a nonword or unintelligible utterance that was socially directed to the caregiver. Crying, laughing, and screaming were not considered potentially communicative vocalizations. A conventional gesture was defined as a purposeful nonverbal movement used to signal toward an object or event of interest. Gestures included reaches, headshakes for no and yes, pointing (proximal or distal), showing, and giving. Manual signing was coded when a child used a clearly defined manual sign with a consistent hand shape. Spoken words were defined as the child using the same sounds for the same referent consistently. Words did not have to be clearly articulated; however, to be counted as a word, the word had to be an acceptably close approximation to an adult pronunciation. Specifically, words needed to contain at least one accurate phoneme occurring in the correct position and have the same number of syllables as the adult pronunciation. Number of child communication acts were measured to capture the frequency of expressive communication.

Weighted Count of Communication Acts

A weighted count of communication acts was estimated using the Infant Toddler Growth Indicator: Early Communication Index procedures (Carta et al., 2010; Greenwood et al., 2010). To calculate a weighted count, the therapist multiplied the number of vocalizations and gestures by one, spoken and manually signed words by two, and multiple word combinations by three before summing the communication acts. Multiplying selected communication forms by a larger number effectively “gives more weight” to communication forms that are theoretically more complex or developmentally within age expectations (Yoder et al., 2018). Child communication acts were measured using a weighted count in addition to a frequency count because research on early social communication skills indicated that weighted counts may be more sensitive to change and predictive of later social communication skills in young children with language delays (Greenwood et al., 2010; Yoder et al., 2009).

Number of Different Words Expressed

The number of different words expressed was defined as a frequency count of the number of different word roots spoken or signed. The same word produced using different modes was counted as a single word root. Number of different words expressed was measured to capture lexical diversity.

Observational Measurement of Caregiver and Child Behaviors

All caregiver- and child-dependent variables were coded from video media files of a brief (10 min), repeated, observational measure of caregiver–child interaction. During each session, the therapist directed the caregiver to (a) gather toys the child would be interested in playing with, (b) indicate when she was ready to play, and (c) play with the child as she normally would for 10 min. Ten minutes were randomly selected from each session, when the caregiver and child played for longer than 10 min to ensure a consistent observation length. The therapist followed the same observational measurement procedures across all conditions (baseline, intervention, maintenance) and service delivery type (in-person, telepractice).

Coders used operational definitions and procedures from the EMT manual for coding a caregiver–child interaction (McCulla et al., 2019). This manual was developed for and used in previous EMT studies, and the reliability for child and adult variables was adequate (e.g., Peredo et al., 2018; Quinn et al., 2020; Roberts & Kaiser, 2015; Wright et al., 2013).

Design

A multiple baseline design across behaviors (i.e., EMT strategies) was used to examine the effect of a hybrid telepractice delivery of EMT on caregivers' accuracy and frequency of EMT strategy use. Within each multiple baseline design, there were four opportunities to demonstrate an intervention effect for the caregiver's accuracy and frequency of EMT strategy use. The multiple baseline design was replicated across four caregiver–child dyads. Thus, there were 16 potential demonstrations of effects. Each design had three conditions: baseline, intervention, and maintenance.

During the baseline condition, the therapist conducted at least five sessions; baseline continued until caregiver responses were stable. A stable baseline was defined as a series of at least four data points in which the two most recent data points did not indicate a therapeutic (accelerating) trend. Decisions about condition changes (e.g., moving from baseline to intervention) were made following structured visual analysis procedures to examine the accuracy of EMT strategy use within and across tiers (Ledford, Lane, & Severini, 2018).

Once the caregiver demonstrated a stable baseline, the intervention was applied to the EMT strategy in the first tier. No training was provided for the EMT strategies in the remaining three tiers. Caregiver instruction for subsequent EMT strategies was introduced after at least five intervention sessions were completed, and the caregiver met or exceeded the performance criterion during three of four consecutive sessions. For each dyad, concomitant changes in child number of different words, communication acts, and weighted count of communication acts were examined in the context of a simple time series design (A–B comparison).

Procedure

Training on Telepractice Equipment

After participants completed eligibility assessments and prior to initiating baseline sessions, the therapist taught the caregiver how to use the iPad Mini, Zoom Mobile app, and Kubi telepresence robot. The therapist also guided each family through a technology test run during which the caregiver practiced all of the steps to participate in video-teleconferencing.

Selection of Home Routines

Home routines were activities performed by the caregiver and child on a regular basis. They were selected by the caregiver and therapist prior to the baseline condition and varied over the course of the intervention based on child and caregiver interests. Two activities were practiced to provide opportunities for generalization. Caregivers were provided a handout describing categories of routines to inform their selections.

Structure of Sessions

To simulate a typical early intervention session and isolate the effects of EMT, the therapist implemented four activities during baseline and intervention sessions. Each session included (a) setting the stage (e.g., reviewing the child's developmental progress and family events), (b) instruction (e.g., providing caregiver education), (c) observation (e.g., watching a 10-min caregiver–child interaction and a 5-min routine), and (d) summarizing the session (e.g., reviewing the session and confirming the future schedule). The session structure was identical across conditions and service delivery (in-person, telehealth). Caregivers were informed of the session structure and the planned activities in each condition prior to starting sessions.

Baseline Sessions

Baseline sessions were conducted in person (43%) and via telepractice (57%). Sessions were conducted twice per week and ranged from 36 to 106 min (M = 58.8 min, SD = 17.3 min). First, the therapist set the stage by asking the caregiver about family events. Next, the therapist provided instruction by sharing information about developmental milestones for the following areas: social–emotional, gross and fine motor skills, early math, and visual spatial skills. Information presented to caregivers was adapted from the Centers for Disease Control Learn the Signs—Act Early public awareness campaign (Centers for Disease Control and Prevention, 2019). Although these developmental domains may overlap with early language skills, the therapist selected materials to highlight the nonverbal aspects of these developmental skills and eliminated explicit references to speech, language, and social communication skills. No instruction was provided regarding language development or the EMT strategies. After discussing developmental milestones, the therapist observed a 10-min caregiver–child interaction with toys and a 5-min home routine. No coaching or feedback was provided. Last, the therapist summarized the session and provided general, positive feedback to the caregiver (e.g., “It looked like he enjoyed you crashing that tower together”). Baseline sessions were assessed for procedural fidelity, described below, which included an assessment of whether instruction involved language strategies or skills, and whether feedback was general and not focused on language development.

Intervention Sessions

Intervention sessions were conducted in person (40%) and via telepractice (60%). Five EMT strategies were taught sequentially across four tiers, Tier 1: matched turns, Tier 2: target talk and expansions, Tier 3: time delays, and Tier 4: milieu teaching episodes. The primary difference between baseline and intervention sessions was that the therapist taught caregivers EMT strategies using the Teach–Model–Coach–Review approach during the session activities. Each EMT strategy was introduced during a caregiver education workshop (one per tier, four total), followed by twice weekly caregiver coaching sessions for 4 months.

Workshop sessions were an average of 85.7 min (SD = 15.01 min, range: 51–111 min) and were conducted in person. First, the therapist set the stage by reviewing the child developmental progress and family events. During workshop instruction, the therapist presented a slideshow that (a) defined the EMT strategy, (b) provided a rationale for the strategy, (c) explained strategy implementation using video exemplars, and (d) role played the strategy with the caregiver. During workshop modeling, the therapist played with the child for 10 min demonstrating the EMT strategy with the child. The therapist pointed out the target EMT strategy while using it with the child while the parent observed. Then, the therapist role played the strategy with the caregiver as guided practice. After teaching and modeling the EMT strategy, the therapist observed and coached the caregiver on how to use the target strategy during a 10-min caregiver–child interaction with toys and 5 min of home routines. Last, the therapist reviewed caregiver implementation of the EMT strategy during the session summary.

After completing the workshop, caregivers participated in twice weekly coaching sessions, which were conducted in person and through telepractice. Coaching sessions were an average of 52 min (SD = 10.96 min, range: 34–78 min). First, the therapist asked about family events, changes in child communication, and the caregiver's use of EMT strategies during the week to set the stage for the current session. Next, caregiver instruction involved reteaching the target strategy definitions and rationale. The therapist also presented graphed data of the caregiver's performance from the previous session and discussed performance goals for the current session. During coaching sessions, the therapist modeled the target EMT strategy by demonstrating the strategy, role playing with the caregiver, and presenting video examples and/or transcripts of the caregiver applying the strategy in the previous session. The therapist then led a discussion to plan for using the EMT strategy during toy play and home routines. After planning, the therapist observed and coached the caregiver while she practiced the target EMT strategy with the child during toy play for 10 min and a home routine for 5 min. Last, the therapist summarized the session by leading a reflection, encouraging the caregiver to review their target EMT strategy use, and the child's response to the therapist. While reflecting, the therapist and caregiver jointly problem solved and planned how to use the EMT strategies.

Coaching strategies applied during intervention sessions included six behaviors (a) reminding the caregiver to use a specific strategy or specific component of a strategy (e.g., mirror-and-map), (b) suggesting a specific target word or phrase to model (e.g., “blocks”), (c) suggesting an action to increase play or engagement (e.g., “roll the car on the ramp”), (d) recommending environmental arrangement strategies (e.g., “clear your space”), (e) providing performance feedback (e.g., “great job offering a choice but try again when he is looking right at you”), and (f) delivering general praise (e.g., “good idea”). Coaching statements were brief, positive in tone, and intended to support caregivers' EMT strategy use during the immediate interaction. Occasionally, the therapist made suggestions about positive behavior supports for the child's behavior if needed (“Let's clean up now [to prevent throwing]”). Therapists reminded the parent to use the EMT strategy at least 2 times during toy play (10 min) and once during the routine (5 min). Therapist provided specific performance feedback at least once per minute during the parent–child interaction (15 min).

Treatment intensity was consistent with prior studies of EMT (Roberts & Kaiser, 2015). Table 3 summarizes the treatment intensity parameters for therapists and caregivers. Average dose frequency was 1.67 sessions weekly. Number of intervention sessions ranged from 24 to 28 sessions (M = 25.5 sessions, SD = 1.91 sessions). Session duration ranged 34–111 min (M = 60.6 min, SD = 18.5 min), with workshops being longer because they included initial instruction and modeling. The variability in session length was also related to child care activities unrelated to the intervention that interrupted the planned session activities (e.g., toileting, attending to the needs of a target child's siblings). All but one participant (Ambyr) had siblings at home for at least some of the intervention sessions. Siblings frequently required the caregiver's attention during the sessions. Session time was calculated to include these intrusions in order to provide an accurate picture of the realities of early intervention service delivery. Cumulative intervention duration was an averaged 24.74 hr (SD = 2.87 hr).

Table 3.

Treatment intensity for therapist and caregiver behaviors.

Treatment parameter	Definition	Therapist parameter	Caregiver parameter M (range)
Dose	Number of properly administered teaching episodes during an intervention session	Clinician teaches the caregiver a target EMT strategy.	Caregiver uses the target EMT strategy with the child
Dose form	Typical task or activity in which the teaching episodes are provided	Semistructured home visit following Teach–Model–Coach–Review	Caregiver applies the EMT strategies during 10 min of toy play and a 5-min routine
Dose frequency	Number of times a dose of intervention is provided per session	• Teach: 1 teaching episode a • Model:1 teaching episode a • Coach: 18 teaching episodes a : 3 reminders of the target strategy and performance feedback 1× per min during a 15-min interaction • Review: 1 teaching episode a	• Matched Turns b : 59.20 (41–88) b • Target Talk b : 45.24 (8–70) b • Expansions b : 5.83 (0–28) b • Time Delays b : 3.75 (0–8) b • Milieu Teaching Episode: 3.05 (0–10)
Total number of doses	Number of doses across all targets per session	21 teaching episodes a	Dependent on intervention tier c • Tier 1: 53.31 (47.00–57.56) c • Tier 2: 105.13 (93.25–122.17) c • Tier 3: 118.95 (111.60–123.00) c • Tier 4: 131.41 (112.13–143.86) c
Number of targets	Number of different targets that will be addressed in each session	One EMT strategy in Tiers 1, 3, 4, and two EMT strategies in Tier 2
Session frequency	Weekly schedule of sessions	1.67 Sessions per week (1–2 sessions per week)
Session duration	Duration of an individual session	60.6 min (34–111 min)
Total intervention duration	Total number of sessions and weeks	25.25 Sessions (24–28 sessions) for an average of 15 weeks
Cumulative treatment intensity	Total number of doses × total intervention duration	21 × 25.5 sessions = 535.5 a	Tier 1: (53.31 × 7 sessions)b,c + Tier 2: 105.13 × 5.75 sessions) b , c + Tier 3: (118.95 × 5.75 sessions) b , c + Tier 4: (131.41 × 6.75 sessions) b , c = 2548.65 b,c

Note. This table is based on definitions introduced in Warren et al. (2007) and informed by the work of Alt et al. (2020). EMT = enhanced milieu teaching.

^a These are the minimum number of teaching episodes required based on procedural fidelity guidelines.

^b Dose frequency for caregiver EMT strategy use is averaged across participations and expected dose frequency differs by EMT strategy. Figures 1–4 depict number of EMT strategies used per session.

^c Expected total number of doses for caregivers differs by session; later sessions have larger total dosage because parents demonstrate proficiency with all five EMT strategies. Treatment parameters are consistent across in-person and telepractice sessions.

Maintenance Sessions

Maintenance sessions began 2 weeks following the completion of the intervention and continued until 6 weeks post-intervention for all participants. Procedures for maintenance sessions were similar to those used in baseline sessions and included three elements: (a) setting the stage, (b) observation, and (c) session summary. First, the therapist asked the caregiver about changes that had occurred, updates on recent successes, current challenges, and new concerns. Next, the therapist observed a caregiver–child interaction with toys for 10 min and a home routine for 5 min. Last, the therapist summarized the session and provided general, positive feedback to the caregiver. No explicit teaching, coaching, or performance feedback on EMT strategies was provided during maintenance sessions.

Post-Intervention Assessment

The therapist administered a post-intervention assessment battery 6 weeks post-intervention. A structured language sample, PLS-5, and MCDI were completed to describe changes in child communication following the intervention and maintenance period.

Interobserver Agreement for Dependent Variables and Procedural Fidelity

A senior research assistant not involved in the conduct of the current study supervised the initial coder training. Prior to coding data, the two coders were trained to 90% point-by-point interobserver agreement on transcription and coding for all dependent variables on three consecutive video media files. One graduate research assistant, a master's degree student in special education, acted as the primary coder and coded all session data. Two graduate research assistants and the first author acted as secondary coders. A randomly selected subsample of 33% of all baseline, intervention, and maintenance sessions across all tiers was selected for each participant to examine the reliability of the dependent variables. The primary coder was unaware of which coded sessions were randomly selected and assessed for interobserver agreement.

First, the primary coder independently transcribed then coded sessions using the System of Analysis of Language Transcripts software. Second, the reliability coder independently transcribed and coded the selected sessions and then compared the two coded samples and calculated point-by-point interobserver agreement for each of the dependent variables. This two-step process ensured that unitizing and classifying errors were included within the calculation of interobserver agreement at the dependent variable level. In the few instances where interobserver agreement for the dependent variables fell below the 80% criterion, the first author met with the primary coder and (a) reviewed and refined operational definitions, (b) specified the coding manual by adding clarifications or examples if needed, and (c) completed consensus coding to minimize observer drift. Consensus coding was only used for instructional purposes. For all caregivers, the average interobserver agreement was above 90% for matched turns (range: 83%–98%), 88% for target talk (range: 70%–98%), 91% for expansions (range: 81%–98%), and 90% for time delays and milieu teaching episodes (range: 50%–100%).Supplemental Material S1.3, p. 7, includes the average interobserver agreement across tiers, conditions, and dyads. For all children, the average interobserver agreement was above 88% for child communication acts (range: 78%–100%) and 94% for number of different words (range: 50%–100%). Interobserver agreement was less than 80% on occasions when the base number of occurrences was low (less than five), or when coders segmented caregiver statements differently. Supplemental Material S1.4, p. 8, also presents the average interobserver agreement for child communication outcomes across conditions and children.

The first author supervised the coder training for measuring procedural fidelity. Prior to assessing procedural fidelity, two coders were trained to 90% point-by-point interobserver agreement on three consecutive video media files. Research assistants, graduate students in master's degree programs in special education and child studies, acted as the primary and secondary coders. Each coder independently assessed the sessions after watching video media files. The coders used an observational checklist (direct systematic observational recording alongside a procedural fidelity checklist) to assess adherence to study procedures during baseline, intervention, and maintenance sessions. Procedural fidelity was measured during at least 33% of sessions across conditions, tiers, and participants. The first author randomly selected a subsample (20% of sessions) to examine the reliability of the procedural fidelity assessment. Both coders were unaware of which sessions were assessed for interobserver agreement.

Fidelity data were analyzed separately for each participant, condition, and tier and are summarized in Supplemental Material S1.5, p. 9. Results of the procedural fidelity assessment indicated that, across participants, the study procedures were implemented with an average of 98% accuracy for baseline sessions (range: 77%–100%), 97% accuracy for intervention sessions (range: 77%–100%), and 98% accuracy for maintenance sessions (range: 68%–100%). Maintenance session fidelity was very low (68%) in one session in which the caregiver and therapist decided to discontinue the session because the child was too fatigued to continue. Average interobserver agreement for procedural fidelity data was 99% for all participants (range: 91%–100%).

Social Validity

Social validity was assessed using two approaches, blind observers and participant questionnaires. Twelve raters, blind to study purpose, watched six randomly selected video clips (45 s) of the telepractice and in-person sessions. They answered nine questions about the feasibility of procedures, effects on parent and child behaviors, and acceptability of the intervention. Two questions inquired about the resources needed to implement the intervention. The raters were graduate students in a masters' level speech-language pathology program, with an average of 6.7 (SD = 4.2) years of experience with young children. The four participating caregivers also completed a questionnaire to describe the perceived feasibility and acceptability of the procedures. The questionnaire included eight questions: two addressed goals, three addressed procedures, and three addressed effects of the intervention. Responses to each question were scored on a 1–5 Likert scale, in which 1 = strongly disagree and 5 = strongly agree.

Visual Analysis and Statistical Analysis

Caregiver Outcomes

Structured visual analysis was the primary analytical approach used to examine the effects of a hybrid telepractice delivery of EMT on the accuracy and frequency of EMT strategy use (Ledford, Lane, & Severini, 2018). Caregiver performance of each EMT strategy was evaluated through visual analysis with an expectation of an immediate increase in level and an accelerating (therapeutic) trend to criterion. A functional relation (intervention effect) was determined by (a) examining the number of data points, level, trend, stability, and overlap of data (Ledford & Gast, 2018); (b) reviewing the data to determine whether the accuracy and frequency of caregiver EMT strategy use increased when and only when the intervention was implemented; and (c) evaluating the patterns of change to determine if the pattern was consistent across EMT strategies and caregiver–child dyads.

Secondary statistical analyses were conducted to quantify the differences between conditions and corroborate the results of visual analysis. These analyses were conducted after completing structured visual analysis. A nonoverlap metric and a parametric effect size were selected to quantify the results because there is not yet consensus on the best approach to calculating effect sizes in single-case design research and both approaches have strengths and limitations (Pustejovsky, 2019; Pustejovsky & Ferron, 2017). Nonoverlap metrics and parametric effect sizes were calculated in R studio using the SingleCaseES R package (Pustejovsky & Swan, 2018).

The nonoverlap of all pairs (NAP; Parker & Vannest, 2009) was selected as the nonoverlap metric because of all overlap-based methods, NAP is least sensitive to variability in procedural factors such as baseline and intervention condition length, which is particularly relevant in multiple baseline designs (Pustejovsky, 2019). NAP quantifies nonoverlapping data by evaluating the extent to which all possible pairs of data overlap in adjacent conditions. Other nonoverlap indices were considered (e.g., Tau-U, Parker et al., 2011) but not selected because they have undesirable measurement characteristics. Specifically, their average magnitude is influenced by procedural factors including the sample size, number of observations, observation length, and condition length (Pustejovsky, 2019; Tarlow, 2017). Since NAP values can be overestimated when the data include baseline trend, we used the more conservative criteria, derived for multiple baseline designs (Petersen-Brown et al., 2012).

The natural log of the ratio of the conditional means or log response ratio increasing (LRRi) was calculated to provide an estimate of the magnitude of effects (Pustejovsky, 2015). LRRi was selected because it is robust to procedural variabilities impacting other metrics. It has several benefits, it is appropriate for behaviors measured as counts or proportions, it is robust to differences in observational measurement systems, it is not influenced by session or condition length, and it can be converted to a percentage change in the level of the outcome from baseline to treatment condition (Pustejovsky, 2019). Nonoverlap methods, including NAP, do not yield measures that characterize the magnitude of effects. However, LRRi provides an estimate of the percentage of change that approximates a measure of magnitude (Pustejovsky & Ferron, 2017).

Child Outcomes

A structured visual analysis protocol was used to examine the effects of caregiver strategy use on the number of child communication acts, weighted count of child communication acts, and number of different words (Ledford, Lane, & Severini, 2018). Increases in child communication behaviors were evaluated with visual analysis with an expectation of a latent increase in level and/or a gradual, accelerating (therapeutic) trend within a month of the caregiver meeting the learning criterion for matched turns. Statistical analysis was not conducted because there was not sufficient experimental control for the child dependent variables planned in the design.

Costs of the Intervention

Expenses for materials, personnel, and travel were calculated to establish the costs of implementing the hybrid telepractice model. Estimating the cost of the intervention provides critical information for educational service districts and health systems that may wish to project the costs of bringing such programs to scale (Hollands & Levin, 2017). Material expenses included assessments, evaluation manipulatives, telepractice equipment (iPads, Kubi), and video-teleconferencing software (Zoom Client for Meetings and Zoom Mobile app). Personnel expenses ($36.00 an hour) included the therapist's time for telepractice sessions, in-person sessions, and travel to and from home visits. Travel expenses included mileage reimbursement for travel to families' homes (0.53 per mile). The expected costs of delivering the intervention to each family entirely in person were also calculated in order to compare the costs of a hybrid telepractice delivery of EMT to the costs of providing in-person EMT.

Results

(RQ1) Is There a Functional Relation Between a Hybrid Telepractice Delivery of EMT and the Accuracy of Caregiver EMT Strategy Use?

The left side of Figures 1 –4 (Graph A) show the accuracy of caregiver EMT strategy use as measured by the percentage of turns in which caregivers used each EMT strategy correctly for each dyad. For all four dyads, data in the baseline condition were stable, at a consistent low or moderate level for four EMT strategies: matched turns, expansions, time delays, and milieu teaching episodes. After the therapist delivered the intervention for these four strategies, there was a consistent and immediate increase in level for all dyads. Across dyads, there was some within-condition variability across EMT strategies but this was not unexpected, and the amount of variability did not impede determinations regarding level changes. During the intervention condition, all dyads met the performance criterion for four EMT strategies: matched turns, expansions, time delays, and milieu teaching episodes. For all dyads, there was some covariation between Tier 1 (matched turns) and Tier 2 (target talk) shown by an accelerating trend during baseline. However, this was expected and typical of the effects of training the individual components of EMT in multiple baseline designs (Hampton, Kaiser, Nietfeld, & Khachoyan, 2020).

Figure 1.

Jessica and Jameson's number and percentage of correct EMT strategies. Closed circles represent caregiver responses during in-person sessions, and open circles represent responses during telepractice sessions. Line Graph B includes a double axis graph; the gray bars depict caregiver responses and correspond with the axis on the left side (scale = 0–30). EMT = enhanced milieu teaching; NAP = nonoverlap of all pairs; LRRi = Log response ratio increasing.

Figure 2.

Elena and Ira's number and percentage of correct EMT strategies. Closed circles represent caregiver responses during in-person sessions, and open circles represent responses during telepractice sessions. Line Graph B includes a double axis graph; the gray bars depict caregiver responses and correspond with the axis on the left side (scale = 0–30). EMT = enhanced milieu teaching; NAP = nonoverlap of all pairs; LRRi = Log response ratio increasing.

Figure 3.

Terry and Ambyr's number and percentage of correct EMT strategies. Closed circles represent caregiver responses during in-person sessions, and open circles represent responses during telepractice sessions. Line Graph B includes a double axis graph; the gray bars depict caregiver responses and correspond with the axis on the left side (scale = 0–30). EMT = enhanced milieu teaching; NAP = nonoverlap of all pairs; LRRi = Log response ratio increasing.

Figure 4.

Sydney and Nora's number and percentage of correct EMT strategies. Closed circles represent caregiver responses during in-person sessions, and open circles represent responses during telepractice sessions. Line Graph B includes a double axis graph; the gray bars depict caregiver responses and correspond with the axis on the left side (scale = 0–30). EMT = enhanced milieu teaching; NAP = nonoverlap of all pairs; LRRi = Log response ratio increasing.

Structured visual analysis indicated there were 16 actual demonstrations of effect out of 16 potential demonstrations of effect. The consistency of behavior changes across conditions, and replication across tiers, suggests a functional relation between the introduction of the intervention and the accuracy of EMT strategy use for all four dyads. A detailed summary of the structured visual analysis for each dyad is provided in Supplemental Materials S1.6, pp. 10–13, and S1.7, pp. 14–21.

NAP and LRRi estimates were calculated to complement the results of structured visual analysis, depicted in text boxes alongside Graph A (see Figures 1–4). Supplemental Material S1.8, p. 22, includes estimates with standard errors, 95% confidence intervals, and size descriptions. According to Petersen-Brown et al. (2012), NAP estimates ranged from small to large. All participants had the smallest NAP estimates for the target talk strategy (range: 0. 55–0.93). LRRi estimates were transformed to percentage change; these estimates varied across participants: Jessica/Jameson (range: 120%–2,141%), Elena/Ira (range: 229%–999%), Terry/Ambyr (range: 41%–352%), and Sydney/Nora (range: 36%–77%). LRRi estimates were not possible to calculate for some EMT strategies (time delays, milieu teaching episodes) because use of these strategies was consistently at zero during the baseline condition. LRRi estimates do not perform well when baseline values are at or near zero.

(RQ2) Is There a Functional Relation Between a Hybrid Telepractice Delivery of EMT and the Frequency of Caregiver EMT Strategy Use?

The right side of Figures 1 –4 (Graph B) shows the frequency of caregiver EMT strategy use as measured by the number of turns in which caregivers used each strategy correctly, for each dyad. For three strategies: expansions, time delays, and milieu teaching episodes, there is a second axis on the right of Graph B, which scales the number of EMT strategies from 0 to 30. A double axis was included to help visualize changes in level of number of EMT strategies used for low-rate behaviors (Ledford et al., 2019). This was important because expansions, time delays, and milieu teaching episodes were expected to occur at a much lower frequency than target talk and matched turns, since they were contingent on infrequent child behaviors. Expansions were in response to child spoken or signed words, and time delays and milieu teaching episodes were in response to nonverbal requests. As mentioned above, time delays and milieu teaching were expected to occur between 1–10 and 1–5 times during the caregiver–child interaction, respectively (Roberts et al., 2014).

Two dyads (Jessica/Jameson and Sydney/Nora) had stable data at zero or very low levels during the baseline condition followed by an immediate increase in level after the intervention was applied for three strategies (expansions, time delays, and milieu teaching episodes). These two dyads had overlapping data and a latent increase in level for two strategies: matched turns and target talk, which impacted determinations of behavior change for those strategies. During the baseline condition, one dyad (Ira/Elena) had stable, low-level data followed by an immediate increase in level after the intervention was applied for four EMT strategies (matched turns, expansions time delays, and milieu teaching episodes). One dyad (Terry/Ambyr) showed low and stable data during the baseline condition followed by an immediate increase in level after the intervention was applied for three EMT strategies (matched turns, time delays, and milieu teaching episodes). However, Terry did not show a clear increase in level for target talk or expansions due to the amount of overlap between conditions. As described above, all four dyads had some covariation between Tier 1 (matched turns) and Tier 2 (target talk) but this was not unexpected.

Visual analysis indicated there were 13 demonstrations of effect out of 16 potential demonstrations of effect. Although the level changes were small, they were immediate and followed a consistent pattern across tiers. Additionally, the frequencies of EMT strategy use were within the expected ranges based on prior research (Roberts et al., 2014). Findings from visual analysis for Terry/Ambyr suggested that there were strong effects from implementing the intervention procedures for three EMT strategies, but there was not sufficient evidence of a functional relation due to the amount of overlap between the baseline and intervention conditions during the first three sessions in Tier 2 (target talk and expansions). The consistency of behavior changes across conditions, and replication across at least three consecutive tiers suggested a functional relation between the intervention and the frequency of EMT strategy use for three dyads: Jessica/Jameson, Elena/Ira, and Sydney/Nora. A detailed summary of the structured visual analysis for each dyad is provided in Supplemental Materials S1.6, pp. 12–13, and S1.7, pp. 14–21.

NAP and LRRi estimates were calculated to complement the results of structured visual analysis, depicted in text boxes alongside Graph B (see Figures 1 –4). Supplemental Materials S1.9, p. 23, includes estimates with standard errors, 95% confidence intervals, and size descriptions. All NAP estimates were significant and ranged from small to large according to Petersen-Brown et al. (2012) guidelines. LRRi estimates were transformed to percentage change; these estimates varied across participants: Jessica/Jameson (range: 60%–16,914%), Elena/Ira (range: 92%–19,676%), Terry/Ambyr (range: 91%–1,337%), and Sydney/Nora (range: 36%–10,551%).

(RQ3) Do Caregivers Continue to Use Each EMT Strategy at Levels Consistent With the Performance Criterion Several Weeks After Completion of the Intervention?

Caregiver performance levels were judged consistent if they were at or above 70% for matched turns, 30% for expansions, 40% for target talk, and 70% for time delays and milieu teaching episodes. During the maintenance condition, all four caregivers continued to use all five EMT strategies at or above the performance criterion for the majority (> 50%) of maintenance sessions (see Figures 1 –4).

(RQ4) Are Increases in Caregiver Use of EMT Strategies Associated With Increases in the Number of Child Communication Acts?

Caregiver strategy use was judged to be associated with increases in the number of child communication acts if the child's data showed a latent increase in level and/or an accelerating, therapeutic trend within a month of the caregiver meeting the learning criterion for matched turns. Figures in Supplemental Material S1.10, pp. 24–27, depict graphs of the number of child communication acts across participants. Caregiver use of EMT strategies was associated with increases in number of child communication acts for one dyad (Elena/Ira). Average performance across baseline, intervention, and maintenance conditions for all four dyads is summarized in Table 4. Three children's (Jameson, Ira, and Ambyr) average number of communication acts increased during the intervention and maintenance conditions compared to the baseline condition but without sufficient evidence to demonstrate that changes were related to the intervention.

Table 4.

Child communication outcomes.

Participant	No. of communication acts			Weighted communication acts			Number of different words
	Baseline M (SD)	Intervention M (SD)	Maintenance M (SD)	Baseline M (SD)	Intervention M (SD)	Maintenance M (SD)	Baseline M (SD)	Intervention M (SD)	Maintenance M (SD)
Jameson	18.96 (8.47)	22.59 (9.79)	27.29 (10.30)	56.37 (23.51)	66.62 (26.13)	69.83 (31.03)	8.24 (2.55)	10.30 (3.50)	13.71 (8.53)
Ira	16.86 (7.91)	29.95 (13.03)	44.60 (22.43)	49.95 (17.06)	84.89 (26.89)	123.80 (43.03)	1.49 (1.43)	6.65 (2.71)	10.00 (4.00)
Ambyr	9.30 (6.86)	18.62 (6.43)	21.60 (13.45)	42.51 (14.97)	56.50 (13.75)	79.80 (22.38)	2.22 (1.90)	5.59 (2.03)	9.25 (3.46)
Nora	10.27 (6.31)	10.25 (3.72)	12.00 (5.76)	52.34 (16.68)	29.65 (9.92)	36.83 (15.17)	1.75 (1.38)	2.43 (1.45)	3.67 (2.87)

(RQ5) Are Increases in Caregiver Use of EMT Strategies Associated With Increases in the Weighted Count of Child Communication Acts?

Caregiver strategy use was judged to be associated with increases in the weighted count of child communication acts if the child's data showed a latent increase in level and/or an accelerating, therapeutic trend within a month of the caregiver meeting the learning criterion for matchedturns. Figures in Supplemental Material S1.10, pp. 24–27, depict graphs of the weighted count of communication acts across participants. Caregiver use of EMT strategies was associated with increases in the weighted count of child communication acts for one dyad (Elena/Ira). Average performance across baseline, intervention, and maintenance conditions for all four dyads is summarized in Table 3. Three children's (Jameson, Ira, and Ambyr) average weighted count of communication acts increased during the intervention and maintenance conditions compared to the baseline condition but without sufficient evidence to demonstrate that changes were related to the intervention.

(RQ6) Are Increases in Caregiver Use of EMT Strategies Associated With Increases in the Child Number of Different Words?

Caregiver strategy use was judged to be associated with increases in the child's number of different words if the child's data showed a latent increase in level and/or an accelerating, trend within a month of the caregiver meeting the performance criterion for matched turns. Visual analysis indicated that caregiver use of EMT strategies was associated with increases in the number of different words expressed during caregiver–child interactions for three children (Jameson, Ira, and Ambyr; see Supplemental Material S1.10, pp. 24–27). Average performance across baseline, intervention, and maintenance conditions for all four dyads is summarized in Table 3. All four children's number of different words increased during the intervention and maintenance conditions compared to the baseline condition but without sufficient evidence to demonstrate that changes were related to the intervention.

(RQ7) Did Stakeholders Judge the Intervention Goals to Be Socially Important, Procedures Socially Acceptable, and Outcomes Socially Significant?

Findings from the SLP graduate student raters naïve to the purposes of the study indicated that they found the intervention goals and procedures feasible and acceptable. They also indicated that hybrid telepractice delivery of EMT was effective. As shown in Table 5, the graduate students strongly agreed that (a) the intervention was implemented similarly during in-person and telepractice sessions, (b) caregivers appeared similarly engaged when the therapist was training via videoconferencing or in person, and (c) the intervention was effective at increasing language use and communication skills in young children.

Table 5.

Social validity assessment items and summary ratings by blind speech-language pathology graduate student raters.

Questions	Session rating (1–5)
	M (range) a	Description
1. The teletherapy intervention was implemented very similarly to the in-person intervention.	4.67 (4–5)	Strongly agree
2. The intervention appears to be just as effective when completed through teletherapy or in person.	4.00 (3–5)	Agree
3. The children were more engaged in learning and playing when the therapist was on video and only the caregiver was present at home.	2.5 (1–3)	Disagree
4. The caregivers appeared just as engaged when the therapist was on video or in person.	4.67 (4–5)	Agree
5. The intervention is effective at increasing fine motor skills in children. b	3.5 (3–5)	Neutral
6. The intervention promotes bonding between the child and their caregiver.	5.00	Strongly agree
7. The intervention is effective at increasing language use and communication skills in children.	4.67 (4–5)	Strongly agree
8. The intervention will likely work with children with a variety of diagnoses.	4.5 (3–5)	Agree
9. The coaching provided to families could be conducted by a speech-language pathologist or developmental therapist working in an EI program.	4.5 (3–5)	Agree
Responses to open-ended questions
What resources, skills, training, or materials would an early interventionist need to implement a hybrid in-person, teletherapy intervention?  • Webcam, computer  • Training specifically how to work with parents and expertise in early intervention  • Coaching sample scripts illustrating how and when to provide feedback to parents  • A few practice sessions with volunteers, overseen by an expert SLP  • Training on how to adapt to various cultures and lifestyles
What resources, training, skills, or materials would caregivers need to deliver and participate in a hybrid in-person, teletherapy intervention?  • Access to technology, high-quality Internet service  • Training regarding therapy approaches  • Understanding of SLP vocabulary (e.g., expansions)  • Variety of toys  • Reduced environmental distractions  • Caregivers should have some practice in person prior to the start of teletherapy sessions.

Note. SLP = speech-language pathologist.

^a Based on 12 responses from students in a speech-language pathology master's degree program.

^b Behavior not a target in the intervention.

Results from the social validity questionnaire showed that all caregivers strongly agreed that the intervention goals made sense for the child, were important for the child's development, and were meaningful to their family (M = 5.00). Regarding the intervention procedures, all caregivers agreed or strongly agreed that the therapist's suggestions were easy to follow, they were able to use the EMT strategies during typical activities and routines, and the duration of the sessions was acceptable for the family (M = 4.50, SD = 0.57). Regarding the intervention goals, all four caregivers strongly agreed that the EMT sessions were effective in helping support the child's language learning and effective in improving the child's communication skills and that they would recommend the program to other parents (M = 5.00). Additional details regarding caregivers' assessment of social validity including mean score for each question are available in Supplemental Material S1.11, pp. 28.

Post-Intervention Assessment

After the intervention and maintenance conditions, the therapist administered the PLS-5, MCDI, and a structured language sample 6 weeks post-intervention to describe the children's language and communication skills. Table 6 shows the outcomes for each assessment for each child. Relative to their performance at study entry, all children showed increases in the number of different words as measured by the MCDI and communication rate as measured by the language sample. Two children (Ira and Ambyr) had observed scores with confidence intervals greater than their performance at study entry on the PLS-5.

Table 6.

Post-intervention assessment results and change from pretest.

Measures	Jameson		Ira		Ambyr		Nora
	Post-intervention	Change from pretest	Post-intervention	Change from pretest	Post-intervention	Change from pretest	Post-intervention	Change from pretest
PLS-5	M (95% CI)		M (95% CI)		M (95% CI)		M (95% CI)
Auditory ComprehensionSS	57 [54, 66]	−10	115 [107, 121]	+34	82 [77, 89]	+16	73 [68, 81]	0
Expressive CommunicationSS	75 [71, 83]	+1	94 [86, 103]	+22	88 [81, 98]	+12	85 [78, 95]	+19
Total ScoreSS	64 [60, 72]	−5	105 [99, 111]	+31	84 [79, 91]	+15	78 [73, 85]	+10
MCDI: Words and Gestures	(95% CI)		(95% CI)		(95% CI)		(95% CI)
Total Words Understood	186 [183, 189]	+128	355 [352, 358]	+269	255 [252, 258]	+42	258 [255, 261]	+115
Total Words Produced	52 [49, 55]	+30	51 [48, 54]	+43	105 [102, 108]	+66	61 [58, 64]	+58
Total Verbs Understood	34	+24	54	+45	42	+4	35	+14
Total Verbs Produced	2	+2	1	+1	18	+14	6	+6
Structured Language Sample
Rate (Communication Acts/Min)	1.75	+0.78	3.04	+1.94	2.67	+2.21	1.04	+0.42
NDW	25	+18	27	+27	17	+17	8	+5

Note. PLS-5 test–retest reliability ranges from r = .86 to .95. PLS-5 standard errors of measurement are reported in standard score units and range from 3.0 to 3.4 for the total language score, 3.7–5.0 for the Expressive Communication subscale, and 4.7–4.5 for the Auditory Comprehension subscale. MCDI test–retest reliability was r > .80 for vocabulary comprehension and productive vocabulary (Fenson et al., 2007). PLS-5 = Preschool Language Scales–Fifth Edition; CI = confidence interval; SS = standard score with a mean of 100 and a standard deviation of 15; MCDI = MacArthur–Bates Communicative Development Inventories; NDW = number of different words.

Intervention Costs

The total cost of the intervention across all four participants was $2,0916.34. Mean cost per participant was $5,229.085. The projected total cost to deliver EMT entirely in person across participants was $2,9595.60, which is $8,679.26 greater than the cost of the hybrid telepractice delivery of EMT. The majority of the cost savings, in the hybrid telepractice delivery of EMT, were accounted for by the reduction in therapist time spent traveling to and from home visits and mileage. Supplemental Materials S1.12, pp. 29, S.13, pp. 30, includes an itemized budget for each EMT delivery approach.

Discussion

Results from this study demonstrated that a hybrid telehealth delivery model of EMT increased caregivers' use of four EMT strategies: matched turns, expansions, time delays, and milieu teaching episodes. Visual analysis suggested a functional relation between the hybrid telepractice delivery of EMT and the accuracy of caregiver EMT strategy use for all four dyads and the frequency of caregiver strategy use for three dyads. Nonoverlap (NAP) and parametric (LRRi) effect size estimates indicated that the change from the baseline to the intervention condition was statistically significant for accuracy and frequency of caregiver EMT strategy use. Caregivers continued to use EMT strategies with fidelity for 6 weeks after the intervention was completed.

Increases in child communication during caregiver–child interactions were modest and variable across children. Visual analysis revealed one child demonstrated an increase in number of communication acts and weighted count of communication acts, and three children demonstrated an increase in number of different words associated with their caregiver's use of EMT strategies. However, the study design did not permit assessment of a functional relation between caregiver training and child outcomes.

Social validity was assessed using multiple methods (naïve raters, questionnaire). Assessments were conducted with stakeholders who represented the end users of the intervention, speech-language pathologists and caregivers of children with language impairments. Findings from the social validity assessments indicated preservice speech-language pathologists and the participating caregivers judged the interventions goals, procedures, and outcomes to be clinically important for children with language impairments.

Costs of the intervention were calculated to inform future studies and wide-scale adoption of EMT in typical practice settings. Not surprisingly, the cost savings for the intervention were related to personnel time spent traveling to family homes and mileage reimbursement for home visits. The farther families lived from the research center, the greater the cost savings associated with delivering the intervention via hybrid telepractice. Since the families who lived the farthest were located in rural communities, the cost analysis may be of particular importance to rural families and early intervention providers. Children in rural communities have significantly higher rates of developmental disabilities compared to children in urban areas (Zablotsky et al., 2019). They are also significantly less likely to have received developmental and behavioral health services like speech-language therapy, early intervention, and special education than their urban peers (Zablotsky et al., 2019).

Results from the current study replicate and expand previous investigations of caregiver-implemented EMT that utilized the Teach–Model–Coach–Review approach in two key ways (Peredo et al., 2018; Roberts & Kaiser, 2015; Roberts et al., 2014; Wright & Kaiser, 2017). First, this study adapted the EMT in-person procedures for a hybrid telepractice delivery. This study provided an initial demonstration of the preliminary efficacy of a hybrid telepractice delivery of EMT. Even though the current study does not provide sufficient experimental control to answer questions about the relative efficacy of a hybrid telepractice delivery compared to in-person delivery, it does provide initial positive evidence for a hybrid telepractice delivery of EMT. As shown in graphs of the session data, caregivers used strategies with similar accuracy during in-person and telepractice sessions. It is noteworthy that caregivers received a dosage of the intervention similar to prior EMT studies and met the learning criterion for each EMT strategy within five to seven sessions, which also is consistent with findings in other EMT studies (Peredo et al., 2018; Roberts & Kaiser, 2015; Roberts et al., 2014). This supports the premise that caregivers were able to learn efficiently given both in-person and telepractice sessions.

Second, the therapist leveraged technology including (a) videos of caregivers performing EMT strategies (video self-modeling), (b) PowerPoint vignettes that scripted conversational turns using EMT strategies, and (c) bar graphs depicting progress toward acquisition of EMT strategies, in order to enhance the Teach–Model–Coach–Review approach. The therapist used technology to include specific coaching practices that have been demonstrated to be effective in teaching adults new instructional practices including modeling, observation, reflection, and performance feedback (Artman-Meeker et al., 2015; Artman-Meeker & Hemmeter, 2013; Friedman & Woods, 2015). Video modeling provided caregivers with more opportunities to observe, reflect, and receive performance feedback from the therapist. The PowerPoint vignettes provided additional practice role-playing EMT strategy use with scenarios from the previous session. The graphs depicted the caregiver's performance for a target EMT strategy with a goal line illustrating the performance criterion. These bar graphs provided a context to discuss caregivers' proficiency with EMT strategies.

Wright & Kaiser (2017) suggested that insuring that caregivers met the performance criteria and providing feedback about their progress toward proficiency was one adult learning approach that was not explicitly addressed in prior studies of EMT (cf. Kaiser & Roberts, 2013; Roberts et al., 2014). Thus, the current study addressed this limitation by reviewing bar graphs as a context for discussing progress toward the performance criterion and providing performance feedback regarding EMT strategy use. Although we cannot estimate the value added by including the videos, vignettes, and graphs, data from the intervention and maintenance conditions suggest that Teach–Model–Coach–Review implemented in the current study was sufficient for caregivers to learn and maintain use of EMT strategies. Moreover, there is strong evidence to suggest coaching, which involves modeling and performance-based feedback, is critical to improving teacher and parent practices (Artman-Meeker et al., 2015; Artman-Meeker & Hemmeter, 2013; Friedman & Woods, 2015).

Findings from the current study are consistent with previous research that demonstrated the positive effects of telepractice interventions on caregiver language strategy use (e.g., McDuffie et al. 2013, 2016; Meadan et al., 2016). Results from this study extended the previous research first by using a rigorous experimental design that met contemporary design standards including the What Works Clearinghouse (2020) and Council for Exceptional Children (Ledford, Zimmerman, Schwartz, & Odom, 2018).

Second, it adds to emerging literature on naturalistic developmental behavioral interventions delivered via telepractice by establishing the initial positive effects of EMT for families of children with language delays and Down syndrome. To date, prior studies have focused on two empirically supported naturalistic and developmental behavioral interventions: Telehealth Early Start Denver Model (Vismara et al., 2012, 2013) and Reciprocal Imitation Training (Wainer & Ingersoll, 2013, 2015). Other studies have used empirically supported approaches to develop new telehealth interventions such as the Naturalistic Parent-Implemented Intervention (McDuffie et al., 2013, 2016) and Internet-based Parent-implemented Communication Intervention (Meadan et al., 2016).

Third, caregiver training utilized the Teach–Model–Coach–Review approach. As outlined in a recent review (Quinn et al., 2017), previous telepractice studies did not report using an empirically supported approach to caregiver instruction to deliver parent instruction. This is an important contribution of the current study because having a fully developed, manualized intervention with a structured coaching approach and implementation fidelity guidelines can help bridge the gap between research to practice and facilitate wide-scale adoption of empirically supported practices (Olswang & Prelock, 2015). Explicating the specific coaching practices allows for greater research replication and clinical adoption.

Fourth, the current study specified explicit performance criteria for caregiver use of EMT strategies. Several of the previous telepractice studies introduced strategies after a specified number of sessions; researchers did not make response-guided decisions based on caregiver behaviors (cf. McDuffie et al., 2013; Vismara et al., 2013). Specifying criteria for caregivers and carefully monitoring their performance are critical because research on practice-based coaching has shown the strong effects of joint planning, reflection, and feedback (Friedman et al., 2012).

Fifth, the current study delivered the telepractice intervention at a high dose in terms of both duration and frequency. Most previous studies delivered the intervention once per week and provided between 12 and 16 intervention sessions (e.g., McDuffie et al., 2013, 2016; Vismara et al., 2012, 2013), although one study provided as many as 25 sessions (Meadan et al., 2016). It is possible that the stronger outcomes observed in the current study and in Meadan et al. (2016) were the result of more frequent intervention and a larger number of coaching sessions both contributing to a more intensive intervention. This study also summarized treatment intensity parameters for the therapist and caregiver, which is an important contribution since little is known about the treatment intensity required to insure optimal effects of caregiver-implemented interventions (Baker, 2012).

A final important difference between the intervention procedures in the current study and other studies of telepractice was use of a hybrid model that involved in-person and video-teleconferencing procedures. These procedures combined the advantages of in-person coaching (relationship building, knowledge of the family's home setting and routines, opportunities to directly support caregiver use of communication and behavior support strategies) with the accessibility and cost advantages of telepractice sessions. To our knowledge, McDuffie et al. (2013, 2016) are the only other caregiver-implemented intervention studies that used a hybrid telepractice approach. Taken together, McDuffie et al. (2013, 2016) and this current study demonstrate the initial efficacy of a hybrid approach. The combination in the current study was 40% in person, 60% telepractice. Based on the current study, we do not know if this combination is optimal or if relatively more telepractice sessions could be included while maintaining the strong effects of the hybrid approach.

Limitations

Although a rigorous multiple baseline design, which controlled for many threats to internal validity and minimized the risk of bias, was implemented in the current study, several constraints of the study design introduced potential for systematic error (Higgins et al., 2011). Potential sources of bias included (a) selection bias, (b) performance bias, (c) detection bias, and (d) measurement error (Reichow et al., 2018).

An overall constraint of the study is the potential for selection bias due to participant sample and demographic characteristics. Participants were recruited from Oregon health care systems, community organizations, early childhood special education programs, and early intervention programs. While the recruitment strategy was designed to reach a diverse sample of children and families, we were only able to capture diversity on one important social determinant of health—rurality. All participants were White, had experience with a digital device and video-conferencing, plus they had reliable broadband Internet service at their homes. Therefore, the results of this study must be viewed in light of these sampling limitations, as the interventions effects may be different, and/or the intervention may need systematic adaptations for racially, ethnically, linguistically, and culturally diverse families, for families experiencing poverty, and for families without reliable Internet service.

There is a high risk of performance bias because research participants and personnel were not blind to when the intervention was implemented or when conditions changed within the intervention. Performance bias is difficult to minimize in single-case design research because of the frequent and intimate interactions between the investigator, participants, and research staff (Reichow et al., 2018).

There is a high risk of detection bias because the first author was unable to blind outcome assessors (coders) to the study condition. A key component of the intervention is coaching the caregivers on five specific EMT strategies used during parent–child interactions. The presence or absence of coaching of a specific behavior was a clear signal to the coder whether the caregiver–child dyad was in the intervention or baseline condition. Blinding of outcome assessments was not possible since the dependent variables (caregiver EMT strategy use, child communication, child number of different words) were measured while the therapist delivered EMT. Consequently, there is also a high risk of observer-caused correlated measurement error (Yoder et al., 2018). This type of correlated measurement error occurs when a coder systematically overestimates the true score during the intervention sessions and/or systematically underestimates the true score during baseline sessions.

Systematic changes in the measurement context between the baseline condition and intervention condition introduced a second type of correlated measurement error, contextual measurement error. As a result of instruction and coaching on EMT strategies, the caregiver was expected to adopt a new interaction style that was more responsive to child communication. Adopting a new interaction style during the intervention condition represented a systematic change in caregiver behavior during the caregiver–child interaction between the baseline and intervention conditions. While this was the desired outcome, using a responsive interaction style during the intervention condition supported the child's communication skills. This may have affected the exploratory dependent variable, making it appear that the child's communication acts and number of different words increased, when the child's skill might not have improved, if measured in a different context. An alternative that would have controlled for this is intermittent generalization probes with a familiar but untrained caregiver.

Other measurement issues that should be considered relate to the child behaviors. Changes were modest and variable across children. The pre- and postmeasures should be interpreted with caution in the absence of a control group. Some measures have limited psychometric information available, leading to cautious interpretation of changes. The study did not include measures of speech production and thus could not assess the impact of individual differences in speech development on the language outcomes.

Future Research

There are three critical avenues for future research. First, the results of the current study must be replicated with a larger and representative sample of children receiving early intervention including (a) caregivers employed outside the home, (b) families receiving early intervention in community-based settings, (c) families from underrepresented racial and/or ethnic groups, and (d) families experiencing poverty. Importantly, telepractice offers benefits for working families including (a) more flexible schedules for sessions, (b) minimizing family and provider travel for services, and (c) less disruption of family activities by home visitors. Peredo et al. (2018) developed EMT en Español and found there were additional linguistic, cultural, and activity adaptations of the intervention needed for resource-limited Spanish-speaking families of toddlers with language impairments. A recent meta-analysis of predictors of outcomes among late talkers found that socioeconomic status significantly predicted expressive language outcomes (Fisher, 2017); however, neither the Roberts and Kaiser (2011) or the Heidlage et al. (2020) meta-analysis reported differential outcomes of caregiver-implemented interventions associated with socioeconomic status. Future studies should include families from varied resource backgrounds to determine if telepractice interventions can be effective across the range of socioeconomic status and can specifically address the needs of children and families experiencing poverty.

Second, there is a compelling need to examine how telepractice interventions can be incorporated to optimize existing evidence-based language interventions. For example, telepractice could be used to provide “booster” sessions, monitor procedural fidelity, and provide individualized coaching in a consultative model. Telepractice holds promise for increasing the intensity of interventions and maximizing the timing of interventions based on the child and family developmental skills. While some caregiver-implemented language interventions have been conducted entirely via telepractice (Meadan et al., 2016), feedback from caregivers in the current study suggested the hybrid delivery was effective in helping them learn language strategies. Variations in delivery models for telepractice should be investigated. However, the goal of telepractice research may not be to show it can replace in-person intervention. Instead, the focus of research may be on how to leverage technology to achieve optimal outcomes, or maintain outcomes that would be difficult or impossible to sustain over time and settings.

Third, few studies have explored which coaching practices are associated with child communication growth (Artman-Meeker et al., 2015). Little is known about the relative impact of various coaching practices, and even less is known about which coaching practices work best for telepractice service delivery. Additional research is needed to build a foundation of evidence-based telepractice coaching strategies. This evidence is critical to insure early intervention providers have the opportunity to learn to use effective coaching practices specific to telepractice service delivery that have been shown to facilitate child language growth (Artman-Meeker et al., 2015; Brown & Woods, 2012; Kemp & Turnbull, 2014). Closely examining the results from the current study alongside related studies of caregiver-implemented intervention can inform future research focused on identifying effective coaching practices for telepractice and hybrid language interventions.

Clinical Implications

Findings from this research have the potential to inform local, regional, and national early intervention programs that may be considering adoption of telepractice service delivery models. The results demonstrated the preliminary efficacy of using hybrid telepractice delivery models to provide naturalistic, caregiver-implemented language interventions such as EMT.

Conclusions

In this study, four caregivers learned EMT strategies and implemented them at home during play and family routines with their young children with language impairments. The effects of the intervention maintained for up to 6 weeks after the intervention was completed. The intervention was delivered using a hybrid in-person and telepractice model. The hybrid service delivery model was cost effective. Results demonstrate the preliminary effects of using video-teleconferencing to teach caregivers language support strategies. Key stakeholders judged the intervention goals and procedures, including the telehealth service delivery model to be acceptable. Findings from the current research have the potential to help early intervention providers maximize resources and expand access to services to children and families who are significantly underserved (Rosenberg et al., 2008).

Funding Statement

The contents of this article were supported in part by the Bonsal Applied Education Research Award from Peabody College at Vanderbilt University and the Doctoral Leadership Training Grant in Early Childhood Special Education (H325D10034A) from the U.S. Department of Education. The content is solely the responsibility of the authors and does not necessarily represent the official views of the U.S. Department of Education.