Telehealth Training in Naturalistic Communication Intervention for Mothers of Children with Angelman Syndrome

Abstract  

Objectives 

Young children with Angelman syndrome have significant delays in expressive communication. Parents of children with Angelman syndrome require training to support their child’s communication development. Unfortunately, parent training focused on the needs of families of children with rare genetic syndromes is unavailable to many families. The purpose of this study was to evaluate a telehealth parent training program on naturalistic communication intervention for young children with Angelman syndrome.

Methods

Using two single-case multiple baseline designs across a total of six parent–child dyads, we evaluated the effects of a telehealth parent training program on parent implementation fidelity of a naturalistic communication intervention, child communication, and child engagement.

Results

With the telehealth parent training program, parent implementation fidelity of naturalistic communication intervention improved, maintained and generalized to untrained home routines. Small effects on child communication and engagement were observed during the program.

Conclusions

Parents of children with Angelman syndrome were successfully taught via telehealth to implement a naturalistic communication intervention with their child at home. Additional research is needed to promote positive child communication outcomes through parent-mediated intervention.

Parents of children with rare genetic syndromes have reported difficulty and increased stress in accessing information specific to their child’s diagnosis (Griffith et al., 2011). Individuals with rare genetic syndromes present with unique characteristics and needs (e.g., Adams et al., 2018; Bush & Scott, 2021). Yet information and resources for families are often aggregated under the umbrella of neurodevelopmental disorders (Morris et al., 2013). This grouping may present challenges for families in accessing recommendations specific to their child’s diagnosis and support needs resulting in isolation, frustration, and helplessness (Griffith et al., 2011). Furthermore, differences in the type of resources parents seek out have been shown to vary across genetic syndrome populations (Pearson et al., 2018). Parents of children with Angelman syndrome, for example, have reported locating information and resources related to communication is a top priority (Radstaake et al., 2014).

Angelman syndrome (AS) is a rare neurogenetic disorder with an estimated prevalence of 1 in 24,000 (Williams & Mueller-Mathews, 2020). AS is caused by abnormalities in chromosome 15 involving either a deletion of the 15q11-q13 chromosome region or mutation of the UBE 3A gene (Williams et al., 2010). AS is characterized by severe to profound intellectual disability, severe language impairment, epileptic seizures, hyperactivity, frequent laughter, and ataxic gait (Van Buggenhout & Fryns, 2009). A core feature of AS is the complete or near absence of vocal speech (Clayton-Smith & Laan, 2003; Didden, et al., 2009; Williams et al., 2010). Individuals with AS often present with deficits in skill areas related to communication, including joint attention and imitation (de Carlos Isla & Baixauli Fortea, 2016). Those individuals with AS who do acquire speech typically acquire only a few words or word approximations (Alvares & Downing, 1998; Hyppa-Martin et al., 2013).

To promote expressive communication, individuals with AS have been taught to communicate using aided augmentative and alternative communication (AAC), such as speech generating devices (SGDs; Martin et al., 2013), picture exchange (Summers & Szatmari, 2009), or unaided AAC, such as gestures and manual sign (Calculator, 2016; Calculator & Diaz-Caneja Sela, 2015) with some individuals utilizing multiple communication modes (Quinn & Rowland, 2017). Although aided and unaided AAC has been provided to support communication for individuals with AS, expressive communication skills often remain low for this population. For example, Quinn and Rowland analyzed datasets from 300 participants with ASD and found that 98% of participants communicated at pre-intentional (e.g., shifting eye gaze or orientation) or early intentional communication (e.g., joint attention by shifting gaze between person and object) levels. Given this, focusing intervention efforts on outcomes related to joint attention, pre-intentional communication, and early intentional communicative behaviors may be key in promoting expressive communication skills for individuals with AS (Calculator, 2016).

Fortunately, interventions have been effective in supporting pre-intentional and early intentional communication with children who have neurogenetic disorders (Yoder et al., 1994). Many of these interventions can be classified as naturalistic communication interventions (NCI). We use the term NCI to refer to a class of interventions focused on promoting social engagement, play, and language development through play and daily routines (Smith et al., 2004). Within NCI approaches, the interventionist arranges the environment to create opportunities for child interaction and communication, follows the child’s lead to capitalize on child motivation, engages in balanced turn taking with the child, and responds to child behaviors through mapping and expanding language (Hancock & Kaiser, 2002).

A strength of NCI is the potential for implementation by caregivers within home contexts (Erturk et al., 2020). Caregivers have been taught to implement language interventions, like NCI, using a variety of instructional tools including information sharing, modeling, prompting, and feedback (Haring et al., 2020). For example, Meadan et al. (2014) implemented one-on-one instruction in homes to teach five parents of young children with Down syndrome to embed NCI during typical daily routines. Training included written and verbal instruction, video examples, joint action planning between the parent and trainer, and opportunities for questions. They found the parent training program resulted in increased parent implementation fidelity of NCI, high levels of parent satisfaction, and parent reports of improved child communication skills.

Despite a growing literature base, much of the research on NCI approaches has exclusively focused on outcomes for children with autism spectrum disorder (Dubin & Lieberman-Betz, 2020) and few studies have evaluated NCI with children who have neurogenetic syndromes (e.g., McDuffie et al., 2016; Meadan et al., 2014; Wright et al., 2013). To date, only four studies (Calculator, 2002; Calculator, 2016; Calculator & Diaz-Caneja Sela, 2015; de Carlos Isla & Baixauli Fortea, 2016) have evaluated NCI with children with AS. Calculator (2016) examined a 10-week parent-implemented NCI consisting of mand-model with time delay prompting and a molding-shaping protocol to support enhanced natural gestures with 18 parent–child dyads. Over half of the parents reported their child had met or exceeded their goal by the conclusion of the program. However, this quasi-experimental study relied on parent report and did not collect direct measures of child communication. de Carlos Isla and Baixauli Fortea (2016) conducted a case study of one mother’s implementation of NCI, the Hanen More than Words program, with her child with AS. The 11-week program consisted of strategies for recruiting child’s attention, following the child’s lead, and engaging the child in predictable routines. Though results showed some improvement in mother–child interactions, a functional relation between the training program and parent use of strategies was not evaluated.

Designing and evaluating parent training programs for families of specific rare genetic syndromes is an understudied area and an area of priority for families (Pearson, Waite, & Oliver, 2018). One method that might be helpful for training parents of children with AS to implement NCI is practice-based coaching. Practice-based coaching (Snyder et al., 2015) is founded upon a collaborative partnership between a coach and coachee who work together to identify strengths and needs around a specific practice. Leveraging strengths, they jointly select goals and design action plans to achieve those goals. Focused observations of the coachee’s use of the practice are conducted followed by joint reflection and feedback based on those observations. This process continues until the practice is mastered. Practice-based coaching has been evaluated and found to be effective with special education and early childhood teachers (e.g., Sutherland et al., 2015), but has not yet been evaluated with parents. The collaborative, individualized, and cyclical nature of practice-based coaching may be well aligned with family-centered approaches to parent coaching (Moore et al., 2014).

Unfortunately, access to professionals with expertise in rare genetic disorders who can train and support parents is limited, particularly for traditionally underserved populations and those that live in rural areas (Bullard et al., 2021; Thomson et al., 2021). Telehealth approaches may be particularly well suited for families of children with rare genetic syndromes, like AS, who face additional barriers to accessing care and specialized services (Hyde, et al., 2020). Telehealth services provide a modality for families across wide geographic areas to access training and treatment from home utilizing video conferencing technologies (Tsami et al., 2019). Interventions delivered via telehealth have been shown to improve communication, social, and adaptive behaviors for children with disabilities (Gevarter, 2021; Simacek et al., 2017; Wainer & Ingersoll, 2014). Leveraging a telehealth approach allows for families across wide geographic areas and without local access resources specific to their child’s diagnosis and support needs (Tsami et al., 2019).

Previous research has evaluated telehealth caregiver training programs for parents of children with disabilities. Unholz-Bowden et al (2020) conducted a review of the literature on caregiver training in behavioral interventions via telehealth and found telehealth training to be associated with improved parent implementation and child outcomes. However, the authors note wide variability in the reporting of training components, the fidelity of those components, and the experimental evaluation of training components on parent implementation outcomes.

The purpose of this study was to evaluate a practice-based coaching training program delivered via telehealth for parents of young children with AS. We sought to evaluate (a) the effect of a telehealth parent training program for parents of children with AS on parent implementation fidelity of NCI practices during home routines, (b) the effect of a telehealth NCI parent training program on child engagement during home routines, (c) the effect of a telehealth NCI parent training program on child communication outcomes, and (d) parent perspectives on the social validity of the telehealth training program.

Method

Participants

Participants were recruited as part of a larger study examining telehealth intervention for caregiver wellbeing for parents of children with AS. Parents were eligible to participate if (a) they were caregivers of a child with AS between the ages of 2 and 8 years, (b) English was the primary language spoken at home, and (c) the family had access to high-speed internet. Participants were recruited through social media and via an email to Angelman syndrome caregiver organizations. Families were randomly assigned to one of two intervention cohorts of four families. During baseline, two dyads (one from each cohort) withdrew from the study. This resulted in a total of three dyads per cohort. Table 1 provides mother and child demographic information including age, ethnicity, education background, geographic setting, target communication mode, household makeup, and targeted home routines.

Table 1.

Participant characteristics 

Caregiver	Sarah	Emma	Diya	Susan	Rachel	Marilyn
Age (years)	36	42	37	49	37	39
Ethnicity	White	White	South Asian	White	White	White
Education	Graduate degree	Some college	Graduate degree	High school	Bachelor’s degree	Some college
Geographic setting	Suburban	Suburban	Urban	Suburban	Suburban	Rural
Child	Chloe	Ashley	Tanay	Michael	Lacey	Jessica
Age (years)	4	2	2	5	8	4
Target communication mode	SGD	SGD	Manual sign	SGD	SGD	SGD
Lives with	Mother, father, two siblings	Mother, father, eight siblings	Mother, father, one sibling	Mother, father, five siblings	Mother, father, two siblings	Mother, father, grandmother, two siblings
Primary routine	Play	Play	Play	Books	Play	Play
Generalization routine	Bath	Meal	Meal	Play	Yoga	Meal

Note. SGD, speech generating device

Cohort One

Sarah was a 36-year-old married White female with a graduate degree. Her child, Chloe, was a 4-year-old female diagnosed with AS at 2 years. The genetic mechanism of her AS was not reported. Chloe was minimally vocal, using some independent vocalizations to communicate, including “mom,” “more” for food, and “ouch” to escape. To augment her limited vocalizations, Chloe also had access to an iPad with an SGD application, Go Talk Pro, that displayed 12 icons. Sarah reported that Chloe did not often use the iPad to communicate but would flick the screen with her fingertips in a repetitive fashion. However, she independently used the “hi” icon to request attention. Sarah chose SGD use with a single icon display as Chloe’s target communication mode. Chloe physically engaged with toys, particularly those that she could flick with her fingertips (e.g., zippers and sequins) but did not interact with others or take turns during play routines. All sessions took place in the family’s home living room and bathroom.

Emma was a 42-year-old married White female with some college education. Her child was Ashley, a 2-year-old female diagnosed with AS at 7 months old. The genetic mechanism of her AS was not reported. Ashley occasionally made vocalizations, but they were mostly unintelligible. She communicated her desires using some gestures, including reaching up to be held. Ashley occasionally communicated rejection by shaking her head. She had access to an iPad at home with an SGD application that included 8 icons on the screen. Emma reported she had attended a workshop on AAC but did not have training in teaching Ashley to communicate with the SGD app. Emma chose SGD use with a two-icon display as Ashley’s target communication mode. Ashley engaged with toys that she could chase after or spin (e.g., wheels) but did not interact with others or share during play routines. All sessions took place in the family’s home kitchen and living room.

Diya was a 37-year-old married South Asian female with a graduate degree. Her child, Tanay, was a 2-year-old male diagnosed with AS at 12 months old. The genetic mechanism of his AS was not reported. Tanay was non-vocal. He communicated his desire for objects, people, or activities by scooting his body on the floor toward the desired stimulus. Tanay communicated rejection by turning his head away and/or crying. Diya chose sign language “more” as Tanay’s target communication mode. Tanay physically engaged with toys by shaking them and watching items that light up but did not interact with others or share during play routines. All sessions took place in the family’s living room or dining room.

Cohort Two

Susan was a 49-year-old married White female with a high school diploma. Her child, Michael, was a 5-year-old male diagnosed with AS shortly after his first birthday. The genetic mechanism of his AS was not reported. Michael was non-vocal, and used some manual sign approximations (e.g., “mine,” “please,” and “more”) and gestures (e.g., nodding and shaking his head, reaching, and waving bye). To communicate rejection, Michael would scream, pull hair, pinch, and occasionally bite, and communicated primarily with an SGD. Michael used a Samsung tablet with the Saltillo Nova Chat 10 SGD app with 10 pages containing 12 icons per page. Michael independently scanned and selected icons within a page but did not navigate across pages. Susan chose SGD use with page navigation as Michael’s target communication mode. Michael physically engaged with toys such as blocks, puzzles, and books, although sometimes struggled due to fine motor deficits. He occasionally took turns and shared during play and book routines. All sessions took place in the living room.

Rachel was a 37-year-old married White female with a bachelor’s degree. Her child, Lacey, was an 8-year-old female with AS, caused by a maternal deletion of chromosome 15. Lacey occasionally made one-syllable vocalizations, but these were mostly unintelligible. She primarily communicating with gestures (e.g., nodding, pointing, physically guiding other person, tapping objects, and waving “come here”) and an SGD. Lacy had access to an iPad with the Dynavox™ app installed. However, Rachel reported that Lacey rarely used the app to communicate. Lacey communicated rejection by screaming, pushing things or people away, and pulling hair. Rachel chose SGD use with 10 icons on screen as Lacey’s communication target mode. Lacey physically interacted with balls and tactile sensory play items (e.g., Play-Doh, puddy) and would appropriately take turns and share with her parents during play routines. All sessions took place in the family’s living room.

Marilyn was a 39-year-old married White female with some college education. Marilyn’s child, Jessica, was a 4-year-old female diagnosed with AS at 15 months old. Jessica’s AS was caused by a maternal deletion of chromosome 15. She was non-vocal and had limited use of a picture board and an SGD for communication. Marilyn reported that Jessica did not yet communicate with the iPad app but would flick and swipe at the screen with her fingertips. To communicate displeasure, Jessica made whining vocalizations. Marilyn chose SGD use with two icons display as Jessica’s communication target mode. Jessica physically interacted with balls and cause and effect toys by flicking or mouthing them but did not take turns or share during play routines. All sessions took place in the living and dining rooms.

Procedures

Setting and Materials

All sessions took place within each family’s home during parent-selected home routines. Participants selected a target routine and a generalization routine during the first intake meeting with the researcher (see Table 1). Routines included play and bath time for Sarah and Chloe, play and breakfast for Emma and Ashley, play and lunch for Diya and Tanay, reading books and play for Susan and Michael, play and yoga for Rachel and Lacy, and play and breakfast for Marilyn and Jessica. During each session, the parent and child were present, while other family members carried on with ongoing activities and routines in the home.

Each dyad was sent a Lenovo Tab 10.1″ tablet to access training materials, to participate in videoconference coaching sessions via Zoom, and to record home routine sessions with their child. The tablet was preloaded with the following apps: Gmail for communicating with researchers via email; CoughDrop™, an open-source app used for creating customizable boards for speech generated communication; Microsoft Word; an internet browser, AppLock™, to ensure that only the necessary apps were accessible; and the TORSH™ app. TORSH™ is a HIPAA compliant observation and data management platform and app with end-to-end encryption. Parents were asked to record all study sessions using TORSH™ app installed on the tablet. In some cases, participants recruited a family member to hold the tablet and record the session. Parent coaching sessions took place in the TORSH Talent™ web platform using the Zoom integration feature. The TORSH Talent™ platform features included participant-specific pages for goal setting and action planning, a video exemplar library, access to participant-uploaded videos, and time-stamped annotated feedback on participant-uploaded videos.

NCI training modules were preloaded to the tablet. The training modules were developed by the first and second author for this study and included four 10-min voice-over PowerPoint™ presentations with embedded video models of parents implementing NCI strategies with their child. The modules were locked on the tablets using AppLock™ to prevent participant access prior to intervention. Upon completion of the baseline phase, training modules were unlocked and made accessible to parents via remote desktop connection. Training modules included four 10-min voice-over PowerPoint™ presentations with embedded brief (approximately 30 s each) video models of parents implementing NCI strategies with their child.

Intake

Following informed consent, parents met with their coach on Zoom for a 60-min intake session to review study expectations, to develop rapport, and to learn about family and child history, and family communication priorities for their child. Parents were asked to provide information on the child’s strengths, diagnostic history, communication modality, familiarity and history with AAC, current services, and daily home routines. The coach guided the family to review their home routines and select a target routine during which the communication intervention would occur and a secondary routine to serve as a measure of generalization. The coach asked the parent to identify the communication mode for their child to focus on within the course of this study. Parent choices included speech generating apps on a tablet device picture exchange, vocalizations, gestures, or manual sign. Parents could elect to use AAC systems they had previously or currently had access to or to request a new SGD application. Five parents selected SGD or SGD apps to which they already had access as the target mode. Diya selected manual sign.

Baseline

Following intake, each parent met on Zoom™ with the coach for 20 min for orientation to technology and to study procedures. Coaches explained and modeled how to operate the tablet and how to access TORSH Talent™ platform and app. The coach instructed participants to record and upload three 5-min videos each week using the TORSH™ app. Two videos were to occur during the target routine and one video during the generalization routine. Participants were told to record themselves during these routines interacting with their child and supporting their child’s communication as they normally would, including the use of any AAC devices to which the child had access. After a session was recorded on the tablet TORSH app, videos were immediately and automatically uploaded to the secure TORSH Talent™ platform. No coaching took place during baseline.

Intervention

A two-step process was utilized as part of the NCI coaching package. First, parents independently watched the four training modules developed for this study on their tablets. Modules consisted of explicit instruction and video models. Each module was approximately 10 min in duration and contained written and voice-over PowerPoint content and three to four brief video examples of parents implementing the focus strategy. Examples of each communication strategy with AAC as a communication modality were described or illustrated through a video example in each module. The first module provided a brief overview on the theoretical and applied aspects of NCI and on varying communication modes. The first module included explicit instruction on embedding communication opportunities into daily home routines. The remaining three modules each described one set of strategies to encourage child communication. These strategies included (a) following the child’s lead, (b) environmental arrangement, and (c) mirroring, mapping, and language expansion. After parents had viewed the modules, they moved into weekly practice-based coaching.

All practice-based coaching session took place within Zoom™ integration feature of TORSH Talent™. Weekly coaching sessions were between 20 and 30 min in duration. Informed by the principles of adult learning, practice-based coaching sessions built upon the explicit instruction of the modules and offered differentiated instruction and opportunities for problem solving and feedback (Birman et al., 2000). During the first coaching session, the coach and parent reviewed training module content. The coach and parent completed a strengths and needs assessment together to assess parent knowledge and confidence for each of the target NCI strategies (i.e., following the child’s lead; environmental arrangement; and mirror, map, and language expansion). The coach and parent used this information to identify a weekly goal for implementing NCI during the target routine. Goals were individualized to each participant’s target communication mode, including SGDs or manual signing. This practice-based coaching format was adapted for telehealth through the use of the shared screen feature during the Zoom™ coaching session. To reduce parent response effort, the coach entered the parent’s implementation goal into the parent’s goal page in TORSH Talent™ and worked with the parent to develop an action plan to meet the goal by sharing example action plans and suggesting possible actions the parent could elect to place on the plan.

During each subsequent practice-based coaching session, the coach and parent reflected on the parent’s use of NCI that week, reviewed progress toward the parent’s goal, revised the goal if needed, and developed or revised the action plan to meet that goal. The coach offered various enhancement coaching strategies to the parent such as suggesting videos to view from the exemplar library, time-stamped written feedback on parent-uploaded videos, or written task analysis and materials for use during home routines. Parents could elect all, none, or some of these enhancement strategies each week. Parents remained in the intervention phase until they reached mastery criteria of three consecutive target routine sessions at or above 85% implementation fidelity.

Maintenance

Five of the six participants completed a maintenance assessment. Maintenance assessments were conducted 4 weeks after the final coaching session. Using the TORSH app, participants were asked to record two 5-min videos, one of the target routine and one of the generalization routine. If participant’s implementation fidelity fell below 85% fidelity on the target routine, a booster session was provided. During the booster session, the coach reviewed the NCI strategies, jointly reflected with the parent of the parent’s use of the strategies and offered enhancement coaching strategies. Following booster coaching sessions, participants were asked to record another 5-min video of the target routine. This process repeated until the participant achieved 85% or higher implementation fidelity.

Social Validity

Social validity of the training program and the telehealth delivery of the program were assessed via electron surveys at the conclusion of the study. Participants completed the Treatment and Acceptability Rating Form-Revised (TARF-R; Reimers et al., 1995), which utilizes a 7-point Likert scale to determine the level of acceptance across nine questions. The TARF-R assesses the willingness of participants to use intervention strategies, their confidence and understanding of strategies, disruption and cost of intervention, and acceptability of procedures in context of specific child needs. Additionally, participants completed a researcher-developed telehealth social validity survey that assessed the ease of technology use and preferences for modality (see supplemental file).

Measures

Parent NCI Implementation Fidelity

Parent NCI implementation fidelity was scored using a 3-point researcher-developed rating scale. The scale included 11 NCI strategies based on the work by Olive et al. (2007), Ertuk et al. (2021), and Kaiser and Roberts (2013). Table 2 depicts the 11 strategies related to AAC use; environmental arrangement; following the child’s lead; and mirroring, mapping, and expanding language. The 3-point scale rated parent use of strategies throughout the session. A score of zero was noted when the parent did not implement that strategy at all during the observation, a score of 1 indicated the parent somewhat implemented the strategy correctly but not on all opportunities, and a score of 2 indicated the parent implemented that strategy on most opportunities throughout the session. The total points earned in each observation were added and divided by the total possible 22 points and multiplied by 100 to give a percentage of points earned (Ledford & Gast, 2018). During sessions in which an item was not applicable (e.g., Tanay did not use an AAC device, the SGD was not present during Chloe’s bath time routine), then the points associated with that item were excluded from the number of points possible.

Table 2.

Parent implementation fidelity

NCI Strategy	“No” (0 points)	“Somewhat” (1 point)	“Yes” (2 points)
If applicable, maintain placement of AAC device within view and reach of child, as necessary for the participant’s specific needs and routines	Device remains out of view and reach of child for nearly all of session	Device is in view and reach of child for some of session	Device is in view and reach of child for nearly all of session
If applicable, model the use of AAC in at least 50% of trials	Does not model AAC use when presented opportunities	Models the use of AAC but in less than 50% of trials	Models the use of AAC in at least 50% of trials
If child does not respond verbally or with AAC, parent ends with a physical prompt	Does not physically prompt when presented opportunities	Physically prompts in 1–3 opportunities, but not all	Physically prompts in 4 or more opportunities OR in every opportunity
Follow the child’s lead	“No” (0 points)	“Somewhat” (1 point)	“Yes” (2 points)
Observe what child was engaged with or attending to and join that activity or comment on that activity	Parent takes the lead for most of the session	Parent and child share lead throughout the session	Child takes the lead in nearly all of the session
Talk about what the child is playing or doing, using language from either parent’s or child’s point of view	Does not narrate action from a participant’s perspective throughout session	Narrates action from a participant’s perspective 1–3 times	Narrates action from a participant’s perspective in 4 or more times
Provide semantic feedback of child communication or acknowledge the child’s response with rich vocabulary	Does not provide semantic feedback when presented opportunities	Provides semantic feedback in 1–3 opportunities, but not all	Provides semantic feedback in 4 or more opportunities OR in every opportunity
Environmental arrangement	“No” (0 points)	“Somewhat” (1 point)	“Yes” (2 points)
Remain within arm’s reach of child and at child’s level	Rarely remains within arm’s reach or at child’s level	Remains within arm’s reach and/or at child’s level for some of the session	Remains within arm’s reach and at child’s level for nearly all of the session
Remove distractions and unused materials before the session began	Distractions exist throughout the session	Distractions allowed to exist for a portion of the session	Distractions removed at the start; any unforeseen distractions that occur are removed promptly
Mirror, map, expand	“No” (0 points)	“Somewhat” (1 point)	“Yes” (2 points)
Mirror: In opportunities in which child does not verbalize, simultaneously physically imitate child’s action before verbal description (mapping) occurs	Does not mirror child’s action when presented opportunities	Mirrors child’s action in 1–3 opportunities, but not all	Mirrors child’s action in 4 or more opportunities OR in every opportunity
Map: Repeat and phonologically or grammatically add or correct something new to the child’s utterance or nonverbal request	Does not map language when presented opportunities	Maps language in 1–3 opportunities, but not all	Maps language in 4 or more opportunities OR in every opportunity
Expand: Adult rephrases what child said in a new way or adds 1–2 words; for nonverbal expansion, after mirroring parent adds language that represents what child would say	Does not expand child’s language when presented opportunities	Expands child’s language in 1–3 opportunities, but not all	Expands child’s language in 4 or more opportunities OR in every opportunity

Child Engagement

Child engagement in the target and generalization routines was measured using 10-s partial interval recording. Engagement during home routines was scored when the child did any of the following: oriented to parent (e.g., looks at parent after joint attention bid), followed parent point or parent verbal directions (e.g., shifted gaze from toy to object parent points to), joint attention by shifting gaze (e.g., looks from parent, to object, and back to parent), or pointing at object in a bid to get parent’s attention (Hansen et al., 2018). The number of intervals with engagement was divided by the total number of intervals in the observation and multiplied by 100% to obtain the percentage of intervals with engagement.

Child Communicative Behaviors

Intentional communicative behaviors were measured by coding video recordings using the Communication Complexity Scale (CCS; Brady et al., 2018). The CCS is a 12-point scale of expressive communication from pre-intentional through symbolic communication and was designed specifically for individuals with limited communication skills. Each numerical score corresponds to an operational definition of communication (See Brady et al., 2018). Scores of 0 represent no intentional communication (e.g., child does not look at or interact with the activity presented or another individual), 1 to 5 for pre-intentional communication (e.g., orienting toward stimuli), 6 to 10 for intentional non-symbolic communication (e.g., joint attention with or without vocalizations), and 11 to 12 for intentional symbolic communication (e.g., one-word or multiword verbalizations, functional communication of contextually appropriate icon on SGD). Examiners were trained by the CCS development team to become research reliable on the CCS prior to scoring.

To evaluate changes in participant communication skills on the CCS, we scored one baseline and one intervention session for each participant. Baseline sessions and intervention sessions were randomly selected using a random number generator. Each 5-min video recorded session was divided into 30-s intervals. Within each interval, the examiner scored the highest observed child communication level using the 0–12-point scale. The mean score throughout the 5-min observation was calculated to determine the overall session score. After scoring the videos, researchers calculated the optimal and typical scores. The optimal score reflects the average of the three highest scores recorded during the video and the typical score is the average of the six middle scores. Comparing the optimal score allows one to see the change in the highest form of communication over the course of the study. The purpose of using the typical score for comparison was to help remove any outlier data points that might have existed and skewed the data. To measure within-subject paired difference on the CCS, we conducted a Wilcoxon signed rank test, given the nonparametric nature of the data and the small sample size.

Interobserver Agreement

Trained graduate and undergraduate research assistants collected reliability data on each dependent variable. Across dependent variables, interobserver agreement (IOA) data were collected during at least 33% of baseline, intervention, and maintenance sessions for each participant, and was calculated by dividing the number of intervals with agreements by the total number of intervals (agreements plus disagreements) then multiplying by 100%. For parent implementation fidelity, IOA was calculated for each item on the fidelity rating scale. An agreement was defined as both observers scoring the same rating for an item. Mean IOA for implementation fidelity across participants and study phases was 88% (range 64 to 100%). For child engagement, an agreement was defined as both observers scoring the occurrence or nonoccurrence of engagement in the same 10-s interval and mean agreement was 89% with a range of 66 to 100%. Low IOA scores were primarily the result of the challenges with video recording audio and visual quality as well as to variations in participant dyad.

Inter-scorer agreement on the CCS was calculated using the optimal and typical scores coded by both graduate students on 30% of videos. Thirty percent of videos were randomly selected from the identified pre-intervention and post-intervention videos and were coded independently by a second trained graduate student who was research reliable on the CCS. After all videos were coded, the optimal and typical scores were calculated for each video and then the intraclass correlation coefficient (ICC) was determined for both scores. For optimal score, the ICC was 0.892 and 0.988 for the typical score. When calculating ICC scores above 0.8 are considered acceptable, and scores above 0.9 are considered excellent (Brady et al., 2012).

Practice-Based Coaching Fidelity

The first and second authors were trained in practice-based coaching and NCI and served as coaches. Practice-based coaching fidelity was measured for at least 50% of coaching sessions for each participant. All coaching sessions were recorded on Zoom integration with TORSH and viewed by trained observers (first and second authors). Observers scored coaching fidelity using a 12-item researcher-developed task analysis (see Table 3) adapted from Snyder et al. (2015). An item was scored correct if the coach implemented that procedure correctly throughout the coaching session. If any error in implementation for a given item was observed, that item was scored incorrect. Coaching fidelity was calculated by dividing the number of items scored as correct by the number of items scored correct plus the number of items scored incorrect and multiplied by 100%. Mean coaching fidelity was 100% across coaches and participants.

Table 3.

Coaching fidelity

Coaching practice
Welcomes parent and states purpose of coaching session
Asks parent about experience, feedback about what they learned from the incidental teaching module
Asks parent to reflect on each of the three areas of NCI from modules: follow the child’s lead; environmental arrangement; mirror, map, and language expansion. Asks parent to identify a strength in each of these areas. Ask parent if there are any areas they would like to focus on this week
Reviews importance and rationale for goal setting
Asks parent to identify one practice to work on related to one of the three NCI areas and helps operationalize practice with parent
Helps parent move practice into a measurable goal that could be accomplished in 1 to 3 weeks. Goal should focus on parent action that can be seen or heard, not child behaviors. Practice should focus on how parent uses a practice, not simply on making materials available
Reviews importance and rationale for action planning
Discusses each item on action plan and completes action plan for parent using screen share feature
Action plan is focused on observable parent actions, has specific steps aligned to the goal, follows a logical sequence, and specifies resources for each step and a realistic timeline within 1 to 3 weeks
Reminds parent coach will bring action plan and update on actions taken at next coaching meeting
Reminds parent to record 2 target routine videos and 1 secondary routine video at least 24 h prior to coaching meeting
Reminds parent of next coaching meeting date, time, and location

Data Analyses

Two concurrent multiple baseline designs across dyads (Ledford & Gast, 2018) for each cohort were used to examine the effects of telehealth parent NCI training program on parent implementation fidelity and child engagement. The study consisted of three phases: baseline, parent training, and maintenance. Within each phase, sessions were conducted within each family’s target routine and probes were conducted during generalization routines. Phase change decisions were made using visual inspection of level, trend, and stability of parent NCI implementation fidelity.

Results

Parent NCI Implementation Fidelity

Figures 1 and 2 depict results for parent NCI implementation fidelity for cohort one and two respectively. All three participants in cohort one demonstrated low and relatively stable implementation fidelity during baseline. Mean baseline implementation fidelity was 2% for Sarah (range 0–9%), 40% for Emma (range 27–50%), and 27% for Diya (range 14–36%). Similar levels of implementation fidelity were observed in the baseline generalization routine for these three participants. With the introduction of the training modules and weekly telehealth practice-based coaching, each participant demonstrated an immediate increase in implementation fidelity during the target routine. Sarah’s implementation fidelity increased from the last data point in baseline (0%) to the first target routine intervention session (60%) and an intervention mean of 69% (range 46–95%). Her implementation fidelity was variable with an increasing trend beginning on session 15 and reaching 100% by sessions 19 and 20.

Fig. 1.

Cohort one parent NCI implementation fidelity. Note. Mothers watched training modules prior to the start of intervention. Coaching sessions for each participant are indicated with asterisks

Fig. 2.

Cohort two parent NCI implementation fidelity. Note. Mothers watched training modules prior to the start of intervention. Coaching sessions for each participant are indicated with asterisks

Emma’s data also demonstrated a change in level with the introduction of the telehealth training program. Emma’s mean implementation fidelity during the intervention phase was 88% (range = 63–100%) with an increasing trend that reached mastery criteria at session 19. Diya’s implementation fidelity increased from 20% correct in baseline to 70% correct in the first intervention session. She demonstrated an overall increasing trend and reached mastery criteria in session 27. Her mean implementation fidelity during the intervention phase was 76% (range 59–94%).

During intervention, participant implementation fidelity during generalization probes increased above baseline levels for all three participants with overlapping levels of performance in the target routine. One maintenance probe was collected in the target routine for all three participants and during the generalization routine for Sarah and Diya. If participant performance dropped, a booster coaching session was provided, followed by an additional probe. During the probe, Sarah’s implementation fidelity dropped to 50%, which was above baseline levels but below the 85% mastery criteria. After one coaching booster session, her implementation fidelity reached 100% in the target routine. Her implementation fidelity performance maintained in the generalization routine. Emma and Diya maintained high levels of implementation fidelity during the maintenance probes.

Figure 2 displays results for cohort two, Susan, Rachel, and Marilyn. Again, low, and stable levels of implementation fidelity were observed in baseline for all participants in both the targeted routine and generalization routine. Susan’s implementation fidelity increased from 36% (range 32 to 41%) in baseline to 76% (range 59 to 95%) in intervention with an immediate change in level from final baseline session to first intervention session. Susan demonstrated an increasing trend in her implementation fidelity and reached mastery criteria in session 18. Rachel’s baseline fidelity was a mean of 42% (range 27 to 55%) and increased to 75% (range 55 to 90%) during intervention. Rachel showed an increasing and slightly variable trend and reached mastery criteria in the target routine in session 21. During baseline, Marilyn’s mean implementation fidelity was 24% (range 9 to 41%). She demonstrated a change in level from 20% in the last baseline session to 90% in the first intention session. She maintained high and stable levels of implementation fidelity with an overall mean of 90% (range 86 to 96%).

Like the first cohort, these three parents had improved implementation fidelity in the generalization routine as compared to baseline, yet none met mastery criteria in the generalization routine. Maintenance data were collected for Rachel and Marilyn, and both maintained mastery criteria during the target routine. Rachel’s implementation fidelity during the generalization maintenance probe was near baseline levels. Marilyn implementation fidelity increased during the generalization maintenance probe.

Child Engagement

Figures 3 and 4 depict results for child engagement in the target and generalization routine sessions. During baseline, Chloe showed a decreasing trend in engagement in the target and generalization routines with a mean of 26% (range 13 to 50%) in the target routine and 23% (range 10 to 37%) in the generalization routine. With the introduction of parent-implemented NCI, her engagement was variable in the target routine with an increasing trend during the first five sessions followed by low levels of engagement in the subsequent 3 sessions and a final increase in the last target routine session. Mean engagement during intervention was 28% with a range of 3 to 57%. Engagement during the generalization routine was consistent with baseline levels with a mean of 14% (range 7 to 30%). During the maintenance probes, there was a decrease in Chloe’s engagement in the target routine and a slight increase in the generalization routine as compared to intervention sessions.

Fig. 3.

Cohort one child engagement

Fig. 4.

Cohort two child engagement

Ashley engagement was low during baseline for both the target (M = 10%, range 3 to 37%) and generalization (M = 15%, range 3 to 33%) routines. This pattern persisted during intervention with similar levels of engagement similar to baseline in both routines (target routine M = 14%, range 0 to 40%; generalization routine M = 11%, range 10 to 13%). During the maintenance phase, Ashley’s engagement increased to 60% during the generalization routine and remained at 20% during the target primary routine.

Tanay had variable levels of engagement in both the target (M = 56%, range 30 to 83%) and generalization (M = 56%, range 30 to 80%) routines during baseline with an overall decreasing trend. This decreasing trend continued during intervention for the target routine and remained variable for the generalization routine. Tanay’s mean engagement in intervention was 34% (range 17 to 50%) and 55% in the generalization routine (range 50 to 60%). He engaged in 23% of intervals during the maintenance probe in the target routine.

Engagements for Michael, Lacey, and Jessica are depicted in Fig. 4. Each participant showed varying levels of engagement across routines in baseline. Michael’s mean baseline engagement was 55% (range 30 to 73%) in the target routine and 53% (range 37 to 70%) in the generalization routine. With the introduction of the parent-implemented NCI, Michael’s engagement levels decreased slightly in the target routine (M = 31%, range 20–43%) and the generalization routine (M = 34%, range 7 to 53%). Maintenance probes were not conducted for Michael due to COVID-19 impacts on the family.

Lacey’s mean baseline engagement in the target routine was 31% with a range of 13 to 60%, and 23% in the generalization routine (range 7 to 33%). During the NCI phase, her engagement decreased to a mean of 12% (range 3 to 40%) in the target phase and remained comparable to baseline levels in the generalization phase (M = 22%, range 7 to 37%). Lacey’s engagement remained low in both the target and generalization maintenance probes.

During baseline, Jessica displayed higher levels of engagement in the generalization routine (M = 37%, range 3 to 63%) than in the target routine (M = 12%, range 0 to 43%). Levels of engagement increased during the first two parent-implemented NCI sessions and returned to baseline levels in both the target and generalization routines. Mean engagement in intervention in the target routine was 35% (range 17 to 70%) and 16% (range 3 to 23%) in the generalization routine. Maintenance probes in both target and generalization routines remained low for Jessica.

Child Communicative Behaviors

CCS optimal and typical scores were used to assess the effects of the intervention on early communication behaviors. Optimal scores report the highest level of communication observed during the session, and typical scores refer to the median level of communication observed across the session. Table 4 presents optimal scores and typical scores for each participant alongside the results of the Wilcoxon test for significance in the change in mean from baseline to end of intervention. Five of six participants displayed an increase in their optimal score from baseline to intervention and four of six participants increased both their optimal and typical score from baseline to intervention. However, the Wilcoxon signed rank test indicated that scores were not significantly higher after the intervention (M = 7.94, n = 6) compared to before (M = 5.67, n = 6), z = 1.78, p = 0.075, with a moderate effect size of r = 0.51. For typical scores, the baseline average was 4.08 and decreased to 3.97 after intervention. This change was also not statistically significant (p = 0.528).

Table 4.

Participant Scores on Communication Complexity Scale

Participants	CCS optimal scores			CCS typical scores
	Baseline	Intervention		Baseline	Intervention
Lacey	7	9		3.33	6.33
Michael	11	9.33		11	5
Jessica	7	11		3.17	5.17
Chloe	3.33	5.67		2	2.33
Ashley	2.67	8.33		2	2.17
Tanay	3	4.33		3	2.83
			p value			p value
Mean	5.67	7.94	.075	4.08	3.97	.528

Note. This table shows the CCS optimal and typical scores for each participant at the beginning of baseline and the end of intervention

Social Validity

Participant responses on the TARF-R indicate the program was well suited for natural home routines occurring, easy to use, unintrusive, and appropriate to address the needs of the child. Of the nine items in the survey, six questions are presented in which high scores indicate participants found the intervention acceptable (e.g., How much do you like the suggested practices?). The mean responses on those items were 6.0 or higher. Conversely, three items on the TARF-R are presented in which high scores indicate participants found the intervention unacceptable (e.g., How disruptive will it be to your home to implement the suggested practices?). Participant scores averaged 1.75 or less on each of these questions. Responses on the telehealth social validity survey indicated participants viewed telehealth as the ideal modality for coaching sessions. Four of the six participants reported that they prefer to engage in programs targeting their child’s communication via telehealth, while only one participant responded that in-person delivery was preferred. Five of the participants rated the ease of technology use as excellent and one rated it as good.

Discussion

The purpose of this study was to evaluate the effects of a practice-based coaching telehealth parent training program designed to teach parents to implement NCI with their children with AS with fidelity. The training program consisted of four brief modules and weekly one-on-one telehealth practice-based coaching. Results showed that all six parent participants increased NCI implementation fidelity to mastery criteria (three consecutive target routines with at least 85% fidelity) during intervention with some evidence of generalization to non-target routines. Results maintained for four of the five participants who completed 4-week maintenance probes. With just one booster coaching session in the maintenance phase, the fifth participant achieved mastery criteria.

This study adds to a small body of work examining parent-implemented NCI for children with AS. Although this was a preliminary study of a telehealth approach to NCI parent training for families of children with AS, our findings align with previous research supporting the effectiveness of telehealth-delivered parent training on parent communication intervention implementation (Akemoglu et al., 2020). The improvement in parent use of NCI strategies suggests that a telehealth approach consisting of explicit instruction and practice-based coaching may be a beneficial approach to supporting parents of children with rare genetic syndromes. Future research is needed to replicate these procedures across additional families and to further evaluate the feasibility and scalability of such an approach with a focus on provider delivery of the program.

In this study, we assessed both maintenance and generalization of effects on parent implementation and child engagement. Although maintenance probes were few and were conducted in close temporal proximity to the training program, high levels of implementation fidelity suggest the training program may result in maintained NCI implementation. This may be due to the ease with which NCI can be embedded into ongoing routines (Kaiser & Roberts, 2013). Although Sarah did not maintain high levels of fidelity at maintenance, she was able to recover previous mastery of the NCI strategies following a single booster coaching session. For parents who do not maintain high levels of implementation fidelity, future research should examine the role and structure of brief booster coaching sessions. During the parent training phase, all six parents’ implementation fidelity in the generalization routine improved and was maintained at 4 weeks. This is a promising outcome that generalization of parent NCI fidelity can occur across routines. Because none of the parents reached mastery criteria in the generalization phase, future research could evaluate effects of adding intervention components designed to facilitate generalization (Stokes & Baer, 1977) and further explore the role of brief booster sessions in supporting generalization across home routines.

Potential mechanisms for the positive parent outcomes may be linked to the structure of the telehealth training program. The program included elements of explicit instruction including written and audio instruction, video examples, practice, and feedback. Explicit instruction has been shown to be an effective adult learning approach that leads to changes in practices (Kirkpatrick et al., 2019). This study suggests these components may be effective for training families of children with rare genetic syndromes as well. In this program, explicit instruction was delivered asynchronously through brief online modules. Such a model may be scalable to larger numbers of families and can be tailored to the needs of specific populations.

In addition to explicit instruction, this program incorporated practice-based coaching. Through joint goal setting, action planning, reflection, and feedback, parents demonstrated continued improvement and ultimate mastery of NCI implementation. Practice-based coaching has produced positive outcomes in teacher practices (e.g., Sutherland et al., 2015) but this is the first adaptation and application of practice-based coaching delivered via telehealth with families of children with AS. The program adhered to core features of practice-based coaching including focus on a specific set of practices, collaborative coaching partnership between the parent and the coach, joint goals and action planning, and focused observation and reflection that have been implemented with teachers in previous research (e.g., Shannon, et al., 2021). However, to facilitate fit of practice-based coaching with families, several adaptations were made. These adaptations including future research are needed to consider what adaptations to this coaching model may be beneficial for families and for telehealth delivery.

Despite improved parent implementation fidelity, no functional relation in child engagement was observed, and the improvements in child communication measures were not statistically significant. However, given the social communication and fine motor deficits present in many individuals with AS, it may not be practical to expect immediate positive outcomes in a low-intensity intervention targeting AAC use (Calculator, 2014). This is one of few studies that has directly measured AS child communication during NCI rather than relying on parent report (e.g., Calculator, 2016) and underscores the importance of using direct measures of child communication in future research. Future research could compare parent report measures to direct observation measures of child communication and evaluate alignment.

We did observe a small but not statistically significant increase in participant optimal scores from pre to post observation on the CCS but no change in the typical scores. Though both typical and optimal scores on the CCS provide insight into an individual’s communication level, optimal scores may be a more appropriate measure for children with AS. Optimal scores reflect the mean of the scores displayed during the three highest scores of the observation. To calculate typical scores, the highest and lowest two scores are removed from analysis and the remaining mean score is obtained. Given that observations were only 5 min and consisted of only ten 30-s intervals, the typical score calculation may have lacked sufficient sensitivity to detect change. Nevertheless, our findings support previous research suggesting that individuals with AS often remain at pre-intentional and pre-symbolic communication levels (Quinn & Rowland, 2017). It is possible that the lack of statistical significance is due to the small sample size. Thus, the gains seen here, though small, have clinical significance, particularly after a such low-intensity communication intervention.

Similarly, even with improved parent implementation fidelity of NCI, child engagement did not increase during the intervention. This again highlights challenges in measuring progress in pre-intentional communication behaviors, such as joint attention, with children with genetic syndromes. For example, Wright et al. (2013) evaluated an NCI approach focused on joint attention, play, and expressive communication with three children with Down syndrome and found the intervention led to improvements in the use of manual sign, but no clear effect on spoken words or joint attention. However, the purpose of this study was to evaluate a low-intensity naturalistic intervention that could be easily incorporated into daily family routines. While we found improvements in parent implementation fidelity and high levels of social validity, the lack of improved child engagement and communication may suggest that a more intensive explicit approach is necessary for this population (Summers & Hall, 2008) and aligns with previous research with this population (Calculator, 2016).

Additionally, because many of the routines involved play, it is possible that engagement results were confounded by play skill deficits (Kasari et al., 2010). Future research should explore the effect of play skill instruction on child engagement during play routines with children who have AS, particularly considering evidence suggesting that a collateral improvement in untargeted play behavior may arise during naturalistic interventions for children with other developmental disorders (Ledbetter-Cho et al., 2017).

All parents reported high levels of satisfaction with the parent training content and the telehealth delivery. This may be because training materials and content were specially designed for families of children with AS, which is often lacking in rare genetic syndrome parent support (Pearson et al., 2018). This alignment between training materials and child characteristic and support needs may have facilitated the successful application of strategies within and across each dyad and resulted in high levels of social validity. Additionally, the naturalistic routines targeted by NCIs are designed to be meaningful for the specific needs of the family, and therefore may be positively perceived by participants even after supports have been removed. Although the effects on child outcomes were minimal, NCIs, particularly when used as a parent-mediated intervention to teach AAC use, may require a higher intensity treatment before benefits are displayed in child social communication skills (Wright & Kaiser, 2017). Future research could explore the adult and child outcomes of a longer term program. Furthermore, this training program offered an efficient (5 weeks) and feasible approach to parent training. Breaking the instructional content into four 10-min modules may have enhanced feasibility for parent viewing. The weekly practice-based 30-min coaching session was scheduled at parent’s convenience. All participants rated the telehealth delivery of the parent training program as high. This may be due to acute circumstances of government-ordered shutdown due to the COVID-19 pandemic or could have reflected a broader desire for telehealth support. Future research should further explore parent perceptions of telehealth services and neurogenetic syndromes.

Limitations and Future Research

There are several limitations of this study. First, to participate in the study, families must have had access to high-speed interest and research is needed to explore avenues for supporting AS families without such access. The quality of videos participants uploaded varied considerably. Poor lighting, child or parent moving out of video frame, and the presence of distractors (e.g., other people in room) complicated coding and contributed to lower IOA scores on some videos. Videos were coded in the order in which the participants uploaded them. Though parents were told to submit two target videos and one generalization video per week, they were not always able to do this. Due in part to the quality and nature of the uploaded videos, it was sometimes challenging to accurately measure participant communicative behaviors and engagement. For example, joint attention depends on eye gaze (Hansen et al., 2018), and it can be difficult to track eye gaze from video recordings. Future research is needed to develop methods for measuring joint attention and pre-intentional communication behaviors from video recordings. The CCS has been used to measure communication skills during scripted and unscripted administrations (Brady et al., 2018). To our knowledge, this is the first study to apply the CCS to home contexts. Additional research may be needed to refine the use of CCS for remote assessment in home environments. Additionally, although a post-intervention survey indicated that participants found the intervention to be appropriate, interviews with the mothers would have further verified the social validity of the study. Future research is needed to translate parent-implemented intervention to maximize positive child outcomes for AS families.

Author Contribution

MR: designed and executed the study, oversaw data analysis, and wrote the manuscript. ES: assisted in execution of study, created figures and assisted in data analysis, assisted in writing the method and results section of the manuscript. CV: assisted in data collection and analysis and in writing the results section of the manuscript. RL: assisted in design of the study and preparation of caregiver training content. RM: assisted in design of the study and in writing of the discussion and editing of the final manuscript. BK: secured funding for this project, assisted in participant recruitment and data collection and writing of the method section.

Funding

This work was funded by the Angelman Syndrome Foundation Caregiver Support Fund and the Office of Special Education Programs, US Department of Education, Award H325D190064a.

Declarations

Ethics Approval

This study was approved by the Purdue University Institutional Review Board IRB-2020–328.

Informed Consent

Written parental consent was obtained for all participants and participants were able to withdraw from the study at any time.

Conflict of Interest

The authors declare no competing interests.