Exploring the feasibility of collecting multimodal multiperson assessment data via distance in families affected by fragile X syndrome

Abstract

Introduction

Telehealth is an important tool in helping to provide services for hard-to-reach populations. One population that might benefit from telehealth are individuals with fragile X syndrome (FXS). Although FXS is the leading inherited cause of intellectual disability, it is nonetheless a low incidence disorder. Individuals with FXS and their families are involved in research studies, clinical trials and receive interventions – many of which are only offered in a few locations in the United States and thus, not easily accessible to many families. The current project explored the feasibility of using telehealth procedures to collect multimodal behavioural and psychological assessment data from these families.

Methods

Participation in the current study involved online surveys, measures of physiological indices of stress, live interviews and observations of mother–child interactions conducted via distance videoconferencing using the family’s own technology when possible. Across all modes of data collection, we obtained information regarding the feasibility of participating entirely via distance by documenting missing data as well as each mother’s overall impression of participating via distance.

Results

Our telehealth procedures were successfully implemented across a wide range of technology platforms with limited difficulty, and we documented little missing data due to technology-related challenges. Perhaps most importantly, however, our sample of mothers reported high satisfaction with participating via distance.

Discussion

These findings suggest that a wide range of services and types of assessments may be amenable to telehealth procedures. Further, the findings have immediate applications as the field shifts towards telehealth due to the coronavirus disease 2019 (COVID-19) pandemic.

Introduction

Telehealth is most often characterized as a means of delivering clinical services via technology-based interactions, including the use of videoconferencing, and has been widely seen as beneficial in easing the burden on the broader healthcare system. ¹ In recent years, telehealth has been utilized in multiple disciplines to better support a variety of patient needs, from stroke treatment to mental health services. ² Telehealth has been shown to be beneficial in bringing meaningful services to hard-to-reach communities, such as those in rural areas, ³ as well as individuals who are impacted by a rare illness or disability and thus are not geographically close to a specialist with expertise relevant to their condition. ⁴ , ⁵ Further, in light of the recent coronavirus disease 2019 (COVID-19) pandemic, the utility of telehealth and related technology-based procedures has been crucial in maintaining connectedness, from clinical care, to education, to daily social interaction, during an unprecedented and prolonged period of limited face-to-face contact. ⁶

One population for which telehealth has special appeal is families of individuals with neurodevelopmental disabilities. ⁷ , ⁸ Individuals with neurodevelopmental disabilities, such as autism spectrum disorder (ASD), Down syndrome and fragile X syndrome (FXS), often have delays in language and cognition, ⁹ , ¹⁰ comorbid challenging behaviours that interfere with social life, poor academic achievement and limited mastery of daily living skills. ¹¹ , ¹² These individuals can benefit from high quality, evidence-based behavioural, social and educational interventions.^13–17 Further, these individuals can benefit from consistent assessments to document improvements and identify high-priority areas for intervention. ¹⁸ Although the current standard is still for these services to be provided through face-to-face in-person interactions in clinical settings, there is growing support for offering many of these services via telehealth practices, especially practices aimed at training parents and other caregivers to deliver therapies directly to their sons and daughters. ⁸ ,^19–22 More recently, there has been research documenting the utility of remote consultations and assessments.^23–28 Utilizing telehealth procedures might be especially useful in families impacted by low-prevalence neurodevelopmental disabilities such as FXS. Although FXS is the leading inherited cause of intellectual disability, it is still a rare condition and is a lower occurring disability compared to autism and other neurodevelopmental disabilities; ²⁹ thus, these families are at a higher risk for being geographically removed from specialists who have expertise in FXS. ³⁰ Further, biological mothers of youth with FXS are carriers of either the FMR1 full or premutation, resulting in their own behavioural phenotype which is characterized by elevated rates of depression and anxiety among other physical health issues. ³¹ These challenges could lead to difficulties in dealing with the demands of telehealth, and thus the acceptability of such an approach needs to be explored further in this population in particular.

Remote delivery of interventions

Individuals with neurodevelopmental disabilities such as FXS benefit from consistent interventions to support a wide range of skills including behavior ³² , ³³ and language. ³⁴ In recent years, many different clinicians have shown the utility of delivering interventions remotely. Studies by McDuffie and colleagues ¹⁹ , ³⁵ have shown the utility of telehealth in teaching parents of children with FXS how to implement language support strategies when interacting with their children through a combination of synchronous and asynchronous interactions. There is additional support in the FXS literature regarding the benefit of telehealth-delivered interventions in reducing problematic child behaviours such as aggression and task social withdrawal. ³⁶ Further, numerous studies in the ASD literature ²² , ³⁷ have also supported the efficacy of telehealth interventions in supporting more optimal child outcomes (e.g., a decreased frequency of problem behaviours). Telehealth interventions in these populations, however, have largely focused on parent training to ultimately support the child outcomes and thus, less is known regarding the feasibility of clinician-to-child interventions via telehealth and should be explored further.

The evidence clearly supports the feasibility and efficacy of a variety of telehealth-delivered interventions. It is of note, however, that many of these same interventions nonetheless rely on behavioural, cognitive, and psychological and other assessment data being collected in person in a clinic or university setting, thereby still placing considerable burden on the families to travel, typically on multiple occasions, during their participation.

Remote delivery of assessments

Families who have a child affected by intellectual and developmental disability identify the difficulty of travelling to a clinic as a barrier to participation in research. ³⁸ Additionally, given the high prevalence of challenging behaviours and cognitive impairment of children with FXS, travel to research facilities is not only burdensome for families, but it can result in less than optimal assessment compliance in a clinical or research setting for participating families and preclude participation for other families. In addition, the findings of such studies may have limited generalizability to settings in the child’s daily life.

Merging principles of telehealth with principles of standardized assessment is one solution to helping support these families that are unable to travel to a clinic for a myriad of reasons. Importantly, there is preliminary support for the feasibility of collecting a wide array of assessment data from behavioural measures to physiological (e.g., heart rate) in infants with Down syndrome, ²³ as well as support for the validity of using telehealth to coach parents on how to use various probes for behaviour as a tool for early autism screening. ²⁴ Beyond infancy, there is also support for clinician-to-child language assessments being conducted through telehealth with some success. ³⁹ Although prior work shows the utility of utilizing telehealth procedures to collect assessment data, there is still a relative dearth of information on the acceptability of such approaches with regards to the participants and the practicality of implementing procedures across a variety of electronic devices, which could impact scalability and broaden the reach of diagnoses and assessment of treatment of outcomes. In the case of FXS, the physical and behavioural health challenges faced by the biological mothers of individuals with FXS by virtue of carrying either the full mutation or the FMR1 premutation may be a barrier to implementing telehealth assessments. Given the utility of telehealth to do just that, future research needs to be done to address this gap in the current literature.

Current study

Given the success of previous studies collecting assessment data via distance with a variety of populations, ⁴⁰ the present study was designed to investigate the feasibility and acceptability to families of individuals with FXS of distance technology-delivered multiperson assessments. In the study, we focused on telehealth for data collection to improve generalizability of our findings while allowing us to obtain direct measures of child and mother characteristics through online parent report, direct observations of parent–child interactions in the home through videoconferencing, and wearable technology to measure physiological markers of stress. Biological mother and child dyads were chosen in particular due to the unique nature of their dyadic relationship in that they both experience inherent biological risk associated with FXS. The following hypotheses were addressed:

1. We hypothesized that distance technology would allow us to collect direct observations of parent-child interactions, interviews, questionnaires, and physiological measures of stress without requiring participating mother–child dyads to visit a clinic. We addressed this hypothesis largely through analysis of the frequency of missing data due to technology-related challenges.

2. We hypothesized that all families would be able to learn to use the technology proficiently and that data collection would be possible across various device platforms (i.e., PC vs. Mac, tablet vs. laptop). This hypothesis was addressed through parent report of their impressions of technology use and challenges throughout the study.

Methods

Twenty-two biological mother–child dyads were recruited to participate in a larger study exploring the relationship between phenotypic characteristics of FXS and the impacts on mother–child interactions. FXS status of the full mutation was determined for the child through genetic documentation provided by the parent confirming their diagnosis. Further, mothers who were either carriers of the FMR1 full mutation or more commonly the premutation, provided documentation of their own genetic testing when available. Of those mothers, 20 provided electronic informed consent to participate prior to completing data collection, whereas the other two were lost to follow-up prior to starting. Mothers were given the option to utilize their own personal technology for the videoconferencing portion of the study or were provided technology by the study when needed/requested. Requirements for the families’ devices were that they had access to a web browser and camera. All technology procedures were developed with the help of the University of California, Davis Health System Information Technology unit and were implemented in accordance with Institutional Review Board (IRB) and Health Insurance Portability and Accountability Act (HIPAA). Families were recruited through national listservs, and given the distance nature of this study, participation was not restricted by geographic location; however, all mother–child dyads did need to be English speakers. Of the 20 families enrolled to participate, 15 states/provinces were represented and two countries (USA and Canada) with families participating remotely between 42 miles and 2996 miles from the MIND Institute (M = 1797 miles, Mdn = 1932 miles, SD = 952 miles). Children were between the ages of 6 and 11 years, and their biological mothers were between the ages of 28 and 47. Further, the mother participants were generally well educated, having completed an average of 16 years of school, which is equivalent to a college education, and had an average IQ of 108 (as assessed directly in the study). The household income for our participants indicated that they were relatively well resourced, with only one family reporting income in the lower range. Additional child and mother demographics can be found in Table 1.

Table 1.

Participant demographics.

	N	%
Child participants (6–11 years old; M = 8.75, SD = 1.83)
Female	4	20
Male	16	80
American Indian/Alaskan Native	1	5
Black or African American	1	5
White	13	65
More than one racea	4	20
Unknown or prefer not to answer race question	1	5
Hispanic	3	15
Not Hispanic	16	80
Unknown or prefer not to answer ethnicity question	1	5
Mother participants (28–47 years old; M = 40.35, SD = 5.27)
American Indian/Alaskan Native	--	--
Black or African American	1	5
White	14	70
More than one racea	4	20
Unknown or prefer not to answer race question	1	5
Hispanic	3	15
Not Hispanic	16	80
Unknown or prefer not to answer ethnicity question	1	5
Household information
Household income
<$50,000	1	5
$50,000–$150,000	9	45
>$150,000	9	45
Preferred not to answer	1	5
Caregiver status
Primary caregiver (mother)	20	100
Two parent/caregiver household	16	80
One parent/caregiver household	4	20

^aNote. 75% of the child participants that selected more than one race families endorsed both Black/African American and White, with the remaining 25% indicating American Indian/Alaskan Native and White. 50% of our mothers reporting more than one race selected Black/African American and White and the other half selected American Indian/Alaskan Native and White.

Use of personal technology devices

Of the 20 families, 17 were successful in using their own devices with minimal complications. For the remaining three families, one elected to have a device provided to them from the onset of the study, one was unable to download the teleconferencing software on their device due to out of date operating software and thus elected to have a device provided to them at that time, and one was lost to follow-up after the technology training session. Although this last family was unable to connect to the video portion of the call using their own device during the technology training session, we were unable to determine whether it was technology issues or other factors that contributed to the family discontinuing their participation. Importantly, we were successful in completing the videoconferencing calls across multiple types of devices, including smart phones (e.g., iPhone®), tablets (e.g., iPad®) and a variety of computers (e.g., laptops running Microsoft® Windows® or macOS®), and with only occasional minor and easily resolvable issues. We did, however, encounter connectivity issues with Google products including a smart phone and Chromebook™ notebook computer with the solution being to use a different device.

Unique study page

Given the multimodal nature of data collection, we elected to develop unique study pages for each parent through SharePoint® utilizing the secure, HIPAA compliant and UC Davis Health Research IT approved Office 365® suite. These pages allowed us to create user friendly access to the videoconferencing call, study questionnaires and a video that was watched by the mother participant during the baseline physiological data collection session described below. Unique, non-identifiable, email addresses were created and provided to the family for gaining access to this website. The email addresses were then deactivated upon completion of the study.

Videoconferencing software

Although a variety of videoconferencing software platforms have been shown to be viable options in connecting with families, Skype for Business™ was the preferred option at the time of the study implementation for creating a secure connection with our participating families as it was a part of the UC Davis Health Office 365® suite. These calls were hosted through a UC Davis account and families joined the secure link as a guest participant. This connection also allowed for research personnel to record the sessions; however, for added security, the participant could not record the sessions. All participants provided informed consent at the onset of the study to being recorded and were notified during each call when the recording was being started.

Tech training

Prior to beginning data collection, a researcher (LB) and the parent completed a technology training session. This session started over the phone and transitioned to the technology as the researcher walked the mother through a provided technology guide individualized to their personal device. The researcher ensured that the mother was able to access all of the required components for the study and ended on the video call to ensure an adequate connection. The researcher also worked with the mother to find proper locations for the various types of calls (e.g., a quiet and private place to complete interviews and cognitive assessments as well as a place for the parent–child interactions to occur). All technology-related issues were discovered during this call, and steps were taken to fix the problems prior to data collection.

Assessment measures

The present study collected data across a variety of assessment approaches as a way to obtain a comprehensive profile of the mother and child participants.

Direct assessments and interviews

The mother participant completed the General Ability Measure for Adults (GAMA), ⁴¹ via distance as a measure of maternal cognitive ability. The mother and research staff were connected by video call and the mother orally reported her answers as the staff member recorded responses and monitored the 25-minute time limit. The mother also reported on her child’s adaptive and developmental level through the Vineland-3, ⁴² a comprehensive interview that takes approximately 1 hour and was completed over video call with research staff. Mothers were given the choice to complete these two tasks in either one or two sessions depending on their availability.

Observations of mother–child interactions

Mother–child dyads also completed a series of semi-structured interactions with research staff observing via videoconferencing. Interactions included the telling of a wordless picture book, playing with a developmentally appropriate set of toys for 15 minutes and while making a snack together. These observations were recorded and later transcribed and coded for maternal language input and measures of child spoken language. Mother–child dyads completed each interaction one time, and the order was randomly distributed across participants. These interactions occurred in separate sessions and on different days with the exception of four dyads who requested to complete two interactions at different times on the same day.

Physiological indices of stress

Mothers were also provided with an E4 Empatica physiological wristband (Empatica Inc, Boston, MA) that was used to collect physiological indices of stress from a distance. Research staff provided detailed instructions on how to use the wristband as well as supported the parent in using the wristband during the various teleconferencing calls. The device is designed to be able to store data internally and then, upon receipt after study completion, data can be extracted. Mothers wore the wristbands during three separate mother–child interaction distance sessions: (1) reading a book together, (2) playing with a provided set of toys and (3) while making a snack. At the start of each session, mothers completed a 5-minute baseline condition, during which they watched a video of waves crashing and then immediately transitioned to one of the three interactions.

Questionnaires

The current study also employed the use of online questionnaires to be completed by the mother who responded on their child’s behaviour as well as on their own feelings of parenting stress and mental health status. These questionnaires were completed via REDCap surveys and were made accessible to the family through their unique study page. Questionnaires were reviewed during the tech training call and completed within 1 week of the mother–child interactions.

Data collection and analysis plan

Data were collected across five to six telehealth sessions with the second session occurring in either one or two sessions depending on family availability (Figure 1). Detailed instructions regarding how to use the necessary technology were provided to the families and reviewed during a distance training session. At each study call, study staff documented any technology-related issues that occurred and reported whether a session needed to be rescheduled due to technology failure. Upon completing data collection, mothers were asked to complete a final questionnaire about their impressions of using technology during the study and participating entirely via distance (Table 2). We assessed telehealth feasibility by (a) documenting the frequency of missed sessions and missing data, (b) the number of technical trouble-shooting calls required and (c) by soliciting maternal impressions of participating via distance and, if they were to participate in research again, whether they would prefer to do so through distance technology or through travel to the research centre. Data analyses to address our hypotheses are largely descriptive summaries of the feasibility measures. The analyses reported include only the 19 mothers who completed the study.

Figure 1.

Timeline of distance sessions (DS) completed over the course of the study.

Table 2.

Mother report of telehealth feasibility.

Question	Median	Interquartile range
1. I am comfortable using technology outside of this project	7	1
2. I found the videoconferencing software easy to use	7	1
3. Wearing the physio-wristband was burdensome	1	1
4. I found the online format for completing questionnaires preferable to paper-and-pencil	7	1
5. I felt comfortable being observed via distance while interacting with my child	7	2
6. The interactions with my child felt representative of their everyday behaviour	7	1
7. I encountered several technology-related problems over the course of the project	1.50	3
8. Participating from a distance allowed for more flexibility in scheduling sessions	7	1
9. I would participate in another study via distance	7	0
10. Participating via distance was preferable to travelling to a clinic	6.50	3

Note. Possible scores for each item ranged from 1 to 7, with 1 being “strongly disagree” and 7 being “strongly agree”.

Results

All but two of the 19 mothers used their own technology for the study. Of these 17 mothers, all indicated that they were able to use their own devices with minimal difficulty and would prefer not to have a device provided to them in future projects. The two mothers who had technology provided to them reported that they were able to use the technology with minimal difficulty and would prefer to have technology provided to them again in future projects.

The data on maternal impressions of the technology are provided in Table 2. All 19 mothers indicated that they were comfortable using technology outside of this project, with the majority (77%) endorsing that they strongly agreed (item #1). The majority of the mothers also indicated that they felt the videoconferencing software was easy to use, with 69% saying they strongly agreed (item #2). The majority of mothers did not think that wearing the physio-wristband was burdensome, with 77% saying they strongly disagreed with it being burdensome (item #3). Although the majority of mothers reported that they preferred the online format for completing the questionnaires compared to paper-and-pencil, there was variation in responses with one mother responding more neutrally and two others preferring paper-and-pencil (item #4). The majority of the mothers felt comfortable being observed via distance, whereas two indicated a more neutral stance on this issue (item # 5). Importantly, all mothers agreed that the interactions felt representative of their child's everyday behaviour, with the majority of mothers (70%) indicating that they strongly agreed (item #6).

Mothers reported a range of responses to whether they encountered technology-related problems over the course of the project, with the majority indicating that they did not, three mothers were neutral, and three mothers reported that they had encountered some problems (item #7). The majority of the mothers indicated that participating from a distance allowed for more flexibility in scheduling sessions, with only one mother indicating that she felt neutral (item #8). All but one mother indicated agreement to participating in another study via distance, with that one mother being neutral (item #9). Finally, there was again variability in responses surrounding the mothers’ preference to participating via distance vs travelling to a clinic. Although a majority of the mothers agreed that to some degree that distance was preferable, there were three mothers who were neutral and one who disagreed with preferring distance (item #10).

Overall, we did not encounter significant data loss due to technology-related challenges. The exception was some loss of physiological data points due to technology issues surrounding the wristband. Specifically, there was one instance in which the mother could not get the watch into data collection mode and thus no data were collected for that participant and three instances in which a mother inadvertently turned the watch off between the baseline and interaction conditions, resulting in partial data loss (13%). Further, the video teleconference recorded mother–child narrative tasks were of sufficient quality so that all were reliably transcribed and coded to address the larger project’s aims.

We did not need to reschedule any sessions due to technology issues beyond those encountered during the initial technology training session. It is of note, however, that calls were frequently rescheduled to accommodate changing schedules and/or the mother indicating that the child was not likely to comply. Overall, it took participating families an average of 26 days to complete all of the distance sessions (range: 4–148 days), with seven of the dyads completing data collection in five sessions and twelve dyads needing six sessions. Scheduling challenges became even more apparent during the COVID-19 pandemic, taking on average 35 days to complete the distance session as opposed to 19 days for the families who participated prior. Further, one mother completed tech training before pausing participation due to the pandemic and then resumed several months later. Importantly, this flexibility in scheduling is a strength of telehealth as it decreased burden on the time of the family and also allowed for a more representative picture of the child’s skills and behaviour while still allowing for complete data sets to be collected.

Discussion

Although this study is limited by a small sample size, the data support the utility of telehealth procedures in remote assessment in the FXS population, which is a rare condition. The data also provide support for not only the utility of telehealth for collecting a wide array of data type, but also that it was positively viewed by the parents involved.

We did not experience substantial missing data or the need to reschedule sessions due to technology-related challenges, and importantly, the data that were collected were of adequate quality to allow for transcription and coding of behaviours needed to address the primary aims of the project as regards to understanding the factors affecting child language development and parent–child interaction. We believe that this success was due to the implementation of a technology training session at the initiation of the family’s participation. During this session, we were able to troubleshoot a variety of technology challenges, including connectivity or microphone issues, environmental set-up, and in some instances, we discovered a need to provide technology to families. It is of note, however, that we did experience several instances in which sessions needed to be rescheduled due to changing availability in schedules or simply the parent’s perception of the child’s likely compliance. The study was further impacted by the COVID-19 pandemic and thus, several families needed additional time to complete the study due to challenges imposed by the pandemic itself. However, this flexibility to reschedule is a positive feature of telehealth delivery, as there was minimal time and no travel effort spent in needing to reschedule; however, this need for flexibility and rescheduling should be considered and accounted for when developing future telehealth protocols. Further, all of the participating families had their own Wi-Fi or hotspot. In studies involving less well-resourced families, it might be necessary for researchers to provide hotspots for use during the study.

Families expressed overwhelming agreement in the feasibility of participating in this type of research from a distance. Although the goal is not to fully replace in-person participation in research or clinical activities, the present findings suggest that telehealth does allow a pathway for many families who might not otherwise be able to or interested in participating in research or accessing onsite clinical services. Further, we were successful in implementing our procedures across a wide variety of device platforms. This is particularly important in that telehealth procedures are expected to continue evolving and, ensuring flexibility of devices and procedures while maintaining the core telehealth principles is paramount to expanded use of telehealth. It is of note, however, that the present sample of mothers represented a relatively technologically savvy group and thus, future research is needed to determine the feasibility of the procedures from this study for more novice users. Further, given the relatively middle- to high-income levels and levels of education of our sample, there is a need for additional information on how the procedures can help bring services to those that might not have as broad of an access to resources.

Implications for future research

Importantly, along with other studies documenting the utility of remote assessment, ²⁵ , ⁴³ this study provides a roadmap of how assessment data can be collected from a distance in families having children with neurodevelopmental disabilities. Although this study was limited to data collection primarily on the part of a cognitively able adult, we were successful in collecting observational data of mother–child interactions with little concern on the child’s part regarding our remote presence. Future work should expand upon remote data collection in this population by examining the utility of connecting directly with the child participant using the parent as a support person when needed. We also hope that findings from this study will help to ease the burden related to travel and participation in larger studies such as clinical trials and behavioural interventions by eliminating the need for onsite assessments. Future work in clinical trials might benefit from exploring the use of remote visits with the overseeing physician as well as staff charged with the collection of behavioural and developmental measures.