Feasibility and acceptability of an online response inhibition cognitive training program for youth with Williams syndrome

Abstract

Williams syndrome (WS) is a genetic neurodevelopmental disorder often accompanied by inhibitory difficulties. Online cognitive training programs show promise for improving cognitive functions. No such interventions have been developed for individuals with WS, but to explore the practicality of large-scale online cognitive training for this population, we must first investigate whether families of those with WS find these programs feasible and acceptable.

Twenty individuals aged 10–17 years with WS, along with parents, participated in a pilot online cognitive training program supervised in real time using videoconference software. We evaluated the feasibility and acceptability of this response inhibition training using three parent questionnaires.

Descriptive data are reported for the measures of feasibility and acceptability. Overall, the online procedures received a positive reaction from families. Parents were likely to recommend the study to others. They indicated training was ethical and acceptable despite feeling neutral about effectiveness. The frequency and duration of sessions were acceptable to families (two 20-to-30-min sessions per week; 10 sessions total). Families provided feedback and offered suggestions for improvement, such as more flexibility in scheduling and decreasing time spent in review of procedures.

1. Introduction

Williams syndrome (WS) is a genetic neurodevelopmental disorder caused by a microdeletion on chromosome 7q11.23 (Hillier et al., 2003), with prevalence estimates of about 1 in every 7500 births (Stromme, Bjornstad, & Ramstad, 2002). Most individuals with WS display mild to moderate intellectual disability and personal strengths in expressive vocabulary and auditory short-term memory (Mervis & Klein-Tasman, 2000). People with Williams syndrome are often “overfriendly” and often show significant attention deficits, high impulsivity, and non-social anxiety (Leyfer, Woodruff-Borden, Klein-Tasman, Fricke, & Mervis, 2006; Mervis & Klein-Tasman, 2000; Rhodes, Riby, Matthews, & Coghill, 2011).

1.1. Inhibitory control and Williams syndrome

Problems with executive function, especially ‘response inhibition’ (RI), are increasingly seen as a contributing factor to the observed behavioral phenotype in WS (Frigerio et al., 2006; Gothelf et al., 2008; Horn, Dolan, Elliott, Deakin, & Woodruff, 2003a; Mobbs et al., 2007; Porter, Coltheart, & Langdon, 2007). RI allows us to stop a movement or inhibit a stimulus-response association, and challenges with RI translate into difficulties stopping unwanted behavior and have been linked with impulsivity and attention problems (Horn et al., 2003; Mobbs et al., 2007). About 64% of those with WS meet criteria for ADHD (Leyfer et al., 2006) and show similar executive function characteristics as those with ADHD, including difficulties with impulsivity, inhibition of responses, concentration, and error recovery (Greer, Riby, Hamiliton, & Riby, 2013; Rhodes et al., 2011; Rhodes, Riby, Park, Fraser, & Campbell, 2010).

The hypersociability that is characteristic of those with WS involves excessive use of social engagement techniques (Reilly, Losh, Bellugi, & Wulfeck, 2004). Research suggests that impaired RI influences social disinhibition in WS and contributes to the overall behavioral phenotype (Carney, Brown, & Henry, 2013). The social disinhibition in WS may create social vulnerability related to overly friendly behavior (Fisher, Moskowitz, & Hodapp, 2013). Additionally, anxiety is highly comorbid in those with WS and is thought to be linked to executive functioning and poor inhibitory control (Derakshan, Smyth, & Eysenck, 2009; Woodruff-Borden, Kistler, Henderson, Crawford, & Mervis, 2010). The interplay of anxiety and inhibition in WS has yet to be fully understood, especially in WS, but it is possible that anxiety impairs the executive functions by making it difficult to shift away from a threat and inhibit an overly fearful response.

Based on this cluster of difficulties, poor inhibition may contribute to challenges with psychosocial functioning across many settings for those with WS. Although some research indicates spared inhibitory function in WS (Capitão et al., 2011; Costanzo et al., 2013), there is support for the position that impaired executive functions and social inhibition in WS make inhibitory control a potential target for improvement.

1.2. Cognitive training and internet-based intervention

Cognitive training involves methods that target improvement in brain processes such as memory, executive skills, and attention. Cognitive training has shown positive to largely promising results in other adult and child populations for a wide variety of cognitive skills (Amir et al., 2009; Cicerone et al., 2000; Karch et al., 2013; Klonoff et al., 2007). Notably, positive results were seen even in early studies designed to build inhibition skills, including in children with ADHD and other disabilities (Baer & Nietzel, 1991; Duckworth, Ragland, Sommerfeld, & Wyne, 1974). Expanding the intervention research that targets functions such as response inhibition for those with WS is an important next step, given that inhibition is crucial for academic and social success, particularly when social vulnerability is high (see Jawaid et al., 2012). Cognitive training may be helpful in targeting compromised processes in the WS brain (Greer et al., 2013); however, little is known about the feasibility or potential efficacy of delivering these online interventions to youth with this complex developmental disorder.

Recently, cognitive training has taken the exciting leap into online, computer-based delivery. The first FDA-approved, digital game-based treatment for ADHD was released earlier this year (EndeavorRx; 2020 Akili Interactive Labs, Inc). Further, telehealth delivery of services has exploded in the wake of the COVID-19 pandemic restrictions enforced in 2020, making online cognitive training programs even more relevant. Internet-based delivery of intervention increases efficiency and dissemination efforts by overcoming barriers including access, cost, time, and geographical location (Ybarra & Eaton, 2005). With regard to cognitive training programs specifically, positive results across age and population type are noted for an impressive spread of cognitive functions, most notably executive function, response control, and impulsiveness (Gagnon & Belleville, 2012; Slate, Meyer, Burns, & Montgomery, 1998).

It is important to consider that improvement sometimes does not generalize to untrained tasks (Simons et al., 2016). It is acknowledged that the literature does contain null results (Enge et al., 2014; Kable et al., 2017). Nevertheless, there is indication of promise for computer-based cognitive training, with documentation of a transfer to untrained skills (Loosli, Buschkuehl, Perrig, & Jaeggi, 2012; Thorell, Lindqvist, Nutley, Bohlin, & Klingberg, 2009; Walton et al., 2015). Further, some authors have reported therapeutic effects on everyday functioning in children with anxiety (Lee, Goetz, Turkel, & Siwiec, 2015). In recent research, Kühn et al. (2017) specifically examined effects of game-based inhibition training in older adults and found that cortical growth in areas associated with RI was directly related to time spent playing the inhibition game as well as improvement on a computer-based measure of inhibition.

1.3. The potential of online interventions for WS

Given the promising effects of online cognitive training programs noted above, there is now a strong basis for extending such training to targeted clinical populations. Online cognitive training for individuals with WS would be an innovative way to improve skills and decrease impairment in everyday life. Large-scale research targeting such populations is nearly impossible due to logistic barriers, but online-delivered programs could be an acceptable and feasible solution. For one, they would be easier to disseminate to a clinical population spread across the country or the world. In addition, such a program could be cost-effective in using free or inexpensive video conferencing tools and requiring less staff and space than traditional intervention programs. Further, children and adolescents may be more drawn to a computerized task that could be made game-like or competitive in nature. This could boost adherence, acceptability, and even enjoyment. If acceptable to families, internet-based delivery options could open doors to intervention beyond RI training.

1.4. The need for studies of feasibility and acceptability

With this vision of improved functioning in WS and enabling greater access via online delivery of interventions, we have begun testing the very first internet-delivered cognitive training for individuals with WS: a program targeting response inhibition in WS youth. We acknowledge that any potential intervention must be accompanied by investigation of specific factors that influence improvement in RI as a result of cognitive training, such as adherence or the feasibility and acceptability of the training program (Owen et al., 2010). Studies of feasibility are a necessary step in refining interventions to improve acceptability and the likelihood of effectiveness, and they are an important prerequisite to larger studies of efficacy (Bowen et al., 2009; Fletcher et al., 2016). To improve adherence and recognize the full potential of an intervention, feasibility and acceptability studies of online training, not just for the WS population but for the general population, are of critical importance (Kaltenthaler et al., 2008). For studies involving youth, understanding parental acceptance, adherence, and engagement is pivotal to intervention development prior to designing larger study trials (Haine-Schlagel & Walsh, 2015).

Feasibility or acceptability assessments have been included in studies of many computerized or internet-based interventions, including programs targeting coping with traumatic stress, anxiety, or depression, building manualized therapy skills, and boosting adherence (Botella, Serrano, Baños, & Garcia-Palacios, 2015; Kassam-Adams et al., 2016; Khanna & Kendall, 2010; Rathus, Campbell, Miller, & Smith, 2015; Spirito et al., 2015; Wenze, Armey, & Miller, 2014). Some computer-based treatment has assessed satisfaction or sought program evaluation from participants, finding positive ratings and high levels of satisfaction and acceptability, including in children with ASD and intellectual disability (Benyakorn et al., 2018; Botella et al., 2015). Other researchers first assessed usability of an intervention before piloting with a small sample, allowing for tailoring of the program and collection of favorable ratings of usability and acceptability (Stark et al., 2016). These studies provide examples and guidance for evaluating feasibility of internet-delivered interventions.

1.5. Summary and rationale

Response inhibition difficulties seem to underlie a cluster of symptoms in WS, and these are manifested through disinhibited behavior, high rates of ADHD, and trouble with shifting or disengaging from threats. While some emerging computer-based interventions target response inhibition, there is little research on either computer-based intervention for WS or on response inhibition in WS. To help those with WS benefit from the newly-emerging computer-based intervention programs that target cognitive processes, we must first determine whether it is feasible and acceptable to apply internet-based, self-administered cognitive training to this population.

1.6. Study aims

This purpose of this study is to examine the feasibility and acceptability of online cognitive training targeting response inhibition improvement in youth with Williams syndrome. The larger pilot study (Brei, 2017) included a cognitive assessment, evaluation of RI functioning and changes in RI, the cognitive training program itself, and parent report of therapeutic effects on everyday functioning, but these results lie beyond the current paper's scope. The current study provides quantitative and qualitative information derived from parent report measures to inform future computer-based cognitive training endeavors for those with WS and possibly for other populations with genetic or developmental differences.

2. Method

2.1. Participants

Participants were 20 children and adolescents with WS (8 females), aged 10–17 years, and 20 parents (one ‘primary’ caregiver alongside each child). The youth had all been genetically diagnosed with Williams syndrome previously, the first and main language spoken in the home was English, and families possessed a computer with internet access as well as a second electronic device capable of video conferencing. Exclusion criteria were: legal adult, previous inhibition training, and comorbid diagnosis of ASD due to possible interference with interpretation of results. No minimum IQ was required, as we sought to use a representative sample of IQ in children with WS. Fliers announcing the larger pilot study were mailed to families seen by the principal investigators in prior studies of WS. Fliers were also distributed by the Williams Syndrome Association via email to attendees of the Williams Syndrome Conference and to families near several Midwestern metro areas. A description of the study was posted on the Williams syndrome Research Registry. The broader pilot study was submitted to the online registry of Clinical Trials. Interested families were instructed to contact the principal investigators to complete a screening form and arrange participation. See Table 1 for a characterization of the sample.

Table 1.

Demographics.

Participant demographics
Population	20 children/adolescents with Williams syndrome, and primary caregivers
Gender	12 males, 8 females
Age	Mean: 14 years, SD: 1.9 years, Range: 10–17 years
Intelligence quotient	Mean standard score: 67; SD: 16; Range: 44–97
ADHD comorbidity	n = 16 (80%): 13 Inattentive type; 3 combined type

This research was conducted with the understanding and written consent of each participant and his or her parent, according to ethical principles outlined by the Institutional Review Board. This research was independently reviewed and approved for ethical soundness by the Institutional Review Board within the Human Research Protection Program at the institution conducting this research. Families were compensated with a total of $40 per child upon completion.

2.2. Measures

Treatment evaluation inventory-short form (TEI-SF; Kelley, Heffer, Gresham, & Elliott, 1989). This is a Likert scale with 8 statements related to treatment acceptability, child consent, discomfort, and overall reaction. Parents rate from “strongly disagree” to “strongly agree.” Higher scores indicate greater agreement. This measure has good psychometric properties. It was administered via online parent self-report immediately after the training program was complete.

Treatment acceptability questionnaire (TAQ; Hunsley, 1992). The TAQ is 7-point Likert scale consisting of 6 questions related to acceptability, effectiveness, and ethics of the treatment and the knowledge and trustworthiness of the providers. Higher scores indicate a more positive rating. This measure was administered via online parent self-report immediately after the training program was complete. While the TEI-SF focuses more on treatment procedures and likability, the TAQ incorporates feedback derived from interactions with the study staff and feelings of ethicality.

After treatment questionnaire (ATQ; Brei & Klein-Tasman, 2016). This is a study-specific measure of feasibility and acceptability created by the principal investigators. It consists of 22 questions about technological preferences, program flow, session length and number, and level of staff involvement. It includes open-ended questions to gain parent feedback. See Table 2 for further description of items included in these three measures.

Table 2.

Description of measures.

Measure	Items/description
Feasibility and acceptability
Treatment evaluation inventory—Short form (TEI-SF; Kelley et al., 1989)	Parents rate on a Likert scale their responses to items assessing the following: acceptability of the program, willingness to use this procedure to target behavior change, acceptability without child consent, likability of procedures, expectations of discomfort, likelihood of improvement, parent-enforced use of the program, overall reaction to the program
Treatment acceptability questionnaire (TAQ; Hunsley, 1992)	Parents rate on a Likert scale their responses to items assessing the following: level of acceptability, degree of ethicality, level of effectiveness, perceptions of side effects, degree of provider knowledge, degree of provider trustworthiness
ATQ (Study-specific questionnaire created by principal investigators)	Parents rate on a Likert scale their responses to items and provide optional comments for the following: phone or email preference, comfort using the study's videoconference tool, alternative videoconference preferences, problems, and comments, ease and independence in computer setup, staff assistance with technology and training, session number, length, and effectiveness, ideas for improving study or staff, more rigorous sessions and child fatigue, likelihood of participation if session frequency or length increased, particular likes or dislikes, staff correspondence and involvement, supervision vs. independent completion, suggestions for improving clarity, likelihood of recommending to others and why
Training and evaluation	Experimental Tasks used for RI assessment and training measures
Go/No-Go task (Lee, 2014)	The 8-min computerized Go/No-Go task adapted for this study consists of 75% go trials requiring a button press after each stimulus (alphabet letter) and 25% no-go trials. It assesses response time and inhibition after the discriminatory stimulus (the letter ‘X')
Motor stroop task (Lee, 2014)	The 10-min computerized Motor Stroop Task adapted for this study requires participants to press a particular button depending on which side of the screen the stimulus appears
Stop-signal task (Lee, 2014)	The 10-min computerized Stop Signal Task adapted for this study instructs participants to press computer keys indicating whether an arrow on the screen points left or right. Participants were to inhibit the response if the stop signal (audible beep) followed the appearance of the stimulus
Response inhibition training program (Lee, 2014)	Computerized training program developed for this study is a game-like computer program tapping into motor inhibition and interference control within an engaging story line. An introduction and ‘practice level’ is administered prior to training. Participants complete three 5-min “levels” per session, and levels incorporate various demands on inhibitory requirement and include feedback. Inhibitory demands become more challenging as participants progress. Adaptations were designed with consideration of the lowered IQ present in WS and the potential for frustration stemming from inability to meet passing criterion. All participants received approximately the same ‘dosage’ of training

Note: The larger pilot study included baseline assessment of cognitive ability, executive functioning, and comorbidities. Additionally, at each time point, parents completed six online questionnaires pertaining to everyday functioning in areas of attention, anxiety, affect, question-asking frequency, and emotion regulation.

2.3. Procedure

While the present study focuses on the feasibility and acceptability of internet delivery of the cognitive training intervention, the larger study's phases are briefly described here to demonstrate the steps involved in this program's development and to help readers understand participants' experience, which informed questionnaire responses. All phases of the study were delivered online except the initial in-person meeting. See Fig. 1 for study flow.

Fig. 1.

Study flow chart. This displays the sequence of training procedures and questionnaire administration. In-person assessment and RI task practice were followed by baseline assessment and training/wait, crossover, and Follow-up. The feasibility and acceptability measures were completed immediately after the training and at Follow-up. Notes: The Response Inhibition (RI) Task and Parent Questionnaires were administered at each time point. The feasibility and acceptability measures (Treatment Evaluation Inventory-Short Form and Treatment Acceptability Questionnaire; TEI-SF and TAQ) were completed immediately after the Post-Training/Crossover assessment. The After Treatment Questionnaire (ATQ) was completed at Follow-up.

Part I: Baseline (2 h total). Study staff met with participants in person to administer a standardized measure of cognitive functioning and a diagnostic interview. Study staff assisted parents in computer setup for the study either in person or via videoconferencing equipment. Participants then met with staff from home via videoconferencing equipment to complete an online, computerized RI assessment lasting about 1 h (see Table 2 for a description of the experimental tasks). Parents also completed an online questionnaire battery (six brief standardized measures) about child mood, behavior, attention, and anxiety. After Baseline, participants were randomly assigned to the Treatment or Waitlist Crossover condition of the cognitive training program, stratifying by intellectual functioning.

Special adaptations for acceptability in WS. To facilitate the acceptability of the RI assessment for participants with WS and their parents, the computerized RI assessment was first introduced to one volunteer child with WS and adjusted based on staff observations and volunteer feedback. Given that intellectual functioning in this population falls in the mildly to moderately delayed range, adaptations to increase participant understanding and acceptance included the addition of detailed instructions presented visually with accompanying audio, teaching of correct responding using the computer keyboard and mouse, and the addition of practice trials. Study participants were then administered this standardized 5-min ‘practice’ to ensure that instructions were understood. In allowing for a supervised, standardized practice for all participants, we hoped to improve treatment acceptability by increasing familiarity with the task demands and reducing frustration, misunderstanding, or forgetting. All participants were supervised until they demonstrated an understanding of the goal of the task, and behavioral observations (including mood and practice progression) were recorded for each child.

Part II: Treatment or waitlist crossover (6–7 h total). The game-like cognitive training program utilized in this phase was developed for a recent series of clinical trials studies and showed promise for improving RI in individuals with disorders characterized by poor RI (Lee, Espil, Bauer, Siwiec, & Woods, 2018). See Table 2 for a description of the training program.

Participants in the Treatment condition completed ten 20- to 30-min sessions of training (about two per week) over the course of about 5 weeks. Time spent training was 15 min total, with practice, pauses, and review built into the session. They were then administered the 1-h RI assessment tasks that had been administered at Baseline. The Waitlist Crossover group waited during those 5 weeks, were administered the RI assessment tasks, completed crossover training, and then were administered the RI assessment tasks once again. Parents repeated the questionnaires about child psychosocial functioning and attention after the wait and after training.

The first two training sessions were supervised by study staff via videoconferencing equipment, and all videoconferencing activity was securely recorded to ensure proper administration. Parents supervised the child at all times but were instructed not to assist the child during the RI assessment or training game. Brief breaks that did not interfere with administration were allowed. For cases in which participants did not respond to a program-generated email prompt to complete a training session, study staff reached out to families via phone to troubleshoot barriers.

Adaptations to improve acceptability for WS. The cognitive training game was adapted from the original version used in clinical trials. It was tailored specifically for youth with WS based on the volunteer participant's feedback and on typical levels of cognitive functioning and ADHD comorbidity in WS. The main adaptation involved adjusting passing criteria to provide more positive feedback for smaller gains, considering the lower IQ present in WS and the potential for frustration if passing criteria were too difficult. Additionally, before each of the ten training sessions, there was a brief review and practice of the training game procedures.

Part III: Follow-up (45 min). Three months after completion of training, each participant completed the Follow-up assessment in which they were again administered the computerized RI assessment tasks after a brief review. Parents again completed the questionnaire battery on child functioning. The Follow-up session occurred as the final phase of the study for all participants.

Assessment of feasibility and acceptability. The TEI-SF and TAQ parent report measures were administered online to parents immediately upon their child's completion of the training, before Follow-up. At Follow-up, the final feasibility and acceptability questionnaire specific to this study, the After Treatment Questionnaire (ATQ), was sent to parents to complete online anonymously, with the option for open-ended comments (refer to Table 2).

3. Results

Results focus on descriptive data from the responses on each of the parent-reported feasibility and acceptability measures. Responses are reported as positive if the parent indicated a more positive than neutral response on each scale. Only one family dropped out of treatment before the completion of the study, just before Follow-up.

3.1. Attendance and program completion

Out of 20 participant families, there was a 100% completion rate for the 10-session training program. There were 19 families who completed the entire program, including the 10 training sessions plus the RI assessment at each time point; one participant did not complete the 3-month follow up RI assessment. Feasibility and acceptability questionnaire completion was similarly high, with 85% and 90% of parents completing the TEI-SF and TAQ measures, respectively, immediately after training had ended, and 70% of parents completing the ATQ at the Follow-up assessment.

3.2. TEI-SF

Results from the Treatment Evaluation Inventory (n  = 17), on a 0–4 acceptability scale, indicate that parents had a favorable response to the intervention. The majority of parents (71%) agreed or strongly agreed that they liked the procedures (M = 2.76, SD = 0.75, where 0 is “strongly disagree” and 4 is “strongly agree”), and most (76%) had a positive reaction to training (M = 2.88; SD = 0.93). 41% of parents agreed or strongly agreed that this intervention was an acceptable way of treating the presenting issues, while 47% reported feeling neutral (M = 2.47; SD = 0.94). Only a smaller subset (12%) thought the training would likely result in permanent improvement; the majority felt neutral (M = 1.94; SD = 0.75). See Fig. 2 for more details about particular item responses.

Fig. 2.

Parental ratings on TEI-SF. This displays key parental responses on items of the treatment evaluation inventory-short form, including responses about acceptability, procedures, perception of improvement, and reaction to training.

3.3. TAQ

Results from the Treatment Acceptability Questionnaire, a 1–7 acceptability scale, (n  = 18) indicate a positive response with regard to acceptability, with 72% of the respondents finding the intervention at least slightly acceptable (39% reported the training was “very acceptable”) (M = 5.44, SD = 1.50, where 1 is “very unacceptable” and 7 is “very acceptable”). The strong majority of parents reported that the intervention was very ethical (M = 6.34; SD = 1.20) and the providers were very knowledgeable (M = 6.34; SD = 1.04) and trustworthy (M = 6.61; SD = 0.85). On average, parents were neutral regarding its effectiveness (M = 4.44; SD = 1.42). See Fig. 3 , which displays all parental responses to items.

Fig. 3.

Parental ratings on TAQ. This displays key parental responses on items of the Treatment Acceptability Questionnaire, including responses about acceptability, ethics, effectiveness, and provider knowledge.

3.4. ATQ

The After Treatment Questionnaire (n  = 14) provided anonymous parental perceptions and feedback informing both feasibility and acceptability, the training process, use of technology, email vs. phone communication, staff involvement, training program difficulty, and participant comfort with various procedures. It allowed parents to provide feedback and suggestions for modification and improvement.

Positive feedback. Parents were enthusiastic about the engaging nature of the 10-session training program. Many reported being pleased with the playful, game-like nature of the training tasks and commented on the engaging graphics. They also noted that staff engagement with the child (e.g, regular encouragement and social reinforcement) boosted the child's enjoyment and engagement during study procedures. They were positive about the study framework and design. They reported feeling comfortable with the videoconferencing tools used and with the use of video throughout the session for supervision and interaction. They appreciated having staff walk them through program installation and setup. Most preferred email communication from staff, which was the primary modality used, as opposed to phone calls. They affirmed the level of staff involvement/supervision provided, and most believed it necessary and helpful to have constant staff supervision during the first two sessions. Parents noted that it was helpful to have flexibility about the time of day their child completed unsupervised training sessions. Some reported feeling excited at seeing their child's improvement during the study. Encouragingly, most parents indicated that they were likely to recommend the study to another family.

Suggestions for modification and improvement. Parents offered several ways to modify the training program for future use. Some were in favor of expanding video conferencing options to include those which were already downloaded or embedded on their device. They also asked for greater flexibility in scheduling the supervised training sessions. Some parents commented that the review and practice of training game procedures before each session should be briefer. A few parents described that more rigorous sessions would be manageable and may result in greater improvement. Some reported issues with videoconferencing (e.g., audio/video lag) or difficulty with computer setup. They felt that limiting the number of parent questionnaires during the study would be helpful, and they suggested having a shorter RI assessment, as this step was repeated multiple times. Parents also offered ideas for making the procedures even more child-friendly, in particular connecting with the child through informal conversation at the start of any supervised session. Most parents would not have supported more rigorous (longer, more frequent, or more intense) sessions, as they felt their child would have been too fatigued, although over half would still have been willing to participate. See Table 3 for results from the ATQ and participant responses to the open-ended questions.

Table 3.

Results from after treatment questionnaire (n = 14)^a.

Relevant question	Percent	Relevant responses in open-ended comment space
Preferred email	71
Comfortable using study video chat	100	•Video chatting was a very easy and smooth way for us to use and communicate. We did not encounter problems
Prefer alternative video chat options	36	•We did not have any issues but another method would be fine
Audio/video problems	36	•We had a few sound glitches but they were solved pretty quickly •There was a bit of a delay, and often there was a background hum
Easy setup	50	•In scheduling, include a few day and evening times to have us pick from •No surprises. Everything was outlined and explained in detail
Difficult setup	29	•Positioning camera to pick up correct video [was hard]
Prefer study staff help with setup	79	•Everything was outlined and explained in detail
Prefer staff supervision	86	•Spend a bit more time chatting with the child before jumping in
Would not support more sessions	64	•There were more than enough sessions •I think the number was about right. It took a bit to become comfortable with it •I think it may have been a little more effective with a few more sessions
Would not support longer sessions	71	•They were long enough. For my son it seemed to tax his attending skills •I think shorter sessions are definitely better. Frustration levels peak if any longer •I don't think my son would have been as cooperative •They were long enough to be productive, but any longer and his attention would have been lost. It was very hard to keep him on task
Would still participate if more or longer sessions	57	•It would be interesting to see 3 ×/3wk results •I thought the 20 min sessions were a good length. Maybe add another session or two
Believed child would fatigue with more rigorous sessions	86	•[It was] too repetitive, easy to get bored and/or frustrated •It wasn't difficult to comply, and if it can help other Williams Syndrome families with children with attention issues, it's very worthwhile! •My son got bored with it but to me (not as video game savvy!) •I would say 10–12 sessions of equal or shorter duration. Twice per week worked well
Assistance was helpful (vs. doing all sessions alone)	50	•[Research staff] were very friendly and professional •[My child] responds well to positive reinforcement so that was something she hoped would happen during the testing. That cheering on from the staff is a wonderful thing
Overall: Would recommend this study	86	•It surprised me that [my child] learned to inhibit his response. I didn't expect an effect •We enjoyed it. I was impressed with the improvements [my child] made. It opened my eyes to see how much she wanted to know her scores. Even though she was frustrated when she missed a cue or made a mistake, she was happy in the end •I would recommend this study because it allowed us to add input to a good cause. It was easy to do and the time commitment was flexible and convenient •It was fun for my daughter to play, and our family is always eager to help researchers •She got a kick out of the noises and music in the game—who thought of that one? •I thought it was explained well. I didn't realize I would be repeating the parent surveys so many times–that's the only thing that surprised me a bit, but it wasn't a problem •We wouldn't recommend this study, because we haven't noticed improvements •I would be eager to see findings from a larger number of participants. I am also curious as to how shorter attention spans influence tasks. Also curious about how you tease out attending difficulties vs. impulse control. I think my son knew what to do and thought about controlling the impulse but his mind would wander during the activities •If it could help you learn how to help our kids I'd recommend it

Note: This table includes all participant responses providing specific information in the comments.

^a70% response rate.

4. Discussion

These results use quantitative and qualitative data to describe the acceptability and feasibility of an online intervention aimed at improving response inhibition in children and adolescents with Williams syndrome. That a study is feasible means it involves procedures that assist researchers in developing formal interventions and can include investigation of constructs such as acceptability, implementation, practicality, and adaptation, to name a few (Bowen et al., 2009). Describing acceptability involves examining possible consumers' perceptions of study procedures and involves constructs such as attitude toward treatment, ethicality, and perceptions of effectiveness (Kazdin, 1980; Sekhon, Cartwright, & Francis, 2017). In this study, feasibility reflected the investigators' ability to deliver the program as intended through the use of videoconference-based supervision, online software for RI assessments and the training game, and online questionnaires and interviews to record parental assessment of child functioning at various time points. Acceptability referred to the degree to which parents reported the study procedures were acceptable, understandable, ethical, and helpful.

Encouraging feedback about the training program's acceptability and procedures was obtained. In particular, parents were likely to recommend the study to others. This overall inclination to endorse the treatment to others is promising for future success of online cognitive interventions, as it indicates parents felt the program would be worthwhile to other families. This likelihood of recommendation was present despite neutral feelings as to whether the training would result in lasting improvement. Low expectations of treatment effectiveness may indicate the need for study staff to have placed more emphasis on possible results, as perceived effectiveness of an intervention is important in establishing its acceptability (Sekhon et al., 2017). Timely completion within the prescribed schedule (2 ×/week for 5 weeks), the ability of WS participants to grasp procedures and complete the tasks, and positive parent feedback about time spent on the procedures all support the study's feasibility. Very low dropout levels and perceptions of ethicality in this study support its acceptability (Sekhon et al., 2017).

There was a high level of completion of study procedures, with 100% of participants completing the training program itself and only one participant failing to complete the final follow-up assessment of RI. This gives a general indication of study feasibility, along with parent feedback about the ease of delivery and completion through online videoconference and training game software. The high completion rate is also likely related to frequent, active email or phone check-ins by study personnel to remind families to complete sessions if a session had not yet been completed as expected during the week. With a low attrition level for the program overall, the absence of some parents' completion of the final measures is not believed to reflect a lack of engagement in treatment or a problem with acceptability. Further, most parents were enthusiastic about participation, which implies there was not an unwillingness to complete the feasibility and acceptability measures.

Almost 90% of parents would recommend this study to others, which is promising given that this study was the first of its kind in this population and has much potential for refinement. With these encouraging results, it is hoped that researchers will continue to explore and improve online training programs for future use with children with Williams syndrome.

4.1. Feasibility and acceptability of specific study features

Computer-based delivery and videoconferencing software. The majority of parents thought computer setup was easy, and about half appreciated help with setup, but some parents did consider this step to be difficult. It may be helpful to create a walk-through installation video that parents can view, as this step became time-consuming to troubleshoot via videoconference. For consistency, our study's procedures allowed only one option for videoconferencing software. However, given that about one-third of parents would have preferred using an alternative or more familiar videoconferencing tool than the one used in this study, it seems that families would be more comfortable if conferencing were offered via the software they already have installed on their electronic device.

Training schedule. A large majority of parents believed their child would have fatigued if sessions had been more rigorous. It is possible that more rigorous sessions themselves would not cause fatigue if the child were not required to complete the online RI assessment three to four times depending on treatment group. While most parents did not believe more or longer sessions would have made training more effective in improving skills, they were not averse to participating in a study involving more or longer sessions. This indicates that many parents are willing and open to explore the possibilities in order to further the research that may benefit their child.

Staff involvement. Stark et al. (2016) suggest that treatments may be enhanced if providers regularly monitor progress and check in with the family. With all parents reporting that study personnel involvement was sufficient, it would be interesting to explore how much researcher supervision and assistance is actually necessary for families to remain satisfied. This could help put the training in families' hands and increase dissemination all the more.

4.2. Innovation, limitations, and future directions

This examination of feasibility and acceptability of an online cognitive training program was the first study of its kind in the child and adolescent Williams syndrome population. Such a technology-based and internet-delivered cognitive training program for clinical populations is relatively new. This study helps promote research in this area, which may help to improve cost-effectiveness and dissemination to populations with rare conditions through easier access to home-based delivery. Further, this study demonstrates that cognitive training programs can be adapted in a feasible and acceptable way for individuals with WS and their families—and potentially to others with developmental conditions or similar IQ. Because every assessment in the current study was supervised and observed live via videoconferencing software, staff were able provide encouragement to participants and families, improve understanding and motivation, assist with technical problems, make general observations, and collect feedback to make adjustments, all which likely influenced the acceptability of the procedures and inform future research.

The limitations of this small-sample study are acknowledged, as is the difficulty of gathering a substantial number of children with a rare genetic condition. However, as noted by Bowen et al. (2009) and Fletcher et al. (2016), it is helpful to begin with a limited number of participants in order to gain candid feedback and inform establishment of a feasible delivery method. It is noted that this sample consisted of families who possessed adequate technological resources, which may not reflect the general population. Additionally, no guidelines exist for study procedures in those with genetic or neurodevelopmental conditions, such as optimal training length or spacing of cognitive training sessions. Future endeavors may aim to determine the optimal amount, quality, and intensity of training and explore whether a condensed time frame improves acceptability. Researchers may find Ritterband et al.’ (2003) steps for developing web-based interventions as a helpful resource.

Limitations of the feasibility and acceptability measures include that formal, written feedback was only collected after children completed training, which omits feedback from parents and children who dropped out of the study (n  = 1) or did not complete all parent measures, as described in Results. Because parental feedback was kept anonymous to encourage transparency, it was not possible to determine whether demographic characteristics or comorbidities influenced perceptions of acceptability.

Formal feedback was only obtained from parents and not from participants themselves. Parental feedback about feasibility and acceptability of interventions is a necessary focus, as parent support of an intervention is critical to the use of that intervention for a child (Stahmer & Pellecchia, 2015). Further, parent participation affects child attendance and engagement in treatments (Haine-Schlagel & Walsh, 2015). Staff observations about participant engagement indicated that children were generally accepting, willing, and even excited to play the training game, and several parents noted their child's reactions on the ATQ. At the same time, children and adolescents with WS could likely provide feedback to developmentally appropriate questions about the ethics and acceptability of such programs and their level of willingness to participate. Future studies should provide a wider range of technology and videoconferencing options, examine the role of parental involvement in the intervention and results, include formal, written participant feedback from youth, and track changes in parental perception of treatment acceptability over time.

5. Conclusion

This study helps to establish the feasibility and acceptability of an online cognitive inhibition training program for children and families with Williams syndrome. Though parents did not anticipate lasting cognitive or behavioral changes, feedback from parents was largely positive and provided information about expanding accessibility and flexibility of treatments for their children. It is hoped that these results will prompt researchers to develop, improve, and disseminate online interventions for children with WS and other developmental disabilities.