Case series of technology-aided interventions to support leisure and communication in extensive disabilities

Abstract

Objectives: This study assessed a new technology-aided program to support independent leisure and communication engagement with people with intellectual and motor or sensory-motor disabilities.

Methods: Six adults participated. The program relied on the use of a Samsung Galaxy Tab S2 LTE tablet with Android 6.0 Operating System, a NANO SIM card, and the WhatsApp Messenger and MacroDroid applications. The participants were presented with leisure and communication (i.e. text messaging) alternatives, and, based on their choices, with series of steps allowing them to access leisure events or send messages.

Results: During baseline, the participants were unable to access leisure events or send messages. During intervention and post-intervention phases, they used the program successfully and spent large percentages of the session time independently engaged in leisure and messaging.

Conclusions: A program based on readily available technology may be functional to support leisure and communication in people like the participants of this study.

Introduction

People with intellectual and motor or sensory-motor disabilities may be unable to engage in functional or leisure activities and may also have extreme difficulties interacting/communicating with others regardless of whether the others are in their immediate context or away from it (Badia et al. 2013, Lancioni et al. 2017, Sutherland et al. 2014, Taylor and Hodapp 2012). In practice, their disabilities may prevent them from performing the motor responses required to move and carry out the steps of functional activities, and to manage the use of the various devices available for engaging in leisure activities (e.g. television, computers, and musical instruments) (Dahan-Oliel et al. 2012, Federici and Scherer 2012, Lancioni et al. 2016, 2017). Their communication difficulties may be related to limited verbal skills and unavailability of alternative communication resources as well as the inability to use telephone devices or other devices for reaching distant partners (Hatakeyama et al. 2015, Lancioni et al. 2014b, McMillan and Renzaglia 2014, Van der Meer et al. 2012).

This situation creates concern within rehabilitation and care contexts. Resorting to the use of high levels of staff supervision is practically unrealistic given the general shortage of staff personnel in many of those contexts. Moreover, staff support would not promote initiative and self-determination with the consequence that people will remain dependent and fail to reach new developmental milestones (Foley and Ferri 2012, Lancioni and Singh 2014). The use of assistive technology is increasingly considered an efficient way to help these people progress in their occupation and communication (Chantry and Dunford 2010, Cullen et al. 2017, Hoppestad 2013, King et al. 2014, Lancioni and Singh 2014, Meder and Wegner 2015, Stasolla et al. 2013, 2015). For example, studies have been conducted with the use of virtual reality and computer-aided programs to promote leisure activities such as playing virtual ball games, managing the use of television, and choosing among different types of music or singers (Lancioni et al. 2014a, 2016, Stasolla et al. 2013, Wang et al. 2011, Yalon-Chamovitz and Weiss 2008). Similarly, studies have been conducted with the use of Speech Generating Devices and similar technology to promote basic communication (requesting) skills (Kagohara et al. 2013, Ricci et al. 2017, Schepis and Reid 1995, van der Meer et al. 2012, 2017a, 2017b) and computer-aided programs to promote independent use of telephone calls and text messaging (Lancioni et al. 2010, 2011, 2014b).

Recently, studies have been conducted, which relied on computer- or smartphone-based programs to support both leisure and communication activities. Indeed, providing the opportunity to freely shift from one type of activity to the other can be relevant to improve the quality of engagement and personal satisfaction. For example, Lancioni et al. (2017) carried out a study in which eight adult participants affected by intellectual disability as well as motor and visual or auditory impairments could choose among leisure and communication options (e.g. songs, sport videos, and telephone calls and/or text messages). The technology involved a pre-arranged computer with screen and sound amplifier, a mobile communication modem, and a microswitch. If a participant chose one of the leisure options, the computer would present a number of related alternatives so that the participant could choose the one he or she wanted to access. If the participant chose the telephone calls or messages option, the computer presented the names and/or photos of the partners available for calls or messages. Once a partner was chosen, the computer set up a telephone call with that partner or presented the messages available (i.e. among which to choose) for that partner. All participants managed to engage in both leisure and communication activities independently.

Lancioni et al. (2018) assessed a smartphone-based program with five participants who presented with intellectual disability and visual and/or motor impairments and limited verbal skills (i.e. participants whose verbal utterances were inadequate to correctly activate the smartphone language recognition functions; see Lancioni et al. 2014b). Given their inadequate speech, the participants were allowed to use mini objects or pictures fitted with frequency code labels for their leisure and communication requests. Two smartphones were available. The participants were to place the pictures or objects on the back of the smartphone nearer to them. This smartphone, which contained a special application, read the object or picture’s label and verbalized the request of the corresponding activity. The verbalization of this smartphone activated the S-Voice of the second smartphone, which then presented the leisure activity or placed the phone call, as requested. All five participants learned to use this program successfully and reached high levels of independent leisure and communication engagement.

While the results of the aforementioned studies are encouraging, new efforts may be required to develop a technology-aided program that (a) can support leisure and communication engagement in participants with intellectual disability, extensive motor or sensory-motor impairments, and lack of speech (i.e. participants who are generally underserved and would only partially or occasionally benefit from the aforementioned smartphone-based program), and (b) is based on advanced, readily available technology (i.e. is practically advantageous compared to the aforementioned computer-aided program involving a mobile communication modem and a microswitch). This study developed and assessed such a new technology-aided program. The program relied on the use of a tablet or a combination of a tablet and a smartphone and was applied to support independent leisure activities and text messaging for six adults affected by extensive disabilities.

Method

Participants

Table 1 lists the six participants by their pseudonyms, and reports their ages and their motor or sensory-motor impairments. All participants, who had congenital encephalopathy, presented with extensive motor impairment and were in a wheelchair. Four of them (Billy, Jerry, Audrey, and Kevin) had sufficient arm-hand movements to activate the proximity sensor of the tablet used for their program. Natalie and Anne had only head movements. The participants’ sensory conditions also varied. Billy, Jerry, and Audrey had typical sensory functioning. Kevin had severe hearing loss. Natalie was blind and Anne had minimal residual vision. The first four (Billy, Jerry, Audrey, and Kevin) were able to discriminate pictorial representations and pictorial scenes conveying simple messages concerning, among others, common activities, greeting and love, food and eating, and questions. Billy, Jerry, and Audrey were also known to understand verbal sentences conveying the same messages. A similar ability to understand a variety of verbal sentences was reported for Natalie and Anne. None of the participants had expressive verbal skills to communicate with others. Yet, they could (a) use a few sound utterances that were interpretable by staff (Billy, Kevin, and Anne), (b) fixate or point to pictures, picture combinations, or few printed words or symbols (Billy, Jerry, Audrey, and Kevin), or (c) produce ‘yes’ and ‘no’ head movements to respond to and choose among options/offers verbally presented by staff (Natalie and Anne). The participants attended rehabilitation and care centers and those centers were seeking support to help them engage in forms of independent occupation and communication. The psychological services of those centers had estimated the participants’ level of functioning to be within the mild to moderate range of intellectual disability, but no IQs were available for them. In fact, no formal testing could be carried out given the participants’ general conditions.

Table 1.

Participants’ pseudonyms, ages, and motor or sensory-motor impairments

Participants	Ages (years)	Motor or sensory-motor impairments
Natalie	23	Extensive motor impairment with only head movements; and blindness
Anne	27	Extensive motor impairment with only head movements; and minimal residual vision
Billy	56	Extensive motor impairment with arm-hand movements to activate the tablet’s optic sensor
Jerry	62	Extensive motor impairment with arm-hand movements to activate the tablet’s optic sensor
Audrey	45	Extensive motor impairment with arm-hand movements to activate the tablet’s optic sensor
Kevin	43	Extensive motor impairment with arm-hand movements to activate the tablet’s optic sensor; and severe hearing loss

The participants represented a convenience sample (Pedhazur and Schmelkin 1991) and were recruited based on three criteria. First, they were dependent on staff and caregivers for their leisure occupation and for communication with other people, particularly people not present in their immediate surrounding. Second, direct observations and staff interviews had indicated that the participants had clear preferences for leisure activities and people and were interested in (a) accessing those activities (e.g. watching videos with or listening to music, comedy sketches, sport events, religious hymns, and kitchen recipes), and (b) connecting with those people (i.e. family and staff members) directly and via messages handled by an intermediary. Moreover, they were happy when they received a message from those people. Third, participants and staff had shown interest in the use of a technology-aided program that would support independent leisure and communication engagement. All participants agreed to be involved in the study. Yet, they could not read or sign any consent forms so their legal representatives signed those forms for them. The study had been approved by a relevant Ethics Committee.

Setting, sessions, and data recording

Familiar rooms of the centers that the participants attended served as the setting for the study. Sessions were conducted on an individual basis, two to four times a day, with the participant sitting in a wheelchair and having a tablet in front or to the side. The sessions lasted 10 min or until any leisure activity or communication event started before the 10-min limit was terminated. Data recording was carried out by research assistants, with experience in technology-aided programs for persons with disabilities, and concerned: (a) the leisure activities accessed and the messages sent out and received independently, and (b) the engagement time for (duration of) those leisure activities and those messages. The engagement time for the activities and messages sent out also included the selection process (see below). Inter-rater agreement was measured in about 25% of the sessions of each participant. During those sessions, a research assistant and a reliability observer took part in data recording. Agreement required that the research assistant and reliability observer (a) recorded the same number of activities and messages and (b) reported total engagement times for activities and messages differing less than 1 min. Agreement was obtained in more than 90% of the sessions of each participant.

Technology

The technology consisted of a Samsung Galaxy Tab S2 LTE tablet with 8-inch screen, Android 6.0 Operating System, proximity sensor, multimedia player, and Google Assistant. The tablet was fitted with (a) a NANO SIM card for Internet connection, (b) the WhatsApp Messenger application, which allowed free message exchanges, and (c) the MacroDroid application, which served for automating the tablet’s functioning in line with the intervention and post-intervention conditions (see below). The tablet was also supplied with numerous multimedia files concerning the choice alternatives for leisure and messages with which the participants were presented, and the contacts for (telephone numbers of) the partners with whom messages could be exchanged.

Table 2.

Questionnaire items.

1. Do you think that the participant enjoys this program situation?

2. Do you think that the participant benefits from this situation?

3. Do you think that this situation improves the social status of the participant?

4. Do you think that this situation could be reproduced within daily contexts?

5. Would you like to be involved in the implementation of this situation?

The leisure activities covered the participants’ preferred topics (i.e. six per participant), which involved, among others, music, comedy sketches, kitchen recipes, circus shows, short stories, religious events, martial arts, and sports (see Participants section). Each topic included six related options (e.g. singers, comedians, clowns, and football). Each topic option contained four or five items (e.g. four or five songs of a specific singer or four or five comedy sketches of a specific comedian).

Each participant was allowed to exchange messages with six preferred partners (i.e. family and staff members; see Participants section). For any of these partners, four message topics (e.g. greetings/love, questions, and daily activities) were available. Within each topic, the participant could choose among four to six different messages (e.g. ‘I love you’, ‘when do you visit me?’, ‘what did you do today?’).

For Natalie and Anne (i.e. the two participants with only head movements), a Samsung Galaxy A3 smartphone with Android 6.0 Operating system, proximity sensor, multimedia player, and MacroDroid application was also available. The smartphone was fixed to their wheelchair’s headrest and served to detect (via proximity sensor) their head responses and relate those responses to the tablet described above. In practice, the smartphone substituted for the proximity sensor of the tablet that these two participants could not activate.

Experimental conditions and data analysis

The study was carried out according to a non-concurrent multiple baseline design across participants (Barlow et al. 2009). The baseline phase, which varied in length for the different participants, was followed by the intervention and post-intervention phases. The research assistants, who were involved in data recording (see above), were also in charge of baseline, intervention and post-intervention sessions. The study was followed by staff interviews about the technology-aided program used in the study (Luiselli et al. 2010). The participants’ baseline and post-intervention engagement times for leisure activities and messages accessed and managed independently were graphed as means per session over blocks of sessions. The Kolmogorov–Smirnov test was to be used for assessing engagement time differences between phases if data overlaps existed (Parker et al. 2011, Siegel and Castellan 1988). Actually, no overlaps occurred.

Baseline

The baseline included four to eight sessions. The participant was provided with the tablet whose functioning had not been automated via MacroDroid (as it was for the intervention and post-intervention phases). At every session, the tablet showed two folders on different background colors and verbalized the names of those folders (i.e. videos and messages) at intervals of about 45 s. If the participant wanted to engage in leisure activity, he or she was to touch the videos folder. In that case, the tablet presented folders and names of six video topics (e.g. music, comedy, and sport). Touching one of those folders led the tablet to present six topic options (e.g. singers and comedians). Touching an option led the tablet to present four related items (e.g. four songs of the selected singer). Touching one of the items led the tablet to play it for 1.5 min (i.e. a time considered satisfactory for the participants).

If the participant wanted to select a partner for a message, he or she was to touch the messages folder. In that case, the tablet presented images (faces) and names of six partners. If the participant touched one of those faces the tablet opened the WhatsApp chat for that partner so that the participant could formulate and send a message to him or her. The research assistant invited the participant to make a choice on (touch) the tablet at intervals of 1–2 min. If the participant did not make progress for about 4 min, the research assistant activated a leisure item or sent a message for him or her (showing and explaining the process).

Intervention

The intervention phase included 8 to 12 sessions during which the participants used the tablet that was automated via MacroDroid to fit the program conditions. Specifically, the MacroDroid was set up to control the procedural sequences available for the two choice areas (i.e. videos and messages), in relation to the participant’s activation of the tablet or smartphone’s proximity sensor (see below). Every session started with the tablet sequentially scanning (illuminating for about 5 s) the two pictures representing the two choice areas, and verbalizing each area while illuminating it. The participant could select either area by (a) approaching with the hand the proximity sensor of the tablet (i.e. Billy, Jerry, Audrey, and Kevin) or (b) turning the head to the side (i.e. Natalie and Anne) within the 5 s in which it was illuminated or 5 s from its verbalization. The head movement activated the proximity sensor of the smartphone fixed to the wheelchair’s headrest.

Figure 1 illustrates the sequence of procedural steps available if the participant initially selected the videos area as well as the sequence of procedural steps available if the participant initially selected the messages area. The sequences ended with the tablet (a) playing the leisure item the participant had selected for 1.5 min (see baseline) or (b) reiterating the name/face of the partner and the message the participant had selected and sending the message to that partner. Figure 2 provides an example of images appearing on the screen to illustrate six video topics following the participant’s selection of the videos area. Figure 3 provides an example of images appearing on the screen to illustrate four message topics following the participant’s selection of the partner to whom the message was to be sent. Images were always scanned individually and verbalized. Images were the only input for Kevin. Verbalizations were the only input for Natalie and Anne. Any choice response, as mentioned above, consisted of activating the tablet or smartphone’s proximity sensor.

Figure 1.

Flowchart illustrating the sequence of procedural steps for videos (leisure activities) and messages.

Figure 2.

Example of images appearing on the screen to illustrate six video topics following the participant’s selection of the videos area. The topics illustrated are: music, comedy sketches, religious events, sports, kitchen recipes, and circus shows.

Figure 3.

Example of images appearing on the screen to illustrate four message topics following the participant’s selection of the partner for a message. The topics illustrated are: food/eating, activities at the center, greetings/love, and questions.

Incoming messages were automatically presented by the tablet at the end of a sequence and before the start of the next. Incoming messages that arrived outside of a session were presented to the participant before the start of a new session. Kevin’s partners had been provided with over 30 pictorial messages, which were stored in their smartphone and could be used in their interaction with Kevin. Each message included the face of the partner sending it and a pictorial component that represented the message content (e.g. greetings/love, answer to Kevin’s question, food, and weather). The other participants’ partners were simply to write their messages, which were then read aloud by the tablet. Before reading the message, the tablet presented the name as well as the face of the partner sending it for Billy, Jerry, and Audrey.

During the initial intervention sessions, the research assistant provided verbal and physical guidance to help the participant practice the selection responses through the video and message sequences. In practice, the research assistant would verbalize and/or point to a choice area and then guide the participant to produce a response at each of the following steps to attain a leisure item or send a message. During the following sessions, the research assistant’s guidance was faded out and eventually eliminated to ensure that the participant was able to manage the sequences of both areas independently.

Post-intervention

During the 76 to 128 post-intervention sessions (with differences in numbers due to participants’ availability), conditions were identical to those in use at the end of the intervention phase. Research assistant’s guidance was not available during post-intervention sessions.

Staff interviews

Twenty staff persons (i.e. education, rehabilitation and care personnel) working with people with severe and multiple disabilities, but not connected to the study or participants, were involved in the interviews. The staff persons were 30 to 49 (M = 40) years old and represented a convenience sample (Pedhazur and Schmelkin 1991). They were divided into two groups of 10. Each group was asked to watch three 2.5 min video clips showing three of the participants (i.e. Audrey, Natalie, and Jerry, or Kevin, Billy, and Anne) while performing the sequences to access a leisure activity and send out a message. Research assistants considered those video clips representative of the participants’ typical performance. After watching the video clips, staff were to answer a five-item questionnaire (see Table 2). For each item, staff could provide a score of 1 to 5, which represented least and most favorable ratings, respectively.

Results

The six panels of Figure 4 summarize the baseline and post-intervention data for the six participants. The intervention sessions are not reported in the figure given that they served as propaedeutic (via guidance and guidance fading) to enable the participants to manage the program and access leisure and communication independently. The participants’ order in the figure reflects the number of baseline sessions available to them. The bars’ height indicates the mean percentages of session time that the participants spent independently engaged over blocks of sessions. The gray and black sections of the bars represent the mean percentages of time spent in the two areas (i.e. videos/leisure and messages/communication), respectively.

Figure 4.

The six panels summarize the baseline and post-intervention data for the six participants who are ordered according to the number of baseline sessions they received. The bars’ height indicates the mean percentages of session time the participants spent independently engaged, over blocks of sessions. The number of sessions included in the blocks is indicated by the numerals above them. The gray and black sections of the bars represent the mean percentages of time spent in the two areas (i.e. videos/leisure and messages/communication, respectively) within the aforementioned blocks of sessions.

During the baseline, none of the participants managed to use the tablet to access leisure activities or send out messages independently. In fact, they could not even receive any message (i.e. could not have any message read out or displayed visually), given that the tablet was not yet automated via MacroDroid. Consequently, their mean percentages of session time independently engaged were zero. During the post-intervention phase, all participants showed high levels of independent engagement. Their mean percentages of session time independently engaged varied between about 80 (Natalie) and close to 90 (Kevin). The mean percentages relative to videos/leisure varied between slightly above 25 (Anne) and about 60 (Kevin); those relative to messages/communication varied between slightly above 25 (Kevin) and about 60 (Anne).

Staff interviews showed that they were quite positive about the program. The mean scores (i.e. computed across the 20 staff persons) for the five items of the questionnaire varied between 4.1 (i.e. on whether the situation could be reproduced in daily contexts) and 4.7 (i.e. on whether the participant could benefit from the situation).

Discussion

The results indicate that the new program was successful in supporting independent leisure and communication engagement. Indeed, each participant learned to use it, irrespective of the disability condition, and thus could remain busy in an independent manner throughout the sessions. Moreover, the staff persons interviewed about the program were supportive of it, providing fairly high scores on all items. In light of these results, a few considerations may be in order.

First, the positive outcome of the study adds strength to previous data in the area showing the relevance of technology-aided programs to help individuals with intellectual and other disabilities reach independent leisure and communication occupation (Lancioni et al. 2018, 2017). The new study could also be seen as a relevant extension of previous research. Indeed, the new program seems to suit participants with different disability conditions (including participants who would not benefit from previous programs) and is based on readily available technology. The suitability of the program is in part connected to the fact that all choice situations were presented visually and verbally. Some participants were able to use both components. Others were able to use only one of them, but still benefited from it. The other reason for the suitability of the program is connected to the use of WhatsApp Messenger for sending and receiving messages. The WhatsApp Messenger presents two advantages, that is, (a) it makes the interaction between the participant and his or her partners free of costs and (b) it allows the use of pictorial material for communication. Pictorial material is critical to enable participants with hearing loss and different types of productive and/or receptive language impairments to communicate with their partners (Flippin et al. 2010, Levén et al. 2014, Li et al. 2013).

Second, the technology used in the study (i.e. tablet or tablet and smartphone) was selected based on two main principles, that is, availability and safety. Indeed, the tablet and smartphone are readily available devices. Similarly available are the WhatsApp Messenger and the MacroDroid applications, which are essential to arrange the messaging process and automate the functioning of both devices, respectively. Availability in this case, particularly as regards the Macrodroid application, should not be taken as an equivalent of operational simplicity. In fact, the use of MacroDroid requires a certain level of expertise that rehabilitation and care personnel may not always have. With regard to the question of safety, one specific point needs to be underlined. The goal in this study was to ensure that a device with Internet and/or Bluetooth connection would not be near the participants’ head. That explains why a smartphone was used as microswitch (i.e. without Internet or Bluetooth connection) to trigger the tablet, rather than as a substitute of the tablet, for Natalie and Anne. One last point to make here is that, although very effective in its connection with the tablet, the smartphone is not the only device that could be used as microswitch.

Third, the results of the interview carried out with the 20 staff persons are encouraging. In fact, they perceived the program as largely beneficial and pleasant for the participants and also indicated their desire to be involved in the running of the program. They also considered the program applicable within daily contexts. With regard to the applicability issue, a cautionary note may be in order here to help avoid failure and frustration. The note concerns the arrangement of the technology for the program. In light of the comments made in the previous paragraph (i.e. about the use of MacroDroid) and the fact that staff may not necessarily have specific technical expertise, it might be useful to seek the help of a consultant with the required technical knowledge to set up the program in daily contexts (Kuo et al. 2013, Plackett et al. 2017, Taherian and Davies 2018).

Fourth, several limitations of the study may need to be discussed here and possibly addressed in future research. The first limitation concerns the relatively small number of participants involved in the study. Such limitation imposes a certain level of caution in making general statements about the results reported and asks for direct and systematic replication efforts to determine the dependability of the program and its usability across situations and participants (Kazdin 2011, Makel and Plucker 2014). A second limitation concerns the lack of assessment of participants’ satisfaction. Regarding this limitation, two points can be made. On one hand, it may be argued that the availability of preferred leisure activities and the possibility of communicating with preferred partners were relevant conditions to make the participants enjoy the program. Actually, informal reports suggested that participants had signs of happiness (e.g. smiles) during the sessions (Dillon and Carr 2007). On the other hand, it is clear that a formal investigation of their satisfaction would have afforded concrete evidence on this aspect (Copeland et al. 2014, Lenker et al. 2013). Such evidence could have answered the question of whether the program played a role in improving the participants’ mood and quality of life (Brown et al. 2013, Verdugo et al. 2012).

A third limitation concerns the absence of an interview with the participants’ partners about their communication experience within the program. The partners’ opinion could be important to understand whether and how much the messaging (communication) events can enhance their interaction with the participants and eventually improve their attitude toward the participants (Lenker et al. 2013, Nicolson et al. 2012). Another possible limitation of the study is the absence of open questions in the social validation questionnaire. Open questions could help one attain an elaborate description of the program’s perceived benefits and suggestions about the steps to take for improving it.

In conclusion, the results of this study indicate that the new program was effective in fostering the participants’ independent leisure and communication engagement. These results confirm and expand previous evidence on the benefits of technology-aided interventions with participants with extensive disabilities. Before one can draw general conclusions on the evidence available, new research should rectify the limitations of the present study and determine practical strategies for ensuring a successful arrangement of the program (i.e. for dealing with the technology requirements) in daily contexts. New research may also find ways of modifying the technology, of making it friendlier for staff and participants.

Ethical Approval

Appropriate institutional approval and written informed consent were obtained for the study. All procedures performed were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the article.