Supporting Remote Participation when Designing with People with Dementia

Abstract

Exploring accessible remote design methods has become the need of the hour for supporting participation in research and collaborative design with individuals with dementia. Existing remote design approaches face specific challenges when facilitating best practices for co-design with participants with dementia. These challenges include, enabling sensory engagement with physical design materials and prototypes and observing these interactions in a natural manner. We present a system architecture and use cases for a portable system with a range of connected devices that support real-time, embodied design activities with individuals with dementia.

INTRODUCTION AND BACKGROUND

As the world navigates the COVID-19 pandemic, older adults have become a vulnerable population [6, 7, 12] and face increased health risks to participating in in-person research. With people with dementia, there have long been additional barriers to in-person research, such as difficulty recruiting [1, 23] and accounting for transportation needs and accessible physical facilities. These barriers might also result in the recruitment of a less diverse group of participants [26], as research may be restricted to populations within university areas. Remote interviews and design techniques (e.g., online forum, telephone, and video-based approaches) have been used by researchers to overcome logistical issues such as travel for geographically distant participants [3, 8] or when stigma might prevent participants from revealing their identity in person [17].However, these approaches pose barriers in light of our emerging understanding of best practices to support people with dementia in human-centered design practices. First, people with dementia experience changes in cognition and communication that make verbal-based human-centered design methods challenging to apply [2, 9]. Recent research identifies the importance of supporting embodied interaction and sensory engagement [13, 14, 19, 25] - which become more challenging to support in remote methods. Second, when involving people with dementia as co-designers, real-time interaction is important given changes that occur in short-term memory, which make recollection more challenging. To address these challenges, we propose a system to support remote synchronous interaction that moves design interactions “of the screen” onto a physical design space by using physical artifacts (e.g. paper prototypes). The proposed system and use cases are a synthesis of literature and observations from the authors’ preliminary research projects.

PROPOSED SYSTEM

We present an architecture for a remote collaborative design system (see Figure 1). The system integrates devices, such as mini printers and pico-projectors, that connect with a video conferencing web-app on the participant’s computer or tablet. The participant would be sent these devices as a “kit” that connects wirelessly or through USB to their devices. The researcher utilizes the web-app and the connected devices at the participants’ locations to facilitate a range of design activities. Our proposed system builds on top of commercially available video-conferencing systems to leverage video supported non-verbal aspects of communication such as reading face and body language [8] - modes of expression that are essential to attend to in dementia [11]. The videoconferencing component of our system is being built as a Node.js [20] web app using the Zoom Web SDK [27] and can be accessed on any device with an HTML5-capable browser. We are choosing to develop a web app rather than a native app to make our system usable across device and OS constraints. We plan to extend the capabilities of videoconferencing through connected devices. Past researchers have enabled remote design activities through software (drawing on digital whiteboards [8], taking, uploading, and responding to photographs [15]). Our system will utilize hardware to support accessible design activities in the physical environment of the individual with dementia. Devices such as portable pico-projectors, mini-printers, and overhead cameras will support design activities involving the sharing of different views (e.g. overhead view), producing physical artifacts for different activities (e.g. paper and 3d prototypes), or demonstrating prototypes. Custom event-based communication using Socket.IO [22] will allow the researcher to remotely trigger these devices to print, project, or stream video. We will select devices that can be controlled via API calls from the participant’s computer (e.g., CUPS [5] for printers) or are compatible with a Raspberry Pi micro-controller [21] intermediary. When using the web app to video conference, a modified version of the Zoom UI with a “Design Devices” window will be presented to researchers conducting the remote design session (Figure 2). This window allows researchers to search for available devices at a participant’s location (connected to the system shipped to them) and send, for example, a file containing the design of a prototype to be printed. Participants will see a traditional Zoom UI in their browser (see Figure 3). We leverage existing familiarity with Zoom in mainstream dementia groups (e.g. monthly online gatherings [10]).

Figure 1:

System Setup

Figure 2:

“Design Devices” window on researcher’s view shows participant’s printer and projector can be used for the session

Figure 3:

Traditional Zoom UI on Participant’s View

USE CASES

Based on prior literature and our experience conducting remote research with people with dementia (two co-design sessions: one remote and one in-person, and 16 remote interviews), we have identified the following use cases that demonstrate the potential of a remote collaborative design system:

Use Case 1: Facilitating Spatial Information Organisation

Changes in visual processing and memory, commonly experienced by people with mild to moderate dementia [4], can pose barriers when conducting remote studies. In our past work, we have repeatedly run into these barriers when conducting remote research with people with dementia. Participants often used split screen to be able to see interview questions and the full screen video of the researcher via video-conference. However, when they needed to utilize additional screens (e.g., to show us how they used certain websites), this caused stress and confusion as they were unable to manage screens that were not visible to them at all times. The envisioned system would assist by having information constantly available on a physical surface. In particular, supporting the researcher in printing study materials for the participant would allow for individuals to easily highlight or trace the progress of the interview questions - an approach that we have found to make verbal design methods more accessible for people with dementia. The projector in the system can also be used to keep the instructions visible and present example interfaces that can be dynamically updated through manipulating the interface on the researcher’s computer when using a “Wizard of Oz” method (Figure 4).

Figure 4:

Walking through an example interface using a pico-projector in a WoZ session

Use Case 2: Supporting and Capturing Natural Interaction

In a pilot, in-person co-design session, we noted how a participant with dementia actively engaged in the activity by using different paper based design materials such as markers and post-its. The visual and tactile feedback provided by the design materials play a key role in facilitating engagement. Our proposed system can create opportunities to support remote participants’ interactions with design materials. The mini-printer and 3D printer allow the researchers to print design materials and physical artifacts as needed. As an example, when using a card sorting approach [24] the connected mini-printer can be used to create physical cards with the information categories for the session (Figure 5) at the participant’s location.In addition to facilitating participants in naturally interacting with design materials in their own space, this system would enable researchers to better capture these interactions during remote design activities. In our past work attempting to understand technology use among people with dementia, we found most participants wanting to show us their use of their smartphone as it is the technology they use most frequently and rely most heavily on. However, it is extremely difficult to remotely observe participants using their phone due to the video-conferencing camera facing directly towards the participant. In these instances the overhead camera in our system could allow researchers to observe participants’ interactions with their devices (or design materials) in their work space and the challenges they encounter (Figure 6). This would be favorable than relying on participants to verbally explain their dificulties, which may not fully capture their experience.

Figure 5:

Creating physical design materials like paper cards using the mini printer

Figure 6:

Using overhead camera to observe participant’s natural interactions

DISCUSSION AND FUTURE WORK

The objective of our proposed system is to support design sessions for distributed teams composed of participants with dementia and researchers. By proposing a system architecture that physical devices can be “plugged” into, we can leverage advances from past research, for example, Macleod et. al’s asynchronous remote communities [16] and Mok et. al’s remote critiquing of prototypes using video conferencing [18]. Some of the questions that we plan to answer through future studies include:

• How do we make the system as usable by the participant with dementia as possible, while supporting a caregiver’s involvement when needed?

• How can we select design activities for participants’ particular functional abilities (e.g. verbal interaction becomes less accessible as dementia advances, while motor memory persists longer)?

• What is an optimal set of devices that would enable different types of studies while keeping the system portable and compatible with familiar technologies?

Some of the answers to these questions might depend on factors such as the participant’s stage of dementia, socioeconomic status, and their familiarity with technology. It is imperative to acknowledge that our system may restrict access to technologically savvy individuals, and to actively work to ensure the system is accessible and productive for all.