Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2023 May 22.
Published in final edited form as: ASSETS. 2022 Oct 22;2022:38. doi: 10.1145/3517428.3544809

Mobile Phone Use by People with Mild to Moderate Dementia: Uncovering Challenges and Identifying Opportunities

Mobile Phone Use by People with Mild to Moderate Dementia

Emma Dixon 1,*, Rain Michaels 2, Xiang Xiao 3, Yu Zhong 4, Patrick Clary 5, Ajit Narayanan 6, Robin Brewer 7,*, Amanda Lazar 8
PMCID: PMC10202486  NIHMSID: NIHMS1865459  PMID: 37283282

Abstract

With the rising usage of mobile phones by people with mild dementia, and the documented barriers to technology use that exist for people with dementia, there is an open opportunity to study the specifics of mobile phone use by people with dementia. In this work we provide a first step towards filling this gap through an interview study with fourteen people with mild to moderate dementia. Our analysis yields insights into mobile phone use by people with mild to moderate dementia, challenges they experience with mobile phone use, and their ideas to address these challenges. Based on these findings, we discuss design opportunities to help achieve more accessible and supportive technology use for people with dementia. Our work opens up new opportunities for the design of systems focused on augmenting and enhancing the abilities of people with dementia.

Keywords: Dementia, Mobile Phones, Accessibility

1. INTRODUCTION

Dementia is a condition which involves changes in cognition and affects the ability to engage in daily tasks and activities [74]. Dementia is typically progressive, with people in more mild stages of dementia experiencing less changes to their functioning than those in more advanced stages [74]. Many everyday technologies are not designed to meet the access needs of individuals with dementia [32, 3638, 79]. At the same time, research shows a trend towards greater mobile phone use by people with mild cognitive impairment and mild dementia, with almost half using smartphones [42, 57]. The rising usage of mobile phones and the documented barriers that exist lead to the opportunity to study the specifics of mobile phone use by people with dementia. This usage, as well as barriers to use, are key to understand in order to design apps and features that are useful and accessible for people with dementia. In this study we address the following research questions:

  • For what purposes do people with dementia use their mobile phones?

  • What challenges, if any, exist with mobile phone use?

  • What opportunities do people with dementia envision to support them when they encounter challenges with their mobile phone use?

Through semi-structured interviews with fourteen people with mild to moderate dementia, we learned that individuals used mobile phones in everyday life not only to accommodate changes in cognitive ability and emotional regulation, but also to stay productive and manage their health. We uncovered three major challenges with mobile phone use: 1) navigating to apps and features; 2) task execution in moments of high stress, fatigue, and time pressure; and 3) re-learning task flows after updates and upgrades. To address these challenges, participants described ideal interactions with mobile phones, including customizable user interfaces, activity-based customization, proactive technology assistance, and extended modalities for voice-based interactions.

Based on these findings, our work makes four primary contributions to the literature. First, this paper provides an empirical account of how fourteen people with mild to moderate dementia use mobile phones and the challenges they face with mobile phone use. Second, it describes participants’ ideas to address challenges with mobile phone use. Third, abstracting from participants’ ideas for future more accessible interactions with mobile phones, this paper discusses design opportunities to help achieve more accessible and supportive technology use for people with dementia. Finally, this work opens up new opportunities for the design of systems focused on augmenting and enhancing the abilities of people with dementia.

2. RELATED WORK

The following section describes research on mobile phone use by neurodiverse users. Following this, we describe research on barriers to general technology use for people with dementia as well the provision of technical support when people with dementia experience challenges with technology use.

2.1. Mobile Phone Use by Neurodiverse Users

Researchers have begun to investigate how neurodiverse users, such as those living with traumatic brain injury, Down syndrome, and autism use mobile phones. For example, researchers have found individuals with traumatic brain injury heavily relied on their phones for reminders out of concern for forgetting upcoming events and that they only used limited features as the more advanced features, which may have been of use, were considered too complex [25]. Research with individuals with developmental disabilities broadly, including people with Down syndrome, autism and other unspecified developmental disabilities, found that mobile phones played a key role in safety [28], increasing independence [28], providing entertainment [58], social connectedness [28, 58], and reminders and scheduling support [29]. Researchers have noted specific phone features which can be barriers for individuals with developmental disabilities (e.g., small buttons and complex menus [28]), as well as how features can be better designed to support neurodiverse users (e.g., large buttons, icons with titles, and a single level menu structure for individuals with traumatic brain injury [73]).

Past researchers have designed and developed prototype apps for tablets and mobile phones to support people across the stages of dementia (e.g., to promote safe walking with GPS tracking on mobile devices [46, 61, 100, 101]). Apps have also been designed to support self-care and health management by people with mild dementia [40, 52, 54]. Much of this past work focuses on the design and evaluation of new prototype applications to support users with mild to moderate dementia in important aspects of life. Given the increasing number of people with mild dementia now using mobile phones in their daily lives [42, 57], there is an opportunity to understand how to design for people with dementia from another angle: by studying existing use.

2.2. Barriers to Technology Use by People with Dementia

Past research has identified barriers that people living with mild to moderate dementia experience with technology use, such as the cost of devices [56], ethical issues [39, 56], attitudinal aspects [39, 56, 84], condition-related challenges [39, 56, 66, 78, 85], and technology-related challenges [39, 56, 78].

In regards to specific attitudinal aspects, researchers have argued that people with dementia face difficulties with using technology due to low digital self-efficacy (belief in one’s capacity to execute technology-related tasks) [6, 39, 78, 79]. Two studies compared digital self-efficacy of people with mild dementia to older adults without dementia, finding participants with mild dementia perceived using technology to be more difficult than participants of the same age [66, 85]. This low digital self-efficacy may impact the uptake of new, potentially useful devices [18].

This low digital self-efficacy may be due, at least in part, to changes that dementia brings which include memory, sensitivity to stress, orientation to place and time, and interpreting and understanding information [78]. Past work investigating barriers to technology use largely focuses on cognitive challenges [39, 56, 66, 78, 85]. Though researchers have also described the unique changes to sensory abilities that can cause barriers to technology, such as changes in speech patterns and language patterns and difficulty identifying different sounds [30]. In addition, researchers have pointed to how the progressive nature of the condition affects technology use [39, 56, 84].

Past research has also investigated difficulties people living with dementia experience due to the design of various technologies, such as websites [35, 43, 55], devices for in-home monitoring and support [9, 12, 18, 21], and computers and cell phones [78]. These difficulties include challenges with familiarity [18], conspicuous devices [18, 21], and complex interfaces [40, 43, 55]. Unlike past work with other neurodiverse users [25, 28, 58], little work has investigated the accessibility of mobile phones for people with mild to moderate dementia. One exception reports difficulty some tech-savvy participants with mild to moderate dementia have typing on their mobile phone due to small key size and identifying notification ringtones and sounds [31]. With this limited understanding of the technologically-related challenges people with mild to moderate dementia experience, in this work, we set out to investigate the specific barriers people with mild to moderate have with mobile phone use. Understanding these barriers is essential to designing future systems to provide the necessary technical assistance to combat these barriers.

2.3. Technical Support for People with Dementia

With the challenges that people with dementia face with technology use, research to date has largely discussed ways informal caregivers can provide technical assistance [27, 37, 53, 54, 68, 72, 77, 84, 88]. Past work found that while assistance from others can be useful, it was not always desirable by people with dementia, with individuals going to great lengths to avoid assistance from other people to avoid burdening loved ones and to avoid others taking away tasks that they had difficulty with [32]. Some participants in this study suggested that technical assistance mediated by technology may be more desirable than human assistance. Researchers have proposed one alternative to provide necessary technical assistance without relying on caregivers - automatic personalization (e.g., [97, 98]) in combination with AI to determine instances when an individual may need assistance and automatically provide this assistance or make suggestions according to the perceived need [31]. Although this past work suggests alternative solutions to provide technical support to people with dementia, researchers have not yet investigated how people living with mild to moderate dementia envision the provision of technical support. In this work, we investigate the perspectives of people with mild to moderate dementia on the optimal provision of technical support when challenges arise with mobile phone use and future opportunities for technological support.

3. METHODS

To understand how people with mild to moderate dementia are using mobile phones (RQ1), the specific challenges they have with mobile phone use (RQ2), and opportunities for technology to support people with dementia when they encounter these challenges (RQ3), we conducted semi-structured interviews with fourteen people with mild to moderate dementia. Below, we present our approach to recruitment, procedures for data collection, participant demographics, analytic approach, and limitations.

3.1. Recruitment

Participants with mild to moderate dementia were recruited through convenience sampling and snow-ball sampling. The first author reached out to 20 potential participants with a brief email describing the study and the eligibility criteria. People were eligible to participate in the study if they self-reported a clinical diagnosis of mild to moderate dementia, owned a mobile phone, and used their mobile phone daily. Seventeen potential participants responded to the email from the first author expressing interest in participating in the study. Three of these potential participants who initially showed interest in participating in the study were not comfortable with the interview being recorded and therefore chose not to participate. The other fourteen potential participants consented to participate electronically, and completed a demographics questionnaire. Aligning with best practices when working with people with mild to moderate dementia [44] as well as legal and ethical best practices in some countries [59, 93], we assumed participants’ capacity to consent. Although people in the mild to moderate stages of dementia can experience changes in cognitive ability [74], they are generally able to participate independently in research studies [44]. All interviews were conducted remotely in November and December of 2021.

3.2. Procedure

We conducted remote, semi-structured video interviews that were split into two segments of questions. The first segment focused on participants present mobile phone use and any challenges they experienced. The second segment focused on opportunities for future mobile device interactions. This is in response to Lewis, Sullivan, and Hoehl’s call to include individuals with cognitive disabilities in the design and development of future more accessible mobile phones [60]. Speculative futuring in the dementia space has primarily used co-design methods, where people with dementia use sketching and paper prototyping to ideate future technologies [61]. Due to changes in dexterity, other researchers have sketched participants’ design ideas as participants describe them, with the participants actively critiquing the designs being drawn [14]. However, researchers have found these low-fidelity prototyping methods are not always effective for people with dementia who sometimes prefer to verbally describe their design ideas over sketching [33]. For this reason, in the second section, we first asked participants to verbally describe their ideas to accommodate the challenges disclosed during the first section of the interview. We used ideas participants shared in earlier interviews to engage later participants in discussions about future mobile phone use.

Past work has also shown how using high-fidelity prototypes helps participants with mild to moderate dementia grasp abstract concepts even with changes in abstract thinking ability [33, 45]. For this reason, after participants had shared their ideas for future technologies, we used a publicly available Android smartphone application as a technology probe [48], to inspire participants to think of new kinds of technology to support their needs and desires. This app was chosen as the technology probe because it was designed for neurodiverse users with cognitive disabilities to provide a simplified user interface with customizable buttons which enable multi-step tasks with a single click (see Figure 1). We demonstrated some functions of the technology probe which we believed may be of specific use to people with dementia, such as providing directions, making phone calls to specific people, using smart-home devices, and opening specific YouTube Channels. See Figure 1 for a visual of the mobile phone home screen demonstrated during the interview as the technology probe.

Figure 1:

Figure 1:

An Android phone home screen showing customizable single click buttons for common multi-step tasks specific to the individual. Customizations include personalized images, labels, and sizing.

The semi-structured nature of the interview allowed us to ask further probing questions to pursue topics guided by the informants themselves. Following each interview, participants received a $75 gift card as an incentive. Interviews ranged from 47 to 67 minutes (average = 54 minutes). The interviews were audio and video recorded, resulting in 13 hours and 27 minutes of data. We provide the full study protocol, including interview questions, in the supplementary materials.

3.3. Participants

The research team conducted 14 semi-structured interviews with people with mild to moderate dementia. This aligns with the average sample sizes for remote interview studies [19]. Table 1 provides more information on participant demographics and mobile phone use. All participants were familiar with voice-assistants and some of their functionality. All participants resided in the U.S. with one participant, Miranda, residing in Canada. Throughout the paper we use pseudonyms for participants.

Table 1:

Participants’ Demographic Information

Pseudonym Gender Age Group Education Racial or Ethnic Group Mobile Phone Operating System Regularly received assists from others with mobile phone use
Thomas Man 55-64 Bachelor’s degree White iOS Yes
Elenora Woman 55-64 Master’s degree White Android No
Kim Woman 45-54 Bachelor’s degree Black or African American iOS No
Tristen Man 65-74 Some college credit, no degree White Android Yes
Sylvia Woman 55-64 Some college credit, no degree Black or African American Android No
Preston Man 55-64 Some college credit, no degree White Android No
Miranda Woman 55-64 Some college credit, no degree White iOS Yes
Kennith Man 65-74 Some college credit, no degree White iOS Yes
Tina Woman 65-74 Master’s degree White iOS No
Malcolm Man 55-64 Master’s degree White iOS Yes
Josslyn Woman 65-74 Some college credit, no degree White Android No
Hall Man 65-74 High school diploma or the equivalent White iOS Yes
Alecia Woman 55-64 Some college credit, no degree White Android No
Sabella Woman 55-64 Some college credit, no degree Black or African American iOS Yes

3.4. Analysis

We used a thematic analysis approach to analyze the interview data [16]. To become familiar with the data, the first author verified computer-generated interview transcripts from audio recorded interviews. The first author then coded each transcript to generate initial codes. The first author grouped initial codes into potential themes and went back through all interview transcripts to gather quotes relevant to those themes. The research team reviewed and discussed themes to ensure that they were relevant to the codes and quotes extracted. The first author created a thematic map, grouping themes under each of the three research questions of the study. The names of the themes within the thematic map were then refined to more clearly describe the findings in each section. Finally, the research team selected the most vivid and compelling examples that related back to the research questions to include in this report.

3.5. Limitations

This work has demographic limitations. First, most participants (10/14) most likely had early onset dementia, representing 9% of dementia cases world-wide [1]. This relatively younger group of participants may be overrepresented in our research due to the contacts we recruited from and the recruitment material language: “we’ll discuss how you use your mobile phone in daily life, any challenges you have with mobile phone use, and your ideas for ways to make technology use easier for you.” As one study showed, technophilia - high enthusiasm for new technologies - was associated with lower age of people with dementia [42]. The second demographic limitation is the lack of people of color. With research showing a higher prevalence of dementia in non-white people [2] and only three participants identifying as Black or African American, these demographics were underrepresented in our sample. The third demographic limitation is that participants resided primarily in the United States. Findings from this study are therefore limited in terms of geographic and cultural settings, which affect healthcare access, socioeconomic status, and network device coverage.

This study was also limited by the one-hour semi-structured interview method, which was chosen to minimize the time commitment and amount of work required of participants. This choice limited the scope of the data collected, as opposed to a longitudinal study design which could provide several weeks for participants to report on mobile phone use and future ideas. Further, we choose to conduct interviews, not co-design sessions, as past work has demonstrated that some people with dementia may experience difficulty sharing sketches over video-conference calls [33]. Instead, we relied on participants’ verbal explanation of their future ideas, where a UX Designer on our team later illustrated these ideas in Figma to include in this report.

4. FINDINGS

Through our interviews we learned of the reliance, desire and interest participants with dementia had with their mobile phones and the individually meaningful activities they used their mobile phones for. Even with the heavy use of and reliance on their mobile phones, we also uncovered challenges participants had with navigating to apps and features; time pressure, high stress and fatigue affecting task execution; and difficulty re-learning task flows after updates and upgrades. To address these challenges participants outlined their ideal interactions with mobile phones, providing opportunities for the design of more accessible mobile phone interactions for people with dementia. Below we elaborate on each of these findings.

4.1. Mobile Phone Use

All participants described their reliance on their mobile phone, emphasizing how they “can’t live without this phone” [Malcolm]. Further, mobile phones enhanced participants’ quality of life, as they described them as “their best and most famous friend” [Kennith] and their “brain” [Thomas, Tristen, Preston]. Most participants described their reliance on the mobile phone to support “memory issues” [Thomas] they experienced affecting their executive functioning skills, leading Kennith to refer to his mobile phone as “the way I cope with my daily existence.” Their reliance on the phone for scheduling and reminders of everyday activities resulted in several participants sharing the adage: “if it’s [an activity or event] not on the phone, it’s not going to happen” [Preston]. Other participants used their phones as a security measure for moments when they struggled to remember where they were. As Hall described, the phone is “something that I have to have with me all the time if I go someplace. That way, [Hall’s wife] can keep track of me and/or call me, or I can call them. So, it’s a safety catch for me.”

4.1.1. Overall Mobile Phone Use.

Participants described their use of their mobile phones for diverse and individually meaningful activities. They described their use of apps for social media, navigation, online shopping, mobile banking, reading the news, listening to music, games, entertainment, productivity, and health management. Please see the appendix for the full list of apps participants reported using. This list does not represent all uses of the mobile phone by participants, only those that participants were able to recall during the interview. Participants’ specific uses of their mobile phones for purposeful work and health management were particularly interesting and unexpected. Therefore, we describe them further below.

4.1.2. Mobile Phones Facilitating Purposeful Work and Connection.

Many participants described the difficulty they had after receiving a dementia diagnosis and subsequent retirement. Malcolm described his experience as feeling like there “was such a void in my life… I have to be busy. I don’t sit around much because I feel if I sit around, I’m not productive and I like to be productive.” Participants described ways that their “phone is the key factor” in “keeping myself pretty busy” [Tina].

All participants described the importance of their calendar to support general productivity through connecting with others and keeping track of daily tasks. A few participants described productivity in relation to helping others through advocacy work. As Miranda described “I do a lot of one-on-one support things with other people with dementia. So quite often I do video calls on my phone with someone who is in need.”

Still others worked on more social and creative activities, which they considered to be purposeful part-time jobs. For example, Thomas works part-time as a freelance court reporter. Sylvia “use[s] my phone a lot on Facebook live… I show people how to make natural herbal lip balm, and I tape the whole thing.” Malcolm started to co-host a podcast, which he posts online from his phone. He explained how his podcast “is sort of my salvation. It’s the thing that without it I think I would probably be very depressed because it’s the one thing that I so look forward to” [Malcolm]. Staying productive through these various methods was as much about enjoyment for participants as it was about “keep[ing] my brain active” through “something to prepare for… I do it all pro bono, but it’s that little job that I have. It is purposeful work, and it keeps me going” [Malcolm].

Participants used their mobile phones to support general productivity, meaningful hobbies, and for more creative activities - all providing purposeful work for participants.

4.1.3. Mobile Phones Making Health Management Easier.

Participants also described using their phones for health management activities. For example, some participants used their phone for medication reminders, asking Siri to “remind me to take my medication every day at 7:00. So, I use that feature and it’s extremely helpful with making sure I take my medication” [Kim]. For Miranda these medication reminders included specific days and dates because “I often struggle with what day we’re on, what the day and date is,” which is important for specific medications she takes.

Participants described how valuable it was to be able to use apps on their phone to keep track of medical results [Josslyn, Sabella] and to collect medical data [Thomas]. Thomas, who is living with type 2 diabetes, demonstrated his use of his “glucose monitor, which I use with my phone” where all he has to do is place the phone up to his left triceps to check his blood sugar. This data is then uploaded to the cloud so physicians can access this information, sparing Thomas from “hav[ing] to stick my finger all the time. They were just able to share the data and see on a day-to-day, hour-to-hour way all my blood sugar.”

The phone was also used for recalling medical information during doctor’s visits. Elenora described how during doctor’s visits they “always ask the same standard information about your medications and surgeries and all this stuff. And I can just pull it up on my phone… and I can share it with the doctors” [Elenora]. This included notes on her phone with “specific links to my prescription list document my supplements document” [Elenora].

Participants also described how valuable the phone is to call emergency medical services. For instance, when Miranda’s “phone is upended or shaken too severely, it will send out an SOS1 and they will actually come on and say, ‘are you okay? Do you need help?”’ In addition to this feature, Miranda has used her phone “to Call 9112 to have an ambulance come for me.”

Participants used their mobile phones for medication reminders, keeping track of medical results, collecting medical data, to recall medical information during doctor’s appointments and to be able to call 911 in emergency medical situations.

4.2. Challenges with Mobile Phone Use

Although participants described their reliance on their mobile phones and used their phones for a variety of individually meaningful activities, they also described the challenges they had with navigation, task execution, and relearning changed interfaces after updates and upgrades.

4.2.1. Difficulty Navigating to Apps and Features.

Participants described experiencing challenges with navigating to apps and features on their mobile phone. Thomas described this as “maneuvering through the phone” where “as it becomes more useful I think it can become more challenging because there’s more stuff squeezed in there and finding it all and maneuvering through it all can be a challenge.”

Participants often knew exactly what they wanted to find on their mobile phone but had difficulty “sift[ing] through the vast amount of information… zoom[ing] down into it” to get to what they were looking for [Thomas]. For example, several participants described their difficulty with “navigating the contact lists” in high stress environments [Miranda]. Preston described how “A lot of people with dementia would forget to scroll. They would think I only can see what’s on that page… Somebody with dementia that gets overwhelmed or finally realizes they’re lost. All a sudden your anxiety is so high. It’s like, ‘Where do I make a phone call? Where’s the number?”’ We can understand this example through the lens of affordances - an interaction element that tells us what we can do with it [75]. Affordances are binary - they either exist or they don’t - and contextual, meaning if we cannot perceive or understand an affordance because of our situation, it does not exist. In the example Preston describes where high anxiety makes scrolling to find a phone number in his address book challenging. For Preston, there is no identifiable indication of how to interact with the phone in those moments, though Preston is able to understand how to do this in other contexts when he is less anxious.

In other instances, participants described difficulty navigation to specific applications and content on their mobile phones. For example, Sabella described an instance when her daughter

“made a post about me on her Snapchat… it was a beautiful post about living with dementia and how she feels about me living with dementia. And so, I was trying to go back to find it to save it and I just could not understand the concept of doing that… I wanted it so bad, but I just couldn’t figure out how to do it [get back to the Snapchat post]. So, I lost it which was the most frustrating for me.”

Participants described difficulty with navigating to apps and features due to the amount of information available on their phone and challenges with remembering where items were saved on their phone.

4.2.2. Time Pressure, High Stress and Fatigue Impairing Task Execution.

Participants experienced challenges executing the necessary steps to complete tasks with precision on their mobile phones though they understood the necessary steps to complete tasks (i.e., the Gulf of Execution [76]). This challenge was primarily described in instances of time pressure or when participants experienced moments of high stress and fatigue.

Participants described feeling time pressure for tasks, which made them more difficult to execute with precision. For example, when inputting calendar events on their mobile phone, participants described experiencing “the pressure of the person that you’re doing business with kind of standing there going ‘Why is this so hard for you?”’ [Alecia] as well as the “people behind you” in line [Josslyn]. In some instances, this time pressure can lead to the event not “get[ting] on the calendar and I’m thinking, I thought I did it. And I think I’m missing that last click” [Josslyn]. In other instances, this time pressure led participants to input the exact month and time of events incorrectly, therefore scheduling events “in the wrong month” and consequently “show[ing] up at wrong appointments at the wrong time because I’ve screwed them up” [Alecia].

Participants also described challenges with executing tasks in moments of high stress and fatigue. For example, Elenora, who lives in a large city, described a time when after a long day of errands she “was very cognitively exhausted and I was waiting for a specific bus to come home”. This was the only way she knew how to get home. But, the bus she was waiting for was delayed for three hours, leaving her sitting at the bus stop into the night. Although Elenora was able to check on her phone to see the bus continued to be delayed, she “didn’t have the cognitive wherewithal to figure anything else out,” meaning use her phone to search for other alternative ways to get home. Similarly, after a stressful day of work and navigating transportation in a large, unfamiliar city, Thomas described how on his phone he was “looking at this map of these train stations and it’s just not making any sense. And all I could think of was the train somehow jumped the tracks into a different line and they’d taken me to the wrong place.” He explained this as “a situation where our brains aren’t working right,” making it difficult to execute routine tasks, such as navigation, on their mobile phones. This created high stress situations for participants who were aware of the increased risk of the harm of being lost for people living with dementia [86, 104].

In these instances of high stress and fatigue, participants were either not willing to ask for help from other people or had difficulty communicating with others. Elenora described how in these moments “I’m certainly not going to ask a stranger on the street because when you get in a fog like that, as friendly as I tend to be and willing to ask for help, when I’m that deep in the fog, I really enclose in on myself.” Thomas was willing to ask for help, but when he “asked the conductor, he was very gruff, and made me feel stupid… and look[ed] at me like I was crazy” but his directions just were “not computing to me.” In this instance, Thomas called his husband who drove to pick him up, describing how “people with challenges need the ability to reach out to a human even more than” other people. Thomas is referring to reaching out to significant others who understand his situation and can provide sympathetic assistance rather than people, such as the conductor, who do not understand his situation and therefore cannot provide the necessary assistance.

Participants also described high stress interactions with voice-based systems which impaired them from executing tasks on their mobile phone. For example, Malcolm describes using his bank’s interactive voice response system regularly to pay his bills. However, these systems do not always understand Malcolm where the system will say “I didn’t hear you.” This could be due to changes in speech patterns that people with dementia experience [71, 82] such as slowed speech, a developed stutter, and greater pauses between words. Preston posits this may be in part due to participants asking “question[s] and say[ing] the wrong word or the wrong adjective sometimes by mistake” [Preston]. In these instances, Malcolm then responds by “yelling and screaming because they don’t hear my voice,” which led to “a very high level of frustration… and I end up hanging up the phone because it’s horrible.” This example shows that execution of tasks on mobile phones may be impaired due to voice assistants not understanding their verbal commands.

Participants described challenges with executing tasks on their mobile phones when under time pressure and in moments of high stress and fatigue, such as after a long day trying to navigate home or when interacting with mobile phone voice-based systems.

4.2.3. Difficulty Re-learning Task Flows After Updates and Upgrades.

Similar to past work [50], participants expressed the challenge it was to re-learn tasks flows after updates and upgrades because “this is the new path I have to go through to do the things that I’m used to doing. I’m not to the point where I can’t do it, but it definitely can give me pause when a program changes or updates” [Thomas]. For Tristen, upgrades were difficult because “nobody gives you instructions that these are the changes that have taken place. It just happens on the phone.” He explained that “sometimes I get very anxious and frustrated, when changes [updates] take place.” Notably, participants did not describe barriers with updates and upgrades in relation to changes made to software and/or interfaces in order to correct accessibility barriers, as described in the Web Content Accessibility Guideline [49]. Instead these barriers were more so due to the change itself, rather than the substance of the change, as well as the fact that these changes are not explicitly called out by the system.

Several participants expressed fear over upgrading their phones to a newer version. Tina described how she has “always kind of updated or upgraded as needed. But now with my Alzheimer’s, I think that if I got another phone that had more fancy things that might make it more difficult… I don’t try to start all over again and get a whole brand-new phone” because she was “afraid I’m gonna mess up and then it’ll screw up my brain.” Malcolm shared this fear: “it’s a five-year-old [phone]. I’m afraid to get a new phone because I’m afraid I’m not going to know how to use all of this stuff. So, I keep the phone. It works fine.”

Participants also described not wanting to switch the operating system they used because of the difficulty they experienced with learning new systems. Josslyn explained that when she retired “I just kept [previous operating system] because I knew how to use it and I didn’t want to have to relearn everything.” One participant, Preston, intentionally got a new phone with a larger screen and memory “with the hopes I probably never have to get a new phone again” as he plans for this phone to last him the rest of his phone use.

Updated, upgraded or switched operating systems of their mobile phones complicated discovery for participants, forcing them to relearn mobile phone skills, which led to frustration and fear for some participants.

4.3. Participants’ Ideas to Address Challenges with Mobile Phone Use

In this section we describe participants’ ideas to address challenges with mobile phone use, including customizable user interfaces, activity-based customization, proactive technology assistance, and extended use of voice-based interactions. These design opportunities are based on feedback provided during the speculative design portion of the interview, where participants first articulated their ideal interactions with mobile phones and then saw the technology probe demonstration.

4.3.1. Customizable User Interface on Mobile Phones.

To address challenges with navigating their mobile phones, participants wanted to customize the size of apps and the icons so that they could more easily recognize icons. For example, Thomas wanted “to sort of mold the device to what you need.” Molding the device included making app icons that are “recognizable… easy for me to identify, even when I’m at my worst with my brain fog with my dementia” [Miranda]. This also included “hide[ing] all those other things that we don’t use or don’t want” [Miranda] and placing only the most used “app[s] on the home screen” [Thomas]. Malcom describes how he needs “less options. I don’t need 90% of the things [apps]… Because I think that’s where I have problems, when there’s a lot of options. Limit the options so I’m not searching for things.”

Relating to the challenges with re-learning task flows after updates and upgrades, participants wanted a pre-specified easy version of their phone rather than having to make user interface customizations themselves. Josslyn proposed the idea of being able to “choose an easy format” where modifications were “set for you” making the phone similar to “these phones advertised for seniors with the simplicity things.” Though when asked if she had considered buying and using one of these phones, Josslyn explained how she would prefer “just a simplified version of my phone” because this is what she is familiar with.

4.3.2. Activity-based Customization on Mobile Phones.

As another way to address challenges with navigating their mobile phones, participants described their need to bundle common activities - meaning to create specific categories of activities - and store these bundles in distinct, easily identifiable places. For example, Josslyn described how she “had to have it [her calendar] compartmentalized” where she “save[s] it [her phone calendar] for doctor’s appointments, or hair appointments or something that I’m leaving the house for. Not just like to remember to do the laundry… routine things are put on the paper calendar.” Josslyn made this modification to her calendar use after her diagnosis as she described “when I worked I had all that” on her phone calendar, “but once I had to simplify my life that’s how I simplified it - paper calendar, phone calendar.”

This idea of bundling common activities and storing them in distinct places was applied to several envisioned futures for mobile phone use. First, after seeing the technology probe could make a phone call using a single click button to a pre-specified individual (a common feature on devices geared towards older adults and people with dementia [72, 90]), Tina described her desire for a “my tribe” button to “push the button that would have the name and the phone number of those seven people that are in my tribe so that I don’t have to look it up” in her contacts. Both iOS [5] and Android smart phones [38] provide this service through groups.

In another example of activity-based customization by bundling common activities, one participant, Sylvia, described her desire to present voice-assistants as support for specific aspects of life. For example, Sylvia wanted her phone to “have five buttons” each with a different “personal assistant… one is for doctor’s appointments. One is to help you with the grocery store. One is to [remind you to] take medication.” Practically this would be one voice-assistant (e.g., Alexa, Google Assistant, Siri) but presented as different types of assistants to help with different bundled aspects of their life. We found this concept compelling, and so proposed it in later interviews to get other participants’ feedback. This idea was well received, with several participants proposing different personal assistance (e.g., life coach assistant to support chores [Alecia, Josslyn], and technical support [Kennith]).

Participants described their desire to have activity-based customization by bundling common activities and storing these bundles in distinct, easily identifiable places to address challenges with navigating their mobile phones.

4.3.3. Proactive Technology Assistance on Mobile Phones.

To assist in moments of pressure, stress and fatigue that can hinder task execution on mobile phones, participants described their desire for more proactive technology, where their mobile phones take some action on their behalf without them asking for this assistance. For instance, Sylvia described her desire for a single button, that when pushed, enabled a personal assistant that could help her during doctor’s visits by “advocat[ing] for me” by “asking [questions] for me” if she forgets to ask them. Sylvia also wanted the personal assistant to tape her conversations with her doctor, because “if I can record it then I can play it back,” supporting her memory changes. In later interviews, we used Sylvia’s idea to probe other participants’ perception of the line between helpful and disconcerting with this kind of assistance from mobile phones. Tina described her perspective: “I don’t think it’s creepy because you’re making the choice to tell [the system] to start filming [recording]… It’s not like she [the system] knows you just got into the doctor’s office and now automatically the microphone’s going to show up.” However, Sabella surfaced a legitimate concern: “the thing about the button [to activate the personal assistant] is you have to remember to tell the button to do it [provide assistance].” Because memory can be increasingly challenging for people with dementia, Tina later concluded: “I think[s] there’s going to be a time where we really need all those crazy voices [voice assistants] kind of helping us” without being prompted.

In addition to wanting more proactive assistance in doctor’s visits, participants also described how they would like their phone to track their activities on their mobile phone to play a part in sustaining their relationships. Participants wanted their phone to help them respond to others and initiate social interactions with others - tasks that were becoming increasingly difficult. For example, Alecia wanted her phone to be “snooping in your text” to then remind you “‘hey, you haven’t checked in on or checked up on so and so since such and such.”’, similar to existing Microsoft Outlook email reminders [69]. Though Alecia describes how it would be important for the system to be “synced to your text” rather than social media in order to only be tracking those more active social connections.

One participant, Preston, proposed another instance when tracking mobile phone use to provide proactive technology assistance may be useful: in identifying unused but potentially useful features (e.g., a stylus). For example, Preston proposed the phone could identify: “‘Hey, he really hasn’t used that stylus pen. Maybe we should send him another opportunity.”’ and then provide “a tutorial” of how to use that feature. When this idea was presented to participants in later interviews it was well received as a potential solution to facilitate learning new task flows after updates or upgrades.

Participants also wanted their phone to provide more proactive assistance by learning and automating routine uses of their phone, which could further support them in executing tasks on their mobile phones in stressful situations or when they experience fatigue. For example, participants described using specific intervals of reminders for every calendar event: “it’s always two days before the event, one day before the event, one hour before the event, and maybe 15 or 20 minutes before the event, and then a five-minute reminder” [Preston]. Preston wanted his phone to be “smart enough to realize, I’m doing the same reminders every time. There should be an option where I can just say, ‘use your common reminders’.” Sabella wanted to completely automate the process of inputting calendar events so that “when you call to say ‘Sabella, I’m going to set us up a Zoom for so and so’ and it automatically just kind of records it in some little strange way and it sets it on the calendar right then… Not for me to do it [input a calendar invite] but its own self.”

Proactive assistance could also take the form of voice-assistants providing more structured prompts in a conversational style to help people execute tasks on their mobile phone. For example, Elenora wanted to be able to

“tell my phone while I’m out: ‘I’m confused’ or ‘I’m not sure where to go.’ Then it could start asking me questions. Not, ‘Do you want to do this?’ But literally ask me questions to help me figure out what it is I’m trying to even do. And, I could just say ‘I’m trying to get home’ and it could say, ‘Well, there’s all kinds of ways. What do you prefer? Want to go home the fastest way? The easiest way? The cheapest way?’ And then I can just pick one.”

This system would allow Elenora to “interact with it in a more natural language… almost have a conversation with the system.” This example reflects the challenges of balancing 100% open-ended conversation with structured prompting (as previous work with older adults has highlighted [17]). In our findings, participants describe their need for structured prompting but in a more conversational style and for voice-assistants to understand less structured responses.

Participants described their desire for more proactive technology assistance through unprompted assistance, more in-depth tracking of their mobile phone use to support relearning task flows after updates and upgrades, as well as automating routine tasks to provide support in moments of high stress or fatigue that impair mobile phone use.

4.3.4. Extending Voice-based Interactions with Mobile Phones.

Participants described their desire to use voice-based interaction to address challenges with navigating their phone. For instance, Tina wanted to just be able to speak to her preferred social media app to have it search for pictures of her and her daughter on vacation rather than having to scroll through her past posts to find the album. Many participants described their desire to “click on the calendar [app] and use your voice and say, ‘doctor’s appointment, this date, this time”’ and have it add an event [Josslyn]. Figure 2 provides an example of adding a calendar event using voice-based interactions with Google Assistant on the mobile phone, which also takes into consideration more proactive technology assistance. Participants also wanted to search for content across apps [Elenora, Josslyn, Alecia, Sabella, Kim, Preston] (e.g., saved recipes on different social media apps [Kim], social connections with people [Thomas]). Preston wanted to “name your phone, which would then turn it on like Alexa” when you spoke its name.

Figure 2:

Figure 2:

Mockup of adding a calendar event using voice-based interactions with Google Assistant on a mobile phone, including proactive technology assistance with reminder intervals.

Several participants described needing an avatar in combination with a voice-based virtual assistant on their mobile phone to help with attention, which would ultimately help them to execute tasks on their mobile phones. For instance, Malcolm described how “I don’t like people talking to me over the phone, it’s difficult for me. This [points at the video conference call camera] is easier.” Preston describes this as “a dual connection because you’re connecting with your eyes and your ears, whereas if my screen is gone, we’ve only got one connection and that’s just listening.” Preston believes this dual connection would help to “draw you in and you’re going to pay more attention than a voice that’s just randomly talking.” Because of this need for a dual connection, participants described the need for voice-based interactions on their phone to have an “artificial person” or “an android person” [Preston]. Sylvia described wanting to personalize her virtual assistant by having “an animated person” where “I can pick his voice. I can pick the culture. Like, it’ll probably be a person of color.” She explained this concept by relating to “the Wii game you can describe your person… You can pick the hair, you pick the glasses, the voice, the outfit. I would like to pick my virtual assistant” [Sylvia].

Participants also described clear lines for acceptable use of avatars with virtual assistants for people with dementia, relating back to the uncanny valley effect [70]. Preston notes that the system needs to make it clear that “it’s not a real person” by having “in the background saying ‘You’re speaking to Robby the Robot’ something that is displayed. So that they know this person isn’t real because for some people they could get confused.” One participant, Alecia, was open to support from avatars in combination with virtual assistants as long as it remained two dimensional where any physical or three-dimensional robots were considered “scary” and “going over the top.”

Even with the enthusiasm of many participants about extending voice-based interactions with their mobile phone, several participants also noted the importance of providing human assistance in high stress or time sensitive instances where voice assistants were not understanding them, which made executing tasks on their mobile phones difficult. Thomas stated: “there are times when we just need, particularly people with cognitive challenges, need the ability to reach out and touch another person. Especially if we’re in a situation where our brains aren’t working right.” Similarly, Miranda describes how she “prefers person-centered help” especially “when you’re talking to an artificial intelligence [about] anything more complex.” Though she elaborates “that doesn’t mean that I don’t think things like [voice assistants] aren’t good. But I think within those [having] the ability to easily access an actual person is also important” [Miranda].

Participants described their desire to extend the capabilities of voice-based interactions by: facilitating voice-based interactions within and across apps to better support navigation, as well as to support task execution by using avatars in combination with voice assistants to support attention and further personalization; and providing access to human assistance in instances when voice assistants did not understand them.

5. DISCUSSION

Through interviews with fourteen people with mild to moderate dementia, we address each of our three research questions. First, we investigated how people with mild to moderate dementia use their mobile phones, finding they use a range of apps for: navigation, mobile shopping, online banking, games, social media, news, entertainment, communication, connection, productivity, and health management. These findings take a first step towards filling an empirical gap in the literature by detailing some ways people with mild to moderate dementia use their mobile phones. Because past research in the development of mobile apps for people with dementia have primarily focused on designing apps to improve memory [40] or for GPS tracking to assist in navigation [46, 61, 100, 101], we also intentionally highlight two common activities on mobile phones, productivity and health management, as promising areas for future development. Additionally, participants described some ways their use of apps changed as they progressed with dementia. For example, they experienced a new reliance on their calendar apps to dictate their daily activities. They also described having to compartmentalize calendars based on different aspects of life, as well as the need for set intervals of reminders for each calendar event. As our findings report participants’ reflections on how their mobile phone use has changed, further research is needed to observe the longitudinal use of mobile phone applications as people with dementia progress with the condition.

Second, we investigated the pain points and barriers to completing tasks on mobile phones that people with mild to moderate dementia experience. We uncovered challenges with: 1) navigating to apps and features; 2) task execution in moments of high stress, fatigue, and time pressure, and 3) re-learning task flows after updates and upgrades. These findings expand on past work on barriers tech-savvy people with mild to moderate dementia have with typing on their mobile phone due to small key size and difficulty identifying the notification ringtones and sounds [31] by providing a more thorough examination of the barriers people with mild to moderate dementia experience with mobile phone use.

Addressing our third research question concerning opportunities for technology to support people with dementia when they encounter challenges with their mobile phones, we uncover four design opportunities based on participants envisioned future interactions with mobile phones: customizing for accessibility, activity-based customization, proactive technology assistance, and extended modalities for voice-based interactions. These findings demonstrate the considerable creativity of people with dementia in generating ideas for future technologies.

Many of these ideas incorporate AI and automation to support more accessible interactions with mobile phones for people with dementia. This provides a different perspective from past work in AI and dementia, which has primarily focused on ways AI could be used to detect and monitor the progress of dementia [8, 41, 87, 92, 96], in smart home environments to support care partners in monitoring the activities of individuals with dementia [3, 24, 26, 63, 80], or to support therapy [13, 20, 95]. Therefore, our work opens up new opportunities for the design of future AI systems to support the abilities of people with dementia, as in ability-based design [105].

In the remainder of the discussion, we describe design opportunities to support individuals with progressive disabilities and tensions with automation for people with dementia.

5.1. Design Opportunities to Support Individuals with Progressive Disabilities

One contribution of this work is taking a first step towards understanding how to design for access needs for a group of neurodiverse users that experience progressive changes in ability, going beyond the traditional binary representation of disability. For instance, our findings demonstrate the importance of adaptive user interfaces to minimize navigation of mobile phone use, which can become increasingly difficult for people with dementia as they progress with the condition. Our findings also demonstrate the importance of providing increasingly proactive technological support for people with dementia as they progress with the condition. In the following section we discuss each of these directions for future design.

5.1.1. Adaptive User Interfaces for Progressive Changes in Ability.

Participants described their desire for adaptive interfaces that could simplify navigating their phones through customizable home screens, and adjustable app sizes and personalizable icons (aligning with guidance from the WCAG Cognitive Accessibility Task Force) [99]. But, they also described challenges with learning new interaction patterns, which are inevitable with adaptive interfaces. As one way to navigate this tension, designers could ensure that users are made aware of any changes with updates and upgrades and then provide training on new task flows.

Another potential way to navigate this tension is to use temporal dimensions for adaptive user-interfaces for people with dementia, building off of previous work on ephemeral adaptations - where only the most used menu items are displayed abruptly and then all other menu items gradually fade in [34]. In the case of mobile phone accessibility for people with dementia, it may be necessary for the less used apps to gradually fade out and no longer be displayed due to changes in visual ability with the progression of the condition, including selecting an object from a visually busy environment [11, 91]. To be clear, we are not advocating for systems that strip away functions of a device to the bare minimum due to the inability of people with dementia to understand complex functions of devices (as in past work [4, 18, 43, 55, 62, 72, 88]). Rather, we are proposing systems which display the most used features in a way that is easy for people with mild to moderate dementia to navigate to, while still providing access to less used features if/when they want them.

Still another potential future direction is the design of adaptable systems in combination with activity-centric thinking [7], which shifts away from traditional application-centric computing and towards human goal-oriented activities, cutting across systems boundaries [7]. Participants wanted their user interfaces to bundle common activities and store these bundles in distinct, easily identifiable places. For instance, participants bundled their calendars by type of activity (e.g., in-home activities vs. out of the house activities). Adapting user interfaces to reflect bundles of common activities may be one way to make systems more accessible to people with dementia as they experience progressive changes in ability.

These are just two examples of potential areas to explore in future work on adaptive user interfaces to support people with dementia. There is room for much further exploration concerning adaptive user-interfaces to support more accessible interactions with technology for people with progressive changes in ability.

5.1.2. Proactive Technological Support.

One way to provide more proactive technological support is through context-aware computing. Participants described the need for their phone to recognize their location as well as their conversational partner (as in the TalkAbout System for people with Aphasia [51]) to assist them in executing tasks in moments of high stress and fatigue. This assistance could be provided through context-aware prompting for tasks, as in [22, 23] which provided prompts for users with cognitive impairments to complete tasks. Though, as participants in our study mentioned, there may come a time with the progression of the condition where prompting will not be enough and the system will need to provide more active assistance. For example, assistance could be provided automatically when the system recognized the context the person with dementia was in (e.g., in a doctor’s appointment speaking with their doctor). One participant even wanted their mobile phones to listen into their phone conversations to automatically add calendar events as they were confirmed during phone calls, as in past work [64, 65]. Future systems should consider utilizing context-aware computing to provide more active support to people with dementia.

Another way proactive technological support could be provided is through tracking technology use. For example, several participants described their desire for their phone to monitor their text and emails to prompt them to respond to and initiate social interactions with others. Participants also wanted their mobile phones to track what features they used and propose new or previously unused features that may be helpful to them based on their historic phone usage. Current systems provide nudges and notifications for new features once rather than providing additional reminders as these additional reminders are assumed to be unhelpful and annoying to users [81]. Our findings suggest that for people with dementia they may need to be reminded about features more than once, due to the progressive nature of the condition and their changes in memory overtime. Future systems could integrate more proactive technological support (e.g., context-aware proactive smart-speakers [83, 102, 103]) by highlighting potentially useful features and providing regular reminders of these features if they begin to go unused.

Although these design directions may provide necessary support to prolong mobile phone use for people with dementia, they also introduce privacy concerns due to the level of data collection necessary to provide this support. As past work has outlined [67], such tracking could be used as a mediator of coercive control and abuse. To manage this tension between the need for more proactive technology support and privacy concerns, we urge researchers, designers and developers to keep privacy considerations central to their work.

5.2. Tensions in Designing Technologies for use by People with Dementia

Participants described extensive future applications for automation, surfacing tensions with autonomy and who would be in control of initiating support, the person or the system. For instance, participants in our study described ways that automation could be used to improve their task execution on their mobile phones, centering their own role in completing the task (e.g., having a button where they could prompt voice assistants). This may be one solution to the concern of some people with dementia in past work towards receiving support from AI to assist with managing their daily life out of concern for the loss of autonomy [32]. However, participants also described how with the progression of the condition, they will need systems to eventually act autonomously to perform tasks on their behalf (e.g., start audio recording in doctor’s visits). Participants described these progressive changes in abilities due to the condition necessitates less autonomy with technology to support them to continue to be active in everyday activities (e.g., attending doctor’s visits independently). However, with more proactive technological support and less autonomy, this could pose additional risks for abuse facilitated via the proactive systems (as described in [67]). With evidence of elder abuse through automation in the form of smart home technologies [15], we join in the call of past work [94] to design automated systems for use by people with dementia rather than on people with dementia.

Participants also described instances where their reliance on technology negatively affected them. For example, moments of intense brain fog can render technology assistance useless (e.g., when navigating), requiring them to reach out to another person for support. These findings suggest future systems should be designed to keep humans in-the-loop (as proposed in [89]) around users with dementia to provide support when necessary. Importantly, participants needed to be connected to sympathetic human assistance. Therefore, future systems could sense frustration or negative emotion from voice interactions, and then automatically connect the user to a friend or loved one to provide human assistance.

Still another tension which emerged from our work concerns the use of a visual representation for voice assistance. Although some participants noted a visual representation may help with attention when interacting with voice assistants, they also noted how such visuals may be confusing or disturbing to some people with dementia who may not be able to discern that the avatar is not a real person. Future work is needed to understand how people with dementia perceive different types of visual representations of voice assistants (e.g., disembodied agents, artificial embodied agent, and photorealistic embodied agents [10]) and if their perceptions of visual representations change with the progression of the condition.

We recognize that this is not a full investigation of all possible tensions that may arise with the design of technologies for use by people with dementia (e.g., trust, explainability). Further work is needed to investigate these tensions and ways for designers, developers, and researchers to better navigate these tensions.

6. CONCLUSION

This work details ways people with mild to moderate dementia use their mobile phones surfaced through an analysis of interviews with fourteen people with mild to moderate dementia. Findings from this study showed three major challenges with mobile phone use: 1) navigating to apps and features; 2) task execution in moments of time pressure, high stress and fatigue, and 3) re-learning task flows after updates and upgrades. To address these challenges participants described their ideal interactions with their mobile phones, which included customizing for accessibility, activity-based customization, proactive technology assistance, and extended modalities for voice-based interactions. This paper contributes to the literature by 1) providing an empirical account of how fourteen people with mild to moderate dementia use mobile phones and the challenges they face with mobile phone use; 2) uncovering design opportunities to help achieve more accessible mobile phone use for people with dementia; and 3) providing new directions for the design of future systems to augment and enhance the abilities of people with dementia. With the pervasive use of mobile phones in our society, these findings will help researchers and creators of technology design environments that assure societal inclusion [47] for people with mild to moderate dementia.

Supplementary Material

Supplementary Material

CCS CONCEPTS.

Human-centered computing → Accessibility; Empirical studies in accessibility; Human computer interaction (HCI); Empirical studies in HCI; Accessibility; Accessibility theory, concepts and paradigms.

ACKNOWLEDGMENTS

Thank you to participants, as well as Shaun Kane, Hernisa Kacorri, Jonathan Lazar, Gregg Vanderheiden, Matthew Goupell and the anonymous reviewers who provided feedback on versions of this paper. This work was supported, in part, by grant 90REGE0008, U.S. Admin. for Community Living, NIDILRR, Dept. of Health & Human Services, NSF Grant IIS-2045679, and the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 1840340. Opinions expressed do not necessarily represent official policy of the Federal government.

A. APPENDICES

Table 2:

Table of Apps and Features from Participants Self-reported Mobile Phone Usage

Type of App or Feature App or Feature Participants who Reported Using that App or Feature Number of Participants
Social Media Facebook Thomas, Elenora, Kim, Tristen, Sylvia, Kennith, Tina, Malcolm, Josslyn, Hall, Alecia 11
YouTube Sylvia, Preston, Kennith, Hall 4
Instagram Elenora, Tina 2
SnapChat Alecia, Sabella 2
TikTok Sabella 1
LinkedIn Thomas 1
Twitter Thomas 1
Navigation Waze Tristen, Kennith, Malcolm 3
Google Maps Elenora, Hall 2
Apple Maps Thomas 1
Life360 Thomas 1
Uber Elenora 1
Lift Elenora 1
Unspecified Kim, Miranda 2
Online Shopping Amazon Kim, Josslyn, Alecia 3
Sams App Kim 1
Ebay Alecia 1
Mobile Banking Capital One App Elenora 1
Apple Wallet Thomas 1
Venmo Alecia 1
unspecified Kim, Tristen, Sylvia, Sabella 4
News BBC Malcolm 1
CNN Malcolm 1
Fox Malcolm 1
NPR Malcolm 1
unspecified Josslyn, Hall 2
Music Pandora Elenora, Hall 2
Shazam Alecia 1
unspecified Thomas, Tristen, Tina, Alecia, Sabella 5
Communication Email Thomas, Elenora, Tristen, Sylvia, Miranda, Tina, Malcolm, Josslyn, Sabella 9
phone calls Kim, Tristen, Preston, Kennith, Josslyn, Hall, Sabella 7
Messenger Kim, Kennith, Tina, Malcolm, Hall, Sabella 6
SMS Tristen, Sylvia, Preston, Josslyn, Alecia 5
FaceTime Miranda, Tina, Malcolm, Sabella 4
Zoom Sylvia, Alecia 2
Google Meet Miranda 1
Evite Kennith 1
Blogs Tina 1
WhatsApp Elenora 1
Games Words with Friends Kennith 1
Solitaire Hall 1
memory games Sabella 1
unspecified Kim, Alecia 2
Entertainment Photos Thomas, Elenora, Sylvia, Preston, Hall, Alecia 6
Audible Kim, Hall, Alecia 3
Bible app Kim, Hall 2
Deer Cast Hall 1
Apple TV Sabella 1
Planter - Garden Planner app Kim 1
ESPN Hall 1
Frameo Alecia 1
GoFan Hall 1
E-books Thomas 1
Productivity Reminders Thomas, Elenora, Tristen, Sylvia, Miranda, Malcolm, Josslyn, Hall 8
Google Search Preston, Miranda, Kennith, Tina, Malcolm, Josslyn, Sabella 7
Calendar Thomas, Kim, Preston, Kennith, Tina, Hall, Alecia 7
Google Calendar Elenora, Tristen, Sylvia, Miranda, Malcolm, Josslyn 6
Calculator Thomas, Elenora, Kim, Alecia 4
Weather Kim, Tina, Hall, Alecia 4
KeepNote Elenora, Sylvia, Alecia 3
Clock Elenora, Tristen, Alecia 3
Google Workspace Thomas, Sylvia 2
Timer Kim 1
Notes Kim 1
Outlook Alecia 1
Evernote Elenora 1
Flashlight Thomas 1
Health Management Fitbit Alecia 1
pharmacy apps unspecified Kim 1
patient portals Sabella 1
glucose monitoring app Elenora 1
fall detection apps Miranda 1

Footnotes

ACM Reference Format:

Emma Dixon, Rain Michaels, Xiang Xiao, Yu Zhong, Patrick Clary, Ajit Narayanan, Robin Brewer*, and Amanda Lazar. 2022. Mobile Phone Use by People with Mild to Moderate Dementia: Uncovering Challenges and Identifying Opportunities: Mobile Phone Use by People with Mild to Moderate

1

SOS is an internationally recognized signal of distress in radio code used especially by ships calling for help.

2

911 is a phone number used in North America to contact emergency services.

Contributor Information

Emma Dixon, University of Maryland.

Rain Michaels, Google Inc..

Xiang Xiao, Google Inc..

Yu Zhong, Google Inc..

Patrick Clary, Google Inc..

Ajit Narayanan, Google Inc..

Robin Brewer, University of Michigan.

Amanda Lazar, University of Maryland.

REFERENCES

  • [1].Alzheimer’s Association. Younger/Early-Onset Alzheimer’s. Alzheimer’s Disease and Dementia. Retrieved March 16, 2020 from https://alz.org/alzheimers-dementia/what-is-alzheimers/younger-early-onset
  • [2].Alzheimer’s Association. 2021. 2021 Alzheimer’s Disease Facts and Figures. Alzheimers Dement, Chicago, IL. [DOI] [PubMed] [Google Scholar]
  • [3].Amiribesheli Mohsen and Bouchachia Abdelhamid. 2015. Smart Homes Design for People with Dementia. In 2015 International Conference on Intelligent Environments, 156–159. 10.1109/IE.2015.33 [DOI] [Google Scholar]
  • [4].Ancient Claire, Good Alice, Wilson Clare, and Fitch Tineke. 2013. Can Ubiquitous Devices Utilising Reminiscence Therapy Be Used to Promote Well-Being in Dementia Patients? An Exploratory Study. In Universal Access in Human-Computer Interaction. Applications and Services for Quality of Life (Lecture Notes in Computer Science), 426–435. 10.1007/978-3-642-39194-1_50 [DOI] [Google Scholar]
  • [5].Apple Inc. 2022. Create and manage groups of contacts on iCloud.com. Apple Support. Retrieved June 24, 2022 from https://support.apple.com/guide/icloud/create-and-manage-groups-mmfba73c71/icloud
  • [6].Archer Norm, Keshavjee Karim, Demers Catherine, and Lee Ryan. 2014. Online self-management interventions for chronically ill patients: Cognitive impairment and technology issues. International Journal of Medical Informatics 83, 4: 264–272. 10.1016/j.ijmedinf.2014.01.005 [DOI] [PubMed] [Google Scholar]
  • [7].Bardram Jakob E., Jeuris Steven, Tell Paolo, Houben Steven, and Voida Stephen. 2019. Activity-centric computing systems. Communications of the ACM 62, 8: 72–81. 10.1145/3325901 [DOI] [Google Scholar]
  • [8].Bertini Flavio, Allevi Davide, Lutero Gianluca, Montesi Danilo, and Calzà Laura. 2021. Automatic Speech Classifier for Mild Cognitive Impairment and Early Dementia. ACM Transactions on Computing for Healthcare 3, 1: 8:1–8:11. 10.1145/3469089 [DOI] [Google Scholar]
  • [9].Boman Inga-Lill, Lundberg Stefan, Starkhammar Sofia, and Nygård Louise. 2014. Exploring the usability of a videophone mock-up for persons with dementia and their significant others. BMC Geriatrics 14, 1: 49. 10.1186/1471-2318-14-49 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [10].Bonfert Michael, Zargham Nima, Saade Florian, Porzel Robert, and Malaka Rainer. 2021. An Evaluation of Visual Embodiment for Voice Assistants on Smart Displays. In CUI 2021 - 3rd Conference on Conversational User Interfaces, 1–11. 10.1145/3469595.3469611 [DOI] [Google Scholar]
  • [11].Borruat François-Xavier. 2013. Posterior Cortical Atrophy: Review of the Recent Literature. Current Neurology and Neuroscience Reports 13, 12: 406. 10.1007/s11910-013-0406-8 [DOI] [PubMed] [Google Scholar]
  • [12].Bossen Ann L, Kim Heejung, Williams Kristine N, Steinhoff Andreanna E, and Strieker Molly. 2015. Emerging roles for telemedicine and smart technologies in dementia care. Smart homecare technology and telehealth 3: 49–57. 10.2147/SHTT.S59500 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Boumpa Eleni, Charalampou Ioanna, Gkogkidis Anargyros, and Kakarountas Athanasios. 2017. Home Assistive System for Dementia. In Proceedings of the 21st Pan-Hellenic Conference on Informatics (PCI 2017), 1–6. 10.1145/3139367.3139435 [DOI] [Google Scholar]
  • [14].Bourazeri Aikaterini and Stumpf Simone. 2018. Co-designing smart home technology with people with dementia or Parkinson’s disease. In Proceedings of the 10th Nordic Conference on Human-Computer Interaction - NordiCHI ’18, 609–621. 10.1145/3240167.3240197 [DOI] [Google Scholar]
  • [15].Brandl Bonnie, Dyer Carmel Bitondo, Heisler Candace J., Otto Joanne Marlatt, Stiegel Lori A., and Thomas Randolph W.. 2006. Elder Abuse Detection and Intervention: A Collaborative Approach. Springer Publishing Company. [Google Scholar]
  • [16].Braun Virginia and Clarke Victoria. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2: 77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]
  • [17].Brewer Robin, Raymundo Cornejo Garcia Tedmond Schwaba, Gergle Darren, and Anne Marie Piper. 2016. Exploring Traditional Phones as an E-Mail Interface for Older Adults. ACM Transactions on Accessible Computing 8, 2: 6:1–6:20. 10.1145/2839303 [DOI] [Google Scholar]
  • [18].Cahill Suzanne Mary, Begley Emer, Faulkner Jon Paul, and Hagen I. 2007. “It gives me a sense of independence” – Findings from Ireland on the use and usefulness of assistive technology for people with dementia. Technology and Disability 19, 2–3: 133–142. 10.3233/TAD-2007-192-310 [DOI] [Google Scholar]
  • [19].Caine Kelly. 2016. Local Standards for Sample Size at CHI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI ’16), 981–992. 10.1145/2858036.2858498 [DOI] [Google Scholar]
  • [20].Carós Mariona, Garolera Maite, Radeva Petia, and Giro-i-Nieto Xavier. 2020. Automatic Reminiscence Therapy for Dementia. In Proceedings of the 2020 International Conference on Multimedia Retrieval, 383–387. Retrieved March 4, 2022 from 10.1145/3372278.3391927 [DOI] [Google Scholar]
  • [21].Cavallo Filippo, Aquilano Michela, and Arvati Marco. 2015. An Ambient Assisted Living Approach in Designing Domiciliary Services Combined With Innovative Technologies for Patients With Alzheimer’s Disease: A Case Study. American Journal of Alzheimer’s Disease & Other Dementias® 30, 1: 69–77. 10.1177/1533317514539724 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [22].Chang Yao-Jen, Chang Wan Chih, and Wang Tsen-Yung. 2009. Context-aware prompting to transition autonomously through vocational tasks for individuals with cognitive impairments. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets ’09), 19–26. 10.1145/1639642.1639648 [DOI] [Google Scholar]
  • [23].Chang Yao-Jen, Tsai Shih-Kai, and Wang Tsen-Yung. 2008. A context aware handheld wayfinding system for individuals with cognitive impairments. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility (Assets ’08), 27–34. 10.1145/1414471.1414479 [DOI] [Google Scholar]
  • [24].Chimamiwa Gibson, Alirezaie Marjan, Banaee Hadi, Köckemann Uwe, and Loutfi Amy. 2019. Towards Habit Recognition in Smart Homes for People with Dementia. In Ambient Intelligence (Lecture Notes in Computer Science), 363–369. 10.1007/978-3-030-34255-5_29 [DOI] [Google Scholar]
  • [25].Chu Yi, Brown Pat, Harniss Mark, Kautz Henry, and Johnson Kurt. 2014. Cognitive support technologies for people with TBI: current usage and challenges experienced. Disability and Rehabilitation: Assistive Technology 9, 4: 279–285. 10.3109/17483107.2013.823631 [DOI] [PubMed] [Google Scholar]
  • [26].Dagoberto Cruz-Sandoval and Jesus Favela. 2016. Human-robot interaction to deal with problematic behaviors from people with dementia. In Proceedings of the 10th EAI International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth ’16), 274–275. [Google Scholar]
  • [27].Davies Richard, Nugent Chris, Donnelly Mark, Hettinga Marike, Meiland Franka, Moelaert Ferial, Mulvenna Maurice, Bengtsson Johan, Craig David, and Dröes Rose-Marie. 2009. A user driven approach to develop a cognitive prosthetic to address the unmet needs of people with mild dementia. Pervasive and Mobile Computing 5, 3: 253–267. 10.1016/j.pmcj.2008.07.002 [DOI] [Google Scholar]
  • [28].Dawe Melissa. 2006. Desperately seeking simplicity: how young adults with cognitive disabilities and their families adopt assistive technologies. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1143–1152. 10.1145/1124772.1124943 [DOI] [Google Scholar]
  • [29].Dawe Melissa. 2007. Understanding mobile phone requirements for young adults with cognitive disabilities. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility (Assets ’07), 179–186. 10.1145/1296843.1296874 [DOI] [Google Scholar]
  • [30].Dixon Emma, Anderson Jesse, and Lazar Amanda. 2022. Understanding How Sensory Changes Experienced by Individuals with a Range of Age-Related Cognitive Changes can Affect Technology Use. ACM Transactions on Accessible Computing. 10.1145/3511906 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [31].Dixon Emma and Lazar Amanda. 2020. The Role of Sensory Changes in Everyday Technology use by People with Mild to Moderate Dementia. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’20), 1–12. 10.1145/3373625.3417000 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [32].Dixon Emma, Piper Anne Marie, and Lazar Amanda. 2021. “Taking care of myself as long as I can”: How People with Dementia Configure Self-Management Systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–14. 10.1145/3411764.3445225 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [33].Dixon Emma, Shetty Ashrith, Pimento Simone, and Lazar Amanda. 2021. Lessons Learned from Remote User-Centered Design with People with Dementia. In Proceedings of the 2021 Dementia Lab Conference (Design for Inclusion). [Google Scholar]
  • [34].Findlater Leah, Moffatt Karyn, McGrenere Joanna, and Dawson Jessica. 2009. Ephemeral adaptation: the use of gradual onset to improve menu selection performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1655–1664. 10.1145/1518701.1518956 [DOI] [Google Scholar]
  • [35].Freeman Ed, Dr Linda Clare, Savitch Nada, Royan Lindsay, Litherland Rachael, and Lindsay Margot. 2005. Improving website accessibility for people with early-stage dementia: A preliminary investigation. Aging & Mental Health 9, 5: 442–448. 10.1080/13607860500142838 [DOI] [PubMed] [Google Scholar]
  • [36].Gibson Grant, Dickinson Claire, Brittain Katie, and Robinson Louise. 2015. The everyday use of assistive technology by people with dementia and their family carers: a qualitative study. BMC Geriatrics 15, 1: 1–10. 10.1186/s12877-015-0091-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [37].Gibson Grant, Dickinson Claire, Brittain Katie, and Robinson Louise. 2019. Personalisation, customisation and bricolage: how people with dementia and their families make assistive technology work for them. Ageing & Society 39, 11: 2502–2519. 10.1017/S0144686X18000661 [DOI] [Google Scholar]
  • [38].Google Inc. 2022. View, group & share contacts - Android - Contacts Help. Google Support - Contacts Help. Retrieved June 24, 2022 from https://support.google.com/contacts/answer/30970?hl=en&co=GENIE.Platform%3DAndroid
  • [39].Guisado-Fernández Estefanía, Giunti Guido, Mackey Laura M., Blake Catherine, and Caulfield Brian Michael. 2019. Factors Influencing the Adoption of Smart Health Technologies for People With Dementia and Their Informal Caregivers: Scoping Review and Design Framework. JMIR Aging 2, 1: e12192. 10.2196/12192 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [40].Guo Yuqi, Yang Fan, Hu Fei, Li Wei, Ruggiano Nicole, and Lee Hee Yun. 2020. Existing Mobile Phone Apps for Self-Care Management of People With Alzheimer Disease and Related Dementias: Systematic Analysis. JMIR Aging 3, 1: e15290. 10.2196/15290 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [41].Palliya Guruge Chathurika, Oviatt Sharon, Haghighi Pari Delir, and Pritchard Elizabeth. 2021. Advances in Multimodal Behavioral Analytics for Early Dementia Diagnosis: A Review. In Proceedings of the 2021 International Conference on Multimodal Interaction, 328–340. Retrieved March 4, 2022 from 10.1145/3462244.3479933 [DOI] [Google Scholar]
  • [42].Guzman-Parra Jose, Barnestein-Fonseca Pilar, Guerrero-Pertiñez Gloria, Anderberg Peter, Jimenez-Fernandez Luis, Valero-Moreno Esperanza, Goodman-Casanova Jessica Marian, Cuesta-Vargas Antonio, Garolera Maite, Quintana Maria, García-Betances Rebeca I, Lemmens Evi, Berglund Johan Sanmartin, and Mayoral-Cleries Fermin. 2020. Attitudes and Use of Information and Communication Technologies in Older Adults With Mild Cognitive Impairment or Early Stages of Dementia and Their Caregivers: Cross-Sectional Study. Journal of Medical Internet Research 22, 6: e17253. 10.2196/17253 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [43].Hattink Bart, Droes Rose-Marie, Sikkes Sietske, Oostra Ellen, and Lemstra Afina W. 2016. Evaluation of the Digital Alzheimer Center: Testing Usability and Usefulness of an Online Portal for Patients with Dementia and Their Carers. JMIR Research Protocols 5, 3. 10.2196/resprot.5040 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [44].Hegde Soumya and Ellajosyula Ratnavalli. 2016. Capacity issues and decision-making in dementia. Annals of Indian Academy of Neurology 19, Suppl 1: S34–S39. 10.4103/0972-2327.192890 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [45].Hendriks Niels, Huybrechts Liesbeth, Wilkinson Andrea, and Slegers Karin. 2014. Challenges in doing participatory design with people with dementia. In Proceedings of the 13th Participatory Design Conference on Short Papers, Industry Cases, Workshop Descriptions, Doctoral Consortium papers, and Keynote abstracts - PDC ’14 - volume 2, 33–36. 10.1145/2662155.2662196 [DOI] [Google Scholar]
  • [46].Holbø Kristine, Bøthun Silje, and Dahl Yngve. 2013. Safe walking technology for people with dementia: what do they want? In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’13), 1–8. 10.1145/2513383.2513434 [DOI] [Google Scholar]
  • [47].Holmes Kat. 2018. Mismatch: How Inclusion Shapes Design. MIT Press, Cambridge, MA, USA. [Google Scholar]
  • [48].Hutchinson Hilary, Mackay Wendy, Westerlund Bo, Bederson Benjamin B., Druin Allison, Plaisant Catherine, Beaudouin-Lafon Michel, Conversy Stéphane, Evans Helen, Hansen Heiko, Roussel Nicolas, and Eiderbäck Björn. 2003. Technology probes: inspiring design for and with families. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’03), 17–24. 10.1145/642611.642616 [DOI] [Google Scholar]
  • [49].W3C Web Accessibility Initiative (WAI). Web Content Accessibility Guidelines (WCAG) Overview. Web Accessibility Initiative (WAI). Retrieved January 9, 2021. from https://www.w3.org/WAI/standards-guidelines/wcag/
  • [50].Kane Shaun K., Jayant Chandrika, Wobbrock Jacob O., and Ladner Richard E.. 2009. Freedom to roam: a study of mobile device adoption and accessibility for people with visual and motor disabilities. In Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility (Assets ’09), 115–122. 10.1145/1639642.1639663 [DOI] [Google Scholar]
  • [51].Kane Shaun K., Linam-Church Barbara, Althoff Kyle, and McCall Denise. 2012. What we talk about: designing a context-aware communication tool for people with aphasia. In Proceedings of the 14th international ACM SIGACCESS conference on Computers and accessibility - ASSETS ’12, 49. 10.1145/2384916.2384926 [DOI] [Google Scholar]
  • [52].Kerkhof Yvonne, Bergsma Ad, Graff Maud, and Dröes Rose-Marie. 2017. Selecting apps for people with mild dementia: Identifying user requirements for apps enabling meaningful activities and self-management. Journal of Rehabilitation and Assistive Technologies Engineering 4: 2055668317710593. 10.1177/2055668317710593 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [53].Kerkhof Yvonne, Kohl Gianna, Veijer Melanie, Mangiaracina Floriana, Bergsma Ad, Graff Maud, and Dröes Rose-Marie. 2020. Randomized controlled feasibility study of FindMyApps: first evaluation of a tablet-based intervention to promote self-management and meaningful activities in people with mild dementia. Disability and Rehabilitation: Assistive Technology 0, 0: 1–15. 10.1080/17483107.2020.1765420 [DOI] [PubMed] [Google Scholar]
  • [54].Kerkhof Yvonne, Pelgrum-Keurhorst Myrna, Mangiaracina Floriana, Bergsma Ad, Vrauwdeunt Guus, Graff Maud, and Dröes Rose-Marie. 2019. User-participatory development of FindMyApps; a tool to help people with mild dementia find supportive apps for self-management and meaningful activities. DIGITAL HEALTH 5: 205520761882294. 10.1177/2055207618822942 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [55].Kort Helianthe S. M. and van Hoof Joost. 2014. Design of a website for home modifications for older persons with dementia. Technology and Disability 26, 1:1–10. 10.3233/TAD-140399 [DOI] [Google Scholar]
  • [56].Kruse Clemens Scott, Fohn Joanna, Umunnakwe Gilson, Patel Krupa, and Patel Saloni. 2020. Evaluating the Facilitators, Barriers, and Medical Outcomes Commensurate with the Use of Assistive Technology to Support People with Dementia: A Systematic Review Literature. Healthcare 8, 3. 10.3390/healthcare8030278 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [57].LaMonica Haley M, English Amelia, Hickie Ian B, Ip Jerome, Ireland Catriona, West Stacey, Shaw Tim, Mowszowski Loren, Glozier Nick, Duffy Shantel, Gibson Alice A, and Naismith Sharon L. 2017. Examining Internet and eHealth Practices and Preferences: Survey Study of Australian Older Adults With Subjective Memory Complaints, Mild Cognitive Impairment, or Dementia. Journal of Medical Internet Research 19, 10: e358. 10.2196/jmir.7981 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [58].Lazar Jonathan, Kumin Libby, and Feng Jinjuan Heidi. 2011. Understanding the computer skills of adult expert users with down syndrome: an exploratory study. In The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility (ASSETS ’11), 51–58. 10.1145/2049536.2049548 [DOI] [Google Scholar]
  • [59].Leo Raphael J.. 1999. Competency and the Capacity to Make Treatment Decisions: A Primer for Primary Care Physicians. Primary Care Companion to The Journal of Clinical Psychiatry 1, 5: 131–141. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [60].Lewis Clayton, Sullivan James, and Hoehl Jeffery. 2009. Mobile Technology for People with Cognitive Disabilities and Their Caregivers – HCI Issues. In Universal Access in Human-Computer Interaction. Addressing Diversity (Lecture Notes in Computer Science), 385–394. 10.1007/978-3-642-02707-9_44 [DOI] [Google Scholar]
  • [61].Lindsay Stephen, Brittain Katie, Jackson Daniel, Ladha Cassim, Ladha Karim, and Olivier Patrick. 2012. Empathy, participatory design and people with dementia. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI ’12, 521. 10.1145/2207676.2207749 [DOI] [Google Scholar]
  • [62].Lopes Philippe, Pino Maribel, Carletti Giova, Hamidi Sofana, Legué Sylvie, Kerhervé Helene, Benveniste Samuel, Andéol Guillaume, Bonsom Pierre, Reingewirtz S, and Rigaud Anne-Sophie. 2016. Co-Conception Process of an Innovative Assistive Device to Track and Find Misplaced Everyday Objects for Older Adults with Cognitive Impairment: The TROUVE Project. IRBM 37, 2: 52–57. 10.1016/j.irbm.2016.02.004 [DOI] [Google Scholar]
  • [63].Lotfi Ahmad, Langensiepen Caroline, Mahmoud Sawsan M., and Akhlaghinia MJ. 2012. Smart homes for the elderly dementia sufferers: identification and prediction of abnormal behaviour. Journal of Ambient Intelligence and Humanized Computing 3, 3: 205–218. 10.1007/s12652-010-0043-x [DOI] [Google Scholar]
  • [64].Lyons Kent, Skeels Christopher, and Starner Thad. 2005. Providing support for mobile calendaring conversations: a wizard of oz evaluation of dual-purpose speech. In Proceedings of the 7th international conference on Human computer interaction with mobile devices & services (MobileHCI ’05), 243–246. 10.1145/1085777.1085821 [DOI] [Google Scholar]
  • [65].Lyons Kent, Skeels Christopher, Starner Thad, Snoeck Cornelis M., Wong Benjamin A., and Ashbrook Daniel. 2004. Augmenting conversations using dual-purpose speech. In Proceedings of the 17th annual ACM symposium on User interface software and technology (UIST ’04), 237–246. 10.1145/1029632.1029674 [DOI] [Google Scholar]
  • [66].Malinowsky Camilla, Almkvist Ove, Kottorp Anders, and Nygård Louise. 2010. Ability to manage everyday technology: a comparison of persons with dementia or mild cognitive impairment and older adults without cognitive impairment. Disability and Rehabilitation: Assistive Technology 5, 6: 462–469. 10.3109/17483107.2010.496098 [DOI] [PubMed] [Google Scholar]
  • [67].McKay Dana and Miller Charlynn. 2021. Standing in the Way of Control: A Call to Action to Prevent Abuse through Better Design of Smart Technologies. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–14. 10.1145/3411764.3445114 [DOI] [Google Scholar]
  • [68].Meiland Franka, Bouman Ans, Sävenstedt Stefan, Bentvelzen Sanne, Davies Richard, Mulvenna Maurice, Nugent Chris, Moelaert Ferial, Hettinga Marike, Bengtsson Johan, and Dröes Rose-Marie. 2012. Usability of a new electronic assistive device for community-dwelling persons with mild dementia. Aging & Mental Health 16, 5: 584–591. 10.1080/13607863.2011.651433 [DOI] [PubMed] [Google Scholar]
  • [69].Microsoft. 2022. Send an email message with a follow-up reminder. Microsoft Support. Retrieved June 24, 2022 from https://support.microsoft.com/en-us/office/send-an-email-message-with-a-follow-up-reminder-740a3b9e-e837-4711-938a-08dd0ea5ac64
  • [70].Mori Masahiro, MacDorman Karl F., and Kageki Norri. 2012. The Uncanny Valley [From the Field]. IEEE Robotics Automation Magazine 19, 2: 98–100. 10.1109/MRA.2012.2192811 [DOI] [Google Scholar]
  • [71].Mueller Kimberly D., Hermann Bruce, Mecollari Jonilda, and Turkstra Lyn S.. 2018. Connected speech and language in mild cognitive impairment and Alzheimer’s disease: A review of picture description tasks. Journal of Clinical and Experimental Neuropsychology 40, 9: 917–939. 10.1080/13803395.2018.1446513 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [72].Mulvenna Maurice, Martin Suzanne, Sävenstedt Stefan, Bengtsson Johan, Meiland Franka, Dröes Rose Marie, Hettinga Marike, Moelaert Ferial, and Craig David. 2010. Designing & evaluating a cognitive prosthetic for people with mild dementia. In Proceedings of the 28th Annual European Conference on Cognitive Ergonomics - ECCE ’10, 11. 10.1145/1962300.1962306 [DOI] [Google Scholar]
  • [73].Nandigam David, Symonds Judith, Kayes Nicola, and McPherson Kathryn. 2010. Mobile phone user interface design for patients with traumatic brain injury. In Proceedings of the 11th International Conference of the NZ Chapter of the ACM Special Interest Group on Human-Computer Interaction (CHINZ ’10), 69–72. 10.1145/1832838.1832850 [DOI] [Google Scholar]
  • [74].National Institute of Aging. 2021. Alzheimer’s Disease Fact Sheet. National Institute on Aging. Retrieved October 11, 2021 from http://www.nia.nih.gov/health/alzheimers-disease-fact-sheet
  • [75].Norman Donald. 1999. Affordance, conventions, and design. Interactions 6: 38–42. 10.1145/301153.301168 [DOI] [Google Scholar]
  • [76].Norman Donald A. and Draper Stephen W. (eds.). 1986. User Centered System Design: New Perspectives on Human-computer Interaction. CRC Press, Hillsdale, NJ. [Google Scholar]
  • [77].Nugent Chris D., Davies Richard J., Donnelly Mark P., Hallberg Josef, Hariz Mossaab, Craig David, Meiland Franka, Moelaert Ferial, Bengtsson Johan E., Savenstedt Stefan, Mulvenna Maurice, and Droes Rose-Marie. 2008. The development of personalised cognitive prosthetics. In 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 787–790. 10.1109/IEMBS.2008.4649270 [DOI] [PubMed] [Google Scholar]
  • [78].Nygård Louise and Starkhammar S. 2007. The use of everyday technology by people with dementia living alone: Mapping out the difficulties. Aging & mental health 11: 144–55. 10.1080/13607860600844168 [DOI] [PubMed] [Google Scholar]
  • [79].O’Connor Siobhan, Bouamrane Matt-Mouley, O’Donnell Catherine A., and Mair Frances. 2016. Barriers to Co-Designing Mobile Technology with Persons with Dementia and Their Carers. Nursing Informatics. 10.3233/978-1-61499-658-3-1028 [DOI] [PubMed] [Google Scholar]
  • [80].Orpwood Roger, Gibbs Chris, Adlam Timothy, Faulkner Ricahrd, and Meegahawatte D. 2005. The design of smart homes for people with dementia—user-interface aspects. Universal Access in the Information Society 4, 2: 156–164. 10.1007/s10209-005-0120-7 [DOI] [Google Scholar]
  • [81].Pielot Martin, Church Karen, and de Oliveira Rodrigo. 2014. An in-situ study of mobile phone notifications. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services - MobileHCI ’14, 233–242. 10.1145/2628363.2628364 [DOI] [Google Scholar]
  • [82].Poole Matthew L., Brodtmann Amy, Darby David, and Vogel. 2017. Motor Speech Phenotypes of Frontotemporal Dementia, Primary Progressive Aphasia, and Progressive Apraxia of Speech. Journal of Speech, Language, and Hearing Research 60, 4: 897–911. 10.1044/2016_JSLHR-S-16-0140 [DOI] [PubMed] [Google Scholar]
  • [83].Reicherts Leon, Zargham Nima, Bonfert Michael, Rogers Yvonne, and Malaka Rainer. 2021. May I Interrupt? Diverging Opinions on Proactive Smart Speakers. In CUI 2021 - 3rd Conference on Conversational User Interfaces (CUI ’21), 1–10. 10.1145/3469595.3469629 [DOI] [Google Scholar]
  • [84].Riikonen Merja, Paavilainen Eija, and Salo Hannu. 2013. Factors supporting the use of technology in daily life of home-living people with dementia. Technology & Disability 25, 4: 233–243. 10.3233/TAD-130393 [DOI] [Google Scholar]
  • [85].Rosenberg Lena, Kottorp Anders, Winblad Bengt, and Nygård Louise. 2009. Perceived difficulty in everyday technology use among older adults with or without cognitive deficits. Scandinavian Journal of Occupational Therapy 16, 4: 216–226. 10.3109/11038120802684299 [DOI] [PubMed] [Google Scholar]
  • [86].Rowe Meredeth A. and Bennett Vikki. 2003. A look at deaths occurring in persons with dementia lost in the community. American Journal of Alzheimer’s Disease & Other Dementias® 18, 6: 343–348. 10.1177/153331750301800612 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [87].Sakai Yoichi, Nonaka Yuuko, Yasuda Kiyoshi, and Nakano Yukiko I.. 2012. Listener agent for elderly people with dementia. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction (HRI ’12), 199–200. 10.1145/2157689.2157754 [DOI] [Google Scholar]
  • [88].Sarne-Fleischmann Vardit, Tractinsky Noam, Dwolatzky Tzvi, and Rief Inbal. 2011. Personalized reminiscence therapy for patients with Alzheimer’s disease using a computerized system. In Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA ’11), 1–4. 10.1145/2141622.2141679 [DOI] [Google Scholar]
  • [89].Shneiderman Ben. 2022. Human-Centered AI. Oxford University Press. [Google Scholar]
  • [90].Sixsmith Andrew, Gibson Grant, Orpwood Roger, and Torrington Judith. 2007. Developing a technology ‘wish-list’ to enhance the quality of life of people with dementia. Gerontechnology 6. 10.4017/gt.2007.06.01.002.00 [DOI] [Google Scholar]
  • [91].Suárez-González Aida, Henley Susie M., Walton Jill, and Crutch Sebastian J.. 2015. Posterior Cortical Atrophy: An Atypical Variant of Alzheimer Disease. Psychiatric Clinics 38, 2: 211–220. 10.1016/j.psc.2015.01.009 [DOI] [PubMed] [Google Scholar]
  • [92].Tanaka Hiroki, Adachi Hiroyoshi, Ukita Norimichi, Kudo Takashi, and Nakamura Satoshi. 2016. Automatic detection of very early stage of dementia through multimodal interaction with computer avatars. In Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI ’16), 261–265. 10.1145/2993148.2993193 [DOI] [Google Scholar]
  • [93].The Health Research Authority. 2005. Mental Capacity Act. Retrieved June 29, 2021 from https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/mental-capacity-act/
  • [94].Tiersen Federico, Batey Philippa, Harrison Matthew J. C., Naar Lenny, Serban Alina-Irina, Daniels Sarah J. C. and Calvo Rafael A.. 2021. Smart Home Sensing and Monitoring in Households With Dementia: User-Centered Design Approach. JMIR Aging 4, 3: e27047. 10.2196/27047 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [95].Tiong Esther Y. C., Powers David M. W., and Maeder Anthony J.. 2018. Dementia virtual assistant as trainer and therapist: identifying significant memories and interventions of dementia patients. In Proceedings of the Australasian Computer Science Week Multiconference (ACSW ’18), 1–9. 10.1145/3167918.3167953 [DOI] [Google Scholar]
  • [96].Tsoi Kelvin KF, Lam Max WY, Chu Christopher TK, Wong Michael PF, and Meng Helen ML. 2018. Machine Learning on Drawing Behavior for Dementia Screening. In Proceedings of the 2018 International Conference on Digital Health (DH ’18), 131–132. 10.1145/3194658.3194659 [DOI] [Google Scholar]
  • [97].Vanderheiden Gregg, Lazar Jonathan, Jordan J. Bern, Ding Yao, and Wood Rachel E.. 2020. Morphic: Auto-Personalization on a Global Scale. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–12. Retrieved January 27, 2022 from 10.1145/3313831.3376204 [DOI] [Google Scholar]
  • [98].Vanderheiden Gregg, Treviranus Jutta, Usero Josée Angel Martinez, Bekiaris Evangelos, Gemou Maria, and Chourasia Amrish. 2012. Auto-Personalization: Theory, Practice and Cross-Platform Implementation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 56: 926–930. 10.1177/1071181312561193 [DOI] [Google Scholar]
  • [99].W3C Working Group. 2021. Making Content Usable for People with Cognitive and Learning Disabilities. Retrieved March 4, 2022 from https://www.w3.org/TR/coga-usable/#dfn-cognitive-and-learning-disabilities
  • [100].Wan Lin, Müller Claudia, Randall Dave, and Wulf Volker. 2016. Design of A GPS Monitoring System for Dementia Care and its Challenges in Academia-Industry Project. ACM Transactions on Computer-Human Interaction 23, 5: 1–36. 10.1145/2963095 [DOI] [Google Scholar]
  • [101].Wan Lin, Müller Claudia, Wulf Volker, and Randall David William. 2014. Addressing the subtleties in dementia care: pre-study & evaluation of a GPS monitoring system. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems -CHI ’14, 3987–3996. 10.1145/2556288.2557307 [DOI] [Google Scholar]
  • [102].Wei Jing, Dingler Tilman, and Kostakos Vassilis. 2021. Developing the Proactive Speaker Prototype Based on Google Home. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1–6. Retrieved March 4, 2022 from 10.1145/3411763.3451642 [DOI] [Google Scholar]
  • [103].Wei Jing, Dingler Tilman, and Kostakos Vassilis. 2022. Understanding User Perceptions of Proactive Smart Speakers. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 4: 185:1–185:28. 10.1145/3494965 [DOI] [Google Scholar]
  • [104].Eleanor Bantry White and Paul Montgomery. 2015. Dementia, walking outdoors and getting lost: incidence, risk factors and consequences from dementia-related police missing-person reports. Aging & Mental Health 19, 3: 224–230. 10.1080/13607863.2014.924091 [DOI] [PubMed] [Google Scholar]
  • [105].Wobbrock Jacob O., Kane Shaun K., Gajos Krzysztof Z., Harada Susumu, and Froehlich Jon. 2011. Ability-Based Design: Concept, Principles and Examples. ACM Transactions on Accessible Computing 3, 3: 9:1–9:27. 10.1145/1952383.1952384 [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material

RESOURCES