User-participatory development of FindMyApps; a tool to help people with mild dementia find supportive apps for self-management and meaningful activities

Short abstract

Objective

There is growing evidence that hand-held touchscreen devices (tablets) can support people with mild dementia to manage their life and engage in meaningful activities. However, as it can be difficult to find apps that match one’s personal needs, wishes and abilities, a person-centred selection tool was developed, called FindMyApps.

Method

To ensure its usability, the FindMyApps selection tool was developed using a ‘user-participatory design’ in which users (persons with dementia and informal carers), and experts (designers, developers and researchers) closely collaborated. In three short iterative rounds – so called ‘sprints’ – the users were invited to test whether the prototypes harmonised with their needs, wishes and abilities.

Results

Each sprint provided insight into potential improvements of the tool. The development team gained an understanding of issues regarding usefulness (e.g. meaningful content of (sub)categories for apps in domains of self-management and meaningful activities), as well as issues to increase the user-friendliness (e.g. intuitive design with instructive navigation support).

Conclusion

The FindMyApps selection tool was conceived as a means to make it easier for people with mild dementia to select apps meeting their needs, wishes and abilities. This provisional end version will be further tested, and, if necessary, improved in a feasibility study.

Introduction

Research shows that 70% of people with dementia stop engaging in activities due to lack of confidence, 50% avoid the neighbourhood and 40% hardly leave their home.¹ Community-dwelling people with dementia and their informal carers report a lack of meaningful activities during the day.^2–5 In the early stage of the disease, people experience insufficient support from professional caregivers and health care services to compensate for self-management disabilities.⁶ The inability to organise one’s own affairs can be very difficult to accept.⁷ As many as 54% of carers feel rather heavily burdened, and 9% feel very heavily burdened or overburdened.⁸ This high burden of carers frequently results in admission of the person with dementia to a long-term care facility.⁹

The current policy in Western countries is to enable people with dementia to live in their own home for as long as possible.¹⁰ This implies that they will need to adapt to and deal with the consequences of dementia in their daily life in so far as that is possible.¹¹ Touchscreen technologies, such as hand-held touchscreen devices (tablets), are promising tools to support people with mild dementia in their ability to self-manage and engage in meaningful activities.^12,13 In the last decade, many applications (apps) for tablets have been developed to support people in managing their daily life and health, staying in touch with their social network and engaging them in activities.¹⁴ Although people with dementia need support to learn how to use touch screen devices,^15–21 there is growing evidence that apps also have potential to support people with mild dementia for these purposes.^{15–20,22–28} In the areas of self-management, for example, there are apps for timely medication intake (MedAlert), social contact (Nextdoor, Skype), daily structure (Dementia app, Pictoplanner), navigation (Blokje Om), language and communication support (Dario App) and a variety of apps for meaningful activities (memory training, art, history, reminiscence, music, games).

However, careful selection is required, as only a small fraction of existing apps are usable for individuals with dementia.^{16,20,29–31} One could formulate a ‘top 10’ of most suitable apps for people with mild dementia to match the needs, wishes and abilities of the person with dementia. Yet this would make no sense since needs, wishes and abilities can differ strongly between people due to specific disabilities and personal, social and environmental factors.³² Moreover, with the rapid development of new apps, this top 10 would quickly be out of date. Nowadays, the use of apps on hand-held touchscreen devices is becoming an integral part of everyday life, also among the older generation.³³ This increases the need to design digital systems that can be used by all, regardless of physical or cognitive impairments.³⁴

The present study therefore intended to develop an interactive selection tool (web application) that would be able to find apps for self-management and meaningful activities that suit individual needs, wishes and abilities of people with dementia. This paper aims to provide a clear insight into this developmental process. The specific goal of the selection tool is to match the preferences and abilities of people with dementia to specific features and app types in order to improve and, ultimately, promote their app usage for the purposes of self-management and engaging in meaningful activities.

Within the context of self-management and meaningful activities, the tool was based on an extensive inventory of the functional selection criteria of apps that related to the activities in which persons with dementia wished to engage, and an inventory of the technical selection criteria that related to their ability to use apps.³⁵ The tool will be used together with an introductory training to help people with dementia and family carers to learn to use the tablet, part of an innovative person-centred tablet intervention called FindMyApps.³⁶

To develop the FindMyApps selection tool, the following research question was addressed:

How to develop a selection tool that helps people with dementia find suitable apps for self- management and meaningful activities (selection criteria) that match their individual needs, wishes and abilities (user profile)?

Methods

Research design

To develop the FindMyApps selection tool, we used a user-participatory design.^37,38 Qualitative research methods were applied to identify needs of users and usability issues (usefulness and user-friendliness). Usefulness refers, among other things, to whether users believe that a website or application fulfil specific needs or whether it helps them to be more effective and productive.³⁹ User-friendliness (ease of use) refers, among other things, to whether users believe that using a website or application will be easy and simple to use.³⁹ The development of the FindMyApps selection tool was inspired by the User Experience (UX) design and Agile methodology. The overall rationale behind UX design is that knowledge comes from user experience; design decisions are made based on how users interact with the design.⁴⁰ Agile is a set of methods that help a team to think more effectively, work more efficiently, and make better decisions.⁴¹ Derived from Agile methods, the working process was structured according to the Scrum and Kanban method. The Scrum method is an innovative method to design and evaluate a temporary product in short iterative sprints.⁴¹ The Kanban method adds that every sprint starts with transparency about all the ‘to do’ actions and the development team deciding, based on the aim to limit the work in progress, which actions are to be achieved in one sprint.⁴¹

Three iterative sprints

Based on the Scrum method and user-participatory methods,^41–43 the following phases represent one sprint:

• Phase I: Design, collecting theoretical data: needs, wishes and abilities of users (user requirements) with regard to the desired design;

• Phase II: Development, converting these data (user requirements) in mock-ups;

• Phase III:Usability tests, testing to ensure that the selection tool meets the user requirements;

• Phase IV: Discover usability issues to improve the selection tool and discover needs for further development (adapting user requirements or creation of new user requirements).

The progression from one sprint to another was conducted as an iterative process, i.e. returning to the phase of design and development based on feedback or new information collected during a sprint. This provided the opportunity to optimise the selection tool constantly according to the needs, wishes and abilities of users. The three sprints were conducted over a 9-month period, from March 2017 to November 2017 (see Figure 1).

Figure 1.

Development of FindMyApps selection tool in sprints (green phases carried out during this study).

During all sprints, researchers from Saxion University, developers and designers from a software company worked together (development team) with potential users (community-dwelling people with mild dementia, informal and formal carers), experts in person-centred dementia care and information and communication technology (ICT) experts (expert team). The result of phase II in the third sprint is a provisional end version of the FindMyApps selection tool. Although originally three complete sprints were pre-planned, sprints 1 and 2 provided us with enough rich data about how the users interacted with the tool. On the basis of these data, new improved prototypes were developed (phase I and II of sprint 3). Subsequently, additional information collection will be needed on more sustainable usability issues. It is expected that users can provide this information only after having used the tool for a longer period of time. We will investigate these usability issues in a controlled pre-posttest feasibility study into FindMyApps (phase III and IV of sprint 3) that is yet to be conducted and will be described in a separate paper.

Methods phases I and II: Design and development

The development team used Confluence – project management software. In Confluence, it is possible to plan and register the actions of sprints, to chat with each other, to share documents and notes of meetings and to present prototypes. The development team met every 2 weeks. They discussed the design and content of the FindMyApps tool with an expert team on a monthly basis. During the first meeting with the expert team, the major principles of the tool were established on the basis of the scientific literature,^6,7,44 our previous study and best practices as well as the intended functionalities in FindMyApps.^35,45–47 The developers and designers from the software company translated these principles into a programme of requirements. Mock-ups were created in each sprint, and these were tested in order to assess whether they met the requirements. The development team and expert team also discussed the additional wishes and needs and inventoried these in order to improve the tool’s prototypes. This process continued until the prototype met its predetermined requirements. A cognitive walkthrough took place within the second (n = 4) and third (n = 5) sprint by both researchers and experts. They explored the tool by performing a series of tasks (assignments) aimed at identifying potential usability problems that could impede the successful completion of a task.⁴⁸ A task is a realistic example of how users can use the tool. This cognitive walkthrough was not performed in sprint 1 because we wanted to record the first spontaneous reactions of participants. After integrating all feedback from a sprint and redesigning the tool to address the usability issues from the cognitive walkthrough, the next phases started, i.e. usability testing with potential users and discovering new usability issues.

Methods phases III and IV: Usability testing and discovering usability issues

Setting and participants

Participants for the usability tests were recruited with help of two Meeting Centres for people with dementia and carers, and one day-care centre for people with dementia, all located in the eastern part of The Netherlands (Enschede and Doetinchem). Inclusion criteria were community-dwelling care-dependent people with mild dementia, with and without a confirmed diagnosis. Inclusion criteria for the informal carers were caring for a person with possible dementia in an earlier stage. For the usability tests of the FindMyApps selection tool, participants had to be willing and able to perform a usability test on a tablet (to trial the tool). Two participants with dementia who volunteered to work with us lacked the ability to perform the tests and their input was not used in our research.

In the first and second sprint eight persons with dementia, eight informal carers and two formal carers participated in the usability tests.

Qualitative methods and study procedure

The prototypes of the selection tool that resulted from phases I and II of sprints 1 and 2, respectively, were installed on tablets. The method of ‘scenario-based testing’ was applied during the usability tests.⁴⁸ The scenarios (three in sprint 1 and five in sprint 2) concerned realistic examples of how users may carry out tasks in a specific context with the tool.⁴⁸ An example of a scenario was: ‘In your younger years you had a special interest in painting. Now you would like to paint again. We invite you to search for an app that may support you in performing this activity’. The Three Step Test-Interview (TSTI) principle was applied within each scenario to identify usability issues with regard to the content and design of the FindMyApps tool. The TSTI method consists of three steps:

1. (Respondent-driven) observation of respondent behaviour as they use the tool while ‘thinking aloud’;

2. (Interviewer-driven) retrieval of additional data by follow-up probing aimed at remedying gaps in observational data;

3. (Interviewer-driven) validation via semi-structured debriefing aimed at eliciting experiences and opinions with regard to tool.⁴⁹

The TSTI method is usually performed by completing every step for each scenario and subsequently going to the next step for each scenario.⁴⁹ However, because of the memory problems of the target group, the TSTI was adapted by performing all three steps per scenario consecutively. In sprint 2, this method was further adapted as we noticed their (limited) short-term memory made it difficult for the target group to share their experiences and opinions afterwards. So, for a more natural sequence we decided to combine steps 2 and 3. Perceiving difficulties in the target group’s ability to imagine the standard scenarios, we adapted the scenario-based testing method in more realistic assignments closer to the individual users’ preferences and interests. An example of an assignment was: ‘Which activity would you like to do? Try to find an app for that activity’.

The interviews were conducted by two researchers (YK and MPK), in the roles of primary interviewer and observer, the latter being responsible for reporting the interview observations. The researchers alternated the roles of interviewer and observer. All interviews were videotaped to capture the full context of the interviews, in particular how the users interacted with the design while they were using the selection tool on the tablet.

To provide an in-depth understanding of the results, quotes supporting the observed behaviour of participants were included. To ensure confidentiality and anonymous presentation of the data all participants were given a number (1–8) with letters to identify whether they were a person with dementia (PwD), an informal carer (IC) or a formal carer (FC). For this article we selected extracts, based on what was most illustrative for the development of the selection tool, to illustrate the main content of the data.

Data analysis

Three types of data were collected during the usability tests and used for the analysis:

1. participant characteristics and tablet/smartphone experiences by means of a short questionnaire;

2. observed behaviour of participants during the interview reported in notes and on video-tape;

3. detailed descriptions of participant observations per scenario/assignment in Microsoft Excel.

These descriptions were based on analysis of the videos and the notes on observed behaviour of participants during the interviews. A deductive analysis was performed, as we were particularly interested in the usability (usefulness and user-friendliness) and needs for further improvement of the FindMyApps selection tool. The usability tests, which contained pre-defined scenario’s and assignments for participants, led to participant reactions that required the use of a deductive analysis procedure. This approach matches the study’s specific interest regarding issues that could help to improve the usability of the tool. The following observations and analyses were applied during the sprints: 1) Two researchers (YK and MPK) independently noted any relevant behaviours that were observed during the usability tests and also made notes whilst viewing the video tapes. Both the transcribed statements as noted behaviours as in free text descriptions, were then coded by both researchers. This procedure of independent coding was followed by a consensus meeting. 2) The researchers mutually grouped the codes into themes. 3) Researchers YK and MPK sent the developers and designers the so-called condensed raw data in an Excel document, which clearly explained the process from the coding to the themes. The developers and designers had instructed the researchers to only present the observed and transcribed behaviours and to refrain from providing possible solutions for the barriers that the users had faced, as this would have influenced the designers’ creative solution process. An example of one such theme was called: input for improving icons. 4) As a last step, the researchers divided the inventoried themes into two main categories, i.e. themes that were relevant to the tool’s content (self-management and meaningful activities), and themes that were relevant to the tool’s design.

A summary of the findings of the usability tests was sent to the software company and discussed in a meeting with the development team. Subsequently, directions were formulated to improve the tool, which were converted into actions for the next sprint.⁴¹

Ethical considerations

The Medical Ethics Committee of the VU University Medical Centre in Amsterdam approved the study protocol. After receiving written and oral information about the research and prior to participation in the usability tests, all participants signed an informed consent form. During the tests with persons with dementia we performed an on-going consent procedure by regularly asking them if they were still comfortable with their participation.⁵⁰ We created a safe environment by spending time getting to know the people, giving them positive feedback, emphasising the importance of their participation, recognising signs of discomfort,⁵⁰ and through the use of a written time schedule (A3 paper size) that allowed participants to see what was going to happen at any given time.

Results

Participant characteristics for sprints 1 and 2 of the usability tests

Both usability tests involved the participation of four community-dwelling persons with mild dementia or mild cognitive impairment (MCI) (one female, seven males; mean age 78.6 years; range 72–86). Five of these participants had Alzheimer’s disease, two had Frontotemporal Dementia and one had MCI. Most of the participants were married, had graduated from college and had experience with using tablets or computers. Five informal carers participated in sprint 1 of the usability tests while three informal carers participated in sprint 2 (four females, four males; mean age 73.1, range 58–82). Most of these participants were married to persons with dementia who had Alzheimer’s disease and each of them had graduated from college and had experience with either tablets or computers. One formal carer participated in each of the usability tests (two females; aged 48 and 54). Both participants had graduated from college and had experience with using tablets.

Results, sprint 1

Results, design and development (phases I and II)

Table 1 presents the decisions regarding the major principles for the basic conditions, content and design of the tool, made by the development and expert teams during the first meetings. However, as user experience with the content of the tool was highly relevant, these principles needed to be confirmed and specified during the development.

Table 1.

Basic conditions, content and design of the tool.

Basic conditions	Content tool	Design tool
A person-centred selection of apps: The tool will ask questions about the individual needs, wishes and abilities of persons with dementia in the area of self-management and meaningful activities (user profile) and match their answers with specific apps (selection criteria).	Determine what (sub)categories of self-management and meaningful activities are most appropriate for the tool and determine how to organise these activities. This was based on functional selection criteria of apps,35 articles about self-management programmes for people with early dementia,6,7,44 the Activity Card Sort,45 and daily activities for young people with dementia.46	Make prototypes right from the start, using a responsive (touchscreen based) website, to try what works and what does not. The website will be made for Android and iOS.
Persons with mild dementia may need help from an informal carer to set the user profile, select and download apps. They can use apps on the tablet independently.	Determine technical selection criteria for rating dementia-friendly apps for the (sub)categories of self-management and meaningful activities, based on technical selection criteria of apps,35 and the App Selection Framework Guidance Manual.47 Next, rate available apps for self-management and meaningful activities on these determined selection criteria. Most of the apps were found on sites of organisations related to dementia.	Design and develop different prototypes not all at once, but better to develop prototypes after each other.
Persons with dementia are seen as the real experts and this means that when they can use the interface without problems, this is the best guarantee that it works.	Determine what selection criteria (personal settings) are needed for the user profile. This was based on user characteristics addressing abilities of users in terms of their physical and cognitive condition and tablet skills.35	The tool will consist of a front-end (looks like (in landscape mode)) and a back-end (administration and definition user profiles/app selection and user statistics). The front-end is based on essential heuristics for interfaces of people with dementia.30,35,47

In the programme of requirements for the first sprint, the following user-interface aspects were prioritised:

1. The organisation of (sub)categories of apps for self-management and meaningful activities and navigation within these (sub)categories;

2. The presentation of apps within a (sub)category;

3. The navigation to the user profile and to change the personal settings, e.g. letter size of apps and letter size of the tool.

Mock-ups and prototypes were created and subsequently tested by the researchers. Prototype 1.3 (see Figure 2 and Table 2 for a description of the prototype) was ready to be trialled in usability tests.

Figure 2.

Prototype 1.3 selection tool as a result of phases I and II of sprint 1 © Bejoy Mobile 2016 and © Steve Sprang (Christoffer Hoel).

Table 2.

Description of prototypes that had been provided for testing with users.

Sprint 1Prototype 1.3	Front-end
	Pages: A search page (‘Zoeken’) with 10 categories on the left side with text, icons of self-management and meaningful activities (hobbies, going out, exercise, nature, social contact, in and around the house, memories, about dementia, memory support and safety). By pressing a category and subcategories (in text blocks) apps became visible. A (sub) could also be searched in the search bar at the top of the screen. A page with an overview of the most frequently used apps per category (‘Mijn apps’). A page for setting the user profile at the bottom of the screen (‘Mijn profiel’). A help button at the top right-hand corner of the screen (not functional in this prototype).
	Navigation to other pages using the navigation bar at the bottom
Sprint 2Prototype 2.4	Front-end
	Pages: A home page (‘Begin’) with two big buttons. One for entering the search page (‘Zoeken’) and one for entering the overview of most used apps (‘Mijn apps’). Search pages (‘Zoeken’): A page presenting 10 categories with text and icons of self-management and meaningful activities (in and round the house, social contact, hobbies, games, exercise, nature, going out, memories, reminders and safety). A page (after pressing a category) where subcategories with text and icons in the same colour of the overarching category become visible. A page (after pressing a subcategory) with app selection. When pressing the image of the app, the choice to download can be made and the instruction video ‘how to download an app’ starts. After that a blue button becomes visible to go to Apple Store (iOS) or Google Play (Android). A page with an overview of most used apps per category (‘Mijn apps’). A page for setting the user-profile (‘Instellingen’) with two big buttons that give access to the following two pages:   ○ A page for using and changing the apps’ personal settings, which contains six questions about the personal preferences for the apps, e.g. large font size; less text and plenty of pictures; only available in Dutch; real photos; simple to operate and includes instructions.35   ○ A page for using and changing the personal settings for the FindMyApps tool, which contains questions about the personal preferences for the FindMyApps tool usage, such as choice of letter size; choice to change the icons of the categories of self-management and meaningful activities into photos, etc. A help page where users can find the instruction video ‘How to find and download an app of your interest’ and where an explanation of several pages was given. A green button (‘Hier’) on every page offered visual and audio instructions on how to navigate the page.
	Navigation to other pages using the navigation bar at the bottom with bigger and clearer navigation icons. A purple bar at the top provided on every page location information and offered the possibility to go back.
Sprint 3Prototype 3.5	Front-end
	A simple log-in page. A page for setting the user profile containing the same six questions (see prototype 2.4) about personal preferences for apps (‘Instellingen’). A page for choosing main categories and subcategories of self-management and meaningful activities (‘Ontdek’). For each subcategory a page with a selection of apps and a sentence describing each app. The best apps will be recommended with a higher score (‘Ontdek’). For each app, a page including (a) more information about the app; (b) app score details based on personal settings; (c) button for accessing the Apple Store or Google Play (‘Ontdek’). A page with an overview of most used apps arranged by subcategory (‘Mijn apps’). A help tutorial on every page (‘Uitleg’).
	Navigation to other pages using the navigation bar at the top.
	Back-end
	Consisting of: (i) a user environment where user profiles are made and where the selection of apps is carried out; (ii) an administration environment where user profiles, apps and selection criteria are defined and (iii) a reporting environment where information about the user profiles and the use of the tool (during the feasibility study and RCT) can be automatically stored.

Results, usability tests and discovered usability issues (phases III and IV)

Scenarios, based on the above described user-interface aspects, were formulated for the usability tests. The section below contains the main observations and quotes that were made by the users for each scenario. Lastly, some general observations and quotes about the tool are mentioned.

Scenario 1: ‘In your younger years you had a special interest in painting. Now you would like to paint again. We invite you to search for an app that may support you in performing this activity’

Most users with dementia needed more support to select the right (sub)category due to three problems: 1) The category names and icons were not clear and there were too many and overlapping options for subcategories. ‘Can I find an app for painting under art or under drawing and painting?’ (PwD1). ‘Too many blocks for subcategories; is too messy and creates too many stimuli for persons with dementia’ (FC1). 2) When a category was selected, apps became visible immediately. This was distracting and users then forgot to make a further selection within the subcategories. 3) Third, the boxes and letter sizes of the subcategories were too small. ‘I have trouble reading the small letters’ (PwD4).

Scenario 2: ‘You have problems with planning and maintaining overview of activities during the week. a) Could you search for an app that may support you in this? b) Choose an app that could help you the best’

None of the users directly selected the category ‘memory support’ (where the planning app can be found). Most users could not decide between ‘social contact’, ‘memories’ and ‘memory support’. Main reasons were that ‘memory support’ is a vague term and that the ‘memory support’ icon was not clear. ‘I think it is a vague term. Can’t visualise what it is. I think it is a professional term’. (IC4).

The same three problems for selecting a (sub)category were observed as in scenario 1 . Most users needed more instructions to select and download an app because of three problems: 1) It was frustrating that the descriptions of apps in the tool were too long, not clear and that they were in English. 2) Most users with dementia were not aware of the possibility to scroll down in the apps list, to see more apps. 3) Steps to be followed were not clear (i.e. enter the page of Apple Store, scroll down, download the app and go back to the tool). ‘I doubt whether this app is the right choice. I should be able to tell immediately. Now the person with dementia has to search for this information and gets discouraged. The person wants to go back, but it is not clear how to go back’ (IC4).

Scenario 3: ‘You think the letter size of the app you selected is too small. You forgot to set the preferred letter size. Adjust the preferred letter size in ‘your profile’

None of the users pressed the button ‘My profile’ in the navigation bar at the bottom because of three problems: 1) The navigation bar was not noticed because users were focussing on the 10 categories. 2) The term ‘My profile’ was not recognisable for users as a page where you can change letter sizes. ‘I have no idea’ (PwD3) (question researcher: do you have a clue what to find behind ‘My profile’). 3) The icons in the navigation bar were too small.

Once users were in the profile, most of them did not understand how to operate the personal settings and they did not understand the meaning of the personal settings. ‘I am not pressing the ‘little’ button, because then the letter sizes become even smaller’ (IC5).

General observations and quotes about the tool

Most users had problems operating the buttons of the prototype. They had to press more than once for buttons to react.

Colour use in the prototype was experienced as clear and quiet. ‘Nice quiet colours no hard colours. It has to have calm colours’ (IC3).

For most users with dementia the functionality of the tool was not immediately clear, but once they were informed they were enthusiastic about it. This was also true for the IC and FC. ‘Nice tool, a progression. When you search for an app, it is already there. That’s easy’ (PwD1). One user with dementia was curious what was in the category ‘about dementia’. It distracted her and kept her from continuing the usability tests. Another user with dementia found it confronting to see this category. I don’t look at ‘about dementia’; it is bad enough that I have it. It doesn’t bother me, but I don’t look at it (PwD3).

None of the users noticed the search bar at the top where they could type in (sub)categories of activities.

Results, sprint 2

Results, design and development (phases I and II)

Based on the discovered usability issues developers and designers of the software company translated these data into a new programme of requirements. In the second sprint it was decided to improve the following user-interface aspects:

1. To provide the users with an instruction video and with more visual and audio instructions for operating the tool on every page.

2. To make a clear distinction between the personal settings of usable apps and the personal settings for the FindMyApps tool.

3. The organisation of (sub)categories of apps for self-management and meaningful activities. Since there were too many and overlapping subcategories, it was decided to delete the subcategories for which no apps were available.

4. To present the subcategories in the same way as the categories. To make use of different colours for each category and apply those same colours to the subcategory.

5. To work on better recognisable icons/pictures for the presentation of the (sub)categories and for the navigation bar.

For point 3 we needed to start selecting dementia-friendly apps in the self-management and meaningful activities domains that were to be included in the library of the FindMyApps tool. Apps were carefully selected by students based on an assessment for dementia-friendly apps (see Table 3). This assessment was based on a set of important app criteria with regard to interaction, feedback, aesthetic design, app design, customisation, obstacles and age appropriateness.^30,47 For each app, a maximum of 30 points could be scored; apps which scored 20 points or higher were added to the FindMyApps library. Of the approximately 400 apps that were assessed, we included 180 of these in the self-management and meaningful activities domains that were dementia friendly.

Table 3.

Score sheet for assessing dementia-friendly apps.

Category	Criterion	Good(3 points)	Sufficient(2 points)	Insufficient(1 point)	Score
1. Interaction: Type of gesture	What type of gesture is needed to operate the app?	Swipe and clicks	Swipe, clicks and one other type of gesture	Other types of gesture than swipe and clicks
2. Interaction: Control panel	Does de app contain large, medium or small buttons?	Large >2 cm	Medium 1–2 cm	Small <1 cm
3. Feedback: Visual and audio	Is there visual & audio feedback when operating buttons?	Both visual and audio	Visual or audio	No visual and no audio
4. Aesthetic design: Size of text	Which size of letters is used in the app?	Large (16)	Medium (11–15)	Small (10)
5. Aesthetic design: Contrast and background	Do colours of interactive elements contrast well against background?	Both good contrast and clear background	Good contrast or clear background	No good contrast and no clear background
6. App design: Intuitive use	How recognisable are the buttons and how user-friendly is the app?	Recognisable buttons and user-friendly	Recognisable buttons or user-friendly	No recognisable buttons and not user-friendly
7. App design: Instructions/ help tutorial	Is there a help tutorial (instructions) and is this usable?	Usable help tutorial	Complex help tutorial	Not available
8. Customisation: Language	Which language possibilities are available?	Only Dutch	Dutch and English	Only English
9. Obstacles: Adverts/ light version	Does the app contain (disrupting) adverts?	No adverts and no light version	Light version	Pop-ups and adverts
10. Age appropriateness: Childish	Is the app childish?	Not childish	For both adults and children	Childish
				Total

As in sprint 1, mock-ups and prototypes were created and tested by researchers of the development team. After the release of prototype 2.3, a cognitive walkthrough took place with researchers and experts to collect additional data concerning the usability of the prototype. Assignments for the cognitive walkthrough were based on the above user-interface aspects. Data was collected to improve the tool, e.g. change of icons; change of text for personal settings; workable audio instructions; the presence of a scroll bar within the presentation of apps and on the page for the overview of most used apps per category. The cognitive walkthrough also showed how to improve the assignments for the usability tests, e.g. the sequence of assignments, and to add an assignment about how the user experienced the download process of apps from the tool. Feedback was incorporated in prototype 2.4 and ready to be tested in usability tests (see Figure 3 and Table 2).

Figure 3.

Prototype 2.4 selection tool as a result of phase I and II of sprint 2.

Results, usability tests and discover usability issues (phases III and IV)

Assignments of cognitive walkthrough were adapted for the usability tests. The section below contains the main observations and quotes that were made by the users for each assignment.

Assignment 1: ‘Find and look at the instruction video in ‘Help’.

All the users with dementia and most of the IC did not find the instruction video in ‘help’ without prompts from the researcher. For all users with dementia it was not clear that it was an instruction video. While the video was playing the users were responding to questions asked in the video, by saying something or by pressing buttons of FindMyApps. All IC and FC felt the video contained too much information for people with dementia. That it was too fast and that the letter font used in the video was too small. ‘It is complicated, too much information at once. You want to look, but it is too small and too much. You want to read but the video continues’ (IC7).

Some IC noticed that other parts of the ‘Help’ page concerning an explanation of several pages in FindMyApps offered too little support when users got stuck.

Assignment 2: a) ‘Which activity would you like to do? Try to find an app for that activity’. b) ’What can you do to download the app’?

Most users did not have problems on the Home page when making a choice to enter the Search page. Most users with dementia knew that they had to press a coloured button to select an activity but had problems finding the activity of their interest because categories and related subcategories were not clear to them and there were to many options to choose for. ‘Playing football is a social activity, so that’s why I pressed social contact’ (PwD7).

Most users needed more instructions to choose and download an app because of five problems: 1) The scroll down instruction was not noticed in the list of apps. 2) It was frustrating that the descriptions of apps in the tool were too long, not clear and that they were in English. ‘It has taken more time to read than I want (refers to text of the app). It is totally unclear to me. Also, after reading the text, I don’t understand the meaning of the app’ (PwD6). 3) It was not clear for some users that they had to press the image of the app. 4) The instruction video ‘how to find and download an app of your interest’ was confusing and not necessary. ‘I am not interested in the video if I want to look at birds. I don’t need it here’ (PwD8). 5) The blue button to go to Apple Store or Google Play was not noticed because it disappeared too quickly.

Assignment 3: The researcher is showing the page of categories of self-management and meaningful activities on the screen. a) ‘In FindMyApps it is possible to change the icons of the activities into photos. The page personal settings of FindMyApps can support you with that, will you please try to do this’? b) ‘Can you turn this question ON and OFF’?

All users had difficulty understanding and changing this setting because of three problems:

1) The difference between the personal settings of the FindMyApps tool and the personal settings of the apps was not clear. 2) They did not know how to operate the settings, turn it ON or OFF. ‘Is the setting ON or OFF’? (PwD5). 3) Once they turned it ON, it was not clear for them where to look for the result of their action. ‘And now I don’t know, it is not clear. I turned the photos on, but nothing happened’ (IC6).

Assignment 4: The researcher is opening the page for the personal settings of apps. a) ‘We are now in the page for the settings of apps. Do you have a clue what the content of these questions is? b) ‘Can you turn these questions ON and OFF’?

The same two problems occurred as in assignment 3 (see points two and three). In addition, most users had trouble understanding the content of these questions because they were not clear. ‘What can I say about this setting, I don’t understand’ (PwD5).

Assignment 5: The researcher is showing the page for the categories of self-management and meaningful activities on the screen. a) ‘What photos are clear/not clear to you’? b) ‘What icons are clear/not clear to you’?

Most users found the photos for the categories clearer than the icons. Most unclear icons for categories were those for: social contact, games, going out, memories, reminders and safety. Most unclear photos for categories were those for: exercise, going out, reminders and safety. ‘I know that these are self-adhesive memos but I would not call those reminders. I do not recognise that in yellow memos’ (PwD6).

General observations and quotes about the tool

Most users had problems operating the buttons of the prototype. They had to press more than once for buttons to react.

The green instruction button (‘Hier’) to get visual and audio instructions how to operate the pages was not noticed, not clear, distracting or confusing. Some users thought it was the home button. ‘Oh, I didn’t even see it’ (IC8).

Most users did not use the navigation bar at the bottom and most users did not notice the ‘go back’ possibility in the purple bar at the top of the screen.

Colour use in the prototype was experienced as clear and calm. ‘Colour use is clear. Quite visible with different colours and much better than white, black and grey’ (IC6).

The formal carer wondered whether some apps were suitable for this generation. ‘The current generation is not familiar with food service at home, so what about the suitability of apps for that’? (FC2).

Results, sprint 3

Results, design and development (phases I and II)

The usability tests provided lots of ideas to improve the simplicity of the FindMyApps selection tool. It was therefore decided in the third sprint to improve the following user-interface aspects:

1. To reorganise the (sub)categories of apps for self-management and meaningful activities. This concerned work on the grouping hierarchy (with fewer possible choice options on one page), the icons and the titles of (sub)categories.

2. To make the personal settings, help instructions and download process of apps more user-friendly.

3. To improve the supply of apps and to provide the users with short and clear information about the apps.

4. To work on the back-end of the tool where user profiles are made and defined and where the selection of apps is performed and defined so that dementia-friendly apps can be submitted. Furthermore, to work on a user-friendly presentation of tool usage by users of FindMyApps (analytics).

Due to practical issues, such as photo copyrights, it was decided to work on dementia-friendly icons instead of photos for (sub)categories of self-management and meaningful activities. So, after reorganising the group hierarchy, additional tests with 10 persons with mild dementia (mostly Alzheimer Dementia, mean age 74, range 68–85) were performed to check whether the selected icons for these (sub)categories were recognised by the majority of users as representative of these activities. This was done in a Meeting Centre in Enschede. All titles and icons of a category and its subcategories were presented on tables, while the person with dementia was invited to walk by the table and try to match each title with the icon he or she thought fitted best. At the end a photo was made of the result (see photo 1).

The photos of the additional tests were analysed by counting the correct and incorrect titles given to icons of (sub)categories of self-management and meaningful activities. This was processed in tables and sent to the designer of the software company as input for improvement.

Next, mock-ups and prototypes were created and tested by researchers of the development team. After the release of prototype 3.4, a cognitive walkthrough took place with researchers and experts to collect additional needs and wishes concerning the usability of the prototype. Assignments for the cognitive walkthrough were based on the first three points of the user-interface aspects described above. Data was collected to improve the tool, e.g. change of icons for subcategories; change of text for personal settings; improve navigation to go back within subcategories so that search did not have to start from the beginning and improve the supply of apps within some subcategories. Feedback was converted in prototype 3.5 (see Figure 4 and Table 2) and ready to be tested in the feasibility study of November 2017. The FindMyApps selection tool now also includes a back-end.

Figure 4.

Flow of ‘FindMyApps’ selection tool (prototype 3.5), including personal settings, selection of main and subcategories, and app recommendation and information pages. © 2013 He Hajo; © 2011 Intuary, Inc; © 2011 M&B Development and © 2013 Afasie Vereniging Nederland.

Photo 1.

Additional tests for dementia-friendly icons.

First, after a simple log-in page, the FindMyApps selection tool will create a user profile by asking six questions about personal preferences for apps. Second, based on personal interest persons select a category in the area of self-management and meaningful activities. Main categories that people can choose are: in and round the house, contacts and leisure time. When persons select a main category, they will be led to subcategories, where they can specify the activity of their interest. When people select a subcategory, recommended apps become visible. Clicking on the ‘information and download’ button will provide specific app information. The apps that best match their user profile will have a higher score. In this way apps are selected that are useful and suitable for the individual person with dementia.

Discussion and conclusion

Overall results of sprints

In this user-participatory design study the FindMyApps selection tool was developed in close collaboration with end users (persons with dementia and informal carers) and other important stakeholders. We succeeded in making a workable tool with an unambiguous routing for finding apps, which requires minimal effort from the target group to master. During sprints users tested the usability of the prototypes in several rounds, which generated important insights into (i) useful content and (ii) user-friendly design of the tool. During the development, we considered the important user-interface aspects that were mentioned in the scientific literature on designing dementia-friendly ICT applications.

One key point in the development of the content was establishing a useful group hierarchy of main and subcategories (see Table 1). On the one hand we wanted to meet the variety of needs regarding self-management and meaningful activities people with dementia may have by offering enough apps.³⁵ On the other hand, we wanted to prevent people with dementia from the need to endlessly click, which is a requirement for dementia-friendly interfaces.^22,35 At the same time, user experiences told us that combining many options for main and subcategories on one screen was not desirable. We therefore decided to incorporate more pages with fewer possible options, supported by a clear and simple navigation.

In addition, user-experiences provided relevant information on suitable dementia-friendly icons, a suitable supply of apps within subcategories, and the use of clear and short explanations of the content of apps. In addition, they clarified how to best formulate the personal settings and help instructions. Requirements about use of relevant icons, minimal use of text and using clear and short sentences for dementia-friendly interfaces were also acknowledged in other studies.^{19,21,22,27,35,51}

With regard to the design, user-experiences with the tool provided us with knowledge for an intuitive design that is easy and attractive in its use. A major insight was that different user-interface elements, such as pages and interactive buttons, had to be simple and logically integrated to support users in intuitively operating and understanding the tool. In previous prototypes (1.3 and 2.4), users had problems accessing and understanding the settings and the Help page. The last version (prototype 3.5), in addition to a Settings page, also asks about the settings during the registration into FindMyApps (immediately after having chosen a username and password) so that users do not have to access a separate setting page when they are into the FindMyApps environment. Instead of having one Help page explaining different functions, the help function was broken down so that each page in FindMyApps has its own explanation. The design of the buttons – i.e. big horizontal and easy to access – made the interaction with the buttons very intuitive, which resulted in easy selection within the main and sub-categories. Requirements for an easy-to-use, attractive and intuitive design, and the use of large accessible buttons were also acknowledged in other studies that designed dementia-friendly applications.^{19,21,22,27,35,52} Furthermore, to meet the requirement of a simple structured interface and to improve recognisability, we used the interface also used for overarching main and subcategories. Subsequently, we simplified the use of username and password by asking the users to set a username and password in the beginning only; after that they stayed logged in.⁵² Other important requirements that were taken into account in the design of the tool were the use of a minimum number of buttons,^19,28 minimising the need for scrolling,^35,53,54 clear contrast between text and background,^35,51,54 use of appropriate text sizes and fonts,^19,35,51,54 navigation comfort and landscape presentation.²²

Still, for some requirements, such as colour use and location of the navigation bar, practice will have to show what works best. In prototype 1.3, colour use was minimised, whereas in prototype 2.4 different colours were used. In both usability tests, users were satisfied about the colours used in the tool. For prototype 3.5, we minimised the colour use again also because of literature insights.^53,54 Both usability tests showed that users hardly noticed the navigation bar at the bottom of the screen. We therefore decided for prototype 3.5 to locate the navigation bar at the top, even though according Riley et al. it is better positioned at the bottom of the screen to reduce fatigue when users hold their arms out to press the screen.²²

The FindMyApps selection tool makes a unique and important contribution to the field of dementia. As far as we know, this is the first tool to be designed for people with dementia that matches personal preferences and abilities (user-profile) with the specific features and types of apps in the self-management and meaningful activities domains. We hope that FindMyApps will ultimately support people with mild dementia in using the relevant apps, and that this will subsequently contribute to a better quality of life. A similar web-based and personalised toolbox is available for young adults to prevent them from developing mental disorders with the help of mobile health apps.⁵⁵ Significant effects were found on mood, energy, rest and sleep trajectories between intervention and control groups.⁵⁵

Strengths and limitations

The main strength of the current study is that end users were involved in developing FindMyApps. This is in contrast with the more traditional ‘waterfall’ method, a more top-down approach that does not include the end user in the development process, which is frequently associated with problems with usability, adoption and attrition.⁵⁶ Span et al. stated that the involvement of people with dementia improved the usefulness and acceptability of IT applications and that it may have empowering effects for them.⁴² In our study, people with dementia and informal carers fulfilled roles of informants and advisors,⁵⁷ and decisions during the designing of FindMyApps were based on how the users interacted with the design and how they experienced it.⁴⁰ Furthermore, we worked according to the Scrum method in demarcated yet iterative sprints that guided the development team in prioritising the working agenda.⁴¹ Also, a development team and an expert team consisting of different disciplines, i.e. researchers, experts in dementia care and developers of the software company, collaborated intensively during the development process, which created a kind of triangulation in developmental issues.

There are also limitations of the study that need to be mentioned. One limitation is that the development team had to make choices based on a limited amount of data. Establishing a useful group hierarchy and selecting suitable icons representing the main and subcategories could have been a separate study, instead of part of the current study. The development team struggled to choose the most suitable icons for people with dementia. In addition, the researchers mentioned that both persons with dementia and informal carers needed some training in the basic working principles of FindMyApps before performing the tasks prescribed by the development team. We presume, and this was also noticed by participants, that they would have performed ‘better’ on tasks during sprint 1 and sprint 2 if they had had the chance to get to know FindMyApps a little better before performing prescribed tasks. We therefore recommend providing instructions and giving users time to practice before starting with tasks and research data collection. These recommendations will be followed in the subsequent FindMyApps feasibility study.³⁶ Last, the developers gave first priority to the technical realisation of the FindMyApps functionality, while design including user-friendly navigation was a second priority. During sprint 1, it became evident that, especially for people with dementia, design and technical functionalities had to be developed ‘hand in hand’.

Further development

Having followed the recommendations from the Medical Research Council framework for the development and evaluation of complex interventions,^58–60 future studies will involve testing the FindMyApps selection tool in one or more feasibility studies, with further improvements where needed, and finally tested in a definite RCT.³⁶ According to Span et al. this is guaranteed to result in a supportive and user-friendly IT application,⁴² because it guarantees the involvement of people with dementia in four development phases:

1. Explorative phase, setting requirements, collected in previous needs studies;³⁵

2. Technical development phase, setting technical requirements (current study);

3. Adaption phase, pilot testing (daily operation of FindMyApps) and identifying usability issues;

4. Evolution phase, measuring effects and impact of FindMyApps.

Second, if FindMyApps is effective and improved based on newly discovered usability issues, a native app will be developed (which will be also made available for other platforms e.g. a smart phone). A native app can be defined as an app downloaded to the user device.⁶⁰ From the beginning, it was decided to build a responsive website, instead of a native app, because of research-driven pros: a website provides the possibility for continuous development and improvement, whereas a native app is more static and can therefore easily result in a poor-quality end product. In addition, a website is less time consuming to build and more user-friendly for updates.⁶⁰ Furthermore, a website avoids problems with the compatibility of FindMyApps on different versions of tablets. There are also some cons to mention regarding the use of a responsive website. During the usability tests it was noticed that users had problems operating the buttons. They had to press more than once for buttons to react, which was caused by a poor internet connection. A native app can be used offline, which would hopefully improve the responsiveness of buttons. In addition, users experienced the page with the overview of most used apps per category (Mijn apps) as less meaningful because the page does not fulfil its intended function, which was to launch (open) the apps downloaded through the FindMyApps tool. A responsive website does not support universal links of apps, which means that apps downloaded through FindMyApps are not automatically stored in the back-end, making it impossible to launch apps from the overview page of FindMyApps (Mijn Apps). With a native app they could be stored in the back-end and therefore launched within the FindMyApps environment.

Third, with the rapid development of new apps we have to find a way to update and maintain the FindMyApps library in a user-friendly way. For example, in future prototypes it would be desirable to add a functionality in which the users can recommend apps based on certain criteria for dementia-friendly apps. The selection of apps that are currently included in the library have not been chosen by potential users, but have been compiled by trained student volunteers.

Fourth, usability tests informed us that users struggled with the distinction between personal settings of individual apps and personal settings of the FindMyApps tool. To improve the simplicity of the tool we dropped the latter. In future prototypes we have to think of a user-friendly manner for users to meet the requirement to also adjust the FindMyApps tool to their personal preferences,^17,19,35,52 e.g. to set photos or icons to represent main and subcategories; to set an extra search bar for typing the activities of their interest; to set the letter sizes, etc.

Fifth, as mentioned before, the researchers found it hard to establish a group hierarchy order for (sub)categories and to select suitable icons representing these (sub)categories. The feasibility study will identify further necessary adaptions based on new insights gained after people with dementia and their informal carers use the FindMyApps selection tool for a longer period of time.

Last, the FindMyApps selection tool may also be of benefit to other vulnerable target groups, such as people with more severe dementia living in nursing homes, and people with intellectual disabilities, autism, psychiatric disorders or acquired brain injuries. New development sprints will be needed to adjust the tool to the specific needs, wishes and abilities of other target groups.

Practical implications

In this study, we found that people with dementia can participate in this type of research where they have to perform tasks on a tablet device. They provided us with valuable feedback to adapt the tool to their wishes, needs and abilities, which hopefully results in an increased usability. In future IT development, researchers and software developers could benefit even more from outcomes of usability tests by providing a little practice/try out beforehand, regardless of which target group they built the application for. In the present study users, researchers, software developers and experts in dementia care worked closely together and this resulted in a thorough understanding of how potential users interact with the user-interface. It also contributed to a better mutual understanding of the researchers’ and developers’ roles, perspectives and use of each other’s jargon. During the development researchers became more aware of logical steps in software development and software developers adopted a research attitude, which was supportive for researchers and of great value for the quality of the end product. This study may contribute to the development of practical guidelines for new dementia-friendly ICT tools. The authors intend to prepare a separate paper on this in the future.

Conclusion

Overall, we can say that, in three sprints, the FindMyApps selection tool, in co-creation with users, researchers, developers and experts, has developed towards a more intuitive design that is easy and attractive to use. The FindMyApps tool was conceived as a means to make it easier for people with mild dementia to select apps that meet their needs, wishes and abilities. It is hypothesised that the use of these selected apps will encourage self-management and meaningful activities. The tool will be further tested and improved in a feasibility study and its effectiveness subsequently evaluated in an RCT.

Acknowledgements

We are grateful for the cooperation of the visitors of the day care centre of care organisation Sensire and the meeting centres in Enschede (the Netherlands). We also thank the informal carers and the staff of these organisations. We thank the developers and designers of the software company for their fruitful cooperation. We furthermore thank Gianna Kohl and Melanie Veijer, students from the University of Twente for their work on the assessment of apps and for entering them in the FindMyApps library. We thank Gerben op den Dries, Sector Art & Technology student at Saxion University of Applied Sciences, for assessing apps and for his ideas and input in this study. We are grateful to the Sector Art & Technology students at Saxion University of Applied Sciences for the first mock-ups of the tool. Finally, special thanks to the Foundation for the Support of VCVGZ for funding this study.

Contributorship

Each author has been sufficiently involved in this submission to take public responsibility for his or her work. This means that each author has made substantial contributions to the concept and design of the study, drafting the article or revising it critically for important intellectual content. All authors have approved the manuscript and agree to its submission.

Declaration of conflicting interests

Conflicting interests: YK is an employee of Saxion University of Applied Sciences and PhD student at the VU University Medical Centre. YK has received a grant from the Stichting tot Steun Vereniging voor Christelijke Verzorging van Geestes- en Zenuwzieken [Foundation for the Support of the Christian Mental Healthcare Association] (VCVGZ). The organisation played no role in the design and execution of the study or in the preparation, review or approval of the manuscript.

Ethical approval

The Medical Ethics Committee of the VU University Medical Centre in Amsterdam approved the study protocol (REC number: 2016.030).

Funding

This work was supported by the Foundation for the Support of VCVGZ [grant number 232].

Guarantor

YK