Usability study of a smartphone app entitled: Living with Parkinson’s disease

Abstract

Background

Mobile healthcare apps have transformed the healthcare industry and these apps can now be used for educational and preventive purposes, as well as providing valuable information for self-care. Apps related to Parkinson's disease can help from diagnoses to treatment, however the purpose of the app in this study is to inform and to educate.

Objective

To describe user appraisal of the refinement of a Parkinson's Disease app through a technical evaluation by researchers in the technological area of Human-computer Interaction HCI and usability from the perspective of the end user.

Methods

Technological development research of heuristic evaluation and user experience study carried out for usability testing, where the profile of users were surveyed and usability was evaluated by HCI researchers and end users.

Results

Parkinsonians with a recent diagnosis showed the greatest interest in the app. The final version of the app was ready after modifications proposed by experts and users. The app achieved an excellent usability rating, when rated on the Brooke scale in regard to user evaluation.

Conclusion

Structured strategies to promote greater adherence contribute to increased usability, and may allow for greater autonomy in care and early identification of symptoms.

1. Introduction

Parkinson's Disease (PD) is a complex neurodegenerative disease defined especially by the loss of dopaminergic neurons in the substantia nigra, with its motor symptoms being the main aspects that define its diagnosis. It is estimated that the incidences range from 5/100,000 to more than 35/100,000 new cases annually, with a substantial increase in the elderly population [1].

It is a well known fact that currently there is no known cure for the disease or effective methods of prevention, and it is necessary to invest in social alternatives of education and self-care, in order to reduce its risks and to improve the life quality of sufferers. However, an important aggravating factor emerged with the start of the Covid-19 pandemic, which required drastic changes in PD care, with the interruption of in-person care and therapies and the increase in online care being notorious [2].

Apps for caregivers and users with health problems have been widely discussed and debated, mainly with regard to their development based on a participatory design (centered on the expectations and needs of users) and on the evaluation of usability, in addition to the interdisciplinary approach. These apps highlight that in many cases, users and their caregivers lack information and even training on how to recognize important changes and the search for trained professionals is essential to educate about self-care and about the problems that users could face [[3], [4], [5]].

Adaptive strategies have been put in place on a large scale worldwide, allowing for the monitoring of symptoms through smartphone apps to replace traditional methods, supporting continuous management of the disease [2]. The search for information about these symptoms and their care, as well as the professionals involved, are some of the points of interest of apps for PD, however despite the wide applicability of these resources from diagnosis and treatment to rehabilitation, the low methodological quality stands out, which impacts on the recommendation of the use of these apps [6]. It is worth noting that after a quick search in the digital stores no apps of an informative nature were found in Brazilian Portuguese during the time of writing this study.

Because of this shortage of apps this study briefly describes the development of a prototype of an app about PD for caregivers and patients alike, with the objective of informing about the main aspects of the disease and its care based on existing information which can contribute to educating users about the disease, which has seen an exponential growth worldwide. With care options limited by the Covid-19 pandemic and following the premise of the low methodological quality already pointed out [3] as well as the importance of apps in health education [4] the main objective of this study was to describe the contribution by users in the improvement of an app by evaluating its usability both in terms of perspective of the end user as already used in the literature [7], and from the technical viewpoint of HCI professionals in the technological area of Human-computer Interaction (HCI), and only after this would the first version of the app be made available for free in app stores.

2. Methods

2.1. Type of study

This is a cross-sectional and observational research, which was submitted and approved by the Research Ethics Committee of the University («omitted for consideration») under CAAE: 43338921.7.0000.8093. Note that all participants signed a consent form (TCLE).

2.2. Application prototyping: a brief summary

After development of a prototype app that would be subject to modification after user evaluation as to if the app meets their needs and specialists regarding the technical evaluation, the DECIDE framework was used to define the scope. DECIDE is a tool that allows for the organization of projects in HCI [8]. At this stage the general objective of the technology, deadlines, priorities, content and tools were defined, with the aim of making the main topics about PD feasible. We used the Brazilian web platform Fábrica de Aplicativos (Application Factory), which makes it possible to develop apps without requiring specific programming knowledge.

The content was based on an educational e-book previously developed by the group, which contains information based on literature from pathophysiology to the treatment of the disease [9]. The content was evaluated by three specialists, a neurologist specialized in PD and two speech therapists. The content was adapted to accessible language, with a readable font and size and graphic association in the form of images and/or short narrated videos (average of 1–2 min duration) and audio.

We based the layout and formatting of the app on the criteria for interface accessibility for the elderly [10], because PD is a disease that mainly effects the elderly and because we did not find any PD specific interface criteria [1]. There is a summary of the criteria which was followed in the checklist below Table 1.

Table 1.

Summary of accessibility items considered for the design of the app.

Item	What was evaluated	App “Vivendo com Parkinson”
Compatibility	Evaluated the presence and absence of resources compatible with the visual and auditory difficulties that the elderly may have.	Textual information was summarized and changed into plain language and in an active voice. Content associated with representative images, audio or short narrated videos. Ensure an there was no advertising or any other resources that may generate visual confusion. Pull down features avoided. Provision of large access buttons.
Flexibility	Evaluated the synchrony between subtitles or sound representation, in addition to the format on different devices.	In the audios created by the research group all the narration conforms to the visual presentations. This app is only available on Android.
Readability	Evaluated the readability of the information also considering the visual difficulties that the elderly may have.	We used a white background with black text, and used bold text to highlight some topics, titles with different sizes. We avoided “serif” fonts, and used 20 point with proper spacing.
User Control	No automatic updates. When there are updates they can be disabled by the user	The user is informed by the app itself if there are any requirements to add or update the content or tools of the app.
Grouping/distinction by location	Verification that related content is logically grouped.	The main menu has general topics, when clicking on one the user is directed to associated subtopics. For example the symptoms menu item has sub-menus such as, initial symptoms, motor symptoms, non-motor symptoms.
Meaning of codes and denominations	Large legible icons that are clearly labeled.	As described above, we paid attention to the clarity of the title of the items, as well as to their arrangement/layout.
Agility	Assessment was made as to whether sufficient information was provided to use the app, as well as the manuals and guidelines of use.	Initially we had not thought about this topic and after modifications a user guide and FAQs were written and both were included in the app.
Minimized interaction	To avoid repetitive actions when using the app.	Short and objective interaction with the app, user access with a click.
Consistency	Information and object interactions that occur repeatedly on different pages, are presented in consistent manner.	We opted for a standard layout for ease of learning
Information density	Pages do not contain irrelevant or repetitive information	To avoid competing for users' attention, the app does not contain irrelevant information such as advertisements or repetitive information.

The last step included an assessment by the present group of researchers specializing in HCI, and following Nielsen's usability heuristics [11] with the aim of scoring previous problems and making the necessary modifications before the stage of usability analysis. After this process, the first version of the app was registered at the National Institute of Industrial Property (INPI) who issued a certificate with Protocol Number: («omitted for evaluation»). The summary of this analysis can be found in Table 2 below:

Table 2.

Evaluation of the app's usability by the research group.

Nielsen heuristic	Place	Severity	Recommendation	Disagreements between experts
Aesthetic and minimalist design.	Menu presentation in two formats	Serious problem - possibility of generating confusion during access	Only use a single menu	There were no disagreements with this item
Flexibility and efficiency in the use of accelerators	Search field disabled	Serious problem - tool dysfunction.	Enable tool or remove it	There were no disagreements with this item
Visibility of system state	Access login - add information to make user recruitment easier for the usability evaluation	Cosmetic issue - no need for immediate modifications, but contributes to future research	Add “City/State” fields	There were no disagreements with this item
Visibility of system state	Menu - no information about the team and linked institutions	Minor issue - may explain to user about the developer group	Include a field “Team” in the main menu	There were no disagreements with this item
Correspondence between the system and the real world	Menu - references: some users may not understand the functionality	Serious problem - a lack of clarification can lead to confusion for a novice user	Add a note to explain what the “References” tab is for, that is, to access additional information	There were no disagreements with this item

2.3. Step one refinement of the app: getting to know the users' profile

According to the concept of participatory design, defining the end users of the app is essential so that it can be tailored according to the needs of the users [3]. To achieve this all of the members of the Associação Parkinson Brasília (Parkinson's Association of Brasilia) were invited to participate in the research and were recruited, in a non-probabilistic way by convenience, which accounted for 63 participants, including parkinsonians, family members, caregivers, students and healthcare professionals. Participants with functional literacy and without cognitive problems such as, memory changes, dementia, Alzheimer's disease were included in the research and signed a consent form and completed a questionnaire used for data collection. For parkinsonians, the time of diagnosis was not considered, since the objective of this step was to learn about the interested parties. The recruitment of these participants was due to the uncertainty of which audience would be defined as the end user, consequently this step was essential to identify them by analyzing the questionnaire as explained below.

Among this population, a questionnaire was created that sought to understand the sociodemographic profile and education, the individual's level of interest in information about PD and the present application, in addition to the technological profile, with the aim of identifying those most interested in and benefiting from the app's proposal, being defined as end users. In addition to the technological profile of these individuals, participants were contacted individually by WhatsApp. The collection of information was not carried out in person but by telephone after the researcher had previously booked a call time. The questions were read and answers collected. The time to complete the questionnaire took on average 10 min. It is worth noting that questionnaires are commonly used to identify a user's profile because it is a simple, low-cost and easy method that meets its intended purpose [12,13].

2.4. Prior and technical analysis by HCI scholars

Considering the profile of the end users as determined in the previous step and to ensure technical compliance before user assessment, an assessment was carried out by the HCI professionals. To do this we considered a minimum of three individuals as recommended by Nielsen [11]. This quantity is commonly used in usability studies in healthcare systems, proving to be useful and highly efficient [14,15]. Professionals and/or students in the area were included, who had HCI experience and already taken part in research projects, scientific initiation or extensions in the area and who had an Android device. We excluded those who did not participate in all the stages of the research and who did not have access to a mobile device or an internet connection.

The predictive paradigm and “ask the experts” technique were used to pre-determine any difficulties with the software. Predictive assessment aims to analyze and predict the type of problem that users could face through inspection by a group of HCI professionals [16]. Considering the profile of primary users, each specialist performed three main tasks using the app 1) access a video, 2) access an audio, and 3) access the written content. For the analysis of these functions, through the Nielsen usability heuristics [11], a score from 1 to 4 was assigned, where 1 signifies (a problem that does not require correction), 2 (a simple problems), 3 (a serious problems) and 4 (a problematical event) reference [15]. Finally, a joint meeting was held to define the consensus among the specialists, formalizing a report containing a) the objectives of the evaluation, b) scope, c) brief description of the methodology used, d) aspects analyzed, e) number and profile of specialists, and f) list of problems found for each problem, location, severity and suggested solution as in Ref. [15]. After this step, the problems were adjusted, and we then proceeded to evaluate them with the users.

Just for informational purposes in HCI in the area of computer science students learn evaluation methods, including heuristic evaluation. In the case of any of the students not being able to master any item of the heuristic evaluations then they could discuss this together and find a solution to improve the interface.

2.5. Usability analysis by end users

The last stage of the study looked at the usability evaluation by end users. Nielsen [11] suggests that 5 individuals are enough to determine 85% of potential usability problems, as used in other studies in healthcare systems, such as in the analysis of the admission module and medical records [14] and the national health system “AVAB” [15]. Only participants characterized as end users in step 2.3 were included independent of gender or age, following the functional literacy criterion for the continuation of the research participants without a mobile device or internet access were excluded.

The last evaluation step was carried out by the primary users who performed the same three tasks as described above. These users were recruited from the first stage through WhatsApp. After 5 days of using the app each user had an individual meeting. Users performed the tasks and were asked to use the Think aloud technique, that is, talking aloud through the entire process of using the app and executing the tasks [17]. During the meetings, difficulties and suggestions for solutions were highlighted by the research group, according Nielsen [11]. In addition, the System Usability Scale (SUS) questionnaire was answered, which was adapted to the context of our app, this has 10 statements with an intensity measurement from the Likert scale of 1–5, where close to 1 refers to strongly disagree, and close to 5 strongly agree [18,19]. The calculation of the score follows a rule: for items 1, 3, 5, 7 and 9, the scale value minus 1 is considered. For 2, 4, 6, 8 and 10, the answer of 5 is subtracted. After that, the score of all items is added and multiplied by 2.5. The considered satisfactory average is 68 points [18,19]. Data from all stages of items 2.3 and 2.4 of this article followed descriptive analysis.

3. Results

3.1. User profile

There were 26 respondents to the questionnaire, of which 30.76% had Parkinson's disease, and 26.92% were relatives of someone who had the disease, followed by 15.38% professionals in the field, 23.07% students and 3.84% volunteers. There was a predominance of parkinsonians and their family members, who were then included in the group of end users. The primary users were on average 59.8 years old (SD ± 15.96) was obtained with 60% of participants aged over 60 years old, resulting in a sample with an adult and elderly profile. In addition, 60% were female and 53.3% had completed a higher education.

The individuals with PD had an average of 10.57 (SD ± 10.40) were seen at the time of diagnosis, with 71.4% diagnosed less than 7 years ago. Among the difficulties pointed out by these individuals were using a mobile phone, 30.7% motor difficulties, 15.3% font size and 7.6% color contrast. The majority of primary users, used devices running android, a total of 60% of the individuals. In addition, 59% used mobile phones to access information, followed by 36% computers and 5% laptops.

The apps identified as the most used were phone calls (23%), WhatsApp (18%), Google (18%) and YouTube (12%). Among these apps were also Instagram (5%). It is worth noting that many users still use unreliable sources of information about PD (13%) compared to scientific articles (8%), in addition to a few who read magazines (2%) and digital books (4%), these are findings that strongly favored the adaptation of the E-book to a mobile app format.

3.2. Technical evaluation in HCI

The technical analysis of the app was done by three 22 year old participants who had an education in HCI and had participated in projects in the area through scientific initiation. Two were female with 1 year of work experience, while onde were male with 6 months of work experience. No problems were encountered in carrying out the tasks, but in general, 13 usability problems were found related to heuristics of compatibility between system standardization and a lack of guidelines/tutorials, and to a lesser extent, a large number of text elements that could overwhelm the user. The main modifications made at this stage were as follows standardization of fonts, colors, size and delimitation of the menu tabs and references, creation of a tutorial and FAQs, textual synthesis, adaptation of the order of the main menu content. An example of how this data was organized is shown in Table 3.

Table 3.

Demonstration, summary, of the management of usability problems found by specialists.

Problem 2	There were no guidelines/tutorials about using the app, so the elderly may have difficulties using it.
Heuristic violation	Help and documentation (10)
Problem area	App in general
Severity	3 assigned by all experts
Solution	To make a user guide containing a tutorial on how to access the app, and about its information and tools, as well as a tab with FAQs
Mods carried out	Creation of a user guide containing a tutorial on how to access the app, its information and tools, as well as a tab with FAQs

3.3. Evaluation of usability by end users

The evaluation of usability by the users was carried out by 5 people with up to 10 years of experience in diagnosis. 80% were male with an average age 56.8 years old (SD ± 9.7) and average time in diagnosis of 4.6 years (SD ± 2.7). During the individual meetings no motor difficulties were seen while carrying out the tasks, however one participant suffered from dysatrophonia which is a speech and voice alteration that impacted the collection of information during the Think aloud technique, while another pointed out difficulty in understanding the information in the registration. According to user suggestions the following adjustments were made as follows information added about parkinsonian rights myths and truths, and scientific articles. In addition, they suggested telling family members and caregivers about the app and to have a space to exchange practical experiences. An example of how this data was organized is shown in Table 4.

Table 4.

A brief demonstration of usability problems found by users were treated.

Participant	SP
age	63
Sex	Male
Time in diagnostics (years)	2
Difficulties found	Did not have any difficulties due to age, health or using the app, but he found the registration confusing
Suggestion	To add guides and information about the myths and truth (scientific research) of Parkinsons, and Parkinsonian rights
Mods made	We made some login fields clearer, such as “City” for “City of Residence” We added the myths and truths tab, with the participation of a neurologist via text and a narrated video. We added the Parkinsonian's rights tab, with the participation of a lawyer via text and a narrated video. We added the Scientific Research tab, with scientific articles on various topics about the disease.

As previously mentioned the SUS scale follows a specific score, with an average of 68 among responses as a satisfactory parameter. This study had averages well above, both among the items and in the general evaluation (88.5), as shown in Table 5, indicating that our app has an excellent level of usability according to the Brooke scale [19] which considers <60 unacceptable, 60–70 good, 80–90 excellent and >90 best possible usability. We found that one participant (n = 5) had a considerably lower score when compared to the others, it is worth noting that this individual suffered from dysarthrophonia during the Think Aloud technique which may have impacted the evaluation process. The comparison between the initial and final version of the app can be found in the appendices section.

Tabela 5.

Results of the system usability scale (SUS) scores.

Participant	1	2	3	4	5
SUS score	97,5	97,5	95	82,5	70
Average of participants	88,5

4. Discussion

This study demonstrated excellent usability of the app for guidance on Parkinson's disease by parkinsonians, in addition to having defined and adjusted any technical problems found by HCI professionals. Apps aimed at Parkinson's Disease have been gradually appearing, especially internationally, but very few consider the user at some stage of development or validation of the app [6]. A study assessed the positive and negative factors in the implementation of technologies of PD [20], demonstrating that when participants are included in the development and/or testing of technologies with regard to user-centered design, the result is greater satisfaction, ease of handling, feeling of confidence, refinement according to their real needs, in addition to greater adherence, especially when the technology can be used in their daily lives, aspects that confirm the objective of the usability analysis methodology as used in this study.

As for the barriers in the use of these technologies in PD, a study [20] pointed out that the lack of understanding of motor issues, technical issues and high cost tend to hinder adherence, such aspects were widely considered during the questionnaire phase to learn about the profiles of users. The HCI technical analysis allowed for predicting and adjusting the problems considering the particularities of the end user, in the observational analysis during the usability evaluation process that allowed for monitoring the impact of motor and cognitive issues when handling the app. It should also be noted that this app will be distributed free of charge which will allow for greater access by different users.

Also considering the importance of knowledge about the disease in the technological proposal, no motor impacts were seen on usability analyzed by end users. It is known that, in the initial stages tremors may not be noticeable to people, in addition to the use of levodopa medication which is used to control the symptoms and helps considerably with tremors and muscle stiffness, especially in the early stages, aspects that may justify such a finding [21]. In addition, it is worth noting that, according to the profile of the users studied, there was a greater demand for information in the early stages of the disease than in the later stages, where the impact of the disease is greater, as also found in other studies [22]. Using our app can help these people in the early stages with PD education, which can help them with therapeutic intervention and management strategies early on. On the other hand, it is necessary to carry out new usability tests with a larger number of participants, in order to verify whether the demand for information is really more intense in the initial stages, as described in other studies [22], or whether further adjustments to the app should be made considering the motor impacts in the later stages of the disease.

Among the suggestions pointed out by the users were disclosure to family members and caregivers which stood out. It is important to emphasize that caregivers and/or family members also suffer and are impacted by the disease, mainly due to the extra demand for care, affection and attention of the sufferer [23]. It is also suggested that tests should also be carried out with caregivers, considering them as an active part in the illness process and possible beneficiaries, depending on the technology [20]. Although the usability test of our study was carried out only with parkinsonians, we saw that during the survey of end users that family members and/or caregivers were the second category that showed the most interest, and could indeed be considered for future analyses. Just to note that the application is not restricted to parkinsonians but available for all and contains valuable information to assist in the learning about the disease and care of the disease in general, including the caregivers themselves in the topics covered by the app.

It should also be noted that interdisciplinarity was fundamental for the success of this study, since interdisciplinary solutions in technology and health are gradually growing and gaining prominence in the areas of prevention, diagnosis, monitoring and dissemination in health. Such aspects gain strength and open up new opportunities for future research in an interdisciplinary manner, as carried out in this study [24]. It should be mentioned that the app is only available for Android devices which is the most commonly used by our public, as seen in the definition of end users, however, the platform used to create the app also allows for indexing to the Apple Store, a step that is being considered for future studies.

The benefits of the app on learning about the disease are still unknown, that is whether the app will actually achieve it's objective of providing guidance and information. For this purpose studies are being carried out by the research group. It should also be noted that the Covid-19 pandemic presented as a major limitation to this study namely that of carrying it out in-person. There were many problems with online evaluation such as network problems, delay in accessing digital meeting platforms, the need to adapt these platforms for greater ease of use by the participant and noisy environments of the participants. To remedy such problems, at times it was necessary to train the participants on how to use the meeting platforms (how to access, how to position themselves, among other factors), in addition to assistance from caregivers when necessary. It is believed that, if it would have been possible to carry out usability analysis of users in a controlled in-person environment, perhaps we could have encountered other problems which may have helped in achieving better management in collection and analysis.

5. Conclusion

Ease of use strategies for greater adherence and to make an app easy to handle should be considered in usability studies. This was relevant to this study regarding not only to the evaluations by the specialists but also in the definition of the primary users and their evaluation, which were a key part of the process. We could see that during the evaluation by the specialists that the Vivendo com Parkinson app needed some technical adjustments aimed at predicting the limitations of the users due to their disease and age. We expected that the users would have motor limitations that would be a factor that hindered their use of the app, but this was not so. This can possibly be explained by their good response to levodopa since they were in the early stages of the disease.

Summary Table

What was already known about the topic

Parkinson's disease is mainly characterized by motor symptoms that generate gradual dysfunction throughout its progression. The need to know about the main changes resulting from the disease is essential for self-knowledge and self-care. The use of apps in the healthcare area has already been highlighted, including for the elderly population, who are the most affected by Parkinson's. There are many apps available but very few for Brazilians, in addition to a lack of apps with global guidance on the disease. Among these, low methodological rigor is seen in relation to usability. Evaluating the usability of a system is essential to adapt technologies to the target audience.

What this study added to our knowledge

We demonstrated, in great detail, the process of creating and evaluating the usability of an educational app about Parkinson's Disease. Understanding a user's profile is an essential part of usability studies, this study highlighted the importance of this analysis for the tools to meet the individual and collective protection of users. The analysis of usability by the specialists was essential to predict the problems of the analyzed tool, made possible thanks to the knowledge of the profile of the primary users. Despite the motor characteristic of the disease, user analysis showed that in the initial stages motor difficulties may not be a problem for using the app. In this study the users were in the early stages of the disease and were more interested in learning about Parkinson's disease, as well as their families and/or caregivers.

Author contribution statement

Naira Rúbia Rodrigues Pereira: Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Letícia Corrêa Celeste: Cristina Lemos Barbosa Furia: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.

André Barros de Sales: Fabiana Freitas Mendes: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Data availability statement

Data will be made available on request.

Data statement

We declare that the data collected from this study will be made available.

The material will be available for research, teaching and study. We are not responsible for the results and interpretations produced by further use of the material.

The data can be found in the “COLLECTED_DATA” document, attached during submission.

Sincerely,

Pereira, NRR, Celeste, LC, De Sales, AB, Mendes, FF, Furia, CLB.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendices. (This app is only available in the Portuguese language)

Appendix A. Supplementary data

The following are the Supplementary data to this article: