Abstract
Background
Mobile apps are evolving in the medical field. However, ongoing discussions have questioned whether such apps are really valuable and whether patients will accept their use in day-to-day clinical life. Therefore, we initiated a usability study in our department.
Objective
We present our results of the first app prototype and patient testing of health-related quality of life (HRQoL) assessment in oncological patients.
Methods
We developed an app prototype for the iOS operating system within eight months in three phases: conception, initial development, and pilot testing. For the HRQoL assessment, we chose to implement only the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire-Core 30 (QLQ-C30; German version 3). Usability testing was conducted for three months. Participation was voluntary and pseudonymized. After completion of the QLQ-C30 questionnaire using iPads provided by our department, we performed a short survey with 10 questions. This survey inquired about patients’ opinions regarding general aspects, including technical advances in medicine, mobile and app assistance during cancer treatment, and the app-specific functions (eg, interface and navigation).
Results
After logging into the app, the user can choose between starting a questionnaire, reviewing answers (administrators only), and logging out. The questionnaire is displayed with the same information, questions, and answers as on the original QLQ-C30 sheet. No alterations in wording were made. Usability was tested with 81 patients; median age was 55 years. The median time for completing the HRQoL questionnaire on the iPad was 4.0 minutes. Of all participants, 84% (68/81) owned a mobile device. Similarly, 84% (68/81) of participants would prefer a mobile version of the HRQoL questionnaire instead of a paper-based version. Using the app in daily life during and after cancer treatment would be supported by 83% (67/81) of participants. In the prototype version of the app, data were stored on the device; in the future, 79% (64/81) of the patients would agree to transfer data via the Internet.
Conclusions
Our usability test showed good results regarding attractiveness, operability, and understandability. Moreover, our results demonstrate a high overall acceptance of mobile apps and telemedicine in oncology. The HRQoL assessment via the app was accepted thoroughly by patients, and individuals are keen to use it in clinical routines, while data privacy and security must be ensured.
Keywords: radiation oncology, healthcare surveys, mobile applications, mobile apps, telemedicine, health-related quality of life, questionnaires, oncology
Introduction
Since the first smartphone was introduced by IBM in 1995 [1], the development of cell phones and mobile apps has been world-changing. The success of medical and health apps (labeled under mobile health [mHealth] or electronic health [eHealth]) is undeniable, with 165,000 programs in the respective leading app stores (Apple and Google) [2]. Apps for registering heart rate, blood pressure, and blood glucose are forthcoming, and apps for depression, body weight reduction, and diabetes are widely accepted. However, most of the apps are not scientifically validated [3-5]. To date, only a few apps in an oncological context exist, which allow for quality of life (QoL) assessment.
An essential key to everybody’s well-being is QoL, which the World Health Organization defined in 1946 as an, “individual's perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns” [6]. This broad definition includes a holistic approach to QoL, which is rarely used in medical research. In the 1980s the concept of health-related quality of life (HRQoL) evolved [7]. HRQoL is a critical criterion in the therapeutic decision-making process undertaken by health care professionals (HCPs) who are torn between the patients’ well-being, outcome, and economic considerations. Especially in oncology, where treatment is often only life-prolonging and not curative, HRQoL assessment is crucial to identify therapeutic benefit and need. The European Organization for Research and Treatment of Cancer (EORTC) developed a wide range of standardized and validated questionnaires to assess HRQoL in oncological patients. In this study we used the EORTC Quality of Life Questionnaire-Core 30 (QLQ-C30) [8], which is the core questionnaire that can be extended by cancer-specific modules.
In earlier series, we asked HCPs [9] and patients [10] about their attitudes regarding telemedicine and mobile apps in oncology. A total of 84.3% of HCPs supported the idea of an oncological app, and 97.8% found the HRQoL assessment very useful or useful. Approximately half of the patients asked were willing to use an oncological app; 75.3% of those patients were keen to send HRQoL data. In general, younger patients were more in favor of such an app (P=.032, r=-0.12).
Our previous results show an existing demand for oncological apps. Hence, we developed an app for HRQoL assessment in oncological patients. This work aims to present our results of the first prototype, the design/development, and patient testing.
Methods
For the first version of the app, we decided to develop the prototype for the iOS operating system (version 8 and later) optimized for iPads (generation 2 and later), but the app also worked on iPhones (generation 4s and later). Further operating systems and device options were scheduled after the first patient testing and validation results were obtained. The development was completed within eight months in three phases: conception, initial development, and pilot testing.
App Design and Development
Considering our previously published work on app use during cancer treatment [9-11], we specified primary functional requirements for the prototype listed in Table 1 with the following four main characteristics:
Table 1.
Functional requirements for the prototype app.
| Requirement | Details |
| User management | A user can register herself/himself with a given user ID and password on a device; after that, login and logout are possible on that specific device |
| Administration | The administrator can delete registered users on the device |
| The administrator can review completed questionnaires and delete them | |
| Questionnaire | A questionnaire can be filled out multiple times |
| A questionnaire can be canceled at any time; results are not saved | |
| Questions can be skipped | |
| Answers can be changed | |
| Completed questionnaires can be saved (submitted) to store the results on the device | |
| Data management | Questionnaires are stored with the information: user ID, date, given answers |
Clear interface and user-friendly design
The prototype should implement only the EORTC QLQ-C30 questionnaire (German version 3; Multimedia Appendix 1 [German], Multimedia Appendix 2 [English])
Data is locally stored on the device; no Internet connection or online data transfer were intended in this version
Login information (user identification [ID] and password) will be generated and provided by the department; this method ensured pseudonymization
The app was developed using Xcode 7.0 and Swift 2.2.
Validation
As an empirical usability evaluation method, we used a questionnaire. The survey was conducted for three months at the Department of Radiation Oncology, Klinikum rechts der Isar, Munich, Germany. Participation was voluntary and pseudonymized. Inclusion criteria for participation were: age older than 18 years, German-speaking, and being physically and mentally able to fill out a structured questionnaire on a mobile device. Research assistants supervised the patients during app use and while completing the questionnaire. The Ethics Committee of the Technical University of Munich approved the nature and content of the study with the project number 321/16 S.
After completion of the QLQ-C30 questionnaire using iPads provided by our department, we conducted the short usability survey with 10 questions (Multimedia Appendix 3). This survey inquired about patients’ opinions regarding general aspects, including technical advances in medicine, mobile and app assistance during cancer treatment, and the app-specific functions (eg, attractiveness, operability, and understandability). Statistical calculations were performed using SPSS Statistics v23 (IBM, Armonk, NY, USA) in a primarily descriptive way.
Results
App Design and Development
After launching the app, the login page appears (Figure 1, left). If accessing the app for first time, a user ID and password need to be registered on the device (Figure 1, right). After login, the start page appears (Figure 2, left). We decided to use a slide bar for the menu, as it is easily extendable for future additional functions of the app. Users can click the button in the top left corner or swipe left on the screen to open the menu. For the prototype, the functions for starting a questionnaire, reviewing answers (administrator only), and logging out were implemented (Figure 2, right).
Figure 1.
Screenshot of the login (left) and registration (right) page.
Figure 2.
Screenshot of the start page (left) and slide bar of the menu (right). Administrator view is displayed.
Figure 3 (left) shows the start page of the EORTC QLQ-C30 questionnaire. We displayed the same information that is written on the paper-based version of the questionnaire. Likewise, all 30 questions were copied from the original QLQ-C30 sheet. No alterations in wording or answers were made. Figure 3 (right) shows the first question. On the bottom of each page, the user can go back to the previous question or skip the current question. After the last question, the user is asked to save the completed questionnaire. These answers can be reviewed by the administrator (Multimedia Appendix 4).
Figure 3.
Screenshot of the start page of the European Organization for Researchand Treatment of Cancer Quality of Life Questionnaire-Core 30 (EORTC QLQ-C30; left) and the first question (right).
Validation
The usability of the app was tested with 81 patients (44 male, 37 female); median age was 55 years (range 21-80 years). When using the app, older patients (>60 years old) distinguished themselves from younger patients (<60 years old). The research assistants observed that while the elderly users had to ask questions about where to click and swipe on the screen, the younger users intuitively knew how to navigate through the app.
The median time for completing the HRQoL questionnaire on the iPad was 4.0 minutes (range 1.5-8.0 minutes). Of all participants, 84% (68/81) owned a mobile device. Similarly, 84% (68/81) would prefer the mobile version of the HRQoL questionnaire instead of the paper-based version that they usually get during a clinical visit. Using the app in daily life during and after cancer treatment would be supported by 83% (67/81) of participants, which corresponds with their general opinion about mobile technologies in medicine (described in Figure 4). Patients were satisfied with the current development and the introduction of the app into clinical life. In the prototype version of the app, data were stored on the device; in the future, 79% (64/81) of the patients would agree to transfer data via the Internet.
Figure 4.
Patient opinion about mobile technologies in medicine (scale from 1 = “I like the development” to 6 = “I don’t like the development”).
The operability and navigation of the app were rated as intuitive by 95% (77/81) and 93% (75/81) of participants, respectively. Six participants stated opportunities for improvements regarding the HRQoL questions. However, these cannot be changed as they follow a standardized structure and wording that has been established by the EORTC. Regarding the attractiveness, two patients wished for a setting option to change the font size; five had difficulties finding the menu at first. Figure 5 shows the patients’ opinions on the app interface and design.
Figure 5.
Patient opinion about the app design in terms of color, text and button size, content, and the amount of content per page (scale from 1 = “I like it” to 6 = “I don’t like it”).
Discussion
The long-term aim of app-assisted care is to generate higher treatment quality and HRQoL for our patients. In this study, we present our first app-prototype for regular HRQoL assessment, along with the patient usability testing. Most of the participants (84%, 68/81) owned a mobile device and therefore were appropriately equipped to use an app. Similarly, 84% (68/81) of participants would prefer a mobile version of HRQoL assessment instead of a paper-based version, while 83% (67/81) would use it in daily life. Such an app would increase clinical efficiency by reducing paperwork and costs, and enhance patient empowerment. Our results show that mobile technologies are widely accepted; 96% (78/81) of the patients scale their opinion about telemedicine as positive (scale 1-3; Figure 4).
The users in this study were cancer patients, and the concept of the prototype was to implement a clean and simple app, with the primary goal to be user-friendly. The oncological patient population includes individuals of young and old age. While implementing apps in clinical routine, age and technical skills always seem to be a problem. In our study, patients >60 years old needed more assistance than younger patients. However, Smith et al showed a growing trend of smartphone use amongst elderly people (>65 years) in the United States, with 18% in 2014 [12] and 27% in 2015 [13]. Older individuals incrementally adapt to mobile technology. Nevertheless, it is essential to develop apps for older patients with a focus on operability and understandability. Intensive preuse teaching might be necessary in some cases.
Although we recently showed a higher usage of Android phones (52.9%) than iOS phones (37.2%) in our patient population [10], we rationally decided to develop the first version of the app for iPhones and iPads. iOS development is cheaper [14] and more comfortable, as apps only need to meet the criteria of two model types. However, a version of our app for Android is planned in the near future.
The interface was designed in light, cool colors and we omitted any animation or gamification. Overall, we used hues of the blue and green family, which tend to psychosocially calm down the already anxious cancer patients [15]. We chose to use dark characters on white background. Piepenbrock et al [16] showed an advantage in positive display polarity for young and old adults. Font size 12 was chosen to be readable by older patients with visual impairment. Darroch et al [17] showed that font size does not affect reading accuracy in both young and old adults, but young adults preferred size 8 and 10 while older adults preferred slightly bigger sizes of 8 to 12. Darroch et al recommend offering a setting option for font size, which was requested by two of our patients.
Button size for the questionnaire was chosen to be 20 pixels high. Anthony [18] recommends a target width of 45-57 pixels wide to allow the user’s finger to fit in the target while the edges are visible when tapping. We chose the width to be the full size of the screen, as our app needs to be comfortable for older patients with visual impairment and lower touch accuracy. After pretesting, we decided to implement a button for skipping questions, as patients can do the same in the paper-based version. Patients who do not want to (or cannot) answer one of the questions can still complete the survey.
Overall, patients were in favor of the presented app design (Figure 5); operability (95%) and navigation (93%) were rated excellent. These factors play a critical role in patient compliance, as poor user-friendliness will automatically lead to reduced usage. Completion time of the paper-based EORTC QLQ-C30 questionnaire was reported by Aaronson et al [8] as approximately 11 minutes, while our mobile version only took median 4 minutes. Both patient populations were different (median age 55 vs 63 years, international multicenter study vs in-house study) [8]. However, our study shows that patients with cancer disease are compliant with an app-based HRQoL assessment, and it is an efficient method.
To date, data are only stored locally on the device and can be transferred via a cable. In the future, patient data will be sent online, and the treating physicians will have access to the data (eg, at aftercare appointments). Here, it is crucial to guarantee data protection and security. We showed previously that 85.2% patients marked pseudonymization and data security as very important/important [10]. As health data is always highly sensitive, data security was the most crucial requirement for using an oncological app [11,19].
The first version of the app only contains the EORTC QLQ-C30 questionnaire. Future functions are planned and in progress. A multilingual approach (English, French, Italian, Russian, Arabic) is necessary, as our department is frequented by many international patients, and furthermore German is not the mother tongue of some local patients (eg, refugees). Moreover, cancer-specific modules such as the EORTC QLQ-BR23 [20] for breast cancer or QLQ-PR25 [21] for prostate cancer will be implemented next. HRQoL data from cancer patients can be used to adjust the individuals’ treatment or offer supportive therapy. Surveys on therapy satisfaction will help to improve the departments’ workflow and the patients’ contentment.
Another function of the app will be the documentation of treatment-related side effects and symptoms. The course of a disease will be monitored and chronicled. This feature can also be used to register study parameters in app-assisted randomized controlled trials (smartRCTs). SmartRCTs can reduce study duration, costs, and subject bias, as well as collect a broader range of data [11]. Certainly, a workflow regarding how to check patient answers regularly and how to handle severe entries by the patients must be developed. However, we could recently show that 94.3% of HCPs were willing to contact the patient [9], which may lead to a quicker detection of progress, as recently stated by Denis et al [22,23]. This group demonstrated that using a Web-based app resulted in a significant improvement in overall survival (12 months vs 19 months) in patients with high-risk lung cancer, and relapses were detected five weeks earlier that the control group [23].
Our usability test showed good results for the presented app. Moreover, our results demonstrate a high overall acceptance of mobile apps and telemedicine in oncology, which is in line with our previous results [9,10]. The HRQoL assessment via the app was accepted thoroughly by patients, and individuals are keen to use it in clinical routine, while data privacy and security must be ensured. Digital medicine (medicine 4.0) is an unstoppable trend and will play a significant role in the future of clinical health care.
Abbreviations
- eHealth
electronic health
- EORTC
European Organization for Research and Treatment of Cancer
- HCP
health care professional
- HRQoL
health-related quality of life
- ID
identification
- mHealth
mobile health
- QLQ-C30
Quality of Life Questionnaire-Core 30
- QoL
quality of life
- smartRCT
smart randomized controlled trial
QLQ-C30 questionnaire paper based version (German).
QLQ-C30 questionnaire paper based version (English).
Patient usability survey (German).
Screenshot of the review screen with completed questionnaires (administrator view).
Footnotes
Conflicts of Interest: None declared.
References
- 1.Sager I. Bloomberg. 2012. Jun 29, [2017-11-21]. Before IPhone and Android came Simon, the first smartphone https://www.bloomberg.com/news/articles/2012-06-29/before-iphone-and-android-came-simon-the-first-smartphone .
- 2.The Economist. 2016. Mar 10, [2017-11-21]. Things are looking app. Mobile health apps are becoming more capable and potentially rather useful https://www.economist.com/news/business/21694523-mobile-health-apps-are-becoming-more-capable-and-potentially-rather-useful-things-are-looking .
- 3.Brouard B, Bardo P, Bonnet C, Mounier N, Vignot M, Vignot S. Mobile applications in oncology: is it possible for patients and healthcare professionals to easily identify relevant tools? Ann Med. 2016 Nov 27;48(7):509–515. doi: 10.1080/07853890.2016.1195010. [DOI] [PubMed] [Google Scholar]
- 4.Collado-Borrell R, Escudero-Vilaplana V, Ribed-Sánchez A, Ibáñez-García S, Herranz-Alonso A, Sanjurjo-Sáez M. Smartphone applications for cancer patients; what we know about them? Farm Hosp. 2016 Jan 01;40(1):25–35. doi: 10.7399/fh.2016.40.1.8993. http://www.aulamedica.es/fh/pdf/8993.pdf. [DOI] [PubMed] [Google Scholar]
- 5.de Jongh T, Gurol-Urganci I, Vodopivec-Jamsek V, Car J, Atun R. Mobile phone messaging for facilitating self-management of long-term illnesses. Cochrane Database Syst Rev. 2012;12:CD007459. doi: 10.1002/14651858.CD007459.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.WHO: Basic Documents Forty-Seventh Edition. Geneva: World Health Organization; 2009. WHO: Constitution of the World Health Organization; p. 1. [Google Scholar]
- 7.National Center for Chronic Disease Prevention and Health Promotion. Division of Population Health, Prevention. 2016. May 23, [2017-11-21]. Health-Related Quality of Life (HRQOL) https://www.cdc.gov/hrqol/concept.htm .
- 8.Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, Flechtner H, Fleishman SB, de Haes JC. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993 Mar 3;85(5):365–76. doi: 10.1093/jnci/85.5.365. [DOI] [PubMed] [Google Scholar]
- 9.Kessel KA, Vogel MM, Schmidt-Graf F, Combs SE. Mobile apps in oncology: a survey on health care professionals' attitude toward telemedicine, mHealth, and oncological apps. J Med Internet Res. 2016 Nov 24;18(11):e312. doi: 10.2196/jmir.6399. http://www.jmir.org/2016/11/e312/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Kessel KA, Vogel MM, Kessel C, Bier H, Biedermann T, Friess H, Herschbach P, von Eisenhart-Rothe R, Meyer B, Kiechle M, Keller U, Peschel C, Schmid RM, Combs SE. Mobile health in oncology: a patient survey about app-assisted cancer care. JMIR Mhealth Uhealth. 2017 Jun 14;5(6):e81. doi: 10.2196/mhealth.7689. http://mhealth.jmir.org/2017/6/e81/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Vogel MME, Combs SE, Kessel KA. mHealth and application technology supporting clinical trials: today's limitations and future perspective of smartRCTs. Front Oncol. 2017;7:37. doi: 10.3389/fonc.2017.00037. doi: 10.3389/fonc.2017.00037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Smith A. Pew Research Center. 2014. Apr 03, Older adults and technology use http://www.pewinternet.org/2014/04/03/older-adults-and-technology-use/
- 13.Smith A. Pew Research Center, Internet & Technology. 2015. Apr 1, U.S. smartphone use in 2015 http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/
- 14.Car T. Infinum. 2015. Oct 27, [2017-11-21]. Android development is 30% more expensive than iOS. And we have the numbers to prove it! https://infinum.co/the-capsized-eight/android-development-is-30-percent-more-expensive-than-ios .
- 15.Cao J. The Next Web. 2015. Apr 7, Web design color theory: how to create the right emotions with color in web design https://thenextweb.com/dd/2015/04/07/how-to-create-the-right-emotions-with-color-in-web-design .
- 16.Piepenbrock C, Mayr S, Mund I, Buchner A. Positive display polarity is advantageous for both younger and older adults. Ergonomics. 2013;56(7):1116–24. doi: 10.1080/00140139.2013.790485. [DOI] [PubMed] [Google Scholar]
- 17.Darroch I, Goodman J, Brewster S, Gray P. The effect of age and font size on reading text on handheld computers. Human-Computer Interaction - INTERACT 2005; INTERACT 2005; September 12-16, 2005; Rome. Berlin, Heidelberg: Springer; 2005. pp. 253–266. https://link.springer.com/chapter/10.1007%2F11555261_23#citeas. [DOI] [Google Scholar]
- 18.Anthony T. Smashing Magazine. 2012. Feb 21, [2017-11-21]. Finger-friendly design: ideal mobile touchscreen target sizes https://www.smashingmagazine.com/2012/02/finger-friendly-design-ideal-mobile-touchscreen-target-sizes/
- 19.Rumbold JMM, Pierscionek B. The effect of the general data protection regulation on medical research. J Med Internet Res. 2017 Feb 24;19(2):e47. doi: 10.2196/jmir.7108. http://www.jmir.org/2017/2/e47/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Sprangers MA, Groenvold M, Arraras JI, Franklin J, te Velde A, Muller M, Franzini L, Williams A, de Haes HC, Hopwood P, Cull A, Aaronson NK. The European Organization for Research and Treatment of Cancer breast cancer-specific quality-of-life questionnaire module: first results from a three-country field study. J Clin Oncol. 1996 Oct;14(10):2756–68. doi: 10.1200/JCO.1996.14.10.2756. [DOI] [PubMed] [Google Scholar]
- 21.van Andel G, Bottomley A, Fosså SD, Efficace F, Coens C, Guerif S, Kynaston H, Gontero P, Thalmann G, Akdas A, D'Haese S, Aaronson NK. An international field study of the EORTC QLQ-PR25: a questionnaire for assessing the health-related quality of life of patients with prostate cancer. Eur J Cancer. 2008 Nov;44(16):2418–24. doi: 10.1016/j.ejca.2008.07.030. [DOI] [PubMed] [Google Scholar]
- 22.Denis F, Lethrosne C, Pourel N, Molinier O, Pointreau Y, Domont J, Bourgeois Hp, Senellart H, Tremolieres P, Lizee T, Bennouna J, Urban T, El Kouri C, Charron A, Septans A, Balavoine M, Rolley C, Landry S, Solal-Celigny P, Letellier C. Overall survival in patients with lung cancer using a web-application-guided follow-up compared to standard modalities: Results of phase III randomized trial. J Clin Oncol. 2016 Jun 20;34(18_suppl):LBA9006–LBA9006. doi: 10.1200/JCO.2016.34.18_suppl.LBA9006. http://ascopubs.org/doi/abs/10.1200/JCO.2016.34.18_suppl.LBA9006#affiliationsContainer. [DOI] [Google Scholar]
- 23.Denis F, Viger L, Charron A, Voog E, Dupuis O, Pointreau Y, Letellier C. Detection of lung cancer relapse using self-reported symptoms transmitted via an internet web-application: pilot study of the sentinel follow-up. Support Care Cancer. 2014 Jun;22(6):1467–73. doi: 10.1007/s00520-013-2111-1. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
QLQ-C30 questionnaire paper based version (German).
QLQ-C30 questionnaire paper based version (English).
Patient usability survey (German).
Screenshot of the review screen with completed questionnaires (administrator view).