User-Centered App Design for Acutely Ill Children and Adolescents

Abstract

Background and Objectives: The high level of acceptance and consistent use of smartphones by children and adolescents present new opportunities to monitor and collect health data. For acutely ill children and adolescents, collecting symptom data via smartphone applications (apps) provides patient-reported data that can be collected daily and offers the potential to provide a more comprehensive picture of the symptom experience. The purpose of this study was to employ user-centered design principles and medical professional input in order to obtain feedback and insight into redesigning our Technology Recordings for better Understanding Blood and Marrow Transplant (TRU-PBMT) app. This redesigned app will be used for children and adolescents with cancer or undergoing blood and marrow transplantation. Method: We interviewed six pediatric blood and marrow transplant patients (ages 10-17 years) who had pilot tested the app, and we surveyed 30 pediatric oncology clinicians. Results: Interview feedback from previous app users and survey feedback from clinicians guided the app redesign. We incorporated suggestions to make the app more engaging, meaningful, personal, and motivating in order to increase symptom reporting. We added emojis to the symptom tracker, a mood scale, and personalized symptom graphs. Conclusion: Leveraging mobile health technologies may be a useful and acceptable approach to obtain symptom data; however, design and software development needs to be evidenced-based and informed by user needs. Our approach using patient and clinician feedback was valuable in the redesign of the TRU-PBMT app and will contribute to symptom research for acutely ill children and adolescents.

Mobile health technologies, such as smartphones, are playing an increasing role in monitoring, studying, and managing health. Smartphones are ubiquitous, widely accepted, and accessible devices, particularly for children and adolescents (Bendixen et al., 2017; Riley et al., 2019). Recent estimates report that 95% of adolescents 13 to 17 years of age currently own or have access to a smartphone (Pew Research Center, 2018; Riley et al., 2019), have it with them most of the time, and use it regularly to stay connected (Rainie & Zickuhr, 2015). Smartphone apps are easy to use, and health care apps have become popular for managing diet, fitness, and medication (Krebs & Duncan, 2015). Smartphone use by children and adolescents presents new opportunities to monitor and collect health data across the care continuum. For example, acutely ill children can record and track their symptoms daily on smartphone apps. Collecting data in this manner provides real-time patient-reported data, which is considered the “gold standard” for assessing symptom burden (Brock et al., 2018; Linder et al., 2018; Shaw et al., 2016). Literature shows that for subjective outcomes such as symptoms, the patient who is experiencing the symptom is considered the expert (Brock et al., 2018; Linder et al., 2018). From a research standpoint, children and adolescents engage with their smartphones multiple times throughout the day, which may provide an increased opportunity to collect symptom data (Stinson et al., 2013; Taylor et al., 2019). Additionally, engagement with the smartphone may result in increased recording compliance compared with the more traditional paper- and web-based data collection methods. Given the prevalence of mobile health tools and their growing potential in health management, the National Institutes of Health (2015) recognizes and supports research in pediatric digital health as an important initiative.

This contemporary way of collecting self-reported data from children and adolescents allows their “voice” to be heard when reporting symptoms and holds the potential to improve symptom understanding by both patients and clinicians, which may then lead to enhanced symptom management strategies. While this technology is promising, the development of smartphone apps must be evidence-based and patient-centered in order to be useful and lead to improved health outcomes (Hilliard et al., 2014; Petersen & Hempler, 2017). A review of mobile apps conducted by Hamel et al. (2019) found that only 11.4% were evidence-based, and, importantly, only 10.3% involved medical professionals in the development process.

In our previous work, we developed a patient-centered app to monitor symptoms in children and adolescents undergoing a pediatric blood and marrow transplant (PBMT) (Vaughn et al., 2018). The initial version of the app, “Technology Recordings for better Understanding PBMT” (TRU-PBMT) was used by children and adolescents enrolled in a pilot study to track symptoms using two mobile devices, a wearable and a smartphone app. The pilot study was conducted to test the study design, procedures, and the devices before embarking a larger feasibility study. The app included features such as a symptom tracker, care goals, a food diary, and bowel movement recordings. Patients could log into the app and navigate to the different pages to record information. The symptom tracker feature allowed patients to record commonly experienced symptoms including pain, nausea, vomiting, fatigue, bleeding, diarrhea, headache, itching, difficulty sleeping, and rash (Figure 1). In addition, patients could record each symptom’s intensity, timing, and if any interventions were used.

Figure 1.

Symptom tracking. Symptom occurrence can be recorded either through emoji or through text, intensities are recorded on a visual analogue scale (0-10), information related to symptom status can be recorded, and interventions used for symptom management can be recorded.

Although the pilot study usability findings were promising, we found that app use decreased over time in the study possibly due to device fatigue or changes in health status. Literature supports that making an app more engaging and user-centered will increase consistent use (Brock et al., 2018; McCurdie et al., 2012). Thus, we sought to increase compliance and engagement in order to optimize symptom data collection for acutely ill children and adolescents.

Well-designed apps begin with the end-user preferences in mind (Bendixen et al., 2017; Hilliard et al., 2014; Sage et al., 2017). They also promote sustained use by incorporating engaging features (Sage et al., 2017). Features that are of interest to users of all ages include the following: (1) educational features that inform users about symptoms or disease, (2) prompts or alerts such as medication reminders (Sage et al., 2017), and (3) gamification and “streaks” designed to create fun and reward positive behaviors (Hilliard et al., 2014; Sage et al., 2017). Still further, adding social networking platforms for users to engage with others, or communication conduits for clinicians, improves app engagement (Hilliard et al., 2014). Finally, including features such as personalization and ease of use also promotes sustained app use (Bendixen et al., 2017). For acutely ill children and adolescents, ease of use is an important consideration to minimize the effort needed to record symptom data and optimize data collection during periods of high symptom distress.

Purpose

The purpose of this study was to employ user-centered design principles and medical professional input in order to obtain feedback and insight into redesigning our TRU-PBMT app for children and adolescents with cancer or undergoing blood and marrow transplantation.

Conceptual Framework

We chose user-centered design principles to guide the redesign of the app. User-centered design is an evidence-based iterative approach guided by the end users’ needs (Cafazzo et al., 2012; McCurdie et al., 2012) and is considered a key feature to improve adoption of and increasing engagement and satisfaction with technologies (Cafazzo et al., 2012; Hilliard et al., 2014; Riley et al., 2019; Sage et al., 2017). The approach uses pilot testing, satisfaction assessment, usability assessment (using interviews and focus groups), and performance analysis (Brunner et al., 2017) of the app’s impact as an effective symptom monitoring tool.

Method: Design, Setting, and Sample

Our two-phase project employed an iterative user-centered design approach. In Phase 1, with institutional review board approval and following parental consent and patient assent, we pilot tested the initial TRU-PBMT app with 10 PBMT patients aged 10 to 17 years, undergoing their first transplant in a hospital in the Southeastern United States. The inclusion criteria also included the ability to read and speak English. Patients were interviewed at the end of the study to obtain their perspectives and feedback on using the app.

In addition, we surveyed 30 pediatric oncology clinicians (medical doctors, nurse practitioners, registered nurses, and child life therapists) from the same institution to obtain their input and feedback for the app redesign. We distributed surveys to each clinician at group sessions. In Phase 2, the design and development team identified possible design features based on the pediatric interview data and clinician survey data and incorporated these features into the new version of the app.

Phase 1

Data Collection

Pediatric Data Collection

We conducted semistructured interviews with six children and adolescents who had used the app during the pilot study. Interview questions focused on their experiences using the app as well as their perspectives on feasibility, usability, and acceptability. We also requested suggestions for additions and improvement for the app. Interviews were recorded using field notes, transcribed, and then analyzed by grouping answers into common themes. Questions were designed to probe areas such as (1) Describe how you used the app each day; (2) Did you enjoy using the app, why or why not? (3) What features did you like? (4) Describe the challenges in using the app; (5) What features, information, or changes would you like to see in the app?

Clinician Data Collection

We employed a user-centered design process to obtain perspectives, ideas, and suggestions from 30 pediatric clinicians with a range of expertise. We conducted group presentations that consisted of three to 10 members and lasted approximately 15 to 30 minutes. A short presentation on the app was first given to the group followed by a demonstration of the original TRU-PBMT app. This was followed by a hands-on interactive session using three to five iPads/iPhones loaded with the app to allow clinicians to engage with the app and explore the app’s features. The group sessions ended with survey data collection. Survey questions focused on asking clinicians’ perspectives on ease of use for patients, assessing the capture of clinically meaningful symptom data for patients, and obtaining suggestions through open-ended questions for the app redesign.

Data Analysis

Pediatric interview data were recorded in real time using written notes and later transcribed by a study team qualitative researcher. Interview data were analyzed by organizing it according to each interview question and developing common themes within each question category (Zhang & Wildemuth, 2009). Descriptive statistics were used to analyze frequencies and percentages of interview and survey data using Microsoft Excel 2013.

Results

In this section, we present the results from the pediatric interviews and the clinician surveys.

Pediatric Results

Six of the 10 hospitalized PBMT patients from the pilot study agreed to be interviewed in order to provide feedback and input for the app redesign (Table 1). These children and adolescents participated in the study ranging from 76 to 118 days and engaged with the app an average of 52% of the days they were in the study. Ages ranged from 10 to 17 years, with the mean age 14.6 years; 83% were female; approximately 67% identified as Black or African American, 17% identified as Hispanic, and 17% identified as White. Three major themes emerged to redesign and improve the app: (1) make it more engaging and fun—endorsed by 100% of the users; (2) make it more personal—endorsed by 83% of the users; and (3) add features such as rewards to motivate users—endorsed by 100% of the users (Table 1).

Table 1.

Interview Themes From Previous Users Who Used the TRU-PBMT App.

Feedback	n (%)	Patientsa	Theme
What features, information, or changes would you like to see in the app?	6 (100)	Make it more fun, add games, emojis.	Engaging and fun
	5 (83)	Let us make an avatar or use our pictures. Add charts and emojis.	Personal
	6 (100)	Reward us for using it. Make levels; let us earn points and streaks. Let us win gift cards for participation.	Reward and motivation

^an = 6.

Clinician Results

We surveyed 30 clinicians (Table 2) and obtained their feedback through survey responses and open-ended questions (Table 3). Clinicians (n = 3) provided detailed requests for daily patient care activities (teeth brushing and walking laps). They (n = 10) suggested consolidating the symptom list to make it less burdensome for the children—for example, listing “bleeding” in the symptom tracker as a solitary symptom rather than having a list of three to four sites where bleeding typically occurs (mouth, nose, stool, and urine). Clinicians (n = 9) suggested adding a mood tracker and emphasized that depression and anxiety were needed symptom additions. Clinicians (n = 7) also suggested adding rewards or incentives to encourage engagement and motivate patients, and clinicians (n = 5) suggested personalizing the app with avatars and custom symptom charts. Similar to the children’s responses, clinicians (n = 3) suggested making the app more fun and engaging by adding emojis, especially for the symptom tracker.

Table 2.

Clinician Demographic Statistics.

Participanta	n (%)	Male, n (%)	Female, n (%)
Medical doctor	4 (13)	1 (3)	3 (10)
Nurse practitioner	3 (10)	1 (3)	2 (7)
Registered nurse	20 (67)	2 (7)	18 (60)
Child life therapist	3 (10)	0 (0)	3 (10)

^an = 30.

Table 3.

Additional Open-Ended Clinician Survey Responses.

Question	n (%)	Pediatric cliniciansa
Anything else (symptoms or features) that would be important that we should consider to add or remove from the app?	10 (33) 9 (30)	Make it easier to use. Too many symptoms, consider combining by topic: pain choices and bleeding choices. Add a “mood tracker” for patients to indicate their mood each day. Add depression and anxiety to the symptom list.
	7 (23)	Add incentives like badges, icons, or levels to motivate kids to move more.
	5 (17)	Personalize the app (with avatars, allow child to add pictures, customize symptom charts for each child).
	4 (13)	Add “streaks” to increase engagement with the app.
	3 (10)	Add daily care goals: laps walked, mouth care, bath, and linen change.
	3 (10)	Add child-friendly language, emojis, and pictures to symptom tracker to make it fun and engaging.

^an = 30.

Phase 2: App Redesign

The first phase of the app redesign process incorporated feedback and suggestions from children and adolescents. The three key areas of redesign included ways to make the app more engaging and personal and to offer rewards for using it.

Engagement

The user experience is a critical feature to address when developing an app; the more positive the experience, the more likely the user will engage with the app (Muralidharan et al., 2017; Riley et al., 2019). To improve the user experience, we increased the appeal and ease of use of the app by adding symptom emojis (Figure 1). Emojis are pictographs that represent emotions, feelings, and moods through facial expressions (Jaeger et al., 2017; Kralj Novak et al., 2015). They are likable, popular, and easy-to-understand communication tools (Kralj Novak et al., 2015; Stonbraker et al., 2019; Thompson et al., 2018) and provide an easy way for children and adolescents to express themselves. Current research shows that emojis can be especially useful for expressing symptoms (Thompson et al., 2018), and efforts are underway to demonstrate that emojis are a valid and reliable measure of patient well-being (Jaeger et al., 2017; Tandyonomanu & Tsuroyya, 2018). The study team made the initial selection of emojis for the app based on face validity.

Personalized

The app serves as a personal symptom diary for each child. Making an app more personal is a user-centered approach that should inspire children and adolescents to use the app (Bendixen et al., 2017) in order to record symptom data. We added personalized symptom charts so they can monitor and visually track their symptom intensities and changes over time (Figure 2). Digital diaries, such as symptom trackers, can be useful for this purpose and have been shown to increase compliance with recording (Metsaranta et al., 2019; Taylor et al., 2019).

Figure 2.

Line charts displaying symptom trends over 12 days.

Motivation and Rewards

Positive reinforcement can be a motivating factor. On the first page of the app, users can earn a gold star for completing their daily care goals (Figure 3). Similar to an icon or a badge, it serves as a reward and motivator. This feature is displayed prominently on the first page. We also included colorful radio buttons used in the TRU-PBMT app that could be filled in to serve as motivating prompts as users progressed toward their daily care goals.

Figure 3.

The first page of the app is the Health page, which prominently displays daily care goals. Users can earn a gold star as a reward for completing goals. The page also reminds users to do something fun and relaxing each day.

In addition to the feedback from children and adolescents, we incorporated changes to the app based on feedback from clinicians. Their suggestions focused on making the app more meaningful, motivational, and easier to use. Additionally, we added features to make it more fun and likable in order to increase engagement with the app.

Meaningfulness

We used clinician feedback to redesign the app in order to help capture meaningful and clinically relevant symptom data. Physical symptoms have historically gained more attention than psychological symptoms; however, symptoms such as depression and anxiety are increasingly identified as significant symptoms to manage and research (Hockenberry et al., 2017; Sourkes, 2018). We added a mood tracker page using emojis (Figure 4) and added depression and anxiety in the text page symptom tracker to collect data on this increasingly important area of symptom research (Figure 1).

Figure 4.

Mood tracker: Mood (happy, angry, sad, tired, and worried) and the intensity on a scale from 0-10 can be recorded each day.

Motivating and Rewarding

Similar to the child and adolescent feedback, clinicians suggested adding rewards as motivation. In addition to the gold star that can be earned, we also included prompts to motivate the children and adolescents to “do something fun, educational, and relaxing” each day (Figure 3).

Easy

Clinicians recommended reducing the total number of symptoms by combining similar symptoms and removing some of the less frequently experienced symptoms. We also included an audible prompt each evening at 6:00 p.m. to serve as a reminder and to make it easier for users to record the data each day. Audible alarms have been shown to increase compliance with recording data in apps (Stinson et al., 2013).

Fun

Clinicians, like the children and adolescents, gave input on making the app more fun and pediatric-friendly. We added emojis to our mood tracker and symptom tracker and found that other researchers have successfully used similar approaches (Thompson et al., 2018).

Discussion

Mobile technologies such as smartphone apps have the potential to facilitate symptom monitoring and management for children with life-threatening illness. They present a novel approach to collecting daily self-report data for research purposes and to improve patient and health care provider understanding of patients’ real-time symptoms and provide insight into how these symptoms change over time (Brock et al., 2018; Fortier et al., 2016; Rodgers et al., 2013). Furthermore, by adding features such as symptom emojis, children and adolescents are able to easily use a familiar and common communication tool. Emojis, expressive symbols used in everyday life, may offer a new way to communicate the symptom experience (Stonbraker et al., 2019), particularly for children and adolescents.

Evidence-based features such as user-centered design principles are needed to make an app useful and more “sticky,” a concept that describes the continued and repeated use of a device over time. In this study, we aimed to improve on our current symptom collection app (TRU-PBMT) by incorporating patient-user and clinician insight. We chose an evidence-based approach and incorporated user-centered design principles during the development phase to ensure that the redesigned app would fulfill the end users’ expectations and meet their needs.

The majority of previous health apps have lacked the input of medical professionals (Hamel et al., 2019; Krebs & Duncan, 2015; Petersen & Hempler, 2017). Therefore, this was a key element for our app redesign to enhance content and make it relevant to children and adolescents with cancer or undergoing a PBMT. Our multidisciplinary app design and development team consisted of a physician researcher, a nurse scientist with digital health expertise, a nurse scientist with symptom science expertise, a nurse who specializes in mobile app development, a pediatric oncology nurse practitioner, and a pediatric nurse. In addition, we incorporated suggestions from diverse groups of pediatric clinicians who possess knowledge of and expertise in understanding the challenges these patients experience. This new version of the app retained the name TRU-PBMT (for patients undergoing PBMT) and was named TRU-Onc (Technology Recordings to better Understand Oncology) for patients with cancer. The app expands our ability to collect symptom data in different populations of acutely ill children who experience similar symptoms from treatment.

The limitations of our study included the following: a small sample size of previous users and that these users were predominantly female. A larger and more diverse sample size may offer insights that this small group did not address. Additionally, we collected data at only one site, which may limit generalizability to all pediatric populations.

We plan to conduct usability testing with the redesigned app with pediatric oncology patients and PBMT patients. In addition, we plan to expand the app to include features based on a child’s age and preferences to take developmental differences into consideration. We also hope to integrate these data into the electronic health record so clinicians can incorporate this information into daily clinical management.

Future work will also increase the evidence base for the use of emojis as a valid measure of symptom data collection. We plan to further assess the validity of each app emoji and explore the similarities and/or differences of emoji meaning across differing cultures and age-groups.

Conclusion

Mobile technologies represent a new frontier in symptom management by fostering real-time, accurate, and reliable symptom data collection in acutely ill children and adolescents. Leveraging mobile health technologies may be a useful and acceptable approach; however, design and software development needs to be evidenced-based and informed by user needs. We employed user-centered design principles and medical professional input to redesign our symptom app. The TRU-PBMT and TRU-Onc apps offer a unique approach as effective user-friendly symptom collection tools that will contribute to symptom research for acutely ill children and adolescents.

Author Biographies

Jacqueline Vaughn, BSN, RN, is a registered nurse and a PhD candidate at the Duke University School of Nursing. Her research focuses on the use of mHealth technology to enhance symptom monitoring and management for acutely ill children.

Nirmish Shah, MD, is an associate professor of Medicine in the Duke University School of Medicine, an associate professor in Pediatrics, and an associate professor in the Duke University School of Nursing. His research interests include pain management in patients with Sickle Cell disease and the use of digital technology to improve patient care outcomes.

Jude Jonassaint, RN, is a registered nurse and CEO of Sickle Soft, a mobile app development company that specializes in health care apps.

Nichol Harris, MSN, RN, CPNP-PC/AC, is a pediatric oncology nurse practitioner with Duke Children’s Health Center.

Sharron Docherty, PhD, PNP, FAAN, is an associate professor in the Duke University School of Nursing. Her research interests include improving outcomes for children, adolescents, young adults and families undergoing intensive treatment for life-threatening and chronic conditions. She studies how to improve care models, symptom management, and decision making from diagnosis through end of life.

Ryan Shaw, PhD, RN, is an associate professor in the Duke University School of Nursing. He is a digital health scientist and health informaticist. His research focuses on how health data from mobile health and novel sensing technologies can be used in real-time care delivery to improve health outcomes.