Development and Evaluation of Internet-based Health Technology in Pediatric Oncology: A Scoping Review

Abstract

Objective:

There is an increased awareness to identify symptomatic experiences in children undergoing chemotherapy. An Internet-based health technology accessible and friendly for children and parents to report health problems during chemotherapy has been well-developed in developed countries. The purpose of this scoping review is to provide a comprehensive view of relevant research related to the emergence of health applications in pediatric oncology so that it can provide information for design and evaluation in the future.

Methods:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines support this scoping review. To investigate the evidence on the development of Internet-based health technology, Science Direct, Scopus, PubMed, EBSCOHOST Medline, IEEEXplore, ProQuest, and Nature databases were searched between 2015 and 2021.

Results:

14 articles met the inclusion criteria with nine Internet-based health technologies. Moreover, four of nine mobile health apps use a theoretical foundation (SPARK for children and family member, Empower Stars!, THRIVE, and Facebook-based “Healthy Teens for Soaam”), three apps apply four stages of development and testing, and all apps have met the category of the degree of attachment of patients to the application.

Conclusion:

The effect of Internet-based health technology through a scientific process by paying attention to the underlying theories, user needs, developer passion, application testing, and evaluation methods is the key to success.

Introduction

Nowadays, cancer is the leading cause of death among children and adolescents in the world (World Health Organization, 2018). Children are diagnosed with cancer approximately 400,000/year (Steliarova-Foucher et al., 2017), and more than 80% of children with cancer in high-income countries are cured (Lam et al., 2019). Health professionals have made many efforts to increase life expectancy for patients, particularly children with cancer, one of which is the discovery of chemotherapy. Since its discovery as a cancer treatment, children who have the disease can survive longer. However, children and adolescents with cancer experience various interrelated symptoms due to illness and treatment (Linder et al., 2015; Torres et al., 2019), which can have an impact on the health-related quality of life and coping abilities of patients (Heydarnejad et al., 2011; Schepers et al., 2016).

During treatment in children with cancer, symptoms are expressions of feelings rather than side effects (McCulloch et al., 2018) and subjective experiences that describe changes in a person’s biopsychological, function, sensation, or cognition (Bender et al., 2018). Symptoms of chemotherapy that are common experienced by children are in the form of both physical and psychosocial symptoms (Torres et al., 2019), including fatigue, nausea, loss of appetite, pain, difficulty sleeping, worry, feelings of sadness, and anxiety (Cook et al., 2019; McCulloch et al., 2018; Nurgali et al., 2018; Poder et al., 2010; Torres et al., 2019), low self-esteem (Sherief et al., 2015). Monitoring and management of symptoms due to effective chemotherapy in patients are essential (McCann et al., 2009) so that no development of severe chemotherapy toxicity develops into additional stress (McCaffrey, 2006; McCann et al., 2009), prolonged hospitalization, and in some cases, ultimately death (McCann et al., 2009), and decreased psychosocial well-being and quality of life for children and adolescents (Torres et al., 2018).

Technological advances have also influenced developments in the health sector regarding the emergence of various web-based and mobile health applications (mHealth apps). There are almost 3 million Android apps in the Google Play store and more than 3 million apps in the Apple App store (Cannon, 2018). The top two design categories are health management and disease management apps, while other types include self-diagnosis, medication reminders, and electronic patient portal apps (Kao and Liebovitz, 2017). These mHealth apps have the potential to provide assistance almost any time with a low cost, quick access to high-quality health information based on scientific evidence on a global scale (Cook et al., 2016; Kao and Liebovitz, 2017), thereby reducing the number of visits to the clinic/hospital (Nkoy et al., 2019), and improving communication of patients and health care providers (Tomines, 2019).

Due to chemotherapy, symptoms are essential aspects that are often difficult to convey by pediatric patients with cancer, so the health care provider does not identify them. As the closest person to a child, even parents can feel stressed and uncertain with the management of complex treatment regimens that produce various side effects for the child. Qualitative studies provide the theme of experience of care as one of the main concerns where parents express some examples of negative experiences that ultimately lead to changes in hospital service providers, namely the need to have easy access to health care providers and knowledge of their children’s daily regimens (Morrison et al., 2016).

Currently, most developed countries have available a variety of web-based or mHealth apps to monitor the symptoms and management of symptoms due to chemotherapy both in mature patients and children with chronic conditions, including cancer. The number of secondary studies reporting on the use of electronic symptom monitors is increasing; however, few publications are available regarding how the process of developing and testing electronic health applications.

Based on the explanation above, this study is a scoping review asking the question: what are the steps of developing the Internet-based health technology, which is useful, acceptable, and child-friendly in pediatric oncology in developed countries?

Review questions/objectives

Our review focuses on exploring critical aspects of developing health applications that affect the ease of use, perceived benefits, and their contribution to improving outcomes for pediatric patients with cancer. To ensure the range of substance from the literature drawn on topics of interest, we submit the following preliminary research statements to guide the search:

1. What types of Internet-based health technology are used in the area of pediatric oncology?

2. What is the content or contents of Internet-based health technology that has been developed?

3. What is the theoretical foundation in developing Internet-based health technology in the area of pediatric oncology?

4. What are the steps for developing and testing an Internet-based health technology?

Materials and Methods

This scoping review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher et al., 2009). This review also refers to the guidelines for conducting scoping reviews (Peters et al., 2015).

Inclusion criteria

Types of Participants

This review searched for studies focusing on pediatric patients with cancer, children with acute lymphocytic leukemia, children with hematopoietic stem cells transplantation (HSCT) from aged 0 to 18 years in the hospital or home as populations. Studies that focused on health care providers (nurses, doctors) in pediatric oncology areas and caregivers (parents or family) involved on the internet-based health technology intervention and development were also included.

Types of Phenomena of Interest

In this review, studies that evaluated the process of development of Internet-based health technology involving either health care providers (nurses or doctors) or children or caregivers (parents or family) were considered.

Types of Studies

Studies using qualitative and quantitative designs were included as inclusion criteria.

Types of Outcomes

In this scoping review, the following items were suggested as outcomes measured in the development of Internet-based health technology:

1. Perceived ease of use

2. Perceived benefit

3. Attitude

4. Behavioral intention or acceptance

5. Actual use

Search strategy

Title, abstract, and study were included if they meet these criteria. Only articles published in English were included in this review. The search was improved by limiting the review to only studies with the full text. Editorials, comments, or reviews about the health application of cancer patient symptom management were included because the purpose of this scoping review was to conclude and report research findings on topics of interest and identify research gaps in the literature to help in planning and implementing in the future.

The following seven electronic databases were used to search literature: Science Direct, Scopus, PubMed, EBSCOHOST Medline, IEEEXplore, ProQuest, and Nature. Furthermore, publication time was limited from 2015 to 2021.

Search strategies used keywords, MeSH terms, and combinations of synonyms with subjects on all items using Boolean (‘AND’ and ‘OR’) operators such as symptom management (OR symptom control) AND pediatric oncology (OR pediatric cancer) AND mHealth (OR mobile health OR mobile application OR mobile app OR smartphone application OR telehealth OR eHealth).

Selection process of article

The electronic search of 7 databases obtained 4,370 articles. Selected articles were entered into Mendeley’s reference manager software, and duplications were eliminated. Then, one author (D.N) and a research assistant (A.W, a bachelor student in Medicine and has experience in articles’ search) separately reviewed the titles and abstracts of the articles relevant to the research questions. In cases where the study’s relevance could not be identified from the abstract, the full-text articles were examined. Non-conformities were resolved through reaching an agreement or consultation with a third reviewer (L.Z). In the next stage, the full-text of the articles was retrieved and reviewed to comply with the inclusion and exclusion criteria. Finally, 14 articles were included. The selection process for this article is illustrated in Figure 1.

Figure 1.

PRISMA Flowchart (Moher et al., 2009)

Extracting and charting the results

Data were extracted from the selected papers using a data extraction form developed for the study. Three reviewers (E.S.H, F.H, E.S) independently checked the full-text of the selected articles and removed the data items. The results were then compared, and disagreements were resolved by discussion, after which the form was revised where necessary. This scoping review presented an apparent description of the existing literature on this topic. Hence, assessment of methodological quality was not performed to include or exclude studies based on quality scores or risk of bias due to the heterogeneity of study designs.

Data collection

Scoping review framework involves making a table of the selected articles. Conclusions were developed from each manuscript related to the author, year, location of research, research design, research methods, sample size, and study results. A list of the studies involved is presented in Table 1.

Table 1.

Included Studies

Authors	Country	Methods	Subject	Summary of Findings
Baggott et al. (2015)	Norway	Descriptive	Children with cancer aged 7-12 years and their parents	Sisom was completed in less than 30 minutes, with a high rating in ease of use and the perceived benefits of parental participants.
Kock et al. (2015)	German	Descriptive	Children aged 15-17 years and family of children	The total scores for the group of children and their families were 130.5 and 131.8, respectively, ranging from 115 - 147. This shows that this software solution has outstanding functionality and ergonomics.
Jibb et al. (2017)	Toronto, Canada, USA	Qualitative	Adolescent aged 12-18 years old	The Pain Squad+ application can be received and completed in an average of 4.3 minutes. There were problems with software malfunctions, lack of interface design and confusing text that need revision.
Bruggers et al. (2018)	Utah, USA	Qualitative and quantitative data	Children, parents, health care provider	The survey picture tool showed that two favorite games were collecting star prizes and terraforming the planet. Travelling from space to the space station was a less attractive component. Two favorite mini-games were the Positron Planet Core Game and the Activity Collecting Game, while the less preferred one was the exercise mini-game, which was the exercise game for choosing healthy foods. The average scores for all statements of the provider and parents were above 3.5 and 3.0, respectively.
Chung et al. (2018)	USA	Mixed methods study	Phase I: HCP and parents of cancer children undergoing treatment. Phase II: 30 parents of children aged 3-17 years who are undergoing cancer treatment	Parents reported high satisfaction with the C-TIPS skills education and training module, stress decreased and relaxation increased significantly (p = 0.004; 0.05).
O'Sullivan et al. (2018)	Toronto, Canada, USA	Cohort	Children with cancer aged 8-18 years	All children understood the application's features, and each of the four concepts was more difficult to understand after using the help menu. 19 out of 20 children think the application was an excellent way to communicate with doctors and nurses.
Cook et al. (2019	USA	Quantitative and qualitative data	Children and adolescent with cancer and bone marrow transplant resipients aged 8-18 years	All children understood the application's features, and each of the four concepts was more difficult to understand after using the help menu. 19 out of 20 children think the application was an excellent way to communicate with doctors and nurses.
Vettese et al. (2019)	USA	Longitudinal study	Children and adolescent with cancer and bone marrow transplant recipients aged 8-18 years	Of the 30 children enrolled, the median day of completing the SPARK application was 5 (range 3-5). Almost all (96.6%) thought that screening for symptoms using SPARK was easy or very easy. All participants understood the SPARK symptom report.
Tsimicalis et al. (2019)	Norway	Qualitative	Children with cancer, aged 6-13 years	Many children (33.85%) conveyed verbal responses to at least one symptom; 1) discussing their thoughts and feelings aloud, 2) categorizing their symptoms as pleasant or unpleasant, 3) reflecting their experiences with various levels of certainty. 23 parents (74%) provided at least one input: 1) clarifying, 2) validating their child's response, 3) guiding their child's response, 4) responding on behalf of their child, and 5) making casual jokes/comments.
Svedberg et al. (2019)	Sweden	Quantitative and qualitative methods	Children aged 0-17 years in four pediatric care centres	The successful implementation of Sisom as an eHealth-based communication tool to strengthen children's participation is following the values and goals of the organization, and the beliefs and expectations of health professionals on the benefits of using Sisom.
Schwartz et al. (2020)	Philadelphia	Pilot RCT	61 AYA (12-25 years)	The THRIVE intervention showed a high level of acceptance and feasibility. 71% of knowledge in the intervention group increased. AYA in the intervention group who chose to increase fruit/vegetable consumption rose by 29% compared to the control group by 15%. One person who decided to quit smoking in the intervention group reported not smoking anymore.
Park et al. (2020)	Korea	Qualitative (focus group discussion)	Analysis phase: 12 adolescents with cancer. Design phase: Four specialists in pediatric hematology-oncology unit with ten years of work experience; 2 nurses with master's education in pediatric nursing; 3 doctors with doctoral education. Testing phase: two experts in usability and computer-human interaction. 11 children aged 13-18 years, with a cancer diagnosis and any treatment.	Adolescents with cancer showed an interest in seeing success stories of childhood cancer and self-management after discharge and preferred multimedia content (video clips) over text. The heuristic and user tests identified usability issues and then revised them to make them usable and valuable for cancer adolescents.
Watling et al. (2020)	Canada, USA	Qualitative	Family members (≥ 16 years) of pediatric patients undergoing cancer treatment were 100 people	SPARK family member was easy to use and understand.
Gilljam et al. (2020)	Sweden	A quasi-experimental design with mixed methods	14 children aged 6-12 years	Children's participation based on Shier's participation model was in the first three levels. Children were listened to encouraged to express their opinions, and children's views are taken into account. The proportion of pediatricians directing discussion in the intervention group was more significant (731 occasions) than in the control group (624 events). The ratio of appointment times was almost the same between the two groups (mean 17.0 minutes vs. 17.6 minutes).

USA, United States of America; HCP , Health Care Providers; C-TIPS, Cancer-Tailored Intervention for Pain and Symptoms; SPARK, Supportive Care Prioritization, Assessment, and Recommendation for Kids; RCT, Randomized-controlled trial; AYA, Adolescents and Young Adults

Results

This scoping review produced 14 articles from 6 countries, namely the United States of America (USA) (Bruggers et al., 2018; Chung et al., 2018; S. Cook et al., 2019; Jibb, Cafazzo, et al., 2017; O’Sullivan et al., 2016; Vettese et al., 2019; Watling et al., 2020), Norway (Baggott et al., 2015; Tsimicalis et al., 2019), Germany (Kock et al., 2015), Sweden (Gilljam et al., 2020; Svedberg et al., 2019), Philadelphia (Schwartz et al., 2020), and Korea (Park et al., 2020). Some articles focused on design and application development (n = 7), pilot study (n = 2), and application eligibility (n=3). Some manuscripts are related to each other where the first article describes the evaluation phase of electronic devices that measure the symptoms of chemotherapy in children (O’Sullivan et al., 2018)) and starting from this developed a web-based application called Supportive Care Prioritization, Assessment, and Recommendation for Kids (SPARK) (Cook et al., 2019).

What types of mHealth apps are used in pediatric oncology?

From the 14 manuscripts reviewed, four mobile-based apps were obtained, namely, Empower Stars!, Aftercare App., THRIVE, and Facebook-based “Healthy Teens for Soaam” (Bruggers et al., 2018; Kock et al., 2015; Park et al., 2020; Schwartz et al., 2020), web-based, namely SPARK (Cook et al., 2019; Watling et al., 2020), C-TIPS (Chung et al., 2018), and Pain Squad+ (Jibb, Cafazzo, et al., 2017) and computer-based, namely Sisom (Gilljam et al., 2020; Svedberg et al., 2019; Tsimicalis et al., 2019) (Table 2).

Table 2.

Characteristic of Health Apps

Name of Application	Aim	Type	Theory-based	Content
SPARK	Symptom screening for children with cancer and family member	Web-based application	Technology acceptance model	1. The landing page with three icons i.e patient, family and health care provider 2. Do SSPedi Now! 3. How will SSPedi help me 4. My SSPedi scores 5. Clinical practice guidelines - fatigue management and mucositis prevention
Sisom	Describe and report symptoms for children with cancer and chronic illness	Interactive computer-based animation tool on a computer touchpad	CHOICE, a module for adult cancer and rehabilitation patients	It contains an avatar trip to visit five islands by sailing using a ship, namely in the Hospital, About Managing Something, My Body, Mind and Feelings, Something That Can Make Fear. It measures 35 physical symptoms and 44 psychosocial problems.
Empower Stars!	Provide intervention of physical exercise and mental empowerment for children with cancer	Mobile video game	Exercise theory, health behavior theory, and video games	1. Story line about imagination space exploration stories 2. Physical activity / training designed to increase strength, flexibility, and mobility of the upper and lower limbs with fun, natural, and comfortable 3. Empowering education through questions. There are two educational formats, the first in the form of text messages about important facts related to cancer and supportive care. The second format relates to specific content in mini game challenges such as the Food Nutrition mini-game, which includes exercises for choosing healthy and unhealthy foods 4. Harware case
Aftercare App.	Follow-up care after therapy of childhood cancer survivors	Mobile app	NR	1. Disease information 2. Reminder service 3. Calender function for check-up coordination
C-TIPS	Provide intervention of pain management information and coping skills training to parents of pediatric cancer patients	Web-based application	NR	1. Twelve tailoring algorithm questions 2. Navigation bar to select soundtrack and image during program 3. Relaxation training and practice in diaphragmatic breathing 4. COPE modules
Pain Squad+	Real-time pain management app for adolescents with cancer	Web-based smartphone application	NR	1. Pain assessment: 22-items pain questionnaire 2. Real-time pain management advice (pharmacological, psychological, physical) 3. Mental games, audio-recorded muscle relaxation-based mindfulness
THRIVE	THRIVE messages aims to raise awareness of health vulnerability, emphasize the importance of ongoing engagement in care, and provide support to motivate uptake and reinforce positive health behaviors	Text messaging on smartphone	SMART, Health Belief Model, social cognitive theory	Three message text domains: 1. INFORM (information on health promotion, acute problems, sequelae, and resources) 2. MOTIVATE (motivation, encouragement, reinforcement, and monitor the achievement of six goals) 3. ENGAGE (engagement in follow-up care, sharing experiences, promoting autonomy, and providing support)
Facebook-based "Healthy Teens for Soaam"	Educational program for teenagers with cancer on Facebook	Multimedian in Facebook Group	NR	Eight modules for eight weeks. Each module in the form of PowerPoint slides consists of learning objectives, structured learning materials, multimedia (video clips or games), and discussion topics. The module contains the characteristics and types of childhood cancer, treatment of childhood cancer, returning to school and society, and the people around the child, i.e family and friends.

Note. SPARK, Supportive Care Prioritization, Assessment, and Recommendation for Kids; SSPedi, The Symptom Screening in Pediatrics Tool; CHOICE, Creating better Health Outcomes by Improving Communication about Patients' Experiences; C-TIPS , Cancer-Tailored Intervention for Pain and Symptoms; COPE, Children's Oncology Pain Education; NR, Not reported; THRIVE, Texting Health Resources to Inform, motiVate, and Engage; SMART, Socio-ecological Model of AYA Readiness to Transition to Adult Care; AYA, Adolescents and Young Adults

What is the content or content of mHealth apps that have been developed?

Of the nine reviewed applications (apps), five apps contained symptoms due to chemotherapy that could be accessed and identified by children with cancer, seven apps provided educational information and six apps had how to manage symptoms due to cancer and chemotherapy. Only two apps contained reminders to implement the interventions (Table 3). The SPARK app has 15 items of disturbing symptoms felt by children who received cancer treatment yesterday or today. The Sisom app includes 35 physical symptoms and 44 psychosocial problems. Children can detect it interestingly through an avatar trip visiting five islands by boat. Pain Squad+ and C-TIPS assess pain using a questionnaire.

Table 3.

Characteristics of Applications' Content

Name of Application	Symptoms monitoring	Information/education	Motivation	Intervention	Support	Interactive	Reminder
SPARK (for children with cancer)	✓	×	×	×	×	×	×
SPARK (for family member)	✓	✓	×	✓	×	×	×
Sisom	✓	×	×	×	×	✓	×
Aftercare App.	×	✓	×	×	×	×	✓
Empower Stars!	×	✓	×	✓	×	✓	×
C-TIPS	✓	✓	×	✓	×	×	×
Pain Squad+	✓	✓	×	✓	×	×	✓
THRIVE	×	✓	✓	✓	✓	×	×
Facebook-based "Healthy Teens for Soaam"	×	✓	×	✓	✓	✓	×

SPARK, Supportive Care Prioritization, Assessment, and Recommendation for Kids; C-TIPS, Cancer-Tailored Intervention for Pain and Symptoms; THRIVE, Texting Health Resources to Inform, motiVate, and Engage

Furthermore, the Empower Stars! is an application that includes a one-day therapeutic intervention in cancer children in video games, physical activity and health education about cancer, supportive care, and food nutrition mini-games. C-TIPS is a web-based application intended for parents with cancer children who are undergoing treatment. This application provides information and education in managing pain in children with cancer empirically both pharmacologically and non-pharmacologically.

Unlike C-TIPS, Pain Squad+ was designed for adolescents diagnosed with cancer to manage the pain independently. Pain Squad+ contains a questionnaire to assess pain, recommendations based on pain responses in pharmacological, psychological, and/or physical. Besides, this application also includes a warning system to complete the review after obtaining recommendations. The next application, Aftercare App, contained information about diseases, reminder services, and consultation schedules.

SPARK for family members currently presents two recommendations to treat fatigue and prevent mucositis in children with cancer. THRIVE and Healthy Teens for Soaam are two very interesting apps for teens with cancer to get information about cancer and its treatment. The THRIVE application provides short, easy-to-understand messages for adolescents about cancer and provides support for adolescents to achieve their healthy goals. Meanwhile, “Healthy Teens for Soaam” provides 8 educational modules in the form of multimedia PowerPoints.

What is the theoretical foundation in developing health applications in the area of pediatric oncology?

The first step to developing an application for people living with cancer is to review what is available through literature studies and the application’s existence (Davis and Oakley-Girvan, 2017). Three of the nine apps use specific theories to develop apps, while others use application modules for cancer in adults and rehabilitation patients, and four apps do not clearly state the theoretical basis used. SPARK uses the Technology Acceptance Model (TAM), and this is the basis for focusing each domain’s evaluation on model theory during application development. This model explains that perceived ease of use and perceived benefits are predictors of behavior and actual use of health technology. Empower Stars! applies the theory of exercise, behavioral health, and video games. Using this theory as a foundation for building this app is designed to be interactive, participatory, exercise promotion, empowering, and portable and can be played in inpatient and outpatient settings. The development of THRIVE is influenced by the SMART (Social-ecological Model of Adolescents and young adults Readiness to Transition to Adult Care) model as well as two other theories, namely the Health Belief Model and social cognitive theory. These models and theories help them to develop educational programs and interventions to promote health behaviors and awareness for adolescents with cancer.

What are the steps for developing and testing a health application?

In general, all apps go through three or four stages: the initial stages of determining ideas and initial concepts, the stages of development, testing, and evaluation (Table 4). The initial phase of SPARK is the development of an appropriate symptom screening tool in children with cancer (described in a different article), the symptom screening in pediatric tools (SSPedi), 15-item paper-based and electronic-based, using a 5-point Likert scale, where the SSPedi is used in apps which were later developed. The electronic version has features specifically designed to facilitate children’s reports, and children find it easy to use. The initial Sisom is in the form of a review of the literature and focuses groups with clinicians (doctors, nurses, psychologists) who review the preliminary symptoms list for relevance, comprehensiveness, completeness, and degree of detail; with six parents of children with cancer to see the integrity of the term. Likewise, the initial phase of C-TIPS forms a task force of health care providers. They have to produce the contents of the app, organize it, and provide feedback on the C-TIPS module. After that, hold a discussion group with parents of cancer children to obtain input related to the app’s contents and use. Unlike the case with Empower Stars!, the concept of video game interventions in children with cancer that combines physical exercise and mental empowerment was born from clinical observation at the level of resilience and determination of children and families when diagnosed and through cancer treatment; significant physical and emotional deterioration during treatment; and the sense of empowerment displayed by children in overcoming illness and barriers related to treatment. “Healthy Teens for Soaam”, in the initial stage, they analyzed the needs of adolescents with cancer through focus group discussions, whereas THRIVE uses fundamental models and theories to find the right app for adolescents with cancer.

Table 4.

Application Development and Testing Stage

Name of Application	Early-Stage	Development Stage	Testing Stage	Evaluation Stage
SPARK (for children with cancer)	SSPedi testing for SPARK content	Development Team and Web Application Research	Low-fidelity phase; Design phase; High-fidelity phase	Evaluation of each step through interview. The quantitative assessment includes understandable applications, uses, and benefits.
SPARK (for family member)	--	Development of recommendations using CPGs and panel discussion	Usability test (thinks aloud and cognitive interviews)	Ease of use
SISOM	Literature review, focus group discussion	--	Interface test on 12 healthy school children; Interview with five healthy school children; and Usability test on 14 healthy children and cancer	Appraisal of a dyad cancer children and their parents to see aspects of the rating, the ease, and benefits of application.
Aftercare App.	--	--	--	--
Empower Stars!	Groundwork clinical observation	Multidiscplinary team, video game developer, academic and healthcare provider team	Qualitative and quantitative on multilevel subject	--
C-TIPS	Phase I. Development of C-TIPS: 1. Form a Taskforce of HCP to produce content, organization, and feedback on the C-TIPS module. 2. Focus group discussion on parents of children with cancer to get feedback on using C-TIPS and module content.	Commercial web companies develop applications	Phase II. Formative evaluation to test the usability	--
Pain Squad+	--	--	Usability testing with an iterative approach (2-3 cycles) -- thinking aloud	--
THRIVE	Based on model and theory	Reify Health built a text messaging platform and algorithm A research team consisting of psychologists, nurse practitoners, oncologists, and young adult students designed the text messages	Pilot RCT for 16 weeks	Feasibilty, acceptability, and questionnaires to assess knowledge, health behavior, health competence beliefs, and quality of life.
Facebook-based "Healthy Teens for Soaam"	Need assessment by focus group discussion	Development of module content, expert reviews, and development of Fb-based intervention program	Usability test (heuristics evaluation and users test)	Modifications and final versions of the program, expert review is carried out if necessary.

SPARK, Supportive Care Prioritization; Assessment, and Recommendation for Kids; C-TIPS, Cancer-Tailored Intervention for Pain and Symptoms; SSPedi, Symptom Screening in Pediatrics Tool; HCP, Health Care Providers; THRIVE, Texting Health Resources to Inform, motiVate, and Engage; RCT, Randomized Controlled Trial; Fb, Facebook

The next stage at SPARK is web applications development through collaboration between research teams and web application development companies. For Empower Stars!, the prototype development stage involved a multidisciplinary process from young video game developers at Spy Hop Youth Media, Academic Faculties, and a team of health care providers specializing in the care of children with cancer such as nurses, child life specialists, psychological, social workers, physical therapists, a pediatric oncologist. Similarly, C-TIPS was developed by commercial web companies. For THRIVE, they cooperate with Reify Health to build a text messaging platform and algorithm. Then, the content of the text messages was designed by a research team consisting of psychologists, practicing nurses, oncologists, and young adult students.

The second stage of “Healthy Teens for Soaam” is quite complex and structured. This stage begins with the development of module content based on the analysis of the needs of adolescents with cancer and literature derived from the publication of articles, nursing, and medical textbooks, information from health organization websites, educational materials in hospitals, and consultations with expert panels. After that, the module content design was reviewed by experts and revised and finalized into a Facebook-based program mockup. Last, this intervention program was developed for eight weeks using the private group functionality.

The third stage is implementation and testing. The app testing phase is at the initial maturity level (prototype and pilot study) in the form of functionality/stability and feasibility/usability (World Health Organization [WHO], 2016). Phase testing of SPARK through three phases, namely a low-fidelity, design, and high-fidelity, were distributed into Cohort 1-10. Through Cohort 1-3 studies, low fidelity is used to evaluate and improve the functionality and usability of SPARK web pages. Design, Cohort 4-7, the pages are modified to achieve the “look and feel” desire during this phase. Catalyst Workshop, Inc. was the design company, collaborating with Translucent Inc. and the research team to complete this phase. There were two sub-phases, namely sub-phase 1 for SSPedi report design (Cohort 4 and 5); subphase 2 (Cohort 6 and 7) for the layout of 4 additional pages, namely landing page, access to SSPedi, SSPedi information, and specific symptom trends. High fidelity, Cohort 8-10, consisted of two phases.

The Sisom application also went through three phases, namely the interface test, to determine whether the school children being interviewed can correctly identify the symptoms from the graphic display and the usability test. Using qualitative and quantitative data collection to obtain constructive feedback on the usability, engagement, and enjoyment was considered critical for the development and improvement of the game in the future. The approach used was a multilevel subject population consisting of three groups: 1) 10 children aged 7-14 years who undergo chemotherapy, 2) one parent of each child, 3) 12 health service providers, including nurses, child life specialists, social worker, and doctor. The measurement instrument used in children is a survey containing 15 statements using a 5-point Likert scale, an image survey using a ranking scale (at least favorite) for eight-game elements, and five mini-games, a simple open-question survey called Three Things. For parents of children and providers, the scale used ten items of statements suitable for parents and providers with a 5-point Likert scale and Three Things survey. C-TIPS and Pain Squad+ applications use a usability test in the form of formative evaluation and thinks aloud. “Healthy Teens for Soaam” used two techniques of usability test that is: 1) Nielsen’s heuristics by two experts in usability and human-computer interaction, and 2) user test through observation and thinks aloud. THRIVE was tested using a randomized method in two groups of adolescents with cancer aged 12-25 years, where the intervention group used THRIVE, and the control group received a handbook for 16 weeks.

The final stage was the evaluation. Several data collection methods and strategies were used during the prototype and pilot study stages to evaluate the mHealth solutions used in the home and clinic context in children with cancer. There were several ways to assess the use of apps, including cognitive interviews and focus group discussions on critical informants and application users (Tsimicalis et al., 2019; Vettese et al., 2019) and surveys on the use and benefits of applications (Vettese et al., 2019). The methods used to evaluate mHealth solutions vary, including various tools/instruments and qualitative and quantitative data collection strategies.

There was an evaluation of SPARK through each phase to see the response after every five interviews, and the research team met to decide whether SPARK needed any minor editing. After every 10-20 conversations and when the research team was satisfied that each phase was complete, the review panel met and made edits or confirmed the stage’s completion. Quantitative evaluations in the low and high-fidelity stages included understandability, usefulness, ease to understand from SPARK laps, and benefits for children with cancer receiving treatment in the future. At Sisom, the evaluation was done by involving pairs of cancer children and their parents by comparing Sisom and Memorial Symptom Assessment Scale (MSAS) in terms of rating, ease of use, and perceived benefits.

Discussion

This article describes the results of a literature search on mobile and web applications for children of cancer patients to analyze methods for developing and testing appropriate mobile and web-based health applications in this population. We identified 14 articles describing mHealth apps for children with cancer. We found mHealth apps in adults and healthy children, but they were not included in this review because we wanted to express the form of the app and its stages in children with cancer.

A review conducted by Jupp et al (2018) of 8 mHealth apps using the Mobile Application Rating Scale (MARS) found that most have a low score on the domain of engagement (range 1-5). MARS is one of the guidelines that can be used to develop new mobile health apps consisting of four fields: engagement, functionality, aesthetics, and information quality. There was no evaluation using MARS in this review because most of the apps obtained were not based on mobile apps. Hence, we based the app development process on the iterative model of software development (testing excellence.com/iterative model), which consists of four phases: requirements, design phase, implementation phase/testing, and evaluation phase. Of the four apps reviewed, only one, SPARK was developed through all the four stages of development and testing.

Creating a more useful product requires a user-centered design approach involving users in each phase, a multidisciplinary team, caring and enlightened management, and determining the user’s goals and objectives (Rubin and Chisnell, 2008). Interestingly, we found that almost all apps have involved user input by discussing and forming a multidisciplinary team in developing applications.

The weaknesses and strengths of mHealth apps can be identified through the degree of engagement of patients with the application (Cerrato and Halamka, 2019). The degree of attachment can be achieved if it meets the patient’s needs category. The categories include providing educational information, reminding patients to take specific actions, tracking medical or health data, presenting patients with data that can be placed on a mobile device, offering recommendations based on the data entered by the patient, allowing patients to send information to family or health care providers, providing social network support, and rewarding patients for changing behavior (Singh et al., 2016). We found that each app was developed based on patients’ needs identified through literature studies, discussions with health care providers and parents of children with cancer, and clinical observations in the field. According to Singh et al., (2016), when the content contained in the apps meets several categories, it is expected to increase the degree of attachment of the app user/users.

Limitations of current research

Nowadays, Internet-based health technologies for cancer children are emerging. Thus, this scoping review examines the development stage of applications that have been used in pediatric oncology. In the current literature and the Google Play store or App Store, there are various Internet-based health technologies used throughout the study to assess the symptoms and treatment of children with cancer using different approaches, making it difficult to generalize.

There are several limitations to our paper. Reviewing research articles is always limited by the time frame of the search and the criteria for selecting manuscripts which can potentially miss the contribution of valuable papers. However, in this review, we have searched for additional information through a hand search on the reference articles obtained. Besides, we have also used seven database search engines, but in the end, we found only 14 articles reviewing established criteria after removing duplicates.

Implications of the review findings

Advances in Internet-based health technology have improved the quality of cancer care in children, while increasing patient satisfaction and the quality of life for cancer children and their families. Continuous maintenance of children with cancer while in the hospital and at home requires symptoms monitoring to obtain appropriate medical and nursing care. The development of Internet-based health technology can help pediatric patients, their parents and health workers identify these symptoms. In future research, it is necessary to pay attention to various aspects in the development of health technology, including ease of use, perceived usefulness, and according to the needs of patients and families in addition to elements of reminders, safety, and comfort. This approach will increase patient engagement and compliance in using these technologies. Apart from that, it is also necessary to pay attention to effective communication between patients, families, and health workers.

In the nursing area, this technology is likely to help ease nurses’ workload in understanding symptoms that cannot be communicated directly by children with cancer. Cancer children and their families can download a personalized Internet-based health technology for use at home or in the hospital. Ultimately, the Internet-based health technology can be integrated into an electronic medical record system to be accessed easily by the authorities in health facilities.

Furthermore, a rapid rise of need for the use of Internet-based technology and mobile apps is happening today, especially since the COVID-19 pandemic. During the first wave of the COVID-19 pandemic, technology played an important role in providing safe healthcare, where telemedicine was the most commonly used technology, followed by videoconferencing (Zoom) and social media platforms (WhatsApp) (Abd-Alrazaq et al., 2021). Similarly, telemedicine and eHealth created innovations and a revolution in providing healthcare to patients with surgery globally (Taha et al., 2021). A review reported that the use of m-Health apps is a suitable method, bypassing the costly infrastructure, and is considered more accessible in terms of providing quality health care services, especially in the oncology services (Mehdizadeh et al., 2019).

In conclusion, mobile and web-based health applications require systematic and scientific monitoring and evaluation efforts through research projects in clinical and community settings. This monitoring and evaluation effort can oversee the design, implementation, and testing process of the health technology to meet the needs of users, both patients and families, and health care providers. Besides, it is also essential to consider the cost-effectiveness, reimbursement process for patients and providers to improve the application’s sustainability. All of these aspects contribute to produce appropriate decision-making on the best approach for integrating mHealth technology into the national health system.

Author Contribution Statement

D.N – wrote, designed the study, collected, and selected the articles, and edited the manuscript. L.Z – reviewed the selected articles. E.S.H – supervised and reviewed the eligible articles. F.H and E.S – reviewed the eligible articles and the manuscripts. All the authors read and approved the final manuscript.