How Is Quality of mHealth Interventions for Cancer Survivors Defined and Described? An Umbrella Review

PURPOSE

Although there are commonly accepted criteria of what defines quality of health care including cancer care, less is known about what defines quality of mHealth interventions in health care. The aim of this review was to identify how quality of mHealth interventions for cancer survivors is described and measured.

METHODS

CINAHL, EmCare, JBI, Medline, SCOPUS, and ProQuest databases from January 2008 to January 2020 were searched. Review papers with search terms related to mobile devices, quality, and cancer relevant to adults with cancer were included. Interventions needed to consist of mHealth technologies, such as mobile applications or short message service, or wearable devices. Title and abstract screening, full-text screening, and data extraction were performed independently by two reviewers. Conflicts were resolved by a third reviewer. Reviews were evaluated for coverage of quality according to six metrics defined by the Institute of Medicine: patient-centeredness, equitability, safety, effectiveness, timeliness, and efficiency. Any additional quality items were recorded. A Measurement Tool to Assess systematic Reviews (AMSTAR) was used to rate the quality of the reviews included.

RESULTS

The initial search yielded 766 papers with seven systematic reviews meeting the eligibility criteria. Four papers were of AMSTAR moderate quality, with three of low quality. The median number of quality metrics reported in a review was two (the range was 1-4). Efficacy and safety and timeliness and efficiency were most reported (n = 4), followed by usability (n = 3), equitability and access (n = 2), privacy and security (n = 2), and patient-centeredness (n = 2).

CONCLUSION

There is great variability in how quality of mHealth interventions is defined with no reviews addressing all quality metrics. A comprehensive approach to measure quality of mHealth interventions is needed.

INTRODUCTION

Digital technology offers a promise of revolutionizing health care for complex conditions such as cancer through improving access to care and supporting self-management.¹ mHealth, defined as digital technology delivered through mobile devices such as mobile phones, patient monitoring devices, and personal digital assistants (PDAs),² has been a relatively new addition to digital health and has been associated with significant interest and uptake by users.³ The uptake of mHealth technology has been particularly significant among cancer survivors with commercial applications (apps) developed for cancer survivors and endorsed by cancer survivors.^4,5 Cancer survivorship care refers to the care of cancer survivors, defined as any person diagnosed with cancer from the time of diagnosis, and focuses on prevention and surveillance for recurrences and new cancers, surveillance and management of physical effects, surveillance and management of psychosocial effects, surveillance and management of chronic medical conditions, and health promotion and disease prevention.⁶ mHealth interventions can be a valuable tool in survivorship care delivery and self-management for cancer survivors, who often have a long and complex journey with many unmet needs.⁷ As peer support is a very important aspect of survivorship care, mHealth interventions that facilitate peer support and/or are endorsed by other survivors are particularly of interest.⁸

CONTEXT

• Key Objective

• Although there are commonly accepted criteria of what defines quality of health care including cancer care, less is known about what defines quality of mHealth interventions in health care. The aim of this research was to identify how quality of mHealth interventions for cancer survivors was described.

• Knowledge Generated

• There was great variability in how quality of mHealth interventions was defined. The most common attributes of quality used included the following: efficacy and safety, timeliness and efficiency followed by usability, equitability and access, privacy and security, and patient-centeredness. None of the studies reviewed looked at all attributes.

• Relevance

• A comprehensive, consistent approach to measure quality of mHealth interventions is needed to ensure quality across all interventions irrespective of setting, population, or type of intervention.

Many mHealth interventions are directly accessible to cancer survivors and may not undergo a stringent evaluation of efficacy and safety that is required for other therapies such as medicines or devices⁹ and therefore may be a source of inaccurate information. Cancer mobile apps are produced by diverse developers including cancer organizations, health care professionals, cancer survivors, digital health researchers, and commercial developers. More importantly, the process of development may or may not include stakeholder consultation and codesign and formal evaluation of the app's efficacy, safety, and acceptability to users. A recent study from the United States that analyzed cancer treatment information shared on social media platforms showed that of all 200 popular cancer articles on social media from 2018 to 2019 articles analyzed, 32.5% (n = 65) contained misinformation and 30.5% (n = 61) contained harmful information. Among articles containing misinformation, 76.9% (50 of 65) contained harmful information.¹⁰ These findings highlight the need for appropriate quality assessment of mHealth interventions. Misinformation may lead patients to abandon evidence-based therapies or seek alternative therapies (defined as unproven cancer treatments administered by nonmedical personnel), which are associated with a decrease in cancer survival.¹¹ Accordingly, the issue of how to define and measure quality of mHealth intervention is an important research consideration.

The Institute of Medicine (IOM) identifies six domains of health care quality including safety, effectiveness, patient-centeredness, timeliness, efficiency, and equitability.¹² Although the IOM criteria apply to general health care and cancer survivorship care, little is known about quality of mHealth interventions for cancer survivors.

To address this gap, the aim of this review was to examine how quality of mHealth interventions for cancer survivors was described and evaluated. Specifically, the objectives of this review were to describe which quality domains of the IOM quality framework were reported and were there any additional domains of quality.

METHODS

The study was a review of reviews (umbrella review). The methodology was selected because of the large volume of literature about mHealth, indicating the maturity of the field. The methodological approach was developed with the assistance of the research librarian following the guidelines on conduct of systematic reviews of reviews.¹³ The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis recommendations.¹⁴

Electronic databases CINAHL, EmCare, JBI, Medline, SCOPUS, and ProQuest were searched systematically using both keywords and mesh terms. The following Medical Subject Heading and keywords were used: combinations of terms for mHealth, cancer, and quality. The full search strategy can be found in the Appendix.

Inclusion and exclusion criteria were determined before commencement of the review. Papers were included if they were a review study evaluating at least one mHealth technology in adults (18 years and older) with any type of cancer. When a paper evaluated several technologies or a technology that presents on several platforms, it was included only if mHealth tools were reported separately. Papers not containing a Methods section were excluded. Medical devices that interface with mobile platforms like ECG monitors that connect to smartphones were not considered. Only full-text available papers published in English and published from 2008 onwards were included, given that mobile phone use and mobile apps gained popularity since the introduction of the first iPhone in 2007.¹⁵

Following the search of electronic databases, search results were imported into EndNote and duplicates were eliminated. Titles and abstracts were then reviewed by the primary author (T.T.). The search results were then imported into Covidence for the purpose of collaboration with the second author (S.G.) to independently screen the titles and abstract. Subsequently, full text for each paper was retrieved and reviewed independently in Covidence by T.T. and S.G. for eligibility. Where conflicts existed, these were resolved by discussion with the third author (B.K.).

The following data were extracted from the final set of papers into a table. The included reviews were further analyzed against the listed six quality items as defined by a framework proposed by the IOM (safety, effectiveness, patient-centeredness, timeliness, efficiency, equitability; see Table 1 for definitions). Any additional quality items were recorded. References to quality items were obtained from the Results section when stated or from the Discussion section. The source of references was annotated in the data extraction.

TABLE 1.

Institute of Medicine Quality Items and Their Definitions

A Measurement Tool to Assess systematic Reviews (AMSTAR) was used to assess the quality of each included study.¹⁶

RESULTS

After removal of duplicates, the literature search yielded 766 papers from six electronic databases. Fifty-one reviews published between 2016 and 2020 were selected for title and abstract screening, 35 were selected for independent full-text screening, and seven systematic reviews met the eligibility criteria (Fig 1). Reviews were from six different countries (United States, Spain, South Korea, United Kingdom, Australia, and Brazil). The included reviews summarized 86 primary studies in 1989 individuals with cancer across 16 cancer types. Primary studies sample sizes ranged from 4 to 125 participants. Seven different mHealth interventions were identified (mobile apps, text messaging, web-based, PDA apps, mobile-based telehealth, and activity trackers), and 19 different mHealth functionalities were identified. The most reported functionality was symptom monitoring (n = 50), followed by diagnosis and early detection of disease (methods unspecified; n = 25), and education and information provision (n = 24; Table 2).

FIG 1.

PRISMA diagram. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analysis.

TABLE 2.

Quantity of mHealth Intervention Functionalities Identified in Reviews

Four papers were classified as moderate quality, and three as low quality. The determining factors for low quality ratings were the absence of methods established before the review, authors not conducting study selection independently, authors not conducting data extraction independently, lack of technique for establishing risk of bias (RoB), and lack of accounting for RoB when interpreting results. Three of seven papers did not state explicitly as having established methods before conducting the review, two did not have authors independently who perform study selection, five did not have authors who independently perform data extraction, four did not have satisfactory techniques for assessing RoB, and four papers did not account for RoB when interpreting results.

The number of quality items reported per paper ranged from 1 to 4. Four papers had findings in the Results and Discussion section, and three papers had findings in the Results section only. Efficacy and safety and timeliness and efficiency were most reported (four papers each), followed by usability (three papers), equitability and access (two papers), privacy and security (two papers), and patient-centeredness (two papers). A summary of the findings is given in Table 3.

TABLE 3.

Main Quality Findings of Reviews

Efficacy and Safety

Four reviews reported on efficacy and safety. All four reviews reported on positive outcomes with intervention. Among positive outcomes that were found in the Results section were increased patient knowledge and confidence in symptom self-management,¹⁷ increased adverse effect reporting,¹⁸ lower pain and distress levels, better self-efficacy outcomes,¹⁸ decreased fatigue and insomnia levels,¹ and increase in moderate to vigorous physical activity.¹⁹ Two reviews reported on the reduced use of health care services with intervention¹⁷ including reduced hospital readmission rates.¹⁸

One of four reviews reported results of negative outcomes including increased anxiety levels after intervention. These were attributed to increased patient knowledge of treatment or the need to use a tablet computer.¹⁷ Two reviews that reported on positive outcomes also reported results of mixed outcomes after intervention in pain and distress levels¹ with mobile apps and weight loss¹⁹ and activity levels¹⁹ with fitness trackers.

Timeliness and Efficiency

Four papers reported on timeliness and efficiency. Two reviews reported results of quicker reporting time, analysis of data reported, and feedback.^20,21 Although not reported in the Results section of the review, the Discussion section of one review of 18 studies on mHealth integration in patients with skin cancer noted that the nature of mHealth apps analyzing skin lesions precludes instantaneous feedback when compared with the conventional approach because of apps requiring images to be of high quality and stored and then forwarded for analysis.²¹

A review of nine studies evaluating mobile apps and web-based apps in patients with breast cancer reported results of a lower number of unexpected consultations, hospital readmissions, and emergency department visits after mobile app–based e-monitoring interventions as it allowed timelier e-consultations if concerns were identified.¹⁸ The Results section of another review of 20 studies evaluating mobile and PDA apps found that symptom monitoring interventions (text message, mobile app, and telehealth) reduced the use of unexpected health care services. Furthermore, patients in the intervention group also had fewer expected clinic visits because of intervention facilitating early resolution of issues with a care coordinator.¹⁷

Equitability and Accessibility

Two reviews reported on equitability and accessibility. In one review of 20 studies evaluating mobile and PDA apps, the results of patient interviews from a randomized controlled trial showed that patients felt that mobile app symptom monitoring intervention enabled easier access to cancer specialists and increased communication with clinicians.¹⁷ Another review of seven studies related to mobile apps suggested that mHealth has a potential wide reach and can transcend diverse population characteristics, demonstrated by participants' varied cancer diagnoses (lung, breast, bowel, lymphoma, and prostate), sex, and origin from different countries.¹

Patient-Centeredness

Two reviews reported on patient-centeredness. Both reviews have findings related to personalization and customization features of mHealth. In the Results section of one review of five studies of mobile apps focused on patients with breast and prostate cancer, evaluated app functionalities include the ability to customize and personalize a menu plan and a calorie tracker to establish an individual profile as demonstrated by a weight loss mobile app.²⁰ The Results section of another review of 13 studies on activity trackers found that there was a lack of personalization from autogenerated advice given by a fitness tracker and that a daily step goal in fitness trackers was seen as one-dimensional by users.¹⁹

Usability

Three reviews reported on usability. A review of 14 studies related to mHealth development, showed in the Results section that ease of use, convenience, and helpfulness were frequently assessed through questionnaires and interviews on the perceived benefits of mHealth. Concerns were also raised for older patients with cancer who may not be familiar or comfortable using mHealth, and it was recommended that developers ensure ease of use for end users or make training available.²² Another review of five studies on mobile apps in adult patients with cancer discussed usability and accessibility concerns as cancer survivors may have a degree of cognitive deterioration from treatment-related effects, subsequently recommending consideration of this issue during mHealth development processes.²⁰ A review of seven studies of adult patients with cancer evaluating mobile app interventions mentioned in the Discussion section that although smartphone ownership rates may be lower among older populations, their willingness to use mHealth is equal to that of younger populations, evident from a relatively high mean age of participants (50-69 years) across studies.¹

Privacy and Security

Two reviews reported on privacy and security. A review of 14 studies on the development of mHealth interventions in adult patients with cancer reported in the Results section that only one primary study discussed data protection and privacy. The authors subsequently recommended that mHealth developers are mindful of concerns regarding data protection and privacy.²² It was discussed in a review of five studies involving mobile apps focused on adult patients with breast and prostate cancer that patients greatly appreciated the use of no personal identifiers or other information stored on devices used, as well as pseudonymizing and other data protection features as per results from questionnaires.²⁰

DISCUSSION

This review of reviews offered an insight into how quality was reported in mHealth among cancer survivors with variability in how quality was reported. Although none of the analyzed reviews had quality in mHealth interventions as their primary aim, all reviews did report on some quality domains. No single review covered all quality domains. Six of seven reviews reported on at least one IOM quality domain. A review of 14 studies, evaluating the development of mHealth interventions for patient cancer care, did not report on any IOM quality domains but had findings in usability and privacy and security.

The results of this review showed that some quality domains were reported more than others. Specifically, effectiveness was the most common quality domain reported. This finding is not surprising and shows consistency with nondigital interventions, whereby effectiveness is commonly reported on and used as a measure of quality. The majority (7 of 13) of findings within the effectiveness and safety domain were related to the effectiveness of symptom management, which reflects symptom monitoring as the most commonly identified mHealth functionality.

All patient-centered features of the mHealth interventions included in this review revolved around the ability to customize and personalize features. The ability to personalize and customize in mHealth has the potential to make the intervention more relevant to patient's needs and circumstances and thus potentially more acceptable.

Our review showed that faster reporting times and more timely consultations resulted in fewer hospital presentations and unnecessary consultation,^17-21 lending support to the potential for mHealth to increase efficiency and reduce burden on resources.

Usability was the quality domain identified in the review but not found in the list of IOM quality domains. Usability is especially important in mHealth as mHealth poses a different paradigm when compared with conventional nondigital health interventions as its application is dependent on self-administration by a user (the patient). Usability concerns related to mHealth usage in older population groups was mentioned in two reviews of the three that reported on usability. This is due to assumptions that older populations may not be familiar with mHealth technologies and that cancer survivors are mostly an older population as demonstrated by the National Cancer Institute's recent statistical data reporting the median age of a cancer diagnosis as 66 years.²³ Cancer survivors may also face unique challenges using mHealth because of long-term chemotherapy-induced cognitive decline²⁴ or chemotherapy-induced peripheral.²⁵ Therefore, usability of mHealth especially among cancer survivors should be regarded as an important quality measure.

Privacy and data protection was another quality domain identified that was not included in the list of IOM quality domains. Many mHealth apps require the storage of personal information with the manufacturer for the purposes of registration or personalization, in contrast to conventional non-mHealth interventions such as drugs where personal information is not required or held by the manufacturer. Understandably, personal data stored on devices or in cloud services or transmitted to third-party organizations remain a concern for the industry and consumers alike, with fears that such data may be vulnerable to cyberattacks.²⁶ An alarming number of mHealth apps on the market still lack sufficient privacy and security measures such as data encryption.²⁶ For example, a study of 137 highly rated mobile apps targeting high-need, high-cost populations found that more than 60% of apps used insecure means of data transmission.²⁷ Our recommendation is that mHealth technologies for cancer survivors implement industry standard data encryption to ensure the security of private information. Apart from security issues, a potential barrier to integration of these interventions is broadly related to interoperability of diverse e-records and systems and variations in regulations on data privacy, transmission, and sharing between different jurisdictions and countries.

There is currently limited regulation among mHealth technologies despite the increased use and availability of mHealth. The Australian Therapeutic Goods Administration has recently introduced regulation on software-based medical devices. Any software that fits the definition of section 41BD of the Therapeutics Goods Act 1989 is considered a medical device, thus requiring regulation. However, software packages that are considered sources of information or lifestyle modification tools are excepted from regulation.²⁸

Standards for mHealth in oncology care are yet to be defined. Conversely, there has been recent development in defining standards for telehealth in oncology, with a systematic review published by ASCO in September 2021 recommending a set of well-defined standards and practice recommendations.²⁹ Although standards are being developed for other aspects of digital technologies in oncology, similar developments should potentially be made in mHealth in oncology. There are emerging efforts to support systematic evaluation of mHealth technology. For example, the Organisation for the Review of Care and Health Apps (ORCHA) reviews health care apps and has a large database of reviews of more than 6,000 apps available for public access.³⁰ ORCHA uses an evaluation method called Digital Technology Assessment Criteria developed by the United Kingdom's National Health Service (NHS) and is primarily focused on meeting NHS standards.³¹ More recently, international standards for health software and wellness apps have been published.³²

More research is recommended in how quality is defined in mHealth among cancer survivors. Further research may help propel the development of a comprehensive quality framework that allows for consistent evaluation of mHealth technologies in cancer survivors.

We are not aware of another umbrella review to explore how quality is defined in mHealth for cancer survivors, making this review the first of its kind to our knowledge. This review also revealed unique and pertinent quality domains to mHealth technologies not featured within the IOM set, namely, usability, privacy, and security.

This research has notable limitations. An umbrella review methodology was used for this review, therefore potentially excluding findings of primary studies that were not included in the analyzed reviews. Non-English papers were excluded, possibly excluding findings from non-English sources and countries and introducing a bias as to how quality of mHealth in cancer survivors is reported. The search term mHealth was used, and the decision was made to avoid eHealth and digital health because of the overwhelming number of reviews with nonmobile-related interventions in the search results. However, this introduces the possibility of potentially excluding reviews pertaining to mHealth published under terms such as eHealth or digital health. Some findings were also derived from the Discussion section of reviews where opinions of the authors of the reviews may contribute to a bias.

Our findings can assist mHealth technology and software developers ensure that a focus is given to additional quality criteria in creating and deploying interventions in cancer survivorship care. In addition, clinicians selecting mHealth interventions for cancer survivors to assist in their care can be more aware of the quality criteria that are important to consider. Future research may guide the development of a comprehensive quality framework to allow for consistent evaluation of mHealth technologies among cancer survivors and to guide the development of these interventions.

In conclusion, although mHealth offers great opportunities to revolutionize cancer care delivery among cancer survivors, there is significant variability in how quality is reported in mHealth interventions. Usability, privacy, and security were found to be important quality domains that are unique to mHealth interventions in addition to the existing IOM-defined quality domains.

APPENDIX

Search Strategy

(mhealth OR "mobile health" OR app? OR "mobile app*" OR "smart watch" OR smartwatch OR "smart device" OR smartphone OR "smart phone" OR fitbit OR "fitness tracker" OR tablet OR ipad OR iphone OR "text messag*" OR "short messaging service" OR sms)

AND

(quality$ OR effectiveness OR safety OR efficacy OR outcome OR evaluation OR measure OR framework OR benefit? OR strength OR limitation OR pro? OR con? OR disadvantage? OR advantage?)

AND

(cancer OR tumo?r OR neoplasm)

AND NOT p?ediatric

AUTHOR CONTRIBUTIONS

Conception and design: Timothy Tune, Patricia A.H. Williams, Bogda Koczwara

Administrative support: Bogda Koczwara

Collection and assembly of data: Timothy Tune, Bogda Koczwara

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).