Developing and validating a content quality evaluation tool for cancer mobile applications

Abstract

Introduction

Cancer remains a leading cause of death, underscoring the importance of reliable health apps. This study aims to design and validate a content quality evaluation tool for cancer mobile applications ensuring they provide standard services and features to users.

Method

This descriptive-analytical study was conducted in steps including identifying mobile application contents, designing and validating evaluation tool. Studies published until February 19, 2023, in PubMed, Web of Science, Scopus, and Scientific Information Database (SID) databases, as well as mobile applications in Myket, Bazaar, and Google Play, were searched. Data extraction and expert panel validation followed, leading to four expert sessions to design the tool. Nineteen oncologists, health information management and medical informatics specialties were included in the study as a research community. The tool’s face validity, content validity and reliability were assessed using CVR (Content Validity Ratio), CVI (Content Validity Index), and Cronbach’s alpha respectively.

Results

Out of 242 initial search results in databases and 125 mobile applications in app stores, 25 articles and 20 mobile applications included based on the search limits. The designed tools consist of 8 main themes (prevention, diagnosis, treatment, follow-up, education, communication, requests/order and other) with 43 question items. All items received scores above 0.79 in CVI and 0.42 in CVR, leading to their approval and the reliability of the tool was accepted with a score of 0.967.

Conclusion

Evaluating cancer mobile applications can be very helpful in developing useful and supportive mobile apps, and assist health policymakers in identifying and recommending high-quality cancer-related apps. This will enable patients to choose the most suitable apps, enhancing the quality of cancer care.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-024-13112-w.

Introduction

Cancer is a disorder characterized by uncontrolled cell proliferation and differentiation that can occur in any tissue of the body and at any age. It can cause severe illness and, ultimately, death by attacking healthy tissues in the body. Despite significant advances in medical science, cancer remains one of the most important diseases of the present century. It is considered the third leading cause of death after cardiovascular diseases and accidents [1–3]. Cancer is one of the most common chronic diseases and accounting for nearly 10 million deaths in 2020, or nearly one in six deaths [4]. Every year, more than 7 million people worldwide lose their lives due to cancer [2, 5]. Currently, cancer is considered one of the major healthcare issues globally and in our country, impacting the patient’s occupation, socio-economic status, and family life, leading to disruptions and devastation in their lives. These impacts particularly affect various aspects of the patient’s quality of life, including mental, emotional, social, economic, and sexual well-being [6]. Limited access to healthcare facilities, along with the rising prevalence of co-morbidities and complications, underscores the growing importance of mobile applications and telemedicine in delivering healthcare services. For this purpose, health and wellness electronic applications and tools (such as websites, web-based applications, and mobile apps) have great potential for empowering self-management of chronic conditions like cancer [7–10]. In addition, mobile applications are a suitable option for those who prefer or are required to use information and communication technologies or are in geographically or physically constrained conditions [11, 12]. Mobile applications and electronic tools have the potential to provide low-cost intervention and preventive services, specifically designed for mental health disorders such as anxiety, depression, and widespread health issues. According to available statistics until the end of 2019, 204 billion mobile applications have been downloaded and installed in various countries worldwide, enabling the majority of the population to easily access these apps. The successful experience of using digital health and mobile applications in identifying and combating common international diseases such as Ebola and Severe Acute Respiratory Syndrome (SARS) has demonstrated that these mobile applications are essential tools for improving health and providing services to the public [13–15].

Given the high prevalence of cancer in middle-aged individuals and the significant financial burden on patients and healthcare systems, the use of information technology, particularly mobile applications, is crucial for providing medical information, evaluating, treating, and monitoring patients [16]. Various studies describe the emergence of smartphone applications related to cancer [16–18], including those designed for breast cancer, prostate cancer, skin monitoring, melanoma diagnosis, and cancer patient medication adherence [17–19].

Studies indicate that mobile applications in the field of cancer do not meet al.l the needs of patients [19, 20]. Moreover, many mobile applications in this field have not defined key phrases for mobile users. Collado-Borrell et al. [21] examined 166 cancer-related mobile applications for patients and found that individual monitoring tools like daily journals or ways to track unwanted effects and treatments are limited. According to their study, medical mobile applications used for cancer patients were mostly educational, diagnostic, and preventive. However, many mobile applications faced issues such as insufficient validation and data update delays. They also recommend that strict regulations should be implemented to prevent mobile applications from becoming a threat rather than a tool for the general public [20, 21].

Some evaluation tools for mobile applications like THESIS and Mobile Application Rating Scale (MARS) are designed to generally evaluate health-related applications and there is significant heterogeneity among these evaluation tools in evaluating criteria [22, 23]. THESIS evaluation tool includes seven evaluation criteria for mobile applications such as transparency, health content, technical content, privacy and security, usability, and mind presence, and the MARS evaluation tool includes seven evaluation criteria for mobile applications such as design, information or content, usability, performance, ethical issues, security and privacy, and perceived value by the user. In various studies, the categorization of evaluation dimensions for mobile applications varies based on their usage, including interaction description, engagement with the target patient population, user rankings and reviews, and application updates [24], design, system reliability, personal security and accuracy [25], interaction, performance, interface design, access, educational content quality, and content design quality [26]. Therefore, the presented mobile applications should undergo content evaluation and validation to ensure their features, and content quality, and provide standard services to individuals. With the increasing adoption of mobile applications and electronic tools that may be used for management purposes and as a means to complement clinical services, all users need to have access to standard features and content [27]. The lack of accuracy and suitability evaluation of health-related mobile applications and electronic tools can endanger the health and safety of users. Moreover, considering the potential risks associated with the use of mobile applications, there is an increasing interest in evaluating the quality and content of mobile applications and digital tools [28].

There are numerous mobile applications in the field of cancer that can be utilized in various areas including screening, triage, disease diagnosis, follow-up, patient education, treatment, diet, exercise, and self-care [11–15]. These applications are very useful and helpful in reminding appointment schedules, treatment follow-up, providing proper disease information, and even contacting healthcare professionals, ultimately reducing costs [29], and can be a useful and effective tool for disease management [13–15]. Mobile applications in this area can be very helpful, but it should be noted that these applications need to be validated to effectively meet the needs of cancer patients [17–19]. Based on the reviewed literature, mobile tools are primarily designed to emphasize the technical features and usability of the tools, signaling the need for the design and validation of tools for content evaluation. Since several mobile applications related to cancer have been designed and developed, the lack of quality and accurate evaluation of content can pose various risks in the health field for individuals and users. Additionally, considering that there is no suitable tool for evaluating these mobile applications from a content perspective [17–19], this research attempts to systematically design a tool for assessing and evaluating cancer-related mobile applications using the results of literature reviews and expert interviews, followed by validation.

Materiel and methods

This study is a descriptive-analytical study with applicable results, conducted in 2022–2023. It was designed in steps including identifying cancer mobile application contents, designing and validating evaluation tool.

Literature and mobile applications stores search

The first step was pertained to the review of studies and mobile applications stores to identify cancer mobile application contents. The review was done until February 19, 2023 with applying research inclusions (original articles and conferences with English and Persian languages). Additionally, the inclusion criteria for mobile applications were: available in Persian and English, executable on the Android operating system, accessible in Iran, related to cancer, and sized between 20 and 30 megabytes. Table 1 demonstrates the research keywords and search strategy in PubMed, ScienceDirect, Web of Science databases, and Scientific Information Database (SID). Moreover, for Persian and English mobile applications, Google Play was used, and for Persian applications, Myket and Bazaar were used with the cancer and neoplasm keywords. Figures 1 and 2 depict the search and screening process for studies and mobile apps.

Table 1.

Search strategy in databases

Database	Search strategy
PubMed	(“Validation“[Title/Abstract] OR “Evaluation“[Title/Abstract]) AND (“medical smartphone applications“[Title/Abstract] OR “mobile health applications“[Title/Abstract]) AND (“cancer“[Title/Abstract] OR “Neoplasms“[Title/Abstract]) (TITLE ABSTARCT)
Science Direct	(Validation OR Evaluation) AND (medical smartphone applications OR Mobile health applications) AND (cancer OR Neoplasms) (TITLE ABSTARCT)
WOS	((TS= (Validation OR Evaluation)) AND TS= (medical smartphone applications OR Mobile health applications)) AND TS= (cancer OR Neoplasms) (Title, abstract, author keywords)
SID	(Mobile application OR Cancer OR Evaluation)

Fig. 1.

PRISMA flow diagram of screening process and search results

Fig. 2.

Diagram of mobile application selection process

PRISMA flow diagram of screening process and search results.

Content analysis

Content analysis entails the collection, organization, and interpretation of data to discern its meaning. In this phase, the research team held two sessions, each lasting 1 h and 30 min, to review and structure the extracted information related to studies and mobile applications. Initially, they identified the primary contents of the mobile applications and categorized them into eight themes. By further examining these contents, they subsequently determined sub-themes, categories, and sub-categories within each theme. In case of any discrepancies, the agreement was reached through discussing between researchers to compare findings and resolve disagreements.

Designing evaluation tool items

For designing the initial draft of the content quality evaluation tool, a panel of experts consisting of physicians, health information management and medical informatics specialists held 4 face-to-face sessions each lasting 1 h. Evaluation questions were designed based on the specified content from the previous step. Each question was individually evaluated by experts, and questions with differing opinions were discussed collectively. After collaboration and discussion, the questions approved by the experts were ultimately designed into an initial draft of the content evaluation tool in the form of a checklist. The designed checklist was consisting of 43 questions with Likert scale.

Validating evaluation tool

Research participants

In the validation phase, the research community including 19 oncologists, health information management, and medical informatics specialists from Shiraz University of Medical Sciences, Iran University of Medical Sciences, and Jahrom University of Medical Sciences were selected through convenience sampling method. The inclusion criteria encompass experts in medicine, particularly cancer, who have a track record of research in e-health or mobile health. Additionally, health information management specialists and medical informatics experts with a minimum of 3 years’ experience as university faculty members are considered.

Evaluation tool validity

At first, the face validity of the tool was confirmed by 5 health information management specialists. Subsequently, the validity and reliability of the designed tool were calculated. In this phase, the evaluation tool checklists were distributed among participants to rate each item of the evaluation tool, based on content validity indices (relevancy, clarity, and simplicity) and also content validity ratio using a Likert scale. To calculate CVR, the following formula based on Lawshe table was used:

In this formula, N represents the total number of experts, and ne denotes the number of experts who have selected the necessary option. All evaluation tool’s items received scores above 0.79 in CVI and 0.42 in CVR, leading to their approval [30]. Finally, the evaluation tool’s reliability was evaluated using Cronbach’s alpha for both the entire tool and each domain. The quantitative results were analyzed using SPSS version 22 software. SPSS software, now known as IBM SPSS Statistics, was originally developed by SPSS Inc., which was based in Chicago, Illinois. Table 2. shows the timetable for the research methodology.

Table 2.

Methodology timetable

Methodology phase	Time
Literature and Mobile Applications Stores Search	6 months
Content analysis	1 month
Designing Evaluation Tool Items	2 months
Validating Evaluation Tool	3 months

Results

In the initial search for studies, by searching the mentioned databases, 242 candidate papers were selected. After deleting duplicated studies, 225 studies remained. Then, based on using the exclusion criteria, 89 articles were investigated meticulously to choose relevant studies. Eventually, 35 studies, including 32 English studies and 3 Persian studies, were applied to obtain an answer to the research questions. Figure 1 represent the search results in a PRISMA flow diagram.

In the initial search for mobile applications, 125 applications (Bazar:65, Myket:10, Google Play = 50) After removing duplicates, 120 applications remained. In the next stage, after applying search limits, 20 applications were selected. Figure 2 shows the diagram of selecting mobile applications.

Information about the content of cancer mobile applications, was extract based on the researcher-made information extraction form of included studies [31–55] and mobile applications (Onco cancer care, Fight colorectal cancer, Onco power, Cancer. Net mobile, Outcomes 4me cancer care, Cancer therapy advisor, Chemowave Health Diary, Foto check skin cancer spots ID, War on cancer join toda, Foto check skin cancer spots ID, Bscan-Breast cancer screening, Vine health: cancer companion, Chemowave Health Diary, War on cancer join today, Breast cancer club, my cancer coach, Belong beating cancer together, Zen onco cancer care apps and, Persian Cancer Bank and Colon cancer applications. Table 3 shows the extracted contents including main themes, subthemes, categories, and sub-categories. According to the determined categories and subcategories, evaluation tool items were designed based on experts’ opinions (Table 4).

Table 3.

Extracted content from studies and applications

Theme	Sub-theme	Category	Subcategory	Freq.
Prevention	Receive information and promote awareness	Disease symptoms		5
		Survive		1
		Healthy diet		6
	Providing preventive strategies for cancer	Symptom management		8
	Determining the risk of infection	Analysis of people’s health records		1
Diagnosis	Calculation of laboratory indicators			1
	Record vital signs			1
	Record the symptoms of the disease	Recording the graphic image of symptoms over time (skin disease)		3
		Rating of symptom severity		2
		ability to export/send data		6
	Analysis based on each patient’s symptoms			2
	Records/History	Registration and review of medical records		2
		Registration and review of family records		1
		Treatment history	Drug therapy	1
			Chemotherapy	1
			Radiotherapy	1
	Periodic self-examination			2
Treatment	Monitoring side effects of treatments (chemotherapy)			6
Follow up	Quality of life	Nutrition	Healthy eating education	4
			Monitor nutrition/ patient energy	2
		physical activity	Physical activity monitoring	5
			Weight management	1
			Teaching related sports	5
	Self-care	Drug therapy		6
		Management of symptoms		7
		Pain self-management	Providing specific training for the use of painkillers for patients	2
			Principles of pain psychology	1
			Health behaviors and pain	1
			Pain coping skills training	3
	Follow up Treatment	Reminder and warning	Appointment reminder	5
			Warning when taking medicine	3
			Remember to record the symptoms that the patient experiences	1
	Mental Health	Mental health monitoring		1
		Provide psychological emotional support	Sending motivational content	4
Education	Public Education	Providing standard articles and guides		9
		Education based on multimedia		1
	Dedicated Training	Dedicated training based on receiving patients’ symptoms		1
Communication	Communication with the Therapist/ Physician	Questions and answers with the doctor		9
		Conduct counseling		5
	Communication with the other patients	Sharing common experiences		7
Request /Order	Online purchase of medicinal and non-medicinal items through the shopping cart in applications			3
Other	Financial Support			3
	Advertising	In-app online shopping ad		1
		Repair of body parts (in patients with breast cancer)		1
		Advertising of future events of foundations and charitable institutions (financial support)		1
		Advertising the performance of an individual surgeon or a group of surgeons (surgery center)		1

Table 4.

Content evaluation items for cancer-related mobile applications

Themes	Question Items
Prevention	1- Can the application provide general information about the symptoms of cancer?
	2- Can the application provide information and awareness about disease survival?
	3- Can the application provide information and awareness about healthy eating?
	4- Does the application provide preventive strategies for cancer in the field of symptom management (to prevent the progression of the disease)?
	5- Does the application program in the field of cancer prevention provide the possibility of determining the risk of contracting the disease through the analysis of health records?
Diagnosis	1- Does the application have the possibility to calculate laboratory indicators related to the patient?
	2- Does the application have the possibility to record vital signs (pulse, breathing, blood pressure, temperature)?
	3- Does the application have the possibility to record the symptoms of the disease in the form of a graphic image?
	4- Does the application have the possibility of ranking the symptoms of the disease based on the severity of the symptoms?
	5- Does the application have the possibility to send data related to the registered symptoms of the disease?
	6- Does the application have the ability to analyze based on symptoms?
	7- Does the application have the possibility to register and check disease records?
	8- Does the application have the possibility to register and check family records?
	9- Does the application have the possibility of recording the patient’s treatment history (drug therapy, chemotherapy, radiotherapy)?
	10- Does the program have the possibility to record periodical self-tests?
Treatment	1- Does the application have the possibility of monitoring the side effects of chemotherapy treatment?
	2- Does the application have the possibility of monitoring the side effects of drug treatment?
	3- Does the application have the possibility of monitoring side effects of radiotherapy treatment?
Follow up	1- Does the application allow improving the quality of life through the provision of healthy change training?
	2- Does the application have the possibility of improving the quality of life through monitoring the nutrition and energy of patients?
	3- Can the application program improve the quality of life through physical activity?
	4- Can the application improve the quality of life through weight management?
	5- Does the application have the possibility of improving the quality of life through teaching sports related to patients?
	6- Does the application in the field of self-care provide the possibility of managing the symptoms of the disease?
	7- Does the application in the field of self-care provide the possibility of drug therapy?
	8- Does the application in the field of self-care provide the possibility of providing special training for the use of painkillers for self-management of pain?
	9- Does the application in the field of self-care provide self-management of pain through the principles of pain psychology (the effect of stress and emotions on pain)?
	10- Does the application provide the possibility of self-management of pain through sleep management?
	11-Does the application program in the field of self-care include pain self-management through the training of coping skills?
	12- Does the application in the field of treatment follow-up have the ability to remind appointments?
	13- Does the application program in the field of treatment follow-up have the ability to warn when to take medicine?
	14- Does the application program in the field of treatment follow-up have the ability to remember the symptoms experienced by the patient?
	15- Does it have a mental health monitoring application?
	16- Does the application have the possibility of providing psychological emotional support through sending motivational content in the form of short messages?
Education	1- Can the application provide articles and standard guides related to cancer?
	2- Does the application have the possibility of providing specific training based on the patient’s symptoms?
	3- Does the application have the possibility to provide training based on multimedia (movies, audio files, images, gifs, links, etc.)?
Relationship	1- Does the application have the possibility of communicating with the therapist/doctor for consultation?
	2- Does the application have the possibility to communicate with the therapist/doctor for questions and answers?
	3- Does the application allow communication with other patients in order to share common experiences?
Requests and orders	1- Does the application allow the purchase of pharmaceutical items online?
	2- Does the application allow the purchase of non-medical items online?
Advertising	1- Does the application have the ability to advertise for in-app purchases?

According to Table 3, the main themes of content of cancer mobile applications include prevention, diagnosis, treatment, follow-up, education, communication, requests and orders, and advertising. The sub-dimensions include receiving information and awareness, presenting preventive solutions, recording symptoms, medical history, clinical self-examination, symptom monitoring, quality of life, self-care, treatment follow-up, mental health, education, communication, requests and orders, and advertising; with corresponding items for each dimension placed in their respective section. After completing the final checklist of the evaluation tool by 19 experts in health information management, medical informatics, and Oncologists, the CVI with 4-point Likert scale and CVR with 3-point Likert scale were calculated. Table 5 shows information related to the participating experts.

Table 5.

Participants Demographic Information

Participants	frequency	frequency percentage
Health information management specialist	8	42.1%
Medical informatics specialist	2	10.5%
Oncologists	9	47.3%
Total	19	100%
Female	14	73.6%
Man	5	26.3%

By calculating the CVI and CVR using SPSS software version 22 and confirming all questions, the validity of the tool was approved. According to Table 6, all evaluation tool’s items received scores above 0.79 in CVI, and 0.42 in CVR, leading to their approval. Table 6 shows the results of the content validity. Finally, the Cronbach’s alpha for the entire tool was calculated as 0.967, which is higher than 0.50 and is considered acceptable.

Table 6.

Results of Content Validity for Tools Items

Objects	CVI			CVR
	Simplicity	Clarity	Relevancy
1- It is possible to provide general information about the symptoms of cancer.	0.94	0.94	1	1
2- It is possible to provide information and awareness about cancer survival.	0.89	0.89	0.89	0.47
3- It is possible to provide information and awareness about healthy eating.	0.94	0.89	0.94	1
4- It provides cancer preventive solutions in the field of symptom management (to prevent the progression of the disease).	1	0.94	1	1
5- In the field of cancer prevention, it provides the possibility of determining the risk of contracting the disease through the analysis of health records.	1	1	0.94	0.47
6- It is possible to register and check the records of the disease.	1	1	1	0.78
7- It is possible to register and check family records.	0.94	0.94	0.94	0.78
8- It is possible to record the patient’s treatment history (drug therapy, chemotherapy, radiotherapy).	1	1	1	0.89
9- It is possible to rank the symptoms of the disease based on the severity of the symptoms.	0.94	0.94	0.94	0.89
10- It is possible to send data related to registered symptoms of the disease.	0.94	0.94	0.94	0.78
11- It has the ability to analyze based on symptoms.	0.84	0.84	0.94	0.68
12- It is possible to record laboratory indicators related to the patient	0.94	0.94	0.94	0.89
13 It is possible to record vital signs (pulse, breathing, blood pressure, temperature).	0.94	0.94	0.94	0.68
14- It is possible to record the symptoms of the disease in the form of a graphic image.	0.94	0.94	0.89	0.47
15- It is possible to register periodical self-tests.	0.89	0.89	0.89	0.57
16- It is possible to monitor the side effects of treatment with different treatment methods (chemotherapy, drug therapy and radiotherapy).	0.94	0.94	1	0.68
17- It allows to improve the quality of life through the provision of healthy nutrition education.	1	1	1	0.57
18- It is possible to improve the quality of life by controlling the nutrition of patients.	0.94	0.94	0.94	0.47
19- It is possible to improve the quality of life through the control of physical activity.	1	1	1	0.57
20- It is possible to improve the quality of life through weight management.	1	1	1	0.47
21- It is possible to improve the quality of life through teaching sports related to patients.	1	1	1	0.78
22- In the field of self-care, it provides the possibility of improving the symptoms of the disease under the supervision of a physician.	0.89	0.84	084	0.78
23- In the field of self-care, it provides the possibility of drug therapy under the supervision of a doctor.	0.89	0.89	0.89	0.78
24- In the field of self-care, it provides the possibility of providing special training under the supervision of a doctor for the use of painkillers for self-management of pain.	0.89	0.84	0.89	0.57
25- In the field of self-care, it provides pain self-management through the principles of pain psychology (the effect of stress and emotions on pain).	0.94	0.89	0.089	0.68
26- Provides the possibility of self-management of pain through offering a healthy lifestyle such as sleep management.	0.94	0.94	1	0.68
27- In the field of self-care, self-management of pain includes the training of coping skills.	0.94	0.94	1	0.78
28- Monitors mental health.	1	1	1	0.68
29- It is possible to provide psychological emotional support through sending motivational content in the form of short messages.	0.89	0.89	0.84	0.68
30- In the field of treatment follow-up, it has the ability to remind appointments.	0.94	0.94	0.94	0.78
31- In the field of treatment follow-up, it has the ability to warn when to take medicine.	1	1	1	0.57
32- In the field of treatment follow-up, it has the ability to remember the symptoms experienced by the patient.	1	0.94	1	0.57
33- It is possible to present articles and standard guides related to cancer.	1	1	1	0.57
34- It is possible to provide specific training based on the patient’s symptoms.	0.94	0.94	0.94	0.47
35- It is possible to provide education based on multimedia (movies, audio files, images, gifs, links, etc.).	0.94	0.94	0.94	0.57
36- It is possible to communicate with the therapist/doctor for consultation.	1	1	1	0.89
37- It is possible to communicate with the therapist/doctor for questions and answers.	0.94	0.94	0.94	0.89
38- It is possible to communicate with other patients in order to share common experiences.	0.89	0.89	0.89	0.68
39- It is possible to buy medicinal items online.	0.94	0.89	0.94	0.47
40- It is possible to buy non-medicinal items online.	0.84	0.84	0.84	0.47
41- It has the ability to advertise for in-app purchases.	0.84	0.84	0.84	0.47
42- Advertisements of future events of foundations and charitable institutions.	0.89	0.84	0.89	0.47
43- Advertising involves the performance of a surgeon or surgery center.	0.89	0.84	0.84	0.47

Discussion

Cancer is one of the common diseases that has affected many people worldwide. Cancer is a disease that, if diagnosed late, can spread throughout the entire body. However, some types of cancer are completely treatable if diagnosed in time [56]. Mobile applications can be useful and effective in various stages of the disease such as screening, triage, early detection of the disease, follow-up, self-care, and providing information and awareness in various disease-related areas [13–15]. They have been successful in the prevention and management of cancer treatment, particularly in areas such as education, pain management, and self-care for patients undergoing chemotherapy. Despite the importance of this issue and the widespread use of health mobile applications among cancer patients, the quality of the content used in these applications, which can significantly impact the quality of life of these patients, has been neglected. It appears that no studies have been conducted to evaluate the content of cancer-related applications, highlighting a key strength of this study.

In this study, the content evaluation tool for mobile applications in the cancer domain was designed around eight main themes: prevention, diagnosis, treatment, follow-up, education, communication, requests/orders, and other (advertising and financial support). The tool includes 43 question items. In this regard, Nouri et al., in their study categorized the criteria for assessing the quality of health-related mobile programs into 7 main categories: design, information or content, usability, performance, ethical issues, security and privacy, and perceived value by the user, with 37 subcategories [22].

Also, Cai et al. [47].identified four main themes (1) Patient and healthcare provider participation in design and development stages; (2) Patient preferences; (3) Patient personalities; and (4) Motivation for using mobile health apps for breast cancer patients, each encompassing multiple sub-themes including functional exercise, lifestyle enhancement, symptom management, and informational and emotional support which, these items were considered in our study.

Additionally, some mobile application evaluation tools such as MARS, THESIS and MAUQ, have been developed to assess health-related mobile applications in general, primarily focusing on the usability aspects such as engagement, functionality and aesthetic. For instance, Levine et al. designed a ranking tool for mobile applications called THESIS, which can enhance informed selection of mobile applications. The THESIS evaluation tool includes seven evaluation criteria for mobile applications such as transparency, health content, technical content, privacy and security, usability, and mind presence [23]. Unlike the current study that focused on content transparency, privacy, security, usability, and mind presence, we specially emphasized the contents of mobile applications in our study.

In this study, by examining mobile applications and reviewing previous studies in the field of cancer and taking into account expert opinions in all research steps, a tool is presented for evaluating the content quality of cancer-related mobile applications. Some studies in mobile applications evaluation, have used MARS. It is an objective and reliable tool incorporating the following domains: Engagement, Functionality, Aesthetics, Information, and Quality. Lu et al. in their study used mobile health apps for monitoring patient-reported outcomes remotely and enhancing communication between patients and healthcare providers. They applied MARS for evaluating apps [53]. Additionally, in a study by Kanjak et al., three assessment tools were employed to evaluate the quality of mobile health apps for cervical cancer. Among these tools, the MARS evaluation tool, specifically designed for analyzing mobile app quality, shared only the content dimension with the present study. The Systematic Evaluation Scorecard for Mobile Apps, comprising objective components (comprehensiveness, cost, in-app purchasing, literature, app acquisition, connection, advertising, text search field, device compatibility, and other components) and subjective components (ease of orientation, mental presentation), also aligned with the present study only in the dimensions of advertising and text search field [36]. Moreover, the App Rating Using Specific Statements encompassed domains such as previous visits, app performance, treatment-related toxicity, new unrelated health issues, and visits and meetings, correlating solely with visits and meetings in the present study. Furthermore, the study used the System Usability Scale (SUS) to assess the usability of the apps and their effectiveness in following up clinical trials [36], which we have not addressed in our study.

Dykes et al., used the Impact of Health Information Technology (I-HIT) Scale designed to provide a tool for measuring nurses’ perceptions regarding how Health Information Technology (HIT) impacts interdisciplinary communication, workflow patterns, and nurse satisfaction with HIT programs and tools currently available in hospitals. The I-HIT scale should be implemented for nurses working in direct care positions to offer valuable insights into the effectiveness of HIT programs and tools available in acute care settings [57]. Unlike the current study, which includes all users of cancer-related apps such as patients, healthcare providers, and researchers, I-HIT is solely used by nurses and researchers and serves as a tool for psychometric evaluation of all health information technology tools and programs, not exclusively focused on a specific domain like cancer. Also, some studies such as Moorthy et al. translated the Mobile Health Application Usability Questionnaire (MAUQ) into German and determining its internal consistency, reliability, and construct validity. MAUQ is specifically designed to assess the usability of mobile health applications. It covers various dimensions such as ease of use, usefulness, ease of learning, and satisfaction, aligning with MARS in these aspects [58]. Our study, however, did not address usability and related issues. Instead, this study focuses on designing content evaluation tool items specifically for cancer-related applications, which can be used alongside other design tools for a comprehensive evaluation of these applications.

Limitation

This study acknowledges limitations in the evaluation tool items. Specifically, even if a mobile app meets all the criteria based on these items, users might still encounter misinformation, and the accuracy of the content cannot be definitively verified. Therefore, further studies are needed to investigate the effectiveness of the content quality evaluation tool, determine high-quality content items, and establish thresholds for item ratings, enabling users to independently assess the content quality of mobile apps.

Conclusion

Cancer progresses quietly, disrupting normal life. While mobile apps offer valuable educational and diagnostic support, their abundance makes it difficult to choose high-quality options. Ranking tools that evaluate app features and content quality assist users in making efficient choices. Accurate evaluation tools are essential for user health and safety, and developing reliable tools can improve the design of supportive cancer apps. If developers of cancer-related applications utilize the results of this study, they can establish better standards for apps content. Additionally, if health policymakers identify high-quality cancer-related apps, they can recommend them to their community. Ultimately, if patients gain a better understanding, they can select the most suitable app based on their needs, contributing to improved quality of care for cancer patients.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Abbreviations

MARS

Mobile Application Rating Scale

SUS

System Usability Scale

MAUQ

Mobile Health Application Usability Questionnaire

SARS

Severe Acute Respiratory Syndrome

CVR

Content Validity Ratio

CVI

Content Validity Index

I-HIT

Health Information Technology Scale

Author contributions

A.B. and N.A. have designed the study. AB, L.E., R.R. and N.A. contributed to data gathering, data interpretation, design and validation of evaluation tool. A.B., N.A. and B. A. contributed to article drafting. All authors have contributed to finalizing the manuscript. A.B. has also supervised the whole.

Funding

Not applicable.

Data availability

All data generated or analyzed during this study are included in this article and its supplementary information files.

Declarations

Ethics approval and consent to participate

This study was approved by the Shiraz University of Medical Sciences Ethics Committee (ID: IR.SUMS.NUMIMG.REC.1401.113). All methods were carried out in accordance with relevant guidelines and regulations under ethics approval and informed consent to participate. In tool validation phase, all participants indicated their consent to take part in this study by completing the informed consent form.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.