Developing a Theoretical Framework for Evaluating the Quality of mHealth Apps for Adolescent Users: A Systematic Review

Abstract

OBJECTIVE

To assess adolescents' preferred mobile app features and to propose a framework for evaluating health-related mobile apps for adolescents.

METHODS

PubMed, CINAHL, PsycINFO, ERIC, HealthIT.gov, and ClinicalTrials.gov were systematically searched in August 2017. Studies pertaining to app development, feasibility, or usability that reported preferred app features and rating criteria on mHealth (mobile health) apps intended for adolescents were included. Quality assessment was performed using the Mixed Methods Appraisal Tool. Qualitative synthesis was performed to develop themes reflecting best practices for evaluating the quality of mHealth apps for adolescents. Using a grounded theory approach, we constructed a theoretical framework of rating criteria that can be used to inform development of an evaluation tool for mHealth apps targeted to adolescents.

RESULTS

Thirteen articles were included. Most commonly preferred features include ability to track test results or self-management progress, connect to social media, and gain points or prizes through app gamification. Common rating criteria include degree of app customizability, ease of use, visual appeal, and interactivity. Five emerging dimensions were used in the theoretical framework: Technical Quality; Engagement; Support System; Autonomy; and Safety, Privacy, and Trust.

CONCLUSIONS

We found that adolescents prefer mHealth apps that are customizable, offer peer support through social media, sustain engagement via gamification, and support the ability to visualize health trends via simplified graphs. Findings may help in the development of mHealth apps that are preferred by adolescents, as well as the development of a quality evaluation tool for mHealth apps targeted to this population.

Introduction

In the United States, over 80% of the nation's annual healthcare expenditure is spent on managing chronic health conditions.¹ This presents an ongoing public health issue for both adults and adolescents. Indeed, 1 in 5 US adolescents between the ages of 12 and 17 years lives with a chronic health condition such as asthma, diabetes, or depression,² and 1 in 3 lives with 3 or more chronic conditions.²

Self-management of chronic conditions often entails continuous activities such as consistent medication taking, symptom monitoring, and other condition-specific treatment activities. These tasks may be more challenging for adolescents compared with adults. Adolescents may be especially prone to barriers resulting from lack of social support, understanding about a prescribed treatment regimen, disease state knowledge, and self-efficacy for self-management behaviors.^3–5 Further, adolescents living with chronic conditions consider friendships, support from family and healthcare professionals, and positive school experiences as key factors in successfully managing their health conditions.⁶ To promote effective and sustainable self-management behaviors in adolescents, there is a need for targeted solutions that consider the unique needs and challenges faced by adolescents living with chronic conditions.

One viable option for improving self-management across a range of chronic conditions in adolescents is to use mobile health (mHealth) application (app) technology for smartphones and tablets. Use of mobile apps may improve adolescent engagement with health behaviors.⁷ Indeed, interventions leveraging mobile apps or text-messaging are effective in improving a broad range of health behaviors in adolescents, including diabetes self-management,⁸ asthma self-management,⁹ mental health,¹⁰ and condition-specific medication adherence.¹¹ Given that 73% of all US teenagers between 13 and 17 years of age own or have access to a smartphone and the familiarity of this age group with using mobile technology,¹² public health programs and interventions disseminated via mobile apps may have increased “reach” within the adolescent population compared with paper-based or face-to-face programs.

Despite the benefits of using mobile apps for engaging adolescents in self-management of health behaviors, there is no published tool to guide the development and evaluation of the quality of mHealth apps designed specifically for adolescent users. Evaluation tools exist for rating the quality of mHealth apps, such as the Mobile App Rating Scale (MARS),¹³ but are not designed specifically to rate apps intended for the unique needs of adolescent end users. Developing a rating tool designed specifically for evaluation of mHealth apps for adolescent users is a critical first step toward widespread design and dissemination of high-quality, evidence-based apps for improving self-management behaviors, and ultimately patient outcomes in this vulnerable population.

The need for such a tool is high. However, little is known about adolescent users' preferences for features in mobile apps. Additionally, relevant evaluation criteria for inclusion in a rating tool that meets adolescents' preferences have not been defined. Thus, the purpose of this systematic review is to assess adolescents' preferred mobile app features and to propose a framework for developing health-related mobile apps for adolescents. A grounded theory approach was used to evaluate the reviewed articles and create a theoretical framework of rating criteria that can be used to inform development of a quality-rating tool for development, evaluation, and comparison of mHealth apps targeted to adolescent users. Developing an adolescent mHealth app to improve medication adherence for chronic conditions is presented as an illustrative example of how the proposed framework may be used.

Materials and Methods

Search Strategy. PubMed, CINAHL, PsycINFO, ERIC, HealthIT.gov, and ClinicalTrials.gov were systematically searched in August 2017 for articles pertaining to mHealth app design and evaluation in an adolescent population. Citations in pertinent articles were also hand-searched to identify relevant articles that may not be indexed in the included databases. Search terms included (“mobile app” OR app OR mobile) AND (criteria OR scale OR rating OR checklist OR design OR develop) AND (child OR adolescent OR teen) AND (health OR medical OR drug OR mHealth).

Study Selection. Articles underwent initial title and abstract screening, followed by full-text screening. Each of these screenings was performed independently by 2 of the authors (RJ and NH), with discrepancies in retained articles resolved through discussion and consensus. English-language articles were included for review if they met the following criteria: 1) published between 2007 and 2017; 2) peer-reviewed cross-sectional analytic studies, quantitative descriptive studies, qualitative interviews or focus groups, or other app development, feasibility, or usability assessments (impact studies, study protocols, ecologic momentary assessments, and review articles were excluded); 3) intended users of apps are between the ages of 12 and 18 years (adolescents) and are normally developing (studies involving adolescents with neurodevelopmental disorders such as autism were excluded); 4) inclusion of preferred app features and rating criteria for the app; 5) inclusion of smartphone or tablet mobile app running on any platform (web-based, text, and personal digital assistant (PDA) applications were excluded); and 6) ability to extract results (rating and design criteria).

Data Extraction and Synthesis. Data were extracted by RJ and NH during full-text screening, using a standardized form. This standardized form contains fields for study location, general description (including study design, sample size, and participant characteristics), objective, measures, and results, as well as app features and rating criteria reported within each study.

Quality assessment was independently performed by RJ and NH using the Mixed Methods Appraisal Tool Version 2011,¹⁴ simultaneous with data extraction. The Mixed Methods Appraisal Tool contains 5 study design categories: qualitative, quantitative randomized controlled, quantitative non-randomized, quantitative descriptive, and mixed-methods. Each category contains 4 criteria (or 3 for mixed-methods), and scores range from 0% to 100%. For qualitative, quantitative randomized controlled, quantitative non-randomized, and quantitative descriptive studies, scores are calculated as a percentage of criteria met. For mixed-methods studies, scores are calculated as the lowest score from among the 3 relevant designs (quantitative, qualitative, and mixed methods). Interrater reliability of quality assessment scores was analyzed in the form of percent agreement.

Qualitative synthesis of the data was performed independently by RJ and NH to develop initial themes reflecting adolescents' preferred mHealth app features and best practices for evaluating the quality of health-related mobile apps for adolescents. Initial themes were discussed and modified by RJ and NH until consensus was reached, followed by review and verification by a third author (BF), to generate final themes. With a grounded theory approach,¹⁵ these final themes were used to construct a theoretical framework of rating criteria that can be used to inform development of a quality rating tool for design, evaluation, and comparison of mHealth apps targeted to adolescent users.

Results

PRISMA. A total of 13 articles met the inclusion/exclusion criteria for this review. Study selection is depicted in Figure 1. The most common reasons for exclusion were not including an mHealth app (e.g., use of a text-based tool); not explicitly listing rating criteria; not focusing on development, design, or rating of apps; and non-eligible study design.

Figure 1.

PRISMA diagram.

Characteristics and Quality of Included Studies. Table 1 contains a list of included studies and a summary of their design and general characteristics. All studies included qualitative design elements; 7 of the 13 studies used a purely qualitative design,^16–22 whereas the remaining 6 studies used a mixed-methods design including both qualitative and quantitative analyses.^8,23–27 The main objective of all retained studies was to design, develop, and/or evaluate mHealth apps for adolescent use. Feasibility, usability, and/or functionality testing were common purposes of app evaluation. Eight studies focused on self-management of asthma,^19–21 cancer,^23,26,27 or type 1 diabetes mellitus^8,16 in adolescents. The remaining 5 studies focused on adolescents' self-management of Sickle Cell Disease,²⁴ sexually transmitted infections,²² lupus,¹⁸ mental health conditions,¹⁷ and medication adherence in solid organ transplant.²⁵ All 13 studies directly included adolescents as participants (age, 12–18 years).^8,16–27 Four studies also supplemented adolescent or patient results with parent or non-professional caregiver results.^8,16,23,25

Table 1.

Characteristics of Included Studies

Article	Country	Study Objectives	Study Design	Sample Characteristics
Cafazzo8	Canada	To design, develop, and pilot an mHealth intervention for the management of type 1 diabetes mellitus in adolescents	Mixed methods (qualitative–cross-sectional analytic) Duration: 12 wk	Sample size: qualitative (adolescents, 6; parents, 6), Pre-post (20) Condition: diabetes Age: 12–16 yr
Carpenter20	USA	To evaluate the strengths of 2 existing apps and develop recommendations for a self-management app for adolescents	Qualitative Duration: 1 wk	Sample size: 20 Condition: asthma Age: 12–16 yr
Crosby24	USA	To describe internet access and use among adolescents and young adults with sickle cell disease, identify barriers to self-management, co-design an app, evaluate the feasibility and acceptability of the app	Mixed methods (qualitative–quantitative descriptive) Duration: not specified	Sample size: 70 Condition: sickle cell disease Age: 16–24 yr
Gkatzidou22	UK	To identify interface design requirements applicable to a sexual health app	Qualitative Duration: 45–60 min	Sample size: 49 Condition: sexually transmitted infections Age: 16–24 yr
Herschman18	Canada	To describe the development of a mobile app for adolescents with lupus	Qualitative Duration: 2 days	Sample size: 23 Condition: lupus Age: 16–59 yr
Holtz16	USA	To use patient-centered research methods to improve the design and functionality of an app for adolescents with type 1 diabetes	Qualitative Duration: adolescents = 72 min; parents = 107 min	Sample size: adolescents (5), parents (7) Condition: type 1 diabetes mellitus Age: adolescents = 10–14 yr; parents = 35–60 yr
Jibb27	Canada	To refine Pain Squad+, a pain self-management app, through usability testing	Mixed methods (qualitative–cross-sectional analytic) Duration: Not specified	Sample size: 16 Condition: cancer Age: 12–18 yr
Kenny17	Ireland	To explore adolescents' needs and concerns in relation to mental health apps	Qualitative Duration: 2 mo	Sample size: 34 Condition: mental health Age: 15–16 yr
Kock23	Germany	To design and implement a mobile app to increase compliance of childhood cancer survivors to aftercare programs	Mixed methods (qualitative–quantitative descriptive) Duration: 4 mo	Sample size: adolescents (13), caregivers (9) Condition: cancer Age: adolescents = 15–17 yr; caregivers = 40–54 yr
Sage21	USA	To assess the usability and user-centeredness of an asthma app intended for adolescents	Qualitative Duration: 1 day	Sample size: 8 Condition: asthma Age: 11–18 yr
Schneider19	USA	To engage teen asthma patients in the developmental stages of product design for an asthma self-management app for teenagers	Qualitative Duration: 10 days	Sample size: 16 Condition: asthma Age: 13–18 yr
Shellmer25	USA	To describe the development and testing of Teen Pocket Path prototype	Mixed methods (qualitative–quantitative descriptive) Duration: 6 wk	Sample size: adolescents (7), caregivers (9) Condition: solid organ transplant Age: adolescents = 11–18 yr; caregivers = 42–61 yr
Stinson26	Canada	To design, develop, and test the usability, feasibility, compliance, and satisfaction of a game-based smart phone app for adolescents	Mixed methods (qualitative–quantitative descriptive) Duration: 2 wk	Sample size: phase I (15), phase II (18), phase III (14) Condition: cancer Age: 9–18 yr

Table 2 contains the quality assessment results. Quality assessment ratings ranged from 50% to 100%, with 7 studies achieving a rating of 75% to 100%.^16–22 Interrater reliability of quality ratings was high (percent agreement, 82.86%).

Table 2.

Article Quality Assessment

Article	MMAT Design Category*	Quality Criteria*		Comments	Quality Score,* %
		Criteria	Criteria Met
Cafazzo8	Mixed methods (qualitative plus cross-sectional analytic)	Recruitment minimizes selection bias	No		50
		Appropriate measures and absence of contamination between groups	Yes
		Groups are comparable or differences controlled for	Yes
		Complete outcome data (≥80%) and acceptable response rate or follow-up rate if applicable	No
		Sources of qualitative data are relevant	Yes
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine	Researchers' influence is not specified
		Mixed method design relevant to the questions	Yes
		Integration of qualitative and quantitative data is relevant	Yes
		Appropriate consideration given to the limitations of this integration	No
Carpenter20	Qualitative	Sources of qualitative data are relevant	Yes		75
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine	Researchers' influence is not specified
Crosby24	Mixed methods (qualitative plus quantitative descriptive)	Is sampling strategy relevant to address research question?	No		50
		Is the sample representative of the population under study?	Yes
		Are measures appropriate and clear?	Yes
		Is there an acceptable response rate?	Unable to determine	Response rate not specified
		Sources of qualitative data are relevant	Yes
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Yes
		Mixed method design relevant to the questions	No
		Integration of qualitative and quantitative data is relevant	Yes
		Appropriate consideration given to the limitations of this integration	Yes
Gkatzidou22	Qualitative	Sources of qualitative data are relevant	Yes		100
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Yes

^* Quality assessed using the Mixed Methods Appraisal Tool (MMAT) – Version 2011. MMAT contains 5 study design categories: qualitative, quantitative randomized controlled, quantitative non-randomized, quantitative descriptive, and mixed methods. Each category contains 4 (or 3 for Mixed Methods) criteria. Scores range from 0% to 100%. For qualitative, quantitative randomized controlled, quantitative non-randomized, and quantitative descriptive studies, scores are calculated as a percentage of criteria met. For mixed methods studies, scores are calculated as the lowest score from among the 3 relevant designs (quantitative [Quan], qualitative [Qual], mixed methods [MM]): 25% when Quan = 1 or Qual = 1 or MM = 0; 50% when Quan = 2 or Qual = 2 or MM = 1; 75% when Quan = 3 or Qual = 3 or MM = 2; 100% when Quan = 4 or Qual = 4 or MM = 3.

Table 2.

Article Quality Assessment (cont.)

Article	MMAT Design Category*	Quality Criteria*		Comments	Quality Score,* %
		Criteria	Criteria Met
Herschman18	Qualitative	Sources of qualitative data are relevant	Yes		75
		Process of analyzing qualitative data is relevant	Unable to determine	The form of data analysis is not clear
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Yes
Holtz16	Qualitative	Sources of qualitative data are relevant	Yes		100
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Yes
Jibb27	Mixed methods (qualitative plus cross-sectional analytic)	Recruitment minimizes selection bias	No		50
		Appropriate measures and absence of contamination between groups	Unable to determine	Source of survey instrument not clear
		Groups are comparable or differences controlled for	Yes
		Complete outcome data (≥80%) and acceptable response rate or follow-up rate if applicable	Yes
		Sources of qualitative data are relevant	Yes
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Yes
		Mixed method design relevant to the questions	Unable to determine	Rationale for using mixed method not mentioned
		Integration of qualitative and quantitative data is relevant	Yes
		Appropriate consideration given to the limitations of this integration	Unable to determine	Limitations associated with mixed method not mentioned
Kenny17	Qualitative	Sources of qualitative data are relevant	Yes		75
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine

^* Quality assessed using the Mixed Methods Appraisal Tool (MMAT) – Version 2011. MMAT contains 5 study design categories: qualitative, quantitative randomized controlled, quantitative non-randomized, quantitative descriptive, and mixed methods. Each category contains 4 (or 3 for Mixed Methods) criteria. Scores range from 0% to 100%. For qualitative, quantitative randomized controlled, quantitative non-randomized, and quantitative descriptive studies, scores are calculated as a percentage of criteria met. For mixed methods studies, scores are calculated as the lowest score from among the 3 relevant designs (quantitative [Quan], qualitative [Qual], mixed methods [MM]): 25% when Quan = 1 or Qual = 1 or MM = 0; 50% when Quan = 2 or Qual = 2 or MM = 1; 75% when Quan = 3 or Qual = 3 or MM = 2; 100% when Quan = 4 or Qual = 4 or MM = 3.

Table 2.

Article Quality Assessment (cont.)

Article	MMAT Design Category*	Quality Criteria*		Comments	Quality Score,* %
		Criteria	Criteria Met
Kock23	Mixed methods (qualitative plus quantitative descriptive)	Is sampling strategy relevant to address research question?	No		50
		Is the sample representative of the population under study?	Yes
		Are measures appropriate and clear?	Yes
		Is there an acceptable response rate?	Unable to determine	Response rate not specified
		Sources of qualitative data are relevant	Yes
		Process of analyzing qualitative data is relevant	Unable to determine
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine	Researchers' influence is not specified
		Mixed method design relevant to the questions	Unable to determine	Rationale for using mixed method not mentioned
		Integration of qualitative and quantitative data is relevant	Yes
		Appropriate consideration given to the limitations of this integration	Unable to determine	Limitations of using mixed methods not discussed
Sage21	Qualitative	Sources of qualitative data are relevant	Yes		100
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Yes
Schneider19	Qualitative	Sources of qualitative data are relevant	Yes		75
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine	Researchers' influence is not specified

^* Quality assessed using the Mixed Methods Appraisal Tool (MMAT) – Version 2011. MMAT contains 5 study design categories: qualitative, quantitative randomized controlled, quantitative non-randomized, quantitative descriptive, and mixed methods. Each category contains 4 (or 3 for Mixed Methods) criteria. Scores range from 0% to 100%. For qualitative, quantitative randomized controlled, quantitative non-randomized, and quantitative descriptive studies, scores are calculated as a percentage of criteria met. For mixed methods studies, scores are calculated as the lowest score from among the 3 relevant designs (quantitative [Quan], qualitative [Qual], mixed methods [MM]): 25% when Quan = 1 or Qual = 1 or MM = 0; 50% when Quan = 2 or Qual = 2 or MM = 1; 75% when Quan = 3 or Qual = 3 or MM = 2; 100% when Quan = 4 or Qual = 4 or MM = 3.

Table 2.

Article Quality Assessment (cont.)

Article	MMAT Design Category*	Quality Criteria*		Comments	Quality Score,* %
		Criteria	Criteria Met
Shellmer25	Mixed methods (qualitative plus quantitative descriptive)	Is the sampling strategy relevant to address question?	No		50
		Is the sample representative of the population under study?	Yes
		Are measures appropriate and clear?	Yes
		Is there an acceptable response rate?	Unable to determine	Response rate not specified
		Sources of qualitative data are relevant	Yes
		Process of analyzing qualitative data is relevant	Unable to determine	Method of analyzing qualitative data not specified
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine	Researchers' influence is not specified
		Mixed method design relevant to the questions	Yes
		Integration of qualitative and quantitative data is relevant	Yes
		Appropriate consideration given to the limitations of this integration	Unable to determine
Stinson26	Mixed methods (qualitative plus quantitative descriptive)	Is the sampling strategy relevant to address question?	No		50
		Is the sample representative of the population under study?	Yes
		Are measures appropriate and clear?	Yes
		Is there an acceptable response rate?	Unable to determine	Response rate not specified
		Sources of qualitative data are relevant	Yes
		Process of analyzing qualitative data is relevant	Yes
		Appropriate consideration given to how findings relate to context	Yes
		Appropriate consideration given to how findings relate to the researchers' influence	Unable to determine	Researchers' influence is not specified
		Mixed method design relevant to the questions	Unable to determine	Rationale for using mixed method not mentioned
		Integration of qualitative and quantitative data is relevant	Yes
		Appropriate consideration given to the limitations of this integration	Unable to determine	Limitations associated with integration not specified

^* Quality assessed using the Mixed Methods Appraisal Tool (MMAT) – Version 2011. MMAT contains 5 study design categories: qualitative, quantitative randomized controlled, quantitative non-randomized, quantitative descriptive, and mixed methods. Each category contains 4 (or 3 for Mixed Methods) criteria. Scores range from 0% to 100%. For qualitative, quantitative randomized controlled, quantitative non-randomized, and quantitative descriptive studies, scores are calculated as a percentage of criteria met. For mixed methods studies, scores are calculated as the lowest score from among the 3 relevant designs (quantitative [Quan], qualitative [Qual], mixed methods [MM]): 25% when Quan = 1 or Qual = 1 or MM = 0; 50% when Quan = 2 or Qual = 2 or MM = 1; 75% when Quan = 3 or Qual = 3 or MM = 2; 100% when Quan = 4 or Qual = 4 or MM = 3.

Qualitative Synthesis. App Features Preferred by Adolescents. The qualitative synthesis performed by RJ and NH revealed mHealth app features preferred by adolescents. Preferred features commonly mentioned include ability to track test results or self-management progress; customize the app; connect to social media; gain points or prizes through app gamification; and easily navigate and read the app (including visual appeal and emphasis on using graphics instead of text). Table 3 contains a complete list of preferences.

Table 3.

Features of Mobile Apps Designed for Adolescent Use, and Rating Criteria for App Evaluation

Article	Platform	App Purpose	Preferred and Non-Preferred Features*	App Rating Criteria and Measures†
Condition: asthma
Carpenter20	Apple	Self-management of asthma in adolescents	Preferred features: medication reminder, chart and tracking features, peak flow records, emergency plan, self-check quizzes, doctor's report, goal setting	1) Perceived usefulness of app features, measured via 5-point Likert-type questions (1=not at all useful and 5=very useful).
Sage21	Not mentioned	Self-management of asthma in adolescents	Preferred features: ability to customize profile/charts, notifications/alerts/reminders, quizzes and badges, charts, logging medication, and tracking triggers, gamification, appealing look and feel	1) Look and feel of the app; 2) Likelihood of usage, measured with a 5-point Likert item (1=not at all likely and 5=very likely); 3) Usefulness, measured via a yes/no item.
			Non-preferred features: Bar charts
Schneider19	Apple	Self-management of asthma in adolescents	Preferred features: prompts to use the app, reminders to take medication or refill, gamification (asthma-related games), personalization, built-in peak flow meter, motivational and supportive messages, direct communication with provider, medication log	1) Ease of use; 2) Number of steps; 3) Degree to which app use is intuitive and self-explanatory; 4) Teen-friendliness of graphics, fonts, font-size, and color.
			Non-preferred features: lack of notifications
Condition: cancer
Kock23	Not mentioned	Management of childhood cancer aftercare	Preferred features: personalization of content	1) Suitability for individualization; 2) Error tolerance; 3) Controllability; 4) Suitability for learning; 5) Conformity with user expectations; 6) Self-descriptiveness; 7) Suitability for the task.
Condition: diabetes
Cafazzo8	Apple	Self-management of type 1 diabetes mellitus in adolescents	Preferred features: fast and discrete transaction, assistance on timely decision-making based on data, simple data display, gamification, ability to see trends, social media	1) Speed; 2) Availability of help screens to guide user interaction; 3) Simplicity of data display; 4) Possession of decision-support prompts or alerts; 5) Safe information sharing.
Holtz16	Not mentioned	Self-management of type 1 diabetes mellitus in adolescents	Preferred features: getting redeemable points through the app, team membership, gamification, social media	1) Motivating factors for use (customizability, interactivity, tangible rewards).
Condition: lupus
Herschman18	Apple	Management of lupus in adolescents	Preferred features: symptom tracking, logs, links to information, gamification, pop-up notifications with minimal text or audio/video options, social media, medication list, medication reminder, appointment reminder, easy navigation, customizable, look and feel	1) Physical (e.g., symptom tracking) components are present; 2) Emotional components are present; 3) Intellectual (knowledge-seeking) components; 4) Social components; 5) Practical components; 6) Technical components; 7) System components; 8) Personalization components.
Condition: mental health
Kenny17	Not mentioned	Management of mental health in adolescents	Preferred features: multimedia features such as music and video included	1) Safety (confidentiality, cyber-bullying, social stigma); 2) Engagement (content, appearance, incentive to use); 3) Functionality (information, access to professional help, improved health outcomes, alternative emotional outlet); 4) Social interaction (communicating with peers, young people helping young people, relating to others' experiences); 5) Promoting awareness (word of mouth, online media, other popular media); 6) Accessibility (ease of use, free of cost); 7) Ability to tailor based on gender; 8) Young people in control.
			Non-preferred features: large blocks of text
Condition: organ transplant
Shellmer25	Android	Medication adherence after solid organ transplant in adolescents	Preferred features: access to medication list, alerts for changes to medication list, access to medication adherence history, automatic messaging to caregiver to support medication taking	1) Ease of use; 2) Simplicity of reminders and warning messages; 3) Satisfaction with the app and its use; 4) Appeal of using the app for a long time period.
Condition: pain
Jibb27	Apple	Pain management in adolescents	Preferred features: real-time pain assessment	1) Ease of use; 2) Ease of understanding; 3) Efficiency; 4) Acceptability (design, content, navigation, utility, customizability).
			Non-preferred features: excessive number of steps required to complete functions (navigation), software malfunction (app crashing), poor responsiveness of buttons, complicated text
Stinson26	Apple	Pain management in adolescents	Preferred features: gamification, audible alarms, color scheme, font, and graphics	1) Likeability; 2) Overall appearance; 3) Ease of use; 4) Interference with daily activities and friendships.
			Non-preferred features: complicated text
Condition: sexual health
Gkatzidou22	Not mentioned	Management of sexual health in adolescents	Preferred features: colors that reflect credibility, help functions, credibility of language	1) Social privacy (password protection, privacy settings, discreet design); 2) Institutional privacy (assurances and disclaimers, disclosures, confidentiality and security policy); 3) Credibility and legitimacy (language, visual aesthetics, user community cues, affiliations, identification of app operator, assurances of medical content accuracy); 4) User journey support (simplification of complex health journeys: visual trackers, overviews, content relevance and logic, specific and appropriate feedback, reassurances, flexibility in the delivery of support); 5) Task-technology-context fit (ubiquity, mobility, customization).
Condition: sickle cell disease
Crosby24	Not mentioned	Self-management of sickle cell disease in adolescents	Preferred features: visualization of self-management progress, customizable profile and goals, social interaction via text messages, ability to perform team competitions	Measured on a 0 to 5 scale: 1) Ease of use; 2) Benefit for tracking symptoms; 3) Degree of tailoring to suit user needs; 4) Extent of ability to choose self-management goals; 5) Extent of ability to communicate with others about self-management strategies; 6) Extent of peer support functions.

^* Preferred and non-preferred features are from the perspective of adolescents.

^† Rating criteria were reported by the study authors and were used by study participants (adolescents, parents, or providers) to evaluate the apps for adolescent use across a range of usability factors.

Table 3.

Features of Mobile Apps Designed for Adolescent Use, and Rating Criteria for App Evaluation (cont.)

Article	Platform	App Purpose	Preferred and Non-Preferred Features*	App Rating Criteria and Measures†
Condition: mental health
Kenny17	Not mentioned	Management of mental health in adolescents	Preferred features: multimedia features such as music and video included	1) Safety (confidentiality, cyber-bullying, social stigma); 2) Engagement (content, appearance, incentive to use); 3) Functionality (information, access to professional help, improved health outcomes, alternative emotional outlet); 4) Social interaction (communicating with peers, young people helping young people, relating to others' experiences); 5) Promoting awareness (word of mouth, online media, other popular media); 6) Accessibility (ease of use, free of cost); 7) Ability to tailor based on gender; 8) Young people in control.
			Non-preferred features: large blocks of text
Condition: organ transplant
Shellmer25	Android	Medication adherence after solid organ transplant in adolescents	Preferred features: access to medication list, alerts for changes to medication list, access to medication adherence history, automatic messaging to caregiver to support medication taking	1) Ease of use; 2) Simplicity of reminders and warning messages; 3) Satisfaction with the app and its use; 4) Appeal of using the app for a long time period.
Condition: pain
Jibb27	Apple	Pain management in adolescents	Preferred features: real-time pain assessment	1) Ease of use; 2) Ease of understanding; 3) Efficiency; 4) Acceptability (design, content, navigation, utility, customizability).
			Non-preferred features: excessive number of steps required to complete functions (navigation), software malfunction (app crashing), poor responsiveness of buttons, complicated text
Stinson26	Apple	Pain management in adolescents	Preferred features: gamification, audible alarms, color scheme, font, and graphics	1) Likeability; 2) Overall appearance; 3) Ease of use; 4) Interference with daily activities and friendships.
			Non-preferred features: complicated text
Condition: sexual health
Gkatzidou22	Not mentioned	Management of sexual health in adolescents	Preferred features: colors that reflect credibility, help functions, credibility of language	1) Social privacy (password protection, privacy settings, discreet design); 2) Institutional privacy (assurances and disclaimers, disclosures, confidentiality and security policy); 3) Credibility and legitimacy (language, visual aesthetics, user community cues, affiliations, identification of app operator, assurances of medical content accuracy); 4) User journey support (simplification of complex health journeys: visual trackers, overviews, content relevance and logic, specific and appropriate feedback, reassurances, flexibility in the delivery of support); 5) Task-technology-context fit (ubiquity, mobility, customization).
Condition: sickle cell disease
Crosby24	Not mentioned	Self-management of sickle cell disease in adolescents	Preferred features: visualization of self-management progress, customizable profile and goals, social interaction via text messages, ability to perform team competitions	Measured on a 0 to 5 scale: 1) Ease of use; 2) Benefit for tracking symptoms; 3) Degree of tailoring to suit user needs; 4) Extent of ability to choose self-management goals; 5) Extent of ability to communicate with others about self-management strategies; 6) Extent of peer support functions.

^* Preferred and non-preferred features are from the perspective of adolescents.

^† Rating criteria were reported by the study authors and were used by study participants (adolescents, parents, or providers) to evaluate the apps for adolescent use across a range of usability factors.

App Rating Criteria. Many studies performed hands-on usability tests with adolescents to assess app quality in terms of meeting end users' needs for app functionality and likability. Methods of measuring app quality varied across studies, including qualitative discussion, survey measures using Likert-type scales, and/or other survey rating measures. Quality criteria most often mentioned included degree of app customizability, ease of use, visual appeal, interactivity (with peers, clinicians, or social media), self-management capability, and condition-specific features (Table 4).

Table 4.

Dimensions for Rating the Quality of mHealth Apps Targeted to Adolescents

Technical Quality	Engagement	Support System	Autonomy	Safety, Privacy, and Trust
Efficiency Task-technology fit Ease of use Intuitiveness Conformity with user expectation Error tolerance Controllability Suitability for learning	Interactivity Customizability Youth-preferred entertainment Gamification Aesthetics Simplicity of text and data display	Behavior change support Decision support Condition-specific support Social support Learning support Institutional support	Degree of adolescent control Cost Platforms	Safety from social stigma Credibility Information security

Theoretical Framework. Figure 2 presents a framework for developing health-related mobile apps for adolescents, based on qualitative synthesis of adolescent-preferred app features and common quality rating criteria from the included studies. Five dimensions emerged: 1) Technical Quality, including 8 constructs, such as app ease of use; 2) Engagement, including 6 constructs, such as app interactivity; 3) Support System, including 6 constructs, such as decision support or behavior change support features; 4) Autonomy, including 3 constructs, such as app accessibility in terms of cost; and 5) Safety, Privacy, and Trust, including 3 constructs, such as app safety for adolescent use.

Figure 2.

Theoretical framework.

Discussion

This study found that adolescents prefer mHealth apps that are customizable, offer peer support through social media, sustain engagement via gamification, and support the ability to visualize health trends via simplified graphs. In terms of rating criteria for adolescent mHealth apps, the degree of ease of use, perceived usefulness, customizability, efficiency, appeal, privacy, credibility, and interactivity are commonly reported as rating criteria in studies that design, develop, or evaluate mHealth apps intended for adolescent use. Five dimensions emerged from the qualitative synthesis of adolescent preferences for mHealth app features and quality rating criteria. The dimensions include Technical Quality, Engagement, Support System, Autonomy, and Safety, Privacy, and Trust. Technical Quality includes quality dimensions pertinent to app usability. Engagement reflects the ability to sustain usage of the app due to the presence of positive features such as gamification. Support System includes mechanisms to aid users in different aspects relevant to adolescent mHealth app usage. Autonomy refers to the app being free of cost and the extent to which adolescent autonomy is enabled. For example, adolescent autonomy could be enabled by giving adolescents the liberty to share information with peers and healthcare providers, engage in an advocated behavior, or change the look and feel of the app without continual parental oversight. Safety, Privacy, and Trust describes the use of features such as a discreet design or password to protect users from negative events such as cyberbullying and social stigma. This dimension also covers credibility in terms of the app's visual appearance and information communicated via the app appearing trustworthy.

Adolescent preferences informed the theoretical framework proposed for the evaluation of mHealth apps targeted to adolescents in this study. Previous studies support our finding that adolescents prefer mHealth apps to be customizable^18,28,29 and include social networking,^30,31 gamification,¹⁸ charts and tracking,³² multimedia,³³ and treatment reminders.^18,28 Our findings on adolescents' need for social interaction features corroborates the findings of previous studies that reported that adolescents prefer their friends to be involved in self-management mHealth apps. For example, Roberts et al³⁰ found that adolescents expressed a need to video-chat with friends, connect with friends from school, and create a communication network via social media within mHealth apps. Similarly, Bendixen (2017) reported that adolescents expressed the need for social media to provide a social support system in apps developed for adolescents.³⁴ Additionally, previous studies support our findings on adolescents' preferences for concise and simple displays of data and textual information, especially in displays of tracked monitoring parameters for health conditions.¹⁸ App information and displays should also appear credible. For example, Nightingale et al³⁵ reported accuracy and trustworthiness of app information as a desirable component of mHealth apps targeted to children and adolescents. This supports our findings that adolescents prefer mHealth apps that appear credible in terms of the visual design and tone or language used. Young adults age 18 to 30 years have also shown similar preferences to adolescents in this regard, reporting reluctance to read text notifications that did not appear credible.³⁶ Finally, adolescents' need for autonomy in terms of the choice to use an mHealth app, sharing personal information within the app, and feeling in control of their health condition when using the app is another adolescent need confirmed by previous studies.^29,37

Common mHealth app preferences between adolescents and adults include features that facilitate social networking, communication with healthcare providers, and customizability.³⁸ Although both adolescents and adults express a need for social networking capabilities in mHealth apps, our findings show that adolescents prefer networking with their peers, or people of a similar age group, whereas adults prefer people with similar conditions, and trusted family and friends.^38,39 Further, although both adults and adolescents prefer mHealth apps to include features enabling communication with healthcare providers, our study found that adolescents were primarily interested in providers' ability to view health data for monitoring purposes and the ability to ask questions and receive feedback from providers.^19,20,24 Conversely, adults may prefer more advanced connections with the healthcare system, such as automation for prescription orders and refill processes, in addition to direct communication with healthcare providers.³⁸ Adolescents may not see the need for these types of automated features because their parents or guardians typically obtain and refill prescriptions on their behalf. In addition to features facilitating communication with providers, adolescents prefer features that facilitate communication of data with parents and peers for informational purposes.⁸

A difference between adolescents' and adults' preferences in the design of mHealth apps is adolescents' need for features that support decision-making. This may include app suggestions for health condition-related actions based on tracked trends in monitoring parameters.^8,25 Also, although both adolescents and adults have concerns about information privacy when using mHealth apps, the context may be different. For instance, cyberbullying may be a unique privacy concern for adolescents. Additionally, while our results indicate that adolescents prefer mHealth apps to include gamification components, such as the ability to earn points based on app use, electronic badges, and interactive quizzes, recent studies on adults' app preferences indicate that adults are less enthusiastic about gamification compared with adolescents, including the use of prizes, rewards, or competition to motivate health behavior.^36,38

App rating criteria most often used by the included studies in adolescent populations are generally consistent with app quality rating criteria employed by app development and usability studies in adult populations. For instance, Birkhoff et al⁴⁰ identified similar rating criteria themes in their app usability study, such as perceived usefulness, ability to track health, availability of reminders, and customizability. Communication, education, and socialization have also emerged as app rating criteria themes in a previous study involving adults.³⁸ Further, several dimensions and constructs in the proposed framework align with constructs from established health behavior and/or technology use theories, theoretical frameworks, or conceptual models. For example, perceived usefulness and ease of use constructs are consistent with constructs from the Technology Acceptance Model and the Unified Theory of Acceptance and Use of Technology model.^41,42

Tools for app quality assessments exist, including the App Quality Evaluation tool (AQEL)⁴³ intended for nutritional apps and the Mobile App Rating Scale (MARS).⁴⁴ The app rating criteria in AQEL consist of Appropriateness for Adults, App Functionality, Skill Building, Knowledge Building, Behavior Change Potential, and Appropriateness for Dietary Approaches to Stop Hypertension dimensions.⁴⁵ The Knowledge Building, Skill Building, Appropriateness for Dietary Approaches to Stop Hypertension, and Behavior Change Potential dimensions are similar to constructs of the proposed framework's Support System dimension, including learning support, condition-specific support, and behavior change support. AQEL's Functionality dimension maps to a subset of the constructs within our proposed Technical Quality dimension, such as efficiency and ease of use. Similarly, AQEL's Appropriateness for Adults dimension achieves the same purpose as the Engagement dimension of our framework by ensuring that the app is targeted toward a specific age group. However, the AQEL framework does not address adolescent safety, privacy, and trust concerns, credibility, tailoring for adolescents in terms of the need for autonomy and social support, and may only be used for rating nutritional mHealth apps.

The MARS tool uses 4 overarching dimensions for evaluating the quality of mobile apps: Engagement, Functionality, Aesthetics, and Information. There are similarities between these dimensions and the proposed framework. For instance, the Engagement and Aesthetics dimensions in MARS are similar to the Engagement dimension in the proposed framework. Constructs of the MARS' Information dimension, such as credibility and accuracy of information, are constructs within the Safety, Privacy, and Trust dimension of the proposed framework. Further, the constructs measured under Functionality in MARS are similar to several constructs under Technical Quality in our framework. Despite these similarities, there are some differences between the proposed framework and MARS. The proposed framework is sensitive to apps created specifically for the adolescent population by capturing app features that enable adolescent autonomy, accessibility in terms of being free of cost, appeal to adolescents, and simplicity of textual and graphical data presentation. It also captures the need for multiple support systems in mHealth apps intended for adolescent use, including behavior change support, decision support, condition-specific support based on particular health conditions, social support, learning support, and institutional support. Finally, the proposed framework differs from MARS by addressing the issue of safety and privacy.

The Example of Medication Adherence. The proposed framework may be adapted to fit the needs of app developers and evaluators addressing a variety of adolescent health conditions and health behaviors. Here we describe an example of how the proposed framework might be adapted to fit general medication-taking behavior (non-condition-specific medication adherence) in adolescents. Dimensions of the framework that may be operationalized in a form befitting a medication adherence app for adolescents are described below. Technical Quality and Safety, Privacy, and Trust are generic and useful for app development or evaluation regardless of the health condition or behavior.

1. Engagement. The gamification construct may be adapted to transform behaviors such as medication taking or logging behaviors into fun activities for adolescents. Electronic badges or redeemable points may be awarded to users as they take and log their medications.

2. Support System. The 6 different types of support systems: behavior change, decision, condition-specific, social, learning, and institutional support may be adapted to support medication taking. First, behavior change support is critical to aid adolescents in forming and maintaining consistent medication-taking behavior. Relevant support may also include reminders to take medication and update medication logs (cues to action), feedback on medication-taking behavior displayed as simplified line graphs (self-regulation), and notifications about levels of adherence to the prescribed medication regimen (self-regulation). Second, decision support may take the form of suggested methods to become more adherent with medication taking if an adolescent's level of adherence is low. Third, a condition-specific support for medication adherence may include an app feature that enables adolescents to input the names and pictures of their medications, a medication logging feature, and a graphical display of medication taking or logging trends for specific medications or conditions. Fourth, the app may have a social networking component with the capability for adolescents to connect with individuals who also use the app for medication adherence purposes. Fifth, the app may offer links to or in-app educational resources for health conditions associated with users' medications, topics pertaining to the benefits of being adherent to medications, and information on typical medication dosages, side effects, drug-drug interactions, and contraindications. The final component of the support system, institutional support, may also be adapted. For instance, features that enable adolescents to share their medication log with school clinics and healthcare providers may be included to facilitate health monitoring or for incentivization purposes.

3. Autonomy. An important construct is adolescent autonomy. In this regard, questions to guide app developers or mHealth program planners are: 1) Does the app enable adolescents to feel in control of their medication-taking behavior? 2) Do adolescents have the capability to independently choose if adherence information is shared with others? 3) Is the app free of cost and available on multiple platforms such that adolescents can more easily access the app independently?

Study Limitations. This study is not without limitations. As the review is based on existing development and usability studies of mHealth apps intended for adolescent use, it is possible that salient preferences and app rating criteria themes may not have been captured by the original studies. However, due to the systematic procedure used in study selection and the relatively high quality of most of the retained studies, we have confidence that the preferences and rating criteria outlined here reflect those most salient to an adolescent population as represented in the current literature, and the proposed theoretical framework is suitable for evaluating mHealth apps intended for adolescent use. Also, this study did not include articles evaluating the usability of apps intended for adolescents with neurodevelopmental conditions, such as autism. This may limit the generalizability of findings to such populations.

Implications for Policy and Practice. This research adds new knowledge on app design concepts that may be adapted for the development of mHealth apps targeted to adolescent users. The dimensions in the proposed framework may also guide the evaluation of adolescent mHealth apps. In addition to including features that offer condition-specific support in mHealth apps targeted to adolescents, there is a need to incorporate features that offer other types of support pertinent to adolescents. This includes decision-making, behavior change, institutional, learning, and social support. Future studies should evaluate the most relevant types of support within mHealth apps that effectively promote healthy behaviors among adolescent users. It is also important for app developers to explore gamification within mHealth apps by using rewards like redeemable points and electronic badges, as well as app customizability and simplicity of textual and graphical information display. However, future studies should clarify which fonts, colors, and graphics are adolescent-friendly in order to facilitate effective implementation of gamification, customizability, and easily understood textual and graphical information displays in mHealth apps intended for adolescent use. Future research should also ascertain the ethical considerations for granting adolescents the autonomy to share health data with peers and healthcare providers via mHealth apps. Lastly, the proposed framework may be tested using exploratory and confirmatory factor analysis, and used to create a rating tool for adolescent mHealth apps after validation in samples of adolescents.

Conclusions

This paper describes adolescents' preferences for the design of mHealth apps, as well as the app rating criteria that have been used for this population. Adolescents value mHealth apps that are customizable and appealing due to the use of adolescent-friendly graphics, fonts, and colors. Apps that appear credible and use elements of gamification to support app use or the intended health behavior are also preferred. The incorporation of features described in this report may help in the development of mHealth apps that are preferred by adolescents, improving adolescents' engagement with and use of mHealth apps, which may in turn lead to improved self-management of chronic conditions and positive health outcomes for adolescents.

ABBREVIATIONS

AQEL

App Quality Evaluation tool

MARS

Mobile App Rating Scale