Skip to main content
NCBI home page
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2018 Oct 24;44(3):333–342. doi: 10.1093/jpepsy/jsy086

Content and Usability Evaluation of Medication Adherence Mobile Applications for Use in Pediatrics

Julia K Carmody 1,2,3,✉, Lee A Denson 4,5, Kevin A Hommel 1,2,3,5
PMCID: PMC6415658  PMID: 30358863

Abstract

Objective

The objective of this study was to systematically evaluate commercially available medication adherence apps for the inclusion of behavior change techniques (BCTs) and to conduct a usability analysis on a subset of apps with adolescents and young adults living with a chronic illness.

Methods

Medication adherence apps were identified via a search of iTunes app store in August 2016. Seventy-five apps meeting initial inclusion criteria were independently coded by two researchers for the presence/absence of 26 BCTs. Twenty adolescents and young adults (ages: 13–20 years) with inflammatory bowel disease conducted usability testing on a subset of apps (n = 4).

Results

Across 75 apps coded for presence/absence of 26 BCTs, only 7 unique BCTs were identified. The number of BCTs per app ranged from 2 to 6, with an average of 3.3 BCTs. In usability testing, quality ratings varied across apps. Medisafe received the highest average scores on engagement, functionality, aesthetics, and information subscales. Medisafe and MyTherapy ranked first and second, respectively, on overall quality and perceived impact ratings.

Conclusion

Content evaluation revealed only a limited number of BCTs that have been translated to medication adherence apps. Among apps with comparable content, clear user preferences emerged based on perceived quality and usability. Greater collaboration is needed between psychologists and health technologists to incorporate more evidence-based BCTs in apps. Findings also indicate a need for app developers to consider and incorporate the preferences of younger end users to improve app quality and engagement for pediatric populations.

Keywords: adherence, behavior change technique, m-health, mobile app, usability

Nonadherence to medication among children, adolescents, and young adults is a pervasive problem. Across chronic conditions, it is estimated that approximately 50% of children and as high as 75% of adolescents and young adults do not take their medications as prescribed (Rapoff, 2010). Consequences of nonadherence to medication include medical complications, treatment failure, impairments to daily functioning and quality of life, and higher health care utilization (McGrady & Hommel, 2013; Rapoff, 2009, 2010). Meta-analytic reviews indicate that pediatric adherence-promoting interventions lead to improved adherence, better health outcomes, enhanced patient and caregiver quality of life, and reduced health-care utilization (Graves, Roberts, Rapoff, & Boyer, 2010; Kahana, Drotar, & Frazier, 2008; McGrady et al., 2015; Pai & McGrady, 2014). The most effective adherence-promoting interventions are multicomponent and include educational and behavioral techniques (Pai & McGrady, 2014). However, patient access to such interventions may be limited by geographical, resource, and time constraints. Mobile adherence applications (apps) represent an innovative and widely available approach to enhancing treatment adherence in youth. Adolescents and young adults in particular stand to benefit from mobile adherence promotion platforms, as 89% of adolescents (ages: 12–17 years) and 98% of young adults (ages: 18–24 years) have access to a smartphone (Pew Research Center, 2017).

Currently, an abundance of apps aimed at helping users take their medications is publicly available; however, the degree to which apps incorporate evidence-based techniques to promote health behavior change is unclear. A recent review by Nguyen et al., (2016) examined the content of medication reminder apps based on a list of features designated by the authors as important for pediatric adherence promotion. Specifically, the authors examined apps for the inclusion of medication reminder features, educational material, and five behavior modification features. Results showed that while most apps contained several medication reminder features (e.g., record medications and set reminder), apps contained on average less than one educational and less than one behavioral modification feature. While this review highlights the available reminder features and lack of educational content in apps, it does not offer a comprehensive review of included behavior change techniques (BCTs). To fully characterize the degree to which apps incorporate a comprehensive range of evidence-based techniques used in traditional health promotion interventions, the current study sought to systematically review available apps according to Abraham and Michie’s taxonomy of BCTs (2008). The taxonomy of BCTs is a coding system that allows researchers to reliably identify operationalized, theory-linked BCTs used in behavior change interventions. Twenty-six individual BCTs are defined in the taxonomy, including techniques such as provide information on behavior–health link, prompt specific goal setting, and provide feedback on performance. The taxonomy has been widely used to evaluate health behavior interventions, including interventions delivered on mobile platforms (Brannon & Cushing, 2015; Dahlke, Fair, Hong, Beaudoin, Pulczinski, & Ory, 2015; Yang, Maher, & Conroy, 2015).

While the most effective adherence promotion interventions for children, adolescents, and young adults include behavioral approaches, there is not clear consensus on which behavioral techniques (e.g., specific goal-setting, self-monitoring, and social comparison) are central to effective medication adherence interventions (Dean, Walters, & Hall, 2010; Pai & McGrady, 2014). In the adult literature, two systematic reviews of medication adherence interventions for adults with chronic illness found that the inclusion of medication monitoring with tailored feedback and reinforcement or rewards significantly predicted intervention effectiveness (Demonceau et al., 2013; Kripalani, Yao, & Haynes, 2007). Van Genugten and colleagues conducted a meta-analysis of 52 Internet-based health behavior interventions and found that barrier identification/problem-solving and providing rewards were significant positive predictors of intervention effect size (van Genugten, Dusseldorp, Webb, & van Empelen, 2016). When used in combination, barrier identification/problem-solving and providing rewards were found to have an even greater influence on intervention effectiveness. Because the pediatric/young adult medication adherence promotion literature has not clarified a specific technique or combination of techniques that are most effective for adherence promotion, the current study coded for the presence or absence of all 26 BCTs included in the taxonomy. Given the findings in the adult literature, our content evaluation also aimed to specifically examine whether those BCTs with the most evidence for adherence promotion in the adult literature (i.e., monitoring with feedback, reinforcement/rewards, and barrier identification/problem-solving) were included in medication adherence apps.

In addition to the inclusion of evidence-based BCTs, the potential for any app to improve health behaviors depends on its usability, meaning the quality of engagement, functionality, aesthetics, and information (Stoyanov et al., 2015). Several studies have evaluated the usability of various mobile health apps according to researcher ratings (Bardus, van Beurden, Smith, & Abraham, 2016; Mani, Kavanagh, Hides, & Stoyanov, 2015; Nguyen et al., 2016). However, to our knowledge, no studies to date have examined usability from the perspectives of adolescent or young adult end users. Therefore, the second aim of this study was to evaluate the usability of a subset of medication adherence promotion apps from the perspectives of adolescents and young adults living with a chronic illness. Taken together, the overarching goal of this study was to systematically evaluate and identify apps to recommend to pediatric and young adult patients with a variety of medical needs and conditions based on app content (included BCTs) and usability ratings.

Methods

Content Evaluation

The US iTunes App Store was reviewed in August 2016 for medication tracking apps. Apps were identified using the following search terms: “medication,” “medication reminder,” “adherence,” “medication adherence,” and “medication compliance.” The initial search yielded 271 apps. In total, 148 unique apps were included after duplicates were removed. App store descriptions were reviewed for the following inclusion criteria: (1) primarily for managing medications, (2) includes medication reminder, and (3) designed for patient use. Apps were excluded based on the following: (1) designed for use by medical professionals only, and (2) only accessible to patients enrolled in an associated medication trial, practice, or insurance policy, and (3) designed exclusively for users with a specific chronic condition (e.g., asthma and HIV). To evaluate and identify apps that could be broadly recommended to pediatric and young adult users with a variety of medical conditions and needs, we chose to exclude apps designed for specific chronic conditions. A detailed description of the search strategy is included in Online Appendix A. After the initial review, 75 apps met inclusion criteria and 73 apps were excluded. Screening and exclusion details are included in the flow chart (Figure 1).

Figure 1.

Figure 1.

Open in a new tab

App search flow diagram.

For all apps that met initial inclusion criteria, the first author coded each app store description (i.e., text, pictures, and videos posted on the iTunes store page) for the presence/absence of 26 BCTs using the Taxonomy of BCT coding manual (Abraham & Michie, 2008). The manual includes definitions of the 26 BCTs and explanations of how to identify each technique in intervention descriptions. BCTs were coded with a “0” if absent and a “1” if present. All 75 apps were then independently coded by a second reviewer. Cohen’s kappa interrater reliability showed substantial agreement (k = .726; p < .005). Coding discrepancies were resolved with discussion.

Usability Testing

The next step was to identify a subset of apps for usability testing with adolescent and young adult users. Apps were selected for usability testing from the 75 apps identified in the initial app store review based on the following: (1) app included more than three BCTs and (2) app included a feature for sharing adherence data with another user (i.e., caregiver) to facilitate real-time monitoring and/or prompting. The decision to conduct usability testing with apps using more than three BCTs was based on recent systematic reviews of technology-based self-management interventions, which indicate that the inclusion of more BCTs is associated with increased intervention effectiveness compared to interventions using fewer BCTs (Morrison et al., 2014). Inclusion of a sharing feature is particularly important for child and adolescent users, as families typically share condition management responsibilities. Furthermore, mobile health interventions with caregiver involvement produce greater change in health behaviors than interventions that target only youth (Fedele, Cushing, Fritz, Amaro, & Ortega, 2017). Seven apps met the aforementioned inclusion criteria for usability testing and two members of the research team downloaded and tested each app for 24 hr. Two apps were subsequently excluded because of frequent crashing/bugs. The remaining five apps selected for usability testing were downloaded and coded for presence/absence of BCTs to compare BCTs identified during app use to BCTs identified from app store descriptions.

Sample Selection

A convenience sample was recruited from the pediatric inflammatory bowel disease (IBD) clinic. Adolescents and young adults of age 13–21 years who were diagnosed with IBD and were prescribed at least one daily medication or supplement in the past year were eligible. The age range was chosen based on the readability score for the usability measure (Flesch–Kincaid index = 70; 8th grade equivalent).

The goal of this study was to systematically evaluate and identify apps to recommend to youth and young adult patients with a variety of medical needs and conditions. Several common characteristics in adolescents and young adults with IBD support generalizability of this sample to other pediatric and young adult populations. First, adolescents and young adults with IBD report many of the same adherence barriers commonly identified in other condition groups, such as forgetting, interference by other activities, being away from home, and regimen complexity (Hanghøj & Boisen, 2014; Hommel & Baldassano, 2010; Ingerski, Baldassano, Denson, & Hommel, 2010). Second, adolescents and young adults with IBD are typically prescribed a number of medications and supplements, often with different dosing schedules for each (e.g., once daily, twice daily, and once weekly). Changes to treatments, including type of medication prescribed, dose, and frequency, are not uncommon, particularly for patients with nonadherence (Robinson, Hankins, Wiseman, & Jones, 2013). Similar variability and complexities in medication regimens are common in other pediatric populations, including asthma, cystic fibrosis, HIV, diabetes, and juvenile arthritis (Santer, Ring, Yardley, Geraghty, & Wyke, 2014; Rapoff, 2009). Third, adolescence represents a critical developmental period as individuals begin to establish independence from caregivers and assume more self-management responsibilities, often resulting in negative consequences for adherence. This pattern is consistently observed across disease groups (DiMatteo, 2004). As such, adolescents and young adults in particular stand to potentially benefit from the use of mobile health tools to support self-management.

Procedures and Participants

The study was approved by the institutional review board. For participants <18 years of age, a parent provided consent and children indicated assent. Participants ≥18 years provided their own consent.

Participants were 5 males and 15 females, 13–20 years old (M = 16.9, SD = 2.00). In total, 19 participants (95%) were Caucasian and 1 participant was African American. Participants tested each app on a study iPod for at least 10 min, which is the length of time recommended by measure developers for testing before completing the usability measure (Stoyanov, Hides, Kavanagh, & Wilson, 2016). Participants were instructed to enter one of their medications, set a reminder, enable the caregiver sharing feature, and examine other available app components (e.g., adherence logs). All study visits and testing occurred in a private clinic room with a member of the research team present to oversee time spent on each app. The order in which each participant tested the five apps was determined via computer-generated randomization.

Immediately after testing each app, participants completed the user version of the Mobile Application Rating Scale (uMARS). The uMARS is a 26-item measure designed and validated for usability testing with mobile health apps (Stoyanov et al., 2016). Participants rated items pertaining to each subscale (engagement, functionality, aesthetics, and quality of information) on a five-point scale, from 1 (inadequate) to 5 (excellent). Participants also completed the subjective quality subscale on the uMARS, which assessed (1) whether users would recommend the app to others, (2) how frequently users would anticipate using the app, (3) whether users would pay for the app, and (4) users’ overall impression of the app (i.e., users provided a star rating). Finally, participants completed the perceived impact scale on the MARS, which assessed user perceptions of how app use would impact awareness of the health behavior (i.e., taking medications), knowledge about the behavior, attitudes and intentions toward improving the behavior, and likelihood that using the app would increase/decrease the health behavior.

Statistical Analyses

Subscale scores for engagement, functionality, aesthetics, and quality of information on the uMARS were calculated by averaging item responses within each domain. The overall quality score for each app was calculated by averaging the four subscale scores. Separate subjective quality and perceived impact scores were determined by averaging responses to items within those domains. We conducted two repeated measures analyses of variance (ANOVAs) to determine if there were differences across apps on overall quality and perceived impact ratings. Paired samples t-tests were conducted for post hoc comparisons between apps. To account for multiple comparisons, results were interpreted using a Bonferroni correction (significance level p < .008). This correction was determined by dividing the desired alpha level (.05) by the number of comparisons (6).

Updated App Store Review and Content Evaluation

To provide an update to the app store search and content review conducted in August 2016, a new search was conducted in July 2018. Using the same search terms as the first app store review, 326 apps were identified. After removing duplicates, app store descriptions for the remaining 168 apps were reviewed for inclusion/exclusion criteria (Figure B1). In total, 54 new apps met inclusion criteria and were coded for the presence/absence of 26 BCTs.

Results

Content Evaluation

Across 75 apps coded for presence/absence of 26 BCTs, only 7 unique BCTs were identified. The most common BCTs were “prompt specific goal setting.” and “teach to use prompts/cues” (Table I). Both of these BCTs are accomplished by the user entering their medication(s), dose frequency, and time(s) of administration while setting a reminder. Based on inclusion criteria, this function was included in all 75 coded apps. In total, 37 apps (49.3%) provided feedback on adherence in the form of an adherence report/log depicting daily, weekly, and/or monthly adherence rates. In total, 20 apps (27%) included at least one of the following additional BCTs: planning for social support, providing rewards, and providing information on consequences of taking medication (Table I). No apps included barrier identification. The number of BCTs per app ranged from 2 to 6 (M = 3.27, SD = 1.16; Figure 2).

Table I.

BCTs Included in Apps

BCT Example applications of BCT in apps Number of (%) apps with BCT
Prompt specific goal setting
Detailed planning of what individual will do, including a specific definition of behavior (e.g., frequency and duration). At least one context (i.e., where, when, how, or with whom) must be specified To set reminders, user enters medication duration, frequency, and planned time of administration 75 (100.0)
Teach to use prompt/cues
Teach individual to identify environmental prompts, which can be used to remind them to perform the behavior (e.g., time of day) User sets reminder alarm prompting user to take medication at specified time 75 (100.0)
Prompt self-monitoring
Individual is asked to keep a record of behavior. App prompts user to record whether medication was taken or missed 42 (56.0)
Provide feedback on performance
Individual receives data about recorded behavior Provides log, report, or graph showing adherence rate over time 37 (49.3)
Plan social support or social change
Involves prompting individual to think about how others could change their behavior to offer help or social support, such as setting up a buddy system Allows user to share adherence reports
User can invite others to “care team” to monitor
Enables care team members to send reminder messages through app for missed doses		 14 (18.7)
Provide contingent rewards
Can include praise or encouragement as well as material reward. Reward must be explicitly linked to performance of behavior User receives congratulatory message, earns badges, and earns money toward charity for taking medication 6 (8.0)
Provide information on consequences
Providing information on what will happen if the person performs the behavior including the benefits and costs of action or inaction User receives information about the health benefits of taking medication 1 (1.3)
Open in a new tab

Note. BCT = behavior change technique.

Figure 2.

Figure 2.

Open in a new tab

Total number of incorporated BCTs across 75 apps.

Note. *Each app counted in one category based on total number of BCTs included in app.

The five apps selected for usability testing were downloaded and coded for presence/absence of BCTs. Results were compared with app store description coding. Identified BCTs were consistent across both coding methods (app and app store description) reflecting 100% agreement between coding methods for all five apps (Medisafe, MyTherapy, Memo Health, Care4Today, and Ceyhello).

Usability Analysis

Participants completed the uMARS for each of the five apps tested. Following testing, one app (Ceyhello) was deleted from the app store and is no longer available. Therefore, results for usability testing with the remaining four apps are presented. Internal consistency of the uMARS as measured by Crohnbach alpha ranged from good (MyTherapy α = .84) to excellent (Medisafe α = .93; Memo Health α = .96; Care4Today α = .96). Average subscale scores on the uMARS ranged from 3.10 to 4.44 out of a possible 5. Medisafe earned the highest average scores on engagement, functionality, aesthetics, and information subscales (Table II). Post hoc analyses revealed Medisafe scored significantly higher on engagement, functionality, and aesthetics than Care4Today and Memo Health. MyTherapy had the second highest scores across all subscales except for information. Post hoc analyses indicated Medisafe did not score significantly higher than MyTherapy on any subscale.

Table II.

UMARS Subscale Scores

Engagement
 Functionality
 Aesthetics
 Information
M SD M SD M SD M SD
Medisafe 4.10 0.65 4.44 0.57 4.27 0.59 4.40 0.65
MyTherapy 3.64 0.68 4.13 0.61 4.05 0.37 3.84 1.00
Memo Health 3.35 1.05 3.81 0.79 3.55 0.81 3.55 1.33
Care4Today 3.38 0.72 3.63 0.97 3.47 0.74 3.85 1.14
Open in a new tab

Note. uMARS = user version of the Mobile Application Rating Scale.

Medisafe again ranked highest on average overall quality, subjective quality, and perceived impact (Table III). A repeated measures ANOVA with a Greenhouse–Geisser correction determined that the mean scores for overall app quality were significantly different across apps, F(2.75, 49.56) = 7.71, p < .001. Paired samples t-tests with Bonferroni corrections revealed Medisafe earned significantly higher overall quality ratings compared with MyTherapy, t(19) = 3.57, p = .002, Care4Today, t(19) = 4.42, p < .001, and Memo Health, t(19) = 5.15, p < .001. The mean scores for perceived impact were also significantly different across apps, F(3.22, 57.94) = 22.257, p = .001. Medisafe was rated significantly higher on perceived impact compared with Care4Today, t(19) = 3.52, p = .002 and Memo Health, t(19) = 3.83, p = .001.

Table III.

App Rankings According to uMARS Total Scores

M SD
App overall quality
 Medisafe 4.30 0.52
 MyTherapy 3.92 0.50
 Care4Today 3.58 0.83
 Memo Health 3.50 0.88
Subjective quality
 Medisafe 3.90 0.80
 MyTherapy 3.46 0.88
 Care4today 2.94 1.00
 Memo Health 2.54 1.15
Perceived impact
 Medisafe 3.69 0.88
 MyTherapy 3.20 0.93
 Care4today 2.97 1.01
 Memo Health 2.75 1.08
Open in a new tab

Note. uMARS = user version of the Mobile Application Rating Scale.

Updated App Store Review and Content Evaluation

In total, 54 apps identified in the updated search conducted in July 2018 were coded for the presence/absence of 26 BCTs. Similar to our results from the previous content review, the most commonly included BCTs were teach to use prompt/cues (100% of coded apps), prompt specific goal setting (98%), prompt self-monitoring (63%), and provide feedback on performance (46%; Online Table B1). The number of included BCTs per app ranged from 1 to 7 (Figure B2) and the average number of BCTs per app was 3.46 (SD = 1.40).

Discussion

The current study evaluated medication adherence apps based on content (i.e., inclusion of BCTs) and usability, with the overarching goal of identifying the most promising apps for promoting medication adherence in child and young adult users. To our knowledge, this is the first study to systematically review and code medication adherence apps from the US iTunes app store using Abraham and Michie’s taxonomy of BCTs. Our content evaluation revealed only a limited number of BCTs that have been translated to medication adherence apps. Thirty (40%) of apps did not include any additional BCTs beyond those techniques inherent to logging a medication and setting a reminder (i.e., goal-setting and teach to use prompt/cues).

Previous systematic and meta-analytic reviews of adult medication adherence promotion interventions indicated that monitoring with feedback, barrier identification/problem-solving, and providing rewards were significant predictors of intervention effectiveness (Demonceau et al., 2013; Kripalani et al., 2007). In the current study, just under half of the coded apps (n = 37) included a feedback feature that allowed users to view their adherence rates over time. Only 8% (n = 6) of apps provided contingent rewards for taking medication as planned. Examples of rewards included praise for taking medication every day of the week and earning money toward a charity after reaching a designated level of adherence. Barrier identification/problem-solving was not included in any of the 75 apps. Similarly, results from the updated app store search conducted in July 2018 showed that while the volume of available medication adherence apps increased, the number of BCTs and types of BCTs included in apps remained largely the same. Less than half of apps coded from the updated search provided behavioral feedback (46%), only three apps included contingent rewards (6%), and none of the apps addressed barrier identification/problem-solving. While the current study did not evaluate app effectiveness, the lack of these evidence-based BCTs suggests that the majority of currently available apps lack what may be the most salient tools to support health behavior change.

Our findings were consistent with a 2015 content evaluation of medication adherence apps available in the Ireland iTunes store (Morrissey, Corbett, Walsh, & Molloy, 2016). Similarly, Morrisey and colleagues found only a small percentage of apps included BCTs beyond those inherent to entering medications and setting a reminder. Owing to time since the previous review, differing inclusion criteria (i.e., Morissey and colleagues included disease-specific apps), and differences in available content between the US and Ireland app stores, only 30 of the apps coded in the present study were also coded in the previous review.

Our results showed that the majority of apps did not include features beyond medication reminders, and these findings are consistent with the 2016 review by Nguyen and colleagues. However, when comparing the top apps identified, Medisafe was the only app highlighted in both reviews. Our initial content evaluation included seven of the eight top apps in the Nguyen review (CardioLite was selected in the Nguyen review but was no longer available at the time of this review). However, the current study identified five of the most promising apps for usability testing based on number of included BCTs and capability for caregiver sharing/monitoring. These criteria were not applied by Nguyen and colleagues to identify top apps, as their review did not use the BCT taxonomy to evaluate content, which explains the difference in top apps selected. Our review highlights a subset of apps that may be most helpful to patients and families based on the number BCTs used, caregiver sharing/monitoring capability, and usability scores. Several recent studies examined weight management and physical activity apps for the inclusion of evidence-based BCTs (Bardus et al., 2016; Conroy, Yang, & Maher, 2014). Findings indicated a greater variety of BCTs included in weight management/physical activity apps compared with medication adherence apps. For example, weight management apps often include a social comparison component via linked social networks, where users can share their progress (Bardus et al., 2016). Physical activity apps often include videos or instructions modeling target behaviors and/or information about consequences of target behaviors (Conroy et al., 2014). These findings offer promising examples of ways to incorporate more complex BCTs in interactive and interesting ways, which may be translatable to medication adherence apps.

Among adolescents and young adults with IBD who participated in usability testing, there was a clear preference for Medisafe over the other apps tested. Usability testing indicated that evaluating apps only based on the inclusion of evidence-based BCTs is insufficient. Among apps initially identified as comparable based on content (i.e., included same number and types of BCTs), clear user preferences emerged based on perceived quality and usability. Furthermore, users were more likely to rate an app as more likely to bring about behavior change if that app was also perceived as higher quality in terms of aesthetics, engagement, functionality, and information.

Based on Medisafe’s inclusion of five BCTs, easy-to-use sharing/social support feature, and high usability ratings, Medisafe may be the most engaging and useful commercially available medication adherence app for young persons and their caregivers. MyTherapy emerged as the runner up in usability testing. In addition to scheduling medication doses and tracking adherence, MyTherapy enables users to track their symptoms and mood and offers users the option to schedule healthy lifestyle goals and appointments. This app may be preferable for users looking to track and schedule more than just their medications. Importantly, efficacy testing to evaluate the impact of app use on medication adherence is a crucial next step and would provide sound clinical implications for this population.

Of note, one of the apps selected for usability testing, Care4Today, underwent a significant upgrade following usability testing. The app now requires a provider code to share adherence data, and it is unclear whether adherence data can be accessed or shared with caregivers once the code is entered. Therefore, for providers and patients looking for apps that include caregiver monitoring/sharing capability, we recommend considering MyTherapy or Medisafe over Care4Today.

The current study has several limitations. First, based on available devices for usability testing (iPods), our review of medication adherence apps was limited to the iTunes app store. Therefore, we were unable to evaluate medication adherence apps exclusively available on other platforms (e.g., Google Play). For usability testing, we used a convenience sample recruited from the pediatric IBD clinic; therefore, there is a potential for selection bias. Total 75% of the sample was female, and 95% of the sample was Caucasian. Future usability testing with a more balanced sample is needed to investigate differences on app quality ratings and preferences by gender and racial background. Additionally, usability testing was conducted with youth and young adults with IBD, which presents a potential limitation for generalizability. While many factors that impact adherence are consistent across illness groups, distinct treatment factors and barriers to adherence exist for specific populations (Hommel, Ramsey, Rich, & Ryan, 2017). Furthermore, apps designed exclusively for medication tracking may not be as useful for disease populations that require multicomponent treatment regimens (e.g., cystic fibrosis). Therefore, app quality ratings and preferences may differ depending on disease group and usability testing in patients with other chronic conditions is warranted. Time spent testing each app is another limitation, as user ratings may have differed after using the app for an extended period. However, the time spent on each app for the current study was consistent with the duration of testing specified by the uMARS developers. An additional limitation is that the uMARS has not yet been validated in children. This study included 11 participants <18 years of age. Despite this limitation, the uMARS was the only validated usability measure for mobile apps available at the time of the study. Furthermore, the uMARS has been used in previous studies with adolescents as young as 13 years of age (Bakker & Rickard, 2018; Hides et al., 2018). Validation of the uMARS with younger populations would strengthen these and future usability analyses.

While apps represent a widely accessible platform for adherence promotion, the current market is limited in terms of the content incorporated in medication adherence apps. A number of different companies and organizations are involved in developing medication apps, including pharmacies (e.g., Walgreens app), pharmaceutical companies (e.g., Care4Today by Johnson & Johnson), and private health technology companies (e.g., Medisafe and MyTherapy). Broadly, greater collaboration is needed between psychologists and health technologists to incorporate more evidence-based BCTs in apps. Apps developed exclusively for health behavior intervention trials have already begun to incorporate a greater variety of BCTs through innovative methods, such as gamification, integrated motivational support networks, and ecological momentary assessment with real-time intervention (Cafazzo, Casselman, Hamming, Katzman, & Palmert, 2012; Franklin, Waller, Pagliari, & Greene, 2006; Kashdan, Ferssizidis, Collins, & Muraven, 2010). Developers should work with behavioral psychologists to incorporate these and other innovative techniques into commercial apps. Across existing commercial apps, research trials are needed to test their efficacy for improving adherence.

Our findings affirm the need for collaborations between health technologists and behavioral scientists in app development and efficacy testing. However, industry and research operate at vastly different paces and approach development and dissemination in different ways, presenting significant challenges to achieving these goals. Traditional academic models for intervention development and testing progress at a much slower pace compared to the rapid pace of health technology development and dissemination. The speed of health technology development was apparent in findings from the current study. Our updated app store review, which was conducted 2 years after our original search, yielded 168 newly identified apps for medication adherence on the Apple platform alone. To address these challenges, experts in the field are looking beyond traditional randomized controlled trials and adapting alternative research designs. Pagoto and Bennett (2013) highlight several alternative approaches to intervention development, refinement, and efficacy testing, including SMART trials and pragmatic randomized controlled trials, in which data-informed intervention enhancements are made on a rapid timeline throughout the intervention. Similarly, agile science provides a framework for rapid iteration and improvement to streamline the adaption of individualized and user-centered behavior change mechanisms in health technology interventions (Hekler et al., 2016).

User preferences must also be carefully considered in the development process. Our content evaluation did not reveal any apps specifically targeted to child, adolescent, or young adult users, signaling a need for developers to target and involve youth as end users. The subset of apps tested by users was variable in terms of quality of engagement, functionality, aesthetics, and information. Ease-of-use and engagement are particularly crucial to the success of any mobile health application. In light of this, commercial apps should seek to incorporate engagement strategies such as user ability to customize design, individualized feedback, graded tasks (e.g., achieving a new level), and individualized information (Garnett, Crane, West, Brown, & Michie, 2015).

Supplementary Data

Supplementary data can be found at: https://academic.oup.com/jpepsy.

Funding

This work was supported by the National Institutes of Health T32HD068223.

Conflicts of interest: None declared.

Supplementary Material

Supplementary Data
Click here for additional data file. (220.2KB, zip)

References

  1. Abraham C., Michie S. (2008). A taxonomy of behavior change techniques used in interventions. Health Psychology, 27, 379.. [DOI] [PubMed] [Google Scholar]
  2. Bakker D., Rickard N. (2018). Engagement in mobile phone app for self-monitoring of emotional wellbeing predicts changes in mental health: MoodPrism. Journal of Affective Disorders, 227, 432–442. [DOI] [PubMed] [Google Scholar]
  3. Bardus M., van Beurden S. B., Smith J. R., Abraham C. (2016). A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. International Journal of Behavioral Nutrition and Physical Activity, 13, 35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brannon E. E., Cushing C. C. (2015). A systematic review: Is there an app for that? Translational science of pediatric behavior change for physical activity and dietary interventions. Journal of Pediatric Psychology, 40, 373–384. [DOI] [PubMed] [Google Scholar]
  5. Cafazzo J. A., Casselman M., Hamming N., Katzman D. K., Palmert M. R. (2012). Design of an mHealth app for the self-management of adolescent type 1 diabetes: A pilot study. Journal of Medical Internet Research, 14, e70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Conroy D. E., Yang C.-H., Maher J. P. (2014). Behavior change techniques in top-ranked mobile apps for physical activity. American Journal of Preventive Medicine, 46, 649–652. [DOI] [PubMed] [Google Scholar]
  7. Dahlke D. V., Fair K., Hong Y. A., Beaudoin C. E., Pulczinski J., Ory M. G. (2015). Apps seeking theories: Results of a study on the use of health behavior change theories in cancer survivorship mobile apps. JMIR mHealth and uHealth, 3, e31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Dean A. J., Walters J., Hall A. (2010). A systematic review of interventions to enhance medication adherence in children and adolescents with chronic illness. Archives of Disease in Childhood, 95, 717–723. [DOI] [PubMed] [Google Scholar]
  9. Demonceau J., Ruppar T., Kristanto P., Hughes D. A., Fargher E., Kardas P., De Geest S., Dobbels F., Lewek P., Urquhart J., Vrijens B.; ABC project team (2013). Identification and assessment of adherence-enhancing interventions in studies assessing medication adherence through electronically compiled drug dosing histories: A systematic literature review and meta-analysis. Drugs, 73, 545–562. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. DiMatteo M. R. (2004). Variations in patients' adherence to medical recommendations: A quantitative review of 50 years of research. Medical Care, 42, 200–209. [DOI] [PubMed] [Google Scholar]
  11. Fedele D. A., Cushing C. C., Fritz A., Amaro C. M., Ortega A. (2017). Mobile health interventions for improving health outcomes in youth: A meta-analysis. JAMA Pediatrics, 171, 461–469. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Franklin V. L., Waller A., Pagliari C., Greene S. A. (2006). A randomized controlled trial of Sweet Talk, a text‐messaging system to support young people with diabetes. Diabetic Medicine, 23, 1332–1338. [DOI] [PubMed] [Google Scholar]
  13. Garnett C., Crane D., West R., Brown J., Michie S. (2015). Identification of behavior change techniques and engagement strategies to design a smartphone app to reduce alcohol consumption using a formal consensus method. JMIR mHealth and uHealth, 3, e73. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Graves M. M., Roberts M. C., Rapoff M., Boyer A. (2010). The efficacy of adherence interventions for chronically ill children: A meta-analytic review. Journal of Pediatric Psychology, 35, 368–382. [DOI] [PubMed] [Google Scholar]
  15. Hanghøj S., Boisen K. A. (2014). Self-reported barriers to medication adherence among chronically ill adolescents: A systematic review. Journal of Adolescent Health, 54, 121–138. [DOI] [PubMed] [Google Scholar]
  16. Hekler E. B., Klasnja P., Riley W. T., Buman M. P., Huberty J., Rivera D. E., Martin C. A. (2016). Agile science: Creating useful products for behavior change in the real world. Translational Behavioral Medicine, 6, 317–328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hides L., Quinn C., Cockshaw W., Stoyanov S., Zelenko O., Johnson D., Tjondronegoro D., Quek L. H., Kavanagh D. J. (2018). Efficacy and outcomes of a mobile app targeting alcohol use in young people. Addictive Behaviors, 77, 89–95. [DOI] [PubMed] [Google Scholar]
  18. Hommel K. A., Baldassano R. N. (2010). Brief report: Barriers to treatment adherence in pediatric inflammatory bowel disease. Journal of Pediatric Psychology, 35, 1005–1010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hommel K. A., Ramsey R. R., Rich K. L., Ryan J. L. (2017). Adherence to pediatric treatment regimens In Roberts M. C., Steele R. G. (Eds.), Handbook of Pediatric Psychology Fifth Edition (pp. 119–133). New York, NY: The Guilford Press. [Google Scholar]
  20. Ingerski L. M., Baldassano R. N., Denson L. A., Hommel K. A. (2010). Barriers to oral medication adherence for adolescents with inflammatory bowel disease. Journal of Pediatric Psychology, 35, 683–691. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kahana S., Drotar D., Frazier T. (2008). Meta-analysis of psychological interventions to promote adherence to treatment in pediatric chronic health conditions. Journal of Pediatric Psychology, 33, 590–611. [DOI] [PubMed] [Google Scholar]
  22. Kashdan T. B., Ferssizidis P., Collins R. L., Muraven M. (2010). Emotion differentiation as resilience against excessive alcohol use: An ecological momentary assessment in underage social drinkers. Psychological Science, 21, 1341–1347. [DOI] [PubMed] [Google Scholar]
  23. Kripalani S., Yao X., Haynes R. B. (2007). Interventions to enhance medication adherence in chronic medical conditions. A systematic review. Archives of Internal Medicine, 167, 540–549. [DOI] [PubMed] [Google Scholar]
  24. Mani M., Kavanagh D. J., Hides L., Stoyanov S. R. (2015). Review and evaluation of mindfulness-based iPhone apps. JMIR mHealth and uHealth, 3, e82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. McGrady M., Ryan J., Gutiérrez‐Colina A., Fredericks E., Towner E., Pai A. (2015). The impact of effective paediatric adherence promotion interventions: Systematic review and meta‐analysis. Child: Care, Health and Development, 41, 789–802. [DOI] [PubMed] [Google Scholar]
  26. McGrady M. E., Hommel K. A. (2013). Medication adherence and health care utilization in pediatric chronic illness: A systematic review. Pediatrics, 132, 730–740. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Morrissey E. C., Corbett T. K., Walsh J. C., Molloy G. J. (2016). Behavior change techniques in apps for medication adherence: A content analysis. American Journal of Preventive Medicine, 50, e143–e146. [DOI] [PubMed] [Google Scholar]
  28. Morrison D., Wyke S., Agur K., Cameron E. J., Docking R. I., MacKenzie A. M., McConnachie A., Raghuvir V., Thomson N. C., Mair F. S. (2014). Digital asthma self-management interventions: A systematic review. Journal of Medical Internet Research, 16, e51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Nguyen E., Bugno L., Kandah C., Plevinsky J., Poulopoulos N., Wojtowicz A., Schneider K. L., Greenley R. N. (2016). Is there a good app for that? Evaluating m-Health apps for strategies that promote pediatric medication adherence. Telemedicine and e-Health, 22, 929–937. [DOI] [PubMed] [Google Scholar]
  30. Pagoto S., Bennett G. G. (2013). How behavioral science can advance digital health. Translational Behavioral Medicine, 3, 271–276. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Pai A. L., McGrady M. (2014). Systematic review and meta-analysis of psychological interventions to promote treatment adherence in children, adolescents, and young adults with chronic illness. Journal of Pediatric Psychology, 39, 918–931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Pew Research Center. (2017). Mobile fact sheet. Retrieved from http://www.pewinternet.org/fact-sheet/mobile/.
  33. Rapoff M. A. (2009). Adherence to pediatric medical regimens. Germany: Springer Science & Business Media. [Google Scholar]
  34. Rapoff M. A. (2010). Consequences of nonadherence and correlates of adherence Adherence to Pediatric Medical Regimens (pp. 33–45). Germany: Springer. [Google Scholar]
  35. Robinson A., Hankins M., Wiseman G., Jones M. (2013). Maintaining stable symptom control in inflammatory bowel disease: A retrospective analysis of adherence, medication switches and the risk of relapse. Alimentary Pharmacology and Therapeutics, 38, 531–538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Santer M., Ring N., Yardley L., Geraghty A. W., Wyke S. (2014). Treatment non-adherence in pediatric long-term medical conditions: Systematic review and synthesis of qualitative studies of caregivers’ views. BMC Pediatrics, 14, 63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Stoyanov S. R., Hides L., Kavanagh D. J., Wilson H. (2016). Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR mHealth and uHealth, 4, e72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Stoyanov S. R., Hides L., Kavanagh D. J., Zelenko O., Tjondronegoro D., Mani M. (2015). Mobile app rating scale: A new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth, 3, e27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. van Genugten L., Dusseldorp E., Webb T. L., van Empelen P. (2016). Which combinations of techniques and modes of delivery in internet-based interventions effectively change health behavior? A meta-analysis. Journal of Medical Internet Research, 18, e155. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Yang C.-H., Maher J. P., Conroy D. E. (2015). Implementation of behavior change techniques in mobile applications for physical activity. American Journal of Preventive Medicine, 48, 452–455. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
Click here for additional data file. (220.2KB, zip)

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES