A Scoping Review and General User's Guide for Facilitating the Successful Use of eHealth Programs for Diabetes in Clinical Care

Abstract

The vast eHealth literature in diabetes can provide a useful foundation to aid in the selection, adoption, and implementation of eHealth methodologies in clinical care. Despite clear potential to enhance reach, efficiency, and clinical effectiveness, research has yielded mixed and often contradictory results, and wide-spread adoption and maintenance of eHealth programs in clinical care has been limited. Furthermore, few reports have identified the unique challenges that clinicians and health systems face when attempting to incorporate eHealth systems into clinical care. To address these gaps, we address two goals in this report: first, to summarize and integrate the major findings of the diabetes-related eHealth literature based on currently available systematic and narrative reviews; and second, based on the review, to provide practical guidelines to assist clinicians and health systems in selecting and implementing eHealth programs into diabetes care using dissemination and implementation science principles and perspectives.

Background

Adults with chronic disease, such as diabetes, make ∼98% of their disease management decisions outside of the clinical care setting.¹ One goal of self-management support (SMS) is to provide these individuals with the knowledge, resources, support, and tools to make informed, evidence-based decisions that are consistent with their personal priorities and that contribute to good health and quality of life over time.^2,3 Fueled by efforts to better engage patients and recent changes in reimbursement policies, there has been a dramatic increase in interest in using new technologies to bring SMS into where patients live, work, socialize, and recreate. What has emerged through the use of smart phones and other digital mobile devices is the development of literally thousands of “apps” and other mobile technologies to aid in this process.^4,5 The COVID-19 epidemic has rapidly expanded the need to make further use of SMS using mobile technologies.

Some mobile technologies link users to their health care teams (HCTs), whereas the majority are free-standing systems that operate independently of health systems: some focus on a single medium of communication (e.g., text messages) or one management target (e.g., steps while walking), whereas others utilize multiple methods of communication and address a variety of management and behavioral health issues.⁴ Thus, the complexity of the technology, the device, and the management targets vary considerably and require different levels of user, clinician and health system digital savvy, and experience. We will use the term “eHealth” to capture this broad area of digital, mobile, and remote disease management technology.

Despite the explosion of both free and proprietary eHealth systems and apps, evidence for their clinical effectiveness, usability, wide-spread adoption, and maintenance of use over time has yielded mixed results.^5,6 Several systematic overviews of this literature have identified major strengths and weaknesses of current eHealth systems for adults with diabetes and other chronic conditions^7,8; but very few have highlighted the unique challenges that clinicians and health systems face when attempting to incorporate them into clinical care. These challenges include not only deciding which, if any, of these technologies should be considered but also exploring if a specific eHealth program can be practically adopted, implemented, and maintained within a specific clinical setting.⁹

To aid in this decision-making process, we address two goals in this report: first, to summarize and integrate the major findings of the eHealth literature based on currently available systematic and narrative reviews; and second, based on the review, to provide practical guidelines to assist clinicians and health systems in selecting and implementing eHealth into care using dissemination and implementation science principles and perspectives.^10,11 We address these goals by asking three questions. First, what are the most frequently used eHealth modalities and the disease-related targets they address? Second, what do past evaluations and reviews of these programs suggest will yield the “biggest bang for the buck” for patients and practitioners in terms of outcomes: effective, safe, fast, patient-centered, equitable, and cost-efficient methodologies¹²? Third, what are the critical challenges that practitioners, clinics, and health systems must address when selecting, adopting, implementing, and maintaining eHealth systems within the clinical setting over time? Although we focus on SMS for diabetes, the discussion presented below also can be salient for other chronic conditions.

Review Strategy

Given the numerous meta-analyses, reviews, and reviews of reviews in the literature over the last 10 years, we used a pragmatic approach to summarizing the literature, called “realist evaluation.”¹³ Rather than focus on a meta-analytic strategy, which primarily evaluates the overall average effect of a specific intervention on one specific outcome, this approach is a practical and applied tool to assist policy and decision makers in summarizing studies that take into account issues of context and setting on a variety of interventions that address multiple outcomes.¹⁴ The fundamental goal of a “realist evaluation” is to assess which interventions work in a given context or set of contexts to answer a specific question relevant to stakeholders. In this case, the key questions are as follows: how effective are eHealth systems for which patient populations, and what health care system contextual factors and priorities need to be considered when selecting a program? We therefore surveyed the literature through PubMed and other search engines using a variety of search terms (e.g., eHealth, mHealth, mobile technologies, diabetes mellitus) to identify technologies that focused on diabetes management and related conditions and targets. Titles and abstracts from 2015 to 2020, covering original studies from 1995 to 2020, were scanned. Fifty-three articles were identified, covering from 6 to 44 studies per article. Each was reviewed and summarized with respect to effectiveness, attrition, cost, clinical or behavioral target, modality, channel of communication, identified problems and benefits, and program characteristics. Overriding themes and conclusions were then abstracted and integrated for each content area to form the basis of this “realist evaluation.” Where contradictions among articles existed, further reviews of the contributing articles were undertaken to form consensus conclusions, taking variations among samples, methods, and approaches into account.

eHealth Definitions and Technologies

The U.S. Food and Drug Administration defines eHealth as, “the delivery of health services and improvements in health outcomes via mobile and wireless devices.”¹⁵ Others have defined eHealth more specifically as “software programs designed for mobile patient phones, tablets, and desk-top computers that aim to promote or support self-management skills to manage key disease markers and symptoms.”⁵ In general, eHealth systems collect or deliver user information, serve as a medium of communication between HCTs and users, or provide suggestions/recommendations based on user-provided information. In this sense, eHealth programs are not interventions in and of themselves; instead they are clinical tools, an extension of care, or a modality of care delivery.¹⁶ To limit the scope of the presentation, we distinguish between eHealth tools and “telemedicine,” which we define as remote clinical encounters similar to in-clinic visits, usually through audio and video linkages.¹⁶ To set the stage for this discussion, in Figure 1, we list the variety of behavioral targets generally addressed by eHealth in diabetes and related chronic conditions, the various communication channels used (one-way and two-way), and the several kinds of eHealth media and technologies utilized. A review of Figure 1 illustrates the heterogeneity and complexity of both the various modalities of eHealth delivery and the management targets in diabetes. Although the most commonly used eHealth modalities include text messaging alerts and reminders that focus on education, motivation/engagement, and behavioral self-monitoring (diet, exercise, blood glucose levels),^17–19 it is apparent that eHealth does not reflect a single approach. The only commonality among the diverse eHealth programs is the use of technologies applied outside of the clinical setting to assist users in better managing their diabetes.²⁰

FIG. 1.

Overview of eHealth approaches to diabetes management. a, mobile health, app-based; b, artificial intelligence; c, self-management support; d, web-based; e, health-system based; HCT, health care team; EHR, electronic health record; SMS, self-management support. Color images are available online.

This diversity also highlights some of the difficulties in which clinicians and health systems find themselves when trying to decide which types of eHealth programs might be best; that is, the “ultimate, pragmatic use question”—which modalities work best with which clinical or behavioral targets for which patients when delivered under which conditions, at what cost; and how/why do these results come about.^21,22 For example, the diversity of eHealth programs makes it difficult to construct a grid with clinical targets along one dimension and eHealth modalities along the other to assess the relative effectiveness and applicability of different combinations. Furthermore, many programs utilize multiple modalities that are directed at multiple clinical targets, making efforts to evaluate effectiveness and to aid in decision-making even more difficult.²³ Also, some programs address proximal outcome targets, such as exercise and calories consumed per day, whereas others address relatively more distal targets, such as weight loss maintenance over time or glycemic control, raising issues of timing, length of program, and use of additional required staffing or IT support over time.²³ Hence, although all can be classified under the category of eHealth, given current definitions, they constitute a highly diverse set of programs and methodologies that defies a simple set of categories that can be used to assess which eHealth platform is optimal to adopt and implement for a given clinical problem, clinic, and patient population.

Despite this complexity, the potential of eHealth to improve outcomes, especially for select populations in clearly defined domains of care, is undeniable. The crucial questions for health systems and HCTs are how effective are eHealth systems for which patient populations; and what health care system contextual factors and priorities need to be considered when selecting a program? We address these questions by summarizing the essential findings from multiple narrative and meta-analytic reviews in bulleted, user-friendly form in Tables 1 through 4. The findings are summarized and integrated in the text below. We conclude by posing 10 crucial questions in Table 5 to serve as a user guide for HCTs and health systems when selecting and implementing eHealth programs.

Table 1.

Potential eHealth Benefits And Problems

Potential benefits2,17,23,26:

▪ Increases reach to vulnerable, geographically dispersed, rural, underserved patients with less access to care, lower health literacy, single parents, less educated, minorities, immigrants.

▪ Enhances health equity.

▪ Extends clinical care to everyday world and natural environment: an expanded interface between HCT and patients.

▪ Can provide immediate, real-time, useable management feedback.

▪ Guides patient management and decision making.

▪ Assists in management problem solving.

▪ Provides information to enhance HCT and user decision support.

▪ Some potential to lower costs for health systems, patients, or both, or to provide for a good clinical return for investment.

▪ Provides real-world data to both HCTs and patients so that in-office visits can be focused.

▪ Provides more opportunities for panel management, with linkages to other health systems and resources.

Potential problems4,20,23,27,61:

▪ High attrition: high refusal to participate and high subsequent program drop-out rate.

▪ Low adherence: many users remain in the program but engage infrequently.

▪ Problems with data security and privacy, with a potential for eHealth interfaces with EHRs to open up access to hackers.

▪ Too much data collected in forms not easily obtainable and useable by HCTs and patients.

▪ Problems with both HCT and patient usability and user-friendliness: technologically complex, with too many whistles and bells.

▪ Best results require customization and ongoing adaptation for both HCTs and patients: user literacy, numeracy, culture, education, age, gender, and technological savvy of end-users are rarely considered.

▪ Variable accuracy of measurement tools: the validity of the tools used to provide feedback to users (carb estimators, physical activity trackers) display poor validity or accuracy compared with gold standard assessment tools.

▪ Lack of clarity of specific short- and long-term clinical objectives, including both proximal and distal clinical outcomes.

▪ Often do not include needed end-user training for both health system and patients.

▪ Lack of user input on the multiple perspectives needed for meaningful program development.

▪ Although the best outcomes occur when the program utilizes multiple media, this often increases cost and complexity.

▪ Variations in eHealth programs use can unintentionally increase health inequities.

▪ eHealth interventions often are unsustainable within health systems because of a lack of clear planning, targeting and integration within health systems operations.

▪ Health systems can become overly dependent upon external, proprietary systems such that the costs incurred in switching or modifying the program become prohibitive.

▪ Difficulty in integrating eHealth data with other data management systems, for example, EHR, public health.

EHR, electronic health record; HCT, health care team.

Table 5.

An eHealth User's Guide: 10 Key eHealth Questions

Question	Answers, options and considerations
1. Which eHealth modality and clinical target is best for us?	eHealth modalities are only clinical tools or modes of care extension—their use is not a goal in and of itself, even though they can be seductively attractive. Actively consider the answers to five subquestions:
	a. What is the clinical or behavioral problem or target that needs to be addressed?
	b. What patient population(s) is of interest?
	c. What aspects of clinic staffing, culture, patient flow, and competing demands need to be considered?
	d. How will expected costs and potential reimbursement drive goal attainment and return on investment?
	e. How to get input from all stakeholders regarding which eHealth modality and clinical/behavioral targets best address the issues raised by questions “a” through “d” above?
2. What are the major benefits of a particular eHealth programs that make it attractive for use in clinical care?	The major benefits of eHealth systems include their potential to reach geographically dispersed, underserved patients, those struggling to meet clinical goals with routine visit frequency, and those who find it difficult to come for care (especially post-COVID). They also can provide effective strategies for patient education, self-management training, peer support, live and algorithm-based, real-time feedback, and assistance for day-to-day disease management and problem solving-for patients and HCTs.
3. Can I apply the published eHealth efficacy data to my clinic population to justify its use?	Whatever the reported research data, they may not apply to your clinic! Because most outcome data are generated from white, educated adults from high income countries, in highly controlled research studies, they may not generalize to your clinic and patient population. Use existing outcomes (or effectiveness) data as a starting point, but then anticipate that the eHealth system will have to be customized for your specific, targeted patient groups, accounting their diversity, literacy, numeracy, beliefs, skills, social determinants challenges, and values.
4. How much improvement in clinical and behavioral targets can I expect when applying an eHealth system?	Support for the effectiveness of eHealth systems is generally good, but there are many exceptions and qualifications to consider. Although statistically significant improvements in clinical targets with eHealth systems have been documented, it is important to decide whether or not the reported statistical improvements are sufficiently clinically meaningful and equitable across different targeted patient groups in your setting.
5. How much HCT and staff involvement, should be included to maximize benefits?	eHealth systems linked to HCTs generally show higher levels of usability and effectiveness than free-standing systems not linked to HCTs. Generally, there appears to be a positive association between the number of contacts between HCTs and users, use of the eHealth system and change in outcomes, although majority of these data are correlational, not experimental.
6. How much training and support will patients and staff require?	Evaluate training needs carefully. Patient access and training for eHealth systems and the availability of real-time technical support are crucial, especially when the eHealth system includes multiple modalities or targets. Training needs depend on the complexity of the system and the preferences and needs of the targeted patient population.
7. What kinds of clinical and behavioral problems or issues are best targeted for change using eHealth systems?	Best are short- or intermediate-term, highly specific behavioral targets, like medication taking, carbohydrates consumed, or steps completed. Long-term targets, such as HbA1C, number of comorbidities/complications, or hospital days are affected by many additional causative factors and may take a far longer time to assess accurately, thus increasing costs and allowing for the input of other complicating factors.
8. How should I consider costs, benefits, and potential reimbursements when deciding upon an eHealth program?	▪ Be realistic: eHealth programs often have good cost-value ratios, but few actually save money by switching existing in-clinic services to eHealth methods. Rapidly emerging and changing reimbursement options are providing more opportunity by offsetting the costs of eHealth programs, but these vary considerably based on the staff involved, the patient's payment system and the content of the action taken.
	▪ Definitive cost analyses are difficult to do accurately as they depend upon quality data for the time spent implementing and sustaining programs, and clear specification of which stakeholder perspectives are included.
	▪ Clearly articulate the system's business case for the modalities, clinical and behavioral targets and patient populations that will be targeted.
	▪ Make sure to fully estimate costs for staff training, patient recruitment and training, integration with existing information technology, program oversight, ongoing support and supervision, disruptions to patient flow, ongoing technical assistance, credentialing, opportunity costs, and potential liability, in addition to the costs incurred by either developing or purchasing an existing system.
	▪ Be sure to consider the time frame being used to estimate costs, benefits, and reimbursements.
9. How do I make sure that my patients will actually use the eHealth system we use?	Consider explicitly: what is the right information needed at the right time for the right user? Have patients test it out in a pilot phase and ask for their suggestions for the eHealth program's design and implementation. Assume you will need to use initially collected data to adapt the program to changing context. Typical reasons for poor use include: user lack of technological savvy, low literacy or numeracy, too many bells and whistles (keep it focused), lack of contact/feedback from HCTs, goals not commensurate with patients' needs (do they see the eHealth target as a problem), values and interests, real-time tech. support., etc.
10. What do I need to consider for the successful adoption, implementation and maintenance of an eHealth system within the clinical setting?	Clinics and health care systems rarely consider the staff time and the investment required to gain and maintain stakeholder support and engagement. Factors that need to be determined up-front by health systems include:
	▪ Regarding staff: “who does what” for the full workflow of eHealth data collection, response, documentation, etc.
	▪ How much initial training and ongoing supervision will these staff need
	▪ How will eHealth data be integrated with existing technical infrastructure
	▪ HCT information sharing
	▪ Address issues of data privacy and security
	▪ Integration with clinic culture and mission
	▪ System maintenance and periodic upgrades
	Useful eHealth systems require full integration within the structures, culture and processes of clinic operations; and ideally coordination and collaboration with other community and public health resources.

The Effectiveness of eHealth Technologies

As indicated in Table 1, reviews of the effectiveness of eHealth programs indicate several recurring methodological and analytical problems that hinder informed selection of programs. These include the often poor reliability and validity of eHealth clinical and behavioral target outcome measures, making it difficult to know when program goals have been achieved; the limited data comparing the larger number of nontheory-based eHealth programs to theory-based eHealth programs; the lack of clear demonstration, where appropriate, that eHealth programs are as effective or more effective than similar programs delivered within the clinic or other settings; the lack of clarity and generalizability of specific eHealth programs for specific kinds of health systems and patient populations; the failure to accurately assess the costs of installing and implementing eHealth solutions; the lack of data on the representativeness of health systems, clinicians and patients included in evaluating eHealth programs; and the lack of evidence that the improvements promised by an eHealth program are not only statistically significant but also clinically meaningful and sustainable. (We use the word “efficacy” to denote studies that assess highly specific interventions with individual outcome targets evaluated in highly controlled research conducted in optimal conditions. In contrast, we use the word “effectiveness” to refer to studies that address more practical, real-world applications of eHealth programs in more representative situations.)

Furthermore, most eHealth technologies have been evaluated predominantly among white, educated, middle class adults in high income-countries,^19,23–25 and studies of their appropriateness for use with geographically dispersed, high risk, culturally diverse, or underserved populations to address health equity and social determinants of health are lacking.^2,23,26,27 These significant gaps need to be considered when evaluating the results of the extant literature and applying them to specific health care settings and patient populations.

Although the findings are not consistent across reviews, the overall literature suggests that eHealth programs are at least modestly effective across a range of self-care behaviors, such as medication taking and glycemic control assessed by HbA1C.^{5,8,17,19,28–39} When reviewing the results targeting improvements in HbA1C, medication adherence, and weight loss, however, results seem best only in the short term, or when very poorly adherent users are excluded, when multimodality programs are used, and when there is a significant focus on user motivation and engagement over time.^{5,29,30,34,37} An additional issue concerns the selection of proximal versus relatively more distal outcomes, such as HbA1C change. For example, targeting improved insulin dosing, a proximal goal, should affect HbA1C in the long run, but the impact of improved dosing to HbA1C change certainly is not 1:1—that is, other factors undoubtedly come into play. Consequently, it may be helpful to target a specific set of proximal behaviors for change, rather than to target the relatively more distal effects of that behavior change on more distal indices such as HbA1C.

Interestingly, there are very few eHealth studies that address smoking, blood pressure, diabetes distress and depression, anxiety, or other behavioral health issues. The focus tends to be on more general diabetes-related “clinical” targets without reference to specific self-care behaviors, mental health, quality of life, or social determinants of health issues.

Several additional caveats are worthy of note (Table 2). First, there is nothing to suggest that any single type of eHealth technology or combination of technologies is best for achieving a particular clinical goal.²³ Although text messaging might be most helpful for simple, one-way communication between HCTs and users, no combination of text messaging, monitoring, or algorithmic technology should be considered standard or most effective to achieve a particular clinical goal. Thus, it appears that effectiveness depends less on the type, modality, or form of an eHealth system selected for use and more on the function of how best to achieve a clinical or behavior goal, considering both user and health system contextual factors.⁴⁰ It may be best, therefore, for decision makers to begin by identifying a specific goal, issue, or concern and proceed from there, rather than starting with the selection of a seductively attractive eHealth modality or program with multiple whistles and bells.

Table 2.

Effectiveness of eHealth Interventions: Fact Sheet

Overall, the effectiveness of eHealth programs is moderate, but there is considerable variability across studies.8,17,28–33,36,38,39,62

Program characteristics related to effectiveness:

▪ Studies can be hard to evaluate because of their use of different eHealth modalities, clinical targets and patient populations. Study sample sizes are small, and study quality is often low, with little information about the accuracy of the information collected; overall program effects and the fact that program components are rarely assessed, only total program effects.8,23,34,37,46

▪ Use of eHealth systems that only monitor behavior or blood glucose levels are less helpful than systems that provide live or algorithm-based feedback that is actionable.33,63

▪ Immediate/timely feedback is best.2,63

▪ Built-in incentives for use, goal attainment, and “gamification” work best only in short term, if at all.43,61

▪ To optimize ease of use, manual input of data should be minimized; instead fostering automated input and linkages to other devices, for example, CGM, accelerometers.46,64

▪ There are mixed results on whether or not use of behavior change theory makes a difference with respect to outcomes.34,49

▪ How long a program should last is unclear—it may depend on when a change in the clinical target can reasonably be expected; longer-term follow-up in studies is infrequent, so proximal goals are most frequently targeted.7,28,39,46,47,56

▪ There is a need for clearly defined and assessed clinical targets37,38—that is, what is the specific, accurately measured behavioral goal—what does the program ask the user to do differently?

▪ Many studies show statistically significant between-group differences even though the differences may not be clinically meaningful for the entire sample or for specific, often at-risk patient subgroups.35,50

Effectiveness using different eHealth modalities

▪ Neither a single eHealth modality nor the sequence of use of eHealth modalities has been shown to be more or less “effective.”17

▪ Use of systems with multiple types of communication channels and components is best, while keeping complexity minimized—allows for user customization and choice.17,33,34,64,65

Effectiveness using free-standing vs. linked programs

▪ Best outcomes are with integrated clinician/user contact,37,38 but the exact number of contacts and their frequency has not been demonstrated for specific eHealth systems—this may be patient/clinic/goal-dependent.5,39

▪ Systems with active clinician involvement have been shown to be more helpful than free-standing systems and user-only programs34,38,46,47,51,63,66,67; although there may be differences in costs, there are little data to suggest whether different kinds of clinicians, for example, nurses, educators, physicians, affect eHealth efficacy differently; this too may be clinic/patient dependent.

▪ Although patient selection bias may be operative, there are some data to suggest that, up to a maximum, more frequent HCT/patient contact through the eHealth system leads to less attrition and better outcomes.20,24,34,35

▪ Some data suggest the utility of inserting remote diabetes monitoring data directly into the EHR through the cloud. Unfortunately, there have been very few published evaluations of this tool and the range and type of data reported to date have been very limited.

Effectiveness related to user issues

▪ Findings from the eHealth literature are limited by the lack of diverse samples included; results generally indicate that the best eHealth outcomes occur among users with high baseline HbA1c who are younger and middle age, so that others might well need additional support and assistance.37,38

▪ eHealth systems may have fewer benefits in advanced health systems where patient/clinician contact and communication are already extensive.23,26,56

▪ It is important to include HCT and patient training in system use.2,23

▪ Context matters and it is multilevel: effective eHealth programs focus on user needs, skills, culture, experience with technology, and literacy. Likewise, clinic context plays a big role: public vs. private health systems, leadership, culture, clinical style, competing clinical priorities, and experience with technology are major contextual factors.2

Effectiveness regarding specific eHealth targets

HbA1c:

▪ Results are mixed: only about half of studies show significant HbA1c reductions with eHealth interventions.8,25,68

▪ Largest HbA1c reductions occur when baseline HbA1c levels are >8.0%.35,38

▪ Reductions are often statistically significant but not necessarily clinically meaningful.35

▪ Best results occur with multicomponent interventions that include education, monitoring and feedback.34,35

▪ The inclusion of peer support does not necessarily lead to greater HbA1c improvement when using eHealth systems.46,63,68

Medication use:

▪ Results of eHealth interventions are modest at best and usually include adults with type 2 diabetes (oral hypoglycemics, lipid-lowering or antihypertensive medications.29

▪ Positive results tend to be short-term.29

▪ Best approaches tend to be through text reminders or use of IVR two-way communication systems.69

Obesity and weight loss:

▪ Results of eHealth weight loss interventions are poor.24,70

▪ Best results tend to be around short-term weight loss; results for maintenance of weight loss over time are poor.31,47,51

▪ Major problems are maintaining good user engagement, many studies include highly diverse user samples, whose variability in response makes evaluation difficult.31,51,70

Second, a clear and unambiguous finding from this literature is that eHealth technologies linked to clinical care tend to be more effective than both free-standing programs with no other “live” contact or to free-standing programs that provide contact with external vendor staff or “coaches.” The essential message is that effectiveness is increased when eHealth-care is seen by the user as part of the care delivered by their HCT—as an extension of existing care.

Third, congruent with behavior change theories and principles,⁴¹ the following eHealth program characteristics are generally associated with greater program effectiveness: delivery of timely, actionable feedback; use of automated (as opposed to manual) data uploads from wearables and related self-monitoring devices; and inclusion of clearly defined, mutually agreed to clinical and behavioral goals. Motivational cues or behavioral “nudges”⁴² provided in a game-like context (gamification) also may be helpful. This sometimes includes financial incentives or “points” to end-users to increase motivation for continued use. However, most studies show that their effects tend to be significant only in the short term.⁴³ Moreover, the most effective programs include implementation strategies that address user context, needs, skills, literacy, and culture, along with appropriate user training and easily accessible technical assistance and support.^29,44

Fourth, most studies compare the use of an eHealth system with usual care. Rarely do studies focus on comparing different kinds of eHealth modalities or differently configured programs for a given clinical target, or by comparing a given clinical activity delivered through eHealth versus one delivered within a clinic or other setting. Finally, a troubling finding is the lack of customization of eHealth systems in terms of complexity, frequency of contact, and so on for use with diverse patient groups with different needs. A one-size-fits-all approach may simplify research designs, but it also may make selection and effective utilization of an eHealth program in a specific setting more difficult.

The Special Problem of Attrition in eHealth Programs

One of the most serious problems concerning both the effectiveness and utility of eHealth programs is attrition over time (Table 3). Rates of attrition vary from report to report, but are often well over 50%, with most users utilizing the technology initially, but then stopping use after only a few days or weeks. Even the definition of attrition can be confusing and published rates may severely overestimate actual use over time. For example, it is not uncommon to find that a large number of potential users decline to try an eHealth program to begin with, and so are not included in attrition statistics; or they begin the program but abandon it quickly, or they stop using the program when the pilot project has been completed, overestimating sustainability statistics.²⁰

Table 3.

Attrition in eHealth Interventions: Fact Sheet

General considerations

▪ General rates of attrition over time are very high: between 5% and 64% in most target areas.5,20,27,30,46,61,68,69,71,72

▪ Usually a large dropout rate occurs initially, with about 25% more dropping out each month: figures do not include those who refuse participation at the outset, those who agree but who never use the program and those who stop use after the program ends.20,73

▪ The primary reasons for attrition are lack of interest, technical problems, and those who feel that they did not need it to begin with.20

Predictors and correlates of attrition

▪ User demographics, clinical characteristics and psychological factors generally do not predict dropout38,69; some data suggest that Hispanics and those with low education drop out more frequently, but the findings are not consistent and may be related to specific eHealth programs and targets.20

▪ The primary associations with dropout rate are issues linked with user engagement, beliefs, and values,5,49–51,63 dropping out may be related to expectations that the rate of improvement is viewed as lower than expected.51,63

▪ Users with relatively lower dropout rates tend to be: women, older users, users with higher education and income, and users with a strong belief in the severity and importance of their health condition.20

Attrition in specific target areas

Obesity and weight loss:

▪ Exceptionally high attrition—upwards of 80%.31,48,51,72

▪ Long term participation to reach goals is minimal.43

Medication adherence (including blood glucose monitoring):

▪ High rates of attrition—over 40% during first several months.37,49

▪ Repeated documentation of increasingly poor user engagement over time.50

Mood, anxiety, depression:

▪ High rates of attrition: 90% attrition for an anxiety-based intervention70; 13% to 41.6% attrition for depression interventions.31

▪ For these kinds of eHealth programs, many potentially eligible users refuse to sign up initially.73

▪ Contributing factors to attrition in mood interventions are concerns about suspicion, distrust, confidentiality, and privacy.73

These low use rates are relatively consistent across different clinical targets, and eHealth modalities, but are especially high in weight loss programs with rates reaching as high as 80% over time.^45–48 It remains unclear, however, if specific user characteristics are associated with attrition: some reviews indicate that user demographics and disease-related characteristics are not significantly associated with attrition, whereas other reports suggest that select user characteristics, such as age and gender, are. Also, the degree to which eHealth is delivered as part of standard clinical practice or is offered as “optional” influence rates of attrition; that is, most studies do not evaluate an eHealth program that is presented as a standard part of clinical practice, instead only including patients who selectively opt-in. In these ways attrition rates reported in the literature are only partial and may not provide clinical decision makers with good estimates of the potential impact of an eHealth program for their overall patient panel.

The reviews clearly indicate that attrition may be reduced by crafting programs for specific groups of users that address their values, beliefs, needs, and motivations.^49–51 For example, one health system may target adolescents with type 1 diabetes who tend to be very tech-savvy, in contrast to another health system that wishes to use the same system with older individuals with poorer technology skills. Each user group has different needs, priorities, and abilities, thus requiring very different forms of eHealth technologies and support. These differences highlight the need to engage potential end-users in the process of eHealth selection, development, and implementation to maximize outcomes, increase usability, and reduce attrition over time. Likewise, it may be best not to consider “attrition” as a single concept or number; rather to divide the problem into more definable, manageable targets to identify specific groups of users who might benefit from a eHealth program; recruit them in patient-tailored ways that will maximize their participation; enhance continued use over time through patient-tailored engagement, motivation, and clinical contact; and engage with users to set criteria as to when continued use is no longer necessary.

Clinic/Heath System Issues in the Selection and Implementation of eHealth Programs

Four themes emerge from the literature that links the context of the clinical setting with its ability to adopt and implement eHealth programs successfully: staffing, data management and infrastructure, systems operation and culture, and costs (Table 4). Detailed planning and ongoing oversight of eHealth programs is a basic, but often neglected, requirement. Such activities need to include clinical skill building, the provision of adequate staff time, supervision, documentation, integration into patient flow, and sustainable reimbursement levels. As with any clinic function, training, supervision, and case-based oversight should continue over time to assure effective and efficient use of eHealth programs. Assuming that a new eHealth system, once introduced, even with initial training and support, will continue to run by itself or be consistently effective over time without continuous monitoring and adaptation is short-sighted and unrealistic.

Table 4.

Clinic/Health System Considerations: Fact Sheet

Staffing issues

▪ Comprehensive and ongoing staff training is often required, including ongoing supervision, a careful delineation of staff roles and oversight.23,29,49 For example, who will respond to emergency alerts from downloaded CGM data.23

▪ To what extent will the demands of the program divert existing staff from other responsibilities to affect ongoing patient flow issues.2,23

▪ Will clinical contacts with users occur with existing staff or with staff external to the clinic (e.g., coaches).4

▪ Added staff burden can occur because of the need to review and respond to eHealth data through the eHealth system, record keeping, and the need to respond in real-time.27,33

eHealth data issues

▪ How are the data integrated into the EHR or other data systems, in a form easily accessible to and understandable by HCTs; most clinics lack an adequate internal technology infrastructure, making clinics dependent on external systems around which they often have little control, and ability to customize the program for their staff and patient population.2,23,74

▪ Can the data received from users be easily shared with multiple members of the care team in ways that make the data actionable.29

▪ Major concerns about data protections: confidentiality, data loss, and vulnerability to Internet hackers.4,46,49

▪ Data transfer from clinic to user and vice versa often raises issues of clinic legal liability.49

Systems and clinical cultural issues

▪ How will an eHealth system conform with clinic culture and mission.56

▪ How will an eHealth system mesh with HCT beliefs, comfort and clinical style.56,66

▪ Which important stakeholders need to be involved in the planning and execution of eHealth programs, for example, staff, payers, users, HCTs.23

▪ eHealth programs are often viewed and funded as time-limited and are rarely fully integrated into clinical care for ongoing use.27,61

▪ There is a strong need for ongoing oversight, governance, commitment of stakeholders (including HCTs), medical and other leadership, coupled with policy reform.61,74

▪ eHealth systems are not solutions in and of themselves—they are only tools that can be used to help reach clinical goals; because they are unique, they have to fit seamlessly into clinic culture and operation to be most effective.27

Costsa

▪ In general, eHealth programs demonstrate good cost-effectiveness when clinical targets are well-defined, sufficient time for data collection is available, and patient populations are well-specified.6,7,25

▪ Overall costs will increase when multiple vs. single user groups are targeted, although costs per user will decrease. This reflects the complexity of assessing eHealth system costs.56

▪ There are at least five different ways of considering eHealth costs, value and benefits, each yielding different results: cost-effectiveness analysis; cost-utility analyses, cost-consequences analyses, cost-minimization analyses; cost-benefit analyses; budget-impact analysis: full analyses are rarely done.7,25,35

▪ eHealth systems may reduce expenses for patients but not for clinics and vice versa.7

▪ In general, costs increase based on the type and number of eHealth system components, which outcome is selected and the length of time of the eHealth program.7,25

▪ Text messaging appears to be the least expensive modality, but it is one-way, not easily customizable and most impersonal.56

▪ It is hard to define a small set of eHealth goals around which costs and benefits can be easily calculated; for example, reducing care inefficiencies, time to diagnosis, complications, hospitalization, etc. can require long-term programs and relatively large investments and numbers of users.6

▪ It is difficult to get compensated for start-up, maintenance, licensing, and credentialing costs, especially when users live across state lines; eHealth programs may have to be modified to insure that they reach criteria for reimbursement.46,49,56

▪ Reimbursement varies depending on the type of staff utilized and the payment system utilized by the patient. Reimbursement also varies by the tasks involved, for example:

▪ Remote physiological monitoring

▪ Telehealth visits

▪ CGM

▪ Diabetes Education provision

▪ Level 1 E&M visits

▪ Medicare Chronic Care Management

▪ Medicare Principal Care Management

^aThe cost issues outlined above reflect a U.S. point of view only and may not be applicable to other countries where different systems of health care funding occurs.

eHealth systems that include two-way communication can be far more complex and demanding of resources than simple one-way programs, such as text messaging or online module-based education. Yet, even one-way systems require well-managed technological infrastructures that allow for integration with existing data management systems, such as electronic health records (EHRs), so that easily accessible information can be shared with all members of the clinical team.

eHealth systems are often placed in silos outside of what are viewed by HCTs as regular clinic function. This can fragment care among team members, specialists, and patients; create competitive and duplicative within-clinic care structures; and undermine team functioning, leading to a loss of HCT support and interest. This is analogous to recent findings that clinic management efforts are most successful when care managers are closely engaged with HCTs and patients such that all aspects of care are coordinated, rather than having care managers operate peripherally from HCTs.⁵²

eHealth systems are often viewed as “cool” for those HCT members who are tech-oriented, but they are rarely viewed as an integral part of the clinic's mission and care structure by many team members and administrators. Thus, many reviews emphasize the need to integrate eHealth programs within the mission and culture of clinical services so that they fit the beliefs and clinical styles of HCTs and staff.^29,34 This usually means engaging with numerous stakeholders, including clinical, administrative, and staff leadership, to ensure that the program fits seamlessly into the clinic's operation. Because of the uniqueness of each clinic and patient panel, this often requires that eHealth systems be customized and adapted to more adequately fit the care setting, rather than using a standard, off the shelf program.^53,54 It also means that HCTs need to be carefully engaged, informed, and educated about their use as the teams are formed and as team functions and staffing change over time. A simple eHealth add-on to existing HCT functioning is rarely sustainable.

Because eHealth systems rely on sometimes complex, mobile technologies, actual use can be reduced by concerns regarding data storage, privacy, and security.^4,46,49 Issues of liability around potential data breaches also can affect use.⁵⁵ These are not inconsequential concerns and for many users who worry about stigma, they can impact their initial willingness to participate and can contribute to attrition over time.

eHealth Costs and “Return on Investment”

eHealth methodologies have the potential to demonstrate a good “return on investment,” dramatically expanding clinic reach to underserved, special, or otherwise geographically dispersed patients or to those who can profit from the real-time motivational, education, and monitoring capabilities of eHealth methodologies.²⁵ Furthermore, they can provide feedback and other assistance with disease management in ways that are simply not possible within the limits of a clinic visit. Although costs remain a critical issue in any form of health care, they need to be placed within the context of well-defined health outcomes, quality of life, and satisfaction with care for larger numbers of clinic patients. This challenging process involves carefully identifying what the current and expected costs might be, for which specific clinical outcomes, over what period of time, and for which patient groups; and identifying potential reimbursement options that will allow the program to be sustainable over time.¹⁶

Notably, however, a “good return on investment” for health systems does not necessarily correspond with a similar cost-benefit balance for patients. For patients who are not tech-savvy, who lack sufficient mobile data plans or smart phones, or who live in “dead zones” without Internet access, the costs of eHealth programs can be prohibitive and usability can be limited. Furthermore, although many text-messaging and one-way emailing systems that remind users about upcoming appointments have been shown to be beneficial by reducing the costs associated with missed appointments,^19,56 it can be difficult to demonstrate the costs and benefits of more complicated single or multicomponent systems.

Different methods of cost analysis can yield different results and results often depend on which kinds of comparisons are made between eHealth and other programs, such as in-office interventions, usual care, and alternative eHealth methods.⁷ The clarity of a cost-effectiveness or cost-benefit analysis is also dependent on the clarity of the desired outcomes, the number of users required to demonstrate benefits, and the expected time period during which costs and outcomes are measured. These issues often complicate the answer to a seemingly simple question—“Will this eHealth program save us money or provide a good “return on investment?” Additional staff time, supervisory costs, and startup and maintenance costs further complicate cost estimation over time. Furthermore, reimbursement levels vary dramatically based on the insurance mix of a clinic's patients, the model of reimbursement (e.g., fee-for-service, health maintenance organization/capitated care model, use of a value-based payment system), and the clinical staff available to review/bill for remote physiological monitoring, diabetes education, and/or chronic care management.^6,7 Although they are critical to a health systems' decision, very few attempts at comprehensive cost analysis include these factors.⁵⁷

Although several reviews indicate that eHealth systems can yield a good “return on investment” for a variety of outcomes, few demonstrate actual cost savings.⁶ Most studies, however, include users who are well-educated, white and tech-savvy, and are delivered by health systems with considerable eHealth experience, therefore limiting their generalizability to low resource clinical settings and patient groups. Often excluded from cost studies are users from rural or geographically dispersed, underserved, high risk areas where the need may be greatest and where the potential for substantive clinical improvement may reside.^6,7,25 Engaging these vulnerable populations, however, may require higher initial expenditures.

Some clinic settings have attempted to reduce per patient costs by expanding eHealth programs to include individuals with a variety of health conditions or clinical problems, such as asthma-Chronic Obstructive Pulmonary Disease, depressive disorders, or cardiovascular disease. Although this approach makes an eHealth program less user and disease-specific, it dramatically increases the number of potential users, thereby substantially reducing per user cost.

An alternative to clinic-housed eHealth systems, which can be expensive to develop and maintain, is for a health system to contract with an often less expensive, external vendor to provide eHealth services. This often leads to a difficult set of trade-offs: externally operated systems while at times less expensive and less demanding of clinic staff, require careful data sharing and communication with clinic systems, good quality control, and the vendor's ability to customize the system for clinic patients and staff in ways that fit the style, culture, and workflow of the HCT, its EHR, and patients. The ability to share data across systems in timely, clear, effective, and actionable ways can be difficult and can be subject to frequent clinical and technological glitches, and long lag-times.

A User's Guide

In Table 5, we provide a practical user's guide to summarize and integrate the various threads of eHealth considerations described above and in Tables 1 to 4 by posing 10 applied questions, the answers to which highlight the critical elements of eHealth selection and implementation. The questions include a focus on characteristics of the eHealth program itself, staff issues, costs, attrition, usability, and clinic context. Addressing these practical questions should help HCTs and health systems select, adapt, and implement the most appropriate eHealth resources for their setting and patients.

Strengths and Limitations

This article has both strengths and limitations. Strengths include a comprehensive synthesis of a very large and extremely diverse literature; a health care setting and systems perspective as opposed to just a focus on individual patient outcomes; an integration of behavioral, clinical, and implementation science perspectives; and presentation of detailed, user-friendly summary tables with a focus on pragmatic questions. Regarding limitations, this is not an exhaustive, systematic review or meta-analysis of original literature. We did not think that such an effort was needed, given the large number of already published reviews and the fact that our stated goal was to consider several pragmatic issues and questions: “what health programs for what purposes under what conditions.” Second, we acknowledge that although our conclusions and recommendations are supported by the references cited, much has been colored by our own experience in diabetes self-management, implementation science, and eHealth.^14,58–60

Conclusions

This article attempts to synthesize the vast eHealth literature and consider its implications for health care systems, HCTs and other entities considering the adoption and implementation of eHealth solutions for diabetes and other chronic conditions.

The extensive literature has produced encouraging, but mixed findings. eHealth programs can be strikingly attractive: they are often cleverly designed and can be engaging. However, many attempt to do far more than is practical, feasible, and even desirable, and many include far too many whistles and bells—simply because they can. Despite the benefits, the literature shows repeatedly that unless an eHealth system addresses an important clinical or behavioral need in a cost-efficient way, unless it appeals to clinicians and patients, unless it can be used easily and productively with recognizable gain, and unless it can be integrated effectively into existing staffing, patient flow, and related data systems, the system will most likely end up on the shelf after a short period of use. Hopefully, this review and the 10-question user's guide will facilitate a more successful outcome.

Author Disclosure Statement

All three authors have no competing financial interests to disclose. L.F. is a consultant to Eli Lilly and Ascensia Diabetes Care and has received speaking honoraria from Dexcom.

Funding Information

L.F. receives research support from The National Institutes of Health through grants RO1 DK121241 and R18 DK108039. R.E.G.'s contributions are partially supported by grant NCI P50 CA244688 and the Eastern Colorado Geriatric Research and Clinical Center. A.H.'s contributions are partially supported by grant NCI P50 CA244688.