Digital health interventions for chronic diseases: a scoping review of evaluation frameworks

Abstract

Background

Monitoring and evaluations of digital health (DH) solutions for the management of chronic diseases are quite heterogeneous and evidences around evaluating frameworks are inconsistent. An evidenced-based framework is needed to inform the evaluation process and rationale of such interventions. We aimed to explore the nature, extent and components of existing DH frameworks for chronic diseases.

Methods

This review was conducted based on the five steps of Arksey and O’Malley’s scoping review methodology. Out of 172 studies identified from, PubMed, Embase and Web of Science, 11 met our inclusion criteria. The reviewed studies developed DH frameworks for chronic diseases and published between 2010 and 2018.

Results

According to WHO guidelines for monitoring and evaluation of DH interventions, we identified seven Conceptual frameworks, two Results frameworks, one Logical framework and one Theory of change. The frameworks developed for providing interventions such as self-management, achieving personal goals and reducing relapse for cardiovascular disease, diabetes, chronic obstructive pulmonary disease and severe mental health. A few studies reported evaluation of the frameworks using randomised clinical trials (n=3) and feasibility testing via Likert scale survey (n=2). A wide range of outcomes were reported including access to care, cost-effectiveness, behavioural outcomes, patient–provider communications, technology acceptance and user experience.

Conclusion

There is a lack of evidence on the application of consistent DH frameworks. Future research should address the use of evidence-based frameworks into the research design, monitoring and evaluation process. This review explores the nature of DH frameworks for the management of chronic diseases and provides examples to guide monitoring and evaluation of interventions.

Key messages.

What is already known about this subject?

• Rapid advancements of digital technology and marketing strategies make it difficult to compare accessibility and affordability of digital products and services across communities intercountries and intracountries.

• Methodological quality of existing evidence around digital health interventions for chronic disease is low and the results are inconsistent.

• The most significant issue influencing the effectiveness of digital health is related to the monitoring and evaluation of such interventions.

• There is a need for a common framework to inform the monitoring and evaluation of digital health projects and to outline the process and rationale of such interventions.

What are the new findings?

• In this scoping review, we reviewed studies that reported the development of a digital health framework for patients with chronic diseases.

• Reviewed frameworks were categorised based on the WHO guidelines for monitoring and evaluating digital health interventions. We identified seven Conceptual frameworks, two Results frameworks, one Logical framework and one Theory of change.

• The frameworks developed for providing interventions, such as self-management, achieving personal goals and reducing relapse.

• A wide range of patients’ outcomes were considered in the reviewed studies including access to care, cost-effectiveness, self-management, improving behavioural outcomes, facilitating patient–provider communications, motivational feedback, improving technology acceptance and user experience.

How might it impact on clinical practice in the foreseeable future?

• Among 11 studies, only three reported validation of proposed frameworks. There is a significant gap regarding the evidence on the application of consistent frameworks in this field and further demonstrated a current lack of consensus surrounding the evaluation and measurement of patient outcomes.

• This review provides examples of frameworks that were introduced by WHO which can guide the evaluation of specific interventions.

Introduction

Digital health (DH) interventions have contributed to the transformation of healthcare delivery in the past decade. Among the wide range of applicability, chronic diseases have been the most notable context of DH research and development. This is attributed to the high cost of healthcare (80%) for the management and treatment of chronic diseases in many countries.¹

Despite the potential for the delivery of healthcare with relatively low cost at scale, implementation of DH interventions is not an easy endeavour. For several reasons, it is almost impossible to have a universal ‘digital recipe’ for managing chronic diseases. From a patient’s perspective, strategies for the self-management of a chronic condition may vary based on sociocultural and economic status of people. From a health system’s perspective, different countries have different legislative and policy implications for adoption of DH interventions.²

DH interventions are complex and altering through stages of maturity. As a result of barriers to implementation and sustainability of evidence-based DH, there are a limited number of successful interventions beyond the pilot or feasibility stage. In general, the methodological quality of the studies on DH for chronic diseases is at a low level. For some fields, the results are mixed, or existing studies are short term and there is no evidence of impact.^3–5 The most significant issue influencing the effectiveness of such interventions concerns monitoring and evaluation (M&E) methods. M&E of DH interventions is critical to assess progress, identify problems and facilitate changes to improve service delivery and achieve the desired outcomes. Previous research showed that there is a lack of knowledge related to the development of frameworks for M&E of DH interventions that can inherit the values of community-based participatory research and the importance of acknowledging diverse cultural perspectives and settings.⁶

Moreover, rapid advancements of digital technology and marketing strategies make it difficult to compare accessibility and affordability of digitally enabled healthcare services across communities intercountries and intracountries. As a result, evaluations of DH interventions for the management of chronic diseases are quite heterogeneous and the evidence around evaluating frameworks is inconsistent. Therefore, there is a need for a common framework to inform the evaluation of DH projects and initiatives.

According to WHO, the level of DH activity is growing in countries. However, evaluation of those activities by member states is very low (12%). Evaluation will need to be incorporated into the project management life cycle to ensure better quality results.⁷ Based on the WHO guideline for M&E of DH interventions, frameworks are required to outline the process and rationale of such interventions and to assist researchers to achieve their goals.⁸ The WHO resource provides step-by-step guidance to improve the quality and value of M&E efforts in the context of DH interventions, also commonly referred to as mHealth or eHealth interventions. The guideline is intended to assist implementers and researchers of DH activities, as well as policy-makers who seek to understand the various stages and opportunities for systematically monitoring fidelity and for evaluating the impact of DH interventions. This resource aims to serve as a reference document for those implementing DH interventions, who need a practical guide for understanding systematic approaches to M&E in the multidisciplinary field of DH. Grounded in the real experiences that have emerged from numerous projects across three continents, this tool offers guidance ranging from development of M&E plan to monitoring implementations, evaluating outcomes, assessing data quality and eventually reporting findings.⁸

Developing a framework will assist researchers to (1) identify the aims and objectives of the intervention; (2) understand the intrarelationship of different objectives; (3) construct the project activities required to achieve aims and objectives and (4) explain the expected outcomes. Framework for M&E of DH interventions is similar to a roadmap. It clarifies the steps needed to assess the processes and outcomes of a programme. Framework addresses indicator selection, related data sources, analysis and synthesis practices (including quality assessment), and communication and use. Frameworks offer a platform that can guide researchers and policy-makers in generating and translating evidence to support future directions and ongoing investment in DH services. Recent debates highlight the importance of fostering evaluation designs, which combine different research methods, using qualitative, quantitative and codesign principles, as well as process measures.⁹

We studied the frameworks based on the WHO four categories. These are Conceptual framework, Results framework, Logical framework and Theory of change, which are briefly described in the following.

Conceptual framework

Conceptual framework also known as casual framework can be a form of diagram that defines and demonstrates intra-relationship of key factors including systemic, organisational and individual, which may affect programme implementations and the success of project’s goal(s) (a health problem). The purpose of this framework is to identify enablers and barriers in the process of evaluating DH interventions. Furthermore, the framework provides insight for understanding of programme goals including underlying factors within the implementing environment and explains analytical assumptions and their effects on programme facilitators or barriers.⁸

Results framework

Results framework is a graphical presentation of a research programme that identifies a strategy to achieve a specific goal/s with the cause and effect linkage.¹⁰ The framework identifies cause and effect relationship between programme objectives and observed impact. The Results framework aims to build consensus and ownership for the activities that constructs the programme. It assists identifying approaches to measure the achievement of proposed programme goals, select appropriate inputs and establish the foundation for planning and refining programme objectives. The results framework consists of the assistance objective (AO) and intermediate results (IRs). They also include hypothesised cause and effect linkages and the critical assumptions that must hold true for the strategy to remain valid. AOs and IRs should be stated clearly and precisely and can be objectively measured. The AO should represent the team’s best assessment of what can realistically be achieved. IR is defined as an essential step to achieving results or outcomes.⁸

Logical framework

A Logical framework is a tool to manage and measure aims and objectives of a project, to identify its key assumptions, and to monitor and evaluate outputs. The Logical framework identifies programme objectives and aids in the clarification of complex relationship between inputs, processes, outputs, outcomes and impacts.¹¹ WHO defines Logical framework as a graphical representation that facilitates involving and communicating with diverse stakeholders, including decision-makers, in a repetitive process, during the programme development or implementation.

Logical frameworks connect inputs (project resources) with processes (required activities to deliver services), outputs (products of processes), outcomes (intermediate changes) and impacts. The framework components are described as follows:

• Assumptions: ideas that guide research and development.

• Methods: strategies, theoretical background, the use of guidelines to assist the development process.

• Inputs: encompass critical resources that go into the model and on which it is dependent to mount its activities.

• Activities: actual events or actions done within the model and its resources. Inter-relationships between activities.

• Outputs: are measurable products of the model inputs and activities.

• Outcomes: the changes that result from the model’s activities in a sequence expressed (short term, intermediate term and long term).⁸

Theory of change

Project outcomes and activities are connected using Theory of change framework to explain how and why the proposed change is occurred and can be measured.¹² Like Logical frameworks, Theory-based frameworks aim to provide a perspective of the casual links between intervention activities and expected results. Despite Logical frameworks, Theory of change does not provide a linear cause and effect relationship but rather defining the multiple factors or causal determinants as well as underlying assumptions. This model can be examined in an ongoing process of reflection to explore change and its procedure.¹³

Although four common frameworks have been introduced by WHO for M&E of DH interventions, there is lack of consensus regarding the essential elements and attributes that support such interventions. Hence, we conducted a scoping review to (1) explore the nature, extent and type of DH frameworks for chronic diseases, (2) identify the gap in the current evidence and limitations in existing knowledge and (3) guide future research directions. The primary aim of this review was to synthesise evidence related to the type of frameworks developed for DH interventions and chronic conditions. The secondary aim was to understand the essential components of the proposed DH frameworks.

Study design

Overview of scoping review methodology

Given the rapid evolution of DH interventions, we chose the scoping review methodology to obtain an overview of the extant literature on DH frameworks for chronic diseases interventions. This review was conducted based on the five steps of scoping review methodology described by Arksey and O’Malley’s scoping review methodology.¹⁴Figure 1 illustrates the steps of the scoping review methodology. The aim of a scoping review is to explore the existing literature in order to compare the nature, type and range of the previous research and to identify knowledge gaps in the available evidence.¹⁴ In this review, we considered studies that described or evaluated frameworks, maps or conceptual structures for DH intervention proposed for patients with chronic diseases.

Figure 1.

Steps of Arksey and O’Malley’s scoping review methodology.

Step 1: identifying the research question

According to WHO, DH is defined as using digital, mobile and wireless technologies to support the achievement of health objectives. The examples of DH interventions include patient–provider communication, point-of-care data exchange, remote monitoring of medical devices, public health alerts, patient education and clinical trials information. The first stage of identifying research questions began with the reviewing of the WHO guidelines for M&E of DH interventions, followed by the identification of the knowledge gap of the DH frameworks in the current literature. To comprehensively understand the range and nature of DH frameworks for chronic diseases management, this scoping review aimed to answer two research questions: (1) to describe the types of frameworks developed for DH interventions and (2) to understand the essential components of the existing DH frameworks.

Step 2: identifying relevant studies

The authors collaboratively planned and developed relevant keywords to identify existing literature that was specific to DH frameworks for the management of chronic conditions. Electronic searches were performed in PubMed, Web of Science and Embase. These databases were selected to ensure all relevant studies were retrieved. These databases were searched for English language articles published between 2010 and 2018. A combination of free-text keywords and Medical Subject Heading terms were used for searching PubMed.^15–17 The search query was refined for searching Web of Science and Embase according to their guides. Search terms were selected based on a combination of the following three main concepts: (digital health) and (chronic conditions) and (frameworks) (online supplementary appendix 1). For chronic diseases, we searched for chronic disease as a general concept, as well as specific diseases including stroke, asthma, chronic heart failure, chronic obstructive pulmonary disease, diabetes, hepatitis, hypertension and mental disease. The search strategy is outlined in online supplementary appendix 1.

Step 3: selecting appropriate articles for the scoping review

The selected titles and abstracts of the existing literature were searched by two reviewers independently. Through involving third reviewer and team consultation conflicts were resolved. Our inclusion criteria were frameworks/models developed for DH technologies focusing on chronic conditions which published in English. Research articles using different methodologies (both qualitative and quantitative) including the articles for theoretical models were considered for inclusion.

Step 4: charting the data

To extract and chart the data from selected articles, we used a descriptive–analytical method. Based on the team consultation, a process was employed to extract the data and charting onto the tables that developed by the first author based on the WHO guidelines. Tables were used to collect, summarise and chart data for team review and decision making. Data extraction items were study characteristics, frameworks’ category and components. General study characteristics were author and year, target population, proposed intervention, expected outcomes and evaluation method if reported. Frameworks were categorised based on the WHO frameworks’ classifications.

Step 5: summarising, analysing and reporting the results

This stage consisted of data analysis, reporting and interpreting the results. This was designed based on the Arksey and O’Malley recommendation on adopting a theoretical framework to gather and summarise the extracted variables in a systematic manner.¹⁴ We adopted the WHO frameworks to summarise the results. Studies then were charted and reported into the following categories to answer the specific research questions from the scoping review step 1: (1) study characteristics, (2) type of frameworks and (3) components of frameworks.

Results

As shown in figure 2, our search identified a total of 2544 papers from searching three different online databases. After excluding duplicate records and screening 2447 records at the title or abstract level, we examined the full text of 172 potentially applicable articles. Finally, 11 studies were included in this review.

Figure 2.

Study flow diagram.

Study characteristics

Four studies conducted in the USA^18–21 and two studies^{9 22} were carried out in the UK. The rest of studies were conducted in different countries.^23–27 Studies reported DH frameworks for different chronic diseases such as cardiovascular disease,^{18 22} diabetes,²⁵ chronic obstructive pulmonary disease,²¹ severe mental illness,²³ HIV/AIDS,²⁰ and combination of chronic diseases^{19 21 26}. One study did not consider any specific condition for the proposed framework⁹ and the other study reported senior consumers (table 1).²⁴

Table 1.

Study characteristics of the included papers

Author/year	Country	Target population	Sample size and setting	Proposed intervention	Expected outcome(s)	Evaluation method
Villarreal 200927	Republic of Panama	Patients with diabetes	Setting: Not reported Sample size: Not reported	mHealth: Mobile app for providing self-control	Self-control, patient monitoring, improving communication between patients and doctor	Not reported
Koutkias 201026	Greece	Patients with chronic disease	Setting: Hypertension outpatient clinic Sample size: Not reported	eHealth: Body area network for providing home care services and increasing self-management	Home care service delivery, interoperability, extensibility, access to drug and patient information, access to care	Not reported
Beatty 201318	USA	Patients with ischaemic heart disease	Setting: Not reported Sample size: Not reported	mHealth: A mobile app for improving access, increasing participation and improving outcomes in patients	Cost-effective, access to care, behaviour change, patient-centred health, reduce rates of rehospitalisation, increase participations	Randomised controlled trial
Dhillon 201324	New Zealand	Senior health consumers	Setting: Not reported Sample size: 43 seniors aged 60–85	Telehealth: Web-based system with a Facebook-like plug-in architecture for increasing patient motivation	Open and extensible system, social and emotional support, Feedback and motivation, access to care, cost-effective	Usability and effectiveness of the framework were evaluated via 6-level Likert scale
Beentjes 201523	Netherland	Patients with severe mental illness	Setting: Inpatient/outpatient clinics Sample size: Not mentioned	eHealth: e-IMR programme for involving other important people, manage achieving personal recovery goals and reducing relapse	Access to care, cost-effective,	Cluster randomised controlled trial
Fico 201525	Spain	Patients with diabetes	Setting: Not reported Sample size: Not reported	eHealth: A model for providing self-management	Cost-effective, maximised usability, user experience, patient engagement, self-management	Not reported
Gee 201519	USA	Patients with chronic disease	Setting: Not reported Sample size: Not reported	eHealth: A model (using mobile devices) for providing self-management	Improving functional and clinical outcomes, patient-centred outcomes, access to care, access to data,	Not reported
Salisbury 201522	UK	Patients with chronic diseases (cardiovascular disease and depression)	Setting: Not reported Sample size: 34 patients completed the questionnaire	Telehealth: A model for providing patient engagement (including care coordination, patient self-management, optimisation of treatment)	Health outcomes, access to care, patient experience, cost-effective	Randomised controlled trials
Schnall 201620	USA	HIV prevention for high-risk men who have sex with men (MSM)	Setting: Not reported Sample size: 5 focus groups with 33 targeted end-users.	mHealth (Mobile App) Mobile App for providing self-management	Behaviour change and self-management, cost-effective, Improving technology acceptance, access to care, to enhance usability, easing the doctor’s duty, usefulness, timesaving	User interface and system function of prototype, end-user usability
Wilhide 201621	USA	Patients with chronic diseases (diabetes, epilepsy, asthma, chronic obstructive pulmonary disease, lupus, HER2 +breast cancer, and low back pain)	Setting: Not mentioned Sample size: Not reported	mHealth; Mobile App for providing self-management	Change healthcare service and self-management behaviour, access to care, cost-effective, user experience	Not reported
Greenhalgh 20179	UK	Individuals/patients who abandon health technologies	Setting: Cardiology departments of hospital Sample size: Not reported	Telehealth: A model for predicting and evaluating the success of technology-supported health and social care programmes	Cost-effective, access to care, client self-refer, patient safety, ease of use, case management,	Not reported

Out of 11 studies, three reported inpatient/outpatient clinics^{23 26} or hospitals⁹ as their study settings. On the basis of input data, the studies in our review described a number of frameworks that were developed for a range of DH interventions including providing self-management,19–22 25–27 achieving personal recovery goals,²³ reducing relapse,²³ increasing patient participation with mobile-based intervention, improving clinical outcomes,¹⁸ increasing motivation with using web-based application,²⁴ and evaluating the impact of technology for health and social care (table 1).⁹

A wide range of patient outcomes considered in the reviewed studies including access to care, cost-effectiveness, self-management, improving behavioural outcomes, facilitating patient–provider communications, motivational feedback, improving technology acceptance and user experience.

Among 11 studies, three reported of the validation of developed frameworks in randomised controlled trials (RCTs)^{18 22 23} and two tested the feasibility of their frameworks using a Likert scale survey.^{20 24} The rest of studies did not report any evaluation methods or outcomes (table 1).9 19 21 25–27

Type of frameworks

Based on the most commonly used frameworks outlined by WHO guidelines for M&E DH interventions, we categorised reviewed frameworks into: (1) Conceptual framework, (2) Results framework, (3) Logical framework and (4) Theory of change. Out of 11 studies, seven reported development of Conceptual frameworks,9 19 20 22 24 26 27 two Results frameworks,^{23 25} one Logical framework²¹ and one Theory of change (figure 3).¹⁸

Figure 3.

Distribution of frameworks based on WHO guidelines.

Components of frameworks

Conceptual frameworks

Based on the WHO guidelines, Conceptual frameworks consist of two main components including factors and anticipated goals.⁸

According to table 2, the reviewed studies considered different factors in their conceptual frameworks. Two studies proposed factors such as involvement of patients and healthcare providers in the process of mHealth development^{22 27} and two studies focused on factors such as partnership between providers.^{9 19}

Table 2.

Conceptual frameworks

Author/year	DH intervention	Factors (systemic, organisational, individual or other)	Anticipated goals
Villarreal 200927	mHealth	Providing better communication between patients and providers, improving patient education, diet control, preventive control based on patient condition, decreasing frequent visits to the doctors, providing continuous patient monitoring	Make a healthy daily routine for patients, making patients’ lives easier, constant control on glucose tendency, ease the day-by-day life of patients, to enhance patients’ self-control, personalised care and advices,
Koutkias 201026	eHealth	Medication management, patients monitoring, vital sign measurement, monitoring patient treatment, adverse drug event recognition.	Extend home care service delivery, personalised medication treatment, communicating patient and provider, personalisation of healthcare, providing two-way communication between the home care system and clinical environment
Dhillon 201324	Telehealth	Patient centric, accessibility with web-based system, easy to use interface, share data among multiple applications by using a triple store database, adding new health apps by using Facebook-like plug-in architecture, using a content management, security with encryption	Improving emotional health and well-being, motivating the patients, consultation with the health professionals, tracking (weight, exercise, vital sign)
Gee 201519	eHealth	Self-management: access, convenience, reminders, alerts, planning, empowerment, engagement delivery system design: care coordination, interoperability, medical jargon, timeliness, policy, content, networking. clinical decision support: graphs, charts, protocols, guidelines, reminders, info buttons. Clinical information systems: Electronic Health Records, Personal Health Records, patient portal, internet, mhealth, smartphone, wearable devices, telehealth. ehealth education: message training, health education, technology training, e-community training, navigation training, accuracy, completeness, volume of information, customisation numeracy, literacy, usability, security	Patient activation, patient engagement, self-management enhancement, support effective patient–doctor interactions and improve health outcomes.
Salisbury 201522	Telehealth	Chronic disease management, engagement of patients and providers, partnership	Health outcomes access to care, patient experience, cost-effectiveness
Schnall 201620	mHealth	User centred, easy to use, contributing patients in app design.	Change behaviour of patients, improving patient self-management,
Greenhalgh 20179	Telehealth	To increase clinician participation, increasing the use of patient-facing technology, well-established sociotechnical infrastructure, improving caregivers respond, patient encouragement to connect with call centre in emergency situations.	Generating the knowledge or making it visible by technology, addresses the knowledge and support needed to use the technology, sustainability by addressing the issues,

DH, digital health.

Effective chronic disease management (including subcomponents of self-management, optimisation of treatment, care coordination) also were considered as factors in studies conducted by Gee et al¹⁹ and Salisbury et al²² used the chronic disease management model as a basis for the framework to evaluate telehealth intervention using parallel RCTs for two different conditions.²² For instance, the framework proposed by Villarreal et al²⁷ was used to form the patterns to generate self-management instructions by measuring patient data, generating patient profiles, identifying educational instructions and communication tools.

Factors such as development of patient-centred intervention^{20 24} and patient education was considered in two studies.^{19 27} While three studies considered technical systems and well-established sociotechnical infrastructure,^{9 20 24} clinical information systems and clinical decision support were reported by.¹⁹ For example, the web-based system proposed by Dhillon et al,²⁴ included a Facebook-like plug-in for adding new health applications, and incorporating social networking functionalities. Authors ensued that health consumers have a positive view of this new telehealth technology, and that it can positively change the attitude of users toward their health (table 2).

We fund two studies that included monitoring vital signs, and patient treatment process as the underlying factors for their frameworks.^{26 27} However, medication and vital sign management only considered by Koutkias et al²⁶ As shown in table 2, Koutkias et al²⁶ considered different factors to monitoring patient treatment procedure, in terms of medication response and patient safety.

Based on the results of our review anticipated outcomes reported by the studies were widely varied. Where some studies considered more than four outcomes such as extending home care telehealth service, personalised medication treatment and personalisation of healthcare,¹⁹ others reported only two outcomes.²⁰ Improving two-way communications between home care system and clinical environment were reported by two studies.^{19 24} Cost-effective care was also reported by two studies.^{9 22}

While three studies reported improving patient health as their outcomes,^{19 22 27} the other three considered improving access to care.^{18 19 23} Improving patient self-management was also considered by two studies.^{19 20} Increasing technology acceptance and improving patient experience were reported by Salisbury et al²² and Villarreal et al.²⁷

Results frameworks

According to WHO guidelines, Results frameworks consists of AOs, IRs, hypothesised cause and effects linkage, and critical assumptions.⁸ As shown in table 3, we categorised two studies in the Results frameworks. Beentjes et al proposed a model using intervention mapping and developed a matrix which covered behavioural outcomes, performance objectives, determinants and proximal change objectives.²³ The elements of the proposed matrix are compatible with the components of the results frameworks. The authors considered objective planning and executive attainable steps toward recovery goals and coping actively with symptoms and stressors as hypothesised causes factors. Reducing relapse, set and achieve recovery goals, problem-solving and communicating skills considered as effects linkage.

Table 3.

Results frameworks

Author/year	Assistance objective	Intermediate results	Hypothesised cause and effects linkage	Critical assumptions
Beentjes 201523	Estimating the added value of e-health will be possible because it is controlled by the standard Illness Management and Recovery (IMR) programme.	Effective education (using hardcopy text books), patient technology usage experience, illness management and recovery	Causes: Objective planning and executive attainable steps towards recovery goals, coping actively with symptoms and stressors ------------------------- Effects: Reducing relapse, set and achieve recovery goals, problem-solving skills and communicating skills.	Having connections with other important people, achieving personal recovery goals, reducing relapse of psychiatric symptoms, achieving goals and reducing relapse
Fico 201525	Participatory development approach, and persuasive design techniques to engage patients in adopting positive self-management behaviours.	Support patient monitoring, Personalised care, follow-up, easy to learn.	Causes: providing physical and virtual spaces to address the needs of health practitioners Effects: to implement more effective care provision, and empower patients to become coproducers of their own healthcare	Improve management and complication risk assessment, adopting positive self-management behaviours

Fico and Arredondo,²⁵ developed a road map as a modular framework to structure the different activities including research, development, business modelling, validation and evaluation in a unified strategy. We categorised the Fico and Arredondo²⁵ model as Results framework. The framework AO is based on a participatory development approach, and persuasive design techniques to engage patients in adopting positive self-management behaviours. A number of IRs also reported including support patient monitoring, personalised care, follow-up and easy to learn. Hypothesised causes included providing physical and virtual spaces to address the needs of health practitioners. Linkage effects were implement more effective care provision and empower patients to become coproducers of their own healthcare.²⁵

Logical frameworks

Wilhide et al²¹ developed a Logical framework for providing self-management intervention for diverse chronic diseases.²¹ The components of the framework are detailed in table 4. The model provides an efficient method of using resources including national guidelines, standards of care, published literatures and behavioural change theories. As presented in table 4, the framework inputs, processes, outputs and outcomes have been identified.

Table 4.

Logical framework

Author/year	Inputs (programme resources)	Process (activates undertaken)	Outputs (products of processes)	Outcomes (intermediate changes)	Impact
Wilhide 201621	National guideline, standards of care, published literature and meta analyses, Behaviour change theories, Disease-specific guidelines.	Identification of stakeholders by interdisciplinary teams, Guiding the design of integrated clinical and behavioural interventions/ supporting actions identification for each essential behaviour/ Intervention identification by analysing on clinical guidelines standards of care, evidence based public health programmes, medicine and healthcare/ categorising interventions based on their strategic intent.	Monitoring support, education support, GPS-facilitated features, support meal planning, universal education videos and tips, logbook, real-time feedback, longitudinal feedback, Medication adherence tool, carb estimation tool restaurant locator Tailored healthcare provider report Time management support, homepage design, Time-based ‘touchpoint’ messages educational skills-building support self-management tools.	Improving access to care, Reducing healthcare costs, Individualisation of user’s experience, self-management educational curriculum, patient–provider communication support, patient engagement, behavioural adherence support, coaching support, Improve clinical decision making.	Improvement of health in patients with chronic disease, Improving quality of life

Theory of change framework

As shown in table 5 Beatty et al¹⁸ proposed a Theory of change framework to evaluate interventions for cardiovascular secondary prevention.¹⁸ The authors noted that cardiac rehabilitation can be considered a behavioural change intervention to promote healthy behaviours in patients with ischaemic heart disease. The authors recommended to address the core components of the framework including behavioural change theory, individual tailoring of features, usability and evaluating intervention in an RCT to assess patient-centred outcomes. The anticipated outcomes reported by Beatty et al,¹⁸ included patient-centred outcome, usability cost and cardiovascular events. The underlying behavioural change assumptions were patient assessment, exercise training and self-management. Individual patient risk factor assessment and management, exercise training, self-management of modifiable risk factors and psychosocial support were reported as process or sequences of interventions activities.¹⁸

Table 5.

Theory of change framework

Author/year	Context	Anticipated outcomes	Process/sequence of interventions (activities)	Assumptions	Diagram and narrative summery
Beatty 201318	Cardiac rehabilitation programme	Patient-centred outcome, usability cost, cardiovascular events	Design an easy-use interface. individual patient risk factor assessment and management, exercise training, self-management of modifiable risk factors and psychosocial support.	Behavioural change (patient assessment, exercise training, self-management)	Yes

Discussion

Principal findings

The results of this scoping review have been presented as follows: (1) characterisations of reviewed studies that developed DH frameworks for patients with chronic diseases; (2) identification of the type of the frameworks and (3) exploring the components of each frameworks based on the four categories introduced by WHO guidelines. In our review, we categorised the frameworks of seven studies as Conceptual frameworks. Based on the WHO guidelines, two frameworks were categorised as Results frameworks, one as Logical and one as Theory of change.

Based on the results of this review, studies mainly developed Conceptual frameworks for DH solutions. Out of seven studies, two developed a framework for addressing an intervention design including medication treatment and diabetic monitoring.^{26 27} The rest of studies reported on developing frameworks for different purposes such as effective use of telehealth among patients with chronic health condition, a novel telehealth system to overcome shortcomings of existing technologies, to help predict and evaluate the success of a technology-based health/social care programme, and to guide for the design of mHealth apps.9 19 20 22 24

A broad range of factors and anticipated outcomes were identified by the reviewed studies as components of conceptual frameworks. Factors reported by Koutkias et al²⁶ were related to patient’s treatment procedure, in terms of medication response and patient safety.²⁶ Villarreal et al²⁷ referred to factors as creating modules and communication tools between each of the measuring devices and the mobile phone.²⁷

The framework proposed by Salisbury et al²² addressed four key factors including engagement of patients and health professionals, effective chronic disease management (including subcomponents of self-management, optimisation of treatment, care coordination), partnership between providers, and patient social and health system context. Dhillon et al²⁴ discussed about the system that was web based, has a Facebook-like plug-in architecture for adding new health applications, and incorporating social networking functionalities.

Key intended outcomes reported by studies varied and covered different measurements including extended home care telehealth service, personalised medication treatment, facilitating patient and provider communication, personalisation of healthcare, providing two-way communication between home care system and clinical environment, improved health, access to care, improved patient experience, cost-effective care, patient safety, ease of use, self-management, improved technology acceptance, enhanced usability and time-saving.9 19 20 22 24 26 27

One study described that although the proposed framework contributes to chronic care model, but it requires significant improvement in several areas particularly, eHealth-related educational materials which is essential for self-care. Furthermore, it is necessary to address the context of community for eHealth support and enhancement with the benefits of the e-Community or virtual communities. A complete feedback loop must be addressed to assure effective technology-based interactions between patient and provider.¹⁹

As reported in the studies conducted by Fico and Arredondo and Beentjes et al,²³ the Results framework provided detailed information including the cause and effects links between the desired programme goal and the IRs. A detailed Logical framework reported by Wilhide et al²¹ to design a mobile app intervention for patients with chronic disease.²¹ Previous research showed that developing the Logical framework for community health assessment assisted researcher to understand and communicate intended impact by considering political, economic, social and technological factors.²⁸ Logical frameworks are considered as management and measurement tools which provide graphical representations that can be served as a facilitator for involving and communicating with diverse stakeholders, including implementers. However, this procedure must be in a repetitive manner to optimise programme design or implementation.

Based on the results of this scoping review Beatty et al¹⁸ proposed a Theory of change framework.¹⁸ Theory of change can be viewed as a tool to map the logical relationship between inputs and outcomes. The mapping of the logical sequence is strengthened by critical thinking about the framework components that may influence the programme, the motivations and contributions of stakeholders and other actors. The logical relationship between inputs and outcomes identifies the different interpretations or assumptions about how and why that sequence of change might occur.

A Theory of change guides identifying key evaluation questions that are expected to address critical points during the process of M&E. This will in turn ensure that the project indicators are set up to measure all relevant steps and processes, and not only to address one factor, such as outputs. A well-established Theory of change framework improves M&E processes whether these are mid-term or end-of-project/programme evaluations and allows researchers to put any unanticipated or unintended outcomes (if they arise) in context.

The exercise of making an evidenced-based framework for M&E of DH interventions, in words and/or figures, is critical to clarify what is known and to identify what is in question or not known.²⁹ argued that ‘assumptions, methods and study designs of experimental science’are not as appropriate in evaluating DH interventions than one that can account for the ‘personal, social, political and ideological components’.²⁹

Learning how to develop and apply integrative DH frameworks is an educational issue for researchers in training, as well as those who train them and those who asses their works.²⁸ DH evaluation might be facilitated through promoting education about the importance of conducting DH research, developing coherence in description of such interventions, agreeing on common outcomes measures, and improving reporting, indexing and systematic reviewing of the literature on DH.³⁰

Catwell and Sheikh³¹ provided a framework for systematic evaluation and discussed the importance of evaluating eHealth interventions during all stages of design, development and deployment.³¹ Results of this review reflect large gaps in knowledge and few standards in practice in a rapidly expanding field.

Previous research conducted by Fatehi et al³² introduced a series of important stages for the formulation of appropriate research questions and design of suitable telehealth studies.³² We recommend considering a number of stages for developing DH frameworks too. Hence, it is recommended to start with a framework which addresses the most important factors of a problem and solution. Conceptual frameworks are appropriate to adopt for the evaluation of DH intervention in early stages as they aim to clarify the nature of the problem and guide the development of possible solutions. Conceptual frameworks are like lighthouses and lenses which illuminate certain parts of the ocean at any given time, other parts are left in the dark.³³ Different Conceptual frameworks emphasise (magnify) different aspects of the problem or elements of the solutions. As a result, more than one Conceptual framework may be required for assessment and monitoring of a DH intervention. Developing a Conceptual framework or a set of frameworks to address problem or assess a programme involves critical thinking and is a task that can be challenging, especially for novice researchers. Although this type of frameworks plays an important and essential role in identifying the nature of problems and proposing suitable interventions, they may not provide specific details of the proposed programme. It can be concluded that, as DH interventions become advanced the more detailed frameworks such as Results frameworks and Theory of change can be adopted.

Using the framework enables researchers to acknowledge the complexity of mHealth design and implementation by involving a broad range of stakeholders and addressing their interests to create remarkable change in healthcare delivery. DH framework can serve multiple important purposes. The framework can be a continuous process of M&E that serves to guide DH projects and provide a thoughtful assessment of the project’s impact on healthcare.

Evidence gap

The systematic methodical and comprehensive assessment of a continuing or completed DH intervention is required to identify the fulfilment of objectives, goals, productivity, effectiveness, impact and maintenance. Based on the results of this review, there is a lack of information regarding the evaluation of proposed frameworks. Five studies did not report whether the developed frameworks have been validated and the proposed intervention evaluated. Hence, additional empirical studies are required to ensure whether interventions which developed based on the proposed frameworks are associated with greater uptake, usage, engagement and improvement in health-related outcomes.

Future research directions

The findings from this review have two implications for the future development of DH interventions. First, this review provides examples of frameworks that were introduced by WHO that may guide the evaluation of specific interventions. Second, this review has identified lack of evidence regarding the application of consistent frameworks in this field and further, demonstrated a current lack of consensus surrounding the evaluation and measurement of patient outcomes.

Strengths and limitations

Compared with prior scoping reviews in DH, this review provides a map of literature on the underexplored topic of DH frameworks for the management of chronic diseases. This scoping study is subject to publication bias. Despite our aim to be as inclusive as possible, searching three separate databases may have resulted to missing a number of studies. This study may be limited by the restrictive search of online reference databases and exclusion of grey literature. Grey literature was excluded from this study to balance the feasibility of this scoping review with the available resources. Given the nature of the scoping review, the evidences related to DH frameworks were not synthesised to determine its effectiveness. Instead, the diversity of the available literature with its varied objectives, nature, components and populations was captured. Consequently, the aim of this scoping review was to explore the existing evidence and provide recommendations for future research directions.

Conclusions

To the best of our knowledge, our study is the first scoping review that explored the nature of existing DH frameworks for the management of chronic diseases. This review was conducted based on the WHO guidelines to map the identified frameworks for M&E of DH interventions. Future DH interventions must adopt evidence-based frameworks into the research design, explore more powerful and reliable assessment and monitoring strategies to enhance and expand DH practice. To unlock the potential of rapidly advancing technologies for the patients with chronic diseases, future DH research should apply evidenced-based frameworks prior assessing and evaluating interventions.