Table 2.

Description of identified aspects of evaluation in electronic health interventions.

Aspects of assessment	Key areas of measurement
Organizational aspect	Organizational setting where the intervention is taking place; it can differ depending on the scale of the intervention (eg, health center, region, and country) [29]; all type of individuals or groups in the health care system that participate in the eHealth intervention, their characteristics, and expectations [30]; organizational performance and professional practice standards [30]; changes in the functions of the health care provider, skills and resource demands, and the roles of the professionals in the organization [29,31-33]; representativeness and participation rates of the health care professionals during the intervention [34]; capability of the organization to implement the intervention [30,34-38] and the extent that the technology fits the organizational strategy, operations, culture, and processes [30]; and sustainability or the degree that the technology becomes accustomed in the daily practice of an organization [29,32,34]
Technological aspect	Ensure trust [38], effectiveness, and contribution of quality of care [30,36] of the technology implemented; system performance: hardware and software requirements, correct functioning of the components [29,38], and system capability to meet users’ needs and fit the work patterns of the health care system’ professionals [30,39,40]; usability: broad experience of the users with the system [29,33,37,40]; privacy and security: safety and reliability of the technology [29], and security of the data managed in the technology [37,38]; technical accuracy: quality of the transfer of data [41]; information quality: relates to accuracy, completeness, and availability of the information produced by the system (eg, patients’ records, reports, images, and prescriptions), and it depends on users’ subjectivity [30,39]; service quality: measures the support and follow-up service delivered by the technology provider [39]; triability: the ability of the innovation to be tested on a small scale before the final implementation [40]; maturity: whether the system has been used on a sufficient number of patients to address all the technical problems [36]; and interoperability: communication between the technology and the pre-existing systems, the fit between the technology and the existing work practices [37]
Human and social aspect	Acceptance and usability satisfaction of the technology used in the intervention [30,31,33,36,38,39,41] where the user can be physicians, nurses and other staff, and patients, depending on the type of the participants in the intervention [41]; system use: volume of use, who is using, purpose of use, and motivation to use the technology [39]; user satisfaction: perceived usefulness, enjoyment, decision-making satisfaction, and overall satisfaction for the technology [30,39]; and psychological aspects such as satisfaction, well-being, and other psychological variables, and social aspects such as accessibility to the technology, the social relationships evolving over the transmission of care, or activities of the patients under the intervention [31]
Clinical aspect	Benefits and unanticipated negative effects of the intervention, biological outcomes including disease risk factors, behavioral outcomes of the participants, staff who deliver the intervention and the sponsors, and quality-of-life outcomes to evaluate participants’ mental health and satisfaction [34] and long-term measurements of the diagnostic and clinical effectiveness [41,35,36], safety of care [33,35,36], and quality of care [33]
Cost and economic aspect	Cost analysis methods to compare the intervention with relevant alternatives in terms of costs and consequences [36]; diverse cost analysis methods can be considered (eg, cost-minimization analysis, cost-effectiveness analysis, cost-benefit analysis, cost-utility analysis, and cost-consequence analysis) [30,31,41] and are conducted from several perspectives such as societal, third-party payers, health care providers, or patient [31]; and diverse costs can be included such as investment cost, monthly user charge of equipment, costs of used communication line, education of the technology, costs of patients and their close relatives [41], wages of doctor and other staff [30,41], expenditure and revenue for the health care organization adopting the technology [36], and resource utilization and opportunity cost of the eHealth intervention [34]
Ethical and legal aspect	Ethical concerns of the app itself and its implementation including all the stakeholders’ viewpoints on using the technology and the key ethical principles associated with the context in which intervention is conducted [35,36] and legal aspect identifies and analyzes the legislative documents and legal obligations that may exist in each context involved in the intervention [30,35,36]
Transferability aspect	Participation and representativeness of the intervention, percentage of persons who receive or are affected by the program, and the characteristics of participants and nonparticipants to investigate the extent that participants are representative and what population group should be a priority for future research [34] and transferability of results from studies of eHealth from one setting to another and the assessment of validity and reliability of the study [36]