Effectiveness of eHealth literacy interventions: a systematic review and meta-analysis of experimental studies

Abstract

Background

eHealth Literacy (eHL) is a set of competencies and skills encompassing the knowledge, comfort and perceived ability to identify, evaluate and apply electronic health information to health problems. Given its role in the appropriate use of health technologies, ensuring equitable access to health information and improving patient outcomes, this study aims to systematically retrieve, qualitatively and quantitative pool and critically appraise available experimental evidence on the effectiveness of eHL interventions across different population groups.

Methods

Following the PRISMA guidelines, we conducted a systematic review in PubMed/Medline, Scopus, Web of Science, Embase, Cochrane Library and ClinicalTrials.gov, including original experimental studies quantifying the effectiveness of interventions aimed at increasing eHL, as assessed by the eHealth Literacy Scale (eHEALS) or other validated scales. We performed a random-effects model meta-analysis comparing changes in eHL levels before and after the interventions, and between the intervention and control groups. Heterogeneity was assessed using I² statistics.

Results

Out of the 504 studies retrieved, 15 studies conducted between 2011 and 2023 met the inclusion criteria. Target populations of eHL interventions included adults in 7 studies, older people in 5 and young people in 4. The meta-analysis included 10 studies that used the eHEALS. Participants showed a mean increase in eHEALS scores of 5.81 points (95% CI = 3.36–8.26, N = 1025) following the eHL interventions compared to the pre-intervention period. In the analysis between the intervention and control groups, we found a statistically significant difference in eHL improvement in favour of the intervention group, with mean eHEALS scores 3.62 points (95% CI = 1.63–5.60, N = 1258) higher in the intervention group than in the control groups. Subgroup analyses by intervention type, stratified by Collaborative Learning (CL) or Individualistic Learning (IL) showed significant increases in eHealth Literacy in the pre-post intervention analysis (CL: UMD = 5.19, CI = 0.01–10.38, N = 402; IL: UMD = 6.05; CI = 3.14–8.97, N = 623) and in the intervention vs. control analysis in the IL group (DMD = 4.98; CI = 1.77–8.12, N = 540).

Conclusions

Our findings support the effectiveness of tailored interventions in significantly enhancing eHL, providing key insights for evidence-based intervention design targeted to different population groups.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-025-21354-x.

Introduction

Advances in access to internet and technology have fostered a rise in the adoption of eHealth services, paving the way for innovative solutions to address the complex health needs of the population. Digital health technologies are defined by the World Health Organization (WHO) as “the field of knowledge and practice associated with the development and use of digital technologies to improve health”, which broadly covers any applications of digital technologies in the context of health, including the use of big data, artificial intelligence and other uses of information and communication technologies to support health [1, 2]. These technologies offer new opportunities to empower individuals in making health decisions, managing their health, meeting information needs and accessing personalised health services [3–6]. This is particularly the case for those with chronic conditions and vulnerability, who can benefit from patient portals, telehealth systems, wearable devices and health-related apps for better health outcomes and novel modes of prevention, treatment, rehabilitation and well-being [7–13].

Several barriers hinder the meaningful adoption of digital health technologies, including organisational and environmental barriers (e.g. lack of dedicated resources and funding, lack of physical access to the Internet and low information and communication technology infrastructure) and individual barriers (e.g. privacy and ethical concerns, lack of motivation, cultural resistance to the use of technology by both professionals and individuals) [9, 14–16].

Appropriate use of digital health technologies also depends on a set of competencies and skills included in eHealth Literacy (eHL). Initially introduced by Norman and Skinner in 2006, eHL was defined as “the ability to seek, find, understand and evaluate health information from electronic sources and to apply the knowledge gained to address or solve a health problem” [17]. This foundational definition conceptualized eHL through the “Lily Model,” which encompassed six core literacies: health, information, science, media, computer, and internet literacy. Subsequent studies have highlighted the need for a more dynamic and holistic understanding of eHL. For instance, in 2015, Norgaard at al. proposed the “eHealth Literacy Framework” (eHLF), which reimagined eHL not only as an individual set of skills but also as a multidimensional construct that considers the interplay between individual competencies and system characteristics. The eHLF identified seven domains that shape an individual’s ability to use and benefit from digital health technologies. These included the ability to process information (domain 1) and actively engage in one’s health (domain 2), which depend on individual capacities, as well as system-level factors like access to functional systems (domain 6) and system adaptability to individual needs (domain 7). Furthermore, the framework addressed the interaction between individuals and systems, considering how users engage with information (domain 3), experience security and control (domain 4), and adopt a positive attitude toward technology (domain 5) [18]. This expanded perspective on eHL integrated both individual and systemic dimensions, offering a comprehensive lens to understand how digital health technologies can be effectively utilized in diverse contexts.

Various studies have shown that low levels of eHL mainly affect the most vulnerable and disadvantaged populations, including those living in rural areas with limited access to technology, people with low cultural levels and individuals with low socio-economic status, contributing to poorer health outcomes and digital marginalisation for those already suffering from health inequalities [19]. Low levels of eHL are also common among older adults, although the lack of skills and knowledge to use eHealth resources could be widespread across all ages [20]. While digital health technologies promise to improve health outcomes and make health information and access more accessible, failing to address users’ eHL needs could exacerbate the digital divide [21, 22], making it imperative to identify the most effective eHL strategies.

eHL interventions take place in different settings and target different population groups, usually focusing on promoting Internet knowledge and literacy, search and evaluation skills, and the ability to apply information in practice [15]. According to some authors, eHL training should not be limited to practical skills such as teaching the use of technological devices and computer literacy, but should include broader concepts such as usability, user-centred design and participatory approach, and awareness of barriers to accessing technology [23].

Evidence regarding the effectiveness of eHL interventions remains limited, particularly in terms of their applicability across diverse population groups and the optimization of learning methods. Current studies reveal disparities in outcomes and methodologies, making it challenging to conduct a comprehensive evaluation of best practices [7, 8, 24, 25]. For instance, one systematic review highlighted the significant variability in the methods and rigor of eHL interventions, noting that these inconsistencies hinder the ability to generalize findings across different demographic groups. This limitation extends beyond older adults to include other populations, such as younger individuals and socially disadvantaged groups, where the evidence base is even less robust [8]. Another study applied an eHL lens to interventions targeting socially disadvantaged groups and identified a need for better alignment between intervention designs and the unique challenges faced by these populations [26]. The lack of standardisation of eHL interventions calls for a better understanding of the most successful features and learning strategies, and for more inclusive research involving a wider range of population groups in an effort to identify universally effective approaches tailored to the specific health needs of different communities.

Therefore, the aim of this systematic review and meta-analysis is to comprehensively retrieve, qualitatively and quantitatively pool, and critically appraise the available evidence from experimental research on the effectiveness of interventions designed to increase eHL.

Methods

We followed the Prepared Items for Systematic Reviews and Meta-Analysis (PRISMA) [27, 28]. A study protocol was developed in advance and registered in the PROSPERO database (ID number CRD42023457021). The research question, search strategy, inclusion and exclusion criteria, primary outcomes, strategy for data extraction, and data synthesis were determined in the protocol.

Search methods and inclusion criteria

Studies identified searching the electronic databases PubMed/Medline, Scopus, Web of Science, Embase, the Cochrane Library, ClinicalTrials.gov written in English with available full text, were included. No publication date restriction was adopted. The search strategy was first developed in PubMed/Medline and then adapted for use in the other databases (Supplementary Table S1). Briefly, we used a combination of free text and exploded MeSH headings, identifying: (i) ‘digital health’, (ii) ‘health literacy’ and (iii) ‘experimental study design’.

The inclusion and exclusion criteria are detailed in Table 1, according to the Population, Intervention, Comparison, Outcomes and Study Design (PICOS) framework [29, 30].

Table 1.

A priori defined inclusion and exclusion criteria according to the Population (P), intervention (I), comparison (C), outcomes (O) and study design (S), (PICOS) framework

Search Strategy	Details
Inclusion criteria	P: general population I: eHealth literacy intervention, defined as any type of intervention with the aim of increasing eHealth Literacy, implemented in both digital and traditional forms C: both comparison with any type of intervention and absence of comparators O: primary outcome: any reported measure of efficacy of the eHealth Literacy interventions studied (i.e., changes in eHealth literacy); secondary outcomes: any measurable health outcomes related to the baseline health condition of the target population indirectly affected by the effect on eHealth Literacy S: randomised and non-randomised controlled trials (i.e., experimental studies)
Exclusion criteria	P: healthcare workers S: observational studies, qualitative studies and reviews, studies not published as peer-reviewed, book, book chapter, thesis, protocol, no full-text papers
Language filter	English
Time filter	From inception through 19th of September 2023
Database	PubMed/Medline; Scopus, Web of Science, Embase, Cochrane, ClinicalTrials.gov

Studies focusing both the general population and specific subgroups, either healthy or affected by disease, of all ages and genders, were included. We only considered evidence from experimental research, including randomised and non-randomised controlled trials. We considered studies reporting original quantitative data and efficacy outcomes for different types of eHL interventions. We considered eHL as defined by Norman et al. 2006 [17] and eHL interventions as any type of intervention with the aim of increasing eHL, implemented in both digital and traditional forms, such as online or in-person training programs, online health communities, web tools, websites, tutorials. Eligible studies comprised both comparisons with any type of intervention and no comparisons.

Primary outcomes of interest were the short- and long-term results of eHL interventions, as assessed by the eHealth Literacy Scale (eHEALS) and other validated scales. eHEALS is an 8-item measure of eHealth Literacy graded on a 5-point Likert scale (subjective evaluation, with possible responses ranging from “completely disagree” to “completely agree”), aiming to assess individuals’ combined knowledge, comfort and perceived ability to find, evaluate and apply electronic health information to health problems [31]. Available data on satisfaction with the intervention, willingness to use it, quality of life results, any measurable health impact related to the condition addressed by the intervention, and changes in health-seeking behavior were considered as additional outcomes.

We did not consider studies evaluating interventions targeted at healthcare professionals. We excluded observational studies, qualitative studies and reviews, studies not published as peer-reviewed, book, thesis, protocol, and opinion papers (i.e., editorials, narrative reviews, commentaries, and letters to the Editor) not providing original data. Systematic reviews were also excluded but screened to retrieve relevant original studies.

Study selection, data extraction and quality appraisal

Identified studies were independently reviewed for eligibility in a two-step process; a first screening was performed by seven authors based on the title and abstract (C.B, E.M., C.G., M.B., M.L., T.B, G.P.V). Then, full texts were retrieved for a second screening by four authors (E.M., C.G., M.B., M.L.). At both stages, disagreements among reviewers were resolved by consensus and discussed with the eighth and ninth authors (A.O., M.D.). Data were independently extracted by five authors (E.M., R.B., C.G., M.B., M.L.), using an ad-hoc developed data extraction spreadsheet. The data extraction spreadsheet was piloted on five randomly selected papers and modified accordingly. Data extraction included: full reference details, country of study conduction, study design, study period, sample size, study population details, including gender and mean age. In addition, a comprehensive description was provided of the characteristics of the intervention, including type of administration (digital or face-to-face), duration and staff involved.

Outcomes of interest, including both eHL levels and other health literacy outcomes, were extracted, with details of the assessment tools used and the definitions considered. Quantitative results were reported separately for each group (intervention and control, if present), providing data at baseline and after any intervention administered. Measures of the effect of the comparison between the intervention and control groups were also specified separately, when available. We reported results as mean difference or standardised mean difference followed by the corresponding standard deviation (SD) when continuous scales were used to assess the effectiveness of the intervention in improving eHL, as in the case of the eHealth Literacy Scale (eHEALS).

Corresponding authors were contacted by e-mail in case of incomplete or unavailable data. Quality appraisal of included studies was carried out by applying the Cochrane risk-of-bias tool for randomised trials (RoB 2) [32] and the Risk Of Bias In Non-randomised Studies - of Interventions tool (ROBINS-I) [33], for randomised or non-randomised controlled trials, respectively.

Data pooling and meta-analysis

We performed descriptive analysis to report and pool the characteristics of included studies using means and standard deviations, separately for intervention (eHL interventions) and control groups, if applicable. A meta-analysis was conducted on all available outcome data on eHL, as measured by the eHealth Literacy Scale (eHEALS). In our analysis, we considered the overall scores obtained from the eHEALS, both as reported in the included studies and as calculated based on the individual items available. We expected variability between studies in terms of methodology and population, we therefore applied random-effects meta-analyses, rather than assuming a single true value in a fixed-effects approach [30]. Two analyses were conducted to assess the difference in eHL levels, as measured by the eHEALS, between pre- and post-intervention, and between the intervention and control groups. Pooled effect sizes for the effectiveness in improving eHL were calculated using the unstandardised mean difference (UMD) with corresponding 95% confidence intervals (95%CI).

Heterogeneity was assessed using the I² statistic and visual inspection of forest plots. We assessed publication bias with funnel plot visual inspection [32] and the Egger test [34]. A “trim and fill” method was used if publication bias was detected to estimate potential missing studies which contribute to the funnel plot’s asymmetry [35, 36]. This method assumes that the most extreme effect size studies have not been reported, biasing the overall effect size estimates [37].

Meta-regression analysis was conducted to investigate sources of heterogeneity and to evaluate the associations between the effect size and potential modifier variables, including subject characteristics (age and level of eHL at baseline). Meta-analyses and meta-regression were performed using ProMeta3^® (Internovi, Milan, Italy) software.

Subgroup analyses were conducted to identify which strategies are most beneficial among different population groups, according to the type of learning approach, whether individualistic (referring to those learning modes in which participants work independently to achieve their goals), or collaborative (referring to various group-based approaches and activities aimed at promoting both individual and collective learning) [38, 39]. Differences in pooled mean differences between the studies reporting subjects receiving Individualistic (IL) or Collaborative Learning (CL) eHL interventions were calculated using the two samples Z-test on RStudio^® software (Posit Software). Statistical significance was set at p = 0.05 for all analyses.

Results

A total of 887 studies were identified by searching the selected databases and listing references of relevant articles. After removing duplicates, 504 records were retrieved. Papers were screened and selected as illustrated in Fig. 1. Following the title and abstract screening, 419 records were excluded, while 70 were excluded after being assessed for eligibility in the full screening process. This resulted in 15 papers meeting the pre-defined inclusion criteria and included in the review [23, 40–53].

Fig. 1.

Flow diagram of the studies selection process. From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71)

The characteristics of the included studies and the study populations examined, together with a description of the eHL interventions administered, are presented in Tables 2 and 3.

Table 2.

Characteristics of included studies

Reference	Country	Study design	Study period	Population		Outcome
				eHL intervention group (intervention group)	usual care group (control group)
Bevilacqua et al., 2021 [23], International Journal of Environmental Research and Public Health	Italy	Experimental non-randomised	NA	n = 58, older adults, 34 M / 24 F, Age mean (SD): 68.2 (5.0)	NA	eHL (eHEALS) Other outcomes: Experiences and feelings toward technology (Survey of Technology Use - SOTU)
Carroll et al., 2019 [40], Journal of General Internal Medicine	USA	Pragmatic RCT	July 2014 - March 2017	n = 180, persons living with HIV, 112 M / 68 F, Age mean (SD): 51.7 (10.7)	n = 179, persons living with HIV, 101 M / 75 F, 3 transgenders, Age mean (SD): 51.2 (11.3)	eHL (eHEALS) Other outcomes: Patient Activation Measure (PAM) Health status (SF-12 mental; SF-12 physical); HIV Adherence Self-Efficacy (HIV Adherence Self-Efficacy scale); Undetectable viral load
De la Hoz et al., 2021 [41], International Journal of Environmental Research and Public Health	Spain	Quasi experimental	2016/2017- 2017/2018	n = 42, university students studying bioscientific training subjects, 14 M / 28 F	NA	eHL (eHEALS) Other outcomes: Zika virus (only post-test) with four open questions about scientific knowledge in health
Güven et al., 2020 [42], Epilepsy & Behavior	Turkey	RCT	November 2017 - April 2018	n = 35, youth with epilepsy (10 focal seizure type, 22 generalized seizure type, 3 both seizure type), (3 more than 1 seizure in a week, 7 more than 1 seizure in a month, 16 more than one seizure in a year, 9 less than 1 seizure in a year). Time to get an epilepsy diagnosis: 6–12 months: 14, 13–36 months: 7, diagnosis 37–72 months: 10, diagnosis 73 month or more: 4, 16 F / 19 M, Age mean (SD) = 13.4 (2.72) n = 35 parents, 30 F / 5 M, Age mean (SD): 40.1 (4.50)	n = 34, youth with epilepsy (2 focal seizure type, 30 generalized seizure type, 2 both seizure type), (0 more than 1 seizure in a week, 4 more than 1 seizure in a month, 17 more than one seizure in a year, 13 less than1 seizure in a year). Time to get an epilepsy diagnosis: 6–12 months: 8, 13–36 months: 11, diagnosis 37–72 months: 10, diagnosis 73 month or more: 5, 15 F / 19 M, Age mean (SD) = 13.1 (2.36) n = 34 parents, 23 F / 11 M, Age mean (SD): 39.6 (5.38)	eHL (eHEALS) Other outcomes: SSES-C (Seizure Self-Efficacy Scale for Children 9–17 years), PeIF (Personal Information Form) Epilepsy Knowledge Test (EKT): CATIS (child attitude toward illness scale), WAMMI (website analysis and measurement inventory), PIF (Parental Identification Form), Parental Anxiety Scale for Seizures (PASS) Pediatric Epilepsy Medication, Self-Management Questionnaire (PEMSQ), Quality of Criteria for Consumer Health Information (DISCERN)
Lyles et al., 2019 [43], Journal of the American Board of Family Medicine	USA	RCT	From June to October 2016 Recruitment April to July 2016 and follow-up period July to December 2016	i) In-Person training arm n = 44, English-speaking patients with 1 + chronic diseases, 23 F / 21 M, Age mean (SD): 56.3 (10) ii) Take-Home training arm n = 49 English-speaking patients with 1 + chronic diseases, 25 F / 24 M, Age mean (SD): 52.5 (14)	n = 2,462 Usual care comparison group	eHL (4-item eHEALS) Other outcomes: Portal log-in (yes/no) 3 to 6 months post-training, assessed via the electronic health record. Self-reported attitudes and skills collected via baseline and follow-up surveys.
Mitsuhashi, 2018 [44], PeerJ	Japan	RCT	Rrecruitment from 14/09/2017 to 19/09/2017 Baseline questionnaire survey was administrated from 29/09/2017 to 03/10/2017 EG was exposed to e-learning program for 14 days, from 10/10/2017 to 23/10/2017 Follow-up online questionnaire from 23/10/2017 to 30/10/2017	n = 148 after the e-learning program 134 complete follow-up survey, 14 did not respond, Inclusion criteria: agreement to participate, interest in e-learning, and interest in health literacy, 74 F / 74 M, Age mean (SD): 40.2 (9.9)	n = 153 after the e-learning program 148 complete follow-up survey, 5 did not respond. Inclusion criteria: agreement to participate, interest in e-learning, and interest in health literacy, 74 F / 77 M, Age mean (SD): 40.2 (10.2)	eHL (eHEALS) Other outcomes: HEL (Healthy Eating Literacy scale), five-item scale that measures interactive and critical literacy about healthy diet.
Nokes et al., 2019 [45], Computers, informatics, nursing	USA	Cluster randomised trial	July 2013 - November 2014	i) Intervention 1 n = 50, members of a treatment program for low-income persons living with HIV (≥ 18 years old, English-speaking) 19 F / 31 M, Age mean (SD): 49.32 (10.46). 86% African American, 10% Hispanic-Latino, 54% more than High School, 44% AIDS diagnosis ii) Intervention 2 n = 50, members of a treatment program for low-income persons living with HIV (≥ 18 years old, English-speaking) 22 F / 28 M, Age mean (SD): 50.58 (8.98). 78% African American, 22% Hispanic-Latino, 52% more than High School, 34% AIDS diagnosis	NA	eHL (eHEALS)
Roh et al., 2023 [46], International Journal of Environmental Research and Public Health	Korea	RCT	During COVID-19 quarantine period	n = 62, female college students, Age mean (SD): 19.4 (0.5)	n = 58, female college students, Age mean (SD): 19.5 (0.6) No group differences in the baseline general characteristics	eHL (eHEALS) Other outcomes: Exercise Self-Schemata (ESS) questionnaire Health Behavior Scale (HBS)
Sanders et al., 2020 [47], Patient education and counseling	USA	RCT	2014–2017	n = 180, English speaking, > 18 years old, with confirmed HIV diagnosis	n = 179, English speaking, > 18 years old, with confirmed HIV diagnosis	eHL (eHEALS) Other outcomes: BEHKA, HIV screening tool with score range 0–8, 1 point for every correct answer: low HIV literacy (0–3), marginal HIV literacy (4–5), adequate HIV literacy (6–8). REALM, validated screening tool for adult literacy in health care settings, each participants has to read 21 medical terms aloud: 0–13 inadequate literacy, 14–18 marginal literacy, 19–21 adequate literacy. NVS screening tool, it includes a nutrition label and 6 questions related to it
Spindler et al., 2022 [48], mHealth	Denmark	RCT	NA	n = 67, adults with heart-failure-related hospitalization within the previous 2 weeks, 51 M / 16 F, Age mean (SD): 61.73 (10.75)	n = 70, adults with heart-failure-related hospitalization within the previous 2 weeks, 54 M / 16 F, Age mean (SD): 61.36 (11.46)	eHL (eHealth Literacy Questionnaire - eHLQ)
Vazquez et al., 2023 [49], JMIR AGING	USA	RCT, 3 × 2 × 3 mixed factorial design	NA	i) CL intervention n = 233, older adults, 75 M / 158 F, Age mean (SD): 70.1 (6.7) ii) IL intervention n = 233, older adults, 79 M / 154 F, Age mean (SD): 70.8 (7.6)	NA	eHL (eHEALS) Other outcomes: computer and web knowledge (ad hoc questionnaire) basic computer and web operation (ad hoc questionnaire) information-seeking skills (ad hoc questionnaire) website evaluation skills (ad hoc questionnaire)
Xie, 2011a [50], Journal of the American Society for Information Science and Technology	USA	Experimental, non-randomised study	August 2009 - June 2010	n = 172, older adults, 50 M / 122 F, Age: 70.4 (8.0)	NA	eHL (eHEALS) Other outcomes: eHealth literacy supplemental measures: (i) perceived usefulness of the internet in helping make health decisions; (ii) perceived importance of being able to access health resources on the internet
Xie, 2011b [51], Journal of the American Society for Information Science and Technology	USA	RCT, 2 × 2 × 2 mixed factorial design with learning method (CL, IL) and presentation channel (visual only, visual plus auditory)	September 2010–February 2011	n = 124, older adults, 82 F / 36 M, Age: 68.15 (9.00)	NA	eHL (eHEALS)
Xie, 2011c [52], Journal of Medical Internet Research	USA	RCT, 2 × 2 mixed factorial design with learning method (collaborative; individualistic) The present study is a part of the Electronic Health Information for Lifelong Learners (eHiLL) research project	February to May 2011	n = 146, older adults, 44 M / 96 F, Age mean (SD): 69.99 (8.12) i) CL intervention n = 72 ii) IL intervention n = 74	NA	eHL (eHEALS)
Zaim et al., 2021 [53], Health Literacy Research and Practice	USA	Quasi experimental (no randomisation nor control group)	2017–2018 (quasi-experimental in summer 2018)	n = 27, teen girls (13–18 yo), 40.7% White, 33.3% African American, 26% Asian, Native Hawaiian, American Indian, or Multiracial 27 F Age mean (SD): 16.1 (1.7)	NA	eHL (eHEALS) Other outcomes: acceptability of the tool

Abbreviations: CL = Collaborative Learning,; eHEALS = eHealth Litearcy Scale; eHL = eHealth Literacy; F = Females; IL = Individual Learning; M = Males; NA = Not Available; RCT = Randomised Controlled Trial; USA = United States of America, yo = years old

Table 3.

Description of intervention and outcomes reported in the included studies

Reference	Intervention		Effect size
	eHL intervention	usual care	eHL intervention Mean (SD)	usual care Mean (SD)	eHL intervention vs. usual care
Bevilacqua et al., 2021 [23], International Journal of Environmental Research and Public Health	Content: 5 modules: (i) raising awareness of eHealth literacy; (ii) digital health apps and skills; (iii) practicing new skills; (iv) social communication; (v) self-evaluation and sustainability of the improvement. Method: online sessions using the GoToMeeting platform Duration: sessions of 90 min, over 4 weeks Type of intervention: IL	NA	Pre (baseline): eHEALS: 24.3 (8.9) Post (last session): eHEALS: 28.4 (8.1) Pre-post: a statistically significant improvement (p < 0.001)	NA	NA
Carroll et al., 2019 [40], Journal of General Internal Medicine	Content: basic HIV literacy, development of basic eHealth competency, use of ePHR, and how to ask questions Method: (i) use of a smart device (Apple Ipod Touch) and a customized electronic Personal Health Record developed for activation in person living with HIV named URHealth pre-loaded on each device; (ii) six group training sessions co-facilitated by staff coaches and trained peer educators; (iii) subsequent individual coaching session (to reinforce skills learned) Duration: group training sessions of 90 min; individual sessions of 20–30 min Type of intervention: CL	Usual care All control group participants received the iPod device after their follow-up evaluation was completed)	Pre (baseline): eHEALS: 28.53 (7.75) Post (12-month outcomes): eHEALS,: 29.81 (1.45)	Pre (baseline): eHEALS: 27.27 (8.52) Post (12-month outcomes): eHEALS,: 27.67 (1.42)	Generalized estimating equation (GEE) models for comparing changes of 2 independent groups Improvement in eHEALS: 2.67 (95% CI = 1.38–3.9)
De la Hoz et al., 2021 [41], International Journal of Environmental Research and Public Health	Content: knowledge and use of digital resources (such as search engines and online bibliographic databases) in Spanish and in English Method: active cooperative methodology (Aronson’s jigsaw technique). After instruction of students, they were divided into groups of 4 members with 4 functions distributed. Each member became an expert of one resource and then shared his experience and learning with others Duration: same intervention in both academic years for three hours Type of intervention: CL	NA	Pre (baseline): eHEALS: overall, 23.93 (4.59); female, 24.36 (4.92); male, 22.85 (4.35) Post (end of intervention): eHEALS: overall, 34.55 (2.69), female, 34.93 (2.99); male, 33.43 (2.62) Pre-post: statistically significant differences (p < 0.05) between the pre-test and post-test results, both globally and by sex.	NA	NA
Güven et al., 2020 [42], Epilepsy & Behavior	Content: web-based epilepsy education program, content evaluation with DISCERN scale for (i) youth with epilepsy and (ii) their parents Method: access to the website, with weekly reminders to use it and direct contact every 2 weeks for technical support Duration: 12 weeks Type of intervention: IL	Usual care in an outpatient clinic with visits at the clinic every 1 to 3 months without the provision of any educational materials (After the post test, the website was made available to the youth in the control group and their parents)	i) Youth with epilepsy Pre (baseline): eHEALS: 2.92 (0.97) Post (after 12 weeks): eHEALS: 3.85 (0.86) Pre-post: Significant increase in eHEALS (p < 0.05) ii) Parents Pre (baseline): eHEALS mean (SD): 2.61 (0.97) Post (after 12 weeks): eHEALS: 3.72 (0.49) Pre-post: Significant increase in eHEALS (p < 0.05)	i) Youth with epilepsy: Pre (baseline): eHEALS: 2.98 (0.86) Post (after 12 weeks) eHEALS: 2.99 (0.85) Pre-post: eHEALS increased but was not statistically significant ii) Parents: Pre (baseline): eHEALS: 2.89 (0.74) Post (after 12 weeks): eHEALS: 2.91 (0.70) Pre-post: eHEALS increased but was not statistically significant	Independent Samples t-test between EG and CG, comparing the means of 2 independent groups: i) Youth with epilepsy eHEALS: t = 4.14 (p = 0.0001) ii)Parents: eHEALS: t = 5.69 (p < 0.0001)
Lyles et al., 2019 [43], Journal of the American Board of Family Medicine	Content: training curriculum with simple instructions and 11 how-to videos for accessing MYSFHEALTH i) In-Person training arm Method: in-person tutorial with a trained research assistant who prompted participants to log into the learning platform and guided them in accessing the training materials Duration: 4 in-person meetings Type of intervention: IL ii) Take-Home training arm Method: article handout with a link to the training materials and an outline of the steps for accessing the training curriculum Type of intervention: IL	Nonrandomized usual care comparison group EHR data of all patients who had visited the 2 primary care clinics during the recruitment period	Overall trial sample (In-Person Training Arm and Take-Home Training Arm): Pre (baseline) 4-item eHEALS 14.4 (3.7) Post (3–6 months follow-up): 4- item eHEALS 16.2 (2.4) Pre – post (t-test): Significant increase in eHEALS p < 0.001)	NA	NA
Mitsuhashi, 2018 [44], PeerJ	Content: e-learning content created by the researcher on (1) reliability of information on the Internet, (2) scientific research methods, and (3) cautions regarding health information posted on social networking websites, included 4 optional quizzes Method: the content was presented to the participants in simple Japanese to facilitate comprehension Duration: two-week period with about 10 min of dedicated application to learning the content per day Type of intervention: IL	Without e-learning Type of intervention: none	Pre (baseline): eHEALS: 24.5 (6.61) Post (Follow-up): eHEALS: 26.8 (5.84)	Pre (baseline): eHEALS: 25.9 (6.18) Post (Follow-up): eHEALS: 26.6 (5.63)	Baseline: delta_score_change = 1.57; 95% CI = 0,09–3,05; p = 0.037 Follow-up: delta_score_change = 0.250; 95% CI = 0.01–0.48; p = 0.037 statistically significant difference in eHEALS between EG and CG
Nokes et al., 2019 [45], Computers, informatics, nursing	Content: use of electronic health information on HIV/AIDS i) “MEDLINE” intervention Method: i) first session for watching the video “Evaluating Internet Health Information”, ii) at-home assignments with instructions to look for HIV health information Web sites, using 6 criteria, without help from family, iii) second session with revision of the at-home assignments, iv) additional 15 min with the expert HIV nurse clinician, where the skills learned in the video were reinforced Duration: 1 week between first and second session Type of intervention: IL ii) “E-HELP” intervention Same intervention without additional 15-minutes meeting with expert (HIV nurse clinician who reinforced video content) Type of intervention: IL	NA	Overall sample Pre (baseline): eHEALS: 23 (7.14) Post intervention: eHEALS: 28 (6.39) i) “MEDLINE” Intervention Post intervention: eHEALS: 30.19 Post (at 1 week after intervention): eHEALS: 31.6 Pre-post: Statistically significant differences in eHEALS before and after session (df = 98; t = -5.020; p = 0.000) and at 1 week after intervention (df = 9 8; t = -7.740; p = 0.000) ii) “E-HELP” Intervention Post intervention: eHEALS: 28.74 Post (at 1 week after intervention): eHEALS: 25.4 Pre-post: Statistically significant differences in eHEALS of CG before and after session (df = 98; t = -7.140; p = 0.000) and at 1 week after intervention (df = 98; t = -4.720; p = 0.003) Between the two interventions there is a significant difference post intervention (P = 0.047), not at 1 week after intervention (P = 0.545)	NA	NA
Roh et al., 2023 [46], International Journal of Environmental Research and Public Health	Content: eHL education program on (i) health-related knowledge building education for acquiring basic information on positive health behaviors (e.g., physical activity, healthy eating, and sleep), (ii) eHL experience-building practice for allowing students to find, assess, and utilize reliable health resources online through a practice program focused on promoting eHL. Method: weekly 2-hour online sessions during physical education (PE) classes Duration: six weeks Type of intervention: IL	Participation in the existing vinyasa yoga PE class curriculum (vinyasa yoga participation and education program) provided by university PE instructors, without receiving any information and contents of the eHL education program Type of intervention: none	Pre (baseline): eHL: 3.36 (0.68) Post (after PE classes): eHL: 3.82 (0.59)	Pre (baseline): eHL: 3.53 (0.82) Post (after PE classes): eHL: 3.59 (0.68)	Change in eHL: Main effect for time: F(1, 118) = 8.277, p = 0.005, partial η2 = 0.066 Main effect for the group: F(1, 118) = 0.064, p = 0.800, partial η2 = 0.001 Interaction effect of the group and time: F(1, 118) = 4.765, p = 0.031, partial η2 = 0.039 The EG showed a significant increase in eHL compared to the CG after the intervention.
Sanders et al., 2020 [47], Patient education and counseling	Content: activation and empowerment of HIV patients in asking their clinician questions through training on how to use a handheld, smart, web-enabled wireless device (Apple iPod TouchTM), introduction to the internet and email, training in use of the HIV personal health record app Method: six, weekly, in-person, 90-minute, group-training sessions, co-led by peer coaches, with the use of a digital device Duration: six weeks Type of intervention: CL	Any specific training Type of intervention: none (Control group received their device after the intervention was complete)	Pre (baseline): eHEALS: 28.53 (7.75) Post (immediate follow-up, T1): eHEALS: 32.23 (5.59) Post, T2 (up to six months after they completed T1): eHEALS: 32.35 (5.29)	Pre (baseline): eHEALS: 27.27 (8.52) Post (immediate follow-up, T1): eHEALS: 28.76 (7.62) Post, T2 (up to six months after they completed T1): eHEALS: 29.09 (7.27)	eHEALS improved significantly between EC and CG, calculating using t-test. At T1: eHEALS: MD = 3.47 (p < 0.001) At T2: eHEALS: MD = 3.26 (p < 0.001)
Spindler et al., 2022 [48], mHealth	Content: educational material on life with heart failure Method: 3 phases: (i) education and titration of medicine (0–3 months); (ii) telerehabilitation in healthcare center (3 months); (iii) everyday life with telerehabilitation (6 months) Duration: 12-month Type of intervention: IL	Traditional rehabilitation program Type of intervention: none	Pre (baseline): not collected Post (6 months): eHLQ1 = 2.98 (0.66) eHLQ2 = 3.21 (0.43) eHLQ3 = 3.11 (0.70) eHLQ4 = 3.22 (0.42) eHLQ5 = 3.12 (0.54) eHLQ6 = 3.07 (0.50) eHLQ7 = 2.89 (0.61) Post (12 months): eHLQ1 = 3.01 (0.60) eHLQ2 = 3.27 (0.44) eHLQ3 = 3.2 (0.56) eHLQ4 = 3.27 (0.47) eHLQ5 = 3.15 (0.50) eHLQ6 = 3.14 (0.37) eHLQ7 = 2.98 (0.54)	Pre (baseline): not collected Post (6 months): eHLQ1 = 2.71 (0.65) eHLQ2 = 3.08 (0.43) eHLQ3 = 2.87 (0.66) eHLQ4 = 3.17 (0.46) eHLQ5 = 2.79 (0.60) eHLQ6 = 2 0.90 (0.54) eHLQ7 = 2.66 (0.68) Post (12 months): eHLQ1 = 2.78 (0.70) eHLQ2 = 3.1 (0.51) eHLQ3 = 2.90 (0.71) eHLQ4 = 3.22 (0.51) eHLQ5 = 2.86 (0.61) eHLQ6 = 2.97 (0.54) eHLQ7 = 2.78( 0.62)	At 6 months: statistically significant differences in i) eHLQ1 ‘using technology to process health information’ EG:2.98(0.66) vs. CG:2.71(0.65), (p = 0.04); ii) eHLQ5 ‘motivated to engage with digital services’ EG:3.12(0.54) vs. CG:2.79(0.60), (p = 0.00) No significant changes from 6 to 12 months on any of the eHQL subscale across the two groups
Vazquez et al., 2023 [49], JMIR AGING	i) CL intervention Content: series of web-based interactive tutorials developed for this study with specific instructions and activities developed for the CL intervention group Method: 2-hours sessions held twice a week, where first the participants watched the tutorial twice with a 5-minute break in between; then, they were given a handout to perform practice activities. To encourage collaboration, groups of 2 or 3 participants shared a computer so that all members of a group could proceed at the same pace Duration: 4 weeks Type of intervention: CL ii) IL intervention Content: series of web-based interactive tutorials developed for this study with specific instructions and activities developed for the IL group Method: 2-hours sessions held twice a week, where first the participants watched the tutorial twice with a 5-minute break in between; then, they were given a handout to perform practice activities. Participants wore headphones and worked on their computers during the entire intervention to avoid interaction with peers Duration: 4 weeks Type of intervention: IL	NA	Overall sample Pre (baseline): eHEALS: 19.86 (8.08) Post (end of training): eHEALS: 33.37 (4.68) Post (6 months follow-ups): eHEALS: 32.06 (5.93) Pre-post eHL improves pre-post intervention in CL and IL (p < 0.001) Post to follow-up From post to follow-up, statistically significant decreases were found in eHL (p < 0.001)	NA	NA
Xie, 2011a [50], Journal of the American Society for Information Science and Technology	Content: toolkit containing 9 modules (modules 1–5 focused on NHISeniorHealth; modules 6–8 focused on MedlinePlus; module 9 focused on improving the ability to appraise health information) Method: small classes (no more than 8 participants per class) that met in a library site twice a week, 2 h each time, in order to look at the toolkit with one computer for each participant Duration:4 weeks Type of intervention: IL	NA	Pre (baseline): eHEALS: 2.59 (0.83) Post intervention: eHEALS: 4.04 (0.49) Pre-post: Significant difference from pre to post in eHEALS (p < 0.001)	NA	NA
Xie, 2011b [51], Journal of the American Society for Information Science and Technology	Four experimental conditions: (i) IL / visual; (ii) IL / visual plus auditory; (iii) CL / visual; (iv) CL / visual plus auditory Content: 16-minute-long tutorial developed by the National Library of Medicine of the National Institutes of Health on the key issues for evaluating the quality of Internet health information. Method: Phases of intervention: - explain learning method - watch the tutorial - brief reflection - rewatch the tutorial − 10-minute practice (individualistic learning condition) or group discussion/reflection (collaborative learning condition) aided by the second set of discussion questions Duration: 1 session Type of intervention: IL, CL	NA	No significant three- or two-way interaction effect among learning method, information presentation channel, and time of measurement on the outcome measures. e-Health literacy efficacy, e-Health literacy skills and perceived usefulness of e-health literacy skills increase significantly from pre to post intervention (p < 0.001 in all three cases; power = 1.00 for both eHL efficacy and eHL skills and 0.98 for perceived usefulness of e-health literacy skills, all at the alpha = 0.05 level)	NA	NA
Xie, 2011c [52], Journal of Medical Internet Research	(i) CL and (ii) IL intervention Content: 4 modules training focused on basic computer terms (module 1), NIHSeniorHealth.gov, a website designed to accommodate age-related changes in cognitive, physical, and sensory abilities (module 2 and 3), and evaluation of the quality of online health information (module 4). Method: Small classes (≤ 8 participants) met in a library site twice a week, 2 h each time, assigned to the CL o IL method. Duration: 4 week Type of intervention: CL, IL	NA	Pre-post: eHL improved significantly, eHEALS: F1,93 = 229.31 (P < 0.001) Cohen’s d = 2.25; statistical power (alpha = 0.1) = 1.00 Learning method had no significant effect on eHEALS	NA	NA
Zaim et al., 2021 [53], Health Literacy Research and Practice	Content: internet-based training tool for teaching teens how to search for, evaluate, and use online health information appropriately Method: instructional video with lessons and testimonials from other teen girls, an interactive game, and short video clips from an adolescent medicine physician to direct teens to the web-page during the targeted ad buy period Type of intervention: IL	NA	Pre (baseline): eHEALS: 27.1 (6.0) Post (30 days later): eHEALS: 32.6 (4.6) Pre-post: statistically significant differences (p < 0.01) between the pre-test and post-test results	NA	NA

Abbreviations:CI = Confidence Interval; CL = Collaborative Learning; eHEALS = eHealth Literacy Scale; EHR = Electronic Health Records eHL = eHealth Literacy; IL = Individual Learning; eHLQ= eHealth Literacy Questionnaire; MD = Mean Difference; NA = Not Available; PE = Physical Education; SD = Standard Deviation;

Included studies were published between 2011 and 2023, with over half (n = 8, 53%) published after 2020 [23, 41, 42, 46–49, 53]. The majority of included studies (n = 9, 60%) were conducted in the USA [40, 43, 45, 47, 49–51, 53], while others were carried out in South Korea [46], Denmark [48], Japan [44], Italy [23], Spain [41], and Turkey [42] (n = 6, 40%, 1 study for each country).

Among included studies, 11 (73%) were randomised controlled trials [40, 42–51] and 4 (27%) were non-randomised or non-control group studies [23, 41, 50, 53].

General characteristics of the study population

Overall, the sample size of included studies ranged from 27 [53] to 466 [49] subjects, with a total sample size of 2510 (males: 1158, 46%; 1352: females, 54%). The eHL interventions administered in the included studies were aimed at different population groups. 5 studies targeted elderly populations (mean age of 69.6 ± 1.1 years) [23, 49–52], 7 studies targeted adults [40, 42–45, 47, 48] and 4 studies targeted younger populations, including children, adolescents, or university students (mean age of 16.3 ± 3.1 years) [41, 42, 46, 53]. Among studies targeting adults, one study considered adults with at least one chronic illness [43], 3 focused on an HIV-affected population [40, 45, 47] and one focused on adults with heart failure [48] (mean age 54.1 ± 4.6 years). One study [44] focused on healthy adults, with a mean age of 40.2 ± 10.2 years, while another study [42] examined a population of parents with an average age of 40.1 ± 4.50 years.

Descriptive analysis and outcome measures

All included studies reported a significant increase in eHL scores, both pre-post intervention (n = 10, 66.7%) and compared to usual care (n = 6, 40%), including one study that assessed both [42].

The majority of included studies reported changes in the 8-item eHEALS (n = 13, 87%) score [23, 40–42, 44–47, 49–53] or, as in one study [43], in a short 4-item version of eHEALS. One study [48] used an alternative scale, the eHealth Literacy Questionnaire, to evaluate the effectiveness of a telerehabilitation intervention in a population of patients with heart failure, taking into account individual eHL as well as participants’ interactions and experiences with digital services, and demonstrated improved outcomes in both literacy and motivation following the intervention.

Over the 70% of included studies evaluated secondary outcomes including digital literacy and user experience outcomes (n = 2, 13% and n = 3, 20% respectively) [23, 42, 43, 49, 53] and health literacy, both general (n = 2, 13%) [40, 46] and related to a specific condition of participants (n = 5, 33%) [40–42, 44, 47]. In terms of digital literacy outcome assessment, for example, one study [23] evaluated experiences and feelings about technology through the Survey of Technology Use (SOTU), a 29-item checklist based on the Matching Person and Technology Model and divided into three subscales to explore positive, negative, or neutral experiences with technology [54]. Authors found that participants with less experience or less positive experiences with technology perceived a greater improvement in their eHL after the course. Another study assessed both eHL and computer and web literacy at different levels, from computer and web knowledge to the development of practical skills. Participants were tested on their ability to navigate websites, search for health information online, and assess the reliability of information, and showed significant improvements before and after the intervention [49]. Examples of secondary outcome assessments included the assessment of mental and physical health status, measured by the SF-12 questionnaire, allowing to monitor participants’ mental and physical well-being [40], or evaluations related to the specific condition of the participants, as regards for example to therapeutic adherence (adherence self-efficacy to antiretroviral therapy in HIV [40]). Another study considered eating literacy as measured by the Healthy Eating Literacy Scale (HEL), to assess whether the changes observed after the eHL intervention were not due to the Hawthorne effect, thus excluding the influence of the awareness of being observed on the study participants [44].

Characteristics of the eHealth literacy interventions

Of the included studies, 6 administered full web-based intervention (40%) [23, 42, 44, 46, 48, 53], while 9 employed a combination of face-to-face teaching and digital resources (60%) [40, 41, 43, 45, 47, 49–52]. Moreover, 9 studies administered Individualistic Learning (IL) interventions (60%) [23, 42–46, 48, 50, 53], 3 studies administered Collaborative Learning (CL) interventions (20%) [40, 41, 47], while 3 studies reported on both approaches (20%) [49, 51, 52].

Details of the interventions administered in included studies are reported in Table 3.

A wide variety of eHL intervention types were found. Interventions targeted to older people included online sessions on eHL and digital skills [23], e-learning content on evaluating health information [49], and small class toolkits focused on basic computer terms and online health information seeking [50, 51]. Interventions targeting younger populations aimed at use of digital resources and health information [41, 53] and curriculum-integrated eHL programmes [46]. HIV patients benefited from group training and individual coaching using smart devices or personal health records [40, 45], while patients with chronic diseases were offered both web-based educational materials and telerehabilitation programmes to manage their conditions [42, 43, 48]. For the latter group, eHL interventions focused on both eHL and health literacy related to the management of their condition.

Study quality evaluation

Quality appraisal of included studies is reported in Supplementary Table S2. Among included randomized studies, 3 had low risk of bias (27%) [42, 44, 46], 7 some concern (64%) [40, 43, 45, 47–49, 52] and 1 high risk of bias (9%) [51]. Among the non-randomised studies, 3 studies presented serious risk of bias (75%) [23, 41, 53] and one had critical risk of bias (25%) [50].

Outcomes’ pooled analysis

Quantitative pooling of effect estimates was conducted on studies with available data on changes in eHL from baseline to post-intervention and between intervention and control groups, as assessed by the eHEALS [23, 40–42, 44–47, 50, 53]. One study [42] reported separate data on two distinct populations, resulting in two datasets. Consequently, as shown in Table 4, a total of 11 datasets were included in the pre-post intervention meta-analysis and 6 data sets were included in the intervention vs. control meta-analysis. Meta-analysis were conducted on a total of 1,025 subjects in the pre-post assessment and on a total of 1,258 subjects in the intervention vs. control assessment.

Table 4.

Meta-analyses assessing eHealth literacy interventions outcomes

Type of analysis	N. of included datasets (k)	ES	95% CI, p-value	N. of participants	Chi2; df	I2	p	Intercept*	t-value*	p-value*
eHEALS difference: pre vs. post intervention
Overall°	11	5.81	(3.36; 8.26) < 0.001	1025	388.25; 10	97.42	< 0.001	-1.26	0.20	0.846
Subgroup analysis by intervention type
Collaborative°	3	5.19	(0.01; 10.38), 0.050	402	137.33; 2	98.54	< 0.001	81.19	2.10	0.283
Individualistic°	8	6.05	(3.14; 8.97) < 0.001	623	228.15; 7	96.93	< 0.001	-4.04	-0.68	0.522
eHEALS difference: intervention vs. control
Overall°	6	3.62	(1.63; 5.60) < 0.001	1258	31.32; 5	84.03	< 0.001	7.49	5.61	0.005
Subgroup analysis by intervention type
Collaborative°	2	1.54	(0.24; 2.85), 0.020	718	1.40; 1	28.48	0.237	NA	NA	NA
Individualistic°	4	4.98	(1.77; 8.19) 0.002	540	23.96; 3	87.48	< 0.001	7.18	4.93	0.039

* Egger’s linear regression test

Values are reported as unstandardised mean differences

Abbreviations: df = degree of freedom; ES = Effect Size; N. = number; NA = not available

As presented in Figs. 2 and 3, all included studies reported an increase in eHL in the pre-post intervention analysis, while in the intervention vs. control analysis a statistically significant difference was found across all the studies, except one [40].

Fig. 2.

Forest plot (A) and funnel plot (B), after trim and fill method, of the meta-analysis assessing the pooled difference in eHEALS scores in the pre- vs. post- intervention datasets

Fig. 3.

Forest plot (A) and funnel plot (B), after trim and fill method, of the meta-analysis assessing the pooled difference in eHEALS scores in the intervention group vs. control group datasets

Pooled estimates of the difference between pre- and post-intervention means indicated a statistically significant increase of 5.81 points (UMD = 5,81, 95% CI = 3.36–8.26, N = 1025) at the eHEALS. The pooled intervention vs. control analysis showed a statistically significant eHL improvement, with the intervention group showing a mean increase of 3.62 points (DMD = 3.62, 95% CI: 1.63–5.60, N = 1258) on eHEALS compared to the control group. Both pre-post and intervention vs. control analyses showed a considerable degree of heterogeneity (pre- vs. post- intervention: P < 0.001, I²= 97.42; intervention vs. control group: P < 0.001, I²=84.03).

The funnel plots resulted asymmetric at visual inspection, showing potential publication bias, confirmed by the Egger linear regression test only for the intervention vs. control analysis (pre- vs. post- intervention: intercept: -1.26, t =-0.20, P = 0.846; Fig. 2B intervention vs. control: intercept: 7.49, t =-5.61, P = 0.005, Fig. 3B). No unpublished data sets were retrieved after the trim and fill method.

A sensitivity analysis was performed to assess the effect of excluding from the intervention vs. control analysis two datasets (Guven 2020 [42]) that were identified as potential outliers by the visual inspection of the forest plot. After exclusion, the mean difference was still significant and positive, although less pronounced than in the previous analysis (UMD = 1.84, 95% CI 0.97–2.72; p < 0.001), with non-significant reduction heterogeneity as assessed by the I² test (p = 0.314; I² = 15.46). Publication bias analysis, performed by inspection of the funnel plot followed by trim and fill method, detected one study trimmed to the left of the funnel plot, but the Egger linear regression test was not significant (intercept: 5.73, t: 1.51 p = 0.270).

Subgroup analyses by intervention type, stratified by Collaborative Learning (CL) or Individualistic Learning (IL), also showed significant increases in eHL, including 3 and 8 data sets respectively in the pre-post intervention analysis (CL: UMD = 5.19, CI = 0.01–10.38, N = 402; IL: UMD = 6.05; CI = 3.14–8.97, N = 623) and 4 data sets in the intervention vs. control analysis (IL: DMD = 4.98; CI = 1.77–8.12, N = 540). Results of the intervention vs. control analysis stratified by CL interventions were not significant (Table 4; Fig. 4).

Fig. 4.

Forest plot and funnel plot after trim and fill method, of the meta-analysis assessing the pooled difference in eHEALS scores in the pre- vs. post- intervention datasets, stratifying by intervention type: Collaborative Learning (A, B), Individualistic Learning (C, D)

The results of the random-effects meta-regression, which was employed to assess the association between pre- and post-intervention outcomes and selected potential moderators (eHL levels at baseline and age), showed a significant association between eHL levels at baseline and change in eHEALS score after the eHL intervention, with larger differences in eHL levels between pre- and post-intervention for participants with lower levels of eHL at baseline (overall: slope = -0.91, P = 0.009; individualistic: slope = -0.96, P = 0.035). No further significant associations were found (Table 5; Figs. 5 and 6).

Table 5.

Meta-regression analysis to assess the association between pre- and post-intervention outcomes and selected potential moderators

Moderator	Intercept	Slope	p-value*
eHEALS difference: pre vs. post intervention (overall)
eHEALS at baseline	28.36	-0.91	0.009*
Age	4.54	0.02	0.698
eHEALS difference: pre vs. post intervention (collaborative)
eHEALS at baseline	52.91	-1.77	0.161
Age	NA	NA	NA
eHEALS difference: pre vs. post intervention (individualistic)
eHEALS at baseline	28.87	-0.96	0.035*
Age	4.49	0.04	0.496

Fig. 5.

Meta-regression analysis to assess the association between pre- and post-intervention outcomes and baseline eHealth Literacy (Overall, A; Collaborative Learning, B; Individualistic Learning, C)

Fig. 6.

Meta-regression analysis to assess the association between pre- and post-intervention outcomes and age (Overall, A; Individualistic Learning, B)

Table 6 presents a summary of the results reported in the pre-post intervention analysis, which examined changes in eHL levels stratified by Collaborative or Individualistic Learning intervention.

Table 6.

Collaborative vs. individualistic intervention eHEALS difference: pre vs. post

Outcomes	Collaborative intervention	Individualistic intervention	P value
eHEALS difference: pre vs. post°	5.19 (0.01–10.38; N1 = 3; N2 = 402)	6.05 (3.14–8.97; N1 = 8; N2 = 623)	0.5918876

Abbreviations: N1 = number of included datasets; N2 = number of participants

* Statistically significant (p-value < 0.05)

°Values are reported as unstandardised mean differences (95% confidence interval [CI])

The z-test revealed no statistically significant difference between the pre- and post-intervention pooled MD of subjects receiving CL or IL eHL interventions (z-score = 0.5361027, p = 0.5918876).

Discussion

This systematic review and meta-analysis summarized the best available evidence from experimental research on the effectiveness of eHL interventions. eHL has received increasing interest in recent years, as evidenced by more than half of the studies included being published after 2020. Our study provided a comprehensive overview of effective strategies for improving eHL, outlining a variety of approaches that tailor content and delivery to the specific needs and contexts of each population. Overall, evidence from the 15 relevant original included studies highlighted the effectiveness of tailored interventions in promoting eHL in different population groups. These findings were further supported by the random-effects model meta-analysis of 11 datasets comparing changes in eHL, as assessed by the eHEALS, before and after the interventions, and between the intervention and control groups, with a total of over 1,000 participants, respectively.

Several insights can be drawn from the results of our study on the effectiveness, modalities and content of eHL promotion strategies in different population groups.

First and more generally, in line with previous systematic reviews and meta-analyses [17, 24], our findings supported the effectiveness of eHL interventions, as participants showed a mean increase in eHEALS scores of over five points following the eHL interventions and over three points higher when compared to usual care. This demonstrated that such interventions are an effective strategy for increasing knowledge, comfort, and perceived ability to locate, evaluate, and apply electronic health information to health problems across different population groups. In addition, compared to previous studies, we further investigated the potential influence of age and eHL at baseline on the effectiveness of eHL interventions. While no significant association was observed for age, we found that interventions were more effective for participants with lower eHL at baseline. These findings have important implications for the design of new eHL interventions, as they suggest that the greatest potential for improvement lies with those with low eHL - those who could actually benefit the most - justifying efforts and resources focused on the most vulnerable groups, rather than on strengthening existing skills in highly literate individuals.

Second, with regard to modality of eHL interventions implemented, we highlighted a wide variety of settings, durations and approaches, ranging for example from online sessions [23] and e-learning content [49] to small class programmes [50, 51] or individual coaching involving smart devices and personal health records [40, 45]. The effectiveness of interventions does not appear to be related to the type of learning approach: our analysis found no statistically significant differences between individual and collaborative learning approaches, suggesting that the current experimental evidence does not support a single standard approach to improving eHL. These findings are consistent with previous studies highlighting the value of tailoring interventions to the specific needs and conditions of the target population [7, 24], and highlight the need to involve end-users in the design of patient-centred solutions [8, 55], to ensure that no health need is left behind.

Third, in terms of content, intervention designs were shown to integrate the promotion of eHL, in most cases assessed by eHEALS, with content and related assessments of digital literacy, user engagement and experience with digital tools - particularly for older people - or even specific domains of health literacy - as in the case of young people or frail patients with specific health needs, suggesting once again the importance of tailoring interventions to the needs of the targeted groups. Furthermore, several of the studies included within this systematic review addressed the evidence gap identified by previous researchers [8] by extending the scope of eHL interventions to include health impacts within the training outcome, contributing to a broader understanding of the interaction between eHL and health status and ultimately providing a more diverse view of how eHL can support improved health practices and outcomes. In addition, another lesson to be considered in future design and implementation of eHL interventions is the shown potential for greater impact when intervention content includes both general and disease-specific health literacy elements, i.e. educational components that reinforce core health literacy skills, such as understanding basic health concepts, navigating the health system and making informed health decisions, as well as more specific disease-related content, such as increased knowledge and empowerment about one’s condition. As noted in a recent systematic review and meta-analysis by Kim et al., which aimed to determine whether an individual’s level of eHL affects actual health-related behaviour, eHL may mediate the process by which health-related information drives health-related behavior change [56]. Thus, integrating these elements alongside eHL training could foster a synergistic effect, improving both general health literacy and eHL outcomes.

Our findings should be interpreted in light of some limitations. The first is the high degree of heterogeneity of our analyses, as assessed with the I² statistic, which might limit the generalizability of pooled effect estimates. This may have been due to the differences among the included studies in terms of study design, population characteristics, settings, and intervention methods. In addition, the limited number of available studies and the heterogeneity of the populations considered did not allow us to perform effectiveness analyses by population subgroups, which limited our ability to address the specific characteristics of the intervention beneficiaries. Furthermore, it must be taken into account the overall quality of the included studies. While the majority of the studies were RCTs, the overall methodological quality was not uniform. Many studies were of low quality, lacking rigorous methodology, adequate sample size, randomisation and controls, presenting a risk of bias, which means that the results must be interpreted with caution. Lastly, the majority of studies in this systematic review used the 8-item eHEALS to assess eHL levels after interventions. This is consistent with previous systematic reviews, which suggest that eHEALS is a favoured scale because of its short length, ease of administration and simple questions [25]. While another advantage of eHEALS is the widespread validation of the 8-item version in different populations and languages [57–62], as noted by previous authors [7, 25], the digital landscape has changed significantly since its development in 2006, paving the way for new tools better suited to the current more dynamic and interactive context. Of the studies included in this review, only one [48] adopted a more recent scale, choosing the eHLQ which takes into account people’s interactions with digital health services and different stakeholders [18, 63]. Thus, the use of eHEALS by most studies considered may not have fully reflected current eHealth Literacy skills due to its outdated nature.

Despite to these limitations, to the best of our knowledge, this systematic review and meta-analysis is the most updated and comprehensive summary of the available evidence on eHL interventions to date. The large total sample size of the pooled analysis (n = 1,025; n = 1,258), the use of rigorous methodology and definitions, and strict adherence to the Prepared Items for Systematics Reviews and Meta-analysis guideline are additional relevant strengths of this paper. The inclusion of only experimental evidence and comparison in the analysis of studies using the same single validated scale for eHL levels assessment facilitated comparability between studies and strengthened the validity of our results. A comprehensive literature review was carried out, using a wide range of databases (n = 6) to increase the robustness and generalisability of the findings, and a broad set of search terms using a combination of MeSH and free text words to maximize the number of studies retrieved and minimize the risk of publication bias. The high degree of heterogeneity among the studies confirmed the choice of a random-effects model. Furthermore, the meta-regression analyses performed allowed us to consider the impact of selected variables.

Conclusions

In conclusion, this systematic review and meta-analysis provides significant evidence of the effectiveness of eHL interventions, offering a solid basis to guide and inform future public health strategies. Our findings present a comprehensive overview of the interventions implemented to date to promote eHL in different population subgroups and highlight key dimensions that should be considered when designing evidence-based eHL interventions, including consideration of the target group’s baseline eHL levels and their specific needs and health conditions.

In terms of future research pathways, while many interventions show promising results, the lack of randomisation and the low quality of the available studies call for an improvement in the quality of evidence through well-designed randomised controlled trials and robust methodologies to establish consistent standards to be applied. In addition, future research should focus on the design and rigorous evaluation of tailored eHL interventions that address the specific needs of different demographic groups, particularly those most vulnerable to the digital divide. This includes improving assessment tools to capture the evolving set of skills required, and including long-term follow-up to assess the durability and scalability of eHL interventions.

As digital health rapidly evolves, eHealth Literacy is emerging as a public health goal in population health promotion, since it enables individuals to access health information and services equitably, use technology to its full potential for improved health outcomes, and navigate current and future health systems.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Abbreviations

CL

Collaborative Learning

CI

Confidence Interval

eHEALS

eHealth Literacy Scale

eHL

eHealth Literacy

IL

Individualistic Learning

PICOS

Population, Intervention, Comparison, Outcomes and Study Design

PRISMA

Prepared Items for Systematic Reviews and Meta-Analysis

PICOS

Population, Intervention, Comparison, Outcomes and Study Design

RoB-2

Risk-of-bias tool for randomised trials

ROBINS-I

Risk Of Bias In Non-randomised Studies - of Interventions tool

UMD

Unstandardized mean difference

Author contributions

Study concept and design: RB, EM; Acquisition of data (Literature search and Study Selection): EM, RB, CG, CB, GPV, TB, MB, EB, ML; Analysis and interpretation of data (literature): CB, GPV, EM, RB, AO; Meta-analysis: CB, GPV; Writing—original draft preparation: CB, EM, RB, TB, MB; Critical revision of the manuscript for important intellectual content: AO, RB, EM; Supervision: AO, MD, ML; Writing—review and editing: AO, EM, RB. CB and EM equally contributed as co-first. AO and RB equally contributed as co-last. All authors read and approved the final manuscript.

Funding

This research was supported by the National Plan for Complementary Investments to the PNRR Hub Life Science-Digital Health (PNC-E3-2022-23683267, DHEAL COM).

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Clinical trial number

Not applicable.

Competing interests

The authors declare no competing interests.