A scoping review of the use of e-learning and e-consultation for healthcare workers in low- and middle-income countries and their potential complementarity

Abstract

Objective

Although the provision of e-learning (EL) training for healthcare workers (HCWs) and provider-to-HCW e-consultation (EC) is considered useful for health outcomes, research on their joint use is limited. This scoping review aimed to create an overview of what is currently known in the literature about the use and implementation of EC and EL by HCWs in LMICs and to answer the question of whether there is evidence of complementarity.

Materials and Methods

Scientific databases were searched and peer-reviewed papers were reviewed systematically according to predefined inclusion/exclusion criteria. Data were extracted including the study focus (EC/EL), year of publication, geographical location, target population, target disease(s) under study, type(s) of study outcomes, and article type.

Results

A total of 3051 articles were retrieved and screened for eligibility, of which 96 were kept for analysis. Of these, only 3 addressed both EL and EC; 54 studies addressed EL; and 39 addressed EC. Most studies looked at gain in knowledge/skills usability, efficiency, competence, and satisfaction of HCW, or barriers/challenges to implementation. Descriptive studies focused on the application of EL or EC for targeting specific health conditions. Factors contributing to the success of EC or EL networks were institutional anchoring, multiple partnership, and capacity building of local experts.

Conclusions

Our review found an important gap in the literature in relation to the complementary role of EL and EC for HCWs in LMICs evidenced by outcome measures. There is an important role for national and international academic institutions, learned medical societies, and networks to support regional experts in providing EL and EC for added value that will help the clinical performance of HCWs and improve health outcomes.

INTRODUCTION

In many low-to-middle-income countries (LMICs), information and communication technologies (ICT) for healthcare delivery, often referred to as e-health, are being applied broadly in an attempt to fill in gaps of limited funding, a shortage of trained medical professionals, and poor infrastructure in these contexts.¹

Within the context of LMICs, most research is focused on the effects and benefits of e-health on patients and less research exists focusing on the effects and benefits for healthcare workers (HCWs).² Two important forms of e-health for HCWs are e-learning (EL) and e-consultation (EC). EL refers to the use of ICT for interaction between health professionals and trainees and can be defined as “an approach to teaching and learning, representing all or part of the educational model applied, that is based on the use of electronic media and devices as tools for improving access to training, communication, and interaction and that facilitates the adoption of new ways of understanding and developing learning.”³ Vaona et al⁴ refer to it as “any educational intervention mediated electronically via the Internet.” EL has become a growing trend for health-related teaching, and it is used for training students, professionals, and patients. EC refers to the use of ICT for interaction between health professionals. Vimalananda et al^5,6 define EC as “asynchronous, consultative, provider-to-provider communications within a shared electronic health record.” We supplement this definition with that of Rasmussen et al,⁷ to include synchronous forms of EC that take place in real time, usually through the means of video-conferencing, whereas asynchronous EC systems, also known as store-and-forward, allow providers to submit written questions and images to a specialist so that they can view the consult at a time convenient for them.

Despite the increasing use of e-health interventions for HCWs, the literature in this area continues to remain fragmented.⁸ Within this fragmented literature, we also hypothesize a lack of mainstream discussion on how various forms of e-health like EL and EC work in conjunction with each other. In a quest to better understand—and therefore help improve—the use of e-health by HCWs, this study is specifically focusing on the use of EL and EC as those are both tools of e-health that are commonly used by HCWs. We performed a scoping review to create an overview of what is currently known in the literature about the use and implementation of EC and EL by HCWs in LMICs and whether there is evidence of complementarity in the joint use of these 2 tools.

METHODS

A first literature search was carried out using various bibliographic databases (Embase, Medline ALL Ovid, Web of Science Core Collection, WHO GHL, SCIELO, CINAHL EBSCOhost, ERIC Ovid) in December 2019 (see Supplementary S1). The search was carried out by an expert librarian who has a PhD degree in developing searches for systematic reviews.^9,10 The search string included variations and synonyms of the following terms: e-learning, e-consultation, HCWs, and LMICs. As there is no consensus within the literature concerning these definitions, a search for similar terms that are commonly used interchangeably with e-learning and e-consultations—such as mHealth or telemedicine—was carried out as well. Subsequently, if the papers’ descriptions matched those of e-learning or e-consultation as defined in this study, the papers were included in the selection despite the differing use of terminology. The search provided us with 2837 items. During our study, an updated search of the literature was carried out in May 2020 in order to include more recent published literature. It provided us with an additional 214 items. As such, the number of papers for evaluation amounted to a total of 3051.

Inclusion and exclusion criteria

Inclusion and exclusion criteria were both established a priori to screening the literature. The inclusion criteria for this study consisted of:

• type of e-health intervention: the papers assess the use of EL or EC.

• Target population: the papers focus on the main target group “health care workers,” also including community health workers, nurses, midwives, and physicians. To refer to HCWs, papers using the following synonyms were also included: frontline workers, rural health auxiliaries, volunteer health worker, outreach worker, traditional birth attendant, community health education worker, community health agent, community health promoter, community health assistant, and health promoter.

• Geographical location: the papers pertain to LMICs as defined by the World Health Organization.

• Year of publication: papers are published in 2010 or later in order to ensure timelines and relevance to today’s e-health context.

• Language: only articles in English are included in the study.

• Article type: papers categorized as scholarly peer-reviewed articles.

The exclusion criteria for this study consisted of:

• papers that do not pertain to the use of technology for the purpose of providing learning materials to or communication between healthcare workers.

• Papers that focus on students and resident doctors.

• Papers that classify as nonacademic article types, such as study protocols, blog posts, and novels.

• Papers that are not retrievable in full text.

Selection procedure and quality assessment

After completion of the literature search, a first screening of the literature focusing on title and abstract—using the established inclusion and exclusion criteria—was undertaken by two of the authors using the EndNote software. Duplicates were automatically discarded. A contestation took place between both researchers after the first screening in order to obtain consensus on the use of the inclusion and exclusion criteria. The contestations between the research got solved through individual discussions on each paper. In order to identify whether to keep or discard a paper, the authors (AI and SD) based themselves on the definitions of EL and EC mentioned in the introduction.

A second round of in-depth screening took place based on the content of the full papers. The following data were extracted from eligible included studies: the study focus (EC/EL), year of publication, geographical location, target population, target disease(s) under study, type(s) of study outcomes, and article type. The data were grouped into categories and the number of studies belonging to each category is presented in graphs and tables.

In accordance with recent guidelines, we created a table for guidance in the inclusion decision-making process.^11,12 Article types were categorized as review studies, randomized controlled trials, nonrandomized studies, descriptive studies, and expert opinions. Papers that did not fit into any given category in the table were discarded. Papers that were included were categorized according to the corresponding section in the table. In the second round of screening, we focused on a list of the following criteria (where relevant): methods; study design; intervention; number of participants; number of controls; outcome measures, results, and follow-up; bias, limitations/strengths; and conclusions, recommendations.

The main findings from the studies were discussed, grouped by article type. Furthermore, descriptive studies are discussed with emphasis on recent accomplishments of EL and EC networks as they present evidence of joint use of EC and El in LMICs. Finally, brief paragraphs are devoted to cost-analysis and HCW perceptions of e-health as these issues have been identified as 2 common themes that repeatedly come up in the literature across the various types of studies and are often noted as crucial reasons for failure or success of implementation and longevity of projects.

RESULTS

Figure  1 shows a flow diagram depicting the selection process for included studies. The initial search in December 2019 provided us with 2837 items. The updated search in May 2020 provided us with an additional 214 items. As such, the number of papers for screening amounted to 3,051. By the end of the first screening round, 406 items were left in the inclusion list. During the full-text screening, a total of 309 articles were excluded as they did not meet the inclusion criteria after all, or of which the full text was not retrievable. Among these 309 articles were 60 papers that were not written in English, as they did not fit our language inclusion criteria. In the end, 96 studies were included in our review. Although the above suggests that 1 out of 5 studies were excluded as they were written in non-English language, it is difficult to know exactly how many non-English studies were missed as our search algorithm included English search terms only.

Figure 1.

Flow diagram depicting the selection process for included studies.

In total, 54 studies addressed EL, 39 studies addressed EC, and 3 studies addressed EL and EC. Most studies on EL and/or EC for HCWs in LMICs were published in the years 2018 (17 publications) and 2019 (14 publications) as shown in Figure  2, left panel. Most studies were performed in Africa (41) and Asia (20), and to a lesser extent Central- and South America (7). Additional studies were performed cross-continental (28) (Figure  2, right panel). The 3 studies on both EL and EC published in 2013, 2015, and 2017 were descriptive studies conducted in Botswana, Sudan, and Tanzania assessing the main outcomes of sustainability, user experience, user perception, efficiency, acceptability, and uptake, focusing on women’s health/radiology/oral medicine/dermatology; family medicine; and maternal/perinatal care, respectively.

Figure 2.

Left panel: year of publication. Studies on EL + EC count in both categories. 2020 is limited until May 2020. Right panel: geographical distribution of the studies. EC, e-consultation; EL, e-learning.

The main medical areas in which the use of EL and EC were described were infectious diseases (14) mostly focusing on HIV/AIDs and also including other conditions, such as tuberculosis. In addition maternal health (14), and NDCs (12) including conditions, such as diabetes and cancer. It is important to note that only 46 out of the 96 studies had a specific disease focus. Regarding the outcomes under study, we found that most studies looked at the gain in knowledge/skills of HCWs (39), followed by barriers/challenges to implementation (33), impact on patient care (28), usability (20), efficiency (17), competence (15), and satisfaction (13) of HCWs (Figure  3). These categories are non-exclusive, and many studies studied a range of the outcomes when assessing the use of the technology.

Figure 3.

Number of EL and EC studies by the type of main outcome under study. EC, e-consultation; EL, e-learning.

As shown in Figure  4, among the 54 studies on the topic of EL, we found 10 scoping and systematic review studies, 8 randomized controlled trials, 2 nonrandomized studies with 2 groups, 5 nonrandomized studies with 1 group, 22 descriptive (pilot) studies, and 7 expert opinions. Among the 39 studies on the topic of EC, we found 6 review studies, 3 randomized controlled trials, 1 non-randomized studies with 1 group, 24 descriptive (pilot) studies, and 5 expert opinions. The 3 studies that looked at both EC and EL were all descriptive studies.

Figure 4.

Number of EL and EC studies by study type. EC, e-consultation; EL, e-learning.

We provide an overview of the study type and focus (EL/EC), year of publication, geographical location of the study, target population, aim, and outcome for all reviewed studies in Supplementary S2 and S3. Below, the main findings from the studies are discussed grouped per type of study.

Review studies

Recent scoping and systematic review studies suggest that EL and EC for HCWs in LMICs are becoming of increasing importance (Supplementary Tables S1A and S1B). Of the 16 review studies, 10 studies did not specify a particular disease focus, 3 on infectious diseases, 2 assessed interventions targeting noncommunicable diseases (NCDs), and 1 on maternal health. The types of outcomes assessed by these studies included: types of use of EL/EC,^13–19 effects on healthcare delivery,^19–23 barriers to implementation.^{13,14,16,18,20,24}

Two systematic reviews addressed both EC and EL among HCWs in LMICs but did not discuss their complementarity.^24,25

Three scoping studies by the same study group,^13,14,21 reviewed 35, 24, and 88—partially overlapping—international studies about HCW EL training. They reported significant variability between ongoing training programs (aimed at improving knowledge, skills, and behavioral change) as most teaching was carried out through in-person tutored courses, and only a small minority of the studies reported the use of EL technologies to deliver or assist ongoing training activities. It should be noted that most studies in the review investigated programs, which combine a range of mobile tools and broadly include any type of activities aimed at facilitating HCW-learning activities (eg, applications that support work-force management and serve as a job-aid tool at the same time). Although digital and blended education may improve knowledge, skills, and comfort of HCW compared to traditional learning, the evidence is heterogeneous and often limited and of low quality.^17,18

A common factor found to be associated with successful development of EC programs was program integration with existing systems applying simple stable and easy-to-use technology. Moreover, the collaboration with international institutions was successful as it reduces the burden on HCW’s workload.²⁶ Smith et al²⁷ reviewed the evidence on how primary HCWs in LMICs obtain health information during EC to support decision-making for prescribing. The authors conclude that internet-based sources are useful when provided content that can be downloaded for offline use and easily updated when there is internet access, though a lack of up-to-date medical information was an important challenge.

Randomized controlled trials

Three studies focused on EC and 8 on EL, though no study mentions the combination of EL and EC (Supplementary Table S1C). The common measures of outcomes assessed by these studies included feasibility,²⁷ gain of knowledge and self-study behavior,^28–34 and effects on healthcare delivery.^{29–32,35–38} An important issue that comes to light in many studies is the importance of differentiating assessment of interventions in terms of effective and successful implementation by HCWs and meaningful outcomes for patient health.³⁵

Nonrandomized trials

Of the 8 studies, 7 focused on EL and 1 on EC (Supplementary Tables S1D and S1E). The measures of outcomes assessed by these studies focused on feasibility and increase of knowledge,³⁹ as well as the effects on healthcare delivery, improving access to expert cardiological consultations,^40,41 and early referral for oncological treatment providing considerable economic benefits not only to the patient but also to the HCW.^42,43 These studies underline the importance of the existence of inherent preferences, in particular for EL, and the importance of form, mode, and length of the instructional content in terms of user engagement.^44–46

Descriptive studies and expert opinion

Most descriptive studies focused on the application of either EL or EC for targeting specific health conditions such as infectious diseases (mostly HIV and TBC),^47–55 maternal and child care,^56–64 and pediatrics (Supplementary Tables S1F and S1G).^65–68

Additional targeted health conditions where EL or EC has been used include cardiology,^41,69 neurology,⁴⁴ dermatology,^70,71 and oncology.^72,73 Our search did not find studies related to other relevant types of health conditions in this area according to Winters,¹⁴ such as mental health, trauma, and disabilities.

Some studies just focussed on ICT tools and evaluation of EL and EC.^{105,108–116}

Telemedicine networks are large-scale EC interventions falling into our definition of EC.^65,74–81 These networks implement EC tools for humanitarian purposes in various countries and their beneficial effects are well documented.^79–83 The performance of seven of these networks was evaluated by Wootton et al.^79–83 It is of specific interest that at least 2 of these 7 networks also provided EL activities; however, these two forms of interventions are not discussed in the same papers, hence compatibility and complementarity cannot be assessed.

Similarly, large-scale EL interventions in the form of networks also form an important part of the literature landscape. EL networks have the common aim of providing remote training options for HCWs in rural communities lacking access to care and with limited opportunities for advancement.^40,85,86 With respect to the development of these tools, there is a need for contextually appropriate training of suitable trainees with relevance to the disease burden and available resources. Moreover, continuous interaction is essential between users and developers evaluating all aspects of usability such as effectiveness, efficiency, and satisfaction.^86–89

As the production and maintenance of online EL courses, whether in the format of small private online courses (SPOC) or as massive open online courses (MOOC), is resource expensive, there is a wide gap between high- and low-to-middle-income countries in terms of access to these courses.^90,91 In order to meet the need of dissemination of information at affordable costs and also to apply rigorous economic evaluation methods to test the components EL and EC in implementing long-term efficiency gain, collaboration is essential between various stakeholders, such as the World Health Organization, international medical learned societies, academic institutions, and national health authorities.^91,92

Main factors contributing to the success of EC and EL networks are institutional anchoring, multiple partnership, and capacity building of local experts. However, as mentioned prior, the question as to whether there are complementary effects and potential added values of joint use EL and EC particularly evidenced by outcome measures is not addressed. Summaries of studies of long-running EL and EC networks are provided in Supplementary S4.

Cost evidence and cost-effectiveness

Cost analysis has often been overlooked in the past but is becoming increasingly more mentioned in this line of research.^26,76,92,93 Although the studies from the last decade show a lack of consistency of methodology for cost analysis, they support the notion that EL and EC are more cost-effective compared to traditional forms of learning and consultations, both from a HCW and patient perspective.⁹⁴ For example, Sissine et al⁹⁵ found that blended EL resulted in savings of up to 42% due to decreased classroom time reducing the costs associated with travel, trainers, and classroom costs. However, although digital and blended education may improve knowledge, skills and comfort of HCWs compared to traditional learning, the evidence is heterogeneous and often limited and of low quality.^17,18

HCW perceptions of EL and EC

There are multiple reviews and studies addressing HCWs perceptions of the relevance of EL and EC and challenges faced by HCWs in LMICs.^{16,26,96–104}

Most studies investigate perception in relation to the use of mobile phones. Major benefits for HCW have been found to be the feasibility of using mobile phones for improving performance, workflow and the work environment, as well as reducing unnecessary referrals. Generally, the EC and EL services are viewed very positively. However, some common challenges are mentioned and include: technological readiness in terms of access and training to adopt new e-health solutions, weak technical support, issues of internet connectivity and cost. In addition, behavioral readiness, such as expected ease, reliability, and effectiveness and satisfaction of using telemedicine systems are important. Furthermore, motivating HCWs—many of whom are volunteers—is important for the sustainability of integrated community case management programs. The incentivization of local health professionals can be promoted through the provision of inter-university diploma’s and remote training/CME certificates.^76,87 Investing in efforts to improve knowledge of community members and recognition of HCW contribution to community health may have a significant impact on HCWs’ motivation and retention in their role.¹⁰⁴

DISCUSSION

This scoping review found that in the course of the last 10 years, the volume of studies in the English literature on the use of EL and EC for HCW in LMICs has increased with some more papers on the topic of EL than of EC. We presume that the greater number of EL articles is due to the fact that assessing EL is easier and more straightforward as outcome criteria, such as gain in knowledge or skills can be better quantified where outcome of EC is harder to evaluate. The majority of the reviewed research originated from Africa and Asia and to a lesser extent from Latin America. Target diseases under study concentrated around maternal and child health and infectious diseases, in particular HIV/AIDS, and to a lesser extent noncommunicable diseases, such as diabetes. Apparently, studies related to additionally relevant types of health conditions as reported by Winters et al,¹⁴ such as mental health, trauma, and disabilities are under-represented. The proportion of systematic and scoping reviews as well as experimental studies was limited, which means that high quality evidence of effectiveness aspects of EC and EL in LMICs is scarce. The majority of the EL studies focused on knowledge and skills development of HCWs, the usability of tools and the barriers of use. Many studies were descriptive reports on pilot studies. As they may not progress past the pilot stage even the term “pilotitis” has been coined.^17,18

In a recent comment on the WHO Digital Health Guidelines 2019 (https://www.who.int/reproductivehealth/publications/digital-interventions-health-system-strengthening/en/; 2019), Labrique et al¹⁰⁶ conclude that the effect and certainty of effect range of provision of EL training to HCWs and similarly provider-to-provider telemedicine (EC) are very low to moderate at best on health system strengthening. It should be noted that studies on EL and EC networks reporting surveys assessing HCWs satisfaction are generally positive.^66,77,84 It is well recognized that implementation studies and large-scale deployment are context-specific, expensive and labor-intensive. Therefore, Barteit et al¹⁰⁷ suggested developing structured programs including a standardized evaluation framework and topic-specific database, registering all EL and EC interventions. In addition, as aspects of cost-effectiveness of EL and EC over traditional learning and consultation are less well described, it would be relevant to also conduct more research on the effectiveness and cost-effectiveness of EC and EL in LMICs in order to provide an evidence-base for large-scale pragmatic roll-out of these technologies.

In this review, we addressed the question whether there might be potential complementarity and added value from the use of EL and EC. Although the identified scoping and systematic reviews suggest that EL and EC targeted at HCWs in LMICs have become important electronic tools with undeniable merits on their own, they are seldom used in combination with each other and discussions about their potential complementarity are lacking. Similarly, our review of trials and descriptive studies of EL or EC networks demonstrate positive outcomes for HCWs but fail to discuss and assess the effects of joint implementation or complementarity.

As pointed out by Peiris,³⁵ it is common in outcome studies that implementation of EL and EC are deemed to be successful if knowledge scores and utilization logistics improve, with often no mention or consideration of health outcomes. Therefore, a well-designed implementation and evaluation strategy would be required. We postulate that explicitly offering EL in combination with EC will enhance usability of EC, as EL provides background and in-depth understanding of underlying pathophysiology and evidence-based management which will improve effectiveness, efficiency and moreover satisfaction and empowerment within the community.¹⁰⁴ In fact, medical specialty societies are uniquely placed to deliver balanced, disease-oriented and patient-centered EL education and provide expert consultation. Moreover, they are well equipped to maintain professional standards and to offer appropriate accreditation.⁹⁰ Research on the complementarity of EL and EC is therefore urgently required, especially evaluating effectiveness in terms of health outcomes.

Limitations of the study

Although we searched several academic databases, there may be a bias of not including papers published in journals listed in other databases. We limited our search to papers published in English, and we noticed that studies from Central and South America are underrepresented as they are probably more often published in Spanish or Portuguese. Furthermore, results of relevant programs and networks currently operative may not have been published yet, as e-health is a rapidly developing field of research. Finally, we focused our review specifically on EL and EC for HCWs in LMICs, which carries the consequence that some studies on mHealth which do not clearly create a distinction between EL and EC may have been omitted. Despite the limitations, we believe that the studies included in this review provide a comprehensive overview of the scope of the English-language literature regarding EL and EC in LMICs.

CONCLUSIONS

The outcomes of this scoping review show and echo the notion that EL and EC have become important electronic tools for HCWs in LMICs. EC and EL are often provided in the form of mHealth technology and generally provide the main benefits of expanding services and access of HCWs to communication and knowledge. EL improves their professional knowledge and skills, particularly if simple, stable and easy-to-use technology is applied and technical problems can be circumvented. EC enables provision of specialized care through supervision and evaluation provided remotely. Although studies generally reported positive effects of EL and/or EC interventions, many were considered to be of poor methodological quality and failed to show effects on patient outcome. Moreover, our review finds an important gap in the literature in relation to the complementary role of EL and EC. As e-health and mHealth technologies continue to expand, an insight into how to utilize the joint use of these tools is needed to maximize potential benefits and provide cost-effective and meaningful interventions.

There is an important role for national and international academic institutions, learned medical societies and networks to support regional experts in providing EL and EC for added value that will help the clinical performance of HCWs and improve health outcomes.

AUTHOR CONTRIBUTIONS

All authors were involved in the conceptualization and design of the study. AI and SD conducted the searches, screened the references, extracted the data, assessed the quality of included studies, and jointly analyzed the data. SCvK and PGMdJ provided extensive comments on the data analysis. All authors were involved in the writing of the manuscript and reviewed the final draft of the review.

ETHICS APPROVAL

No ethics approval was required.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.