Abstract
Background
Despite acceleration of remote consulting throughout the COVID-19 pandemic, many health care professionals are practicing without training to offer teleconsultation to their patients. This is especially challenging in resource-poor countries, where the telephone has not previously been widely used for health care.
Objective
As the COVID-19 pandemic dawned, we designed a modular online training program for REmote Consulting in primary Health care (REaCH). To optimize upscaling of knowledge and skills, we employed a train-the-trainer approach, training health workers (tier 1) to cascade the training to others (tier 2) in their locality. We aimed to determine whether REaCH training was acceptable and feasible to health workers in rural Tanzania to support their health care delivery during the pandemic.
Methods
We developed and pretested the REaCH training program in July 2020 and created 8 key modules. The program was then taught remotely via Moodle and WhatsApp (Meta Platforms) to 12 tier 1 trainees and cascaded to 63 tier 2 trainees working in Tanzania’s rural Ulanga District (August-September 2020). We evaluated the program using a survey (informed by Kirkpatrick's model of evaluation) to capture trainee satisfaction with REaCH, the knowledge gained, and perceived behavior change; qualitative interviews to explore training experiences and views of remote consulting; and documentary analysis of emails, WhatsApp texts, and training reports generated through the program. Quantitative data were analyzed using descriptive statistics. Qualitative data were analyzed thematically. Findings were triangulated and integrated during interpretation.
Results
Of the 12 tier 1 trainees enrolled in the program, all completed the training; however, 2 (17%) encountered internet difficulties and failed to complete the evaluation. In addition, 1 (8%) opted out of the cascading process. Of the 63 tier 2 trainees, 61 (97%) completed the cascaded training. Of the 10 (83%) tier 1 trainees who completed the survey, 9 (90%) would recommend the program to others, reported receiving relevant skills and applying their learning to their daily work, demonstrating satisfaction, learning, and perceived behavior change. In qualitative interviews, tier 1 and 2 trainees identified several barriers to implementation of remote consulting, including lacking digital infrastructure, few resources, inflexible billing and record-keeping systems, and limited community awareness. The costs of data or airtime emerged as the greatest immediate barrier to supporting both the upscaling of REaCH training and subsequently the delivery of safe and trustworthy remote health care.
Conclusions
The REaCH training program is feasible, acceptable, and effective in changing trainees’ behavior. However, government and organizational support is required to facilitate the expansion of the program and remote consulting in Tanzania and other low-resource settings.
Keywords: remote consultation, mobile consulting, digital health, telehealth, mHealth, eHealth, mobile health, health care, cascade, train the trainer, low- and middle-income countries, rural areas, Tanzania, Kirkpatrick, consultation, training, low- and middle-income, rural, COVID-19
Introduction
Background
Essential health services are not available for over a third of the world’s population, and most of this population is in low- and middle-income countries (LMICs) [1]. Marginalized communities, including those living in rural areas and informal settlements or slums, have least access to high-quality health care [2]. High-quality care includes appropriate and timely treatment and follow-up [2], and its provision forms part of the United Nations Sustainable Development Goal for health [3].
Even prior to the COVID-19 pandemic, remote consulting was considered to have the potential to increase access to quality health care, especially in rural communities [4-6]. It is estimated that 85% of individuals across LMICs own a mobile phone [7]. Although Tanzania has lower rates of ownership, it still has 75% mobile phone ownership across the population and 90% among health workers [8]. Mobile phone ownership is lower amongst rural, older, illiterate, and female populations compared to other population groups but is rapidly increasing [4,7]. Patients find remote consulting acceptable and appreciate the consistency and continuity of care achieved through improved communication [9].
From the beginning of the COVID-19 pandemic, the World Health Organization recommended remote consultation using phones or videoconferencing as an option for protecting the safety of patients and health workers and to enable continued health care provision [10,11]. Worldwide, in the face of the pandemic, remote consulting increased but often with little preparation and training [12]. This lack of training in the use of health technology is a key barrier to the acceptance and uptake of remote consulting in LMICs [4,13], along with health workers’ worries about increasing personal workload [9].
Worldwide, continuing medical education delivered remotely has been shown to be acceptable, feasible, and desirable [14]. It enables greater geographic accessibility and time flexibility [15] and has been shown to be as effective as traditional teaching methods and far more effective than no training [16,17]. Issues of network connectivity, costs of data/airtime, access to electricity, and usability of the device are challenges that need to be addressed [18].
This paper first describes a remotely delivered education program called remote consulting in primary health care (REaCH) aimed at equipping health care workers in LMICs with knowledge, skills, and confidence to conduct remote consulting. We then present a 2-phase approach to evaluation: (1) a pretest phase to establish technical and face validity, and (2) our feasibility study of the delivery of the REaCH training to registered health workers and its cascade to other health workers, and the perceived impact of training on the delivery of health care remotely.
REaCH Training Program and Its Development
REaCH training aims to equip health workers with an understanding of the variety, benefits, challenges, and organizational changes associated with remote consulting and the skills for implementation of remote consulting in their health care facilities. The training was developed in partnership between St Francis University College of Health and Allied Sciences (SFUCHAS) (Tanzania), King’s College London (KCL, UK), and the University of Warwick (UK). The REaCH training, and a sample presentation of the training materials, can be freely accessed on a not-for profit basis at the Warwick Medical School website [19].
The REaCH training, developed in April and May 2020, is designed for registered health workers (eg, nurses, doctors, clinical medical officers) with access to smartphones, at least intermittent access to Wi-Fi, and an ability to learn in English. We refer to these trainees as tier 1 trainees. They engage in self-directed learning using written and video materials. Activities and assignments are included, which encourage trainees to apply what they learn to their local context. Training materials are in English and can be downloaded as PDF files where network access is challenging. A facilitator introduces the 8-module course to the trainees and interacts with them via a social media platform to discuss the learning and assignments. Each module is designed to take 1-3 hours. The facilitator supports these tier 1 trainees to cascade their learning to health workers in their local team (tier 2 trainees) using the train-the-trainer approach. It is left to the discretion of the tier 1 trainees to decide what learning to cascade to the tier 2 trainees. Tier 2 trainees need to own a feature phone (ie, no internet or up to 2G enabled). In our pilot, the learning cascade was completed in the local language, Swahili.
The content of each module is described in Table 1. REaCH is delivered via Moodle [20], an open-source blended-learning app. For the facilitated discussions, in our pilot, we used WhatsApp (Meta Platforms) [21] as it was popular locally and content is encrypted; trainees did not share patient information on the group. An information and communication technology (ICT) officer provided telephone support to trainees when they encountered difficulty with Moodle and suggested solutions when internet access was difficult (eg, travelling to a local village to download the materials).
Table 1.
REaCHa modules.
| Module | Description |
| Introduction | Why is remote consulting important? |
| 1 | What devices and platforms are used in remote consulting? |
| 2 | How does my role change and the care I provide my patients? |
| 3 | What are the risks and benefits of remote consulting? |
| 4 | What patient outcomes can I expect, including limiting COVID-19 spread? |
| 5 | What new issues arise in remote consulting that are different from face-to-face care? |
| 6 | What is my plan for delivering my work remotely (and that of my team/colleagues)? |
| 7 | How can I evaluate my own remotely delivered health care practice (and that of my team/colleagues)? |
| 8 | How can I influence others to change to remote consulting? |
aREaCH: remote consulting in primary health care.
We used the talent, resource, alignment, implementation, and nurturing (TRAIN) framework to optimize our train-the-trainer approach [22]. The facilitators who delivered the tier 1 training and the tier 1 trainees themselves were health professionals willing and able to train others (talent). We provided airtime and internet for facilitators, and each tier 1 trainee received £60 (US $74.30) for airtime and internet (resource). We provided tier 1 trainees with a certificate of course completion so they could add this to their training portfolio (alignment). Embedded within the REaCH training are teaching and activities related to implementation of remote consulting and how to cascade learning (implementation). There is opportunity for the tier 1 trainees to maintain contact on social media after the course for peer support (nurturing).
The facilitator is supported by a facilitator’s guide incorporating pedagogical principles underpinning the course, logistics, expectations, and tips to optimize trainee engagement. The learner is provided with a guide covering learning expectations, how to seek help, how to organize cascade training, and other logistical issues.
In July 2020, we pretested the first iteration of the REaCH Moodle course to establish technical and face validity with university-based professionals, 11 from SFUCHAS and 1 from the United Kingdom. The test demonstrated that it was possible and acceptable to use Moodle for delivering the course.
Based on feedback from this test, we included the WhatsApp group for facilitator support, developed the facilitator and trainee guides, and notes on how to cascade each module, an introductory video, and the option of downloading course materials as PDF files to enable studying to continue when digital access was interrupted. This version of REaCH was used in the second iteration feasibility study (August 2020) described in this paper. During this period, we obtained funding to support the airtime requirements of leaners to undertake and cascade training and deliver remote consultations to their patients. This timeline is presented in Table 2.
Table 2.
Timeline: REaCHa development, training process, and feasibility study.
| Period | Training phase | Details |
| April-May 2020 | Training development |
|
| July 2020 | Pretesting |
|
| July-August 2020 | Adaptation |
|
| August-September 2020b | Feasibility study (as reported in this paper) |
|
| ||
| August 2020-March 2022 | Trial |
|
aREaCH: remote consulting in primary health care.
bData collected and analyzed in this paper.
Feasibility Study Objectives
The feasibility study objectives were to evaluate the trainees’:
Response to REaCH training, their engagement levels, and their perceptions of the content and process (reaction)
Perceptions of their level of understanding of the topic, including knowledge, skills, and attitudes, after undertaking the training (learning)
Intended changes in how they deliver health care after completing the training and how the training contributed to this (behavior)
Methods
Study Design
In this feasibility study, we implemented and then evaluated the REaCH training using a survey, qualitative interviews, and documentary analysis. Our study was informed by Kirkpatrick’s model [23] for assessing informal and formal learning (Figure 1).
Figure 1.
Summary of 4 levels of Kirkpatrick’s model.
We assessed the reaction and self-reported learning and intended behavior change.
Ethical Considerations
We used the Frascati definition of research, as summarized by the University of Warwick, to determine whether this study was considered research. We considered it not to be research, as its purpose was testing and standardization [24].
We subsequently checked our decision using the UK Medical Research Council and National Health Service (NHS) Health Research Authority tool for assessing ethics review, which indicated we did not need ethical review [25].
We received permission from the district medical officer (Ulanga District) for the participants (health care workers) to participate in the training and its evaluation.
Trainees and Setting
Tier 1 trainees were enrolled from health facilities in Ulanga District of remote rural Tanzania. Ulanga has a population of 265,203, with 1 hospital, 2 health centers, and 23 dispensaries [26,27]. Tier 1 trainees were selected using purposive and referral sampling and fulfilled the following criteria: they consulted with patients, worked in a rural area, owned a smartphone or computer, had access to Wi-Fi, were prepared to include remote consultations by phone as part of their health care practice, and were willing to cascade training to 7 other health workers in their team.
Tier 2 health workers were enrolled by the tier 1 trainees. They had to consult with patients, own a feature phone, and be prepared to add remote consultations by phone to their health care practice. The training was delivered between August 10 and September 2, 2020. All trainees received information about the evaluation and verbally consented to it.
Data Collection
The Survey
Questionnaires were developed by authors TM and SP for completion by trainees after each module and at the end of the training. These were structured around Kirkpatrick’s model of learning [25], with Kirkpatrick’s second and third levels (learning and behavior) assessed by self-report. We asked trainees about the process of undertaking the training (dichotomous questions and open-ended questions) and about their satisfaction (reaction), learning, and any intended changes to health care delivery as a result of the training (behavior) using 5-point Likert scales. A link to the survey was emailed to trainees and completed via SurveyMonkey [28].
Qualitative Interviews
The facilitator, all tier 1 trainees who completed the training, and, from each of their groups of tier 2 trainees, a convenience sample of 2 tier 2 trainees were invited for in-depth semistructured interviews. These were conducted by telephone following completion of the training by a researcher (TM) experienced in qualitative methods. Interviews explored participants’ perceptions and experiences of the training and their views about remote consulting. Each interview was recorded digitally, transcribed verbatim, and translated by this researcher.
Documentary Analysis
We collated WhatsApp texts and emails between facilitator and trainees and reports written by tier 1 trainees after they had cascaded the training to tier 2 trainees. The tier 1 trainees reported on their experiences of cascading training, including topic selection, duration of training, preparedness for teaching and learning, how they motivated the tier 2 trainees, and advantages and disadvantages of the REaCH Moodle training approach.
Data Analysis and Trustworthiness
The survey results were analyzed descriptively. Interview transcripts were independently coded and analyzed thematically [29] by 3 team members (authors TM, BC, AD). Coding disagreements were resolved through discussion within the wider research team. TM analyzed the written documents thematically [29]. The research team held weekly debriefing meetings to reflect on the training and evaluation, identify/respond to challenges, share insights, and collectively make sense of the data [30]. We triangulated and integrated our findings in interpretation [31].
Results
Trainees
In total, 12 tier 1 trainees were enrolled within the REaCH training program, 3 (25%) women and 9 (75%) men. Tier 1 trainees were predominantly senior medical figures in participating health facilities (mostly doctors or assistant medical officers). In addition, 63 tier 2 trainees received cascaded training. The tier 2 trainees included a variety of health practitioners in the region: clinical officers, nurses, medical attendants, community health workers, and pharmacists, as well as 3 (5%) laboratory technicians and 2 (3%) radiologists, who were anecdotally delivering remote consulting. Of the 63 tier 2 trainees, 24 (38%) were women. Trainee characteristics are presented in Table 3. Of the 12 tier 1 trainees, all completed the training; however, 2 (17%) faced delays due to difficulty with internet connection and subsequently did not complete the evaluations or the cascading process. In addition, 1 (8%) tier 1 trainee faced personal circumstances, which precluded them from completing the cascading process. Thus, 9 (75%) tier 1 trainees cascaded their training to tier 2 health workers in their teams (N=63).
Table 3.
Trainee characteristics.
| Cadres | Tier 1 trainees | Tier 2 trainees | ||||
|
|
Enrolled (N=12), n (%) | Training completed (N=12), n (%) | Training cascaded (N=9), n (%) | Enrolled (N=63), n (%) | Training completed (N=61), n (%) | |
| Medical doctors | 6 (50) | 6 (50) | 5 (56) | 1 (2) | 1 (2) | |
| Assistant medical officers | 5 (42) | 5 (42) | 3 (33) | 2 (3) | 2 (3) | |
| Clinical officers and assistant clinical officers | N/Aa | N/A | N/A | 18 (29) | 18 (30) | |
| Pharmacists | 1 (8) | 1 (8) | 1 (11) | 3 (5) | 3 (5) | |
| Community health workers | N/A | N/A | N/A | 2 (3) | 2 (3) | |
| Radiologists | N/A | N/A | N/A | 2 (3) | 2 (3) | |
| Laboratory technicians | N/A | N/A | N/A | 3 (5) | 3 (5) | |
| Nurses | N/A | N/A | N/A | 27 (43) | 25 (41) | |
| Medical attendants | N/A | N/A | N/A | 5 (8) | 5 (8) | |
| Gender (female) | 3 (25) | 3 (25) | 3 (33) | 24 (38) | 23 (38) | |
aN/A: not applicable.
The Survey: Training Process Questionnaires
Survey questions about the process of training were completed by 9 (75%) of 12 tier 1 trainees; 3 (25%) trainees were unable to complete the survey due to a poor internet connection.
All 9 responding tier 1 trainees had studied in their own personal time; 7 (78%) said that they also studied during working hours, 8 (89%) had completed all 8 REaCH modules, and 1 (11%) respondent completed 5 modules. All completed the assignments associated with the modules studied. Respondents spent 1-3 hours studying per module. Of the 9 respondents, 5 (56%) completed these modules in the allocated 6-day time frame, while the other 4 (44%) completed it within 8 days. Delays were due to busyness, device and network challenges, and initial low technological competence. All respondents found the assistance from the ICT officer and facilitators to be effective and timely.
The Survey: Reaction, Learning, and Behavior Questionnaires
Survey questions on reaction, learning, and behavior [23] were completed by 10 (83%) of 12 respondents (see Multimedia Appendix 1). There was little disagreement with the questions. All 10 respondents agreed that the training was useful and that facilitation was sufficient and timely. All 10 respondents appreciated the online and WhatsApp method of teaching and found that the learning outcomes were realistic and achievable. In addition, 9 (90%) of 10 respondents would strongly recommend this type of training to other health care workers.
Every respondent reported receiving the skills needed to learn remote consulting and to apply these skills to their jobs. They each reported already using the training in their daily work and being able to train other health care workers in this content.
Qualitative Interviews
Telephonic interviews were carried out with the tier 1 training facilitator, 9 (75%) tier 1 trainees, and 16 (25%) tier 2 trainees. Interviews lasted between 15 and 30 minutes each, with tier 1 interviews conducted in English and tier 2 interviews in Swahili. See Multimedia Appendix 2 for the interview question scaffold.
We present the results under the following themes: perceptions of the REaCH program, challenges encountered during the training, learning from REaCH training, how the training could be improved, and trainees’ views on implementation of remote consulting into their routine practice. Trainees are labeled according to their tier of training and the order in which they were interviewed, as follows:
Tier 1: participant A, B, C,…
Tier 2: participant AA, BB, CC,….
Perceptions of the REaCH Training Program
Overall, the trainees appreciated the program and recommended continuation and expansion among their peers.
I wish to congratulate the initiators of this program. I would recommend this knowledge to be taught in the health colleges so that we now begin to recruit new doctors with high experience in remote consultation.
Tier 1, participant D
Generally, the participants perceived the course as a good course, something [that] is also a success.
Tier 1, facilitator
Challenges Encountered During the Training
Over half of the tier 1 trainees reported challenges with their digital technology, including storage capacity of smartphones, low technological competence, and network challenges.
There is no stable internet connection in this area, and this was 1 of the challenges I faced during the training. Ooh, likewise, the mobile network we use in our area is not stable.
Tier 1, participant E
Nevertheless, trainees found the assistance from ICT personnel and facilitators to be effective and timely.
There were several technical challenges and some issues concerning the arrangements of the modules; therefore, we used to seek instruction from the facilitator and ICT personnel, [and] actually, they were responding as soon as possible. We were told how to download the modules and the way we could go about reading them.
Tier 1, participant A
Although some trainees found it difficult to schedule the training around their work, others appreciated the flexibility of the online training.
The shortage of enough time…I spent many hours at work, so I had to make sure I read the modules in my extra time.
Tier 1, participant D
The training time planning was well arranged because it allowed us to engage in learning at any daily time.
Tier 2, participant CC
Learning From REaCH Training
All participating health care workers felt that their knowledge increased and that their behavior had changed since the training program. Some trainees were learning about remote consultation for the first time.
Yes, there are some changes; as you know, the modules have insisted on practicing remote consultation instead of face-to-face consultation, which we only trusted before. Recently, we have noticed that remote consultations are also appropriate, and actually, this alternative will work properly…!
Tier 1, participant E
Many remarked upon the usefulness of remote consulting during COVID-19.
Actually, this would assist much during this time of COVID-19 spread because it avoids the chance for having physical interaction between the patients and doctors.
Tier 1, participant F
Over half of the interviewees reported paying attention to privacy and confidentiality during remote consultations.
I do the remote consultation in a professional way by making sure I ask for consent, ensuring privacy as well as keeping their records, and making sure I continue to make follow-up on the patient's progress.
Tier 1, participant A
Trainees reported talking to patients before they attended clinic and following them up by phone rather than face-to-face. This included conducting remote consultations for patients who hesitated to attend face-to-face consultations out of fear of stigmatization.
I have started to offer advice to patients with shameful diseases remotely. You know the patients with gonorrhea can feel free to talk to a health care provider remotely rather than face-to-face.
Tier 1, participant H
Trainees had worked out how to bill for remote consultations.
I enjoyed this learning style because I have discovered that through this…this can give us an extra alternative to get money!
Tier 1, participant D
Recommendations for Improvement of the REaCH Training
Tier 1 trainees recommended a face-to-face meeting at the beginning of the course and additional time at the start to familiarize themselves with Moodle.
My advice is this…we should be making face-to-face meetings at least once at the beginning of a course that will be helpful in making things more clear! We can be taught physically on how to go through the Moodle and students’ forum as well.
Tier 1, participant D
The facilitator agreed that additional time was needed at the start of the course for familiarization and supported the tier 1 trainees in including this during cascading. The facilitator supported some tier 1 trainees in producing printed materials for the tier 2 trainees. However, we found that many of the tier 2 trainees had smartphones and received the online materials easily.
The first thing that has been successful in this training [is] the majority of us received the learning materials on time simply because we have got smartphones through which we received them.
Tier 2, participant MM
The facilitator was keen to see the addition of incentives to engagement, such as accreditation and payment for time spent undertaking the training.
Implementation of Remote Consulting in Routine Practice
To further apply their learning in practice, trainees said they needed airtime and internet packages, suitable electronic devices, and improved infrastructure.
Our digital devices are not modern ones...we should be assisted with the internet packages to support online processes during the moment of interacting with the remote clients.
Tier 1, participant B
We can’t provide the remote consultation if the supporting infrastructures like mobile networks are not working very well. Therefore, the government should ensure all necessary infrastructures for remote consultations.
Tier 1, participant I
Trainees emphasized the importance of governmental recognition to ensure adequate compensation for their work.
This needs some money…workers should be paid for this extra duty.
Tier 1, participant D
They recognized the need to inform the community about remote consulting.
Moreover, we need to make the community be aware and recognize this kind of consultation.
Tier 1, participant I
The lack of pharmacies and pathology laboratories in rural areas was identified as a barrier to successful remote consulting.
First of all, it will be difficult to make a physical examination, and the second challenge will be a shortage of pharmacies in remote areas, something [that] will make the remote clients fail to get medicines after consultation.
Tier 2, participant CC
The government should allow individuals to establish laboratories in remote areas. You know there are many laboratories that have been stopped due to the fact that they don’t meet the eligibility requirements. So, we should have enough laboratories in rural areas so that clients may have test[s].
Tier 2, participant DD
Trainees noted that some members of the community would not have easy access to a phone, as they are owned by the heads of families and sharing phones can reduce confidentiality.
You know most of the mobile owners in the family level are heads of the families; therefore, the other family members will not be free enough to use those phones. So far, sharing phones will reduce the confidentiality of the clients’ information.
Tier 2, participant MM
Older members of the community were unlikely to afford a phone, and there were community members who were illiterate and so unable to use text messaging.
Most of the community members, especially elders, are not possessing mobile phones, so they can’t make consultations by themselves without asking the help from their neighbors.
Tier 1, participant J
Generally, this is a good idea, but I am doubting whether the elders will afford to pay the costs for remote consulting.
Tier 2, participant NN
So far, some of them are not aware of reading and writing; therefore, they can’t send a text to the doctor when required to do so.
Tier 1, participant J
Some trainees were concerned with how to keep records of remote consultations.
We are still lacking the best alternative to keep the remote clients’ records…we should find how to solve this challenge.
Tier 1, participant B
One said there needed to be a different way of referring patients between health care workers when they were consulting remotely.
To make referrals to remote clients, there should be an alternative for facilitation referrals from junior to senior HCWs [health care workers].
Tier 1, participant L
Qualitative findings from the interviews with tier 1 and 2 trainees were compared with analysis of the reports on the cascading process by tier 1 trainees.
Documentary Analysis: Tier 2 Training Process
Tier 2 training was completed over 3 days, with 2 pretraining preparation days where tier 1 trainees informed the tier 2 trainees about the aims of the course, its contents, and the training style and answered any questions. Tier 1 trainees selected modules 1, 2, 3, 5, and 7 for cascade as they were deemed to be the most clinically relevant (Table 1).
Introduction to training occurred via phone conferences and WhatsApp chats, and learning was primarily conducted through smartphone and featured phones with phone calls, texts, phone conferences, WhatsApp message group, and emails.
Soon after receiving the modules, the [tier 2] trainees started learning independently…when issues could not be understood, they used to make calls and send texts for more discussion and elaboration.
Tier 1, participant E
Where there were unstable internet connections, tier 2 trainees traveled to their nearest colleagues to pick up the module PDF files or to a nearby area with a stronger internet connection.
The participants from the areas with [an] unstable internet connection were advised to move to the areas with [an] internet connection in order to download the materials.
Tier 1, participant A
Of the 63 tier 2 trainees enrolled, 61 (97%) trainees completed the course, with 53 (87%) completing it within 3 days, while 2 (3%) did not complete the course due to personal reasons.
One participant’s child got sick during the week of training that made her fail to complete the training in time.
Tier 1, participant F
Modifications used to ensure engagement included using reminder texts and phone calls to gauge and maintain attention of the trainees, using group discussions to increase teamwork, and conducting face-to-face conversations when the trainees and trainers were working in the same health facilities.
Sometimes, we were sending texts through the phone and WhatsApp media to remind them about the discussion time.
Tier 1, participant D
I also used to put some question in [the] WhatsApp platform to assess the trainees’ understanding.
Tier 1, participant F
Participants’ chats in WhatsApp assisted to assess the participation rate. You know, we were making calls to [the] training facilitator once per day to report on cascading progress and share the technical experience.
Tier 1, participant B
Modifications to solve logistical issues included translating the training documents into Swahili to overcome language barriers, providing downloadable materials that trainees could access from nearby villages when they had unstable internet connections, and moving group calls to early morning and evening hours to avoid working hours.
All in all, when I posted learning materials on the WhatsApp media, I tried to elaborate in Swahili to make them understand the contents.
Tier 1, participant A
Discussion
Principal Findings
This feasibility study found that remotely delivered professional REaCH training [19] using the Moodle app supported by cascade training infrastructures is technically and pedagogically feasible and well received by trainees in rural Tanzania. They were satisfied with the course and would recommend the program to other health care workers (reaction). They expressed that they learned skills needed to remotely consult within the health system, including how to bill patients for the consultations, and they were able to cascade the teaching (learning). Trainees reported confidently implementing remote consulting and increased understanding of topics, such as medical ethics of remote consulting and behavior change theory (behavior) [23].
Barriers to remote consulting implementation identified by our trainees include lacking digital infrastructure and few resources, inflexible billing and record-keeping systems, and limited community awareness about remote consulting. Having local technical support for learners proved invaluable to delivery and receipt of training. The greatest immediate barrier to supporting both the upscaling of REaCH training in LMICs and subsequently the delivery of safe and trustworthy remote health care is the cost of the data or airtime for the health workers themselves.
Comparison to Prior Work
Our REaCH training responds to a need identified by current research. In a systematic review of 14 studies assessing the feasibility and efficacy of remote consulting in LMICs, all studies identified that with adequate training, health care workers were able to learn to use mobile phones to deliver health care, but the review emphasized that sufficient initial and ongoing training is required to support the implementation of remote consulting [32]. In a systematic review of the barriers to remote consulting, lack of training was likewise identified as a key barrier [33]. Furthermore, during the COVID-19 pandemic, in a survey of physicians in Libya, 638 (94.8%) of 673 of participants expressed willingness to participate in a telemedicine training course [34].
Ediripulge et al. [35] literature review of 9 studies that described the delivery of training in telehealth not only emphasized the importance of adequate training to ensure integration of remote consulting in health systems but also found that the programs were predominantly conducted online and were a mixture of continuous professional development and university courses [35].
A scoping review, published after the development of the REaCH curriculum, describes the range of topics covered by courses that train health personnel for remote consulting [36]. Our course covered the key topics commonly taught, and included topics less commonly taught, including ethics, professionalism, and challenges of remote consulting. In this review, only 2 (5%) of 43 studies were conducted in LMICs [37,38]. One of these papers, similar to our study, evaluated its program using Kirkpatrick level 3 evaluation, while the other paper also included Kirkpatrick level 4 evaluation [39,40].
As in our study, the train-the-trainer approach to remote consulting education was successfully used in Rwanda to train community health care workers in monitoring pregnancy and pregnancy-related complications remotely and in Liberia to upskill traditional midwives to use mobile technology for short messaging service (SMS) texting [39,40]. Also replicated in our findings, remote delivery of remote consulting training in Brazil and India has been successful [38,41]. These trials and other similar remote consulting training programs in LMICs have been well received with high completion rates, as with our pilot study [42,43].
Strengths and Limitations
This study has some key strengths. We tested the program at several stages, undertook intensive evaluation at each stage, and were thus able to improve the program multiple times. We collected quantitative data and qualitative data to evaluate the training.
REaCH training and its pilot evaluation were undertaken at speed in response to the urgent need to support Africa’s low-resource health care system in the face of the COVID-19 pandemic. Consequently, it has some limitations. The results are based on a relatively small number of health workers. Kirkpatrick’s model informed the evaluation, but the second and third levels (learning and behavior) were assessed by self-report, with no external observation or validation. Although Kirkpatrick’s model has its limitations for assessing medical education, it is useful for an evaluation, such as this that assesses immediate effects [44]. The survey questions were developed and delivered in a short time frame. Although each question captures 1 area of interest, some include 2 issues, which we are unable to tease out. There was a marked positive skew in the survey results, although in the exploratory semistructured interviews, respondents talked about both positives and negatives. The evaluation was conducted at 1 site, a single region in a single country, and by the team that developed the training.
We are currently running a stepped-wedge trial of REaCH training in Tanzania and Nigeria to evaluate actual behavior change and results in terms of the impact on health care delivery [45].
Conclusion
The REaCH program, providing training on remote consulting, is feasible and acceptable and successfully initiated behavior change in health care workers in a rural district in Tanzania. Trainees identified a need for resourcing of data/airtime and a technical and device infrastructure for the implementation of remote consulting.
Acknowledgments
We are deeply grateful to the health care workers who participated in the training and its evaluation. We thank the journal editors and 2 anonymous reviewers for the constructive, thoughtful comments on our manuscript. We also thank our colleagues in the Digital Innovation for Development in Africa (DIDA) project for the collegiality and shared insights into remote consulting.
This study was supported by funding from the UK Research and Innovation (UKRI) Global Challenges Research Fund for Digital Innovation for Development in Africa: Engineering and Physical Sciences Research Council (Grant EP/T030240/1) and the King’s Together Fund: “How Can Remote Consulting Training for health workers in Rural Tanzania Be Optimised to Support Upscaling to Remote and Marginalised Communities of East and West Africa?”
Abbreviations
- ICT
information and communication technology
- LMIC
low- and middle-income country
- REaCH
remote consulting in primary health care
- SFUCHAS
St Francis University College of Health and Allied Sciences
Survey responses aligned with Kirkpatrick’s model (n=10 respondents).
Interview questions scaffold.
Footnotes
Conflicts of Interest: The authors FG, BH, SP, and JS have an IP share in the licensing of REaCH training.
References
- 1.World Health Organization . Primary Health Care on the Road to Universal Health Coverage: 2019 Monitoring Report. Geneva: World Health Organization; 2019. [Google Scholar]
- 2.Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, Adeyi O, Barker P, Daelmans B, Doubova SV, English M, García-Elorrio E, Guanais F, Gureje O, Hirschhorn LR, Jiang L, Kelley E, Lemango ET, Liljestrand J, Malata A, Marchant T, Matsoso MP, Meara JG, Mohanan M, Ndiaye Y, Norheim OF, Reddy KS, Rowe AK, Salomon JA, Thapa G, Twum-Danso NAY, Pate M. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Global Health. 2018 Nov;6(11):e1196–e1252. doi: 10.1016/S2214-109X(18)30386-3. https://linkinghub.elsevier.com/retrieve/pii/S2214-109X(18)30386-3 .S2214-109X(18)30386-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.United Nations Transforming Our World: The 2030 Agenda for Sustainable Development. [2022-05-11]. https://sdgs.un.org/2030agenda .
- 4.World Health Organization . Global Diffusion of eHealth - Making Universal Health Coverage Achievable: Report of the Third Global Survey on eHealth. Geneva: World Health Organization; 2017. p. 154 pp. [Google Scholar]
- 5.Griffiths F, Watkins JA, Huxley C, Harris B, Cave J, Pemba S, Chipwaza B, Lilford R, Ajisola M, Arvanitis TN, Bakibinga P, Billah M, Choudhury N, Davies D, Fayehun O, Kabaria C, Iqbal R, Omigbodun A, Owoaje E, Rahman O, Sartori J, Sayani S, Tabani K, Yusuf R, Sturt J. Mobile consulting (mConsulting) and its potential for providing access to quality healthcare for populations living in low-resource settings of low- and middle-income countries. Digit Health. 2020;6:2055207620919594. doi: 10.1177/2055207620919594. https://journals.sagepub.com/doi/10.1177/2055207620919594?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub%3dpubmed .10.1177_2055207620919594 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Harris B, Ajisola M, Alam R, Watkins J, Arvanitis T, Bakibinga P. Mobile consulting (mConsulting) as an option for accessing healthcare services for communities in remote rural areas and urban slums in low- and middle- income countries: a mixed methods study. Digital Health. 2021 doi: 10.1177/20552076211033425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Rowntree O. GSMA. Connected Women. The Mobile Gender Gap Report. London: GSMA; 2019. [Google Scholar]
- 8.Silver L, Johnson C. Majorities in Sub-Saharan Africa Own Mobile Phones, but Smartphone Adoption Is Modest. [2022-05-11]. https://www.pewresearch.org/global/2018/10/09/majorities-in-sub-saharan-africa-own-mobile-phones-but-smartphone-adoption-is-modest/
- 9.World Health Organization Recommendations on Digital Interventions for Health System Strengthening. [2022-05-11]. https://www.who.int/publications/i/item/9789241550505 . [PubMed]
- 10.World Health Organization . Classification of Digital Health Interventions v1.0: A Shared Language to Describe the Uses of Digital Technology for Health. Geneva: World Health Organization; 2018. [Google Scholar]
- 11.World Health Organization Operational considerations for case management of COVID-19 in health facility and community. Interim guidance. Pediatr Med Rodz. 2020 Jul 14;16(1):27–32. doi: 10.15557/pimr.2020.0004. [DOI] [Google Scholar]
- 12.Webster P. Virtual health care in the era of COVID-19. Lancet. 2020 Apr 11;395(10231):1180–1181. doi: 10.1016/S0140-6736(20)30818-7. http://europepmc.org/abstract/MED/32278374 .S0140-6736(20)30818-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Busagala L, Kawono G. Underlying challenges of e-health adoption in Tanzania. Int J Inform Commun Technol Res. 2013;3(1):34–41. [Google Scholar]
- 14.World Health Organization Web supplement 1: evidence-to-decision frameworks. WHO Guideline: Recommendations on Digital Interventions for Health System Strengthening. [2022-05-11]. https://apps.who.int/iris/bitstream/handle/10665/311979/WHO-RHR-19.6-eng.pdf?ua=1 . [PubMed]
- 15.Ruggeri K, Farrington C, Brayne C. A global model for effective use and evaluation of e-learning in health. Telemed J E Health. 2013 Apr;19(4):312–21. doi: 10.1089/tmj.2012.0175. http://europepmc.org/abstract/MED/23472702 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in medical education. Acad Med. 2006 Mar;81(3):207–12. doi: 10.1097/00001888-200603000-00002.81/3/207 [DOI] [PubMed] [Google Scholar]
- 17.Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008 Sep 10;300(10):1181–96. doi: 10.1001/jama.300.10.1181.300/10/1181 [DOI] [PubMed] [Google Scholar]
- 18.Lall P, Rees R, Law GCY, Dunleavy G, Cotič. Car J. Influences on the implementation of mobile learning for medical and nursing education: qualitative systematic review by the digital health education collaboration. J Med Internet Res. 2019 Feb 28;21(2):e12895. doi: 10.2196/12895. https://www.jmir.org/2019/2/e12895/ v21i2e12895 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Griffiths F. Remote Consulting for Healthcare (ReaCH): Training Health Workers to Continue Health Care Delivery Digitally in Low Resource Countries. [2021-08-12]. https://warwick.ac.uk/fac/sci/med/research/hscience/sssh/research/lyncs/COVID19 .
- 20.Moodle. [2021-05-15]. https://moodle.org/
- 21.WhatsApp. [2021-05-15]. https://www.whatsapp.com/
- 22.Mormina M, Pinder S. A conceptual framework for training of trainers (ToT) interventions in global health. Global Health. 2018 Oct 22;14(1):100. doi: 10.1186/s12992-018-0420-3. https://globalizationandhealth.biomedcentral.com/articles/10.1186/s12992-018-0420-3 .10.1186/s12992-018-0420-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Kirkpatrick D, Kirkpatrick J. Evaluating Training Programs: The Four Levels. Oakland, CA: Berrett-Koehler; 2006. [Google Scholar]
- 24.Cochrane C. Frascati Definition of Research. [2022-02-25]. https://warwick.ac.uk/services/ris/systems/dataquality/frascati/
- 25.NHS Health Research Authority Do I Need NHS REC Review? [2022-02-21]. http://www.hra-decisiontools.org.uk/ethics/
- 26.The Planning Commission Dar es Salaam. Regional Commissioner's Office Morogoro Morogoro Region Socio-Economic Profile. [2022-05-11]. http://www.tzonline.org/pdf/Morogoro.pdf .
- 27.National Bureau of Statistics, Ministry of Finance. Office of Chief Government Statistician, President’s Office, Finance, Economy and Development Planning 2012 Population and Housing Census: Population Distribution by Administrative Areas. [2022-05-11]. http://tanzania.countrystat.org/fileadmin/user_upload/countrystat_fenix/congo/docs/Census%20General%20Report-2012PHC.pdf .
- 28.SurveyMonkey. [2022-05-11]. https://www.surveymonkey.co.uk/
- 29.Boyatzis R. Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: SAGE Publications; 1998. [Google Scholar]
- 30.McMahon SA, Winch PJ. Systematic debriefing after qualitative encounters: an essential analysis step in applied qualitative research. BMJ Global Health. 2018;3(5):e000837. doi: 10.1136/bmjgh-2018-000837. https://gh.bmj.com/lookup/pmidlookup?view=long&pmid=30233833 .bmjgh-2018-000837 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Noble H, Heale R. Triangulation in research, with examples. Evid Based Nurs. 2019 Jul 14;22(3):67–68. doi: 10.1136/ebnurs-2019-103145.ebnurs-2019-103145 [DOI] [PubMed] [Google Scholar]
- 32.Agarwal S, Perry HB, Long L, Labrique AB. Evidence on feasibility and effective use of mHealth strategies by frontline health workers in developing countries: systematic review. Trop Med Int Health. 2015 Aug;20(8):1003–14. doi: 10.1111/tmi.12525. doi: 10.1111/tmi.12525. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Kruse C, Betancourt J, Ortiz S, Valdes Luna SM, Bamrah IK, Segovia N. Barriers to the use of mobile health in improving health outcomes in developing countries: systematic review. J Med Internet Res. 2019 Oct 09;21(10):e13263. doi: 10.2196/13263. https://www.jmir.org/2019/10/e13263/ v21i10e13263 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Elhadi M, Elhadi A, Bouhuwaish A, Bin Alshiteewi F, Elmabrouk A, Alsuyihili A, Alhashimi A, Khel S, Elgherwi A, Alsoufi A, Albakoush A, Abdulmalik A. Telemedicine awareness, knowledge, attitude, and skills of health care workers in a low-resource country during the COVID-19 pandemic: cross-sectional study. J Med Internet Res. 2021 Feb 25;23(2):e20812. doi: 10.2196/20812. https://www.jmir.org/2021/2/e20812/ v23i2e20812 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Edirippulige S, Armfield NR. Education and training to support the use of clinical telehealth: a review of the literature. J Telemed Telecare. 2017 Feb;23(2):273–282. doi: 10.1177/1357633X16632968.1357633X16632968 [DOI] [PubMed] [Google Scholar]
- 36.Stovel RG, Gabarin N, Cavalcanti RB, Abrams H. Curricular needs for training telemedicine physicians: a scoping review. Med Teach. 2020 Nov;42(11):1234–1242. doi: 10.1080/0142159X.2020.1799959. [DOI] [PubMed] [Google Scholar]
- 37.Zafar A, Belard JL, Gilani S, Murad F, Khan M, Merrell RC. The impact of curriculum on a national telehealth program. Telemed J E Health. 2008 Mar;14(2):195–8. doi: 10.1089/tmj.2007.0029. [DOI] [PubMed] [Google Scholar]
- 38.Manjunatha N, Kumar CN, Math SB, Thirthalli J. Designing and implementing an innovative digitally driven primary care psychiatry program in India. Indian J Psychiatry. 2018;60(2):236–244. doi: 10.4103/psychiatry.IndianJPsychiatry_214_18. http://www.indianjpsychiatry.org/article.asp?issn=0019-5545;year=2018;volume=60;issue=2;spage=236;epage=244;aulast=Manjunatha .IJPsy-60-236 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Ngabo F, Nguimfack J, Nwaigwe F, Mugeni C, Muhoza D, Wilson DR, Kalach J, Gakuba R, Karema C, Binagwaho A. Designing and implementing an innovative SMS-based alert system (RapidSMS-MCH) to monitor pregnancy and reduce maternal and child deaths in Rwanda. Pan Afr Med J. 2012;13:31. https://www.panafrican-med-journal.com/content/article/13/31/full/ [PMC free article] [PubMed] [Google Scholar]
- 40.Lori JR, Munro ML, Boyd CJ, Andreatta P. Cell phones to collect pregnancy data from remote areas in Liberia. J Nurs Scholarsh. 2012 Sep;44(3):294–301. doi: 10.1111/j.1547-5069.2012.01451.x. http://europepmc.org/abstract/MED/22672157 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Carvalho VS, Picanço MR, Volschan A, Bezerra DC. Impact of simulation training on a telestroke network. Int J Stroke. 2019 Jul;14(5):500–507. doi: 10.1177/1747493018791030. [DOI] [PubMed] [Google Scholar]
- 42.dos Santos AdF, Alves HJ, Nogueira JT, Torres RM, Melo MDCB. Telehealth distance education course in Latin America: analysis of an experience involving 15 countries. Telemed J E Health. 2014 Aug;20(8):736–41. doi: 10.1089/tmj.2013.0291. [DOI] [PubMed] [Google Scholar]
- 43.Ishibashi Y, Juzoji H, Kitano T, Nakajima I. ITU e-health training program for Pacific Island community with the support of the Sasakawa peace foundation. J Med Syst. 2011 Jun 12;35(3):345–51. doi: 10.1007/s10916-009-9371-9. [DOI] [PubMed] [Google Scholar]
- 44.Yardley S, Dornan T. Kirkpatrick's levels and education 'evidence'. Med Educ. 2012 Jan;46(1):97–106. doi: 10.1111/j.1365-2923.2011.04076.x. [DOI] [PubMed] [Google Scholar]
- 45.Sturt J. Determining Trustworthiness and Safety of Remote Consulting in Primary Healthcare for Chronic Disease Populations in Africa. [2022-02-28]. https://www.kcl.ac.uk/research/reach-trial .
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Survey responses aligned with Kirkpatrick’s model (n=10 respondents).
Interview questions scaffold.