Abstract
Background:
Health worker training is an essential component of epidemic control; rapid delivery of such training is possible in low-middle income countries with digital platforms.
Methods:
Based on prior experience with the Ebola outbreak, we developed and deployed a bespoke InStrat COVID-19 tutorial app, to deliver accurate and regularly updated information about COVID-19 to frontline health workers and epidemic response officers across 25 states of Nigeria. The potential effectiveness of this app in training frontline health workers was assessed through online pre- and post-tests and a survey.
Results:
A total of 1051 health workers from 25 states across Nigeria undertook the e-learning on the InStrat COVID-19 training app. Of these, 627 (57%) completed both the pre- and post-tests in addition to completing the training modules. Overall, there were statistically significant differences between pre- and post-tests knowledge scores (54 increasing to 74). There were also differences in the subcategories of sex, region and cadre. There were higher post-test scores in males compared with females, younger versus older and southern compared with northern Nigeria. A total of 65 (50%) of the participants reported that the app increased their understanding of COVID-19, while 69 (53%) stated that they had applied the knowledge and skills learnt at work. Overall, the functionality and usability of the app were satisfactory.
Conclusion:
Capacity building for epidemic control using e-health applications is potentially effective, can be delivered at minimal cost and service disruption and can serve as a tool for capacity building in similar contexts.
Keywords: COVID-19, e-Health, e-Learning at scale, frontline health workers, Nigeria, pandemics
Introduction
The COVID-19 pandemic is the largest global health crisis occurring in the digital era, with catastrophic and long-term consequences for the communities affected by it.1 Digital health solutions have been identified by the World Health Organization (WHO)2 as a promising approach to mitigate the effects of the pandemic, given the vast potential to leverage digital tools for prompt dissemination of information, track disease transmission in real time and facilitate day-to-day operations by creating virtual venues for meetings.3 In some contexts, digital technology has been used to build multiple COVID-19-specific tools to support outbreak management, including scripted triaging, real-time data analytics and telemedicine capabilities.3
The African continent is at high risk for the spread of COVID-19 due to her weak healthcare systems, limited surveillance and laboratory capacity, as well as the scarcity of human and financial resources.4 Nigeria, with a population of 200 million, identified its first case of COVID-19 on 27 February 2020.5,6 In response, the Nigerian government rapidly passed an emergency economic stimulus bill, made cash transfers, provided food rations to vulnerable households and strengthened its health system’s capacity to test and treat COVID-19 cases.7 However, modelling studies rate Nigeria as having a moderate capacity to control COVID-19.8
Health worker training is critical to controlling pandemics such as COVID-19, as it facilitates triage, rapid assessment, isolation, management and reporting of new cases.9 Evidence shows that Nigeria has human resources for health (HMH) density of 2.0 per 1000 population, compared to 2.2 per 1000 for African countries and 14 per 1000 in Europe.10 Nigeria’s health service is characterised by weak and irregular staff training and an inequitable distribution of qualified staff with higher ratios per population in the urban compared with rural areas. Rapid delivery of health worker training at scale should be possible with digital platforms. It is projected that there will be 634 million mobile subscribers across sub-Saharan Africa by 2025, representing 52% of the continent’s population.11 With an estimated 25–40 million smartphone users in Nigeria,12 the use of smartphones for training of frontline health workers (even in remote locations) holds great prospects. This is supported by the results of a recent survey, conducted by InStrat Global Health Solutions, (an indigenous technology company) in 339 health facilities in 10 states (with representative geographic coverage of Nigeria). This survey showed that there was an average of seven health workers per health facility, with an average of seven phones per facility; of which, five were smartphones capable of running applications. Of these 339 facilities, 226 (66%) were rural and 116 (34%) were urban.13
Accessing up-to-date and evidence-based information by frontline health workers, especially those in remote areas, is challenging. There is evidence from the literature of modest improvement in knowledge and attitudinal change toward the Ebola virus disease among frontline health workers, when an e-health tablet application was used to educate them during the Ebola virus disease outbreak of 2014 in Nigeria.14 To further validate our strategy of using e-health training solutions in this context, we designed a tutorial app consisting of training modules and tests to train frontline health workers on COVID-19; the aim was to bridge the gap between social distancing and the provision of essential up-to-date training in a safe environment. The app was designed to allow health-care workers access to up-to-date COVID-19 training modules and near real-time updates via any android device. Once downloaded, the content could be viewed as many times as required, to ensure complete understanding.
InStrat worked with Plus91 Technologies to deploy the app in Ogun State in south-western Nigeria. Details of the deployment of this app have been reported previously.15 The app allowed for (1) training of health workers on COVID-19; (2) provision of users with desk guides to rapidly screen, identify and manage suspected COVID-19 cases.16 This app was subsequently deployed at scale to 25 states of Nigeria. We describe the results of the nationwide deployment of the InStrat COVID-19 tutorial app to frontline health workers as a public health intervention during the COVID-19 pandemic in Nigeria.
Method
Design
The study involved quantitative cross-sectional surveys with pre- and post-intervention knowledge, attitude, and practice (KAP) assessments of frontline health workers.
Study setting
This nationwide electronic learning (e-Learning) intervention was implemented across 25 states of Nigeria, that have a population of 150 million people. Primary healthcare (PHC) facilities are administered by Local Government Areas (LGAs) and funded through state government statutory allocations.
Participants
Eligible participants were doctors, nurses, midwives, laboratory technicians, pharmacists, community health officers (CHO) and Community Health Extension Workers (CHEW) drawn from 275 health facilities across 25 states of Nigeria (Figure 1). We obtained the contact details of potential participants from facility managers, after obtaining consent for the study from the state health authorities to assess the effectiveness of the e-Learning intervention. Potential participants were invited to partake in the study via email, WhatsApp and SMS, with a link explaining the objectives of the study. Potential respondents were provided with project information that explained the objectives of the study, to help them decide on whether to participate in the study (Supplementary File). Those who agreed to participate signed the consent forms electronically before proceeding with the training. Once signed, the consent forms were automatically stored in an electronic repository, which was accessible by the study team. Monitoring calls were then made to guide HWs through the training process. The monitoring of responses, pre-test, training module completion and post-test assessment were also performed by InStrat.
Figure 1.
Map reflecting the distribution of health workers who used the InStrat COVID-19 training App.
e-Learning intervention
The e-Learning intervention was implemented between 30 March and 20 June 2020. The intervention consisted of an InStrat COVID-19 tutorial app (Figure 2), developed using the MediXcel Lite health technology platform. The COVID-19 education module was based on content developed by the Foundation for Healthcare Innovation and Development (FHIND) (www.fhind.org). The training module (consisting of video, audio and text-based learning materials, was created in English and then translated to three major languages: Hausa, Igbo and Yoruba. Training materials were drafted by AO, EE, OO, AO and JW, drawing on information from the Nigeria Centre for Disease Control (NCDC)17 and the WHO.18 The InStrat COVID-19 tutorial app was built to deliver accurate and regularly-updated COVID-19 modules to frontline health workers, epidemic response officers and strategic personnel. User logins were created and sent to the participants via WhatsApp, email and SMS by the staff of InStrat to download onto their android mobile phones. This was to enable them to log into the app and work their way through the module, which also tracked their progress automatically. We estimated that it would take the participants at least 60 minutes to complete the module.
Figure 2.
Home page of the InStrat COVID-19 tutorial App.
We collected all data via the app used by the FHWs, with the data automatically uploaded onto remote servers before being accessed by the research team. To assess the extent of change in health workers’ COVID-19 related knowledge, all participants accessing the COVID-19 tutorial app first took a multiple-choice pre-intervention test (‘pre-test’) that assessed their reported COVID-19 knowledge on the following seven topics:
Overview of COVID-19 and origin of the novel Coronavirus.
Signs and symptoms of COVID-19.
Risk factors of contracting COVID-19.
Precautions against COVID-19.
Specimen collection and management guidelines.
A World Health Organization COVID-19 tutorial video to provide additional background information for health workers.
Official NCDC daily COVID-19 updates and case summaries.
The post-intervention test (‘post-test’) questions were the same as the pre-test questions. Participants who completed the COVID-19 training modules were automatically prompted via their phones to take the post-test. The estimated time duration for completion of each test (either pre- or post-test) was five minutes. The participants were allowed to retake the post-test as many times as they wished; the best post-test score was captured for each participant. A user satisfaction survey (Survey Monkey) was also provided within the training app to assess the overall satisfaction of health workers with the training content. Feedback was obtained on knowledge gained from use of the COVID-19 training app and its applicability to the work environment.
Statistical analysis
Relative frequency was used to describe the socio-demographic characteristics of the participants. The Shapiro–Wilk test and Levene’s test were used to test for normality and the homogeneity of variance of the pre-test and post-test scores. Logarithmic transformation of the post-test scores was conducted to address the observed skewness of the data; parametric tests were used for the analysis afterward.
An independent t-test was used to summarise the mean and standard deviation (SD) of the pre-test and post-test scores of the participants, whose selection was independent of each other. The paired t-test was used to describe the mean and SD of paired observations.
The independent variables were sex, region of Nigeria, age group and cadre of health workers (categorical variables), while the post-test scores were the continuous, dependent variable. One-way ANOVA was used to compare the mean pre-test and post-test scores between the independent variables. In the modelling, multivariable linear regression of predictors of post-test scores was not performed because all independent variables were categorical variables. As a result, a four-way factorial ANOVA was done to determine the relationship between the independent variables and post-test score.
All analysis was conducted using Stata 14 at a 5% significance level. Statistically significant variables were those whose p-values were less than 0.05.
Ethics approval
The study protocol was approved by the National Health Ethics Review Committee of Nigeria with assigned number NHREC/01/01/2007.
Results
A total of 1051 health workers from 25 states across Nigeria registered for, and undertook, the e-learning on the InStrat COVID-19 training app. Of these, 627 (57%) completed both the pre- and post-tests, in addition to completing the training modules. The distribution of the 1051 participants by age group was as follows: 20–29 years (136; 13%); 30–39 years (340; 32%); 40–49 years (343; 33%); 50–59 years (223; 21%); and ⩾60 years (9; 1%). There were 520 (50%) females and 495 (47%) males. Data on sex was missing for 36 (3%) of the participants. There were more participants from the South (676; 64%) compared with 375 (35.7%) from the North. The cadre of health workers were as follows: CHEW (444; 42.3%); doctor (79; 7.5%); laboratory technician (43; 4.1%); nurse (153; 14.6%); pharmacist (26; 2.5%); and other (this included environmental health officers and biomedical scientists) 266 (25.2%). Data on cadre was missing for 40 (3.8%) of the participants. The Shapiro–Wilk test and Levene’s test of variance for the pre-test scores showed normality, with a p-value = 0.230 and homogeneity, with p = 0.121. For the post-test score, normality was = 0.01 and homogeneity was = 0.49. After logarithmic transformation of the post-test scores, the Shapiro–Wilk test for normality became non-significant [p = 0.994. Using a one-way ANOVA test, there was a statistically significant difference in the mean pre-test scores in the age groups F (df: 4622) = 3.64, p = 0.006], while there were no statistically significant differences in the mean pre-test scores in sex (p = 0.430), region of Nigeria (p = 0.131) and cadre of health workers (p = 0.185). See Table 1.
Table 1.
Comparison of the pre-test scores across the socio-demographic characteristics of the respondents.
| Variable | Pre-test score (Mean ± SD) | Test statistic | p-value |
|---|---|---|---|
| Sex | |||
| Female (n = 339) | 54.28 ± 12.05 | Independent t-test (0.791) | 0.430 |
| Male (270) | 53.52 ± 11.40 | ||
| Region | |||
| North (n = 214) | 54.91 ± 10.78 | Independent t-test (1.514) | 0.131 |
| South (n = 413) | 53.41 ± 12.56 | ||
| The age group (years) | |||
| 20–29 (n = 101) | 57.53 ± 13.67 | One-way ANOVA F test value (3.64) | 0.006* |
| 30–39 (n = 192) | 53.33 ± 11.41 | ||
| 40–49 (n = 199) | 53.17 ± 10.42 | ||
| 50–59 (n = 130) | 52.85 ± 12.01 | ||
| ⩾60 (n = 5) | 62.00 ± 8.37 | ||
| Cadre | |||
| CHEW (n = 259) | 53.36 ± 12.24 | One-way ANOVA F test value (1.51) | 0.185 |
| Doctor (n = 37) | 54.87 ± 13.04 | ||
| Lab tech (n = 24) | 52.92 ± 13.02 | ||
| Nurse (n = 98) | 52.14 ± 11.15 | ||
| Pharmacist (n = 15) | 56.67 ± 12.34 | ||
| Other (n = 176) | 55.57 ± 12.22 | ||
| Overall mean ± SD | 53.92 ± 11.72 | ||
| Range | 20–90 | ||
Statistically significant association.
CHEW, Community Health Extension Workers; SD, standard deviation.
A Bonferroni post hoc test for the difference in the mean pre-test score by age group was conducted (Table 2). This test revealed that there were statistically significant differences in the mean pre-test scores between the 20–29 and 30–39 (p = 0.035), 20–29 and 40–49 (p = 0.022) and 20–29 and 50–59 (p = 0.025) year age groups. Participants from southern Nigeria had higher mean post-test scores than those from northern Nigeria (75.6 versus 70.4) and this difference was statistically significant (p < 0.001). Using one-way ANOVA, the mean post-test scores differed between the age groups F (df: 4624) = 3.13, p = 0.015) and cadre of health worker’s F (df: 5605) = 7.01, p < 0.001). See Table 3.
Table 2.
Bonferroni post hoc test for the difference in the mean pre-test score by age group.
| Pre-test score | Age group (years) | |||||
|---|---|---|---|---|---|---|
| 20–29 | 30–39 | 40–49 | 50–59 | ⩾60 | F test (p-value) | |
| Pre-test score | 57.53 ± 13.67 | 53.33 ± 11.41 | 53.17 ± 10.42 | 52.85 ± 12.01 | 62.00 ± 8.37 | 3.64 (0.006) |
| Mean difference, (p-value) | ||||||
| 20–29 | – | – | – | – | – | |
| 30–39 | −4.19, (0.035) | – | – | – | – | |
| 40–49 | −4.36, (0.022) | −0.17, (1.000) | – | – | – | |
| 50–59 | −4.68, (0.025) | −0.49, (1.000) | −0.32, (1.000) | – | – | |
| ⩾60 | 4.48, (1.000) | 8.67, (1.000) | 9.93, (0.936) | 9.15, (0.844) | – | |
Table 3.
Changes in the post-test scores across the socio-demographic characteristics of the respondents.
| Variable | Post-test score (Mean ± SD) | Test statistic | p-value |
|---|---|---|---|
| Sex | |||
| Female (n = 339) | 72.98 ± 12.81 | Independent t-test (−1.329) | 0.184 |
| Male (272) | 74.30 ± 11.44 | ||
| Region | |||
| North (n = 216) | 70.42 ± 10.88 | Independent t-test (−4.742) | <0.001* |
| South (n = 413) | 75.55 ± 12.49 | ||
| The age group (years) | |||
| 20–29 (n = 101) | 70.10 ± 10.63 | One-way ANOVA F test value (3.13) | 0.015* |
| 30–39 (n = 192) | 74.90 ± 12.11 | ||
| 40–49 (n = 200) | 74.45 ± 12.06 | ||
| 50–59 (n = 131) | 72.75 ± 12.95 | ||
| ⩾60 (n = 5) | 76.00 ± 16.73 | ||
| Cadre | |||
| CHEW (n = 259) | 75.41 ± 13.21 | One-way ANOVA F test value (7.01) | <0.001* |
| Doctor (n = 38) | 79.74 ± 11.27 | ||
| Lab tech (n = 24) | 71.67 ± 16.06 | ||
| Nurse (n = 98) | 72.35 ± 10.73 | ||
| Pharmacist (n = 15) | 78.67 ± 13.56 | ||
| Other (n = 177) | 70.17 ± 9.74 | ||
| Overall mean ± SD | 73.55 ± 12.17 | ||
| Range | 30–100 | ||
Statistically significant association.
CHEW, Community Health Extension Workers; SD, standard deviation.
A Bonferroni post hoc test showed statistically significant differences in the mean post-test scores between the 20–29 and 30–39 (p = 0.013) and 20–29 and 40–49 (p = 0.028) year age groups reported in Table 3. Similarly, there were statistically significant differences in the mean post-test scores between CHEW and other cadres of health workers (p < 0.001), doctors and nurses (p = 0.019) and doctors and other health workers (p < 0.001) (Table 4).
Table 4.
Bonferroni post hoc test for the mean difference in the post-test scores in age groups and cadre of health workers.
| Post-test score | Age group (years) | |||||
|---|---|---|---|---|---|---|
| 20–29 | 30–39 | 40–49 | 50–59 | ⩾60 | F test (p-value) | |
| Mean difference, (p-value) | 3.13 (0.015) | |||||
| 20–29 | – | – | – | – | – | |
| 30–39 | 4.80, (0.013) | – | – | – | – | |
| 40–49 | 4.35, (0.028) | −0.45, (1.000) | – | – | – | |
| 50–59 | 2.65, (0.984) | −2.15, (1.000) | −1.70, (1.000) | – | – | |
| ⩾60 | 5.90, (1.000) | 1.10, (1.000) | 1.55, (1.000) | 3.25, (1.000) | – | |
| Cadre of health workers | ||||||
| Post-test score | CHEW | Doctor | Lab tech | Nurse | Other | F test (p-value) |
| Mean difference, (p-value) | 7.01 (<0.001) | |||||
| Doctor | 4.33, (0.555) | – | – | – | – | |
| Lab technologist | −3.74, (1.000) | −8.07, (0.145) | – | – | – | |
| Nurse | −3.06, (0.465) | −7.39, (0.019) | 0.68, (1.000) | – | – | |
| Pharmacist | 3.26, (1.000) | −1.07, (1.000) | 7.00, (1.000) | 6.32, (0.848) | 8.50, (0.124) | |
| Other | −5.24, (<0.001) | −9.57, (<0.001) | −1.50, (1.000) | −2.18, (1.000) | – | |
Overall, there was a statistically significant difference (p < 0.001) in the pre-test and post-test scores of the participants who responded to both online tests (Table 5). With the exception of the ⩾60-year age group, in which there was no statistically significant difference in both the pre-test and post-test scores (p = 0.206), there were statistically significant differences (p < 0.001) in all subcategories of sex, region and cadre. A four-way ANOVA was run on a sample of 608 participants to examine the effect of age, region of Nigeria, sex and cadre of health workers on post-test scores (Table 6). Region (p = 0.010), sex (p = 0.007) and the interaction between age and sex (p = 0.021) was significantly associated with post-test scores. However, there was no significant four-way interaction, F (df: 4528) = 1.54, p = 0.189.
Table 5.
Changes in the pre-test and post-test scores for all participants who responded to both surveys.
| Variable | Pre-test score | Post-test score | Paired t-test | p-value |
|---|---|---|---|---|
| Mean ± SD | Mean ± SD | |||
| Total (n = 627) | 53.92 ± 11.72 | 73.51 ± 12.17 | −37.425 | <0.001* |
| Sex | ||||
| Female (n = 339) | 54.28 ± 12.05 | 72.98 ± 12.81 | −24.902 | <0.001* |
| Male (n = 270) | 53.52 ± 11.40 | 74.22 ± 11.44 | −28.486 | <0.001* |
| Region | ||||
| North (n = 214) | 54.91 ± 10.78 | 70.28 ± 10.83 | −17.769 | <0.001* |
| South (n = 413) | 53.41 ± 12.16 | 75.18 ± 12.49 | −34.533 | <0.001* |
| The age group (years) | ||||
| 20–29 (n = 101) | 57.52 ± 13.67 | 70.10 ± 10.63 | −9.385 | <0.001* |
| 30–39 (n = 192) | 53.33 ± 11.41 | 74.90 ± 12.11 | −22.894 | <0.001* |
| 40–49 (n = 199) | 53.17 ± 10.42 | 74.37 ± 12.04 | −25.287 | <0.001* |
| 50–59 (n = 130) | 52.85 ± 12.02 | 72.69 ± 12.99 | −17.736 | <0.001* |
| ⩾60 (n = 5) | 62.00 ± 8.37 | 76.00 ± 16.73 | −1.510 | 0.206 |
| Cadre | ||||
| CHEW (n = 259) | 53.36 ± 11.24 | 75.41 ± 13.21 | −28.348 | <0.001* |
| Doctor (n = 37) | 54.87 ± 13.04 | 79.46 ± 11.29 | −9.841 | <0.001* |
| Lab tech (n = 24) | 53.92 ± 13.02 | 71.67 ± 16.06 | −6.912 | <0.001* |
| Nurse (n = 98) | 52.14 ± 11.15 | 72.35 ± 10.73 | −17.148 | <0.001* |
| Pharma (n = 15) | 56.67 ± 12.34 | 78.67 ± 13.56 | −7.873 | <0.001* |
| Other (n = 176) | 55.57 ± 12.22 | 70.11 ± 9.74 | −15.052 | <0.001* |
Statistically significant association.
NB: The disparity in the total number of paired observations between the variables ‘Region’ and ‘Age group’ (n = 627) and the variables ‘Sex and cadre of health workers’ (n = 609) was due to missing data in the latter.
CHEW, Community Health Extension Workers; SD, standard deviation.
Table 6.
Four-way ANOVA to determine relationship between socio-demographic and geographical independent variables and their interaction effect on a post-test scores.
| Variable | F value | p-value |
|---|---|---|
| Age group | 0.97 | 0.421 |
| Region | 6.75 | 0.010 |
| Sex | 7.22 | 0.007 |
| Cadre | 1.43 | 0.212 |
| Age group*Area | 0.74 | 0.562 |
| Age group*Sex | 2.93 | 0.021 |
| Area*Sex | 0.07 | 0.791 |
| Age group*Cadre | 0.96 | 0.492 |
| Area*Cadre | 1.12 | 0.350 |
| Sex*Cadre | 0.91 | 0.477 |
| Age group*Area*Sex | 0.91 | 0.438 |
| Age group*Area*Cadre | 1.40 | 0.162 |
| Age group*Sex*Cadre | 0.80 | 0.653 |
| Area*Sex*Cadre | 1.56 | 0.199 |
| Age group*Area*Sex*Cadre | 1.54 | 0.189 |
An online Survey Monkey was sent out to the participants within a week of completing the training: 141 participants completed the survey (i.e. 23% of participants that registered for the intervention). From the responses provided, 94 (69%) of the respondents stated that they had received previous training on COVID-19 before using the Instant COVID-19 tutorial app. Forty-three (31%) of them had not received prior training on COVID-19 and 4 did not answer this question. A total of 74 (57%) strongly agreed that they were interested in taking part in the training (Table 7). Sixty-five (50%) respondents strongly agreed that the InStrat COVID-19 tutorial app had provided them with a better understanding of COVID-19. A total of 78 (60%) strongly agreed that the skills they had learned from going through the e-learning app were applicable at work. Sixty-nine (53%) strongly agreed that they had applied the knowledge and skills they had learned from the e-learning app at their workplace. A total of 72 (61%) agreed that the level of training material was adequate, while 31 (26%) strongly agreed that they would prefer e-Learning to face-to-face training in the future.
Table 7.
Responses to Survey Monkey questions.
| Question | Strongly agree n (%) | Agree (%) | Undecided (%) | Disagree (%) | Strongly disagree (%) | Skipped (%) |
|---|---|---|---|---|---|---|
| I was interested in taking part in this training | 74 (57) | 55 (43) | 1 (1) | 0 | 0 | 11 (8) |
| The app has provided me with a better understanding of COVID-19 | 65 (50) | 59 (45) | 4 (3) | 0 | 2 | 11 (8) |
| I learned skills that I can use at work | 78 (60) | 48 (37) | 3 (2) | 2 (1) | 0 | 10 (7) |
| I have since applied the knowledge and skills I learned at my workplace | 69 (53) | 56 (43) | 4 (3) | 1 (1) | 0 | 11 (8) |
| I was satisfied by the level of the training material | 40 (34) | 72 (61) | 4 (3) | 1 (1) | 1 (1) | 23 (16) |
| I would prefer e-Learning courses to face-to-face learning in the future | 31 (26) | 60 (60) | 13 (11) | 13 (11) | 1 (1) | 23 (16) |
| The app has improved my willingness and ability to train and mentor others | 50 (42) | 64 (534) | 5 (4) | 0 | 0 | 22 (16) |
When asked what additional assistance they would require implementing what they had learned on the course, forty-two (35%) respondents stated that they would require supervisory support, 29 (24%) stated they would require videos, 18 (15%) stated they would like print material and a total of 13 (11%) stated that they would like classroom-based training. When asked to identify which aspects of the app they found most useful/applicable, a total of 102 (84%) respondents chose hand hygiene, 90 (74%) chose respiratory hygiene and cough etiquette, 94 (77%) chose appropriate use of personal protective equipment and 54 (44%) chose patient placement. A total of 88 (72%) chose safe handling, cleaning and disinfection of patient care equipment, while 69 (57%) chose environmental cleaning – beds, trolleys and other surfaces. Fifty-five (45%) respondents chose safe handling and cleaning of soiled linen, while 82 (67%) chose waste management and 64 (53%) chose sharps management.
Discussion
This study demonstrates the potential effectiveness of the InStrat COVID-19 tutorial app in training frontline health workers. Statistically significant differences were recorded between pre- and post-tests in the subcategories of sex, region and cadre. Males had higher post-test scores than females, participants from the south performed better those from the north and those younger than 60 years did better than older participants. In addition, the app enabled better understanding of COVID-19 and the application of knowledge and skills learned at health workers’ workplace.
The findings of this study add to the growing body of knowledge that supports the use of e-Learning in outbreak settings to rapidly and effectively train large numbers of frontline health workers, some of whom may be practicing in remote settings. A systematic review conducted by Imperial College London researchers, and commissioned by the WHO, found that e-Learning was as effective as traditional methods for training health professionals.19 This point is further supported by a more recent systematic review, which found that e-Learning on its own was superior to no learning (and as good as face-to-face learning) in improving the knowledge and skills of healthcare workers.20
Some of the reported advantages of e-Learning include time and location flexibility and accessibility, lower training costs and time commitment, self-directed learning, standardised course delivery and the invaluable benefit of unlimited access to the training materials.21,22 With widening health inequities in access to care in low- and middle-income countries (LMICs) such as Nigeria, e-Learning offers a promising approach to bridge the gaps in capacity building for health workers that can be extended beyond boosting pandemic preparedness. We rapidly deployed this tutorial app in real-time during the COVID-19 pandemic, which was characterised by restrictive measures to reduce gatherings and social distancing as a key control strategy. We have demonstrated that the use of an e-Learning app can successfully mitigate the challenge of getting training out to health workers without exposing them to the risk of in-class training which could jeopardise their safety. In addition, the use of versions translated into the three main languages in Nigeria is likely to have facilitated the uptake of the contents of this tutorial app. By incorporating pre- and post-test assessments, it was easy to evaluate the effectiveness of the e-Learning app and this was done at no additional cost.
Our approach is similar to that adopted by a team at the KEMRI-Wellcome Trust Research Programme (KWTRP) in Kenya and the University of Oxford, who developed and deployed the Life-saving Instruction for Emergencies (LIFE) smartphone-based virtual learning platform.23 The LIFE platform has been successfully used to train healthcare workers in Africa to manage children with suspected COVID-19.
There is evidence from Nigeria of how mHealth strategies involving creation of maps using a geographical information system helped with the detection of polio cases in physically inaccessible areas of Nigeria.24 The information generated in real time by these mHealth strategies was useful in identifying missed or partially covered settlements. Sending SMS text reminders to clients was also shown to be effective in improving the surveillance of acute flaccid paralysis due to polio in Nigeria.25
In our study, the non-significant difference in the pre and post-test scores for those who were ⩾60-years of age may reflect their level of digital literacy; in addition, it may also be that older adults are slower and less effective than younger people in learning new skills.26 However, this may change as health technology literate people become older. The participants from the southern part of Nigeria had significantly higher test scores than those from the north. This could be due to the educational inequalities between the north compared with the south that have been previously reported.27 It is unclear why the post-test scores of the male participants were significantly higher than that of the female participants. Given the significant difference in mean test scores among various age groups and professions we recorded, it may be beneficial to further customise such training modules to cater for an individual’s age and profession.
Cell phone ownership in Nigeria has progressively increased over the years and presently stands at 200 million.28 Recent data from the National Bureau of Statistics of Nigeria indicate a 17.1% increase in internet usage in the first quarter of 2020 compared with 2019. This degree of teledensity confers an enormous advantage which can be leveraged for all aspects of control of disease outbreaks.
The use of e-Learning within the Nigerian space is not without its challenges. Power outages, inequitable access to both the internet and electronic devices are some barriers that have been previously identified.29 These would have to be factored in and potentially mitigated (where possible) before launching similar training apps.
The findings of our study have several implications for health policy, practice and future research. Infectious disease outbreaks with accompanying restrictions require rapid upscaling of the training of frontline health workers for quick control. Given the widespread ownership of internet capable phones across urban and rural communities, where the primary healthcare centre is located, the adoption of e-Learning platforms is a compelling strategy. The ability to provide high-quality, customised information with real-time updates to health workers in a safe and cost-effective way supports that response. User friendly health apps, such as our COVID-19 tutorial app, can bolster that response by facilitating quicker delivery of much needed health information to help the control of epidemics. The evaluation of learning and skills acquisition using the same platform ensures that in-service training is received in the least disruptive way; the cost of training is contained while ensuring that provision of care is maintained and the quality of care not compromised.
This training app approach should be attractive in LMICs struggling with low health budgets, and where capacity building of the health workforce towards health systems strengthening is a priority. Poor digital literacy, electricity and internet connectivity are constraints in the widespread deployment of e-health strategies that support the use of face-to-face training. However, as only a small proportion of participants in our study still favoured this mode, a blended/hybrid approach is more desirable. The utility of this approach has been shown in the management of other conditions such as HIV and tuberculosis.30,31 E-health interventions have been criticized for failing to incorporate relevant experiential and situated learning theories relevant to the workplace.32 We took this point into consideration while developing the contents of our COVID-19 tutorial app by presenting the materials in varying formats (language, graphics, videos) for ease of assimilation and access by all participants.
This study had several limitations. Firstly, the use of pre- and post-test scores in the same cohort of participants to assess learning may not represent a true assessment of effectiveness. A randomised controlled trial would have been a stronger study design. However, this wasn’t feasible; the survey design used was pragmatic in the context of a pandemic, where rapid deployment and assessment are essential. Secondly, findings on the functionality, the usability and the applicability of the app may have been better explored using qualitative methods. However, the COVID-19 pandemic has undoubtedly had a negative impact on the mental and psychological state of many health workers. We therefore opted for a less intrusive method of obtaining feedback by deploying a simple survey tool, with minimal, but targeted, questions. Finally, the app was designed to function on android devices and was not available to those using Apple iPhones.
In conclusion, the InStrat COVID-19 tutorial app appears to be effective for educating health workers at scale in the context of a pandemic, with high acceptability rates among them. The applicability of such an app is likely to extend beyond pandemic preparedness to other healthcare contexts. These could include patient education, supportive supervision and targeting other disease conditions. While the key challenges of implementation and sustainability will need to be addressed, the cost-effective and flexible nature of the e-Learning apps make them feasible for health worker training in LMIC settings.
Supplemental Material
Supplemental material, sj-pdf-1-tai-10.1177_20499361211040704 for Training health workers at scale in Nigeria to fight COVID-19 using the InStrat COVID-19 tutorial app: an e-health interventional study by Akaninyene Otu, Okey Okuzu, Emmanuel Effa, Bassey Ebenso, Soter Ameh, Nrip Nihalani, Obiageli Onwusaka, Tomisin Tawose, Adebola Olayinka and John Walley in Therapeutic Advances in Infectious Disease
Footnotes
Author Contributions: AO, EE, BE and OO conceptualised the study. AO, EE, BE, OO, NR, ObO, JW, AdO and TT designed the study. AO, EE, OO, SA, ObO and TT were involved in data collection, data analysis and interpretation. AO wrote the first draft of the article. EE, BE, JW, OO, SA and AdO critically revised the first draft. All the co-authors reviewed and approved the version of the article to be published.
Conflict of interest statement: The authors declare that there is no conflict of interest.
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We are appreciative of the intellectual contributions from the Foundation for Healthcare Innovation and Development (FHIND) and the UKaid/DFID funded COMDIS-HSD research programme consortia.
ORCID iDs: Akaninyene Otu 
https://orcid.org/0000-0002-6009-2707
Emmanuel Effa
https://orcid.org/0000-0002-5654-2963
Supplemental material: Supplemental material for this article is available online.
Contributor Information
Akaninyene Otu, Department of Infection and Travel Medicine, Leeds Teaching Hospitals NHS Trust, Leeds, UK; Department of Internal Medicine, College of Medical Sciences, University of Calabar, Calabar, Cross River State Nigeria; Foundation for Healthcare Innovation and Development, Calabar, Cross River State, Nigeria.
Okey Okuzu, InStrat LLC, Montclair, NJ, USA; InStrat Global Health Solutions, Abuja, Nigeria.
Emmanuel Effa, Department of Internal Medicine, College of Medical Sciences, University of Calabar, Calabar, Cross River State, Nigeria; Foundation for Healthcare Innovation and Development, Calabar, Cross River State, Nigeria.
Bassey Ebenso, Nuffield Centre for International Health and Development, Leeds Institute of Health Sciences, University of Leeds, Leeds, UK.
Soter Ameh, Department of Community Medicine, Faculty of Medicine, College of Medical Sciences, University of Calabar, Calabar, Nigeria; Department of Global Health Population, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA.
Nrip Nihalani, Plus91 Technologies Pvt Ltd., Pune, India.
Obiageli Onwusaka, Foundation for Healthcare Innovation and Development, Calabar, Cross River State, Nigeria; Faculty of Allied Medical Sciences, College of Medical Sciences, University of Calabar, Calabar, Nigeria.
Tomisin Tawose, InStrat Global Health Solutions, Abuja, Nigeria.
Adebola Olayinka, World Health Organization Country Office Abuja, Nigeria.
John Walley, Nuffield Centre for International Health and Development, Leeds Institute of Health Sciences, University of Leeds, Leeds, UK.
References
- 1.Fagherazzi G, Goetzinger C, Rashid MA, et al. Digital health strategies to fight COVID-19 worldwide: challenges, recommendations, and a call for papers. J Med Internet Res 2020; 22: e19284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.World Health Organization. Infection Prevention and Control (IPC) for COVID-19 virus, https://openwho.org/courses/COVID-19-IPC-EN (2020, accessed 8 March 2021).
- 3.Reeves JJ, Hollandsworth HM, Torriani FJ, et al. Rapid response to COVID-19: health informatics support for outbreak management in an academic health system. J Am Med Inform Assoc 2020; 27: 853–859. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nkengasong JN, Mankoula W.Looming threat of COVID-19 infection in Africa: act collectively, and fast. Lancet 2020; 395: 841–842. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ebenso B, Otu A.Can Nigeria contain the COVID-19 outbreak using lessons from recent epidemics? Lancet Glob Health 2020; 8: e770. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Nigeria Centre for Disease Control. First case of COVID-19 confirmed in Nigeria, https://ncdc.gov.ng/news/227/first-case-of-corona-virus-disease-confirmed-in-nigeria (2020, accessed 8 March 2021).
- 7.Dixit S, Ogundeji YK, Onwujekewe O.How well has Nigeria responded to COVID-19? Brookings, https://www.brookings.edu/blog/future-development/2020/07/02/how-well-has-nigeria-responded-to-covid-19/ (2020, accessed 8 March 2021).
- 8.Gilbert M, Pullano G, Pinotti F, et al. Preparedness and vulnerability of African countries against importations of COVID-19: a modelling study. Lancet 2020; 395: 871–877. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Bhaumik S, Moola S, Tyagi J, et al. Community health workers for pandemic response: a rapid evidence synthesis. BMJ Global Health 2020; 5: e002769. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.World Health Organization. Health workforce requirements for universal health coverage and the sustainable development goals, https://apps.who.int/iris/handle/10665/250330 (2016, accessed 8 March 2021).
- 11.Newsroom. More than half of Sub-Saharan Africa to be connected to mobile by 2025, finds new GSMA study, https://www.gsma.com/newsroom/press-release/more-than-half-of-sub-saharan-africa-to-be-connected-to-mobile-by-2025-finds-new-gsma-study/ (2020, accessed 8 March 2021).
- 12.Statsitica. Number of smartphone users in Nigeria from 2014 to 2025 (in millions), https://www.statista.com/statistics/467187/forecast-of-smartphone-users-in-nigeria/ (2020, accessed 8 March 2021).
- 13.Forenbacher I, Husnjak S, Cvitić I, et al. Determinants of mobile phone ownership in Nigeria. Telecomm Policy 2019; 43(7): 101812. DOI: 10.1016/j.telpol.2019.03.001. [Google Scholar]
- 14.Otu A, Ebenso B, Okuzu O, et al. Using a mHealth tutorial application to change knowledge and attitude of frontline health workers to Ebola virus disease in Nigeria: a before-and-after study. Hum Resour Health 2016; 14: 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Otu A, Okuzu O, Ebenso B, et al. Introduction of mobile health tools to support COVID-19 training and surveillance in Ogun State Nigeria. Front Sustain Cities 2021; 3: 10. [Google Scholar]
- 16.COMDIS-HSD. COVID-19 clinical and operational guidelines for healthcare facilities, https://comdis-hsd.leeds.ac.uk/ (2020, accessed 8 March 2021).
- 17.Nigeria Centre for Disease Control COVID-19 resources, https://covid19.ncdc.gov.ng/resource/ (2020, accessed 8 March 2021).
- 18.World Health Organization. Responding to community spread of COVID-19: interim guidance, https://apps.who.int/iris/handle/10665/331421 (2020, accessed 8 March 2021).
- 19.Wong S. eLearning as good as traditional training for health professionals. Imperial News. London: Imperial college. https://www.imperial.ac.uk/news/163029/elearning-good-traditional-training-health-professionals/ (2015, accessed 11 August 2021). [Google Scholar]
- 20.Rohwer A, Motaze NV, Rehfuess E, et al. E-learning of evidence-based health care (EBHC) to increase EBHC competencies in healthcare professionals: a systematic review. Campbell Syst Rev 2017; 13: 1–147. [Google Scholar]
- 21.Ahmed H.Hybrid e-learning acceptance model: learner perceptions. Decision Sci J Innov Educ 2010; 8: 313–346. [Google Scholar]
- 22.Omar A, Kalalu D, Alijani G.Management of innovative e-learning environments. Acad Educ Leadership J 2011; 15: 37–64. [Google Scholar]
- 23.Nuffield Department for Medicine. Centre for Tropical Medicine and Global Health. New COVID-19 simulation training for smartphones helps African healthcare workers save lives, https://www.tropicalmedicine.ox.ac.uk/news/new-covid-19-simulation-training-for-smartphones-helps-african-healthcare-workers-save-lives (2020, accessed 8 March 2021).
- 24.Barau I, Zubairu M, Mwanza MN, et al. Improving polio vaccination coverage in Nigeria through the use of geographic information system technology. J Infect Dis 2014; 210(Suppl. 1): S102–S110. [DOI] [PubMed] [Google Scholar]
- 25.Kazi AM, Jafri LA.The use of mobile phones in polio eradication. Bull World Health Organ 2016; 94: 153–154. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Picchio M.Is training effective for older workers? IZA world of labour, https://wol.iza.org/articles/is-training-effective-for-older-workers/long (2020, accessed 8 March 2021). [Google Scholar]
- 27.Umar H, Ismail R, Abdul-Hakim R.Regional Inequality of Educational Attainment in Nigeria. J Econo Manage Trade 2014; 4: 420–430. [Google Scholar]
- 28.Nigerian Communications Commission. Subscriber statistics, https://www.ncc.gov.ng/statistics-reports/subscriber-data#monthly-subscriber-technology-data (2020, accessed 8 March 2021).
- 29.Dele-Olowu S, Ogunrinu T, Chukwu D, et al. Leveraging e-learning to train health workers during the COVID-19 pandemic in Nigeria, https://www.clintonhealthaccess.org/leveraging-e-learning-to-train-health-workers-during-the-covid-19-pandemic-in-nigeria/ (2020, accessed 8 March 2021).
- 30.Manyazewal T, Marinucci F, Belay G, et al. Implementation and evaluation of a blended learning course on tuberculosis for front-line health care professionals. Am J Clin Pathol 2017; 147: 285–291. [DOI] [PubMed] [Google Scholar]
- 31.Bertman V, Petracca F, Makunike-Chikwinya B, et al. Health worker text messaging for blended learning, peer support, and mentoring in pediatric and adolescent HIV/AIDS care: a case study in Zimbabwe. Hum Resour Health 2019; 17: 41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Winters N, Langer L, Geniets A.Scoping review assessing the evidence used to support the adoption of mobile health (mHealth) technologies for the education and training of community health workers (CHWs) in low-income and middle-income countries. BMJ Open 2018; 8: e019827. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, sj-pdf-1-tai-10.1177_20499361211040704 for Training health workers at scale in Nigeria to fight COVID-19 using the InStrat COVID-19 tutorial app: an e-health interventional study by Akaninyene Otu, Okey Okuzu, Emmanuel Effa, Bassey Ebenso, Soter Ameh, Nrip Nihalani, Obiageli Onwusaka, Tomisin Tawose, Adebola Olayinka and John Walley in Therapeutic Advances in Infectious Disease