Online capacity building for the health workforce: the case of the Integrated Disease Surveillance and Response for the African region

Abstract

The World Health Organisation (WHO) Regional Office for Africa (AFRO) has developed a comprehensive capacity devel-opment programme to support the successful implementation of the Integrated Disease Surveillance and Response 3rd edition Technical Guidelines (IDSR). As part of the learning program, a series of asyn-chronous online courses are offered on OpenWHO in English, French and Portuguese. This paper describes the use of five IDSR online courses and reports on feedback received from learners on Course 1 in the English series. An online learner survey was developed, and a descriptive analysis was conducted. This paper also reports on use related empirical metadata from the OpenWHO platform. Overall, learners (97%-n/N) of Course 1 IDSR English series indicated a positive perception toward their online learning experience because of the quality of course content, its organization, ease of use and relevance to their workplace needs. In addition, 88% (n/N) of learners reported that they had used their acquired knowledge at least sometimes and 54.4% (n/N) had shared their learning with others. Lastly, the quiz analyses showed an average of right answers of 78.97% for quiz 1 and 69.94% for quiz 2. Online learning is an essential component of a blended capacity development programme and provides cost effective, equitable and impactful learning. Learners who have a learning goal and find their needs met in courses tend to show more satisfaction and motivation to share their learning.

Introduction

OpenWHO is WHO's an interactive online platform for critical knowledge-sharing during health emergencies and beyond. It al-lows frontline responders, healthcare professionals, and anybody interested worldwide to access free, self-paced courses developed and delivered by WHO experts. One of WHO's core values is equity in learning and it translates into accessible content in a wide range of languages and improving learner comprehension and the ease with which courses can be utilised and adapted to different contexts (1).

The Integrated Disease Surveillance and Response (IDSR) strategy 2020-2030 for the WHO African Region (AFRO) is a comprehensive, evidence-based approach for strengthening national public health surveillance and response systems. It was first, adopted by Member States in 1998 with the latest, third revision in 2019. The IDSR strategy objectives are 1) to reinforce national capacity for early detection, complete recording, timely reporting, regular analysis, and prompt feedback of IDSR priority diseases, events, and conditions at all levels, 2) to strengthen national and supranational laboratory capacity to confirm IDSR priority diseases, events, and conditions and 3) to strengthen capacity for public health emergency preparedness and response at all levels (2,3).

To ensure the consistent availability of skilled health workers, the WHO Africa region continues to support Member States to increase the availability of IDSR-trained human resources by scaling up capacity development. This is a critical success factor for IDSR at all levels, and especially at peripheral-level facilities and in the community.

The IDSR capacity development programme provides face-to-face workshops, asynchronous online learning and synchronous webinars. WHO provides the entire curriculum online via the OpenWHO platform in five online courses in 3 different languages (English, French and Portuguese). This blended learning is a combination of conventional face-to-face learning methodologies along with asyn-chronous or synchronous online learning (3), which has provided a promising approach for health-related learning as it is characterised by both traditional and new digital learning techniques. The development of blended learning approaches has significantly increased in the past few years, especially in response to the challenges of delivering face-to-face training posed by the COVID-19 pandemic public health and social measures (PHSMs). Blended learning approaches are now broadly utilised in health training.

Virtual classes, webinars and communities of practice have been developed by the WHO Africa region in collaboration with the University of New Mexico Institute ECHO (Extension for Community Healthcare Outcomes). Adopting a blended learning strategy has shown to be effective in health-related contexts (4). In fact, according to a meta-analysis, blended learning may improve knowledge acquisition for a wide range of learners and disciplines directly relevant to health professions. Blended learning is promising and worthwhile for use in the health professions (5). Furthermore, systematic review has shown that when compared to traditional learning in health education, blended learning consistently produced greater knowledge outputs (6). Synchronous IDSR webinars were organised by Project ECHO and AFRO and have so far provided training to close to 300,000 health professionals on various health emergencies topics. Synchronous webinars for IDSR capacity development programme consists of the virtual classrooms and remote communities of practice. The ECHO learning model (7) is an example of a virtual community of practice (8); it is a case-based low dose, high frequency educational intervention that strengthens knowledge and clinical and public-health practice through a combination of 1) video-conferencing to bridge geography and connect learners, and 2) a case-based virtual community of practice, in order to provide access to equitable, public health knowledge and collaborative peer-to-peer and novice-to-expert problem solving to underserved people globally.

In this paper we conduct descriptive analysis of learner data of the IDSR online courses. The aim is to gain insights from the feedback received from the participants of course 1 (i.e., Introduction to IDSR Technical Guidelines 3rd Edition) in English series. Course 1 consists of four modules: Module 1: An Introduction to the IDSR online courses, Module 2: An overview of IDSR Part 1, Module 3: An Overview of IDSR Part 2, Module 4: Community Based Surveillance.

Literature Review

Training for professional or skills development in health care is considered a common approach. In fact, research comparing no intervention with training intervention has found training essential for knowledge acquisition or performance improvement (6). Organizations adopt different delivery formats to conduct training, including traditional in-person, online or blended. Some of the online training programmes are delivered in a self-paced format, while some use live webinars. Research on online or blended learning indicates a number of factors such as learner characteristics, course design, social presence, and technology affecting a learner's learning experience (9-11). A comparison of different delivery formats (i.e., online, blended, and traditional, which refers to in-person lectures) has found blended learning the most effective because it offers more flexibility and autonomy to learners while learner interaction is maintained through in-person or virtual meetings (6,12). It can therefore be concluded that peer learning is an important part of the learning process.

Research on training effectiveness concludes that major indicators to learner outcomes and satisfaction are social presence, teaching presence, cognitive presence and quality of (4,9,13). Most important, the increased self-efficacy as a result of training has been found to be an important rationale for training effectiveness (7). Cognitive presence, according to (14), is ‘the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry’ (14). The term social presence is defined as the ‘ability of participants to identify with the community, communicate purposefully in a trusting environment, and develop interpersonal relationships by projecting their individual personalities’ (15). An-derson et al (2019) defined teaching presence as ‘the design, facilitation and direction of cognitive and social presences for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes’ (16). Using accessible technology and employing these elements of community of inquiry can result in meaningful learning experiences.

For learners with limited access to quality training, technology can be an effective medium to facilitate their knowledge acquisition and skills devel-opment. A study by Arora and colleagues (2011) using the extension for community healthcare out-comes (ECHO) mode, which offers a peer-to-peer learning platform, found using technology to provide mentorship to healthcare professionals in rural areas resulted in increased HCV cure rate and patient out-comes. Another study by Hunt and colleagues (2022) reported the implications of using ECHO for peer-to- peer telementoring a successful approach for clinical learning, hence better care to patients (17).

Methods

This paper includes commentary analysis that relies on a learner survey and draws on use of related empirical data from the OpenWHO platform itself. The sources of data included responses to the end- of-course survey of one of the courses, called participants feedback survey (PFS), and course metadata of all five courses, which included learner demograph-ics, their quiz performance and activities on the IDSR courses. The survey was administered to one out of five courses of IDSR English series due to the high number of enrolments and completion rate in Course 1 (4345 learners). We also aimed to pilot the survey and apply our learning to other IDSR courses being developed at the time.

Instrumentation

In October and November 2021, in close collaboration with the WHO Regional Office for Africa, an IDSR Participants Feedback Survey (PFS) tool was developed and implemented with the learners who had enrolled in Course 1: Introduction to IDSR Technical Guidelines 3rd Edition of the English series.

Figure 1.

The purpose of the survey was to receive feedback on:

Learners' perception toward online learning experience Course quality and levels of learner satisfaction

3. How the learning was being used with regard to the implementation of the IDSR Technical Guidelines 3rd Edition.

The survey tool and how to strengthen it

The survey consisted of 17 questions. It included multiple-choice, four-point ranking scales (i.e., fully agree, somewhat agree, somewhat disagree, completely disagree), and open-ended questions with focus on the demographics, learning goals, perceptions toward the course and the survey quality. The survey was administered from 22 October to 9 November 2021. At the time of the survey, about 10,000 learners had successfully completed or enrolled in at least one IDSR English series online course: Course 1 launched in February 2021, Course 2 launched in April 2021, Course 3 Course 4 and 5 launched in June 2021 June.

Procedures

The course metadata for all OpenWHO IDSR online courses, launched between 8 February 2021 and 14 April 2022, were obtained from OpenWHO user analytics on a data run dated until 30 June 2022. IDSR course data included the analytics from the OpenWHO integrated statistical data on five IDSR English online courses, five French online courses and one Portuguese online course.

Anonymized course reports data sets were extracted from the OpenWHO reporting tool, providing raw data including basic demographics on OpenWHO users (e.g., self-declared age, gender, professional affiliation, and nationality information), which learners entered at the time of registration on the platform.

OpenWHO course reports also included course-specific learners' performance and course activity indicators (such as module completion), including videos, self-assessments, download activity status, quiz performance, and obtention of the certificate, as well as tracked average session duration and time-stamped activity usage patterns.

To administer the survey, on 22 October 2021, 4,345 learners from OpenWHO IDSR English Course 1 were invited to take part in the survey. 375 responses were received, giving a response rate of 8.6%. Participants were sent three reminders, the first after a week, the second after two weeks and the third just before the survey closed.

Data Analysis

Course registration and completion data captured by the OpenWHO analytics systems were analyzed using descriptive analysis methods to understand learner demographics, certifications, and attendance on all IDSR courses. Course completion was defined as watching all videos and completing the end-of-course learning quizzes with a score of at least 80%. Questions were scored as correct or incorrect and no partial credit was granted. Demographic information, including age, gender, and professional affiliation were not mandatory; therefore, analyses on these variables were based on the learners who provided information voluntarily.

Analysis of the data from the survey is based on the total number of enrolled learners by 22 October 2021 on English Course 1 which was launched in February 2021. A descriptive analysis was used to analyse the data.

Results

OpenWHO metadata

Enrolments to the OpenWHO IDSR online courses have been steady since the launch of the first course in February 2021. By September 2022, there had been a total of 20,154 enrolments to all courses. By November 2022, eleven courses had been opened, all five of the English and French and 2 in Por-tuguese. Overall, 8,184 learners had obtained Record of Achievement (see Fig. 1).

The learner demographic indicated that 64 percent of the learners were male and 36% female and 0.7% selected ‘other’ for gender. The largest learner group (40%) were aged 20-29, followed by (33.1%) age group 30-39, and clearly less participants (13.7%) were aged 40-49. 4.8% were 50-59 and just 1% were 60-69.

Enrolments in African countries were high with thirteen of the top twenty countries from the African continent, the highest use was in Nigeria, Ethiopia and Kenya. The top five learners' affiliations were: Student 28.9%, Health Care Professional 26.6%, Ministry of Health 10.1%, Other 6.7% and Volunteer 5.4% (see Table I). The geographic distribution of learners is displayed in the table as well.

Demographic characteristics of the total enrolled learners to all IDSR online courses from 9 February 2021 to 30 September 2022.

Results of the Participant Feedback Survey of English Course 1

The survey was administered in October and November 2021 to receive feedback on learners' perception toward online learning experience. In total 4345 learners from OpenWHO IDSR English Course 1 were invited to take part in the survey. 375 responses were received, giving a response rate of 8.6%. Survey responses were analyzed using frequency and descriptive analysis. Of the 375 learners who responded to the survey, 100 had also completed all 5 English courses. The learners who had completed all five courses in the English series were disaggregated in the analysis. Survey results of Course 1 (i.e., Introduction to IDSR) are reported in this paper.

41.6 percent indicated that they heard about the courses via OpenWHO email and 40.8% from the WHO website. On reasons for the course enrollment, 38.67% of learners reported that they enrolled on an IDSR online course to be prepared to carry out a specific professional responsibility. Further, 22.67% reported that the course was to assist them with future job opportunities or roles, while 13.33% said it was to assist them with the implementation of the IDSR Technical Guidelines 3rd Edition. Lastly, 8% indicated it was mandatory training for their role, with 8% out of interest and 1.87% for other reasons.

On perceptions toward the course, the survey results indicated high levels of participant satisfaction with the IDSR online courses, with a lack of time given as the main reason for not completing a course. A majority of the learners (97%) reported that they had acquired new IDSR knowledge and/or skills as a result of participating in an IDSR online course, with 41.6% reporting that their knowledge was most improved regarding the early detection, preparedness and response to epidemic-prone diseases and all other public health emergencies (See Fig. 2). Most of the learners (88%) reported that they had used their learning at least sometimes, with 54% reporting that they often used their learning.

Most of the learners who had completed Course 1 found it well organised and easy to follow (72.5%) and that they were satisfied with the course (71.6%). A majority of the learners indicated that the course met their learning needs (71.5%), while 72.6% indicated the course had increased their knowledge and it was relevant to their work expectations (72.2%). Lastly, 73.3% indicated that they would recommend the course to other learners. For preferred learning methods, survey responses showed videos, downloadable documents, slides, quizzes and learning exercises as the top five preferred methods (see Fig. 3).

Table I.

Demographic characteristics of the total enrolled learners to all IDSR online courses from 9 February 2021 to 30 September 2022.

Characteristics	Values, n (%)
Total enrolments	20,154(100)
Gender
Female	2,670 (36.13%)
Male	4,716 (63.81%)
Other	5 (0.7%)
Age group (years)
<20	628 (7.1%)
20-29	2,875 (40%)
30-39	2,144 (33.1%)
40-49	878 (13.7%)
50-59	292 (4.8%)
60-69	59 (1.0%)
70<	17 (0.3%)
Professional affiliation (top 10)
1. Student	1,941 (28.9%)
2. Health Care Professional	1,788 (26.6%)
3. Ministry of Health	677 (10.1)
4. Health Expert	543 (8.1%)
5. Other	447 (6.7%)
6. Volunteer	363 (5.4%)
7. NGO	290 (4.3%)
8. WHO Staff	259 (3.9%)
9. Health Institute	155 (2.3%)
10. International Organisation	124 (1.8%)
11. Other Ministry	83 (1.2%)
Geographic distribution of learners (top 10)a
1. India	708 (11.2%)
2. Nigeria	707 (11.2%)
3. Ethiopia	324 (5.3%)
4. Kenya	254 (4.0%)
5. Cameroon	237 (3.7%)
6. South Africa	215 (3.4%)
7. Ghana	210 (3.3%)
8. China	196 (3.1%)
9. Pakistan	181 (2.9%)
10. DRC	177 (2.8%)
11. USA	169 (2.7%)
12. Philippines	141 (2.2%)
13. Somalia	116 (1.8%)
14. Sudan	103 (1.6%
15. Zambia	98 (1.5%)
16. Uganda	96 (1.5%)
17. Rwanda	88 (1.4%)
18. UK	83 (1.3%)
19. Egypt	79 (1.2%)
20. Tanzania	69 (1.1%)

Figure 2.

Learner satisfaction and relevance to learning needs in Course 1.

Figure 3.

Preferred learning material formats on online course.

For knowledge gain on Course 1 objectives, 43.2% reported that their knowledge on surveillance activities at all levels was most improved. 41.6 percent reported that their knowledge on the early detection, preparedness and response to epidemic-prone diseases and all other public health emergencies was improved. 36 percent reported that they gained more knowledge on their work level-specific responsibilities. 34.3 percent reported that their knowledge on the IDSR Technical Guidelines as a resource for developing training, supervision, monitoring and evaluation materials and resources was improved. 31.9 percent reported that they gained more knowledge on standard case definitions for threshold levels that initiate action for responding to specific diseases.

On the use of materials, the learners indicated that they used their learning materials offline with 54.4% having shared them in meetings with colleagues and partners, while 37.8% with communities. Only 4% of the learners indicated that they did not intend to share their learning. On learners' perception toward accreditation, 94% showed support for it, indicating Public Health accreditation as the most valuable (70%) followed by Continuous Professional Development (45%). Lastly, on the quality of the survey and ways to improve it, 24% indicated that the questions should be made easier to understand, thus, the current survey tool in use on the OpenWHO IDSR online courses has been modified.

Discussion

Overall, the survey produced good evidence of high levels of participant satisfaction with the IDSR on-line Course 1 and how learning acquired online is used to implement the IDSR Technical Guidelines 3rd Edition. Results of this study provide strong evidence that the overall satisfaction of learners with a course depends on the extent to which they perceive it as useful for their present and future professional development, and how well organized the course is (18). Aligned with previous studies, learners show a positive perception toward a course that meets their learning needs in that the training should meet their expectations (19,20). Findings of the study support the importance of relevance of learning materials for on-the-job performance, hence an effective training design (19,20), as adult learners seek specific gains in relation to their learning experience.

Our survey analyses yielded a time-efficient preliminary measure for course effectiveness that measures more than the participants' reactions. Surveys, if well-designed, have been shown appropriate tools to effectively measure a course quality and nudge the learners to take the desired actions (e.g., being accountable, applying or sharing their newly acquired knowledge or skills) (21,22). As training evaluations can be time-consuming and labour-intensive (23), surveys can be used to inform the teams of inherent barriers to learning and preliminary quality enhancement initiative (24).

Results of this paper also demonstrated that the cognitive outcomes (i.e., enhanced knowledge and better mental models) of a course can be accomplished if course objectives, learning materials, and the needs of the target audience are well aligned. Course 1 learners scored, on average, high on both quizzes indicating their knowledge gain. Additionally, learners' motivation to enrol according to their country of origin-the highest enrolment was in Nigeria, Ethiopia and Kenya-in line with their affiliations indicated that learners seek learning based on existing or emerging needs. Therefore, the affective learning outcomes of training (i.e., motivation or self-efficacy) plays an important role in the acquisition of knowledge or skills as a result of a training program (19).

The online IDSR Course 1 showed an increase in learner's knowledge acquisition. Further, their knowledge sharing with co-workers and community showed that the course met the learner's expectations and they were fully cognizant of the importance of completing this course along with other IDSR courses. The positive perception of learners toward the IDSR course and their online learning experience indicated that the course was relevant and well-organized meeting their expectations.

Conclusion

The successful implementation of the IDSR Technical Guidelines 3rd Edition requires a well-trained, competent and dedicated workforce. This workforce is essential for the operation of a functional health system that can detect and respond to the leading causes of illness, death, and disability. Recent health emergencies, such as the COVID-19 pandemic, have shown the need for IDSR technical guideline related learning. Noting that workshop-based face-to-face training is unlikely to reach all health workers due to logistic and economic constraints, online learning is encouraged as a complementary approach to reaching participants.

Providing online learning for knowledge acquisition can be considered an effective approach to learning as part of a blended learning model. Enrolments in African countries were high with thirteen of the top twenty countries from the African continent. In addition, for acquiring a new skill or honing an existing one, support within a community of practice and in-person meetings through which learners can observe, practice, demonstrate, and receive feedback are essential. In short, online learning was found to be an essential component of a blended capacity development programme, providing cost effective, equitable and impactful learning.

In this study, using a survey tool provided sufficient information on learners' perception of the IDSR course they completed, their learning experience, and use of the materials. While relying on self-report surveys have been considered insufficient to draw conclusions on course effectiveness, it can certainly provide preliminary measures for quality enhancement.

Limitations

We acknowledge the limitation of self-reported data as respondents might report inaccurate information due to recall or response biases. We were not able to validate the responses provided. Also, the traditionally low response rate in the MOOCs industry might lead to misrepresentation and skew results, yet the limitation is well-known and described in the literature (25,26). Finally, the descriptive nature of analysis does not allow us to draw cause-effect relationships and should be explored in further studies.