What knowledge is available on massive open online courses in nursing and academic healthcare sciences education? A rapid review

Abstract

Background

With the recent challenges due to the Coronavirus 2019 outbreak, distance learning has been largely introduced in healthcare sciences curricula, and universities have been called upon to share learning opportunities with each other to ensure continuity of education and delivery of new graduates to the health system. However, decisions about its introduction should be supported by up-to-date evidence capable of providing an overview of available knowledge.

Objectives

To map the (a) state of research on massive open online courses in undergraduate and postgraduate health sciences education, (b) evaluation methods and tools used to measure learning outcomes, and (c) factors increasing their effectiveness as documented to date.

Design

A rapid review following the preferred reporting items for systematic reviews and meta-analysis guidelines.

Methods

PubMed, the Cumulative Index to Nursing and Allied Health Literature, Cochrane, Scopus, PsycInfo and Medline (via Ovid) were searched. Primary studies reporting one or more massive open online course (1) devoted to undergraduate and/or postgraduate students in nursing and healthcare sciences (2), written in English (3) with abstract available (4) and published up to February 18th, 2020 were all included. After having assessed the need for a review and the topic itself (a), the literature search was performed (b), studies were screened and selected (c), data was extracted (d), and the findings were summarised (e).

Results

Thirty-six studies emerged with mainly an explorative/descriptive or case study design. The courses have been developed mainly by universities alone or in collaboration with institutions mainly in US, Sweden and the UK. Their delivery has been performed at multi-national levels, mainly in English, and with a number of participants ranging from 45 to >23,000. The duration spanned from two weeks to six months on clinical topics (e.g., emergency medicine) to methods (e.g., statistics). The target audience has been mainly mixed, including students, healthcare professionals, and lay citizens. Evaluation methods and tools have been described in 28 studies, and multiple-choice questions were most frequently adopted. Factors affecting the effectiveness of massive open online courses have been identified analysing the courses themselves and the participants.

Conclusion

Massive open online courses have recently started to be studied in healthcare sciences: these can be useful to educate students, mainly as elective courses, and to educate a massive audience, thus embodying the third mission of the university. The complexity of factors increasing effectiveness suggests the need for a multidisciplinary approach both in their design and implementation.

1. Introduction

In the new century, digital technologies have become the pre-eminent strategy to expand academic accessibility for all students (Bendezu-Quispe et al., 2017; Daniel, 2012). In this context, massive open online courses (MOOCs) have been developed as a new form of education devoted to different topics as science, engineering, arts (Hew, 2015), and recently also healthcare sciences (Stathakarou et al., 2018b). The word “MOOC” was first used in 2008 to describe a course taught by Stephen Downes and George Siemens (Downes, 2008) entitled “Connectivism and Connective Knowledge” at the University of Manitoba (Canada). While it became internationally popular, Stanford University (US) offered three courses in 2011 in which >100,000 students were enrolled from 190 different countries (Alturkistani et al., 2019).

MOOCs are intended to be: (1) “massive” because thousands of students can access them, (2) “open” because participants do not pay fees, (3) “online” because they are offered through the Web, and (4) “courses” because they are shaped around specific learning objectives by offering structured contents (Hoy, 2014). To date, two main type of MOOCs have been identified, namely cMOOC (or Connectivist MOOC) and xMOOC (or ExtendedMooc). Connectivist MOOCs represented the original pedagogical frameworks developed by Downes and colleagues emphasising social learning and network interactions to enhance knowledge (Downes, 2008; Rodriguez, 2013; Siemens, 2012). Extended MOOCs were instead established later reflecting a passive learning with a unidirectional teaching process (Chan et al., 2015). Both pedagogical frameworks have been defined to justify the use of different methodologies and tools, including short pre-recorded video lessons, PowerPoints, (Hendriks et al., 2019), discussion forums, and feedbacks from teachers (de Jong et al., 2020), as well as exercises with virtual patients (Kononowicz et al., 2015).

Moreover, to date, MOOCs have been documented to be a potential strategy for distance learning for healthcare students (Chan et al., 2019), representing an opportunity for innovation of education in this context. Furthermore, MOOCs have been considered able to educate a large number of students (Hendriks et al., 2019) thus providing an answer to the expected shortage of healthcare workers globally of around 12.9 million individuals globally (World Health Organization, 2013).

Despite several advantages, also limits hindering the effectiveness of MOOCs have been reported to date, encompassing the degree of computer literacy and language skills (Liyanagunawardena and Williams, 2014) and the accessibility of electronic devices and Internet access (Pickering and Swinnerton, 2017). Low completion rates and the difficulties in assessing the knowledge learned (Jia et al., 2019) have been also identified as main issues. Firstly, the completion rates of MOOCs in the medical healthcare sciences field have been reported to range between 4.3% and 11% (Maxwell et al., 2018), mainly due to participants' limited interactions (Fricton et al., 2015) and the lack of face-to-face sessions thereby generating a sense of isolation and disconnection (Aboshady et al., 2015). Secondly, the voluntary nature of MOOCs, as well as the ease with which participants can register or stop the course, prevent their completion (Fricton et al., 2015). Thirdly, the assessment in MOOCs occurs in different forms (e.g., traditional multiple-choice questions, peer feedback) depending on the course (Chan et al., 2019), thus lacking a reliable, standardised methodology. Accordingly, institutions have been reported to be hesitant to accredit MOOCs (McAuley et al., 2010) also in healthcare sciences and available evidence (Alturkistani et al., 2020) has grown more slowly compared to other academic fields (Kearney et al., 2016).

In this context and despite its limits, MOOCs have become increasingly popular in the healthcare educational context as a response to social distancing and physical isolation imposed by governments due to the Coronavirus 2019 (COVID-19) outbreak (International Association of Universities, 2020). Accordingly, universities worldwide have introduced different strategies for distance learning (e.g., MOOCs, online lessons) in several healthcare curricula, including nursing (Dewart et al., 2020). However, decisions about its introduction should be supported by up-to-date evidence capable of providing an overview of available knowledge in addition to that summarised by recent systematic reviews on their efficacy in teaching in medical education (Zhao et al., 2018) and their effectiveness (Rowe et al., 2019) and evaluation methods (Alturkistani et al., 2020).

Therefore, in order to address the need for strategies to overcome the limits affecting MOOCs and to fill gaps in literature, we conducted a rapid review with the aim of informing the scientific and academic community on how MOOCs devoted to nursing students and health care students have been studied in the literature, reporting their evaluation methods and factors affecting their effectiveness. Accordingly, we aimed to summarise (a) the state of the knowledge on MOOCs; (b) assessment methods of MOOCS; and (c) factors affecting effectiveness of MOOCs in the field of nursing and health science education.

2. Methods

2.1. Study design

A Rapid Review as a knowledge-generation strategy, capable of summarising evidence and using “abbreviated” systematic review methods to provide university stakeholders and policymakers with relevant and state-of-the-art evidence has been undertaken (Tricco et al., 2017). The steps proposed by Tricco et al. (2017) were followed: (a) performing an assessment of needs and selecting/redefining the topic, (b) performing a literature search, (c) screening and selecting studies, (d) extracting the data, and (e) summarising the results. Methods and results have been reported in accordance with the preferred reporting items for systematic reviews and meta-analysis guidelines (Moher et al., 2009) (Supplementary Table 1).

2.2. Assessing the needs, selecting and refining the topic

Due to the COVID-19 outbreak that forced universities in Italy to close, it was decided that all healthcare professional (HCP) undergraduate and postgraduate students had to stay at home; consequently, there was the need for an immediate revision of the curriculum delivery from in person to distance learning. Amid the challenges posed by the pandemic, a meeting was held among researchers (see authors). In this context, given the changes in the curriculum (Morin, 2020), as well as the need to educate immediately students about the pandemic using open online resources (BMJ Best practice, 2020), and the lack of resources in some universities calling for cooperation in sharing learning resources (Marinoni and de Wit, 2020), three research questions were identified:

• (1)“What is the state of research published to date on MOOCs in undergraduate and postgraduate health sciences education?”
• (2)“What evaluation methods and tools have been used to date to measure MOOC's learning outcomes in this population?” and
• (3)“What factors have been documented to date to influence the effectiveness of MOOCs in this context?”

2.3. Performing the literature search

Five databases including PubMed, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane, Scopus, PsycInfo, and Medline (via Ovid) were searched by adopting the following keywords: “Massive open online course”, “MOOC”, “health”, “nurse”, “nursing”, and “medicine”. According to each database, the search strings were changed slightly (Supplementary Table 2).

2.4. Screening and selecting studies

As inclusion criteria, we considered primary studies with the following features: (1) that described one or more delivered MOOCs, (2) that concerned MOOCs devoted to undergraduate or postgraduate students in nursing and healthcare sciences, (3) that were written in English and Italian, (4) had an abstract available, and (5) were published up to February 18th, 2020 with no start date.

In screening and selecting the literature, the search strategy was broader and not limited to nursing students for two main reasons: firstly (a), MOOCs are intended to be massive and open in order to reach and educate a large audience of learners (especially in cMOOCs) (Downes, 2015); and secondly (b) the nursing discipline itself is encouraged to undertake—when appropriate—inter-professional educational strategies aimed at shaping future attitudes of students in working in multidisciplinary teams (Lennen and Miller, 2016).

Therefore, we excluded opinions, letters, and Delphi studies with full texts not available. A total of 1156 studies emerged. Two researchers (AP, JL) independently evaluated the eligibility (titles and abstracts) of the 773 studies retrieved, thereby excluding 704 studies. Three researchers (AP, BDC, JL) analysed the full text of 69 eligible studies and included 36 studies (Fig. 1 ).

Fig. 1.

Flow chart of the search strategy and results according to the PRISMA statement (Moher et al., 2009).

Abbreviations: CINAHL, Cumulative Index to Nursing and Allied Health Literature; MOOC, Massive Open Online Course; n, sample size; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

2.5. Extracting the data

According to the research questions, the following data was extracted:

• (1)research question n. 1: author(s); year of publication; study aim(s); study design; country(ies) that developed and distributed the MOOC(s); MOOC design data when available, encompassing (a) platform(s) and provider(s); (b) topic(s); (c) language(s) (mono or multilingual); (d) target population (including type, number, mono- or multidisciplinary, duration, dropout rates); (e) teaching methodologies; and (f) pedagogical foundations, if described;
• (2)research question n. 2: evaluation method(s) and tools adopted;
• (3)research question n. 3: facilitators of and barriers to MOOC's effectiveness.

2.6. Summarising the findings

Data from the studies included was collected and summarised describing the following: (1) the main features of MOOCs to provide a summary of the extent and the nature of the existing literature regarding MOOCs targeting undergraduate and postgraduate nursing and healthcare science students; (2) the MOOCs' evaluation systems documented to date: also in this case, a summary of the methods used has been developed by analysing data extracted from the primary studies; and (3) factors affecting the effectiveness of the MOOCs that have been categorised in the MOOC itself and in the participants. For each factor identified independently by two researchers and then agreed upon, the influence – as improving or hindering the effectiveness of the MOOCs- has been summarised.

3. Results

3.1. MOOCs in healthcare education: the state of the research

Thirty-six studies published from 2014 (Subhi et al., 2014) to 2020 (Canavese et al., 2020) emerged; around one third of them (13; 36.1%) were based on a descriptive/explorative study design (e.g., Canavese et al., 2020) and 13 (36.1%) on case study designs (e.g., Castle et al., 2016). Three studies were retrospective (e.g., Frank et al., 2016), and one was prospective in nature (Jia et al., 2019); only three (8%) were randomized control trials (Bowen et al., 2014; Hossain et al., 2015; Stathakarou et al., 2018b) (Table 1 ). The remaining were qualitative or mixed-methods study designs.

Table 1.

Characteristics of studies included.

Authors Year Country developed Country delivered	Study aims Study design	Platform providers	Topic or title Mono/multidisciplinary Mono/multilingual	Target students Number registered Duration Dropouts	Methods (e.g. video, audio) Underlying theory	Evaluation and assessment systems
Canavese et al. 2020 Brazil Brazil	To describe the design, development, and assessment process of an online-based course that uses the MOOC format to offer a new and pioneering training on the topic of sexual rights and health involving intersex (LGBTI+) health promotion in Brazil Descriptive and analytic study	Moodle software and a distance learning platform University and other institutions (Ministry of Health and Federal Institute)	Sexual rights and health involving intersex (LGBTI+) health promotion in Brazil Multidisciplinary NR	Global audience but especially HCPs, included students 3000 30 h 80%	Video or podcast; scientific text and optional complementary material NR	The final certificate was awarded after approval in both modules (minimum grade of 75%). Test with 5 questions at the end of each module
Hendriks et al. 2020 NR NR	To investigate the quality of the instructional design of medical MOOCs that are eligible for integration in formal campus education Explorative (33 MOOCs)	10 different platforms (45% MOOCs used Coursera) Universities (26 MOOCs), partnership of institutions (3 MOOCs), health organisations (2 MOOCs)	Medical condition or disease NR Monolingual, English	Not explicit (general audience of students) NR NR NR	Formulation of personal goals, (learning or performance goals); real patients describing their experiences; videos of operations; open-access research articles; feedback largely automated or by peers, not by experts; collective knowledge via learning from each other; activities attempt to activate learners' relevant prior knowledge or experience; complex or ill-structured problems with multiple solutions NR	NR
Alturkistani et al. 2019 NR NR	To trial data collection methods to inform course development and to reflect on evaluation methodology for future course runs Qualitative study	NR Health Q	Data Science Essentials: Real World Evidence Multidisciplinary NR	Undergraduate students in data science within healthcare 191 2 months NR	NR	Data from pre-course and post-course surveys, quizzes, and tests
Beer 2019 UK NR	To evaluate a 2-week MOOC as part of a MSc in nursing to establish whether learners are demonstrating transformative learning Case study	FutureLearn University	Healthcare research for HCPs Multidisciplinary Monolingual, English	MSc nursing students, members of the public 1160 2 weeks NR	Video; audio; discussion of an article; peer review; quiz; test Transformative learning	Rubric§ for a direct measure of learning that is consistent in its evaluation
Chan et al. 2019 Guatemala 85% participants from Spain, Guatemala, Colombia, Mexico, Perù	To explore students' behaviour from a MOOC on Health Emergencies, analysing the completion and the drop-out rates Explorative	Telescope project (.LRN platform) University	Health Emergencies to introducing first aid and emergency treatment Multidisciplinary NR	Global audience (60% of participants studied at university) 2144 5 weeks 97.6%	Introductory unit (general aspects and methodology of MOOC); learning units (1 per week); each unit had between 3 and 5 videos (6–9 min in length); support materials (PPT presentation, interactive animations, learning activities supported by cloud-based tool) show different real-life situations in which students have to make a decision or solve a case; collaborative discussion forums with experts solving doubts; mind map Connectivism theory	Peer assessment (with a rubric to evaluate classmates for each learning activity); self-assessment tests; multiple-choice questions; post-questionnaire with 20 questions (Likert scale); a questionnaire related to dropout aspects for participants who did not finish the course
Hendriks et al. 2019 NR NR	To specify the materials and teaching approaches available in medical MOOCs that qualify for integration in formal student education Explorative (33 MOOCs)	10 different platforms Universities (26 MOOCs), partnership of institutions (3 MOOCs), health organisations (2 MOOCs)	Medical conditions or diseases NR Monolingual, English	Not explicit (general audience of students) NR NR NR	Digital text or textbook and videos (100%); link to external online resources (94%); illustrations or simulations (48%); prompts to use external links (42%); independent activities related to content and PPT slides with voice-over (30%); PPT slides (18%); audio files (9%); recorded traditional lecture and thought trees or word clouds (6%); flashcards (3%); discussion boards for asking questions or for discussing course materials (94%), introducing oneself (70%), answering question prompted (61%), prompting to respond to peers (9%); virtual patient cases (55%); games (12%); virtual microscope activities (3%) Only objectivist-individual or constructivist-individual (6%); focusing constructivism (45%), objectivism (39%), equal variety in objectivist/constructivist (12%); all course favoured individuals teaching modes (60–100% of course teaching modes)	Formal assessment structures (88%), of these 4 also offered an optional exam for continuing medical education credit. Multiple-choice questions (100%); open-ended questions with long answer (39%); open-ended questions peer reviewed (21%); open-ended questions with short answer (9%); most courses offering weekly assessment
Jia et al. 2019 China NR	To compare the differences between the “blended learners” and “social learners” in course completion, participation, performance, and online interactions Prospective cohort study	Chinese MOOC platform iCourse 163 University	Health assessment NR NR	Blended learners (2nd year nursing undergraduates), social learners (global audience) 4106 16 weeks 92%	Videos; tests; reading materials; case discussions; participating in the online discussions NR	433 test questions; all videos >10 min had a resident question in the middle and an in-class test with 2–15 questions at the end, test for each topic with 15–20 questions; final exam with 70 questions
Lan et al. 2019 China NR	To investigate learners' behaviours and correlate patterns of self-regulated learning with performance and achievement Case study	Coursera University	Implant dentistry Monodisciplinary NR	Undergraduate dentistry students, fresh graduates, junior clinicians and senior, experienced practitioners 7608 5 weeks 83.2%	5 modules in which there were lectures, clinical procedures videos, short webinars, discussion boards, suggested readings, practical tutorials, case studies; regular “checkpoints” and mentoring by experienced tutors and peers NR	Completed graded course assessments expressed a final grade with passing grade > 75%; non-graded peer assessment; self-assessment; multiple-choice quizzes; possibility to purchase a certificate
Roller-Wirnsberger et al. 2019 Austria Europe	To provide background information on MOOCs in general, and specifically to describe a MOOC under the umbrella of the international project “Screening for Chronic Kidney Disease among Older People across Europe” Case study	ICT platform iMoox Pan-European Consortium; Horizon 2020 programme of the European Commission; University	Screening for chronic kidney disease among older people across Europe; chronic kidney disease and its management in older patients with complex care needs Multidisciplinary Monolingual, English	Medical and nursing students during their first years of education and training as well as general practitioners NR NR NR	3 courses designed with chapters and learning goals; video clips (15–20 min.); documents, links, asynchronous communication possibilities, textual description, graphics animations and audio (interactive mode); initial-guided course; a teacher available to guide students through the whole MOOC; every 2 weeks 1 course has to be finished Extended MOOC: theories of “cognitivism”	Assessment after all modules with passing grade > 75% score and SCOPE “C certificate”
Bakayev et al. 2018 Russia Russia	To show the demand for the developed online course “Physical Culture: Theory” in the educational process among students Case study	Open Education University	Physical Culture: Theory Multidisciplinary NR	University students, independent trainees, lecturers, university staff 4400 6 weeks 46.5%	Lectures; recording videos; 6 modules, 22 academic hours, including 18 in electronic formats, individual work of students and its control takes 50 h; kick-off lecture for university students is mandatory NR	Successful completion of the course to get a certificate, opportunity to transfer credit in higher education institution; final test; for external different variants of passing final test (online testing, “blended online and offline proctoring format” in another university classroom equipped with video cameras)
Magaña-Valladares et al. 2018 Mexico Mexico	To show that carefully designed educational interventions can improve service professionals' competencies and that regardless of the modality, face-to-face, blended learning, or MOOC, high graduation rates can be achieved Pre-post	Moodle Local ministries of health	Breast cancer Multidisciplinary NR	Primary HCPs and medical students 11,569 40 h 12%	Video; interactive exercises; gamification. Stratification based on different profession profile with different learning strategies Constructivism theory	Assessment for every module. A “cube of competencies” was designed.
Shang & Liu 2018 China China	To describe a blended learning course, which combine online self-learning with classroom teaching and to evaluate the teaching effects Case-control study	iCourses (Chinese university MOOC platform) University	Medical physiology NR Monolingual, Chinese	Nursing students at 4-yr bachelor's degree programme at university 54–55 2 weeks NR	Video lesson (5–15 min.); 6 study blocks, each containing a study outline, 1–3 micro-lesson videos, and 1 online quiz; 1-h classroom question session and a 2-h student presentation and discussion session during and after the online study weeks; online homework and quizzes; experimental videos; PPT lecture NR	8 online quizzes; final student grade was then composed of a final examination (70%), online credit (20%), and practical assessment (10%); multiple-choice questions; long- and short-answer questions; 13 online tasks to get full online credit.
Stathakarou et al. 2018a Sweden NR	To describe the process of improving the quality of the virtual patients components by the application of literature frameworks and extending them with a feedback module using the learning outcomes as a starting point Case study	OpenLabyrinth 3 open source Virtual patients system University	Urology NR NR	Not explicit; learners (medical students) 378 NR NR	Interactive patient scenarios: bladder cancer virtual patients; branched virtual patients: possibility to identify relevant information from a set of anonymous patient-related data, conduct physical exams, laboratory tests and make diagnostic and therapeutic decisions without any real-world repercussions; learners select best available option at each stage; choices and the available paths represent different clinical scenarios and outcomes NR	NR
Stathakarou et al. 2018b Sweden Participants from 172 countries (most from USA, Sweden, India)	To explore the learners' interaction pattern with Virtual Patients in MOOCs; how branching points in a virtual patient case may influence the dropout rate of learners within the virtual patients RCT	edX University	Introduction to Urology (common symptoms related to the urinary tract) Multidisciplinary NR	Medical students 4925 5 weeks 89.5%	Video components; multiple-choice questions; 3D models; glossary; discussion forum; webinar; 2 virtual patient cases NR	Final exam; students could acquire course credits for the activity completion (not mandatory)
Berman et al. 2017 Sweden Participants from 185 countries (most from USA, India, UK)	To explore learners' perceptions of using virtual patients in a behavioural medicine MOOCs and there by describe innovative ways of disseminating knowledge in health-related areas Case study	edX University	Behavioural Medicine and Motivation to Change Multidisciplinary NR	Global: HCPs, students in health professions, other professional groups, laypeople 19,236 5 weeks 96.2%	5 sections which contain 2-branched narrative interactive virtual patients (with stress and sleep problems) consisting of video recordings of a live standardised patient, with multiple clinical decision points and narration unfolding depending on learners' choices; audio, video, interactive elements; discussion forum; learners see the outcomes of their choices NR	Certification was released with completion of 65% of course materials; final course online exit survey; multiple-choice questions; free text questions; post shared in the discussion forum; 4 questions at the end of each virtual patient assignment; final course project assignment, concerned participants experience with virtual patients
Darcy & Lock 2017 USA NR	To describe the development of 3 technology-based innovations aimed at improving outcomes for children and adolescents with eating disorders Case study	NR University and US National Institutes of Health	Family-based therapy for adolescents with anorexia nervosa Multidisciplinary NR	Medical doctor psychiatrists, doctoral-level psychologists, master's level family therapists, doctoral-level graduate students, registered dietician 45 12 weeks 22%	Training modules: 6–7 lectures, comprising 5–6 very short (3–4 min.) didactic videos on treatment model and a role-play therapy session (or series of short role-played scenarios) with a typical case of anorexia nervosa; prescribed reading; course is delivered sequentially, with a new lecture delivered every Monday morning; weight chart to track progress, intake evaluation report; discussion forum Case method teaching	Completion: finishing >80% of videos and assignments. Tested clinical decision making; standardised assessments
Evans et al. 2017 USA 170 countries	To describe the development and evaluation of MOOC on Ebola virus disease as the 2014–2015 epidemic was ongoing Mixed method	Coursera University and partner institutions	Ebola virus Multidisciplinary Monolingual, English	Global audience, included HCPs (learners with medical, nursing, or public health backgrounds) 7082 6 weeks 90%	Individual lectures; panel discussions; PPT; videos lessons with questions or reflections for self-checks generally at the end of each video; open-access resources and discussion board NR	Quizzes (9–11 true/false, closed-ended and multiple-choice). Grades were calculated based on quizzes and participation in discussion boards. Quizzes were averaged and accounted for 90% of the grade. The other 10% was based on discussion posts; six were required to earn full credit. Verified certificate; passing grade (70%).
Harvey et al. 2017 NR 110 countries	To describe 2 MOOCs about the management of spinal cord injuries; to determine whether the MOOCs increased usage of an existing freely available online-learning module created by the International Spinal Cord Society upon which the MOOC was based Descriptive study	Physiopedia International society	Physiotherapy management of spinal cord injuries Monodisciplinary NR	Physiotherapy students and junior physiotherapists 13,509 10 weeks 43–35%	Read around a topic. Closed Facebook page (2 or 3 discussion threads were posted each week), video clip NR	Voluntary; obligatory for credits. Opportunities at different stages to assess own knowledge and clinical reasoning skills; multiple-choice self-assessments and interactive activities dispersed throughout the content; pre- and post-MOOC knowledge assessment with 20 multiple-choice questions. Online Google analytic tracking software was used to record daily usage.
Koch & Hägglund 2017 Sweden Participants from 162 countries (most from USA, India, UK)	To evaluate how students and teachers perceived the course Case study	edX University	eHealth – Opportunities and Challenges Multidisciplinary NR	Global audience of students; 13,302 6 weeks 95.7%	Topic released at the beginning of each week; introductory video lecture followed by a series of short videos about subtopics and literature; every course assignment of the preceding weeks (estimated workload: 4–6 h per week); video clips in the form of interviews with clinicians, industry representatives, policymakers, patients, and informal carers from different parts of the world; hand-drawn illustrations of patient scenarios; 1 teaching assistant and teacher responsible for answered questions in discussion forum; course runs three times: first and third as a session-based course, second as self-paced course NR	The course was given in three different versions: (1–2) without the possibility of paid-for certificates; (3) with the possibility to paid-for certificates. Quizzes
Pickering & Swinnerton 2017 UK NR	To assesses the use of an anatomy MOOC as part of a blended learning medical anatomy curriculum; to provide valuable information by using an anatomy MOOC to investigate the demographic profile, patterns of engagement and self-perceived benefits to healthcare professionals Explorative	FutureLearn University	Exploring Anatomy: The Human Abdomen MOOC Multidisciplinary NR	HCPs and students 2711 3 weeks 97%	Video lectures; research and discussion forum NR	Automated self-assessment 21 survey questions
Swinnerton et al. 2017 UK 1) NR 2) UK	To investigate the impact of an anatomy MOOC on, in a first phase, the general public's uptake and engagement with a science-based course and, in the campus phase, how campus-based medical students currently studying anatomy at the host institution's medical school would use the MOOC as part of their year 1 curriculum Explorative	FutureLearn University	Exploring Anatomy: The Human Abdomen MOOC Multidisciplinary NR	(1) Pre-university and undergraduate healthcare students; HCPs: (2) First-year medical students at the university (1) 8597 (2) 9786 3 weeks (1) 94.1% (2) 96.3%	Video-based lectures; discussion forum with educators and peers; introductory, core, and advanced materials; short introductory scene-setting videos, bespoke mini-lectures using hand-drawn and animated images, detailed screencasts; interview-style videos with experts; clinical case studies and links to current medical research; transcripts, subtitles, audio recordings; cadaver demonstration videos Adult learning	Formative assessments with instant feedback positioned throughout each week; online formative multiple-choice questions
Castle et al. 2016 USA Participants from 47 countries	To evaluate the impact of MOOC on participants Case study	Coursera Women's Health Research Institute and university	Introduction to Reproduction (biological foundations of reproductive health) Multidisciplinary NR	Global audience of students 289 NR NR	Lecture videos; animations; 3-dimensional anatomical illustrations; virtual teaching assistants with personalities voice emblemised by cartoon visuals; online reading materials supplement the short video presentations and links to reproductive health tools (e.g. app for menstrual cycle tracking); real-life case studies discussion boards; online reproductive health lexicon (Repropedia) NR	Content-based quiz (10 multiple-choice questions) after each module. Score of 70% or higher to adequately complete the course and certify the completion; reflective questionnaires combined with knowledge-based quizzes
Frank et al. 2016 USA Participants from 145 countries	To compare the scores on the Society of Academic Emergency Medicine exam of students trained by a “democratically open, outstanding hybrid of internet-aided, computer-aided, and human-aided education” educational tool versus traditionally trained students Retrospective study	NextGenU University, International Federation of Emergency Medicine, and the Society of Academic Emergency Medicine	Emergency Medicine for Senior Medical Students Monodisciplinary NR	Senior medical students at the university 4000 A semester (Q3) 0%	Readings on common emergency medicine diseases and presentations; it was instructor-led NR	Certification obtained through assessment with objective multiple-choice question testing; through interactions with, and assessed by, local and/or remotely available peers and mentors; Society of Academic Emergency Medicine exam; quizzes, multiple-choice questions, peer and mentor assessment
Milligan & Littlejohn 2016 USA 168 countries	To address the research question ‘How do professionals self-regulate their learning in a MOOC?’ The study examined the ‘Fundamentals of Clinical Trials’ MOOC and presented narrative descriptions of learning drawn from interviews with 35 course participants. Mixed method	edX University	Fundamentals of Clinical Trials Multidisciplinary Monolingual, English	HCPs and student for HCPs 22,000 12 weeks NR	Video lectures; exercises; quizzes; weekly readings; textbook; articles NR	Survey instrument designed to provide a measure of their self-regulation Multiple-choice assessment
Robinson 2016 USA NR	To compare learner evaluations and ratings of a course that was previously delivered by traditional methods that is now delivered as a MOOC Retrospective study	Udemy University	Medicine as a Business elective Multidisciplinary NR	Open to anyone, included fourth-year medical students 286 5 weeks 44%	Video presentations, reading materials, discussion forum, that could be accessed on a smartphone, tablet, or traditional computer via the internet; option of meeting with the faculty to discuss course content; a system to send messages to the course instructor NR	Completion of all course sections is required for successful completion; Multiple-choice questions (5-point Likert scales) for courses quality evaluation
Aboshady et al. 2015 Egypt Egypt	To assess the prevalence of awareness and use of MOOCs among medical undergraduates in Egypt as well as to identify the limitations from enrolling in and completing these courses along with the satisfaction level with using them Cross-sectional	Coursera, edX, Udacity, FutureLearn Universities	Medical MOOCs Monodisciplinary NR	Medical students in all 6 undergraduate years in 10 Egyptian institutions 136 NR 82.4%	Video lectures (72%) NR	Exams and assignment (64%)
Fricton et al. 2015 USA 179 countries	To describe the course concepts in preventing chronic pain, the analytic data from the course, the course participants' pain assessments, and post- course evaluation forms Explorative	Coursera University	Preventing Chronic Pain: A Human Systems Approach Multidisciplinary Monolingual, English	HCPs, students, patients, and consumers 23,650 20 modules 91%	Video with interactive components; discussion forum; exercises; music video Transformative care and human systems theory	Voluntary. Assessment quiz that can be completed at any time during the course; a homework essay at the end
Hossain et al. 2015 Australia Participants from 108 countries	To compare 2 ways of providing online education about spinal cord injuries to physiotherapy undergraduate students in Bangladesh to understand if MOOCs improve knowledge or confidence and lead to greater satisfaction RCT	Physiotherapy International Spinal Cord Society (University of Sydney)	Physiotherapy management of spinal cord injuries Monodisciplinary Monolingual, English	Physiotherapy students 3523 5 weeks (3 h/week) 0%	14 lessons; short didactic overview and between 2 and 7 activity modules each week; online discussion through Facebook where 2–5 discussion points were posted each week by coordinators; 3 h per week to study; guidance through the content, including course curriculum, objectives, weekly study plan; 1367 screens; videos; interactive lesson contains 150 videos of people with spinal cord injuries and interviews with both physiotherapists and patients from diverse countries Adult learning theory	Online assessment. Multiple-choice self-assessments; pre-MOOC and post-MOOC knowledge assessment (20 multiple-choice questions)
Kononowicz et al. 2015 Sweden Participants mostly from USA, India, UK	To analyse technical challenges and solutions for offering virtual patients in health-related MOOCs and describe patterns of virtual patient use in one such course Case study	edX University	KIBEHMEDx: Behavioural Medicine—A Key to Better Health (science of changing behaviour to improve health and quality of life) Multidisciplinary NR	Students (not specified) 19,236 5 weeks 96.1%	OpenLabyrinth for Virtual Physiological Human with branching paths navigation model; scenario with treatment of stress-related symptoms; treatment of sleep problems; virtual patient scenarios consisted of 80 and 61 screen card types or decision nodes; text description; discussion forums; videos involved a professional actor, 2 clinicians, a film team (16 s–6 min in length); all videos were posted on YouTube and embedded in the virtual patient scenarios using an internal frame; some videos from week 2 were repeated in week 3, forming review nodes; virtual patient activity was planned for 1 h NR	Certificate for completed course; free text questions; multiple-choice questions
Stokes et al. 2015 UK Participants from 79 countries; most from UK (77%)	To describe how a relatively new style of online learning, a MOOC, may be used to raise aspirations and widen participation in dental professions Case study	FutureLearn University	Discover Dentistry Monodisciplinary Monolingual, English	Potential students of dental professions, dental professionals and members of the public 4224 6 weeks 74.7%	Discussion forum; 2–3 h of engagement per week; video; supporting text; transcript NR	Not offering any academic credit; short multiple-choice question assessment each week, with a longer ‘final assessment’ in week 6 covering content from throughout the course; pre- and post-course survey
Bowen et al. 2014 USA University campuses in Northeast and Mid-Atlantic	To measure the effect on learning outcomes of a prototypical interactive learning online statistics course (hybrid format with machine-guided instruction accompanied by 1 h of face-to-face instruction each week vs traditional format with 3 h of face-to-face instruction each week) on public university campuses RCT	Carnegie Mellon University's platform University	Introduction to statistics Monodisciplinary NR	University students of 6 public university campuses 313 1 semester NR	Textual explanations of concepts; inventory of worked examples and practice problems; feedback to student answers; supplemented by a 1 h per week face-to-face session NR	Not mandatory; self-assessment; comprehensive assessment of outcomes in statistics with 40-item multiple-choice assessment at beginning and end of semester
Goldschmidt & Greene-Ryan 2014 USA USA	To provide a course overview, pilot data, and suggestions for further research Case study	NR University	Gateway to Online Learning Monodisciplinary NR	RN students 49 5 modules NR	modules: blackboard, discussion board, technological support, podcast, link to online journals, online writing centre NR	Data from course discussion board, anonymous Drexel course evaluation, student evaluations, student satisfaction survey; 3 credits if students completed the course
Harvey et al. 2014 NR 108 countries	To audit participation in and satisfaction with a MOOC for teaching physiotherapy students and physiotherapists about spinal cord injuries Descriptive audit	Physiopedia and Facebook International society	Management of spinal cord injuries Monodisciplinary Monolingual, English	Physiotherapy students; physiotherapists 3523 5 weeks 59%	3 tasks each week: specific lessons; selected readings from book that are free for course; online discussion through a closed Facebook group NR	Voluntary, obligatory for credits. Pre- and post-MOOC knowledge assessments; multiple-choice questions at the end of each session; multiple self-assessments; an online course satisfaction evaluation; Facebook activity and internet-based usage statistics
Liyanagunawarden & Williams 2014 North America, Canada, Australia, Spain, UK, Ireland, Denmark, West Indies, Switzerland, China NR	To provide a review of MOOCs related to health and medicine offered by various MOOC platforms in 2013, by analysing and comparing the various offerings, their target audience, typical length of course, and credentials offered Explorative (98 MOOCs)	Coursera, Open2Study, CourseSites, Canvas, Miriada, FutureLearn, NovoEd, P2PUniversity, Rwaq, VentureLab University	Health and medicine (food, nutrition, nursing in healthcare, health for all through primary care, contraception, social context of mental health and illness, genes and the human condition, health informatics, work in the pharmaceutical industry) Multidisciplinary Multilingual	Global audience included students and HCPs NR 3–20 weeks, average 6.7 weeks NR	NR NR	Verified certificate or credits for continuing professional development
Stathakarou et al. 2014 Sweden NR	To investigate the potential offered by virtual patients for the purpose of clinical reasoning skills training Case study	edX integrated with OpenLabyrinth open source VP system University	Medical education NR NR	NR NR NR NR	Integration of the edX MOOC platform with Open Labyrinth virtual patients system: (1) provide virtual patients containing ill-defined clinical problems (e.g. with sparse or conflicting clinical data); (2) collective repurposing of cases and division of discussion into subgroups focusing on local variances in healthcare; (3) building short cases focusing just on the most important step in the decision making process Activity theory	Collection of responses in the script concordance testing approach
Subhi et al. 2014 North America, Southern Europa, Australia NR	To review and to evaluate all available courses offered by the largest MOOC providers and the relevance of those courses to the seven roles identified in the CanMEDS framework Explorative (594 MOOCs)	Coursera, edX University	Major depression, clinical terminology, genomic and precision medicine, safety in healthcare, rationing and allocating scarce medical resources, fundamentals of clinical trials Mono- and multidisciplinary (39% possibly relevant for more than one role) NR	Medical expert and scholar, communicator, collaborator, manager, health advocate, professionals NR 6–9 weeks NR	Homework, exercises, repetitions, discussions with peers NR	NR

Abbreviations: HCPs, healthcare professionals; ICT, information and communication technology; KIBEHMEDx, Behavioural Medicine: A Key to Better Health (edX); LGBTI+, intersex; LRN, learning resource network; MOOC, massive open online course; MSc, Master of Science; NR, not reported; PPT, PowerPoint; P2P, Peer 2 Peer; RCT, randomized controlled trial; UK, United Kingdom; USA, United States of America; +, plus; >, greater than.

§ as “a set of criteria specifying the characteristics of an outcome and the levels of achievement in each characteristic” (Odden, 2017).

From the creation point of view, a total of 24 (66.7%) studies reported that MOOCs have been developed by universities, while eight (22.2%) reported joint projects between universities and other institutions (e.g., the Minister of Health; e.g. Hendriks et al., 2019) and to lesser extent with scientific societies (e.g., Frank et al., 2016). The US (nine; e.g., Evans et al., 2017), Sweden (six; e.g., Berman et al., 2017) and UK (four; e.g., Beer, 2019) emerged as the countries that most often developed MOOCs.

On the other hand, the MOOCs documented have been offered mainly at multi-national levels (14; 39% e.g., Stathakarou et al., 2018b): however, no data regarding where MOOCs have been delivered (14; 39% e.g., Darcy and Lock, 2017) as well as the language used (24; 66.6%, e.g., Alturkistani et al., 2019) have been reported in several studies. When being documented (12; 33.3%), ten studies used English (e.g., Harvey et al., 2014; Hossain et al., 2015) while platforms most commonly used were Coursera, edX, and Futurelearn (e.g., Kononowicz et al., 2015).

The students target has been varied, ranging from a general audience of students to a global audience with healthcare professionals (HCPs) to laypeople (six studies; e.g., Berman et al., 2017) or to single individuals (five; e.g., Bakayev et al., 2018) such as: nursing students (four; e.g., Goldschmidt and Greene-Ryan, 2014); medical students alone (four; e.g., Frank et al., 2016) or among a general healthcare audience (three; e.g., Magaña-Valladares et al., 2018); students included in a general healthcare audience (three; e.g., Milligan and Littlejohn, 2016); physiotherapist students and physiotherapists (two; e.g., Harvey et al., 2017); dentistry students in a global healthcare audience (two; e.g., Lan et al., 2019); physiotherapist students (Hossain et al., 2015); undergraduate students in data science in healthcare (Alturkistani et al., 2019); and medical and nursing students with HCPs (Roller-Wirnsberger et al., 2019). However, the population was not clarified in four studies (e.g., Stathakarou et al., 2014).

The number of participants ranged from 45 when the recruitment process of participants was selective (Darcy and Lock, 2017) to >23,000 (Fricton et al., 2015); however, high percentages of dropout rates (>90%) have been reported in several MOOCs (e.g., Berman et al., 2017; Pickering and Swinnerton, 2017). Topics varied broadly, ranging from clinical subjects like depression, genomic medicine, chronic pain, and emergency medicine (e.g., Frank et al., 2016) to research and statistics (e.g., Milligan and Littlejohn, 2016).

The reported duration of MOOCs spanned from two weeks (e.g., Shang and Liu, 2018) to six months (e.g., Canavese et al., 2020) with a mean length of five or six weeks (e.g., Berman et al., 2017). Moreover, MOOCs have been documented to be available for a defined period, such as a semester (Bowen et al., 2014) or six months (Darcy and Lock, 2017).

As reported in Table 1, the pedagogical foundations of MOOCs have been described only in 10 studies (27.8%) referring to “adult learning theory” (four; e.g., Evans et al., 2017), while other theories have been used as a foundation in other studies as for example “constructivism” (three e.g., Hendriks et al., 2020), “connectivism” (Chan et al., 2019), “transformative learning” (Beer, 2019) or “cognitivism” (Roller-Wirnsberger et al., 2019). In one study, the combination of more theories was reported (Magaña-Valladares et al., 2018).

Video lessons were used most often (25 studies; 69.4%; e.g., Aboshady et al., 2015; Jia et al., 2019), followed by podcasts (six; e.g., Berman et al., 2017), PowerPoint presentations (five; e.g., Frank et al., 2016) and more complex tools such as three-dimensional anatomical illustrations or virtual patients (e.g., Castle et al., 2016). As supportive strategies, discussion forums and social media groups have been reported in almost all studies to increase interactivity and collaboration both between peers and between students and teachers, technical staff or experts (e.g., Fricton et al., 2015). Didactic resources have been described mainly as scientific readings or articles, textbooks, and website links (e.g., Milligan and Littlejohn, 2016).

3.2. MOOCs in healthcare education: assessment methods and tools

As reported in Table 1, data regarding the assessment of learning outcomes has been reported in the majority of studies (28; 77.8%; e.g., Goldschmidt and Greene-Ryan, 2014), but only three explicated the voluntary non-mandatory nature of the final assessment (e.g. Bowen et al., 2014) or its formative intent (e.g. Swinnerton et al., 2017). Nearly half of the studies also described the evaluation and/or the assessment timing (16; 44.4%; e.g., Hendriks et al., 2019), as during and post- (11; e.g., Canavese et al., 2020) or pre- and post-MOOCs attendance (five; e.g., Harvey et al., 2017).

Among the written assessments, the multiple-choice questionnaire was the main tool used (16; 44.4%, e.g., Alturkistani et al., 2019), along with validated instruments such as the “Comprehensive Assessment of Outcomes in Statistics” (Bowen et al., 2014) and less often with open-ended questions (three; e.g., Kononowicz et al., 2015). Peer evaluations and feedback (six; e.g., Chan et al., 2019) have been also documented as assessment methods, even if only with formative intent (three; e.g., Lan et al., 2019). However, when reported, methods used to assess have been mixed, by using—for example—free text and multiple-choice questions, and multiple self-evaluations (Harvey et al., 2014), including practical assessments (Shang and Liu, 2018). Moreover, satisfaction and reasons for dropping out have been reported as being assessed in five studies (e.g., Harvey et al., 2017) and one (Chan et al., 2019) study, respectively.

The delivery of a final certificate was reported in 11 studies (30.5%; e.g., Frank et al., 2016), of which five described how to achieve it: four studies defined a cut-off passing grade point (e.g., Evans et al., 2017) and one the need to pay fees (Koch and Hägglund, 2017). To gain transferable credits for university was possible in two MOOCs for nursing students (Goldschmidt and Greene-Ryan, 2014; Shang and Liu, 2018), while obtainable, non-specified credits for completion of the course were available in four studies (e.g., Evans et al., 2017) and credits for continuing education in two reviews of MOOCs (Hendriks et al., 2019; Liyanagunawardena and Williams, 2014).

3.3. Factors affecting MOOCs' effectiveness

Factors affecting the effectiveness of the MOOCs have been identified in the MOOC itself and in its participants. For the first element (Table 2 ), the following have been reported to increase effectiveness: (a) promoting discussion and collaboration through forums, collective tasks, and social media groups—e.g., Facebook—(e.g., Aboshady et al., 2015); (b) using multiple resources and delivery systems—e.g., lecture, video, homework assignment, practical training via computer or mobile phone—(e.g., Goldschmidt and Greene-Ryan, 2014); (c) facilitating interactivity with face-to-face sessions or tools as virtual patients (e.g., Bowen et al., 2014); and (d) ensuring constructive communication, feedback, and support by teachers and staff (e.g., Chan et al., 2019). Difficulty accessing technology and internet connection issues have also been identified in some studies as barriers to completing MOOCs, especially in developing countries, while asynchronous resources, offline and lower-resolution versions have been adopted in order to overcome these barriers (e.g., Fricton et al., 2015). Furthermore, regular assessments have been reported to monitor participants' progress, thus preventing a flawed learning process through the detection of missed activities (Bakayev et al., 2018). MOOCs delivered in a single language or in a limited geographical area have been reported to prevent access and understanding (e.g., Evans et al., 2017).

Table 2.

Factors affecting MOOCs' effectiveness.

Factors	Aboshady et al., 2015	Alturkistani et al., 2019	Bakayev et al., 2018	Beer, 2019	Berman et al., 2017	Bowen et al., 2014	Canavese et al., 2020	Castle et al., 2016	Chan et al., 2019	Darcy and Lock, 2017	Evans et al., 2017	Frank et al., 2016	Fricton et al., 2015	Goldschmidt and Greene-Ryan, 2014	Harvey et al., 2014	Harvey et al., 2017	Hendriks et al., 2019	Hendriks et al., 2020	Hossain et al., 2015	Jia et al., 2019	Koch and Hägglund, 2017	Kononowicz et al., 2015	Lan et al., 2019	Liyanagunawardena and Williams, 2014	Magaña-Valladares et al., 2018	Milligan and Littlejohn, 2016	Pickering and Swinnerton, 2017	Robinson, 2016	Roller-Wirnsberger et al., 2019	Shang and Liu, 2018	Stathakarou et al., 2014	Stathakarou et al., 2018a, Stathakarou et al., 2018b	Stathakarou et al., 2018a	Stokes et al., 2015	Subhi et al., 2014	Swinnerton et al., 2017
Regarding MOOCs
Discussion forum/social groups	↑			↑					↑	↑					↓	↑	↑	↑	↑↓	↑	↑				↑	↑	↑		↑	↑		↑			↑	↓
Multiple methods/resources	↑					↑	↑		↑	↑	↑		↑	↑	↑							↑				↑		↑	↑	↑			↑	↑		↑
Interactivity (e.g., face-to-face)	↑				↑	↑			↑						↑	↑					↑					↑			↑	↑			↑		↑
Teacher/guide/staff communication	↑				↑				↑					↑		↑		↑		↑	↑					↑			↑	↑			↑
Peer collaboration e/o assessment	↑								↓									↑↓			↑														↑
Promoting self-regulated learning																		↑					↑						↑	↑	↑
Official certificate/credits/assessment	↑					↑						↑			↑				↑				↑	↑		↑			↑
Asynchronous delivery of resources	↑									↑		↑						↓		↓								↑							↑
Offline resources	↑				↑															↑		↑
Access to technology and internet	↑										↑								↑			↑		↑						↑				↑
Regular assessment/monitoring			↑					↑																	↑		↑						↑			↑
Not free access												↑↓	↓	↓												↓			↓					↓	↑
Representing a small geographic area								↓													↓													↓	↓
Organised, logical, simple structure									↑						↑								↑													↑
Passive learning approach																	↑↓									↓			↓		↓
Clinical case close to clinical work																					↑				↑		↑		↑
Expert involvement/feedback																			↑					↑					↑					↑
Short video/not too time-consuming								↑														↑								↑						↑
Regarding participants
Learners' motivation and/or engagement							↑		↑							↑				↑		↑	↑			↑	↑							↑
Background knowledge and skills	↑								↑					↑						↑				↑	↑	↑
Time management skills	↑								↑					↑				↑					↑			↑
Adequate time to complete sessions					↑					↑					↑																				↑	↑
Personalised learning and goals					↑								↑					↑							↑	↑↓										↑
Multiple languages											↑		↑						↑					↑		↑

Legend: MOOCs = Massive online open courses; ↑ = presence of factor increases MOOCs' effectiveness; ↓ = presence of factor decreases MOOCs' effectiveness.

From the perspective of participants, learners' motivation and engagement have been identified as affecting completion of MOOCs (e.g., Jia et al., 2019). In addition, a high knowledge level, skills and e-health literacy (e.g., Aboshady et al., 2015), and time-management skills (e.g., Lan et al., 2019) have been reported as increasing MOOC completion. However, allocating adequate time to complete sessions, personalising the schedules, and preventing excessively time-consuming activities (e.g., Castle et al., 2016) have been reported as preventing drops outs.

4. Discussion

4.1. MOOCs in healthcare education: the state of the research

To the best of our knowledge, this is the first rapid review of literature on MOOCs performed to date. The recent so-called “healthcare sciences education disruption” (Dewart et al., 2020) generated by the COVID-19 outbreak, calling for immediate curricula redesign towards a forced and wide introduction of distance learning also at the academic levels, has generated a new interest in online courses. Therefore, summarising the research in this area might support educators and policymakers who have the responsibility to undertake decisions in these challenging times.

Studies on MOOCs devoted to undergraduate and postgraduate students in nursing and healthcare fields emerged in 2014, although articles citing the term “massive online open courses” in healthcare education had been written previously (Skiba, 2012). This educational strategy seems to have grown more slowly, (Kearney et al., 2016) with around six studies/year, mainly with descriptive or case study designs instead of longitudinal studies or trials capable of also measuring MOOCs' effectiveness.

Universities alone or together with other relevant institutions (e.g., Minister of Health) have been reported as the main MOOC developers; only in six studies has the MOOC been delivered at the national level (Brazil, China, Egypt, Mexico, Russia, and the US), suggesting that this educational strategy is aimed at embracing an international perspective. Moreover, 22 studies have documented >1000 participants, and a mix of subgroups (students of different disciplines, HCPs, or citizens) have been educated: specifically, only four studies have been focused on MOOCs devoted to nursing students. The benefits of learning with other students in interprofessional courses has already been established (Lennen and Miller, 2016). However, the picture that has emerged seems to suggest that MOOCs reflects substantially the third mission of universities, which is to develop knowledge and to create alliances with society and its representatives aiming to gain this knowledge by combining “explicit” (formal, taught at the university) and “implicit” (acquired in the practice, possessed by HCPs) dimensions of knowledge (Laredo, 2007). Moreover, MOOCs seem to include mainly undergraduate students (the so-called “mass tertiary education”) and only in a few studies (Beer, 2019; Darcy and Lock, 2017) master's or doctoral students were involved (the so-called “professional specialised higher education and research”) (Darcy and Lock, 2017). However, despite the importance of second and third levels of education internationally, MOOCs seem to be used only rarely.

No trends in the topics taught or on the duration of the MOOCs, suggesting the wide flexibility of the tool, have emerged from the studies available. Moreover, the context in which these courses are offered (e.g., citizens, HCPs and students) as well as the high number of dropouts seem to reflect that they are mainly elective or voluntary university courses. For what concerns their pedagogic foundations, when data has been reported, theories consistent with academic education (“adult learning theory”, e.g., Evans et al., 2017) have been used. In this context, the complexity of the didactical method used (from videos to podcasts) supported by virtual communities suggests that MOCC design and implementation require competences that should be developed by universities.

Likely due to its novelty, studies available are not homogenous in the data reported and lack several data: therefore, with the additional intent of performing in the future systematic reviews and meta-analysis, a homogenous set of data should be reported in all studies investigating MOOCs.

4.2. MOOCs in healthcare education: assessment methods and tools

Data regarding the evaluation methods have been variable across studies, suggesting that in this field standardisation of the information is required. However, multiple assessment methods have emerged as pre- and post-evaluations, written or oral, or implying composite tools, with formative and summative intents. In general, findings suggest that the evaluation is designed according to the needs of the target, e.g., to acquire university credits (e.g., Shang and Liu, 2018) or credits for continuing education (Liyanagunawardena and Williams, 2014), confirming their flexibility for diverse audiences. Methods that emerged and the tools suggest that evaluation is conducted mainly to assess the knowledge acquired (e.g., Canavese et al., 2020) and only in a few cases the competences. However, university qualification frameworks established across the World require the specification of the learning outcomes based on acquired competencies, rather than on knowledge, that should, in turn, be steadily updated to current healthcare needs (Di Giulio et al., 2020). This might be one of the reasons why among healthcare sciences MOOCs have played a limited role (Kearney et al., 2016).

4.3. Factors affecting MOOCs' effectiveness

A list of factors has emerged as promoting or hindering the effectiveness from the perspective of the MOOC itself and that of its participants, suggesting that MOOCs should be well designed and delivered by considering also the needs of the attendees or a degree of flexibility thus allowing each participant to self-design their educational pathway. In a few studies, factors documented are in conflict with each other (e.g., Hendriks et al., 2020) or to those reported in other studies (e.g., Frank et al., 2016 vs Fricton et al., 2015) suggesting that—although mainly with descriptive studies and case studies—the body of evidence available tends to be cumulative. In general, a MOOC based on multi-didactical strategies, methods, and resources, both delivered online and offline, promoting a constructive approach to the learning process, supported by peers and teachers, with regular feedback, and tailored to the background and the needs of attendees seems to increase the effectiveness. Further studies are needed to assess these factors in terms of effectiveness both alone and as a whole according to their reciprocal influence.

4.4. Limitations

The study has several limitations firstly due to the approach used, based on a Rapid Review where studies may have been missed (Tricco et al., 2017). In particular, we have not performed a search in grey literature and we have included only English and Italian primary studies. The selected languages might have introduced a selection bias in the studies with a consequence that studied performed in high-income English language countries (e.g., US, Sweden and the UK) merged, thus missing studies from low-income countries or written in different languages.

Moreover, the data extraction has been performed around three research questions, where factors affecting a MOOC's effectiveness have been not differentiated with regards to their potential effects on learning outcomes, dropouts, or the degree of satisfaction reported by participants. Furthermore, factors that emerged have been summarised according to their main influence (as increasing or decreasing MOOCs' effectiveness) without reporting quantitative data—according to the main intents of this Rapid Review. Given that the purpose of this review was to survey the evidence rather than to evaluate specific effects, a critical appraisal of the studies was not performed (Tricco et al., 2017).

5. Conclusion

MOOCs have recently started to be studied in undergraduate and postgraduate healthcare sciences education mainly with descriptive or case studies: in addition to the lack of standardisation in the data reported across studies, the descriptive nature of those available suggests the need to increase research efforts in this research field by also establishing guidelines in order to ensure homogeneity in study reporting, allowing systematic reviews and meta-analysis.

To date, target audiences have been wide and mixed and have included students along with citizens and HCPs. Therefore, at the policymaker level, this seems to suggest that MOOCs might have two intents: as a tool to educate healthcare students, mainly in interdisciplinary elective general courses, and as the third mission where universities transfer the developed knowledge to a massive, diverse audience of both professionals and students.

According to the findings, MOOCs are capable of targeting different audiences with different needs and backgrounds, as well as with different learning expectations by also ensuring transferable credits both for university and continuing education. However, more robust systems of competence evaluation should be introduced and analytically reported in studies aimed at assessing MOOCs' contribution to the acquisition of skills expected by graduate students.

Multi-didactical strategies and methods, tailored to the needs of attendees, and delivered online and offline, promoting a constructive approach to the learning processes, supported by peers and teachers, increase MOOCs' effectiveness. Therefore, while designing a MOOC, these factors should be considered by educators; however, considering their complexity, MOOCs require a multidisciplinary approach and, for those involved in their implementation, specific education to support the transition from traditional teaching methods to massive online courses, including also non-conventional students such as citizens and HCPs, is needed.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

None.

Appendix A. Supplementary data

Supplementary tables