Massive Open Online Course (MOOC) learning builds capacity and improves competence for patient safety among global learners: a Prospective Cohort Study

Abstract

Background:

Education about patient safety has historically been limited in health professionals curricula and largely inaccessible to the general public. We developed a massive open online course (MOOC), The Science of Safety in Healthcare, to present the foundational patient safety principles to a broad global audience of health professionals, learners, and patients interested in patient safety.

Objectives:

To describe the Science of Safety in Healthcare MOOC, its effects on patient safety competence, and the satisfaction of course participants.

Methods:

The five-week video-based course was delivered in 2013 and 2014, and was organized in five modules: 1) overview of science of safety and safety culture, 2) enabling and contextual factors that influence patient safety and quality, 3) methods to improve safety and quality, 4) translating evidence intro practice and leading change, and 5) summary and opportunities for capacity building. Each module had three or four segments. Participants were introduced to key concepts, and tools and skills to promote patient safety. Participants completed the Health Professional Education in Patient Safety Survey (H-PEPSS), which measures patient safety competence, and a course satisfaction survey. Pre- and post- course H-PEPSS scores were compared using paired t-tests. Course satisfaction surveys were administered at the completion of the course and six months later.

Results:

A total of 20,957, and 9,679 participants enrolled in the course in 2013 and 2014, respectively. About half of participants were 25–44 years old (57%), and female (54%). Participants joined from over 100 countries. The majority were health professionals (61%) or health professionals in training (7%). Mean H-PEPSS scores improved after course completion, with significant increases on all survey domains in both years (p<0.01). Mean score differences were: Teamwork 0.68 (95% CI: 0.64, 0.71), Communication 0.70 (95% CI: 0.66, 0.73), Managing Risk 0.79 (95% CI: 0.76, 0.82), Human Environment 0.64 (95% CI: 0.61, 0.68), Recognizing and Responding 0.64 (95% CI: 0.61, 0.68), and Culture 0.72 (95% CI: 0.68, 0.75). About 8% of participants in each cohort earned a certificate of completion. At 6-months post-course, the majority of the participants agreed or agreed strongly that the course content was useful (93%) and that the course positively influenced their clinical practice (69%) and communication (84%).

Conclusions:

The MOOC course allowed educators to reach a large, diverse audience. The course was well-received and participants reported a significant increase in patient safety competence. As with most MOOCs, rates of completion were low.

BACKGROUND

Patient safety is a global health priority. Adverse health events due to unsafe care are one of the 10 leading causes of death globally.¹ In 2002, the World Health Assembly passed a resolution urging countries to address patient safety by: developing global norms and standards, promoting evidenced-based policies, promoting mechanisms to recognize excellence in patient safety internationally, and encouraging research and providing assistance to countries in several key areas for patient safety.² In its landmark report, To Err Is Human: Building a Safer Health System, the National Academy of Medicine (NAM, formerly the Institute of Medicine) described the magnitude of the problem and the urgent need for reform.³ The report recommended the inclusion of patient safety content in health professional education.³ In its subsequent report, Health Professions Education: A Bridge to Quality, the NAM further emphasized the need for learners and active health professionals to develop and maintain proficiency in five areas: delivering patient-centered care, working as part of interdisciplinary teams, practicing evidence-based medicine, focusing on quality improvement, and using information technology.⁴

Historically, patient safety content has not been included in health professionals’ education, and it has been difficult to incorporate it into curricula.⁵ Contributing factors include the conservative discipline-based organization of higher education, lack of qualified faculty to teach this content,⁶ the limited available time in the medical and nursing schools curricula,⁷ and the inflexibility of health professionals’ academic program and training schedules.^8,9 The NAM called for a more innovative health professionals’ education model that allows for lifelong learning.⁴ The creation of new learning opportunities is essential for patient safety content to be successfully incorporated into health professional’s training and practice.

For nurses, physicians, and allied health professionals to work collaboratively, they must all achieve specific Core Competencies for Interprofessional Collaborative Practice.¹⁰ Interprofessional education is required by licensing bodies for the health professions.¹⁰ This includes understanding the roles and responsibilities of both professions, engaging in effective communication, collaborating around shared ethics and values, and engaging in teamwork. Interprofessional training programs have been developed to increase health professionals’ and learners’ capacity to lead patient safety and quality improvement efforts. Examples of the programs are a 3-day patient safety intersession course that focused on improving learners’ teamwork and communication skills and system-based thinking^11,12 and an interprofessional simulation on assertive communication and use of structured communication tools.¹³

Patient safety is also relevant for the general public and especially for patients and their families, whose engagement is essential to improving patient safety and healthcare quality.¹⁴ However, patient safety education is typically tailored to health care professionals and difficult for the members of the lay public to access.

MOOCs are online courses that differ from other conventional online education programs because they have no formal requirements for entry, no participation limit, generally yield no resulting credentials, and are free of charge.¹⁵ Any individual with access to internet can enroll in a MOOC. MOOC providers, such as Coursera, edX, and Udacity, provide thousands of free courses from universities around the world, including prestigious, top-ranking institutions. For example, Coursera has served over 66 million learners from 190 countries across more than 4300 different courses offered from dozens of universities including Johns Hopkins.¹⁶ By offering free online access to courses taught by faculty at prestigious universities, MOOCs increase the accessibility and inclusivity of higher education. From an educator’s perspective, MOOCs offer a unique opportunity to reach a large, diverse local-to-global learner population. MOOCs may be especially important for topics such as patient safety which need to be broadly taught yet many organizations lack content experts to teach learners.

MOOC course delivery is asynchronous, which means that learners can participate from anywhere at a time that is convenient to them.¹⁶ MOOCs are structured to be more interactive and dynamic than passive recordings of lectures; MOOC providers promote that they have a better foundation in pedagogy, theory, learner success, and learning outcome planning than traditional online courses.¹⁷ Most MOOCs consist of short, focused video lectures, interactive quizzes, and a discussion board to communicate with classmates.^16,17 Since MOOCs have no prerequisites and no limits on participation, there is a variety in the level of participation, experience, and education among the learners.¹⁸ Uncapped and largely tuition-free, MOOC courses largely rely on automated processes, including for assessment, and often have low completion rates.¹⁹

MOOCs create the potential for affordable, lifelong learning and professional development, which makes MOOCs an attractive vehicle for patient safety for both practicing health professionals and learners. In 2013, an interprofessional team of patient safety experts at Johns Hopkins developed a Massive Open Online Course (MOOC) entitled The Science of Safety in Healthcare, to introduce patient safety concepts to a broad and international audience. To our knowledge, The Science of Safety in Healthcare was the first MOOC developed to address patient safety, though several more have been introduced in the last years.²⁰ The purpose of this paper is to describe this course, which was delivered in June – July 2013 and June – July 2014, learner participation rates, and impact on participants’ self-assessed competence in patient safety.

METHODS

Study Design

This study was a prospective, pre-post, study of the Science of Safety in Healthcare MOOC effects on patient safety competence. Course completion rates, participant satisfaction, and impact on future practice was also examined.

Course Description

The primary audience of this course were health professionals and health professions learners. However, faculty also aimed to keep content relevant to other audiences. The 5-week course was designed to introduce the foundational principles of the science and culture of safety in healthcare. The course was organized into five modules (Table 1). Two-to-five hours of work was expected from the participants each week. The Coursera platform permitted self-paced learning. Participants were introduced to key concepts, tools and skills required to promote patient safety and reduce preventable harm. Case studies were developed to allow participants opportunities to apply patient safety concepts and improvement tools to realistic, engaging scenarios. Structured exercises challenged participants to consider patient safety concepts, principles, and best practices. For example, one assignment asked learners to identify a patient safety defect they would like to address and explain a strategy addressed in the course that could be implemented to improve patient safety. Weekly discussion board forums fostered small group communication on specific topics generated by participants and interaction with course faculty. There were two cohorts of course participants: one cohort who took the course when it first ran from June – July 2013, and a second cohort who took it when it ran again in June – July 2014. The first cohort of course participants used a common social media venue, Facebook, to create a page dedicated to this course and networked during and after course completion.

Table 1.

The Science of Safety in Healthcare MOOC course content

Module	Description of Module Content
1	An overview of the science of safety and an introduction to a culture of safety in healthcare.
2	Enabling and contextual factors, including communication, teamwork, and healthcare human factors, that influence patient safety and quality were explored in this module. This module also included patient-centered care and strategies for patients and families to contribute to patient safety.
3	Methods to improve safety and quality: Given the system complexity and various sources of healthcare safety and quality defects, multiple methodologies including sound measurement approaches are required to improve safety and quality. In this module, several examples of available methodologies to improve safety as well as measurement strategies were examined.
4	Translating evidence into practice and leading change: In this module, learners explored the Translating Research Into Practice (TRiP )Model* for translating evidence into practice, reviewed an integrated approach to improving the reliability of care, and distinguished the technical and adaptive challenges of safety and quality improvement.
5	Summary and opportunities for capacity building: In this final module, the course was summarized and key concepts reinforced. In addition, further opportunities to build capacity in patient safety and quality improvement were presented.

^*: Pronovost PJ, Berenholtz SM, Needham DM. Translating evidence into practice: a model for large scale knowledge translation. BMJ. 2008;337:a1714. Published 2008 Oct 6. doi:10.1136/bmj.a1714a1714

Study Measures

Patient Safety competence

Participants were asked to complete the Health Professional Education in Patient Safety Survey (H-PEPSS) in the immediate post-course period. ²¹ H-PEPSS measures self-reported patient safety competence in six domains: culture of safety, working in teams with other healthcare providers, effective communication, managing risk, optimizing human and environmental factors, and recognizing and responding to adverse events to measure patient safety learning.²¹ For all six domains, the Cronbach’s alpha ranged from 0.81 to 0.85 both in this study and in previous literature.²¹ H-PEPSS consists of 16 items using a 5-point Likert type scale ranging from 1 “strongly disagree” to 5 “strongly agree.” The items begin with ‘I feel confident in what I learned about…’ and include a ‘don’t know’ option. In this study, we asked participants to rate their level of confidence on each item before and after attending the course.

Participant Satisfaction

A post-course survey was administered at 6 months following course completion to measure participant satisfaction and application of the knowledge and skills developed through the course. Participants were asked to rate their level of agreement using a 5-point Likert type scale that included a “not applicable” option to the following five statements:

1. The content I learned in the online safety course was useful.

2. I have applied the content I learned in this course in my clinical practice.

3. I would recommend this online course to other colleagues.

4. The online course has positively influenced my clinical practice.

5. The online course has positively influenced my communication with other healthcare providers.

Additional Post-Test Measures

There were additionally two free-text questions: 1) “Please comment on how this course has influenced your approach to patient safety and /or your practice. Provide examples whenever possible”; and 2) “Since your participation in our course, have you participated (or planning to participate) in any additional education activities to learn about patient safety?”

Other variables

Sociodemographic data, including sex, age, race, education level, and current career position were assessed at baseline. Participants additionally responded to questions on motivation for taking the course and prior knowledge and experience in the course topic.

Statistical Analysis

All analyses were carried out using Stata 15.0. We calculated descriptive statistics for all variables included in the analysis. We presented categorical variables as frequencies and percentages and continuous variables as mean and standard deviation. We assessed differences between the two cohorts by chi-squared tests. We combined the two cohorts to analyze participant satisfaction and perceived change in patient safety competencies. H-PEPPS domain pre- and post-course means were compared using paired t-tests. We conducted a Shapiro-Wilks Test which confirmed that the pre- and post-course scores were normally distributed, thus paired samples t-tests were appropriate for within-subject analyses. We performed sensitivity analyses with the two cohorts separated to determine if findings were consistent with the two cohorts assessed separately. We hypothesized that perceived patient safety competency would increase following the course.

Ethical Approval and Informed Consent:

Johns Hopkins School of Medicine’s Institutional Review Board (IRB) exempted the course evaluations from requiring IRB approval (NA_00086513). The IRB determined that the evaluations of learner learning and engagement poised very minimal risk to the learners, were not likely to adversely impact learners’ opportunity to learn and that the identity of the learners who completed the surveys could not readily be ascertained.

RESULTS

The course participants had a broad range of experience in healthcare, ranging from no prior training to licensed clinicians with experience in leading patient safety efforts (Table 2). A total of 20,957 and 9,679 participants from over 100 countries enrolled in the course in 2013 and 2014, respectively. United States had the highest number of registrants both years, followed by India and Canda. Over half of the participants were 25–44 years old (57%) and female (54%). The majority were health professionals (61%) or health professionals in training (7%), though a sizable proportion (27%) were not health professionals.

Table 2.

Baseline demographic characteristics, motivation for taking the course, and familiarity with the patient safety field among MOOC course participants, by year

	2013 (N=3382)* % (n)	2014 (N=950)* % (n)	χ2	P Value
Country, % of total participants
North America	59 (1887)	47 (449)	104.15	<0.05
Europe	13 (411)	7 (66)
South America	6 (199)	3 (28)
Asia	16 (500)	16 (152)
Other	11 (384)	27 (256)
Female Sex, %	56 (1894)	51 (484)	3.24	0.07
Age in years, %
13–17	1 (34)	1 (11)	19.45	<0.05
18–24	15 (507)	14 (133)
25–34	37 (1251)	35 (332)
35–44	19 (643)	23 (218)
44–54	15 (507)	15 (142)
55–64	9 (304)	9 (86)
65+	4 (135)	3 (28)
Highest degree, %
Doctorate degree	6 (203)	8 (76)	19.45	<0.05
Professional school degree	15 (507)	17 (161)
Master’s degree	26 (879)	28 (266)
Bachelor’s degree	30 (1015)	27 (256)
Associate degree	6 (203)	5 (47)
High school diploma	14 (472)	13 (123)
Some high school or less	3 (100)	2 (19)
Current Position, %
Nurse	34 (1149)	30 (282)	13.64	<0.05
Physician	17 (575)	18 (168)
Pharmacist	3 (101)	3 (26)
Hospital Administrator	4 (135)	4 (39)
Healthcare Learner	7 (237)	10 (94)
Healthcare Technology/Device Professional	6 (203)	5 (51)
Other	29 (913)	30 (290)
Motivations for taking course, % important or very important**
Subject relevant to academic field of study	72 (2359)	71 (635)	0.52	0.47
Teaches skills that will help job/career	82 (2704)	90 (855)	26.0	<0.05
To enhance my CV/resume	49 (1613)	49 (465)	0.08	0.77
Offered by a prestigious university	53 (1740)	57 (541)	4.22	<0.05
Will be fun and enjoyable	62 (2040)	67 (636)	5.67	<0.05
Personal growth and development	87 (2876)	92 (874)	14.43	<0.05
Curiosity about online course	82 (2690)	26 (247)	12.42	<0.05
Prior knowledge / experience in course topic, %
Degree in this field			18.55	<0.05
Significant work experience in this field	10 (322)	7 (65)
Completed some coursework in this field	26 (853)	25 (235)
Some work experience in this field	11 (357)	13 (125)
Like to explore this subject	30 (1009)	31 (290)
Mostly new to the subject	14 (455)	12 (108)

^*: Not all registered course participants completed the baseline questionnaire on demographic characteristics, which is reflective of the low level of participation in MOOCs. Even among the course participants who did complete the baseline questionnaire, some participants did not answer each question.

^**: Learners were asked “Why did you enroll in this course?” and were given a preset list of motivations for enrolling in the course, They ranked each motivation on a scale from 1 “not important” to 5 “very important.”

Of the 30,636 participants enrolled across the two years, 1826 (6%) completed the H-PEPSS survey. The pre-course and post-course domain mean scores are shown in Table 3 Participants’ mean scores on the H-PEPSS scales improved significantly (p<0.05) after completion of the program. The mean change in scores for each domain were: Teamwork 0.68 (95% CI: 0.64, 0.71), Communication 0.70 (95% CI: 0.66, 0.73), Managing Risk 0.79 (95% CI: 0.76, 0.82), Human Environment 0.64 (95% CI: 0.61, 0.68), Recognizing and Responding 0.64 (95% CI: 0.61, 0.68), and Culture 0.72 (95% CI: 0.68, 0.75). When the two cohorts’ mean scores were separately analyzed, there remained a significant increase (p<0.05) across all H-PEPSS scales (Table A1).

Table 3.

Self-reported Patient Safety Competence before and after MOOC course participation, as measured by the domain scores of the Health Professional Education in Patient Safety Survey (H-PEPSS)^*, N=1826

	Pre Mean (SD)	Post Mean (SD)	Change Mean (SD)	P value	95% CI
Teamwork	2.78 (0.73)	3.45 (0.60)	0.68 (0.75)	<0.05	0.64, 0.71
Communication	2.88 (0.73)	3.58 (0.57)	0.70 (0.73)	<0.05	0.66, 0.73
Managing risk	2.70 (.75)	3.49 (.58)	0.79 (0.72)	<0.05	0.76, 0.82
Human environment	2.98 (0.75)	3.52 (0.57)	0.64 (0.72)	<0.05	0.61, 0.68
Recognize and respond	2.89 (0.79)	3.54 (0.58)	0.64 (0.77)	<0.05	0.61, 0.68
Culture	2.95 (0.72)	3.66 (0.52)	0.72 (0.71)	<0.05	0.68, 0.75

^*: The range of scores for each domain is 1 (strongly disagree) to 5 (strongly agree). Across all domains pre- and post-course, the reported scores ranged from 1–5.

Participants reported high levels of satisfaction at 6-months post course completion. (Figure 1). The majority of the participants reported that they agreed or strongly agreed that the content learned in the course was useful (93%), that the course positively influenced their clinical practice (69%) and communication (84%), and that they would recommend the course to others (91%). Examples of open-ended feedback in the 6-month post survey included: “As an attorney specializing in medical staff issues, it is very useful to look at issues from a systems perspective, rather than an individual physician responsibility perspective.” “In working on large-scale implementation, the course broadened my knowledge base of quality improvement change and how to approach clinicians and evaluate clinical situations to help institute change and quality improvement implementation strategies.” “The focus on practical ways to measure and address organizational culture has been very useful for my work.” Multiple participants commented that the interaction was their favorite part of the course. Participants were able to communicate with each other through the discussion forum, and even created a Facebook group to continue discussions after the course ended. The majority of participants indicated that they participated in, or are planning to participate in, additional educational activities to learn about patient safety.

Figure 1.

Participants’ ratings of course usefulness and impact on practice at 6 months post course completion.

Participation in the MOOC is shown in Figure 2. Of the 14972 enrolled participants, 1229 (8%) earned a certificate of completion in 2013. In 2014, 674 (8%) of the 8047 participants earned a certificate of completion. We offered Continuing Education Unit credits in 2014, which 87 participants earned.

Figure 2.

Course Participation Over the Five-Week Period

Discussion

The Science of Safety in Healthcare MOOC implemented in 2013 and 2014 is to our knowledge the first MOOC on patient safety. It reached a large and diverse global audience with over half of participants residing outside the US. Course participants reported increased competence in patient safety domains and high levels of satisfaction with the course. While course completion rates were low, this finding is typical across MOOCs.^19,22 The findings support the utility of MOOCs to deliver basic patient safety content to a broad audience.

The high satisfaction scores with this MOOCs and increases in content-based knowledge were similar to what has been reported in the literature.^20,22,23 Initial studies of the MOOC format suggest that it may be a useful adjunct to conventional modes of instruction for health professions education. In a Midwest medical school, a medical school elective for fourth-year medical learners delivered via a MOOC was well-received by participants.²⁴ Learner evaluations were favorable in ratings of effective teaching and meeting personal learning objectives, which indicates that MOOCs may be well-received to deliver content in medical schools. Another university integrated a MOOC into a course on anatomy found that learners enjoyed having access to the material, though they did not see it as a viable replacement for campus-based teaching.²⁵ There may be value in using a MOOC to supplement a traditional course in healthcare professionals’ curriculum.

The interactive component was a central piece to the delivery of this MOOC. Health care professionals from multiple disciplines and patients were able to share powerful ideas regarding patient safety. In a randomized-controlled trial by Hossain and colleges (2015), while a five-week MOOC with a requirement to join the course Facebook group for discussion had similar rates of satisfaction to an online module with no opportunity to interact, the learners reported that the ability to interact with learners from other countries through the discussion forum was a positive experience.²⁶ Westerlaken and colleagues similarly found that being able to interact socially online in addition to having didactic online modules was perceived as valuable by the learners.²⁷

The MOOC course platform allowed educators to reach a large, diverse audience, though rates of completion were low. While a large number of learners around the world enroll in MOOCs, MOOCs have high attrition rates estimated to be about 90%, which is likely related to the lack of academic credit and the lack of financial commitment on the part of participants since most MOOCs are free.^18,19

An important aspect of this course was its interprofessional nature. The flexible format of this MOOC allowed interprofessional leaders in healthcare to reach learners from various healthprofessions and the course platform enabled social interactiosn and discussions amongst them. Interprofessional collaboration is essential to improving patient care.^28,29 While there is a consensus that interprofessional education is important to healthcare professionals training, it has been difficult to incorporate in the current curricula for health care professions.^10,30 Interprofessional simulations and didactic programs are two common methods for providing interprofessional education, however both require matching busy schedules, adequate space, administrative support, and faculty training.³¹ An examination of 16 medical schools’ experiences implementing interprofessional education found that financial support and scheduling were common barriers.³²

MOOCs have the potential to generate revenue for an institution. They can serve as a lead generator for other learning activities and advertise for that institution. The second time this particular MOOC was offered, participants were given a chance to earn CEU credits. The revenue from offering CEUs partially offset the cost of producing the MOOC. Adding the Certificate of Completion for an additional fee to “The Science of Safety” course also led to revenue for the institution. MOOCs have the capability to stimulate participants’ interest for enrolling in other courses at the institution that are not free.

Limitations

This study has some limitations which must be taken into consideration when interpreting results. There was both a low completion rate and a low response rate, which could have introduced selection bias. Since the course was delivered entirely online, we could not validate individuals’ given information. While the course took place in 2013 and 2014, we believe consistency in the diversity of the audience reached, level of participation, and participants’ reports of increased competency across the two years suggest that the course’s content and delivery format have lasting value that is relevant to the current times when online education is increasingly prevalent.

Conclusion

The Science of Safety in Healthcare, a MOOC, provided a flexible, affordable and effective format for educating health professionals, patients, and others, on the fundamentals of patient safety. Patient safety is an essential competency for healthcare professionals, but including it in all healthcare professionals’ curricula has been difficult. The MOOC format can provide a valuable adjunct to conventional modes of education, and has the potential to expand the education of healthcare professionals on patient safety.