A Multidisciplinary Self-Directed Learning Module Improves Knowledge of a Quality Improvement Instrument: The HEART Pathway

Abstract

We created and tested an educational intervention to support implementation of an institution wide QI project (the HEART Pathway) designed to improve care for patients with acute chest pain. Although online learning modules have been shown effective in imparting knowledge regarding QI projects, it is unknown whether these modules are effective across specialties and healthcare professions. Participants, including nurses, advanced practice clinicians, house staff and attending physicians (N = 486), were enrolled into an online, self-directed learning course exploring the key concepts of the HEART Pathway. The module was completed by 97% of enrollees (469/486) and 90% passed on the first attempt (422/469). Out of 469 learners, 323completed the pretest, learning module and posttest in the correct order. Mean test scores across learners improved significantly from 74% to 89% from the pretest to the posttest. Following the intervention, the HEART Pathway was used for 88% of patients presenting to our institution with acute chest pain. Our data demonstrate that this online, self-directed learning module can improve knowledge of the HEART Pathway across specialties—paving the way for more efficient and informed care for acute chest pain patients.

Introduction

Emergency Departments (EDs) in the United States care for 8–10 million patients annually with a complaint of chest pain (Owens et al., 2010). Traditionally, providers have pursued comprehensive testing strategies, often including admission for further evaluation in order to avoid missing the crucial diagnosis of acute coronary syndrome (ACS). This strategy leads to 50–70% of these patients being admitted, but ultimately less than 10% of these patients are diagnosed with ACS, resulting in increased costs, false-positive testing, and suboptimal resource utilization (Pines et al., 2010; Fleischmann et al., 2002).

The HEART (History, Electrocardiogram, Age, Risk Factors, Troponin) Pathway is a decision aid used by ED care providers, which is designed to focus cardiac testing and care resources on patients most likely to benefit (Mahler et al., 2015). Recent studies have shown that this strategy, which combines an easy-to-use clinical decision rule with two serial troponin measurements, results in reduced cardiac testing, admissions, and hospital length of stay without an increase in missed adverse cardiac events (Mahler et al., 2011; 2013; Six et al., 2013; Backus et al., 2013). Given this evidence, US health systems have begun to implement the HEART Pathway, or similar care pathways, to improve the quality and efficiency of care delivery for patients with acute chest pain.

Within our academic medical center, health system leadership and clinical leaders from emergency medicine (EM), internal medicine (IM), family medicine (FM), and cardiology chose to pursue implementation of the HEART Pathway as a quality improvement (QI) initiative. To implement the HEART Pathway, health professionals across the institution required training to achieve familiarity with the scientific basis of the decision aid and the implications of its consistent use. To meet these training needs, an online, self-directed learning course was created. In this study, we sought to determine whether this modality of training would be an effective tool for translating new scientific knowledge for multidisciplinary and interprofessional front-line providers.

Online self-learning modules to support quality improvement initiatives have been developed in other arenas (Cameron et al., 2014; Pelayo et al., 2011) and in their many forms, they have been shown to be effective in imparting knowledge and changing behavior for particular groups of healthcare professionals (Starkey et al., 2014). Internet-based learning in general has been shown to be effective for healthcare professionals, though the literature continues to be somewhat limited by the small size and heterogeneity of studies (Cook et al., 2008). What has not been thoroughly demonstrated, however, is the utility of this type of platform when administered to providers across specialties and healthcare professions.

Methods

Study Design

We conducted a pre/post study from October 2014 to December 2014. A grant, which was supplied by the Donaghue Foundation and administered by the Association of American Medical Colleges (AAMC), supported the conduct of the study. Participants were included under a waiver of consent. This study was approved by the Institutional Review Board of the sponsoring organization and is part of the HEART Pathway Implementation Study, which is registered with clinicaltrials.gov (clinical trial number NCT02056964).

Setting

The study institution is a tertiary care academic medical center located in the Piedmont Triad area of North Carolina, serving urban, suburban, and rural populations. The institution supports the training of over 600 residents and fellows. The ED is staffed by board certified or board eligible emergency physicians 24 hours per day, 7 days a week who directly provide care and oversee care provided by residents, physician assistants, and nurse practitioners. ED patient volume in 2013 consisted of approximately 104,000 patient encounters. The medical center contains 885 licensed beds and is affiliated with an allopathic medical school. Providers from an affiliated free standing ED with 24-hour emergency physician coverage were also included in the educational intervention.

Participants

Participants included nurses, advanced practice clinicians, house staff, and attending physicians of Wake Forest Baptist Health. Departmental and institutional leadership strongly encouraged completion of this module by ED nurses and house staff, advanced practice clinicians (APCs), and house staff and attending physicians in the fields of EM, IM, FM, and cardiology. These participants were chosen because they routinely care for the target population for the HEART Pathway: patients with acute chest pain. Testing and module completion occurred in the fall of 2014.

Self-Directed Learning Module

An Articulate online module, including graphics and interactive features, was created and published on PeopleSoft Learning Management System for an academic medical center. An example of the slides comprising the module can be seen in Figure 1, and the provider module is included as Supplemental Digital Content (http://links.lww.com/JHQ/A12). The module was tailored specifically to participant roles, including customized role-specific components. For example, the mechanics of calculating a HEART score was emphasized in the emergency physician module and de-emphasized in the nursing counterpart. The base module consisted of 15 slides, one of which contained embedded slides with further descriptions of HEART Pathway components, and another one included two embedded participant response questions. Two slides were dedicated to role-specific instructions for using HEART Pathway decision support within our institution’s electronic medical record.

Figure 1.

Screenshot of Articulate Educational Module.

Pre/Post Testing

The impact of the education program on HEART Pathway knowledge and application improvement was measured via a pre/post testing design. Both tests included 10 knowledge-based or application-based questions, and five of the questions on the posttest were repeated from the pretest. Questions were multiple choice, single correct answer, with four possible choices given. Two versions of the posttest were created such that different questions were repeated. A “passing” score for the posttest was defined as 80% correct or greater.

Statistical Analysis

Descriptive statistics were used to determine training module completion rate, pass rate, and pretest/posttest scores among participants. Test score distributions were assessed visually and found to be normally distributed. Therefore, scores are described as means and standard deviations (SD) reported. Paired Student’s t tests were used to determine differences in pretest and posttest scores. Repeated measures analysis of variance was conducted to model the differences in pre/post scores based on learner department (EM, IM, FM, cardiology, and nursing) and provider type (nurse, APC, house officer, or attending physician). Statistical analysis was performed using SAS 9.3 (Cary, NC).

Results

Of 486 learners enrolled, 97% (469/486) completed the training, of which 90% (422/469) passed the training on the first attempt. The cohort consisted of 45% house staff, 34% attending physicians, 14.5% nurses, and 6.5% APCs. In addition, 42% were classified as IM, 34% EM, 14.5% nursing, 11% FM, and 8.5% cardiology. Learners who completed testing in the correct order (pretest, module, then posttest) were included in the pretesting/posttesting analysis (N = 323).

Among participants included in the analysis (n = 323), the mean pretest score was 74% (SD ± 16%). The mean posttest score was 89% (SD ± 9%). Participants’ scores increased by a mean of 15% (95% CI 14–16%) from the pretest to posttest, p < 0001. Mean pretest and posttest scores by specialty are illustrated in Figure 2, and the mean, standard deviation, median, and interquartile range for each group can be seen in Table 1. Likely due to high pretest baseline scores, EM and cardiology providers had less change in pretest and postscores than IM, FM, and nursing providers (p <.0001). Results were similar among nurses, APCs, house staff, and attending physicians (p =.11). Preliminary analysis after the implementation of this QI intervention shows that the HEART Pathway tool was used for 88% of patients presenting to our institution for acute chest pain.

Figure 2.

Mean pretest and posttest scores by specialty (out of 10 total questions) with 95% confidence intervals.

Table 1.

Results by Discipline

Specialty	N	Pretest Mean (SD)	Posttest Mean (SD)	Pretest Median (IQR)	Posttest Median (IQR)
Cardiology	28	7.4 (1.5)	8.7 (1.0)	7 (6–9)	9 (8–9.5)
Emergency   medicine	82	8.5 (1.4)	9.2 (0.7)	9 (8–9)	9 (9–10)
EM—nursing	47	7.2 (1.9)	8.8 (1.0)	7 (6–9)	9 (8–10)
Family medicine	32	7.4 (1.2)	9 (0.8)	7.5 (6.5–8)	9 (8.5–10)
Hospitalists	35	6.9 (1.4)	8.6 (1.1)	7 (6–8)	9 (8–10)
Internal medicine	99	6.8 (1.5)	8.8 (1.0)	7 (6–8)	9 (8–10)

Discussion

Translating new scientific evidence such as the safety and efficacy of the HEART Pathway into practice requires time, persistence, and educational effort. It has been estimated that new discoveries require an average of 17 years to be fully adopted into medical practice (Balas and Boren, 2000). In order to deliver the best available care to patients, this cycle must be accelerated, and educational vehicles are needed in order to do so. This kind of quality improvement and up-to-date educational initiative may be of particular importance in the setting of an academic medical center, where practice patterns and QI habits for new physicians are established (Kelz et al., 2013). In acknowledgement of this, the AAMC has created programs to more closely align the goals of continuing medical education and other educational functions of academic medical centers with the goals of quality improvement (Davis et al., 2013). This educational and QI initiative represents one example of the benefits of that sort of alignment.

Our intervention benefited from widespread participation across the medical center likely because of support from institutional leadership. Announcements and several e-mails about the educational module were sent to potential participants by their supervisors in the medical center, and participation was tracked, though not mandatory. The participant pool thus included learners from diverse backgrounds including nursing staff, APCs, house staff at levels ranging from interns to fellows, and attending physicians across a broad array of specialties. The fact that our modules were created specifically with each of these learners in mind may have contributed to the broad uptake of the QI material.

We found that each group, including nurses, house staff and attending physicians from different medical specialties, all demonstrated improved knowledge of the HEART Pathway after our intervention. Medical providers in EM and cardiology showed less improvement than some of their peers, but this appears to be due to high baseline levels of performance as demonstrated on the pretest. The variance in pretest performance among groups may have been affected by other factors including previous exposure to the HEART Pathway literature and familiarity with QI concepts. The intervention appears to have been effective across a broad range of learners because there were not significant differences between the improvements made by nurses, house staff, APCs, or attending physicians. In fact, the similar performance demonstrated by nursing staff when compared with physicians argues that the stark division between educational initiatives targeting these groups may at times be unnecessary. These findings would support the notion that future QI projects can target both nurse and physician learners simultaneously.

These results are promising for a number of reasons. The strong performance and improvement for learners across the interdisciplinary and interprofessional spectrum suggests that this kind of educational platform can effectively support quality improvement initiatives throughout the diverse terrain of a large medical center. New knowledge in the hands of lone providers may ultimately not help patients if other providers and medical staff are not aware and are unable to participate in implementing important changes. The strong support from leadership at the medical center and throughout academic departments also seems to have contributed to robust participation. The association between health system leadership involvement and success of an educational/QI initiative has previously been demonstrated (Kulawik et al., 2009).

There are several limitations to this study. Our study only includes one academic medical center, and the enthusiasm with which this new clinical paradigm was embraced may not be easily replicated elsewhere. The high pretest scores both limited the demonstrable impact of the educational intervention and provided evidence of some diffusion of this information within the institution prior to our study period. An institution with less prior investment in a given QI initiative may realize larger gains in preanalysis/postanalysis. A large number of participants could not be included in preanalysis/postanalysis because they did not complete the components in the correct order, a problem that appeared to disproportionately affect nursing staff and potentially introduced some degree of selection bias. The learning management software used here was able to prevent users from accessing the posttest before the learning module, but was not able to prevent learners from viewing the module or posttest before completing the pretest, likely accounting for these errors in order and the exclusion of those learners from the analysis. It is also not yet clear how much of this new knowledge will be retained in the long term, nor how much regular medical practice was actually modified. Further study will be needed to quantify those changes. Other investigations, however, do suggest that educational interventions such as this can lead to long-term improvements in actual medical practice (Zisblatt et al., 2013; Joyner et al., 2014). Although we are encouraged by the finding that the HEART Pathway has been utilized with 88% of patients presenting with acute chest pain in our institution, this finding is limited by the fact that clear comparison data are not available and it is impossible to say exactly what role the educational module played in this development.

Online, self-directed learning modules improved knowledge of the HEART Pathway among a multidisciplinary and interprofessional cohort of healthcare providers. This kind of educational platform and institutional directive may form the basis of future educational endeavors to support quality improvement initiatives and improve the translation of scientific discoveries to enhanced care at the bedside.

Biographies

Nicholas D. Hartman, MD, MPH is an assistant professor of Emergency Medicine at Wake Forest School of Medicine and Wake Forest Baptist Health, Winston-Salem, NC. He serves as assistant program director for the emergency medicine residency there and has research interests in medical education, assessment, training related to cardiovascular emergencies and quality improvement.

Erin N. Harper, MSHS, CCRP is a Project Manager II at Wake Forest University Health Sciences, Winston-Salem, NC, where she oversees the clinical research activities and research staff within the department of Emergency Medicine.

Lauren M. Leppert MSN, RN, CCRN, CPN is a Registered Nurse with a Master’s Degree in Nursing Administration, as well as a Certified Pediatric and Critical Care Registered Nurse. She is currently a Standards Compliance Consultant working in the Clinical Compliance and Regulatory Services Department at Wake Forest Baptist Medical Center in Winston-Salem, NC. Her job accountabilities include overseeing performance in relation to compliance with State, Federal regulations and The Joint Commission standards, facilitating an environment conducive to patient safety and safe patient outcomes, and developing, initiating, and maintaining policies and procedures as they relate to State and Federal regulations and the Joint Commission standards.

Brittany M. Browning, BHS is Associate Administrative Coordinator for Nursing Education and Clinical Compliance at Wake Forest Baptist Medical Center in Winston-Salem, NC. She assists nurse educators with creating education for nursing and medical staff and works with compliance officers to assist with regulatory state survey visits and policies for the medical center.

Kim Askew, MD is an associate professor of Emergency Medicine at Wake Forest School of Medicine in Winston-Salem, NC. He currently serves as the Director of the Clinical Curriculum for the medical school.

David E. Manthey, MD, FACEP, FAAEM is the Vice Chair of Education and Professor of Emergency Medicine for Wake Forest School of Medicine in Winston-Salem, NC. Nationally, he is known for being the founding chair of the Clerkship Directors in Emergency Medicine. He works as an educational consultant to the Dean of Academic Affairs of the Wake Forest Medical School designing the new curriculum. He is also involved with faculty development initiatives at the local and national level.

Simon A. Mahler, MD, MS, is an associate professor of Emergency Medicine at Wake Forest Baptist Health, Winston-Salem, NC. He has emerging national recognition for the delivery of innovative methods to improve care for patients presenting to the Emergency Department with acute chest pain. Dr. Mahler completed a Master of Science in Clinical and Population Translational Sciences at Wake Forest University and a mentored research program in Quality Care and Outcomes Research in Cardiovascular Disease (a T32 fellowship sponsored by the National Heart Lung and Blood Institute).