A patient safety education program in a medical physics residency

Abstract

Education in patient safety and quality of care is a requirement for radiation oncology residency programs according to accrediting agencies. However, recent surveys indicate that most programs lack a formal program to support this learning. The aim of this report was to address this gap and share experiences with a structured educational program on quality and safety designed specifically for medical physics therapy residencies. Five key topic areas were identified, drawn from published recommendations on safety and quality. A didactic component was developed, which includes an extensive reading list supported by a series of lectures. This was coupled with practice‐based learning which includes one project, for example, failure modes and effect analysis exercise, and also continued participation in the departmental incident learning system including a root‐cause analysis exercise. Performance was evaluated through quizzes, presentations, and reports. Over the period of 2014–2016, five medical physics residents successfully completed the program. Evaluations indicated that the residents had a positive experience. In addition to educating physics residents this program may be adapted for medical physics graduate programs or certificate programs, radiation oncology residencies, or as a self‐directed educational project for practicing physicists. Future directions might include a system that coordinates between medical training centers such as a resident exchange program.

1. INTRODUCTION

Patient safety and quality improvement are key components of any residency program in radiation oncology. This is well‐recognized through the ACGME requirements on quality training for physicians1 in place since 2006 and the CAMPEP requirements for medical physicists.2, 3 It is also reflective of trends in reimbursement models (e.g., MACRA in the US) that explicitly call for the inclusion of quality metrics.4 Rigorous education in quality and safety is also needed if physicians and physicists are expected to be leaders in this arena, as has been suggested.5, 6 Additionally, the recent AAPM Task Group‐100 report advocates that safety and quality “need to be incorporated in training programs for all radiation oncology disciplines.”7 It is also called for in other reports providing recommendations around patient safety.8

Although the need for quality and safety education is apparent, most residency programs currently lack a formal program for supporting this learning.9, 10 In evaluations during a one‐day safety course for physician and physics residents, only 22% of participants agreed with the statement “I have adequate education in quality improvement for my current role.”9 A recent survey of medical and physics residents in the US also noted that only 40% of medical physics residents agreed with the statement that “formal teaching of patient safety” was adequate in their program.10 Although some reports have appeared providing recommendations for a safety and quality education curriculum,2, 11 specific details are lacking and, to our knowledge, no report has yet appeared in the literature describing experiences with formal patient safety educational programs in radiation oncology residencies.

The purpose of this report was to share experience with a patient safety and quality education program developed at the University of Washington specifically for medical physics residents. While the main focus of this report is on the medical physics residency program, the experience may be useful for physician residencies and may be relevant to Maintenance of Certification through the ABR and practice accreditation programs such as APEx, in which patient safety and quality of care is a key pillar.12

2. METHODS

The patient safety and quality educational program consists of a didactic component (reading and lectures on fundamental concepts) coupled with practice‐based learning. The majority of this content is delivered in the form of a mentored rotation that is one month in duration, and there is also a component which extends throughout the residency and includes participation in the departmental incident learning program.

In designing an educational program around a broad subject like patient safety and quality, it is important to select appropriate topic areas of focus. This is particularly important in the context of a medical physics residency program. Since most programs are 24 months, the total time dedicated specifically to patient safety and quality education will necessarily be limited. In formulating focus topics we consulted the following resources: AAPM Report No. 249 on guidance for CAMPEP‐accredited residencies,2 the CAMPEP residency standards,3 the “Safety is No Accident” report from ASTRO,12 the report of AAPM Task Group 100,7 and the AAPM 2013 Summer School Proceedings on Quality and Safety.13 Additionally, the study of Dunscombe8 provides a broad summary and references to the literature, which is valuable. Table 1 shows the five topic focus areas selected for the program. The resulting topic areas are very similar to those proposed by Yeung and Greenwalt,11 are consistent with the recommendations from the ACGME for Pathways to Excellence,1 and include the specific safety‐related topics listed in the CAMPEP residency standards.3

Table 1.

Topic areas for the patient safety and quality educational program. Reference materials are assigned as reading materials (c.f. Table 2)

Topic area	Learning objectives	Reference materials
Accidents and outcomes	Appreciate a few select watershed accidents in radiation oncology and the associated issues. Understand the data linking quality of treatment with outcomes.	34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44
Failure mode and effect analysis (FMEA)	Understand the FMEA formalism and become competent in its use.	7, 45, 46, 47
Incident learning and safety culture	Appreciate the various drivers of error and contributing factors. Know the various types of events, near‐misses and incidents, and how these are monitored in clinical practice. Understand the role of culture in incident learning safe practices.	25, 28, 29, 48, 49, 50, 51, 52
Principles in error proofing and quality improvement	Gain familiarity with common error‐proofing techniques and the variable effectiveness. Appreciate the difference between identifying an error (e.g., QA) and addressing the drivers of error.	53, 54, 55, 56
Quality audits	Appreciate the role of independent audits in commissioning. Gain familiarity with patient‐specific quality audits performed by physicists.	12, 57, 58, 59, 60, 61

The topic focus areas in Table 1 were grounded in a didactic component of the program that included a series of readings (see the citations in Table 1) and six lectures. A “flipped classroom” arrangement was used whereby the lectures were recorded, and the resident reviewed them in advance.14 This allows for a more rich discussion of the topics during in‐person meetings. These lectures and discussions were supplemented with online video lecture resources, primarily from the AAPM virtual library (aapm.org), which includes presentations from a variety of speakers (28 lectures are found between 2010 and 2015 under the keywords “patient safety”). Another resource is vimeo.com where much of the same content is available also to non‐AAPM members (e.g., search “AAPM patient safety”). Another resource (not listed in the tables below) is the Learning program on Safety and Quality the from the International Atomic Energy Agency, IAEA (https://rpop.iaea.org/RPOP/RPoP/Content/News/e-learning.htm). This course consists of 12 modules, seven of which are focused on incident learning. A certificate is available after the course is complete.

A project‐based learning component builds on and reinforces the base knowledge of concepts and content areas of the didactics. During the one‐month rotation each resident is expected to complete one project in one of the topic areas in Table 1. This often takes the form of an FMEA for a particular clinical process of the resident's choice and often involves several other department members such as dosimetrists, physicians, and therapists who are involved in the clinical workflow under study.

In addition to the rotation project, the resident participates in the incident learning program of the department. This includes attendance at the weekly safety committee meetings where near‐miss events are discussed and analyzed. The department operates a high‐volume near‐miss incident learning system where approximately 25 quality reports are analyzed every week.15 The resident is expected to be involved in at least one root‐cause analysis (RCA) exercise, usually motivated by findings in the incident learning system. Since reportable medical incidents are exceedingly rare in this clinic, the RCA exercise often focuses on a potentially serious near‐miss event. This has the advantage of allowing the resident to analyze a case without the time pressures and other factors associated with a reportable incident. Our experience with this approach has been positive, and we recommend that at least the first experience of a resident with RCA be focused on a near‐miss event. The expectation for RCA is decoupled from the one‐month rotation since there are often no events reported during the one‐month period that are worthy of an RCA.

Resident learning is evaluated and feedback provided in several ways: (a) A written quiz near the end of the rotation. (b) A presentation to physics faculty at the end of the rotation. These presentations are conducted at the end of every rotation in the medical physics residency and include the learning objectives for the rotation and a presentation of the rotation project. At least three physics faculty must be present and must include the rotation mentor and at least one member of the residency steering committee. (c) A written report about the rotation project. (d) A presentation of the RCA project at one of the monthly all‐staff departmental safety meetings. (e) Feedback about performance and the rotation from the mentor to the resident and also from the resident to the program director. If the resident fails to meet the educational objectives, a plan for remediation is in place which involves further study and project work. To date, however, this has not been necessary.

3. RESULTS

Table 2 provides an example, detailed schedule for the one‐month rotation including topic areas and assignments.

Table 2.

Sample rotation outline including reading assignments and related lectures

Time	Topic area & associated references	Rotation project (Example here: FMEA)	Lecture
Week 1	Watershed accidents:34, 35, 36, 39, 42 Overview of QM chapter:37 (Ch 12) Harm estimates:40 Studies linking quality and outcomes:38, 43, 44 FMEA TG100:47 (Ch 4)	FMEA process map	1: Overview lecture 2: Watershed accidents 3: FMEA
Week 2	FMEA:7, 45, 46 Incident learning papers:28, 48, 49, 50, 51 Root‐cause analysis:62 National incident learning system:25	FMEA: collect failure modes	4: Incident learning and RCA
Week 3	Safety culture:29, 52 Error proofing:53 (pg. 1–13 and Ch 7) Barrier analysis:54 Automatic error detection:56	FMEA: score and rank failure modes	5: Safety culture 6: National and international incident learning systems
Week 4	Safety is no accident:12 (Ch 3) Safety profile assessment:57, 59 Audits and trial accreditation:60, 61 EORTC phantom experience:58	FMEA: interventions and rescore	7: Error Proofing and Design
Ongoing	Participation in incident learning program. Conduct at least one RCA on a near‐miss event.

The program was developed in an iterative manner, with feedback from the residents being used to develop it into the form outlined here. One example of this was a request from the residents that they attend all weekly meetings of the departmental incident learning program starting from the beginning of the residency instead of only during the rotation since this was viewed as a rich learning environment. This is now a residency program‐level expectation. The residents start by observing these meetings and later, as their general clinical experience grows, they participate and contribute more actively to the discussion.

Over the period of 2014–2016, five medical physics residents have completed the program described here. In all cases the rotation was successfully completed according to the educational evaluations listed above. The rotation projects generally consisted of FMEA exercises on various clinical processes including: treatment planning for SBRT lung cases, the final steps of preparing a treatment plan for treatment in dosimetry, and the process for the use and approval of patient in vivo dosimetry checks. In one case the project was reported as a manuscript,16 and provided the opportunity to compare the results of FMEA with clinical experience by way of incident learning, which revealed some novel insights. Feedback from residents about the rotation was positive, for example, “a very useful and well‐done rotation.”

4. DISCUSSION

The structured educational program on quality and safety described here has been in place since 2014 and is well‐received by medical physics residents. A key feature of the program is a concentrated rotation lasting one‐month with learning objectives focused on topic areas that are important for a medical physicist in clinical practice. The program also includes continuous learning through active participation in the departmental near‐miss incident learning system.

This program is designed to address the gap in quality and safety education in residencies that has been reported in recent studies.9, 10 In one of these, a national survey of physician and physics residents, a majority of respondents reported that formal teaching of patient safety was inadequate in their program.10 The need for a structured learning program is also motivated by the requirements of residency accrediting bodies,1, 3 the recent trends around reimbursement models that include quality reporting components (e.g., MACRA in the USA), and the need for physicians and physicists to serve as leaders in this arena.5, 6 Training is listed as a key component in essentially all reports providing recommendations around patient safety.8 The program described here is also part of a broader movement to include specialized rotations in medical physics residencies and to share curricular ideas, as recently highlighted in an AAPM symposium.17 These specialized rotations include not only quality and safety but also clinical shadowing, introductory rotations, and other topics.

One of the strong‐points of the program described here is the focus area of incident learning and root‐cause analysis (RCA). This aspect of the program benefits from the incident learning system operated by the department.15 This is a high‐volume system focused on near‐miss reports that provides unique opportunities for learning18 and is especially valuable for residents. Incident learning and RCA are particularly important for several reasons: it is one of the pillars of safety and a requirement for practice accreditation,12 it is not well understood or utilized by trainees in general,19 and it is one of the safety topics with the largest gaps in understanding and comfort among residents within radiation oncology.10 A recent survey of physician and physics residents suggested that programs with an incident learning system are more effective at providing education in safety and quality.10 Although there may be a variety of factors accounting for this correlation, one reason is that participation in a clinical incident learning safety program engages the learner because the incidents are so relevant to radiotherapy practice.

The effectiveness of incident learning is supported by many educational theories and principles. For example, Experiential Learning, as described by Kolb20 requires not only hands‐on practice, but a cycle of (a) participating in a concrete experience, (b) reflective observation, (c) abstract conceptualization to make sense of the experience and finally, (d) active experimentation, where the experience is tried again, applying the new insights gained from steps 2 and 3. This cycle is remarkably similar to a Plan, Do, Study, Act (PDSA) cycle of quality improvement. Learning is very contextual, and is more effective when the learner is situated in the professional practice, and they can participate in its activities, such as the apprenticeship model of resident and fellow education. They also learn more when the content is legitimate and meaningful, as it is with RCA of safety incidents, even if they are still under supervision and still somewhat peripheral to the action.21 However, as trainees advance through a program, they also need to solidify their learning not just through activities and tasks, but also understanding the culture and context, which is especially important for incident learning and safety.22

To our knowledge, there is no report in the literature which describes the details of a formal patient safety educational program in radiation oncology residencies. Several related reports have appeared, however. Across healthcare in general there are reports about the experience of residents particularly in the realm of practice‐based quality improvement and associated metrics.23 Yeung and Greenwalt11 have proposed a framework specific to radiation oncology that would satisfy ACGME requirements for patient safety and quality improvement. The suggested topic areas are very similar to those outlined here. Fogh and Pawlicki9 reported on a specialized training course on quality and safety and reported substantial improvements even from this single‐day event. The University of Pennsylvania has developed a Quality and Safety Culture Education program and recently reported their five‐year experience.24 Though this program was developed for all staff, it did include a component specifically targeted for residents. Excellent engagement was demonstrated by physician and physics residents. There was an improvement over time for the self‐reported scores that gauged culture of safety, although the measured changes for physics residents and medical residents were not statistically significant.

A variety of challenges might be anticipated in implementing an educational program like the one described here. First, the mentor(s) might not have an extensive expertise in the topic and may feel uncomfortable teaching quality and safety concepts. This report may partially address this challenge by providing a baseline structure as well as references and background materials. It is also expected that the projects suggested here (e.g., RCA or FMEA) provide a rich opportunity for learning for both the resident and the mentor(s). These activities can also serve for Maintenance of Certification activities for the American Board of Radiology. In particular, it may provide credit for a self‐directed educational program (SDEP) which is a type of Self‐Assessment Continuing Education credit available to medical physicists (SA‐CE; see Appendix A for more details).

A second challenge might be in delivering the didactic content that supplements reading materials. The program described here employed a “flipped classroom” (i.e., recorded lectures) to deliver didactic content. This facilitates a more interactive discussion of the topics during in‐person meetings. Programs may find it challenging to develop such content. However, recorded lectures may be borrowed from colleagues or may be easily developed using low‐cost screen‐capture software such as Camtasia (TechSmith Corp., Okemos, MI, USA). The AAPM virtual library is another resource for teaching materials.

Scheduling may also represent a challenge. The program described here used a one‐month dedicated rotation. This was found to be a very compressed time scale and the rotation will be extended to 1.5 months in the future. Even with a compressed rotation, however, some residency programs may find it challenging to set aside this time from a typical 24‐month program, given the many other demands of the program. While there is no direct solution to this, it may be possible to deliver much of the learning described here outside of a rotation with a dedicated time period, such as in a weekly or monthly journal club.

A final challenge may arise if the program does not have a departmental incident learning system and, related to that, does not have a strong culture of safety. For programs which do not have a departmental incident learning system, it may be possible to review case studies from the literature or from RO‐ILS™: Radiation Oncology Incident Learning System25 which are publicly available.26 It can be expected that this will be less of an issue in the future since departmental incident learning systems are specifically called for by ASTRO and other professional societies12 and, in the US at least, will be incentivized by reimbursement models under the new MACRA rules. The needs of the educational program may provide an additional impetus for engaging in such a program. The tools for incident learning in radiotherapy are now well understood and described in the literature18, 27, 28 and specialty‐specific systems are available, such as the RO‐ILS system25 sponsored by ASTRO and AAPM and the Center for Assessment of Radiological Sciences PSO system (cars‐pso.org). Related to this is the need for a strong culture of safety, which is a driving force for quality care. Culture of safety has been linked to patient outcomes29 and has been specifically mentioned in the radiation oncology context.5, 24

While this report has focused on medical physics residents, parts of the program described here may have a broader applicability. Parts of the program may be adapted to serve the continuing medical education needs of faculty and staff (see Appendix A). It may also be possible to adapt the program for physician residents, but changes would need to be made including a modification of the topic focus areas (e.g., Table 1). For instance, incident learning should include the issue of error disclosure and professional aspects.30 Principles of error proofing (Table 1 & 2, for example, barrier analysis, QA, and automatic error detection) could be modified to have less of a physics focus, and quality audits could be modified to focus less on commissioning and phantoms and more on issues like physician peer review.31, 32 Adapting aspects of the curriculum may also be of interest to graduate or certificate program directors.

There are some limitations to this study. One important limitation is that not all safety‐related topics can be included in an educational program like the one presented here, and Table 1 should not be taken as an exhaustive list. For example, one topic which is not included is statistical process control. This technique, which has attracted attention recently, involves analyzing and tracking the results of some procedure or test over time to establish clinical action levels33 and has an analog with physician‐reported performance metrics, which have been called for in the context of trainee education.23 While such topics are called for in some reports,11 it must be acknowledged that not all aspects of patient safety and quality of care can be included in a relatively focused educational program like the one described here.

The program described here is expected to evolve and develop further. One future direction might be a distributed program that is coordinated between a few centers. This might include a combined teaching component, which leverages the expertise of faculty at various centers, and/or a resident exchange program which facilitates a shared learning experience. Another direction may be to encourage partnerships between physician and physics residents to conduct projects together. Any future effort would benefit from a certification process such as that eLearning course from the IAEA (https://rpop.iaea.org/RPOP/RPoP/Content/News/e-learning.htm).

CONFLICT OF INTEREST

The authors declare no conflict of interest.

APPENDIX A.

MAINTENANCE OF CERTIFICATION CREDITS FOR MEDICAL PHYSICISTS UNDER THE SDEP MECHANISM

Summary of SDEP mechanism: Self‐directed educational programs (SDEP) are one method that the American Board of Radiology (ABR) recognizes for Medical Physicist for fulfilling Part 2 of maintenance of certification (MOC) requirements (i.e., Lifelong Learning and Self‐Assessment). Current ABR requirements call for 75 Category 1 CE credits in a three‐year period. At least 25 of these credits must be Self‐Assessment CE (SA‐CE). One common type of SA‐CE is ABR‐prequalified SAMs credits offered at meetings and elsewhere. Another type of SA‐CE credit specifically available to medical physicists is the SDEP. Details are found at www.theabr.org/moc-rp-comp2: “A maximum of one SDEP may be recorded yearly. Fifteen CE credits are given for each completed SDEP, and project samples can be found on the ABR web−site.”

The template provides an example for SDEP project related to the educational program described in this study.

SDEP project for ABR MOC credit

Title: Develop a patient safety and education program for medical physics residents

Category: Education

Date Initiated: TBD

Date Completed: TBD

A. SIGNIFICANCE

Patient safety and quality of care are key core concepts for medical physicists and are a competency of medical physics residents required by CAMPEP. As future leaders it is essential that residents have a strong foundation in the concepts and practice of safety and quality. The program developed here will support these educational needs.

B. APPROACH

Example: I will develop the structure for an educational program for residents in our medical physics program. This will consist of a one‐month rotation. The rotation will include a series of background readings on quality and safety. In this SDEP I will also develop a series of four lectures to support this learning. A future SDEP project (planned for next year) is to develop a rotation project centering on failure mode and effects analysis (FMEA) which will allow the resident to experience hands‐on learning.

C. EVALUATION OF ACHIEVEMENT

1. Prospective statement (provided when SDEP is initiated)

Example: I will assemble a list of readings and review them myself in detail. I will develop and deliver four power‐point lectures. At the end of the rotation, residents will provide an evaluation of the rotation which will help assess the success of this project.

2. Final statement (provided when SDEP is completed)

Example: The project has resulted in a finalized reading list of 30 key studies, three book chapters and four educational videos from the AAPM virtual library (see list below). These are assigned to the resident when they start the rotation. I have also developed and delivered four power‐point presentations on the following topics: “Example errors in radiation oncology,” “QA — What does it mean?”, “Root‐cause analysis and how to investigate an error” and “Culture of Safety: Incident learning and me.” Resident assessments have been positive. One resident, for example, noted: “This is better than quantum physics II. I loved it!”

D. IMPACT ON PRACTICE/OUTCOME STATEMENT

1. Prospective statement (provided when SDEP is initiated)

Example: The project addresses the important need for a structured program in quality and safety for radiation oncology physics residents. This is expected to have a significant impact on their understanding as they move toward more independent practice.

2. Final statement (provided when SDEP is completed)

Example: The implementation of this program has helped residents to better understand key concepts in patient safety and quality of care. They have mastered the key concepts and have a basic facility with some of the tools for quality management. The program has also provided an opportunity for mentors to further learn these topics in detail.