Improving patient safety by instructional systems design

Abstract

Education and training are important elements in patient safety, both as a potential contributing factor to risks and hazards of healthcare associated injury or harm and as an intervention to be used in eliminating or preventing such harm. All too often we have relied on training as the only interventions for patient safety without examining other alternatives or realizing that, in some cases, the training systems themselves are part of the problem. One way to ensure safety by design is to apply established design principles to education and training. Instructional systems design (ISD) is a systematic method of development of education and training programs for improved learner performance. The ISD process involves five integrated steps: analysis, development, design, implementation, and evaluation (ADDIE). The application of ISD using the ADDIE approach can eliminate or prevent education and training from being a contributing factor of health associated injury or harm, and can also be effective in preventing injury or harm.

Health care is unique among various professions because of the relationship between the education and training of health professionals and the delivery of patient care. Hospitals and clinics around the world depend upon trainees to deliver major portions of care to patients. These trainees, whether nursing or medical students, graduate medical trainees, pharmacists, or allied health professions students, are often placed at what Reason¹ refers to as the “sharp end”—that is, closest to the action where an error can be critical to patient safety. The linking of education to the delivery of patient care has created what Perrow² defines as a tightly coupled system, where errors in one part of the system place other components at significant risk. The learning process is filled with error as the learner tries to develop and refine his or her new professional role.

In many error critical situations (where even a minor error can have disastrous consequences) such as aviation, this learning process takes place in a protected environment such as a simulator to minimize the risk to the public, the teacher, and the learners themselves.³ However, this is not always the case in health care, especially in teaching hospitals and clinics where trainees deliver a significant level of care to patients. Thus, our educational systems can and often do represent what Reason¹ calls a “latent hazard” within the healthcare system.

Gawande,⁴ in his work on surgical education, describes having to attempt new skills on patients for the first time. There is a significant learning curve before expertise can be achieved. This concept of using patients to practice essential skills represents a significant patient safety risk—for example, in the area of bariatric surgery there is strong evidence that a 150–200 case learning curve exists before competency is achieved.⁵ What patient would like to be among those first cases during the learning curve? A significant ethical question arises: can we put today's patients at risk for the benefit of training tomorrow's practitioners? When introducing new clinical procedures we must be able to design our education and training systems in order to achieve maximum learning and development of competency without endangering patients.

The maintenance of competence or lack thereof is another potential patient safety risk. Once a practitioner is deemed competent, no matter at what level, there is no assurance that the competence will remain. When new clinical procedures and practices are introduced, the “trained professional” is just as potentially dangerous to the patient as the new trainee. In fact, some would say they are even more dangerous because they are operating without supervision.

Unfortunately, education and training have often been looked upon as the way to correct patient safety problems identified in event reporting systems. This author has noted the prevalence of corrective actions listed as follow up activities to patient safety events as “individual retrained and procedure modified”. There is a tendency to focus on finding the guilty party and then to retrain them. This has become known as the “blame and train” syndrome.⁶ Training when improperly applied as an intervention can make matters worse by not addressing all of the related contributing factors in the chain of events that lead up to an adverse event.

The challenge in patient safety is to examine carefully the role of education and training, both as a potential latent system hazard as well as a potential intervention. Thus, education and training activities must be carefully designed in order to be effective. With this in mind, this paper will present the concept of instructional systems development (ISD). ISD is a systematic method that, if carried out correctly, will improve the outcome of any education and training experience. The ISD process involves five integrated steps: analysis, design, development, implementation, and evaluation (ADDIE).⁷ The concept of ISD is not new to health care nor to health professions education. In fact, the author has taught this model in graduate programs in biomedical communications and in faculty development programs for primary care specialists for over a quarter of a century.⁸,⁹ While this approach is used in health professions education and is well known to medical educators, it has had limited application as an approach to patient safety. This paper will describe how the ADDIE approach to ISD can be applied to patient safety as a means of providing safety by design.

Instructional systems development (ISD)

The concept of ISD is built upon some basic concepts about learning and instruction. Learning takes place within an individual and involves both cognitive and motor functions that lead to behaviors. As Rogers¹⁰ has noted in his work on learning, you can never really teach anybody anything, you can only facilitate learning. Instruction is the formal act of facilitating learning. One must therefore design the learning experience/instruction in such a manner that it will optimize learning. The application of ISD principles and approaches is one way to optimize learning through the design of the instructional process.

Planning, development, delivery, and evaluation of instruction should be conceived in terms of general systems theory.¹¹ Instructional systems are composed of various interrelated components that function together to facilitate learning. ISD is the process of designing learning through instruction where the focus is on learning rather than on teaching.¹² There are levels of learning or cognitive process of information that are essential elements for the design of effective instruction/learning.¹³,¹⁴ The US military and industry were some of the first organizations to embrace the concept of instructional development and design. In 1975 the US Department of Defense published its five integrated steps approach to ISD involving analysis, design, development, implementation, and evaluation (ADDIE).⁷ The concept of ISD has been widely accepted and incorporated within the education profession as a distinct specialty and there are now numerous textbooks on the subject.¹⁵,¹⁶,¹⁷

The ADDIE method

The A in ADDIE refers to the Analysis phase which involves identifying the target audience or learners to determine their learning needs. The first D is for the Development phase which involves delineating entry behavior, learning outcomes, and learning sequence. The second D is for the Design phase which involves specifying the content to be covered and selecting the medium instruction to be used to deliver the content and two stages of prototype testing. Implementation involves delivering the instructional program and providing logistical support for the program. Evaluation is focused not on the learner per se but on whether or not the instructional program was effective in obtaining the desired changes in learner behavior and/or attitude.

Analysis

The first step in the analysis phase is to identify the target audience for the instruction. Different groups of learners have very different needs based on different roles and functions that they are expected to perform. There are a variety of methods that can be used during the analysis process including task analysis of job functions, focus groups, surveys, and consensus development of needs and requirements. For established roles or job functions, a task analysis may well be the desired approach to needs assessment. Many of the types of analysis methods used for ISD are exactly the same or very similar to the tools and techniques used by human factors specialists in design work and safety systems.

For new or emerging roles such as that of the patient safety officer, one might select an expert consensus development approach. This author has used the Ideals concept of design combined with the Delphi and Nominal Group Technique to identify roles and functions as part of needs assessment.⁸,⁹,¹⁸ It is only after a complete analysis of these different needs that one should precede to the next step in the process—development.

Development

The development phase involves specifying the learning outcomes and performance objectives for the instructional activities. From the early work on objectives by Tyler¹⁹ in the 1930s through the development of behavioral objects by Mager²⁰ in the 1960s, there is extensive guidance available for this essential element of instruction. The existence of well formulated objectives guides those managing the instructional process as well as the learners themselves. In the case of highly motivated learners, when given the objectives of instruction, these learners can often succeed in spite of their teachers or faculty. It is at the development stage that one would identify entry level skills and behavior and specify the sequence and structure of the instruction to be carried out.

Design

The design phase involves taking the specifications from the analysis and development phases and actually designing the instruction itself. The design involves specifying the learning events or activities and developing an instructional management and delivery system for the instruction. It is at this stage that one begins to map the learning needs and requirements with the appropriate learning experiences and stimuli that will lead to the desired learning outcome. Dale²¹ conceptualized learning stimuli as a cone of experience ranging from direct purposeful experiences (delivery of care to patients) through abstract visual and verbal symbols (reading a book). Figure 1 is an adaptation based on Dale's original cone of experience. At the base of the cone are the direct experiences; next are contrived experiences such as simulation. Moving up the cone, one moves away from direct involvement of the learner to more passive observations and mediated experiences such as learning stimuli. At the top of the cone are verbal symbols, most often represented by the printed word. The task of the instructional designer is to match the learning requirements with the most appropriate and available learning stimulus. In healthcare education we all too often rely heavily on direct experiences, which may in fact place patients at extreme risk of injury and harm. At the other end of the spectrum we also rely too heavily on verbal symbols or printed text to provide the learning stimulus when visual and mediated stimuli may be required. We are fortunate today to have available a wide variety of instructional media from which to choose. With the advent of the computer and the widespread acceptance of the world wide web or the internet, we now have a multimedia delivery vehicle that is global in dimension. In selecting the medium of instruction we must be mindful of Marshall McLuhn's warning that “the medium is the message”.²² The attributes of any particular medium convey their own unique message. Another cautionary note is that advocates of any particular mode of instruction often oversell its benefits. The old saying that “if the only tool you have is a hammer, then the world appears as a nail” applies in the case of instructional media. There is no magic instructional bullet that can be applied effectively in all situations. Careful selection and analysis of the strengths and weakness of any mode of instruction are necessary from traditional leader led instruction to complex high fidelity simulators or web based self‐instruction. In some cases it is not necessary actually to create new materials but, rather, to use existing resources that can be gathered together to form the instructional program. Often a set of instructional materials can be assembled in teaching or learning kits to support instruction.²³

Figure 1 The cone of experience. Adapted from Dale.²²

The last step in the design process is to complete prototype testing. It is essential, at various points in the design process, that prototype versions of the instructional system be tested. At least two types of prototype testing are advisable—one in paper format (the “content format/storyboard draft stage”) and the other is a production prototype.²⁴,²⁵ Testing need not be extensive at the earlier stages but should include a larger sample at the prototype stage.

Implementation

The implementation phase of the ISD process involves activating the instructional management plan and conducting the instruction. The deployment of the instruction must be as carefully planned as it is during the design and development phases. Many an excellent instructional program fails because of a weak or non‐existent implementation plan. In most educational settings there are instructional gate keepers who serve to provide access to learners. These gate keepers are training managers or teachers better known as faculty. There is always someone in virtually every instructional setting who serves this critical function. Thus, any implementation plan must include a strategy to involve these instructional gate keepers in order to reach the intended audience. Special in‐service for faculty development sessions may be required as part of this stage in the ADDIE process. One also must assess the learning environment in which the instruction is to take place and be sure that all of the resources required for the instructional program are present.

Evaluation

The evaluation phase actually begins during the design phase with the two stages of prototype testing previously described. These evaluative steps are the initial formative evaluation of the instructional program. It is essential that internal evaluation be conducted before the program is widely disseminated or distributed. However, if the instructional program is to be used in multiple settings, it should be tested in a representative sample of these settings during prototype testing. It may be advisable to have an external evaluation of a large scale instructional program to ensure an objective and complete review of the effectiveness of the program itself. While much of the evaluation will be an assessment of the learning outcomes of individual learners, it is important to remember that the real focus of the evaluation of an instructional program is on the program and not the individual per se.

Kirkpatrick²⁶ outlined four types (levels I, II, III, and IV) of evaluation of learning outcome measures: reactions, learning, behaviors, and results. Reactions are simply measures of how well trainees liked or valued a training program and certain aspects of the instructional program itself. Learning involves measuring the extent to which trainees understand and retain principles, facts, and techniques imparted during the instructional program. Behavior refers to measures of any behavioral changes that occur as the result of the instruction. Results refer to the measurement of the impact of training on organizational criteria. In the case of instruction in patient safety, these criteria would be reductions in risks and hazards associated with the delivery of patient care.

Another aspect of evaluation that is often overlooked is the actual adoption of the instructional program. The concept of adoption is Roger's concept of diffusion of innovation.²⁷ It is possible that a very effective instructional program could be designed and implemented only to find that no one outside the developing institution uses the new resource. To be considered truly effective an instructional program system must be used; many an effective program gathers dust on some library shelf and never reaches the intended audience. Thus, an adoption component must be included in an overall approach to the evaluation of instruction.

Applying ISD for safety

Safety by design in the area of education and training will involve two distinct and different challenges—the redesign of existing instructional systems and the development of training in new skills needed to address patient safety issues. The first challenge can be considered as re‐engineering our educational systems to eliminate their potential as latent hazards.

Education as latent conditions

One can think of latent conditions as hazards resulting from the delayed consequences of technical and organizational actions and decisions. These underlying conditions may predispose individuals at the “sharp end” to fail.¹,²⁸ One way to conceive the potential for latent conditions caused by education is to use a continuum extending from the “sharp end” of those delivering care to patients to the “blunt end” of those allocating resources and establishing policy and procedures that impact on the work environment of caregivers at the “sharp end”. Figure 2 is an illustration of this continuum. At the “sharp end” are the actual providers of care, both those in training and already in practice. In the case of the in‐training group, one would need to ensure learning while protecting the patient from harm during the learning process or that those providing care were competent. For the practising professional, one would focus on continuing education and development of new skills based on application of new approaches to care for improved safety. The next level of the continuum is the mid level supervisors and managers who develop policies and procedures that influence or shape clinical work. Included in this level would be teaching faculty and clinical supervisors, program directors, and department heads. At the final level of the continuum are the organizational leaders who allocate resources and manage the clinical system (deans, hospital CEOs, and chief nursing and medical officers). These are the individuals who are often responsible for unintentionally creating latent hazards or system failure through their action/inaction and decision in managing organizational resources. As Reason²⁹ has pointed out, the higher up the organizational ladder one is, the greater one's capacity for generating latent hazards. At each level of the continuum, individuals can cause harm to patients when they interact with a hazard. When would one find faculty induced hazards or errors? An example would be when faculty state that learners need only simulate hand washing in Standardized Patient (SP) experiences to save time. The learners thus gain the attitude that hand washing is time consuming and there is insufficient time for hand washing between patients. Another example of latent conditions would be work hours of both trainees and staff. By ignoring the problem of sleep deprivation, the CEO or dean who states that work hours cannot be reduced is being a hazard. While this is important, in many cases it may miss the point by not looking at what is being done during those hours.

Figure 2 A continuum of patient safety learners.

Removing these latent conditions may be the most challenging for the instructional designer/patient safety expert because it will involve changing the standard approaches to teaching that have existed for centuries. Separating the learner from the patient during skills development and finding other ways to provide safer learning spaces should be a required safety step in health professions education. The adoption of simulators and skills laboratories will be needed. Considerable resistance can be expected. One often hears the expression at curriculum committees: “we don't need simulation when we have plenty of patient material available in the hospitals and clinics”. Many patients would be shocked and distressed to learn that some faculty consider them as teaching material for their students. Some of this redesign of instruction will also accompany the redesign of clinical work altogether in order to improve patient safety.

Designing in safety

Significant changes are occurring in health care as new technologies are being introduced; new facilities are being built which significantly shape the nature of the patient care experience. Traditionally, the educational impact of these changes in facilities and practices have been afterthoughts—if thoughts at all. Including education instruction as part of overall systems changes will be required lest we continue to inadvertently introduce latent conditions. An example of including education and training as part of overall systems design comes from the military in its approach to new technology and systems. When the US Department of Defense contracts for the development of a new fighter plane or new ship, they also contract for the development of the educational system for that new system. Thus, when Lockheed Martin built the C5 cargo plane, they also built all of the skills trainers and simulators for operating and maintaining the C5. A secondary effect of this approach to systems training is that defense contractors are now some of the most experienced educational development and instructional development groups around. As we introduce or contemplate new technologies, especially HIT, we need to incorporate good instructional design and development as part of these new systems in order to avoid creating new latent conations and risk and hazards. The literature on patient safety has a growing list of cases where this mismatch has occurred.

A second challenge is the design and implementation of entirely new instructional programs to develop new skills in dealing with patient safety. This challenge is best illustrated by a new professional role that is emerging in healthcare institutions—the patient safety officer. There is a growing requirement for every hospital to have a patient safety officer, but what are the knowledge skills and attitudes needed for this new role? The development of an instructional system to support the education and training of a patient safety officer is a perfect opportunity to apply the ISD method to the field of patient safety. How we meet this challenge may well influence the course of an entire field of endeavor.

An example of a patient safety intervention

One area where this is being applied is in the area of teamwork and communications. Teamwork and poor communications have been shown to be leading contributing factors in patient safety events. The author has been actively involved in a project to improve teamwork and communications skills for health professionals. The US Agency for Healthcare Research and Quality (AHRQ) and the US Department of Defense (DoD) are engaged in a cooperative activity to develop and deploy teamwork curriculum and support materials that can be used by healthcare institutions to improve teamwork and communications skills of healthcare personnel. The curriculum and resource materials resulting from this effort have been entitled TeamSTEPPS (Team Strategies and Tools to Enhance Performance and Patient Safety).TeamSTEPPS has been developed over the past 4 years as an evidence based resource for conducting teamwork training through a “train the trainer” approach. TeamSTEPPS is a flexible training kit or FLEXTRA kit²³ which provides all the necessary materials to support leader led workshops. The FLEXTRA kit includes a leader's guide, learner's guide, presentation materials in PowerPoint and video vignettes and group exercises, and evaluation tools. The kit is being distributed as a CD with the printed materials and presentation modules and one DVD with all the video vignettes. The TeamSTEPPS kit is also downloadable from the web. Table 1 is a brief description of the application of ISD to this project. For those readers interested in examining how well the ISD principles were applied in TeamSTEPPS, they can examine the product from the AHRQ web site at www.ahrq.gov.

Table 1 Applying the ADDIE ISD model to patient safety training (TeamSTEPPS).

ISD step	Actions
Analysis	Reviewed patient safety event reports to assess risks for communication and teamwork
	Reviewed the evidence for teamwork and team training in health care and other industries28
	Conducted extensive needs analysis of healthcare facilities, both civilian and military facilities
Development	Developed a curriculum plan with core concepts and identified knowledge skills and attitudes in four key team skills. Developed outcome measures of team performance and clinical indicators of “teamness”29
Design	Selected the FLEXTRA kit model for leader led “train the trainer” workshops.24 Materials include instruction guide, learner's guide, presentation materials as PowerPoint slides and video vignettes. Conducted storyboard testing of video vignettes prior to actual production
Implementation	Prototype tested TeamSTEPPS in military treatment facilities worldwide. Coordinated initial implementation with large health systems interested in improving teamwork. Plans in place for national implementation through Quality Improvement Organization (QIO) in each of the states and territories of the US
Evaluation	Conducting independent institutional trials in military and civilian hospitals. Adoption assessment to be conducted

Conclusion

Education and training are an important part of patient safety. On the one hand, education and training can represent an embedded latent hazard to patients where the training of new health professionals can place a patient at increased risk of heath care associated injury. On the other hand, education and training can serve as an effective intervention for helping to improve knowledge skills and attitudes needed to make patient care safer. This paper has presented in brief form the ADDIE method of ISD that can be used to improve instructional outcomes. While other ISD approaches exist in the literature that are also effective, this author has found that the ADDIE approach to instructional design is very effective and has been able to apply the method in a variety of situations. Applying well established principles of ISD using the ADDIE approach for the creation and use of instructional programs is one way in which we can make health care safer by design.