The practice of simulation-based assessment in respiratory therapy education

Abstract

Clinical simulation has gained prominence as an educational approach in many Canadian respiratory therapy programs and is strongly associated with improved learning, clinical and nonclinical skill, future performance, and patient outcomes. Traditionally, the primary assessment approach employed in clinical simulation has been formative debriefing for learning. Contextual factors, such as limited opportunities for learning in clinical practice and technologically oriented perspectives on learning in clinical simulation, are converging to prompt a move from using formative debriefing sessions that support learning in simulation to employing high-stakes testing intended to measure entry-to-practice competencies. We adopt the perspective that these factors are intricately linked to the profession’s regulatory environment, which may strongly influence how simulation practices become embedded with respiratory therapy educational programs. Through this discussion we challenge the profession to consider how environmental factors, including externally derived requirements, may ultimately impact the effectiveness of simulation-based learning environments.

BACKGROUND

During the mid-twentieth century increasingly complex respiratory technologies and procedures, such as patient interfaces for therapeutic gas delivery and mechanical ventilation, were being introduced into the Canadian healthcare system [1]. It was during that time that the profession of respiratory therapy emerged from a need in the medical and anaesthesia communities for appropriately trained individuals to support these new technologies and therapeutic procedures [1, 2]. Since that time respiratory therapists have evolved from being primarily technical healthcare workers trained in hospital-based programs to highly educated and skilled professionals who function as part of an interdisciplinary team of health professionals [3]. While Canadian respiratory therapists now work in a wide variety of health-related settings, providing a broad range of service from hospital-based to community and primary care, their practice remains largely centered in acute care settings [3].

Respiratory therapy has also evolved over time and entry-to-practice education is now provided by institutes of higher education across Canada through 3-year diploma and 4-year degree programs [4]. Because respiratory therapy is a competency-based profession, where practice occurs in clinical settings, respiratory therapy education necessarily occurs in both the classroom and clinical practice environments. Respiratory therapy students are required to engage in learning the skills, attitudes, and behaviours of professional practice in authentic environments. Clinical simulation-based education has, in part, been rapidly adopted by respiratory therapy educational programs, and by those of other health professions, because it offers a safe environment in which learners can develop professional skills without the risk of causing harm to actual patients [5].

WHY HAS RESPIRATORY THERAPY ADOPTED SIMULATION-BASED EDUCATION?

In recent years, a growing interest in assuring patient safety has been fuelled in large part by an Institute of Medicine report documenting the magnitude of medical errors in US hospitals [6]. The report determined that at least 44,000 people, and perhaps as many as 98,000, die in hospital each year due to medical errors [6]. The report was particularly alarming to the respiratory therapy community given that its primary practice contexts were noted by the report to be those areas of healthcare segments most at risk. For example, the report highlighted that the highest error rates and most serious consequences are most likely to occur in critical care units, emergency rooms, and operating rooms [6]. In response to emerging concerns over patient safety in healthcare systems, clinical simulation has been identified as an educational tool that enables learning experiences for health professionals in an environment that does not compromise patient safety [7].

Compounding the impacts that concerns over patient safety have had on health professions’ education programs, limited access to adequate and appropriate opportunities for learning in clinical environments has made assuring sufficient experiential learning opportunities for health professional students increasingly challenging [8]. This limitation appears to be particularly evident in specialty practice areas (e.g., critical care and pediatrics), which also typify the primary practice environments of respiratory therapists. As is the case in the educational contexts of other health professions, clinical simulation-based education in respiratory therapy has thus also emerged as a technique that is said to facilitate learners to “engage in the same critical thinking and clinical decision-making skills required in actual clinical practice” [9]. It is suggested, therefore, that clinical simulation can offer a means of ensuring learners are optimally prepared to safely begin practice in real settings, while at once helping to address the resource shortcomings of the clinical education context.

THE FOUNDATIONS OF SIMULATION-BASED EDUCATIONAL PRACTICE

Clinical simulation theory has longstanding historical roots in the aviation industry, from which health professions’ education has adopted many of its early simulation-based educational practices [10]. The past two decades have witnessed an expansive growth in the use of clinical simulation in the education of healthcare professionals to address issues of patient safety and quality care and to enhance the traditional apprenticeship model of medical education [10, 11]. When employed with a well-designed formative feedback mechanism, clinical simulation in this context has been shown to be useful in supporting student learning needs [7, 12].

There also exists a move to ensuring support for higher ordered learning in health professional education for practice in increasingly complex environments [5]. For example, to meet the regulatory requirements for licensure and subsequent entry-to-practice for the profession of respiratory therapy, graduates must “perform continuous self-evaluation,” “demonstrate critical judgement in professional practice,” “demonstrate problem-solving skills,” and “demonstrate decision-making skills,” among other competencies [13]. “The concepts of meta-cognition and self-directed learning provide the theoretical mechanism for designing and implementing meaningful and worthwhile educational practice. That is, they describe the processes by which higher-ordered learning occurs” [14]. Simulation is seen as one solution to this educational need [5].

Simulations encompass very carefully crafted reconstructions of realistic scenarios, or they may simply replicate a component or group of components of a clinical context to provide a degree of reality [5]. Chiniara et al [11] delineated a variety of simulation modalities along technological lines including: computer-based simulation, simulated patients, simulated clinical immersion, and procedural simulation. Cook et al [15] defined clinical simulation as “an educational tool or device with which the learner physically interacts to mimic an aspect of clinical care for the purpose of teaching or assessment.” In prefacing their instructional design framework for clinical simulation, Chiniara et al [11] problematized simulation technologies as often considered an educational method, noting that the use of any particular technology may vary widely.

Indeed, concerns with persistent technological-centric approaches to clinical-simulation practice surface regularly in the literature and they are responded to with calls for greater educational theorization in the field and recognition of clinical simulation as a social practice [8, 16]. In doing so, the utility of clinical simulation as an educational approach may be understood as resting in the interactions that occur between many elements of this complex learning system, including those elements that can be designed for (e.g., assigned tasks, the technology, choice of participants) and those which cannot be designed for (e.g., sense of community, emergent activity) [17, 18]. In particular, the social aspects of the learning environment (e.g., factors which might affect learners’ emotional or psychological status or which may impact their sense of safety and trust) represent those nondesignable elements of clinical simulation that underlie the effectiveness of the learning environment [16, 18].

ASSESSMENT PRACTICES IN SIMULATION-BASED EDUCATION

Clinical simulation is now a well-established practice in health professional education, and formative debriefing for learning has traditionally been employed as the primary assessment strategy in that practice [10, 19, 20]. However, having first acknowledged that engaging learners in a debriefing practice that focuses on multiple forms of feedback is the most important and frequently cited assessment practice through which to promote effective learning; a review by McGaghie et al [20] also identified the opportunities for increasing the application of simulation in high-stakes examinations, such as those used to evaluate readiness of health professionals for licensure.

The use of debriefing in the clinical simulation context is associated with improvements in various areas including: learning, clinical and non-clinical skill, future performance, and patient outcomes [8, 19, 21]. The methods of debriefing performance that have been examined in relation to their impact on learning generally include facilitators’ provisions of critically constructive and empathetic feedback, and providing opportunities for learners to engage in self- [22] and peer-assessments [8]. Eppich and Cheng [23] further differentiated these formative debriefing approaches into three broad categories: (i) learner self-assessment, (ii) focused facilitation to promote critical reflection and deeper understanding of events, and (iii) directive performance feedback. Contending that each approach can at times be useful as an educational approach, Eppich and Cheng [23] have advocated for a blended approach to debriefing that can respond flexibly to specific educational goals. While the existence of a variety of approaches to debriefing are evident in the literature [23, 24], the practice of debriefing in simulation-based education remains predominantly formative in nature.

The movement towards using simulation for competency-based evaluation for certification or licensure in health professions, as noted by McGaghie et al [20], is echoed throughout the literature [25–27], though existing practices amongst health professions in Canada vary with respect to the use of simulation for assessment of practice readiness. This growing trend, both in research and application, appears to be primarily occurring in the areas of procedural specialties in medicine [20, 28, 29]. Within the professions of respiratory therapy and nursing in Canada, clinical simulation-based examinations for entry-to-practice have not yet been implemented [30, 31].

RESPIRATORY THERAPY REGULATORY ENVIRONMENT: EDUCATIONAL IMPLICATIONS

The potential influences of a regulatory environment on respiratory education programmatic decisions, including assessment approaches, can be critically examined through a social practice theory lens. Social practice theory provides a structure for examining what people do, what they value, and which meanings they derive from participating in a shared, situated practice [32]. By adopting a social practice theory perspective—one that acknowledges the influences that socio-cultural environments may have on the activities of a community of practitioners—a previous exploration of assessment practices in health professional education contexts exposed the professional regulatory environments as an influential factor [18]. Building on this understanding, there is value in exploring the regulatory environment that exists within the profession of respiratory therapy as an important contextual factor that may influence the practices of its educational programs.

In 2003 the profession of respiratory therapy was amongst the first health professions in Canada to adopt competency-based entry-to-practice requirements [33]. The National Competency Profile for Respiratory Therapists in Canada dictates the competencies that an entry-level respiratory therapist is expected to be able to perform in the workplace, and it identifies the outcomes that must be achieved by the conclusion of educational programs in respiratory therapy in Canada [13]. In response, Canadian institutions offering respiratory therapy education programs have implemented curriculums founded on the principles of competency-based education that comply with the discipline’s accreditation requirements [34]. Frank et al [35] defined competency-based education as an approach to preparing health professionals “for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs.” Given that the clinical simulation-based education technique aligns well with the achievement of objectives mandated by competency-based education, it is not surprising that uptake of simulation-based education has grown since implementation of the first National Competency Profile for Respiratory Therapy in Canada.

In response to growing interest in the use of clinical simulation to support development of professional competencies in respiratory therapy, the 2011 iteration of the national competency profile identified specific competencies that could be assessed for entry-to-practice in clinical-simulation environments. Many of the competencies that the regulatory bodies have determined may be assessed using simulated environments consist primarily of procedural skills. A representative example is pediatric endotracheal intubation [13]. In the case of those competencies in which clinical simulation may be used to assess competency, the profile further delineates that characteristics of the clinical-simulation technologies that can be employed to assess specific competencies (i.e., either high or low fidelity) [13].

Low-fidelity simulation technologies have been defined as having the capacity to replicate “an aspect of a task” [36], e.g., arterial cannulation; therefore, they are commonly referred to as “part-task trainers.” High-fidelity simulation technologies have been defined as having the capacity to recreate “an entire working environment such as the operating theatre” [36]. Linkages have been made among use of specific simulation devices and influences on learning outcomes [37] as well as between competencies learned and practiced in high-fidelity environments and positive transfer between differing levels of simulation and, more importantly, from the simulator to the clinical environment [36]. Simultaneously, calls have been made to increase research emphasis on “techniques used to facilitate learning during simulation” [37] and to explicitly focus attention on the “educational processes that underpin simulator training [to ensure] deliberate practice, reflection and feedback” [36].

Within the 2011 Respiratory Therapy National Competency Profile there is no literature cited, or currently available, to validate the effectiveness of fidelity standards in assuring respiratory therapy educational outcomes, such as competency at the entry-to-practice level [38]. Nevertheless, respiratory therapy educational programs must comply with the minimum standards set by Canadian regulatory bodies to maintain their accreditation standards and to ensure their graduates are eligible for professional licensure upon graduation [34]. It is plausible that these standards play a role in encouraging educational programs to employ clinical simulation for high-stakes assessment of competencies in place of formative assessment techniques traditionally used. Moreover, the distinction between educational approaches (i.e., high vs. low fidelity) and the identification of competencies which may or may not be developed through simulation exemplify some prevailing perspectives on simulation that exist within the profession. The need to examine emerging approaches to simulation-based education within the profession is underscored by these technologically centric perspectives on simulation identifiable both in practice and within the literature.

THE FUTURE OF SIMULATION-BASED EDUCATION IN RESPIRATORY THERAPY

The previous issue of the Canadian Journal of Respiratory Therapy shared the recommendations of a national advisory group on the use of clinical simulation that was struck by the National Alliance of Respiratory Therapy Regulatory Bodies (NARTRB) [38]. The role of the advisory group was to inform the NARTRB on issues relating to attainment and demonstration of competency using clinical simulation, in particular relating to its use to supplement and/or replace clinical practice to attain and demonstrate competence. The working group offered the NARTRB a range of literature- and practice-informed recommendations on the use of clinical simulation [38]. One of the final recommendations of the report speaks to concerns that exist within the community regarding clinical simulation-based assessment practices:

Several environmental factors have been identified as essential in creating an effective debriefing environment in clinical simulation, including: fostering a supportive learning environment, ensuring participants feel comfortable, and establishment of trust within the circle of participants. In light of the importance of fostering a debriefing environment that supports learning, there is value for educators and regulatory bodies to carefully consider that including high-stakes examination of [respiratory therapy] learners may impact those essential environmental factors. We need to ensure that any move towards employing high-stakes examinations in [respiratory therapy] education does not threaten to undermine the effectiveness of the clinical simulation learning environment. [38]

This recommendation illustrates some of the tensions emerging within the discipline’s discourse relative to the use of clinical simulation as an assessment strategy. It challenges policies that may ultimately impact clinical simulation-based environments and encourages the profession’s educational programs to engage in practices that foster learning in these environments.

CONCLUSION

Clinical simulation practice and research are often times characterized by techno-centric perspectives on clinical simulation that is educationally undertheorized. In the past, an organizational culture amongst Canadian respiratory therapy regulatory bodies emerged where the technological aspects of clinical simulation, such as high-fidelity manikins and low-fidelity part-task trainers, were promoted as fundamental tools for achieving learning outcomes. Currently, it is commonplace that clinical simulation is used in the curricula of Canadian respiratory therapy education programs. Moreover, the observation has been made that the use of high-stakes clinical simulation-based assessments that occur throughout respiratory therapy program curriculum are becoming increasingly prevalent.

The literature supports that the use of debriefing as an assessment strategy in the clinical simulation context is associated with improvements in various areas including: learning, clinical and non-clinical skill, future performance, and patient outcomes. As has been explored here, a variety of pressures, including concerns for the safety of patients, limited opportunities for learning in clinical practice, and the professional regulatory environment, are converging to prompt a move towards the use of clinical simulation for high-stakes assessment of learners in entry-to-practice respiratory therapy educational programs. In response, it is incumbent upon the respiratory therapy community to carefully consider how clinical simulation learning designs, including approaches to assessment, may impact environmental factors that are fundamental to learning in this context (e.g., the sense of support, trust, and comfort that participants may experience in clinical simulation). Discourse and reflection on those factors that influence educational practices, including the professional regulatory environment, may prove instrumental to informing future practice.