ASPiH Conference 2019 keynote paper. Quality improvement through simulation: a missed opportunity?

Abstract

As the use of simulation has become more established in the delivery of healthcare education and training, there has been a corresponding increase in healthcare simulation research. Simulation-based research can be divided into research about simulation (answers research questions in which the focus is on simulation itself) and research through simulation (simulation as a method/tool for research). However, there are barriers, particularly for smaller less well-resourced simulation centres, that may prohibit participation in research. Therefore, it is suggested that quality improvement (QI) through simulation may be a pragmatic way in which simulation centres of all sizes can contribute to improving patient care beyond education and training. QI is defined as systematic, data-guided activities designed to bring about immediate, positive changes in the delivery of healthcare. Although not the case in healthcare, other industries routinely used simulation to support QI. For example, in aviation simulation is used to inform the design of the working environment, the appropriate use of technology, to exercise emergency procedures and to ‘re-fly’ flights following an adverse event as part of the mishap investigation. Integrating simulation within healthcare QI can support the development of novel interventions as well helping to address heretofore intractable issues.

Introduction

As the use of simulation has become more established in the delivery of healthcare education and training, there has been a corresponding increase in healthcare simulation research.^1–3 Research is identified as one of the three core standards by the Society for Simulation in Healthcare (the other core standards are assessment and teaching). This recognises the potential of research to improving patient safety through the use of simulation technologies and methods.⁴ However, there are barriers, particularly for smaller less well-resourced simulation centres, that may prohibit participation in research. This paper will describe the broad types of simulation-based research, identify the major barriers to simulation research, provide a brief overview of quality improvement (QI), and suggest that simulation though QI may be a pragmatic way in which simulation centres of all sizes can contribute to improving patient care beyond education and training.

Simulation-based research

Simulation-based research can be divided into two broad types:

• Research about simulation: answers research questions in which the focus is on simulation itself (eg, evaluating the efficacy of simulation-based training).

• Research through simulation: the focus of this research is not simulation itself, but rather uses simulation as a method/tool for research (eg, evaluating the effectiveness of different alarm frequencies in a simulated clinical environment).⁵

Research about simulation dominates the healthcare simulation research literature.³ In a bibliometric review of the 100 most cited articles in healthcare simulation, the majority of studies (86%) were concerned with education and training, with 28% concerned with evaluating the impact of simulation-based interventions.^{1 3 5} When used under the right conditions, there is now sufficient evidence that healthcare simulation is an effective educational intervention.^{1 3} Therefore, future research about simulation must shift the focus from questions of ‘does healthcare simulation work?’ to more nuanced questions such as ‘under what conditions is simulation most effective?’, or ‘how can simulation-based interventions be scaled across multiple sites?’ Answering these nuanced research questions requires well-designed and supported studies.

In the healthcare industry, research through simulation is considerably less common that research about simulation.⁵ However, this is the opposite in other industries in which the use of simulation is used for education and training. In industries such as aviation, research through simulation dominates. Flight simulators have been used to answer a range of research questions. For example, what is the impact of workload on pilot performance⁶?; what is the effect of pilot fatigue on performance⁷?; or how do pilots react to flight deck alerts⁸? Therefore, in agreement with other authors, the “potential of simulation for conducting research has remained underexploited”.⁵ However, irrespective of the type of simulation-based research planned, the enterprise of research itself can be fraught with challenges.

General Data Protection Regulation (GDPR) means that accessing research participants and data for research has become increasingly challenging. Ethics approval is also more difficult to achieve due to GDPR requirements and also because the ethical standards for research have become increasingly stringent. Arguably, a greater focus on protecting research participants, and their data, is long overdue. However, the result of this focus is that obtaining ethical approval, and addressing data protection concerns, is becoming increasingly time consuming and burdensome. In a survey of 42 simulation centres from across the world, only 26 (62%) reported research activities related to simulation. Moreover, it was found that 98% of all research activities originated from only six major centres.⁹ However, despite the challenges involved in undertaking simulation research, there are still great opportunities for simulation centres to make meaningful contribution to patient safety by simulation through QI. It is postulated that QI allows for a quicker, and potentially more impactful, way to realise changes to healthcare systems and processes than research.

Quality improvement

QI has been defined as ‘systematic, data-guided activities designed to bring about immediate, positive changes in the delivery of healthcare’.¹⁰ Evidence of poor care experiences and patient harm have prompted the growth of QI initiatives in healthcare.¹¹ A QI approach considers processes within a healthcare system in order to identify variation, and then implement change based on testing different approaches to achieve the desired outcome.¹²

There are many different models to support QI. However, they all tend to align with the scientific experimental method.¹³ An increasingly common approach to QI is the plan–do–study–act cycle model for improvement. These four cycles align with the experimental method of developing a hypothesis (plan), implementing an intervention or change to effect an outcome (do), collecting data to test the effects of the change on the outcome (study) and analysing the data in order to make inferences to allow changes to be made to the hypothesis (act).¹³ This model for QI enables rapid assessment of an intervention and provides flexibility to quickly make changes based upon feedback.^{14 15}

Often QI studies do not require review by an ethics review board, as they are not considered human subjects research. However, it is important to indicate that research and QI are not necessarily mutually exclusive. Research is designed to develop or contribute to generalisable knowledge.¹⁰ Although the focus of a QI study may be on improving patient care in a specific hospital, the findings may also have implications for other hospitals. For example, a QI study designed to bring about local change (eg, use of simulation to assess level of alertness at during different shift patterns) is likely to have implications for other hospitals. Therefore, a determination needs to be made as to whether a study is research or QI. Also, the fact that an ethical review is not required does not mean that care should not be taken to ensure the ethical conduct of a QI study, and many hospitals have committees in place specifically to monitor the conduct of QI studies.¹⁶

Just as is the case for reporting traditional simulation-based research,² there are specific reporting guidelines for QI studies called SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence).¹⁷ As might be expected, the SQUIRE 2.0 reporting guidelines have less of a focus on research issues (eg, a priori sample size calculations, randomisation, statistical methodologies). Instead, SQUIRE 2.0 is focused on reporting the impact of an intervention at a local site. Also, although a study may be considered QI, and not research, this does not preclude it from publication. As can be seen from the references included in the next section, journals are increasingly willing to publish well-conducted QI studies.

QI through simulation

Use of simulation to initiate or evaluate QI initiatives is consistent with the assertion that an important role for simulation is to contribute to learning within healthcare systems.¹⁸ A learning healthcare system can be defined as ‘one in which science, information, incentives, and culture are aligned for continuous improvement and innovation, with best practices seamlessly embedded… and new knowledge captured as an integral by-product of the care experience’.¹⁹ In the aviation industry, simulation is used to inform the design of the working environment, the appropriate use of technology, to exercise emergency procedures and to ‘re-fly’ flights in the simulation following an adverse event as part of the mishap investigation.²⁰

Simulation can be used to investigate a range of performance shaping factors. These attributes include the individual (eg, stress), teams (eg, communication), technology (eg, usability), the work environment (eg, interruptions) and systems (eg, workflow). Simulation may be applied to QI in a number of ways, such as examining patient flow processes within healthcare facilities, examining the physical environment of a healthcare facility, supporting the conduct of Failure Modes and Effects Analysis, recreating adverse events to promote learning and prevention,^{12 21} testing new equipment prior to introduced in the clinical setting^{12 22} and testing the emergency preparedness of a hospital.²³ The effective medical response to the Boston Marathon bombing in 2013 and the Paris terrorist attacks in 2015 were at least partially attributed to the simulation-based training and preparedness at the hospitals where the injured were received.^{24 25}

There is also the potential for patient inclusion in the simulation activities. For example, patients could be involved in simulated exercises, and participate in the subsequent debrief to ensure that their thoughts, experiences and opinions are captured and reflected in plans for organisational change.¹² Positive impacts of applying simulation within QI demonstrated within the small body of extant work have included improved safety (eg, reduced falls risk to patient), improved efficiency (eg, factors which inhibit efficient staff workflow addressed) and improved patient experience (eg, a wheelchair at reception for patients).¹²

Conclusion

Most simulation centres were established to support front line staff to deliver patient care through the provision of education and training. This means that these centres may not be resourced to carry out research about simulation. However, healthcare simulation is an under-used tool that has enormous potential to also support the improvement of care by research or QI through simulation. Integrating simulation within QI in particular can support the development of novel interventions as well helping to address heretofore intractable issues.