Auditing simulation practice against ASPiH standards: an experience for a district general hospital

Introduction

Simulation enhanced learning (SEL) provides unique medical education by allowing the application of theory in a realistic environment.¹ In 2016, the Association for Simulated Practice in Healthcare (ASPiH) developed 21 standards with corresponding guidance as a framework for centres to refer to during simulation design.² In 2018, ASiPH announced an accreditation process for centres complying with their standards.³ ASPiH claim that through accreditation, facilities will benefit in terms of simulation quality assurance, organisational networking, formalisation of the simulation process and improved meeting of simulation stakeholders’ needs.⁴ Furthermore, patronage will serve as a marker of excellence and help secure funding and resources for the simulation department. Thus, we audited the simulation program at Princess Alexandra Hospital (PAH) National Health Service Trust to assess both adherence with these standards and the usefulness of the ASPiH standards as a method to improve the quality of SEL.

Methodology

Currently, there is no validated tool to assess ASPiH standard compliance. Therefore, we developed an audit tool to assist with this process (online Supplementary appendix 1). To assess whether a standard had been achieved, we developed criteria using the corresponding recommendations present in the guidance document. Some standards, such as standard 6, ‘the patient’s perspective is considered and demonstrated within educational planning’, were explicit enough to be a single criterion standard. Single criterion standards could be deemed complete or not complete depending on whether that criterion was achieved. Broader standards had multiple criteria depending on the amount of published guidance. For example, standard 1 ‘Faculty ensure that a safe learning environment is maintained for learners and encourages self-reflection on learning’ was composed of three criteria: (1) clear ground rules set up prior to starting any simulation regarding confidentiality and the need for the environment to be open and honest; (2) the simulation rooms are checked to be safe and free from hazards and (3) faculty encourage self-reflection on the simulated experience. Standards in which there were multiple criteria could be deemed complete, partially complete or not complete if all, some or none of the criteria were achieved, respectively. Furthermore, some standards were specific to particular forms of SEL such as in situ simulation (standards 12, 13 and 14) and simulated patients (standards 15 and 18). If the simulation was not in situ or did not involve simulated patients that standard would be deemed not applicable (N/A). The newly designed audit tool was used to assess all sessions delivered by PAH medical education centre within a 6-week period, eventually resulting in three audited sessions. One session was an in situ obstetric simulation, while two sessions were delivered in the simulation suite.

Results

Across the three sessions, the mean average for fully completed standards was 7.33, range (5–10), for partially completed standards 7.33, range (6–8), for not completed standards 3.67, range (1–4) and for not applicable 3.67, range (2–5). The full data set is demonstrated in table 1.

Table 1.

Audited simulation results

Standard heading	Standard number	Simulation sessions			Completed	Partially completed	Not completed	N/A
		SH4FT	Obstetric	Nursing perceptorship
Faculty	1	C	P	C	2	1	0	0
	2	C	P	P	1	2	0	0
	3	P	P	P	0	3	0	0
Technical personal	4	P	P	P	0	3	0	0
Activity—programme	5	C	P	P	1	2	0	0
	6	N	C	N	1	0	2	0
	7	P	P	P	0	3	0	0
	8	P	P	P	0	3	0	0
Activity—assessment	9	C	C	C	3	0	0	0
	10	P	P	P	0	3	0	0
	11	C	C	C	3	0	0	0
Activity—ISS	12	N/A	C	N/A	1	0	0	2
	13	N/A	C	N/A	1	0	0	2
	14	N/A	C	N/A	1	0	0	2
Resource—SFT	15	N	N/A	N/A	0	0	0	3
	16	C	C	N	2	1	0	0
	17	C	C	C	3	0	0	0
	18	N/A	N/A	N/A	0	0	0	3
Resource—MLD	19	N	C	C	2	0	1	0
	20	N	N	N	0	0	3	0
	21	P	C	P	1	2	0	0

	SH4FT	Obstetric	Nursing perceptorship	Average
Total completed	7	10	5	7.33
Total partially completed	6	8	8	7.33
Total not completed	4	1	3	3.67
Total N/A	4	2	5	3.67

Summary of collected data from audited simulations.

C, is the standard in which all criteria have been completed; P is the standard in which the criteria have been partially completed; N is the standard in which none of the criteria have been completed and N/A is the standard which is not applicable to the audited simulation.

ISS, in situ simulation; MLD, management, leadership, development; SFT, simulation, facility, technologies; SH4FT, Simulation at harlow for foundation trainees.

Discussion

The results demonstrated a good foundation for future development of SEL. The key outcome from the study highlighted that rebranding of the PAH simulation centre to include a clear mission statement and formalisation of simulation design procedures would be key to both increase compliance with ASPiH standards and improve the quality and consistency of the SEL overall. This was apparent as standards within the ‘resources—management, leadership and development’ theme (standards 19, 20 and 21) were most consistently not achieved. By considering ASPiH standards during simulation design (standards 6 and 10), a more holistic simulation experience could be developed for learners in which patients’ perspective as well as the participants own psychology could be explored more consistently. Post-session debriefing, if employed, would enable reflection on the SEL experience from a faculty perspective and enable future improvement of that session. Furthermore, the assignment of a named individual with organisational influence that could champion the department at senior meetings and procure stable funding would aid in the development of a more sustainable department.

There were two key limiting factors to the study. First, due to several last-minute cancellations of planned sessions only three were audited in the time-period allocated. Often this was because the hospital department requesting a simulation session could not release enough staff from a shift and still maintain safe staffing numbers. It was also a frequent occurrence that technical personnel delivering the simulation were not provided with guidance from the requesting department until the day before the simulation, if at all. This resulted in rushed, last-minute preparations, poorly aimed at the stakeholders needs, only for the simulation to be cancelled the next day. This turned out to be a substantial waste of educational resources and be preventable should, through accreditation, PAH rebrand its simulation centre and implement a formalised procedure for simulation session procurement by an interested department. In addition, in view of applying for accreditation, the auditing process stimulated reflection within the department that highlighted multiple possible avenues for improvement which were not previously considered. Having a sample size of three limits the ability of the study to highlight exact areas for improvement. A larger sample would more accurately identify areas for improvement as well as stimulate more departmental reflection.

The ASPiH standards and guidance were initially designed to be broad enough to apply to multiple simulation programmes. However, it was difficult to gauge whether a standard had been achieved. More explicit guidance from ASPiH would aid centres in assessing their own adherence prior to undertaking the accreditation process.

Conclusion

In conclusion, the ASPiH standards provide the first framework to assess the quality of simulation, formalised by the ASPiH accreditation process. It is an excellent first step in standardising SEL on a national scale to ensure consistent high-quality SEL across multiple institutions. Within PAH, rebranding of the simulation centre involving formalisation of procedures would both increase adherence with ASPiH standards and decrease waste of educational resources. However, further advice from ASPiH would assist institutions to assess current standard adherence prior to starting the accreditation application.