Evaluation of junior doctors’ retention of knowledge and skills after simulation training in shockable rhythm cardiac arrest in a low-resource setting in Nepal

Abstract

Aims

To test junior doctors’ abilities to retain advanced life support psychomotor skills and theoretical knowledge in management of shockable rhythm cardiac arrest.

Methods

A repeated measure pre-post study design was used with 43 junior doctors, recruited after notifying them with robust method of attraction through flyers, brochures, email and phone calls. Written and performance tests, initial pre-test, immediate post-training, 30-days post-training and 60-days post-training, using simulation-based scenarios with a low-fidelity manikin were used with recording performance of ALS.

Instrumentation

Resuscitation Council UK ALS algorithms and guidelines¹ were used in a simulated testing environment.

Results

There was a highly significant improvement in knowledge immediately after training (p < 0.00), with a net gain of marks from a mean value of 63.2% before training to 87.7% after training by 24.5% (95% CI 19.4, 29.6).

There was a gradual decline of retained knowledge with time from immediate post-training over, 30-days and 60-days post-training (p < 0.00). The simulation pre-training assessments and immediate post-training assessments results were statistically significant (p < .00). The mean difference was 44.1% (95% CI 50.11, 38.10). There was a statistically significant decline of the competency with time (p < .00). Unlike for the knowledge test, the drop was significant on the 30th day (p < .00) with a mean difference of −10.5% (95% CI −13.55, −7.40).

Conclusion

The training of junior doctors in shockable rhythm cardiac arrest in a low resource setting, improved knowledge and skills in the participants after training. However, retention of knowledge declined at 30 days and more significantly after 60 days and retention of skill was declined more significantly at 30 days.

Introduction

Training in ALS in the UK is usually undertaken via Advanced Life Support Training (ALS), Resuscitation Council UK, 2021¹ which is based on models of care from high income countries² with a refresher course in the UK every four years.³ In Nepal and in other low resource settings,² there are no registered ALS training centres, with the cost of a course outside of the country exceeding several months’ salary and acting as a significant barrier in Nepal and in other low resource settings.² When doctors are trained in Advanced Life Support (ALS) and it is applied correctly, it can have an impact on restoring circulation in 40–60% of patients⁴ providing oxygenation and restoring spontaneous circulation and brain function. Advanced Life Support training is an evidence-based proven standard for the management of cardiac arrest restoring life, leading to a return of spontaneous circulation and has a huge impact in increasing the rate of survival to hospital discharge.⁵ Honarmand et al.⁶ found that when clinicians deviated from ALS guidelines, there is greater risk of a lower Return of Spontaneous Circulation (ROSC) and low survival to hospital discharge. Hence, the emphasis on statutory resuscitation training to support adherence to ALS guidelines⁶ is critical to decrease mortality and morbidity. The incidence of in Hospital Cardiac Arrest (IHCA) was found to be higher in lower and middle income countries where survival outcomes were lower than for higher income countries.⁷

The aim of this research was to design, implement and evaluate an accessible ALS simulation based training programme for junior doctors in low resource settings, which would ultimately increase the rate of survival for cardiac arrest patients. Without an accessible ALS course in this low resource setting for most clinicians, the depreciation rate of knowledge and skills was predicted to occur, hence it was a key objective to evaluate the training at 30 and 60 days. A decision to focus on shockable rhythm cardiac arrest was influenced by the higher survival rates of 49% compared to 10.5% in non-shockable rhythm.⁸ The spontaneous circulation and survival to discharge survival rate was higher in shockable rhythm in both high and low income countries.

Methods

This study aimed to analyse the short-term (immediate) and medium term (30 days and 60 days) impact on knowledge, skills and confidence of junior doctors after simulation training in shockable rhythm cardiac arrests in a tertiary hospital in a low-resource setting in Nepal. RS designed and led this research as part of a collaborative four year Hybrid International post graduate Emergency Medicine Fellowship programme (Nepal and UK)⁹ and the research study was approved by the Research Ethics department at the Teaching Hospital (Ethical approval number: 121-076/077).

Design

A prospective, repeated-measure pre-post study, tested participants at four points: (1) pre-training, (2) immediate post-training, (3) 30-days post-training and (4) 60-days post-training.

Sample

Junior Doctors including medical interns, medical officers, first year junior residents and first year junior registrars from the emergency department were invited to attend the training with the distribution of a flyer, brochure, emails and phone calls. Those who were included in the research were a sample of self-selected junior doctors representing the targeted group who consented to the study during an open period of recruitment between February 2022 and March 2022. The study took place between March 2022 and July 2022. All participants provided informed consent.

Method

The setting for this research was an emergency department in a tertiary hospital, about 150 km distance to the east of Kathmandu, with 80–100 patients a day in ED and 60 beds. The cardiac arrest team in the hospital consisted of junior ED doctors, middle grade ED doctors, EM consultant and ED nurses, with senior or more experienced ED doctor in the shift usually taking the team leadership role in a cardiac arrest situation. The crash trolley including drugs, equipments and biphasic defibrillator was available in the Emergency Department. A feasibility study was undertaken in the first year of the four year Fellowship programme which informed the design and submission of the simulation based research (SBR) to the hospital research ethics committee. A second feasibility study was undertaken in year four of the Fellowship programme, after completion of ALS training and Training the Trainers course in the UK by RS and one month prior to the implementation of the research. Dependent variable were knowledge and skills gained after training, retention of knowledge and skills after training and independent variables were gender, age and experience of the participants. Reporting guidelines for health care simulation research were followed and an analysis of the feasibility data informed the final design of the training which was framed within four research phases:

Phase 1: Pre-test phase

A pre-test phase involved the lead trainer (RS), who managed the executation of tests and an EM consultant¹ whose role was to reduce the risk of bias by distributing the live simulation test assessments and to inviligate the completion of these tests. A pre-training knowledge assessment (with 15 questions) was carried out using single based answer (SBA) questions from a bank of questions based on UK Resuscitation Council 2021¹ guidance. Each question had five options among which one was the best answer and it covered questions on chest compression, drugs, energy, chain of survival, initial evaluation, rhythm and effective resuscitation team.⁸ Secondly each participant was called into a simulation room, where individually they were asked to lead a simulation based scenario involving a patient in cardiac arrest with shockable rhythm. Low-fidelity manikins, airway adjuncts, breathing adjuncts, circulation adjuncts, and defibrillator were used and supported by three volunteer team members, who were briefed about what role is expected of them in the simulation. The team members were a mix of junior and middle grade doctors who were briefed only to act as per the participant’s intstructions and leadership. A simulation checklist was used which contained twenty-five actions informed by UK Resusitation Council guidance on shockable cardiac arrest.¹

Phase 2: Intervention

Immediately after the pre- training knowledge assessment and simulation based scenario, the educational materials and videos of cardiac arrest management from Resuscitation Council UK, the links of guidelines and websites were all sent to participants via email. Participants were divided randomly into five groups and on Day 1, were given educational theoretical sessions about the cardiac arrest followed by simulation with shockable rhythm (use of defibrillator) demonstration session for an hour. The sessions were led by RS and supervised by a second Emergency Medicine consultant. This was followed by Day 2, which immediately followed day 1 and involved a six-hour simulation workshop (led by RS and supervised by the second EM consultant) where each participant was given a chance to lead a simulation based scenario which lasted for about 20–30 minutes consisting of details of scenario including age, gender of the patient and circumstances leading to shockable cardiac arrest and be a team member in other team colleagues scenarios. The simulation based scenarios included debriefing using Pendleton method at the end of each simulation led by RS which lasted for about 5–10 minutes.

Phase 3: Immediate post-training test

A post training knowledge assessment was provided and invigilation was undertaken by a third Emergency Medicine consultant, using a single based answer (SBA) knowledge questionnaire containing fifteen questions based on guidance from the UK Resuscitation Council 2021.¹ The questionnaire covered similar questions to the pre-training phase which were selected from the question bank and related to: chest compression, drugs, energy, chain of survival, initial evaluation, rhythm and effective resuscitation team⁸ and arranged in a different order. Secondly each participant was called into a simulation room, where they were asked to individually lead a simulation based scenario involving a patient in cardiac arrest. Low-fidelity manikins, airway adjuncts, breathing adjuncts, circulation adjuncts and defibrillator were used and three volunteer team members provided assistance. A practical simulation test was led by RS and supported by the third emergency medicine consultant, who assessed participants against a simulation checklist form. The checklist contained twenty-five actions informed by UK Resusitation Council guidance on shockable cardiac arrest.⁸

Phase 3 & 4: 30 and 60 days post-training

Participants were called via email and phone calls for taking the 30 and 60-days post-training knowledge test. They were individually called for simulation tests, where they were assessed by the third emergency medicine consultant. The simulation tests were conducted on the same day or the following day of the 30 and 60 day questionnaire, dependant on the availability of each participant.

Data analysis

For Phase 1–4, data were entered in Microsoft Excel 2020. All paired comparisons before and after training were done using the paired t-test, Wilcoxon Signed-Rank Test and Friedmann test, as required.

Results

Demographics

A total of 43 junior doctors participated in the study. Participants ranged from age of 22 to 36 years with a mean age of 27.3 (SD = 2.94) years, with 30 (69.8%) men and 13 (30.2%) women doctors participating in the research (p =.01). The largest group was medical interns (19, 44.2%), followed by house officers (10, 23%), junior General Practice and Emergency Medicine (GP-EM) residents (5, 11%) and junior Emergency Medcine (EM) registrars (9, 20%). Work experience of them managing CPR is given in Table 1.

Table 1.

Characteristics and experience of the participants.

Aspect	Stratification	Number	Percentage
Experience as a Doctor (years)	<0.5	18	41.9
	0.5–1	8	18.6
	1–2	3	7.0
	>2	14	32.6
No of cardiac arrest managed as a team member	0	9	20.9
	1–5	14	32.6
	6–10	5	11.6
	>10	15	34.9
No of cardiac arrest managed as a team leader	0	26	60.5
	1–5	11	25.6
	6–10	5	11.6
	>10	1	2.3
Previous ALS Course	Taken	6	14.0
	Not taken	37	86.0
Time since previous ALS course	<6 months	2	33.3
	6–12 months	2	33.3
	>12 months	2	33.3
Challenges for taking the ALS course	Unaware	20	46.5
	Inaccessible	19	44.2
	Expensive	21	48.8
	Unsupportive employer	1	2.3
	Uninterested	0	0

Assessment results

Marks scored in each session are shown in Table 2, Fig. 1, Fig. 2. The expected standard cut off point was put as 90%, as a high standard universal grade of assessment score.¹⁰

Table 2.

Above 90% scorers.

Timing of the Assessment	Total Number of Participants	Knowledge Assessment		Simulation Assessment
		Number	Percentage	Number	Percentage
Pre-Training	43	2	4.6	1	2.3
Immediate Post-training	43	19	44.2	29	67.4
30-Days Post-training	41	16	39.0	9	22.0
60-Days Post-training	40	7	17.5	18	45.0

Fig. 1.

Box plot of knowledge score.

Fig. 2.

Box plot of simulation scores.

Knowledge

There was a statistically highly significant improvement in knowledge immediately after training (p < 0.00), with a net gain of marks from a mean value of 63.2% before training to 87.7% after training (Fig. 1) by 24.5% (95% CI 19.4, 29.6).

There was a gradual decline of retained knowledge with time from immediate post-training over, 30-days and 60-days post-training (p < 0.00) (Fig. 1). However, this decline was most noticeable only after 60 days of gaining knowledge (p < 0.00). Although there was a slight decline of retention on the 30th day this was not statistically significant (p = 0.20). However, the difference between the 30th day and the 60th day was statistically significant (p = 0.01).

Age did not predict pre-training knowledge [R² = 0.039, F (1, 42) = 1.66, p =.21] and immediate post-training knowledge [R² = 0.005, F (1, 42) = 0.20, p =.66]. Interestingly, none of gender, designation, experience as a doctor, previous experience of managing cardiac arrest as a team leader or member had a statistically significant impact on the pre-existing knowledge and immediate post-training knowledge of the participants (Table 3).

Table 3.

Pre-training and immediate post-training knowledge and simulation scores vs demographic data.

Variable	Category	Pre-Training		Immediate Post-Training
		Median (Z)	p-value	Median (Z)	p-value
KNOWLEDGE
Gender	Male	67	0.10	90	0.59
	Female	53		87
Designation	EM Junior Registrar	73	0.06	87	0.17
	GP-EM Junior Resident	67		87
	Medical Intern	60		87
	Medical Officer	60		93
Experience as a doctor	<6 months	60	0.12	90	0.37
	6–12 months	53		93
	1–2 years	60		93
	>2 years	73		87
No of Cardio-Respiratory Arrests managed as a Team Member	None	60	0.86	93	0.38
	1–5	67		87
	5–10	67		87
	>10	67		87
No of Cardio-Respiratory Arrest managed as a Team Leader	None	67	0.21	93	0.26
	1––5	60		87
	5–10	73		93
	>10	77		70
Whether taken Adult ALS course before	No	67	0.62	93	0.31
	Yes	70		87
SIMULATION SKILLS
Gender	Male	50	0.18	98	0.51
	Female	44		96
Designation	EM Junior Registrar	72 (2.97)	0.00	100	0.05
	GP-EM Junior Resident	68 (1.76)		92
	Medical Intern	44 (−1.69)		92
	Medical Officer	36 (−2.21)		98
Experience as a doctor	<6 months	42 (−2.02)	0.00	90	0.13
	6–12 months	36 (−2.25)		100
	1–2 years	68 (0.38)		96
	>2 years	70 (3.78)		100
No of Cardio-Respiratory Arrests managed as a Team Member	None	44 (−1.51)	0.02	88 (−2.54)	0.04
	1–5	40 (−1.98)		98 (0.10)
	5–10	72 (1.53)		100 (1.27)
	>10	68 (2.20)		100 (1.21)
No of Cardio-Respiratory Arrest managed as a Team Leader	None	42 (−3.15)	0.01	94	0.71
	1–5	60 (0.95)		96
	5–10	76 (2.61)		100
	>10	72 (1.44)		94
Whether taken Adult ALS course before	No	48	0.90	96	0.33
	Yes	52		100

Simulation

The pre-training skill assessments and immediate post-training assessments results were statistically significant (p < .00) (Fig. 2). The mean difference was 44.1% (95% CI 50.1, 38.1).

There was a statistically significant decline of the competency with time (p < .00). Unlike for the knowledge test, the drop was significant on the 30th day (p < .00) with a mean difference of −10.5% (95% CI −13.5, −7.4).

The same was true for immediate post-training and 60-days post-training simulation assessments (p < 0.00) showing a drop of −8.5%, 95% CI(−11.8, −5.2). The slightly positive increase between the 30-days post-training and 60-days post-training simulation assessment was not statistically significant (p = 0.18).

Age did not predict pre-training simulation skills [R² = 0.085, F(1, 42) = 3.81, p =.06] and immediate post-training simulation skills [R² = 0.002, F(1, 42) = 0.07, p =.79]. Gender and previous ALS course experience didn't have impact whereas designation, experience as a doctor, previous experience of managing cardiac arrest as a team leader or member had a statistically significant impact on the pre-existing knowledge and immediate pre-training simulation score of the participants (Table 3).

Discussion

A simulation based training programme in a low resource hospital setting increased doctors knowledge and performance in shockable rhythm cardiac arrest. While this training was effective, even by 30 days post-training, there was a loss of skill and knowledge which is consistent with other studies. Problems accessing ALS training in low resource settings is a global problem and doctors working in low resource setting are unlikely to be ALS trained, with 86% in this study not having any previous access to an ALS course. This is higher than other countries, for example in India it is 76%,¹¹ Tanzania 44%,¹² and Nigeria 69%.¹³ The low level of access to ALS courses by junior doctors in low resource settings is a high risk factor for patient safety and well designed training is required in this area.

This study showed that baseline knowledge of junior doctors was low at 63% compared to the expected 90%. The physicians started with low score, but they improved in knowledge after the training compared to pre-training scores. There was significant gain in knowledge from 63% to 87% post training although there was still room for improvement. Knowledge started to decline at 30-days post training assessment (Fig. 1) and further decreased at 60-days post-training. Analysis of pre-training and immediate post-training assessments and 60-days post-training results were statistically significant while 30-days post-training was not. This analysis suggests a need for further research with a larger sample to identify the optimum assessment of knowledge at both immediate, 30 and 60 days, while also extending assessment to 90 days days post-training. Immediately after training, more than 40% of the doctors scored above 90%, whereas by the 60th day this dropped to less than 20% of doctors.

In simulation, the pre-training simulation score was low at 50%. The scores were lower than results found by Morgan and Westerland,¹⁴ where 67% of the doctors felt competent, but higher than in another study,¹⁵ where only 46% passed the pre skill assessment test. This research found that skills increased significantly after the training 94%. The pre-training score and post intervention training score were higher than study done by Lin et al.¹⁶ in 2018. However, the skills declined subsequently at 30 and 60-days post-training. The scores in 30-days and 60-days post-training assessments still remained higher than the simulation pre-training assessments. Simulation training in shockable rhythm declined less than the decline in knowledge at 60 days, which is the opposite finding to a literature review which find skills decline faster than knowledge.¹⁷ This finding could relate to the high level of vocational learning with mentoring in the work place due to the lack of access to ALS training courses. These findings suggest there is a need to design ALS training that is best suited for the context of a low resource setting such as Nepal, rather than to invest in ALS training designed for high income countries.

The data analysis concluded that pre-training and immediate post-training simulation test was statistically significant, so was immediate post-training and 30-days and 60-days post-training. However, the 30-days and 60-days post-training wasn't statistically significant.

Junior doctors are an integral part of the resuscitation team in cardiac arrest11, 14 thereby requiring appropriate skills and knowledge to recognize and treat a respiratory or cardiac arrest.18, 19 However, most house-officers report not having received any formal training in life support and are not adequately skilled in ALS.¹⁴ Access to ALS training is not just a problem for low resource settings as previous research conducted in the UK¹⁴ found that 49% of junior doctors who were part of a resuscitation team had not received ALS training. Despite the poor knowledge level in cardiopulmonary resuscitation amongst junior doctors, training them with a well structured training programme can increase knowledge on resuscitation and management of cardiac arrest.10, 14 Junior doctors should undergo regular CPR training every 6 months in order to maintain practical CPR skills.14, 19, 20 Without refresher training at 6 months–1 year¹⁵ skills and knowledge will decline, with skills declining faster than knowledge.¹⁷ This research shows a promising outcome for developing high quality, rigorous ALS training suited to low resource settings, where economic access to ALS training continues to be a major barrier.

Limitations

The study included a small number of self selecting participants on to an ALS training programme in a low resource setting. This could have contributed to results bias as those selecting to attend were motivated and engaged to learn. The setting was a single centre study which may not reflect knowledge and performance of doctors in other hospitals. While the lead researcher and trainer consistently led and attended all phases of the research, the pre - and post-training assessors were different which might have contributed to observer bias. However all assessors were tasked with invigilating and ensuring compliance with testing and there was no variation in this role observed.

Conclusion

The training of junior doctors in shockable rhythm cardiac arrest in a low resource setting, improves knowledge and skills in the participants after training, however retention of knowledge declines at 30 and 60 days. While the aim of this research set out to measure depreciation of knowledge and skills at 30 and 60 days, the analysis found that there was no statistical difference in post training and 30 days results and therefore a longer repeated study at post training, 60 and 90 days should be planned. A future study should correlate training outcomes with longitudinal data measuring survival rates of shockable and non shockable cardiac arrest for patients in the emergency department.

Funding source

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CRediT authorship contribution statement

Rojina Shrestha: Conceptualization, Methodology, Investigation, Data curation, Writing – original draft, Visualization, Project administration. Buddhike Sri Harsha Indrasena: Formal analysis, Software. Prakash Subedi: Conceptualization, Methodology, Writing – original draft. Dayaram Lamsal: Project administration, Investigation, Supervision. Chris Moulton: Writing – review & editing. Jill Aylott: Conceptualization, Methodology, Validation, Writing – review & editing, Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.