A Simulation-based PPE orientation training curriculum for novice physicians

Summary

Background

Personal protective equipment (PPE) is effective in preventing coronavirus disease (COVID-19) infection. Resident knowledge of proper use and effective training methods is unknown. We hypothesise that contamination decreases and knowledge increases after a formalised PPE educational session.

Methods

Participants included first year interns during their residency orientation in June 2020. Before training, participants took a knowledge test, donned PPE, performed a simulated resuscitation, and doffed. A standardised simulation-based PPE training of the donning and doffing protocol was conducted, and the process repeated. Topical non-toxic highlighter tracing fluid was applied to manikins prior to each simulation. After doffing, areas of contamination, defined as discrete fluorescent areas on participants' body, was evaluated by ultraviolet light. Donning and doffing were video recorded and asynchronously rated by two emergency medicine (EM) physicians using a modified Centers for Disease Control and Prevention (CDC) protocol. The primary outcome was PPE training effectiveness defined by contamination and adherence to CDC sequence.

Results

Forty-eight residents participated: 24 internal medicine, 12 general surgery, 6 EM, 3 neurology, and 3 psychiatry. Before training, 81% of residents were contaminated after doffing; 17% were contaminated after training (P<0.001). The most common contamination area was the wrist (50% pre-training vs. 10% post-training, P<0.001). Donning sequence adherence improved (52% vs. 98%, P<0.001), as did doffing (46% vs. 85%, P<0.001). Participant knowledge improved (62%–87%, P <0.001). Participant confidence (P<0.001) and preparedness (P<0.001) regarding using PPE increased with training.

Conclusion

A simulation-based training improved resident knowledge and performance using PPE.

Introduction

Due to the bedside nature of the profession, physicians and other healthcare workers are at an increased risk of occupational exposure to communicable diseases [1]. This has become especially important during the SARS-CoV-2 pandemic where hundreds of healthcare workers have died [2]. Unfortunately, the growth of the world population, urbanisation, and international trade has increased the risk of global spread of infectious disease likely making it easier for pandemics to evolve [3]. As such, an important aspect in pandemic responses will be the effective use of personal protective equipment (PPE) by healthcare workers. Despite this, there is evidence that suggests that many physicians do not have formal training in proper use of PPE [4].

There is limited published research on formal effective curricula to educate medical students prior to exposure to hazards in the clinical environment of residency training [5]. The core focus of medical school curricula is to prepare the graduate for medical practice. However, an assessment has shown the majority of medical students have inadequate, incomplete, or non-existent PPE training [6]. This lack of PPE training for recent healthcare graduates is evident across many different medical education settings globally [7]. We describe a simulation-based method for PPE training that can be easily implemented in quality improvement programs. Our study evaluates PPE training effectiveness defined by contamination and adherence to Centers for Disease Control and Prevention (CDC) sequence.

Methods

Study design, participants and setting

This study was a prospective observational study of quality improvement measures implemented in residency orientation training in June 2020. First year resident physicians who needed American Heart Association (AHA) life support certifications (advanced cardiac life support, basic life support, etc.) were enrolled. There were three distinct phases to this process: pre-training, training, and post-training (Figure 1). All phases of the study were conducted at a single university simulation center. The study was determined not to be human subjects research by the university's institutional review board.

Figure 1.

Phases of training.

In the first phase, participants were asked to fill out a survey to establish baseline knowledge of PPE usage and previous training in medical school. Next, participants were asked to don PPE in the manner they felt was most effective while being video recorded. Participants then performed cardiopulmonary resuscitation (CPR) and positive pressure ventilation using a bag valve mask (BVM) on manikins (unmodified CAE iStan, CAE Healthcare INC, Sarasota, FL). Topical non-toxic invisible highlighter fluid was used as a tracer [8], which was applied with equal amounts distributed on the face, neck, and chest. After performing two rounds of compressions and one round of ventilations, participants were asked to doff PPE while being video recorded. Participants were then examined under long wave ultraviolet (UV) light for evidence of the tracer, indicating areas of contamination (Figure 2).

Figure 2.

Ultraviolet light assessment of contamination.

The second phase consisted of a standardised 30-minute formative training program utilising modified CDC guidelines on donning and doffing procedures of PPE. Participants were asked to reflect on their performance during the simulated code and give their opinion why they may have had contamination. The CDC training program included a step-by-step simulation-based review of proper PPE the donning and doffing sequence with deliberate practice and feedback (Supplement 1).

The final phase was similar to the first. Participants were asked to don PPE, perform compressions and ventilations, and then doff their PPE prior to examination for contamination under long-wave UV light. Participants then completed a post-training survey including a knowledge test and a curricular evaluation.

Measurements and outcomes

The primary study outcome was to evaluate PPE training effectiveness defined by contamination and adherence to CDC sequence. The secondary outcomes were to assess participant knowledge on PPE usage and isolation precaution types and to evaluate participant perceptions of the training and use of fluorescent tracer to visualise errors.

There were three sets of data gathered during this study: participant pre- and post-training survey data, contamination data, and video data of donning and doffing procedures for participants before and after training.

Survey data included questions on previous PPE education, precaution types, previous clinical experience, pre- and post-training knowledge on PPE usage, as well as confidence with PPE utilisation (Supplement 2).

Contamination was defined as discrete areas of fluorescence separated by greater than 3 cm. Two independent assessors were used to judge contamination. Discrepancies were discussed and adjudicated and a third independent observers was involved, if necessary. The number of contaminated areas was recorded for each specific body area, including face, neck, chest, arms, forearms, wrists, hands, and legs.

Video data was scored by two of five randomly assigned independent raters, all emergency medicine attending physicians, using a summarised scoring sheet of CDC PPE recommendations (Supplement 3). Each rater assessed performance of specific and defined aspects of PPE donning or doffing. Donning was scored from one to five points and doffing was scored from one to seven points. The evaluators completed a video training session and inter-rater reliability was assessed using intraclass correlation coefficient (ICC). Final ICC was 0.96, indicating excellent inter-rater reliability. Data management of all measurements were preformed using spreadsheet software (Microsoft Excel, Microsoft Corporation, Redmond, WA).

Analysis

General summary statistics were performed on participant survey data. Descriptive statistics on overall and specific body part contamination were performed and compared for participants before and after training. Donning and doffing scores were compared with respect to training and comparative categorical univariate statistics. Descriptive and univariate statistics were performed using Stata Release 16 (StataCorp, College Station, TX) and SPSS Statistics for Windows, version 27 (IBM Corp., Armonk, NY).

Results

A total of 48 residents completed the training course: 24 internal medicine, 12 general surgery, 6 emergency medicine, 3 neurology, and 3 psychiatry.

Survey

Survey data revealed that 71% of participants had not undergone any PPE training. Only 23% had hands-on general PPE training in addition to either video-based content training, written, or didactic based training. Only 23% of participants had any COVID-19 specific PPE training and only 6% had any clinical experience with COVID-19 patients. Many participants stopped clinical rotations in March 2020 (46%) when they were medical students. Furthermore, only 15% of participants had a rotation in the past three months prior to residency. Of note, 15% had not had clinical exposure in the past six months prior to residency.48% of residents had never used PPE in that time period and 79% of those participants donned and doffed PPE less than 10 times.

Overall participant knowledge scores increased after training (62% vs. 87%; difference 25%; 95% CI: 19%–31%; P<0.001). Knowledge of doffing sequence significantly increased, but no significant change was noted in donning sequence. Participant knowledge of transmission of tuberculosis, chickenpox, SARS, Norwalk virus and West Nile virus improved after the education, but no significant change in COVID-19 (Table I).

Table I.

Knowledge Questions Results Summary, mean (95% CI)

Knowledge question	Pre-training correct	Post-training correct	Difference	P-value
Donning sequence	81%	92%	10% (-3%–24%)	0.133
Doffing sequence	52%	94%	42% (27%–56%)	<0.001
Tuberculosis transmission	75%	98%	23% (11%–35%)	0.001
Chickenpox transmission	27%	54%	27% (12%–43%)	0.001
SARS transmission	69%	94%	25% (10%–40%)	0.002
Norwalk virus transmission	50%	90%	40% (23%–56%)	<0.001
West Nile virus transmission	75%	94%	19% (6%–32%)	0.005
COVID-19 transmission	67%	81%	15% (-3%–33%)	0.109
Overall score	62%	87%	25% (19%–31%)	<0.001

Participant confidence in donning and doffing PPE safely and preparedness to appropriately use PPE when caring for COVID-19 patients significantly increased after training (Table II). The post-training survey data showed that 44% felt that practicing donning and doffing was one of the most beneficial parts of the training while 46% of people felt the most beneficial aspect was the use of a fluorescent tracer to visualise errors. 98% of participants felt the use of fluorescent tracer was helpful and 92% felt it improved their ability to visualise errors during the donning and doffing process. Only 4% felt that the training would have no effect on their future PPE use. Overall, 98% of participants agreed or strongly agreed that all incoming residents should receive PPE training prior to starting work in the healthcare setting.

Table II.

Opinion questions

	Top 2 boxesa, n (%)			Median (IQR)
	Pre	Post	Difference (95% CI)	Pre	Post	P-value
Confidence in PPE donning/doffing	6 (13%)	48 (100%)	88% (78%–97%)	3 (2–3)	5 (4–5)	<0.001
Concerned of COVID-19 exposure	17 (35%)	13 (27%)	-8% (-24%–7%)	3 (2–4)	3 (2–4)	0.311
Prepared to use PPE	8 (17%)	47 (98%)	81% (70%–93%)	2 (2–3)	5 (4–5)	<0.001
Residents should receive PPE education	47 (98%)	47 (98%)	0% (-6%–6%)	5 (5–5)	5 (5–5)	0.279

^aResponses of 4 or 5 on 5-point Likert scale.

Contamination

Prior to training, 81% (39/48) of participants had contamination compared to 17% (8/48) after training (P<0.001). Half of all participants had at least one wrist contaminated, which was the most common area of contamination. The hand and forearm were the next two most common areas of contamination. With training, all areas of contamination reduced, with no contamination of the face after PPE training (Table III). Prior to training, statistically significant correlations between contaminated body areas were noted of the arm and face, wrist and face, and hand and wrist (Table IV).

Table III.

Subjects with Contamination Results Summary, n (%)

Body surface	Pre-training contamination	Post-training contamination	Difference (95% CI)	P-value
Face	10 (21%)	0 (0%)	-21% (-33%–9%)	0.001
Neck	10 (21%)	0 (0%)	-21% (-33%–9%)	0.001
Chest	4 (8%)	0 (0%)	-8% (-16%–0%)	0.044
Arm	3 (6%)	2 (4%)	-2% (-12%–7%)	0.659
Forearm	11 (23%)	5 (10%)	-13% (-27%–2%)	0.083
Wrist	24 (50%)	5 (10%)	-40% (-55%–24%)	<0.001
Hand	14 (29%)	0 (0%)	-29% (-43%–16%)	<0.001
Leg	0 (0%)	0 (0%)	0%
Any area	39 (81%)	8 (17%)	-65% (-80%–49%)	<0.001

Table IV.

Correlations between contaminated body areas pre training

	Face	Neck	Chest	Arm	Forearm	Wrist	Hand	Leg
Face	-	-0.14	0.03	0.29a	-0.04	0.31b	0.12	-
Neck	-0.14	-	0.03	0.08	-0.16	0.00	0.01	-
Chest	0.03	0.03	-	0.23	-0.16	0.00	-0.03	-
Arm	0.29a	0.08	0.23	-	0.06	0.08	0.02	-
Forearm	-0.04	-0.16	-0.16	0.06	-	0.05	-0.24	-
Wrist	0.31b	0.00	0.00	0.08	0.05	-	0.37c	-
Hand	0.12	0.01	-0.03	0.02	-0.24	0.37c	-	-
Leg	-	-	-	-	-	-	-	-

^aSignificant at P=0.044.

^bSignificant at P=0.033.

^cSignificant at P=0.010.

Donning and doffing

Average total score for donning and doffing of participants significantly increased with training. This was true for both average donning scores (52% vs. 98%; difference 47%; 95% CI: 37%–57%; P<0.001) and doffing scores (46% vs. 85%; difference 39%; 95% CI: 33%–46%; P<0.001). Participants had significantly increased success rate for each step of donning and doffing (Table V, Table VI).

Table V.

Successful donning steps before and after training

	Pre-training success	Post-training success	Difference (95% CI)	P-value
1: Sanitize hands	76%	100%	24% (11%–37%)	<0.001
2: Don isolation gown	36%	97%	61% (48%–74%)	<0.001
3: Don N95 mask	40%	99%	59% (45%–72%)	<0.001
4: Don face shield or goggles	49%	100%	51% (37%–65%)	<0.001
5: Put on gloves	56%	100%	44% (29%–58%)	<0.001
Total	52%	98%	47% (37%–57%)	<0.001

Table VI.

Successful doffing steps before and after training

	Pre-training success	Post-training success	Difference (95% CI)	P-value
1: Remove gloves	26%	100%	74% (62%–87%)	<0.001
2: Remove gown	24%	96%	71% (58%–85%)	<0.001
3: Leave contaminated area	6%	49%	43% (27%–58%)	<0.001
4: Sanitize hands	52%	70%	18% (0%–36%)	0.048
5: Remove face shield or goggles	74%	96%	21% (10%–32%)	<0.001
6: Remove/waste N95	83%	99%	16% (7%–25%)	0.001
7: Sanitize hands	54%	86%	32% (15%–49%)	0.001
Total	46%	85%	39% (33%–46%)	<0.001

Discussion

Our study highlights the need for PPE training for newly graduated medical students entering residency. Prior PPE use knowledge was poor, and PPE education is an essential aspect of orientation prior to clinical work. It is also important for clinicians to know the transmission modes of virulent diseases for proper body substance isolation in avoiding contamination. During donning and doffing of PPE, frequent contamination occurred and a pattern was identified in this study. Wrist contamination was the most common area and correlated with face contamination. Most importantly, PPE training had clear effect in reducing contamination after doffing. Our study also found that the most common error was poor adherence to PPE donning and doffing sequence. The simulation-based training was positively perceived by participants. Post-survey data showed it improved confidence and preparedness of PPE usage for clinical care of COVID-19 patients.

Previous studies have utilised ultraviolet fluorescent tracer during simulated patient history and physical exams, however this study is unique in its use of CPR as the defining simulation event [9]. Despite this difference, our results indicate that during PPE usage, the wrist is the major source of contamination which has also been shown in previous research [[10], [11], [12], [13]]. Our study further shows that contamination of the wrists and arms appears to correlate with contamination of the face, a result not reported previously. This is important as face contamination likely increases the risk of mucous membrane exposure and infection since healthcare workers do not routinely wash their faces after doffing their PPE [13].

There are a variety of unique errors occur during PPE use due to its numerous and complex sequence of steps to properly perform [13,14]. Most errors occurred because of failure to adhere to the recommended step-by-step protocol (e.g. not sanitizing hands before placing isolation gown) or improper technique. Many of these errors occurred because participants did not secure the gowns behind their bodies correctly during donning, allowing for the gown to become dislodged during CPR. The investigators noticed that this appeared to result in significantly more touching between the gown and the provider's hands as attempts were made to readjust the gown. Previous research suggests this lack of proper sequence is not uncommon and that errors in doffing are high risk for contamination since these occur after attending to the patient [15].

The extremely low number of participants that underwent hands-on PPE training prior to starting internship is surprising and reflects the need for training. This is highlighted by participant feedback regarding the usefulness of the training program and is also shown in previous simulation-based training in medical education [16]. Post-curricular evaluations overwhelmingly suggest that tracer utilisation was a highly beneficial component of training methodology as it provided objective feedback regarding contamination. While previous research suggests fluorescent tracer may not delineate all clinically relevant areas of contamination [12], it appears to be effective in estimating how effectively someone utilises PPE and their risk for potential contamination [13,17,18]. Further, the most beneficial aspect of this training regimen utilising tracer may lie in its ability to give real-time visual feedback as someone is performing simulated training exercises. Previous research also supports this type of training as both direct feedback and repetition of educational goals is effective in trainee understanding [19].

While it is not clear if improved PPE use will translate into future reductions in workplace-caused infections, an immediate secondary benefit is the reduced anxiety regarding the risk of work-related occupational exposure. Previous research has also shown this type of benefit [20]. Additionally, there is a general increased psychological risk with prolonged use of PPE [21]. With a better knowledge on the donning and doffing technique and lessened anxiety regarding iatrogenic exposure, clinicians may be more willing to remove and replace PPE for breaks. This is especially important as resident interns are particularly prone to excessive stress, anxiety and burnout [[22], [23], [24], [25]].

This study has several important limitations. While our results suggest that PPE training is both highly effective and essential to proper usage, it is unclear if this leads to decreased actual transmission of communicable diseases. However, during a pandemic, it is unlikely that a randomized trial of PPE training would be undertaken for ethical reasons. Further, this study only included physicians. It did not study non-physicians. However, there is no clear reason that these individuals would not also benefit from this type of training.

Another limitation is that use of the fluorescent tracer is based on contact and not aerosolization. There are tracers that simulate aerosol exposure and may be more applicable for viral transmission. Lastly, the training and testing occurred on the same day and it is unclear what interval of re-training may be needed to solidify long-term habits regarding PPE usage.

Conclusions

A simulation-based curriculum utilizing fluorescent tracer demonstrated improved confidence, improved knowledge, reduced contamination, and improved performance of donning and doffing PPE in simulated scenarios for first year residents.

Credit author statement

Spencer W. Greaves: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing – Original Draft, Writing – Review and Editing, Visualization Scott M. Alter: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data Curation, Writing – Original Draft, Writing – Review and Editing Rami A. Ahmed: Conceptualization, Methodology, Validation, Investigation, Writing – Original Draft, Writing – Review and Editing Kate E. Hughes: Conceptualization, Methodology, Validation, Investigation, Writing – Original Draft, Writing – Review and Editing Devin Doos: Conceptualization, Methodology, Validation, Investigation, Writing – Original Draft, Writing – Review and Editing Lisa M. Clayton: Conceptualization, Methodology, Validation, Investigation, Writing – Original Draft, Writing – Review and Editing Joshua J. Solano: Conceptualization, Methodology, Validation, Investigation, Writing – Original Draft, Writing – Review and Editing Sindiana Echeverri: Conceptualization, Methodology, Investigation, Resources, Writing – Original Draft, Writing – Review and Editing Richard D. Shih: Methodology, Investigation, Writing – Original Draft, Writing – Review and Editing Patrick G. Hughes: Conceptualization, Methodology, Validation, Investigation, Resources, Writing – Original Draft, Writing – Review and Editing, Visualization, Supervision.

Conflict of interest statement

Research support: This research received no external financial or non-financial support.

Relationships: There are no additional relationships to disclose.

Patents and Intellectual Property: There are no patents to disclose.

Other Activities: There are no additional activities to disclose.

Funding

No funding to declare.

Ethics approval

University IRB reviewed study and determined it not to be human subjects research.

Appendix A. Supplementary data

The following are the Supplementary data to this article: