Tracheal intubation while wearing personal protective equipment in simulation studies: a systematic review and meta-analysis with trial-sequential analysis

Abstract

Background

Tracheal intubation in patients with coronavirus disease-19 is a high-risk procedure that should be performed with personal protective equipment (PPE). The influence of PPE on operator’s performance during tracheal intubation remains unclear.

Methods

We conducted a systematic review and meta-analysis of simulation studies to evaluate the influence of wearing PPE as compared to standard uniform regarding time-to-intubation (TTI) and success rate. Subgroup analyses were conducted according to device used and operator’s experience.

Results

The TTI was prolonged when wearing PPE (eight studies): Standard Mean Difference (SMD) -0.54, 95% Confidence Interval [-0.75, -0.34], p <  0.0001. Subgroup analyses according to device used showed similar findings (direct laryngoscopy, SMD -0.63 [-0.88, -0.38], p < 0.0001; videolaryngoscopy, SMD -0.39 [-0.75, -0.02], p =  0.04). Considering the operator’s experience, non-anesthesiologists had prolonged TTI (SMD -0.75 [-0.98, -0.52], p < 0.0001) while the analysis on anesthesiologists did not show significant differences (SMD -0.25 [-0.51, 0.01], p = 0.06). The success rate of tracheal intubation was not influenced by PPE: Risk Ratio (RR) 1.02 [1.00, 1.04]; p = 0.12). Subgroup analyses according to device demonstrated similar results (direct laryngoscopy, RR 1.03 [0.99, 1.07], p = 0.15, videolaryngoscopy, RR 1.01 [0.98, 1.04], p =  0.52). Wearing PPE had a trend towards negative influence on success rate in non-anesthesiologists (RR 1.05 [1.00, 1.10], p = 0.05), but not in anesthesiologists (RR 1.00 [0.98, 1.03], p = 0.84). Trial-sequential analyses for TTI and success rate indicated robustness of both results.

Conclusions

Under simulated conditions, wearing PPE delays the TTI as compared to dressing standard uniform, with no influence on the success rate. However, certainty of evidence is very low. Performing tracheal intubation with direct laryngoscopy seems influenced to a greater extent as compared to videolaryngoscopy. Similarly, wearing PPE affects more the non-anesthesiologists subgroup as compared to anesthesiologists.

Introduction

At the beginning of May 2021 the coronavirus disease 2019 (COVID-19) pandemic has approached almost 165 million diagnosed cases in 191 countries/regions, causing over 3.300.000 deaths according to the data from the Center for Systems Science and Engineering at Johns Hopkins University.¹ It has been estimated that around 2% of patients with COVID-19 will eventually require tracheal intubation for acute respiratory failure.²

A cornerstone in the management of COVID-19 pandemic has been the use of personal protective equipment (PPE) in order to ensure adequate protection to healthcare staff. Indeed, COVID-19 is a highly contagious disease and all maneuvers spreading aerosol particles (such as tracheal intubation) are considered as high-risk procedures for contamination of healthcare personnel. During the 2003 Severe Acute Respiratory Syndrome epidemic, Loeb et al. identified the following activities at high-risk for contagion in healthcare professionals: performing and assisting for tracheal intubation, suctioning before intubation, manipulating the oxygen mask.³ Such risks were subsequently confirmed by a systematic review.⁴

In the attempt to safely manage airways of COVID-19 patients and to minimize the risk of viral transmission, several recommendations have been issued in the course of the current pandemic.5, 6, 7 Among these, experts have suggested to wear full PPE and to use a videolaryngoscope (VLS) already at the first tracheal intubation attempt, possibly using a device with distant screen in order to enable further distancing of the operators from the patient’s mouth. Of note, these recommendations are mostly based on a theoretical background and on the experience acquired during present⁵ and previous⁸ pandemics. However, there is currently no strong evidence supporting these recommendations and it remains unclear if VLS is truly the best choice. Moreover, it remains uncertain if the use of PPE has different impact on tracheal intubation according to the operator’s experience. Therefore, several aspects pertaining the approach to tracheal intubation during a pandemic need to be systematically addressed.

Whilst it seems likely that the indispensable use of PPE under hazardous conditions may render airway management more challenging due to operator’s constraints, one study reported faster performance by operators wearing PPEs as compared to their results whilst dressing in standard uniform.⁹ Moreover, several other studies showed neutral findings.10, 11, 12 In light of the above considerations, we thought it was urgent to pool the results of the available evidence. Therefore, we conducted a systematic review and a meta-analysis of simulation studies to evaluate the impact of wearing PPE as compared to standard uniform, both in term of time to intubation (TTI) and of success of the procedure, with sub-analyses conducted according to the type of device used or to the operator’s experience.

Methods

This systematic review and meta-analysis is reported in accordance with PRISMA guidelines.¹³ A protocol was written before starting the current review but the registration with the international prospective register of systematic reviews (PROSPERO) was not feasible as the register itself does not currently consider systematic reviews and meta-analyses on simulation studies.

Eligibility criteria

We included prospective studies conducted in simulated adult scenarios where participants with any level of experience in airway management performed tracheal intubation both under standard uniform dressing and whilst wearing any level of PPE. Regarding the simulation scenarios, we decided to include prospective simulated studies performed both on manikins or cadavers, and irrespectively of a normal or difficult airway scenario. The outcomes of interest were TTI and success rate (see Supplementary Digital Content 1 for PICOS criteria – Population, Intervention, Comparison, Outcomes, Study design).

Exclusion criteria

Studies including a population below 10 participants were considered for sensitivity analyses only. Paediatric studies were excluded. We applied a language restriction and only articles providing an abstract and published in the English language were considered for inclusion.

Search strategy

Two systematic independent literature searches of the electronic databases were performed through the NHS Healthcare Databases Advanced Search. We systematically searched the PubMed, MedLine, and EMBASE databases with the last update on September 3^rd, 2020; the searches consisted of the combination of the MESH term “airway” with at least one term from each of two groups: 1) “simulat*” or “manikin*” or “mannequin*”, and 2) “protective equipment” or “CBRN” (Chemical, Biological, Radiological, and Nuclear hazards) or “protective clothing” or “PPE” or “biohazard” or “protective gear” for the second group. A further independent manual search was performed by two authors (ST, VLR).

Study selection and data extraction

Two pairs of assessors screened the titles and abstracts for suitability (FS, ST, VLR, PM), with a fifth assessor (MA) arbitrating any disagreements. Full text articles identified as potentially relevant were assessed against PICOS criteria. Discrepancies were resolved by consensus and/or by involving another author (MA). All the authors conducted also an independent search on PubMed to check for further evidence. Two reviewers (ST, VLR) independently extracted data from individual studies and entered information into a pre-designed data collection form, which was cross-checked by two other authors (FS, PM). If needed, we planned to contact the corresponding authors to get more data.

Synthesis of evidence and outcomes of interest

The two primary outcomes were time-to-intubation (TTI) and success rate of the intubation procedure. These outcomes were considered for participants performing tracheal intubation wearing any type of PPE as compared to standard work dressing. In this regard, it must be noted that the level of protection offered by PPEs varies. In fact, PPEs have been classified by the Environmental Protection Agency and by the Occupational Safety and Health Administration into four levels of protection,14, 15 each one identified by a capital letter (A, B, C, D). The level A PPE offers the highest protection, whilst level D identifies the standard uniform (standard precautions for healthcare professionals as gloves, splash protection, etc.). Details of PPE level are provided as Supplementary Digital Content 2. For each study, we evaluated in detail the equipment worn by participants before confirming the level of PPE. Despite apparent differences in PPE levels and in the protection offered, they all increase constraints and discomfort for the operator. For such reason, and because we expected a relatively low number of prospective studies with a heterogeneous design regarding devices and operator’s experience,¹⁶ we preventively decided to perform the primary analysis grouping all the PPE levels (A, B, and C) as compared to standard uniform (level D).

Subgroup and secondary analyses

Subgroup analyses of the outcomes of interest (TTI, success rate) were conducted separating studies results according to: a) the type of device used, or b) the operator’s experience. Regarding the type of device, we divided data of participants in those performing tracheal intubation with direct laryngoscopy (DL, with or without the use of aids) or with a VLS. In case of studies where DL was performed both alone or with the use of aids (i.e., stylet, bougie), we included only the DL without use of aids. In the second subgroup analysis, we separated studies where all participants were anaesthesiologists (or anesthesiology residents) from those where the population was mostly represented by non-anaesthesiologists. In particular, in the case of a study with a mixed population, it was classified in the subgroup of non-anaesthesiologists if the anaesthesiologists were less than 50% of the study population.

We also performed secondary analyses. As discussed, PPEs are classified into 3 levels (A, B, C). In the secondary analyses, we repeated the analyes for TTI and success rate separating studies according to the type of PPE. Thus, we performed three secondary analyses. Another secondary analysis was performed pooling together the PPE of level A and B, as we judged similar the level of discomfort/constraint caused by them (main difference is that PPE level A is fully encapsulating, while B is not).

Statistical analysis, heterogeneity, risk of bias, quality of assessment, and publication bias

Number of participants, mean values, and standard deviation were collected for the outcome analysis of TTI. If data were reported only as median and interquartile range or confidence interval (CI), we followed the approaches suggested by Luo et al.¹⁷ and Wan et al.¹⁸ For the analysis of the success rate, we collected the number of attempts and the success at first attempt. A random effect model was used, the continuous outcome (TTI) and the categorical variable (success rate) differences were analyzed using the inverse variance method with a 95%CI. Values for TTI are reported as standard mean difference (SMD), while success rate is reported according to Risk Ratio (RR); p-values were two-tailed and considered significant if p < 0.05.

The presence of statistical heterogeneity was assessed using the X² (Cochran Q) test. Heterogeneity was likely if Q > df (degrees of freedom) and confirmed if p ≤ 0.10. Quantification of heterogeneity was performed using I² statistic. Values of 0–24.9%, 25–49.9%, 50–74.9%, and > 75% were considered as none, low, moderate, and high heterogeneity, respectively. Meta-analysis was performed using review manager (Revman, Version 5.4. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014).

As already performed in other meta-analyses in the field of anesthesiology,19, 20 we conducted trial-sequential analyses (TSAs) and assessment of the quality of evidence of our findings. In particular, we performed the TSA in order to evaluate the effect of random error and calculate the information size (the power of the meta-analysis) for the overall analyses of TTI and success rate. We used the freely-available TSA Software (Copenhagen Trial Unit’s TSA Software®; Copenhagen, Denmark). The information size was computed assuming an alpha risk of 5%, a beta risk of 20%. The estimated effects (RR and SMD) were computed averaging results of the classical meta-analysis method. Further details on TSA and its interpretation are available elsewhere.²¹

There were no deviations from the protocol. The risk of bias was performed using the RoB 2.0 for “Individually-randomized, cross-over trials”²² which was deemed the most appropriate after consultation with senior statisticians. Publication bias was investigated by visual inspection of funnel plots for the primary outcomes. The quality of evidence was generated in accordance with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group. Using the GRADEpro online software, we performed a summary of the overall analyses in order to assess the quality and certainty of evidence for each outcome (pooled data expressed as SMD or RR).23, 24, 25 The flow of references was managed with the Endnote X7 citation manager.

Results

The two independent literature searches produced 31 titles on Medline, 48 on PubMed, and 51 on EMBASE. The PRISMA flowchart of the systematic search and qualitative synthesis is reported as Figure 1. After removal of duplicates and after screening of titles and abstracts, 17 articles were considered of interest. Six were excluded because participants performed simulated tracheal intubation wearing PPEs but not under a standard uniform (missing control group). Two other studies were excluded for not reporting the data required (we contacted twice the corresponding author without success). Therefore, we finally included 9 studies. Of these, one was a four-arm study (controls, PPE A, B, and C),¹⁰ another evaluated tracheal intubation with both levels B and C PPEs as compared with controls.⁹ Of the remaining seven studies, five compared level C PPEs to standard uniform,11, 12, 26, 27, 28 and two involved the use of level A PPEs.29, 30 All studies performed tracheal intubation with DL (± aids), one included also attempts with a McCoy laryngoscope,³⁰ and five performed tracheal intubation with VLS(s).9, 11, 27, 28^,30 As shown in Table 1, data extracted from each study included the PPE level, the type of device(s) used for tracheal intubation, operator’s experience, and the outcome(s) of interest reported.

Figure 1.

PRISMA flowchart.

Table 1.

Characteristics of the included studies.

First Author, Year	PPE(s) level	Device(s) for tracheal intubation	Outcome reported
Journal		Operator’s experience
Schumacher, 2020	B, C	DL; VLSs	TTI,
Anaesthesia		25 Anaesthesiologists	Success rate
Plazikowski, 2018	C	DL; VLS	TTI,
Inf Control and Hosp		30 Anaesthesiologists	Success rate
Taylor, 2018	C	DL; VLS	–
Am J Emerg Med		19 (15 EM residents; 4 EMS personnel)	Success rate
Wang, 2016	C	DL+stylet	TTI,
Eur J Emerg Med		40 EM residents	Success rate
Shin, 2013	C	DL+stylet; VLS	TTI,
Emerg Med J		31 medical doctors passing the national boards exam	Success rate
Castle, 2011 Dec	A	DL, DL+stylet, DL+bougie; McCoy; VLS	TTI,
Resuscitation		66 final-year paramedic students	Success rate
Castle, 2011 Maya	A	DL	TTI,
Resuscitation		48 final-year paramedic students	Success rate
Burns, 2010	C	DL	TTI
Prehosp and Disaster Med		47 EMS personnel	–
Garner, 2004	A, B, C	DL	TTI
Emerg Med Australasia		16 mixed (3 paramedics; 8 prehospital doctor; 3 EM physicians; 2 anaesthesiologists)	–

DL, direct laryngoscopy; EM, emergency medicine; EMS, emergency medical service; PPE, personal protective equipment; TTI, time-to-intubation; VLS, videolaryngoscope.

^aThis study evaluated the tracheal intubation in four different manikin's positions. We used the values reported in the ambulance trolley (60 cm height) position and discarded results reported regarding the intubation performed lying prone, kneeling and sitting.

Outcome analyses – TTI

Eight studies reported the TTI comparing simulated procedure performed both dressing standard uniform (controls) and wearing PPE.9, 10, 11, 12, 26, 28, 29, 30 In particular, regarding the intervention group, seven studies reported data on level C PPE, two with level B, and three level A.

The primary outcome analysis (pooling together results from all levels of PPEs) showed that tracheal intubation was significantly shorter wearing standard uniform (SMD -0.54, 95%CI [-0.75, -0.34], p <  0.0001, I² = 69%; 19 studies, 1306 procedures; very low certainty of evidence). As shown in Fig. 2, we found no subgroup differences according to the device used for tracheal intubation, being TTI shorter with standard uniform using both DL (SMD -0.63, 95%CI [-0.88, -0.38]; p < 0.0001, I² = 69%) or VLS (SMD -0.39, 95%CI [-0.75, -0.02]; p = 0.04, I² = 72%). As shown in Supplementary Digital Content 3, when dividing subgroups according to the operator’s experience, anesthesiologists had a non-significant trend towards shorter TTI whilst wearing standard uniform (SMD -0.25, 95%CI [-0.51, 0.01]; p = 0.06, I² = 72%). Conversely, in the subgroup of non-anesthesiologists (which included also Garner et al.¹⁰ where 2 out of 16 participants were anesthesiologists) the TTI was significantly shorter wearing standard uniform as compared to PPE (SMD -0.75, 95%CI [-0.98, -0.52]; p < 0.0001, I² = 61%).

Figure 2.

Forest plot comparing the Time to Intubation wearing standard uniform as compared with personal protective equipment (PPE) of level A, B, or C. Subgroups are divided according to the type of device used for the tracheal intubation.

CI, confidence interval; DL, direct laryngoscopy; IV, inverse variance; SD, standard deviation; VLS, videolaryngoscopy.

For each study, we indicated the first author, the journal and year of publication, the type of device used for tracheal intubation, and the level of PPE worn (in brackets).

In order to provide more clinical sense of potential delay of intubation under the constraints of wearing PPE, we also calculated the weighted means from the groups. We found that weighted TTI was on average prolonged by 11.3 seconds when wearing PPE uniform (38.8 vs. 27.5 seconds in the standard uniform).

Outcome analyses – success rate

Seven studies reported the success rate during simulated attempts of tracheal intubation performed by participants dressing standard uniform (controls) and wearing PPE(s).9, 11, 12, 27, 28, 29, 30 We pooled data from five studies where participants worn level C PPE, one where they dressed level B PPE and two with level A PPE. The primary outcome analysis showed that wearing PPE did not significantly influence the success rate of tracheal intubation (RR 1.02, 95%CI [1.00, 1.04]; p =  0.12, I² = 25%; 17 studies, 1192 procedures; very low certainty of evidence; Fig. 3). This finding was valid for both the use of DL (RR -1.03, 95%CI [0.99, 1.07]; p = 0.15, I² = 49%) or VLS [(RR 1.01, 95%CI [0.98, 1.04]; p = 0.52, I² = 0%).

Figure 3.

Forest plot comparing the success rate of tracheal intubation wearing standard uniform as compared with personal protective equipment (PPE) of level A, B, or C. Subgroups are divided according to the type of device used for the tracheal intubation.

CI, confidence interval; DL, direct laryngoscopy; IV, inverse variance; VLS, video-laryngoscopy.

For each study, we indicated the first author, the journal and year of publication, the type of device used for tracheal intubation), and the level of PPE worn (in brackets).

When the analysis was performed dividing subgroups according to the operator’s experience, we found that success rate in anesthesiologists was not influenced by wearing PPEs as compared to standard uniform (RR -1.00, 95%CI [-0.98, 1.03]; p =  0.84, I² = 0%). Conversely, the subgroup of “non-anesthesiologists” had a trend towards lower success rate when wearing PPEs (RR 1.05, 95%CI [1.00, 1.10]; p =  0.05, I² = 57%, Supplementary Digital Content 4).

In order to provide more clinical understanding on the impact of wearing PPE on success rate at first attempt of the intubation procedure, we calculated the weighted success rates, which was, on average, almost 5% lower when wearing PPE uniform (93.6% vs. 98.5% in the standard uniform).

Trial-sequential analyses

The two TSA performed for the overall TTI and success rate showed similar results (Supplementary Digital Content 5 and 6, respectively). The TSA on the overall TTI crossed the trial sequential monitoring boundary, showing that meta-analysis results are well-powered. Indeed, the two-sided alpha-spending boundary according to O’Brien-Fleming method showed that the number of intubation procedures needed to be included in the meta-analysis to reach the desired level of significance and power was 148, which was passed very early by the z-curve. Therefore, the result on TTI seems very robust and unlikely to be biased.

The z-curve on the overall success crossed the futility boundary and almost reached the required sample size of 1243 procedures. This indicates that the finding of no difference in success rate between standard uniform and PEE is robust and no further studies are needed.

Secondary analyses

Table 2 shows the results of the primary analyses on TTI and success rates together with the findings of the analyses conducted according to the level of PPE used. As the studies of Schumacher et al.⁹ and Garner et al.¹⁰ compared the control group with two and three level of PPE respectively, we also conducted further post-hoc analyses of the primary outcomes considering one level of PPE only (vs. control group) for these two studies. None of these analyses changed the results.

Table 2.

Summary of the results of the primary and secondary analyses comparing tracheal intubation wearing personal protective equipment (PPE) vs. standard uniform (control group).

Outcome and type of PPE	Overall	DL	VLS	Anesthesiologists	Non-anesthesiologists
TTI	SMD -0.54 [-0.75, -0.34]	SMD -0.63 [-0.88, -0.38]	SMD -0.39 [-0.75, -0.02]	SMD -0.25 [-0.51, 0.01]	SMD -0.75 [-0.98, -0.52]
PPE all types (n = 19)	p < 0.00001; I2 = 69%	p < 0.0001; I2 = 67%	p = 0.04; I2 = 72%	p = 0.06; I2 = 46%	p < 0.00001; I2 = 61%
TTI	SMD -0.37 [-0.62, -0.12]	SMD -0.35 [-0.61, -0.09]	SMD -0.40 [-0.95, 0.15]	SMD -0.27 [-0.66, 0.13]	SMD -0.47 [-0.79, -0.15]
PPE C (n = 10)	p = 0.004; I2 = 56%	p = 0.008; I2 = 37%	p = 0.16; I2 = 76%	p = 0.19; I2 = 62%	p = 0.005; I2 = 52%
TTI	SMD -0.27 [-0.56, -0.02]	SMD -0.51 [-0.95, -0.07]	SMD -0.08 [-0.47, 0.31]	SMD -0.22 [-0.57, 0.12]	not reported as based on one study only
PPE B (n = 4)	p = 0.07; I2 = 0%	p = 0.02; I2 = 0%	p = 0.69; I2 = 0%	p = 0.21; I2 = 14%
TTI	SMD -1.03 [-1.21, -0.84]	SMD -1.09 [-1.31, -0.88]	not reported as based on one study only	no studies on anesthesiologists	SMD -1.03, [-1.21, -0.84]
PPE A (n = 5)	p < 0.00001; I2 = 0%	p < 0.00001; I2 = 0%			p < 0.00001; I2 = 0%
TTI	SMD -0.74 [-1.01, -0.46]	SMD -0.97 [-1.18, -0.76]	SMD -0.36 [-0.94, 0.21]	SMD -0.22 [-0.57, 0.12]	SMD -0.99 [-1.17, -0.82]
PPE A/B (n = 9)	p < 0.00001; I2 = 65%	p < 0.00001; I2 = 15%	p = 0.22; I2 = 77%	p = 0.21; I2 = 14%	p < 0.00001; I2 = 0%
Success rate	RR 1.02 [1.00, 1.04]	RR 1.03 [0.99, 1.07]	RR 1.01 [0.98, 1.04]	RR 1.00 [0.98, 1.03]	RR 1.05 [1.00, 1.10]
PPE all types (n = 17)	p = 0.12; I2 = 25%	p = 0.15; I2 = 49%	p = 0.52; I2 = 0%	p = 0.84; I2 = 0%	p = 0.05; I2 = 57%
Success rate	RR 1.01 [0.98, 1.04]	RR 1.01 [0.97, 1.05]	RR 1.02 [0.97, 1.06]	RR 1.00 [0.97, 1.04]	RR 1.04 [0.97, 1.11]
PPE C (n = 10)	p = 0.43; I2 = 0%	p = 0.59; I2 = 0%	p = 0.52; I2 = 21%	p = 0.79; I2 = 0%	p = 0.28; I2 = 44%
Success rate	RR 1.00 [0.96, 1.04]	not reported as based on one study only	RR 1.00 [0.95, 1.06]	RR 1.00 [0.96, 1.04]	no studies on non-anesthesiologists
PPE B (n = 3)	p = 1.00; I2 = 0%		p = 1.00; I2 = 0%	p = 1.00; I2 = 0%
Success rate	RR 1.06 [0.98, 1.15]	RR 1.07 [0.96, 1.20]	not reported as based on one study only	no studies on anesthesiologists	RR 1.06 [0.99, 1.14]
PPE A (n = 4)	p = 0.15; I2 = 73%	p = 0.20; I2 = 82%			p = 0.09; I2 = 71%
Success rate	RR 1.03 [0.99, 1.08]	RR 1.05 [0.97, 1.14]	RR 1.01 [0.96, 1.06]	RR 1.00 [0.96, 1.04]	RR 1.06 [0.99, 1.14]
PPE A/B (n = 7)	p = 0.19; I2 = 53%	p = 0.21; I2 = 75%	p = 0.75; I2 = 0%	p = 1.00; I2 = 0%	p = 0.09; I2 = 71%

Results of Time-To-Intubation (TTI) are reported as Standardized Mean Difference (SMD, with in square brackets the 95% confidence interval), followed by p- value and statistical heterogeneity (I²). Results of success rate are reported as Risk Ratio (RR, with in square brackets the 95% confidence interval), with p-value and statistical heterogeneity (I²). Results are in bold font if statistically significant. Results with a trend towards statistically significant finding are indicated in italic font.

DL: direct laryngoscopy; VLS: videolaryngoscope.

Risk of bias, publication bias, and quality of evidence assessment

The assessment of the risk of bias performed with the RoB 2.0 showed that all studies were at high-risk (Supplementary Digital Content 7).

We investigated publication bias by visual inspection of funnel plots for the primary outcomes (Supplementary Digital Content 8). We found no evidence of publication bias regarding the TTI outcome. The success rate included a total of seventeen comparisons, with ten of them reporting a RR value of 1.00 and similar sample size. Therefore, several studies had an overlap on the funnel plot. Only one comparison of the three reported by Castle et al.³⁰ behaved as outlier (RR = 1.20, intubation with McCoy DL). Its exclusion did not change the overall and the subgroups’ results.

Quality of evidence generated in accordance with the GRADE Working Group resulted very low for both outcomes (Table 3), suffering from the high-risk of bias in the included studies and the indirectness of the simulation scenarios.

Table 3.

Evaluation of quality of evidence according to Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group.

Does wearing PPE compared to standard uniform influence the operator's performances during tracheal intubation?
Patient or population: Operators performing tracheal intubation	Setting: Simulation studies
Intervention: Wearing PPE	Comparison: Wearing standard uniform

Outcomes	Difference wearing PPE as compared to standard uniform	Relative effect (95% CI)	N. of participants (studies)	Certainty of the evidence (GRADE)	Comments
Time to intubation	SMD 0.54 SD lower (0.75 lower to 0.34 lower)	–	1306 (19 RCTs)	⨁◯◯◯ VERY LOWa, b, c	Quality of evidence was downgraded because the high-risk bias in included studies, the inconsistency due to the different level of experience in airway management across studies' participants, and the indirectness of findings due to the simulation environment. Result of the trial sequential analysis shows that the meta-analysis is well-powered.
Success rate	19 more per 1.000	RR 1.02 (1.00 to 1.04)	1192 (17 RCTs)	⨁◯◯◯ VERY LOWa, b, c	Quality of evidence was downgraded because the high-risk bias in included studies, the inconsistency due to the different level of experience in airway management across studies' participants, and the indirectness of findings due to the simulation environment. Result of the trial sequential analysis shows that the meta-analysis is well-powered.

*The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).

CI, confidence interval; SMD, standardised mean difference; RR, risk ratio.

^aAs per RoB 2.0.

^bDifferent levels of operator's experience.

^cFindings are from simulation studies.

Discussion

We conducted a systematic review and meta-analysis with the aim of investigating whether performing tracheal intubation under the constraints of wearing PPE may worsen the operator’s performance, both in terms of TTI and success rate. The most important finding of our meta-analysis is that wearing PPE significantly prolonged the TTI without affecting the overall success rate of tracheal intubation as compared with standard uniform. The two TSAs showed that both analyses reached the appropriate information size. Therefore, wearing PPE prolongs the TTI without worsening the success rate of intubation with no further research required to confirm these findings. However, the high risk of bias in the included studies, the inconsistency due to the different level of experience in airway management across studies’ participants and the indirectness of findings due to the simulation environment contributed to the very low certainty of evidence for the two primary outcomes as assessed with the GRADEpro software. The larger effect size found for TTI allowed an upgrade of certainty of evidence.

From a clinical perspective, as the use of SMD and the RR are not entirely intuitive in describing to what extent PPE hinders tracheal intubation, we calculated the weighted means of TTI and success rate. On average, we found that PPE increased the TTI of about 11 seconds, whilst decreasing 5% the probability of success at first attempt. As guaranteeing the operators’ safety during intubation is certainly of utmost importance, our meta-analysis numerically supports that wearing PPE has a mild clinical impact on intubation practice.

We also performed subgroup analyses according to the device used for tracheal intubation or to the operator’s experience. The first subgroup analysis (taking into account the type of device used) confirmed that wearing PPE increases the TTI regardless of the device used. However, the delay in achieving tracheal intubation was longer using DL as compared with VLS (see values of SMD and relative 95%CI – weighted means around 13 s for DL and 9 s for VL). The success rate was not different between subgroups.

The other subgroup analysis was performed considering the experience in airway management. In this analysis, the TTI was significantly prolonged in operators with lower experience in airway management, whilst the impact on anesthesiologists did not reach a statistically significant result (p =  0.06). Regarding the success rate at first attempt, wearing PPE did not influence the performance of anesthesiologists, whilst a trend towards lower success rate in non-anesthesiologists was noted (p = 0.05).

Noteworthy, one should keep in mind that, among several limitations of pooling evidence from simulated scenarios, the included studies were not set up for difficult intubation. This setting deserves further investigations as, theoretically, the constraints of wearing PPE may further worsen performances in difficult airway scenarios. Nonetheless, despite these and other limitations (see dedicated paragraph), our meta-analysis provides some support to the current (and possibly future) guidelines developed on the airway management in highly hazardous conditions,5, 6, 7 such as in COVID-19 patients. Current guidelines suggest that tracheal intubation procedure should be performed from the beginning with a VLS, also suggesting the use a VLS equipped with a distant screen to enable greater distance between the operator’s face and the patient’s mouth. As we did not find enough studies evaluating the use of VLS with distant screen, we could not investigate if this type of VLS offers better performances as compared to VLS with “screen-on-blade”. Indeed, there was a large heterogeneity regarding the devices for tracheal intubation in the included simulation studies. Therefore, a quantitative analysis comparing the performances of each device was not reasonable, and we rather pooled studies in broad subgroups according to the type of device (DL or VLS). This results in a certain degree of clinical heterogeneity as in the subgroup DL were also included procedures performed with the aid of a stylet,12, 28 as well as results obtained with the use of McCoy blade.³⁰ Similarly, we included in the VLS subgroup both studies performed with the device with screen attached (Pentax AWS²⁸ and Airtraq9, 11, 30) or distant from the blade (Airtraq VL⁹). We acknowledge the heterogeneity of pooling together different levels of PPE, but this choice was justified by the expected (and confirmed) relatively low number of studies.

Though some analyses were not feasible due to the low number of studies, in the attempt to look for difference between PPE levels, we performed secondary analyses in this regard confirming the results of the primary outcomes. In particular, wearing PPE delays the TTI but does not affect the overall success rate of tracheal intubation. Moreover, the findings of subgroup analyses also seemed similar, with greater delay when tracheal intubation is performed with DL or by less experienced operators (non-anesthesiologists).

Of note, we excluded one study where the participants (anesthesiology consultants and residents) performed tracheal intubation with DL wearing various PPEs but the study did not include a control group (lower level of protection was equivalent to PPE level C³¹).

Limitations

Our meta-analysis has several limitations. First, the studies presented heterogeneity in the definition of TTI, and the chronometer count did not have the same start-point for all the studies. Moreover, not all studies were clear on the TTI calculation in case of failure. In such instance, authors have two main options: 1) calculate the TTI averaging only the time taken for successful intubations, or 2) also counting the time for failed procedures attributing a “failure time” (pre-established cut-off) for each failed procedure. In other words, some studies explicitly reported in their design a predefined cut-off time to declare a failed tracheal intubation attempt (i.e., 120 seconds). In these cases, authors may have either counted the cut-off time for each failed attempt, or rather discarded the case. As in most cases authors were unclear in this regard, we preferred to use the overall number of participants for the calculation of the TTI. Since the success rates were similar between groups, we think that it is unlikely that the analysis was significantly biased by this reporting issue.

Second, the validity of subgroup analyses according to the operator experience is probably limited. Indeed, the group of anesthesiologists consisted of 55 participants only, whilst the experience in the remaining non-anesthesiologist participants was highly variable. Moreover, the between-study heterogeneity cannot be fully corrected using a random-effects analysis. In this regard, one way to investigate the influence of factors contributing to the variability of study findings would have been to perform a meta-regression analysis accounting for operator experience (anesthesiologist or non-anesthesiologist), type of model for performing intubation (manikin or cadaver), positioning of the operator (lying or in upright position), and possibly others. We unfortunately don't have the required skills to perform such advanced analysis and lack of meta-regression analysis is one limitation of our study.

Third, we investigated simulation studies and the included studies cannot account for all the human factors involved in the airways management of patients with highly infectious diseases, such as COVID-19. Indeed, while some factors are potentially accounted also in simulation studies as related to the uniform itself, other issues that may influence the operator performance (such as the fear of self-contamination with a highly infectious disease)32, 33 are probably not well-replicated by simulation environment. Moreover, in a computed tomography study, Schebesta et al. showed that manikins do not fully reflect the upper airways anatomy of actual patients.³⁴ Therefore, the generalizability of our findings and those of all manikin studies is certainly limited.

Fourth, from a statistical perspective, the decision to pool together the studies regardless the level of PPE was dictated by the expectation of a low number of studies.³⁵ This means that for the two studies exploring more than one level of PPE,9, 10 the performances of tracheal intubation with each PPE level were plotted against the same control group. However, post-hoc analyses ruled out this issue.

Fifth, we found some difficulties in finding the best scale of evaluation for the risk of bias. After consultation with a senior statistician, we agreed that the RoB 2.0 for “Individually-randomized, cross-over trials” was the most appropriate.

Conclusion

In conclusion, under simulated conditions, wearing PPE prolongs the time to achieve successful tracheal intubation as compared to dressing in standard uniform, without influence on the success rate of the procedure. The influence of wearing PPE seems greater when performing tracheal intubation with DL as compared with VLS. The performance of anesthesiologists seem less influenced than of non-anesthesiologists. Although, the overall analyses are well-powered according to the TSA, the strength of our findings is heavily weakened by the high risk of bias in the included studies and the very low/low certainty of evidence found in the GRADE assessment. The clinical impact of our meta-analysis is weakened by the variability in the included studies regarding providers, PPE used, operator experience, and by the indirectness of findings due to the simulation setting.

The PRISMA checklist is provided as supplementary material.

Conflicts of interest

The authors declare no conflicts of interest.

Appendix A. Supplementary data

The following are Supplementary data to this article: