Intubation Competence During Neonatal Fellowship Training

Abstract

Download video file ^{(24.4MB, mp4)}

DOI: 10.1542/6250519457001

Video Abstract

OBJECTIVES

To characterize neonatal-perinatal medicine fellows’ progression toward neonatal intubation procedural competence during fellowship training.

METHODS

Multi-center cohort study of neonatal intubation encounters performed by neonatal-perinatal medicine fellows between 2014 through 2018 at North American academic centers in the National Emergency Airway Registry for Neonates. Cumulative sum analysis was used to characterize progression of individual fellows’ intubation competence, defined by an 80% overall success rate within 2 intubation attempts. We employed multivariable analysis to assess the independent impact of advancing quarter of fellowship training on intubation success.

RESULTS

There were 2297 intubation encounters performed by 92 fellows in 8 hospitals. Of these, 1766 (77%) were successful within 2 attempts. Of the 40 fellows assessed from the start of training, 18 (45%) achieved procedural competence, and 12 (30%) exceeded the deficiency threshold. Among fellows who achieved competence, the number of intubations to meet this threshold was variable, with an absolute range of 8 to 46 procedures. After adjusting for patient and practice characteristics, advancing quarter of training was independently associated with an increased odds of successful intubation (adjusted odds ratio: 1.10; 95% confidence interval 1.07–1.14).

CONCLUSIONS

The number of neonatal intubations required to achieve procedural competence is variable, and overall intubation competence rates are modest. Although repetition leads to skill acquisition for many trainees, some learners may require adjunctive educational strategies. An individualized approach to assess trainees’ progression toward intubation competence is warranted.

What’s Known on This Subject:

Tracheal intubation is a critical procedure for neonatology fellows to learn. The number of intubation encounters needed for fellows to achieve procedural competence is not defined.

What This Study Adds:

The number of intubations needed for fellows to achieve procedural competence is highly variable. An individualized assessment of intubation skill acquisition may be more useful than a uniform quota of intubation encounters.

Tracheal intubation is a fundamental life-saving procedure for critically ill newborns.^1–3 Because of changes in clinical neonatal practice, neonatal intubation is performed less frequently than in previous eras.^4–6 Although neonatal training programs must ensure learners have adequate opportunities to acquire this skill, little is known regarding how much exposure is needed for trainees to obtain procedural competence.

Previous observational studies demonstrated increasing intubation success rates with trainee advancement from residency to fellowship in both the NICU and pediatric critical care medicine settings.^7–11 However, training was characterized either by overall role (ie, resident versus fellow) or year of training. In addition, in these studies, researchers did not evaluate individual progression toward competence throughout training. In disciplines such as anesthesiology, pediatric critical care, and emergency medicine, between 26 and 75 intubations attempts are necessary to achieve intubation proficiency.^12–15 The number of intubations needed to develop procedural competence has not been defined for neonatal-perinatal medicine (NPM) fellows.

Our primary objective was to define the number of intubation encounters necessary for individual NPM fellows to achieve procedural competence. In addition, we sought to assess the impact of training duration on NPM fellows’ intubation competence using a more granular exposure of the quarter of fellowship training. We hypothesized that NPM fellows’ intubation procedural success rates would improve as training progressed and the number of attempts to develop procedural competence would vary from individual to individual.

Methods

Study Design and Participants

This was a retrospective cohort study by using prospectively gathered data in the multicenter National Emergency Airway Registry for Neonates (NEAR4NEOS). NEAR4NEOS is an international neonatal airway registry that has been previously described.^16–18 Participating sites collect and contribute data for all neonatal intubations performed in the NICU or delivery room (DR), including patient, provider, and practice characteristics as well as outcome data, using standardized operational definitions.

We included intubations from October 2014 through December 2018 performed at eligible NEAR4NEOS North American academic centers. The study start time corresponded to the inception of the NEAR4NEOS database. Eligible US and Canadian sites were defined as centers where at least 90% of fellows’ intubation attempts were captured in the NEAR4NEOS database and a direct method of linking each intubation encounter to a specific neonatology fellow existed. All intubations with any fellow participation as an airway provider within the intubation encounter were included, regardless of whether the fellow was the first airway provider or not. Intubations in both the NICU and DR were included. Tracheal tube exchanges were excluded from this study, because tube exchange with ongoing ventilation through the existing endotracheal tube during the procedure represents a different skillset from endotracheal intubation without an endotracheal tube in place. At all sites, the NEAR4NEOS registry was granted institutional review board approval or was designated as quality improvement exempt from institutional review board oversight.

Definitions and Outcome Measures

On the basis of NEAR4NEOS operational definitions, an intubation encounter is defined as one sequence of completed advanced airway management, which can include multiple intubation attempts. An attempt is defined as a single advanced airway maneuver starting with the insertion of an airway device into the patient’s mouth or nose.

We ascertained the start date of fellowship training for all included fellows. For this analysis, the exposure of interest was fellowship training experience, which we defined in 2 ways: sequential intubation encounters (for the cumulative sum [CUSUM] analysis), and the quarter of fellowship training (for the aggregate analysis).

Individual CUSUM Analysis

The exposure was defined as the sequential encounter count, starting from each individual’s first intubation encounter during the fellowship. Only fellows with intubation data captured from the start of their fellowship training were included in this analysis.

Aggregate Analysis

The exposure of interest was the quarter of fellowship training. We calculated the quarter of the fellowship at the time of intubation using the start date of training and date of the intubation encounter. All fellow intubation encounters during the study period were included in this analysis.

The primary outcome for both analyses was intubation success confirmed by chest rise, auscultation, second independent laryngoscopy, carbon dioxide detection, and/or chest radiography, as previously described.¹⁶ We defined success from the perspective of the individual fellow as a successful intubation within 2 attempts performed by that fellow. Of note, if a fellow had 1 unsuccessful attempt and the subsequent attempt was performed by a new provider, this was classified as a failure. If >1 fellow participated in the encounter, each fellow was assessed separately on the basis of these definitions.

Statistical Analysis

Using Stata 16.1 (Stata Corp, College Station, TX), we compared baseline provider, patient, and practice characteristics between successful and unsuccessful intubations with standard bivariate tests (Wilcoxon rank and χ²). Intubation success rates by fellowship training quarter were calculated as the percentage of successful intubations per quarter, with 95% confidence intervals computed by using the binomial distribution.

CUSUM Analysis

CUSUM calculations were performed to assess individual performance. CUSUM evaluation requires specifying acceptable (p0) and unacceptable (p1) intubation rates as well as the probability of type I (α) and II (β) error. The acceptable failure rate was set at 20% for successful intubation within 2 attempts (corresponding to 80% success), whereas the unacceptable failure rate was set at 40%. The probability of type I (α) and II (β) error was set at .1. There is no uniformly accepted definition for neonatal intubation competence. We selected an 80% success rate because this is consistent with previous definitions used in neonatal studies^8,9 and comparable with that of observational studies reporting intubation success rates of 72% to 88% for attending neonatologists.^7,11,16 In a post hoc sensitivity analysis, we assessed different definitions of success, using a less stringent acceptable failure rate of 30% (corresponding to 70% success) and more stringent acceptable failure rate of 10% (corresponding to 90% success). In each sensitivity analysis, we set the unacceptable failure rate as twice the acceptable failure rate, as in the primary CUSUM analysis.

The formula used for the CUSUM calculation was similar to those used in studies by Ishizuka et al¹³ and Komatsu et al.¹⁹ From the values of p0, p1, α, and β, the variable s is calculated. Individual fellow charts began at 0; for each success, s was subtracted from the previous CUSUM value, and, for each failure, the amount 1-s was added. A negative trend demonstrates progression toward competence. This is achieved when the line crosses the lower decision limit (h0), which corresponds to p0, indicating the true failure rate does not differ significantly from an acceptable failure rate. Similarly, a positive trend demonstrates progression away from proficiency. If the upper decision limit (h1) is crossed, the true failure rate is significantly greater than the acceptable failure rate (ie, deficiency is reached).

Aggregate Analysis

For all fellow intubations, we assessed the impact of fellow experience on intubation success within 2 attempts with a mixed effects logistic model using the independent variable of the quarter of fellowship training (from 1 to 12). The model included a random intercept for individual fellows and fixed effect for the hospital. The model included the following a priori characteristics that have been associated with success during neonatal intubation: paralytic premedication use, use of video laryngoscopy, and patient weight at the time of intubation.^1,16,18,20 A P value <.05 was considered statistically significant.

Results

During the study period, a total of 92 fellows participated in 2297 tracheal intubation encounters in 8 hospitals representing 6 fellowship training programs (Table 1). Across these, 1766 of 2297 (77%) intubation encounters were successful within 2 attempts. Fellow characteristics and factors associated with intubation success are described in Table 2.

TABLE 1.

Participating Site Characteristics

Hospital
	1	2	3	4	5	6	7	8
Training program (for CUSUM analysis)	A	B	B	C	D	D	N/A	N/A
Country	United States	United States	United States	United States	United States	United States	United States	Canada
NICU level	IV	IV	III	IV	III	IV	III	IV
No. beds	68	98	36	56	47	32	64	65
No. admissions per y	961	1318	600	600	510	450	1060	1100
No. intubations per y	200	400	100	220	140	100	300	200
Delivery hospital	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes
NP and/or PA present	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Pediatric residents present	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Pediatric residents perform intubation	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
No. neonatal fellows per year	3	6	6	2	2	2	2	4
Fellow intubation curriculum present	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Didactic	Yes	Yes	No	No	No	No	No	No
Simulation	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Mandatory supervised intubation	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes
No. supervised intubations	5	5	5	5	2	2	N/A	3

N/A, not applicable; NP, nurse practitioner; PA, physician assistant.

TABLE 2.

Provider, Patient, and Intubation Practice Characteristics for Successful and Unsuccessful Intubations

	Successful (n = 1766)	Not Successful (n = 531)	P
Provider characteristics
Quarter of fellowship training, median (IQR)	4 (2–7)	3 (1–6)	<.001
Fellow was the first airway provider, n (%)	1438 (81)	446 (84)	.17
Patient characteristics
Patient gestational age at birth, median (IQR), wk	28 (25–36)	27 (25–32)	<.001
Patient wt at time of intubation, median (IQR), g	1730 (950–3000)	1176 (736–2440)	<.001
Patient wt <1000 g at time of intubation, n (%)	476 (27)	217 (41)	<.001
Indication for intubation,a n (%)
Oxygen failure	441 (25)	144 (27)	.32
Ventilation failure	414 (23)	132 (25)	.50
DR: clinical indication	382 (22)	119 (22)	.70
Practice characteristics, n (%)
Intubation in NICU location	1255 (71)	366 (69)	.34
Use of video laryngoscopy	633 (36)	112 (21)	<.001
Paralytic premedication use	747 (42)	137 (26)	<.001

^aMore than 1 is possible; only the most common 3 are listed.

We included 40 fellows from 4 US training programs in the CUSUM analysis. Of these, 18 (45%) fellows achieved procedural competence during the period assessed, and 12 (30%) met the failure threshold (Fig 1). Among fellows who achieved competence, the median number of intubations to meet this threshold was variable, with a median of 18 (interquartile range [IQR] 15–25) and absolute range of 8 to 46 intubations. The median number of intubations to cross the deficiency threshold was 7 (IQR 5–14), with some fellows remaining deficient despite exposure to 50 intubation encounters (Supplemental Table 4). Each program had at least 1 fellow who met the deficiency criteria.

FIGURE 1.

CUSUM curve for tracheal intubation by NPM fellows. The lines demonstrate the learning curves of individual fellows, represented by different colors. The h1 and h0 lines demonstrate upper and lower control limits. Success is displayed as a decrease in CUSUM. A priori defined competency criteria was met if the fellow crossed h0, whereas deficiency occurred if the fellow crossed h1.

The remaining 10 fellows did not cross the success or failure thresholds during the period assessed. These fellows may not have performed a sufficient number of intubations to reach competence during the study period. The study period did not encompass all years of fellowship training in some cases, but other trainees remained undifferentiated at the completion of fellowship training (Supplemental Table 4).

We performed post hoc sensitivity analyses using variable definitions for competence. None of the fellows achieved procedural competency, defined more stringently as 90% success within 2 attempts. There were 29 of 40 (73%) of fellows who reached competence by using a less conservative definition of 70% success within 2 attempts. For those trainees, a median of 7 (IQR 4–11) intubations was needed to achieve this less stringent competence definition.

Across the entire cohort, intubation encounters were more frequent in the first year of training (quarters 1–4), and intubation success rates improved with advancing quarter of fellowship training (Fig 2). In univariable analysis, successful intubation encounters were performed by fellows who were slightly more advanced in the quarter of fellowship training (median quarter: 4; IQR 2–7), compared with fellows involved in unsuccessful intubations (median quarter: 3; IQR 1–6; P < .001; Table 2). In the mixed effects logistic model adjusted for paralytic premedication, video laryngoscopy use, and patient weight <1000 g, increasing quarter of fellowship training was significantly associated with an increasing odds of intubation success (adjusted odds ratio: 1.10; 95% confidence interval 1.07–1.14; Table 3).

FIGURE 2.

Intubation success rates with 95% confidence intervals by fellowship training quarter. Data are presented from 2297 intubation encounters performed by 92 NPM fellows. Success was defined as successful intubation within 2 attempts.

TABLE 3.

Multivariable Model of Factors Independently Associated With Intubation Success

Characteristic	Adjusted Odds Ratio	95% Confidence Interval	P
Quarter of fellowship training	1.10	1.07–1.14	<.001
Paralytic premedication	1.77	1.35–2.33	<.001
Video laryngoscopy	1.67	1.19–2.33	.003
Patient wt <1000 g	0.61	0.48–0.77	<.001

Discussion

Neonatal intubation is a critical procedure for neonatologists, but the amount of exposure needed for novice providers to acquire this skill is unknown. We were able to leverage data collected in a large multicenter neonatal intubation registry to address this question. Although intubation success rates generally increased with ongoing fellowship training, there was important variability in the number of intubations needed for individual fellows to achieve procedural competence. Equally important, some fellows were identified as deficient or undifferentiated (neither competent nor deficient), despite having exposure to 50 intubations. These findings suggest an individualized approach to assess trainee’s acquisition of intubation competence may be more useful than a uniform quota of intubation encounters.

To our knowledge, this is the first study to describe the number of neonatal intubation attempts required to achieve competence in NPM fellows. Although neonatal intubation is designated a core competence for pediatric residents by the Accreditation Council for Graduate Medical Education (ACGME) and Royal College of Physicians and Surgeons of Canada, contemporary pediatric residents have few opportunities to participate in intubations.^{9,11,16,21–23} Therefore, neonatal intubation skill acquisition occurs almost exclusively in neonatology fellowships.

Previously described intubation success rates range from 52% in a single attempt to 78% within 2 attempts across for neonatology fellows, regardless of the training year or quarter.^7,9,11,16 In the study by Haubner et al,¹¹ third-year fellows achieved 70% success rates (similar to attending physician success rates of 72%), compared with 44% for first-year fellows and 20% for pediatric residents. Our results were similar, with 77% of intubations successful across all fellows in aggregate. Additionally, we identified improving success rates by each quarter of fellowship training.

Among fellows who achieved competence in our study, the median number of intubations to reach this threshold was 18 (IQR 15–25). The number of intubation attempts required for trainees to develop competence in other disciplines ranges from 26 to 75. This contrast is likely due to differences in definitions for procedural competence, which was set at a 90% first attempt success rate in other disciplines.^12–15,24 Consistent with previous neonatology literature, we defined competence more leniently, as an 80% success rate.^8,9 This was also informed by reported neonatology attending physicians’ success rates of 64% for the first attempt and 72% to 88% within multiple attempts.^7,11,16

As has been previously described, changing definitions for acceptable and unacceptable failure thresholds yielded different results.²⁵ In the post hoc sensitivity analysis, no fellows in our study met the more stringent competence threshold of 90% success within 2 attempts. Going forward, it may be valuable for accreditation agencies and medical educators to define uniform competence and acceptable failure rates that can be used across studies.

Perhaps the most notable study finding was the variability among fellows’ progressive intubation performance, with several trainees not improving or worsening over time. Training programs have often used the number of procedural experiences as criteria for determining procedural competence.²⁶ Repeated practice has generally been associated with acquisition of skilled motor tasks in adults both in and outside of medicine.^27–29 However, our results suggest that repeated clinical exposure alone may not result in competence for some trainees.

Programs should develop methods to identify fellows who are unlikely to intubate successfully at an acceptable rate after fellowship completion. A real time cumulative assessment of procedural performance, such as a CUSUM plot, could help training programs assess learners’ progression toward competence and identify trainees who require additional educational interventions. In the current study, most fellows who met the deficiency threshold did so within 5 to 14 intubation encounters. In addition, given that more intubation encounters occurred in the earlier quarters of training, it is possible that some fellows’ procedural skills degrade later in training. The CUSUM method could be applied to assess skill retention by depicting whether providers continue to meet the defined competence threshold over time.

Future studies should focus on optimal methods to train fellows who are unlikely to achieve procedural competence through repeated clinical exposure alone. Deliberate practice with tailored feedback in both the simulated and clinical setting is a promising strategy.^30–32 In addition, practice characteristics, such as paralytic premedication and video laryngoscopy, were independently associated with an improved odds of procedural success; implementing these strategies could optimize trainees’ intubation success and skill acquisition. Finally, we observed important differences in the number of intubations performed by fellows at similar stages of training in our cohort. Training programs may need to strategically allocate intubation procedures for clinicians who require neonatal airway management skills and provide additional intubation training opportunities, such as rotations with anesthesiology in the operating room.

We acknowledge study limitations. The CUSUM method assesses a sequential count of encounters and does not account for the interval of time between procedures. In addition, we do not have information about fellows’ intubation experiences before starting fellowships. We anticipate that most fellows had limited intubation experience in residency, given the decline in resident participation in intubation.^{9,11,16,21,22} Ultimately, this potential variability in baseline skill at the start of training further supports an individualized approach to assessing competence, rather than establishing a uniform quota for trainees. Finally, we were unable to assess intubations performed outside of the NICU or DR settings (such as on transport). However, all centers were selected on the basis of having at least 90% data capture for fellows when accounting for intubations in other locations.

Study strengths include analysis of a large number of intubations captured prospectively by using standardized operational definitions in the NEAR4NEOS registry. We were able to link individual fellow identifiers with intubation data captured in the registry to characterize individual fellows’ intubation learning curves. The CUSUM method provides more granular information about providers’ progressive procedural performance, compared with assessing the proportion of successful intubations among all historical encounters. In addition, the multi-site NEAR4NEOS setting increases the generalizability of our study results.

Conclusions

The number of neonatal intubations required to achieve procedural competence is variable among fellows, and the overall intubation competence rates are modest. Some trainees did not improve or even worsened with repeated intubation encounters. These results provide insights for neonatal educators regarding the optimal number of intubation encounters required to achieve procedural competence, tools to identify fellows at risk for not achieving competence, and the need to individualize intubation training for trainees.

Glossary

CUSUM

cumulative sum

DR

delivery room

IQR

interquartile range

NEAR4NEOS

National Emergency Airway Registry for Neonates

NPM

neonatal-perinatal medicine