Patient safety events in out-of-hospital paediatric airway management: a medical record review by the CSI-EMS

Abstract

Objective

To describe the frequency and characterise the nature of patient safety events in paediatric out-of-hospital airway management.

Methods

We conducted a retrospective cross-sectional medical record review of all ‘lights and sirens’ emergency medicine services transports from 2008 to 2011 in patients <18 years of age in the Portland Oregon metropolitan area. A chart review tool (see online supplementary appendix) was adapted from landmark patient safety studies and revised after pilot testing. Expert panels of physicians and paramedics performed blinded reviews of each chart, identified safety events and described their nature. The primary outcomes were presence and severity of patient safety events related to airway management including oxygen administration, bag-valve-mask ventilation (BVM), airway adjuncts and endotracheal intubation (ETI).

Results

From the 11 328 paediatric transports during the study period, there were 497 ‘lights and sirens’ (code 3) transports (4.4%). 7 transports were excluded due to missing data. Of the 490 transports included in the analysis, 329 had a total of 338 airway management procedures (some had more than 1 procedure): 61.6% were treated with oxygen, 15.3% with BVM, 8.6% with ETI and 2% with airway adjuncts. The frequency of errors was: 21% (71/338) related to oxygen use, 9.8% (33/338) related to BVM, 9.5% (32/338) related to intubation and 0.9% (3/338) related to airway adjunct use. 58% of intubations required 3 or more attempts or failed altogether. Cardiac arrest was associated with higher odds of a severe error.

Conclusions

Errors in paediatric out-of-hospital airway management are common, especially in the context of intubations and during cardiac arrest.

Strengths and limitations of this study.

• This study provides an in-depth look at paediatric out-of-hospital airway management from a patient safety perspective.

• It uses a rigorously developed chart review process.

• This study includes all emergency medicine service-performed airway management interventions from a specific geographic area during the study period.

• It was conducted in a single large urban area in the USA and results may not be generalisable to other geographical areas.

• This is a retrospective study.

Introduction

The US National Quality Forum defines ‘patient safety’ as: the prevention and mitigation of harm caused by errors of omission or commission that are associated with healthcare.¹ The patient safety movement in medicine, triggered by retrospective studies of clinical care, started two decades ago and has triggered massive efforts to improve care in hospital-based medicine.^{2 3} Out-of-hospital care, in particular paediatric care, has little published literature regarding patient safety and the nature of safety events is largely unknown. This knowledge gap has limited our ability to improve care on a system-based level.

Airway management is a critical component of resuscitation during many paediatric emergencies and includes a set of technical procedures that are potentially high risk for errors. The skill set required for paediatric out-of-hospital airway management includes oxygen administration, bag-valve-mask ventilation (BVM), oral and nasal airway insertion, supraglottic device insertion, and endotracheal intubation (ETI). ETI has long been considered ‘definitive’ airway management for patients of any age and is practised by many emergency medicine service (EMS) agencies throughout the world.⁴ Paediatric ETI is taught in paramedic training programmes and is part of the US National Registry of Emergency Medical Technicians (NREMT) practical examination.⁵

Although paediatric ETI is considered an essential skill for paramedics, the single existing controlled trial found no benefit compared with BVM and reported harm in some subgroups.⁶ Other studies have demonstrated low success rates for paediatric out-of-hospital ETI and increased complications compared with in-hospital ETI.^7–10 In addition, paramedics perform paediatric ETI infrequently, perhaps only once every 5 years, and rapidly lose skills after training.^11–13 As a result of these factors, ETI for children in the out-of-hospital setting is controversial. There are little data on other aspects of out-of-hospital paediatric airway management such as airway adjuncts and supraglottic devices. Several existing paediatric studies have been conducted in patient simulators and found high success rates; however, an adult study on the laryngeal mask airway demonstrated high success in patient simulators (100%) with substantially lower success in practice (64%).^14–17 A national Delphi study recently found that airway management is the most high-risk scenario for errors in paediatric out-of-hospital care.^{18 19} Another recent study performed in a large national database found that intubation remains the most commonly used paediatric advanced airway technique out-of-hospital with lower success rates than in adults, with the lowest success being among patients <1 year of age.²⁰ Though this study addressed success rates, it did not include a detailed review of the charts and was thus unable to identify the rates of specific types of errors in airway management such as tube depth, tube size and the potential harm associated with the errors.

The objective of this study is to describe the rates and nature of patient safety events related to paediatric out-of-hospital airway management in a cohort of critically ill paediatric transports from a large metropolitan area.

Methods

Study design and setting

This is a retrospective cross-sectional study representing one portion of the Children's Safety Initiative-EMS (CSI-EMS) using a chart review designed to capture a broad range of potential and manifest safety events. The CSI-EMS is a National Institutes of Health (National Institute of Child Health and Human Development (NICHD) R01HD062478) funded mixed-methods study. The goal of the CSI-EMS is to identify, describe and classify the occurrence of safety events in out-of-hospital paediatric emergency care.²¹ We subclassified safety events as: unintended injury or consequences, near misses, suboptimal actions, errors and management complications. This classification scheme had not been previously used but was developed by patient safety experts on our research team to capture the spectrum of errors as broadly as possible.

In this report, we present a retrospective medical record review of individual EMS transports in the Portland, Oregon (USA) metropolitan area. This metro area has a ‘dual advanced life support (ALS)’ system where separate ALS fire and transport agencies respond to all calls. The transport and fire agencies in this study serve a population of over 700 000 residents. Airway management procedures in scope of practice in this system include oxygen administration, BVM, oral and nasal airways, supraglottic airways (King LT), and ETI including rapid sequence intubation. The cardiac arrest survival in Portland, OR is among the highest in the USA, with EMS treated cardiac arrest survival of 10.4% reported in a 2008 study using the Resuscitation Outcomes Consortium sites. These high survival rates most likely result from rapid response times and an effective (or ‘high functioning’) EMS system.²²

The transport agency units respond with two paramedics in each ambulance. Fire units include four to five person teams with at least one paramedic per team. Though fire units respond to each call, the transporting units can elect to dismiss them. If the fire department responders do not intervene, they do not complete a chart. In this system, fire units respond to 90% of calls in <7 min and transport units respond to 90% of calls in <8 min. Fire department crews work in 24-hour shifts. All transport paramedics work rotating shifts and experienced paramedics are equally distributed during all times of day.

Paramedics in this system are all required to participate in annual airway management training using simulation with adult, paediatric and infant manikins. In addition to this training, they are all required to maintain Pediatric Advanced Life Support certification. There were no specific protocols designating which paramedic, among those responding, would perform the intubation. The protocol for paediatric intubation, at the time of the study, called for using BVM or a rescue device if two attempts at intubation failed. However, at the time of the study, the King LT was the rescue device being used and it was not available in sizes suitable for most children under the age of 8. Providers did have access to oral and nasal airways in all paediatric sizes.

Selection of participants

We reviewed records for transports from 2008 to 2011 for all patients <18 years of age who were transported ‘code 3’ (lights and sirens) indicating a critical transport. This transport priority is used at the discretion of the treating providers for patients felt to have a life-threatening or limb-threatening condition. Reviews included charts from the transporting agency and, when applicable, the fire response unit. This group was chosen to identify a subset of patients more likely to need and receive interventions.

Chart reviewtool development

Our chart review tool and review methods were based on forms from the Harvard Medical Practice Study and the Utah and Colorado Medical Practice Study that were the foundation for the Institute of Medicine Report on patient safety in medicine.²³ In turn, the chart review tool was adjusted to the out-of-hospital setting based on results from our EMS focus group study.^{2 3 21} The tool was iteratively revised and finalised in several rounds of testing which included ‘talk aloud’ sessions and pilot reviews using 30 sample EMS charts.

Given the lack of data defining patient safety events in paediatric EMS, the presence or absence of a safety event was based on judgement of the expert chart reviewers. Given the somewhat subjective nature of judging certain potential safety events, each chart was reviewed in tandem by a paramedic and emergency physician using a standardised review tool. A paediatric emergency physician, with expertise in paediatric EMS, performed a third review to arbitrate differences between the initial reviews. Chart reviewers could not be blinded to the study objectives since the review tool specifically focused on safety events. However, reviewers were blinded to results of all interim analysis and study hypotheses. All chart reviewers received a 2-hour training session on the chart review tool, completed test cases before and after the session, and were provided feedback on their test reviews. Based on results of test reviews and questions during feedback sessions, a guidebook was created and provided to all reviewers as a resource to ensure consistency and quality. Reviews were completed online via SurveyMonkey. The expert chart review panel consisted of 13 paramedics and 7 physicians who were not affiliated with the agencies submitting charts for review. Paramedic reviewers were recruited by word of mouth in the local EMS community, and physician reviewers included one paediatrician and emergency physicians who worked in the paediatric emergency department of the metro area's academic medical/trauma centre and provided online medical control and one paediatrician.

As a final quality control measure, all charts were arbitrated by one of two paediatric emergency physician investigators with experience in EMS. Inter-rater reliability was established between the two paediatric emergency physician investigators prior to arbitration. A κ statistic of 0.615 was achieved between the two arbiters on the presence or absence of safety events in the major domains described below. Following the initial blinded review to establish inter-rater reliability, the arbiters met and discussed all discrepancies in the reviews to achieve consensus.

The review tool was designed to identify errors in the following major domains: resuscitation; assessment, impression/diagnosis and clinical decision-making; airway/breathing; fluids and medication; procedures; equipment; environment; and system. Where available, data were abstracted from the chart electronically (scene time, transport time, time of day). Through a series of checkboxes, Likert-type questions and open-ended responses, the reviewers manually abstracted chart data, not available electronically, including age, sex, weight, scene location and transport priority. The reviewers then identified details of care including the dispatch complaint, clinical impression, all procedures performed including airway management procedures, whether or not an error occurred with any of the procedures or in other specific domains, details about the nature and cause of the error, and the degree of potential harm to the patient. The degrees of harm were assessed by the chart review tool using the following question: Using your best clinical judgement, to what degree could the (specific domain inserted, eg, airway management) issue have harmed the patient? The following were the potential responses: (1) no harm likely or a near miss, (2) mild or temporary harm, including additional treatment and (3) permanent or severe harm including death.

Analysis

First, we identified the cohort of patients who had airway management procedures including oxygen administration, BVM, airway adjuncts (oral, nasal, supraglottic airways) and ETI out of the complete study population of critical transports as indicated by the chart reviewers. Next, we performed a descriptive analysis of the age, sex, scene location and dispatch complaint of all patients who had airway management interventions. We then described the clinical impression of the paramedics documented in the electronic patient care report in cases where airway management procedures were performed. We then identified all patients who had an error in airway management indicated by the reviewers. The reviewers also described the potential severity of harm caused by each error using three categories including ‘no harm’, ‘mild or temporary harm’ and ‘permanent or severe harm including death’ using their best judgement based on the information available in the EMS patient care report. The reviewers described the nature of the error using a free-text field where problems, such as ‘tube too deep’, could be entered. This list of variables was created based on a priori hypotheses developed by the study team of factors we felt could be associated with errors based on the experience of EMS professionals on the team as well as previous studies. Univariate analysis was performed on variables thought to be predictors of severe airway error and then used to select variables to be put in a regression model for severe airway error. Univariate analysis of continuous variables versus severe airway error was performed using logistic regression. Variables with a p value <0.2 from univariate logistic regression were entered into the model.

Univariate analysis of categorical variables was performed using χ² analysis. Fisher's exact test was used when there were small counts. The φ coefficient or Cramer's V was used to measure the strength of association between severe airway error and variables for which the χ² test was found to be significant. Variables with a mild association to very strong association (|φ or Cramer's V| ≥0.15) were entered into the regression model. Variables with a mild-to-weak association that were suspected to be effect modifiers were entered into the model. One variable with a φ or Cramer's V >0.40 but <0.50 was not entered into the model (reason for dispatch) because of concerns that the variable distorted the parameters of the other predictor variables, that is, collinearity with other predictor variables.

When performing χ² analysis on nominal variables, categories with cell counts <5 were not included in the analysis. ORs and CIs from 2×2 tables for categorical variables are reported in table 2 with the results of the univariate analysis.

Table 2.

Univariate analysis of factors associated with a severe airway management error*

Characteristic	OR	95% CI	p Value
Patient characteristic
Age†			<0.001
0–28 days	6.62	1.68 to 26.0
29 days–11 months	5.73	1.91 to 17.2
1–5 years	1.07	0.33 to 3.47
6–11 years	1.65	0.46 to 5.99
12–17 years (reference category)	1	–	–
Gender‡,§			0.51
Male	0.79	0.39 to 1.60
Call characteristic
Reason for dispatch†,¶			<0.001
Trauma (reference category)	1	–	–
Seizure or ALOC	0.14	0.02 to 1.12
Respiratory distress	1.4	0.47 to 4.16
Cardiorespiratory arrest	15.61	5.65 to 43.1
Other (including birth/delivery)	0.95	0.05 to 18.5
Fire department involvement§			0.006
Fire chart	2.75	1.30 to 5.82
Call during the night§			0.032
(22:00–8:00)	2.21	1.05 to 4.62
Scene time**
(per 1 min increase)	1.03	0.99 to 1.07	0.14
Transportation time
(per 1 min increase)	0.96	0.91 to 1.01	0.10

*Three hundred and thirty-eight observations (338 out of 490 charts reviewed; airway error information not available for 152 charts).

†ORs and CIs calculated from 2×2 tables of each category against the reference category. Reported p value from a Fisher's exact test performed on all categories. Cramer's V was found to be >0.20 from analysis of a table of age group and severe airway management error that included four age groups (0 days–11 months, 1–5, 6–11 and 12–17 years) so that the expected value for each cell in the table was >5.

‡Three hundred and thirty-six observations (gender not available for two charts).

§ORs and CIs calculated from 2×2 tables. Reported p value from a χ² test on the 2×2 table; −0.20<φ<0.20.

¶ORs and CIs calculated from 2×2 tables of each category against the reference category, except for any categories for which the 2×2 table contained a zero cell. Reported p value from a Fisher's exact test performed on all categories. Cramer's V was found to be >0.20 from analysis of a table of reason for dispatch and severe airway management error that included three of the categories of reason for dispatch (trauma, cardiac arrest and respiratory distress) so that the expected value for each cell in the table was >5.

**Three hundred and thirty-seven observations (scene time not available for one chart).

ALOC, altered level of consciousness.

The study team reviewed the free-text responses regarding the specific nature of the airway management errors (eg, tube too deep) and then divided them into specific categories. In addition, one study team member reviewed each cardiac arrest chart and abstracted the airway intervention and medication administration times from the event log in the patient care report to perform an analysis of the time to first epinephrine administration in patients who had advanced airway attempts versus those who did not as well as total time spent prior to securing the airway (defined as the interval between time of first patient contact and documented success of ETI). Analysis was conducted using SAS software V.9.4 (SAS Institute, Cary, North Carolina, USA).

Results

From a total of 11 328 paediatric transports during the study period, we identified 497 ‘code 3’ transports (4.4%). We eliminated two transports due to missing data and five transports because we were unable to obtain the associated fire department chart, leaving a total of 490 charts for review. Table 1 presents the characteristics of the 490 paediatric patients and details for those who received specific airway interventions. Approximately two-thirds of patients who were transported code 3 had an airway management intervention of some kind (329/490, 67%) and 25.51% of patients had more than one airway management intervention. The median number of interventions was ‘1’. Of those transported code 3, 61.6% (302/490) were treated with oxygen, 15.3% (75/490) with BVM, 8.6% with ETI (42/490) and 2.0% (10/490) with airway adjuncts. Half of all intubations (21/42) were performed in children <12 months of age. The most common reason for dispatch among patients with airway management procedures besides oxygen administration was cardiopulmonary arrest, the second most common reason was seizure and the third most common reason was trauma. In contrast, the most common reason for dispatch in the entire study population was trauma, followed by nearly equal proportions of interfaculty transports and seizures/altered level of consciousness.

Table 1.

Characteristics of patients who had airway management procedures

	All patients	Any airway management	Oxygen	BVM	Airway adjuncts*	Intubation
	N=490	N=329	N=302	N=75	N=10	N=42
	n (%)	n (%)	n (%)	n (%)	n (%)	n (%)
Age
0–28 days	23 (4.7)	18 (5.5)	12 (4.0)	7 (9.3)	0 (0.0)	6 (14.3)
29 days–11 months	61 (12.5)	50 (15.2)	41 (13.6)	22 (29.3)	3 (30.0)	15 (35.7)
12 months–5 years	169 (34.5)	117 (35.6)	110 (36.4)	19 (25.3)	0 (0.0)	6 (14.3)
6–11 years	84 (17.1)	56 (17.0)	55 (18.2)	7 (9.3)	1 (10.0)	2 (4.8)
12–17 years	153 (31.2)	88 (26.8)	84 (27.8)	20 (26.7)	6 (60.0)	13 (31.0)
Female	195 (39.8)	134 (40.7)	120 (39.7)	33 (44.0)	3 (30.0)	20 (47.6)
Scene location
Home	219 (44.7)	174 (52.9)	154 (51.0)	51 (68.0)	6 (60.0)	31 (73.8)
School	25 (5.1)	9 (2.7)	9 (3.0)	3 (4.0)	1 (10.0)	1 (2.4)
Street/highway	117 (23.9)	64 (19.5)	63 (20.9)	10 (13.3)	1 (10.0)	7 (16.7)
Hospital/clinic	110 (22.5)	69 (21.0)	63 (20.9)	9 (12.0)	0 (0.0)	2 (4.8)
Recreation/sport	14 (2.9)	10 (3.0)	10 (3.3)	2 (2.7)	2 (20.0)	1 (2.4)
Other	5 (1.0)	3 (0.9)	3 (1.0)	0 (0.0)	0 (0.0)	0 (0.0)
First responder
Ambulance	183 (37.4)	121 (36.8)	112 (37.1)	20 (26.7)	5 (50.0)	13 (31.0)
Fire	181 (36.9)	129 (39.2)	119 (39.4)	33 (44.0)	3 (30.0)	18 (42.9)
Police	14 (2.9)	6 (1.8)	6 (2.0)	0 (0.0)	0 (0.0)	1 (2.4)
Unknown	112 (22.9)	73 (22.2)	65 (21.5)	22 (29.3)	2 (20.0)	10 (23.8)
Reason for dispatch
Cardiopulmonary arrest	35 (7.1)	35 (10.6)	19 (6.3)	33 (44.0)	5 (50.0)	26 (61.9)
Trauma	203 (41.4)	94 (28.6)	91 (30.1)	11 (14.7)	2 (20.0)	10 (23.8)
Seizure or ALOC	107 (21.8)	95 (28.9)	93 (30.8)	17 (22.7)	1 (10.0)	2 (4.8)
Ingestion/poisoning/intoxication	29 (5.9)	15 (4.6)	15 (5.0)	4 (5.3)	2 (20.0)	2 (4.8)
Respiratory distress	79 (16.1)	73 (22.2)	68 (22.5)	9 (12.0)	0 (0.0)	2 (4.8)
Allergic reaction/anaphylaxis	11 (2.2)	7 (2.1)	7 (2.3)	1 (1.3)	0 (0.0)	0 (0.0)
Pain (non-trauma)	12 (2.5)	5 (1.5)	5 (1.7)	0 (0.0)	0 (0.0)	0 (0.0)
Other	14 (2.9)	5 (1.5)	4 (1.3)	0 (0.0)	0 (0.0)	0 (0.0)

*Airway adjuncts: airway adjuncts in the chart review included nasopharyngeal or oropharyngeal airways n=9 and one King LT supraglottic airway device.

ALOC, altered level of consciousness; BVM, bag-valve-mask ventilation.

Overall, 27.8% (94/338) of charts with airway management were judged to have airway management errors. Errors were related to oxygen use in 21% (71/338) of cases, to BVM in 9.8% (33/338), to airway adjunct use in 0.9% (3/338) and to ETI in 9.5% (32/338). Nine per cent of cases had errors in more than one aspect of airway management.

Over 70% of cases in which ETI was performed contained an error of some type. The specific nature of errors relating to ETI were: endotracheal tube (ETT) placed too deep in 25% of cases, incorrect ETT size used in 29% (78% of the time too small) and dislodgement of the ETT during transport in 13% of cases. Reviewers felt that 24% of the time, ETI was not indicated when performed. The most common type of error in ETI was technical difficulty with the procedure, accounting for 58% of errors, defined as failure to secure ETI or requiring three or more attempts.

The most common clinical scenario in which ETI was performed was cardiac arrest. In cardiac arrest scenarios, intubation was attempted 74% of the time. On average, in cardiac arrest cases where ETI was attempted, the mean time to the first dose of epinephrine was 11 min. In cases where ETI was not attempted, the mean time to the first dose of epinephrine was 7 min. In addition, in cases where ETI was attempted, 10 min passed on average prior to securing an airway or abandoning further attempts. This may indicate that on average providers conclude advanced airway management prior to administration of epinephrine.

Further analysis of other types of airway management errors found that 48% of the time, oxygen was administered when it was not indicated, and 32% of the time oxygen was not administered when it was indicated. There was a broad range of clinical scenarios where oxygen was applied and reviewers felt it was not indicated. Errors related to BVM were most commonly due to lack of use when indicated or failure to attempt for a long enough period prior to performing ETI. Airway adjuncts were infrequently used (n=10) and all errors identified were related to lack of use when indicated, rather than inappropriate use or a complication related to their use.

Figure 1 presents the distribution of errors in airway management by age. Neonates had the highest rate of severe errors followed by infants, with a trend towards decreased rate of severe errors with increasing age. Figure 2 shows the level of harm of the errors according to airway management procedures. Intubation had the highest rate of severe errors among the airway management procedures.

Figure 1.

Severity of airway management errors by patient age. O₂, oxygen; BVM, bag-valve-mask ventilation; UNSEM, unintended consequence, near miss, suboptimal action, error, management complication.

Figure 2.

Severity of airway management errors by type of procedure. O₂, oxygen; BVM, bag-valve-mask ventilation; UNSEM, unintended consequence, near miss, suboptimal action, error, management complication.

Errors in airway management were commonly associated with errors in other domains of care. In cases with any airway management error, 64% had errors in decision-making, 62% in resuscitation and 48% in medications (p<0.01 for all). In cases where there were errors in BVM or ETI, more than 90% of cases also had an error identified in resuscitation, over 65% had errors in assessment/decision-making and over 60% had errors in medications (p<0.01 for all).

We performed univariate analyses to identify factors associated with severe airway management errors (table 2). In these unadjusted analyses, an increased risk of a severe error related to airway management was strongly associated with patient age <28 days (OR 6.62, 95% CI 1.68 to 26.0), <12 months (OR=5.73, 95% CI 1.91 to 17.20) patients in cardiopulmonary arrest (OR=15.61, 95% CI 5.65 to 43.12), fire department involvement (fire department team and transport team both provided significant care; OR=2.75, 95% CI 1.30 to 5.82), and calls occurring between 22:00 and 08:00 (OR=2.21, 95% CI 1.05 to 4.62).

In multivariate logistic regression analyses (table 3), 0–28 days and age <12 months remained statistically significant with strong associations with severe errors. We noted that fire department involvement was associated with increased odds of severe errors, and increased transport time was mildly protective against severe errors, while increased scene time was associated with increased odds of severe errors.

Table 3.

Multivariate analysis of factors associated with a severe airway management error*†

Characteristics	OR	95% CI	p Value
Patient characteristic
Age
Newborn (0–28 days)	7.50	2.11 to 26.6	<0.001
Infant (29 days through 11 months)	5.69	1.13 to 28.8	0.02
1–5 years	1.52	0.34 to 6.86	0.22
6–11 years	0.96	0.4 to 3.92	0.43
12–17 years (reference category)	1	–	–
EMS scene care
Fire department involvement	3.91	1.55 to 9.85	0.004
Transport time in minutes	0.94	0.88 to 1.00	0.04
Scene time in minutes	1.04	1.00 to 1.09	0.03

*Three hundred and thirty-one observations (out of 490 charts, after excluding charts with missing info on airway errors, gender, and scene time and excluding outliers); likelihood-based pseudo R²=0.12 maximum-rescaled R²=0.26.

†Four outliers with a standardised deviance residual of >2.50 were removed from the final model.

Discussion

In this analysis, we found that airway management errors are relatively common, occurring in >25% of cases where airway management was performed, and that a high proportion of errors had the potential to cause severe patient harm. Patients <12 months of age and those with cardiac or respiratory arrest were at highest risk for severe airway management errors and were the patient populations who most commonly needed airway management. Of airway management procedures, ETI had the highest proportion of errors and required three or more attempts or was unsuccessful 58% of the time bringing into question the safety of this procedure. Airway adjuncts and BVM were judged to be underused.

Several previous investigations have documented paediatric ETI success rates from 50% to 100%, and major complications between 1.8% and 50%.^24–27 The only controlled trial on out-of-hospital paediatric ETI found no benefit compared with BVM but has been criticised for inadequate training.⁶ Our study also provides additional insight into the nature of ETI related errors by identifying incorrect tube size, incorrect tube depth, multiple attempts, failure to successfully place the tube and unnecessary intubation as the specific types of errors.

We also found that patients with cardiac arrest had significantly increased odds of an airway management error. In these cases, EMS providers may face cognitive and emotional overload.²⁸ Similar to our study, an analysis of more than 10 years of data from San Diego, California, found that the majority of paediatric out-of-hospital intubations are for cardiac arrest, indicating that this is an important area of focus.²⁹

Infants experience a disproportionate portion of airway management errors. The airway anatomy of children <1 year of age differs substantially from that of older children. In addition, the small size of the patient makes performing simultaneous interventions such as airway management and cardiopulmonary resuscitation (CPR) challenging. In cases with airway management errors, we also found high rates of errors in resuscitation, assessment/decision-making and medication administration. This indicates that airway management errors are not likely due to one specific skill deficit, but while multiple critical decisions and interventions are being performed quickly.

One reason paediatric ETI may not benefit patients is that the potential advantages of an endotracheal tube securing the airway are mitigated by procedural complications and by detrimental effects on other aspects of resuscitation. A recent study carried out in the emergency department found that significant adverse events increase rapidly with the number of intubation attempts.³⁰ We found that in cases of cardiac arrest, on average, ETI was being performed prior to administration of epinephrine, which contradicts American Heart Association guidelines.^{31 32} This suggests that ETI may have a negative effect on other important components of out-of-hospital paediatric resuscitation.

Our analysis adds to the literature by evaluating a broad range of airway interventions. We found that BVM and airway adjuncts were at times not used when indicated. Previous simulation studies have found technical problems with prehospital BVM, which may reflect the challenging nature of this procedure in children or lack of experience in its use.^{33 34} In addition, BVM in paediatric patients may be resource intensive and require two providers to achieve adequate mask seal.

Airway adjuncts such as supraglottic devices, oropharyngeal airway (OPAs) and NPAs appear to be used very infrequently in children, even when indicated. Simulation-based studies have shown high success rates with supraglottic devices in paediatric manikins placed by EMS providers and may be a promising solution.^{16 35}

We found that the presenting complaint of seizure/altered level of consciousness trended towards decreased odds of errors. We believe that this is because these patients were most likely overtriaged to code 3 transport, given the benign natural course of most paediatric seizures. Errors were not unexpectedly associated with calls during the night-time hours. Increased scene time was associated with increased odds of errors most likely reflecting increased time at risk. Increased transport time was associated with reduced odds of errors after controlling for scene time. This could be due to more stable patients at lower risk for errors being transported longer distances due to bypassing the nearest hospital in favour of a children's hospital a greater distance away. Finally, we found that fire department involvement had a significant association with increased odds of a severe error. This is most likely due to confounding by indication since locally the fire department only provides ongoing care to the most severely ill patients.

There are several potential mechanisms to improve the safety of paediatric airway management based on the results of this study. Each provider in our system undergoes paediatric airway simulation training, but it is clearly not adequate. Increased time in simulation training low cost models that are more realistic are potential solutions. Simulation training is resource intensive, so this may not be sustainable or generalisable. Given that the highest rate of errors was in the youngest patients, limiting intubation to older children may be reasonable. This study was not powered to detect differences by age group, so we cannot comment on what the correct age should be. Increased use of supraglottic airways may be a promising option in children as well, though further study is needed.

Limitations

Our study has several limitations. First, it was conducted in a single metropolitan area. Next, this is a retrospective study and limited to the data available in the medical record. Our reliance on the written record most likely biases our results towards underestimation of errors.³⁶ Though a standard definition was used, assessment of harm was based on the judgement of chart reviewers and is inherently subjective, although this practice has been used in other landmark patient safety studies.^{2 3} We used a rigorous chart review tool development and training process with multiple blinded reviewers to mitigate this source of bias. Finally, this study was conducted in a well-developed EMS system with short response times using paramedics as initial responders to all calls, available medication facilitated intubation, and with a relatively high cardiac arrest survival.²² In general, this is a ‘high functioning’ EMS system which may have lower rates of errors than in other systems. Our findings may not be generalisable to other EMS systems. Categories of nominal predictor variables with one or less case of severe airway error were not included when performing the χ² test on reason for dispatch; we chose to focus on categories of dispatch reason in which there was more than one severe airway error reported. Owing to the small number of cases in this sample, we cannot be sure that severe airway error would not be present in greater numbers in certain categories of reason for dispatch (poisoning, allergy or anaphylaxis, and pain) when looking at a larger sample. The small sample size precluded the inclusion of interaction terms to investigate possible effect modification in the multiple regression model.

In conclusion, our study finds high error rates in paediatric out-of-hospital airway management. ETI was the modality associated with the highest rate of errors, and the youngest patients as well as those with cardiopulmonary arrests were at highest risk of errors. These findings raise serious concerns about the safety of advanced airway management procedures. Future directions for this work could include expanding this research to other types of EMS systems and conducting prospective trials assessing the efficacy of different airway management strategies.