Anticipation of the difficult airway: preoperative airway assessment, an educational and quality improvement tool

Abstract

Background

Assessment of the potentially difficult airway (DA) is a critical aspect of resident education. We investigated the impact of a new assessment form on airway prediction and management by anaesthesia residents. We hypothesized that residents would demonstrate improvement in evaluation of DAs over the study duration.

Methods

After IRB approval, anaesthesia residents were randomized into two groups: control (existing form) and experimental (new form). Data were collected prospectively from August 2008 to May 2010 on all non-obstetric adult patients undergoing non-emergent surgery.

Results

Eight thousand three hundred and sixty-four independent preoperative assessments were collected and 8075 were analysed. The experimental group had the higher completion rate than the control group (94.3% vs 84.3%, P=0.001). DA prediction was higher for the control group (71.2%) compared with the experimental group (69.1%; P=0.032). A significant improvement in prediction rates was found over time for the experimental group (likelihood estimate=0.00068, P=0.031).

Conclusions

The use of a comprehensive airway assessment did not improve resident ability to predict a DA in an academic, tertiary-based hospital, anaesthesiology residency training programme.

Editor’s key points.

• This study addresses the impact of a comprehensive airway assessment form, including all 11 ASA's airway risk factors, on resident education, while assessing is of questionable value.

• Completion of this specially designed airway assessment form led to significantly better documentation; however, it did not appear to have a clinically significant impact.

Airway management remains one of the most important responsibilities of an anaesthetist,¹ yet documentation of the clinical assessment, which is a professional requirement,² is often incomplete.³ Poor airway management has been recognized as a serious patient safety concern for almost three decades,^4–7 highlighting the need for careful airway assessment before the induction of anaesthesia.^8–21 While improvements in patient monitoring,²² airway devices,²³ and clinical protocols and training^24–30 have reduced the risk associated with an unpredicted difficult airway (DA), these changes have not reduced the incidence of unexpected DAs in clinical practice. Since the consequences of an unanticipated DA are potentially catastrophic, proper education and training are a continued necessity.

This study addresses this gap in knowledge by evaluating the impact of a comprehensive airway assessment form on resident education, while assessing is of questionable value.⁸ We hypothesized that a new comprehensive airway assessment form would result in greater resident recognition of the 11 important airway features recommended by the ASA.¹ Based on this hypothesis, the overall aim of the present investigation was to document the effect of a more comprehensive airway assessment form on resident education.

Methods

This prospective, randomized, single-blind study was conducted from August 2008 to May 2010 at a Level 1 academic trauma center (Memorial Hermann Hospital, Texas Medical Center, Houston, TX, USA). After obtaining IRB approval (HSC-MS-07-0144), adult patients non-obstetric presenting for elective surgery requiring general anaesthesia, which did not already have their airway secured, were enrolled in the study. For patients who received more than one anaesthetic during the study period, data were collected independently for each anaesthetic encounter. Patients were provided with a written document describing the study and gave a verbal consent to participate.

All residents enrolled at the University of Texas Medical School at Houston anaesthesia residency programme for the academic year 2008–9 were recruited at the start of the study. An additional 24 incoming residents were enrolled during the second academic year (2009–2010) of the study, giving a total of 91 residents (Fig. 1). Each resident provided a verbal agreement to be enrolled on the quality improvement project. One resident joined the programme off-cycle in March 2008 and was enrolled in the study as a CA 1 (a first year anesthesia resident) (experimental group) for the remaining 2008–9 academic year and was also enrolled for the second year. No residents dropped from, or transferred in or out of the programme. Residents were randomized into two groups—an experimental group, which used the comprehensive airway assessment form in addition to the existing anaesthesia record, and a control group, which only used the existing anaesthesia record. Randomization was performed after stratification by year of training and based on 1:1 randomization. Experimental and control residents had a one-on-one tutorial (1 day, 3–4 practice assessments) on how to complete the data forms, respectively, followed by 1 month of validation at the beginning of each resident's rotation. Afterward, once a month, the residents' assessments were audited for quality control.

Fig 1.

Resident randomization. CA 1, first-year residents during first year of study; CA 2, second-year residents during first year of study; CA 3, third-year residents during first year of study; CA 1–1, first-year residents during second year of study; CA 1–2, second-year residents during second year of study; CA 2–3, third-year residents during second year of study.

The new comprehensive airway assessment form (Appendix 1) required a detailed assessment of the patient's airway history and physical examination compared with the existing anaesthesia preoperative assessment form (Appendix 2). A common form to the experimental and control group was used to collect postoperative outcome data (Appendix 3).

For the purposes of this study, difficult mask ventilation (DMV) was defined as difficulty in maintaining a mask seal and obtaining satisfactory capnography (end-tidal CO₂ and tidal volume).² If mask ventilation was attempted and determined to be difficult, the severity was graded either mild (requiring oral/nasal airway), moderate (can ventilate with assistance), severe (cannot ventilate with a facemask), or extreme (cannot ventilate with the supraglottic device). However, the use of neuromuscular blocking agent, type, dosage, time of administration, and use as a rescue were not included as outcome.

Difficult supraglottic airway (DSGA) was defined as either inability to physically place a supraglottic device or inadequacy of ventilation, oxygenation, or airway protection after placement that required conversion to an alternative technique. If placement of a supraglottic device was attempted, level of difficulty, number of attempts, and type of device were documented. Difficult direct laryngoscopy (DDL) was defined as the difficulty in visualizing any portion of the vocal cords after conventional laryngoscopy requiring more than one attempt.² If direct laryngoscopy was attempted, the type of blade, number of attempts, Cormack–Lehane grade, and any difficulties encountered were documented. Difficult intubation (DI) was defined as proper insertion of the tracheal tube with conventional laryngoscopy requiring multiple attempts.² If intubation was attempted, the number of attempts and the presence of any difficulties were documented. Difficult surgical airway (DSA) was defined as a difficult cricothyrotomy or tracheostomy, open or percutaneous, performed electively or emergently, to manage a DA due to bleeding, poor orientation and difficult instrumentation, and defined by the surgeon as technically difficult. DA was defined as the occurrence of DMV, DSGA, DDL, DI, or DSA. If a surgical airway was attempted, it was classified as either emergent or elective and either difficult or easy, with the number of attempts recorded.

Statistical analysis

Our database includes 9117 postoperative encounters; a small subset of patients received multiple anaesthetics (n=155, 1.7%). Patients who received multiple anaesthetics were excluded from statistical analysis. All statistical analyses were conducted using SAS 9.2 (SAS Institute, Cary, NC, USA). Data on the completeness of documentation were analysed using a χ² test; accuracy of prediction was defined as the sum of correct assessments and significance determined using a χ² test, while the changes of prediction accuracy over time among groups were analysed by a logistic regression. A P-value of <0.05 was considered significant.

Results

A total of 8364 independent preoperative assessments were completed. Incomplete assessments (>4 risk factors not completed) were excluded, resulting in a total of 8075 assessments included in our analysis. Three thousand three hundred and thirty-two (41%) were performed by the experimental group and 4743 by the control group (59%) (Fig. 2). A total of 1560 (17%) of all postoperative assessments (n=9117) were reported as DA.

Fig 2.

Assessment distribution. DMV, difficult mask ventilation; DDL, difficult direct laryngoscopy; DI, difficult intubation; DSGA, difficult supraglottic airway; DA, difficult airway.

No difficult surgical or invasive airways were reported. The frequency of each particular event was similar between all groups (Fig. 2), and ranged from 7.17% to 8.79% for DMV, 5.59% to 5.64% for DDL, 4.09% to 4.98% for DI, and 1.38% to 1.43% for DSGA.

Completeness of airway examination documentation

Results are shown in Table 1 and demonstrate significant differences between the two groups. The experimental group had a higher rate of completion than the control group (94.3% vs 84.3%; P<0.001).

Table 1.

Completeness of airway exam documentation. MP, Mallampati. χ² test comparisons of completeness for each variable between control and experimental were all statistically significant (P=0.005)

	Control	%	Experimental	%	P-value
Total records	4882	100.00	3372	100.00
MP complete	4581	93.83	3255	96.53	<0.00001
Neck mobility complete	4251	87.07	3265	96.83	<0.00001
Both complete	4114	84.27	3182	94.37	<0.00001
One complete	4718	96.64	3302	97.92	0.0038

Overall recognition of the DA

The experimental group correctly predicted a DA in 2397 out of 3471 patients (69.1%). The control group predicted 3551 out of 4984 patients (71.2%) correctly, which was significantly higher (P=0.032) than the experimental group (Table 2).

Table 2.

Accuracy of difficult airway prediction in the experimental and control groups. Per χ² test, statistical significance between the groups is P=0.032

	Correct	%	Incorrect	%	Total
Control	3551	71.20	1433	28.80	4984
Experimental	2397	69.10	1074	30.90	3471
Total	5948		2507		8455

Impact on resident education

Prediction accuracy for each day of the study was calculated and graphed as a 30 day moving average for the entire study period (Fig. 3). The multiple logistic regression model created to analyse the differences in the rates of accuracy between resident groups showed significant relationships with both patient factors (age, weight, Mallampati, jaw protrusion, interincisor distance, thyromental distance, sternomental distance, and neck circumference) and resident factors (day of study and cohort). Inclusion in the CA 1–1 cohort was the largest negative predictor of correctness (likelihood estimate −0.62, P=0.001) followed by the CA 2–3 cohort (likelihood estimate −0.36, P=0.0025). Compared with the CA 3 cohort, the odds ratios for both the CA 1–1 (0.267, CI 0.130–0.549) and CA 2–3 (0.347, CI 0.171–0.705) cohorts were statistically significant. Inclusion in the CA 1–2 cohort was associated with a positive likelihood estimate (0.28, P=0.018), but compared with the CA 3 cohort, the odds ratio was not significant (0.660, CI 0.330–1.321). A significant positive likelihood estimate was also noted when the day of the study was used as a predictor (0.00068, P=0.031), indicating a small improvement in correct prediction during the study period.

Fig 3.

Thirty day moving average of prediction accuracy over duration of study. CA 1–2 are residents who enrolled as first-year residents when the study began and were second-year residents at the end of the study; CA 2–3 are residents who enrolled as second-year residents when the study began and were third-year residents at the end of the study; CA 1–1 are residents who enrolled as first-year residents the second year of the study; CA 3 are residents who enrolled as third-year residents the first year of the study.

Discussion

This study is one of few studies that focus on comprehensive airway assessment implemented on a large scale for training purposes. This study demonstrates that a comprehensive airway assessment form using all 11 of the ASA's proposed airway risk predictors did improve documentation of airway assessment, but not the accuracy of such predictions.

The rates of DDL, intubation, and supraglottic device placement events reported in our study are similar to those that have already been reported in the literature.^2–16,31,32 Our overall rate of DMV (5.8%) is similar to the rates reported by Langeron and colleagues⁷ (5%) and Yildiz and colleagues³³ (7.8%). However, it is significantly higher than the rate reported by Kheterpal and colleagues⁵ (1.5%), Asai and colleagues³⁴ (1.4%), and Rose and Cohen¹⁶ (0.9%), but differences in definitions, institutional practices, provider mix, and self-reporting may account for this difference.

For example, we did not include the contribution of neuromuscular blocking agents in our assessment of DMV and we did not control for oral airway placement, as it is routine practice to place an oral airway before any mask ventilation attempt and also to wait for proper ventilation before the administration of neuromuscular blocking agent. We left the opportunity to comment (open field) on the use of the neuromuscular blocking agent as rescue of a DMV, but no comments on such a specific topic were found in the database. We only included data from residents rather than a mix of attending anaesthetists, CRNAs, and AAs, as is presented by Kheterpal and colleagues.²¹

Additionally, although other studies have reported a decrease in the number of surgical airways in conjunction with a comprehensive airway education programme,²⁶ no difficult surgical airway events occurred during the study period. Because such events are quite rare and because we excluded emergent cases, we probably lacked the appropriate sample size to address this issue. Despite these differences, the frequency of DA events in this study is comparable with other reported literature.

Completeness of airway examination documentation

The experimental form did significantly improve completeness of the airway assessment when compared with the standard anaesthetic record, although complete documentation was not necessarily equivalent. With respect to comparing the completeness of documentation, we only looked at two fields (Mallampati and neck mobility), as only these were present on both forms.

Overall recognition of the DA

The accuracy of predicting difficulty was greater for the control group than the experimental group (71% vs 69%, P=0.032). This difference, small and probably not of significant clinical impact, may have been related to the additional work required to complete the form—effort which may have distracted from the actual assessment; for example, the requirement to perform several measurements, some of which may have been normal, could actually have directed attention away from an obviously abnormal appearance. Alternatively, the form may have provided false reassurance and led to an incorrect prediction (i.e. false negative), underscoring the lack of sensitivity of the predictors (more is not necessarily better).

Impact on resident education

The logistic regression analysis demonstrated significant differences in prediction accuracy between cohorts of residents in the experimental group (Fig. 3); however, considering the low rate of overall correct prediction, it is questionable in its clinical value.

In particular, the low rate of correct prediction observed for the CA 2–3 residents compared with the CA 1–2, particularly at the start of the study, was unexpected. This may reflect difficulty by the CA 2–3 in adjusting an established work flow to complete the form, but did not affect the CA 1–2 because they had not established a routine yet. It may also reflect an intermediate level of expertise in the CA 2–3 where residents are more likely to make errors. The decrease in accuracy seen in the CA 1–2 cohort starting in January 2009 and reaching a zenith in July 2009 is also consistent with this effect. The consistently high rate of correct prediction seen in the CA 3 might also reflect a different aspect of this effect, where passage from an intermediate to a more senior phase of training leads to lower rates of error. Other possible explanations for observed patterns may reflect changes in educational emphasis during the study and the impact of external events on resident performance. For example, the difference in initial frequency of correct prediction between the CA 1–2 class at the start of the study and the entrance of the CA 1–1 class in August 2009 may reflect the impact of a larger educational effort at the start of the study than at its mid-point. The decrease in the rate of correct prediction during the initial months of the study may be related to the increased demands placed on residents after hurricane Ike. Similarly, the decrease in the rate of correct prediction seen in July 2009 may reflect the annual turnover of residents. Simply being aware that this type of variability exists may be useful for planning educational efforts and managing operating theatre resources. However, we recognize our assertions remain speculative and require further confirmations.

Although it is encouraging that the considerable effort required by the residents to complete this study did result in improved prediction, we had expected to see a larger effect than was actually observed. It is possible that the educational effort applied at the beginning of the study resulted in an early improvement in prediction that lessened somewhat over time, resulting in a smaller improvement by the end of the study period.

Our form was intended to function as a cognitive aid analogous to those described by Hutchins³⁵ that supports pilots while landing an airplane. However, it is possible that the complexity of the information on our form did not adequately support clinician cognitive processes and could have actually contributed to poor performance.³⁶ We hypothesize that by fragmenting the provider's focus, the form may have distracted residents from the task of interpreting the data and performing an integrated assessment.

There are few limitations to this study. Although this study was large, it reflects only a single institutional experience. A multicentre trial is necessary to help limit this effect. Additionally, the interobserver variability between assessments presents a limitation, as does the variability of patients encountered between groups. The time from study design to study implementation was significant and the scope was limited by changes in technology and research findings that had occurred in the interim. It is possible that changes in work flow not directly related to airway assessment impacted resident performance. Although we anticipated this impact to be small, it is important to consider this factor to have potential impact on resident performance.

Conclusions

Based on our study, the use of an airway assessment that includes all 11 ASA's airway risk factors did not result in clinical improvement of resident prediction. Although completion of a specially designed airway assessment form led to significantly better documentation, it did not appear to have a clinically significant impact. Future studies pertaining to DA evaluation may benefit from a focus on the interpretation process rather than strictly data-driven prediction.

Declaration of interest

C.A.H. was the recipient of the 2007 Foundation for Anesthesia Education and Research Grant. She is a consultant for Storz, Ambu A/S, Aircraft Medical, and serves on the speaker bureau for Covidien, LMA North America, and Ambu A/S. D.C. is a paid consultant for Smiths Medical and serves on the speaker bureau for Cadence. D.C. and C.A.H. equally contributed to the development of the Airway Assessment Form.

Funding

C.A.H. and D.I. initiated this study through funding by the Foundation for Anesthesia Education and Research (FAER). P.V.K. is supported by a training fellowship from the Keck Center NLM Training Program in Biomedical Informatics of the Gulf Coast Consortia (NLM grant no. T15LM007093).

Appendix 1: Experimental group airway assessment form

This comprehensive airway assessment form was used by the experimental and research groups to assess patient airways before operation. The form contains descriptions of all 11 of the ASA's recommended DA predictors and requires a prediction as to the expected difficulty of the airway and the anaesthetic plan that will be used.

Appendix 2: Control group airway assessment form

This form was used by the control group to assess the expected difficulty of the patient airway before operation. The physical examination includes Mallampati score and neck mobility assessments.

Appendix 3: Postoperative outcome data form

This form was completed for all patients enrolled in the study and documented difficulties (if any) experienced with mask ventilation, supraglottic airway device, direct laryngoscopy, intubation, and surgical airway. Advanced airway techniques were documented if alternative devices were used. The number of attempts for all procedures was documented.