Pilot Error in Air Carrier Mishaps: Longitudinal Trends Among 558 Reports, 1983–2002

Abstract

Background

Many interventions have been implemented in recent decades to reduce pilot error in flight operations. This study aims to identify longitudinal trends in the prevalence and patterns of pilot error and other factors in U.S. air carrier mishaps.

Method

National Transportation Safety Board investigation reports were examined for 558 air carrier mishaps during 1983–2002. Pilot errors and circumstances of mishaps were described and categorized. Rates were calculated per 10 million flights.

Results

The overall mishap rate remained fairly stable, but the proportion of mishaps involving pilot error decreased from 42% in 1983–87 to 25% in 1998–2002, a 40% reduction. The rate of mishaps related to poor decisions declined from 6.2 to 1.8 per 10 million flights, a 71% reduction; much of this decrease was due to a 76% reduction in poor decisions related to weather. Mishandling wind or runway conditions declined by 78%. The rate of mishaps involving poor crew interaction declined by 68%. Mishaps during takeoff declined by 70%, from 5.3 to 1.6 per 10 million flights. The latter reduction was offset by an increase in mishaps while the aircraft was standing, from 2.5 to 6.0 per 10 million flights, and during pushback, which increased from 0 to 3.1 per 10 million flights.

Conclusions

Reductions in pilot errors involving decision making and crew coordination are important trends that may reflect improvements in training and technological advances that facilitate good decisions. Mishaps while aircraft are standing and during push-back have increased and deserve special attention.

Commercial aviation has experienced major changes since the early 1980s. Increasing numbers of flights have placed greater demands on pilots, air traffic controllers, and airports. Changes include advanced computers and other technology that have altered pilots’ tasks. Pilot training has relied increasingly on flight simulators. Crew resource management has been emphasized to improve communication and interaction among crewmembers (3). Beginning in 1997, the majority of commuter flights that had been governed by 14 CFR Part 135 regulations were required to fly under the stricter Part 121 regulations that previously applied only to aircraft designed for 30 or more passengers (7).

Many of these changes have been undertaken to reduce the problem of “pilot error,” long considered the most prominent contributor to aviation crashes. In an attempt to determine whether the desired beneficial effect has been achieved, we examined the question of whether pilot error has diminished in importance or changed in its characteristics during recent decades. Our primary objective was to identify temporal trends in the prevalence and patterns of pilot error and other factors in air carrier mishaps.

METHOD

Data

This study, part of a larger project that focused on age-associated changes in commercial pilots (6), was a case-series analysis of crashes and other mishaps of domestic air carrier flights (14 CFR Part 121), both scheduled and nonscheduled, that occurred during 1983 – 2002. The mishaps were identified from National Transportation Safety Board (NTSB) records. The NTSB is responsible for investigating transportation mishaps that occur in the United States.

The NTSB defines an aviation accident as an event associated with aircraft operation resulting in serious injury, death within 30 d, or damage to the aircraft requiring major repair or component replacement. Although the term ‘crash’ is preferred by most injury prevention professionals, the present study uses the term “mishaps” (i.e., ‘accidents’ as defined by the NTSB) because some injury-producing events do not involve crashes, but are either encounters with turbulence or events while the aircraft is on the ground and not moving. Turbulence in flight was classified by the NTSB as an “accident” if it resulted in serious injury.

Data gathered in aviation investigations are recorded on the Factual Report Aviation form (NTSB Form 6120.4), which includes coded data describing the circumstances and probable causes of each mishap as well as a narrative summary. Records for all air carrier mishaps between 1983 and 2002 were reviewed by a project consultant (an experienced commercial pilot and flight instructor), who coded the circumstances, contributing factors, and pilot errors that were identified in the NTSB report.

Denominator data on the number of flights (departures) were obtained from the NTSB website ( 6 ) and NTSB personnel. The number of flights was preferred to flight hours as a denominator because flight hours are greatly influenced by the large proportion of hours spent at high altitudes, where mishaps are not common. In contrast, the number of flights reflects the more hazardous periods near or on the ground.

Analysis

The outcome measure of primary interest was the rate of air carrier mishaps per 10 million flights. To counteract the problem of numbers that would be too small for meaningful analysis, we divided the two decades into four 5-yr periods and calculated mishap rates for each period. Analyses first addressed the question of whether the rate of air carrier mishaps, as a whole, had changed during 1983 –2002. The proportions and rates of mishaps involving pilot error, mechanical failure, turbulence, and bad weather were then determined. Pilot errors were analyzed in relation to the phase of flight in which the mishap occurred.

Secondly, we examined longitudinal trends in the overall rate of mishaps involving pilot error, as well as trends in rates for the major categories of pilot error: carelessness, poor decisions, mishandling aircraft kinetics, mishandling wind or runway conditions, and poor crew interaction. As a measure of the severity of crashes and other mishaps, the case fatality rate was calculated as the percent of crashes that resulted in death of the pilot-in-command. Including all mishaps that were fatal to anyone in the numerator would have distorted the proportion because of the great variation in the number of passengers. In order to examine longitudinal trends in the outcomes of interest, observed crashes were compared to expected crashes using Chi-square tests. The data analyses were performed using Statistical Analysis Software (SAS), version 9.1 (SAS Institute Inc., Cary, NC) and Intercooled STATA 6 (StataCorp, College Station, TX).

RESULTS

The NTSB reported 604 ‘accidents’ (i.e., exclusive of terrorist or criminal actions) involving domestic air carriers in the United States during 1983 –2002. We included 558 (92%) in our analysis. Of the remaining 46 mishaps, 39 were excluded due to missing information and 7 for miscellaneous reasons. The rate of mishaps over the 20-yr period averaged 33 per 10 million flights, with no consistent longitudinal trend (Table I).

TABLE I.

MISHAP RATES PER 10 MILLION FLIGHTS BY PILOT ERROR AND TIME PERIOD, U.S. AIR CARRIERS, 1983–2002.

	N					Rate
Pilot Error Involved	83–87	88–92	93–97	98–02	Total	83–87	88–92	93–97	98–02	Total
Yes	46	40	47	47	180	14.2	10.2	10.9	8.5	10.6
No	64	70	105	139	378	19.8	17.9	24.2	25.2	22.2
Total	110	110	152	186	558	34	28.1	35.1	33.7	32.8

Bad weather, excluding turbulence, played a role in 265 (47%) mishaps, clear air turbulence in 138 (25%), and mechanical failure in 117 (21%). The proportion of mishaps in which weather was a factor declined slightly, by 17%. No statistically significant longitudinal trends were evident in any of these circumstances.

Pilot error was noted in 180 mishaps (32%). The percentage of mishaps that involved pilot error declined from 42% in 1983 – 87 to 36% in 1988 – 92, 31% in 1993 – 97, and 25% in 1998 – 2002 (P = 0.02), an overall decrease of 40%. In 56 mishaps, at least two types of pilot error were noted. The most common types of pilot error were carelessness (26% of errors), poor decisions (23%), mishandling aircraft kinetics (21%), and poor crew interaction (11%) (Table II).

TABLE II.

NUMBER AND PERCENT OF PILOT ERRORS, BY TYPE OF ERROR, U.S. AIR CARRIERS, 1983–2002.

	Total Errors*
Category of Pilot Error	N	Percent
Careless	62	26.3
Flawed decision	55	23.3
Mishandled aircraft kinetics	50	21.2
Poor crew interaction	27	11.4
Mishandled wind or runway conditions	17	7.2
Other	25	10.6
Total	236	100.0

^*Includes up to two pilot errors per mishap, if in different categories.

Carelessness is illustrated by a case in which the pilot-in-command omitted the check for hydraulics when reading the checklist prior to landing. As a result, there was no hydraulic pressure for the landing gear or flaps and the DC9 made a gear-up landing. Poor crew interaction was evident in a hard landing following an unstabilized approach, during which the copilot twice urged the captain not to land without first executing a go-around; the captain ignored him. An example of mishandling aircraft kinetics was poor response to a bounced landing. Flawed decisions were exemplified by a case in which the aircraft had been put on hold because weather at the destination did not meet minimum criteria for landing. When the weather cleared, air traffic control asked whether they could make the approach from their altitude. The pilot-in-command made a poor decision, saying “ yes ” although the aircraft was too high. As a result, the airplane came in too high and too fast, landed 7000 feet down the runway, and ran off the end of the runway.

During the period studied, the rate of mishaps related to pilot error declined by 40%, from 14.2 to 8.5 per 10 million flights (P = 0.02) (Table I). No trend was evident in the rate of mishaps related to carelessness or to mishandling aircraft kinetics. Pilot error categorized as flawed decision declined from 6.2 to 1.8 per 10 million flights, a 71% reduction (P = 0.006) (Fig. 1). The most common type of flawed decisions, those related to weather, declined by 76%, from 4.6 to 1.1 per 10 million flights (P = 0.0001). Poor crew interaction declined from 2.8 to 0.9 per 10 million flights, a 68% reduction (P = 0.003). Mishandled wind conditions or runway conditions declined by 78%, from 2.5 to 0.54 per 10 million flights (P = 0.03).

Fig. 1.

Mishap rate per 10 million flights by type of pilot error and time period, U.S. air carriers, 1983–2002.

The proportion of mishaps involving pilot error varied with phase of flight (Table III). Pilot error was most common (44 – 64% of mishaps) during taxiing, takeoff, final approach, and landing. Pilot error was least common while the aircraft was being pushed back from the gate (11%) or standing, i.e., not in motion (9%). Examples of events while aircraft were not in motion included aircraft struck by a ground vehicle, ramp personnel struck by a jet blast or propeller, and injuries during a precautionary evacuation.

TABLE III.

MISHAPS INVOLVING ANY PILOT ERROR, BY PHASE OF FLIGHT, U.S. AIR CARRIERS, 1983–2002.

		Mishaps with Pilot Error
Phase of Flight	Total Mishaps	N	Percent
Landing	83	53	64
Taxiing	54	28	52
Final approach	17	8	47
Takeoff	52	23	44
Climbout	53	15	28
Cruise	100	19	19
Descent	74	12	16
Pushback	37	4	11
Not in motion	67	6	9
Other	21	12	57
Total	558	180	32

Three phases of flight showed significant trends (Fig. 2). The rate of mishaps during takeoff declined by 70%, from 5.3 to 1.6 per 10 million flights (P = 0.01). Mishaps while aircraft were not in motion more than doubled, from a rate of 2.5 to 6.0 per 10 million flights (P = 0.03). Mishaps during pushback increased from none in 1983–87 to 3.1 per 10 million in 1998–2002 (P = 0.02).

Fig. 2.

Mishap rate per 10 million flights by phase of flight and time period, U.S. air carriers, 1983–2002.

The case fatality rate for pilots (i.e., the percent of pilots-in-command who were killed) declined steadily during the period, from 11.8% in 1983–87 to 1.6% in 1998 – 2002, an 86% reduction (P = 0.002). No changes in pilot error rates were apparent in association with the major regulatory change in 1997 that included most Part 135 flights under Part 121 regulations.

DISCUSSION

This report documents an impressive 40% reduction in the rate of air carrier mishaps involving pilot error between 1983 and 2002 as well as a 40% reduction in the proportion of mishaps that involved pilot error. By defining specific types of pilot error and by measuring changes in incidence per 10 million flights, the study offers insight into where progress has (and has not) been made in recent decades. Much of the decrease in pilot-error mishaps was associated with improved decision making by pilots. In particular, weather-related mishaps involving pilot error declined by 76%. This reduction cannot be explained by better weather patterns during flights, since the proportion of mishaps in which weather was a factor declined by only 17% over the 20 yr studied.

The decreasing trend in pilot-error mishaps probably reflects, in part, advances in technology such as cockpit displays that permit pilots to avoid threatening weather and to determine their exact location in relation to airports, runways, etc. Emphasis in recent years on crew resource management has likely been another important factor in the 68% reduction in mishaps involving poor crew coordination.

High-profile airline disasters generally lead to recommendations by the NTSB for changes in training, flight procedures, and aircraft design that might avert future crashes and deaths, for example by improving flight crew training with regard to weather and crew resource management (9). In illustration, a crash in 1987 of a DC9 at the Detroit Metropolitan Airport resulted in an NTSB recommendation that airline operators “emphasize rigorous adherence to prescribed checklist procedures” (10). In addition to making changes prompted by crashes, NTSB recommendations, and FAA regulations, airlines have created databases that analyze flights, identifying threats to safety such as adverse weather (the most common threat), radio congestion, and operational pressures, and indicating whether the flight crews identified and managed the threats (13). The results help airlines to identify and address problems within their operations.

Li and colleagues, in a study based on NTSB-coded data, found that during 1983 –1996 pilot error was cited by NTSB investigators as a probable cause in 34% of air carrier crashes ( 4 ). In the present study, we found that in 1998–2002, pilot error was associated with only 25% of mishaps. This encouraging trend is not likely to be an artifact since coding was consistent throughout the study: a single researcher who was a commercial pilot read and evaluated NTSB briefs of each of the 558 crashes.

Our finding that 22% of errors in 1983 –2002 were related to poor decisions by pilots is consistent with the results of analyses of 44 air carrier mishaps in 1990 – 1996 using the Human Factors Analysis and Classification System (HFACS) ( 14 ). In their review, Wiegmann and Shappell identified decision errors in 25% of mishaps that were at least partly attributable to the aircrew. Their investigation excluded some cases included in our analysis such as turbulence-related mishaps and any mishaps with no pilot error. Several of their major categories were similar to ours: decision errors, crew-resource mismanagement, and skill-based errors (which would have included our categories of mishandling aircraft kinetics and mishandling wind and runway conditions). Although the authors reported finding no improvements during the 7 yr studied, the relatively short time period would have made it hard to identify a longitudinal trend.

The severity of mishaps apparently decreased during our study period, judging from the 86% reduction in pilot fatality rates. The reported number of passenger deaths and serious injuries per million passenger emplanements also declined (8). The increase in survival rates in airline crashes may reflect, in part, improved crashworthiness associated with changes in aircraft design, including changes intended to reduce the risk of fire, which is the major determinant of crash survivability ( 5 ). Probably the most important influence on crash severity and the likelihood of fatality is the phase of flight in which a mishap occurs; the increase in mishaps while aircraft were standing or during pushback would have reduced the proportion of mishaps likely to cause the death of a pilot.

The overall air carrier mishap rate remained stable during the 20-yr study period, with the decrease in pilot-error mishaps offset by an increase in mishaps in which pilot error was not evident. This does not mean that no one made errors. Rather, the increase in mishaps on the ground suggests that an increasing number of mishaps may have involved ground personnel and air traffic control. In recent years, the number of flight operations has increased with relatively little increase in airport capacity (11), resulting in greater density of air traffic at airports and a situation that is increasingly challenging to all airport workers ( 2). Research based upon airline data indicated that air traffic control problems such as radio congestion were second only to adverse weather conditions in their contribution to threats to the safety of flights of air carriers (13).

A special strength of this study is the provision of rates, using available information on the numbers of flights (departures) of air carrier flights (7). Too often, aviation research emphasizes differences in proportions rather than rates, because relevant denominator data are usually unavailable. Unlike rate calculations, proportional analyses have the disadvantage that if one component decreases, there is a corresponding increase elsewhere.

A limitation of the study is that the mishaps analyzed were 8% fewer than the total number of domestic air carrier mishaps recorded by the NTSB during the 20-yr study period. Therefore, it is probable that the rates presented are approximately 8% lower than actual rates. This is unlikely to bias the results, since there was no indication of any relationship between pilot-error mishaps and the missing data that caused exclusion of 8% of cases.

The results are specific to the years 1983–2002 and, therefore, cannot be generalized to more recent years. However, recent rates reported by the NTSB (7) indicate that airline crashes during 2003 –06 decreased compared to earlier years. It is possible that the decrease in crash rates since 2002 is associated with the decline in pilot-error mishaps identified in this study. Our analysis is limited to errors by pilots and does not include the role of organizational factors and supervision. These important determinants of aviation safety are among the human factors analyzed and discussed in a recent report by Shappell and colleagues (12).

It is noteworthy that during the two decades when the rate of mishaps involving pilot error declined by 40%, there was no decline in the overall rate of air carrier mishaps per 10 million flights ( Table I ) despite the fact that part way through the study period, in 1997, scheduled commuter flights with 10 or more passenger seats came under air carrier regulations (i.e., 14 CFR Part 121). The regulatory change might have been expected to increase the Part 121 crash rate because previously the rate for scheduled commuters had been substantially higher than the rate for Part 121 flights (1). In the decade prior to the 1997 regulatory change, for example, the accident rate per 10 million flights was 57 for scheduled commuters and 35 for airlines ( 7). It is possible that aircraft carrying fewer than 10 passengers, which were not included in the change to coverage by 14 CFR 121, were largely responsible for the high rates associated with Part 135 operations.

Conclusion

Improved decision making by pilots and better crew coordination are important trends that may reflect improvements in training as well as technological advances that facilitate good decisions. The decrease in pilot-error mishaps was offset by an increase in mishaps in which pilot error was not implicated. The increase in mishaps while aircraft are not in motion or being pushed back deserves attention.