Validity of Suspected Alcohol and Drug Violations in Aviation Employees

Abstract

Introduction

In the United States, transportation employees who are suspected of using alcohol and drugs are subject to reasonable-cause testing. This study aims to assess the validity of suspected alcohol and drug violations in aviation employees.

Methods

Using reasonable-cause testing and random testing data from the Federal Aviation Administration for the years 1995 through 2005, we calculated the positive predictive value (PPV) and positive likelihood ratio (LR+) of suspected alcohol and drug violations. The true status of violations was based on testing results, with an alcohol violation being defined as a blood alcohol concentration of ≥40 mg/dL and a drug violation as a test positive for marijuana, cocaine, amphetamines, phencyclidine, or opiates.

Results

During the 11-year study period, a total of 2,284 alcohol tests and 2,015 drug tests were performed under the reasonable-cause testing program. The PPV was 37.7% [95% confidence interval (CI), 35.7–39.7%] for suspected alcohol violations and 12.6% (95% CI, 11.2–14.1%) for suspected drug violations. Random testing revealed an overall prevalence of 0.09% (601/649,796) for alcohol violations and 0.6% (7,211/1,130,922) for drug violations. The LR+ was 653.6 (95% CI, 581.7–734.3) for suspected alcohol violations and 22.5 (95% CI, 19.6–25.7) for suspected drug violations.

Discussion

The discriminative power of reasonable-cause testing suggests that, despite its limited positive predictive value, physical and behavioral observation represents an efficient screening method for detecting alcohol and drug violations. The limited positive predictive value of reasonable-cause testing in aviation employees is due in part to the very low prevalence of alcohol and drug violations.

INTRODUCTION

The United States federal government started the Drug Free Workplace Program in 1986, which included provisions for testing employees with safety-sensitive functions for illicit drugs (i.e., marijuana, cocaine, amphetamines, phencyclidine, and opiates). To facilitate the implementation of the Drug Free Workplace Program, the Department of Health and Human Services established scientific and technical standards for laboratory certification and performance of the drug tests. Spurred by a series of high-profile transportation incidents in which alcohol was implicated as a probable cause, including the 1989 Exxon Valdez oil spill in Alaska, the 1990 conviction of three Northwest Airline pilots, and the 1991 New York subway crash, the US Congress enacted the Omnibus Transportation Employee Testing Act (OTETA) of 1991, making alcohol testing mandatory for transportation employees with safety-sensitive functions [1]. As mandated by the OTETA, the Department of Transportation (DOT) established detailed guidelines for implementing workplace alcohol and drug testing programs. Codified in title 49 of the Code of Federal Regulations (CFR) Part 40, these guidelines prescribe the procedural protocols as to how alcohol and drug tests are to be conducted by covered employers [2]. Following these guidelines, DOT’s operational agencies developed alcohol and drug testing programs applicable to their specific industries.

The rules and procedures for alcohol and drug testing in aviation are specified in the Federal Aviation Administration (FAA)’s Anti-Drug and Alcohol Misuse Prevention Programs for Personnel Engaged in Specified Aviation Activities [1]. Major airlines (Part 121 certificate holders), commuter air carriers and air taxis (Part 135 certificate holders), and air traffic control facilities that are not operated by the FAA are required to implement the alcohol and drug misuse prevention programs in employees with safety-sensitive functions, including flight crew members, flight attendants, flight instructors, aircraft dispatchers, maintenance personnel, aviation screeners, ground security coordinators, and air traffic controllers. These employees are subject to the following alcohol and drug testing programs: pre-employment testing (conducted after a contingent job offer); random testing (conducted on randomly selected employees); reasonable-cause testing (conducted on employees suspected of being under the influence of alcohol or drugs); post-accident testing (conducted on all employees who may have contributed to the accident); return-to-duty testing (conducted on employees who have successfully completed substance abuse treatment before returning to safety-sensitive duty); and follow-up testing (conducted on employees who have returned to safety-sensitive duty following substance abuse treatment).

Random testing, presumably a major deterrent of alcohol and drug use in the workplace, is the predominant component of the mandatory alcohol and drug testing program, accounting for 97% of all alcohol tests and 96% of all drug tests performed on aviation employees [3]. Unlike random testing, reasonable-cause testing is primarily aimed at the detection of alcohol and drug violations. Thus, it targets employees suspected of using alcohol or illicit drugs. A supervisor trained in the detection of alcohol and drug use determines the need for reasonable-cause testing. Suspicion must be based on physical appearance, behavior, breath, body odor, speech, or performance indicating probable alcohol or drug use.

Mandatory alcohol and drug testing, particularly reasonable-cause testing, has been controversial because of ethical, legal and economic concerns [6]. Although mandatory alcohol and drug testing in truck drivers has been linked to a significant reduction in crashes [7–9], the safety benefit to other modes of transportation appears to be negligible [3,10]. Moreover, there is inadequate research on the effectiveness of specific testing programs [6,11–22]. The purpose of this study is to assess the validity of suspicions of alcohol and drug violations on which reasonable-cause testing is based.

METHODS

Data for this study came from the FAA’s alcohol and drug testing information management system. Federal law (49 CFR 40) requires that employers report alcohol and drug test results to the U.S. Department of Transportation using standard protocols and procedures [2]. The Drug Abatement Division of the FAA is responsible for collecting, maintaining, and analyzing alcohol and drug testing data submitted by aviation employers. Each year, employers must submit annual reports to the FAA by February 15, and the submitted data must be signed by a representative of the employer to certify the accuracy of the data reported. False statements or reports may constitute a criminal offense and are punishable by law.

Federally mandated drug tests must be performed by laboratories certified by the US Department of Health and Human Services, while alcohol tests must be conducted by certified technicians using a DOT-approved device [2,4,5,23]. Random testing is performed on randomly selected employees immediately before, during, or immediately after their work shift. An alcohol violation is defined as an alcohol concentration level of ≥ 0.04 g/dL (equivalent to 0.04 grams of alcohol per 210 liters of breath), shy lung or refusal to submit to testing. Shy lung is recorded when an employee fails to provide a breath specimen and a physician has determined, through a required medical evaluation, that there was no adequate medical explanation for the failure. For alcohol tests, a result <0.02 g/dL is considered negative. A confirmation test is required when the result is ≥0.02 g/dL. A drug violation is defined as a positive test for marijuana (≥ 50 ng/ml), cocaine (≥ 300 ng/ml), amphetamines (≥1000 ng/ml), phencyclidine (≥ 25 ng/ml), or opiates (≥ 2000 ng/ml), adulterated tests, substituted tests, shy bladder or refusal to submit to testing. Shy bladder is recorded when an employee fails to provide a sufficient amount of urine when directed, where medical evaluation determines that there is no adequate medical explanation for the failure. For drug tests, a result below the cutoff is considered negative. If the result is at or above the cutoff, then a urine confirmation test is conducted. When a confirmation test is below the confirmation cutoff, the result is considered negative; when a test result is at or above the cutoff it is considered positive. Specimens are evaluated to detect if the sample is consistent with human urine and then are examined for signs of adulterants, foreign substances, dilution, and substitution. When a test is confirmed as positive or when drug testing was not conducted due to detected adulterants or substitution, a test of the split specimen may be requested. The split specimen is then tested by a different laboratory [2].

Annual alcohol and drug testing data for the years 1995 through 2005 were analyzed. The validity of suspected alcohol and drug violations was measured by positive predictive value (PPV) and positive likelihood ratio (LR+). PPV, referring to the probability of an actual alcohol/drug violation in an employee suspected of using alcohol/drugs, is calculated by dividing the number of violations detected through the reasonable-cause testing program by the number of tests performed under the reasonable-cause testing program. LR+, defined as the probability of a reasonable-cause test being performed on an employee who was positive for alcohol/drugs divided by the probability of a reasonable-cause test being performed on an employee who was not, was computed according to the following formula [24]:

$$\text{LR}+=[{\mathrm{p}}_2(1-{\mathrm{p}}_1)]÷[{\mathrm{p}}_1(1-{\mathrm{p}}_2)]$$

where p₁ denotes the proportion of positives in random tests and p₂, the proportion of positives in reasonable-cause tests (numerically equivalent to PPV). Together, PPV and LR+ measure the validity of trained supervisors’ suspicion of alcohol and drug violations on which reasonable-cause testing is based.

RESULTS

During the 11-year study period, a total of 2,284 alcohol tests were administered under the reasonable-cause testing program; of them, 861 were positive, yielding a PPV of 37.7% [95% confidence interval (CI), 35.7–39.7%] for suspected alcohol violations. During the same period, a total of 2,015 drug tests were performed under the reasonable-cause testing program; of them, 254 tested positive, resulting in a PPV of 12.6% (95% CI, 11.2–14.1%) for suspected drug violations. Positive predictive values of suspected alcohol violations remained fairly stable during the study period, ranging from 29.9% in 1995 to 45.9% in 1997 (Table 1). Similarly, PPVs of suspected drug violations fluctuated over the years but did not show any apparent time trend (Table 1).

Table 1.

Positive Predictive Values (PPVs) and 95% Confidence Intervals (CIs) of Suspected Alcohol and Drug Violations in Aviation Employees by Year, 1995–2005

	Suspected Alcohol Violations	Suspected Drug Violations
Year	PPV (95% CI)	PPV (95% CI)
1995	0.299 (0.223–0.376)	0.130 (0.064–0.196)
1996	0.364 (0.295–0.434)	0.204 (0.138–0.271)
1997	0.459 (0.384–0.534)	0.144 (0.091–0.197)
1998	0.432 (0.357–0.507)	0.169 (0.103–0.235)
1999	0.301(0.235–0.367)	0.128 (0.084–0.173)
2000	0.373 (0.312–0.434)	0.134 (0.089–0.180)
2001	0.350 (0.292–0.409)	0.094 (0.060–0.127)
2002	0.344 (0.288–0.401)	0.111 (0.067–0.154)
2003	0.395 (0.330–0.461)	0.142 (0.092–0.192)
2004	0.374 (0.309–0.439)	0.103 (0.058–0.148)
2005	0.450 (0.386–0.513)	0.082 (0.043–0.120)
Total	0.377 (0.357–0.397)	0.126 (0.112–0.141)

During 1995–2005, the Federal Aviation Administration recorded a total of 649,796 alcohol tests and 1,129,922 drug tests under the random testing program, which detected 601 (0.09%) alcohol violations and 7,211 (0.6%) drug violations, respectively. Reasonable-cause testing was much more likely to be performed on employees who were positive for alcohol (LR+ 653.6, 95% CI, 581.7–734.3) or drugs (LR+ 22.5, 95% CI 19.6–25.7) than on employees who were not. Of all the alcohol violations detected, approximately 90% were based on testing results, 9% due to refusals to submit to testing, and 1% due to shy lung. Of all the drug violations detected, 91% were based on testing results, 6% due to refusals to submit to testing, and 3% due to shy bladder or adulterated/substituted urine samples.

The prevalence of alcohol violations detected by random testing increased from 0.07% during 1995–1997 to 0.11% during 1998–2005 (p<0.0001), when the annual alcohol testing rate decreased from 25% to 10% (Fig. 1). Lowering the annual testing rate for drugs from 50% to 25% in 1998 was not associated with a significant change in the prevalence of drug violations (0.65% during 1995–1997 and 0.63% during 1998–2005, p= 0.4355; Fig. 1).

Figure 1.

Prevalence of Alcohol and Drug Violations Based on Random Testing Data for Aviation Employees, United States, 1995–2005

DISCUSSION

The results of this study indicate that the positive predictive value of suspected alcohol and drug violations in aviation employees is rather low; the majority of the employees suspected of using alcohol or drugs tested negative for these substances. The limited accuracy of suspected alcohol and drug violations is due in part to the very low prevalence of alcohol and drug violations in aviation employees. It also reflects the difficulty of identifying alcohol and drug violations through observation of employees’ physical and behavioral characteristics by trained personnel. The exceptionally low PPV of suspected drug violations suggests that it is harder to detect a person under the influence of illicit drugs than a person under the influence of alcohol based on physical appearance, body odor, behavior, and job performance.

On the other hand, the very high positive likelihood ratios indicate that, on a relative scale, reasonable-cause testing is a method with formidable discriminative power for distinguishing employees who are under the influence of alcohol and drugs from those who are not. The discriminative power of reasonable-cause testing suggests that, despite its limited positive predictive value, physical and behavioral observation represents an efficient screening method for detecting alcohol and drug violations. Based on data presented in this study, under the reasonable-cause testing program it takes an average of 3 tests for detecting one alcohol violation and 8 tests for detecting one drug violation.

Both random testing and reasonable-cause testing aim to reduce alcohol and drug violations and may have a deterrent effect. The deterrent effect of random testing is presumably a function of the testing rate. Although little is known about the relationship between the testing rate and the effect size of deterrence, it is reasonable to assume that the deterrent effect increases as the testing rate increases, and vice versa (3). Results of the present study provide empirical evidence that the deterrent effect of random alcohol testing may diminish as the testing rate decreases. Part of the observed increase in the prevalence of alcohol violations during 1998–2005 might be due to the decrease in annual testing rates from 25% to 10%. The relationship between testing rate and deterrent effect appears to be substance-specific and may be susceptible to extraneous confounders, as evidenced by the lack of any statistically significant change in the prevalence of drug violations after the annual testing rate for drugs was lowered from 50% to 25% in 1998.

Kraus and Li [25] documented the importance of reasonable-cause testing in detecting alcohol violations among flight crew members. During 1990 and 2006, newspapers reported on a total of 13 incidents of alcohol-impaired flying involving 17 US airline pilots; of the 13 reported incidents, nine were identified by airport personnel and two by passengers based on suspicion of alcohol use by the pilot [25]. Widespread publicity generated by these incidents may have served as an important mechanism of the deterrent effect of the reasonable-cause testing program.

Although it is not as efficient as reasonable-cause testing in detecting alcohol and drug violations, random testing as an indiscriminate program may be more acceptable to employees than reasonable-cause testing. Moreover, random testing provides essential data for determining the prevalence and monitoring the time trends of alcohol and drug violations, and for evaluating the effectiveness of intervention programs. Therefore, it is important to keep random testing in the mandatory alcohol and drug testing program. The challenge facing researchers and policy-makers is to optimize the different components of the testing program so that the testing program is most cost-effective and cost-beneficial. To meet this challenge, research is needed to quantitatively define the relationship between the annual testing rate for random testing and the deterrent effect on alcohol and drug use behaviors and to understand the extent to which they meet their respective objectives.

This study has several limitations. First, our analysis relied on aggregated alcohol and drug testing data. Employee-level data were not available. Therefore, it was not possible to examine alcohol and drug violations by demographic characteristics. Second, the validity of suspected alcohol and drug violations was measured only by positive predictive value and positive likelihood ratio. We did not have data necessary for estimating the sensitivity and specificity of reasonable-cause testing. The negative predictive values of reasonable-cause testing are likely to be higher than 99%, given the very low prevalence (<1%) of alcohol and drug violations in aviation employees. Third, we did not examine the intra- and inter-observer reliability of suspicions of alcohol and drug violations. The reproducibility of physical and behavioral observations is likely to vary with many factors, such as timing, individual response to alcohol and drugs, and observer’s experience. To enhance the performance of reasonable-cause testing requires improving both the validity and reliability of the observational method for detecting physical and behavioral characteristics related to alcohol and drug use.

The mandatory alcohol and drug testing policy for employees with safety-sensitive functions has been challenged on a number of occasions regarding its constitutionality, admissibility and reliability of the test results, and conflict with the National Labor Relations Act (26). The mandatory testing programs were opposed by employers on the basis of unnecessary costs to their businesses and by unions on the grounds of unreasonable search in violation of the Fourth Amendment (26). The alcohol and drug testing policy has been upheld by the courts of law because it is implemented according to uniform and standardized procedural protocols and because it is supplemented by other components of the Federal Drug-Free Workplace Program, such as the employee assistance program that provides confidential counseling and referral to rehabilitation services. Findings from this study and other studies (3, 25) indicate that the mandatory testing programs hold sufficient safeguards against alcohol and drug abuse by aviation personnel.