Abstract
OBJECTIVES
Alcohol use disorders (AUDs) are common among patients admitted to an intensive care unit, yet systematic screening is rarely performed. We sought to confirm the construct validity of the full Alcohol Use Disorders Identification Test (AUDIT) and to evaluate the performance of the brief 3 item AUDIT-C using the full AUDIT as a proxy gold standard in a population of patients with a medical critical illness.
DESIGN
Secondary Analysis
SETTING
The Acute Respiratory Distress Syndrome (ARDS) Network, a consortium of 12 university centers (44 hospitals) dedicated to conducting multicenter clinical trials in patients with ARDS.
SUBJECTS
Patients meeting consensus criteria for Acute Respiratory Distress Syndrome
INTERVENTIONS
None
MEASUREMENTS AND MAIN RESULTS
Of 1,133 patients enrolled in one of three ARDS Network studies, 1,037 (92%) had full AUDIT data available. Of the included patients, 236 (23%) scored above the screening threshold for an alcohol use disorder on the full AUDIT. Construct validity analysis of the full AUDIT supported a three factor model. Compared to the full AUDIT, the AUDIT-C had an area under the receiver operating characteristic (AuROC) curve of 0.99 for men and 0.98 for women. The optimal cutoff was 4 for both genders. At this cutoff, the AUDIT-C had a sensitivity of 95% (95% CI 92%, 98%) and specificity of 94% (95% CI 92%, 96%) for men and sensitivity of 89% (95% CI 82%, 96%) and specificity of 99% (95% CI 98%, 100%) for women.
CONCLUSIONS
Though a 3 factor structure for the AUDIT was confirmed in ICU patients with ARDS, the first 3 questions focusing on alcohol consumption provide information that is comparable to the full 10 item AUDIT screening questionnaire. This study is limited by the lack of a true gold standard and the performance of the AUDIT-C is likely overestimated due to this limitation.
Keywords: alcohol use disorders identification test, acute lung injury, alcohol use disorder, alcohol misuse, unhealthy alcohol use
INTRODUCTION
Alcohol use disorders are prevalent in the intensive care unit (ICU). For example, up to 30% of patients with septic shock have an alcohol use disorder (AUD).1 Despite the high prevalence of AUDs in patients admitted to an ICU, systematic screening for AUDs is rarely performed. Because laboratory testing and clinical intuition inaccurately identify AUDs, systematic screening with questionnaires is necessary.2 Furthermore, identifying patients with an AUD may alter the clinical approach in regard to differential diagnosis and treatment plan by identifying patients at risk for an alcohol related condition.
Several validated alcohol screening questionnaires are available to screen for AUDs. One such screening questionnaire is the Alcohol Use Disorders Identification Test (AUDIT). The AUDIT consists of three domains or factors. It contains 3 questions regarding recent consumption, 3 questions focused on symptoms of dependence, and 4 questions focused on harmful alcohol use. Despite concerns that patients may provide false statements regarding their alcohol use, patients complete the AUDIT accurately when compared to the gold standard of diagnostic interviews and when corroborated by surrogate reports.3–5
In an effort to streamline screening for AUDs in clinical practice, prior studies have compared the first three questions of the AUDIT to the full AUDIT. These three questions focus on recent alcohol consumption and are referred to as the AUDIT-C. In the primary care setting, the AUDIT-C provides a similar sensitivity and specificity when compared to the full AUDIT for the identification of AUDs.5 However, the AUDIT-C has not been compared to the full AUDIT in patients with a medical critical illness. Because of the integral role of AUDs in Acute Respiratory Distress Syndrome (ARDS), the NHLBI ARDS network began collecting full AUDIT scores on all patients enrolled into clinical trials in 2007.1 We sought to confirm the construct validity of the full AUDIT and to evaluate the performance of the AUDIT-C using the full AUDIT as a proxy gold standard in a population of patients with a medical critical illness in 3 recently completed ARDS network clinical trials.
MATERIALS AND METHODS
We conducted a secondary analysis of data from 3 recent ARDS network trials. The ARDS network is a consortium of 12 academic medical centers (44 hospitals) dedicated to conducting clinical trials in patients with ARDS. Institutional Review Boards at each of the participating institutions reviewed and approved study protocols and informed consent was obtained from the patient or his or her surrogate prior to enrollment in the parent studies. This study using de-identified data was formally evaluated by the Colorado Multiple Institutional Review Board and was determined to not constitute human subjects research. The full inclusion and exclusion criteria for each study are described in the parent manuscripts.6–8 Patients included in this analysis were enrolled in the parent studies between 2007 and 2011. Data collected at the time of enrollment included the full 10 item AUDIT questionnaire completed by the patient or his or her surrogate. Data regarding demographics and clinical characteristics was collected as described in the parent manuscripts. 6–8
To analyze the construct validity of the full AUDIT in this population, confirmatory factor analysis was performed fitting the model to polychoric correlations using weighted least squares (WLS) and robust weighted least squares (RWLS). Models with one, two, and three factors were developed to determine which factor model best explained the variation in the data. Best model fit was determined by Satora-Bentler scaled chi-square (SB χ2), Comparative Fit Index (CFI), Root Mean Square Approximation (RMSEA), and the Tucker Lewis Index (TLI).
Scores on the full AUDIT questionnaire range from 0 to 40. Though there is debate about the proper screening threshold for the full AUDIT, we chose a cutoff of 5 for both men and women as a proxy gold standard for an alcohol use disorder. This cutoff is associated with a clear increase in the probability that a patient has an alcohol use disorder and is recommended by several authors as the appropriate threshold for screening.9
AUDIT-C scores range from 0–12. The performance of the AUDIT-C was analyzed using the full AUDIT as a proxy gold standard with a cutoff of 5 for both men and women. Gender specific receiver operating characteristic (ROC) curves were generated for the AUDIT-C and sensitivity and specificity were calculated for the AUDIT-C at different screening thresholds.
RESULTS
There were 1,133 patients enrolled in the 3 clinical trials; 1,037 (92%) had full AUDIT data available. In 39 of the 96 patients without a full AUDIT, the AUDIT-C was completed. Patients without full AUDIT data were more likely to be male (63% vs 51%; p = 0.03) but had a similar age, ethnicity, and severity of illness. Rates of AUDIT completion did not differ by admission location. Patients included in this sample had an average age of 52 (±16) years, were predominantly non-Hispanic whites (68%), and had an average Acute Physiology and Chronic Health Evaluation III score of 91 (±27). Overall, 236 (23%) patients scored above the screening threshold on the full AUDIT. Patients with alcohol misuse were younger, more likely to be male, had higher rates of current smoking, higher rates of comorbid cirrhosis, and were more likely to have trauma or aspiration as an ARDS risk factor (Table 1).
Table 1.
Characteristics of patients with and without alcohol misuse based on full AUDIT scores.
| No Alcohol Misuse (n=801) |
Alcohol Misuse (n=236) |
P-Value | |
|---|---|---|---|
| Age (mean, SD) | 53 (±17) | 48 (±13) | <0.001 |
| Race/Ethnicity (n, %) | 0.55 | ||
| White, Non-Hispanic | 548 (69) | 158 (67) | |
| African American | 133 (17) | 39 (17) | |
| White, Hispanic | 63 (8) | 16 (7) | |
| Other | 50 (6) | 21 (9) | |
| Gender (male) | 360 (45) | 166 (70) | <0.001 |
| Acute Physiology and Chronic Health Evaluation (APACHE) III (mean, SD) | 92 (±27) | 90 (±27) | 0.42 |
| Smoking Status (n, %) | <0.001 | ||
| Never Smoker | 397 (50) | 44 (19) | |
| Current Smoker | 211 (26) | 153 (65) | |
| Former Smoker | 190 (24) | 37 (16) | |
| Location (n, %) | 0.21 | ||
| Medical | 524 (65) | 147 (62) | |
| Surgical | 102 (13) | 41 (17) | |
| Medical/Surgical | 175 (22) | 48 (20) | |
| ARDS Risk Factor * | |||
| Trauma | 43 (5) | 25 (11) | <0.01 |
| Sepsis | 465 (58) | 133 (56) | 0.64 |
| Pneumonia | 569 (71) | 159 (67) | 0.28 |
| Aspiration | 129 (16) | 77 (33) | <0.001 |
| Comorbidity | |||
| Cirrhosis | 28 (4) | 23 (10) | <0.001 |
| Diabetes | 245 (31) | 40 (17) | <0.001 |
| CHF | 58 (7) | 7 (3) | 0.01 |
Patients may have more than 1 acute respiratory distress syndrome risk factor.
Construct validity analysis of the full AUDIT using the SB χ2 implied best fit for a three factor model. The RMSEA values were 0.029 for the three factor model, 0.031 for the two factor model, and 0.047 for the one factor model indicating that the three factor model had the best fit but one and two factor models were acceptable. The CFI and TLI both had values of 0.99 for all three models, indicating each model was equally acceptable.
In logistic regression models, the area under the ROC values for the AUDIT-C were 0.99 for men and 0.98 for women. The ROC curves demonstrated that a screening threshold of 4 maximized sensitivity and specificity for both men and women. At a cut off value of 4 the AUDIT-C had a sensitivity of 95% (95% CI 92%, 98%) and specificity of 94% (95% CI 92%, 96%) for men. For women, this cut of value had a sensitivity of 89% (95% CI 82%, 96%) and specificity of 99% (95% CI 98%, 100%). The sensitivity of the AUDIT-C for women was improved to 93% (95% CI 87%, 99%) with a cut off value of 3, while maintaining a specificity of 95% (95% CI 93%, 97%) (Table 2). When using a pre-test probability of 23%, the negative predictive value was 97% and 98% for men and women respectively at a cut off value of 4.
Table 2.
Performance of the AUDIT-C at different screening thresholds
| AUDIT-C Cutoff |
Sensitivity† | 95% CI | Specificity† | 95% CI |
|---|---|---|---|---|
| Male | ||||
| 2 | 0.99 | 0.97-1.00 | 0.83 | 0.79-0.87 |
| 3 | 0.98 | 0.96-1.00 | 0.90 | 0.87-0.93 |
| 4 | 0.95 | 0.92-0.98 | 0.94 | 0.92-0.96 |
| 5 | 0.84 | 0.78-0.90 | 1.00 | 1.00-1.00 |
| 6 | 0.72 | 0.65-0.79 | 1.00 | 1.00-1.00 |
| 7 | 0.61 | 0.53-0.68 | 1.00 | 1.00-1.00 |
| Female | ||||
| 2 | 0.96 | 0.91-1.00 | 0.91 | 0.88-0.94 |
| 3 | 0.93 | 0.87-0.99 | 0.95 | 0.93-0.97 |
| 4 | 0.89 | 0.82-0.96 | 0.99 | 0.98-1.00 |
| 5 | 0.80 | 0.71-0.89 | 1.00 | 1.00-1.00 |
| 6 | 0.69 | 0.58-0.80 | 1.00 | 1.00-1.00 |
| 7 | 0.53 | 0.41-0.65 | 1.00 | 1.00-1.00 |
Sensitivity and specificity are calculated using the full AUDIT as a proxy gold standard
DISCUSSION
In a secondary analysis of over 1000 patients with ARDS, approximately one quarter of patients had alcohol screening scores above the screening threshold for an AUD. Screening thresholds for the AUDIT-C determined in this population of patients with a medical critical illness were similar to those validated in other populations. Confirmatory factor analysis of the full AUDIT supported the three factor model previously established in the primary care setting. Most importantly, we demonstrate that the AUDIT-C provides information comparable to the full AUDIT. However, the AUDIT-C can be completed in less than a minute compared to the full AUDIT which may take up to 4 minutes to complete.
This analysis focuses on patients enrolled in ARDS clinical trials. Therefore, these results may not be applicable to critically ill patients in general. This study also lacked a gold standard. Instead, we relied on the full AUDIT as a proxy gold standard. While not a true gold standard for alcohol use disorders, the full AUDIT has area under the ROC value of 0.93 for men and 0.94 for women for identifying past year alcohol dependence in primary care patients.9 While previous studies have suggested that the full AUDIT may be more accurate in identifying an AUD than the AUDIT-C, this study is unable to assess accuracy due to lack of a true gold standard.10 Finally, this study was not meant to compare the AUDIT or AUDIT-C to other alcohol screening tests such as the Cut down, Annoyed, Guilty, Eye opener (CAGE) test or the Short Michigan Alcohol Screening Test so other alcohol screening questionnaires may be superior.
Although the AUDIT-C shows promise as a simple, consumption-based screening test for AUDs in clinical practice, further work is necessary to determine if it is the best screening test for patients with a medical critical illness. Future investigations of the AUDIT-C in the ICU should include a gold standard of diagnostic interviews in patients able to complete them as well as alcohol related biomarkers for comparison. Determining feasible and accurate screening tests for AUDs in the ICU setting may also pave the way for better epidemiological data on AUDs in the ICU as many current databases lack high quality alcohol screening data.
Acknowledgments
Supported by National Institutes of Health grants K23 AA-021814 (BJC), K24-HL-089223 (MM), and N01 HR56167.
Copyright form disclosures: Dr. Clark received support for article research from the National Institutes of Health (NIH). His institution received grant support from the NIH/NHLBI. Dr. Reisinger is employed University of Colorado Denver (Internal Medicine Residency Stipend). Dr. Moss received support for article research from the NIH. His institution received grant support from the NIH/NHLBI (award#K24HL089223), received support for travel and for participation in review activities from a NHLBI award, and received grant support from other NIH grant submissions.
REFERENCES
- 1.Moss M, Parsons PE, Steinberg KP, et al. Chronic Alcohol Abuse Is Associated with an Increased Incidence of Acute Respiratory Distress Syndrome and Severity of Multiple Organ Dysfunction in Patients with Septic Shock. Crit Care Med. 2003;31:869–877. doi: 10.1097/01.CCM.0000055389.64497.11. [DOI] [PubMed] [Google Scholar]
- 2.Gentilello LM, Villaveces A, Ries RR, et al. Detection of acute alcohol intoxication and chronic alcohol dependence by trauma center staff. J Trauma. 1999;47:1131–1135. doi: 10.1097/00005373-199912000-00027. discussion 5–9. [DOI] [PubMed] [Google Scholar]
- 3.Kershaw C, Martin GS, Moss M. Patient and Family Member Agreement on the Alcohol History of ICU Patients. Am J Respir Crit Care Med. 2003;167:A251. [Google Scholar]
- 4.Donovan DM, Dunn CW, Rivara FP, Jurkovich GJ, Ries RR, Gentilello LM. Comparison of trauma center patient self-reports and proxy reports on the Alcohol Use Identification Test (AUDIT) J Trauma. 2004;56:873–882. doi: 10.1097/01.ta.0000086650.27490.4b. [DOI] [PubMed] [Google Scholar]
- 5.Frank D, DeBenedetti AF, Volk RJ, Williams EC, Kivlahan DR, Bradley KA. Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups. J Gen Intern Med. 2008;23:781–787. doi: 10.1007/s11606-008-0594-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.National Heart L, Blood Institute Acute Respiratory Distress Syndrome Clinical Trials N. Matthay MA, et al. Randomized, placebo-controlled clinical trial of an aerosolized beta(2)-agonist for treatment of acute lung injury. Am J Respir Crit Care Med. 2011;184:561–568. doi: 10.1164/rccm.201012-2090OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.National Heart Lung, Blood Institute Acute Respiratory Distress Syndrome Clinical Trials Network. Rice TW, et al. Initial trophic vs full enteral feeding in patients with acute lung injury: the EDEN randomized trial. JAMA. 2012;307:795–803. doi: 10.1001/jama.2012.137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Rice TW, Wheeler AP, Thompson BT, et al. Enteral omega-3 fatty acid, gamma-linolenic acid, and antioxidant supplementation in acute lung injury. JAMA. 2011;306:1574–1581. doi: 10.1001/jama.2011.1435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Rubinsky AD, Dawson DA, Williams EC, Kivlahan DR, Bradley KA. AUDIT-C scores as a scaled marker of mean daily drinking, alcohol use disorder severity, and probability of alcohol dependence in a U.S. general population sample of drinkers. Alcohol Clin Exp Res. 2013;37:1380–1390. doi: 10.1111/acer.12092. [DOI] [PubMed] [Google Scholar]
- 10.Kriston L, Holzel L, Weiser A, Berner MM, Harter M. Meta-analysis: Are 3 Questions Enough to Detect Unhealthy Alcohol Use? Ann Int Med. 2008;149:879–887. doi: 10.7326/0003-4819-149-12-200812160-00007. [DOI] [PubMed] [Google Scholar]