Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 1.
Published in final edited form as: Drug Alcohol Depend. 2016 Sep 22;168:196–202. doi: 10.1016/j.drugalcdep.2016.09.015

Comparison of AUDIT-C Collected via Electronic Medical Record and Self-Administered Research Survey in HIV Infected and Uninfected Patients

Kathleen A McGinnis 1, Janet P Tate 1,2,3, Emily C Williams 4, Melissa Skanderson 1, Kendall J Bryant 5, Adam Gordon 6,7, Kevin L Kraemer 6, Stephen A Maisto 8, Steven Crystal 9, David A Fiellin 1,2,3, Amy C Justice 1,2,3
PMCID: PMC5086273  NIHMSID: NIHMS820251  PMID: 27694059

Abstract

Background

Using electronic medical record (EMR) data for clinical decisions, quality improvement, and research is common. While unhealthy alcohol use is particularly risky among HIV infected individuals (HIV+), the validity of EMR data for identifying unhealthy alcohol use among HIV+ is unclear. Among HIV+ and uninfected, we: 1) assess agreement of EMR and research AUDIT-C at validated cutoffs for unhealthy alcohol use; 2) explore EMR cutoffs that maximize agreement; and 3) assess subpopulation variation in agreement.

Methods

Using data from the Veterans Aging Cohort Study (VACS), EMR AUDIT-C cutoffs of 2+, 3+, and 4+ for men (2+ and 3+ for women) were compared to research AUDIT-C 4+ for men (3+ for women). Agreement was compared by demographics, HIV, hepatitis C infection, and alcohol related diagnosis.

Results

Among 1,082 HIV+ and 1,160 uninfected men, 14% and 22% had an EMR and research AUDIT-C 4+, respectively. Among 32 HIV+ and 115 uninfected women, 9% and 14% had an EMR and research AUDIT-C 3+. For men, EMR agreement with the research AUDIT-C 4+ was highest at a cutoff of 3+ (kappa = 0.49). For women, EMR agreement with AUDIT-C 3+ was highest at a cutoff of 2+ (kappa = 0.60). Moderate agreement was consistent across subgroups.

Conclusions

EMR AUDIT-C underestimates unhealthy alcohol use compared to research AUDIT-C in both HIV+ and uninfected individuals. Methods for improving quality of clinical screening may be in need of investigation. Researchers and clinicians may consider alternative EMR cutoffs that maximize agreement given limitations of clinical screening.

Keywords: Alcohol consumption, population-based screening, AUDIT-C, electronic health record, veterans, HIV

1. INTRODUCTION

Using electronic medical record (EMR) data for clinical decisions, quality improvement, and research is common; however, the validity of EMR data for identifying unhealthy alcohol use among those infected with HIV (HIV+) is unclear. Unhealthy alcohol use includes any amount of drinking above “low risk” drinking and ranges from risky drinking to alcohol use disorders (Bradley et al., 2009; Saitz, 2005). Unhealthy alcohol use is common among those with HIV with estimates of 8% for alcohol abuse (Galvan et al., 2002; McGinnis et al., 2013), and 20% to 41% for hazardous use (Conigliaro et al., 2003; Justice et al., 2016; McGinnis et al., 2013). Among HIV+ individuals, unhealthy alcohol use has been associated with non-adherence to antiretroviral therapy (ART; Hendershot et al., 2009), worse HIV disease (Hahn et al., 2010), mortality and physiological frailty (Justice et al., 2016) and worse quality of care (Korthuis et al., 2012). Some HIV+ patients do not take ART in advance of drinking to avoid medication interactions (Kalichman et al., 2015; Bilal et al., 2016). Further, intoxication, which occurs at lower levels of drinking for HIV+ relative to HIV- patients (McGinnis et al., 2016), is associated with increased risky sexual behavior (Cook et al., 2006; Stall et al., 1986; Scott-Sheldon et al., 2013, 2016), which may increase risk of HIV transmission (Cook et al., 2006; Petry et al., 1999; Stall et al., 1986; Shuper et al., 2010). HIV+ individuals may be more sensitive to the effects of alcohol (McCance-Katz et al., 2012; McGinnis et al., 2016) and experience increased mortality and physiologic injury at lower levels of alcohol use compared to those uninfected (Justice et al., 2016). Identifying unhealthy alcohol use among HIV+ can provide an opportunity for intervention, and guide clinical quality improvement and research efforts designed to provide tailored patient care.

The Alcohol Use Disorders Identification Test – Consumption (AUDIT-C) questionnaire has been routinely collected and entered into the EMR system in the Veterans Health Administration (VHA) since 2006 (Bradley et al., 2006) and provides a source of information regarding unhealthy alcohol use for a large cohort of HIV+ and uninfected veterans. However, studies have found that the prevalence of unhealthy alcohol use is lower when based on EMR AUDIT-C, which are collected primarily face-to-face in a clinical setting (Williams et al., 2015), compared to AUDIT-C collected via mailed surveys (Bradley et al., 2011; Hawkins et al., 2007, Lapham et al., 2013). Lower prevalence using EMR AUDIT-C compared to self-administered survey AUDIT-C could be due to the mostly face-to-face mode of collection of EMR data as there is: 1) documented variation in how AUDIT-C is administered and documented by clinic staff including skipping the 3rd AUDIT-C item and altering items (and potential response options) in attempt to enhance patient comfort (Williams et al., 2015); and 2) social desirability bias inherent in face-to-face collection of sensitive issues - disclosure of sensitive information is lower with face-to-face administration compared to paper administration (Bowling, 2005). In contrast, wording and response items on paper surveys are fixed and consistent and responses may be less subject to social desirability bias, especially if the information is collected confidentially.

Patient comfort and social desirability bias may be an even greater issue in HIV clinics due to potential implications or consequences for reporting alcohol use. Treatment for conditions that are commonly comorbid with HIV, such as HCV, have traditionally been contraindicated for patients who use alcohol, which may be a disincentive for accurately reporting alcohol use in clinical settings. Therefore, agreement of EMR with research AUDIT-C screening results may vary by HIV status and this has not been previously examined. Similarly, screening results may vary by HCV, race/ethnicity, site, and/or alcohol related diagnosis.

The main goals of the current study were to 1) assess agreement of EMR and research AUDIT-C at validated cutoffs for identifying unhealthy alcohol use; 2) explore EMR AUDIT-C cutoffs that maximize agreement with research AUDIT-C; and 3) assess subpopulation variation in agreement.

2. MATERIALS AND METHODS

2.1 Veterans Aging Cohort 8 Site Study (VACS-8)

We used data from the VACS 8 Site Study (VACS-8) which has been described in detail previously (Justice et al., 2006). Briefly, VACS-8 is a prospective cohort study conducted at eight VHA facilities in the United States (Atlanta, Georgia; Baltimore, Maryland; Bronx, New York; Houston, Texas; Los Angeles, California; New York City, New York; Pittsburgh, Pennsylvania and Washington, District of Columbia). Recruitment and enrollment in VACS began in June, 2002 and is ongoing. HIV+ individuals are recruited from Infectious Disease clinics at participating sites. Uninfected controls are recruited from General Internal Medicine clinics at the same sites, and are targeted to match the demographics of the Infectious Diseases clinics by 5-year age blocks, race/ethnicity, and year of active VHA healthcare use. After patients were consented to the study and at annual follow-ups, participants completed a comprehensive self-administered paper survey that includes the AUDIT-C. Usually surveys were completed before, after, or in between appointments; sometimes patients took the survey and returned it to the research coordinator at a later time. The coordinator was not typically present when the subjects were completing the surveys and in some situations (the minority of the time) the follow-up surveys were mailed to the participants. The study consent informs participants that their survey responses will remain confidential. For this analysis, we include AUDIT-C data from a VACS participant’s first completed survey between October, 2007 and September, 2012. This could be the baseline survey or follow-up waves 4, 5, and 6 for previously enrolled participants. These survey data are referred to as “research AUDIT-C”.

2.2 Veterans Health Administration (VHA) EMR Data

We used AUDIT-C data available through the VHA Corporate Data Warehouse (CDW) from October, 2007 to September, 2012. The CDW is a national repository that incorporates data from clinical and administrative systems into one standard database structure and is described in detail elsewhere (McGinnis et al., 2011). AUDIT-C and information on a variety of health behaviors (such as smoking and depressive symptoms) are collected nationally using clinical reminder prompts to clinicians (Bradley et al., 2011; McGinnis et al., 2011; Williams et al., 2015). We identified an EMR AUDIT-C reported within 90 days before or after the research AUDIT-C. The 90 day window was utilized because it was previously reported to provide similar results as narrower timeframes (Bradley et al., 2011). These data are referred to as “EMR AUDIT-C.”

2.3 Analyses

We describe demographic characteristics and AUDIT-C results by HIV status for VACS participants with an EMR AUDIT-C within 90 days of a research AUDIT-C. The t-test was used to compare age by HIV status and chi-square tests were used to compare categorical variables by HIV status.

For men, we assessed agreement between EMR AUDIT-C and research AUDIT-C 4+ using agreement percent, sensitivity, specificity, and kappa statistics. Because prior research found there is under-reporting of unhealthy alcohol use with EMR AUDIT-C, we evaluated EMR AUDIT-C cutoffs of 2+, 3+, and 4+. The threshold of greater than or equal to 4 for research AUDIT-C is based on prior work demonstrating that sensitivity and specificity of unhealthy alcohol use is maximized for men at this cutoff (Babor et al., 1989; Bradley et al., 2003, 2007; Bush et al., 1998). For the purpose of calculating sensitivity and specificity (two types of agreement), we use research AUDIT-C 4+ as the reference group because, as mentioned previously, we believe it is subject to less social desirability bias and data collection errors than EMR AUDIT-C. We calculated kappa statistics for the overall sample and separately by HIV status. The kappa statistic measures agreement beyond that expected by chance alone and ranges from 0 to 1, with 1 representing perfect agreement. Landis and Koch suggest interpreting intermediate values as follows: 00. – poor; .00 to .20 – slight; .21 to .40 – fair; .41 to .60 – moderate, .61 to .80 – substantial, and .81 to 1.00 – almost perfect (Landis and Koch, 1977).

We calculated kappa statistics, percent agreement, sensitivity, and specificity in subgroups by age, race/ethnicity, site, HCV, and alcohol related diagnosis. Kappa statistics were compared by HIV status and gender; and for men by age, race/ethnicity, site, hepatitis C status, and alcohol related diagnosis. As a sensitivity analysis, overall kappa statistics were also generated allowing for only 60 and 30 days between EMR and research AUDIT-C. Z-tests were used to determine whether kappa statistics were statistically significantly different between groups.

Because the recommended AUDIT-C cutoff for unhealthy drinking for women is 3+ (vs. 4+ for men) (Bradley et al., 2003; Bradley et al., 2007), we compared EMR AUDIT-C cutoffs of 2+ and 3+ to research AUDIT-C 3+ for women. The sample size for women is much smaller than for men so we did not compare agreement between subgroups for women. Analyses were run using Stata 13.0.

3. RESULTS

Of the 7,515 individuals (3,629 HIV+ and 3,501 uninfected men; and 99 HIV+ and 286 uninfected women) who completed a survey for VACS Baseline 2002 from June, 2002 to September, 2012, 6,766 were alive as of October, 2007 and 5,491 had a baseline or follow-up survey between October, 2007 and September, 2012. Of those, 5,179 (94%) had a non-missing research AUDIT-C value on their first VACS survey within the time frame, and 5,344 (97%) had at least one EMR AUDIT-C. Of those, 2,242 men (1,082 HIV+ and 1,160 uninfected), and 147 women (32 HIV+ and 115 uninfected) had an EMR AUDIT-C within 90 days before or after the research AUDIT-C.

3.1 HIV+ and Uninfected Men

The mean age at the time of the VACS survey was 53 years for HIV+ and 56 years for uninfected men. Of the HIV+ and uninfected men, respectively, 68% and 65% were African-American, 20% and 23% were white, and 12% and 12% were of Hispanic or other race/ethnicity. For HIV+ and uninfected men, fewer had an EMR AUDIT-C 4+ (14% and 15%) than a research AUDIT-C 4+ (21% and 23%) and more had an EMR AUDIT-C of zero (45% and 53%) than a research AUDIT-C of zero (39% and 45%) (Table 1).

Table 1.

Characteristics of Male Participants with EMR and VACS Research AUDIT-C

HIV+ Uninfected P-Value
N 1,082 1,160
Mean Age, (SD) 53 (8.6) 56 (9.4) <.001
Race/Ethnicity (%) .3
  White 20 23
  African American 68 65
  Hispanic/Other 12 12
HCV 51 32 <.001
Site
  1 20 12 <.001
  2 5 8
  3 15 15
  4 12 14
  5 13 19
  6 13 9
  7 18 21
  8 4 4
Alcohol Related Dx 17 26 <.001
EMR AUDIT-C (%)
  0 45 53 <.001
  1–2 34 26
  3 8 6
  4+ 14 15
Research AUDIT-C (%)
  0 39 45 .001
  1–2 30 24
  3 10 9
  4+ 21 23
Open in a new tab

EMR, electronic medical record; SD, standard deviation

Table 2 shows agreement of EMR AUDIT-C (cutoffs of 4+, 3+ and 2+) with research AUDIT-C (4+). For both HIV+ and uninfected men, using research AUDIT-C 4+ as the reference group, EMR AUDIT-C 2+ had the highest sensitivity, indicating that of those with research AUDIT-C 4+, 74% and 71% had EMR AUDIT-C of 2+. EMR AUDIT-C 3+ had sensitivity of 59% and 59%, and EMR AUDIT-C 4+ the lowest sensitivity (42% and 47%). Conversely, EMR AUDIT-C 4+ had the highest specificity, indicating that of those with research AUDIT-C<4, 94% and 95% of HIV+ and uninfected men also had an EMR AUDIT-C of <4. EMR AUDIT-C 3+ had specificity of 88% and 90%, and EMR AUDIT-C 2+ had the lowest specificity (78% and 82%). Based on kappa statistics, EMR AUDIT-C 3+ had better overall agreement than EMR 2+ and EMR 4+ for both HIV+ and uninfected men (kappa = 0.47 and 0.51 vs. 0.42 and 0.47; and 0.41 and 0.48; all p<.05) (Table 2).

Table 2.

Comparison of EMR and Research AUDIT-C Screening Results in Men

HIV+ Uninfected
Research Research
AUDIT-C 4+ AUDIT-C <4 N AUDIT-C 4+ AUDIT-C <4 N
EMR (n=225) (n=857) (n=266) (n=894)
AUDIT-C 4+ 95 (42%) 55 (6%) 150 126 (47%) 45 (5%) 171
AUDIT-C <4 130 (58%) 802 (94%) 931 140 (52%) 849 (95%) 989
Kappa 0.41 0.48
EMR
AUDIT-C 3+ 133 (59%) 100 (12%) 233 157 (59%) 85 (10%) 242
AUDIT-C <3 92 (41%) 757 (88%) 848 109 (41%) 809 (90%) 918
Kappa 0.47 0.51
EMR
AUDIT-C 2+ 166 (74%) 192 (22%) 357 189 (71%) 164 (18%) 353
AUDIT-C <2 59 (26%) 665 (78%) 724 77 (29%) 730 (82%) 807
Kappa 0.42 0.47
Open in a new tab

Bold numbers represent agreement. All percentages are column percents

To evaluate agreement within subgroups we compared EMR AUDIT-C 3+ cutoff to research AUDIT-C 4+. Agreement was moderate (kappa >0.40) for all subgroups examined except site 2. Overall agreement, based on kappa statistics, was statistically significantly higher for: men compared to women; and among men: HIV uninfected compared to HIV+, those ages 50–64 and 65+ compared to those <50, white compared to African-American, Hispanic, and other race/ethnicity, sites 1, 4 and 8 compared to other sites, HCV uninfected vs. HCV+, and those without an alcohol related diagnosis compare to those with (Table 3). Kappa statistics were statistically significantly different between all age groups except 50–64 and 65+; all racial/ethnic groups; all sites; and by HCV status and alcohol related diagnosis (yes/no).

Table 3.

Unhealthy Alcohol Use in Men in the Veterans Aging Cohort Study (VACS), by Demographics and Characteristics, Based on VACS Research Survey and Electronic Medical Record AUDIT-C

N Research
AUDIT-C 4+
(%)		 EMR
AUDIT-C 3+
(%)		 Agreement
(%)		 Sensitivity*
(%)		 Specificity*
(%)		 Kappa
All 2242 21.9 21.2 82.8 59.1 89.4 0.49
HIV+ Yes 1082 20.8 21.5 82.2 59.1 88.3 0.47
No 1160 22.3 20.9 83.3 59.0 90.5 0.51
Age <50 602 25.7 24.8 79.1 57.4 86.6 0.45
50–64 1394 21.9 20.8 83.4 59.7 90.1 0.51
65+ 246 12.6 14.6 88.2 61.3 92.1 0.50
Race/
Ethnicity 		White 485 20.6 17.5 87.0 61.0 93.8 0.58
African-
Amerian		 1489 21.6 22.1 81.9 59.2 88.1 0.49
Hispanic 185 29.7 23.8 77.8 52.7 88.5 0.44
Other 83 18.1 20.5 85.5 66.7 89.7 0.54
Site 1 351 19.4 19.9 86.9 67.7 91.5 0.59
2 142 27.5 24.6 76.1 51.3 85.4 0.38
3 331 22.1 17.2 81.3 46.6 91.1 0.41
4 293 25.3 23.9 83.6 64.9 90.0 0.56
5 360 23.3 20.8 81.9 56.0 90.0 0.48
6 244 17.2 23.8 81.1 64.3 854. 0.42
7 439 20.7 20.7 83.1 59.3 89.4 0.49
8 82 24.4 23.2 86.6 70.0 91.9 0.63
HCV Yes 921 22.3 21.1 81.6 56.1 89.0 0.46
No 1321 21.7 22.3 83.6 61.2 89.9 0.51
Alcohol Yes 487 32.4 33.1 76.3 64.6 82.0 0.46
Related Dx No 1755 19.0 17.9 84.6 56.5 91.1 0.49
Open in a new tab
*

Sensitivity and Specificity assume that research AUDIT-C is the reference group; sensitivity represents concordance of EMR AUDIT-C with positive research AUDIT-C; specificity represents concordance of EMR AUDIT-C with negative research AUDIT-C.

Compared to using the 90 day window, kappa statistics were similar for 60 and 30 day windows (kappa = 0.49, n = 2,242 for the 90 day window; kappa = 0.49, n = 1,694 for the 60 day window; and kappa = 0.51, n = 1,127 for the 30 day window).

3.2 HIV+ and Uninfected Women

The mean age at the time of the VACS survey was 49 years for HIV+ and 50 years for uninfected women. Race/ethnicity was similar by HIV status; overall 72% were African-American, 17% were white, and 11% were of Hispanic or other race/ethnicity. For HIV+ and uninfected women, 9% and 9% had an EMR AUDIT-C 3+ and 16% and 13% had a research AUDIT-C of 3+. EMR AUDIT-C 2+ had higher sensitivity than EMR AUDIT-C 3+ for both HIV+ and uninfected women (60% and 67% vs. 40% and 40%). However, EMR AUDIT-C 3+ had higher specificity than AUDIT-C 2+ for both HIV+ and uninfected women (96% and 96% vs. 93% and 87%).

Overall agreement, based on kappa statistics, was higher for EMR AUDIT-C 2+ (kappa = 0.46) than for EMR AUDIT-C 3+ (kappa = 0.42), using research AUDIT-C 3+ as the reference group. Agreement of EMR AUDIT-C 2+ with research AUDIT-C 3+ was moderate for both HIV+ (kappa = 0.53) and uninfected women (kappa = 0.44), and the difference is statistically significant (p<.05). Overall agreement was better for men compared to women (kappa= 0.49 vs. 0.46, p<.05).

4. DISCUSSION

EMR AUDIT-C identifies a lower percent with unhealthy alcohol use compared to the same cutoff for research AUDIT-C for both HIV+ and uninfected men and women. Using a lower threshold for EMR AUDIT-C (3+ for men; 2+ for women vs. 4+ for men; 3+ for women) identifies a higher percent with potentially unhealthy alcohol use and results in better agreement with research AUDIT-C (4+ for men; 3+ for women). This pattern is consistent for all subgroups examined and agreement between EMR AUDIT-C and research AUDIT-C was moderate for almost all subgroups compared.

Finding a lower percent with a positive AUDIT-C screen using EMR compared to research AUDIT-C is consistent with prior research. Three studies that compared VA EMR AUDIT-C to AUDIT-C from a mailed survey reported that the percent with a positive screen was lower for EMR compared to the mailed survey data (Bradley et al., 2011; Hawkins et al., 2007; Lapham et al., 2013). This finding is also consistent with a study on the effects of mode of question administration which found that disclosure of sensitive information was lower with face to face administration compared to paper administration (Bowling, 2005).

To put these findings in the context of prior studies, sensitivity reflects agreement/concordance of EMR AUDIT-C with a positive research AUDIT-C and specificity reflects agreement/concordance of EMR AUDIT-C with a negative research AUDIT-C. Our overall finding in men of 59% sensitivity and 89% specificity is similar to findings in other studies of 61% sensitivity and 93% specificity using AUDIT-C 4+ cutoffs (Hawkins et al., 2007) and 39% sensitivity and 99% specificity using AUDIT-C 5+ cutoffs (Bradley et al., 2011). Finding that sensitivity is lower in African-American and Hispanic men compared to white men and for certain sites is consistent with Bradley’s research which also found that agreement was worse for African-Americans and for certain VHA networks (Bradley et al., 2011). Another VHA study by Lapham et al (2013) examined agreement between EMR and survey AUDIT-C using cutoffs of 4+ for men and 3+ for women among patients with 1 to 4 prior negative screens. Prevalence of positive AUDIT-C was lower for EMR than for survey screening, and sensitivity was low and decreased from 40.6% to 17.4% as the number of prior negative screens increased from 1 to 4. The EMR screen performed similarly for women and men, but performed better for older (≥ 65 years) compared to younger (< 50 years) patients in this study by Lapham et al. (2013).

That EMR AUDIT-C 3+ had better agreement with research AUDIT-C 4+ than EMR AUDITC 4+ suggests that using a cutoff of 3 may optimize sensitivity and specificity for identifying both HIV+ and uninfected men with unhealthy alcohol use when using the current VHA EMR data. Although other studies have compared EMR AUDIT-C to survey or research AUDIT-C, no other studies have made the comparisons in an HIV+ population, and none have identified lower cut-points that may more accurately identify patients with unhealthy alcohol use based on EMR screening. Identifying HIV+ individuals with unhealthy alcohol use provides an opportunity to provide alcohol interventions which, in addition to improving drinking outcomes, may ultimately improve adherence, overall health and mortality risk, and also reduce the risk of HIV transmission (Samet, et al., 2010; Parsons et al, 2007a; Chander et al., 2015; Hasin et al., 2013). Having more accurate assessments of unhealthy alcohol use could also be important for alerting providers to monitor more closely for alcohol related liver injury and drug interactions. Because alcohol impacts those with HIV at lower levels of use than among uninfected, it may be especially important to use a lower cutoff of AUDIT-C among HIV+.

We recommend that the EMR AUDIT-C screening be improved in order to reduce administration error and social desirability bias. Consistent with recommendations by Williams et al (2015), use of patient-administered screening tools, such as laminate, paper-based, or web-based tools, may address some of the current limitations of clinical screening.

There are several strengths of this analysis. We used data from the VHA, which benefits from one of the most highly-developed health information systems in the world (Corrigan et al., 2003; McQueen et al., 2004). The sample is demographically diverse, comprises multiple sites, and did not require substantial data resources or any additional participant burden. Compared to prior studies that used data on predominantly white populations in 2003–2004, 2006–2008, and 2007–2008, the VACS study population is comprised predominantly of minorities and uses AUDIT-C collected from 2007 to 2012.

There are several limitations to this study. Only 6% of the sample is women so we were underpowered to run analyses on subgroups for women, and due to the small number of HIV+ women in this study, caution is warranted when interpreting the results for women. Both methods of identifying unhealthy alcohol use are based on a 3-item screen for unhealthy alcohol use (the AUDIT-C), as opposed to an in-depth assessment. While the AUDIT-C has been validated with high sensitivity and specificity for identifying unhealthy alcohol use and accounts for under-reporting (Bradley et al., 2003; Bush et al., 1998; Bradley et al., 2007; Frank et al., 2008), some mis-categorization is expected by design. However, similar to the validation studies, the research AUDIT-C questions and response options are fixed whereas there is variation in the way EMR AUDIT-C questions and response options are administered in the VHA, as noted previously (Williams et al., 2015). Although the demographics of the HIV+ and uninfected groups are similar, some differences exist (e.g., differential representation of women) and the demographic make-up of these groups may have influenced results. Similarly, it is unknown how alcohol use in this sample that includes 8 VHA sites compares to that of the general population of VHA outpatients. While patterns of any and unhealthy alcohol use have not been directly compared between the VACS matched sample and general VHA outpatients, survey AUDIT-C distributions from the present study were similar to those reported in a previous study among a national sample of VHA outpatients (Williams et al., 2015).

Despite limitations of EMR AUDIT-C, EMR data hold the potential to help clinicians and researchers identify unhealthy alcohol use overall and in HIV+ and other subgroups. For example, these data provide the opportunity to generate large cohorts of patients who consume alcohol, evaluate health outcomes by AUDIT-C levels, track change in alcohol use over time, enroll patients into alcohol intervention programs, assess the impact of alcohol use interventions, and measure performance for quality improvement initiatives. EMR data can be retrieved efficiently, longitudinally, at low cost, and in a comprehensive cohort of patients. In addition, this methodology for using EMR data can serve as a useful model for other healthcare organizations as they use or transition to EMRs. Findings from this study add to previous literature suggesting limitations regarding the quality of clinical alcohol screening administered in this large healthcare system and help identify cut-points that maximize identification of unhealthy alcohol use in these populations given the current limitations of clinically-administered screening. Researchers and clinicians using AUDIT-C from EMRs should consider replicating this work, particularly in a sample containing a greater number of women, to determine optimal EMR AUDIT-C cutoffs relevant to their research or clinical purposes and taking into consideration the type of mode and administration of the AUDIT-C being used.

Highlights.

  • Electronic medical record (EMR) AUDIT-C under represents alcohol consumption

  • EMR-based AUDIT-C may warrant lower thresholds

  • Strategies to improve the quality of EMR-based AUDIT-C should be pursued

Acknowledgments

Role of funding source

The funders of this study had no further role in its design, collection, analysis and interpretation of data, writing of the report, or in the decision to submit the paper for publication.

The views are not those of the Department of Veterans Affairs or the United States Government. COMpAAAS/Veterans Aging Cohort Study, a CHAART Cooperative Agreement, is supported by the National Institutes of Health: National Institute on Alcohol Abuse and Alcoholism (U24-AA020794, U01-AA020790, U01-AA020795, U01-AA020799; U10 AA013566-completed) and in kind by the US Department of Veterans Affairs. Dr. Williams is supported by a Career Development Award from VA Health Services Research & Development (CDA 12–276).

List of Abbreviations

VACS

Veteran Aging Cohort Study

HIV+

HIV Infected

AUDIT-C

Alcohol Use Disorder Identification Test- Consumption

HCV

Hepatitis C Virus

Alc Rel Dx

alcohol related International Classification of Diseases, Ninth Revision diagnosis

EMR

Electronic Medical Record

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributors
  • Kathleen A. McGinnis-Dr. McGinnis’ contributions to this submission were conceptionand design, analysis and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content and statistical analysis.
  • Janet P. Tate-Dr. Tate’s contributions to this submission were conception and design, acquisition of data, analysis and interpretation of data, critical revision of the manuscript for important intellectual content, and supervision.
  • Emily C. Williams-Dr. Williams contributions to this submission were analysis and interpretation of data and critical revision of the manuscript for important intellectual content.
  • Melissa Skanderson-Ms. Skanderson’s contributions to this submission were conception and design, acquisition of data, analysis and interpretation of data, critical revision of the manuscript for important intellectual content, and supervision.
  • Kendall J. Bryant-Dr. Bryant’s contributions to this submission were conception and design, analysis and interpretation of data and critical revision of the manuscript for important intellectual content.
  • Adam J. Gordon-Dr. Gordon’s contribution to this submission was critical revision of the manuscript for important intellectual content.
  • Kevin L. Kraemer-Dr. Kraemer’s contributions to this submission were analysis and interpretation of data and critical revision of the manuscript for important intellectual content.
  • Stephen A. Maisto-Dr. Maisto’s contributions to this submission were analysis and interpretation of data and critical revision of the manuscript for important intellectual content.
  • Steven Crystal-Dr. Crystal’s contributions to this submission were analysis and interpretation of data and critical revision of the manuscript for important intellectual content.
  • David A. Fiellin-Dr. David Fiellin’s contributions to this submission were conception and design, analysis and interpretation of data and critical revision of the manuscript for important intellectual content.
  • Amy C. Justice-Dr. Justice’s roles for this submission were conception and design, acquisition of data, analysis and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, statistical analysis, obtaining funding, administrative, technical or material support and supervision.

Conflicts of interest

The manuscript has not been previously published and is not being considered for publication elsewhere. The authors have all approved the manuscript and have no conflicts of interest to declare.

References

  1. Babor TF, de la Feunte JR, Saunders J, Grant M. World Health Organization. Geneva: WHO/MNH/DAT 89.4; 1989. AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care; pp. 1–30. [Google Scholar]
  2. Bilal U, Lau B, Lazo M, McCaul ME, Hutton HE, Suldowski MS, Chander G. Interaction between alcohol consumption patterns, antiretroviral therapy type, and liver fibrosis in persons living with HIV. AIDS Patient Care ST. 2016;30:200–207. doi: 10.1089/apc.2016.0010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bowling A. Mode of questionnaire administration can have serious effects on data quality. J. Public Health (Oxf.) 2005;27:281–291. doi: 10.1093/pubmed/fdi031. [DOI] [PubMed] [Google Scholar]
  4. Bradley KA, Bush KR, Epler AJ, Dobie DJ, Davis TM, Sporleder JL, Maynard C, Burman ML, Kivlahan DR. Two brief alcohol-screening tests from the Alcohol Use Disorders Identification Test (AUDIT): validation in a female Veterans Affairs patient population. Arch. Intern. Med. 2003;163:821–829. doi: 10.1001/archinte.163.7.821. [DOI] [PubMed] [Google Scholar]
  5. Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol. Clin. Exp. Res. 2007;31:1208–1217. doi: 10.1111/j.1530-0277.2007.00403.x. [DOI] [PubMed] [Google Scholar]
  6. Bradley KA, Kivlahan DR, Williams EC. Brief approaches to alcohol screening: practical alternatives for primary care. J. Gen. Intern. Med. 2009;24:881–883. doi: 10.1007/s11606-009-1014-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bradley KA, Lapham GT, Hawkins EJ, Achtmeyer CE, Williams EC, Thomas RM, Kivlahan DR. Quality concerns with routine alcohol screening in VA clinical settings. J. Gen. Intern. Med. 2011;26:299–306. doi: 10.1007/s11606-010-1509-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bradley KA, Williams EC, Achtmeyer CE, Volpp B, Collins BJ, Kivlahan DR. Implementation of evidence-based alcohol screening in the Veterans Health Administration. Am. J. Manag. Care. 2006;12:597–606. [PubMed] [Google Scholar]
  9. Braithwaite RS, Conigliaro J, Roberts MS, Shechter S, Schaefer A, McGinnis K, Rodriguez MC, Rabeneck L, Bryant K, Justice AC. Estimating the impact of alcohol consumption on survival for HIV+ individuals. AIDS Care. 2007;19:459–466. doi: 10.1080/09540120601095734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Braithwaite RS, Justice AC, Chang CC, Fusco JS, Raffanti SR, Wong JB, Roberts MS. Estimating the proportion of patients infected with HIV who will die of comorbid diseases. Am. J. Med. 2005a;118:890–898. doi: 10.1016/j.amjmed.2004.12.034. [DOI] [PubMed] [Google Scholar]
  11. Braithwaite RS, McGinnis KA, Conigliaro J, Maisto SA, Crystal S, Day N, Cook RL, Gordon A, Bridges MW, Seiler JF, Justice AC. A temporal and dose-response association between alcohol consumption and medication adherence among veterans in care. Alcohol. Clin. Exp. Res. 2005b;29:1190–1197. doi: 10.1097/01.alc.0000171937.87731.28. [DOI] [PubMed] [Google Scholar]
  12. Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch. Intern. Med. 1998;158:1789–1795. doi: 10.1001/archinte.158.16.1789. [DOI] [PubMed] [Google Scholar]
  13. Chander G, Hutton HE, Lau B, Xu X, McCaul ME. Brief intervention decreases drinking frequency in HIV-Infected, heavy drinking women: results of a randomized controlled trial. J. Acquir. Immune Defic. Syndr. 2015;70:137–145. doi: 10.1097/QAI.0000000000000679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Cook RL, McGinnis KA, Kraemer KL, Gordon AJ, Conigliaro J, Maisto SA, Samet JH, Crystal S, Rimland D, Bryant KJ, Braithwaite RS, Justice AC. Intoxication before intercourse and risky sexual behavior in male veterans with and without human immunodeficiency virus infection. Med. Care. 2006;44:S31–S36. doi: 10.1097/01.mlr.0000223710.35008.d9. [DOI] [PubMed] [Google Scholar]
  15. Cook RL, Clark DB. Is there an association between alcohol consumption and sexually transmitted diseases? A systematic review. Sex. Transm. Dis. 2005;32:156–164. doi: 10.1097/01.olq.0000151418.03899.97. [DOI] [PubMed] [Google Scholar]
  16. Corrigan JM, Eden J, Smith BM. Leadership by Example: Coordinating Government Roles in Improving Health Care Quality (Quality Chasm) Washington, DC: National Academies Press; 2003. [Google Scholar]
  17. Frank D, DeBenedetti AF, Volk RJ, Williams EC, Kivlahan DR, Bradley KA. Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups. J. Gen. Intern. Med. 2008;23:781–787. doi: 10.1007/s11606-008-0594-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Freeman RC. Toward development of enhanced preventive interventions for HIV sexual risk among alcohol-using populations: confronting the ‘mere pause from thinking’. AIDS Behav. 2016;20:1–18. doi: 10.1007/s10461-015-1179-7. [DOI] [PubMed] [Google Scholar]
  19. Galvan FH, Bing EG, Fleishman JA, London AS, Caetano R, Burnam MA, Longshore D, Morton SC, Orlando M, Shapiro M. The prevalence of alcohol consumption and heavy drinking among people with HIV in the United States: results from the HIV Cost and Services Utilization Study. J. Stud. Alcohol. 2002;63:179–186. doi: 10.15288/jsa.2002.63.179. [DOI] [PubMed] [Google Scholar]
  20. Hasin DS, Aharonovich E, O'Leary A, Greenstein E, Pavlicova M, Arunajadai S, Waxman R, Wainberg M, Jelzer J, Jonston B. Reducing heavy drinking in HIV primary care: a randomized trial of brief intervention, with and without technological enhancement. Addiction. 2013;108:1230–1240. doi: 10.1111/add.12127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hawkins EJ, Kivlahan DR, Williams EC, Wright SM, Craig T, Bradley KA. Examining quality issues in alcohol misuse screening. Subst. Abuse. 2007;28:53–65. doi: 10.1300/J465v28n03_06. [DOI] [PubMed] [Google Scholar]
  22. Hendershot CS, Stoner SA, Pantalone DW, Simoni JM. Alcohol use and antiretroviral adherence: review and meta-analysis. J. Acquir. Immune Defic. Syndr. 2009;52:180–202. doi: 10.1097/QAI.0b013e3181b18b6e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Justice AC, Lasky E, McGinnis KA, Skanderson M, Conigliaro J, Fultz SL, Crothers K, Rabeneck L, Rodriguez-Barradas M, Weissman SB, Bryant K. Medical disease and alcohol use among veterans with human immunodeficiency infection: a comparison of disease measurement strategies. Med. Care. 2006;44:S52–S60. doi: 10.1097/01.mlr.0000228003.08925.8c. [DOI] [PubMed] [Google Scholar]
  24. Justice AC, McGinnis KA, Tate JP, Braithwaite RS, Bryant KJ, Cook RL, Edelman JE, Fiellin LE, Freiberg MS, Gordon AJ, Kraemer KL, Marshall BDL, Williams EC, Fiellin DA. Risk of mortality and physiologic injury evident with lower alcohol exposure among HIV infected compaered to uninfected men. Drug Alcohol Depend. 2016;161:95–103. doi: 10.1016/j.drugalcdep.2016.01.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kahler CW, Liu TC, Bryant PA, Pinkston MM, Kojic EM, Onen N, Baker JV, Hammer J, Brooks JT, Patel P. Direct and indirect effects of heavy alcohol use on clinical outcomes in a longitudinal study of HIV patients on ART. AIDS Behav. 2016 doi: 10.1007/s10461-016-1474-y. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kalichman SC, Kalichman MO, Cherry C, Hoyt G, Washington C, Grebler T, Welles B, Merely C. Intentional medication nonadherence because of interactive toxicity beliefs among HIV-positive active drug users. J. Acquir. Immune Defic. Syndr. 2015;70:503–509. doi: 10.1097/QAI.0000000000000776. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Korthuis PT, Fiellin DA, McGinnis KA, Skanderson M, Justice AC, Gordon AJ, Doebler DA, Asch SM, Fiellin LE, Bryant K, Gibert CL, Crystal S, Goetz MB, Rimland D, Rodriguez-Barradas MC, Kraemer KL. Unhealthy alcohol and illicit drug use are associated with decreased quality of HIV care. J. Acquir. Immune. Defic. Syndr. 2012;61:171–178. doi: 10.1097/QAI.0b013e31826741aa. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Landis JR, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
  29. Lapham GT, Rubinsky AD, Heagerty PJ, Williams EC, Hawkins EJ, Maynard C, Kivlahan DR, Bradley KA. Annual rescreening for alcohol misuse: diminishing returns for some patient subgroups. Med. Care. 2013;51:914–921. doi: 10.1097/MLR.0b013e3182a3e549. [DOI] [PubMed] [Google Scholar]
  30. Lim JK, Tate JP, Fultz SL, Goulet JL, Conigliaro J, Bryant KJ, Gordon AJ, Gibert C, Rimland D, Goetz MB, Klein MB, Fiellin DA, Justice AC, Lo RV., III Relationship between alcohol use categories and noninvasive markers of advanced hepatic fibrosis in HIV-infected, chronic hepatitis C virus-infected, and uninfected patients. Clin. Infect. Dis. 2014;58:1449–1458. doi: 10.1093/cid/ciu097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. McCance-Katz EF, Lum PJ, Beatty G, Gruber VA, Peters M, Rainey PM. Untreated HIV infection is associated with higher blood alcohol levels. J. Acquir. Immune Defic. Syndr. 2012;60:282–288. doi: 10.1097/QAI.0b013e318256625f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. McGinnis KA, Brandt CA, Skanderson M, Justice AC, Shahrir S, Butt AA, Brown ST, Freiberg MS, Gibert CL, Goetz MB, Kim JW, Pisani MA, Rimland D, Rodriguez-Barradas MC, Sico JJ, Tindle HA, Crothers K. Validating smoking data from the Veteran's Affairs Health Factors dataset, an electronic data source. Nicotine Tob. Res. 2011;13:1233–1239. doi: 10.1093/ntr/ntr206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. McGinnis KA, Justice AC, Kraemer KL, Saitz R, Bryant KJ, Fiellin AD. Comparing alcohol screening measures among HIV-infected and -uninfected men. Alcohol. Clin. Exper. Res. 2013;37:435–442. doi: 10.1111/j.1530-0277.2012.01937.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McGinnis KA, Fiellin DA, Tate JP, Cook RL, Braithwaite RS, Bryant KJ, Edelman EJ, Gordon AJ, Kraemer KL, Maisto S, Justice AC. Number of drinks to "feel a buzz" varies by HIV status and viral load in men. AIDS Behav. 2016;20:504–511. doi: 10.1007/s10461-015-1053-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. McQueen L, Mittman BS, Demakis JG. Overview of the Veterans Health Administration (VHA) Quality Enhancement Research Initiative (QUERI) J. Am. Med. Inform. Assoc. 2004;11:339–343. doi: 10.1197/jamia.M1499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Parsons JT, Golub SA, Rosof E, Holder C. Motivational interviewing and cognitive-behavioral intervention to improve HIV medication adherence among hazardous drinkers: a randomized controlled trial. J. Acquir. Immune Defic Synd. 2007a;46:443–450. doi: 10.1097/qai.0b013e318158a461. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Petry NM. Alcohol use in HIV patients: what we don’t know may hurt us. Int. J. STDS AIDS. 1999;10:561–570. doi: 10.1258/0956462991914654. [DOI] [PubMed] [Google Scholar]
  38. Saitz R. Clinical practice. Unhealthy alcohol use. N. Engl. J. Med. 2005;352:596–607. doi: 10.1056/NEJMcp042262. [DOI] [PubMed] [Google Scholar]
  39. Samet JH, Horton NJ, Meli S, Freedberg KA, Palepu A. Alcohol consumption and antiretroviral adherence among HIV-infected persons with alcohol problems. Alcohol. Clin. Exp. Res. 2004;28:572–577. doi: 10.1097/01.alc.0000122103.74491.78. [DOI] [PubMed] [Google Scholar]
  40. Samet JH, Horton NJ, Traphagen ET, Lyon SM, Freedberg KA. Alcohol consumption and HIV disease progression: are they related? Alcohol. Clin. Exp. Res. 2003;27:862–867. doi: 10.1097/01.ALC.0000065438.80967.56. [DOI] [PubMed] [Google Scholar]
  41. Samet JH, Walley AY. Interventions targeting HIV-infected risky drinkers: drops in the bottle. Alcohol Res. Health. 2010;33:267–279. [PMC free article] [PubMed] [Google Scholar]
  42. Scott-Sheldon LA, Carey KB, Cunningham K, Johnson BT, Carey MP, MASH Research Team Alcohol use predicts sexual decision-making: a systematic review and meta-analysis of the experimental literature. AIDS Behav. 2016;20(Suppl 1):19–39. doi: 10.1007/s10461-015-1108-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Scott-Sheldon LA, Walstrom P, Carey KB, Johnson BT, Carey MP, MASH Research Team Alcohol use and sexual risk behaviors among individuals infected with HIV: a systematic review and meta-analysis 2012 to early 2013. Curr. HIV AIDS Rep. 2013;10:314–323. doi: 10.1007/s11904-013-0177-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Shuper PA, Neuman M, Kanteres F, Baliunas D, Joharchi N, Rehm J. Causal considerations on alcohol and HIV/AIDS--a systematic review. Alcohol Alcohol. 2010;45:159–166. doi: 10.1093/alcalc/agp091. [DOI] [PubMed] [Google Scholar]
  45. Stall R, McKusick L, Wiley J, Coates TJ, Ostrow DG. Alcohol and drug use during sexual activity and compliance with safe sex guidelines for AIDS: The AIDS Behavioral Research Project. Health Educ. Behav. 1986;13:359–371. doi: 10.1177/109019818601300407. [DOI] [PubMed] [Google Scholar]
  46. Williams EC, Achtmeyer CE, Thomas RM, Grossbard JR, Lapham GT, Chavez LJ, Ludman EJ, Berger D, Bradley KA. Factors underlying quality problems with alcohol screening prompted by a clinical reminder in primary care: a multi-site qualitative study. J. Gen. Intern. Med. 2015;30:1125–1132. doi: 10.1007/s11606-015-3248-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Williams EC, Rubinsky AD, Chavez LJ, Lapham GT, Rittmueller SE, Achtmeyer CE, Bradley KA. An early evaluation of implementation of brief intervention for unhealthy alcohol use in the US Veterans Health Administration. Addiction. 2014;109:1472–1481. doi: 10.1111/add.12600. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Wong MD, Cunningham WE, Shapiro MF, Andersen RM, Cleary PD, Duan N, Liu HH, Wilson IB, Landon BE, Wenger NS. Disparities in HIV treatment and physician attitudes about delaying protease inhibitors for nonadherent patients. J. Gen. Intern. Med. 2004;19:366–374. doi: 10.1111/j.1525-1497.2004.30429.x. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES