Clinician Suspicion of an Alcohol Problem: An Observational Study From the AAFP National Research Network

Abstract

PURPOSE

In clinical practice, detection of alcohol problems often relies on clinician suspicion instead of using a screening instrument. We assessed the sensitivity, specificity, and predictive values of clinician suspicion compared with screening-detected alcohol problems in patients.

METHODS

We undertook a cross-sectional study of 94 primary care clinicians’ office visits. Brief questionnaires were completed separately after a visit by both clinicians and eligible patients. The patient’s anonymous exit questionnaire screened for hazardous drinking based on the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) and for harmful drinking (alcohol abuse or dependence) based on 2 questions from the Diagnostic and Statistical Manual of Mental Disorders. After the visit, clinicians responded to the question, “Does this patient have problems with alcohol?” with answer options including “yes, hazardous drinking” and “yes, alcohol abuse or dependence.” Analyses assessed the associations between patients’ responses to screening questions and clinician’s suspicions.

RESULTS

Of 2,518 patients with an office visit, 2,173 were eligible, and 1,699 (78%) completed the exit questionnaire. One hundred seventy-one (10.1%) patients had a positive screening test for hazardous drinking (an AUDIT-C score of 5 or greater) and 64 (3.8%) for harmful drinking. Clinicians suspected alcohol problems in 81 patients (hazardous drinking in 37, harmful drinking in 40, and both in 4). The sensitivity of clinician suspicion of either hazardous or harmful drinking was 27% and the specificity was 98%. Positive and negative predictive values were 62% and 92%, respectively.

CONCLUSION

Clinician suspicion of alcohol problems had poor sensitivity but high specificity for identifying patients who had a positive screening test for alcohol problems. These data support the routine use of a screening tool to supplement clinicians’ suspicions, which already provide reasonable positive predictive value.

INTRODUCTION

Of adults in the United States, 15% have had at least 1 episode of binge drinking in the previous 30 days¹ and 8.4% meet criteria for an alcohol use disorder in the past year.² When alcohol problems are detected and addressed by clinicians, patients can be helped to reduce consumption. For example, brief interventions by primary care clinicians in efficacy trials resulted in 40% of hazardous drinkers reducing their alcohol consumption to safe levels compared with 20% of the control group.^3–5

Alcohol problems are not commonly identified during the course of routine health care services.^6–13 Several studies suggest, however, that when clinicians do ask about alcohol consumption, they are more likely to do so for patients who do have hazardous drinking or alcohol use disorders.^{7,8,11,14–16} In a nationally representative survey, 5.3% of respondents were found to have a positive Alcohol Use Disorders Identification Test¹⁷ (AUDIT score of 8 or greater), and this group was 1.65 times more likely to report that a clinician asked them about alcohol consumption in the past year than were those with a negative AUDIT screen.⁷ In an observational Veterans Affairs study, higher AUDIT scores were strongly associated with receipt of advice by a clinician about alcohol use.¹⁴ Yet screening rates are still low. In a national sample of persons aged 18 to 39 years, 49% of those who had seen a doctor in the past year recalled having been asked about their alcohol consumption.¹³ Of those who exceeded per-day or per-week safe limits, only 14% recalled being advised about safe-drinking limits, and 7% were advised to cut down. These proportions were somewhat higher among those who exceeded both per-day and per-week limits; 24% were advised about safe limits and 21% to cut down.¹³

Screening for alcohol consumption has not yet been integrated into routine primary care¹³ even though alcohol problems are prevalent,^18,19 expensive,^20,21 and major causes of morbidity and mortality,²² and screening is recommended (grade B) by the US Preventive Services Task Force.^23,24 Screening can be accomplished by asking only 1 or a few questions,^25–29 and subsequent brief interventions for problem drinking by primary care clinicians can reduce hospital days and health care costs³⁰ and mortality.³¹ In the absence of systematic screening, detection of alcohol problems relies on clinicians’ suspicion. Our study assessed the sensitivity, specificity, positive predictive value, and negative predictive value of clinician suspicion of alcohol problems and compared them with those for validated, written, brief screening instruments.

METHODS

The study was conducted from October 2007 through September 2008 in practices recruited in the American Academy of Family Physicians (AAFP) National Research Network and affiliated networks, including Kentucky Ambulatory Network, Research Involving Outpatient Settings in New Mexico, Missouri’s Show-Me Research Network, and State Networks of Colorado Ambulatory Practices and Partners. Data from patients and their clinicians were originally collected to examine the association of alcohol use and sleep problems, and those findings have been reported previously.³² Clinicians who agreed to participate were asked to enroll 30 adult patients from consecutive visits. E-mails and letters were sent to 530 clinicians; 113 responded that they were interested, and 94 returned data. Recruitment letters were sent to 14 regional practice-based research networks and 4 participated, as well as to 69 residency programs, of which 5 participated.

The study was approved by the AAFP Institutional Review Board (IRB) and 11 local IRBs. For practices without a local IRB, clinicians participated as unaffiliated investigators under AAFP’s Federalwide Assurance. Patients were provided an informational sheet about the study at the start of their office visit and were not required to sign an informed consent form. The 1-page patient information sheet identified the study and its purpose (identification of patients’ sleep problems, alcohol consumption, and chronic diseases in primary care settings), assured participants of the confidentiality of the study, and invited patients to participate (“You are invited to participate in a study that examines lifestyles that can affect sleep in patients.”). Patients were eligible if they were aged 18 years or older, cognitively intact, not in severe distress, able to read English, and had not already participated in the study.

If an eligible patient agreed to participate, both patient and clinician completed separate questionnaires immediately after the visit. Patients were instructed to seal their anonymous, 23-item, self-administered questionnaire in an envelope and leave it at the front desk when checking out. Clinician and patient questionnaires were linked by a study identification number unique to that encounter.

The patients’ questionnaire included 5 alcohol questions. Three were Alcohol Use Disorders Identification Test-Consumption (AUDIT-C)^17,28,33 questions: “In the past 12 months, how often have you had a drink containing alcohol?” “... how many drinks containing alcohol have you had on a typical day when you are drinking?” and “... how often have you had 6 or more drinks on one occasion?” Each was scored 0 to 4 points and summed, with possible scores ranging from 0 to 12, where 0 reflects no alcohol use. In a stratified random sample of the US population, Dawson and Grant found that for past-year drinkers (with both sexes combined), a cut point of 5 or greater had a sensitivity of 82.6% and a specificity of 81.3% in detecting either hazardous or harmful drinking,³³ the threshold currently used in Veterans Affairs health care to prompt intervention.³⁴

The other 2 alcohol questions were derived from the Diagnostic and Statistical Manual of Mental Disorders-4th Edition (DSM-IV)³⁵ Diagnostic Interview Schedule³⁶: “In the past 12 months, how often have you had a lot more to drink than you intended to have?” and “In the past 12 months, how often have you been under the influence of alcohol in situations where you could have caused an accident or gotten hurt?” For these 2 questions, a report of monthly or more often on both items was considered a positive screen for harmful drinking in patients who also had a positive screening result on the AUDIT-C, consistent with the 2-question instrument’s original validation.³⁷ Responses to these questions have been shown to be moderately sensitive and specific in identifying alcohol use disorders^37,38 based on a structured interview and diagnostic criteria in the DSM-IV.³⁵

For our analyses, patients were classified into 4 drinking status categories (Table 1). A nondrinker responded negatively to all questions. A moderate drinker acknowledged alcohol use but was negative on the AUDIT-C. A hazardous drinker had positive screening results on the AUDIT-C but negative on the DSM-IV–based questions. A harmful drinker had positive screening results on both the AUDIT-C and the 2 DSM-IV–based questions. In this report, we use the term alcohol problem to refer to the patient whose screening test was positive for hazardous drinking, harmful drinking, or both. The scoring key for the AUDIT-C can be found in the Supplemental Appendix at http://www.annfammed.org/content/11/1/53/suppl/DC1.

Table 1.

Alcohol Use Categories Based on Patient Self-Report

Category	Standard Definitiona	Study Definitionb
No alcohol consumption		Denies alcohol use on AUDIT-C and DSM-IV–based questions
Moderate drinking	On 1 occasion drinking <4 drinks for a woman or <5 drinks for a man, and in a week drinking <8 drinks for a woman or <15 drinks for a man	Negative on both the AUDIT-C and the 2-question DSM-IV– based screen
Hazardousdrinkingc	On 1 occasion drinking >3 drinks for a woman or >4 drinks for a man, or in a week drinking >7 drinks for a woman or >14 drinks for a man	Positive AUDIT-C but negative on the 2-question DSM-IV–based screen
Harmfuldrinkingc	Alcohol use disorders, ie, alcohol abuse or alcohol dependence defined using DSM-IV criteria	Positive on the 2-question DSM-IV–based screen, with or without a positive AUDIT-C

AUDIT-C = Alcohol Use Disorders Identification Test-Consumption; DSM-IV = Diagnostic and Statistical Manual of Mental Disorders-IV.

^aFrom the National Institute on Alcohol Abuse and Alcoholism.^39,40

^bBased on patients’ responses to our screening instruments. See the Supplemental Appendix for the AUDIT-C scoring key.

^cAlcohol problems defined as either hazardous or harmful drinking as identified on screening tests noted above.

Clinicians were asked, “Does this patient have problems with alcohol (check each that applies)?” Answer options were “no,” “yes, hazardous drinking,” “yes, alcohol abuse or dependence,” “don’t know,” and “patient is a recovering alcoholic and/or ex-drinker.” When the clinician checked either affirmative answer, we considered it to signify a suspicion of an alcohol problem.

We first conducted bivariate analyses of drinking status with each demographic factor we measured. Analyses of the sensitivity and specificity of clinician suspicion controlled for patient sex, age, educational attainment, race/ethnicity, and clustering by clinician. Age was examined as a continuous variable. Educational attainment and race/ethnicity were specified by a series of dummy variables with low educational attainment (less than high school) and white race as the referent categories. The thresholds for a positive AUDIT-C that we used have reasonable sensitivity³³ but are still screening questions and do not constitute a reference standard for identifying alcohol problems. We therefore repeated key analyses using higher thresholds on the AUDIT-C. We report sensitivity, specificity, and predictive values, and area under the receiver-operating-characteristic curves. For all analyses, we used Stata SE 10.1 (StataCorp, LP).

RESULTS

We recruited 94 clinicians from 40 primary care practices for the study. Figure 1 shows derivation of the patient sample. The mean age of study patients was 50 years and 67% were women. Ten percent reported less than a high school education, whereas 58% reported at least some college education. Based on responses to the AUDIT-C and 2-item DSM-IV–based screening questions, the proportions in 4 alcohol consumption categories were 40.4% nondrinkers, 49.6% moderate drinkers, 6.3% hazardous drinkers, and 3.8% harmful drinkers. The prevalence of alcohol problems, meaning either hazardous or harmful drinking, was higher in men, younger patients, and those with greater educational attainment. Blacks were marginally less likely to report alcohol problems than whites (P = .1).

Figure 1.

Flowchart of patient enrollment.

Forty-two clinicians reported discussing alcohol with 92 patients either during the study visit or a previous visit. This discussion involved 69 (85%) of the 81 patients suspected of having an alcohol problem.

Clinicians suspected either hazardous or harmful drinking in 81 patients (4.9%). Because the 2 levels of clinician suspicion did not differentiate between harmful and hazardous drinking, based on patient report, our analyses report the psychometric properties of clinician suspicion for either type of alcohol problem.

For detection of screening-identified alcohol problems, clinician suspicion had a sensitivity of 27.0%, specificity of 97.9%, positive predictive value of 61.7%, and negative predictive value of 91.5%. The area under the receiver-operating-characteristic curve—controlling for patient age, sex, educational attainment, race, and clustering by clinician—was 0.785, and the statistic was 0.331. Although the sensitivity of clinician suspicion was low, the specificity was high, with few false positives. Logistic models adjusting for patient characteristics showed that clinician suspicion of alcohol problems was associated with an odds ratio of 16.3 (95% CI, 9.3–28.4) of detecting an alcohol problem identified by patients’ screening responses.

Using the AUDIT-C thresholds recommended by Bradley and her colleagues (an AUDIT-C score of 3 or higher in women, 4 or higher in men),²⁸ the sensitivity of clinician suspicion was 16.1% and specificity 98%. Examining only the third AUDIT-C question, which asks about the frequency of drinking “6 or more drinks on 1 occasion,” with patient responses that they drank that much “weekly” or “daily or almost daily,” the sensitivity was 38.7%. Sensitivity of clinician suspicion was greater with higher AUDIT-C thresholds (Table 2).

Table 2.

Sensitivity of Clinician Suspicion Using Different Criteria to Identify Which Patients Have an Alcohol Problem

Criteria and Threshold for Identifying Patients With Alcohol Problem	Sensitivity %
Total AUDIT-C score: a positive screening test defined as a score at the given level or above
≥3 (women), ≥4 (men)	16.1
6
7
8
9
10	61.1
Changing a positive screening test to negative if drinking on a “typical day” was ≤2 drinks and if the frequency of episodic heavy drinking was less often than monthly	21.6
Changing a positive screen to negative if drinking on a “typical day” was ≤2 drinks regardless of the frequency of episodic heavy drinking	21.7
Using only the third AUDIT-C question on episodic heavy drinking
Monthly	27.9
Weekly	38.7
Daily or almost daily	73.3

AUDIT-C = Alcohol Use Disorders Identification Test-Consumption.

In post-hoc analyses, clinician suspicion of alcohol abuse or dependence was more strongly associated with a positive screening test for either level of alcohol problem than was clinician suspicion of hazardous drinking. In logistic regression models controlling for patient age, sex, education, and race, as well as for clustering, the odds ratio associated with clinician suspicion of hazardous drinking was 8.7 in detecting hazardous drinking and 9.4 in detecting harmful drinking. Those odds ratios changed only slightly (to 8.4 and 9.2, respectively) when both levels of suspicion were entered together. The odds ratio associated with suspicion of harmful drinking was 19.3 in detecting hazardous drinking and 21.8 in detecting harmful drinking, changing to 18.9 and 21.5, respectively, when the 2 levels of suspicion were entered together. These results suggest that if the clinician believes the patient has a serious alcohol problem, the probability that he or she has some level of problem is higher.

DISCUSSION

In 40 primary care practices with 94 clinicians participating, we found that clinician suspicion of alcohol problems missed most patients with a potential problem as identified by confidential patient responses to brief screening instruments. Conversely, when the clinician suspected an alcohol problem, most of these patients independently acknowledged in responses to screening questions that they were drinking at hazardous or harmful levels. When the clinician suspected a patient of having an alcohol problem, they reported discussing alcohol consumption with 85% of these individuals during that visit or a previous one.

Our study is unique in being conducted in 40 geographically dispersed primary care practices and in a large sample of 1,600 patients. Most studies on this topic have smaller patient samples from a single setting. Several studies have also reported a low sensitivity of clinician suspicion.^41,42 A German primary care study reported a sensitivity of 15% based on chart review in detecting alcohol abuse or dependence, but when asked directly in interviews, clinicians correctly identified 50% of patients with a positive CAGE screening test or short Michigan Alcoholism Screening Test (MAST) and 74% of patients with diagnosis based on a structured diagnostic interview.⁴³

Our analysis also extends previous research by examining not only clinicians’ ability to detect more severe harmful drinking but also their ability to identify less severe hazardous drinking. Clinicians suspected only 27% of the patients with either alcohol problem. By comparison, in a national probability sample of 43,093 people, the sensitivity and specificity of the AUDIT-C screen were, respectively, 83.7% and 83.1% in detecting an alcohol use disorder and 92.6% and 92.0% in detecting hazardous drinking.³³

Integrating alcohol screening into routine primary care remains a major challenge.^13,44–47 To increase the feasibility of screening for alcohol problems in practice, a validated single screening question can be used. For example, for the question, “When was the last time you had more than X drinks in one day?” where X is 4 for women and 5 for men, an answer of any time in the past 3 months was 86% sensitive and 86% specific in detecting alcohol problems compared with a structured, researcher-administered interview.^25,26 A slightly different single-question screen has also been validated²⁷: “How many times in the past year have you had X or more drinks in a day?” where X is 5 for men and 4 for women, and a response of 1 or more is considered positive. Patients may be put off by any questions about alcohol use, however.⁴⁸ Indeed, after a randomized trial of a brief intervention in Denmark for hazardous drinking in primary care, the researchers interviewed the participating clinicians and found that they had stopped screening because of lack of time and the perception that the patients thought it distracted from addressing their reason(s) for the visit.⁴⁹ These challenges will need to be addressed when integrating screening into clinical practice.

Our study found that the specificity of clinical suspicion is reassuringly high. After adjustment for patient characteristics, we observed a 16-fold increase in the adjusted odds of the patient having an alcohol problem compared with no suspicion. The positive predictive value of clinician suspicion was 62%, which indicates that the clinician should discuss alcohol use with the patient without a great concern about raising a topic that is not relevant to the patient’s health.

In regard to study limitations, we could not evaluate the basis of the clinician’s suspicion of an alcohol problem, and our construct likely encompasses a spectrum that ranges from speculation to certitude. We have complete data from 1,664 patients, only 77% of whom were eligible for our study, and we do not know the effect that the missing patients would have had on our findings. Our use of an AUDIT-C threshold of 5 or greater increases its specificity while lowering its sensitivity. In a stratified random sample of the US population, Dawson et al found that for past-year drinkers (both sexes combined), a cut point of 5 or greater had a sensitivity of 82.6% and a specificity of 81.3% in detecting either hazardous or harmful drinking.³³ This threshold is also currently used by the Veterans Affairs to prompt an intervention.³⁴ Others have recommended using a lower threshold.^28,29,50,51 Although these 5 questions have reasonable sensitivity and specificity,^28,36,37 they are inferior to a criterion-standard diagnostic interview. To determine whether the low sensitivity of clinician suspicion in this study is due using a lower AUDIT-C threshold that may allow for more false positives, we examined the sensitivity of clinician suspicion of persons with very heavy alcohol use based on the third AUDIT-C question and found that the sensitivity of clinician suspicion only increased to 39%.

Training materials are now available that can help clinicians learn what to say when counseling about alcohol problems^39,40 and how to say it⁵² (see, for example, the free training modules at https://adept.missouri.edu). Our study affirms that systems need to be in place, possibly through team-based care, to screen systematically for alcohol problems with a validated question or series of questions and to address this health threat using evidence-based approaches. This process can improve patient health and perhaps save some of the $223 billion economic cost of alcohol in the United States (in 2006).²⁰

To read or post commentaries in response to this article, see it online at http://www.annfammed.org/content/11/1/53.

Funding support: This study was funded by a grant from the Robert Wood Johnson Foundation to the University of Pennsylvania, “Program of Research Integrating Substance Use Information into Mainstream Healthcare,” Barbara J. Turner, MD, MSED, MA, principal investigator.