Abstract
Background
Unhealthy alcohol use is a major worldwide health problem. Yet few studies have assessed provider adherence to the alcohol-related care recommended in clinical practice guidelines, nor links between adherence to recommended care and outcomes.
Objectives
To describe quality of care for unhealthy alcohol use and its impacts on drinking behavior
Research Design
Prospective observational cohort study of quality of alcohol care for the population of patients screening positive for unhealthy alcohol use in a large Veterans Affairs health system.
Participants
A total of 719 patients who screened positive for unhealthy alcohol use at one of 11 primary care practices and who completed baseline and 6-month telephone interviews.
Main Measures
Using administrative encounter and medical record data, we assessed three composite and 21 individual process-based measures of care delivered across primary and specialty care settings. We assessed self-reported daily alcohol use using telephone interviews at baseline and 6-month follow-up.
Key Results
The median proportion of patients who received recommended care across measures was 32.8% (range < 1% for initiating pharmacotherapy to 93% for depression screening). There was negligible change in drinking for the study population between baseline and 6 months. In covariate-adjusted analyses, no composites were significantly associated with changes in heavy drinking days or drinks per week, and just one of nine individual measures tested was significantly associated. In a subsample of patients drinking above recommended weekly limits prior to screening, two of nine individual measures were significantly associated.
Conclusions
This study shows wide variability in receipt of recommended care for unhealthy alcohol use. Receipt of recommended interventions for reducing drinking was frequently not associated with decreased drinking. Results suggest deficits in provision of comprehensive alcohol care and in understanding how to improve population-based drinking outcomes.
KEY WORDS: alcoholism and addictive behavior, primary care, quality assessment, substance abuse, veterans
INTRODUCTION
Unhealthy alcohol use, ranging from risky drinking to alcohol use disorder (AUD),1 is highly prevalent2 and associated with consequences at both individual and societal levels.3,4 Most at-risk individuals, however, receive no clinical care for their unhealthy drinking.2,5,6 Primary care settings hold promise for engaging a larger proportion of patients with unhealthy drinking in treatment. This study aims to test the link between delivery of guideline-based alcohol care among routine primary care patients and reduced alcohol consumption.
Quality of care for unhealthy alcohol use can be assessed using quality measures that describe the extent to which care is consistent with clinical practice guideline recommendations. These guidelines combine scientific evidence and expert opinion to guide providers in delivering appropriate care for specific conditions. Current guidelines on substance use disorders include recommended care for the full spectrum of unhealthy alcohol use.7,8 Most alcohol-related quality measures, in contrast, focus on patients with an AUD diagnosis and on care delivered in specialty care settings9,10 and are not applicable to patients with undiagnosed or milder, but potentially harmful, alcohol use.11 This study uses a set of measures that assess alcohol-related care in primary care or specialty care settings and applies the measures across the full spectrum of primary care patients screening positive for unhealthy alcohol in 11 Veterans Health Administration (VHA) primary care practices.
Historically, quality measures have often been evaluated based on face validity relative to scientific evidence and expert opinion and the feasibility of measure implementation. With increased use of quality measures,12 validating process-based quality measures in terms of their ability to predict meaningful improvements in patient outcomes has become increasingly important.13 Prior studies have evaluated the predictive validity of some alcohol-related quality measures and have demonstrated a relationship between better care and improved clinical outcomes,14 increased employment,15 decreased arrests,16 and decreased mortality.15,17 However, we know of no prior studies that focused on assessing whether a comprehensive set of alcohol quality of care measures predicted subsequent changes in alcohol use when applied to a broad primary care population.
The objectives of this study were to describe quality of care for unhealthy alcohol use among patients visiting primary care settings and to determine whether better quality of alcohol care was linked to reduced alcohol consumption 6 months later.
METHODS
Research Design and Study Cohort
Participants were recruited following a positive screen for unhealthy alcohol use during routine screening at one of 11 primary care practices in the Greater Los Angeles VA Healthcare System (GLAVA) from February 2013 to January 2014, and were followed prospectively for 6 months. The clinical reminder system within the electronic medical record prompts providers to perform screening using the Alcohol Use Disorders Identification Test – Consumption (AUDIT-C).18 A positive screen was defined as a score of 5 or higher, the threshold used by VHA to indicate follow-up interventions.19 Additional study eligibility criteria included no AUD outpatient or inpatient encounter in the prior 90 days, at least 18 years of age, ability to speak and understand English, have a valid telephone, no cognitive impairment, and engaged in care (at least one outpatient visit within GLAVA in the prior 12 months). The RAND Human Subjects Protection Committee and the GLAVA Institutional Review Board approved all procedures.
Data Collection
Eligible patients received a letter followed by a telephone invitation. Interviewers explained study participation and obtained verbal informed consent. Baseline telephone interviews were completed an average of 15.3 days after the positive screen visit (range six to 87). Follow-up telephone interviews were conducted 6 months after the positive screen visit (mean 187.2 days, range 174 to 299). Participants received incentives for baseline ($50) and follow-up ($60).
Measures
Patient Characteristics
Gender and age were obtained from administrative records. Marital status, education, employment, race/ethnicity, income, receiving all or most care at VHA, depression symptoms,20 physical and mental health functioning,21 and drug use (illegal and prescription misuse) were collected from the baseline survey.
Process Measures
Candidate process measures were based on a review of existing measures and clinical practice guidelines and were selected and refined using a modified RAND/UCLA expert consensus panel approach22 and then operationalized, yielding a total of 21 measures.23 We applied the measures using administrative data (i.e., outpatient visits, pharmacy, laboratory) and medical record review to assess outpatient care across both primary and specialty settings in the 6 months following identification of unhealthy alcohol use (Table 1). Six measures required only administrative data, while 15 measures integrated both administrative and medical record review data. Two registered nurses used a structured abstraction tool to collect data elements from medical record text notes. Abstractors achieved high interrater reliability (percent agreement 99.1%, kappa 98.0%).
Table 1.
Performance on Quality Measures for Unhealthy Alcohol Use
| Measure name (number*) | Numerator | Denominator | Eligible patients | Pass rate (%) |
|---|---|---|---|---|
| Assess for Alcohol Use Disorders (1) | Patients assessed for AUD within 30 days after screen-positive visit | Patients with unhealthy alcohol use | 719 | 25.5 |
| Screen for Depression (2) | Patients screened for depression within 30 days before or after identification of high-risk alcohol use | Patients with high-risk alcohol use | 302 | 93.0 |
| Assess for Depression (3) | Patients assessed for depression within 30 days before or 60 days after identification of high-risk alcohol use or receipt of a behavioral health visit | Patients with high-risk alcohol use who received at least one behavioral health specialty visit | 59 | 61.0 |
| Evaluate for Suicide Risk (4) | Patients evaluated for suicide risk within 30 days before or after identification of high-risk alcohol use | Patients with high-risk alcohol use | 302 | 42.4 |
| Screen for Other Substance Use (5) | Patients screened for other substance use within 30 days before or after the screen-positive visit | Patients with unhealthy alcohol use | 719 | 38.4 |
| Assess for Other Substance Use (6) | Patients received an assessment (type, frequency, and recency) of other substance use within 30 days before or after the screen-positive visit or behavioral health visit | Patients with high-risk alcohol use and a behavioral health specialty visit 30 days before or after the screen-positive visit | 43 | 34.9 |
| Test Liver Function (7) | Patients screened for liver disease within 90 days before or after identification of high-risk alcohol use | Patients with high-risk alcohol use | 302 | 80.5 |
| Conduct Brief Intervention (8) | Patients received a brief intervention within 7 days before to 30 days after the screen-positive visit or SUD specialty care within 30 days before or after the screen-positive visit | Patients with unhealthy alcohol use | 719 | 83.7 |
| Discuss Treatment Options (9) | Patients received counseling on multiple alcohol-related treatment options within 30 days after identification of high-risk alcohol use | Patients with high-risk alcohol use | 302 | 34.4 |
| Offer Psychosocial Intervention (10) | Patients offered a psychosocial intervention for alcohol use within 30 days after identification of high-risk alcohol use | Patients with high-risk alcohol use | 302 | 32.8 |
| Offer Pharmacotherapy (11) | Patients offered pharmacotherapy for alcohol dependence within 30 days after AUD diagnosis | Patients diagnosed with AUD within 90 days after the screen-positive visit | 195 | 3.1 |
| Psychosocial Intervention Dose (12) | Patients received four or more psychosocial intervention visits within 90 days of the first psychosocial visit | Patients with high-risk alcohol use and at least one psychosocial intervention visit within 90 days after the screen-positive visit | 7 | 28.6 |
| Psychosocial Intervention Quality (13) | Patients received at least one psychosocial visit that includes elements of an evidenced-based psychosocial intervention | Patients with high-risk alcohol use and at least one psychosocial intervention visit | 15 | 33.3 |
| Receipt of Pharmacotherapy for Alcohol Dependence (14) | Patients received pharmacotherapy for alcohol dependence within 90 days of AUD diagnosis | Patients diagnosed with AUD within 90 days after the screen-positive visit | 195 | 0.5 |
| Refer to Recovery Support for the Community (15) | Patients referred to recovery support within 183 days after the screen-positive visit | Patients diagnosed with AUD within 30 days after the screen-positive visit | 183 | 18.0 |
| Treatment for Co-occurring Mental Health and Substance Use (18) | Patients received at least two visits where both MH and AUD were addressed in each visit | Patients with co-occurring AUD and MH diagnoses | 49 | 36.7 |
| Receipt of Integrated Co-occurring Disorder Treatment (19) | Patients with either: (a) an integrated treatment plan including both conditions or (b) two visits where both conditions were addressed in the same visit | Patients with co-occurring AUD and MH diagnoses | 49 | 32.7 |
| Treatment Initiation (21) | Patients with an SUD encounter (inpatient, outpatient) within 14 days after the qualifying SUD visit | Patients with a qualifying visit with an SUD diagnosis who have not had an SUD diagnosis or any SUD-related services in the 60 days prior to the date of diagnosis | 201 | 5.0 |
| Treatment Engagement | Patients with treatment initiation and at least two SUD encounters within 30 days after treatment initiation | Patients with a qualifying visit with an SUD diagnosis who have not had an SUD diagnosis or any SUD-related services in the 60 days prior to the date of diagnosis | 201 | 1.5 |
| Reassess Alcohol Use (24) | Patients received an alcohol use reassessment (quantity, frequency) between 30 and 90 days after screen the positive screen visit | Patients with unhealthy alcohol use | 719 | 22.9 |
| Repeat Brief Intervention (25) | Patients received at least two brief interventions within 7 days before to 60 days after the screen-positive visit or two SUD specialty care visits within 30 days before to 60 days after the screen-positive visit | Patients with unhealthy alcohol use | 719 | 16.4 |
*Measure numbers correspond to the detailed technical specifications documented elsewhere.23AUD, alcohol use disorder; unhealthy alcohol use, AUDIT-C greater than or equal to 5; high-risk alcohol use, AUDIT-C greater than 8 or an AUD diagnosis; SUD, substance use disorder; MH, mental health
We defined a formative composite24 to summarize documented care processes identified by the expert panel that each patient received for unhealthy alcohol use. We created an overall percentage composite measure, defined as the proportion of individual process-based quality measures (described below) for which each patient received the recommended care relative to all for which the patient was eligible. Each eligible measure is weighted equally for each patient. Composite measures have several advantages,24,25 including the potential to increase reliability and maximize the number of patients included in the analyses relative to individual measure analyses that are limited to patients in the denominator of an individual measure. For inclusion in our composite measures, individual measures needed to have a denominator of at least 20 eligible patients, a numerator of at least four, and a pass rate no smaller than 2% and no larger than 98%. Using these 17 qualifying measures, we developed two sub-composites: (1) screening including screening and assessment for unhealthy alcohol use (eight measures: 1–7, 24; see Tables 1 and 2) and (2) treatment including brief intervention, initial discussion of treatment options, initiation of treatment, and receipt of treatment for unhealthy alcohol use (nine measures: 8–11, 15, 18, 19, 21, 25).
Table 2.
Demographic and Clinical Characteristics of Study Sample at Baseline (n = 719)
| Patient characteristic | N (weighted %) |
|---|---|
| Male | 697 (96.9%) |
| Married/living as married | 264 (36.5%) |
| Education | |
| Did not complete high school | 24 (3.5%) |
| High school graduate or GED | 185 (26.4%) |
| Some college | 370 (51.3%) |
| College graduate or beyond | 139 (18.9%) |
| Employment | |
| Employed | 278 (39.7%) |
| Unemployed | 96 (13.6%) |
| Out of labor force | 344 (46.7%) |
| Race/ethnicity | |
| Black | 92 (13.0%) |
| Hispanic | 175 (25.4%) |
| White, non-Hispanic | 383 (52.0%) |
| Other* | 69 (9.6%) |
| Income | |
| $15,000 or under | 214 (31.1%) |
| $15,001–$30,000 | 185 (26.8%) |
| $30,001–$60,000 | 209 (29.0%) |
| $60,001 or more | 91 (13.1%) |
| All or most medical care through VHA | 611 (86.1%) |
| Illegal drug/prescription drug misuse in the past 30 days | 65 (9.3%) |
| Weighted mean (SD) | |
| Age | 55.3 (16.0) |
| PHQ-9 | 5.2 (5.6) |
| Functioning | |
| Physical health | 46.6 (11.2) |
| Mental health | 51.7 (11.5) |
Ns are observed frequencies; means, SDs, and percentages are weighted for survey non-response. *Other race/ethnicity includes American Indian/Alaska Native, Asian, Native Hawaiian/other Pacific Islander, multiple race/ethnicities, and unknown
Alcohol Use Measures
We assessed past 30-day drinking using the Timeline Followback (TLFB) at both baseline and follow-up.26 We examined two drinking outcomes: percent of heavy drinking days in the past 30 days out of days available (i.e., not incarcerated or hospitalized) and mean drinks per week in the past 30 days. A heavy drinking day was defined as five or more drinks for men and four or more drinks for women. Drinking above recommended weekly limits was defined as mean drinks per week of more than 14 drinks per week for men and more than seven per week for women.27 These measures are distinct from the positive alcohol screen for study eligibility, which was an AUDIT-C score of 5 or higher during a routine screen.
The goal of the baseline interview was to capture alcohol use prior to the screen-positive visit, as interventions delivered immediately following the positive screen could affect alcohol use. Past 30-day alcohol use included some days after the screen-positive visit, so we used drinking data for all days prior to the screen-positive visit for each patient to characterize baseline drinking. We imputed missing daily alcohol consumption data, both at baseline and 6-month follow-up, using a hot-deck multiple imputation approach28 (see Appendix).
Response Rate and Analytic Sample
The adjusted response rate (AAPOR response rate 129) at baseline was 54% (raw rate: 51%; 973 completes of 1922 potentially eligible). The probability of baseline non-response was not significantly associated with any characteristics available for all eligible patients (age, gender, income, marital status, or service era; results not shown). Of 1032 who consented to participate, 46 were excluded for reporting alcohol-related encounters within the past 90 days and 13 declined further participation. The follow-up adjusted response rate was 82% (raw rate, 79%; 770 completes of 973 baseline completers). Removal of patients sampled in error (e.g., updated administrative data indicated non-eligibility) resulted in a sample of 913 at baseline and 719 at follow-up. Two patients reported no days available for drinking in the past 30 days at follow-up and were excluded from modeling analyses (n = 717).
Statistical Analyses
Patient Characteristics and Receipt of Recommended Care
We used descriptive statistics, weighted for survey non-response, to describe the characteristics of the study sample and the proportion of patients who received recommended care on each quality measure.
Composite Measure Analyses
To assess the association between composite measures and risk of heavy drinking, we fit quasi-Poisson models to account for overdispersion and estimated robust standard errors for regression coefficients.30,31 We regressed the number of heavy drinking days in the 30 days preceding the follow-up interview on the target composite measure. Models were run using the overall composite as the predictor of interest, as well as separate models containing both sub-composites (assessment; treatment). All models included age, marital status, education, race/ethnicity, income, mental and physical functioning, receiving all or most care in VHA, and the baseline value of alcohol use, derived from daily drinking data preceding the positive screening visit. Since 47.6% (n = 342) reported no heavy drinking days at baseline, we also fit these models including only a targeted subset of 375 patients who had at least one heavy drinking day prior to positive screening.
We fit linear regression models for the average number of drinks per week outcome. We transformed the outcome (taking the log of the average number of drinks per week + 1) so the model would be better aligned with linear regression assumptions. Under this model, the composite quality coefficient estimate can be expressed in terms of the expected percent change in drinks per week associated with receiving all versus none of the care recommended for a patient. We adjusted for the same covariates described above. We also fit these models on a targeted subsample of 357 patients who drank above recommended weekly limits in the period prior to positive screening (i.e., > 14 drinks per week for men; > seven per week for women).
Individual Process Measure Analyses
For both alcohol consumption outcomes, we similarly examined nine individual measures: Assess for Alcohol Use Disorders, Evaluate for Suicide Risk, Screen for Other Substance Use, Test Liver Function, Conduct Brief Intervention, Discuss Treatment Options, Offer Psychosocial Intervention, Reassess Alcohol Use, and Repeat Brief Intervention. These were the subset of measures for which the sample size was sufficient to detect a medium-sized association (Cohen’s d = 0.5 standard deviations32 (SD)) between the individual measure and the logarithm of average number of drinks per week. We fit models using these measures in place of the composites for both full sample and targeted subsample analyses as described above.
RESULTS
Sample Characteristics
Patient demographics and clinical characteristics are provided in Table 2. Almost all patients were male (97%), and nearly half reported a race/ethnicity other than White (48%). On average, participants were 55 years old. Nearly two fifths of the sample was married (36%), two fifths were employed (40%), and one third earned $15,000 or less a year (31%).
Alcohol use at baseline and 6-months is described in Table 3. Despite all patients screening positive for unhealthy alcohol use in the routine clinical setting, only 83% of patients had an AUDIT-C of 5 or higher during the baseline interview. One-quarter had been diagnosed with an AUD in the past 2 years (26%). Overall, there was little change in drinking between baseline and follow-up (e.g., Cohen’s d = 0.11 SD and d = 0.12 SD differences for percent heavy drinking days and drinks per week, respectively).
Table 3.
Characteristics of Alcohol Use at Baseline and 6-Month Follow-up (n = 719)
| Baseline Weighted mean (SD) |
Follow-up Weighted mean (SD) |
|
|---|---|---|
| Percent heavy drinking days*, of days available | 20.9 (31.4) | 17.6 (28.3) |
| Drinks per week | 18.3 (17.3) | 16.3 (17.5) |
| N (weighted %) | N (weighted %) | |
| AUDIT-C Score | ||
| Less than 5 | 120 (16.8%) | 186 (26.1%) |
| 5 to less than 8 | 362 (50.2%) | 341 (47.3%) |
| 8 or more | 237 (33.0%) | 191 (26.6%) |
| Encounter with alcohol use disorder (AUD) diagnosis in 2 years prior to screen-positive visit† | 189 (26.2%) | – |
| AUD diagnosis on day of screen-positive visit day | 171 (24.3%) | – |
Ns are observed frequencies; means, SDs, and percentages are weighted for survey non-response. *Heavy drinking days were defined as five or more drinks for men and four or more drinks for women in a day. †Patients were ineligible if there was an AUD encounter within 90 days prior to the screen-positive visit. Data shown above is based on survey responses at baseline and 6-month follow-up except AUD diagnoses, which were obtained from administrative VH records
Care for Alcohol Misuse
Table 1 describes care received for unhealthy alcohol use among patients followed at 6 months. Five patients (0.7%) had documentation of receiving all recommended care, and 28% had documentation of receiving at least half of the recommended care processes. Performance on the process measures varied widely, ranging from less than 1 to 93% (median, 33%). Processes of care with performance rates higher than 75% included screening for co-occurring depression (93%), delivering a brief intervention (84%), and testing patients’ liver function (81%). One measure of whether patients who received specialty mental health care were given a diagnostic assessment for depression had moderate performance (61%). The remaining 17 measures had lower performance (i.e., fewer than 50% of patients received recommended care). Among these, the lowest measures were treatment initiation (5.0%), offering pharmacotherapy (3.1%), treatment engagement (1.5%), and receipt of pharmacotherapy (0.5%).
Multivariable Model Results
Table 4 presents estimated changes in the percentage of heavy drinking days and percent changes in drinks per week in the 30 days prior to the follow-up interview after adjustment for baseline drinking, demographics, mental and physical functioning, and receiving all or most of care from VHA. In analyses of the full sample and of targeted subsamples, neither the overall composite nor either of the two sub-composites were associated with statistically significant changes in the percentage of heavy drinking days or changes in drinks per week.
Table 4.
Association Between Quality of Care for Unhealthy Alcohol Use Composite Measures and Change in Drinking Outcomes
| Inclusion criteria | Full sample | Targeted subsamples | ||
|---|---|---|---|---|
| Percent heavy drinking days | Drinks per week | Percent heavy drinking days | Drinks per week | |
| All | All | > 1 heavy drinking day at baseline | Men > 14 drinks Women > 7 drinks |
|
| N | 717 | 717 | 375 | 357 |
| % Change (95% CI) | % Change (95% CI) | % Change (95% CI) | % Change (95% CI) | |
| Overall composite model | ||||
| 7.3 (− 41.8, 97.8) | − 18.1 (− 45.2, 22.4) | 10.1 (− 40.4, 103.4) | − 21.6 (− 57.4, 44.4) | |
| Two-composite model | ||||
| Screening | 35.3 (− 9.8, 102.8) | 0.20 (− 23.7, 31.6) | 37.91 (− 10.4, 112.2) | 3.7 (− 32.5, 59.1) |
| Treatment | − 23.7 (− 50.4, 17.3) | − 20.0 (− 38.6, 4.3) | − 23.5 (− 50.0, 17.0) | − 26.1 (− 49.5, 8.3) |
Covariates included for adjustment were age, marital status, college education, non-Hispanic white, income, mental and physical health functioning, receiving all or most care in VHA, and the pre-screen-positive visit value of the outcome variable (i.e., heavy drinking percentage or drinks per week). Joint tests of the composites in the two-composite model were not significant for either outcome in full or targeted subsamples. Positive (negative) estimates indicate the composite measure is associated with higher (lower) drinking changes
Table 5 presents model results for nine selected individual quality measures. Brief Intervention was associated with a statistically significant reduction in drinks per week within the targeted subsample that had exceeded recommended drinking limits in the 30 days preceding the positive screening visit. Those in the subsample who received a brief intervention reduced their adjusted drinks per week by 25.4% (95% CI 6.0, 40.8; p = 0.013) on average. Reassessment of Alcohol Use was associated with a significant increase in the percent of heavy drinking days in both the full sample and the targeted subsample. In the full sample, patients with documentation of reassessment increased their adjusted percent of heavy drinking days by 30.75% (95% CI 5.3, 62.4; p = 0.015), and among those who had at least one heavy drinking day prior to the positive screen visit, the increase was 36.3% (95% CI 8.1, 71.8; p = 0.009).
Table 5.
Association Between Individual Measures of Quality of Care for Unhealthy Alcohol Use and Change in Drinking Outcomes
| Full sample | Targeted subsamples | ||||||
|---|---|---|---|---|---|---|---|
| Percent heavy drinking days | Drinks per week | Percent heavy drinking days | Drinks per week | ||||
| Inclusion Criteria | All | All | > 1 heavy drinking day | Men > 14 drinks Women > 7 drinks |
|||
| N | % Change (95% CI) | % Change (95% CI) | N | % Change (95% CI) | N | % Change (95% CI) | |
| Assess for Alcohol Use Disorder (1) | 717 | − 6.5 (− 26.9, 19.6) | − 1.8 (− 18.0, 17.6) | 375 | − 5.1 (− 26.7, 22.9) | 357 | − 1.4 (− 25.4, 30.5) |
| Evaluate for Suicide Risk (4) | 301 | − 3.0 (− 24.7, 25.1) | − 5.2 (− 26.4, 22.1) | 198 | − 5.6 (− 28.2, 24.2) | 198 | − 17.6 (− 39.7, 12.5) |
| Screen for Other Substance Use (5) | 717 | − 4.1 (− 22.9, 19.4) | − 8.7 (− 22.1, 7.0) | 375 | 2.2 (− 18.3, 27.9) | 357 | − 3.8 (− 23.6, 21.2) |
| Test Liver Function (7) | 301 | 34.0 (− 8.0, 95.1) | 12.0 (− 19.1, 54.9) | 198 | 47.0 (− 4.3, 125.7) | 198 | 14.9 (− 25.4, 76.9) |
| Brief Intervention (8) | 717 | − 18.5 (− 36.7, 5.0) | − 15.0 (− 28.3, 0.8) | 375 | − 19.9 (− 37.2, 2.1) | 357 | − 25.4 (− 40.8, − 6.0)* |
| Discuss Treatment Options (9) | 301 | 17.7 (− 10.7, 55.0) | 13.5 (− 15.9, 53.1) | 198 | 13.4 (− 13.5, 48.5) | 198 | 27.5 (− 10.0, 80.5) |
| Offer Psychosocial Intervention (10) | 301 | 12.5 (− 16.2, 51.0) | 15.8 (− 14.6, 57.0) | 198 | 6.6 (− 19.8, 41.6) | 198 | 25.6 (− 12.0, 79.2) |
| Reassess Alcohol Use (24) | 717 | 30.8 (5.3, 62.4)* | − 1.1 (− 18.8, 20.6) | 375 | 36.3 (8.1, 71.8)* | 357 | 3.6 (− 22.6, 38.8) |
| Repeat Brief Intervention (25) | 717 | 2.7 (− 21.8, 34.8) | − 13.4 (− 30.1, 7.2) | 375 | − 7.1 (− 29.4, 22.3) | 357 | − 21.2 (− 42.5, 7.9) |
Covariates included for adjustment were age, marital status, college education, non-Hispanic white status, income, mental and physical health functioning, receiving all or most care in VHA, and the pre-screen-positive visit value of the outcome variable (i.e., heavy drinking percentage or drinks per week). *p < 0.05
DISCUSSION
This study comprehensively applied measures of the quality of alcohol-related processes of care to an alcohol screen–positive primary care population seeking care in one of 11 primary care practices. The process measures span care received in both primary and specialty care settings, were based on the best available evidence for importance and feasibility, used both administrative encounter data and medical review data, and were the result of an expert panel process. We observed wide variability in the proportion of patients receiving recommended care, and the median performance rate across these measures was only 33%. Further, we observed negligible change in alcohol use in the 6 months following the positive screen. These findings highlight the need for improvements in the quality of care delivered to patients with unhealthy alcohol use (or at least care that is documented).
We found receipt of higher overall quality of care for unhealthy alcohol use among a diverse population of patients who screened positive on the AUDIT-C in primary care clinics was not associated with significant reductions in drinking during our follow-up period. Composite measures separately capturing screening/assessment and treatment similarly were not associated with reduced drinking. Subgroup analyses of patients with at least one heavy drinking day or who were drinking above recommended weekly limits at baseline yielded similar results. Receipt of brief intervention significantly predicted decreased drinking in patients who were drinking above recommended weekly limits. However, cautious interpretation is warranted, as up to three significant findings (p < 0.05) would be expected by chance alone.33
It remains unclear why we did not detect an overall process-outcome link between higher-quality alcohol care and reduced drinking, particularly when we used composites designed to capture multiple aspects of recommended care—an approach similar to prior work that found a process-outcome link in depression care.34 Low rates of receipt of effective interventions (e.g., pharmacotherapy) prevented us from adequately testing some measures. Perhaps evidence-based interventions identified in research and assessed by our measures were not applied with sufficient adherence. Alternatively, the approach to capture whether recommended care occurred may not reflect actual care delivered. For example, providers may document in the medical record that an assessment or intervention was conducted when it was not done with adequate fidelity to evidence-based practice.
Some limitations should be noted. Almost half of patients at baseline reported no heavy drinking days, limiting the amount of change we could detect and reducing the sample size of those with more severe drinking. There was negligible change in drinking outcomes overall (e.g., Cohen’s d = 0.11–0.12), leaving little room for reductions in drinking to be associated with composite and individual measures. Few patients received specialty care, limiting our ability to test composites and certain measures. VHA uses clinical reminders to prompt screenings, which may result in detecting a higher proportion of patients with unhealthy alcohol use. The focus on VHA clinics also resulted in fewer women and younger men being included in our sample, particularly because we used the VHA’s AUDIT-C positive screen cut point of 5 or greater for both men and women. Further, although we adjusted for observable risk factors, there could be residual confounding of our estimates by other unmeasured factors. For example, 95% confidence intervals for some measures (Test Liver Function, Reassess Alcohol Use, Discuss Treatment Options) suggested receipt of recommended care was associated with increased drinking. Perhaps providers detected and responded to signals of increased alcohol use that were not detected by our severity measures or other observable characteristics. Finally, our response rate (54% at baseline) may impact generalizability of our findings; there could be unobserved differences between study participants and the population despite the lack of significant differences on observed demographics.
Results indicated that few patients received recommended care following a positive screen for unhealthy alcohol use in routine practice. Our analyses did not demonstrate that overall receipt of a greater percentage of recommended care processes predicted decreases in drinking at 6-month follow-up among patients in our sample. Subgroup analyses suggest, however, that providing brief interventions to higher-risk patients is linked to reduced drinking. Results highlight a critical need for greater focus on how to provide comprehensive alcohol care to patients across both primary care and specialty care settings.
Acknowledgements
We thank the VA Greater Los Angeles HSR&D Center for the Study of Healthcare Innovation, Implementation & Policy and Lisa Altman, MD, for their collaboration and organizational support of this research. Further, we are grateful for the extensive contribution of Daniel Kivlahan, PhD, to this study and for his critiques on earlier versions of this manuscript. Portions of this paper were presented at the 2017 Addiction Health Services Research (AHSR) conference, Madison, WI; the 2017 International Network on Brief Interventions for Alcohol and Other Drugs (INEBRIA) conference, New York, NY; and the 2017 Research Society on Alcoholism (RSA) conference, Denver, CO.
Appendix. Alcohol Use Hot-Deck Imputation Approach
We imputed missing daily alcohol consumption data, both at baseline and at 6-month follow-up, using a hot-deck multiple imputation approach. We considered drinking data separately for the baseline pre-screening period, baseline post-screening period, and 6-month follow-up period. For patients missing fewer than 7 days of drinking data within a period, we replaced missing data by randomly selecting, with replacement, a non-missing observation from the patient’s own data from the same period, matching on weekday.
For patients missing seven or more days of drinking data, we randomly selected, with replacement, a donor patient whose drinks per week value during the period was within 0.5 drinks of the recipient, who matched the recipient in having an AUDIT-C score either below 10 or 10 or more, and who had valid drinking data on at least one of each weekday within the period. Once a suitable donor was identified, we sampled with replacement individual days from the donor’s drinking data to impute each missing day for the recipient, matching on day of the week. Follow-up drinking data were not imputed for two patients who reported zero days available for drinking during that 30-day period.
Patients who had their screening visit more than 23 days before the baseline interview lacked a full seven days of pre-screening drinking data, as we considered the screening visit day to be in the post-screening period. For these patients, we treated the deficit as missing data and identified appropriate donors as described above, imputing only enough pre-screening data needed to bring them to a full seven days. We conducted these procedures five times independently, creating five separate analytic datasets for use in our analyses. Our analytic models were fit on each of the imputed datasets, and regression results were combined using standard multiple imputation techniques to arrive at our final results.
Funding Information
This research was supported by a grant from the National Institute on Alcohol Abuse and Alcoholism (R01AA019440). Dr. Hoggatt was funded through a Department of Veterans Affairs, Veterans Health Administration (VA) Health Services Research & Development/Quality Enhancement Research Initiative (HSR&D/QUERI) Career Development Award (CDA 11-261) at the VA Greater Los Angeles Healthcare System.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Disclaimer
The views expressed within are solely those of the authors, and do not necessarily represent the views of the Department of Veterans Affairs or of the United States government.
Contributor Information
Kimberly A. Hepner, Phone: 310-393-0411, Email: hepner@rand.org
Susan M. Paddock, Email: paddock@rand.org
Katherine E. Watkins, Email: kwatkins@rand.org
Katherine J. Hoggatt, Phone: 310-478-3711, Email: khoggatt@g.ucla.edu.
Lisa V. Rubenstein, Phone: 310-393-0411, Email: lisar@rand.org
Andy Bogart, Email: abogart@rand.org
Praise O. Iyiewuare, Email: piyiewua@rand.org
Susan C. Rosenbluth, Phone: 310-268-3632, Email: Susan.Rosenbluth@va.gov
Harold Alan Pincus, Phone: 646-774-5400, Email: pincush@nyspi.columbia.edu
References
- 1.U.S. Preventive Services Task Force. Screening and behavioral counseling interventions in primary care to reduce alcohol misuse: Recommendation statement. AHRQ publication no. 12–05171-EF-3. Available at: http://www.uspreventiveservicestaskforce.org/uspstf12/alcmisuse/alcmisusefinalrs.htm. Accessed October 10, 2018.
- 2.Grant BF, Goldstein RB, Saha TD, et al. Epidemiology of DSM-5 alcohol use disorder: Results from the national epidemiologic survey on alcohol and related conditions III. JAMA Psychiatry. 2015;72(8):757–66. doi: 10.1001/jamapsychiatry.2015.0584. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bouchery EE, Harwood HJ, Sacks JJ, Simon CJ, Brewer RD. Economic costs of excessive alcohol consumption in the U.S., 2006. Am J Prev Med. 2011;41(5):516–24. doi: 10.1016/j.amepre.2011.06.045. [DOI] [PubMed] [Google Scholar]
- 4.U.S. Burden of Disease Collaborators The state of U.S. health, 1990-2010: Burden of diseases, injuries, and risk factors. JAMA. 2013;310(6):591–606. doi: 10.1001/jama.2013.13805. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hingson RW, Heeren T, Edwards EM, Saitz R. Young adults at risk for excess alcohol consumption are often not asked or counseled about drinking alcohol. J Gen Intern Med. 2012;27(2):179–84. doi: 10.1007/s11606-011-1851-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Watkins KE, Pincus HA, Paddock S, et al. Care for veterans with mental and substance use disorders: Good performance, but room to improve on many measures. Health Aff. 2011;30(11):2194–203. doi: 10.1377/hlthaff.2011.0509. [DOI] [PubMed] [Google Scholar]
- 7.National Institute for Health and Care Excellence. Alcohol-use disorders: Diagnosis, assessment and management of harmful drinking and alcohol dependence (Clinical guideline [CG115]). Available at: https://www.nice.org.uk/guidance/cg115. Accessed October 10, 2018.
- 8.U.S. Department of Veterans Affairs and U.S. Department of Defense. VA/DoD clinical practice guideline for the management of substance use disorders - Version 3.0–2015. Available at: http://www.healthquality.va.gov/guidelines/MH/sud/VADoDSUDCPGRevised22216.pdf. Accessed October 10, 2018.
- 9.Garnick DW, Lee MT, Chalk M, et al. Establishing the feasibility of performance measures for alcohol and other drugs. J Subst Abuse Treat. 2002;23(4):375–85. doi: 10.1016/S0740-5472(02)00303-3. [DOI] [PubMed] [Google Scholar]
- 10.McCorry F, Garnick DW, Bartlett J, Cotter F, Chalk M. Developing performance measures for alcohol and other drug services in managed care plans. Washington Circle Group. Jt Comm J Qual Improv. 2000;26(11):633–43. doi: 10.1016/s1070-3241(00)26054-9. [DOI] [PubMed] [Google Scholar]
- 11.Johnson JA, Lee A, Vinson D, Seale JP. Use of AUDIT-based measures to identify unhealthy alcohol use and alcohol dependence in primary care: A validation study. Alcohol Clin Exp Res. 2013;37(Suppl 1):E253–9. doi: 10.1111/j.1530-0277.2012.01898.x. [DOI] [PubMed] [Google Scholar]
- 12.Esposito ML, Selker HP, Salem DN. Quantity over quality: How the rise in quality measures is not producing quality results. J Gen Intern Med. 2015;30(8):1204–7. doi: 10.1007/s11606-015-3278-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.National Quality Forum. Measure evaluation criteria and guidance for evaluating measures for endorsement. Available at: http://www.qualityforum.org/WorkArea/linkit.aspx?LinkIdentifier=id&ItemID=79434. Accessed October 10, 2018.
- 14.Harris AHS, Humphreys K, Bowe T, Tiet Q, Finney JW. Does meeting the HEDIS substance abuse treatment engagement criterion predict patient outcomes? J Behav Health Serv Res. 2010;37(1):25–39. doi: 10.1007/s11414-008-9142-2. [DOI] [PubMed] [Google Scholar]
- 15.Dunigan R, Acevedo A, Campbell K, et al. Engagement in outpatient substance abuse treatment and employment outcomes. J Behav Health Serv Res. 2014;41(1). 10.1007/s11414-013-9334-2. [DOI] [PMC free article] [PubMed]
- 16.Garnick DW, Horgan CM, Acevedo A, et al. Criminal justice outcomes after engagement in outpatient substance abuse treatment. J Subst Abuse Treat. 2014;46(3):295–305. doi: 10.1016/j.jsat.2013.10.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Paddock SM, Hepner KA, Hudson TJ, et al. Association between process-based quality indicators and mortality for patients with substance use disorders. J Stud Alcohol Drugs. 2017;78(4):588–96. doi: 10.15288/jsad.2017.78.588. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Arch Intern Med. 1998;158(16):1789–95. doi: 10.1001/archinte.158.16.1789. [DOI] [PubMed] [Google Scholar]
- 19.Williams EC, Rubinsky AD, Chavez LJ, et al. An early evaluation of implementation of brief intervention for unhealthy alcohol use in the U.S. Veterans Health Administration. Addiction. 2014;109(9):1472–81. doi: 10.1111/add.12600. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Löwe B, Kroenke K, Herzog W, Gräfe K. Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9) J Affect Disord. 2004;81(1):61–6. doi: 10.1016/S0165-0327(03)00198-8. [DOI] [PubMed] [Google Scholar]
- 21.Fleishman JA, Selim AJ, Kazis LE. Deriving SF-12v2 physical and mental health summary scores: a comparison of different scoring algorithms. Qual Life Res. 2010;19(2):231–41. doi: 10.1007/s11136-009-9582-z. [DOI] [PubMed] [Google Scholar]
- 22.Hepner KA, Watkins KE, Farmer CM, Rubenstein L, Pedersen ER, Pincus HA. Quality of care measures for the management of unhealthy alcohol use. J Subst Abuse Treat. 2017;76:11–7. doi: 10.1016/j.jsat.2016.11.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Mattox T, Hepner KA, Kivlahan D, et al. Quality measures to assess care for alcohol misuse: Measure technical specifications (TL-197-NIAAA). Santa Monica, CA: RAND Corporation; 2016. http://www.rand.org/pubs/tools/TL197.html. . Accessed 21 Nov 2018.
- 24.Agency for Healthcare Research and Quality. Part II. Introduction to measures of quality. Available at: https://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/perfmeasguide/perfmeaspt2.html. Accessed October 10, 2018.
- 25.Shwartz M, Restuccia JD, Rosen AK. Composite measures of health care provider performance: A description of approaches. Milbank Q. 2015;93(4):788–825. doi: 10.1111/1468-0009.12165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Sobell LC, Brown J, Leo GI, Sobell MB. The reliability of the Alcohol Timeline Followback when administered by telephone and by computer. Drug Alcohol Depend. 1996;42(1):49–54. doi: 10.1016/0376-8716(96)01263-X. [DOI] [PubMed] [Google Scholar]
- 27.National Institute on Alcohol Abuse and Alcoholism. What’s “low-risk” drinking? Available at: http://rethinkingdrinking.niaaa.nih.gov/IsYourDrinkingPatternRisky/WhatsLowRiskDrinking.asp. Accessed October 10, 2018.
- 28.Little RJA, Rubin DB. Statistical analysis with missing data. New York, NY: Wiley; 1987. [Google Scholar]
- 29.The American Association for Public Opinion Research . Standard definitions: final dispositions of case codes and outcome rates for surveys. 9. Oakbrook Terrace, IL: The American Association for Public Opinion Research; 2016. [Google Scholar]
- 30.Lumley T. Analysis of complex survey samples. J Stat Softw. 2004;9(1):1–19. [Google Scholar]
- 31.Lumley T. Survey: analysis of complex survey samples. 2014. R package version. 2014;3:5.
- 32.Cohen J. Statistical power analysis for the behavioral sciences. 2. Mahwah, NJ: Lawrence Erlbaum Associates; 1988. [Google Scholar]
- 33.Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine--reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357(21):2189–94. doi: 10.1056/NEJMsr077003. [DOI] [PubMed] [Google Scholar]
- 34.Hepner KA, Rowe M, Rost K, et al. The effect of adherence to practice guidelines on depression outcomes. Ann Intern Med. 2007;147(5):320–9. doi: 10.7326/0003-4819-147-5-200709040-00007. [DOI] [PubMed] [Google Scholar]