The ATTAIN Solution Tested: Initial Pilot Results of an Automated, Web-based Screening Tool for Unhealthy Drinking Behaviors

Abstract

Background:

Automated Alcohol Misuse Interventions (ATTAIN) is an automated, computer-based tool that screens people for unhealthy alcohol use and offers web-based counseling with minimal involvement of health-care personnel. We describe the outcomes of the initial quality improvement pilot tests done to examine the screening component of ATTAIN.

Methods:

This is a data-only study that reports the results from the ATTAIN–quality improvement database. A detailed description of the quality improvement methods is presented in a related article. The primary outcome was the ATTAIN questionnaire response rate. Secondary outcomes included the percentage of individuals who screened positive for unhealthy drinking behaviors and for alcohol use disorders, respondents’ comfort level with ATTAIN, and the cost of ATTAIN.

Results:

A total of 301 members were included in the pilot tests. The ATTAIN response rate was 46%. The prevalence of unhealthy drinking behaviors and alcohol use disorders was 16% and 5%, respectively. These rates were noninferior to the Healthcare Effectiveness Data and Information Set 2019 rates submitted from the same organization for the Unhealthy Alcohol Use Screening and Follow-up measure. In the post-ATTAIN paper surveys, 100% agreed to the statement: “I felt comfortable answering [ATTAIN] questions . . . .” The cost of ATTAIN screening was estimated to be one-tenth the cost of office-based screening.

Conclusion:

ATTAIN was well accepted by eligible adults, appeared noninferior to office-based screening, and added several potential advantages in terms of screening for alcohol use disorders and readiness to change.

INTRODUCTION

Excessive use of alcohol contributes to negative health, social, and financial outcomes for all age groups.¹ Thus, the Centers for Medicare and Medicaid Services and the National Committee for Quality Assurance have recommended that health-care organizations incorporate alcohol screening as a quality measure [Healthcare Effectiveness Data and Information Set (HEDIS)].^2-4 Some studies have tested digital or computer-based interventions to address unhealthy drinking behaviors, but have focused on hospitals, community-based settings, and smaller population subsets (such as veterans or college students).^5-29 There is a gap in research covering computer-based programs that address unhealthy drinking behavior screening and intervention in integrated, managed-care organizations.

In a separate, related article, we describe our development of an automated, computer-based tool that screens people for unhealthy drinking behaviors and alcohol use disorders, and offers appropriate web-based counseling with minimal involvement of health-care personnel.³⁰ This tool was integrated into the electronic medical record–patient portal interface of the integrated, managed-care health care organization (Kaiser Permanente, Southern California). It was developed by the Southern California Permanente Medical Group (SCPMG), which serves more than 2.68 million adult Kaiser Permanente health plan members, who were eligible for unhealthy drinking behavior screening. We named this tool ATTAIN, AuTomaTed Alcohol misuse INterventions. We planned a series of quality improvement (QI) pilots to test and refine ATTAIN so as to optimize the process of screening members for unhealthy drinking behaviors and alcohol use disorders, and offering brief interventions when appropriate.

Here we describe the outcomes of the initial QI pilots done to test the feasibility of screening members for unhealthy drinking behaviors and alcohol use disorders using the ATTAIN tool.

METHOD

Study Type and Procedures

This is a data-only study that reports results from an existing QI database. It includes all the data in the QI database without exclusion. The institutional review board at the institution where the QI project took place reviewed and approved this study. The specialty practice was a preventive medicine/health education-based clinic that received referrals for health plan members mainly looking to lose weight or improve their metabolic health (not related to alcohol).

QI Workflow

Eligible participants for the QI project were identified by scanning upcoming visits for 1 SCPMG provider in a specialty practice (using ATTAIN version 1.0 and, later, 1.1) and for 1 SCPMG provider in a primary care practice (using ATTAIN version 1.0).

Inclusion criteria were age ≥ 18 years and an active account with the health plan’s internet-based electronic medical record patient portal (www.kp.org). An invitation message with the attached ATTAIN questionnaire link was sent to these member accounts. As with all messages sent from the health plan, the message triggered an email to members’ personal email account, alerting them of a care-related message on their www.kp.org account. They could access their account on a computer or a portable electronic device. Messages were sent 1 day, 3 days, 7 days, or 14 days prior to their appointment date. Members who followed the link in the alert email were routed to their portal, where they could log in to their account. Once logged in, they could see the message (as well as a 30-second video) describing the purpose behind ATTAIN and asking them to complete the questionnaire by clicking a link. Patients’ responses became a part of their medical record and were returned to the sending provider’s in-basket (a unique electronic medical record inbox for each provider).

QI Survey Questions

In ATTAIN version 1.0, the initial question asked, “How often did you have a drink containing alcohol in the last year?” For those who answered “Never,” the questionnaire ended. For all others, 2 additional follow-up questions were asked to screen for unhealthy drinking behaviors relating to drinking beyond the recommended daily and weekly limits. These questions were age and gender specific. Those who screened positive on either of these 2 questions were given additional questions (based on the Diagnostic and Statistical Manual of Mental Disorders, 5th edition criteria) to screen for alcohol use disorders.³¹ Patients were also asked whether they were interested in cutting down (either on their own or via help) and whether they were interested in receiving medication or a specialty clinic referral. In ATTAIN version 1.1, the survey branching was modified, based on National Committee for Quality Assurance feedback, so that all participants were asked at least the first 3 questions (the initial question and the 2 additional questions just described for version 1.0).

In the specialty clinic pilot (ATTAIN version 1.0), a paper survey was given to all the invitees at their in-person visit to solicit feedback about their experience with the online survey.

Outcome Measures

The primary outcome was the ATTAIN questionnaire response rate (number of respondents who completed the questionnaire/number who were invited). We reviewed the existing QI database to determine this.

The secondary outcomes included the percentage who screened positive for unhealthy drinking behaviors; the percentage who screened positive for alcohol use disorders; any differences in response rates based on the number of days before an appointment that the invitation was sent (specialty clinic only); and the responses to questions regarding the interest in cutting down, interest in medication, and the interest in referral to a specialty clinic. We also noted demographic data (average age, gender, and language), which were recorded in the QI database for the specialty clinic group (but not recorded for the primary clinic group).

The ATTAIN questionnaire response rate and the percentage of patients that screened positive for unhealthy drinking behaviors were compared to HEDIS 2019 rates submitted from the SCPMG to the National Committee for Quality Assurance for the Unhealthy Alcohol Use Screening and Follow-Up measure.⁴

The cost of ATTAIN screening was estimated to be 10 cents × the total number of the population. The cost of office-based screening was estimated based on the following assumptions and calculations: [(Cost per person assuming 1 minute of nurse time to screen and $60/h cost per nurse) × (No. of members screened in the office in 2019)] + [(Cost per person assuming 2 minutes of physician time and $180/h cost per clinician) × (No. of members who screened positive in 2019 and would require further screening by the physician)].³²

We reported descriptive statistics from the QI database with the responses to the paper survey about the ATTAIN process. This paper survey was given in the office after the specialty clinic pilot (ATTAIN version 1.0) and results were reported in Excel graphs. We described whether participants reported receiving the email invitation, whether they completed the survey (for those who received it), their frequency of checking their email, whether they felt comfortable answering the ATTAIN questions, whether they felt they answered the questions honestly, whether they found the survey helpful, whether if they found it easy to complete, and whether they felt it was an invasion of privacy.

Statistical Analysis

The χ² test was used for analysis, with an alpha of 0.05 or less.³³

RESULTS

ATTAIN Total Results: Combined Specialty and Primary Care Clinics

A total of 301 patients met the inclusion criteria for the ATTAIN pilots (Table 1 and Figure 1). The average age was 47 years and 73% of the respondents were female. Ninety-five percent preferred English as their first language. The ATTAIN questionnaire response rate was 46% (137 of 301). The 2019 HEDIS reported screening rate by the SCPMG was 43% (SCPMG unpublished report).

Table 1.

Demographic characteristics of Automated Alcohol Misuse Interventions invitees in the primary and specialty clinics (N = 301)

Characteristics	Value
Mean age, y	47
Gender, n (%)
Male	80 (27)
Female	221 (73)
Language, n (%)
English	286 (95)
Spanish	13 (4.3)
Other	2 (0.6)

Figure 1.

Results flowchart of primary and secondary outcomes. ATTAIN, Automated Alcohol Misuse Interventions.

The prevalence of unhealthy drinking behaviors among the ATTAIN group was 16% (22 of 137). The 2019 HEDIS-reported positive screen rate by the SCPMG was 11.5% (SCPMG unpublished report).

The prevalence of alcohol use disorders among ATTAIN responders was 5% (7 of 137). Three participants indicated interest in further help, 9 indicated they would cut back on their own, and 6 participants indicated they were not interested in cutting back.

ATTAIN Results: Specialty Clinic

A total of 252 members (213 with ATTAIN version 1.0 and 39 with ATTAIN version 1.1) met the inclusion criteria in the specialty clinic pilots. The average age of participants was 45 years and 80% (201 of 252) were female. Ninety-four percent (238 of 252) preferred English as their first language. The ATTAIN questionnaire response rate was 44% (112 of 252) (Table 2).

Table 2.

Gender, language, and message timing in respondents vs nonrespondents in the specialty clinic (N = 252)

Variable	Responded to survey, n (%)	Did not responded to survey, n (%)	p Value
Gender			0.19
Male	92 (45)	109 (55)
Female	20 (39)	31 (61)
Preferred language			0.029a
English	110 (46.2)	128 (53.7)
Spanish	2 (15.4)	11 (84.6)
Messaging timing			0.18
1 d before	32 (48)	34 (52)
3 d before	45 (48)	49 (52)
1 wk before	23 (33)	46 (66.6)
2 wk before	12 (52)	11 (48)

^aSignificant.

The response rates by messaging timing in reference to their upcoming appointment were 48% (32 of 66) for 1 day before, 48% (45 of 94) for 3 days before, 33% (23 of 69) for 1 week before, and 52% (12 of 23) for 2 weeks before (p = 0.18). The response rates by gender were 46% (92 of 201) for females and 39% (20 of 51) for males (p = 0.19).

Of the 6% (13 of 252) of members in the QI database who reported Spanish rather than English as their preferred language, 15% (2 of 13) completed the ATTAIN questionnaire survey and both screened negative for unhealthy drinking behaviors (p = 0.29).

Among those who completed the survey, the prevalence of unhealthy drinking behaviors and alcohol use disorders was 11.6% (13 of 112) and 3.5% (4 of 112), respectively. The breakdown by gender for unhealthy drinking behaviors was 10% (9 of 92) for females and 20% (4 of 20) for males.

Overall, 2 participants indicated interest in further help. One of these screened positive for unhealthy drinking only and was interested in medications and in learning more about what we can offer. The other screened positive for a severe alcohol use disorder and was interested in medications, learning more about what we can offer, and in a referral to a specialist. Seven participants indicated they would cut back on their own, including 2 of those who screened positive for a moderate alcohol use disorder. Three participants indicated they were not interested in cutting back, including one who screened positive for a mild alcohol use disorder.

During follow-up for those who screened positive (for either unhealthy drinking or an alcohol use disorder), 4 subjects stated they had quit, 2 said they would quit right away, 3 stated they cut down markedly, and 1 said they would cut down to low risk levels.

ATTAIN Results: Primary Care

Forty-nine members (ATTAIN version 1.0) met the inclusion criteria in the primary care clinic pilot. The average age of participants was 55 years and 40% (20 of 49) were female. Forty-eight listed English and 1 listed either English or Spanish as their first language. The ATTAIN response rate was 51% (25 of 49). The response rates by gender were 45% (9 of 20) for females and 55% (16 of 29) for males (p = 0.7).

Among those who completed the survey, the prevalence of unhealthy drinking behaviors and alcohol use disorders was 36% (9 of 25) and 12% (3 of 25), respectively. The breakdown by gender for unhealthy drinking behaviors was 33% (3 of 9) for females (2 of whom did not complete the secondary questions for alcohol use disorders) and 37% (6 of 16) for males.

Overall, 1 participant was interested in learning more about what we can offer. Two participants (1 screened positive for unhealthy drinking and another for an alcohol use disorder) indicated they would cut back on their own. Three participants indicated they were not interested in cutting back, and 2 did not complete that part of the questionnaire.

Participant Perception Survey Results about the ATTAIN Process: Specialty Clinic

Thirty-six participants (of the 213 from ATTAIN version 1.0) completed paper surveys about their perception of the ATTAIN questionnaire process (Table 3 and Figure 2). Seventy-eight (28 of 36) indicated they received the email invitation to do the survey and, of those, 75% (21 of 28) indicated they completed it. Of 30 participants who answered questions about their email use, 73% (22 of 30) said they check email at least every 3 days and 27% (8 of 30) said they check their email once a week or less frequently. Of those who completed the survey, 100% (21 of 21) agreed to the statement “I felt comfortable answering questions on the survey,” 57% (12 of 21) agreed to the statement “I found the survey to be helpful,” 95% (20 of 21) agreed to the statements “It was easy to complete the survey” and “I answered the questions honestly,” and 5% (1 of 21) agreed to the statement “I felt like this survey was an invasion of privacy.”

Table 3.

Automated Alcohol Misuse Interventions feedback survey in the specialty clinic (N = 36)

Questions	n (%)
Received email survey?
Yes	28 (77.8)
No	8 (22.2)
Survey completed?
Yes	21 (75.0)
No	7 (25.0)
How often do you check your email?
Daily	21 (70.0)
Every 2–3 d	1 (3.3)
Weekly	4 (13.3)
> Weekly	4 (13.3)

Figure 2.

Participants’ perceptions of Automated Alcohol Misuse Interventions (ATTAIN) in the specialty clinic (N = 21). NA, not applicable.

Cost Estimates of Screening

The cost of screening via ATTAIN is estimated to be 10 cents per person whereas the costs of screening done in the traditional primary care office is estimated to be about $1 (assuming 1 minute of nurse time to screen and $60/h cost per nurse).

DISCUSSION

The burden of disease from alcohol in the US is very high. Although the US Preventive Services Task Force concludes with moderate certainty that screening and brief behavioral counseling interventions for unhealthy alcohol use is of moderate net benefit, many physicians do not perform this screening consistently.³⁴ Some reported barriers to screening include time constraints and a lack of knowledge about best practices.³⁴

ATTAIN addresses these barriers, providing a cost-effective way to screen and counsel adults. The rate achieved by ATTAIN in screening for unhealthy drinking behaviors (the primary outcome of this study) was noninferior to the rate of traditional primary care office screening for this condition reported in the health plan’s HEDIS measure submissions. The lower cost of ATTAIN, estimated to be one-tenth the cost of office screening, makes this a particularly encouraging result. The secondary outcomes suggested that ATTAIN 1) might detect more unhealthy drinkers compared to office-based screening, 2) included further screening for alcohol use disorders, 3) screened people for interest in medication or specialty services to help reducing drinking, 4) performed well in both the primary and specialty care settings, and 5) was well accepted by the target population. Furthermore, internal presentations and discussions of the data have led to an interest in expanding the process to automate other screenings (eg, for drug abuse, tobacco use, and obesity-related concerns), thus expanding ATTAIN’s potential beyond alcohol use.

Limitations

These pilots tested only the first goal of ATTAIN: to screen. They did not include integration of automated counseling (goal 2). Second, these pilots relied on subjects with upcoming appointments to a doctor’s office. It is not known whether ATTAIN would work similarly if sent to random health plan members (an important option because not all eligible adults see a provider each year). It is also not known whether it can be replicated in less integrated health-care organizations. Also, 6% of the participants in the QI database indicated Spanish rather than English as their preferred language, but still received the (English) ATTAIN questionnaire (a Spanish version was not available then). It is not known whether a language barrier may have affected the accuracy of these member’s responses as well as their low response rate. In addition, these results only prove ATTAIN as a supplement to office-based screening. If it was to replace office-based screening adequately, it should yield a completed questionnaire rate much greater than 44%. Last, the small sample size in the ATTAIN pilots limited its power in terms of comparison to the control group. Testing ATTAIN on a larger sample can show whether it is more accurate in detecting those with unhealthy drinking behaviors.

CONCLUSION

ATTAIN, an automated, web-based screening tool for unhealthy drinking behaviors, was well accepted by eligible adults and proved noninferior to traditional office-based screening in an integrated health-care organization. In addition, it added several potential advantages in terms of screening for alcohol use disorders and for indications of readiness to change. Further testing of ATTAIN on a larger sample of the population and with integrated automated counseling is needed.