Stand Down–Think Before You Drink: protocol for an effectiveness-implementation trial of a mobile application for unhealthy alcohol use with and without peer support

Abstract

Introduction

Mobile apps can increase access to alcohol-related care but only if patients actively engage with them. Peers have shown promise for facilitating patients’ engagement with mobile apps. However, the effectiveness of peer-based mobile health interventions for unhealthy alcohol use has yet to be evaluated in a randomised controlled trial. The goal of this hybrid I effectiveness-implementation study is to test a mobile app (‘Stand Down–Think Before You Drink’), with and without peer support, to improve drinking outcomes among primary care patients.

Methods and analysis

In two US Veterans Health Administration (VA) medical centres, 274 primary care patients who screen positive for unhealthy alcohol use and are not currently in alcohol treatment will be randomised to receive usual care (UC), UC plus access to Stand Down (App), or UC plus Peer-Supported Stand Down (PSSD—four peer-led phone sessions over the initial 8 weeks to enhance app engagement). Assessments will occur at baseline and 8-, 20- and 32-weeks postbaseline. The primary outcome is total standard drinks; secondary outcomes include drinks per drinking day, heavy drinking days and negative consequences from drinking. Hypotheses for study outcomes, as well as treatment mediators and moderators, will be tested using mixed effects models. Semi-structured interviews with patients and primary care staff will be analysed using thematic analysis to identify potential barriers and facilitators to implementation of PSSD in primary care.

Ethics and dissemination

This protocol is a minimal risk study and has received approval from the VA Central Institutional Review Board. The results have the potential to transform the delivery of alcohol-related services for primary care patients who engage in unhealthy levels of drinking but rarely seek treatment. Study findings will be disseminated through collaborations with healthcare system policymakers as well as publications to scholarly journals and presentations at scientific conferences.

Trial registration number

NCT05473598.

STRENGTHS AND LIMITATIONS OF THIS STUDY

• This study was designed through a collaboration between researchers and leadership of the US Department of Veterans Affairs and informed by feedback from veteran stakeholders.

• The study design allows for an evaluation of the effectiveness of a mobile health intervention with and without peer support and its potential for wider-scale implementation.

• Although the study design is consistent with recommendations for pragmatic trials, imbalances in time and attention across study conditions may limit inferences on the specific mechanisms for improving patient outcomes.

Introduction

Unhealthy alcohol use—representing the spectrum from drinking above recommended limits to an alcohol use disorder (AUD)—is linked to numerous societal costs worldwide.^1–3 Despite this, most individuals who engage in unhealthy alcohol use receive little to no treatment for their drinking.⁴ In the US Veterans Health Administration (VA), 15%–30% of patients seen in primary care screen positive for unhealthy alcohol use on the Alcohol Use Disorder Identification Test for Consumption (AUDIT-C).⁵ However, less than 30% of these patients receive any intervention during the primary care visit to reduce their alcohol use.⁶

Mobile applications (apps) can expand access to care for those engaging in unhealthy alcohol use.⁷ Apps eliminate travel time to in-person appointments and provide a care option that is private and discreet, thus addressing the stigma that is often a barrier to individuals entering alcohol use treatment.⁸ Mobile apps for substance use problems are generally viewed as useful and easy to use⁹; however, evidence for their effectiveness from randomised controlled trials (RCTs) is limited.^{10 11} Mobile apps can only be effective for improving drinking outcomes if patients actively engage with them. Yet, poor patient engagement continues to be the Achilles’ heel of mobile apps for alcohol use self-management.^{12 13}

Peers may play a valuable role in facilitating patients’ engagement with mobile apps. In large healthcare systems such as VA, peer specialists are non-clinical staff with lived experience of substance use and/or mental health problems who are now in recovery and have been trained to provide services to others who currently struggle with these problems.¹⁴ Peer specialists are embedded within behavioural health and primary care teams to help patients navigate the care system, serve as self-care role models and provide social support.^{15 16} Such support has been found to facilitate engagement in substance use treatment by countering self-stigma related to addiction and alleviating patients’ mistrust of the healthcare system.¹⁷ Accordingly, peer specialists may be ideally suited to facilitate patients’ engagement with mobile apps for alcohol use. Peer specialists can orient patients to an app, provide assurance that the app is secure and coach patients on how to apply app content to their drinking-related goals.¹⁸ This approach is consistent with the supportive accountability model of e-health, which posits that adherence to digital therapeutics is enhanced through accountability to a coach who is seen as trustworthy and benevolent.¹⁹

We developed and piloted a protocol for peer specialists to enhance engagement with an app called, Stand Down–Think Before You Drink.²⁰ Stand Down is a veteran version of the Step Away app, which was found in an RCT to improve drinking outcomes in a community-sample of young adults with AUD.¹¹ Grounded in motivational enhancement and cognitive-behavioural therapy, the app offers assessment and personalised feedback to enhance awareness of drinking-related problems, establishes and monitors progress towards a drinking goal, provides in-the-moment coping tools to manage cravings and connects app users with supportive persons. In a single-arm trial, 31 veterans who screened positive for unhealthy drinking during a primary care visit had access to Stand Down for 4 weeks and received weekly phone support from a peer specialist to facilitate app engagement.²¹ Acceptability in terms of app usage and patient satisfaction ratings exceeded a priori benchmarks, and patients reported significant reductions in total standard drinks, drinks per drinking day (DPDD) and heavy drinking days (HDD). Although promising, the effectiveness of peer-supported use of Stand Down for improving drinking outcomes has yet to be demonstrated in an RCT. Further, knowledge of facilitators and barriers to implementing peer-based mobile health interventions for unhealthy alcohol use in primary care settings is limited.²²

In this protocol paper, we describe a pragmatic RCT to test the effectiveness and implementation potential of the Stand Down app, with and without peer support, to improve drinking outcomes among primary care patients. The overarching study goals are to evaluate whether Stand Down reduces drinking among primary care patients and for whom peer support of the app is more effective than the app alone.

Methods and analysis

Patient and public involvement

Prior to funding acquisition, the investigator team collaborated with leadership from VA’s Office of Mental Health and Suicide Prevention, VA’s Office of Connected Care, and the VA Palo Alto’s Veteran and Family Engagement Council to solicit feedback on study goals, design, outcomes and recruitment strategies.

Study design

This trial is a hybrid type 1 study.²³ The primary goal of such studies is to evaluate the effectiveness of an intervention in an RCT, with a secondary goal of gathering data from stakeholders on the implementation context of the intervention. At two VA medical centres, primary care patients will be enrolled in the study, complete a baseline phone interview, randomly assigned to one of the three conditions—usual care (UC), UC+access to Stand Down (App) or UC+Peer-Supported Stand Down (PSSD)—and complete assessments by phone at 8-weeks, 20-weeks and 32-weeks postbaseline. We hypothesise that over this timeframe patients in the App condition (vs UC) and PSSD (vs UC and vs App) will have greater reductions in total standard drinks (Aim 1—Hypothesis 1a), DPDD, HDD and negative consequences from drinking (Aim 1—Hypothesis 1b), and that the effects of App and PSSD on outcomes will be mediated by increases in readiness to change drinking, increases in self-efficacy to reduce drinking and greater app usage (Aim 1—Hypothesis 1c). In Aim 2, we hypothesise that App patients will report higher satisfaction than UC patients, and PSSD patients will report higher satisfaction than App patients. Aim 2 will also involve qualitative interviews with primary care staff and patients from each site who were randomised to PSSD. The goal of this process evaluation is to identify potential barriers and facilitators to the implementation of PSSD within primary care. Aim 3 will explore moderators of associations between PSSD and better outcomes.

Participants

Patients will be recruited into the RCT who are at a study site and (1) had a positive AUDIT-C (score of >5 for men, ≥4 for women) during a primary care visit in the past month, (2) have documented receipt of a brief alcohol intervention following a positive AUDIT-C, (3) did not receive any outpatient, inpatient or residential care for alcohol use in the month after their positive AUDIT-C, (4) own a smartphone and (5) have no active diagnoses of a psychotic disorder or cognitive disorder. Power analyses focused on expected effect size reductions for the primary outcome—total standard drinks. Based on the pilot study,²¹ we expect the PSSD condition will generate an effect size change (Cohen’s d) of 0.57 over the study period, the App condition will generate an effect size change of 0.39, and the UC condition will generate effect size change ranging from d=0.0–0.5. Based on these estimates, the variance in effect sizes across the three study conditions is expected to range from 0.067 to 0.072. Using the more conservative estimate of 0.067, in a two-way analysis of variance across three arms and two sites, with an alpha of 0.05 and 80% power, 73 patients per arm (219 total) will be needed to detect this effect size variance. To account for 20% attrition at follow-ups, we will recruit 274 patients.

For the process evaluation, at each site we will conduct a semi-structured interview by telephone with 12 primary care staff and 12 patients who were randomised to PSSD. Staff interviews will take place after all patient participants have completed the intervention phase of the RCT. Patients will be compensated $25 for this interview, which will occur after the participant has completed or dropped out of the intervention.

RCT recruitment and randomisation

Eligible patients will be identified from VA’s administrative databases, mailed a recruitment letter and contacted via phone by a research assistant (RA) to assess interest and complete a brief screening interview to confirm study eligibility (eg, positive AUDIT-C). For those who are interested and eligible, verbal consent will be obtained and the baseline phone interview will be administered. After this interview, patients will be randomly assigned to study conditions. Randomisation schedules for each site will be created by the Study Biostatistician and made accessible to only the study coordinator and the site investigators to ensure blinding of the outcomes assessment. Randomisation will occur in random block sizes of 3, 6 and 9 and be stratified by gender and AUDIT-C scores from the phone screen. Following the baseline interview, an RA will notify the study coordinator who will contact the participant to inform them of condition assignment. Recruitment for the RCT started on 1 December 2022 and is scheduled to end on 30 November 2024.

Study conditions

UC condition

All study patients will have access to UC for unhealthy alcohol use in primary care, including annual screening via the AUDIT-C and a brief intervention following a positive AUDIT-C (ie, feedback from a provider to reduce drinking and the impact of alcohol use on health). VA guidelines also recommend that patients be referred to specialty care for AUD if they have a prior AUD diagnosis, an AUDIT-C score >8, or a comorbid mental health or medical condition that can be exacerbated by alcohol use.²⁴

App condition

Patients assigned to this condition will receive UC and be provided with a unique code by the study coordinator to access the Stand Down app. The study coordinator will also assist patients with entering their code and downloading the app to their smartphone to ensure exposure to the intervention. The app comprises eight modules: (1) Drinking patterns—assessment and personalised, norm-referenced feedback on drinking patterns and problems; (2) Goals—selecting moderation or abstinence as a drinking goal; (3) Rewards—setting up rewards for meeting a drinking goal; (4) Cravings—information on alcohol cravings and coping strategies to manage them; (5) Strategies—behavioural strategies for relapse prevention; (6) Supportive persons—identifying and sharing progress towards a drinking goal with family and/or friends; (7) Reminders—creating verbal and visual reminders of reasons to change drinking and (8) New Activities—scheduling non-drinking activities instead of drinking. In addition to these modules, app users can enter high-risk times for drinking and receive alerts when these times are approaching, and daily notifications to complete in-app assessments to track their alcohol consumption and cravings. A ‘Get Help’ feature that provides immediate assistance with alcohol cravings and psychological distress, as well as outreach to other types of support (eg, VA crisis line) are also available in the app, as needed.

PSSD condition

Patients assigned to this condition will receive UC, access to the Stand Down app, plus four bi-weekly phone sessions with a peer specialist over the initial 8 weeks of the study. Sessions will be approximately 15–30 min in length, and focus on enhancing patients’ engagement with the app (see figure 1). In session 1, peers will introduce themselves, provide an overview of their role in the intervention, reassure patients about the privacy of the information they enter into the app, and assist with app set-up. In sessions 2 and 3, peers will inquire about patients’ use of the app since the last session, discuss the app content and provide suggestions on how it could be applied to patients’ drinking goals, help patients navigate the app and understand its functionality, and encourage ongoing usage of the app via action plans tailored to patients’ needs. In session 4, peers will cover all components of sessions 2 and 3 as well as review any benefits patients report from using the app, encourage ongoing usage until the 32-week follow-up and provide referrals to care, as needed. Throughout the four sessions, peers will provide emotional support and share their lived experiences with alcohol use and recovery.

Figure 1.

Main components of the Peer-Supported Stand Down intervention.

Study peers will complete a half-day training led by the site investigators to review the study procedures and intervention components, which are detailed in a manual. The manual includes templates for peers to document their sessions in the patients’ medical records. To monitor fidelity to the protocol, all sessions will be audio-recorded (with patient consent). For each peer, 25% of all sessions will be randomly selected each month by an RA and evaluated against a 4-item Fidelity Checklist. Group clinical supervision will be provided through weekly, 1 hour long video meetings between all peers and the site investigators.

Data collection

Randomised controlled trial

Treatment outcomes, mediators and moderators will be measured primarily through phone interviews conducted at four-time points (baseline and 8-weeks, 20-weeks and 32-weeks postbaseline) by RAs who are blind to condition. Participants will be compensated $50 after completion of each interview. We will follow the intent-to-treat principle so will retain all randomised patients in analyses regardless of study completion status. Table 1 provides an overview of the primary and secondary outcomes and hypothesised mediators and moderators, the data sources used to measure them, and the timeframe for when data will be collected.

Table 1.

Variables, measures and timeframes for data collection for the randomised clinical trial

Variables	Outcome type	Data source	Baseline	8 weeks	20 weeks	32 weeks
Total standard drinks	Primary	Timeline follow back (TLFB)	X	X	X	X
Drinks per drinking day	Secondary	TLFB	X	X	X	X
Heavy drinking days	Secondary	TLFB	X	X	X	X
Negative consequences from drinking	Secondary	Short Inventory of Problems-Revised	X	X	X	X
Satisfaction with care	Secondary	Client Satisfaction Questionnaire		X
Readiness to change drinking	Mediator	Readiness Ruler	X	X	X	X
Self-efficacy to reduce drinking	Mediator	Situational Confidence Questionnaire	X	X	X	X
Stand Down app usage	Mediator	Here & Now Systems, LLC Dashboard	X	X	X	X
Age and other sociodemographics	Moderator	Addiction Severity Index	X
Alcohol use severity	Moderator	Structured Clinical Interview for DSM-5 (SCID; past-year symptoms of alcohol use disorder)	X
Drug use severity	Moderator	SCID (past-year symptoms of drug use disorder)	X
Depression symptoms	Moderator	Patient Health Questionnaire-9	X	X	X	X
Trauma symptoms	Moderator	PTSD Checklist-DSM-5 version	X	X	X	X

Primary and secondary outcomes

The timeline follow back will provide information on quantity and frequency of alcohol use in the past 30 days.²⁵ Data from this reliable and well-validated interview will be used to measure the primary outcome (total standard drinks) and secondary outcomes of DPDD and HDD (ie, ≥5 and ≥4 standard drinks for men and women, respectively). Additional secondary outcomes include negative consequences from drinking, measured via the 15-item self-reported Short Index of Problems-Revised,²⁶ and satisfaction with care, measured via the 8-item self-reported Client Satisfaction Questionnaire (CSQ).²⁷

Hypothesised mediators

To measure the potential mediators of readiness to change drinking and self-efficacy to reduce drinking, we will administer the self-reported Readiness Ruler²⁸ and Situational Confidence Questionnaire (SCQ),²⁹ respectively. Responses on the Readiness Ruler are on a 1 (not ready to change) to 10 (trying to change) scale. The SCQ consists of 14 items, which ask patients to rate their level of confidence in resisting alcohol use in specific situations. Responses are averaged across items to produce a total score, with higher scores indicating more self-efficacy. Additional potential mediators will use Stand Down usage data (eg, modules used, days of use, frequency of use, in-app assessments completed) collected throughout the study duration and available to the study team through a secure dashboard housed by the app developer (Here & Now Systems).

Hypothesised moderators

Exploratory moderators measured at baseline will include age, alcohol and drug use severity, and comorbid psychiatric symptoms. Age and other demographic data (gender, race, ethnicity, marital status, education, employment status, income, housing status) will be gathered from selected portions of the Addiction Severity Index.³⁰ Alcohol and drug use severity will be operationalised via past-year symptom counts for alcohol and drug use disorders using the relevant modules from the Structured Clinical Interview for DSM-5.³¹ For drug use severity, questions will focus on the substance that the patient reports currently using the most. Comorbid psychiatric symptoms will be operationalised via total scores on the Patient Health Questionnaire-9 (PHQ-9),³² a 9-item self-report measure of symptoms of depression in the past 2 weeks, and the PTSD Checklist-5 (PCL-5),³³ a 20-item self-report measure of DSM-5 PTSD symptom severity in the past month.

Process evaluation

Qualitative interviews will be completed with patients randomised to PSSD to assess their general impressions of the intervention, how they used Stand Down, what they found most (and least) helpful about the app and the peer phone sessions, and suggestions for improving the intervention as a whole.³⁴ To ensure blinding, only the study coordinator will conduct these interviews.

Interviews with primary care staff will be guided by the Consolidated Framework for Implementation Research (CFIR).³⁵ Staff will be queried on CFIR constructs in domains of Intervention Characteristics (relative advantage; complexity—eg, ‘How complicated would it be for peers to support patients’ use of the Stand Down app in your clinic?’), Outer Setting (patient needs and resources—eg, ‘How do you think patients served by your clinic will respond to having peers support their use of the Stand Down app?’), Inner Setting (access to information and knowledge; compatibility—eg, ‘How well would use of peers to support patient engagement with the Stand Down app fit with existing work processes in your clinic?’), Characteristics of Individuals (knowledge and beliefs about the intervention—eg, ‘Do you think using peers to support patients’ use of the Stand Down app in your clinic will be effective? Why or why not?’) and Process of Implementation (planning—eg, ‘What planning would be needed if you were to use peers to support patients use of the Stand Down app in your clinic?’).

Analytic plan

Randomised controlled trial

To test the study hypotheses, we will employ generalised linear mixed-effects regression models (GLMMs) using full maximum likelihood estimation. These models will compare the App condition (vs UC) and PSSD (vs UC and vs App) on total standard drinks, DPDD, HDD and negative consequences from drinking over the follow-up period. To adjust for multiple comparisons due to the three-group comparison and multiple drinking outcomes, we will use the false discovery rate method with a q value threshold of 0.05.³⁶ All models will include main effects for time, condition and study site. We will assess the effectiveness of (1) App versus UC, (2) PSSD versus UC and (3) PSSD versus App using a Condition×Time interaction term, which will estimate the change in the outcome measures across the four assessment points for the planned contrasts. To detect variations in outcomes across sites, we will test a condition×site interaction term and retain this term in final models if it achieves significance. To compare the conditions on satisfaction with care (CSQ total scores at 8 weeks), analyses will follow the approach described above but exclude a term for the effect of time since this outcome will only be measured once. To address missing data, we will use model-based multiple imputation and impute missing values using multiple imputation by chained equations.³⁷

Regarding hypothesised mediators, we will examine whether readiness to change drinking (higher Readiness Ruler ratings) and self-efficacy to reduce drinking (higher SCQ total scores) mediate the relationship between the App and/or PSSD conditions (vs UC) and better outcomes at the follow-ups. We will also examine whether Stand Down usage mediates outcome differences between the App and PSSD conditions. To test these hypotheses, we will use methods for simultaneous testing of direct effects (condition assignment predicting the outcome) and indirect effects (whether a potential mediator accounts for the association between condition assignment and the outcome).³⁸ We will test separate models for each of the potential mediators, and bias-corrected bootstrapped CIs of the indirect effects will be calculated; mediation will be supported if the CIs do not include zero.

Regarding hypothesised moderators (ie, older age, greater AUD or drug use disorder severity, or higher PHQ-9 or PCL-5 total scores will be associated with better outcomes), using the GLMMs described above we will include the main effect of one of these moderators at baseline and its interaction with a dummy variable for condition, and Condition×Time interactions. If inclusion of the interaction terms for a given moderator yields significantly better model fit, this will provide evidence for moderation. For significant interaction terms, conditional moderators (±1 SD) will be evaluated to assess the direction and magnitude of effects within subgroups.

Process evaluation

Deidentified audio-files of the qualitative interviews will be transcribed and analysed using deductive and inductive coding techniques.^{39 40} This approach allows for identification of a priori codes informed by themes identified in studies of peer-supported mobile health,^{20 41 42} as well as new codes based on an initial review of the textual data. This process will identify themes related to potential barriers and facilitators to implementation of PSSD within primary care teams across the VA healthcare system. Themes will be categorised into tables, which will include potential solutions based on evidence-based implementation strategies⁴³ that could be considered to support more widespread implementation of PSSD.

Strengths and limitations

A key strength of the current study is its examination of the use of peers to enhance patients’ engagement with the Stand Down app, given findings that poor patient engagement severely limits the potential effectiveness of mobile apps for alcohol use.^{12 13} Providing peer support is a novel approach to potentially overcoming the problem of poor app engagement. The hybrid study approach is another strength as findings can be used to inform strategies to address potential barriers and facilitators to future implementation of PSSD in large integrated healthcare systems such as the VA, should the intervention be found to be of greater benefit in the RCT. The study findings will also suggest potential modifications to PSSD to enhance its effectiveness for improving drinking outcomes with the target population. Finally, the current study will address an important gap in the literature on effectiveness of mobile apps in primary care where most patients are older adults and may have limited awareness of their alcohol use problems. Evidence to date for the effectiveness of mobile apps for improving drinking outcomes has been limited to younger adults¹¹ and patients discharged from residential AUD care.¹⁰

A limitation of this study is that the UC and PSSD conditions are not balanced on time and attention. Thus, if PSSD is found to be more effective than UC it will remain uncertain if these effects are due to the components of this intervention or simply the greater time and attention that participants received. Further, there is no peer support-only condition to afford a test of whether such services are comparable or superior to use of an app to improve drinking outcomes. These limitations notwithstanding, this study’s approach is consistent with recommendations on the design of pragmatic trials in which the goal is to test whether a given intervention is more effective than the current standard of care for a patient population.^{44 45} Additional limitations include lack of biochemical verification of alcohol use and potential lack of generalisability of findings to non-veteran populations or veterans receiving primary care outside of VA.

Ethics and dissemination

This study is anticipated to have minimal risks to participant safety. All patient participants will continue to receive standard care for unhealthy alcohol use, and patients who decide to engage in substance use treatment after enrolling in the study will remain eligible for participation. For primary care staff, potential participants will be informed that their decision to participate or not is voluntary and that their supervisor will not be informed as to whether or not they agreed to participate. All research material will be collected for study purposes only. All participant data will remain confidential after it is collected and stored in a format that is identifiable only by a study-specific ID number rather than participants’ personally identifiable information. Only approved members of the study team will have access to the study data.

This study is being conducted in partnership with senior leadership in the VA Office of Mental Health and Suicide Prevention and Office of Connected Care. Throughout the study, the investigators will meet with these partners to discuss emerging results and identify opportunities for dissemination of findings on national calls and cyberseminars attended by primary care staff, peer specialists, and other VA and non-VA healthcare providers. In addition, study results will be shared with the scientific community through publications in peer-reviewed journals and presentations at national and international conferences.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Institutional review board approval was granted by VA’s Central IRB (Study #22-14) on May 20, 2022, which approved verbal informed consent by phone for participation. Details on data management, reporting of advere events, protection of confidentiality, data access, are documented in the IRB protocol. All methods were performed in accordance with the relevant guidelines and regulations of this institution. An independent data safety monitoring board is provided by VA HSR&D.