Skip to main content
PMC home page
NPJ Digital Medicine logoLink to NPJ Digital Medicine
. 2024 Jun 29;7:174. doi: 10.1038/s41746-024-01169-7

Do engagement and behavioural mechanisms underpin the effectiveness of the Drink Less app?

Claire Garnett 1,2,✉,#, Larisa-Maria Dinu 1,#, Melissa Oldham 1, Olga Perski 3,4, Gemma Loebenberg 1, Emma Beard 1, Colin Angus 5, Robyn Burton 6,7, Matt Field 8, Felix Greaves 9,10, Matthew Hickman 11, Eileen Kaner 12, Susan Michie 13, Marcus Munafò 2,14, Elena Pizzo 15, Jamie Brown 1
PMCID: PMC11217434  PMID: 38951560

Abstract

This is a process evaluation of a large UK-based randomised controlled trial (RCT) (n = 5602) evaluating the effectiveness of recommending an alcohol reduction app, Drink Less, compared with usual digital care in reducing alcohol consumption in increasing and higher risk drinkers. The aim was to understand whether participants’ engagement (‘self-reported adherence’) and behavioural characteristics were mechanisms of action underpinning the effectiveness of Drink Less. Self-reported adherence with both digital tools was over 70% (Drink Less: 78.0%, 95% CI = 77.6–78.4; usual digital care: 71.5%, 95% CI = 71.0–71.9). Self-reported adherence to the intervention (average causal mediation effect [ACME] = −0.250, 95% CI = −0.42, −0.11) and self-monitoring behaviour (ACME = −0.235, 95% CI = −0.44, −0.03) both partially mediated the effect of the intervention (versus comparator) on alcohol reduction. Following the recommendation (self-reported adherence) and the tracking (self-monitoring behaviour) feature of the Drink Less app appear to be important mechanisms of action for alcohol reduction among increasing and higher risk drinkers.

Subject terms: Public health, Human behaviour

Introduction

Reducing increasing and higher risk alcohol consumption (defined as scoring 8 or more on the Alcohol Use Disorders Identification Test [AUDIT]) is a public health priority1. Digital interventions, such as websites and smartphone applications (‘apps’), potentially have a broad reach and low incremental costs for delivering alcohol interventions at scale2. Apps are a particularly promising mode of intervention delivery because smartphones have become increasingly affordable to end users, with approximately 84% of the UK population having access to a smartphone3. Meta-analyses have shown there is evidence for the effectiveness of digital interventions at reducing alcohol consumption4,5. Nonetheless, most of the interventions included in these meta-analyses were websites rather than apps. Furthermore, despite many alcohol reduction apps being available on app stores in the United Kingdom (UK), none have been evaluated in a randomised controlled trial (RCT) among the adult general population. To address this gap, we conducted an RCT (the iDEAS trial; iOS Drink Less, evaluating the Effectiveness of an Alcohol Smartphone app) to evaluate the effectiveness of recommending the evidence- and theory-informed app, Drink Less, in reducing alcohol consumption among increasing and higher risk drinkers in the UK, compared with usual digital care6,7. After accounting for missing data using multiple imputation, we found a two-unit reduction (95% CI = −3.76 to −0.24) in weekly alcohol consumption among the Drink Less group after 6 months compared with usual digital care, though the effect without imputation (where non-responders were assumed to be drinking at baseline levels) was weaker (−0.98 units, 95% CI = −2.67 to 0.70)7.

In addition to establishing whether an alcohol reduction app is effective, it is critical to understand its mechanisms of action, or, in other words, why it is effective. Understanding the underlying processes through which an intervention has its effects can help design more effective interventions8. To this end, process evaluations can help test hypothesised causal pathways using quantitative data9. In the current study, the overarching theoretical framework underpinning the Drink Less app is the COM-B model of behaviour10, and the proposed mechanisms of action were engagement with the interventional components of the Drink Less app (see ‘Methods’ section)11,12, which, in turn, influences participants’ behavioural characteristics, including urges to drink, motivation to drink less and self-regulatory behaviour (see Fig. 1 for the logic model).

Fig. 1. Logic model of the Drink Less app.

Fig. 1

Open in a new tab

Logic model showing which behaviour change techniques map onto the evidence-based Drink Less app modules and the proposed mechanisms of action: engagement with the interventional components of the Drink Less app, which, in turn, influences participants’ behavioural characteristics, including urges to drink, motivation to drink less and self-regulatory behaviour, leading to short-term, intermediate and long-term outcomes.

This study used data from the iDEAS trial comparing the effectiveness of the Drink Less app with usual digital care in reducing alcohol consumption in increasing and higher risk drinkers, focusing on participants’ behavioural characteristics and engagement with the intervention as part of the embedded mixed-methods process evaluation. The qualitative component evaluating the acceptability of the digital tools is reported elsewhere13 and found that Drink Less was perceived as being ethical, easy, user-friendly and effective. The following research questions were addressed:

  1. To what extent do participants self-report adhering to their recommended digital tool and how does this differ by group?

  2. Among participants in the intervention group, to what extent do participants engage with the Drink Less app in terms of (i) downloading the app; (ii) depth; (iii) frequency; (iv) duration; and (v) amount of use over the 6-month period from the date of recommendation?

  3. Does motivation to drink less at baseline moderate the effect of the intervention on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up?

  4. Do (i) urges to drink, self-regulatory and self-monitoring behaviour at 6-month follow-up, and (ii) self-reported adherence at 1- or 6-month follow-up to the recommended digital tool mediate the effect of the intervention on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up?

  5. Among participants in the intervention group, does extent of behavioural engagement with the Drink Less app mediate the effect of self-reported adherence on intervention effectiveness on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up?

Results

RQ1: Participant self-reported adherence to recommended digital tool

Among the intervention group, 78.0% (95% CI = 77.6–78.4) self-reported adherence (at either 1- or 6-month follow-up) to their digital tool, which was significantly greater than the 71.5% (95% CI = 71.0–71.9) in the comparator group (t = 18.470, p = 0.034), see Table 1. The pattern of results was similar when conducting a complete case analysis (see Table 1).

Table 1.

Self-reported adherence to their recommended digital tool by group

1-month follow-up 6-month follow-up 1- or 6-month follow-up
Multiple imputation for missing data (n = 5602)
Intervention (Drink Less), n = 2788 72.0% (95% CI = 71.6–72.4) 67.4% (95% CI = 67.0–67.9) 78.0% (95% CI = 77.6–78.4)
Comparator (NHS alcohol advice webpage), n = 2814 64.3% (95% CI = 63.8–64.7) 57.0% (95% CI = 56.5–57.5) 71.5% (95% CI = 71.0–71.9)
Complete case analysis
Intervention (Drink Less) 79.9% (n = 1435), 95% CI = 77.9–81.7 69.3% (n = 1481), 95% CI = 67.3–71.2 78.8% (n = 1782), 95% CI = 77.0–80.4
Comparator (NHS alcohol advice webpage) 71.2% (n = 1336), 95% CI = 69.1–73.2 56.3% (n = 1236), 95% CI = 54.2–58.4 70.9% (n = 1662), 95% CI = 69.0–72.7
Open in a new tab

RQ2: Extent of behavioural engagement with Drink Less among intervention group

Among participants in the intervention group, 1858 participants (66.6%, 95% CI = 64.9–68.4%) downloaded the Drink Less app and entered their email address. Ten participants downloaded the app outside of the 6-month period and were excluded.

Of the 1858, 128 participants downloaded the app multiple times (109 participants downloaded the app twice, 15 downloaded it three times, 3 downloaded it four times and 1 participant downloaded it five times). The median length of time for participants to download the app from being recommended it in the baseline survey was 3 min (mean = 1 day).

Among all participants in the intervention group (n = 2788), i.e., including the 33% of participants who did not download the app at all, they had a mean number of 34 sessions, spent a mean of 54 min on the app, used it for a mean of 25 days and viewed a mean of 17 unique screens, see Table 2. The figures were higher among only those who downloaded the app.

Table 2.

Engagement data for participants in the intervention group

Engagement All participants in intervention group (n = 2788) Participants who downloaded Drink Less (n = 1858)
Number of sessions (frequency)
Mean (SD) 34.3 (65.06) 51.4 (73.96)
Median (IQR) 5 (0, 32) 16 (5, 67)
Time on app in minutes (amount)
Mean (SD) 54.0 (115.25) 81.1 (133.20)
Median (IQR) 10 (0, 54) 32 (10, 95)
Number of days used (duration)
Mean (SD) 25.3 (44.44) 38.0 (49.83)
Median (IQR) 4 (0, 26) 13 (4, 52)
Number of unique screens viewed (depth)
Mean (SD) 17.4 (14.94) 26.1 (10.40)
Median (IQR) 18 (0, 30) 26 (18, 34)
Open in a new tab

SD standard deviation, IQR interquartile range.

RQ3: Motivation to drink less as a moderator of intervention effectiveness

No significant interaction was detected between participants’ motivation to drink less at baseline and their group allocation (intervention versus comparator) on intervention effectiveness (F1,5011.39 = 0.285, p = 0.594; log-transformed coefficient = 0.741, 95% CI = −1.927 to 3.410). This indicates that there was insufficient evidence to support a moderating effect of motivation to drink less at baseline having a differential effect on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up (using multiple imputation for missing outcome data at 6-month follow-up).

The same pattern of results was found in the sensitivity analysis (when using a complete case approach at 6-month follow-up) with no significant interaction detected between motivation to drink less and group allocation on the primary outcome (F1,3628546.53 = 0.023, p = 0.590; log-transformed coefficient = 0.751, 95% CI = −1.979 to 3.481).

RQ4: Urges to drink, self-regulatory and self-monitoring behaviour and self-reported adherence as mediators of intervention effectiveness

Among all participants, self-reported adherence to the recommended digital tool partially mediated the effect of the intervention on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up (average causal mediation effects [ACME] = −0.250, 95% CI = −0.42 to −0.11) and there was a direct effect of intervention group on alcohol reduction when not considering the path of self-reported adherence as a mediator (average direct effects [ADE] = −1.966, 95% CI = −3.68 to −0.13).

Self-monitoring behaviour partially mediated the effect of the intervention group on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up (ACME = −0.235, 95% CI = −0.44 to −0.03) and there was a direct effect of intervention group on alcohol reduction when not considering self-monitoring behaviour as a mediator (ADE = −1.966, 95% CI = −3.65 to −0.12).

No mediation of urges to drink (ACME = −0.096, 95% CI = −0.48 to 0.33) or self-regulatory behaviour (ACME = 0.142, 95% CI = −0.28 to 0.60) was detected.

In the sensitivity analysis among only those participants who completed the 6-month follow-up, both mediation results persisted but in both cases there was no significant direct effect of intervention group on alcohol reduction when not considering the mediators (see Supplementary Table 8).

RQ5: Extent of behavioural engagement as a mediator of the effect of self-reported adherence on intervention effectiveness in the intervention group

Among all participants in the intervention group (n = 2788), with treatment status as self-reported adherence to the app, no causally mediating effects of number of sessions (ACME = −0.009, 95% CI = −0.10 to 0.07), time on app (ACME = −0.017, 95% CI = −0.12 to 0.06), number of days used (ACME = −0.009, 95% CI = −0.10 to 0.06), or unique screens viewed (ACME = −0.024, 95% CI = −0.15 to 0.06) on alcohol reduction at 6-month follow-up were detected. This indicates that there was no causal mediation effect on alcohol reduction among the treatment status (i.e., those who self-reported adherence to the app) as a result of the mediator (i.e., extent of behavioural engagement). Among all participants in the intervention group, no direct effects of the treatment status (i.e., self-reported adherence to the app) on alcohol reduction at 6-month follow-up were detected, when not considering the engagement measures as mediators (number of sessions, ADE = 0.386, 95% CI = −2.49 to 3.31; time on app, ADE = 0.468, 95% CI = −2.55 to 3.56; number of days used, ADE = 0.413, 95% CI = −2.42 to 3.34; unique screens viewed, ADE = 0.523, 95% CI = −2.54 to 3.57).

In the sensitivity analysis among only the participants in the intervention group who completed the 6-month follow-up, no causally mediating effects were detected (see Supplementary Table 9). However, there were direct effects of the treatment status (i.e., self-reported adherence to the app) on alcohol reduction at 6-month follow-up, when not considering the behavioural engagement measures as mediators (see Supplementary Table 9). This indicates that while there were no causal mediation effects on alcohol reduction among the treatment status (i.e., those who self-reported adherence to the app) as a result of the engagement measures as a mediator, there was a direct effect of self-reported adherence to the app on alcohol reduction.

Discussion

This process evaluation investigated engagement with an alcohol reduction app, Drink Less, and participants’ behavioural characteristics as potential mechanisms of action underlying the effectiveness of the app. Self-reported adherence to both digital tools was over 70% and was significantly higher for Drink Less than for usual digital care. Self-reported adherence to the intervention and self-reported self-monitoring behaviour both partially mediated the effect of Drink Less, compared with usual digital care, on alcohol reduction.

Participants’ self-reported adherence was 78% for Drink Less and 72% for the NHS alcohol advice webpage. While 78% of participants in the intervention group self-reported using Drink Less, app data showed that only 67% downloaded the app. App downloads is a behavioural measure of adherence; however, it may be that some participants downloaded the app but did not enter their email address on the onboarding page, meaning that their app data could not be linked with their participation in the trial.

The extent of behavioural engagement with the Drink Less app among all participants randomised to the intervention (i.e., including the 33% of participants who did not download the app) was also investigated. Participants had a mean number of 34 sessions, spent a mean of 54 min on the app, used it for a mean of 25 days and viewed a mean of 17 unique screens. This data was positively skewed with participants using the app a median number of 5 sessions, 10 min, 4 days and viewing 18 unique screens. This suggests that there was a smaller group of ‘super users’ who were engaging intensively with the app and we are planning future research to characterise this group. The presence of highly engaged users has been found for other digital interventions14. When excluding participants who did not download the app, the mean number of 51 sessions, 81 min on the app, used it for 38 days and viewed 26 unique screens. While there are currently no guidelines for what constitutes adequate engagement, the data reported here suggest relatively high engagement with the Drink Less app when compared with engagement metrics reported in other studies of mobile health apps15. Comparing directly with a previous study of 672 Drink Less users in 2016, over a one-month period, people who downloaded the app had a median number of 5 sessions, 17 min on the app and used it for 4 days16. The current study measured engagement over a 6-month period and while the median values for users of the app were higher (median sessions = 16; median time = 32 min; median days used = 13) they were not six times higher. This could be due to the way in which participants found the app: in the 2016 study, users found the app in the iTunes store while in the current study participants were recommended (but did not choose themselves) to use the app (the recommendation was made remotely at the end of the baseline survey). Alternatively, it could relate to engagement tending to be higher in the earlier weeks and months and tailing off over time17.

Participants self-reported adherence to the intervention and self-reported self-monitoring behaviour (i.e., how often they kept track of how many units of alcohol they drank each week) both partially mediated their alcohol reduction at 6-month follow-up. This aligns with a previous investigation of behavioural engagement in a large sample of Drink Less users (over 19,000) which found that 85% of screen views occurred in the Self-monitoring & Feedback module of the app17 suggesting that this app module is how most users engage with the app. Furthermore, recent systematic reviews of mobile health apps have reported positive associations between self-monitoring behaviour and higher user engagement18, and self-monitoring behaviour and behaviour change19. These findings suggest that following the recommendation to use a digital intervention and tracking alcohol consumption are mechanisms which underlie the effectiveness of the Drink Less app.

There was no evidence that baseline motivation to drink less had a moderating effect of the intervention on alcohol reduction in increasing and higher risk drinkers at 6-month follow-up. This could be partially explained by a ceiling effect: the trial sample consisted of participants who were all motivated to cut down on alcohol reduction, which was reflected by the large reduction in alcohol consumption that was seen across both groups in the trial7. We found no evidence that self-regulatory behaviour or urges to drink had a mediating effect of the intervention on alcohol reduction, though there were reductions in both how difficult participants found it to control their drinking and in their strength of urges to drink from baseline to 6-month follow-up across both groups. The relationship between self-regulatory behaviour and alcohol reduction in the literature is mixed, with some research indicating that self-regulatory behaviour is related to alcohol-related consequences and not with alcohol consumption per se20,21. Furthermore, a contextual model of self-regulation change mechanisms in individuals with addictive disorders has been proposed, emphasising that the role of self-regulatory behaviour as a mechanism of behaviour change might depend on individual contextual factors22. The lack of evidence for self-regulatory behaviour mediating the effect of the intervention on alcohol reduction could be related to the fact that contextual factors were not specifically assessed in the iDEAS trial. However, research focusing on the importance of context in alcohol consumption is currently underway23.

Among participants in the intervention group (for whom we had data on their behavioural engagement), the effect of self-reported adherence on alcohol reduction did not appear to be mediated by the extent of behavioural engagement with the intervention (amount, duration, frequency or amount). This suggests that downloading the Drink Less app and following the recommendation is the critical engagement mechanism of the app’s effectiveness. That we did not detect a mediation effect of behavioural engagement with the app leading to better outcomes could be because behavioural engagement extends over time (e.g., frequency is an aggregated indicator, over time) and there might be dynamic feedback loops that were not accounted for here. It is also possible that any individual effect related to the behavioural engagement measures recorded within the app were overridden by the direct effect of self-reported adherence on alcohol reduction. We are planning future research to conduct more detailed modelling on exactly how users engage with the app in terms of which components, in what order and for how long to try and unpick further how engagement may relate to effectiveness.

This study reports on the findings from an embedded process evaluation of a large RCT of a theory- and evidence-based alcohol reduction app, Drink Less. We identified links between the outcome measures and participant engagement with the intervention and potential behavioural mechanisms of action.

Nonetheless, there are limitations. Firstly, given that self-reported adherence to the app was higher than the objective number of downloads, this analysis may have missed some participants who downloaded the app but failed to provide linkage data. Secondly, it was a strong assumption that self-reported adherence and engagement measures only relate to the pre-treatment covariates given that engagement is dynamic and fluctuates in response to time-varying factors24. Furthermore, we assumed each mediator was independent of each other, which they may not be, and it might be that engagement leads to changes to the behavioural mechanisms of actions, which then led to alcohol reduction. Using dynamic structural equation modelling could enable examination of multiple independent variables, mediators or outcomes25,26 and could be considered in future research. Another limitation was the reliance on self-report data for adherence and alcohol consumption. The self-reported adherence was higher than the objective app downloads though this may be due to the limitations in the ability to automatically link app usage with trial data, with some users potentially not including their email address or a different one to that used in the trial. However, the experimental design and remote recruitment with minimal research contact means there is likely limited differential bias between the two groups in the self-report data.

Finally, we were unable to assess whether behavioural engagement measures mediated the effect of the intervention on alcohol reduction as we were unable to measure behavioural engagement in the comparator group for the NHS alcohol advice webpage. As a result, we could only test whether engagement was a mediator among the intervention group, and whether engagement mediated the effect of self-reported adherence on alcohol reduction. However, it may have been that any individual mediation effect due to engagement was obscured by the direct effect of the self-reported adherence on alcohol reduction.

This study showed that engagement with the recommended digital tools was high in a large sample of digitally literate and motivated increasing and higher risk drinkers, with over 70% self-reporting adherence to either the Drink Less app or the NHS alcohol advice webpage. Self-reported adherence to the recommended digital tool partially mediated the effect of the intervention on alcohol reduction at 6-month follow-up indicating the importance of following the recommendation. Self-monitoring behaviour, i.e., how often users tracked their drinking, partially mediated the effect of the intervention on alcohol reduction suggesting that the Self-monitoring & Feedback module of the Drink Less app appears to be an important mechanism of action.

Methods

Ethics

Ethics approval was granted by the ethics committee at University College London (16799/001) and the trial was registered on the ISRCTN registry for clinical trials (ISRCTN64052601).

Pre-registered protocol

The pre-registered study protocol can be found on the Open Science Framework: https://osf.io/2s7ft. Two changes were made to the pre-registered study protocol. Namely, the sensitivity analysis for research question 5 to test for the possible existence of unobserved pre-treatment covariates using the medsens function in R was not conducted, as this function could not be used on multiply imputed datasets. Furthermore, three descriptive research questions were dropped from the main manuscript, but all are reported in Supplementary File 1.

Design

This was a process evaluation that assessed engagement and mechanisms of action of an intervention, the Drink Less app, embedded within a larger RCT6. Findings pertaining to the intervention acceptability are reported elsewhere7.

Setting and sample

As described elsewhere6, participants from the UK were invited to take part in a trial evaluating the long-term effectiveness and cost-effectiveness of the digital recommendation of the Drink Less app, compared with advice from the National Health Service (NHS) alcohol advice webpage (usual digital care), in reducing alcohol consumption. Participants were recruited between July 2020 and March 2022 and had to be aged 18+, increasing and higher risk drinkers (AUDIT score ≥8), live in the UK, have access to an iOS device and want to drink less alcohol.

Digital tools

Drink Less is a stand-alone app-based intervention that is freely available via the Apple app store in the UK27. Drink Less was developed for increasing and higher risk drinkers to help them reduce their alcohol consumption. Drink Less consists of evidence-based modules to help users change their drinking behaviour: Goal Setting, setting weekly ‘drinking reduction’ goals; Self-monitoring & Feedback, monitoring alcohol consumptions and seeing progress on goals; Action Planning, creating plans for dealing with difficult drinking situations; Normative Feedback, providing personalised feedback on how an individual’s drinking behaviour compares to the norm; Cognitive Bias Re-training, a game for retraining users’ automatic biases for alcoholic drinks; Behavioural Substitution, planning to substitute drinking with a neutral behaviour; and Information about Antecedents, providing users with information about situations and events, emotions and cognitions that predict their drinking. These evidence-based modules map to behaviour change techniques (see Fig. 1). The development, refinement, and content of the original Drink Less version is reported in full elsewhere11,12.

There were no specific requirements for participants when using the Drink Less app. On downloading Drink Less, users are asked to complete the AUDIT, provide sociodemographic details and then receive the Normative Feedback. Users are then guided through Goal Setting and shown how the key features of the app work before arriving on the landing page of the app with suggestions for the user to complete each day. The app provides a toolbox of features for users to choose from as and when they want. The app is not tailored to the user except for personalised feedback in two modules: Normative Feedback and Self-Monitoring & Feedback.

The comparator group received the recommendation to view the NHS alcohol advice webpage on ‘Tips on cutting down’28. This can be considered reflective of ‘usual digital care’ in this context as it is the digital support currently available to treatment-seeking individuals from the NHS.

Measures

Participants self-enrolled into the study and responded to a web-based screening questionnaire, which assessed the inclusion and exclusion criteria, including the full AUDIT. Informed consent was obtained from all eligible participants. The baseline and follow-up surveys (measuring alcohol consumption, sociodemographic characteristics, self-reported adherence and behavioural characteristics) were conducted online using Qualtrics.

Self-reported weekly alcohol consumption was measured at baseline and at the 6-month follow-up, calculated using the 3-item Alcohol Use Disorders Identification Test—Consumption (AUDIT-C)29. The AUDIT-C asks about frequency (‘How often do you have a drink containing alcohol’ with five response options: Never; Monthly or less; 2 to 4 times per month; 2 to 3 times per month; 4 times of more per month), quantity (‘How many units of alcohol do you drink on a typical day when you are drinking?’ with five response options: 0 to 2; 3 to 4; 5 to 6; 7 to 9; 10 or more) and frequency of heavy episodic drinking (‘How often have you had 6 or more units on a single occasion in the last year?’ with five response options: Never; Less than monthly; Monthly; Weekly; Daily or almost daily).

Sociodemographic characteristics were recorded at baseline. Participants were asked to report their age (in years), gender (% female), ethnicity (% white), education (% post-16 education qualifications), occupation (to derive social grade ABC1: managerial, professional and intermediate occupations versus C2DE: skilled, semi-skilled, unskilled manual and lowest grade worked or unemployed) and income level (% > £26,000).

Self-reported adherence was measured at the 1-and 6-month follow-up surveys for all participants. It asked ‘Did you look at or use the digital tool we recommended? It doesn’t matter either way, you will still be paid, but it will help us draw more accurate conclusions if you answer honestly.’ with Yes/No response options. It is an alternative measure of engagement for cases where it is not possible to automatically measure engagement (e.g., for participants randomised to the comparator).

Behavioural engagement can be automatically measured through app usage logs for individuals randomised to the intervention over a 6-month period from the date of the recommendation (i.e., when the iDEAS trial baseline survey was completed). Engagement with digital interventions can be defined as ‘the extent of digital behaviour change intervention use (e.g., frequency, amount, duration, depth)’30. Frequency of engagement was assessed by number of sessions, where a new session was defined as a new screen view after 30 min of inactivity31. Amount of engagement was assessed by time on app, in minutes. Duration of engagement was assessed by the number of days the app was used. Depth of engagement was assessed by the number of available screens viewed (without considering time spent viewing each screen). While not fine-grained, these measures provide more objective information on how participants interact with the app, to supplement data on self-reported adherence of the intervention. The number of participants who downloaded the app multiple times is also reported. If a participant downloaded the app multiple times (and entered their email address each time), their total engagement over the multiple instances was summed. For any participant who did not download Drink Less, the app download was recorded as ‘no’ and the engagement measures were recorded as 0.

Behavioural characteristics measures were assessed as potential mechanisms of action in the baseline and 6-month follow-up surveys using four measures for all participants: urges to drink; motivation to drink less; self-regulatory behaviours (how difficult to control drinking) and self-monitoring behaviours (how often alcohol units are tracked) (see Fig. 1 for the Logic Model). Strength of urges to drink was measured with the question ‘How strongly have you felt the urge to drink alcohol in the past 24 hours?’ with a 6-point scale for responses: not at all; slight; moderate; strong; very strong; extremely strong. Motivation to drink less was measured with the Motivation to Stop Scale32,33 where participants were asked: ‘Which of the following describes you?’ with the following options: (1) I REALLY want to cut down on drinking alcohol and intend to in the next month; (2) I REALLY want to cut down on drinking alcohol and intend to in the next 3 months; (3) I want to cut down on drinking alcohol and hope to soon; (4) I REALLY want to cut down on drinking alcohol but I don’t know when I will; (5) I want to cut down on drinking alcohol but haven’t thought about when; (6) I think I should cut down on drinking alcohol but don’t really want to; (7) I don’t want to cut down on drinking alcohol. Self-regulatory behaviour was measured with the question ‘How difficult do you find it to control your drinking?’ with a 5-point scale for responses: not at all; slightly; moderately; very; extremely34. Self-monitoring behaviour was measured with the question ‘How often, if at all, do you keep track of how many units of alcohol you personally drink each week?’ with a 5-point scale for responses: never; rarely; sometimes; very often; always.

Statistical analyses

All statistical analyses were conducted in R Studio (v2023.06). Multiple imputation (R package: Amelia) using baseline characteristics (gender, ethnicity, education, occupation, age and income) using 20 imputed datasets combined using Rubin’s rules was used for self-reported adherence, behavioural characteristics and alcohol reduction for the 20% of trial participants who did not respond to the 6-month follow-up survey35,36. Sensitivity analyses were also conducted using only the data from those participants responding to 6-month follow-up. Mediators of interest were assessed for skewness and log-transformed if necessary: motivation to drink less at baseline was negatively skewed and log-transformed, and all behavioural engagement measures were positively skewed and log-transformed (after adding 1 to avoid issues of infinity when log-transforming). For descriptive statistics, medians and interquartile ranges were reported to account for any potential skewness in the data.

For RQ1 (participant self-reported adherence to the recommended digital tool), among all participants, the proportion and 95% confidence interval (CI) of participants in each group self-reporting using their recommended digital tool (intervention or comparator) at (1) 1-month follow-up, (2) 6-month follow-up and (3) 1- or 6-month follow-up are reported.

For RQ2 (extent of behavioural engagement with Drink Less among intervention group), among participants in the intervention group (i.e., recommended to download the Drink Less app), the proportion and 95% CI of participants who downloaded the Drink Less app (i.e., input their email address when requested) is reported. Furthermore, the above analysis is repeated as a secondary analysis with those participants who did not download the app excluded. The length of time for participants to download the app from being recommended is also reported.

For RQ3 (motivation to drink less as a moderator of intervention effectiveness), the main analysis model from the main trial paper is reported (a one-way ANCOVA examining the effect of group allocation on the primary outcome, weekly alcohol consumption at 6-month follow-up, adjusting for baseline consumption using multiple imputation for missing outcome data at 6-months), including an interaction term between the moderator of interest (motivation to drink less at baseline, log transformed due to negative skewed data) and group allocation on the primary outcome (using multiple imputation for missing data at 6-month follow-up), adjusting for alcohol consumption at baseline, see Fig. 2.

Fig. 2. Moderation analysis for motivation to drink less.

Fig. 2

Open in a new tab

Motivation to drink less as a moderator of intervention effectiveness was assessed by including an interaction term between the moderator of interest (motivation to drink less at baseline, log transformed due to negative skewed data) and group allocation on the primary outcome in the primary analysis model from the main trial paper (a one-way ANCOVA examining the effect of group allocation on the primary outcome, weekly alcohol consumption at 6-month follow-up, adjusting for baseline consumption using multiple imputation for missing outcome data at 6-months).

For RQ4 (urges to drink, self-regulatory and self-monitoring behaviour and self-reported adherence as mediators of intervention effectiveness), a series of model-based causal mediation analyses are run to assess the following potential causal mechanisms for the effect of randomisation to the Drink Less app or NHS alcohol advice webpage on the primary outcome: (1) self-reported adherence to the recommended digital tool; (2) self-regulatory behaviour at 6-month follow-up, adjusting for baseline; (3) self-monitoring behaviour at 6-month follow-up, adjusting for baseline; (4) urges to drink at 6-month follow-up, adjusting for baseline. Two statistical models are specified, where we test the causal relationship between (1) the treatment status and the mediator (mediator model) and (2) the treatment status and the outcome (outcome model) (see Fig. 3 for the directed acyclic graph; DAG):

  1. The mediator model, where the mediator of interest (mediator) is modelled as a function of the recommendation of the digital tool (treatment status) and pre-treatment covariates (baseline alcohol consumption, age, gender, education level and occupation).

  2. The outcome model, where the outcome variable is alcohol consumption at 6-month follow-up, and the explanatory variables include the mediator, treatment status, and the same set of pre-treatment covariates as in the mediator model.

Fig. 3. Directed acyclic graph showing potential causal mechanism for the effect of Drink Less on self-reported adherence, self-regulatory and self-monitoring behaviour and urges to drink.

Fig. 3

Open in a new tab

Directed acyclic graph showing potential causal mechanism for the effect of randomisation to the Drink Less app on the outcome, weekly alcohol consumption at 6-month follow-up. A series of model-based causal mediation analyses were run for each potential mediator: (1) self-reported adherence, (2) self-regulatory behaviour, (3) self-monitoring behaviour and (4) urges to drink. The unmarked blue ovals refer to ancestors of outcome, the green oval refers to exposure, and the blue oval with an ‘I’ refers to the outcome.

The key identifying assumption is sequential ignorability which implies that, conditional on covariates, there is no unmeasured confounding of the treatment-mediator, treatment-outcome and mediator-outcome relationships. This assumption is satisfied if the treatment is randomised, as is the case with this analysis.

The ‘mediation’ package in R37 and the mediate function are used to estimate the average causal mediation effects (ACME) and the average direct effects (ADE), which represent the population averages of the causal mediation and direct effects. The ACME is the estimated average causal mediation effect in alcohol reduction among the treatment status (i.e., intervention vs. comparator group) as a result of the mediator, rather than ‘directly’ from the treatment. The ADE represents the average change in the outcome variable directly influenced by the treatment status. The mediate function is run on each of the 20 multiple imputed datasets and then combined the components of the output using the amelidiate function (this function does not pass the information required for calculation of p values).

For RQ5 (extent of behavioural engagement as a mediator of the effect of self-reported adherence on intervention effectiveness in the intervention group), a series of model-based causal mediation analyses are run to assess, among participants in the intervention group, the following measures of engagement as a causal mechanism for the effect of self-reported adherence (whether the participant downloaded Drink Less) on the primary outcome: (1) depth; (2) frequency; (3) duration; and (4) amount of use over the 6-month period from the date of recommendation. Two statistical models are specified:

  1. The mediator model, where the mediator of interest (extent of engagement) is modelled as a function of self-reported adherence (treatment status) and pre-treatment covariates (baseline alcohol consumption, age, gender, education level and occupation).

  2. The outcome model, where the outcome variable is alcohol consumption at 6-month follow-up, and the explanatory variables include the mediator, treatment status, and the same set of pre-treatment covariates as in the mediator model.

The average causal mediation effects (ACME) and the average direct effects (ADE) were estimated using the same R packages and procedures as specified above for RQ4.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Supplementary information

Supplementary Information (386.8KB, pdf)
Reporting summary (1.9MB, pdf)

Acknowledgements

This project is funded by the National Institute for Health and Care Research (NIHR) [Public Health Research Programme (project reference NIHR127651)]. The views expressed are those of the author(s) and not necessarily those of the National Institute for Health Research or the Department of Health and Social Care, or Public Health England (PHE). C.G. is funded by NIHR (NIHR302923). E.B. and J.B. are also funded by Cancer Research UK (CRUK: PRCRPG-Nov21\100002). Drink Less was funded by the NIHR School for Public Health Research (SPHR), the UK Centre for Tobacco and Alcohol Studies (UKCTAS), the Society for the Study of Addiction (SSA) and CRUK. E.P. is supported by the NIHR ARC North Thames. M.H. acknowledges NIHR Health Protection Research Unit in Evaluation, and M.M. and M.H. acknowledge NIHR Biomedical Research Centre at Bristol. J.B. and M.M. are part of the SPECTRUM Consortium, UK. O.P. is supported by a Marie Skłodowska-Curie Postdoctoral Fellowship from the European Union (Grant Agreement number: 101065293). E.K. is supported by an NIHR Senior Investigator and is Director of the NIHR funded Applied Research Collaboration Northeast and North Cumbria. The funders played no role in the design, conduct or analysis of the study, nor in the interpretation or reporting of study findings.

Author contributions

C.G. and L.D. are co-first authors and equally contributed to writing the original draft of the manuscript. All authors equally contributed to the editing and reviewing the final manuscript. C.G. and J.B. equally contributed to the funding acquisition with E.B., C.A., R.B., M.F., F.G., M.H., E.K., S.M., M.M. and E.P. supporting this. C.G., L.D., M.O., O.P., G.L., E.B. and J.B. equally contributed to conceptualisation of the study with C.A., R.B., M.F., F.G., M.H., E.K., S.M., M.M. and E.P. supporting this. C.G., L.D., M.O., O.P., M.M. and J.B. equally contributed to the methodology with G.L., E.B., C.A., R.B., M.F., F.G., M.H., E.K., S.M. and E.P. supporting this. M.O., G.L. and L.D. equally contributed to the investigation with C.G. and J.B. supporting this. L.D. and G.L. equally contributed to the project administration. C.G. and J.B. equally contributed to supervision. C.G., M.O. and E.B. equally contributed to data curation with L.D. and J.B. supporting this. C.G. led the formal analysis with support from L.D. and E.B. C.G. and E.B. equally contributed to validation.

Data availability

The anonymised data and data dictionary are available online at OSF (https://osf.io/2j9df/).

Code availability

The anonymised code is available online at OSF (https://osf.io/2j9df/).

Competing interests

O.P., C.A., M.M., G.L., L.D., M.F., E.B., E.P. and S.M. declare no conflicts of interest. M.O.’s salary is partially funded by Medical Research Council (MR/W026430/1). J.B. has received unrestricted research funding to study smoking cessation from Pfizer and J&J, who manufacture smoking cessation medications and sits in an unrenumerated role on the scientific advisory board for the SmokeFree app. C.G. and M.O. have done paid consultancy work for the behaviour change and lifestyle organization, ‘One Year No Beer’, providing fact checking for blog posts. F.G. is employed by NICE and previously by Public Health England PHE; he has no other conflicts of interest. R.B. is currently employed by the Office for Health Improvement and Disparities. M.H. is co-director of NIHR Health Protection Research Unit in Behavioural Science and Evaluation and a trustee for the Society for the Study of Addiction. Outside the submitted work, E.K. has previously co-authored papers that analysed raw market research consumer-based data provided to Newcastle University under a direct contract with Kantar Worldpanel at no cost to Newcastle University. Kantar Worldpanel received reimbursement from AB InBev to cover the costs of the data, Kantar WordPanel having similar commercial relationships with other customers who pay to have data collected on food and non-food items available for sale in supermarkets and other retail outlets covered by the WorldPanel.

Footnotes

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Claire Garnett, Larisa-Maria Dinu.

Supplementary information

The online version contains supplementary material available at 10.1038/s41746-024-01169-7.

References

  • 1.Babor, T., Higgins, J., Saunders, J. & Monteiro, M. AUDIT: The Alcohol Use Disorders Identification Test Guidelines for Use in Primary Care (World Health Organisation, 2001).
  • 2.West, R. & Michie, S. A Guide to Development and Evaluation of Digital Interventions in Healthcare (Silverback Publishing, 2016).
  • 3.Boyle, M. Mobile phone and internet usage in the UK. finder. https://www.finder.com/uk/mobile-internet-statistics (2022).
  • 4.Kaner EFS, et al. Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. Cochrane Database Syst. Rev. 2017 doi: 10.1002/14651858.CD011479.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Riper H, et al. Effectiveness and treatment moderators of internet interventions for adult problem drinking: an individual patient data meta-analysis of 19 randomised controlled trials. PLoS Med. 2018;15:e1002714. doi: 10.1371/journal.pmed.1002714. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Garnett C, et al. Evaluating the effectiveness of the smartphone app, Drink Less, compared with the NHS alcohol advice webpage, for the reduction of alcohol consumption among hazardous and harmful adult drinkers in the UK at 6‐month follow‐up: protocol for a randomised con. Addiction. 2020 doi: 10.1111/add.15287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Oldham M, et al. Effectiveness of a smartphone app (Drink Less) versus usual digital care for reducing alcohol consumption among increasing-and-higher-risk adult drinkers in the UK: a two-arm, parallel-group, double-blind, randomised controlled trial. eClinicalMedicine. 2024;70:102534. doi: 10.1016/j.eclinm.2024.102534. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Carey RN, et al. Behavior change techniques and their mechanisms of action: a synthesis of links described in published intervention literature. Ann. Behav. Med. 2019;53:693–707. doi: 10.1093/abm/kay078. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Moore GF, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. doi: 10.1136/bmj.h1258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement. Sci. 2011;6:42. doi: 10.1186/1748-5908-6-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Garnett C, Crane D, West R, Brown J, Michie S. The development of Drink Less: an alcohol reduction smartphone app for individuals who drink excessively. Transl. Behav. Med. 2019;9:296–307. doi: 10.1093/tbm/iby043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Garnett C, et al. Refining the content and design of an alcohol reduction app, Drink Less, to improve its usability and effectiveness: a mixed methods approach. F1000Res. 2021;10:511. doi: 10.12688/f1000research.51416.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Oldham M, et al. Evaluating the acceptability of the Drink Less app and the NHS alcohol advice webpage: a qualitative interviews process evaluation. JMIR Prepr. 2023 doi: 10.2196/preprints.42319. [DOI] [Google Scholar]
  • 14.Perski O, Crane D, Beard E, Brown J. Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digit. Health. 2019;5:2055207619880676. doi: 10.1177/2055207619880676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Purkayastha S, Addepally SA, Bucher S. Engagement and usability of a cognitive behavioral therapy mobile app compared with web-based cognitive behavioral therapy among college students: randomized heuristic trial. JMIR Hum. Factors. 2020;7:e14146. doi: 10.2196/14146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Garnett C, et al. Predictors of engagement, response to follow up, and extent of alcohol reduction in users of a smartphone app (Drink Less): secondary analysis of a factorial randomized controlled trial. JMIR Mhealth Uhealth. 2018;6:e11175. doi: 10.2196/11175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Bell L, et al. Engagement with a behavior change app for alcohol reduction: data visualization for longitudinal observational study. J. Med. Internet Res. 2020;22:e23369. doi: 10.2196/23369. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Milne-Ives M, Homer SR, Andrade J, Meinert E. Potential associations between behavior change techniques and engagement with mobile health apps: a systematic review. Front. Psychol. 2023;14:1227443. doi: 10.3389/fpsyg.2023.1227443. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Teasdale N, et al. Systematic review and meta-analysis of remotely delivered interventions using self-monitoring or tailored feedback to change dietary behavior. Am. J. Clin. Nutr. 2018;107:247–256. doi: 10.1093/ajcn/nqx048. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Neal DJ, Carey KB. A follow-up psychometric analysis of the self-regulation questionnaire. Psychol. Addict. Behav. 2005;19:414–422. doi: 10.1037/0893-164X.19.4.414. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Hustad JTP, Carey KB, Carey MP, Maisto SA. Self-regulation, alcohol consumption, and consequences in college student heavy drinkers: a simultaneous latent growth analysis. J. Stud. Alcohol Drugs. 2008;70:373–382. doi: 10.15288/jsad.2009.70.373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Roos CR, Witkiewitz K. A contextual model of self-regulation change mechanisms among individuals with addictive disorders. Clin. Psychol. Rev. 2017;57:117–128. doi: 10.1016/j.cpr.2017.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Stevely, A., Holmes, J., Garnett, C., Jones, A. & Oldham, M. Optimising measurement of information on the context of alcohol consumption within the Drink Less App amongst people drinking at increasing and higher risk levels: a mixed-methods usability study. Under review.
  • 24.Perski O, et al. Do daily fluctuations in psychological and app-related variables predict engagement with an alcohol reduction app? A series of N-of-1 studies. JMIR Mhealth Uhealth. 2019;7:e14098. doi: 10.2196/14098. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Ou L, Hunter MD, Chow SM. What’s for dynr: a package for linear and nonlinear dynamic modeling in R. R. J. 2019;11:91–111. doi: 10.32614/RJ-2019-012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Gunzler D, Chen T, Wu P, Zhang H. Introduction to mediation analysis with structural equation modeling. Shanghai Arch. Psychiatry. 2013;25:390–394. doi: 10.3969/j.issn.1002-0829.2013.06.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.West, R. Drink Less on iTunes store. https://itunes.apple.com/gb/app/drink-less-get-help-reducing/id1020579244?mt=8 (2016).
  • 28.NHS. Alcohol support—tips on cutting down. https://www.nhs.uk/live-well/alcohol-support/tips-on-cutting-down-alcohol/ (2019).
  • 29.Bradley KA, et al. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin. Exp. Res. 2007;31:1208–1217. doi: 10.1111/j.1530-0277.2007.00403.x. [DOI] [PubMed] [Google Scholar]
  • 30.Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl. Behav. Med. 2016;7:254–267. doi: 10.1007/s13142-016-0453-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Google Analytics. How a web session is defined in Analytics. https://support.google.com/analytics/answer/2731565?hl=en (2017).
  • 32.de Vocht F, et al. Temporal patterns of alcohol consumption and attempts to reduce alcohol intake in England. BMC Public Health. 2016;16:917. doi: 10.1186/s12889-016-3542-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Kotz D, Brown J, West R. Predictive validity of the Motivation To Stop Scale (MTSS): a single-item measure of motivation to stop smoking. Drug Alcohol Depend. 2013;128:15–19. doi: 10.1016/j.drugalcdep.2012.07.012. [DOI] [PubMed] [Google Scholar]
  • 34.Georgiades C, West R. Time spent with urges to smoke and strength of urges as independent predictors of self-rated difficulty not smoking during abstinence. J. Smok. Cessat. 2009;4:48–51. doi: 10.1375/jsc.4.1.48. [DOI] [Google Scholar]
  • 35.Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev. Sci. 2007;8:206–213. doi: 10.1007/s11121-007-0070-9. [DOI] [PubMed] [Google Scholar]
  • 36.Rubin, D. Multiple Imputation for Nonresponse in Surveys (John Wiley & Sons, 2004).
  • 37.Tingley, D., Yamamoto, T., Hirose, K., Keele, L. & Imai, K. mediation: R package for causal mediation analysis. J. Stat. Softw.59, 1–38 (2014).

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information (386.8KB, pdf)
Reporting summary (1.9MB, pdf)

Data Availability Statement

The anonymised data and data dictionary are available online at OSF (https://osf.io/2j9df/).

The anonymised code is available online at OSF (https://osf.io/2j9df/).

Articles from NPJ Digital Medicine are provided here courtesy of Nature Publishing Group

RESOURCES