Table 5.

Linking visualization to the design of a micro-randomized trial.

What we learnt from these analyses	Which visualization or analyses showed us this	How this informed the design of our randomized trial
The present notification appears to be a key driver of engagement	Figure 1, plot A: heat map of total sessions. Figure 1, plot B: heat map of total time on app (hours) Table 4: estimated risk ratio with 95% CI for the associations between exposure to the notification and app use within each cluster, at 3 time points (days 1, 7, and 30)	We chose to undertake a micro-randomized trial to both understand the causal effect of the notification on engagement, and to further optimize the delivery of notifications with respect to time-varying covariates, notifications, and outcomes
The impact of the notification seems to be strongest in the hour preceding delivery	Figure 1, plot A: heat map of total sessions	We set the time window to measure the proximal (ie, near-term) effect as 1 hour after delivery
Evenings seem to be an opportune and acceptable moment to engage with Drink Less. It is also a time of increased vulnerability to excess drinking	Figure 1, plot B: heat map of total time on app (hours)	We moved the delivery time of the notification to 8 PM
The notification may encourage the reporting of alcohol-free days more than drink consumed. This may be due to competing pressures for time at 11 AM	Figure 3: frequency distributions of when alcohol-free days and alcohol units are recorded	We intervened in the evenings to see if this is a more acceptable and opportune time to report drinks consumed
The notification may reduce the median time per session during the reminder of the day	Figure 2: line plot of median time spent on app (seconds)	We included a no-notification arm in our trial to capture a momentary assessment of engagement when no notifications are sent
The depth of engagement with Drink Less is low	Multimedia Appendix 2: summaries of use by module for all users	We trialed new notifications which target the perceived usefulness of Drink Less to encourage broader engagement
Slow disengagers (3679/19,233, 19.13%) have a high probability of engagement during the first week, but by day 30, this group has a low probability, suggesting a loss of motivation	Figure 4: probability of use on day after download by cluster group	We tested 30 new messages to increase novelty and motivation to remain engaged with Drink Less (Multimedia Appendix 8)
Exogenous impacts, such as public health campaigns, are likely to influence the cohort of users over time	Figure 6: time to disengagement (defined as the first day of 7 or more consecutive days of no use) by the engagement cluster	We included a standard app version arm in the trial, to provide an exchangeable sample to compare the fixed and random notification policies