Table 3.

Summary of usability.

Study [reference]; app name	Usability measures	Usability
Aharonovich et al, 2017 [30]; HealthCall	Engagement—proportion of days used out of total possible days, and satisfaction—7 items rated 1 (low) to 5 (high).	Engagement: 95% (range 68.7%-100%) of the possible 60 days. Satisfaction: mean 4.5 (SD 0.8).
Baskerville et al, 2018 [31]; Crush the Crave	Four satisfaction items (used frequently, easy to use, well laid out, and confidence in using) measured on a 5-point Likert scale (“strongly agree” to “strongly disagree”), overall satisfaction item using same scale as above, an overall helpfulness item on a 10-point Likert scale.	Mean (SD): Used frequently: 3.6 (1.2); Easy to use: 2.3 (1.1); Well laid out: 2.5 (1.1); Confidence in using: 2.8 (1.1); Overall satisfaction: 2.6 (1.3); Overall helpfulness; 4.3 (2.7).
Boendermaker et al, 2016 [32]; Alcohol/Avoid	User experience measured on 5-point Likert scale, 1 (strongly disagree) to 5 (strongly agree).	Mean (SD): Ease of use:15.77 (2.11); Player enjoyment:13.19 (2.98); Player involvement Mean (SD) 11.23 (2.13); Task compliance:6.07 (1.53).
Bricker et al, 2014 [33]; SmartQuit	Treatment satisfaction measured on 5-point scale, 1 (not at all) to 5 (very much), and utilization: self-reported number of times opened app.	85% said app organized; 53% said app useful for quitting; 59% were satisfied overall; app mean use 37 times over 8 weeks (no prompts).
Crane et al, 2018 [34]; Drink Less	Four usability measures rated on a 5-point scale, 1 (not at all) to 5 (extremely).	Mean (SD): Helpfulness: NFa: 3.05 (0.88); CBb: 3.02 (0.98); MFc: 3.18 (0.93); APd: 3.04 (1.02); ICe:3.09 (0.97). Ease of use: NF: 3.45 (0.97); CB: 3.45 (0.97); MF: 3.59 (1.00); AP: 3.56 (1.07); IC: 3.57 (1.00). Recommend: NF: 2.99 (1.23); CB: 2.91 (1.23); MF: 3.25 (1.22); AP: 3.08 (1.23); IC: 3.15 (1.16) . Satisfaction: NF: 3.22 (0.95); CB: 3.20 (0.97); MF: 3.36 (1.00); AP: 3.26 (1.00); IC: 3.25 (0.95).
Dulin et al, 2014 [50]; LBMI-A	Helpfulness and ease of use of each tool rated on a 7-point scale, 1 (extremely unhelpful or extremely easy to use) to 7 (extremely helpful or extremely difficult to use).	Mean (SD): Ease of use: 5.6 (1.7), with the Drink Monitor Tool being the easiest to use, 6.6 (1.8) and the High-Risk Location Tool being the most difficult, 4.3 (1.6). Helpfulness: High-Risk Location Tool was least helpful, 3.8 (2.2) and the Daily Interview Tool was most helpful, 6.1 (1.1).
Gajecki et al, 2014 [38]; PartyPlanner	Self-reported app usage and questions on ease of use, suitability, and likelihood of recommending to a friend, on a 5-point scale.	Mean (SD): Self-reported usage (any): PPf, 41.4%; PKg, 74.1%. Ease of use: PP: 3.2 (1.1); PK: 4.0 (1.1). Suitability: PP: 3.6 (1.2); PK 3.4 (1.2). Would recommend: PP: 3.6 (1.2); PK 3.7 (1.3).
Hasin et al, 2014 [41]; HealthCall	Twelve satisfaction questions on a 3-point scale, assessing feelings of safety and privacy, effects on recall and knowledge of own drinking patterns, motivation and self-confidence to reduce consumption, and app’s ability to prompt drinking goals.	Intervention group (“agree”): perceived responses as safe (94.59%), concerned about privacy (37.84%), liked using app (91.89%), graphs increased interest in app (86.49%), and graphs increased perceived benefit of app (91.89%). Of the 30 daily suggestions for cutting down drinking, 13 were rated as “helpful”/“very helpful” by over half the patients.
Hides et al, 2018 [43]; Ray’s Night Out	Mobile Application Rating Scale (5-point rating scale, 23 items), assessing engagement, functionality, aesthetics, and information quality.	Mean (SD): The MARSh indicated the app had a good level of overall app quality: 3.82 (0.51); Functionality: 3.98 (0.69); Aesthetics: 4.03 (0.62); Information: 4.0 (0.56). Participants reported that they were unlikely to pay for the app:1.25, (0.69) and gave it a 3 out of 5-star rating: 3.13 (0.76).
Liang et al, 2018 [46]; S-Health	Seven usability questions (5-point scale), assessing ease of use, recall feasibility, willingness to provide responses, etc.	Intervention group (“strongly agree” or “agree”): “The survey questions were easy to understand” (55.3%); “I was comfortable answering these questions” (68.1%); “I was able to remember the number of days or frequency using alcohol or drugs in the past week” (53.2%); “The smartphone screen was easy to use” (72.3%); “I prefer to answer these questions myself on a cellphone instead of having a person ask me” (46.8%).
McTavish et al, 2012 [51]; A-CHESS	Passive app use data: which service selected; duration of use for each service; which pages viewed; messages sent or received.	93.5% accessed the system during the first week after leaving treatment. The A-CHESS services used by the greatest percentage of participants included discussions, my messages, my team, and weekly surveys. The least used services were instant library, frequently asked questions, and web links.
Pocuca et al, 2016 [52]; Ray’s Night Out	Mobile Application Rating Scale -youth version (5-point rating scale, 23 items), assessing engagement, functionality, aesthetics, and information quality.	Mean (SD): Entertaining: 3.78 (SD 0.83); Interesting: 3.67 (0.71); Customizable: 3.00 (0.58); Interactive: 2.63 (0.74); Speed and accuracy of function: 4.78 (0.44); Ease of use: 4.44 (0.73); Flow and logic: 4.33 (0.50); Layout design: 4.33 (0.50); graphics quality 4.56 (0.73); quality and credible information 3.71 (0.58); would you recommend 2.78 (1.09); overall star rating 3.11 (0.60).

^aNF: personalized normative feedback.

^bCB: cognitive bias retraining.

^cMF: monitoring and feedback.

^dAP: action planning.

^eIC: identity change.

^fPP: PartyPlanner app.

^hMARS: Mobile App Rating Scale.

^gPK: Promillekoll app.