Table 2.

User engagement indicators.

Authors/Year	User engagement strategies	Factors	Measures	Main findings
Achtyes et al. (26)	Notifications/prompts	Age Gender Education level Technical support	Demographic variables were assessed for each participant Data about app usage (each time a participant accessed FOCUS, the date and time-stamped login was recorded within the tool)	Subjects in this study accessed FOCUS half the time (38,139 times, 50.6%) during off-hours, indicating that approximately half the “dose” of this intervention occurred when the clinic was closed, an important extension of care during times when access to a clinician may be difficult or impossible A subset of “high utilizers” (n = 152, 43.8%) self-initiated use of all five FOCUS modules both on- and off-hours “High utilizers” tended to be women, >35 years old, and had a high school diploma or greater Patients older than 35 years engaged well with FOCUS when given ongoing technical support from the mental health technology coach
Ben-Zeev et al. (27)	Notifications/prompts	30-min training session (demonstration focused on in-the-moment use and selection of resources from the different on-demand options. Users then had the opportunity to practice using FOCUS and ask questions as needed)	Demographic and clinical variables were assessed for each participant [Positive and Negative Syndrome Scale (PANSS) for Schizophrenia; Beck Depression Inventory, Insomnia Severity Index, Brief Assessment of Cognition in Schizophrenia, Brief Medication Questionnaire] 26-item self-report acceptability/usability measure comprised of adapted items from the System Usability Scale Post Study System Usability Questionnaire Technology Assessment Model Measurement Scales Usefulness, Satisfaction, and Ease questionnaire Participants were asked to rate their agreement with a series of statements about the intervention	Completers used the system on 86.5% of days they had the device, an average of 5.2 times a day Approximately 62% of use of the FOCUS intervention was initiated by the participants, and 38% of use was in response to automated prompts Approximately 90% of participants rated the intervention as highly acceptable and usable
Ben-Zeev et al. (36)	Notifications/prompts	Study staff call participants if it appears that there is a problem with passive data collection	Demographic variables and comprehensive baseline assessment of symptoms and functioning (semi-structured clinical interviews and questionnaires) App data (active and passive user assessments)	Participants had unique digital indicators of their psychotic relapse. For some, self-reports provided clear and potentially actionable description of symptom exacerbation prior to hospitalization. Others had behavioral sensing data trends (e.g., shifts in geolocation patterns, declines in physical activity) or device use patterns (e.g., increased nighttime app use, discontinuation of all smartphone use) that reflected the changes they experienced more effectively Not all participants in the ongoing CrossCheck study complete a full year of data collection, adhere to the study protocol (i.e., carry the smartphone with them, charge the battery, complete self-reports regularly), or experience hospitalizations
Ben-Zeev et al. (28)	Notifications/prompts	Brief weekly calls from an mHealth support specialist who assisted them in all technical and clinical aspects of the intervention	Demographic and clinical variables were assessed for each participant (Symptom Checklist, Beck Depression Inventory, Psychotic Symptom Rating Scales, Recovery Assessment Scale) Satisfaction 5-item questionnaire Data about app usage (engagement in treatment was calculated for each participant weekly by using his or her FOCUS use data; participants were considered engaged if they used the app on at least five of 7 days a week (that is, approximately 70%). A FOCUS “use” event is recorded as such only if, following a prompt, participants elect to engage in a clinical status assessment or if they self-initiate one of the FOCUS on-demand tools)	Averaging across all participants assigned to FOCUS, participants used the app on 5.4 ± 2.4 days in the first week of the intervention, on 4.6 ± 2.7 days in the third week, on 4.3 ± 2.7 days in the sixth week, on 3.9 ± 2.7 days in the ninth week, and on 3.8 ± 2.9 days in the last week Satisfaction was high
Ben-Zeev et al. (29)	Notifications/prompts Use of videos	Concerns about privacy Apprehensive about using video interventions in public for fear of public stigmatization	Demographic and clinical variables were assessed for each participant (Psychotic Symptom Rating Scales, Beck Depression Inventory) Semi-structured interview about their experiences with the video and written interventions 11-item measure in which they were asked to indicate their intervention modality preference 12-item measure examining usability, acceptability, and satisfaction	On average, participants found videos to be more personal, engaging, and helpful in supporting their illness management. Participants responded to 67% of system-delivered prompts to engage FOCUS-AV, and 52% of FOCUS-AV use was initiated by the users. On average, participants used interventions 6 days a week, four times daily. Participants used video functions an average of 28 times. They chose video over written interventions on 67% of the times they used on-demand functions but opted for written content 78% of the times they responded to pre-scheduled prompts Written interventions were rated as more favorable in letting users proceed at their own pace
Bonet et al. (41)	Notifications/prompts Clinicians contact patients by phone in response to preset alarms	Clinicians’ workload Limited efforts to integrate these apps into care settings Excessive eHealth communications could be regarded as intrusive or irritating	Demographic and clinical variables were assessed for each participant (Clinical Global Impression Severity of Illness scale, Global Assessment of Functioning, Positive and Negative Syndrome Scale (PANSS), Premorbid Adjustment Scale) Feasibility, efficacy, compliance, and engagement measures: number of times patients answered the questionnaires when presented and number of months using the app, patients’ dropouts, plus number of urgent consultation requests	Only 20% (12/59) of patients from the ReMindCare group had a relapse, while 58% (18/31) of the TAU patients had one or more relapses (χ2 = 13.7, P = .001) Moreover, ReMindCare patients had fewer visits to urgent care units (χ2 = 7.4, P = .006) and fewer hospitalizations than TAU patients (χ2 = 4.6, P = .03). The mean of days using the app was 352.2 (SD 191.2; min/max: 18–594), and the mean of engagement was 84.5 (SD 16.04)
Bucci et al. (37)	Notifications/prompts Participants could view written and visual “in-app” instructions Multiple messages and images associated with each exchange minimize boredom and repetition within the app Users can customize the aesthetics of the Actissist interface	45-min phone set-up training session All participants received a weekly phone call from the project manager to troubleshoot equipment functions	The primary outcome was feasibility, which was assessed in terms of uptake (the proportion of eligible participants consenting to the study), attrition, the proportion of participants completing user, and alert-initiated data entries across participants (>33% data points), and the proportion continuing for 12 weeks (both arms) Satisfaction with technology was also measured Acceptability of the Actissist intervention was assessed via participant feedback Other secondary outcome measures related to functioning and symptom were assessed	Actissist was feasible (75% participants used Actissist at least once/day; uptake was high, 97% participants remained in the trial; high follow-up rates), acceptable (90% participants recommend Actissist), and safe (0 serious adverse events), with high levels of user satisfaction. Treatment effects were large on negative symptoms, general psychotic symptoms and mood. The addition of Actissist conferred benefit at post-treatment assessment over routine symptom-monitoring and treatment as usual
Eisner et al. (39)	Notifications/prompts (above-threshold increases in app-reported psychotic symptoms prompted a telephone interview to assess relapse)	Participants were telephoned by the researcher (weekly for 4 weeks; monthly thereafter) to encourage participation and troubleshoot any difficulties with app use	Demographic and clinical variables were assessed for each participant (Schizophrenia Proneness Instrument Adult Version interview, Fear of Recurrence Scale, Hospital Anxiety and Depression Scale, Psychotic Symptom Rating Scales) Qualitative interviews App engagement was measure through percentage of assessments completed	Participants completed 65% app assessments and 58% telephone interviews. App items showed high concurrent validity with researcher-rated psychotic symptoms and basic symptoms over 6 months. There was excellent agreement between telephone call and face-to-face assessed psychotic symptoms. The primary relapse definition, based on telephone assessment and case notes, compared well with a case note-only definition but had better specificity. Mixed-effects models provided preliminary evidence of concurrent and predictive validity: early signs and basic symptoms were associated with most app-assessed psychotic symptom variables the same week and with psychotic symptoms variables 3 weeks later; adding basic symptoms to early signs improved model fit in most of these cases
Fulford et al. (48)	Goal setting Push notifications administered three times per day (morning, mid-day, and evening) directed users to app content focused on their identified social goal and steps Provide more specific structure/guidance on steps to achieve social goals, and increasing in-person check-ins during app use Use of videos	Support for technical problems with the smartphone/app	Clinician interviews Focus groups with patients Usability test: quantitative and qualitative feedback to inform subsequent revision of the app through an online survey, regarding their experience with the app, including discussing barriers and facilitators to using the app and any additional feedback to be implemented in a future iteration of the app	Preliminary evidence of acceptability and feasibility of the MASS app is promising given the limited available mobile apps focused on addressing social skills and motivation specifically through a standalone app Key features that were identified by participants as helpful included the video content, the reminders of steps necessary to complete their goal, and the ease of using the app. Some notable suggestions for improving the app experience included decreasing the number of daily notifications, adding more video content, providing more specific structure/guidance on social goals, and increasing in-person check-ins during app use
Garety et al. (43)	Use interactive features including information, animated vignettes, games, and personalized thought bubbles Notifications/prompts	Inclusive, human-centered design	Demographic and clinical variables were assessed for each participant (Paranoid Thoughts Questionnaire, Psychotic Symptom Rating Scales, Maudsley Assessment of Delusions Schedule, Fast and Slow Thinking Questionnaire, Warwick-Edinburgh Mental Well-being Scale, Manchester Short Assessment of Quality of Life, Brief Core Schema Scales, Penn State Worry Questionnaire) Semi structured interviews investigated the feasibility and potential acceptability of the intervention Mobile app adherence was operationalized as at least 1 home screen interaction after a minimum of 3 therapy sessions and was recorded by system analytics	The mean (SD) number of SlowMo sessions attended was 6.8 (2.6), increasing to 7.3 (1.9) for those attending 1 or more sessions. Among the 181 participants in the SlowMo arm, 145 (80.1%) completed all 8 therapy sessions, 13 (7.2%) attended no sessions, and 23 (12.7%) discontinued therapy between sessions 1 and 7. Mean (SD) session duration, including behavioral work, was 75 (29) minutes. SlowMo plus TAU was not associated with greater reductions than TAU alone in the primary outcome of GPTS total paranoia score at 24 weeks (Cohen d, 0.20; 95% CI, −0.02 to 0.40; P = .06)
Gire et al. (44)	Notifications/prompts Use of multimedia such as music, images, or videos If the App detected low mood/paranoia, participants were offered tailored interventions; when symptoms are exacerbated causing severe distress, the crisis response may include contacting the early intervention services or an agreed designated contact	Financial implications of mobile ownership and connectivity to the internet Confidentiality and security	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS), Psychotic Symptom Rating Scales) The success criterion for feasibility was the recruitment of ≥50% of eligible participants the acceptability of the intervention was assessed based on the amount of engagement and usage of the TechCare App, with the success criterion for compliance being set at ≥33%	A total of 83.33% (n = 10) of participants completed the 6-week feasibility study, with 70% of completers achieving the set compliance threshold of ≥33% engagement with the TechCare App system. Analysis of the qualitative data suggested that participants held the view that the TechCare was both an acceptable and feasible means of delivering interventions in real-time
Hanssen et al. (30)	The application was personalized for all participants, both with and without feedback, according to the personal preferences of the participant—they filled in enjoyable activities, several social contacts, comforting thoughts, and relaxing activities Notifications/prompts	Some participants indicated that there were too many beeps during the day and that they sometimes felt disturbed in their activities by the beep On day 2 and day 7 participants were contacted by phone to check for technical difficulties and whether they had any additional questions. A contact number was provided for technical support	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS), Community Assessment of Psychic-Experiences, Social Functioning Scale, and two subtests of the Wechsler Adult Intelligence Scale) Six short Experience Sampling Method (ESM) questionnaires daily when prompted by a beep, for a duration of 3 weeks and an evening questionnaire Evaluation questionnaire of the app	The response rate was 64% for the ESM questionnaires. In the feedback group, participants indicated that on 49% of the ESM days they acted on at least one personalized feedback prompt per day. Momentary psychotic symptoms significantly decreased over time only in the feedback group. Momentary loneliness and questionnaire-assessed psychotic symptoms decreased over time, irrespective of feedback. Participants rated the SMARTapp as easy to use (94%) and appealing (95%), indicated that questions were clear (80%), and generally felt that they could reflect their experiences well through the questions provided by the application (68%). Seventy-four percent of the participants said they used the coping tips, and 54% found them useful (43% neutral, 3% not useful)
Kidd et al. (35)	Notifications/prompts Content customization regarding daily wellness and goal attainment check-ins to inform content delivery and highlight mental health trajectories Medication, appointments, and event reminders are custom set by the user during onboarding and can be added throughout app use A peer-peer engagement platform that facilitates strategy/tip-sharing between A4i users (anonymous and moderated) Passive collect data on phone use as a proxy for sleep and activity levels	Illness severity Gender Age	Demographic and clinical variables were assessed for each participant (Brief Symptom Inventory, Personal Recovery Outcome Measure, Brief Adherence Rating Scale) The mHealth use and utility scale used by Ben-Zeev and colleagues was employed with minor modification to specify A4i use Qualitative data included the use of field notes to capture information gathered in weekly check-in contacts and, at post assessment, a semi-structured interview was used to capture what was more and less helpful/engaging in participants’ experiences of using A4i Metrics collected through the app included overall time used, time and frequency of using specific functions, frequency of app refresh, a sleep-proxy metric, and daily wellness check-in responses. Descriptive data analysis included a descriptive profile of participants, their app use, and ratings of app utility	Among the 38 individuals with a primary psychosis who participated, there was no research attrition and classic retention on the app was 52.5%. Significant improvement was observed in some psychiatric symptom domains with small-medium effects. Significant change in recovery engagement and medication adherence were not observed after controlling for multiple comparisons. Those who interacted with the app more frequently were more depressed and had higher hostility and interpersonal sensitivity at baseline. Satisfaction with the app was high and qualitative feedback provided insights regarding feature enhancements
Kreyenbuhl et al. (51)	Notifications/prompts Personalized motivational feedback Use of gamification	Informed by the Information-Motivation-Behavioral (IMB) Skills Model and using the iterative process of user-centered design Possibility to integrate the interface information into a patient's medical records (more work is needed to understand how to better enable service systems and providers to integrate real-time patient information into existing workflows to facilitate measurement-based care)	During the 2-week study period, all patient-participant use of the application (i.e., button presses, responses EMAs) and psychiatrist-participant use of the clinician interface were continuously uploaded to a secure study server At follow-up, psychiatrist- and patient-participants completed surveys querying them about the acceptability and likeability of the application and clinician interface, respectively The primary outcomes of the open trial were the feasibility and acceptability of the application and clinician interface	MedActive was determined to be both feasible and acceptable, with patient participants responding to 80% of all scheduled EMAs and providing positive evaluations of their use of the application. Psychiatrist participants were interested in viewing the information provided on the MedActive clinician interface but cited practical barriers to regularly accessing it and integrating into their daily practice
Krzystanek et al. (47)	Notifications/prompts Software also contained a library of videos and audiobooks accessible to the patients at will	Patients were able to report the need for a televisit Age Illness severity Reduced cognitive functions	The clinical status was measured using the Positive and Negative Syndrome Scale (PANSS), Calgary Depression Scale for Schizophrenia, and Clinical Global Impression-Severity clinical scales The number of hospitalizations and visits to outpatient clinics was gathered directly from the MONEO platform Medical adherence was monitored based on feedback messages sent by the patient	Adherence data gathered for the 12-month study on the MONEO platform ranged from 61% to 85% Patients using the full version of the MONEO platform exhibited a pronounced reduction in the schizophrenic symptoms Use of the MONEO platform did not influence the rate of hospitalization and visits to outpatient clinics
Lewis et al. (38)	Notifications/prompts	Illness severity	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS)) Qualitative analyses (interviews) assessed acceptability of the system	Participant's adherence rate (defined as responding to >33% of alerts) was 84% Of 38 participants who completed 12 weeks of the trial, three (8%) reported significant events: 1 reported increased anxiety prompted by questions; 1 reported increased irritation due to the alert beeps, and 1 had their charger explode Qualitative analyses supported the acceptability of the system to participants and staff App use was associated with psychotic symptom improvement in recent-onset participants, but not those with longstanding illness, supporting the notion of improved self-management
Lim et al. (52)	Use of gamification (e.g., points, challenges, badges) to encourage participant engagement Personalization Relatability of content In-app feedback functions Use of video Youth-friendly design and navigation	Age User participation in design Fits with routine	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS), Calgary Depression Scale for Schizophrenia, Social Skills Performance Assessment, Revised UCLA Loneliness Scale, Social Interaction Anxiety Scale, Scales of Psychological Well-being) A series of questionnaires were created to assess acceptability, feasibility, and usability A 20-item questionnaire designed for the study was used to assess how helpful each module was for the participants	+Connect helped participants to increase their social confidence, enjoy life, look forward to being with other people, and feel more connected with others Participant interviews supported these results, with participants highlighting the app's strengths in providing useful information, stimulating self-reflection, fostering positive affect, and encouraging transfer of skills into their social interactions Participants completed 95.47% of the +Connect exceeding the a priori criteria of app completion (33 out of 42 days). Eight out of the ten participants also remained engaged in the program 3 months following the end of treatment assessment. Of the ten participants, two reported early difficulties integrating the use of the app into their routine Preliminary findings indicated that +Connect yielded high levels of acceptability and feasibility
Moitra et al. (31)	Notifications/prompts	Lack of technology fluency Access to Internet Technical support	Demographic and clinical variables were assessed for each participant (Antipsychotic Medication Beliefs and Attitudes Scale, Brief Adherence Rating Scale, Brief Coping Orientation to Problems Experienced, Brief Psychiatric Rating Scale, World Health Organization Disability Assessment Schedule 2.0) System usability scale, Client Satisfaction Questionnaire-8, and the Usefulness, Satisfaction and Ease of Use Questionnaire	Participants completed about one session per day on average as expected. Overall, measures of MACS usability and satisfaction were positive Participant feedback was mostly positive as many noted the benefits of being prompted to reflect on their symptoms and functioning, as well as in receiving brief CBTp-based support
Moore et al. (45)	Notifications/prompts Personalization	Unexpected technology issues Therapeutic relationship—app as an extension of the therapist	Semi-structured interview	Participants perceived Ecological Momentary Assessment and Intervention (EMA/I) technology as helping capture their experience more accurately and communicate this more effectively to the therapist, which, in combination with coping prompts developed in-session, deepened the therapeutic relationship
Niendam et al. (40)	Active and passive data collection Dashboard notifications appeared when participants did not complete surveys more than 3 days in a row, or when passive data were not being collected from a participants’ telephone	Monetary incentives (participants were paid $0.50 per daily and $1.50 per weekly survey, i.e., maximum $20 monthly payment) Device breakage/loss	Demographic and clinical variables were assessed for each participant (Brief Psychiatric Rating Scale) Participants completed self-report surveys at the end of the study evaluating satisfaction and perceived impact on clinical care	High survey completion rates (average 77% weekly; 69% daily) were not related to symptom severity or length of time in treatment at the clinic, indicating that even symptomatic individuals early on in treatment are able and willing to engage in smartphone-based surveys as part of their treatment 97% of participants reported that the app was easy to use 40% of participants indicated the app helped them remember to take their medication and increased motivation for symptom management and treatment engagement. This suggests that, for some individuals, simply tracking symptoms and medication adherence via an app may be an intervention in and of itself
Palmier-Claus et al. (55)	Notifications/prompts	Fits with routine Illness severity Repetitiveness of the questions Socioeconomic status Reading ability Researcher support (phone calls)	Demographic variables were assessed for each participant Qualitative interviews were conducted to explore participants’ perceptions and experiences of the devices, and thematic analysis was used to analyze the data. Three themes emerged from the data: (i) the appeal of usability and familiarity, (ii) acceptability, validity, and integration into domestic routines, and (iii) perceived impact on clinical care	Compliance as defined by completion of at least 33% of all possible data-points over 7 days was 82%. ClinTouch is a valid form of self-assessment, which could facilitate the real-time monitoring of symptoms in schizophrenia in research and clinical management settings. In addition to overcoming the constraints of rater training and limited reliability, recall bias and averaging, it potentially offers advantages over semi-structured interview administered scales allowing finer-grained analysis over briefer time periods, with potential inclusion of external contingency data, diurnal and short-term variability and adding in of other behavioral data gathered by the same device, such as sleep pattern and activity
Schlosser et al. (46)	Motivation coaches Use of gamification (challenges) Goal setting Peer-peer community (users may send messages directly to each other and can also capture and share positive, spontaneous moments in their daily life with the whole PRIME community short bio) Personalization	Technical support	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS), Trust Task, Motivation and Pleasure Self-Report Scale, Role Functioning Scale, Quality of Life Scale—Abbreviated, Dysfunctional Attitudes Scale, Beck Depression Inventory, Revised Self-Efficacy Scale) PRIME acceptability was assessed during an exit interview at the 12-week time point (post-trial) where participants rated their satisfaction with the specific features of PRIME. To evaluate feasibility, the following metrics were used: login frequency (average number of days logged in per week), average number of challenges completed (both overall and by individual challenge category), challenge completion percentage, and the average number of peer and coach interactions. To determine PRIME acceptability, the average ratings from the PRIME satisfaction survey were assessed. To investigate PRIME feasibility, descriptive statistics for the following PRIME metrics were examined: login frequency, challenges completed, spontaneous and goal achievement moments, peer and coach interactions, and active use rate	The overall 74% retention rate for the treatment (and 88% retention post-intervention), demonstrated that this intervention was very well tolerated. Participants rated their overall satisfaction with PRIME highly. On average, participants logged in a little over 4 day/week. Over a 12-week period, participants were highly engaged in the platform, with 5,152 direct messages sent from participants to coaches. In terms of peer-to-peer interactions, participants-initiated interactions with each other a total of 497 times. Participants initiated about 10 interactions with coaches for every initiated peer interaction. All 38 participants initiated at least 1 message to a coach and 13 (33%) initiated more than the average of 128.8 coach interactions. Participants completed an average of 1.5 challenges per week
Simões de Almeida et al. (32)	Goal setting Contact with therapists in crisis situations	Age Data security Onset of illness Context of care delivery (receiving ambulatory care) Illness severity Influence of clinicians User participation in design	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS), Recovery Assessment Scale, Empowerment Scale, General Self-Efficacy Scale, Social Support Satisfaction Scale, Personal and Social Performance Scale) A usability brief questionnaire was also applied, asking participants their ability to use de app, satisfaction, and difficulties	Concerning the usability, most participants (45%) used the app two or three times a week and recognize the “Anxiety Management” and “Goal Setting” modules and the possibility to exchange messages with the therapist the modules being most used by them. Participants showed that were satisfied with the weCope system, considering the application useful for the management of the disease (89%), and 78% said they would continue to use the app
Steare et al. (33)	Goal setting Notifications/prompts	Age Regular smartphone use patterns Technical difficulties Influence of clinicians (clinicians were asked to discuss recovery goals and relapse prevention plans in routine appointments with participants, and assist with entering these into the appropriate My Journey 3 sections)	Demographic and clinical variables were assessed for each participant (Positive and Negative Syndrome Scale (PANSS), Social Outcomes Questionnaire, Mental Health Confidence Scale, Questionnaire about the Process of Recovery, Warwick-Edinburgh Mental Well-being Scale) DIALOG scale, Service Engagement Scale, app patient records and assessments through the app Semi-structured interview	83% and 75% of participants were retained in the trial at the 4- and 12-month assessments. All treatment group participants had access to My Journey 3 during the trial, but technical difficulties caused delays in ensuring timely access to the intervention. The median number of My Journey 3 uses was 16.5 (IQR 8.5–23) and median total minutes spent using My Journey 3 was 26.8 (IQR 18.3–57.3)
Steare et al. (34)	Communication with peers and clinicians Notifications/prompts	Age Data security and privacy Recovery stage Illness severity	Demographic variables were assessed for each participant DIALOG scale, Service Engagement Scale Clinical semi-structured interviews assess feasibility and acceptability of the intervention and barriers and facilitators to its use To assess acceptability of the intervention and user engagement, My Journey 3 usage data were collected for all participants in the treatment group from the training session until the 12-month time point	Many service user participants found My Journey 3 to be acceptable. The symptom and medication trackers were described as helpful. A smaller number of service users disliked the intervention. Clinicians tended to report that My Journey 3 was a potentially positive addition to service users’ care, but they often felt unable to provide support due to competing demands in their work, which in turn may have impacted acceptability and usage of the app
Taylor et al. (50)	Goal setting Supportive feedback Content was tailored to participant responses using branching logic Notifications/prompts	30 min therapist support weekly (included “technical”, “use” and “clinical” support to overcome any barriers related to engagement, understanding and implementation of strategies as required) Ease of remembering to use Fits with routine	Demographic and clinical variables were assessed for each participant (Pittsburgh Sleep Quality Index, Paranoid Thoughts Scale, Specific Psychotic Experiences Questionnaire, Depression, Anxiety and Stress Scale 21-itens, Warwick-Edinburg Mental Well-being Scale, Work and Social Adjustment Scale) Acceptability of the intervention was assessed by app engagement, an Experience Feedback Questionnaire and a semi-structured interview focused on the participant's experience of the app, barriers to engagement and their perception of the impact of the intervention	It is feasible and acceptable to deliver therapist-guided CBT for sleep problems by smartphone app for individuals with psychosis. On average, each participant engaged with 5.6 of 7 available modules. Engagement declined over time: all participants (n = 13; 100%) engaged with the week one modules, reducing to 61.5% (n = 8) by week 6. Responses to reminders similarly declined over time, starting at a mean of 44.6% (SD = 33.8%) completed in week 1, reducing to 29.5% (SD = 36.7%) at week 6. On average, participants engaged with 13.7 (34.3%; SD = 10.4) of the 40 reminders. Qualitative feedback indicated the intervention was considered helpful and would be recommended to others
Terp et al. (42)	Personalization Notifications/prompts	Illness severity User participation in design Fears and worries of restraint (surveillance/data sharing)	Semi-structured interviews	The findings demonstrate that young adults diagnosed with schizophrenia are amenable to use a smartphone app to monitor their health, manage their medication, and stay alert of the early signs of illness exacerbation. This may empower them to stay on track with their illness, thus in control of it. Participants in the evaluation described MindFrame as easy and intuitive to use. Some participants terminated use within 1 month (n = 5), others terminated use within 2–3 months (n = 4), and others used MindFrame for 6–12 months, terminating their use when the intervention period stopped (n = 4). Reasons given for self-initiated termination of MindFrame included boredom, lack of motivation and energy, fatigue, and problems quantifying their mental health