Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2026 May 1.
Published in final edited form as: Psychol Serv. 2024 Jun 6;22(2):221–231. doi: 10.1037/ser0000869

Feasibility and Acceptability of a Mobile Intervention for Patients with Psychosis Following Psychiatric Hospitalization: A Pilot Randomized Controlled Trial

Brandon A Gaudiano 1,2,3, Madeline Ward 1,2, Madeline B Benz 1,2, Christopher Hughes 1,2, Jennifer E Johnson 4, Ethan Moitra 2
PMCID: PMC11878221  NIHMSID: NIHMS2055438  PMID: 38842850

Abstract

This pilot randomized control trial (RCT) examines the feasibility and acceptability of a novel mHealth intervention for patients with schizophrenia-spectrum disorders (SSDs) following discharge from inpatient hospitalization. Using cognitive behavior therapy for psychosis (CBTp) strategies, the app provides just-in-time assessment and intervention for individuals to promote healthy coping skills and treatment adherence. We assessed the mHealth intervention relative to a comparison app that included mobile assessment plus psychoeducation alone. Patients were assessed at hospital discharge, as well as 1-, 2-, and 4-months post-discharge. Forty-two adults with SSDs discharging from inpatient care participated in the study. Our a priori-defined feasibility and acceptability goals were mostly achieved during the study, in terms of the proposed recruitment and retention rates, mHealth app engagement, app satisfaction ratings, clinical improvement observed over time, and absence of adverse events related to the study. Participants were significantly more engaged in the mHealth intervention (74%) vs the comparison app (43%). Over the course of the study, dysfunctional coping and psychiatric symptoms significantly declined in both groups. Future larger trials are needed to confirm the efficacy of the mHealth intervention.

Keywords: schizophrenia, cognitive behavioral therapy, mobile health, care transition, psychiatric hospitalization

1. Introduction

Schizophrenia-spectrum disorders (SSDs) are a form of serious mental illness responsible for over $343 billion dollars per year in treatment costs and lost wages across the United States (Kadakia et al., 2022). SSDs are associated with high rates of relapse (up to 82% within 5 years; Robinson et al., 1999) and rehospitalization (up to 64% within 5 years; Hudson, 2019), complicating treatment and resulting in frequent and disruptive transitions in care. A period of particularly high risk occurs in the transition from hospital to outpatient care, when patients often experience increased stress, suicidality, and difficulties with housing stability and treatment access (Pompili et al., 2007). In addition, high rates of treatment dropout and medication nonadherence often occur following hospital discharge (Kimhy et al., 2004; Olfson et al., 2000). The costs of rehospitalization in SSDs due to medication nonadherence alone is estimated at over $1.5 billion yearly (Wander, 2020). Despite this, there is a lack of available and effective transitional services supporting patients’ successful return to the community and long-term recovery (Velligan et al., 2016). Transitional interventions that are potentially cost-effective and efficient are needed to support treatment adherence and coping post-hospital discharge.

In response to the high relapse, dropout, and safety risks associated with the post-discharge period, mental health providers are increasingly exploring mobile health (mHealth; i.e., smartphones and related devices) interventions to augment (not replace) in-person services (Bidargaddi et al., 2020). mHealth is being used across a wide range of age groups and conditions to inconspicuously and efficiently collect behavioral, psychosocial, and physiological data in real-time, as well as provide in-the-moment intervention and crisis management in the person’s natural environment (Marzano et al., 2015). mHealth interventions show promise for use with the SSD population, as the technology is feasible and accessible to these patients, and rates of smartphone ownership are increasingly similar to the general public’s rate of smartphone ownership of over 85% (Young et al., 2020; Pew Research Center, 2021; Moitra et al., 2021; Naslund et al., 2015). Existing studies in patients with SSDs support the initial efficacy of mHealth related relapse prevention, medication adherence, coping, and symptom monitoring (Naslund et al., 2015). However, less work to date has been conducted testing mHealth during the critical transition from hospital to outpatient care for those with SSDs (Brunette et al., 2016).

Our research group recently created an mHealth intervention for patients with SSDs addressing the period immediately following inpatient care. The Mobile After-Care Support (MACS) smartphone application (“app”) monitors symptoms and functioning. It also uses cognitive behavior therapy for psychosis (CBTp) strategies to provide just-in-time interventions based on patients’ responses to support treatment adherence and foster healthy coping skills (Moitra et al., 2021). CBTp was chosen because it is readily adaptable to the mHealth format (Depp et al., 2018), promotes successful coping with psychosis (Schlier et al., 2020), and has demonstrated efficacy through increasing medication adherence, reducing symptom severity, and improving functioning (Farhall & Thomas, 2013; Jones et al., 2012; Mueser et al., 2013; Turner et al., 2020). Though CBTp is a first-line, evidence-based treatment recommended by the National Institute for Health and Care Excellence (NICE), research finds low rates of CBTp are offered and received in the year after hospital discharge (Jacobsen et al., 2023), further justifying the use of an app to address this treatment gap.

Our initial open trial (n = 10) demonstrated that MACS was a feasible, acceptable, and potentially effective intervention with individuals with SSDs when used immediately post-discharge as part of standard care (Moitra et al., 2021), supporting the need for further study. We also conducted a qualitative study of non-patient stakeholders (n = 18; administrators, support staff, clinicians) to inform the development of MACS. Stakeholders were supportive of mHealth services for patients with SSDs post-discharge but emphasized the need to appropriately tailor them to the population, integrate them with traditional care, and show added benefits for patients and providers that justified the costs involved (Moitra et al., 2022).

The primary aim of the current pilot randomized controlled trial (RCT) was to examine the feasibility and acceptability of MACS relative to a comparison condition. We randomized patients to MACS versus a psychoeducation app condition that contained mobile assessment and psychoeducational information alone without offering specific CBTp strategies. We collected longitudinal clinical outcomes by assessing patients’ symptom severity, functioning, medication adherence, and coping skills over 4 months. In line with recent calls to examine multiple harm indices in psychosis research (Ellett, 2023), we examined safety features of the MACS app. We hypothesized that MACS would be feasible and acceptable in this population in terms of demonstrating our ability to recruit, retain, and engage patients with SSD with MACS following hospital discharge (primary aim). Given the pilot nature of this trial, we were not powered to detect differences between the two treatment conditions. However, we expected that those receiving MACS would demonstrate significant improvements over time on our main outcomes (i.e., reduced symptom severity, increased treatment adherence, and improved coping skills) over 4-month follow-up (secondary aim). Lessons learned from the current and previous projects were designed to inform decisions about a next step, fully-powered RCT to test the effectiveness and implementation of an mHealth transitions of care intervention.

2. Material and methods

2.1. Participants

A total of 42 adults with SSDs participated in this study. Inclusion criteria were: (a) currently or recently hospitalized (psychiatric acute care inpatient facility); (b) diagnosed with schizophrenia, schizoaffective, schizophrenia unspecified/other, or mood disorder with psychotic features (bipolar or major depressive disorder) based on the Structured Clinical Interview for DSM-5 (SCID-5; [First, Williams, Karg, & Spitzer, 2015]); (c) 18 years or older; (d) prescribed oral psychotropic medication upon discharge; and (e) able to speak and read English. Exclusion criteria were: (a) substance use severe enough to interfere with study participation post-discharge, (b) planned discharge to a long-term supervised living setting or participation in formal outpatient treatment adherence programs, or (c) pregnancy or other medical condition (e.g., dementia) contraindicating use of certain psychotropic medications. Psychotropic medication was an inclusion criterion because this is a recommended primary treatment for this population (American Psychological Association, 2012), and the MACS app was designed to be an adjunctive intervention that specifically addressed medication adherence.

2.2. Procedures

Recruitment occurred during or immediately following (i.e., within 1 week of discharge) participants’ index admission at an acute-care psychiatric hospital in the northeast region of the U.S. The study was approved by the hospital’s Institutional Review Board. Electronic medical records for newly admitted patients were screened for initial eligibility after obtaining a Protected Health Information waiver for this purpose. Patients were approached with permission from their treatment team and completed the informed consent process if interested in the study. Participants were told they would be randomly chosen to receive one of two apps: one offering coping skills and the other providing information about diagnosis and symptoms. Participants completed a baseline assessment prior to or shortly after hospital discharge to confirm eligibility and were then asked to complete 1-month, 2-month, and 4-month follow-ups. We chose a 4-month follow-up window consistent with previous studies (e.g., Bach et al., 2013) and to begin to examine the longer-term effects of the intervention within a timeframe that was feasible to conduct for a pilot study of this nature.

Assessments occurred in person or remotely for participants’ convenience (half of the recruitment occurred during the COVID-19 pandemic). Although we originally intended to conduct blind follow-ups, we were unable to do so because of unexpected staffing disruptions and patient flow problems stemming from the COVID-19 pandemic. Research assistants (RAs) were trained to initial interrater reliability (kappa > .80) on the interview-administered measures, with periodic checks to prevent drift. Participants were compensated for completing study assessments and were also paid a small amount to complete app sessions in both conditions (US $0.25 per session for a maximum of $22.50) during the first month only to improve engagement.

A staff person not affiliated with the study created a randomization sequence using a computer-generated list held in a secure electronic database (via REDCap data management system) at the start of the trial. Randomization allocation was concealed by the computer program prior to assignment and study staff did not have access to view the randomization list. Because of the smaller sample size and to prevent possible differences between conditions, we stratified randomization by diagnosis (schizophrenia/schizophreniform vs schizoaffective) because patients with schizoaffective disorder tend to have a better prognosis (Benabarre et al., 2001), and by birth sex (men vs women) because schizophrenia is more prevalent among younger men (Li et al., 2022).

At baseline, either the MACS app or psychoeducation (PE) app was downloaded onto the participant’s smartphone or if needed, an Android operating system-based mobile device was provided with the app pre-loaded to participants during the study. All participants practiced responding to app sessions in the presence of staff to familiarize themselves with the program and troubleshoot technical problems prior to discharge.

2.3. Mobile app interventions (MACS and PE)

Both apps were programmed using a mobile software service (mema.ilumivu.com), which provided a secure, HIPAA-compliant application (Android or Apple IOS compatible). The protocol consisted of 3 randomly scheduled prompts during daytime hours (9am – 9pm) for the first month and then they were prompted once daily for the remaining 3 months. Additionally, users could initiate a session “on demand” whenever they wished. Each session was designed to take about 5 minutes to complete. For both apps, sessions began with the administration of brief ecological momentary assessment (EMA) items adapted from traditional measures and those used in other mobile studies in psychosis assessing the following constructs: a) affect (Watson & Clark, 1994), b) psychosis (Granholm et al., 2008), c) social support (Zimet et al., 1988) d) life satisfaction (Harper et al., 1998), e) functioning (WHO, 2010), f) treatment adherence (Weiden et al., 1994), g) substance use (Bush et al., 1998; Berman et al., 2003), h) suicidal ideation/safety (Kroenke et al., 2001).

In both the MACS and PE groups, participants were contacted by a study clinician if they indicated significant suicidal ideation (SI) while completing the app surveys based on alerts automatically sent from the device. If participants reported consistent and unchanging levels of passive suicidal ideation (e.g., “a little”), they would be contacted initially to check-in, then contacted only once weekly or if their SI was increasing.

Psychoeducation (PE) app.

Following EMA questions, the PE app delivered brief psychoeducation content about illness adapted from publicly available sources (e.g., British Psychological Society, 2017). Psychoeducation content was presented randomly following EMA items and not linked to patients’ responses. PE participants were given the research team’s contact information and encouraged to call with any questions regarding the app following discharge and to follow their regular treatment plan. The rationale for the choice of the PE condition was to control for the time and attention of mobile device use and to examine whether the response-adaptive MACS app was more effective at changing proposed targets compared with non-tailored mobile assessment/information alone. Additionally, we wanted to design a believable comparison to prevent differences in perceived credibility by patients. Furthermore, psychoeducation is recommended as part of the treatment of schizophrenia in current evidence-based practice guidelines (American Psychological Association, 2012).

MACS app.

In the MACS app, based on responses to the initial EMA questions, participants were provided with individualized intervention skills. The MACS app focused on teaching participants active coping strategies to manage illness-related distress, as well as fostering adherence to medications and treatment appointments. MACS was constructed from CBTp, including previous mobile interventions, to improve self-coping and adherence behaviors (Depp et al., 2010; Granholm et al., 2012). Participants were asked to choose a variety of possible reasons for their reported medication nonadherence and missed treatment appointments. Based on the reported reason for nonadherence, participants were given logistical problem-solving (e.g., instructions to contact their provider) or CBTp techniques (e.g., cost vs. benefits of medication) to promote adherence. If adherence was initially reported or following the provision of the adherence intervention, the MACS app also offered patients the choice of completing additional modules related to social engagement/support, emotion regulation, improving life satisfaction, and reducing substance abuse. For more information on the content of the MACS app, refer to (Moitra et al., 2021).

Within 48 hours of discharge from the hospital, MACS participants were given a brief clinical check-in call to remind them of their discharge plan and to encourage use of the app. After obtaining releases of information from the participant, brief (one-page) clinical reports that summarized MACS app data and symptom severity data from clinical interviews were sent to participants’ outpatient providers after baseline and follow-up assessments to promote care coordination. These reports explained that the individual was using the MACS app and summarized app-collected data related to symptoms and functioning to aid in treatment planning for their non-study clinicians.

2.4. Acceptability and satisfaction measures.

The Client Satisfaction Questionnaire-8 (CSQ-8; [Larsen, Attkisson, Hargreaves, & Nguyen, 1979]) is an 8-item measure designed to assess satisfaction with services or an intervention. The System Usability Scale (SUS; [Brooke, 1996]) is a validated 10-item self-report that was adapted to specifically examine the usability of the apps. Example items included, “I would imagine that most people would learn to use this app very quickly.” Higher scores on the aforementioned measures indicated greater satisfaction and usability, respectively.

2.5. Symptom and functioning measures.

The Brief Psychiatric Rating Scale (BPRS; [Overall & Gorham, 1962]) is an 18-item interviewer-rated measure of overall psychiatric symptoms, including anxiety, depression, and psychosis. Higher scores indicate more severe symptoms. The brief version of the World Health Organization Disability Assessment Schedule (WHODAS 2.0; [WHO, 2010]) is a 12-item self-report measure of functional impairment in activities of daily living, cognition, mobility, self-care, and socialization. The Alcohol Use Disorders Identification Test and Drug Use Disorders Identification Test (AUDIT; [Bush et al., 1998], DUDIT; [Berman et al., 2003]) are brief self-report measures which assess alcohol and drug use, respectively.

2.6. Treatment measures.

The Brief Adherence Rating Scale (BARS; [Byerly, Nakonezny, & Rush, 2008]) is an interviewer administered measure assessing the percentage (0–100%) of primary psychotropic medication doses taken vs. prescribed over the past month. The Treatment History Interview (THI-4; [Linehan & Heard, 1987]) was adapted to assess missed and scheduled mental health appointments, rehospitalizations and quantity of overall mental health treatment utilization. The THI-4 includes information on the type and amount of mental health treatment received over follow-up and was cross-checked with patients’ medical records where possible.

2.7. Process measures.

The Brief Coping Orientation to Problems Experienced (Brief COPE; Carver, 1997) is a 28-item self-report measure of various coping approaches, including problematic or maladaptive approaches. We used the three subscales constructed by Coolidge et al. (2000): emotion-focused coping strategies, problem-focused strategies, and dysfunctional coping strategies. Higher scores indicate higher utilization of that type of these coping strategies. The Medication Beliefs and Attitudes Scale (AMBAS; [Martins et al. 2019]) is validated 12-item self-report that assesses medication beliefs and attitudes, including factors related to shame and stigma. Higher scores mean more positive medication beliefs.

2.8. Statistical analyses

Guided by recommendations from Leon et al. (2011), the primary aim of this pilot trial was to evaluate the feasibility and acceptability of the intervention itself as well as study procedures. We set the following a priori acceptability/feasibility benchmarks for the pilot study based on the clinical trial experience from the research team and similar to other treatment development projects (Gaudiano, Davis, et al, 2020; Gaudiano, Ellenberg, et al., 2023): average recruitment of at least 2 participants per month, app engagement rate ≥ 75% of participants used app, drop out/loss to follow-up ≤ 20%, CSQ score ≥ average 24, significant improvement on BPRS scores over time, and no serious adverse events due to study involvement.

All statistical analyses were conducted in SPSS. We calculated descriptive statistics for measures of acceptability and feasibility. Given the pilot nature of the study (including the smaller sample size), we chose not to adjust alpha (p < .05) in our statistical tests to balance issues of type I vs type II errors (Lee et al., 2014). Chi Square or t-tests, as well as related effect sizes (Cohen’s d), were used to compare groups on variables of interest. For longitudinal outcomes, we conducted a series of multilevel models (MLM; Bryk & Raudenbush, 1992) to assess change in outcomes of interest; MLM was selected because models can accommodate the multilevel nature of the data (repeated assessments nested within individuals) and are robust to missing data (e.g., due to study dropout). Instead of using listwise deletion or another method of imputation, MLM takes into account within-person variability and models the effects based on all available data using maximum likelihood estimation (Baker, 2019). Specifically, measures of interest were entered as outcome variables, predicted by the fixed-effects of time (baseline, one-, two- and four-month follow-up assessments) and treatment condition (active control vs MACS condition) as well as random-effects for subject (allowing individuals’ intercept to vary)1. While analyses were underpowered to detect between-group (PE vs MACS) differences due to relatively small sample size, visual trends in data were examined and effect size estimates are presented (proportion of variance explained; Xu, 2003).

3. Results

3.1. Baseline characteristics

See Table 1 for a description of demographic and clinical characteristics by condition. Additionally, almost all participants had a cell phone at baseline (MACS: 17 [94.4%] vs. PE: 17 [89.5%]). Of those, the majority were smartphone users (MACS: 17 [100%] vs. PE: 16 [94.1%]). Furthermore, almost all participants had a data plan on their phone (MACS: 16 [94.1%] vs. PE: 15 [88.2%]).

Table 1.

Demographics and Baseline Variables

PE (n=23) MACS (n=19)
Demographics N (%) N (%)
Age [M(SD)] 35.1 (11.2) 32.6 (11.7)
Gender
 Female 8 (34.7) 7 (36.8)
 Male 15 (65.2) 11 (57.9)
 Other 0 (0.0) 1 (5.3)
Education1
 Less than high school 4 (19.0) 1 (6.3)
 High school /GED/Some college 12 (57.1) 12 (75.0)
 College/Master’s Degree 5 (23.8) 3 (18.8)
Marital Status1
 Married/co-habitating 2 (9.1) 2 (10.5)
 Single/Divorced/Separated/Widowed 20 (90.9) 17 (89.5)
Race
 White/Caucasian 13 (56.5) 13 (68,4)
 Black or African American 5 (21.7) 1 (5.2)
 Other or multiracial 5 (21.7) 5 (26.3)
Ethnicity
 Hispanic/Latino 4 (17.4) 6 (31.6)
 Not Hispanic/Latino 19 (82.6) 13 (68.4)
Employment Status1
 Employed Full- or Part-time 4 (18.2) 2 (10.5)
 Student/Homemaker/Retired 1 (4.5) 5 (26.3)
 Unemployed 5 (22.7) 4 (21.1)
 Disability 12 (54.5) 8 (42.1)
Income1
 $0–29,999 13 (68.4) 13 (72.2)
 $30,000–59,999 1 (5.3) 1 (5.6)
 $60,000 or greater 5 (26.3) 4 (22.2)
Primary Diagnosis
 Schizophrenia/schizophreniform 7 (30.4) 5 (26.3)
 Schizoaffective disorder 7 (30.4) 7 (36.8)
 Bipolar w psychotic features 0 (0.0) 2 (10.5)
 Major depression w psychotic features 3 (13.0) 2 (10.5)
 Unspecified psychosis 6 (26.1) 3 (15.8)
Chart diagnosis of Substance Use Disorder 7 (30.4) 8 (42.1)
Open in a new tab

PE = Psychoeducation app, MACS = Mobile After Care Support app

1

Missing data for 1 or more participants, percentages represent available data.

3.2. Recruitment and retention

See Figure 1 for participant flow diagram. Overall, our approach-to-decline rate was 50% (67 declined vs 134 approached). The COVID-19 pandemic restricted access to the inpatient units for recruitment for periods of time during the study. However, when we were permitted to recruit, the average number of participants consented per month during the study was 2.32 (SD = 1.3), consistent with our benchmark of 2 per month. In the PE group, two participants withdrew and six were lost to follow up. In the MACS group, one participant withdrew, and one was lost to follow-up. Thus, the rates of follow-up were better for MACS than PE, but this difference was only approaching significance (follow-up rates: MACS = 16 [84.2%] vs. PE = 15 [65.2%],χ2 = 3.37, p = .07). Our benchmark for follow-up was ≤ 20%, which was achieved in the MACS group but not in the PE group.

Figure 1.

Figure 1.

Open in a new tab

CONSORT Chart

3.3. Satisfaction and Acceptability

Both groups reported similar levels of satisfaction with the app at month 4 (PE: 22.5 ± 7.6 vs MACS: 23.7 ± 4.4, t = −0.44, n.s.) out of a total score of 32 on the CSQ. Our target benchmark for the CSQ was an average score of 24, which was achieved for the MACS group but not for the PE group. Additionally, both groups rated the apps as “good”, or a grade of B on the System Usability Scale at month 4 (PE: [n = 10] 70.75 ± 20.45 vs MACS [n = 13] 76.35 ± 17.67, t = −0.70, n.s.). Scores > 68 represent “above average” usability (Gao et al., 2018).

3.4. App engagement

MACS participants were significantly more likely to have used the app at all compared with the PE group (MACS: 14 [73.7%] vs. PE: 10 [43.5%], χ2 = 3.88, p < .05). Our benchmark for app engagement was 75% was within range for the MACS condition but not for the PE condition. In general, participants completed app sessions every 1–3 days on average. There was a trend for participants in the MACS group to complete significantly more app sessions on average per day compared with those in the PE group (MACS: .40 daily ± .63 vs. PE: .17 daily ± .29, t = −1.58, p = .061, d = .47)

3.5. Treatment Utilization

During the follow-up period, there were no significant differences in the number of psychiatric emergency room visits (MACS: 5 [26.3%] vs. PE: 2 [8.7%], χ2 = 2.32, n.s.), rehospitalizations (MACS = 9 [47%] vs PE = 9 [39%], χ2 = 0.28, n.s.), partial hospitalizations (MACS: 6 [31.6%] vs. PE: 5 [21.7%], χ2 = 0.52, n.s.), or crisis/suicide hotline calls (MACS: 0 [0%] vs. PE: 3 [13.0%], χ2 = 2.67, n.s.) between the MACS and PE groups. Additionally, there were no significant differences between groups for those attending outpatient mental health treatment during follow-up (MACS: 16 [84.2%] vs. PE: 15 [65.2%], χ2 = 1.94, n.s.).

3.6. MACS coping skills used

See Table 2. MACS participants most frequently chose to receive coping skills related to distressing thoughts and voices (46.8% of skills chosen), followed by emotions (16.5%), daily activities (15.3%), social support (9.6%), life satisfaction (8.8%) and substance use (2.9%). MACS participants found coping skills focused on appointment treatment adherence helpful (moderately to extremely) 100% of the time they were received. Participants rated medication adherence coping skills as helpful 94.4% (slightly to very) of the time. Skills focused on coping with psychotic symptoms were rated as helpful 97.2% (slightly to extremely) of the time. Emotion-focused skills were rated as helpful 94.3% (slightly to extremely), substance-focused skills as helpful 90.9% (slightly to extremely), and behavioral activation skills as helpful 100% (slightly to extremely) of the time.

Table 2.

Satisfaction with MACS App Coping Skills

Skill Type Not at all helpful N (%) Slightly helpful N (%) Moderately helpful N (%) Very helpful N (%) Extremely helpful N (%)
Treatment Adherence - Appointments 0 (0) 0 (0) 5 (62.5) 0 (0) 3 (37.5)
Treatment Adherence - Medications 2 (5.6) 13 (36.1) 17 (47.2) 4 (11.1) 0 (0)
Psychosis 10 (2.8) 140 (39.4) 136 (38.3) 54 (15.2) 15 (4.2)
Emotions 7 (5.7) 22 (17.9) 44 (35.8) 30 (24.4) 20 (16.3)
Substance Use 2 (9.1) 2 (9.1) 5 (22.7) 3 (13.6) 10 (45.5)
Behavioral Activation 0 (0) 29 (26.1) 52 (46.8) 17 (15.3) 13 (11.7)
Quality of Life 4 (6.2) 15 (23.1) 16 (24.6) 13 (20.0) 17 (26.2)
Social Support 1 (1.4) 12 (16.2) 31 (41.9) 13 (17.6) 17 (23.0)
Open in a new tab

3.7. Safety data

Psychiatric inpatient rehospitalizations and suicide attempts were deemed expected and unrelated study events given the history and nature of the sample. There were no significant differences between the groups in number of reported suicide attempts (MACS = 2 [10.5%] vs PE = 0 [0%], χ2 = 2.54, n.s.). One participant in the MACS group reported paranoia while using the app and this was successfully managed when identified. A similar number of participants in each condition endorsed some level of SI on the app during the study (MACS: 10 [71%] vs. PE: 5 [50%], χ2 = 1.14, n.s.).

3.8. Clinical outcomes

Results from the MLMs indicated that there was a significant fixed-effect of time on coping strategy use and psychiatric symptom severity (see Tables 3 and 4). Specifically, dysfunctional coping declined over time, regardless of condition (βtime = −0.86; SE = 0.27; 95%CI = −1.41 to −0.32), as did total psychiatric symptoms (βtime = −3.89; SE = 0.56; 95%CI = −5.01 to −2.77). Improvement over time in medication beliefs was only approaching significance (βtime = 1.89; SE = 1.05; 95%CI = −0.19, 3.98). No significant changes over time were found for functioning, medication adherence, problem-focused or emotion-focused coping, or drug use. We did not identify differences over time on the above outcomes by treatment condition.

Table 3.

Fixed Effects of Time on Outcomes using Multilevel Modeling from Baseline through 4 Months Post-Hospital Discharge

Variable B SE 95% CI (B) t/z Proportion of variance explained
BPRS Total
Time −3.89 0.56 −5.01, −2.77 −6.89** 33.03%
Condition −3.02 2.99 −9.07, 3.03 −1.01 0.82%
COPE Dysfunctional
Time −0.86 0.27 −1.41, −0.32 −3.15* 9.97%
Condition −0.43 2.12 −4.73, 3.87 −0.20 0.05%
COPE Problem Focused
Time −0.12 0.17 −0.47, 0.22 −0.71 6.10%
Condition 1.59 1.24 −0.92, 4.10 1.28 6.56%
COPE Emotion Focused
Time −0.35 0.26 −0.88, 0.17 −1.34 3.05%
Condition −0.57 1.72 −4.06, 2.92 −0.33 0.93%
AMBAS Total
Time −0.23 0.27 −0.77, 0.31 −.086† 3.45%
Condition −1.37 2.15 −5.73, 2.99 −0.64 0.33%
BARS
Time 1.89 1.05 −0.19, 3.98 1.81 2.80%
Condition −2.58 5.49 −13.69, 8.53 −0.47 0.21%
WHODAS
Time −0.01 0.01 −0.03, 0.01 −1.47 1.00%
Condition −0.08 0.06 −0.20, 0.04 −1.42 0.46%
AUDIT
Time −0.34 0.27 −0.87, 0.20 1.24 0.93%
Condition −2.60 1.38 −5.34, 0.13 −1.89† 3.05%
DUDIT
Time −0.72 0.41 −1.53, 0.10 −1.76 2.65%
Condition −1.79 2.85 −7.58, 3.99 −0.63 0.36%
Open in a new tab

Notes:

†

>.1

*

>.05

**

>.001

Table 4.

Estimated Means and Standard Errors from Multilevel Models

Time Baseline Month 1 Month 2 Month 4
Condition PE MACS PE MACS PE MACS PE MACS
BPRS Total Mean (SE) 49.81 (2.23) 53.12 (2.41) 38.91 (2.54) 42.24 (2.58) 37.50 (2.55) 40.81 (2.61) 33.13 (2.57) 36.44 (2.59)
[95% CI] 45.36, 54.26 48.31, 57.93 33.87, 43.95 37.10, 47.36 32.44, 42.57 35.61, 46.01 28.03, 38.23 31.29, 41.60
COPE DF Mean (SE) 28.14 (1.52) 28.58 (1.63) 26.13 (1.64) 26.58 (1.72) 26.26 (1.65) 26.70 (1.71) 24.40 (1.66) 24.84 (1.72)
[95% CI] 25.07, 31.21 25.30, 31.87 22.85, 29.42 23.13, 30.03 22.96, 29.56 23.27, 30.14 21.08, 27.71 21.39, 28.30
COPE PF Mean (SE) 18.05 (0.91) 16.47 (0.97) 16.44 (0.98) 14.87 (1.03) 17.16 (0.98) 15.58 (1.02) 17.25 (0.99) 15.68 (1.03)
[95% CI] 16.23, 19.86 14.53, 18.41 14.49, 18.40 12.81, 16.92 15.19, 19.12 13.54, 17.63 15.27, 19.23 13.62, 17.73
COPE EF Mean (SE) 25.16 (1.27) 25.74 (1.35) 24.29 (1.39) 24.87 (1.45) 24.10 (1.40) 24.68 (1.44) 23.67 (1.42) 24.25 (1.45)
[95% CI] 22.60, 27.71 23.02, 28.46 21.50, 27.08 21.96, 27.78 21.29, 26.90 21.79, 27.57 20.84, 26.49 21.34, 27.16
AMBAS Total Mean (SE) 29.54 (1.52) 30.87 (1.66) 28.44 (1.63) 29.77 (1.74) 30.64 (1.63) 31.97 (1.73) 28.10 (1.64) 29.44 (1.74)
[95% CI] 26.48, 32.59 27.52, 34.22 25.18, 31.70 26.27, 33.27 27.36, 33.94 28.49, 35.46 24.81, 31.40 25.94, 32.93
BARS Mean (SE) 83.84 (4.41) 86.21 (4.76) 89.42 (4.66) 91.80 (4.70) 92.25 (4.63) 94.62 (4.82) 92.03 (4.75) 94.40 (4.88)
[95% CI] 75.05, 92.63 76.72, 95.70 80.15, 98.70 82.42, 101.18 83.02, 101.47 85.00, 104.24 82.57, 101.49 84.68, 104.12
WHODAS Mean (SE) 0.33 (0.04) 0.41 (0.05) 0.29 (0.05) 0.37 (0.05) 0.33 (0.05) 0.41 (0.05) 0.27 (0.05) 0.35 (0.05)
[95% CI] 0.24, 0.41 0.32, 0.50 0.20, 0.38 0.28, 0.47 0.24, 0.42 0.32, 0.51 0.18, 0.36 0.26, 0.45
AUDIT Mean (SE) 3.20 (1.25) 6.13 (1.32) 2.69 (1.15) 5.61 (1.21) 2.31 (1.33) 5.24 (1.36) 1.70 (1.05) 4.62 (1.03)
[95% CI] 0.69, 5.72 3.47, 8.79 0.38, 4.99 3.17, 8.05 −0.40, 5.03 2.46, 8.02 −0.42, 3.81 2.53, 6.72
DUDIT Mean (SE) 6.28 (2.12) 8.09 (2.20) 5.38 (2.28) 7.19 (2.36) 4.85 (2.30) 6.66 (2.35) 3.36 (2.32) 5.17 (2.36)
[95% CI] 2.02, 10.54 3.66, 12.52 0.81, 9.95 2.45, 11.92 0.25, 9.45 1.95, 11.37 −1.27, 7.99 0.43, 9.90
Open in a new tab

Notes: DF = Dysfunctional; PF = Problem Focused; EF = Emotion Focused

4. Discussion

Results showed that MACS was generally feasible and acceptable to deliver as a treatment support aid for patients with SSD transitioning from an acute psychiatric hospitalization back into outpatient care. Our approach/consent rate was 50%. We were able to meet our goal of consenting at least 2 participants per month, suggesting that recruitment of inpatients is feasible for this type of mHealth study. We were able to recruit for the study despite significant challenges and limitations presented by the COVID-19 pandemic (e.g., having to start and stop recruitment for extended periods of time due to unit quarantines and remove to remote assessment and administration).

Overall, both groups reported that the apps were acceptable to use, and participants were largely satisfied with their experience. Average satisfaction ratings were similar for both groups (PE = 22.5 vs MACS = 23.7) and were slightly below our target of 24 on the CSQ for the PE group (representing good to very good satisfaction). Although we exceeded our anticipated 80% follow-up rate for the MACS condition (84%), we did not achieve this for the PE condition (65%). We also found significantly higher engagement with the app in the MACS group (74%) compared with the PE group (43%). In fact, the PE group’s engagement rate was well below our target app engagement rate of 75%. These findings suggest that overall the PE app was less acceptable and feasible than the MACS app and produced less follow-up engagement. Future trials should consider more assertive follow-up procedures, as participants might find the comparison condition less engaging and thus be more inclined to drop out. Additional strategies could include coordinating with community agencies where participants are receiving follow-up treatment or selecting friends or family members to be involved in the study to improve completion rates.

Rehospitalizations and suicide attempts were expected events given the nature of the sample, but the frequency of these events did not differ between conditions. The MACS group reported slightly (non-significantly) more suicidal ideation during app monitoring compared with the PE group. However, this was likely confounded by the overall higher level of engagement in the MACS group, leading to more opportunities to report suicidal ideation for those using MACS.

Both groups showed significant improvement over time in total psychiatric symptoms (33% of variance explained representing a large effect size), which was our proposed primary clinical outcome. The sample also demonstrated significant decreases in their use of dysfunctional coping strategies over follow-up (10% of variance explained representing a medium effect size), which was one of our proposed mechanisms of action for the intervention. Furthermore, self-reported medication adherence was high and similar across groups, so there was limited opportunity of identifying changes in this variable. Although we attempt to test for group differences over time in order to demonstrate the feasibility of doing so and to collect information relevant to powering a future trial, the sample size was too small to meaningfully interpret group differences over time.

We are only aware of one previous study that was similar to the current investigation in that it tested an app (called FOCUS) for patients with SSD who were enrolled post-discharge during community treatment, although in this case within 60 days of hospital discharge. This quasi-experimental (non-randomized) effectiveness trial of FOCUS (n = 342) found that 82% of patients engaged with the app (Brunette et al., 2016). Compared to usual care, FOCUS was associated with an average reduction of 4.6 days of hospitalization over 6-month follow-up (Homan et al., 2022). However, compared with the previous FOCUS study, the current trial tested an app among patients enrolled during a current hospitalization and who started to use the app immediately post-discharge. This is important to emphasize because previous use of FOCUS was studied in a sample that was already engaged with community-based services. In reality, many patients never make it that far: they do not seek out or engage in outpatient care immediately post-discharge (“fall between the cracks”) until they reappear in the emergency room, a re-hospitalization, or worse at a later point. In addition, the FOCUS trial did not use a randomized design, and the app was delivered as part of a larger mHealth program that included computerized CBTp, a daily support website, and a prescriber decision support system, so it was not possible to differentiate the effect of the FOCUS app specifically. To our knowledge, MACS is the first standalone app intervention employed and tested during the direct transition from inpatient to outpatient care in patients with SSD.

Several limitations and areas for future study should be considered. First, follow-up assessments were not conducted in a blind fashion. Also, this was a pilot study with a smaller sample and so we were not powered to detect group differences on clinical outcomes. However, we were able to examine our intended feasibility benchmarks to inform future larger-scale trials. Second, the two app conditions both contained EMA and only differed in the therapeutic content they delivered (i.e., psychoeducation vs cognitive-behavioral techniques). Thus, there was significant overlap between conditions and potentially their effect on outcomes. However, we did find evidence that the PE app was less engaging for participants relative to MACS as it resulted in lower app use and follow-up. Thus, future trials might instead consider comparing treatment as usual alone with or without MACS to verify the overall efficacy of the aftercare treatment app for patients with SSD, as this might produce more informative results for implementation purposes. Additionally, a future trial of MACS might consider using a Multiphase Optimization Strategy (MOST) design so that specific components of the intervention can be selected (or removed) and tested to determine optimal performance on mechanisms and outcomes (Collins et al, 2007).

Third, participants completed an app session on average once every 1–3 days during the study. Because most patients are likely to have only one outpatient session per week with a provider at best, the use of this technology provides opportunities for enhanced monitoring and early intervention compared with traditional treatment alone. However, there was significant variability in the number of app sessions completed by participants (range 1–125) even though we provided a small amount of compensation for app sessions during the first month. Therefore, future research should consider ways of making the MACS app more appealing and useful to further enhance uptake, which could take the form of a “digital navigator” or peer support person to encourage app usage (Wisniewski & Torous, 2020).

Finally, the study was only conducted at one hospital and future work is needed to examine expansion of MACS to different settings. The clinical population at the recruitment site is known to be similar in diagnostic and sociodemographic characteristics compared with other hospitals in our region, but settings with more elderly patients or those with more significant social determinants of health issues may find MACS implementation more challenging. Also, this project was conducted during the COVID-19 pandemic which negatively impacted both enrollment efforts and aftercare treatment availability for many patients. Thus, the pandemic and its consequences could have negatively influenced the utility of MACS as an adjunctive intervention. Future reach should test MACS further under more typical circumstances to better inform the generalizability of our findings.

In conclusion, apps that offer monitoring and self-management skills like MACS appear to be feasible, acceptable, and potentially efficacious for enhancing the aftercare treatment of patients with SSD immediately following an acute hospitalization. Although patients in both groups who used the apps reported that they were feasible and acceptable overall, our data suggest that the MACS app that offered CBTp skills (i.e., emphasizing treatment adherence and coping with illness) was more engaging than the PE app that provided psychoeducation alone. We also observed significant change over time in psychiatric symptoms and one of our proposed mechanisms of decreased dysfunctional coping among participants with large and medium effects, respectively. However, given the pilot nature of the current trial, we were not able to collect and conduct formal cost-effectiveness analyses. Such an analysis would be relevant to include in a future large-scale implementation trial to ensure equitable implementation and access to the MACS app for different patient subgroups. Future trials are needed with larger samples to confirm the clinical utility of the MACS app for improving outcomes relative to those receiving routine care alone or to examine specific contributions of the various components of the intervention.

Impact Statement.

Individuals with psychosis leaving psychiatric hospitals encounter frequent gaps in care and poor transitions that result in rehospitalization and relapse. Mobile technology can help bridge the gap between hospital and community treatment and serve as an adjunct to treatment. This study showed that a mobile intervention was feasible and acceptable for patients with psychosis leaving the hospital and contributed to improved symptoms and coping in the community up to 4 months following discharge.

Authors notes:

This project was funded by National Institute of Mental Health grant # R34 MH115144 (PIs: Gaudiano & Moitra). Madeline Ward, B.A. is now affiliated with the Department of Psychology, Case Western Reserve University.

Footnotes

Potential conflicts of interest: None.

1

We used an iterative approach for model testing for each outcome variable such that we initially tested a null model (with no fixed effect predictors, only a random intercept by participant ID), we then fit a model with fixed effects of time and condition with a random intercept but not slope, followed by a model with the addition of a random slope for time, and finally a model with the time by condition interaction. Model fit was compared for each, and the best fitting model for each outcome is presented in the present manuscript. Coincidentally, despite all models selected for each outcome independently from that of other outcome variables, the best fitting model for all outcome variables was one containing fixed effects for time and condition with a random intercept (but no time by condition interaction or random slopes)—the addition of the cross-level interaction or random slopes, produced models with worse fit.

References

  1. American Psychiatric Association (2021). The American Psychiatric Association Practice Guideline for the Treatment of Patients with Schizophrenia (3rd ed). APA: Washington, DC. [Google Scholar]
  2. Attkisson CC, & Zwick R (1982). The client satisfaction questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5(3), 233–237. 10.1016/0149-7189(82)90074-X [DOI] [PubMed] [Google Scholar]
  3. Bach P, Gaudiano BA, Hayes SC, & Herbert JD (2013). Acceptance and commitment therapy for psychosis: intent to treat, hospitalization outcome and mediation by believability. Psychosis, 5, 166–174. [Google Scholar]
  4. Baker SG (2019). Maximum likelihood estimation with missing outcomes: From simplicity to complexity. Statistics in Medicine, 38, 4453–4474. 10.1002/sim.8319 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Berman AH, Bergman H, Palmstierna T, & Schlyter F (2003). Drug use disorders identification test. European addiction research. Available at: https://paihdelinkki.fi/sites/default/files/duditmanual.pdf [DOI] [PubMed] [Google Scholar]
  6. Bidargaddi N, Schrader G, Klasnja P, Licinio J, & Murphy S (2020). Designing m-Health interventions for precision mental health support. Translational psychiatry, 10(1), 222. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. British Psychological Society. (2017). Understanding psychosis and schizophrenia. Available at: https://www.bps.org.uk/guideline/understanding-psychosis-and-schizophrenia (Accessed: 30 June 2023).
  8. Brooke J (1996). SUS-A quick and dirty usability scale. Usability evaluation in industry, 189(194), 4–7. [Google Scholar]
  9. Brunette MF, Rotondi AJ, Ben-Zeev D, Gottlieb JD, Mueser KT, Robinson DG, Achtyes ED, Gingerich S, Marcy P, Schooler NR, Meyer-Kalos P, & Kane JM (2016). Coordinated Technology-Delivered Treatment to Prevent Rehospitalization in Schizophrenia: A Novel Model of Care. Psychiatric services, 67(4), 444–447. 10.1176/appi.ps.201500257 [DOI] [PubMed] [Google Scholar]
  10. Bryk AS, & Raudenbush SW (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage Publications. [Google Scholar]
  11. Jacobsen PC, Teale A, & Burgess-Barr S (2023). A study investigating the implementation of NICE recommended psychological interventions for people with psychosis following a psychiatric inpatient admission. PsyArXiv; 2023. DOI: 10.31234/osf.io/yemqb [DOI] [PubMed] [Google Scholar]
  12. Benabarre A, Vieta E, Colom F, Martinez-Aran A, Reinares M, & Gasto C (2001). Bipolar disorder, schizoaffective disorder and schizophrenia: epidemiologic, clinical and prognostic differences. European Psychiatry, 16, 167–172. [DOI] [PubMed] [Google Scholar]
  13. Bush K, Kivlahan DR, McDonell MB, Fihn SD, & Bradley KA (1998). The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Archives of internal medicine, 158(16), 1789–1795. 10.1001/archinte.158.16.1789 [DOI] [PubMed] [Google Scholar]
  14. Byerly MJ, Nakonezny PA, & Rush AJ (2008). The Brief Adherence Rating Scale (BARS) validated against electronic monitoring in assessing the antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. Schizophrenia research, 100(1–3), 60–69. 10.1016/j.schres.2007.12.470 [DOI] [PubMed] [Google Scholar]
  15. Carver CS (1997). You want to measure coping but your protocol’s too long: consider the brief COPE. International journal of behavioral medicine, 4(1), 92–100. 10.1207/s15327558ijbm0401_6 [DOI] [PubMed] [Google Scholar]
  16. Collins LM, Murphy SA, & Strecher V (2007). The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions. American Journal of Preventive Medicine, 32, S112–S118. 10.1016/j.amepre.2007.01.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Coolidge FL, Segal DL, Hook JN, & Stewart S (2000). Personality Disorders and Coping Among Anxious Older Adults. Journal of Anxiety Disorders, 14(2), 157–172. 10.1016/S0887-6185(99)00046-8 [DOI] [PubMed] [Google Scholar]
  18. Depp CA, Howland A, Dumbauld J, Fontanesi J, Firestein D, & Firestein GS (2018). Development of a game-based learning tool for applied team science communication in a virtual clinical trial. Journal of Clinical and Translational Science, 2(3), 169–172. 10.1017/cts.2018.8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Depp CA, Mausbach B, Granholm E, Cardenas V, Ben-Zeev D, Patterson TL, ... & Jeste DV. (2010). Mobile interventions for severe mental illness: design and preliminary data from three approaches. The Journal of nervous and mental disease, 198(10), 715–721. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Ellett L (2023). Mindfulness for psychosis: Current evidence, unanswered questions and future directions. Psychology and Psychotherapy. 10.1111/papt.12480 [DOI] [PubMed] [Google Scholar]
  21. Farhall J, & Thomas N (2013). Cognitive and behavioural therapies for psychosis. Australian and New Zealand Journal of Psychiatry, 47(6), 508–511. 10.1177/0004867413483370 [DOI] [PubMed] [Google Scholar]
  22. First MB, Williams JBW, Karg RS, & Spitzer RL (2016). User’s guide for the SCID-5-CV Structured Clinical Interview for DSM-5® disorders: Clinical version. American Psychiatric Publishing, Inc. [Google Scholar]
  23. Gao M, Kortum P, & Oswald F (2018). Psychometric Evaluation of the USE (Usefulness, Satisfaction, and Ease of use) Questionnaire for Reliability and Validity. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62(1), 1414–1418. 10.1177/1541931218621322 [DOI] [Google Scholar]
  24. Gaudiano BA, Davis CH, Miller IW, & Uebelacker L (2020). Pilot randomized controlled trial of a video self-help intervention for depression based on acceptance and commitment therapy: Feasibility and acceptability. Clinical Psychology & Psychotherapy, 27, 396–407. 10.1002/cpp.2436 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Gaudiano BA, Ellenberg S, Johnson JE, Mueser KT, & Miller IW (2023). Effectiveness of acceptance and commitment therapy for inpatients with psychosis: Implementation feasibility and acceptability from a pilot randomized controlled trial. Schizophrenia Research, 261, 72–79. 10.1016/j.schres.2023.09.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Granholm E, Ben-Zeev D, Link PC, Bradshaw KR, & Holden JL (2012). Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophrenia bulletin, 38(3), 414–425. 10.1093/schbul/sbr155 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Granholm E, Loh C, & Swendsen J (2008). Feasibility and validity of computerized ecological momentary assessment in schizophrenia. Schizophrenia bulletin, 34(3), 507–514. 10.1093/schbul/sbm113 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Harper A, Power M, & Working Group. (1998). Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological Medicine, 28(3), 551–558. [DOI] [PubMed] [Google Scholar]
  29. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of biomedical informatics, 42(2), 377–381. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Homan P, Schooler NR, Brunette MF, Rotondi A, Ben-Zeev D, Gottlieb JD, Mueser KT, Achtyes ED, Gingerich S, Marcy P, Meyer-Kalos P, Hauser M, John M, Robinson DG, & Kane JM (2022). Relapse prevention through health technology program reduces hospitalization in schizophrenia. Psychological medicine, 1–7. 10.1017/S0033291722000794 [DOI] [PubMed] [Google Scholar]
  31. Hudson CG (2019). Five-year rehospitalization experience of a state-wide cohort of persons with schizophrenia. Social Psychiatry and Psychiatric Epidemiology, 54(7), 861–870. 10.1007/s00127-018-1650-7 [DOI] [PubMed] [Google Scholar]
  32. Jones C, Hacker D, Cormac I, Meaden A, & Irving CB (2012). Cognitive behavioural therapy versus other psychosocial treatments for schizophrenia. The Cochrane Database of Systematic Reviews, 4, CD008712. 10.1002/14651858.CD008712.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kadakia A, Fan Q, Shepherd J, Dembek C, Bailey H, Walker C, & Williams GR (2022). Healthcare resource utilization and quality of life by cognitive impairment in patients with schizophrenia. Schizophrenia Research: Cognition, 28, 100233. 10.1016/j.scog.2021.100233 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kimhy D, Harkavy-Friedman JM, & Nelson EA (2004). Identifying Life Stressors of Patients With Schizophrenia at Hospital Discharge. Psychiatric Services, 55(12), 1444–a. 10.1176/appi.ps.55.12.1444-a [DOI] [PubMed] [Google Scholar]
  35. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of general internal medicine, 16(9), 606–613. 10.1046/j.1525-1497.2001.016009606.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Lee EC, Whitehead AL, Jacques RM, & Julious SA (2014). The statistical interpretation of pilot trials: should significance thresholds be reconsidered? BMC Medical Research Methodology, 14, 41. 10.1186/1471-2288-14-41 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Leon AC, Davis LL, & Kraemer H,C. (2011). The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research, 45, 626–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Li X, Zhou W, & Yi Z (2022). A glimpse of gender differences in schizophrenia. General Psychiatry, 35, e100823. 10.1136/gpsych-2022-100823 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Linehan MM, & Heard H (1987). Treatment History Interview (THI). University of Washington. [Google Scholar]
  40. Martins MJRV, Pinto AM, Castilho P, Macedo AF, Pereira AT, Bajouco M, Leite R, Matos O, Morais S, Madeira N, Nogueira V, Roque C, Santos P, Santos T, Santos T, Santos V, Troia AF, & Barreto-Carvalho C (2019). Assessing beliefs and attitudes towards antipsychotic medication from a recovery-based perspective: Psychometric properties of a new scale. Psychiatry Research, 273, 325–330. 10.1016/j.psychres.2019.01.043 [DOI] [PubMed] [Google Scholar]
  41. Marzano L, Bardill A, Fields B, Herd K, Veale D, Grey N, & Moran P (2015). The application of mHealth to mental health: Opportunities and challenges. The Lancet Psychiatry, 2(10), 942–948. 10.1016/S2215-0366(15)00268-0 [DOI] [PubMed] [Google Scholar]
  42. Mueser KT, Deavers F, Penn DL, & Cassisi JE (2013). Psychosocial Treatments for Schizophrenia. Annual Review of Clinical Psychology, 9(1), 465–497. 10.1146/annurev-clinpsy-050212-185620 [DOI] [PubMed] [Google Scholar]
  43. Naslund JA, Marsch LA, McHugo GJ, & Bartels SJ (2015). Emerging mHealth and eHealth Interventions for Serious Mental Illness: A Review of the Literature. Journal of Mental Health, 24(5), 321–332. 10.3109/09638237.2015.1019054 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Olfson M, Mechanic D, Hansell S, Boyer CA, Walkup J, & Weiden PJ (2000). Predicting Medication Noncompliance After Hospital Discharge Among Patients With Schizophrenia. Psychiatric Services, 51(2), 216–222. 10.1176/appi.ps.51.2.216 [DOI] [PubMed] [Google Scholar]
  45. Overall JE, Gorham DR (1962). The brief psychiatric rating scale. Psychological Reports, 10, 799–812. [Google Scholar]
  46. Pew Research Center (2021). Mobile fact sheet: Internet, Science & Tech. Available at: https://www.pewinternet.org/fact-sheet/mobile (Accessed: 30 June 2023).
  47. Pompili M, Amador XF, Girardi P, Harkavy-Friedman J, Harrow M, Kaplan K, Krausz M, Lester D, Meltzer HY, Modestin J, Montross LP, Bo Mortensen P, Munk-Jørgensen P, Nielsen J, Nordentoft M, Saarinen PI, Zisook S, Wilson ST, & Tatarelli R (2007). Suicide risk in schizophrenia: Learning from the past to change the future. Annals of General Psychiatry, 6(1), 10. 10.1186/1744-859X-6-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Turner DT, McGlanaghy E, Cuijpers P, van der Gaag M, Karyotaki E, & MacBeth A (2018). A Meta-Analysis of Social Skills Training and Related Interventions for Psychosis. Schizophrenia Bulletin, 44(3), 475–491. 10.1093/schbul/sbx146 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Ustün TB, Chatterji S, Kostanjsek N, Rehm J, Kennedy C, Epping-Jordan J, Saxena S, von Korff M, Pull C, & WHO/NIH Joint Project. (2010). Developing the World Health Organization Disability Assessment Schedule 2.0. Bulletin of the World Health Organization, 88(11), 815—823. 10.2471/blt.09.067231 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Velligan DI, Roberts DL, Sierra C, Fredrick MM, & Roach MJ (2016). What Patients With Severe Mental Illness Transitioning From Hospital to Community Have to Say About Care and Shared Decision-Making. Issues in Mental Health Nursing, 37(6), 400–405. 10.3109/01612840.2015.1132289 [DOI] [PubMed] [Google Scholar]
  51. Wander C (2020). Schizophrenia: Opportunities to improve outcomes and reduce economic burden through managed care. The American Journal of Managed Care, 26(Suppl 3), S62–S68. 10.37765/ajmc.2020.43013 [DOI] [PubMed] [Google Scholar]
  52. Watson D, & Clark L (1999). The PANAS-X: Manual for the positive and negative affect schedule-expanded form. Psychology Publications. [Google Scholar]
  53. Weiden P, Rapkin B, Mott T, Zygmunt A, Goldman D, Horvitz-Lennon M, & Frances A (1994). Rating of medication influences (ROMI) scale in schizophrenia. Schizophrenia bulletin, 20(2), 297–310. 10.1093/schbul/20.2.297 [DOI] [PubMed] [Google Scholar]
  54. Wisniewski H, & Torous J (2020). Digital navigators to implement smartphone and digital tools in care. Acta Psychiatrica Scandinavica, 141(4), 350–355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Xu R (2003). Measuring explained variation in linear mixed effects models. Statistics in medicine, 22(22), 3527–3541. [DOI] [PubMed] [Google Scholar]
  56. Young AS, Cohen AN, Niv N, Nowlin-Finch N, Oberman RS, Olmos-Ochoa TT, Goldberg RW, & Whelan F (2020). Mobile Phone and Smartphone Use by People With Serious Mental Illness. Psychiatric Services, 71(3), 280–283. 10.1176/appi.ps.201900203 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Zimet GD, Dahlem NW, Zimet SG, & Farley GK (1988). The multidimensional scale of perceived social support. Journal of personality assessment, 52(1), 30–41. [DOI] [PubMed] [Google Scholar]

RESOURCES