Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Apr 1.
Published in final edited form as: Contemp Clin Trials. 2024 Mar 1;139:107481. doi: 10.1016/j.cct.2024.107481

A Hybrid Type 1 trial of a multi-component mHealth intervention to improve post-hospital transitions of care for patients with serious mental illness: study protocol

Ethan Moitra 1,a, Toni M Amaral 2, Madeline B Benz 1,2, Simranjeet Cambow 2, A Rani Elwy 1,3, Zachary J Kunicki 1, Zhengduo Lu 2, Neil S Rafferty 2, Ana Rabasco 1,2, Rita Rossi 2, Heather T Schatten 1,2, Brandon A Gaudiano 1,2,3
PMCID: PMC10960682  NIHMSID: NIHMS1972419  PMID: 38431134

Abstract

Background:

The transition from acute (e.g., psychiatric hospitalization) to outpatient care is associated with increased risk for rehospitalization, treatment disengagement, and suicide among people with serious mental illness (SMI). Mobile interventions (i.e., mHealth) have the potential to increase monitoring and improve coping post-acute care for this population. This protocol paper describes a Hybrid Type 1 effectiveness-implementation study, in which a randomized controlled trial will be conducted to determine the effectiveness of a multi-component mHealth intervention (tFOCUS) for improving outcomes for adults with SMI transitioning from acute to outpatient care.

Methods:

Adults meeting criteria for schizophrenia-spectrum or major mood disorders (n = 180) will be recruited from a psychiatric hospital and randomized to treatment-as-usual (TAU) plus standard discharge planning and aftercare (CHECK-IN) or TAU plus tFOCUS. tFOCUS is a 12-week intervention, consisting of: (a) a patient-facing mHealth smartphone app with daily self-assessment prompts and targeted coping strategies; (b) a clinician-facing web dashboard; and, (c) mHealth aftercare advisors, who will conduct brief post-hospital clinical calls with patients (e.g., safety concerns, treatment engagement) and encourage app use. Follow-ups will be conducted at 6-, 12-, and 24-weeks post-discharge to assess primary and secondary outcomes, as well as target mechanisms. We also will assess barriers and facilitators to future implementation of tFOCUS via qualitative interviews of stakeholders and input from a Community Advisory Board throughout the project.

Conclusions:

Information gathered during this project, in combination with successful study outcomes, will inform a potential tFOCUS intervention scale-up across a range of psychiatric hospitals and healthcare systems.

Keywords: serious mental illness, mHealth, ecological momentary intervention, post-acute care, psychiatric hospitalization, Hybrid Type 1

1. Introduction

Psychiatric disorders of serious mental illness (SMI), including bipolar disorder, major depression, and schizophrenia-spectrum disorders, are highly burdensome to patients and society. Worldwide, they affect millions of adults and significantly contribute to years of life lost due to disability and mortality [1]. With a lifetime risk of 9–27% [24], suicide is the leading cause of death among individuals with SMI and this group is over-represented among U.S. adults who die by suicide each year [5]. Additionally, the economic burden of SMI is substantial, ranging from $156 billion per year for schizophrenia-spectrum disorders [6] to $211 billion per year for major depressive disorder [7].

Although efficacious treatments are available for SMI, including pharmacotherapy [8, 9] and psychosocial interventions [10], engagement in these treatments is often suboptimal. For example, up to 50% of adults with schizophrenia-spectrum and mood disorders are medication nonadherent [1114], which undermines the benefits of efficacious treatments and contributes to high rates of rehospitalization and relapse risk among those with SMI [15, 16]. Poor engagement is particularly prevalent when patients with SMI transition from acute care (i.e., psychiatric hospitalization) to outpatient services. Indeed, recent psychiatric hospitalization predicts treatment nonadherence [6, 1518], and the transition from inpatient to outpatient services confers the highest risk of nonadherence and lack of follow through with recommended care [19, 20]. Additionally, the highest risk period for suicide among individuals with SMI appears to be in the first month after discharge [21, 22]; yet risk levels remain elevated for up to one year post-discharge, with 30–60 fold higher suicide risk following hospitalization than in community samples [2224]. Despite unreliable treatment engagement and elevated suicide risk following hospital discharge, clinical settings often lack feasible and effective services to support patients’ return to the community and re-engagement with outpatient providers.

Although engagement and illness self-management interventions for adults with SMI in the community have produced promising, albeit mixed results, limited research has focused on the immediate post-acute care period [2528]. To improve treatment engagement and effective coping with illness, a growing body of research focuses on developing and testing interventions that leverage digital or mobile mental health services (i.e., mHealth or mobile technology-supported resources that can improve patients’ functioning and reduce symptoms) [29]. mHealth for patients with SMI has been shown to be feasible, acceptable, and efficacious [30, 31]. Moreover, emergent research from our group shows that an mHealth intervention is feasible and acceptable among patients with SMI transitioning from inpatient to outpatient care [32].

This protocol paper describes a 5-year Hybrid Type 1 [33] effectiveness-implementation study, in which a randomized controlled trial (RCT) will be conducted to determine the effectiveness of a multi-component mHealth intervention for improving symptoms, recovery, coping, and treatment engagement for adults with SMI during a transition of care from psychiatric hospitalization to outpatient community treatment. Factors potentially impacting future implementation of the mHealth intervention will also be examined.

We will use an adapted version of a multi-modal mHealth illness self-management system for people with SMI, called FOCUS. Findings suggest that FOCUS is feasible, acceptable, and engaging [34, 35] for people with SMI, and produces medium to large effects in reduction of symptoms of general psychopathology, psychosis, depression, and improvement in recovery [3538]. However, previous research on the FOCUS intervention was conducted in samples already receiving community-based services. FOCUS has never been tested in patients enrolled during a current hospitalization and followed immediately post-discharge. This is important because, in reality, many patients never make it to community-based services, as they do not seek or engage in outpatient care following hospital discharge [17]. Consistent with a Hybrid Type 1 design, a mixed methods process evaluation of the study will also be conducted to inform future implementation of the intervention.

2. Materials and methods

2.1. Overview

In this Hybrid Type 1 effectiveness-implementation trial, our primary and secondary aims focus on assessing effectiveness outcomes, while our tertiary aim will assess barriers and facilitators to future implementation, guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM [39]) and the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS [40]) frameworks. We designed our Hybrid Type 1 trial to balance issues of internal (i.e., blind outcome raters, randomization, treatment standardization, fidelity monitoring) and external validity (i.e., recruitment from routine hospital setting, use of typical hospital providers) [33]. Our study will also be informed by input from a Community Advisory Board (CAB), consisting of persons with lived experience, payors, clinicians, and administrators representing inpatient and outpatient mental health settings.

2.2. Setting

All study activities will take place at Butler Hospital in Providence, R.I., which is the primary adult psychiatric hospital in the area, providing inpatient, outpatient, and partial hospitalization services. Butler Hospital treats approximately 2,500 unique patients per year with bipolar, major depression, and schizophrenia-spectrum disorders.

2.3. Participants and sample size

We aim to recruit n=180 participants over the 5 year project period. Inclusion criteria are: (a) currently hospitalized at Butler Hospital (inpatient or partial hospital programs); (b) meets DSM-5 criteria for schizophrenia-spectrum disorders (schizophrenia, schizophreniform, schizoaffective disorder) or major mood disorders (major depressive disorder, bipolar I/II disorder) based on structured diagnostic interview (other psychiatric comorbidities will be permitted) [41]; (c) receiving ongoing mental health treatment post-discharge; (d) 18 years or older; and (e) ability to speak and read English. Exclusion criteria are: (a) lack of smartphone; (b) homelessness or housing instability preventing reliable follow-up; or (c) discharge to a restricted living setting. Individuals without a smartphone will be excluded to reflect real-world practice in which resources can be limited. However, if an individual does not have a smartphone but is otherwise eligible, we will assist them in obtaining a free phone from an applicable government sponsored program.

2.4. Recruitment

To recruit participants, our staff will screen the electronic health record (EHR) for newly admitted patients with the target diagnoses and meeting other study inclusion criteria. A HIPAA waiver will be obtained from the local IRB to allow screening of potential participants. A research assistant will approach potentially eligible patients during hospitalization to discuss study participation after receiving permission from their treatment team. If interest and potential eligibility are indicated, the research assistant will obtain participant consent based on IRB-approved procedures and conduct the baseline assessment to confirm eligibility.

2.5. Retention and attrition

We aim to achieve high retention rates, even if participants decide to drop out of clinical care at the recruitment site. Staff will call participants to schedule assessment appointments, which can be conducted via telephone, video, or in person, depending on participant preference. Participants will provide at least two contact persons we can ask for information about participants’ whereabouts should we lose contact. We will compensate participants for completion of study assessments, but not for use of the mHealth app, to best align with real-world practice. Participants will be compensated up to $200 total ($50 for each time point) for assessments at baseline, 4, 12, and 24 weeks in the form of gift cards. We will arrange/reimburse travel for participants to return to our offices for follow-up assessments.

2.6. Interventions

All participants will receive unrestricted treatment-as-usual (TAU), consisting of inpatient, partial hospital, and outpatient clinical care following discharge as typically provided in the respective treatment settings. Inpatient treatment usually includes daily medication management sessions with a psychiatrist, community meetings, group and occupational therapy, open air walks, discharge planning, and other services (e.g., medical testing). Partial hospital TAU includes regular medication management sessions with a prescriber, daily individual and group psychotherapy, family meetings, and discharge planning. Outpatient services include medication management overseen by a prescriber, as well as individual and group therapy.

2.6.1. TAU + Transition FOCUS (tFOCUS)

In addition to unrestricted TAU, participants randomized to the active condition will receive the mHealth intervention, tFOCUS. Our scientific collaborators developed and tested FOCUS, a multi-modal 12-week mHealth illness self-management system for people with SMI, showing that FOCUS is feasible, usable, highly engaging, and effective among people with SMI [3436, 42, 43]. We will adapt the FOCUS app for this project, which we will rename Transition-FOCUS (tFOCUS), to engage participants with SMI immediately following a current hospitalization. The tFOCUS system has three elements: (a) application (app); (b) a clinician dashboard; and, (c) mHealth aftercare advisors.

tFOCUS application (app).

The smartphone system includes pre-programmed daily self-assessment prompts and on-demand functions that can be accessed 24-hours a day via a “toolbox” (see Figure 1). Self-management content targets six broad domains: Voices (i.e., coping with auditory hallucinations via cognitive restructuring, distraction, guided hypothesis testing), Mood (i.e., managing depression and anxiety via behavioral activation, relaxation techniques, supportive content), Sleep (i.e., sleep hygiene, relaxation, health/wellness psychoeducation), Social (i.e., cognitive restructuring of persecutory ideation, anger management, activity scheduling), Medication (i.e., behavioral tailoring, reminders, psychoeducation), and Safety (i.e., cognitive and behavioral problem-solving related to suicidality). The app uses audio prompts, video content, cartoons, and written text to deliver coping strategies in the form of interactive questions, suggestions, and guided demonstrations.

Figure 1.

Figure 1.

FOCUS prompt and homescreen with intervention modules.

The app will be installed on participants’ smartphones prior to hospital discharge. Research staff will engage with participants in a shared decision-making conversation to determine which of the six domains are most relevant to them, leading to the selection of three domains. Every day, a morning, afternoon, and evening prompt will occur with each related to a different domain. All participants who report a history of suicidal ideation at baseline will use Safety as one of their check-in domains. App training will include testing one module prior to discharge, as well as testing the toolbox feature.

As part of TAU, standardized suicide Safety Plans [44] listing individualized (a) warning signs; (b) internal coping strategies; (c) social support; and, (d) crisis contacts are typically developed prior to hospital discharge. We will leverage this safety plan by prompting users via the tFOCUS app to refer to this information if they report suicidal ideation (SI). Participants will be contacted and assessed further by an aftercare advisor if they initially report SI with plan or intent and/or if their SI level significantly increases from baseline, as clinically appropriate, and in coordination with their other treatment providers.

Clinician dashboard:

tFOCUS users’ responses to daily self-assessments are securely transmitted to a remote server. The information is processed and displayed on an online dashboard with a real-time summary of engagement, module selection, and reported symptom severity over the last week, which is accessible to authorized staff to inform and enhance their clinical services (see Figure 2). If dashboard data suggest clinical deterioration and that a participant requires additional assistance, staff will call to assess further and provide consultation.

Figure 2.

Figure 2.

Clinician Dashboard.

mHealth aftercare advisors:

tFOCUS participants are engaged throughout the treatment period by trained aftercare advisors (i.e., Master’s level clinicians) to encourage their adherence to tFOCUS by assisting them with all technical and clinical aspects of the intervention and to review safety concerns via the dashboard. Aftercare advisors will maintain regularly scheduled contact with participants throughout the treatment phase and will initiate check-ins if clinical concerns are detected via the dashboard.

2.6.2. TAU + CHECK-IN (comparison)

Participants assigned to the comparison condition, “CHECK-IN,” will receive current best practice hospital discharge planning, including a personalized safety plan, list of crisis resources, outpatient mental health appointments, medication instructions, and referrals to other community treatments/services as needed. We will ensure that the individual’s standard discharge plan includes these elements, and if not, a study aftercare advisor will work with the participant to offer the missing elements of the plan. Participants in both conditions will also receive one call from a study aftercare advisor within 48 hours of their discharge to ascertain their current safety, review their post-discharge plan and upcoming treatment appointments, and provide additional referrals if needed.

2.6.3. Aftercare advisors

The study aftercare advisors will be individuals with at least a Master’s mental health degree or equivalent and experience working with the target clinical population. Their primary role will be to provide participants with both clinical and technical support following hospital discharge. For participants in the CHECK-IN condition, the aftercare advisor will outreach the individual via telephone for a single post-discharge check-in call (within 48 hours). For participants in the tFOCUS condition, aftercare advisors will complete the initial post-discharge check-in call within 48 hours of discharge and cover the same content as the CHECK-IN condition. The aftercare advisors will also review the clinical dashboard to assess app usage, provide technical assistance to the individual if they experience any issues with the app, and encourage continued app usage. Participants in the intervention condition will then receive 6–7 brief (10–15 minutes each) check-in calls of decreasing frequency: weekly calls during the first month (including the 48 hour check-in call), bi-weekly calls during the second month, and 1 call at the end of the third month. Aftercare advisors will review the dashboard prior to each call and provide positive reinforcement for app usage, troubleshoot any barriers to app engagement, and encourage the participant to discuss the skills they have learned from the app modules. Time spent in these activities will be captured in work logs to characterize adoption of tFOCUS for future implementation. Study staff will also complete weekly monitoring of the dashboard and outreach the participant as necessary if any clinical (e.g., SI) or technical issues are identified.

2.7. Randomization

Participants will be randomized in a 1:1 ratio to either TAU+tFOCUS or TAU+CHECK-IN. Randomization will be stratified by: (a) sex at birth (man/woman); (b) diagnosis (primary psychotic/mood disorder); and, (c) admission type (partial/inpatient). Treatment arm assignments will be made in blocks of 4 or 6 participants (randomly ordered) and an independent team member will set up randomization tables in REDCap [45, 46] (a secure web-based service for building and managing online surveys and databases). Other research staff will not have access to the tables. Participants will be randomized following the baseline assessment using the randomize feature in REDCap.

2.8. Study assessment schedule

Follow-up assessments will occur at 6- (mid-treatment), 12- (post-treatment), and 24- (follow-up) weeks post-discharge either in-person or remotely. See Table 1 for summary of study assessments, including primary and secondary outcome measures and hypothesized mediators and moderators.

Table 1.

Study assessments.

Assessments Construct Aim Method Time
Mini Mental Status Examination (MMSE) Cognitive Impairment MOD INT BL
Mini International Neuropsychiatric Interview-7 for DSM-5 Psychiatric Diagnosis MOD INT BL
DSM-5-TR self-report ratings form Psychiatric Symptoms MOD SR BL, 6, 12, 24
Brief Psychiatric Rating Scale (BPRS) Overall Symptoms OC INT BL, 6, 12, 24
Suicidality Composite Suicidality OC INT BL, 6, 12, 24
Treatment History Interview-4 (THI-4) Appointment Attendance OC INT BL, 6, 12, 24
Electronic Health Record (EHR) review Hospitalizations MOD/OC OBJ BL, 6, 12, 24
Medication Adherence Rating Scale (MARS) Medication Adherence OC SR BL, 6, 12, 24
Patient Health Questionnaire-9 (PHQ-9) Depression OC SR BL, 6, 12, 24
General Anxiety Disorder-7 (GAD-7) Anxiety OC SR BL, 6, 12, 24
World Health Organization Disability Assessment Schedule (WHO-DAS 2.0) Physical Disability OC SR BL, 6, 12, 24
Recovery Assessment Scale (RAS) Illness Recovery OC SR BL, 6, 12, 24
Alcohol Use Disorders Identification Test (AUDIT) Alcohol Use OC SR BL, 6, 12, 24
Drug Use Disorders Identification Test (DUDIT) Drug Use OC SR BL, 6, 12, 24
Difficulties in Emotion Regulation Scale – Short Form (DERS-SF) Emotion Regulation MED SR BL, 6, 12, 24
Multidimensional Scale of Perceived Social Support (MSPSS) Social Support MED SR BL, 6, 12, 24
Mental Help Seeking Attitudes Scale (MHSAS) Help Seeking MED SR BL, 6, 12, 24
Behavioral Activation for Depression Scale – Short Form (BADS-SF) Behavioral Activation MED SR BL, 6, 12, 24
Usefulness, Satisfaction, & Ease of Use Questionnaire (USE) App Acceptability SAT SR 6, 12
Client Satisfaction Questionnaire-8 (CSQ-8) Treatment Satisfaction SAT SR 6, 12, 24

BL=baseline, MOD=moderator, OC=outcome, MED=mediator, SAT=satisfaction, INT=interview, SR=self-report, OBJ=objective.

2.9. Quality control and safety

2.9.1. Aftercare advisor fidelity

Clinical supervision will be provided by the study’s lead investigators during weekly supervision meetings. Advisors will log clinical interactions in REDCap, including notes from participant interactions, suicide risk assessment items, and participant reported app usage data. Additionally, check-in calls will be audio-recorded and monitored for protocol fidelity.

2.9.2. Data management and blinding

REDCap will also function as the study’s data repository. Follow-up assessments will be completed by staff members blinded to participants’ assigned study condition. Recruitment numbers and follow-up rates will be monitored and discussed in weekly staff meetings.

2.9.3. Participant safety

All contact from aftercare advisors will include a suicide risk assessment, guided by the Columbia-Suicide Severity Rating Scale [47]. Staff conducting baseline and follow-up assessments will be trained to assess suicide risk. As needed, staff will consult with the licensed study clinicians who will determine next steps. tFOCUS participants’ clinical dashboards will be monitored weekly for indications of safety risk and symptom decline and participants will be contacted as needed to follow-up based on clinical judgement. Participants will be reminded that app data are not reviewed in real-time and provided with instructions via the app on how to access immediate crisis services if needed.

2.10. Statistical power and analyses

We will randomize 90 per group in this 2-arm RCT and will use an intent-to-treat approach to ensure that our computations are conservative. However, we will use 72 as the effective per-group sample size to be conservative in our estimates because 20% attrition is expected throughout follow-up, but we will use all available data when conducting statistical analyses. Concerning our primary outcomes, using Lehr’s equation adjusted for an autocorrelation of Brief Psychiatric Rating Scale (BPRS; [48]) scores r = .60 and assuming a 5% type-I error level [49] we are powered at 80% to detect d = 0.38. If we assume the autocorrelation of BPRS scores can range from r = .50 to .80, the minimum detectable effect ranges from d = 0.28 to 0.41. In prior studies, FOCUS had effects on overall psychopathology ranging from d = 0.44 to 0.73 so we can be confident the current study is powered to detect effects consistent with our previous research.

Concerning our secondary outcomes, because there are multiple variables to consider, we will use a form of the Bonferroni adjustment procedure that accounts for the correlations among the outcome measures [50]. After this adjustment, the usual critical value for hypothesis test (z = 1.96) is moved to z = 2.69. This changes the minimum detectable effects calculated using Lehr’s equation to range from d = 0.38 to 0.50, assuming the correlations range from .50 to .80.

Per intent-to-treat principles, missing data will be handled with multiple imputation or full-information maximum likelihood estimation and assumptions checked with sensitivity analyses. We will test for baseline differences between groups on demographic and clinical characteristics. Because treatment is randomly assigned and stratified, most variables should be evenly distributed. However, we will adjust for baseline values of outcome measures, and variables used in the stratification of randomization: sex at birth, primary diagnosis, and admission type. As participants will be receiving other non-study treatments during the trial, we will conduct secondary outcome analyses that introduce statistical control for post-randomization variables (e.g., other treatments received) as well as explore other possible treatment effect modifiers or subgroups that differentially respond to treatment to inform future intervention studies [51].

2.10.1. Primary outcomes

Compared to CHECK-IN, we hypothesize that tFOCUS will reduce overall psychiatric symptom severity by 24-week follow-up based on the Brief Psychiatric Rating Scale (BPRS; [48]). This will be tested with a generalized linear mixed effect model, with BPRS score at 12- and 24-weeks as repeatedly-observed outcomes, and with treatment group, baseline BPRS, and other variables defined above as covariates [52]. A time factor will be included in the model for week 12- and 24-week assessments.

A linear mixed effect model will be used if the residuals from such a model are normally distributed. Otherwise, an alternative link function will be used. Given these design parameters, the hypothesis that tFOCUS results in greater reduction of overall psychopathology relative to CHECK-IN at week 24 will be tested with the sign and significance of the treatment group indicator, and expressed using marginal effects [53]. We will supplement these with expressions of clinical significance and effect size [54].

2.10.2. Secondary outcomes

Secondary outcomes over the 24 week follow-up include: psychiatric rehospitalization duration and time to rehospitalization, SI severity and suicide behaviors composite (i.e., total number of suicide attempts and related behaviors), first post-discharge mental health follow-up appointment attendance, illness recovery, depression, anxiety, physical disability, alcohol and other substance use, and medication adherence. This outcome set includes continuous measures (e.g., SI severity, number of suicide attempts/behaviors, illness recovery, depression severity) that will be analyzed using the generalized linear mixed model approach described for the primary hypothesis. Other outcomes are discrete, time-to-event, or count outcomes (rehospitalization, duration of rehospitalization episodes, number of suicide attempts/behaviors), which will be addressed with quasi-parametric survival models or Poisson regression (or generalizations to address possible over-dispersion, such as the negative binomial model or zero-inflated versions of the Poisson or negative binomial model).

2.10.3. Mediator and moderator analyses

Compared to CHECK-IN, we hypothesize that tFOCUS will result in improved behavioral activation, emotion regulation, help seeking, and social support. We will also examine the mediational effects of engagement with the tFOCUS app. We will test the effects of treatment condition on potential moderating variables (cognitive impairment, diagnosis: schizophrenia-spectrum vs. mood disorders, number of previous hospitalizations, overall DSM-5, and type of index hospitalization: inpatient vs. partial), which may identify individuals who will benefit more or less from tFOCUS, as compared with CHECK-IN. For these analyses, we will add putative mediators, or moderators (and their interaction with treatment) to outcome models described above. Mediation effects will be described comparing treatment effect estimates with and without statistical control for the putative mediator. Interval estimates for mediation effects will be generated using bootstrap methods [55]. Moderation effects will be expressed similarly and their effects displayed graphically. These analyses are considered post-hoc and exploratory and their main use will be to inform the future development of treatment group targeting or treatment refinement.

2.11. Assessing the Implementation Context: Community Engagement and Process Evaluation

2.11.1. Implementation determinants and outcomes

The Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM [39]) and the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS [40]) frameworks will guide our process evaluation of tFOCUS use during inpatient to outpatient transitions, to plan for a future Hybrid Type 3 implementation trial. We chose this framework because of its focus on innovation and recipients (patients and providers), particularly the subconstructs that may be informing views of the innovation and recipient behavior. RE-AIM guides the evaluation of programs, while i-PARIHS is a determinants framework, examining the potential factors contributing to or preventing implementation success. In terms of RE-AIM, Reach (eligible patients who engage in tFOCUS), Effectiveness (the degree of success of tFOCUS vs CHECK-IN on outcomes), Adoption (engagement in tFOCUS by aftercare advisors), and Implementation (adherence to how tFOCUS should be used by providers and participants) will be assessed via RCT activities. Maintenance (the potential for tFOCUS to become part of the routine organizational practices and policies within outpatient and inpatient departments) will be informed by bi-annual Community Advisory Board (CAB) meetings, where eight sustainability constructs related to the Program Sustainability Assessment Tool will be discussed and action plans created [56]. The CAB will help us understand barriers and facilitators to future implementation and how to design for sustainability from the beginning. In CAB meetings, we plan to talk about how the technology could fit into clinical care, which relates to patient buy-in, challenges for clinicians, more focus on case manager usage, and challenges with digital literacy. We also plan to discuss the role of the aftercare advisors, and to consider billing codes and how those codes could support sustainable use.

Guided by i-PARIHS, the project will also assess stakeholder perceptions of: Innovation (degree of fit with inpatient and outpatient mental health settings and hospital mental health providers and staff), Recipients (tFOCUS participants’ motivation for using the mHealth app and engaging with aftercare advisors), and Context of the setting (organization and cultural values concerning tFOCUS and its implementation feasibility in Butler Hospital and the surrounding Providence, R.I. community).

2.11.2. Community partner interviews and analysis

We will conduct individual, in-depth qualitative interviews until reaching saturation with key community partners (e.g., administrators, public health officials, clinicians, payors, persons with lived experience) and individuals from other diverse hospitals and healthcare systems to assess tFOCUS barriers and facilitators. The data from these assessments will be used to create specific strategies for the scale-up, spread, and sustainability of tFOCUS’s implementation in other healthcare systems.

All interviews will be audio recorded, transcribed, and reviewed for accuracy. Data will be analyzed iteratively during data collection to allow for adaptation of the interview guides as needed and to determine data collection saturation. The coding team will use the Rapid Assessment Process [57] to provide a focused understanding of the tFOCUS implementation context. Data will be consolidated using a summary template of each transcribed interview that abstracts the overall responses to each of the 2–3 main questions within each of the i-PARIHS constructs. The coding team will meet weekly to review the interview summaries and revise the template as needed. Upon template finalization, interviews will be coded by individual team members and the abstracts next placed into a matrix [58] that shows the respondent and main questions within each domain (along with highlighted quotations). The data can then be examined by respondent type and other factors.

Mapping of identified barriers to potential implementation strategies will take place following the steps used in our prior work [59]. We will name, define, and specify these strategies [60] to address the identified challenges, which will be critical for the scaling and widespread use of tFOCUS in future implementation trials.

3. Discussion

Currently available interventions for improving illness self-management and treatment engagement in SMI can be efficacious, but often are not routinely deployed in real-world clinical settings due to barriers related to feasibility, cost, and access. Moreover, limited attention has been paid to supporting engagement and coping during the important transition from acute psychiatric hospitalization to outpatient services. The field is turning to the use of mHealth approaches as a means of more efficiently streamlining service delivery and providing real-time assessment. However, there are notable gaps in existing research.

This study will address these gaps through: (a) enrollment of patients pre-discharge and then targeting the initial weeks/months of aftercare following an acute hospitalization for SMI; (b) integration of mHealth tools into routine discharge planning to support patients’ successful return to the community; (c) improvement of clinician monitoring and feedback through a remote “dashboard,” which will provide real-time reports of symptoms, functioning, and safety (i.e., SI) based on individuals’ mobile data post-discharge; and (d) data collection to generate hypotheses about implementation strategies via a multi-pronged approach involving key partners (CAB for overall guidance, qualitative interviews for in-depth hypothesis generation) and reviewing other supporting data (aftercare advisor work logs, patient EHRs) to support the future adoption of tFOCUS in clinical settings.

3.1. Conclusions and future directions

The products of this project will inform potential tFOCUS scale-up, in combination with successful study outcomes. Payors have expanded and continue to explore models for increasing their coverage of telehealth services since the emergence of the COVID-19 pandemic, which may include reimbursement for selected evidence-based apps and associated services. While these are not firm commitments yet, we intend to continue engaging with payors, leveraging insights and data from the project to advance potential wide-scale adoption and use of the technology.

During the last months of the project, final revisions will be made to the future trial design and protocols, which will provide support for a future Hybrid Type 3 trial based on results of the current Hybrid Type 1 trial. Our preliminary hypotheses in the current project are that tFOCUS will be superior to CHECK-IN in improving post-discharge clinical outcomes. tFOCUS effect sizes may be more modest compared with intensive interventions; however, the ease of disseminating an mHealth intervention could make the “reach” and thus public health impact of the intervention quite large. Based on the results and lessons learned from this study, we should be well-positioned to investigate which methods work best in facilitating effective clinical implementation of tFOCUS across more diverse psychiatric hospitals, in a Hybrid Type 3 trial. Currently, there is no gold standard for psychiatric post-hospital aftercare. Existing psychosocial interventions have had mixed results and those that have shown effectiveness are highly specific to certain patients or highly resource intensive. Thus, studying the best practices for implementation and scalability of mHealth interventions, such as tFOCUS, across healthcare systems warrants further research.

Highlights.

  • Effectiveness-implementation trial of a multi-component mHealth intervention.

  • Adults with serious mental illness transitioning from acute to outpatient care.

  • Potential to inform scale-up across a range of psychiatric and healthcare settings.

Funding:

This project was funded by a grant from the National Institute of Mental Health to Drs. Gaudiano and Moitra (R01 MH130496) and partial funding to support contributions by Drs. Benz and Rabasco (T32 MH126426). The funding agency was not involved in the study design; in the collection, analysis and interpretation of data; in the writing of this report; and, in the decision to submit this article for publication.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflicts of Interest: The authors have no competing interest to declare.

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  • 1.Rehm J and Shield KD, Global Burden of Disease and the Impact of Mental and Addictive Disorders. Current Psychiatry Reports, 2019. 21(2). [DOI] [PubMed] [Google Scholar]
  • 2.Chan JKN, et al. , Excess mortality and life-years lost in people with bipolar disorder: an 11-year population-based cohort study. Epidemiology and Psychiatric Sciences, 2021. 30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Lu L, et al. , Prevalence of suicide attempts in individuals with schizophrenia: a meta-analysis of observational studies. Epidemiology and Psychiatric Sciences, 2020. 29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.John A, et al. , Premature mortality among people with severe mental illness - New evidence from linked primary care data. Schizophrenia Research, 2018. 199: p. 154–162. [DOI] [PubMed] [Google Scholar]
  • 5.Schmutte T, et al. , Comparisons between suicide in persons with serious mental illness, other mental disorders, or no known mental illness: Results from 37 US states, 2003–2017. Schizophrenia Research, 2021. 228: p. 74–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Wander C, Schizophrenia: Opportunities to Improve Outcomes and Reduce Economic Burden Through Managed Care. American Journal of Managed Care, 2020. 26(3): p. S62–S68. [DOI] [PubMed] [Google Scholar]
  • 7.Greenberg PE, et al. , The Economic Burden of Adults with Major Depressive Disorder in the United States (2010 and 2018). Pharmacoeconomics, 2021. 39(6): p. 653–665. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Davis JM, Chen N, and Glick ID, A meta-analysis of the efficacy of second-generation antipsychotics. Archives of General Psychiatry, 2003. 60(6): p. 553–564. [DOI] [PubMed] [Google Scholar]
  • 9.Solmi M, et al. , Safety, tolerability, and risks associated with first- and second-generation antipsychotics: a state-of-the-art clinical review. Therapeutics and Clinical Risk Management, 2017. 13: p. 757–777. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Demissie M, et al. , Psychological interventions for bipolar disorder in low- and middle-income countries: systematic review. Bjpsych Open, 2018. 4(5): p. 375–384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Staring ABP, et al. , Treatment adherence therapy in people with psychotic disorders: randomised controlled trial. British Journal of Psychiatry, 2010. 197(6): p. 448–455. [DOI] [PubMed] [Google Scholar]
  • 12.Faden J, How do we select an antipsychotic for those with schizophrenia? Expert Opinion on Pharmacotherapy, 2019. 20(18): p. 2195–2199. [DOI] [PubMed] [Google Scholar]
  • 13.Sajatovic M, et al. , Treatment adherence in individuals with rapid cycling bipolar disorder results from a clinical-trial setting. Journal of Clinical Psychopharmacology, 2007. 27(4): p. 412–414. [DOI] [PubMed] [Google Scholar]
  • 14.Baskaya E and Demir S, Effect of treatment adherence training given to patients with bipolar disorder on treatment adherence, social functioning and quality of life: A pilot study. Complementary Therapies in Clinical Practice, 2022. 46. [DOI] [PubMed] [Google Scholar]
  • 15.Ruscher SM, deWit R, and Mazmanian D, Psychiatric patients’ attitudes about medication and factors affecting noncompliance. Psychiatric Services, 1997. 48(1): p. 82–85. [DOI] [PubMed] [Google Scholar]
  • 16.Bener A, Dafeeah EE, and Salem MO, A Study of Reasons of Non-Compliance of Psychiatric Treatment and Patients’ Attitudes towards Illness and Treatment in Qatar. Issues in Mental Health Nursing, 2013. 34(4): p. 273–280. [DOI] [PubMed] [Google Scholar]
  • 17.Marcus SC and Olfson M, Outpatient antipsychotic treatment and inpatient costs of schizophrenia. Schizophrenia Bulletin, 2008. 34(1): p. 173–180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Beebe LH, Smith K, and Phillips C, Descriptions and Correlates of Medication Adherence, Attitudes, and Self-Efficacy in Outpatients With Schizophrenia Spectrum Disorders (SSDs). Archives of Psychiatric Nursing, 2016. 30(3): p. 400–405. [DOI] [PubMed] [Google Scholar]
  • 19.Kreyenbuhl J, et al. , Time to discontinuation of first- and second-generation antipsychotic medications in the treatment of schizophrenia. Schizophrenia Research, 2011. 131(1–3): p. 127–132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Vega D, Acosta FJ, and Saavedra P, Nonadherence after hospital discharge in patients with schizophrenia or schizoaffective disorder: A six-month naturalistic follow-up study. Comprehensive Psychiatry, 2021. 108. [DOI] [PubMed] [Google Scholar]
  • 21.Haglund A, et al. , Suicide Immediately After Discharge From Psychiatric Inpatient Care: A Cohort Study of Nearly 2.9 Million Discharges. Journal of Clinical Psychiatry, 2019. 80(2). [DOI] [PubMed] [Google Scholar]
  • 22.Forte A, et al. , Suicidal Risk Following Hospital Discharge: A Review. Harvard Review of Psychiatry, 2019. 27(4): p. 209–216. [DOI] [PubMed] [Google Scholar]
  • 23.Goldacre M, Seagroatt V, and Hawton K, Suicide after-Discharge from Psychiatric Inpatient Care. Lancet, 1993. 342(8866): p. 283–286. [DOI] [PubMed] [Google Scholar]
  • 24.Baldessarini RJ, Tondo L, and Hennen J, Lithium treatment and suicide risk in major affective disorders: Update and new findings. Journal of Clinical Psychiatry, 2003. 64: p. 44–52. [PubMed] [Google Scholar]
  • 25.Nieuwlaat R, et al. , Interventions for enhancing medication adherence (Review). Cochrane Database of Systematic Reviews, 2014(11). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Miklowitz DJ, et al. , Adjunctive Psychotherapy for Bipolar Disorder A Systematic Review and Component Network Meta-analysis. Jama Psychiatry, 2021. 78(2): p. 141–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Cahaya N, et al. , Interventions to Improve Medication Adherence in People with Schizophrenia: A Systematic Review. Patient Preference and Adherence, 2022. 16: p. 2431–2449. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Loots E, et al. , Interventions to Improve Medication Adherence in Patients with Schizophrenia or Bipolar Disorders: A Systematic Review and Meta-Analysis. International Journal of Environmental Research and Public Health, 2021. 18(19). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Graham AK, Lattie EG, and Mohr DC, Experimental Therapeutics for Digital Mental Health. Jama Psychiatry, 2019. 76(12): p. 1223–1224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Alvarez-Jimenez M, et al. , Online, social media and mobile technologies for psychosis treatment: A systematic review on novel user-led interventions. Schizophrenia Research, 2014. 156(1): p. 96–106. [DOI] [PubMed] [Google Scholar]
  • 31.Bell IH, et al. , Ecological Momentary Assessment and Intervention in the Treatment of Psychotic Disorders: A Systematic Review. Psychiatric Services, 2017. 68(11): p. 1172–1181. [DOI] [PubMed] [Google Scholar]
  • 32.Moitra E, Park HS, and Gaudiano BA, Development and Initial Testing of an mHealth Transitions of Care Intervention for Adults with Schizophrenia-Spectrum Disorders Immediately Following a Psychiatric Hospitalization. Psychiatric Quarterly, 2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Curran GM, et al. , Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care, 2012. 50(3): p. 217–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Ben-Zeev D, et al. , Feasibility, Acceptability, and Preliminary Efficacy of a Smartphone Intervention for Schizophrenia. Schizophrenia Bulletin, 2014. 40(6): p. 1244–1253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Ben-Zeev D, et al. , Video-Based Mobile Health Interventions for People With Schizophrenia: Bringing the “Pocket Therapist” to Life. Psychiatric Rehabilitation Journal, 2018. 41(1): p. 39–45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Ben-Zeev D, et al. , Development and Usability Testing of FOCUS: A Smartphone System for Self-Management of Schizophrenia. Psychiatric Rehabilitation Journal, 2013. 36(4): p. 289–296. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Ben-Zeev D, et al. , Mobile Health (mHealth) Versus Clinic-Based Group Intervention for People With Serious Mental Illness: A Randomized Controlled Trial. Psychiatr Serv, 2018. 69(9): p. 978–985. [DOI] [PubMed] [Google Scholar]
  • 38.Wykes T, et al. , Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr Bull, 2008. 34(3): p. 523–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Glasgow RE, Vogt TM, and Boles SM, Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health, 1999. 89(9): p. 1322–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Harvey G and Kitson A, PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implementation Science, 2016. 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Sheehan DV, The Mini International Neuropsychiatric Interview for DSM 5 (MINI). 2016: Harm Research Institute. [Google Scholar]
  • 42.Ben-Zeev D, et al. , mHealth for Schizophrenia: Patient Engagement With a Mobile Phone Intervention Following Hospital Discharge. JMIR Ment Health, 2016. 3(3): p. e34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Ben-Zeev D, et al. , Strategies for mHealth Research: Lessons from 3 Mobile Intervention Studies. Administration and Policy in Mental Health and Mental Health Services Research, 2015. 42(2): p. 157–167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Stanley B and Brown GK, Safety Planning Intervention: A Brief Intervention to Mitigate Suicide Risk. Cognitive and Behavioral Practice, 2012. 19(2): p. 256–264. [Google Scholar]
  • 45.Harris PA, et al. , The REDCap consortium: Building an international community of software platform partners. Journal of Biomedical Informatics, 2019. 95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Harris PA, et al. , Research electronic data capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 2009. 42(2): p. 377–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Posner K, et al. , The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry, 2011. 168(12): p. 1266–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Overall JE, The Brief Psychiatric Rating-Scale (Bprs) - Recent Developments in Ascertainment and Scaling - Introduction. Psychopharmacology Bulletin, 1988. 24(1): p. 97–99. [PubMed] [Google Scholar]
  • 49.Lehr R, Sixteen S-squared over D-squared: a relation for crude sample size estimates. Stat Med, 1992. 11(8): p. 1099–102. [DOI] [PubMed] [Google Scholar]
  • 50.Pocock SJ, Geller NL, and Tsiatis AA, The analysis of multiple endpoints in clinical trials. Biometrics, 1987. 43(3): p. 487–98. [PubMed] [Google Scholar]
  • 51.Kraemer HC, et al. , Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry, 2002. 59(10): p. 877–83. [DOI] [PubMed] [Google Scholar]
  • 52.Everitt BS and Wessely S, Clinical trials in psychiatry. 2008: John Wiley & Sons. [Google Scholar]
  • 53.Williams R, Using the margins command to estimate and interpret adjusted predictions and marginal effects. The Stata Journal, 2012. 12(2): p. 308–331. [Google Scholar]
  • 54.Kraemer HC and Kupfer DJ, Size of treatment effects and their importance to clinical research and practice. Biol Psychiatry, 2006. 59(11): p. 990–6. [DOI] [PubMed] [Google Scholar]
  • 55.Efron B, The jackknife, the bootstrap and other resampling plans. 1982, Philadelphia: Society for Industrial and Applied Mathematics. [Google Scholar]
  • 56.Luke DA, et al. , The Program Sustainability Assessment Tool: A New Instrument for Public Health Programs. Preventing Chronic Disease, 2014. 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Vindrola-Padros C and Johnson GA, Rapid Techniques in Qualitative Research: A Critical Review of the Literature. Qualitative Health Research, 2020. 30(10): p. 1596–1604. [DOI] [PubMed] [Google Scholar]
  • 58.Averill JB, Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qualitative Health Research, 2002. 12(6): p. 855–866. [DOI] [PubMed] [Google Scholar]
  • 59.Elwy AR, et al. , Using formative evaluation methods to improve clinical implementation efforts: Description and an example. Psychiatry Research, 2020. 283. [DOI] [PubMed] [Google Scholar]
  • 60.Proctor EK, Powell BJ, and McMillen JC, Implementation strategies: recommendations for specifying and reporting. Implementation Science, 2013. 8. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES