A Randomized Pilot Study of Rhythms And You (RAY): An internet-based program for bipolar disorder administered with and without clinical helper support in primary care

Abstract

Background:

Rhythms And You (RAY) is an online intervention for bipolar disorders (BD) based on Interpersonal and Social Rhythm Therapy. We examined RAY’s feasibility and acceptability for individuals with BD recruited from primary care. Because online interventions may be more effective when paired with human support, we evaluated RAY with and without weekly brief (~5 minutes) calls from clinical helpers (CH).

Methods:

Participants (n=47) meeting criteria for BD I, II or other specified BD, presenting for primary care, were randomly assigned to RAY, RAY-CH, or Adjunctive Reading Material (ARM) control. RAY consisted of 12 weekly online modules. ARM consisted of 12 weekly emails. Participants were assessed at baseline, 4, 8, and 12 weeks.

Results:

RAY showed high completion rates and Client Satisfaction Questionnaire scores (36/47, 77% and 25.1±5.5, respectively; no group differences). Effect sizes for RAY- CH ranged from small [Internal State Scale-Activation Subscale (ISS-ACT); d=0.3] to large [SF-12 Mental Health Composite Score (SF-12 MHC); d=1.3]. ARM also showed moderate effects (ISS-ACT d=0.7; Quick Inventory of Depressive Symptoms, d=0.8). SF-12 MHC scores showed a time*group interaction (F=2.38, df=6,32, p=0.05) favoring RAY-CH. Number of logins trended toward significant association with improved social rhythm regularity (F=4.09, df=1, 17, p=0.06).

Limitations:

Sample size is small, limiting conclusions that can be drawn.

Conclusions:

Remote delivery of RAY for individuals with BD is feasible and acceptable. More time spent engaged in RAY was associated with greater improvement in social rhythm regularity. Preliminary evidence suggests adding brief human support to RAY may yield better outcomes.

1. INTRODUCTION

Bipolar disorders (BD) are characterized by high rates of impairment (Bonnin et al., 2012), marked psychosocial disability (Ferrari et al., 2016), huge societal costs (Dilsaver, 2011), and significant medical comorbidity (Forty et al., 2014). Approximately a third of individuals with BD are treated exclusively in primary care settings (Cerimele et al., 2013) where the prevalence of BD ranges from 0.5%−4.3% (Cerimele et al., 2014). Medications, which can be prescribed by primary care physicians (PCPs), hasten recovery from illness and improve long-term course of BD illness, but relapse and residual symptoms are common when individuals are treated with pharmacotherapy alone (Judd et al., 2002). Best practices treatment for BD includes adjunctive, BD-specific psychotherapy as a key element of effective care (Yatham et al., 2018), and yet, evidence-based BD-specific psychotherapies are typically unavailable in primary care settings.

In primary care, most practices have few resources available to support psychosocial interventions for BD; online psychotherapy may increase access to this critically important aspect of treatment. A fully automated, self-help intervention is appealing because it imposes almost no staffing burden and facilitates easy access for patients. In the context of the COVID-19 pandemic, remote interventions that facilitate social distancing may be especially desirable. Yet data suggest that online-delivered interventions for patients with psychiatric disorders are more effective when paired with human support provided via email, telephone, or text (Cuijpers et al., 2019; Rollman et al., 2018). This may be especially important for individuals with BD who are often poorly adherent to treatment regimens (Sajatovic et al., 2008).

Adherence is a prerequisite for the effectiveness of any internet intervention (Jimison et al., 2008). Human support may enhance adherence by increasing participants’ accountability to another person (Mohr et al., 2011). Accountability is defined as knowing that one will have to justify use or non-use to another individual at some future time (Lerner and Tetlock, 1999). We previously showed that coaching based on a supportive accountability model (Mohr et al., 2011) produces greater adherence to online depression programs (Mohr et al., 2013). Specialized clinical helpers (CH) who deliver support and an expectation of accountability via telephone and email may improve the effectiveness of online psychotherapy in primary care by promoting patient adherence to the online intervention (Rollman et al., 2018). This approach is likely beneficial for individuals with BD (Fletcher et al., 2018), although the hypothesis has never been formally tested.

Interpersonal and social rhythm therapy (IPSRT) (Frank et al., 2005) is designed to help individuals establish more consistent daily routines. One of the few evidence-based psychotherapies for BD endorsed by international treatment guidelines (Yatham et al., 2018), IPSRT is informed by research implicating circadian dysfunction in the genesis of bipolar mood episodes (Logan and McClung, 2019). The social Zeitgeber (“time keeper”) hypothesis argues that regularity of daily routines of sleep, work, meals, and leisure activities (“social rhythms”) serve to entrain the circadian system, thereby stabilizing mood symptoms (Ehlers et al., 1988; Grandin et al., 2006). IPSRT helps individuals to stabilize their social rhythms. Individuals with bipolar disorders are especially vulnerable to the impact of routine-destabilizing events, including loss of social structures experienced during quarantine or lockdown, increasing the potential salience of a routine-focused, online intervention during the COVID-19 pandemic (Murray et al., 2021).

Information about IPSRT-informed online treatments is sparse. A small open study tested online daily monitoring of social rhythms (Lieberman et al., 2011), but the program did not incorporate other features of the IPSRT intervention. Another group, working collaboratively with patients and IPSRT clinicians, developed a digital version of the Social Rhythm Metric (Monk et al., 2002), a central feature of IPSRT, that provided text message reinforcement for achieving more regular routines and was well received by users (Abdullah et al., 2016).

The majority of online applications for BD that have been developed and evaluated so far are based on psychoeducation or cognitive behavioral therapy (CBT) (Barnes et al., 2015; Gliddon et al., 2019; Proudfoot et al., 2012; Smith et al., 2011; Todd et al., 2014). One small study tested a mindfulness-based program (Murray et al., 2015). Outcomes of online interventions for bipolar disorder have been mixed, with some studies showing positive results (Gliddon et al., 2019; Lauder et al., 2015; Lieberman et al., 2011; Murray et al., 2015) and others no effects (Proudfoot et al., 2012; Smith et al., 2011; Todd et al., 2014). No other trial, to our knowledge, has tested an online program based on principles of IPSRT.

Rhythms And You (RAY) is a fully automated, internet-based program for BD, based on Social Rhythm Therapy (SRT) (Crowe et al., 2020), a component of IPSRT. It consists of social rhythm tracking tools and text-based lessons, videos, interactive exercises designed to increase regularity of daily routines. The primary aim of this pilot study was to evaluate the feasibility and acceptability of delivering RAY, with and without CH support, compared to a bibliotherapy control condition (Adjunctive Reading Materials; ARM), for individuals with BD who receive care from primary care physicians (PCPs). Secondary exploratory aims were to examine the impact of RAY and RAY +CH on symptomatic and functional outcomes to inform the design of future clinical trials.

2. METHODS

The study (Clinicaltrials.gov identifier NCT02448108) was conducted from March 1, 2017 to January 30, 2019 and enrolled patients at a large, urban, university-affiliated, academic primary care practice that uses the Epic electronic medical record (EMR) system. All study procedures were reviewed and approved by the Institutional Review Board of the University of Pittsburgh prior to the start of participant enrollment. Potential participants provided informed written consent after receiving a complete description of the study. The data that support the findings of this study are available from National Institute of Health Data Archive (www.nda.nih.gov) DOI 10.15154/1519309.

2.1. Study design overview

This was a 12-week, randomized, parallel-group study in adult participants with BD I, II, or other specified and related disorders with elevated depressive or hypomanic symptoms. We exposed PCPs to an EMR “Best Practice Alert” (BPA) reminder about their patient’s potential eligibility for our study at the time of the clinical encounter. It launched automatically for all patients aged 18 to 75 years whenever bipolar disorder was on their electronic problem list or lithium was on their electronic medication list (Rollman et al., 2008). We also enrolled study participants through our University’s online research registry and advertisement. All study procedures were completed remotely (by computer and telephone). Consent forms were sent to participants electronically and then reviewed with the participant by telephone.

2.2. Eligibility

Participants were any gender, adults, who met the following criteria: (1) age 18 or older; (2) lifetime diagnosis of BD I, II, or other specified and related disorders according to DSM 5 criteria,(American Psychiatric Association, 2013) (3) Patient Health Questionnaire-9 (PHQ-9) (Kroenke et al., 2001) score ≥9 or Internal State Scale-Activation Subscale (ISS-ACT) (Bauer et al., 2000) score ≥155; (4) access to broadband internet connection and a telephone; (5) receiving care from a primary care physician at the designated study site; (6) ability to read and speak English.

Individuals were excluded from the study if they met any of the following criteria: (1) self-reported visual impairment that would prevent completion of study procedures; (2) diagnosis of psychotic disorder, substance use disorder, or current manic episode; (3) planning to leave the primary care practice in the next 3 months; (4) active suicidal ideation. We originally excluded individuals currently receiving psychiatric services (including psychotherapy) outside of the PCP’s office but removed that exclusion criteria in the middle of the study to enhance recruitment.

2.3. Allocation

As shown in the CONSORT diagram (Figure 1), 206 potential participants agreed to initial screening for inclusion in the protocol during the study period to yield 47 individuals eligible for randomization. Participants were randomly allocated to either Rhythms And You ( RAY; n=16), RAY + Clinical Helper (RAY +CH; n=15) or Adjunctive Reading Material (ARM; n=16) by an independent data manager not otherwise involved in study procedures using a permuted block strategy (Matts and Lachin, 1988).

Figure 1.

CONSORT Diagram

2.4. Outcome Measures

Raters blind to treatment assignment conducted assessments by telephone at baseline and 4, 8, and 12-week follow-up except as indicated below. Self-report measures were collected online. Demographic data were recorded on standardized research forms. Lifetime and current mood diagnoses were assigned using the Structured Clinical Interview for DSM 5, Clinician Version (SCID I) (First et al., 1995) and all other psychiatric diagnoses were ascertained with the Mini-international Neuropsychiatric Interview (MINI) (Sheehan et al., 1998).

The primary outcomes for the trial were feasibility and acceptability. Feasibility was assessed by examining number of dropouts, benchmarked against other eHealth studies which average dropouts of 74% for unsupported interventions and 38% for interventions supported by non-clinical personnel (Richards and Richardson, 2012). Acceptability of the intervention was assessed at visit 12 with the Client Satisfaction Questionnaire (CSQ) (Attkisson and Greenfield, 1994) which yields scores ranging from 8–32 with higher scores indicating greater levels of satisfaction. Mean CSQ scores for in-person treatment are reported as 27.2 (± 4.0) (Attkisson and Greenfield, 1994), thus any score above 23.2 (one standard deviation below the benchmark mean) would indicate satisfactory levels of acceptability. Number of logins and number of modules completed were also evaluated as markers of feasibility and acceptability.

Supportive accountability, another primary outcome that is hypothesized to explain differences in the supported versus unsupported RAY conditions, was measured using a revised version of the Supportive Accountability Questionnaire (SAQ), a 22-item scale (range 1–7) developed by out group (Mohr et al., 2011), with higher scores indicating greater expectancy of supportive accountability.

Symptomatic and functional measures were considered secondary outcomes. Depressive symptoms were assessed using the Quick Inventory of Depressive Symptomatology-Self Report (QIDS-SR) (Rush et al., 2003) and the PHQ-9 (Kroenke et al., 2001). QIDS-SR scores and PHQ-9 scores both range from 0–27, with higher scores indicating greater depression severity. Mania symptoms were rated using the Internal State Scale—Activation Subscale (ISS-AC) which consists of 5 items from the 15-item Internal State Scale. Possible scores range from 0–500 with scores >155 used to discriminate between euthymia and mania/hypomania (higher scores indicate more mania/hypomania) (Bauer et al., 2000). Health-related quality of life was evaluated using the Short Form 12 (SF-12) (Ware et al., 1996), which yields Physical and Mental Health Composite Scores (PCS, MCS). Standardized scores range from 0–100, with a mean of 50 and higher scores indicating better health. The clinician-rated Functioning Assessment Short Test (FAST) (Rosa et al., 2007), a 24-item measure with higher scores indicating more impairment, was used to assess functioning.

Regularity of routines was assessed with the Brief Social Rhythm Scale (BSRS) (Margraf et al., 2016), a 10 item scale that ranges from 0–60, with lower scores indicating more regularity, and the Social Rhythm Metric (SRM) (Monk et al., 2002), a 5-item scale that ranges from 1–7, with higher scores indicating more regularity.

2.5. Interventions

Rhythms and You (RAY):

RAY is a fully automated, internet-based, psychotherapy for bipolar disorder. The intervention is based on the Social Rhythm Therapy (Crowe et al., 2020) component of IPSRT (Swartz et al., 2012). RAY builds on evidence that disordered circadian biology contributes to the development and maintenance of psychiatric symptoms in BD (Logan and McClung, 2019) and that helping individuals develop more regular routines and social patterns can help regulate underlying circadian abnormalities, thereby reducing symptoms and improving outcomes (Gottlieb et al., 2019). Through RAY, participants are encouraged to develop more regular sleep and activity patterns and modify their social interactions to promote mood stability. RAY uses didactic and problem-solving approaches to help individuals regularize their social rhythms to entrain underlying disturbances in circadian and sleep/wake regulation. The application consists of 12 interactive modules delivered over 12 weeks. Lessons are designed to be completed in 15–20 minutes. Lessons consist of text-based didactic material (“slides”), videos/animations, and interactive exercises (“tools”). Badges (“clocks”) are unlocked weekly when participants complete a lesson. Table 1 summarizes individual RAY lessons and screenshots of the application are provided in eAppendix. Participants complete an online version of the SRM (Monk et al., 2002) to monitor regularity of their social routines which takes no more than 1–2 minutes to complete daily.

Table 1.

Description of Rhythms And You (RAY) Lessons

RAY LESSONS
Week	Content
1	Rhythms and You: Meet Your Body Clock
2	Rhythm and Blues: Mood and Daily Rhythms
3	Finding the Beat: Daily Anchors for Rhythms and Mood
4	Bipolar Disorder and Physical Health: Your PCP, Your Rhythms, and You
5	Rhythms and Sleep: Understanding Sleep
6	Even Better Sleep: Rules for Better Sleep
7	Skipped Beats: Finding the Beat after Rhythm Disruptions
8	Social Rhythms: People, Changes, Loss, and Mood
9	Let’s Talk: How Communication Helps Your Rhythms
10	Relationships and Rhythms: Disagreements and Mood
11	Timing Tomorrow: Anticipating Future Challenges
12	Staying in Rhythm: What You Need to Know to Keep the Beat

Clinical Helper (CH):

CH sessions were designed to engage the participant in RAY and assist participants in accessing the site. The intervention consisted of a 30-minute initial phone call followed by weekly 5–10 minute phone calls and reminder emails as needed. The CH manual is based on a pretreatment engagement session which utilizes principles of motivational interviewing (Zuckoff et al., 2008) and was administered by masters and bachelors level research staff who received a half day training as well as ongoing supervision of audio-recorded CH call by the first author (H.A.S.). Formal fidelity monitoring of CH calls was not conducted.

Additional Reading Material (ARM):

ARM consists of written information emailed weekly to study participants on topics related to physical and mental health. Handouts included information about the symptoms of BD, circadian rhythms, social rhythms, approaches to insomnia and maintaining good sleep, and strategies for better interpersonal communication. ARM was designed to approximate the content of RAY, but without the interactive components.

2.6. Statistical analysis

Analyses were conducted using SAS v.9.4 and R version 3.6.3. For baseline demographic and clinical characteristics, Wilcoxon tests and Fisher exact tests were used to compare continuous and categorical variables, respectively. Effect sizes (Cohen’s d) were calculated by subtracting the mean difference between baseline and 12 week scores for two groups, divided by the pooled standard deviation of the two groups. The study was adequately powered (0.82) to detect a large effect size (d=1.2, 2 sample t-test, alpha=0.05) with 16 participants per group and 30% dropout. To achieve 90% power to detect a moderate effect size (d=0.5, 2 sample t-test, alpha=0.05), the study would have needed 123 participants per group (30% dropout), which was beyond the scope of a pilot study.

Longitudinal data were analyzed using an intent-to-treat approach with mixed effect models. Subjects were treated as random effects in repeated measures models. Time and treatment group were treated as categorical variables to allow for non-linear effects. Individual variabilities were taken into account by including random intercept and random slope for linear mixed models and participant as a random term for repeated measures mixed models. Graphs depict raw means of each group at each time point with one standard deviation above and below.

The association between outcomes and login counts were analyzed using Spearman’s rho correlation tests. Linear regression was used to explore mediation effects of login counts on change over twelve weeks. To meet normality assumptions for the linear regression, login count data were log-transformed.

Of note, the trial ended prior to completing expected enrollment (n=75) as the result of difficulties with recruitment. Therefore, the cell sizes were uneven (truncation of randomization occurred in the middle of a block).

3. RESULTS

3.1. Baseline Demographic and Clinical Characteristics

The sample was predominantly female (35/47; 74%), white (31/47; 66%), and with a mean age of 41.8 years (± 13.5). Over half of the sample met criteria for BD I (25/47; 53%). There were no significant differences in baseline demographic and clinical characteristics of the groups except PHQ 9 scores, with higher scores in the RAY group (17.0 ± 5.1) compared to RAY + CH (12.5 ± 4.8) and ARM (12.9 ± 6.1) (Table 2). One participant in the ARM condition met all study eligibility requirements but did not participate in the intervention.

Table 2.

Baseline Demographic and Clinical Information

Variable	RAY (N = 16)	RAY +CH (N = 15)	ARM (N= 16)	P (kruskal wallis)
Gender (n, % male)	5, 0.31	3, 0.19	4, 0.25	0.78
Age (mean, SD)	41.9, 9.97	41.4, 15.44	42.0, 15.42	0.91
Ethnicity (n, %Hispanic)	0,0	0, 0	0, 0	NA
Race
N, % Caucasian	12, 0.75	8, 0.53	11, 0.69	0.43
N, % African American	4, 0.25	4, 0.27	5, 0.31	0.92
N, % other	0, 0	3, 0.20	0, 0	0.04
Marital Status (n, %)
Never Married	11, 0.69	11, 0.73	11, 0.69	0.95
Married/Living as Married	1, 0.06	3, 0.20	2, 0.13	0.53
Separated/Divorced/Widowed	4, 0.25	1, 0.07	3, 0.19	0.40
Education (highest level attained) (n, %)
High School Degree or Less	3, 0.19	1, 0.07	4, 0.25	0.40
Some college or Associates Degree	6, 0.38	7, 0.47	5, 0.31	0.68
Bachelor’s Degree	5, 0.31	5, 0.33	3, 0.19	0.62
Graduate or Professional Degree	2, 0.13	2, 0.133	4, 0.25	0.58
Total Income per Year (n, %)
<$30,000	10, 0.63	13, 0.87	15, 0.94	0.07
$30,000–$74,999	5, 0.31	2, 0.13	1, 0.06	0.16
≥$75,000	1, 0.06	0, 0	0, 0	0.38
Reciving Psychotherapy (n, %)	8, 0.5	8, 0.53	8, 0.5	0.98
Diagnosis (n, %)
Bipolar 1	8, 0.5	8, 0.53	9, 0.56	0.94
Bipolar 2	8, 0.5	7, 0.47	6, 0.38	0.77
Other Bipolar	0, 0	0, 0	1, 0.06	0.38
Patient Health Questionnaire-9 (mean, SD)	17.0, 5.1	12.7, 4.7	12.6, 6.1	0.04
Quick Inventory of Depressive Symptoms (mean, SD)	15.9, 4.5	13.5, 3.5	13.6, 4.4	0.22
Internal State Scale—Activation (mean, SD)	270.8, 121.0	200.0, 149.6	225.0, 93.1	0.19
SF-12 Mental Health Component (mean, SD)	29.1, 8.7	30.9, 8.7	30.0, 9.5	0.76
Brief Social Rhythm Scale (mean, SD)	43.2, 9.2	36.9, 8.0	36.9, 8.0	0.20

3.2. Primary Aims: Feasibility, Acceptability, and Supportive Accountability

Over half (115/206) of potential participants were not enrolled because they could not be reached by research staff after BPA referrals by the PCPs or other methods of recruitment (Figure 1).

To assess feasibility, we assessed dropout rates. Thirty eight percent (6/16) in RAY and 27% (4/15) in RAY + CH dropped out, which is below the expected threshold dropout rates of 74% for unsupported interventions (i.e., RAY) and 38% for supported interventions (i.e., RAY + CH) (Richards and Richardson, 2012). Dropout rates between RAY and RAY + CH were not statistically significant.

To assess acceptability, we examined CSQ scores, number of logins, and number of modules completed. There was a moderate effect (d=0.7) for CSQ scores at week 12 comparing RAY (23.8 ± 5.9) versus RAY + CH (27.6 ± 3.6). Overall CSQ (satisfaction) scores were high 25.1 ±5.5 and exceeded the threshold for acceptability on the CSQ (23.2), but they did not differ significantly across the three groups. There was a moderate effect (d=0.5) for number of logins over 12 weeks in RAY + CH (77.9 ± 72.0) versus RAY (47.2 ± 27.6). Mean number of lessons completed did not differ between RAY (mean 3.9 ± 5.4, range 0–12) and RAY + CH (mean 4.1 ± 5.6, range 0–12). Twenty-seven percent (4/15) and 18% (3/16) of individuals in RAY + CH and RAY, respectively, completed all 12 lessons.

Effect sizes for SAQ scores for RAY + CH versus RAY and RAY + CH versus ARM were large (d=0.9 and d=1.0, respectively). The effect size for RAY versus ARM was small (d=0.1). (See Table 4). SAQ scores showed a trend for change over time (F= 2.83, df= 2, 27, p = 0.08) with mean scores numerically highest at week 12 in RAY + CH (5.0 ± 0.4) compared to RAY (4.4 ± 0.8) and ARM (4.5 ± 0.6). The difference at week 12 trended toward significance, favoring RAY + CH (chi-squared = 5.5, df = 2, p-value = 0.06).

Table 4.

Group Comparison Effect Sizes on Clinical Measures

Variable	Effect Size (Cohen’s D)1,2
	RAY + CH v. RAY	RAY + CH v. ARM	RAY v. ARM	(RAY −CH plus RAY) v. ARM
Quick Inventory of Depressive Symptoms (QIDS)	0.5	−0.3	−0.8	−0.5
Functioning Assessment Short Test (FAST)	0.4	1.0	0.3	0.6
Internal State Scale—Activation (ISS-ACT)	−0.3	−1.0	−0.7	−0.6
Patient Health Questionnaire (PHQ9)	0.1	0.2	0.1	−0.1
Short Form-Physical Health Component (SF-12 PHC)	−0.5	0.1	0.7	0.5
Short Form-Mental Health Component (SF-12 MHC)	1.3	1.4	−0.1	0.5
Brief Social Rhythm Scale (BSRS)	0.4	0.0	−0.4	−0.5
Social Rhythm Metric (SRM)	−0.2	−0.2	−0.1	−0.2
Client Satisfaction Questionnaire (CSQ)3	0.7	0.6	0.0	0.3
Supportive Accountability Questionnaire (SAQ)3	0.9	1.0	−0.1	0.4
# logins over 12 weeks	0.5	NA	NA	NA

¹Calculated by subtracting the mean difference between baseline and week 12 scores for relevant group comparisons, divided by the pooled standard deviation of the two groups, unless otherwise noted.

²Positive valence for Cohen’s D favors first group for “better” outcomes; negative valence favors the second group

³Calculated by subtracting mean week 12 scores, divided by the pooled standard deviation of the two groups

3.3. Secondary Exploratory Aims: Symptomatic and Functional Outcomes

Mean baseline and 12-week scores for outcome measures are summarized in Table 3. Pre- to post- effect sizes in the RAY + CH group ranged from medium to large on five out of eight measures evaluated [QIDS (d= 1.2), FAST (d=0.8), PHQ-9 (d=0.4), SF-12 MHC (d=1.2), BSRS (d=0.4)] with a small effect on one out of eight measures [SRM (d=0.3)]. Effect sizes for pre- to post- changes in the RAY group were small to medium on three measures: ISS-ACT (d=0.3), SF-12 PHC (d=0.5), and SRM (d=0.7). Pre- to post- effect sizes for the ARM group ranged from medium to large on four measures: QIDS (d=0.8), ISS-ACT (d=1.1), BSRS, (d=0.8) and SRM (d=0.7).

Table 3.

Mean Scores on Clinical Measures at Baseline and Week 12 with Pre- to Post-treatment Effect Sizes

Variable (mean, SD)	RAY (N = 16)			RAY +CH (N = 15)			ARM (N= 16)
	Baseline (n=16)	Week 12 (n=12)	Pre to post Cohen’s d1	Baseline (n=15)	Week 12 (n=13)	Pre to post Cohen’s d1	Baseline (n=16)	Week 12 (n=9)	Pre to post Cohen’s d 1
Quick Inventory of Depressive Symptoms (QIDS)	15.9, 4.5	16.0, 6.7	0.0	13.5, 3.5	9.2, 3.0	1.2	13.6, 4.4	9.6, 5.3	0.8
Functioning Assessment Short Test (FAST)	38.9, 17.9	37.9, 18.4	0.1	41.8, 12.9	31.4, 9.8	0.8	32.9, 15.7	36.0, 22.0	−0.2
Internal State Scale—Activation (ISS-ACT)	270.8, 121.0	229.6, 124.1	0.3	200.0, 149.6	210.0, 141.5	−0.1	225.0, 93.1	112.5, 95.8	1.1
Patient Health Questionnaire (PHQ9)	17.0, 5.1	16.1, 5.5	0.2	12.7, 4.7	10.5, 6.0	0.4	12.6, 6.1	12.2, 5.7	0.1
Short Form-Physical Health Component (SF-12 PHC)	40.5, 13.0	33.1, 15.2	−0.5	41.8, 7.0	42.8, 8.9	0.1	35.5, 13.7	35.9, 12.3	0.0
Short Form-Mental Health Component (SF-12 MHC)	29.1, 8.7	28.0, 9.6	−0.1	30.9, 8.7	43.2, 10.0	1.2	30.0, 9.5	31.3, 12.3	0.1
Brief Social Rhythm Scale (BSRS)	43.2, 9.2	43.4, 13.6	0.0	36.9, 8.0	33.5, 10.0	0.4	39.6, 10.9	30.8, 8.9	0.8
Social Rhythm Metric (SRM)	2.4, 1.0	3.4, 1.5	0.7	2.7, 1.3	3.2, 1.8	0.3	3.1, 1.4	4.0, 0.2	0.7

¹Positive valence for Cohen’s D indicates a pre- to post- improvement in symptoms, regardless of the whether higher/lower scores for the measure itself indicate improvement

Group comparison effect sizes using mean standardized change scores from baseline to week 12 are summarized in Table 4.

Effect sizes for RAY + CH to ARM comparisons favored RAY + CH on most (6/10) secondary outcome measures and ranged from small (PHQ-9, d=0.2) to medium (CSQ, d=0.6) to large (FAST, d=1.0; SF-12 MHC, d=1.4; SAQ, d= 1.0). Effect sizes favored ARM (v. RAY + CH) for 3/10 measures: QIDS (d=0.3), ISS-ACT (d=1.0), and SRM (d=0.2) scores. There were no or negligible effects on BSRS (d=0) and SF-12 PHC (d=0.1) scores.

ARM was favored (v. RAY) with medium to large effects on 4/10 measures: QIDS (d=0.8), ISS-ACT (d=0.7), SF-12 PHC (0.7). RAY (v. ARM) was favored with small effects on 2/10 measures: FAST (d=0.3) and BSRS (d=0.4) scores. There were no or negligible effects on PHQ-9 (d=0.1), SF-12 MHC (d=0.1), SRM (d=0.1), SAQ (d=0.1), and CSQ (d=0).

Effect sizes for RAY + CH v. RAY comparisons favored RAY + CH on 6/10 measures and ranged from small (ISS-AC, d=0.3) to medium (BSRS, d=0.4; QIDS, d=0.5; CSQ, d= 0.7) to large (SF-12 MHC, d=1.3; SAQ, 0.9). Effect sizes favored RAY (v. RAY + CH) on 2/10 measures: ISS-ACT (d=0.3) and SRM (d=0.2) scores.

RAY + CH and RAY were combined into a single group (“active”) and then compared to ARM. Combined active group (v. ARM) was favored with small to medium effect sizes on 4/10 measures: FAST (d=0.6), SF-12 MHC (d=0.5), CSQ (d=0.3) and SAQ (d=0.4) scores. ARM (v. active) was favored with small to medium effect sizes on 5/10 measures: QIDS (d=0.5), ISS-ACT (d=0.6), SF-12 PHC (0.5), BSRS (d=0.5), and SRM (d=0.2). There were negligible effects on PHQ-9 (d=0.1).

3.4. Exploratory Inferential Analyses of Secondary Outcomes

In the entire sample, QIDS scores (F=3.27, df=3,24, p=0.038), SF-12 MHC scores (F=3.64, df=3, 25, p=0.026), BSRS scores (F=7.5, df=1, 24, p=0.012), and SRM scores (F=8.73, df= 1,184, p = 0.004) improved significantly over time. ISS-AC scores (F=1.65, df=1, p >.05), FAST scores (F=0.29, df=3 and 14.7, p > .05), and SF-12 PCS scores (F=1.71, df=3,29, p >.05) did not change over time. There were no significant time by group interactions on any of these variables except SF-12 MHC which showed a significant time by group interaction favoring RAY + CH (F=2.40, df= 6, 33, p=0.049) (see Figure 2 and Table 5).

Figure 2.

Change in symptom scores over twelve weeks (raw mean scores with one standard deviation above and below).

Table 5.

Number of Response per Assessment Time Point

	Baseline	Week 4	Week 8	Week 12
RAY+CH	15	13	12	10
RAY	16	12	12	9
ARM	16	10	9	8

We used Spearman’s rho correlation tests to assess the associations between number of logins and change in symptomatic and functional measures over 12 weeks, examining RAY and RAY + CH individually and as a single group (“active” condition). Number of logins was significantly associated with improvement in FAST scores (rho=0.77, p=0.04) in the RAY group and with improvement in QIDS scores (rho=0.46, p=0.05), in the “active” (RAY and RAY + CH) group. There was a trend towards a significant association between number of logins and SF12-MHC scores (rho=0.44, p=0.06) and BSRS scores (rho=0.41, p=0.08) in the “active” group.

A linear regression model fit using stepwise selection to evaluate the relationship among number of logins, baseline BSRS scores, and group, showed a trend toward a negative relationship between number of logins and change in BSRS scores (F=4.09, df=1, 17, p=0.06) such that a higher number of logins was associated with greater decreases (improvement) in BSRS scores.

4. DISCUSSION

The primary aim of this study was to assess the feasibility and acceptability of using a novel, online program for individuals with bipolar disorder followed by a PCP, administered with and without a clinical helper. Feasibility was demonstrated by rates of completion of 63% and 73% for RAY and RAY + CH, respectively, which compare favorably with other online interventions for mood disorders which estimate average levels of retention at 26% and 62% for unsupported and supported interventions, respectively (Richards and Richardson, 2012). Mean number of logins varied from 4 to 7 per week, indicating good engagement with the program, although only about a quarter of participants completed all 12 modules.

Acceptability, as assessed by mean scores on the satisfaction measure (CSQ), was high overall with mean scores of 27.6 ± 3.6 and 23.8 ± 5.9 in RAY + CH and RAY, respectively. These outcomes are comparable to those found in other behavioral health programs (Attkisson and Greenfield, 1994), but with a large effect size (d=0.7) favoring RAY +CH. When comparing RAY and RAY+ CH (“active group”) vs. ARM and RAY + CH vs. ARM acceptability, outcomes favored the active treatment groups with small to medium effects. No benefit of RAY alone was seen in the RAY vs. ARM comparison on CSQ scores (See Table 4). These findings support the conclusion that when RAY and RAY + CH are considered in aggregate, they are more acceptable than ARM. The observation of larger effects for acceptability in RAY + CH versus RAY (d=0.7) is consistent with extant literature showing better engagement (as a proxy for acceptability) in supported versus unsupported eHealth interventions (Richards and Richardson, 2012).

Patterns of effect sizes (valence and magnitude) in the pre- to post- analyses (Table 3) reveal that the RAY + CH group showed robust improvements on most measures whereas the RAY group showed very little change (improvement) in symptom scores over time. The ARM group was intermediate to RAY and RAY+ CH. The ARM group out-performed the RAY-CH group on some metrics, including hypomania and depression scores. ARM also outperformed RAY on hypomania, depression, and social rhythm regularity scores. For RAY+ CH, the biggest effects (large effect sizes) were seen on depression symptoms, functioning, and global mental health. Smaller effects were noted on social rhythm regularity. The estimated effect size for RAY + CH compares favorably with meta-analyses of computer-based interventions for depression which estimate an overall effect size of d=0.56, with somewhat higher estimates for supported studies (supported by a non-professional) of d=0.58 (Richards and Richardson, 2012).

It is important to note that some participants did well with ARM, which consisted of emailed psycho-education about BD, including information about circadian rhythms and mood. This suggests that written information about BD and circadian rhythms may be sufficient to produce positive outcomes in some participants, especially on measures of depression and social rhythm regularity.

Although the primary goal of the study was to examine acceptability and feasibility, exploratory inferential analyses showed that RAY + CH produced significantly greater improvement in global mental health, as measured by the SF-12 MHC, compared to RAY and ARM. However, we did not observe similar differences in measures of depression, hypomania, functioning, or global physical health. Possible explanations for these findings include the fact that entry criteria included either an elevated depression score OR hypomania score, making it more difficult to detect group differences in change on specific symptom measures, whereas a global measure of mental health quality may have been more sensitive to change. Global physical health outcomes are less likely to show changes in response to a psychosocial intervention, which may explain the absence of a signal on this measure (SF12-PHC). Although we did not find significant differences in functioning on the FAST in inferential analyses, there were medium to large effect sizes for group differences on functioning (Table 4) in the RAY + CH versus ARM and RAY plus RAY + CH (“active”) versus ARM comparisons, suggesting that failure to find a group effect for FAST may related to sample size (type II error).

Number of logins was significantly correlated with the change in QIDS scores from baseline to week 12 and showed a trend toward correlation with SF12-MHC and BSRS scores. Consistent with studies showing that greater exposure to an intervention improves outcomes (Donkin et al., 2011; Rollman et al., 2018), our findings suggest that when participants were able to engage in the program, they experienced improvement in mood, global mental health, and regularity of social rhythms. Notably, improvement in BSRS scores was related to login frequency, suggesting that RAY may be especially beneficial for those with irregular social rhythms.

Our findings support studies showing that unguided self-help interventions (i.e., patients work through the application on their own, without any outside help) benefit when co-administered with support from a coach or clinical helper (Cuijpers et al., 2019; Rollman et al., 2018). As posited by Mohr and colleagues (Mohr et al., 2011), it seems likely that human support increases participant accountability to another person, thereby enhancing their adherence to the intervention. Thus, when RAY and RAY + CH were compared on measures of feasibility and acceptability, there was a medium effect for RAY + CH on number of logins (d=0.5) and treatment satisfaction (d=0.7). Similarly, there was a moderate to large effect on supportive accountability for RAY + CH (d=0.9).

Mohr’s Supportive Accountability model further asserts better outcomes are achieved by increased exposure to the intervention (increased “dose”) (Donkin et al., 2011). This may explain differences in RAY + CH v. RAY outcomes on some secondary measures such as the mental health component of health-related quality of life where the effect size was quite large (d=1.3). Although the study was underpowered to detect differences on measures where effects sizes were small to moderate such as the QIDS and ISS-AC, the overall pattern favored RAY + CH over RAY (Figure 2), in accordance with a supportive accountability model.

Our study has several limitations. First, because of difficulties with recruiting from PCP offices, it did not achieve expected recruitment goals. Dropout rates were high (38% and 27% for RAY and RAY +CH, respectively) which tempers conclusions that can be drawn about treatment effects, although we note that these figures are well below what has been previously found in trials of online interventions. The study is small, and the sample is under-powered to detect differences among groups. Analyses have not been corrected for multiple comparisons, therefore type 1 errors may occur. Individuals were receiving care concurrently from PCPs and, in many cases, from specialty mental health services, which makes it difficult to isolate the effects of the active intervention. Study exclusion criteria (comorbid substance use disorders, lack of access to broadband connection) and the fact that we were unable to reach 56% (115/206) of potential participants limit the generalizability of findings.

Feasibility of recruitment in PCP offices was challenging. Although the study received many referrals, over half could not be contacted following initial referral by PCPs. Contributing factors likely included PCPs being too busy to explain/endorse the study after responding to the EMR alert, PCP lack of interest in promoting online psychotherapy for bipolar disorder, patient-participants being overwhelmed, patient-participants’ lack of interest in online psychotherapy for bipolar disorder, low levels of research staffing for a pilot study, and problems with communication between referring PCPs and the research team.

Implementing referral strategies that reduce burden on busy PCP’s may enhance feasibility of using RAY in primary care settings. Future trials should consider positioning RAY+CH as part of a collaborative care model where embedded specialty mental health professionals facilitate access to the program. Deploying the application from a patient-facing web-based portal, either embedded in the EMR or as a free-standing platform, may also improve access to RAY. It seems likely that psychotherapeutic support for BD in primary care should be delivered within such a collaborative care framework, ensuring integration of mental health treatment with primary care services. The question of whether in-person interventions would fare better than an online approach cannot be answered by this study but should be addressed in future trials.

Several participants indicated that they prefer to launch RAY from their phones rather than computers, therefore feasibility and acceptability may be enhanced by moving RAY from an entirely web-based application to a phone-based app. RAY could be further improved by creating an app-based interface for launching CH calls, sending SMS reminder messages to users, and adding gamification features which may increase engagement with mental health technologies (Cheng, 2020). Finally, shortening the program may improve engagement since only approximately one quarter of participants completed all 12 modules.

In conclusion, RAY is a feasible and acceptable online intervention for bipolar disorder based on principles of IPSRT. Acceptability, feasibility, and symptomatic outcomes are enhanced by administering RAY in conjunction with brief supportive calls from a clinical helper to increase supportive accountability, thereby increasing engagement with the intervention. Future research should include conducting an adequately powered RCT to formally evaluate the efficacy and effectiveness of RAY as an adjunctive treatment for bipolar disorder, in both primary and specialty health care settings.

Highlights.

• Rhythms And You (RAY) is an online intervention for bipolar disorders (BD) based on Interpersonal and Social Rhythm Therapy

• Remote delivery of RAY for individuals with BD is feasible and acceptable

• More time spent engaged in RAY was associated with greater improvement in social rhythm regularity

• Preliminary evidence suggests adding brief human support to RAY may yield better outcomes.