The MARIGOLD study: Feasibility and enhancement of an online intervention to improve emotion regulation in people with elevated depressive symptoms

Abstract

Background:

This manuscript describes the first two phases of pilot testing MARIGOLD, an online self-guided positive emotion skills intervention for adults with elevated depressive symptoms, along with enhancements to overcome retention and adherence problems reported in previous research.

Methods:

Adults with elevated depressive symptoms were recruited online and assessed at baseline, post-intervention, 1- and 3-month follow-up. Phase 1 participants (n=58) were randomized to MARIGOLD, daily emotion reporting, or waitlist. Phase 2 participants (n=79) were randomized to MARIGOLD plus one enhancement: online discussion board (ODB), virtual badges (VB), or facilitator contact (FC). Post-intervention interviews assessed acceptability. Intention-to-treat analyses examined retention, adherence, and preliminary efficacy.

Results:

In both phases, retention and adherence did not differ between groups. MARIGOLD skills were highly acceptable, but qualitative results indicate web-based features (e.g., log-in, ODB, VB) require refinement prior to larger testing. Neither phase demonstrated between-group differences in preliminary efficacy. In Phase 1 within-group analyses, MARIGOLD and emotion reporting control demonstrated a similar pattern of findings (stable depressive symptoms, increased positive emotion, decreased negative emotion and stress), whereas the waitlist group significantly increased in depressive mood. Most Phase 2 within-group analyses demonstrated the expected pattern of results (i.e., decreases in PHQ-8 and negative emotion, increases in positive emotion). However, CES-D scores were stable in FC; perceived stress was stable in FC and ODB.

Limitations:

This pilot study is not powered to evaluate efficacy.

Conclusion:

Positive emotion skills, plus enhancements for web-based, self-guided delivery, warrant additional study in people with elevated depressive symptoms.

Introduction

Despite continued innovation in treatment methods, depression remains one of the most prevalent mental disorders in the US (Pratt and Brody, 2014). Epidemiologic data indicate 12-month and lifetime prevalence rates of 10.4% and 20.6%, respectively (Hasin et al., 2018). At both subclinical and clinical levels, depression is detrimental to biopsychosocial functioning and quality of life (Cuijpers et al., 2004; Fogel et al., 2006; Goldney et al., 2004; Gotlib et al., 1995; Hasin et al., 2018; Jeuring et al., 2016; Judd et al., 1994).

These negative outcomes have driven depression research, which has focused largely on the role of negative emotion. However, mounting evidence suggests that positive emotion plays a distinct role in predicting depression (Benning and Ait Oumeziane, 2017; Clark et al., 1994; Clark and Watson, 1991; Nitschke et al., 2001). This new perspective has arisen from findings demonstrating that the relationship between positive and negative emotion is more nuanced than previously assumed, often acting in relative independence (Goldstein and Strube, 1994; Larsen et al., 2017). Several studies show an association between diminished positive emotion and elevated depressive symptoms, independent of levels of negative emotion (Feldman et al., 2008; Nelis et al., 2015; Raes et al., 2012; Werner-Seidler et al., 2013). Additionally, positive emotion is associated with a heightened capacity for coping with stressful life events (Fredrickson et al., 2003; Ong et al., 2006; Tugade and Fredrickson, 2004) and with enhanced biopsychosocial functioning (Carrico and Moskowitz, 2014; Chida and Steptoe, 2008; Danner et al., 2001; Pressman and Cohen, 2005; Wilson et al., 2017). Thus, although gross excesses in positive emotion can be harmful (e.g., in bipolar disorders), evidence increasingly suggests that healthy levels of positive emotion may have a protective role, including in cases of depression.

Emerging research on interventions designed to increase positive emotion shows promise for alleviating depression. Meta-analyses of positive psychological interventions (PPIs) delivered to clinical and community samples have reported small but significant effects for reducing depressive symptoms, particularly in trials with symptomatic (vs. healthy/nondepressed) individuals (Bolier et al., 2013; Sin and Lyubomirsky, 2009). More crucially, PPIs have withstood comparison to traditional psychotherapy and pharmacotherapy by demonstrating comparable efficacy and lasting effects (Chaves et al., 2017; Sin and Lyubomirsky, 2009). Additional studies have determined that such interventions help develop coping skills for depression and prevent relapse (Fredrickson and Joiner, 2002; Ruini and Fava, 2012).

Though effective interventions, including those focusing on positive emotions, have been developed, many individuals with elevated depressive symptoms remain untreated or undertreated (González et al., 2010; Hasin et al., 2018). Online interventions may be instrumental in increasing access to care by overcoming some of the barriers to conventional treatment. Specifically, internet-based interventions offer ease of access and anonymity – features which circumvent stigma and other barriers to face-to-face treatment (Andrews et al., 2001; Demyttenaere et al., 2004; Lintvedt et al., 2008; Rickwood et al., 2007; Saddichha et al., 2014; van Zoonen et al., 2015). Furthermore, online interventions provide a time-efficient and cost-effective means to disseminate treatment to a broader audience (Saddichha et al., 2014).

Online interventions for depression have demonstrated widespread appeal (Markle-Reid and McAiney, 2016; Rice et al., 2014), and meta-analyses have demonstrated their efficacy for reducing depressive symptoms (Christensen et al., 2009; Newman et al., 2011; Van’t Hof et al., 2009). However, the promise of online interventions remains tempered by concerns of poor adherence and retention (Christensen et al., 2006; Eysenbach, 2005; Geraghty et al., 2010). Self-guided PPIs face similar concerns, suggesting that features that improve adherence may be critical for maximizing their benefits (Bolier et al., 2013).

The current study therefore sought to develop an intervention, MARIGOLD (Mobile Affect Regulation Intervention with the Goal of Lowering Depression), to increase positive emotion in adults with depression, using an online, self-guided format that was designed to keep participants engaged (Cheung et al., 2018). This manuscript describes the first two phases of pilot testing: (1) the positive emotion skills tailored for adults with elevated depressive symptoms (MARIGOLD); and (2) three enhancements designed to overcome retention and adherence problems reported in previous Web-based, depression research. The results of Phases 1 and 2 will be used to refine study procedures, intervention delivery and enhancements for subsequent testing in Phase 3.

Methods

Study methods were previously reported in detail (Cheung et al., 2018). A brief description of Phases 1 and 2 is provided below.

Participants

Phase 1 and 2 participants were recruited online (e.g., ResearchMatch, Backpage, Craigslist, Google Ads, stress- and depression-related Reddit threads) and completed telephone screening for eligibility, which required: (a) age ≥18; (b) daily access to internet; (c) own a mobile phone; (d) live in US; (e) able to read and write in English; (f) elevated depressive symptoms (PHQ-8 score ≥5). Phase 1 participants were ineligible for Phase 2. Procedures were approved by institutional review boards at participating institutions, and the study was registered with ClinicalTrials.gov (Phase 1: #; Phase 2: #). All participants provided informed consent.

Randomization and Study Conditions

Consenting participants completed a 7-day run-in period involving daily and momentary assessment of positive and negative emotions. Only participants who completed ≥4 daily emotion reports were randomized. In Phase 1, participants were randomized to MARIGOLD, active control (daily emotion reporting during the intervention period), or waitlist control.

MARIGOLD.

The MARIGOLD intervention is based on revised Stress and Coping Theory and the Broaden-and-Build Theory of positive emotion (Folkman, 1997; Fredrickson, 1998). It teaches eight positive emotion skills: Noticing and Amplifying positive events, Gratitude, Activation, Mindfulness, Positive Reappraisal, Strengths, and Acts of Kindness (Cheung et al., 2018). Previous versions of the positive emotion skills intervention have demonstrated feasibility and efficacy across a broad range of illness contexts (e.g., diabetes, HIV, cancer) and delivery formats (e.g., individual, group, in-person) (Addington et al., 2018; Cheung et al., 2017; Cohn et al., 2014; Moskowitz et al., 2017). In MARIGOLD, the skills are delivered in a web-based, self-guided format. (See Figure 1 for a sample of the skill content.) Participants can access MARIGOLD by logging into the study website on any browser, using any device with an internet connection. Each week for five weeks, participants receive access to didactic material and daily practice exercises for 1–3 skills. They are encouraged to read each skill lesson at least once and to engage in the accompanying home practice exercises (positive events: noticing and savoring, gratitude journal, activation, mindfulness meditation, informal mindfulness, positive reappraisal, strengths, acts of kindness) daily or for as many days as possible during that week. Participants can review skills from a previous week at any time, but they can only access new skills after completing the prior ones. While the entire course is designed to be completed in five weeks, MARIGOLD is a self-paced intervention that allows participants up to seven weeks to complete all eight skills.

Figure 1.

Sample of MARIGOLD skill content as displayed for web-based, self-guided delivery

In Phase 2, participants were stratified by gender and depression severity (PHQ-8 score 5–9, 10–14, ≥15) prior to randomization. All randomized participants received MARIGOLD plus one of three enhancements: access to an online discussion board, virtual badges, or facilitator contact. These features were based on characteristics of the most efficacious online interventions, which established that frequent personal contact, high interactivity, incentivized participation, and multicomponent learning lead to greater engagement and enhanced outcomes (Andersson and Cuijpers, 2009; Cheung et al., 2017; Christensen et al., 2009, 2004, 2002; Clarke et al., 2009; Moskowitz et al., 2017; Proudfoot et al., 2003; van den Berg et al., 2004).

1. On the online discussion board (ODB), moderated by trained study staff, participants were able to interact and discuss skill use with each other via posts, comments, and likes, while maintaining their privacy through the use of avatars and assigned usernames. (See Figure 2.) They were able to personalize their ODB presence by creating a profile, which involved responding to a range of questions such as, What three words describe you? What puts a smile on your face? What’s your favorite quote? The ODB was based on evidence that peer support through online social platforms enhances intervention adherence (Horvath et al., 2013; Perini et al., 2009; Pfeiffer et al., 2011; Proudfoot et al., 2012; Titov et al., 2010).

2. In the virtual badges (VB) condition, participants received a virtual garden that filled with flowers as they accomplished proximal, attainable goals ( e.g., logging into the website for seven consecutive days, completing home practice for four consecutive days). Flowers filled the garden in a random pattern, and participants could rearrange them (see Figure 3). Prior research showed that such benchmarks provide challenge and reward that make tasks more memorable and engaging (Klopfer et al., 2009; Linehan et al., 2011).

3. In the facilitator contact (FC) condition, trained facilitators provided five-minute, weekly telephone check-ins during the intervention period. When facilitators were unable to reach participants by phone, they followed up via email. Facilitators did not provide instruction in the skills, but maintained brief, regular contact with participants in accordance with findings that personal contact increases intervention adherence (Andersson and Cuijpers, 2009; Clarke et al., 2005; Ebert et al., 2018; Proudfoot et al., 2003; Titov et al., 2013).

Figure 2.

Excerpt from Phase 2 ODB

Figure 3.

Example of a VB “garden” arranged by a participant

With the exception of the Phase 1 waitlist control group, all participants in Phase 1 and Phase 2 were asked to complete daily emotion reporting (referred to as “Feel” in participant-facing content), in which they indicated the frequency of 13 positive and 13 negative emotions over the past day (Cheung et al., 2018; Fredrickson, 2013). These groups also received daily email messages, the primary purpose of which was to encourage participants to complete the daily emotion reporting. For participants assigned to groups who also received the MARIGOLD skills (i.e., the Phase 1 intervention group and all three groups in Phase 2), the daily emails also included a reminder to complete the skills and home practice.

Measures

Participants in both phases completed online, self-reported assessments at baseline (BL, pre-intervention), post-intervention (POST), one-month post-intervention (follow-up 1, FU1), and three months post-intervention (follow-up 2, FU2).

Feasibility and acceptability.

Phases 1 and 2 were designed as pilot studies to evaluate the feasibility and acceptability of the MARIGOLD intervention and the three enhancements. As such, outcomes of primary interest were as follows:

Retentio was defined as completing post-intervention and follow-up assessments, categorized at each assessment as a binary outcome (1 = retained, 0 = not retained).

Adherence was defined a priori in two ways: a) number of skills accessed, and b) proportion of intervention completed (i.e., number of pages viewed out of the total possible pages across all skill lessons in the intervention) (Cheung et al., 2018). In secondary analyses, we evaluated an additional aspect of adherence: the total number of home practice exercises completed over the course of the study. The MARIGOLD platform in both Phases 1 and 2 automatically recorded participants’ use of the site, such that all three indicators of adherence were objectively measured; participants were not asked to self-report how many skill lessons they completed or how often they completed home practice.

Acceptability.

Course acceptability was measured quantitatively and qualitatively for both Phases 1 and 2. Participants who received the MARIGOLD intervention completed telephone interviews to obtain their feedback about study procedures, intervention content, and, in Phase 2, the enhancements. Respondents were asked whether they would recommend MARIGOLD skills (a) to a friend and (b) to someone else with depression (0 definitely not, to 10 definitely yes). Open-ended questions asked whether/how MARIGOLD met participants’ expectations, whether/how it was helpful, and whether they disliked any part of the course. In Phase 2, participants were asked about the enhancement they were assigned to receive (i.e., FC, VB, or ODB); items included their rating (0–10, definitely not to definitely yes) for how much they enjoyed the enhancement and open-ended items evaluating any benefits, drawbacks, and effect on their use of the website. The feedback survey also elicited participants’ opinions about surveys/emotion reporting and technological features (e.g., email reminders).

Preliminary efficacy outcomes.

Preliminary efficacy outcomes of primary interest were based on the target sample (depression) and the intervention’s basis in theories related to emotions and stress. Accordingly, we evaluated depressive symptom severity using the PHQ-8 (Kroenke et al., 2009), as well as depressive symptomatology using the CES-D (Radloff, 1977). Participants also reported the frequency of positive and negative emotions over the past week using the 26-item modified Differential Emotions Scale, which provides mean subscale scores for both positive and negative emotions (Fredrickson, 2013). Stress was assessed by the 10-item Perceived Stress Scale (Cohen, 1988).

Statistical Analyses

Data analysis was conducted in SPSS Version 23 using intention-to-treat analyses to examine retention, adherence, and preliminary efficacy.

Feasibility.

We tested whether the proportion of participants retained through the final follow-up assessment (FU2) differed as a function of randomization arm, using dummy variables to represent each arm. In Phase 1, we reported descriptive statistics of participants’ adherence rates in the intervention arm. In Phase 2, we conducted analyses of variance (ANOVA) tests to examine whether adherence differed as a function of enhancement arm (ODB vs. VB vs. FC).

Preliminary efficacy.

For each preliminary efficacy outcome, we conducted multilevel modeling (Singer and Willett, 2003) to assess change in outcome as a function of randomization arm over time. We used months elapsed since baseline assessment as our metric of time. We modeled time (centered at baseline) at Level 1 and randomization arm at Level 2, using dummy variables to represent each arm. For each outcome, we tested for both linear and quadratic change over time and present results for the best-fitting longitudinal change pattern.

Phase 1 Results

Feasibility

Participants.

Phase 1 included 58 participants randomized to the MARIGOLD intervention, daily emotion reporting control, or waitlist control (Figure 4). Most participants (Table 1) were female (65.5%), White (63.8%), and well-educated (67.2% with bachelor’s degree or higher). The mean PHQ-8 score (15.69) fell in the moderately severe range.

Figure 4.

Phase 1 CONSORT diagram.

Table 1.

Phase 1: Demographics by Condition

	Total Sample (N = 58)	MARIGOLD Intervention (N = 26)	Emotion Reporting Control (N = 17)	Waitlist Control (N = 15)
Age Mean (SD)	42.16 (14.47)	45.20 (14.53)	37.24 (15.83)	42.67 (11.95)
Gender N (%)
Male	19 (32.8%)	5 (19.2%)	9 (52.9%)	5 (33.3%)
Female	38 (65.5%)	20 (76.9%)	8 (47.1%)	10 (66.7%)
Another Answer	1 (1.7%)	1 (3.8%)	0 (0%)	0 (0%)
Baseline PHQ-8 Mean (SD)	15.69 (4.00)	15.04 (3.26)	16.94 (4.92)	15.40 (3.96)
Race Ethnicity N (%)
Black or African American	8 (13.8%)	5 (19.2%)	2 (11.8%)	1 (6.7%)
White/European	37 (63.8%)	17 (65.4%)	9 (52.9%)	11 (73.3%)
Asian or Asian American	7 (12.1%)	3 (11.5%)	3 (17.6%)	1 (6.7%)
Native American or Eskimo	4 (6.9%)	2 (7.7%)	1 (5.9%)	1 (6.7%)
Mixed/Other	5 (8.6%)	3 (11.5%)	2 (11.8%)	0 (0%)
Education N (%)
< High School	0 (0%)	0 (0%)	0 (0%)	0 (0%)
High School	2 (3.4%)	0 (0%)	2 (11.8%)	0 (0%)
Some College	11 (19.0%)	6 (23.1%)	3 (17.6%)	2 (13.3%)
College Graduate	29 (50.0%)	12 (46.2%)	7 (41.2%)	10 (66.7%)
Professional Degree (PhD, MD, JD Other)	10 (17.2%)	6 (23.1%)	3 (17.6%)	1(6.7%)
Other	4 (6.9%)	2 (7.7%)	1 (5.9%)	1 (6.7%)
Missing	2 (3.4%)	0 (0%)	0 (0%)	1 (6.7%)
Income N (%)
< $10,000	6 (10.3%)	5 (19.2%)	0 (0%)	1 (6.7%)
$10,000 to < $19,999	10 (17.2%)	3 (11.5%)	4 (23.5%)	3 (20.0%)
$20,000 to < $29,999	11 (19.0%)	4 (15.4%)	2 (11.8%)	5 (33.3%)
$30,000 to < $49,999	8 (13.8%)	4 (15.4%)	4 (23.5%)	0 (0%)
$50,000 to < $74,999	8 (13.8%)	4 (15.4%)	3 (17.6%)	1 (6.7%)
$75,000 to < $99,999	4 (6.9%)	0 (0%)	2 (12.5%)	2 (13.3%)
$100,000 to < $199,999	8 (13.8%)	6 (23.1%)	1 (6.3%)	1 (6.7%)
Over $200,000	1 (1.7%)	0 (0%)	0 (0%)	1 (6.7%)
Missing	2 (3.4%)	0 (0%)	1 (5.9%)	1 (6.7%)

Retention and adherence.

With 89.7% of participants retained through FU2 (Figure 4), retention was very good and did not differ as a function of randomization condition, all ps > .73. Intervention adherence also was very good: Participants randomized to MARIGOLD accessed 7.50 of 8 total skills (SD = 1.45, range: 3 to 8) and completed 87% of intervention content (SD = 24%, range: 12% to 100%). Over the course of the intervention period, they completed a total of 19.73 home practice exercises (SD = 10.19, range: 0 to 40), which equates to approximately 3–4 home practice exercises completed per week.

Acceptability.

In feedback interviews with intervention participants (n = 16), the vast majority indicated they would be highly likely to recommend skills taught in the course to a friend (M = 9.27, SD = 1.16, range: 6–10; n = 1 (6%) rated <8) and to someone else with depression (M = 9.40, SD = 1.12, range: 7–10; n = 2 (13%) rated <8). When asked about favorite skills, the most popular were gratitude (n = 6, 37.5%) and acts of kindness (n = 4, 25%); the remaining skills were selected by 1–2 participants each. Regarding least favorite skills, the most common responses were noticing/capitalizing on positive events, personal strengths, and gratitude (n = 3, 20% each); the remaining skills were selected by 1–2 participants each.

All respondents indicated that the course met or exceeded their expectations and that it was helpful to extremely helpful. In describing how the course was helpful, they indicated that it “trained [my] brain to think differently about life” and taught them “that it’s okay to notice when something good happens and sit with it and savor it instead of brushing it aside,” for example. None indicated dissatisfaction with the course overall, though two did not like components of the technology (i.e., having to log in to the website for each activity). In addition, three participants did not find the email reminders to be helpful; one suggested an option to receive reminders via text message, which we incorporated into Phase 2.

Feedback regarding daily emotion surveys was mixed, with participants’ falling evenly into three groups: favorable (e.g., helpful, important, increased emotional awareness), neutral (e.g., fine, okay), and negative (e.g., inconvenient). Two participants found the emotion surveys aversive because, for example, “Some days I didn’t want to fill them out because I didn’t want to think about feelings of sadness. I was ignoring these feelings previously.”

Preliminary Efficacy (Table 2)

Table 2.

Phase 1: Preliminary Efficacy Outcomes

	BL		POST		FU1		FU2		Condition × Time Interaction (Linear Effect)		Within-Group Trajectory
Outcomes	M	SE	M	SE	M	SE	M	SE	b	p	b	p
PHQ-8
MARIGOLD Intervention	15.46	1.45	14.78	0.99	14.75	0.99	14.70	1.13	Reference		−0.02	.89
Emotion Reporting Control	17.05	1.37	16.18	1.21	15.70	1.25	14.85	1.49	−0.34	.27	−0.36	.17
Waitlist Control	15.46	1.45	15.92	1.28	16.17	1.28	16.61	1.43	0.21	.46	0.20	.42
CES-D
MARIGOLD Intervention	24.03	2.18	24.63	1.99	25.01	2.00	25.68	2.26	Reference		0.29	.40
Emotion Reporting Control	26.41	2.67	26.83	2.44	27.09	2.51	27.55	2.98	−0.09	.88	0.20	.70
Waitlist Control	25.42	2.84	27.93	2.58	29.51	2.58	32.30	2.86	0.91	.11	1.21	.008
Positive Emotion
MARIGOLD Intervention	3.84	0.31	4.23	0.28	4.47	0.28	4.90	0.32	Reference		0.18	.001
Emotion Reporting Control	4.01	0.38	4.50	0.34	4.81	0.35	5.36	0.42	0.05	.57	0.24	.004
Waitlist Control	3.96	0.40	4.07	0.36	4.14	0.36	4.27	0.40	−0.13	.12	0.05	.36
Negative Emotion
MARIGOLD Intervention	4.96	0.30	4.56	0.27	4.32	0.27	3.88	0.31	Reference		−0.18	<.001
Emotion Reporting Control	5.19	0.37	5.06	0.35	4.38	0.34	3.84	0.41	−0.05	.58	−0.24	.001
Waitlist Control	5.16	0.40	4.70	0.33	5.00	0.35	4.89	0.39	0.14	.09	−0.04	.61
Perceived Stress
MARIGOLD Intervention	3.73	0.14	3.48	0.13	3.32	0.13	3.05	0.14	Reference		−0.12	<.001
Emotion Reporting Control	3.75	0.17	3.59	0.15	3.49	0.16	3.28	0.18	.04	.30	−0.08	.009
Waitlist Control	3.64	0.18	3.57	0.16	3.52	0.16	3.44	0.18	.08	.01	−0.04	.08

For all Phase 1 preliminary efficacy outcomes, linear models fit the data best. As such, below we report only the results from the linear models.

Depression.

The Condition × Time interactions for depressive symptom severity (PHQ-8) did not emerge as significant, ps > .27. Simple effects tests revealed that, in all three conditions, depressive symptom severity did not significantly change over time, ps > .17. Tests of between-group differences in depressive symptom severity were not significant at any time point, ps > .51.

Although the Condition × Time interactions for depressive symptomatology (CES-D) were not significant, ps > .10, simple effects tests revealed that waitlist participants showed increased depressive symptomatology over time, b = 1.21, p = .008, whereas MARIGOLD and emotion reporting control conditions showed no significant changes in depressive symptomatology over time, ps > .40. Once again, there were no significant between-group differences at any time point, ps > .20.

Positive emotion.

Although the Condition × Time interactions did not emerge as significant for positive emotion, ps > .12, simple effects tests revealed that MARIGOLD participants showed significant increases in positive emotion over time, b = 0.12, p < .001, whereas positive emotion in the waitlist remained stable over time, b = 0.05, p = .36. Unexpectedly, participants in the emotion reporting condition also showed increases in positive emotion over time, b = 0.24, p = .004, that did not differ from the increases shown by participants in the MARIGOLD intervention arm, p = .57. Once again, there were no significant differences between the three groups in positive emotion at any time point, ps > .17.

Negative emotion.

We found a marginally-significant Condition × Time interaction for negative emotion, reflecting a small difference in the magnitude of change in negative emotion between MARIGOLD participants and waitlist control participants over time, b = 0.14, p = .09. Specifically, simple effects tests revealed that participants in the MARIGOLD intervention showed reductions in negative emotion over time, b = −0.18, p < .001, whereas negative emotion in waitlist participants remained stable, b = −0.04, p = .61. Consistent with positive emotion findings, participants in the emotion reporting condition also showed reductions in negative emotion over time, b = −0.24, p = .001, that did not differ in magnitude from reductions shown by participants in the MARIGOLD intervention condition, p = .58. There were no significant between-group differences in negative emotion at any time point, ps > .13.

Perceived stress.

There was a significant Condition × Time interaction for perceived stress, reflecting a significant difference in the magnitude of change in perceived stress between MARIGOLD and waitlist control participants over time, b = 0.08, p = .01. Specifically, simple effects tests revealed that participants receiving the MARIGOLD intervention showed statistically-significant reductions in perceived stress over time, b = −0.12, p < .001, whereas waitlist control participants showed smaller, marginally-significant reductions, b = −0.04, p = .08. Participants in the emotion reporting condition also showed reductions in perceived stress over time, b = −0.08, p = .009, that did not differ in magnitude from the reductions shown by participants in the MARIGOLD intervention condition, p = .30. There were no significant between-group differences in perceived stress at any time point, ps > .24.

Phase 2 Results

Participants.

Phase 2 included 79 participants randomized to receive the MARIGOLD intervention plus ODB, VB, or FC (Figure 5). Most participants (Table 3) were White (73.4%); just over half were female (58.2%) and well-educated (54.4% with bachelor’s degree or higher). The mean PHQ-8 score (13.43) was in the moderate range.

Figure 5.

Phase 2 CONSORT diagram

Table 3.

Phase 2: Demographics by Condition

	Total Sample (N = 79)	Intervention + FC (N = 27)	Intervention + ODB (N = 26)	Intervention + VB (N = 26)
Age Mean (SD)	34.44 (13.41)	37.33 (15.78)	29.58 (7.84)	36.31 (14.26)
Gender N (%)
Male	31 (39.2%)	11 (40.7%)	10 (38.5%)	10 (38.5%)
Female	46 (58.2%)	15 (55.6%)	15 (57.7%)	16 (61.5%)
Another Answer	2 (2.5%)	1 (3.7%)	1 (3.8%)	0 (0%)
Baseline PHQ-8 Mean (SD)	13.43 (5.73)	13.44 (5.66)	12.77 (5.41)	14.08 (6.23)
Race Ethnicity N (%)
Black or African American	11 (13.9%)	1 (3.7%)	2 (7.7%)	8 (30.8%)
White/European	58 (73.4%)	24 (88.9%)	20 (76.9%)	14 (53.8%)
Asian or Asian American	10 (12.7%)	2 (7.4%)	2 (7.7%)	6 (23.1%)
Native American or Eskimo	2 (2.5%)	1 (3.7%)	0 (0%)	1 (3.8%)
Mixed/Other	5 (6.3%)	1 (3.7%)	4 (15.4%)	0 (0%)
Education N (%)
< High School	1 (1.3%)	0 (0%)	1 (3.8%)	0 (0%)
High School	4 (5.1%)	1 (3.7%)	1 (3.8%)	2 (7.7%)
Some College	23 (29.1%)	9 (33.3%)	7 (26.9%)	7 (26.9%)
College Graduate	28 (35.4%)	9 (33.3%)	8 (30.8%)	11 (42.3%)
Professional Degree (PhD, MD, JD Other)	15 (19.0%)	5 (18.5%)	5 (19.2%)	5 (19.2%)
Other	8 (10.1%)	3 (11.1%)	4 (15.4%)	1 (3.8%)
Missing	0 (0%)	0 (0%)	0 (0%)	0 (0%)
Income N (%)
< $10,000	11 (13.9%)	4 (14.8%)	3 (11.5%)	4 (15.4%)
$10,000 to < $19,999	9 (11.4%)	2 (7.4%)	2 (7.7%)	5 (19.2%)
$20,000 to < $29,999	9 (11.4%)	3 (11.1%)	5 (19.2%)	1 (3.8%)
$30,000 to < $49,999	14 (17.7%)	5 (18.5%)	6 (23.1%)	3 (11.5%)
$50,000 to < $74,999	7 (8.9%)	4 (14.8%)	1 (3.8%)	2 (7.7%)
$75,000 to < $99,999	13 (16.5%)	6 (22.2%)	2 (7.7%)	5 (19.2%)
$100,000 to < $199,999	12 (15.2%)	2 (7.4%)	6 (23.1%)	4 (15.4%)
Over $200,000	2 (2.5%)	0 (0%)	1 (3.8%)	1 (3.8%)
Missing	2 (2.5%)	1 (3.7%)	0 (0%)	1 (3.8%)

Retention and adherence.

Retention was adequate, with 49.4% of participants retained through FU2 (Figure 5), and did not differ as a function of randomization condition, ps > .13. Intervention adherence was good: Overall, participants accessed 5.90 of 8 total skills (SD = 2.85, range: 0 to 8) and completed 70% of intervention content (SD = 36%, range 0% to 100%). On average, participants completed a total of 13.48 home practice exercises (SD = 16.93, range: 0 to 102), which amounts to approximately 2–3 home practice exercises completed per week. We did not find differences as a function of enhancement arm for the number of skills accessed (M_ODB = 5.50, SE = 0.56; M_VB = 5.96, SE = 0.56; M_FC = 5.50, SE = 0.56), the proportion of intervention content completed (M_ODB = 72%, SE = 7%; M_VB = 72%, SE = 7%; M_FC = 66%, SE = 7%), or the total number of home practice exercises completed (M_ODB = 12.46, SE = 3.31; M_VB = 17.54, SE = 3.31; M_FC = 10.56, SE = 3.25), all ps > .41.

Acceptability.

Most Phase 2 participants who completed a feedback interview (n = 40) indicated they would be highly likely to recommend skills taught in the course to a friend (M = 8.81, SD = 1.74, range: 2–10; n = 6 (15%) rated <8) and to someone else with depression (M = 8.80, SD = 1.85, range: 2–10; n = 5 (13%) rated <8). As in Phase 1, Phase 2 participants demonstrated heterogeneity regarding skill preference. The most common favorite skills were: noticing positive events (n = 7; 17.5%), mindfulness in daily activities (n = 7; 17.5%), acts of kindness (n = 7; 17.5%, each), and gratitude (n = 6; 15.0%). There was greater consensus among participants regarding their least favorite skill, with 9 (27.3%) selecting breath awareness (formal mindfulness meditation) as their least favorite skill.

The vast majority (n = 38, 95%) indicated that the course met their expectations. All were able to provide examples of how the course was helpful. For example, one participant stated, “It’s well put together. It went in depth enough and was varied enough that it felt useful and applicable; it didn’t feel like a one-shot gimmick. It felt scientific rather than like pop psychology….” Another stated that “[the course] was very clear in terms of what the strategies were and provided practical situations/examples.” Only one participant indicated overall dissatisfaction with the course, stating that the content was “static and a little boring.” Two participants were dissatisfied with the home practice exercises, referring to them as “a bit tedious” and “clunky.”

Perspectives on the daily emotion surveys in Phase 2 again were mixed, with approximately two-thirds (n = 20) describing them favorably (e.g., helpful) and one-third (n = 12) negatively (e.g., redundant, monotonous, difficulties with timing or technology). Some participants stated that completing daily emotion surveys “reinforced” the skills, such as mindfulness or positive reappraisal. Several noted that the surveys increased their selfawareness; some found this helpful, but others disliked being reminded of their negative emotions.

The online discussion board received minimal use. Although 88% created a profile, very few made statements on the board (n = 2, 8%), commented on (n = 2, 8%) or liked (n = 7, 27%) others’ posts on the board. Not surprisingly, ODB also was the enhancement that was most negatively rated (M = 3.88, SD = 2.75, range: 0–8; n = 9 (75%) rated <7). Negative feedback focused on the lack of activity on the board, its failure to capture participants’ attention, or difficulties navigating the board. Two participants stated that they appreciated seeing that others were also in the course and getting feedback from other participants. Four participants indicated that an improved ODB – for example, one that was more personalized and user-friendly, with more active members – could be a useful feature.

Most VB participants (n = 24, 92%) earned at least one badge; on average, they earned a total of 7.5 badges (range: 0 to 57), including a mean of 3.81 different badges (range: 0 to 8). The most commonly received badges were: reading a skill (n = 22, 85%), reading all of the skills (n = 15, 58%), recording emotions ≥4 days in the past week (n = 12, 46%), and logging into the website for 7 consecutive days (n = 11, 42%). Of all the enhancements, ratings of the virtual badges were the most mixed (M = 4.58, SD = 3.57, range: 0–10), with approximately one-third indicating they did not enjoy them (n = 3 rated 0–2), another third rating them neutrally (n = 3 rated 5–6), and another third indicating high levels of enjoying the “garden” (n = 4 rated 7–10). Those providing negative ratings indicated that they “didn’t notice” the virtual badges, while those providing positive ratings liked the “reinforcement” of the badges and the visual indicators of their progress.

On average, participants assigned to receive facilitator contact completed 3.15 total phone calls; 5 (19%) completed none. Most FC participants indicated that they enjoyed that enhancement at least somewhat (M = 6.81, SD = 2.06, range: 2–10; n = 3 (13%) rated <7). Most found that the contact (whether by phone or email) maintained their motivation or “good habits” for staying engaged in MARIGOLD. However, a few found the time commitment or scheduling to be a “hassle,” while two mentioned that facilitator contact promoted feelings of guilt associated with missing calls or feeling “like a professor was checking on my work.”

Preliminary Efficacy Outcomes (Table 4)

Table 4.

Phase 2: Preliminary Efficacy Outcomes

	BL		POST		FU1		FU2		Within-Group Trajectory (Time) (Linear Effect)		Within-Group Trajectory (Time2) (Quadratic Effect)
Outcomes	M	SE	M	SE	M	SE	M	SE	b	p	b	p
PHQ-8
Intervention + ODB	12.83	1.11	7.42	1.24	6.11	1.27	5.91	1.54	−3.66	<.001	0.45	.02
Intervention + VB	14.02	1.11	9.69	1.17	8.60	1.19	8.28	1.40	−2.76	.001	0.32	.05
Intervention + FC	13.50	1.11	10.12	1.10	9.06	1.111	8.07	1.29	−2.07	.009	0.19	.22
CES-D
Intervention + ODB	31.21	2.10	22.48	2.35	20.70	2.41	21.61	2.98	−5.92	.001	0.81	.02
Intervention + VB	34.27	2.10	27.88	2.22	25.95	2.24	24.34	2.69	−3.73	.01	0.35	.24
Intervention + FC	32.48	2.10	29.60	2.07	27.74	2.09	23.35	2.47	−1.10	.51	−0.16	.64
Positive Emotion
Intervention + ODB	4.50	0.29	4.93	0.27	5.13	0.30	5.53	0.38	0.21	.005	-	-
Intervention + VB	3.82	0.29	4.25	0.26	4.45	0.28	4.85	0.35	0.21	.004	-	-
Intervention + FC	4.31	0.29	4.61	0.26	4.75	0.27	5.02	0.32	0.15	0.04	-	-
Negative Emotion
Intervention + ODB	4.41	0.31	3.02	0.34	2.79	0.35	3.39	0.36	−0.98	<.001	0.15	< .001
Intervention + VB	4.88	0.31	3.51	0.32	3.26	0.33	3.15	0.43	−0.95	<.001	0.14	.008
Intervention + FC	4.30	0.31	3.50	0.30	3.33	0.31	3.50	0.39	−0.54	.02	0.07	.13
Perceived Stress
Intervention + ODB	20.54	1.40	17.53	1.56	16.48	1.60	15.21	1.96	−1.69	.10	0.13	.54
Intervention + VB	23.74	1.40	18.92	1.47	17.69	1.49	17.30	1.78	−2.98	.005	0.35	.10
Intervention + FC	22.04	1.41	19.35	1.40	18.68	1.38	18.53	1.63	−1.82	.09	0.22	.29

In Phase 2, linear models fit the data best for positive emotion, and we report only the results from the linear models for this outcome. Quadratic models fit the data best for depression (both PHQ-8 and CES-D), negative emotion, and perceived stress, and we report the results from the quadratic models for these outcomes.

Depression.

Change in depressive symptom severity (PHQ-8) did not differ as a function of enhancement arm (Condition × Time interaction (linear effect) and Condition × Time² interaction (quadratic effect): ps > .42). Simple effects tests revealed that participants in all three enhancement arms (intervention + ODB, intervention + VB, intervention + FC) showed significant decreases in depressive symptom severity over time (linear effects: ps < .009, quadratic effects: ps = .02, .05, and .22 respectively). We did not find significant between-group differences in depressive symptom severity at any time point, ps > .22.

Change in depressive symptomatology (CES-D) demonstrated a trend towards differing as a function of enhancement arm (Condition × Time interaction (linear effect): F(2, 140) = 2.42, p = .09, Condition × Time² interaction (quadratic effect): F(2, 134) = 2.31, p = .10). Specifically, simple effects tests revealed that participants in the ODB and VB enhancement arms showed significant decreases in depressive symptomatology over time (linear effects: ps < .01, quadratic effects: ps = .02, and .24 respectively), whereas participants in the FC arm showed stable levels of depressive symptomatology over time (linear and quadratic effects: ps ≥ .51). In addition, participants in the FC arm had higher levels of depressive symptomatology relative to those the ODB arm at POST and FU1, although between-group differences were only marginal in statistical significance, ps = .07 and .08 respectively. We found no other between-group differences in depressive symptomatology at any time point, all ps > .26.

Positive emotion.

Change in positive emotion did not differ as a function of enhancement arm (Condition × Time interaction (linear effect), p = .72). Simple effects tests revealed that participants in all three enhancement arms showed significant increases in positive emotion over time, ps < .04. We did not find significant between-group differences in positive emotion at any time point, ps > .21.

Negative emotion.

Change in negative emotion did not differ as a function of enhancement arm (Condition × Time interaction (linear effect) and Condition × Time² interaction (quadratic effect): ps > .26). Simple effects tests revealed that participants in all three enhancement arms showed significant decreases in negative emotion over time (linear effects: ps < .02, quadratic effects: ps = .001, .008, and .13 respectively). We did not find significant between-group differences in negative emotion at any time point, ps > .44.

Perceived stress.

Change in perceived stress did not differ as a function of enhancement arm (Condition × Time interaction (linear effect) and Condition × Time² interaction (quadratic effect): all ps > .57). However, simple effects tests revealed that only participants in the VB enhancement arm showed statistically significant decreases in perceived stress over time (linear effect: p = .005, quadratic effect: p = .10). The FC and ODB enhancement arms showed marginally-significant decreases in perceived stress over time (linear effects: ps = .09 and .10, respectively). We did not find significant between-group differences in perceived stress at any time point, ps > .29.

Discussion

Phases 1 and 2 of the current study involved initial pilot testing and iterative refinement of the MARIGOLD intervention, a web-based, self-guided delivery of positive emotion skills for people with elevated depressive symptoms. Phase 1 demonstrated the feasibility of recruiting and retaining participants randomized to receive the MARIGOLD intervention or a control. Phase 2 demonstrated the feasibility of delivering the MARIGOLD intervention with enhancements designed to increase retention and adherence. Inability to contact potential participants by telephone was a primary barrier to enrollment in both Phases 1–2, suggesting that web-based, self-guided interventions might be most accessible if the enrollment process is automatized, rather than requiring enrollment through a study coordinator. Phase 3 of the MARIGOLD study therefore will test a more streamlined and automated enrollment process. This also will increase the external validity of the Phase 3 trial of MARIGOLD, in that self-guided interventions frequently are disseminated to the public via platforms that do not require telephone contact with a staff member (e.g., independently downloading a program from a mobile app store).

While Phase 2 retention – i.e., the proportion of participants completing the post-intervention and follow-up assessments – was lower than in Phase 1, it was comparable to other online intervention studies (Christensen et al., 2009; Eysenbach, 2005; Geraghty et al., 2010). The recruitment methods and sociodemographic characteristics of both samples were comparable and thus are unlikely to account for the decline in retention and adherence from Phase 1 to Phase 2. These differences may stem from artefacts of the relatively small samples or changes in the online platform and survey software between Phases 1 and 2 (Qualtrics and REDCap, respectively). For example, only the Phase 1 platform displayed participants’ progress in each skill (see Figure 1), which might have encouraged more complete participation. The Phase 2 assessments included 7 additional scales not administered in Phase 1 (Cheung et al., 2018); retention differences may therefore be attributed to differences in assessment length.

Furthermore, differences between Phases 1 and 2 may result from the addition of enhancements in Phase 2. Although the enhancements were intended to improve retention and adherence, they could have had the unintentional and paradoxical effect of deterring retention, for example, by overwhelming participants or diffusing their attention. This possibility will be tested in Phase 3 (N = 600), which includes an emotion-reporting only group and an intervention-only group, in addition to groups receiving one or more of the three enhancements (Cheung et al., 2018). In addition, randomization of Phase 3 participants will be stratified by baseline depression severity, allowing us to examine whether retention, adherence, or efficacy differ by clinical presentation.

In both Phases 1 and 2, participants accessed the majority of the MARIGOLD content (i.e., skill lessons), but their usage of the online home practice exercises was lower than expected. Some participants may have practiced the skills outside of the web-based platform, such as a pen-and-paper gratitude journal or mentally savoring positive events without recording them on the study website. However, it is likely that many participants read the didactic material without incorporating either formal or informal skill practice. Compliance with and quality of psychotherapy homework have demonstrated small to moderate effects on therapy outcomes across a range of studies targeting depression (Kazantzis et al., 2016; Mausbach et al., 2010), and mobile and web-based platforms have been proposed as user-friendly mechanisms for supporting homework adherence (Tang and Kreindler, 2017). Therefore, studies of m/eHealth interventions should continue to explore the role of home practice and strategies for increasing it. The larger sample (N = 600) planned for Phase 3 of the MARIGOLD study will facilitate further investigation of home practice adherence and acceptability, differences in home practice associated with receipt of varying enhancements, and relationships between home practice and efficacy of MARIGOLD for improving key outcomes such as depression and positive affect.

Future studies will also need to examine research-specific and participant characteristics that influence the efficacy of web-based programs. For example, the small sample size in the Phase 2 group assigned to receive the ODB may have contributed to its low usage rates. The fact that most participants completed their profile suggests that they may have had initial interest in the ODB, but additional activity on the board (e.g., messages, likes) never became prolific or interesting enough to engage participants. This is consistent with the qualitative feedback from the ODB group. Thus, discussion boards may be useful only when the participant pool is large enough and active enough to generate an engaging level of discussion board content.

Supportive features also may need to vary based on clinical needs. For example, web-based peer support may benefit individuals who are socially isolated (Callan et al., 2017). Recent reviews of technology-based programs have indicated that clinician support may be particularly important for people whose depressive symptoms meet diagnostic criteria, while self-guided programs may better serve individuals with subthreshold symptoms (Callan et al., 2017). Additionally, enhancement features may have different appeal based on personal characteristics, such as the positive association between the efficacy of gamification and responsiveness to external motivation (Brown et al., 2016). Thus, participant- (e.g., symptom severity, social isolation, motivation/reward preferences) and study-level (e.g., sample size) moderators should be examined in future trials and meta-analyses.

Quantitatively and qualitatively, participants indicated that the positive emotion skills were highly acceptable. Nonetheless, there was great heterogeneity in which skills participants selected as their favorite. This is not surprising, given prior evidence that different skills work for different people (Lyubomirsky et al., 2005). Future studies therefore should continue offering positive emotion skills as a multicomponent intervention, to allow participants to experience a range of approaches for increasing positive emotions. Participants can select the skills that work best for them in different situations over time. Future studies with larger samples may benefit from examining moderators to identify which skills work best for whom and in which contexts. This would allow clinicians to proactively suggest specific skills to people who are most likely to enjoy and benefit from them.

Participants in both phases provided some critical and constructive feedback, including minor inconveniences related to the technology (e.g., log-in). Some Phase 2 participants reported that ODB and VB were not engaging or user-friendly; some did not even notice these features, which may partially explain the low rates of ODB usage. This suggests that users may need more formal introduction to these enhancements and that these features need to be more centrally located on the platform, which we have integrated into Phase 3. For example, participants in Phase 3 will see the ODB (“Activity Feed”) and VB (“Personal Garden”) at the top of the homepage (see Figure 6) and can access brief videos and written instructions for using these features. As technology becomes increasingly integrated into daily life, with applications and users becoming more advanced, web-based interventions will need to mimic the features of popular, publicly available mobile- and internet-based applications.

Figure 6.

Phase 3 homepage display of access to VB (“personal gardens”) and ODB (“activity feed”)

Feedback on the daily emotion survey was mixed. Across the two phases, most participants described it as useful or neutral, but others described it as a hassle or even unpleasant. The Phase 1 group randomized to daily emotion reporting only demonstrated within-group improvements in several outcomes – namely, increases in positive emotion and decreases in both negative emotion and perceived stress. Given that depression is associated with alexithymia and deficits in emotional awareness or clarity (Visted et al., 2018), it is conceivable that daily emotion reporting contributed to improvements in these facets of emotion regulation. These plausible mechanisms were not measured in the current study, and thus further research is needed to better understand the effects of daily emotion reporting. Nonetheless, these results suggest that daily emotion reporting is, at minimum, an appropriate and tolerable active control group. Conversely, the waitlist control group in Phase 1 demonstrated a statistically significant within-group increase in CES-D (but not PHQ-8) scores. Future trials with people with elevated depressive symptoms should therefore proceed cautiously when selecting control group(s), giving consideration to including only active control groups rather than using a waitlist control design.

Based on within-group analyses, Phase 1 provided preliminary evidence that positive emotion skills may increase positive affect and decrease negative affect and perceived stress. Phase 2, in which all three groups received the MARIGOLD intervention, demonstrated similar results, with additional findings of within-group decreases in depressive symptoms/mood. Although not all within-group changes over time were significant in Phase 2, no between-group differences in feasibility or efficacy outcomes were uncovered in Phase 2. This suggests that positive emotion skills delivered in a web-based, self-guided format have potential to improve psychological outcomes in adults with elevated depressive symptoms, while the online discussion board, virtual badges, and facilitator contact warrant further investigation of their effects on retention and adherence.

Limitations

Both Phases 1–2 represent formative processes of examining the feasibility and acceptability of the MARIGOLD intervention and enhancements. As such, these pilot tests were not designed to provide the statistical power necessary to evaluate efficacy. Future studies, such as the forthcoming MARIGOLD Phase 3, are needed to investigate whether the enhancements affect adherence and whether the positive emotion skills (with or without enhancements) improve depression and other outcomes in adults with elevated depressive symptoms.

Furthermore, results cannot be generalized to populations beyond those represented in this sample. Future studies will particularly need to examine feasibility, acceptability, and efficacy of MARIGOLD among people of color and participants who have not completed college. Although the majority of participants was female, both Phase 1 and 2 samples included a sizable proportion of males (32.8%, 39.2%, respectively). Nonetheless, the preponderance of female participants is externally valid, given that rates of depression are higher among females (Pratt and Brody, 2014). Moreover, in both samples, around half reported annual income less than $50,000, indicating that MARIGOLD may be appealing across a range of economic levels. This further suggests that web-based, self-guided positive emotion skills programs may play an important role in filling gaps in treatment for those who lack financial resources to obtain in-person care.

Conclusions

Positive emotion skills, plus enhancements for web-based, self-guided delivery, warrant additional study in larger samples of people with elevated depressive symptoms. Future trials should consider using an active control condition, such as daily emotion reporting, in lieu of waitlist control. After incorporating feedback and refining the study based on Phase 1–2 results (e.g., enhancing visibility of ODB and VB, eliminating telephone requirement for enrollment), Phase 3 of the MARIGOLD study will conduct a factorial trial in which participants (N=600) are randomized to one of nine groups: emotion reporting control, intervention only, or intervention plus 1–3 enhancements (ODB ± VB ± FC) (Cheung et al., 2018). This will facilitate further examination of the effects of daily emotion reporting, along with investigation of varying combinations of enhancements for increasing feasibility and efficacy of the positive emotion skills. This three-phase process will optimize the web-based delivery format of the MARIGOLD intervention for people with elevated depressive symptoms.

Highlights.

• Online self-guided positive affect skills are acceptable in adults with depression.

• Enhancements to encourage retention and adherence warrant further study.

• Larger trials are necessary to examine efficacy.

Abbreviations

BL

baseline (assessment 1)

CES-D

Center for Epidemiologic Studies – Depression scale

FC

facilitator contact

FU1

follow-up 1 (assessment 3)

FU2

follow-up 2 (assessment 4)

MARIGOLD

Mobile Affect Regulation Intervention with the Goal of Lowering Depression

PHQ-8

Patient Health Questionnaire – 8 items

POST

post-intervention/control (assessment 2)

PPI

positive psychological intervention

ODB

online discussion board

VB

virtual badges