A randomized controlled trial of the online OurRelationship program with varying levels of coach support

Abstract

Online programs that reduce relationship distress fill a critical need; however, their scalability is limited by their reliance on coach calls. To determine the effectiveness of the online OurRelationship program with varying levels of coach support, we conducted a comparative effectiveness trial with 740 low-income couples in the United States. Couples were randomly assigned to full-coach (n_couples = 226; program as originally designed), automated-coach (n_couples = 145; as a stand-alone program with tailored automated emails only), contingent-coach (n_couples = 145; as an adaptive program where tailored automated emails are followed by more coaching if couples did not meet progress milestones), or a waitlist control condition (n_couples = 224). All analyses were conducted within a Bayesian framework. Completion rates were comparable across conditions (full-coach: 65 %, automated-coach: 59 %, contingent-coach: 54 %). All intervention couples reported reliable pre-post gains in relationship satisfaction compared to waitlist control couples (d_full = 0.46, d_contingent = 0.47, and d_automated = 0.40) with no reliable differences across intervention conditions. Over four-month follow-up, couples in full- and contingent-coach conditions maintained gains in relationship satisfaction and couples in the automated-coach condition continued to improve. Given the comparable completion rates and minimal differences in effect sizes across intervention conditions, all three coaching models appear viable; therefore, the choice of model can vary depending on available resources as well as couple or stakeholder preferences. This study was preregistered (ClinicalTrials.govNCT03568565).

Highlights

• •Comparable completion rates across all coaching conditions.
• •All coach conditions improved relationship satisfaction relative to the waitlist.
• •No differences in relationship satisfaction gains between coaching levels.
• •The OurRelationship program can be effective with automated coach support.

1. Introduction

The prevalence of relationship distress in the United States is troubling. Approximately one-third of marriage are unhappy (Lavner and Bradbury, 2010; Whisman et al., 2008) and 40 %–50 % of first marriages end in divorce (Smock and Schwartz, 2020). Furthermore, relationship difficulties disproportionately impact lower-income couples (Bramlett and Mosher, 2002; Trail and Karney, 2012). Relationship distress has substantial negative impacts on individual mental and physical health (Slatcher and Selcuk, 2017; Robles et al., 2014). Fortunately, couple therapy (Roddy et al., 2020b) improves relationship satisfaction (Cohen's d = 0.91) alongside communication (d = 0.76) and emotional intimacy (d = 0.39) compared to control. Likewise, relationship education (Hawkins et al., 2008) ameliorates relationship distress (ds = 0.30 to 0.36) and communication difficulties (ds = 0.43 to 0.45). Given the inherent limitations in the reach of in-person relationship interventions, technology-delivered interventions that have the capacity to reach many couples are needed (Doss et al., 2017).

The RE-AIM Framework defines five areas to evaluate health promotion interventions to ensure external validity and improve adoption and implementation (Glasgow et al., 1999): reach, effectiveness, adoption, implementation, and maintenance. OurRelationship, an online adaptation of Integrative Behavioral Couple Therapy (Doss et al., 2013), is able to reach significantly more couples than traditional couple therapy (Georgia Salivar et al., 2020) with substantial effectiveness. OurRelationship consists of online e-learning modules and brief calls with a paraprofessional coach and is the most well-researched online program for distressed relationships with five randomized trials to date demonstrating its effectiveness (Doss et al., 2016; Roddy et al., 2018; Rothman et al., 2019; Doss et al., 2020; Hatch et al., 2022). In randomized controlled trials with a nationally representative sample (Doss et al., 2016) and two independent low-income samples (Doss et al., 2020, Hatch et al., 2022) in the United States, OurRelationship improved multiple domains of relationship functioning (Cohen's ds = 0.46–0.69) as well as individual mental and physical health (ds = 0.13–0.50) (Doss et al., 2016; Roddy et al., 2020a) with effect sizes similar to more intensive, in person interventions. Further, these effects are maintained for at least 12 months (Doss et al., 2019; Roddy et al., 2021).

While technology-delivered programs such as OurRelationship help address concerns of reach and effectiveness (Georgia Salivar et al., 2020), coaching is the primary rate-limiting factor for its adoption and implementation. Coaches – paraprofessionals with mental health experience and training – require initial training and ongoing clinical supervision. Moreover, the cost of training, salaries, and supervision do not decrease at the same rate as the digital program itself when increasing scale. While an ideal program would have both high completion and high effectiveness, moderate completion and moderate effectiveness would still have a substantial impact on the population if barriers to adoption and implementation were decreased. A greater understanding of how coaching impacts both completion and effectiveness of the program – as well as if more sophisticated automation can alter the need for coaching – can inform potential adaptations to be made during delivery to reduce implementation barriers.

In the individual intervention literature, evidence suggests that any additional form of support, be it automated emails or coach phone calls, can substantially increase program adherence and treatment gains and are key to the success of web-based interventions (Oromendia et al., 2016; Kleiboer et al., 2015; Levin et al., 2021; Berger et al., 2011). However, questions remain regarding how adherence and treatment gains in online couple-based relationship interventions vary depending on the types and levels of coaching. Previous research on OurRelationship investigated the amount of coaching needed for program completion and effects on relationship satisfaction, an omnibus measure of relationship functioning. Couples completing a brief version of OurRelationship following a 6-week waitlist were randomized to do the program with or without a coach. Couples with a coach experienced improvements in relationship satisfaction (within-group d = 0.40) and 71 % completed the program (Roddy et al., 2017). In contrast, couples not assigned a coach did not experience significant improvements in relationship satisfaction and only 42 % completed the program (Roddy et al., 2017). An open trial of OurRelationship comparing couples who received four coach calls (“high” support) to a single coach call (“low” support) found similar effects on relationship satisfaction (within-group ds = 0.61 and 0.52 for high and low support, respectively) but higher program completion in the high (66 %) than low support (36 %) group (Roddy et al., 2018). Finally, a second open trial of OurRelationship without any eligibility or enrollment requirements and without any coach support saw only 6 % of couples completing the program (Rothman et al., 2019). While these studies suggest that a coach may play a potentially important role in ensuring substantial effects and completion, the absence of a control group makes it unclear what the intervention effects are with reduced coaching. Moreover, it remains unclear what the optimal role of a coach is and how technological advances, such as automated reminders, might shift that balance. To help answer this question, OurRelationship was updated to include extensive automated reminders based on progress through the program and responses to key programmatic questions. These technological advances may replace some of the accountability typically provided by coaches (Mohr et al., 2011), and in turn increase program adherence and treatment gains for couples receiving low or automated coach support.

Furthermore, it is possible that couples may vary in their need for support to complete the program. In the individual literature, one meta-analysis of internet-based cognitive behavioral therapy for depression found coaching provided substantial benefit only to individuals with moderate-to-severe depressive symptoms (Karyotaki et al., 2021). Adaptive interventions mirror clinical work by adjusting the dose or intensity of an intervention based on client characteristics (Collins et al., 2004). For OurRelationship, adding a coach when a couple is experiencing difficulty completing the program in a timely manner – rather than assigning everyone the same dose of coaching at the start of the program – could be an effective middle ground. Examining the effectiveness of OurRelationship with reduced coach support and comparing the relative effectiveness with varying levels of coaching has important implications for adapting and implementing OurRelationship on a large scale.

The current study seeks to contribute to the existing knowledge of OurRelationship and online couple-based relationship intervention more generally by testing and comparing OurRelationship delivered with three levels of coaching: (1) full coach (i.e., as originally designed where couples receive one introductory coach call followed by three coach calls), (2) automated coach (i.e., as a stand-alone program where couples receive one introductory coach call followed by tailored automated emails), and (3) contingent coach (i.e., as an adaptive program where couples receive one introductory coach call followed by either tailored automated emails or additional coach calls depending on participant engagement). Specifically, we examined (1) differences in completion rates across intervention conditions, (2) effects on relationship satisfaction (as an omnibus measure of relationship functioning) for OurRelationship delivered with reduced coaching relative to a waitlist control (effects of full coach relative to a waitlist group was reported in Hatch et al., 2022), and (3) differences in intervention effects on relationship satisfaction across intervention conditions.

2. Material and methods

2.1. Participants

Data from a larger partial/limited sequential multiple assignment randomized trial design (Lei et al., 2012) involving 1246 couples (N = 2494 individuals; see Fig. 1 for the consort diagram) was used. As documented in the study registration (ClinicalTrials.gov NCT03568565), participants were first randomized to one of five groups – a waitlist control, OurRelationship (with and without additional coaching), and ePREP (with and without additional coaching); those initially assigned to either program without additional coach support and who were not completing activities in a timely manner were then re-randomized to either continue with or without additional coach support. A waitlist control group was selected for two reasons: (1) this is what has been done in previous studies of OurRelationship, making comparisons of between-group effect sizes across trials possible, and (2) a waitlist control group likely reduces the chances that couples would seek other interventions to improve their relationship during the study; thus, it may yield a more accurate estimate of the effect of the program versus no intervention. The current sample is comprised of 740 couples (n = 1480 individuals) who were in the waitlist control group (n_couples = 224) or one of the three coaching conditions of OurRelationship: full-coach (n_couples = 226), contingent-coach (n_couples = 145; initially not assigned to receive additional coaching but re-randomized to receive coaching if non-responsive) and automated-coach (n_couples = 145; initially not assigned to receive additional coaching and re-randomized to continue without coaching if non-responsive). Data from the waitlist control group and the OurRelationship full-coach condition used in the current study has been reported in Hatch et al. (2022). Unequal group size is due to planned unbalanced randomization (see supplement). Eight individuals withdrew consent, resulting in a final analytic sample size of 740 couples (n = 1472 individuals). Couples (n_couples = 506) randomized to the other intervention program, ePREP, were not included because of differences in definitions of program completion and “non-responsive” (triggering additional coaching).

Fig. 1.

Consort diagram.

Note. ¹Four individuals withdrew consent, ²two individuals withdrew consent, ³two individuals withdrew consent, ⁴not included in these analyses.

Participants were on average 33.25 years old (SD = 7.92) and racially identified as White (68.8 %), Black (18.4 %), multiracial (5.0 %), Asian (0.8 %), American Indian or Alaskan Native (1.7 %), Native Hawaiian or Pacific Islanders (0.2 %), or Other (5.0 %). Ethnically, 13.3 % identified as Hispanic or Latino. Participants were predominantly in mixed-gender relationships (92.3 %), with 7.7 % in same-gender relationships; 47 % identified as male, with 53 % identified as female. Couples identified their current marital status as married (45.2 %), engaged (28.7 %), and cohabiting (26.1 %) and reported being in a committed relationship with their current partner for an average of 5.76 years (SD = 5.27). Educational achievement ranged from having no degree (7.7 %), to having a GED or equivalent (14.8 %), high school diploma (16.6 %), vocational degree (9.4 %), college experience that did not culminate in a degree (25.4 %), Associate's degree (10.7 %), Bachelor's degree (12.6 %), or advanced degree (Master's or Doctoral level; 2.8 %). Less than half of the sample were employed full-time (41.2 %), with the remainder of participants employed part time (13.0 %), temporarily or seasonally (9.2 %), or employed with variable hours (6.6 %). Thirty percent of participants were not currently employed. Most participants reported an average individual annual income of under $25,000 (71.0 %), with the remainder reporting between $25,000—$50,000 (24.9 %), $50,000—$75,000 (3.6 %), between 75,000–100,000 (0.4 %), and over $100,000 (0.1 %).

2.2. Procedure

Permission was granted from the Institutional Review Board at University of Miami prior to participant recruitment. This study was preregistered (ClinicalTrials.gov NCT03568565), and all data and R scripts are made available on the Open Science Framework (https://osf.io/yk63v/?view_only=62db2da4b4ca40b39a0928acc38d9005). Recruitment was conducted using paid advertising on Google and social media. Couples were eligible if they were married, engaged, or cohabiting with their partner for at least six months; had a household income <200 % of the federal poverty line; lived in the United States; were between the ages of 18–64; could read and write in English fluently; had high speed or 3G internet access; agreed to forego couples treatment for 6 months; had not previously participated in OurRelationship or PREP programs; and did not endorse fear of (i.e., quite, very, or extremely afraid) or severe (e.g., forced sex, choking, threated with a gun or knife) intimate partner violence.

2.2.1. Intervention

OurRelationship is an online adaptation of Integrative Behavioral Couples Therapy (Christensen et al., 2004) designed to improve relationship satisfaction for distressed couples. The program consists of three phases of online activities that take 6–8 h to complete in total. The first phase, Observe, is devoted to helping couples identify and define a relationship problem (known as the core issue) that they feel is at the center of their relationship difficulties. In the next phase, Understand, couples learn to develop a deeper understanding of what causes, maintains, and exacerbates the core issue. In the final phase, Respond, couples are taught skills to problem-solve the core issue (Doss et al., 2013).

2.2.2. Coaching

During enrollment, all couples completed a fifteen-minute introductory videoconference call with a coach to outline procedures, inform couples about what they could expect from the program, and obtain verbal consent; only after this call were couples granted access to the online materials. Couples assigned to the full-coach condition had the opportunity to schedule three additional fifteen-minute videoconference calls with the coach, as originally designed, after completing Observe, Understand, and Respond, respectively. On these calls, the coach discussed program content, practiced skills learned from the program activities, and addressed any technical problems or barriers to activity completion. To ensure reliability and quality control, coaches followed a structured, semi-scripted manual for each call.

Couples assigned to the automated-coach condition completed online materials without additional coach calls following each phase, allowing them to progress through the materials entirely on their own timeline. They also received automated emails from their coach regarding their progress and reminding them to complete their materials.

Couples assigned to the contingent-coach condition received automated emails from their coach (similar to the automated-coach condition). However, if they were deemed “non-responsive” – defined as taking >14 days to reach a particular milestone in the online activities since completing the most recent activity – they would be offered the opportunity to schedule coach calls for the remainder of the program. For example, if couples were in the Observe phase when deemed “non-responsive”, they would have the opportunity to complete the remaining three coaching calls—parallel to the full coach condition; if couples were in the Respond phase, they would only have one additional coach call.

2.3. Outcome measures

Program completion by each phase and the entire program were coded at the couple level (complete = 1, incomplete = 0). Couples were considered to have completed a certain phase if both partners finished all the online activities required before and in the corresponding phase.

Relationship satisfaction was measured using the abbreviated, four-item version of the Couples Satisfaction Index (Funk and Rogge, 2007). Participants rated how much they agreed with questions such as: “I have a warm and comfortable relationship with my partner.” Scores range from 0 to 21, with higher scores indicating more satisfaction ($\alpha$ = 0.92). Satisfaction was assessed five times: baseline, mid-assessment (roughly 1 month from baseline), post-assessment (roughly 2 months from baseline), and 2- and 4-month follow-ups, which were roughly 4 and 6 months from baseline, respectively.

2.4. Data analysis

Analyses were conducted within a Bayesian framework using the brms package in R (v2.19.0; (Bürkner, 2017)). Bayesian approach has several advantages over frequentist approach including a de-emphasis on p-values, the ease of interpretation, and the ability to integrate prior information. Instead of 95 % confidence intervals, 95 % Bayesian credible intervals were reported, which indicate that there is a 95 % probability that the true estimate would lie within the interval based on the data and prior knowledge. When the 95 % Bayesian credible interval does not include 0, we conclude that the parameter is reliably different from 0, indicating that there is an effect. Model convergence was evaluated based on $\widehat{R}$ statistics and visual inspection of whether the trace plots depicting the sample chains showed a converging pattern. $\widehat{R}$s exceeding 1.05 would be indicative of possible non-convergence (Vehtari et al., 2021).

To examine differential completion rates, three two-level logistic regression models were run (i.e., couples nested within coaches) – one for each phase. Each were fitted with 2 parallel chains, default priors, 1000 warm-up iterations and a total of 2000 iterations. The automated-coach condition was treated as the reference group. Two binary variables were used to represent assignment to the full-coach or the contingent-coach condition.

To capture differential relationship satisfaction changes, a piecewise three-level multilevel model was utilized. Multilevel models account for dependencies in the data (i.e., repeated measures nested within couples nested within coaches). Slopes modelling the effect of time during and following the program, respectively, were centered at the post assessment and allowed to vary across couples and coaches. The waitlist control was treated as the reference group. Differential changes in each of the three intervention conditions relative to the waitlist control were modeled by including the main effects of group assignments and their two-way interactions with effects of time during and following the program, respectively. There were 9.6 %, 14.4 %, 24.1 % and 33.8 % missing data at each of the four assessments following baseline, respectively. Missing data was handled using Blimp – a multilevel imputation program (Enders et al., 2020; Enders et al., 2018; Keller and Enders, 2019). Variables associated with relationship satisfaction and its missingness were identified and included in the imputation model to reduce bias. Ten imputed datasets were created and used for analysis. The model was fitted with 16 parallel chains, default priors, 5000 warm-up iterations and 45,000 post-warm-up draws per chain per imputed dataset. Cohen's d was calculated by multiplying the respective unstandardized coefficients for time during and following the program from the multilevel piecewise model by the corresponding median number of weeks elapsed (7.71 from baseline to post and 20.5 from post to 4-month follow-up), and then divided by the baseline standard deviation.

3. Results

Descriptive statistics for relationship satisfaction are presented in Table S1. The visual inspection of trace plots indicated proper mixing and convergence for all models. All R^{^} were smaller than 1.05, indicating good model convergence.

3.1. Program completion

Fig. 2 presents the percentage of couples completing by each of the three sequential program phases by intervention condition. Of 145 contingent-coach couples, 124 were deemed “non-responsive” and offered additional coaching. However, less than half (n = 59) subsequently completed one or more coach calls.

Fig. 2.

Program phase completion by intervention condition.

The odds of couples completing by Observe, Understand, and Respond phases were not reliably different across conditions (i.e., the 95 % credible interval [CI] for an odds ratio included 1; see online supplement for odds ratios and CIs for paired comparisons). Further, the posterior distributions indicated that the probability of full-coach couples having the higher odds (i.e., odds ratio > 1) of completing Observe, Understand, and Respond phases, relative to automated-coach couples, was 28.4 %, 43.9 %, and 77.4 %, respectively; the probability of full-coach couples having the higher odds of completing Observe, Understand, and Respond phases, relative to contingent-coach couples, was 65 %, 70 %, and 92.7 %, respectively; and the probability of contingent-coach couples having the higher odds of completing Observe, Understand, and Respond phases, relative to automated-coach couples, was 15.6 %, 22.6 %, and 21.7 %, respectively.

3.2. Relationship satisfaction

Table 1 summarizes the aggregated results from the 10 imputed datasets and Fig. 3 displays within-group effect sizes. From baseline to post-program, couples in all three intervention conditions, compared to waitlist control couples, experienced reliably greater gains in relationship satisfaction (between-group Cohens' d [d]_full = 0.46, 95 % CI [0.33, 0.59]2; d_contingent = 0.47, 95 % CI [0.32, 0.61]; d_automated = 0.40, 95 % CI [0.25, 0.55]). No reliable differences were found across intervention conditions (d_{contingent-full} = −0.004, 95 % CI [−0.15, 0.14]; d_{automated-full} = 0.06, 95 % CI [−0.22, 0.10]; d _{contingent-automated} = 0.06, 95 % CI [−0.10, 0.23]).

Table 1.

Differential program effects on relationship satisfaction during and following the program.

	B	95 % CI
Fixed effects
Intercept (at post)
Waitlist	10.50	[9.90, 11.09]
Full Coach	1.96	[1.17, 2.75]
Contingent Coach	1.80	[0.91, 2.70]
Automated Coach	1.90	[0.99, 2.81]
Time 1 (Intervention Period)
Waitlist	0.08	[0.02, 0.14]
Full Coach	0.30	[0.21, 0.38]
Contingent Coach	0.30	[0.21, 0.39]
Automated Coach	0.26	[0.16, 0.36]
Time 2 (Follow-up Period)
Waitlist	0.03	[0.00, 0.05]
Full Coach	−0.01	[−0.04, 0.03]
Contingent Coach	0.01	[−0.03, 0.05]
Automated coach	0.05	[0.01, 0.09]
Random effects (couple-level)
Sd (Intercept)	3.89	[3.66, 4.13]
Sd (Time 1)	0.24	[0.19, 0.28]
Sd (Time 2)	0.10	[0.08, 0.12]
Cor (Intercept, Time 1)	0.29	[0.14, 0.42]
Cor (Intercept, Time 2)	−0.23	[−0.35, −0.09]
Cor (Time 1, Time 2)	0.05	[−0.20, 0.36]
Random effects (coach-level)
Sd (Intercept)	0.25	[0.01, 0.69]
Sd (Time 1)	0.03	[0.00, 0.09]
Sd (Time 2)	0.01	[0.00, 0.03]
Cor (Intercept, Time 1)	−0.08	[−0.90, 0.85]
Cor (Intercept, Time 2)	−0.07	[−0.90, 0.85]
Cor (Time 1, Time 2)	0.02	[−0.87, 0.88]

Note. CI = credible interval. The intervention indicator variables (i.e., Full-coach, Contingent-coach, Automated-coach) were coded such that couples in the waitlist control = 0 and Full-/Contingent-/Automated-coach = 1 for those in the corresponding intervention assignment. Sd = standard deviation; Time during both the intervention (Time 1) and follow-up periods (Time 2) were centered around the date of posttest assessment.

Fig. 3.

Magnitude of within-group changes by the end of the program.

Note. CI = credible interval.

During the 4 months follow-up, compared to waitlist control couples, full-coach couples (d = −0.03, 95 % CI [−0.18, 0.12]3) and contingent-coach couples (d = 0.04, 95% CI [-0.12, 0.21]) maintained their gains, whereas automated-coach couples experienced some additional gains (d = 0.19, 95% CI [0.02, 0.36]). Changes were reliably different between full-coach and automated-coach couples (d _{automated-full} = 0.22, 95% CI [0.05, 0.39]) but were not reliably different between full-coach and contingent-coach couples (d _{contingent-full} = 0.07, 95% CI [-0.09, 0.24]), or between contingent-coach and automated-coach couples (d _{contingent-automated} = −0.15, 95% CI [-0.31, 0.02]).

4. Discussion

The current study examined differential completion rates and effects of the online OurRelationship program delivered with three different levels of coaching. Overall, completion rates were comparable across conditions. Further, compared to waitlist couples, couples in all active intervention conditions demonstrated reliable improvements in relationship satisfaction that were maintained through the short follow-up period. This comparative effectiveness trial demonstrated that all intervention conditions were effective with subtle differences between conditions with regard to program completion and global relationship satisfaction improvements, providing meaningful implications for implementation.

4.1. Program completion

Completion rates were comparable across conditions. This is consistent with Levin et al. (2021)'s work on online acceptance and commitment therapy where no differences were found in program engagement between email prompts only condition and email plus phone coaching condition. These findings also complement other findings in the individual intervention literature, which suggest that minimal contact that is built into the program may play a critical role in engaging participant. Specifically, Berger et al. (2011) found a higher completion rate (56 %) for an internet-based treatment of depression that included scheduled e-mail contact with a therapist, compared to without any support (36 %). Similarly, Oromendia et al. (2016) found a lower drop-out rate and better treatment adherence for an internet-based treatment for panic disorder that included mandatory psychological support from a therapist, compared to with support only upon patients' request. Moreover, completion rates for the automated- and contingent-coach conditions in the current study were meaningfully higher than the no or low support condition in previous open trials (Roddy et al., 2018), suggesting that the automated reminders may have aided in program completion. Taken together, these findings suggest that minimal engagement such as tailored automated emails may be a promising approach to balancing affordability and scalability. Despite comparable completion rates, the posterior distributions suggested that the probabilities of full-coach couples having higher odds of completing the program than those in the other two conditions were >77 %. This indicates something unique about consistent coach contact that may not be fully replaced with the extensive automated reminders. The posterior distributions also suggested a 78 % probability that automated-coach couples had higher odds of completing the program than contingent-coach couples. Since less than half of the couples completed additional coach calls when given the opportunity, the contingent-coach condition may need to be more sensitive, trigger coaching earlier, or trigger alternative sources of support.

4.2. Improvements in relationship satisfaction

Despite slightly lower completion rates in conditions with reduced coaching, all three conditions created important and impactful improvements for couples that were maintained over the 4-month follow-up, with minimal differences between conditions. The overall lack of differences aligns with findings from studies in the individual intervention literature that have compared conditions involving therapeutic support from a professional or paraprofessional with conditions involving only automated and technical support (Levin et al., 2021; Karyotaki et al., 2021).

Effect sizes at post-program for the full-coach condition were slightly lower than those observed in previous trials (Doss et al., 2016; Doss et al., 2020; Hatch et al., 2022). Below, we compare within-group effect sizes, as not all trials have included a control condition. The within-group effect size for the full-coach condition in the current study (d = 0.58) was lower than in prior trials with low-income couples (d = 0.72; Doss et al., 2020) as well as nationally representative samples (ds = 0.61 and 0.96; Roddy et al., 2018, Doss et al., 2016). The smaller effect sizes observed may be due to the increasing number of couples served in each trial, from 151 in the first trial to 516 in the current study, which could have diluted the effects. More couples served has implications for project staffing, expansion of advertising to additional groups, and the training of additional coaches. Furthermore, each subsequent trial has enrolled a broader set of couples, and it is common for effect sizes to slightly decrease as interventions are scaled up (Parry et al., 2013). In contract, the effect sizes at post-program for the contingent- (d = 0.58) and automated-coach (d = 0.52) conditions in the current trial were similar in magnitude to the previously tested “low support” model (d = 0.52; Roddy et al., 2018), which speaks to the scalability of programs requiring less coach contact. Despite the slightly lower completion rates of these two conditions, they have the potential to produce a significant population-level impact when paired with moderate-sized effects.

4.3. Limitations

There are several limitations to note. First, we assessed a relatively short follow-up period; differences between conditions could become evident with time. Although previous trials of OurRelationship have demonstrated maintenance of gains through one year (Doss et al., 2019; Roddy et al., 2021) for the full-coach condition, it is unclear if this will be replicated with automated- and contingent-coach couples; however, it is notable that couples in the automated-coach condition continued to report gains after the program ended. Second, we used a waitlist control group. It is important to consider design differences when comparing effect sizes in the current study with other studies that employed a different design as the anticipation of receiving intervention may lead to under- or over-estimated effect sizes. Finally, it is possible that the repeated research assessments may have aided engagement in all intervention conditions.

4.4. Future directions

Given the minimum differences between coach conditions and significant improvements relative to the control group in all three conditions, OurRelationship can be offered with various levels of coaching. Therefore, the specific level of coaching that is most appropriate may vary by setting, resources, and the unique needs of the populations. From a cost-effectiveness perspective, omitting coaching would significantly decrease the cost of delivery; however, that option may be less appealing to some users – which could limit adoption of the intervention. The advent of artificial intelligence chatbots is an exciting future direction that could be incorporated into the OurRelationship program to further personalize the automated responses (Carlbring et al., 2023). Additionally, the contingent-coach condition may need adaptations to be more sensitive to couples falling behind so that more coach contact can be added more immediately when couples begin experiencing difficulties.

Furthermore, there may be characteristics of couples that would moderate these effects. For example, more satisfied couples may benefit more from the self-paced, automated-coach condition than would severely-distressed couples who might need a coach to help mediate challenging conversations. Alternately, comfort with technology could moderate effects such that those with higher technological comfort could derive less benefit from the inclusion of a coach. It is also possible that there would be differences across conditions in more narrow aspects of relationship functioning (e.g., communication, intimacy) or in the mechanisms that drove changes in relationship satisfaction. These and other potential moderators should be explored to inform future adaptations.

4.5. Conclusions

To optimize an intervention for implementation, considerations of affordability and scalability must be balanced with effectiveness (Collins, 2018). The amount of coaching offered has significant implications for both affordability and scalability of the OurRelationship program. While an intervention optimized for the lowest cost would select the automated-coach model for maximum scalability and affordability, contextual factors such as client and provider preferences should also be considered in future implementation efforts.

Funding sources

Funding for this project was approved by the U.S. Department of Health and Human Services, Administration for Children and Families, Grant 90FM0063. MKR is supported by the Vanderbilt Faculty Research Scholars. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the U.S. Department of Health and Human Services, Administration for Children and Families. Publication fee was supported by Open Access Publication Equity Fund from University of Denver awarded to YL.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Brian Doss, PhD, senior author, is a coinventor of the intellectual property used in this study and an equity owner in OurRelationship LLC.

Appendix A. Supplementary data

Supplementary material