Relating Engagement to Outcomes in Prevention: The Case of a Parenting Program for Couples

Abstract

Analyses of program engagement can provide critical insight into how program involvement leads to outcomes. This study examines the relation between participant engagement and program outcomes in Family Foundations (FF), a universal preventive intervention designed to help couples manage the transition to parenthood by improving coparenting relationship quality. Previous intent-to-treat outcome analyses from a randomized trial indicate FF improves parental adjustment, interparental relationships, and parenting. Analyses for the current study use the same sample, and yield statistically reliable relations between participant engagement and interparental relationships but not parental adjustment or parenting. Discussion considers implications for FF and the difficulities researchers face when examining the relation between engagement and outcomes in preventive interventions.

Program engagement is presumably a critical element of successful prevention program implementation leading to positive outcomes of the target population (Staudt, 2007). Without engaging the target population, most community-based interventions cannot be effective (Snell-Johns, Mendez, & Smith, 2004). Although a number of studies have examined the predictors of program engagement, relatively little research examines how participant engagement relates to program outcomes. Studying how engagement predicts outcomes can inform program improvement efforts by identifying the program elements most critical to success. Further, studies of engagement and outcomes can enhance understanding of how much program dosage is necessary to achieve outcomes (Charlebois, Brendgen, Vitaro, Normandeau, & Boudreau, 2004). The goal of this study is to examine the relation between engagement and several outcomes previously shown to result from exposure to Family Foundations, a preventive intervention designed to promote positive individual and family adjustment among couples across the transition to parenthood (Feinberg & Kan, 2008).

Family Foundations

Family Foundations is a universal intervention for couples expecting a first baby, delivered in the non-stigmatizing context of a hospital's childbirth education department alongside or integrated into standard childbirth education classes. The program was developed based on a theory that the coparenting relationship is a proximal and key influence on parent adjustment, parenting, and child well-being (Feinberg, 2002; 2003). Accordingly, intervention content focuses on factors considered to promote positive coparental relations, such as emotional regulation, conflict management, problem solving, communication, and mutually supportive co-parenting strategies. Couples attend eight interactive, skills-based classes that each last approximately two hours. Four classes occur before childbirth and four after. Couples also complete homework assignments between classes.

Results from a randomized trial found intervention effects on co-parenting, parenting, parent-child relations, child self-regulation, maternal depression and anxiety at child age 6 months and one year (Feinberg & Kan, 2008; Feinberg, Kan, & Goslin, 2009). Most recently, outcome analyses when children were 3 years old indicate enhanced parental efficacy, co-parenting quality, relationship satisfaction, and child social competence in the intervention group. Further, intervention group families who had boys experienced reduced parenting stress, depression, parenting overreactivity, parenting laxness, and child behavior problems (Feinberg, Jones, Kan, & Goslin, 2010). In the most recent outcome analyses, parental efficacy, co-parenting quality, parenting stress, and depression were assessed at child age 6 months, 12 months, and 3 years. Outcome analyses identified an overall effect across years, which emerged at 6 months and did not change significantly across time (Feinberg, Jones, Kan, & Goslin, 2010).

Previous Research Relating Engagement to Outcomes

Program engagement is a multifaceted construct representing an individual's level of involvement in an intervention. Appropriate measures of engagement vary across intervention contexts. In multi-session curriculum-based interventions such as Family Foundations, three inter-related engagement indicators have been used in previous research (e.g. Hansen & Warner, 1994; Reid, Webster-Stratton, & Baydar, 2004) and are used in this study: 1) session attendance; 2) homework completion; and 3) group leader ratings of engagement during sessions. The underlying notion is that participants who are more engaged, as indexed by these three indicators, will derive relatively greater benefit from a program that relies on active participation and practice for skill development.

In a prior study, we examined which individual and couple factors at baseline predicted engagement in Family Foundations. Findings indicated that marital status is the most powerful predictor of engagement. Sociodemographic variables, including parent gender, education, income, and age also predicted program engagement to a limited extent (Brown, Feinberg, & Kan, 2011).

Several studies have identified a relation between participant engagement in intervention/prevention programs and participant outcomes. For example, this relation was examined in a study of The Incredible Years (Webster-Stratton, Reid, & Hammond, 2004), a prevention program aimed at reducing the development of conduct problems during early childhood though improvements in maternal parenting behavior. Maternal engagement was related to improvements in children's observed pro-social behaviors and conduct problems (Reid, Webster-Stratton, & Baydar, 2004). Similarly, amount of attendance in Preparing for the Drug (Free) Years predicted increased parenting skills and improved parent-child affective quality (Spoth & Redmond, 1996; Spoth, Redmond, Haggerty, & Ward, 1995). Findings from studies of The Incredible Years and Preparing for the Drug (Free) Years are consistent with a meta-analytic review of participant engagement in parent-training programs (Reyno & McGrath, 2006). Meta-analysis findings indicated that participant attendance in parent-training classes was a significant predictor of treatment outcome, though effect sizes were small. Further, early dropout in outpatient treatment predicted poorer treatment outcome for children presenting with conduct problems (Kazdin, Mazurick, & Siegel, 1994).

Participant engagement has also been examined in relation to academic outcomes. For example, dosage (i.e., child attendance at training sessions) was examined in relation to academic and behavioral outcomes following a school-based intervention for disruptive, young boys from highly stressed, low-income families (Charlebois et al., 2004). In the sample as a whole, attendance was significantly related to post-intervention academic outcomes but not to behavioral outcomes. For boys living in single-parent homes, attendance was related to post-intervention pro-social behaviors.

Dosage was also relevant to outcomes related to child risk for substance abuse in a parent-training program delivered to parents in the workplace. Participants who attended at least 80% of the program (high dosage group) reported significant, lasting improvements in child behavior problems, child positive behaviors, parenting behaviors, and parental attitudes against substance abuse (Felner, Brand, Mulhall, Counter, & et al., 1994). In contrast, participants who received low dosage (< 80% program attendance) did not report significant improvements in these domains or reported improvements that did not sustain at long-term follow-up.

Although several studies demonstrate a relation between engagement and outcomes, we were unable to identify studies that directly examined how engagement in different program elements relates to outcomes. The closest we found was a study that examined differences between intent-to-treat effects and effect sizes when treatment group members who skipped specific program elements were excluded from analyses (Kosterman, Hawkins, Haggerty, Spoth, & Redmond, 2001). Such analyses can be particularly useful for program improvement because they can aid in the identification of program elements that have the strongest influence on program outcomes. As such, we investigated the ability of prenatal and postnatal program engagement in Family Foundations to predict outcomes separately. These analyses were exploratory, as we did not have hypotheses about whether prenatal or postnatal engagement would be more closely related to outcomes.

In this study, we examined the relation between engagement and the following outcome domains: parental adjustment, interparental relationship, and parenting. We expected engagement to be associated with outcomes that demonstrated significant program effects in the randomized trial findings using the same sample (Feinberg et al., 2010). More specifically, we hypothesized engagement to have negative relations with parental stress, parent depression, parenting overreactivity, and parenting laxness. We hypothesized engagement would have positive relations with parental efficacy, coparenting quality, and couple relationship satisfaction.

Method

Participants

Participants in this study were 89 heterosexual couples (178 individuals) assigned to the treatment condition in a randomized trial of Family Foundations. To be included in the study, couples had to be expecting their first child, living together, and 18 or older at the time of recruitment. Childbirth education centers in two hospitals recruited 81% of couples by first mailing a letter and then calling couples on the phone. Twenty-three percent of the couples called agreed to participate in the randomized trial. Reasons offered for not participating include a lack of time, scheduling conflicts, and a lack of perceived need for the intervention. The remaining participants were recruited through advertising, which required interested individuals to return a postcard or call a phone number to enroll in the study. Of the 89 couples assigned to the FF intervention condition, 96% participated in one or more classes.

At the start of the study, 82% of couples were married. Most participants (91% of mothers and 90% of fathers) were white, with the remaining participants including African American, Asian, Hispanic, or other. Median annual family income was $65,000 (σ = $34,372). Average educational attainment was 15.06 years for mothers (σ = 1.82) and 14.51 years for fathers (σ = 2.19); 86% of mothers and 71% of fathers had received at least some post-secondary education. Expectant mothers and fathers had an average age of 28.3 (σ = 4.9) and 29.8 (σ = 5.6). Families were recruited from medium-sized cities in Pennsylvania and were generally representative of families from these regions. Further details on recruitment are available elsewhere (Feinberg & Kan, 2008).

Pretest data were collected from mothers and fathers during home interviews when mothers were pregnant (average weeks gestation = 22.9, σ = 5.3). Intervention couples, who are the focus of this study, received the manualized FF program consisting of 4 prenatal and 4 postnatal sessions. Group leaders underwent three days of training and received ongoing feedback from supervisors who observed some of the sessions. Observer ratings indicated that the intervention was implemented as intended, with an average of 95% of the curriculum content covered (Feinberg & Kan, 2008). Observer ratings of the clarity of group leader presentations averaged 4.7 on a 5 point scale, with 4 = very clear and 5 = extremely clear. Classes contained only study participants, with an average group size of 9 couples. Couples were offered free child care during all postnatal sessions.

Follow-up data collection occurred across three non-equally spaced waves: when the child was roughly six months (wave 2), 12 months (wave 3) and 36 months old (wave 4). Data from four families were not utilized in analyses because of child medical problems (n=2), and/or multiple births (n=2). Study attrition for mothers and fathers at wave 4 was approximately 17% and 23%, respectively. Roughly 85% of families provided data (from either parent) at wave 4. We used multiple imputation to estimate the missing data caused by attrition (Schafer, 1997; Sinharay, Stern, & Russell, 2001).

Measures

Engagement

We used three separate indicators to measure engagement – session attendance, group leader ratings of engagement, and homework completion. Table 1 provides descriptive statistics for all engagement indicators, both pre and postnatal, along with all covariate and outcomes scales. Group leaders took attendance at each of the eight classes and the attendance variable represents the number of classes attended (overall mean = 5.4 classes; σ = 2.5). Group leaders also rated participants on the degree to which they were actively engaged in the classes. The four-item measure used a five-point response format, with participants receiving an average rating of 3.6 (σ = 1.1; 3 represents sometimes engaged, 4 represents usually engaged). The items measured leaders' perceptions of each group member's level of interest in the material, engagement with his/her partner during the session activities, participation in group discussions, and resistance to the material (reverse scored). Group leaders rated each participant after the first four sessions and again after the last four sessions. Homework completion was measured using self-report from participants, who answered yes/no questions about whether they completed each of the 14 homework assignments (overall mean = 7.5; σ = 5.0). To minimize the number of analyses, we created three composite predictor variables representing overall engagement, prenatal engagement, and postnatal engagement by taking the average standardized value of attendance, homework completion, and group leader ratings of engagement. Combining the three indicators of engagement into a composite variable is both conceptually and empirically sound. Conceptually, attendance, homework completion, and group leader ratings of engagement are all indicators of engagement that have not only been used in previous research but also combined to create a composite variable (Hansen & Warner, 1994; Reid, et al., 2004). Combining the three indicators of engagement is empirically sound because each indicator is highly related to the other indicators of engagement. Correlations between attendance, homework completion, and group leader ratings range from .67 to .92. The engagement scale is also internally consistent (alpha = .94 overall, .90 prenatal, and .95 postnatal). Overall engagement and postnatal engagement are normally distributed but prenatal engagement faces a ceiling effect with skewness = −1.92 for mothers, −1.83 for fathers and kurtosis = 2.96 for mothers, 2.58 for fathers. Transformations creating a normally distributed prenatal engagement variable did not alter the direction or significance of regression coefficients so results from the untransformed variable are presented to maintain consistency with postnatal and overall engagement.

Table 1.

Sample descriptive statistics.

			Mothers		Fathers
	Wave*	Range	Mean	SD	Mean	SD
Engagement:
Prenatal attendance	×	0–4	3.22	1.13	3.19	1.15
Postnatal attendance	×	0–4	2.29	1.71	2.17	1.73
Prenatal homework completion	×	0–6	4.54	2.06	4.54	2.05
Postnatal homework completion	×	0–8	3.76	3.38	3.48	3.49
Prenatal group participation	×	1–5	3.96	0.95	3.91	0.98
Postnatal group participation	×	1–5	3.33	1.61	3.15	1.62
Control variables:
Age	1	18–41	28.6	4.7	29.9	5.3
Education (years)	1	9–17	15.0	1.8	14.4	2.2
Family Income (in thousands)	1	2.5–162	66.1	36.2	67.3	34.5
Financial Strain1	1	0.5–1.1	0.6	0.2	0.6	0.2
Social Desirability	1	5.0–30.7	17.5	5.0	17.8	4.4
Couple Love	1	57.5–81.0	74.5	5.0	72.7	6.1
Anxiety	1	1.1–15.9	7.2	3.3	5.8	3.7
Interparental Relationship:
Coparenting Quality	2 – 4	2.5–6.0	4.3	1.2	4.4	1.2
Relationship Satisfaction2	4	0.9–2.2	1.4	0.3	1.4	0.3
Parental Adjustment:
Parental Stress	1 – 4	1.0–3.3	1.9	0.5	2.0	0.5
Parental Efficacy	1 – 4	64–112	97.0	9.9	91.5	12.1
Depression (CES-D)2	1 – 4	0–1.5	0.4	0.4	0.3	0.4
Parenting:
Overreactivity	4	1.0–4.0	2.2	0.6	2.4	0.7
Laxness	4	1.0–4.5	2.2	0.7	2.5	0.8

^*Wave 1 = pretest; 2 = 6 months post-birth; 3 = 12 months post-birth; 4 = 3 years post-birth; × = collected during intervention, between waves 1 and 2). For measures collected at multiple waves, wave 4 data used for descriptive statistics.

¹Log transformed

²Square root transformed

Coparenting and couple relationship response scales

We assessed coparenting relationship quality with the 31-item Coparenting Scale, which was created based on prior work (e.g., Abidin & Brunner, 1995; Cordova, 2001; Frank, Olmstead, Wagner, & Laub, 1991; Margolin, Gordis, & John, 2001; McHale, 1997). The overall score represents an average of items covering theoretically important domains: coparental agreement, support, undermining, and exposure of the child to interparental conflict (alpha = .72 for mothers, .65 for fathers). To assess the quality of the couple relationship, we included a measure of relationship satisfaction from the Quality of Marriage Index (Norton, 1983). This score was created from an average of six items – combining five items that ask parents to rate their agreement to statements about their relationship using a 7-point likert scale (e.g., “We have a good relationship”) with one item asking them to rank how happy they feel their relationship is on a scale of 1 to 10 (alpha = .97 for mothers, .95 for fathers; Items were standardized before averaging).

Parental adjustment response scales

We assessed parent efficacy with the 16-item Parenting Sense of Competence scale (Gibaud-Wasston & Wandersman, 1978), asking mothers and fathers how they feel about their parental role (e.g., “I feel confident in my role as a parent”) using a 7-point response scale (alpha = .84 for mothers and .83 for fathers). Self-report of parental stress was measured through the total score on the Parenting Stress Index (PSI; Abidin, 1997). This scale consisted of 27-items asking parents to rate their agreement with certain statements (e.g., “I feel trapped by my responsibilities as a parent”), utilizing a five-point likert scale (alpha = .90 for mothers, .87 for fathers). We assessed parental depression using an abbreviated version of the Center for Epidemiological Studies Depression Scale (CES-D) which consists of seven items asking the respondent to indicate their feelings and outlook within the past week (alpha = .86 for mothers, .83 for fathers; Radloff, 1977). For instance, subjects indicated the degree to which they felt lonely using a 4-level response scale ranging from rarely/none of the time to always/most of the time.

Parenting response scales

The Parenting Scale assesses parent discipline practices in children from 18–48 months (Arnold, O' Leary, Wolff, & Acker, 1993). Our assessment focused on two outcomes: The Laxness scale (based on 11 items assessing permissive parenting; alpha = .85 for mothers, .82 for fathers) and the Overreactivity scale (9 items assessing the degree of authoritarian parenting; alpha = .76 for mothers, .78 for fathers).

Control Variables

Regression models included several control variables representing important background participant characteristics, including respondent age, parent gender, child gender, marital status, family income, and respondent education. Additional control variables included in all models were Couple Love, Anxiety, Social Desirability, and Financial Strain. Aside from Social Desirability, all control variables were included because previous research indicates they are important determinants of parenting and child outcomes (Ahlborg & Strandmark, 2006; Burstein, Ginsburg, & Tein, 2010; D'Onofrio et al., 2003; Gutman, McLoyd, & Tokoyawa, 2005; Van Egeren, 2003; Waylen & Stewart-Brown, 2010). Couple Love was measured using the relationships questionnaire (Braiker & Kelley, 1979), which contains 9 items asking about the extent to which respondents have a loving, giving, committed, intimate, and cohesive relationship with their partner (alpha = .84). We measured Anxiety with the 20-item short form of the Taylor Manifest Anxiety Scale (Bendig, 1956; Taylor, 1953), which measures chronic anxiety by asking respondents about things they do or feel (alpha = .73). Social Desirability consisted of 33 items including such statements as “I am always courteous, even to people who are disagreeable;” (Crowne & Marlow, 1964), and was included as a control for tendency to respond in a favorable manner (alpha = .75 for mothers, .69 for fathers). Financial Strain was measured with three items using a five-point response scale, asking parents to indicate their degree of current financial hardships (alpha = .75 for mothers, .78 for fathers; Kessler, Turner, & House, 1988).

Statistical analyses

We used separate regression models to estimate the influence of engagement on each of the program outcomes of interest. All models used outcomes reported by both mothers and fathers, so we used multilevel models in SAS Proc Mixed to account for the nesting of parents within family. Table 1 lists the wave(s) at which each outcome variable in the study were collected.

For outcomes where data were available across multiple waves, we used three-level growth models, where time-specific outcomes (level 1) were nested within parents (level 2), who were nested within families (level 3). Modeling the developmental trajectory of the longitudinal outcome variables enables examination of the influence of engagement on not only the magnitude of the outcome variable across the three time points but also variation in the outcome across time. In all models, engagement was entered as an independent variable at the parent level. We specified random intercept terms at both the parent- and family-levels of the model, thus acknowledging differences in levels of the outcomes across families in general, and across parents over time. We specified a random slope at the parent level of the model to allow unobserved variation in change over time by parent. Within the three level growth models, we also tested the impact of engagement on variation in the random slope term (using time*engagement interaction term). If the time*engagement interaction term was not significant, we did not retain the effect in the final model (which was always the case in our analyses).

All regression models included all previously listed control variables. If available, we also controlled for pretest scores of the outcome variable, which existed for parent depression and parenting efficacy. We also assessed moderation of the influence of engagement on the dependent variables by child gender and parent gender. However, child and parent gender interactions were never significant, and were thus dropped from the final models.

Results

Table 2 summarizes the results of our analyses, providing standardized estimates of the relation between engagement and each outcome of interest. Findings indicate a positive relation between engagement and the interparental relationship. Specifically, a one standard deviation increase in overall engagement predicted a 0.34 standard deviation unit increase in relationship satisfaction (p < .05). Additionally, although only a trend relation, a one standard deviation increase in overall engagement predicted a 0.14 standard deviation unit increase in coparenting quality (p < .10). Postnatal but not prenatal engagement predicted a statistically reliable (p < .05) increase in coparenting quality and relationship satisfaction. More specifically, a one standard deviation increase in postnatal engagement predicted a 0.15 standard deviation unit increase coparenting quality and a .32 standard deviation unit increase in relationship satisfaction.

Table 2.

Results from separate regression models using program engagement to predict outcomes (only coefficients for the engagement variable are presented).

	Overall Engagement			Prenatal Engagement			Postnatal Engagement
Outcomes	Est.	S.E.	95% C.I.	Est.	S.E.	95% C.I.	Est.	S.E.	95% C.I.
Interparental Relationship:
Coparenting Quality^	.14†	.08	−.02 – .30	.01	.09	−.17 – .19	.15*	.07	.01 – .29
Relationship Satisfaction	.34*	.14	.07 – .61	.16	.16	−.15 – .47	.32*	.13	.07 – .57
Parental Adjustment:
Parental Stress^	−.01	.09	−.19 – .17	.08	.09	−.10 – .26	−.04	.08	−.20 – .17
Parental Efficacy^	.04	.07	−.10 – .18	.01	.07	−.13 – .15	.03	.07	−.11 – .17
Depression (CES-D)^	−.01	.07	−.15 – .13	.01	.07	−.13 – .15	−.03	.06	−.15 – .09
Parenting
Overreactivity	.03	.12	−.21 – .27	−.09	.13	−.34 – .16	.09	.12	−.15 – .33
Laxness	−0.12	0.11	−.34 – .10	−.13	.11	−.35 – .09	−.06	.11	−.27 – .16

^†p < .10

^*p < .05

^{^}three waves of post-intervention data included (waves 2 – 4); full model results available from first author.

Relations between engagement and indicators of parental adjustment and parenting were not significant. Separate regression models using prenatal and postnatal engagement as predictors of parental adjustment and parenting yielded similar null findings. The control variables were also generally unrelated to the outcome variables. For simplicity, we only report control variable findings from models using overall engagement as a predictor of each dependent variable. Only 8 of 70 estimates were significant at the p < .05 level when examining overall engagement. Marital status, child gender, family income, social desirability, and financial strain were not significant in any of the models. Parent age had a significant positive effect on parenting laxness (b = .33). Mothers had significantly higher scores on parenting efficacy (b = .65) and lower scores on parenting laxness (b = − .48) than fathers. Number of years of parent education had a significant positive effect on parenting efficacy (b = .20) and negative effect on parental stress (b = −.20). Parent anxiety had significant positive effects on parent depression (b = .19) and parental stress (b = .20). Finally, couple love had a significant positive effect on coparenting quality (b = .14).

Discussion

Consistent with study hypotheses, overall engagement predicted enhanced interparental relationships, as measured by both coparenting quality and relationship satisfaction. However, the relation between overall engagement and relationship satisfaction was only a trend, and is thus more likely to be spurious. Analyses examining the influence of prenatal and postnatal engagement separately found that postnatal but not prenatal engagement reliably predicted increases in both coparenting quality and relationship satisfaction. The finding that postnatal but not prenatal engagement predicts enhanced interparental relationships may indicate that the content and/or timing of the postnatal sessions is more conducive to achieving program outcomes than that of prenatal engagement. As discussed further in the next two paragraphs, there are conceptual reasons to believe both the postnatal content and the postnatal intervention timing are likely to have a strong influence on coparenting quality and relationship satisfaction.

Prenatal sessions deal with the rudiments of communication, support, and problem-solving as expectant parents discuss childrearing values, plans, and possible arrangements. Postnatal sessions build on this foundation, focusing more on the coordination of actual parenting tasks, such as how to collaborate effectively to promote secure attachment. The effective coordination of parenting tasks is central to both coparenting quality and couple relationship satisfaction (Feinberg, 2002). Thus, it is possible that the prenatal sessions introduced material that was less helpful to couples, whereas the postnatal material was more effective. However, without the collaborative foundation built during the prenatal sessions, the postnatal sessions may not have been effective.

Another possibility is that the timing of the sessions was a determining factor for the finding that postnatal but not prenatal engagement significantly predicted interpersonal relationships. Postnatal sessions may be most helpful because they occur while participants are facing the daily challenge of maintaining a positive coparenting relationship. It is possible that couples in the prenatal sessions did not have the same sense of urgency to incorporate and learn from the intervention content. The timing of interventions is known to be an important predictor of intervention efficacy (Herman & Mandiberg, 2010). From a life course perspective, there are critical periods of development where problematic behaviors are likely to develop and targeted preventive interventions are best able to influence developmental trajectories (Halfon & Hochstein, 2002). Coparenting begins when a couple's first baby is born. Thus, interventions seeking to improve coparenting may be most successful when coparenting patterns are first emerging.

It is also possible that differences in the distributions of engagement across pre and postnatal sessions influenced the results. Prenatal engagement demonstrated a ceiling effect, with negative skew and positive kurtosis, which may have interfered with its predictive ability. The level of postnatal engagement was lower overall and had greater variability; postnatal engagement was also normally distributed. The different levels and distributions of engagement may reflect a different meaning of engagement across the span of the intervention, as postnatal engagement is likely to be more difficult because of the newborn child. It is also possible that couples with stronger interparental relationships may have been more motivated to engage in the postnatal sessions. The significant findings may not have persisted if we were able to control for baseline levels of co-parenting quality and relationship satisfaction. However, we were able to control for baseline levels of couple love and marital status in all analyses, which served as a proxy for interparental relationship quality. Given that engagement rates were lower in the postnatal period, and that postnatal engagement predicted enhanced interpersonal relationships, program implementers may want to focus their engagement efforts on the postnatal period.

In contrast to the findings for interparental relationships, engagement did not predict any measure of parental adjustment or parenting. The null findings are unexpected because analyses of the randomized trial using the same sample indicated that couples assigned to the intervention condition had significantly better outcomes compared to the control group on all measures examined in this study. Thus, we expected that greater engagement would have allowed couples to derive greater benefit from the exercises and skill development practice of the program. There are several potential explanations for the null findings.

It may be easier to detect relations between engagement and outcomes in the interparental relationship domain because the proximal target of the Family Foundations intervention is enhanced interparental relationships. According to the program theory, the impact of engagement on parent adjustment and parenting would be mediated by improvements in interparental relations. The indirect (i.e., mediated) link between engagement and the other outcome domains may exist but could be too small to detect with a sample size of 89 couples. Based on the 95% confidence intervals, we can however conclude that a one standard deviation increase in engagement is unlikely to predict more than a 0.19 standard deviation unit change in parental adjustment or more than a 0.33 standard deviation unit change in indicators of parenting.

Another possible explanation is that engagement or outcomes are poorly measured. However, our multidimensional measure of engagement is based on three normally distributed indicators – attendance, homework completion, and group leader ratings of engagement. Further, outcome measures are generally well-established and were all sensitive enough to detect change in the randomized trial.

Another unlikely explanation for this study's findings is that the intervention does not work. Analyses of the randomized trial provide rigorous evidence that the intervention was helpful for the sample under study. The only potentially reasonable explanation for inaccurate randomized trial findings in this sample is that the intervention group felt obligated to report higher scores after the intervention. However, if this unlikely explanation were true, we would expect the high engagement group to feel more obligated to report particularly favorable outcomes, thus leading to a stronger relation between engagement and outcomes.

An alternative, plausible explanation of findings is that intervention participants can make appropriate decisions about program engagement and disengagement. High-engagers may have decided to engage at a high level because they perceived a need for the intervention, which then led to positive outcomes for high engagers. Those choosing a low level of engagement may not have perceived a need for the program, as they may have recognized that they were likely to attain positive outcomes without the intervention. If perceptions of need are accurate and predict program engagement, we would not expect high engagers to have better outcomes after the intervention than low engagers. This scenario would suppress a relation between engagement and outcomes despite the effectiveness of the intervention. As such, the magnitude of the relation between engagement and outcomes for those individuals who can benefit from the intervention may be larger than was identified in this study.

If individuals can make appropriate decisions about engagement and disengagement, providing them with the opportunity to engage is critical. People who engage at a higher level may have greater need and benefit more from engagement. However, people with greater need may also face greater barriers, such as transportation difficulties, additional life stressors that distract from engagement, or pessimistic attitudes that undermine motivation. Although it is difficult to know when engagement decisions are appropriate, the positive outcomes from the randomized trial suggest that participant engagement decisions regarding Family Foundations were at least partially appropriate, as those who chose to engage benefited enough to produce an average causal effect for everyone who had the opportunity to engage.

We also note that in order to detect a relation between engagement and outcomes, engagement decisions cannot be appropriate in all cases: participants who would benefit must sometimes disengage or there will be no variance in engagement among the individuals poised to benefit from the intervention. Further, relations between engagement and outcomes can only be detected among those individuals capable of benefiting from the intervention. Future research on moderators of treatment efficacy may be able to identify subgroups that benefit most from participation.

The current study provides insight into when engagement in FF program elements is most critical. Specifically, variability in postnatal engagement appears to be more critical than prenatal engagement. Additionally, this study provides insight into the difficulties researchers face when examining the relation between engagement and outcomes. Sample heterogeneity in the consequences of engagement can mask strong relations between engagement and outcomes in the subgroups best poised to benefit from participation. The modest sample size of this study also prevents the generation of precise estimations of effect size. Future research with a larger sample may be able to identify the subgroups that do benefit from engagement and better pinpoint which program elements promote outcomes.