Change in Participant Engagement During a Family-Based Preventive Intervention: Ups and Downs with Time and Tension

Abstract

The efficacy of preventive interventions is related to both the delivery of content and the uptake of that content. While much research has focused on the quality of delivery, few studies have examined the factors that influence uptake. This study examines how and why participants’ engagement, conceptualized as a dynamic process wherein participants interact with each other, the interventionists, and the intervention curriculum, changes over time. We apply growth curve models to repeated measures of engagement obtained from 252 families during a 7-week intervention trial. In the models, we examine (1) whether and how engagement changes over time, the extent of between-person differences in change, and (2) how those changes and differences are related to chronic and session-specific aspects of family tension, while also testing/controlling for differences across parent sex and two versions of SFP 10–14. Results show that, on average, engagement increased over time, linearly with some deceleration, with substantial differences in both level and rates of change. Higher in-session chronic family tension was related to lower initial levels of engagement but not rates of change. Sessions when families displayed more session-specific tension were characterized by different levels of engagement for parents, depending on their level of chronic tension. Overall our results highlight the importance of considering engagement as a dynamic construct that changes over time in complex ways. Further understanding of the many factors that influence engagement can promote both better delivery and better uptake of intervention curriculum.

A prevention program’s efficacy is strongly related to the quality with which it is implemented (Durlak & Dupree, 2008; Dane & Schneider, 1998). So far, program evaluation efforts have focused on evaluating consistency in delivery (i.e., implementation fidelity), especially with respect to adherence and quality (Dane & Schneider, 1998; Dusenbury, Brannigan, Falco, & Hansen, 2003). However, the focus on consistent delivery covers only the interventionist-driven aspect of program success and tends to ignore the role participants play in implementation (Schulte, Easton, and Parker, 2009). Participants are not just passive recipients of content, but active learners with varying levels of cognitive, affective, and behavioral involvement in the intervention process. Effective implementation is thus a complex process involving both the successful delivery of an intervention by an interventionist and the receipt and use of the intervention concepts by participants (Berkel, Mauricio, Schoenfelder, & Sandler, 2011). Our interest in this study is to examine how and why participants’ engagement - a dynamic process wherein a participant interacts with other participants, the interventionists, and the curriculum – changes over time.

Participant Engagement: More than Just Attendance

Generally, greater participant engagement has been associated with improved outcomes in evidence-based interventions (e.g., Newcomb, Rabow, Hernandez, & Monto, 1997; Nix et al., 2009). Studies have found positive associations between program attendance and outcomes, although effects are inconsistent across studies. Some studies suggest that higher attendance (Prado, Pantin, Schwartz, Lupei, & Szapocznik, 2006; Gross et al., 2009) is related to improved program outcomes, while others do not find effects of attendance on outcomes (e.g., Ogden & Amlund-Hagan, 2008).

Compared to attendance, behavioral aspects of engagement are less often studied but appear to be consistently associated with program outcomes (e.g., Breitenstein et al., 2010; Nix et al., 2009; Teti et al., 2008). These studies have focused on behavioral aspects of participant engagement, including level of active in-session participation (Nock and Kazdin, 2005), quality of participation, completion of homework (Nix et al., 2009; Dumas, Nissley-Tsiopinis, & Moreland, 2007), interest (Orrell-Valente, Pinderhughes, Valente, Laird, & the Conduct Problems Prevention Research Group, 1999), paying attention, being supportive of other participants (Breitenstein et al., 2010), better memory of sessions (Buckley & Sheehan, 2009) positive attitude toward interventionist, and understanding of content (Korfmacher, Kitzman, & Olds, 1998). The relations between engagement and program outcomes may be more consistent than dosage effects because observed behaviors that are consistent with learning introduce discriminating information among those participants who attend a given session. Said another way, despite sufficient attendance, some participants may not learn curriculum content because they attended sessions that were not relevant for them, or because they were not attentive and involved (i.e., engaged) in sessions that they attended. At the same time, some participants may learn content despite low attendance because they compensated for absence by actively engaging in other sessions.

Because family-based prevention programs are often delivered in a group-based format to multiple parents together, conceptualizations of engagement may also need to attend to group-level dynamics. Measures of participant engagement in group therapy often focus on multiple dimensions including positively contributing to group process (e.g., Cunningham & Henggeler, 1999; Levenson, Macgowan, Morin, & Cotter, 2009; Macgowan, 1997; Tetley, Jinks, Huband, & Howells, 2011). Engagement in group therapy has been found to be associated with more positive outcomes (e.g., Smith, Duffee, Steinke, Huang, & Larkin, 2008). In sum, evaluation of prevention programs could benefit from further exploring whether and how participants’ engagement with group members and whole-group dynamics contribute to intervention outcomes.

Changes and Predictors of Participant Engagement

A critical limitation in the studies of engagement thus far is that engagement has been measured and treated analytically as a static, or trait-like, characteristic of the individual even though there are many reasons to expect that engagement changes across the course of an intervention. Coatsworth and his colleagues (2006) found that several dimensions of engagement, including positive and negative alliance, participants’ level of leadership in interventions, and participants’ relationships with group members and interventionists, do change over time. In an effectively designed and implemented intervention, engagement is likely to increase over time as participants become more familiar with the people and context, experience positive interactions with both interventionists and other participants, and gain confidence in the relevance and quality of the curriculum. Other possibilities are that engagement may decline steadily over time as the content gets repetitive, irrelevant, or difficult. Or, engagement may fluctuate up and down depending on the personal relevance of particular content and/or changes in personal circumstances and stress levels. There seems much to gain from tracking how participants’ engagement changes over time as individuals progress through a curriculum and from identifying specific individual, family, and contextual factors influence engagement. We would then be better positioned to optimize intervention uptake, and the benefits that follow.

A wide variety of factors are likely to influence participant engagement. For example, in family-based interventions (the focus of this study), parents are asked to reflect on their experiences parenting their child, and family members are often asked to interact with each other. For these reasons, concurrent family dynamics are very likely to influence engagement with and uptake of the curriculum. The first studies relating family functioning to participant engagement suggest either that families who are experiencing conflict or tension have higher attendance (Connell, Dishion, Yasui, & Kavanagh, 2007), or that families who are experiencing less conflict or tension have higher attendance (Perrino, Coatsworth, Briones, Pantin, & Szapocznik, 2001; Tolan & McKay, 1996). In either case, the implication is that family tension is a factor likely to influence more than just attendance, but also other aspects of engagement. Examining how session-to-session changes in family tension over the duration of an intervention are related to changes in engagement provides a first step towards identifying the specific factors that may influence participant engagement. This previous research has examined family tension as differences between individuals; a family’s experience of tension can also vary over the duration of an intervention. Thus, it may be that higher-than-usual tension sparks low engagement, for example.

The Present Study

There were two primary goals in this study. The first goal was to confirm that engagement does indeed change over time. To do so, we applied growth curve models to repeated measures of engagement obtained during families’ participation in a seven-week preventive intervention program, the Strengthening Families Program: For Parents and Youth Ages 10–14 (SFP 10–14). The program curriculum is described more fully in terms of structure and protective factors targeted in the Method section. To examine questions of how engagement changes over the course of the program we identified the shape of the engagement trajectory and the extent of between-person differences in change. Based on findings by Coatsworth and colleagues (2006), we hypothesized a general increase in engagement over time, but also that there would be substantial between-person differences in the direction and rate of change.

The second goal was to examine how the observed changes in engagement were related to family functioning, particularly family tension. Generally, we hypothesized that higher levels of tension would be associated with less favorable engagement trajectories. For example, engagement would generally be lower and increase less rapidly in families with higher overall levels of tension. Considering the ongoing dynamics of families, we additionally hypothesized that parents’ engagement would be lower than usual during those sessions when their family’s tension was higher than usual. Our models additionally consider whether trajectories of engagement differed across parent sex (male, female) and two versions of the SFP 10–14 curriculum. In terms of program fidelity our interest was both to test whether trajectories of engagement differed by sex and/or altered by our program modifications and to control for any such differences.

Method

Data are drawn from a randomized controlled trial (RCT) testing the efficacy of (1) SFP 10–14 (Molgaard, Kumpfer, & Fleming, 2001), (2) a modified version of the program Mindfulness-enhanced Strengthening Families Program (MSFP; Coatsworth et al., 2014), and (3) a literature control condition. SFP 10–14 is a family intervention that targets youth substance use initiation by (a) teaching parents a variety of skills, including monitoring, effective discipline and clear communication about expectations, (b) teaching youth a variety of skills, including peer-resistance, and stress management, and (c) promoting positive family interactions through model activities (Molgaard, Spoth & Redmond, 2000). The program consists of seven weekly two-hour sessions where, during the first hour of each session, SFP 10–14 “facilitators” separately lead a group of parents (one interventionist) and a group of youth (two interventionists) in skill building exercises. During the second hour, all three facilitators lead youth and parents through skills practice and family activities. The modified program, MSFP, follows the same timing and structure of SFP 10–14, but mindfulness activities and principles are infused into the parent component of each session (Coatsworth et al., 2014).

Participants

All families of 6^th and 7^th graders from four communities in central Pennsylvania were eligible to participate in the RCT. Participants were recruited through school and community events. Families registered for the research study were randomly assigned to one of the three conditions, maintaining balance of sex and grade of child, number of parents participating, and number of siblings needing childcare in order to ensure feasibility of active participation. The trial was conducted in six cohorts over three years and in four communities, resulting in 16 SFP 10–14 groups and 16 MSFP groups.

Participants included in this study were 252 parents from the first four cohorts (26 implementation groups) of the RCT who were (a) assigned to either an SFP 10–14 or MSFP group and (b) attended at least three of the seven sessions. Parents in the control group and those with very low attendance (n = 74) were not included in the sample because either engagement with the literature materials was not assessed, or the number of repeated measures was deemed insufficient for tracking nonlinear change and/or reliably quantifying within-person associations. Demographic characteristics and other descriptors are summarized in Table 1. Notably, although the high proportion of White parents reflects the demographics of the region, the sample was diverse in terms of income, education, and marital status. The youths’ ages are reflective of the recruitment-targeted school grades (M = 12.05, SD = 0.64). Parents who attended less than 3 sessions were similar to those who attended more sessions in many respects (income, curriculum version, partnered status, parent Latino ethnicity, youth grade, youth sex, or youth Latino ethnicity, ps > .05), but differed in that parents who attend less often had slightly older children (z = 2.43, p < 05), greater likelihood of being fathers (χ² = 12.73, p < .05), and lower initial (z = −2.07, p < .05) and average level of engagement (z = −3.49, p < .05). Based on the included sample, the results of this study can be generalized to parents who attend SFP 10–14 for three or more sessions; these parents are similar to the population that enrolls in SFP 10–14 except that they are more likely to be mothers, attend with a younger children, and show higher engagement than the rest of the population.

Table 1.

Descriptives for sample of N = 252 parents participating in 26 SFP 10–14 and MSFP groups

Characteristic	Statistic
	M	SD
Annual family income1	$111,727.17	$336,285.45
Attendance (n sessions)a	5.76	1.19
Youth age	12.05	0.64
	% Parents	% Youth
Modified curriculum	50.79
Partnered2	82.84
Current marital status3
Single, never married	4.85
Widowed	0.00
Divorced	8.81
Separated	5.73
Living in a marital-like relationship	7.93
Married and living with spouse	72.69
Education4
Partial high school	2.82
High school graduate/GED	22.58
Partial college or specialized training	26.61
College graduate	24.19
Graduate training	23.79
Male	32.54b	45.63c
Latino	2.38	7.54
Race
Black/African American	11.90	14.29
Asian	5.56	5.56
White	89.76	75.21
More than one of listed	1.98	4.37
Other	0.79	1.59

Note. Complete (N = 252) data unless otherwise noted.

^aSample includes only parents who attended three or more of seven sessions

^bPercent of parents who were fathers/male caregivers

^cPercent of parents who attended with sons/male youth

¹Percent of N = 242 who chose to respond

²Percent of N = 239 who chose to respond

³Percent of N = 227 who chose to respond

⁴Percent of N = 248 who chose to respond

Measures

Participant engagement and family tension were assessed directly by interventionists through observation and consensus. Prior to the first session, interventionists were trained to observe and rate specific aspects of family members’ behavior. Throughout the study period, the interventionists collaboratively discussed the ratings with each other and their supervisors in order to achieve consensus ratings and to maintain calibration of scores across groups and time.

Participant engagement

Each week, the interventionist leading the parent component of the session rated each parent on five dimensions of behavioral engagement: engagement/participation, interest, resistance (eventually removed from scale), positive affect toward leaders, and positive affect toward other parents/group members. For example, the interventionist noted the extent to which each “Parent was actively engaged and readily participated in parent session group discussion/activities” using a “Rarely or never” = 1 to “Always or almost always” = 4 scale. Examination of item covariances and response distributions suggested that the ‘resistance’ item tapped a set of behaviors unrelated to the other aspects of engagement. Removing that item, a composite score was calculated as the sum of responses to four items (Cronbach α = .875), with higher scores indicating greater engagement of the parent in that session. Across the 1445 ratings the average parent was engaged “Often (3)” or “Always or Almost Always (4)” (M = 3.514, SD = 0.577). However, as seen in Figure 1, parents differed in both their overall levels of engagement and how their engagement changed across the seven sessions.

Figure 1.

Changes in Engagement Across Sessions. Lines depict how a randomly selected subset of 35 parents’ engagement changed over seven weekly intervention sessions. Readily apparent are both between-parent differences and within-parent changes (both trends and week-to-week fluctuations).

Family tension

Throughout final portion of each session, the interventionists circulated among the families, noting the quality of each family’s interactions. After the session, the three interventionists collectively discussed their observations of each family to reach a single, consensus rating of each family’s level of tension. Specifically, unanimous agreement was reached for a response to the item, “There appeared to be some tension or disagreement between family members during the activities,” using a Rarely/Never = 1 to Always/Almost always = 4 scale. Across the 1348 ratings level of family tension was generally low (M = 1.289, SD = 0.669). Acknowledging that families differ in characteristic levels of tension, and that tension may vary within-family from week to week, the repeated measures were used to calculate separate indices of in-session chronic family tension, computed as the mean level of family tension across all available (3+) observations, and session-specific tension, computed as the deviation from those family-specific means (see e.g., Schwartz & Stone, 2007). On average (N = 252) chronic family tension was low (M = 1.298, SD = 0.438). Session-specific tension fluctuations typically stayed within one rating scale category (across 1348 ratings, SD = 0.551).

Time

Progression through the program is indexed by weekly session numbers, coded 0 to 6. Time, thus, represents overall progression of the program, acknowledging that specific content differs from week to week.

Control Variables

Two additional binary variables were included in analyses to test and/or control for differences in engagement by parent sex (female/mother vs. male/father) and program type (SFP 10–14 vs. MSFP).

Data Analysis

Growth curve models, implemented in a multilevel framework (Ram & Grimm, 2007; Singer & Willett, 2003), were used to accommodate the nested nature of program delivery and data collection across 4 levels of sampling (repeated measures nested within parents nested within groups nested within communities/school districts with slightly different demographic characteristics). Specifically, three sets of models were used to examine how variance was distributed across levels of analysis (Model 0), the prototypical pattern of change in parent engagement across time and the between parent and between group differences therein (Model 1), and how family tension and other predictors were related to those changes (Model 2).

Hierarchical structure

In a preliminary step we sought to establish a hierarchical model that would parsimoniously accommodate the complexity of the nesting and accurately represent the distribution of variance across levels. Using a series of unconditional means models (no predictors), all possible nestings were fit to the data and evaluated with respect to both substantive and statistical features. For example, although community/school district were intentionally selected to broaden the demographic characteristics of the sample, community-level variance components were non-significant. Thus, the community-level nesting was deemed unnecessary for the accurate representation of changes in engagement and not examined further. Similarly, although parents were nested within families, within-family variance would not interpretable because in 98 of 176 families data from only one parent was available. Thus, the family level of nesting was not examined. The best baseline model (Model 0) indicated that variance in parent engagement was distributed across three levels, with differences across the repeated measures, parents, and groups contributing 58.88% (σ²_εjit = 0.200, p < .05), 20.46% (σ²_vji0 = 0.069, p < .05), and 20.68% (σ²_uj00 = 0.070, p < .05) of total variance, respectively.

Change in parent engagement across sessions

The 3-level structure was then used to examine how parent engagement changed over time. Specifically, the fits of a series of models with polynomials of time (time, time², time³, etc.) included as predictors were compared to determine the prototypical pattern of change (linear, quadratic, cubic, etc.) and establish the most appropriate structure of the between-parent and between-group differences therein. Models with various random effects structures for the intercepts and slopes (e.g., Level 2 variances and covariance for intercept and linear slope; Level 3 variance in intercept only) were compared using AIC and BIC fit statistics and the best structure selected (Model 1 in Table 2).

Table 2.

Results from Growth Models of Parents’ Engagement

	Model 1: Growth curve model		Model 2: Growth curve model with predictors
	Estimate	SE	Estimate	SE
Fixed effectsa
Intercept, γ000	3.296*	0.089	3.274*	0.094
Time, γ001	0.131*	0.038	0.151*	0.043
Time2, γ002	−0.011*	0.005	−0.015*	0.006
Session-specific family tension, γ003			0.012	0.035
Chronic family tension, γ020			−0.157*	0.054
Chronic * Session-specific family tension, γ013			−0.159*	0.071
Father, γ010			−0.016	0.045
Curriculum Version, γ100			−0.117	0.110
Random effects
Level 2 (person)
Variance intercept, σ2νji0	0.075*	0.010	0.077*	0.010
Covariance intercept, session-specific family tension, σ2νji0vji3			0.009	0.009
Variance session-specific family tension, σ2νji3			0.015*	0.007
Level 3 (group)
Variance intercept, σ2uj00	0.180*	0.058	0.201*	0.067
Covariance intercept, time, σ2uj00uj01	−0.057*	0.022	−0.068*	0.026
Variance time, σ2uj01	0.028*	0.011	0.037*	0.014
Covariance intercept, time2, σ2uj00uj02	0.007*	0.003	0.008*	0.003
Covariance time, time2, σ2uj01uj02	−0.003*	0.001	−0.004*	0.002
Variance time2, σ2uj02	0.000*	0.000	0.001*	0.000
Residual variance, σ2εjit	0.155*	0.006	0.145*	0.007
−2LL	1868.3		1667.6
AIC	1884.3		1687.6

Note. Unstandardized estimates and standard errors. Effects are scaled in units of engagement scale scores (1–4) per session. Model based on up to 7 occasions nested within 252 participants for a total of 1445 observations with engagement scores (Model 1) 1284 observations with all predictor variables (Model 2). Participants comprised 26 implementation groups. −2LL = −2 Log Likelihood; AIC = Akaike Information Criterion, relative model fit statistics. Model 2 parameters sometimes indicate higher variance than Model 1 parameters because of change in centering location caused by the interaction term in Model 2.

^aPositive effects should be interpreted for higher session-specific tension, higher chronic tension, later sessions, fathers, and modified curriculum. Intercept represents session 1 for the average participant.

^*p < .05

Predictors of change in parent engagement

The best model of change was then carried forward and expanded to examine how family tension and other predictors were related to those changes. Specifically, parent sex (male, female) and program type (SFP 10–14, MSFP), session-specific family tension, and chronic family tension variables were included as predictors. The initial model included all possible main effects and interactions. Then, non-significant higher-order interactions were trimmed iteratively for parsimony. The final model (Model 2 in Table 2) was

$$\text{Level}1:{\mathit{engagement}}_{\text{jit}}={\mathrm{\pi }}_{\text{ji}0}+{\mathrm{\pi }}_{\text{ji}1}{\mathit{time}}_{\text{jit}}+{\mathrm{\pi }}_{\text{ji}2}{{\mathit{time}}^{\mathit{2}}}_{\text{jit}}+{\mathrm{\pi }}_{\text{ji}3}{\mathit{sessionfamtension}}_{\text{jit}}+{\mathrm{e}}_{\text{jit}}$$ (1)

$$\text{Level}2:{\mathrm{\pi }}_{\text{ji}0}={\mathrm{\beta }}_{\mathrm{j}00}+{\mathrm{\beta }}_{\mathrm{j}10}{\mathit{father}}_{\text{ji}}+{\mathrm{\beta }}_{\mathrm{j}20}{\mathit{chronicfamtension}}_{\text{ji}}+{\mathrm{v}}_{\text{ji}0}$$ (2)

$${\mathrm{\pi }}_{\text{ji}1}={\mathrm{\beta }}_{\mathrm{j}01}$$ (3)

$${\mathrm{\pi }}_{\text{ji}2}={\mathrm{\beta }}_{\mathrm{j}02}$$ (4)

$${\mathrm{\pi }}_{\text{ji}3}={\mathrm{\beta }}_{\mathrm{j}03}+{\mathrm{\beta }}_{\mathrm{j}13}{\mathit{chronicfamtension}}_{\text{ji}}+{\mathrm{v}}_{\text{ji}3}$$ (5)

$$\text{Level}3:{\mathrm{\beta }}_{\mathrm{j}00}={\mathrm{\gamma }}_{000}+{\mathrm{\gamma }}_{100}{\mathit{msfp}}_{\mathrm{j}}+{\mathrm{u}}_{\mathrm{j}00}$$ (6)

$${\mathrm{\beta }}_{\mathrm{j}10}={\mathrm{\gamma }}_{010}$$ (7)

$${\mathrm{\beta }}_{\mathrm{j}20}={\mathrm{\gamma }}_{020}$$ (8)

$${\mathrm{\beta }}_{\mathrm{j}01}={\mathrm{\gamma }}_{001}+{\mathrm{u}}_{\mathrm{j}01}$$ (9)

$${\mathrm{\beta }}_{\mathrm{j}02}={\mathrm{\gamma }}_{002}+{\mathrm{u}}_{\mathrm{j}02}$$ (10)

$${\mathrm{\beta }}_{\mathrm{j}03}={\mathrm{\gamma }}_{003}$$ (11)

$${\mathrm{\beta }}_{\mathrm{j}13}={\mathrm{\gamma }}_{013}$$ (12)

where the sample-level parameters represent the prototypical initial level of engagement (γ₀₀₀), differences across curriculum versions in parents’ initial level of engagement (γ₁₀₀), differences between mothers and fathers in initial levels of engagement (γ₀₁₀), differences in engagement attributable to differences in chronic family tension (γ₀₂₀), the prototypical linear and quadratic rates of change (γ₀₀₁, γ₀₀₂), differences in engagement attributable to session-specific family tension (γ₀₀₃), and the extent to which chronic family tension moderates the effect of session-specific family tension on engagement (γ₀₁₃). Residual us, vs and e_jit are unexplained between-group, between-parent and within-parent differences in engagement that were allowed to covary within level, but not across levels.

All models were fit to the data using SAS 9.3 (proc mixed; Littell, Milliken, Stroup & Wolfinger, 1996) with full information maximum likelihood (FIML) estimation. Incomplete data were assumed missing at random (MAR; Little & Rubin, 1987) based on clinical reports that absences were not systematically related to parents’ moods or aspects of program content and delivery but appeared to stem primarily from factors such as schedule-conflict and illness. All predictors (except time) were sample and/or person centered to facilitate interpretation of model parameters as representing effects for the prototypical parent (as described by the demographics in Table 1) and program.

Results

Change in Parent Engagement across Sessions

Parameters from the best model of change in parent engagement, Model 1 in Table 2, indicate a prototypical pattern of change with initial levels of engagement of γ₀₀₀ = 3.296, with modest, but significant linear increases in engagement of γ₀₀₁ = 0.131 per session, and a quadratic component providing some concave curvature (γ₀₀₂ = −0. 011). This prototypical trajectory, increasing engagement with a slight deceleration in change as the program progressed, is depicted graphically by the solid line in Figure 2. Cubic and quartic terms, which would accommodate additional curvature, were not significant.

Figure 2.

Changes and Differences in Parent Engagement across Sessions. Predicted trajectory for the prototypical parent is indicated by the solid bold line, indicating increases in engagement across session with some deceleration as the program progressed. Dashed lines indicate how parents with high (+1SD = short dash) levels of chronic family tension have lower initial levels of engagement compared to parents with low (minimum sore of −0.298 = long dash) levels of chronic family tension. Thin, gray lines depict parent-specific predicted trajectories based on final model (Model 2).

Between-parent and between-group differences were apparent in some components of change, but not others. Examining the random effects (bottom portion of Table 2), there was evidence of significant between-group differences in initial level of engagement (σ²_uj00 = 0.178) and both linear and quadratic rates of change (σ²_uj01 = 0.028, σ²_uj02 = 0.0004). However, within a group, parents only appeared to differ in their initial levels (σ²_νji0= 0.075), but not in how those trajectories proceeded over time (σ²_νji1 and σ²_νji2 were not significant and thus trimmed from the final model reported in Table 2). There were, of course, still substantial session-to-session changes in engagement unaccounted for by the systematic trends (σ²_ejit = 0.155).

Predictors of Change in Parent Engagement

Working from the model above, we examined how the noted changes and differences in engagement were related to family tension, parent sex, and curriculum version. Parameters from the final model (Model 2) are given in Table 2. Initial levels of engagement (γ₀₀₀) and linear and quadratic components of change in engagement (γ₀₀₁ and γ₀₀₂) were similar to those in Model 1. As shown in Figure 2, chronic family tension was related to engagement, such that families with higher chronic tension experience lower initial levels of engagement (γ₀₂₀ = −0.165). Contrary to expectations, session-specific family tension was not related to engagement for the prototypical parent (γ₀₀₃ = 0.012) and chronic family tension was not systematically related to parents’ rates of change in engagement (all non-significant interactions, e.g. chronicfamtension*time, were trimmed from the final model). The control variables, father (γ₀₁₀ = −0.016) and curriculum version (γ₁₀₀ = −0.117), were not systematically related to levels of parents’ engagement or rates of change in engagement. There was evidence that chronic family tension moderated the relation between session-specific family tension and engagement (γ₀₁₃ = −0.159). As shown in Figure 3, on occasions when low-chronic tension families experienced high session-specific tension, those parents tended to showed higher engagement, whereas on occasions when high-chronic tension families experienced high session-specific tension, those parents tended to show lower engagement. There was also evidence of within-group differences in the association of session-specific family tension and engagement (σ²_vji3 ⁼ 0.015) after accounting for the other predictors. This indicates that session-to-session fluctuations in family tension impacted parents in different ways, some more than others.

Figure 3.

Chronic Family Tension Moderates the Extent to which Session-specific family tension Influences Engagement. Each dot represents a parent’s predicted engagement score for each observed level of session-specific family tension. Bold lines represent the extent of the relation between session-specific family tension and engagement for low (minimum sore of −0.298, black) and high (+1SD, grey) levels of chronic family tension.

Overall, pseudo-R², calculated as the correlation between observed engagement scores and Model 2 predicted scores, was r = .272.

Discussion

This study systematically examined how parents’ engagement changed across a seven-week intervention and identify how specific factors such as family tension may influence that change. Two main findings emerged from our analyses: (1) the average parents’ engagement during SFP 10–14 and MSFP increases over time, linearly with some deceleration, and (2) aspects of family tension were, as expected, related to both initial levels of engagement and session-to-session changes in engagement.

Change in Parent Engagement across Sessions

The first goal of this study was to confirm that engagement is an aspect of participant experience/behavior that does indeed change over time rather than a static person-level characteristic. Applying growth curve models to repeated measures of engagement obtained during families’ participation in a preventive intervention curriculum, we found that there were, on average, significant increases in participants’ engagement across the seven weeks of the interventions. While the specific pattern of change (linear increase with some concave curvature) is specific to this study context (the structure of the SFP 10–14 curriculum), our results are consistent with other studies showing that implementation process measures such as positive and negative alliance and participant leadership change over time (Coatsworth et al., 2006). It is especially promising that overall engagement is high, even at the first session, so that engagement is increasing from an already-high level. This finding suggests that the participants find the structure and content of SFP 10–14 interesting, enjoy participating, and develop positive affect toward both leaders, and other parents/group members. The overall increase in engagement across sessions indicates that, in this implementation, participants played an increasingly important role in a process that involved dyadic and group interactions among participants and interventionists, with each other and with the curriculum (Berkel et al., 2011; Schulte, Easton, and Parker, 2009). In sum, the participant driven aspects of the process appear to have been successful.

Beyond the average trajectories, there was evidence of differences among groups. In particular, the random effects in Models 2 and 3 indicated significance between-group differences in linear change. That is, some groups’ engagement increased faster than others. Interpreted with respect to the interventionist-driven aspects of program delivery, these differences in rates of change may indicate differences in adherence or quality of delivery across groups in this sample. Interpreted with respect to the participant-driven aspects of program receipt, differences in change across groups may indicate how groups differed may be received by groups of participants differently. Because of the interactive nature of implementation, it is most likely that the differences in change between groups reflect differences in both delivery of an intervention by an interventionist and the receipt and use of the intervention concepts by groups of participants.

Predictors of Change in Parent Engagement

The second main finding of this study provides insight into the complex ways that family tension may impact intervention engagement. Our findings suggest that there is a main effect of chronic family tension such that families displaying high tension across sessions tended to be less engaged than low-tension families, which is similar to other research suggesting that low-functioning families attend fewer sessions (Perrino et al., 2001; Spoth et al., 1996). A parallel within-family process was also present but moderated by level of chronic family tension such that session-specific family tension have a different impact on engagement, depending on the normative level of tension in the family. The results suggest that parents from lower-tension families respond to higher-than-usual tension by increasing their engagement in that session. Session-specific discord may motivate these parents to engage in the session. This interpretation is consistent with previous studies showing that higher family conflict and tension motivates attendance (Connell et al., 2007; Prado et al., 2006). In complement, parents from higher-tension families respond to higher-than-usual tension by decreasing their engagement in that session. One potential explanation for this finding is that high-tension families experiencing sessions with high tension may be more distressed, overloaded, and unable to constructively leverage this tension for change. This interpretation is consistent with psychological studies showing that hyper-stress causes overloads that impair functioning (Hancock, 1989). The result of such interplay between chronic and session-specific tension may be that higher tension, higher need families miss some of the important content of the intervention during days when they might be able to benefit most from the intervention.

The more general implication of this finding is that participants’ current states and proximal experiences, including other factors not measured or modeled here, influence participants’ observed engagement – processes that, in turn, are likely to affect how effective the intervention can be. To the extent that this is true, group delivery may need to be modified in order to dynamically, in real-time, respond to participants’ current needs in order to ensure optimum engagement for each individual at each session. For example, for high-tension families in the programs studied here, SFP 10–14 facilitators should try to mitigate and process high session-specific family tension to ensure that parents are kept engaged and deal constructively with this tension.

In terms of practical relevance to the field, research on participant engagement has much to contribute by illuminating the stable (time-invariant) and time-varying factors that influence engagement, and how engagement may change. Armed with such knowledge, intervention programs can be designed in ways that dynamically optimize participant engagement, and improve intervention effectiveness.

Limitations and Future Directions

This study provides a first look at how engagement changes over the course of the intervention and identifying specific factors that contribute to those changes. The findings are compelling, but some cautions are warranted. We studied changes in engagement with a sample of families participating in a specific family-based intervention, SFP 10–14 and MSFP. The patterns of change in engagement found here are specific to the structure of these programs’ curriculum and delivery. Trajectories of engagement will likely be somewhat different in programs with different content, structure, group size, and number of sessions. As well, we limited our sample to those N = 252 families that had attended more than 2 of 7 weekly sessions. Prioritizing examination of within-family associations, we excluded from our analysis the 74 families that either did not attend or provided data on only 1 or 2 occasions. Our rationale for treating the severely incomplete data in this way was that these “infrequent attenders” may represent a different population of parents/families than “consistent attenders.” Indeed, previous research on parent attendance has demonstrated that parents who do not attend and attend fewer sessions have different characteristics than parents who reliably attend an intervention program (e.g., Bloomquist et al., 2009; Coatsworth et al., 2006; Dillman Carpentier et al., 2007; Eisner & Meidert, 2011). Further work should examine repeated measures for evidence of different subpopulations and describe any differences in how engagement changes (e.g., using growth mixture modeling; Ram & Grimm, 2009). Likely, as in substance use, there may be early, later, and sporadic engagers.

The outcome measure of engagement used here was a four item composite that captured differences and changes in engagement/participation, interest, and positive affect toward leaders and other parents/group members. Extending measurement of engagement in ways that capture additional processes would allow for both broader conceptions of engagement and/or finer grained (e.g., item-level) analyses (Bamberger & Coatsworth, 2013). Our measures of parent engagement and family tension were based on interventionists’ ongoing observations of parent behavior and family interactions, respectively, during the program sessions. Training and post-session discussions were used to obtain consensus scores for each family for each session. This data collection procedure invoked consistent evaluation of behaviors across weeks through discussion among multiple trained raters, but precluded statistical assessment of inter-rater reliability. Future studies may also consider obtaining multiple ratings from trained observers, calculating inter-rater reliability each week and making adjustments based on those statistics. Real-time data flow will open new possibilities for data-informed discussion and adjustment of measurement procedures.

Our analysis used the hierarchical structure of weekly measures nested within families nested within groups to examined the role group-level (SFP 10–14 vs MSFP), family-level (parent sex, chronic family tension), and session-level (time, session-specific tension) factors have on parent engagement. We only included a few variables at each level. Future research should be purposively designed to examine a wide array of factors. For example, at the individual and family levels of analysis, demographic, behavioral, and relationship characteristics likely all play some role (Baydar, Reid, & Webster-Stratton, 2003; Connell et al., 2007; Dumas et al., 2007; Haggerty, MacKenzie, Skinner, Harachi, & Catalano, 2006; Kazdin & Mazurik, 1994; Nix et al., 2009; Orrell-Valente et al., 1999; Perrino et al., 2001; Prado et al., 2006; Spoth, Redmond, Kahn, & Shin, 1997; Teti et al., 2008). At the group level, we can study how specific aspects of the intervention context, including group dynamics, interventionist characteristics, and delivery quality, influence engagement. Notably, such “group-level” factors may themselves change over time, with the cadence of assessment carrying with is some important assumptions. For example, although we assessed family tension through observation of the behaviors exhibited during the intervention session, it is unclear whether the observed tension was induced in the moment by curriculum designed to challenge the prevailing family organization (cf. Minuchin & Fishman, 1981) or carried over into the session from family interactions occurring earlier in the day. Including assessments of family conflict or tension outside the session throughout the day may provide a fuller picture of the multitude of factors (e.g., problem behavior at home, parent stress, social calendar) influencing engagement.

In summary, our results indicate that parent engagement in preventive interventions should be addressed as a rich and multi-level dynamic process. Research is needed to further illuminate how engagement changes, how personal, family, group dynamics, and the dynamics of the intervention context influence engagement. We look forward to taking the next step to examine how and which aspects of participant engagement provide conditions for learning, intervention uptake, and families’ and youths’ improvement in intervention-targeted outcomes.