Mechanisms of Change in a Cognitive Behavioral Couples Prevention Program: Does Being Naughty or Nice Matter?

Abstract

Although there is a body of evidence suggesting beneficial effects of premarital prevention, little research directly examines the mechanisms of effect. One study that examined changes in communication following training in the Prevention and Relationship Enhancement Program (PREP) found that, although couples made the expected communication gains pre to post PREP, female gains in positive communication were paradoxically associated with worse, not better, outcomes (Schilling et al., 2003). Using two samples, the current investigation did not yield evidence of such an association. We discuss issues related to replication studies (e.g., failure to reject null hypotheses), challenges in analyzing and interpreting dyadic data, and implications for prevention practice.

For couples entering marriage, the risk for marital distress and dissolution is high, with various negative implications for adult and child welfare (Amato, 2000). As a result, a range of public and private sector marriage and relationship education activities have received growing emphasis in public policy discussions (Parke & Ooms, 2002), including efforts to prevent marital distress and divorce (e.g., Stanley, 2001). Although concerns have been raised about whether or not such interventions are reaching couples most likely in need of them (Sullivan & Bradbury, 1997), various reviews of the effectiveness of premarital, preventive education note positive findings (Carroll & Doherty, 2003; Halford, Markman, Kline, & Stanley, 2003; Sayers, Kohn, & Heavey, 1998). Furthermore, analyses in a large, random sample, suggest that the positive effects of premarital education extend across lines such as income and race, although access to such services is more limited for those who are less religious or who have lower income levels (Stanley, Amato, Johnson, & Markman, 2006).

A widely held view has been that communication quality early in marriage and prior to it will be partly determinative of future outcomes. Early on, Markman found that communication and conflict management deficiencies that exist early in a relationship lead to an erosion of satisfaction in marriage later on (e.g., Markman, 1981; Markman & Floyd, 1980). Since then, numerous longitudinal studies have suggested that negative communication (e.g., escalating arguments, withdrawal, name-calling or invalidations) predates or exacerbates the development of marital distress and divorce (e.g., Clements, Stanley, & Markman, 2004; Gottman, 1993; Karney & Bradbury, 1995; Markman & Hahlweg, 1993). As such, many premarital prevention programs include a focus on communication skills. There are, of course, other views of marital deterioration that have empirical support (e.g., Huston, Caughlin, Houts, Smith, & George, 2001), as well as a trend in the broader field of marital and relationship research toward considering more positive and complex dynamics, including those that are potentially transformative (Fincham, Stanley, & Beach, in press). Communication is, nevertheless, widely viewed by both researchers and practitioners as a crucial element in understanding and affecting couple outcomes.

Coie et al. (1993) describe how outcome studies can be used to revise theory, with revised theory subsequently affecting prevention programs, and so forth. A specific enactment of this goal is to examine mechanisms of change (e.g., whether the targeted changes in behavior are actually related to future outcomes). At the same time, addressing mechanisms of change in prevention research with couples is particularly challenging. Not only does such work raise complex methodological issues about the analysis of dyadic processes (a focus of the current paper), it also raises issues about the nature of prevention work with couples more generally. Specifically, the analysis of change in prevention samples is complicated by ceiling effects when the focus is primary prevention around risk factors that may be ubiquitous but not universal. For example, although negative communication is a clear risk factor for marital distress and divorce, a representative sample of premarital couples will contain many couples who do not demonstrate poor communication. Prevention efforts focused on reducing negative communication may have little impact on such couples (W. Kim Halford, personal communication, February 19, 2006). Further, some effects that are truly preventive may be harder to assess since they have to do more with preventing declines over long periods of time than with the need for immediate improvements in functioning.

A recent paper directly tested whether changes in communication quality during premarital education affected longer term outcomes (Schilling, Baucom, Burnett, Allen, & Ragland, 2003). Schilling et al. evaluated a weekend version of the Premarital and Relationship Education Program (Markman, Stanley, Blumberg, Jenkins, & Whiteley, 2004; Stanley, Blumberg, & Markman, 1999), a cognitive-behavioral prevention program for couples. In their sample of 39 premarital couples, they found that men and women showed reductions in objectively coded negative communication and increases in objectively coded positive communication from pre to post intervention, as in numerous other studies on variations of PREP (Hahlweg, Markman, Thurmaier, Engl, & Eckert, 1998; Halford, Sanders, & Behrens, 2001; Markman, Renick, Floyd, Stanley, & Clements, 1993; Stanley et al., 2001). Further, couples generally maintained high levels of marital satisfaction for several years afterward.

Also consistent with assumptions about preventive effects, Schilling et al. (2003) found that reductions in men’s negative communication and increases in men’s positive communication were associated with better marital outcomes over time. However, they also obtained a paradoxical finding wherein increases in women’s positive communication were predictive of more risk for distress. Such a finding was not without precedent. For example, Gottman and Krokoff (1989) found that higher levels of positive communication by women predicted greater marital satisfaction cross-sectionally but lower levels of satisfaction years later. Similarly, Karney and Bradbury (1997) reported that female positive communication relative to negative communication was associated with greater declines in satisfaction over time among newlyweds (see also Heavey, Layne, & Christensen, 1993). More recently, Baucom, Hahlweg, Atkins, Engl, and Thurmaier (2006) showed that nine of 75 women demonstrated extreme increases in nonverbal positive communication following a German version of PREP (effect sizes up to 4.6), and these nine were subsequently at greater risk for distress than women with less extreme changes. These studies suggest that some kinds of “negative” communication may benefit marriages and some kinds of “positive” communication may harm marriages. However, this is an inconsistent and poorly understood phenomenon, and one that could merely be a methodological artifact (Woody & Costanzo, 1990).

The purpose of the present study was to test whether the pattern found by Shilling et al. could be replicated in longer term and/or larger samples. Perhaps of even greater importance, we also focus on statistical issues that affect outcomes in the analysis of communication behavior within dyads. We present analyses from two studies on the effects of PREP. One sample is from the Denver Family Development Study (DFDS; see Markman, Floyd, Stanley, & Storaasli, 1988; Markman et al., 1993). The DFDS used an early version of PREP that closely matches the version used in the Schilling et al. (2003) paper (for a description, see Baucom, Epstein, Burnett, & Rankin, 1993). The other is from the Family Stability Project (FSP) (see Stanley et al., 2001), which uses a more recent version of PREP in a broad-based, carefully constructed community sample. Consistent with the model of prevention science developed by Coie et al. (1993), PREP is continually updated to reflect new research findings, although the core communication skills taught to couples have remained consistent over the years. The DFDS sample offered long-term follow-up data on the entire sample but with a small sample size. The FSP sample is much larger, but fewer long-term data are available because recruitment occurred over a period of seven years and ended only recently. We choose not to combine the two samples because, although analyses of both samples can speak to the question at hand, the studies are different in terms of cohort, version of PREP, and outcomes. In both of these studies, couple communication quality was coded using the same procedures and coding system as in Schilling et al.’s study.

Method

As noted earlier, we used two separate samples to evaluate the research questions. They were collected as part of separate studies, so the methods for the two samples (FSP and DFDS) are detailed here individually. For the sake of parsimony and to facilitate comparison of findings, the results for the two separate samples are presented in a single results section.

FSP Participants and Procedure

Methods for FSP are described in detail elsewhere (e.g., Stanley et al., 2001). For this larger project, 306 couples initially were recruited through the religious organizations where they planned to marry. Each religious organization that couples were recruited from was randomly assigned to one of three premarital interventions (two conditions consisted of training in PREP). Of the 306, 249 couples completed both the pre-training assessment (PRE) and the post-training assessment (POST). As noted earlier, we chose to use only couples who had received PREP (no control couples) so that our results would be most comparable to Schilling et al.’s (2003). (The same decision was made for the related Baucom et al. (2006) publication.) Of these 249 couples, 153 completed PREP either by university professionals or by clergy members trained in PREP. Of these, 126 couples had provided data at least one assessment following POST. However, there were 8 couples who were missing data on key variables and were excluded. Further, for 13 couples, the dichotomous variable regarding distress development (described later) indicated distress following POST, but they were already distressed at PRE. Therefore, they did not develop distress, as was our operationalization of distress; they merely stayed distressed throughout the study, so they were excluded. Thus, there were 105 couples who met our criteria to be included in the present study.¹

Men ranged from 19 to 53 years of age (M = 27.42, SD = 5.24), and women ranged from 18 to 51 (M = 26.49, SD = 5.54) at the pre-intervention assessment (PRE). On average, women’s annual income was between $15,000 and $29,999, and they had 15.30 (SD = 1.87) years of education. Men’s annual income, on average, ranged from $20,000 to $39,999, and they had 15.50 (SD = 1.86) years of education. Women in this sample were 83.5% White and 14.7% other and men were 87.4% White, 12.6% other. At the couple level, 76% were composed of two White partners; for 24% of the couples, one or both partners identified themselves as belonging to an ethnic minority group. Most participants (98%) were entering first marriages.

Couples came into the laboratory for PRE and POST, and returned roughly once per year thereafter. Participants were paid $40 for PRE, and $50 for POST and follow-up assessments. During each visit to the lab, partners completed questionnaires individually on computer or by paper and pencil and engaged in two videotaped 10–15 minute discussions. One was on a positive topic of their choosing and was not used for the present study. The discussion that was used here was a problem-solving discussion based on the couple’s most problematic issue or area of conflict, as identified on the self-report Marital Agendas Protocol (Notarius & Vanzetti, 1983). These problem-solving discussions were later coded using the Interaction Dynamics Coding System (see below). All procedures were approved by a university Institutional Review Board, and each individual in the study provided written informed consent.

The recruitment of couples took place between 1995 and 2002. As such, we include couples at widely different stages in the longitudinal procedures with regard to completion of follow-up assessments. The most complete data are from the first follow-up assessment (FU1). For reasons detailed later, our multiple regression analyses use only FU1 outcome data while the logistic regressions are based on all available data (up to FU5 for 36 of the couples).

FSP Measures

Couple interactions

The Interaction Dynamics Coding System (Julien, Markman, & Lindahl, 1989; Kline et al., 2004), a global coding-system for couples’ discussions of relationship problems, was used for our observational data on couple interaction. The system codes various positive and negative dimensions that have discriminated between distressed and non-distressed couples (Julien et al., 1989; Prado & Markman, 1998). Inter-coder reliability for the larger study from which the present study’s data were drawn is high; intra-class correlations range from .66 to .95, with a median intra-class correlation of .87 (Kline et al., 2004). For the present study, positive and negative interaction subscales were used (created based on factor analyses). The positive subscale included the positive affect, problem solving skills, support/validation, and communication skills dimensions (coefficient alpha for females’ subscale scores was .88; for males’ scores it was .89). The negative communication subscale included the negative affect, denial, dominance, withdrawal, conflict, and negative escalation dimensions (the coefficient alpha for females’ subscale scores was .88; for males’ scores it was .86).

Relationship adjustment

The Marital Adjustment Test (MAT; Locke & Wallace, 1959) is a widely used measure of relationship quality with acceptable reliability and validity (Markman et al., 1993; Stanley et al., 2001) as well as the ability to discriminate between distressed and non-distressed couples (Crane, Allgood, Larson, & Griffin, 1990). The coefficient alpha for the MAT for FU1 in this sample was .70. Scores in the present study ranged from 84 to 156, and the mean is consistent with generally happy, premarital/newlywed couples (M = 128.06, SD = 14.52).

DFDS Participants and Procedure

The second study used archival data from the DFDS; methods from this study are described in detail elsewhere (e.g., Markman et al., 1993). Twenty-nine couples from this study who participated in an early version of PREP were used here (Markman, Floyd, Stanley, & Jamieson, 1984). Couples were selected for the analyses presented here only if they received PREP, eventually married, and completed POST in addition to at least one follow-up.

Men in the DFDS sample used here range from 19 to 32 years of age (M = 25.10, SD = 3.96) at PRE; women ranged from 18 to 31 (M = 23.28, SD = 3.60) at PRE. On average, the women’s annual income was between $5,000 to $14,999 ($10,750 – $32,248 in 2005 dollars, according to the Consumer Price Index), and they had 15.07 (SD = 1.90) years of education. Men’s average annual income also ranged from $5,000 to $14,999, and they had 15.62 (SD = 1.57) years of education. Women and men in this sample were 95% White and 5% other.

Couples who were engaged or planning marriage were recruited through community-wide advertising. Couples visited the laboratory before premarital intervention (PRE), soon after intervention (POST), and at yearly intervals thereafter. They were paid for their participation throughout the study (starting at $25 per research session in the first years of the project and gradually rising to $50). During each visit to the lab, partners completed questionnaires and engaged in a videotaped 10–15 minute problem-solving discussion using the same procedure described in the FSP sample. As with the FSP sample, these problem-solving discussions were later coded using the Interaction Dynamics Coding System. All procedures were approved by a university Institutional Review Board, and each individual provided written informed consent.

DFDS Measures

We used the MAT and IDCS described above. The coefficient alpha for the MAT in the DFDS sample at FU1 was .70. PRE scores on this measure were high, as they were in the FSP sample, ranging from 54 to 152 (M = 124.11, SD = 12.08). As reported elsewhere (Julien et al., 1989), inter-rater agreement and internal consistency for the IDCS in the DFDS sample were both acceptable.

Results

Our analyses were designed to test for replication of Schilling et al.’s (2003) finding that increases in women’s positive communication, following PREP, were associated with higher risk for marital distress. We first present analyses that are most similar to those conducted by Schilling et al.: logistic and multiple regressions with both partners’ communication change scores entered simultaneously. We then present analyses with only one partner’s communication change score entered at a time. In doing so, we draw attention to statistical issues that arise when conducting regression analyses with dyadic data. To be consistent, we also tested for replication of the finding that decreases in negative communication were associated with lower risk.

Dichotomous Outcomes: Reliable Change in Relationship Adjustment and Divorce

Schilling et al. focused on predicting two types of individuals: maritally distressed versus non-distressed. We used the same reliable change method (see Jacobson & Truax, 1991) that Schilling et al. used to categorize participants as distressed or non-distressed at later follow ups. Given that dichotomous outcomes can squander important variance and limit power, we also present analyses with continuous outcomes (MAT scores) to further test for replication of Schilling et al.’s findings.

Classifying couples as distressed or non-distressed

Based on MAT reliability estimates in both the FSP sample and the DFDS sample (which had very similar reliabilities for the MAT [.70] and means), individuals with change scores from POST² of at least 22.4 and MAT scores < 100 at any follow-up were classified as distressed.³ In the FSP sample, the last follow-up that was included was FU5; 36 (34.9%) of the couples had completed this assessment. In the DFDS sample, data were available through FU5 for 25 (86.2%) of the couples, and through FU7 (on average, 8 years following PRE) for 15 (51.7%) of the couples. Although the DFDS sample has less power than the FSP sample because of sample size, the distressed outcome variable in the DFDS sample has more variability (a greater percentage had become distressed) because more time had passed to allow for deterioration in participants’ marriages. We used a cut-off of 100 on the MAT because other research has established this value as distinguishing between distressed and non-distressed couples (e.g., Clements et al., 2004; Rogge & Bradbury, 1999). Any couple that was separated or divorced at the time of the analyses was categorized as distressed regardless of prior adjustment scores. Individuals who landed in the distressed category due to the MAT score at one follow-up but who then “rebounded” to having a MAT score above 100 at a later time point (6 in FSP, 2 in DFDS) were treated as non-distressed. However, the results of the logistic regressions presented later were essentially the same whether rebounders were coded as distressed or non-distressed.

Positive Communication Change Predicting Distress Development: Logistic Regression

We used logistic regression to examine whether changes in positive communication were associated with the development of distress (i.e., whether distress developed or not during the study). In contrast, Schilling et al. used a modified form of survival analysis⁴ to assess the relationship between communication changes and marital outcomes. Survival analysis is a form of logistic regression that is typically used when a researcher wishes to examine the timing and odds of a dichotomous event (e.g., how long couples who divorce are typically married before divorcing; Keiley & Martin, 2005; Willett & Singer, 1993). However, the timing of onset was not the focus of Schilling et al.’s paper, nor is it the focus here. Because we were not interested in the timing of distress onset as much as whether it occurred or not, we used logistic regression instead of survival analysis as it seemed more parsimonious and better suited to these research questions. As we demonstrate below, we believe the crucial methodological issue that emerges in these analyses concerns the number and type of terms added as simultaneous predictors.

For all analyses, we formed residualized change scores for PRE to POST communication change, as did Schilling et al. (2003). They note that there are various issues and preferences expressed in the literature about the most appropriate change scores (e.g., the residual of POST scores after removing the variance of PRE scores, or simple difference scores). We share their preference for the residual method, though we have found virtually the same results regardless of the method used. Here, a larger value for positive communication residual scores indicates increased positive communication following intervention, and a larger value for negative communication residual scores indicates increased negative communication.

For the first set of logistic regression analyses, we simultaneously entered the terms that Schilling et al. entered to predict distress development: PRE female and male positive communication scores, female and male communication change scores, and the interaction between female and male change scores (see Table 1). For both the FSP sample and the DFDS sample, the overall logistic regression models were non-significant for both wife and husband distress development (p >.10). Thus, we did not replicate the paradoxical association Schilling et al. found between female positive communication changes and increased risk for distress development, nor do we replicate their association between male positive communication changes and decreased risk for distress development. Findings were not altered by including or excluding the interaction term (female change score X male change score).

Table 1.

Summary of Logistic Regression Analyses for Observed Positive Communication Variables Predicting Marital Distress Development for Wives and Husbands

	Wives			Husbands
Predictor	B	SE B	eB	B	SE B	eB
The FSP sample
Female pretest	−.22	.33	.80	−.27	.31	.77
Female increase	−.29	.36	.75	.01	.34	1.01
Male pretest	.28	.32	1.32	.23	.30	1.25
Male increase	.34	.36	1.41	.02	.32	1.02
Male increase X female increase	.10	.14	1.10	.03	.14	1.03
Constant	−2.16			−1.47
Model χ2	2.92			.90
df	5			5
% (N) distressed or divorced	14.3(15)			16.5(17)
The DFDS sample
Female pretest	.49	.39	1.63	.42	.39	1.53
Female increase	1.14	.79	3.12	.52	.71	1.67
Male pretest	.27	.42	1.31	.30	.40	1.35
Male increase	−1.01	.68	.36	−.48	.58	.62
Male increase X female increase	−1.24	.88	.29	−.61	.67	.54
Constant	−4.08			−4.16
Model χ2	6.81			3.46
df	5			5
% (N) distressed or divorced	34.5(10)			34.5(10)

Note. B = unstandardized logit regression parameter estimates, SE B = standard error of the estimate, e^B = exponentiated B. All variables were entered simultaneously. For the FSP sample, there were 105 wives and 103 husbands. In the DFDS sample, there were 29 wives and 29 husbands.

Positive Communication Change Predicting Relationship Adjustment: Multiple Regression

The above logistic regression model is most similar to the survival analysis model used by Schilling et al. To further examine the paradoxical pattern that Schilling et al. found, we decided to use a slightly different outcome: a continuous measure of relationship adjustment rather than a dichotomized distress development variable. We decided to use a continuous outcome variable because it might be more sensitive to changes in the predictor (though it cannot capture information about dissolution). We chose to examine FU1 MAT scores in both the FSP and the DFDS samples because we have the most power to detect differences at FU1 (because we have the largest sample at this follow-up in both samples). As with the logistic regressions already reported, these multiple regression equations include PRE female and male positive communication scores, and female and male changes scores, all entered simultaneously. In addition, we entered PRE MAT scores, to control for initial levels of relationship adjustment. The interaction term (male residual score X female change score) was not a significant predictor; therefore, the results presented here are without the interaction term.

As can be seen in Table 2, we found a trend in the FSP sample for the female positive communication change score predicting higher female FU1 MAT scores, controlling for all other variables in the model. In the DFDS sample, there was no evidence of a significant association between female changes and female MAT scores in either direction (Table 2). With regard to male FU1 MAT scores, none of the positive communication variables approached significance as predictors in either sample (Table 2).

Table 2.

Summary of Multiple Regression Analyses for Observed Positive Communication Variables Predicting Relationship Adjustment for Wives and Husbands

	Wives			Husbands
Predictor	B	SE B	β	B	SE B	β
The FSP sample
Pretest MAT	.58	.15	.41***	.48	.80	.32**
Female pretest	3.69	2.43	.21	−.19	2.51	−.01
Female increase	4.82	2.71	.29†	1.88	2.83	.12
Male pretest	−3.08	2.14	−.20	.07	2.19	.01
Male increase	−3.19	2.52	−.21	−.81	2.60	.01
Model statistics	F(5, 75) = 4.18,** adjusted R2 = .17			F(5, 73) = 2.00†, adjusted R2 = .06
The DFDS sample
Pretest MAT	.41	.20	.37*	.30	.25	.27
Female pretest	3.69	2.47	.33	.30	2.99	.04
Female increase	−3.25	3.86	−.19	−4.40	4.59	−.26
Male pretest	−5.79	2.37	−.55*	−1.66	2.63	−.16
Male increase	3.84	3.23	.26	1.64	3.71	.12
Model statistics	F(5, 22) = 2.91,* adjusted R2 = .26			F(5, 22) = .87, adjusted R2 = −.03

^***Note. p <.001,

^**p <.01,

^*p <.05,

^†p <.10. For wives, her pretest MAT score was entered, for husbands, his pretest MAT was entered instead. All variables were entered simultaneously.

The analyses in Tables 1 and 2 highlight an interesting phenomenon. There appears to be a tendency for the male and female positive change scores to predict in opposite directions (as one will note by examining the valences of the regression coefficients). That is, when the coefficient for one gender is positive, the other is typically negative (though in different directions in the two samples). In both samples, the examination of the zero order correlations between the variables in Table 3 shows that neither male nor female positive change scores are associated with relationship adjustment. But, the apparent reversals in signs appear when both partners’ scores are entered. This valence reversal is an important statistical phenomenon and dilemma for those conducting dyadic research (see Discussion).

Table 3.

Correlations among Variables of Interest for the FSP Sample and the DFDS Sample

Variable	1	2	3	4	5	6	7	8	9	10	11	12
1. Female PRE positive comm.	−	.62***	.12	−.05	−.46*	−.32	.16	.36	−.01	.33	−.04	−.01
2. Male PRE positive comm.	.66***	-	.06	.12	−.22	−.53**	−.10	−.01	−.15	.17	−.38*	−.10
3. Female positive comm. increase	−.06	.02	-	.60***	.12	.21	−.58***	−.02	−.19	−.26	−.09	−.27
4. Male positive comm. increase	.05	.06	.76***	-	.07	.13	−.63***	−.46*	−.03	−.05	.06	−.08
5. Female PRE negative comm.	−.53***	−.34**	−.03	−.12	-	.71***	−.11	−.23	−.25	−.50**	−.23	−.48**
6. Male PRE negative comm.	−.42***	−.60**	−.05	−.14	.82***	-	−.05	−.01	−.23	−.30	−.01	−.38*
7. Female negative comm. increase	.07	−.10	−.59***	−.42***	.02	.10	-	.62***	−.16	−.03	.07	.10
8. Male negative comm. increase	.02	.01	−.48***	−.59***	.09	.04	.74***	−	.09	−.03	.04	.09
9. Female PRE MAT	.33**	.20	−.03	.06	−.28*	−.19	.05	−.14	-	.42*	.48*	.60***
10. Male PRE MAT	.25*	.20	.11	−.04	−.34**	−.25*	−.03	−.07	.40***	-	.24	.32
11. Female FU1 MAT	.17	.01	.07	.01	−.15	−.09	−.05	−.16	.41***	.31**	-	.70***
12. Male FU1 MAT	.06	.06	.11	.02	−.14	−.15	−.17	−.26*	.22	.33*	.67***	-

^***Notes. p ≤.001,

^**p ≤.01,

^*p <.05.

Correlations for the FSP sample (N = 78) appear below the diagonal, correlations for the DFDS sample (N = 28) appear above the diagonal.

Individual-Level Analyses on Communication Change: Positive and Negative Change

Due to our concerns about entering both partners’ positive communication scores simultaneously into one model (whether it be multiple regression, logistic regression, or survival analysis), we proceeded by examining communication change by entering only one partner’s communication score at a time as a predictor. Doing so eliminates the need to interpret the effects of changes in one partner’s communication while controlling for changes in the other’s.⁵ We examined both negative and positive communication. Additionally, we used both distress development and relationship adjustment as outcomes and therefore present logistic regressions and then multiple regressions.

Logistic regression analyses

In neither sample did female or male positive communication change predict distress development when only one partner’s score was used as a predictor (see Table 4).⁶ Likewise, in the FSP sample, neither female nor male negative communication change predicted distress development when only one partner’s score was entered (see Table 5). However, in the DFDS sample, logistic regression analyses indicated a trend (p = .054) for increases in male negative communication PRE to POST to predict female distress development (see Table 5). For a one-unit increase in male negative communication change, the odds of being a distressed wife increase by a factor of 2.75. Here, the standard deviation for the male residualized change score is .96, so a one-unit change on the measure is essentially a one standard deviation change.

Table 4.

Summary of Logistic Regression Analyses for Changes in Positive Communication Variables Predicting Marital Distress Development for Wives and Husbands

	Wives			Husbands
Predictor	B	SE B	eB	B	SE B	eB
	The FSP sample
Model 1
Male positive increase	.08	.19	1.08	.01	.19	1.01
Constant	−1.42			−1.44
Model χ2	.17			.00
df	1			1
% (N) distressed or divorced	19.4(20)			19.2(20)
Model 2
Female positive increase	.06	.20	1.07	.02	.21	1.02
Constant	−1.42			−1.43
Model χ2	.10			.01
df	1			1
% (N) distressed or divorced	19.4(20)			19.2(20)
	The DFDS sample
Model 1
Male positive increase	−.40	.46	.69	−.21	.46	.81
Constant	−.66			−.81
Model χ2	.81			.21
df	1			1
% (N) distressed or divorced	34.5(10)			31.0(9)
Model 2
Female positive increase	.24	.52	1.27	.03	.53	1.03
Constant	−.65			−.80
Model χ2	.21			.00
df	1			1
% (N) distressed or divorced	34.5(10)			31.0(9)

Notes. e^B = exponentiated B. For the FSP sample, there were 103 wives and 104 husbands. In the DFDS sample, there were 29 wives and 29 husbands.

Table 5.

Summary of Logistic Regression Analyses for Changes in Negative Communication Variables Predicting Marital Distress Development for Wives and Husbands

	Wives			Husbands
Predictor	B	SE B	eB	B	SE B	eB
	The FSP sample
Model 1
Male negative increase	−.08	.27	.92	−.05	.26	.95
Constant	−1.41			−1.43
Model χ2	.09			.04
df	1			1
% (N) distressed or divorced	19.4(20)			19.2(20)
Model 2
Female negative increase	−.36	.29	.70	−.21	.28	.81
Constant	−1.43			−1.43
Model χ2	1.67			.61
df	1			1
% (N) distressed or divorced	19.4(20)			19.2(20)
	The DFDS sample
Model 1
Male negative increase	1.01†	.61	2.75	.68	.52	1.97
Constant	−.70			−.84
Model χ2	3.70†			1.90
df	1			1
% (N) distressed or divorced	34.5(10)			31.0(9)
Model 2
Female negative increase	.64	.72	1.89	.33	.73	1.39
Constant	−.66			−.81
Model χ2	.79			.21
df	1			1
% (N) distressed or divorced	34.5(10)			31.0(9)

^†Notes. p <.10,

e^B = exponentiated B. For the FSP sample, there were 104 wives and 103 husbands. In the DFDS sample, there were 29 wives and 29 husbands.

Multiple regressions for communication change

In neither study did changes in male or female positive communication significantly predict FU1 MAT scores (p’s >.10). However, in the FSP sample, increases in male negative communication were significantly predictive of lower FU1 MAT scores for husbands (F(2, 76) = 7.69, p <.05, adjusted R² = .15) but not wives. In the DFDS sample, the multiple regressions revealed that there were no significant associations between negative communication change and FU1 relationship adjustment (p’s >.10).

Discussion

The specific research question that we examined was whether changes in positive communication (during problem solving discussions) were associated with distress and divorce, as found by Schilling et al. (2003). The broader issues raised by this research have to do with two matters: statistical issues in dyadic analyses and preventive education with couples. On one hand, we agree with the clinical conclusions drawn by Schilling et al. (2003). Practitioners should make sure that participants of skills-based relationship education do not misinterpret messages about healthy interaction patterns. On the other hand, in direct tests for replication, we did not find evidence for the paradoxical finding of Schilling et al. in either of our two samples. Increases in female positive communication were not associated with increased risk of adverse marital outcomes. In fact, in one of the analyses most similar to Schilling et al.’s models, we found a trend in the opposite direction. In contrast to the absence of effects for positive communication changes, we found some overlap with the findings of Schilling et al. with regard to negative communication changes. There was modest evidence that decreases in men’s negative communication were associated with reductions in marital risk over time. Like Schilling et al.’s findings, changes in female negative communication were not associated with marital outcomes.

If the findings obtained here were to be broadly replicated, it would raise important questions about why increases in positive communication are unrelated to future marital outcomes and why negative communication changes are not more robustly related. One possibility is that the limits of this commonly used problem discussion paradigm are obscuring information about how communication behavior is related to marital outcomes. Likewise, ceiling effects with generally positive premarital or newlywed samples might make it hard to detect the importance of communication changes. It may also be the case that positives are just not as consequential as negatives (for a general discussion of positive vs. negative, see Baumeister, Bratslavsky, Finkenauer, & Vohs, 2001) or that they are differentially meaningful to men and women (e.g., Gottman, 1994; Notarius & Markman, 1993; Stanley, Markman, & Whitton, 2002).

Possible Explanations for Non-Replication

Based on our results here, as well as extensive exploration of how such statistical models work, it appears to us that reversals in the direction of effect of positive communication change could be an important artifact of statistical procedures rather than a substantive phenomenon. It is an artifact that highlights a difficulty faced by researchers who use dyadic data. Before addressing these methodological concerns, we briefly describe other factors that may have contributed to the differences between our findings and Schilling et al.’s.

Although both their study and this one examined outcomes following PREP training, there could have been important differences in ways PREP was delivered that account for differences in findings. Additionally, sample differences could account for the failure to replicate findings regarding positive communication. Their sample (N = 39 wives and 38 husbands) was drawn from one church, whereas both samples analyzed here resulted from community-wide recruiting efforts, with the newer FSP sample being recruited from 41 religious organizations. Further, Schilling et al.’s participants could choose between PREP or the church’s naturally occurring premarital services; here, in the FSP sample, the couples were merely told that PREP was what was available for those marrying. Additionally, our studies included more frequent assessment of couples. Further, the current investigation included a much larger sample (FSP) and much longer-term follow-up (DFDS).

Important methodological questions arose when we closely examined the patterns of coefficients in the logistic and multiple regression models that included communication scores for both partners. For example, how could there be a medium-sized and marginally significant association in the FSP sample between female increases in positive communication and female FU1 MAT scores (β = .29) when the zero-order correlation between these variables is virtually zero and non-significant (r = .07)? The fact, of course, is that the interpretation of multiple regression coefficients is in the context of holding constant all other variables in the model (Cohen & Cohen, 1983); zero-order correlations are not affected by other variables. Schilling et al. found the paradoxical finding in the context of controlling for many other variables simultaneously, including female PRE MAT scores, female PRE communication score, male partner’s PRE communication score, and male partner’s communication change score. The matter is all the more complex when the interaction between female and male change is also being added to the model, as they did because it was significant.

Because of all the other variables in the model, it is especially difficult to discern what one may conclude when female positive communication change is associated with later relationship outcomes. Part of what makes the interpretation of such models so difficult is that partners’ communication change scores are highly correlated (as would be expected; see Table 3; r = .77 in the FSP sample and r = .60 in the DFDS sample). Could multicollinearity cause regression coefficients to go in unexpected, paradoxical directions? Multicollinearity can cause reversals in directionality of associations, leading to difficulties in the interpretation of model coefficients (Montgomery & Peck, 1982). That is, there could be a positive association (or no association) between two variables when only Pearson correlations are examined and a paradoxical association between the same two variables when additional variables are simultaneously included. Schilling et al. did not present Pearson correlations, and we have been unable to learn what that association is at the simplest level, so we cannot comment on whether they could show that an increase in female positive communication change is associated with later distress when no other variables are included in the analysis. If it is not significant in a more parsimonious analysis, it is risky to interpret a coefficient in a highly complex analysis. We have never seen this result in any simple analysis of the underlying question in any sample.

In addition to multicollinearity, there are other possible explanations for differences in directionality between zero-order correlations and multiple regression coefficients. Non-detected influential observations can greatly affect patterns of results. Just one or two extreme observations can force a best-fit line in the opposite direction from what would be expected (Draper & Smith, 1981; Montgomery & Peck, 1982), especially in small samples such as Schilling et al.’s (Mendenhall & Sincich, 2003). Further, model misspecification based on the omission of important variables (Cohen & Cohen, 1983; Reichardt & Gollob, 1986), nonlinearity (Cohen & Cohen, 1983), and restriction of range (Draper & Smith, 1981; Montgomery & Peck, 1982) can affect directionality.

Beyond these statistical matters, we believe that the findings of Schilling et al. and our non-replication of those findings raise an important conceptual question for researchers using dyadic data, whether in outcome research or basic science research. Specifically, when do couple-level analyses poorly represent what is actually happening within couples? Calls have been made for more sophisticated and appropriate analysis of dyadic data (e.g., Kenny, 1998), and we strongly agree with the importance of doing such work. However, analyzing couple-level data is difficult to do well because partners’ scores, by nature, violate assumptions of independence. There is a tension between analyses that take into account the multiple dependencies and the inter-relatedness among partner’s scores and simpler analyses that have the advantages of parsimony. Schilling et al. attempted something too often avoided among couple researchers; they focused on the interplay of partners’ communication and couple outcomes.

We have spent hours talking with other researchers and statisticians about what the coefficient that represents change in female positive communication means if the change in her partner’s communication is controlled for. The very fact that we could spend so much time discussing such a question, without deriving simple answers of conceptual utility, suggests that the matter is extraordinarily complex. Including both partners scores as independent variables in a regression holds constant each in the interpretation of the other. Doing so, in effect, removes the variance that is likely most interesting to couple researchers in the first place: the partners’ behaviors that covary. In models like these, the coefficient for the female positive communication change represents the part of female change that is unrelated to male change. To us, it doesn’t make sense to examine only the part of someone’s communication that is unrelated to the communication of her partner when the goal is to change the communication between partners. One possible solution to these analytic problems could be to use ratings of dyadic rather than individual behavior. Another could be to compute a composite score that reflects the couple’s behavior. Additionally, there have been many statistical and conceptual advances in the couple research field that will help future research address questions about how partners influence one another, such as dyadic multilevel modeling and actor-partner interdependence models (see Atkins, 2005; Kenny, Kashy, & Cook, 2006).

For the present, we suggest that the direction of communication changes be interpreted with caution unless the simplest and most parsimonious analyses are significant and in same direction as more complex procedures. In the absence of simple associations indicating that increases in female positive communication are associated with later distress, we are inclined to doubt the conclusion that risk for distress increases when women communicate more positively. Nevertheless, it seems good practice to make sure that couples taking part in prevention programs properly understand what is and is not being recommended.

Even beyond these specific methodological issues, assessing mechanism of effects in primary prevention efforts is complex. In the relationship research area, prevention samples often contain many premarital or newlywed couples who are near the ceiling in terms of positive functioning on many variables, including the risk and protective factors being targeted. Thus, some couples may not make changes, or apparent and easily measured changes, immediately following prevention. Some would not need to make any changes. Conversely, the couples who are at greatest risk are the ones who can demonstrate the most change on targeted variables. Still, these couples who demonstrate change may not reach the same level of positive functioning as other couples, even following preventive intervention. If these different types of couples do exist in many prevention samples, complex patterns could be easily missed. For example, those who may make the greatest gains in the short run on targeted risk variables may also, for many reasons, continue to be couples at higher risk. Therefore, initial changes could appear to be associated with greater long-term risks, even when the intervention has been beneficial.

For a number of reasons, Halford et al. (2001) suggest focusing specific prevention efforts on specific types of risks. While we have been skeptical of the feasibility of this approach on a larger scale, the approach may lead to prevention samples in which participants are better identified in terms of specific, pre-existing risks. With a stronger focus on identifying risk factors, patterns of change and outcomes may be more interpretable.

As we discussed briefly in the results section, another issue that prevention researchers must contend with is whether to dichotomous outcome variables. Dichotomizing variables that are continuous in nature can limit the power to detect meaningful associations. Conceptually, it can also be quite difficult to determine the cut-offs for group membership. For example, two couples who are just a few points away from each other on the MAT (and less than even a ½ SD) could be classified into different groups. For some couples this distinction will be meaningful, in others it will not be. At the same time, dichotomization can ease interpretation, as Jacobson and Truax (1991) note, when it is based on meaningful or clinically significant criteria, it can be more useful than examining continuous outcomes. It is imperative that researchers carefully consider the implications of dichotomizing continuous variables for their interpretations.

There are a number of limitations to the research presented here. The strongest support for our assertion that Schilling et al.’s findings are not replicable in our sample would have come from finding robust associations between the key variables in the opposite direction. Thus, this research is limited to failing to reject the null hypothesis. Of course, failure to reject the null does not prove that the null hypothesis is correct. This conundrum is inherent in all research, and it is a particular challenge in replication research. Indeed, if non-significant tests of replication were routinely precluded from being interpreted as evidence of non-replication, anything that was ever significant in any study would be forever significant in our field. The best antidote to this fundamental dilemma is having large enough samples to have strong statistical power, along with multiple samples. Here, the FSP sample was large enough to provide power for the detection of small to medium effects (indeed, this sample was twice as large as Schilling et al.’s), but the DFDS sample had a small sample size that may have obscured effects. However, findings about negative communication that are consistent with the findings in the FSP sample as well as the negative communication findings of Schilling et al. mitigate this concern.

Missing data were a challenge in this investigation, as in most longitudinal research. Having missing data likely reduced statistical power because the sample size was limited. Further, it could be that couples who become distressed are more likely to miss assessments, thus distressed couples may have been underrepresented in this research.

These samples are also limited in terms of generalizability to minority couples, though the FSP sample had better representation than most studies on the effects of premarital intervention. Lastly, premarital couples are unique in that they are, on average, quite happy. Measuring relationship adjustment and distress only in the first few years of marriage may not capture what we really need to know.

In summary, we did not replicate the paradoxical findings of Schilling et al. We believe their findings and those presented here demonstrate some of the complex statistical issues that arise in understanding dyadic communication behavior. These complicated and important issues related to the analysis of couple data must be addressed by prevention and intervention researchers. Further, the findings presented here point out the difficulties in measuring mechanisms of change in prevention research with couples. It may be that knowledge of mechanisms of change in couple therapy (see Doss, Thum, Sevier, Atkins, & Christensen, 2005) needs to be more carefully thought through before it is applied to prevention efforts. Perhaps maintenance of positive and constructive relationship behaviors over time is a more salient short-term effect of relationship education with premarital couples.