Abstract
Expressed emotion measures, encompassing dimensions of criticism (CRIT), and emotional overinvolvement (EOI) are increasingly being used to assess the parent–child relationship in child clinical populations, despite the lack of studies assessing their validity. We examined the correspondence between CRIT, EOI, and parent–child interactions as observed by neutral coders in a sample of 252 clinic-referred children and adolescents, ages 7–17 years. We found support for the validity of the CRIT code, with high critical parents showing more antagonism, negativity, disgust, harshness, and less responsiveness, compared to parents who scored in the low or borderline ranges. In contrast, none of the observed behaviors were found to correspond with parental EOI, suggesting either that this construct lacks validity with juvenile samples or that behaviors that correspond to EOI are difficult to observe. We conclude that high parental CRIT can be used as an index of problematic parent–child interactions.
Keywords: expressed emotion, parent-child interaction, criticism, emotional overinvolvement
Evidence is accumulating that some childhood disorders, such as depression, are associated with low levels of family support (Barrera & Garrison-Jones, 1992; Feldman, Rubenstein, & Rubin, 1988), high family conflict (Forehand et al., 1988; Wierson, Forehand, & McCombs, 1988), or poor family relations (Puig-Antich et al., 1993). Expressed emotion (EE) is one measure that has been used to assess the parent–child relationship among adults, and more recently children, with psychological disorders. Having a caregiver score high on the EE measure has been associated with poorer outcome for adults with mood, anxiety, and schizophrenic disorders, and with the presence of internalizing and externalizing disorders for children (Asarnow, Tompson, Woo, & Cantwell, 2001; Butzlaff & Hooley. 1998; Chambless & Steketee, 1999; Stubbe, Zahner, Goldstein, & Leckman, 1993; Vostanis, Nicholls, & Harrington, 1994). The assumption underlying EE is that the way parents talk about a child is indicative of the way they treat the child on a day-to-day basis (Chambless, Bryan, Aiken, Steketee, & Hooley, 1999). Given the suggested association between parent–child interactions and these disorders and given the increasing use of EE as an index of dyadic relationship problems within the family, it is important to know whether the expressed emotion measure actually corresponds to such interactions.
EE is assessed via interview or an open-ended query posed to the parent or caregiver. Commonly, EE is measured using the Five-Minute Speech Sample (Magana et al., 1986) wherein caregivers are asked to describe their child and their relationship with him/her and responses are coded for two different components of EE-criticism (CRIT) and emotional overinvolvement (EOI), with a high score on either indicating a “High EE profile.” Despite this convention, researchers have recently favored dismantling the overarching EE construct into its separate CRIT and EOI components, particularly in child samples (Hirshfeld, Biederman, Brody, & Faraone, 1997; McCarty & Weisz, 2002; Stubbe et al., 1993; Wamboldt, O’Connor, Wamboldt, Gavin, & Klinnert, 2000). Because the two components of EE are intended to measure distinct parental attitudes, they should, in theory, be associated with different behaviors. However, the validity of the EOI scoring system and the cohesiveness of the measure among parents of children and adolescents have recently been questioned, which might suggest that EOI may not be related to parent–child interactions at all. The measurement of EOI using the traditional, adult-based EE scoring system has been suggested to be problematic in that it attempts to measure problematic parenting (“overinvolvement”), and yet it is scored from some criteria that may actually be developmentally appropriate when referring to juvenile children (Daley, Sonuga-Barke, & Thompson, 2003; McCarty & Weisz, 2002; Wamboldt et al., 2000). The measure also has not demonstrated cohesiveness as a measure, in terms of its internal consistency and the way that the individual criteria relate to child psychopathology (McCarty & Weisz, 2002). To increase our understanding of the appropriateness of applying EOI to child clinical samples, investigation of its concurrent validity with behavioral data is warranted.
Thus far, the research relating observed interactions to EE has mostly focused on families of adults with or at risk, for schizophrenia-spectrum disorders (Miklowitz, Goldstein, Falloon, & Doane, 1984; Strachan, Goldstein, & Miklowitz, 1986; Valone, Norton, Goldstein, & Doane, 1983;) bipolar disorder (Simoneau, Miklowitz, & Saleem, 1998), or depression (Mundt, Fiedler, Ernst, & Backenstrab, 1996) and has assessed interactions between relatives as a function of overall EE in general, not differentiating CRIT from EOI The studies that have assessed the two EE components separately suggest that relatives high on the CRIT dimension express a significantly greater number of criticisms during interactions than relatives who score low on this dimension, and show extreme negative escalation patterns (Hahlweg et al., 1989). Parental CRIT has also correlated positively with “belittling and blaming” statements made by parents, and negatively with “disclosing and expressing” and “trusting and relying” (Hubschmid & Zemp, 1989). The way that EOI relates to interactions between parents/relatives and patients has generally been less consistent across the literature compared to CRIT (Miklowitz et al., 1984; Mueser et al., 1993). Some studies have found that high EOI parents make more intrusive statements (Strachan et al., 1986) or more ambiguous and unclear statements (Hubschmid & Zemp, 1989); others have found that they do not differ in behavioral interactions compared to low EOI parents (Hahlweg et al., 1989).
Although these studies provide some evidence for the validity of the separate EE components (particularly CRIT) for parents of adult children, the comparable studies examining behavioral correlates for parents of juveniles either (a) have not examined CRIT and EOI separately or (b) have been conducted on nonclinical samples. In a study of boys with Attention Deficit Hyperactivity Disorder, parents’ overall EE toward their child was a good predictor of parents’ verbal coercive processes and their negative affective style in interactions (Marshall, Longwell, Goldstein, & Swanson, 1990), but differences between high CRIT and EOI parents were not reported. A study examining the behavioral correlates of EE among families of clinic-referred adolescents found that low EE dyads were less likely to escalate negative interactions compared to high EE dyads (Cook, Strachan, Goldstein, & Miklowitz, 1989); but in this study, too, CRIT and EOI were not analyzed separately.
Two studies with asthmatic youth have assessed parent–child interactions in relation to CRIT and EOI separately, and relatively good evidence has been found to support the validity of CRIT, with weaker evidence for EOI. In the first, high CRIT mothers were found to give more criticisms and have more frequent sequences of negative verbal interaction with their asthmatic children (Hermanns, Florin, Dietrich, Rieger, & Hahlweg, 1989). A second study involving asthmatic children and adolescents found high parental CRIT was related to parent negative affect, poorer parent and child problem-solving skills, and poorer attunement to the other person. In this study, high EOI was related to poorer adolescent boundaries, but no differences emerged in parental interactions with children as a function of EOI (Wamboldt et al., 2000)
Given the limitations of prior studies, it remains to be shown whether reliable behavioral correlates of parental CRIT and EOI can be identified among clinic-referred children/adolescents and their parents. For example, do parents who express criticism toward their child during the speech sample when alone with an interviewer actually behave differently when interacting with their children? It is possible that some parents may be critical in a tape-recorded statement made about their child in the child’s absence, yet still appear conflict avoidant in face-to-face interaction with the child (Wamboldt, Wamboldt, Gavin, Roesler, & Brugman, 1995). Moreover, the developmental validity issues raised around EOI beg the question of whether or not this measure is a proxy for a particular style of parenting when referring to juvenile, not adult, children.
It is important to consider both parental and child behavior in relation to EE. A host of theories, including Sameroff’s transactional theory (Sameroff, 1975), Patterson’s theory of coercive family processes (Patterson, 1982), and Thornberry’s interactional theory (Thornberry, 1987), suggest that the behaviors of parents and children influence each other and are reciprocally associated. The literature shows strong evidence that child behavior can be a very powerful determinant of how a mother responds to a child (Anderson, Lytton, & Romney, 1986; Sanders, Dadds, & Bor, 1989; Williams & Forehand, 1984). Children with behavior problems that are difficult to tolerate may cause parents to develop critical attitudes toward them. In other words, high CRIT parents may be reacting to genuinely problematic child behaviors that are causing them distress. Researchers who study externalizing problems among children have suggested that mothers’ irritability and depression may be partly a reaction to their failure in parental control and the chronic level of coercive behaviors they experience from their children (Hetherington & Martin, 1986; Patterson, 1982). Experimental studies confirm that deviant child behavior is causally related to parental stress and negative mood (Pelham et al., 1987, 1988). Thus, it is possible that parental critical attitudes develop over time after parents experience difficulty in managing their children’s aversive behavior. Alternatively, parents who are overly negative and critical might instigate oppositional and aggressive behavior in their children. According to coercion theory, (Patterson, 1982), children exposed to aggressive interactions with their parents are at increased risk for aggression themselves because irritable exchanges within the family are thought to provide a basic training for aggression that generalizes to other settings. Thus, children may act out in response to high CRIT parents. Overall, we suspect that any association between parental attitudes and child behavior most certainly is transactional in nature, and the direction of effect is likely bidirectional. The current study examines associations between the two components of EE and both observed parent and observed child behaviors during interactions.
In summary, we set out to determine whether parental attitudes captured by CRIT and EOI measures correspond to parent or children’s behavior in structured interactions together—assessing for a wide variety of behaviors and utilizing a large clinic-referred sample. We chose to use structured, clinic-based observation during family problem-solving discussions so that we could structure the nature of the tasks, with a trend toward increasing conflict, as high-conflict tasks generally elicit greater differences between distressed and nondistressed groups (Christensen & Margolin, 1988; Gottman, 1979).
METHOD
Participants
Participants were 252 youth (164 boys and 88 girls) and their primary caregivers who had been referred to one of nine outpatient community mental health clinics in central and southern California participating in a larger study of clinic-based care. Child ages ranged from 7 to 17 years (M = 11.29, SD = 2.55). The sample included 49.0% Caucasian, 14.5% African American, 15.8% Hispanic/Latino, 2.1% Asian American, and 18.6% multiethnic children. Some 40.2% of the sample reported annual family incomes below $15,000 per year, 29.3% between $15,000 and $30,000, and the remaining 30.5% above $45,000. Ninety percent of the youth were interviewed with a female primary caregiver (80% biological mother, 1% stepmother, 6% other female relative, 2% other female nonrelative, 1% adoptive mother). The other youth were interviewed with their biological fathers or father figures. About 75% of the mothers had received a high school diploma. Marital status for the primary caregivers were as follows: 40% married, 30% divorced, 9% separated, 4% widowed, 7% living with partner, and 10% never married and not living with partner.
This clinical sample of youth evidenced high levels of psychopathology. Reports on the Child Behavior Checklist (see later) placed children at the 95th percentile for externalizing problems and at the 93rd percentile for internalizing problems. The mean number of diagnoses in the sample, according to parent or child report on the Diagnostic Interview Schedule for Children, was 2.68 (SD = 2.19). Rates for specific categories of diagnosis are provided in Table I.
Table I.
Diagnostic category | informant | Percent Meeting criteria |
---|---|---|
Depressive disorder(s) | Parent or Child | 33.3 (n =77/231) |
Major depressive disorder | Parent or Child | 24.2 (n = 56/231) |
Dysthymic disorder | Parent or Child | 26.0 (n = 60/231) |
Anxiety disorder(s) | Parent | 37.6 (n = 111/231) |
Agoraphobia | Parent | 4.5 (n = 11/242) |
Avoidant disorder | Parent | 5 8(n =14/242) |
Obsessive–compulsive disorder | Parent | 1.7 (n =4/242) |
Overanxious disorder | Parent | 15.3 (n = 37/242) |
Panic disorder | Parent | 0.0 (n = 0/242) |
Separation anxiety disorder | Parent | 12.9 (n = 31/241) |
Simple phobia | Parent | 17.8 (n = 43/242) |
Disruptive behavior disorder(s) | Parent or Child | 52.1 (n = 122/234) |
Conduct disorder | Parent or Child | 23.5 (n =53/226) |
Oppositional defiant disorder | Parent | 45.4 (n = 109/240) |
Attention deficit hyperactivity disorder | Parent | 37.0 (n = 88/238) |
Note. DISC data for individual diagnoses were missing for 5.9–10.3 of the sample.
Procedures and Measures
Families who agreed to participate were interviewed near the time of their intake at the clinic. The interview involved multiple measures, with child measures and parent measures administered by different interviewers in separate rooms. Parents and children were brought together for three interaction tasks (described later) about midway through their individual interviews.
The Five-Minute Speech Sample (Magana et al., 1986)
The FMSS requires the parent to speak for 5 minutes into a tape recorder without interruption about his/her child and how they get along together. Audiotapes were scored by the original developer of the EE measure (A. Magana), who was blind to all other information about the parents and youth. For both CRIT and EOI, parents were rated as high, borderline, or low, using the standard EE-scoring approach. A FMSS was scored as high on the CRIT dimension if any of the following criteria was met: negative initial statement, negative relationship rating, or one or more criticisms were present. A negative relationship rating was scored when parents indicated that they and their child do not get along together or are unable to communicate. Parents were scored as borderline CRIT if they expressed dissatisfaction with their child, but it was not extreme enough to be rated as a criticism (Magana et al., 1986). A high EOI rating was assigned if the parent expressed self-sacrificing or overprotective behavior, displayed excessive emotion (crying), or provided five or more positive remarks about the child combined with a statement of affection or excessive detail about the child’s past. Excessive detail was scored when the parent gave an inordinate amount of extraneous information about the child’s distant past (e.g., describing them as an infant). Parents were rated as borderline-high EOI if they exhibited one of the following during the speech sample: moderate levels of self-sacrificing or overprotective behavior, statements of love, excessive detail, or more than five positive remarks about the child. A subset of 47 random tapes were double-coded by the first author to check for reliability. The weighted kappa statistic, a generalization of the simple kappa coefficient that uses weights to quantify ratio-scaled degrees of disagreement, was calculated. It is the recommended coefficient of agreement for ordered category data (Cohen. 1968). Weighted kappas between the two raters were .72 for EOI and .68 for CRIT.
Child Behavior Checklist
Parents reported on child behavior problems over the 6 months prior to the interview using the Child Behavior Checklist (CBCL; Achenbach, 1991). The CBCL is a reliable measure with alphas for the internalizing problems (including social withdrawal, somatic complaints, and anxious/depressed behaviors) and externalizing problems (including aggressive and delinquent behaviors) of 0.89, arid 0.93, respectively (Achenbach, 1991). Test-retest reliabilities obtained from mothers’ ratings of non-referred children (aged 4–16) over a 1 -week period were 0.89 for internalizing problems, and 0.93 for externalizing problems (Achenbach, 1991).
Diagnostic Interview Schedule for Children
The Diagnostic interview Schedule for Children (DISC-2.3; National institute of Mental Health [NIMH], 1991) was administered to parents and children to obtain information about diagnostic status for the sample of youth. Parents received the full DISC, with children receiving the sections on mood disorders, conduct disorder, and substance use disorders. The DISC has been used extensively in both clinical and academic settings to assess youth psychopathology. Adequate validity, interrater reliability, and test-retest reliability of the measure have been documented in several investigations (Fisher et al., 1993; Schwab-Stone et al., 1996; Shaffer; Fischer, Dulcan. & Davies, 1996). Data collection spanned the last years of DSM-III-R and some years of DSM-IV. For consistency, we used DSM-III-R diagnoses throughout the study. Studies of agreement across versions suggest that most DSM-III-R diagnoses are reasonably generalizable to DSM-IV (e.g., Biederman, Faraone, Weber, Rater, & Park, 1997; Kendall & Warman, 1996).
ChiId’s Report of Parental Behavior Inventory
Parent and youth reports on various parenting sub-scales of the Child’s Report of Parental Behavior Inventory (CRPBI; Schaefer, 1965; Schludermann & Schludermann, 1970) were used as validity checks for the observational codes. The CRPBI has been found to have good internal consistency (Schwartz, Barton-Henry, & Pruzinsky, 1985), replicable factor structure (Schludermann & Schludermann, 1970), and cross-ethnic validity (e.g., Knight, Tein, Shell, & Roosa, 1992). Note that the child and parent versions of the CRPBI were only administered to a subset of this study sample (n = 121 child, n = 183 parent), as these measures were dropped from the assessment battery after the first year (beginning with the child version) in order to condense the interview time. The following subscales were used: Acceptance, Acceptance of Individuation, Control, Control Through Guilt, Detachment, Hostile Control, Hostile Detachment, Instilling Persistent Anxiety, Intrusiveness, Rejection, and Withdrawal of Relations.
Interaction Tasks
Three interaction tasks were arranged for parent–child dyads to participate in together: (1) a teaching task, (2) a planning task, and (3) a conflict task. The order of tasks was not counterbalanced because the emotionally arousing nature of the conflict task seemed likely to influence the behavior of parents and children during the other two tasks.
In the teaching task, the parent was asked to teach the child “how to plan and shop for the groceries needed to prepare a healthy dinner for a family of six, people” The parent was given index cards with pictures of various foods, and asked to use the pictures in teaching the child. The teaching task lasted 3 min. In the planning task, participants were given 4 min to plan a 2-day vacation, including (a) where they would go, (b) how they would get there, (c) where they would stay overnight, and (d) what they would do during the day. This task was an adaptation of the Family Interaction Task (Grotevant & Cooper, 1985) and was designed to elicit the expression and coordination of different viewpoints on a topic to which the child could contribute. The conflict discussion task was a variation of Strodbeck’s Revealed Differences Technique (Strodbeck, 1957). Prior to the task, the parent and child were individually asked to rate how much they disagreed about different topics. The interviewer then identified the topic that maximized the conflict ratings by both while minimizing the discrepancy between the two, and the parent and child were given 6 min to try to resolve this mutually identified conflict.
Rating Systems for Parent-ChiId Interactions
The coding and rating systems developed for this study were designed to include behavioral analogs of CRIT and EOI, as well as to capture more general indices of parent and child behavior toward each other. The Affective Style Scoring Manual (Doane, West, Goldstein, Rodnick, & Jones, 1981), which includes codes that parallel the EE measures but was created for interactions between adult patients and their caregivers, was examined to help create codes and ratings that would be relevant to observing CRIT and EOI behaviors relevant for interactions with parents and their nonadult children.
Both microanalytic codes7 and global ratings were created to reflect parental CRIT and EOI. The global ratings that were created to reflect the EOI dimensions were intrusive control and fostering independence, with the latter expected to relate negatively to EOI. Affective punishment, a microanalytic code, was created to include the concept of guilt induction described in the Affective Style manual. Three global ratings (antagonism, negativity, and disgust) were created to provide behavioral indices of CRIT The antagonism rating was based upon the parent expressing feelings of anger, resentment, and ill will toward the child, as well as physical and verbal displays of aggression. Negativity included general irritability or antagonism. Disgust was defined to be distancing behavior, including mocking and/or insulting the child.
Coding Process
The parent–child interactions were transcribed verbatim by research assistants using both audiotapes and videotapes as needed. Nonverbal behaviors that appeared to have interpersonal significance were also written into the transcript for coding. Each transcript was checked and edited by a second transcriber to ensure accuracy of the transcript. We trained a group of undergraduate students and research volunteers (n = 13) in applying the coding and rating criteria to these videotaped parent–child interactions. After participating in a 50-hr training program, coding and rating 15 pilot tapes, and achieving reliability of at least 70% agreement with the criterion ratings, observers independently coded the videotaped interactions. Observers were instructed to view the videotape once before coding, and then code the videotape, with the order of viewed tasks being randomly selected for the coders. Observers were informed that they would be spot-checked for reliability on several randomly selected tapes throughout the coding process in order to minimize observer drift and to obtain accurate measures of reliability.
Global Rating System (McCarty, Lau, & Valeri, 2001)
Global ratings for parent and child behavior were completed by the observers after viewing the entire interaction sequence, using 7-point Likert scales (where 1 = not at all, and 7 = very much). Fourteen global dimensions of behavior were rated for both parents and children, and two additional dimensions were rated for parents only (“Authoritarian” and “Fostering Independence”). All interactions were rated independently by two separate observers, and the mean rating across the observers was used, both at the item level and in the creation of composite variables.
Global composites were created after running factor analyses for the positive global ratings, and the negative global ratings separately, for both parent and child. We constructed composites by taking the sums of individual ratings, based on both the factor analyses and considerations of interrater and internal consistency reliability. Three parent and three parallel child composites (harsh, enthusiastic, responsive) were used in analyses, with intraclass correlations (ICCs) ranging from .60 to .78, as shown in Table II. Although reliability estimates were on the low end for EOI-specific codes as a result of coder disagreement and lower base rates for some of these behaviors, these codes showed good concurrent validity, as evidenced by moderate correlations in the expected directions with relevant parenting scales on the Child’s Report of Parental Behavior Inventory, parent and child reports (Schaefer, 1965; see Table III).
Table II.
Behavioral code | Parent | Child |
---|---|---|
CRIT-specific codes | ||
Antagonism (G) | 0.67 | |
Negativity (G) | 0.48 | |
Disgust (G) | 0.62 | |
EOI-specific Codes | ||
Affective punishment (M) | 0.32 | |
Intrusive control (G) | 0.30 | |
Fostering independence (G) | 0.47 | |
Global rating composites (n = 333) | ||
Harsh | 0.60 | 0.75 |
Enthusiastic | 0.63 | 0.71 |
Responsive | 0.62 | 0.78 |
Note. (G) = Global rating, (M) = Microanalytic code.
Table III.
Affective punishment | Intrusive control | Fostering independence | |
---|---|---|---|
Parent report (ns = 142–183) | |||
Acceptance of individuation | −.16* | .01 | .19** |
Control | .00 | .20** | −.29** |
Control through guilt | .09 | .22** | −.26** |
Detachment | .18 | .27** | −.35** |
Hostile control | .16* | .32** | −.39** |
Instilling persistent anxiety | .22** | .29** | −.37** |
Intrusiveness | .03 | .27** | −.29** |
Withdrawal of relations | .05 | .23** | −.34** |
Child report (ns = 96–121) | |||
Acceptance | −.23 | −.07 | .28** |
Acceptance of individuation | −.15 | −.14 | .19* |
Hostile control | .02 | .09 | −.23** |
Hostile detachment | .09 | .15 | −.44** |
Rejection | .13 | .13 | −.36** |
Withdrawal of relations | .06 | .14 | −.39** |
p < .05,
p < .01.
RESULTS
Because EE is assumed to provide some indication of how parents act toward their child, parent and child behaviors during the interaction tasks were compared for children of high, borderline, and low EE parents, with the CRIT and EOI dimensions analyzed separately. First, we assessed the specific codes and ratings created to index parental CRIT and EOI behaviors, to test the validity of EE as an index of parents’ interactions with their children, using separate ANOVAs for each code. We were not as concerned about controlling for Type I error in analyses of EE-specific behaviors, because these codes were theoretically related to CRIT and EOI. Then, we ran analyses on the broader range of global composites to assess other parent behaviors and child behaviors that might be related to CRIT and EOI, but were not designed to be specific indicators of them (i.e., harsh, enthusiastic, responsive). For these analyses, we used a more conservative (MANOVA) approach, as this group of dependent variables was not theoretically linked to CRIT and EOI, and thus we sought more control for Type I error. We used a categorical MANOVA approach because the scoring systems for CRIT and EOI are categorical and only lead to three levels. Post hoc differences between pairs of means for the three CRIT/EOI subgroups were conducted using Scheffe tests based on univariate tests. All analyses were conducted while controlling for child age, child gender, and level of maternal education. This was accomplished by regressing these three covariates on each of our observational measures, and using the unstandardized residuals as dependent variables in our subsequent analyses of variance.
What Behaviors Are Related to Parental Criticism?
In order to understand whether scores on the CRIT dimension of EE predict actual behavior in interactions, we examined the CRIT-specfic observational behaviors and the more general global composites as a function of parental CRIT grouping. The first set of analyses were one-way ANOVAs comparing the CRIT-specific behavior ratings for parents who scored high, borderline, or low on the CRIT dimension. We found significant differences on each of the three CRIT-specific ratings, including parental antagonism, F(2, 228) = 6.24, p = .002, parental negativity, F(2. 228) = 6.97, p < .001, and parental disgust, F(2, 228) = 430, p = .02. Parents who scored high on CRIT were rated higher on each of these behaviors compared to parents who scored low on CRIT; Significance levels, group means, and post hoc contrasts are reported in Table IV.
Table IV.
High | Borderline | Low | Direction | p Value | |
---|---|---|---|---|---|
CRIT-specific behaviors | |||||
Parent antagonism | 2.78 | 1.95 | 2.11 | H > L | .01 |
H > B | .01 | ||||
Parent negativity | 3.30 | 2.70 | 2.60 | H > L | .002 |
Parent disgust | 2.83 | 2.22 | 2.29 | H > L | .04 |
Parent global composites | |||||
Parent harsh | 3.29 | 2.65 | 2.67 | H > L | .002 |
H > B | .02 | ||||
Parent responsive | 3.18 | 3.33 | 3.58 | L > H | .01 |
Note. H = High CRIT group, B = Borderline CRIT group, L = Low CRIT group. Values reflect means for each group (after covarying child age, child gender, and maternal education level). Means are based on 7-point Likert scale, ranging from 1 (not at all) to 7 (very much).
Next we examined differences on other, non-CRIT-specific behaviors using separate MANOVAs for parent and child behaviors. Significant differences emerged between the CRIT groups on the parental global composites, F(8, 450) = 2.67, p = .001. Univariate tests suggested differences in parental harshness F(2, 228) = 7.58, p = .001 and responsiveness F(2, 228) = 4.80, p = .009. Using Scheffe post hoc tests, we found that parents who scored low or borderline on CRIT were significantly less harsh during the interactions with their children, compared to parents who scored high on CRIT. Parents who scored low on the CRIT EE dimension also were rated to be more responsive with their children, compared to parents who scored high on CRIT. Differences in parental behaviors for the three groups of CRIT parents are shown in Table IV. Children’s behavior during the interactions was nor significantly different on the global composites, F(8, 450) = 1.46, p = .17, as a function of parental CRIT.
What Behaviors are Related to Parental EOI?
In order to understand whether EOI scores were associated with distinctive parent or child behavior, the behavioral codes and ratings from the interaction tasks were examined with respect to parental EOI status. To parallel the analyses conducted with parental CRIT, ANOVAs were run in which the frequencies of the EOI-specific observational behaviors were tested for differences according to parental EOI grouping. MANOVAs were run for the global composites of behavior for parents and children.
Results revealed that high EOI parents did not differ from low or borderline parents on any of the three EOI-specific behavior codes, which included intrusive control, F(2, 228) = 0.07, p = .94; fostering independence; F(2, 228) = 0.85, p = .43; and affective punishment, F(2, 228) = 0.70, p = .50. Moreover, no differences were found among EOI subgroups in parent global composites of behavior, F(8, 450) = 0.31. p = .96. In terms of children’s behavior as a function of parental EOI, no differences emerged for the child global rating composites, F(8, 450) = 1.13, p = .34. Thus, neither parent nor child behavior during structured interactions was observed to differ as a function of EOI status.
DISCUSSION
The EE measure has been used extensively in studies of adult psychopathology, and is a fairly robust predictor of relapse for schizophrenia, depression, and other psychiatric problems in adults (Butzlaff & Hooley, 1998). Although it has been extended to use with, children, little data on its validity with juvenile populations exists. This study investigated the behavioral correlates of parental EE in a sample of clinic-referred youth, to examine what this measure might reveal about family interactions. One of the most important findings is that parental critical attitudes, measured by EE, do manifest themselves at a behavioral level. Parents who score high on CRIT EE were more antagonistic, negative, and disgusted in interacting with their child. They also were observed, to be more harsh in the interactions, although the empirically derived harsh composite was somewhat redundant with the CRIT-specific codes, as it was comprised of the following individual codes: antagonism, negativity, intrusive control, authoritarian parenting, and disgust. High critical parents were rated lower on responsiveness compared to noncritical parents. In all cases, it was the parents who scored “high” on CRIT who could be differentiated from parents who scored in the low or borderline ranges. Parental CRIT was not associated with aspects of children’s behavior toward their parents.
The meaning of EOI is less clear, given the lack of significant relationships found in the current study. EOI was not linked to observable behaviors, and at least two explanations must be considered for this null finding: (a) EOI-EE does not correspond very closely to observable behavior, or (b) it is difficult to find clearly and reliably observable behaviors that reflect EOI. The first explanation suggests that EOI is not a proxy for any parental behaviors. This result is consistent with the lack of association found between EOI and parental affect and behavior with asthmatic children and adolescents in Wamboldt and colleagues’ study (Wamboldt et al., 2000). The combined effects of that study and the current study may suggest a lack of concurrent validity for EOI, as measured by the Five-Minute Speech Sample; or more fundamentally, a lack of construct validity for EOI with juvenile populations. The EOI measure comprises varied parental attitudes, as it is coded from such diverse criteria as parents saying that they love their child, giving detail about the child’s past (i.e., infancy), and making five or more positive remarks about their child when asked to describe them. It is arguable that these statements may be developmentally appropriate or even adaptive when made by parents of youngsters. Our data suggest that parents who express such attitudes do not differ in their behavior with their child along the dimensions we measured from those who do not express such attitudes, and it is possible that the label “emotional, over-involvement” is a misnomer for this particular measure when applied to parent-youth relations.
An alternate explanation that merits contemplation is that the types of behaviors that would reflect EOI are difficult to observe. Although we attempted to create codes and ratings that would parallel the current interpretation of parents who score high on the EOI measure as being intrusive, guilt-inducing, and fostering dependence, there may be aspects of being overinvolved that are difficult to observe, particularly during structured interactions. Indeed, in our sample, the base rates for the affective punishment code were quite low, and our coders, although well trained, had more difficulty in agreeing upon the EOI-specific codes. This methodological limitation makes interpretation of these null findings for EOI more convoluted, particularly because diminished reliability on these codes limits our ability to draw strong conclusions about validity. Despite low interrater agreement, we did find modest relations in the expected directions between the EOI observational codes and the CRPBI (particularly the parent-report version). It is recommended that replication and extension of these findings be produced to and in weighing the relative plausibility of each of these explanations.
These results converge with the findings of other researchers (e.g., Stubbe et al., 1993; Chambless et al., 1999) in suggesting that CRIT be used separately from EOI, rather than collapsed into the broader measure of EE as most studies have done. As Chambless and colleagues state in their discussion (1999), “Combining separate and unrelated constructs into a single variable before such a composite variable has been demonstrated to be warranted in a given sample risks muddying the results of prediction analyses” (p. 74). Because the current study found CRIT to be related to many important variables in parent–child interactions, and EOI to be essentially unrelated to parent and child behavior in interactions, it appears that combining CRIT and EOI may risk diminishing or obscuring important relationships.
The current study cannot determine the causal direction of the relationships between parental critical EE and parent and child interactional behavior, because they were both assessed concurrently. What is clear is that there is a reliable and consistent relationship among these variables, and that high levels of parental criticism on the EE measure are indicative of problematic parent–child interactions. Previous research has shown that maternal and child diagnoses, as well as some self-report parent measures of hostility, negativism, and family conflict are predictive of high maternal EE (Hibbs et al., 1991; Hibbs, Hamburger, Kruesi, & Lenane, 1993). By contrast, low EE has been related to the absence of psychopathology and to a more functional family environment (Hibbs et al., 1993). The current study adds to that literature by examining behavioral correlates, as rated by independent observers, of parental CRIT and EOI separately. In particular, the study suggests that parental criticism, as measured by the Five-Minute Speech Sample, is a useful index of relatively dysfunctional interactions between parents and children. Scoring “high” on CRIT is an indicator of more distressed parent–child relations, in contrast to those who score “low” or “borderline.” From a broader perspective, findings of the study make two general contributions to the literature on EE and child and parent functioning. First, the findings illustrate the importance of dismantling the composite EE construct into its components and assessing how each relates to family relations and child psychopathology. And second, the findings underscore the value of linking direct observation of parent–child interactions to assessment of parental altitudes. Although the present findings do not provide definitive answers to all the important questions in this complex field, they do suggest intriguing hypotheses that may profitably be explored in future research.
Acknowledgments
The study was supported by a National Research Service Award F31 MHI1845 (to Carolyn McCarty), and by National Instituie of Mental Health research grants R01 MH49522, R01 MH57347, and K05 MHO1161 (to John Weisz), which we gratefully acknowledge. We are appreciative to the staff of the participating clinics for their supportive involvement, and to Mandy Jensen, Vanessa Robin Weersing, Kristin Hawley, Lynne Sweeney, Susan Han, and Christopher Thurber, for their careful management of the data set.
Appendix
Composite Constituent ratings | Brief description |
---|---|
Parent harsh | |
Parent antagonism | Feelings of anger, resentment, and ill will toward child |
Parent negativity | General irritability not necessarily directed toward child |
Parent intrusive control | Degree to which parent tries to exert influence over child in a manipulative way |
Parent authoritarian | Direction of the child’s behavior through the assertion of power (autocratic and unilateral) |
Parent disgust | Parent is cold toward child and puts distance between them |
Parent enthusiastic | |
Parent happy | Parent displays happiness through facial expression, tone of voice, or body language |
Parent interest | Increased energy that is positive in valence, engagement in activity with child |
Parent warmth | Warmth, affection, and love the parent gives to the child, expressed both physically and verbally |
Parent responsive | |
Parent fostering independence | Respecting the child’s opinions and encouraging expression of them |
Parent sensitive | Attentiveness and attunement in interacting with child |
Parent fair | Treated child justly; had reasonable requests |
Parent willing to bend | Willing to compromise, flexible about approaching tasks |
Parent understood child | Empathic, communicated appreciation for child’s perspective |
Child harsh | |
Child antagonism | Feelings of anger, resentment, and ill will toward parent |
Child negativity | General irritability not necessarily directed toward parent |
Child intrusive control | Degree to which child tries to exert influence over parent in a manipulative way |
Child disgust | Child is cold toward parent and puts distance between them |
Child enthusiastic | |
Child happy | Child displays happiness through facial expression, tone of voice, or body language |
Child interest | Increased energy that is positive in valence, engagement in activity with parent |
Child warmth | Warmth, affection, and love the child gives to the parent, expressed both physically and verbally |
Child responsiveness | |
Child sensitive | Attentiveness and attunement in interacting with parent |
Child fair | Treated parent justly; had reasonable requests |
Child willing to bend | Willing to compromise, flexible about approaching tasks |
Child understood parent | Empathic, communicated appreciation for parent’s perspective |
Note. Global ratings were intended to capture the general, over-arching behaviors and affects communicated by parents and children during the entire interaction (all three tasks), and were made after viewing interactions in their entirety.
Footnotes
Aside from the use of affective punishment as an indicator of EOI behavior, none of the other microanalytic codes (content or affect-based)are used in this paper. A number of the microanalytic codes were difficult to code reliabily, and analyses that were conducted did not indicate meaningful group differences. More information about this coding system can be obtained by contacting the first author.
References
- Achenbach, T.M. (1991). Manual for the Child Behavior Cheklisti 4-18 and 1991 Profile. Burlington: University of Vermont, Department of Psychiatry.
- Anderson KE, Lytton J, Romney DM. Mothers’ interactions with normal and conduct-disordered boys; Who affects whom? Developmental Psychology. 1986;22:415–419. [Google Scholar]
- Asarnow JR, Tompson M, Woo S, Cantwell DP. Is expressed emotion a specific risk factor for depression or a non-specific correlate of psychopathology? Journal of Abnormal Child Psychology. 2001;29:573–583. doi: 10.1023/a:1012237411007. [DOI] [PubMed] [Google Scholar]
- Barrera M, Garrison-Jones C. Family and peer social support as specific correlates of adolescent depressive symptoms. Journal of Abnormal Child Psychology. 1992;20:1–46. doi: 10.1007/BF00927113. [DOI] [PubMed] [Google Scholar]
- Biederman J, Faraone SV, Weber W, Rater RL, Park K. Correspondence between DSM-III-R and QSM-IV attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36:1682–1687. doi: 10.1097/00004583-199712000-00016. [DOI] [PubMed] [Google Scholar]
- Butzlaff RL, Hooley JM. Expressed emotion and psychiatric relapse. Archives of General Psychiatry. 1998;55:547–552. doi: 10.1001/archpsyc.55.6.547. [DOI] [PubMed] [Google Scholar]
- Chambless DL, Bryan AD, Aiken LS, Steketee G, Hooley JM. The structure of expressed emotion: A three-construct representation. Psychological Assessment. 1999;11:67–76. [Google Scholar]
- Chambless DL, Steketee G. Expressed emotion and behavior therapy outcome: A prospective study with obsessive-compulsive and agoraphobic outpatients. Journal of Consulting and Clinical Psychology. 1999;67:658–665. doi: 10.1037//0022-006x.67.5.658. [DOI] [PubMed] [Google Scholar]
- Christensen, A., & Margolin, G. (1988). Conflict and alliance in distressed and nondistressed families, In R. A. Hinde & J. Stevenson-Hinde (Eds.), Relationship with families: Mutual influences (pp. 263–282). London: Oxford University Press.
- Cohen J. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin. 1968;70:213–220. doi: 10.1037/h0026256. [DOI] [PubMed] [Google Scholar]
- Cook W, Strachan A, Goldstein M, Miklowitz D. Expressed emotion and reciprocal affective relationships in families of disturbed adolescents. Family Process. 1989;28:337–348. doi: 10.1111/j.1545-5300.1989.00337.x. [DOI] [PubMed] [Google Scholar]
- Daley D, Sonuga-Barke EJS, Thompson M. Assessing expressed emotion in mothers of pre-school AD/HD children: Psychometric properties of a modified speech sample in a clinical group. British Journal of Clinical Psychology. 2003;42:53–69. doi: 10.1348/014466503762842011. [DOI] [PubMed] [Google Scholar]
- Doane JA, West KL, Goldstein MJ, Rodnick EH, Jones JE. Parental communication deviance and affective style: Predictors of subsequent schizophrenia spectrum disorders in vulnerable adolescents. Archives of General Psychiatry. 1981;38:679–685. doi: 10.1001/archpsyc.1981.01780310079008. [DOI] [PubMed] [Google Scholar]
- Feldman SS, Rubenstein JL, Rubin C. Depressive affect and restraint in in early adolescents: Relationships with family structure, family process, and friendship support. Journal of Early Adolescence. 1988;8:279–296. [Google Scholar]
- Fisher PW, Schaffer D, Piacentini JC, Lapkin J, Kafantaris V, Leonard H, et al. Sensitivity of the Diagnostic Interview Schedule for Children, (2nd ed.; DISC 2.1) for specific diagnoses of children and adolescents. Journal of the American Academy of Child and Adolescent Psychiatry. 1993;32:666–673. doi: 10.1097/00004583-199305000-00026. [DOI] [PubMed] [Google Scholar]
- Forehand R, Brody GH, Slotkin J, Fauber R, McCombs A, Long N. Young adolescent and maternal depression: Assessment, interrelations, and family predictors. Journal of Consulting and Clinical Psychology. 1988;56:422–426. doi: 10.1037//0022-006x.56.3.422. [DOI] [PubMed] [Google Scholar]
- Gottman, J. M. (1979). Marital interaction: Experimental investigations. San Diego, CA: Academic Press.
- Grotevant HD, Cooper CR. Patterns of interaction in family relationships and the development of ideniity exploration in adolescence. Child Development. 1985;56:415–428. [PubMed] [Google Scholar]
- Hahlweg K, Goldstein MJ, Nuechterlein KH, Magana AB, Mintx J, Doane JA, et al. Expressed emotion and patient–relative interaction in families of recent-onset schizophrenics. Journal of Consulting and Clinical Psychology. 1989;57:11–18. doi: 10.1037//0022-006x.57.1.11. [DOI] [PubMed] [Google Scholar]
- Hermanns J, Florin I, Dietrich M, Rieger C, Hahlweg K. Maternal criticism, mother–child interaction, and bronchial asthma. Journal of Psychosomatic Research. 1989;33:469–476. doi: 10.1016/0022-3999(89)90008-1. [DOI] [PubMed] [Google Scholar]
- Hetherington, E., & Martin., B. (1986). Family factors and psychopathology in children. In H, Quay and J. Werry (Eds.). Psychopathological disorders of childhood (3rd ed., pp. 332–390). New York; Wiley.
- Hibbs ED, Hamburger SD, Kruesi MJP, Lenane M. Factors affecting expressed emotion in parents of ill and normal children. American Journal of Orthopsychiatry. 1993;63:103–112. doi: 10.1037/h0079395. [DOI] [PubMed] [Google Scholar]
- Hibbs ED, Hamburger SD, Lenane M, Rapoport JL, Kruesi MJP, Keysor CS, et al. Determinants of expressed emotion in families of disturbed and normal children. Journal of Child Psychology and Psychiatry. 1991;32:757–770. doi: 10.1111/j.1469-7610.1991.tb01900.x. [DOI] [PubMed] [Google Scholar]
- Hirshfeld DR, Biederman J, Brody L, Faraone SV. Associations between expressed emotion and child behavioral inhibition and psychopathology: A pilot study. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36:205–213. doi: 10.1097/00004583-199702000-00011. [DOI] [PubMed] [Google Scholar]
- Hubschmid T, Zemp M. Interactions in high- and low-EE families. Social Psychiatry and Psychiatric Epidemiology. 1989;24:113–119. doi: 10.1007/BF01788635. [DOI] [PubMed] [Google Scholar]
- Kendall PC, Warman MJ. Anxiety disorders in youth: Diagnostic consistency across DSM-III-R and DSM-IV. Journal of Anxiety Disorders. 1996;10:453–463. [Google Scholar]
- Knight GP, Tein JY, Shell R, Roosa M. The cross-ethnic equivalence of parenting and family interactions among Hispanic and Anglo-American families. Child Development. 1992;63:1392–1403. doi: 10.1111/j.1467-8624.1992.tb01703.x. [DOI] [PubMed] [Google Scholar]
- Magana AB, Goldstein MJ, Karno M, Miklowitz DJ, Jenkins J, Falloon RH. A brief method for assessing expressed emotion in relatives of psychiatric patients. Psychiatry Research. 1986;17:203–212. doi: 10.1016/0165-1781(86)90049-1. [DOI] [PubMed] [Google Scholar]
- Marshall VG, Longwell L, Goldstein MJ, Swanson JM. Family factors associated with aggressive symptomatology in boys with attention deficit hyperactivity disorder: A research note. Journal of Child Psychology and Psychiatry. 1990;31:629–636. doi: 10.1111/j.1469-7610.1990.tb00802.x. [DOI] [PubMed] [Google Scholar]
- McCarty, C. A., Lau, A. S., & Valeri, S. M. (2001). Global coding interpretive manual: UCLA clinic: study. Unpublished manuscript.
- McCarty CA, Weisz JR. Correlates of expressed emotion in mothers of clinically-referred youth: An examination of the five minute speech sample. Journal of Child Psychologya and Psychiatry and Allied Disciplines. 2002;43:759–768. doi: 10.1111/1469-7610.00090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miklowitz DJ, Goldstein MJ, Falloon IRH, Doane JA. Interactional correlates of expressed emotion in the families of schizophrenics. British Journal of Psychiatry. 1984;144:482–487. doi: 10.1192/bjp.144.5.482. [DOI] [PubMed] [Google Scholar]
- Mueser KT, Bellack AS, Wade JH, Sayers SL, Tierney A, Morrison RL. Expressed emotion, social skill, and response to negative affect in schizophrenia. Journal of Abnormal Psychology. 1993;102:339–351. doi: 10.1037//0021-843x.102.3.339. [DOI] [PubMed] [Google Scholar]
- Mundt, C. Fiedler, P., Ernst, S., & Backenstrab, M. (1996). Expressed emotion and marital interaction in endogenous depressive patients. In C. Mundt, M. J. Goldstein. K. Hahlweg, & P. Fiedler (Eds.). Interpersonal factors in the origin and course of affective disorders (pp. 240–256). London: Royal College of Psychiatrists.
- National Institute of Mental Health (NIMH), (1991). The NIMH Diagnostic Interview Schedule for Children. Rockville, MD.
- Patterson, G. (1982). Family coercive process. Eugene, OR: Castalia Press.
- Pelham WE, Lang AR, Atkeson B, Murphy DA, Gnagy EM, Greiner AR, et al. Effects of deviant child behavior on parental alcohol consumption: Stress-induced drinking in parents of ADHD children. American Journal on Addictions. 1998;7:103–114. [PubMed] [Google Scholar]
- Pelham WE, Lang AR, Atkeson B, Murphy DA, Gnagy EM, Greiner AR, et al. Effects of deviant child behavior on parental distress and alcohol consumption in laboratory interactions. Journal of Abnormal Child Psychology. 1997;25:413–424. doi: 10.1023/a:1025789108958. [DOI] [PubMed] [Google Scholar]
- Puig-Antich J, Kaufman J, Ryan ND, Williamson DE, Dahl RE, Lukens E, et al. The psychosocial functioning and family environment of depressed adolescents. Journal of the American Academy of Child and Adolescent Psychiaty. 1993;32:244–253. doi: 10.1097/00004583-199303000-00003. [DOI] [PubMed] [Google Scholar]
- Sameroff AJ. Early influences on development: Fact or fancy? Merrill-Palmer Quarterly. 1975;21:267–294. [Google Scholar]
- Sanders MR, Dadds MR, Bor W. Contextual analysis of child oppositional and maternal aversive behaviors in families of conduct-disordered and nonproblem children. Journal of Clinical Child Psychology. 1989;18:72–83. [Google Scholar]
- Schaefer ES. A configurational analysis of children’s reports of parent behavior. Journal of Consulting Psychology. 1965;29:552–557. doi: 10.1037/h0022702. [DOI] [PubMed] [Google Scholar]
- Schludermann S, Schludermann E. Sociocultural change and adolescents perceptions of parent behavior. Child Development. 1983;19:674–685. [Google Scholar]
- Schwab-Stone M, Shaffer D, Dulcan MK, Jensen PS, Fisher P, Bird HR, et al. Criterion validity of the NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC 2.3) Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:878–888. doi: 10.1097/00004583-199607000-00013. [DOI] [PubMed] [Google Scholar]
- Schwartz JC, Barton-Henry ML, Pruzinsky T. Assessing child-rearing behaviors: A comparison of ratings made by mother, father, child, and sibling on the CRPBI. Child Development. 1985;56:462–479. [PubMed] [Google Scholar]
- Shaffer D, Fisher P, Dulcan MK, Davies M. The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC 2.3): Description, acceptability, prevalence rates, and performance in the MECA study. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:865–877. doi: 10.1097/00004583-199607000-00012. [DOI] [PubMed] [Google Scholar]
- Simoneau TL, Miklowitz DJ, Saleem R. Expressed emotion and interactional patterns in the families of bipolar patients. Journal of Abnormal Psychology. 1998;107:497–507. doi: 10.1037//0021-843x.107.3.497. [DOI] [PubMed] [Google Scholar]
- Strachan, A. M., Goldstein, M. J., & Miklowitz, D. J., (1986). Do relatives express expressed emotion? In M. J., Goldstein, I. Hand, & K. Hahlweg (Eds.). Treatment of Schizophrenia: Family Assessment and Intervention. Heidelberg: Springer-Verlag.
- Strodbeck F. Husband-wife interaction over revealed differences. American Sociological Review. 1951;16:57–68. [Google Scholar]
- Stubbe DE, Zahner GEP, Goldstien MJ, Leckman JF. Diagnostic specificity of a brief measure of expressed emotion: A community study of children. Journal of Child Psychology. 1993;34:139–154. doi: 10.1111/j.1469-7610.1993.tb00976.x. [DOI] [PubMed] [Google Scholar]
- Thornberry TP. Toward an interactional theory of delinquency. Criminology. 1987;25:863–891. [Google Scholar]
- Valone K, Norton JP, Goldstein MJ, Doane JA. Parental expressed emotion and affective style in an adolescent sample at risk for schizophrenia spectrum disorders. Journal of Abnormal Psychology. 1983;92:399–407. doi: 10.1037//0021-843x.92.4.399. [DOI] [PubMed] [Google Scholar]
- Vostanis P, Nicholls J, Harrington R. Maternal expressed emotion in conduct and emotional disorders of childhood. Journal of Child Psychology and Psychiatry and Allied Disciplines. 1994;35:365–376. doi: 10.1111/j.1469-7610.1994.tb01168.x. [DOI] [PubMed] [Google Scholar]
- Wamboldt FS, O’Connor SL, Wamboldt MZ, Gavin LA, Klinnert MD. The five minute speech sample in children with asthma: Deconstructing the construct of expressed emotion. Journal of Child Psychology and Psychiatry and Allied Disciplines. 2000;41:887–898. [PubMed] [Google Scholar]
- Wamboldt FS, Wamboldt MZ, Gavin LA, Roesler TA, Brugman SM. Parental criticism and treatment outcome in adolescent hospitalized for severe, chronic asthma. Journal of Psychosomatic Research. 1995;39:995–1005. doi: 10.1016/0022-3999(95)00507-2. [DOI] [PubMed] [Google Scholar]
- Wierson M, Forehand R, McCombs A. The relationship of early adolescent functioning to parent-reported and adolescent-perceived interparental conflict. Journal of Abnormal Child Psychology. 1988;16:707–718. doi: 10.1007/BF00913480. [DOI] [PubMed] [Google Scholar]
- Williams CA, Forehand R. An examination of predictor variables for child compliance and noncompliance. Journal of Abnormal Child Psychology. 1984;12:491–504. doi: 10.1007/BF00910661. [DOI] [PubMed] [Google Scholar]