Abstract
Expressed emotion (EE) is a robust predictor of outcome in bipolar disorder. Despite decades of research, interventions to reduce EE levels have had only modest effects. This study used an expanded model of EE to develop an intervention. Research has demonstrated a strong link between attributions and EE in families of patients with psychiatric disorders. There is also substantial research to suggest that anger can drive blaming attributions. Combining these ideas, this study built on previous psychoeducation interventions through the addition of an acceptance component designed to decrease anger and blaming attributions among family members of those with bipolar disorder. Twenty-eight family members attended a 1-day or 2-evening multifamily group workshop and completed a follow-up assessment 1 week later. At follow-up, participants demonstrated more knowledge about bipolar disorder. Anger, blaming attributions, and number of criticisms remained unchanged. Results of this study are consistent with others in that it is difficult to change EE. Implications for future clinical research in this area are addressed.
The construct of expressed emotion (EE) is defined as critical, hostile, or emotionally overinvolved (EOI) attitudes toward a relative with a psychiatric disorder (Hooley, Rosen, & Richters, 1995). EE has been linked with poor patient outcome for several psychiatric disorders (Barrowclough & Hooley, 2003; Butzlaff & Hooley, 1998). Butzlaff and Hooley (1998) found a meta-analytic effect size of r=.39 across six studies that examined the relationship between EE and relapse within mood disorders. More recent research suggests that, although EE was unrelated to manic relapse, high EE was related to a five-fold increase in the odds of a depressive recurrence, even after controlling for symptom severity (Yan, Hammen, Cohen, Daley, & Henry, 2004). Criticism has been found to be the most important element of EE for understanding the course of mood disorders (Hooley et al., 1995) and for bipolar depression (Kim & Miklowitz, 2004). Thus, EE has an influence on bipolar relapse, more specifically for depression.
What drives high levels of criticism? According to Weiner’s (1995) attribution-affect model, attributions influence the way people experience emotion. In particular, when an individual perceives that another person is in control of his or her behavior, this leads to a judgment that the other person is responsible for his or her behavior. A judgment that the other person is responsible elicits anger and criticism. High EE relatives who are critical have a tendency to view symptoms of mental illness as behaviors that are within the patient’s control (Barrowclough, Johnston, & Tarrier, 1994). Because high EE relatives believe the patient is in control of his or her behavior, they believe the patient must also be capable of modifying this behavior. Thus, relatives struggle to normalize the patient’s behavior through negative feedback and controlling behavior of their own (Hooley & Campbell, 2002).
Several studies of schizophrenia and depression have shown that high EE is related to personal, internal, and controllable attributions (Barrowclough et al., 1994; Brewin, MacCarthy, Duda, & Vaughn, 1991; Hooley & Licht, 1997; Wendel, Miklowitz, Richards, & George, 2000). Wendel et al. (2000) found that high EE relatives made more personal and controllable attributions than low EE relatives, whereas stability and internality of attributions were not related to EE. Attributions of control were particularly related to the criticism and hostility components of EE. In sum, family beliefs that a patient’s symptoms are personal and controllable appear related to criticism. As such, unfairly blaming attributions may lead to poorer patient outcomes.
Weiner’s attributional theory has been the foundation for many family psychoeducation programs in schizophrenia and other psychiatric disorders (Dixon, Adams, & Lucksted, 2000). The purpose of psychoeducation is to reduce family members’ guilt, confusion, helplessness, and sense of overresponsibility. The ultimate goal is to enable family members to be less critical of their ill relative by addressing misattributions for illness (Lefley, 1996).
Several studies have demonstrated that family psychoeducation programs can increase knowledge about the illness, increase familial support, reduce family burden, increase self-efficacy, and reduce relapse rates in psychiatric disorders (Abramowitz & Coursey, 1989; Cozolino, Goldstein, Nuechterlein, & West, 1988; Dixon et al., 2000; Honig, Hofman, Rozendaal, & Dingemans, 1997; Pitschel-Walz, Leucht, Baeuml, Kissling, & Engel, 2001; Solomon, Draine, Mannion, & Meisel, 1996). In bipolar disorder, family, couple, and parent psychoeducation programs also have significantly improved nonverbal interactions, caregiver knowledge of the disorder, patient understanding of the illness, positive family interactions, caregiver distress, coping, attributions, patient functioning, and patient medication adherence (Bland & Harrison, 2000; Brent, Poling, McKain, & Baugher, 1993; Clarkin, Carpenter, Hull, Wilner, & Glick, 1998; Fristad, Goldberg-Arnold, & Gavazzi, 2003; Simoneau, Miklowitz, Richards, Saleem, & George, 1999; van Gent & Zwart, 1991).
These psychoeducation programs, however, were developed with the aim of reducing EE. Whereas they have attained positive effects on many outcomes, the results are less uniform when considering EE levels. Specifically, these programs failed to improve negative verbal interactions (Fristad et al., 2003; Simoneau et al., 1999). To date, we can identify only one study that documented significant reductions in EE with treatment compared to control (Honig et al., 1997). Honig et al. reported a significant change in EE for a treatment group compared to a control group: results of a six-session multifamily psychoeducational intervention showed that 31% of relatives of patients with bipolar disorder in the treatment group changed from high EE to low EE, compared to none of the relatives in the control group. The authors, however, did not report if changes to EE were specific to criticism or overinvolvement. Taken together, findings suggest that currently available interventions have achieved only limited success with the core goal of reducing EE.
One way to improve interventions might be to consider other variables driving EE. Substantial literature suggests that emotions provide fuel for quick heuristic judgments of behavior that match a person’s mood state. For example, anger triggers attributions that blame people rather than the situations people face (Keltner, Ellsworth, & Edwards, 1993; Lerner & Keltner, 2001; Smith & Ellsworth, 1985). Longer-lasting mood states also guide appraisals (Siemer, 2005). Given that anger would be expected to lead to attributions of responsibility for behavior and, as noted above, that attributions of responsibility for behavior can increase anger, it is important to consider a bidirectional relationship between attributions of responsibility and angry mood.
As described above, traditional psychoeducation programs were based on the notion that the provision of accurate information about the illness would modify attributions and thereby decrease EE. However, attributions of blame might be intensified by anger, and traditional psychoeducational programs do not address anger. In this study, basic psychoeducation was augmented by targeting anger in order to reduce blaming attributions.
To reduce anger, materials from Christensen and Jacobson’s integrative behavioral couple therapy (IBCT) were adapted for this intervention (Christensen & Jacobson, 2000). IBCT was designed to decrease negative interactions among partners by emphasizing emotional acceptance (Christensen et al., 2004). In contrast to traditional behavioral couple therapy (TBCT), which seeks to change behavior of spouses, the goal of IBCT is to help spouses accept, without promoting resignation, aspects of their partners that were previously unacceptable (Jacobson, Christensen, Prince, Cordova, & Eldridge, 2000). Christensen and Jacobson (2000) define acceptance as the ability “to tolerate what you regard as an unpleasant behavior of your [relative], probably to understand the deeper meaning of that behavior, certainly to see it in a larger context, and perhaps even to appreciate its value and importance in your relationship” (p. 124).
IBCT uses the acceptance technique known as “unified detachment” to help partners gain a degree of emotional distance from their problems (Cordova, Jacobson, & Christensen, 1998). The goal of this technique is to help family members learn to talk about their problems without engaging in conflicts. It also encourages family members to recognize that certain aspects of their relative are unchangeable, such as bipolar illness, and to be accepting rather than pushing them to change.
IBCT has been shown to significantly improve relationship satisfaction and to decrease global distress among couples (Christensen et al., 2004). Of more relevance, IBCT interventions have been shown to produce large increases in nonblaming discussions as compared to TBCT (Cordova et al., 1998). Thus, it appears that IBCT may be useful for decreasing angry mood and criticism in family members.
The goal of this study was to test whether a brief psychoeducational intervention would lead to a decrease in family criticism. Relatives of people with bipolar disorder participated in a 1-day or 2-evening multifamily group workshop that combined psychoeducation and acceptance. The intervention was designed as a brief program so that it could be practically implemented in community settings. We hypothesized that the workshop would increase family members’ knowledge about bipolar disorder and decrease family members’ anger. We further hypothesized that increased knowledge and decreased anger would predict more positive and less blaming attributions regarding relatives’ symptoms of bipolar disorder.
Method
OVERVIEW OF DESIGN
Relatives of people with bipolar disorder were recruited for a 1-day or 2-evening intervention. All participants completed assessments before and 1 week after the intervention.
PARTICIPANTS
Participants were recruited from the community in South Florida through local outpatient hospitals, support groups, fliers, and print and Internet advertising. Callers were eligible if they were between the ages of 18 and 70 and if they had a relative older than the age of 12 diagnosed with bipolar disorder. Participants were between ages 21 and 70. Up to three persons per family were allowed to participate in the multifamily group.
Eighty-two callers were screened by phone. Callers were asked whether they had a relative who had been previously diagnosed with bipolar disorder. This was followed by a structured interview, modeled after the Structured Clinical Interview for DSM-IV (SCID-IV; First, Spitzer, Gibbon, & Williams, 1997), to question the caller about lifetime presence of manic symptoms in their relative. Callers were invited to participate if their relative demonstrated elevated mood accompanied by at least two additional symptoms or irritable mood accompanied by at least three additional symptoms. No DSM-IV diagnoses of bipolar disorder were made. In addition, prescreening was conducted to determine whether the caller demonstrated a cognitive or linguistic inability, such as inability to read, write, or speak English well enough to comfortably participate in the workshop and complete the measures. Of those screened, participants were excluded if they did not have a relative diagnosed with bipolar disorder (n=6), if they did not feel that their relative would sign a consent form allowing them to participate (n=2), if the person with bipolar disorder was younger than the age of 12 (n=1), or if they were no longer with their romantic partner or their relative with bipolar disorder was deceased (n=2). Eleven people decided after the phone screen that they were not interested in the program. Hence, of the initial 82 callers who contacted us by phone, 60 families were interested in and eligible for the study. Within these families, 24 were interested in having more than one family member attend.
Of the 84 potential participants from 60 families, 42 (13 male, 29 female) adults from 32 families attended the workshops. There were no differences between individuals who did and did not attend on the type of relative diagnosed with bipolar disorder, whether they lived with that relative, whether their relative had done anything to cause trouble for themselves or for the family, or whether the relative had been hospitalized for mania. Eight families had 2 members attend, and 1 family had 3 members attend. There were no differences in pre or post levels of criticism between individuals who attended with an additional family member and those who attended alone. Fifty-five percent of the sample were parents of a child with bipolar disorder (mean age of child=25.93, SD=9.81), 9.5% were siblings of a person with bipolar disorder, 5% were adult children of a person with bipolar disorder, and 31% were genetically unrelated (i.e., spouse or partner) to the person with bipolar disorder. Data from 3 participants were excluded because of technical difficulties with the Five Minute Speech Sample (FMSS; Magaña, Goldstein, Karno, & Miklowitz, 1986). Five participants did not complete follow-up assessments. A chi-square analysis indicated that participants who did and did not complete follow-up did not differ in EE. The remaining 35 participants were from 28 families. For analyses, the family member who currently had the most contact per week, as indicated by self-report, was selected. If the family members spent an equal amount of time with the patient, then one of the family members’ data were randomly selected for analyses.
Demographics for the final sample (9 men, 19 women) are shown in Table 1. Overall, the sample was highly educated, predominantely Caucasian, English speaking, and employed either full or part time. This sample was also highly educated about and experienced with bipolar disorder and had obtained information from a psychiatrist, therapist, or other sources. The family members with bipolar disorder (50% female) had an average age of 35.20 years (SD=16.98), and most were offspring or spouses of the participating relatives. Most of the sample reported that their relative had been coping with bipolar disorder for more than 5 years and had experienced multiple episodes of mania and depression. Sixty-five percent of the family members with bipolar disorder had been hospitalized at least once for their illness. Only two families attended the workshop after their relative experienced their first manic episode.
Table 1.
Variable | |
---|---|
Gender | |
Male | 32.1% |
Ethnicity | |
Caucasian | 71.4% |
Hispanic | 21.4% |
African-American | 3.6% |
Marital Status | |
Single | 7.1% |
Married | 75% |
Divorced | 17.9% |
Employment Status | |
Full-time | 53.6% |
Part-time | 10.7% |
Language | |
English | 82.1% |
Bilingual | 32.1% |
Education | |
College Graduate or higher | 78.5% |
High School or Some College | 21.4% |
INTERVENTION
To ensure comparability to other interventions, the psychoeducation module was adapted from several well-developed sources of family psychoeducation for bipolar disorder to increase relatives’ knowledge (Miklowitz & Goldstein, 1997; Poling, Brent, & Birmaher, 1999; Rigsby-Jones et al., 1994). The acceptance component was drawn from the marital therapy literature, specifically from Christensen and Jacobson’s (2000) IBCT. IBCT focuses on the emotions that might underlie a couple’s bitter communications. More detailed information regarding the content of the workshop is included in Table 2.
Table 2.
Module | Goals | Content |
---|---|---|
Psychoeducation (adapted from Miklowitz & Goldstein, 1997; Poling, Brent, & Birmaher, 1999) |
|
|
Making sense of the patient’s behaviors and relatives’ responses (adapted from Rigsby-Jones et al., 1994; Christensen & Jacobson, 2000) |
|
|
Acceptance (adapted from Christensen & Jacobson, 2000) |
|
Acceptance coping skills:
|
The intervention was held at the University of Miami and was co-led by two advanced clinical psychology graduate students specializing in bipolar disorder. All participants received supplemental written materials and worksheets. The intervention was designed as a 1-day (9:00 A.M. – 4:00 P.M.) or 2-evening (5:30 P.M. – 9:00 P.M.) multifamily group workshop. The evening workshop was adapted from the 1-day workshop, and each topic received an equal amount of time in both workshops. Only one 2-evening workshop was run to accommodate individuals who could not attend on a Saturday. Single multifamily group sessions have been shown to offer several advantages (Moltz & Newmark, 2002), including reductions in stigma (Miklowitz, 2004) and providing an opportunity to build a support network with other families going through the same experience (Anderson, Reiss, & Hogarty, 1986; McFarlane et al., 2003).
In this study, multifamily groups consisted of 3 to 11 people from three or more families. The relatives with bipolar disorder themselves were not involved in the workshop. Having the patient present has been shown to have some negative effects on the group process, with relatives in patient-present groups making fewer personal and topic-relevant comments (Reilly, Rohrbaugh, & Lackner, 1988). Previous psychoeducation programs have excluded patients to provide a forum in which the relatives can talk freely (Cozolino et al., 1988; Linszen et al., 1996; Reinares et al., 2004; Solomon et al., 1996).
PROCEDURE
Participants were screened initially by a research assistant over the phone to determine if they were eligible to participate. All participants completed measures of knowledge about bipolar disorder, angry mood, attributions, and a speech sample to assess EE before and after the intervention. At baseline, participants were asked to provide demographic data and treatment history for themselves and history of illness of the identified patient. Participants also provided client satisfaction feedback about the content and the group leaders after the intervention. Follow-up assessments of knowledge, angry mood, attributions, and EE were conducted by phone 1 week after the intervention.
MEASURES
Knowledge about illness
The Understanding Mood Disorders Questionnaire (UMDQ) was designed to assess family members’ knowledge of mood disorders (Gavazzi, Fristad, & Law, 1997). The measure includes 59 questions about illness attributions, knowledge of symptoms, course, and treatment of the disorder, which are summed to produce scores ranging from 0 to 59. The UMDQ is sensitive to posttreatment changes in knowledge regarding mood disorders and demonstrates good reliability (α=.73; Gavazzi et al., 1997). In this sample, reliability was good both preworkshop (α=.81) and postworkshop (α=.75).
Expressed emotion
The Five Minute Speech Sample (FMSS) is designed to assess criticism and dissatisfaction among family members towards a mentally ill relative (Magaña et al., 1986). The FMSS was developed as a brief alternative to the Camberwell Family Interview (CFI), the traditional assessment for EE (Brown & Rutter, 1966).
Relatives are asked to speak freely into a tape recorder for 5 minutes about their ill family member (Van Humbeeck, Van Audenhove, De Hert, Pieters, & Storms, 2002). The recording is then coded on four dimensions: (1) quality of the initial statement, (2) quality of relationship, (3) criticism, and (4) emotional overinvolvement. The dimension of criticism, measured as the number of critical comments, was used as the primary outcome measure.
All FMSS were audiotaped, transcribed, and coded using both tapes and transcripts. Four undergraduate raters were trained in FMSS ratings by a FMSS-certified graduate student rater. To begin training, all raters completed an initial didactic session. Raters then coded six training tapes and met as a group to review and establish consensus. Consensus was further established on four speech samples from this study. A random sample of 10 tapes was selected for reliability analyses. Reliability between the graduate student rater and the undergraduate student raters was determined to be inadequate (ICC<.60). Ratings from the unreliable raters were discarded. A new undergraduate rater was trained using the same procedures and re-rated tapes by the unreliable raters. To eliminate possible biases in ratings, if the undergraduate student rated the preworkshop FMSS tape, the postworkshop tape was coded by the FMSS-certified graduate student rater and vice versa. Both raters in this study were unaware of participants’ treatment status, and their individual ratings were used in analyses. To ensure that ratings were reliable, two trained graduate student raters independently reviewed 10 randomly selected FMSS tapes. Interrater reliability was adequate for number of critical comments, ICC (9, 18)=.71.
EE ratings using the FMSS have a concurrent validity of 75% with the EE ratings from the CFI, although the FMSS has a tendency to underestimate the score of the CFI in 20% to 30% of samples (Magaña et al., 1986; Van Humbeeck et al., 2002). Shimodera et al. (2002) showed the sensitivity with the FMSS was 66.7% and the specificity was 95.2% in a sample of Japanese families of patients with mood disorders.
Attributions
The Causal Dimension Scale (CDS-II) is a self-report measure of how people perceive causes of particular situations (McAuley, Duncan, & Russell, 1992). The scale includes three items. Participants were asked to describe their thoughts about the cause of their relative’s bipolar disorder and then rate this on dimensions of controllability, intentionality, and responsibility. Scores range from 3 to 27, with higher scores indicating a tendency to attribute causality to the patient. The CDS-II demonstrated good reliability (Cronbach’s α=.81 and .84) in previous studies (Friedman, 2005; Weisman & Lopez, 1997). In this study, reliability was adequate pre- and postworkshop (α=.76 and .66).
A Behavior Controllability subscale was also included. Following the same format of the CDS-II, participants are asked to identify two behaviors related to their relative’s disorder and rate them on the same dimensions of controllability, responsibility, and intentionality. Scores of the total scale (CDS-II and Behavior Controllability subscale) range from 9 to 56, with higher scores indicating more blaming attributions for cause of the disorder and for behaviors. All items together had Cronbach’s alphas of .85 and .81 before and after the workshop, respectively.
Angry mood
The State Trait Anger Expression Inventory (STAXI) is a 44-item questionnaire designed to capture the intensity and frequency of anger (Spielberger, 1988). Although the scale includes five scales and two subscales, only the 15-item State Anger (S-Anger) scale was used (Spielberger & Sydeman, 1994). Factor analysis studies of the S-Anger scale have demonstrated that the items from this scale load onto two factors: feel angry and feel like expressing anger (Forgays, Forgays, & Spielberger, 1997). The feel angry factor was the primary measure of angry mood state. Items (i.e., “I am furious”) are rated on a 4-point intensity scale from “not at all” to “very much so.” The instructions of the scale were modified to capture the intensity of angry feelings towards the ill relative in the past week. The S-Anger scale has demonstrated a high degree of internal consistency (α=.90 or higher; Spielberger, 1988). In this study, the reliability for the Feeling Angry subscale of the State Anger scale of the STAXI was high pre- and post-workshop (α=.88 and α=.91).
Program evaluation
A modified version of the 8-item Client Satisfaction Questionnaire (CSQ-8) was used in this study to assess overall client satisfaction with the intervention (Nguyen, Attkisson, & Stegner, 1983). This self-report measure demonstrates good internal consistency, Cronbach’s alpha=.87, and high correlations (r=.60 to .80) with other consumer satisfaction instruments (Attkisson & Greenfield, 1994). Cronbach’s alpha for the modified CSQ-8 was high (α=.86).
Clients also evaluated the group leaders using the Working Alliance Inventory (WAI; Horvath & Greenberg, 1986, 1989). The WAI was developed to measure alliance factors in all types of therapy (Horvath, 1994). Each item is rated on a 7-point scale (1=never, 7=always). Research has shown strong support for the reliability of the WAI, ranging from α=.85 to .93 (Horvath & Greenberg, 1989). This study used a modified version of the WAI to capture the participants’ perspective on the alliance in a one-time group workshop. Each therapist was rated by participants on items on a 7-point scale of agreement (1=strongly disagree, 7=strongly agree). Cronbach’s alpha for the modified WAI in this study was good (α=.85).
In addition, each workshop was either videotaped or audiotaped. The tapes were coded by trained raters using a modified version of the Behavioral Family Management Therapist Competency and Adherence Scale (BFM-TCAS), designed to assess therapist competency and treatment adherence (Weisman et al., 1998). The modification to the scale included questions that specifically addressed the different modules of the workshop: psychoeducation and acceptance. The revised measure contained a total of 8 items rated on a scale of 1 to 7. After establishing reliability, two undergraduate raters each rated four tapes. Cronbach’s alpha for the modified TCAS in this sample was .74.
Results
OVERVIEW OF ANALYSES
Before conducting analyses of hypotheses, participant satisfaction and therapist adherence and competence were examined. Next, distributions were examined for normalcy and outliers. The primary goal of the current study was to determine whether participants demonstrated increased levels of knowledge about bipolar disorder, decreased anger, fewer blaming attributions, and fewer critical comments toward their relative with the illness. To do so, paired dependent t-tests were conducted to examine whether these variables changed from baseline to follow-up.
PARTICIPANT SATISFACTION, THERAPIST ADHERENCE AND COMPETENCE
Workshop participants reported a high degree of satisfaction with the program on the Client Satisfaction Questionnaire (M=25.26, SD=2.69, of a total possible score of 28, N=42) and with the workshop facilitators on the Working Alliance Inventory (M=75.27, SD=6.95, of a total possible score of 77, N=42). Eighty-four percent were very satisfied with the quality of service, 90% felt that the program met their needs, and 97% felt that they would be able to deal with their problem more effectively after the workshop. One participant expressed dissatisfaction that the workshop did not address the process of hospitalization. Some written comments by relatives who participated in the intervention included the following: “In 31 years, I’ve never discussed my [relative’s] disorder and behavior with a group, so it was a great value for me to be open and honest about who he is and what he’s done and how I’ve coped with it”; “It was a great experience for me and I would recommend it to anyone that needs help.”
Therapists demonstrated a high degree of competency and strong adherence to the intervention. Across seventapes, the mean score on the Competency Rating Scale was 50.43 (SD=2.37) out of a possible total of 56.
ANALYSES
The pattern of missing data suggested no systematic response bias. That is, missing values were not clustered for a particular variable or for any participant. Missing data were imputed for number of hours spent with relative and age of relative before the workshop using the expectation maximization (EM) method in SPSS Missing Value Analysis software (Little & Rubin, 1987). The percentage of missing data for these variables was 36% and 7%, respectively. This algorithm does not impute missing data for categorical variables.
Before conducting tests of hypotheses, univariate distributions for all continuous variables were examined to determine if a transformation of a variable was required. Because distributions mirrored those expected for a relatively healthy community sample, distributions were not transformed. Means and standard deviations of all measures are reported in Table 3. Participants came into the workshop with high levels of knowledge as measured by the UMDQ and mild levels of anger as measured by the STAXI. For nonpsychiatric populations, the mean state anger score for females age 30 and over is 17.50 (SD=4.52) and for males is 18.43 (SD=5.63) (Spielberger, 1988). The majority of families expressed no criticisms at baseline, which is consistent with other studies (Rein et al., 2006). The number of criticisms was not related to the number of hours spent with the relative (r=.32, p>.10). Summarizing across measures, most people in the sample experienced relatively little anger and tended not to make many criticisms.
Table 3.
Measure | Pre Means | Post Means | Mean Difference | SE | t (27) | p | Effect size |
---|---|---|---|---|---|---|---|
Knowledge | 30.12(4.86) | 33.89(3.16) | −3.77 | 0.85 | −4.44 | <.001 | .79 |
Feeling Angry | 10.57(3.96) | 9.40(3.87) | 1.17 | 0.88 | 1.33 | .20 | .21 |
Blaming Attributions | 26.85(12.38) | 26.07(12.42) | 0.79 | 1.84 | 0.43 | .67 | .09 |
Number of criticisms | 0.25(0.44) | 0.14(0.45) | 0.11 | 0.13 | 0.83 | .42 | .10 |
Before conducting primary analyses, correlations between the measures administered before and after the workshop were examined (Table 4). Before the workshop, knowledge was negatively correlated with blaming attributions. Anger was positively correlated with blaming attributions. It is noteworthy that the correlation between blaming attributions and number of criticisms was unexpectedly low (r=.02) and not significant.
Table 4.
PREWORKSHOP | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
---|---|---|---|---|---|---|---|---|
1. Knowledge | 1 | −.19 | −.39* | −.20 | .44* | .06 | −.32 | −.08 |
2. Feeling Angry | 1 | .62** | −.06 | −.26 | .29 | .55** | .16 | |
3. Blaming Attributions | 1 | .02 | −.16 | .09 | .69** | .41* | ||
4. Number of Criticisms | 1 | −.11 | −.17 | .02 | −.19 | |||
POSTWORKSHOP | ||||||||
5. Knowledge | 1 | −.17 | −.43* | .14 | ||||
6. Feeling Angry | 1 | .31 | .07 | |||||
7. Blaming Attributions | 1 | .27 | ||||||
8. Number of Criticisms | 1 |
p<.05;
p<.01;
p<.001.
Next, the correlations between each measure before and after the workshop were examined. As would be expected, blaming attributions correlated positively. There were moderate correlations between pre- and postworkshop measures of knowledge. Pre- and post-workshop measures of anger and number of criticisms were not correlated.
In an examination of the correlations among measures administered after the workshop, blaming attributions remained negatively correlated with knowledge. The relationship between anger and attributions was not significant. Number of criticisms after the workshop did not correlate with any other postworkshop variables.
TESTS OF HYPOTHESES
Using a within-subjects pre-post design, paired dependent t-tests were used to test the hypothesis that after the intervention relatives would demonstrate an increased knowledge of bipolar disorder, less anger about their ill relative, fewer blaming attributions, and fewer critical comments. Analyses demonstrated that knowledge increased significantly (t(27)=−4.44, p<.001; effect size=.79), but no significant changes were observed in blaming attributions, anger, or in the number of criticisms (Table 3). In examining the knowledge variable further, the area relatives improved most was knowledge about symptoms of the disorder, t(27)=4.42, p<.001.
Discussion
Many family psychoeducation programs for psychiatric disorders have been created with the goal of decreasing EE, particularly criticism and hostility, towards the ill relative. These programs have been designed based on the theory that attributions people make about the illness and related behaviors can lead to blame, which often results in critical comments and hostile attitudes. Providing psychoeducation should theoretically lead to changes in attributions and fewer critical comments. Although such programs have yielded changes on many outcome measures (Bland & Harrison, 2000; Brent et al., 1993; Clarkin et al., 1998; Fristad et al., 2003; Simoneau et al., 1999; van Gent & Zwart, 1991), they have failed to produce changes in EE or negative behaviors (Fristad, Gavazzi, & Mackinaw-Koons, 2003; Simoneau et al., 1999), with one exception (Honig et al., 1997). The findings of this study are consistent with the majority of the literature on psychoeducation programs.
To date, most programs have provided the facts about the disorder to decrease blaming attributions. This study drew on research that suggests that mood state, in particular an angry mood state, can also drive blaming attributions. As criticism rather than overinvolvement has been the component of EE linked to relapse, this study augmented traditional psychoeducation with an acceptance module to decrease anger, with the ultimate goal of modifying blaming attributions and reducing criticism.
The intervention had some positive results. The results of the consumer satisfaction feedback and the therapist evaluations demonstrated that people were satisfied with the information they received. Independent raters judged therapist adherence and competence to be high across all workshops. The results from these evaluations are comparable to results found using the same measures in studies with similar methodologies and goals (Friedman, 2005; Glueckaf et al., 2002; Weisman et al., 1998). Consistent with the findings of previous research, participant knowledge increased, specifically about symptoms of the disorder, despite the fact that the sample was highly knowledgeable before the study (Bernhard et al., 2006; Reinares et al., 2004). Although there were low levels of anger before the intervention, anger was highly correlated with blaming attributions before the workshop, suggesting that anger may still be an important target that warrants a direct intervention. However, it may be more difficult to significantly modify anger in a brief intervention.
Unfortunately, this program did not lead to significant decreases in anger or criticism. Several factors could help explain the null results. The small sample size resulted in limited statistical power. There may have also been confounds within the sample. The participants were relatively knowledgeable and largely not angry or critical, even before the intervention. This may be a reflection of the methods used to capture these constructs. Low levels of angry mood may be due to the fact that anger was assessed using a self-report instrument and may have been more accurately measured through behavioral observation of an interaction with the relative. In addition, the reliance on the FMSS may have also interfered with the ability to assess criticism, as researchers have noted that the FMSS may fail to identify 20% (Hooley & Parker, 2006) to 31% (Heikkila et al., 2002) of those who would be categorized as such using the CFI. Also, we did not assess current mood state of the relative with bipolar disorder or obtain diagnostic information on the participants, which may have influenced responses to questionnaires or to the intervention. Another methodological limitation was the length of the follow-up period. After 1 week, participants may not have had enough time to implement, practice, and fine-tune the skills learned in the workshop (Bernhard et al., 2006; Doss, Thum, Sevier, Atkins, & Christensen, 2005). Nonetheless, it must be stated that this intervention may simply not be effective.
If this intervention is not an effective one, what types of changes might be important in future research on family interventions? It may be more useful to offer repeated sessions rather than a day-long workshop, as participants were tired by the end of the day when the acceptance module was covered. For angry and critical family members, it may take a more in-depth intervention lasting 8 to 12 sessions to bring about true change, which may also have a broader positive impact on the life of the participant. Also, it may be more effective to talk about acceptance with the patient present. One might envision that a longer acceptance intervention with all family members present might increase empathy and compassion, promote a descriptive view of the problem, and build tolerance by enhancing the positive aspects of the negative behavior. Another alternative might be to offer the program to families independently, rather than in groups; this might enhance the ability of therapists to address issues specific to each family. It is also possible that this intervention might be effective in a population that is less knowledgeable about the disorder or among families experiencing a more recent diagnosis.
Psychoeducation programs have been created with the intention of decreasing EE, but perhaps EE is too narrow a target for intervention programs. That is, even low EE family members may need other services. Low EE individuals in this sample reported that they received benefits from participating. Programs targeting the broader construct of caregiver burden may be warranted, particularly given the high rates of depression and mental health concerns among these families (Ogilvie, Morant, & Goodwin, 2005; Perlick, Hohenstein, Clarkin, Kaczynski, & Rosenheck, 2005).
In conclusion, findings of this study are, for the most part, consistent with those of other brief intervention programs. After the workshop, caregivers demonstrated increased knowledge about and understanding of the illness. We were not successful in modifying blaming attributions or, consistent with other brief multifamily interventions, in reducing levels of criticism. More research is needed to identify treatment targets to refine family interventions for bipolar disorder.
Acknowledgments
The authors thank Stephanie McMurrich for her assistance with the workshops. The authors thank Stephanie Wasserman, Ashley Cain, Lindsey Pepin, Ana Maria LiRosi, Gavin MacKenzie, Rebecca Siegel, Dan Fulford, and Philip Garza for their assistance with data collection and coding.
References
- Abramowitz IA, Coursey RD. Impact of an educational support group on family participants who take care of their schizophrenic relatives. Journal of Consulting and Clinical Psychology. 1989;57:232–236. doi: 10.1037//0022-006x.57.2.232. [DOI] [PubMed] [Google Scholar]
- Anderson CM, Reiss DJ, Hogarty GE. Schizophrenia and the family. New York: Guilford Press; 1986. [Google Scholar]
- Attkisson CC, Greenfield TK. The Client Satisfaction Questionnaire-8 and the Service Satisfaction Scale-30 (SSS-30) In: Maruish ME, editor. The use of psychological testing for treatment planning and outcome assessment. Hillsdale, NJ: Lawrence Erlbaum; 1994. pp. 402–420. [Google Scholar]
- Barrowclough C, Hooley JM. Attributions and expressed emotion: A review. Clinical Psychology Review. 2003;23:849–880. doi: 10.1016/s0272-7358(03)00075-8. [DOI] [PubMed] [Google Scholar]
- Barrowclough C, Johnston M, Tarrier N. Attributions, expressed emotion, and patient relapse: An attributional model of relatives’ response to schizophrenic illness. Behavior Therapy. 1994;25:67–88. [Google Scholar]
- Bernhard B, Schaub A, Kummler P, Dittmann S, Severus E, Seemuller F, et al. Impact of cognitive-psychoeducational interventions in bipolar patients and their relatives. European Psychiatry. 2006;21:81–86. doi: 10.1016/j.eurpsy.2005.09.007. [DOI] [PubMed] [Google Scholar]
- Bland R, Harrison C. Developing and evaluating a psychoeducation program for caregivers of bipolar affective disorder patients: Report of a pilot project. Research on Social Work Practice. 2000;10:209–228. [Google Scholar]
- Brent DA, Poling K, McKain B, Baugher M. A psychoeducational program for families of affectively ill children and adolescents. Journal of the American Academy of Child and Adolescent Psychiatry. 1993;32:770–774. doi: 10.1097/00004583-199307000-00010. [DOI] [PubMed] [Google Scholar]
- Brewin CR, MacCarthy B, Duda K, Vaughn CE. Attribution and expressed emotion in the relatives of patients with schizophrenia. Journal of Abnormal Psychology. 1991;100:546–554. doi: 10.1037//0021-843x.100.4.546. [DOI] [PubMed] [Google Scholar]
- Brown GW, Rutter M. The measurement of family activities and relationships: A methodological study. Human Relations. 1966;19:241–263. [Google Scholar]
- Butzlaff RL, Hooley JM. Expressed emotion and psychiatric relapse. Archives of General Psychiatry. 1998;55:547–552. doi: 10.1001/archpsyc.55.6.547. [DOI] [PubMed] [Google Scholar]
- Christensen A, Atkins DC, Berns S, Wheeler J, Baucom DH, Simpson LE. Traditional versus integrative behavioral couple therapy for significantly and chronically distressed married couples. Journal of Consulting and Clinical Psychology. 2004;72:176–191. doi: 10.1037/0022-006X.72.2.176. [DOI] [PubMed] [Google Scholar]
- Christensen A, Jacobson NS. Reconcilable differences. New York: Guilford Press; 2000. [Google Scholar]
- Clarkin JF, Carpenter D, Hull J, Wilner P, Glick ID. Effects of psychoeducational intervention for married patients with bipolar disorder and their spouses. Psychiatric Services. 1998;49:531–533. doi: 10.1176/ps.49.4.531. [DOI] [PubMed] [Google Scholar]
- Cordova JV, Jacobson NS, Christensen A. Acceptance versus change interventions in behavioral couple therapy: Impact on couples’ in-session communication. Journal of Marital and Family Therapy. 1998;24:437–455. doi: 10.1111/j.1752-0606.1998.tb01099.x. [DOI] [PubMed] [Google Scholar]
- Cozolino LJ, Goldstein MJ, Nuechterlein KH, West KL. The impact of education about schizophrenia on relatives varying in expressed emotion. Schizophrenia Bulletin. 1988;14:675–687. doi: 10.1093/schbul/14.4.675. [DOI] [PubMed] [Google Scholar]
- Dixon L, Adams C, Lucksted A. Update on family psychoeducation for schizophrenia. Schizophrenia Bulletin. 2000;26:5–20. doi: 10.1093/oxfordjournals.schbul.a033446. [DOI] [PubMed] [Google Scholar]
- Doss BD, Thum YM, Sevier M, Atkins DC, Christensen A. Improving relationships: Mechanisms of change in couple therapy. Journal of Consulting and Clinical Psychology. 2005;73:624–633. doi: 10.1037/0022-006X.73.4.624. [DOI] [PubMed] [Google Scholar]
- First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV Axis I Disorders. Washington, DC: American Psychiatric Press; 1997. [Google Scholar]
- Forgays DG, Forgays DK, Spielberger CD. Factor structure of the State-Trait Anger Expression Inventory. Journal of Personality Assessment. 1997;69:497–507. doi: 10.1207/s15327752jpa6903_5. [DOI] [PubMed] [Google Scholar]
- Friedman M. Unpublished doctoral dissertation. Boston: University of Massachusetts; 2005. The facing and coping together (FACT) program: A brief psychoeducation program for relatives’ of individuals with schizophrenia. [Google Scholar]
- Fristad MA, Gavazzi SM, Mackinaw-Koons B. Family psychoeducation: An adjunctive intervention for children with bipolar disorder. Biological Psychiatry. 2003;53:1000–1008. doi: 10.1016/s0006-3223(03)00186-0. [DOI] [PubMed] [Google Scholar]
- Fristad MA, Goldberg-Arnold JS, Gavazzi SM. Multi-family psychoeducation groups in the treatment of children with mood disorders. Journal of Marital and Family Therapy. 2003;29:491–504. doi: 10.1111/j.1752-0606.2003.tb01691.x. [DOI] [PubMed] [Google Scholar]
- Gavazzi SM, Fristad MA, Law JC. The Understanding Mood Disorders Questionnaire. Psychological Reports. 1997;81:172–174. doi: 10.2466/pr0.1997.81.1.172. [DOI] [PubMed] [Google Scholar]
- Glueckaf RL, Liss HJ, McQuillen DE, Webb PM, Dairaghi J, Carter CB. Therapeutic alliance in family therapy for adolescents with epilepsy: An exploratory study. The American Journal of Family Therapy. 2002;30:125–139. [Google Scholar]
- Heikkila J, Karlsson H, Taiminen T, Lauerma H, Ilonen T, Leinonen KM, et al. Expressed emotion is not associated with disorder severity in first-episode mental disorder. Psychiatry Research. 2002;111:155–165. doi: 10.1016/s0165-1781(02)00134-8. [DOI] [PubMed] [Google Scholar]
- Honig A, Hofman A, Rozendaal N, Dingemans P. Psychoeducation in bipolar disorder: Effect on expressed emotion. Psychiatry Research. 1997;72:17–22. doi: 10.1016/s0165-1781(97)00072-3. [DOI] [PubMed] [Google Scholar]
- Hooley JM, Campbell C. Control and controllability: Beliefs and behaviour in high and low expressed emotion relatives. Psychological Medicine. 2002;32:1091–1099. doi: 10.1017/s0033291702005779. [DOI] [PubMed] [Google Scholar]
- Hooley JM, Licht DM. Expressed emotion and causal attributions in the spouses of depressed patients. Journal of Abnormal Psychology. 1997;106:298–306. doi: 10.1037//0021-843x.106.2.298. [DOI] [PubMed] [Google Scholar]
- Hooley JM, Parker HA. Measuring expressed emotion: An evaluation of the shortcuts. Journal of Family Psychology. 2006;20:386–396. doi: 10.1037/0893-3200.20.3.386. [DOI] [PubMed] [Google Scholar]
- Hooley JM, Rosen LR, Richters JE. Expressed emotion: Toward clarification of a critical construct. In: Miller GA, editor. The behavioral high-risk paradigm in psychopathology. New York: Springer; 1995. pp. 88–120. [Google Scholar]
- Horvath AO. Empirical validation of Bordin’s pantheoretical model of the alliance: The Working Alliance Inventory perspective. In: Horvath AO, Greenberg LS, editors. The working alliance: Theory, research, and practice. New York: Wiley; 1994. pp. 109–128. [Google Scholar]
- Horvath AO, Greenberg LS. The development of the Working Alliance Inventory. In: Greenberg LS, Pinsof WM, editors. The psychotherapeutic process: A research handbook. New York: Guilford; 1986. pp. 529–556. [Google Scholar]
- Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36:223–233. [Google Scholar]
- Jacobson NS, Christensen A, Prince SE, Cordova J, Eldridge K. Integrative behavioral couple therapy: An acceptance-based, promising new treatment for couple discord. Journal of Consulting and Clinical Psychology. 2000;68:351–355. doi: 10.1037//0022-006x.68.2.351. [DOI] [PubMed] [Google Scholar]
- Keltner D, Ellsworth PC, Edwards K. Beyond simple pessimism: Effects of sadness and anger on social perception. Journal of Personality and Social Psychology. 1993;64:740–752. doi: 10.1037//0022-3514.64.5.740. [DOI] [PubMed] [Google Scholar]
- Kim EY, Miklowitz DJ. Expressed emotion as a predictor of outcome among bipolar patients undergoing family therapy. Journal of Affective Disorders. 2004;82:343–352. doi: 10.1016/j.jad.2004.02.004. [DOI] [PubMed] [Google Scholar]
- Lefley HP. Family caregiving in mental illness. Thousand Oaks, CA: Sage Publications; 1996. [Google Scholar]
- Lerner JS, Keltner D. Fear, anger, and risk. Journal of Personality and Social Psychology. 2001;81:146–159. doi: 10.1037//0022-3514.81.1.146. [DOI] [PubMed] [Google Scholar]
- Linszen D, Dingemans P, Van Der Does JW, Nugter A, Scholte P, Lenior R, et al. Treatment, expressed emotion, and relapse in recent onset schizophrenic disorders. Psychological Medicine. 1996;26:333–342. doi: 10.1017/s0033291700034723. [DOI] [PubMed] [Google Scholar]
- Little RJA, Rubin DB. Statistical analysis with missing data. New York: Wiley; 1987. [Google Scholar]
- Magaña AB, Goldstein MJ, Karno M, Miklowitz DJ. A brief method for assessing expressed emotion in relatives of psychiatric patients. Psychiatry Research. 1986;17:203–212. doi: 10.1016/0165-1781(86)90049-1. [DOI] [PubMed] [Google Scholar]
- McAuley E, Duncan TE, Russell DW. Measuring causal attributions: The revised Causal Dimension Scale (CDSII) Personality and Social Psychology Bulletin. 1992;18:566–573. [Google Scholar]
- McFarlane WR, Dixon L, Lukens E, Lucksted A. Family psychoeducation and schizophrenia: A review of the literature. Journal of Marital and Family Therapy. 2003;29:223–245. doi: 10.1111/j.1752-0606.2003.tb01202.x. [DOI] [PubMed] [Google Scholar]
- Miklowitz DJ. The role of family systems in severe and recurrent psychiatric disorders: A developmental psychopathology view. Development and Psychopathology. 2004;16:667–688. doi: 10.1017/s0954579404004729. [DOI] [PubMed] [Google Scholar]
- Miklowitz DJ, Goldstein MJ. Bipolar disorder: A family-focused treatment approach. New York: Guilford Press; 1997. [Google Scholar]
- Moltz DA, Newmark M. Multifamily groups for bipolar illness. In: McFarlane WR, editor. Multifamily groups in the treatment of severe psychiatric disorders. New York: Guilford Press; 2002. pp. 220–243. [Google Scholar]
- Nguyen TD, Attkisson CC, Stegner BL. Assessment of patient satisfaction: Development and refinement of a service evaluation questionnaire. Evaluation and Program Planning. 1983;6:299–313. doi: 10.1016/0149-7189(83)90010-1. [DOI] [PubMed] [Google Scholar]
- Ogilvie AD, Morant N, Goodwin GM. The burden on informal caregivers of people with bipolar disorder. Bipolar Disorders. 2005;7:25–32. doi: 10.1111/j.1399-5618.2005.00191.x. [DOI] [PubMed] [Google Scholar]
- Perlick DA, Hohenstein JM, Clarkin JF, Kaczynski R, Rosenheck RA. Use of mental health and primary care services by caregivers of patients with bipolar disorder: A preliminary study. Bipolar Disorders. 2005;7:126–135. doi: 10.1111/j.1399-5618.2004.00172.x. [DOI] [PubMed] [Google Scholar]
- Pitschel-Walz G, Leucht S, Baeuml J, Kissling W, Engel RR. The effect of family interventions on relapse and rehospitalization in schizophrenia: A meta-analysis. Schizophrenia Bulletin. 2001;27:73–92. doi: 10.1093/oxfordjournals.schbul.a006861. [DOI] [PubMed] [Google Scholar]
- Poling K, Brent DA, Birmaher B. Understanding and coping with bipolar illness: A survival manual for families. Pittsburgh: University of Pittsburgh, Western Psychiatric Institute and Clinic; 1999. [Google Scholar]
- Reilly JW, Rohrbaugh M, Lackner JM. A controlled evaluation of psychoeducation workshops for relatives of state hospital patients. Journal of Marital and Family Therapy. 1988;14:429–432. [Google Scholar]
- Rein Z, Perdereau F, Curt F, Jeammet P, Fermanian J, Godart N. Expressed emotion and anorexia nervosa: The validation of the Five Minute Speech Sample in reference to the Camberwell Family Interview. International Journal of Eating Disorders. 2006;39:217–223. doi: 10.1002/eat.20245. [DOI] [PubMed] [Google Scholar]
- Reinares M, Vieta E, Colom F, Martinez-Aran A, Torrent C, Comes M, et al. Impact of a psychoeducational family intervention on caregivers of stabilized bipolar patients. Psychotherapy and Psychosomatics. 2004;73:312–319. doi: 10.1159/000078848. [DOI] [PubMed] [Google Scholar]
- Rigsby-Jones J, Zelonis P, Morrison F, Bland R, Harrison C, Hammer A. Bipolar disorder family support program. Brisbane, Queensland: Princess Alexandra Hospital; 1994. [Google Scholar]
- Shimodera S, Mino Y, Fujita H, Izumoto Y, Kamimura N, Inoue S. Validity of a five-minute speech sample for the measurement of expressed emotion in the families of Japanese patients with mood disorders. Psychiatry Research. 2002;112:231–237. doi: 10.1016/s0165-1781(02)00242-1. [DOI] [PubMed] [Google Scholar]
- Siemer M. Mood congruent cognitions constitute mood experience. Emotion. 2005;5:296–308. doi: 10.1037/1528-3542.5.3.296. [DOI] [PubMed] [Google Scholar]
- Simoneau TL, Miklowitz DJ, Richards JA, Saleem R, George EL. Bipolar disorder and family communication: Effects of a psychoeducational treatment program. Journal of Abnormal Psychology. 1999;108:588–597. doi: 10.1037//0021-843x.108.4.588. [DOI] [PubMed] [Google Scholar]
- Smith CA, Ellsworth PC. Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology. 1985;48:813–838. [PubMed] [Google Scholar]
- Solomon P, Draine J, Mannion E, Meisel M. Impact of brief family psychoeducation on self-efficacy. Schizophrenia Bulletin. 1996;22:41–50. doi: 10.1093/schbul/22.1.41. [DOI] [PubMed] [Google Scholar]
- Spielberger CD. Professional Manual for the State-Trait Anger Expression Inventory (Research edition) Odessa, FL: Psychological Assessment Resources; 1988. [Google Scholar]
- Spielberger CD, Sydeman SJ. State-Trait Anxiety Inventory and State-Trait Anger Expression Inventory. In: Maruish ME, editor. The use of psychological testing for treatment planning and outcome assessment. Hillsdale, NJ: Lawrence Erlbaum; 1994. pp. 292–301. [Google Scholar]
- van Gent EM, Zwart FM. Psychoeducation of partners of bipolar-manic patients. Journal of Affective Disorders. 1991;21:15–18. doi: 10.1016/0165-0327(91)90013-i. [DOI] [PubMed] [Google Scholar]
- Van Humbeeck G, Van Audenhove C, De Hert M, Pieters G, Storms G. Expressed emotion: A review of assessment instruments. Clinical Psychology Review. 2002;22:321–341. doi: 10.1016/s0272-7358(01)00098-8. [DOI] [PubMed] [Google Scholar]
- Weiner B. Judgments of responsibility: A foundation for a theory of social conduct. New York: Guilford Press; 1995. [Google Scholar]
- Weisman AG, Lopez S. An attributional analysis of emotional reactions to schizophrenia in Mexican and Anglo American cultures. Journal of Applied Social Psychology. 1997;27:223–244. [Google Scholar]
- Weisman AG, Okazaki S, Gregory J, Goldstein MJ, Tompson MC, Rea M, et al. Evaluating therapist competency and adherence to behavioral family management with bipolar patients. Family Process. 1998;37:107–121. doi: 10.1111/j.1545-5300.1998.00107.x. [DOI] [PubMed] [Google Scholar]
- Wendel JS, Miklowitz DJ, Richards JA, George EL. Expressed emotion and attributions in the relatives of bipolar patients: An analysis of problem-solving interactions. Journal of Abnormal Psychology. 2000;109:792–796. doi: 10.1037//0021-843x.109.4.792. [DOI] [PubMed] [Google Scholar]
- Yan LJ, Hammen C, Cohen AN, Daley SE, Henry RM. Expressed emotion versus relationship quality variables in the prediction of recurrence in bipolar patients. Journal of Affective Disorders. 2004;83:199–206. doi: 10.1016/j.jad.2004.08.006. [DOI] [PubMed] [Google Scholar]