Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Apr 25.
Published in final edited form as: J Clin Psychol. 2009 Mar;65(3):319–335. doi: 10.1002/jclp.20547

Coping Mediates Outcome Following a Randomized Group Intervention for HIV-Positive Bereaved Individuals

Nathan Grant Smith 1, Nalini Tarakeshwar 2, Nathan B Hansen 3, Arlene Kochman 4, Kathleen J Sikkema 5
PMCID: PMC3636308  NIHMSID: NIHMS456609  PMID: 19152338

Abstract

The purpose of this study was to examine the mechanisms responsible for the beneficial psychological effects of a coping-focused group intervention for HIV-positive individuals who had lost loved ones to AIDS. Data from 235 HIV-positive men and women enrolled in a randomized controlled clinical trial testing a coping-focused group intervention were analyzed using a multiple-indicator-multiple-cause (MIMIC) structural equation model. Results revealed that the effects of the intervention on decreases in depression and grief were mediated by decreases in avoidant coping. Specifically, participants in the intervention condition decreased their use of avoidant coping. Decreases in avoidant coping, in turn, were related to decreased depression and grief. The results of this study help to validate the use of coping-focused interventions for HIV-positive bereaved individuals.

Keywords: HIV, AIDS, coping, bereavement, group psychotherapy, structural equation modeling

Introduction

Based on decades of research on coping, researchers maintain that how people cope with stress reduces or amplifies the effects of adverse life events on both short-term and long-term emotional distress and functioning (Skinner, Edge, Altman, & Sherwood, 2003). Among individuals with HIV, AIDS-related bereavement and multiple losses that occur over brief periods are a relatively common source of stress (Martin, 1988; Sikkema, Kochman, DiFranceisco, Kelly, & Hoffman, 2003) and subsequent mental health problems. A number of coping-focused group interventions using a cognitive–behavioral framework has been implemented with HIV-positive individuals, primarily among gay men (e.g., Antoni et al., 2000; Chesney, Folkman, & Chambers, 1996). Studies suggest that these interventions are associated with positive psychological outcomes for participants. Notably, few intervention studies have focused specifically on AIDS-related bereavement (Goodkin et al., 1999; Sikkema, Hansen, Kochman, Tate, & DiFranceisco, 2004). Likewise, few studies that focused on nonbereaved HIV-positive individuals have examined the underlying psychological mechanisms that are arguably responsible for the observed positive outcomes. Moreover, no studies focused on bereaved HIV-positive individuals have examined these underlying psychological mechanisms. However, recent findings suggest that use of specific coping strategies, such as cognitive coping and denial, are related to quality of life among HIV-positive individuals in general (Leiberich et al., 1997; Swindells et al., 1999; Vosvick et al., 2002; Weaver et al., 2004). The purpose of this study was to (a) identify the specific changes in coping that occur among HIV-positive men and women enrolled in a coping-focused intervention for AIDS-related bereavement, (b) examine how these changes in coping are related to changes in grief and depression, and (c) examine whether changes in coping mediate the effectiveness of the intervention.

Background

According to Lazarus and Folkman (1984), coping is process-oriented and contextual. Specifically, coping is viewed as a process involving an individual’s appraisals of an event and appraisals of his or her available resources. If these appraisals result in heightened negative emotion, then the person will choose some sort of response to diminish the negative emotion (i.e., coping strategy or response). Two dimensions of coping that are often used to categorize the coping responses of individuals are problem-focused versus emotion-focused coping and active versus avoidant coping. Problem-focused coping involves attempts to modify or eliminate sources of stress through one’s behavior; emotion-focused coping includes behavioral or cognitive responses to manage the emotional consequences of stressors (Lazarus & Folkman, 1984). Active coping includes attempts to actively confront the stressful situation whereas avoidant coping represents cognitions or behaviors aimed at avoiding the stressful situation. Conclusions on the effectiveness of each of these types of coping are not unanimous and depend on the nature of the stressor, its controllability, and duration of coping efforts required in dealing with it (Lazarus, 1999).

In general, studies on stress and coping report that coping strategies in which an individual is actively engaged in managing the stressful situation are associated with better psychological outcomes (e.g., Billings & Moos, 1984; Hobfoll, Dunahoo, Ben-Porath, & Monnier, 1994; Hovanitz & Kozora, 1990; Kershaw, Northouse, Kritpracha, Schafenacker, & Mood, 2004). Examples of such strategies include finding alternative solutions to the problem, rediscovering alternative sources of significance, positive reframing, and seeking emotional support. In contrast, greater use of strategies wherein an individual engages in efforts to escape or avoid dealing with the stressful situation has been associated with poorer psychological outcomes. Strategies in this category include wishful thinking, denial, passive acceptance of the situation, and derogatory thinking.

Coping With AIDS-Related Bereavement

Bereavement in general represents a stressful life event. AIDS-related bereavement among persons with HIV, however, is unique for several reasons. First, AIDS continues to be a stigmatized disease and disproportionately affects disenfranchised groups (e.g., gay and bisexual men, women of color; Centers for Disease Control and Prevention, 2002). As such, the loss of a loved one to AIDS is complicated by the societal context wherein the loss occurs (Lund & Caserta, 1998). This type of grief has been termed “disenfranchised grief” (Doka, 1989) because the loss “cannot be openly acknowledged, publicly mourned, or socially supported” (p. 4). Second, in addition to the stigma associated with AIDS-related bereavement, persons with HIV may experience multiple losses. Research on the impact of number of losses on adjustment has been equivocal, with some studies finding a positive relationship between number of losses and depression (Dean, 1995; Martin, 1988) and other studies finding no relationship (Folkman, Chesney, Collette, Boccellari, & Cooke, 1996; Martin & Dean, 1993; Neugebauer et al., 1992). Even in the era of highly active antiretroviral treatment (HAART), there is still no cure for AIDS, so AIDS-related losses continue to be a stressor for individuals affected by HIV. Despite uncertainty over the effect of multiple losses, persons with HIV may have more bereavement-related stress with which to contend than persons who are not HIV-positive. Finally, AIDS-related bereavement may affect the person with HIV by highlighting his or her own mortality in conjunction with a life-threatening illness.

Within the context of AIDS-related bereavement, as with other stressors, psychological adjustment depends on the individual’s choice of coping strategies. A few studies on AIDS-related bereavement yield support for the distinct relationships between an individual’s choice of coping strategies (whether active or avoidant) and psychological adjustment. For instance, active coping strategies such as positive reappraisal, engaging in problem-solving and setting goals, engaging in spiritual activities, and subscribing meaning to events are related to positive immunologic and health outcomes (Bower, Kemeny, Taylor, & Fahey, 1998; Folkman, 1997; Hansen et al., 2006). In contrast, avoidant strategies such as emotional suppression, self-blaming, wishful thinking, and denial are related to poorer psychological outcomes (i.e., greater grief and depressed mood) postbereavement (Folkman et al., 1996; Hansen et al.; Ingram, Jones, & Smith, 2001; Sikkema et al., 2000).

These types of findings tend to be consistent among studies examining HIV-positive individuals who are dealing with a variety of stressors: There is growing support for the link between active coping and positive outcomes, as well as the link between avoidant coping and negative outcomes. Among persons with HIV, Vosvick and colleagues (2002) found that mental disengagement, behavioral disengagement, denial, and venting were related to negative health outcomes, such as increased health distress, cognitive problems, and decreased mental health.

Bereavement Interventions

A number of interventions have been targeted toward persons who have lost loved ones. Most aim at facilitating normal grieving (e.g., reviewing the lost relationship, mourning the deceased) through a process that is consistent with a specific psychological theory or perspective, such as cognitive–behavioral, psychoanalytic theory, or attachment theory (Raphael, Minkov, & Dobson, 2001). Research on the effectiveness of bereavement interventions has revealed mixed results, with some studies demonstrating positive outcomes (e.g., Horowitz, Marmar, Weiss, DeWitt, & Rosenbaum, 1984; Yalom & Vinogradov, 1988) and others failing to demonstrate positive outcomes (see Jordan & Neimeyer, 2003 for a review). The reasons for these contradictory findings may be attributable to the type of population targeted for intervention. Studies have indicated that bereavement interventions for persons experiencing noncomplicated grief are not effective and may actually be harmful (Jordan & Neimeyer, 2003; Neimeyer, 2000). However, interventions for those at risk of or currently experiencing complicated grief have been shown to be effective at lowering distress (Center for the Advancement of Health, 2004; Schut, Stroebe, Van Den Bout, & Terheggen, 2001). Unlike interventions for HIV-positive individuals, it is unclear the exact reasons why bereavement interventions are effective, whether it is because of an increase in actively confronting the loss, the perceived social support derived in belonging to a support group, the passage of time, and/or other variables. To date, little empirical data are available that explicate how bereavement interventions facilitate the grieving process.

A small number of group bereavement interventions have been developed specifically for HIV-positive individuals, primarily among gay men, and most have used stress and coping theory as a heuristic framework. Goodkin et al. (1999) conducted a randomized controlled trial evaluating the efficacy of a 10-session bereavement support group intervention with both HIV-positive and HIV-negative gay men who had recently lost a close friend or partner to AIDS. Participants in the intervention condition showed significantly reduced psychological distress and accelerated grief reduction as compared with participants in the control group, who did not receive an intervention beyond the level of psychosocial and medical care used prior to baseline. However, as with interventions for bereavement in general, interventions for HIV-positive bereaved individuals have not been validated to determine the specific psychological mechanisms, such as coping behaviors, that are assumed to change as a result of the intervention.

Although it is unclear why AIDS-related bereavement interventions are effective, research on interventions for nonbereaved HIV-positive individuals has shed light onto possible mechanisms of change. A number of interventions has been developed to assist persons, mostly gay men, to cope with the stresses associated with an HIV-positive diagnosis. Research into the effects of cognitive–behavioral stress-management group interventions for HIV-positive gay men has revealed that these interventions lead to positive mental health outcomes, increases in active coping, and decreases in avoidant coping. These changes in coping mediated the effects of the interventions on mental health outcomes (Carrico et al., 2006; Carrico, Antoni, Weaver, Lechner, & Schnedierman, 2005; Cruess et al., 2002; Lutgendorf et al., 1998). Similar results were found for a group intervention focused specifically on coping-effectiveness among HIV-positive gay men. Chesney, Chambers, Taylor, Johnson, and Folkman (2003) found that those in the intervention group evidenced decreased burnout and stress and that those effects were mediated by increases in coping self-efficacy. It is not yet known whether group interventions specifically focused on AIDS-related bereavement, or interventions provided to samples diverse in terms of sex and sexual orientation, will evidence similar results.

The current intervention was based on a cognitive–behavioral model that combined elements of cognitive theory of stress and coping (Folkman et al., 1991; Lazarus & Folkman, 1984) with models of coping with grief and bereavement (Rando, 1984; Worden, 1991) for men and women with HIV who were coping with AIDS-related bereavement. In an evaluation of efficacy (Sikkema et al., 2004), participants were randomly assigned to a 12-week cognitive–behavioral group intervention or to an individual therapy on request comparison condition. Analyses indicated that participants in the group intervention had significant reductions in overall psychiatric distress compared to controls. Further, women in the group intervention demonstrated greater reductions in grief and depression than all other participants.

Given the effectiveness of the intervention, the purpose of the current study was to test the validity of the model by examining the impact of the intervention on the coping strategies used by individuals in the two conditions. Specifically, we wanted to determine if the psychological benefits associated with attending the cognitive–behavioral group intervention were due to modification in participants’ coping patterns, i.e. whether they exhibited an increase in active coping strategies, decrease in avoidant coping strategies, or both. We hypothesized that, relative to HIV-infected participants assigned to the comparison condition, HIV-infected participants in the AIDS-related bereavement coping intervention would exhibit at the postintervention assessment: (a) greater use of active coping strategies, and (b) less use of avoidant coping strategies. Further, we predicted that at postintervention assessment: (a) greater use of active coping strategies would be related to decreased severity of grief reaction and fewer symptoms of depression, and (b) greater use of avoidant coping strategies would be related to increased severity of grief reaction and more symptoms of depression.

Method

Participants

Participants included 235 HIV-positive men (n = 150, 64%) and women (n = 85, 36%) who completed baseline and posttest assessments of a randomized clinical trial. The mean age of participants was 40 years (range = 21–60 years). Approximately half of participants were African American (53%, n = 125). Twenty-eight percent (n = 66) were Caucasian; 13% (n = 30) were Latino/a; and 6% (n = 14) were of other racial/ethnic groups. Of the male participants, 67% (n = 101) identified as gay; 21% (n = 32) identified as heterosexual; and 11% (n = 17) identified as bisexual. Of the female participants, 92% (n = 78) identified as heterosexual; 7% (n = 6) identified as bisexual; and 1% (n = 1) identified as lesbian. The mean time since being diagnosed with HIV was 86.6 months (SD = 48) and 45% of the sample reported having an AIDS diagnosis.

Procedure

Participants were recruited in New York, NY and Milwaukee, WI through printed media and contacts with AIDS service organizations and health care providers. Interested HIV-positive individuals who had lost a loved one to AIDS were instructed to call for an appointment. Participants underwent a two-part screening interview to determine eligibility. Inclusion criteria included (a) health care provider verification of HIV-positive serostatus, (b) loss of a loved one to AIDS not less than 1 month or more than 2 years ago, (c) not currently psychotic, and (d) no more than mild cognitive impairment. Participants who met the eligibility criteria then completed a baseline assessment. Following baseline assessment, participants were randomly assigned to a comparison or intervention condition. Participants were randomized via mood, anxiety, and/or substance abuse disorder stratification. Initially, participants had an equal chance of being randomized to either condition. However, to ensure adequate numbers of intervention participants, randomization was changed mid-way such that there were two participants randomized to the intervention group for each participant randomized to the comparison group. See Figure 1 for a description of the progress of participants through each stage of the clinical trial.

Figure 1.

Figure 1

The progress of participants through each stage of the clinical trial.

Participants in the comparison condition were offered individual therapy upon request; participants in the intervention condition received the coping-focused group intervention. Services provided to participants in the comparison condition approximated to what clients would receive at a community mental health center. These services were provided by masters- and doctoral-level therapists not trained in the intervention. Because not all participants in the comparison condition requested individual services (29% of those in the comparison condition received services through the study), these services were not standardized. Whereas a no-treatment control group would have provided the optimal setting in which to evaluate the intervention, the community-standard of care (i.e., individual services upon request) was offered for ethical reasons. All participants in the comparison condition were offered the group intervention after the completion of the research project.

Participants in the intervention condition were enrolled into a 12-week cognitive–behavioral coping intervention group that focused on coping with loss and being HIV-positive. The groups included 8–10 participants each and were led by two trained masters- or doctoral-level therapists. The semistructured 90-minute group sessions focused on a number of themes, including development of social support, expression of loss due to bereavement and participants’ own experiences of living with HIV, identification of stressors and coping strategies, and development and implementation of adaptive coping strategies. Group leaders assisted participants in evaluating the adaptiveness of their current coping strategies. Participants then identified short- and long-term goals related to coping with bereavement. Finally, participants rehearsed adaptive coping strategies in an effort to reduce distress. The intervention focused on increasing participants’ sense of control, expressing and managing anger, decreasing guilt, encouraging empowerment, and establishing healthy relationships. Group facilitators followed a structured protocol, but tailored their activities to the specific needs and dynamics of each group (e.g., focusing on children and parenting among women, focusing on dual stigma among gay and bisexual men, focusing on male gender role issues among heterosexual men).

Following the completion of the group sessions, participants in both the comparison and intervention conditions returned to complete posttest assessments that included the same measures as the baseline assessments. Participants received monetary compensation for their involvement in the research project. Participants provided written informed consent. An institutional review board approved all study procedures prior to data collection. For more information on the study design and intervention content, see Sikkema et al. (2004).

Measures

Because the analyses were conducted using structural equation modeling, the measures section will be organized by construct, rather than by scale. In addition, each construct had three to four indicators that were formed by using the scale score, subscale score, or parcels of a scale.

Coping measures

Coping was measured using the Ways of Coping Questionnaire (WCQ; Folkman & Lazarus, 1988) and the Coping with Illness scale (CWI; modified from the Dealing with Illness Inventory; Namir, Wolcott, Fawzy, & Alumbaugh, 1987). The WCQ is a 66-item questionnaire that measures coping behaviors pertaining to a particular stressor (in this case, AIDS-related loss). The CWI is a 31-item scale that measures coping behaviors pertaining to illness (in this case, being HIV-positive).

The WCQ includes eight subscales; however, the eight factors of the WCQ tend to be unstable across different samples and different stressors (van Heck & de Ridder, 2001). For this reason, and because this sample had unique sample characteristics and context-specific stressors, the WCQ and the CWI items were combined and factor analyzed. A principal components factor analysis with varimax rotation was computed to identify stressor-specific coping constructs in this sample. The factor analysis revealed that a 5-factor solution provided the best fit to the data. The five factors were labeled Active Coping, Avoidant Coping, Self-destructive Behavior, Seeking Social Support, and Spiritual Coping (for more information on the results of the factor analysis, see Tarakeshwar, Hansen, Kochman, & Sikkema, 2005). Given the emphasis in the literature of focusing on active versus avoidant coping, for the present study, only the Active and Avoidant Coping factors were utilized. These factors also represented the most stable factors in the 5-factor solution, and allowed for a clearer estimation of adaptive versus maladaptive coping strategies. The Active Coping construct contained 29 items and reflected problem-focused coping strategies or strategies wherein the individual was engaged in reflecting about the stressor (e.g., “I just concentrated on what I had to do next,” “Thought more about the meaning of life”); the Avoidant Coping construct contained 16 items and reflected behavioral or cognitive avoidance coping strategies (e.g., “I refused to believe that it had happened,” “I criticized or lectured myself”). In the current study, Cronbach’s alphas were .93 for Time 1 Active Coping, 0.94 for Time 2 Active Coping, 0.91 for Time 1 Avoidant Coping, and 0.92 for Time 2 Avoidant Coping.

To form the indicators for the Active and Avoidant Coping constructs, each subscale was separately factor analyzed, extracting only one factor. The factor loadings for the items on each subscale (either Active or Avoidant Coping) were examined and four parcels were formed for each construct. The four items with the highest factor loadings served as the anchors for each parcel. Subsequent items were added to the four anchors, such that items with high and low factor loadings were distributed equally across all parcels (based on recommendations by Hall, Snell, & Foust, 1999).

Grief symptoms

Grief symptoms were measured using the Grief Reaction Index (GRI; Lennon, Martin, & Dean, 1990). The GRI is a 12-item scale that assesses common grief symptoms such as numbness, denial, and intrusive thoughts. In the current study, Cronbach’s alphas for the GRI were 0.88 for Time 1 and 0.90 for Time 2. To form the indicators for grief symptoms, the GRI was factor analyzed and three parcels were created using the methods described above, consistent with Hall et al. (1999).

Depressive symptoms measures

Depression was measured using the Symptom Checklist-90-Revised (SCL-90-R; Derogatis, 1983), the Structured Interview Guide for the Hamilton Depression and Anxiety Scales (SIGH-AD; Williams, 1988), and the Beck Depression Inventory (BDI; Beck & Steer, 1987). The SCL-90-R is a 90-item scale designed to assess global psychiatric distress, as well as specific types of distress symptoms. The SCL-90-R contains a global severity index and nine specific subscales including depression, somatization, anxiety, obsessive–compulsive, interpersonal sensitivity, hostility, paranoid ideation, psychoticism, and phobic anxiety. For the current study, only the depression subscale of the SCL-90-R was used. The SCL-90-R depression subscale yielded Cronbach’s alphas of 0.93 for both Time 1 and Time 2. The SIGH-AD contains both the Hamilton Depression Scale (HAM-D) and the Hamilton Anxiety Scale (HAM-A). The HAM-D contains 17 items and was designed to measure depressive symptoms. For the current study, only the HAM-D was used, which yielded Cronbach’s alphas of 0.85 for both Time 1 and Time 2. The BDI is a 21-item scale designed to measure depressive symptoms. The BDI contains a cognitive–affective subscale and a somatic–performance subscale. Because HIV-related physical problems can be confounded with the somatic– performance subscale, only the cognitive–affective subscale was used in the current study. Cronbach’s alphas for the cognitive–affective subscale of the BDI were 0.88 for Time 1 and 0.91 for Time 2. The construct of depression was defined by using the depression subscale of the SCL-90-R, the HAM-D, and the cognitive–affective subscale of the BDI as the three indicators.

Results

All estimation of model parameters and fit statistics were performed using full information maximum likelihood in LISREL 8.72 (Jöreskog & Sörbom, 2005). We considered comparative fit index (CFI) greater than 0.95, root-mean-square error of approximation (RMSEA) less than 0.06, and standardized root-mean-square residual (SRMR) less than 0.08 as indicative of adequate model fit (Hu & Bentler, 1999). The analysis was intention-to-treat; therefore data from all participants who completed the baseline and posttest assessments were analyzed.

Measurement Model

The measurement model provided an excellent fit to the data χ2, (71) = 78.78, ns; χ2/df = 1.11; RMSEA = 0.02, 90% CI = 0.00, .05; CFI = 0.99; SRMR = 0.05. All parcel and scale indicators significantly loaded on their respective latent variables (see Table 1). These findings indicate that the indicators adequately measured the latent variables.

Table 1.

Factor Loadings for Latent Variables.

Variable Unstandardized factor loading SE z-score Standardized factor loading
Change in active coping
Parcel 1 0.47 0.03 13.77 0.78
Parcel 2 0.46 0.03 15.16 0.84
Parcel 3 0.45 0.03 14.64 0.82
Parcel 4 0.52 0.03 14.43 0.81
Change in avoidant coping
Parcel 1 0.44 0.04 11.88 0.72
Parcel 2 0.45 0.04 12.21 0.74
Parcel 3 0.44 0.04 11.69 0.71
Parcel 4 0.44 0.04 12.96 0.77
Change in depression
HAM-D 3.72 0.49 7.61 0.56
SCL-90-R 0.43 0.05 9.15 0.68
BDI 3.51 0.44 8.03 0.59
Change in grief
Parcel 1 2.26 0.19 11.72 0.73
Parcel 2 2.62 0.22 11.80 0.73
Parcel 3 3.28 0.23 14.55 0.87

Note: N = 235. HAM-D = Hamilton Depression Scale; SCL-90-R = Symptom Checklist-90-Revised; BDI = Beck Depression Inventory. All factors loadings are significant at p<0.001.

Structural Model

To examine the effect of treatment on changes in avoidant coping, changes in active coping, changes in depression, and changes in grief, we used a MIMIC (multiple indicator-multiple cause) model. A MIMIC model allows latent variables to be identified by both endogenous item indicators and exogenous background variables, such as treatment (Mast & Lichtenberg, 2000). The addition of background variables allows for the examination of population heterogeneity (Muthén, Tam, Muthén, Stolzenberg, & Hollis, 1993); that is, differences among the intervention and comparison groups can be assessed. Simply put, a MIMIC model allows a measured variable to serve as an exogenous (independent) variable that predicts a latent endogenous (dependent) variable.

To account for participants’ baseline levels of active and avoidant coping, depression and grief, change scores were computed. Specifically, to account for differences between scores at Time 1 and Time 2, for each latent variable, each participant’s Time 1 parcel score was subtracted from his or her corresponding Time 2 parcel score to provide a difference score for each indicator. These difference indicators were used to measure latent changes from Time 1 to Time 2.

The model provided a good fit to the data, χ2 (83) = 127.33, p<0.01; χ2/df = 1.53; CFI = 0.97; RMSEA = 0.05, 90% CI = 0.03, 0.06; SRMR = 0.13. Although the SRMR was above 0.08, the CFI was above 0.96; Hu and Bentler (1999) note that when the SRMR is above 0.09 or 0.10, a CFI value above 0.96 will result in adequate model fit. The model, with standardized path coefficients indicated (with unstandardized coefficients followed by standard errors in parentheses), is displayed in Figure 2. Treatment significantly and negatively predicted changes in avoidant coping, indicating that the treatment group decreased their use of avoidant coping significantly more than the control group. Changes in avoidant coping, were related, in turn, to changes in grief, such that decreases in avoidant coping predicted decreases in grief. Likewise, changes in avoidant coping related to changes in depression, such that decreases in avoidant coping predicted decreases in depression. Though treatment did not have a significant effect on changes in active coping, decreases in active coping significantly predicted decreases in depression.

Figure 2.

Figure 2

The model with standardized path coefficients indicated (with unstandardized coefficients followed by standard errors in parentheses) is displayed.

To test whether changes in avoidant coping mediated the effect of treatment on changes in depression and grief, two Sobel (1982) tests were conducted. Specifically, the Sobel test was calculated using the formula

z=babb(ba2seb2+bb2sea2)

where ba refers to the path coefficient for the effect of treatment on changes in avoidant coping, bb refers to the path coefficient for the effect of changes in avoidant coping on changes in depression (or grief), and se refers to their respective standard errors. The Sobel test for depression was significant, z = −2.31, p<0.05, indicating that changes in avoidant coping mediated the effect of treatment on changes in depression. The Sobel test for grief was also significant, z = −2.24, p<0.05, indicating that changes in avoidant coping mediated the effect of treatment on changes in grief. Note that the unstandardized path coefficients displayed in Figure 2 have been rounded to two decimal places; the above z-tests were calculated before rounding so the results will be slightly different.

Discussion

The purpose of this study was to examine the psychological mechanisms by which decreases in depression and grief are seen in HIV-positive bereaved individuals who participated in a coping-focused group intervention. Previous research found that this coping-focused intervention was effective in decreasing depression and grief among this population (Sikkema et al., 2004). Because the group intervention focused on developing adaptive coping behaviors, we hypothesized that changes in coping strategies would account for the salubrious effects of the intervention. Results revealed that, indeed, changes in coping were related to decreased depression and grief in participants in the intervention condition. Specifically, participants in the intervention condition decreased their use of avoidant coping strategies. This decrease in use of avoidant coping was, in turn, related to decreased depression and grief. Moreover, the effect of the treatment on both depression and grief was mediated by change in avoidant coping. That is, the effect of the treatment on grief and depression was only seen through the effect of treatment on avoidant coping. Contrary to predictions, the intervention condition did not outperform the control condition in producing change in active coping.

There are a number of strengths of this study. First, the results of this study lend credence to the efficacy and validity of psychological interventions aimed at helping persons living with HIV cope with stressors. Although evidence exists concerning the efficacy of coping-focused group interventions for HIV-positive individuals in general (e.g., Antoni et al., 1991, 2000; Chesney et al., 1996; Kelly et al., 1993) and for HIV-positive bereaved individuals specifically (Goodkin et al., 1999; Sikkema et al., 2004), heretofore there has been little examination of why these interventions work, especially among bereaved HIV-positive persons. These interventions are aimed at changing coping strategies, and according to the theory of stress and coping (Lazarus & Folkman, 1984), the use of more adaptive coping strategies should lead to positive health outcomes. The results of this study provide verification of that hypothesis. Though the results cannot be generalized beyond the stressor of AIDS-related bereavement, this study provides some initial validation of the use of a theoretically based coping intervention for persons with HIV. Thus, this type of intervention may be beneficial to persons living with HIV who are dealing with a variety of stressors. Second, this study is one of the few to examine a group intervention for HIV-positive positive individuals who are diverse in terms of gender, race, and sexual orientation. As women of color now represent one of the fastest growing groups of persons newly diagnosed with AIDS (Centers for Disease Control and Prevention, 2002), it is important that interventions are targeted toward women of color and tailored to their specific needs.

These results are also consistent with previous research. Avoidant coping has been shown to predict psychological distress in HIV-positive individuals (e.g., Namir et al., 1987; Pakenham, Dadds, & Terry, 1994; Simoni & Ng, 2000). Moreover, avoidant coping strategies are more prevalent among HIV-positive members of disadvantaged groups, such as persons of color, intravenous drug users, women, and those with lower socioeconomic status (Fleishman & Fogel, 1994). The results of research on active coping among persons with HIV/AIDS, however, have been mixed (Simoni & Ng, 2000). Some studies have found positive associations between active coping and adjustment (e.g., Namir et al., 1987); others have not (e.g., Nicholson & Long, 1990). In addition to finding a positive relationship between changes in avoidant coping and changes in distress, the results of the current study indicate that avoidant coping is amenable to change via intervention.

The intervention did not produce significant changes in active coping. It may be possible that it is more difficult to develop new coping strategies (i.e., active) than it is to decrease the use of existing nonadaptive (i.e., avoidant) ones. With the myriad stressors encountered by persons living with HIV, adopting new behaviors may be too burdensome. However, with the recognition that current coping strategies are not helpful, it may be easier to abandon those strategies. Another possibility is that the adoption of new behaviors has a less certain outcome than the abandonment of current behaviors. That is, persons contemplating adding adaptive coping strategies to their behavioral repertoire may be unsure if these behaviors will be successful in promoting health. However, when persons are able to recognize that their current behaviors are health-compromising, they may be more confident that abandoning those behaviors will lead to a positive outcome. This explanation is consistent with social cognitive theory (Bandura, 1986), which asserts that persons are more likely to change their behaviors if they expect (via outcome expectations) that the behavior will result in a positive outcome. It may be more likely for HIV-positive bereaved individuals to expect positive outcomes from abandoning maladaptive coping strategies than from adopting new, more adaptive coping strategies. Finally, it is also likely that for many participants, with the removal of avoidant coping strategies, the active coping strategies being utilized were working, and new strategies were not needed.

There are several limitations to this study. First, the coping constructs were based on an exploratory factor analysis. Examining the factor structure of the avoidant and active coping constructs in a subsequent sample of bereaved HIV-positive individuals (via confirmatory factor analysis) would provide more evidence of validity. Second, the generalizability of the findings to bereaved HIV-positive individuals in cities outside of New York, NY and Milwaukee, WI is not known. Individuals in cities with fewer resources may have a more limited array of coping strategies from which to choose. As such, an intervention such as the one described here may have differential effects on changes in coping strategies. Finally, the analyses only included two time points; as such, a clear causal sequence between mediator and outcome, which is a requirement of mediation, has not been demonstrated (Cole & Maxwell, 2003).

Though this study focused on the reason why a coping-focused group intervention was effective, it did not examine the specific ways in which the intervention changed coping strategies. It is possible that specific intervention components (e.g., didactic, experiential, homework assignments), but not others, were effective in helping persons living with HIV/AIDS change their coping strategies. However, it is also possible that participants changed their coping strategies simply by being surrounded by supportive others. Future research should focus on identifying the specific components of the invention that are implicated in positive outcomes. For example, studies could examine the impact of the inclusion or exclusion of participants evaluating the adaptiveness of their current coping strategies. Likewise, to determine whether the effects were the result of simply being in a supportive group, future studies could compare the coping-focused group intervention to a nondirective support group. By comparing the coping-focused intervention to a nondirective support group, researchers can determine whether the intervention has positive effect above and beyond the effectiveness of general supportive group therapy.

In summary, this study was the first to examine the mechanisms by which a coping-focused intervention produced beneficial psychological outcomes among HIV-positive bereaved individuals. AIDS-related bereavement among HIV-positive persons represents a unique stressor, and helping patients to cope effectively should be an important goal of treatment providers and researchers. Coping-focused group interventions can be effective at improving psychological adjustment by helping patients to change their coping behaviors. These types of interventions, especially when tailored to meet the needs of diverse populations, can help persons dealing with AIDS-related bereavement avoid the negative psychological consequences of complicated grief.

Acknowledgments

This research was supported by grants RO1-MH54958, P30-MH62294, and T32- MH20031 from the National Institute of Mental Health. The authors gratefully acknowledge our community collaborations with the AIDS Resources Center for Wisconsin, the Madison AIDS Support Network, and the Callen-Lorde Community Health Center in New York City. We also thank William Cunningham, Rene Paulson, and Randall Schumaker for their assistance with data analysis.

Contributor Information

Nathan Grant Smith, McGill University.

Nalini Tarakeshwar, The Children’s Investment Fund Foundation, UK.

Nathan B. Hansen, Yale University

Arlene Kochman, Duke University.

Kathleen J. Sikkema, Duke University

References

  1. Antoni MH, Baggett L, Ironson G, LaPerrierre A, August S, Klimas N, et al. Cognitive-behavioral stress management intervention buffers distress responses and immunologic changes following notification of HIV-1 seropositivity. Journal of Consulting and Clinical Psychology. 1991;59:906–915. doi: 10.1037//0022-006x.59.6.906. [DOI] [PubMed] [Google Scholar]
  2. Antoni MH, Cruess DG, Cruess S, Lutgendorf S, Kumar M, Ironson G, et al. Cognitive-behavioral stress management intervention effects on anxiety, 24-hr urinary norepinephrine output, and T-cytotoxic/suppressor cells over time among symptomatic HIV-infected gay men. Journal of Consulting and Clinical Psychology. 2000;59:906–915. doi: 10.1037//0022-006x.68.1.31. [DOI] [PubMed] [Google Scholar]
  3. Bandura A. Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall; 1986. [Google Scholar]
  4. Beck AT, Steer RA. Beck Depression Inventory manual. San Antonio, TX: The Psychological Corporation; 1987. [Google Scholar]
  5. Billings AG, Moos RH. Coping, stress, and social resources among adults with unipolar depression. Journal of Personality & Social Psychology. 1984;46:877–891. doi: 10.1037//0022-3514.46.4.877. [DOI] [PubMed] [Google Scholar]
  6. Bower JE, Kemeny ME, Taylor SE, Fahey JL. Cognitive processing, discovery of meaning, CD4 decline, and AIDS-related mortality among bereaved HIV-seropositive men. Journal of Consulting and Clinical Psychology. 1998;66:979–986. doi: 10.1037//0022-006x.66.6.979. [DOI] [PubMed] [Google Scholar]
  7. Carrico AW, Antoni MH, Duran RE, Ironson G, Penedo F, Fletcher MA, et al. Reductions in depressed mood and denial coping during cognitive behavioral stress management with HIV-positive gay men treated with HAART. Annals of Behavioral Medicine. 2006;31:1155–1164. doi: 10.1207/s15324796abm3102_7. [DOI] [PubMed] [Google Scholar]
  8. Carrico AW, Antoni MH, Weaver KE, Lechner SC, Schneiderman N. Cognitive-behavioural stress management with HIV-positive homosexual men: Mechanisms of sustained reductions in depressive symptoms. Chronic Illness. 2005;1:207–215. doi: 10.1177/17423953050010030401. [DOI] [PubMed] [Google Scholar]
  9. Center for the Advancement of Health. Report on bereavement and grief research. Death Studies. 2004;28:491–575. doi: 10.1080/07481180490461188. [DOI] [PubMed] [Google Scholar]
  10. Centers for Disease Control and Prevention. HIV/AIDS and minority women. Atlanta, GA: Author; 2002. [Google Scholar]
  11. Chesney MA, Chambers DB, Taylor JM, Johnson LM, Folkman S. Coping effectiveness training for men living with HIV: Results from a randomized clinical trial testing a group-based intervention. Psychosomatic Medicine. 2003;65:1038–1046. doi: 10.1097/01.psy.0000097344.78697.ed. [DOI] [PubMed] [Google Scholar]
  12. Chesney M, Folkman S, Chambers D. Coping effectiveness training for men living with HIV: Preliminary findings. International Journal of STD & AIDS. 1996;7:75–82. doi: 10.1258/0956462961917690. [DOI] [PubMed] [Google Scholar]
  13. Cole DA, Maxwell SE. Testing mediational models with longitudinal data: Questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology. 2003;112:558–577. doi: 10.1037/0021-843X.112.4.558. [DOI] [PubMed] [Google Scholar]
  14. Cruess S, Antoni MH, Hayes A, Penedo F, Ironson G, Fletcher MA, et al. Changes in mood and depressive symptoms and change processes during cognitive-behavioral stress management in HIV-infected men. Cognitive Therapy and Research. 2002;26:373–392. [Google Scholar]
  15. Dean L. The epidemiology and impact of AIDS-related death and dying in New York’s gay community. In: Sherr L, editor. Grief and AIDS. Oxford, UK: Wiley; 1995. pp. 29–43. [Google Scholar]
  16. Derogatis LR. SCL-90-R: Administration, scoring, and procedure manual-II. Towson, MD: Clinical Psychometric Research; 1983. [Google Scholar]
  17. Doka KJ. Disenfranchised grief. In: Doka KJ, editor. Disenfranchised grief: Recognizing hidden sorrow. Lexington, MA: Lexington Books; 1989. pp. 3–11. [Google Scholar]
  18. Fleishman JA, Fogel B. Coping and depressive symptoms among people with AIDS. Health Psychology. 1994;13:156–169. doi: 10.1037//0278-6133.13.2.156. [DOI] [PubMed] [Google Scholar]
  19. Folkman S. Positive psychological states and coping with severe stress. Social Science & Medicine. 1997;45:1207–1221. doi: 10.1016/s0277-9536(97)00040-3. [DOI] [PubMed] [Google Scholar]
  20. Folkman S, Chesney MA, Collette L, Boccellari A, Cooke M. Postbereavement depressive mood and its prebereavement predictors in HIV+ and HIV− gay men. Journal of Personality and Social Psychology. 1996;70:336–348. doi: 10.1037//0022-3514.70.2.336. [DOI] [PubMed] [Google Scholar]
  21. Folkman S, Chesney MA, McKusick L, Ironson G, Johnson DS, Coates TJ. Translating coping theory into an intervention. In: Eckenrode J, editor. The social context of stress. New York: Plenum Press; 1991. pp. 239–260. [Google Scholar]
  22. Folkman S, Lazarus RS. Manual for the Ways of Coping Questionnaire: Research edition. Palo Alto, CA: Consulting Psychologists Press; 1988. [Google Scholar]
  23. Goodkin K, Blaney NT, Feaster DJ, Baldewicz T, Burkhalter JE, Leeds B. A randomized controlled clinical trial of a bereavement support group intervention in human immunodeficiency virus type 1-seropositive and -seronegative homosexual men. Archives of General Psychiatry. 1999;56:52–59. doi: 10.1001/archpsyc.56.1.52. [DOI] [PubMed] [Google Scholar]
  24. Hall RJ, Snell AF, Foust MS. Item parceling strategies in SEM: Investigating the subtle effects of unmodeled secondary constructs. Organizational Research Methods. 1999;2:233–256. [Google Scholar]
  25. Hansen NB, Tarakeshwar N, Ghebremichael M, Zhang H, Kochman A, Sikkema KJ. Longitudinal effects of coping on outcome in a randomized controlled trial of a group intervention for HIV-positive adults with AIDS-related bereavement. Death Studies. 2006;30:609–636. doi: 10.1080/07481180600776002. [DOI] [PubMed] [Google Scholar]
  26. Hobfoll SE, Dunahoo CL, Ben-Porath Y, Monnier J. Gender and coping: The dual-axis model of coping. American Journal of Community Psychology. 1994;22:49–82. doi: 10.1007/BF02506817. [DOI] [PubMed] [Google Scholar]
  27. Horowitz MJ, Marmar C, Weiss DS, DeWitt KN, Rosenbaum R. Brief psychotherapy of bereavement reactions: The relationship of process to outcome. Archives of General Psychiatry. 1984;41:438–448. doi: 10.1001/archpsyc.1984.01790160024002. [DOI] [PubMed] [Google Scholar]
  28. Hovanitz CA, Kozora E. Life stress and clinically elevated MMPI scales: Gender differences in the moderating influence of coping. Journal of Clinical Psychology. 1990;45:766–777. [PubMed] [Google Scholar]
  29. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55. [Google Scholar]
  30. Ingram KM, Jones DA, Smith NG. Adjustment among people who have experienced AIDS-related multiple loss: The role of unsupportive social interactions, social support, and coping. Omega. 2001;43:287–309. [Google Scholar]
  31. Jordan JR, Neimeyer RA. Does grief counseling work? Death Studies. 2003;27:765–786. doi: 10.1080/713842360. [DOI] [PubMed] [Google Scholar]
  32. Jöreskog KG, Sörbom D. LISREL 8.72. Lincolnwood, IL: Scientific Software International, Inc; 2005. [Google Scholar]
  33. Kelly JA, Murphy DA, Bahr GR, Kalichman SC, Morgan MG, Stevenson LY, et al. Outcome of cognitive-behavioral and support group brief therapies for depressed, HIV-infected persons. American Journal of Psychiatry. 1993;150:1679–1686. doi: 10.1176/ajp.150.11.1679. [DOI] [PubMed] [Google Scholar]
  34. Kershaw T, Northouse L, Kritpracha C, Schafenacker A, Mood D. Coping strategies and quality of life in women with advanced breast cancer and their family caregivers. Psychology & Health. 2004;19:139–155. [Google Scholar]
  35. Lazarus RS. Stress and emotion: A new synthesis. New York: Springer; 1999. [Google Scholar]
  36. Lazarus RS, Folkman S. Stress, appraisal, and coping. New York: Springer; 1984. [Google Scholar]
  37. Leiberich P, Engeter M, Olbrich E, Rubbert A, Schumacher K, Brieger M, et al. Longitudinal development of distress, coping and quality of life in HIV-positive persons. Psychotherapy & Psychosomatics. 1997;66:237–247. doi: 10.1159/000289141. [DOI] [PubMed] [Google Scholar]
  38. Lennon MC, Martin JL, Dean L. The influence of social support on AIDS-related grief reaction among gay men. Social Science and Medicine. 1990;31:477–484. doi: 10.1016/0277-9536(90)90043-r. [DOI] [PubMed] [Google Scholar]
  39. Lund DA, Caserta MS. Future directions in adult bereavement research. Omega. 1998;36:287–303. [Google Scholar]
  40. Lutgendorf SK, Antoni MH, Ironson G, Starr K, Costello N, Zuckerman M, et al. Changes in cognitive coping skills and social support during cognitive behavioral stress management intervention and distress outcomes in symptomatic human immunodeficiency virus (HIV)-seropositive gay men. Psychosomatic Medicine. 1998;60:204–214. doi: 10.1097/00006842-199803000-00017. [DOI] [PubMed] [Google Scholar]
  41. Martin JL. Psychological consequences of AIDS-related bereavement among gay men. Journal of Consulting and Clinical Psychology. 1988;56:856–862. doi: 10.1037//0022-006x.56.6.856. [DOI] [PubMed] [Google Scholar]
  42. Martin JL, Dean L. Effects of AIDS-related bereavement and HIV-related illness on psychological distress among gay men: A 7-year longitudinal study, 1985–1991. Journal of Consulting and Clinical Psychology. 1993;61:94–103. doi: 10.1037//0022-006x.61.1.94. [DOI] [PubMed] [Google Scholar]
  43. Mast BT, Lichtenberg PA. Assessment of functional abilities among geriatric patients: A MIMIC model of the functional independence measure. Rehabilitation Psychology. 2000;45:49–64. [Google Scholar]
  44. Muthén B, Tam T, Muthén L, Stolzenberg RM, Hollis M. Latent variable modeling in the LISCOMP framework: Measurement of attitudes toward career choice. In: Krebs D, Schmidt P, editors. New Directions in attitude measurement. Festschrift for Karl Schuessler. Berlin: Walter de Gruyter; 1993. pp. 277–290. [Google Scholar]
  45. Namir S, Wolcott DL, Fawzy FI, Alumbaugh MJ. Coping with AIDS: Psychological and health implications. Journal of Applied Social Psychology. 1987;17:309–328. [Google Scholar]
  46. Neimeyer RA. Searching for the meaning of meaning: Grief therapy and the process of reconstruction. Death Studies. 2000;24:541–558. doi: 10.1080/07481180050121480. [DOI] [PubMed] [Google Scholar]
  47. Neugebauer R, Rabkin JG, Williams JBW, Remien RH, Goetz R, Gorman JM. Bereavement reactions among homosexual men experiencing multiple losses in the AIDS epidemic. American Journal of Psychiatry. 1992;149:1374–1379. doi: 10.1176/ajp.149.10.1374. [DOI] [PubMed] [Google Scholar]
  48. Nicholson WE, Long BC. Self-esteem, social support, internalized homophobia, and coping strategies of HIV-positive gay men. Journal of Consulting and Clinical Psychology. 1990;58:873–876. doi: 10.1037//0022-006x.58.6.873. [DOI] [PubMed] [Google Scholar]
  49. Pakenham KI, Dadds MR, Terry DJ. Relationships between adjustment to HIV and both social support and coping. Journal of Consulting and Clinical Psychology. 1994;62:1194–1203. doi: 10.1037//0022-006x.62.6.1194. [DOI] [PubMed] [Google Scholar]
  50. Rando TA. Grief, dying, and death: Clinical interventions for caregivers. Champaign, IL: Research Press Company; 1984. [Google Scholar]
  51. Raphael B, Minkov C, Dobson M. Psychotherapeutic and pharmacological interventions for bereaved persons. In: Stroebe MS, Hansson RO, Stroebe W, Schut H, editors. Handbook of bereavement research: Consequences, coping, and care. Washington, DC: American Psychological Association; 2001. pp. 587–612. [Google Scholar]
  52. Schut H, Stroebe MS, Van Den Bout J, Terheggen M. The efficacy of bereavement interventions: Determining who benefits. In: Stroebe MS, Hansson RO, Stroebe W, Schut H, editors. Handbook of bereavement research: Consequences, coping, and care. Washington, DC: American Psychological Association; 2001. pp. 705–737. [Google Scholar]
  53. Sikkema KJ, Hansen NB, Kochman A, Tate DC, DiFranceisco W. Outcomes from a randomized controlled trial of a group intervention for HIV positive men and women coping with AIDS-related loss and bereavement. Death Studies. 2004;28:187–209. doi: 10.1080/07481180490276544. [DOI] [PubMed] [Google Scholar]
  54. Sikkema KJ, Kalichman SC, Hoffman R, Koob JJ, Kelly JA, Heckman TG. Coping strategies and emotional well-being among HIV-infected men and women experiencing AIDS-related bereavement. AIDS Care. 2000;7:463–475. doi: 10.1080/095401200750003798. [DOI] [PubMed] [Google Scholar]
  55. Sikkema KJ, Kochman A, DiFranceisco W, Kelly JA, Hoffman RG. AIDS-related grief and coping with loss among HIV-positive men and women. Journal of Behavioral Medicine. 2003;26:165–181. doi: 10.1023/a:1023086723137. [DOI] [PubMed] [Google Scholar]
  56. Simoni JM, Ng MT. Trauma, coping, and depression among women with HIV/AIDS in New York City. AIDS Care. 2000;12:567–580. doi: 10.1080/095401200750003752. [DOI] [PubMed] [Google Scholar]
  57. Skinner EA, Edge K, Altman J, Sherwood H. Searching for the structure of coping: A review and critique of category systems for classifying ways of coping. Psychological Bulletin. 2003;129:216–269. doi: 10.1037/0033-2909.129.2.216. [DOI] [PubMed] [Google Scholar]
  58. Sobel ME. Asymptotic confidence intervals for indirect effects in structural equation models. In: Leinhardt S, editor. Sociological methodology 1982. Washington, DC: American Sociological Association; 1982. pp. 290–312. [Google Scholar]
  59. Swindells S, Mohr J, Justis JC, Berman S, Squier C, Wagener MM, et al. Quality of life in patients with human immunodeficiency virus infection: Impact of social support, coping style and hopelessness. International Journal of STD & AIDS. 1999;10:383–391. [PubMed] [Google Scholar]
  60. Tarakeshwar N, Hansen N, Kochman A, Sikkema KJ. Gender, ethnicity, and spiritual coping among bereaved, HIV-positive individuals. Mental Health, Religion & Culture. 2005;8:109–125. [Google Scholar]
  61. van Heck GL, de Ridder DTD. Assessment of coping with loss: Dimensions and measurement. In: Stroebe MS, Hansson RO, Stroebe W, Schut H, editors. Handbook of bereavement research: Consequences, coping, and care. Washington, DC: American Psychological Association; 2001. pp. 449–469. [Google Scholar]
  62. Vosvick M, Gore-Felton C, Koopman C, Thoresen C, Krumboltz J, Spiegel D. Maladaptive coping strategies in relation to quality of life among HIV+ adults. AIDS & Behavior. 2002;6:97–106. [Google Scholar]
  63. Weaver KE, Antoni MH, Lechner SC, Duran REF, Penedo F, Fernandez MI, et al. Perceived stress mediates the effects of coping on the quality of life of HIV-positive women on highly active antiretroviral therapy. AIDS & Behavior. 2004;8:175–183. doi: 10.1023/B:AIBE.0000030248.52063.11. [DOI] [PubMed] [Google Scholar]
  64. Williams JBW. The Structured Interview Guide for the Hamilton Anxiety and Depression Rating Scales (SIGH-AD) New York: Biometrics Research Division, New York State Psychiatric Institute; 1988. [Google Scholar]
  65. Worden JW. Grief counseling and grief therapy: A handbook for the mental health practitioner. New York: Springer; 1991. [Google Scholar]
  66. Yalom ID, Vinogradov S. Bereavement groups: Techniques and themes. International Journal of Group Psychotherapy. 1988;38:419–446. doi: 10.1080/00207284.1988.11491130. [DOI] [PubMed] [Google Scholar]

RESOURCES