Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Nov 29.
Published in final edited form as: AIDS Educ Prev. 2012 Apr;24(2):117–131. doi: 10.1521/aeap.2012.24.2.117

REAL MEN ARE SAFE–CULTURALLY ADAPTED: UTILIZING THE DELPHI PROCESS TO REVISE REAL MEN ARE SAFE FOR AN ETHNICALLY DIVERSE GROUP OF MEN IN SUBSTANCE ABUSE TREATMENT

Donald A Calsyn 1, A Kathleen Burlew 2, Mary A Hatch-Maillette 3, Jerika Wilson 4, Blair Beadnell 5, Lynette Wright 6
PMCID: PMC4247752  NIHMSID: NIHMS640688  PMID: 22468973

Abstract

Real Men Are Safe (REMAS) was effective at reducing the number of unprotected sexual occasions for men in substance abuse treatment compared to an HIV education control intervention. Utilizing a modified Delphi process, modules from REMAS were compared to similar-content modules from other CDC-approved, culturally tailored HIV prevention interventions. Utilizing ratings and recommendations obtained from an independent expert panel, REMAS was subsequently revised to be more culturally adapted for an ethnically diverse group of men. Ratings suggested REMAS was culturally fair, but that in certain areas the culturally tailored interventions were more in tune with African American and Hispanic men. Revisions to REMAS include an added focus on how culture, social norms, and upbringing affect a man’s sexual behavior and relationships.

HIV-RELATED HEALTH DISPARITIES FOR MEN IN SUBSTANCE ABUSE TREATMENT

The Centers for Disease Control and Prevention (CDC; Centers for Disease Control, 2007) report that compared to Whites, African Americans and Hispanics are over-represented among men with HIV infection. For example, African Americans and Hispanics each accounted for 13% of the total population in the states monitored by the CDC, but among men accounted for 43.9% and 19.6% of the HIV infections, respectively. Whites accounted for 72% of the population, but for only 34.5% of the HIV infections among men. This disproportionate rate of HIV in African American and Hispanic males, relative to White males, highlights the need for evidence-based HIV prevention interventions with African Americans and Hispanics.

The Clinical Trials Network (CTN) of the National Institute on Drug Abuse (NIDA) recently completed a randomized clinical trial evaluating the Real Men Are Safe (REMAS) HIV prevention intervention targeting men in substance abuse (SA) treatment. Men assigned to the REMAS groups engaged in significantly fewer self-reported unprotected vaginal and anal intercourse occasions during the 90 days prior to the 3- and 6-month post-intervention follow-ups compared to men assigned to a standard HIV education group (Calsyn et al., 2009). In addition, men assigned to the REMAS groups were less likely to have engaged in sex under the influence of drugs or alcohol during their most recent sexual encounter prior to the 3-month follow-up compared to men assigned to a standard HIV education group (Calsyn et al., 2010b). Based on these findings, the Centers for Disease Control and Prevention (2009) added REMAS to their compendium of evidenced-based HIV prevention interventions.

The study was not powered to determine if there were ethnic differences in effectiveness of the REMAS intervention (the study sample was 30% African American and 12% Hispanic). However, post hoc analyses suggested that Whites benefitted more from the intervention than African Americans in their rates of condom use. One goal of the intervention was to increase condom use with casual sexual partners. Looking at the total study sample, the percent of participants using condoms for more than 80% of all vaginal and anal intercourse occasions with casual partners increased from baseline to the 3- and 6-month follow-ups by 34.1% and 17.2%, respectively, for REMAS participants, compared to 20.9% and 2% increase for HIV education participants (Calsyn et al., 2009). However, a closer look at ethnicity revealed that for White REMAS attenders (n = 27), frequent condom use with casual partners increased from 5.8% at baseline to 38.9% at 6-month follow-up (p < .05, McNemar binomial), whereas for African Americans (n = 9), there was an increase (11.1% to 22.2%) that was not statistically significant (Calsyn et al., 2010a, November). Although there was insufficient sample size to evaluate increased condom use with casual partners for Hispanic men, the data indicated that none of the Hispanic REMAS-attending men who completed the 6-month follow-up were using condoms frequently with their casual sex partners. These results suggested a differential effect for White, compared to African American and Hispanic men.

CONCEPTUAL FRAMEWORK FOR CULTURAL ADAPTATION OF EVIDEN-BASED INTERVENTIONS

Previous work suggests that cultural sex role beliefs and cultural differences in masculinity ideology may predict condom attitudes and condom use (Noar & Morokoff, 2002; Williams, Wyatt, Resell, Peterson, & Asuan-O’Brien, 2004). Accordingly, Wyatt’s Sexual Health Model (2009) and other experts maintain that HIV behavioral interventions that ignore the complex set of historical, environmental, and cultural factors that influence sexual behaviors are less effective than interventions that integrate culture and cultural values into behavior interventions (Airhihuhbuwa, Okoror, Shefer, Brown, et al., 2009; Lipscomb, 2002; Nobles, Goddard, & Gilbert, 2009; Wyatt, 2009).

The available literature provides compelling evidence that cultural adaptations may improve the outcomes of promising interventions. In an earlier review of 37 HIV prevention projects, Janz and colleagues (1996) concluded that prevention interventions adapted to the culture in which the interventions were being implemented were more effective than others. More recently, two other reviews revealed that culturally adapted interventions were more effective than standard interventions that did not consider culture (Castro, Barrera, & Holleran-Steiker, 2010; Griner & Smith, 2006). Breland-Noble, Bell and Nicolas (2006) demonstrated that culturally adapting interventions may enhance treatment participation. Others have demonstrated the benefits of cultural adaptations in working with specific populations, such as families (Bernal, 2006; Fraenkel, 2006; Santisteban et al., 2006; Uebelacker, Hecht, & Miller, 2006), low-income depressed mothers (Boyd, Diamond, & Bourjolly, 2006), and parents of hyperactive children (Matos et al., 2006).

The strong support in the literature for pursuing cultural adaptations of evidenced-based prevention interventions and the findings noted above suggesting RE-MAS was less effective with non-Whites provided the rationale for adapting REMAS to make it more culturally appealing to African American and Hispanic men, while maintaining the fundamental core components of the intervention that led to its inclusion by the CDC as a promising evidenced-based HIV prevention intervention on the DEBI (Diffusion of Effective Behavioral Interventions) website (Centers for Disease Control, 2009). We hoped that by making REMAS more relevant to these men, we could create an intervention that was effective for an ethnically diverse group of men in SA treatment rather than for one cultural group. This would be important given that many SA treatment programs have culturally diverse clientele and it is economically unfeasible to have HIV interventions for each subgroup. This issue of generalizability is also reflected on the DEBI website, where there are many evidence-based, culturally tailored HIV prevention interventions that focus on specific cultural groups but few with demonstrated effectiveness with multiple subgroups.

Some guidance exists on intervention adaptation in the context of HIV (Wingood & DiClemente, 2008) and culture (Castro et al., 2010). Both of these investigative teams stress the need to maintain core elements of the intervention when adapting the intervention for increased relevance to the new targets.

Further, Resnicow and colleagues (2000) conceptualize a culturally sensitive intervention as one that reflects the “ethnic/cultural characteristics, experiences, norms, values, behavioral patterns, and beliefs of a target population.” In addition to the work of Castro and colleagues (2010) mentioned above, several authors also have described procedures to make health promotion materials more culturally appropriate for targeted ethnic populations (Bernal & Scharrondel-Rio, 2001; Castro & Garfinkle, 2003; Dévieux et al., 2005). Bernal, Bonilla, and Bellido (1995) and Yuen (2004) established criteria for identifying a culturally tailored intervention which include ensuring (1) use of language, idioms, and expressions of the target group; (2) use of the symbols and concepts of the target population; (3) presentation of the material in a manner that is consistent with the knowledge, cultural values, and customs of the target group; (4) incorporation of activities that enhance ethnic identity; and (5) use of materials that demonstrate an understanding of the social context that surrounds the behavior and living situation of the target group. Thus, we found that the literature provided guidance on the concepts of cultural adaptation, but that an evidence-based process for the adaptation was also necessary.

THE DELPHI PROCESS

With the above guidelines as a framework for our manual revision, we felt a modified Delphi process could be effectively used to accomplish our goal. The Delphi process as described by DeVilliers, DeVilliers, and Kent (2005) uses a series of questionnaires to aggregate expert opinion in an anonymous fashion. This takes place over a series of rounds. First, a question (in this case the cultural relevance of intervention modules being examined) is formulated and expanded upon in a set of assumptions, solutions, or options (in our case, options for different approaches to delivering desired intervention concepts). Second, an expert panel is identified and invited to provide opinions (in our case, by means of ratings and revision suggestions). The responses are analyzed and ranked. A second questionnaire is developed using the results and feedback from the first round. Participants again record their opinions, which are collated and assessed for consensus. The process terminates when an acceptable degree of consensus is reached. Two to three rounds are usually sufficient to achieve this, with the largest adjustments usually occurring between rounds one and two.

METHODS

OVERVIEW

A panel of academic and community-based experts, knowledgeable in HIV prevention in African-American and Hispanic communities, was convened. Utilizing the Delphi process, panel members reviewed the REMAS materials along with non-REMAS HIV prevention materials from other culturally tailored interventions for African Americans or Hispanics that were not specific to SA treatment. These materials were rated by panel members for cultural sensitivity. Panel members were encouraged to make specific suggestions for revisions to REMAS that would render it culturally viable for a mixed group of Whites, African Americans and Hispanics.

THE EXPERT PANEL

The expert panel consisted of individuals from academic settings and from community SA treatment programs. Panel members from academic settings needed to have at least one peer-reviewed publication in either the area of health disparities among African Americans or Hispanics or in the area of developing culturally appropriate health intervention or behavioral change treatment materials. The investigators identified potential academic experts through their personal network of professional contacts. However, only one of the four academic expert panel members was personally known to any of the investigators prior to organizing the panel.

To identify potential community treatment–based experts, the investigators elicited nominations from members of the NIDA CTN Community Treatment Programs Caucus. The investigators reviewed the resumes of nominees submitted, conducted phone interviews with eight nominees, and selected six to be on the expert panel.

All expert panel members were either African American or Hispanic, and all but two were men. One of the academic experts had to withdraw from the study when increased professional demands independent from the study rendered study participation infeasible. Thus, our panel consisted of a total of 3 academic and 6 community experts.

ORIGINAL REMAS INTERVENTION

The original Real Men Are Safe (REMAS) was an HIV prevention intervention of five 90-minute group sessions targeting men in SA treatment and developed for the CTN “Safer Sex for Men” protocol (CTN0018). REMAS drew heavily from the NIMH Multisite HIV Prevention Trial Group’s (1998) Project Light and Bartholomew and colleagues’ (2000) Time Out! For Men. In addition to lecture material, there was liberal use of role-plays, peer group discussions, and self-assessment motivational exercises. There was nearly an equal focus on information delivery and skill building, with a somewhat smaller focus on motivation. A more detailed description of the intervention is available in Calsyn and colleagues (2009). There was no attempt in Real Men Are Safe, or either of the interventions from which it borrowed heavily, to address cultural issues related to HIV prevention. Rather, these interventions were designed to be culturally neutral.

NON-REMAS HIV PREVENTION INTERVENTION MATERIALS

With the original REMAS intervention as a starting point, the Delphi process required first identifying sources of new material that matched REMAS as closely as possible in content focus, but that was more culturally sensitive. The authors identified all HIV prevention interventions tailored toward African Americans or Hispanics that were evidence-based and vetted through the Diffusion of Effective Behavioral Programs (DEBI) or Effective Programs Plus (REP+) programs (CDC, 2009) and that also had publically available treatment manuals for review. However, none were specifically designed for use in SA treatment settings. The following four interventions met these criteria and were selected by the investigators for expert panel review:

  1. Nia—a video-based motivational skills-building intervention consisting of 6–10 participants in each group and targeting African-American men who have sex with women (Kalichman, Cherry & Browne-Sperling, 1999; Pyeatt & Tirado, 2008). The intervention includes videos, movie clips, and discussion to educate men about HIV/AIDS, elevate their mood, and entertain them while reinforcing information and motivating behavior change.

  2. d-up: Defend Yourself! (d-up)—a community-level intervention that seeks to mobilize an existing social network of Black men who have sex with men (MSM) to support condom use and improve their sense of self-worth (Jones et al., 2008). d-up uses specific social network members, called opinion leaders, who are respected and trusted by their peers, to promote the benefits of consistent condom use and increase self-worth among their friends and acquaintances.

  3. Many Men, Many Voices (3MV)—a group-level intervention that addresses behavioral and social determinants influencing the HIV/STI risk and protective behaviors of Black MSM (Wilton et al., 2009). Cultural, social, and religious norms, identity of Black MSM and their degree of connectedness to the Black and gay communities, HIV/STI interactions, sexual relationship dynamics, and the social influences of racism and homophobia are also addressed.

  4. ¡Cuídate! (Take Care of Yourself)—a small group, culturally based intervention to reduce HIV sexual risk among Latino youth (Villarruel, Jemmott, & Jemmott, 2005, 2006). The intervention consists of six 60-minute modules delivered to small, mixed-gender groups. ¡Cuídate! incorporates salient aspects of Latino culture, including familialism and machismo. These cultural beliefs are used to frame abstinence and condom use as culturally accepted and effective ways to prevent sexually transmitted diseases, including HIV.

DELPHI PROCESS: ROUND 1

Prior to initiating round 1 of the Delphi process, three packets of materials were assembled for the expert panelists. One packet consisted of articles from the professional literature reviewing conceptualizations and procedures previously employed to make health promotion materials more culturally appropriate for targeted ethnic populations (Bernal et al., 1995; Bernal & Sáez-Santiago, 2006; Castro et al., 2010; Resnicow et al., 2000). These articles identify two approaches to culturally tailoring a generic intervention. The first is a “surface structure” strategy. This approach uses intervention materials and messages that match the observable characteristics (e.g., race, language, music, food) of the target group. Essentially, the message is unchanged but instead is delivered in a culturally consistent manner. The other approach is a “deep structure” strategy. This approach incorporates cultural values as well as social, psychological, and historical factors into the intervention. For the panelists, the research literature was meant to provide a consistent context from which they could consider the forthcoming REMAS and new HIV intervention materials.

The second packet included the 13 modules that comprise the five sessions of REMAS (Calsyn et al., 2009). The third and final packet included modules from the aforementioned non-REMAS, evidence-based, HIV prevention materials. Each REMAS module was paired with 2–3 modules from the culturally tailored non-REMAS interventions. For example, the REMAS module “Overcoming Barriers to Condom Use” was paired with the Nia module “Pros and Cons of Condom Use,” the d-up module “What Can You Do with a Condom,” and the ¡Cuídate! module “Overcoming Barriers to Condom Use.” In choosing modules from d-up and Many Men, Many Voices for the expert panel to review, the investigators avoided modules with an MSM-specific focus. This was because REMAS and its cultural adaptation were intended to be neutral and appropriate for men of all sexual orientations. Similarly, adolescent-focused material from ¡Cuídate! was minimized in choosing comparison modules. In instances where MSM or adolescent-focused material was selected for comparison, expert panel members were reminded that the target group consisted of ethnically diverse adult men of diverse sexual orientation in substance abuse treatment, the majority of whom would be heterosexual in most instances.

For each combination of modules, the panel members received a rating form with four dimensions taken from Bernal and colleagues (1995) and Bernal and Sáez-Santiago (2006). For each dimension (A–D), the panel member rated the module on a 1 to 5 metric: (A) Use of Language/Expressions of the Target Group (1 = High use of unfamiliar language/expressions for target group; 5 = High use of unique language/expressions for target group); (B) Activities That Enhance Ethnic Identity (1 = High use of activities that detract from ethnic identity; 5 = High use of activities that enhance ethnic identity); (C) Consistent with Norms, Knowledge, Cultural Values of Target Group (1 = Presentation is highly inconsistent with norms, knowledge, cultural values of target group; 5 = Presentation is highly consistent with norms, knowledge, cultural values of target group); and (D) Understands Social Context That Surrounds the Behavior and Living Situation of Target Population (1 = Material suggests high insensitivity to the social context/living situations of target group; 5 = Material suggests high sensitivity to the social context/living situations of target group). Separate ratings were generated for use of the modules with African American and Hispanic men. A rating difference between a REMAS module and any corresponding culturally tailored module for either African Americans or Hispanics of greater than 0.5 was a priori identified as rationale for a possible major revision of the REMAS module.

In addition to completing the ratings, panel members were asked to make specific suggestions for revising the REMAS materials. Suggestions could range from no change to minor/major revisions to completely replacing the module with different material.

For two REMAS modules (Experience with Sex and Drugs/Enhancing Sex without Drugs; Coping with Sexual Dysfunction), there were no corresponding modules from the culturally tailored interventions.

Panel members received the packets via e-mail or U.S. mail and returned completed rating forms and suggestions for revision in a similar manner. Consistent with Delphi process procedures, each panel member completed the rating forms and revision suggestions independent from other panel members, and the panelists were anonymous to each other. Round 1 of the Delphi process took approximately 10 weeks.

DELPHI PROCESS: ROUND 2

The round 1 responses were analyzed by calculating central tendency and variability measures for the eight ratings (four dimensions by two ethnicities) for each intervention module. Experts’ suggestions for revisions were de-identified and summarized in a table. Based on module ratings and the revision suggestions, the investigative team prepared a revised version of REMAS that was identified as REMAS–CA (culturally adapted). For round 2 of the Delphi process, panelists received: (1) the summary of revision suggestions from round 1; (2) a response by the investigative team to each suggestion for revision which ranged from “completely accept,” “completely reject,” or “partially accept” (i.e., implement some aspects or themes from the suggestion); (3) a REMAS–CA manual; and (4) rating forms. The panelists were asked to rate each REMAS–CA module on the four dimensions utilized in round 1 on the same 1–5 metric. Again, the panelists were asked to indicate whether the module was acceptable “as is” or needed further revision. If further revision was recommended, specific suggestions for revisions were elicited. Rating forms and suggestions for revisions were returned to the Delphi process manager. Further, relatively minor revisions to the REMAS–CA manual were completed in response to the round 2 ratings and suggestions for revisions. At this point, the investigative team felt sufficient consensus had been obtained such that a third round of the Delphi process was not needed. Round 2 of the Delphi process took approximately 4 weeks.

RESULTS

Generally speaking, raters tended to rate consistently across the four dimensions. Hence, to simplify analysis, the ratings across the four dimensions were averaged for each module, separately for each race and intervention program. For rounds 1 and 2, respectively, the median Cronbach’s alpha for the four ratings was .80 and .81, with 80% and 92% of modules having item alphas >.60.

Presented in Table 1 are the round 1 mean ratings for each REMAS and non-REMAS module, averaged across the nine raters and, as noted above, averaged across the four cultural element dimensions. Ratings were averaged across dimensions because the ratings between dimensions were highly correlated for each module being rated. Consistent with the “culturally neutral” development of REMAS, there was little evidence that the REMAS modules were culturally biased or insensitive to cultural dimensions for either African Americans or Hispanics, as all of the modules received a mean rating either very near 3 (neutral) or greater than 3. For African Americans (Table 1, top) and Hispanics (Table 1, bottom), there was a rating difference of 0.5 on only three modules: Getting Started, Relationships/Social Norms, and Sexual Communication Skills. As expected, experts rated the non-REMAS modules that were actually geared toward African Americans as more culturally relevant to African Americans than the similar content REMAS modules. For Hispanics, two of the non-REMAS modules that experts rated higher than REMAS modules targeted Hispanics, and a third one targeted African American men. Besides making major changes to these three modules, additional significant changes were made to 5 others based on specific reviewer recommendations that were not fully reflected in the mean ratings presented in Table 1.

TABLE 1.

Round 1 Expert Panelist Mean Ratings for REMAS Modules and Their Corresponding Non-REMAS Modules

HIV Prevention Interventions
REMAS Nia d-up Many Men, Many Voices ¡Cuídate!
Modules and Target Audience Mean (sd) Mean (sd) Mean (sd) Mean (sd) Mean (sd)
African American
Getting Started 3.03 (0.73) 3.53 (1.33) 3.09 (0.76) 3.06 (1.18)
Risk Reduction Self-Assessment 3.11 (0.31) 3.00 (0.49)
HIV/AIDS Information 3.19 (0.70) 3.47 (0.71) 3.66 (0.91) 3.50 (0.88) 3.13 (0.66)
Condom Use Skills 2.86 (0.36) 3.30 (0.54)
Healthy Options 3.00 (0.33) 3.08 (0.54) 3.36 (0.88)
Condom Barriers 3.03 (0.16) 3.31 (0.56) 3.37 (0.57) 3.09 (0.59)
Relationships/Social Norms 3.14 (0.31) 3.58 (0.72) 4.17 (0.65) 3.06 (1.17)
Prevention Planning 3.31 (0.50) 3.62 (0.74)
Communication Skills for Safe Sex 3.11 (0.73) 3.94 (0.97) 3.14 (0.61)
Summary 3.56 (0.66) 3.59 (0.72)
Review Prior Sessions 3.11 (0.36) 3.42 (0.41)
Sex Under the Influence 3.22 (0.34)
Coping with Sexual Dysfunction 2.92 (0.50)
Hispanic
Getting Started 2.94 (0.70) 3.30 (1.05) 3.00 (0.61) 4.30 (1.08)
Risk Reduction Self-Assessment 3.14 (0.36) 3.06 (0.45)
HIV/AIDS Information 3.28 (0.66) 3.34 (0.42) 3.22 (0.52) 3.15 (0.46) 3.59 (0.88)
Condom Use Skills 2.72 (0.32) 3.14 (0.41)
Healthy Options 2.97 (0.29) 3.11 (0.44) 3.11 (0.40)
Condom Barriers 3.13 (0.23) 3.13 (0.40) 3.22 (0.63) 3.54 (0.42)
Relationships/Social Norms 3.06 (0.39) 3.14 (0.65) 3.39 (0.82) 4.78 (0.44)
Prevention Planning 3.25 (0.54) 3.56 (0.67)
Communication Skills for Safe Sex 3.14 (0.75) 3.64 (0.96) 3.58 (0.66)
Summary 3.56 (0.61) 3.50 (0.66)
Review Prior Sessions 3.06 (0.37)
Sex Under the Influence 3.19 (0.39)
Coping with Sexual Dysfunction 2.92 (0.50)
Open in a new tab

Note. Bolded values represent a difference of 0.5 or more in the rating from that for REMAS.

Presented in Table 2 are the mean ratings for each REMAS (round 1) and REMAS–CA (round 2) modules, averaged across the 9 raters and averaged across the four cultural element dimensions. For round 2, several content areas were broken down into more narrowly focused modules. In those cases, each more narrowly focused module from round 2 was compared to the larger content area from round 1. Also listed are the resulting changes incorporated into the final REMAS–CA manual, most of which occurred between rounds 1 and 2 of the Delphi process. Modules with mild to moderate changes maintained the core elements of the original REMAS module, but they were revised to be more cultural sensitive. Modules that were added shared a stronger focus on understanding how cultural and socialization experiences contribute to a man’s past or current sexual behavior. Modules that were deleted were removed because room was needed for the new modules, their content was likely to be offered through existing clinic programming, and/or their content was already contained in other REMAS–CA modules.

TABLE 2.

Mean Rating for Each REMAS Module in Round 2 and Changes Made for REMAS–CA

Topic Area African American Hispanic Changes from REMAS to REMAS-CA
M (sd) Change from
M (sd) Change from
Round 1 Round 1
Getting Started 3.58 (0.64) 0.55 3.47 (0.69) 0.53 Participants now generate their own group norms/rules, rather than have group leaders dictate them. Participants identify something about themselves of which they are proud.
Risk Reduction Self-Assessment 3.47 (0.64) 0.36 3.47 (0.64) 0.33 Little change.
HIV/AIDS Information 3.50 (0.61) 0.31 3.42 (0.66) 0.14 Participants more actively involved; myths associated with HIV more actively addressed.
Condom Use Skills
Male Condom Demonstration & Practice 3.31 (0.39) 0.45 3.31 (0.39) 0.59 A partner rating of practice removed.
Female Condom Demonstration & Practice 3.22 (0.36) 0.36 3.22 (0.36) 0.50 A partner rating of practice removed.
Healthy Options 3.42 (0.33) 0.42 3.39 (0.44) 0.42 Revised to focus on drug use and sex risk hierarchies/continuums rather than harm reduction.
Condom Barriers 3.42 (0.38) 0.39 3.44 (0.35) 0.31 Participants more actively involved; cultural barriers also identified.
Relationships/Social Norms
Who’s Got the Power? 3.78 (0.59) 0.64 3.83 (0.50) 0.77 New module exploring observations of others’ intimate relationships during participants’ social development.
Culture Values and Intimate Relationships 4.00 (0.61) 0.86 4.08 (0.57) 1.02 New module exploring how developmental history and culture affected participants’ own intimate relationships.
Ideal Man/Ideal Woman 3.81 (0.57) 0.67 3.86 (0.44) 0.80 Revised to explore cultural differences also.
Changing Social Norms 3.56 (0.50) 0.42 3.56 (0.50) 0.50 New module challenging participants to explore norms in their social networks and ways to change those norms.
Prevention Planning
 Building Skills for Making Safer Sex Decisions: Movie Clips 3.44 (0.69) 0.13 3.39 (0.71) 0.14 Revised to utilize movie clips as stimulus material for planning how participants might address future sexually risky situations.
 Personal commitment to Sexual Safety 3.25 (0.48) −0.06 3.19 (0.62) −0.06 Little change.
Communication Skills for Safe Sex
Communicating about Safe Sex: Talk Tools 3.28 (0.54) 0.17 3.22 (0.58) .08 Little change.
Talk Role Plays 3.78 (0.87) 0.67 3.67 (0.71) 0.53 New role-plays written by expert panel members using language they felt was more in tune with African American or Hispanic men.
Practice Talk Tools: Responding to Excuses for Not Using Condoms 3.53 (0.67) 0.42 3.50 (0.64) 0.36 New module requiring participants to practice responding to reasons for not using condoms.
Practice Talk Tools: Responding to Come-ons for Sex Under the Influence 3.72 (0.76) 0.61 3.75 (0.79) 0.61 New module requiring participants to practice drug use refusal when confronting a possible sex under the influence opportunity.
Summary na na na na na na Not included in REMAS–CA.
Review Prior Sessions 3.56 (0.53) 0.45 3.53 (0.55) 0.47 Little change.
Sex Under the Influence
Experience with Sex and Drugs 3.33 (0.66) 0.11 3.28 (0.68) 0.09 Little change.
Enhancing Sex without Drugs 3.06 (0.30) −0.16 3.06 (0.30) −0.13 Little change.
Coping with Sexual Dysfunction 3.22 (0.49) 0.30 3.25 (0.52) 0.33 Removed from REMAS–CA.
Talking Circle 3.78 (0.81) na 3.75 (0.62) na Added to close each session.
Open in a new tab

Note. Italicized topics are modules created for round 2; they are listed under the modules they replaced and differences from round 1 were calculated compared to those. Bold values represent a difference of 0.5 or more in the rating from that for REMAS.

DISCUSSION

Castro and colleagues (2010) suggest that cultural adaptation may be appropriate when an intervention is ineffective for a particular subgroup. Consistent with that proposition, the aim of the current project was to revise the REMAS intervention to be more culturally relevant for African American and Hispanic men. The Delphi process, a structured method for obtaining consensus anonymously, provided a workable framework for soliciting feedback for cultural adaptation of REMAS for African American and Hispanic men. Even though the initial expert ratings suggested that REMAS was culturally neutral rather than culturally biased, the experts proposed ways to increase further its cultural relevance for the target groups. The revised modules focused more on the influence of the social context and developmental socialization on male decisions about sexual behaviors, and less on information delivery.

Interestingly, the revisions to REMAS coincided with existing literature on working effectively with African American and Hispanic men. For example, Wyatt (2009) proposes that relevant gender and culture-bound norms (e.g., “tough” façade, status in sexual conquests), along with individual health beliefs, affect the likelihood that African-American men will engage in HIV risk behaviors. Wyatt maintains that behavioral interventions that ignore such contextual factors are less effective. The revised activities in REMAS–CA are similar to the “talk and listen,” conflict resolution, and sexual ownership techniques that Wyatt advocates.

THE DELPHI PROCESS AS A TOOL FOR CLINICAL RESEARCHERS

The Delphi process possesses several strengths and was well suited for our cultural adaptation efforts for the following reasons. First, it gives equal weight to the opinions of each expert. For our purposes, we utilized an expert panel with great diversity in academic experience and achievement, yet the anonymity of the Delphi process eliminated the potential for higher status or more persuasive members to influence the group dynamics disproportionately. Second, the Delphi process allows more time for experts to think through their recommendations rather than confining their participation to a one two-day intensive planning session. For this study, each panel member was able to work at his/her own pace. Some of the treatment provider panelists even “tried out” treatment modules being rated in their treatment programs, which further enriched their round 2 feedback. Despite the advantages, the Delphi process also presented some challenges in this study. First, the investigative team had less opportunity than a face-to-face meeting to probe the experts about their recommendations. Second, the Delphi process inhibited the investigative team from capitalizing on any synergy emerging from a group discussion. Moreover, since the panel members only had access to a written record of the results from earlier rounds, they lacked the opportunity to probe each other about responses. While a deliberate choice for the current project, other investigators/projects might feel this was a disadvantage in situations where group discussions could substantially benefit consensus-building. A third disadvantage to the Delphi process is its reliance on written feedback. In settings where panelists would be uncomfortable or less skilled in providing written feedback, the Delphi process may be less desirable or need significant modification to achieve the goals put forward.

Several broader issues influence the cultural adaptation of interventions. First, the investigative team needed to find a reasonable balance between adaptation and fidelity. DePue and colleagues (2010) raise the question of how much an intervention can be adapted before becoming a different intervention altogether. Second, interventions for treatment samples face some unique issues irrelevant in many other settings. For example, a clinic serving a diverse clientele may not have the luxury of using an intervention tailored to only one racial/ethnic group. Moreover, the clinic faces the challenge of integrating the intervention into the rest of the treatment. Other aspects of treatment may also influence a client’s response to the intervention.

In Table 3, to further distill our process, we provide a user-friendly summary of how the Delphi process could be used by clinical researchers and intervention developers to adapt existing material for new purposes.

TABLE 3.

User-Friendly Summary for Using the Delphi Process in Intervention Adaptation

Identify what original material is to be adapted, and for what purpose
 -Who is the target audience?
Identify an expert panel
 Determine who will be the experts—researchers, consumers, clinicians?
 Estimate the amount of time needed from experts
Identify what material will serve as comparison to the original intervention
 Existing interventions that have desirable elements?
 Newly developed material?
Identify what form expert feedback will be in
 Rating scales? (if so, develop)
 Written comments/suggestions?
 New material to be developed
Prepare Delphi process round 1 materials
 Provide background/basic information to experts so they’re oriented and “on the same page” for the task at hand?
 Provide original intervention and comparison material to experts
 May need to break it down and/or organize it in “digestible” chunks (e.g., grouping similar-content sections so experts can make more specific comparisons)
 Provide method for experts to relay feedback (rating scales, forms to organize written feedback, etc.)
 Provide clear instructions for the task
Delphi process round 1
 Communicate a deadline for return of feedback
 Be prepared to proactively check in with experts to address confusion or problems with delivery or materials
Process feedback from round 1
 Analyze ratings (if applicable)
 Review feedback, identify areas of panel consensus and disagreement
 Prepare summary report of feedback from round 1 to be used in round 2
 Respond to feedback by revising original intervention material
Prepare Delphi process round 2 materials
 Provide summary report of feedback from round 1
 Provide newly revised intervention material
 Provide method for experts to relay feedback
 Provide clear instructions for task
Delphi process round 2
 Communicate a deadline for return of feedback
 Be prepared to proactively check in with experts to address confusion or problems with delivery or materials
Process feedback from round 2
 Analyze ratings (if applicable)
 Review feedback, identify areas of panel consensus and disagreement
 Prepare summary report of feedback from round 2 to be used in round 3, if necessary
 Identify areas of consensus for further revisions
 If adequate consensus has been obtained, finalize revisions
 If adequate consensus has not been obtained, conduct round 3 by repeating round 2 protocol
Revised intervention is ready to be tested in the field once adequate consensus has been obtained.
Open in a new tab

The investigative team is currently piloting REMAS–CA in four clinics: in Los Angeles, California, Hartford, Connecticut, Columbia, South Carolina, and Santa Fe, New Mexico. Preliminary and anecdotal evidence suggests that REMAS–CA has been well received by its target audience, an ethnically diverse group of men in outpatient substance abuse treatment. Modules that encourage exploration of the influence of cultural norms on sexual and risk decision-making and gender roles have been particularly powerful at eliciting reflections from men. Study participants and the REMAS–CA counselors collectively report that such content and its relation to relapse prevention, communication/negotiation of safer sex, and responsibility for self and community are unique to programming in their treatment clinics. REMAS–CA therefore appears to be offering new content to these treatment-seekers. Along with evaluating the treatment outcomes, focus groups will occur after the last group sessions to gather feedback from a sample of participants. This feedback will be used to conduct a final revision of REMAS–CA.

LIMITATIONS

Although the goal of the study was to develop a culturally adapted HIV prevention intervention for a culturally diverse group of men in substance abuse treatment, we did not ask our expert panel to rate the cultural relevance of REMAS or REMAS–CA modules for White men since evidence of REMAS efficacy for that group already existed, nor did we include Whites as members of the expert panel. These shortcomings are somewhat modulated by having two White co-investigators who were experienced in conducting the REMAS intervention with ethnically diverse groups of men, reviewing the Delphi process. Their experience with both the original REMAS and REMAS–CA provided a continual point of reference for how this intervention performed with White men.

A second potential limitation is not including patient consumers in our expert panel for the Delphi process. However, given the substantial volume of academic material to read, evaluate, and rate, we felt that consumer input would be more valuable once the intervention had been fully revised. Therefore, we are currently seeking specific consumer feedback via focus groups from men who attended REMAS–CA groups within their substance abuse treatment programs during the ongoing pilot trial.

In summary, this study sought to revise an existing evidence-based HIV/STI prevention manual for use in SA treatment settings. Revisions were aimed to increase the intervention’s relevance for, and efficacy in, an ethnically diverse group of men. The study successfully used a Delphi process framework to organize the revision process. Future intervention design efforts may find the Delphi process a useful and proven method for achieving consensus.

Acknowledgments

The study was supported by a grant from the National Institute on Drug Abuse (RC1 DA028245).

The investigators wish to thank Marilyn Pyeatt and Anne Freeman, University of Texas Southwestern, for their assistance in providing materials from the Nia intervention and suggestions on cultural adaptation. They also wish to thank the expert panel members for their significant contribution to the project: Felipe Castro, University of Arizona; Ronald Braithwaite, Morehead School of Medicine; Cynthia Prather, Center for Disease Control and Prevention; Larry Wallace, Volunteers of America; Sandra Larios, University of California San Francisco; Angel Valencia, La Frontera Center; Jerome Turner, Western Psychiatric Institute and Clinic; Marion Coletrane, Lexington/Richmond Alcohol and Drug Abuse Council; and Carlos Freytas, Healthcare Alternative Systems, Inc.–Transitional Housing Program.

Footnotes

Findings reported in this manuscript were presented in part at the annual meetings of the American Public Health Association, November 9, 2010, in Denver, Colorado, and at the Health Disparities in Boys and Men: Innovative Research to Reduce Addiction, Trauma and Related Co-morbidities Conference, November 1, 2010, in Washington, D.C.

Contributor Information

Donald A. Calsyn, Alcohol & Drug Abuse Institute and the Department of Psychiatry & Behavioral Sciences at the University of Washington in Seattle

A. Kathleen Burlew, University of Cincinnati and the Crossroads Center in Cincinnati.

Mary A. Hatch-Maillette, Alcohol & Drug Abuse Institute and the Department of Psychiatry & Behavioral Sciences at the University of Washington in Seattle

Jerika Wilson, University of Cincinnati and the Crossroads Center in Cincinnati.

Blair Beadnell, School of Social Work at the University of Washington in Seattle.

Lynette Wright, Alcohol and Drug Abuse Institute in Seattle.

References

  1. Airhihuhbuwa C, Okoror T, Shefer T, Brown D, et al. Stigma, culture, and HIV and AIDS in the Western Cape, South Africa: An application of the PEN-3 cultural model for community-based research. Journal of Black Psychology. 2009;35(4):407–432. doi: 10.1177/0095798408329941. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bartholomew NG, Hiller ML, Knight K, Nucatola DC, Simpson DD. Effectiveness of communication and relationship skills training for men in substance abuse treatment. Journal of Substance Abuse Treatment. 2000;18:217–225. doi: 10.1016/s0740-5472(99)00051-3. [DOI] [PubMed] [Google Scholar]
  3. Bernal G. Intervention development and cultural adaptation research with diverse families. Family Process. 2006;45:143–151. doi: 10.1111/j.1545-5300.2006.00087.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bernal G, Bonilla J, Bellido C. Ecological validity and cultural sensitivity for outcome research: Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology. 1995;23:67–82. doi: 10.1007/BF01447045. [DOI] [PubMed] [Google Scholar]
  5. Bernal G, Sáez-Santiago E. Culturally centered psychosocial interventions. Journal of Community Psychology. 2006;34:121–132. [Google Scholar]
  6. Bernal G, Scharrondel-Rio MR. Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cultural Diversity and Ethnic Minority Psychology. 2001;7:328–342. doi: 10.1037/1099-9809.7.4.328. [DOI] [PubMed] [Google Scholar]
  7. Boyd RC, Diamond GS, Bourjolly J. Developing a family-based depression prevention program in urban community mental health clinics: A qualitative investigation. Family Process. 2006;45:18–103. doi: 10.1111/j.1545-5300.2006.00090.x. [DOI] [PubMed] [Google Scholar]
  8. Breland-Nobel AM, Bell C, Nicolas G. Family first: The development of an evidence-based family intervention for increasing participation in psychiatric clinical care and research in depressed African American adolescents. Family Process. 2006;45:153–169. doi: 10.1111/j.1545-5300.2006.00088.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Calsyn DA, Burlew AK, Hatch-Maillette MA, Wilson J, Beadnell B, Wright L. Culturally adapting Real Men Are Safe (CTN0018). Poster session presented at the annual meeting of the American Public Health Association; Denver, Colorado. 2010a. Nov, [Google Scholar]
  10. Calsyn DA, Crits-Christoph P, Hatch-Maillette MA, Doyle SR, Song YS, Coyer S, et al. Reducing sex under the influence of drugs or alcohol for patients in substance abuse treatment. Addiction. 2010b;105:100–108. doi: 10.1111/j.1360-0443.2009.02812.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Calsyn DA, Hatch-Maillette M, Tross S, Doyle SR, Crits-Christoph P, Song YS, et al. Motivational and skills training HIV/STI sexual risk reduction groups for men. Journal of Substance Abuse Treatment. 2009;37:138–150. doi: 10.1016/j.jsat.2008.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Castro FG, Barrera M, Jr, Holleran Steiker LK. Issues and challenges in the design of culturally adapted evidence-based interventions. Annual Review of Clinical Psychology. 2010;6:213–239. doi: 10.1146/annurev-clinpsy-033109-132032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Castro FG, Garfinkle J. Critical issues in the development of culturally relevant substance abuse treatments for specific minority groups. Alcoholism: Clinical and Experimental Research. 2003;27:1381–1388. doi: 10.1097/01.ALC.0000080207.99057.03. [DOI] [PubMed] [Google Scholar]
  14. Centers for Disease Control and Prevention. Update to racial/ethnic disparities in diagnoses of HIV/AIDS—33 states, 2001–2005. Morbidity and Mortality Weekly Report. 2007;56:189–193. [PubMed] [Google Scholar]
  15. Centers for Disease Control and Prevention. 2009 Compendium of evidence-based HIV prevention interventions. Updated August. 2009;24:2009. http://www.cdc.gov/hiv/topics/research/prs/evidence-based-interventions.htm. [Google Scholar]
  16. Dévieux JG, Malow RM, Rosenberg R, Jean-Gilles M, Samuels D, Ergon-Pérez E, et al. Cultural adaptation in translational research: Field experiences. Journal of Urban Health. 2005;82(S3):iii82–iii91. doi: 10.1093/jurban/jti066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. DePue JD, Rosen RK, Batts-Turner M, Bereolos N, House M, Held RF, et al. Cultural translation of interventions: Diabetes care in American Samoa. American Journal of Public Health. 2010;100:2085–2093. doi: 10.2105/AJPH.2009.170134. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. De Villiers MR, De Villiers PJT, Kent AP. The Delphi technique in health sciences education research. Medical Teacher. 2005;27:639–643. doi: 10.1080/13611260500069947. [DOI] [PubMed] [Google Scholar]
  19. Fraenkel P. Engaging families as experts: Collaborative family program development. Family Process. 2006;45:237–257. doi: 10.1111/j.1545-5300.2006.00093.x. [DOI] [PubMed] [Google Scholar]
  20. Griner D, Smith TB. Culturally adapted mental health interventions: A meta-analytic review. Psychotherapy: Theory, Research, Practice, Training. 2006;43:531–548. doi: 10.1037/0033-3204.43.4.531. [DOI] [PubMed] [Google Scholar]
  21. Janz N, Zimmerman M, Wren P, Israel BA, Freudenberg N, Carter R. Evaluation of 37 AIDS projects: Successful approaches and barriers to program effectiveness. Health Education Quarterly. 1996;23(1):80–97. doi: 10.1177/109019819602300106. [DOI] [PubMed] [Google Scholar]
  22. Jones KT, Gray P, Whiteside YO, Wang T, Bost D, Dunbar E, et al. Evaluation of an HIV prevention intervention adapted for black men who have sex with men. American Journal of Public Health. 2008;98:1043–1050. doi: 10.2105/AJPH.2007.120337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kalichman SC, Cherry C, Browne-Sperling F. Effectiveness of a video-based motivational skills-building HIV risk-reduction intervention for inner-city African American men. Journal of Consulting and Clinical Psychology. 1999;67:959–966. doi: 10.1037//0022-006x.67.6.959. [DOI] [PubMed] [Google Scholar]
  24. Lipscomb R. How to help African American daughters survive in the new millennium: An empowerment perspective. Race, Gender, & Class. 2002;9(3):96–101. [Google Scholar]
  25. Matos M, Torres R, Santiago R, et al. Adaptation of parent-child interaction therapy for Puerto Rican families: A preliminary study. Family Process. 2006;45:205–222. doi: 10.1111/j.1545-5300.2006.00091.x. [DOI] [PubMed] [Google Scholar]
  26. NIMH Multisite HIV Prevention Trial Group. The NIMH multisite HIV prevention trial: Reducing HIV sexual behavior risk. Science. 1998;280:1889–1894. doi: 10.1126/science.280.5371.1889. [DOI] [PubMed] [Google Scholar]
  27. Noar S, Morokoff P. The relationship between masculinity ideology, condom attitudes, and condom use: Stage of change; a structural equation modeling approach. International Journal of Men’s Health. 2002;1:43–55. [Google Scholar]
  28. Nobles WW, Goddard LL, Gilbert DJ. Culturecology, women, and African-centered HIV prevention. Journal of Black Psychology. 2009;35(2):228–246. [Google Scholar]
  29. Pyeatt M, Tirado A. Nia facilitator’s guide. Dallas, TX: University of Texas Southwestern; 2008. [Google Scholar]
  30. Resnicow K, Soler R, Braithwaite R, Ahluwalia JS, Butler J. Cultural sensitivity in substance use prevention. Journal of Community Psychology. 2000;28:271–290. [Google Scholar]
  31. Santisteban DA, Suarez-Morales L, Robbins MS, Szapocznik J. Brief strategic family therapy: Lessons learned in efficacy research and challenges to blending research and practice. Family Process. 2006;45:259–271. doi: 10.1111/j.1545-5300.2006.00094.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Uebelacker LA, Hecht J, Miller IW. The family check-up: A pilot study of a brief intervention to improve family functioning in adults. Family Process. 2006;45:223–236. doi: 10.1111/j.1545-5300.2006.00092.x. [DOI] [PubMed] [Google Scholar]
  33. Villarruel AM, Jemmott LS, Jemmott JB. Designing a culturally based intervention to reduce HIV sexual risk for Latino adolescents. Journal of the Association of Nurses in AIDS Care. 2005;16:23–31. doi: 10.1016/j.jana.2005.01.001. [DOI] [PubMed] [Google Scholar]
  34. Villarruel AM, Jemmott JB, Jemmott LS. A randomized controlled trial testing an HIV prevention intervention for Latino youth. Archives of Pediatric Adolescent Medicine. 2006;160:1–6. doi: 10.1001/archpedi.160.8.772. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Williams JK, Wyatt GE, Resell J, Peterson J, Asuan-O’Brien A. Psychosocial issues among gay- and non-gay-identifying HIV-seropositive African American and Latino MSM. Cultural Diversity and Ethnic Minority Psychology. 2004;10(3):268–286. doi: 10.1037/1099-9809.10.3.268. [DOI] [PubMed] [Google Scholar]
  36. Wilton L, Herbst JH, Coury-Doniger P, Painter TM, English G, Alvarez ME, et al. Efficacy of an HIV/STI prevention intervention for black men who have sex with men: Findings from the Many Men, Many Voices (3MV) project. AIDS and Behavior. 2009;13:532–544. doi: 10.1007/s10461-009-9529-y. [DOI] [PubMed] [Google Scholar]
  37. Wingood GM, DiClemente RJ. The ADAPT-ITT model: A novel method of adapting evidence-based HIV interventions. Journal of Acquired Immune Deficiency Syndromes. 2008;47:S40–S46. doi: 10.1097/QAI.0b013e3181605df1. [DOI] [PubMed] [Google Scholar]
  38. Wyatt G. Enhancing cultural and contextual intervention strategies to reduce HIV/AIDS among African Americans. American Journal of Public Health. 2009;99:1941–1945. doi: 10.2105/AJPH.2008.152181. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Yuen RK. Unpublished doctoral dissertation. Loyola University; Chicago, Illinois: 2004. The effectiveness of culturally tailored interventions: A meta-analytic review. [Google Scholar]

RESOURCES