Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Nov 9.
Published in final edited form as: Contemp Clin Trials. 2015 Nov 9;45(0 0):244–251. doi: 10.1016/j.cct.2015.10.005

Passing the Baton: Community-based ethnography to design a randomized clinical trial on the effectiveness of oral pre-exposure prophylaxis for HIV prevention among Black men who have sex with men

Jonathan Garcia a,*, Paul W Colson b,c, Caroline Parker b, Jennifer S Hirsch b
PMCID: PMC4674295  NIHMSID: NIHMS736931  PMID: 26476286

Abstract

Although HIV interventions and clinical trials increasingly report the use of mixed methods, studies have not reported on the process through which ethnographic or qualitative findings are incorporated into RCT designs. We conducted a community-based ethnography on social and structural factors that may affect the acceptance of and adherence to oral pre-exposure prophylaxis (PrEP) among Black men who have sex with men (BMSM). We then devised the treatment arm of an adherence clinical trial drawing on findings from the community-based ethnography. This article describes how ethnographic findings informed the RCT and identifies distilled themes and findings that could be included as part of an RCT. The enhanced intervention includes in-person support groups, online support groups, peer navigation, and text message reminders. By describing key process-related facilitators and barriers to conducting meaningful mixed methods research, we provide important insights for the practice of designing clinical trials for ‘real-world’ community settings.

Keywords: HIV, pre-exposure prophylaxis, adherence trial, ethnography, men who have sex with men, African American

INTRODUCTION

HIV prevention researchers have argued that the next generation of interventions should focus on high impact combination prevention, which would integrate behavioral, biomedical and structural approaches to preventing infection among those most at-risk (13). Pre-exposure prophylaxis (PrEP) represents a promising tool for high impact prevention: recent studies have demonstrated the efficacy of daily oral antiretroviral (ARV) medications to prevent HIV acquisition in men who have sex with men (MSM) and discordant couples, although there have been discrepant findings in women (46). However, PrEP’s effectiveness, particularly in the populations most vulnerable to HIV infection (who may also face significant structural barriers to health systems engagement) depends on its coordinated delivery with behavioral and community-based prevention interventions (710).

Black men who have sex with men (BMSM) face among the highest incidence of HIV infection in the US (1114). Although biomedical prevention methods such as PrEP offer an important new method of protection, BMSM face structural and sociocultural barriers to engagement with health services, as well as vulnerabilities related to high community viral loads and dense sexual networks (15,16). BMSM constitute a group likely to benefit from PrEP, but to fulfill this promise it is critical to understand better the multidimensional factors that influence uptake and acceptability of PrEP among Black MSM, as well as the optimal components of a PrEP package of care that will promote adherence.

The challenge of providing PrEP to Black MSM in the current environment of HIV incidence and health services provides justification for research that calls upon the combined strengths of qualitative, quantitative and clinical methods of inquiry. Mixed methods have grown in popularity in public health and medicine for intervention development (17) and, to a lesser extent, in clinical trials (1821). A recent systematic review of 30 RCTs that employed qualitative methods found that more than two thirds of studies did not refer to any specific approach, while approximately one third mentioned theoretically-informed methods such as grounded theory, ethnography, action research, and narrative approaches (22), suggesting that ethnographic findings have been only marginally integrated in designing clinical trials. In interdisciplinary studies (e.g. with epidemiologists and social scientists), there was often no evidence of integration at the level of interpretation of findings, and the nature and extent of collaboration between ethnographic and clinical trial teams is rarely documented (23).

This article makes two contributions to the literature on the incorporation of ethnography into RCTs. First, it provides an example of the use of ethnographic research to design an intervention that is used in an RCT. Second, it discusses the process through which ethnographic findings were applied to the design; while mixed-methods research is increasingly viewed as an important methodological strategy in health research, publications infrequently describe the actual process through which the qualitative component of the research shapes the clinical component. To fill that gap, we briefly describe the community ethnography, and then discuss the process of using ethnographic findings to design the treatment arm in an adherence RCT.

METHODS

Ethnographic Component

This component focused on describing the critical behavioral domains relevant to BMSM engagement with a PrEP package of care (health systems engagement, sexual relationships, and HIV prevention practices). Ethnography has been a highly appropriate approach to design and implement culturally appropriate prevention methods, although its use has not been well documented in clinical trials (17,2428). Ethnography, which often includes a range of specific data collection methods, offers methodological strengths that include: (1) long term repeated interaction with participants, allowing the ethnographer to assess and re-assess initial interpretations, improving the validity of findings compared to single interviews or focus groups; (2) the ability to check narrative data through direct observation of actual behavior – or the ability to triangulate between what people say they do and what they actually do; and (3) the engagement of local communities in the construction of culturally appropriate clinical trials (21,24,29). Data collection methods included repeated in-depth interviews with 31 BMSM (3 90-minute sessions per participant), 17 60-minute key informant interviews, and 11 months of participant observation documented in field notes, maps, and systematic jottings. Table 1 describes each of these methods. The lead ethnographer (first author) conducted in-depth and key informant interviews and participant observation. His identity as a racial-ethnic minority and a gay man facilitated development of rapport with communities. All procedures were approved by the Columbia University Medical Center Institutional Review Board.

Table 1.

Ethnographic Methods

Data collection method Specific data elicited Sample description
Participant Observation Organizational behavior, group composition, ways people discuss sexuality, race, gender, age, class;/interaction, spoken rules of conduct and implicit cultural norms expressed, enforced, followed and navigated. 11 months of observation in: private spaces (homes, parties); public spaces (parks, streets, events); virtual spaces (chat rooms, blogs, social media); and institutions (community organizations, health centers, religious institutions).
Key Informant Interviews Organizational mission; role in organization/group/community; knowledge and attitudes about black MSM, HIV vulnerability, institutions and networks available to Black MSM; views on PrEP and other for HIV/STI prevention 17 informants, including 2 physicians; 3 mental health providers; 4 community organization program administrators; 5 outreach workers; 3 community mobilizers
In-depth Interviews Session 1: History of family relations, coming of age, education, housing, making money, friends; community, recreation
Session 2: Sexual history, including desire, casual and steady relations, sexual identity and racial identity
Session 3: Perceptions of health and risk; practices and attitudes about medications and seeking health services; knowledge and attitudes about HIV prevention
31 participants

The methods described in Table 1 were chosen to describe individual experiences and behavior, as well as community institutions and structures. Our analytic framework reflects the importance of understanding both cultural and structural influences on individual behavior in order to understand barriers to uptake of and adherence to biomedical prevention.

Analytic framework

The community ethnography, which sought to capture ‘real-world’ contextual factors that shape individual behavior, was grounded in the social ecological framework (30,31). This framework examines individual-level factors (behaviors, attitudes), social factors (social risk, life projects, social networks) and institutional factors (community organizations, health systems, the labor market, the housing context) to understand variation in individual behaviors. The goal was to understand the broad range of factors at each level of the ecological framework that might shape why men would seek out and be adherent to PrEP, in order to develop an adherence intervention that addressed those factors. The ethnographic data collection was designed to collect information related to individual-level knowledge, attitudes, and behaviors, as well as three theory-driven community-level domains: social risk, life projects, and social networks. Much research on why people do not adopt health-enhancing practices (e.g., abstain from sex, use condoms or clean needles, or seek anti-retroviral treatment) and/or continue to engage in behaviors potentially detrimental to their health (e.g., unprotected intercourse) has conceptualized risk rather narrowly as “health risk” (3236). The concept of social risk, in contrast, points to the fact that the fear of losing important social resources (e.g., relationships or reputation) is often more immediate than the fear of illness. Therefore in any given context, the logical course of action may be to prioritize continued access to these social resources over health considerations (3742). The notion of social risk necessarily implies an exploration of stigma and discrimination, since the intense and multi-layered discrimination faced by Black MSM is part of what makes the possibility of disclosure of sexual behavior socially risky.

The related notion of life projects sets out a rubric within which to explore the culturally variable and socially-structured nature of what people want, and what others expect them to achieve, from their lives (4346). The concept has already been used successfully to explain how people engage with anti-retroviral therapy, (4749) but it has much greater potential to shed light on the ways that sexual relationships and engagement with health care services fit more broadly into people’s efforts to build a satisfying life.

Social risk and life projects are embedded within and shaped by social networks. Family members, peers, and sex partners shape exposure to, attitudes toward, and uptake of, health promotion interventions; and interventions targeting MSM have relied on those social networks and connections to influential individuals as a means to obtaining community-level reductions in HIV risk (50).

Analytic Methods

The analytic process began with the selection of an analytic framework that would capture multiple dimensions of the community where the clinical trial would take place, as described above. Through monthly full-team meetings that brought together the community ethnography subgroup and the clinical trial subgroup, we discussed recruitment, data collection instruments, and ongoing findings. We presented on-going data analysis reports to the clients and providers in community advisory boards (CABs) to validate salient themes and perform member checking (i.e., community members collaborated in data analysis to check its veracity and applicability). The client CAB participated in group data analysis activities, including the construction of a detailed map of New York City that described the social risks and health risks associate with different areas. The ethnographic team developed a codebook that was developed based on domains (code families) derived from the interview guides and through open coding. Two members of the research team (first and third authors) coded interview data and field notes, with inter-coder agreement greater than 80%. Data were analyzed and triangulated between and across cases and methods using Atlas.ti 7.0 qualitative software. This analysis was then used to inform an evidence-based PrEP adherence intervention relevant to the lives of BMSM.

Clinical Trial Component

This component will, in a randomized controlled trial, test the effectiveness of an enhanced adherence package versus standard adherence PrEP package in a community-based setting; examine healthcare providers’ knowledge, attitudes and practices (KAP) related to PrEP including acceptability and feasibility; examine PrEP acceptability and feasibility for at-risk Black MSM; and identify characteristics associated with adherence to PrEP. We hypothesize that experimental participants will have greater adherence and reduced sexual risk behaviors.

RESULTS

Ethnographic findings

Findings, which provided insights about how social context could affect PrEP adherence, are grouped here into six domains (Table 2). First, structural issues, including limited access to housing, racial discrimination from police, unemployment, food and education, emerged as factors that contribute to HIV vulnerability, through undermining access to health services and through promoting sexual risk behaviors including sex work and condomless sex. While all key informants (community stakeholders and HIV advocates) emphasized the importance of addressing structural issues, most felt that it was rarely possible to incorporate structural approaches into existing interventions.

Table 2.

Summary of Main Findings from Community Ethnography

Summary Representative Quote Distilled Theme
Structural
 Most men were unemployed, experienced limited educational opportunities, and had limited access to health services at community- based clinics. “I feel like I have bigger fish to fry…Like, I need a job. And then with a job, then I could look into the resources on how to get some type of health coverage.” (27 year-old) Lack of non-medical services (employment, housing, other benefits)
Life Projects
 Most men wanted to have careers, get a GED or higher education, to have stable housing; a few planned to be fathers; some considered healthiness a goal; leisure included going to parties, playing sports, dancing. “Planning a better job, number one. Finding my own place, number two, and those two things and possibly go back to school… If I do find a new place, making sure jobs are stable and the money is decent to support myself in paying the rent and stuff.” (47 year-old) Goal-oriented self-care needed for adherence
Social Networks
 Many men had been rejected by their families, churches, and peers at school because of their homosexuality. “Negativity comes with being homosexual from heterosexuals, from religion, from your grandma, from your aunt, from your uncle’s buddy, from your uncle – they could say everything about faggots and lesbians… It’s not supported in the black community; it’s definitely not supported through church.” (46 year-old) Lack of social support from trusted individual
Social Risk
 Straight, discreet, and bisexual men, and some gay men were cautious avoiding feminine men in public, having sex in parks, and meeting men online for the sake of reputation, social status and to maintain social support. “It’s threatening to be with a feminine man… because you don’t want people looking at you in a certain way. Stereotypes are for feminine men. They tend to be loud, obnoxious, ghetto, flamboyant, and just messy.” (18 year-old) Avoiding stigma (HIV, femininity, sexuality) led to health risk
HIV Prevention
 Majority had condomless sex in past year; few had steady relationships, and some had both male and female partners concurrently. Several enjoy sex on alcohol, marijuana, cocaine, and poppers; most met partners through social media. All were had knowledge of transmission, most tested every three months, most worried about HIV-related stigma in the black community. “If you are HIV positive, people in Harlem, they don’t refer to you being HIV positive and/or having AIDS diagnosis, they call it the monster … they’re looked down upon it…You don’t say you have the monster if you have Herpes…. if you have Herpes, you got burned. If you have chlamydia, you got burned.” (39 year-old) Felt racism and HIV stigma, poor risk reduction strategies
PrEP
 Few heard about PrEP, and once described some thought it would be an effective prevention tool, although several worried about risk of acquiring other STIs, increased sexual risk behavior, drug side effects, and mistrusted medical institutions based on perceived institutional racism. “I feel like if a person is like, they’re using it to maximize their protection, they’re like, ‘Yes, I’m going to use this. I’m going to use condoms. I’m going to get tested regularly,’ then I’m all for it. But I don’t want it to be the kind of thing where it’s like, ‘Okay, I have Truvada, so I don’t need to worry about using condoms,’ … since HIV is like considered such a big deal like nationwide and like worldwide, people forget that there are still other things that can be spread, like there are other STIs that can really damage your body.” (17 year-old) Reluctance to take pills, poor PrEP knowledge, medical mistrust

Second, men’s life projects reflected their desire to overcome many of these structural challenges. The most commonly mentioned life projects included getting a GED, finding stable housing, finding full-time work, and establishing a career. Short-term goals included obtaining enough money to hang out with friends, continuing to use drugs, and engaging in casual sex. Men who engaged in health-protective behaviors frequently made connections between condom use and realizing their broader life projects of “self-actualization” such as being “good fathers” and becoming comfortable with their homosexuality. Based on these observations, we recommended that education about PrEP be framed as integral to achieving life projects, and that the content of several components of the intervention should include creating spaces (online, through in-person support and peer navigators) that focus on PrEP as a way to achieve life projects that are most important to men.

Third, social and sexual networks affected how BMSM engaged with HIV prevention. Men who were rejected by their families or religious institutions reported using opportunistic sexual relationships to secure material and social resources including housing, food, comfort and reassurance of self-worth. Moreover, peer networks had a critical effect on men’s engagement in risky practices, through the enforcement of cultural norms of masculinity that emphasized invincibility and that were linked to not seeking health services. Lack of support from social networks or fear of losing social status severely limited men’s ability to negotiate condom use. Findings in this domain informed several components of the treatment arm of the RCT by providing opportunities for BMSM to seek support online (i.e., to maintain discretion about their sexualities) and in groups of like-minded peers.

Fourth, all men discussed ways in which a concern with social risk, rather than health risks, shaped their behavior. Social risks included potentially adverse social consequences of behaviors, such as disclosing sexuality, failing to perform socially expected masculine norms, and exposing social class position. For many, disclosing their homosexuality meant risking the support of their families and losing housing stability. This risk was particularly acute for young men who depended on one of their parents and for men in primary relationships with women. Spatial context shaped many men’s perception of social risk. For example, observation in streets revealed that feminine men risked being discriminated against and being verbally or physically assaulted, and many men who described themselves as masculine avoided the “threat” of being seen with feminine men or with men who were socially open about their homosexuality. Many men were reluctant to discuss serostatus because of HIV-related stigma and gossip within sexual networks. Findings in this domain alerted us to BMSM’s need for “safe spaces”, and shaped the strategies to circumvent social risk employed by the treatment arm.

Fifth, men generally had HIV and STI tests, but did not feel the need to go to the doctor for non-HIV related check-ups. For most men, the incentive for HIV testing resulted from inconsistent condom use. Many men tested every three months and in one case every month. In some cases, men tested because they were linked to drop-in centers or CBOs that encouraged it. Some key informants (HIV outreach workers) claimed that the services and resources available to people who are HIV+, but not to the community in general, served as an incentive to become infected, and told stories of men who did not care if they were infected because they saw HIV as a non-issue (no more campaigns of people dying of AIDS) or because they would have more public support (e.g., housing, case management) if they seroconverted. Men’s expressions of greater concern about HIV than about other potential health problems provided key insights about how perceptions of risk could affect to adherence to PrEP. This informed our approach to HIV and PrEP-related education in the treatment arm.

Lastly, most men had not heard of PrEP, and those who had commonly confused it with post-exposure prophylaxis (PEP). Among those who knew about PrEP, some feared that users would face HIV-related stigma owing to Truvada’s status as a HIV medication, and some thought it would lead to increased condomless sex and exposure to other STIs. Moreover, many men reported not wanting to take pills because of a lack of control over their side effects. Some refused to take medications for chronic conditions (high cholesterol, depression and diabetes) because they did not trust medicine or doctors, often based on perceptions of racism. Many key informants attributed mistrust to the history of unethical clinical trials, “overmedicating black men with mental illness” and to slavery. These findings revealed the kinds of discussions that would be most helpful in addressing men’s negative perceptions of medicine and medical institutions in order to improve adherence to PrEP.

Using Ethnographic Findings to Design an Adherence Support Intervention

The literature on medication adherence, particularly with respect to antiretroviral medications, offers several principles for effective intervention design. Successful programs generally offer combination interventions rather than a single method (5153). While thorough patient education is essential upon treatment initiation (52), it is essential to offer strategies beyond education, including addressing motivation (54), peer support (53,55,56), and reminders (53).

Some key organizational elements of our study facilitated the process of translating findings from the community ethnography into components of the adherence RCT. By interviewing leaders and staff at the clinic where the RCT would take place, and by attending meetings of the community clinic’s “PrEP Task Force,” we learned about the culture of the institution and the types of programs already provided as “standard of care.” This allowed us to devise an intervention that provided standard care as well as the additional resources mentioned in Table 3. We also presented ethnographic findings to the staff at the community health clinic to engage them in the process of analyzing our findings and devising the intervention.

Table 3.

Intervention Components

PURPOSE ACTIVITIES
1) PEER NAVIGATORS (MSM who knows his status, may have taken/is taking PrEP, and is supportive of PrEP adherence)
  • provide social support and navigation

  • provide support about discussing PrEP with partners, family, or friends

  • provide information about HIV risk reduction, PrEP, PEP, services, referrals, etc.

  • address barriers to services

  • help identify referrals for needed non-medical services

  • accompany clients to initial appointment at referred organizations, and subsequent appointments as needed

  • informal counseling about adherence: adherence tips, discussion of specific barriers to adherence

  • remind clients about appointments via telephone/SMS/email

  • assist with efforts to re-engage missing participants in care

2) STUDY-SPECIFIC SUPPORT GROUPS (face to face)
  • provide social support (informational support, emotional support, esteem support, network support)

  • promote social learning

  • promote critical awareness and ways to confront stigma (related to HIV, sexuality masculinity, race, class)

  • provide information about PrEP, PrEP side effects, PEP, acute HIV infection, HIV risk reduction

  • discuss adherence

  • share information about needed non-medical services

  • discuss larger issues (“life projects”)

  • decide on content of SMS messages as a group

  • Initial sessions will focus more on health information about HIV, PrEP, acute HIV infection

  • As group evolves, it will determine agenda

  • New members may need “booster” of health education information already provided to entire group

3) SOCIAL NETWORKING (e.g., invitation-only on-line discussion group)
  • provide social support, group connectedness

  • provide information about HIV risk reduction, PrEP, PEP, services, referrals, etc.

  • Share information on non-medical services

  • Share information on PrEP, PrEP side effects, PEP, acute HIV infection, HIV risk reduction

  • Discuss PrEP adherence, including specific problems with adherence

  • When postings require a more detailed response than available on webpage, members can be encouraged to attend next on-site support group to discuss

  • Post reminders for on-site support group meetings and activities

4) SMS MESSAGING
  • send health-supportive messages (“cues to action”)

  • each participant to receive SMS texts from study

  • content of messages to be decided in study-specific support groups

In addition, the client and provider community advisory boards (CABs) were critical to our study. The CABs, which met frequently throughout the ethnography and has continued to meet throughout the clinical trial, provided a sounding board for recruitment strategies, refining instruments, and data interpretation. For example, drawing on ethnographic findings about sexual discretion and stigma, the CAB suggested broadening the inclusion criterion for “high-risk” men to include men who reported either unprotected insertive and receptive anal sex. Moreover, full team meetings, where the ethnography team and clinical team came together, facilitated discussion about what types of information were important for informing the RCT and reviewing current PrEP interventions. These were critical for “passing the baton,” which initially consisted of a report of ethnographic findings. The outline for this report was co-produced by both teams, and we held full team and CAB meetings to review the findings. We presented the preliminary results to the 10 staff members at a local community health clinic serving predominantly black and Latino LGBT persons. Through this process we were able to distill ethnographic findings, as shown in Table 2, into components of the clinical trial.

Preliminary results from the community ethnography highlighted the need for intervention at multiple levels to address challenges related to PrEP acceptance and adherence, including policy interventions, community-level interventions, and behavioral interventions. Only some of these findings, however, were amenable to direct incorporation in an RCT to support individual adherence to PrEP. The ethnographic component was critical to designing a culturally meaningful and tailored RCT that incorporated distilled themes into 4 components (Figure 1).

Figure 1.

Figure 1

How ethnographic findings were reflected in the design of the RCT treatment arm

The ethnography identified a number of issues which are important for the lives of black MSM but which could not be directly addressed in an adherence-based intervention in a clinical setting. While the adherence intervention will be unable to provide direct solutions to structural problems such as unemployment, housing insecurity, limited educational opportunities, these topics are included addressed in the in-person and on-line support groups and through peer navigation. Other challenges to adherence, such as difficulties in obtaining health insurance, the cost of PrEP, and possible variations in provider acceptance or competence are either obviated (i.e., the study will provide PrEP free of charge and will seek health insurance for any uninsured participants) or become untestable in an experimental setting located with a community-based clinic (i.e., the study will partner with one community agency with a small number of physicians).

  1. Peer Navigation – Navigators play several roles in assisting individuals with adherence issues (5658). As people who have experienced similar life challenges, peer navigators can serve as role models (59). Peers can answer questions about taking PrEP and side effects that may be difficult to discuss in a group setting and can provide information and accompany participants to access non-medical programs and services, which were identified as a significant need in the ethnography. Most importantly, they can use their personal experience to discuss strategies to avoid challenges to taking PrEP, such as HIV- and sexuality-related stigma.

  2. Support Groups – In-person support groups offer the opportunity to provide useful information while also facilitating discussion of emotional issues (5658). Specific information is being provided about PrEP including the PrEP regimen itself, PrEP side effects, Post-Exposure Prophylaxis (PEP), acute HIV infection, and HIV risk reduction. Support groups are designed to normalize PrEP with a network of peers to address stigma related to PrEP use (sex negativity, shaming) and provide ways to navigate social pressures. Groups address the lack of knowledge and misconceptions identified in the ethnography, including participants’ mistrust of medication and medical institutions. They provide a safe space to discuss stigma (related to racism, sexuality, masculinity), life projects and structural challenges, such as employment instability, housing insecurity and limited educational opportunities (60). Groups also provide information on how to navigate available resources to address structural issues.

  3. Social Networking – The ethnographic findings suggest that BMSM did not disclose their homosexuality to social networks other than through social media. The intervention offers an interactive website that functions as an on-line support group. This website offers an opportunity for participants who do not attend in-person support group meetings to discuss the same topics through a virtual community (61,62). This may be especially important for men who are uncomfortable talking about sexuality and HIV-related stigma due to social risk.

  4. Text Message Reminders - Participants are reminded through Short Messaging Service (SMS) texts to adhere to daily PrEP use and to attend medical appointments. This modality has proven effective in supporting adherence, along with retention in care (63,64). Reflecting ethnographic findings, these messages contextualize “staying healthy” as being “strong”, framing PrEP adherence in relation to ideologies of masculinity and men’s life projects. The content of the SMS messages will be decided through discussion in the in-person support groups.

Table 3 lists the purpose and specific activities associated with each of these four interventions. Consistent with the adherence literature, the interventions address the importance of forming new habits, offering social support individually and through groups, and supporting retention in care, along with connections to needed external services (49, 50).

DISCUSSION

Overall, our ethnographic findings offered a rich contextual description of the lives of BMSM, revealing issues that we could address through a clinical trial as well as helping us think critically about what could be done to provide opportunities for men to address issues, such as housing, educational opportunities and employment, for which we could not provide direct solutions in a randomized trial. One key challenge to incorporating ethnographic findings into the design of the RCT was the project’s timeline. Ethnographic data analysis can frequently take as long as the collection of the data itself, and the project timeline required a much more rapid development of recommendations for the intervention. This was addressed by emphasizing the descriptive findings, which were summarized in the aforementioned report, before moving onto the development of more analytical and theory-driven approaches to working with the data (56,60,65). In addition, efforts to engage staff at the community-based clinic were challenged by staff-turnover during the study. Through key informant interviews we found this was a common problem at clinics that provide HIV prevention, treatment and care services, underscoring the need for regular engagement with staff members to compensate for turnover.

This article describes key methodological processes seldom described in the literature on mixed-methods studies or clinical trial design. First, we identified how ethnographic findings informed components of the RCT along six domains. Although some findings could be addressed in the RCT, structural factors that have been documented drivers of the HIV epidemic (e.g., unemployment, unstable housing) could not be directly addressed by a clinical trial focused on individual adherence to treatment.

We also noted elements of our research and RCT design process that facilitated and challenged “passing the baton” from the community ethnography to the clinical team. Key insights for conducting mixed-methods research in the context of RCT design included:

  • Having client and provider community advisory boards with members who participate in both the ethnographic and clinical components of the study

  • Establishing and re-establishing relationships with staff and stakeholders in the context of the community-based health clinic

  • Considering more fully the time required to analyze ethnographic and qualitative findings to effectively inform the design of the clinical trial

  • Holding regular full team meetings throughout both components of the study, so that the handoff is a continuous dialogue and not a one-time communication.

Our project goes beyond the individual approach so commonly found in clinical research, exploring the tensions between ideology and individual behavior and attending to how structural factors shape the range of choices available to individuals. Lewin et al. (2009) indicate that “qualitative studies remain relatively uncommon alongside trials of complex interventions” and qualitative studies within clinical trials lacked rigor as most “had important methodological shortcomings and the findings were poorly integrated with those of the trials”(22). Overall, ethnography has seldom been used to design the content of intervention in clinical trials (e.g., the “treatment”); rather, it has been used more frequently to study the context under which the RCT will take place. Few studies employed participant observation, the method that clearly distinguishes other qualitative work from ethnographic research. Lesser et al. (2005) used ethnographic methods (including in-depth individual interviews and participant observation) to inform the design of a HIV prevention program for teenage parents (66). Some studies have used other qualitative approaches including focus groups and in-depth interviews to describe knowledge, beliefs and behaviors to inform the design behavioral interventions in RCTs to prevent STDs among minority women (67,68). Others have used less intensive formative research prior to RCT implementation, interviewing only staff and focusing observation at the clinic where the RCT would take place (69).

Our mixed methods strategy features two key innovations relevant more generally to integrating ethnographic research into RCTs. First, the ethnographic research was conducted prior to and in parallel with the tailoring of the RCT. This allowed for tailoring the content and components of the RCT by incorporating an understanding of barriers to success into the intervention itself, rather having the ethnographic research take place after the technology is made available to explain why (almost inevitably) there seem to be community-level barriers to uptake. Second, the overall study offers access to data on how PrEP rollout might work in a naturalistic environment, enhancing the ecological validity of the research. The goals of engaged ethnographic research included informing RCT design, identifying evidence that the intervention will have the desired effect, reviewing the theoretical basis of the intervention, and identifying potential barriers to successful RCT implementation in a community health clinic.

Acknowledgments

Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under award number R01 MH098723. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Hankins CA, de Zalduondo BO. Combination prevention: a deeper understanding of effective HIV prevention. AIDS. 2010 Oct;24(Suppl 4):S70–80. doi: 10.1097/01.aids.0000390709.04255.fd. [DOI] [PubMed] [Google Scholar]
  • 2.CDC. High-Impact HIV Prevention: CDC’s Approach to Reducing HIV Infections in the United States [Internet] Centers for Disease Control and Prevention; 2011. p. 12. cited 2014 Aug 8. Available from: http://www.cdc.gov/hiv/pdf/policies_NHPC_Booklet.pdf. [Google Scholar]
  • 3.Grossman CI, Purcell DW, Rotheram-Borus MJ, Veniegas R. Opportunities for HIV combination prevention to reduce racial and ethnic health disparities. Am Psychol. 2013;68(4):237–46. doi: 10.1037/a0032711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Baeten JM, Donnell D, Ndase P, Mugo NR, Campbell JD, Wangisi J, et al. Antiretroviral Prophylaxis for HIV-1 Prevention among Heterosexual Men and Women. N Engl J Med. 2012 Aug 2;367(5):399–410. doi: 10.1056/NEJMoa1108524. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Grant RM, Lama JR, Anderson PL, McMahan V, Liu AY, Vargas L, et al. Preexposure Chemoprophylaxis for HIV Prevention in Men Who Have Sex with Men. N Engl J Med. 2010 Dec 30;363(27):2587–99. doi: 10.1056/NEJMoa1011205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Thigpen MC, Kebaabetswe PM, Smith DK, Segolodi TM, Chillag K, Kasonde M, et al. Daily oral antiretroviral use for the prevention of HIV infection in heterosexually active young adults in Botswana: results from the TDF2 study. 2011 [Google Scholar]
  • 7.Kim SC, Becker S, Dieffenbach C, Hanewall BS, Hankins C, Lo Y-R, et al. Planning for pre-exposure prophylaxis to prevent HIV transmission: challenges and opportunities. J Int AIDS Soc. 2010 Jul 12;13:24. doi: 10.1186/1758-2652-13-24. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Mayer KH, Venkatesh KK. Chemoprophylaxis for HIV Prevention: New Opportunities and New Questions. J Acquir Immune Defic Syndr 1999. 2010 Dec 15;55(Suppl 2):S122–7. doi: 10.1097/QAI.0b013e3181fbcb4c. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Michael NL. Oral Preexposure Prophylaxis for HIV — Another Arrow in the Quiver? N Engl J Med. 2010 Dec 30;363(27):2663–5. doi: 10.1056/NEJMe1012929. [DOI] [PubMed] [Google Scholar]
  • 10.Underhill K, Operario D, Mimiaga MJ, Skeer MR, Mayer KH. Implementation science of pre-exposure prophylaxis: preparing for public use. Curr HIV/AIDS Rep. 2010 Nov;7(4):210–9. doi: 10.1007/s11904-010-0062-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Mayer KH, Wang L, Koblin B, Mannheimer S, Magnus M, del Rio C, et al. Concomitant Socioeconomic, Behavioral, and Biological Factors Associated with the Disproportionate HIV Infection Burden among Black Men Who Have Sex with Men in 6 U.S. Cities. In: Yang R, editor. PLoS ONE. 1. Vol. 9. 2014. Jan 31, p. e87298. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.van Griensven F, Stall RD. Racial disparity in HIV incidence in MSM in the United States: how can it be reduced? AIDS. 2014 Jan;28(1):129–30. doi: 10.1097/QAD.0000000000000097. [DOI] [PubMed] [Google Scholar]
  • 13.Maulsby C, Millett G, Lindsey K, Kelley R, Johnson K, Montoya D, et al. HIV Among Black Men Who Have Sex with Men (MSM) in the United States: A Review of the Literature. AIDS Behav. 2014 Jan;18(1):10–25. doi: 10.1007/s10461-013-0476-2. [DOI] [PubMed] [Google Scholar]
  • 14.Millett GA, Peterson JL, Flores SA, Hart TA, Jeffries WL, Wilson PA, et al. Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in Canada, UK, and USA: a meta-analysis. Lancet. 2012;28(380):9839. doi: 10.1016/S0140-6736(12)60899-X. [DOI] [PubMed] [Google Scholar]
  • 15.Malebranche DJ, Peterson JL, Fullilove RE, Stackhouse RW. Race and sexual identity: perceptions about medical culture and healthcare among Black men who have sex with men. J Natl Med Assoc. 2004 Jan;96(1):97–107. [PMC free article] [PubMed] [Google Scholar]
  • 16.Levy ME, Wilton L, Phillips G, Glick SN, Kuo I, Brewer RA, et al. Understanding Structural Barriers to Accessing HIV Testing and Prevention Services Among Black Men Who Have Sex with Men (BMSM) in the United States. AIDS Behav. 2014 May;18(5):972–96. doi: 10.1007/s10461-014-0719-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Creswell JW, Fetters MD, Plano Clark VL, Morales A. Mixed Methods Intervention Trials. In: Andrew S, Halcomb EJ, editors. Mixed Methods Research for Nursing and the Health Sciences [Internet] Oxford, UK: Wiley-Blackwell; 2009. pp. 159–80. cited 2015 May 26. Available from: http://doi.wiley.com/10.1002/9781444316490.ch9. [Google Scholar]
  • 18.Donovan J. Quality improvement report: Improving design and conduct of randomised trials by embedding them in qualitative research: ProtecT (prostate testing for cancer and treatment) study * Commentary: presenting unbiased information to patients can be difficult. BMJ. 2002 Oct 5;325(7367):766–70. doi: 10.1136/bmj.325.7367.766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Cooper C, O’Cathain A, Hind D, Adamson J, Lawton J, Baird W. Conducting qualitative research within Clinical Trials Units: Avoiding potential pitfalls. Contemp Clin Trials. 2014 Jul;38(2):338–43. doi: 10.1016/j.cct.2014.06.002. [DOI] [PubMed] [Google Scholar]
  • 20.Riley T, Hawe P, Shiell A. Contested ground: how should qualitative evidence inform the conduct of a community intervention trial? J Health Serv Res Policy. 2005 Apr 1;10(2):103–10. doi: 10.1258/1355819053559029. [DOI] [PubMed] [Google Scholar]
  • 21.Corneli AL, Piwoz EG, Bentley ME, Moses A, Nkhoma JR, Tohill BC, et al. Involving communities in the design of clinical trial protocols: The BAN Study in Lilongwe, Malawi. Contemp Clin Trials. 2007 Jan;28(1):59–67. doi: 10.1016/j.cct.2006.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ [Internet] 2009:339. doi: 10.1136/bmj.b3496. cited 2014 Sep 19. Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2741564/ [DOI] [PMC free article] [PubMed]
  • 23.de Salis I, Tomlin Z, Toerien M, Donovan J. Qualitative research to improve RCT recruitment: Issues arising in establishing research collaborations. Contemp Clin Trials. 2008 Sep;29(5):663–70. doi: 10.1016/j.cct.2008.03.003. [DOI] [PubMed] [Google Scholar]
  • 24.Parker R, Ehrhardt AA. Through an Ethnographic Lens: Ethnographic methods, comparative analysis, and HIV/AIDS research. AIDS Behav. 2001;5(2):105–14. [Google Scholar]
  • 25.Agar M. Speaking of ethnography [Internet] Beverly Hills: Sage Publications; 1986. p. 79. Available from: http://www.loc.gov/catdir/enhancements/fy0654/85050781-d.html http://www.loc.gov/catdir/enhancements/fy0654/85050781-t.html. [Google Scholar]
  • 26.Greene J. An ethnography of nonadherence: Culture, poverty, and tuberculosis in urban Bolivia. Cult Med Psychiatry. 2004;28(3):401–25. doi: 10.1023/b:medi.0000046429.55801.c8. [DOI] [PubMed] [Google Scholar]
  • 27.McCann S, Campbell M, Entwistle V. Recruitment to clinical trials: a meta-ethnographic synthesis of studies of reasons for participation. J Health Serv Res Policy. 2013 Oct 1;18(4):233–41. doi: 10.1177/1355819613483126. [DOI] [PubMed] [Google Scholar]
  • 28.Price N, Hawkins K. Researching sexual and reproductive behaviour: a peer ethnographic approach. Soc Sci Med. 2002;55(8):1325–36. doi: 10.1016/s0277-9536(01)00277-5. [DOI] [PubMed] [Google Scholar]
  • 29.Creswell JW. Designing and conducting mixed methods research. 2. Los Angeles: SAGE Publications; 2011. p. 457. [Google Scholar]
  • 30.Stokols D. Translating Social Ecological Theory into Guidelines for Community Health Promotion. Am J Health Promot. 1996 Mar;10(4):282–98. doi: 10.4278/0890-1171-10.4.282. [DOI] [PubMed] [Google Scholar]
  • 31.Wilson PA. A Dynamic-Ecological Model of Identity Formation and Conflict among Bisexually-Behaving African-American Men. Arch Sex Behav. 2008 Oct;37(5):794–809. doi: 10.1007/s10508-008-9362-9. [DOI] [PubMed] [Google Scholar]
  • 32.Amaro H. Love, sex, and power. Considering women’s realities in HIV prevention. Am Psychol. 1995 Jun;50(6):437–47. doi: 10.1037//0003-066x.50.6.437. [DOI] [PubMed] [Google Scholar]
  • 33.Gómez CA, Marín BV. Gender, culture, and power: Barriers to HIV-prevention strategies for women. J Sex Res. 1996 Jan 1;33(4):355–62. [Google Scholar]
  • 34.Salazar LF, Crosby RA, DiClemente RJ, Wingood GM, Lescano CM, Brown LK, et al. Self-esteem and theoretical mediators of safer sex among African American female adolescents: implications for sexual risk reduction interventions. Health Educ Behav Off Publ Soc Public Health Educ. 2005 Jun;32(3):413–27. doi: 10.1177/1090198104272335. [DOI] [PubMed] [Google Scholar]
  • 35.Wingood GM, DiClemente RJ, Mikhail I, Lang DL, McCree DH, Davies SL, et al. A randomized controlled trial to reduce HIV transmission risk behaviors and sexually transmitted diseases among women living with HIV: The WiLLOW Program. J Acquir Immune Defic Syndr 1999. 2004 Oct 1;37( Suppl 2):S58–67. doi: 10.1097/01.qai.0000140603.57478.a9. [DOI] [PubMed] [Google Scholar]
  • 36.Wingood GM, DiClemente RJ. Enhancing adoption of evidence-based HIV interventions: promotion of a suite of HIV prevention interventions for African American women. AIDS Educ Prev Off Publ Int Soc AIDS Educ. 2006 Aug;18(4 Suppl A):161–70. doi: 10.1521/aeap.2006.18.supp.161. [DOI] [PubMed] [Google Scholar]
  • 37.Hirsch JS, Meneses S, Thompson B, Negroni M, Pelcastre B, del Rio C. The Inevitability of Infidelity: Sexual Reputation, Social Geographies, and Marital HIV Risk in Rural Mexico. Am J Public Health. 2007 Jun;97(6):986–96. doi: 10.2105/AJPH.2006.088492. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Hirsch JS, Wardlow H, Smith DJ, Phinney H, Parikh S, Nathanson CA. The secret: Love, marriage, and HIV. vanderbilt university Press; Nashville: 2009. [Google Scholar]
  • 39.Parikh SA. The political economy of marriage and HIV: the ABC approach, “safe” infidelity, and managing moral risk in Uganda. Am J Public Health. 2007;97(7):1198. doi: 10.2105/AJPH.2006.088682. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Phinney HM. Rice is essential but tiresome; you should get some noodles”: DoiMoi and the political economy of men’s extramarital sexual relations and marital HIV risk in Hanoi, Vietnam. Am J Public Health. 2008 Apr;98(4):650–60. doi: 10.2105/AJPH.2007.111534. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Smith DJ. Modern Marriage, Men’s Extramarital Sex, and HIV Risk in Southeastern Nigeria. Am J Public Health. 2007 Jun;97(6):997–1005. doi: 10.2105/AJPH.2006.088583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Wardlow H. Men’s Extramarital Sexuality in Rural Papua New Guinea. Am J Public Health. 2007 Jun;97(6):1006–14. doi: 10.2105/AJPH.2006.088559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Kondo DK. Crafting Selves: Power, Gender, and Discourses of Identity in a Japanese Workplace. University of Chicago Press; 2009. p. 366. [Google Scholar]
  • 44.Social Reproduction and History in Melanesia [Internet] Cambridge University Press; [cited 2015 May 12]. Available from: http://www.cambridge.org/us/academic/subjects/anthropology/social-and-cultural-anthropology/social-reproduction-and-history-melanesia-mortuary-ritual-gift-exchange-and-custom-tanga-islands. [Google Scholar]
  • 45.Wiley. Life’s Work: Geographies of Social Reproduction - Katharyne Mitchell, Sallie A. Marston, Cindi Katz [Internet] [cited 2015 May 12]. Available from: http://www.wiley.com/WileyCDA/WileyTitle/productCd-1405111348.html.
  • 46.Wikan U. The Self in a World of Urgency and Necessity. Ethos. 1995 Sep 1;23(3):259–85. [Google Scholar]
  • 47.Gore-Felton C, Rotheram-Borus MJ, Weinhardt LS, Kelly JA, Lightfoot M, Kirshenbaum SB, et al. The Healthy Living Project: an individually tailored, multidimensional intervention for HIV-infected persons. AIDS Educ Prev Off Publ Int Soc AIDS Educ. 2005 Feb;17(1 Suppl A):21–39. doi: 10.1521/aeap.17.2.21.58691. [DOI] [PubMed] [Google Scholar]
  • 48.Smith DJ, Mbakwem BC. Life projects and therapeutic itineraries: marriage, fertility, and antiretroviral therapy in Nigeria. AIDS Lond Engl. 2007 Oct;21( Suppl 5):S37–41. doi: 10.1097/01.aids.0000298101.56614.af. [DOI] [PubMed] [Google Scholar]
  • 49.Smith DJ, Mbakwem BC. Antiretroviral therapy and reproductive life projects: mitigating the stigma of AIDS in Nigeria. Soc Sci Med 1982. 2010 Jul;71(2):345–52. doi: 10.1016/j.socscimed.2010.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Kelly JA, St Lawrence JS, Diaz YE, Stevenson LY, Hauth AC, Brasfield TL, et al. HIV risk behavior reduction following intervention with key opinion leaders of population: an experimental analysis. Am J Public Health. 1991 Feb;81(2):168–71. doi: 10.2105/ajph.81.2.168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Marcus JL, Buisker T, Horvath T, Amico KR, Fuchs JD, Buchbinder SP, et al. Helping our patients take HIV pre-exposure prophylaxis (PrEP): a systematic review of adherence interventions. HIV Med. 2014 Aug;15(7):385–95. doi: 10.1111/hiv.12132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Golub SA, Operario D, Gorbach PM. Pre-exposure prophylaxis state of the science: empirical analogies for research and implementation. Curr HIV/AIDS Rep. 2010 Nov;7(4):201–9. doi: 10.1007/s11904-010-0057-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Chaiyachati KH, Ogbuoji O, Price M, Suthar AB, Negussie EK, Bärnighausen T. Interventions to improve adherence to antiretroviral therapy: a rapid systematic review. AIDS. 2014 Mar;28:S187–204. doi: 10.1097/QAD.0000000000000252. [DOI] [PubMed] [Google Scholar]
  • 54.Amico KR, Mansoor LE, Corneli A, Torjesen K, van der Straten A. Adherence Support Approaches in Biomedical HIV Prevention Trials: Experiences, Insights and Future Directions from Four Multisite Prevention Trials. AIDS Behav. 2013 Jul;17(6):2143–55. doi: 10.1007/s10461-013-0429-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Binford MC, Kahana SY, Altice FL. A Systematic Review of Antiretroviral Adherence Interventions for HIV-Infected People Who Use Drugs. Curr HIV/AIDS Rep. 2012 Dec;9(4):287–312. doi: 10.1007/s11904-012-0134-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Garcia J, Parker C, Parker RG, Wilson PA, Philbin M, Hirsch JS. Psychosocial Implications of Homophobia and HIV Stigma in Social Support Networks Insights for High-Impact HIV Prevention Among Black Men Who Have Sex With Men. Health Educ Behav. 2015 Aug;26:1090198115599398. doi: 10.1177/1090198115599398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.McDonald HP, Garg AX, Haynes RB. Interventions to Enhance Patient Adherence to Medication Prescriptions: Scientific Review. JAMA. 2002 Dec 11;288(22):2868. doi: 10.1001/jama.288.22.2868. [DOI] [PubMed] [Google Scholar]
  • 58.DiMatteo MR. Social Support and Patient Adherence to Medical Treatment: A Meta-Analysis. Health Psychol. 2004;23(2):207–18. doi: 10.1037/0278-6133.23.2.207. [DOI] [PubMed] [Google Scholar]
  • 59.Arnold EA, Hazelton P, Lane T, Christopoulos KA, Galindo GR, Steward WT, et al. A qualitative study of provider thoughts on implementing pre-exposure prophylaxis (PrEP) in clinical settings to prevent HIV infection. PloS One. 2012;7(7):e40603. doi: 10.1371/journal.pone.0040603. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Garcia J, Parker C, Parker R, Wilson P, Philbin M, Hirsch JS. “You’re really gonna kick us all out?” Sustaining safe spaces for community-based HIV prevention and control among Black men who have sex with men. PLoS ONE. doi: 10.1371/journal.pone.0141326. forthcoming. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Bull SS, Levine DK, Black SR, Schmiege SJ, Santelli J. Social Media–Delivered Sexual Health Intervention. Am J Prev Med. 2012 Nov;43(5):467–74. doi: 10.1016/j.amepre.2012.07.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Jaganath D, Gill HK, Cohen AC, Young SD. Harnessing Online Peer Education (HOPE): Integrating C-POL and social media to train peer leaders in HIV prevention. AIDS Care. 2012 May;24(5):593–600. doi: 10.1080/09540121.2011.630355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Muessig KE, Pike EC, Fowler B, LeGrand S, Parsons JT, Bull SS, et al. Putting Prevention in Their Pockets: Developing Mobile Phone-Based HIV Interventions for Black Men Who Have Sex with Men. AIDS Patient Care STDs. 2013 Apr;27(4):211–22. doi: 10.1089/apc.2012.0404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders: AIDS. 2011 Mar;25(6):825–34. doi: 10.1097/QAD.0b013e32834380c1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Garcia J, Parker R, Parker C, Wilson PA, Philbin M, Hirsch JS. Black MSM’ as a category: Why gender, sexuality, and desire still matter for social and biomedical HIV prevention methods. Glob Public Health. 2015 doi: 10.1080/17441692.2015.1134616. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Lesser J, Verdugo RL, Koniak–Griffin D, Tello J, Kappos B, Cumberland WG. Respecting and Protecting Our Relationships: A Community Research HIV Prevention Program for Teen Fathers and Mothers. AIDS Educ Prev. 2005 Aug;17(4):347–60. doi: 10.1521/aeap.2005.17.4.347. [DOI] [PubMed] [Google Scholar]
  • 67.Shain RN, Piper JM, Newton ER, Perdue ST, Ramos R, Champion JD, et al. A randomized, controlled trial of a behavioral intervention to prevent sexually transmitted disease among minority women. N Engl J Med. 1999;340(2):93–100. doi: 10.1056/NEJM199901143400203. [DOI] [PubMed] [Google Scholar]
  • 68.Ramos R, Shain RN, Johnson L. MEN I MESS WITH DON’T HAVE ANYTHING TO DO WITH AIDS. Sociol Q. 1995;36(3):483–504. [Google Scholar]
  • 69.Sussman AL, Davis S. Integrating formative assessment and participatory research: Building healthier communities in the CHILE Project. Am J Health Educ Am Alliance Health Phys Educ Recreat Dance. 2010 Jan 1;41(4):244–9. doi: 10.1080/19325037.2010.10599150. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES