Recruitment and Retention of Diverse Populations in Antiretroviral Clinical Trials: Practical Applications from the Gender, Race And Clinical Experience Study

Abstract

Background

Women, particularly women of color, remain underrepresented in antiretroviral (ARV) clinical trials. To evaluate sex-based differences in darunavir/ritonavir-based therapy, the Gender, Race And Clinical Experience (GRACE) study was designed to enroll and retain a high proportion of women representative of the racial/ethnic demographics of women with HIV/AIDS in the United States. The recruitment and retention strategies used in GRACE are described in this article.

Methods

Recruitment and retention strategies targeting women included selecting study sites that focused on women, involving community consultants, site-specific enrollment plans, access to other ARV drugs, study branding, site and patient toolkits, targeted public relations, site grants for patient support, and subsidized child care and transportation.

Results

The recruitment strategies were successful; 287 (67%) women were enrolled, primarily women of color (black, n=191 [67%], Hispanic, n=60 [21%]). Despite the focus on retention, a greater proportion of women (32.8%) discontinued compared with men (23.2%).

Conclusions

The successes of GRACE in enrolling a representative population of women were rooted in pretrial preparation, engagement of community advisors, enrollment quotas, choice of study sites and site support. Lessons learned from GRACE may be applied to future study design. Further focus on factors that influence discontinuation is warranted.

Introduction

Approximately 25% of new HIV diagnoses in the United States are in women,^1–3 and the rate of these new infections has been increasing steadily over the past decade.⁴ Women of color are disproportionately affected; black and Hispanic women constitute 25% of the women in the United States, yet in 2008, black and Hispanic women accounted for an estimated 80% of new HIV/AIDS diagnoses in this patient population.^1,2,5 Despite this increase, women, particularly those who are treatment experienced, remain underrepresented in clinical trials of antiretroviral (ARV) agents for HIV.⁶ Consequently, current treatment guidelines are based primarily on data from clinical trials that predominantly enrolled men.

Although data are limited on sex-based differences in HIV therapy, the results of some studies that have made gender comparisons (albeit with limited numbers of women) have suggested that there may be differences between women and men in the pharmacokinetics, efficacy, and safety of ARVs.^4,7,8 In contrast, post hoc analyses of several other studies have not identified sex-based differences in the efficacy and safety of ARV therapy; however, these studies were not designed for sex-based comparisons.^9–11 Although the number of treatment-naïve women enrolled into clinical trials has been steadily increasing, recruitment and retention of treatment-experienced women in trials have been less successful.^4,9,12 Clinical trials that are specifically designed to recruit and retain a sufficient number of HIV-infected women will ensure that studies are adequately powered for between-sex comparisons and provide meaningful data on the effects of ARVs in women.

There are many barriers to the successful recruitment and retention of women in clinical trials. Typically, women occupy primary care roles in their families and may place family commitments above their own personal medical care.¹³ Additional factors for some women that may impact regular access to healthcare and lead to missed medical appointments include domestic violence or the threat of violence, use of illicit drugs, and perceived stigma about their HIV status.¹³ To overcome these barriers and achieve successful participation of women in clinical trials, innovative approaches to trial design and conduct are required.

The Gender, Race, And Clinical Experience (GRACE) study was specifically designed to enroll a high proportion of women in order to investigate sex-based differences in ARV therapy. An array of approaches and strategies, some of them novel, were applied in GRACE in an effort to maximize recruitment and retention of women. GRACE successfully enrolled 67% women, thereby challenging the collective sentiment that such a goal could not be accomplished. As individual patient's motivations for participation in the trial were not investigated, it was not possible to determine the relative effectiveness of the independent recruitment and retention tactics. The focus of this article is a review of the enrollment strategies that were applied, the challenges faced with retention of women, and the key lessons learned throughout the planning and conduct of the study.

Materials and Methods

Study design, implementation, and conduct

The GRACE study was specifically designed to compare the efficacy, incidence of adverse events, and tolerability over 48 weeks of darunavir/ritonavir (DRV/r) 600 mg/100 mg twice daily combined with an investigator-selected optimized background regimen in ARV-experienced women and men. This open-label, phase 3b study was conducted at 65 sites across the United States, Puerto Rico, and Canada.

Pretrial planning

Early in protocol development, the study sponsor, Tibotec Therapeutics, partnered with clinicians, researchers, and community advisors for input into the planning of a women-focused study. The study design was not preconceived, although the sponsor envisioned a study in women that would expand the current knowledge base for the protease inhibitor, DRV. Initially, the study sponsor hosted a 2-day internal brainstorming session to discuss issues that might have an impact on GRACE, including where to find study sites and how to motivate women to participate in the trial. The sponsor also examined the outcomes of previous trials for strategies and tactics that could be applied to this study.

A series of advisory boards was subsequently hosted with treatment advocates, researchers, and clinicians involved in the care of women. Advisors provided insight into the needs of women and the goals and unique challenges of conducting a study in this population. The discussion included potential confounding baseline factors that could influence study design, implementation, or outcomes, such as variations in care; differences in access to medications; differences in the experience level of the study site or principal investigator (PI); presence or absence of support personnel, such as social workers or peer advocates; and an affiliation of the study site with an AIDS Service Organization. The study concept, details, and protocol were refined during additional advisory boards held over several months before the start of the study.

Study population and enrollment

All inclusion and exclusion criteria were reviewed for the potential to serve as a barrier to participation for women or people of color, and adjustments were made as necessary. No CD4+count criterion was imposed, prior treatment allowances were broad, enrollment of patients on treatment interruptions of at least 4 weeks was allowed, and investigators were permitted to enroll patients actively using illicit drugs or alcohol if they believed the patient could comply with the study protocol. Full inclusion and exclusion criteria have been described.¹⁴ The sample size was determined based on a noninferiority model, a hypothesized, clinically relevant difference in response rates between women and men (15%) and 80% statistical power, with an allowance for discontinuations. Women were ultimately overrepresented to allow greater insight into safety and tolerability in this population.

Site selection and engagement

The site selection process considered epidemiologic trends, as women with HIV/AIDS are not evenly distributed across the United States.² Almost 50% of selected sites were in the southeastern United States,¹⁵ reflecting the higher concentration of HIV/AIDS among black and Hispanic women in this region (Fig. 1). Moreover, guidance received during the planning stage recommended that the study sponsor identify sites where women receive HIV primary care and have preestablished relationships with healthcare providers. It was hoped that use of primary care sites would foster a strong commitment to GRACE on behalf of both provider and patient, minimize the travel and time burden associated with clinical care, and aid patients in overcoming the challenges to continued trial participation. Thus, the study sponsor did not limit participation to sites with extensive prior experience in conducting clinical trials, but instead broadened site recruitment to include centers that had strong community relationships, could support flexible office hours, and had the capacity to and interest in exceeding the standards of care for their female patients. Community advisors provided an additional list of more than 30 possible study sites and reached out to women in their communities to identify women-focused healthcare providers. The ability of each site to conduct a successful trial was assessed using a feasibility questionnaire that was specifically modified to assess issues important for GRACE. Following initial evaluation, in-person site feasibility visits were used to confirm the appropriateness of sites.

FIG. 1.

Study sites and geographical distribution of women enrolled in the Gender, Race And Clinical Experience (GRACE) study.

In all, eight novel sites with a focus on women but with limited clinical trial experience participated in GRACE. Most novel study sites were paired with more experienced sites for support and guidance throughout the study. These new sites had to be trained by the sponsor on Good Clinical Practices, monitored more closely on an ongoing basis, and visited more frequently by field clinical personnel and consultants to assist with enrollment strategies and data collection. Unlike in other clinical trials, the Clinical Research Organization (CRO) was afforded special monitoring rights that enabled it to visit sites and provide support when required.

Investigator meeting

After identification of the participating study sites and before study initiation, a meeting brought together investigators, study coordinators, community advisors, the GRACE Study Team, and the study sponsor. Various sessions included HIV-positive women sharing their experiences of living with HIV and participating in clinical trials, protocol review, site questions and feedback, identification of potential challenges at the site level, and brainstorming about tools or tactics for success. In small group sessions, each site was required to draft an action plan for site outreach. This meeting also allowed sites that were new to clinical research an opportunity to network with each other and with more experienced clinical sites.

Community advocates as consultants

A patient advocate with significant personal and professional experience working with women living with HIV/AIDS was hired as a consultant for GRACE. This consultant provided important input to the study sponsor in all aspects of study design and implementation. Significant emphasis was placed on providing study sites with direct and frequent access to the consultant for various types of assistance. Moreover, the consultant worked to schedule meetings with community-based groups, connect study sites with local resources, and conduct specialized GRACE presentations to community groups, such as the Southern AIDS Coalition. A second community consultant who served as an advisor during the early planning stages was later enlisted to assist with study recruitment, primarily in the Los Angeles area.

Recruitment and retention strategies

Discussions during pretrial planning with the consultant led to the application of several novel tactics for study recruitment. First, each site was initially required to enroll at least 3 women before enrolling a man, and thereafter each site was required to maintain at least 70% female enrollment. Because it was expected that the majority of women participating in GRACE would be black or Hispanic, the enrollment of white men was limited to 25% of the total planned population of men to promote a similar racial/ethnic distribution between sexes for meaningful statistical comparisons.

Second, the name GRACE, created by the consulting patient advocate, was designed to appeal to women. The decision to brand GRACE with a unique multicolored butterfly logo subtly intertwined with women's silhouettes (Fig. 2) to give the study an identity was conceived during the advisory process and was received with enthusiasm by both women and their healthcare providers. The logo was designed in partnership with the community consultant to represent diversity, community, and a connectedness that transcends the common dividing lines of race, socioeconomic status, and geography. Even the script font was chosen specifically to appeal to women. The GRACE logo became the backbone of study-related materials, allowing the study to resonate with physicians and patients at a personal level and, as such, was an essential component of study outreach.

FIG. 2.

The GRACE logo for unique branding over the course of the study.

In addition to the recruitment approaches mentioned and unlike many other ARV studies, the launch of GRACE also included comprehensive and targeted media and community outreach in communities of color. The goals of the media and community outreach, which was designed as a public relations campaign rather than paid advertising, were to raise public awareness of, generate excitement for, and facilitate enrollment of women of color into the study. Black and Hispanic media outlets in local markets were targeted in both English and Spanish. A diverse array of media was used, including print advertisements, public service announcements, community list-serves, newsletters, and faith-based forums. Television spots, news stories, and live radio interviews were used where possible, and HIV treatment press was enlisted to help disseminate the study plan to specific communities of color. These efforts were continued throughout the trial, engaging patients who may not normally have access to information regarding the conduct and availability of clinical trials.

The recruitment of a large proportion of underrepresented women of color was a key goal in GRACE, but the retention of trial participants was another critical objective. Thus, strategies intended to foster retention of women also were incorporated into the trial design. A stipend for child care services and transportation to trial sites was offered to all participants to reduce barriers and encourage ongoing clinical care visits. Although transportation support is included in many trial protocols, it is not often publicized to the site staff or patients. Thus, significant efforts were made within GRACE to communicate that these resources were available, and all prospective participants were made aware, via the informed consent form, of the stipend for child care services or transportation costs.

Modest site grants were available to fund activities that would foster the retention of patients. In all, 12 of 65 sites applied for and received grants to support activities, such as clinic-based support groups, lunch and learns, and other educational programs. The goal of these retention activities was to foster a sense of connectedness and support in a place where women could learn more about the study. Additionally, women could discuss and share their perceptions and concerns about clinical trial participation and general challenges related to HIV and its management. Some sites also provided more frequent and personal access to physicians and nurses, which allowed for flexibility in clinic attendance and facilitated a reduction in potential barriers to continued trial participation. Other sites offered prefilled pillboxes, pillbox storage/refrigeration, or individual reminders for scheduled visits in order to simplify some of the practical aspects of disease management that can contribute to the discontinuation of care. Staff from at least one study site conducted home visits as unique circumstances for individual patients arose.

Although most strategies were used solely for either recruitment or retention purposes, some tactics were employed for both. The provision of several ARVs, including an investigational agent to facilitate construction of a full treatment regimen, helped eliminate any potential barriers to accessing medication. This approach also served to differentiate GRACE from other clinical trials for treatment-experienced patients. In addition, toolkits were used as both a recruitment and a retention strategy. Institutional Review Board-approved kits were designed based on feedback from community and physician advisors, as well as ideas generated by site staff at the investigator meeting, and were branded only with the GRACE study logo or components of that logo to identify the materials. The outreach toolkits included GRACE posters and fliers, inclusion/exclusion cards for providers, a booklet on engaging women in clinical trials, “Ask me about GRACE” buttons for site staff, and GRACE balloons for outreach events. A CD-ROM with templates for fliers and advertisements, draft public service announcements, a “Dear Colleague” letter for the purpose of piquing interest among colleagues in the local area, and a supplemental grant request form were also included. The patient toolkits included a branded canvas bag containing a day planner, GRACE pen, water bottle, single-dose pillbox, site contact card, insulated lunchbox, and a description of study visits and procedures. Acknowledgment certificates, intended for patients, site staff, and partners in the local community, were used to recognize individual contributions to the study, and GRACE birthday cards gave sites a mechanism to celebrate and recognize their patients as people rather than solely as study participants.

Results

Impact of recruitment and retention strategies

The novel recruitment strategies employed in GRACE were successful; the recruitment goals were met on time. A total of 429 patients were recruited to the trial, including 287 (67%) women, most of whom were women of color (black, n=191 [67%], Hispanic, n=60 [21%]).¹⁴ The ability of the GRACE study to provide meaningful information about women with HIV hinged on the mandated enrollment of at least 70% women at each study site. This strategy proved to be highly effective, as approximately halfway into recruitment, only 10% of the participants were men. At that point, the enrollment requirement was discontinued to assure that enough men would participate to meet the study goals. Of the 65 sites, a total of 59 enrolled patients within the enrollment time line, and only 1 site closed during the study period. Patients were not required to document their motivation for participating in the study at the time of enrollment, so it was difficult to quantify the impact of individual enrollment strategies. However, some patients did specifically request to participate in the study as a result of the media campaigns.

An interim analysis of GRACE revealed that among the first 200 patients enrolled, nearly 25% of women had discontinued the study through week 24. Before the availability of the interim analysis, the main focus of the study team and sites was on recruitment of women. At this point, efforts shifted to reducing subsequent discontinuations among GRACE participants. The team reviewed all discontinuations for trends in causes or patient/site characteristics, but the review did not reveal any consistent trends or clear opportunities for additional interventions. From that point forward, however, the study team and sites committed to trying to improve the retention rate for women.

Among the 429 patients who ultimately enrolled in the study, a significantly greater proportion of women discontinued the trial vs. men (32.8% vs. 23.2%, respectively, p<0.05) (Fig. 3) before the final week 48 visit.¹⁴ The higher rate of discontinuation among women in the GRACE trial was driven by reasons other than a lack of response to study medication; only 6 (2.1%) women discontinued because of virologic failure. Instead, the primary reasons for study discontinuation among women were loss to follow-up, adverse events, and reasons categorized as “other” by the study investigator (e.g., site closed, patient moved, pregnancy). The rate of discontinuation due to adverse events was slightly higher for women than men (Fig. 3). In addition, rates of discontinuation in GRACE were also higher in black patients (32.6%) vs. white (26.2%) or Hispanic (24.0%) patients. Individual study sites likely applied different tactics to improve study retention based on the resources available to them, but the relative success or failure of individual tactics was not tracked.

FIG. 3.

Study disposition. *Other classification was selected by the investigator as reason for discontinuation. (From Currier J, et al. Ann Intern Med 2010;153:349–357.¹⁴ Reprinted with permission. Copyright 2010 The American College of Physicians.)

Interestingly, novel study sites were more successful in recruitment of trial participants and also had somewhat better retention in the study compared with other sites. The novel sites recruited an average of 2 more patients per site to the study and had a discontinuation rate 33% lower than other sites (overall discontinuation rates of 21% at novel sites vs 33% at other sites).

Discussion

Although it is difficult to assess the individual contributions of the many recruitment and retention techniques employed in GRACE, in retrospect, we believe that the two most important components of the study plan were early engagement of experienced, passionate clinicians and community advocates, and thoughtful identification and selection of study sites. The researchers and community advisors were dedicated partners in the GRACE study, with a vested interest in successfully generating ARV treatment data in women. Their input throughout the study was indispensible in designing a successful trial that would fulfill the goals of all participating parties. Equally as important as the involvement of these advisors was the decision to include novel sites with limited clinical trial experience but with ready access to the population of interest—women. Novel sites not only proved to be successful in recruitment but also had somewhat better retention in the study, thus demonstrating that such sites are fully capable of conducting clinical trials if provided with appropriate support. An indirect, but important, outcome of the engagement of novel study sites was enhancing the capacity of these sites to conduct future clinical trials; several of these sites have since successfully participated in other ARV studies.

In general, provision of individualized consulting support for sites proved to be important to the overall success of the trial and to study recruitment in particular. Because challenges at each site were unique, the support provided was individually tailored. Providing access to a consultant with significant experience in the HIV community gave sites an opportunity to work through challenges and maintain their enthusiasm and momentum for the duration of the study. Because other HIV-focused studies had failed in their efforts to recruit women and many individuals voiced serious doubts that GRACE would be able to meet enrollment goals, the focus of the study team and the consultant was first and foremost on recruitment; the shift in focus to patient retention did not occur until too late in the study. One of the lessons learned was that retention of women should have been a critical focus from the initial planning stages of the study.

Planning and conduct of the GRACE study was associated with slightly higher costs (e.g., additional CRO costs, consulting fees, toolkit costs) and resource requirements (e.g., public relations support for media outreach, extensive site support) than traditional studies. Collectively, these tactics all contributed to successful enrollment within the preestablished time frame for the study and were likely worth the additional investment. Although the individual contributions of these components cannot be assessed, it is believed that setting enrollment quotas alone would not have been sufficient to reach the desired enrollment goals, as previous trials using enrollment quotas have not achieved the same level of success in enrollment as GRACE. The Safety, efficacy and Pharmacokinetics of tipRanavir boosted with low-dose ritonavir IN 400 racially and Gender-diverse HIV-positive treatment-experienced population (SPRING) study, for example, aimed to enroll 200 women and 200 men at 72 sites across eight countries, including white, black, Hispanic, Asian, and American-Indian patients. The study was terminated, however, not because of efficacy or safety concerns but because of poor enrollment.^16,17

The higher discontinuation rates seen in women (compared with men) and blacks (compared with nonblacks) in GRACE have also been reported in other studies. The Context Study, which assessed treatment-experienced patients, reported a discontinuation rate of 47% in women compared with 22% in men; however, women comprised only 15% of the 315 patients enrolled, limiting the interpretation of these data.⁹ Additionally, in a study that enrolled 254 patients, including a high proportion of women (50%), black (40%), and Hispanic (37%) treatment-naïve patients, Kumar et al.¹⁸ reported an overall discontinuation rate of 49%; specific differences by sex were not reported. The Women First Study was specifically designed to assess the efficacy and safety of combination ARV therapy in protease inhibitor-naïve women and also to evaluate treatment compliance and continued study participation. Specific support for patients was provided and included labeled pillboxes with dosing schedules, dosing instruction cards, day planners, and education on HIV and management of the disease.¹⁹ Although ARV treatment was generally well tolerated in this study, only 22 of the 68 patients enrolled were available for analysis at week 48.^19,20

In GRACE, the rate of discontinuation due to adverse events was slightly higher for women than men. Although the difference was not statistically significant, it is possible that additional women stopped therapy and disappeared from care (i.e., categorized as “lost to follow-up”) for issues related to tolerability. Indeed, there were subtle differences in tolerability between women and men in GRACE—women experienced more nausea and vomiting, whereas men reported more diarrhea.¹⁴ These findings are supported by other ARV studies as well.^8,10,21 One approach to reducing the rate of discontinuation in clinical trials may be for healthcare providers to monitor patients more intensively for subtle side effects, proactively communicate on a regular basis with patients about how they are tolerating treatment, and provide patients with information about management of side effects if they occur.

Although more women discontinued from GRACE than men and more blacks discontinued than nonblacks, discontinuation rates were highly variable by site. Indeed, for at least one site in the southeastern United States, several black women generally considered to be at high risk for discontinuation successfully completed the study. In such cases, a strong patient-provider relationship and the dedication of the investigator likely contributed to the patients' success, enabling patients to discuss their concerns and overcome challenges before they were compelled to discontinue. The impact of such a relationship cannot be underestimated when it comes to continued engagement in clinical care or in clinical trial conduct, and the higher retention rates for novel sites in GRACE may have hinged on such relationships. Novel sites tended to be primary care sites and have fewer other active studies, which likely allowed more focus by the site study staff on individualized support of the GRACE patients. It is possible that increased use of primary care sites, whether experienced in research or not, may be important in improving patient retention in future studies.

The successes and challenges associated with GRACE provided insight into data gaps that can be addressed in future trials. First, capturing socioeconomic data (such as housing status, education level, number of dependents, and income level) from patients and including more psychosocial measures at baseline may aid in understanding the factors that influence patients' motivation to stay in care. In the case of GRACE, having access to such information may have led to a clearer understanding of the reasons for study discontinuation and lower response rates among black patients. Second, collecting information at study entry about intolerance to prior ARV regimens and rates of treatment failure may provide additional insight into subtle differences that exist among treatment-experienced patients. Identifying up front which study participants are predisposed to challenges with adherence and tolerability may allow study staff to more effectively mitigate the risk of poor adherence or study discontinuation due to intolerance. Third, collecting detailed information from patients at the time of study discontinuation would provide more insight than collecting data in the typical broad categories of “loss to follow-up” and “noncompliance.” In GRACE, specific details related to study discontinuations would have helped the study sponsor and sites better understand what proportion of discontinuations was preventable and subsequently develop strategies to address the root causes of discontinuation.

Over the course of GRACE, we identified three main areas that could have been improved and that we would approach differently in the future. First, we would place more emphasis on study retention during study startup and routine study monitoring. In theory, early identification of discontinuation trends would allow for earlier intervention, which may allow for earlier patient-focused efforts (e.g., from the screening phase forward) to foster retention. Second, more novel study sites would be enlisted, particularly focusing on sites that serve as a source of primary care for women. Finally, more education would be provided around the site grants and how to use them effectively; many sites did not access these funds and were unsure of how to put the resources to best use. Ideally, technical assistance could be provided to sites to help implement grant-related activities and to support data collection regarding factors contributing to study discontinuation.

Conclusions

GRACE was the largest study in North America to date to focus on treatment-experienced women with HIV receiving ARV therapy. The successes of GRACE were rooted in pretrial preparation, engagement of community advisors, mandated enrollment quotas, choice of study sites, and providing study sites with the resources and flexibility to adapt practices as necessary to support patients. These strategies were effective in the recruitment of underrepresented women of color to the trial. In one third of the women, however, retention efforts fell short of overcoming the well-rooted and very real barriers to continuous healthcare faced by women and other disadvantaged populations. Lessons learned during the planning and conduct of the GRACE study form a solid foundation for the development and design of future ARV clinical trials and may contribute to a better general understanding of how to enroll and retain diverse populations in clinical trials and care.

Acknowledgments

The GRACE study was funded by Tibotec Therapeutics. Gilead Sciences provided emtricitabine, tenofovir disoproxil fumarate, and emtricitabine/tenofovir disoproxil fumarate. J.C. is supported in part by a grant from the National Institute of Allergy and Infectious Diseases, National Institutes of Health (K24 AI56933). We thank Samantha Taylor, Ph.D., Medicus International New York, for her editorial assistance. Lori DeLaitsch of Tibotec Therapeutics Clinical Affairs reviewed and provided input on this article. Editorial support was funded by Tibotec Therapeutics.

Disclosure Statement

R.F., D.S., R.R., and J.M. are employees of Tibotec Therapeutics. J.C. has served as an advisor for Tibotec, Bristol-Myers Squibb, Merck, and GlaxoSmithKline; has received research grants from Tibotec, Merck, Schering Plough, and Theratechnologies, and has been a DSMB member of Achillion and Koronis. K.S. has served as an advisor/consultant for and has received research grants from Tibotec, Gilead, Merck, GlaxoSmithKline, and Schering Plough; has served as an advisor for Tobira; has received a research grant from Biocryst; and has been a data and safety monitoring board (DSMB) member of Pfizer. D.H. has served as a consultant for Tibotec, Gilead, Merck, Bristol-Myers Squibb, Abbott, HealthHIV, and Health Matters CME and has served as a speaker for Boehringer Ingelheim. D.A.B. has served as a consultant for Tibotec, Gilead, Merck, Bristol-Myers Squibb, and Boehringer Ingelheim.