Recruiting and retaining low-income, multi-ethnic women into randomized controlled trials: Successful strategies and staffing

Abstract

Developing effective recruitment and retention strategies in populations with traditionally high attrition rates is critical to the success of Randomized Controlled Trials (RCTs). Data on successful participation of women from low-income, minority populations in RCTs of behavioral interventions are limited. This is problematic given the multiplicity of Healthy People 2020 goals that target health disparities in these populations. This paper reports successful recruitment and retention methods from two separately funded NIH clinical trials of primary care-based prenatal interventions to increase breastfeeding among ethnically diverse, low-income women in urban medical centers in the Bronx, NY. It also presents the required staff effort necessary to conduct such a successful RCT, in terms of full-time equivalents (FTEs).

Results include timely recruitment of 941 participants over 29 months, with 98.1% completing 1̄ follow-up interview. A recruitment and retention plan that maximized study staff access and availability to the participant, as well as strong study staff rapport with participants, addressed previously reported barriers in this population, optimizing follow-up rates. A qualitative assessment of the participants' study experience suggesting that high retention was due to strong rapport with participants, short interviews requiring little time commitment, and participants' perception of the study as informative, provides further evidence of our approach's effectiveness.

Logistical protocol procedures and staff management strategies relating to successful recruitment/retention are provided to propose a practical, cost-effective and translational recruitment–retention plan for other researchers to adopt.

1. Introduction

Successful recruitment and retention is critical to the success of randomized controlled trials (RCT). Difficulties with either can potentially compromise RCTs' validity and reliability by reducing study power, introducing bias, and decreasing generalizability [1,2]. High attrition is more pronounced among multi-ethnic, low income populations [3–6]. NIH and other scientific groups promulgate guidelines regarding the inclusion of minorities and women in research [7]. Among minority women, the myriad barriers to successful study participation include: language barriers, economic constraints, transient living situations, unreliable contact information, transportation difficulties, cultural differences, distrust of the medical community, and negative patient experiences [4,8–16].

The incontrovertible benefits of breastfeeding to mother and child [17] underlie Healthy People 2020 goals (Maternal Infant and Child Health 21–24) [18]. Given the lower breastfeeding rates among minority and/or low-income women [19,20], data for effective programs are particularly needed in these populations. Yet, this study population presents unique challenges, including: time constraints due to work and childcare commitments, risk-aversiveness during pregnancy and lack of rapport between research staff and participants [5,21,22]. Assessing interventions that span the prenatal through post-partum periods is complicated by the stress and increased time commitments in the first few months post partum [22].

Little is known about the logistics of successful recruitment and retention in minority pregnant women, a critical target population for Healthy People 2020 goals [18]. Studies examining recruitment and retention among minority pregnant women may highlight successful strategies [14,23], but do not provide detailed information on the staff effort required to carry out these methods. Most studies examining human effort necessary to carry out an RCT do so cumulatively per participant or in annual dollar amounts [9,24–27]. More importantly, this literature does not examine behavioral health promotion trials in this population.

This paper presents the strategies employed to achieve an outstanding retention rate—98% completing ≧1 follow-up interviews by 6-months postpartum—in our RCTs of primary care based breastfeeding promotion interventions among low to moderate income, ethnically diverse women. In order to aid future researchers wishing to work with this target population, this paper presents a) the trials' 1, 3, and 6 month recruitment and follow-up rates, b) the contact and staff effort required to achieve these rates, c) the trials' 10 point recruitment and retention plan, created to address commonly cited obstacles, and d) qualitative data from participant exit interviews about study participation.

2. Methods

2.1. Study description

This analysis includes data from two single-blinded RCTs of routine prenatal care based interventions to increase breastfeeding intensity and duration among multiethnic women. Participants were recruited from two urban Bronx, New York clinics serving multi-ethnic, low income populations. Both studies were approved by the governing institutional review board and were separately funded by the National Institute of Health (NIH) [28,29]. All study materials were available in Spanish and English. The eligibility criteria, recruitment methods, follow-up protocols and data collection process were identical for the two studies. The study RAs rotated between the sites, and the Study Coordinator oversaw the trials at both sites. Given that this analysis examines these process measures, data has been combined for the purpose of this analysis.

Participants were randomized into a Lactation Consultant (LC) only, Electronic Prompt (EP) only, Electronic Prompt and Lactation Consultant (EP + LC), or a Standard of Care (Control) group. Intervention details are documented elsewhere [30]. Briefly, participants in the EP intervention group had 5 electronic prompts inserted into their electronic medical records, instructing the provider to discuss breastfeeding guidelines and benefits, and to ask about the participant's breastfeeding plans and support systemat each routine prenatal visit. Those randomized to the LC group met with an International Board Certified lactation consultant; the study's LC protocol included 2 prenatal visits at the participant's prenatal care site, 1 hospital visit, and both telephone and home follow-up visits up until 3 months post-partum or until the participant was no longer breastfeeding.

2.2. Recruitment and retention

Both RCTs enrolled English and Spanish speaking women aged ≥18 years, in the 1st or 2nd trimester of a singleton pregnancy. Recruitment took place from February 2008 through July 2010, and follow-up continued through September 2011. BINGO recruited from a federally qualified health center that serves as a teaching site, and PAIRINGS recruited from a faculty practice. Both trials were affiliated with the same Bronx-based medical center.

Eligibility for both trials was determined by a prenatal screener administered by a bilingual Research Assistant (RA) during a routine prenatal visit. Participants were provisionally consented and enrolled pending provider approval. The baseline interview was administered by a RA at the time of enrollment; it collected demographic information as well as attitudes and knowledge about feeding practices.

Follow-up consisted of 1, 3 and 6 month post-partum telephone interviews conducted by RAs. Interviews collected information on the birth experience and feeding practices. The study's project director maintained a data management system (Microsoft Access), which alerted staff to send out reminder postcards for upcoming interviews. Participants had a 14 day window for completion of the 1 month interview, a 20 day window for completion of the 3-month interview, and a 28 day window for completion of the 6-month interview. If a participant missed the 1-month interview, missing birth information was collected during the 3 or 6 month interview, but these individuals were considered to be “Missed” for their 1-Month interview.

RAs were masked to treatment group in several ways. Participant paper files did not indicate treatment group, nor did any of the electronic databases accessed by the RAs. However, there was no explicit attempt to have a different RA assess outcomes (vs. the RA who enrolled the participant). The first follow-up generally occurred anywhere from 3 to 7 months after enrollment.

A random 20% of participants, across both trials, were selected to partake in a 6-Month qualitative exit interview. Participants were randomized by study identification number. The qualitative exit interviews (n = 67) assessed participant's perception of intervention effectiveness, fidelity of intervention and their overall experience in the study.

All follow-up data were recorded by a RA using Microsoft Access database; information was password protected for confidentiality purposes and was checked weekly by the Study Coordinator (SC) to ensure accuracy of identifying information and birth related data. This database contained documentation of date of participant enrollment, preferred language, RA method of contact, number of contacts, date & time of successful contacts and/or interview completions, and status of incentive deliveries. At time of enrollment, consent (approved by governing IRB) included access to participants' electronic medical records. Study staff worked with medical site administrators at each site to remotely access their administrative databases to track upcoming visits for the purposes of updating contact information or conducting an interview if the participant had not been reachable by phone.

2.3. Analyses

As this paper does not address the trials' outcomes (breastfeeding at 1, 3, and 6 months) data for both trials were combined. This decision is consistent with this study's objectives: to describe and analyze process-related measures of RCTs in low/moderate income populations. The justification for doing this is that the eligibility criteria, recruitment strategies, follow-up strategies, and data collection were identical in both RCTs. Furthermore, pooling the data increases the heterogeneity of the target samples, and thus is more likely to increase the external validity of our results.

We carried out descriptive analyses on recruitment and 1, 3, and 6 month post-partum interview retention rates (see Table 2). The latter were compared with targeted retention rates, based upon a prior study [31]. Average number of contact attempts to successfully complete interviews at 1, 3, and 6 months are presented, as well as the distribution of the method of contact through the entirety of the study.

Table 2.

Follow-up rates and contact attempts.

Post-partum interview (N=907)	Completed, n	Non-completeda, n (%)	Actual completion rate	Target completion rate	Average attempt for successful contact (range)
1-Month	848	59(6.5%)	93.4%	89.7%	3.3 (1–22)
3-Month	857	50 (5.5%)	94.5%	84.6%	3.4 (1–20)
6-Month	850	57 (6.3%)	94.0%	80.0%	3.9 (1–20)
Completed Any	890	–	98.1%	–	3.3 (1–22)
Completed Only 1	16	–	1.7%	–	6.3 (1–11)
Completed Only 2	84	–	9.2%	–	4.8 (1–20)
Completed All 3	790	b17 (1.8%)	87.1%	–	3.2 (1–16)

^aWithdrawn treated as “non-completed”.

^bmissed all 3 interviews.

Further, recruitment and follow-up rates were assessed over time in relation to study staff full time equivalents (FTE) based upon a 35 hour work week. Additionally, we assessed personnel effort to determine the amount of time required to recruit and retain participants, and used staff recall to evaluate what strategies translated best into practice. Qualitative exit data were recorded and transcribed. Through an iterative process, data were coded until 80% inter-rater reliability was reached by the project team. Themes focused on participants' attitudes towards the study, particularly the logistics. Data saturation was achieved at 67 interviews, at which point they were analyzed using the code matrix/frequency function part of the MAX.qda software. Further details are described elsewhere [32].

3. Results

3.1. Study population

We randomized 941 women into a treatment group; 34 were excluded due to “protocol violations” (i.e., rendered ineligible after provisional enrollment or miscarried),¹ yielding N = 907 participants used in this analysis. These 907 participants include those who withdrew (n = 5), whose infant deceased (n = 1) or who we were unable to be contacted for follow-up (n = 13). Overall, the sample includes participants who gave birth to a live infant, for which they had custody. The sample's demographics by study are shown in Table 1. They were primarily Hispanic, about one-third were born outside the US, and the majority received WIC. Three fourths had at least a high school degree, and expected to return to work or school by their child's first birthday (Table 1).

Table 1.

Maternal baseline characteristics.

	n (%)
Na	907
Age, mean ± SD	27.6 ± 5.8
Race/ethnicityb
White/non-Hispanic	54 (5.6)
Black/non-Hispanic	288 (30.0)
Hispanic	529 (55.2)
Asian/Other	66 (6.9)
Nativity Status
US born	607 (66.9)
Foreign born	300 (33.0)
Preferred language
English	829 (91.4)
Spanish	78 (8.6)
Mother and/or child enrolled in WIC	492 (54.2)
High school graduate	722 (79.6)
Nulliparous	361 (39.8)
Plans to work 1st year postpartum	661 (72.8)
Plans to attend School 1st Year postpartum	434 (47.8)

^aProtocol violations were not included in this analytic sample.

^bThose marked two or more races were counted two or more times; for race/ethnicity N=957, although we only have a total of 907 participants.

3.2. Recruitment and retention rates

Among all eligible participants (N = 907), 98.1% completed one or more interviews, while 87.1% completed all 3 post-partum interviews (Table 2). Completion rates for the 1, 3 and 6 month post-partum interviews were 93.4%, 94.5% and 94.0% respectively, and well exceeded target rates (Table 2). Retention rates did not vary by control or an intervention group (p=.989). There were no significant differences in the proportion of participants completing 0, 1, 2 or 3 follow-up interviews by study site (p=.959). The sites' 1, 3, and 6 month follow-up interview rates were: BINGO=93%, 94%, 95%; PAIRINGS=95%, 94%, 91%.

3.3. Staff effort and recruitment–retention by FTE

3.3.1. FTE over the course of the study

The recruitment periods for both studies lasted 29 months, beginning in February 2008, and reaching our target enrollment of 941 participants in June 2010. The retention process spanned from May 2008 until September 2011, for a total of 40 months of follow-up. Recruitment for both trials began within 1 month of each other, thus majority of recruitment and retention efforts were conducted simultaneously. Though recruitment ended 2 months earlier in the smaller trial, the staff effort remained the same.

The Study Coordinator (SC) allocated 25% FTE during the first 2 years of the study, and 10% FTE thereafter to recruitment and retention activities (Fig. 1). The initial 4 months focused on recruitment, with 2 full-time RAs. This was the height of recruitment, during which we enrolled 153 participants (Fig. 1). From October 2008 to May 2010, RA staff effort was increased from 2 FTEs to 3 FTEs to manage the rise of post-partum interviews that needed to be administered. Retention efforts peaked during the February 2009–May 2009 period, as there were 389 follow-up interviews administered at that time (Fig. 1). When target recruitment was reached in June 2010, the number of RA FTEs decreased to 2 full-time RAs (Fig. 1) dedicated to follow-up and data entry duties. The remaining months of the study (June 2011–September 2011) required only 1 full-time RA, as recruitment efforts were complete and only 8 interviews needed to be conducted (Fig. 1).

Fig. 1.

Recruitment and follow-up completion by monthly periods & FTE.

3.3.2. Contact effort for follow-up completion

A well-built data management system using Microsoft Access alerted study staff of which participants needed to be contacted for follow-up interviews. On a daily basis, the SC reviewed which participants needed to be interviewed, and assigned RAs an equal number to contact. Thus, each RA had a list of participants to call on the days they were not recruiting.

Our RA staff made a total of 10,781 attempts to contact participants over the course of the study. Contact attempts were 94.1% via phone, 3.3% via mail (not including reminder postcards, which everyone received), 2% via email and 0.6% in person. The average number of contact attempts necessary to reach participants for their 1, 3 and 6 month questionnaires was 3.4, 3.3, and 3.9 respectively (Table 2). The number of contact attempts resulting in successful completion ranged from 1 to 22. Almost all interviews were completed by phone.

Those participants who could not be reached by the 3rd contact were placed on a “hard to reach” list, which the RAs updated weekly. The SC monitored the clinic schedule for these patient's appointments, allowing RAs to complete the interview in-person (if an appointment was within an interview window) or to update contact information. In total, RAs met participants at their scheduled clinic appointments 67 times. RAs successfully completed interviews at more than half of these appointments (56.7%). At the remaining appointments for which an RA was dispatched, the participant either missed the appointment (26.8%), or provided study staff with an alternate contact(s) (16.4%), which led to future successful follow-up. On average, the RA contacted a participant 5 times via phone/email/mail before successfully completing an interview “in person.”

3.3.3. Recruitment and retention logistics

The SC monitored recruitment and retention rates throughout the study to ensure targeted goals were met. She dedicated 10–25% FTE to supervising the RAs to ensure recruitment goals were met, all calls to participants were made, incentives were mailed out, data entry was complete and completed interviews were scanned into the database for future analysis. Below is a breakdown of the actual time each RA spent on every step of the recruitment–retention process.

At the smaller (PAIRINGS) site, the RA recruited 2 days per week, while at the larger (BINGO) site, the RA recruited 3 days a week. RA staff rotated between sites as needed. RAs spent 2–5 days per week at the clinics, and the remaining time went towards retention activities. The entire recruitment process (from approaching a patient to giving them their incentives) lasted 15–20 min and was usually done in an exam room as patients waited for the doctor. However, fragmented recruitment sessions were common and occurred wherever space was available (i.e.,: empty exam rooms, hallways).

When retention activities were at their peak (Fig. 1) each RA called 8–10 participants per day for follow-up interviews. Administration of the 1 month questionnaire took on average 12–15 min; 3 and 6 months interviews (same form used for latter two) took 7–10 min. All contact attempts were documented in a paper log and later entered into an electronic database. This data entry took 5–10 min, depending on the length of the call log.

The remaining post-interview retention activities consisted of checking and photocopying interviews, filing hard-copies of interviews away for future scanning, and sending out gift-cards and thank you flyers. These activities took 15 min per participant. On average, retention activities from the beginning of a questionnaire to the scanning of the interview data took 40 min per participant to complete.

3.4. Recruitment–retention plan addresses known obstacles

The 10 Point plan (see Appendix A) was created to reduce attrition rates in an ethnically diverse population cited to be difficult to recruit and retain. We implemented a multitude of strategies, synthesizing effective recruitment retention strategies found in the literature. We present which points of our recruitment–retention plan were most effective in overcoming barriers to recruitment and retention among our population of minority women (Table 3). There were three overarching themes of our strategy: maximum access to participants, study accessibility, and quality incentives.

Table 3.

Barriers to recruitment/retention and ways in which 10 point plan addresses them.

Known barriers to recruitment/retention	Way (s) in which 10 point plan addresses barrier
Language barriers	Bi-lingual staff
Time Constraints	Short, 10–15 min, phone interviews, and increased RA availability by providing participants the opportunity to complete interviews during evenings and weekends.
Economic Constraints	$20 gift card incentives for each of the interviews
Transient living conditions	Obtaining participants' contact information as well as that for 2 other contacts, and having access to clinics appointments and updated contact information provided by prenatal clinic.
Lack of transportation	Conducting telephone interviews instead of in-person interviews, and the pre-natal interview (which is in person) is done while the participant is already in the clinic
Distrust of medical community	Research assistants build rapport with the participants as they are familiar faces at the clinic and often engage in conversations with participants when they come for their pre-natal check-ups.

Maximized access to participants was accomplished by requiring a third party contact at time of enrollment, and maintaining the most up-to-date information from clinic records. Increasing availability of our bilingual staff to complete the brief (10–15 min) telephone interviews during evenings and weekends was very effective in addressing language barriers, and time constraints due to work obligations and infant/child care. We strove to optimize participants' knowledge of and access to the study by providing them with business cards and magnets with study contact information, and by sending reminder postcards 7–10 days prior to interviews. Knowing distrust of the medical community was a potential barrier in our population, study staff worked hard to establish good rapport with participants. The RAs' consistent presence at the prenatal clinic during recruitment provided the opportunity for friendly relations with participants, which contributed to our high follow-up. Lastly, we encouraged continued participation by offering Babies ‘R’ Us gift-card incentives at recruitment and for each post-partum interview.

3.5. Qualitative assessment of participant's experience during follow-up

Assessment of qualitative responses revealed 3 major themes that likely contributed to recruitment and retention success: RA rapport, perception of study participation as educational and useful, and convenience of the follow-up process. Analysis of coding indicated that the first two themes greatly overlapped, thus these two themes are discussed under 1 subheading.

3.5.1. Strong rapport and useful education

Regarding rapport, the 2–3 day/week presence of the RAs at the prenatal clinic encouraged conversations between study staff and enrolled participants. Participants had positive regard for the study staff, as indicated by the following quotes:

“It's been great talking to you guys. It was a good study. You know, I enjoyed talking to you guys and learning and all that. And it helped.”[B262]“I liked that, you know, the people are not forcing you to really answer question… They are considerate of your convenience…how they approach it is very nice.”[B103]

The perceived utility and educational benefits of study participation is clear from our qualitative interviews.

“It really helped me a lot… I did learn a lot of things. And I think it was a really good study.”[B132]“I liked the follow-ups and the questions about nursing my child or if I needed assistance, they provided a lot of information and a breast pump—they were very hands on, which is very important” [B323]

The quality of study staff rapport with participants was further illustrated by participants expressing feeling cared for and comforted.

“It's like a blessing to have a study going on where people are concerned with what you are doing with your baby how your baby is eating, how your baby is responding…. You know you guys also have a way to help out, you know, to figure out what we can do. That's what I like about it.”[B482]“It did not make me feel uncomfortable. It made me feel like I was talking to somebody who really wanted, you know, know about how things are going. Cause being a new mother is hard.”[B463]

These positive relations, incentives tailored to support mothers of infants' and the nature of the interview questions led participants to feel a heightened sense of care, educational opportunity and general interest towards their own and their child's health and well being.

3.5.2. Convenience

Retention was clearly aided by the study's attention to questionnaire content and length, and study procedures. Participants' stated that the post-partum questionnaires were both short (10 min) and non-invasive, while the interviews were viewed as convenient and flexible.

“The phone calls were cool. They weren't long or drawn out… They were brief and straight to the point.”[P163]“I think it was pretty, pretty good as far as the time, was concerned. You were pretty flexible, so whenever I was not available, you would call me back or I would call you back, so… And I did get those cards in advance to notify me that you guys were calling so that was—that was very good.” [P182]

Additionally, the Babies ‘R’ Us gift card incentives were perceived as very helpful, but was not a major reason for participation.

“It was a plus. It was a bonus. If it wasn't given I still would have taken part.”[P132]“It helped out. I put it towards the baby's diapers. That's what I bought with the card—towards the diapers. A big box of diapers at Toys R Us… It helped me a lot, thanks.”[B262]

4. Discussion

This paper reports the strategies and efforts deployed in an RCT of behavioral interventions in which 98% of participants completed at least one of the three follow-up interviews by the final 6 months post-partum outcomes assessment. Specifically, our trial of breastfeeding promotion interventions in a multi-ethnic and low/moderate income population attained 1, 3, and 6 month follow-up rates of 93%, 95%, and 94%, respectively. In contrast, such trials in comparable US samples ranged from 71% to 88% at 3 months, [33–35] and 71% to 75% at 6 months, [33,35]. To guide future researchers, this paper presents the strategies that aided in recruitment and retention, and the associated full-time equivalent (FTE) efforts deployed, in an enrolled sample of over 900 women.

Findings reported here are evidence that our “user-friendly” protocol maximized participation rates. Rapport was enhanced by the bi-lingual, congenial RAs consistent presence at the prenatal care site. Logistically, participants described the follow-up interviews as brief and easy to schedule. Our recuitment–retention plan included multiple contacts, upcoming reminders, and the option to complete interviews at upcoming medical center visits and/or during evenings/weekends. These strageies maximized access for follow-up interviews and have been reported as efficacious in this population [10]. However, most reports [8,11,14,36,37] have not employed the breadth of approaches that we posit was key to our successes. Other studies that employed a multiplicity of strategies did not have the challenge of non-Englsh speaking participants [4,38]. Futheremore, none of the foregoing provide detailed FTE estimates.

Our findings suggest that multiple methods of contact information (mail, email, in person) played a signficant role in our high retention rate. Indeed, while average contact attempts in difficult to reach population have ranged from 4.6 to 10, [39,40] our average number of attempted contacts per follow-up interview for those who completed all 3 interviews was 3.2. Considering the importance of multiple contact modalities, and the current popularity of text messaging, we recommend that future studies incorporate text message alerts to remind participants of upcoming interviews.

The trials' success is especially notable given their clinical translational context. That is, participants were recruited from busy urban prenatal care clinic waiting rooms, with actual enrollment usually occurring in the exam rooms. While the RA presence in the waiting rooms engendered rapport, RA staff had to contend with limited space, hectic settings, wariness on the part of potential participants, and interruptions in the baseline interview when providers entered the exam room. Successful navigation of these barriers required poise and perseverance on the part of the RA staff, and a host site that was invested in the research [30].

It takes considerable attention to the logistics of implementing trials, particularly in hard to reach populations, to ensure successful study participation. Our study contributes to the literature because we employed perhaps the widest range of recruitment/retention strategies, present them in the context of staff FTE, and provide qualitative from participants about their effectiveness and acceptability. Thus, findings presented here offer practical guidance to researchers in drafting budgets and budget justifications, as well as participant engagement and retention strategies.

Appendix A. Ten point recruitment–retention plan used to recruit and retain participants

Note: Study materials were available in both Spanish and English, and our staff included bilingual RAs.

1. During recruitment, participants were required to provide contact information for themselves as well as two other contact persons. Participants were asked for address, cell phone number, home phone number, work phone number, and email addresses.

2. At each of the post-partum telephone interviews (1, 3 and 6 months), participants would receive an additional $20 gift card via mail to encourage follow-through.

3. Participants were provided with study labeled materials, such as logo branded magnets and brochures, which provided the participant with the study's contact information.

4. Study staff obtained biweekly updates of changes in participant contact information within the electronic medical records.

5. Research staff mailed logo branded Postcards to participants to remind them of upcoming interviews. The postcards were mailed approximately 10 days before the start of an interview window. The postcard listed the dates of the interview window, as well as study contact information.

6. Research Assistants (RAs) conducted weekday evening and weekend calls for participants that were consistently unreachable during normal working hours, 9 am to 5 pm.

7. Participants were provided with a business card containing RA cell phone numbers.

8. RAs sent out personalized letters and e-mails to alert participants that they were unable to reach them to conduct the interviews. Study contact information was provided in this correspondence.

9. A hard to reach list was compiled weekly, and listed participants that were deemed difficult to reach. As detailed in the consent form, research assistants utilized the electronic medical record system to obtain new contact information for participants who were hard to reach.

10. RAs searched participants' medical records for upcoming clinic visits for the hard to reach participant or baby. RAs would meet mothers at their visits to either issue an in-person reminder of the upcoming interview, or to conduct the interview in person if the visit was within the interview window. Updated contact information was also obtained at this time.