Retaining Traditionally Hard to Reach Participants: Lessons Learned from Three Childhood Obesity Studies

Abstract

Retaining underserved populations, particularly low-income and/or minority participants in research trials, presents a unique set of challenges. In this paper, we describe initial retention strategies and enhanced retention strategies over time across three childhood obesity prevention trials. Hip Hop to Health Jr. (HH) was a randomized controlled trial (RCT) testing a preschool-based obesity prevention intervention among predominately African-American children. Retention was 89% at 14 weeks, 71% at 1-year, and 73% at 2-year follow up. Primary retention strategies for HH included: 1) collaboration with a community-based organization to enhance program credibility; 2) continuity of data collection locations; 3) collecting detailed contact information and provision of monetary compensation; and 4) developing a detailed tracking/search protocol. In a follow-up trial, Hip-Hop to Health Jr. Obesity Prevention Effectiveness Trial (HH Effectiveness), 95% of participants completed assessment at 14 weeks and 88% completed assessment at 1 year. For this trial, we emphasized staffing continuity in order to enhance participant relationship building and required data collection staff to have relevant community service experience. In a third study, we assessed dietary quality among participants in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) across three time points following the WIC food package shift instituted nationally in 2009. Retention rates were 91% at 12 months and 89% at 18 months. For our WIC study, we augmented retention by developing a home data collection protocol and increased focus on staff diversity training. We conclude with a summary of key strategies and suggestions for future research.

1. Introduction

Despite recent improvements, the rates of obesity among preschool children remain alarmingly high.^[1] Without intervention, obese children are more likely to remain obese into adulthood, increasing the risk of poor health outcomes.^[2–4] Hispanic and African American children ^[1] and children from low income families ^[5] are especially at risk for excess weight gain as they age. Results from the 2011–2012 National Health and Nutrition Examination Survey (NHANES) found that among 2- to 5-year olds, 3.5% of Non-Hispanic Whites, 11.3% of Non- Hispanic Blacks, and 16.7% of Hispanics are obese, and the prevalence of obesity is even higher among 6–11 years olds (13.1% of Non-Hispanic Whites, 23.8% of Non-Hispanic Blacks, and 26.1% of Hispanics).^[1] Thus, the preschool years represent a critical period to address weight- related health behaviors among children at the highest risk for obesity. ^[6–7]

One possible contributing factor to the existing obesity disparities among preschool children is that minority and low-income participants are often underrepresented in obesity prevention and treatment trials. ^[8–10] It is critical for researchers conducting obesity prevention trials to recruit and retain low-income and ethnic and racial minority groups to address existing health disparities ^[11] and to increase external validity and the ability to generalize findings. ^[12] Many barriers exist to recruiting and retaining underserved, low-income and ethnic/racial minority participants, including both logistical and attitudinal barriers. ^[8] Some of the identified barriers include transportation, childcare, cultural and language differences, inflexible work hours, communication barriers, ^[13] distrust in health-related research, and less reliable contact information. ^{[8, 10; 14–17]}

To date, there have been few obesity prevention trials targeting preschool children, and even fewer targeting low-income and African American and Hispanic children. ^[18] Monasta and colleagues ^[18] conducted a systematic review of 7 available randomized controlled trials for obesity prevention among preschoolers. Most of the papers reviewed did not report the ethnicity of the participants randomized, ^[19–22] one trial randomized mostly Non-Hispanic White participants ^[23], one was specifically for Native American children ^[24], and a third specifically recruited racial minority participants (e.g., primarily African American participants). ^[25] Williams and colleagues developed a dietary intervention for low-income minority preschoolers attending Head Start programs which aimed to reduce cholesterol intake. ^[26] They reported a retention rate of 90.4% at 6 month follow up, but specific retention strategies were not described. .

Over the past 15 years, our research team has designed and implemented several obesity prevention trials for low-income and ethnic/racial minority parent/child dyads. ^{[7, 27]} Each of the trials included longer-term follow up with participants (12 months or greater). The purpose of this paper is to describe retention strategies across three childhood obesity prevention trials. Specifically, we present how strategies evolved and were tailored given our previous “lessons learned” and unique considerations given study population and design. Given the lack of research on retention strategies for obesity prevention trials among ethnic and racial minority preschoolers, we identified common barriers to engagement in research from the health disparities literature ^{[8, 10; 13; 14–17]} and built our retention strategies in order to problem solve around these barriers given our unique population and study designs. We present each of the study designs and results, followed by retention rates and retention strategies. Over the course of the three studies, we learned from experience and enhanced our retention strategies, when necessary, in order to improve our retention rates (See Table 1). We highlight the challenges of reconciling the inherent discrepancies between the needs of a highly structured research protocol and lack of structure and predictability in the lives of many from underserved and minority backgrounds. We provide strategies and discuss future directions in order encourage additional research among underserved populations.

Table 1.

Retention Strategies Utilized Across Three Trials

	HH	HH Effectiveness	WIC
Collaborated with a community-based organization to enhance program credibility	X	X	X
Continuity of data collection locations	X	X	X
Collection of detailed contact information and provision of monetary compensation	X	X	X
Developed a detailed participant tracking/search protocol	X	X	X
Staff continuity		X	X
Strategic hiring of data collection staff		X	X
Developed a home data collection protocol			X
Increased focus on staff diversity training			X

Note. HH = Hip Hop to Health, Jr. Efficacy Trial; HH Effectiveness = Hip-Hop to Health Jr. Obesity Prevention Effectiveness Trial; WIC = Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)

2. Obesity Prevention Trials: Study Design, Results, and Retention

2.1 Hip-Hop to Health Jr. (HH)

2.1.1 Study Design and Results

Hip Hop to Health Jr. (HH) was a school-based nutrition and physical activity obesity prevention program designed for a predominantly African-American preschool population (81% of children were African-American, and 11% identified as either Latino or Multiracial).^[7] The HH program included nutrition and physical activity programming directed to both children and parents. The child component, delivered by trained early childhood educators during school hours, included a 40-minute intervention delivered 3 times weekly for 14 weeks. The parent component included weekly newsletters that complemented topics covered during the child sessions. Parents were asked to assist their child in completing a weekly homework assignment related to the in-school activity. We evaluated the intervention by comparing changes in body mass index (BMI [kg/m²]) and BMI Z-score in 3–5 year old children randomized to either HH or a general health intervention (GH). Results at the 1 and 2 year follow-ups showed that children in HH had significantly smaller increases in BMI and Z-scores compared to children in the GH control group. HH was the first intervention to exhibit positive effects on BMI in low-income, preschool children. ^[7] At post-intervention (14 weeks), we collected follow-up height and weight from 362 of our original 409 randomized participants (89%). At 1 year follow up, we were able to collect anthropometric data from 71% of our originally randomized participants, and this percentage increased at 2-year follow up (73%; See Figure 1). There were no statistically significant differences in retention by treatment group.

Figure 1.

Hip-Hop to Health Jr. Study Design and Participant Flow

2.1.2 Key Retention Strategies

1) Collaboration with a community-based organization to enhance program credibility

Previous research has found that partnering with known community-based organizations can help to improve retention rates among minority populations. ^[9,15] The HH study was designed to be conducted in Head Start preschool programs. Our partnership with Head Start centers increased the projects credibility in the eyes of potential participants because Head Start is a known and trusted community educational organization. Therefore, we were more able to retain participants because of an enhanced sense of trust due to our relationship with Head Start.

2) Continuity of data collection locations

Because the intervention was conducted in a preschool, the initial recruitment and baseline interviews were conducted at the school, as were the majority of follow-up interviews. Our research teams recruited parents at each participating school over a three-week period at drop-off and pick-up times. During that time, our research teams were able to foster relationships with parents, teachers and school support staff. Face-to-face interactions in the schools during the recruitment and baseline interview phases were helpful as we worked to retain participants for follow-up interviews. Developing relationships with school staff was also critical following the active intervention, as school staff were often more available to communicate with parent participants and able to provide helpful information regarding difficult-to-contact participants.

3) Collection of detailed contact information and provision of monetary compensation

Parents completed measures at post-intervention (14 weeks) and 1- and 2-year follow-up. The effort to retain participants over the course of the study influenced the interview protocol. As part of the interview, we asked the participant to provide contact information for two people who were close friends or family who did not reside with the participant, but would be aware of their location. This information often proved critical when we attempted to locate difficult to reach and more transient participants. Additionally, we provided $25 in compensation for participants at baseline and increased that amount by $5 at each follow-up.

4) Developed a detailed participant tracking/ search protocol

We developed a tracking protocol with detailed procedures for maintaining contact with participants, and a tracking database that staff could readily access in order to maintain an up-to-date record of contact attempts and outcomes. Our tracking protocol included phone calls and mailings at scheduled intervals between data collection points. We contacted participants bi-monthly by telephone to verify contact information and "check in." We attempted to contact a parent by phone no more than three times over a two-week period, and if we were not successful, we would send a letter requesting a return phone call. We mailed thank-you cards to participants following each interview and also sent cards on Mother’s/Father’s Day, the child’s birthday, and the winter holiday season. For each mailing, we would track and record all returned mail.

For participants we could no longer reach during the tracking and follow-up interview periods, we developed a search protocol. As part of that protocol, we called and then sent letters to the contacts provided by participants, and we conducted online searches that included the white pages, credit bureau, prison system, and death record index of Cook County.

2.2 Hip-Hop to Health Effectiveness Trial (HH Effectiveness)

2.2.1 Study Design and Results

The Hip-Hop to Health Jr. Obesity Prevention Effectiveness Trial was an NIH-funded 14-week effectiveness trial delivered by classroom teachers and modeled after the HH trial. ^[27] Ninety-four percent of participants enrolled in HH Effectiveness were African-American, and the remaining 6% identified as either Latino or Multiracial. Improvements in screen time and physical activity were observed at 14 weeks and improvements to diet was observed at 1 year post intervention; however, no significant between group differences were observed in BMI z-score at either time point. At 14 weeks, height and weight were measured for 589 of the 618 participants (95%), and 4 participants completed some measures but not height and weight. At one year post-intervention, height and weight were measured for 543 out of 618 children (88%), and 4 completed some measures but not anthropometric measurements (Figure 2). There were no statistically significant differences in retention by treatment group.

Figure 2.

Hip-Hop to Health Effectiveness Study Design and Participant Flow

^a Follow-up was considered complete if the child’s height and weight were measured.

This figure was previously published in Fitzgibbon et al. ^[27]

2.2.2 Key Retention Strategies

1) Staff Continuity

We made an effort to maintain consistent data collection staff throughout the course of the study. During the HH study, we learned that undergraduate students were often not ideal hires as data collectors due to frequent school scheduling changes, which presented a barrier to consistent availability. For the HH Effectiveness study, we took steps to ensure that data collectors could be available consistently during preschool hours. Data collection teams were small, with 2–3 person teams who maintained a regular presence in the schools. This procedure ensured that schools were less likely to encounter unfamiliar data collection staff members. Taking the lessons learned from the HH study, we budgeted for and purposely hired staff that could commit to staying with the study throughout all follow-up data collection periods. Consistent staffing allowed for greater trust and relationship building between the data collection staff and participating families. It also allowed for the retention of detailed information about participating families that were hard to reach, increasing the likelihood that we would be successful in conducting follow-up interviews.

2) Strategic Hiring of Data Collection Staff

Data collection staff for the HH study primarily consisted of recent college graduates with limited research experience. For the HH Effectiveness study, we re-visited our hiring practice to prioritize previous community-based social service experience. It became evident that people with a social service background had more experience with both interpersonal and relationship skill building. Since the relationships our data collection and tracking staff were able to build with subjects had proved to be pivotal in retention for the HH study, we created full-time, longer-term jobs with benefits to encourage staff retention over the course of the study. The changes led to the creation of well-rounded staff positions, and data collectors achieved a multi-faceted understanding of the study. Weekly all-staff meetings allowed for data collectors to share their experiences and problem solve with input from investigators, project coordinator, and data collection staff.

2.3 Analysis of 18-month Impact of WIC Revisions

2.3.1 Study Design and Results

Study Design, Results, and Retention

In 2009, Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) revised its food packages and provided whole grains, fruits, and vegetables and fewer foods with high saturated fat content. ^[28–30] We assessed dietary intake among WIC recipients prior to the revisions, and then 6, 12 and 18 months after the changes were made. ^[31] Greater than 90% of adults in the overall study sample were women and self-identified as either African American or Hispanic. Overall, dietary changes were modest; however, both African-American and Hispanic mother-child dyads decreased whole milk intake at 6 and 18 months after food package revisions were implemented. Before WIC food package revisions were enacted, we recruited 398 parents of 2–3 year old children participating in WIC for a cross-sectional study. Later, participants from the cross-sectional study were invited to participate in the current longitudinal study and a total of 298 parents agreed to participate. Retention was high throughout this study; 295 dyads completed the 6 month interview, 272 dyads completed the 12-month interview (91% retention), and 266 completed the 18-month interview (89% retention; Figure 3).

Figure 3.

Chicago Family Food Survey, enrollment and retention.

^a In 3 cases, the parent/guardian from the cross-sectional study was not available, and a different parent enrolled in the longitudinal study.

2.3.2 Key Retention Strategies

1) Developed a home data collection protocol

During the HH Effectiveness study, we began home-based data collection for more difficult-to-reach participants. It was often an option of last resort reserved for participants who had missed more than three scheduled appointments or could not be reached through a variety of formats. During the WIC study, we realized that unlike our previous school based studies, the WIC sites were not a daily destination for the participants. Therefore, participants were offered a home interview in the scheduling call if that was their preference. Also, if the participant missed their first scheduled appointment, follow-up efforts were made for a home interview. This avoided wasting staff resources and time for participants unlikely to come to a scheduled appointment at the WIC site. Our increased willingness to conduct home data collection greatly increased our retention for follow-up data collection periods.

2) Increased focus on staff diversity training

Although diversity training had always been a component of our staff training, the need for attention to culture was magnified in our longitudinal study because we did not have the opportunity to build trust through the schools as we did with our intervention trials. We were keenly aware of the importance of linguistic and cultural matching with our Latino participants and the precise use of words in our consent form and measures that included foods more commonly consumed among Latinos (e.g., fruits such as cherimoya and guava; vegetables such as tomatillos and nopales (cactus)). We ensured that we had bilingual data collectors on staff, and consulted with native speakers to ensure that our questionnaires would be relevant and comprehendible for Latino families. Additionally, we included cultural competency curriculum as a part of our data collector training. Our training stressed knowledge about the cultures we were working with an emphasis on reflecting on potential bias and the having the utmost respect for our participants. Much of cultural competency training linked well with the regular training in the responsible conduct of research that all of our faculty and staff participate in on a regular basis.

3. Discussion

Successful retention is essential to the success of RCTs and longitudinal studies, with high attrition leading to the introduction of bias and decreased generalizability.^[32–33] Higher loss to follow-up is a significant problem among low-income and minority populations due to transportation challenges, transient living situations, distrust of research, inconsistent contact information, and language barriers. ^[9,14,16]

This paper reported on the strategies utilized over three separate childhood obesity prevention studies targeting traditionally hard to reach participants. ^{[7, 27,31]} Across the three studies, we learned valuable lessons regarding retention, and we modified and enhanced strategies as needed. It is important to plan for successful retention a priori, rather than devise strategies as the study progresses. . ^[17] Prior to recruitment for the three studies, we strategically considered retention strategies given our specific setting, population, and study design. We planned strategies for retention while developing the grant proposal in order to appropriately budget for staffing needs and incentives. Retaining participants across follow-up visits requires significant staff time.^[34] Therefore, resources required for retention should be considered to ensure that appropriate funds are allocated. ^[35] On average, over the course of a 5-year study, we dedicated approximately 20% of our overall budget to staff salary and incentives. The majority of the 20% was spent on staff (~17%) and the other 3% was devoted to incentives. In terms of annual budget allowance, a higher percentage of the budget was used for salary and incentives in the second (~22%) and third years (~25%) when the most participants were simultaneously active in the study. Although we hope that these percentages are helpful, we would like to acknowledge that the percentage of the budget suggested for successful retention may vary across studies depending on sample size, number and role of data collection staff, length of assessment, type of data being collected, and number of follow-up visits. We found it important to work with our grants management team in order to create a budget that best reflected the needs of each study.

Although retention strategies across studies varied, all three studies had a primary focus on staff retention, collaboration with trusted community organizations, and relationship building. These strategies are similar to others reported in similar trials ^[8,16–17], and we believe they are essential to building trust between research staff, organizations, and participants. It was very apparent that the relationship between our academic setting and the research setting (i.e, Head Start programs, WIC ) needed to a cooperative partnership, especially in the school setting where our intervention did not address traditional academic content. Establishing and maintaining mutually respectful relationship with key stakeholders and having ongoing communication at all levels is key. ^[9] All stakeholders were involved in the project from the earliest point of planning the study and developing the details. They were able to provide us with their own policies and regulations that informed the study design, including the curriculum. We incorporated Head Start and the state of Illinois curriculum requirements so that the curriculum would help teachers to fulfill their class time requirements instead of taking time away. We worked with all stakeholders so that we had “buy in” from our community partners, from central office administrators to site managers and Principals to the sites’ janitorial staff. In the case of WIC, we also bought childcare supplies and nutrition education material as a thank you gift for each site that participated.

We recommend recruiting staff for full time positions which include benefits to increase staff retention and also recruit staff with prior relevant field experience. Once hired, our staff was trained on how to build and maintain relationships with organizations/schools and with research participants. Collaborating with trusted community organizations improved initial recruitment of participants and their identification with the study. Retention progress was also monitored in weekly meetings with the principal investigator, project coordinator, and study staff to problem solve around barriers to participant involvement.

In addition to these cross-cutting strategies, we recommend tailoring specific strategies to the specific study setting, population, and study design. For example, for our longitudinal study investigating the impact of WIC revisions on dietary quality, we realized that our setting and participants required additional strategies. As compared to the school setting where parents and children are generally present 5 days a week, our WIC settings were visited less frequently. Therefore, we developed a home data collection protocol to reach participants who had transportation barriers. Additionally, we increased focus on diversity training to facilitate relationship building. Study-specific training and cultural competency training are keys to improving retention efforts. ^[36–38] Although our studies mainly consisted of African American and Latino participants, we believe that training in cultural competency across ethnic and racial groups might be important in improving trust and ultimately, improving retention rates.

In our literature review, we found that many published childhood obesity prevention studies lacked critical information. For example, many studies did not include the race/ethnicity ^[19–22] of participants, making it difficult to compare their retention rates to ours. One study developed a dietary intervention for low-income minority preschoolers attending Head Start programs and reported a high retention rate, but retention strategies were not described. . ^[26] We suggest that future studies report retention data at all time points and provide some description of retention strategies used to help inform future work. . We also suggest that future studies explicitly plan for retention prior to recruitment to improve the external validity of research findings.

Overall, we have found that successful retention requires planning ahead, experience, flexibility, and appropriate allocation of resources. We were able to refine strategies we used in earlier studies to increase retention rates over time. When attrition is reduced, a foundation is established for making stronger statements regarding results. It is our hope that future research will utilize and test these strategies in order to provide further support of their efficacy, and that we can work toward developing an evidence-based model that will improve retention among populations at highest risk for health disparities.