Retention Strategies for Health Disparities Preventive Trials: Findings from the Early Childhood Caries Collaborating Centers

Abstract

Objectives

To identify successful strategies for retention of participants in multi-year, community-based randomized controlled trials (RCTs) aiming to reduce early childhood caries in health disparities populations from diverse racial/ethnic backgrounds and across diverse geographic settings.

Methods

Four RCTs conducted by the Early Childhood Caries Collaborating Centers (EC4), an initiative of the National Institute of Dental and Craniofacial Research, systematically collected information on the success of various strategies implemented to promote participant retention in each RCT. The observational findings from this case series of four RCTs were tabulated and the strategies rated by study staff.

Results

Participant retention at 12 months of follow-up ranged from 52.8% to 91.7%, and at 24 months ranged from 53.6% to 85.9, across the four RCT. For the three RCT that had a 36 month follow-up, retention ranged from 53.6% to 85.1%. Effectiveness of different participant retention strategies varied widely across the RCT.

Conclusions

Findings from this case series study may help to guide the design of future RCTs to maximize retention of study participants and yield needed data on effective interventions to reduce oral health disparities.

Introduction

Over the past two decades, the field of oral health disparities research has advanced from observational studies aimed at understanding the causes and consequences of oral health disparities to theory-based interventions aimed at reducing and eliminating disparities. In 2008, the National Institute of Dental and Craniofacial Research (NIDCR), a component of the US National Institutes of Health, funded a consortium of collaborating research centers to conduct community-based caries prevention clinical trials, each using a ‘hard’ clinical outcome, aimed at reducing early childhood caries (ECC) over a two- to three-year follow-up period. Four separate randomized controlled trials (RCTs), each working with underserved populations that have been disproportionately burdened by ECC, were conducted under the auspices of the Early Childhood Caries Collaborating Centers (EC4).

The EC4 is comprised of three separately funded research centers – the Center for Native Oral Health Research (CNOHR), University of Colorado Anschutz Medical Campus, Aurora, CO; the Center to Address Disparities in Children’s Oral Health (known as CAN DO), School of Dentistry, University of California San Francisco, San Francisco, CA; and the Northeast Center for Research to Evaluate and Eliminate Dental Disparities (CREEDD), Boston University Henry M. Goldman School of Dental Medicine, Boston, MA; all three supported by a single Data Coordinating Center (DCC), based at the University of California San Francisco. The community contexts for the RCTs include urban public housing developments (CREEDD), primarily Hispanic communities near the US–Mexican border (CAN DO), and rural American Indian reservation areas (CNOHR). These communities have a high prevalence of ECC and are most likely to benefit from the identification of effective, sustainable and culturally acceptable interventions.

Recruitment and retention of participants from such populations is recognized to be challenging.^1,2 Socioeconomic characteristics may influence the rates of retention, with participants of lower socioeconomic status finding participation intrusive or burdensome.³ Retention of hard-to-reach participants has been a longtime topic of interest, particularly when pertaining to clinical interventions for conditions that require recurring outpatient appointments and consistent follow-up, such as HIV⁴, PTSD⁵, depression⁶, and ADHD.⁷ Moreover, research integrity may suffer if RCTs fail to adequately include and retain hard-to-reach populations, leading to decreased generalizability or validity of findings.⁸ Not overcoming challenges to retention may also contribute to underrepresenting or excluding minorities from research.⁹

The EC4 has earlier reported on recruitment strategies used¹⁰ to enroll over 4,000 participants across four RCTs. Similarly challenging is the retention of enrolled participants over an RCT’s multi-year duration, necessary to assess a hard clinical outcome such as ECC prevention. Trials may fail to yield valid results due to reduced power from significant overall attrition, or to differential attrition between RCT arms. Adequate retention is thus recognized to be essential to an RCT’s success, in regard to its internal validity as well as to its external validity and thus the generalizability of findings to the communities of interest.

Despite the continuing need for, and support of, community-based intervention studies to address oral health disparities, there is a surprising paucity of information in the peer-reviewed literature on successful participant retention strategies for clinical trials in oral health disparities populations. With the continuing need to rigorously test interventions to reduce oral health disparities, the findings reported here should be of value to other investigators planning work in this important area of research. The recruitment and retention of minority populations in clinical and community-based research, particularly that focusing on reducing health disparities, remain challenging tasks. While none of the EC4 RCTs formally tested and compared different retention strategies, as such work was beyond the focused scope of each study, the observations made by each study and their ‘lessons learned’ may nevertheless be useful in guiding the design of future RCTs. In addition, our findings may serve to identify promising research questions for systematic investigation. Assessing different retention strategies in oral health disparities intervention studies may be viewed as a worthy research aim in itself, and may serve to further our understanding of how to effectively reduce and eliminate oral health disparities.

Methods

Study participants and data sources

Data were obtained on participant attendance at scheduled study follow-up visits within the four separate RCTs, for up to three years of follow-up depending on the study. The participating EC4 research centers, their principal investigators, and their RCTs are listed in Table 1. A single Data Coordinating Center (DCC), based at the University of California, San Francisco, was established to serve the EC4 RCTs, and the NIDCR constituted a single Data and Safety Monitoring Board to oversee all the trials. The DCC compiled data on participant retention for the four RCTs. Each RCT’s staff compiled its narrative experiences in implementing various retention strategies and reported them to an EC4 Working Group (WG) on Recruitment and Retention. The Recruitment and Retention WG members represented each RCT on regularly scheduled teleconferences to share information based on their experiences and lessons learned, discuss common problems, and explore innovations on participant recruitment and retention. Each RCT received approval for its protocol from the NIDCR and from each funded center’s Institutional Review Board(s). In addition, each of the two RCTs working with American Indian groups received approval from the relevant tribal review boards.

Table 1.

Randomized Controlled Trials Conducted by the Early Childhood Caries Collaborating Centers

ClinicalTrials.gov NCT	Principal Investigators	Study Title
NCT0111672611	Terrence Batliner, DDS, MBA	CNOHR-I: Promoting Behavioral Change for Oral Health in American Indian Mothers and Children
NCT0111673912, 13	Patricia Braun, MD, MPH David Quissell, PhD	CNOHR-II: Preventing Caries in Preschoolers: Testing a Unique Service Delivery Model in American Indian Head Start Programs
NCT0112944014	Stuart Gansky, DrPH Francisco Ramos-Gomez, DDS, MS, MPH	GIFVT: Glass Ionomer Sealant and Fluoride Varnish Trial to Prevent Early Childhood Caries
NCT0120597115	Michelle Henshaw, DDS, MPH, Belinda Borrelli, PhD	TSHS: Tooth Smart Healthy Start: Oral Health Advocates in Public Housing

The CNOHR Study I is aimed at assessing the efficacy of a Motivational Interviewing intervention to prevent caries in children through knowledge and behavior change in AI mothers in a Northern Plains tribe.¹¹ The RCT’s enrollment goal was 600 AI caregiver–child dyads; enrollment began in September 2011 and was completed in February 2014. Each caregiver-child dyad is recruited when the child is a newborn and followed up until the child is 36 months old. The RCT has completed all intervention activities, and final follow-up assessments will be completed in early 2017.

The CNOHR Study II, a cluster-RCT, enrolled children 3 to 5 years old in 52 Navajo Nation Head Start Programs.¹² It tested whether an intervention of fluoride varnish (FV) administration and oral health promotion education in Head Start classrooms could reduce the number of decayed, missing, and filled primary tooth surfaces in children, as compared to children receiving usual care which included regular dental services provided by the Indian Health Service. Lay tribal members (Community Oral Health Specialists), who had received brief training from oral health and behavioral experts delivered the intervention. FV was offered up to four times/school year; oral health promotion education was provided to enrolled children (5 times/school year) and their caregivers (4 times/school year) in separate sessions for up to two years. Children were followed for up to three years. A total of 1,016 caregiver-child dyads were enrolled. This RCT has completed all data collection and the main outcome paper has been published.¹³

The CAN DO Glass Ionomer Fluoride Varnish Trial (GIFVT) RCT enrolled children aged 2.5 to 3 years, primarily Latinos, at two community health centers (CHCs) in San Diego County, California, to compare the caries prevention efficacy of FV applied every six months as compared to the combination of FV applied every six months and the annual application of fluoride-releasing glass ionomer sealants to the eligible occlusal surfaces of primary molar teeth. The primary outcomes include caries incidence and increment over 3 annual follow-up assessments. This RCT enrolled 597 participants, completed all intervention activities, and completed 36 month follow-up assessments in February 2016.¹⁴

The CREEDD Tooth Smart Healthy Start (TSHS) trial ¹⁵, a cluster-RCT, enrolled mother-child dyads, with children aged 0 to 5 years, residing in public housing developments in the greater Boston area. The youngest eligible child was designated as the ‘index’ child, but study procedures were made available to all age-eligible children in the family. This report includes 1,065 dyads enrolled across 26 separate housing developments. The RCT compares two community-based multimodal interventions: (i) Motivational Interviewing (counseling) delivered to participating parents/legal guardians by dental health advocates (lay providers who were also mothers residing in public housing), child FV application, child oral health assessment, and referral to dental health services as needed; and (ii) written oral health education materials regarding ECC prevention, child FV application, child oral health assessment, and referral to dental health services as needed. All intervention components, including FV, are delivered quarterly for two years. The primary outcome is 2-year incidence of caries. This RCT is currently ongoing. The final 24-month outcome assessment on all participants is expected to be completed in November 2016.

All studies offered various levels and forms of compensation for attendance at study visits. The various retention strategies developed and implemented for each trial were summarized and tabulated. The observations made by each trial regarding their experiences with various strategies were categorized by their strengths and weaknesses and the strategies’ effectiveness rated on a four-point scale (A=excellent; B=good; C=fair; D=poor) by each trial’s staff.

Recruitment and Follow-up

Descriptive statistics were generated regarding numbers of enrolled participants, children and/or their caregivers, who did or did not return for scheduled study follow-up visits at the 12, 24, and 36 month study examination windows, as related to each RCT’s protocol. The follow-up percentages at scheduled follow-up visits for each RCT were tabulated through December 2015 (“% with follow-up”). Because three RCTs were still actively following participant dyads as of December 31, 2015, the calculation of each follow-up rate was restricted to dyads that already had been due for the corresponding follow-up visit. To convey the progress of each RCT for each follow-up visit, RCT summaries also include the percentage of dyads for which the time of scheduled visit had passed (“% with visit window closed”). For each RCT, the summary statistics are based upon the sample that will be the focus of the trial’s main outcome analyses.

Results

As of December 31, 2015, only one trial, CNOHR-II, had completed all data collection activities (Table 2). For the CNOHR-II study, over 50% of eligible participants attended their 12, 24 and 36 month visits. The CNOHR-I study will complete all its scheduled 36 month visits in mid-2017. Through 2015, CNOHR-I participant attendance at the 12, 24 and 36 month visits has exceeded 80% (Table 2). Similarly, over 80% of eligible GIFVT participants attended their 12, 24 and 36 month visits. The TSHS study has achieved an over 70% attendance rate for its scheduled 12 and 24 month visits (Table 2).

Table 2.

RCT Participant Characteristics, Enrollment, and Percentage Follow-up^*

ClinicalTrials.gov NCT	Study location	N children enrolled	Mean age±SD at enroll-ment	Participant Race/ethnicity	12-month follow-up		24-month follow-up		36-month follow-up
					% with follow-up	% with visit window closed §	% with follow-up	% with visit window closed §	% with follow-up	% with visit window closed §
NCT01116726	CNOHR-I: Pine Ridge Reservation and environs, South Dakota	579	0 ± 0	American Indian	91.7	100	85.9	99.5	83.2	57.5
NCT01116739	CNOHR-II: 52 Navajo Nation Head Start classrooms	897	3.7 ± 0.5	American Indian	52.8	100	53.6	100	53.6**	100**
NCT01129440	GIFVT: Two community health centers in San Diego County, CA	597	3.3 ± 0.5	Hispanics	87.6	100	84.9	100	85.1†	97.6†
NCT01205971	TSHS: 26 public housing developments in Boston, MA, area	1065	2.7 ± 1.7	Hispanic, African American, Non-Hispanic white	70.8	100	82.2††	89.3††	n/a§§	n/a§§

SD = standard deviation

^*Follow-up data cut-off date: December 31, 2015

^**Among the n=442 dyads that enrolled in 2011 and were therefore eligible for a 36 month follow-up

^†2 months of follow-up remaining as of data cut-off date (Dec 31, 2105)

^††11 months of follow-up remaining as of data cut-off date (Dec 31, 2015)

^§the percentage of dyads for which the time of the scheduled follow-up visit had passed, as of December 31. 2015

^§§TSHS designed for only up to 24 months follow-up.

Project-specific Lessons Learned (Table 3)

Table 3a.

CNOHR–I (ordered from top to bottom, most effective to least effective, and graded A to D)

Retention Strategy/Strategy Rating	Description	Start	End	Strengths	Weaknesses
Facebook A	Use private messaging to schedule appointment and get updated contact information. Oral health messages on the project page	9/18/12	Ongoing	Messages sent to people to let them know a team will be in the area and to remind them of upcoming appointments	Difficult to find people unless they provide their FB name
Concentrated scheduling A	Scheduling as many participants in 1 area as possible, to minimize staff travel time.	10/1/12	Ongoing	Saves travel costs and field team time; team can make appointments or call when in the community; allows the field team to be more flexible in scheduling	Field team is committed to one area for the entire day; cannot see participants who walk-in to the office
Post cards B	Birthday post cards: sent in the month prior to a child’s birthday reminding about visit and dental screening Reminder post cards: mailed or left at participant’s home as needed to get in touch with participant 30-Month contact post cards sent to participants who cannot be reached by phone at 30 months. Serves as a bridge between 24 and 36 month visits. Asked to mail postcards back with updated contact info. Postage paid envelope is included.	9/18/12	Ongoing	Still can reach people who have their phones turned off	Cost, difficult to send/receive mail on the reservation; people move so addresses maybe incorrect
Billboards B	5 around the reservation		Ongoing	Brands the study and keeps people familiar with the study and oral health benefits	Costly; difficult to maintain; storage issues
Community activities B	Participation at Halloween parades, basketball games, monthly health fairs in the districts Lakota Nation Invitational (LNI) tournament Oglala Pow Wow Kids Fair Homeless Connect Give-away for children	10/31/12	Ongoing	General oral health message to the community; exposure for the project in the community; see hard-to-reach participants at community events and update contact information; builds team pride and community connections; gets much-needed oral health supplies to community	Additional costs not covered by grant; takes field team away from field work
Opening up the semi-annual CAB meetings to family members B		12/2/13	Ongoing	Provide feedback and spread the word about the study; higher participation	More expensive; more distractions with kids
Literature for participants C	Future appointment magnets Retention tear-off pads Information brochure	9/18/12	Ongoing	Visual reminder for participants; contains field office contact information	Additional cost; can have incorrect information (phone numbers, dates)
Newspaper ads with picture of field staff C		1/1/14	Ongoing	Makes people familiar with field staff and they often get recognized in the community. People consider them “safe” to let into their homes.	Adds to cost

For both CNOHR studies, many challenges were related to working in remote and rural settings. Involving the community in the development of retention strategies and delivery of interventions was critical. The CNOHR-I Motivational Interventionists and the CNOHR-II Community Oral Health Specialists were all community members. They were familiar with local customs, the geography of each Reservation, spoke the local tribal language and were sensitive to the life-circumstances of study participants. These characteristics allowed them to recognize and meet the needs of study participants.¹⁶

Using the local media, including newspaper ads, billboards with oral health messages, and airing interviews with the study PI and field staff on the local tribal radio helped to spread the word and establish trust for CNOHR-I on the Reservation (Table 3a). Moreover, the field staff became recognized in the community as representatives of the study, and participants felt safe and comfortable having them in their homes.¹⁷ This rapport between the research staff and the community helped in retaining study participants and tracking participants who had been lost to follow-up.

For CNOHR-II, sending personalized letters was one of the most effective retention strategies. Other strategies (Table 3b) such as, telephone calls, text and email had varying success.¹⁷ In the final year of data collection, home visits were added that facilitated retaining participants as it was convenient for the participants and accommodated their work schedules. These visits, however, added to study costs.

Table 3b.

CNOHR–II (ordered from top to bottom, most effective to least effective, and graded A to D)

Retention Strategy	Description	Start	End	Strengths	Weaknesses
Mailed letters A	Letters sent to caregivers notifying them of the fall data collection and the times staff would be in their community. Letters were sent to secondary contacts if first attempt unsuccessful.	Fall 2011 Spring 2012 Fall 2012 Fall 2013 Fall 2014	Winter 2012 July 2012 Winter 2013 Winter 2014	Letters with study logo went to specific participant.	Families moved/ changed address; didn’t assess if caregiver received/read letters
Home Visits B	Families were given the option of a home visit for final data-collection event.	Fall 2014	Winter 2014	Convenient for families; provided flexible after-school times for children and after-work times for caregivers.	Additional cost for travel. Difficult to find participant homes; non-productive use of staff time to find/travel to home; occasionally families not home when staff arrived.
Phone calls B	Phone numbers of participants and secondary contacts were collected at enrollment and confirmed/updated at each data collection event. Team members called families to communicate study-related events.	Fall 2011 Spring 2012 Fall 2012 Fall 2013 Fall 2014	Winter 2012 July 2012 Winter 2013 Winter 2014	Working telephone numbers provided a method of communication with participants. If participant was not home—family member passed on message.	Some phones out of service/disconnected, out of service-range; calls not always answered; some participants preferred to be texted
Phone texting B	Study staff would text messages regarding study-related activities	Fall 2012 Fall 2013 Fall 2014	Winter 2013 Winter 2014	Economical way for participants to receive study information; efficient use of cell-phone minutes	Difficult to track if messages received and/or ignored.
Fliers C	Head Start teachers sent home generic study fliers to children to give to caregivers with date/time of study recruitment/enrollment events.	Fall 2011 Spring 2012 Fall 2012	Winter 2012 July 2012 Winter 2013 Winter 2014	Fliers did not require a specific participant address.	Not sure if caregiver received/read flier.
Events at Chapter Houses C	Conducting enrollment/retention events at Chapter Houses.	Fall 2011 Spring 2012 Fall 2012	Winter 2012 July 2012 Winter 2013 Winter 2014	Use of Chapter Houses provided a community location with which participants were familiar; demonstrated community support of the project; some Chapter Houses waived the usual-use fee for the use of the space.	Little to no flexibility of the time and date of the event; time/date not always convenient for participants; required families to get own transportation to event; loss of productive staff time if no or few families showed; some Chapter Houses closed at 5:00; other community events occasionally superseded our events
Booths at local fairs/community events C	Staff would attend/rent booth at local community fairs/pow-wow	Fall 2011 Fall 2012 Fall 2013 Fall 2014	Winter 2012 July 2012 Winter 2013 Winter 2014	Took advantage of large community events that most members of community attended; used materials with logo to help community members identify project.	Many other activities happening at events at the same time.
Newspaper ads C	An insert was put into the local newspapers	Fall 2013	n/a	Broad distribution to community members	Few participants saw fliers.
Word of mouth C	Head Start teachers contacted participants on occasion; participants occasionally notified others regarding study	Throughout project	Throughout project	Personal communication with community member	Difficult to track these activities.
Fliers sent home by Elementary School Teachers D	Elementary school teachers sent home generic fliers with kindergarten and first grade children	Fall 2014	Winter 2014	Reach out to children once graduated out of Head Start	Children graduated into > 100 different schools; difficult to assess if caregiver received flier.

GIFVT staff built upon knowledge from a previous study in the same community which focused on barrier reduction, incentives, and relationship building.¹⁸ GIFVT investigators found that having a community outreach specialist who either comes from the community or manages to integrate with the community helped build trust (Table 3c). In order to integrate with the community, the outreach specialist and the rest of the study team should not only be able to communicate in the predominant language (in GIFVT’s case, Spanish) but also be familiar with the culture that forms the community. While in GIFVT most participants were of Mexican origin, the study composition varied from families who had just arrived in the United States with a first language of Spanish, to families who have lived in the U.S. for several decades with communication mainly in English, and who may have a separate culture within the community.

Table 3c.

GIFVT (ordered from top to bottom, most effective to least effective, and graded A to D)

Retention Strategy	Description	Start	End	Strengths	Weaknesses
Community Outreach Specialist  A	A bilingual/bicultural “case manager” for difficult/hard-to-reach participants follows up with participants via telephone, postcards and while waiting for other appointments at the health centers. The coordinator often searches for new ways to get in contact with individuals whose telephone numbers are no longer in service through secondary numbers.	Study Initiation	Study End	This role allows for a “patient-centric” relationship with participants and their caregivers, and allows the coordinator to identify the causes of missing visits; which can be directly addressed and may help prevent attrition in participation.	This is a time consuming task, and often the coordinator faces difficult people on the other side of the call. However, it is a very effective way to find participants who missed appointments and who have moved out and then have returned to San Diego.
Personalismo and Familismo A	Establishing rapport between staff and participants is vital; thus hiring staff that understands, appreciates and prioritizes the personal touch is extremely important.	Study Initiation	Study End	Staff is bilingual and bicultural and are familiar with the community; moreover, group building activities like birthday celebrations, group outings to help impoverished communities (outside work schedule) motivated the team to work together to improve the lives of the communities served.	Finding the adequate staff may be time-consuming and a challenge when hiring through a federally qualified health center. Finding bilingual/bicultural individuals who also presented computer literacy was challenging, thus bilingualism/biculturalism was leveraged with other essential desired features. Constant training, trial and error and micromanagement at early stages proved to work. Some of the measures to consider are the turnover rate, absenteeism, and conflict rate.
Family/Child Friendly Environment A	The operatory room at one site is decorated with animal characters, toys and a small table is found in the examining room and in the interviewing room. Children are invited to play while the caregiver is being interviewed.	Study Initiation	Study End	Easy to implement; however if the setting does not have enough room, it will be difficult to add a table and chairs with blocks, books and other toys.	It is difficult to measure the impact of this setting; however children often stop by since they are drawn by the decorations and the toys. In addition, parents state how much they like the setting (the nontraditional dental setting)
Invite back participants who miss or skip study visits (Retention Visits) A	Participants are always invited to return for missed or skipped visits. This is important to all visits, but specifically to final (36 month) visits.	Study Initiation	Study End	Easy to implement and allows for continuing contact with families via telephone, postcards or walk-ins.
Post Cards A	Staff identifies participants who have missed a visit or have a disconnected/wrong phone number and send them postcards. The outreach coordinator and other staff members mail out “Reminder” postcards to every participant a week prior to the scheduled visit. The site’s contact information is included in case participants need to reschedule. All materials are available in English and Spanish.	Study Initiation	Study End	The outreach coordinator and other staff members record the number of postcards mailed out in the clinical trials management system (CTMS). In addition, most contacts with participants are also recorded in the CTMS. Returned postcards suggest participants that may have moved away. However, the team continues mailing postcards and some participants return after long periods of absence.
Compensation – FV/Glass Ionomer Sealants Application B	Participants receive a $30 gift card when baseline, 6, 12, 18, 24, 30 month visits are completed; and a $60 gift card when 36 month visit is completed ($240 in 3 years total) A sub-sample participates in a sub-study which includes additional activities. These participants receive $60 gift card when baseline, 6, 12, 18, 24, 30 month visits are completed; and $90 gift card when 36 month visit is completed ($450 in 3 years total) All children enrolled in the study receive fluoride varnish applications semiannually for 3 years. Individuals in the intervention group receive Glass Ionomer Sealants at baseline, and if required at 12 and 24 month visits.	Study Initiation	Study End	Provides oral health prevention to a population at high risk for ECC. For some participants this is the first contact with an oral health provider, and for others this is the only way to access oral care. In addition, Glass Ionomer Sealants are applied in primary molars to individuals in the intervention group, which is not the standard of care, and hence some families feel very motivated about this treatment.	This population faces multiple obstacles to attend visits (e.g., transportation, relocation, family illness, ill participant, work and school schedule of the individual, caregiver and/or sibling conflict with study visits, weather, etc.). Likewise, the compensation and services received may affect individuals differently depending on expectations and needs.
Raffle C	All participants who attend a study visit in a given month are entered into a drawing to win a $60 gift card. At each study visit participants are informed they are participating in a drawing. At the end of the month, two participants are randomly selected.	2/1/13	Study End	Easy to implement and participants seem motivated by this initiative.	It may be difficult to measure the impact of this initiative.
Birthday Cards C	GIFVT sends a birthday card to each participant in the study each year.	1/2013	Study End	Easy to implement; requires few resources	The effect of the birthday cards is difficult to measure, though anecdotally we have received positive feedback from participants.

Creating a family and child-friendly space in the Community Health Center (CHC) dental home, as done in a prior study¹⁸, made families more comfortable and encouraged them to attend study visits. For example, an office with toys for siblings and participants, and with enough room to store strollers made parents feel welcomed. In addition, having diapers at the CHC and other items needed to care for children may have positively impacted attendance. Furthermore, having staff who can naturally and comfortably interact with toddlers was an asset to GIFVT (e.g., genuinely speak to children about the latest cartoon characters, movies, toys).

Finally, after assessing community patterns of attendance by using historical data¹⁸, the study team was able to anticipate broken appointments, and hence add several tools to stay in contact with the families, such as telephone calls and postcards, and by also reaching out to secondary and tertiary contacts. In a context where pre-paid mobile phones are normative, the team had to find alternative ways to stay in contact with participants with expired mobile phone minutes or changed phone numbers.

Building trust within community was also key to retention for the TSHS RCT. The community includes the participant population, the management staff at the Boston Housing Authority (BHA) sites, and the tenant task force groups. The majority of the TSHS staff were bilingual and were current or previous residents of the BHA sites, which allowed them to relate to the participants and develop rapport. TSHS staff were individually assigned to BHA sites and each endeavored to build relationships between the research project and that particular site. This promoted BHA’s trust in participating in research, and staff members felt comfortable conducting home visits for those who were difficult to reach by phone, text or email. Many times staff members had been to the participant’s home before and knew the best time of day to reach her. Over time, staff members began to ascertain the best days and times to reach participants, and would often travel to their homes during these times to conduct surveys or counseling for participants who were difficult to reach using other methods.

To further build trust with the community, including the management offices and tenant task force groups, the TSHS team became actively engaged with the community. For example, each summer the team would participate in “Back-to-School” events at the different sites by setting up “Tooth Smart Healthy Start” booths and creating activities for the children. Additionally, the coordinators and other staff members attended community meetings to give updates on the progress of the study. To acknowledge the hard work and assistance of BHA management staff, TSHS put together holiday gifts each year and personally delivered them to the sites.

Throughout the TSHS RCT, post cards were mailed, and also left at participants’ homes by staff members for those participants who were unable to be reached to schedule appointments or who were not home at the time of a scheduled visit. The post cards were developed in a way that allowed the team to personalize each post card with a handwritten note in the participant’s language of preference. The post cards highlighted the visit type for each participant (e.g., assessment or counseling) and whether she would be receiving a gift card after completion. Additionally, this method of retention via postal service allowed TSHS to identify when a participant had moved from her current residence, as the post card would be returned to the office.

The most effective approach to retention for TSHS was the Community Outreach Specialist position, which was created in the latter half of the RCT. This bilingual specialist’s time was entirely devoted to contacting and conducting study visits with hard-to-reach participants. The methods used by the specialist included going to the housing site at different times throughout the week, such as weekends and evenings, speaking with the management office regarding participants who relocated, and calling numbers that were previously not working. The Community Outreach Specialist was someone who was skilled in effectively communicating with participants who were considering leaving the study or who had missed multiple study visits. A main focus for the position was on having participants complete their final study visit. This effort resulted in improved attendance at the final 24-month visit over the interim 12-month visit.

Discussion

Our findings illustrate the successes and challenges with using a variety of retention methods that are common to other longitudinal research, namely mailings, phone calls and text messaging, and compensation. Mailings included four main types of contacts: use of birthday or holiday cards, reminders for scheduled visits, check-ins for participants who missed several visits, and notifying the community regarding staff presence in the area and data collection times. Overall, mailings were beneficial, especially for participants who were hard-to-reach by other means of communication or for identifying participants that had moved from their addresses on file, a strategy commonly employed by others.^{18, 19} Mailings were easy to implement and required few resources; however, there is difficulty in quantifying the effectiveness of mailings such as birthday or holiday cards. Also, while identification of outdated addresses might provide valuable information for the research team, it may not overcome the challenge of reaching participants.

Extensive contact has been shown to improve retention in studies with demanding protocols^{18, 20}, with two common methods being use of phone calls and text messaging. Phone calls provided valuable opportunities to have consistent contact with participants. However, challenges included phone services being disconnected or calls not being answered. Some participants preferred text messages, consistent with research showing that texting is particularly useful for tracking and contacting hard-to-reach populations.²¹ Possible reasons for preferring texting could include the ubiquity of cellular phones, the less-obtrusive aspect of texting as opposed to voice calls, and possible lower costs of cellular phone texts versus voice communications especially for cellular plans with unlimited texting. This preference could be easily honored early on by including a “best way of contacting” item to enrollment forms, a strategy suggested by a Navajo nation Community Advisory Board (CAB).²² Barriers still remain, such as poor cellphone reception (particularly in rural areas), lack of cellphone minutes or data, and changing cellphone numbers.²²

Compensation has been shown to improve retention, particularly when payments are higher and in cash, but perceived coercion^{18, 23} is an important consideration.²⁴ Cash compensation may be used more often by participants to purchase essentials²³. Gift cards, depending on the vendor, may also be useful for purchase of essentials. However, monetary compensation may not be a critical motivator for all participants across a population, especially those studied here, who face multiple obstacles to attending visits. Similar to other findings, while compensation has some impact, it is difficult to quantify and may have differential impact across participants and studies.²⁵ As further noted by Abernethy et al.²⁵, compensation is closely related to issues of access to care and disparities in care. In that regard, retention may be high in studies that provide intervention services that study participants highly value.²⁶ This may be true for the provision of FV to all children in TSHS and for the use of glass ionomer sealants in GIFVT. As such sealants are not the usual standard of care for primary teeth, families may have felt motivated by receiving this extra treatment at no cost.

While various common methods were used, these four studies also had to pay particular attention to the communities with whom they worked in order to have successful retention. This community engagement included attendance at community activities and the use of CABs, which were required by the NIDCR funding mechanism supporting these centers. CABs have been used previously for retention among American Indian populations for behavioral interventions to reduce ECC¹⁶, and CNOHR continued to use them while opening up CAB meetings to family members. This allowed for more feedback from community members and also created respectful relationships with families, which has also been shown to increase retention.²⁶

Most importantly, successful retention was aided by creating rapport with the community. Retention strategies are most effective when there is a sense of trust between participants and study staff^{18, 27}, and strategies for retention must involve the entire community.²⁸ These four studies used a variety of means for community engagement tailored to the target population. Activities included community-wide celebrations, holiday parties, health fairs, and meetings at local community gathering places, such as tribal Chapter Houses.

Approaches used paid particular attention to cultural sensitivity, in order to establish valuable cultural connections.²⁹ For example, cultural sensitivity in the Latino communities may mean incorporating family as part of recruitment, showing respect by using formal titles and listening carefully, and having interactions that are warm and personal.^{18, 30} The GIFVT study demonstrated this by focusing on personalismo and familismo, both of which were population-specific values that helped to establish rapport between staff and participants.¹⁸ Bilingual staff, in particular, are crucial to increasing retention^{18, 28, 31}, and both GIFVT and TSHS studies made efforts to hire staff who were bilingual and bicultural in order to increase staff-participant rapport. However, there were challenges to this. Finding bilingual/bicultural individuals with adequate qualifications was challenging, though mediated by a number of strategies, such as extensive training and close supervision in early stages of the studies. Despite these strategies, staff turnover and absenteeism were common challenges.

Our findings, while novel and potentially important, have several limitations. This report represents a “case series” of observations made by four distinct RCTs. None of these RCTs formally tested or directly compared different retention strategies, as such work was beyond the focused scope of each RCT. The findings reported on each RCT’s retention strategies, while collected systematically, are not experimentally based and are essentially anecdotal in nature. In addition, each trial focused on particular populations that varied greatly among the four RCTs in the race/ethnicity of participants and in their geographic settings.

Nevertheless, these observations may be useful in guiding the design of future RCTs and identifying promising research questions for systematic investigation in disparities populations. For example, future research could include exploring how health behavior theory constructs operate within the realm of retention and whether certain theoretical constructs are more or less predictive of participant retention. Approaches could follow the work of Spoth and Redmond³² on how constructs of the Health Belief Model are weighted differently in predicting a behavioral outcome, while also taking into account context features that may simultaneously influence beliefs.³² For the purpose of studies like ours in the EC4, the intended behavioral outcome is ongoing participation, i.e., retention. Analysis of retention methods’ effectiveness could apply Spoth and Redmond³² in two primary ways: (a) studies could focus on the interrelationship of various constructs of social behavioral theories, such as the Health Belief Model, and how constructs affect each other and ultimately predict the intended outcome, while (b) noting relevant participant characteristics unique to each hard-to-reach study population and describing the effect of the context on health behavior theory constructs. Future studies, in their formative phase, could include collection of in-depth baseline data from participants regarding various constructs, e.g., perceived susceptibility. After studying the relative predictive value of each construct, findings could be used to inform how to best achieve retention with methods that reflect the most salient predictive constructs in their chosen population.

In conclusion, improving efforts to recruit and retain hard-to-reach populations will improve the rigor, quality and efficiency of intervention studies aimed at reducing oral health disparities. An increase in this evidence base is necessary for improving oral health outcomes. Our findings may help to guide research teams on how to better design future RCTs in order to maximize retention of study participants and yield needed data on effective interventions to reduce oral health disparities.

Table 3d.

TSHS

Retention Strategy	Description	Start	End	Strengths	Weaknesses
Community Outreach Specialist A	A “case manager” for difficult/hard-to-reach participants, especially 24-month participants. The outreach workers focuses directly on the retention of difficult participants through home visits and the ability to build relationships with these participants.	9/2014	Ongoing	Adequate time to devote towards the retention of hard-to-reach participants. Ability to conduct multiple home visits at different times of day. Ability to focus solely on these participants unlike the regular TSHS staff.	As the study progresses, there is a larger accumulation of hard to reach participants, so it is difficult to assign all of these participants to a single case worker.
Home Visits (Retention Visits) A	Retention Visits are conducted both by part-time outreach workers and TSHS staff. A retention list consisting of hard-to-reach participants with multiple missed visits or no phone numbers, is created by the coordinators. A retention visit consists of knocking on a participant’s door and trying to obtain a new number or scheduling an appointment if the caregiver is home. If no one is home, a postcard is left at the door with a phone number for them to reach the OHAPH staff.	Study Initiation (more extensive visits were conducted when the outreach workers were hired)	Ongoing	Provides a way to reach individuals with disconnected phone numbers who do not respond to postcards or other means of communication.	Home visits are resource intensive, requiring significant time from staff members. Time of day that retention visits are scheduled is limited to staff availability; sites where keys are required for building access must be obtained from the management offices.
Compensation, FV Application & MI environmental supports B	Participants receive a $40 gift card when baseline, 12 and 24 month visits are completed ($120 in total) All children enrolled in the study receive FV applications quarterly for 2 years (9 times in total). Individuals in the intervention group receive environmental supports to assist with achieving goals set during the MI visit.	Study Initiation	Ongoing	Provides oral health promotion to a population at high risk for ECC. Provides incentives for attending visits.	Participants have different motives for continuing in the study. These incentives will not always work as motivators for retention.
Raffle B	During each study visit attended, participants are given a raffle ticket. The raffle is drawn on a monthly basis, and the winner receives a $50 gift card.	7/2011	Ongoing	Easy to implement and requires little resources.	The raffle has a different effect on the participants depending on how it is presented by the staff. The staff sometimes get discouraged by the raffle if their participants do not win.
Post Cards B	The Coordinators identify participants who have missed several visits or have a disconnected/wrong phone number and a postcard is sent to them. Postcards tailored to the participants’ specific status in the study have been developed in both languages.	Study Initiation	Ongoing	Provides participants the means to contact staff on their own when they are difficult to reach. Returned postcards are also a good way to identify participants that may have moved.
Dental Health Advocate/participant assignments B	The participant stays with the same Dental Health Advocate (DHA) throughout the entire study. The DHA delivers the MI intervention quarterly.	Study Initiation	Ongoing	Participants form a relationship with their DHA. Therefore, DHAs sometimes have more success contacting the participants.	Staffing turnovers have caused some participants to be switched to a different DHA.
Birthday Cards C	Participant birthdays are tracked, and TSHS sends a birthday card each year.	4/2012	Ongoing	Easy to implement; requires few resources	The effect of the birthday cards is difficult to measure.
Unity Days C	Unity Days is an annual celebration at each development, sponsored by the Boston Housing Authority. The TSHS staff members set up a table with arts and crafts for the children and provide attendees with toothbrushes. During 2013 and 2014 events, in order to encourage more people to come over to the table, TSHS held a $20 raffle.	Study Initiation	Ongoing	Build visibility and trust within communities. BHA/development sponsored, organized, and implemented resulting in minimal effort and minimal burden on site. Allows staff to re-connect with study participants.	This event was helpful for recruitment but less effective for retention, as there is no guarantee that participants will be at the Unity Days. Finding sufficient staff to attend Unity Days can be difficult. Attending events is an opportunity cost; during the time spent at the Unity Day, staff could be conducting participant visits.