Maintaining Participation and Momentum in Longitudinal Research Involving High-Risk Families

Abstract

Purpose

The purpose of the current study was to identify and describe strategies available to optimize retention of a high-risk research cohort and assist in the recovery of study participants following participant dropout.

Design and Methods

The Maternal Lifestyle Study (MLS), which investigated the effects of prenatal substance exposure (cocaine or opiates) on child outcome, is a prospective longitudinal follow-up study that extended from birth through 15 years of age. Retention strategies to maximize participation and factors that might negatively impact compliance were examined over the course of five follow-up phases.

Findings

At the conclusion of the 15-year visits, MLS had successfully maintained compliance at 76%. Retention rates did not differ by exposure group.

Conclusions

Maintaining ongoing participation of enrolled study subjects is a critical element of any successful longitudinal study. Strategies that can be used to reengage and maintain participants in longitudinal research include persistence, flexibility with scheduling, home visits, long-distance trips, increased incentives, and development of a computerized tracking system. Establishing rapport with families and ensuring confidentiality contributed to overall participant retention. The use of multiple tracking techniques is essential.

Clinical Relevance

Researchers are challenged to maintain participants in longitudinal studies to ensure the integrity of their research.

Longitudinal follow-up of high-risk populations presents many challenges to researchers (Zook et al., 2010). Maintaining long-term active participation of study participants is a critical element of successful long-term studies (Aitken, Gallagher, & Madronio, 2003; Marcellus, 2004). Multiple factors, including poverty and substance use, contribute to unstable living situations and unreliable communication techniques that can complicate ongoing interaction with study participants (Bada et al., 2008; Robles, Flaherty, & Day, 1994). Concerns about data validity and the resultant success of peer-reviewed publications are factors often compromised by excessive attrition (Moser, Dracup, & Doering, 2000; Ribisl et al., 1996). The implementation of the Health Insurance Portability and Accountability Act (HIPAA) privacy rule in 2003 greatly impacted research in general, with added costs and research delays and added impediments to communication with families (Ness, 2007).

A lifestyle complicated by substance use is one of several obstacles that may interfere with the ability and willingness to continue participation in a longitudinal project (Howard, 1991). The home environment of substance-abusing parents is frequently characterized as chaotic and unstable. Living in poverty also adds numerous challenges to ongoing study participation that include multiple caretakers and residence changes, limited or unreliable transportation, loss of telephone service, and lack of adequate child care. There may also be efforts to achieve anonymity for fear of being located regarding outstanding fiscal responsibilities, immigration issues, or ongoing illegal substance use.

This article discusses the numerous approaches available to maintain optimal participation as exemplified in the Maternal Lifestyle Study (MLS). Strategies that contributed to the successful reengagement of families following variable periods of dropout as well as proactive ways to promote ongoing retention of a medically complex, high-risk sample are discussed. Overall approaches and individually tailored approaches are presented. Differential compliance of families with drug-exposed versus non-drug-exposed children is also examined.

Methods

Sample

The MLS was a longitudinal follow-up study of 1,388 children, approximately half of whom were initially identified as cocaine or opiate exposed (n = 658), who were then matched to a non-cocaine or non-opiate exposed control (n = 730) based on gestational age, race, and gender. Four university research centers (Brown University, University of Miami, University of Tennessee-Memphis, and Wayne State University) participated, and the study was approved by the institutional review board (IRB) at each site. A certificate of confidentiality from the National Institute on Drug Abuse was obtained to ensure confidentiality of information regarding each participant’s drug use. Written informed consent was obtained for the longitudinal follow-up at the time of the initial visit and at the beginning of each of the study’s five phases.

Procedures

The MLS was initially envisioned as an acute perinatal study with a 3-year developmental follow-up, but was subsequently extended to 15 years of age. During the infant’s first year, families were seen at 1, 4, 8, 10, and 12 months of age. They were then seen in the clinic setting every 6 months until the child was 6 years old and annually thereafter. There was one standardized assessment conducted entirely in the home at age 5½ years. Each visit had an established “window” during which the evaluation needed to be completed.

Staffing at each site was standard, including a principal investigator, a study coordinator, a nurse or clinical nurse specialist or nurse practitioner, a psychologist, interviewers, social workers, and psychometricians. Each site had at least one designated person responsible for tracking, maintaining contact with families, and scheduling appointments.

Child health, cognitive ability, language, attention skills, school achievement, and developmental psychopathology as well as caretaker information were assessed. Results of any standardized tests were discussed with the mother or primary caretaker at the end of each visit, and appropriate referrals were made for follow-up if necessary.

Comprehensive contact information was recorded upon recruitment and updated by the tracker at each subsequent study visit. This information included full names, nicknames, home addresses, home telephone, work telephone, cell phone numbers, social security numbers, and, in more recent years, e-mail addresses. Families were always asked for the preferred way to be contacted. Families were also given an extra $10 incentive if they moved or changed telephone numbers and contacted the MLS office with their new contact information.

Families were asked to provide the name and contact information for at least one additional person who would always know their whereabouts and who would be able to contact them. In addition, with parental consent, the child’s school information was recorded. School board IRB approvals were obtained to conduct school visits for record reviews and teacher interviews at 7, 9, and 11 years, and using parental school consents was a means of obtaining current addresses and telephone numbers. Medical information (primary care provider, specialty clinics, etc.) and any other agencies with which the family may be involved were also recorded. Confidentiality was assured, and HIPAA guidelines were followed.

Given the sensitivity of the interviews conducted, assuring confidentiality was of the utmost importance. An explanation of the methods used to keep information confidential was reviewed with the family at each clinic visit and included in the informed consent. MLS also operated with a certificate of confidentiality issued by the U.S. Department of Health and Human Services. The certificate of confidentiality applied specifically to any answers regarding the use of drugs and ensured that no one could be forced to identify them in any federal, state, or local questioning whether it was civil or criminal.

Initially, individual computer programs at each site recorded information on previous, current, and potential future contacts. At the 11-year visit, a uniform electronic tracking system, to manage and record all family contact information, contact attempts, and visit status, was developed, allowing for a comparative database that identified common reasons for attrition and shared successful techniques for reengagement. The tracking system included a visit scheduler that facilitated appointments, kept a history of visit compliance, and assisted in the planning of visit appointments within the study window. Additional patient-specific reports were available that would alert the interviewers to important upcoming special events, such as birthdays. Attendance profiles generated by the system enabled staff to identify the most difficult and least compliant families, who could then be targeted for early and more intense scheduling efforts.

Contact with families began 4 to 8 weeks before the opening date of their window, allowing time to accommodate special needs, such as weekend or evening appointments, and to pursue more extensive tracking efforts if the family could not be located using their previous contact information. As children approached school age, appointments were routinely scheduled on weekends, evenings, and during school breaks to accommodate work and school schedules.

Once the visit had been initially scheduled, postcards were then sent to families to reconfirm the date and time of the scheduled visit. Families were again contacted 3 to 7 days before their scheduled appointment to confirm the date and time and to ensure their means of transportation to the clinic. After the visit, families were sent thank you letters that included standardized testing results and any pertinent follow-up information.

Transportation to and from the clinic visits was provided through either university-leased vehicles, local taxicab services, or reimbursement for public transportation. For families providing their own transportation, parking was reimbursed. Provision of transportation services for up to 90% of families was required at some sites, without which compliance would have been significantly impacted.

Families who had moved out of their original site catchment area presented a challenge. For these families the project protocol was amended to allow the interviews and assessments to be conducted in the home. A significant number of families (more than 80) were reengaged in the study after agreeing to home-based evaluations. Off-site visits included single full-day trips of several hundred miles as well as long-distance trips exceeding a thousand miles to adjacent states where multiple visits were accomplished over several consecutive days. Traveling teams from one site often assessed children originally seen at other sites if they had moved within the area of travel. This effort significantly improved compliance for all sites. Local home visits were also conducted when traveling to the clinic may have created a hardship, as in the case of elderly or ill caretakers, or severely developmentally delayed children. Staff safety was of paramount importance in high-risk areas, with visits being conducted either in teams or with security officers.

Families were remunerated at each visit in accordance with individual site IRB-approved practices. Families could choose from cash, gift cards, and developmentally appropriate toys or books. Meals were provided to the families, along with a birthday celebration (cake and photos) at each annual visit. Over the years incentive amounts were increased. Incentives were also given for keeping the first scheduled appointment and for being on time for appointments. Raffles were held for larger gifts, such as bicycles and televisions, and families became eligible for these when they attended their clinic visit.

Families who were lost to follow-up because no current address or working telephone was available presented a significant challenge. Previous contact information was carefully reviewed, and home visits were attempted to reestablish contact. Outside agencies that may have been involved with the family were also pursued. Over the course of the study, technological advances (e.g., the use of Internet search engines, etc.) also became an integral part of locating missing families.

Even when families declined participation for a specific visit, or had missed several visits, communication was maintained when possible. Because individual circumstances and family situations change frequently over time, numerous instances occurred in which families became reengaged in the study after several years of nonparticipation. The centralized tracking system facilitated improved communication with study participants by providing a historical record of all previous tracking and follow-up efforts regardless of the outcomes. This was instrumental in improving and maintaining participant retention.

Analyses

In a high-risk, substance-exposed population, noncompliance was a significant challenge. To identify possible factors in the exposed and non-exposed groups that may negatively impact retention rates, the two groups were compared on demographic characteristics recorded at the initial visit (child’s gender, birth weight, and gestational age and mother’s race, insurance, age, education, and drug use) using chi-square tests. Similar comparisons were made for those who attended the 15-year visit versus those who were lost to follow-up. Retention rates were computed as the percentage of children who attended each visit compared with the original 1,388 sample, excluding those who had died. Logistic regression analyses were conducted to compare retention at each visit among exposed versus. nonexposed children.

Results

Table 1 shows the demographic characteristics of the original 1,388 study children and their mothers. About half of the children were male and 42% were born preterm. The majority of the sample (77%) was Black, 39% of the mothers had less than a high school education, and 83% were on Medicaid. About half of the mothers used cocaine or opiates (47%), tobacco (54%), or alcohol (59%) while pregnant, and 23% used marijuana.

Table 1.

Sample Characteristics by Cocaine/Opiate Exposure and Attendance at 15-Year Visit

	All n (%)	At 15-yr visit n (%)	Not at 15-yr visit n (%)	p	Exposed n (%)	Not exposed n (%)	p
n	1,388	1,041	347	—	658	730	—
Male	727 (52)	527 (51)	200 (58)	.024	342 (52)	385 (53)	.776
Mean birth weight in grams (SD)	2,630 (819)	2,612 (816)	2,710 (823)	.059	2,570 (760)	2,683 (865)	.010
Preterm birth (< 37 weeks)	577 (42)	434 (42)	143 (41)	.885	279 (43)	298 (41)	.533
Mother’s mean years of age (SD)	28 (6)	28 (6)	27 (6)	.008	30 (5)	26 (6)	< .001
Black	1,063 (77)	823 (79)	240 (69)	< .001	493 (75)	570 (78)	.165
Less than high school education	545 (39)	400 (39)	145 (42)	.261	318 (48)	227 (31)	< .001
Medicaid	1,131 (83)	858 (83)	273 (80)	.125	564 (87)	567 (79)	< .001
Cocaine/opiate exposure	658 (47)	484 (46)	174 (50)	.238	--	--	--
Tobacco exposure	748 (54)	559 (54)	189 (54)	.804	537 (82)	211 (29)	< .001
Alcohol exposure	825 (59)	630 (61)	195 (56)	.155	463 (70)	362 (50)	< .001
Marijuana exposure	324 (23)	238 (23)	86 (25)	.432	253 (39)	71 (10)	< .001

Participants who remained in the study at 15 years were generally similar to those who were lost to follow-up, with no significant differences on preterm birth, maternal education, Medicaid enrollment, cocaine or opiate use, tobacco use, alcohol use, and marijuana use. However, those who remained in the study were more likely to be female and Black and to have a lower birth weight.

Table 2 provides background information on additional demographic characteristics of the sample that may vary over time and that can present obstacles to study participation. In this sample, 86% of children had their biological mothers as caretakers at 1 year of age, with this number consistently declining over time to a low of 68% at 15 years of age. At each visit, a substantial number of participants (23% to 35%) had moved since the previous visit. Over the course of the study, 33% to 59% had caretakers who were employed. About half of the study families (45% to 53%) had five or more people in the household, with a similar proportion (41% to 53%) having three or more children in the household. About a quarter of the sample at each time point (22% to 28%) had mothers who had biological children who were not living with her.

Table 2.

Percentage of Children With Time-Varying Demographic Characteristics by Visit

Visit Year	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
n	1,097	1,106	1,092	962	929	971	977	1,017	1,029	1,029	1,022	1,031	1,035	1,027	1,041
Mother is caretaker (%)	86	82	81	80	78	76	74	73	73	71	70	69	69	68	68
Moved since previous visit (%)	23	34	35	32	28	27	26	32	30	26	29	32	31	31	31
Caretaker is employed (%)	33	42	47	50	53	57	58	60	56	57	56	59	58	57	52
5+ people in household (%)	50	49	50	51	51	51	50	53	53	50	50	49	48	45	45
3+ children in household (%)	50	50	51	52	53	52	53	54	53	50	51	50	48	44	41
Mother has children not living with her (%)	24	25	26	24	25	26	26	28	27	28	27	25	24	24	22

Figure 1 illustrates retention rates by exposure status from year 1 through year 15. Retention rates for the first 3 years of the study were relatively stable at 79% to 80% but fell to a low of 65% at the 4½ year visit and 68% at the 5 year visit. There was significant recapture at the 5½ year visit to 77%, which may be attributed to this being a home visit. For the remaining years, retention rates remained fairly consistent, increasing from a low of 71% in year 6 to a high of 75% to 76% in years 9, 10, and 12 through 15.

Figure 1.

Retention Rates by Year and Exposure to Cocaine and/or Opiates.

Compared to nonexposed children, exposed children generally had mothers with fewer resources and greater substance use (see Table 1). However, retention rates did not differ significantly across exposed and nonexposed children at any of the visits (p > .05; see Figure 1), suggesting that the study procedures were equally effective at retaining exposed and nonexposed children in the study, regardless of the differences between these groups in terms of demographics and family dynamics, which may place exposed children at particular risk for loss to follow-up.

Changes in staff over time were reviewed in relation to its possible impact on compliance. During the transition into an extended study, 32 staff changes occurred across the four MLS sites, with 13 (40%) being in positions felt to be key in establishing rapport with families: the tracker, the social workers, the interviewers, and the nurses. The potential impact of these changes and strategies to mitigate their impact and maintain study compliance are discussed below.

Discussion

Factors and strategies identified as contributing to successful retention in longitudinal studies have been previously reported. In a review of the successful retention of 86% of the participants for 10 years, Lyons and colleagues (2004) reported using specific, detailed tracking information and search engines, along with building relationships, compensating participants, and remaining flexible. In a review of challenges to recruitment and retention of participants to research studies, Gul and Ali (2010) stressed the importance of eliminating barriers for participants. However, for MLS, additional challenges encountered in a population of perinatal substance users and their infants added multiple layers of complexity to the maintenance of participant retention.

The MLS was originally designed as a two-phase project and mothers were asked to participate for only 3 years when they were approached prenatally for initial informed consent. Given, Keilman, Collins, and Givens (1990) found that at the beginning of a study, expectations should be clearly communicated to participants to enhance retention. MLS retention rates were relatively consistent at the four participating study sites over the first 3 years, with a mean of 77%. When the study was unexpectedly extended to include visits between 4 and 7 years of age, many families refused to extend their original commitment and others began to miss scheduled appointments. Retention rates fell significantly to their lowest point, with a mean of 65% during the first year after the extension. It is speculated that families’ original expectations that the study was time-limited made them reluctant to extend their participation.

In addition, there were significant staff changes at all sites during this transition, which impacted the rapport and trust established with the families during the initial 3-year follow-up. Many staff members at all sites, in anticipation of the original announced closure of the study, had sought new positions. Key factors for enhancing compliance, such as family comfort and confidence, especially when revealing and discussing very personal and confidential issues, might have been perceived as compromised because of the loss of familiar personnel with whom the families were comfortable and with whom they had established open communication over the first 3 years with the frequent visits and contacts.

The staff at each site was challenged to recapture these lost families and to convince disinterested families to reengage in order to return compliance to more acceptable rates. Sensitivity to family needs was paramount, and extensive and creative attempts to address intellectual, emotional, physical, and social impediments to ongoing participation were used. In addition, maintaining current information on families who had moved out of the original catchment areas was important to facilitate either local or long-distance home assessments.

Although it is difficult to measure the impact that establishing a congenial and trusting rapport with the family has on maintaining interest and involvement, it is clear that this is one of the major factors that optimizes retention. Having the families see the program staff as their trusted advocates and a resource was a significant factor in retaining and recapturing participants. In a study of low-income urban parents enrolled in a parent training study, Gross, Julion, and Fogg (2001) report that more than 90% of the parents cited the personality and trustworthiness of the recruiter as an important incentive for their participation. Ely and Coleman (2007) stated that creating a partnership with parents was the key to successful retention of their study subjects. McGregor, Parker, LeBlanc, and King (2010) stated that successful longitudinal studies depend on developing trusting relationships between the researchers and the participants.

The MLS staff had been a constant in the lives of both the children and their primary caretakers for more than a decade. This relationship was a source of permanency and support given the inconsistencies, negativity, and chaos that often characterize the life circumstances of many of these families. Families were always treated with respect, and they recognized the program as a source of positive support in their lives. Often in difficult economic times or in emergency medical situations, families saw the program as their advocate to call for assistance. Although needed services were facilitated and were not provided, families still identified program staff as caring advocates with whom they could talk, problem solve, and receive direction. Families always appreciated the program staff being there for them in times of unanticipated crisis or need. The caring and trusting rapport established with most families, even those who had long absences from participation, was clearly a factor that positively impacted retention and commitment. It was evident that the dedication and the genuine affection that study staff exhibited for children as they grew from infancy into adolescence and the respect and admiration they felt for the families they watched struggle to overcome adversities, and who shared occasions of pride and happiness, were effectively conveyed to the families.

Persistence was an essential quality needed for success. The majority of study participants who were lost at times from the study did not openly express their desire to permanently withdraw. Being too busy was the primary reason that families gave for not committing to a scheduled visit. However, not responding to telephone calls, appointment letters, messages, etc. was the most usual passive response. Many families would repeatedly agree to a visit when contacted, but would not show up for scheduled visits. Although at times avoidant, several parents eventually commented, “Thank you for not giving up on me.”

Incentives have always played a significant role in encouraging families to keep their appointments. Although families find money an attractive incentive, smaller incentives were also important because they became part of the ritual of the visit and provided a more personal touch. Certificates that acknowledge the completion of each of their assessments were always important to the family and to the child, and were reported by caretakers as being permanently displayed in the home. Birthday parties and birthday cakes were always eagerly anticipated. The cakes and the entire staff singing “Happy Birthday” to the child each year contributed to a celebratory atmosphere, reflecting that staff members were genuinely happy to see the child and the family. The anticipation of the party and birthday cake continued to serve as an important reminder that around the time of the child’s birthday, their annual assessment was due. For some families, the MLS birthday cake and party was the only celebration that the child would receive.

It was important for families to understand in a meaningful way why they were participating in the study and what they were contributing. In their review of factors influencing retention, Davis, Broome and Cox (2002) found that participants are more likely to remain in a study if they understand its importance. Although the MLS study goals were explained at the time of initial enrollment and again whenever informed consent was renewed, it was clear that parents were often not able to explain or discuss the significance of the study to their friends, relatives, teachers, and others. At the 10-year visit, a booklet was prepared for the children themselves that explained the study in age-appropriate language. Parents were also able to use these booklets to better understand the study in everyday terms. It explained how families were selected to be in the study, that it was a multisite study, and discussed why the study had been extended beyond the original 36-month planned time frame. The booklet did not identify the study as a “drug exposure” study, but it did describe the diversity of the study population and it conveyed the importance that each family contributed to the study. It was an excellent tool to help the children specifically understand their own contribution and it emphasized how their participation was valued and appreciated. It also reinforced the family’s resolve that they were not only potentially improving their own child’s outcome, but that they were contributing significantly to a much larger research effort that had the potential to improve the future and optimize the outcome of many other children in similar circumstances.

It was important to recognize that as the children aged, they appreciated and enjoyed their involvement in the long-term follow-up process. As their birthdays approached, many children asked when their next study visit would occur. And as they became adolescents, some took the initiative for scheduling their own appointments.

Conclusions

Our experiences over the 15 years of the MLS have shown that maximizing compliance and retention requires extensive time and effort. Innovative strategies and the dedication, flexibility, and ingenuity of project staff results in family satisfaction and appreciation, and helps to ensure study success.

Clinical Resources.

• National Institutes of Health National Institute on Aging. This valuable website for all researchers provides guidance not only on recruitment and retention, but also on all aspects of conducting a research study, http://www.nia.nih.gov/research/dgcg/clinical-research-study-investigators-toolbox/startup