Participant Retention in a Longitudinal National Telephone Survey of African American Men and Women

Abstract

Objective

The purpose of this article is to describe participant demographic factors related to retention, and to report on retention strategies in a national study of African Americans re-contacted 2.5 years after an initial baseline telephone interview.

Design & Setting

The Religion and Health in African Americans (RHIAA) study was originally developed as a cross-sectional telephone survey to examine relationships between religious involvement and health-related factors in a national sample of African Americans. The cohort was re-contacted on average of 2.5 years later for a follow-up interview.

Participants

RHIAA participants were 2,803 African American men (1,202) and women (1,601).

Interventions

RHIAA used retention strategies consistent with recommendations from Hunt and White.¹ Participants also received a lay summary of project findings.

Main outcome measures

Retention at the follow-up interview.

Results

Retention rates ranged from 39%–41%. Retained participants tended to be older and female. In age- and sex-adjusted analyses, retained participants were more educated, single, and in better health status than those not retained. There was no difference in religious involvement in adjusted analyses.

Conclusions

Although overall retention rates are lower than comparable longitudinal studies, RHIAA was not originally designed as a longitudinal study and so lacked a number of structures associated with long-term studies. However, this project illustrates the feasibility of conducting lengthy cold call telephone interviews with an African American population and helps to identify some participant factors related to retention and study strategies that may aid in retention.

Introduction

Survey research with historically underrepresented populations can involve challenges around recruitment and retention. Factors such as emotional stress and suspicion of research institutions occur due to a legacy of discrimination. Historical events such as the US Public Health Service Syphilis Study at Tuskegee forged mistrust concerning the protection of human subjects in medical research, particularly in African American communities.^1,2

Despite the difficulties, research with African Americans contributes to our understanding of factors that enhance or diminish health in this population. Researchers have documented race-related health disparities in most chronic diseases.³ Underrepresentation of racial/ethnic groups in clinical trials can reduce the generalizability of data.⁴ Consequently, Healthy People 2010 called for an initiative to improve recruitment and retention strategies increasing minority participation in health research.⁵

Longitudinal Studies and Retention

Longitudinal studies provide the opportunity to examine change over time and enhance confidence in testing causal models. However, over time, successfully recruited individuals may decide not to comply with research protocols or choose not to take part in follow-up data collection.⁶ This issue may be particularly amplified in research with underrepresented or medically underserved, disparities-impacted, or highly mobile populations. Compared to other groups, ethnic minorities are more likely to opt out of such studies.⁷ This may help explain the scarcity of longitudinal studies that focus specifically on populations such as African Americans. Though some large-scale longitudinal efforts have included a significant proportion of African Americans,^8–11 race-specific retention rates are more difficult to identify.

Background: Religion and Health in African Americans

The Religion and Health in African Americans (RHIAA) study is a national longitudinal cohort study focusing on African Americans.¹² The purpose of the RHIAA study is to help explain complex relationships between religious involvement and health-related factors (eg, health behaviors, physical/emotional functioning). In the RHIAA study, African American men and women completed an initial telephone interview. While there were no original plans for a subsequent study, when support later became available for a follow-up interview, the RHIAA cohort was re-contacted and, thus, the study became longitudinal in nature.

The purpose of this article is to describe participant demographic factors related to retention and to articulate retention strategies used in the study. This article makes a unique contribution to the literature by focusing on a national longitudinal sample of African Americans recruited for a single contact study. Little is known about re-recruitment and retention efforts with individuals who participated in a cross-sectional study and who were re-contacted for subsequent research to document longitudinal trends in the original sample.¹³ Findings may have implications for others engaged in longitudinal research with medically underserved populations.

Methods

Telephone Survey Methods

The RHIAA study is a national telephone survey of African American households and is based on a probability sample. The RHIAA study contains two sub-samples: 1) the RHIAA-I sub-sample, which consisted of 2,000 participants who completed a 45-minute interview; and 2) the RHIAA-II sub-sample, which was from a companion study to RHIAA-I, using the same methodology. In RHIAA-II, 803 participants completed a briefer, 30-minute interview with many of the same study measures as RHIAA-I participants. Both sub-samples were recruited through OpinionAmerica, an external data collection subcontractor. RHIAA-I participants completed measures focusing on self-esteem, self-efficacy, affect, social support, religious involvement,¹⁴ and health-related behaviors (dietary, physical activity, smoking, alcohol use, cancer screening), while the RHIAA-II participants completed measures of personality constructs, affect, social support, religious involvement, and physical and emotional functioning.

The RHIAA data collection methods have been reported elsewhere.¹² Using probability-based methods, a professional sampling firm generated a list of households from publicly available data such as motor vehicle records in all 50 states in the nation. Trained interviewers dialed telephone numbers from this call list, asking to speak to an adult who lived at the household. The interviewers introduced the project and, if the contact expressed interest, administered a brief eligibility screener to determine whether they self-identified as African American and aged ≥21 years. Eligible individuals were screened for cancer history because cancer diagnosis was an exclusion criterion. Eligible contacts provided verbal assent after hearing an informed consent script. Upon completion of the interview, participants received a $25 gift card by mail.

Retention Methods

General Approach

Using a community-engaged perspective, our research was characterized by a sense of appreciation for RHIAA participants and recognition of their value as stakeholders in the outcome of the research. We operated under the assumption that the participants’ input is vital to our development of the science, and that the participants’ stake in study outcomes is as much personal and practical as it is academic. We treated participants with dignity and respect and believe that made a difference. We operationalized our dignity stance through the careful crafting and thoughtful implementation of all study materials and protocols, which were intended to communicate a positive regard and sensitivity to participants. Successful retention was defined as a fully completed follow-up interview.

Specific Retention Techniques

We used recommended retention techniques consistent with Hunt and White⁶ but also tailored retention efforts to the study population. These included bonding activities such as: developing a study logo and theme; sending study updates of findings; having regular contact with participants; using a tracking system; having professional and well-trained interviewers and staff; use of scheduling flexibility including evening and weekend calls; and use of study incentives including tokens of appreciation with the study logo. However, a significant difference in which the current study varied from the recommendations was in the enrollment, consent, and baseline activities involving careful screening of potential participants for their willingness to participate in a longitudinal study with a long-term commitment; and fully informing participants about the long-term nature of the study.

After completion of the baseline interview, the project team mailed all participants a lay summary of project findings. This report was four pages in length and printed in color. In preparation for the follow-up, the project team mailed an advance letter to those (88.51%) who had indicated at baseline that they would be willing to participate in another interview. The letter notified these participants that they would receive a call to complete the interview. This was the first indication that participants received that there would be data collection subsequent to their baseline interview. These advance letters were mailed in batches corresponding with interview call dates. Participants were thanked for their previous participation and told that they would be called in the weeks ahead for a 30- (RHIAA-II) or 45-minute (RHIAA-I) telephone interview on health and wellness in the African American community. The letters contained a toll free number that participants could use to schedule an interview time or conduct an on-the-spot interview if they chose to do so.

After 90 days of calling and when most participants had been attempted up to 10 times, nonrespondents who indicated at baseline that they would agree to do another interview were mailed a cloth grocery/shopping bag with the study name and University logo on it, another copy of the study lay report as well as a letter inviting their participation. Finally, an attempt was made to reach those participants who indicated at baseline that they would not be interested in completing a subsequent interview (10.88% of baseline completes). Study staff mailed these participants a targeted “conversion” letter that asked them to contact the call center to set up an interview appointment. Study staff did not call these individuals directly. This mailing also included the grocery/ shopping bag and study lay report.

We used the Peoplefinders.com service to locate correct/current addresses for participants who had relocated or changed telephone numbers (n = 38 addresses found and letters re-mailed). This service also was used to verify participant addresses prior to mailing study letters. When returned mail was received and an address could not be verified, the information was logged into an Excel file as undeliverable (n = 232). On occasion (eg, roughly once a month), participants called the principal investigator directly to update their contact information. Additionally, study staff followed up individually with any participant who reported a lost or stolen gift card (19 participants).

Statistics

All analyses were conducted using SAS (Version 9.3). The baseline demographic and socioeconomic characteristics of participants who were retained were first compared with those of participants not retained using unadjusted bivariate analysis. For continuous variables, ordinary linear regressions were used to compare these two groups. Logistic regressions or multinomial logistic regressions were used for comparison on categorical variables. Adjusted analyses that controlled for age and gender were then conducted. Significance was set at P<.05 level.

Results

Retention Findings

The overall retention rates for RHIAA-I and RHIAA-II sub-samples were 41%, and 39%, respectively (Table 1). Among the 2,481 participants who at baseline indicated that they would be willing to participate in another interview, 1,047 (42.2%) were retained. Among the 305 participants who indicated that they would not be willing to participate in another interview, 73 (23.9%) were retained. Another 17 participants refused to answer this question. Of these participants, 4 (23.5%) were retained. For those not retained, the mean number of call attempts was 13.53. Of the overall baseline sample (N = 2,803), 25% were not retained due to a non-working phone number. Of the overall baseline sample (N = 2,803), 31% were not retained due to not being able to be contacted (eg, deceased 2%; no answer or no such person at the number 29%). Very few participants refused to do the interview (2.5%).

Table 1.

Baseline to follow-up retention rates

	Baseline n Complete	Follow-up Completes, n (%)
RHIAA-I	2000	810 (40.50)
RHIAA-II	803	314 (39.10)

Factors Associated with Retention

Compared to participants who were not retained, participants were older (P<.0001), female (P = .0001), with college or higher education (P<.05), and were more engaged in religious behaviors (eg, attendance), (P<.05) (Table 2). Retained participants were also more likely to be widowed than currently married (P<.05), and retired (P<.001) or working part-time (P<.05) than working full-time than those not retained. There were no significant differences in religious beliefs, self-reported health status, or annual household income.

Table 2.

Comparison of demographics and religious involvement between retained and not retained participants (RHIAA-I and RHIAA-II combined)^a

	Baseline N Complete (N=2803)	Follow-up Completes (n=1124)	Not Retained (n=1679)	Unadjusted Odds Ratio (95% CI)	P	Adjusted Odds Ratio (95% CI)b	P
Age, years, mean (SD)	54.86 (14.64)	58.06 (13.18)	52.72 (15.18)	-	<.0001	-	-
Religious beliefs, mean (SD) [out of 20]	17.69 (2.85)	17.74 (2.92)	17.66 (2.81)	-	.4426	-	.9249
Religious behaviors,mean (SD) [out of 21]	16.61 (3.11)	16.77 (3.03)	16.49 (3.17)	-	.0297	-	.5537
Sex
Female	1601 (57.1)	692 (61.6)	909 (54.1)	1.357 (1.163, 1.583)	.0001	-	-
Male	1202 (42.9)	432 (38.4)	770 (45.9)	-
Education
≥College	1505 (53.7)	635 (56.5)	870 (51.8)	1.208 (1.037, 1.406)	.0150	1.284 (1.099, 1.500)	.0016
≤High school	1298 (46.3)	489 (43.5)	809 (48.2)	-		-	-
Marital status
Never been married	374 (13.4)	137 (12.2)	237 (14.2)	.851 (.667, 1.084)	.1916	1.172 (.907, 1.515)	.2253
Currently single	450 (16.1)	180 (16.1)	270 (16.1)	.981 (.784, 1.228)	.8664	1.331 (1.049, 1.688)	.0186
Separated or divorced	501 (17.9)	185 (16.5)	316 (18.9)	.861 (.692, 1.072)	.1806	.792 (.633, .990)	.0405
Widowed	390 (14.0)	182 (16.2)	208 (12.4)	1.287 (1.020, 1.625)	.0336	.885 (.681, 1.149)	.3588
Currently married or living with partner	1080 (38.6)	437 (39.0)	643 (38.4)	-		-	-
Employment
Part-time employed	329 (11.8)	143 (12.8)	186 (11.1)	1.476 (1.144, 1.904)	.0027	1.327 (1.022, 1.723)	.0337
Not currently employed	371 (13.3)	143 (12.8)	228 (13.7)	1.204 (.941, 1.542)	.1402	1.222 (.948, 1.574)	.1220
Retired	783 (28.1)	374 (33.4)	409 (24.5)	1.756 (1.449, 2.128)	<.0001	1.001 (.770, 1.301)	.9942
Receiving disability	318 (11.4)	121 (10.8)	197 (11.8)	1.179 (.908, 1.532)	.2168	.885 (.673, 1.162)	.3788
Full-time employed	987 (35.4)	338 (30.2)	649 (38.9)	-		-	-
Health status
Poor	166 (5.93)	58 (5.2)	108 (6.4)	.819 (.558, 1.203)	.3085	.613 (.413, .910)	.0152
Fair	620 (22.1)	249 (22.1)	371 (22.1)	1.024 (.783, 1.338)	.8641	.836 (.634, 1.102)	.2035
Good	955 (34.1)	390 (34.7)	565 (33.7)	1.053 (.820, 1.352)	.6866	.874 (.675, 1.131)	.3051
Very good	709 (25.3)	288 (25.6)	421 (25.1)	1.043 (.804, 1.355)	.7501	.946 (.723, 1.237)	.6827
Excellent	351 (12.5)	139 (12.4)	212 (12.6)	-		-	-
Income
≤$30,000	1595 (56.9)	647 (57.6)	948 (56.5)	1.046 (.898, 1.218)	.5646	.922 (.787, 1.079)	.3113
>$30,000	1208 (43.1)	477 (42.4)	731 (43.5)	-		-	-

^aData are n (%) unless specified otherwise.

^bAdjusted for age and sex.

Adjusted Analyses

Because a number of the above findings may be related to age and sex, we conducted another set of analyses that controlled for these factors. In these analyses, education remained significant (P<.01), however, religious behaviors were no longer significant (Table 2). Marital and work status remained significant; however, the categories in which the retained vs not-retained shifted. Retained participants were more likely to be single (P<.05) and less likely to be separated/divorced (P<.05) than to be married or living with a partner. Retained participants were more likely to be employed part-time than to be working full-time (P<.05). When controlling for age and sex, self-reported health status became significant, with participants in poor health condition less likely to be retained (P<.05) than those reporting excellent health.

Discussion

Our study provided an opportunity to identify factors associated with retention in a longitudinal study of African American men and women. This is a unique contribution given the population of focus and that the study was not originally designed for participant re-contact.

Retention techniques followed those previously recommended in a review by Hunt and White.⁶ Because RHIAA did not start as a longitudinal study, not all procedures were possible, and likely led to our lower retention rates. In comparison, other studies that originated with longitudinal designs had better retention rates.^8,15–18

Interestingly, the length of the interview, 30 vs 45 minutes, did not appear to be associated with different retention rates. However, a number of demographic factors were associated with loss to follow-up. Consistent with previous research,¹⁹ women were more likely to complete the follow-up interview than men. It is possible that women’s greater social vulnerability as compared to men was offset by the lack of telephones among males. Similarly, older individuals were more likely to be retained than younger people, which also is consistent with previous research.²⁰ Perhaps older people were more likely to be reached at their home by telephone, had more time, or due to social isolation were more inclined to engage with the interviewer for a lengthy telephone interview. These patterns have the potential to introduce bias into the data from underrepresentation of men and younger people.

School and Work

The adjusted analyses indicated that individuals with a college education were more likely to complete the interview than those with high school or less. Perhaps those with more education were more open to research or had had previously been exposed to research through their education or work experiences. Another possibility is that those who were less educated had less time to engage in the interview, perhaps due to increased responsibilities such as multiple jobs. People who worked part-time were more likely to be retained than the reference group of those working full-time, which may be related to having more available time to do the interview. The modest incentive for participation may have been more attractive to individuals working part-time as well, particularly in the context of an economic downturn. However, no significant differences in income were found for participants who were retained vs not retained. These patterns could introduce bias into the data that may manifest in terms of socioeconomic factors associated with higher education or income such as behavioral patterns (eg, smoking, access to health care).

Family Structure

Individuals who were retained were more likely to be single and less likely to be separated or divorced relative to married participants. Separated or divorced participants may have had less availability to complete the interview due to competing demands including parenting. However, without additional follow-up such as qualitative methods to verify why particular groups of people were lost to follow-up, it is difficult to speculate on the role of marital status factors in retention rates. These patterns are likely to introduce bias into the data that may manifest in terms of health-related factors associated with family structure such as dietary patterns (eg, food preparation, eating out, children in the home).

Preaching to the Choir

The team gave serious consideration to whether a study on “religion and health” would draw a cohort of participants characterized by a disproportionate number of religiously active and engaged individuals. In an effort to recruit a broader sample, interviewers were trained and provided recruitment scripts that did not emphasize religion per se, but rather focused on general wellness and health. Indeed, it does not appear from the adjusted analysis that there were retention differences related to religious beliefs or behaviors. This is a positive factor for the overall RHIAA study in terms of not retaining a biased, overly religious sample. Finally, consistent with previous research,¹⁵ individuals reporting poor health were less likely to be retained at follow-up than those reporting excellent health. This may be a function of decline in health from baseline to follow-up, or those in poor health being less able to complete the interview due to competing demands or stressors.

Comparison with Previous Research

Retention rates overall for this project were roughly 40%, which is considerably lower than Allman and colleagues’ University of Alabama-Birmingham (UAB) 2011 Study of Aging, which achieved a 71% retention rate among older African American participants in Alabama. In that study, 355 participants completed a phone interview at the 48-month mark. At year four, of a baseline of 500 African American individuals, 114 were deceased, 3 had withdrawn, and 28 could no longer be contacted.¹⁵ Although our study achieved lower retention rates, there are three notable differences. First, the UAB Study of Aging was designed as a longitudinal study from the outset; therefore, participants had the expectation of re-contact. Second, the study comprised a regional sample who may have had familiarity with the institution and, therefore, may have been more likely to be retained due to a local relationship. Third, the sample consisted of older adults, who, similar to the RHIAA sample, may have had more available time for participation and therefore were more likely to be retained, or may be less mobile due to work- or family-related relocations. In addition, the UAB Study of Aging included follow-up phone interviews at 6-month intervals throughout the four-year time period in an effort to increase retention.¹⁵

Limitations and Conclusions

Our findings are limited by a number of factors. Primarily, retention rates are affected by the fact that at baseline there were no plans for a longitudinal study. Had these plans been in place, additional measures could have been taken to maximize retention. It is also recommended to collect extensive collateral information for use in participant tracing (eg, social security numbers, family contact information). This was not deemed feasible in our study population of African American men and women, who have a history of mistrust of research.²¹ Further, collecting information such as social security numbers may not be advisable in the current climate of identity theft, security breaches and ever-increasing cyber security concerns. Our study did not track whether participants completed the interviews on cellular phones or land lines, limiting our ability to draw conclusions related to cellular phone use. Finally, though the population of focus may be viewed as a unique factor and strength, the focus on African American men and women does limit external validity of the current findings.

We believe, however, that the current findings have implications for those doing longitudinal research in underrepresented populations. The current findings illustrate that it is possible to conduct a lengthy cold calling initial telephone interview with African American men and women. Those most likely to be retained over the study period were women, older, educated, single, and in excellent health. Strategies that help retain participants include use of advance letters, skilled/ trained/professional interviewers, provision of an incentive, report-back of study findings, conversion attempts, participant tracing, having a live person available by telephone, and individual participant follow-up as necessary.⁶ We treated each RHIAA participant with dignity and believe that made a difference.