Reflections of a Bean Counter: The Costs of Recruiting

Abstract

Although there is published information regarding the costs of participant recruitment for large-scale clinical trials, these findings may not be applicable for small feasibility studies. To determine the most effective recruitment strategy we adapted a metric developed by Chin Feman et al. (Chin Feman et al., 2008). We compared costs and enrollment rates associated with various recruitment strategies. We recruited persons with mild to moderate dementia for a feasibility study that sought to adapt laboratory measures of daytime function to the home. We recruited from 1) enrollees in a Memory Research Center, 2) Alzheimer’s disease support groups, 3) the Reynolds Senior Health Center Clinic at the University of Arkansas for Medical Sciences, and 4) senior citizens housing. We determined that recruitment through the Memory Research Center was most effective (enrollment rate 67%, salary costs $49.47 per participant). These findings have implications for investigators preparing budgets for studies involving populations with dementia.

Introduction

When developing research, the ability to design and carry out an effective plan for participant recruitment is essential to study implementation and data collection. Robust findings depend upon recruiting an adequately sized sample that is appropriate to the study. Without an adequate sample size, the power to detect meaningful results is compromised (Tarlow & Mahoney, 2000). In addition, when research involves persons with Alzheimer’s disease (AD), using limited resources (the valuable time of research participants as well as grant monies) without assurance that recruitment problems will not negatively affect the project presents both moral and ethical problems. Participant recruitment, the first step in data collection, is not a passive process. Some investigators use the phrase data generation to clearly describe the hard work of this purposeful and important activity (Ben Hannigan, 2006).

Effective participant recruitment for research with the growing population of individuals with Alzheimer’s disease (AD) is particularly important. Recruitment for research involving this vulnerable population is essential because this is a growing population. By the midpoint of the 21^st century, the annual number of new AD cases will more than double to 959,000 in 2050 (Hebert, Beckett, Scherr, & Evans, 2001). Approximately 5 million are now afflicted or burdened with care-giving for persons with AD (Anderson H.S. & Kuljis, 2004).

Effective participant recruitment for research with persons with AD can be challenging for several reasons. First, obtaining informed consent from persons with AD can be a complex process. Persons with AD cannot automatically be excluded from the informed consent process based on diagnosis however even in minimal risk studies; the person consenting to treatment must be competent to do so. (Cole, 2005) Therefore, their ability to participate in the informed consent process must first be determined and documented. If the person with AD is unable to participate in the informed consent process, the legally authorized representative provides informed consent. Also, the research design for studies involving persons with AD often involves the caregiver. Therefore, successful recruitment requires that both members of the dyad are willing to participate and meet all inclusion criteria. Finally, effective participant recruitment in studies involving this vulnerable population must decrease the respondent burden and the stress of study participation (Tarlow & Mahoney, 2000).

Although effective recruitment strategies using variable metrics have been reported for some clinical trials (Chin Feman et al., 2008);(Kelly, Ahmed, Martinez, & Peralez-Dieckmann, 2007); (Knobf et al., 2007); (Robinson et al., 2007); (Tarlow & Mahoney, 2000), these findings may not be applicable for small feasibility studies. Without adequate information, beginning investigators may be unprepared to deal with recruitment problems. To determine the most effective recruitment strategy for our feasibility study we adapted a metric developed by Chin Feman et al. (Chin Feman et al., 2008). Our research questions were:

1. What is the recruitment yield (proportion of participants contacted to those enrolled)?

2. What is the recruitment completion rate (proportion of potential participant contacts later enrolled in the study)?

3. What are the total costs associated with recruiting AD/family caregiver dyads.

4. What is the most cost effective strategy?

Method

Target Population

The purpose of the original study was to explore the feasibility of obtaining repeated measures of neurobehavioral performance in persons with AD and determine if we could tailor a measurement model used with healthy adults to the home. The original study was funded by The Tailored Biobehavioral Interventions Research Center at UAMS, an exploratory center funded by the National Institute of Nursing Research (P20 NR009006-01).

Study overview

Dr. David F. Dinges and his group at The University of Pennsylvania have experimentally studied neurobehavioral performance related to sleep in healthy adults using repeated measures of attention (simple reaction times with the Psychomotor Vigilance Task [PVT]) (Dinges & Powell, 1985;, Doran, Van Dongen, Dinges DF., 2001; Drummond et al., 2005;, Van Dongen & Dinges, 2003). They also used behavioral monitoring in a laboratory setting to ensure optimal PVT performance. We proposed that unfortunately, in persons with AD, the standardized conditions of the laboratory may actually preclude optimal PVT performance. For example, unfamiliar surroundings may increase stress of study participation, anxiety, and would adversely affect test performance. Therefore, the specific aim for the original feasibility study was to determine the feasibility of using repeated PVT measures with persons with AD in the home setting.

The questions were:

1. Can laboratory controls (activity, diet, light, posture, sound, temperature, and time) be replicated in the home?

2. What is the correlation between PVT trials done in the laboratory and in the home?

3. Where do persons with AD perform PVT trials optimally (most minutes, fastest reaction times, fewest total errors, least percent change over the ten minute trial)?

4. What are the characteristics of participants who complete all PVT trials; a) In the home, b) In the laboratory?

5. What factors disrupt PVT data collection in the home and in the laboratory setting?

6. What are the perceptions and preferences of persons with AD and their caregivers regarding home and laboratory PVT trials?

The original feasibility study involved two steps. In step one 1 the person with AD completed PVT trials every two hours for two days, one day in the General Clinical Research Center (GCRC) at UAMS and one day in the home setting. In Phase 2 the person with AD and their caregiver participated in a phone survey to determine ways to further tailor this measurement method to the unique characteristics of persons with AD.

Inclusion and Exclusion Criteria

We selected participants based on the following criteria:

• (a) Enrolled in the UAMS Memory Research Center (MRC) database;

• (b) A consensus diagnosis of AD (Clinical Dementia Rating [CDR] ≥ 1).

• (c) Either on a consistent dose of a cholinesterase inhibitor or NMDA receptor antagonist for at least 7 days prior to the study or not receiving these drugs;

• (i) Availability of caregiver to stay with the participant in the GCRC and home and assist with behavioral and dietary monitoring;

• (j) Ambulatory and able to complete activities of daily living;

• (k) Capable of performing PVT trials as evidenced by:

  1. The ability to sit upright in a chair and manipulate the push button on the PVT,

  2. Report normal corrected vision,

  3. Ability to hear, understands, and responds verbally to instructions. (Mini Mental State Exam of 9 or higher.

Exclusion criteria were self-report or caregiver report of acute medical or psychological illness and receiving anxiolytics or hypnotics or presence of depression. Depression was measured with a score of 11 or more on the Geriatric Depression Scale (GDS) (Yesavage et al., 1982).

Primary Recruitment Plan

We first recruited persons with a consensus diagnosis of AD through The UAMS Memory Research Center (MRC) patient registry. The UAMS MRC made initial telephone contact to determine the participant’s level of interest in our project. If the enrollee agreed to meet with the principal investigator to further discuss the research project a meeting at a time and place convenient to the participant was determined. During that meeting, we explained the study, obtained consents and we collected baseline data. Our target for enrollment was 20 dyads (person with AD and caregiver), 40 persons total.

Expanded Recruitment Plan

In the event an adequate sample was not obtained from the UAMS MRC we also proposed to expand recruitment efforts to the Donald W. Reynolds Institute on Aging Clinic on the campus of UAMS, participants in Alzheimer’s Arkansas support groups, and senior citizens housing sites. Initially, we made contact via telephone with the chief administrator (pastor, executive director, etc.) of the indicated group. The purpose of the first contact was to obtain agreement to proceed. We explained the purpose of the study and answered any questions and concerns. We requested that the administrator designate a primary contact and resource to determine the best process to identify potential participants. We anticipated that these processes would vary depending upon the setting. In the event we expanded recruitment beyond the UAMS Memory Research Center, we also proposed to modify one portion of the study’s inclusion criteria: UAMS Memory Research Center Enrollee or Physician Diagnosis of Alzheimer’s disease documented in the medical record.

Results

The final sample consisted of 14 dyads. Eleven dyad participants were Caucasian and three were African-American. Of the persons with AD, 5 were female (35.7%) and 9 were male (64.3%). Their average age was 79.1 (SD 5.64), with range from 65 to 88. Of the persons with AD, 3 had advanced degrees, 4 had completed college, 3 had partial college, 2 completed high school, and 2 had less than a 10^th grade education. The average MMSE score was 22.06 (SD±4.52). The average GDS score was 6.58 (SD 4.31).

Twenty-three dyads signed the informed consent document and 19 completed baseline data collection. Fifteen dyads met the study criteria. Three potential participants scored higher than 11 on the GDS and one participant withdrew consent during baseline data collection leaving a final sample of 14 dyads with complete data (Table 1).

Table 1.

Home Visits, Enrollees, Completers and Cost by Recruitment Site.

Site	Home Visit	Enrolled	Enrollment Rate	Completer	Cost per Completer
Memory Research Center	15	10	67%	9	$49.47
Reynolds Senior Clinic	23	9	39.1%	5	$450.15
Alzheimer’s Support Groups	2	0	0	0	--
Seniors Housing	2	0	0	0	--
Total	42	19	45.2%	14	$196.84

1. What is the recruitment yield (proportion of participants contacted to those enrolled)? We approached 305 persons regarding this study (including all contacts however brief) and 42 expressed an interest in the study. Of that number 23 signed the informed consent for an overall recruitment yield of 45.2%.

2. What is the recruitment completion rate (proportion of potential participant contacts later enrolled in the study)? Of the 42 potential participant contacts, 14 completed the study, for a recruitment completion rate of 14/42 or 33.3%

3. What are the total costs associated with recruiting AD/family caregiver dyads. Based on salary costs of our research assistant and estimated hours spent in all recruitment efforts we determined that the cost per participant was $196.81.

4. What is the most cost effective strategy? The Memory Research Center was most effective (enrollment rate 67%, salary costs $49.47 per participant).

Conclusions

We found that recruitment from Memory Research Center enrollees was the most efficient and cost effective method. Unfortunately, a sample that consists entirely of Memory Research Center enrollees may not be representative of all persons with AD and this may therefore limit the generalizability of findings. Further, changes in the recruitment environments that were unpredictable affected achievement of our recruitment goals. That is, because of changes in the funding status of our Memory Research Center, recruitment of new enrollees ended, limiting our ability to recruit a complete sample from Memory Research Center enrollees, we learned that it is essential to have a flexible recruitment strategy with well thought out alternative plans. The recruitment plan should include many paths to identify potential participants and these multiple paths should be implemented simultaneously and early in the study.

Additionally, the manpower requirements and cost of recruitment must be realistically considered when developing a proposed budget. Although this report provides a rough estimate of the direct costs of participant recruitment based on salary and fringe benefit costs of study personnel, we did not include the cost of participant incentives, mileage, or indirect costs (e.g., overhead, community resources, and non-project personnel time). Clearly, these additional costs should also be considered to gain a clear picture of the costs of recruitment. We believe that, in future projects, it will be relatively easy to record and track additional costs such as participant incentives, mileage, community resources and non-project personnel time. We have yet to determine the significance of these costs. However, indirect costs such as overhead may be more difficult to ascertain and, fortunately, most funded research projects account for overhead by applying indirect costs to the base budget. Overhead expenses are usually thought of as a percentage of the direct labor cost and include indirect costs such as accounting, advertising, depreciation, indirect labor, insurance, interest, legal fees, rent, repairs, supplies, taxes, telephone, and utilities.

Finally, it is essential to actively manage the recruitment process. We applied these principles of continuous quality improvement to participant recruitment (Deming, 1986): the process of participant recruitment can and should be made better, efforts to improve participant recruitment should be continuous; the participant recruitment process itself can yield data and information on how well the process works; data and information are essential to improving the recruitment process. To operationalize these principles the first author and research assistants established weekly team meetings, including reports from each team member on all recruitment activities. We established daily and weekly goals for each recruitment activity (e.g., phone calls, clinic visits, home visits, etc.) and evaluated whether goals were met. In this way we were able to celebrate successes, analyze the root cause of failures and hold each team member accountable for performance. We learned to focus on short-term recruitment goals as well as long-term goals and to take action as soon as short-term goals are not being met. Finally, we recorded the minutes of these meetings to provide data for future process improvements.

Our findings have implications for investigators recruiting vulnerable populations, such as persons with Alzheimer’s disease. Recruiting an adequate sample from this vulnerable population can be challenging for many reasons, some of which we cannot control. Because this group is already dealing with the day to day challenges of Alzheimer’s disease, adapting investigative methods in ways to minimize respondent burden can positively affect participant recruitment. When potential participants expressed an interest in our work, we found that they were extremely committed to complete the study. Similarly, we must maintain our commitment to study and ultimately improve care for this vulnerable population.

Figure 1.

Recruitment Yield