Challenges in Recruitment to a Randomized Controlled Study of Cardiovascular Disease Reduction in Sleep Apnea: An Analysis of Alternative Strategies

Abstract

Study Objectives:

A challenge in conducting randomized controlled trials of sleep apnea is the timely recruitment of participants to active and control arms. This study assesses the costs and efficiencies of alternative recruitment methods.

Design:

Analysis of recruitment data from the Best Apnea Intervention in Research planning study.

Setting:

Sleep clinics and cardiology practices.

Participants:

One hundred forty-eight individuals with an apnea-hypopnea index ≥ 15 and cardiovascular (CV) risk factors randomized from a pool of more than 30,000 potentially eligible patients.

Interventions:

Comparisons: (1) modes of recruitment: face-to-face (F2F) recruitment versus mail-based recruitment (MBR); (2) recruitment source (sleep versus cardiology clinics).

Measurements and Results:

Recruitment yield was defined as the ratio of the number of subjects randomized to the number of screened records. Recruitment costs were estimated based on staff time. Of the 148 randomized subjects, 25 were recruited from sleep clinics using F2F recruitment and 123 were recruited from cardiology using a F2F (n = 35) or MBR (n = 88) strategy. F2F recruitment yields were 0.17% and 0.30% for sleep versus cardiology sources, respectively (P = 0.04). A comparison of F2F to MBR showed recruitment yields of 1.11% and 0.90% and costs per randomized subject of $2,139 and $647, respectively.

Conclusions:

Large resources may be needed to meet the recruitment goals of sleep apnea intervention trials. Recruitment source and mode influence efficiencies. For a trial comparing active versus sham continuous positive airway pressure in patients with cardiovascular risk factors, recruiting from cardiology was more efficient than from sleep clinics. Mail-based recruitment was three times less costly than face-to-face recruitment.

Citation:

Gleason K, Shin D, Rueschman M, Weinstock T, Wang R, Ware JH, Mittleman MA, Redline S. Challenges in recruitment to a randomized controlled study of cardiovascular disease reduction in sleep apnea: an analysis of alternative strategies. SLEEP 2014;37(12):2035-2038.

INTRODUCTION

Sleep disorders affect an estimated 50 to 70 million Americans,¹ with annual costs in the United States estimated at $75 billion.² Observational data provide strong evidence that sleep disorders, particularly sleep apnea, contribute to an increased risk of diabetes, cardiovascular disease (CVD), stroke, and certain cancers.³ This large health burden indicates the potential for sleep interventions to reduce morbidity, mortality, and health care costs. Although meta-analyses support the efficacy of several interventions such as continuous positive airway pressure (CPAP) for treatment of sleep apnea-associated sleepiness,⁴ there is a paucity of high- level evidence from randomized controlled trials (RCTs) to guide treatment decisions. In fact, between 2000 and 2012, the health impact of sleep apnea treatment was only evaluated in 39 small RCTs, most with durations of less than 1 y.⁵

Although RCTs are needed to provide the evidence necessary to guide treatment decisions, such studies are challenging. One obstacle is timely enrollment of subjects who both meet eligibility criteria and are willing to accept random treatment allocation.^6–8 Suboptimal recruitment can reduce the sample size below that needed for hypothesis testing, increase study costs, and delay study findings.⁸ Notably, fewer than 50% of studies have achieved their recruitment targets without extending the duration of the trial.^9–11

The Best Apnea Interventions for Research (BestAIR) study was designed as a National Institutes of Health planning study to address challenges in conducting future large-scale trials of sleep apnea treatment. We describe the challenges and effect of alternative recruitment methods on recruitment yield and study efficiency.

METHODS

Trial Design

The BestAIR study aimed to randomize approximately 180 patients with sleep apnea who had either established CVD or at least three CVD risk factors into a 12-mo, four-arm intervention study. All arms included nightly use of nasal dilator strips and conservative medical therapy (CMT) defined as education on healthy sleep and lifestyle. The two active arms were: (1) active CPAP delivered using standard support from a trained sleep technician; and (2) active CPAP enhanced by a behavioral promotion intervention delivered by a behavioral therapist. The two control arms were: (1) sham-CPAP; and (2) CMT alone. The primary physiological outcome was change in mean 24-h systolic blood pressure. Process measures were recruitment yields, retention rates, and CPAP adherence levels.

Participants

Eligibility criteria included: moderate to severe sleep apnea (apnea-hypopnea index (AHI) ≥ 15), and age 45 to 75 y with established CVD or diabetes, or age 55 to 75 y for those with three or more CVD risk factors. Patients with severe nocturnal hypoxemia, excessive sleepiness (Epworth Sleepiness Scale [ESS] score > 14), severe chronic health conditions, a recent major cardiac event, or prior CPAP use were ineligible.

Face-to-Face Recruitment

Starting in March 2011, potential subjects were identified by screening the electronic medical records of patients referred to two sleep clinics affiliated with academic health centers in Boston, MA. With permission from healthcare providers, potentially eligible patients were interviewed by study staff at the time of their sleep clinic appointments to determine eligibility and interest. In October 2011, face-to-face (F2F) recruitment was expanded to include cardiology clinics affiliated with the same academic centers.

Mail-Based Recruitment

A mail-based recruitment (MBR) approach was initiated for cardiology clinics at one medical center in October 2011, and expanded over 9 mo to include specialty clinics at three other centers in Boston, MA. Potentially eligible patients were identified through screening electronic medical records, which at three sites included submitting search queries to an electronic patient registry. With permission from participating providers, brief study invitation letters and screening questionnaires were sent to individuals meeting initial eligibility criteria. Eligibility criteria were further determined using the returned questionnaire and a follow-up phone interview.

Recruitment Procedures

Figure 1 shows the patient flow pathways possible in the trial. Participants moved through three prerandomization phases. (1) Patients who had not yet undergone polysomnography (PSG), but met all other eligibility criteria, were administered a home sleep test (HST) to determine AHI; patients eligible by prior PSG directly entered a run-in trial. (2) Patients determined to be fully eligible initiated a 2-w run-in trial, designed to identify subjects who could not tolerate a nasal mask. The subject wore a CPAP mask nightly without any pressure applied (open to room air). (3) After the run-in trial, subjects were scheduled for a baseline research examination, where they were randomized to treatment.

Figure 1.

Flow of patients into the trial using face-to-face (F2F) versus mail-based recruitment (MBR) strategies.

Recruitment Cost Analysis

Costs included staffing and materials. Staff costs were calculated using the average hourly salaries of study staff and the number of hours spent on recruitment (e.g., screening records, interviewing patients). Materials costs for MBR included postage, paper, and envelopes. Overhead costs were excluded because they were invariable.

Recruitment Yields

Recruitment yields were compared by clinic source (sleep versus cardiology clinics) and recruitment mode (F2F versus MBR). Recruitment yield by clinic source was defined as the ratio of the number of subjects randomized to the number of screened records. Because the MBR approach used an electronic query to identify potentially eligible subjects, recruitment yield by mode was defined as the ratio of the number of subjects randomized to the number of patients who were potentially eligible on initial screening criteria (i.e., age, CVD, etc.). This yield analysis was repeated using the total number of records screened as the denominator. Because MBR was never used in sleep clinics and used almost exclusively starting in January 2013, the comparison of yield by source was restricted to the F2F strategy for participants recruited prior to January 1, 2013. To compare the yield and costs by mode, only data from cardiology settings during a 7-mo period when the two strategies were used concurrently were analyzed.

RESULTS

Participant Characteristics

Of the 169 participants randomized, 148 were recruited through the main pathways under evaluation (Figure S1, supplemental material). Of these, 25 subjects were recruited from sleep clinics (F2F), and 123 were recruited from cardiology clinics (35 F2F; 88 MBR). The sample had a mean (± standard deviation) AHI of 29.2 (± 16.5) and ESS of 8.3 (± 4.5). The distributions of subject characteristics were similar across the recruitment groups (Table S1, supplemental material).

F2F Recruitment Yields (Sleep Versus Cardiology Clinics)

For F2F recruitment, a total of 23,846 patients were screened during the analyzed period. Of these, 13,736 were identified from sleep clinics, 23 of whom were randomized to treatment (0.17%). Major reasons for ineligibility were sleepiness, prior CPAP use, absence of CVD, or lack of equipoise by the patient or physician. Of the 10,110 patients identified from screening electronic medical records in cardiology clinics, 30 were randomized (0.30%; P = 0.04 for the difference in the proportion randomized by source). Major reasons for ineligibility were lack of sleep apnea symptoms, co-morbid conditions, or low AHI on screening.

F2F Versus MBR Recruitment Yields (Cardiology Clinics)

During the period analyzed, 1,744 patients were identified as potentially eligible on initial medical record screening and targeted for F2F recruitment in cardiology clinics, whereas 3,993 were targeted for MBR using electronic query tools. Of these, 19 were randomized using the F2F strategy (1.11%) and 36 were randomized using MBR (0.90%), demonstrating similar yields by mode (P = 0.55). However, if F2F recruitment yield were assessed considering all 8,401 patients screened for F2F recruitment, the yield would decrease to 0.23%, nearly four times lower than the MBR strategy (P < 0.001).

Cost Comparisons

Staff support for F2F recruitment efforts included time for screening and interviewing potential participants, as well as “down time” in clinic waiting for patients. On average, 58 h of recruiter time per week were spent on F2F recruitment activities. Using an average recruiter salary plus fringe of $24.85/h with an average of 0.67 randomizations per week resulted in a cost per randomized subject of $2,139 for those recruited through F2F contact.

For MBR, most staff time was spent contacting by telephone subjects who had returned questionnaires, with much less time spent reviewing medical records and virtually no “down time.” On average, 111 mailings yielded one randomized subject. Approximately 143 mailings were sent per week, yielding an average of 1.29 randomizations and requiring an average of 26 h of recruitment staff time weekly. Using $1.10 as the average material and postage cost for one mailing and about 3 h of student time (at $8/h) preparing mailing materials per 100 letters resulted in a cost of $647 per randomized subject, a value more than two thirds lower than the cost of F2F recruitment per randomized subject (Table S2, supplemental material).

DISCUSSION

Although a critical step in clinical trial conduct is successful participant recruitment, the overall yields, efficiencies, and costs of alternative approaches are not well understood, particularly for sleep apnea trials where interventions can be complex. We observed very low recruitment yields when screening patients from sleep or cardiology clinic settings. This underscores the need to identify large patient pools when designing RCTs with numerous eligibility criteria, such as moderate sleep apnea without sleepiness or prior CPAP use, CVD risk factors, and willingness to enroll in a year-long protocol. Notably, recruitment yields were better when recruiting from cardiology compared to sleep clinics. Although recruitment yields will vary by condition, setting, and design, several studies have reported similar low yields for other conditions.^12–14

Significantly lower costs were associated with MBR compared to F2F recruitment. Whereas some studies report greater success in proactive (e.g., F2F) recruitment,^15–17 others advocate the use of reactive (e.g., MBR) recruitment.^18–20 Although F2F recruitment has the advantage of directly connecting the research staff with potential participants, this recruitment mode can be inefficient because of the need to coordinate activities around routine clinical activities and the lack of research infrastructure within clinic settings. In contrast, electronic databases often have efficient search procedures for identifying patients who meet core eligibility criteria and can be leveraged to identify large pools of potential participants. A strong electronic medical record was particularly useful in improving efficiencies and reducing costs by approximately 70% compared to F2F recruitment. Studies of recruitment methods for other conditions have similarly demonstrated the cost-effectiveness of MBR,^12,21,22 and underscore the potential for electronic medical records to facilitate clinical research.

This analysis had several strengths and limitations. Data were collected prospectively and provide a fairly complete categorization of key costs and yields. However, activities were based in academic medical centers in one geographic location, and thus data may not be generalizable to other settings. This analysis applies to a trial comparing an accepted treatment (CPAP) to CMT and a sham-CPAP device; patient and provider acceptance may vary for different conditions or interventions. The cost estimates focus on recruitment efficiency, but do not account for costs associated with collection of alternative clinical data from each source (e.g., sleep apnea screening tests for patients with no prior PSG). Yield estimates were based on both total numbers of screened individuals and numbers of screened individuals meeting initial eligibility. Higher yields would have been estimated had the yields been based on the actual number of subjects directly contacted.

In summary, this study provides new data relevant to the emerging areas of RCTs for sleep apnea regarding yield and costs associated with recruitment to a sleep apnea RCT. Although specific recruitment yields likely depend on many factors, the findings highlight the need for large patient pools and the potential improved efficiencies possible through leveraging electronic medical record repositories to conduct MBR. Understanding the factors that influence recruitment before initiating a study can be key to the success of trials that generate the data ultimately needed to practice evidence-based medicine. With greater constraints on funding and more emphasis on “results-based” trials, there is a need to continue to identify best practices for subject recruitment and for optimizing research interfaces within clinical medicine.

DISCLOSURE STATEMENT

This study was supported by NIH NHLBI 1U34HL105277 and a grant from ResMed Foundation. Equipment was donated by ResMed Inc. and Philips-Respironics. The authors have indicated no other financial conflicts of interest.

SUPPLEMENTAL MATERIAL

Figure S1

Cumulative participant accrual over the course of the study for each of the sources (sleep clinic vs cardiology clinic) and recruitment mode (F2F vs MBR).

Table S1

Baseline characteristics of randomized subjects.

Table S2

Summary of cost analysis.