A web-based screening and accrual strategy for a cancer prevention clinical trial in healthy smokers

Abstract

Screening and recruitment of qualified subjects for clinical trials is an essential component of translational research, and it can be quite challenging if the most efficient recruitment method is not utilized. In this report, we describe a successful web-based screening and accrual method used in a randomized prospective chemoprevention clinical trial with urinary biomarker endpoints. The targeted study population was a group of at-risk healthy current smokers with no evidence of lung disease. Craigslist was used as the sole recruitment modality for this study. All interested subjects were directed to a pre-screening website, in which subject questionnaire responses were linked to the study coordinator's secure e-mail account. Of the 429 initial inquiries, 189 individuals were initially eligible based on the questionnaire response. One hundred twenty-two people were telephone-screened, of whom 98 subjects were consented, 84 were randomized and 77 subjects completed the study successfully. Utilizing this single web-based advertising strategy, accrual for the trial was completed 7 months prior to the projected date. Craigslist is a cost effective and efficient web-based resource that can be utilized in accruing subjects to some chemoprevention trials.

1. Background

Recruitment and retention of an adequate number of qualified subjects is essential for the successful completion of a clinical trial [1,2]. Every clinical trial faces the challenge of meeting the accrual goal within a specified time and budget. Most trials utilize various recruitment strategies that can be costly and labor intensive [3]. Often studies report a need to extend the proposed accrual period or the need for additional funding in order to meet the study goals [4]. Some of the reasons for such recruitment problems include inadequate planning, overestimating the accrual yield based on utilized methods, lack of proper recruitment monitoring and inability to change recruitment strategy when accrual is lagging [5]. In addition, changing medical practice (such as revised screening or treatment guidelines) or newly identified rare toxicities of preventive agents can delay recruitment to clinical trials. Identifying and utilizing cost effective recruitment methods allowing for consistent subject accrual is essential.

Chemoprevention trials suffer from similar recruitment challenges. These trials are generally performed in healthy individuals at risk of cancer but without evidence of disease. Many of these trials have stringent inclusion and exclusion criteria to capture the individuals at highest risk of disease and may require invasive procedures. In the case of lung cancer prevention trials, substantial tobacco exposure but otherwise overall good health is generally required for participation in order to minimize the potential risks and optimize the overall risk–benefit ratio. Given that many smokers have significant co-morbidities that would exclude them from clinical trials, an efficient screening approach to identify potentially eligible individuals while excluding ineligible subjects is critical. Here we describe our experience with Craigslist as an effective, low-cost web-based resource to identify potential participants to a lung chemoprevention trial. Our study consisted of a short agent intervention with urinary biomarkers as endpoints and no requirement for invasive procedures, thereby being considerably less complex than typical phase II lung cancer prevention trials. However, the efficient identification of eligible healthy smokers suggests that this approach is effective in accruing normal healthy volunteers and could also be useful as a first screening step for studies that subsequently require additional invasive procedures to qualify participants for clinical trials.

2. Methods

2.1. Study entry criteria and schema

This was a randomized, open label biomarker trial studying the role of zileuton or zileuton plus celecoxib in modulating the urinary biomarkers of lung injury. Based on sample size calculations, 80 subjects were required to enroll in the trial. To qualify for the study, the subjects were required to be 18 years of age or older, healthy (ECOG performance status of 0 or 1) current smokers with ≥10 pack-years of smoking (calculated by multiplying the estimate of the average number of cigarettes smoked per day by the number of years of smoking divided by 20). The exclusion criteria included active cancer, history of cancer, chronic inflammatory conditions (e.g. Crohn's disease, arthritis, and ulcerative colitis), active infection, concurrent use of certain medications (e.g. celecoxib, zileuton, corticosteroids, nonsteroidal anti-inflammatory drugs (NSAIDs), or other investigational drugs within 30 days of consent), contraindication to celecoxib or zileuton (pregnancy, peptic ulcer disease, sulfa allergy, prior cardiovascular events), and kidney or liver disease. After meeting all screening criteria and randomization, the participants were started on the study agents for 6 ± 1 days. The trial consisted of 2 arms (administration of zileuton 1200 mg twice daily; combined administration of zileuton 1200 mg twice daily plus celecoxib 200 mg twice daily) with 60 and 20 participants in the single agent and combined agent arms, respectively. There was no additional post-intervention follow-up after the final clinic visit. The study schema is presented in Fig. 1.

Fig. 1.

Study schema.

2.2. Recruitment methods and activity

Originally, a multifaceted recruitment approach was planned, as had been successfully used in a previous trial targeting a similar population representative of individuals at risk of smoking-related lung diseases [6]. These strategies include flyer posting, public service announcements, and internet and e-mail based circulation. According to our previous recruitment history, the study duration of 2 years was calculated based on a prior track record of 3.3 subject accruals per month. Prior to adopting this multifaceted strategy, a different approach was tried first. At the beginning of the trial, a pre-screening webpage with study information was developed within the university hospital website allowing the interested subjects to answer basic questions on demographics and smoking history and provide contact information for the study coordinator. This site was designed based on the inclusion and exclusion criteria. Advertisements on Craigslist were used to direct potential participants to the pre-screening webpage. Craigslist was chosen because it is a centralized online network that features free classified advertisements that are organized by interest categories, thereby allowing the potential viewer to rapidly identify the desired service or product. A clinical trial advertisement with the IRB approved text was placed in the sequence of sections within Craigslist: New York Craigslist > Manhattan > Community > Volunteers. Subjects were referred to a hyperlink directing them to the study website in order to complete the initial questionnaire. Once a prospective subject completed and submitted the initial online questionnaire, an e-mail would automatically be generated with the subject's information addressed to a secure university e-mail account accessible by the study coordinator. Based on this information, the site coordinator contacted potentially eligible participants for a pre-screening phone call to ascertain basic study eligibility. Since Craigslist ads expire after 1 week such that the solicitations are no longer available for viewing, the ad was renewed on a weekly or bi-weekly basis based on the number of responses. Ads were posted between April 2010 and August 2011. Given the excellent response rate, Craigslist was used as the sole modality for trial accrual. All but two participants, who were referred by friends, were recruited via Craigslist. The participants received $40 for the first two visits and $190 at the final visit after the completion of the study for travel and parking expenses.

2.3. Screening procedures and randomization

The initial pre-screening was performed based on the web-based questionnaire submitted by the interested subjects. This provided basic demographics, age and smoking pack-year history, thereby allowing the coordinator to determine early eligibility. Eligible individuals were then contacted via e-mail and a phone interview was conducted. During the phone interview, the study was explained in detail and eligibility was further confirmed based on the set criteria before scheduling subjects for a screening visit. At the screening visit, a physician directed interview was performed to determine eligibility based on the inclusion/exclusion criteria, followed by liver function and pregnancy tests (if applicable). Once the final eligibility was confirmed (normal laboratory tests) and the second stage of registration was completed, participants were assigned a randomization number pre-generated by the study biostatistician.

Eligible subjects were randomized to one of the two arms (zileuton only or zileuton and celecoxib) in a 3:1 ratio. During the baseline visit, a full medical history and physical exam were performed and urine was collected. Participants received the study drug(s) at this visit and a pill diary to document the dosage taken every day. The pill diaries were used to insure proper medication compliance. Participants were also made aware that drug levels in the blood would be measured at the completion of the trial as a biological measure of drug compliance.

Each subject received a phone call prior to the final visit to encourage drug compliance and was reminded of the final scheduled visit. At the final visit, subjects returned any remaining pills along with the pill diary to follow-up with medication compliance. Additional urine and blood (plasma) were collected post study medication(s). Recruitment strategy was monitored at monthly protocol meetings and documented in the minutes. Outcomes, including the number of participants enrolled per month, were tracked. If recruitment were to lag by 20% or more of the target rate, modified strategies to enhance recruitment were to be implemented. Due to the short-term nature of the trial, few withdrawals were anticipated. Participant withdrawal and loss to follow-up were recorded.

2.4. Data collection and analysis

All participant data were collected prospectively and maintained in a secure database. Travel time and distance were calculated based on the provided address utilizing Google maps. Characteristics of the participants at each stage of the recruitment were summarized. Association between subject characteristics and recruitment success was examined using logistic regression. Poisson regression was used to examine the association between response rate per day and number of days from latest ad renewal. To investigate if the response rate to the ad varies depending on the day of the week or the month of the year the ad was renewed, we examined the association between the response rate per ad renewal (i.e. the number of responses received following the latest ad renewal) and the day of ad renewal and between the response rate per ad renewal and the month of ad renewal using the non-parametric Kruskal–Wallis test. Association with p-value <0.05 was considered statistically significant.

3. Results

The trial opened in April 2010, with the first subject enrolling in May of 2010, and accrual was completed in August of 2011, approximately 7 months ahead of the projected completion date (Fig. 2). On average, 5.76 subjects per month were consented and 4.94 subjects per month were randomized to the study. At the end of subject accrual, 429 individuals had responded to the Craigslist ad. Of these, 189 were initially eligible based on smoking history (years smoking and cigarettes smoked per day) and age response sent from the website questionnaire. Of the 122 individuals who were telephone-screened, 108 were potentially eligible. Ninety-eight of the 108 of the subjects were scheduled for the screening visits and consented. A total of 84 subjects were randomized, 80 subjects started the study medication(s), and 77 subjects completed the entire study. The demographics and travel distance of the subjects responding and enrolled in the study are summarized in Table 1. On the initial response to the ad, the gender distribution was similar for both sexes, but 61% of the subjects completing the study were male and 39% were female, with a median overall age of 45 years. Similar numbers of black and white subjects participated in the study (48% and 49% respectively), with only 3% Asian participants.

Fig. 2.

Enrollment progress. Actual enrollment was speedier than the anticipated enrollment, resulting in earlier than anticipated study completion.

Table 1.

Characteristics of subjects included in each stage of the trial recruitment process.

	All	Initially eligible	Phone interview	Phone interview eligible	Randomized	Completed
n	429	189	122	108	84	77
Age
Mean ± SD	34.2 ± 11.8	43.3 ± 9.4	44.0 ± 8.4	43.3 ± 8.3	43.6 ± 8.6	43.8 ± 8.8
Median (range)	31 (17, 69)	44 (22, 69)	45 (24, 65)	44 (24, 64)	44.5 (24, 64)	45 (24, 64)
Unknown	10	3	1	1	0	0
Gender
Female	216 (50.5%)	88 (46.6%)	45 (36.9%)	40 (37.0%)	33 (39.3%)	30 (39.0%)
Male	212 (49.5%)	101 (53.4%)	77 (63.1%)	68 (63.0%)	51 (60.7%)	47 (61.0%)
Unknown	1	0	0	0	0	0
Race
Asian	2 (2.1%)	2 (2.1%)	2 (2.1%)	2 (2.1%)	2 (2.4%)	2 (2.6%)
Black	46 (48.4%)	46 (48.4%)	46 (48.4%)	46 (48.4%)	39 (47.0%)	37 (48.1%)
White	47 (49.5%)	47 (49.5%)	47 (49.5%)	47 (49.5%)	42 (50.6%)	38 (49.3%)
Unknown	334	94	27	13	1	0
Smoking (pk yr)
Mean ± SD	11.3 ± 12.1	21.3 ± 12.0	21.2 ± 11.5	21.1 ± 11.1	21.1 ± 10.5	21.7 ± 10.8
Median (range)	7.08 (0, 78)	18 (8, 78)	19 (8, 68)	19 (8, 68)	19 (10, 68)	20 (10, 68)
Unknown	1	1	1	0	0	0
Distance (miles)
Mean ± SD	12.5 ± 25.3	12.0 ± 24.5	9.4 ± 12.7	9.0 ± 12.1	9.2 ± 11.8	8.9 ± 12.1
Median (range)	8 (0.1, 322)	7.8 (0.1, 231)	6.6 (0.1, 101)	6.3 (0.1, 101)	6.8 (0.1, 101)	6.6 (0.1, 101)
Unknown	8	3	0	0	0	0

Reasons for ineligibility at the time of initial response were a self-reported smoking history below the requisite 10 pack-years and/or less than an average of 10 cigarettes per day. Ineligibility after telephone screening was generally based on preexisting medical conditions and/or concomitant medications (based on exclusion criteria; primarily NSAID use). Ten individuals who were eligible after the telephone screening and scheduled for screening visits were lost to follow-up or chose not to continue with the study. An additional 14 individuals who were consented were excluded prior to randomization due to inconsistent response in self-reported smoking history, concomitant medications (based on exclusion criteria; primarily NSAID use), abnormal liver function tests or positive pregnancy test.

Among the 84 randomized participants, 2 were lost to follow-up, 1 withdrew from the study, and 1 was ineligible after the base line visit. Eighty subjects started the study agent(s) and 77 subjects successfully completed all study requirements, indicating a 3.75% loss to follow-up. The excellent retention rate is likely due to the short duration of the study and the favorable tolerability of the study drugs. Additionally, the phone contact prior to the final visit and the subject compensation provided upon completion of all visits may have influenced the participant retention rate.

The characteristics that are associated with accrual success in our study population were evaluated. The association between recruitment success (randomized subjects) and subject characteristics was examined using logistic regression. Age, gender, and distance to the trial site were significantly associated with successful recruitment (P<0.001, P = 0.02, P = 0.01, respectively) in univariate analysis. Specifically, increasing age, male gender, and lesser distance from the trial site were associated with increasing odds of recruitment success. In multivariable logistic regression analysis with covariates of age, gender and distance, age remained statistically significant (P<0.001), while the effects due to gender and distance were no longer statistically significant (P = 0.18 and 0.17, respectively).

Since each ad expired after 1 week, the ad was renewed 68 times during the participant accrual period to ensure good visibility. The number of responses attributed to a renewal of the ad ranged from 1 to14 (Fig. 3A). The significance of the timing of ad renewal in relation to successful subject accrual was evaluated. As expected, the daily response rate was the highest on the day of ad renewal and declined significantly each day from the latest ad renewal (P < 0.001, Poisson regression; Fig. 3B). There is no statistically significant relationship between the response rate per ad renewal and the day of the week that the ad was renewed (P = 0.76, Kruskal–Wallis test). There is a strong suggestion of difference in the response rates per ad renewal and the month of ad renewal (P = 0.05, Kruskal–Wallis test). Specially, the summer month of July had the lowest response rate per ad renewal with the median of 3 (range 2–5) (Fig. 3C). To give an idea of the workload associated with this recruitment strategy, Fig. 3D shows the number of responses received per day (range 1–10) and the number of work days during which each number of responses was received. Of the 210 work days during which ad responses were received, six or fewer responses were received per day on all but 1 day and 1–3 responses were received on more than 75% of the work days. Ten responses were received on 1 day only.

Fig. 3.

Relationship between ad renewal and responses to the ad. (A) Response rate per ad renewal. The distribution of the response rate per ad renewal, defined as the number of responses attributed to the latest renewal of the study advertisement, was examined. (B) Effect of increasing time since ad renewal on daily ad response rate. The relationship between the response rate per day and the number of days from latest ad renewal was summarized by mean and standard error of the mean. (C) Effect of month of ad renewal on ad response rate. The relationship between the response rate per ad renewal and the month of placement of the ad renewal was examined. (D) Workload associated with ad placement. The burden of work resulting from ad renewals was assessed by examining the number of work days on which one or more ad responses were received. The majority of work days were associated with three or fewer responses.

4. Discussion

Cost effective and timely recruitment methods are the cornerstone of conducting a successful clinical trial. Delays in recruitment may result in increased duration and cost, may lead to premature termination of the trial, and may adversely affect the power of the study [7]. In addition to the strategies used in recruiting subjects, factors influencing successful recruitment include the study design and the participant characteristics. Study inclusion criteria, duration, intervention, subject compensation and location greatly influence the accrual process. Additionally, the desired characteristics of the study subjects such as age, gender, socioeconomic, and health status affect the population pool for a trial. Recruitment strategies should be tailored such that they can reach the largest potential pool of subject for any given clinical trial.

The majority of clinical trials use multiple strategies for recruiting subjects. In the Concerned Smoker Study, overall newspapers, television and radio advertisements accounted for 93% of randomizations with the mean efficiency of approximately 10% [3]. We initially planned a multifaceted recruitment process utilizing flyers, newspaper advertisement, media advertisement and internet ads. However, based on our previous experience with placement of flyers and public service announcements, we found these modalities to be unreliable and inconsistent in accruing subjects. We therefore chose to use a web-based advertisement using Craigslist as the primary method of subject accrual. The recruitment was closely monitored by monthly reviews to assess the need for the additional recruitment strategies. Additionally, since there was no cost associated with placing advertisements in Craigslist, this method was chosen over other web-based strategies such as Google Ad words. Based on previous experience in recruiting subjects for a chemoprevention trial using multimodality advertising methods, an accrual rate of 3.3 subjects per month was predicted. However, using this single method, we were able to randomize on average 5 subjects per month to the trial and complete the study in advance of the projected date. Unlike posting flyers that require substantial staff effort, this method only requires ad renewal at chosen time intervals. As anticipated, this time interval is critical, and the number of responses per ad decreased with the number of days after renewal, with minimal response after 6 days.

Recruitment of diverse population of participants such as minorities, women and the elderly is desired for many clinical trials [8]. The use of Craigslist resulted in the recruitment of an equal number of black and white participants to the study, with a wide age range of 24–64 years (median 45 years). This is in part due to the population diversity of New York City. According to the 2010 census, the demographic makeup of New York City is 44% white, 25.5% black and 12.7% Asian with overall median age of 35.5 years (http://quickfacts.census.gov). On the other hand, and not unexpectedly, the median age of our population was relatively young (45 years) for lung chemoprevention trials. A recently reported trial in ex-smokers showed a median age of 59 years, which is more typical of lung cancer prevention trials [9]. The reasons for the younger age in our trial are partially due to the less stringent tobacco exposure history (10 pack-years versus the more typical 30 pack-year requirements of most lung cancer prevention trials focusing on bronchial dysplasia), but may also be due to differing use of the internet by populations of different ages.

Variable success rate has been reported for various recruitment techniques. In a lung cancer chemoprevention trial involving heavy ex-smokers, radio advertisement generated the most inquires (71%), followed by internet posting (11.8%), with flyers and mass mailing generating 4.4% and 2.7% of the inquires respectively [9]. Internet classified postings showed a lower cost per randomization compared to other strategies used. Mass mailing has been used as the primary recruitment method in some studies with variable success [10–12]. In one study, more than 3.4 million letters were sent with only a 4.3% overall response rate [13]. In a trial evaluating the role of an herbal agent in preventing dementia, mass mailing was the most successful approach to recruiting elderly in both rural and urban settings [14]. Conversely, in the Lung Health Study, direct mailing was the least cost effective and resulted in only 1.5% of the accrual [11]. Broadcast media and print media had only 0.5% and 0.4% response rate, respectively. This is in contrast to the more recent chemoprevention study that showed radio ads were the most effective recruitment strategy in a study performed in a high density and a largely commuting population (Los Angeles) [9]. Utility of telephone surveys in recruiting subjects for primary prevention trials has also been reported; however, this method is costly and labor intensive [15].

Advertising cost per participant is rarely reported in clinical trials. In a heart failure trial, the average marketing cost per enrollee and the participant who completed the trial was $29.2 and $41.96, respectively, using a multimodality recruitment strategy [16]. In this trial, self-referral in response to flyers was the most economical marketing strategy but with only a 4.9% response rate. Since there is no cost associated with placing ads in Craigslist, the only costs associated with recruitment in our study were the initial phone call made to the participants who met the basic inclusion criteria and a portion of the site coordinator's time. Thus, no cost analysis was performed for our trial.

Unwillingness of subjects to take additional medications, give up supplements, and undergo invasive procedures, transportation problems, or concern for development of side effects are some of the barriers encountered in prevention trials [9,12,14]. The differences in recruitment success discussed above highlight the point that the recruitment strategies should be tailored to the characteristics of the study, desired population and the location of the trial. Recently, web-based recruitment registry services such as ResearchMatch have been shown to be effective methods of subject accrual for clinical studies [17]. However, such services require familiarity with the registry by the prospective participant since they ask for self-enrollment by prospective participants for potential future studies. Initial data show that recruitment to clinical trials using FDA-approved drugs was much less effective than recruitment to behavioral or psychosocial studies (0.73% vs. 4.2% of contacted individuals were enrolled, respectively) using ResearchMatch. Recruitment to studies using investigational agents or devices was even less robust. The overall small number of participants, limited public awareness, variation in the number of registered volunteers by state and region, and involvement in other trials may limit the effectiveness of such registries. Nevertheless, with the increase in use of social media globally, such approaches may eventually become an important avenue in recruiting subjects for trials. At the current time, direct comparison between different recruitment strategies utilizing web-based advertising and registries is not yet possible due to the lack of publicly available data.

Although Craigslist worked well for our study, one should recognize the strength and limitations of this approach. We were looking for healthy current smokers at risk of lung disease to participate in a brief clinical trial with minimal study demands in a highly populated urban setting. The study location and broad inclusion criteria made this recruitment technique an ideal choice for our trial. However, Craigslist may not be an efficient recruitment strategy in low population density areas and its role in recruitment for disease specific studies requiring the presence of histologic precursors to cancer, such as bronchial dysplasia identified during bronchoscopy, remains to be determined. Furthermore, the use of an Internet-based approach, which requires Internet access and a certain level of education and literacy, may skew the trial population towards higher education and socioeconomic status. Nevertheless, the brisk accrual in our study demonstrates that this is a viable method of accruing healthy at-risk individuals to chemoprevention trials. It will be of interest to investigate whether such an approach could be applied to the initial screening of potential subjects for more complex studies that subsequently require additional invasive procedures for participant qualification. With increasing access to the internet, web-based advertising methods such as Craigslist are efficient strategies for reaching a wide and diverse potential participant pool for clinical trials, especially in the urban setting.