Comparison of characteristics and outcomes by initial study contact (website versus staff) for participants enrolled in a weight-management study

Abstract

Background

Traditional recruitment methods for clinical trials, such as telephone, mail, and print-media, are often inefficient, costly and use large amounts of staff time and resources.

Purpose

This analysis was conducted to determine whether retention, demographics, and outcomes differed between enrolled participants who responded to recruitment outreach using an Internet-based information and registration system and enrollees whose first contact was with study staff via telephone.

Methods

We identified potentially eligible participants from Kaiser Permanente Northwest databases and mailed brochures inviting them to participate in the Life weight-loss maintenance study. We also used employee newsletters, a member-directed website, and messages to employee email distribution lists to publicize the study. All outreach methods contained both a website address and a telephone number through which respondents could register for an information session. The website contained the same information as was provided by staff over the telephone.

Results

Out of 2122 potential participants who expressed interest in the study, 70% did so through the website. There was no difference in retention rates between enrollees who initiated contact through the website (WEB = 308) and enrollees who contacted the study by telephone (Staff = 161). The WEB group was younger (p = 0.01), had higher income (p = 0.01) and education (p < 0.01) levels, and lower body mass index (p < 0.01). There was a trend toward greater weight loss in the WEB group (p = 0.06).

Limitations

We did not conduct a formal cost analysis of the two methods. Also, the population for this analysis was mostly Caucasian and middle income; thus, we cannot draw conclusions about the generalizability of our findings to more racially and economically diverse populations.

Conclusion

Enrolled participants who used a website to register for an initial study information session had similar study retention and outcome performance as enrollees who used a more traditional telephone method. For larger clinical trials, a website may help researchers more efficiently and cost-effectively achieve recruitment, eligibility, and randomization goals. More research is needed to determine whether similar recruitment and retention patterns are observed among racially and economically diverse populations when these and similar methods are compared.

Introduction

Obtaining a sufficient number of participants to meet study objectives is an enormous challenge for many clinical trials. Published studies show that fewer than 50% of clinical trials meet recruitment targets, and many only reach targets by extending the length of the accrual period. [1;2] Slow recruitment increases the cost of trials and may lead to underpowered studies that fail to answer the questions addressed. A recent Cochrane review called on trialists to publish evaluations of recruitment strategies in order to help remedy these problems [1].

Clinical trial recruitment often initially relies on print media, radio and television outreach. After potential recruits respond, study personnel traditionally spend time and resources on telephone calls and in-person visits to provide information, schedule and register potential participants, and screen out those not eligible. Such intensive and repetitive methods are not efficient and can be expensive [3]. Therefore, evaluating other methods and identifying processes that result in efficient identification of potential participants likely to be eligible for studies is essential to control the costs of conducting clinical trials and to improve the robustness of the data they provide by increasing the numbers of participants.

Although the possibility of using the Internet for outreach and recruitment into clinical trials has been discussed for many years, this method only became feasible when Internet access expanded to a large portion of the population [4]. Internet access in the United States has increased in the last decade: the 44% of the US population able to access the Internet in 2000 grew to 74% in 2009 [5]. In addition, some initial studies examining the success of recruitment via the Internet have been promising, including a study comparing a letter-only recruitment method with a letter + website method that reported a response rate significantly higher among those who received the letter + the website option [6]. Similarly, authors of a study that used a website specifically designed for initial screening of potential participants concluded that the 20% yield from screening to consent using the website reduced recruitment cost from a previously experienced yield of 10% when screening was done solely through in-person telephone calls [7].

The Internet holds huge potential for helping researchers more efficiently inform, register, and screen recruits for clinical trials. However, little is known about participants who use the Internet to enter a trial and whether retention rates – or outcomes – for Internet recruits differ from retention rates and outcomes in participants reached through more traditional methods; so far that potential has remained largely untapped. The purpose of this analysis was to determine whether retention, demographics, and outcomes differed between enrolled participants who initially responded to recruitment outreach using an Internet-based information and registration system or using the telephone to speak to study staff.

Methods

The Life study is a randomized clinical trial testing the difference in one-year weight-loss maintenance between an experimental intervention and a control arm following an intensive weight loss program. A complete description of the rationale, design, and methods for the Life study has been published elsewhere [8]. Briefly, the Life study recruited participants to a standard, behavioral, group-based, six-month weight-loss program (Phase I) followed by six months of weight-loss maintenance group sessions (Phase II). At the end of the weight-loss program, participants who lost at least 10 pounds and attended at least 70% of the weekly weight-loss group sessions were randomized into Phase II. The Life study was sponsored by the National Center for Complementary and Alternative Medicine (NCCAM) and approved by the Kaiser Permanente Northwest IRB; all participants provided written informed consent. The methods we describe in detail here are for a secondary analysis comparing characteristics of enrollees who responded to recruitment outreach using the Internet and enrollees who responded through more traditional telephone methods.

Study Design

We recruited participants from the Kaiser Permanente Northwest (KPNW) membership. Our main recruitment outreach strategy was an informational brochure mailed to potentially eligible participants we identified from KPNW databases; the brochure contained an invitation to seek more information. Based on our experience recruiting for weight loss maintenance clinical trials involving similar participant commitment, we estimated it would take ~15,000 brochures to yield a target of 510 Phase I enrollees and 288 randomized Phase II participants. We mailed 14,750 brochures, in addition to using other unquantifiable outreach methods that included employee newsletter announcements, information posted on a member-directed web site, and messages sent to employee email distribution lists.

All outreach information included both a website address and a telephone number that allowed potential participants to express an interest in the study and register for an information session. The first step in the study-qualification process was to register for and attend a group-based information session. We designed the website to be a tool for potential participants to register for a study information session without having to contact a study-staff member. Our purpose was to allow potential participants the convenience of a self-registration process while providing the same information given to those who chose to contact a study staff member by telephone.

Website method

The website began with a welcome message about the length of the registration process (about five minutes), the information necessary to complete the registration (a KPNW health identification number and contact phone number), and the dates, times, and locations of upcoming information sessions. The website provided a concise study description and a brief explanation about participation requirements. Potential participants selected “yes” or “no” to high-level exclusion criteria (e.g. Are you younger than 30 years old? or Are you taking insulin?) Answering “yes” to any of the questions indicated ineligibility and ended the session. Whenever the session continued, potential participants provided contact information and selected a date and location for an information session. Only sessions with space available were listed as choices, eliminating the need for a message that a session was full or not available. A confirmation page displayed the date, time, and location of the information session for which the participant had registered, including a map to the facility where the session would take place. The potential participant was encouraged to print the confirmation page and informed that no reminder would be sent. At any time during the online registration session, potential participants could select a “quit” option to end the session.

Traditional telephone method

As an alternative to using the website to register for a study information session, potential participants could call the study office and register by telephone. Staff members asked the same set of high-level screening health questions, provided the same study description and the same set of expectations as those listed on the website, and registered potential participants for an information session. Upon registration into an information session, the potential participant was asked to write down the information about time and locations because no additional reminders would be provided once the registration telephone call had concluded.

Screening and enrollment

After completing the information session, interested potential participants could schedule an individual screening visit to determine whether they met the eligibility criteria and to sign the informed consent form. During the screening process, we did not conduct reminder telephone calls for visits prior to study enrollment. Potential participants could reschedule individual screening visits by calling prior to the originally scheduled appointment.

Whenever a potential participant met all eligibility criteria, she or he was scheduled into the Phase I weight loss program. Eligible participants were men and women over 30 years of age with body mass index (BMI) between 30-50 kg/m² (weight in kilograms divided by height in meters squared) who lived in the Portland, Oregon, metropolitan area. Those who had had treatment for cancer in the last two years, were taking insulin for diabetes, had congestive heart failure, had experienced a cardiovascular event such as a stroke in the past two years, were taking weight-loss medications, had a history of gastric-bypass surgery or liposuction, or were unwilling to keep a food record during the screening process and unwilling to agree to the diet and exercise intervention recommendations were excluded. Attendance at the first Phase I weight loss program group session completed the participant enrollment process.

Analysis

The comparison for this analysis is between enrolled participants who registered for the study information session using the study website (WEB), and enrollees who registered by talking with a study staff member using the telephone (Staff). The sample is limited to enrolled participants because data collection began at enrollment. We used SAS version 9.1 to perform Chi-square tests to compare proportions and t-tests to assess differences in means. In preparation for analysis, data were audited and missing values (for demographic data, food records, and final weight) were replaced using multiple imputation [9].

Results

Figure 1 shows the flow of the screening process and the number retained at each step. Of the 2122 potential participants who expressed interest in the study, 1483 (70%) did so through the website. Of the 1344 people who scheduled an information session, 916 (68%) used the website. Once the initial information session was scheduled, the proportion retained at each screening step persisted equally for both groups throughout the remainder of the screening process. At the completion of screening, the initial outreach efforts provided a yield of 469 enrollees in the study. Overall more enrolled participants used the website (308) to express initial interest in the study than the number who spoke to a staff member by telephone (161). However, the enrollment yield (% enrolled per number contacting the study) was higher in the Staff group, (WEB 21% (308/1483) vs. Staff 25% (161/639)). Of the 26 individuals who were not eligible after attending the individual screening visit, 15 were in the WEB group and 11 were in the Staff group. Ineligibility reasons varied, but the most common were not completing the screening food-record requirement and expressing an unwillingness to follow the intervention recommendations for diet and exercise.

Table 1 shows baseline characteristics of the 469 enrolled participants. The WEB group was younger, had higher education levels, higher income and lower BMI (p ≤ 0.01). There were no group differences with respect to gender and race. Table 2 shows a comparison of intervention outcomes. There was no difference between the two groups in the number of sessions attended or number of food records kept during Phase I. There was a trend toward greater weight loss in the WEB group, but the difference was not statistically significant. A higher proportion of the WEB group was randomized (WEB 64% vs. Staff 52%), however the randomization yield from initial contact was similar between groups (WEB 13% (198/1483) and Staff 13% (85/639)).

Table 1.

Characteristics of enrolled participants (N=469)

	WEB (n=308)	Staff (n=161)
Age, years, mean (SD)	54.0 (11.8)	57.7 (11.2)
Education, % college degree or higher	59.1	42.9
Income, % > 60K	57.8	45.1
BMI; mean (SD)	37.3 (5.1)	38.6 (5.4)
Gender, % female	81.5	86.3
Race, % non-white	8.8	11.9

Table 2.

Comparison of intervention outcomes

	WEB (n=308)	Staff (n=161)
Percentage of intervention sessions attended, mean (SD)	74.5 (26.3)	70.6 (27.2)
Daily food records kept per wk (0-7), mean (SD)	5.1 (2.3)	5.0 (3.0)
Change in weight, kg, mean (SD)	-6.8 (8.2)	-5.5 (12.1)
Randomized into Phase II (% of group)	198 (64.3)	85 (52.8)

Discussion

This analysis demonstrates that the Internet may be a helpful tool for increasing outreach and recruitment into large clinical trials. As the Internet continues to expand, and its technological sophistication increases, so does its potential to make a significant impact on research, including on outreach and recruitment for clinical trials. Potential participants are increasingly able and willing to access clinical trials through non-traditional means. While website development can be costly, websites limited to study description, orientation, and self-registration may be feasible, cost effective, and have the potential to save large amounts of recruitment staff time and resources. A website is available at any time of the day or night and can handle high volumes of activity, whereas a recruitment call center must be staffed by several employees who take only one call at a time. While completely eliminating recruitment staff does not seem feasible or even desirable, utilizing Internet recruitment tools as part of an overall recruitment plan, particularly for simple, repetitive functions such as registration into information sessions, is clearly a potentially cost-saving endeavor which holds enormous potential to reach large numbers of people.

In this analysis, the WEB and Staff groups differed in certain characteristics, but the differences were not clinically significant, (age and BMI) or were expected (education and income). While all our outreach efforts listed both options for contacting the study, i.e., website address and telephone number, two thirds of enrollees chose the website option; there was no difference between the WEB and Staff groups in terms of retention at each screening step after scheduling an information session. It is worth noting that a slightly higher proportion of enrollees started out by talking to a staff member. This higher enrollment yield may signal the importance of personal communication, but comes at a cost that would likely be negated for larger trials. The fact that both methods of enrollment yielded low rates of ineligible potential participants and of participants who lost interest after completing the individual screening visit further signals the benefits of using a website to help screen out ineligible people before costly staff time has been spent.

The overall study enrollment and randomization yields were very close to what we predicted based on our previous recruitment experience, despite using a website as one method for potential participants to indicate interest. The similar randomization yields between groups indicate that both options for expressing interest are viable. Our results should help to counter skepticism that people using a website to express interest in a clinical trial may be prone to miss visits or to take the study less seriously because they did not have an in-person conversation.

One limitation of this analysis was that we did not collect cost data to allow us to compare cost-yield of the two options for responding to recruitment outreach. However, based on our previous experience with studies that used telephone as the only response option for participants to initiate contact, we estimate that the development costs of this website were offset by reductions in recruitment staff time. The 66% of enrollees who registered for the screening visit through the website did not consume any staff resources from the time they logged on to the website to the time they attended an information session, other than the resources that went into website development. Cost-saving would be magnified in larger studies with target sample sizes approaching 500 or more because of the one-time fixed cost of website development compared to an additive cost of staff to field individual telephone calls. An additional limitation of our analysis was the small number of non-white participants, which may limit applicability to studies that seek to enroll racially and economically diverse populations.

The differences between the WEB and Staff groups in income and education levels were not surprising but could be informative for researchers who plan to use a website for recruitment efforts. Conversely, the difference in average age, 54 vs. 58 years, is likely to have very little clinical significance with regard to considerations for using a website to recruit participants. Unexpectedly, the WEB group had lower initial average BMI and a trend toward more weight loss in Phase I. We speculate that the greater success of the WEB group may be related to the higher socio-economic status of this group at entry, but our analysis did not yield any definitive causal explanations for this difference.

We conclude that potential participants, especially in higher socio-economic areas, are willing and able to use a website to indicate interest in participating in a clinical trial. Further, for large clinical trials (~500+ participants) a website may be more efficient than a staffed telephone line to help researchers achieve recruitment, eligibility, and randomization goals. In this population, it was promising that those in the WEB group were as committed to the study as those in the Staff group based on retention and adherence rates.

There are many issues associated with using Internet-based systems as part of recruitment that our analysis did not explore. For example, we could not measure reach or representativeness of all individuals who contacted the study because data collection did not begin until after the initial information session. These results indicate that before the promise of the Internet to increase recruitment and lower study costs can be realized, more research is needed. This analysis shows that, at least in a mostly white and middle-income relatively educated population, fears that a study population using a website to contact a study will lack commitment, drop out, or have poor outcomes, were not borne out. Determining whether our findings generalize to other, more diverse populations and how to harness the power of the Internet to contact specific hard-to-reach populations are two research areas that should be high priorities for those seeking to lower the costs and increase the efficiency of conducting future trials.