Effort Required in Eligibility Screening for Clinical Trials

This study provides results based on prospectively captured effort to estimate the largely nonreimbursed costs of eligibility screening and suggests that screening can be a significant financial burden to an institution.

Abstract

Purpose:

Determining eligibility for a clinical trial (CT) typically requires a lengthy manual review of data for a single evaluation. The cost associated with eligibility screening is typically not compensated through contracts supporting CTs.

Methods:

We used a real-time tracking system that captures CT evaluations and provides information on evaluation outcomes and time spent on each eligibility screening by research staff. Using these data, we describe the effort and costs of eligibility screening overall and per enrolled patient for cancer CTs. The study sample included all completed eligibility assessment (evaluation) records for the 18-month study period. We used generalized multinomial modeling to predict evaluation outcomes and then used the resulting parameter coefficients to estimate the effort associated with each participant, adjusted for probability of being enrolled. From these data, we calculated cost associated with eligibility screening.

Results:

We found substantial variation in attributed cost by study type and phase. The cost of eligibility screening ranged by study phase from $129.15 to $336.48 per enrolled patient. The estimated annual cost of screening was more than $90,000.

Conclusion:

This study provides results based on prospectively captured effort to estimate the largely nonreimbursed costs of eligibility screening and suggests that screening can be a significant financial burden to an institution. Centers performing CTs may need to acknowledge the differences in screening costs for different study types when negotiating contracts with funding organizations. Information such as that captured here could support such negotiations to reduce the gap between reimbursed and nonreimbursed costs.

Introduction

Clinical trials (CTs) serve as the mechanism through which research is translated into standards of care. This translational process is dependent on the number of participants enrolled onto trials, yet this represents less than 4% of all adult patients with cancer.^1–5 The success of a CT depends on its capability to enroll a prespecified number of study participants within a planned timeframe.⁶ The speed and success in completing trials are dependent on rapid and efficient eligibility screening. Eligibility screening typically requires a manual review of clinical data sources,^7,8 which may entail hours for a single evaluation. This limits the number of patients who can be evaluated.⁸ This screening process is made more difficult by the need to repeatedly evaluate patients over their disease course and by the number of CTs and detailed eligibility requirements that research staff and physicians must consider at each patient visit.^2,7,8 Despite the intensity of the effort to determine eligibility, the cost associated with eligibility screening is typically not compensated through contracts supporting CTs.

Estimating the resources used to identify and screen patients is important; however, there is limited published information on the level of effort required to determine CT eligibility. Published studies have typically focused on the costs associated with recruitment methods (personal and direct mailing, media advertisements, referrals, public-site posters and flyers, presentations to local groups, and so on).^9–20 The literature describing recruitment costs most frequently reports on nontherapeutic trials, such as screening studies, or interventions that pose minimal risk to study participants.^{3,8,11,13,14,16,21–23} Data for nontherapeutic trials may not accurately represent the effort required to screen patients for therapeutic trials that have more clinically complex protocol requirements such as assessing multiple comorbidities and completing detailed reviews of treatment history.

We therefore estimated the effort as a measure of time spent and the probability of successful enrollment associated with eligibility screening for CTs. We used data from a real-time tracking system that captures evaluations for patients screened for CTs at our cancer center. The system provides information on the outcome of each evaluation (enrollment and eligibility status) and captures the time spent on these evaluations, as reported by the research staff performing the evaluations. From these data, we present estimates of effort (as time) and cost of eligibility screening in cancer CTs.

Methods

Data on eligibility evaluations were captured through the Clinical Trial Eligibility Database (CTED). CTED is an automated software system that facilitates the clinical research evaluation process by linking longitudinal patient data from multiple electronic sources, including clinical reports (eg, pathology, radiology reports, and inpatient and outpatient clinical notes), clinical laboratory results, billing records, and scheduled visits. The system serves as a repository for capturing information on the eligibility of patients with cancer for specific CTs by the clinical research staff (CRS; clinical research associates/coordinators and research nurses). CRS record a variety of parameters relevant to patients' CT eligibility status for each evaluation. These include: study (linked to the CT management system), research division, outcome for that evaluation (ineligible, refused, or enrolled), and time required to perform the activities necessary to complete the evaluation. Patient demographic characteristics are captured and integrated from the patient scheduling system. A detailed description of CTED functionality has been provided in a prior publication.⁸

Study Sample

The sample for this study was derived from all patients with a completed eligibility assessment (evaluation record) in CTED between January 2010 through July 2011 at the Virginia Commonwealth University Massey Cancer Center. The analytic unit is the evaluation that represents a unique combination of patient, study, and workup start and end dates. Patients may have been evaluated for more than one study or for the same study multiple times as their clinical status changed, potentially resulting in multiple evaluation records per patient.

There were 15 research nurses and 18 clinical research associates/coordinators who performed eligibility screening for CTs. Approximately 80% of the screening was performed by the research nurses because of the complexity of the clinical requirements for cancer trials.

Analytic Variables

CT characteristics.

Study type represented CTs grouped as therapeutic and nontherapeutic. Therapeutic studies were further classified as phase I to III. Nontherapeutic studies included observational/epidemiologic, ancillary, correlative, and companion studies. Study type was not available for 4.8% of the evaluations, because a patient might be determined ineligible or might refuse to participate in a CT before screening for a specific study. Research division corresponded with disease area (breast, gynecologic oncology, hematologic malignancy, medical oncology) or category of study (cancer prevention and control, palliative care).

Patient characteristics.

Demographic information, sex, race, and age were extracted from patients' scheduling or billing records. We categorized race as black, white, or other. Age at the time of the evaluation was grouped into five age groups: 19 to 45, 46 to 50, 51 to 64, 65 to 75, and > 75 years.

Evaluation characteristics.

The outcome for each evaluation record was the final eligibility status determination. Evaluation outcomes were categorized as: enrolled, refused, or ineligible. Ineligible included all evaluations where the patient was determined to be ineligible or the patient consented but was not enrolled. For patients with more than one evaluation, we assigned an order ranking to the evaluation to control for correlation between frequency of evaluation and time (effort) required to complete the evaluation. Order was coded as first, second, third, and fourth through ninth based on the workup start date for that evaluation.

Patient identification source.

Patients were classified as being identified through three sources: direct referrals by physicians or clinical nursing staff, multidisciplinary conferences, and electronic data sources. The latter category included: CTED, inpatient census lists, electronic health record, and manual lists from the patient scheduling system or from a daily list that was extracted by the decision support system.

Categorical Definition of Effort

The time required for eligibility determination was defined as the time recorded by CRS from identification of the patient as a potential study candidate through final eligibility determination. This represented the effort associated with that eligibility determination. The time intervals are recorded by the research staff either in real time or shortly after the completion of the eligibility workup. The time intervals are selected from a set of six effort categories, which were defined by the research staff. The effort categories include: cursory (< 10 minutes), tertiary (≥ 10 to ≤ 30 minutes), minimal (30 minutes), average (> 30 to < 60 minutes), complex (≥ 60 to < 120 minutes), and super complex (≥ 120 to 240 minutes). The time reported in the eligibility screening process may be modified over the course of the evaluation process.

Key activities performed as part of the effort (time interval) include: review of the patient medical record to determine eligibility for the specific study, discussions with patient and family about the study, and time associated with efforts to obtain medical records from other locations where the patient may have received care.

Statistical Approach

Summary statistics of patients and evaluations according to the analytic variables were reported with frequencies and proportional distributions using IBM SPSS Statistics (version 19; SPSS, Chicago, IL). Only evaluations without missing demographics or effort were included in this analysis. These descriptive characteristics were based on the 3,467 complete evaluations.

Reporting on the direct measure of effort and time based on the variables listed is insufficient because of the potentially confounding relationship between many of the variables of interest (such as research division and CT characteristics) and the outcome measure of effort based on time. The use of statistical modeling permits us to control for key variables that may be collinear with our outcomes of interest. We therefore estimated the effort per enrolled patient as the probability of enrollment status for each patient (using a generalized multinomial model) and the probability of being in each effort (time interval) category (using a continuation ratio ordinal logistic model). Patient enrollment and CRS effort are both functions of patient characteristics (age group, race, and sex) and other factors (evaluation order, where the patient was identified, clinical research division, and study type and phase). Therefore, to account for the correlation between these two estimated probabilities, we simultaneously evaluated them and used them at the patient level to estimate the expected effort per enrolled patient. Separate models were used to estimate the effort for evaluations for therapeutic and nontherapeutic studies. All variables were tested, and only those variables that produced the most parsimonious and best-fit models were included to predict probability of enrollment and the probabilities of effort for observational studies and therapeutic studies separately. The effect and significance of the covariates differ for each of these models. The nonsignificant covariates were removed when evaluating models to provide the best fit. These data were used to calculate the average number of hours spent on eligibility screening for each type (or phase) of CT for each enrollment category. Finally, we estimated the effort as average time spent on evaluating patients to successfully enroll a single patient in each type of CT.

Estimation of Cost

Only costs associated with personnel (research nurses and clinical research associates) were estimated in this study. Cost for eligibility determination per enrolled patient was attributed according to the average salary for a research nurse or clinical research associate at our institution. Hourly cost was allocated based on an 80% attribution of effort associated with eligibility screening by research nurses and 20% by CRS. We assumed that there were 2,080 work hours per year for each staff member. Hours were assigned to each category of effort as the median time for that interval. During the study period, an additional effort category was introduced into CTED, named convoluted, with time allocation greater than 240 minutes. This category was a selection option for a period of 3 months (May to July 2011). Fifteen evaluations (0.4%) had this effort category selected. We allocated 180 minutes to convoluted and combined it with the super complex category. This provided a conservative estimation of the total time for eligibility screening.

Using the effort as the average number of hours spent on eligibility screening to enroll a single patient (T̂_phase) obtained from these models for each study phase or type, we then estimated the average cost of enrolling one patient in a CT as:

This estimated cost per enrolled patient was then used in combination with the number of enrolled patients and evaluations from CTED to calculate the total cost of screening for each CT phase during the study period as well as the annualized total eligibility screening cost to the cancer center.

Accessing of Data

Institutional review board approval was obtained for this study under a waiver of informed consent from Virginia Commonwealth University.

Results

There were 3,467 complete evaluations during the 18-month study period, representing 2,315 patients screened for eligibility for 130 open CTs. Fifty-seven percent were evaluations for therapeutic studies, and 38% were for observational studies; 4.8% had no study associated with the evaluation. There were 727 enrollments, with 67.5% in observational studies. Table 1 provides the distribution of evaluations according to patient characteristics. Patients in the age group of 51 to 64 years, women, and white patients were more likely to have at least one evaluation. Most patients had one evaluation (67.3%), but 4.1% had four to nine evaluations.

Table 1.

Distribution of Patients and Frequency of Evaluations by Patient Demographic Characteristics

Characteristic	Patients With One Evaluation		Patients With Two Evaluations		Patients With Three Evaluations		Patients With > Three Evaluations		Total (N = 2,315)
	No.	%	No.	%	No.	%	No.	%	No.	%
Age category, years
19-45	206	69.4	61	20.5	15	5.1	15	5.1	297	12.8
46-50	146	64.6	57	25.2	10	4.4	13	5.8	226	9.8
51-64	654	64.9	237	23.5	76	7.5	41	4.1	1,008	43.5
65-75	400	69.1	117	20.2	39	6.7	23	4.0	579	25.0
> 75	152	74.1	42	20.5	7	3.4	4	2.0	205	8.9
Sex
Male	820	66.0	281	22.6	88	7.1	53	4.3	1,073	46.3
Female	738	68.8	233	21.7	59	5.5	43	4.0*	1,242	53.7
Race
White	885	67.4	293	22.3	84	6.4	51	3.9	1,313	56.7
Black	602	66.4	201	22.2	60	6.6	44	4.9	907	39.2
Other/unknown	71	74.7	20	21.1	3	3.2	1	1.1	95	4.1
Total	1,558	67.3	514	22.2	147	6.3	96	4.1

The distribution of effort category by evaluation outcome is summarized in Table 2. There is an association between evaluation outcome and time required for the evaluation process, with increasing effort for enrolled patients (Pearson χ² = 1617.5; P < .001).

Table 2.

Distribution of Effort According to Nonpatient Characteristics

Characteristic	Evaluations by Effort Category (minutes)												Total (N = 3,467)
	< 10		≥ 10 to < 30		30		≥ 30 to < 60		≥ 60 to 120		≥ 120 to 240
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%*
Research division†
Cancer control	12	7.0	46	26.7	27	15.7	57	33.1	27	15.7	3	1.7	172	5.0
Gynecologic oncology	2	1.6	36	27.9	36	27.9	20	15.5	17	13.2	18	14.0	129	3.7
Hematologic malignancies	58	8.3	352	50.1	94	13.4	65	9.2	37	5.3	97	13.8	703	20.3
Medical oncology	107	13.7	394	50.5	129	16.5	52	6.7	23	2.9	75	9.6	780	22.3
Breast cancer	13	4.5	72	24.9	57	19.7	68	23.5	57	19.7	22	7.6	289	8.3
Palliative care	106	10.6	255	25.4	151	15.1	169	16.8	201	20.0	121	12.1	1,003	28.9
Radiation oncology	6	1.5	86	22.0	80	20.5	100	25.6	100	25.6	19	4.9	391	11.3
Study type‡
Phase I	43	9.5	214	47.3	81	17.9	48	10.6	21	4.6	45	10.0	452	13.0
Phase II	57	10.2	163	29.1	110	19.6	99	17.7	64	11.4	67	12.0	560	16.2
Phase III	73	7.6	377	39.0	178	18.4	134	13.9	105	10.9	99	10.2	966	27.9
Observational	119	9.0	349	26.4	199	15.1	243	18.4	269	20.3	143	10.8	1,322	38.1
Not applicable	12	7.2	138	82.6	6	3.6	7	4.2	3	1.8	1	0.6	167	4.8
Identification source§
Electronic data	141	9.8	575	39.8	196	13.6	186	12.9	209	14.5	126	8.7	1,443	41.6
Multidisciplinary clinic	110	10.8	440	43.1	175	17.2	124	12.2	76	7.5	95	9.3	1,020	29.4
Referral	53	5.2	226	22.3	203	20.0	221	21.8	177	17.5	134	13.2	1,014	29.2
Evaluation outcome‖
Enrolled	3	0.4	74	10.2	30	4.1	92	12.7	296	40.7	232	31.9	727	21.0
Refused	27	6.1	101	22.9	91	20.6	146	33.1	33	7.5	43	9.8	441	12.7
Ineligible	274	11.9	1,066	46.4	453	19.7	293	12.7	133	5.8	80	3.5	2,299	66.3
Total	304	8.8	1,241	35.8	574	18.6	531	15.3	462	13.3	355	10.2

^*Percent equals column % for that variable.

^†χ² = 594.2; P < .001.

^‡χ² = 335.6; P < .001.

^§χ² = 203.6; P < .001.

^‖χ² = 1617.5; P < .001.

Table 2 also represents the distribution of the proportion of evaluations according to the level of effort (columns) and by nonpatient characteristics. The largest proportion of evaluations (35.8%) required 10 to 30 minutes, but more than 10% required between 2 to 4 hours for completion. There were significant differences in the distribution of effort by time interval for research division (χ² = 594.2; P < .001), study type (χ² = 335.6; P < .001), and source from which the patient was identified (χ² = 203.6; P < .001).

The estimated time per evaluation and per enrollment and the associated cost are listed in Table 3. There was substantial variation in cost according to study type and within therapeutic studies by phase. Observational studies had the lowest eligibility screening costs ($129 per enrollment), and phase I studies had the highest, at $336 per enrollment. The average number of patients screened per enrolled patient ranged from three for observational studies to 13 for phase I studies. The total estimated annual cost spent on the eligibility screening process for our center was $90,505.

Table 3.

Time and Cost Associated With Eligibility Evaluations by Study Type

Study Type	No. of Evaluations	No. of Enrollments	Mean No. of Patients Screened per Enrollment	Hours per Evaluation		Hours per Enrollment		Cost per Enrollment*	Total Cost of Screening in Study Period†	Annualized Total Cost of Screening per Year
				Mean	SD	Mean	SD
Phase I	490	39	13	0.74	0.26	8.78	4.50	$336.48	$13,122.90	$8748.60
Phase II	639	92	7	0.87	0.26	6.33	3.94	$242.59	$22,318.36	$14,878.91
Phase III	1,099	132	8	0.84	0.25	7.55	4.60	$289.35	$38,193.70	$25,462.47
Observational	1,510	518	3	0.94	0.22	3.37	3.38	$129.15	$62,122.05	$41,414.70
Total	4,005‡	781	5						$135,757.02	$90,504.68

Abbreviation: SD, standard deviation.

^*Cost per enrollment = (average hours per enrolled patient associated with eligibility screening for that phase) × ([0.8 × research nurse hourly salary] + [0.2 × clinical research associate hourly salary]).

^†Study period was 18 months.

^‡267 evaluations were not associated with a specific study.

Discussion

Identifying eligible patients represents one of the most challenging and important components in the CT accrual process, yet most sponsors typically do not reimburse for the eligibility screening process. Screening usually occurs before consent and includes activities such as reviewing medical records and obtaining information from other institutions and providers. In this study, we prospectively captured time spent on screening in the context of patient eligibility, demographic characteristics of the patient, and other CT characteristics. The average time spent to find, screen, and enroll a patient varied from 3.4 to 8.8 hours, and cost varied from $129 to $336, respectively. These costs of screening represent 6.5% to 16.7% of the average total amount reimbursed per enrolled patient (based on a $2,000 reimbursement per enrolled patient for federally sponsored trials). The total cost of screening to an institution is substantial, at $90,505 annually for our cancer center. Depending on the number of enrollments and study types (eg, more phase I studies), cost to an individual institution may be substantially higher. The proportion of federally versus industry-sponsored trials at an institution will also affect the proportion of reimbursement per enrolled patient that is used in prescreening, because this varies by sponsor,^24,25 with the lowest reimbursement for government-sponsored research. At our cancer center, 85% of CTs are either government sponsored or locally initiated. Therefore, the dollars spent on prescreening represent a larger proportion of the total reimbursement for a CT, compared with cancer centers with a greater proportion of industry-sponsored trials.

The screening process for cancer therapeutic trials may be more time consuming and labor intensive because of the detailed clinical requirements associated with these studies and the involvement of professional staff (oncology research nurses), resulting in higher costs. The pool of potential participants in cancer therapeutic trials is specific; therefore, recruitment efforts mainly target patients diagnosed and/or treated at the institution or practice of the principal investigator. Thus, the published literature reporting on cost of recruitment strategies and the staff effort and cost are not applicable to cancer trials.

The published literature estimates screening costs from $120 to $2,508 per enrolled patient.^{12,13,19,26–28} The difference compared with our results is likely the result of differences in cost estimation methods used in the published literature, such as retrospective data capture, inclusion of nontherapeutic trials, and inclusion of recruitment-related costs (eg, advertising). Our cost estimates may also be slightly lower because there may be some efficiencies in the accrual and screening process at cancer centers through patient identification in multidisciplinary clinics. Furthermore, physicians at a cancer center may be more active in patient recruitment.

Several advantages in methodology are represented in this report. Of the few studies focusing on cost and effort associated with recruitment, most relied on retrospective analysis of cost, with data collected either at the study end or after the recruitment process was completed.^13,19 Our study captured these data prospectively, and although the data are based on the recording of time spent by the research staff to determine eligibility, the near real-time data capture reduces potential under- or over-reporting resulting from recall errors. This study also reports the time spent for eligibility screening across a large number of trials and study types (ie, different phases and 130 CTs for many cancer sites). The majority of the published literature focused on a single or limited set of studies.^{13,14,26–28} Finally, we used statistical methods that accounted for the ordinal nature of the effort and the multinomial distribution of the evaluation outcome, accounted for correlations in effort and evaluation outcome according to patient characteristics and other factors, and adjusted for multiple evaluations per patient. This enhances the reliability of our estimates of time and cost.

Recent publications on the current practice of CTs have demonstrated a clear trend toward under-reimbursement for study-related activities. The reimbursement for cancer trials has not changed since 1999, representing a 20% decrease when adjusted for inflation.²⁵ Although the average reported profit margin for sites is approximately 2%,²⁹ increasing protocol complexity and regulatory and administrative requirements likely reduce even this profit margin. For example, between 1995 and 2000, one hospital was able to break even; however, by 2005, the hospital subsidized $1.29 for every dollar of revenue from research.²⁹ Data on screening costs could be used in contract negotiations that take into account variation in cost for type of study (ie, phase I studies).

There are some limitations to our study. The costs involved in eligibility assessment included only the costs attributable to personnel and did not include all the costs for accrual, such as physician involvement, advertising, and so on. For cancer therapeutic CTs, physician involvement is extensive because of the protocol complexity and critical need for physician involvement in referral. Thus, the study may have underestimated the total cost for trial accrual. Underestimation might also have occurred based on use of the mean for the category with the greatest effort. In discussions with the research nurses, some evaluations may have taken as long as 9 hours.

In summary, the cost of recruiting patients to CTs is high, with most studies not fully covering the cost of the trial itself.^24,25,29 To our knowledge, this is the first study that prospectively captured the amount of time spent and estimated the largely nonreimbursed cost of eligibility screening. The cost associated with the eligibility screening process is highly variable based on factors related to study type and complexity and patient characteristics. Eligibility screening can be a significant financial burden to an institution, especially if the institution is performing early-phase CTs. Information such as that captured here could serve as a point of negotiation with sponsors to reduce the gap between reimbursed and unreimbursed costs.

Authors' Disclosures of Potential Conflicts of Interest

The author(s) indicated no potential conflicts of interest.

Author Contributions

Conception and design: Lynne T. Penberthy, Bassam A. Dahman

Administrative support: Valentina I. Petkov

Collection and assembly of data: Lynne T. Penberthy, Valentina I. Petkov

Data analysis and interpretation: All authors

Manuscript writing: All authors

Final approval of manuscript: All authors