Abstract
Rationale: The randomized clinical trial has been an important tool for expanding our knowledge of disease. This study is the first to compare trial participants to the entire eligible population.
Methods: We performed a cohort analysis using data from the Cystic Fibrosis Foundation Registry database between 1992 and 1998.
Measurements and Main Results: There were 8,735 patients older than 6 yr followed for the entire period. Of the patients, 2,635 patients (30.2%) were enrolled in at least 1 of 32 Institutional Review Board–approved clinical trials, with an average annual participation rate of 7%. Patients enrolled in clinical trials had more advanced disease as judged by FEV1% predicted (68 vs. 77%, p < 0.001), higher rates of Pseudomonas aeruginosa infection (71 vs. 65%, p < 0.01), and were more likely to have private insurance (odds ratio [OR], 1.25; 95% confidence interval [CI], 1.14–1.37) and be white (OR, 1.98; 95% CI, 1.44–2.70). No sex differences were noted. Despite the worse clinical status at baseline, clinical trial participants had a lower average annual rate of decline in lung function (1.33%/yr; 95% CI, 1.20, 1.46; compared with 1.52%; 95% CI, 1.43–1.60).
Conclusions: These results show that the overall participation rate is very high. Despite more advanced disease at baseline, lung function decline was lower in trial participants; the cause of this difference is unclear. The differences seen in insurance status are concerning. Efforts should be made to ensure adequate representation from different social demographic groups.
Keywords: clinical trial, cystic fibrosis, generalizability, participation
Cystic fibrosis (CF) is one of the most common inherited fatal diseases in whites, with a reported incidence from 1 in 2,000 to 1 in 3,200 live births (1). Advancing the care of patients with CF requires the continued search for more efficacious treatments that can potentially modify the natural history of the disease and improve symptom control and prognosis. The randomized clinical trial has been a vital tool for expanding our knowledge of disease and response to therapies and is considered the gold standard for assessing the effectiveness of various therapies and interventions.
Important differences may exist between those eligible but not participating compared with those who participate, which limits the generalizability of the trial results. Differences between those who participate in trials compared with those who refuse to participate have been reported in the literature, but rarely has the entire eligible population for the trial been known (2–5). These studies have compared those patients who participated and those who declined participation without comparing them to the entire eligible population. It is also well known that the proportions of patients eligible for clinical trials may substantially exceed those that actually enter a given trial (6). There are many reasons for lack of participation in clinical trials. Patients worry about side effects, uncertainty of therapy, consent, and the additional demands of the trials themselves (7). Patients may not be offered an opportunity to participate because research studies are not conducted at a geographic location near to the study site or the provider may deem the subject an inappropriate candidate (8). Thus, numerous types of barriers may limit participation in clinical trials: financial, social, intellectual, geographic, and educational. Resultant selection biases may skew study results and affect study interpretation.
There is no published literature about the differences between patients with CF who participate in clinical trials and those who do not. For this reason, we analyzed data from the Cystic Fibrosis Foundation's Patient Registry (CFFPR) to investigate whether patients who participate in clinical trials differ from those who do not participate in clinical trials, and whether participants in clinical trials have different clinical outcomes than do nonparticipants. The phase 3 intermittent administration of inhaled tobramycin was used as a case study in the analyses. There are few patient registries in the United States that enroll a vast majority of all patients with a particular disease. The CFFPR captures more than 85% of the CF population in the United States (9). For this reason, these analyses may have broader implications for clinical researchers attempting to obtain representative populations in their studies. For the first time, the denominator in addition to the numerator is known. Some of the results of these studies have been previously reported in the form of an abstract (10).
METHODS
Study Subjects
The CFFPR contains data collected annually from participants at U.S. CFF-accredited centers (9). The data are anonymous and without patient identifiers. We defined several study populations. Cohort 1 included all patients in the CFFPR in 1992 who were older than 6 yr in 1992 (Figure 1). Cohort 2 included all patients in the CFFPR in 1992 who were older than 6 yr in 1992 and who did not die or become lost to follow-up in the subsequent 7 yr. Cohort 3 included a subset of subjects who fulfilled the age and lung function entry criteria for the phase 3 intermittent administration of inhaled tobramycin (> 6 yr of age and FEV1 > 25% of predicted and < 75% of predicted) and were followed in the registry in 1995 and 1996. This cohort was chosen to assess the impact of participation in one of the major studies with both upper and lower lung function entry criteria during the period. The risk period for Cohorts 1 and 2 was defined as the 7 yr of follow-up of the cohort (1992–1998). The study was approved by the CFF and Institutional Review Board (IRB) at the University of Washington, Seattle.
Figure 1.
Diagram of different cohorts that study used.
Study Design
The design was a cohort study. A priori, the predictors of trial participation were as follows: sex, race, insurance status, highest level of education attained, marital status, and employment status. Highest level of education attained was also specified by age (i.e., high school education in those > 18 yr old). The primary outcome was participation in at least 1 of 32 IRB-approved clinical trials during 7 yr (1992–1998). Clinical trial participation was defined as participation in an IRB- approved research trial. Each site picked from a list of CFF-funded and industry-sponsored studies known to be actively enrolling during the time of data collection. For the purpose of this analysis, only those subjects who participated in studies that clearly involved a therapy or device were defined as clinical trial participants. Participation in the Epidemiologic Study of Cystic Fibrosis, the only observational study noted in the CFFPR during this period, was not treated as a clinical trial participation. Insurance status was categorized as commercial insurance, government-funded insurance, uninsured, or unknown. Lung function was assessed using FEV1 and FVC. Percent predicted for both FEV1 and FVC was calculated using reference equations by Knudson and colleagues (11). CF pulmonary exacerbation was defined as a CF-related pulmonary condition requiring admission to the hospital or use of home intravenous antibiotics. Weight percentiles were based on current age-specific values from the U.S. population (http://www.cdc.gov/nccdphp/dnpa/growthcharts/). Pancreatic insufficiency was assessed by use of pancreatic enzymes. Pediatric subjects were defined as those younger than 18 yr.
Statistical Analysis
Continuous variables were compared between groups using Student's two-sample t test or Mann-Whitney rank-sum tests as appropriate. χ2 tests were used to compare nominal data. A two-sided p value of less than 0.05 was considered statistically significant.
The primary goal of this analysis was to assess the associations between the predictors of interest and the outcome (trial participation). Logistic regression models were used to estimate the odds of clinical trial participation during the follow-up period. Model fit for logistic regression was assessed with the Hosmer-Lemeshow statistic (12). In each case, influential data points and outliers were assessed. Observations with missing data were excluded from the analysis. Lung function was also an important clinical outcome variable; change in lung function was initially assessed by comparing absolute change among groups. Repeated-measures linear regression (13) models were used to assess the effect of trial participation study on FEV1% predicted over time employing the Huber/White estimator of variance. Each patient contributed a maximum of seven measurements (best FEV1% predicted in each year of observation), and the effect of time on lung function was captured in the model using time in calendar years. An interaction term between participation status and time provided an estimate of the average annual rate of lung function decline (in FEV1% predicted) associated with trial participation. A subsequent analysis compared patients who fulfilled eligibility criteria for the phase 3 intermittent administration of inhaled tobramycin in patients with CF (14), but were not enrolled compared with those patients who were enrolled. Analyses were performed with SAS 8.2 (SAS Institute, Inc., Cary, NC) and Stata 7.0 (StataCorp, College Station, TX).
RESULTS
Cohort Description
There were a total of the 13,041 subjects with CF in the registry in 1992 who fulfilled entry criteria for this study (cohort 1). There were a total of 3,780 (29%) of the initial 1992 cohort of 13,041 subjects with CF (cohort 1) who participated in at least one IRB-approved trial. Of the cohort 1 subjects, 8,735 subjects were still noted to be in the CFFPR in 1998 (cohort 2). A total of 2,238 subjects died during the follow-up period (1992–1998). A total of 2,068 patients were lost to follow-up during this period or had no evaluable data in 1998. Of the 8,735 subjects in the final cohort, 2,635 (30%) participated in at least one IRB- approved trial. The average annual rate of participation was 7% during the study period. There was a total of 6,100 (70%) of the cohort 2 who did not participate in a trial during the follow-up period. Overall clinical trial participation for all subjects in the registry by year demonstrates significant variation in numbers of subjects per year (Figure 2), with peaks of subject participation in 1992 to 1993 and 1995, the enrollment periods of two major randomized clinical trials in CF, recombinant human DNase and inhaled tobramycin phase 3 trials (14, 15).
Figure 2.
Comparison of annual trial participation rate: all patients versus final cohort.
Characteristics of the cohort 2 study population of 8,735 patients are described in Table 1 along with the characteristics of the subjects who participated in clinical trials and those who did not participate. Subjects who were involved in clinical trials were more likely to be older, have commercial health insurance, be white, be colonized with Pseudomonas aeruginosa, have worse lung function, and have more office visits. The absolute differences between the two groups, those who participated in clinical trials and those who did not participate in clinical trials, were small except in regard to lung function, P. aeruginosa detection, and median number of office visits. Importantly, gender did not differ between participants and nonparticipants (47 and 46% female subjects, respectively; p = 0.53). We also found no difference in age at diagnosis between nonparticipants and participants (median age 3 vs. 2.5 mo, p = 0.24). Increasing time from diagnosis to cohort entry was associated with study participation (mean age, 12.8 yr in nonparticipants vs. 14.2 yr in participants; p < 0.01). Because excluding subjects who died and were lost to follow-up could bias the results, the data were analyzed after including all those subjects who died or were lost to follow-up (cohort 1 [n = 13,041]). Inclusion of subjects who died or were lost to follow-up yielded similar results; trial participants were more likely to be older and have markers of more advanced disease (lower lung function and P. aeruginosa infection).
TABLE 1.
STUDY POPULATION, CLINICAL TRIAL PARTICIPANTS COMPARED WITH NONPARTICIPANTS
| Characteristic | Trial Participants | Trial Nonparticipants | p |
|---|---|---|---|
| n | 2,635 | 6,100 | |
| Age, mean (± SD) | 16.6 (9.2) | 15.1 (8.4) | < 0.01 |
| Sex, n (% female) | 1,233 (47) | 2,810 (46) | 0.53 |
| Genotype | |||
| Homozygous for Δ F508, n (%) | 983 (54) | 1,961 (54) | < 0.01 |
| Heterozygous for Δ F508, n (%) | 681 (38) | 1,299 (36) | < 0.01 |
| Nonwhite, n (%) | 49 (2) | 220 (4) | < 0.01 |
| Commercial health insurance, n (%) | 1,532 (58) | 3,218 (53) | < 0.01 |
| Mean height percentile (SD) | 34.5% (26.8) | 33.7% (26.7) | 0.21 |
| Median weight percentile (interquartile range) | 19.0% (6,45) | 23.0% (7,48) | < 0.001 |
| FEV1, L, mean (± SD) | 1.69 (0.80) | 1.83 (0.86) | < 0.001 |
| FEV1% predicted (mean ± SD) | 68% (26) | 77% (25) | < 0.001 |
| FVC, L, mean (± SD) | 2.41 (1.09) | 2.46 (1.15) | 0.06 |
| FVC % predicted, mean (± SD) | 81% (22) | 88% (22) | < 0.001 |
| Pancreatic insufficiency, n (%) | 2,488 (95) | 5,728 (94) | 0.27 |
| Median annual outpatient visits in 1992 (interquartile range) | 5 (3–10) | 4 (2–5) | < 0.001 |
| Colonized by Pseudomonas aeruginosa, n (%) | 1,710 (71) | 3,216 (65) | < 0.01 |
| Colonized by Burkholderia cepacia, n (%) | 72 (3) | 112 (2.1) | 0.025 |
Characteristics were assessed at cohort entry. Total population: n = 8,735.
Tables 2 and 3 shows the cohort 2 stratified by age in 1992 (⩾ 18 and < 18 yr of age). The age cutoff of 18 yr was used because it represents the age at which subjects can individually provide consent for clinical trial participation. We found that 5,808 (66%) of the subjects were younger than 18 yr at the initiation of the cohort. When comparing characteristics associated with clinical trial participation in adults and pediatric subjects, similar results were found compared with cohort 1. The primary exceptions were that differences in height and weight seen in the overall cohort and in the adults were not observed in the pediatric group. The clinical trial participants continued to have worse lung function (as measured by percentage of predicted) and more annual clinic visits.
TABLE 2.
PEDIATRIC STUDY POPULATION IN 1992, CLINICAL TRIAL PARTICIPANTS COMPARED WITH NONPARTICIPANTS
| Pediatric
|
|||
|---|---|---|---|
| Characteristic | Trial Participants | Trial Nonparticipants | p |
| n | 1,587 | 4,221 | |
| Age, mean (± SD) | 10.7 (3.3) | 10.7 (3.3) | 0.66 |
| Sex, n (% female) | 764 (48.1) | 1,974 (46.8) | 0.35 |
| Nonwhite, n (%) | 34 (2.1) | 164 (3.9) | 0.001 |
| Commercial health insurance, n (%) | 919 (58.0) | 2,216 (52.6) | < 0.001 |
| Mean height percentile (SD) | 32.0% (26) | 31.8% (26) | 0.76 |
| Median weight percentile (interquartile range) | 25.0% (8,52) | 26.0% (9,52) | 0.26 |
| FEV1, L, mean (± SD) | 1.50 (0.66) | 1.64 (0.74) | < 0.001 |
| FEV1% predicted, mean (± SD) | 77.5% (24.2) | 83.6% (22.7) | < 0.001 |
| FVC, L, mean (± SD) | 1.94 (0.84) | 2.07 (0.92) | < 0.001 |
| FVC % predicted, mean (± SD) | 87.8% (21.1) | 91.7% (20.1) | < 0.001 |
| Median annual outpatient visits in 1992 (interquartile range) | 5 (3–9) | 4 (2–5) | < 0.001 |
Total population: n = 5,808.
TABLE 3.
ADULT STUDY POPULATION IN 1992, CLINICAL TRIAL PARTICIPANTS COMPARED WITH NONPARTICIPANTS
| Adult
|
|||
|---|---|---|---|
| Characteristic | Trial Participants | Trial Nonparticipants | p |
| n | 1,048 | 1,879 | |
| Age, mean (± SD) | 26.4 (6.9) | 25.8 (6.8) | 0.02 |
| Sex, n (% female) | 469 (44.5) | 836 (44.8) | 0.89 |
| Nonwhite, n (%) | 15 (1.4) | 56 (3.0) | 0.01 |
| Commercial health insurance, n (%) | 613 (58.6) | 1,002 (53.4) | 0.01 |
| Mean height percentile (SD) | 21.6% (21.2) | 24.1% (23.0) | 0.003 |
| Median weight percentile (interquartile range) | 12.0% (5,32) | 15.0% (5,36) | 0.01 |
| FEV1, L, mean (± SD) | 1.98 (0.88) | 2.24 (0.97) | < 0.001 |
| FEV1% predicted, mean (± SD) | 54.8% (22.3) | 62.0% (24.8) | < 0.001 |
| FVC, L, mean (± SD) | 3.10 (1.05) | 3.34 (1.12) | < 0.001 |
| FVC % predicted, mean (± SD) | 73.4% (20.7) | 78.9% (22.2) | < 0.001 |
| Median annual outpatient visits in 1992 (interquartile range) | 5 (3–12) | 3 (2–5) | < 0.001 |
Total population: n = 2,927. Characteristics were assessed at cohort entry for both groups.
Characteristics Associated with Trial Participation
Using univariate logistic regression to compare social and demographic characteristics between study participants and nonparticipants in cohort 2, the odds ratio (OR) for trial participation in any clinical trial was 1.22 (95% confidence interval [CI], 1.11–1.34) for patients who had commercial or private health insurance compared with subjects with noncommercial health insurance (Figure 3). Being a nonwhite with CF was associated with nonparticipation (OR, 0.54; CI, 0.4–0.74); 18.2% of nonwhites participated compared with 30.5% of whites who participated. When looking solely at subjects who were older than age 18 at study entry, being employed full time was associated with study participation when compared with not working (OR for participation, 1.26; CI, 1.04–1.53). In this same subpopulation (> 18 yr), having at least a high school degree was associated with study nonparticipation (OR for participation, 0.7; 0.25–0.89) compared with those subjects without a high school degree. When both education and employment were evaluated in the same regression model, ORs did not change. Marital status was not associated with trial participation and was not kept in the model. We found no statistically significant interactions between educational status, employment, and marital status. We also found no significant interaction between marital status and insurance status in relation to trial participation. The subjects with the highest odds of participation were those without a high school education who worked full time (OR for participation, 1.26; CI, 1.04–1.53) when compared with those who did not work and did not hold a high school degree. Those subjects with a high school education who did not work had the lowest odds of participation (OR, 0.45; CI, 0.24–0.85) when compared with those who did not work and did not hold a high school degree. Those subjects who worked full time and had a high school education had a nonsignificant decreased odds of trial participation (OR, 0.56; CI, 0.29–1.09). In the population older than 25 yr, the highest level of education was not associated with study participation; having a college education or higher degree was not associated with clinical trial participation when compared with those subjects with only a high school degree (OR, 0.87; CI, 0.35–2.17). Neither marital status nor employment status in the population older than 25 yr was also not associated with trial participation.
Figure 3.
Associations between socioeconomic/demographic characteristics and trial participation. †Odds ratio (OR) compares nonwhite to white patients; ‡OR compares patients older than 18 yr with commercial insurance to those with noncommercial insurance; ¥OR compared patients who are married with those who are single; ¶OR compares patients older than 18 yr who are employed compared with those who are unemployed; *OR compares patients older than 18 yr with a high school degree with those without a degree; ∈OR compares patients older than 25 yr with a college degree or higher with those without a high school degree.
Clinical Outcome of Trial Participants
Lung function change was only assessed in the subjects in the primary cohort who had pulmonary function measurements obtained both in 1992 and 1998, a subset of cohort 2. The highest value for FEV1 and FVC from those 2 yr was used in the analysis. Change in mean FEV1% predicted differed between subjects who participated in clinical trials and those who did not. Trial participants had a 7.6% absolute drop in FEV1% of predicted, whereas trial nonparticipants had a 9.0% absolute drop (p = 0.01) from 1992 at study entry to 1998 (Table 4). Mean absolute change in FVC percent predicted fell 6.2% in trial participants compared with 6.0% in trial nonparticipants (p = 0.72) over the 7-yr study period. Thus, progression of lung disease appeared to be less in trial participants compared with trial nonparticipants despite having poorer lung function at the beginning of the cohort.
TABLE 4.
MEAN FEV1% PREDICTED IN 1992 COMPARED WITH 1998
| Mean FEV1% Predicted (SD)
|
|||
|---|---|---|---|
| Group | 1992 | 1999 | Mean Absolute Change (% predicted) |
| Trial participants, n = 2,366 (90%) | 68% (26) | 62% (26) | 7.6% (21)* |
| Trial nonparticipants, n = 5,196 (85%) | 77% (25) | 69% (26) | 9.0% (21)* |
Comparison of participants with nonparticipants, p = 0.01.
We then assessed the rate of lung function decline in trial participants compared with nonparticipants using all of the available lung function data in the intervening years between 1992 and 1998 using generalized estimating equations. The annual rate of decline in lung function was assessed by evaluating an interaction between time and trial participation status. On average, during the study period, trial participants had an FEV1% predicted that was 7.6% (95% CI, 6.6–8.7%) less than the trial nonparticipants with an absolute annual rate of lung function decline in percent predicted during the study period of 1.46% (95% CI, 1.39–1.53). The average number of annual lung function values reported per patient during the 7-yr period was 6.6. When annual rate of decline of lung function was assessed using repeated-measures analysis, FEV1% predicted decreased at a slightly lower rate in trial participants than nonparticipants (1.33%/yr; 95% CI, 1.20–1.46) when compared with trial nonparticipants (1.52%; 95% CI, 1.43–1.60; p = 0.02). We found no change in the point estimate or CIs when adjusting for baseline FEV1% predicted and age in years. We then adjusted for baseline number of annual office visits, hospital admissions, and pulmonary exacerbations. No differences in the estimate for the average rate of decline of FEV1% predicted were seen when baseline pulmonary exacerbation rate and hospitalization rate were added to the model. The effect of clinical trial participation on lung function decline was attenuated and became nonsignificant after adding the annual number of office visits in the model, suggesting that baseline access to care may explain the differential health outcomes noted between clinical trial participants and nonparticipants.
Phase 3 Trials
We then looked at a subset of subjects who fulfilled entry criteria for the phase 3 intermittent administration of inhaled tobramycin in patients with CF but did not participate and compared them with those patients who participated as a case study, n = 6,168 (cohort 3). Using the CFFPR, we were able to identify 404 (78%) of the original 520 patients who were enrolled in the trial. Using the age cutoff for trial inclusion (⩾ 6 yr) and lung function as measured with FEV1 ⩾ 25% of predicted and ⩽ 75% of predicted followed in the registry in 1995 and 1996 (the years of enrollment for the trial) and a respiratory culture positive for P. aeruginosa, we found a comparison group of 5,764 who were not in the study. In the study population, 45.5% were female compared with the comparison group of 47.8% (p = 0.39). We found no differences in race or government-sponsored insurance. We also found no differences in rates of pancreatic insufficiency (94.1 vs. 93.7%, p = 0.77) and CFTR genotype. Mean age (20.3 and 20.9 yr, p = 0.9) did not differ between the trial participants and nonparticipants. We did find small and nonsignificant differences in mean FEV1% predicted (49.2 and 51.1%, p = 0.18), with trial participants having slightly lower lung function with no difference in mean weight percentile (http://www.cdc.gov/nccdphp/dnpa/growthcharts/; 18.9 and 18.8%, p = 0.11) and mean age of diagnosis (3.4 and 3.5 yr, p = 0.9). Subjects who participated in the tobramycin study had more clinic visits (6 interquartile range [4,9] compared with 4 IQR [3,7], p < 0.01), but not total hospitalizations (1 IQR [0,2] compared with 1 IQR [0,2], p = 0.19) than those who did not participate.
Assessment of Bias
The majority of clinical trials in CF have inclusion and exclusion criteria that limit patients based a lower cutoff of lung function of 25 to 40% of predicted (inhaled recombinant human DNase, phase 3, excluded subjects with an FEV1% predicted < 40% of predicted) (14, 15). One major clinical trial during this period (inhaled tobramycin phase 3 trial [14]) had exclusion criteria for subjects with lung function less than 25% of predicted and greater than 75% of predicted. The entry criteria for these studies are included in Table 5. Thus, a subcohort of subjects with an FEV1% predicted of 25% or greater and 75% or less was used to assess how the lung function inclusion criteria might affect the results. Clinical trial participants were still more likely to have commercial insurance and be white; no sex differences were found. Trial participants were also more likely to be culture positive for P. aeruginosa (80.6 compared with 75.0%, p < 0.01), have worse lung function (FEV1% predicted of 52.9 compared with 55.5%, p < 0.01), and more office visits (median of five compared with a median of four, p < 0.01).
TABLE 5.
INCLUSION AND EXCLUSION CRITERIA FOR THE INHALED RECOMBINANT HUMAN DEOXYRNase, PHASE 3 STUDY, AND THE INHALED TOBRAMYCIN PHASE 3 STUDY
| Deoxy RNase, phase 3 study (15) | |
| Inclusion criteria | a) 5 yr of age or older |
| b) A confirmed diagnosis of cystic fibrosis (a sweat chloride value higher than 60 mmol/L) | |
| c) An FVC > 40% of the predicted value (based on sex, age, and height) | |
| d) To ensure that patients were enrolled when they were clinically stable, they had to have been receiving a consistent regimen of antibiotics, or no antibiotics, during the 14 d before randomization | |
| Tobramycin, phase 3 study (14) | |
| Inclusion criteria | a) 6 yr of age or older |
| b) A documented diagnosis of cystic fibrosis | |
| c) A respiratory tract culture yielding Pseudomonas aeruginosa | |
| d) An FEV1 that was > 25%, but no more than 75% of the predicted value | |
| e) Ability to perform reproducible pulmonary function tests | |
| Exclusion criteria | a) Receipt of antipseudomonal antibiotics within the previous 2 wk |
| b) Known hypersensitivity to aminoglycosides | |
| c) Compromised renal function (serum creatinine level, 2 mg/dl [177 mmol/L]) | |
| d) Recovery of Burkholderia cepacia from the respiratory tract within the previous 2 yr |
We also limited the cohorts to those subjects who participated in studies other than the inhaled recombinant human DNase, phase 3 study, and the inhaled tobramycin phase 3 study. We found very similar results. In this subset of patients, we found no sex difference between clinical trial participants and nonparticipants (47 compared with 46%, p = 0.41). Clinical trial participants were more likely to be white than nonparticipants, but the absolute difference was small (97.4 compared with 96.2%, p = 0.01). Clinical trial participants were still more likely to have commercial insurance (56.4 compared with 50.1%, p < 0.001). These results suggest that the findings in the overall cohort are not dominated solely by two phase 3 studies.
DISCUSSION
Clinical trials have become the most accepted method for assessing the effectiveness and efficacy of clinical therapies and devices in medicine. The randomized, double-blind, controlled trial has been termed “the most definitive tool for evaluation of the applicability of clinical research” (16). Although clinical trials can be a tremendous asset for furthering the advancement of medicine, they are largely dependent on adequate study populations who represent the overall population well in regard to both clinical and demographic characteristics.
The CFFPR provided an ideal tool to test the hypothesis of whether clinical trial participants with CF differed from those subjects with CF who did not participate in clinical trials. Similar comparisons of participants and nonparticipants have been published, but, to our knowledge, this is the first study to be able to compare clinical trial participants with the majority of the target population in a given country and the first study to do so in CF. We have shown that the overall rate of participation of CF in clinical trials is very high (30% of our cohort). In contrast to the other reported studies, subjects with CF participating in clinical trials had worse lung function, lower weight percentile, higher rate of P. aeruginosa colonization, and more interactions with health care providers as assessed by the increased number of annual office visits even in a restricted subset who fulfilled limited lung function requirements for a large phase 3 clinical trial during the study period. Most studies have shown that clinical trial participants have less severe disease compared with nonparticipants (2, 3). The two major published clinical trials in CF look primarily at patients with an FEV1% predicted greater than 25% and any condition that in the view of the investigator would interfere with study participation (14, 15). Thus, the selection bias (both set by inclusion criteria and by physician and patient selection) should have led to a clinical trial population with milder disease. Our findings suggest that patients who have more severe disease with more frequent encounters with the health care system or sites participating in clinical trials may perceive clinical trial participation as a benefit and thus be more likely to participate.
Previous work evaluating clinical trial participants and nonparticipants has primarily been in the fields of oncology and human immunodeficiency syndrome. Differences between trial participants and nonparticipants have been noted, but these studies have been limited by the inability to capture the entire target population. Hutchins and colleagues (2) evaluated patients enrolled in the 164 Southwest Oncology Group treatment trials between 1993 and 1996. They found that patients older than 65 were much less likely to participate in clinical trials than those younger than 65, even after excluding all trials with age limits as enrollment criteria. Antman and colleagues (3) noted that patients eligible for adjuvant doxorubicin for intermediate- and high-grade sarcoma but those not enrolled in clinical trials had significantly higher stage lesions and subsequent worse disease-free survival. Our findings are very different from these studies; subjects participating in clinical trials appear to have more advanced disease, despite the exclusion of patients with CF in clinical trials who have severe disease.
We also found that subjects with commercial/private insurance were more likely to participate than those with government insurance although the absolute differences were small; surprisingly, given time constraints, being employed was also associated with study participation. The differences in participation between whites and nonwhites were statistically significant, but small in absolute terms. Although the magnitude of the differences were small, this finding is concerning. Are subjects with government insurance (thus with probable low socioeconomic status) and unemployed subjects not getting adequate opportunity to participate in clinical trials in CF? This is especially relevant considering that participants had a lower rate of lung function decline during the 7 yr of follow-up than trial nonparticipants. Importantly, we did not find any sex bias in our study even after subgroup analyses, as has been shown in other disease processes (4).
When clinical outcomes were assessed in the cohort, we found that despite the increased severity of illness of trial participants, absolute change in lung function as assessed by FEV1% predicted and rate of lung function decline were lower in the clinical trial participant group compared with the nonparticipants. This finding was not affected by the baseline lung function. When we adjusted for a marker of health care utilization, number of annual office visits, we found that the effect of trial participation on lung function decline disappeared. Thus, access to care appears to be in the causal pathway explaining the association CF lung function decline and clinical trial participation. Perhaps clinical trial participation enhances access to care; potentially, as a component of clinical trial participation, physicians increase the evaluation of more routine symptoms when compared with usual care. In essence, such detailed clinical evaluations that occur when subjects are under study in a clinical trial uncover clinical problems that would otherwise not be detected at that time with usual care.
To approach the question of generalizability of a major therapy in CF, we compared the clinical and demographic of trial participants in the phase 3 intermittent administration of inhaled tobramycin to nonparticipants who fulfilled the entry criteria. Unlike our comparisons of participants versus nonparticipants in the main cohort, we found no significant clinical differences in the populations (including insurance status and total number of hospitalizations). This finding suggests that the trial population was very representative of the entire eligible population and did not represent a biased sample. It is unclear why differences in employment, insurance status, and severity of illness were not seen in this analysis as in the larger cohort. What is encouraging is that the concerns of participation in clinical trials presented in the analyses were not reflected in this important phase 3 study of an agent that is now widely used around the world to treat CF-related lung disease. These analyses are unusual in that we were able to compare clinical trial participants compared with the entire population at large who fulfilled inclusion criteria for the study; such an analysis of a randomized controlled trial has never before been performed. Although our results showed no evidence of selection bias, systems should be put in place to perform population-wide postmarketing ecologic assessments of novel drug entities to ensure that drug efficacy is maintained in the target population.
Equal representation of patient groups in clinical trials was advocated in the Belmont Report of 1979 (http://ohsr.od.nih.gov/guidelines/belmont.html) to ensure that the risk of participation was spread out among the broader population. More recently, as noted in the National Institutes of Health guidance document in 1994 and the amending of this document in 2001, clinical trial participation was believed to provide a benefit to participants. Participants may obtain access to new therapies and procedures for conditions with limited treatment options. In the current study, we have shown that clinical trial participants, despite having more severe disease, had lower rates of lung function decline over the intervening 7 yr. One could hypothesize numerous reasons for this association; the least likely of which is that clinical trial participation itself improved lung function. The finding that the impact on lung function disappeared when adjusted for the number of clinic visits argues against the conclusion clinical trial participation alone altered lung function decline. Two of the primary studies included in this time period did turn out to improve lung function in CF; access to these two new agents could account for some of the lung function findings of this study (14, 15). Lung function is intimately associated with prognosis, and survival as has been noted in several recent survival prediction models for CF (17, 18). The difference that we found in insurance status, though small, and the difference in outpatient visits may be a reflection of access to care. Clinical trial participants may have better access to care; it is unclear whether this is due to study participation or a marker of those more likely to participate. Given that there may be potential benefits to study participation, CF clinicians needs to ensure adequate opportunities for participation in studies for all eligible subjects.
The primary limitation of these analyses is our inability to assess reasons for nonparticipation. Patients may have been eligible for trial participation, but refused, or they may not have been offered an opportunity to enroll. Given that differences between insurance status and race were seen, a better understanding of barriers to participation is needed. It is encouraging that we found no gender bias to participation and that the population enrolled in a major phase 3 trial was very representative of all the eligible subjects. Another important limitation of this analysis is that patients who died or were lost to follow-up were excluded from the analysis. This population might be quite different from the remaining population and may have very different levels of clinical trial participation. We did find very similar overall rates of study participation when including all subjects who were lost to follow-up or who died (29 compared with 30%), arguing against a significant bias and no significant differences to our results when subjects who died or were lost to follow-up were included in the analyses. Last, when we compared patients enrolled in a major phase 3 study with those who fulfilled entry criteria during the clinical trial years, we could only account for 78% of the original study population. Thus, some enrolled subjects are likely to have been classified as unenrolled, potentially biasing our results.
The results of this study have addressed some of the important questions related to generalizability of CF clinical trials and help define populations with poor participation rates and underrepresentation in major clinical trials in CF. We did find differences between clinical trial participants and nonparticipants that should be further explored. Are there barriers that patients who have government insurance face in regard to study participation? Is it merely a factor of convenience that patients seen more often in clinics participate in clinical trials? Because we had the ability to look at the entire population of potential trial participants, we are able to comment on the generalizability of a major therapy for CF lung disease: inhaled tobramycin. The study population for the phase 3 study was representative of the eligible population as a whole. To continue to move the science of clinical research forward, one must continue reevaluate how representative the research will be and ensure that all patients with CF have an opportunity to participate in clinical research if they are interested.
Acknowledgments
The authors thank Robert J. Beall, Ph.D., President and CEO, CF Foundation; Preston Campbell, III, M.D., Vice President for Medical Affairs, CF Foundation; Bruce Marshall, M.D., Director of Clinical Affairs, CF Foundation; Monica Brooks, B.S., CF Foundation; and the CF Foundation and the CF community for their support of this project and for providing the CF registry data.
Supported by Leroy Matthew Physician Scientist, Cystic Fibrosis Foundation 1 K23 HL72017-01, National Heart, Lung, and Blood Institute.
Originally Published in Press as DOI: 10.1164/rccm.200502-273OC on September 28, 2005
Conflict of Interest Statement: None of the authors have a financial relationship with a commercial entity that has an interest in the subject of this manuscript.
References
- 1.Rosenstein BJ, Cutting GR. The diagnosis of cystic fibrosis: a consensus statement. Cystic Fibrosis Foundation Consensus Panel. J Pediatr 1998;132:589–595. [DOI] [PubMed] [Google Scholar]
- 2.Hutchins LF, Unger JM, Crowley JJ, Coltman CA Jr, Albain KS. Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med 1999;341:2061–2067. [DOI] [PubMed] [Google Scholar]
- 3.Antman K, Amato D, Wood W, Carson J, Suit H, Proppe K, Carey R, Greenberger J, Wilson R, Frei E III. Selection bias in clinical trials. J Clin Oncol 1985;3:1142–1147. [DOI] [PubMed] [Google Scholar]
- 4.Stone VE, Mauch MY, Steger K, Janas SF, Craven DE. Race, gender, drug use, and participation in AIDS clinical trials: lessons from a municipal hospital cohort. J Gen Intern Med 1997;12:150–157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Albert SM, Sano M, Marder K, Jacobs DM, Brandt J, Albert M, Stern Y. Participation in clinical trials and long-term outcomes in Alzheimer's disease. Neurology 1997;49:38–43. [DOI] [PubMed] [Google Scholar]
- 6.McCusker J, Wax A, Bennett JM. Cancer patient accessions into clinical trials: a pilot investigation into some patient and physician determinants of entry. Am J Clin Oncol 1982;5:227–236. [DOI] [PubMed] [Google Scholar]
- 7.Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol 1999;52:1143–1156. [DOI] [PubMed] [Google Scholar]
- 8.Kemeny MM, Peterson BL, Kornblith AB, Muss HB, Wheeler J, Levine E, Bartlett N, Fleming G, Cohen HJ. Barriers to clinical trial participation by older women with breast cancer. J Clin Oncol 2003;21:2268–2275. [DOI] [PubMed] [Google Scholar]
- 9.Fitzsimmons SC. The changing epidemiology of cystic fibrosis. J Pediatr 1993;122:1–9. [DOI] [PubMed] [Google Scholar]
- 10.Goss CH, Rubenfeld GD, Ramsey B, Aitken ML. Clinical trial participants compared to non-participants in cystic fibrosis. Pediatr Pulmonol 2000;S20:315. [Google Scholar]
- 11.Knudson RJ, Lebowitz MD, Holberg CJ, Burrows B. Changes in the normal maximal expiratory flow-volume curve with growth and aging. Am Rev Respir Dis 1983;127:725–734. [DOI] [PubMed] [Google Scholar]
- 12.Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Stat Med 1997; 16:965–980. [DOI] [PubMed] [Google Scholar]
- 13.Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika 1986;73:13–22. [Google Scholar]
- 14.Ramsey BW, Pepe MS, Quan JM, Otto KL, Montgomery AB, Williams-Warren J, Vasiljev K, Borowitz D, Bowman CM, Marshall BC, et al. Intermittent administration of inhaled tobramycin in patients with cystic fibrosis. Cystic Fibrosis Inhaled Tobramycin Study Group. N Engl J Med 1999;340:23–30. [DOI] [PubMed] [Google Scholar]
- 15.Fuchs HJ, Borowitz DS, Christiansen DH, Morris EM, Nash ML, Ramsey BW, Rosenstein BJ, Smith AL, Wohl ME. Effect of aerosolized recombinant human DNase on exacerbations of respiratory symptoms and on pulmonary function in patients with cystic fibrosis. The Pulmozyme Study Group. N Engl J Med 1994;331:637–642. [DOI] [PubMed] [Google Scholar]
- 16.Friedman M, Furberg CD, DeMets DL. Fundamentals of clinical trials, 3rd ed. St. Louis: Mosby-Year Book; 1996.
- 17.Mayer-Hamblett N, Rosenfeld M, Emerson J, Goss CH, Aitken ML. Developing cystic fibrosis lung transplant referral criteria using predictors of 2-year mortality. Am J Respir Crit Care Med 2002;166: 1550–1555. [DOI] [PubMed] [Google Scholar]
- 18.Liou TG, Adler FR, Fitzsimmons SC, Cahill BC, Hibbs JR, Marshall BC. Predictive 5-year survivorship model of cystic fibrosis. Am J Epidemiol 2001;153:345–352. [DOI] [PMC free article] [PubMed] [Google Scholar]