Using ClinicalTrials.gov to Understand the State of Clinical Research in Pulmonary, Critical Care, and Sleep Medicine

Abstract

Rationale: ClinicalTrials.gov is the largest trial registry in the world. Strengthened registration requirements, including federal mandates in 2007, have increased study representation. A systematic evaluation of all registered studies has been limited by the absence of an aggregate data set and specialty-specific search terms.

Objectives: We leveraged a newly transformed database containing annotated data from ClinicalTrials.gov to define the portfolio of interventional clinical research in pulmonary, critical care, and sleep medicine.

Methods: Analysis was restricted to studies registered after September 2007 through September 2010 and defined as “interventional” (n = 40,970). A specialty-specific study data set (n = 2,226) was created using disease condition terms provided by data submitters and medical subject heading terms generated by a National Library of Medicine algorithm. Trial characteristics were extracted and summarized using descriptive statistics.

Measurements and Main Results: Pulmonary, critical care, and sleep medicine trials composed 5.4% of all interventional studies registered over the 3-year period. In contrast, oncology and cardiovascular disease composed 21.9 and 8.4% of trials, respectively. Within pulmonary trials, asthma and chronic obstructive pulmonary disease were the most studied conditions (27.4 and 21.8% of studies, respectively), and measures of lung function or safety were the most frequent primary outcomes. Nearly two-thirds of trials indicated enrollment of 100 patients or fewer, and a majority of studies were phase II or III trials. The single largest funding source (43.5%) was industry, and study characteristics varied by funding source.

Conclusions: We applied a novel approach to describe the portfolio of interventional clinical research in pulmonary medicine. Our results indicate a disparity between trial representation and the burden of respiratory disease. Resources should be targeted across the spectrum of pulmonary research to address this discrepancy.

Respiratory diseases are a major public health concern, with chronic lower respiratory tract disease now representing the third leading cause of death in the United States (1). At the same time, the number of patients cared for in the intensive care unit is projected to increase sharply over the next decade (2). The combined economic burden is overwhelming—chronic respiratory disease alone accounts for annual expenditures in excess of $100 billion (3).

Clinical trials provide the foundation for evidence-based medicine and are critical to inform public policy; however, stakeholders lack methods by which to assess characteristics of the clinical research enterprise in pulmonary, critical care, and sleep medicine (PCCSM) as a whole. Without this knowledge, it remains unclear whether the quality and scope of available research adequately reflect population disease burden and are sufficient to inform health-care providers and policy makers alike. With growing concern about how clinical research is designed, performed, and funded (4), an aggregate analysis of the PCCSM clinical research portfolio is timely.

The ClinicalTrials.gov database was initially launched in response to congressional mandates for a publicly available registry of interventional trials; further actions by the International Committee of Medical Journal Editors and the U.S. Food and Drug Administration (FDA) made journal publication conditional on registration and mandated results and adverse events reporting (Table 1) (5, 6). A systematic evaluation of this public database, however, has been hindered by inadequate access to the comprehensive annotated data. Members of the Clinical Trials Transformation Initiative have bridged this gap by creating a high-quality, publicly available, annotated database of the information contained in ClinicalTrials.gov—the database for Aggregate Analysis of ClinicalTrials.gov (AACT, https://www.ctti-clinicaltrials.org/project-topics/clinical-trials.gov/aact-database) (7). Addition of PCCSM and other specialty-specific search capabilities have further enhanced the functionality of AACT.

Table 1.

Milestones in clinical trial registration requirements

Date	Milestone
November 21, 1997	FDAMA section 113 is enacted, requiring creation of a registry of clinical trials
February 29, 2000	First version of ClinicalTrials.gov registry is made publicly available
September 16, 2004	ICMJE publishes policy requiring trials registration as a condition of publication*
September 27, 2007	FDAAA section 801 expands scope and mandates registration, enacting penalties for noncompliance
September 27, 2008	Reporting of summary “basic trial results” is mandated
September 27, 2009	Reporting of adverse events is mandated

Definition of abbreviations: FDAAA = Food and Drug Administration Amendments Act; FDAMA = Food and Drug Administration Modernization Act; ICMJE = International Committee of Medical Journal Editors.

^*This policy applied to any clinical trial starting enrollment after July 1, 2005. Trials that began enrollment before this date were required to be registered by September 13, 2005.

Adapted by permission from Reference 11.

We leveraged this transformed database to provide a recent snapshot of interventional clinical trials in PCCSM, including basic trial characteristics, funding source, and distribution of diseases and primary study outcomes. Additionally, we sought to determine if trial characteristics varied by funding source. Using descriptive statistics, we provide the first comprehensive overview of recent clinical trial activity within our discipline. We establish that AACT is a valuable tool that can be used to monitor trends in trial activity and study characteristics over time within a target subspecialty area. Furthermore, it may provide a foundation on which to construct a mechanism to more critically assess the public and private investment in PCCSM trials moving forward. These study results were initially presented in abstract form as an oral presentation at the American Thoracic Society (8).

Methods

Creation of the ClinicalTrials.gov Data Set

Figure 1 summarizes the process by which the final PCCSM study data set was created. In brief, a database of 96,346 clinical studies registered at ClinicalTrials.gov was downloaded on September 27, 2010. Subsequently, the data set was locked, and the relational AACT database was designed to facilitate aggregate analysis. An article detailing the creation of the AACT database and the development, implementation, and validation of methods for regrouping by clinical specialty has been previously published (9).

Figure 1.

Creation of the pulmonary, critical care, and sleep medicine (PCCSM) specialty data set. *Medical subject heading (MeSH) IDs relevant to transplant medicine were reviewed for a future report; however, studies identified by these IDs were excluded from the current analysis.

Creation and Subcategorization of the PCCSM Study Data Set

Analysis was restricted to interventional studies, defined as those in which participants are assigned according to a research protocol to receive specific interventions, registered after September 2007 through September 2010 (n = 40,970). We chose to include trials after September 2007, given that this date corresponds with the enactment of the legal obligation for sponsors to register applicable interventional trials. Studies were censored in September 2010 to account for the natural lag between actual study completion and revision of the study record at ClinicalTrials.gov.

The PCCSM study data set was created using disease condition terms provided to ClinicalTrials.gov by data submitters at the time of trial registration and additional condition medical subject heading (MeSH) terms generated by a National Library of Medicine algorithm. Condition terms from a subset of the 2010 MeSH thesaurus (10) and non-MeSH condition terms that appeared in five or more studies in the selected analysis data set were reviewed by clinical specialists (J. L. T. and S. M. P.) and annotated according to their relevance to PCCSM topics (Y = relevant, N = not relevant). Studies with at least one associated relevant term were included in the initial study data set (n = 2,888). A final study data set (n = 2,226) was created after manual review of the ClinicalTrials.gov study record. The studies were then subcategorized according to disease and primary outcome. Probable funding source was derived from submitted lead sponsor and collaborator information using an algorithm developed by the Library of Medicine (11). Additional details on methods used to develop and subcategorize the specialty data set in addition to steps taken to standardize the analysis are provided in Appendix E1 in the online supplement.

Statistical Analysis of PCCSM Study Data Set

Descriptive statistics were used to characterize the portfolio. Unless otherwise indicated, percentages were calculated at the study level and exclude studies with missing values in the denominator. The composition of the PCCSM clinical trial portfolio was stratified according to several basic study characteristics. These results were then compared with all interventional trials registered in the same 3-year time frame not included in the PCCSM study data set (“non-PCCSM” trials). SAS software (version 9.2; SAS Institute) was used for all statistical analyses.

Results

Basic Trial Characteristics

PCCSM trials composed 5.4% (2,226 of 40,970) of all interventional clinical trials registered between October 2007 and September 2010. In contrast, oncology and cardiovascular disease accounted for 21.9 and 8.4% of registered interventional trials, respectively, during the same 3-year period (11, 12). Within the PCCSM data set, the number of clinical trials started per year increased slightly over the included time frame, from 473 studies during the first year of analysis to 576 during the final year analyzed.

A complete report of all evaluated basic trial characteristics can be found in Table E1. As expected, the overwhelming majority of trials were treatment trials, with drugs, devices, and procedures accounting for more than three-fourths of the interventions (Table 2). A majority of trials were reported as phase II or III, composing 65.8% of all studies, whereas 11.6% were phase I and 22.6% were phase IV. With respect to trial design, 79.0% (1,714 of 2,170) of trials were randomized, with 47.2% (1,033 of 2,188) reporting double blinding, 11.5% (252 of 2,188) single blinding, and 41.3% (903 of 2,188) using an open-label design. Additionally, most studies followed a parallel group interventional model (59.3%, 1,302 of 2,194), with fewer reporting a crossover or factorial study design (18.6%, 409 of 2,194, and 1.3%, 29 of 2,194, respectively).

Table 2.

Distribution of all pulmonary, critical care, and sleep medicine clinical trials (n = 2,226) by intervention type

Intervention Type*	Studies (n)	% of Total
Drug	1,262	56.7
Device	368	16.5
Procedure	177	8.0
Behavioral	136	6.1
Dietary supplement	72	3.2
Biological	41	1.8
Genetic	2	0.1
Radiation	2	0.1
Other	334	15.0

^*Studies may indicate more than one intervention type.

Notably, 64.5% (1,418 of 2,198) of trials indicated an actual or anticipated enrollment of fewer than 100 patients, whereas only a small fraction (0.5%, 11 of 2,198 actual; 3.2%, 71 of 2,198 anticipated) included more than 1,000 patients (Figure 2). Nearly one-third of the studies excluded patients older than 65 years, and further analysis demonstrated that most of these studies were related to asthma (data not shown). Of all the trials in the PCCSM data set, 12.7% were noted to be strictly limited to the pediatric population.

Figure 2.

Pulmonary, critical care, and sleep medicine clinical trial enrollment. Studies with missing enrollment values (n = 28) are not represented. Data reflect actual enrollment for completed studies and anticipated enrollment for active studies.

Data on the lead sponsor and collaborators were available for all included studies, and the probable funding source was derived as described in Appendix E1. Of all trials in the PCCSM data set, 43.5% (969 of 2,226) were industry-funded, whereas the number of National Institutes of Health (NIH)-funded trials was substantially lower (5.4%, 119 of 2,226). A large percentage of trials (51.1%, 1,138 of 2,226) maintained other funding sources. Of the top agencies, either lead sponsor or collaborator, that were categorized as “other” funder, the preponderance were universities, health-care institutions or large hospital networks, and private foundations. In particular, the Cystic Fibrosis Foundation (CFF) was the number one collaborator on trials with other funding sources. A list of the top 15 other funding agencies can be found in Table E2.

The majority of trials included enrolling sites in North America (50.0%, 1,022 of 2,046) and/or Europe (42.5%, 870 of 2,046), and fewer reported sites in other regions. When all trials in the PCCSM data set were compared with all non-PCCSM trials, there were no obvious differences in study start/completion year, allocation, blinding, phase, enrollment, or reported sponsor, although PCCSM studies were more likely than non-PCCSM studies to report an enrolling site in Europe (42.5%, 870 of 2,046, vs. 29.4%, 10,441 of 35,474).

Disease Distribution

The 20 most frequent conditions represented are outlined in Table 3, and a list of less common disease conditions can be found in Table E3. Asthma, chronic obstructive pulmonary disease (COPD), and sleep-disordered breathing composed more than one-half of the registered trials, with other conditions being much less commonly studied. Strikingly, clinical trials related to sepsis and shock accounted for only 6.4% (136 of 2,121) of all PCCSM studies. Even when studies of sepsis, shock, and other nonrespiratory failure–related critical care topics were combined, they accounted for less than 10% of all trials in the data set. When examining exclusively pediatric trials, asthma, hypoxic respiratory failure, and cystic fibrosis (CF) made up greater than two-thirds of the diseases studied—specifically, asthma accounted for 40.6% (data not shown) of the pediatric trials.

Table 3.

Distribution of all pulmonary, critical care, and sleep medicine clinical trials (n = 2,121) by disease condition

Condition*	Studies (n)	% of Total
Asthma	582	27.4
COPD	463	21.8
Sleep-disordered breathing	207	9.8
Cystic fibrosis	156	7.4
Pulmonary hypertension	138	6.5
Sepsis and shock	136	6.4
Respiratory failure, not specified as hypoxic	112	5.3
Hypoxic respiratory failure†	99	4.7
Critical care, not specified as sepsis or shock	70	3.3
Intubation and airway management	52	2.5
Interstitial lung disease	33	1.6
Venous thromboembolic disease	27	1.3
Bronchiolitis	24	1.1
VAP and tracheobronchitis	21	1.0
Bronchiectasis	19	0.9
Cough	19	0.9
Sarcoidosis	16	0.8
Pleural disease	14	0.7
Pneumothorax	14	0.7
Cardiopulmonary resuscitation	11	0.5
Other‡	69	3.3

Definition of abbreviations: COPD = chronic obstructive pulmonary disease; PCCSM = pulmonary, critical care, and sleep medicine; VAP = ventilator-associated pneumonia.

Using the described methodology, at least one condition was identified for 95.3% (2,121 of 2,226) of studies in the PCCSM data set.

^*Studies may indicate more than one condition of interest.

^†Includes acute lung injury and ARDS.

^‡These studies are outlined in Table E3 in the online supplement.

When we stratified trial enrollment according to disease of study, COPD and asthma accounted for the majority of clinical trials enrolling more than 1,000 patients (32.9%, 27 of 82, and 24.4%, 20 of 82, respectively); this was not surprising, given their overall preponderance in the data set. Interestingly, however, 20.7% (17 of 82) of these large trials evaluated sepsis or shock.

Primary Outcomes of Interest

Of the diverse range of primary outcomes that were represented, measures of lung function, safety, and cytokines or biomarkers were evaluated most frequently. The 10 most commonly reported primary outcomes in the PCCSM data set as a whole are shown in Table E4. To test our free text coding methodology, primary outcomes were examined by trial phase; as anticipated, earlier-phase studies were more likely to examine safety and pharmacodynamic or kinetic endpoints, whereas later-phase trials were more likely to consider measures of lung function, mortality, and exacerbation (data not shown).

Influence of Funding Source on Study Characteristics

Trials including more than 500 patients were more likely to be funded by industry than by the NIH or other sources (n = 120, 20, and 62, respectively). Asthma and COPD were the top two funded disease categories for all derived funding sources. Pulmonary hypertension was funded at a higher priority by industry, whereas hypoxic respiratory failure and sleep-disordered breathing were a higher priority of studies funded by the NIH and other sources, respectively. Table 4 defines the frequency of the most common primary outcomes in the data set as a whole stratified by funding source. Interestingly, in comparison with studies that were industry-funded, the NIH-funded studies contained a much broader scope of primary outcomes and were more likely to include mortality, quality of life, and measures of cytokines or biomarkers.

Table 4.

Stratification of top 10 primary outcomes by derived funding source

Primary Outcome*	Industry, % (n = 962)	NIH, % (n = 119)	Other. % (n = 1,112)
Lung function	30.8	15.1	7.9
Safety and tolerability	17.2	4.2	3.1
Cytokines and biomarkers	3.6	8.4	9.4
Change in vital signs/serum parameters	4.3	3.4	5.2
Mortality	2.9	10.9	5.3
Pharmacodynamics and kinetics	7.4	1.7	2.2
Quality of life	1.8	6.7	3.9
Exercise capacity other than 6MWD	1.5	2.5	3.9
Exacerbation	4.0	3.4	1.3
6MWD	2.6	2.5	2.4

Definition of abbreviations: 6MWD = 6-minute walk distance; NIH = National Institutes of Health.

^*Studies can indicate more than one primary outcome. No outcome could be determined for 33 studies based on the information submitted at ClinicalTrials.gov.

Discussion

Using the AACT database, a newly created resource that includes annotated data from ClinicalTrials.gov, we completed the first comprehensive examination of recent interventional clinical research in PCCSM. Milestones and attributes relevant to ClinicalTrials.gov registration led us to restrict the portfolio analysis to studies registered over a fixed period from 2007 to 2010. From this analysis we demonstrate that PCCSM studies make up a small fraction of all registered clinical trials. Furthermore, we determined that a majority of trials are of limited sample size, focus on asthma or COPD, and use lung function measures as the primary outcome. We also demonstrate that industry funds a substantial proportion of trials and that study size, primary outcomes, and disease focus vary based on funding source.

The scope of the PCCSM clinical research portfolio is not surprising, with overrepresentation of common respiratory diseases with existing pharmacologic therapies, such as COPD or asthma. Our observation that PCCSM studies account for only 5.4% of all registered trials, however, is somewhat unexpected, particularly in the context of increasing chronic respiratory disease–related mortality in recent decades (13). In contrast, cardiovascular disease and cancer, which represent a larger proportion of all interventional trials, have demonstrated consistent downward trends in associated mortality over time (1, 13). Comparing CDC morbidity estimates with number of available trials for cancer, heart disease, asthma, and COPD (Figure 3) further highlights this disparity.

Figure 3.

Morbidity estimates within the U.S. population for heart disease, cancer, chronic obstructive pulmonary disease (COPD), and adult asthma compared with number of trials, stratified by phase, that were registered in ClinicalTrials.gov between September 2007 and September 2010. *Trials indicating phase as “N/A” are not included. Trial numbers for cancer are derived from prior work published by Hirsch and colleagues (12). Trial numbers for heart disease are derived from unpublished work by Dr. Karen Alexander and Dr. David Kong. †Morbidity estimates according to the Centers for Disease Control summary health statistics for U.S. adults: National Health Interview Survey, 2011 (19).

As trial number, at least in part, may reflect the availability of plausible therapies, our results appear to support recent position statements from respiratory organizations arguing the tangible consequences of relative underfunding at the basic science level and lagging translation of basic research into new therapeutic targets for pulmonary diseases (3, 14). In fact, for some disease areas, such as pulmonary fibrosis or sepsis, perhaps what is most needed is increased investment in basic and translational approaches to achieve the scientific rationale necessary to support more successful movement into clinical trials.

We learned that PCCSM trials are generally of small sample size, with nearly two-thirds of studies reporting an anticipated or actual enrollment of 100 or fewer patients. This finding could not be attributed to a high proportion of early-phase studies, as a majority of trials were phase II or III, and was somewhat surprising particularly given the preponderance of COPD and asthma studies within the portfolio. Although sample size does not exclusively reflect trial quality and sufficiency, limited sample sizes directly impact statistical power to accurately assess efficacy and may predispose to type II errors, resulting in unjustified disbandment of potential therapeutic interventions. Similarly, small sample sizes may influence the ability to detect important safety signals. Perhaps one strategy moving forward would be to invest in fewer, but larger sized and appropriated clinical studies within the discipline or, alternatively, to leverage nonparallel trial designs for studies of less common pulmonary diseases.

As might be expected, the most used primary outcomes in PCCSM studies as a whole were those related to lung function parameters. The relative emphasis on change in lung function as an approval endpoint by the FDA (15) (and in turn a deemphasis on biomarker data and patient-reported outcomes, such as quality of life) likely explains the high prevalence of lung function as an outcome of interest among industry-funded trials. Interestingly, we noted a disparity between industry- and non–industry-funded trials, with the latter including more biomarker and quality-of-life primary outcomes. Failing to include data on these outcomes in industry trials may represent a missed opportunity to learn more about variations in treatment response and the impact of treatment on the patient’s sense of well-being.

Despite its relative rarity, affecting an estimated 30,000 individuals in the United States (16), CF was the fourth most commonly evaluated disease among all PCCSM studies. We believe this illustrates a successful paradigm by which patient advocacy groups can partner with public and private resources to dramatically affect the research landscape. When CF and non-CF studies were compared, there were no apparent differences in trial characteristics (data not shown); however, the CFF was the most frequent collaborator for studies funded by private agencies or institutions. The CFF recently published its model for drug discovery and development (17), citing a multifaceted approach that includes a nationwide clinical trials network designed to facilitate rapid evaluation of new CF therapies and public awareness campaigns to enhance trial enrollment. This may represent an optimal model that can be applied for other less common pulmonary diseases to advance drug development and improve patient care.

There are several limitations to our analysis, most of which relate to the quality and comprehensiveness of the data registered at ClinicalTrials.gov. We selected studies from 2007 onward, as mandatory reporting requirements increased data completion after this time. A lack of standard ontology and free text entry for some data elements, however, makes aggregate analysis difficult. It should also be noted that although ClinicalTrials.gov is the largest clinical trial registry in the world, now representing more than 100,000 trials from 182 countries (18), it is most likely to be complete for interventional drug or device studies sponsored by U.S.-based or multinational organizations. As such, our results do not apply to noninterventional studies and are not inclusive of data from a number of smaller clinical trials registries around the world. Finally, we recognize that although the focus of our work relates to the availability and characteristics of clinical trials within pulmonary medicine, our analysis does not encompass wide-scale public health strategies or missed public health opportunities to limit tobacco and other environmental exposures, which are likely to be of greatest long-term benefit in mitigating the burden of respiratory disease within the population.

In summary, by leveraging the newly transformed AACT database, we provide the first comprehensive analysis of the PCCSM interventional clinical research portfolio. Our study offers unique insights into the scope of PCCSM research and points to opportunities for trial design improvement. Furthermore, the impact of the FDA, funding sources, and patient advocacy groups in shaping the research landscape is readily apparent. Our analysis introduces key stakeholders, including study sponsors, clinicians, researchers, policy makers, and public advocacy groups, to a novel resource by which to begin to better understand and more critically assess the state of PCCSM clinical research. For example, our observation that PCCSM trials compose a relatively narrow proportion of all registered interventional trials, particularly when compared with other disease areas of similar public health concern, provides an opportunity for these stakeholders to evaluate the focus and direction of interventional research within our discipline moving forward. Through such an effort, the impact of the PCCSM research portfolio can be maximized to ensure fulfillment of existing and projected public health needs.