Trial Registration at ClinicalTrials.gov between May and October 2005

Abstract

BACKGROUND

Clinical trial registration allows interested parties to obtain information about ongoing and completed trials, but there are few data indicating the quality of the information provided during the registration process. We used information in the publicly available ClinicalTrials.gov database to describe patterns of trial registration before and after the implementation by journal editors of a new policy requiring registration as a prerequisite for publication.

METHODS

We reviewed ClinicalTrials.gov records to determine patterns of completion of the “Intervention Name” and “Primary Outcome Measure” data fields for trials registered on May 20 and October 11, 2005, and for trials registered during the interval between these two dates, inclusively.

RESULTS

During the interval studied, the number of registrations in ClinicalTrials.gov increased by 73 percent from 13,153 to 22,714. The percentage of interventional trials registered by industry with nonspecific Intervention Name entries (attributable to four drug companies) decreased from 10 percent to 2 percent; all other industry and nonindustry records contained specific entries in this field. Of the 2670 studies registered by industry between the two dates, 76 percent provided information in the Primary Outcome Measure field, although these entries varied markedly in their degree of specificity. In the remaining 24 percent of the records, this field was blank.

CONCLUSIONS

During the summer of 2005, there were large increases in the number of clinical trial registrations. Overall, the data contained in records were more complete in October than they were in May, but there still is room for substantial improvement.

Concern about previously undisclosed safety problems with drugs such as paroxetine (Paxil, GlaxoSmithKline) and rofecoxib (Vioxx, Merck) has increased the public’s desire for more complete information about clinical research studies.^1,2 The provision of basic information about clinical trial protocols in a publicly accessible registry and the public identification of all trials, whether or not their results are subsequently published, have been advocated as ways to address this issue.^3–6 Numerous groups have called for comprehensive registration by issuing statements or convening meetings to discuss policy and implementation details.^7–15

In the United States, the Food and Drug Administration (FDA) Modernization Act, section 113 (FDAMA 113), mandates the registration of all private and public trials that test effectiveness for “serious or life-threatening” conditions submitted to the FDA under investigational-new-drug applications (IND).¹⁶ A Web-based registry, ClinicalTrials.gov, was established in 2000 by the National Library of Medicine on behalf of the National Institutes of Health as a result of this law.^17,18 Although FDAMA 113 mandates the registration of certain data elements, ClinicalTrials. gov also includes a broad set of optional data elements. In addition, ClinicalTrials.gov permits the registration of any clinical trial, regardless of its IND status, the type of intervention (e.g., surgical procedure, device, or drug), the medical condition, or the country of origin. As of late October 2005, the registry contained more than 23,000 trials.

We examined the completeness and utility of the information contained in trial-registration records in ClinicalTrials.gov from May 20 through October 11, 2005. This period includes time both before and after September 13, 2005, the date of implementation of the International Committee of Medical Journal Editors (ICMJE) policy requiring the registration of clinical trials as a prerequisite for consideration for publication.^7,8

METHODS

THE REGISTRY

Sponsors, principal investigators, or other persons or organizations with primary responsibility for a given clinical trial (called “data providers”) can register with ClinicalTrials.gov through a Web-based system (http://prsinfo.ClinicalTrials.gov).¹⁹ In some instances, “intermediary trial registries,” such as that of the National Cancer Institute (www.cancer.gov), provide trial data. The database uses both open-ended responses and menu-based options, and terms from the National Library of Medicine Unified Medical Language System^20,21 are used to facilitate subsequent information retrieval. Trials with the same protocol that are conducted at multiple sites are considered one trial in the registry. The complete entry in the registry for a given trial is referred to as a record in the database.

ClinicalTrials.gov includes both mandatory and optional data elements. Trials cannot be registered without the completion of all mandatory data elements, which include both FDAMA 113 and registry-imposed requirements. In addition, the ICMJE requires completion of some of the optional data elements. Members of the National Library of Medicine staff manage the quality of information in the registry by rejecting records that do not have all required fields completed, reviewing entries for appropriate content and internal consistency, ensuring that links are active and relevant, checking contact information for recruiting studies, and confirming that approval from an institutional review board has been obtained. In addition, sponsoring organizations must electronically sign off on all entries (and subsequent revisions) before they are made available on the Web site.

STUDY 1: ALL INTERVENTIONAL TRIALS

We described the numbers and types of trials registered in ClinicalTrials.gov. Because records in ClinicalTrials.gov can be modified at any time by data providers to keep the information current, this study was conducted with data that were available to the public on May 20, 2005 (before the September 13, 2005, implementation of the ICMJE policy on trial registration) and on October 11, 2005 (four weeks after implementation of the policy). We also reviewed data from trials registered between these two dates, inclusively, which we refer to as the interval sample (Fig. 1A). Searches of ClinicalTrials.gov were accomplished with the use of a National Library of Medicine reporting tool, although they could be replicated with the use of the public search function in combination with individual inspection of those data.

Figure 1. Categories of Trials Registered in ClinicalTrials.gov on or between May 20 and October 11, 2005.

Panel A shows the origins and types of trials registered in ClinicalTrials.gov on or between May 20 and October 11, 2005. Trials were categorized as observational or interventional. Study 1 focused on the interventional trials (shaded). Trials were registered by the National Institutes of Health and other U.S. federal agencies, industry, and universities, foundations, and other organizations (denoted “University”). “Interval” denotes the period between May 20 and October 11. Panel B shows interventional trials registered in ClinicalTrials.gov on or between May 20 and October 11, 2005, according to whether they included at least one drug or vaccine intervention (if so, they are denoted “drug trials”). Study 2, which assessed the use of the Intervention Name field, focused on the trials that included a drug or vaccine (shaded). Panel C shows interventional trials registered in ClinicalTrials.gov on or between May 20 and October 11, 2005, according to whether they were registered by industry (denoted “industry”) or by other data providers (denoted “nonindustry”). Study 3, which assessed the use of the Primary Outcome Measure field, included only the trials that were registered by industry during the interval period (shaded).

STUDY 2: “INTERVENTION NAME” FIELD

We reviewed ClinicalTrials.gov records to determine patterns of completion for the ICMJE-required data element termed “Intervention Name.” FDAMA 113 mandates completion of this field, although it does not specify how informative the entry must be, thereby limiting our ability to enforce the use of specific drug names.

We reviewed Intervention Name fields to see whether the information provided gave clinically meaningful insight into the specific treatment that was being tested. For example, a preliminary review of records showed that nonspecific terms such as “investigational drug,” rather than the name of the drug under study, were occasionally used. We limited our review of this field to interventional trials of drugs or vaccines (Fig. 1B). Records were considered acceptable if they specified at least one drug name or unique company identifying serial number. We did not evaluate the completeness of information provided about comparison interventions in a study. For example, a record that lists in the Intervention Name field “acetylsalicylic acid compared with active comparator” would have been considered acceptable for the purpose of this study, even though the information contained was not as clinically meaningful as it would have been if specific names for both drugs had been given.

STUDY 3: “PRIMARY OUTCOME MEASURE” FIELD

We reviewed ClinicalTrials.gov records to determine patterns of completion for the field termed “Primary Outcome Measure.” This field requests information about the outcome measure used to determine the statistical power of the study. It reflects the primary effect that the intervention is designed to modify. The definition of this data element in the registry states that it should include the measure used and the time of measurement relative to the start of the intervention, such as “death at 180 days after the start of treatment.”²² This field has been available in ClinicalTrials.gov only since October 2004, and completion was initially mandatory for most nonindustry data providers, whereas it has always been optional for industry providers. (Before June 2005, completion of the field was mandatory for all non-IND studies, which accounted for 79 percent of the nonindustry studies and 4 percent of the industry studies. Completion of the field is now optional for all data providers.) To examine how the field is used by data providers in the absence of enforcement by ClinicalTrials.gov, we limited our analysis to industry-registered interventional trials registered between May 20 and October 11, 2005 (Fig. 1C).

We first tabulated the number of trials with and without any entry in the Primary Outcome Measure field. We stratified data for the top 20 pharmaceutical companies, ranked according to volume of U.S. drug sales.²³ We examined the relationship between completion of this field and the phase of the study. We also assessed the quality of the entries in this field by noting whether or not they specified a measure and a time point. This subjective assessment was made (by one of us) on a sample of the records for phase 2, 3, and 4 drug studies registered by the top 10 drug companies²³ during the interval between May 20 and October 11.

STATISTICAL ANALYSIS

We report primarily descriptive statistics. We used a chi-square test to examine the relationship between completion of the Primary Outcome Measure field and phase of study.

RESULTS

STUDY 1: ALL INTERVENTIONAL TRIALS

On May 20, 2005, there were 13,153 records in ClinicalTrials.gov; the number had increased to 22,714 as of October 11, 2005. This increase was largely attributable to a spike in registrations during the period immediately before and after September 13, 2005 (Fig. 2). Table 1 contains data on the number of trials registered according to key trial characteristics. There were increases in registered trials from all categories of data providers. The sharpest rise was in the category comprising universities, foundations, and other nongovernmental, nonindustry providers. The number of trials registered by commercial sponsors more than doubled, including an increase in the number of IND studies, from 2010 to 3516, and an increase in the number of non-IND studies, from 77 to 1348. Overall, the number of data providers increased from 667 to 1969 during this time. Among commercial sponsors, the number of companies registering trials rose from 328 to 575; among the latter were all of the top 20 pharmaceutical companies (according to volume of sales in the United States in 2005)²³ and 14 of the top 20 medical-device companies (according to estimated volume of global sales in 2004).²⁴

Figure 2. New Trials Registered in ClinicalTrials.gov, According to Week.

The figure shows the number of new registrations per week (beginning on the date indicated) from mid-May through early October 2005. The “Industry” category includes all commercial data providers; the “Federal” category includes the National Institutes of Health and other U.S. federal data providers; and the “University” category includes universities, foundations, and other providers.

Table 1.

Trials Registered at ClinicalTrials.gov on May 20 and October 11, 2005.

Variable	No. of Trials
	May 20, 2005	Oct. 11, 2005
Total	13,153	22,714
Type of trial
Observational	2,128	3,359
Interventional	11,025	19,355
Type of interventional trial
Provider category
Federal (including NIH)*	8,688	9,796
Industry	2,064	4,734
University, foundation, or other	273	4,825
Intervention category†
Drug‡	9,492	15,498
Device	143	755
Procedure	3,863	5,218
Behavioral or other	903	1,900

^*NIH denotes National Institutes of Health.

^†A trial record may include more than one intervention type.

^‡The “drug” category includes drugs and vaccines.

We examined the interval sample to determine whether there was a change in registration behavior coincident with the implementation of the ICMJE policy. This sample included 2670 interventional studies registered by industry: 6 percent were phase 1, 28 percent phase 2, 47 percent phase 3, and 19 percent phase 4 trials. FDAMA 113 requires commercial sponsors to register only trials performed under an IND application. However, among the trials added to the database during the interval examined, 45 percent were non-IND studies, as compared with only 4 percent on May 20, 2005. As of October 11, 2005, 59 percent of the 1167 data providers from universities, foundations, and other nongovernmental, nonindustry organizations were based outside the United States. During the interval period, 52 percent of the 5307 trials registered by these 1167 data providers were conducted outside the United States, as compared with 21 percent of the 249 trials registered by this sort of provider before May 20, 2005.

STUDY 2: INTERVENTION NAME FIELD

The Intervention Name field was completed with a specific entry in 100 percent of the nonindustry records at both time points. The percentage of industry records with a nonspecific entry dropped from 10 percent to 2 percent during the study period (Table 2). All the nonspecific entries at both time points were attributable to four drug companies: Merck, GlaxoSmithKline, Pfizer, and Lilly. On May 20, 2005, the percentage of trials with nonspecific entries in this field varied from 91 percent (Merck) to 3 percent (Lilly). Between May 20 and October 11, 2005, only two companies, GlaxoSmithKline and Pfizer, created new records with nonspecific intervention names, in 1 percent and 6 percent of their entries, respectively. Merck, GlaxoSmithKline, and Pfizer also added specific information to previously vague entries during the study period; Merck made the most dramatic changes, by reducing their number of nonspecific entries from 91 percent on May 20 to less than 1 percent on October 11 (Table 2). However, on October 11, there were noninformative entries in 21 percent of GlaxoSmithKline records and 11 percent of Pfizer records.

Table 2.

Number and Disposition of Records from Industry Providers for Interventional Trials with Nonspecific Entries in the “Intervention Name” Field.

Provider*	May 20, 2005	Interval Period (May 20–Oct. 11, 2005)				Oct. 11, 2005
	Records with Nonspecific Entries	Records Corrected with Addition of Company Serial Number	Records Corrected with Addition of Drug Name	Records Not Corrected	New Records with Nonspecific Entries	Records with Nonspecific Entries
	no./total no. of trials		no.	no./total no. of trials
Merck	120/132	25	94	1	0/52	1/184
GlaxoSmithKline	53/104	2	4	47	1/128	48/232
Pfizer	22/75	2	2	18	14/224	32/299
Lilly	3/96	0	0	3	0/136	3/232
Other industry	0/1619	0	0	0	0/1849	0/3468
Total	198/2026	29	100	69	15/2389	84/4415

^*Specific providers are listed in descending order of the number of nonspecific records as of May 20, 2005.

STUDY 3: PRIMARY OUTCOME MEASURE FIELD

Use of the Primary Outcome Measure field was assessed in the interval sample only. Information had been entered in this field in 2033 of 2670 records registered by industry (76 percent) during the study interval. Seventy percent of the records (range, 0 percent to 100 percent) from the top 20 drug companies included information in this field (Table 3). The rates of completion of this field were 77 percent for phase 1 studies, 79 percent for phase 2 studies, 76 percent for phase 3 studies, and 65 percent for phase 4 studies (χ² = 26.21, with 3 df; P<0.001).

Table 3.

Use of the “Primary Outcome Measure” Field by 20 Drug Companies from May 20 through October 11, 2005.

Rank According to U.S. Drug Sales*	Company	No. of Records with Primary Outcome Measure	Total No. of Records	Percentage of Records with Primary Outcome Measure
1	Pfizer	221	224	99
2	GlaxoSmithKline	63	66	95
3	Johnson & Johnson	57	63	90
4	Merck	9	46	20
5	AstraZeneca	51	52	98
6	Novartis	8	239	3
7	Amgen	65	70	93
8	Sanofi–Aventis	19	45	42
9	Bristol-Myers Squibb	53	60	88
10	Lilly	121	136	89
11	Wyeth	53	53	100
12	Abbott	19	34	56
13	Hoffmann–La Roche	0	13	0
14	TAP Pharmaceutical	22	22	100
15	Boehringer Ingelheim	48	48	100
16	Teva (Teva Neuroscience)	14	14	100
17	Schering-Plough	1	11	9
18	Forest Laboratories	1	1	100
19	Eisai	31	35	89
20	Watson	15	15	100
Total for the 20 companies		871	1247	70

^*Data on rank according to volume of U.S. sales are from IMS Health.²³

The clinical value of the information provided in the Primary Outcome Measure field varied. Table 4 shows five categories of quality based on the specificity of the information about the primary outcome measure and the inclusion of information about the time it was measured. The 657 phase 2, 3, or 4 records from the top 10 drug companies that had entries in this field were reviewed and assigned to one of these categories. Table 4 shows that 17 percent of the entries were vague, whereas the others had varying degrees of useful information.

Table 4.

Attributes of Entries in “Primary Outcome Measure” Field.

Attribute	Frequency (N = 657)*	Examples from ClinicalTrials.gov
	%
Vague	17	Clinical response Tolerability
Domain without specific measure	19	Glucose regulation Severity of symptoms of schizophrenia
Specific measure without time frame	23	Intravenous glucose-tolerance test Structured clinical interview — positive and negative syndrome scale No. of hospitalizations
Time frame without specific measure	10	Tumor response at 3 mo Freedom from progression at 2 yr Improvement in glucose control over 16-wk period
Specific measure and time frame	31	Change in glycosylated hemoglobin from baseline to 6 mo Mortality from any cause at 30 days

^*Frequencies are based on a review of 657 records from the top 10 drug companies, ranked according to data from IMS Health on the volume of U.S. sales.²³ Phase 2, 3, and 4 trials were included.

DISCUSSION

Our findings support the conclusion that ICMJE policy has had an effect on trial-registration practices. Among commercial sponsors, there was an increase in the registration of both IND and non-IND studies. Nonindustry data providers also dramatically changed their registration behavior around the time of the ICMJE deadline. The 73 percent increase in trials registered during this time was associated with a 195 percent increase in the number of data providers from around the world. Since these new providers seem to have registered in order to comply with ICMJE policy, it is likely that they will continue to register trials.

Examination of data-element usage in ClinicalTrials.gov suggests that the act of registration alone is not a good indicator of adherence to registration policies. When trial sponsors have the option of providing information of marginal clinical value in a particular data field, our findings show that some companies provide useful information and others do not. This heterogeneous behavior may indicate varying degrees of comfort with different levels of disclosure. For example, among data elements not examined in this trial, there has been a learning curve, with some companies being slower than others to provide mandatory items such as the name of the sponsor and the location of the trial.²⁵

Completion of the Intervention Name field is mandatory for all trials in ClinicalTrials.gov, but the use of specific terms has not been enforced. We determined that three industry data providers — Merck, GlaxoSmithKline, and Pfizer — used a nonspecific term, such as “investigational drug,” between 29 percent and 91 percent of the time in trials registered as of May 20, 2005. These three companies are ranked in the top five according to volume of U.S. drug sales. Lilly used nonspecific intervention names in 3 of its 96 entries (3 percent). In contrast, other data providers, including 571 other industry providers, entered specific information (either a name or a serial number) in this field in all their records. Between May 20 and October 11, only two drug companies, GlaxoSmithKline and Pfizer, used a nonspecific term in this field, and then only rarely. In addition, many of the previously identified nonspecific records were corrected with the addition of drug names or serial numbers.

Our assessment of the quality of information in the Intervention Name field is limited by our methods. Our search revealed only records that had an easily identified term, such as “investigational drug,” in lieu of a drug name. As a result, entries such as “tyrosine kinase inhibitor” or “antibiotic” were not captured in our search for nonspecific terms. In addition, we were not able to evaluate the degree to which interventions in all groups in a study were delineated. Such information is critical to the full description of a clinical trial. Structures for collecting and monitoring the quality of this information need to be developed.

The Primary Outcome Measure field has been available since October 1, 2004. Before May 20, 2005, this field was commonly left blank by industry and other data providers. Since then, 76 percent of industry records have included an entry in this field, although the percentages vary widely according to company (Table 3). In general, information in this field is more likely to be omitted for phase 4 trials. In addition, the quality and completeness of the entries vary with respect to standard attributes of outcome measures. The attributes presented in Table 4 are consistent with those identified in global standards, such as the Tripartite Harmonised ICH (International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use) guideline E3²⁶ and the ICMJE statement.⁸ Although we examined only industry records, the use of this field by all data providers will need to be monitored and discussed. It is not clear how ClinicalTrials.gov can best provide information about outcome measures to the full range of interested parties, including patients, clinicians, researchers, and policymakers. In the meantime, more structured guidelines for listing outcome measures might enhance the utility of data in ClinicalTrials.gov and other registries.

Evaluation of compliance with the legal mandate for trial registration, FDAMA 113, shows improving but imperfect compliance on the part of industry.²⁵ Although we cannot judge the degree of compliance with ICMJE policy, which is not legally binding, without knowing the number of clinical trials overall, our data indicate large increases in trial registration from all sectors. Some commercial organizations and other stakeholders note that the mandatory registration of exploratory trials (roughly, phase 1 and 2 trials) and the prospective disclosure of certain data elements, including intervention name and primary outcome measure, raise critical proprietary issues.²⁷ These concerns may explain some of the variations in registration practices that are evident in our data.