Sponsorship, conflict of interest, risk of bias, and reporting of participant's flow and baseline demographic information in studies applicable to the federal law to post the results in clinicaltrials.gov

1. Introduction

The US Congress passed the Food and Drug Administration Modernization Act and Amendments Act, mandating that investigators post protocols and the results of all clinical studies enrolling human subjects in the United States in the trial registry clinicaltrials.gov within 1 year after study completion [1], [2]. Sponsors and principal investigators who do not comply with the federal law are subject to financial penalties up to $10,000/day for failing to register or submit the results of trials. Numerous previous analyses suggested poor compliance with this policy [3], [4], [5], [6]. However, previous analyses were not able to precisely identify studies' applicability to federal law, because this information is not in the publicly available Web version of the trial registry, clinicaltrials.gov [4], [7]. The Aggregate Analysis of ClincalTrials.gov (AACT) database enabled us to analyze the entire set of registered studies and precise information about studies' applicability to federal law. We aimed to investigate sponsor compliance with federal law to post complete high-quality information about study and participant characteristics and outcomes. We assumed noncompliance when sponsors did not post the results or posted ambiguous data. The law clearly states a sponsor's responsibility for posting timely, complete, and accurate information about the study, sponsors, investigators, and participants [1], [2]. Therefore, we assumed sponsor responsibility was the sole reason and did not speculate on other reasons for not posting information about studies labeled by registry administrators as “applicable clinical trial.” We analyzed sponsorship, conflict of interest, study design and risk of bias, and reporting of participants' flow and baseline demographics based on provided information in the database extracts from the Clinical Trials Transformation Initiative (CTTI) (https://www.ctti-clinicaltrials.org/aact-database).

2. Methods

We downloaded, reformatted, and analyzed all studies available in the CTTI database as of May 2016 in the HPCC platform (High-Performance Computing Cluster, https://hpccsystems.com/). The CTTI database has 2 variables indicating whether a study is an “applicable clinical trial” as defined in US Public Law 110-85, Title VIII, Section 801 and whether a trial is a Food and Drug Administration–regulated intervention (Appendix Table 2). We relied on straightforward law language regarding sponsor responsibility for compliance. Therefore, in contrast with previous publications, we did not speculate or hypothesize what other factors can result in missing or ambiguous data [3], [5], [8].

Appendix Table 2.

Definitions of the data elements that are available for downloading from www.clinicaltrials.gov (* mandatory fields required by the federal law).

Field name	Definition of the data element	Utilization in our analysis
NCT ID*	The ClinicalTrials.gov identifier	Unique study identifier
Other IDs	Other identification numbers assigned to the protocol, including unique identifiers from other registries and NIH grant numbers	Not used
Title*	Official name of the protocol provided by the study principal investigator or sponsor	Not used
Acronym	Acronym or initials used to identify this study	Not used
FDA Regulated Intervention?	FDA Regulated Intervention? (FDAAA) Yes/No Definition: Indicate whether this trial includes an intervention subject to US Food and Drug Administration regulation under section 351 of the Public Health Service Act or any of the following sections of the Federal Food, Drug and Cosmetic Act: 505, 510(k), 515, 520(m), and 522.	Analyzed
Is section 801?	Section 801 Clinical Trial? (FDAAA) Yes/No Definition: If this trial includes an FDA regulated intervention, indicate whether this is an “applicable clinical trial” as defined in US Public Law 110-85, Title VIII, Section 801. Briefly, applicable drug trials include controlled clinical investigations, other than Phase I investigations, of a drug or biologic subject to US FDA regulation. Applicable device clinical trials are controlled trials with health outcomes of devices subject to FDA regulation, other than small feasibility studies, and pediatric postmarket surveillance.	Analyzed
Funded*	Funding source as industry, NIH, U.S. Federal Government, Network, or other	We categorized as industry funded or other funding
Sponsors*	Name of primary organization that oversees implementation of study and is responsible for data analysis	Analyzed
Recruitment*	#Enrolling by invitation: participants are being (or will be) selected from a predetermined population #Active, not recruiting: study is ongoing (i.e., patients are being treated or examined), but participants are not currently being recruited or enrolled #Completed: the study has concluded normally; participants are no longer being examined or treated (i.e., last patient's last visit has occurred) #Suspended: recruiting or enrolling participants has halted prematurely but potentially will resume #Terminated: recruiting or enrolling participants has halted prematurely and will not resume; participants are no longer being examined or treated #Withdrawn: study halted prematurely, prior to enrollment of first participant	We used the exact categories as reported in trial registry
Conditions*	Primary disease or condition being studied, or focus of the study. Diseases or conditions should use the National Library of Medicine's Medical Subject Headings (MeSH) controlled vocabulary when possible.	Normalized and analyzed
Study Types*	Interventional or observational studies	We used the exact categories as reported in trial registry
Study Designs	Purpose, phase, treatment allocation, masking of the treatment status; type of primary outcome or endpoint that the protocol is designed to evaluate	Analyzed
Phases*	Phase of investigation, as defined by the US FDA for trials involving investigational new drugs	We used the exact categories as reported in trial registry
Study Results	-Participant Flow -Baseline Characteristics -Outcome Measures and Statistical Analyses -Adverse Events Information -Administrative Information “Applicable clinical trials” generally include interventional studies (with one or more arms) of drugs, biological products, or devices that are subject to FDA regulation, meaning that the trial has one or more sites in the U.S, involves a drug, biologic, or device that is manufactured in the US (or its territories), or is conducted under an investigational new drug application (IND) or investigational device exemption (IDE).	We categorized the studies into 2 categories: with posted results and without posted results. We analyzed reporting of the number of participants who completed studies and the number of participants who discontinued the study due to adverse effects
Interventions*	-Drug (including placebo) -Device (including sham) -Biological/Vaccine -Procedure/Surgery -Radiation -Behavioral (e.g., Psychotherapy, Lifestyle Counseling) -Genetic (including gene transfer, stem cell and recombinant DNA) -Dietary Supplement (e.g., vitamins, minerals)	We categorized interventions as drug, procedure, radiation, biologics, or behavioral according to the categories in ClinicalTrials.gov
Outcome Measures	Specific key measurement(s) or observation(s) used to measure the effect of experimental variables in a study, or for observational studies, to describe patterns of diseases or traits or associations with exposures, risk factors or treatment.	Not used
Gender*	Physical gender of individuals who may participate in the protocol	We used the exact categories as reported in trial registry
Age Groups	Age of participants	We used the exact categories as reported in trial registry
Enrollment*	Number of subjects in the trial	We used the reported numbers, excluding ambiguous values e.g., enrollment values of more than 99,999 participants registry
First Received	Date the protocol information was received	Not used
Start Date*	Date that enrollment to the protocol begins	We calculated the length of studies as the time period between start and completion dates
Completion Date	Final date on which data was (or is expected to be) collected
Last Updated	Date the protocol information was updated	Not used
Last Verified*	Date the protocol information was last verified	Not used
Primary Completion Date*	The date that the final subject was examined or received an intervention for the purposes of final collection of data for the primary outcome, whether the clinical trial concluded according to the prespecified protocol or was terminated	We calculated the length of studies as the time period between start and primary completion date when completion dates were missing.
Has Expanded Access?*	Indicate whether any non-protocol access is to be provided for the investigational drug or device. If so, an Expanded Access record should also be created for this IND/IDE.	Analyzed
Accepts Healthy Volunteers?*	Indicate if persons who have not had the condition(s) being studied or otherwise related conditions or symptoms, as specified in the eligibility requirements, may participate in the study. Select Yes/No.	Analyzed
Maximum age*	Maximum age of participants.	Analyzed
Minimum age*	Minimum age of participants.	Analyzed
Organization's Unique Protocol id*	Unique identification assigned to the protocol by the sponsoring organization, usually an accession number or a variation of a grant number. Multiple studies conducted under the same grant must each have a unique number.	Analyzed
Primary Completion Date Type*	A “Type” menu is also included, with options Anticipated and Actual. For active studies, set Type to Anticipated and specify the expected completion date, updating the date as needed over the course of the study. Upon study completion, change Type to Actual and update the date if necessary.	Analyzed
Lead Sponsor or Collaborators*	Name of primary organization that oversees implementation of study and is responsible for data analysis. For applicable clinical trials, sponsor is defined in 21 CFR	Analyzed

We conducted frequency analysis of all fields required by the World Health Organization for the registration of clinical studies (Appendix Table 1). We identified clinical trials and conducted frequency analysis of the fields describing study sponsorship, conflict of interest by principal investigators, risk of bias in study design, and the reporting of participants' flow and baseline demographics. We also analyzed reporting of all mandatory fields required by the federal law (Appendix Table 2). For each study applicable to Section 801, we calculated proportions of missing data among all mandatory fields required by federal law.

Appendix Table 1.

(at http://www.who.int/ictrp/network/trds/en/index.html).

The minimum amount of trial information that must appear in a register in order for a given trial to be considered fully registered. There are currently 20 items in the WHO Trial Registration Data Set. It is sometimes referred to as the TRDS. 1.Primary Registry and Trial Identifying Number Name of Primary Registry, and the unique ID number assigned by the Primary Registry to this trial. 2.Date of Registration in Primary Registry Date the trial was officially registered in the Primary Registry. 3.Secondary Identifying Numbers Other identifiers, if any allocated by the Primary Registry. These include: ○ The Universal Trial Number (UTN) ○ Identifiers assigned by the sponsor (record Sponsor name and Sponsor-issued trial number (e.g. protocol number)) ○ Other trial registration numbers issued by other Registries (both Primary and Partner Registries in the WHO Registry Network, and other registries) ○ Identifiers issued by funding bodies, collaborative research groups, regulatory authorities, ethics committees/institutional review boards, etc.<listbend> All secondary identifiers will have 2 elements: an identifier for the issuing authority (e.g. NCT, ISRCTN, ACTRN) plus a number. There is no limit to the number of secondary identifiers that can be provided. 4.Source(s) of Monetary or Material Support Major source(s) of monetary or material support for the trial (e.g. funding agency, foundation, company, institution). 5.Primary Sponsor The individual, organization, group or other legal entity which takes responsibility for initiating, managing and/or financing a study. The Primary Sponsor is responsible for ensuring that the trial is properly registered. The Primary Sponsor may or may not be the main funder. 6.Secondary Sponsor(s) Additional individuals, organizations or other legal persons, if any, that have agreed with the primary sponsor to take on responsibilities of sponsorship. A secondary sponsor may have agreed to: ○ take on all the responsibilities of sponsorship jointly with the primary sponsor; or ○ form a group with the Primary Sponsor in which the responsibilities of sponsorship are allocated among the members of the group; or ○ act as the Primary Sponsor's legal representative in relation to some or all of the trial sites. 7.Contact for Public Queries Email address, telephone number and postal address of the contact who will respond to general queries, including information about current recruitment status. 8.Contact for Scientific Queries There must be clearly assigned responsibility for scientific leadership to a named Principal Investigator (PI). The PI may delegate responsibility for dealing with scientific enquiries to a scientific contact for the trial. This scientific contact will be listed in addition to the PI. The contact for scientific queries must therefore include: ○ Name and title, email address, telephone number, postal address and affiliation of the Principal Investigator, and; ○ Email address, telephone number, postal address and affiliation of the contact for scientific queries about the trial (if applicable). The details for the scientific contact may be generic (that is, there does not need to be a named individual): e.g. a generic email address for research team members qualified to answer scientific queries. 9.Public Title Title intended for the lay public in easily understood language. 10.Scientific Title Scientific title of the study as it appears in the protocol submitted for funding and ethical review. Include trial acronym, if available. 11.Countries of Recruitment The countries from which participants will be, are intended to be, or have been recruited at the time of registration. 12.Health Condition(s) or Problem(s) Studied Primary health condition(s) or problem(s) studied (e.g., depression, breast cancer, medication error). If the study is conducted in healthy human volunteers belonging to the target population of the intervention (e.g. preventive or screening interventions), enter the particular health condition(s) or problem(s) being prevented. 13.Intervention(s) For each arm of the trial record a brief intervention name plus an intervention description. Intervention Name: For drugs use generic name; for other types of interventions provide a brief descriptive name. ○ For investigational new drugs that do not yet have a generic name, a chemical name, company code or serial number may be used on a temporary basis. As soon as the generic name has been established, update the associated registered records accordingly. ○ For non-drug intervention types, provide an intervention name with sufficient detail so that it can be distinguished from other similar interventions. Intervention Description: Must be sufficiently detailed for it to be possible to distinguish between the arms of a study (e.g. comparison of different dosages of drug) and/or among similar interventions (e.g. comparison of multiple implantable cardiac defibrillators). For example, interventions involving drugs may include dosage form, dosage, frequency and duration. If the intervention is one or more drugs then use the International Non-Proprietary Name for each drug if possible (not brand/trade names). For an unregistered drug, the generic name, chemical name, or company serial number is acceptable. If the intervention consists of several separate treatments, list them all in one line separated by commas (e.g. “low-fat diet, exercise”). For controlled trials, the identity of the control arm should be clear. The control intervention(s) is/are the interventions against which the study intervention is evaluated (e.g. placebo, no treatment, active control). If an active control is used, be sure to enter in the name(s) of that intervention, or enter “placebo” or “no treatment” as applicable. For each intervention, describe other intervention details as applicable (dose, duration, mode of administration, etc). 14.Key Inclusion and Exclusion Criteria Inclusion and exclusion criteria for participant selection, including age and sex. Other selection criteria may relate to clinical diagnosis and co-morbid conditions; exclusion criteria are often used to ensure patient safety. If the study is conducted in healthy human volunteers not belonging to the target population (e.g. a preliminary safety study), enter “healthy human volunteer”. 15.Study Type Study type consists of: ○ Type of study (interventional or observational) ○ Study design including: ▪ Method of allocation (randomized/non-randomized) ▪ Masking (is masking used and, if so, who is masked) ▪ Assignment (single arm, parallel, crossover or factorial) ▪ Purpose ○ Phase (if applicable) For randomized trials: the allocation concealment mechanism and sequence generation will be documented. 16.Date of First Enrollment Anticipated or actual date of enrolment of the first participant. 17.Target Sample Size Number of participants that this trial plans to enrol in total. 18.Recruitment Status Recruitment status of this trial: ○ Pending: participants are not yet being recruited or enrolled at any site ○ Recruiting: participants are currently being recruited and enrolled ○ Suspended: there is a temporary halt in recruitment and enrolment ○ Complete: participants are no longer being recruited or enrolled ○ Other 19.Primary Outcome(s) Outcomes are events, variables, or experiences that are measured because it is believed that they may be influenced by the intervention. The Primary Outcome should be the outcome used in sample size calculations, or the main outcome(s) used to determine the effects of the intervention(s). Most trials should have only one primary outcome. For each primary outcome provide: ○ The name of the outcome (do not use abbreviations) ○ The metric or method of measurement used (be as specific as possible) ○ The timepoint(s) of primary interest Example: Outcome Name: Depression. Metric/method of measurement: Beck Depression Score. Timepoint: 18 weeks following end of treatment. 20.Key Secondary Outcomes Secondary outcomes are outcomes which are of secondary interest or that are measured at timepoints of secondary interest. A secondary outcome may involve the same event, variable, or experience as the primary outcome, but measured at timepoints other than those of primary interest. As for primary outcomes, for each secondary outcome provide: ○ The name of the outcome (do not use abbreviations) ○ The metric or method of measurement used (be as specific as possible) ○ The timepoint(s) of interest

Quality assurance of the trial registry uses automated business rules to detect missing data or inconsistencies in the data elements [9]. However, this methodology has limitations, including the lack of an independent source of study data to verify each data element [9]. As stated by the registry director, “posting does not guarantee that the record is fully compliant with either ClinicalTrials.gov or legal requirements.” [9] We detected missing or ambiguous data but did not contact the institutional review boards to confirm the accuracy of the data in ClinicalTrials.gov.

Following the federal financial disclosure guidance [10], we concluded that there was a conflict of interest when study sponsors were employers of principal investigators. We categorized study funding by industry for all studies funded by pharmaceutical or device companies exclusively or in combination with individuals, universities, or community-based organizations.

We defined high risk of bias in study design according to the criteria outlined by the Agency for Healthcare Research and Quality, including termination status, non-random allocation of study subjects, and open allocation status [11].

We identified all completed studies that did not post results on clinicaltrials.gov in compliance with federal law (Section 801 and criteria without posted results). We estimated the fee for noncompliance with federal law as a minimum $1/day up to a maximum $10,000/day for all days between the primary completion day and the end of 2014. We excluded from the cost-analysis studies completed in 2015–2016 to address a possible time lag between posting the results on the Web and in the downloadable database.

3. Results

We downloaded 217,258 studies from the database in the format of related tables. We linked all related tables by unique study, treatment, and outcome identifiers. We were able to analyze 211,437 studies. We identified 44,635 studies applicable to Section 801 (Table 1). Industry was involved in sponsoring 63% of these studies. The majority of the studies (75%) did not report employment of principal investigators by a sponsoring organization, and obvious conflict of interest was identified in 4% of the studies in which principal investigators were employed by sponsoring organizations (Table 1). The majority of the studies were randomized clinical trials (60%), but only 38% of all studies applicable to Section 801 were double-blind studies. More than 3000 studies (9%) applicable to Section 801 were terminated, mostly due to poor recruitment. Information about attrition of study subjects (18%) and treatment discontinuation due to adverse effects (8%) was available only in a small proportion of studies (Table 1). Baseline participant age was available only in 18% of studies, and enrollment of racial and ethnic minorities was available in less than 5% of studies (Table 1).

Table 1.

Sponsorship, conflict of interest, risk of bias, and reporting of participant's flow and baseline demographic information in studies that must comply with the US Public Law 110-85 to post the results in clinicaltrials.gov.

Study characteristics	Number of studies	% of total applicable studies
Applicable clinical trial as defined in US Public Law 110-85, Title VIII, Section 801	44,635
Industry involvement in sponsoring the study	27,995	62.7
Principal investigators are employed by sponsoring organization	1910	4.3
Principal investigators are not employed by sponsoring organization	9325	20.9
Employment of principals investigators by sponsoring organization is not reported	33,400	74.8
Risk of bias
Allocation of subjects: randomized	26,635	59.7
Allocation of subjects: non-random	5526	12.4
Allocation of subjects not reported	12,474	27.9
Terminated	3789	8.5
Double-blind study	17,089	38.3
Open Label study	22,748	51.0
Single Blind study	2357	5.3
Masking not reported	2441	5.5
Study design
Phase not reported	6998	15.7
Phase 0	292	0.7
Phase 1	3890	8.7
Phase 1/Phase 2	3063	6.9
Phase 2	14,534	32.6
Phase 2/Phase 3	1051	2.4
Phase 3	9553	21.4
Phase 4	5254	11.8
Participants flow
Reported attrition ( # of subjects not completed the study)	8133	18.2
Reported study discontinuation due to adverse effects	3366	7.5
Baseline patient characteristics
Reported baseline age of enrolled patients	8218	18.4
Both genders enrolled	38,379	86.0
Female studies	3782	8.5
Male studies	1855	4.2
Gender not reported	619	1.4
Reported # of Asian patients	1918	4.3
Reported # of African-American patients	1928	4.3
Reported # of Hispanic patients	486	1.1
Reported # of Native American patients	1481	3.3

We identified 29,992 applicable clinical trials (according to Section 801) that did not post results on clinicaltrials.gov. We restricted our analysis to 14,476 studies completed before 2015 that failed to post results in clinicaltrials.gov and therefore did not comply with federal law (Fig. 1). These 14,476 studies enrolled 3,660,385 participants. The results from 12,479 studies have not been published in journals; therefore, they are unavailable to the public. We estimated that the penalty of noncompliance would be from a minimum of $28.6 million to a maximum of $286 billion. We identified 22 sponsors that failed to post the results from >50 studies each (the list is available upon request). The majority of noncomplying studies were industry-sponsored (65.4%) randomized clinical trials (67.5%).

Fig. 1.

The number of the studies that did not comply with the federal US Public Law 110-85, Title VIII, Section 801about posting the results in clinicaltrials.gov by the primary completion dates as reported in clinicaltrials.gov (only studies completed before 2015 are included).

4. Discussion

In concordance with multiple previous publications, our findings indicated poor compliance with the federal law requiring certain studies to make results available [3], [5], [8]. There are several ongoing policy improvement efforts, including the Trial and Experimental Studies Transparency (TEST) Act, National Institutes of Health (NIH) policy, and growing public discussion about emergent need in sharing of clinical trial results [12], [13], [14]. The NIH recently issued a final policy mandating the registration and posting of the results of all NIH-funded clinical trials regardless of coverage by the Food and Drug Administration Amendments Act requirements [15]. The final rule is expected to promote public trust in clinical research, to fulfill an ethical obligation to the public and trial participants, to optimize the public investment in research, and to ensure accountability via the public reporting of information [16]. Much better enforcement is needed. Lack of accountability mechanisms result in enormous reporting bias in evidence analyses that inform policy, coverage, and clinical decisions [17], [18]. Policy efforts must address enrollment of racial and ethnic minorities and members of other underrepresented populations including elderly people, children, and women.

Our study has several limitations. We relied on information submitted by principal investigators and did not contact sponsors or investigators requesting submission of the missing data. We relied on the trial registry for identification of applicable studies and assumed noncompliance when the results were not available or ambiguous. We projected ranges of financial penalties and found no publicly available evidence of sponsor payments in cases of delayed posting.

Nevertheless, our analysis demonstrates a clear need for better accountability from sponsors and investigators who do not comply with the federal law about posting results in clinicaltrials.gov. The final rule estimated an additional cost of $56 million for the public and the sponsors related to study registration and the posting of results [16]. However, modern technology (http://www.lexisnexis.com/risk/) allows harmonization of institutional review board databases with trial registries. This harmonization will allow sponsors to submit study information once, providing instant analysis of compliance and the ability to check the quality control of the data as it is entered.