Challenges and opportunities in accessing trial data: an example from statin primary prevention trials

In 2013, the Cochrane Heart Group published an update to the systematic review of statins for primary prevention of cardiovascular disease (CVD). The objectives were to assess the effects, both harms and benefits, of statins in people with no history of CVD. Our results showed that statins reduced all-cause mortality (OR=0.86, 95% CI 0.79 to 0.94) and fatal and non-fatal cardiovascular events (RR=0.75, 95% CI 0.70 to 0.81) with no increased risk of adverse events (RR=1.00, 95% CI 0.97 to 1.03).¹

Despite consistent findings of the benefits of statins for primary prevention of CVD, the validity of these results has been challenged by some authors.² Observational studies have demonstrated associations between statin exposure and adverse outcomes,such as impaired cognition and dementia,³ but it is uncertain whether or not such relationships are causal given the non-randomised nature of these study designs. It is also uncertain whether these types of adverse events or other outcomes such as health-related quality of life, which was reported in only 2 of the 19 statin primary prevention trials as of 2013, were even collected in statin primary prevention clinical trials.

Since 2012, models have been developed to respond to calls for greater sharing of clinical trial data sharing from organisations such as the AllTrials campaign (alltrials.net) as well as the National Academy of Medicine⁴ and International Consortium of Medical Journal Editors.⁵ These principles for broader data sharing have garnered support from the Pharmaceutical Research and Manufacturers of America, the European Federation of Pharmaceutical Industries and Associations, the Biotechnology Industry Organization and even the United Nations.⁶ Data-sharing models include academic organisations serving as data hosts and intermediaries for requests, governmental institutions responding to information requests and pharmaceutical industry-sponsored sites where users can access trial data through secure platforms. New data-sharing models have been recently proposed by trialist groups.^{7 8}

While the published experience in accessing open data is limited and not without reported challenges,^{9 10} we sought to access unpublished statin trial clinical study reports and case report forms through available data-sharing mechanisms to update the 2013 Cochrane systematic review. We sought to understand if there were data collected but not yet reported on adverse events and health-related quality of life. Our hypothesis was that no new adverse event data were collected but that health-related quality of life data were collected given their ease and frequency of collection among other trials, including HOPE-3,¹¹ yet had not been yet fully reported. In this Viewpoint, we describe the challenges in accessing trial data.

We identified the pharmaceutical sponsors for statin primary prevention trials included in the 2013 review (n=6). Two of these sponsors (Novartis, Daiichi Sankyo) are cosponsors of the data-sharing website, http://www.clinicalstudydatarequest.com, which facilitates data requests for 13 companies. As of September 2016, data from statin primary prevention trials were not available through this site. Nevertheless, in April 2015, February 2016, and September 2016, we completed online inquiries for deidentified case report forms and clinical study reports from Novartis and Daiichi Sankyo, respectively. We submitted proposals to other sponsors (Bristol-Myers-Squibb (April 2015), Merck (June 2015), AstraZeneca (June 2015) and Pfizer (July 2015)) according to the guidelines on data sharing published on the companies' websites.

In August 2015, we also submitted inquiries to the Food and Drug Administration, European Medicines Agency, Medicines and Healthcare Products Regulatory of the UK and Health Products Regulatory Authority of Ireland for any copies of case report forms submitted as part of statin trials. Within the Cochrane Collaboration, we sought and received guidance from Drs Tom Jefferson and Peter Doshi who have published extensively on their efforts to obtain unpublished data.¹²

Of 10 requests, we received 8 responses, all of which were a decline of our request or a referral to publicly accessible documents; we are still awaiting responses from two companies. Of the companies that did respond, the response time varied from 1 to 11 months. Reasons for declining requests were related to company policies regarding sharing of case report forms (AstraZeneca), timing of trials that preceded the company's disclosure period that began in 2008 (Bristol-Myers-Squibb) or exclusion of statin-related trials in data-sharing agreements (Daiichi Sankyo, via clinicalstudydatarequest.com). Moreover, the retention period for the statin trials had expired, and the documents were unavailable. Quality-of-life data were also unavailable for another company (Pfizer).

The European Medicines Agency reported that the agency did not hold any clinical study reports related to any of the statin drugs because these drugs were not centrally approved medicinal products. Based on the European Medicines Agency's response, we proceeded to contact regulatory authorities in the UK and Ireland but learned that it was contrary to their policy to release information to third party individuals. The Food and Drug Administration directed us to publicly available information regarding approval documents for approved drugs. Approved statins (n=7) can be found through online searching (http://www.accessdata.fda.gov/scripts/cder/drugsatfda/) and contain original submissions, including summary safety data. However, data are pooled across diverse studies in these reports (ie, short-term and long-term trials, randomised and open-label trials, approved and unapproved doses, placebo and active comparator groups, (including other statins) coadministration with other lipid-lowering drugs, parallel group and **crossover trial designs), which made comparison of adverse event rates in intervention and placebo groups with published reports of long-term primary prevention trials infeasible (table 1). Some applications also included redactions of safety data. We found neither clinical study reports nor case report forms through searching these documents.

Table 1.

Reported adverse events in intervention and comparator groups among participants in statin primary prevention randomised controlled trials, updated to include HOPE-3 trial data published in 2016¹¹

	Statin		Usual care or placebo
Study ID	Events	Total	Events	Total	Weight (%)	Risk ratio (fixed effect)
Adult Japanese MEGA Study 2006	451	3866	397	3966	5.7	1.17 (1.03 to 1.32)
AFCAPS/TexCAPS 1998	1045	3304	1126	3301	16.3	0.93 (0.87 to 0.99)
ASPEN 2006	361	959	334	946	4.9	1.07 (0.95 to 1.20)
Bone 2007	413	485	102	119	2.4	0.99 (0.92 to 1.08)
CAIUS 1996	21	151	21	154	0.3	1.02 (0.58 to 1.79)
CARDS 2008	1390	1428	1376	1410	20.1	1.00 (0.99 to 1.01)
Derosa 2003	0	24	0	23		Not estimable
HOPE-3 2016	1643	6361	1633	6344	23.7	1.00 (0.95 to 1.06)
JUPITER 2008	1352	8901	1377	8901	20.0	0.98 (0.92 to 1.05)
KAPS 1995	107	214	95	212	1.4	1.12 (0.91 to 1.36)
METEOR 2010	583	700	226	281	4.7	1.04 (0.97 to 1.11)
PHYLLIS 2004	12	253	14	254	0.2	0.86 (0.41 to 1.82)
PREVEND IT 2004	13	433	22	431	0.3	0.59 (0.30 to 1.15)
Total (95% CI)		27 079		26 342	100.0	1.00 (0.97 to 1.02)
Total events	7391		6723

Heterogeneity: χ²=16.43, df=11.

(p=0.13); I²=33%.

Through other Cochrane reviewers, we received one clinical study report, including its corresponding case report form, from a small (n=335 participants) trial evaluating the effects of atorvastatin in patients with familial hyperlipidaemia. No data on health-related quality of life were collected. These documents are available on the Drug Industry Documents website (http://www.industrydocuments.library.ucsf.edu).

To update the 2013 Cochrane review on statins for the primary prevention of CVD, we have sought new data on adverse events and health-related quality of life from unpublished clinical study reports and case report forms. However, we received limited responses and were largely unable to find new sources of information to support or refute our hypothesis or to address concerns from individuals and groups who have challenged the validity of our previous results. Despite broad support for greater trial sharing, our experience suggests that the operational infrastructure for data sharing outside of trialist collaborations remains under development.

Data sharing of statin primary prevention trials may be particularly difficult since most, but not all, primary prevention trials were performed prior to the widespread use of electronic case report forms. The lack of infrastructure, costs, or heavy workload for data sharing from older trials may be potential reasons why companies have focused on more recent (and subsequent) trials for data sharing. On the other hand, data sharing for commonly used drugs such as statins may be even more important than newer drugs with lower market penetration.

We note recent examples where data sharing appears to be functioning. For example, since its inception in 2013, clinicalstudydatarequest.com has received 234 research proposals as of July 2016, and of these, 149 (64%) proposals were approved, although an unknown proportion of which are conditional approvals. The reasons for refusal are also published (n=12, 5%) including those that did not meet requirement checks, usually because the request did not include anonymised individual participant-level data such as ours. Only 4 (2%) of the requests have led to publications, but an additional 18 (8%) publications are in process. Yale's Open Data Access project hosts data from 123 Johnson & Johnson trials and approved 100% (29 of 29) requests for data access in its first year.¹³ The largest potential change could be through the European Medicines Agency, which aims to implement its policy for clinical report data sharing for all marketing approval applications by the end of 2017.

We summarise data-sharing mechanisms, examples, strengths and limitations in table 2. The potential for investigators to access data is greater than it has ever been; yet, these opportunities are not being fully leveraged. On the other hand, challenges remain, particularly the long lag time from request to data access⁹ and in accessing older trial data, clinical study reports and case report forms, the latter which would be useful to evaluate the potential for reporting bias.

Table 2.

Potential strengths and limitations of clinical trial data-sharing models

Data-sharing mechanism	Examples	Strengths	Limitations
Pharmaceutical company or sponsor-based secure portal for analysis	Clinicalstudydatarequest.com	Data remain secure, and analyses can be monitored by trial sponsors.	Execution of proposal review and data use agreements can be time-consuming.
Third-party dataholder+secure portal for analysis	Yale Open Data Access project	Scientific merit of proposed secondary analyses can be vetted by experienced, third-party investigators.	Execution of proposal review and data use agreements can be time-consuming; potential for demonstrating inaccurate or misleading results; privacy concerns for rare diseases and in very elderly.
Data available for download and analysis may be conditional on evidence of local institutional review board approval	National Institute of Health Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC); European Medicines Agency	Broadest access to data, which may facilitate new discoveries more quickly; concerns for analytical overlap were not borne out by 2017 SPRINT Open Data Challenge experience.	Potential for demonstrating inaccurate or misleading results; privacy concerns; conflicts of interest may not be fully vetted prior to data sharing.
Fee-for-service data sharing with trialists	No examples known at present; concept raised by International Consortium of Investigators for Fairness in Trial Data Sharing	Trialists are compensated for their work in the primary trial.	Investigators from low resource groups or settings may be inherently limited in accessing these data.

Data sharing is in its infancy, and our experience exposes some limitations of the current process. Individual participant data sharing represents an important component of potentially available data, but other data sources may also help provide a more complete picture of the trial performance. Researchers who have synthesised data from unpublished sources such as clinical study reports have called for updating of the 1995 guidelines from International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use to improve the format for accessing and analysing data from these and other sources.¹² Substantial investment will be needed to develop capacity and infrastructure among researchers, sponsors, companies, and patient and professional groups for data sharing that meets the needs of these diverse stakeholders, all while minimising unnecessary duplicative efforts.¹⁴ While some trialists have expressed concerns about research opportunism associated with data sharing, we argue that greater transparency will lead to greater trust and ultimately greater confidence in the beneficial and harmful effect estimates of medical interventions, including statins for the primary prevention of CVD. The optimal system for data sharing to achieve that goal remains in evolution.