Press releases for Phase 2 clinical trial topline results: Have the objective pre-specified efficacy results been disclosed?

Phase 2 clinical trials are of vital importance in the drug development process as they usually gather preliminary evidence of efficacy of potentially new therapies and support the go/no-go decision for Phase 3 pivotal trials. Topline results of Phase 2 trials are typically first disclosed through press releases so that key stakeholders (patients and their advocacy groups, physicians, clinical trials practitioners, investors, etc.) can have timely access to a high level summary of the important findings. The sponsors of the trials often will save more detailed findings for future medical conference presentations and/or peer-reviewed journal publications, and as a result there may be an extended period of time where only the topline results are available on which stakeholders can rely. It is therefore critical for trial sponsors to release objective findings and avoid selective disclosure of favorable results.

We have reviewed a large number of press releases for Phase 2 clinical trial results. Given some of the common issues we have encountered during the review we would like to highlight the following points for trial sponsors to consider when drafting a press release:

• •If a sponsor decides to report the p-value for the primary endpoint analysis, the pre-specified analysis method associated with that p-value should be disclosed. The American Statistical Association recently [1] released a statement on p-values, which includes the definition of a p-value and some guiding principles on the reporting and interpretation of p-values. The statement pointed out that proper interpretation requires full reporting and transparency. To quote from the statement: “Conducting multiple analyses of the data and reporting only those with certain p-values (typically those passing a significance threshold) renders the reported p-values essentially uninterpretable”. In addition, it is important to clarify whether one- or two-sided p-values are reported. Particularly, if a one-sided p-value is reported, it should be clearly stated in the press release.
• •The analysis population should be clearly defined, and if some enrolled patients have been excluded from the analysis, both the criteria for the exclusions and whether these criteria were pre-specified prior to observing the trial results should be stated. We have encountered a number of cases where a press release mentions the total number of patients enrolled in a trial leaving readers with the impression that the analyses were performed with data from all the patients, only to later learn in a publication that a large percentage of patients were excluded from the analyses for various reasons. Common reasons for exclusions include 1) patients taking medications during the trial that may confound the outcome assessments; 2) patients randomized to the treatment arm having serum drug concentration levels below a certain threshold, leaving the sponsor to believe that there were dosing compliance issues; and 3) patients being assessed for efficacy not close enough to a specified study visit date. There is the potential for bias if some of the reasons are determined after observing the trial results. For example, for a 6-month efficacy assessment one patient may be assessed 5 days earlier and another patient may be assessed 6 days later. If the analysis requires patients to be assessed within 5 days of a particular assessment time point, then the latter patient will be excluded.
• •If results for a pre-specified subgroup are included, more details on the pre-specified subgroups should be disclosed. In particular, the number of subgroups and whether some of the subgroups were hypothesized to be more likely to demonstrate a treatment effect than others would help readers interpret the totality of the subgroup results.

To further elaborate on the importance of stating the pre-specified analysis method, we have seen many cases where the press releases claim positive trial results with small p-values, and we learned in subsequent presentations/publications that some post hoc and uncommon, if not invalid, analysis methods were used to generate the small p-values. We conducted a simulation study to illustrate the magnitude of the inflation of false positive rates when selecting the smallest p-value based on multiple analysis methods. In this study we simulated 10,000 randomized Phase 2 trials comparing an experimental treatment with the standard of care (SOC) with 50 patients per arm, where the experimental treatment is not superior over the SOC and both treatment arms have a true response rate of 40%. For each patient we considered three baseline variables, the first variable being binary with equal chance of taking either outcome (e.g. gender), the second variable being continuous with a uniform distribution taking values between 0 and 1, and the third variable being continuous with a standard normal distribution. For each simulated trial we analyzed the trial results with Fisher's exact test and logistic regression including one or more of the variables and their interactions with treatment and calculated the smallest p-value corresponding to these analyses. Our simulation results show that about 15% of the simulated trials had the smallest two-sided p-value being less than 0.05, and about 30% of the simulated trials had the smallest one-sided p-value being less than 0.05. For those stakeholders who consider a trial to be positive when the p-value passes the significance threshold of 0.05, the false positive rate has been tripled when the smallest two-sided p-value is presented and even sextupled when the smallest one-sided p-value is presented without further details. The false positive rate will be further inflated if more analysis approaches (e.g. stratified analysis or analyses conducted with some patients excluded) are considered. This example highlights the importance of disclosing the pre-specified analysis approach for a Phase 2 trial.

The Journal strives to promote the objective disclosure of clinical trial topline results. We welcome the publication of trial design articles that include detailed descriptions of the pre-specified analysis approach, and we welcome our readers to bring to our attention potentially misleading press releases. The global clinical trials community will benefit from more dedicated effort on the dissemination of objective clinical trials findings.