Update in statistical analysis plan of the RENOVATE trial

To the Editor:

We are writing to report an update to the statistical analysis plan (SAP) of the randomized clinical trial comparing high flow nasal cannula (HFNC) versus non-invasive positive pressure ventilation (NIPPV) in acute respiratory failure (ARF) (RENOVATE) trial protocol, recently published in the journal. This update to the SAP of RENOVATE (“High-flow nasal catheter therapy versus noninvasive positive-pressure ventilation in acute respiratory failure”) is necessary because of changes in the distribution of causes of acute respiratory failure among patients entering the trial, resulting from the COVID-19 pandemic beginning at the end of 2019. On November 2022, patients with COVID-19 comprised 63% of our trial population, followed by 19% with hypoxaemic acute respiratory failure without immunocompromise, 8% with cardiogenic acute pulmonary oedema (APE), 8% with hypoxaemic acute respiratory failure and immunocompromise, and 2% with acute chronic obstructive pulmonary disease (COPD) exacerbations. These proportions are substantially different from the design assumptions made in February 2019 and used in trial simulations published in our protocol and the SAP and published in this journal in March 2022 Volume 24 Number 1.

These differences were reviewed by the trial steering committee without knowledge of comparative outcome data. The steering committee decided to aggregate all COVID-19 patients, independent of previous group allocation, into a new group for analysis. Trial simulations were updated to account for this new group and for time-varying effects on outcome. The revised statistical analysis model is included as supplementary aterial accompanying this letter. While the revised statistical design can no longer be considered prespecified, it (i) is a good-faith effort to match the original spirit of the design while addressing the emergence of the COVID-19 population and (ii) changes were recommended by the steering committee that was and continues to be unaware of comparative (unblinded) outcome data.

The primary changes to the design were the following:

• 1.Aggregation of all COVID-19 patients in the new COVID-19 group, in addition to the four prespecified groups: hypoxaemic nonimmunocompromised, cardiogenic acute pulmonary oedema APE, hypoxaemic immunocompromised, and chronic obstructive pulmonary disease;
• 2.Introduction of a time-varying effect for the COVID-19 group to account for a potential change in standard of care and outcomes for COVID-19 patients over time; and
• 3.Adjustments in thresholds for stopping the trial for noninferiority, efficacy, or futility were performed to achieve desired operating characteristics with the new model structure (see supplementary material).

The original design used a mixture model with two different cluster patterns to allow for potentially differential effect in the APE group. The updated design uses a mixture model but with four different cluster patterns to allow for the possibility of a differential effect in the COVID-19 and APE groups. To keep the type I error rate for the “null superiority” (treatment is equivalent to control in all groups) and “null noninferiority” (the treatment effect is at the noninferiority margin of 0.442 [in log-odds] in all groups scenarios) similar to the original design, the stopping rules for declaring success and noninferiority were made more stringent as shown in supplementary material. Simulations were performed to assess operating characteristics of the updated design. The simulations did not attempt to account for past decision made in the trial, but were rather simulated as if the trial were operating in this manner from the beginning. Power was slightly decreased for the subgroups for the main noninferiority hypothesis, compared to the original design, except in the COVID-19 group where it is higher than 90% and in the APE group where a notable drop was seen. The Type I error rate is comparable to the original design. Adding time-varying effect for the COVID-19 group did not meaningfully change the power of the trial (see the supplementary material).

ClinicalTrials.gov: NCT03643939.

Ethical approval: CONEP n. 3.734.371.

Competing interests

Drs. Fitzgerald and Lewis declare that they are employees of Berry Consultants, LLC, a statistical consulting firm that specializes in Bayesian adaptive and platform trial design. All other authors declare that they do not have any potential conflict of interest in relation to this manuscript.

Funding

PROADI-SUS/Hospital do Coração (HCOR). Brazilian Ministry of Health.

Appendix A. Supplementary data

The following is the Supplementary data to this article: