ALN-D-20-01584 Reply

To the Editor:

We thank Dr. Grocott for his interest in our article. We share his stated commitment to the core tenets of clear and transparent reporting that support scientific reproducibility. In pursuit of this commitment, we pre-registered (NCT02856594) and published the parent clinical trial protocol including key elements of the statistical analysis plan,¹ and clearly communicated the context of the analyses underlying our retrospective cohort sub-study.²

Dr. Grocott’s concerns primarily relate to our stated hypotheses and the accompanying inferential framework, finalizing our inferential framework after data access, and the positive results we emphasized. We contend that Dr. Grocott’s letter raises some of the challenges inherent to the closed peer-review process, and the need for continued education on the nuances innate to interpreting multivariable regression models. At initial submission, we hypothesized an association between cognitive impairment and burst-suppression during cardiopulmonary bypass. For our inferential framework, we constructed a covariate-adjusted logistic regression model. During the peer review process, it was rightfully suggested that an analysis of postoperative delirium was of interest to our specialty despite the double-blinded ongoing parent trial. Therefore, in our revised submission, we stated an additional hypothesis, “electroencephalogram burst-suppression during cardiopulmonary bypass mediates the effect of cognitive impairment on delirium.” Restating our initial hypothesis or increasing our sample size, at this stage, ran counter to our commitment to clear and transparent reporting (please see the limitation section of our discussion for the explicit acknowledgment that we powered our study to analyze the association between abbreviated Montreal Cognitive Assessment scores and burst suppression during cardiopulmonary bypass). Thus, our final analysis method was refined after the initial data were accessed to accommodate additional inferences on delirium. Indeed, we intended to convey this by the provided statement concerning the development of the statistical analyses after accessing the data. We acknowledge that it would have been even more precise to have stated that the analyses were modified during peer review.

It is important to note that we did not refer to any of our hypotheses as “the” hypothesis. This is because electroencephalogram hypotheses emanating from the parent trial are exploratory, as stated in our trial protocol.¹ We addressed both our hypotheses using a structural equation model framework, which implied two estimation stages. Thus, we did not deviate from the inferential framework we utilized at initial submission. Specifically, we estimated the association between burst-suppression during cardiopulmonary bypass and numerous variables in a multivariable model, including the abbreviated Montreal Cognitive Assessment, our cognitive variable of interest. This regression model should be interpreted as follows: adjusting for covariates of interest, the study authors asked whether cognitive impairment was associated with intraoperative electroencephalogram burst-suppression during cardiopulmonary bypass. The final model result, which addressed this hypothesis, is appropriately summarized in the causal diagram we presented in figure 3 (please see supplemental table 7 for univariate results).²

In the second stage of estimation, we examined the association between delirium and the same variables in the first stage with the addition of burst suppression. This regression model should be interpreted as follows: adjusting for covariates of interest, the study authors asked whether burst suppression was associated with delirium. We reported point estimates and 95% confidence intervals for this association, and others, to allow readers to evaluate our effect sizes and their plausible values. Thus, we fittingly minimized the sole use of p-values for inferences as we fully understand that the use of null hypothesis testing can be challenging in analyses with non-trivial model uncertainties. Draper³ provides additional background that helps with interpreting and assessing model uncertainties. Nevertheless, we reported False Discovery Rate p-values to help the reader interpret hypothesis tests where appropriate (i.e., univariate regression) throughout the manuscript.

We acknowledge that most studies are rarely definitive. As such, and as stated in our discussion,² our study would benefit from replication studies, including those that adjust for covariates such as dexmedetomidine or multi-component delirium prevention interventions. However, we believe that the burst-suppression findings and the potentially modifiable physical function findings we reported deserved due emphasis because they are biologically plausible and have clinical implications.