Non-inferior. By what margin?

Superiority, non-inferiority, and equivalence – these terms often appear in the rationale and reporting of randomized controlled trials, yet even seasoned scientific reader can find them confusing. However, understanding the differences between these concepts has become increasingly important for clinicians who must decide whether to update their clinical practices based upon the latest clinical trial data. In this issue of Anesthesiology, Dr. Sekela et al report a Phase III, double blinded clinical trial, The Red Cell Pathogen Inactivation (ReCePI) trial (NCT03459287), designed to examine whether amustaline/glutathione pathogen-reduced red blood cells were non-inferior to conventional transfusions during after cardiac surgery.¹ The primary outcome to assess non-inferiority was acute kidney injury (AKI) at 48 hours postoperatively. 321 randomized patients were transfused with study red blood cells, of which 159 received pathogen-reduced red blood cells and 162 received conventional red blood cells. The incidence of AKI by 48 hours was 46/157 (29.3%) in the pathogen-reduced, and 45/161 (28.0%) in the conventional arm (adjusted treatment difference 0.7%, 95% Confidence Interval [CI] −8.9, 10.4%). Importantly, the threshold for determining non-inferiority defined by the trial investigators was if the upper bound 95% CI of the adjusted treatment difference in AKI incidence between study groups was less than half of the incidence of AKI in the control arm, or a 14% absolute margin.

Non-inferiority trials are important and increasingly common. Classically, a non-inferiority trial may be designed to compare a new or alternative treatment with an existing one where the intention of the trial is to establish that there is not a clinically meaningful difference between the two treatments for a chosen outcome of interest. Notably, the new or alternative treatment may offer a potential benefit but may have a downside in efficacy or safety. The primary outcome selected for a non-inferiority study is typically an efficacy or safety endpoint pertinent to the specific characteristics of a treatment. In circumstances where the new treatment has a favorable profile – such as less burdensome duration of treatment (e.g. single dose versus multi dose regimen) or fewer side effects – it could also mean that it would be clinically acceptable if the new treatment has a slightly decreased efficacy or some other tradeoff. In these circumstances, what is the quantity of difference where it would be reasonable to conclude that specific outcomes of interest are “not that much worse”? This is the non-inferiority margin.

In many settings, this clinically relevant non-inferiority margin may be quite small, reflecting a narrow window for the clinically acceptable impact of a treatment on a particular outcome. In the current work, the authors examined if the potential benefit (which was not directly tested in this trial) of reduced pathogen exposure would come at the cost of an unacceptable increased risk of harm from acute kidney injury emerging from the impact of the pathogen reduction process on red blood cells.

It is crucial to understand the intended design of a clinical trial when interpreting the results – is the trial attempting to establish superiority, non-inferiority, or equivalence? In the case of non-inferiority trials, there are many established methods of determining the non-inferiority margin. Fixed-margin, point estimate, and syntheses methods are common methods for defining the non-inferiority margins for studies, but all have their limitations.² Dr. Sekela et al utilized the fixed-margin method, stating that non-inferiority would be, “claimed if the upper bound 95% CI of the treatment difference was less than half (50%) of the observed conventional arm incidence.” Once the trial was completed and the control arm AKI incidence was known to be 28%, this method established that the upper bound of the confidence interval for the difference could not cross 14% (28% * 0.50). Although this approach is a common selection for non-inferiority trials, it could be both a wide and clinically meaningful range and is challenging to understand as a clinician. What is the point estimate of the difference, if the 95% confidence interval could be as high as almost 14%? In the context of a trial with a modest sample size, primary outcome with a relatively high incidence, and a wide confidence interval, this is not obvious.

The authors use a conservative method of calculating the difference between groups, the Miettinen and Nurminen approach, which potentially increases this challenge of interpretation. This approach, in smaller sample size studies, yields larger confidence intervals than seen by other methods for calculating the adjusted treatment difference and may increase the risk of concluding that non-inferiority was not established.³

While the authors’ approach is statistically valid and is supported by prior literature,^2,3 the definition of a non-inferiority margin is not a statistical rule but ultimately a clinical decision. How different can a treatment be to be “non-inferior” or “not that much worse”? That emerges from clinical context. While transmission of an unknown pathogen is a potential hazard of transfusion, the United States blood supply is safe, subject to substantial regulation and testing for known blood borne diseases. In this context, the risk of a novel or untested for agent seems quite low, when stacked up against a very real risk of acute kidney injury and consequent harm including long term morbidity and mortality.^{4, 5, 6} In other settings around the world, with less effective blood testing regimes or extremely high incidences of blood borne pathogens,⁷ a larger difference between the two groups may entirely consistent with non-inferiority. In the US, just a few years ago, in the height of the unknowns of the COVID-19 pandemic, these potential risks may have causes others to weigh this balance differently and potentially choose a different non-inferiority margin.

As the authors note, there are potential added benefits of pathogen-reduced red blood cell transfusions, especially for patients undergoing repeated transfusions or susceptible to transfusion-associated graft versus host disease, although this benefit was not specifically studied or quantified in this study. In the context of this potential benefit and the outcome of interest, the authors selected an excellent study population, as the cardiac surgery population has a high incidence of AKI, commonly receives transfusions, and may undergo repeated transfusions. The authors do acknowledge the small (yet adequately powered) sample size, although stop short of linking this small sample size to the large 95% confidence intervals noted. Some readers may conclude that a primary limitation of the study is the statistically sound, but clinically very large non-inferiority margin.

In future trials, special attention should be paid to the definition of non-inferiority and balance the statistical and clinical relevance of the established effect sizes that would be considered non-inferior by the methods and definitions utilized in a trial. We must view any proposed difference from a clinical and context dependent perspective. In clinical research, just as in business, it’s important to ask: what’s the margin?