Orange You Wishing There Were Definitive Randomized Controlled Trials Already?

Sepsis affects approximately 1,750,000 people in the United States each year with half requiring ICU admission and 20–30% of cases resulting in death (1). Among those who survive, many experience long-term physical, emotional, and cognitive dysfunction (2). Improvements in sepsis outcomes have largely been the result of early recognition and initiation of care bundles that incorporate early antibiotics, targeted fluid resuscitation, and hemodynamic support with vasopressors. Guidelines additionally advocate for the consideration of corticosteroids in some patients (3). Despite these interventions, sepsis continues to account for up to 50% of all in-hospital deaths in the United States (4), and outcomes worldwide are similarly bleak. For these reasons, there is enthusiasm for interventions that may offer a therapeutic benefit, especially if affordable and associated with few adverse effects.

One candidate therapeutic that has gotten much attention over the last 5 years is high-dose IV vitamin C. Enthusiasm for this potential therapeutic has grown due to biological plausibility, demonstrated deficiency in the setting of critical illness, and the suggestion of clinical benefit in early randomized controlled trials (RCTs) (5–8). In some studies, vitamin C is supplemented with thiamine, another essential micronutrient frequently deficient in the critically ill, and corticosteroids, which have an established hemodynamic benefit and may act synergistically with vitamin C (7). Several RCTs have now been completed to assess the “metabolic resuscitation cocktail” using some combination of these drugs compared with placebo regimens with or without corticosteroids. Unfortunately, none have been practice-changing due to the small numbers of patients enrolled.

In this issue of Critical Care Medicine, two systematic reviews with meta-analyses (SRMAs) (9, 10) examining the effect of high-dose IV vitamin C and metabolic resuscitation in patients with sepsis are presented. The review by Sato et al (9) includes 11 RCTs (n = 1737) in which patients were randomized to high-dose IV vitamin C with standard of care versus standard of care alone. The main outcome of interest was short-term mortality, whereas secondary outcomes included duration of vasopressor use and change in Sequential Organ Failure Assessment (SOFA) score between baseline and 72–96 hours. Investigators also examined the relative effect of vitamin C based on severity of sepsis and dose of vitamin C (higher dose ≥ 25 mg/kg every 6 hr vs lower dose 1.5 g every 6 hr) through subgroup analyses. Although no mortality benefit was observed, patients randomized to the intervention groups were dependent on vasopressors for less time and exhibited a greater reduction in SOFA score at 72–96 hours. Other outcomes were not different between groups.

The review by Assouline et al (10) included eight RCTs (n = 1,335 patients) in which the intervention groups received the full metabolic resuscitation cocktail (i.e., vitamin C, thiamine, and corticosteroids), whereas patients in the control groups received standard of care, which could include corticosteroids. In this analysis, the main outcome of interest was change in SOFA score at 72 hours. Secondary outcomes included duration of vasopressor requirement, 28-day mortality, and a composite outcome indicative of kidney injury (Kidney Disease: Improving Global Outcomes stage 3 or need for renal replacement therapy). As in the review by Sato et al (9), vitamin C was associated with a decrease in SOFA score and a shorter duration of requiring vasopressors. Again, there were no differences in other outcomes including mortality.

Both reviews conducted risk of bias (RoB) assessments, which is appropriately a standard expectation for systematic reviews (11). These assessments rate the quality of study designs in an attempt to clearly highlight the risks of bias for the included studies. Although casual readers may look only at the summary forest plots of a systematic review, such plots may include studies with important flaws or limitations. Ideally, subgroup analyses should be conducted examining whether relative effects differ between studies at low RoB versus those at high RoB. If relative effects and conclusions differ qualitatively, then reviewers may either lower their certainty in conclusions or choose to emphasize analyses from only studies with low RoB. Importantly, RoB assessments can be subjective, so summary figures should also be considered with this in mind.

Evaluation of overall certainty of effect estimates generated by meta-analyses can be useful to contextualize results for readers. Summary estimates with issues related to bias (as described above), imprecision, or inconsistency may be less certain than those without these issues. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework can be useful in assessing certainty in pooled estimates. Although both reviews (9, 10) discussed here transparently report RoB assessments, only the study by Assouline et al (10) used GRADE to assess the overall certainty (moderate) in the effect estimate for the main outcome of interest (change in SOFA score). It would be valuable to know the level of certainty for other outcomes of interest as well, but this was not reported.

Of note, Assouline et al(10) used trial sequential analysis (TSA) to evaluate the precision of point estimates (12). Similar to traditional RCTs using frequentist statistics, SRMAs are at risk for type-1 and type-2 errors. TSA is a statistical tool used in meta-analyses that considers the number of participants enrolled from contributing trials, event rates, effect sizes, maximum acceptable risk for a type-1 error, desired power, and the impact of repeated testing. These data are analyzed to estimate a required information size and boundaries of significance (benefit or harm) and futility (Fig. 1). The boundaries, at a given information size (i.e., number of participants in the analysis at the time a trial is included), are wider when accrued participants are few in number to reflect greater uncertainty of point estimates. At these lower numbers, a greater cumulative effect (z score) has to be observed to achieve statistical significance. As more participants are accrued, the boundaries for efficacy and harm narrow, and those for futility widen. In essence, the TSA is akin to a post hoc sample size calculation to evaluate whether the required information size for the meta-analysis is reached. If not, the likelihood that any of the findings are due to chance alone is higher.

Figure 1.

In trial sequential analysis (TSA), the cumulative z score (y-axis) is plotted against number of participants. The Z-curve is a plot of the cumulative z score as each trial is added to the analysis. In an analysis that favors an intervention, the Z-curve will cross the TSA boundary of benefit (Z-curve A). In an analysis that is unlikely to ever demonstrate benefit or harm, the Z-curve will cross into the futility area (Z-curve B). In an analysis that favors the control, the Z-curve would cross the TSA boundary of harm.

In the study by Assouline et al (10), the boundary for efficacy is crossed for the outcomes of change in SOFA score at 72 hours and for vasopressor duration. Because the required information size was exceeded, it is less likely the results were due to chance, and it increases the certainty in the overall finding. However, although there were no issues with precision given these TSA results, the finding of a reduced SOFA score with metabolic therapy was still rated down to moderate certainty due to RoB issues in the included studies.

Where does this leave us? First, it is important to note that decreased duration of vasopressors will also almost certainly decrease SOFA score. Consider that norepinephrine at any dose adds 3 points to the SOFA score compared with a normotensive patient who does not require vasopressors. Therefore, the improvement in SOFA scores at 72–96 hours observed in both analyses is, at least in part, attributable to the cessation of vasopressors.

Second, Assouline et al (10) make a reasonable argument that the beneficial effect on duration of vasopressors is not due to steroids alone. Patients in the control groups of many of the included studies were either allowed to receive corticosteroids or were given them per protocol. And, analyses limited to control group patients who received corticosteroids still show earlier cessation of vasopressors in the intervention groups (13, 14). Indeed, vitamin C has previously been shown to improve endothelial integrity, function as a stress hormone, augment native and exogenous vasopressor responsiveness, and is a required cofactor for the production of endogenous catecholamines (7). For these reasons, it is plausible that patients in the intervention groups required vasopressors for a shorter duration due to vitamin C. The importance of this outcome to patients, when considered in isolation, is debatable.

Third, both sets of authors suggest that reductions in SOFA scores are clinically significant and can reasonably be assumed to act as a surrogate for mortality benefit (15). However, it is notable that Assouline et al (10) report a pooled risk ratio for mortality of 1.02 (95% CI, 0.86–1.20). Using TSA, they report a required information size of 3,109 patients for this endpoint, a threshold that was not met; Sato et al (9) report a risk ratio of 0.88 (95% CI, 0.73–1.06). Given that the CIs from both reviews do not exclude important benefit or important harm, and because the required information size of TSA in the study by Assouline et al (10) is not met, we walk away from these results with ongoing uncertainty regarding the true effect of vitamin C on mortality with no clear evidence of benefit given the currently available data.

Favorable trends in duration of vasopressor use and SOFA scores will likely perpetuate further study of vitamin C and metabolic resuscitation in patients with sepsis. Fortunately, more data are coming with the anticipated randomization of another 1,900 patients with sepsis to treatments including vitamin C versus placebo in the next 2 years (NCT04747795, NCT03592693, NCT03680274, NCT03872011). Until these results are available, we do not believe there is sufficient evidence to support the routine use of high-dose IV vitamin C ± thiamine and steroids for patients with sepsis.