What’s to Be Found in the Wisdom of the Crowd?

Despite the passage of two decades since the publication of Early Goal Directed Therapy (EGDT), the optimal resuscitation strategy for patients with sepsis and septic shock remains unclear (1). EGDT, as well as three subsequent studies, all compared an intervention to “usual care” (2–4). A common limitation in these subsequent trials is that usual care, where it was measured, often did not differ greatly from EGDT. The nature of other key interventions, such as when to start a second vasopressor or corticosteroids, were largely uncharacterized (5). Relatively little evidence exists to guide actual strategy and how best to apply this data beyond a “norepinephrine first” approach (6).

In this issue of AnnalsATS, Bosch and colleagues (pp. 2049–2057) have published a methodologically rigorous investigation describing current practice patterns for the use of adjunctive corticosteroids and vasopressors in the treatment of septic shock (7). Their study describes real-world practice and applies high-quality methods to evaluate factors associated with differences in the threshold at which providers utilize adjunctive therapies. The authors reported significant hospital-level practice variation for the threshold of norepinephrine, which triggers administration of an adjunctive therapy, including a ninefold difference between hospitals in the threshold at which a second vasopressor was started. They report similar variation in the threshold for adjunctive corticosteroids. Practices for adjunctive vasopressor initiation thus appear to be overly dependent on shared provider preferences within hospitals rather than evidence or patient factors. This variation is perhaps more disappointing than surprising, reflecting the dearth of papers addressing this issue in the 20 years that have elapsed since EGDT’s publication (8). The only major trial of adjunctive vasopressor therapy in the last decade barely reported clinical outcomes, leaving an evidence vacuum for clinicians considering the role for the tested therapy in their patients (9).

Admittedly, it appears that intensivists are not entirely arbitrary. The favored second vasopressor was vasopressin, a practice supported by trial data and societal guidelines (6). Sicker and more comorbid patients were more likely to receive vasopressors before corticosteroids, which supports a pattern of using early additional vasopressors to meet hemodynamic endpoints in extreme acute illness. The pattern of later utilization of corticosteroids is also supported by evidence for their use in patients who do not respond to vasopressors (10). The greatest contribution of this study is the report of the commonalities observed, which comprises usual care. Most hospitals added vasopressin at around 10–30 μg/min of norepinephrine about 7 hours after starting norepinephrine, whereas corticosteroids were added around 5–15 μg/min of norepinephrine about 18 hours after starting norepinephrine.

Trial design is difficult. One challenge facing investigators is the design of a control arm. Proponents of usual care as a control contend that for an intervention to be deemed superior and to be adopted, it must be tested against usual care, which allows for determining safety and effectiveness. However, usual care is subject to unexplained clinical variation, may change with secular trends, and may be itself affected by the research environment (11). If a study demonstrated superiority of an intervention over usual care, it may be difficult to determine why. In contrast, omitting usual care in a study by comparing two therapies avoids the problems of usual care but risks irrelevance by testing two therapies not necessarily used in practice.

The two studies in critical care with perhaps the greatest clinical impact have taken different approaches: EGDT was compared against usual care, whereas the Acute Respiratory Distress Syndrome Network (ARDSNet) low tidal volume ventilation study omitted it (1, 12). Clinicians still struggle with how to incorporate these key studies into their practice. Suggestions for a three-arm trial, which compare two strategies to usual care, might offer more information but at the cost of decreased statistical power, increased duration and cost, or both (13). Many studies in critical care are barely powered to answer the question between two arms, let alone three.

Well-conducted observational studies like the one by Bosch and colleagues therefore not only offer a reference for clinicians but also simultaneously highlight evidence gaps—reflected in large variations in practices independent of patient factors—that are ripe for clinical trials. Studies that characterize usual care will help investigators design trials to fill these gaps by spelling out a control arm that approaches usual care. These trials will benefit from less unexplained variation and superior generalizability. Understanding usual care can also be an important step in assessing feasibility for an interventional study. For instance, a prospective planning study found that usual care was similar enough to the planned intervention that the contemplated interventional trial would have been infeasible (14).

Every July, new interns in the intensive care unit ask the questions, “When should I start steroids, and when should I start a second pressor?” Although this study cannot answer these questions, it can at least inform us of what everyone else does, which can shape behavior. In behavioral economics, “herding” is the phenomenon of individuals imitating group behaviors rather than independently determining their behavior. The economist John Maynard Keynes attributed this behavior as an individual’s response to perceived self-ignorance or perception that others were more informed (15). Herding can reinforce behavior as more individuals adopt that behavior. Herding has been prominent this past year, with various communities embracing unproven therapies advocated on social media, including the critical care community.

The work of Bosch and colleagues demonstrates that despite considerable unexplained variation, we are not immune to a herd mentality: We tend to follow guidelines when published, but the variation between hospitals indicates that much of our practice is determined by local culture and local experts. In fact, one unintended consequence of the publication of the work by Bosch and colleagues is that it may homogenize care. Readers may gravitate, wittingly or unwittingly, toward the published median thresholds when considering adjunctive therapies in their future patients or incorporate them into protocols, even though the authors did not attempt to link hospital-level practices with hospital-level outcomes.

It is regrettable that we are able to concretely describe usual practice in so few areas of medicine. Had we known what usual care was before studying EGDT or low tidal volume ventilation, we might have been able to refine the study protocols to be more effective. It is understandable why an investigator might prefer to forgo characterization of usual care: Characterizing practice may not seem as prestigious as testing an intervention. However, the majority of research results are wasted with poor generalizability, underpowered studies, and other methodological problems (16), leading journal editors to call for greater rigor in their submissions. The necessary first step for an interventional study is the characterization of usual care. The dearth of such observational studies is an impediment to high-quality science and high-quality care. We need more studies like that of Bosch and colleagues.