Abstract
How to cite this article: Sampath S. Data Analysis will not Result in Knowledge Production about Sepsis. Indian J Crit Care Med 2021;25(7):750–751.
Keywords: Data analysis, Fragility index, Knowledge, Scientific method, Sepsis
The Surviving Sepsis Campaign (SSC) guidelines are not new to controversy. Calls have been made asking for the guidelines to be retired, and the SSC has been accused of “making strong recommendations based on weak evidence”.1 A rebuttal has cited many studies with improved survival following SSC guidelines and concluded with emotional appeals that “our loved ones should be cared for in institutions” which presumably follow SSC guidelines.3 Predictably, further high-quality reviews were suggested to develop evidence-based guidelines.1
The review by Choupoo et al. in this issue of the journal attempts to assess the strength of the evidence behind some of the SSC guidelines.2 Fragility index (FI) and reverse fragility index (RFI) were the metrics in their review. This has restricted their study to randomized controlled trials (RCTs) with binary outcomes. The SSC guidelines are based on a variety of sources including RCTs. The authors have correctly emphasized that statistical significance and clinical significance are independent entities. They have avoided dichotomizing their results based only on arbitrary cutoffs of p values of 0.05. The rationale of assessing robustness of evidence behind SSC guidelines based only on FI and RFI could have been explained in more detail. An explanation for why this study was attempted and what gap it tried to fill in the SSC literature would have provided a necessary background and perspective.
Choupoo et al. have brought together FI and evidence behind SSC guidelines, and this has not been done previously in this topic. RCTs in critical care tend to be negative, and unsurprisingly the authors have shown that the same trends hold true in most of the RCTs which they have analyzed.4 Trials with and without statistical significance have been compared, but it is not clear at what p value the results have been tabulated. A meta-analytic synthesis with trial sequential analysis would have complemented this analysis and probably provided more information. Elements of the SSC guidelines which were amenable to such an analysis should have had their FIs tabulated. The study identifies the size of treatment effect as an important limitation. The study concludes by stating that RCTs which did not reveal statistical significance are more robust. What does robustness mean? Robustness implies that the direction of results may not change with changes in population characteristics. This suggests that RCTs which supported interventions are more likely to have their findings overturned in the future. The FI is in itself not “robust,” and the p value and confidence intervals will convey the same information and can be used with binary and other outcomes.5
How this study furthers growth of knowledge in sepsis has to be examined from a broader perspective. An ideal scientific theory of sepsis should have the following: (a) have clear definitions and a mathematical model, (b) functional forms between predictors in theory and outcome measures should be clearly specified, (c) appropriate data relevant to the above, and (d) robust statistical analysis of collected data that supports the theory.6 In a recent article, the lack of a clear objective definition of sepsis and substantial limitations of current approaches of identifying sepsis were highlighted.7 Clearly, data collection and statistical analyses, however robust, cannot assist in knowledge production when the theory and functional model are not clearly defined and measurable. The results of the study show that even in the limited subset of sepsis data analyzed, the data for nonsignificant trials are more robust.
A Google Scholar search of the phrase “further studies are indicated sepsis” showed over 700 articles. Clearly, the scientific community feels that more work (more data collection) has to be done in the field of sepsis. Developing testable theories and predictive models for complex systems like sepsis is difficult. It has been suggested that data-driven predictive analysis in ICU will create new knowledge from “big data” which is already being collected in ICUs as part of electronic health records.8 An opposing view point has been put that “too much data is just like no data at all” and a sound theoretical foundation is essential for knowledge production.9,10 Fifty years ago, Simpson pointed out that the application of statistical techniques without prior knowledge of causative mechanisms will lead to fallacious reasoning.11 We need breakthroughs in basic research of metabolic, immunological, and cytokine pathways to uncover the causative mechanisms of organ dysfunction in sepsis.11–14
Orcid
Sriram Sampath https://orcid.org/0000-0001-7521-7106
Footnotes
Source of support: Nil
Conflict of interest: None
References
- 1.Marik PE, Farkas JD, Spiegel R, Weingart S, Aberegg S, Beck-Esmay J, et al. POINT: should the surviving sepsis campaign guidelines be retired? Yes. Chest. 2019;155(1):12–14. doi: 10.1016/j.chest.2018.10.008. DOI: [DOI] [PubMed] [Google Scholar]
- 2.Choupoo NS, Das SK, Saikia P, Dey S, Ray S. How Robust are the Evidences that formulate Surviving Sepsis Guidelines? An Analysis of Fragility and Reverse Fragility of Randomized Controlled Trials that were referred in these Guidelines. Indian J Crit Care Med. 2021;25(7):773–779. doi: 10.5005/jp-journals-10071-23895. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Levy MM, Rhodes A, Evans LE, Antonelli M, Bailey H, Kesecioglu J, et al. COUNTERPOINT: should the surviving sepsis campaign guidelines be retired? No. Chest. 2019;155(1):14–17. doi: 10.1016/j.chest.2018.10.012. DOI: [DOI] [PubMed] [Google Scholar]
- 4.Laffey JG, Kavanagh BP. Negative trials in critical care: why most research is probably wrong. Lancet Respir Med. 2018;6(9):659–660. doi: 10.1016/S2213-2600(18)30279-0. DOI: [DOI] [PubMed] [Google Scholar]
- 5.Seligman H, Teixeira-Pinto A, Nowbar A, Francis D. Fragility of the bond between cardiovascular investigators and their readers. Circ Cardiovasc Qual Outcomes. 2019;12:e006271. doi: 10.1161/CIRCOUTCOMES.119.006271. DOI: [DOI] [PubMed] [Google Scholar]
- 6.Coveney PV, Dougherty ER, Highfield RR. 2016 Big data need big theory too. Philos Trans R Soc A. 374:20160153. doi: 10.1098/rsta.2016.0153. DOI: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Shappell CN, Klompas M, Rhee C. Surveillance strategies for tracking sepsis incidence and outcomes. J Infect Dis. 2020;222(Supplement_2):S74–S78. doi: 10.1093/infdis/jiaa102. DOI: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Carra G, Salluh JI, da Silva Ramos FJ, Meyfroidt G. Data-driven ICU management: using big data and algorithms to improve outcomes. J Crit Care. 2020;60:300–304. doi: 10.1016/j.jcrc.2020.09.002. DOI: [DOI] [PubMed] [Google Scholar]
- 9.Succi S, Coveney PV. Big data: the end of the scientific method? Philos Trans Roy Soc A. 2019;377(2142):20180145. doi: 10.1098/rsta.2018.0145. DOI: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Pearl J. Theoretical impediments to machine learning with seven sparks from the causal revolution. arXiv preprint arXiv:1801.04016. 2018 [Google Scholar]
- 11.Hernán MA, Clayton D, Keiding N. The Simpson's paradox unraveled. Int J Epidemiol. 2011;40(3):780–785. doi: 10.1093/ije/dyr041. DOI: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Van Wyngene L, Vandewalle J, Libert C. Reprogramming of basic metabolic pathways in microbial sepsis: therapeutic targets at last? EMBO Mol Med. 2018;10(8):e8712. doi: 10.15252/emmm.201708712. DOI: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hattori Y, Hattori K, Suzuki T, Matsuda N. Recent advances in the pathophysiology and molecular basis of sepsis-associated organ dysfunction: novel therapeutic implications and challenges. Pharmacol Ther. 2017;177:56–66. doi: 10.1016/j.pharmthera.2017.02.040. DOI: [DOI] [PubMed] [Google Scholar]
- 14.Chousterman BG, Swirski FK, Weber GF. Cytokine storm and sepsis disease pathogenesis. Semin Immunopathol. 2017;39(5):517–528. doi: 10.1007/s00281-017-0639-8. DOI: [DOI] [PubMed] [Google Scholar]