Premise for Standardized Sepsis Models

Abstract

Sepsis morbidity and mortality exacts a toll on patients and contributes significantly to healthcare costs. Preclinical models of sepsis have been used to study disease pathogenesis and test new therapies, but divergent outcomes have been observed with the same treatment even when using the same sepsis model. Other disorders such as diabetes, cancer, malaria, obesity and cardiovascular diseases have used standardized, preclinical models that allow laboratories to compare results. Standardized models accelerate the pace of research and such models have been used to test new therapies or changes in treatment guidelines. The National Institutes of Health (NIH) mandated that investigators increase data reproducibility and the rigor of scientific experiments and has also issued research funding announcements about the development and refinement of standardized models. Our premise is that refinement and standardization of preclinical sepsis models may accelerate the development and testing of potential therapeutics for human sepsis, as has been the case with preclinical models for other disorders. As a first step towards creating standardized models, we suggest 1) standardizing the technical standards of the widely used cecal ligation and puncture model and 2) creating a list of appropriate organ injury and immune dysfunction parameters. Standardized sepsis models could enhance reproducibility and allow comparison of results between laboratories and may accelerate our understanding of the pathogenesis of sepsis.

1. Potential benefits of Standardized Sepsis Models

• Standardized sepsis models should allow comparison of results between laboratories, increasing scientific integrity and directly addressing the data reproducibility mandate from the NIH.

• Standardized sepsis models should ensure that the heterogeneity in the response is due to the disease process, rather than technical differences between laboratories.

• New investigators entering the field of sepsis, or new lab members, may be trained to standardized models to ensure data reproducibility.

• Testing therapeutic interventions may proceed more quickly and reproducibly with standardized models.

• Fewer experimental animals may be required when results could be compared to standardized models, as proposed for other preclinical models.

• Defining appropriate organ injury and immune dysfunction parameters will allow consistent reporting of results, without dictating that those parameters need to be measured.

• A web-based repository of results, such as Research Electronic Date Capture (REDCap), could be created for sharing information, and become an integral part of the resource sharing plan for grant applications.

• Standardization should be applied to more than one sepsis model. No single sepsis model recapitulates the heterogeneity of sepsis, similar to how a single cancer model cannot replicate all cancer biology. For example, preclinical breast cancer models allowed the testing of antibodies directed against the epidermal growth factor receptor (Herceptin® and Perjeta®) which are now in routine clinical use. Although these revolutionary drugs will not reduce mortality in other types of cancer, they greatly improve survival in patients with certain types of breast cancer.

2. Introduction – The problem of sepsis

Sepsis imposes a substantial burden on society with an annual estimate of 31.5 million cases and 5.3 million deaths across the world (1). The mortality of sepsis has decreased in the past decade (2) although this may be due to sepsis being diagnosed earlier and more frequently (3). The 28 day mortality still remains above 20% even with optimal care (4), excess mortality continues after 28 days (5) and many sepsis survivors do not return to full function or paid employment (6, 7). More needs to be done beyond searching for the “magic bullet” that will cure sepsis (8). Standardized sepsis models may accelerate the discovery of disease pathways and testing of new therapies.

3. Benefits of Standardization

Numerous examples demonstrate that standardization, driven by checklists, provides better results in disciplines ranging from academic libraries (9) to clinical care (10). Checklists have been proposed to standardize animal experiments to improve rigor and reproducibility (11). Standardization has been applied to diseases such as diabetes, which has been investigated using more than one standardized model. For example, there are more than two dozen transgenic murine models of type I diabetes (12) and a “roadmap” was proposed to allow better interpretation of therapies for type I diabetes. Importantly, this roadmap did not suggest a single model. Table 1 provides examples of pre-clinical models that have been standardized or proposed for standardization. The NIH recognizes the need to develop standardized animal models to ensure reproducible results with several Research Funding Announcements addressing a range of diseases. For example, PAR-16-215 on developmental disorders specifically requests “… rigorous, controlled and standardized preclinical animal” (emphasis added).

Table 1.

Standardized pre-clinical models of human disease. This table lists pre-clinical models that are either already standardized or have been proposed for standardization. Included in the table are examples with suggested reporting standards.

Malaria (56) Type I diabetes (12, 57) High fat feeding to mice to induce type 2 diabetes (57) Stroke studies (17) Bone fractures (58) Post-herpetic preclinical pain model (59) Diet induced obesity (60) Atherosclerosis (61) Epilepsy (15)

There is value in standardizing preclinical models of disease, analogous to how standardization delivers better quality health care. Healthcare workers are in a unique position to understand the benefits of standardization. When they were surveyed about the benefits of animal research, respondents felt that results should be reproducible between laboratories (13). Despite the importance of standardization, an analysis of dozens of manuscripts found that the published methodology was not sufficient to verify standardization of the experiments (14). Standardized sepsis models would address these issues, allowing investigators to include a simple statement in the methods such as “The studies were performed using a standardized sepsis model”, and then detail if any modifications were incorporated.

The creation of a web-based repository of results, such as REDCap, could provide an additional benefit for sepsis studies. Investigators using pre-clinical models of epilepsy proposed creating a REDCap database with common design elements to allow harmonization of preclinical studies (15). One of the goals of the epilepsy proposal was to assist underpowered preclinical studies by providing a database for comparison. A REDCap database of sepsis results, such as changes in white blood cell counts, would centralize data to allow investigators quick access to data.

4. Initiatives for standardization in non-sepsis preclinical models

There have been initiatives to increase standardization and rigor in preclinical studies in other areas such as cardiovascular disease to increase the reproducibility of the results. A 2017 paper (16) analyzed the level of rigor in preclinical cardiovascular studies by examining nearly 30,000 published studies. The authors observed significant shortcomings in the description of experimental details and noted that the reporting had not improved over the last 10 years. The only cardiovascular pre-clinical studies showing improvement were in the area of stroke. A checklist for stroke studies has been published (17) resulting in the publication of higher quality science (18). Several authors have proposed checklists to standardize other animal models (11, 19).

The NIH has supported standardized approaches for conducting experiments and reporting results. The NIH has used the R funding mechanism to support studies to develop criteria for new models. For example, R01 HL126429 created a central IRB model called ‘IRBchoice’ (IRBchoice.org). Another study was funded to determine the criteria when research results should be returned to patients (R21 HG00612). Despite not being hypothesis driven, both of these programs were supported to increase standardization. The NIH has also launched an initiative to improve the rigor of data and reproducibility of results (20, 21). Grant proposals are now expected to address data reproducibility in the research plan and a specific page needs to specify authentication of key resources. Even with these initiatives, NIH leaders have raised concerns about preclinical research and “subtle changes in protocol” (20). There was specific praise for the use of a checklist when reporting methodological details.

5. Prior standardization successes using non-sepsis pre-clinical models

An important element of the premise is that preclinical models of disease will provide information that actually translates into better patient care. The literature was reviewed to determine if findings from pre-clinical models of disease have been reproduced in clinical trials. A review of 2000 animal studies showed that 37% were replicated in subsequent human trials (22). While not all studies could be reproduced, there are notable successes when translating results from preclinical studies when standardized disease models were used (23) which are listed in Table 2. Included in this group are the Utstein criteria for resuscitation research (24, 25).

Table 2.

Successes using pre-clinical models of disease. This table lists examples where preclinical models of disease resulted in specific effective therapies or approaches to the study of diseases.

Disease	Clinical intervention
Polycystic kidney disease (62)	Tolvaptan (63)
Paroxysmal nocturnal hemoglobinuria (PNH) (64)	Eculizumab both PNH and hemolytic uremic syndrome (65, 66)
Resuscitation research Utstein criteria for animal models (24, 25)	Registries for tracking outcomes of clinical care (67)

6. Applicability of preclinical sepsis models to human sepsis

There has been robust discussion about the applicability of murine models of acute inflammation for the study of human disease. A high profile paper showing a lack of correlation of the gene signatures in mice and men after inflammatory insults (26) triggered vigorous debate (27). One rebuttal manuscript provided more than two dozen examples where animal models predicted the human responses to sepsis (23), including the failure of TNF inhibitors for the treatment of sepsis before the clinical trials (28). A 2017 paper highlighted the heterogeneity in the cause of death among septic patients where a careful analysis showed that the majority of septic patients die from either refractory shock or changing to comfort care only (29). Standardized animal models should replicate the heterogeneity of human sepsis, and ensure that different responses are part of the septic response rather than differences in the technical aspects of the procedure to cause sepsis. Standardized sepsis models may also help identify specific subgroups who will benefit from therapeutic interventions (30).

A review of pre-clinical sepsis models concluded that sepsis models are heterogeneous, but standardization may reduce the observed differences between animal and human studies, enhancing translational value (31). This paper identified areas of concern including the duration of the experiment, monitoring of animals and use of supportive therapy, issues in virtually all preclinical sepsis models. Numerous examples demonstrate that lack of standardization leads to discordant results. The cecal ligation and puncture (CLP) is a widely used model of sepsis that has been used to test different therapies to improve survival (32, 33). Table 3 gives five examples where divergent results were obtained when testing a potential treatment option or examining the role of a specific molecule in the pathogenesis of the disease. For example, the role of the cannabinoid 2 receptor was examined using knockout mice. One study showed improved survival (from 20 to 60% (34)) while another showed a decline (from 60 to 20% (35)). There were two important variations in the CLP procedure that probably accounted for these differences. In wild type mice, the higher mortality study used a larger gauge needle (20 vs 23) and did not include post-operative warming or oxygen.

Table 3.

Five examples where modulating the same pathway yields different outcomes. This table shows the 5 day survival using a CLP model of sepsis. The treatment targeted the same molecule or pathway, but the outcomes were different. This table highlights how a standardized sepsis model may be used to define the role of a potential mediator in the pathogenesis of sepsis. Currently an investigator may selectively cite the literature which supports her/his position to further investigate the molecule of choice.

Molecule or pathway	Better		Worse or no change
	Control	Treated	Control	Treated
Cannabinoid 2 receptor	20%	60% (34)	60%	20% (35)
Leukotriene synthesis inhibitor (MK886)	23%	100% (68)	90%	43% (69)
Anti-IL-6	60%	100% (70)	78%	80% (71)
Cyclooxygenase 2	0%	45% (72)	65%	20% (73)
Adenosine 2B receptor	40%	80% (74)	95%	50% (75)

Table 3 demonstrates that models with low survival demonstrate better survival with different interventions, while high survival models show no benefit or a worse outcome, regardless of treatment. This survival dependent benefit effect was previously described when evaluating either pre-clinical models or actual human sepsis studies (36).

7. A preclinical model can be reproducible

The CLP model of sepsis was described 37 years ago in a landmark paper that has been cited more than 1,200 times (32). Despite the availability of this model there is tremendous variation in the CLP procedure resulting in different mortality, as highlighted in Table 3. It was proposed over 7 years ago that a standardized preclinical model of sepsis would accelerate the pace of discovery (31). That manuscript specifically showed that many animal models of sepsis do not include the administration of antibiotics, one of the most important elements in the treatment of sepsis. It is acknowledged that absolute reproducibility will probably not be achieved, even with a standardized model.

The CLP model of sepsis has also been criticized since source control (resecting the necrotic cecum (37) or peritoneal drainage) is typically not done although several investigators do a second procedure to remove the dead tissue (38, 39). However, with fluid resuscitation and appropriate antibiotics it is possible to have 20% lethality over 7 days (40) even without resection, resulting in mortality similar to patients. It is important to use a lower lethality model since it has the potential to discover both injurious pathways in the mice that die, as well as protective pathways in the mice that live. A clinical scoring system that predicts mortality in the standardized model will identify mice several hours prior to their death to allow careful dissection of the pathways that lead to organ injury or immune dysfunction. Such sepsis scoring systems have previously been published (41–43). Additionally, physiological monitoring (44) may also be included to identify animals at risk of death (45) in addition to well described plasma biomarkers (41, 46, 47). The premise for standardized sepsis models is that such models will embrace the inherent heterogeneity of the host response to infection, even when treated with appropriate antibiotic therapy and fluid resuscitation (48, 49). Standardized models may also identify subgroups who respond to treatment (30).

Therapies effective in standardized sepsis models may be more easily translated into the clinical arena, since an inappropriate model does not translate into better patient care. As one specific example, findings from an animal model led to the development of antibodies to TNFα for the treatment of sepsis (50). The original model was a non-human primate, acute bacteremia model where blocking TNFα improved survival. In subsequent large scale clinical studies, TNFα inhibitors were not effective, since the animal model did not recapitulate the human disease. However, when the CLP model of sepsis was used, antibodies to TNFα were not effective. These results were published in 1992 (51), more than 3 years before clinical trials reported that blockade of TNFα did not improve survival (52). The murine study also showed that the TNFα inhibitor accelerated mortality, foreshadowing one clinical trial where TNF inhibition actually increased mortality in patients (53).

8. Limitations to standardized sepsis models

We recognize that some investigators will not embrace developing standardized models. Grant and journal reviewers may consider that studies done using a non-standardized model are not valid. This concern could be easily rebutted by providing information in the methods about how the model differs from the standard with an explanation for the deviation. Some regulatory agencies may not approve experiments if they do not conform to the standardized model. However, the availability of standardized protocols should streamline approvals when using that model. If not using a standardized model, the deviations could be clearly delineated, which is not a significant change from current approval processes. The changes from standardized models will also be easier to defend since there is a clear standard for comparison. The initial standardized models will probably not include co-morbid conditions such as diabetes or advanced age which are frequently found in patients (48, 49). A standardized model actually makes subsequent models easier to develop and implement, since there will be baseline for comparison. Investigators may be concerned about the cost of measuring the parameters to document organ injury. Standardized models would not dictate that every parameter be measured, but they would provide guidelines.

9. Next Steps

Development of standardized sepsis models would be a first step towards improving preclinical sepsis experiments so that the model more closely recapitulates the heterogeneity of human sepsis. We propose developing two sets of criteria for standardized sepsis models, starting with the CLP model of sepsis. The first set would consist of standardizing the technical elements for the model, since technical differences resulting in different mortality have been well described by several laboratories (32, 37, 54). Despite these prior publications, there is still wide variation in how this model is actually used, as highlighted in table 3. As a starting point, we propose discussing the technical elements listed in table 4 to reach consensus. It should be noted that these technical elements such as resuscitation would also be applicable to pre-clinical studies with larger animals such as sheep or pigs (49). The second criteria are probably more important, and would describe the parameters that should be measured to document organ injury or the dysregulated response to infection. These parameters are listed in table 5 and would be measured regardless of the sepsis model. This table lists the different organs and immune functions that should be evaluated, with the understanding that not every study would need to measure all parameters. As an example, we anticipate that the recommendations would follow previously published guidelines, such as the American Thoracic Society guidelines on measurements of acute lung injury (55).

Table 4.

Examples of the technical elements that could be standardized in the CLP sepsis model. CLP would be one of the models where standardization could be achieved, but would not be the only standardized sepsis model.

Surgical considerations Peritoneal cavity closure method Skin closure method Analgesia Need for second surgery to remove necrotic cecum Size of needle used for puncture Mortality % mortality # of days to follow for mortality Perioperative care Antibiotics yes or no Duration and type of antibiotics Fluid resuscitation including route Anesthesia for surgery Humane endpoints for euthanasia Location of venous sampling Post-op warming and delivery of oxygen Timing of therapeutic interventions

Table 5.

Examples of organ injury and immune dysfunction parameters. This table provides examples of parameters that would be appropriate to measure to document organ injury or dysregulated response to infection.

Organ Injury Pulmonary injury Liver injury Kidney injury Hematology changes CNS alterations Dysregulated host response to infection Plasma biomarkers, i.e. IL-6 Complete blood count with differential Temperature measurements Blood and peritoneal cultures at time of sacrifice when humane criteria are met Frequency of measurements Daily # of days

We would envision a series of white papers with recommendations for standardized sepsis models. These standardized sepsis models represent an important initial stage of increasing the reproducibility of sepsis science, leading to the development and implementation of better therapies to address this lethal disease.