Skip to main content
PMC home page
Implementation Science : IS logoLink to Implementation Science : IS
. 2018 Jan 18;13:15. doi: 10.1186/s13012-017-0706-5

The German Quality Network Sepsis: study protocol for the evaluation of a quality collaborative on decreasing sepsis-related mortality in a quasi-experimental difference-in-differences design

Daniel Schwarzkopf 1,2,#, Hendrik Rüddel 1,2,#, Matthias Gründling 3, Christian Putensen 4, Konrad Reinhart 1,2,✉
PMCID: PMC5774030  PMID: 29347952

Abstract

Background

While sepsis-related mortality decreased substantially in other developed countries, mortality of severe sepsis remained as high as 44% in Germany. A recent German cluster randomized trial was not able to improve guideline adherence and decrease sepsis-related mortality within the participating hospitals, partly based on lacking support by hospital management and lacking resources for documentation of prospective data. Thus, more pragmatic approaches are needed to improve quality of sepsis care in Germany. The primary objective of the study is to decrease sepsis-related hospital mortality within a quality collaborative relying on claims data.

Method

The German Quality Network Sepsis (GQNS) is a quality collaborative involving 75 hospitals. This study protocol describes the conduction and evaluation of the start-up period of the GQNS running from March 2016 to August 2018. Democratic structures assure participatory action, a study coordination bureau provides central support and resources, and local interdisciplinary quality improvement teams implement changes within the participating hospitals. Quarterly quality reports focusing on risk-adjusted hospital mortality in cases with sepsis based on claims data are provided. Hospitals committed to publish their individual risk-adjusted mortality compared to the German average. A complex risk-model is used to control for differences in patient-related risk factors. Hospitals are encouraged to implement a bundle of interventions, e.g., interdisciplinary case analyses, external peer-reviews, hospital-wide staff education, and implementation of rapid response teams. The effectiveness of the GQNS is evaluated in a quasi-experimental difference-in-differences design by comparing the change of hospital mortality of cases with sepsis with organ dysfunction from a retrospective baseline period (January 2014 to December 2015) and the intervention period (April 2016 to March 2018) between the participating hospitals and all other German hospitals. Structural and process quality indicators of sepsis care as well as efforts for quality improvement are monitored regularly.

Discussion

The GQNS is a large-scale quality collaborative using a pragmatic approach based on claims data. A complex risk-adjustment model allows valid quality comparisons between hospitals and with the German average. If this study finds the approach to be useful for improving quality of sepsis care, it may also be applied to other diseases.

Trial registration

ClinicalTrials.gov NCT02820675.

Keywords: Sepsis, Mortality, Quality improvement, Hospitals, Risk adjustment, Administrative claims, Interdisciplinary health team

Background

Sepsis is one of the major challenges for health care today [1]. It is the leading cause of death among infectious diseases [2] and might also be the leading cause of preventable deaths in hospitals [3, 4]. With estimated yearly costs in the USA and Germany of 27 billion USD and 7.7 Euro respectively, sepsis is one of the most expensive inpatient diseases [5, 6]. Recognizing severe shortcomings in prevention, diagnosis, and treatment of sepsis, the World Health Organization adopted a resolution in May 2017, which urges all member states to invest in improvement of sepsis care [7].

Guidelines for treating sepsis recommend early recognition by standardized screening, early administration of antibiotics, early adequate fluid resuscitation, and source control [8]. Several studies have shown positive effects of early therapy on patients’ outcomes, but at the same time guideline adherence is often low [3, 4, 9–16]. By using multifaceted interventions to increase guideline adherence on early adequate diagnosis and therapy of sepsis, several regional, national, and international quality initiatives were able to substantially reduce sepsis mortality in the participating hospitals [17–23].

In 2013, 279.530 patients were diagnosed with sepsis, severe sepsis, or septic shock in German hospitals. Comparable to other high-income countries, sepsis incidence had increased from 2007 to 2013 by 5.7% per year [5, 24]. Hospital mortality among patients with severe sepsis in Australia, England, and the USA decreased from 35 to 18.5%, from 45.5 to 32.1%, and from 37 to 29%, respectively during the 2000s [25–27]. In Germany, it decreased by only 4.2% from 47.8 to 43.6% between 2003 and 2013 [24, 28].

The MEDUSA trial (Medical EDUcation for sepsis Source control and Antibiotics, ClinicalTrials.gov identifier NCT01187134) was the first multi-center quality initiative for reducing mortality among inpatient cases with severe sepsis in Germany [12, 29]. This cluster randomized controlled trial involving 42 hospitals used a quality reporting based on prospectively gathered clinical data. A bottom-up approach led by the local intensive care departments was used to implement a multifaceted intervention within a 2-year intervention period. The intervention led to no substantial improvement in guideline adherence and no decrease in 28-day mortality [29]. Expert interviews and surveys of local quality improvement teams identified lacking support by hospital management, lacking support of the project by other departments, and lacking time for data documentation and implementation of interventions as major barriers to change [30]. Incomplete inclusion of cases with sepsis and delayed documentation led to decreased validity of reporting of quality indicators and reduced power to detect effects of the intervention. Major lessons drawn from these experiences are as follows: (a) clear commitment by hospital management is needed to assure resources for QI and guarantee the involvement of all relevant departments in the QI effort, (b) an approach for gathering quality data assuring inclusion of the majority of cases with sepsis is needed, and (c) structures for hospital-wide QI efforts need to be established for achieving a sustainable long-term reduction of sepsis mortality.

Using claims data for performance measurement has the advantages to cover all diagnosed cases with data being readily available and needing minimal time and costs [31]. This approach is extensively used within quality initiatives in the USA [32] and has also achieved first promising results in Germany [33].

Objectives

The primary objective of this study is to decrease sepsis-related mortality in the participating hospitals compared to the average of all German hospitals. To obtain this objective, all hospitals regularly receive quality reports based on claims data of all in-patient cases with sepsis and implement a multifaceted intervention.

Methods

Context

The German Quality Network Sepsis (GQNS) was founded in February 2016. The participating hospitals committed to improve the quality of care for patients with sepsis and thereby to decrease sepsis-related hospital mortality. The start-up period of the GQNS is funded by grants from the German Federal Ministry of Education and Research (BMBF) via the integrated research and treatment Center for Sepsis Control and Care (CSCC) at the Jena University Hospital and runs from August 2015 to July 2018. The funded start-up phase and its scientific evaluation use the acronym ICOSMOS (quality Improvement in infection COntrol and Sepsis management in MOdel regionS). Additional funding is obtained via an annual fee paid by the participating hospitals. After the start-up-period, the GQNS will continue its work based on additional funding and on the fees of the participating hospitals.

This study protocol describes the conduction and evaluation of the start-up-period of the GQNS. Figure 1 describes the timeline of the study. The intervention started in March 2016 with the first retrospective reporting of quality indicators including the years 2014–2015 to the participating hospitals. This study protocol takes into account the SQUIRE 2.0 recommendations [34].

Fig. 1.

Fig. 1

Open in a new tab

Study timeline. Study timeline and milestones are presented for the 48 hospitals initially enrolled in the German Quality Network Sepsis. A final analysis comparing these 48 intervention hospitals to all other German hospitals will be done with a delay of 1.5 years when the German national claims data will be made available by the Federal Bureau of Statistics

Major principles

The GQNS fulfills the essential features of a quality collaborative [35] by being an organized multifaceted approach which (a) has the specific topic of improving quality of inpatient sepsis care, (b) involves clinical and QI experts who provide ideas and support, (c) implements QI teams in the participating hospitals, (d) uses measurable targets by sepsis-related hospital mortality, and (e) applies a series of structured interventions.

Major principles of the GQNS are as follows: (a) focusing on outcome quality by using sepsis-related hospital mortality as major quality indicator; (b) measuring quality based on claims data readily available at each participating hospital; (c) assuring validity of quality comparisons by implementing a complex risk-adjustment model; (d) aiming at hospital-wide quality improvement by formation of local multidisciplinary QI teams led by committed local champions; (e) implementing education based on Surviving Sepsis Campaign guidelines among all involved health care workers; (f) assuring full support by senior management of participating hospitals by their commitment to an annual participation fee and the publication of risk-adjusted sepsis-related hospital mortality; (g) ownership of the collaborative by the participants by establishing structures that assure participatory action; and (h) bringing the expertise available within the collaborative to action by a system of peer-reviews between the participating hospitals.

Project organization

The organization of the GQNS is regulated in bylaws ratified by the participating hospitals. All responsibility for the QI efforts is at the side of the individual participating hospitals. Local multidisciplinary QI teams have repeatedly been recommended for enhancing guideline implementation especially regarding sepsis [12, 19, 21, 36, 37]. Likewise, participating hospitals of the GQNS are encouraged to name local champions and establish QI teams involving all relevant departments for improving quality of sepsis care, e.g., intensive and intermediate care units, surgical and medical wards, emergency departments, and quality management departments.

The central study coordinating bureau, hosted by the CSCC, supports the QI efforts of the participating hospitals and coordinates accompanying scientific studies. Means of supporting the QI teams are as follows: (a) developing the model for risk-adjustment and specifications for quality reporting; (b) hosting the study homepage providing literature, instructions, educational material, technical and status reports, and online tutorials; (c) organizing national and regional study meetings; (d) coordinating communication and information exchange within the GQNS; (e) counseling local QI teams; (f) measuring the progress of QI efforts and barriers to change and giving feedback on these data to QI teams and senior hospital managements on a regular basis; (g) organizing online lectures and discussions with clinical experts; and (h) organizing web-based discussions with champions of successful international QI initiatives on sepsis care.

Structures of the GQNS guarantee participatory action and ownership of the collaboration by the member hospitals. Major decisions are made in the general assembly of representatives of all participating hospitals. A steering committee is elected among the members of the GQNS, to supervise and assist the work of the coordinating bureau. A committee of experts recruited from the participating hospitals organizes the peer-reviews. Claims data are collected and processed to make quality reports by the medical information technology service provider 3M Health Information Systems (3M). The GQNS is part of the Global Sepsis Alliance’s Quality Improvement Committee [38].

Participating hospitals

Eligible for participation in the GQNS were acute care hospitals with at least one adult intensive care unit. Invitation letters were sent to managements of hospitals which were participating in former or ongoing sepsis-related quality initiatives or research networks, to regional and national hospital groups, and to all German university hospitals. The active recruitment process started in Aug 2015 and ended in Feb 2016 and aimed at including 30 hospitals. Forty-eight hospitals were enrolled in March 2016, when the first quality reports were distributed. Characteristics of these hospitals are presented in Table 1.

Table 1.

Characteristics of 48 initially enrolled hospitals

Characteristic n (%)
Level of care
 Primary or secondary level 29 (60%)
 Tertiary level 19 (40%)
Academic status
 Non-university hospital 39 (81%)
 University hospital 9 (19%)
Number of beds
 ≤ 400 15 (31%)
 401–900 17 (35%)
 901–1200 8 (17%)
 > 1200 8 (17%)
Open in a new tab

Forty-eight hospitals, which were enrolled at the beginning of the intervention phase (March 2016), will be used for the evaluation of the intervention effect of the start-up period

Interventions

The core interventions in the GQNS are (a) reporting and publication of quality indicators; (b) case analyses within the participating hospitals; (c) peer-reviews for hospitals, which are outliers in the quality reports; and (d) hospital-wide staff education in participating hospitals.

The only mandatory intervention is the reporting, benchmarking, and publication of quality indicators. All hospitals committed to provide their claims data for calculation of quality indicators and to publish their main quality indicators in comparison to the German average. The study coordination bureau provides encouragement, information, and support regarding the conduction of case analyses and staff education. It also offers peer-reviews to hospitals where quality indicators suggest low quality of care. But the responsibility for implementing case analyses and staff education is on the side of the participating hospitals and the participation in a peer-review is voluntary. Hospitals are free to develop and implement further interventions suitable to their local conditions.

Reporting and publication of quality indicators

There is consensus that valid and scientifically rigorous measures of performance are a means to improve health care and a central aspect of successful quality initiatives [39–42].

The primary quality indicator measuring outcome quality of sepsis care is risk-adjusted hospital mortality among cases with sepsis with organ dysfunction including septic shock. It also serves as the primary end-point of the study.

Measurement of risk-adjusted hospital mortality

To achieve inclusion of all cases diagnosed with sepsis and to make quality measurement also feasible for small hospitals without specialized research staff, risk-adjusted hospital mortality of cases with sepsis is measured based on claims data [31, 33]. A diagnosis-related group (DRG) system is used to structure reimbursement of hospitals by health insurances in Germany. Every hospital provides a standardized data set (DRG data) of all treated cases to the Federal Bureau for Hospital Reimbursement on a yearly base. Additionally, the German Federal Bureau of Statistics provides the anonymized DRG data of all German hospitals for scientific research with a 1.5-year delay (e.g., data of 2015 available 2017). DRG data contain demographic characteristics, International Classification of Diseases codes in its 10th version (ICD-10), codes of the German Classification of Procedures in Medicine, treating hospital departments, length of stays per department, and information regarding hospital admission and discharge.

Every participating hospital sends its pseudonymized DRG data to 3M on a quarterly basis (Fig. 2). Based on specifications provided by the study coordination bureau, 3M calculates quality indicators and provides quality reports to the participating hospitals. Cases with sepsis are identified based on specific ICD-10-codes and classified in subgroups according to the 1992 ACCP/SCCM sepsis definitions (Appendix) [43]. Incidence and risk-adjusted mortality is reported for cases with sepsis and organ dysfunction including septic shock as well as the subgroup of patients receiving more than 24 h of ventilation. For cases with sepsis without organ dysfunction, only incidence and observed mortality rate are reported.

Fig. 2.

Fig. 2

Open in a new tab

Flows of information in the German Quality Network Sepsis. The study coordinating bureau develops the risk-model based on analyzing the German population claims data hosted by the Felderal Bureau of Statistics. It further provides the specifications for calculating quality indicators to 3M Health Information Systems. Based on quarterly transmissions of claims data by the participating hospitals, 3M produces quality reports and delivers them to the hospitals. Hospitals committed to publish their sepsis-related risk-adjusted hospital mortality in comparison to the German average after 2 years of participation

Statistical methods of risk adjustment are needed to control for differences in patient-related risk-factors between hospitals and allow for valid comparisons of quality [32]. Risk-adjusted performance comparisons based on claims data have been applied to hospital profiling regarding several diseases in the USA [44–46]. The GQNS uses an external risk adjustment algorithm [47] by developing a risk model within the DRG data of all German hospitals and applying this model to the sample of hospitals within the GQNS (Fig. 2). This resembles the methodology used by the US Agency for Health Care Research and Quality Improvement (AHRQ) Inpatient Quality Indicators [48]. The risk model currently in use was developed following established methodologies [44, 49] based on all DRG-cases coded with severe sepsis in German hospitals in 2015 (manuscript in preparation). The model involves about 50 risk factors, e.g., comorbidity categories based on Charlson and Elixhauser indexes [50, 51], and achieves a predictive validity comparable to recently published models based on US American claims data [52, 53]. The risk model is applied to the hospitals in the GQNS by calculating expected mortality rates, standardized mortality ratios (SMR, “Observed over Expected Estimator”), and risk standardized mortality rates (RSMR) with their respective confidence intervals [47, 48, 54] among cases with sepsis with organ dysfunction, cases with septic shock, and cases with organ dysfunction including septic shock and ventilation of more than 24 h. The risk-model and adjustment methodology are constantly improved based on feedback by participating hospitals, research results, and yearly updates of the German hospital population DRG data.

Reporting of risk-adjusted mortality to hospitals

Quality indicators are provided to the hospitals via 3M on a quarterly base. A web-browser-based tool with numerical as well as graphical presentations is used. Incidence, observed mortality and risk-adjusted mortality within subgroups of cases with sepsis are presented for each hospital. Reports allow comparison of risk-adjusted mortality to the German average, among the hospitals of the GQNS, among hospitals of specific levels of care, and over time. Displaying confidence intervals prevents interpreting merely random differences. Figure 3 provides a schematic example of a graphical report of SMRs.

Fig. 3.

Fig. 3

Open in a new tab

Schematic example of graphical reporting of quality indicators. Presented are SMRs (standardized mortality ratios) with 90% confidence intervals (CI) for cases with sepsis with organ dysfunction (including septic shock) in 2016. Internal quality reporting uses 90% CIs to increase sensitivity for finding true deviations from average quality. Publications of quality indicators by participating hospitals will use 95% CIs. GQNS: German Quality Network Sepsis

Additionally, each hospital receives a report using Microsoft Excel© including information on incidence and risk-adjusted mortality of sepsis cases within each of its departments. This report also presents lists of pseudonymized individual cases of the respective hospital including their individually predicted probability of death based on the risk model. Based on these lists, hospitals can perform in-depth single-case analyses (see below).

Publication of risk-adjusted mortality indicators

Public comparative reporting of health care provider performance measures is regarded a key element for improving quality of health care [55, 56]. There is evidence that public reporting has greater effects on organizational improvement efforts than private disclosure of the same data alone [57–60]. Senior managements of hospitals within the GQNS consented to publish their hospitals’ risk-adjusted mortality indicators compared to the average of the German DRG data after 2 years of participation. By doing this, hospitals document their commitment to quality improvement, accountability for quality of sepsis care, and commitment to principles of transparency in health care. Figure 2 presents the system of quality measurement and reporting within the GQNS.

Case analyses

Analysis of individual case records can reveal preventable deaths as well as underlying shortcomings in care [61–63]. Analysis of adverse events among cases with sepsis has revealed that problems in guideline adherence regarding diagnosis and treatment, like delayed recognition of deterioration, delayed source control, and inadequate fluid and antimicrobial therapy, but also structural problem like hand-over practices from normal ward to ICU cause preventable deaths [3, 4].

Within the GQNS, individual probabilities for death are predicted based on the risk model and provided to the participating hospitals. The study coordination bureau provided information on how to conduct case analyses. Deceased cases with a predicted low risk of in-hospital mortality are used to identify possible problems in the quality of care or ICD-10 coding. Hospitals can depseudonymize these cases and analyze individual medical records using established methods [62, 64, 65]. Interdisciplinary case discussions moderated by members of the local QI teams can be used to consent on goals for quality improvement.

Peer-reviews

Peer-review, a process by which physicians evaluate each other’s performance, has a long tradition in efforts to improve health care quality [66]. A system of peer-reviews using external peers from other hospitals is used within a quality initiative involving 400 hospitals (Initiative Qualitätsmedizin [IQM]) [67]. It has been shown to be able to detect areas of possible improvement of care and to be related to decreases in mortality in diseases like heart failure and pneumonia [33]. Based on this methodology, a peer-review committee consisting of specifically trained clinicians is set up within the GQNS and working instructions have been defined. An external peer-review is suggested to hospitals with SMRs for patients with sepsis with organ dysfunction including shock substantially higher than the German average. Additionally, hospitals can call for a peer-review by themselves. External peers investigate a sample of the retrospective case records. These records are identified among cases that died despite of a low predicted risk. Based on case-analysis, external peers will discuss improvement strategies with involved local departments.

Staff education

The main focus of staff education is the implementation of strategies for increasing awareness and early recognition of sepsis as well as the implementation of key elements of the updated Surviving Sepsis Campaign guidelines among physician and nurses, as well as all other health care workers involved in care for patients with sepsis [8, 68, 69]. The content of the education is focused on a small number of essentials (“Sepsis Bundles”). Education is implemented by the local QI teams. The study coordination bureau supports the local QI teams by providing educative material (presentations, pocket cards, posters) developed during the MEDUSA trial. Furthermore, online lectures and discussions involving QI improvement experts are provided.

Further interventions

Participating hospitals will be encouraged to implement a protocol for early recognition of sepsis in the emergency department, recognition of deteriorating patients or patients at risk on regular wards (standardized screening for sepsis), and to establish rapid response teams, which are mandatory or widely implemented in some countries but not in Germany [70]. Furthermore, hospitals will be encouraged to get active in at least one component of the national hospital surveillance program (Krankenhaus-Infektions-Surveillance-System KISS) (http://www.nrz-hygiene.de/en/surveillance/hospital-infection-surveillance-system/).

Study of the interventions

The effect of participating in the start-up period of the GQNS will be evaluated by comparing the participating hospitals to all other German hospitals regarding the difference between the retrospective baseline period from January 2014 to December 2015 to the intervention period from April 2016 to March 2018 (quasi-experimental difference-in-differences design) [71]. This approach is more valid than uncontrolled before-after comparisons that were used in previous quality initiatives on sepsis [23], since it is able to distinguish the intervention effect from secular changes that would have occurred without the intervention [71]. Beside secular changes, the so called Will Rogers phenomenon might bias changes in observed mortality across time [71, 72]. This phenomenon occurs if an intervention increases awareness for a disease, which results in increased detection and coding also of less severely ill cases. By incorporating an adjustment for risk factors in the difference-in-differences comparison, the analyses will try to correct for this bias. The analyses will evaluate the overall effect of participating in the GQNS, but they will not be able to distinguish which intervention elements cause possible changes.

Measures for studying processes and implementation of the interventions

The study coordination bureau of the GQNS receives a study report by 3M on a quarterly base that informs about the quality indicators of each participating hospital as well as about hospitals that failed to provide their claims data. Hospitals that do not provide their claims data despite several reminders are excluded from the quality collaborative. To assess fidelity and dose of the local implementation of interventions in the participating hospitals, the coordination bureau of the GQNS regularly surveys local QI team leaders by standardized online questionnaires. These questionnaires contain items regarding departments participating in the QI teams, extend of interdisciplinary analyses of quality indicators, extend of interdisciplinary case analyses, extend of education of staff, and implementation of further interventions like existence and scope of rapid response teams. The questionnaire also contains items regarding barriers to change like lacking resources or support by management, as well as items that evaluate the quality of tools and support provided by the coordination bureau. The questionnaire was developed based on results of qualitative interviews conducted during the MEDUSA trial [30]. Peer-review visits are documented and evaluated using paper-pencil surveys using standardized questionnaires both by the external peers and the involved local clinicians. These questionnaires were adapted from existing tools used in other German quality initiatives [67, 73].

Outcome measures

Hospital mortality of cases with sepsis with organ dysfunction (including septic shock) in the participating hospitals is the primary end-point of the study, since the newly revised consensus definitions of sepsis and septic shock (“Sepsis-3”) exclude cases without organ dysfunction [2]. Since the evaluation of the study effect relies on claims data, it is not possible to fully assess and guarantee completeness and accuracy of data. To allow hospitals to identify possible under-coding of cases with sepsis the incidence of sepsis, severe sepsis and septic shock is presented and compared between hospitals in the quality reports.

Analyses

Analysis of the primary outcome

The final analysis of the intervention effect will be conducted using German national DRG data. The change in hospital mortality between the baseline period (January 2014 to December 2015) and the intervention period (April 2016 to March 2018) will be compared between the 48 hospitals initially enrolled in the GQNS and control hospitals. These are all non-participating German hospitals, whose DRG data are also available from the Federal Bureau of Statistics. Hospitals entering the GQNS after March 2016 will be excluded from this comparison. This comparison will be conducted in 2020 when the national DRG data for 2018 will be available from the Federal Bureau of Statistics. The primary analysis will evaluate the difference-in-differences in a hierarchical generalized linear model controlling for clustering of cases in hospitals by testing the statistical interaction between period (baseline vs. intervention) and group (48 hospitals enrolled in GQNS vs. control hospitals). Patient-related risk-factors will be considered as covariates. As a secondary analysis, the trajectories of change across time before and after intervention will be compared between intervention and control hospitals by a piecewise hierarchical model [74].

A preliminary evaluation of the study effect will be performed after completion of the intervention period in March 2018 but only relying on the data of the 48 intervention hospitals (before-after-comparison). A secondary preliminary analysis will be done by a piecewise hierarchical model to investigate the change in the trajectory of hospital mortality over time between periods.

Sensitivity analyses and planned subgroup analyses

Differences across time between hospitals of the GQNS and control hospitals might result from changes in coding in the intervention hospitals, e.g., improved ICD-coding of sepsis or comorbidities. Therefore, incidences of sepsis without organ dysfunction, sepsis with organ dysfunction and septic shock as well as incidences of mortality risk-factors will be compared across time between intervention and control hospitals. Analyses of the primary outcome will be repeated within these three subgroups of sepsis cases.

Additionally, hospitals will be categorized based on the degree of their QI efforts during the intervention period but without inspection of their primary outcome. The categorization will be based on the results of the regularly conducted surveys of local QI team leaders. The analyses of the primary end-point will be repeated within the subgroups of hospitals with high QI effort and hospitals with low QI effort.

Sample size considerations

Concerning the baseline period (January 2014 to December 2015), 45,419 cases with sepsis were included among the initially enrolled 48 hospitals, of which 27,331 (60%) were diagnosed with sepsis with organ dysfunction (including septic shock), resulting in approximately 1000 cases per hospital across both the baseline and intervention period. Comparing the intervention hospitals with all other German hospitals across a baseline and intervention period resembles a controlled before-after trial on the cluster level. Therefore, our sample size calculation used a design factor developed for cluster-randomized before-after trials [75]. We observed a baseline mortality of 44% and an interclass correlation (ICC) of 0.02 among the included 48 hospitals. Based on the results of the Surviving Sepsis Campaign [15], a reduction in mortality to 40% was expected for the 2-year intervention period. A Fisher’s exact test comparing intervention and control conducted at α-level 0.05 with a power of 0.9 would require a total sample size of 6476. Correcting this sample size by the obtained design factor of 3.75, a total of 26 hospitals (13 control and 13 intervention) each including 1000 cases would be necessary in a cluster randomized before-after trial. Given 48 intervention hospitals and more than 1000 control hospitals in the national German DRG data, our study should be adequately powered to detect the expected intervention effect.

Ethical considerations

The study underwent review by the internal review board of the Jena University Hospital (IRB protocol 4536-08/15) and the data protection supervisor of the Free State of Thuringia, Germany. Since individual case data are only obtained in the form of pseudonymized claims data, the need for informed consent by patients was waived. Quality reports present only aggregated data by hospital. Hospital names are pseudonymized, and each hospital knows only its own pseudonym. Only 3M and the study coordination bureau can de-pseudonymize all hospitals. Individual case data are only reported to the hospital where the cases were treated. For all kinds of surveys of QI team members, hospital staff, or peer-reviewers, participants will be informed about aims of the surveys and methods for protection of data privacy. Informed consent will be implied by participation in the survey, survey data will be stored in pseudonymized or anonymized form only, and all public reports of survey data will contain only aggregated data allowing no identification of individual participants.

Project status

The GQNS was officially started with a kick-off meeting in February 2016. The first quality reports were distributed in March 2016 to the initially enrolled 48 hospitals. Till October 2017, quality reports have been distributed five times. Based on the public awareness of the GQNS, additional hospitals entered the network after the active recruitment. In September 2017, the number of participating hospitals reached 75, clearly surpassing the initially planned number. Participating hospitals are located all over Germany and include hospitals of all levels of care, among them 18 university hospitals (of 39 in Germany). In August 2016 and August 2017, two surveys of local QI team leaders were performed to assess the current progress and barriers to change in the participating hospitals and results were reported to senior hospital managements. Between March 2017 and September 2017, the first four online lectures and between May 2017 and September 2017 the first three external peer-reviews took place.

Discussion

The goal of this project is to establish structures for quality improvement of care for patients with sepsis and thereby to decrease sepsis-related in-hospital mortality. The GQNS is the first quality collaborative using claims data and a complex risk-adjustment to measure and improve quality of care for patients with sepsis. Because of this pragmatic approach 75 hospitals signed contracts for participation.

Previous QI initiatives to improve sepsis care relied on gathering prospective clinical data in an uncontrolled before-and-after design [15, 17–21]. Based on these studies, it is impossible to distinguish the intervention effects from general secular trends [71]. The first cluster-randomized trial which set out to test for effects of elements of a QI collaborative on sepsis-related mortality (MEDUSA trial) failed partly because of incomplete and delayed inclusion of cases with sepsis. Our study design overcomes both of these shortcomings. Using claims data makes all patients with coded sepsis available for analysis. Based on a German national claims database, it is possible to evaluate the effect of the QI initiative in a difference-in-differences design comparing intervention hospitals to all other German hospitals.

The approach of our study of course has some limitations. First, under- or over-coding of sepsis might occur in claims data [76]. However, for every hospital, the degree of under-documentation has decreased considerably, as since 2007 appropriate coding of sepsis cases has become a financial interest for the hospitals. On the other hand, over-coding of sepsis cases for financial reasons is systematically controlled for by the Medical Review Board of the Statutory Health Insurance Funds. A second limitation is the time-lag of 1.5 years until German national claims data are available for analysis delaying the final analysis of the intervention effect. A preliminary evaluation of the effect can only rely on data of the intervention hospitals. Finally, quality improvement in sepsis care has been described to be a marathon and not a short sprint [77]. The 2-year intervention period of the start-up phase of the GQNS might be too short for attaining substantial effects on hospital mortality in a large sample of hospitals. The aim of this start-up period is to build a long-lasting quality initiative and to establish structures for sustained quality improvement. A reevaluation of the effects of participating in the GQNS after a longer period of time—comparable to the duration of other successful large-scale QI initiatives [15]—might be necessary.

If this study finds that the approach of using claims data in combination with a complex risk-adjustment is useful for measuring and improving quality of sepsis care, the approach may also be applied to other diseases.

Acknowledgements

Not applicable.

Funding

The study is funded by grants from the German Federal Ministry of Education and Research (BMBF; FKZ 01EO1502) via the integrated research and treatment Center for Sepsis Control and Care (CSCC). The BMBF and CSCC play no role in the design of the study or implementation of interventions. Additional funding is received from all participating hospitals by a yearly fee between 500€ and 2000€, depending on size of the hospital, for participation in the quality collaborative. DS is fully and HR is partly funded by BMBF grants via the CSCC.

Availability of data and materials

Not applicable.

Abbreviations

3M

3M Health Information Systems

CSCC

Center for Sepsis Control and Care

DRG

Diagnostic related groups

GQNS

German Quality Network Sepsis

MEDUSA trial

Medical EDUcation for sepsis Source control and Antibiotics, ClinicalTrials.gov identifier NCT01187134

QI

Quality improvement

RSMR

Risk standardized mortality rate

SMR

Standardized mortality ratio

Appendix

Definition of cases with sepsis by German ICD-10 codes

Sepsis without organ dysfunction: A40.0, A40.1, A40.2, A40.3, A40.8, A40.9, A41.0, A41.1, A41.2, A41.3, A41.4, A415.1, A41.52, A41.58, A41.8, A41.9, R65.0.

Sepsis with organ dysfunction but without septic shock: R65.1.

Septic shock: R57.2.

Authors’ contributions

KR conceptualized the study design, acquired the funding for the study and supervises the conduction of the study. DS contributed mainly to the design of measurement and reporting of quality indicators as well as the planned statistical analyses. HR contributed mainly to the development of the study interventions and coordinates the work of the central study coordinating bureau. MG and CP are members of the steering committee and participated in the planning of study interventions. DS and HR drafted the manuscript. KR, MG, and CP critically revised the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

The study was approved by the internal review board of the Jena University Hospital. Since a secondary analysis of claims data is done, the need for informed consent of patients was waved. The study was additionally approved by the data protection supervisor of the Free State of Thuringia, Germany.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Daniel Schwarzkopf, Email: Daniel.Schwarzkopf@med.uni-jena.de.

Hendrik Rüddel, Email: Hendrik.Rueddel@med.uni-jena.de.

Matthias Gründling, Email: gruendli@uni-greifswald.de.

Christian Putensen, Email: christian.putensen@ukbonn.de.

Konrad Reinhart, Phone: +49 3641 9 323192, Email: Konrad.Reinhart@med.uni-jena.de.

References

  • 1.Angus DC, van der Poll T. Severe sepsis and septic shock. N Engl J Med. 2013;369(9):840–851. doi: 10.1056/NEJMra1208623. [DOI] [PubMed] [Google Scholar]
  • 2.Singer M, Deutschman CS, Seymour CW, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3) JAMA-J Am Med Assoc. 2016;315(8):801–810. doi: 10.1001/jama.2016.0287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Clinical Excellence Commission. Recognition and management of sepsis. 2012. http://www.cec.health.nsw.gov.au/__data/assets/pdf_file/0004/259375/patient-safety-report-sepsis-2012.pdf. Accessed 15 Jan 2017.
  • 4.Goodwin APL, Srivastava V, Shotton H, et al. Just say sepsis! A review of the process of care received by patients with sepsis; 2015. http://www.ncepod.org.uk/2015report2/downloads/JustSaySepsis_FullReport.pdf. Accessed 15 Jan 2017. 
  • 5.Fleischmann C, Scherag A, Adhikari NKJ, et al. Assessment of global incidence and mortality of hospital-treated sepsis. Am J Respir Crit Care Med. 2016;193(3):259–272. doi: 10.1164/rccm.201504-0781OC. [DOI] [PubMed] [Google Scholar]
  • 6.Torio CM, Moore BJ. National inpatient hospital costs: the most expensive conditions by payer, 2013. Internet. Rockville: Agency for Healthcare Research and Quality; 2016. [PubMed] [Google Scholar]
  • 7.World Health Assembly Executive Board. EB140.R5 Improving the prevention, diagnosis and management of sepsis. 2017 [PDF]. http://apps.who.int/gb/ebwha/pdf_files/EB140/B140_R5-en.pdf. Accessed 26 Oct 2017.
  • 8.Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Intensive Care Med. 2013;39(2):165–228. doi: 10.1007/s00134-012-2769-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Ferrer R, Artigas A, Suarez D, et al. Effectiveness of treatments for severe sepsis a prospective, multicenter, observational study. Am J Respir Crit Care Med. 2009;180(9):861–866. doi: 10.1164/rccm.200812-1912OC. [DOI] [PubMed] [Google Scholar]
  • 10.Ferrer R, Martin-Loeches I, Phillips G, et al. Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program. Crit Care Med. 2014;42(8):1749–1755. doi: 10.1097/CCM.0000000000000330. [DOI] [PubMed] [Google Scholar]
  • 11.Gaieski DF, Mikkelsen ME, Band RA, et al. Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department. Crit Care Med. 2010;38(4):1045–1053. doi: 10.1097/CCM.0b013e3181cc4824. [DOI] [PubMed] [Google Scholar]
  • 12.Bloos F, Thomas-Ruddel D, Ruddel H, et al. Impact of compliance with infection management guidelines on outcome in patients with severe sepsis: a prospective observational multi-center study. Crit Care. 2014;18(2):R42. doi: 10.1186/cc13755. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kumar A, Roberts D, Wood KE, et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med. 2006;34(6):1589–1596. doi: 10.1097/01.CCM.0000217961.75225.E9. [DOI] [PubMed] [Google Scholar]
  • 14.Levy MM, Artigas A, Phillips GS, et al. Outcomes of the surviving sepsis campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919-24. [DOI] [PubMed]
  • 15.Levy MM, Rhodes A, Phillips GS, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5-year study. Crit Care Med. 2015;43(1):3–12. doi: 10.1097/CCM.0000000000000723. [DOI] [PubMed] [Google Scholar]
  • 16.Seymour CW, Gesten F, Prescott HC, et al. Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med. 2017;376(23):2235–2244. doi: 10.1056/NEJMoa1703058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Burrell AR, McLaws ML, Fullick M, et al. SEPSIS KILLS: early intervention saves lives. Med J Aust. 2016;204(2):73. doi: 10.5694/mja15.00657. [DOI] [PubMed] [Google Scholar]
  • 18.Miller RR, Dong L, Nelson NC, et al. Multicenter implementation of a severe sepsis and septic shock treatment bundle. Am J Respir Crit Care Med. 2013;188(1):77–82. doi: 10.1164/rccm.201212-2199OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Ferrer R, Artigas A, Levy MM, et al. Improvement in process of care and outcome after a multicenter severe sepsis educational program in Spain. JAMA-J Am Med Assoc. 2008;299(19):2294–2303. doi: 10.1001/jama.299.19.2294. [DOI] [PubMed] [Google Scholar]
  • 20.Castellanos-Ortega A, Suberviola B, Garcia-Astudillo LA, et al. Impact of the Surviving Sepsis Campaign protocols on hospital length of stay and mortality in septic shock patients: results of a three-year follow-up quasi-experimental study. Crit Care Med. 2010;38(4):1036–1043. doi: 10.1097/CCM.0b013e3181d455b6. [DOI] [PubMed] [Google Scholar]
  • 21.Noritomi DT, Ranzani OT, Monteiro MB, et al. Implementation of a multifaceted sepsis education program in an emerging country setting: clinical outcomes and cost-effectiveness in a long-term follow-up study. Intensive Care Med. 2014;40(2):182–191. doi: 10.1007/s00134-013-3131-5. [DOI] [PubMed] [Google Scholar]
  • 22.Scheer CS, Fuchs C, Kuhn S-O, et al. Quality improvement initiative for severe sepsis and septic shock reduces 90-day mortality: a 7.5-year observational study. Crit Care Med. 2016; Publish Ahead of Print [DOI] [PubMed]
  • 23.Damiani E, Donati A, Serafini G, et al. Effect of performance improvement programs on compliance with sepsis bundles and mortality: a systematic review and meta-analysis of observational studies. PLoS One. 2015;10(5):e0125827. doi: 10.1371/journal.pone.0125827. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Fleischmann C, Thomas–Rueddel DO, Hartmann M, et al. Fallzahlen und Sterblichkeitsraten von Sepsis-Patienten im Krankenhaus. Dtsch Arztebl Int 2016;113(10):159-166. [DOI] [PMC free article] [PubMed]
  • 25.Kaukonen KM, Bailey M, Suzuki S, et al. Mortality related to severe sepsis and septic shock among critically III patients in Australia and New Zealand, 2000-2012. JAMA-J Am Med Assoc. 2014;311(13):1308–1316. doi: 10.1001/jama.2014.2637. [DOI] [PubMed] [Google Scholar]
  • 26.Shankar-Hari M, Harrison DA, Rowan KM. Differences in impact of definitional elements on mortality precludes international comparisons of sepsis epidemiology—a cohort study illustrating the need for standardized reporting. Crit Care Med. 2016; Publish Ahead of Print [DOI] [PubMed]
  • 27.Lagu T, Rothberg MB, Shieh MS, et al. Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007. Crit Care Med. 2012;40(3):754–761. doi: 10.1097/CCM.0b013e318232db65. [DOI] [PubMed] [Google Scholar]
  • 28.Engel C, Brunkhorst FM, Bone HG, et al. Epidemiology of sepsis in Germany: results from a national prospective multicenter study. Intensive Care Med. 2007;33(4):606–618. doi: 10.1007/s00134-006-0517-7. [DOI] [PubMed] [Google Scholar]
  • 29.Bloos F, Rüddel H, Thomas-Rüddel D, et al. Effect of a multifaceted educational intervention for anti-infectious measures on sepsis mortality: a cluster randomized trial. Intensive Care Med. 2017;43(11):1602–1612. doi: 10.1007/s00134-017-4782-4. [DOI] [PubMed] [Google Scholar]
  • 30.Matthaeus-Kraemer CT, Thomas-Rueddel DO, Schwarzkopf D, et al. Barriers and supportive conditions to improve quality of care for critically ill patients: a team approach to quality improvement. J Crit Care. 2015;30(4):685–691. doi: 10.1016/j.jcrc.2015.03.022. [DOI] [PubMed] [Google Scholar]
  • 31.Iezzoni LI. Assessing quality using administrative data. Ann Intern Med. 1997;127(8):666–674. doi: 10.7326/0003-4819-127-8_Part_2-199710151-00048. [DOI] [PubMed] [Google Scholar]
  • 32.Iezzoni LI. Risk adjustment. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S, editors. Performance measurement for health system improvement: experiences, challenges and prospects. New York: Cambridge University Press; 2009. pp. 251–285. [Google Scholar]
  • 33.Nimptsch U, Mansky T. Quality measurement combined with peer review improved German in-hospital mortality rates for four diseases. Health Aff (Millwood) 2013;32(9):1616–1623. doi: 10.1377/hlthaff.2012.0925. [DOI] [PubMed] [Google Scholar]
  • 34.Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2015; [DOI] [PMC free article] [PubMed]
  • 35.Schouten LMT, Hulscher M, van Everdingen JJE, et al. Evidence for the impact of quality improvement collaboratives: systematic review. Br Med J. 2008;336(7659):1491. doi: 10.1136/bmj.39570.749884.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Schouten LM, Hulscher ME, van Everdingen JJ, et al. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008;336(7659):1491–1494. doi: 10.1136/bmj.39570.749884.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Hulscher M, Schouten LMT, Grol R, et al. Determinants of success of quality improvement collaboratives: what does the literature show? BMJ Qual Saf. 2013;22(1):19–31. doi: 10.1136/bmjqs-2011-000651. [DOI] [PubMed] [Google Scholar]
  • 38.Global Sepsis Alliance Quality Improvement Committee. https://www.global-sepsis-alliance.org/qic/. Assessed 17 Oct 2017.
  • 39.Brook RH, McGlynn EA, Cleary PD. Quality of health care .2. Measuring quality of care. N Engl J Med. 1996;335(13):966–970. doi: 10.1056/NEJM199609263351311. [DOI] [PubMed] [Google Scholar]
  • 40.Duckers MLA, Spreeuwenberg P, Wagner C, et al. Exploring the black box of quality improvement collaboratives: modelling relations between conditions, applied changes and outcomes. Implement Sci. 2009;4:12. doi: 10.1186/1748-5908-4-74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Ovretveit J, Gustafson D. Evaluation of quality improvement programmes. Qual Saf Health Care. 2002;11(3):270–275. doi: 10.1136/qhc.11.3.270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Kaplan HC, Brady PW, Dritz MC, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q. 2010;88(4):500–559. doi: 10.1111/j.1468-0009.2010.00611.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Bone RC, Balk RA, Cerra FB, et al. American College of Chest Physicians/Society of Critical Care Medicine consensus conference: definitions for sepsis and organ failure and guidelines for use of innovative therapies in sepsis. Crit Care Med. 1992;20(6):864–874. doi: 10.1097/00003246-199206000-00025. [DOI] [PubMed] [Google Scholar]
  • 44.Krumholz HM, Wang Y, Mattera JA, et al. An administrative claims model suitable for profiling hospital performance based on 30-day mortality rates among patients with an acute myocardial infarction. Circulation. 2006;113(13):1683–1692. doi: 10.1161/CIRCULATIONAHA.105.611186. [DOI] [PubMed] [Google Scholar]
  • 45.Krumholz HM, Lin Z, Drye EE, et al. An administrative claims measure suitable for profiling hospital performance based on 30-day all-cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4(2):243–252. doi: 10.1161/CIRCOUTCOMES.110.957498. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Bratzler DW, Normand SLT, Wang Y, et al. An administrative claims model for profiling hospital 30-day mortality rates for pneumonia patients. PLoS One. 2011;6(4) [DOI] [PMC free article] [PubMed]
  • 47.DeLong ER, Peterson ED, DeLong DM, et al. Comparing risk-adjustment methods for provider profiling. Stat Med. 1997;16(23):2645–2664. doi: 10.1002/(SICI)1097-0258(19971215)16:23<2645::AID-SIM696>3.0.CO;2-D. [DOI] [PubMed] [Google Scholar]
  • 48.Agency for Health Care Research and Quality Improvement. Quality indicator emprical methods—revised November 2014. 2015 [PDF]. http://www.qualityindicators.ahrq.gov/Downloads/Resources/Publications/2015/Empirical_Methods_2015.pdf. Accessed 13 Oct 2017.
  • 49.Krumholz HM, Brindis RG, Brush JE, et al. Standards for statistical models used for public reporting of health outcomes—an American Heart Association scientific statement from the quality of care and outcomes research interdisciplinary writing group—cosponsored by the council on epidemiology and prevention and the stroke council—endorsed by the American College of Cardiology Foundation. Circulation. 2006;113(3):456–462. doi: 10.1161/CIRCULATIONAHA.105.170769. [DOI] [PubMed] [Google Scholar]
  • 50.Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic co-morbidity in longitudinal studies—development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
  • 51.Elixhauser A, Steiner C, Harris DR, et al. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27. doi: 10.1097/00005650-199801000-00004. [DOI] [PubMed] [Google Scholar]
  • 52.Thomas-Rüddel D, Schwarzkopf D, Fleischmann C, et al. Development of a sepsis mortality prediction model for use with German hospital claims data. Intensive Care Med Exp. 2016;4(1) ESICM LIVES 2016: part two-Abstract A2504
  • 53.Ford DW, Goodwin AJ, Simpson AN, et al. A severe sepsis mortality prediction model and score for use with administrative data. Crit Care Med. 2016;44(2):319–327. doi: 10.1097/CCM.0000000000001392. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Hosmer DW, Lemeshow S. Confidence-interval estimates of an index of quality performance based on logistic regression models. Stat Med. 1995;14(19):2161–2172. doi: 10.1002/sim.4780141909. [DOI] [PubMed] [Google Scholar]
  • 55.Christianson JB, Volmar KM, Alexander J, et al. A report card on provider report cards: current status of the health care transparency movement. J Gen Intern Med. 2010;25(11):1235–1241. doi: 10.1007/s11606-010-1438-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Emmert M, Hessemer S, Meszmer N, et al. Do German hospital report cards have the potential to improve the quality of care? Health Policy. 2014;118(3):386–395. doi: 10.1016/j.healthpol.2014.07.006. [DOI] [PubMed] [Google Scholar]
  • 57.Fung CH, Lim YW, Mattke S, et al. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med. 2008;148(2):111–123. doi: 10.7326/0003-4819-148-2-200801150-00006. [DOI] [PubMed] [Google Scholar]
  • 58.Hafner JM, Williams SC, Koss RG, et al. The perceived impact of public reporting hospital performance data: interviews with hospital staff. Int J Qual Health Care. 2011;23(6):697–704. doi: 10.1093/intqhc/mzr056. [DOI] [PubMed] [Google Scholar]
  • 59.Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood) 2005;24(4):1150–1160. doi: 10.1377/hlthaff.24.4.1150. [DOI] [PubMed] [Google Scholar]
  • 60.Contandriopoulos D, Champagne F, Denis JL. The multiple causal pathways between performance measures’ use and effects. Med Care Res Rev. 2014;71(1):3–20. doi: 10.1177/1077558713496320. [DOI] [PubMed] [Google Scholar]
  • 61.Hogan H, Healey F, Neale G, et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf. 2012;21(9):737–745. doi: 10.1136/bmjqs-2011-001159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Hogan H, Healey F, Neale G, et al. Learning from preventable deaths: exploring case record reviewers’ narratives using change analysis. J R Soc Med. 2014;107(9):365–375. doi: 10.1177/0141076814532394. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Nguyen HB, Lynch EL, Mou JA, et al. The utility of a quality improvement bundle in bridging the gap between research and standard care in the management of severe sepsis and septic shock in the emergency department. Acad Emerg Med. 2007;14(11):1079–1086. doi: 10.1111/j.1553-2712.2007.tb02392.x. [DOI] [PubMed] [Google Scholar]
  • 64.Wu AW, Lipshutz AM, Pronovost PJ. EFfectiveness and efficiency of root cause analysis in medicine. JAMA. 2008;299(6):685–687. doi: 10.1001/jama.299.6.685. [DOI] [PubMed] [Google Scholar]
  • 65.Pronovost PJ, Holzmueller CG, Martinez E, et al. A practical tool to learn from defects in patient care. Joint Comm J Qual Patient Saf. 2006;32(2):102–108. doi: 10.1016/S1553-7250(06)32014-4. [DOI] [PubMed] [Google Scholar]
  • 66.Edwards MT. A longitudinal study of clinical peer review’s impact on quality and safety in US hospitals. J Healthc Manag. 2013;58:369–384. doi: 10.1097/00115514-201309000-00011. [DOI] [PubMed] [Google Scholar]
  • 67.Rink O. Wie wir Qualität verbessern. In: Martin J, Rink O, Zacher J, editors. : Handbuch IQM; 2014.
  • 68.Lilly CM. The ProCESS trial—a new era of sepsis management. N Engl J Med. 2014;370(18):1750–1751. doi: 10.1056/NEJMe1402564. [DOI] [PubMed] [Google Scholar]
  • 69.Yealy DM, Kellum JA, Huang DT, et al. A randomized trial of protocol-based care for early septic shock. N Engl J Med. 2014;370(18):1683–1693. doi: 10.1056/NEJMoa1401602. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Chen J, Ou L, Flabouris A, et al. Impact of a standardized rapid response system on outcomes in a large healthcare jurisdiction. Resuscitation. 2016;107:47–56. doi: 10.1016/j.resuscitation.2016.07.240. [DOI] [PubMed] [Google Scholar]
  • 71.Iwashyna TJ, Angus DC. Declining case fatality rates for severe sepsis good data bring good news with ambiguous implications. JAMA-J Am Med Assoc. 2014;311(13):1295–1297. doi: 10.1001/jama.2014.2639. [DOI] [PubMed] [Google Scholar]
  • 72.Feinstein AR, Sosin DM, Wells CK. The will Rogers phenomenon. N Engl J Med. 1985;312(25):1604–1608. doi: 10.1056/NEJM198506203122504. [DOI] [PubMed] [Google Scholar]
  • 73.Felsenstein M. Peer Review: Lernen auf Gegenseitigkeit. Dtsch Arztebl Int. 2011;108(31–32):A-1688-A-1688. [Google Scholar]
  • 74.Gerber JS, Prasad PA, Fiks AG, et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians a randomized trial. JAMA-J Am Med Assoc. 2013;309(22):2345–2352. doi: 10.1001/jama.2013.6287. [DOI] [PubMed] [Google Scholar]
  • 75.Hemming K, Taljaard M. Sample size calculations for stepped wedge and cluster randomised trials: a unified approach. J Clin Epidemiol. 2016;69:137–146. doi: 10.1016/j.jclinepi.2015.08.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Jolley RJ, Sawka KJ, Yergens DW, et al. Validity of administrative data in recording sepsis: a systematic review. Crit Care. 2015;19:12. doi: 10.1186/s13054-015-0847-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Bhardwaj A, Mikkelsen ME. Sepsis quality improvement initiatives: prepare for the marathon, not the sprint. Crit Care Med. 2017;45(2):374–375. doi: 10.1097/CCM.0000000000002110. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Not applicable.

Articles from Implementation Science : IS are provided here courtesy of BMC

RESOURCES