A Roadmap for Successful State Sepsis Regulations—Lessons From New York

Supplemental Digital Content is available in the text.

OBJECTIVES:

New York state implemented the first state-level sepsis regulations in 2013. These regulations were associated with improved mortality, leading other states to consider similar steps. Our objective was to provide insight into New York state’s sepsis policy making process, creating a roadmap for policymakers in other states considering similar regulations.

DESIGN:

Qualitative study using semistructured interviews.

SETTING:

We recruited key stakeholders who had knowledge of the New York state sepsis regulations.

SUBJECTS:

Thirteen key stakeholders from three groups included four New York state policymakers and seven clinicians and hospital association leaders involved in the creation and implementation of the 2013 New York state sepsis regulations, as well as two members of patient advocacy groups engaged in sepsis advocacy.

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

We used iterative, inductive thematic analysis to identify themes related to participant perceptions of the New York state sepsis policy, factors that influenced the policy’s perceived successes, and opportunities for improvement. We identified several factors that facilitated success. Among these were that policymakers engaged a diverse array of stakeholders in development, allowing them to address potential barriers to implementation and create early buy-in. Policymakers also paid specific attention to the balance between the desire for comprehensive reporting and the burden of data collection, narrowly focusing on “essential” sepsis-related data elements to reduce the burden on hospitals. In addition, the regulations touched on all three major domains of sepsis quality—structure, process, and outcomes—going beyond a data collection to give hospitals tools to improve sepsis care.

CONCLUSIONS:

We identified factors that distinguish the New York sepsis regulations from less successful sepsis polices at the federal level. Ultimately, lessons from New York state provide valuable guidance to policymakers and hospital officials seeking to develop and implement policies that will improve sepsis quality.

Sepsis is the dysregulated immune response to infection that results in life-threatening organ dysfunction (1). Sepsis is a leading cause of morbidity and mortality, resulting in more than 750,000 hospitalizations in the United States each year with a mortality rate up to 30% (2). Hospital spending is in excess of $20 billion dollars per year on sepsis, making it the costliest medical condition in U.S. hospitals (3, 4). Evidence-based practices such early antibiotics and fluid resuscitation can reduce sepsis mortality and are strongly recommended by clinical practice guidelines (5, 6). However, patients frequently do not receive guideline concordant care, creating a serious gap between clinical evidence and clinical practice (7–9).

To address this problem, sepsis is increasingly the target of local, regional, and national performance improvement initiatives designed to increase the use of evidence-based practices (10, 11). Recently, several state governments have taken these efforts a step further, issuing regulations that mandate the use of protocols for early recognition and treatment of sepsis (11). In 2013, New York state became the first to issue such regulations, known as “Rory’s Regulations” after a 12-year-old boy, Rory Staunton, who died from sepsis in a New York hospital (12). These regulations mandate that all hospitals in the state adopt sepsis protocols, provide sepsis education to hospital staff, and report protocol adherence and patient outcomes to the state government (Table 1) (13). These regulations were followed by similar policies in Illinois and New Jersey, and over one-third of state governments are considering similar policies (10).

TABLE 1.

Description of the New York State Sepsis Regulations

Component	Description
Protocols for timely recognition	Algorithms or alert systems for sepsis recognition for both adults and children
	Procedures for obtaining blood cultures and identifying infectious source prior to antibiotics
Protocols for timely treatment	Treatment protocols with specific physiologic targets and specified time frames based on:
	• Timely antibiotics
	• Timely IV fluid
	• Appropriate hemodynamic monitoring using either invasive or noninvasive means
	Separate protocols and procedures for adult and pediatric patients
Staff education	Regular training of emergency department and inpatient staff regarding protocol implementation
Mandatory reporting	Submission of protocols to NY Department of Health for review
	Reporting of adherence to sepsis measures consistent with National Quality Forum to NY Department of Health
	Eventual public reporting of sepsis quality data

NY = New York.

Authors’ summary of Parts 404.2 and 405.4 of the New York Codes, Rules, and Regulations.

Early evidence suggests that the New York state sepsis regulations are achieving their desired impact. Difference-in-difference analyses show that sepsis mortality decreased at a greater rate after the regulations in New York compared with states without sepsis regulations, among both adult and pediatric populations (14, 15). There is also little evidence of serious unintended consequences associated with the regulations, such as increased Clostridium difficile infections that might result from overuse of antibiotics, increased use of intensive care, or increased costs (14–16).

Despite this evidence, state sepsis regulations are not without controversy. The New York state regulations go beyond traditional government efforts to improve the quality of care, which are typically limited to either financial incentives (e.g., pay-for-performance) or market-based incentives (e.g., public reporting of quality data). Instead, they take the unprecedented step of directly mandating the use of specific evidence-based practices (13). This strategy is controversial because clinical evidence can change over time, and overly prescriptive regulations can limit clinicians’ ability to individualize care at the bedside (17). Another concern with state sepsis regulations is that they can conflict with federal sepsis regulations. The U.S. Centers for Medicare and Medicaid Services (CMS) recently added sepsis to the list of conditions included in its Hospital Inpatient Quality Reporting Program (18). Yet the CMS program uses a different definition for sepsis and different data elements compared with the New York state regulations, complicating the work of hospitals that must reconcile state and federal sepsis policies (19).

Against this backdrop, we interviewed key stakeholders involved in the design and implementation of the New York state regulations, including policymakers, clinicians, hospital association leaders, and representatives from patient advocacy groups. We sought to understand “what worked” and “what did not” in the development of the New York state regulations. Our goal was to provide insight into the policy making process and create a roadmap for policymakers in other states considering the implementation of similar evidence-based regulations.

MATERIALS AND METHODS

We performed a qualitative study of key stakeholders involved in the creation and implementation of the 2013 New York state sepsis regulations. We targeted three distinct stakeholder groups: New York state policymakers involved in drafting the regulations, physician clinicians and hospital association leaders who were involved in developing and implementing the regulations, and members of patient advocacy groups engaged in national sepsis advocacy during the development and implementation process. We used snowball sampling to identify informants with relevant knowledge and experience. Potential respondents were contacted via electronic mail, and all interviews were conducted via telephone. The University of Pittsburgh Human Subjects Protection Office approved this study (study number STUDY19070034). All participants provided informed consent.

We used a semistructured interview guide developed through iterative discussion among the investigators using the Donabedian model to understand how New York state sought to improve sepsis care through quality improvement processes (20). To refine the interview guide, we conducted pilot interviews with four nonstudy participants and made revisions for content and clarity based on their feedback. The final interview guide consisted of 11 questions in four thematic areas: motivators for the policy, perceptions of the policy, factors that influenced perceived successes, and opportunities for improvements in sepsis policy development or implementation. The full interview guide is available in Supplementary File 1 (http://links.lww.com/CCX/A765).

Interviews were conducted between April and August 2019. We conducted interviews until thematic saturation was reached, which we defined as no new themes identified after two consecutive interviews (21). Interviews were audio-recorded, transcribed verbatim, deidentified, and uploaded into the NVivo software package (QSR International, Chadstone, VIC, Australia) for analysis. We then used iterative, inductive coding to identify the key organizational and environmental factors that influenced the effectiveness of New York state regulations. Coding was guided by the Consolidated Framework for Implementation Research, a conceptual model that emphasizes the influence of contextual and environmental factors in the implementation process (22). The codebook included thematic categories and subcategories, each with a unique definition, and included code inclusion and exclusion criteria. Three researchers individually coded transcripts, and after each transcript was coded, the coders met to discuss and resolve discrepancies through consensus for codes that fell below 80% agreement. The results are presented as a series of themes based on these codes, along with supportive quotes.

RESULTS

Data Collection

Thematic saturation was reached after 13 interviews, with the 13th interview adding no additional themes to the codebook. The final sample included four physician clinician representatives on the New York Department of Health Clinicians Advisory Committee, four New York state policymakers involved in drafting the regulations, three hospital association leaders who were involved in developing and implementing the regulations, and two members of patient advocacy groups who participated in national sepsis advocacy during the development and implementation process. Interviews averaged 39 minutes in length.

Factors That Facilitate Success

Thematic content analysis revealed five key facilitators for success of the New York state regulations: 1) using a multidisciplinary stakeholder advisory group, 2) thinking strategically about data collection and management, 3) emphasizing flexibility throughout the process, 4) keeping a broad focus in terms of quality domains and policy levers, and 5) capitalizing on existing expertise and infrastructure in the state (Table 2).

TABLE 2.

Key Facilitators for Effective State-Level Sepsis Regulations

Domain	Illustrative Quotes
Multidisciplinary advisory group	The Department of Health has been super inclusive and collaborative in its development …they took that to heart, the novelty of that...they spend a great deal of time hearing from each and every stakeholder.—Hospital Association Leader
Data collection and management	So there were a lot of potential barriers around ensuring data quality, what the burden looks like for hospitals to do that, in terms of reporting accurate data and complete data. So there was a lot of this back and forth about what types of data are needed.—Clinician
Flexibility	The flexibility that the state department of health was willing to roll out this program with was also very important because they knew that as much as we tried, the first time out was not going to be where it landed…—Hospital Association Leader
	But the science wasn’t…the foundation; the framework wasn’t as solid as it is for other conditions. So…each hospital was allowed the latitude to develop the protocol that would work for their organization.—Hospital Association Leader
Breadth of focus	“what do we do with this [data]? How do we learn from it? And, how do we improve”…publicly releasing data…includes hospitals looking at their own data, benchmarking, and trying to understand, to put together outcome data, structure data and process data.—Policymaker
	I would caution them against looking at this as a data collection exercise only…I would urge them to focus on the clinical processes and the education, and have the measurement come second.—Hospital Association Leader
Capitalizing on existing infrastructure	Hospital associations really helped with trying to put hospitals that were having more challenges in touch with hospitals who were perhaps doing a little bit better, or were a little bit further along from them…—Clinician

With regard to the role of a multidisciplinary advisory group, interviewees described how New York convened a large advisory group that included clinicians, hospital representatives, epidemiologists, and data science experts. They took a “big tent” approach—there was no such thing as too many stakeholders. This group helped create buy-in among hospitals, ensuring that the regulations flowed from bedside to policy rather than policy to bedside. That is, policymakers first asked, “what do we want to do at the bedside?” and then asked, “what regulations can help get us there?,” rather than first envisioning a policy and then determining how it would work at the bedside. Through this group, clinicians had early input about what elements were most important, leading to not only transparency but also a feeling that stakeholder input improved the regulations, increasing trust in the regulatory process.

With regard to data collection and management, interviewees described how from the outset they sought to strike a balance in terms of data collection. They knew that hospitals would need to collect a plethora of data on sepsis patients for there to be rigor and accountability through public reporting. Yet, they also knew that too great a data collection burden might lead to opposition among hospitals. To strike this balance, regulators decided to require only the bare minimum number of data elements necessary—no “add-ons” were allowed for research or other purposes. The regulators also acknowledged up front that the data collection tools (i.e., the “data dictionary” that would inform data collection and reporting) were living documents that would evolve over time. Indeed, as of this writing, they are on their 10th iteration. Communicating this gave hospitals the sense that they would continue to have input and that any concerns about the first iteration could be addressed in future iterations. In addition, the regulators also made use of outside statisticians and employed regular data audits. These decisions ensured that hospitals viewed the system as neither unfair nor overly burdensome.

With regard to flexibility throughout the process, there was the upfront realization among all stakeholders that the regulations themselves would change over time. In discussions, policymakers acknowledged that they were not likely to get it right on the first attempt. This issue was most salient to the idea of the data dictionary. As noted above, the data dictionary was designed not only to evolve but to evolve in a way that would be driven by the data collectors themselves. This issue was also salient in the context of the regulations writ large, which were written broadly so that hospitals could be flexible with implementation. Rather than proscribing a “one-size-fits-all” approach, hospitals could implement the regulations in a manner that fit within their institutional culture and processes.

The idea of flexibility reflected the belief among regulators that science itself is not settled, that is, there are still gaps in our understanding of ideal sepsis management. It was important that policymakers explicitly acknowledged that new knowledge about sepsis was likely to develop over time, and the regulations should adapt to include new evidence. Interviewees cautioned that if regulators were to imply that the evidence is better than it really is, clinicians would not support the regulations and the effort would fail. By acknowledging that the science is evolving, policymakers won the respect of the clinicians, who, in turn, acknowledged that, in the face of uncertain evidence, it was better to do something than do nothing.

With regard to breadth of focus, interviewees highlighted several ways in which the regulations succeeded in part because they took a broad approach toward sepsis quality improvement. For example, regulators intentionally wrote the regulations to address multiple domains of quality. Specifically, the regulations address all three components of the classic Donabedian model of quality: structure, process, and outcome. They address structure by mandating protocols and education as quality improvement facilitators; they address process by specifying multiple evidence-based sepsis care practices (i.e., early antibiotics and fluid resuscitation), and they address outcome by including risk-adjusted mortality as a quality measure. By including structure, the regulations enabled immediate accountability and told hospitals what steps to take to address sepsis care. By also including outcomes, the regulations ensure that the focus remains on patients. This “soup to nuts” approach to regulation stands in contrast with more traditional regulatory strategies that only address process, such as the CMS Severe Sepsis and Septic Shock: Management Bundle, known as SEP-1 (18).

The regulations are also broad in terms of the various policy levers employed. At their core, the regulations mandated adoption of protocols for evidence-based practice. Yet interviewees also saw value in inclusion of public reporting of quality data via a state website—this helped get the attention of hospital administrators and provided strong incentive to comply. Interviewees also saw value in how the regulations mandated staff education—this ensured that the regulations were not just a data exercise but involved engagement with bedside providers.

With regard to capitalizing on existing expertise and infrastructure, the regulations benefited from the presence of several active hospital associations in New York state. These associations served as de facto quality improvement committees, helping low-performing hospitals learn from high-performing hospitals. Policymakers realized that sepsis regulations do not exist in vacuum nor are they a panacea for sepsis quality. Even after implementing the regulations, hospitals had to address local-quality barriers. Thus, a goal of the regulations was to supercharge existing quality improvement and support ongoing efforts, not necessary to spur new quality improvement, which is a much harder task.

Potential Road Blocks

In addition to these five facilitators for success, thematic content analysis revealed two key roadblocks to look out for during the development and implementation process: 1) failure to justify the need for a policy response and 2) failure to synergize with existing federal sepsis policies (Table 3).

TABLE 3.

Potential Roadblocks to Success

Domain	Illustrative Quotes
Failure to justify the need for a policy response	Mortality was high and the variation in care was significant and unexplained by…seemingly unexplained by patient differences. That made the ability to make the argument for the need for this a little bit easier than it might have been otherwise.—Policymaker
	This wasn’t about just reporting good numbers. It was actually about fixing the underlying system issues that prevented us from providing topnotch care to some patients with sepsis.—Clinician
Failure to synergize with existing federal sepsis policies	We’re looking at ways to align this with other requirements; the national requirements, and also just what makes sense for the day to day because the department as well as the clinicians don’t want nonvalue added busy work that’s not contributing to good outcomes for the patient, which is what this is all about.—Hospital Association Leader
	Sep One just requires that you report data, but they don’t require that you have a protocol in place in your hospital to help you identify patients with sepsis early, or protocols to manage patients with sepsis, or have a structured quality improvement process for sepsis in your hospital. And I think all of those are really key elements.—Clinician

With regard to the risk of failing to justify a policy response, interviewees acknowledged that regulations of this scale were largely unprecedented at the time. Hospitals might reasonably not support the effort if there were concerns about why large-scale regulations were needed. To overcome this problem, it was necessary to repeatedly emphasize the public health burden of sepsis, acknowledge even hospitals with active sepsis quality improvement efforts still can experience poor outcomes, and stress that failure to deliver evidence-based sepsis care is a system failure, not a failure of individual clinicians.

With regard to the risk of not synergizing with the federal program known as SEP-1, interviewees noted the importance of alignment with SEP-1 specifically with regard to data collection to minimize the burden on hospitals. Otherwise, hospitals will be frustrated collecting similar but not identical data for two different programs. Yet interviewees also felt strongly that SEP-1 was not enough and that, even in the presence of SEP-1 regulations, the New York state regulations were needed. Key differences noted by the interviewees were that SEP-1 applied only to adults, not children; and that SEP-1 is simply a reporting exercise and does not include a mandate for hospitals to take steps to improve sepsis care delivery processes. Ultimately, SEP-1 provides a floor, not a ceiling. By implementing regulations that enable hospitals to innovate on top of SEP-1, the regulations could be much more impactful than SEP-1 alone.

DISCUSSION

Our study outlines the lessons learned in New York state during the regulatory process, thereby providing a tractable roadmap that policymakers can follow should they choose to adopt sepsis regulations similar to those in New York state. Given the extensive evidence that the New York state regulations were associated with improvements in sepsis quality and the ongoing efforts of multiple patient advocacy groups, it is likely that statehouses across the country will be grappling with this issue in years to come. The context provided by our study of the successful development of the New York state sepsis regulations is an essential first step to supporting the successfully implementation of sepsis regulations in other states.

Some of our findings are broadly generalizable to all healthcare policy making. For example, the advice to begin with a multidisciplinary stakeholder group and to leverage existing quality improvement networks in order to facilitate buy-in is important no matter what policy is under discussion (23, 24). However, other findings in our study are specific to quality mandates, which have recently emerged as a novel approach to incentivizing quality improvement at the state level (12). For example, the idea that flexibility and the capacity to evolve should be paramount to regulation and is advantageous to preventing overly rigid mandates from meeting untoward resistance, ultimately hampering effectiveness of the regulations. Additionally, alignment with federal sepsis definitions and metrics as a base from which to build more comprehensive state regulations reduces burden associated with new regulations and shifts the focus toward the value added by additional state-level regulations (25, 26).

In addition to providing a roadmap for designing and implementing effective healthcare policy, our study identifies several reasons why traditional policies to incentivize healthcare policy often fail to affect meaningful change. Multiple studies demonstrate that the impacts of public reporting and pay-for-performance are modest at best and can exacerbate health disparities (27–31). In the field of sepsis, the CMS regulations have notably had no discernable impact on either performance or outcomes (32). Our findings demonstrate that substantial impact can be achieved through more comprehensive efforts that address multiple domains of quality within a single disease and employ multiple levers for quality improvement. It is true that regulations of this type are necessarily intensive. Further, even a comprehensive approach to sepsis regulations may not be a recipe for success in all settings, since the overall impact of these regulations is likely dependent on highly local factors such as the availability of hospital resources, leadership buy-in, and sustained efforts to educate providers and staff about sepsis care. However, the clear lesson of the New York state regulations is that even intensive regulations can be successful at a population-level when they are accompanied by comprehensive efforts to achieve local stakeholder engagement.

Our results also provide early insight on why the New York state regulations appear to have worsened health disparities for sepsis (33). The New York regulations were powerful in part because they were comprehensive, but they were also resource-intensive. Additionally, they capitalized on existing quality improvement networks created by state hospital associations. Hospitals serving underrepresented minorities are less likely to be well-resourced and, thus, at a disadvantage to quickly and practically respond to the regulations (34–37). These hospitals are also likely to be actively engaged in networks that help facilitate hospital quality improvement in response to the regulations (38). It is important that policymakers anticipate these issues when designing sepsis regulations that can support all hospitals efforts to achieve equity in sepsis performance.

First, we describe the experience of a single state, New York, such that our results may not generalize to other states. However, New York is a large state with a heterogeneous population and regions, and it was the first state to attempt and successfully implement sepsis regulations. In addition, robust evidence indicates that the New York regulations led to substantial reductions in sepsis mortality (14, 15). As such, understanding the New York experience is critically important to helping other states implement successful sepsis regulations. Second, we interviewed a relatively small number of stakeholders and included only physician clinicians. Nonetheless, we achieved thematic saturation making additional interviews duplicative, and our respondents comprised a broadly representative group of individuals present throughout the conception, development, and implementation of the regulations. Third, as a qualitative study, we were not able to definitively test hypotheses about which factors most contributed to the success of the regulations (39). Rather, this study is intended to provide context and best practices for other states considering adoption of evidence-based sepsis regulations. This study has several limitations.

CONCLUSIONS

Overall, state policymakers and advocacy groups can capitalize on these lessons when developing state sepsis regulations. State-level mandates for quality are, by definition, proscriptive and invasive. Yet, as we demonstrate through interviews with stakeholders engaged in developing and implementing the New York state sepsis policy, it is possible to design these policies in ways that are acceptable to hospitals and achieve substantial positive results.