Perioperative Quality Improvement: Opportunities and Challenges for Pragmatic Trial Designs

In this issue of Anesthesia & Analgesia, Carr and colleagues describe their successful departmental efforts to increase the rate of neuromuscular monitoring in patients undergoing non-cardiac surgery with pharmacological neuromuscular blockers.¹ Using an interrupted times series approach to the analysis, the authors demonstrate the association of their educational quality improvement initiative with a clinically significant increase in train-of-four monitoring. Although quality improvement initiatives encompass a broad spectrum of activities, they are usually defined as small-scale cycles of assessments and interventions that have the goal of improving the process, outcome, and efficiency of complex healthcare systems.² The goals of quality improvement have been described in several frameworks, with one of the most widely accepted originating from the Institute of Medicine (IOM) in its “Crossing the Quality Chasm: A New Health System for the 21st Century,”³ which defines six aims (Table 1).

Table 1: The Six Aims of Healthcare Quality.

Adapted from the Institute of Medicine with permission.³

Aim	Definition
Safe	Avoiding harm to patients from the care that is intended to help them.
Effective	Providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoiding underuse and misuse, respectively).
Patient-centered	Providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions.
Timely	Reducing waits and sometimes harmful delays for both those who receive and those who give care.
Efficient	Avoiding waste, including waste of equipment, supplies, ideas, and energy.
Equitable	Providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status.

Quality improvement efforts are omnipresent in a modern healthcare system. For individuals driving these efforts, sharing the results of successful—and unsuccessful—efforts to achieve the six aims of healthcare quality (Table 1) is in society’s best interest and thus should be encouraged. In this context, the question regularly arises as to whether or not a quality improvement effort constitutes research. The revised 2018 United States Department of Health and Human Services defines research as “…a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge.”⁴ One can argue that any quality improvement initiative aims to generate “generalizable knowledge,” so the practical utility of this definition has been called into question. Two criteria to distinguish quality improvement from research have been proposed: (1) the majority of patients should likely benefit directly from the knowledge to be gained, and (2) risks or burdens beyond the standard of practice should not occur.²

Carr and colleagues used an elegant, interrupted times series approach to carefully assess the association of their three-staged educational intervention to improve the uptake of neuromuscular monitoring by anesthesia clinicians.¹ The rationale for using the interrupted times series analysis is to account for time-dependent trends in the outcome of interest that are not related to the intervention. An inherent weakness of any before-and-after study include systematic error or bias, such as regression to the mean: a period of exceptionally low neuromuscular monitoring use is likely to be followed by a period of higher neuromuscular monitoring use unrelated to the intervention, or the Hawthorne effect: if clinicians are aware that they are being monitored, they may be more likely to use neuromuscular monitoring regardless of any education. Nevertheless, these sources of error or bias can be accounted for in a before-and-after study using appropriate design and analysis.⁵

Pragmatic clinical trials using cluster randomization seek to address the effects of time-dependent confounding. Rather than randomizing participants at the patient level, participants are often exposed to the intervention of interest based on an organizational unit, which is referred to as a cluster.⁶ Examples of clusters include clinics, hospital wards,⁷ or even different hospitals.⁸ In an alternating multiple crossover cluster controlled trial, a single before-after test of an intervention is conducted multiple times, thereby minimizing systematic error from regression to the mean and Hawthorne effect.⁶

Due to the increasing popularity of pragmatic trial designs, many institutional review boards have developed processes to evaluate pragmatic clinical trials. Pragmatic trials are unique in that they are commonly characterized by: (1) a large number of participants, with few exclusion criteria, (2) a real-world setting, (3) a choice of patient-centered outcomes, and (4) the generation of results that are more broadly applicable to clinical practice.⁶ These characteristics overlap with the six aims of healthcare quality that define quality improvement projects (Table 1). Yet, pragmatic trials are primarily conceived as research projects, and, hence, institutional review boards are tasked to evaluate their risks and benefits to ensure that risks to human subjects are minimized. A key difference to the approval of a pragmatic trial compared to a conventional randomized controlled trial with patient-level randomization is that a pragmatic trial often includes a waiver of consent from participants. An institutional review board may approve a waiver of consent if: (1) the research involves no more than minimal risk to the subjects; (2) the waiver will not adversely affect the rights and welfare of the subjects; (3) the research could not practicably be carried out without the waiver; and (4) the subjects will be provided with additional pertinent information after participation, if appropriate.⁹

When approached from a cost-effectiveness perspective, perioperative care can be delivered in a high-value, low-value, or no-value format. High-value care means reducing costs while improving outcomes, increasing costs modestly for improved outcomes, or decreasing costs with acceptable declines in clinical outcome. Low-value care means higher costs with only small improvement in outcomes, or lower costs with substantial deterioration of outcomes. Finally, no-value care harms patients.¹⁰ Accordingly, quality improvement projects should be designed and then proven to increase high-value care.

The integration of local perioperative quality improvement into multi-institutional collaboratives may further provide an opportunity for cost-savings. In a recent retrospective observational study comparing eight hospitals participating in the Anesthesiology Performance Improvement and Reporting Exchange Collaborative Quality Initiative with eight matched, non-participating hospitals, researchers observed a significant USD719 reduction in total episode payments for major high-volume surgical procedures in the participating hospitals.¹¹ Determined value from these projects is not just cost, although that is certainly the easiest to measure. A healthcare system looks at the opportunity for standardization with increased quality, decreased cost, reduced adverse events, and improved workflow and work stress of our care teams. Each of these measures has both a cost and a reputational benefit.^{12, 13}

In addition to leveraging a multi-center approach to assist in benchmarking and learning from other clinicians and healthcare systems, applying pragmatic design approaches, advanced analytic solutions to existing data, and assessments of costs can be key to increase the positive impact of perioperative quality improvement efforts in the future.