From Experiment to Excellence: The Impact of Patient Safety Learning Laboratories

Abstract

To rapidly advance patient safety research, in 2014 the US Agency for Healthcare Research and Quality launched a radically different research initiative by supporting patient safety learning laboratories (PSLLs) using systems perspectives and engineering approaches to advance patient safety. The 5-phase systems engineering methodology uses diverse methods and devotes particular attention to health care safety problem analysis, followed by design, development, implementation, and evaluation. PSLL projects have demonstrated decreases in mortality as well as increases in diagnostic accuracy, reduction in adverse drug events, decreased medication errors, improved early detection of adverse events, and reduction in the number of prenatal adverse events. PSLLs have developed guidance and resources to prevent as well as mitigate patient harm and improve the safety, efficiency, and effectiveness of health care delivery. By fusing approaches ranging from human-centered design to AI-driven analytics applied to health services research, PSLLs have produced influential, evidence-based, scalable interventions that strengthen health care delivery processes and improve outcomes for society, health care organizations, providers, and—most importantly—patients and their families.

Discussions surrounding the definition and quantification of harm from health care skyrocketed following the publication of To Err is Human ¹ in 2000, prompting exam of health care delivery processes that impede patient safety. However, the subsequent decade focused on addressing health care delivery “errors,” based on the underlying assumption that health care providers were the source of failure in health care systems rife with inherent hazards. This perspective yielded some improvements in patient safety but likely contributed to provider frustration and low morale.

Amid this far-from-ideal state of health care, in 2014 the US Agency for Healthcare Research and Quality (AHRQ) launched a radically different research initiative—the patient safety learning laboratories (PSLLs)—to advance patient safety.² The ultimate goal was to improve the safety of health care systems; however, an important additional intention of the program was to shift the paradigm and make health services research more robust by addressing health care delivery through a systems perspective and by integrating engineering approaches, methods, tools, and techniques into health care research.³

PSLL projects use a systems engineering 5-phase methodology, which is an approximation of the widely used V-model.⁴ That methodology begins with devoting resources to problem analysis and proceeds through design, development, implementation, and evaluation. The first step, defining the problem, is often the most difficult part of an improvement process, particularly if there is a rush to get underway.⁵ Problem analysis goes beyond literature and evidence reviews and utilizes a broad range of systems and human factors (HF) engineering tools such as work and cognitive task analysis, patient journey and process mapping, direct clinical observations, simulation, and focus groups and interviews with thematic analysis. The analysis is designed to anticipate the potential for evolving patient care conditions in dynamic environments.5

PSLL researchers identify an important patient safety problem to solve and explore solutions without prematurely closing on a hypothesis that may not address enough of the multiple competing objectives or demands inherent in health care delivery. A critical step is accomplished during problem analysis—getting to know the requirements, needs, and expectations of all stakeholders and users involved in the problem and potentially in the proposed solution.

To understand health care delivery processes, PSLLs use a variety of schematic techniques adapted from systems engineering including workflow, affinity, causal loop, and classification tree diagrams. The selection of tools and techniques varies depending on the focus and characteristics of the problem, time dependency, potential for quantification, and whether problem analysis proceeds from big picture to small details, or vice versa. This step also demonstrates a paradigm shift in health care. While these tools are usually used retrospectively, to look for what went wrong when an intervention failed to deliver the desired outcome, the PSLLs also use them prospectively. The breadth and diversity of systems engineering and human factors methods used proactively and successfully across the PSLLs demonstrate the value of using these techniques to solve health care delivery problems.

As PSLL researchers progress through the next 4 phases of design, development, implementation, and evaluation, insights gained from actual patient care circumstances are iteratively included as interventions are conceptualized, developed, and studied. Unlike traditional health services research, where a proposed intervention is well-defined before the study initiation, the PSLLs take advantage of human-centered design or design thinking as a fundamental framework, utilizing a wide variety of collaborative and inclusive methods such as participatory or codesign sessions. PSLLs use diverse narrative and visualization techniques, such as designing solutions based on scenarios, storyboards or journey maps, developing persona such as avatars, brainstorming, and seeking to understand the users’ wants and needs, to develop tangible design prototypes, such as wireframe visual guides for apps, and mockups, and virtual or augmented reality. These tangible artifact designs are developed in an iterative manner through rapid prototyping, testing, and refining cycles until an appropriate evaluation and feedback mechanism indicates a satisfactory version that passes initial usability evaluation, validation, and verification.

The PSLL process requires multidisciplinary collaboration; each phase of the methodology is improved by incorporating diverse, complementary sources of information and experience. Grantees approach patient safety problems and potential solutions from a systems perspective. They form a multidisciplinary team to design and develop an intervention, use methods, tools, and techniques from other disciplines, engage people who would use or be impacted by potential solutions from the beginning to the end, and make a scientifically rigorous effort to improve the patient’s therapeutic, diagnostic, or quality of life experience.

The insights of health care providers, who understand and appreciate the conditions and complexities at the front line of health care delivery and the realities of how work is actually done⁶ are enriched by the perspectives of experts in design, architecture, systems and HF engineering, implementation science, and other sciences. These complementary types of expertise inform work processes that are safer, more effective and efficient, and satisfying.

As the federal agency charged with improving the quality and safety of health care delivery,⁷ AHRQ has incorporated the 5-phase methodology into a series of notices of funding opportunities (NOFOs) for PSLLs for about a decade. AHRQ’s support for the PSLL projects and their unique approach exemplifies how AHRQ has been, and continues to be, a leader in the integration of:

1. Attention to the complex, adaptive nature of health care delivery systems.

2. Development of interventions that address the interactions across health care delivery systems.

3. Assessment of workload and work-system processes directly.

While much health care delivery research uses proxy measures, such as staff turnover, the outcomes of many of the PSLL projects are improvements in entire work-systems and their components and processes that can be measured directly. The PSLLs examine the work-system, including people, tasks, technology, and the organization, to determine whether the implemented interventions produce changes among the work-system components that then improve conditions for health care workers and safety outcomes for patients.

The PSLL research program has supported 6 distinct cohorts of PSLLs. The PSLL projects have covered a significant part of the health care landscape, including emergency and acute care, medication and diagnostic safety, early detection of patient deterioration, neonatal, pediatric and maternal care, chronic disease management, surgical and perioperative safety, and mental and behavioral health.

PSLL projects have encompassed all populations, with far-reaching impact. More than half of the PSLLs include low-income populations, followed by minority groups (46.8%), special needs care (36.2%), children (34.0%), chronic care (27.7%), and elderly (23.4%) populations.

Findings have been broadly cited, influencing patient care, policy development, and future research in patient safety. Thus far, a total of 47 PSLL grants, 18 of which are ongoing, have resulted in 673 publications with a total of more than 11,700 citations, supported by ∼$125 million in total funding. Several projects have each generated more than 1000 citations.

The impact of these grants extends beyond citations; 36 projects generated toolkits, 27 projects generated training programs, and 26 generated clinical guidance, in addition to online resources, databases, conferences, learning networks, fact sheets, and commentaries.

PSLL funding has been distributed across the country, with Maryland receiving 6 awards, followed by California, Massachusetts, Ohio, Michigan, and South Carolina, which each received 4 awards. More than 90% of the PSLLs involve multiple institutions, with 10 projects extending nationally and 2 internationally to influence broader health care systems and policy decisions.

Many PSLL projects have already demonstrated clear, measurable impacts. For example, Blike’s PSLL at Dartmouth⁸ and Chaudhry’s PSLL at Yale University⁹ reported decreases in mortality due to their interventions. Hinson at Johns Hopkins University,¹⁰ Horwitz at New York University,¹¹ and Dalal and Bates at Brigham and Women’s Hospital¹² demonstrated improved diagnostic safety; Sarkar at the University of California-San Francisco¹³ improved medication adherence; Sapirstein at Johns Hopkins¹⁴ decreased medication errors; Singer at Harvard¹⁵ improved postoperative adverse events; and Stuebe at the University of North Carolina¹⁶ reduced the number of perinatal adverse events. Quality improvement outcomes include Halamek at Stanford¹⁷ improving communication, simulation training, and environmental design to strengthen systems that support perinatal safety; Pickering at Mayo Clinic¹⁸ designing informatics systems to address diagnostic complexity; Smith at the University of Wisconsin-Madison¹⁹ developing a patient safety passport to support safer transitions of older adults; and Walsh at Cincinnati Children’s Hospital Medical Center²⁰ creating tools to improve at-home medication safety and patient/caregiver engagement. Research led by Blike, Halamek, Joseph, and Bayramzadeh are included in this supplement issue.

In addition, several PSLLs reported on sustaining their PSLLs beyond AHRQ funding by securing funding from other sources to continue with their research, by incorporating grant artifacts into teaching programs, or by introducing new practices in their sites. Lead PSLL investigators have also been recognized for their innovative contributions and promoted to higher positions in their institutions, where their impact on health care delivery has been amplified.

The artifacts from some PSLLs, such as Benneyan’s hospital surge capacity tools²¹ (Principle Investigator: Singer) from Northeastern University,²² and Dykes and Bates’ Fall TIPS: A Patient-Centered Fall Prevention Toolkit²³ from Brigham and Women’s Hospital,²⁴ have been adopted by other institutions resulting in widespread use and sustainment of the research. PSLLs led by Joseph at Clemson and Bayramzadeh at Kent State provided design guidelines for future operating rooms²⁵ and trauma centers.²⁶ Sapirstein’s PSLL at Johns Hopkins University^14,27 modeled the beneficial application of systems engineering when designing an automated infusion pump connected to an electronic health record for optimal safety. Bonafide’s PSLL at Children’s Hospital of Philadelphia won the 2021 Association for the Advancement of Medical Instrumentation (AAMI) and Becton Dickinson’s Patient Safety Award²⁸ for outstanding achievement in advancing patient safety by designing and developing more responsive alarm technology, a decision support system, and national guidance for pediatric monitoring.²⁹

Investments in the PSLL program at AHRQ have supported skilled and innovative researchers whose work has significantly improved the safety of health care delivery. By fusing approaches ranging from human-centered design to AI-driven analytics applied to health services research, PSLLs have produced influential, evidence-based, scalable interventions that strengthen health care delivery processes and improve outcomes for health care organizations, providers, and—most importantly—patients, and their families.