Improving healthcare quality through organisational peer-to-peer assessment: lessons from the nuclear power industry

Abstract

Healthcare has made great efforts to reduce preventable patient harm, from externally driven regulations to internally driven professionalism. Regulation has driven the majority of efforts to date, and has a necessary place in establishing accountability and minimum standards. Yet they need to be coupled with internally driven efforts. Among professional groups, internally-driven efforts that function as communities of learning and change social norms are highly effective tools to improve performance, yet these approaches are underdeveloped in healthcare. Healthcare can learn much from the nuclear power industry. The nuclear power industry formed the Institute of Nuclear Power Operators following the Three Mile Island accident to improve safety. That organization established a peer-to-peer assessment program to cross-share best practices, safety hazards, problems and actions that improved safety and operational performance. This commentary explores how a similar program could be expanded into healthcare. Healthcare needs a structured, clinician-led, industry-wide process to openly review, identify and mitigate hazards, and share best practices that ultimately improve patient safety. A healthcare version of the nuclear power program could supplement regulatory and other strategies currently used to improve quality and patient safety.

Patients continue to experience significant preventable harm. Some common approaches to improve quality include empirically measuring it and pressuring clinicians to improve performance, implementing quality improvement efforts, conducting internal physician peer review and requiring regulatory accreditation.¹ The incentives for these approaches generally separate into those driven by external forces, largely from regulators, and those motivated by internal forces, largely professional norms within a professional society or provider organisation.

Quality and patient safety efforts encounter the same issues as clinical medicine. There may be strong evidence that a treatment works (eg, preventing venous thromboembolism) and a recommended process of care (eg, assessment and prophylaxis), but we need effective strategies to persuade providers to use them. There are several externally driven strategies that promote uptake, such as regulated safety standards (eg, The Joint Commission mandate for medication reconciliation) and accreditation, economic incentives (eg, pay for performance programs) and the regulatory requirement that hospitals perform peer review to evaluate a physician's competence and performance.

Such regulatory approaches are needed and beneficial in establishing minimum standards and accountability in healthcare. Yet, they also have limitations, and are unlikely to create a healthcare system that optimises outcomes and continuously improves; indeed, patient harm continues seemingly unabated. For example, hospital peer reviews are limited by a lack of valid measurement tools and too few reviewers with the requisite ability to provide completely objective ratings.² Moreover, reviews usually focus on physician performance, failing to assess the systems in which care is delivered. Some professional societies are changing physician peer review through audit and feedback, simulation, academic detailing and multisource feedback.³

To date, regulation has driven most efforts to improve the uptake of patient safety interventions, although internally motivated efforts are growing rapidly. Local clinician-led efforts that work through communities and change social norms are extremely effective, yet the most underdeveloped in healthcare.⁴ Communities of practice⁵ and quality improvement clinical communities⁶ are examples of local efforts that build relationships, network, learn and share, and have resulted in successful improvement efforts. Quality improvement collaboratives attempt to corral clinicians, provide a common goal and interventions to reach this goal, and network these communities to support each other in achieving this goal. Examples include learning collaboratives (eg, Institute for Healthcare Improvement Breakthrough work⁷) and quality improvement clinical communities, in which interventions are created with, rather than over, clinicians (eg, the Michigan Keystone ICU Project and the national On the CUSP: Stop BSI program to reduce bloodstream infections,^{8 9} and the Vermont Oxford Network to improve care for newborn infants¹⁰).

Another type of internally driven improvement method is organisational peer-to-peer review, in which one provider organisation (without formal regulatory authority) evaluates another organisation, offering an authentic and objective assessment of opportunities to improve. Unlike physician peer review, an organisational review focuses largely on systems. There are several outstanding examples of this approach. The Northern New England Cardiovascular Study used organisational peer-to-peer review to improve the care of cardiac surgery patients¹¹ and the National Health Service in the UK used it to improve the care of patients with lung disease.¹² Beyond these examples, formal organisational peer-to-peer reviews are relatively rare in healthcare.

Informal reviews are frequent but sporadic occurrences in healthcare. Healthcare organisations commonly invite outside consultants to informally evaluate a clinical program or individual clinician. Such assessments, however, are usually ad hoc, seldom use validated evaluation tools, and infrequently disseminate learning on a broad scale. Professional societies have developed guidelines, self-assessment tools and performance measures, but they rarely conduct formal assessments of healthcare organisations.

While a mixture of regulatory and internally motivated improvement efforts are needed to ensure high quality care, the majority of improvement efforts are driven by external forces. Such forces will likely ensure adherence to minimum standards, but they will not promote innovation, optimise care or continuously improve. Internally motivated efforts have promoted innovation to increase the routine practice of recommended care and had significant success in reducing infection rates and mortality.^13–15 A more formal internal effort is needed to evaluate system issues and improve safety, and the nuclear power industry may provide a model.

The nuclear power industry has a compelling history of improving and sustaining safety from which healthcare could learn. While this industry is well known for strict regulatory bodies, they also have a mature internally driven peer-to-peer assessment program that promotes the sharing of best practices among power plants and personnel worldwide. They have used this program, which fosters information exchange and cooperative organisational learning, coupled with regulatory and other approaches, to achieve substantial and sustainable safety improvements. This commentary explores how the nuclear power industry's learning-based peer-to-peer assessment approach can be expanded in healthcare.

Peer-to-peer assessment in the nuclear power industry

After the nuclear power facility accident at Three Mile Island, the nuclear industry, largely chief executive officers (CEOs) of the nuclear companies, formed the US-based Institute of Nuclear Power Operators (INPO) that eventually transitioned into the World Association of Nuclear Operators. How INPO helped to improve nuclear safety is elegantly told in the book Hostages of Each Other.¹⁶ The author states, “what truly distinguishes INPO from many other forms of regulatory ordering… is INPO's role in promoting… a distinctive kind of community in the nuclear power industry. This movement toward community, I also found, has led to the creation of a responsibility-centered industrial culture, a distinctive set of unifying principles and practices that spells out what conduct is virtuous (professional versus unprofessional, for instance) and what goals are legitimate and desirable. I call this form of regulatory ordering “communitarian regulation”. To generalize, a well-developed system of communitarian regulation has a well-defined industrial morality that is backed by enough communal pressure to institutionalize responsibility among its members”.¹⁶ While INPO lacked formal external regulatory control, it created a set of industry-wide norms and generated strong normative pressures to improve, establishing an industrial morality. Such pressures were largely responsible for the success of the Michigan Keystone ICU Project,⁶ and the subsequent spread of this initiative across the USA, and to the UK and Spain. Clinicians' perceptions of accountability shifted from one patient where infections were deemed inevitable to a unified sense of safety for a larger population of patients in whom complications were deemed preventable and clinicians felt they were capable of reducing harm.⁶

INPO established the peer-to-peer assessment program to cross-share best practices, safety hazards, problems and actions that improved safety and operational performance.¹⁷ This program is active today and involves a robust, in-depth, objective evaluation of plant operations by an independent, international team of peers with extensive expertise and technical skills to identify risks and best practices. Nuclear power plant employees participate in the program by developing evaluation tools, conducting the peer reviews and producing safety scores. Importantly, INPO is internally motivated and lacks regulatory authority, fostering open discussions between the review team and plant managers. At a nuclear power plant's voluntary request, an INPO-assembled team uses validated instruments to observe and evaluate plant activities and conditions, conduct interviews and review performance, identifying strengths that could benefit other plants and weaknesses in plant safety and reliability that need improvement. Confidential reports describing the team's findings and recommendations are returned to plant managers. INPO-coordinated peer-to-peer assessments have increased and, combined with external validation and reporting, have led to measurable improvements in safety.¹⁷

Important attributes of an organisational peer-to-peer assessment model

Healthcare could benefit from building upon successful and internally motivated peer-to-peer programs, thereby creating a structured, clinician-led, industry-wide process to openly review, identify and mitigate hazards, and share best practices that ultimately improve patient safety.^{6 18} A healthcare version of the INPO program could supplement the current approaches to improving safety, including efforts by regulators, and provide constructive and trusted feedback, allowing providers to assess and improve their safety, helping to unify the industrial morality and, if coupled with the appropriate tools, evaluate patient outcomes and individual clinician performance.³ If healthcare is to implement the methods of INPO, a model will be needed to support this initiative.

The new organisational peer-to-peer assessment model should have the following:

1. Systems-based focus: A robust review of systems could identify the underlying factors that contribute to errors, and help develop ways to mitigate safety hazards.

2. Horizontal learning: All parties mutually benefit from the review by learning from each other and cross-sharing best practices.

3. Voluntary participation: Voluntary participation will help ensure the unit or organisation is motivated and engaged in assessments.

4. Non-punitive approach: A non-punitive and confidential review will facilitate the open sharing of information and transparency without fear of reprimand, sanction, personal disparagement or financial risk.

5. Multidisciplinary external peer reviewers: Multidisciplinary review teams comprised of peers and technical experts from an outside organisation are essential for independent, objective, valid and unbiased peer reviews. This team would include clinicians, human factors and systems engineers, psychologists, sociologists, informatics personnel, health services researchers and biostatisticians.

This model should likely be managed within one organisation.

An organisational peer-to-peer assessment process in healthcare

To create a robust and effective organisational peer-to-peer assessment process, healthcare needs to: (1) establish or identify organisations, led by clinicians and supported by administrators and technical experts, to coordinate and oversee an independent, confidential and external peer-to-peer assessment process; (2) develop and validate tools and a reliable process; (3) establish a training model and train peer evaluators; and (4) create a sustainable financial model.

Healthcare could create a not-for-profit entity like INPO to coordinate and manage organisational peer-to-peer reviews. This entity could draw clinical and technical experts (eg, human factors engineers and cognitive psychologists) from professional societies, healthcare organisations and universities to work with clinicians to identify potential hazards. Quality improvement organisations could potentially help fill this role, although they would likely need stronger clinician and technical input. To establish financial stability, hospitals could pay for a review or use a barter system, whereby hospitals would receive reviews in exchange for having physicians who work at these hospitals carry out reviews at other sites. If this new assessment process is linked to current efforts within hospitals to improve safety and conduct peer review, the burden of these efforts could be reduced. These are formidable tasks. Nonetheless, the harm that patients continue to experience in healthcare with the associated costs, and the prior success of community-based approaches should motivate the industry to explore innovative methods. Organisational peer-to-peer review could supplement existing quality improvement approaches and significantly improve patient safety. Healthcare should explore its use.

Despite the potential for improved care with organisational peer-to-peer review, healthcare differs from the nuclear industry in one important aspect: the nuclear industry has a strong industrial morality. In creating INPO, Lelan Sillin, chairman of the INPO board of directors and utility CEO stated to fellow CEOs: “In establishing INPO, the nuclear utility industry took the unprecedented step of embracing the concept of self- improvement and self-regulation. In doing so, the industry assumed a major responsibility…. We adopted a philosophy by which all of the nuclear facilities would operate, and we committed ourselves individually and collectively to achieve a standard of excellence in the conduct of our nuclear power responsibilities”.¹⁶ In reality, INPO created this industrial morality. Before INPO, the commission investigating the Three Mile Island event concluded that nuclear facilities were “so consumed by the enormous task of complying with the “voluminous and complex” maze of NRC [Nuclear Regulatory Commission] rules that satisfying regulatory requirements—going by the book—was equated with safety”.¹⁶

The parallels between the pre-INPO and current healthcare safety cultures are hauntingly similar. Efforts to improve quality and safety in healthcare have also been driven to satisfy a growing number of regulatory requirements, equating safety with meeting these requirements. Consider the Universal Protocol to reduce surgical never events. The number of wrong-site or wrong-patient procedures remains high despite hospitals reporting broad implementation of this mandatory requirement.¹⁹ Hospitals with high rates of publically reported infections or other shortcomings receive little pressure from peers to improve. If healthcare is to improve, it will need to strengthen its industrial morality. A healthcare peer-to-peer organisation, modelled after INPO, may help.