Creation of the Prevention of Organ Failure Checklist. A Multidisciplinary Approach Using the Modified Delphi Technique

Abstract

Rationale: Respiratory failure represents a major risk for morbidity and mortality. Although generally managed in the intensive care unit (ICU), respiratory failure often begins elsewhere. Checklists of care processes to minimize the duration of mechanical ventilation and adverse events are routinely used in the ICU, but are uncommonly used outside the ICU.

Objectives: To develop consensus among a multidisciplinary expert panel on care practices to include in a checklist of best practices for critically ill patients with respiratory failure before and after ICU admission.

Methods: A multidisciplinary expert panel was assembled. The panel was tasked with creating a checklist of care processes aimed at decreasing progression to respiratory failure, duration of mechanical ventilation, mortality in mechanical ventilation, and adverse events. Over the course of multiple teleconferences and e-mail communications, the Prevention of Organ Failure Checklist list was reviewed, refined, and voted upon. Items that received greater than 75% of the vote were included in the final checklist.

Measurements and Main Results: Using a modified Delphi process, the expert panel was able to compile Prevention of Organ Failure Checklist into 20 items that aimed to decrease mechanical ventilation by assessing the causes of acute respiratory failure, ventilation strategies, sedation, and general critical care processes, as well as to avoid unwanted or nonbeneficial interventions.

Conclusions: The modified Delphi process identified readily available preventative interventions suitable for checklist implementation in patients with or progressing to respiratory failure even before ICU admission.

The expansion of critical care has led to great advances in modern medicine. However, respiratory failure and mechanical ventilation persist as markers of high likelihood for morbidity and mortality. A review of nearly 6.5 million patient discharges showed that, although the incidence of mechanical ventilation was less than 3%, the mortality was 34.5%, and that less than half of the patients who were mechanically ventilated were discharged home (1).

Much effort has been made in standardizing care to prevent adverse events and reduce duration of mechanical ventilation. The Institute for Healthcare Improvement created ventilator bundles to assist physicians in treating mechanically ventilated patients and in decreasing complications, such as aspiration pneumonitis, ventilator-associated pneumonia, deep-vein thrombosis (DVT), and gastric stress ulcers (2).

Although the effectiveness of checklists depends on their implementation and the local environment in which they are implemented, the use of checklists and protocols has become commonplace to promote the timely, consistent delivery of high-quality care (3–5). Checklists help minimize complications of respiratory failure, improve mechanical ventilator management, decrease adverse events, and improve communication (6–8). Although checklists have been established in critical care, use outside and before admission to the intensive care unit (ICU) is variable (9), especially given that patients often develop respiratory failure and critical illness in other areas of the hospital. From 2001 to 2009, there was a reported 79% increase in ICU admissions from the emergency department (ED) (10), and 14% of adult patients hospitalized outside of the ICU had severe sepsis, with three-quarters of these patients having multiorgan failure (11).

As such, patients in the ED and on the wards represent a population that would be greatly served by systematic clinical response and treatment to impending respiratory distress (11). Rather than determining care for a patient based upon their location, the use of a checklist based upon the critical illness of a patient would be more patient centric, context appropriate, and reflective of the spectrum of respiratory failure.

A multidisciplinary expert panel was assembled to evaluate evidence and come to consensus agreement on what care practices are important to implement for patients that have developed or are progressing toward respiratory failure. The proposed deliverable, called the Prevention of Organ Failure Checklist (PROOFCheck), represents a list of expert recommendations that can be used in standardizing evidence-based supportive care for patients with or at risk for respiratory failure anywhere in the hospital.

Methods

PROOFCheck was developed based on the Checklist for Lung Injury Prevention (CLIP) (12–14). Evidence-based recommendations for treating patients with acute critical illness or on mechanical ventilation were compiled based on review of current guidelines set forth by critical care societies. In addition, a database search of critical care and emergency medicine literature on PubMed and the Agency for Healthcare Research and Quality National Guideline Clearinghouse was performed using the terms “acute respiratory failure” and “guidelines” (see Table E1 in the online supplement). The results were reviewed by the authors to determine their applicability to PROOFCheck. Once complete, the new criteria were added to CLIP with the goal of preventing adverse events in critical illness or reducing the duration of mechanical ventilation.

A total of 13 physicians participated in an expert panel for the modified Delphi process (15) of developing PROOFCheck, which took place over a series of teleconferences and e-mail discussions. These physicians represented the fields of critical care, emergency medicine, anesthesia, and surgery. These physicians also represented experts in the fields of chronic critical illness, quality and safety, and clinical decision support. The members of the panel represented urban and rural academic hospitals from across the United States, as well as professional clinical and quality improvement societies that span all practice types.

The panel was charged with coming to a consensus about which care practices to include in PROOFCheck. They were instructed to vote for care processes that fulfill one of the following goals in patients with progressive respiratory failure:

• 1.Prevent worsening of respiratory failure or lung injury.
• 2.Decrease duration of mechanical ventilation and/or decrease mortality in patients on mechanical ventilation.
• 3.Decrease preventable hospital-acquired adverse events and complications that can prolong mechanical ventilation or increase organ failure or mortality.
• 4.Decrease unwanted and nonbeneficial interventions by encouraging early discussion about goals of care.

The first stage of the modified Delphi process involved review of the initially proposed recommendations in an open discussion to add any recommendations the panel thought were missing from the initial draft of PROOFCheck. Panelists were encouraged to provide their thoughts and ideas, and no time limit was set for discussion so that all opinions and feedback could be equally voiced. The second stage was reiterative voting of each item until there was consensus on what to keep and what to exclude. To guide the panelists in their decision making for each item, they used the American Heart Association Guidelines for Making Recommendations for Practice Guidelines (16). These guidelines were chosen due to their ease of use as well as their reliance on expert opinion for risk versus benefit, as opposed to other guideline-creation protocols that focus on clinical trials, as many of the accepted practices among critically ill patients have not been and are unlikely to be studied in clinical trials involving patients outside the ICU.

For the first round of voting, the panel voted for inclusion into the checklist of any care practice in which the potential benefit is equal to or greater than the risk (class I, IIa, or IIb in the American Heart Association guidelines). In the second round, the panel voted to further restrict the checklist to practices considered to have more benefit than risk and that should be done or are reasonable to do in usual clinical practice (class I or IIa). Items remained in PROOFCheck if they achieved greater than 75% of panelist agreement on inclusion. See Figure 1 for the flowsheet of the drafting process. The website, PollEverywhere (https://pollev.com) (17), was used to ensure anonymous voting in real time.

Figure 1.

Flowchart of the modified Delphi process for creating the Prevention of Organ Failure Checklist (PROOFCheck). CLIP = Checklist for Lung Injury Prevention (12–14).

This project was approved by the Institutional Review Board of the Albert Einstein College of Medicine (The Bronx, NY).

Results

The PROOFCheck was created by a multidisciplinary panel of experts from across a wide range of specialties. The checklist was designed to suggest common clinical practices that may prevent the progression of respiratory insufficiency to severe acute respiratory failure requiring prolonged mechanical ventilation, as well as to decrease mortality and adverse complications and organ failure in patients at high risk for prolonged mechanical ventilation.

In the first stage of the modified Delphi process, 25 items were proposed as the initial recommendations for PROOFCheck. Upon further discussion, eight new items were added. After the final stages of discussion and voting, nine items were specifically excluded from PROOFCheck by not receiving over 75% consensus. The remaining items were condensed into the final checklist (Table 1). See Tables E2 and E3 to review panel consensus and excluded items.

Table 1.

The Prevention of Organ Failure Checklist (PROOFCheck)

Checklist Item		Recommendation	Action
1. Does the patient have a potential infection?	Yes	Start antibiotics; draw blood and fluid cultures as appropriate; check a lactate level (18).	Go to item 1a
	No	—	Go to item 2
1a. Is the patient on appropriate antibiotics?	Yes	Broaden antibiotic coverage (18).	Go to item 1b
	No	Start antibiotics.	Go to item 1b
1b. Was the patient evaluated for source control?	Yes	Obtain source control as quickly as possible, evaluate for other potential sources of infection (18).	Go to item 2
	No	—	—
2. Does the patient have signs of shock?	Yes	Start resuscitation and follow institutional guidelines for shock. If patient has potential infection, start institutional protocol for septic shock. Consider critical care consultation.	Go to item 3
	No	Consider rechecking labs and lactate, frequent vitals evaluation.	Go to item 3
3. Is the head of the bed at 30 degrees?	Yes	Elevation of the head of the bed to 30° helps avoid aspiration (25).	Go to item 4
	No	—	—
4. Does the patient need transfusion of blood products?	Yes	Unless the patient is actively bleeding, limit PRBC transfusion to Hg <7.0 g/dl. Limit platelet and plasma transfusion unless patient has active bleeding or severe thrombocytopenia <10,000 platelets/ul (28, 29).	Go to item 5
	No	—	—
5. Does the patient have or need a urinary catheter?	Yes	Remove or exchange urinary catheter if infection suspected or as soon as it is no longer needed to prevent infection (31).	Go to item 6
	No	—	—
6. Does the patient have or need a central venous catheter?	Yes	Remove central venous catheter as soon as it is no longer needed to prevent infection, or insert new catheter if still needed, but there is concern for infection (18, 30).	Go to item 7
	No	—
7. Was the patient assessed for early mobilization and physical therapy?	Yes	Early mobilization and physical therapy, if not contraindicated, can decrease delirium and duration of mechanical ventilation (32, 33).	Go to item 8
	No	—
8. Was there a discussion about goals of care in last 24 h?	Yes	Early discussion of goals of care is recommended to inform patients and families of the patient condition and gauge the desired level of life-sustaining interventions and therapeutic goals of the patient (34, 38).	Go to item 9
	No	—
9. Does the patient meet criteria for a palliative care consult?	Yes	Indications for palliative care consult: severe or terminal condition in which death within 6 mo is not surprising, difficult to control physical or psychological symptoms, unclear goals of care, disagreement between family and staff on preferences for resuscitation and life support (34–38).	Go to item 9a
	No	—
9a. Were palliative care services offered, if there are unmet palliative care needs?	Yes	—	Go to item 10
	No	—
10. Is the patient on invasive mechanical ventilation?	Yes	—	Go to item 10a
	No	—	Go to item 11
10a. Was low volume ventilation started?	Yes	Consider low tidal volume ventilation if patient at risk for ARDS (23, 24).	Go to item 10b
	No	—	—
10b. Was oral hygiene care initiated?	Yes	Improved oral hygiene decreases ventilator associated pneumonia (2).	Go to item 10c
	No	—	—
10c. Was sedation assessed and titrated to RASS of -1 to 0?	Yes	Minimize or stop sedation. Consider bolus as opposed to continuous sedation to target RASS −1 to 0. Avoid benzodiazepines unless specifically indicated (26).	Go to item 10d
	No	—	—
10d. Was sedation vacation done today?	Yes	Daily sedation breaks, when no contraindication is present, decreases duration of mechanical ventilation (27).	Go to item 10e
	No	—	—
10e. Is patient on DVT prophylaxis?	Yes	DVT prophylaxis is part of the recommended ventilator bundle and decreases incident of DVT and pulmonary embolism (2).	Go to item 10f
	No	—	—
10f. Is patient on GI prophylaxis?	Yes	Stress ulcer prophylaxis is part of the ventilator bundle and prevents hospital-acquired gastric ulcers (2).	—
	No	—	—
11. Does patient have acute hypercapnia?	Yes	Evaluate patient’s oxygen saturation while on oxygen therapy. High levels of oxygenation can worsen hypercapnia in patients with acute hypercapnic respiratory failure (19).	Go to item 12
	No	—	—
12. Is patient on noninvasive mechanical ventilation?	Yes	Re-evaluate patient within 30 minutes after trial of noninvasive oxygen supplementation. If minimal or no benefit from noninvasive ventilation, consider intubation or critical care/rapid response consultation. Evaluate for changes in mental status, PaCO2, work of breathing. Noninvasive ventilation has proven beneficial for patients with COPD and CHF exacerbation, but not for patients with pneumonia or ARDS (20–22).	—
	No	—	—

Definition of abbreviations: ARDS = acute respiratory distress syndrome; CHF = congestive heart failure; COPD = chronic obstructive pulmonary disease; DVT = deep-vein thrombosis; GI = gastrointestinal; PRBC = packed red blood cell; RASS = Richmond Agitation–Sedation Scale.

Discussion

PROOFCheck incorporates many well tested concepts from critical care literature for treating patients with acute respiratory failure. Each recommendation in PROOFCheck fit into at least one of its four domains: prevention of progression of respiratory failure or lung injury; decreasing preventable hospital-acquired adverse events and complications that can prolong mechanical ventilation or increase organ failure or mortality; preventing common hospital-acquired adverse events; and avoiding unwanted or nonbeneficial interventions.

Recognition of conditions that can lead to respiratory failure and lung injury is critical in preventing prolonged mechanical ventilation and its consequences. Although the systemic inflammatory response syndrome is controversial in terms of predicted outcomes, it is still useful for recognition of potential sepsis. PROOFCheck items 1, 1a, 1b, and 2 focus on recognition of sepsis and recommend the initiation of sepsis bundles (18), as determined by individual hospital policy.

For patients with respiratory failure, the panel identified care practices that could prevent progression to prolonged invasive mechanical ventilation (items 11 and12). For patients not on mechanical ventilation with acute hypercapnia on supplemental oxygen, the panel recommended a conservative oxygen saturation goal of 88–93% to avoid worsening of hypercapnia in acute hypercapnic respiratory failure (19). The panel agreed with the recommendation of noninvasive ventilation for patients with chronic obstructive pulmonary disease or congestive heart failure (20). However, as there is no consensus for benefit of noninvasive ventilation for hypoxemic respiratory failure, such as acute respiratory distress syndrome (ARDS) (21, 22), the panel recommended early reassessment of the benefit of noninvasive ventilation in acute hypoxemic and hypercapnic respiratory failure.

For patients that progress to respiratory failure and require intubation, the panel recommended low tidal volume ventilation if there is suspicion of ARDS to limit the risk of lung injury, as the benefit outweighs the risk in the early period of acute respiratory failure (23) (item 10a). Low tidal volume ventilation has been shown to reduce ventilator-associated lung injury and decrease hospital length of stay for patients with and without ARDS (24). Furthermore, the panel felt that there should be efforts to prevent common complications of respiratory failure, such as aspiration and ventilator-associated pneumonia (items 3 and 10b) (25).

The panel recommended avoidance of deep sedation, in accordance with existing clinical standards (items 10c and 10d) (26). The panel also recommended early consideration for sedation vacation, which, in combination with ventilator weaning, may lead to improved ventilator outcomes and possible earlier liberation from the ventilator (27).

The panel made several recommendations for preventing adverse hospital-acquired events, including: restrictive transfusion protocols (item 4) (28, 29); evaluating the continued need for invasive catheters versus the risk of infection (items 5 and 6) (18, 30, 31); assessment for mobilization (item 7) as early as possible (32, 33); and that all mechanically ventilated patients be assessed for DVT, and stress ulcer prophylaxis (items 10e and 10f) (2).

The panel recommended evaluating patients for terminal conditions and unmet palliative care needs, and to offer palliative or comfort care services, if appropriate, after discussion with the patient and family (items 8, 9, and 9a) (34). Early discussion between clinicians and patients or their families can help direct subsequent clinical decision making and improve patient-centered care (35–37). Unrecognized and unmet palliative care needs have been identified as a crucial and common deficit by national palliative care professional societies (38).

Among the panelists, concern was voiced that having too many items may lead to poor compliance. However, the panel determined that, because most physicians outside the ICU likely have few patients for whom the checklist might apply at one time, the length would not be severely limiting or time consuming, and that high compliance levels with the protocol could be attained.

PROOFCheck represents a list of expert recommendations that can be used anywhere in the hospital for treatment of patients with respiratory failure. However, it does have limits. Its usefulness and effectiveness will depend largely on appropriate implementation of the protocol, and identification of patients who are at high risk for respiratory failure will be key. There are currently different strategies used clinically and in research settings to identify such patients. The Lung Injury Prediction Score has been validated to identify patients at high risk for progression to ARDS, and could be used (39). Other early warning scores, like the Cardiac Arrest Risk Triage Score, have been used to identify patients at risk for deterioration, including respiratory failure (40). In conjunction with PROOFCheck, we have developed and validated an electronic medical record–based clinical decision support tool to identify patients who are likely to require mechanical ventilation within 48 hours. We intend to test, in a pragmatic clinical trial, whether early notification of clinicians of patients at risk for prolonged mechanical ventilation, coupled with a checklist of best practices, could improve outcomes in patients with acute respiratory failure (NCT02488174; www.clinicaltrials.gov).

Recognition of critical illnesses, such as respiratory failure, remains an essential step in treating all patients. Early recognition, whether in the ED, on the wards, or in the ICU, remains the rate-limiting step in treatment of respiratory failure. Detection, early progressive treatment, and implementation of protocols, such as PROOFCheck, may help prevent prolonged mechanical ventilation and improve outcomes for those patients at high risk for respiratory failure, wherever they may be in the hospital.