Perceptions of rounding checklists in the intensive care unit: a qualitative study

Abstract

Background

Rounding checklists are an increasingly common quality improvement tool in the intensive care unit (ICU). However, effectiveness studies have shown conflicting results. We sought to understand ICU providers’ perceptions of checklists, as well as barriers and facilitators to effective utilisation of checklists during daily rounds.

Objectives

To understand how ICU providers perceive rounding checklists and develop a framework for more effective rounding checklist implementation.

Methods

We performed a qualitative study in 32 ICUs within 14 hospitals in a large integrated health system in the USA. We used two complementary data collection methods: direct observation of daily rounds and semistructured interviews with ICU clinicians. Observations and interviews were thematically coded and primary themes were identified using a combined inductive and deductive approach.

Results

We conducted 89 interviews and performed 114 hours of observation. Among study ICUs, 12 used checklists and 20 did not. Participants described the purpose of rounding checklists as a daily reminder for evidence-based practices, a tool for increasing shared understanding of patient care across care providers and a way to increase the efficiency of rounds. Checklists were perceived as not helpful when viewed as overstandardising care and when they are not relevant to a particular ICU’s needs. Strategies to improve checklist implementation include attention to the brevity and relevance of the checklist to the particular ICU, consistent use over time, and integration with daily work flow.

Conclusion

Our results provide potential insights about why ICU rounding checklists frequently fail to improve outcomes and offer a framework for effective checklist implementation through greater feedback and accountability.

INTRODUCTION

Intensive care unit (ICU) providers do not consistently implement evidence-based practice, with many patients not receiving care practices proven to save lives.¹ One approach to address this problem is through the use of ICU rounding checklists. A rounding checklist can be defined as a tool used to remind clinicians to address specific topics related to healthcare quality or efficiency while on daily rounds.² In industries like manufacturing and aviation, checklists are known to increase safety and prevent errors.³ In the ICU checklists might have similar benefits, enabling delivery of evidence-based practice by prompting consistent discussion of beneficial treatments by the interprofessional care team during rounds.⁴

Despite a strong conceptual rationale, the evidence demonstrating that checklists actually improve clinical outcomes is limited.^4–8 Some studies suggest that checklists improve the use of evidence-based practice and aid in error avoidance.^4–6 However, other studies show persistent quality gaps despite the use of rounding checklists,⁷ and a recent randomised controlled trial showed no association between checklist implementation and in-hospital mortality for ICU patients.⁸ These data suggest that checklists do not reliably improve patient outcomes, despite being viewed as a powerful quality improvement tool. To better understand why checklists are not achieving their intended result, we conducted a qualitative study to investigate ICU providers’ knowledge, attitudes and practices regarding checklist use. Our goal was to understand how ICU providers perceive rounding checklists and develop a framework for more effective rounding checklist implementation.

METHODS

Study design

We conducted a qualitative study using direct observation and semistructured interviews with care providers within 32 ICUs in 14 hospitals in a large US integrated health system during 2016.⁹ Each hospital with at least one adult ICU was included. Because we were interested in how ICUs implement evidence-based practice even without checklists, we did not limit the study to only ICUs that were currently using checklists but instead studied all adult ICUs in the health system. Unit directors were contacted via email to assess willingness to participate, determine baseline ICU characteristics including checklist use, the approximate date when the checklist was implemented, and the frequency and structure of rounds. We defined a rounding checklist as ‘a tool used to remind clinicians to address specific topics related to health care quality or efficiency while on daily rounds’. After collecting baseline information, we obtained copies of available rounding checklists via email correspondence with unit directors and scheduled visits to the ICUs for direct observation.

Data collection

Checklist review

To provide context for our observations and interviews, we reviewed all in-use checklists for structure and content, including the number of items, the wording of items (statements vs questions), the style of use (single-use or reusable), the presence or absence of instructions, and the presence and absence of key content domains. Two investigators developed and tested an abstraction form for inter-rater reliability and all disagreements were resolved through iterative discussion.

Direct observations

The purpose of the observations was to observe rounding checklist use in action, as well as observe, in ICUs that did not use a formal rounding checklist, how evidence-based practices were addressed on rounds. A total of four investigators performed the observations, although no more than a single observer was present per rounding team at one time. All ICUs were observed once unless we were alerted to unit-specific circumstances that may have made that day’s observations atypical. In these cases, we conducted observations a second time on a different day. Participants were told that the study was to observe rounds and learn more about rounding checklists but were given no other information. All observations were performed by a researcher trained in qualitative methods. The research team has experience in the ICU setting, having completed a number of qualitative research projects in intensive care.^10–12 The researchers took notes by hand during direct observations and transcribed notes to an electronic document following each observation period.

Semistructured interviews

Interviews with care providers were performed directly after each rounds. The purpose of the interviews was to provide deeper insight into the use of checklists, as well as to triangulate data from the direct observations.⁹ We targeted four types of providers: nurses, physicians, respiratory therapists and pharmacists. We targeted these clinicians because they are typically present during interprofessional rounds and were thought to have the most insight into the use of evidence-based practice. The preliminary interview guide was developed from our experience conducting ethnography in the ICU setting along with input from members of a system-wide ICU quality improvement committee. The interview guide was then augmented with a review of the literature related to checklists in the medical field. We piloted the guide on a convenience sample of five practising ICU clinicians, making revisions based on their feedback. The final interview guide contained three major domains: the purpose of daily rounds and checklists, opinions of checklists and preferred content, and either checklist use in the interviewee’s unit for ICUs with checklists or how evidence-based practice is reviewed for ICUs without a rounding checklist. All questions were framed as open-ended questions and probes were used to provide further clarification when needed. A copy of the interview guide is included as an online supplementary appendix.

Potential interviewees were identified by direct approach after each rounds. When rounding team members were not available immediately following rounds for interviews, participants were asked if they would be willing to take part in an interview at a later time or via telephone. Interviews were audio-recorded and transcribed verbatim. Interviewee demographics were collected via a written form at the time of interview.

Analysis

ICU characteristics, checklist characteristics and provider demographics were summarised using standard statistics. Direct observation notes and interview transcripts were uploaded into commercially available qualitative analysis software (NVivo, QSR International, Melbourne, Australia). A thematic codebook was developed that included both deductive and inductive constructs. Deductive constructs relate to aspects related to diffusion of innovations, including relative advantage, compatibility and complexity.¹³ Inductive constructs were developed from an iterative review of observation notes and interviews.¹⁴ Four researchers independently coded sources with all coding questions decided by group consensus. After identifying key themes and their relationships, we revisited the data to look for how these themes manifested across checklist use and non-use units, as well as perceptions of the themes as reflected in interviews and what was carried out in practice as reflected in observations.

Results are presented as a series of primary themes along with how these themes manifested across ICUs that do and do not use rounding checklists. Themes are illustrated with representative quotes and, when applicable, notes from the observations.

The overall work was judged to be quality improvement because the primary goal was to improve care within the health system by identifying strategies to optimise the use of checklists during ICU rounds.¹⁵ De-identified observations of group behaviour during rounds were deemed not to exceed normal work-related risks, and therefore observation participants were not asked to provide consent.¹⁵ Additionally, since there was no direct patient contact and no patient data involved in this project, patients or their surrogates were not required to provide consent. All interviewees provided verbal consent. Observers and interviewers were students or university-affiliated project staff working under the authority of the health system.

RESULTS

In total we studied 32 ICUs in 14 hospitals (table 1). Of the 32 ICUs in the study, 17 reported that they had a form of a rounding checklist. However, on observation in each unit, only 12 ICUs actively used a checklist during morning rounds at the time of study.

Table 1.

Characteristics of participating ICUs (n=32)

Characteristics Age	n
Checklist use
Yes	12
No	20
Duration of checklist use (n=12)
<1 year	1
1 year to <5 years	8
≥5 years	3
Multidisciplinary rounds
Yes	31
No	1
Trainees present on rounds
Yes	22
No	10
Number of ICU beds
<10	7
10–20	20
≥20	5
Physician staffing model*
Open	21
Closed	10
Mixed	1

^*ICUs were categorised according to the framework developed by Pronovost and colleagues.19 Under this framework, an ‘open’ ICU is one in which any hospital physician can admit a patient to the ICU and provide critical care. A ‘closed’ ICU is one in which only intensivist physicians can admit patients to the ICU—non–intensivists must transfer their patients to the intensivists' care. ‘Mixed’ ICU used a mixture of these models, typically requiring intensivists' consult on patients but not requiring transfer of care.

ICU, intensive care unit.

Checklist characteristics from all ICUs that used a checklist (n = 12) are shown in table 2. In terms of formatting, only two checklists contained instructions on how the checklist was to be used. Ten checklists were disposable (ie, they were on paper meant to be either inserted in the medical record or thrown away), while only two were reusable (ie, they were laminated plastic cards meant to be used repeatedly across many patients). None of the checklists were automatically part of the electronic health record. Some of the single-patient paper-recorded checklists were later entered into the patient’s medical record. However, most checklists were either disposed of or compiled to audit use. The number of items on each checklist varied from 21 items to 115 items. Half of the checklists framed items as a mix of questions and statements, while the rest had either all statements or all questions.

Table 2.

Checklist characteristics (n=12)

Characteristics	n
Instructions on how to use	2
Style of use
Reusable	2
Disposable (one patient)	6
Disposable (multiple patients)	4
Number of items
21–25	5
26–35	4
36+	3
Framing of items
Questions	5
Statements	1
Mixed	6

Analysis of checklist content revealed relatively consistent content domains across the study checklists (figure 1). All checklists included items related to nursing care, prophylaxis against ICU complications and infection control, and nearly all contained items related to physical activity and mechanical ventilation. Within the nursing domain, the most common elements were nutrition and restraints, while the least common were height and weight, and fall prevention. Within the prophylaxis and infection control domains, the most common elements were deep vein thrombosis and stress ulcer prophylaxis, and urinary catheter and central venous catheter removal.

Figure 1.

Content domains of study intensive care unit rounding checklists (n=12).

We performed data collection once in 29 ICUs and twice in 3 ICUs, resulting in 114 hours of direct observation and 89 interviews with members of the rounding teams (table 3). In total, three providers declined to participate in an interview, while five additional providers agreed but did not participate in an interview (97 providers were approached for an interview, resulting in a 92% inclusion rate). Initially, our objective was to target staff including physicians, nurses, pharmacists and respiratory therapists at highest importance. Our goal was to have at least one of each of these professions from each ICU participate in an interview. As our study evolved, we found it difficult to obtain interviews from all four of these professions in each ICU as pharmacists and respiratory therapists were typically unavailable following rounds, so ultimately the majority of our participants are physicians and nurses. That said, we were able to collect interviews from pharmacists, respiratory therapists and other staff in some units. The mean interview time was 12.0 min (range: 3.4–23.1).

Table 3.

Demographic characteristics of interview participants (n=89)

Characteristics	n
Age (years)
18–29	30
30–39	23
40–49	22
50+	12
Missing	2
Female gender Profession	60
Nurse	49
Physician	26
Pharmacist	7
Respiratory therapist	3
Other	4

Thematic analysis of the observations and interviews revealed three major themes: perceptions related to the purposes of checklists, potential negative aspects of checklist use, and perceived barriers and facilitators to effective checklist use in the ICU.

Theme 1: purposes of checklists

Respondents in ICUs that used checklists identified three main purposes: to serve as a reminder to discuss important topics that might otherwise be omitted from discussion, to create a shared understanding among the care team about patients’ medical problems and goals for the day, and to increase efficiency on rounds. In serving as a reminder to discuss all important topics, checklists can standardise care and help to ensure the use of evidence-based practice. In creating a shared understanding about the patient, checklists get the team ‘on the same page’ about patients’ progress, prognosis and care plan. In improving efficiency, respondents noted that checklists can provide a predictable structure as to who presents what information at rounds. Together, these roles all relate to checklists as a tool for facilitating effective communication across team members. By reframing rounding discussions in a common language, checklists can help address problems that arise when team members bring unique communication styles or are unfamiliar with each other.

So checklists are tools for reminders; for memory aid if you will. We’re humans and we make mistakes.... I can remember 100% of the items that I need to evaluate in one patient, in two patients, but not in 100 patients. So over time, it’s easy to overlook items. And the objective of the checklist is to standardize medical care by assuring that every single patient have review of minimal standards of care. (Physician in ICU that used a checklist)

I think for us, the checklist has done a nice job of helping to standardize the way that we approach rounds...I just feel like our rounds run more smoothly now than they did in the past, because the resident knows when it’s their turn to talk. The nurse knows when it’s their turn to talk. The pharmacist knows when it’s their turn to talk. And then we wrap things all up together and run through the checklist. It feels like it makes rounds more predictable from an attending to attending basis. (Physician in ICU that used a checklist)

Basically it helps me. So I need to make sure I have an order. I need to make sure that there is a family meeting done within 5 days of admission. I need to know exactly where we’re going with code-status. It actually helps me take care of the patient; like where we’re going. (Nurse in ICU that used a checklist)

These issues were reflected in the observations, wherein units that used checklists showed checklists that facilitated fluid yet organised discussion between rounding team members.

Attending [physician] states, ‘So let’s run through orders and make sure we have everything.’ Attending reads through goals from the checklist noting orders needed for each one. Attending then runs through the checklist item by item. He states each item verbally, saying yes, no, or contraindicated. He gives reasons for contraindication. Group says ‘Yep’ after each item. Attending hands the checklist back to [the nurse]. (Observation in an ICU that used checklists)

Attending physician joins the team. Attending asks fellow if attending should write on the checklist. Fellow replies ‘I’ve got it, it helps me process’. Attending responds, ‘OK’. Resident continues discussing the patient’s goals…. Fellow writes in goal section of checklist while resident speaks. (Observation in an ICU that used checklists)

Respondents in ICUs that do not use checklists tended to describe the purpose of a rounding checklist as task-based reminders rather than holistic tools for communication. Frequently, individuals in ICUs that do not use a rounding checklist mentioned that they used an internal, or mental, checklist to help them stay on task for each patient. Mental checklists were individually oriented in that they were focused on the tasks that that individual needed to accomplish, while true rounding checklists were group-oriented in that they were focused on group-level tasks.

Just to make sure you’re getting everything done that needs done in the unit that day. You’re doing what’s expected of you. (Respiratory therapist in ICU that does not use a checklist)

The perceived value of checklists was modified by the relationship between the checklist contents and the patients in the specific ICU. Checklists were perceived as useful when all the items related to the patient under discussion, and checklists were perceived as less useful when items were not relevant to the patient under discussion.

Maybe it [the checklist] doesn’t fit to that patient as much. Because sometimes we get patients that don’t have a ventilator, they don’t have any pain…It depends on why they’re here. (Nurse in ICU that used a checklist)

Theme 2: negative aspects of checklist use

Respondents described two predominant negative aspects of rounding checklists. The first is that checklists may depersonalise care by fostering an environment of overly standardised patient care: some respondents referred to this as ‘cookbook’ or ‘cookie-cutter medicine’. This could lead to negative consequences by preventing providers from approaching each patient individually and with sufficient flexibility to provide optimal care.

…perhaps they can cause harm if people focus too much on the checklist and not enough on the actual patient and the patient assessment. And kind of if you look at everybody as a format, you’re not looking at them as an individual. (Nurse in ICU that used a checklist)

The second is a risk that checklists unnecessarily wasted time, frustrating already busy providers. This risk was particularly salient when the checklist contained items that were seen as redundant or not salient to the patient at hand.

[We] fill it out just to fill it out. It’s redundant, is meaningless, is a waste of time. (Nurse who worked in ICUs that both use and do not use checklists)

The biggest thing is time. They’re busy. It’s a busy unit. If they spend 20 min rounding on a patient and then 10 min filling out the checklist to make sure it’s done, it’s going to end up being counter-productive. (Pharmacist in ICU that does not use a checklist)

Observations afforded a nuanced understanding of how these concerns manifest in practice. For example, this observation illustrates a time when without a checklist the team discusses only elements of care seen as immediately important at the time. Yet at the same time, the conversation is unstructured and circular with little interprofessional discussion.

We move to the next bed. The nurse for this patient presents the patient, and reads from his paper: overnight issues, intravenous lines, blood pressure, oriented, pupils, symptoms and conditions, Electrocardiogram, afebrile, insulin drip, secretions, speech [therapy] up to see him, incontinent overnight so couldn’t measure urine output, skin. The physicians discuss the patient’s glucose/insulin. Physician 2 clicks through the patient’s electronic medical record as they talk. Physician 1 runs through his list: cardiovascular, ACE inhibitor, respiratory, renal, sodium, gastrointestinal (bowel regimen), infectious disease, afebrile, white count, will take antacid if he starts eating, glucose (sliding scale). I notice a brain scan on Physician 1’s monitor. I also notice that when the physicians talk after the nurse presents the patient, there isn’t a lot of discussion with other members of the rounding team. (Observation in an ICU not using a checklist)

Theme 3: barriers and facilitators to implementation

Respondents identified several key barriers and facilitators to checklist implementation. Barriers include overly long checklists, checklist items that were not relevant to the ICU’s case-mix and checklist items that did not reflect ongoing quality gaps. Facilitators were the converse: brief checklists and checklists with only salient items were viewed as easier to implement. Relatedly, implementation was supported when the checklist was viewed as a ‘living document’, able to evolve with the changing needs of an ICU.

The most important is having the appropriate checklist. If you have a checklist for 40 elements, that checklist is not going to work for you. You have to have the leanest and the most efficient checklist. Because if you have it that way, that’s going to work. The combination of people understanding and having a lean and efficient checklist, I think that’s a good combination that make checklists work. (Physician in ICU that used a checklist)

They need to be very simple. I’ve seen some that are so involved, they’re just overwhelming to look at. Maybe that works for some people. But I think that they can be really overwhelming to even look at. The biggest problem here with checklists was getting everyone to do it because I think they found it annoying. (Physician in ICU that does not use a checklist)

I think that if the checklist was not relevant and it was not changing with the needs of the unit, then it would not be helpful. In fact, it would probably be discarded unconsciously very quickly, if it stopped being relevant. So I think it’s important for a checklist to be a live document that changes and adapts to the changing unit. (Physician in ICU that used a checklist)

Another major facilitator is use that is consistent. Consistent use (ie, use that does not vary over time or between different attending physicians) creates shared normative expectations among the care team. Conversely, when a checklist is occasionally not used, team members question the checklist’s value, which in turn creates momentum towards non-use.

I have one physician who absolutely will not use it. So unless I have somebody on who’s willing to sneak around behind him and take the initiative to fill it out, it doesn’t get done when that physician is here. (Nurse in ICU that used a checklist)

I’m sure you’ve seen the bell-curve of, you’ve got your early adopters, and your middle and late adopters, but this is the kind of thing where even if you have a couple late adopters, it really drags the whole thing down. Because if it’s not routine every day; with the attendings, if you have one bad apple in the bunch, it kind of screws it all up. Because then the nurses are like, ‘well we didn’t need to do it last week and various expectations…’ (Physician in ICU that does not use a checklist)

A final facilitator is use that is integrated into the rounding process. Some ICUs were overt in their checklist use, openly discussing it. In other ICUs, individuals were seen silently completing the checklist as rounds progressed.

I can’t say that there is (a standard way the checklist is used). Sometimes it’s a physician using it, other times we will give to the Physician’s Assistant or the charge nurse. I’ve seen them all basically be responsible for kind of using it and filling it out. So I would say that we don’t necessarily have a standard. (Pharmacist in ICU that used a checklist)

The charge nurse has the paper checklist sitting on top of her computer on wheels. She listens as the residents, the bedside nurse, and physician speak and makes check marks in column 1. (Observation from an ICU that used a checklist)

The Resident reads from goals list. The Attending reads through the checklist stating each item out loud…’Did you do DVT [deep vein thrombosis] prophylaxis?’ Fellow, ‘Yes’ Attending continues, ‘antibiotic dosing repeat…’ Attending finishes the items on the checklist and returns it to the bedside nurse. (Observation from an ICU that used a checklist)

DISCUSSION

In a qualitative study of ICU rounding checklists, front-line ICU providers reported that checklists have the potential to be used as standardisation and communication tools within the ICU. Yet they also noted key negative connotations with checklists, including the idea that checklists are ‘cookbook’ medicine that lack the necessary flexibility to respond to individualised patient needs, add time to already overburdened rounds and often fail to achieve their potential due to inconsistencies in use.

Based on these results we developed a framework for effective adoption based on the key barriers and facilitators to checklist use (figure 2). Under this framework, checklist adoption is a continuous cycle of development, implementation and review. Development should focus on attention to brevity and relevance to the ICU’s specific population and evidence. Implementation should include efforts to ensure consistent and integrated use. Review is the process by which the checklist is updated as the needs of the ICU change, including rotating items in and out of the checklist as needs evolve (ie, item rotation), as well as efforts to support the buy-in of key providers.

Figure 2.

Framework for effective intensive care unit (ICU) rounding checklists.

Our results provide insight into the possible reasons why checklists do not consistently improve ICU quality.^4–8 For example, a recent negative clinical trial used the same checklist in 188 different ICUs.⁸ We show the importance of customising checklists to individual ICU settings and case-mix. Additionally, that trial operationalised the checklist in the same way in every ICU in that the checklist was read aloud by a nurse, while we find value in affording ICUs’ flexibility in how checklists are used while maintaining consistency and integration.

Most of these recommendations are important tenets of quality improvement writ large, not just specifically for checklist implementation.¹⁶ For this reason, checklists should be seen not as quality improvement per se but merely as a tool for quality improvement. Like other quality improvement tools it is important to target the intervention to known gaps, use a local champion, foster buy-in and regularly assess impact, lest the effort fail to improve quality.

Interestingly we did not find direct evidence of ‘checklist fatigue’, an idea expressed as a potential barrier to checklist use.¹⁷ The lack of evidence for checklist fatigue suggests that checklists in and of themselves are not inherently flawed as a quality improvement tool. Rather, it is possible that when implemented intelligently checklists can be a sustainable strategy for quality improvement.

Our project has several limitations. Although we obtained input from a wide variety of clinician types, the majority of our respondents were nurses and the findings may not be completely reflective of the interprofessional care team. Additionally, as in all qualitative research, there is potential for bias on the part of researchers as well as interviewee respondents and rounding team members. In addition, knowledge about the study goals may have influenced observations; however, given the concordance between our interview findings and direct observations, we doubt that this issue substantially influenced our results. We took several steps to minimise bias and increase the trustworthiness of our results, including the use of a standardised, well-piloted interview guide, and use of multiple methods of data collection, which enabled triangulation of our key findings.¹⁸

CONCLUSIONS

Rounding checklists are intended to align providers’ understanding of the goals of patient care and act as a reminder for the use of evidence-based practice. Our results suggest that effective adoption of rounding checklists requires addressing important concerns about their perceived value on the part of care providers. Potential strategies to ensure effective checklist implementation include ensuring that checklist content is relevant to the rounding team and working to ensure use that is consistent and integrated with patient care. The implementation framework we developed is a useful starting point in order to maximise the chance that checklists improve clinical outcomes in the ICU.