Characterizing intensive care unit rounding teams using meta-data from the electronic health record

Abstract

Purpose:

Teamwork is an important determinant of outcomes in the intensive care unit (ICU), yet the nature of individual ICU teams remains poorly understood. We examined whether meta-data in the form of digital signatures in the electronic health record (EHR) could be used to identify and characterize ICU teams.

Methods:

We analyzed EHR data from 27 ICUs over one year. We linked intensivist physicians, nurses, and respiratory therapists to individual patients based on selected EHR meta-data. We then characterized ICU teams by their members’ overall past experience and shared past experience; and used network analysis to characterize ICUs by their network’s density and centralization.

Results:

We identified 2327 unique providers and 30,892 unique care teams. Teams varied based on their average team member experience (median and total range: 262.2 shifts, 9.0–706.3) and average shared experience (median and total range: 13.2 shared shifts, 1.0–99.3). ICUs varied based on their network’s density (median and total range: 0.12, 0.07–0.23), degree centralization (0.50, 0.35–0.65) and closeness centralization (0.45, 0.11–0.60). In a regression analysis, this variation was only partially explained by readily observable ICU characteristics.

Conclusions:

EHR meta-data can assist in the characterization of ICU teams, potentially providing novel insight into strategies to measure and improve team function in critical care.

1. Introduction

Effective critical care delivery is increasingly dependent on interprofessional teams composed of physicians, nurses, respiratory therapists, and other health care professionals [1]. For example, a large number of evidence-based practices in the intensive care unit (ICU) rely on the coordinated activities of multiple different provider types, including lung protective ventilation, daily interruption of continuous sedation and daily spontaneous breathing trials, among others [2–4]. Yet a substantial body of literature indicates that these team-based practices are incompletely adopted—many eligible patients don’t receive these evidence-based practices, leading to preventable morbidity and mortality [5–7]. Research is urgently needed to close this gap [8].

Given the essential link between evidence-based practice and interprofessional care, a greater understanding of the nature of interprofessional care team may yield important insight into strategies to close the evidence gap. Emerging data demonstrate that team characteristics such as shared experience can influence team performance over and above individual characteristics [9,10]. These data suggest that intelligent team design (i.e. intentionally pairing team-members to build shared experience) could increase their ability to implement team-based evidence-based practices, more rapidly bringing effective new treatments to the bedside [12]. At present ICU teams are generally formed through ad hoc scheduling processes rather than through concerted efforts to pair team members together based on their individual competencies or shared experience [13]. Shifting that paradigm towards more intelligent scheduling might be a novel strategy for improving patient outcomes.

Yet despite the apparent value of interprofessional care, relatively little is known about the characteristics of high-performing ICU teams, and few studies have identified effective strategies to improve ICU team function [14–16]. One reason for this deficiency is that ICU teams are difficult to characterize. ICU teams possess low temporal stability, in that team membership changes from day to day as providers rotate in and out of the ICU [17]. ICU teams also change within the course of the day as providers enter and leave rounds depending on which patient is under discussion. This problem makes it difficult to identify the individual members of the ICU team and therefore study them at a large scale. To work around this problem, studies of ICU teamwork often conceptualize the team as all staff members that ever work in the ICU over the course of a year [18,19], which does not accurately reflect the dynamic nature of ICU rounding teams. Alternatively, some studies use prospective methods such as surveys or direct observation to study individual ICU teams [20,21]. However, these methods are not feasible at scale [22].

The rise of the electronic health record (EHR) offers a potential solution to this problem. ICU providers use the EHR to place orders, perform routine charting, and write progress notes [23]. In doing so, they leave electronic signatures in the form of meta-data in the charts of individual patients. These meta-data contain unique identifiers for each provider as well as time stamps indicating that the provider was involved in a patient’s care at a given time. In theory, these data can be used to attribute specific providers to specific patients on specific days, creating a comprehensive picture of the interprofessional care team [24].

In this study we sought to determine the potential value of EHR meta-data for identifying and characterizing ICU teams. First, we applied validated algorithms to EHR records from 27 ICUs in a large multi-hospital health system to link ICU providers to specific patients on specific days. Next, we used those linkages to define ICU rounding teams at the level of the patient-day. Finally, we described the characteristics of those ICU teams through two different lenses: the experience of rounding team members and the networks formed by individual interactions within the team.

2. Methods

2.1. Study design and data

We performed a retrospective analysis of EHR data from UPMC, a multi-hospital health system in Western Pennsylvania, New York, and Maryland. All hospitals in UPMC share a single EHR (Cerner PowerChart, Cerner Corporation, Kansas City) and contribute patient data to an ICU registry used for system-wide performance measurement and quality improvement. Data elements in the ICU registry include unique identifiers for each patient and hospital admission, demographic variables, ICU and hospital admission source, ICU and hospital discharge destination, vital signs, laboratory values, medication administration records, microbiology records, and respiratory care assessments including ventilator settings. This registry has been used in multiple studies examining ICU patient outcomes, and complete details on the registry contents are available in those reports [25–27]. For this study we limited the analysis to 27 ICUs across 10 hospitals in which an intensivist physician rounded daily on all ICU patients, based on an annual organizational survey.

We used data from 09/01/2019–08/31/2020, the most recent year of data available at the time of this analysis. This date range overlaps with the dates of the COVID-19 pandemic, which led to significant changes in ICU staffing patterns in some hospitals [28]. However, the area in which this study took place did not see significant numbers of COVID patients until November 2020, making it unlikely that the pandemic could affect our results.

We supplemented the registry data with selected meta-data designed to identify ICU providers working in study ICUs. We focused on three types of providers: intensivist physicians, bedside nurses, and respiratory therapists. We chose these provider types because they are the core members of the interprofessional rounding team and because they were active bedside care providers in all ICUs during the study period. All meta-data were related to selected instances in which providers entered data into the patient’s chart. For intensivists, relevant data included critical care progress notes. For bedside nurses, relevant data included clinical assessments such as cardiac rhythms and neurological status. For respiratory therapists, relevant data included ventilator settings and respiratory assessment forms. For each data point, we collected the following metadata: the provider’s name, the provider’s unique numeric ID number specific to the EHR system, the patient’s unique hospitalization ID, and a date and time stamp for when the data were entered into the patient’s chart. Additional information on the rationale for our choice of meta-data is available in the Supplementary Methods.

2.2. Creating patient-provider linkages

We used deterministic hierarchical algorithms to link providers to specific patients on specific days. During this step we made no patient exclusions—all ICU patients and ICU patient-days were eligible for link-age. We defined a day as 7:00 AM to 6:59:59 AM in order to align days with typical shift schedules. For intensivists, we created the patient-provider linkages using the date and time stamp of the critical care progress notes. Only one intensivist could be assigned on each day. When multiple physicians wrote progress notes on a single day, we prioritized physicians by whether critical care was the physician’s primary specialty and the timing of the note. If no note was written, the patient was not assigned an intensivist for that day. Additional details on this process are provided in the Supplementary Methods.

For nurses and respiratory therapists, we created patient-provider linkages using the date and time stamps of charting instances specific to each provider type. When multiple nurses or respiratory therapists charted during a shift, we prioritized providers by the frequency of charting and the nearness of the charting to the middle of the shift. If no nurse or respiratory therapist was identified after applying the algorithm, the patient was not assigned a nurse or respiratory therapist during that shift. Additional details on this process are provided in the Supplementary Methods.

2.3. Constructing the rounding team

We defined the rounding team at the level of the patient-day, assigning up to one intensivist, nurse, and respiratory therapist to each patient for each day they were in the ICU. Providers comprised a team if they shared a specific patient on a specific day. For intensivists, this meant the entire 24-h period, while for nurses and respiratory therapists this meant the day shift (7:00:00 AM to 6:59:59 PM). Thus, teams could have as many as three members (one intensivist, one nurse, and one respiratory therapist, which we define as a “complete team”) or as few as zero members (if no providers were identified on a certain day), and providers could be on multiple teams each day. We defined teams in this way because the ultimate goal of this work is to identify team factors associated with patient outcomes, and conceptually this group of providers is directly responsible for the quality of care delivered to each patient on a given day.

To simplify the analysis, we constructed rounding teams only for patient-days in which the patient received some mechanical ventilation, defined using ventilation start and stop times in the ICU registry. We made this decision since patients not receiving mechanical ventilation would be unlikely to have an assigned respiratory therapist and therefore would not have a complete team.

2.4. Descriptive analysis

We then performed a series of descriptive analyses designed to understand the characteristics of the ICU teams. As above, all analyses considered only patient-days in which the patient received some mechanical ventilation. First, we summarized the data at the ICU level, presenting general ICU characteristics and summary data on admitted patients and clinicians. Second, we summarized the data at the provider level, presenting data on their working patterns and team membership patterns. Third, we summarized data at the patient-day level, presenting data on how many patient-days were successfully ascribed a complete team. Fourth, we summarized data at the team level, describing both the number of unique complete teams and quantifying their average individual experience and shared experience over the course of the study period (additional details on how we defined these measures are provided in the Supplementary Methods).

2.5. Network analysis

Finally, we performed a network analysis to better understand and quantify patterns of team interaction [29]. For this analysis we defined nodes as individual providers and ties as any shared team membership. We created the networks within ICUs rather than over the entire system since the ICU is the standard unit of analysis for most studies of critical care structure and outcome [30].

We examined three properties of the networks: density, degree centralization, and closeness centralization [31]. We selected these network properties based on their theoretical relationship to information spread within the network, and therefore their potential influence on dissemination of new knowledge or application of evidence-based practices [11]. These properties are defined as follows:

• Density refers to the proportion of connected ties in a network relative to the number of possible ties. All else being equal, density is higher in networks with greater numbers of connections. In the ICU, greater density would mean that more staff members tend to work with more other staff members, such that there are greater connections and fewer cliques.

• Degree centralization refers to the number of ties that a certain node has, averaged over the network. All else being equal, degree centralization is greater in networks that have individual nodes of greater importance. In the ICU, greater degree centralization would mean that some ICU members are very important to the network—they have lots of ties and may serve to bring people together.

• Closeness centralization refers to the shortest number of connections between a node and all other nodes, averaged over the network. All else being equal, closeness centralization is lower in networks in which nodes are generally closer to each other. In the ICU, lower closeness centralization means that even staff members that aren’t direct connected are still close to other team members; they have fewer “degrees of separation” and information may therefore travel faster throughout the ICU.

After calculating these network measures, we used linear regression to examine the relationship between network measures and readily observable ICU characteristics, including ICU type, ICU size, and the number of unique providers in the ICU. We performed this analysis to address the possibility the network measures (density, degree centralization, and network centralization) were simply alternative representations of factors already known to vary across ICU. If the regression showed that observable characteristics explained nearly all of the variation in the network measures, then the network measures would be providing no new information and therefore would not be useful in describing ICU structure. If, however, there was significant unexplained variation even after controlling for observable ICU characteristics, in would indicate that the network measures provide new and potentially important insight into ICU structure.

2.6. Software and ethics approval

Data management was performed using Microsoft SQL Server Management Studio version 18.11.1 and Stata version 17.0. The descriptive analysis was performed using Stata version 17.0. The network analysis was performed using the igraph package in R version 4.2.0. This research was reviewed and approved by the University of Pittsburgh Human Research Protections Office (PRO18040357).

3. Results

ICU characteristics are shown in Table 1. Of the 27 study ICUs, the plurality were surgical (n = 13, 48.1%) and most had 10–20 ICU beds (n = 15, 55.6%). The number of admissions requiring mechanical ventilation over the study period ranged from 106 in the smallest ICU to 735 in the largest ICU (median = 323). Within ICUs the number of unique intensivists identified ranged from 6 to 41 (median = 11); the number of unique nurses ranged from 34 to 168 (median = 79); and the number of unique respiratory therapists ranged from 13 to 78 (median = 42).

Table 1.

Intensive care unit characteristics (N = 27).

Characteristic	Value
ICU type (N, %)
Medical	5 (19%)
Surgical	13 (48%)
Mixed	9 (33%)
ICU size
<10 beds	9 (33%)
10–20 beds	15 (56%)
>20 beds	3 (11%)
Unique admissions
Median [IQR]	323 [181, 541]
Range	106–735
Unique intensivists
Median [IQR]	11 [8, 12]
Range	6–41
Unique nurses
Median [IQR]	79 [63, 114]
Range	34–168
Unique respiratory therapists
Median [IQR]	42 [36, 64]
Range	13–78

All values refer to the index year (September 1, 2019 to August 31, 2020).

ICU = intensive care unit; IQR = interquartile range.

Provider characteristics are shown in Table 2. On average, respiratory therapists tended to work in a greater number of different ICUs compared to intensivist physicians and nurses. On average, intensivist physicians tended to work more days and see more patients per day and more patients per year compared to nurses and respiratory therapists.

Table 2.

Provider characteristics.

Characteristic	Intensivists N = 176	Nurses N = 1689	Respiratory therapists N = 462
ICUs worked
Mean (SD)	1.8 (1.0)	1.4 (0.8)	2.7 (1.6)
Median [IQR]	2 [1,2]	1 [1, 2]	3 [1,3]
Range	1–5	1–10	1–10
Days worked
Mean (SD)	49 (41)	24 (22)	35 (34)
Median [IQR]	42 [12, 76]	18[6, 37]	25 [7, 54]
Range	1–161	1–141	1–165
Average patient encounters per day
Mean (SD)	4.6 (2.2)	1.2 (0.2)	2.5 (1.1)
Median [IQR]	4.6 [2.8, 6.0]	1.2 [1.0, 1.3]	2.4 [1.7, 3.2]
Range	1.0–14.2	1.0–2.0	1.0–6.3
Total patient encounters over year
Mean (SD)	258.0)	29 (28)	107 (125)
Median [IQR]	209 [47, 374]	22 [6, 45]	61 [10, 168]
Range	1–1519	1–191	1–652
Number of unique complete teams
Mean (SD)	256 (256)	27 (26)	99 (113)
Median [IQR]	209 [46, 369]	20 [6, 40]	56 [9, 150]
Range	1–1506	0–180	0–637

ICU = intensive care unit; SD = standard deviation; IQR = interquartile range.

In total we identified 34,749 unique teams, 30,892 (88.9%) of which were complete (i.e. all three provider types were identified). Most incomplete teams occurred when the patient was admitted or discharged part-way through a day, so they did not encounter all provider types. The incomplete teams were usually comprised of only a nurse and respiratory therapist (3392 teams, 9.8%), with the remaining teams comprised of two other providers or just one provider (465 teams, 1.3%). For complete teams, the measures of average team member experience and shared team experience over the study period are shown in Fig. 1. Teams varied markedly in both average experience (range: 9.0 to 706.3 shifts, median, IQR: 262.2, 177.7–334.7) and shared experience (range: 1.0 to 99.3 shifts, median, IQR: 13.2, 6.0–16.7).

Fig. 1.

Distribution of average critical care team member experience (Panel A) and critical care team shared experience (Panel B) for complete teams (n = 30,892).

Average team member experience (Panel A) is the mean of the individual experience of each of the three team members. Shared team experience (Panel B) is the total number of times each of the three team members has worked with each other.

The results of the network analysis are shown in Table 3. ICUs varied based on their density (range: 0.1 to 0.2, median, IQR: 0.1, 0.1–0.2), degree centralization (range: 0.4 to 0.6, median, IQR: 0.5, 0.5–0.6), and closeness centralization (range: 0.1 to 0.6, median, IQR: 0.5, 0.4–0.5). Since these values do not have units they are primarily interpretable relative to each other, i.e., by comparing ICUs along the metrics. More information about the relative interpretation of these measures can be found in the methods (above). Representative sociograms for these measures are shown in Fig. 2. The regression analyses examining the observable factors associated with these network measures is shown in Table 4. No readily observable factor was consistently significantly associated with density, degree centralization, or closeness centralization, meaning that network properties are not simply reflections of the number different providers. Instead, they are emergent properties related to how those providers interact. Further, the amount of variance (R²) explained by these models did not exceed 68%, indicating that the network measures relate to factors besides ICU type, ICU size, and number of unique providers.

Table 3.

Intensive care unit network measures (N = 27).

Measure	Value
Number of nodes
Mean (SD)	148 (54)
Median [IQR]	139 [113, 184]
Range	59–297
Number of edges
Mean (SD)	1353 (778)
Median [IQR]	1200 [803, 1685]
Range	366–3995
Network density
Mean (SD)	0.13 (0.04)
Median [IQR]	0.12 [0.10, 0.17]
Range	0.07–0.23
Network degree centralization
Mean (SD)	0.50 (0.08)
Median [IQR]	0.50 [0.47, 0.55]
Range	0.35–0.65
Network closeness centralization
Mean (SD)	0.45 (0.10)
Median [IQR]	0.45 [0.42, 0.51]
Range	0.11–0.60

SD = standard deviation; IQR = interquartile range.

Fig. 2.

Representative sociograms for density, a measure of the overall network connectivity (Panel A); degree centralization (Panel B), a network-wide summary measure of individual node importance; and closeness centralization, a network-wide summary measure of individual node connectedness (Panel C).

ICU = intensive care unit.

Sociograms on the left show the ICU with the lowest value for that measure, and sociograms on the right show the ICU with the highest value for that measure. The ICU with the lowest density is also the ICU with lowest closeness centralization; and the ICU with the highest degree centralization is also the ICU with the highest closeness centralization.

Table 4.

Relationship between network measures and ICU characteristics (n = 27).

	Density	Degree Centralization	Closeness Centralization
ICU type (surgical vs not)	−0.003 (−0.028 to 0.022) p = 0.82	−0.003 (−0.060 to 0.055) p = 0.93	0.045 (−0.053 to 0.143) p = 0.35
ICU size (<10 vs not)	−0.024 (−0.056 to 0.007) p = 0.13	0.008 (−0.065 to 0.081) p = 0.83	−0.0134 (−0.137 to 0.110) p = 0.82
Intensivist count	0.001 (−0.001 to 0.002) p = 0.66	−0.008 (−0.012 to −0.004) p < 0.01	−0.003 (−0.010 to 0.004) p = 0.34
Nurse count	−0.001 (−0.001 to 0.000) p = 0.02	0.001 (0.001 to 0.003) p = 0.02	0.001 (−0.001 to 0.003) p = 0.53
Respiratory therapist count	−0.001 (−0.002 to −0.001) p = 0.01	0.000 (−0.002 to 0.002) p = 0.77	−0.001 (−0.004 to 0.003) p = 0.70
Intercept	0.239 (0.199 to 0.280) p < 0.01	0.484 (0.392 to 0.578) p < 0.01	0.449 (0.291 to 0.607) p < 0.01
Model R2	0.68	0.51	0.14

All values except Model R² are the regression coefficients, their 95% confidence intervals, and p-values from Wald tests.

ICU = intensive care unit.

4. Discussion

We demonstrate that meta-data from the EHR can be repurposed to identify and characterize ICU rounding teams. In addition we show that teams vary substantially among key metrics—some teams have a great deal of experience working together over the previous year, while other teams have extremely little experience working together over the previous year. ICUs also vary in terms of the connections between staff. Some ICUs have high density networks, in that most staff members tend to have experience working with most other staff members, while some ICUs have low density networks, in that staff members tend to work with just a few other staff members. Some ICUs are highly centralized, in that just a few connections link all members (i.e., team members have fewer “degrees of separation”), while some ICUs are less centralized, in that it takes many connections to link different team members. A greater understanding of these differences help explain why patterns of culture and communication vary across ICUs [19].

These methods open up several new lines of inquiry in the effort to understand how team structure and function influence patient outcomes in the ICU. For example, these methods enable research examining how team experience relates to clinical outcomes like mortality, complication rates, and length of stay. Individual and shared experience among team members is known to improve team performance in other settings [32] and may underlie the volume-outcome relationship in critical care [33]. These methods also enable work examining how team continuity for patients across their ICU stays might influence patient outcomes. Continuity of care is an important determinant of outcomes in the outpatient setting and may also be important in inpatient settings like the ICU [34].

Our study expands upon past efforts to characterize team structure by observing provider-patient and provider-provider interactions in the EHR. Costa et al. demonstrated that EHR data can be used to examine network ties between ICU nurses [35]. Soulakis and Carson used a similar approach to study team-based care for patients hospitalized with cardiology conditions, demonstrating that team structures vary across patients and might influence collaboration patterns [36–38]. Other investigators have applied EHR meta-data to understand care delivery patterns in a variety of different settings [39–42]. We extend and expand upon these studies by examining a broader set of providers, focusing specifically on the ICU, and adding clinical context by focusing on the key care providers most relevant to evidence-based practice.

Our study has several limitations. First, we only examined three core provider types—intensivist physicians, respiratory therapists, and nurses—neglecting other potentially relevant provider types like advanced practice providers, clinical pharmacists, dieticians, social workers, and non-intensivist physicians. We made this decision because these providers are most directly responsible for the quality of care for mechanically ventilated patients on a day-to-day basis. Second, for the same reasons, we elected to define the team at the level of the patient day. We acknowledge that alternative definitions of the team (e.g. all the providers who worked in the ICU in a given day or all the providers who worked on a single patient over the course of the patient’s ICU day) could be relevant in other circumstances. Third, we used only a small amount of the meta-data available to us. We made this decision because other types of meta-data (e.g. from other types of charting or generated by the simple act of opening the patient’s electronic chart) might identify individuals not directly involved in the patient’s care. Fourth, our algorithms intentionally omitted providers that did not interact with the medical record. We took this approach in part by necessity (since our approach was EHR based) and in part to avoid false positives in team attribution. However this choice meant that we may have omitted some important team members. Fifth, since there is no agreed upon reference standard for ICU team membership, we could not directly assess the accuracy of our algorithms. Future work should develop a robust conceptual model for ICU team membership enabling direct assessment of the accuracy of different approaches for identifying team members.

5. Conclusions

EHR meta-data provide a feasible and scalable method for identifying and characterizing the members of the ICU care team for individual patients. Future investigations based on this method should directly examine the link between team structure and patient outcomes, potentially revealing novel targets for leveraging the ICU team to improve the quality of care.