Temporal Clustering of Critical Illness Events on Medical Wards

Samik Doshi; Saeha Shin; Lauren Lapointe-Shaw; Robert A Fowler; Michael Fralick; Janice L Kwan; Kaveh G Shojania; Terence Tang; Fahad Razak; Amol A Verma

doi:10.1001/jamainternmed.2023.2629

. 2023 Jul 10;183(9):924–932. doi: 10.1001/jamainternmed.2023.2629

Temporal Clustering of Critical Illness Events on Medical Wards

Samik Doshi ^1,², Saeha Shin ², Lauren Lapointe-Shaw ^2,^3,⁴, Robert A Fowler ^2,^3,⁵, Michael Fralick ^2,⁶, Janice L Kwan ^2,⁶, Kaveh G Shojania ^2,⁵, Terence Tang ^2,⁷, Fahad Razak ^1,^2,³, Amol A Verma ^1,^2,^3,^✉

¹General Internal Medicine and Li Ka Shing Knowledge Institute, St Michael’s Hospital, Unity Health Toronto, Toronto, Ontario, Canada

²Department of Medicine, University of Toronto, Toronto, Ontario, Canada

³Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada

⁴Department of Medicine, University Health Network, Toronto, Ontario, Canada

⁵Department of Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada

⁶Department of Medicine, Sinai Health System, Toronto, Ontario, Canada

⁷Institute of Better Health, Trillium Health Partners, Mississauga, Ontario, Canada

Accepted for Publication: April 11, 2023.

Published Online: July 10, 2023. doi:10.1001/jamainternmed.2023.2629

^✉

Corresponding Author: Amol A. Verma, MD, MPhil, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Two Queen St East, Room 714-2, Toronto, ON M5C 3G7, Canada (amol.verma@mail.utoronto.ca).

Author Contributions: Ms Shin had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Razak and Verma served as co–senior authors.

Concept and design: Doshi, Shin, Shojania, Razak, Verma.

Acquisition, analysis, or interpretation of data: Doshi, Shin, Lapointe-Shaw, Fowler, Fralick, Kwan, Tang, Razak, Verma.

Drafting of the manuscript: Doshi, Shin, Razak, Verma.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Doshi, Shin, Razak.

Obtained funding: Razak, Verma.

Administrative, technical, or material support: Shin, Fralick, Razak, Verma.

Supervision: Shojania, Razak, Verma.

Conflict of Interest Disclosures: Dr Fralick reported receiving personal fees from Proof Diagnostics Inc outside the submitted work. Dr Razak reported being a salaried employee of Ontario Health, the agency overseeing many aspects of health care delivery in Ontario, during the conduct of the study. Dr Verma reported being a part-time employee of Ontario Health outside the submitted work and being the coinventor of an artificial intelligence–based early warning system to detect patient deterioration, which was acquired by a start-up company, Signal 1. No other disclosures were reported.

Funding/Support: The development of the GEMINI data platform was supported with funding from the Canadian Cancer Society, the Canadian Frailty Network, the Canadian Institutes of Health Research, the Canadian Medical Protective Association, the Digital Research Alliance of Canada, Green Shield Canada Foundation, the Natural Sciences and Engineering Research Council of Canada, Ontario Health, the St Michael’s Hospital Association Innovation Fund, and the Department of Medicine at the University of Toronto, with in-kind support from partner hospitals and the Vector Institute. Dr Verma is supported by the Temerty Professor in AI Research and Education in Medicine, University of Toronto.

Role of the Funder/Sponsor: The funding sources had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2.

^✉

Corresponding author.

PMCID: PMC10334292 PMID: 37428478

Key Points

Question

Is a critical illness event (death or intensive care unit transfer) associated with an increase in the near-term risk of critical illness in other patients on the same medical ward?

Findings

In this cohort study of 118 529 hospital admissions at 5 hospitals, patients were more likely to die or be transferred to an intensive care unit within 12 hours after another patient experienced a critical illness event on the same ward.

Meaning

Findings suggest that critical illness events tend to cluster on medical wards, and efforts to better understand this association represent important opportunities to improve patient safety.

Abstract

Importance

Recognizing and preventing patient deterioration is important for hospital safety.

Objective

To investigate whether critical illness events (in-hospital death or intensive care unit [ICU] transfer) are associated with greater risk of subsequent critical illness events for other patients on the same medical ward.

Design, Setting, and Participants

Retrospective cohort study in 5 hospitals in Toronto, Canada, including 118 529 hospitalizations. Patients were admitted to general internal medicine wards between April 1, 2010, and October 31, 2017. Data were analyzed between January 1, 2020, and April 10, 2023.

Exposures

Critical illness events (in-hospital death or ICU transfer).

Main Outcomes and Measures

The primary outcome was the composite of in-hospital death or ICU transfer. The association between critical illness events on the same ward across 6-hour intervals was studied using discrete-time survival analysis, adjusting for patient and situational factors. The association between critical illness events on different comparable wards in the same hospital was measured as a negative control.

Results

The cohort included 118 529 hospitalizations (median age, 72 years [IQR, 56-83 years]; 50.7% male). Death or ICU transfer occurred in 8785 hospitalizations (7.4%). Patients were more likely to experience the primary outcome after exposure to 1 prior event (adjusted odds ratio [AOR], 1.39; 95% CI, 1.30-1.48) and more than 1 prior event (AOR, 1.49; 95% CI, 1.33-1.68) in the prior 6-hour interval compared with no exposure. The exposure was associated with increased odds of subsequent ICU transfer (1 event: AOR, 1.67; 95% CI, 1.54-1.81; >1 event: AOR, 2.05; 95% CI, 1.79-2.36) but not death alone (1 event: AOR, 1.08; 95% CI, 0.97-1.19; >1 event: AOR, 0.88; 95% CI, 0.71-1.09). There was no significant association between critical illness events on different wards within the same hospital.

Conclusions and Relevance

Findings of this cohort study suggest that patients are more likely to be transferred to the ICU in the hours after another patient’s critical illness event on the same ward. This phenomenon could have several explanations, including increased recognition of critical illness and preemptive ICU transfers, resource diversion to the first event, or fluctuations in ward or ICU capacity. Patient safety may be improved by better understanding the clustering of ICU transfers on medical wards.

This cohort study investigates whether a critical illness event (death or intensive care unit transfer) is associated with an increase in the near-term risk of critical illness in other patients on the same medical ward.

Introduction

Clinical deterioration resulting in transfer to an intensive care unit (ICU) or death occurs in 5% to 10% of adult hospitalizations.^1,2,3,4,5 Patients transferred to the ICU from inpatient wards have a higher mortality and longer hospital stays than those admitted directly to the ICU from the emergency department.^4,5,6 Preventing and recognizing in-hospital clinical deterioration is important to improve health outcomes and reduce costs of care.

Clinical prediction tools for patient deterioration are based on individual risk factors.^{3,7,8,9,10,11,12,13,14} A 2016 single-center study found that the likelihood of cardiac arrest or ICU transfer was higher after another such event occurred on the same hospital ward in the preceding 6 hours,¹⁵ which suggests that critical illness events may themselves be a risk factor for further events beyond individual patient risk. Resource strain is known to be associated with adverse patient outcomes, and critical illness events may be a marker of this or may divert resources from other patients.^16,17,18 To our knowledge, the association between critical illness events on the same hospital ward has not been reported in other institutions or geographic settings and the potential causes have not been investigated.

We studied the association between critical illness events and subsequent critical illness in other patients on the same general internal medicine (GIM) wards at 5 hospitals in Toronto, Canada. General internal medicine wards care for nearly 40% of patients admitted from the emergency department at these hospitals.¹⁹ In accordance with the previous study,¹⁵ we hypothesized that the recognition or risk of critical illness as defined by ICU transfer or death would be greater after a critical illness event occurred on the same ward. We sought to confirm this finding, strengthen the investigation of potential causality with a “negative control” analysis, and provide preliminary insights into potential explanations for the association.

Methods

Design, Setting, and Participants

This retrospective cohort study involved 5 academic hospitals in Toronto, Ontario, Canada, and was approved by the research ethics board of St Michael's Hospital as the delegated board of record for all sites under Clinical Trials Ontario. The need for informed patient consent was waived by the board because this was a large, retrospective database study. Hospitals were participating in the General Medicine Inpatient Initiative (GEMINI) research collaborative,¹⁹ which collected data about all GIM hospitalizations. The GIM services at these hospitals include teaching services involving undergraduate and postgraduate trainees and a small number of nonteaching teams.¹⁹ All hospitals have on-site intensivist-led ICUs and ICU rapid response teams for deteriorating patients on the ward. Nurse to patient ratios on GIM wards ranged between 1:4 and 1:6, depending on staff availability, patient volume, and time of day.

We included patients who were admitted to or discharged from a GIM ward between April 1, 2010, and October 31, 2017. Room transfer data were used to identify each patient’s ward assignment throughout the hospital stay. Study investigators at each site labeled inpatient wards as “exclusively GIM” (ie, only GIM patients were cared for on this ward) or not. We included patients who spent at least part of their hospitalization on an exclusive GIM ward to avoid potential confounding associated with wards with mixed types of patients, some of whom might be hospitalized for planned procedures that could result in transfer to the ICU, whereas GIM patients are admitted almost exclusively for emergency reasons without planned ICU stays (Figure 1).

Figure 1. — GIM indicates general internal medicine.

Data Collection

GEMINI^19,20 collects administrative health data from hospitals as reported to the Canadian Institute for Health Information and electronic clinical data extracted from hospital information systems. Manual validation of 3300 hospitalizations determined that the sensitivity and positive predictive value of extracted data in GEMINI for transfer to the ICU were 100% and 96%, respectively, and for death were 100% and 100%, respectively.²⁰

Exposure

Each hospital admission was structured into nonoverlapping 6-hour intervals. In each 6-hour interval, we measured the total number of ICU transfers or deaths that occurred on the same GIM ward, categorized as 0, 1, or greater than 1. Because we were interested in measuring unplanned critical illness events, we included only ICU transfers from a GIM ward directly to the ICU. Postoperative ICU transfers were not considered to be unplanned ICU events and therefore were not included in outcomes. We examined the association between events in one 6-hour interval (exposure) and those in the next 6-hour interval (outcome) on the same medical ward (eFigure 1 in Supplement 1). The occurrence of an event for 1 patient was not counted as an exposure for the same patient.

Outcomes and Measures

The primary outcome was a composite of critical illness events, defined as number of ICU transfers and in-hospital deaths per 1000 6-hour patient blocks. We defined ICUs according to the Canadian Institute of Health Information definition, “locations where critically ill patients receive life supporting care,”²¹ and included medical, surgical, trauma, and coronary ICUs. Secondary outcomes were the number of ICU transfers alone and number of in-hospital deaths without ICU transfer.

Covariates

Measured patient characteristics included time-constant admission characteristics and time-varying factors. Admission characteristics included age, sex, Charlson Comorbidity Index score (scores range from 0 to 24, with higher scores reflecting greater comorbidity and serving as a validated predictor of in-hospital mortality),²² day of admission to the hospital (weekend or holiday vs weekday), time of admission (daytime [8 am to 4:59 pm] vs nighttime [5 pm to 7:59 am]), season of admission, month of admission, emergency department boarding time, and admitting hospital. We also calculated the Laboratory-based Acute Physiology Score (LAPS)^23,24 at hospital admission, a validated predictor of patient mortality, with a possible range of scores from 0 to 256 and higher scores reflecting greater risk. We adjusted for time-varying factors for each 6-hour interval, including day at the start of each interval (weekday vs weekend), time at the start of each interval (day vs night), interval number (the count of the number of 6-hour intervals since hospital admission), and the GIM census ratio on the calendar day of the interval. Because hospital capacity strain has been associated with adverse patient outcomes,²⁵ we calculated the ratio of the inpatient GIM census on every day of the study period to the median GIM census for the hospital in that year. This census ratio has been used in previous studies^26,27 to account for time-varying changes in patient volume compared with typical patient volumes and is a proxy for capacity strain on the GIM wards. Among patients who experienced an outcome, we further described the events with the day of outcome, time of outcome, and the LAPS at 24 and 48 hours before the outcome. Among patients transferred to the ICU, we report those who received invasive mechanical ventilation and those who died in the hospital.

Because covariates and outcomes were largely composed of data that are mandatorily reported to the Canadian Institute for Health Information, there were no missing data. We used a slightly simplified version of the original LAPS, in which all missing values are considered normal and which has been validated in the GEMINI data set,²⁸ whereas the original LAPS assigned higher scores to missing arterial pH, troponin, and white blood cell count values for patients with higher predicted mortality risk.^23,24

Statistical Analysis

We compared patient characteristics among exposed and unexposed groups, using standardized differences, with differences greater than 0.10 reflective of imbalance between groups.²⁹ Exposures in each 6-hour interval were assessed for their association with outcomes on the same GIM ward in the subsequent 6-hour interval. To obtain unadjusted rates of outcomes per 1000 6-hour intervals and 95% CIs, we used a Poisson model with robust standard errors. A discrete-time mixed-effects complementary log-log generalized linear model allowed us to model the occurrence of events within each mutually exclusive 6-hour interval and estimate the odds ratio (OR) with 95% CI.³⁰ Hospitalizations and wards were included as crossed random effects to account for clustering within hospitalizations and within wards across 6-hour interval observations. We adjusted for time-constant and time-varying covariates as described. In addition to the overall model, we fit models for each hospital separately to assess the consistency of results. To explore potential explanations for an observed association, we compared patient and event characteristics among those who experienced an outcome across exposure categories. To investigate the association between exposures and in-hospital mortality after ICU transfer, we analyzed the subset of patients who were admitted to an ICU. We constructed a mixed-effects logistic regression model, adjusting for hospital and time-constant characteristics as described earlier, as well as the characteristics of the interval in which the ICU transfer occurred (GIM census ratio, day of interval, and time of interval). The results are presented as ORs with 95% CIs, considering the random effect of the ward level.

Sensitivity Analyses

We repeated the analyses with 12- and 24-hour intervals. We excluded patients who received surgical procedures because this might result in planned postoperative ICU care (Figure 1). We excluded deaths in which patients received palliative care from the outcome, as a proxy for “expected” deaths. We identified palliative care with the International Statistical Classification of Diseases and Related Health Problems, 10th Revision discharge diagnosis code Z51.5.

Negative Control Analysis

We repeated the analyses to test the association of critical illness events on one GIM ward with outcomes occurring on a different GIM ward in the same hospital. Instead of testing the association between outcomes and exposures from the same GIM ward, the outcomes in every 6-hour patient interval were linked to exposures in the prior 6-hour interval from a different GIM ward in the same hospital (eFigure 2 in Supplement 1). We hypothesized that an association between events occurring on different wards would reflect hospital- and time-level factors, whereas if ward-level factors predominated, there would be no association between events occurring on different GIM wards. The number of GIM wards in the 5 hospitals was 5, 5, 3, 2, and 1. For hospitals with more than 2 GIM wards, each 6-hour interval was paired with the prior 6-hour interval from a different randomly selected GIM ward. One hospital had only a single GIM ward and could not be included in this analysis. All analyses were performed with R, version 4.0.2 (R Foundation for Statistical Computing). Data analysis occurred between January 1, 2020, and April 10, 2023 (with the prolonged duration due to interruption because of the COVID-19 pandemic). This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

Results

The study cohort included 118 529 hospitalizations on a GIM ward (Figure 1). The median age was 72 years (IQR, 56-83 years), 50.7% were male, and 49.3% were female (Table 1). Data about patient race and ethnicity are not collected routinely or in standardized fashion in Canadian hospitals and therefore could not be reported for this study. Of all 118 529 GIM patient hospitalizations, 35 301 (29.8%) were exposed to no critical illness events on the same ward and 83 228 (70.2%) were exposed to at least 1 event. Patients exposed to prior events were somewhat older (72 vs 70 years) and had higher Charlson Comorbidity Index scores (Table 1).

Table 1. Patient Characteristics.

Characteristic	All	Exposed to 0 events^a	Exposed to 1 event^a	Exposed to >1 event^a	SMD (0 vs 1 event)	SMD (0 vs >1 event)
No. of admissions	118 529	35 301	48 758	34 470	NA	NA
Age, median (IQR), y	72 (56-83)	70 (53-83)	72 (57-84)	72 (58-83)	0.113	0.142
Sex, No. (%)
Male	60 040 (50.7)	17 623 (49.9)	24 662 (50.6)	17 755 (51.5)	0.013	0.032
Female	58 489 (49.3)	17 678 (50.1)	24 096 (49.4)	16 715 (48.5)	0.013	0.032
Charlson Comorbidity Index score, No. (%)^b
0	44 201 (37.3)	16 338 (46.3)	18 040 (37.0)	9823 (28.5)	0.201	0.410
1	18 261 (15.4)	5532 (15.7)	7728 (15.8)	5001 (14.5)
≥2	56 067 (47.3)	13 431 (38.0)	22 990 (47.2)	19 646 (57.0)
Most common discharge diagnoses, No. (%)
Heart failure	5887 (5.0)	1603 (4.5)	2428 (5.0)	1856 (5.4)	0.021	0.039
Pneumonia	4877 (4.1)	1527 (4.3)	2049 (4.2)	1301 (3.8)	0.006	0.028
COPD	3032 (2.6)	1007 (2.9)	1276 (2.6)	749 (2.2)	0.014	0.043
Urinary tract infection	3815 (3.2)	1155 (3.3)	1652 (3.4)	1008 (2.9)	0.006	0.020
GI hemorrhage	2223 (1.9)	790 (2.2)	901 (1.8)	532 (1.5)	0.028	0.051
LAPS, median (IQR)^c	16 (6-27)	16 (6-27)	17 (6-29)	15 (6-26)	0.093	0.045
Weekend admission, No. (%)	28 888 (24.4)	9000 (25.5)	11 792 (24.2)	8096 (23.5)	0.030	0.047
Night admission, No. (%)	91 359 (77.1)	27 385 (77.6)	37 359 (76.6)	26 615 (77.2)	0.023	0.009
ED boarding time, median (IQR), h	13.1 (9.3-21.0)	13.0 (9.4-20.0)	13.2 (9.3-21.0)	13.5 (9.3-21.7)	0.039	0.074
GIM census ratio, mean of all periods (SD)^d	1.0 (0.1)	1.0 (0.1)	1.0 (0.1)	1.0 (0.1)	0.032	0.075
Season, No. (%)
Spring	30 233 (25.5)	8716 (24.7)	12 647 (25.9)	8870 (25.7)	0.038	0.059
Summer	31 800 (26.8)	9684 (27.4)	13 138 (26.9)	8978 (26.0)
Fall	29 373 (24.8)	9084 (25.7)	11 924 (24.5)	8365 (24.3)
Winter	27 123 (22.9)	7817 (22.1)	11 049 (22.7)	8257 (24.0)

Open in a new tab

Abbreviations: COPD, chronic obstructive pulmonary disease; ED, emergency department; GI, gastrointestinal; GIM, general internal medicine; LAPS, Laboratory-based Acute Physiology Score; NA, not applicable; SMD, standardized mean difference.

^{^a}

Hospital visits were categorized according to their maximum exposure to critical illness events in 6-hour periods across the duration of admission. Hospital visits with multiple exposures were grouped into the category based on the largest number of events in a 6-hour period.

^{^b}

Scores range from 0 to 24, with higher scores reflecting greater comorbidity and serving as a validated predictor of in-hospital mortality.²²

^{^c}

LAPS^23,24 is a validated predictor of patient mortality when combined with age and comorbidity and has a possible range of scores from 0 to 256, with higher scores reflecting greater risk of mortality.

^{^d}

Defined as the GIM census on every day of the study period divided by the median GIM census of that year.

Study Outcomes

The primary outcome of ICU transfer or death occurred in 8785 of the 118 529 hospitalizations (7.4%), with ICU transfer in 4062 (3.4%) and death without ICU transfer in 4723 (4.0%). Before adjustment, critical illness events were more common in the 6-hour intervals after exposure to 1 critical illness event (3.8 events per 1000 6-hour patient blocks; 95% CI, 3.6-4.0) or more than 1 critical illness event (4.4 events per 1000 6-hour patient blocks; 95% CI, 3.9-4.9) compared with exposure to no events (2.6 events per 1000 6-hour patient blocks; 95% CI, 2.5-2.6) (Figure 2). An ICU transfer was more common, but death without ICU transfer was not (Figure 2).

Figure 2. — Error bars represent 95% CIs. ICU indicates intensive care unit.

In the adjusted model, compared with periods with no exposure, patients were more likely to experience the outcome after exposure to 1 event (adjusted OR [AOR], 1.39; 95% CI, 1.30-1.48) or more than 1 event (AOR, 1.49; 95% CI, 1.33-1.68) in the prior 6-hour interval (Table 2). The odds of ICU transfer were greater for patients exposed to prior critical illness events compared with those without exposure (1 event: AOR, 1.67; 95% CI, 1.54-1.81; >1 event: AOR, 2.05; 95% CI, 1.79-2.36), but there was no significant association for death alone (1 event: AOR, 1.08; 95% CI, 0.97-1.19; >1 event: AOR, 0.88; 95% CI, 0.71-1.09) (Table 2). These associations were generally consistent across all participating hospitals (eFigures 3 and 4 in Supplement 1).

Table 2. Regression Models of Primary and Secondary Outcomes.

Outcome	No. of exposure events	OR (95% CI)^a
		Association on same GIM ward		Association on different GIM wards^b
		Unadjusted	Adjusted	Unadjusted	Adjusted
Primary (in-hospital death or ICU)	1	1.40 (1.32-1.49)	1.39 (1.30-1.48)	1.11 (1.01-1.17)	1.04 (0.95-1.10)
Primary (in-hospital death or ICU)	>1	1.59 (1.42-1.78)	1.49 (1.33-1.68)	1.16 (0.99-1.27)	1.09 (0.93-1.20)
In-hospital death without ICU	1	1.07 (0.97-1.18)	1.08 (0.97-1.19)	1.07 (0.95-1.15)	0.97 (0.86-1.05)
In-hospital death without ICU	>1	0.91 (0.74-1.12)	0.88 (0.71-1.09)	0.78 (0.61-0.92)	0.85 (0.68-0.98)
ICU transfer	1	1.68 (1.56-1.82)	1.67 (1.54-1.81)	1.24 (1.10-1.33)	1.12 (0.98-1.22)
ICU transfer	>1	2.19 (1.92-2.50)	2.05 (1.79-2.36)	1.33 (1.07-1.52)	1.25 (0.99-1.43)

Open in a new tab

Abbreviations: GIM, general internal medicine; ICU, intensive care unit; OR, odds ratio.

^{^a}

Odds ratio of outcome compared with exposure to 0 events.

^{^b}

One hospital was excluded from analysis because it had only 1 GIM ward.

Characteristics of Events Across Exposure Categories

Among the 8785 patients who experienced the primary outcome, 7381 (84.0%) were exposed to no critical illness events in the 6-hour interval before the outcome, 1086 (12.4%) were exposed to 1 event, and 318 (3.6%) were exposed to more than 1 event (Table 3). Compared with unexposed patients, those with exposure to prior events were younger (77 vs 72 years [exposed to 1 event] or vs 71 years [exposed to >1 event]), had similar sex distribution (55.0% vs 56.4% male [exposed to 1 event] or vs 58.5% male [exposed to >1 event]; 45.0% vs 43.6% female [exposed to 1 event] or vs 41.5% female [exposed to >1 event]), had similar Charlson Comorbidity Index scores, had lower LAPS values calculated at hospital admission (median score, 24 [IQR, 12-38] vs 21 [IQR, 11-34] for those exposed to 1 event or vs 19 [IQR, 11-30] for those exposed to >1 event), and were more likely to have laboratory test results measured at 24 and 48 hours before the outcome (laboratory results unavailable at 24 hours before outcome, 3189 of 7381 [43.2%] unexposed vs 367 of 1086 [33.8%] exposed to 1 event or vs 92 of 318 [28.9%] exposed to >1 event; laboratory results unavailable at 48 hours before outcome, 2140 of 7381 [29.0%] unexposed vs 239 of 1086 [22.0%] exposed to 1 event or vs 45 of 318 [14.2%] exposed to >1 event). Measured LAPS values were lower 24 hours before the outcome (median LAPS, 22 [IQR, 11-43]) among individuals exposed to multiple prior events than among those with no exposure (median LAPS, 27 [IQR, 13-45]). There was no apparent imbalance between groups in day or time of hospital admission, duration of emergency department boarding time, or GIM census ratio on the day of the event. The primary outcome appeared more likely to occur in the nighttime after exposures (4204 of 7381 [57.0%] unexposed, 720 of 1086 [66.3%] exposed to 1 event, and 201 of 318 [63.2%] exposed to >1 event) (Table 3).

Table 3. Patient Characteristics Among Those Who Experienced Outcome, Categorized by Exposure in the 6 Hours Before the Event.

Patient characteristic	Exposed to 0 events	Exposed to 1 event	Exposed to >1 event	SMD
No. of admissions	7381	1086	318	NA
Age, median (IQR), y	77 (63-87)	72 (59-83)	71 (58-82)	0.170
Sex, No. (%)
Male	4057 (55.0)	612 (56.4)	186 (58.5)	0.047
Female	3324 (45.0)	474 (43.6)	132 (41.5)	0.047
Charlson Comorbidity Index score, No. (%)^a
0	1469 (19.9)	217 (20.0)	59 (18.6)	0.075
1	828 (11.2)	117 (10.8)	46 (14.5)
≥2	5084 (68.9)	752 (69.2)	213 (67.0)
Most common discharge diagnoses, No. (%)
Heart failure	406 (5.5)	48 (4.4)	13 (4.1)	0.044
Pneumonia	295 (4.0)	36 (3.3)	12 (3.8)	0.024
COPD	142 (1.9)	25 (2.3)	10 (3.1)	0.052
Urinary tract infection	104 (1.4)	8 (0.7)	5 (1.6)	0.052
GI hemorrhage	96 (1.3)	7 (0.6)	3 (0.9)	0.045
GIM census ratio, mean of all periods (SD)^b	1.0 (0.1)	1.0 (0.1)	1.0 (0.1)	0.028
LAPS at admission, median (IQR)^c	24 (12-38)	21 (11-34)	19 (11-30)	0.136
LAPS 24 h before outcome, median (IQR)^c	27 (13-45)	30 (16-50)	22 (11-43)	0.124
Laboratory results unavailable 24 h before outcome, No. (%)	3189 (43.2)	367 (33.8)	92 (28.9)	0.200
LAPS 48 h before outcome, median (IQR)^c	33 (18-52)	34 (20-56)	29 (16-49)	0.089
Laboratory results unavailable 48 h before outcome, No. (%)	2140 (29.0)	239 (22.0)	45 (14.2)	0.244
Weekend admission, No. (%)	1865 (25.3)	268 (24.7)	85 (26.7)	0.031
Night admission, No. (%)	5518 (74.8)	806 (74.2)	240 (75.5)	0.019
Weekend event, No. (%)	2046 (27.7)	256 (23.6)	88 (27.7)	0.063
Night event, No. (%)	4204 (57.0)	720 (66.3)	201 (63.2)	0.129
ED boarding time, median (IQR), h	12.6 (8.7-20.8)	12.9 (8.5-20.8)	11.8 (7.9-19.1)	0.076
Time from admission to outcome, median (IQR), h	133.6 (55.1-300.7)	127.3 (53.9-342.3)	121.5 (45.7-358.2)	0.051
Season, No. (%)
Spring	1884 (25.5)	296 (27.3)	78 (24.5)	0.098
Summer	1865 (25.3)	281 (25.9)	71 (22.3)
Fall	1865 (25.3)	252 (23.2)	79 (24.8)
Winter	1767 (23.9)	257 (23.7)	90 (28.3)

Open in a new tab

^{^a}

Scores range from 0 to 24, with higher scores reflecting greater comorbidity and serving as a validated predictor of in-hospital mortality.²²

^{^b}

Defined as the GIM census on every day of the study period divided by the median GIM census of that year.

^{^c}

Among patients transferred to the ICU, LAPS values at admission appeared somewhat lower for those exposed to more than 1 prior event (median score, 19 [IQR, 11-30]) compared with the other groups (median score for those exposed to 1 event, 20 [IQR, 8-34]; median score for those exposed to 0 events, 23 [IQR, 12-37]) (eTable 1 in Supplement 1). An ICU transfer occurred more often at night after exposures (1600 of 3179 patients [50.3%] unexposed, 410 of 656 [62.5%] exposed to 1 event, and 141 of 227 [62.1%] exposed to >1 event). Patients transferred to the ICU after exposure to more than 1 event appeared less likely to receive invasive mechanical ventilation on the day of transfer (33 of 227 [14.5%]) compared with unexposed patients (686 of 3179 [21.6%]) or those exposed to 1 event (149 of 656 [22.7%]). In-hospital mortality after ICU transfer occurred in 222 of 3179 unexposed patients (7.0%), 49 of 656 patients exposed to 1 event (7.5%), and 18 of 227 patients exposed to greater than 1 event (7.9%). After adjusting for patient and situational factors, the risk of in-hospital mortality after ICU transfer was not significantly greater among exposed patients compared with unexposed ones (1 event: AOR, 0.87; 95% CI, 0.61-1.24; >1 event: AOR, 0.84; 95% CI, 0.48-1.48).

Sensitivity Analyses

When the intervals were extended to 12 or 24 hours, there were no significant associations between exposures and outcomes after adjustment (eTable 2 in Supplement 1). Our main results were consistent after exclusion of 1094 hospitalizations involving a surgical procedure (eTable 3 in Supplement 1) and after exclusion of deaths coded as palliative (eTable 4 in Supplement 1).

Negative Control Analysis

In the negative control analysis, the association between critical illness events was substantially attenuated. There was no significant association between critical illness events on one GIM ward and outcomes on another GIM ward after adjustment (Table 2).

Discussion

In this multicenter study, we found that patients were more likely to die or be transferred to the ICU after another critical illness event (ICU or death) on the same GIM ward. Our findings identified a significant association with ICU transfers but not deaths, and the association was more pronounced after 2 or more events occurred on the same ward. This association was evident across 6-hour intervals, which permitted a maximum of 12 hours between exposures and outcomes. The association was not observed across 12- or 24-hour intervals and was attenuated when events on different GIM wards were linked within the same hospital. This strengthens the hypothesis that ward-level factors were most important for explaining the association between clustered critical illness events because hospital-related factors, time-related factors, or both would likely result in a similar association between events on different GIM wards.

Several hypotheses may explain why critical illness events cluster in location and time. First, events may be associated with common causes such as excess patient volume in a hospital. We made numerous efforts to control for such factors and conducted a negative control analysis to assess how unmeasured hospital- and time-varying factors, such as critical care capacity, might have affected our observations. Our findings suggest that the elevated risk likely exists beyond such causes and is more likely explained by ward-level associations. Second, the initial event may divert resources and attention from other patients, leading to delays or lapses in care that result in critical illness. There is a well-established association between nurse staffing and mortality on inpatient wards,^16,18 highlighting that resource diversion could plausibly have this effect. Third, in the face of critical illness events, clinicians might be more cautious in the care of subsequent patients, resulting in preemptive or proactive ICU transfers. These decisions may be deliberate and appropriate. For example, patients might be preemptively transferred to the ICU for safer care if a ward is unable to care for multiple deteriorating patients. However, such decisions could also be subconsciously influenced by cognitive biases,^31,32,33 wherein the initial critical illness event prompts clinicians to overreact regarding subsequent patients. These theories are not mutually exclusive.

Our findings offer preliminary insights but do not definitively explain the observed clustering of critical illness events on GIM wards. Patients who experienced outcomes after exposure to prior critical illness events were more likely to have laboratory tests measured in the 24 hours before the event than patients who experienced outcomes without exposure, whereas resource diversion to the initial event might have resulted in less investigation of other patients. Patients who were transferred to the ICU after exposure to multiple prior events had less laboratory-measured physiologic derangement than patients with ICU transfers and no exposure, were less likely to receive invasive mechanical ventilation, were more likely to be transferred at nighttime, and did not have increased mortality. These findings could be consistent with earlier recognition of deteriorating patients and preemptive transfers to the ICU. Poorer staffing at nighttime could be associated with an increased need for preemptive ICU transfers. However, this interpretation is speculative. Additional research is needed to understand why critical illness events tend to cluster on a medical ward and whether some of these events might be preventable. Studies have shown that only a small subset of ICU transfers are considered preventable, such as those caused by incorrect triage from the emergency department.^6,34,35

Our results are consistent with findings from a single-center study of 83 723 admissions at the University of Chicago, in which cardiac arrest or ICU transfer was more common in the 6 hours after a prior event, with AORs of 1.18 and 1.53 for exposure to 1 or more than 1 event, respectively.¹⁵ We found that the AORs for death or ICU transfer were 1.39 and 1.49 after exposure to 1 or more than 1 event, respectively. The Chicago study showed that the association between critical illness events persisted at a 12-hour window, but we did not find the same. We extend the previous literature by adding a negative control analysis to control for hospital- and time-level factors and by examining differences in patient and event characteristics that might provide insights into the observed clustering of critical illness events on medical wards.

Limitations

This study has limitations. First, it was conducted only in academic centers, which may limit generalizability. Our findings were generally consistent across 5 hospitals and with the previously published study in an academic center from a different country,¹⁵ suggesting these findings may generalize across academic centers. Second, patient vital signs were not consistently recorded in electronic format during the study period (many hospitals still used paper charting), limiting our description of patient acuity to laboratory-based measures. Third, we were not able to measure important potential confounders, including patient goals of care and ICU capacity. We would not expect clustering associated with patient preferences, and if that affected the association, we might expect clustering at 12- or 24-hour intervals, which was not observed. Capacity of ICUs is not defined by beds alone but also by dynamic staffing availability and is difficult to measure. We were able to control for GIM ward occupancy, which we expect would correlate somewhat with ICU capacity. The attenuation of temporal association between events on different GIM wards suggests ICU capacity was not the major explanation for our findings. Fourth, we were unable to measure nurse staffing, which could explain ward-level associations at given periods. Further research should consider nurse staffing as an important, and potentially modifiable, explanation. Fifth, we were unable to reliably capture cardiac arrests. Typically, cardiac arrest would result in either survival and transfer to the ICU or death without transfer; thus, it is unlikely that we missed an important number of critical illness events. Sixth, we were unable to distinguish expected from unexpected deaths, although our results were consistent when deaths coded as palliative were excluded.

Conclusions

Patients are more likely to experience critical illness events within 12 hours of other patients’ critical illness events on the same GIM ward. This association was observed for ICU transfers but not deaths without ICU transfer. Future research should seek to understand why ICU transfers tend to cluster and how these might inform efforts to improve patient safety.

Supplement 1.

eTable 1. Patient Characteristics Among Those Who Had a Transfer to ICU, Categorized by Exposure in the 6 Hours Prior to the Event

eTable 2. Regression Models of Primary and Secondary Outcomes, 12-Hour and 24-Hour Sensitivity Analysis

eTable 3. Regression Models of Primary and Secondary Outcomes, Including and Excluding Patients Who Underwent Surgery

eTable 4. Regression Models of Primary and Secondary Outcomes, Including and Excluding Deaths Coded as Palliative

eFigure 1. Analytic Design

eFigure 2. Schematic of Negative Control Analysis

eFigure 3. Unadjusted Rates of Critical Illness Events Categorized by Exposure to Prior Events, Separated by Hospital

eFigure 4. Regression Models of Primary and Secondary Outcomes, Fit Separately for Each Hospital

Click here for additional data file.^{(591.8KB, pdf)}

Supplement 2.

Data Sharing Statement

Click here for additional data file.^{(14.2KB, pdf)}

References

1.Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8(1):13-19. doi: 10.1002/jhm.1979 [DOI] [PubMed] [Google Scholar]
2.Lyons PG, Klaus J, McEvoy CA, Westervelt P, Gage BF, Kollef MH. Factors associated with clinical deterioration among patients hospitalized on the wards at a tertiary cancer hospital. J Oncol Pract. 2019;15(8):e652-e665. doi: 10.1200/JOP.18.00765 [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Kipnis P, Turk BJ, Wulf DA, et al. Development and validation of an electronic medical record–based alert score for detection of inpatient deterioration outside the ICU. J Biomed Inform. 2016;64:10-19. doi: 10.1016/j.jbi.2016.09.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Gold CA, Mayer SA, Lennihan L, Claassen J, Willey JZ. Unplanned transfers from hospital wards to the neurological intensive care unit. Neurocrit Care. 2015;23(2):159-165. doi: 10.1007/s12028-015-0123-z [DOI] [PubMed] [Google Scholar]
5.Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224-230. doi: 10.1002/jhm.964 [DOI] [PubMed] [Google Scholar]
6.Dahn CM, Manasco AT, Breaud AH, et al. A critical analysis of unplanned ICU transfer within 48 hours from ED admission as a quality measure. Am J Emerg Med. 2016;34(8):1505-1510. doi: 10.1016/j.ajem.2016.05.009 [DOI] [PubMed] [Google Scholar]
7.Linnen DT, Escobar GJ, Hu X, Scruth E, Liu V, Stephens C. Statistical modeling and aggregate-weighted scoring systems in prediction of mortality and ICU transfer: a systematic review. J Hosp Med. 2019;14(3):161-169. doi: 10.12788/jhm.3151 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649-655. doi: 10.1164/rccm.201406-1022OC [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Liu VX, Lu Y, Carey KA, et al. Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit. JAMA Netw Open. 2020;3(5):e205191. doi: 10.1001/jamanetworkopen.2020.5191 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Pimentel MAF, Redfern OC, Malycha J, et al. Detecting deteriorating patients in the hospital: development and validation of a novel scoring system. Am J Respir Crit Care Med. 2021;204(1):44-52. doi: 10.1164/rccm.202007-2700OC [DOI] [PMC free article] [PubMed] [Google Scholar]
11.McGinley A, Pearse RM. A national early warning score for acutely ill patients. BMJ. 2012;345:e5310. doi: 10.1136/bmj.e5310 [DOI] [PubMed] [Google Scholar]
12.Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med. 2020;383(20):1951-1960. doi: 10.1056/NEJMsa2001090 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Muralitharan S, Nelson W, Di S, et al. Machine learning–based early warning systems for clinical deterioration: systematic scoping review. J Med Internet Res. 2021;23(2):e25187. doi: 10.2196/25187 [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Nestor B, McCoy LG, Verma AA, et al. Preparing a clinical support model for silent mode in general internal medicine. Published 2020. Accessed May 6, 2022. https://static1.squarespace.com/static/59d5ac1780bd5ef9c396eda6/t/5f22ccecc4a74012a31d4b4a/1596116209243/155_CameraReadySubmission_155_nestor2020preparing.pdf
15.Volchenboum SL, Mayampurath A, Göksu-Gürsoy G, Edelson DP, Howell MD, Churpek MM. Association between in-hospital critical illness events and outcomes in patients on the same ward. JAMA. 2016;316(24):2674-2675. doi: 10.1001/jama.2016.15505 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of registered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care. 2007;45(12):1195-1204. doi: 10.1097/MLR.0b013e3181468ca3 [DOI] [PubMed] [Google Scholar]
17.Elliott DJ, Young RS, Brice J, Aguiar R, Kolm P. Effect of hospitalist workload on the quality and efficiency of care. JAMA Intern Med. 2014;174(5):786-793. doi: 10.1001/jamainternmed.2014.300 [DOI] [PubMed] [Google Scholar]
18.Needleman J, Liu J, Shang J, Larson EL, Stone PW. Association of registered nurse and nursing support staffing with inpatient hospital mortality. BMJ Qual Saf. 2020;29(1):10-18. doi: 10.1136/bmjqs-2018-009219 [DOI] [PubMed] [Google Scholar]
19.Verma AA, Guo Y, Kwan JL, et al. Patient characteristics, resource use and outcomes associated with general internal medicine hospital care: the General Medicine Inpatient Initiative (GEMINI) retrospective cohort study. CMAJ Open. 2017;5(4):E842-E849. doi: 10.9778/cmajo.20170097 [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Verma AA, Pasricha SV, Jung HY, et al. Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience. J Am Med Inform Assoc. 2021;28(3):578-587. doi: 10.1093/jamia/ocaa225 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Canadian Institute for Health Information . Care in Canadian ICUs. CIHI; 2016. [Google Scholar]
22.Quan H, Li B, Couris CM, et al. Updating and validating the Charlson Comorbidity Index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173(6):676-682. doi: 10.1093/aje/kwq433 [DOI] [PubMed] [Google Scholar]
23.Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232-239. doi: 10.1097/MLR.0b013e3181589bb6 [DOI] [PubMed] [Google Scholar]
24.van Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2010;63(7):798-803. doi: 10.1016/j.jclinepi.2009.08.020 [DOI] [PubMed] [Google Scholar]
25.Eriksson CO, Stoner RC, Eden KB, Newgard CD, Guise JM. The association between hospital capacity strain and inpatient outcomes in highly developed countries: a systematic review. J Gen Intern Med. 2017;32(6):686-696. doi: 10.1007/s11606-016-3936-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Zannella VE, Jung HY, Fralick M, et al. Bedspacing and clinical outcomes in general internal medicine: a retrospective, multicenter cohort study. J Hosp Med. 2022;17(1):3-10. doi: 10.1002/jhm.2734 [DOI] [PubMed] [Google Scholar]
27.Bai AD, Srivastava S, Tomlinson GA, Smith CA, Bell CM, Gill SS. Mortality of hospitalised internal medicine patients bedspaced to non-internal medicine inpatient units: retrospective cohort study. BMJ Qual Saf. 2018;27(1):11-20. doi: 10.1136/bmjqs-2017-006925 [DOI] [PubMed] [Google Scholar]
28.Roberts SB, Colacci M, Razak F, Verma AA. A simplification of the Kaiser Permanente inpatient risk adjustment methodology accurately predicted in-hospital mortality: a retrospective cohort study. medRxiv. Published online January 9, 2023. doi: 10.1101/2023.01.06.23284273 [DOI] [PMC free article] [PubMed]
29.Austin PC. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun Stat Simul Comput. 2009;38(6):1228-1234. doi: 10.1080/03610910902859574 [DOI] [Google Scholar]
30.Austin PC. A tutorial on multilevel survival analysis: methods, models and applications. Int Stat Rev. 2017;85(2):185-203. doi: 10.1111/insr.12214 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.O’Sullivan ED, Schofield SJ. Cognitive bias in clinical medicine. J R Coll Physicians Edinb. 2018;48(3):225-232. doi: 10.4997/jrcpe.2018.306 [DOI] [PubMed] [Google Scholar]
32.Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: a systematic review. BMC Med Inform Decis Mak. 2016;16(1):138. doi: 10.1186/s12911-016-0377-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Lighthall GK, Vazquez-Guillamet C. Understanding decision making in critical care. Clin Med Res. 2015;13(3-4):156-168. doi: 10.3121/cmr.2015.1289 [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68-72. doi: 10.1002/jhm.812 [DOI] [PubMed] [Google Scholar]
35.Wang J, Hahn SS, Kline M, Cohen RI. Early in-hospital clinical deterioration is not predicted by severity of illness, functional status, or comorbidity. Int J Gen Med. 2017;10:329-334. doi: 10.2147/IJGM.S145933 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1.

eTable 1. Patient Characteristics Among Those Who Had a Transfer to ICU, Categorized by Exposure in the 6 Hours Prior to the Event

eTable 2. Regression Models of Primary and Secondary Outcomes, 12-Hour and 24-Hour Sensitivity Analysis

eTable 3. Regression Models of Primary and Secondary Outcomes, Including and Excluding Patients Who Underwent Surgery

eTable 4. Regression Models of Primary and Secondary Outcomes, Including and Excluding Deaths Coded as Palliative

eFigure 1. Analytic Design

eFigure 2. Schematic of Negative Control Analysis

eFigure 3. Unadjusted Rates of Critical Illness Events Categorized by Exposure to Prior Events, Separated by Hospital

eFigure 4. Regression Models of Primary and Secondary Outcomes, Fit Separately for Each Hospital

Click here for additional data file.^{(591.8KB, pdf)}

Supplement 2.

Data Sharing Statement

Click here for additional data file.^{(14.2KB, pdf)}

[ioi230041r1] 1.Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8(1):13-19. doi: 10.1002/jhm.1979 [DOI] [PubMed] [Google Scholar]

[ioi230041r2] 2.Lyons PG, Klaus J, McEvoy CA, Westervelt P, Gage BF, Kollef MH. Factors associated with clinical deterioration among patients hospitalized on the wards at a tertiary cancer hospital. J Oncol Pract. 2019;15(8):e652-e665. doi: 10.1200/JOP.18.00765 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r3] 3.Kipnis P, Turk BJ, Wulf DA, et al. Development and validation of an electronic medical record–based alert score for detection of inpatient deterioration outside the ICU. J Biomed Inform. 2016;64:10-19. doi: 10.1016/j.jbi.2016.09.013 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r4] 4.Gold CA, Mayer SA, Lennihan L, Claassen J, Willey JZ. Unplanned transfers from hospital wards to the neurological intensive care unit. Neurocrit Care. 2015;23(2):159-165. doi: 10.1007/s12028-015-0123-z [DOI] [PubMed] [Google Scholar]

[ioi230041r5] 5.Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224-230. doi: 10.1002/jhm.964 [DOI] [PubMed] [Google Scholar]

[ioi230041r6] 6.Dahn CM, Manasco AT, Breaud AH, et al. A critical analysis of unplanned ICU transfer within 48 hours from ED admission as a quality measure. Am J Emerg Med. 2016;34(8):1505-1510. doi: 10.1016/j.ajem.2016.05.009 [DOI] [PubMed] [Google Scholar]

[ioi230041r7] 7.Linnen DT, Escobar GJ, Hu X, Scruth E, Liu V, Stephens C. Statistical modeling and aggregate-weighted scoring systems in prediction of mortality and ICU transfer: a systematic review. J Hosp Med. 2019;14(3):161-169. doi: 10.12788/jhm.3151 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r8] 8.Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649-655. doi: 10.1164/rccm.201406-1022OC [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r9] 9.Liu VX, Lu Y, Carey KA, et al. Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit. JAMA Netw Open. 2020;3(5):e205191. doi: 10.1001/jamanetworkopen.2020.5191 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r10] 10.Pimentel MAF, Redfern OC, Malycha J, et al. Detecting deteriorating patients in the hospital: development and validation of a novel scoring system. Am J Respir Crit Care Med. 2021;204(1):44-52. doi: 10.1164/rccm.202007-2700OC [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r11] 11.McGinley A, Pearse RM. A national early warning score for acutely ill patients. BMJ. 2012;345:e5310. doi: 10.1136/bmj.e5310 [DOI] [PubMed] [Google Scholar]

[ioi230041r12] 12.Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med. 2020;383(20):1951-1960. doi: 10.1056/NEJMsa2001090 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r13] 13.Muralitharan S, Nelson W, Di S, et al. Machine learning–based early warning systems for clinical deterioration: systematic scoping review. J Med Internet Res. 2021;23(2):e25187. doi: 10.2196/25187 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r14] 14.Nestor B, McCoy LG, Verma AA, et al. Preparing a clinical support model for silent mode in general internal medicine. Published 2020. Accessed May 6, 2022. https://static1.squarespace.com/static/59d5ac1780bd5ef9c396eda6/t/5f22ccecc4a74012a31d4b4a/1596116209243/155_CameraReadySubmission_155_nestor2020preparing.pdf

[ioi230041r15] 15.Volchenboum SL, Mayampurath A, Göksu-Gürsoy G, Edelson DP, Howell MD, Churpek MM. Association between in-hospital critical illness events and outcomes in patients on the same ward. JAMA. 2016;316(24):2674-2675. doi: 10.1001/jama.2016.15505 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r16] 16.Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of registered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care. 2007;45(12):1195-1204. doi: 10.1097/MLR.0b013e3181468ca3 [DOI] [PubMed] [Google Scholar]

[ioi230041r17] 17.Elliott DJ, Young RS, Brice J, Aguiar R, Kolm P. Effect of hospitalist workload on the quality and efficiency of care. JAMA Intern Med. 2014;174(5):786-793. doi: 10.1001/jamainternmed.2014.300 [DOI] [PubMed] [Google Scholar]

[ioi230041r18] 18.Needleman J, Liu J, Shang J, Larson EL, Stone PW. Association of registered nurse and nursing support staffing with inpatient hospital mortality. BMJ Qual Saf. 2020;29(1):10-18. doi: 10.1136/bmjqs-2018-009219 [DOI] [PubMed] [Google Scholar]

[ioi230041r19] 19.Verma AA, Guo Y, Kwan JL, et al. Patient characteristics, resource use and outcomes associated with general internal medicine hospital care: the General Medicine Inpatient Initiative (GEMINI) retrospective cohort study. CMAJ Open. 2017;5(4):E842-E849. doi: 10.9778/cmajo.20170097 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r20] 20.Verma AA, Pasricha SV, Jung HY, et al. Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience. J Am Med Inform Assoc. 2021;28(3):578-587. doi: 10.1093/jamia/ocaa225 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r21] 21.Canadian Institute for Health Information . Care in Canadian ICUs. CIHI; 2016. [Google Scholar]

[ioi230041r22] 22.Quan H, Li B, Couris CM, et al. Updating and validating the Charlson Comorbidity Index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173(6):676-682. doi: 10.1093/aje/kwq433 [DOI] [PubMed] [Google Scholar]

[ioi230041r23] 23.Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk-adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232-239. doi: 10.1097/MLR.0b013e3181589bb6 [DOI] [PubMed] [Google Scholar]

[ioi230041r24] 24.van Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2010;63(7):798-803. doi: 10.1016/j.jclinepi.2009.08.020 [DOI] [PubMed] [Google Scholar]

[ioi230041r25] 25.Eriksson CO, Stoner RC, Eden KB, Newgard CD, Guise JM. The association between hospital capacity strain and inpatient outcomes in highly developed countries: a systematic review. J Gen Intern Med. 2017;32(6):686-696. doi: 10.1007/s11606-016-3936-3 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r26] 26.Zannella VE, Jung HY, Fralick M, et al. Bedspacing and clinical outcomes in general internal medicine: a retrospective, multicenter cohort study. J Hosp Med. 2022;17(1):3-10. doi: 10.1002/jhm.2734 [DOI] [PubMed] [Google Scholar]

[ioi230041r27] 27.Bai AD, Srivastava S, Tomlinson GA, Smith CA, Bell CM, Gill SS. Mortality of hospitalised internal medicine patients bedspaced to non-internal medicine inpatient units: retrospective cohort study. BMJ Qual Saf. 2018;27(1):11-20. doi: 10.1136/bmjqs-2017-006925 [DOI] [PubMed] [Google Scholar]

[ioi230041r28] 28.Roberts SB, Colacci M, Razak F, Verma AA. A simplification of the Kaiser Permanente inpatient risk adjustment methodology accurately predicted in-hospital mortality: a retrospective cohort study. medRxiv. Published online January 9, 2023. doi: 10.1101/2023.01.06.23284273 [DOI] [PMC free article] [PubMed]

[ioi230041r29] 29.Austin PC. Using the standardized difference to compare the prevalence of a binary variable between two groups in observational research. Commun Stat Simul Comput. 2009;38(6):1228-1234. doi: 10.1080/03610910902859574 [DOI] [Google Scholar]

[ioi230041r30] 30.Austin PC. A tutorial on multilevel survival analysis: methods, models and applications. Int Stat Rev. 2017;85(2):185-203. doi: 10.1111/insr.12214 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r31] 31.O’Sullivan ED, Schofield SJ. Cognitive bias in clinical medicine. J R Coll Physicians Edinb. 2018;48(3):225-232. doi: 10.4997/jrcpe.2018.306 [DOI] [PubMed] [Google Scholar]

[ioi230041r32] 32.Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: a systematic review. BMC Med Inform Decis Mak. 2016;16(1):138. doi: 10.1186/s12911-016-0377-1 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r33] 33.Lighthall GK, Vazquez-Guillamet C. Understanding decision making in critical care. Clin Med Res. 2015;13(3-4):156-168. doi: 10.3121/cmr.2015.1289 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi230041r34] 34.Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68-72. doi: 10.1002/jhm.812 [DOI] [PubMed] [Google Scholar]

[ioi230041r35] 35.Wang J, Hahn SS, Kline M, Cohen RI. Early in-hospital clinical deterioration is not predicted by severity of illness, functional status, or comorbidity. Int J Gen Med. 2017;10:329-334. doi: 10.2147/IJGM.S145933 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Temporal Clustering of Critical Illness Events on Medical Wards

Samik Doshi, MD, MSc

Saeha Shin, MPH

Lauren Lapointe-Shaw, MD, PhD

Robert A Fowler, MDCM, MSc

Michael Fralick, MD, PhD

Janice L Kwan, MD, MPH

Kaveh G Shojania, MD

Terence Tang, MD, MSc

Fahad Razak, MD, MSc

Amol A Verma, MD, MPhil

Key Points

Question

Findings

Meaning

Abstract

Importance

Objective

Design, Setting, and Participants

Exposures

Main Outcomes and Measures

Results

Conclusions and Relevance

Introduction

Methods

Design, Setting, and Participants

Figure 1. Study Flow Diagram With Inclusion and Exclusion Criteria.

Data Collection

Exposure

Outcomes and Measures

Covariates

Statistical Analysis

Sensitivity Analyses

Negative Control Analysis

Results

Table 1. Patient Characteristics.

Study Outcomes

Figure 2. Unadjusted Rates of Critical Illness Events, Categorized by Exposure to Prior Events.

Table 2. Regression Models of Primary and Secondary Outcomes.

Characteristics of Events Across Exposure Categories

Table 3. Patient Characteristics Among Those Who Experienced Outcome, Categorized by Exposure in the 6 Hours Before the Event.

Sensitivity Analyses

Negative Control Analysis

Discussion

Limitations

Conclusions

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases