Postoperative Overtriage to an Intensive Care Unit is Associated with Low Value of Care

Abstract

Objective:

We test the hypothesis that for low-acuity surgical patients, postoperative intensive care unit (ICU) admission is associated with lower value of care compared with ward admission.

Summary of Background Data:

Overtriaging low-acuity patients to ICUs consumes valuable resources and may not confer better patient outcomes. Associations among postoperative overtriage, patient outcomes, costs, and value of care have not been previously reported.

Methods:

In this longitudinal cohort study, postoperative ICU admissions were classified as overtriaged or appropriately triaged according to machine learning-based patient acuity assessments and requirements for immediate postoperative mechanical ventilation or vasopressor support. A nearest neighbors algorithm identified risk-matched control ward admissions. The primary outcome was value of care, calculated as inverse observed-to-expected (O:E) mortality ratios divided by total costs.

Results:

Acuity assessments had area under the receiver operating characteristic curve of 0.92 in generating predictions for triage classifications. Of 8,592 postoperative ICU admissions, 423 (4.9%) were overtriaged. These were matched with 2,155 control ward admissions with similar comorbidities, incidence of emergent surgery, immediate postoperative vital signs, and Do Not Resuscitate order placement and rescindment patterns. Compared with controls, overtraiged admissions did not have lower incidence of any measured complications. Total costs for admission were $16.4K for overtriage and $15.9K for controls (P=.03). Value of care was lower for overtriaged admissions (2.9 [2.0–4.0]) compared with controls (24.2 [14.1–34.5], P<.001).

Conclusions:

Low-acuity postoperative patients who were overtriaged to intensive care units had increased total costs, no improvements in outcomes, and received low-value care.

Mini-Abstract

Our objective was to determine whether postoperative overtriage to an intensive care unit is associated with low value of care. In this longitudinal cohort study, overtriage was associated with no improvements in observed-to-expected hospital mortality or other short-term outcomes and had increased total costs and lower value compared with risk-matched ward admissions. These observations provide a rationale and framework for clinical decision support platforms to augment postoperative triage decisions and optimize value of care.

Introduction

Every year, approximately 15 million major, inpatient surgeries are performed in the United States alone.¹ Immediately thereafter, surgeons must decide whether to triage their patient to an intensive care unit (ICU) that offers near-continuous vigilance by personnel trained to prevent and treat organ dysfunction. Triaging low-acuity patients to ICUs consumes valuable resources, costing $2–10K USD per day compared with less than $1K USD per day for a ward (floor) bed, and may not improve outcomes.^2–4 Furthermore, unnecessary ICU surveillance and diagnostic testing may induce harm from unnecessary treatments.⁵ Finally, opportunity cost is impacted: one less ICU bed is available for another patient who may have greater illness severity; denied ICU admission is associated with increased short-term mortality.⁶

Despite the potential consequences of postoperative overtriage, there is no consensus definition of overtriage and there is sparse evidence associating overtriage, patient outcomes, and value of care.⁷ Triage patterns reported in medical and mixed-medical surgical populations may not be generalizable to postoperative patients, who are uniquely vulnerable to hemorrhage, respiratory failure, opiate toxicity, and sepsis.^8–10 Most systems for classifying patient acuity would require further gains in accuracy and integration with clinical and digital workflows before real-world implementation.^{11, 12} Without efficient, effective decision-support platforms to augment postoperative triage decisions, surgeons must rely on individual judgement and hypothetical-deductive reasoning, which may contribute to variability in surgical care costs and failure-to-rescue across US hospitals.^{13, 14}

In a longitudinal, retrospective cohort of patients undergoing major, inpatient surgery, we test the hypothesis that for low-acuity patients, postoperative intensive care unit (ICU) admission is associated with lower value of care compared with ward admission. Overtriaged ICU admissions were identified by a validated machine learning model using automated electronic health record data to generate risk predictions at surgery end time.^{15, 16} We compared patient outcomes, total costs, and value of care among admissions that were appropriately triaged to ICUs, overtriaged to ICUs, and risk-matched control ward admissions.

Methods

Study Design

We generated a longitudinal, retrospective cohort of all subjects aged 18 years or greater who underwent inpatient surgery at the University of Florida Health. ICU admissions were considered overtriaged if they received no immediate postoperative mechanical ventilation or vasopressor support and had predicted risk for hospital mortality and prolonged ICU stay (48h or greater) below median risk thresholds among all inpatient surgeries. Risk was predicted according to a validated, random forest machine learning model using pre- and intraoperative features extracted directly from electronic health records, generating predictions at surgery end time; we applied risk thresholds to these predictions.^{15, 16} A nearest neighbors algorithm identified risk-matched control ward admissions. The primary analysis compared O:E mortality, total costs (i.e., total dollars of direct costs incurred by the hospital in providing care per single hospital admission), and value of care among admissions that were appropriately triaged to ICUs, overtriaged to ICUs, and risk-matched control ward admissions. The University of Florida Institutional Review Board approved this study (IRB# 201802284). This study complies with Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines (see table, Supplemental Digital Content 1, our completed STROBE checklist).

Data Source

We used the University of Florida Integrated Data Repository as an honest broker in assembling a study cohort of 71,065 hospital admissions that included inpatient surgery at University of Florida Health in Gainesville, FL during a 6-year period ending August 20^th, 2020. Supplemental Digital Content 2 illustrates derivation of the study population. Briefly, we excluded anesthesia outside the operating room, organ donation surgery, and admissions for <24 hours, which are often considered “observation only” and may not accurately represent major, inpatient surgery. To avoid excessively narrow prediction windows, we excluded deaths within 24 hours. We split the study cohort into development and validation cohorts chronologically to minimize the impact of dataset drift on predictions according to the Type 2b analysis category of Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) recommendations.^{17, 18} The final dataset contained granular pre- intra- and postoperative information.^{15, 16} Supplemental Digital Content 3 describes and lists all data elements, model input features, preprocessing methods, and model development and validation.

The study design and data source mirror previous work by the authors describing postoperative undertriage to hospital wards. Briefly, among 2,155 risk-matched control ward admissions reported herein, 97% were drawn from a previously reported cohort of 11,042 appropriately triaged ward admissions; 3% were drawn from a previously reported cohort of 1,306 undertriaged ward admissions. The 3% overlap is an artifact of unsupervised nearest neighbor matching.

Identifying Overtriaged ICU Admissions

This study uses observed provision of mechanical ventilation or vasopressor support as well as predicted risk for hospital mortality and prolonged ICU stay to identify postoperative overtriage. This approach differs from previously described methods for identifying overtriaged medical patients using risk for 30-day mortality. The rationale for using expanded classification criteria in the present study is that advanced organ support and prolonged critical illness suggest that ICU admission is appropriate, and therefore warrant inclusion in overtriage classification systems.¹⁹

ICU admissions were considered appropriate if they included mechanical ventilation or vasopressor administration within 2 hours of surgery end time, consistent with guidelines that ICU admission is indicated for patients requiring advanced organ support.^{20, 21} Other factors, such as admitting services desiring continuous, titratable medication infusions or frequent pulse checks or neurologic exams that preclude ward admission, may also affect postoperative triage decisions but are not represented in our EHR consistently and reliably and were therefore not used to identify overtriage, which allowed for such admissions to be classified as overtriaged. In identifying admissions with low risk for hospital mortality, the median risk threshold was chosen to maintain consistency with peer reviewed literature; in our cohort, the median risk threshold for hospital mortality corresponded with 2% in-hospital mortality.^{19, 22–24} Regarding risk for prolonged ICU stay, there is no precedent in peer-reviewed literature; the median risk threshold was chosen to maintain consistency with the hospital mortality cutoff. In the US, there are approximately 9 to 10 ICU beds per 100 hospital beds.²⁵ Therefore, it may be reasonable to classify all patients with risk for hospital mortality and prolonged ICU stay less than 90^th percentile as overtriaged. However, this approach could encourage undertriaging of high-risk patients to hospital wards, where postoperative complications may progress to critical illness and cardiac arrest in between less frequent patient assessments. To minimize undertriage, which has strong associations with adverse events, we chose the more conservative median cutoff.^{22, 23, 26–28}

Identifying Matched Control Ward Admissions

We identified a control cohort of risk-matched postoperative ward admissions by fitting a nearest neighbors brute-force algorithm (https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.NearestNeighbors.html) on predicted risk for hospital mortality and prolonged ICU stay among ward admissions, using the same risk predictions that were used to identify overtriaged ICU admissions. We then applied the fit model to overtriaged admissions using a parameter space radius of 1 and a Minkowski distance metric with p=2. To account for the small number of overtriaged admissions and the different distributions of predicted risk values for hospital mortality and prolonged ICU stay, we performed three separate searches that each identified the two nearest neighbor (k=2) ward admissions according to predicted risk for (1) hospital mortality, (2) prolonged ICU stay, and (3) composite risk for hospital mortality or prolonged ICU stay, which was represented by the average z-score across both outcomes. The union of these three searches matched 2,155 control admissions with 423 overtriaged admissions. Supplemental Digital Content 4 summarizes five other permutations of the clustering approach using different K-values and distance metrics. The primary analysis permutation was chosen to generate the largest sample of control admissions while maintaining clinical characteristics that were similar to those of the overtriage cohort.

Hospital Stations

We extracted hospital station entrance and exit datetimes from EHRs. ICU admissions included intensive and intermediate status, with patient-to-nurse ratios ≤2:1 or ≤3:1, respectively. In this study, the ICU was considered as a single entity that provides low patient-to-nurse ratios, specialized critical care resources and personnel, and daily rounds by a critical care attending. Grouping intensive and intermediate status ICU patients into a single ICU cohort allowed accurate representation of total ICU bed occupancy, which has important implications for ICUs reaching full bed capacity and resulting increased mortality for those denied ICU admission.⁶

Statistical Analysis

We are unaware of any prior reports associating overtriage with value of care. Therefore, we performed a power analysis using associations between postoperative patient acuity and raw mortality rates. In a study by Cutti et al.²⁹, 35,931 hospitalizations with grade 4 surgical complexity had 5.18% in-hospital mortality, representing a high-risk cohort according to peer-reviewed literature; 13,484 hospitalizations with grade 1 surgical complexity had 0.36% in-hospital mortality.^{22–24, 29} Assuming 80% power, alpha=.05, imbalanced cohorts, and hospital mortality ≥2.70% for appropriately triaged ICU admissions, our sample would detect a 1.70% difference in hospital mortality. We performed the power analyses with PASS software (NCSS, Kaysville, Utah).

For training the prediction model, we replaced missing values with a distinct “missing” indicator; for reporting results, we imputed missing values with medians. Continuous variables are reported as median values with interquartile ranges and compared by the Kruskal-Wallis test. Discrete variables are reported as raw numbers with percentages and compared by Fisher’s Exact test. O:E mortality ratios were calculated by dividing observed mortality by estimated mortality according to our prediction model in the primary analysis and also according to American Society of Anesthesiologists (ASA) Physical Status Classification scores in a secondary analysis, using previously reported associations between ASA score and 30-day mortality in a random sample of major General, Neurologic, Orthopedic, Plastic, Thoracic, and Vascular Surgery procedures.³⁰ O:E mortality ratios were compared by Fisher’s Exact test, and presented with odds ratios and 95% confidence intervals. To calculate value of care, inverted O:E mortality ratios were divided by median total costs and multiplied by a constant to set value of care for the entire study population to 1, equating to [(1 / O:E mortality) / total cost] × 23,957.³¹ Sensitivity analyses were performed by reassessing associations between triage classifications and patient outcomes when excluding all cases performed by Plastic and Reconstructive Surgery or Otolaryngology, as these services are responsible for most cases requiring hourly tissue flap nursing assessments that may be associated with ICU transfer in the absence of organ dysfunction or high risk for adverse outcomes, and also excluding all cases performed by Burn Surgery, as our prediction model does not consider the percentage of total body surface area involved by burn wounds to generate predictions. All outcome analyses were adjusted for multiple comparisons using the Benjamini-Hochberg procedure. Model performance was assessed by calculating area under receiver operating characteristic curves (AUROC), precision-recall curves (AUPRC), sensitivity, specificity, positive predictive value, and negative predictive value with 95% confidence intervals (CI). All statistical tests were 2-sided with alpha=.05. We performed modeling and statistical analyses using Python 3.8.8 software.

Results

Study Population Characteristics

Table 1 and Supplemental Digital Content 5 summarize patient characteristics. Relative to the higher-risk appropriate ICU triage cohort, the lower-risk overtriage cohort had lesser illness severity and risk for adverse events, manifested as lower Charlson comorbidity index scores (1.0 vs. 2.0) and lesser proportions of admissions via the emergency department (21% vs. 37%), undergoing emergent surgery (14.2% vs. 24.9%), and receiving red cell transfusions before surgery (0.2% vs. 6.6%) and during surgery (0.0% vs. 1.4%, all P<.001). Compared with the control ward admission cohort, the overtriage cohort had a lesser proportion of admissions from the emergency department (21% vs. 33%, P<.001) but a higher distribution of ASA scores (3.0 [3.0–3.0] vs. 3.0 [2.0–3.0], P=.003); other illness severity indicators were similar, including the incidence of chronic heart, lung, liver, and kidney disease. Supplemental Digital Content 6 lists primary surgical services. Supplemental Digital Content 7 lists top 10 most common primary procedural codes for each cohort.

Table 1.

Characteristics of postoperative admissions appropriately triaged to intensive care units, overtriaged to intensive care units, or controls triaged to hospital wards with risk profiles matching the overtriage cohort.

Characteristics	Appropriate triage N=8,169 No (%)	Overtriage N=423 No (%)	Controls N=2,155 No (%)	Pa	Pb
Demographics
Female	3,479 (42.6)	198 (46.8)	1152 (53.5)	.10	.01
Age (years), median [IQR]	63.0 [51.0–72.0]	56.0 [41.0–69.0]	58.0 [43.0–70.0]	<.001	.29
ADI national rank, median [IQR]	68.0 [46.0–84.0]	68.0 [46.0–84.0]	69.0 [46.0–85.0]	.56	.75
ADI state rank, median [IQR]	7.0 [5.0–9.0]	7.0 [5.0–9.0]	7.0 [4.0–9.0]	.85	.97
Illness Severity
ASA class, median [IQR]	3.0 [3.0–4.0]	3.0 [3.0–3.0]	3.0 [2.0–3.0]	<.001	.003
Congestive heart failure	1581 (19.4)	28 (6.6)	156 (7.2)	<.001	.76
Chronic obstructive pulmonary disease	2223 (27.2)	90 (21.3)	442 (20.5)	.008	.69
Moderate-severe liver disease	128 (1.6)	1 (0.2)	7 (0.3)	.02	>.99
End-stage renal disease	332 (4.1)	13 (3.1)	84 (3.9)	.37	.49
Charlson comorbidity index, median [IQR]	2.0 [0.0–3.0]	1.0 [0.0–2.0]	1.0 [0.0–2.0]	<.001	.20
Admitted from emergency department	3,026 (37.0)	89 (21.0)	714 (33.1)	<.001	<.001
Had preoperative red cell transfusion	543 (6.6)	1 (0.2)	22 (1.0)	<.001	.16
Had intraoperative red cell transfusion	114 (1.4)	0 (0.0)	3 (0.1)	.007	>.99
Surgery priority
Elective	5,753 (70.4)	348 (82.3)	1746 (81.0)	<.001	.59
Urgent	384 (4.7)	15 (3.5)	121 (5.6)	.34	.10
Emergent	2,032 (24.9)	60 (14.2)	288 (13.4)	<.001	.64

IQR: interquartile range; ADI: area deprivation index, ASA: American Society of Anesthesiologists. P values correspond to significance tests comparing the appropriate triage, overtriage, and control cohorts by each variable listed in the “Characteristics” column.

^aAppropriate triage versus overtriage.

^bOvertriage versus controls.

Triage Classifications

In the validation cohort, hospital mortality predictions had AUROC .92 (95% CI .91–.93), AUPRC .26 (95% CI .23–.30), sensitivity .89 (95% CI .80–.92), specificity .81 (95% CI .78–.90), positive predictive value .08 (95% CI .07–.13), and negative predictive value .99 (95% CI .99–.99); prolonged ICU stay predictions had AUROC .92 (95% CI .92–.92), AUPRC .85 (95% CI .85–.86), sensitivity .85 (95% CI .83–.87), specificity .83 (95% CI .81–.86), positive predictive value .67 (95% CI .65–.70), and negative predictive value .93 (95% CI .92–.94). Model AUROC and AUPRC were similar across all racial categories containing 10 or more subjects, as listed in Supplemental Digital Content 8. According to these predictions, and when considering immediate postoperative mechanical ventilation or vasopressor administration as indicating appropriate ICU admission, 8,169 postoperative ICU admissions were appropriate and 423 (4.9%) were overtriaged. The overtriage cohort was matched with 2,155 control postoperative ward admissions according to risk for hospital mortality and prolonged ICU stay. Supplemental Digital Content 9 and 10 list the 20 most important features for predicting hospital mortality and prolonged ICU stay, respectively. Primary procedure, scheduled postoperative location, and intraoperative minimum alveolar concentration measurements or inhalational anesthetic duration were among the 5 most important features for both predictions.

Perioperative Factors

Table 2 summarizes perioperative factors. Sixty-eight percent of the overtriage cohort was originally planned for postoperative triage to hospital wards, compared with 22% of the appropriate triage cohort and 97% of the control cohort (both P<.001). Using vital signs measured within 4 hours of surgery end time, compared with appropriately triaged admissions, overtriaged admissions had lower median heart rate (79.3 vs. 81.9, P<.001), higher systolic (127.8 mmHg vs. 125.0 mmHg, P=.01) and diastolic blood pressure (71.1 mmHg vs. 64.7 mmHg, P<.001), and lower oxygen saturation (95.9% vs. 97.0%, P<.001). Lower oxygen saturation in the overtriage cohort occurred in the context that none of the overtriaged admissions received immediate (within 2 hours) postoperative mechanical ventilation or vasopressor support, according to triage definitions; in the appropriate triage cohort, 36% received immediate postoperative mechanical ventilation and 14% received immediate postoperative vasopressors (both P<.001). There were no significant differences in vital signs between overtriage and control cohorts. Although the proportions of admissions with a Do Not Resuscitate (DNR) order before the first surgery were similar between overtriage (0.2%) and appropriate triage cohorts (1.0%, P=.13), the overtriage cohort had a lower proportion of DNR orders placed anytime during admission (0.7% vs. 5.7%, P<.001). There were no significant differences in DNR order placement or rescindment between overtriage and control cohorts.

Table 2.

Perioperative factors for postoperative admissions appropriately triaged to intensive care, overtriaged to intensive care, or controls triaged to hospital wards with risk profiles matching the overtriage cohort.

Perioperative factors	Appropriate triage N=8,169 No (%)	Overtriage N=423 No (%)	Controls N=2,155 No (%)	Pa	Pb
Planned postoperative destination
Intensive care unit, intensive status	5,735 (70.2)	105 (24.8)	48 (2.2)	<.001	<.001
Intensive care unit, intermediate status	598 (7.3)	31 (7.3)	8 (0.4)	>.99	<.001
Post-anesthesia care unit and ward	1,804 (22.1)	287 (67.8)	2094 (97.2)	<.001	<.001
Unknown	32 (0.4)	0 (0.0)	5 (0.2)	.41	>.99
Immediate postoperative vital signs c
Heart rate, median [IQR]	81.9 [71.9–93.2]	79.3 [69.3–89.4]	80.2 [71.3–89.6]	<.001	.25
Systolic blood pressure (mmHg), median [IQR]	125.0 [110.1–141.4]	127.8 [114.8–140.7]	125.8 [114.2–138.1]	.01	.10
Diastolic blood pressure (mmHg), median [IQR]	64.7 [56.8–73.8]	71.1 [61.6–79.8]	70.5 [62.4–77.9]	<.001	.35
Respiratory rate, median [IQR]	16.0 [14.1–18.3]	16.1 [14.5–18.3]	16.1 [14.7–17.7]	.15	.17
Oxygen saturation (%), median [IQR]	97.0 [95.2–98.7]	95.9 [94.5–97.5]	96.1 [94.6–97.5]	<.001	.73
Temperature (Celsius), median [IQR]	37.2 [36.8–37.5]	37.2 [37.1–37.5]	37.2 [37.1–37.4]	<.001	.73
GCS Eye Opening Response, median [IQR]	4.0 [3.4–4.0]	4.0 [4.0–4.0]	4.0 [4.0–4.0]	<.001	.08
Do Not Resuscitate Order status
Order placed before first surgery	84 (1.0)	1 (0.2)	14 (0.6)	.13	.49
Order placed during admission	463 (5.7)	3 (0.7)	22 (1.0)	<.001	.79
Order rescinded during admission	112 (1.4)	0 (0.0)	2 (0.1)	.007	>.99

IQR: interquartile range, GCS: Glasgow Coma Scale. P values correspond to significance tests comparing the appropriate triage, overtriage, and control cohorts by each variable listed in the “Perioperative factors” column.

^aAppropriate triage versus overtriage.

^bOvertriage versus controls.

^cWithin 4 hours of surgery end time.

Resource Use, Patient Outcomes, and Value

Table 3 and Supplemental Digital Content 11 summarize resource use, postoperative procedures, complications, charges, costs, and value of care. Relative to the appropriate triage cohort, the overtriage cohort had significantly less resource use, manifested as shorter length of stay in the ICU (1.3 days vs. 3.8 days) and hospital (3.3 days vs. 7.7 days), lower proportions of prolonged ICU stay (27.7% vs. 70.1%), prolonged mechanical ventilation (0.5% vs. 14.3%), and postoperative red cell transfusion (6.4% vs. 26.0%), as well as lower incidence of postoperative arterial catheter placement (5.0% vs. 15.5%), central venous catheter placement (0.9% vs. 5.3%), bronchoscopy (0.2% vs. 3.0%), and having a second surgery during admission (2.8% vs. 22.6%, all P<.001). Forty-seven percent of overtriaged admissions were transferred subsequently to hospital wards and the overtriage cohort had 0.0 median ICU-free hospital days, suggesting frequent hospital discharge directly from the ICU. Compared with controls, in addition to having longer ICU length of stay, overtriaged admissions had greater incidence of electric cardioversion (0.5% vs. 0.0%, P=.04) and persistent acute kidney injury without renal recovery (5.7% vs. 3.2%, P=.04). Predicted mortality rates for appropriate triage, overtriage, control cohorts were 5.1%, 1.0%, and 1.2%, respectively, such that O:E hospital mortality ratios were 0.75 for appropriate triage (313/416.6), 0.48 (2/4.2) for overtriage, and 0.04 for controls (1/23.7, P=.07 vs. overtriage).

Table 3.

Outcomes for postoperative admissions appropriately triaged to intensive care units, overtriaged to intensive care units, or controls triaged to hospital wards with risk profiles matching the overtriage cohort.

Outcomes	Appropriate triage N=8,169 No (%)	Overtriage N=423 No (%)	Controls N=2,155 No (%)	Pa	Pb
Resource use
ICU days, median [IQR]	3.8 [1.8–7.4]	1.3 [0.9–2.1]	0.0 [0.0–0.0]	<.001	<.001
ICU admission for ≥48 hours	5725 (70.1)	117 (27.7)	31 (1.4)	<.001	<.001
ICU-free hospital days, median [IQR]	1.0 [0.0–4.0]	0.0 [0.0–3.0]	3.0 [2.0–5.0]	<.001	<.001
Mechanical ventilation for ≥48 hours	1167 (14.3)	2 (0.5)	4 (0.2)	<.001	.31
Had transfer from ICU to ward	4823 (59.0)	200 (47.3)	0 (0.0)	<.001	<.001
Hours between end of surgery and ward arrival, median, [IQR]	73.0 [40.0–139.0]	35.0 [25.0–50.2]	2.0 [1.0–3.0]	<.001	<.001
Had postoperative red cell transfusion	2126 (26.0)	27 (6.4)	93 (4.3)	<.001	.11
Had any red cell transfusion	2394 (29.3)	27 (6.4)	106 (4.9)	<.001	.29
Hospital length of stay (days), median [IQR]	7.7 [4.3–13.8]	3.3 [1.5–5.1]	3.2 [1.7–5.3]	<.001	.79
Complications
In-hospital mortality	313 (3.8)	2 (0.5)	1 (0.0)	<.001	.10
Postoperative cardiac arrest	195 (2.4)	1 (0.2)	2 (0.1)	.002	.47
Unplanned reintubation	305 (3.7)	5 (1.2)	7 (0.3)	.006	.05
Acute kidney injury
Rapid reversal	1089 (13.3)	38 (9.0)	92 (4.3)	.02	<.001
Persistent, with renal recovery	685 (8.4)	3 (0.7)	22 (1.0)	<.001	.82
Persistent, without renal recovery	768 (9.4)	24 (5.7)	70 (3.2)	.01	.04
Total costs, $x1,000, median [IQR]	41.7 [26.3–66.0]	16.4 [12.1–24.2]	15.9 [11.1–22.4]	<.001	.03
Value of care, median [IQR]	0.8 [0.5–1.2]	3.1 [2.1–4.2]	35.8 [25.3–51.2]	<.001	<.001

IQR: interquartile range, ICU: intensive care unit. P values were adjusted for multiple comparisons using the Benjamini-Hochberg procedure. P values correspond to significance tests comparing the appropriate triage, overtriage, and control cohorts by each variable listed in the “Outcomes” column.

^aAppropriate triage versus overtriage.

^bOvertriage versus controls.

Compared with risk-matched controls, overtriaged admissions had higher median professional service charges ($15.3K vs. $13.7K, P<.001), similar total charges for admission ($63.4K vs. $64.6K, P=.79), and higher total costs for admission ($16.4K vs. $15.9K, P=.03). The overtriage cohort had significantly lower hospital mortality and total costs than the higher-risk appropriate triage cohort, such that value of care was greater in the overtriage cohort (3.1 [2.1–4.2] vs. 0.8 [0.5–1.2], P<.001). Value of care for overtriaged admissions was significantly less than that of risk-matched controls (35.8 [25.3–51.2], P<.001), as illustrated in Figure 1. Similar differences in value of care between cohorts were observed in secondary analyses using ASA scores to represent expected mortality (overtriage: 8.9 [6.1–12.1], controls: 72.2 [51.1–103.2], p<0.001 vs. overtriage; appropriate triage: 1.2 [0.7–1.9]; p<0.001 vs. overtriage).

Figure 1. Low-acuity postoperative patients who were overtriaged to intensive care had lower value of care relative to risk-matched controls triaged to hospital wards.

O:E mortality: observed-to-expected hospital mortality ratios. To calculate value of care for each admission, inverted O:E mortality ratios for cohorts were divided by total cost for each admission and multiplied by a constant to set value of care for the entire study population to 1, equating to [(1 / O:E mortality) / total cost] × 23,957.

Sensitivity Analyses

Sensitivity analyses excluding cases performed by Plastic and Reconstructive Surgery, Otolaryngology, and Burn Surgery demonstrated associations between triage classifications and patient outcomes that were similar to those observed in the primary analysis, as summarized in Supplemental Digital Content 12. The overtriage cohort had significantly lower hospital mortality and total costs than the higher-risk appropriate triage cohort, such that value of care was greater in the overtriage cohort (2.7 [1.8–3.5] vs. 0.9 [0.5–1.3], P<.001). Value of care for overtriaged admissions was significantly less than that of risk-matched controls (33.5 [24.0–49.9] P<.001).

Discussion

Approximately 5% of all postoperative ICU admissions represented low-acuity, overtriaged patients. Compared with risk-matched control ward admissions, overtriaged admissions had similar risk-adjusted mortality, higher costs, and lower value of care. Full ICUs may increase risk for short-term mortality among other patients who are denied ICU admission, underscoring the potential opportunity costs of overtriage.⁶ Our results are consistent with clinical intuition that intensive monitoring of low-acuity patients incurs greater expense without improving outcomes. Overtriaged admission were accurately identified using automated EHR data from pre- and intraoperative phases of care, suggesting opportunities for data-driven decision-support. These results could be operationalized via EHR advisories and overrides triggered by ICU admission orders placed for low-acuity postoperative patients.

While guidelines and recommendations for ICU admission criteria provide useful frameworks, they lack objectivity, which hinders clinical implementation.^{20, 21} Data-driven methods could address this gap. Chen et al.¹⁹ used a patient-level 30-day mortality risk threshold of 2% or less to identify low-acuity medical patients who were overtriaged to ICUs in Veterans Affairs hospitals; according to this definition, more than half of all ICU admissions were overtriaged. Compared with the 5% overtriaged in our study, this greater incidence may be secondary to the exclusion of immediate postoperative mechanical ventilation and vasopressor administration in the classification process; instead, Chen et al.¹⁹ adjusted mortality risk for illness severity, admitting diagnosis, and ICU occupancy and complexity level. Previous work has suggested disadvantages for making postoperative triage decisions based on procedure type rather than the full spectrum of patient-specific risk factors. In a multicenter analysis of patients undergoing endovascular aortic aneurysm repair, routine postoperative ICU admission was associated with longer length of stay and no difference in complications.³² Our analysis demonstrated a greater proportion of women in the control cohort, relative to both overtriaged and appropriately triaged ICU admissions, consistent with prior work suggesting that women may have lower probability of receiving ICU resources for neurologic or cardiovascular disease.³³ Associations among gender, postoperative triage patterns, and patient outcomes require further investigation.³⁴ We are unaware of previous reports describing risk-adjusted outcomes and costs for postoperative patients who are overtriaged to intensive care, which precludes further comparison of our results with others.

Previous work has established the importance and efficacy of avoiding unnecessary health care resource use. The Choosing Wisely Campaign seeks to reduce expenditures and avoid harm from unnecessary tests and treatments.^{35, 36} Of 110 Choosing Wisely recommendations made by 17 surgical societies, 83 (75%) focus on non-procedural practices (e.g., ancillary imaging, medications, and laboratory tests); many of these recommendations could be fulfilled by streamlining digital workflows.⁵ In a cluster randomized trial, a default EHR order system decreased unnecessary daily imaging during palliative radiotherapy by more than 50%.³⁷ Implementing an EHR order system for postoperative triage recommendations based on patient acuity profiling would require real-time predictions, ideally on a fully automated platform that avoids manual data entry.¹² Our platform was built to meet these criteria.

Results from this study are subject to selection bias and may not be generalizable to other practice settings and surgeon-specific practice patterns. External validation is needed to assess generalizability. We sought to minimize selection bias by including all eligible, consecutive admissions and using an unsupervised, data-driven procedure for matching overtriaged patients with controls. Outcomes among overtriaged admissions may have been adversely affected by risk factors that were hidden from EHRs but apparent to clinicians, as suggested by a higher distribution of ASA scores that invoke potentially informative subjectivity and intuition. It is difficult to ascertain retrospectively why triage decisions were made, but many factors that are represented inconsistently in EHR data, such as desire for continuous, titratable medication infusions rather than intermittent dosing or frequent pulse checks or neurologic exams, could have disproportionately affected patients in the overtraiged cohort, which allows for patients who may have merited close observation in the ICU for their specific clinical needs to be classified as overtriaged. The retrospective data used in this study is unhindered by delays in clinical documentation and recording vital signs; these elements could degrade model performance during real-time implementation. Additionally, value of care was defined as inverse O:E mortality ratios divided by total costs, which does not account for other important patient outcomes, like quality of life.^{31, 38} Qualitative factors affecting human intuition and decision-making are not represented in the objective EHR data presented herein; prospective, qualitative investigation is needed to understand these factors. Finally, given the lack of similar studies in peer-reviewed literature, our power analysis was targeted loosely, underscoring the importance of accruing more evidence regarding postoperative triage. Future investigations should seek to minimize selection bias and clarify associations between overtriage and value of care by accruing EHR data prospectively, including both quantitative and qualitative data regarding reasons for triage decisions.

In conclusion, low-acuity postoperative patients who were overtriaged to ICUs had similar outcomes compared with risk-matched ward patients but had higher costs and lower value of care. The potential hazards of needlessly occupying an ICU bed are further emphasized by prior observations that denied ICU admission requests are associated with increased short-term mortality. These results provide a rationale and framework for developing and validating automated, real-time clinical decision support platforms to augment postoperative triage decisions and optimize value of care.