A novel, evidence-based, comprehensive clinical decision support system improves outcomes for patients with traumatic rib fractures

Abstract

Background

Traumatic rib fractures are associated with high morbidity and mortality. Clinical decision support systems (CDSS) have been shown to improve adherence to evidence based (EB) practice and improve clinical outcomes. The objective of this study was to investigate if a rib fracture CDSS reduced hospital length of stay (LOS), 90-day and 1-year mortality, unplanned ICU transfer, and the need for mechanical ventilation . The independent association of two process measures, an admission EB orderset and a pain-inspiratory-cough (PIC) score early warning system (EWS), with LOS were investigated.

Methods

The CDSS was scaled across 9 U.S. trauma centers. Following multiple imputation, multivariable regression models were fit to evaluate the association of the CDSS on primary and secondary outcomes. As a sensitivity analysis, propensity score matching was also performed to confirm regression findings.

Results

Overall, 3,279 patients met inclusion criteria. Rates of EB practices increased following implementation. On risk-adjusted analysis, in hospital LOS pre vs post-intervention was unchanged (IRR 1.06, 95% CI 0.97–1.15, p=0.2) but unplanned transfer to the ICU was reduced (OR 0.28, 95% CI 0.09–0.84, p=0.024), as was 1-year mortality (HR 0.6, 95% CI 0.4–0.89, p=0.01). Provider utilization of the admission order bundle was 45.3%. Utilization was associated with significantly reduced LOS (IRR 0.87, 95% CI 0.77–0.98, p = 0.019). The EWS triggered on 34.4% of patients; however, was not associated with a significant reduction in hospital LOS (IRR 0.76, 95% CI 0.55–1.06, p=0.1).

Conclusions

A novel, user-centered, comprehensive CDSS improves adherence to EB practice and is associated with a significant reduction in unplanned ICU admissions and possibly mortality, but not hospital LOS.

Level of Evidence:

2, Therapeutic/care management

Media summary:

A novel, user-centered, comprehensive clinical decision support system was implemented across 9 trauma centers. It was found to improve adherence to evidence based practice and is associated with a significant reduction in unplanned ICU admissions and 1 year mortality, but not hospital length of stay.

Background

Traumatic rib fractures occur in two-thirds of chest trauma(1) and are associated with significant morbidity and mortality, especially among older patients. Patients over the age of 65 have been shown to have an overall mortality rate of 22%(2) compared to 11%(3) in younger patients. Evidence-based (EB) practices have been shown to improve clinical outcomes for patients with rib fractures(4, 5) and include intensive care unit (ICU) admission for older patients with multiple rib fractures(6), Nexus chest computed tomography (CT) criteria(7), neuraxial blockade(8, 9), multimodal analgesia, operative rib fixation for flail chest(10), and use of incentive spirometry (IS)(11). Unfortunately, adherence to rib fracture EB practices varies significantly across United States hospitals(1, 4).

Clinical decision support systems (CDSS) are quality improvement (QI) interventions which have been shown to improve EB adherence, reduce health disparities, and improve clinical outcomes in other patient populations(12). CDSS deliver EB nudges to providers either in an interruptive fashion or passively within the electronic health record (EHR). The purpose of CDSS is to bring evidence to the point of care. CDSS have not been extensively used in trauma patients, an especially vulnerable population that benefits from standardization of care.

The objective of this study was to investigate if a comprehensive rib fracture CDSS pathway implemented across 9 trauma centers as a non-randomized stepped wedge trial was effective in reducing hospital length of stay (LOS). This endpoint was chosen by system quality improvement leadership and supported by a pilot study which achieved a 20% reduction in LOS. Secondary effectiveness objectives included a reduction in unplanned ICU transfer, mortality, and need for mechanical ventilation. Finally, the independent association of two process measures, an admission EB orderset and a pain-inspiratory-cough (PIC) score early warning system (EWS), with LOS were investigated.

Methods

Context and CDSS development and implementation

For this QI intervention, a previously published CDSS (3) was scaled across a 9-hospital trauma system via a 3-stage implementation process modeled after the User Acceptance and System Adaptation Design (UASAD) framework(13) (Supplemental Figure 1). Beginning in 2018, this CDSS was first piloted at an urban level 1 trauma center (3). Due to a successful pilot the CDSS was planned for implementation across a larger 9-hospital trauma system. However, due to differences in processes of care between the 9-hospital trauma system and the pilot site it was unknown how well such a CDSS would translate. For example, the majority of patients admitted at the 9-hospital system were primarily managed by non-surgical services. Furthermore, the 9-hospital system had variable hospital-specific resources, including both small rural hospitals ranging to a large urban academic hospital. Given the differences between these two health systems, the UASAD framework was used to optimize the originally piloted CDSS for the new 9-hospital trauma system which includes: 1 ACS-designated level 2 trauma center, 3 University of Minnesota Department of Health (DoH) designated level 3 trauma centers, and 5 University of Minnesota DoH designated level 4 trauma centers. UASAD is a three staged process. During stage 1, qualitative interviews were conducted to characterize barriers and facilitating factors for acceptance of the CDSS at each site(14). The CDSS was adaptively redesigned, developed within the EHR, and then a formal heuristic and end-user evaluation was performed to optimize usability and workflow integration (15). The final CDSS stage included the introduction of cognitive support resources for clinicians to categorize patients into mild, moderate, or severe risk, beginning with their arrival in the emergency department (ED). The cognitive support resources are available for the duration of patient care, as well as for post-discharge remote patient monitoring for 6-months. Following admission, the cognitive support included nursing specific resources utilizing the Epic Brain feature to ensure patients receive routine respiratory therapy, IS documentation, pain control, and ambulation. The CDSS also included an EWS to preemptively alert nurses about patients at risk for deterioration. Post-discharge remote monitoring included a patient facing mobile application that required the patient to input data at regular intervals for up to 180 days. It also included the integration of multiple validated patient reported outcome measure surveys. All patient responses were monitored by a 24/7 nursing triage service. Red and yellow alerts were developed based on responses. In the event of a yellow alert the patient was contacted the following morning by nursing and scheduled an urgent primary care evaluation. In the event of a red alert the patient was contacted immediately by the triage service and recommended to present to the nearest ED. This process was continued for 180 days post discharge. Details of this final CDSS care map have been previously published (14, 15).

Implementation began in November, 2020. The CDSS was scaled across all trauma centers by August, 2021 via a non-randomized stepped wedge design (Supplemental Figure 2).

Outcomes

Preliminary outcomes from the pilot study identified a 20% reduction in in-hospital LOS (3). Therefore, in-hospital LOS was identified as the primary outcome variable of interest for this study. Additional outcomes examined for the study include:

1. Unplanned transfer to the ICU

2. Days until mortality defined as days from hospital admission until death

3. Need for mechanical ventilation

Data Source

To evaluate CDSS association with high-quality care we used our health system’s “Learning Health System (LHS) data platform”. The trauma registry was not used as most hospitals were level 3 and 4 trauma centers with a limited trauma registry. The LHS data platform includes over 7000 variables on all patients with an ED visit or hospitalization across the health system from January 1, 2011 to present and is generated by automated extraction of data models from multiple data sources including, but not limited to, the EHR and other key real-world data sources (Table 1). The data platform includes data from two different Epic instances due to a health system merger in 2017.

Table 1:

List of electronic health record and real-world data elements available in the Learning Health Systems database

Data Element	Standard Coding System	Examples of Data elements
Medical Diagnoses	ICD-10-CM, SNOMED	Cirrhosis, Type 1 Diabetes, Type 2 Diabetes, Hypertension, Heart Failure, End-stage renal disease, COPD, etc.
Procedures	CPT, ICD-10-PCS, HCPCS	Rib Fixation, Epidural, Paravertebral Blockade
Laboratory Results	LOINC	WBC, Creatinine, etc
Vital Signs	LOINC	HR, RR, Temperature, etc
Provider-level data
Nursing Flowsheet data		Incentive spirometry volume, pain score, respiratory therapy vital capacity assessment, ventilation settings
Elixhauser comorbidity index
Inter-facility transfer information		Admission date, discharge date, presentation to other facility (ED only, Admitted from ED, direct admission), facility name
Medications	RxNorm	NSAIDS, diuretics, etc.
Trauma Mortality Prediction Model
Outcome Variables		Hospital Length of Stay (LOS), ICU and ICU LOS, Ventilation
Date of Death	Department of Health Death Certificate Database
Oxygen Requirements		Fraction of inspired oxygen
Oxygen Delivery Mechanism		Ordinal Variable: room air, nasal cannula, facemask, CPAP/BiPAP/HFNC, mechanical ventilation
Demographics		Age, Legal Sex, Self-identified Race and Ethnicity
Process Measures		Orderset utilization, CDS alerts, PIC score early warning system

Abbreviations: ICD-10-CM, international classification of disease, version 10, clinical modification; CPT, current procedural terminology; ICD-10-PCS, international classification of disease, version 10, procedure coding system; HCPCS, healthcare common procedure coding system; LOINC, logical observation identifiers names and codes; COPD, chronic obstructive pulmonary disease; WBC, white blood cell; HR, heart rate; RR, respiratory rate; ED, emergency department; NSAID, non-steroidal anti-inflammatory drugs; LOS, length of stay; ICU, intensive care unit; CPAP, continuous positive airway pressure; BIPAP, bilevel positive airway pressure; HFNC, humidified high flow nasal cannula; CDS, clinical decision support system; PIC, pain inspiration and cough

Inclusion / Exclusion Criteria:

All patients age 18 years and older with rib fractures were included. Patients with rib fractures were identified using the criteria listed below and confirmed by aligning the identified patients with those indicated in our trauma registry. This 2-factor validation process was necessary to differentiate between patients that might have a rib fracture secondary to other causes such as cardiopulmonary resuscitation.

Inclusion criteria:

1. Combination of thoracic diagnosis related group (DRG) and a rib fracture internal classification of diseases (ICD) code

   1. At least one of the following medicare severity (MS)-DRG codes: 183, 184, 185, 166, 167, 168, 199, 200, 201, 533, 534, 535, 536, 537, 538, 562, 563, 957, 958, 959, or all patients refined (APR)-DRG 135, 930
      AND
   2. one of the following ICD-10-clinical modification (CM) codes: S22.3*, S22.4*, S22.5*, or S22.9*
2. Primary trauma diagnosis code and a rib fracture ICD code

   1. Any trauma ICD-10-CM code listed as the “primary diagnosis”
      AND
   2. One of the following ICD-10-CM codes: S22.3*, S22.4*, S22.5*, or S22.9*

3. As part of the CDSS, providers have the option of manually inputting that the patient has a rib fracture

The following patients were excluded from final analysis:

1. Patients that were not admitted to observation or inpatient status (discharged from the ED), n=4,978

2. Patients that were admitted to three non-trauma hospitals within the health system were excluded from analysis, n=301

3. A period for staff training and education during a 4-month period prior, and including the initial month of, CDSS implementation. This period represents potential contamination due to education of staff regarding best practices and incomplete rollout of the CDSS, n=202

Statistical Analysis

For descriptive purposes, data were expressed as the mean and standard deviation (SD) for continuous variables with a normal distribution, median and interquartile range (IQR) for continuous variables with a skewed distribution, and as percentages for categorical variables. Student t-tests, Pearson χ², and Mann Whitney U tests were used for univariate analysis.

Multivariable Regression Analysis to evaluate the association of pre- vs post-implementation on outcomes (Supplemental Table 1):

The independent variable for the multivariable regression models listed below was the binary variable (exposure) of 0 if a patient was treated during the pre-implementation phase and 1 if the patient was treated post-CDSS implementation (Supplemental Figure 2).

Confounding variables were chosen based on known clinical association with more complicated disease. All regression models were adjusted for the following confounding variables (Supplemental Table 1): age (in years), body mass index (BMI), gender, smoking status, race/ethnicity, maximum pain score on ED arrival (0–10 ordinal scale), lowest systolic blood pressure (SBP), lowest oxygen saturation, highest heart rate (HR), highest temperature, and highest respiratory rate (RR) within 8 hours of ED arrival, first recorded saturation to FiO2 ratio in the ED, palliative care consultation, anesthesia consultation, comfort care status, initial sodium, hemoglobin, creatinine, and bicarbonate level, presence of flail chest, initial oxygen delivery needs (ordinal variable: room air, nasal cannula, facemask, CPAP/BiPAP/HFNC, mechanical ventilation), Elixhauser Comorbidity Index, if a CT chest was obtained, epidural/paravertebral block placement, rib fixation, trauma mortality prediction model (TMPM, a trauma risk adjustment tool shown to have superior discrimination in predicting mortality as compared to Injury Severity Score)(16, 17), admission hospital, admission to a surgical (vs. medical) service, first 24 hour morphine equivalents, IS or VC assessment (binary variable if the patient had IS/VC evaluated in the first 48 hours), and indicator variables if a complete blood count or basic/complete metabolic panel were obtained within the first 24 hours.

Missingness was assessed for each variable and ranged from 0% to 16.77% (Supplemental Table 2). To account for missing data, we used the Stata multiple imputation (mi) suite of commands(18), with 20 imputations for each missing value. A self-reported race was missing for 11 patients (0.3%). Missing self-reported race was treated as a separate category and not imputed (19).

A total of 8 models were trained (Supplemental Table 1). Four negative binomial regression models were fit for the primary outcome of hospital LOS. A negative binomial regression model was used instead of linear regression as hospital LOS had a skewed distribution. The four models included a (model 1) complete case analysis with bootstrapping (100 replicates) and (model 2) a model following multiple imputation. We hypothesized a priori that older patients or patients with more severe disease were more likely to deem benefit from the CDSS, thus 2 subgroup analyses were performed: (model 3) patients aged 66 years and older and (model 4) patients with a predicted mortality of 5% or higher via TMPM (Supplemental Table 1).

Three additional logistic regression models were fit for the secondary outcomes of interest, namely: unplanned transfer to the ICU, 90-day mortality and need for mechanical ventilation. All models were adjusted for the confounding variables described previously (Supplemental Table 1).

To evaluate the secondary outcome of one year mortality, a Cox proportional hazard model was fit. Censoring time is the number of days from admission until either the occurrence of the event (death) or the last time the patient was known to be alive (to a maximum of 365 days). Log-Log plots were generated and reported to assess the proportional hazards assumption.

Sensitivity Analysis

Given the large discrepancy in cohort sizes between the pre- and post-implementation groups a propensity matched analysis was also conducted. Using a randomly selected multiple imputation dataset, propensity scores were estimated with logistic regression using the previously described list of confounding variables (See Confounding Variable List A, Supplemental Table 1). The area under the curve (AUC) of this logistic regression model to generate the propensity scores was 0.82. A 1:1 nearest neighbor matching approach was used to generate two evenly matched pre- vs post-implementation cohorts with the common caliper set at 0.1. Even distribution of propensity scores was confirmed between matched groups and standardized differences were evaluated pre- vs post- for all confounding variables (Supplemental Figure 3). Univariate negative binomial regression was used to compare LOS for pre- (vs post-) implementation patients among the matched cohort. Univariate logistic regression models were also used to evaluate unplanned transfer to the ICU, the need for mechanical ventilation, and 90-day mortality. Kaplan-Meier survival curves for one year mortality were also estimated and compared using a log-rank test.

Multivariable Regression Analysis to evaluate the association of the admission order bundle and hospital LOS.

We also sought to evaluate the association between the admission orderset bundle and hospital LOS. As part of the CDSS, patients were risk stratified into one of three prognostic groups: mild, moderate, and severe. On admission the EHR would automatically risk-stratify the patients and recommend the appropriate admission orderset. The orderset differed based on the prognostic group the patient was placed in, with patients in the severe group receiving the most aggressive treatment including a recommendation for ICU admission. It was the decision of the clinician if they utilized the orderset bundle or manually placed individual orders. A negative binomial analysis was fit to investigate this association (Supplemental Table 1). The use of the admission orderset bundle was the independent variable and hospital LOS was the dependent variable. The multiple imputed dataset generated previously was used. Confounding variables are listed in Supplemental Table 1 and are the same as the previously described models.

Multivariable Regression Analysis to evaluate the association of the early warning system (EWS) on hospital LOS.

We sought to evaluate the association of the implementation of a modified PIC score EWS with hospital LOS. To generate this independent variable we created a binary variable set to 0 for patients who were admitted to the floor and met criteria for the EWS to trigger but did not receive the EWS. This variable was set to 1 if the patient was admitted to the floor, met criteria for the EWS to trigger and received the EWS nursing alert. A negative binomial analysis was fit to investigate this association (Supplemental Table 2). The multiple imputed dataset generated previously was used. Confounding variables are listed in Confounding Variable List B, Supplemental Table 2. In addition to the list of confounding variables used in the previous models, this model also adjusted for a patient’s worst pain score, lowest SBP, highest RR and HR, and lowest IS volume as these were triggering elements of the EWS.

The implementation of the CDSS for QI was submitted and reviewed by our institutional IRB and determined as “not human subjects research” (STUDY00005353) as the implementation of the decision support system was deemed a quality improvement activity. The post implementation statistical evaluation of the CDSS was also submitted to the institutional IRB and determined to meet the criteria for exemption from IRB review (STUDY00014515) as it was deemed exempt by category 4 “secondary research”. We set alpha at 0.05, 2-tailed. For all statistical analyses, we used Stata MP, version 17 (StataCorp, College Station, TX). The STROBE guideline was used to ensure proper reporting of methods, results, and discussion.

Results

Patient characteristics

After we applied our inclusion and exclusion criteria, 3,279 patients were eligible for this study (Figure 1). The median age of the patients was 74.0 (IQR, 59.8–85.4) years, 1,713 were male (52.3%), 3,035(92.6%) were white, 64 Black (1.95%), 66 Asian (2.01%), 20 Latinx (0.6%), 17 Native American/Alaskan (0.52%), 5 Hawaiian/Pacific Islander (0.15%), 2 Other (0.06%), 56 declined to specify their self-identified race/ethnicity (1.7%), and 11 were missing race/ethnicity (0.34%). On univariate analysis post-intervention patients had lower saturation to FiO2 ratios (325.9 vs 336.9, p=0.017), higher initial minimum SpO2 (96% vs 95%, p=0.004), lower initial hemoglobin (12.9 vs 13.2, p=0.004), lower initial sodium (138 vs 139, p<0.001), higher Elixhauser comorbidity score (11 vs 9, p=0.027), and higher TMPM probability of mortality (3.7% vs 3.6%, p=0.007) (Table 2).

Figure 1:

Study diagram detailing selection of patients in the LHS database

Abbreviations: LHS, learning health systems; CDSS, clinical decision support system

Table 2.

Pre vs post-intervention patient and clinical characteristics.

	Pre-Intervention	Post-Intervention	p-value
	N=2,811	N=468
Age (years)	73.56 (59.47–85.44)	76.48 (63.265–85.675)	0.026
Body Mass Index (BMI)	26.37 (22.925–30.2)	25.61 (22.86–29.45)	0.055
Male	1,479 (52.6%)	234 (50.0%)	0.29
Race/Ethnicity
White	2,621 (93.2%)	416 (88.9%)	0.002
Black	53 (1.9%)	11 (2.4%)
Asian	52 (1.8%)	14 (3.0%)
Latinx	14 (0.5%)	6 (1.3%)
Native American/Alaskan	15 (0.5%)	2 (0.4%)
Hawaiian/Pacific Islander	3 (0.1%)	2 (0.4%)
Other	2 (0.1%)	0 (0.0%)
Patient Declined	40 (1.4%)	17 (3.6%)
Missing	11 (0.4%)	0 (0.0%)
Trauma Center
Hospital 1	65 (2.3%)	9 (1.9%)	<0.001
Hospital 2	130 (4.6%)	8 (1.7%)
Hospital 3	424 (15.1%)	47 (10.0%)
Hospital 4	128 (4.6%)	11 (2.4%)
Hospital 5	445 (15.8%)	77 (16.5%)
Hospital 6	615 (21.9%)	160 (34.2%)
Hospital 7	603 (21.5%)	102 (21.8%)
Hospital 8	185 (6.6%)	33 (7.1%)
Hospital 9	216 (7.7%)	21 (4.5%)
Worse Incentive Spirometry (mL)	1000 (750–1500)	1000 (700–1500)	0.85
Max Pain Score (Initial 24 hours)	8 (7–10)	8 (7–10)	0.84
Lowest 24-hr Oxygen Saturation/FiO2 ratio	337.04 (272.22–362.96)	326.67 (278.79–350)	0.023
Lowest (initial 8 hour) Systolic BP, mmHg	136 (120–154)	138 (122–157)	0.21
Lowest (initial 8 hour) Oxygen Saturation	95 (93–97)	96 (94–98)	<0.001
Highest (initial 8 hour) Heart Rate	82 (71–94)	81 (70–95)	0.54
Highest (initial 8 hour) Temperature	98 (97.7–98.4)	98 (97.6–98.35001)	0.17
Highest (initial 8 hour) Respiratory Rate	18 (16–20)	18 (16–20)	0.010
Initial WBC	9.7 (7.6–12.5)	9.7 (7.7–12.5)	0.88
Initial Hemoglobin	13.2 (11.8–14.4)	12.8 (11.5–13.9)	<0.001
Initial Sodium	139 (136–141)	138 (135–140)	<0.001
Initial Creatinine	.91 (.76–1.13)	.9 (.73–1.15)	0.43
Initial CO2	26 (24–28)	26 (24–29)	0.093
Flail Chest	33 (1.2%)	5 (1.1%)	0.84
Smoker	1,324 (47.1%)	210 (44.9%)	0.37
Elixhauser Comorbidity Index	9 (0–20)	11 (3–22)	0.011
TMPM Probability Mortality	3.56% (1.34–3.69%)	3.69% (2.1–3.9%)	0.003
Adherence Rib Fracture Orderset	0 (0.0%)	212 (45.3%)	<0.001
Adherence Nexus Chest CT	1,717 (61.1%)	378 (80.8%)	<0.001
Rib Fixation	18 (0.6%)	5 (1.1%)	0.30
Neuraxial Blockade	99 (3.5%)	19 (4.1%)	0.56
Admitted to ICU from ED	273 (9.7%)	25 (5.3%)	0.002
Admitted to a surgical service	920 (32.7%)	128 (27.4%)	0.021
Morphine Equivalents (Initial 24 hrs), mg	42.75 (19.5–76.75)	32.5 (15–59.5)	<0.001
Morphine Equivalents (24–48 hrs), mg	30 (15–66)	20 (11.25–41.75)	<0.001
Morphine Equivalents (48–72 hrs), mg	30 (15–62.5)	19.5 (8–38.5)	<0.001
Anesthesiology Consultation	237 (8.4%)	111 (23.7%)	<0.001
Palliative Medicine Consultation	192 (6.8%)	58 (12.4%)	<0.001
ED Respiratory Support
Room Air	1,349 (48.5%)	254 (54.6%)	0.18
Nasal Cannula	1,108 (39.9%)	160 (34.4%)
Facemask	247 (8.9%)	40 (8.6%)
CPAP/BiPAP/HFNC	37 (1.3%)	6 (1.3%)
Mechanical Ventilation	38 (1.4%)	5 (1.1%)
Day 1 Respiratory Support	1,174 (52.8%)	227 (57.3%)	0.28
Room Air	824 (37.1%)	135 (34.1%)
Nasal Cannula	142 (6.4%)	26 (6.6%)
Facemask	46 (2.1%)	4 (1.0%)
CPAP/BiPAP/HFNC	38 (1.7%)	4 (1.0%)
Mechanical Ventilation	847 (51.5%)	185 (60.3%)	0.007
Day 2 Respiratory Support	634 (38.6%)	94 (30.6%)
Room Air	75 (4.6%)	20 (6.5%)
Nasal Cannula	53 (3.2%)	5 (1.6%)
Facemask	35 (2.1%)	3 (1.0%)
CPAP/BiPAP/HFNC	1,349 (48.5%)	254 (54.6%)	0.18
Mechanical Ventilation	1,108 (39.9%)	160 (34.4%)
Hospital Length of Stay, days	3.01 (1.76–5.19)	3.16 (2.0–5.26)	0.080
1 Year All-Cause Mortality	339 (12.1%)	36 (7.7%)	0.006
Transitioned to Comfort Care Status	67 (2.4%)	19 (4.1%)	0.036
Mechanical Ventilation	98 (3.5%)	11 (2.4%)	0.20
Admitted to ICU during hospital stay	403 (14.3%)	29 (6.2%)	<0.001
Unplanned Transfer to ICU	130 (4.6%)	4 (0.9%)	<0.001

Abbreviations: mL, milliliters; FiO2, fraction of inspired oxygen; BP, blood pressure; WBC, white blood cell; TMPM, trauma mortality prediction model; CT, computed tomography; ICU, intensive care unit; ED, emergency department; CPAP, continuous positive airway pressure; BiPAP, bilevel positive airway pressure; HFNC, high flow nasal cannula

Patients post-intervention had higher rates of the following practices: adherence with NEXUS Chest CT criteria (80.8% vs 61.1%, p<0.001), lower first 24 hour morphine equivalents (32.5 vs 42.75 mg, p<0.001), higher rate of anesthesia consultation (23.7% vs 8.4%, p<0.001)), higher rate of palliative care consultation (12.4% vs 6.8%, p<0.001)(Table 2). Post-intervention patients were less likely to be admitted to a surgical service (27.4% vs 32.7%, p=0.02). 45.3% of admission orders post-intervention included the rib fracture admission order bundle. 4.1% (vs 3.5%) of patients received an epidural/paravertebral block, 67% vs 45% of patients had IS documented, and 1.1% vs 0.6% of patients received rib fixation.

On univariate analysis post-intervention patients had a lower 1 year mortality (7.7% vs 12.1%, p=0.006). There was no difference in hospital LOS on univariate analysis (3.2 vs 3.0 days, p=0.08) (Table 2).

ICU utilization post- (vs pre-) implementation

We noted a lower rate of ICU admission from the ED (Post- 5.3% vs Pre- 9.7%, p=0.002) (Table 2). To evaluate if this observation was due to differences in ICU utilization due to the COVID-19 pandemic we investigated the pre-implementation ICU admission rates from March 2020 to present and observed a similar rate of ICU admission from the ED, 9.19%. The CDSS included criteria for ICU admission: (1) age > 65 with 3 or more rib fractures, (2) Saturation / FIO2 ratio < 235, (3) Pain score of 7–10 despite IV narcotic challenge, (4) Respiratory rate < 10 or > 29 breaths per minute, (5) Intubation. When we applied this criteria to characterize unnecessary (overtriaged) ICU admissions in the post- (vs pre-) implementation cohorts, we observed no unnecessary ICU admissions post-implementation vs 5.23% unnecessary ICU admissions pre-implementation. This suggests approximately 54% of the 9.7% pre- ICU admissions were potentially overtriaged to the ICU. On univariate analysis post- (vs pre-) implementation patients were significantly less likely to require unplanned transfer to the ICU (0.9% vs 4.6%, p<0.001) (Table 2).

Effectiveness of intervention to reduce hospital LOS

We performed a series of regression models to evaluate the association of the intervention on hospital LOS. First we performed a complete case risk-adjusted analysis and noted a significant reduction in hospital LOS for patients in the post (vs pre-) implementation period (incidence rate ratio (IRR) 0.77, 95% CI 0.65–0.92, p=0.005) (Table 3). Given data missingness, multiple imputation was then performed. All subsequent analyses reported were performed on the multiple imputation dataset. Following multiple imputation, there was no difference in LOS for patients in the post (vs pre-) implementation period (IRR 1.06, 95% CI 0.97–1.15, p=0.2) (Table 3, Supplemental Table 3). Due to sample size differences between the pre- vs post-implementation period a propensity matched analysis was performed as a sensitivity analysis. Using the imputed dataset, two propensity matched cohorts with 345 patients per cohort were generated. Following propensity matching we did not observe a significant difference in hospital LOS (Pre [median 3.02 days] vs Post [median 3.01 days] Mann-Whitney p = 0.9, unadjusted negative binomial regression: OR 1.02, p = 0.7). These findings were similar to our findings for the multivariable regression analysis.

Table 3:

Effectiveness analysis of Rib Fracture Clinical Decision Support Intervention

Primary Outcome – Hospital LOS	IRR	95% CI	p-value
Complete Case Analysis - regression	0.77	0.65–0.92	0.005
Multiple Imputation - regression*	1.06	0.97–1.15	0.2
Multiple Imputation – prop match*	1.02	0.89–1.17	0.7
Primary Outcome - Subgroup Analysis	IRR	95% CI	p-value
Older patients (age > 65 years) - regression *	1.09	0.996–1.2	0.06
TMPM probability mortality > 5% - regression*	0.89	0.71–1.13	0.3
Secondary Outcomes	OR	95% CI	p-value
Unplanned ICU transfer - regression*	0.28	0.09–0.84	0.024
Unplanned ICU transfer – propensity match*	0.14	0.05–0.4	< 0.001
Need for mechanical ventilation - regression*	0.58	0.26–1.28	0.18
Need for mechanical ventilation – propensity match*	0.55	0.24–1.26	0.16
90-day mortality - regression*	0.56	0.28–1.12	0.1
90-day mortality – propensity match*	0.6	0.32–1.1	0.099
	HR
1-year mortality – Cox PH regression*	0.6	0.4–0.89	0.01

Effectiveness analysis of rib fracture clinical decision support intervention.

^*All analyses were conducted using the imputed dataset.

Abbreviations: LOS, length of stay; IRR, incidence rate ratio; CI, confidence interval; TMPM, trauma mortality prediction model; ICU, intensive care unit; PH, proportional hazards; HR, hazard ratio

Effectiveness of intervention to reduce hospital LOS: subgroup analysis

One of the a priori hypotheses was that patients with a worse prognosis would be more likely to benefit from the CDSS intervention. To investigate, two subgroup analyses were performed. In older patients (age>65 years) there was no difference in hospital LOS for post (vs pre-) implementation patients (IRR 1.09, 95% CI 0.996–1.2, p=0.06). In patients with a >5% probability of mortality (based on TMPM) there was no difference in hospital LOS (IRR 0.89, 95% CI 0.71–1.13, p=0.3) (Table 3).

Effectiveness of intervention for secondary clinical outcomes

Multivariable logistic regression analysis identified a significant 74% reduction in unplanned ICU transfer for post (vs pre-) patients (OR 0.28, 95% CI 0.09–0.84, p=0.024) (Table 3, Supplemental Table 4). A similar finding was observed on the propensity-matched analysis (Pre [27/345, 7.83%] vs Post [4/345, 1.16%], Chi squared p < 0.001, unadjusted logistic regression OR 0.14, p < 0.001)

Multivariable logistic regression analysis did not identify a significant difference in need for mechanical ventilation in post (vs pre-) patients (OR 0.58, 95% CI 0.26–1.28, p=0.18) (Supplemental Table 5). A similar finding was observed on the propensity-matched analysis (Pre [16/345, 4.64%] vs Post [9/345, 2.61%], Chi squared p = .15, unadjusted logistic regression OR 0.55, p = 0.16)

Cox proportional hazard analysis was performed to investigate if the intervention was associated with a significant reduction in time until 1 year mortality. This analysis noted a significant 41% reduction in 1-year mortality (HR 0.6, 95% CI 0.40–0.89, p=0.01) (Supplemental Table 6). The hazard function and log-log plots are shown in Supplemental Figure 4. In addition to investigating time until 1 year mortality, we also fit a multivariable logistic regression model for the binary endpoint of 90-day mortality and observed a trend towards reduced 90-day mortality (OR 0.56, 95% CI 0.28–1.12, p =0.1). Similar findings were observed on the propensity-matched Kaplan-Meier analysis evaluating the association between the intervention and 1 year mortality (Figure 2, log-rank test p = 0.1). Similar to the regression analysis, we also investigated the binary endpoint of 90-day mortality in the propensity-matched analysis and observed a similar trend towards reduced 90-day mortality (Pre [29/345, 8.41%] vs Post [18/345, 5.22%], Chi squared p = 0.1).

Figure 2:

One-year Kaplan-Meier survival estimates for propensity-matched cohort

Association of the admission orderset bundle and hospital LOS

Our pilot study identified that the implementation of a rib fracture evidence-based bundled admission orderset was associated with significant reduction in hospital LOS. While this study included a similar admission orderset, we did not observe a significant reduction in hospital LOS. However, one limitation of implementation was only 45% of patients received the admission orderset (Table 2). We sought to investigate the independent association of the admission orderset bundle on hospital LOS as to guide future QI efforts. Patients that received (vs those that did not receive) the admission orderset bundle had significantly shorter hospital LOS (IRR 0.87, 95% CI 0.77 – 0.98, p = 0.019).

Rib Fracture Early Warning System (modified PIC score) (EWS)

161 patients initially admitted had the EWS alert. Of the 161 EWS alerts, 88(55%) resulted in the appropriate action - provider notification by nursing. We observed the implementation of the EWS was not associated with reduced hospital LOS (IRR 0.76, 95% CI 0.55 – 1.06, p = 0.1) on multivariable negative binomial regression.

The reason for each EWS trigger was tracked; when due to tachypnea patients had the longest LOS (8.5 days) followed by IS<15 mL/kg (7 days), hypotension (6.8 days), poor cough (5.0 days), pain (4.2 days), and tachycardia (4.1 days) (Figure 3a). Patients with the EWS trigger due to tachycardia had a 25% 90-day mortality, followed by IS<15 mL/kg (8.3%), tachypnea (8%), hypotension (5.5%), poor cough (4.8%), and pain (2.7%) (Figure 3b).

Figure 3:

Rib Fracture modified PIC (early warning system [EWS]) score

Abbreviations: PIC, pain-inspiration-cough; EWS, early warning system; LOS, length of stay; HR, heart rate; RR, respiratory rate; IS, incentive spirometry; SBP, systolic blood pressure

Discussion

This CDSS was originally piloted at a level 1 trauma center. Given differences between our trauma system and the pilot system, we modified the original CDSS through formal qualitative interviews and usability optimization to tailor it for system resources. We then conducted a nonrandomized stepped wedge QI study to investigate the effectiveness of the modified CDSS ultimately implemented across our 9-hospital trauma system. In this study, we had the following novel findings: (1) adherence with process measures was much lower than the pilot study despite a similar education and training strategy, (2) the CDSS intervention significantly reduced unnecessary ED to ICU admissions and reduced unplanned ICU transfers; however, did not reduce in-hospital LOS, (3) adherence with admission order bundle was associated with decreased LOS, and (4) the modified PIC EWS was not associated with decreased LOS and the most important triggers were RR, SBP, HR, and IS.

Rib fractures are one of the five leading causes of trauma mortality in older adult patients(20). A previous study leveraging the National Trauma Databank identified that rib fracture mortality in older adult patients significantly improved between 2007 and 2015(20). One reason for this may be the characterization of optimal EB practice for rib fracture patients(21). In the last two decades, EB practices identified for rib fractures include: ICU admission for older adults with multiple rib fractures(4, 6), the NEXUS Chest CT score(7), the PIC score(22), multimodal analgesia(23–25), neuraxial blockade(8, 9), operative rib fixation for flail chest(10), and IS(11). Unfortunately, despite numerous trials, these practices are not routine across US trauma centers.

Comprehensive care maps are one tool QI researchers can leverage to bundle multiple EB practices together and improve adherence with not just one best practice, but multiple. One of the earliest approaches to this in rib fracture patients was undertaken by Todd et al(26). They implemented a pathway with assessments and interventions targeting pain control and pulmonary toileting (PIC score) for patients with multiple rib fractures and demonstrated shorter ICU and hospital LOS as well as decreased rates of pneumonia and mortality. In the modern era, CDSS are used to deploy these pathways. One benefit of CDSS is their portability as they are executable tools that can be shared and scaled across EHRs. The CDSS used for this study was developed and implemented at another trauma system by Macheel et al(3). Similar to Todd’s study, Macheel et al, observed a significant reduction in hospital LOS and a trend towards reduced pulmonary complications. Given this, we hypothesized that the CDSS would also reduce LOS within our system. Unfortunately, we did not identify a reduced LOS. One potential reason may be that LOS was already low in the pre-implementation group (4 days) which was the same LOS in Macheel et al’s post-intervention cohort (4 days). However, the alternative interpretation may highlight the reality of “real world” trauma care in the U.S. and the need for further research in quality improvement within non-tertiary trauma centers. Our system and Macheel et al’s urban level 1 trauma center differ vastly. Given this, it was unlikely that the originally piloted CDSS could be implemented unchanged and be successful within our system which includes both community and rural hospitals with limited resources. Thus we needed to not only tailor the CDSS to our system but to each individual hospital’s resources. In this study we leveraged the UASAD framework to translate the CDSS. While EB practice did improve, it did not reach the level reported by Macheel et al. For example, utilization of the admission order bundle was only 45% (as compared with 88.3% in Macheel et al’s study). Rates were even lower for other EB practices such as rib fixation and neuraxial blockade. This highlights important real-world difficulties in deploying trauma QI initiatives across large complex health systems. It is likely easier to deploy initiatives at tertiary trauma centers where patients are admitted to a singular trauma surgery service. However, the majority of patients in the U.S. receive care at non-trauma centers where they are primarily managed by hospitalist services with limited access to intensive care units, thoracic surgery for rib fixation or regional anesthesia services for neuraxial blockade (27, 28).

While adherence was variable, we did observe that overall the intervention significantly reduced unplanned transfer to the ICU, trended towards reduced need for mechanical ventilation, and potentially reduced 1-year mortality. We hypothesize that these effects would be even more pronounced with improved CDSS adherence. Our health system includes 9 trauma centers, both urban and rural, with a range of bed sizes from a 34 bed rural hospital to a 1,700 bed quaternary academic research center. Three of our trauma centers have dedicated trauma services, whereas the other 6 do not, with trauma care primarily provided by the hospitalist service with general surgeon consultation. We believe that our system represents a microcosm to study the reality and complexity of trauma care nationally, where only 43% of trauma patients receive care at a tertiary trauma system(29) and the remainder receive care at designated level 3, 4, and non-designated trauma centers. Thus, it is critical that techniques and tools are developed which can facilitate improved adoption of EB practice at non-tertiary centers.

We have taken significant steps to improve adoption and engagement and will re-evaluate after our next improvement cycle. Specifically, we do not believe that booster education sessions or learning modules will bridge this implementation gap across such a large and heterogeneous system. We plan to rely on technology to better automate the integration of the rib fracture admission bundle elements for patients with a known rib fracture. We have also developed a real-time monitoring system that tracks admission order bundle utilization, neuraxial blockade, flail chest, rib fixation, and the EWS. This will be utilized by trauma program managers, medical directors, and QI champions centrally to provide microeducation interventions to non-compliant providers in real-time. The benefit of such a microeducation intervention is that it allows for a patient to still receive the EB practice during their admission. Additionally, future directions are investigating the potential role for diagnostic artificial intelligence to identify traumatic injuries on chest x-ray and autonomously notify providers that specific EB practices are indicated (30–32).

We did observe a significant reduction in unnecessary ED to ICU admissions via standardization in ICU admission criteria. Of the approximately 9% ICU admissions in the pre-implementation period, 5% did not meet the new ICU admission criteria. After deploying these criteria as decision support within the electronic health record’s admission navigator section, we noted a large reduction in ED to ICU admissions independent of ICU utilization during the COVID-19 pandemic. Our interpretation of this finding, is that prior to the CDSS implementation, ICU admission was subjectively determined based on physician intuition resulting in variability in ED to ICU vs floor admission. We believe, this resulted in high overtriage (approximately 5% based on ICU admission criteria) and undertriage (approximately 4% based on unplanned ICU transfer rate). Following CDSS implementation we observed a significant reduction in unplanned ICU transfer.

This study has several limitations. Only inpatients from one region of the country were included in this study therefore our findings may not be generalizable to other areas of the country. Our dataset is not manually extracted and thus suffers from limitations associated with real-world EHR data. The ability to reliably demonstrate associations between the EWS and outcomes is limited by our small sample size of 161 patients. Finally, this CDSS was implemented during the first two years of the COVID-19 pandemic, when resources were limited and surgical services were disrupted.

Conclusions

A comprehensive CDSS improves adherence to EB practices and is associated with a significant reduction in rates of unplanned ICU transfer; however it is not associated with a significant reduction in hospital LOS. Use of a rib fracture specific order bundle is associated with a significant reduction in hospital LOS.

Supplementary Material

Supplemental Tables 1-6

Supplemental Table 1: List of regression models performed in main analysis

Supplemental Table 2: Missingness of confounding variables prior to multiple imputation

Supplemental Table 3: Full model output for imputed multivariable negative binomial regression model to investigate association of the intervention on hospital LOS

Supplemental Table 4: Full model output for imputed multivariable logistic regression model to investigate association of the intervention on unplanned transfer to the ICU

Supplemental Table 5: Full model output for imputed multivariable logistic regression model to investigate association of the intervention on the need for mechanical ventilation

Supplemental Table 6: Full model output for imputed Cox proportional hazard regression model to investigate association of the intervention on one year mortality

Supplemental Figure 1

Supplemental Figure 1: Enhanced evaluation and implementation as a part of the User Acceptance and System Adaptation Design framework for design of CDSS

Supplemental Figure 2

Supplemental Figure 2: Stepped wedge model implementation of a clinical decision support system across a 9-hospital trauma system

Supplemental Figure 3

Supplemental Figure 3: Standardized difference of confounding variables before vs. after propensity matching

Supplemental Figure 4

Supplemental Figure 4: Cox proportional hazard model diagnostics