Effects of Telemedicine ICU Intervention on Care Standardization and Patient Outcomes: An Observational Study

Supplemental Digital Content is available in the text.

Abstract

Objectives:

Given the numerous recent changes in ICU practices and protocols, we sought to confirm whether favorable effects of telemedicine ICU interventions on ICU mortality and length of stay can be replicated by a more recent telemedicine ICU intervention.

Design, Setting and Patients:

Observational before-after telemedicine ICU intervention study in seven adult ICUs in two hospitals. The study included 1,403 patients in the preintervention period (October 2014 to September 2015) and 14,874 patients in the postintervention period (January 2016 to December 2018).

Intervention:

Telemedicine ICU implementation.

Measurements and Main Results:

ICU and hospital mortality and length of stay, best practice adherence rates, and telemedicine ICU performance metrics. Unadjusted ICU and hospital mortality and lengths of stay were not statistically significantly different. Adjustment for Acute Physiology and Chronic Health Evaluation Version IVa score, ICU type, and ICU admission time via logistic regression yielded significantly lower ICU and hospital mortality odds ratios of 0.58 (95% CI, 0.45–0.74) and 0.66 (95% CI, 0.54–0.80), respectively. When adjusting for acuity by comparing observed-over-expected length of stay ratios through Acute Physiology and Chronic Health Evaluation IVa methodology, we found significantly lower ICU and hospital length of stay in the postintervention group. ICU mortality improvements were driven by nighttime ICU admissions (odds ratio 0.45 [95% CI, 0.33–0.61]) as compared to daytime ICU admissions (odds ratio 0.81 [95% CI, 0.55–1.20]), whereas hospital mortality improvements were seen in both subgroups but more prominently in nighttime ICU admissions (odds ratio 0.57 [95% CI, 0.44–0.74]) as compared to daytime ICU admissions (odds ratio 0.73 [95% CI, 0.55–0.97]), suggesting that telemedicine ICU intervention can effectively supplement low intensity bedside staffing hours (nighttime).

Conclusions:

In this pre-post observational study, telemedicine ICU intervention was associated with improvements in care standardization and decreases in ICU and hospital mortality and length of stay. The mortality benefits were mediated in part through telemedicine ICU supplementation of low intensity bedside staffing hours.

Telemedicine in ICU (tele-ICU) services are now implemented in 15–20% of adult ICU beds (1, 2). Tele-ICU as an evolving care delivery model continues to grow (1). The goal of tele-ICU services is to increase access to critical care specialists, to improve clinical outcomes, and increasingly to manage ICU capacity and resources (3). Publications reporting their respective experiences date back to 2000 (4). Since that time several studies have examined outcomes including ICU and hospital mortality and length of stay (LOS). Several meta-analyses have found ICU mortality benefit and mixed results with regard to hospital mortality (5–8). Sensitivity analysis has suggested a potentially stronger association with positive outcomes and age of the study (8). Our own recent systematic review and meta-analysis found that although overall ICU mortality reductions can be expected through tele-ICU intervention, the largest ICU mortality improvements can be expected in ICUs with high preintervention standardized ICU mortality ratios (6).

Our understanding of which elements of the multifactorial tele-ICU intervention are most effective has also evolved. Review of new patient admissions within 1 hour of arrival, frequent collaborative data reviews, rapid laboratory/alert review, and interdisciplinary rounds have been associated with more favorable outcomes (9). It is likely that outcomes improvement is at least partially related to systemic process improvement, data review, and care standardization efforts coordinated through the tele-ICU.

Given that critical care standards and outcomes, for example in sepsis management, have evolved since inception of earlier tele-ICU programs up to almost 20 years ago, it is important to re-examine whether favorable effects on outcomes can in fact be replicated in more recent tele-ICU interventions, across the spectrum of acute care hospital systems, geographical regions, and operational models (centralized Hub and spoke vs decentralized models) (10). We therefore report our recent experience and outcomes before and after tele-ICU intervention herein. Given the recently identified factors associated with tele-ICU success, we delineate associated changes in ICU best practice adherence and describe the internal tele-ICU performance optimization itself.

METHODS

Setting

We performed a retrospective pre-post study examining outcomes before and after intervention of tele-ICU services in our hospital system of mixed academic and community hospitals (six ICUs in the main academic medical center [medical, neurologic, cardiothoracic, cardiac, trauma, surgical, with a total of 83 beds] and one mixed medical-surgical ICU in an affiliated community hospital [15 beds]).

Program Description

Details on bedside ICU physician coverage models, tele-ICU staffing, and workflows are provided in the Supplemental Digital Content (http://links.lww.com/CCX/A235). Daily tele-ICU team tasks involve nurse auditing of each continuously monitored patient bid to ensure best practices are being followed. The covered best practices are blood transfusion thresholds, venous thrombosis prophylaxis, stress ulcer prophylaxis, glucose control, medication dosing appropriateness, central catheter utilization, and lung-protective mechanical ventilation settings (11–14). Additionally, physicians review each new admission within 1 hour, respond to alerts indicating physiologic instability, manage ventilator variables, and proactively survey high risk or physiologically worsening patients to provide recommendations for care augmentation. Tele-ICU physicians are given full authority to intervene if needed in all units. ICU performance meetings on ICU and hospital mortality, LOS, and detailed best practice adherence are conducted with the stakeholders of each individual ICU as well as joint stakeholders across ICUs (e.g. respiratory therapy, pharmacy) and administrative leadership every 3 months. Tele-ICU physicians and nurses receive quarterly performance feedback on metrics related to patient outcomes. In 2018, bed placement and transfer center functions were moved into the eHealth center to streamline communications and workflows (3, 9). Interdisciplinary conditional ICU bed sharing coordinated by the tele-ICU team was implemented in 2018. In 2017, the tele-ICU started an initiative to reduce the duration of nontunneled central venous catheter (CVC) utilizations in an attempt to decrease the frequency of CVC related complications like catheter-associated thromboses and catheter-related bloodstream infections. Starting in 2017, the tele-ICU intensivist reviewed CVC utilization daily and reminded the bedside teams of CVCs with long utilization times (phase I). Starting in 2018, the insertion and removal of all CVCs were captured in a central registry and at threshold utilization times (4 d for femoral CVCs and 8 d for internal jugular or subclavian locations), the tele-ICU physician reached out to the bedside team to discuss the continued indication for and alternatives to continued CVC utilization (phase II). Rates of CVC-related deep vein thromboses and central line–associated bloodstream infections (CLABSIs) were monitored through the National Healthcare Safety Network Database.

Study Design

This study was conducted between October of 2014 and December of 2018. October 2014 to September 2015 constituted the preintervention period. During this preintervention period, consensus protocols on systemwide ICU best practices were generated by multidisciplinary task-forces representing all system ICUs (care standardization of transfusion practices, venous thromboembolism [VTE], stress ulcer prophylaxis [SUP], glycemic control, and acute respiratory distress syndrome network [ARDSnet] ventilation variables/weaning protocol). October 2015 to December 2015 constituted the start-up and run-in period during which all tele-ICU functions were sequentially introduced. The postintervention period started in January of 2016 with full tele-ICU functionality and concluded in December of 2018. Data on patient demographics, by Acute Physiology, Age and Chronic Health Evaluation Version IVa (APACHE-IVa) scores, ICU mortality, hospital mortality, ICU LOS, and hospital LOS were electronically collected and calculated for all postintervention period patients. A representative sample of preintervention patients was randomly generated, and corresponding data elements were manually abstracted using the same methodology as for the postintervention period. The number of preintervention patients abstracted was proportional to the postintervention volume of the respective ICU. Minor variations in the pre-to-post ratios are attributable to preintervention sample size modeling after 2017 admission volumes. To ensure the reliability and reproducibility of the data abstraction, registered nurses were trained in abstraction methodology. A subset of records was randomly selected and reviewed by a second independent trained abstractor to ensure concordance. Clinical best practice compliance rates were measured as the percentage of eligible patients receiving the respective best practice. The tele-ICU software (eCare Manager 4.1.1; Philips, Amsterdam, The Netherlands) provides the methodology to measure best practice compliance rates based on interfaced and tele-ICU entered datapoints for each best practice.

Patient acuity and risk adjusted mortality and LOS were calculated by APACHE-IVa methodology (15).

The analytical plan set the primary outcome as difference in risk adjusted ICU mortality between pre and post tele-ICU intervention groups. Secondary outcomes were set as hospital mortality, ICU LOS, hospital LOS, CVC utilization, and compliance rates with core best clinical practice measures (blood transfusion thresholds, VTE prophylaxis, SUP, lung-protective ventilation compliance, and glycemic control). Secondary tele-ICU performance metrics included ICU-to-tele-ICU engagement and timeliness of tele-ICU video assessment for newly admitted patients from time of ICU admission.

The study was reviewed and approved by the Westchester Medical Center/New York Medical College Program for the Protection of Human Subjects (Institutional Review Board (no14-287).

Statistical Methods

Descriptive statistics were calculated for independent variables as well as outcome variables. Comparisons between two groups on continuous outcomes variables were made using the Mann-Whitney U test. Comparisons between three or more groups on continuous variables were made using one-way analysis of variance (ANOVA). Comparisons between groups on categorical variables were made using the Fisher exact test or the chi-square test.

Dichotomous outcomes including ICU and hospital mortality were modeled through multivariable logistic regression. Statistical analyses and modeling were performed through Graphpad Prism Version 7.0a (GraphPad Software, La Jolla, CA) and IBM SPSS Version 26 (IBM Corp., Armonk, NY).

RESULTS

A total of 15,328 ICU admissions were identified for the postintervention period. One-hundred sixty-nine admissions were excluded due to an ICU LOS of less than 4 hours. One-hundred twenty-five ICU admissions were excluded for missing data elements. Thirty-three ICU admissions were excluded for invalid data elements. Sixty-eight ICU admissions were excluded due to nonpredictive APACHE-IVa admission diagnoses. Forty-three ICU admissions were excluded due to patient age less than 18 years. Sixteen ICU admissions were excluded due to transfer from another ICU. Of the remaining 14,874 ICU admissions, 14,383 (93.8%) yielded APACHE-IVa ICU predictions on mortality and LOS and 13,162 (85.9%) ICU admissions yielded APACHE-IVa Hospital predictions on mortality and LOS.

Preintervention phase patients were selected by random number generation and abstracted manually if no exclusion criteria were present. Any records with missing or invalid data were excluded. Random charts were abstracted until the preintervention ICU admission volume reached 10% of the corresponding postintervention ICU volume for 2016. Due to this abstraction strategy, no ICU admissions had to be excluded for missing or invalid data.

Data on patient volume by ICU, patient demographics, acuity, and primary diagnosis by organ system are presented in Table 1.

TABLE 1.

Patient Characteristics in the Preintervention and Postintervention Groups

The average age of patients in the postintervention group was significantly lower than in the preintervention group, whereas the average Acute Physiology Scores (APS) and APACHE-IVa scores were significantly higher by 22.2% and 18.3%, respectively (36 vs 44 and 46.9 vs 55.5, respectively). There was a significant relative increase in volume of neurologic admission diagnoses from 20.2% to 22.6% of admissions, decrease of “sepsis of unknown organ system” admissions from 2.0% to 3.4%, and decrease of “Other” admission diagnoses from 0.5 to 0.1%. All other admission diagnosis categories were similar between the groups.

Unadjusted ICU and hospital mortality rates were not significantly different, but adjustment for APACHE-IVa score, ICU type, and admission time (daytime vs nighttime) via logistic regression yielded significantly lower ICU and hospital mortality odds ratios (ORs) of 0.58 (95% CI, 0.45–0.74) and 0.66 (95% CI, 0.54–0.80), respectively. Similarly, average unadjusted ICU and hospital LOS also was not significantly different between the pre- and postintervention groups, but adjustment for acuity by comparing observed-over-expected LOS ratios (through APACHE-IVa methodology) yielded significantly lower ICU and hospital LOS in the postintervention group (Table 2).

TABLE 2.

Mortality and Length of Stay Outcomes Before and After Intervention of Telemedicine ICU Service

Unadjusted ICU and hospital mortality rates were not statistically significantly different between the pre- and postintervention period for the subgroups of patients admitted during daytime versus nighttime hours (Table 3). When adjusted for APACHE-IVa scores by logistic regression, ICU mortality ORs for nighttime ICU admissions were significantly lower post intervention (OR, 0.45 [95% CI, 0.33–0.61] p < 0.001), whereas for daytime ICU admissions, they were not (OR, 0.81 [95% CI, 0.55–1.20] p = 0.292). Hospital mortality ORs for both nighttime and daytime ICU admissions were significantly lower post intervention: (OR, 0.57 [95% CI, 0.44–0.74] p < 0.001) and (OR, 0.73 [95% CI, 0.55–0.97] p = 0.033), respectively (Table 3).

TABLE 3.

Unadjusted and Acuity-Adjusted ICU and Hospital Mortality Rates Pre- Versus Post Telemedicine ICU Intervention for the Subgroups of Daytime Versus Nighttime ICU Admissions

Preintervention ICU best practice adherence rates compared with postintervention best practice adherence rates after 1, 2, and 3 years of tele-ICU support (2016, 2017, 2018) are shown in Table 4, (A). Venous thromboembolism prophylaxis, stress ulcer prophylaxis, and ARDSnet lung-protective ventilation compliance rates are shown in percent of eligible patients. Glycemic control performance data are shown as percent of time weighted daily glucose averages of less than 180 mg/dL. Transfusion threshold compliance is shown as percent of packed RBC transfusions triggered by hemoglobin values of less than 7 mg/dL. All patients with active hemorrhage, acute coronary syndrome, trauma in the first 24 hours, and sepsis in the first 24 hours were excluded from the transfusion best practice statistic.

TABLE 4.

Pre- and Post Telemedicine ICU Intervention Best Practice Adherence, Telemedicine ICU Performance Metrics Related to Patient Outcomes, and Telemedicine ICU Engagement Volume and Interventions by Category

Table 4, (B) shows tele-ICU provider specific performance data, specifically the average time for tele-registered nurses and tele-intensivists to conduct a first comprehensive video assessment of newly admitted patients for 2016, 2017, and 2018. Table 4, (C) shows quantitative data on tele-ICU engagement, that is, number of admission evaluations, video assessments, and interventions by category. Interventions can either be initiated by the bedside team or the tele-ICU team.

Figure 1 shows the cumulative CVC utilization duration in days by year and quarter for phase II of the CVC utilization initiative (commenced in 2018). Discussions between the tele-ICU intensivist and the bedside teams on the continued indication for CVC utilization and on alternatives resulted in the elimination of long duration CVC usage and in a reduction of the average CVC duration from 18.26 ± 1.33 days in 2018Q1 to 12.82 ± 0.39 days in 2019Q2 (Group comparison by one-way ANOVA: p < 0.0001). CLABSI rates were calculated as standardized infection ratios (SIRs) as per National Healthcare Safety Network Database. The CVC utilization initiative reduced the average preintervention SIR (National Healthcare Safety Network Database) across all participating ICUs from 15% below predicted (SIR 0.85) in 2016 to 47% below predicted (SIR 0.53) during phase I of the CVC utilization initiative in 2017 and 41% below predicted (SIR 0.59) during phase II of the CVC utilization initiative in 2018.

Figure 1.

Central venous catheter duration in days by year and quarter (mean and sem depicted in red). Standardized telemedicine ICU interventions were commenced in 2018 and resulted in the elimination of long catheter durations as well as in reduction of average durations from 18.26 ± 1.33 days in 2018Q1 to 12.82 ± 0.39 days in 2019Q2 (mean ± sem). Group comparison by one-way analysis of variance: p < 0.0001.

DISCUSSION

Our study confirms favorable effects of tele-ICU intervention on ICU and hospital mortality and LOS in a healthcare system with academic affiliation, composed of a tertiary and quaternary University hospital and a community hospital with a wide geographical catchment area in the northeastern United States. In addition to the setting, we also designed our tele-ICU components and performance goals largely based on the evidence reported by Lilly et al (9, 16). The major difference between the two interventions is that they happened about a decade apart from each other. Our ICU and hospital mortality rates compare favorably with data reported by Lilly et al (16) with comparable acuity as per APACHE-IVa scores. Lilly et al (16) observed a 28.9% increase in average APACHE-IVa scores from pre- to postintervention phase (2005–2006 to overlapping 2006–2007). Almost exactly 10 years later, we encountered a comparably smaller increase of 18.3%. Compared with Lilly et al (16), we observed a much larger proportion of intermediate interventions, which may be due to differences in workflow priorities and/or documentation thresholds.

The statistically significant decrease in average patient age in the postintervention group may be related to a relative increase in overall transfer volume to our tertiary and quaternary medical center for advanced procedures in the postintervention period. Our network has seen an increase in overall ICU transfer volume from 590 cases per month in 2015 to 690 cases per month in 2018. The neurointerventional service line, admitting to the neuroscience ICU, has seen a 12% increase in transfer volume with an average patient age of 52.3. The increase of neurologic APACHE-IVa admission diagnoses from 20.2% to 22.6% corresponds to this systemwide increase in neurologic and neurosurgical case volume we have observed over the study period.

The combination of increased acuity and decreased patient age in the postintervention period might be related to changes in ICU resource utilization over the time period investigated, restricting ICU resources to younger and sicker patients. Lilly et al (16) by comparison had not only found a significant increase in average patient age in the tele-ICU intervention group but also found significantly increased acuity by average APS and APACHE-IVa scores by 39.4% and 28.9%. The increase in acuity of 28.9% seen by Lilly et al (16) is much larger than the 18.3% increase we observed in our study. The manual preintervention abstraction methodology was optimized to be analogous to the postintervention abstraction, but the nature of the abstraction is retrospective and manual for the preintervention group and prospective, real-time, and electronic for the postintervention group. The overall low number in the “Sepsis of unknown source” category limits the interpretability of pre-post differences, but we speculate that they may relate to a difference in the likelihood that retrospective abstraction will still not have assigned an organ system to the sepsis diagnosis compared with the prospective and real-time abstraction technique.

In a subgroup analysis of daytime versus nighttime ICU admissions, the postintervention period was associated with a significantly lower OR of ICU and hospital mortality for patients admitted during nighttime hours, whereas for patients admitted during daytime hours, a significantly lower OR of hospital mortality but not ICU mortality was observed. The finding that the tele-ICU intervention improved the ORs for ICU and hospital mortality disproportionately in patients admitted at low intensity bedside staffing, that is, nighttime hours is important as it may imply that the tele-ICU can function to augment low intensity bedside ICU physician staffing hours. Pronovost et al (17) have shown outcomes improvement with high intensity ICU physician staffing compared with low intensity staffing. In our view, adding tele-ICU support can effectively turn a low intensity bedside staffing ICU into a high intensity staffing ICU. In addition to staffing models, the issue of ICU operational models may be associated with differential tele-ICU effects on outcomes. Young et al (8) have raised the question as to whether tele-ICUs confer different outcomes benefits in “closed” versus “open” ICUs. Our study setup did not allow a robust assessment of whether tele-ICU interventions differentially effect outcomes in “closed”, “open”, or hybrid ICUs, as only one ICU in our system operates under a “closed” model (Trauma-ICU). Larger multicenter studies are needed to address this potential association.

We have learned that the value that tele-ICUs can add depends on how the intervention is planned and implemented. Important elements are administrative buy-in and support, consensus on care standardization, and periodic review of performance data (9). Table 4, (A) illustrates the positive effect of tele-ICU care standardization on ICU best practices for our healthcare system. Tele-ICU clinician evaluation of newly admitted ICU patients within 1 hour of their admission has been shown to correlate with improved ICU mortality and LOS (9). Table 4, (B) illustrates how the timely use of performance data through periodic review with the tele-ICU physicians and nurses improved the average time to new patient evaluation. On a global level, Table 4, (C) shows quantitative data on new patient evaluations, interventions, and video interactions and illustrates that the tele-ICU intervention of this study represents a very active system with on average between 10.3 (2016) and 15.4 (2018) engagements per patient.

A major limitation of this study is its retrospective observational design with the resulting inability to prospectively randomize patients. We cannot conclude causality between any of the best practice or tele-ICU performance metrics. We implemented the intervention in all ICUs making a difference in differences analysis not feasible. In totality, though a causal relationship is very plausible at the least. Our study was a single-center study. Our results are similar to a prior single-center study, thereby reducing the risk of single-center study report bias (16).

We agree with the importance of disclosure of any potential financial conflicts of interest and funding sources as mandated by Young et al (8, 18, 19). We report no conflicts of interest or external funding for this study.

CONCLUSIONS

In this pre-post observational study, tele-ICU intervention was associated with improvements in care standardization, adherence to ICU best practices, and decreases in ICU and hospital mortality and LOS. The mortality benefits were mediated at least in part through tele-ICU supplementation of low intensity bedside staffing hours.