Abstract
Background
Intensive care unit (ICU) telemedicine is an increasingly common strategy for improving the outcome of critical care, but its overall impact is uncertain.
Objectives
To determine the effectiveness of ICU telemedicine in a national sample of hospitals and quantify variation in effectiveness across hospitals.
Research design
We performed a multi-center retrospective case-control study using 2001–2010 Medicare claims data linked to a national survey identifying United States hospitals adopting ICU telemedicine. We matched each adopting hospital (cases) to up to 3 non-adopting hospitals (controls) based on size, case-mix and geographic proximity during the year of adoption. Using ICU admissions from 2 years before and after the adoption date, we compared outcomes between case and control hospitals using a difference-in-differences approach.
Results
132 adopting case hospitals were matched to 389 similar non-adopting control hospitals. The pre- and post-adoption unadjusted 90-day mortality was similar in both case hospitals (24.0% vs. 24.3%, p=0.07) and control hospitals (23.5% vs. 23.7%, p<0.01). In the difference-in-differences analysis, ICU telemedicine adoption was associated with a small relative reduction in 90-day mortality (ratio of odds ratios: 0.96, 95% CI = 0.95–0.98, p<0.001). However, there was wide variation in the ICU telemedicine effect across individual hospitals (median ratio of odds ratios: 1.01; interquartile range 0.85–1.12; range 0.45–2.54). Only 16 case hospitals (12.2%) experienced statistically significant mortality reductions post-adoption. Hospitals with a significant mortality reduction were more likely to have large annual admission volumes (p<0.001) and be located in urban areas (p=0.04) compared to other hospitals.
Conclusions
Although ICU telemedicine adoption resulted in a small relative overall mortality reduction, there was heterogeneity in effect across adopting hospitals, with large-volume urban hospitals experiencing the greatest mortality reductions.
Keywords: critical care, intensive care units, telehealth, mechanical ventilation
INTRODUCTION
Treatment in an intensive care unit (ICU) staffed by appropriately trained intensivist clinicians improves survival in critically ill patients.1 However, many patients lack access to this level of care, particularly in small hospitals and in rural areas.2 ICU telemedicine is an innovative critical care delivery approach specifically designed to address this problem.3 Using ICU telemedicine, trained intensivist clinicians in regional hubs can monitor and treat patients at remote hospitals in conjunction with the bedside care team, potentially improving the overall quality of critical care.4 Based on its initial promise, the use of ICU telemedicine has expanded dramatically in recent years, with over 10% of all ICU beds in the United States covered by a telemedicine program.5
Despite this rapid expansion, concerns persist about the effectiveness of ICU telemedicine.6 Existing studies are limited, with most data coming from single center before-after studies that lack concurrent controls.7 Moreover, the existing literature is conflicting, with some studies showing substantial reductions in mortality8,9 and others showing no benefit.10,11 To better understand this issue, we conducted a national study of ICU telemedicine effectiveness using Medicare claims data, examining mortality before and after the introduction of ICU telemedicine in a large number of adopting hospitals and comparing these temporal changes to control hospitals that did not adopt ICU telemedicine. Given the conflicting results across existing studies, we assessed not only the national effects of ICU telemedicine but also the effects at individual adopting hospitals, quantifying variation in program effectiveness.
METHODS
Study design and data sources
We performed a retrospective study of fee-for-service Medicare beneficiaries admitted to US hospitals between 2001 and 2010. Patient-level data on hospital admissions were obtained from the Centers for Medicare and Medicaid Services (CMS) Medicare Provider Analysis and Review (MedPAR), which as the only national data source of US hospital admissions was a unique resource for this study. We obtained patient death dates from the Medicare Beneficiary Summary File, and hospital characteristics (including the hospital bed counts, ICU bed counts, and geographic locations) from the CMS Healthcare Cost Reporting Information System (HCRIS).
We obtained data on which hospitals adopted ICU telemedicine and their individual dates of adoption from a previously published list of ICU telemedicine sites in the US.12 This list was created through a comprehensive working group composed of representatives from critical care professional societies, commercial telemedicine vendors and early adopting hospitals, and the list development process included multiple validation steps to ensure accuracy and completeness.12 The final list enumerated hospitals that are the targets of a telemedicine program (i.e. they housed the ICU patients receiving care under telemedicine), independent of whether the hospital also housed the ICU support center. The list only included programs that performed continuous monitoring of ICU patients, rather than programs that performed purely periodic consultation, since consultation-only programs are systematically different than programs that also involve continuous monitoring.13 From the list we excluded Veterans Affairs hospitals which are not included in MedPAR.
Although ICU telemedicine may hold greatest promise in small rural hospitals, we chose not to focus exclusively on those hospitals in order to best understand the impact of ICU telemedicine as it is currently deployed, which includes hospitals both large and small; and hospitals located in both urban and rural areas.5
Description of ICU telemedicine
All adopting hospitals but one used a telemedicine system provided by the predominant private vendor (eICU, Phillips, the Netherlands). Components of this system include one- or two-way videoconferencing with remote-controlled cameras and audio speakers in each covered ICU bed, real-time streaming of vital signs from the bedside monitors, smart alarms for recognition of physiological deterioration, and a comprehensive electronic health record.14 This system enables nurses and physicians in the support center to continuously monitor critically ill patients and rapidly communicate with the bedside team as necessary. Within this framework programs may have varied with respect the make-up of the support center team, the hours the support center was staffed, and the specific activities of the support center clinicians.
Patients
All hospital admissions in MedPAR involving an ICU stay were initially eligible for the analysis. We identified ICU stays using ICU-specific revenue codes.15 To increase homogeneity in the Medicare sample we limited the analysis to patients aged over 65 years at admission. To avoid interdependence of observations, when patients had multiple hospital admissions involving ICU stays we randomly selected one admission. We used MedPAR to determine patient age; sex; admission source (categorized as direct, emergency department, inter-hospital transfer and others); comorbidities in the manner of Elixhauser;16 mechanical ventilation using International Classification of Diseases version 9.0—Clinical Modification procedure codes;17 surgical vs. medical status using All-Patient Refined Diagnosis Related Groups (APR-DRGs); discharge location (categorized as home, post-acute care, acute care transfer, dead and hospice); ICU length of stay, and hospital length of stay.
Hospitals and cohort construction
Of hospitals that adopted ICU telemedicine, we excluded hospitals that did not have at least 120 ICU patients in the years before, during and after adoption. We made this decision to prevent low reliability of mortality estimates at very small hospitals from biasing our results. Additionally, so that we would have 2 years of lead-in and follow-up time for each hospital, we excluded hospitals that adopted ICU telemedicine before 2003 or after 2009. Finally, we excluded hospitals for which there was uncertainty about adoption timing, including hospitals for which we could not determine the exact adoption dates, hospitals in which not all ICUs were covered by telemedicine, and hospitals in which different ICUs were covered at different times.
As our control group we selected non-adopting hospitals that were similar in size, case-mix and geography to adopting hospitals during the year of case hospital adoption.18 To identify controls we stratified all US hospitals by 7 key characteristics determined from either MedPAR and HCRIS: number of hospital beds (<250 or ≥250), percent of hospital beds devoted to the ICU (<10 or ≥10), teaching status of the hospital based on resident full-time equivalents (0, non-teaching vs. >0, teaching); level of urbanization based on Medicare’s classification scheme (rural vs. urban or suburban); annual ICU admission count (<500 vs. ≥500); percentage of ICU patients with a surgical APR-DRG (<20 or ≥20); and percentage of ICU patients receiving mechanical ventilation (<15 vs. 15). We chose these cut points as natural cut points that were at or near the median. This step created 128 unique strata. At this stage we chose not to stratify by state or region, as such a strategy would have led to too few hospitals in each stratum.
Next, we matched each case hospital to up to 3 control hospitals that were in the same strata as the case hospital during the year of adoption. Hospitals were only eligible as controls if they also had at least 120 ICU patients in the years before, during and after the adopting hospital’s adoption year. We randomly selected control hospitals from the pool of eligible hospitals within the same state as the adopting case hospital. If we could not identify 3 control hospitals within the state, we broadened the eligibility to neighboring states, then to US census regions, and then to entire nation, thereby minimizing geographic differences between case hospitals and control hospitals.
We excluded case hospitals that did not have at least 1 eligible control hospital after the matching procedure was complete. We excluded patients admitted in the three months before and after the adoption date for both case hospitals and their matched control hospitals to address potential variance surrounding the implementation periods. The final sample included all eligible ICU admissions in the two years immediately before and after this 6-month implementation period, for a total of four years of data in each hospital.
Analysis
We compared hospital and patients characteristics across case and control hospitals using t-tests, chi-square tests, or Fisher’s exact tests, as appropriate. We used multivariate logistic regression to determine the independent association between ICU telemedicine adoption on 90-day mortality from the date of hospital admission. This time period was chosen because it is sufficiently close to the admission date to be influenced by the quality of ICU care, but sufficiently distal to not be influenced by variation in post-acute care utilization.19 To estimate the effect of ICU telemedicine on 90-day mortality in case hospitals we used a difference-in-differences regression approach that adjusts for underlying trends in outcomes, assuming that those trends are not different between case and control hospitals.20 Under this approach, the estimator of interest was the interaction term between period (pre/post) and adoption status (case/control). We accounted for clustering within matched hospitals using conditional regression.21 All regression models controlled for age (as linear splines), sex, admission source, and Elixhauser comorbidities.
We performed two primary regressions. First, we estimated the national effect of ICU telemedicine by grouping all hospitals together. This difference-in-difference estimate represents the population-averaged effect of ICU telemedicine. We exponentiated the regression coefficient of the difference-in-difference estimator and present the results as a ratio of odds ratios.20 We also used indirect standardization to calculate adjusted mortality rates in the pre- and post-adoption period for both case and control hospitals.22 To specifically examine the effect at small, rural hospitals which may be particularly likely to benefit from remote ICU monitoring we repeated this analysis in three subgroups: hospitals located in rural areas (according to Medicare designation); non-teaching hospitals (resident-to-bed ratio = 0); and small hospitals (<100 total beds).
Second, we estimated the effect of ICU telemedicine in each individual case hospital. Here, we performed a separate difference-in-differences regression for each case hospital and its matched controls. Again, we exponentiated the regression coefficient of the difference and difference estimator and present the results as ratios of odds ratios.20 To understand variation in the effect of ICU telemedicine, we plotted the ratio of odds ratios according to hospital’s relative effectiveness rank. To understand whether selected hospital characteristics were associated with a greater telemedicine effect, we divided all case hospitals into three groups based on whether their mortality significantly increased, decreased, or remained unchanged post-adoption. We compared hospital characteristics across these groups using chi-squire tests or ANOVA, as appropriate. Hospital characteristics of interest were selected a priori and included year of adoption, hospital bed count, academic status, ICU admission count, percentage of ICU admissions with surgical APR-DRGs, percentage of ICU admissions receiving mechanical ventilation, level of urbanization, and metropolitan statistical area size.
The University of Pittsburgh Institutional Review Board reviewed and approved this research. Data management and statistical analyses were performed in SAS 9.4 (Cary, NC). A p-value of ≤0.05 was considered significant.
Role of the funding source
The funding source played no role in the design, conduct and reporting of this study.
RESULTS
Of 5,650 acute care hospitals in MedPAR during the study period, 215 (3.8%) hospitals adopted ICU telemedicine. Of the adopting hospitals we excluded 83: 60 with small admission volumes, 13 for which exact adoption dates could not be determined, 9 for which not all ICUs were covered by telemedicine, and 1 for which we could not find a suitable control, leaving 132 case hospitals in the final analysis. We matched these hospitals to 389 control hospitals: 126 cases (95.4%) had 3 controls, 5 cases (3.8%) had 2 controls, and 1 case (0.8%) had 1 control. Of control hospitals, 204 (52.4%) were within the case hospitals’ state, 123 (31.6%) were within a neighboring state, 40 (10.3%) were outside a neighboring state but within the same census region, and 22 (5.7%) were outside the census region. Hospital characteristics were well-matched between cases and controls (Table 1).
Table 1.
Hospital characteristics categorized by adoption status
| Characteristics | Adopting hospitals (n=132) |
Non-adopting hospitals (n=389) |
p-value |
|---|---|---|---|
| Adoption year | |||
| 2003 – 2004 | 26 (19·7) | 77 (19·9) | 0·96 |
| 2005 – 2007 | 85 (64·1) | 250 (64·3) | |
| 2008 – 2009 | 21 (15·9) | 62 (15·9) | |
| Hospital Beds | 194 ± 161 | 185 ± 118 | 0·51 |
| ICU Beds | 29 ± 33 | 27 ± 27 | 0·65 |
| Academic Status (n, %) | |||
| Non-teaching | 82 (62·1) | 230 (59·1) | 0·74 |
| Small Teaching | 31 (23·5) | 105 (27·0) | |
| Large Teaching | 19 (14·4) | 54 (13·9) | |
| ICU Admissions | 849 ± 866 | 834 ± 859 | 0·87 |
| ICU Admissions receiving MV | 134 ± 143 | 125 ± 107 | 0·47 |
| Percent Surgery | 28·3 ± 10·1 | 28·2 ± 10·2 | 0·91 |
| Urbanicity (n, %) | |||
| Urban | 111 (84·1) | 327 (84·0) | 0·99 |
| Rural | 21 (15·9) | 62 (16·0) | |
| MSA Size (n, %) | |||
| Small | 20 (15·2) | 71 (18·3) | 0·55 |
| Medium | 49 (37·1) | 138 (35·5) | |
| Large | 63 (47·7) | 180 (46·3) | |
| Census Region (n, %) | |||
| Northeast | 14 (10·6) | 43 (11·1) | 0·45 |
| Midwest | 68 (51·5) | 172 (44·2) | |
| South | 20 (15·2) | 82 (21·1) | |
| West | 30 (22·7) | 92 (23·7) |
All values refer to the year of adoption (for adopting hospitals) or the corresponding year (for matched non-adopting control hospitals)· Values are frequency (percent) or mean ± standard deviation
ICU = intensive care unit; MV = mechanical ventilation; MSA = metropolitan statistical area
Patient characteristics between case and control hospitals in both the pre and post-adoption periods are shown in Table 2. Comparing patient characteristics between case and control hospitals in the pre-adoption period, patients in case hospitals were older, had more co-morbid conditions, were more likely to require mechanical ventilation, had longer hospital lengths of stay, and had higher unadjusted 90-day mortality. Comparing patient characteristics in the case hospitals in their pre and post-adoption periods, patients post-adoption were older, were more likely to be admitted from the emergency department or an outside hospital, had more comorbidities, were more likely require mechanical ventilation, and had longer ICU and hospital lengths of stay. For both comparisons many differences were small and not likely to be clinically significant.
Table 2.
Patient characteristics categorized by adoption status and time period.
| Adopting hospitals | Non-adopting hospitals | P-values | ||||
|---|---|---|---|---|---|---|
| Variable | Pre-period (n=147517) |
Post-period (n=145119) |
Pre-period (n=419466) |
Post-period (n=411461) |
Pre-period Adopting vs. non-adopting |
Adopting Pre-period vs. post-period |
| Age | 77·9 ± 7·4 | 78·0 ± 15·3 | 77·8 ± 7·4 | 78·0 ± 7·6 | <0·01 | <0·01 |
| Female | 74937 (50·8) | 73503 (50·7) | 213876 (51·0) | 208566 (50·7) | 0·21 | 0·42 |
| Race | ||||||
| White | 131754 (89·3) | 128686 (88·7) | 373177 (89·0) | 364356 (88·6) | 0·02 | <0·01 |
| Black | 10351 (7·0) | 10711 (7·4) | 31042 (7·4) | 30741 (7·5) | ||
| Other | 5412 (3·7) | 5722 (3·9) | 15247 (3·6) | 16364 (4·0) | ||
| Admission Source | ||||||
| Direct | 51023 (34·6) | 46568 (32·1) | 151974 (36·2) | 143466 (34·9) | 0·26 | <0·01 |
| ED | 80787 (54·8) | 80690 (55·6) | 226235 (53·9) | 225088 (54·7) | ||
| Other Hospital | 12537 (8·5) | 13610 (9·4) | 31270 (7·5) | 31696 (7·7) | ||
| Other | 3170 (2·1) | 4251 (2·9) | 9987 (2·4) | 11203 (2·7) | ||
| Comorbidities (count) | ||||||
| 0 | 12267 (8·3) | 10933 (7·5) | 36454 (8·7) | 32366 (7·9) | <0·01 | <0·01 |
| 1 | 35764 (24·2) | 33721 (23·2) | 103629 (24·7) | 98241 (23·9) | ||
| 2 | 44195 (30·0) | 42705 (29·4) | 125562 (29·9) | 121301 (29·5) | ||
| 3+ | 55291 (37·5) | 57760 (39·8) | 153821 (36·7) | 159553 (38·8) | ||
| Comorbidities | ||||||
| CHF | 7440 (5·0) | 15053 (10·4) | 20657 (30·6) | 41092 (10·1) | 0·07 | <0·01 |
| COPD | 37308 (25·3) | 34063 (23·5) | 104772 (25·0) | 95491 (23·2) | 0·02 | <0·01 |
| Diabetes mellitus | 26513 (18·0) | 25026 (17·2) | 76760 (18·3) | 74483 (18·1) | 0·01 | <0·01 |
| Liver disease | 1543 (1·0) | 1542 (1·1) | 4297 (1·0) | 4343 (1·1) | 0·48 | 0·66 |
| Metastatic cancer | 4811 (3·3) | 4826 (3·3) | 12864 (3·1) | 13039 (3·2) | <0·01 | 0·33 |
| Other cancer | 5504 (3·7) | 3967 (2·7) | 15518 (3·7) | 10612 (2·6) | 0·58 | <0·01 |
| MV | 22213 (15·1) | 22790 (15·7) | 61864 (14·7) | 62633 (15·2) | <0·01 | <0·01 |
| ICU length of stay | 4·7 ± 6·6 | 5·0 ± 6·6 | 4·8 ± 6·6 | 4·8 ± 6·4 | 0·04 | <0·01 |
| Hospital length of stay | 8·6 ± 9·3 | 8·3 ± 8·9 | 8·2 ± 9·1 | 7·8 ± 8·7 | <0·01 | <0·01 |
| Discharge Location | ||||||
| Home | 82344 (55·8) | 77334 (53·3) | 237769 (56·7) | 222969 (54·2) | <0·01 | <0·01 |
| Post-acute care | 37375 (25·3) | 39808 (27·4) | 106016 (25·3) | 112640 (27·4) | ||
| Dead | 19621 (13·3) | 18618 (12·8) | 53011 (12·6) | 50414 (12·3) | ||
| Acute care | 5056 (3·4) | 4492 (3·1) | 14462 (3·4) | 12976 (3·2) | ||
| Hospice | 3121 (2·1) | 4822 (3·3) | 8195 (2·0) | 12382 (3·0) | ||
| 90-Day Mortality | 35422 (24·0) | 35265 (24·3) | 98370 (23·5) | 97584 (23·7) | <0·01 | 0·07 |
All values are mean ± standard deviation or frequency (percent).
ED = emergency department; CHF = congestive heart failure; COPD = chronic obstructive lung disease; MV = mechanical ventilation; ICU = intensive care unit
In the national differences-in-differences analysis, adoption of ICU telemedicine was associated with a small but statistically significant relative reduction in the overall odds of 90-day mortality (ratio of odds ratios = 0.96, 95% confidence intervals 0.94–0.98, p<0.001). The adjusted probabilities show that although the risk of mortality increased slightly in case and control hospitals after the adoption time, the risk increased less in case hospitals (Figure 1). These results were similar in the subgroup analyses focusing on hospitals theoretically most likely to benefit from remote ICU monitoring (Table 3).
Figure 1.
Adjusted mortality before and after the adoption period in both adopting (n= 132) and non-adopting (n=389) hospitals. Models are adjusted for age, gender, admission source, and patient comorbidities. Error bars indicate 95% confidence intervals.
Table 3.
The effect of telemedicine on 90-day mortality in all hospitals and in three subgroups: rural hospitals, non-teaching hospitals, and hospitals with <100 beds, adjusted for patient characteristics.
| Hospital Group | Case Hospitals |
Control hospitals |
Ratio of odds ratios (95% CI) |
P-value |
|---|---|---|---|---|
| All eligible hospitals | 132 | 389 | 0.96 (0.94 – 0.98) | <0.01 |
| Rural hospitals | 21 | 62 | 1.06 (0.99 – 1.13) | 0.09 |
| Non-teaching hospitals | 82 | 230 | 0.97 (0.95 – 1.01) | 0.11 |
| Small hospitals (<100 beds) | 37 | 102 | 0.97 (0.92 – 1.03) | 0.36 |
In the hospital specific difference-in difference analyses we found wide variation in the effect of ICU telemedicine across individual hospitals (median ratio of odds ratios: 1.01; interquartile range 0.85–1.12; range 0.45–2.54; Figure 2). Of the 132 case hospitals, 16 (12.1%) had statistically significantly reduced mortality post-adoption, 107 (81.1%) had no statistically significant change in mortality, and 8 (6.1%) had statistically significantly increased mortality (Table 3). Hospitals with a statistically significant reduction in mortality tended to have larger admission volumes (<0.01), and be located in large urban areas (0.04).
Figure 2.
Hospital specific ratios of odds ratios indicating relative effect of ICU telemedicine accounting for case-mix and temporal trends. Hospitals are ranked according to their relative effect. Hospitals below 1.0 demonstrated a relative reduction in mortality after adoption, and hospitals above 1.0 demonstrated a relative increase after telemedicine. Error bars indicate 95% confidence intervals. The dotted horizontal line indicates the national difference-in-differences estimate.
DISCUSSION
In a national multi-center study of ICU patients we found that adoption of ICU telemedicine was associated with a small but statistically significant relative reduction in the odds of 90-day mortality. Although mortality rose slightly in both groups, the magnitude of the increase was smaller in case hospitals. However, we also found wide heterogeneity in the effect of ICU telemedicine across adopting hospitals, with most hospitals seeing no significant effect, some hospitals seeing reduced mortality, and a few hospitals seeing increased mortality. Large, urban hospitals tended to see greater benefit than other hospitals.
Our results significantly add to the existing ICU telemedicine literature, which is predominantly characterized by small studies with few sites. Most of these studies lack concurrent controls and thus are confounded by temporal trends, and fail to follow patients past hospital discharge and thus are biased by differential discharge practices across hospitals. We improve upon those studies by both including contemporaneous controls and using 90-day mortality as our endpoint, increasing the validity of our findings.
Our results also provide important context to the existing telemedicine literature. Many published studies show dramatic mortality reductions,8,9 however, others show no benefit11 and still others show increased mortality in some patients10 or hospitals.12 By studying a large number of adopting hospitals, we demonstrate that the treatment heterogeneity evident in the single-center literature is not necessarily an artifact of study design but is instead an inherent characteristic of ICU telemedicine, with some ICUs greatly benefiting from the technology and others receiving no significant benefit.
There are several potential mechanisms for this heterogeneity. For hospitals with reduced mortality, ICU telemedicine may promote evidence-based practice via prompting and checklists,23,24 facilitate early recognition and treatment of physiological deterioration,25 and improve care coordination between interprofessional care providers.26 For hospitals in which telemedicine did not affect mortality, the technology may be underutilized, with infrequent contact between the ICU telemedicine “hub” unit and the target ICUs27 or skepticism among the ICU staff that the technology is useful.28 For hospitals with increased mortality, ICU telemedicine may disrupt communication, as can occur after introduction of new technology.29 ICU telemedicine may also lead to “diffusion of responsibility” between the ICU team and the telemedicine team, a phenomenon that occurs when too many people are responsible for the same task, in this case monitoring critically ill patients, leading to neglect.30
Our study provides new insight that explains some of this heterogeneity. In particular, we found that telemedicine was most effective in large urban hospitals. This result contrasts with the conventional wisdom that telemedicine is particularly useful for bringing medical expertise to patients in rural areas separated by large distances from urban referral centers.31 It is possible that higher volume centers may gain greater experience with ICU telemedicine which in turn translates into improved outcomes.32 Additionally, large urban hospitals may staff the ICU telemedicine unit with nurses and physicians that also work in the target ICUs, engendering trust and improving communication between the telemedicine and bedside clinicians.33 Yet much of this heterogeneity remains unexplained. More research is needed to understand variation in program effectiveness and develop strategies to improve quality in existing programs and optimize impact in new programs.
A key limitation of our study was exclusion of hospitals with case volumes less than 120 patients per year. We made this decision to maximize the internal validity of our results, but in doing so we excluded small hospitals where, at least in theory, the benefit of telemedicine might be greatest. Small hospitals are less likely to be staffed by trained intensivists and are known to suffer worse risk-adjusted outcomes compared to larger hospitals.2,32 Accordingly they might be most likely to benefit from remote ICU monitoring. Unfortunately, due to issues of sample size and reliability, quantitative methods are poorly suited for understanding system-level changes in very small hospitals. Future qualitative work should be directed at understanding the effectiveness of ICU telemedicine in these hospitals.
At the same time, it’s important to note that at present, the majority of ICU telemedicine use is not in small, rural hospitals. Our study therefore reports on telemedicine where it is currently deployed, and not in the sub-cohort where use is not well established but in which greater use-benefit may be realized. Although we show that the impact of telemedicine is, perhaps paradoxically, greatest at large hospitals, it is possible that the modest overall impact we observed may be due to its current relatively low use at very small rural hospitals. However, such differential impact can be proven only after additional adoption in this group.
Our work has several other limitations. First, we used an administrative database that lacked detailed clinical risk adjustment. However, by matching hospitals based on size, case-mix and geography, and by using a difference-in-differences approach, we were likely able to partially mitigate differences in severity of illness across adopting case hospitals and non-adopting controls. Second, we studied a Medicare population aged 65 and over. Although these patients comprise the majority of adult ICU admissions nationally, our work may not generalize to younger patients. Third, our only outcome was mortality. Although mortality is arguably the most important outcome of intensive care, it is possible that telemedicine impacts other patient-centered outcomes such as quality of death and dying.34,35 Fourth, we only studied hospitals that used telemedicine for continuous monitoring of ICU patients. Future research is necessary to understand other models of ICU telemedicine, such as periodic consultation models. Fifth, we did not have data on the specific activities of the telemedicine clinicians, and therefore can’t describe the telemedicine “dose” or it’s effect on program effectiveness. However, almost every program in the study used a single telemedicine vendor, such that there is at least standardization of the technology across sites.
Despite these limitations, our work provides important insight into the effectiveness of ICU telemedicine, and in doing so can help guide future adoption. We show that, at least in some settings, ICU telemedicine has the potential to significantly improve outcomes in critical illness. Yet at the same time gains in outcome are not assured and there may be risk for harm. It is incumbent on ICU telemedicine programs to monitor effectiveness and, when necessary, refine programs to ensure that they are leading to the intended improvements in processes and outcomes. The value of ICU telemedicine relates much more to how it is used than if is used.36 Given the dual needs to improve ICU outcomes and reduce costs, it is essential that we target new technologies and new care models to the hospitals most likely to benefit.
Table 4.
Hospital characteristics categorized by the effect ICU telemedicine on 90-day mortality
| Characteristics | Significantly Increased mortality (n=9) |
No mortality difference (n=107) |
Significantly reduced mortality (n=16) |
p-value |
|---|---|---|---|---|
| Adoption period | ||||
| 2003 – 2004 | 2 (22.2) | 16 (15.0) | 8 (50.0) | 0·16 |
| 2005 – 2007 | 4 (44.4) | 75 (70.0) | 6 (37.5) | |
| 2008 – 2009 | 3 (33.3) | 16 (15.0) | 2 (12.5) | |
| Hospital Beds | 195 ± 143 | 183 ± 142 | 265 ± 261 | 0·17 |
| Academic Status | ||||
| Non-teaching | 6 (66.7) | 68 (63.6) | 8 (50.0) | 0·85 |
| Small Teaching | 2 (22.2) | 24 (22.4) | 5 (31.3) | |
| Large Teaching | 1 (11.1) | 15 (14.0) | 3 (18.8) | |
| ICU Admissions | 1037 ± 1027 | 738 ± 640 | 1484 ± 1598 | <0·01 |
| Percent mechanical ventilation | 16.5 ± 8.3 | 16.3 ± 8.2 | 15.7 ± 8.11 | 0·96 |
| Percent Surgery | 25.3 ± 6.8 | 28.3 ± 10.4 | 29.7 ± 9.2 | 0·58 |
| Level of urbanization | ||||
| Urban | 4 (44.4) | 44 (41.1) | 12 (75.0) | 0·04 |
| Suburban | 3 (22.2) | 47 (44.0) | 2 (12.5) | |
| Rural | 3 (33.3) | 16 (15.0) | 2 (12.5) | |
| MSA Size | ||||
| Large | 4 (44.4) | 46 (43.0) | 13 (81.3) | 0·10 |
| Medium | 3 (33.3) | 45 (42.1) | 1 (6.3) | |
| Small | 2 (22.2) | 16 (15.0) | 2 (12.5) |
Values are mean ± standard deviation or frequency (percent)
ICU = Intensive Care Unit; MSA = Metropolitan Statistical Area
Acknowledgments
This work was funded by the United States National Institutes of Health (R01HL120980).
Footnotes
There are no financial conflicts of interest to disclose.
This work was first presented in abstract form at the American Thoracic Society International Conference in Denver Colorado in May, 2105
REFERNCES
- 1.Pronovost PJ, Angus DC, Dorman T, Robinson KA, Dremsizov TT, Young TL. Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA. 2002 Nov 6;288(17):2151–2162. doi: 10.1001/jama.288.17.2151. [DOI] [PubMed] [Google Scholar]
- 2.Angus DC, Shorr AF, White A, Dremsizov TT, Schmitz RJ, Kelley MA, et al. Critical care delivery in the United States: distribution of services and compliance with Leapfrog recommendations. Crit Care Med. 2006 Apr;34(4):1016–1024. doi: 10.1097/01.CCM.0000206105.05626.15. [DOI] [PubMed] [Google Scholar]
- 3.Nguyen Y-L, Kahn JM, Angus DC. Reorganizing adult critical care delivery: the role of regionalization, telemedicine, and community outreach. American Journal of Respiratory and critical care medicine. 2010 Jun 1;181(11):1164–1169. doi: 10.1164/rccm.200909-1441CP. [DOI] [PubMed] [Google Scholar]
- 4.Lilly CM, Zubrow MT, Kempner KM, Reynolds HN, Subramanian S, Eriksson EA, et al. Critical care telemedicine: evolution and state of the art. Crit Care Med. 2014 Nov;42(11):2429–2436. doi: 10.1097/CCM.0000000000000539. [DOI] [PubMed] [Google Scholar]
- 5.Kahn JM, Cicero BD, Wallace DJ, Iwashyna TJ. Adoption of ICU telemedicine in the United States. Crit Care Med. 2014 Feb;42(2):362–368. doi: 10.1097/CCM.0b013e3182a6419f. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Berenson RA, Grossman JM, November EA. Does Telemonitoring Of Patients--The eICU--Improve Intensive Care? Health Affairs. 2009 Sep 8;28(5):w937–w947. doi: 10.1377/hlthaff.28.5.w937. [DOI] [PubMed] [Google Scholar]
- 7.Wilcox ME, Adhikari NK. The effect of telemedicine in critically ill patients: systematic review and meta-analysis. Crit Care. 2012 Jul 18;16(4):R127. doi: 10.1186/cc11429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Breslow MJ, Rosenfeld BA, Doerfler M, Burke G, Yates G, Stone DJ, et al. Effect of a multiple-site intensive care unit telemedicine program on clinical and economic outcomes: An alternative paradigm for intensivist staffing*. Crit Care Med. 2004 Jan;32(1):31–38. doi: 10.1097/01.CCM.0000104204.61296.41. [DOI] [PubMed] [Google Scholar]
- 9.Lilly CM, Cody S, Zhao H, Landry K, Baker SP, McIlwaine J, et al. Hospital mortality, length of stay, and preventable complications among critically ill patients before and after tele-ICU reengineering of critical care processes. JAMA. 2011 Jun 1;305(21):2175–2183. doi: 10.1001/jama.2011.697. [DOI] [PubMed] [Google Scholar]
- 10.Thomas EJ, Lucke JF, Wueste L, Weavind L, Patel B. Association of telemedicine for remote monitoring of intensive care patients with mortality, complications, and length of stay. JAMA. 2009 Dec 23;302(24):2671–2678. doi: 10.1001/jama.2009.1902. [DOI] [PubMed] [Google Scholar]
- 11.Nassar BS, Vaughan-Sarrazin MS, Jiang L, Reisinger HS, Bonello R, Cram P. Impact of an Intensive Care Unit Telemedicine Program on Patient Outcomes in an Integrated Health Care System. JAMA Intern Med. 2014 Jul 1;174(7):1160. doi: 10.1001/jamainternmed.2014.1503. [DOI] [PubMed] [Google Scholar]
- 12.Fifer S, Everett W, Adams M, Vincequere J. In: Critical care, critical choices, the case for tele-ICUs in intensive care. King N, Coffman G, Handt J, Melissinos D, editors. Cambridge: New England Health Care Institute and Massachusetts Technology Collaborative; 2010. p. 68. [Google Scholar]
- 13.Rogove HJ, McArthur D, Demaerschalk BM, Vespa PM. Barriers to telemedicine: survey of current users in acute care units. Telemed J E Health. 2012 Jan;18(1):48–53. doi: 10.1089/tmj.2011.0071. [DOI] [PubMed] [Google Scholar]
- 14.Breslow MJ. Remote ICU care programs: current status. J Crit Care. 2007 Mar;22(1):66–76. doi: 10.1016/j.jcrc.2007.01.006. [DOI] [PubMed] [Google Scholar]
- 15.Iwashyna TJ. Critical care use during the course of serious illness. American Journal of Respiratory and critical care medicine. 2004 Nov 1;170(9):981–986. doi: 10.1164/rccm.200403-260OC. [DOI] [PubMed] [Google Scholar]
- 16.Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi J-C, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Medical care. 2005 Nov;43(11):1130–1139. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
- 17.Quan H, Parsons GA, Ghali WA. Validity of Procedure Codes in International Classification of Diseases, 9th revision, Clinical Modification Administrative Data. Medical care. 2004 Aug 1;42(8):801. doi: 10.1097/01.mlr.0000132391.59713.0d. [DOI] [PubMed] [Google Scholar]
- 18.Taylor JM. Choosing the number of controls in a matched case-control study, some sample size, power and efficiency considerations. Stat Med. 1986 Jan;5(1):29–36. doi: 10.1002/sim.4780050106. [DOI] [PubMed] [Google Scholar]
- 19.Vasilevskis EE, Kuzniewicz MW, Dean ML, Clay T, Vittinghoff E, Rennie DJ, et al. Relationship between discharge practices and intensive care unit in-hospital mortality performance: evidence of a discharge bias. Medical care. 2009 Jul;47(7):803–812. doi: 10.1097/MLR.0b013e3181a39454. [DOI] [PubMed] [Google Scholar]
- 20.Mullahy J. Interaction effects and difference-in-difference estimation in loglinear models. Cambridge: National Bureau of Economic Research; 1999. [Google Scholar]
- 21.Vittinghoff E, Glidden DV, Shiboski SC, McCulloch CE. Regression methods in biostatistics. New York: Springer; 2005. [Google Scholar]
- 22.Localio AR, Margolis DJ, Berlin JA. Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression. J Clin Epidemiol. 2007 Sep;60(9):874–882. doi: 10.1016/j.jclinepi.2006.12.001. [DOI] [PubMed] [Google Scholar]
- 23.Weiss CH, Moazed F, McEvoy CA, Singer BD, Szleifer I, Amaral LAN, et al. Prompting physicians to address a daily checklist and process of care and clinical outcomes: a single-site study. American Journal of Respiratory and critical care medicine. 2011 Sep 15;184(6):680–686. doi: 10.1164/rccm.201101-0037OC. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Kahn JM, Gunn SR, Lorenz HL, Alvarez J, Angus DC. Impact of nurse-led remote screening and prompting for evidence-based practices in the ICU. Crit Care Med. 2014 Apr;42(4):896–904. doi: 10.1097/CCM.0000000000000052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hravnak M, DeVita MA, Clontz A, Edwards L, Valenta C, Pinsky MR. Cardiorespiratory instability before and after implementing an integrated monitoring system. Crit Care Med. 2011 Jan;39(1):65–72. doi: 10.1097/CCM.0b013e3181fb7b1c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Kim MM, Barnato AE, Angus DC, Fleisher LA, Fleisher LF, Kahn JM. The effect of multidisciplinary care teams on intensive care unit mortality. Arch Intern Med. 2010 Feb 22;170(4):369–376. doi: 10.1001/archinternmed.2009.521. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Lilly CM, Fisher KA, Ries M, Pastores SM, Vender J, Pitts JA, et al. A national ICU telemedicine survey: validation and results. Chest. 2012 Jul;142(1):40–47. doi: 10.1378/chest.12-0310. [DOI] [PubMed] [Google Scholar]
- 28.Mullen-Fortino M, DiMartino J, Entrikin L, Mulliner S, Hanson CW, Kahn JM. Bedside nurses' perceptions of intensive care unit telemedicine. Am J Crit Care. 2011 Dec 31;21(1):24–31. doi: 10.4037/ajcc2012801. quiz32. [DOI] [PubMed] [Google Scholar]
- 29.Edmondson AC, Bohmer RM, Pisano GP. Disrupted Routines: Team Learning and New Technology Implementation in Hospitals. Administrative Science Quarterly. 2001 Dec;46(4):685. [Google Scholar]
- 30.Dayton E, Henriksen K. Communication failure: basic components, contributing factors, and the call for structure. Jt Comm J Qual Patient Saf. 2007 Jan;33(1):34–47. doi: 10.1016/s1553-7250(07)33005-5. [DOI] [PubMed] [Google Scholar]
- 31.Institute of Medicine. The role of telehealth in an evolving healthcare environment. Washington, D.C: National Academy Press; 2012. [Google Scholar]
- 32.Kahn JM, Goss CH, Heagerty PJ, Kramer AA, O'Brien CR, Rubenfeld GD. Hospital volume and the outcomes of mechanical ventilation. N Engl J Med. 2006 Jul 6;355(1):41–50. doi: 10.1056/NEJMsa053993. [DOI] [PubMed] [Google Scholar]
- 33.Moeckli J, Cram P, Cunningham C, Reisinger HS. Staff acceptance of a telemedicine intensive care unit program: a qualitative study. J Crit Care. 2013 Dec;28(6):890–901. doi: 10.1016/j.jcrc.2013.05.008. [DOI] [PubMed] [Google Scholar]
- 34.Reineck LA, Wallace DJ, Barnato AE, Kahn JM. Nighttime intensivist staffing and the timing of death among ICU decedents: a retrospective cohort study. Crit Care. 2013;17(5):R216. doi: 10.1186/cc13033. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kerlin MP, Harhay MO, Kahn JM, Halpern SD. Nighttime intensivist staffing, mortality, and limits on life support: a retrospective cohort study. Chest. 2015 Apr;147(4):951–958. doi: 10.1378/chest.14-0501. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Kahn JM, Hill NS, Lilly CM, Angus DC, Jacobi J, Rubenfeld GD, et al. The research agenda in ICU telemedicine: a statement from the Critical Care Societies Collaborative. Chest. 2011 Jul;140(1):230–238. doi: 10.1378/chest.11-0610. [DOI] [PMC free article] [PubMed] [Google Scholar]