The Effects of Health Information Technology Change Over Time: A Study of Tele-ICU Functions

Abstract

Objectives

Longitudinal studies exploring the evolution of health information technology functions provide valuable information about how technology systems are integrated and exploited in situ. This study reports changes in the distribution of functions for a specific health information technology, the tele-ICU, over time. The studied tele-ICU provided care to six remote ICUs within a local geographic region in the same state and had nursing staff around the clock.

Methods

The intervention logs of tele-ICU nurses were collected during two discrete times and coded into nine emergent functional categories, who initiated the intervention and, if required, subsequent escalation. All coded functional categories were investigated for significant changes over time in the nursing logged interventions.

Results

A total of 1927 interventions were coded into the nine emergent functional categories. Seven of the nine categories (78%) were significantly different between 2005 and 2007. The functions of the tele-ICU system continue to change and develop over time.

Conclusion

These findings suggest that the tele-ICU increased support when ICU nurses were off the unit, inter-hospital coordinating and adherence to best practices, while simultaneously decreasing real-time support for ICU nurses. This research suggests that sustaining safety features in a new technology over time have post-conditions after implementation.

1. Introduction

The traditional gold standard for research studies has been the Randomized Controlled Trial (RCT). For this reason, many systematic reviews limit study inclusion to RCTs, including for Health Information Technology (HIT; cf Misono et al., 2010 [1]). Conversely, research hypotheses framed in terms of causal inference are often not appropriate for complex, context-sensitive interventions with confounding interaction variables [2], which is the case for most, if not all, HIT interventions. One methodological solution for HIT intervention studies is to block the analysis by periods based upon when substantial planned modifications to the HIT are implemented (e.g. addition of drug-drug interaction checks to a Computerized Physician Order Entry system) [3]. Although this is an important method, there is currently no known methodology for justifiably blocking in HIT intervention changes over time for reasons other than design modifications. This paper begins to fill this gap through the study of a particular HIT, the tele-ICU.

Prior research suggests that people adapt their use of technology dynamically as part of a complex adaptive system, which includes both small adaptations and transformative changes in how work is configured [4, 5]. In other words, people in multiple roles and at multiple levels of an organization adapt their strategies, modify their roles and interactions, change performance expectations, and tailor technology devices to meet their needs in demanding operational environments [6, 7]. As one example, changes in user beliefs and attitudes may increase their willingness to use automated HIT functions over time [8].

The primary objective of the tele-ICU is to improve patient safety for critically ill patients via remote, real-time tele-monitoring of ICU patients by experienced critical care nurses and intensivist physicians [9–11]. ICU patients typically have multiple diagnoses and require timely responses to events and erroneous treatment actions since they are critically ill [12–14]. Nurses in tele-ICUs continuously monitor automated clinical data and have access to bedside cameras (described in Sapirstien) [15]. All documentation of patient care activities in the clinical record is accessible to tele-ICU personnel. Tele-ICU nurses contact ICU nurses via a phone call or electronic notes, and ICU nurses can contact the tele-ICU nurses via phone or an emergency call button located near the bedside. The tele-ICU nurses conduct their own patient rounds and support reporting clinically relevant information.

A recent prospective study of adult ICU patients reported that a tele-ICU intervention was associated with reduced patient mortality, reduced hospital length of stay (LOS), higher rates of adherence to best practices, and lower rates of preventable complications [16]. Similarly, another study had mixed findings, higher physician tele-ICU use decreased hospital LOS while low use increased hospital LOS, and there was no significant effect for patient mortality, overall LOS, or total hospital cost [17]. Some smaller studies have found benefits from using a tele-ICU for ICU LOS and/or patient mortality [18–22].

To date, telemedicine presence is in approximately 5000 ICU beds in the United States hospital networks and more than a million patients have been monitored and configurations are highly variable [23, 24]. This study aims to extend the current literature on tele-ICUs, by examining what functions the tele-ICU contributes to the hospital system to achieve the reported benefits. In addition, we suspect that over time as the tele-ICU becomes more embedded within the hospital system that there will be changes in the way that the tele-ICU is used.

2. Methods

2.1 Setting

This study was conducted at a single tele-ICU that provided care to six remote ICUs within a local geographic region in the same state. Staffing at the participating tele-ICU was nurse-intensive. During each 12-hour shift, 5–6 nurses monitored a total of 178 patients across the 6 ICUs; there were no physician’s staffed during the day. Each nurse was responsible for 25 to 35 patients and had an average of 15 years of experience. A single intensivist physician staffed the tele-ICU during the night shift (9 pm to 7 am) and was responsible for providing care to all the patients monitored by the tele-ICU.

Nurses monitored patients via a control center, which had access to patients’ physiological data, treatment plans, and medical records. Specifically, the tele-ICU had access to real-time vital sign data (e.g. blood pressure, heart rate), graphic representation of vital sign data over time, remote 2-way audio and 1-way video feeds, and the electronic patient record, which included the history, labs, and notes. In addition to the patient data, the tele-ICU had “smart alerts”, where sensors connected to patients had alarms that were triggered based on a change in the patient’s vitals over time. Finally, the tele-ICU supported the collection of data for management reports about outcomes, practice patterns, resource utilization and clinical operations.

2.2 Data Source

After IRB approval was obtained, the data source was an exhaustive convenience sample of nursing intervention data logged for operational purposes at a single tele-ICU. The time units were one month in 2005 (December; 744 entries) and one month in 2007 (October; 1193 entries), which was the only intervention data researchers were able to obtain.

The intervention logs recorded every interaction between the tele-ICU nurses and the remote ICUs, regardless of whether the tele-ICU or ICU initiated the intervention. Intervention log data included: patient identifier; a narrative description of the intervention and a code classifying the intervention, made by the tele-ICU nurse who documented the intervention; the individual who initiated the interaction, and, occasionally, the time and follow-up/outcome of the intervention.

2.3 Coding Procedure

One investigator (SA) uniquely categorized all log entries, with the exception of nine interventions, which were coded as two categories. These log entries each contained multiple interventions undertaken by the nurse (e.g. called the nurse about a critical lab value and then attending physician to get a medication order). All entries were coded in addition to the nursing codes which were clinically-based (e.g. vital sign change or lab follow-up) rather than functions of the tele-ICU. The codes for the interventions were iteratively generated (see Anders et al, 2007 for the complete code evolution and conceptual frameworks employed in the coding schemes) [25]. Codes were initially generated by the research team and tele-ICU personnel through a process of iteratively grouping and categorizing tele-ICU observational data. This process included a combination of top-down synthesis of macrocognitive functions [26] and bottom-up data analysis to iteratively generate and delineate a set of agreed upon codes used for this analysis. A second reviewer independently coded a random subset of 100 log entries for which the Kappa coefficient was 0.74. Final codes and examples are provided in ►Table 1. The functions grouping the codes were generated after coding was completed to facilitate data analysis.

Table 1.

Functions and associated codes for logged tele-ICU interventions

Function	Code	Example
Timely response to events	Change Detection	The tele-ICU nurse alerted the bedside nurse that a patient was in distress, with a high respiratory rate (>50/min) and low oxygen saturation (95%).
	Error Detection	The tele-ICU nurse notified the ICU nurse that a patient’s blood pressure was low (66/46) because a propofol drip was erroneously left at a high dose.
	Targeted Monitoring	The bedside nurse requested that the tele-ICU nurse monitor one patient while he was taking his other patient to interventional radiology.
	Immediate Patient Risk Detection	The tele-ICU nurse told the patient to stop pulling out his endotracheal tube and alerted the bedside nurse.
Access specialized expertise	Physician Action	The bedside nurse asked the tele-ICU physician to provide guidance on the placement of the nasogastric feeding tube by reading the X-ray showing the kidney, ureter, and bladder (KUB).
	Knowledge	The bedside nurse asked the tele-ICU nurse to confirm that intravenous (IV) fluid could be given through an infusion port on a Swan-Ganz catheter.
Reduce workload	Coordination Tasks	The tele-ICU nurse asked the blood bank to send blood for a bedside nurse’s patient.
Increase adherence to best practices	Best Practices	The tele-ICU nurse notified the bedside nurse that no prophylaxis medication had been ordered to prevent blood clots for a patient on bed rest.
Fix problems with alarms, equipment	Alarm Management	The tele-ICU nurse alerted the charge nurse that the oxygen saturation alarm was not working for a patient.

3.4 Data analysis

Any overall test for differences between 2005 and 2007 was conducted using a Chi-square goodness of fit test statistic. Differences in the relative proportions of coded elements between 2005 and 2007 were tested using a Chi-square goodness of fit test. A stepwise Bonferroni adjustment was made for nine multiple simultaneous tests (one for each code category). In addition, a Chi-square goodness of fit analysis was conducted on changes from 2005 to 2007 in the initiating organization for the intervention (the tele-ICU or one of the six ICUs).

3. Results

Overall, 1927 intervention codes, 744 from 2005 and 1183 from 2007, were analyzed. Interventions were conducted at all six of the ICU’s in both time periods, with the proportion of interventions at a single remote ICU ranging from 3.4% to 30.6% of the overall interventions. The results reveal that 42.6% of the logged tele-ICU interventions were related to change detection, such as phone calls from the eICU to the ICU about abnormal lab values or vital signs. The least frequent interventions were educational (0.6%). There was a significant difference between expected versus observed frequencies of categories between 2005 and 2007, (χ² = 123.92, df = 8, p<0.001).

As displayed in ►Figure 1, seven of the nine categories were significantly different (p<0.05) between 2005 and 2007, respectively. These were: decreased change detection (χ² = 95.74), decreased error detection (χ² = 7.42), increased targeted monitoring (χ² = 9.32), increased coordination tasks (χ² = 38.68), increased best practices (χ² = 24.67), increased physician action (χ² = 10.17) and increased alarm management (χ² = 22.41).

Fig. 1.

Comparison of percentage change in the coded functions for tele-ICU interventions over time

Overall, 81.9% of the interventions originated in the tele-ICU while only 18.1% were the result of the ICU calling the tele-ICU. More interactions were initiated by the ICUs over time (►Figure 2), as opposed to initial interactions primarily being initiated by the tele-ICU (8.8% in 2005 to 23.5% in 2007, χ² = 62.82, p<0.05).

Fig. 2.

Changes in the initiation of logged intervention

4. Discussion

The results reveal significant changes in the relative proportion of types of interventions conducted by a single tele-ICU providing care to the same set of ICUs between 2005 and 2007. In a majority (7/9; 78%) of the tele-ICU functions, these changes in relative proportion were significant. In addition, no new functions emerged, and no functions were eliminated.

This is the first study that we are aware of that actually reports changes in the frequencies of functions in a tele-ICU over time. These findings suggest that this tele-ICU increased support when ICU nurses were off the unit (targeted monitoring) and decreased real-time support for ICU nurses directly caring for patients (change detection and error detection). In addition, the tele-ICU nurses increased coordinating with other hospital units, such as the blood bank. Finally, functions with longer-term benefits increased, such as encouraging adherence to best practices and fixing problems with alarms and equipment.

Based on these findings, it appears that there were both increases and decreases to the workload for ICU nurses. In addition to sharing information with the tele-ICU, which required some additional documentation and sharing tasks by ICU personnel, some of the interactions to adhere to best practices and fix problems with alarms and equipment might have added workload, if only due to additional burden of coordination. On the other hand, the tele-ICU nurses took on coordination tasks, including interacting with the tele-ICU physician, thereby eliminating those tasks for ICU nurses. Because these findings imply a change from a primarily ‘push’ interaction model (the tele-ICU identifying opportunities for interventions) to a ‘mixed push-pull’ model (ICUs initiating contact 23% of the time), we assume that ICU nurse workload was not dramatically increased as the use of the technology evolved. This change in the distribution of work tasks may be a result of a cultural shift within the ICU towards acceptance of the tele-ICU from “big brother” to an integral part of ICUs.

Regarding staffing, the findings do not suggest mechanisms to run the tele-ICU more economically with less specialized staff. Although more of the logged events involved nursing functions as compared to physician functions, the physicians were often subsequently involved in the intervention (25.1% and 27.6%, respectively, of interventions were escalated to the bedside or tele-ICU physician). These findings could be interpreted as tele-ICU nurses trending over time to act less like “an extra set of eyes” and more like a care coordinator and knowledge center; however, the most frequent category of intervention remained detecting patient events for more timely bedside responses. Previous, studies have suggested that a likely reason for decreased mortality and LOS is that the tele-ICU functions as “an extra set of eyes”, which would suggest that this is still a critically important function of the tele-ICU, but may change over time [19].

Based on these findings, replacing either the tele-ICU nurses or the tele-ICU intensivist physician with less knowledgeable personnel or automation would be anticipated to reduce the effectiveness of the interventions and potentially negatively impacting the utilization of this technology resource. All of the categories benefitted from specialized expertise, with the possible exception of alarm management. If tele-ICU personnel changes were made to reduce costs, it is possible that tracking changes in these functions over time might provide insight as to which functions are most strongly associated with specialized expertise, because their use would decrease substantially if no longer available or minimally available.

Finally, as the implementation and use of tele-ICUs increases, it is possible for the tele-ICU functions described in this research to be automatically monitored using intervention logs. Where possible, this could help inform future implementations as well as both administrative and research purposes, including monitoring evolutionary changes within the ICU system.

More broadly, these findings suggest that HIT interventions are dynamic, with constantly evolving emphases on multiple, interacting functions in complex, context-sensitive ways. Therefore, the assumption embedded in traditional RCT study designs that an HIT intervention over a study time period is a single, unitary intervention, analogous to a medication, may need to be reconsidered. Longitudinal studies may provide uniquely valuable empirical information to explain non-intuitive findings about changing HIT effectiveness over time, difficulties in generalizing findings from one site to another, and other surprising findings from HIT implementation.

Because the study was conducted at a single tele-ICU, the ability to generalize to other tele-ICUs is limited. At the time the study was conducted, of the six tele-ICUs we interviewed, this was the only tele-ICU with an intervention log. The intervention logs were created by tele-ICU nurses, and therefore missed physician only interventions. In addition, it is possible that the model of care delivery at the participating tele-ICU has changed since the time of data collection due to the rapidly changing nature of telemedicine technologies in healthcare. Nevertheless, there continues to be interest in using telemedicine to remotely provide care to critically ill patients, and the methods employed in this study may be useful for monitoring changes in functions over time for both research and administrative purposes.

5. Conclusion

This study demonstrated that a specific HIT, the tele-ICU, functionally changed over time as it was further integrated into the ICU system of care. Specifically, the tele-ICU increased support when ICU nurses were off the unit, inter-hospital coordinating and adherence to best practices, while simultaneously decreasing real-time support for ICU nurses. This research suggests that the frequency of new technology functions change after implementation. Further study to delineate important changes that may adversely impact the benefits of the HIT should be undertaken such as monitoring indicators of change along predictable trajectories. Longitudinal studies of systems in use may be useful to monitor changes so as to maximize the benefits of HIT while revealing potential gaps in use.

Implications of Results for Practitioners

The findings from this study provide a functional understanding of what the tele-ICU does in realtime to support ICU nurses and ICU patient care. This research reveals that the functions included in a new technologies may change over time, which may result in use not consistent with intended use, including those that may erode safety. Further research should examine how the tele-ICU and other HIT use evolves over time and its effect on practitioners work.

Human Subjects Protections

The study was reviewed by the Ohio State University Institutional Review Board for compliance with the World Medical Association’s Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects.

Conflicts of Interest

The authors have no conflicts of interest in the research to declare.

Disclaimers

The views expressed in this article are those of the authors and do not necessarily represent the view of the granting agency.

Financial Support

Dr. Anders was supported in part by a training grant from the National Library of Medicine (Gadd, T15LM007450).