A Cost-Benefit Analysis of Automated Physiological Data Acquisition Systems Using Data-Driven Modeling

Abstract

Precision medicine and the continuous analysis of “Big data” promises to improve patient outcomes dramatically in the near future. Very recently, healthcare facilities have started to explore automatic collection of patient-specific physiological data with the aim of reducing nursing workload and decreasing manual data entry errors. In addition to those purposes, continuous physiological data can be used for the early detection and prevention of common, and possibly fatal, diseases. For instance, poor patient outcomes from sepsis, a leading cause of mortality in healthcare facilities and a major driver of hospital costs in the USA, can be mitigated when detected early using screening tools that monitor the changing dynamics of physiological data. However, the potential cost of collecting continuous physiological data remains a barrier to the widespread adoption of automated high-frequency data collection systems. In this paper, we perform cost-benefit analysis (CBA) of machine learning applied to various types of acquisition systems (with different collection intervals) to determine if the benefits of such systems will outweigh their implementation costs. Although such systems can be used in the detection of various complications, in order to showcase the immediate benefits, we focus on the early detection of sepsis, one of the major challenges of hospital systems. We present a general approach to conduct such analysis for a wide range of hospitals and highlight its applicability using a case study for a small hospital with 150 beds and 3000 annual patients where the acquisition system would collect data at 1-min intervals. Lastly, we discuss how the analysis may help guide incentives/policies with regard to adopting automated data acquisition systems.

Introduction

Precision medicine has been identified as a key component of our future healthcare system [1]. Adoption of precision medicine principles is expected to deliver significant benefits to the health of individuals and society at large due to the incorporation of individual-specific factors in clinical decision-making. A significant aspect of precision medicine includes physiological measures such as heart rate (HR) generated from electrocardiograms (ECG), respiratory rate (RR) from impedance respiratory waveform (IRW), and oxygen saturation (S_pO₂) from photoplethysmography (PPG). In the current hospital environment, these measures are manually entered by a nurse into the electronic medical records (EMRs). Some modern hospital systems, however, have started to incorporate automated acquisition software into their systems to reduce the burden of manual entry on the nursing staff, and also to reduce the potential for human error or delay [2]. However, to the best of our knowledge, the exact utility of the automated data acquisition method, with regard to its predictive analytic potential, collection interval, and storage requirements, has yet to be investigated from a cost-benefit perspective, beyond their direct impact on clinical workflow. While clinical decision support systems have long been integrated into the EMR, healthcare practitioners have only recently started to use some of the continuously captured physiological data in early warning systems. For instance, one of the major challenges in hospitals and a leading cause of death, namely sepsis, can be vastly mitigated using automated data entry which enables automatic detection/prediction [3, 4].

Sepsis is a systemic response to infection which, in the absence of early diagnosis and timely treatment, may ultimately result in death. Similar to major fatal conditions such as heart failure, stroke, and multi-organ dysfunction, the time between the onset and detection of sepsis and the type of treatment administered greatly affect patient outcome [5]. Severe sepsis occurs when sepsis is left untreated and may contribute to multi-organ dysfunction, and possibly death [6, 7]. In general, it is reported that each hour of delayed diagnosis of sepsis results in an 8% increase in the mortality rate [8]. Severe sepsis is among the most common causes of death in intensive care units (ICUs) and is associated with billions of dollars in healthcare costs each year [9]. In general, sepsis is among the most expensive conditions to treat in the USA, with costs exceeding 24 billion dollars annually [10]. Hence, early identification of sepsis is an essential component to improved patient outcomes, thereby limiting costs, morbidity, and mortality [11, 12].

Clinicians employ several severity of illness screening tools in order to detect symptoms potentially related to sepsis. For instance, quick sequential (sepsis-related) organ failure assessment (qSOFA) relies on information obtained from blood pressure, respiratory rate, and altered mental status to identify patients that may be at risk of having sepsis [13]. Those screening tools integrate a variety of physiological data such as respiratory rate, blood pressure, level of consciousness, and temperature to identify abnormal acute physiological changes associated with septic patients. However, those tools have been designed to detect sepsis after those physiological trends manifest at the bedside. In addition, some of those measures may be subjective in nature, e.g., the Glasgow Coma Scale (GCS) and may result in inconsistent care across clinicians and hospital systems. Thus, a transition to EMRs has encouraged the use of additional variables to detect septic conditions much before clinical recognition occurs [8, 14]. The available tools have different levels of sensitivity and specificity for detection of sepsis and can rely on the data abstracted from multiple sources, including EMR, laboratory systems, pharmacy systems, and automated bedside monitor acquisition software [15]. However, in most cases, generating specific measures used in these severity of illness scores manually can be cumbersome and time-consuming for the already inundated bedside staff [16, 17]. Hence, an automated implementation of screening algorithms may reduce the burden of capturing and interpreting physiological data.

Physiological Data Streams

The hospital environment generates significant data volumes on a continuous basis. For instance, consider the intensive care units (ICUs), which can generate continuous data at hundreds of times per second. In ICUs, the bedside patient monitor samples ECG from several leads typically at a sampling rate of 500 data samples per second, along with other signals such as PPG and arterial blood pressure, whose combination can rapidly bring the data volume into thousands of data points per second [18]. Indeed, a physiological data storage system could generate over 13 GB of data over a period of 1 month from a 200-bed facility at a sampling rate of only one data point per minute, after adjusting for dynamic hospital census. Furthermore, adjusting for other forms of medical data, such as images, clinical notes, increased physiological sampling rates, and hospital information systems, this data volume could increase significantly.

Most of the clinical data, in the past, were stored and handled in paper-based form; however, the current trend in the healthcare industry is toward expeditious digitization [19]. In order to acquire and store data, many healthcare providers have already made investments in EMR systems. However, it is currently unclear whether there are tangible benefits in upgrading existing physiological data management, beyond their traditional use for immediate visual consumption. It is generally believed that those systems can ameliorate many of the challenges associated with delayed and inaccurate data entry in EMRs and help with medical decision-making [8, 14]. However, those data acquisition systems can produce significant data volumes that make long-term archival impractical given existing workflows [20]. Therefore, there is a need to investigate the tradeoff that exists between capturing high-resolution physiological data at various predefined intervals, as opposed to standard EMR-based vitals that are generally recorded once every hour or more for potential use in clinical decision-making at the bedside. Currently, no standards have been defined as to where that balance exists, in terms of data collection interval and its required infrastructure, and the associated influences on algorithm development [20].

Cost and Infrastructural Impact of Physiological Data Stream Analytics

It is widely acknowledged that EMR systems are associated with several benefits such as an increase in the quality of patient care and the prevention of certain medical errors [21]. However, the various costs associated with EMR systems such as installation, implementation, operation, and maintenance costs continue to be a barrier to the widespread adoption in healthcare facilities. As a result, there is an important existing body of work that uses cost-benefit analyses to determine whether EMR systems improve the financial performance, in terms of several key performance indicators (KPIs) of healthcare organizations [22–25]. These KPIs include return on investment (ROI), patient length of stay (LOS), net present value (NPV), benefit-cost ratio (BCR), and the discounted payback period (DPP) to name a few. In general, the cost-benefit analyses on EMR systems report positive returns for the adoption of these systems with varying magnitudes of the considered KPIs [23].

Most of the existing cost-benefit analyses, however, do not investigate the degree of EMR system integration by healthcare facilities; instead, they simply compare the adoption of EMR systems with conventional paper-based systems. Moreover, existing studies do not investigate the effect of automated data acquisition to truly complement the electronic integration of various medical data sources. While some studies have investigated the automated acquisition of physiological data in order to support nursing workflow, limited work has been contributed in the areas in which predictive analytics can be applied. Furthermore, studies have not considered the influence of various collection intervals and their impact on KPIs considering the benefits and costs associated with data collection at each interval. Limited work is also present in the literature that investigates the impact of predictive analytics on various small, medium, and large hospitals in relation to their perceived utility and expected costs. For instance, it may be better for a smaller healthcare facility to adopt an EMR system that captures and stores data at a different collection interval (therefore requiring different data acquisition and storage infrastructure) than for a larger facility with significantly larger bed counts. Finally, disease incidence and mortality rates for various facility sizes may also influence the decision on the degree of EMR implementation [15]. Therefore, the optimal requirements with regard to an EMR system and its specifications may be case-specific and not generalizable; however, these elements have not been studied to date and hence remain open research questions.

In this paper, we investigate the costs and quantify the potential benefits of implementing automated data collection in hospitals that have already adopted EMRs from a machine learning, i.e., predictive analytics, perspective. The costs are due to implementation, operation, and maintenance of the automated data acquisition system and the benefits would be due to the possibility of using predictive analytics on the collected data for earlier detection/prediction of sepsis. Specifically, we use the physiological data collected by an automated data stream acquisition system of a large hospital on a minute-by-minute basis to investigate the impact of collection interval on the accuracy, sensitivity (also referred to as recall or true positive rate), and specificity, and next translate these values into benefits. We employ versatile machine learning algorithms, namely random forest (RF), recurrent neural network (RNN), and multilayer perceptron (MLP) to generate predictive models that classify sepsis/non-sepsis patients using subsets of the original dataset obtained after controlling for the data collection interval. Our goals are to investigate the influence of varying collection intervals on the accuracy and sensitivity of the sepsis detection algorithm, and to use that knowledge to develop a generalizable model that can be used to assess the utility of capturing continuous physiological data across small, medium, and large hospitals. We identified sepsis as a potentially powerful case to demonstrate the influence of both applying predictive modeling and demonstrating the utility of continuously monitored data.

This paper is structured as follows. In Section 2, we provide an overview of the dataset and the variables used in the classification. We then discuss details of the models used to distinguish between sepsis and non-sepsis patients. Also, we discuss the various costs and benefits from a societal perspective, associated with different degrees of EMR systems implemented in hospitals for different cases with regard to existing data collection infrastructure. Next, in Section 3, we present the results for the classification performance at various data collection intervals and conduct cost-benefit analyses. In Section 4, we discuss the results obtained and how the healthcare industry could benefit from the observations. Lastly, we conclude in Section 5.

Materials and Methods

Patient Data

This IRB-approved, retrospective study uses continuous physiological data streams captured using the Cerner CareAware iBus® system from a large tertiary hospital and stored in a secure facility. A list of the top 15 variables collected by the automated physiological acquisition system is presented in Table 1. The acquisition software integrates bedside medical monitors located across inpatient and intensive care units and inserts data at 1-min intervals into a buffer of the last 12 h. We developed software that accessed data from the 12-h buffer and continuously saved them for retrospective analysis.

Table 1.

Patient dataset parameters and their occurrence

Parameter	Number of data points (in millions)
Heart rate	23.3
Mean arterial pressure (cuff)	21.3
Diastolic blood pressure (cuff)	21.3
Systolic blood pressure (cuff)	21.3
SPO2 (oxygen saturation)	19.5
Measured PEEP/CPAP (cmH2O)	3.3
Mean arterial pressure (arterial)	2.7
Diastolic blood pressure (arterial)	2.7
Systolic blood pressure (arterial)	2.7
Fraction of inspired oxygen (FiO2)	2.4
Respiratory rate	2.1
Inspiratory time (Ti)	1.9
Central venous (Rt atrial) pressure	1.7
Slope time	1.3
Sensitivity (LPM)	1.3

In addition to the physiological data, we also capture specific clinical and demographic data. We mapped patients with physiological data of at least 12 h to the clinical and demographic dataset. Patients included in this study were over the age of 18 and were admitted to the hospital system between February 1, 2017 and May 31st, 2017. A total of 32,190 patients were identified in the dataset, only 5953 of which however had physiological data collected on a continuous basis. The dataset contained patients with a wide array of medical conditions; however, in this study, we focus on patients with sepsis. In addition, we included patients for whom both HR and systolic BP are collected for at least 12 h after admission. This resulted in a secondary dataset of a total of 2995 patients. Out of these patients, 343 patients are diagnosed with sepsis (based on their ICD 10 diagnosis codes). For this specific study, we were interested in the classification of sepsis and non-sepsis patients as reported by their diagnostic codes for the purpose of identifying cost metrics; hence, we did not evaluate a continuous score such as the SOFA or qSOFA.

Costs and Benefits

In this section, we present an overview of the costs and expected benefits of implementing an automated data acquisition system with various data collection intervals. We consider two main categories of benefits, namely the direct costs saved as well as the monetary value of “lives saved” due to the early detection/prevention of sepsis, since the outcomes of detecting life-threatening diseases at an early stage affect the society at large in many ways. It should be emphasized that we are not comparing EMR systems and conventional paper-chart systems; rather, we are investigating the adoption of automated data acquisition systems for hospitals with existing EMR systems at various levels of physiological data monitoring infrastructure and sepsis prevalence. The benefits are estimated conservatively as factors such as the decrease in nursing staff workload fall outside of the scope of this paper. Subject matter expert opinion is used when data are not available.

The benefits associated with implementing an automated data acquisition system include the direct costs saved and the monetary value of lives saved due to the early detection of sepsis using sophisticated machine learning algorithms [23]. Early detection of sepsis may not only allow for a shorter treatment period as a result of identifying sepsis at an early stage, but contributes to improved short-term and long-term patient outcomes [8]. Sepsis is approximately 70% more expensive than the average hospital treatment cost associated with other conditions. One of the primary reasons for the high cost of sepsis treatment is that sepsis patients’ LOS is 75% longer than for other conditions [26]. Early detection of sepsis would therefore reduce the overall treatment period, allowing for a reduced patient turnaround time, which can possibly result in higher revenue for the healthcare facility due to an increase in the number of patients being admitted. Note that in this study, we only focus on the direct costs due to inherent difficulties in quantifying other benefits. The average direct cost of hospitalization due to sepsis was estimated at $18,000 [27]. The average LOS for sepsis patients is on the order of 3–9 days [26, 28]. However, patients with severe sepsis can experience much longer stays, which result in a significantly higher direct cost to hospitals. In one study, the median LOS for patients with severe sepsis is reported at approximately 12.5 days, resulting in approximately $26,000 in direct costs [29]. This figure is somewhat consistent with the average cost of hospital stays in which patients die [30]. Hence, we may use $18,000 for the direct cost of stays in which a severe sepsis patient is discharged alive. If a sepsis patient dies in hospital, however, we use the direct cost of $26,000. As reported, the early detection of sepsis may decrease the average LOS for sepsis patients by an estimated 3.7 days [31]. Note that the average cost of stay for patients who are discharged alive is on the order of $9500 [30]. Hence, we estimate the direct cost of stay in which sepsis is detected early at $11,000, mainly due to the shorter LOS compared to severe sepsis patients but a slightly longer stay than average patients.

In addition, implementing an automated data acquisition system can provide benefits to society in the form of the monetary value of lives saved due to the early detection of sepsis. Numerous estimates for the economic value of human life exist, ranging from $4 million to $9.1 million [32]. In this study, we consider the average economic value of life to be $600,000 for sepsis patients based on the average age of sepsis patients, life expectancy in the USA, and the standard for the monetary value of per year of quality of life [15, 33, 34]. More specifically, the average age for sepsis patients is approximately 66 years whereas the life expectancy in the USA is 78 years. The monetary value per year for quality of life has been estimated to be $50,000 [33]. As a result, we estimate the average economic value of life for patients succumbing to sepsis at (78 − 66) × $50,000 = $600,000.

Attempting to detect sepsis at an early stage may produce false positives (false alarms). A false alarm would result in utilizing the clinical staff time to run an unnecessary blood work and possibly lengthening LOS, resulting in additional costs to the hospital. In addition, false alarms in general can lead to alarm fatigue among the bedside staff. The cost of additional blood work can itself be site-specific [35]. In addition, it is exceptionally difficult to calculate the true cost of false alarms. A recent industry report estimates that 11.08% of nursing time is spent on responding to non-actionable alarms [36]. This figure, however, does not include the potential costs of alarm fatigue and its short-term and long-term implications. In this study, we use an estimated cost of $1000 per false alarm to capture its associated costs. We acknowledge that this estimated cost does not include all potential costs of alarm fatigue.

The direct costs associated with implementing an automated data acquisition system can be categorized into system components, data storage, and operating and maintenance costs [21]. System component costs comprise of sensor devices, display units, and a central data capture station. EMR systems can monitor multiple patients continuously, whereas the physiological data comes from multiple single unit devices that transmit data to the central data capture station via a wired local area network (LAN). Alarms alert clinicians when the system deems that a patient’s vital signs are deteriorating based on certain predefined specifications for an algorithm trained for specific patient conditions. System component costs may vary from $4000 to $6500 per bed for physiological monitoring devices without ECG, and anywhere from $8000 to $50,000 per bed for devices with ECG [37, 38]. Operating and maintenance costs for these devices are estimated at 10% of the initial capital expenditure, after a 1-year warranty, and increase 5% annually [21, 39]. Data storage costs depend on the type of storage solution implemented by the facility. Currently, there are two main storage solutions, namely on-site data storage in racks of servers or cloud-based storage services. The choice of a hospital’s data storage solution is generally influenced by the security and control of the data (HIPAA compliance), redundancy in emergency situations, connectivity, and cost of ownership. In 2011, data storage costs per patient were estimated at $0.55 per patient per year [40]. Lastly, based on recent reports, operating and maintenance costs of the data storage facility are approximately $1300 per TB per year [40].

Methods

Machine learning techniques such as random forest, RNN, and MLP models have been widely implemented in classification applications in the healthcare industry [41–45]. In short, random forests rely on random subsets of the dataset features to build classification trees and then use the classification with the most “votes” from all the generated trees [46]. A random forest model is preferred in certain classification problems since it is not sensitive to noise in datasets and is also robust against overfitting to training datasets [47]. Random forest models are also known for their fast computation time for large datasets, compared to neural networks or deep learning techniques [48, 49], and usually outperform the other tree-based algorithms such as decision tree learning [50] and tree bagging [51]. An RNN contains connections between neural network nodes to form a directed graph along a sequence of data. RNN models, particularly those composed of long short-term memory (LSTM) units, are renowned for their ability to exhibit dynamic temporal behavior for time series data [52]. MLP is a feed forward neural network that consists of one or more hidden layers and is often used in various classification applications [53, 54]. In general, the performance of deep learning models may be dependent on the hyper-parameter values and network architecture used; however, there exist several methods to select the network architecture and hyper-parameters to improve performance [55].

From the dataset, we identify sepsis and non-sepsis patients based on their ICD 10 codes at discharge including A41.9, R65.20, and R65.21 for sepsis, severe sepsis, and septic shock, respectively. We use balanced datasets for training and testing to avoid favoring any particular category of patients. More specifically, our overall dataset consists of 343 sepsis patients and 343 non-sepsis patients randomly sampled from the remaining 2652 patients. The balanced data set is split into 80%/20% training and testing data sets respectively for the random forest model. For the RNN and MLP models, we use a 60%/20%/20% split for training, validation, and testing data sets. Also, for the RNN and MLP models, we implement an early stopping algorithm to maximize the validation set accuracy and avoid overfitting.

In our analysis, we use demographical and physiological data collected for all sepsis and non-sepsis patients in the first 12 h after admission. The objective is to develop models that can correctly detect/predict whether or not a patient has sepsis or would develop sepsis by only relying on the information from the first 12 h after admission. We use gender, age, and race of the patient in all models. For random forest and MLP models, we extract a series of features from heart rate and systolic blood pressure data within the first 12 h. More specifically, we use discrete wavelet transforms [56] to extract five sub bands from heart rate and systolic blood pressure signals and then calculate common statistical features and signal entropy for each signal. The set of features used in the models are listed in Table 2. For the RNN models, we use the stream of heart rate and blood pressure data along with patient demographics to make predictions at hour 12 after admission.

Table 2.

Features used in random forest and MLP models

Feature category	Description
Common statistical features	Minimum, maximum, mean, standard deviation, variance, skewness, kurtosis of signal for time period
Signal information	Entropy of signal for time period
Discrete wavelet transforms	Minimum, maximum, mean, standard deviation, and variance of the five sub bands extracted from each signal
Demographics	Gender, age, and race of patient

To evaluate the significance of the data collection interval on model performance (particularly on classification accuracy, sensitivity, and specificity), we build random forest, RNN, and MLP models, assuming that the data were collected at every 1-, 15-, 30-, 60-, 90-, 120-, 180-, and 240-min time intervals for the 12 h under consideration. Consequently, we select the model resulting in the highest sensitivity among all at each data collection interval and use them to evaluate the effect of collecting data at various intervals in the CBA.

Results

In this section, we first present the effect of data collection interval lengths on the accuracy, sensitivity, and specificity of classification. Next, we estimate the costs of investing in automated data collection systems and perform CBA for a small hospital from a societal perspective. We also present the generalized results for hospitals of different sizes. In this paper, similar to previous works, we classify different-sized hospitals as follows: Small hospitals have less than 200 beds, medium-sized hospitals have between 200 and 499 beds, and large hospitals have 500 beds or more [15, 57].

Accuracy, Sensitivity, and Specificity Versus Collection Interval

We randomly generated ten different balanced splits for training and test datasets from 686 patients for any given data collection interval in order to achieve robust results and provide confidence intervals. We trained ten models for all considered data collection intervals for each classification technique, and report the average accuracies, with 95% confidence intervals, as well as the average sensitivities and specificities of the ten corresponding test sets. Figure 1 illustrates these classification accuracies, sensitivities, and specificities for the various data collection intervals considered for the random forest, RNN, and MLP models. As seen in Fig. 1a, b, these three performance metrics are highly correlated and monotonically decrease in data collection interval length for the random forest and MLP classifiers. That is, consistent with the intuition, the more frequently the physiological data is collected, the higher the overall classification accuracy and sensitivity, and hence the higher the potential for early detection of sepsis. As seen in Fig. 1c, sensitivity obtained using the RNN models on average follow a similar profile to those obtained using the random forest and MLP models. However, as seen in the figure, specificity, and therefore accuracy, seem to be more robust against changes in data collection interval when using RNN models.

Fig. 1.

Average sepsis classification accuracies, with 95% confidence intervals, and average sensitivities and specificities of the test sets at various collection intervals considered. a Random forest. b MLP. c RNN

As briefly mentioned in Section 2.3, at every collection interval, we select the model resulting in the highest sensitivity when conducting a CBA as this metric is most relevant to detecting sepsis patients. Table 3 presents the selected model and its corresponding sensitivity and specificity at each data collection interval to be used in the CBA. In addition, Table 4 presents the normalized confusion matrices of obtained using these selected models for the 1-min and 240-min collection intervals for the dataset examined. As seen in the table, we observe approximately 7% increase in model sensitivity and 6% increase in specificity at the collection interval length of 1 min compared with the collection interval length of 240 min (or 4 h). Clearly, the former data can only be obtained using automated data acquisition systems where the latter may be attributed to the traditional EMR systems where the data is entered manually. The results obtained on larger interval lengths (e.g., greater than 1 h) are consistent with, or in some cases slightly better than, the reported values in the literature. For instance, it has been reported that EMR-based systems can fail to detect up to 30% of patients with sepsis, while an additional 30% are misclassified [15]. Hence, to be on the conservative side, we use the models built on collection interval length of 240 min to estimate the sepsis early detection rate of traditional EMR systems. Recall that in the CBA case study, we aim to investigate the effect of early detection of sepsis and hence, the metrics most relevant to us are sensitivity and specificity. Therefore, consistent with the results in Fig. 1 and Table 4, we use the point estimates of 66% and 65% for the sensitivity and specificity of models developed using the data collected from an EMR system.

Table 3.

Sensitivity and specificity of the selected models at each data collection interval

Collection interval (minutes)	1	15	30	60	90	120	180	240
Sensitivity	73.03%	71.47%	70.86%	69.20%	68.24%	67.58%	66.89%	66.37%
Specificity	73.36%	68.83%	68.05%	67.49%	66.57%	66.74%	67.35%	65.41%
Selected model	RF	RF	RF	RF	RF	RNN	RNN	RNN

Table 4.

The normalized confusion matrices for 1-min and 240-min collection intervals, along with the average accuracies, sensitivities, and specificities

		Predicted+	Predicted−	Accuracy (%)	Sensitivity (%)	Specificity (%)
1-min collection interval	Sepsis+	0.37	0.14	73.19	73.03	73.36
	Sepsis−	0.13	0.36
240-min collection interval	Sepsis+	0.32	0.16	65.87	66.37	65.41
	Sepsis−	0.18	0.34

Cost-Benefit Analysis

Costs to Hospitals for Adopting an Automated Data Acquisition System

In this section, we first present the cost of adopting an automated data acquisition system with 1-min data collection intervals (to complement the existing EMR system) for a small hospital (150 beds, 3000 annual patients) with no existing automated data acquisition infrastructure. We then use the same approach to obtain the costs for automated data acquisition systems with 15-, 30-, 60-, 90-, 120-, and 180-min intervals.

We estimate the costs for a period of 5 years. Considering the system component cost of $8000 per bed [37], the initial capital expenditure for 150 beds equals $1.2 million. In addition, the operating and maintenance cost is estimated at $120,000 from the second year of operation with an annual increase of 5%. Hence, in years 2–5, the operating and maintenance cost will be as follows: $120,000, $126,000 (=$120,000 × 1.05), $132,300, and $138,920. We use the available reports [40] and expert opinion to estimate the data storage cost for the automated data acquisition system as follows. We estimate the storage cost at $1 per patient per year. Based on previous reports, large hospitals generate on the order of 30 terabyte (TB) of data per year [40]. In this study, to be on the conservative side, we use 20 TB in our calculations to account for the small hospital size and also assume that the volume of data generated remains constant over time. Note that the operating and maintenance costs of data storage are on the order of $1300 per TB per year. As a result, the data storage cost for 3000 patients and operating and maintenance cost of 20 TB in the first year are estimated to be 3000 × $1 + $1300 × 20 = $29,000. It is also recommended to keep the collected data up to at least 7 years [40]. Hence, in years 1–5, the data storage cost will be as follows: $29,000, $55,000 ( = $29,000 + $1300 × 20), $81,000, $107,000, and $133,000.

However, it is expected that the use of HIPPA-certified cloud storage services would lower the cost of data storage dramatically in the near future. Therefore, if using a conservative value of $202.5 per 500 GB [58], the storage cost per 1 TB per year would equal to approximately $405. Hence, for our case, i.e., storing 20 TB per year, the cost of using this service would equal to $8100 (=20 × $405) in the first year, and $16,200 (=$8100 + $405 × 20), $24,300, $32,400, and $40,500 in years 2–5, respectively. The costs are summarized in Table 5. It is seen that the type of data storage option does not have a significant influence on the total cost for the 5-year period. In the CBA, we use the cloud storage data cost.

Table 5.

The costs (in $1000) to a small-sized hospital for adopting a data acquisition system with a 1-min data collection interval

Item	Year 1	Year 2	Year 3	Year 4	Year 5	Total
System	1200					1200
Operating and maintenance		120	126	132.30	138.92	517.22
Data storage (on-site)	29	55	81	107	133	405
Data storage (cloud storage)	8.1	16.2	24.3	32.4	40.5	121.5
Total annual costs (on-site)	1229	175	207	239.30	271.92	2122.22
Total annual costs (cloud storage)	1208.1	136.2	150.3	164.7	179.42	1838.72

Similarly, we may estimate the costs for various data collection intervals including 15-, 30-, 60-, 90-, 120-, and 180-min intervals, and various hospital sizes. The major difference across these cases is in data storage cost due to the lower volume of data that would be stored as a result of longer collection interval lengths. Specifically, we estimate the volume to be on the order of 10 TB, 7.5 TB, 5 TB, 2.5 TB, 1 TB, and 0.75 TB for 15-, 30-, 60-, 90-, 120-, and 180-min data collection intervals. In addition, the volume of data generated and system costs grow proportionally to the hospital size. However, larger hospitals may have better bargaining power, which may help reduce the overall costs for them.

Cost-Benefit Analysis from a Societal Perspective

In this section, we perform a CBA to illustrate the effect of adopting an automated data acquisition system from the societal perspective. We first estimate the overall benefits of adopting the system to society. Next, using the costs described in Section 3.2.1, we present the CBA for a 1-min data collection interval. Lastly, we extend the discussion to 15-, 30-, 60-, 90-, 120-, and 180-min intervals and discuss the implications for medium- and large-sized hospitals.

To evaluate the expected benefit of adopting an automated data acquisition system, we use a decision tree, which we calibrate with the data reported in the literature, the model developed in Section 3.1 and expert opinion. The decision tree is illustrated in Fig. 2. We particularly use this decision tree to calculate the relative economic benefit for a certain collection interval versus the base case (EMR system).

Fig. 2.

Decision tree for sepsis patient with a 1-min data collection interval from a societal perspective

The incidence rate of sepsis is approximately 6% across all patients admitted to hospitals [9]. As discussed earlier, the rate of early detection and false alarm (or equivalently, the sensitivity and the complement of specificity of each classification model developed in Section 3.1) is a function of the availability of data. In Fig. 2, the mean sensitivity is estimated from the results of the random forest models presented in Fig. 1 and Table 4, and is set to 73% for the 1-min data collection interval. Similarly, specificity is set to 73% for this data collection interval. Compare these numbers with the 66% sensitivity and 65% specificity for EMR systems achieved using RNN models. We also performed the analysis when using patient data from the first 6 h after admission and obtained a slightly lower, but overall similar performance to using 12 h data with respect to sensitivity and specificity.

As discussed in Section 1, there is a clear distinction between sepsis patient outcomes following early and late detection. In Fig. 2, the probabilities for survival and death for both early and late detection are based on results from the literature [7] and expert opinion in cases where data are not readily available. A conservative estimate of the mortality rate of patients following early and late detection is 20% and 80%, respectively [7]. Note that the elevated mortality rate due to the late detection of sepsis, which may contribute to severe sepsis and ultimately septic shock, is very well established in the literature [7]. However, the exact figures on the overall mortality rate following discharge for sepsis survivors after early and late detection are not widely available. Early and late detection of sepsis contribute to long-term outcomes of patients. Patients discharged with sepsis have an elevated risk of death, further sepsis, and readmission [59–67]. Severe sepsis survivors are at 20% risk of death following discharge [68]. The readmission rate of severe sepsis survivors is reported to be as high as 42.7% within 90 days [64]. In this study, we use point estimates of 5% and 30% for death due to sepsis complications for survivors following early and late detection, respectively [8]. Note that all estimated probabilities are generally site- and patient population-specific. The probabilities can hence be estimated for the given patient population of a given hospital using the data stored in the EMR system.

In Fig. 2, we use the estimated monetary value of $600,000 for death, along with the direct costs estimated in Section 2.2. Hence, for instance, in Fig. 2, if a severe sepsis patient receives treatment and dies, we account for the cost of death in addition to the direct cost of $26,000, hence the total cost of $626,000. Based on the decision tree in Fig. 2, the expected cost following early and late detection, respectively, equals 0.8 × (0.95 × $11,000 + 0.05 × $626,000) + 0.2 × $626,000 = $158,600, and 0.2 × (0.7 × $18,000 + 0.3 × $626,000) + 0.8 × $626,000 = $540,880. Hence, using a 1-min collection interval, with a 73% sensitivity, the total expected cost per sepsis patient may be estimated at 0.73 × $158,600 + 0.27 × $540,880 = $261,815.6. Compare this number with costs under the EMR system at 0.66 × $158,600 + 0.34 × $540,880 = $288,575.2. Hence, using the 1-min collection interval to complement the EMR system provides an additional $288,575.2 − $261,815.6 = $26,759.6 in relative benefit per sepsis patient from a societal perspective.

As discussed in Section 2.2, false positives may arise when attempting to predict sepsis patients, each of which costs the hospital on the order of $1000. Hence, based on the decision tree in Fig. 2, the expected cost per non-sepsis patient using a 1-min collection interval, with a 73% specificity, equals (1 − 0.73) × $1000 = $270. Compare this number with costs under the EMR system at (1 − 0.65) × $1000 = $350. Hence, using the 1-min collection interval to complement the EMR system provides an additional $350 − $270 = $80 in relative benefit per non-sepsis patient.

Finally, for the hospital with 150 beds, 3000 patients annually, and a sepsis incidence rate of 6%, an automated data acquisition system with 1-min collection interval holds a 3000 × (0.06 × $26,759.6 + 0.94 × $80 ) = $ 5,042,328 ≅ $ 5.04 million relative economic benefit per year.

We have conducted CBAs for data collection intervals including 1-, 15-, 30-, 60-, 90-, 120-, and 180-min intervals. Figure 3 presents the BCRs and payback periods for a 5-year period at 1-, 15-, 30-, 60-, 90-, 120-, and 180-min data collection intervals. As seen in the figure, for a 1-min collection interval, the BCR equals 13.71 ≫1, with a payback period of only 0.24 years. Hence, for the small-sized hospital considered, the benefits of adopting the automated data acquisition system with a 1-min data collection interval far outweighs the costs from a societal perspective. In addition, from Fig. 3, it is clear that a 1-min collection interval outperforms other choices in terms of the important KPIs, i.e., the BCR and the payback period.

Fig. 3.

Benefit-cost ratios and payback periods for various collection intervals for a small-sized hospital adopting a data acquisition system from a societal perspective

The incidence of sepsis and mortality rates generally vary based on hospital sizes. For instance, larger hospitals are reported to have a higher rate of mortality rates among sepsis patients [15]. As discussed, the volume of data generated and system costs generally increase with hospital size. Despite these variations, and based on our estimated values, the results of the CBAs conducted for medium- and large-sized hospitals are consistent with the results for small-sized hospitals. That is, in general, a 1-min collection interval results in higher KPIs across our numerical studies. These results suggest that the data storage costs associated with small collection intervals are better offset by the expected benefits obtained due to increase in performance of predictive models in detecting sepsis patients.

Discussion and Future Work

From the results obtained for the CBA reported in this paper, it is clear that the expected benefits associated with collecting granular data are at its maximum for small data collection intervals. The CBA indicates a high BCR and a low payback period of 13.71 and 0.24 years, respectively, for the minute-by-minute collection interval when accounting for all costs and benefits for small hospitals. The results are also replicated for medium and large hospitals. This indicates that an automated data acquisition system is highly beneficial from a societal perspective, especially when the data is leveraged using powerful machine learning algorithms. Hence, healthcare policy-makers may use the results to guide decision-making and perhaps design incentives with regard to the use of automated data collection systems in healthcare facilities considering its potential societal impacts.

In general, the approach proposed in this study may serve as a template to guide healthcare facilities in their investment decisions with regard to automated data acquisition systems in order to compare its associated costs and benefits. In addition, the CBA may be performed from the perspective of health insurance companies, take into account the costs to government-run programs such as Medicaid/Medicare, or be calibrated for specific patient populations and their corresponding costs/benefits, e.g., for the Veterans Health Administration. For instance, it would be interesting to examine the CBA from the perspective of health insurance companies to determine whether or not the cost of adopting automated data acquisition systems would result in an overall reduction in their health care expenditure due to the high cost of treating late-stage sepsis, the subsequently high rate of hospital readmission, and the long-term complications. In addition, a higher rate of patient survival would allow insurance companies to collect additional insurance premiums for years in the future from the survivors. Similarly, such an analysis could be performed from a hospital perspective when accounting for the total reimbursement cost of sepsis patients. For instance, based on the records from a major hospital in the USA, given the same primary diagnosis, the net loss to the hospital for the subset of patients who went on to develop severe sepsis was at least on the order of $4200 more than those who did not develop severe sepsis. Considering homogeneity in the payer source and procedures performed, this highlights the monetary benefits of early sepsis detection to hospitals.

Lastly, the same approach can be used to guide future decisions on wearable health devices, e.g., Apple Watch. The performance and operating capabilities of such devices are greatly dependent on the data collection interval, which affects storage, processing, and battery life specifications.

Conclusions

Precision medicine promises to bring patient-specific healthcare by considering the nuances of the individual’s physiology rather than extrapolating based simply on population averages. A significant component of this approach involves the capture and analysis of continuous physiological data. An increasing number of hospital systems are now electing to incorporate automated physiological data acquisition systems to streamline the largely manual effect involved in incorporating physiological data in the EMR. In this paper, we demonstrate the benefits of adopting automated data acquisition systems collecting physiological data that can be used in a predictive model against a major health problem. We identified the costs that are expected across small, medium, and large hospital systems that utilize continuous physiological data for both automating nursing workflow, and in potential predictive modeling scenarios. We demonstrate through our approach that a cost model can be developed to identify optimal sampling frequency of physiological data at a local hospital system level. Finally, we demonstrate that individual institutions may be able to gain further leverage of their costs by using our presented model to assess the utility of precision medicine and its impact on hospital expenditure.

Compliance with Ethical Standards

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Disclaimer

This manuscript has been co-authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).