A Hybrid Residual Network and Long Short-Term Memory Method for Peptic Ulcer Bleeding Mortality Prediction

Abstract

The prediction of patient mortality, which can detect high-risk patients, is a significant yet challenging problem in medical informatics. Thanks to the wide adoption of electronic health records (EHRs), many data-driven methods have been proposed to forecast mortality. However, most existing methods do not consider correlations between static and dynamic data, which contain significant information about mutual influences between these data. In this paper, we utilize a deep Residual Network (ResNet) consisting of many convolution units, which can jointly analyze different variables, to capture correlation information in and between static and dynamic variables. Furthermore, the Long Short-Term Memory (LSTM) method is used to extract temporal dependencies information from dynamic data. Finally, a deep fusion method is used to integrate these different types of information to improve mortality prediction. Experiment results on Peptic Ulcer Bleeding (PUB) mortality prediction show that the proposed method outperforms existing methods and achieves an AUC (area under the receiver operating characteristic curve) score of 0.9353.

Introduction

The prediction of patient mortality, enabling detection of patients with high-risk of unfavorable clinical outcomes, is a significant yet challenging research problem. The wide application of electronic health records¹ (EHRs) in hospitals has produced a large volume of detailed digital health data (diagnosis codes, laboratory parameters, medications, etc.) of patients. It provides an unprecedented opportunity for developing data-driven methods, especially deep learning techniques^2,3, to promote clinical predictive research^4,5, such as patient mortality prediction^6,7, early detection of disease^8,9, and clinical decision making¹⁰. EHRs often contain both static data (e.g., birthdate, gender, total doses of concomitant drugs) and dynamic sequence data (e.g., lab test results).

Medical dynamic time series are observations of several important indexes changing over time and contain significant information about changes in patient health status. Extracting temporal dependencies information from medical time series to build clinical prediction models has become a hot topic in recent years^3,11,12,13. There are also some machine learning approaches which make use of static information to build medical prediction approaches^14,15. However, further improvement of these single dynamic-information-based or single static-information-based methods with respect to their prediction accuracy is highly challenging, because of the inherently limited value of the information contained in the input variables.

Some new methods also combine static and dynamic data to build medical forecasting models^16,17. However, these methods treat static and dynamic data as two isolated objects and simply concatenate features extracted from static data and features extracted from dynamic time series data. Then, this concatenation is mapped to the outputs without considering the correlations between the static and dynamic variables. However, the correlations between static and dynamic variables are important because of the relationship of mutual influences between them. Static variables affect the future trajectories of dynamic sequence variables. For example, differences in static variables, such as gender and birthdate, can correspond to differences among patients with respect to their personal physiques, leading in turn to differences in their dynamic time series. Conversely, dynamic time series can influence the value of some static variables. For example, a doctor may use the dynamic changes of a patient’s measurements to determine the patient’s total doses of drugs during treatment.

In this paper, we propose a hybrid Residual Network (ResNet) and Long Short-Term Memory (LSTM) method, which integrates correlation information between static and dynamic data, and temporal dependencies information contained in dynamic data, to improve Peptic Ulcer Bleeding (PUB) mortality prediction accuracy. Specifically, the deep ResNet contains many convolution units to analyze different variables jointly and has a residual learning framework to enable the ResNet to become deeper than traditional networks without the vanishing gradient problem. So it has stronger modeling capacity than traditional networks. We utilize the deep ResNet method to capture information about inner-channel correlations among different original static variables (static variables collected from EHRs directly), different new static variables (static variables refined from dynamic data), and moreover cross-channel correlations between original and new static variables. Furthermore, to extract temporal dependencies information from irregularly sampled PUB dynamic data, we modify the standard Dynamic Time Warping (DTW) model to align a large quantity of irregularly sampled time series. Then, the LSTM model, which avoids the vanishing gradient problem by controlling different gates in the memory cell, is used to model the aligned sequence data. Finally, we use a deep Multi-Residual Multi-Scale Network to fuse the correlation information extracted from diversity layers of ResNet and temporal dependencies information extracted from various layers of the LSTM to improve patient mortality prediction. Experiment results demonstrate that our hybrid ResNet and LSTM approach can well integrate different types of information to achieve higher prediction accuracy than other state-of-the-art methods.

Related Algorithms

Dynamic Time Warping Classical Dynamic Time Warping¹⁸ (DTW) is an algorithm designed to align a single pair of series. Suppose there are two time series T and O with lengths of n and m respectively:

$$\begin{array}{cc}T=t_1,t_2,\cdots ,t_i,\cdots ,t_n\hfill & \hfill O=o_1,o_2,\cdots ,o_j,\cdots ,o_m\end{array}$$

To align these two time series, we build an n-by-m matrix, at which the (ith, jth) element contains the distance between t_i and o_j. The Euclidean distances are often used to measure this distance, namely d(t_i, o_j) = (t_i - o_j)². The warping path W consisting of a contiguous set of matrix elements is used to define the aligning relationship between series T and O, whose kth element is defined as $w_k={(i,j)}_k$. W can be represented as:

$$\begin{array}{cc}W=w_1,w_2,\cdots ,w_k,\cdots ,w_K& \max (n,m)\le K<n+m-1\end{array}$$

W is subjected to three conditions:

Boundary: w₁ = (1,1) and w_K = (n, m).

Monotonicity: Given w_k = (i, j)_k then $w_{k-1}={(i^{\prime },j^{\prime })}_{k-1}$ where $i-i^{\prime }\ge 0$ and $j-j^{\prime }\ge 0$.

Continuity: Given $w_k={(i,j)}_k$ then $w_{k-1}={(i^{\prime },j^{\prime })}_{k-1}$ where $i-i^{\prime }\le 1$ and $j-j^{\prime }\le 1$.

The optimal path can be efficiently found by using dynamic programming to minimize the dissimilarity distance of aligned series under the three conditions given above.

Deep Residual Network Deep Residual Networks¹⁹ are neural networks containing many basic blocks that consist of a residual module F, namely a stack of several layers, and a shortcut connection. By connecting several layers with shortcuts, the network can avoid the vanishing gradient problem, which is a problem that very deep plain neural networks often suffer from. As a result, the Residual Network can train very deep networks and gain stronger modeling capacity than other plain neural networks. Using X_i to represent the input of the ith basic block, the output is:

$$Y_i=F_i(X_i)+X_i$$

where F_i(·) is the connection of some batch normalization units²⁰, convolution units and rectified linear units.

Long Short-Term Memory Long Short-Term Memory³ (LSTM) is a modified version of recurrent neural networks (RNNs), specifically introduced to solve the vanishing gradient problem. LSTM has a memory unit to encode what knowledge has been learned, and learn when to forget and update hidden states when new information is given. The operation of the memory unit is controlled by three gates: input gate i, forget gate f, and output gate o. The update function and output are defined as follows:

$$\begin{array}{cc}{i}_t=\sigma (W_{ix}{x}_t+W_{im}{m}_{t-1}+b_i)\hfill & \hfill f_t=\sigma (W_{fx}{x}_t+W_{fm}{m}_{t-1}+b_f)\\ o_t=\sigma (W_{ox}{x}_t+W_{om}{m}_{t-1}+b_0)\hfill & \hfill g_t=\phi (W_{cx}{x}_t+W_{cm}{m}_{t-1}+b_c)\\ c_t=f_t\odot c_{t-1}+i_t\odot g_t\hfill & h_t=o_t\odot c_t\hfill \end{array}$$

where $\sigma (\text{x})={(1+e^{-x})}^{-1}$ is a sigmoid nonlinearity that maps inputs into the range of [0, 1] and φ(x) is a hyperbolic tangent nonlinearity. W are matrices that represent the parameters of the gates that are trained in the learning process and ⊙ means the product with the value of a gate. By controlling multiple gates, the LSTM method can prevent the vanishing gradient problem and capture temporal dependencies.

Methods

Figure 1 gives the overview of our proposed method for PUB patient mortality prediction. We first extract several types of important static information, which can represent the general situation of the time series data, from the irregularly sampled dynamic data and use a modified DTW method to align this huge amount of irregularly sampled PUB sequences data. Then, we utilize the deep Residual Network to capture information about correlations in and between the original static data and the new static data refined from the dynamic data. Furthermore, we use the LSTM method to extract temporal dependencies information from the aligned time series data. Finally, we utilize a deep fusion method to integrate these different types of information to improve mortality prediction.

Figure 1.

The overview of the proposed method.

Data processing

The irregularly sampled PUB dynamic dataset causes difficulty in building a prediction model because time series with different sampling intervals are not comparable. We transform the original data into regularly spaced time series by calculating the average value of the time series in time intervals with equal length (in the experiments, we set the length of the time interval to half a year or one year, and compare their respective results). This averaging process helps us to capture the essential trends of original series, thus reducing sensitivity to measurement error and improving robustness. In the next step, we will extract several types of static information and align dynamic data.

1) Extracting Static Information

To mine correlations between static and dynamic variables, we extract several new static variables that can represent the general state of the dynamic data. This will allow us to capture information about correlations between static data and dynamic data by analyzing the relationships between the original and newly extracted static variables. The first variable we extract is the average value of each dynamic time series. This information is crucial because it provides a basic overall idea of the total time series. The second variable we extract is whether the dynamic series have missing data at different time intervals. Most existing methods solve the missing data problem by simply using zero or mean values to impute missing values¹⁶. In this study, we believe that missing data can also be valuable, because they reflect doctors’ judgments about the health of their patients. For example, if a doctor believes that a patient is in a good state, the doctor will most likely arrange for the patient to take fewer measurements to spare the patient pain and expense. Based on this consideration, we utilize moving window periods to generate labels indicating whether time series have missing data at different time intervals so as to describe the changing characteristics of the dynamic time series.

2) Time Series Alignment

The PUB dynamic series data have different lengths. However, the modeling of machine learning algorithms for time series data is often built on the assumption that the data are aligned. Temporal discrepancies in series data will lead to ill-generalizable models, which in turn affect forecasting performance. Thus, it is necessary for dynamic series data to be aligned. There are many alignment algorithms²¹, among which DTW is one of the most popular algorithms. DTW aims at aligning two sequences by warping the time axis iteratively until an optimal match is found. However, standard DTW is designed to align a single pair of time series²² and is not practical for multiple time series (MTS) alignment because it cannot ensure that different pairs of aligned time series have the same length. Although some enhanced methods have been proposed for MTS alignment^23,24, they can only handle a small number of time series. However, for EHR dataset, the number of time series data is often thousands or even millions.

To solve this problem, we propose a simple but practical aligning framework based on two basic ideas: 1) For each dynamic feature, there exists a template time series that can represent the essential changing tendency of this feature for this specific group of people; 2) The standard DTW method is modified to align time series to the length of the template time series. To identify a suitable template time series, we first filter out dynamic data that are affected by the missing data problem, and then calculate mean values of remaining time series at different moving window periods and use these mean values as the template time series, which has strong robustness and can represent the essential changing trends of the chosen feature within this group of people. Then, we modify the standard DTW method by adjusting its continuity constraint. For instance, we set the step size as {[1, 0], [1, 1]}. Taking [1, 0] as an example, the value 1 means that the warping path moves one step along the direction of the template time series T, while the value 0 at the second element means that the warping path does not move along the direction of the time series O. It can be seen that all these step size settings have a value of 1 at the first element, which means that no matter whether the warping path is [1, 0] or [1, 1], it always has a sub-movement along the direction of T. Together with the boundary condition that the start and end cells of the warping path are the start and end points of these aligning time series, the length of aligned time series is equal to the length of T. This enables us to align a large quantity of PUB time series.

Extracting Correlation and Temporal Information

Through the above data processing, the data are now regularly spaced and have the same length. Next, we extract correlation and temporal information from the PUB data. First, we extract correlation information among different types of data. As mentioned above, the PUB data include both static and dynamic data. There are correlations both in and between different static and dynamic variables. Thus, we utilize a deep Residual Network, which has many convolution units to jointly analyze several variables, to capture information on the correlations among the original static variables and the new static variables refined from the dynamic data. Furthermore, Residual Network (ResNet) has a residual learning framework, which facilitates the training process and allows it to become deeper than traditional networks, and hence has stronger modeling ability than traditional methods. The architecture of the ResNet that we use to extract correlation features is given in Figure 2. The schemas of four kinds of basic blocks with different numbers of filters are shown in Figure 2(a)-(d) and the overall structure is given in Figure 2(e). Each basic block consists of a stack of 3 convolution layers, namely 1×1, 3×3, and 1×1 convolution units, and a shortcut connection. These convolution units can analyze several variables at the same time and thus capture the correlation information between static and dynamic variables. Meanwhile, the shortcuts ensure that these basic blocks can be stacked to increase the depth of the network, increasing the modeling capacity of the ResNet while avoiding the vanishing gradient problem. Furthermore, to ensure that the ResNet can extract enough correlation information features at the last few layers, several full connection layers with different numbers of units are connected to the averaging pool. We implement the ResNet in Keras platform and train the network for 100 epochs by using Adam²⁵ to optimize the model parameters. The learning rate starts from 0.001 and then decays by 50% every 10 epochs.

Figure 2.

The architecture of the Residual Network. (a)-(d) Schemas of four kinds of single block with different numbers of filters. (e) The overall Residual Network structure.

Second, we extract temporal information from the dynamic time series data. The temporal dependencies information in medical time series data can reflect important information about changes in patient conditions and hence help detect high-risk patients. However, traditional methods tend to summarize the sequence ensemble into aggregate features, which ignores the temporal relationships among different elements. To improve prediction accuracy, we utilize an LSTM method to extract temporal information from the time series data. The LSTM model has memory cells that capture information about what has been calculated so far and decide how to use this information for further calculations. That is, the model uses sequential information for modeling instead of assuming that all inputs are independent of each other as traditional neural networks do. Furthermore, by controlling the opening and closing of input gate, forget gate and output gate, the LSTM method avoids the vanishing gradient problem, thus capturing temporal dependencies from sequence data more accurately. This motivates our choice of utilizing LSTM to capture temporal information from dynamic medical series. The LSTM used in our method contains five LSTM layers, followed by seven full connection layers whose numbers of units are 512, 256, 128, 64, 32, 16, and 2 respectively. Similarly to the ResNet, the LSTM is also implemented in Keras platform and trained by using Adam. We train the LSTM for 500 epochs with learning rate starting from 0.001 and then decaying by 50% every 50 epochs.

Feature Fusion

With more layers, the deep neural networks can go deeper and extract features that have more informative relationships with the outputs. However, it is inevitable that some important information may be lost during information processing in the early layers of the deep neural networks. Considering this problem, we not only extract features from the last few layers of the deep neural network, but also from several shallow layers of the network, i.e., the correlation information and temporal dependencies information are extracted from multiple layers of the Residual Network method and the LSTM method respectively.

Figure 3 shows the schema for the Multi-Residual Multi-Scale Network that we use to fuse correlation and temporal features information. As shown in Figure 3(a), the basic unit of this fusion method is a Multi-Scale residual block. Unlike ordinary residual blocks, this block has different sizes of convolution units and thus can analyze different variables at multiple scales and from different aspects at the same time, thus fusing different features more comprehensively. Furthermore, as shown in Figure 3(b), this deep feature fusion method has a Multi-Residual network architecture. By increasing the number of residual functions, this deep Multi-Residual Multi-Scale feature fusion method becomes wider and can utilize more residual information in model learning, thus improving final prediction accuracy. Again, we implement the Multi-Residual Multi-Scale Network in Keras platform and use Adam as the optimizer for training the network. We train the network for 50 epochs. The learning rate starts at 0.0003 and is divided by 5 every 10 epochs.

Figure 3.

The schema for the Multi-Residual Multi-Scale feature fusion method

Results

Dataset and Experimental Design

The experiments are performed on a Peptic Ulcer Bleeding (PUB) EHR dataset collected at the Endoscopy Center, Prince of Wales Hospital, Hong Kong, from January 1, 2007 to December 31, 2016. This dataset is a retrospective cohort of 6,367 patients who were diagnosed with PUB. The dataset contains 35 types of static variables (e.g., birthdate, gender, and total doses of some concomitant drugs that patients took during the treatment) and 7 types of dynamic time series of lab test results. We use these 35 types of static variables as original static data and the 7 types of lab test results as dynamic data. The goal of our research is to predict whether the patient dies within 10 years after being diagnosed with PUB. 48.77% of all the patients are mortality positive (i.e., the patient died). Accurate mortality prediction can help detect high-risk patients and optimize the patient management plan. We conduct this research using a two-fold cross-validation methodology. The experiments are evaluated using two indexes: receiver operator characteristic (ROC) curves and the area under the ROC curves (AUC).

As mentioned before, our experiments explore the influences on prediction accuracy of adding mean values and missing data labels to the original static data. Therefore, we use three kinds of combinations of static data: Static-I includes the original static data only; Static-II includes Static-I and mean values; Static-III includes Static-II and missing data labels. Our experiments also explore the influence of time interval length when resampling the irregularly sampled data into regularly spaced time series. We experimentally compare two resampling frequencies: Frequency-I (twice a year) and Frequency-II (once a year). As baseline methods we use Logistic Regression (LR) and Random Forests (RF), using the dynamic data with the resampling frequency of Frequency-II as the input. We implement these baseline methods using the scikit-learn²⁶ package.

Time Series Alignment Results

Figure 4 shows an example of using the modified DTW method to align two time series. As shown in Figure 4(a), the original time series and template time series vary in speeds of changes. We can observe that both time series contain a trend of first decreasing and then increasing. For the template time series, this trend appears between 2015 and 2017. However, for the original time series, the trend appears between 2014 and 2016. This difference causes an unequal length problem, making it difficult to mine temporal dependencies information from the original time series directly. By utilizing the modified DTW method to align these series, the unequal length problem can be solved. As shown in Figure 4(b), the aligned time series have the same length and trend. At the same time, the aligned time series still preserve some important information about specific changes in the health status of the patient. This demonstrates that the modified DTW method can successfully align a large quantity of PUB dynamic series, such that they all have the same length as the template series and can well align irregular time series data into regular time series data. These capabilities are very important for the subsequent modeling using machine learning methods.

Figure 4.

Example of using the modified DTW method to align time series. (a) Template time series and original time series before applying the modified DTW method. (b) Original time series is stretched to the length of the template time series after applying the modified DTW method.

Experiment Results

Table 1 demonstrates the AUC scores of different methods for the PUB patient mortality prediction. It can be seen that the proposed hybrid Residual Network and LSTM method achieves the best prediction performance among all of these methods with the highest AUC score, i.e., 0.9353. Furthermore, the statistical analysis demonstrates that the 95% confidence interval (CI) of the AUC scores of our proposed method is distributed between 0.9261 and 0.9440, which is higher than 95% CI of the AUC scores of other methods. These results statistically prove that our hybrid method outperforms other comparison methods. Similarly, by comparing the ROC curves of different methods, as given in Figure 5, we can see that the proposed hybrid method has better ROC curve than all the other comparison methods, which further demonstrates that the proposed method has the best mortality forecasting performance. All of these results support that by capturing the correlation information between different variables and integrating this information with temporal information, our proposed method can improve PUB mortality forecasting accuracy. Besides this, several other conclusions can also be obtained.

Table 1.

AUC scores of different methods

Figure 5.

ROC curves of different methods.

Firstly, prediction results of Residual Network (ResNet) for different static data are different. It can be observed from Table 1 that when the mean values of the dynamic time series are added as new static data, the AUC score increases from 0.8683 to 0.8937, which demonstrates that prediction accuracy increases after adding information about the mean values of the dynamic data. Similarly, as shown in Figure 5(a), the ROC curve of the ResNet for static variables becomes higher when the mean values of the dynamic time series are added as new static data. It can be inferred that the ResNet can jointly analyze static and dynamic variables to improve the prediction accuracy of PUB patient mortality. Furthermore, by comparing the prediction results of the ResNet for Static-II and Static-III, as shown in Table 1 and Figure 5(a), we can see that the ResNet has higher accuracy for the latter, which demonstrates that the use of missing-data labels can improve forecasting performance. This proves that missing data labels contain valuable information about the judgments of doctors regarding the health conditions of patients.

Secondly, the prediction results for data with different resampling resolutions are different. By comparing the ROC curves of the ResNet for Static-III with Frequency I and II (Figure 5(a)), of LSTM for dynamic variables with Frequency I and II (Figure 5(b)), and of our proposed hybrid method with Frequency II (Figure 5(d)), we can find that for each method, the ROC curve of a higher resampling resolution is better than for the corresponding method with a lower resampling resolution. Likewise, when the resampling resolution increases, the AUC scores of the same method also increase. This demonstrates that with a higher resampling resolution, more valuable information is preserved in the medical time series data regarding the dynamics of the disease, resulting in better prediction performance. For dynamic data with the same resampling frequency, it can be seen from Figure 5(c) that LSTM outperforms Logistic Regression (LR) and Random Forests (RF), which is consistent with the corresponding AUC scores obtained by these methods given in Table 1. These results demonstrate that deep learning models have stronger modeling capacity for the PUB clinical data than the traditional baseline models.

Thirdly, experiment results demonstrate that the integration of correlation information among different variables and temporal dependencies information in the dynamic time series data can increase the prediction performance for PUB patient mortality. From Figure 5(e) we can see that our proposed hybrid method has a higher ROC curve than the methods built on single correlation information or single temporal information. A similar trend can be observed from Figure 5(d) by comparing the ROC curves of different methods for data resampled at Frequency II. It can be concluded that for both resampling frequencies, the integration of correlation information and temporal dependencies information improves the mortality forecasting performance, and that the higher the resampling resolution, the better the forecasting.

Finally, experiment results demonstrate that our method outperforms two state-of-the-art approaches described in references [16] and [17], which also build prediction models utilizing both static and dynamic data. As shown in Figure 5(f), our method has a higher ROC curve than these two methods. The AUC score of our method is also larger. Both ROC curve and AUC score prove that our method outperforms these two state-of-the-art methods. It can be explained as follows. The methods in references [16] and [17] simply concatenate the static and dynamic features, and use a prediction layer to map these features to the output. In contrast, our method utilizes a deep ResNet to jointly analyze the original static variables and the new static variables extracted from the dynamic data to capture correlation information between these data. Therefore, although both our hybrid method and these two methods use the same data to build models, and both utilize similar methods, which all are special kinds of recurrent neural network (RNN), to extract temporal dependencies information, our proposed method utilizes the ResNet to capture correlation information and as a result achieves better mortality forecasting performance.

Discussion

In this study, we demonstrate that our proposed hybrid ResNet and LSTM method can successfully capture the correlation information between static data and dynamic data, and integrate this correlation information with temporal information extracted from dynamic data to improve mortality prediction of PUB patients. There are some critical discoveries in this study. Firstly, cross-channel correlations between static and dynamic data can improve forecasting performance. It is intuitively plausible that there should be correlations between static and dynamic variables, because they can influence each other. In recent years, although some methods that combine static and dynamic data to predict patient mortality have been proposed, these methods tend to simply concatenate static and dynamic features without considering their correlations. In our research, we utilize a ResNet with many convolution units to analyze different variables jointly, enabling us to capture correlation information between different data. Experiment results demonstrate that our method outperforms two other state-of-the-art methods, which combine static and dynamic data to build medical forecasting models without considering correlations between them. This proves that the mining of correlation information helps to improve mortality prediction performance, which can better detect patients with high risks of negative outcomes. Secondly, our research exploits the missing data phenomenon to improve mortality forecasting accuracy. This strongly implies that missing data contain information about the judgments of doctors regarding the conditions of patients. Experiment results prove the effectiveness of utilizing missing data labels to improve forecasting performance. Therefore, instead of simply using zero or mean value to implement missing values, we can find out the mechanism that causes the missing data phenomenon, and in turn make use of this phenomenon to improve our medical forecasting models. Thirdly, our research confirms an intuitive assumption about the influence of sampling resolution on the prediction performance, i.e., increasing sampling resolution can to some extent help reserve more valuable information about the conditions of patients and thus improve forecasting accuracy. Our experiments compare two resampling frequencies, i.e., twice a year and once a year, and find that models using twice-yearly resampled data achieve higher prediction accuracy. This suggests that if patients undergo health tests at the hospital more frequently, the recorded data will contain more information about their health conditions. Finally, our research proves that the integration of correlation information and temporal information can effectively improve the forecasting accuracy of patient mortality. It can be inferred that the EHR data contain multiple kinds of information, including temporal and correlation information. Mining and integrating different useful information will enable us to model changes of disease more accurately and obtain better clinical prediction results.

Limitations and Future Work

Despite promising performances for mortality prediction, there are still aspects in which the proposed method can be improved, which we will address in future work. Firstly, in this study, we only use numeric data records to build the models. Besides numeric datasets, there are also some text datasets, such as diagnoses records. The utilization of these records will tap new sources of information on patients’ conditions and improve prediction performance. In our future work, we will utilize text mining and natural language processing methods to extract useful information from text datasets of medical diagnoses and use this text information to improve patient health status prediction models. Secondly, we compare prediction performances for dynamic time series data resampled at two resampling frequencies, namely twice a year and once a year. However, we have not conducted experiments at higher resampling resolution. In our future work, we will increase the sampling resolution and compare corresponding prediction performances, to identify an optimal sampling frequency that can preserve all of the necessary information about the dynamics of the disease, while minimizing times that patients visit hospitals to record their health data, which can be expensive and painful. Thirdly, we believe that by performing sub-analyses on the population, we will be able to group patients into different types according to their disease characteristics and build a personalized method according to the characteristics of each sub-group of patients to improve prediction performance. However, due to the limited size of the PUB dataset, such sub-analyses would divide the dataset into relatively small groups, which may impair the training of the model. In our future work, we will perform sub-analyses on bigger clinical datasets to build more accurate personalized prediction methods.

Conclusion

The forecasting of patient mortality, which helps detect high-risk patients, is a significant yet challenging research problem. In this study, we propose a hybrid method that makes use of a deep Residual Network and the LSTM method to extract correlation information between static and dynamic data, and temporal dependencies information contained in dynamic time series data, and utilizes a deep Multi-Residual Multi-Scale Network to fuse these different types of information to improve mortality prediction of PUB patients. Experiment results demonstrate that our proposed hybrid approach outperforms several existing approaches.