An Early Warning System for Hypoglycemic/Hyperglycemic Events Based on Fusion of Adaptive Prediction Models

Abstract

Introduction

Early warning of future hypoglycemic and hyperglycemic events can improve the safety of type 1 diabetes mellitus (T1DM) patients. The aim of this study is to design and evaluate a hypoglycemia/hyperglycemia early warning system (EWS) for T1DM patients under sensor-augmented pump (SAP) therapy.

Methods

The EWS is based on the combination of data-driven online adaptive prediction models and a warning algorithm. Three modeling approaches have been investigated: (i) autoregressive (ARX) models, (ii) auto-regressive with an output correction module (cARX) models, and (iii) recurrent neural network (RNN) models. The warning algorithm performs postprocessing of the models′ outputs and issues alerts if upcoming hypoglycemic/hyperglycemic events are detected. Fusion of the cARX and RNN models, due to their complementary prediction performances, resulted in the hybrid autoregressive with an output correction module/recurrent neural network (cARN)-based EWS.

Results

The EWS was evaluated on 23 T1DM patients under SAP therapy. The ARX-based system achieved hypoglycemic (hyperglycemic) event prediction with median values of accuracy of 100.0% (100.0%), detection time of 10.0 (8.0) min, and daily false alarms of 0.7 (0.5). The respective values for the cARX-based system were 100.0% (100.0%), 17.5 (14.8) min, and 1.5 (1.3) and, for the RNN-based system, were 100.0% (92.0%), 8.4 (7.0) min, and 0.1 (0.2). The hybrid cARN-based EWS presented outperforming results with 100.0% (100.0%) prediction accuracy, detection 16.7 (14.7) min in advance, and 0.8 (0.8) daily false alarms.

Conclusion

Combined use of cARX and RNN models for the development of an EWS outperformed the single use of each model, achieving accurate and prompt event prediction with few false alarms, thus providing increased safety and comfort.

Introduction

Prompt recognition of upcoming hypoglycemic and hyperglycemic situations is crucial for the safety of insulin-dependent patients. New technologies have permitted the combination of an insulin delivery pump with a continuous glucose monitor (CGM) system to provide the user with subcutaneous glucose readings with high frequency, which is called sensor-augmented pump (SAP) therapy. Randomized controlled trials have shown that SAP therapy reduces glycosylated hemoglobin (HbA1c) levels and glycemic variability, as compared with the multiple daily insulin injection (MDII) scheme, in both adults and children. ^1–4 Furthermore, CGM provides greater insight into the patient’s glycemic status and can lead to more efficient glucose prediction in the near future.

Various approaches have been proposed for predicting glucose concentration, most of them based on regression models⁵ such as autoregressive (AR) and autoregressive moving average (ARMA) models and machine learning approaches, such as artificial neural networks (ANNs) and support vector machines. Regression models range from simple AR models using batch identification ^6,7 to more complex approaches, such as recursive ARMA models⁸ and AR models using smoothed data and regularized least squares.^9,10 External inputs including insulin and carbohydrate (CHO) have also been considered (ARX), using both batch and recursive identification.¹¹ Methods for adaptive ARMA model identification using weighted recursive least squares have been proposed for early hypoglycemia detection.¹² In an effort to capture the complex, nonlinear dynamics of glucose time series, machine learning approaches have been introduced into the field of glucose prediction. These mainly include feed-forward gradient-descent back propagation ANN using multiple input information, such as insulin dosage and nutritional intake^13–15 and feed-forward Levenberg–Marquardt back propagation ANNs, in which only present and past glucose samples from a CGM system are applied as inputs.¹⁶ Recurrent neural networks (RNNs) have also been proposed in combination with compartmental models to estimate plasma insulin concentration and meal absorption information, either for control ^17,18 or for glucose prediction based on CGM, MDII schemes, and food data. In all cases, they outperformed the feed-forward ANNs.¹⁹ A new method has been proposed in which glucose prediction is based on support vector regression²⁰ using information of insulin food and physical activity.

The various algorithms proposed for glucose prediction present different advantages and drawbacks. Moreover, a single model’s output is subjected to measurement noise sensitivity and model-specific properties. Fusion of multiple models’ outputs may provide a more generalized and robust performance by combining the different and, in many cases, complementary advantages of each modeling approach. The combination of multiple predictors has been reported in various applications with superior results than with a single model use.^21,22 Five prediction algorithms, based on statistical and numerical prediction, filtering, and linear extrapolation, have been fused in order to formulate a voting system for triggering hypoglycemia alarms.²³ In another study, Bayesian methods were used to combine multiple plasma glucose predictors.²⁴

The aim of this study is to develop an early warning system (EWS) dedicated to prompt and efficient prediction of abnormal metabolic situations in type 1 diabetes mellitus (T1DM) patients under SAP therapy. The system combines real-time adaptive, data-driven glucose prediction models and a warning algorithm for the postprocessing of the models’ outputs and the generation of alerts if a hypoglycemic or hyperglycemic event is expected to start in the near future. The present study follows our previous work in glucose prediction,²⁵ in which online adaptive AR, ARX- and RNN-based models were developed and comparatively assessed with in silico data.

The new contribution of this study comprises extension of the ARX models with an output correction module, investigation of patient-specific fusion of multiple models’ outputs, combination of the prediction models with a warning algorithm to develop an EWS and evaluation using real patient data.

Methods

Subject Data

Sensor glucose and insulin pump data from 23 T1DM patients (age 17–70 years, HbA1c 7.3% ± 0.7%, body mass index 24.2 ± 3.0 kg/m²) under SAP therapy were utilized in the study. All patients used Medtronic insulin pumps (Medtronic MiniMed Inc., Northridge, CA) combined with a real-time, CGM system during everyday living conditions. The sensor glucose values were equally sampled every 5 min. For each patient, half of the data set was used as training set for the identification of the models’ architecture and parameters. The other half was used for the evaluation of the models. Table 1 presents the statistical parameters of the data set.

Table 1.

Statistical Parameters of the Patient Data Set in Mean ± Standard Deviation Values

Data statistics per patient	Training set	Evaluation set
Data collection time (days)	5.30 ± 1.40	4.83 ± 1.80
Number of CGM samples	1382 ± 419	1313 ± 523
Number of hypoglycemic eventsa	7.43 ± 6.64	6.57 ± 5.66
Number of hyperglycemic eventsa	14.00 ± 9.22	13.22 ± 7.17
Hypoglycemic event duration (min)	51.55 ± 26.31	51.82 ± 36.16
Hyperglycemic event duration (min)	122.60 ± 55.86 55(169.3)	137.73 ± 69.98
Data statistics total data set
Data collection time (days)	122	111
Number of hypoglycemic events in data set	171	151
Number of hyperglycemic events in data set	322	304

^aA hypoglycemic (hyperglycemic) event was defined as sensor glucose values below 70 mg/dl (over 180 mg/dl) for at least 10 min (two consecutive sensor glucose measurements).

Glucose Prediction Models

Two data-driven glucose modeling strategies were investigated in the study, all able to provide prediction of glucose profile in prediction horizons (PHs) of 15, 30, and 45 min.

Online Adaptive Autoregressive Models with Output Correction Module

Autoregressive models using past and present glucose and insulin information were developed. The models were individually identified for each patient in terms of architecture (model order) and parameters. The parameter vector is online adaptive during evaluation in order to capture intrapatient variability. An extensive and detailed description of the models’ algorithmic background and design methods is presented in our previous work.²⁵

A novel module was added to the ARX model in order to correct the model’s output based on the estimated prediction error. The principle behind model output correction lies on the development of a submodel for the association of the prediction error of a specified PH to current glucose features and the use of this submodel during evaluation to modify the ARX model’s output.²⁶ After identification of the architecture and parameters, the model is applied to glucose prediction in PH = 15, 30, and 45 min on the training data set. Let Ĝ (t + PH|t^p) be the prediction of glucose at time t + PH by the model given the glucose and insulin information of period t^p = {t ,…, t − p}, where t − p is the oldest time sample used and t is the time when the prediction is performed. For each PH, prediction errors E₁₅, E₃₀, and E₄₅ are computed as

$$E_{\text{PH}}(t+\text{PH})=G(t+\text{PH})-\widehat{G}(t+\text{PH}|t^p),$$ 1

where G (t + PH) is the real glucose value at time t + PH. The prediction error was modeled as a linear combination of three glucose features: current glucose G (t) value and first- and second-order glucose derivatives, DG (t) and D²G (t), respectively, as shown in Equation (2). A different model was used for each PH (15, 30, 45 min):

$${\widehat{E}}_{\text{PH}}(t+\text{PH}|t)=a_{\text{PH}}G(t)+b_{\text{PH}}DG(t)+c_{\text{PH}}{D}^{2}G(t),$$ 2

where Ê _PH (t + PH|t) is the estimation of the prediction error at time t + PH based on the glucose features at time t when the prediction is performed. The parameters a_PH, b_PH, and c_PH were identified in least-squares sense for each patient and each PH based on the paired training glucose and prediction error values. The first-order glucose derivative was estimated as

$$DG(t)=\frac{G(t)-G(t-s)}{s},$$ 3

where s is the sampling time of glucose time series, in our case, s = 5 min. The second-order derivative was estimated similarly. During evaluation, the ARX model provides predictions of glucose for the three PHs. Each prediction is subsequently corrected based on the estimated expected prediction error Ê_PH as

$${\widehat{G}}_C(t+\text{PH}|t^p)=\widehat{G}(t+\text{PH}|t^p)+{\widehat{E}}_{\text{PH}}(t+\text{PH}|t),$$ 4

and Ĝ_C is the corrected model output.

Online Adaptive Recurrent-Neural-Network-Based Models

Recurrent, fully connected neural networks with two feedback loops were developed and individually identified for each patient. During evaluation, the models’ architecture (input/hidden/output neurons, hidden layers) remains unchanged, yet the models’ parameters (weights and biases) continue to adapt online based on new information on glucose and insulin. More details on the design and methods of the RNN-based models can be found in our previous work.²⁵

Early Warning System

The system’s purpose is to issue a warning when a hypoglycemic (hyperglycemic) event is predicted to start in the near future. The system incorporates a glucose prediction model and a warning algorithm for processing the model’s output and issuing an alert if needed. Prior to the design of an EWS for the prediction of hypoglycemic (hyperglycemic) events, some important notions should be defined.

• •Definition of event: A hypoglycemic (hyperglycemic) event was defined as sensor glucose values below 70 mg/dl (over 180 mg/dl) for at least 10 min or two consecutive sensor glucose measurements. This decision was made in order to exclude erroneous sensor fluctuations outside normoglycemia due to presence of noise.
• •Definition of correct warning: A correct warning is one that can be matched to a real event. However, matching of a warning to a real event is not a straightforward procedure. In order to distinguish correct warnings from false alarms, a time-range before the start of the event should be defined as acceptable for the inclusion of a matching (i.e., correct) warning. In this study, the maximum acceptable time range prior to an event start was defined as 45 min, equal to the largest PH used in the study.

Warning Algorithm

The prediction models naturally present errors and time lags (TLs), which cannot be avoided or neglected. The presence of noise in the sensor glucose measurements is an important deteriorating factor of the models’ performance. It has been further observed that increase in the PH results in higher prediction errors and TLs. On the other hand, the higher accuracy of short PHs is counteracted by the inadequately short-term notice for the patient, which may not leave enough time for the needed actions (insulin injection or CHO intake). To this end, in order to enhance the reliability of the predictions and the prompt event detection, the warning algorithm is designed to issue warnings based on the combined predictions of all three PHs. Based on this reasoning and the aforementioned definitions, a warning is issued if all the following rules hold:

• Rule 1. The current glucose value is within the normoglycemic range.

• Rule 2. The 15 min-ahead prediction and at least one of the 30 or 45 min-ahead predictions fall outside the normoglycemia bounds.

• Rule 3. The event is predicted for the first time.

Evaluation

The EWS was evaluated for its ability to predict upcoming hypoglycemic and hyperglycemic events using the following criteria:

• 1.Percentage of correct warnings: A warning is defined as correct if it was issued, at most, 45 min before the start of a real event (see definition described earlier).
• 2.Event detection time: Refers only to correct warnings and denotes the time between the issued warning and the start of the event.
• 3.Daily false alarms: A false alarm is a warning that could not be matched to a real event within 45 min from its triggering time.

Hybrid Early Warning System

A hybrid EWS, which combines more than one prediction model, has been investigated with the aim to explore the improvement of the aforementioned evaluation criteria when two different modeling strategies are fused. The fusion was implemented as a linear combination of the two models’ outputs. The final output of the fused model is formed as shown in Equation (5):

$${\widehat{G}}_{\text{fused}}=a{\widehat{G}}_1+(1-a){\widehat{G}}_2,$$ 5

where Ĝ_i is the output of model i and 0 < a > is a balancing factor between the two outputs. Increasing the value of a moves the fused model closer to model 1 and away from model 2. The balancing factor a is chosen based on the maximization of the following cost function:

$$J=\frac{{(\text{Percentage \hspace{0.17em} of \hspace{0.17em} Correct \hspace{0.17em} Warnings})}^2+{(\text{Event \hspace{0.17em} Detection \hspace{0.17em} Time})}^2}{1+{(\text{Daily \hspace{0.17em} False \hspace{0.17em} Alarms})}^2}$$ 6

All three values, before inserted in the cost function, are normalized in [0 1].

Figure 1 presents the flowchart of the hypoglycemia and hyperglycemia prediction process and warning generation.

Figure 1.

Flowchart of the hypoglycemia and hyperglycemia prediction process and warning generation.

Results and Discussion

Glucose Prediction Models

As a first step, the performance of the ARX, autoregressive with an output correction module (cARX), and RNN prediction models was evaluated on the basis of the root mean square error (RMSE), TL, and correlation coefficient (CC) for PH = 15, 30, and 45 min; the results are presented in Table 2.

Table 2.

Median (5th–95th Percentiles) Prediction Performance of the ARX, cARX, and RNN Models and All Prediction Horizons

PH	Criteria	ARX	cARX	RNN
PH = 15 min	TL	10.0 (0.5−10.0)	5.0 (0.0−9.5)	5.0 (0.5−10.0)
	RMSE	14.1 (9.5−26.9)	16.8 (11.3−33.8)	11.9 (7.7−22.7)
CC	0.97 (0.89−0.98)	0.96 (0.87−0.97)	0.98 (0.95−0.99)
PH = 30 min	TL	25.0 (15.0−25.0)	15.0 (10.0−24.5)	10.0 (5.5−15.0)
	RMSE	25.1 (17.5−44.7)	27.7 (19.0−49.5)	18.9 (12.8−32.3)
CC	0.90 (0.70−0.94)	0.90 (0.71−0.93)	0.94 (0.89−0.96)
PH = 45 min	TL	35.0 (30.0−40.0)	30.0 (20.5−39.5)	20.0 (10.0−25.0)
	RMSE	35.6 (24.4−51.4)	37.0 (25.4−61.1)	26.1 (17.2−39.8)
CC	0.80 (0.59−0.89)	0.82 (0.58−0.88)	0.90 (0.78−0.93)

It can be seen in Table 2 that the cARX model presents lower TL, increased RMSE, and comparable CC to the ARX model. The correction module increased the responsiveness of the model, especially to fast glucose changes, which resulted in faster reactions but, at the same time, larger fluctuations. This observation can be more easily illustrated in Figure 2, where an example of glucose prediction profile by the two models is plotted.

Figure 2.

Example of prediction profile from the ARX (red) and cARX (green) model in PH = 30 min.

The Wilcoxon rank sum statistical test, suitable for data not following normal distribution, was applied for the investigation of statistical differences between the ARX and cARX model. The test has shown that their difference was statistically significant in terms of TL (p − value <.05) but nonsignificant in terms of RMSE (p − value =.2) and CC (p − value =.6). This fact shows that the correction module contributed in the reduction of the TL without significant deterioration of RMSE or CC. From Table 2, it can be further seen that the RNN model outperforms both ARX and cARX with lower TL and RMSE and higher CC in all cases. The later result is in line with the evaluation of the models based on in silico data²⁵ and our preliminary results using real patient data.²⁷

Early Warning System

The EWS based on the ARX, cARX, and RNN models was evaluated according to the three previously described criteria and the performance is presented in the first three columns of Table 3. The Wilcoxon rank sum statistical test was used to detect statistical differences between the ARX-, cARX-, and RNN-based EWSs on the total number of hypoglycemic and hyperglycemic events of the evaluation data set.

Table 3.

Performance of the ARX-, cARX-, RNN-, and Fused cARN-Based Systems in Median (5th–95th Percentiles) for Hypoglycemia and Hyperglycemia Event Prediction

Evaluation criteria	ARX	cARX	RNN	cARN
Hypoglycemia
Correct alarms (%)	100.0 (94.0− 100.0)	100.00 (100.0−100.0)	100.00 (58.1−100.0)	100.00 (100.0−100.0)
Detection time (min)	10.0 (5.0−24.5)	17.5 (11.8−31.0)	8.4 (0.3−14.9)	16.7 (10.0−25.0)
Daily false alarms	0.7 (0.0−1.94)	1.5 (0.5−4.4)	0.1 (0.0−0.6)	0.8 (0.0−1.2)
Hyperglycemia
Correct alarms (%)	100.0 (90.1−100.0)	100.0 (93.8−100.0)	92.0 (70.5−100.0)	100.0 (95.3−100.0)
Detection time (min)	8.0 (1.7−13.9)	14.8 (8.8−20.6)	7.0 (4.7−15.2)	14.7 (5.1−19.25)
Daily false alarms	0.5 (0.0−1.2)	1.3 (0.4−3.2)	0.2 (0.0−0.7)	0.8 (0.0−1.4)

From Table 3, it can be seen that both ARX- and cARX-based EWSs present high accuracy with very small variability in correct warning generation. Nonsignificant statistical difference is found between the two systems both in hypoglycemia (p − value = .6) and hyperglycemia prediction (p − value = .9). The RNN-based EWS is less accurate, as it presents high variability among the patients and cases where many hypoglycemic and hyperglycemic events were missed (p − values <.001). In terms of detection time, the cARX-based EWS outperforms the other systems, as it predicts the hypoglycemic and hyperglycemic events 7.5 and 6.8 min earlier than the ARX-based EWS and 9.1 and 7.8 min earlier than the RNN-based EWS, while, in 95% of the events, the detection time was higher than 11.8 min for hypoglycemia and higher than 8.8 min for hyperglycemia. The ARX- and RNN-based EWSs present similar detection times with nonsignificant differences both in hypoglycemia (p − value = .3) and hypoglycemia (p − value = .9) prediction. In terms of daily false alarms, the RNN-based system presents the best performance, with very few false alarms and very low variability, while the cARX-based is outperformed by both ARX- and RNN-based EWSs. All three systems were statistically different regarding this criterion.

The higher detection times and false alarms of the cARX-based EWS compared with the ARX-based EWS can be directly associated to the previously described differences between the cARX and ARX models. From the statistical analysis, it can be also seen than the ARX-based EWS is statistically close to the cARX-based EWS in terms of correct warning accuracy and to the RNN-based EWS in terms of detection time performance, while, in all criteria, it is outperformed by one of the two other systems. The results indicate that different modeling strategies can contribute to different aspects of an EWS performance, a fact that leads naturally to the next step of fusing their outputs. Based on the performance of the three model-based EWSs, the cARX-based and RNN-based systems were found to present the most complementary advantages, thus a hybrid EWS based on fusion of the cARX and RNN models was designed.

Hybrid Early Warning System

As mentioned in the previous section, fusion of the cARX and RNN models is designed as their linear combination balanced by factor a. The balancing factor a was individually chosen for each patient based on the maximization of Equation (⁶). Furthermore, a different value of a was chosen for hypoglycemia and hypoglycemia warning. The output of the fused model (autoregressive with an output correction module/recurrent neural network [cARN]) is formulated as

$${\widehat{G}}_{\text{cARN}}=a{\widehat{G}}_{\text{cARX}}+(1-a){\widehat{G}}_{\text{RNN}}.$$ 7

The performance of the cARN-based EWS is presented in the fourth column of Table 3 for the optimal choice of a for each patient. The cARN-based EWS presents the highest accuracy of correct alarms with the lowest variability for both hypoglycemia and hyperglycemia and median detection time of 16.7 and 14.7 min, respectively. Considering the pharmacokinetics of insulin and glucose, the detection times achieved by the cARN-based EWS can be sufficient for avoiding the upcoming hypoglycemic or hyperglycemic event if the patient takes the necessary actions for each occasion. In both cases, the EWS presented limited daily false alarms. Statistical analysis showed that the cARN- and cARX-based EWSs present nonsignificant statistical difference in terms of correct warnings accuracy and detection time (p − values < .001), in which criteria the latter showed the most outperforming behavior. In terms of daily false alarms, the hybrid EWS was found statistically different from both cARX- and RNN-based EWSs but significantly similar to the ARX-based EWS, which also presented limited daily false alarms. Figure 3 presents the histograms of the detection times of all the 151 hypoglycemic and 304 hyperglycemic events of the evaluation data set when the EWS is based on the cARX, ANN, and cARN models. It can be seen that the cARX-based system presents a relatively smooth distribution for all detection times, whereas the RNN-based EWS is concentrated in lower detection times. The hybrid cARN-based EWS shows increased frequencies in high detection times compared with the RNN-based EWS, with stronger peak around 10 min compared with the cARX-based EWS.

Figure 3.

Histograms of the detection times of the cARX-, ANN-, and cARN-based EWSs for the 151 hypoglycemic events (blue) and the 304 hyperglycemic events (red) of the total evaluation data set.

The results clearly show that fusion of models with complementary properties can achieve high prediction performances by preserving the qualities of each modeling approach.

Figure 4 presents an example of a hypoglycemic event prediction by the EWS based on the cARX, RNN, and fused cARN models.

Figure 4.

Prediction of a hypoglycemic event start and alarm generation by the system when based on cARX, RNN, and hybrid cARN models. The vertical arrows denote alarm time, and the transparent squares denote the estimated time range of event start.

Conclusions

Data-driven glucose prediction models were proven useful for early recognition of upcoming dangerous metabolic situations. Real-time adaptive ARX, cARX, and RNN models were developed and evaluated with satisfactory performance in both hypoglycemia and hyperglycemia prediction. The combination of the three models with a warning algorithm to generate hypoglycemic and hyperglycemic alarms resulted in the development of a personalized EWS. The cARX- and RNN-based EWSs presented complementary qualities, leading to the development of the hybrid cARN-based EWS. The hybrid system managed to preserve the high accuracy and detection times of the cARX-based EWS and the low false alarms of the RNN-based EWS, combining both safety and comfort. The results indicate that different modeling techniques should be extensively investigated and comparatively assessed to determine complementary properties. Successful combination of complementary approaches can lead to more accurate representation of the glucose time series and significant improvement of the prediction performance. Future research steps will include the integration of the EWS into a control algorithm for the artificial pancreas in order to function as a safety-supervision mechanism. Augmentation of the system with additional modeling algorithms as well as alternative fusion techniques for multiple models’ outputs will be also investigated.

Glossary

(ANN)

artificial neural network

(AR)

autoregressive

(ARMA)

autoregressive moving average

(ARX)

autoregressive with external input

(cARN)

autoregressive with an output correction module/recurrent neural network

(cARX)

autoregressive with an output correction module

(CC)

correlation coefficient

(CGM)

continuous glucose monitor

(CHO)

carbohydrate

(EWS)

early warning system

(HbA1c)

glycosylated hemoglobin

(MDII)

multiple daily insulin injection

(PH)

prediction horizon

(RMSE)

root mean square error

(RNN)

recurrent neural network

(SAP)

sensor-augmented pump

(T1DM)

type 1 diabetes mellitus

(TL)

time lag

Funding

This work was funded by the Bern University Hospital, “Inselspital,” Switzerland.