An Enhanced Model Predictive Control for the Artificial Pancreas Using a Confidence Index Based on Residual Analysis of Past Predictions

Abstract

Background:

Model predictive control (MPC) performance depends on the accuracy of the prediction model implemented by the controller. Complex physiology and modeling limitations often prevent the ability to provide long and accurate glucose predictions, which results in the need to account for prediction errors.

Method:

Optimal insulin dosage by Zone-MPC is calculated by solving an optimization problem in which a scalar index is minimized by penalizing relative input deviations and glucose predictions out of the reference zone. The controller’s tuning parameters are the penalties on the input variable (insulin). Positive and negative relative inputs are penalized differently. A dynamic adaptation of the tuning parameters based on the accuracy of the model in recent history is implemented in this article and compared in silico to aggressive and conservative tunings of the same controller structure.

Results:

Similar average glucose and time in the safe glucose range (70-180 mg/dL) are achieved for the adaptive design and traditional controller configurations. However, percentage time under 70 mg/dL is significantly reduced, both for announced meals using bolus compensation and unannounced meals with a meal detection algorithm triggered bolus. No differences in the average insulin delivered were observed between the adaptive design and the conservative or aggressive tuning for the bolus strategy, and the adaptive controller delivered less insulin in the other scenario considered.

Conclusions:

The adaptive strategy provides safe and effective glucose management as well as significant reduction of hypoglycemia events. No abnormal insulin delivery profiles were observed upon the application of the adaptive strategy.

The artificial pancreas (AP) is a device that aims to automate diabetes treatment (close the loop) by continuously adjusting insulin delivery based on glucose fluctuations.¹ Numerous advances on the AP have been done in recent years, especially for treatment of type 1 diabetes mellitus, with several research teams completing closed-loop experiments in controlled environments,^2-4 and more recently in an ambulatory setting and at home.^5-8

The core of any AP is a control algorithm that calculates the appropriate insulin dosing to maintain glucose concentration at a safe healthy level at all times. The most common controllers used in the AP are algorithms based on model predictive control (MPC),^9,10 proportional-integral-derivative approach,¹¹ and fuzzy-logic design.¹² The author’s group has been evaluating an MPC algorithm that is personalized to the patients clinical information¹³ and an MPC with a reference defined as a glucose zone.^14-16

MPC controllers use prediction models to estimate future glucose trajectories. However, mathematical model predictions introduce uncertainty, and model individualization remains unsatisfactory for data-based models^17-19 or physiology-based models. To improve patient safety, this uncertainty can be quantified,^20-23 and a confidence interval can be estimated around the current glucose value so that the control actions are adapted²⁴ to act within the boundaries of confidence.

In the work presented in this article, the model prediction accuracy is quantified by a rigorous analysis of the prediction residuals. The integration of the information from the prediction residuals into the control algorithm may lead to a new family of adaptive control algorithms. These algorithms would act aggressively to eliminate both hyper- and hypoglycemia in case of a good confidence in glucose predictions and be very conservative in case of poor past predictions. As such a safer and more effective glucose control will be derived.

Methods

Matlab R2013b (Mathworks, Natick, MA) was used for the in silico simulations and analysis. p-values for statistical comparison where calculated using nonparametric permutations methods.²⁵ The data sets were generated using the Universities of Virginia/Padova metabolic simulator,^26-28 which has been accepted by the FDA in lieu of animal trials. The simulator implements 10 virtual subjects that represent the interpatient variability equivalent to the general diabetic population. A key part of the simulator is a continuous glucose monitor (CGM) model.²⁹ The CGM error is different for each execution of the software, introducing differences between simulations. The variability introduced by the random number generator for each simulation impedes a clear comparison of the 10 patients under different treatments. To minimize this effect, every subject is simulated 100 times under every scenario to sum a total of 1000 data sets. By doing this, the noise introduced by the CGM model is averaged over the different simulations of each subject, while still maintaining the variability that the CGM model introduces on the experiments.

The simulations consist of 24 hours of CGM data with 3 meals of increasing carbohydrates load (50 g, 75 g, and 100 g). Every simulation was preceded by 2 hours of open-loop monitoring as a warm up period. Two different meal compensation strategies were considered: (1) meal bolus compensation using the patient’s insulin-to-carbohydrate ratio and (2) unannounced meals to the controller, but detecting a meal occurrence by means of a meal detection algorithm (glucose rate increase detector, GRID³⁰). GRID triggers an alarm when certain conditions of glucose concentration and glucose trend are met, and an insulin compensation dose is administered, equivalent to a prandial bolus strategy. The compensation bolus is the equivalent to a 75 g carbohydrate (CHO) meal correction, based on the patient’s insulin-to-carbohydrate ratio. The controller also incorporates routines for patient safety including an algorithm that avoids delivering extra insulin according to recent insulin delivery history, and a security interlock in case of missing CGM samples.

zMPC Controller

The Zone MPC controller (zMPC) was developed originally by Grosman et al¹⁴ using a constant glucose reference band. This control scheme was later expanded using a control oriented prediction strategy proposed by van Heusden et al.³¹ The cost function for the MPC problem is given by:

$J(\cdot ):={\displaystyle \sum }_{k=1}^{N_p}{z}_k^2+{\displaystyle \sum }_{k=0}^{N_u-1}\left(\hat{R}{\hat{u}}_k^2+\stackrel{⌣}{R}{\stackrel{⌣}{u}}_k^2\right)$

$${\hat{u}}_k:=\max \left(u_k-u_{BASAL},0\right)\forall k\in {\mathbb{Z}}_0^{N_u-1}$$

$${\stackrel{⌣}{u}}_k:=\min \left(u_k-u_{BASAL},0\right)\forall k\in {\mathbb{Z}}_0^{N_u-1}$$

where $N_p$ is the prediction horizon (45 minutes), $N_u$ is the control horizon (25 minutes) and $z_k$ is the output error, understood as the distance of the prediction to the reference zone. $u_k$ is the insulin input, and $u_{BASAL}$ is the patient’s basal insulin. For additional details about the MPC problem see.^15,16,32 The parameters used to tune the controller in this work are $\hat{R}$ and $\stackrel{⌣}{R}$. Both parameters modulate the insulin delivery, but $\hat{R}$ penalizes insulin actions greater than the patient’s basal rate, while $\stackrel{⌣}{R}$ acts on the insulin action when below basal rate. Increasing the value of $\hat{R}$ makes the controller less responsive to upper glucose trends (hyperglycemia) which results in a more conservative control action (delivering less insulin). Increasing the value of $\stackrel{⌣}{R}$ impedes insulin attenuation/suspension. This asymmetry on the tuning parameters leads to the next definition of aggressiveness:

• An aggressive controller is prone to deliver more insulin. It would act fast to counter hyperglycemic excursions, and it would be slow to suspend insulin infusion in presence of hypoglycemia. An aggressive configuration for the zMPC is selected for the rest of the article as $\hat{R}$=800, $\stackrel{⌣}{R}$=500.

• A conservative controller delivers less insulin overall. It shuts down pump infusion fast to minimize hypoglycemia, but it reacts slow to hyperglycemic excursions. The selected conservative configuration is $\stackrel{⌣}{R}$=50000, $\stackrel{⌣}{R}$=10

Note that the definition of aggressiveness used here differs from its traditional meaning. An aggressive controller is usually understood to be a very responsive controller that would act fast against hypo or hyperglycemic excursions, either by injecting more insulin or by shutting down insulin suspension fast. An aggressive controller here would act fast only to counter hyperglycemic excursions, and it is a controller that is more likely to give more insulin than a conservative one, to cope with both hypo and hyperglycemia. Also note that the ratio between $\hat{R}$ and $\stackrel{⌣}{R}$ provides a measurement of the asymmetry of the controller, but this ratio is not enough to characterize the aggressiveness of the controller. The magnitude of $\hat{R}$ and $\stackrel{⌣}{R}$ separately penalize the input costs (as shown in equation 1) relatively to the weight of the output error, which is fixed.

The 2 controllers selected differ from the default settings ($\hat{R}$ =7000, $\stackrel{⌣}{R}$=100) of the controller that were used in previous studies.^8,16 Note that the “aggressive” and “conservative” configurations used in this study are evaluated as the extreme limits of operation for the zMPC, and they may be unsafe in clinical practice.

Trust Index

The method proposed in this article to quantify the trust of the model prediction makes use of the available information provided by the residuals of the predictions of that same model. Each of the $N_p$ residuals $(re{s}_k^t)$ is stored in a pool of the most recent $N_f$ residuals, resulting in a matrix $N_p\times N_f.$ The residual at time $t$ and for a $k$ steps ahead prediction is defined as:

$$re{s}_k^t=y_{t|t-k}-G_t\forall k\in {\mathbb{Z}}_1^{N_p}$$

where $y_{t|t-k}$ is the predicted glucose at the current sample $t$ that was predicted $k$ samples in the past. $G_t$ is the current CGM. The residuals are penalized using the forgetting function:

$${\widetilde{res}}_{k,i}^t=re{s}_k^{t-i}\cdot \frac{\log \left(N_f-i+1\right)}{\log N_f}\forall i\in {\mathbb{Z}}_1^{N_f},\forall k\in {\mathbb{Z}}_1^{N_p}$$

where $t$ stands for the current time, $re{s}_k^{t-i}$ are the progressively more distant residuals, and ${\widetilde{res}}_{k,i}^t$ is its analog penalized residual. Each ${\widetilde{res}}_{k,i}^t$ is an element of the matrix of penalized residuals $(N_p\times N_f)$ where each row ${\widetilde{res}}_k^t$ comprises an empirical sample of the probabilistic distribution of errors of the prediction model at current time $t$ and predicted$k$ samples before. Note that for each time stamp $t$ the residuals are penalized differently, thus the ${\widetilde{res}}_{k,i}^t$ matrix is time varying. More recent residuals are considered equally meaningful in the pool of residuals, and the relevance of older residuals decays fast when approaching $N_f,$ where $N_f=60.$ $N_f$ was chosen so that the pool of residuals is large enough to extract significant statistical metrics, and so that predictions older than 5 hours are forgotten. Both the limit and the forgetting function are choices of the designer, and can be altered if required for testing other algorithms. The normalized decay function is depicted in Figure 1.

Figure 1.

Logarithmic forgetting function applied to the residuals.

A good estimation of the confidence interval of the current prediction error can be calculated from the 5th and 95th percentiles of this sample:

$${\hat{B}}_k=\max \left(percentil{e}_{95}\left({\widetilde{res}}_k^t\right),0\right)\forall k\in {\mathbb{Z}}_1^{N_p}$$

$${\stackrel{⌣}{B}}_k=\min \left(percentil{e}_5\left({\widetilde{res}}_k^t\right),0\right)\forall k\in {\mathbb{Z}}_1^{N_p}$$

where ${\hat{B}}_k$ and ${\stackrel{⌣}{B}}_k$ are the empirically defined upper and lower boundaries of the confidence interval of the $k$-steps-ahead prediction error, and $percentil{e}_x(\cdot )$ is the $x$-th percentile function. Note that both boundaries are dependent on the residual pool at the current time $t$, so they are time-varying values. The subindex $k$ is dropped from now on, and only the first row of the residuals pool is used for the calculation of the trust index. Similarly, only the 1 step-ahead predictions are considered relevant in the rest of the article.

A scalar value that comprises both the uncertainty of previous model predictions and the coherence of the new measurement with that uncertainty is computed next:

$$T_t:=\hat{B}-\stackrel{⌣}{B}+d_H\left(CG{M}_t,\left[\hat{B}+y_t,\stackrel{⌣}{B}+y_t\right]\right)$$

$$d_H\left(x,\left[a,b\right]\right):=\{\begin{array}{l}0ifx\in \left[a,b\right]\\ x-bifx>b\\ a-xifx<a\end{array}$$

where $T_t$ is the uncertainty scalar value, $d_H\left(x,\left[a,b\right]\right)$stands for the Hausdorff distance from a point $x$ to an interval $\left[a,b\right]$, and $y_t$ stands for the model’s prediction of the current CGM sample. $T_t$ is low whenever the confidence interval is small, and it grows whenever the boundaries grow, or if the current experimental measurement falls out of the bounds. The boundaries are not fixed either, and eventually grow whenever enough measurements fall out of bounds, permanently increasing the trust index (until the out-of-bounds measurements are forgotten due to equation 5). A very high confidence (100% trust) is defined by design at$T_{t}5$ mg/dL and the lowest possible confidence correspond to an index $T_t\ge 30$ mg/dL, which is equivalent to a 0% trust. These thresholds are chosen empirically and can be tuned for different prediction horizons and controllers.

The trust index is here implemented into the closed-loop scheme presented in Gondhalekar et al¹⁶ by relating it to the aggressiveness tuning of the controller. Every possible scenario of the correlation between the trust index and the controller aggressiveness is explained in Table 1.

Table 1.

Controller Behavior for Every Glucose Scenario and Each Trust Index Value.

	High trust index(More insulin delivered)	Low trust index(Less insulin delivered)
Hyperglycemia	Controller delivers insulin freely and aggressively	Controller refrains on acting aggressively, but still delivers insulin
Hypoglycemia	Controller reduces insulin delivery; controller may deliver basal insulin if excursions are small and predicted glucose is rising	Controller shuts off the pump; may not deliver insulin even if model predictions indicate rising glucose levels

For a $T_{t}5$ mg/dL (high trust on the predictions) the aggressive tuning is selected, that is, $\hat{R}$=800, $\stackrel{⌣}{R}$ =500. On the other hand, when the index surpasses the upper limit $T_t\ge 30$ mg/dL, the conservative tuning is used: $\hat{R}$=50000, $\stackrel{⌣}{R}$=10. Anywhere in between those thresholds both and $\hat{R}$ and $\stackrel{⌣}{R}$ are linear functions of $T_t$ for every time stamp $t,$ that is, the controller gains are updated in real time as $T_t$ is updated. This dependence of the tuning parameters on $T_t,$ which ultimately makes them dependent on time, is what defines the aggressiveness tuning as adaptive. A graphical representation of the adaptation of both parameters is shown in Figure 2.

Figure 2.

Adaptation scheme for parameters $\hat{R}$ (top) and $\stackrel{⌣}{R}$ (bottom). For trust values close to 100% (almost perfect predictions) the parameters assume the aggressive configuration ($\hat{R}$ =800, $\stackrel{⌣}{R}$ =500). At a low trust, the parameters are set to the conservative mode ($\hat{R}$=50000, $\stackrel{⌣}{R}$=10). Note the different scales for both subplots.

Results

Performance metrics for the experiments described in the methods section are reported in Table 2. Metrics are reported as mean plus-minus standard deviation of the in silico population. The performance metrics for the “default,” “aggressive,” and “conservative” tuning are statistically compared to the proposed adaptive algorithm.

Table 2.

Mean and standard deviation of the control performance metrics of the zMPC controller for the Default, Aggressive, Conservative, and Adaptive tuning.

Tuning	Adaptive	Default	Aggressive	Conservative
Bolus strategy
Average glucose [mg/dL (p value)]	134 ± 6	134 ± 6 (.2)	131 ± 6 (<.005)	137 ± 7 (<.005)
Total insulin [IU (p value)]	24.9 ± 5.6	24.8 ± 5.5 (.33)	25.5 ± 5.8 (.0057)	24.4 ± 5.5 (.03)
Time in target range 70-180 [% (p value)]	91 ± 5.6	90.7 ± 5.9 (.16)	92.3 ± 6.1 (<.005)	89.8 ± 5.8 (<.005)
Time in hypoglycemia <70 [% (p value)]	0 ± 0*	0.01 ± 0.16 (<.005)	0.1 ± 0.57 (<.005)	0 ± 0 (1)
Time in hyperglycemia >300 [% (p value)]	0.01 ± 0.13	0.01 ± 0.22 (.25)	0.01 ± 0.21 (.22)	0 ± 0 (.08)
GRID strategy
Average glucose [mg/dL (p value)]	149 ± 10*	147 ± 8 (<.005)	143 ± 7 (<.005)	154 ± 13 (<.005)
Total insulin [IU (p value)]	23.4 ± 5.2*	23.8 ± 5.1 (.03)	24.5 ± 5.8 (<.005)	22.8 ± 5.1 (.01)
Time in target range 70-180 [% (p value)]	74.6 ± 6.9	75 ± 7.1 (.12)	77 ± 7.2 (<.005)	71.2 ± 9.4 (<.005)
Time in hypoglycemia <70 [% (p value)]	0.16 ± 0.84*	0.67 ± 1.82 (<.005)	1.04 ± 2.43 (<.005)	0.15 ± 1 (.44)
Time in hyperglycemia >300 [% (p value)]	1.07 ± 2.84	1.02 ± 2.31 (.34)	0.7 ± 1.62 (<.005)	1.37 ± 3.5 (.02)

p values are calculated to show significance between the proposed (adaptive) and other 3 controllers. Values with an asterisk show statistical significance (p < .05) between the proposed tuning and the default column.

In the bolus strategy, the adaptive controller shows very similar performance (in mean glucose and time in range) to that of the “default” controller. However, the adaptive strategy is able to reduce significantly the time spent in hypoglycemia (0.01% vs 0.0%, p < .005). Hypoglycemia is also rarely seen when using the conservative controller, but at the cost of a higher average glucose (137 mg/dL vs 134 mg/dL, p < .005).

In the GRID scenario, the adaptive algorithm shows significantly lower average glucose than the conservative tuning (149 mg/dL vs 154 mg/dL, p < .005). Furthermore, the proposed algorithm shows a significant reduction in hypoglycemia with respect to the aggressive case (0.16% vs 1.04%, p < .005), with minimal increase in average glucose and good time in the safe glucose range. The time in hypoglycemia is not different for the conservative case and the proposed adaptive algorithm. Time in hypoglycemia for the proposed adaptive algorithm is significantly lower than for the default controller (0.16% vs 0.67%, p < .005), but in this case at the cost of a higher average glucose. However, the time in range for the proposed algorithm is not significantly different than that of the default tuning.

In Figure 3 the glucose concentration, insulin infusion, and trust index for the adaptive scenario and default controller are plotted. The average glucose is marginally lower for the default controller, and the interquartile range is larger, especially after the larger meal. The way GRID is designed,³⁰ an insulin bolus (considering insulin-to-CHO ratio, recent insulin delivered, and IOB constraints) is triggered when a meal is detected, which explains the relatively small and delayed insulin impulse after meals. Significant drops in the trust index can be observed for time of the meals and glucose peak time, corresponding to changes in the glucose trend that yield larger prediction errors.

Figure 3.

Median glucose concentration and interquartile range (top) and mean insulin pump administration (middle). The default static tuning is plotted in red (continuous line), the proposed adaptive tuning in blue (discontinuous line). The displayed glucose and insulin profiles correspond to the GRID compensated meals. The bottom plot displays the trend of the Median Trust Index against time for the adaptive version of the controller.

Discussion

A detailed comparison of the proposed adaptive strategy with the default controller is necessary to appreciate the benefits of our novel approach. Looking at the bolus strategy in Table 2, the adaptive strategy and the default tuning show very similar performance, both in average glucose (134 mg/dL) and time in range (24.9% and 24.8%). However, the adaptive controller is able to avoid hypoglycemia more efficiently, reducing the time below 70 mg/dL to virtually zero, while the default controller still has spurious episodes of hypoglycemia. This low occurrence of hypoglycemia is also observed for the conservative version of the controller, but the performance of that tuning is subpar when compared with the other tested controllers. The aggressive tuning of the controller is able to achieve significantly better performance than any of the other controllers, with an average glucose of 131 mg/dL and a time in range of 92.3%. However, the number of hypoglycemic episodes triggered by that controller is also higher than those of the adaptive simulation.

When comparing strategies GRID and Bolus from Table 2, a larger disparity can be observed for the results of each controller configuration for the GRID strategy. This is a direct result of the prandial bolus dosing limiting the amount of insulin that the controller is able to utilize in the postprandial period, which is one of the safety mechanisms implemented in the controller structure. This yields, for the bolus strategy, more homogeneous results than those of the GRID strategy.

As for the GRID strategy results, the adaptive controller produces significantly larger average glucose than the default controller configuration (149 mg/dL vs 147 mg/dL). However, this difference is clinically irrelevant, with an estimated A1C (based on the average glucose) of 6.81% for the adaptive control and 6.74% for the default controller configuration. The “A1C-average glucose” regression was calculated following:³³ $A1C=\left(avg\_glucose+46.7\right)/\mathrm{28.7.}$The performance difference is not significant when looking at the time in range (74.6% vs 75% p = .12). Furthermore, the adaptive scheme is able to significantly reduce the time spent in hypoglycemia, down to 0.16% from the 0.67% of the default controller configuration. The time spent in hypoglycemia for the adaptive scheme is not significantly different than that achieved by the conservative controller configuration. Similarly to what was observed for the bolus strategy experiment, the conservative controller produces a higher average glucose than that of the adaptive control, while also showing a significantly lower time in range. The aggressive controller is able to lower the average glucose more than any of the other configurations, at the cost of much higher frequency of hypoglycemia (1%).

Due to the forgetting function introduced in equation 5, the algorithm gradually converges to a particular value if the prediction error remains bounded. Since the prediction errors are bounded by the nature of the output variable (CGM values and its rate of change are bounded), we conclude that the computation of the trust index is stable and convergent.

zMPC uses a control-relevant model that is different from the validation model. Given that there is an evident model-plant mismatch, a practical approach was used to address stability of the closed-loop system, as it has historically been done for MPC applications.³⁴ We performed 2000 simulations under a variety of scenarios, clinical conditions and realizations of additive noise, to verify the stability of the algorithm and the closed-loop system. Our simulations resulted in stable MPC treatments every time, reassuring experimental stability of the algorithm. Furthermore, the controller response is bounded between 2 stable $\hat{R}$ and $\stackrel{⌣}{R}$ operation points. Finally, the MPC problem has no hard constraints on the output (CGM) or states, thus it always has feasible solutions.

For both scenarios tested, the proposed scheme achieves similar or better control than the other configurations by good use of already available information and without any additional risk to the patient’s health.

Even though the results seem promising, the performance of the proposed control benefits when put in context with existing clinical experiments from literature. In Table 3 some of the results of this article are displayed along similar results from literature, using the same controller implementation as described in this article for simulated and real patients.

Table 3.

Performance comparison for the announced meals scenario between the controllers tested in this study and previous studies present in the literature using the same controller scheme.

	Percentage of time in range 70-180 mg/dL	Percentage of time <70 mg/dL
Adaptive (bolus strategy)	91 ± 5.6	<0.01
Default (bolus strategy)	90.7 ± 5.9	0.01 ± 0.2
Aggressive (bolus strategy)	92.3 ± 6.1	0.1 ± 0.6
Conservative (bolus strategy)	89.8 ± 5.8	<0.01
Harvey et al30 in silico (announced meals)	89 [72-96]	0 [0-2]
Harvey et al3 inpatient (unannounced meals)	71.2 ± 15.67	1.8 ± 1.4
Dassau et al8 outpatient (announced meals)	79.1 [60.4-86.1]	1.37 [0-2.84]

Metrics presented in average ± standard deviation or median [min-max], as per the original source.

Rows 1 to 5 of Table 3 correspond to in silico experiments, while the last 2 rows are clinical experiments, both in clinic (row 6) and outpatient (row 7). Rows 2 and 5 are the same experiment, with 1 difference: in this article we have simulated each patient 100 times. We can observe that the results are very similar to those achieved in this article.

Performance results for virtual patients are better than those seen in clinical studies, in particular when looking at frequency of hypoglycemia, which is extremely low in simulation. However, hypoglycemia occurrence is still a very important concern when implementing controllers in clinical practice, which justifies our focus on hypoglycemia reduction.

Conclusions

A novel strategy for adaptive tuning of a zone based MPC controller has been presented and its application to a virtual cohort of people with type 1 diabetes. Repeated residuals from the model predictions are stored and a scalar trust index is deduced quantifying the uncertainty of the model predictions and the coherence of the glucose sample at each step. The MPC controller is then tuned in real time based on the value of the trust index. The resulting adaptive controller was tested against several static counterparts producing safer control in hypoglycemia.