Enhanced Model Predictive Control (eMPC) Strategy for Automated Glucose Control

Abstract

Development of an effective artificial pancreas (AP) controller to deliver insulin autonomously to people with type 1 diabetes mellitus is a difficult task. In this paper, three enhancements to a clinically validated AP model predictive controller (MPC) are proposed that address major challenges facing automated blood glucose control, and are then evaluated by both in silico tests and clinical trials. First, the core model of insulin-blood glucose dynamics utilized in the MPC is expanded with a medically inspired personalization scheme to improve controller responses in the face of inter- and intra-individual variations in insulin sensitivity. Next, the asymmetric nature of the short-term consequences of hypoglycemia versus hyperglycemia is incorporated in an asymmetric weighting of the MPC cost function. Finally, an enhanced dynamic insulin-on-board algorithm is proposed to minimize the likelihood of controller-induced hypoglycemia following a rapid rise of blood glucose due to rescue carbohydrate load with accompanying insulin suspension. Each advancement is evaluated separately and in unison through in silico trials based on a new clinical protocol, which incorporates induced hyper- and hypoglycemia to test robustness. The advancements are also evaluated in an advisory mode (simulated) testing of clinical data. The combination of the three proposed advancements show statistically significantly improved performance over the nonpersonalized controller without any enhancements across all metrics, displaying increased time in the 70–180 mg/dL safe glycemic range (76.9 versus 68.8%) and the 80–140 mg/dL euglycemic range (48.1 versus 44.5%), without a statistically significant increase in instances of hypoglycemia. The proposed advancements provide safe control action for AP applications, personalizing and improving controller performance without the need for extensive model identification processes.

Graphical abstract

1. Introduction

Type 1 diabetes mellitus (T1DM) is an autoimmune disease characterized by a lack of endogenous insulin production. Insulin plays a critical role in the regulation of blood glucose (BG), by promoting glucose uptake and therefore reducing the BG concentration. There are over 18 million people around the world currently suffering from T1DM, with over 15,000 new cases of T1DM among children and adolescents in the US alone, each year.^1–2 People with T1DM require exogenous delivery of insulin to prevent long term complications of high BG concentrations (hyperglycemia). Unfortunately, even a slight overdose of insulin may lead to low BG concentrations (hypoglycemia) with life threatening consequence. If left untreated, severe hypoglycemia may result in death in a matter of hours or minutes.

To compensate for these potential difficulties, people with T1DM typically manage their BG using either multiple daily insulin injections or continuous exogenous insulin delivery devices (e.g., continuous subcutaneous insulin infusion (CSII) pumps) paired with infrequent capillary reference measurements using small blood draws by pricking the fingertips. Some people augment these measurements with a continuous glucose monitor (CGM), which senses subcutaneous glucose concentrations. ³ The limitations of clinical practice in BG management mean that hypo- and hyperglycemic events are a routine part of the daily challenges in the lives of people with T1DM from the onset of the disease.^4–5 Such exposure to routine hypo- and hyperglycemic events have drastic short and long term effects on the health of people with T1DM, and the life expectancy of people with T1DM is at least a decade shorter than that of the overall population.⁶

Automation of BG management through the application of advanced control algorithms in an artificial pancreas (AP) system has been shown to be a clear way forward to improve the lives of people with T1DM, and reduce hypo- and hyperglycemia. By utilizing a control system to automate insulin delivery via a CSII pump based on feedback from a CGM, an AP system can significantly improve the quality of life of people with T1DM.⁷ Multiple clinical studies have previously shown significant improvements in BG control and a reduction in the number of hypo- and hyperglycemic events for people under closed-loop control in comparison to open-loop (manual control).^{5, 7–15} A web archive of recent published clinical trials of AP is available in a database created by the authors’ research group: (http://thedoylegroup.org/apdatabase/). The use of AP devices is considered by many to be a paradigm-shifting advance in the management of BG in people with T1DM.

However, despite a history dating back to the 1970’s,¹⁶ commercial applications of AP have still not been realized. Although a variety of control strategies, such as proportional-integral-derivative^17–20 and fuzzy logic controllers^{11, 13, 21}, have been proposed for AP, the authors’ group has focused on model predictive controllers (MPC)^{15, 22–25} due to their flexibility and many other algorithmic benefits. This paper addresses three major challenges faced by all AP controllers:

1. Inter- and intra-individual variations in insulin sensitivity

2. Asymmetry in the effects of hyper- and hypoglycemia

3. Vulnerability to rebound hypoglycemia after preventative CHO ingestion following prolonged pump suspension

Specific innovations are presented for each challenge that individually, but especially when combined, demonstrate significant improvements in performance compared to prior work. Each innovation is evaluated both separately and in unison, both in silico using the Universities of Virginia/Padova (UVA/Padova) metabolic simulator,²⁶ and also using advisory mode conditions with data from clinical trials of closed-loop AP.

2. Experimental Methods

2.1 Medically inspired personalization of the underlying model in the MPC for AP

One of the greatest difficulties in the widespread adoption of an AP system is the challenge that faces all biomedical applications of technology: the wide spectrum of characteristics of the human population.^27–29 In particular, the BG response to insulin delivery varies greatly based on each individual’s demographic profile and lifestyle, and may also undergo significant changes in a matter of hours, based on the time of day and state of health.³⁰ This is an extremely important factor in designing an AP controller, as even a mere 20% variation in insulin sensitivity between one person and the next may lead to significant changes in insulin response for the same quantity of insulin, and thus the difference between euglycemia (normal BG concentrations) and dangerous hypoglycemia.

The wide variations in insulin response of people who can benefit from AP devices mean that reliance on a fixed model for the development of MPC control algorithms for AP applications may lead to significant inaccuracy in insulin delivery. The consequences range from unsatisfactory control to safety concerns. One way to compensate for this model-patient mismatch is to perform individual model identification for every person with T1DM. However, not only is conducting such an identification for each person with T1DM not feasible, the slow response of the system, and potential for hourly variations mean that there is little confidence that data gathered through such an identification is a sufficient representation. Another method to compensate for this mismatch is to perform on-line tuning of the system, but the relatively long timescales within the system mean that an effective on-line tuning must be considered over a longer timescale than the hourly changes. As a result, the first focus of this work is to compensate for the interpersonal variations through the personalization of a population average model of insulin-BG dynamics, without any individual model identification.

Although excitation of a system is useful for identifying accurate models, the collection of rich BG data-sets, i.e., data that span the extremes of BG values, is not desirable due to the risks of hyper- and hypoglycemia. Thus, a fixed, control-relevant model of insulin-BG dynamics³¹ was initially derived based on frequency response data collected from the UVA/Padova simulator.²⁶ This simulator has been accepted by the US Food and Drug Administration (FDA) in lieu of animal trials prior to clinical studies. It provides 100 in silico subjects whose distribution of clinical parameters related to insulin response intentionally reflects a range wider than the distribution found in the general T1DM human population. Recent literature reports close reproduction of insulin and BG traces in an actual clinical trial as validation of the simulator’s performance.³² The utilized model is the 3rd order discrete-time linear time-invariant transfer function with a sampling time of 5 minutes,³¹

$${\mathrm{M}}_{\mathrm{r}}({\mathrm{z}}^{-1})=\frac{\mathrm{G}({\mathrm{z}}^{-1})}{\mathrm{I}({\mathrm{z}}^{-1})}=\frac{{\mathrm{K}}_{\mathrm{f}}{\mathrm{z}}^{-3}}{(1-p_1{\mathrm{z}}^{-1}){(1-p_2{\mathrm{z}}^{-1})}^2}$$ (1)

where z⁻¹ is the discrete-time, backward-shift operator, I is insulin delivery (pmol/min), G is BG concentration (mg/dL), K_f =2.005 × 10⁻⁴ ([mg/dL]/[pmol/min]), and p₁=0.98, p₂=0.965.

In van Heusden et al.³¹, K_f is modified by a correction factor based on each subject’s total daily insulin (TDI), a clinical parameter that is commonly available for people with T1DM, then multiplied by a “safety factor” that ranges from 1.25 to 3. While utilizing TDI is a good starting point to personalize the controller, it is limited because the glucose management of T1DM subjects is generally divided into two categories – basal compensation, and meal compensation. Basal compensation accounts for each person’s insulin needs separate from meals, and represents the background insulin requirement for glucose control during fasting periods. On the other hand, meal compensation accounts for the additional quantity of insulin beyond basal delivery that is needed to account for the carbohydrate content of meals. Generally, the basal delivery accounts for between 40 to 60% of TDI; however, this proportion can vary significantly based on meal composition and lifestyle.³ As a result, relying solely on TDI results in vulnerability to cases where it is based on a day with unusually large meals, and thus a larger TDI would result in a more aggressive response than is appropriate. Moreover, the determination of the “Safety factor” is based on a decision tree that changes according to the expected uncertainties of the subject parameters, and does not take into account the subject’s diurnal insulin sensitivity as reflected by their basal profile, another commonly available clinical parameter. Consequently, this sliding scale for conservative control results in a significantly conservative response under most scenarios.

In this paper, we expand and enhance the core model in a clinically inspired manner by replacing the Safety Factor with a new parameter that is based on the subject’s basal insulin profile. This approach allows the controller to be adjusted for the mismatch between TDI and basal profiles, and tailors its aggressiveness to individual subjects. The proposed approach takes into account both the TDI and the basal profile, allowing the controller to be designed with each person’s individual BG management experience; thus it acts more aggressively in case of subjects with low insulin sensitivity and reduces insulin delivery when subjects are highly insulin sensitive. Moreover, incorporation of the basal parameter also allows the controller to compensate for diurnal variations in insulin sensitivity accordingly. Specifically, the model gain can be both personalized and made diurnally time-dependent using a priori clinical parameters as

$$K(t)=K_{i}c{S}_b(t)$$ (2)

where K_i (function of TDI) and c (conversion factor) are specified in an identical manner to van Heusden et al.³¹ K_i can also be specified by the subjects themselves through the correction factor clinical parameter, rather than by heuristics based on TDI. The first novel contribution in this paper is an individualized scaling factor that is based on the person’s basal profile. This factor S_b(t) is calculated as

$$S_b(t)=\frac{b_{\mathit{calc}}}{b(t)},$$ (3)

where b_calc is a subject’s recommended basal profile based on his or her TDI at the start of his or her diabetes care. It is calculated as

$$b_{\mathit{calc}}(t)=\frac{x·\mathit{TDI}}{24h}$$ (4)

where x ranges from 0.4 to 0.6, based on the person’s estimated activity level.³ This was considered a tuning parameter, and was fixed at 0.4 after evaluating 10 subjects in silico for best performance (minimize hypoglycemia and maximize time in euglycemia under controllers designed based on this model). The details of this determination can be found in Lee et al.³³ This formulation allows the incorporation of the subject’s actual basal profile b(t). This basal profile plays the most important role in that person’s daily BG control in the absence of meal disturbances. Consequently, b_calc may change the model’s glucose response based on a measure of sensitivity to insulin that is greater, or less, than the standard.

2.2 Asymmetric, exponential penalties to glucose excursions

In addition to the inter- and intrapersonal variations in insulin sensitivity, any AP control algorithm must also take into account the unique, asymmetric nature of the human insulin-BG system³⁴. Specifically, while the consequences of hyperglycemia can be described as more chronic, long-term, and occurring over a wide range of BG, the consequences of hypoglycemia are acute, deadly, and occur within a much smaller range of BG.³ Under the standard formulation of a cost function in MPC, an equal penalty is applied for deviations above and below the target. Instead, we propose an exponential-quadratic shaping of the MPC cost function that penalizes deviations above and below the target asymmetrically. The proposed cost function applies an exponential cost to excursions below the glucose target and a quadratic penalty to the excursions above the target, in a medically inspired manner. This modification resulted in significant improvements in the attenuation of insulin delivery during impending and actual hypoglycemia, while still nimbly adjusting for hyperglycemia (particularly after meal disturbances), which are the two primary objectives for BG control.

In a previous clinical trial, an MPC controller was implemented with the personalized model as detailed in equations (1)–(4) through the standard MPC formulation.¹⁵ Specifically, an MPC algorithm allows for explicit implementation of a discrete model of arbitrary order. Reformulating the poles of the personalized model in equation (1) as

$$b_1=p_1+2p_2=2.91$$ (5)

$$b_2=-2p_1{p}_2-p_2^2=-2.82,$$ (6)

$$b_3=p_1{p}_2^2=0.91,$$ (7)

the LTI model in equation (1) is implemented under the standard MPC formulation, in state space form, as

$$x_{k+1}=A{x}_k+B{I}_k$$ (8)

$$G_k=C{x}_k.$$ (9)

The parameter matrices for the non-personalized controller are formulated as

$$A=\left[\begin{array}{ccc}{b}_1& b_2& b_3\\ 1& 0& 0\\ 0& 1& 0\end{array}\right]$$ (10)

$$B_r=\left[\begin{array}{c}{K}_i\\ 0\\ 0\end{array}\right]$$ (11)

$$C=[\begin{array}{ccc}0& 0& 1\end{array}].$$ (12)

A state estimator (Luenberger observer³⁵) is implemented as

$${\widehat{x}}_{k+1}=A{\widehat{x}}_k+B{I}_k^{\prime }-L({G^{\prime }}_k-{{\widehat{G}}^{\prime }}_k)$$ (13)

$${{\widehat{G}}^{\prime }}_k=C{\widehat{x}}_k$$ (14)

which is a linear time-invariant system with x̂_k representing estimated states of x_k and Ĝ′_k representing the estimated BG G_k. The following equations define the gain L

$$L=K^T$$ (15)

$$K=-{({\mathit{CPC}}^T+\widehat{R})}^{-1}{\mathit{CPA}}^T,$$ (16)

and P satisfies the discrete algebraic Riccati equation

$$P={\mathit{APA}}^T+\widehat{Q}-{\mathit{APC}}^T{({\mathit{CPC}}^T+\widehat{R})}^{-1}{\mathit{CPA}}^T$$ (17)

where Q̂ and R̂ are positive definite design parameters, set as 1 and 1000, respectively. The cost function of this personalized MPC (pMPC) was implemented as

$${\mathrm{J}}^{\text{pMPC}}\left({\{{\mathrm{I}}^{\prime }(\mathrm{k}+\mathrm{j})\}}_{\mathrm{j}=0}^{\mathrm{N}-1}\right)=\sum _{\mathrm{j}=1}^{\mathrm{P}}{\Vert {\mathrm{G}}^{\mathrm{e}}(\mathrm{k}+\mathrm{j})\Vert }^2+\mathrm{R}\sum _{\mathrm{j}=0}^{\mathrm{N}-1}{\Vert {\mathrm{I}}^{\prime }(\mathrm{k}+\mathrm{j})\Vert }^2$$ (18)

where G^e(k+j), denoted glucose excursions above and below the target setpoint G_SP=110 as

$${\mathrm{G}}^e(\mathrm{k}+\mathrm{j})={\mathrm{G}}^{\prime }(\mathrm{k}+\mathrm{j})-{\mathrm{G}}_{\text{SP}}.$$ (19)

Here, R=11,000 is a design parameter that weighs the insulin inputs I′(k+j) above or below basal, G^e(k+j) represents the excursions of the predicted glucose G′(k+j) above or below the setpoint, k and j respectively represent the actual time step and the prediction step. N=5 and P=12 represent the control and prediction horizons. Larger values of R indicate a greater penalty on the corresponding inputs, and thus smaller input deviations away from basal. Note that while the glucose and insulin excursions are scalar values, a norm notation is used within equation (18) and similar equations in this paper to maintain the equations in a general framework for future applications where vector operations may be more appropriate (such as algorithm design for dual hormone controllers).

Although this cost function for the pMPC showed good response to glucose excursions above 110 mg/dL in clinical trials, it was gained at the cost of potentially increased hypoglycemic risk. One of the reasons for this tradeoff was that the glucose excursions were penalized by a quadratic function in both directions, such that only a scalar multiplier determined the differences between the risk of hypo- and hyperglycemia.

A zone-objective MPC (ZMPC) was also implemented with the initial model as in equation (1) in a different clinical trial.¹⁴ The cost function was

$$\begin{array}{c}{\mathrm{J}}^{\text{ZMPC}}\left({\{{\mathrm{I}}^{\prime }(\mathrm{k}+\mathrm{j})\}}_{\mathrm{j}=0}^{\mathrm{N}-1}\right)=\sum _{\mathrm{j}=1}^{\mathrm{P}}{\Vert {\mathrm{G}}^{\text{zone}}(\mathrm{k}+\mathrm{j})\Vert }^2+{\mathrm{R}}_{+}\sum _{\mathrm{j}=0}^{\mathrm{N}-1}{\Vert {\mathrm{I}}_{+}^{\prime }(\mathrm{k}+\mathrm{j})\Vert }^2\\ +{\mathrm{R}}_{-}\sum _{\mathrm{j}=0}^{\mathrm{N}-1}{\Vert {\mathrm{I}}_{-}^{\prime }(\mathrm{k}+\mathrm{j})\Vert }^2,\end{array}$$ (20)

where R₊=7,000 and R₋=100 were design parameters that weighted the insulin inputs above or below basal, and G^zone corresponds to the glucose excursions beyond the target zone, defined as

$${\mathrm{G}}^{\text{zone}}(\mathrm{k}+\mathrm{j})=\{\begin{array}{cc}{\mathrm{G}}^{\prime }(\mathrm{k}+\mathrm{j})-140& \text{if}{\mathrm{G}}^{\prime }(\mathrm{k}+\mathrm{j})>140\\ 80-{\mathrm{G}}^{\prime }(\mathrm{k}+\mathrm{j})& \text{if}{\mathrm{G}}^{\prime }(\mathrm{k}+\mathrm{j})<80\\ 0& \text{otherwise}\end{array}.$$ (21)

This cost function for the ZMPC also showed good overall closed-loop performance. However, the existence of the quiescent zone in glucose control meant that there were concerns regarding the sluggish response to hyperglycemic excursions.

In the proposed approach, a fundamentally different scaling is applied to the glucose values above versus below the targets, in order to obtain an enhanced glucose control. The cost function is reshaped to apply an exponential penalty on glucose excursions below the setpoint, while maintaining a quadratic penalty on glucose excursions above the setpoint. The resulting enhanced MPC (eMPC) cost function J′ for which the MPC is optimized is defined as

$$\begin{array}{c}{\mathrm{J}}^{\prime }\left({\{{{\mathrm{I}}^{\prime }}^{(\mathrm{k}+\mathrm{j})}\}}_{\mathrm{j}=0}^{\mathrm{N}-1}\right)=\sum _{\mathrm{j}=1}^{\mathrm{P}}{\Vert {\mathrm{G}}_{+}^{\mathrm{e}}(\mathrm{k}+\mathrm{j})\Vert }^2+\sum _{\mathrm{j}=1}^{\mathrm{P}}exp(\mathrm{a}\Vert {\mathrm{G}}_{-}^{\mathrm{e}}(\mathrm{k}+\mathrm{j})\Vert )\\ +{\mathrm{R}}_{+}\sum _{\mathrm{j}=0}^{\mathrm{N}-1}{\Vert {\mathrm{I}}_{+}^{\prime }(\mathrm{k}+\mathrm{j})\Vert }^2+{\mathrm{R}}_{-}\sum _{\mathrm{j}=0}^{\mathrm{N}-1}{\Vert {\mathrm{I}}_{-}^{\prime }(\mathrm{k}+\mathrm{j})\Vert }^2,\end{array}$$ (22)

where R₊=7,700, R₋=2,500, and a=0.18 is a design parameter representing the exponential coefficient. In this cost function, an exponentially greater penalty is applied on glucose excursions below the target to compensate aggressively for hypoglycemia, while keeping a quadratic penalty scaling on excursions above the target to maintain a reasonable, but less aggressive, response to hyperglycemia. In addition, the exponential formulation means that glucose excursions near the target have a reasonably conservative, but nonzero, response to fluctuations around the target, while maintaining the ability to respond quickly to larger excursions. A summary of the different objective functions with each proposed cost function formulation and their reference is shown in Table 1.

Table 1.

A brief summary of the references for each cost function referenced in section 2.2.

Name	Target	Reference
MPC	Setpoint (110 mg/dL)	Dassau, et al.8
Zone-MPC (ZMPC)	Range (80–140 mg/dL)	Grosman, et al.36
personalized MPC (pMPC)	Setpoint (110 mg/dL); personalized gain	Lee, et al.33
enhanced MPC (eMPC)	Setpoint (110 mg/dL); exponential penalty	Lee, et al.37

Figures 1a and 1b³⁷ show how the scaled costs of the MPC cost functions vary with glucose concentration. The scaled low blood glucose and high blood glucose indices (LBGI/HBGI),³⁸ a widely used clinical metric to assess BG control performance is also included. As is shown in Figure 1, the asymmetry introduced by the proposed modification combines the advantages of both the pMPC and ZMPC cost function formulations, while minimizing their disadvantages. Additional details about this reshaping of the cost function can be found in Lee et al.³⁷

Figure 1.

The scaled costs of excursions above or below the target are shown for several MPC cost functions, and the LBGI/HBGI scale. The green zones represent the 80–140 mg/dL euglycemic zone, and the yellow zones represent the 70–180 mg/dL safe glycemic zone. (A) Penalties for BG between 0 to 250 mg/dL. (B) 10x zoomed view of the penalties near 110 mg/dL.³⁷

2.3 Enhanced insulin-on-board constraints for robust performance

In addition to the need for personalized compensation of insulin sensitivity and asymmetric penalties to excursions beyond the target, AP systems are also subject to the overarching need for safety in a medical device. Unlike many electrical or mechanical systems, there is a significant time delay after insulin delivery until the peak insulin action time, and the effects of each control action may remain for up to eight hours after the initial delivery.³⁹ As a result, a variety of additional constraints on controller action have been developed and tested under clinical conditions, to reduce the risk of overdelivery of insulin. One such widely used augmentation is called insulin-on-board (IOB), which explicitly takes into account the previous insulin delivery remaining in the system. The basis for these constraints was developed by Ellingen et al.,³⁹ and utilized to estimate the percentage of prior insulin delivery that remains unutilized in the system, represented as 2, 3, …, 8 hour decay curves that range from 100% to 0%. These IOB values were defined as

$${\mathit{IOB}}_k={\mathit{IOB}}_{m,k}+{\mathit{IOB}}_{o,k},$$ (23)

and incorporated as one of the constraints for which the MPC solution was calculated while minimizing the cost function as defined in the previous section. The IOB_k value represents the current IOB constraint at the k^th sampling instant and is divided into IOB_o and IOB_m representing IOB constraints due to insulin delivery in compensation for meals and insulin delivery for other glucose variations.

The IOB curves are represented as 96 (8h/5 min=96) element row vectors in the current implementation, expressed as D_μ with μ= 2, 4, 6, or 8, to describe the remaining insulin that is active in the eight hours following delivery. The IOB value can be expressed as

$${\mathit{IOB}}_{m,k}=D_2{I}_{m,k}$$ (24)

$${\mathit{IOB}}_{o,k}=\{\begin{array}{ll}{D}_2{I}_{o,k}\hfill & \text{if}{G}_k>300\hfill \\ D_4{I}_{o,k}\hfill & \text{if}300\ge G_k>200\hfill \\ D_6{I}_{o,k}\hfill & \text{if}200\ge G_k>140\hfill \\ D_8{I}_{o,k}\hfill & \hfill \text{otherwise}\hfill \end{array},$$ (25)

where I_m represents insulin delivered as a meal bolus, and I_o represents all other insulin delivery. An IOB constraint must be dynamic to allow insulin dosing by the controllers as needed.³⁹ Consequently, when BG is near, or below, the target range, longer decay curves, and thus a slower relaxation of the constraint, are applied to prevent excessive insulin delivery. Progressively shorter IOB decay curves, and thus a quicker relaxation of the constraint, are applied with higher BG values to allow for increased insulin delivery by the controllers. Similarly, as insulin delivery accompanying meals is associated with correction of meals rather than glucose values, meal boluses are also paired with the shortest IOB decay curve.

The IOB is then compared with the insulin that would be required (IOB_req) to bring the subject to the target setpoint G_SP (110 mg/dL), according to

$${\mathit{IOB}}_{\mathit{req},k}=\frac{{\mathit{CGM}}_k-G_{SP}}{CF}.$$ (26)

Correction factor CF represents the amount of BG (mg/dL) that would decrease with 1 U of insulin delivery. It is another clinical parameter that is widely used in the manual BG control for people with T1DM. The IOB constraint is then calculated as

$${\mathrm{I}}_{\text{IOB},\mathrm{k}}=\{\begin{array}{ll}{\text{IOB}}_{\text{req},\mathrm{k}}-{\text{IOB}}_{\mathrm{k}}+\frac{\mathrm{T}}{60}{\mathrm{b}}_{\mathrm{k}}\hfill & \text{if}{\text{IOB}}_{\text{req},\mathrm{k}}>{\text{IOB}}_{\mathrm{k}}\hfill \\ \hfill \frac{\mathrm{T}}{60}{\mathrm{b}}_{\mathrm{k}}\hfill & \text{otherwise}\hfill \end{array}$$ (27)

where T=5min represents the sampling interval, and this expression is implemented as a constraint in the MPC calculations.

Unfortunately, this implementation led to oscillatory behavior in rare situations during clinical validations due to insufficient activation of the IOB constraints. Specifically, the controller could command an extended period of pump attenuation or suspension if there was a persistent monotonic decrease in a subject’s BG concentration. Ingestion of rapid acting carbohydrates following a period of zero pump delivery in such situations can in some cases lead to rebound hyperglycemia, which is a natural consequence of human physiology due to the lack of previous insulin delivery. A controller might over-respond to this rapid increase in BG by rapidly delivering more insulin as the glucose rises, leading to potential rebound hypoglycemia.

An enhanced, dynamic IOB strategy is proposed in this paper to reduce this potential controller-driven hypoglycemia without compromising the controller’s ability to react to hyperglycemia. This strategy recognizes a monotonic decrease in BG as the primary driver of extended pump attenuation. Once triggered, the insulin delivery history for the IOB algorithm is populated with the subject’s basal delivery rates to maintain its effectiveness and to prevent the undesirable oscillatory behavior.

Pump suspension is defined as an instance of q consecutive steps of no delivery of insulin over r steps of history, u_kh:

$$u_{kh}=[\begin{array}{ccccc}{u}_{k-r}& u_{k-r+1}& u_{k-r+2}& \dots & u_{k-1}\end{array}].$$ (28)

In the proposed application, q=3 (i.e., defining qT=15 minutes of zero delivery as pump suspension) and r=12 (defining one hour as the window of insulin delivery history). This history is initialized as the basal rate in the beginning of the closed-loop trial. The enhanced IOB algorithm takes into account all such instances of pump suspension during the considered delivery history.

This proposed modification improves the existing IOB constraints for MPC and is implemented by equations (23)–(25). Specifically, the proposed algorithm modifies any series of q or more consecutive steps of zero insulin delivery in last n steps in the delivery history for the IOB calculation as

$$I_{s,k}=max(I_{o,k},I_{b,k}),$$ (29)

where I_b,k is the basal value (in U) corresponding to the k^th step in controller action, and I_o,k is the original insulin delivery. The augmented calculation is then

$${\mathit{IOB}}_{a,k}=D_{\mu }{I}_{s,k},$$ (30)

which is supplied to the MPC algorithm for the controller calculation. A flowchart that describes the operation of this modified IOB is shown in Figure 2.

Figure 2.

A flowchart of the proposed dynamic IOB algorithm to prevent rebound hypoglycemia.

3. Results

3.1. Preliminary evaluation of the personalization and exponential asymmetry through in silico clinical trials

The personalization and exponential asymmetry were initially tested in silico on 10 virtual adult subjects with T1DM using the UVA/Padova simulator. Since the in silico patients do not exhibit the oscillatory behavior after rebound hypoglycemia, a complete validation of the enhanced IOB algorithm was conducted through advisory mode simulations using clinical data.⁴⁰ Specifically, after simulated ingestion of fast absorbing carbohydrates after an extended period of pump suspension, the in silico subjects within the UVA/Padova simulator did not experience the accelerated and sustained rise in glycemia which was observed in some subjects during clinical trials. As the enhanced IOB algorithm was specifically designed to compensate for this challenge, verification was conducted through advisory mode rather than in silico trials. All other features were tested through the in silico trial protocol as described below.

A clinical protocol identical to the one utilized in human clinical trials was available, and used as the basis for the in silico tests of each controller.^14–15 This protocol begins at noon, and includes two announced dinner and breakfast meal challenges with 65 and 50 g carbohydrate (CHO) content (meal accompanied by an insulin bolus calculated according to subjects’ insulin to carbohydrate (I:C) ratio) at 7:00 PM and 7:00 AM, and one unannounced lunch meal challenge with 65g CHO content at 12:00 PM on the second day of the study. The protocol also includes an overnight period, which potentially has the greatest risk of hypoglycemia due to slower response by the subjects to physiological symptoms.

Two additional challenges were introduced to the controller to further test the controller’s ability to compensate for unexpected situations that may occur in reality. First, the simulator was modified to incorporate the effects of diurnal variations in insulin sensitivity by decreasing the insulin sensitivity of each subject by 50% from 5:00 AM to 10:00 AM to simulate the well-known dawn phenomenon. Moreover, an edge-case robustness validation was conducted by incorporating a scenario in which extreme hypoglycemia and hyperglycemia occurred successively. A 5U intravenous insulin bolus was given at 1:00PM without informing the controller. A pump occlusion was simulated for the next 2h, combined with a rescue carbohydrate treatment 1h after the unknown intravenous insulin delivery. These modifications help make the proposed scenario have a significantly closer resemblance to the possible difficulties faced in reality, and allow each advancement to be challenged for robustness and stability in a scenario similar to real life. The BG and insulin delivery profiles of each subject were recorded for 30 h. A timeline summarizing this protocol is shown in Figure 3.

Figure 3.

A timeline of the clinical protocol for 10 adult in silico subjects within the UVA/Padova simulator with insulin delivery by the proposed controllers.

There is a range of BG values that is generally considered to be close to the range for individuals without T1DM and thus is defined to be an acceptable range (80–140 mg/dL). A slightly wider range of values is considered to be a clinically safe range (70–180 mg/dL). Consequently, the time spent in a specified BG range is one of the most widely utilized metrics in evaluating BG control strategies. Temporary excursions above these ranges are deemed acceptable given the chronic, rather than acute, nature of the effects of hyperglycemia. The percentage of trial time each subject spends within the ranges is a good assessment of control performance. The comparisons of the simulated performances of the application of each proposed augmentation of the MPC control algorithm are seen in Figures 4 to 6 and Table 2, which characterize BG control performance based on the time-in-range metrics.

Figure 4.

Blood glucose control performance characterized by glucose and insulin delivery traces of the most insulin-sensitive in silico subject within the UVA/Padova simulator controlled by the baseline MPC controller and the same controller enhanced with personalized gain, asymmetric cost function, and a combination of all advancements, under the protocol described above. The green shaded region represents the 70–180 mg/dL safe glycemic range. The thin lines within the insulin delivery represent controller recommended delivery during that period, indicating the insulin delivery that was prevented by the pump occlusion.

Figure 6.

(A) Blood glucose control performance characterized by the median cumulative distribution function plots of 10 in silico subjects within the UVA/Padova simulator controlled by the baseline MPC controller and the same controller enhanced with personalized gain, asymmetric cost function, and a combination of all advancements. (B) A zoomed-in view of the differences in % time below the value for the region from 140 to 180 mg/dL.

Table 2.

Summary of mean clinical metrics that characterize glucose control performance of 10 adult in silico subjects controlled by the baseline and eMPC algorithms for 30 h sessions, including an unannounced 65 g carbohydrate dinner, an unannounced 50 g carbohydrate breakfast, and an announced 65 g carbohydrate lunch. This clinical protocol also incorporated induced hypoglycemia followed by hyperglycemia at 1:00PM, and a simulated dawn phenomenon from 5:00AM to 10:00AM to more closely reflect the challenges faced by T1DM patients in reality.

	Baseline	pMPC	eMPC (no personalization)	eMPC	p-value Baseline vs. eMPC
OVERALL STUDY
Mean glucose (mg/dL)	151 [12.0]	149 [9.06]	148 [10.2]	142 [7.67]	0.002*
% time <70 mg/dL	4.10 [1.50]	3.55 [0.57]	4.04 [1.50]	3.99 [1.34]	0.223
% time 70–180 mg/dL	68.8 [9.52]	72.3 [7.29]	70.4 [8.92]	76.9 [6.94]	0.002*
% time > 180 mg/dL	27.2 [9.08]	24.2 [7.33]	25.6 [8.38]	19.1 [6.32]	0.003*
OVERNIGHT PERIOD
Mean glucose (mg/dL)	116 [7.75]	120 [5.95]	114 [6.34]	115 [5.59]	0.570
% time <70 mg/dL	2.12 [5.92]	0.00 [0.00]	1.88 [5.95]	0.00 [0.00]	0.287
% time 70–180 mg/dL	97.9 [5.92]	99.9 [0.37]	98.1 [5.95]	100 [0.00]	0.287
% time > 180 mg/dL	0.00 [0.00]	0.12 [0.37]	0.00 [0.00]	0.00 [0.00]	N/S
5 HOUR PERIOD AFTER UNANNOUNCED MEAL
Mean glucose (mg/dL)	194 [25.7]	186 [23.3]	187 [21.4]	181 [19.1]	0.190
% time <70 mg/dL	0.00 [0.00]	0.00 [0.00]	0.00 [0.00]	0.00 [0.00]	N/S
% time 70–180 mg/dL	41.6 [23.9]	48.0 [21.1]	46.6 [21.1]	54.1 [14.7]	0.177
% time > 180 mg/dL	58.4 [23.9]	52.1 [21.1]	53.4 [21.1]	45.9 [14.7]	0.177

Brackets represent the standard deviation. Asterisks represent significance vs. baseline controller with α = 0.05.

Figures 4 and 5 show the mean glucose and insulin traces of the most sensitive in silico subject and the mean of 10 in silico subjects controlled by the baseline controller as well as the controller augmented with each proposed advancement for the clinical protocol described above. As is shown in the figures, the proposed advancements each show significant improvements in maintaining the subjects within the 80–140 mg/dL euglycemic, and the 70–180 mg/dL safe glycemic zones. For instance, the controller with all proposed advancements brings the subject’s BG back into the euglycemic range more than an hour earlier than the baseline controllers. These improvements are particularly apparent during the overnight low seen at approximately 2:00 AM for the most sensitive subject. Although the addition of each modification shows some improvement in reducing the extent of this hypoglycemic event, it is only with the combination of all advancements that the minimum BG remains above the 70 mg/dL hypoglycemic threshold.

Figure 5.

Blood glucose control performance characterized by the mean glucose and insulin delivery traces of 10 in silico subjects within the UVA/Padova simulator controlled by the baseline MPC controller and the same controller enhanced with personalized gain, asymmetric cost function, and a combination of all advancements, under the protocol described above. The thin lines within the insulin delivery represent controller recommended delivery during that period, indicating the insulin delivery that was prevented by the pump occlusion.

Panel A of Figure 6 shows the individual cumulative times in range of each modification, and panel B shows a zoomed-in view of the differences in the cumulative times in range around the hyperglycemic threshold of 180 mg/dL. As is shown in this figure, the controller that incorporates both advancements significantly outperforms the baseline controller in maintaining the subjects below 180 mg/dL by nearly 10%. The results and statistical significance of these changes is shown in Table 2. The combined controller shows statistically significant improvements in all metrics in comparison to the baseline controller by paired t-test (p<0.05), with reduced mean glucose and maximum glucose throughout the trial period (142 versus 151 and 245 versus 269 mg/dL, respectively), and increased times in both the 70–180 mg/dL safe glycemic range (76.9 versus 68.8%) and the 80–140 mg/dL euglycemic range (48.1 versus 44.5%). The complete controller also significantly reduces hyperglycemic risk, reducing the time in the hyperglycemic range (3.9% versus 5.37%) for the entire trial period. The proposed controller maintains these improvements without any statistically significant differences in hypoglycemic risk. It is important to note that the calculations of these metrics, such as the percent times in range for the overall trial period, were conducted including the hypoglycemic and hyperglycemic episodes induced by the design of the protocol. The proposed advancements show even more apparent improvements over the baseline design under more standard scenarios. The incremental benefits of incorporating each advancement are evident both through the cumulative times in range plot and the table of overall times in range even with the incorporation of these additional challenges.

3.2. Validation of the personalization and dynamic IOB through clinical trials

While clinical subjects with T1DM modulate their basal profiles by as many as ten segments or more per day, the in silico subjects of the simulator have a time-invariant basal rate. As a result, it is important to validate the proposed advancements to the algorithm to ensure stability in cases of time-varying basal profiles. Preliminary results of this validation of the personalization scheme for in vivo subjects with different basal segments exhibited no stability or robustness concerns despite cases where the subject’s basal parameters varied by 50% or more.

Panels A and B of Figure 7 show a validation of the MPC controller with the personalization scheme for a subject whose basal rates varied by as much as 33% (from 0.3 to 0.4 U/h). The clinical trial protocol for this subject was similar to that utilized in the previous section, but with announced dinner and breakfast meals followed by one unannounced lunch, and no induced hyper- or hypoglycemia (ClinicalTrials.gov NCT01987206). The MPC controller produces an insulin trace that maintains the subject’s BG concentrations within the acceptable (euglycemic) range of 80–140 mg/dL throughout the time period, without significant adverse events. In particular, although the last meal (lunch, 65g) was not announced to the controller, the algorithm successfully attenuated the BG excursion and returned the subject’s BG to the euglycemic range in a conservative manner.

Figure 7.

Clinical trial results and advisory mode analysis for a subject with a time-varying basal profile and the baseline MPC controller with the personalization enhancement. Panels (A) and (B) represent the CGM and insulin delivery traces during the actual closed-loop clinical trial. Panel (C) represents advisory mode calculations of the proposed control algorithm with dynamic IOB, given previous CGM trajectory and insulin delivery trajectory at each point.¹⁵

Although the overall BG control of this subject is exemplary, some areas of improvement can be noted. In panels A and B, the first low BG alert at 10:00 PM was preceded by an extended period of little or no insulin delivery. In fact, the controller limited insulin delivery to basal or lower values beginning at 5:00 PM, and attenuated insulin delivery even further from 8:00 PM on, when the subject’s BG value was above 150 mg/dL. Despite this extended period of pump attenuation, the subject’s BG decreased to 75 mg/dL. This situation was detected by a health monitoring system (HMS) ⁴¹ that was tuned to generate an alert if the predicted BG trajectory crosses the 65 mg/dL threshold during the next 15 min; consequently, the subject was alerted twice, at 9:05 PM (21:05) and 9:40 PM (21:40).

Ingestion of fast-acting CHO accompanying these alerts led to a rapid increase in BG, which resulted in aggressive actions by the setpoint MPC. Although hyperglycemia was prevented, this was achieved at the cost of oscillatory behavior. Excess delivery of insulin resulted in a rebound hypoglycemia that was alerted at 00:30. Although the time in the 80–140 mg/dL target range was satisfactory during this sequence of events, such oscillatory behavior is undesirable in clinical applications.

The enhanced dynamic IOB algorithm to minimize these oscillatory behaviors has been validated for the same clinical data through the advisory mode evaluation shown in panel C of Figure 7. An advisory mode validation supplies current glucose and insulin delivery history, as well as the current subject parameters, to the proposed controller at every control step. The controller is augmented with the enhanced IOB algorithm to calculate the insulin delivery. Thus, an advisory mode plot shows the point-by-point recommendations by proposed algorithm versus the original algorithm, and allows a validation of the controller action versus actual clinical outcomes (i.e., reduction of insulin delivery when hypoglycemic risk was observed later in the trial). These point-by-point advisory mode calculations can then be compared with the insulin delivery values during the trial to assess safety.

The results of this advisory mode evaluation of the enhanced IOB algorithm show that the proposed algorithm successfully attenuates the excess insulin delivery for the first low BG alert starting at 22:00. The additional delivery was persistently reduced by as much as 50% or more for the point-by-point comparison throughout the relevant time period. This comparison strongly suggests that the second low BG event might be avoided with the implementation of the enhanced dynamic IOB algorithm. It is important to note that this improvement occurs without a significant decrease in insulin deliveries to compensate for meal disturbances, with only minor changes to the controller’s responses to both the announced breakfast and unannounced lunch meals.

4. Discussion

Three major advancements to a previously validated MPC algorithm for AP systems are proposed that each address a significant challenge in automated BG control. Personalization of the underlying model of the MPC algorithm based on each subject’s basal parameter improves the controller’s ability to compensate for both interpersonal and intra-personal variations in insulin sensitivity, and particularly the diurnal variations in sensitivity. Specifically, the basal profile clinical parameter was incorporated into the core model that was introduced by van Heusden et al.³¹ to better tune the model to each subject in a medically inspired design. This modification improves upon the original decision based “safety factor” approach by recognizing, and compensating for, the reality that people with T1DM vary in the proportion of their TDI that is compensated by their basal profile versus correctional insulin deliveries (for meals and other disturbances). Moreover, this personalization scheme allows the controller to more closely tailor its actions to actual insulin requirements of people with T1DM by incorporating the time-varying basal profile values. An exponential, asymmetric weighting of the glucose excursions in the cost function for the MPC algorithm was also implemented for improved performance. Glucose excursions were penalized asymmetrically, with exponential scaling for movements below the target to respond nimbly to glucose excursions significantly below the setpoint, and quadratic scaling for excursions above the target, to maintain the reasonable responses to unannounced meals seen in clinical settings. This takes advantage of the clinical knowledge regarding the asymmetric nature of the BG scale, and allows the controller to have more robust responses to potential hypoglycemic events. Finally, a dynamic IOB algorithm to address rebound hypoglycemia that was observed for a number of clinical studies of these controllers was also validated. This enhanced IOB algorithm provides a dynamic constraint to minimize the hyper-and hypoglycemia rebounding effects that were observed during some previous clinical trials, following ingestion of fast-acting CHO after an extended period of pump suspension.

The in silico study protocol utilized for validation of these innovations was designed with realistic challenges in mind. Specifically, in addition to the bolused meal and meals with a missed bolus scenario, the controllers were also faced with induced hypoglycemia due to the delivery of an IV bolus unknown to the controller, followed by an induced hyperglycemia due to a simulated period of pump occlusion and accompanying rescue carbohydrate ingestion. These successive challenges show a stronger validation on the robustness of any controller designs that successfully maintain BG near the target despite such unknown disturbances. Additionally, these challenges highlight the advantage of preclinical in silico validations of these controllers prior to ambulatory studies, as while induced hypo- and hyperglycemia on in vivo subjects is dangerous, most, if not all published clinical studies in literature reported multiple instances of such events during the trials.

While each advancement individually showed an incremental improvement to the baseline controller during these in silico validations, the combination of both the personalization and the asymmetric cost function performed the best overall. A statistically significant improvement was seen across nearly all metrics, including improvements in percent time in the safe glycemic range, reduction in % time in the hyperglycemic range, and decrease in the overall mean glucose. These performances increases were despite the additional challenges introduced by the induced hypo- and hyperglycemia. The enhanced IOB was also validated on an advisory mode test of actual clinical data, and showed expected reductions in insulin delivery during scenarios in which a period of rapid glucose rises followed long pump suspensions.

It is important to acknowledge that there are several limitations to this study. While the controllers were evaluated with both in silico clinical trials with challenges beyond those typically seen in such scenarios and advisory mode testing on real clinical data, it is still not possible to definitively determine how the algorithms will perform in actual home-use settings. Moreover, there is a variety of everyday challenges, such as exercise, which may adversely affect the proposed AP controller. Finally, although these controllers were validated under advisory mode settings of clinical data, additional validations may be necessary for cases in which subjects’ basal parameters vary by an even greater degree.

We conclude that the combination of personalization of the controller, incorporation of exponential asymmetry into the controller cost function, and implementation of the enhanced IOB scheme, each successfully addresses a major challenge inherent in the field of controller design for AP. Future clinical studies of the complete controller with all enhancements on in vivo subjects will be necessary prior to its implementation for widespread use.

ABBREVIATIONS

AP

artificial pancreas

MPC

model-predictive control

T1DM

Type 1 diabetes mellitus

IOB

Insulin-on-board

BG

Blood glucose

ZMPC

Zone-MPC

pMPC

personalized-MPC

eMPC

exponential-MPC

CHO

Carbohydrates

CGM

Continuous glucose monitor

CSII

Continuous subcutaneous insulin injection

FDA

Food and Drug Administration

TDI

Total Daily Insulin

mg/dL

milligrams per deciliter

pmol/min

picomoles per minute

LBGI

Low blood glucose index

HBGI

High blood glucose index

UVA/Padova

Universities of Virginia/Padova