Design of a Dual-Hormone Model Predictive Control for Artificial Pancreas with Exercise Model

Abstract

The Artificial Pancreas (AP) is a new technology for helping people with type 1 diabetes to better control their glucose levels through automated delivery of insulin and optionally glucagon in response to sensed glucose levels. In a dual hormone AP, insulin and glucagon are delivered automatically to the body based on glucose sensor measurements using a control algorithm that calculates the amount of hormones to be infused. A dual-hormone MPC may deliver insulin continuously; however, it must avoid continuous delivery of glucagon because nausea can occur from too much glucagon. In this paper, we propose a novel dual-hormone (DH) switching model predictive control and compare it with a single-hormone (SH) MPC. We extended both MPCs by integrating an exercise model and compared performance with and without the exercise model included. Results were obtained on a virtual patient population undergoing a simulated exercise event using a mathematical glucoregulatory model that includes exercise. Time spent in hypoglycemia is significantly less with the DH-MPC than the SH-MPC (p=0.0022). Additionally, including the exercise model in the DH-MPC can help prevent hypoglycemia (p < 0.001).

I. Introduction

Diabetes mellitus is a physiological disorder where the body either is resistant to or does not sufficiently secrete insulin. Type 1 diabetes is the less prevalent but more challenging type of diabetes mellitus in which the pancreas fails to produce insulin for glucose regulation. Without insulin, glucose cannot be absorbed into tissues, and blood glucose (BG) levels increase, resulting in hyperglycemia (BG > 180 mg/dl). Difficulty controlling glucose is compounded by meals which require additional insulin. On the contrary, when glucose utilization is high, for example during physical activity, tissues absorb glucose more rapidly, and the BG can fall very steeply. The American Diabetes Association identifies a lower boundary of 70 mg/dl for the BG, below which is defined hypoglycemia, where severe symptoms may occur if untreated for even short periods [1]. The replacement of insulin in type 1 diabetes requires both basal insulin delivery, either with long-acting insulin or by continuous infusion of fast-acting insulin, and bolus insulin doses for meals with fast-acting insulin.

Insulin pump therapy (open-loop therapy) is a common method for people with type 1 diabetes to maintain euglycemia; the patient can use the insulin pump to infuse basal insulin and can also deliver boluses prior to meals [2]. For closed-loop insulin delivery, also known as the artificial pancreas (AP), insulin delivery is automatically adjusted based on sensed glucose levels and a control algorithm. In single hormone AP systems, only insulin is dosed whereas in dual hormone APs, glucagon is also dosed [3].

A number of closed loop algorithms utilize a proportional integral derivative controller [4, 5]. Our group has previously described variations of PID or PD AP control algorithm [6]. We also have described an exercise detection and dosing adjustment algorithm that can be incorporated [7]. Other groups have implemented model predictive control (MPC) to manage BG such as a SH MPC by Magni et al. [8], a DH MPC by Boiroux et al. [9], and a SH MPC by Hovorka et al. [10]. Boiroux et al. [9] used a switching controller that switches between insulin and glucagon SH MPC based on the patient’s glucose. When glucose levels are below a threshold (e.g. 85 mg/dl), the glucagon SH-MPC would be used, otherwise, insulin SH-MPC would be used. To the best of our knowledge, no prior groups have incorporated exercise directly into the process model of a DH-MPC.

In this paper, we introduce a DH-MPC approach that can switch between dual hormone and single hormone operation based on the sensed glucose level of the patient. Our DH-MPC algorithm includes a model for exercise, such that if exercise is detected or if a patient announces an exercise event to the controller, the algorithm can respond appropriately. We test the feasibility of the algorithm in the presence of moderate exercise. We also compared the performance when including exercise versus not including the exercise model in the MPC. We linearize the non-linear equations within the process model to reduce the processing time in the controller.

II. Material and Method

A. Process Model

The prediction of BG in our MPC algorithm is achieved using a glucoregulatory model, consisting of a glucose kinetics model, insulin kinetics and dynamics models, and glucagon kinetics and dynamics models. The model describes the relationship between subcutaneously delivered insulin and glucagon and blood glucose concentration. The glucose kinetics model [10] defines the effect of the insulin and glucagon actions on the blood glucose, as follows:

$$\begin{array}{l}{\stackrel{.}{\mathrm{Q}}}_1={\mathrm{x}}_1{\mathrm{Q}}_1-{\mathrm{F}}_{01}^{\mathrm{c}}-{\mathrm{F}}_{\mathrm{R}}+{\mathrm{k}}_{12}{\mathrm{Q}}_2+{\mathrm{u}}_{\mathrm{G}}+{\text{EGP}}_0(1-{\mathrm{x}}_3+\mathrm{Y}+{\text{kg}}_3\stackrel{.}{\mathrm{Y}})\\ {\stackrel{.}{\mathrm{Q}}}_2={\mathrm{x}}_1{\mathrm{Q}}_1-{\mathrm{k}}_{12}{\mathrm{Q}}_2-{\mathrm{x}}_2{\mathrm{Q}}_2\end{array}$$ (1)

where Q₁ and Q₂ are the masses of glucose in the accessible and non-accessible compartments respectively, in mmol/L. ${\mathrm{F}}_{01}^{\mathrm{c}}$ and F_R represent non-insulin mediated glucose uptake and renal glucose clearance respectively, in mmol/L/min. u_G represents gut absorption rate in mmol/L/min, which is considered zero in this study. EGP₀ is the basal endogenous glucose production in mmol/L/min at a theoretical zero insulin concentration [10]. The source of non-linearity in these equations comes from the interactions between the effect of the insulin on EGP and the distribution and disposal within the measurable and non-measurable glucose compartments. These nonlinear equations are linearized based on a Taylor series expansion at each time point, where the higher order terms are excluded.

The insulin kinetics model (Eq. 2) represents the insulin absorption rate from the short-acting insulin dosing [11].

$$\begin{array}{l}{\stackrel{.}{\mathrm{Q}}}_{1\mathrm{a}}={\text{ku}}_{\mathrm{I}}-{\mathrm{k}}_{\mathrm{a}1}{\mathrm{Q}}_{1\mathrm{a}}-\frac{{\mathrm{V}}_{max}{\mathrm{Q}}_{1\mathrm{a}}}{{\mathrm{k}}_{\mathrm{m}}+{\mathrm{Q}}_{1\mathrm{a}}}\\ {\stackrel{.}{\mathrm{Q}}}_{1\mathrm{b}}=(1-\mathrm{k}){\mathrm{u}}_{\mathrm{I}}-{\mathrm{k}}_{\text{ia}2}{\mathrm{Q}}_{1\mathrm{b}}-\frac{{\mathrm{V}}_{max}{\mathrm{Q}}_{1\mathrm{b}}}{{\mathrm{k}}_{\mathrm{m}}+{\mathrm{Q}}_{1\mathrm{b}}}\\ {\stackrel{.}{\mathrm{Q}}}_2={\mathrm{k}}_{\text{ia}1}{\mathrm{Q}}_{1\mathrm{a}}-{\mathrm{k}}_{\text{ia}1}{\mathrm{Q}}_2\\ {\stackrel{.}{\mathrm{Q}}}_3={\mathrm{k}}_{\text{ia}1}{\mathrm{Q}}_2+{\mathrm{k}}_{\text{ia}2}{\mathrm{Q}}_{1\mathrm{b}}-{\mathrm{k}}_{\mathrm{e}}{\mathrm{Q}}_3\end{array}$$ (2)

where Q_1a and Q₂ represent the insulin mass through the slow absorption pathway, and Q_1b represents a faster channel for insulin absorption. Q₃ represents plasma insulin mass [11]. All insulin masses are in mU/kg. The sources of nonlinearity in these equations come from the local degradation of insulin at the injection site. These non-linear equations are linearized using a Taylor series. The insulin dynamics model is based on a study from Hovorka et al [10].

$$\begin{array}{ll}{\stackrel{.}{\mathrm{x}}}_1\hfill & =-{\mathrm{k}}_{\mathrm{a}1}{\mathrm{x}}_1+\frac{{\mathrm{k}}_{\mathrm{b}1}}{{\mathrm{V}}_{\mathrm{d}}{\mathrm{I}}_{\mathrm{N}}}{\mathrm{Q}}_3\hfill \\ {\stackrel{.}{\mathrm{x}}}_2\hfill & =-{\mathrm{k}}_{\mathrm{a}2}{\mathrm{x}}_2+\frac{{\mathrm{k}}_{\mathrm{b}2}}{{\mathrm{V}}_{\mathrm{d}}{\mathrm{I}}_{\mathrm{N}}}{\mathrm{Q}}_3\hfill \\ {\stackrel{.}{\mathrm{x}}}_3\hfill & =-{\mathrm{k}}_{\mathrm{a}3}{\mathrm{x}}_3+\frac{{\mathrm{k}}_{\mathrm{b}3}}{{\mathrm{V}}_{\mathrm{d}}{\mathrm{I}}_{\mathrm{N}}}{\mathrm{Q}}_3\hfill \end{array}$$ (3)

where x₁ (min⁻¹), x₂ (min⁻¹) and x₃ (unitless) represent the effect of insulin on glucose distribution, glucose disposal and suppression of endogenous glucose production, respectively and V_{d_IN} is the insulin distribution volume (L.kg-1) [10]. The glucagon kinetics model, which represents the absorption rate of subcutaneously injected glucagon, is reported in [12].

$$\begin{array}{ll}{\stackrel{.}{\mathrm{x}}}_{1\mathrm{g}}\hfill & =-({\mathrm{k}}_{\mathrm{g}1}+{\mathrm{k}}_{\text{ge}1}){\mathrm{x}}_{1\mathrm{g}}+{\mathrm{u}}_{\mathrm{g}}\hfill \\ {\stackrel{.}{\mathrm{x}}}_{2\mathrm{g}}\hfill & ={\mathrm{k}}_{\mathrm{g}1}{\mathrm{x}}_{1\mathrm{g}}-{\mathrm{k}}_{\mathrm{g}2}{\mathrm{x}}_{2\mathrm{g}}\hfill \\ {\stackrel{.}{\mathrm{x}}}_{3\mathrm{g}}\hfill & ={\mathrm{k}}_{\mathrm{g}2}{\mathrm{x}}_{2\mathrm{g}}-{\mathrm{k}}_{\text{ge}2}{\mathrm{x}}_{3\mathrm{g}}\hfill \end{array}$$ (4)

where x_1g and x_2g represent subcutaneous glucagon mass compartments and x_3g is plasma glucagon mass, all measured in mg/kg. The nominal values of the parameters for the glucagon kinetics model used in this study have been provided from Lv et al [12]. The glucagon dynamic model, which describes the interaction between the glucagon and the glucose concentration, was published by us previously [7].

$$\begin{array}{l}\stackrel{.}{\mathrm{Y}}=\frac{{10}^6\times {\mathrm{k}}_{\mathrm{c}}\times {\mathrm{S}}_{\text{fGG}}}{{\mathrm{V}}_{\text{dGG}}}{\mathrm{x}}_{3\mathrm{g}}-{\mathrm{k}}_{\mathrm{c}}\mathrm{Y}={\mathrm{k}}_{\mathrm{g}}{\mathrm{x}}_{3\mathrm{g}}-{\mathrm{k}}_{\mathrm{c}}\mathrm{Y}\\ \stackrel{.}{\mathrm{Y}}=\mathrm{Z}\Rightarrow \stackrel{.}{\mathrm{Z}}={\mathrm{k}}_{\mathrm{g}}{\mathrm{k}}_2{\mathrm{x}}_{2\mathrm{g}}-{\mathrm{k}}_{\mathrm{g}}{\mathrm{k}}_{\mathrm{c}2}{\mathrm{x}}_{3\mathrm{g}}-{\mathrm{k}}_{\mathrm{c}}\mathrm{Z}\end{array}$$ (5)

Y represents the effect of glucagon on endogenous glucose production. Since the change on Y has an effect on EGP, we introduce a new variable Z as another state, which is used in the MPC algorithm. The values of the parameters of the glucagon kinetics are described previously [7]. These values change per subject and make a virtual patient population (see section 2.D).

B. Model Predictive Controller

Model predictive control is an optimization based control algorithm, which considers the dynamic model of the plant. Unlike PID controllers, MPC is able to predict the future outputs and optimize the inputs to the plant accordingly.

Figure 1 shows the structure of the MPC control methodology. The glucoregulatory model is the MPC process model. Typically, the plant would be the patient using the AP. The plant (or human) receives hormones as an insulin infusion rate (IIR) or glucagon infusion rate (GIR) and has a blood glucose level measured by a sensor (BG). For our simulations, the plant is approximated using the same glucoregulatory state equations described above. However, while the process model parameters are kept constant during the simulation and are set to the mean values from prior published studies [10–13], the plant model parameters vary for each virtual patient tested during simulation.

Figure 1.

MPC Schematic

There are 13 state variables, which include two state variables from the glucose kinetics model, four from the insulin kinetics model, three from the insulin dynamic models, three from the glucagon kinetics model and two from the glucagon dynamic model. Since the derivative of the glucagon concentration also affects the glucose concentration (equation 1), we define the second glucagon dynamic model state variable (variable Z in equation 5) for this effect. The final linearized form of the MPC equations is as follows:

$$\begin{array}{l}{x}_m(k+1)=A_m{x}_m(k)+B_{m}u(k)+d_m(k);\\ y(k)=C_m{x}_m(k);\end{array}$$ (6)

x_m(k) is the state vector, u(k) is the 2-dimensional input vector (insulin and glucagon) and d(k) includes the constant terms resulting from the linearization. Since the controller requires a history of the output for future predictions, it is essential to relate the input vector to the output. We define a new vector as follows x (k) = [Δx_m (k)^T y (k)]^T and the augmented state equations are re-arranged below:

$$\begin{array}{l}\underset{x(k+1)}{\underbrace{\left[\begin{array}{c}\mathrm{\Delta }{x}_m(k+1)\\ y(k+1)\end{array}\right]}}=\underset{A}{\underbrace{\left[\begin{array}{cc}{A}_m& o_m^T\\ C_m{A}_m& 1\end{array}\right]}}\underset{x(k)}{\underbrace{\left[\begin{array}{c}\mathrm{\Delta }{x}_m(k)\\ y(k)\end{array}\right]}}+\underset{B}{\underbrace{\left[\begin{array}{c}{B}_m\\ C_m{B}_m\end{array}\right]}}\mathrm{\Delta }u(k)+\underset{D}{\underbrace{\left[\begin{array}{c}1\\ C_m\end{array}\right]}}\mathrm{\Delta }{d}_m(k);\\ y(k)=\underset{C}{\underbrace{\left[\begin{array}{cc}{o}_m& 1\end{array}\right]}}\left[\begin{array}{c}\mathrm{\Delta }{x}_m(k)\\ y(k)\end{array}\right];\end{array}$$ (7)

The predicted outputs are calculated using equation 8.

$$Y_P=Fx(k)+\mathrm{\Phi }\mathrm{\Delta }U+\mathrm{\Psi }\mathrm{\Delta }D,$$ (8)

where the matrices F, Φ and Ψ are presented in [14] and

$$\begin{array}{l}{Y}_P={[y(k+1)y(k+2)\cdots y(k+N_p)]}^T,\\ \mathrm{\Delta }U={[\mathrm{\Delta }u(k)\mathrm{\Delta }\mathrm{u}(\mathrm{k}+1)\cdots \mathrm{\Delta }\mathrm{u}(\mathrm{k}+{\mathrm{N}}_c-1)]}^T,\\ \mathrm{\Delta }D={[\mathrm{\Delta }{d}_m(k)\mathrm{\Delta }{d}_m(k+1)\cdots \mathrm{\Delta }{d}_m(k+{\mathrm{N}}_c-1)]}^T,\end{array}$$

We chose a 300-minute prediction horizon as the action of insulin is several hours [10]. We chose a 20-minute control horizon as the results did not change substantially with a longer horizon. The cost function is defined in equation 9, which includes the reference trajectory (R_s) and the predicted outputs, and the tuning control parameter (R̄).

$$J={(R_s-Y_P)}^T(R_s-Y_P)+\mathrm{\Delta }{U}^T\overline{R}\mathrm{\Delta }U,$$ (9)

We compute the output of the optimizer (future inputs) by setting the derivative of the cost function with respect to ΔU to zero and; after some calculations, the optimal ΔU is defined in equation 10.

$$\mathrm{\Delta }U={({\mathrm{\Phi }}^T\mathrm{\Phi }+\overline{R})}^{-1}{\mathrm{\Phi }}^T(R_s-Fx(k)-\mathrm{\Psi }\mathrm{\Delta }D),$$ (10)

At the next step, we impose some constraints on the insulin and glucagon delivery. We set the maximum amount of insulin infusion rate (IIR) and glucagon infusion amount (GIR) to 15 units per hour and 50 microgram, respectively. In this paper, we show the performance of (1) a single hormone MPC algorithm (SH-MPC) in which the insulin is the only hormone used for blood glucose regulation, and (2) a dual hormone algorithm (DH-MPC). For the dual-hormone, we consider different ways of switching between single hormone and dual hormone operation. One option (DH-Thr) is to use a SH when glucose levels are greater than a threshold (85 mg/dl) and then use DH when glucose levels drop below this threshold. A second option (DH-Pred) is to use SH when glucose levels are predicted to rise above 70 mg/dl during the prediction horizon, otherwise switch to DH-MPC.

C. Exercise model

Exercise can have profound effects on glucose levels in a person with type 1 diabetes, causing hypoglycemia if dosing is not adjusted. We incorporated an exercise model described by Hernandez et al. [13] into our process model. The exercise model, which models aerobic exercise, affects the influence of insulin on glucose transport and disposal, and endogenous glucose production and causes increased insulin sensitivity by influencing equations 3 (k_b1, k_b2 and k_b3) as follows:

$$\begin{array}{l}{\mathrm{k}}_{\mathrm{b}1}^{\ast }={\mathrm{M}}_{\text{PGU}}\times {\mathrm{M}}_{\text{PIU}}\times {\mathrm{k}}_{\mathrm{b}1}\\ {\mathrm{k}}_{\mathrm{b}2}^{\ast }={\mathrm{M}}_{\text{PGU}}\times {\mathrm{M}}_{\text{PIU}}\times {\mathrm{k}}_{\mathrm{b}2}\\ {\mathrm{k}}_{\mathrm{b}3}^{\ast }={\mathrm{M}}_{\text{HGP}}\times {\mathrm{k}}_{\mathrm{b}3}\end{array}$$ (11)

The model parameters (k_b1, k_b2 and k_b3) are updated during the exercise period according to equation 12.

$$\begin{array}{l}{\mathrm{M}}_{\text{PGU}}=1+\frac{{\mathrm{\Gamma }}_{\text{PGUA}}\times \text{PAMM}}{35\text{mg}/min};\\ {\mathrm{M}}_{\text{PIU}}=1+2.4\times \text{PAMM};\\ {\mathrm{M}}_{\text{HGP}}=1+\frac{{\mathrm{\Gamma }}_{\text{HGPA}}\times \text{PAMM}}{155\text{mg}/min};\end{array}$$ (12)

where Γ_PGUA and Γ_HGPA, which are related to the percent of maximum oxygen consumption (PVO_2max), are calculated based on a dynamic model that is explained further in [13]. PAMM is the percent of active muscular mass.

D. Virtual patient population

We defined a virtual patient population for our in-silico simulations. We changed the most sensitive inter-subject parameters (EGP₀, k_b1, k_b2, k_b3, S_fGG, k_c and k_g3) of the process model across each subject. First, we produced a normal distribution based on the information of each parameter, given prior studies [7, 10, 11], and then randomly selected samples, using a random number generator that is weighted by the parameter distribution, in order to generate the virtual population. Each parameter set was then subjected to a battery of tests to determine the physiologic feasibility. If a virtual subject did not pass each of the following 4 criteria, they were excluded from further testing. 1) Steady-state glucose levels – in the absence of insulin, this should be > 300 mg/dl. 2) Delivery of high-dose insulin (15 units/h) should result in a low steady-state glucose level < 100 mg/dl. 3) Delivery of high-dose glucagon (20 mcg/kg) should result in a significant rise in glucose within 2 hours of the dose > 50 mg/dl above the baseline steady state glucose. 4) Delivery of a small dose of glucagon (0.2 mcg/kg) should not result in a response > 100 mg/dl above baseline. A total of 163 out of 400 virtual subjects passed the above criteria and were selected for our in-silico simulations.

E. Test scenario and statistical analysis

All virtual subjects completed the following scenario. First, their glucose levels were brought to a steady state value of 160 mg/dl at time t=0. Next, subjects completed the equivalent of 45 minutes of exercise starting at time t=10 minutes at 60% PVO_2max and 80% PAMM. We compared the control algorithms and assessed performance in terms of the average time spent in hypo/hyperglycemia and the average blood glucose across the virtual subjects. We used the Student’s t-test and the Wilcoxon rank-sum test to compare the average blood glucose and the time spent in hypo-/hyperglycemia, respectively, across different process model configurations. The significance level was 95%.

III. Results and Discussion

Figure 2 shows the benefit of using DH compared with SH. Some subjects within the 25%–75% interquartile range using the SH process model crossed the hypoglycemia threshold interquartile range, whereas subjects using the DH model generally did not go hypoglycemic (only the lower 2.5% of subjects went hypoglycemic under DH). The time the virtual subjects spent in hypoglycemia was significantly less for the DH (p-value = 0.0022). In DH, the glucagon delivery not only prevented more subjects from becoming hypoglycemic, but also increased the median blood glucose across the subjects. Preventing hypoglycemia and reducing time spent in hypoglycemia can prevent symptoms related to hypoglycemia including nervousness, shakiness, dizziness and nausea. It is important to note that both the SH and DH process models included exercise in these results.

Figure 2.

Interquartile range along with the 95% intervals of the blood glucose across the virtual subjects are shown for DH-Thr (red line) and SH (blue line) methods with the exercise model included.

Figure 3 shows the importance of including the exercise model in the process model. When the exercise model is not included, hypoglycemia occurs, whereas when it is included, hypoglycemia is generally avoided. When the exercise model is not included, the algorithm cannot anticipate forthcoming exercise-induced hypoglycemia and thereby turn off insulin and increase glucagon dosing as shown in the insulin and glucagon delivery panels of Figure 3. The time spent in hypoglycemia was significantly less when exercise was included (p-value < 0.001). This makes intuitive sense. Aerobic exercise can cause rapid drops in glucose levels because of increased glucose uptake during exercise. If the process model is not aware of this change and dosing is not adjusted, then hypoglycemia results.

Figure 3.

Interquartile ranges along with the 95% intervals of the blood glucose across the virtual subjects are shown for the DH-Thr with exercise model (red line) and DH-Thr without exercise model (cyan line).

There was a difference in performance when the process model switched between DH and SH based on a fixed glucose threshold (DH-Thr) compared with a predicted drop below 70 mg/dl (DH-Pred). DH-Pred resulted in some post-exercise hyperglycemia. While time spent in hypoglycemia with the DH-Pred approach did not change significantly in comparison to the DH-Thr approach (p-value = 0.011), the time spent in hyperglycemia, as well as the median blood glucose, were significantly higher for DH-Pred (p-value < 0.001). Results here indicate that DH-Thr is optimal for controlling glucose during exercise.

A limitation of this study is that the exercise model only models the effect of aerobic exercise on the glucoregulatory system. Anaerobic exercise can result in a less significant drop in glucose compared with aerobic exercise [15]. Furthermore, the exercise model was static for all subjects, whereas different people can respond differently to exercise. In the future, we plan to integrate anaerobic exercise and high intensity interval training into the exercise model.

In conclusion, the threshold-based dual-hormone MPC with exercise in the process model outperformed the other controlling approaches. It was critical to include exercise in the controller process model so that the time spent in hypoglycemia would be reduced. In the future, we plan to include a meal model to the process model (u_G would not be zero in equation 1) and evaluate in human subjects.