Abstract
Background
The development of artificial pancreas has received a new impulse from recent technological advancements in subcutaneous continuous glucose monitoring and subcutaneous insulin pump delivery systems. However, the availability of innovative sensors and actuators, although essential, does not guarantee optimal glycemic regulation. Closed-loop control of blood glucose levels still poses technological challenges to the automatic control expert, most notable of which are the inevitable time delays between glucose sensing and insulin actuation.
Methods
A new in silico model is exploited for both design and validation of a linear model predictive control (MPC) glucose control system. The starting point is a recently developed meal glucose–insulin model in health, which is modified to describe the metabolic dynamics of a person with type 1 diabetes mellitus. The population distribution of the model parameters originally obtained in healthy 204 patients is modified to describe diabetic patients. Individual models of virtual patients are extracted from this distribution. A discrete-time MPC is designed for all the virtual patients from a unique input–output-linearized approximation of the full model based on the average population values of the parameters. The in silico trial simulates 4 consecutive days, during which the patient receives breakfast, lunch, and dinner each day.
Results
Provided that the regulator undergoes some individual tuning, satisfactory results are obtained even if the control design relies solely on the average patient model. Only the weight on the glucose concentration error needs to be tuned in a quite straightforward and intuitive way. The ability of the MPC to take advantage of meal announcement information is demonstrated. Imperfect knowledge of the amount of ingested glucose causes only marginal deterioration of performance. In general, MPC results in better regulation than proportional integral derivative, limiting significantly the oscillation of glucose levels.
Conclusions
The proposed in silico trial shows the potential of MPC for artificial pancreas design. The main features are a capability to consider meal announcement information, delay compensation, and simplicity of tuning and implementation.
Keywords: artificial pancreas, diabetes, model predictive control, simulation, virtual patients
Introduction
The development of artificial pancreas, e.g., a closed-loop control system for maintaining normoglycemia in type 1 diabetes mellitus (T1DM), has been envisaged and discussed since the 1970s.1,2 However, devices requiring intravenous blood glucose sampling and intravenous glucose and insulin delivery, such as the Biostator™, were not suitable for outpatient use. Recent technological advancements in subcutaneous continuous glucose monitoring and subcutaneous insulin delivery systems have paved the way to the development of minimally invasive glucose control systems.2,3
However, the availability of innovative sensors and actuators, although essential, does not guarantee the achievement of optimal glycemic regulation under all conditions. Closed-loop control of blood glucose levels still poses technological challenges to the automatic control expert.
The principal obstacle to satisfactory closed-loop control is the presence of significant disturbances (i.e., meals and physical activity) and delays in the effect of meals and subcutaneous insulin on glycemia and furthermore from glycemia to measured subcutaneous glucose. Moreover, the control system must satisfy constraints on both plasma glucose levels and insulin delivery rates. These features explain the difficulties encountered when standard proportional integral derivative (PID) controllers are employed. Model predictive control (MPC) is likely to be the most suitable approach to design control systems in the presence of delays and constraints.4,5 Compensation for delays by means of feed-forward action, as well as constraint handling, is naturally incorporated in the design process. For the possible application of MPC strategies to glucose control in T1DM, the reader is referred elsewhere.2,6,7
The comparison of different control algorithms is facilitated greatly by the availability of reliable large-scale simulation models. In fact, in silico trials are perhaps the best way to address the robustness of the artificial pancreas against interindividual variability prior to conducting in vivo clinical trials. Until recently, the drawback of large-scale computer simulation models was the difficulty of identifying all relevant parameters from plasma concentration measurements. Recently, a new generation in silico model of the glucose–insulin system has been developed from the analysis of 204 nondiabetic individuals with various degrees of glucose tolerance who underwent a triple tracer meal protocol.8 This way, it was possible to obtain glucose and insulin fluxes during a meal independently of the model. Exploiting the knowledge of glucose production, utilization, rate of appearance in plasma, and pancreatic insulin secretion, it is possible to identify the various unit processes of the system through a subsystem forcing function strategy.
In this article, the new in silico model is exploited for both design and validation of a linear MPC system. First, the model is modified to provide a good description of the metabolism in T1DM. Then, a linearization of the model around basal values is used to design a linear MPC scheme. A major problem for the artificial pancreas is guaranteeing satisfactory performance under conditions of metabolic disturbance and interindividual variability. In order to validate this aspect, the population distribution of the model parameters of the healthy 204 individuals was modified to obtain the parameter distribution of diabetic patients. Individual models of virtual patients are then extracted from this distribution. The availability of realistic individual models is the basis for conducting an in silico trial: the closed-loop control can be tuned individually and then tested on each virtual patient, possibly injecting disturbances and uncertainties in order to assess robustness of control. The protocol simulates 4 days during which the patient receives breakfast, lunch, and dinner each day. The performance of the MPC system is compared to the performance of a standard PID controller.
Methods
In order to synthesize and test the controller, we used the meal glucose–insulin model.8 Some modifications have been introduced in order to simulate the metabolic specifics of T1DM.
Model of Glucose–Insulin Dynamics
Glucose Intestinal Absorption. Glucose intestinal absorption was modeled by a recently developed three-compartment model9
(1) |
where Qsto(mg) is the amount of glucose in the stomach (solid, Qsto1, and liquid phase, Qsto2), Qgut(mg) is the glucose mass in the intestine, kgri is the rate of grinding, kabs is the rate constant of intestinal absorption, f is the fraction of intestinal absorption that actually appears in plasma, d(mg/min) is the rate of ingested glucose, BW (kg) is body weight, Ra (mg/kg/min) is the glucose rate of appearance in plasma, and kgut is the rate constant of gastric emptying, which is a time-varying nonlinear function of Qsto
where t and tf are the initial and final times of the last ingestion, while a, b, kmax, and kmin are model parameters.
Glucose Subsystem. A two-compartment model is used to describe glucose kinetics10
(2) |
where Gp (mg/kg) and Gt (mg/kg) are glucose masses in plasma and rapidly equilibrating tissues and in slowly equilibrating tissues, respectively, EGP is endogenous glucose production (mg/kg/min), E (mg/kg/min) is renal excretion, Uii and Uid are insulin-independent and -dependent glucose utilizations, respectively (mg/kg/min), and k1 and k2 are rate parameters.
Glucose Renal Excretion. Renal excretion, which occurs if plasma glucose exceeds a certain threshold, is modeled as follows11
(3) |
where ke1 is the glomerular filtration rate and ke2 is the renal threshold of glucose.
Endogenous Glucose Production. EGP comprises a direct glucose signal and a delayed insulin signal12
(4) |
where the delayed insulin signal Id (pmol/liter) is given by
(5) |
where I (pmol/liter) is the plasma insulin concentration, kp1 is the extrapolated EGP at zero glucose and insulin, kp2 is liver glucose effectiveness, kp3 is a parameter governing the amplitude of insulin action on the liver, and ki is the rate parameter accounting for the delay between insulin signal and insulin action.
Glucose Utilization. Glucose utilization consists of two components: an insulin-independent glucose utilization Uii, which represents the glucose uptake by the brain and erythrocytes, and an insulin-dependent component Uid, which depends nonlinearly on glucose concentration in the tissues13:
(6) |
where Km, Vm0, and Vmx and are model parameters, X (pmol/liter) is the remote insulin signal, Ib (pmol/liter) is the basal insulin level, and p2U is a rate constant of insulin action on peripheral glucose utilization.
Subcutaneous Insulin Kinetics. This article adopts a variation of a model described in Verdonk et al.13:
(7) |
where u(t) (pmol/kg/min) represents the administration (bolus and infusion) of insulin. The first compartment represents the amount of nonmonomeric insulin in the subcutaneous space, which is partly transformed into monomeric insulin (second compartment) and partly enters the circulation with rate constants of insulin absorption ka1 and kd, respectively; the monomeric insulin is finally absorbed with rate constant ka2.
Insulin Subsystem. The model equations are
(8) |
where Ip(t) = VII(t) (pmol/kg) and Il (pmol/kg) are insulin masses in plasma and in liver, respectively, VI (liter/kg) is the body weight-normalized insulin volume, and mi, i = 1,…,4 are model parameters.
Subcutaneous Glucose Kinetics. Subcutaneous glucose concentration GM (mg/dl) is obtained as
(9) |
where VG (dl/kg) is the body weight-normalized glucose volume and ksc is a rate constant.
Virtual Patient Generation. In order to obtain parameter joint distributions in T1DM, the parameters identified in 204 healthy subjects were used as a starting point.8 Some modification was needed to realistically describe the metabolism of a person with T1DM. The basal glucose concentration was assumed to be on average 50 mg/dl higher than in nondiabetic individuals, the insulin concentration (due to an external insulin pump) was assumed to be on average four times higher than in nondiabetic individuals, endogenous glucose production was assumed to be 35% higher than in nondiabetic individuals, and insulin clearance was assumed to be one-third lower than in nondiabetic individuals. Parameters relating to insulin action on both glucose production and utilization were assumed to one-third lower than in nondiabetic individuals. For all parameters and variables, the same intersubject variability of nondiabetic subjects was maintained. The parameters were assumed to be log-normal distributed to guarantee that they were always positive. A covariance matrix (26 × 26) was calculated using the log-transformed parameters. One hundred subjects were generated using the joint distribution, i.e., 100 realizations of the log-transformed parameter vector were extracted randomly from the multivariate normal distribution with a mean equal to the mean of the log-transformed parameters and a 26 × 26 covariance matrix. Finally, the parameters in the 100 virtual subjects were obtained by antitransformation.
Performance Assessment
Virtual Protocol. The performance of closed-loop glucose control was tested on a 4-day virtual protocol:
simulation starts at basal value and the first meal is dinner at 7:30 pm of day 1; the patient has breakfast at 9:30 am with 45 grams of glucose, lunch at 1:30 pm with 75 grams of glucose, and dinner at 7:30 pm with 85 grams of glucose
in the first part of the simulation, the virtual “patient” receives a subcutaneous bolus based on an open-loop strategy, while at 9:30 pm of day 2 the controller is plugged in. Thereafter, the piecewise constant insulin delivery is governed by the closed-loop controller and no further bolus is administrated.
The virtual protocol has been designed so as to reproduce a likely clinical trial conducted on real patients. In particular, the first open-loop phase serves as an observation window during which individual patient information may be collected. Insulin delivery during closed-loop control is piecewise constant and is updated every 30 minutes. Shorter sampling intervals are technologically possible but are not compatible with medical supervision likely to be required in the first clinical trials on real patients.
Performance Indices. Some established indices of glucose control are considered.
- Low blood glucose index (LBGI)14: given n samples of plasma glucose concentration Gp(i)
where rl(·) = 10(g((ln(·))a – b))2 if g((ln(·))a – b) <0 and zero otherwise. The positive parameters g, a, and b are such that rl(70) = rl(280) = 25 and rl(50) = r(400). This index captures the propensity of the algorithm to overshoot the target and possibly trigger hypoglycemia. - High blood glucose index (HBGI)14: directly linked to LBGI, it captures the propensity of the algorithm to stay above the target range
where rh(·) = 10(g((ln(·))a – b))2 if g((ln(·))a – b) >0 and zero otherwise.
Coefficients of LBGI and HBGI have been modified with respect to literature values to better suit control performance results.
Minimum of blood glucose concentration Gp/VG (Min_Glycemia).
Maximum of blood glucose concentration Gp/VG(Max_Glycemia).
In order to allow for the transition from open-loop to closed-loop regulation, all indices are computed for two different periods: commutation from 9:30 pm of day 2 to 8:00 am of day 3 and regulation after 8:00 am of day 3.
Model Predictive Control
The glucose metabolism model can be rewritten in the following compact way:
(10) |
where x = [Qsto1, Qsto2, Qgut, Gp, Gt, Ip, X, I1, Id, Il, S1, S2, GM], and f is derived from the model equations reported in the previous section. In the following, it is assumed that meal announcement is available, i.e., the disturbance signal d (the meal) is known in advance.
The MPC control law is based on the solution of a finite horizon optimal control problem (FHOCP), where a cost function J(x,u) is minimized with respect to the input u subject to the state dynamics of a model of the system. Letting u° be the solution of the FHOCP, according to the receding horizon paradigm, the feedback control law u = kMPC(x) is obtained by applying only the first element of the optimal solution to the system. This way, a closed-loop control strategy is obtained solving an open-loop optimization problem.
Model predictive control laws can be formulated for both discrete- and continuous-time systems. In this article, a discrete-time MPC is derived from a unique input–output-linearized approximation of the full model based on the average population values of the parameters.
Given the average basal values of Gp, Gt, and Ip in the population, the associated equilibrium point with d = 0 is indicated by (x,u,d,y). Around this equilibrium point, assuming kgut(t, Qsto) = (kmax – kmin)/2, the system is linearized and discretized with sample time Ts = 30 minutes, yielding
(11) |
where δx(k) = x(kTs) − x, δu(k) = u(kTs) − u, and δy(k) = y(kTs) − y. After some passages, including a model reduction step derived through a balanced realization of the linearized system and a truncation of the state vector (the MATLAB Control Systems Toolbox instruction modred was used), the system is rewritten in the following state-space (nonminimal) representation
(12) |
where xIO(k + 1) = [δy(k + 1), δy(k), δy(k–1), δu(k), δu(k–1), d(k), d(k–1)]′, and the matrices AIO, BIO, MIO, and CIO are defined accordingly.
In order to derive the MPC control law, the following quadratic discrete-time cost function is considered
(13) |
where q is a positive constant.
The solution of the optimization problem has the following structure:
(14) |
where y0 is the future (constant) value of the set point, D(k) = [d(k), d(k + 1),…, d(k + 7), d(k + 8)] is the disturbance signal (meal), and , , GD are suitable matrices.
If the calculated insulin rate u(t) is negative, a zero value will be applied to the system. The fulfilment of the state constraints, however, cannot be guaranteed; it is only possible to tune the parameter q so as to improve the regulation performance. The major advantages of this input–output MPC scheme are that an observer is not required (xIO is made of past input and output values) and that it is easily implementable because real-time optimization is avoided.
Model predictive control, in general, has several independent tuning parameters: control and prediction horizon, output and input weights, and terminal penalty. However, as better illustrated in the Results section, the main advantage of the adopted choice is the possibility to achieve satisfactory results tuning only one parameter (the output weight q, which is also equal to the weight in the terminal penalty) in a quite straightforward and intuitive way.
With a relatively small increase of the computational burden it is possible to consider both input and state constraints explicitly by solving a constrained linear quadratic optimization problem. In this article, results obtained with constrained linear MPC are not reported because they did not show any significant improvement in our experiments. In fact, the explicit consideration of only input constraints does not improve the performance of the unconstrained saturated control law, whereas the fully constrained problem, i.e., also with state constraints, introduces nontrivial feasibility problems as a consequence of the approximation error caused by linearization and model reduction. Further work is required to explore this issue.
At the cost of a significant increase of the computational burden, a nonlinear MPC approach could be pursued. The main advantages would be the possibility to take into account nonlinear dynamics and a more robust fulfilment of state constraints. Of course, this calls for a well-identified patient model. Some preliminary results are presented in Magni and colleagues.15
Proportional Integral Derivative Control
In order to assess the performance of the proposed control methodology, comparison with a classical PID control law is considered. The control law in the different experiments has been tuned on either the average patient or individually. In the latter case, only the proportional gain Kp has been modified. Moreover, the PID controller incorporates a feed-forward action in order to take advantage of the knowledge of meal amount. The parameters of the PID are Ti = 210 and Td = 40, where Ti and Td are integral and derivative times, respectively. The feed-forward action from the meal signal to insulin rate is given by a transfer function with gain 0.0022. Both the PID and the feed-forward action have been implemented in discrete time with sampling period Ts = 30 minutes. In order to obtain discrete-time implementation, an FOH approximation method was used together with an antiwindup scheme.
Results
Experiment 1: The ingested amount of glucose is exactly as considered in the protocol. One hundred subjects are simulated using an MPC control law synthesized with q = 0.003 for all subjects and the set point is 112 mg/dl.
Experiment 2: The same as experiment 1, but this time q has been tuned for each subject.
Experiment 3: The same as experiment 2, but without meal announcement.
Experiment 4: The ingested amount of glucose is varied randomly within ±40% of the nominal value for all 100 patients. The MPC control law has the same parameters as those used for experiment 2 and relies on the nominal glucose dose to decide the feed-forward action.
Experiment 5: The same as experiment 1, but using a PID control law with Kp = −7.09 × 10−4 for all subjects.
Experiment 6: The same as experiment 5, but the gain of the PID is tuned individually.
Experiment 7: The same as experiment 6, but without meal information.
Experiment 8: The same as experiment 4, using a PID control law with meal information.
Experiment Evaluation
Figure 1 shows the scatter plots of Min_Glycemia vs Max_Glycemia during regulation for experiments 1–8. In particular, the six panels in Figure 1 contain the following comparisons: (A) experiments 1 and 2, (B) experiments 5 and 6, (C) experiments 2 and 3, (D) experiments 6 and 7, (E) experiments 2 and 4, and (F) experiments 6 and 8. Note that the panels in the left column (A, C, and E) refer to MPC, whereas the results of PID control are reported in the right column (B, D, and F). Note that in the scatter plot well-regulated patients should stay close to the lower left corner.
Figure 2 shows the effect of a change of the MPC parameter q on the scatter plot (Min_Glycemia, Max_Glycemia) for experiment 2. More precisely, the individually tuned values of q were all scaled by the constant factor 0.8. The corresponding points, before (full dots) and after (star-circle) scaling, are connected to each other.
In Figure 3, the box plots for LBGI and HBGI are reported for experiments 1–8 during both commutation (“c”) and regulation (“r”) periods.
In Figure 4, the MPC and PID control schemes are compared in subject 36 showing plasma glucose and external insulin evolution. In order to have a global comparison of the relative performance of the two control strategies for all subjects, the scatter plots of Min_Glycemia vs Max_Glycemia in Figure 5 are reported for experiments 2 and 6.
In reference to the regulation period, it is apparent from Figure 1 that both MPC and PID control achieve good regulation performance even if their design is based only on the average patient model. However, in view of significant interindividual variability, their performance is enhanced considerably if the control parameters (MPC parameter q and PID gain, respectively) are tuned individually (see Figures 1A and 1B). The tuning of parameter q is straightforward and intuitive: a reduction of q makes the control action less aggressive, thus using less insulin. This implies an increase of both Min_Glycemia and Max_Glycemia, as shown in Figure 2. The ability of MPC to take advantage of meal announcement is shown in Figure 1C. As seen in Figure 1D, there is performance deterioration when meal information is not considered also with PID control. Imperfect knowledge of the amount of ingested glucose (experiments 4 and 8) causes only a marginal deterioration of performance of the regulator for MPC, whereas a greater deterioration is observed for PID control (Figures 1E and 1F). This difference is also highlighted by the LBGI and HBGI box plots reported in Figure 3. Robustness in the face of meal announcement information is essential because the feed-forward action relies on presumed knowledge of meals that are 4 hours ahead. It is remarkable that MPC achieves satisfactory results even with a 40% meal uncertainty.
As evident from Figure 4, the MPC controller normalizes glycemia very quickly, even if starting from unfavorable initial conditions. The transient of external insulin shows that the insulin flux increase anticipates the meals. This is because of the predictive ability of MPC that computes the current value of the insulin infusion also based on the future values of the meals (see the vector D(k) in the control law14).
Finally, Figure 5 shows that, using the best implementations (e.g., individual tuning), MPC produces a better regulation than PID, limiting glucose oscillation significantly.
Conclusions
The in silico trial has demonstrated that linear output feedback MPC achieves satisfactory glycemic regulation in a population of simulated “type 1 diabetic patients.” The proposed scheme is robust with respect to uncertainty in the meal announcement information. Robustness with respect to sensor errors could be investigated by complementing the simulator with a probabilistic model of sensor noise. Another future research direction concerns the development of nonlinear MPC that could take advantage of the knowledge of the nonlinear dynamics described by the large-scale in silico model.
Abbreviations
- FHOCP
finite horizon optimal control problem
- HBGI
high blood glucose index
- LBGI
low blood glucose index
- MPC
model predictive control
- PID
proportional integral derivative
- T1DM
type 1 diabetes mellitus
References
- 1.Bequette BW. A critical assessment of algorithms and challenges in the development of a closed-loop artificial pancreas. Diabetes Technol Ther. 1998;7:28–47. doi: 10.1089/dia.2005.7.28. [DOI] [PubMed] [Google Scholar]
- 2.Hovorka R. Continuous glucose monitoring and closed-loop systems. Diabet Med. 2005;23:1–12. doi: 10.1111/j.1464-5491.2005.01672.x. [DOI] [PubMed] [Google Scholar]
- 3.Klonoff DC. The artificial pancreas: how sweet engineering will solve bitter problems. J Diabetes Sci Technol. 2007;1(1):72–81. doi: 10.1177/193229680700100112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Mayne DQ, Rawlings JB, Rao CV, Scokaert PO. Constrained model predictive control: stability and optimality. Automatica. 2000;36:789–814. [Google Scholar]
- 5.Garcia CE, Prett DM, Morari M. Model predictive control theory and practice–a survey. Automatica. 1989;25(3):335–348. [Google Scholar]
- 6.Dua P, Doyle FJ, III, Pistikopoulos EN. Model-based blood glucose control for type 1 diabetes via parametric programming. IEEE Trans Biomed Eng. 2006;53:1478–1491. doi: 10.1109/TBME.2006.878075. [DOI] [PubMed] [Google Scholar]
- 7.Parker RS, Doyle FG, 3rd, Peppas NA. A model-based algorithm for blood glucose control in type I diabetic patients. IEEE Trans Biomed Eng. 1999 Feb;46(2):148–157. doi: 10.1109/10.740877. [DOI] [PubMed] [Google Scholar]
- 8.Dalla Man C, Rizza RA, Cobelli C. Meal simulation model of the glucose-insulin system. IEEE Trans Biomed Eng. 2007;54:1740–1749. doi: 10.1109/TBME.2007.893506. [DOI] [PubMed] [Google Scholar]
- 9.Dalla Man C, Camilleri M, Cobelli C. A system model of oral glucose absorption: validation on gold standard data. IEEE Trans Biomed Eng. 2006;53:2472–2478. doi: 10.1109/TBME.2006.883792. [DOI] [PubMed] [Google Scholar]
- 10.Vicini P, Caumo A, Cobelli C. The hot IVGTT two-compartment minimal model: indexes of glucose effectiveness and insulin sensitivity. Am J Physiol. 1997;273:E1024–E1032. doi: 10.1152/ajpendo.1997.273.5.E1024. [DOI] [PubMed] [Google Scholar]
- 11.Breda E, Cavaghan MK, Toffolo G, Polonsky KS, Cobelli C. Oral glucose tolerance test minimal model indexes of beta-cell function and insulin sensitivity. Diabetes. 2001;50:150–158. doi: 10.2337/diabetes.50.1.150. [DOI] [PubMed] [Google Scholar]
- 12.Dalla Man C, Toffolo G, Basu R, Rizza RA, Cobelli C. A model of glucose production during a meal. Proceedings of IEEE-EMBS Conference; 2006; New York. [DOI] [PubMed] [Google Scholar]
- 13.Verdonk CA, Rizza RA, Gerich JE. Effects of plasma glucose concentration on glucose utilization and glucose clearance in normal man. Diabetes. 1981;30:535–537. doi: 10.2337/diab.30.6.535. [DOI] [PubMed] [Google Scholar]
- 14.Kovatchev BP, Clarke WL, Breton M, Brayman K, McCall A. Quantifying temporal glucose variability in diabetes via continuous glucose monitoring: mathematical methods and clinical application. Diabetes Technol Ther. 2005;7:849–862. doi: 10.1089/dia.2005.7.849. [DOI] [PubMed] [Google Scholar]
- 15.Magni L, Raimondo DM, Dalla Man C, De Nicolao G, Kovatchev B, Cobelli C. Model predictive control of glucose concentration in subjects with type 1 diabetes: an in silico trial. World Congress; submitted. [Google Scholar]