Experimental Evaluation of a Recursive Model Identification Technique for Type 1 Diabetes

Abstract

Background

A model-based controller for an artificial β cell requires an accurate model of the glucose–insulin dynamics in type 1 diabetes subjects. To ensure the robustness of the controller for changing conditions (e.g., changes in insulin sensitivity due to illnesses, changes in exercise habits, or changes in stress levels), the model should be able to adapt to the new conditions by means of a recursive parameter estimation technique. Such an adaptive strategy will ensure that the most accurate model is used for the current conditions, and thus the most accurate model predictions are used in model-based control calculations.

Methods

In a retrospective analysis, empirical dynamic autoregressive exogenous input (ARX) models were identified from glucose–insulin data for nine type 1 diabetes subjects in ambulatory conditions. Data sets consisted of continuous (5-minute) glucose concentration measurements obtained from a continuous glucose monitor, basal insulin infusion rates and times and amounts of insulin boluses obtained from the subjects' insulin pumps, and subject-reported estimates of the times and carbohydrate content of meals. Two identification techniques were investigated: nonrecursive, or batch methods, and recursive methods. Batch models were identified from a set of training data, whereas recursively identified models were updated at each sampling instant. Both types of models were used to make predictions of new test data. For the purpose of comparison, model predictions were compared to zero-order hold (ZOH) predictions, which were made by simply holding the current glucose value constant for p steps into the future, where p is the prediction horizon. Thus, the ZOH predictions are model free and provide a base case for the prediction metrics used to quantify the accuracy of the model predictions. In theory, recursive identification techniques are needed only when there are changing conditions in the subject that require model adaptation. Thus, the identification and validation techniques were performed with both “normal” data and data collected during conditions of reduced insulin sensitivity. The latter were achieved by having the subjects self-administer a medication, prednisone, for 3 consecutive days. The recursive models were allowed to adapt to this condition of reduced insulin sensitivity, while the batch models were only identified from normal data.

Results

Data from nine type 1 diabetes subjects in ambulatory conditions were analyzed; six of these subjects also participated in the prednisone portion of the study. For normal test data, the batch ARX models produced 30-, 45-, and 60-minute-ahead predictions that had average root mean square error (RMSE) values of 26, 34, and 40 mg/dl, respectively. For test data characterized by reduced insulin sensitivity, the batch ARX models produced 30-, 60-, and 90-minute-ahead predictions with average RMSE values of 27, 46, and 59 mg/dl, respectively; the recursive ARX models demonstrated similar performance with corresponding values of 27, 45, and 61 mg/dl, respectively. The identified ARX models (batch and recursive) produced more accurate predictions than the model-free ZOH predictions, but only marginally. For test data characterized by reduced insulin sensitivity, RMSE values for the predictions of the batch ARX models were 9, 5, and 5% more accurate than the ZOH predictions for prediction horizons of 30, 60, and 90 minutes, respectively. In terms of RMSE values, the 30-, 60-, and 90-minute predictions of the recursive models were more accurate than the ZOH predictions, by 10, 5, and 2%, respectively.

Conclusion

In this experimental study, the recursively identified ARX models resulted in predictions of test data that were similar, but not superior, to the batch models. Even for the test data characteristic of reduced insulin sensitivity, the batch and recursive models demonstrated similar prediction accuracy. The predictions of the identified ARX models were only marginally more accurate than the model-free ZOH predictions. Given the simplicity of the ARX models and the computational ease with which they are identified, however, even modest improvements may justify the use of these models in a model-based controller for an artificial β cell.

Introduction

Recent publications have considered a variety of empirical, black-box type models identified from type 1 diabetes data. Many of these studies use data obtained from computer simulations of physiological models of glucose–insulin dynamics for type 1 diabetes.^1–3 Other studies have evaluated identification techniques for data obtained from human subjects with type 1 diabetes, both for clinical conditions (largely disturbance free) and for ambulatory conditions. Finan and colleagues² analyzed ambulatory data from two type 1 diabetes subjects in ambulatory conditions, comparing model identification characteristics of these data to simulated versions of data in order to determine the effects of unmeasured, unmodeled disturbances in subject data (e.g., stress and exercise effects on the glucose concentration).

Ståhl and Johansson⁴ performed an in-depth analysis of ambulatory data from one subject. Glucose data werecollected via fingerstick measurements (not using continuousglucose monitoring), however, and thus were sparse. The black-box models they identified from data were autoregressive moving average (ARMA) models, ARMA with exogenous inputs (ARMAX) models, subspace-identified models, general transfer function models, and nonlinear ARMAX models.

Gani and associates⁵ identified autoregressive (AR) models from continuous glucose data from nine type 1 diabetes subjects and quantified the accuracy of short-term (<60-minute) predictions. They used a data-smoothing technique and a regularization technique for the estimation of model parameters.

The publications cited earlier investigated time-invariant or nonrecursive models. It is well known, however, that the glucose–insulin dynamics in type 1 diabetes subjects can change significantly over a range of timescales.^6–8 To ensure satisfactory performance for changing conditions, a model-based controller for an artificial β cell should be equipped with a model adaptation strategy in which model parameters can be updated recursively. Conceptually, this technique will ensure that the most accurate set of model parameters for the current conditions is used in making glucose predictions, which in turn affect the calculated control actions.

Only a few previous studies have investigated recursive identification techniques for type 1 diabetes data. Hovorka and colleagues⁹ used a Bayesian technique to recursively estimate parameters in their physiological model for patients in a clinical setting. Their glucose measurements were obtained from intravenous blood samples but were delayed for 30 minutes to simulate subcutaneous measurements. They performed closed-loopmodel predictive control during fasting conditions based on the predictions from the model.

Sparacino and colleagues¹⁰ recursively estimated parameters for first-order polynomial and AR models based on type 1 diabetes subject data. Data in their study were for ambulatory conditions, but were filtered to remove noise spikes. Eren-Oruklu and associates^11,12 recursively estimated parameters for low-order AR and ARMA models for both in-clinic and ambulatory subject data. The subjects in the study were normal, healthy people and people with type 2 diabetes.

In this article, the modeling techniques were evaluated for unfiltered data collected during ambulatory conditions. Data are representative of both normal conditions and conditions of medicinally induced reductions in insulin sensitivity. This condition of reduced insulin sensitivity is achieved through administration of a medication, prednisone (see the following section). Inclusion of these data is of central importance to the recursive model identification scheme investigated in this article. A fundamental advantage of a recursive estimation technique is its ability to adapt to changing conditions. By using reduced insulin sensitivity data in the analysis, we have effectively created a change in the system (i.e., subject) dynamics, thus providing the recursively identified models a changing condition to which to adapt.

Methods

Autoregressive Exogenous Input (ARX) Models

The models identified in this study were empirical, linear dynamic models known as autoregressive exogenous input models. They are linear difference equation models that describe the relationship between the current output variable and previous values of the output and input variables. For this diabetes application, the ARX model has the form

$$A\left(q^{-1}\right)G\left(t\right)=B_1\left(q^{-1}\right)u_{ins}\left(t\right)+B_2\left(q^{-1}\right)u_{meal}\left(t\right)+\gamma +\epsilon \left(t\right)$$ (1)

where G(t) is the glucose concentration (mg/dl), the output variable; u_ins(t) is the insulin infusion rate (U/h), an input variable; u_meal(t) is the meal amount [grams carbohydrate (CHO)], an input variable; γ is a constant disturbance; ε(t) is white Gaussian noise; and t is the discrete-time sampling instant, i.e., t = 1,2,…

The parameters of the model are the disturbance γ and the coefficients of q⁻¹ in the polynomials A, B₁, and B₂:

$$\begin{array}{l}A\left(q^{-1}\right)=1+a_1{q}^{-1}+a_2{q}^2+\cdots +a_n{q}^{-n}\\ B_X\left(q^{-1}\right)=b_{X,1}{q}^{-1}+b_{X,2}{q}^{-2}+\cdots +b_{X,n}{q}^{-n},X=1,2\end{array}$$ (2)

where q⁻¹ is the backward shift operator, i.e., q⁻¹x(t) = x(t − 1). The model order n corresponds to the number of parameters to be estimated in each polynomial. For this research, third-order models were identified, i.e., n = 3.

An attractive feature of ARX models is that their parameters are estimated readily from a set of training data. In fact, estimates of the model parameters values are obtained analytically from the well-known least-squares solution, which minimizes the sum of the squares of the one-step-ahead prediction errors for training data.^13,14

To identify nonrecursive, or batch, ARX models is to estimate the model parameters from a specific set of training data. Once the parameters are estimated, they are held constant. This batch model can then be used to predict new test data. Conceptually, if the glucose–insulin dynamics have not changed, then the batch models should give predictions for test data approximately as accurate as for training data.

If, however, the glucose–insulin dynamics have changed, then it is appropriate to employ a recursive identification technique that updates the model parameters as new data become available. This recursive technique, then, enables model parameters to adapt to changing conditions.

The parameters for the recursively identified ARX models are updated via a least-squares solution, as in the batch case, but with a variation. In order to adapt more quickly to changing conditions, the parameters can be updated with a weighted least-squares criterion, which places more importance on recent information than on older information. This weighting is accomplished with forgetting factor λ, a number slightly less than one. The forgetting factor can be interpreted as follows: a sample that is τ samples old carries a weight of λ^τ compared to a weight of one for the most recent sample.

A range of values for the forgetting factor was investigated, and it was determined that λ = 0.99 resulted in the most accurate model predictions of future data. Data in this study were sampled at 5-minute intervals; thus, this value of λ implies that a data sample approximately 1 day inthe past is weighted at 5% of the most current data sample.

ARX Model Gains

In general, a model gain is the steady-state sensitivity of the model output to a model input. This gain can be interpreted as the steady-state change in the output resulting from a unit step change in the input. Knowledge of the effect of insulin and meals on glucose concentration indicates two fundamental properties of the gains in a diabetes model: (1) the gain associated with the insulin input should be negative (i.e., an increase in insulin results in a decrease in glucose concentration) and (2) the gain associated with the meal input should bsitive (i.e., a meal results in an increase in glucose concentration). However, depending on the nature of training data, there is no guarantee that the estimated gains will have the correct sign.

The gains of ARX models are calculated as follows. In general terms, the steady-state gain from an input u to an output y is Δy/Δu, where Δy and Δu are the final, steady-state changes in y and u after a step change in u. At steady state,

$$\begin{array}{l}\Delta u=\Delta u\left(t-1\right)=\Delta u\left(t-2\right)=\Delta u\left(t-3\right)\\ \Delta y=\Delta y\left(t\right)=\Delta y\left(t-1\right)=\Delta y\left(t-2\right)=\Delta y\left(t-3\right)\end{array}$$ (3)

Therefore, the insulin-to-glucose and meal-to-glucose steady-state gains for these ARX models are calculated as in Equations (4) and (5), respectively:

$$\frac{\Delta G}{\Delta u_{ins}}=\frac{{\displaystyle {\sum }_{i=1}^3{b}_{1,i}}}{1+{\displaystyle {\sum }_{k=1}^3{a}_k}}$$ (4)

$$\frac{\Delta G}{\Delta u_{meal}}=\frac{{\displaystyle {\sum }_{j=1}^3{b}_{2,i}}}{1+{\displaystyle {\sum }_{k=1}^3{a}_k}}$$ (5)

Prediction Horizons

The prediction horizon for these dynamic models refers to how far into the future they predict. This quantity can be expressed in terms of either samples or time units. For the research presented in this article, the sampling interval for all data was 5 minutes.

In general, there is an inherent trade-off between the length of the prediction horizon and the accuracy of the model predictions. In order to investigate the relationship between the accuracy of model predictions and the length of the prediction horizon, a range of prediction horizons was investigated in this research that may be relevant to model-based control—30 to 90 minutes.

Zero-Order Hold (ZOH) Predictions

Given data and a model prediction for those data, there are several ways to quantify the “goodness” of the model prediction. (Two of these techniques are discussed later.) Regardless of the metric used to quantify the model prediction, this metric may not have much meaning unless it is compared to a lower limit. In this study, predictions are compared to corresponding lower limits, which are metrics obtained from making zero-order hold (ZOH) predictions of data. A ZOH prediction is simply the projection of the current glucose measurement p steps into the future, where p is the prediction horizon.

Figure 1 illustrates the difference between an ARX model prediction and a ZOH prediction. In this simple example, the current sample is t = 2 and the prediction horizon is p = 6 steps. Thus, at sample t = 2, the ARX model attempts to predict the glucose value at t = 8; the ZOH model merely holds the current value constant for six steps. The ARX prediction is closer to the actual glucose value at t = 8. The improvement of the ARX prediction compared to the ZOH prediction for this particular data point is given by (36 – 26)/36 = 28%.

Figure 1.

ARX model prediction and ZOH prediction for one data point. The solid line is subject data, and the prediction horizon is six steps.

Prediction Metrics

An inherent difficulty in quantifying model predictions is selecting one or more suitable metrics that most closely characterize the “goodness” of the prediction. For this research, two types of prediction metrics are used: a relative metric and an absolute metric.

The relative metric used in this article is the FIT value, a statistical metric that quantifies how much of the variability in data is explained by the model prediction¹⁵:

$$FIT=\left(1-\frac{\Vert G-\widehat{G}\Vert }{\Vert G-\overline{G}\Vert }\right)\times 100\%$$ (6)

where G is the vector of measured glucose values, Ĝ is the vector of model predictions, G is a vector whose elements are the mean of the measured glucose values, and the norms are Euclidean. (For example, if vector Z contained three elements, $Z=\left[\begin{array}{lll}{z}_1\hfill & z_2\hfill & z_3\hfill \end{array}\right]$, then $\Vert Z\Vert =\sqrt{z_1^2+z_2^2+z_3^2}$). Thus, FIT = 100% is obtained for a perfect prediction, FIT = 0 can be obtained by predicting the mean of the measured glucose at every sample, and FIT < 0 is obtained for very poor model predictions.

The absolute metric used in this article is the square root of the mean squared prediction error, or root mean squared error (RMSE). This metric is a quantification of the magnitudes of the prediction errors and is thus given in physical units (i.e., mg/dl).

Ambulatory Subject Data

Nine adult type 1 diabetes subjects (six women and three men) participated in the study, which was carried out in 2004 and 2005. Each of the subjects signed an informed and witnessed consent form approved by the Cottage Health Systems Internal Review Board. Subjects were eligible to participate if they had type 1 diabetes without major complications and were using a continuous subcutaneous insulin infusion pump. The subjects were trained in the proper use of a continuous glucose monitoring system (CGMS) device (CGMS®, Medtronic MiniMed, Inc., Northridge, CA) and the OneTouch® UltraSmart® blood glucose meter (LifeScan, Inc., Milpitas, CA), and entered at least four blood glucose meter values per day into the CGMS for calibration purposes.

For each subject, several data sets were collected during normal, ambulatory conditions. Data consisted of continuous (5-minute) glucose measurements obtained from the CGMS, insulin pump records of basal rates and bolus amounts and times, and subject-recorded estimates of the times and CHO content of meals. Each data set spanned 2 to 8 days. For each data set, third-order batch ARX models were identified from the first half of the data set (training data) and were used to predict the second half of the data set (test data). These predictions were compared to ZOH predictions, the designated lower limit for prediction accuracy.

Prednisone Data

In the second portion of the study, six of the nine subjects were administered prednisone (60 mg/day), a steroid medication that induces lowered insulin sensitivity, for 3 consecutive days.⁸ (The three subjects who did not take the prednisone declined to participate in this portion of the study.) For these data, it was postulated that, due to reduced insulin sensitivity, improvements in model prediction accuracy could be obtained using a recursive ARX model identification technique that would allow the model parameters to adapt to the patients' changing glucose–insulin dynamics.

Table 1 lists descriptive statistics for the six subjects who took the prednisone medication for both their normal data and their data collected when on the prednisone. Table 1 indicates that, on average, the prednisone succeeded in lowering the subjects' insulin sensitivity. On average, the subjects' mean daily glucose increased by more than 30 mg/dl (133 to 164 mg/dl) and their daily insulin totals increased by more than 50% from normal (38.0 to 60.7 units), while their daily CHO intake remained roughly the same (160 grams vs 168 grams).

Table 1.

Statistics for Six Subjects Who Participated in the Prednisone Portion of the Study

Subject number	Data type	Average daily glucose(mg/dl)	Average daily insulin totals			Average daily CHO intake(g)
			Basal (U)	Bolus (U)	Total (U)
1	Normal	130	13.2	12.5	25.7	80
	Prednisone	222	13.2	42.5	55.7	94
2	Normal	95	18.6	32.5	51.1	168
	Prednisone	102	21.3	66.4	87.7	165
5	Normal	134	24.4	20.0	44.4	182
	Prednisone	202	30.7	31.6	62.3	219
6	Normal	131	14.6	13.0	27.6	146
	Prednisone	138	19.8	24.0	43.8	156
7	Normal	149	16.3	22.3	38.6	144
	Prednisone	153	23.9	29.3	53.2	154
8	Normal	157	20.9	19.7	40.6	241
	Prednisone	168	30.0	31.2	61.2	222
Mean	Normal	133	18.0	20.0	38.0	160
	Prednisone	164	23.2	37.5	60.7	168

For these data sets, batch ARX models were identified from normal data collected prior to prednisone administration (training data) and were used to predict prednisone data (test data). In addition, ARX models were identified recursively throughout the entire data set and were used to predict prednisone data. Both batch and recursive ARX model predictions were compared to ZOH predictions to characterize model accuracy.

Results and Discussion

Batch ARX Identification for Normal Data

Table 2 shows results for normal subject data in terms of FIT values. Three prediction horizons are shown (30, 45, and 60 minutes) for both training data used to develop the models and independent test data for each subject. The improvement of the ARX model predictions relative to the ZOH predictions was modest, indicating considerable unexplained variability in these ambulatory data. In terms of FIT values, the improvement increased with increasing prediction horizon, with maxima at 39% and 18% for 60-minute predictions for training and test data, respectively. For training data, the ARX model predictions are substantially better than the ZOH predictions, regardless of the prediction horizon. This is an expected result given that the ARX model parameters are estimated so as to optimize the model predictions for these training data. For test data, however, there is no guarantee that the ARX models will perform as well as for training data. Test data in Table 2 show that the degree to which the ARX model predictions were more accurate than the ZOH predictions was highly dependent on the subject. For 60-minute predictions, for example, the improvement of the ARX models compared to the ZOH predictions ranged from –10% for subject 3 (i.e., the ZOH predictions were actually better than the ARX predictions) to 70% for subject 8. This broad range of results indicates a high degree of intersubject variability; data from some subjects are explained more readily by these types of mathematical models than other subjects.

Table 2.

Modeling Results for Batch ARX Models and Normal Data^a

Subject number	Model	Prediction horizon
		30 min		45 min		60 min
		Train	Test	Train	Test	Train	Test
1	Batch	74	56	67	48	61	41
	ZOH	71	56	63	48	56	41
2	Batch	59	57	47	48	38	40
	ZOH	54	57	41	47	30	38
3	Batch	54	49	44	36	38	27
	ZOH	47	50	34	39	23	30
4	Batch	65	68	53	56	43	45
	ZOH	53	62	37	48	25	37
5	Batch	68	54	59	38	51	26
	ZOH	63	51	51	36	41	22
6	Batch	68	63	58	51	50	40
	ZOH	57	59	43	44	32	32
7	Batch	69	61	57	47	48	36
	ZOH	59	55	44	40	32	27
8	Batch	72	62	61	47	50	34
	ZOH	62	52	48	35	36	20
9	Batch	66	54	54	39	44	28
	ZOH	58	50	44	34	31	22
Mean	Batch	66	58	56	45	47	35
	ZOH	58	55	45	41	34	30
ARX improvement over ZOH (%)		13	6	24	11	39	18

^aAverage FIT values (%) for training and test sections for each subject. Improvement values were calculated before rounding the table values.

Table 3 lists modeling results for normal subject data in terms of RMSE values for training and test data and for the same three prediction horizons as in Table 2: 30, 45, and 60 minutes. For training data, the improvement was reasonable, about 20%. However, only approximately half of this improvement was observed for test data, being on average only about 9%. Again the degree to which the ARX models improved upon the ZOH predictions was highly subject dependent. For 60-minute predictions of test data, the ZOH predictions were actually more accurate than the ARX predictions for subject 3 (as was true for the FIT values, also), but for subject 8 the ARX predictions were significantly better (about 17%).

Table 3.

Modeling Results for Batch ARX Models and Normal Data^a

Subject number	Model	Prediction horizon
		30 min		45 min		60 min
		Train	Test	Train	Test	Train	Test
1	Batch	24	32	30	38	36	43
	ZOH	27	32	34	39	41	44
2	Batch	15	16	19	19	22	22
	ZOH	16	16	21	19	25	22
3	Batch	18	19	22	24	24	28
	ZOH	20	19	26	23	29	27
4	Batch	23	30	32	41	38	52
	ZOH	31	35	42	48	51	59
5	Batch	24	30	31	40	36	48
	ZOH	28	32	36	42	44	51
6	Batch	19	24	24	32	29	39
	ZOH	25	26	33	36	40	44
7	Batch	23	26	31	36	38	43
	ZOH	30	31	41	41	50	50
8	Batch	19	25	27	35	35	44
	ZOH	26	32	36	43	44	53
9	Batch	24	28	32	37	39	43
	ZOH	29	30	39	39	48	47
Mean	Batch	21	26	28	34	33	40
	ZOH	26	28	34	37	41	44
ARX improvement over ZOH (%)		19	9	20	9	20	9

^aAverage RMSE values (mg/dl) for training and test sections for each subject. Improvement values were calculated before rounding the table values.

In terms of improvement numbers, the RMSE metric was less sensitive to the prediction horizon than the FIT metric. For RMSE values, the improvement of the ARX models relative to the ZOH predictions was approximately constant over the range of prediction horizons. The improvement numbers for the FIT metric, in contrast, varied from 13 to 39% for training data and 6 to 18% for test data as the prediction horizon varied from 30 to 60 minutes.

One possible explanation for the only modest improvement in prediction accuracy for ARX models compared to ZOH predictions is the presence of key unmeasured and unmodeled disturbances in these ambulatory-type data, notably, exercise and stress levels. These factors can have a significant influence on the glucose concentration. However, because they were unmeasured (and therefore unmodeled), they confounded the identification of the simpler glucose–insulin–meal models identified in this research.

A second factor that can contribute to inaccuracies in the identification of the models is the inaccuracy of data themselves. Continuous glucose sensors are subject to measurement errors, and meal records are simply patient-reported estimates of the times and CHO content of meals. There are most certainly significant errors in these meal records for at least some subjects, but it is difficult to quantify or even classify them.

Recursive ARX Estimation for Prednisone Data

Table 4 summarizes the modeling results for the six type 1 diabetes mellitus subjects who participated in the prednisone part of the study, i.e., the part of thestudy in which lowered insulin sensitivity was induced medicinally. The batch ARX method resulted in predictions as accurate or more accurate than the recursive ARX method for all subjects except one, subject 5. Mean values for all subjects indicate that the batch method and the recursive method produced approximately the same prediction performance, despite the fact that batch models were identified from normal data.

Table 4.

Modeling Results for Conditions of Reduced Insulin Sensitivity^a

Subject number	Model	FIT (%)			RMSE (mg/dl)
		30 min	60 min	90 min	30 min	60 min	90 min
1	Batch	60	42	28	37	54	67
	Recursive	62	43	26	36	53	69
	ZOH	61	44	30	37	52	65
2	Batch	66	45	31	18	29	37
	Recursive	66	37	7	18	34	50
	ZOH	63	39	21	20	33	43
5	Batch	53	27	6	35	55	71
	Recursive	62	45	29	28	42	54
	ZOH	61	43	29	29	43	54
6	Batch	71	50	35	23	39	50
	Recursive	67	46	27	25	42	57
	ZOH	65	40	21	27	46	61
7	Batch	66	41	23	29	51	67
	Recursive	63	33	11	32	58	77
	ZOH	59	31	9	35	60	78
8	Batch	68	37	15	23	45	61
	Recursive	68	38	15	23	45	61
	ZOH	57	24	−1	31	54	72
Mean	Batch	64	40	23	27	46	59
	Recursive	65	40	19	27	45	61
	ZOH	61	37	18	30	48	62
Batch ARX improvement over ZOH (%)		6	9	26	9	5	5
Recursive ARX improvement over ZOH (%)		6	9	5	10	5	2
Recursive ARX improvement over batch ARX (%)		1	0	−17	1	0	−4

^aAverage FIT values (%) and RMSE values (mg/dl) for each subject. Improvement values were calculated before rounding the table values. Subjects 3, 4, and 9 did not participate in the prednisone portion of the study.

On average, both modeling methods produced better predictions than the ZOH method. For 90-minute predictions, improvement of the batch ARX models relative to the ZOH predictions was a significant 26% in terms of FIT values, but was only 5% in terms of RMSE values.

For the shortest prediction horizon of 30 minutes, the recursive models tended to produce slightly better predictions than the batch models. For the midrange prediction horizon of 60 minutes, the two methods produced nearly identical predictions. For the longest prediction horizon of 90 minutes, the batch models were more accurate than the recursive models.

Figure 2 shows 60-minute model predictions for subject 5 for the batch and recursive ARX model. Predictions forthe recursively identified model are clearly more accurate than for the batch model. Figure 3 shows the calculated gains and disturbance term for the recursive model. Correlated, erratic estimates of the model gains are evident near t = 11 h and near the end of the data set.

Figure 2.

60-minute predictions for conditions of reduced insulin sensitivity for subject 5. Solid line: subject data. Dotted line: Batch-identified ARX model (FIT = 27%, RMSE = 55 mg/dl). Dashed line: recursive ARX model (FIT = 45%, RMSE = 42 mg/dl).

Figure 3.

Model gains and the disturbance term for the recursive ARX model for subject 5.

Figure 4 shows 60-minute model predictions for subject 6 for the batch ARX model and the recursive ARX model. Predictions for the recursively identified model are slightly less accurate than for the batch model. Figure 5 shows the corresponding gains and disturbance term for the recursive model. Again, a section of data near t = 21 h shows correlated, erratic estimates of the gains in the model.

Figure 4.

60-minute predictions for conditions of reduced insulin sensitivity for subject 6. Solid line: subject data. Dotted line: Batch-identified ARX model (FIT = 50%, RMSE = 39 mg/dl). Dashed line: recursive ARX model (FIT = 46%, RMSE = 42 mg/dl).

Figure 5.

Model gains and the disturbance term for the recursive ARX model for subject 6.

The erratic estimates of model parameters seen in Figures 3 and 5 may be due to the nature of ambulatory subject data, which at times are deficient of useful information. For instance, a stretch of time with no bolus or meal is deficient of information regarding the effects of these inputs on the glucose concentration. In these cases, it is not uncommon that the recursive least-squares parameter estimator becomes sensitive to new inputs and disturbances and consequently results in inaccurate estimates of model parameters, leading to inaccurate predictions.^16,17

These results for test data suggest that the model adaptation had a minimal effect on the prediction accuracy. Also, reducing forgetting factor λ resulted in highly erratic and inaccurate predictions (results not shown; see Finan¹⁸). Thus, for the changes in insulin sensitivity induced in the subjects in this study, the simple ARX models were unable to improve their prediction accuracy significantly through adaptation.

Again, plausible explanations for these results relate to the nature of ambulatory data and the presence of unmeasured and unmodeled disturbances that greatly confound the accurate identification of the ARX models.

It is also possible that the changes introduced into the subjects were not of a great enough magnitude to warrant reestimation of model parameters. Table 1 lists some quantifications of the induced insulin sensitivity change, but even with these data, it is still difficult to determine if the model should be reidentified.

Conclusions

The predictive accuracy for the batch and recursive ARX identification methods in this article was found to be, on average, very similar. These estimation and prediction methods were investigated for data in which states of reduced insulin sensitivity were induced in six type 1 diabetes subjects in ambulatory conditions.

For normal data (i.e., days without the prednisone medication), predictions of the batch ARX models were significantly better than the ZOH predictions, especially for training data. This result is reasonable because the parameters of the models were estimated so that the model fit for training data was optimized. The improvement of the ARX predictions relative to the ZOH predictions increased with increasing prediction horizon when FIT values were used to quantify the predictions. When RMSE values were used, the improvement was largely independent of prediction horizon.

For data characterized by a decrease in insulin sensitivity, the recursive ARX models demonstrated an insignificant improvement over the batch ARX models for the 30-minute prediction horizon and no improvement for the 60-minute prediction horizon. As the prediction horizon was extended to 90 minutes, however, the batch models resulted in better predictions.

These ambulatory subject data included a number of key unmeasured disturbances, most importantly exercise and stress levels. Although states of reduced insulin sensitivity were induced in subjects via prednisone medication, the effect of the medication was difficult to quantify and therefore was not modeled. Reliable, quantitative measures of stress and exercise levels would likely result in more accurate models and perhaps justify the recursive estimation of model parameters.

Abbreviations

AR

autoregressive

ARMA

autoregressive moving average

ARMAX

ARMA with exogenous inputs

ARX

autoregressive exogenous input

CGMS

continuous glucose monitoring system

CHO

carbohydrate

RMSE

root mean square error

ZOH

zero-order hold