Examining Sensor Agreement in Neural Network Blood Glucose Prediction

Abstract

Successful measurements of interstitial glucose are a key component in providing effective care for patients with diabetes. Recently, there has been significant interest in using neural networks to forecast future glucose values from interstitial measurements collected by continuous glucose monitors (CGMs). While prediction accuracy continues to improve, in this work we investigated the effect of physiological sensor location on neural network blood glucose forecasting. We used clinical data from patients with Type 2 Diabetes who wore blinded FreeStyle Libre Pro CGMs (Abbott) on both their right and left arms continuously for 12 weeks. We trained patient-specific prediction algorithms to test the effect of sensor location on neural network forecasting (N = 13, Female = 6, Male = 7). In 10 of our 13 patients, we found at least one significant (P < .05) increase in forecasting error in algorithms which were tested with data taken from a different location than data which was used for training. These reported results were independent from other noticeable physiological differences between subjects (eg, height, age, weight, blood pressure) and independent from overall variance in the data. From these results we observe that CGM location can play a consequential role in neural network glucose prediction.

Introduction

Diabetes mellitus is one of the major health crises plaguing America. Diabetes mellitus is commonly classified as Type 1 Diabetes (T1D), characterized by autoimmune β-cells destruction, or Type 2 diabetes (T2D) which is characterized by initial insulin resistance followed by β-cell failure. ¹ This β-cell failure results in insufficient insulin secretion and consequent hyperglycemia. Physiological complications of prolonged hyperglycemia can be devastating, leading to both macrovascular and microvascular complications. ²

For patients with diabetes, accurate and precise monitoring of the glucose profile plays a vital role in maintaining health and minimizing complications. For many years, patients with diabetes self-monitored their blood glucose (SMBG) through multiple daily fingersticks. SMBG has been important in adjusting therapy, particularly if the patient’s treatment program included insulin.^1-4

Although glucose monitoring by SMBG has been the traditional means for tracking a patient’s glycemic profile, advances in the field have led to the increased use of interstitial continuous glucose monitors (CGMs) for more detailed and nuanced characterization of a patient’s glycemic profile. With traditional SMBG, glucose measurements are taken multiple times per day (typically 4-7) which incompletely captures glycemic trends such as overnight or postprandial levels.^5,6 In contrast, CGM devices utilize a small sensor inserted just under the skin which captures interstitial glucose levels every 5-15 minutes for up to 14 days, depending on the device. ⁷ The benefit of using CGM to capture the glycemic profile is the ability to record 10 to 20 times the number of glycemic measurements relative to SMBG. Typically used in conjunction with insulin pumps or multiple daily shots of insulin, CGM improves glycemic control, lowers glycemic variability and improves quality of life. ⁸ In 2016, the Food and Drug Administration (FDA) approved CGM for non-adjunctive use, which means CGM can fully replace SMBG for treatment decisions.^9-11

Despite the approval of CGM to replace SMBG, concerns about CGM remain, particularly with regards to accuracy. These accuracy issues are more apparent during hypoglycemia or hyperglycemia.^12-15 Multiple factors contribute to CGM accuracy in measuring glucose levels. ¹⁶ These factors include device-tissue forces and pressures, relative device-tissue motion (including micromotion), tissue type, implantation site, patient physiology, sensor calibration values and variability in insulin action.^14,17-21 A number of solutions have been successfully implemented including introduction of sensor delays, various de-noising strategies and calibration with fingerstick data.^14,22,23

Moving beyond just measuring glucose levels, there remains considerable interest in mathematical forecasting of glucose levels using CGM data. Many different methods have been implemented, including autoregressive models, latent variable models, and deterministic physiological models.^24-27 Recently, there has been significant investigation into neural network-based predictive methods. In particular, recent work by Li, et al. has shown that neural networks can effectively forecast glucose behavior 30 minutes into the future.^28-32 Improved automated glucose prediction is of interest since it enables treatment decisions to be more precise and patient specific. Furthermore, modern reduction in computational cost makes neural network glucose prediction more accessible to the average patient because machine learning can now be implemented by low-cost computers.

However, prediction algorithms still depend on sensor accuracy and do not account for sensor location in their design. In particular, one recent study by Liu, et al. demonstrates significant differences between left-arm and right-arm placed CGMs in their participants. ³³ While previous studies have only implemented multiple sensor locations as redundancy, ³⁴ we examine CGM sensor agreement between 2 different locations recommended by the manufacturer. Since an accurate, precise prediction is an important aspect of therapy, understanding the relevance of CGM placement is of paramount importance.^35,36 In response, we examined the neural network (NN) performance of patient specific prediction algorithms trained on the right arm versus algorithms trained on the left arm. We hypothesize that neural networks trained on sensors from different physiological locations will increase blood glucose prediction error (P < .05).

Data Acquisition

Study Characteristics

The data used in this paper were collected from patients participating in the “Role of CGMS Usage in Predicting Risk of Hypoglycemia” study at the University of Minnesota (NCT03481530). The University of Minnesota’s Institutional Review Board approved the protocol (ID: STUDY00002113). All participants provided written informed consent before participation.

Data were collected from 13 subjects with T2D who were not actively using CGM to monitor their glucose profile. Each subject wore 2 blinded FreeStyle Libre Pro CGM systems (Abbott, Alameda, CA), one on the back of each arm for 12 weeks. The CGMs were replaced every 2 weeks. Subjects were instructed to continue with their usual diabetes treatment program, diet and SMBG monitoring. The subjects could not view their CGM results during the 12 week intervention and received a summary of their CGM results only at study conclusion (“blinded” study). These sensors are approved for non-adjunctive use by the FDA, meaning they do not have to be augmented by fingerstick blood glucose measurements when providing treatment. ³⁷ The FreeStyle Libre Pro system is a 14-day continuous monitoring system with a 15-minute sampling rate which is intended to be used as a standalone clinical diabetes monitoring system.^38,39 In this system, glucose is measured from the interstitial fluid through a single sensor which is replaced every 2 weeks, and clinicians accessed raw data by means of a wireless data acquisition unit. We utilized this system because it provides clinically accurate readings without the need for fingerstick calibration.

Methods and Materials

Problem Formulation and Neural Network Criteria

Drawing from current blood glucose neural network predictors found in literature, we determined specific aims used to provide an algorithmic framework for examining the effect of sensor location.

• 1) Patient Specific Prediction—Individually trained algorithms help isolate whether variability occurs from location or from inter-patient differences.

• 2) Clinically Accurate—Prediction error should reflect values which are clinically acceptable.

• 3) Multiple Prediction Horizons—Since previous works have demonstrated the effects of prediction horizon on prediction accuracy, we chose to examine 15, 30, 45, and 60 minute prediction horizons for a robust analysis.^29,30 Note that prediction horizon refers to the forecasting time period.

• 4) Overlapping Time Periods—In order to rule out other potential sources of predictor differences, we utilized data taken from days in which both glucose sensors were functioning properly.

The purpose of these criteria was to reduce variability within the baseline predictor. In other words, these criteria were used to isolate sensor location as the testable variable. For each patient, NNs were used to formulate 4 glucose algorithms: 2 for predicting future values from right arm sensor data and 2 from left arm sensor data. For each arm, one NN was built using training data from the same arm and one was built using data from the opposite arm. The root mean square error (RMSE) was compared between the 2 predictions to determine whether the algorithm would produce a repeatable forecast. For instance, an algorithm that was right arm trained-right arm tested (right-right) was compared to an algorithm that was left arm trained-right arm tested (left-right). Similarly for the left arm, the left-left algorithm was compared to the right-left algorithm. This process is highlighted in Figure 1.

Figure 1.

Our procedure for testing consisted of constructing 4 predictors per patient to determine whether sensor location influences predictor performance. 5×2 cross validation (5×2 CV) methodology is presented in Methods and Materials: 5x2 CV Method.

Algorithm Approach

For the purposes of this analysis, a simple neural network was implemented. In order to predict future glucose values, our NN utilized 3 previous glucose values (ie, 45 minutes of data) to perform a prediction for a future glucose concentration. In other words, since the FreeStyle Libre Pro measures glucose every 15 minutes, the 3 previous values cover the preceding 45 minutes of data.

To achieve this, data were first preprocessed and organized into a usable set. Erroneous readings were eliminated, and a portion of the data was removed for final testing. Our NNs were all trained to be patient specific. An overview of the patient data sets is reported in Table 1. The RMSE is a highly used, well-accepted method for examining the performance of blood glucose predictors. There are numerous examples of this metric in literature includingwork by Mougiakakou et al., Perez-Gandia et al., Robertson et al., Pappada et al., and Zecchin et al. among others.^29,31,40-42 We trained networks using the 5×2 cross-validation (5×2 CV) method in order to provide a statistical framework for comparison.

Table 1.

Patient Characteristics of Our 13 Patient Set.

Patient characteristics
Gender		Height (cm)
Female	6	Mean ± SD	170.6 ± 9.7
Male	7	Range	156-184
Age		Weight (kg)
Mean ± SD	60.4 ± 12	Mean ± SD	99.7 ± 16.6
Range	45-79	Range	73-122
HbA1c (%)		Left sensor observations per patient
Mean ± SD	7.5 ± 1.2	Mean ± SD	8070 ± 3456
Range	5.8-10.2	Range	2328-12553
Fructosamine (mmol/L)		Right sensor observations per patient
Mean ± SD	308.2 ± 42.2	Mean ± SD	7465 ± 3573
Range	263-425	Range	1923-12723

Preprocessing

Patient data were first organized for the 5×2 CV method, and erroneous (e.g., non-overlapping data time periods) were removed. Neural network inputs x (with bias layer $x_0=1$ ) were built from glucose time series by taking 3 lagged data points and using the data point at time t as the reference output r. Additionally, data were normalized to prevent saturation of the layers.

$$\begin{array}{c}\mathit{x}=\left\{x_0,x_{t-1},x_{t-2},x_{t-3}\right\}\end{array}$$ (1)

$$\begin{array}{c}\mathit{r}=\left\{x_t\right\}\end{array}$$ (2)

Neural Network Architecture

To build the neural network predictor, a standard backpropagation approach for nonlinear regression multilayer perceptrons was used; ⁴³ our networks used 3 hidden layers. This approach was used for each NN. The error function used to determine loss during backpropagation was one half the sum of the squared Euclidean distances across the dataset (equation (3)) where N represents the number of observations (as presented by Alpaydin ⁴³ ).

$$\begin{array}{c}E=\frac{1}{2}\sum _{t=1}^N{\left(r^t-y^t\right)}^2\end{array}$$ (3)

Algorithm Evaluation

To evaluate the overall algorithm performance, 2 metrics commonly used in glucose prediction were considered. The root mean square error (RMSE) and mean absolute relative difference (MARD) were chosen to provide a multi-faceted approach to understanding algorithm performance. In particular, the RMSE was used to statistically compare our algorithms to each other as well as provide an insight into overall model performance while MARD was used as an additional tool to understand general predictor performance.

$$\begin{array}{c}{E}_{RMSE}=\frac{1}{5}\sum _{k=1}^5\sqrt{\frac{1}{N^k}\sum _t^{N^k}{\left(r^t-y^t\right)}^2}\end{array}$$ (4)

$$\begin{array}{c}{E}_{MARD}=\frac{1}{5}\sum _{k=1}^5\frac{1}{N^k}\sum _t^{N^k}\frac{\left|y^t-r^t\right|}{y^t}\end{array}$$ (5)

Note that these metrics were averaged due to the use of the 5×2 cross-validation method presented in the following sections.

5×2 CV Method

5×2 Cross-validation is a well understood method for machine learning comparison in which 5 replications of two-fold cross-validation are performed. ⁴⁴ To compare predictor error rates, we utilized the combined 5×2 CV F test introduced by Alpaydin. ⁴⁵ As presented in Section Methods and Materials: Problem Formulation and Neural Network Criteria, our process compared an algorithm trained on one sensor to an algorithm trained on a sensor from a separate physiological location. The process for performing our algorithm comparisons for each patient dataset is as follows:

• 1) Random data split—Data observed from one sensor were randomized with a repeatable random number generator and split equally into training and validation sets.

• 2) Train and validate—Blood glucose predictors were trained with the training set, then performance was tested on the validation set.

• 3) Cross-validate—The algorithm was re-trained and re-validated using the opposite data sets from the equal-sized random split (the validation set becomes the training set and vice versa).

• 4) Re-split and repeat cross-validation—The data were randomized and split before repeating the cross-validation process. This was performed 5 times in total to complete the 5×2 CV process.

• 5) Repeat the 5×2 CV process for the opposite sensor—This process was repeated for data observed from the opposite sensor. These steps resulted in algorithms tested with data observed from the same sensor from which it was trained (eg, right arm trained-right arm tested).

• 6) Test each predictor with data from the opposite sensor—For each predictor in the 5×2 CV process, the trained NN was also tested on validation data from the opposite sensor (eg, right arm trained-left arm tested). In this manner, we exposed our algorithms to data from the opposite sensor while building a robust framework for statistical comparison.

• 7) Collect error estimates and compare performance via the F test—The RMSE for each predictor was collected for use with the F test.

The F test introduced by Alpaydin is more well behaved than the paired t test introduced by Dietterich. ⁴⁵ For our purposes, we used the RMSE as the comparison metric in the F test.

Results

General Predictor Performance

The training data were comprised of observations from patients using the FreeStyle Libre Pro system over extended periods of time. While the quantity of usable data varied by patient, every training data set had at least 20 days of continuous sampling. For the majority of patients, we had over 50 days of glucose data per sensor (approximately 5000 observations per sensor). We observed no noticeable difference in predictor performance relative to training set size. Through the 5×2 CV method we trained and tested hundreds of predictors in order to ensure a robust comparison. Figure 2 is an example of the prediction algorithm forecasting across multiple prediction horizons.

Figure 2.

Our algorithm was able to track closely with predicted values for a variety of prediction horizons. In these instances, we present one-day examples of glucose forecasts which yielded low RMSE and MARD.The left sensor graph is data from October 11, 2019 while the right sensor graph is taken from November 6, 2019.

The predictors we built were first evaluated for general performance by examining $E_{RMSE}$ and $E_{MARD}$ . For the 5×2 CV method, the mean and standard deviation values for RMSE and MARD (n = 10 per Train-Test pair) are presented in Tables 2 and 3. For each set of predictors, we report the average and standard deviation of the metric. We observe that for both RMSE and MARD we experience low variance between the metrics.

Table 2.

The RMSE Values for Predictors are Presented Below Where Significant Differences (P < .05) in RMSE are Denoted by †. Datasets with Significant Differences (P < .05) in Variance are Denoted by *. These Results are Presented Graphically in Figure 3.

Prediction horizon (min)	Algorithm setup (trained-tested)	RMSE (mg/dL) patient 1	RMSE (mg/dL) patient 2	RMSE (mg/dL) patient 3	RMSE (mg/dL) patient 4	RMSE (mg/dL) patient 5	RMSE (mg/dL) patient 6	RMSE (mg/dL) patient 7	RMSE (mg/dL) patient 8	RMSE (mg/dL) patient 9	RMSE (mg/dL) patient 10	RMSE (mg/dL) patient 11	RMSE (mg/dL) patient 12	RMSE (mg/dL) patient 13
15	Left-Left	11.66 ± 0.16	9.99 ± 0.18	10.4 ± 0.06	7.8 ± 0.15	8.91 ± 0.14	9.11 ± 0.39	11.07 ± 0.35	7.67 ± 0.41	8.25 ± 0.46	5.47 ± 0.29	7.93 ± 0.33	8.39 ± 0.15	9.03 ± 0.56
	Right-Left	14.35 ± 0.11†	34.24 ± 1.12*†	61.31 ± 2.25*†	7.97 ± 0.16*	8.95 ± 0.14	9.08 ± 0.46	11.1 ± 0.36	7.77 ± 0.41†	9.3 ± 0.43†	9.19 ± 0.18*†	11.62 ± 0.25†	8.33 ± 0.14*	9.6 ± 0.48†
	Right-Right	8.58 ± 0.06	7.32 ± 0.37	7.05 ± 0.1	8.01 ± 0.25	9.09 ± 0.09	8.73 ± 0.12	11.04 ± 0.41	7.51 ± 0.59	8.73 ± 0.15	10.05 ± 0.19	10.76 ± 0.17	8.24 ± 0.17	9.58 ± 0.08
	Left-Right	10.59 ± 0.16†	10.73 ± 0.27*†	11.15 ± 0.16*†	7.83 ± 0.29*	9.11 ± 0.12	8.62 ± 0.26*	10.98 ± 0.39	7.41 ± 0.59†	10.53 ± 0.24†	60.91 ± 2.05*†	26.18 ± 0.61*†	8.32 ± 0.16*†	9.12 ± 0.16
30	Left-Left	18.45 ± 0.34	13.39 ± 0.3	14.14 ± 0.07	15.36 ± 0.32	18.59 ± 0.36	17.32 ± 0.39	21.63 ± 0.28	14.12 ± 0.36	15.28 ± 0.31	11.15 ± 0.2	15.67 ± 0.33	18.61 ± 0.23	18.02 ± 0.45
	Right-Left	19.68 ± 0.33†	23.19 ± 0.92*†	30.64 ± 0.58*†	15.57 ± 0.28*	18.61 ± 0.31	17.22 ± 0.38	21.64 ± 0.3	14.18 ± 0.37	15.52 ± 0.35	12.61 ± 0.22*†	16.86 ± 0.25†	18.58 ± 0.23*	18.15 ± 0.44†
	Right-Right	18.79 ± 0.14	14.77 ± 0.36	15.2 ± 0.21	15.73 ± 0.15	19.14 ± 0.13	16.89 ± 0.51	21.58 ± 0.41	13.79 ± 0.3	13.54 ± 0.19	11.53 ± 0.11	15.12 ± 0.18	17.53 ± 0.14	19.42 ± 0.31
	Left-Right	19.22 ± 0.2†	15.66 ± 0.34*†	16.51 ± 0.21*†	15.61 ± 0.24*	19.13 ± 0.15	16.84 ± 0.4*	21.55 ± 0.41	13.73 ± 0.28	14.13 ± 0.3	31.21 ± 2.03*†	22.29 ± 0.7*†	17.61 ± 0.14*†	19.37 ± 0.34
45	Left-Left	25.11 ± 0.49	20.17 ± 0.32	21.48 ± 0.2	21.87 ± 0.56	27.26 ± 0.53	24.08 ± 0.74	29.18 ± 0.19	18.89 ± 0.51	20.91 ± 0.29	15.51 ± 0.19	21.98 ± 0.49	27.1 ± 0.3	24.74 ± 0.17
	Right-Left	26.37 ± 0.51†	40.47 ± 1.08*†	62.37 ± 1.87*†	22.31 ± 0.46*	27.29 ± 0.5	23.99 ± 0.81	29.21 ± 0.21	18.94 ± 0.47	21.47 ± 0.27†	18.03 ± 0.21*†	23.9 ± 0.37†	27.09 ± 0.33	24.92 ± 0.18
	Right-Right	25.69 ± 0.34	20.95 ± 0.37	21.83 ± 0.28	22.18 ± 0.27	28.05 ± 0.5	23.81 ± 0.72	29.17 ± 0.26	18.4 ± 0.3	19.3 ± 0.22	17.78 ± 0.17	21.98 ± 0.31	25.19 ± 0.27	26.91 ± 0.64
	Left-Right	26.17 ± 0.31†	22.66 ± 0.29*†	23.99 ± 0.35*†	22.02 ± 0.36*	28.02 ± 0.46	23.76 ± 0.72*	29.12 ± 0.25	18.35 ± 0.31	20.48 ± 0.41†	59.77 ± 2.51*†	34.54 ± 0.85*†	25.37 ± 0.26*†	26.84 ± 0.65
60	Left-Left	28.26 ± 0.41	22.64 ± 0.4	24.47 ± 0.31	27.33 ± 0.53	34.5 ± 0.46	29.34 ± 0.72	35.3 ± 0.49	22.8 ± 0.51	25.87 ± 0.11	18.47 ± 0.15	26.86 ± 0.44	33.45 ± 0.45	29.64 ± 0.17
	Right-Left	28.68 ± 0.44†	30.1 ± 1.03*†	33.19 ± 0.48*†	27.88 ± 0.51*†	34.55 ± 0.43	29.2 ± 0.7	35.32 ± 0.49	22.81 ± 0.38	26.2 ± 0.21	19.38 ± 0.25*†	27.48 ± 0.41†	33.51 ± 0.47	29.89 ± 0.2†
	Right-Right	29.67 ± 0.33	25.87 ± 0.37	26.91 ± 0.33	27.59 ± 0.43	35.29 ± 0.81	29.07 ± 0.69	35.19 ± 0.35	22.18 ± 0.31	23.58 ± 0.28	18.27 ± 0.16	25.22 ± 0.38	31.15 ± 0.36	32.22 ± 0.35
	Left-Right	29.76 ± 0.32	26.36 ± 0.33*†	27.66 ± 0.35*†	27.41 ± 0.46*	35.26 ± 0.82	29.1 ± 0.62*	35.14 ± 0.33	22.12 ± 0.44	24.2 ± 0.31†	29.34 ± 1.5*†	29.11 ± 0.31*†	31.41 ± 0.35*†	32.21 ± 0.34

Table 3.

For Our Purposes, We Did Not Use the MARD to Perform Algorithm Comparisons. However, the Metric is Useful in Evaluating Overall Predictor Performance and is Presented Below.

Prediction horizon (min)	Algorithm Setup (trained-tested)	MARD (%) patient 1	MARD (%) patient 2	MARD (%) patient 3	MARD (%) patient 4	MARD (%) patient 5	MARD (%) patient 6	MARD (%) patient 7	MARD (%) patient 8	MARD (%) patient 9	MARD (%) patient 10	MARD (%) patient 11	MARD (%) patient 12	MARD (%) patient 13
15	Left-Left	7.1 ± 0.1	6.09 ± 0.16	5.41 ± 0.11	3.31 ± 0.11	2.8 ± 0.07	2.98 ± 0.1	3.56 ± 0.04	3.54 ± 0.09	2.73 ± 0.06	3.13 ± 0.05	2.68 ± 0.08	4.08 ± 0.06	4.45 ± 0.07
	Right-Left	7.73 ± 0.08	15.16±0.59	15.96±0.41	3.33±0.09	2.74±0.06	2.9±0.06	3.56±0.05	3.76±0.1	3.18±0.06	5.44±0.07	3.96±0.06	4.02±0.05	4.69±0.06
	Right-Right	5.34±0.08	4.29±0.1	3.89±0.05	3.87±0.07	2.81±0.05	3.07±0.11	3.53±0.1	3.46±0.13	3.17±0.03	6.39±0.12	3.55±0.06	4.83±0.06	5.6±0.19
	Left-Right	6.51±0.09	6.25±0.09	6.03±0.08	3.86±0.11	2.88 ± 0.05	3.12 ± 0.12	3.52 ± 0.08	3.3 ± 0.09	3.49 ± 0.06	31.76 ± 1.15	6.58 ± 0.14	4.94 ± 0.05	5.31 ± 0.19
30	Left-Left	11.49 ± 0.22	8.03 ± 0.25	7.42 ± 0.06	6.77 ± 0.23	5.84 ± 0.18	5.95 ± 0.13	7.12 ± 0.07	6.99 ± 0.09	5.34 ± 0.07	6.8 ± 0.08	5.3 ± 0.12	8.94 ± 0.11	9.45 ± 0.16
	Right-Left	12.36 ± 0.24	13.16 ± 0.41	13.08 ± 0.18	6.82 ± 0.18	5.79 ± 0.16	5.87 ± 0.14	7.12 ± 0.06	7.17 ± 0.11	5.41 ± 0.08	7.56 ± 0.09	5.69 ± 0.11	8.72 ± 0.11	9.53 ± 0.17
	Right-Right	11.8 ± 0.12	8.64 ± 0.13	8.6 ± 0.12	7.57 ± 0.1	5.91 ± 0.07	6.13 ± 0.19	7.04 ± 0.13	6.73 ± 0.16	4.83 ± 0.04	7.31 ± 0.13	4.8 ± 0.03	9.92 ± 0.19	11.4 ± 0.2
	Left-Right	12 ± 0.15	9.08 ± 0.16	9.01 ± 0.16	7.6 ± 0.15	5.96 ± 0.06	6.2 ± 0.16	7.03 ± 0.12	6.6 ± 0.12	5.16 ± 0.11	18.17 ± 1.25	7.02 ± 0.18	10.41 ± 0.06	11.31 ± 0.24
45	Left-Left	15.97 ± 0.27	12.71±0.18	11.7±0.12	9.88±0.3	8.59±0.24	8.5±0.24	9.57±0.04	9.64±0.12	7.45±0.13	9.68±0.09	7.41±0.12	13.34±0.17	13.35±0.17
	Right-Left	17.04±0.35	22.5±0.56	21.89±0.62	9.95±0.29	8.55 ± 0.25	8.39 ± 0.26	9.6 ± 0.04	9.8 ± 0.12	7.57 ± 0.1	11.1 ± 0.1	8.01 ± 0.09	12.96 ± 0.16	13.51 ± 0.12
	Right-Right	16.72 ± 0.27	12.46 ± 0.13	12.59 ± 0.17	10.76 ± 0.2	8.68 ± 0.19	8.81 ± 0.23	9.51 ± 0.1	9.19 ± 0.17	6.93 ± 0.02	11.65 ± 0.14	7.02 ± 0.08	14.45 ± 0.25	16.18 ± 0.24
	Left-Right	16.89 ± 0.23	13.47±0.21	13.28±0.24	10.92±0.23	8.73±0.18	8.89±0.26	9.48±0.09	9.09±0.16	7.5±0.14	35.6±1.07	10.65±0.24	15.47±0.13	16.02±0.32
60	Left-Left	18.41±0.24	14.43±0.22	13.65±0.16	12.75±0.3	10.99±0.2	10.55±0.19	11.63±0.07	11.87±0.15	9.34±0.07	11.77±0.12	9.17±0.08	17.05±0.25	16.33±0.15
	Right-Left	18.99±0.25	19.27±0.66	17.76±0.27	12.8±0.33	10.96 ± 0.25	10.41 ± 0.18	11.66 ± 0.08	11.97 ± 0.15	9.36 ± 0.08	12.08 ± 0.13	9.33 ± 0.06	16.54 ± 0.25	16.58 ± 0.16
	Right-Right	19.96 ± 0.36	15.67 ± 0.21	15.84 ± 0.15	13.61 ± 0.22	11.09 ± 0.2	10.94 ± 0.26	11.52 ± 0.12	11.24 ± 0.18	8.43 ± 0.03	11.96 ± 0.18	8.04 ± 0.06	18.36 ± 0.35	19.62 ± 0.09
	Left-Right	19.82 ± 0.31	15.91 ± 0.24	15.65 ± 0.18	13.95 ± 0.3	11.12 ± 0.25	11.09 ± 0.34	11.5 ± 0.1	11.19 ± 0.27	8.86 ± 0.13	18.6 ± 1.01	9.64 ± 0.08	19.86 ± 0.21	19.43 ± 0.18

Predictor Performance Based on Physiological Location

Recall that the algorithms being compared are the 2 predictors which are tested on data observed from the same sensor. Thus, for each patient we were concerned with comparing Left-Left versus Right-Left and comparing Right-Right versus Left-Right.

For 10 of the 13 participants with T2D, we observed at least one significant difference in predictor behavior based on the location of the CGM (P < .05). These results are reported in Table 2 and graphically in Figure 3.

Figure 3.

The results of the predictor comparisons show that there were a number of instances where there were differences in performance due to a change in sensor location. Consider the 2 comparisons of predictors: Left-Left versus Right-Left and Right-Right versus Left-Right. Mean and standard deviations are reported in Table 2. Significant instances (P < .05) are denoted by an asterisk (*) of the same color listed in the legend.

We also considered whether differences in variance between training and test data sets may be a contributing factor. We observed 12 data sets with significant (P < .05) variance out of the 26 sets being compared. We observed no differences in RMSE behavior which corresponded to differences in variance.

Finally, among patient predictors with significant error rate differences, we considered whether other physiological factors were associated with increased error rates. For the purposes of our analysis, we considered patients which had more than one instance of a different error rate versus the remaining patients.

We observed that baseline differences in height, age, weight, risk score for hypoglycemia over 5 years, ⁴⁶ blood pressure, fructosamine levels, Hemoglobin A1c (HbA1c) and heart rate of patients with significant increases in RMSE were not significantly different than patients without significant RMSE (P < .05).

Discussion

In terms of clinical implications, we made the novel observation that CGM location (right vs. left) can play a critical role in constructing NN based prediction models. Our predictor comparisons demonstrate that algorithm error rates can vary significantly due to the physiological location of the sensor regardless of significant differences in variation between training and test data sets. We believe that our algorithm design choices, particularly the range of prediction horizons and patient specific features, effectively isolated physiological location as a testable variable. Importantly, we observed that very significant differences in RMSE occurred in cases with no significant difference in training versus testing set variation. With no significant physiological differences between patient groups (e.g., age, weight, fructosamine levels), we conclude that physiological sensor location should be considered and accounted for in glucose prediction analysis. Further investigation is needed.

There are many possibilities for future investigation. Examples include mitigation techniques for the differences due to sensor location, persistence of these findings in alternative populations (ie, T1D, healthy people without diabetes) and utilization of larger data sets.

This study has several strengths. We enrolled a population of patients with T2D, varying by 5-year risk for hypoglycemia, and had them wear 2 blinded CGM simultaneously (one on each arm) continuously for 12 weeks. Therefore, this analysis used a rich dataset allowing for direct comparison between the 2 sensors. Our simple, uncomplicated algorithm design allowed us to make direct comparisons of location effects. This also allowed us to train and test hundreds of neural networks with low computational cost. Furthermore, we were able to utilize data collected under real-world conditions whereas other studies often must rely on synthetic data. We also acknowledge several weaknesses. We exclusively used the Freestyle Libre Pro CGM and placed it on the back of the right/left arm, as per the manufacturer’s instructions. Whether these results are generalizable to other CGM systems or other body locations (not recommended by the manufacturer) remains unknown. While training patient specific predictors was necessary to test the effect of sensor location, each predictor was trained on a subset of our total available data. Larger training sets will be leveraged in future work in order to improve robustness to variance. We also acknowledge that our algorithms did not include methods of de-noising and preprocessing. These will be investigated in future work.

Conclusion

Patients with diabetes who use CGM may consider location effects if their CGM results are more variable/unexpected than previous values; perhaps these patients may be better served by consistently wearing the CGM on one side. In terms of glucose predictor design, we believe that sensor location should be considered when evaluating an algorithm for variance robustness. In several notable comparisons, we observed that significant increases in RMSE did not correspond with significant differences in variance between the train-test sets. Results in this study may be used to motivate additional analyses of glucose predictor robustness.