Predicting and Preventing Nocturnal Hypoglycemia in Type 1 Diabetes Using Big Data Analytics and Decision Theoretic Analysis

Abstract

Background: Despite new glucose sensing technologies, nocturnal hypoglycemia is still a problem for people with type 1 diabetes (T1D) as symptoms and sensor alarms may not be detected while sleeping. Accurately predicting nocturnal hypoglycemia before sleep may help minimize nighttime hypoglycemia.

Methods: A support vector regression (SVR) model was trained to predict, before bedtime, the overnight minimum glucose and overnight nocturnal hypoglycemia for people with T1D. The algorithm was trained on continuous glucose measurements and insulin data collected from 124 people (22,804 valid nights of data) with T1D. The minimum glucose threshold for announcing nocturnal hypoglycemia risk was derived by applying a decision theoretic criterion to maximize expected net benefit. Accuracy was evaluated on a validation set from 10 people with T1D during a 4-week trial under free-living sensor-augmented insulin-pump therapy. The primary outcome measures were sensitivity and specificity of prediction, the correlation between predicted and actual minimum nocturnal glucose, and root-mean-square error. The impact of using the algorithm to prevent nocturnal hypoglycemia is shown in-silico.

Results: The algorithm predicted 94.1% of nocturnal hypoglycemia events (<3.9 mmol/L, 95% confidence interval [CI], 71.3–99.9) with an area under the receiver operating characteristic curve of 0.86 (95% CI, 0.75–0.98). Correlation between actual and predicted minimum glucose was high (R = 0.71, P < 0.001). In-silico simulations showed that the algorithm could reduce nocturnal hypoglycemia by 77.0% (P = 0.006) without impacting time in target range (3.9–10 mmol/L).

Conclusion: An SVR model trained on a big data set and optimized using decision theoretic criterion can accurately predict at bedtime if overnight nocturnal hypoglycemia will occur and may help reduce nocturnal hypoglycemia.

Introduction

People with type 1 diabetes (T1D) require lifelong exogenous insulin treatment to maintain adequate blood glucose control; however, intensive glucose control increases the risk of level 2 hypoglycemic episodes (<3 mmol/L) with potentially fatal consequences.¹ Despite the advances in insulin delivery technologies, including sensor-augmented insulin-pumps and artificial pancreas systems, hypoglycemia still constitutes a barrier to achieving optimal glycemic control.^2–6 Fear of hypoglycemia can lead to poor glucose management decisions such as insulin underdosing or extra intake of carbohydrates.⁷ Nocturnal hypoglycemia, which accounts for 55% of level 2 hypoglycemia events^8–10 in patients with T1D and 75% of level 2 events in children,¹¹ is particularly risky because patients are unlikely to recognize symptoms while sleeping or awaken in response to hypoglycemia alarms from continuous glucose monitors.^12,13 In addition to the serious short-term effects of nocturnal hypoglycemia episodes, untreated nocturnal hypoglycemia can further impair the physiological counter-regulatory system and contribute to hypoglycemia unawareness,^14–17 which may eventually result in recurrent asymptomatic hypoglycemia¹⁸ and in some cases the dead in bed syndrome due to prolonged exposure to extremely low blood glucose levels during sleep.¹⁹

Prediction of overnight nocturnal hypoglycemia that can be estimated at bedtime has been less studied than short-term nocturnal hypoglycemia prediction in patients with T1D. Algorithms for short-term hypoglycemia prediction have been used in single- and dual-hormone automated insulin delivery systems (i.e., artificial pancreas) and proven to be effective in reducing the occurrence and duration of nocturnal hypoglycemia.^20–22 Although such delivery systems are effective at improving glycemic control,^23–26 the majority of patients with T1D continue to manage their glucose using sensor-augmented pump therapy and multiple daily insulin injections (MDI)²⁷ therapy.²⁸ Without the benefit of automated insulin delivery and glucose sensing, patients using open-loop sensor-augmented pump therapy and MDI must be proactive about adjusting their insulin or consuming a carbohydrate before bed to prevent nocturnal hypoglycemia.

There have been several published articles describing the prediction of nocturnal hypoglycemia before bedtime. For instance, Jensen et al.²⁹ described a linear discriminant analysis classifier for estimating nocturnal hypoglycemia. They were able to achieve an area under the receiver operating curve of 0.79 with a sensitivity and specificity of 75% and 70%, respectively. Their algorithm was trained and tested on 463 people with T1D and 4721 nights of data. Sakurai et al.³⁰ proposed a mathematical formulation to predict the lowest nocturnal glucose in patients with insulin-treated type 2 diabetes. The authors showed that the linear combination of age, fasting blood glucose level, and daily basal insulin dose could predict the lowest nocturnal glucose concentration within the A and B regions of the Clarke error grid³¹ with a root-mean-square-error of about 1.72 mmol/L. This prediction model was developed and validated on a small data set.

The increasing use of continuous glucose monitoring (CGM) and smart insulin delivery devices has resulted in growing data availability that can be used to train machine-learning algorithms. These large data sets provide an opportunity to leverage machine-learning and big data analytic methodologies to develop robust data-driven models^32,33 that can be used in T1D to anticipate and help prevent glucose excursions. The goal of the current work is to optimize and validate a support vector regression (SVR) model that may be used to predict and ultimately prevent nocturnal hypoglycemia by notifying a patient at bedtime to take action if there is substantial risk of nocturnal hypoglycemia. We utilized a large data set collected from 124 people with T1D (22,804 valid nights of data) under free-living conditions to train and evaluate an optimal nocturnal hypoglycemia alerting algorithm optimized to maximize the benefit of an accurate nocturnal hypoglycemia prediction and to minimize the cost of an inaccurately predicted event using decision theory.

Methods

Data sets

Training data set

A subset of the 4000+ subjects from the Tidepool Big Data Donation Data Set (Tidepool, Palo Alto, CA) was used to select the optimal parameters of the SVR model. The training data set contained a total of 27,466 days and 22,804 valid nights of time-matched CGM and insulin dosed to 124 T1D donors (age 31 ± 19 years, 15 ± 14 years since T1D diagnosis) who are insulin pump users. Data were gathered from multivendor CGM and insulin pump devices through the Tidepool.org platform. Tidepool.org does not provide information about the devices' vendors or models associated with collected data. Tidepool.org also does not provide information about patient demographics other than age, which is why we could not include these data in this article or in the prediction algorithm. CGM readings were obtained every 5 min. This data set included 17,166 nights when there was no hypoglycemia (<3.9 mmol/L) observed and 5638 nights when hypoglycemia was observed. Level 2 hypoglycemia (<3 mmol/L) occurred on 2379 nights.

Validation data set 1

A separate data set was used for model validation. The validation data set 1 was collected during a clinical study in which 10 people with T1D (age 34 ± 6 years, 6F, and 18 ± 10 years since T1D diagnosis) were continuously monitored during a 4-week clinical trial approved by the Institutional Review Board (IRB) at the Oregon Health & Science University (OHSU; clinicaltrials.gov register NCT02687893).^34,35 Participants in the validation study (Table 1) were evaluated under free-living sensor-augmented insulin-pump therapy. Glucose data were collected every 5 min using Dexcom G4 or G4 Share CGM devices (Dexcom, Inc., San Diego, CA), and people with T1D managed their glucose using their own insulin pump. A total of 117 nights of data were used for validation. Nights when the participants consumed a meal after 11 PM were excluded from the data set because consumption of carbohydrates was considered an intervention to prevent hypoglycemia. Of these 117 nights of data, nocturnal hypoglycemia (<3.9 mmol/L) was observed in 17, while no hypoglycemia was observed in the other 100 nights. Level 2 hypoglycemia (<3 mmol/L) occurred on two nights.

Table 1.

Characteristics of Participants of Study Used for Model Validation as Acquired from Reddy et al.^34,35

Description	Mean ± STD
	n = 10 subjects
Demographics
Age, years	33.7 ± 5.8
Female, n (%)	6 (60)
T1D duration, years	17.8 ± 10.2
Glycemic control
Hemoglobin A1c, %	7.4 ± 1.0
Total daily insulin requirement, U	41.0 ± 7.3
Estimated average glucose, mmol/L	9.16 ± 1.57
Validation nights, n	117
Nocturnal hypoglycemia events, n	38 ± 33
Subjects with two or more nocturnal hypoglycemia events, n (%)	8 (80)
Body composition
Body mass, kg	73.6 ± 9.5
BMI, kg/m2	24.4 ± 2.1
Physical fitness
VO2 max, mL/kg	46.8 ± 11.6
Resting heart rate, BPM	62.8 ± 7.8

BMI, body mass index; BPM, beats per minute; STD, standard deviation; T1D, type 1 diabetes.

Validation data set 2

We used another data set to further validate the model and determine how different bedtimes impact the accuracy of the algorithm. We used data from 20 people with T1D using sensor-augmented pump therapy under free-living at-home conditions over two nights as a part of a control arm from a closed-loop study that we published previously.^23,36 This data set enabled us to further validate the algorithm over 40 additional nights while also comparing how different bedtimes impact accuracy of the algorithm.

Virtual patient population data set

We used a validated and published virtual patient population³⁷ to optimize the operation point of the SVR algorithm, through which we selected an optimal prediction threshold below which a carbohydrate would be administered to a patient before bedtime. Specifically, we generated 20 virtual patients, using this virtual patient population,³⁷ through which each virtual patient was matched by weight and total daily insulin requirement to the physiology of an actual patient with T1D from validation data set 2.^23,36 Each virtual patient had a different insulin sensitivity that was statistically sampled from a distribution as described further in Resalat et al.³⁷ Twenty real-world meal scenarios acquired from people with T1D during a 4-day outpatient study²³ were given to these virtual patients and insulin dosing problems were also imposed. Ten of these virtual patients were used to determine the prediction threshold for the SVR. Validation of the algorithm was done on the remaining 10 virtual patients. Validation comprised giving a patient either 15 g of carbohydrate or alternatively a risk-based amount of carbohydrate varying between 15 and 30 g before bedtime if overnight nocturnal hypoglycemia was predicted to occur at bedtime. The model in the simulator assumes 100% availability of the glucose within the carbohydrate consumed.

SVR algorithm development

Feature extraction

A total of 59 features were extracted from CGM, insulin, and meal data. Glucose-related features included daytime glucose statistics for different time frames (calculated using data collected between 6:00 AM and 11:00 PM), glucose control metrics, and history of nocturnal glucose concentrations. Insulin-on-board and projected insulin-on-board overnight descriptors were also considered as well as consumed carbohydrates.

Glucose features were calculated across different time frames ranging from 1 to 15 h before bedtime (i.e., 11:00 PM), including statistical descriptions of the measured glucose concentrations (average, standard deviation, coefficient of variation, skewness, and kurtosis), time in hypoglycemia (glucose <3.9 mmol/L), and time in hyperglycemia (glucose >10 mmol/L). Additional glucose descriptors included bedtime glucose measurements (taken approximately at 11:00 PM), the glucose trend estimated as the slope of a linear model fit to glucose measurements during the 15 min preceding bedtime, the average of the minimum nocturnal glucose, and the likelihood of nocturnal hypoglycemia over the 7 days before the prediction.

Insulin features were represented as inputs to the model as insulin-on-board. Insulin-on-board was calculated as a sum of basal insulin and meal boluses given over time with an expected metabolic disposal and normalized by the person's total daily insulin requirement. In this work, a simplified model for insulin-on-board calculation was used. It was assumed that the action of rapid-acting insulin lasted ∼4 h with a linear decay between the administration time and the 4-h limit. Features obtained from the normalized insulin data and input into the SVR model included insulin-on-board at bedtime and insulin-on-board projected 4 h past bedtime.

Meal-related features were estimated by adding the amount of carbohydrates entered by patients into the smart bolus calculator feature of their pump. Meal sizes during the 6 h preceding bedtime were considered. However, carbohydrate intake data are inherently inaccurate since patients do not always use the bolus calculator wizard to calculate their meal boluses or they might use the calculator to calculate correction boluses.

Feature selection

The goal of feature selection is to find a subset of only the most relevant features for a specific learning problem. Key predictors for nocturnal hypoglycemia were selected using a relevance criterion called mutual information.^38,39

Figure 1 shows the features used to train the proposed machine-learning algorithm ranked by the mutual information criterion. Including additional features did not improve the accuracy of the algorithm. These features were standardized as a preprocessing step. The mean and standard deviation of each feature are presented in the Supplementary Table S1. Glucose features, particularly bedtime glucose, were the most relevant predictors of the minimum nocturnal glucose concentration. Other features derived from glucose measurements, including low blood glucose index (LBGI) and high blood glucose index (HBGI), were considered.⁴⁰ LBGI was not included in the model because the mutual information between LBGI and the minimum nocturnal glucose concentration was 0.040, which is lower than the mutual information of the other features used. If LBGI was used as a stand-alone feature, the training area under the receiver operating characteristic (ROC) curve (AUC) using only LBGI to predict nocturnal hypoglycemia was 0.62. The HBGI was correlated with other features derived from time in hyperglycemia for several time frames and so it was not included. Insulin and meal features were less relevant. This result is consistent with the findings reported by Wilson et al.⁶ from a large clinical study in which the factors associated with nocturnal hypoglycemia were retrospectively analyzed in teenagers and young adults with T1D.

FIG. 1.

Features used to train the proposed machine-learning algorithm ranked by the mutual information criterion.

In the training data set, we found age to be an important predictor for nocturnal hypoglycemia based on the mutual information criterion (Fig. 1). Nocturnal hypoglycemia rate was 23.0% ± 13.0% in the children/adolescent group (age ≤19 years as defined by the World Health Organization) versus 27.0% ± 13.2% in the adult group.

Machine-learning approach

An SVR⁴¹ model was optimized to predict minimum overnight nocturnal glucose level using the 13 most relevant features extracted from daytime glucose data, previous minimum nocturnal glucose concentration, and likelihood of nocturnal hypoglycemia based on a 7-day history. SVR is a widely used regression technique that has yielded competitive performance in many medical applications, including in T1D for short-term prediction of glucose concentrations.^42–45 SVR is trained to find a linear or nonlinear function that maps input features to a target variable constraining the differences between estimated and actual target values to be within an error precision threshold ε. ε defines an error tolerance margin (ε-margin) within which no penalty is associated in the training loss function with points predicted within a distance ε from the actual value. For those points predicted outside the ε-margin, an error penalty is applied according to the penalty parameter C, which essentially softens the hard margin defined by ε. ε and C are the hyperparameters of our model and were tuned through fivefold cross-validation. In k-fold cross-validation (k = 5 in this article), the training data set is randomly split into k groups. At each iteration, data from one of the groups are held out for validation, and the remaining k-1 folds are used for training. The performance of a model, given a set of hyperparameters, is the average of the evaluation scores calculated for each validation group. The hyperparameters that yield the best accuracy are selected to train the final model. The machine-learning pipeline is given in Supplementary Figure S1.

In this work, nocturnal hypoglycemia is defined as any event of any duration in which a patient experienced glucose levels below 3.9 mmol/dL based on CGM readings between 11:00 PM and 6:00 AM. The algorithm described in this article does not predict the time when hypoglycemia occurred, it only predicts that a hypoglycemia event occurs at some time during the overnight time window. For each patient, for each night in the data set, we choose the action (to announce nocturnal hypoglycemia, or not announce) to maximize the expected net benefit (enb) of the action, given the probability of nocturnal hypoglycemia derived from the obtained SVR linear model. It can be shown that this choice is equivalent to choosing to announce nocturnal hypoglycemia if and only if the predicted nocturnal minimum of CGM is less than a constant prediction threshold, which is derived using a cost/benefit approach.⁴⁶

Using decision theory to select prediction threshold

The goal of a decision support tool that warns a patient of impending nocturnal hypoglycemia is to make an optimal decision regarding whether or not to notify the patient. The decision support tool should optimally consider the benefits of giving the recommendation when it is correct as well as the costs of giving an incorrect recommendation. However, there are costs and benefits of not trusting the recommendation and not giving any recommendation at all. This is an example of a type of problem called decision under uncertainty.⁴⁶ Under mild conditions, it can be shown that the rational strategy is to choose the action that has the greatest enb.⁴⁷ This decision analytic approach is a formalization of intuitive considerations, which make it possible to expose, discuss, and revise specific assumptions about the problem, and then derive the consequences of those assumptions.

For the problem of recommending a bedtime hypoglycemia treatment, the actions considered are to predict nocturnal hypoglycemia or absence of nocturnal hypoglycemia. Given that nocturnal hypoglycemia might actually be present or actually be absent, there are four possible outcomes: (1) predict nocturnal hypoglycemia and nocturnal hypoglycemia is actually present, (2) predict nocturnal hypoglycemia and nocturnal hypoglycemia is actually absent, (3) predict absence of nocturnal hypoglycemia and nocturnal hypoglycemia is actually present, and (4) predict absence of nocturnal hypoglycemia and nocturnal hypoglycemia is actually absent. These outcomes are referred to as true positive (TP), false positive (FP), false negative (FN), and true negative (TN). The benefits of these outcomes are B_TP, B_FP, B_FN, and B_TN, respectively.

The probability of nocturnal hypoglycemia $p\left(NH\right)$ is derived from the SVR prediction model for minimum nocturnal glucose concentration and its associated error model. The probability of the absence of nocturnal hypoglycemia is therefore

$$p\left(\sim NH\right)=1-p\left(NH\right).$$ (Equation 1)

Then the enb for the action of predicting nocturnal hypoglycemia is

$$enb\left(NH\right)=p\left(NH\right)B_{TP}+p\left(\sim NH\right)B_{FP},$$ (Equation 2)

and the enb for the action of predicting absence of nocturnal hypoglycemia is

$$enb\left(\sim NH\right)=p\left(NH\right)B_{FN}+p\left(\sim NH\right)B_{TN}.$$ (Equation 3)

Nocturnal hypoglycemia will be predicted if $enb\left(NH\right)>enb\left(\sim NH\right)$, which is equivalent to predicting hypoglycemia if

$$p\left(NH\right)>\frac{\left(B_{TN}-B_{FP}\right)}{\left(B_{TN}-B_{FP}+B_{TP}-B_{FN}\right)}.$$ (Equation 4)

When there is a large positive reward for high sensitivity (i.e., B_TP) or a large negative cost for a missed diagnosis (i.e., B_FN), then nocturnal hypoglycemia is predicted for smaller values of $p\left(NH\right)$, which is equivalent to raising the prediction threshold for alerting persons that they will become hypoglycemic overnight. A similar, complementary conclusion follows from a consideration of B_TN and B_FP: as those terms become larger, nocturnal hypoglycemia is only predicted for larger values of $p\left(NH\right)$ or low cutoff values of the prediction threshold.

Once the critical value of $p\left(NH\right)$ is calculated and given a prediction error model, the corresponding minimum nocturnal glucose prediction threshold for alerting the patient can be found by solving for $g_{th}$ using the Gaussian cumulative distribution function

$$\begin{array}{cc}cdf& \left(x\le 3.9mg∕dL\right)=p\left(NH\right)\\ & =0.5\ast \left[1+erf\left(\frac{x-\left(g_{th}+{\mu }_e\right)}{\sqrt{2}{\sigma }_e}\right)\right],\end{array}$$ (Equation 5)

where ${\mu }_e$ and ${\sigma }_e$ are the average and standard deviation of the errors made by the prediction model, respectively. ${\mu }_e=0.07$ mmol/L and ${\sigma }_e=1.96$ mmol/L were estimated from the training data set.

The benefits B_TP, B_FP, B_FN, and B_TN were selected by analyzing the benefits of correct diagnosis of nocturnal hypoglycemia and the cost associated with missed diagnosis and potential overtreatment on 10 virtual subjects during a 5-week in-silico simulation experiment. The LBGI and HBGI were used to estimate associated costs and benefits on subjects' glucose control using Equation 6:

$$B_j=\left(LBG{I}^{ni}-LBG{I}^i\right)+\left(HBG{I}^{ni}-HBG{I}^i\right),$$ (Equation 6)

where $j\in \left\{TF,FP,FN,TN\right\}$, superscripts ni and i correspond to the metrics calculated when no intervention is performed and when an intervention is performed based on the predictions of the SVR algorithm for several decision thresholds, respectively. The individual costs and benefits were normalized to account for the imbalance in the number of nocturnal hypoglycemic events versus the number of nights where subjects did not experience hypoglycemia. The costs and benefits for positive and negative samples were scaled using a class weight calculated as the ratio of the total number of simulated nights for the 10 virtual subjects to twice the number of nights in which subjects experienced nocturnal hypoglycemia (positive class) or the number of nights in which subjects did not experience hypoglycemia (negative class), respectively.

Calculated average values for B_TP = 10.18, B_FP = −2.64, B_FN = 3.59e-3, and B_TN = 1.80e-4 resulted in critical $p\left(NH\right)=0.21$, which is equivalent to a prediction threshold of minimum nocturnal glucose concentration of $g_{th}=5.4$ mmol/L. This means that if the SVR algorithm predicted that glucose would drop below 5.4 mmol/L during the night, we would recommend to the patient to consume a carbohydrate at bedtime. The choice of the prediction threshold of 5.4 mmol/L was optimal for maximizing the enb to the patients.

The decision under uncertainty procedures described above were implemented using 20 virtual patients from the OHSU T1D virtual patient population. Ten of the virtual patients and 13,200 nights of data were used to select the optimal prediction thresholds for both the SVR and the simple bedtime glucose heuristic. And 10 of the virtual patients and 3300 nights of data were used to evaluate the effect of the intervention.

Simple bedtime glucose heuristic algorithm

We compared the SVR prediction accuracy with a simple heuristic that is oftentimes used by to avoid nocturnal hypoglycemia; if bedtime glucose is below a threshold, the patient should consume a carbohydrate before bed. Again calculating the threshold by decision theoretic analysis using a subset of 10 of the virtual patients, we found that a threshold of 8.28 mmol/L ($gbedtim{e}_{th}=8.28$ mmol/L) was optimal. Using the simple bedtime glucose heuristic, patients should consume a carbohydrate if their bedtime glucose is less than 8.28 mmol/L. The SVR algorithm was compared with the simple bedtime glucose heuristic using the evaluation metrics below.

Evaluation metrics

We evaluated the accuracy of predicting nocturnal hypoglycemia using the AUC, sensitivity, and specificity. The 95% confidence interval (CI) for the AUC was calculated using the Hanley and McNeil method.⁴⁸ Confusion matrices were generated to characterize the sensitivity and specificity of the prediction algorithms in predicting nocturnal hypoglycemia. The 95% CIs for the sensitivity and specificity of the SVR algorithm and the simple bedtime glucose heuristic at the selected operating threshold were calculated to be the exact Clopper/Pearson CIs.⁴⁹

We evaluated the accuracy of the SVR prediction algorithm by calculating the Pearson correlation between the predicted and actual minimum glucose value during the nighttime hours of 11 PM to 6 AM. And we estimated the root-mean-square-error between the actual and predicted minimum glucose during the night. The impact of using either the SVR algorithm or the simple bedtime glucose heuristic on overall glucose control within an in-silico virtual patient population³⁷ is also reported using percent time in hypoglycemia (<3.9 mmol/L), percent time in hyperglycemia (>10 mmol/L), and percent time in target range (3.9–10 mmol/L) as the outcome measures.

We compared the impact of the carbohydrate before bed intervention for both algorithms and also using an oracle, which had perfect knowledge of when a nocturnal hypoglycemia event would occur. The simulations done with the oracle ensured that a carbohydrate was given to the patient before every nocturnal hypoglycemia event and that a carbohydrate was never given if nocturnal hypoglycemia was not indicated. The oracle simulations thereby provide an upper bound on performance above which the prediction algorithms would never exceed.

Software

The development of the algorithms presented in this study was done using Python 3.6. Pandas 0.24.1 and NumPy 1.16.1 libraries were used for data processing, Scikit-learn 0.21.3 was used for machine-learning algorithm development and testing, and Matplotlib 3.0.2 was the chosen library for data visualization. Simulations of virtual subjects were done using MATLAB 2017a.

Results

Accuracy of nocturnal hypoglycemia prediction

Figure 2 shows the ROC curve of the proposed SVR model and the selected operating point compared with the operation point of the simple bedtime glucose heuristic when validating on validation data set 1 using data from nights in which subjects did not eat after 11:00 PM. A snack after the predefined bedtime was considered an intervention to avoid nocturnal hypoglycemia. The SVR prediction model performed better than the simple bedtime glucose heuristic in terms of specificity (72.0% vs. 61.0%) for the same value of sensitivity. The SVR algorithm and the simple bedtime glucose heuristic predicted 94.1% of all nocturnal hypoglycemia events (<3.9 mmol/L) and all episodes of level 2 hypoglycemia (<3 mmol/L) on data from the validation clinical study. The SVR sensitivity and specificity were derived from the confusion matrix shown in Table 2. Moreover, the SVR model predictions of minimum overnight glucose were well correlated with actual minimum nocturnal glucose values (Fig. 3). A summary of the performance metrics calculated for the SVR algorithm is presented in Table 3. The recommendation of a bedtime snack based on the predictions of the SVR model could potentially result in 2.5 overtreated cases per month (8.5% of the nights as shown in Table 3), through which the algorithm recommends a snack before bed when the person's nocturnal glucose does not actually drop below 5.4 mmol/L. However, we show in the next section that this small number of overtreatments does not result in subsequent increases in hyperglycemia.

FIG. 2.

ROC curve of the proposed SVR model and the selected operating point compared with the operation point of the simple bedtime glucose heuristic. AUC, area under the ROC curve; ROC, Receiver operating characteristic; SVR, support vector regression.

Table 2.

Confusion Matrix Resulting from the Validation of the Support Vector Regression Algorithm on the Validation Data Set Using $g_{th}=5.4$ mmol/L

n = 117 nights	True condition
Predicted condition	Nocturnal hypoglycemia (yes)	Absence of nocturnal hypoglycemia (no)	Total
Nocturnal hypoglycemia (yes)	True positives = 16	False positives = 28	44
Absence of nocturnal hypoglycemia (no)	False negatives = 1	True negatives = 72	73
Total	17	100

FIG. 3.

Correlation between the SVR model predictions of minimum overnight glucose and actual minimum nocturnal glucose values.

Table 3.

Support Vector Regression Performance in Predicting Nocturnal Hypoglycemia on the Validation Data Sets Using $g_{th}=5.4$ mmol/L

Performance metric	Value
	Clinical data	Simulated data
	n = 117 nights	n = 2706 nights
Area under the ROC curve for nocturnal hypoglycemia prediction
AUC	0.86 (95% CI, 0.75–0.98)	0.89 (95% CI, 0.86–0.91)
Nocturnal low-glucose excursion prediction (classification threshold = 3.89 mmol/L)
Sensitivity, %	94.1 (95% CI, 71.3–99.9)	88.9 (95% CI, 77.4–87.0)
Specificity, %	72.0 (95% CI, 62.1–80.5)	60.0 (95% CI, 57.8–61.8)
Correct level 2 nocturnal hypoglycemia detection rate, %	100.0	88.0
Overtreated cases (actual minimum glucose ≥5.4 mmol/L), %	8.5	14.8
Minimum nocturnal glucose concentration (regression)
Pearson correlation	R = 0.71, P < 0.001	R = 0.86, P < 0.001
RMSE, mmol/L	1.95	1.68

AUC, area under the receiver operating characteristic (ROC) curve; CI, confidence interval; ROC, receiver operating characteristic; RMSE, root mean squared error.

Using validation data set 2, we also explored how different bedtimes would affect the accuracy of our prediction algorithm using a data set from 20 people using pump therapy over two nights as a part of a control arm from an AP study that we published previously.^23,36 On validation data set 2, the sensitivity and specificity of the SVR model when nocturnal hypoglycemia is predicted at 11:00 PM are 83.3% and 90.0%, respectively. These results are comparable with the results of the analysis on validation data set 1. We found that applying the same SVR algorithm for predictions earlier or later in the night yields lower accuracy, with sensitivity values ranging from 66.7% to 80.0% and specificities from 66.7% to 100.0%, as presented in Supplementary Table S2.

Impact of carbohydrate intervention before bedtime

Table 4 shows results from validating the impact of recommending either a 15 g or variable 15–30 g carbohydrate bedtime snack based on the predictions of both algorithms. Results in Table 4 are based on data acquired from the 10 simulated T1D patients that were not used to optimize the nocturnal hypoglycemia treatment recommendation threshold across 3300 nights using real-world meal scenarios. The SVR and simple bedtime glucose heuristic were compared with no intervention, and performance of an oracle-based intervention (i.e., perfect knowledge of nocturnal hypoglycemia) was also evaluated. Without an intervention, the patients spent 6.1% ± 5.9% time in hypoglycemia overnight. The oracle-based intervention reduced the time in nocturnal hypoglycemia down to 2.0% ± 2.8% for a fixed 15 g carbohydrate intervention. When the carbohydrate intervention varied between 15 and 30 g as determined based on hypoglycemia risk, the oracle-based intervention reduced nocturnal hypoglycemia to 0.9% ± 1.6%. The intervention with variable carbohydrate amount given before bedtime based on the SVR-based algorithm reduced hypoglycemia to 1.4% ± 2.0% with no statistically significant negative impact on the percent time in target range either overnight or during the 24-h period after the person went to sleep. While the simple bedtime glucose heuristic with risk-based variable carbohydrate intervention reduced the percent time in nocturnal hypoglycemia to 1.6% ± 2.1%, there was a substantial amount of overtreatment that resulted in increased time in nocturnal hyperglycemia through which the percent time in target range was statistically significantly lower for the simple bedtime glucose heuristic compared with the oracle (58.4% ± 11.7% vs. 68.0% ± 9.1%, P < 0.001).

Table 4.

In-Silico Evaluation of the Effect of Hypoglycemia Treatment Recommendations at Bedtime

Glucose control metric	Intervention
Mean ± STD	None	Bedtime snack 15 g			Bedtime snack 15–30 g
n = 10 subjects (3300 nights)		Oracle-based	Bedtime glucose	SVR-based	Oracle-based	Bedtime glucose	SVR-based
Time in hypoglycemia (<3.89 mmol/L), %
Nighttime	6.1 ± 5.9	2.0 ± 2.8a	2.3 ± 2.9a	2.1 ± 2.7a	0.9 ± 1.6a	1.6 ± 2.1a	1.4 ± 2.0a
24 h	3.6 ± 2.4	1.7 ± 1.1a	1.8 ± 1.1a	1.8 ± 1.1a	1.2 ± 0.7a	1.5 ± 0.8a	1.4 ± 0.8a
Time in hyperglycemia (>10 mmol/L), %
Nighttime	30.2 ± 10.4	30.6 ± 10.3	36.5 ± 12.2a,b	33.2 ± 9.9a	31.1 ± 10.0	40.0 ± 13.4a,b	34.2 ± 10.1a
24 h	36.9 ± 9.9	37.2 ± 9.8	39.8 ± 9.9a,b	38.4 ± 9.2a,b	37.4 ± 9.8	41.5 ± 9.9a,b	38.9 ± 9.1a
Time in range (3.89–10 mmol/L), %
Nighttime	63.6 ± 6.5	67.4 ± 8.5a	61.2 ± 10.0b	64.7 ± 8.0b	68.0 ± 9.1	58.4 ± 11.7b	64.4 ± 8.7
24 h	59.5 ± 9.7	61.1 ± 9.2a	58.3 ± 9.3b	59.8 ± 8.7b	61.4 ± 9.2	57.0 ± 9.3b	59.70 ± 8.5

^aStatistically significant difference with respect to no intervention (P < 0.01).

^bStatistically significant difference with respect to oracle-based intervention (P < 0.01).

SVR, support vector regression.

Discussion

We demonstrate that machine-learning methodologies combined with decision uncertainty theoretic analysis can be successfully applied to nocturnal hypoglycemia prediction and treatment. The SVR model trained on a data set collected from T1D patients under free-living conditions was able to accurately predict the occurrence of nocturnal hypoglycemia with high sensitivity on people with T1D. The positive impact of hypoglycemia treatment recommendations was demonstrated on simulated T1D subjects.

The proposed SVR algorithm performs better than a simple bedtime glucose heuristic by improving the specificity of the prediction and yielding increased time in target glucose range. However, in the absence of a CGM, smart phone, or other device to acquire the necessary features to run the SVR algorithm, the methods presented in this article also offer an optimally selected bedtime glucose threshold of 8.28 mmol/L that may be used by anyone with T1D to make better decisions before bedtime to minimize nocturnal hypoglycemia. We have shown that 8.28 mmol/L is an optimal bedtime glucose threshold below which a person with T1D should consume a carbohydrate to reduce the risk of nocturnal hypoglycemia. Both the SVR and simple bedtime glucose heuristic algorithm can have multiple operating points defined by a threshold on the predicted minimum nocturnal glucose concentration such that the sensitivity and specificity can be tuned to match patients' alarm preferences.

SVR, which has the flexibility of modeling nonlinear relationship between input variables and the target variable using a kernel-based data transformation, is only one method for regression-based machine learning. There are many other regression-based machine-learning approaches that may be used to predict nocturnal hypoglycemia such as multiple linear regression and also other nonlinear methods such as multivariate adaptive regression splines, neural networks, k-nearest neighbors, and regression trees.⁵⁰ We explored an alternative linear-based prediction method by training a multiple linear regression model using the same explanatory variables used to train the SVR algorithm. The coefficients $\left\{A\right\}$ given in the order of importance of explanatory variables from Figure 1 and the intercept b of the multiple linear regression model are as follows:

$$A=\left\{\begin{array}{c}0.589,0.0181,-0.0818,-0.0292,0.2485,\\ -0.1021,-0.0122,0.0232,-0.0321,\\ -0.0351,0.0657,0.0385,-0.0249\\ \end{array}\right\}$$

and $b=2.263E-15$. The multiple linear regression model had similar performance with the SVR model when evaluated on the validation data set 1 indicated by a sensitivity of 94.1% and a slightly lower specificity of 70.0% when compared with 72.0% obtained with the SVR algorithm. The fact that comparable accuracy was achieved with a linear prediction method implies that the complexity of a nonlinear method may not be necessary, although the SVR did achieve higher specificity than the multivariable linear regression model.

An article that can be used for comparative analysis is the study by Sakurai et al.³⁰ The patients presented in that study were people with type 2 diabetes compared with T1D in the current article. Authors created a linear regression formula to predict minimum nocturnal glucose using age, glucose, and insulin features, achieving a root-mean-square error of 1.72 mmol/L. Relevant predictors and reported accuracy were consistent with the findings of our study. However, the authors were unable to validate their algorithm in T1D or on a larger data set.

An important contribution of this work is the in-silico demonstration of the positive effect of recommending a bedtime snack whose size can be varied based on the minimum glucose predicted by the SVR algorithm or a simple bedtime glucose heuristic. We found that a risk-based intervention with a recommended variable size carbohydrate intake at bedtime can reduce nocturnal hypoglycemia by up to 77.0% with no impact on the overall time in range. While a carbohydrate before bed has been shown here to be a useful intervention to help prevent nocturnal hypoglycemia, other methods may also be used. For example, if the person is a pump user, he or she may reduce the overnight basal insulin rate. They may also consider eating a snack that includes a mix of carbohydrates, proteins, and fats to slow gastric emptying and prolong the effect of the snack overnight. The in-silico model that we used in this article does not include a model for proteins or fats and so we were unable to consider these additional bedtime interventions.

The work presented here provides a basis for performing a clinical study, through which one group of participants use the SVR algorithm within a decision support smart-phone app that recommends bedtime carbohydrates if nocturnal hypoglycemia is predicted. Another study group would be the control group that does not use the app. An alternative third group could be considered that uses a decision support app that uses the simple bedtime glucose heuristic to provide decision support with regard to carbohydrate consumption before bed.

One limitation of this study is that the data used for developing the machine-learning model were limited to pump users. Future validation or algorithmic adjustments will be done on patients using multiple daily injection therapy. Second, the algorithm was trained using data from multiple CGM systems; the accuracy of the resulting model could have been affected by the suboptimal accuracy of some CGM systems in the hypoglycemic range. Third, the training data set lacks data about the subjects' physical activity. We expect that the development of a new model that includes physical activity variables would provide superior results, especially with regard to improving the specificity of the algorithm. While the sensitivity of the prediction of this algorithm is high at 94.1%, there is room for improvement in the specificity, which was 72.0%. To improve specificity in future development efforts, we plan to incorporate additional variables related to physical activity and more accurate meal information in our prediction model. Also, we plan to try different approaches at data sampling during model fitting to reduce the FP cases. Alternative methods at data sampling may be particularly important to prevent overestimation of very low and underestimation of very high nocturnal glucose estimations; this type of over- and underestimation can be observed in Figure 3. Another limitation is that the algorithm was trained using a presumed bedtime, 11 PM. We recognize that some people sleep during the day and others go to sleep earlier than 11 PM. We used the cutoff of 11 PM because the training data set that we used generally contained the most meal-free periods of time between the hours of 11 PM and 7 AM. Our analysis on validation data set 2 showed that the accuracy was lower when bedtime occurred either earlier or later than 11 PM. This implies that we would need to do further training of the algorithm earlier and later than 11 PM to achieve comparable accuracy.

Conclusion

An SVR model designed using decision theory and trained on insulin pump users can predict at bedtime whether nocturnal hypoglycemia will occur any time during the night with high sensitivity, accurately identifying 94.1% of nocturnal hypoglycemia events and all level 2 hypoglycemia events with a specificity of 72% when validated on a cohort of 10 people with T1D who are pump users. A simple bedtime glucose heuristic (i.e., consume a carbohydrate before bed if glucose is less than 8.28 mmol/L) with the threshold chosen using decision theory can also predict and help prevent nocturnal hypoglycemia. While the SVR algorithm requires use of CGM and a computational device, it yields higher specificity and better glycemic outcomes than the simple bedtime glucose heuristic. Big data analytics and machine-learning methodologies have the potential to transform diabetes care by providing new ways to effectively prevent complications associated with nocturnal hypoglycemia and improve glycemic control.

Author Disclosure Statement

P.G.J. and J.R.C. have a financial interest in Pacific Diabetes Technologies Inc., a company that may have a commercial interest in the results of this research and technology. For all other authors, no competing interests exist.

Funding Information

This work was supported by The Leona M. and Harry B. Helmsley Charitable Trust, grant 2018PG-T1D001 and a grant from NIH/NIDDK 1DP3DK101044-01.

Supplementary Material

Supplementary Figure S1

Supplementary Table S1

Supplementary Table S2