Efficacy of an AI-Enabled Low Glucose Prediction: A Pooled Performance Analysis With Capillary Blood Glucose as Ground Truth

Abstract

Background:

Hypoglycemia is a critical challenge for insulin-dependent people with diabetes using multiple daily injections (MDI), who rely on reactive responses to continuous glucose monitoring (CGM) alerts. To meet the need for a proactive safety tool, we evaluated the performance of the Low Glucose Predict (LGP) feature in the Accu-Chek SmartGuide Predict App.

Methods:

This retrospective analysis pooled data from three prospective trials, including 85 subjects over 2709 recording days. The LGP feature uses a XGBoost model to predict low glucose events up to 30 minutes in advance. Performance was assessed rigorously against both capillary blood glucose (BG) and CGM values, including an analysis with “close-call” predictions (+10 mg/dL above the threshold). Metrics included sensitivity, specificity, and ROC-AUC.

Results:

Against the stringent capillary BG reference, LGP showed high performance: sensitivity of 87.13% and specificity of 97.43% (ROC-AUC 0.9787). Including close-call events improved sensitivity to 91.89% and specificity to 98.09%. Referenced against CGM, sensitivity was 94.40% and specificity was 98.25%. The system provided an actionable mean lead time of 14.71 ± 8.30 minutes (CGM reference), with a low average daily true notification rate of 1.31 (2.60 including close-calls).

Conclusion:

The LGP feature is an accurate, highly sensitive, and specific tool for timely, proactive low glucose prediction, validated against both capillary BG and CGM. This predictive intelligence is a crucial mechanism for people with diabetes to safely mitigate hypoglycemia risk, addressing a significant clinical gap and potentially reducing fear of hypoglycemia and diabetes distress.

Introduction

Hypoglycemia remains the single most critical and persistent challenge in insulin-dependent diabetes management, hindering the achievement of optimal time in range (TIR) goals. The serious and potentially life-threatening nature of hypoglycemia instills a profound fear of hypoglycemia (FoH) that often compels people with diabetes to intentionally maintain elevated glucose levels, thereby compromising long-term glucose management.^1,2

Real-time continuous glucose monitoring (rtCGM) has fundamentally transformed diabetes self-management by providing continuous, dynamic glucose data, enabling the identification of critical trends, variability, and patterns of hypo- and hyperglycemia, and thereby significantly helping with decision-making.^3-11 While rtCGM use is proven to significantly reduce hypoglycemia incidence and improve quality of life, it also adds to the individual burden due to the ongoing need for close attention to minute-to-minute data.^4,12,13 This burden is exacerbated by the fact that impaired awareness of hypoglycemia (IAH), which affects 18% to 25% of individuals with type 1 diabetes (T1D) and is associated with depression, anxiety, and heightened diabetes distress, makes people with diabetes especially vulnerable to dangerous, “surprising” low glucose events. ¹⁴ Initially, rtCGM alerts were fundamentally reactive, only signaling when a glucose value has already crossed a preset threshold. This reliance on response rather than anticipation trapped many people with diabetes in a “firefighting” mode, leading to heightened diabetes distress and alarm fatigue.^15,16

Although hybrid-closed loop systems have become the standard of care in T1D in many developed countries, significantly reducing daily burden and helping people with diabetes stay on track, ¹⁷ multiple daily injections (MDI) insulin therapy with CGM will remain the main approach for diabetes management in T2D in developed countries in the years to come.^16,18 This emphasizes the urgent need for advanced tools tailored for the MDI population.

This clinical gap necessitates the integration of CGM-based predictive features. Low glucose prediction models offer a critical pathway addressing this need, enhancing safety and significantly reducing burden. These models forecast glucose trajectory minutes in advance and issue predictive alerts, such as the widely adopted “Urgent Low Soon” feature in Dexcom’s G6 and G7 CGM systems^19,20 The benefits of this predictive alarm have been clearly demonstrated: it promotes significant reductions in time spent in hypoglycemia, helps mitigate rebound hyperglycemia, and empowers people with diabetes to maintain glycemic targets without being surprised by alarms announcing a commencing hypoglycemia.^19,20 This shift to anticipatory management is particularly crucial for those with IAH, providing the time needed to intervene before severe symptoms manifest.

The next generation glucose prediction technology presented in this manuscript is the AI-enabled Low Glucose Predict (LGP) feature of the Accu-Chek SmartGuide Predict App, a companion application for the Accu-Chek SmartGuide CGM system, which will hereafter be referred to as the Predict App. These predictive features are powered by individual algorithms which were trained applying machine-learning methodology. ²¹ Critically, this solution is intended to provide sophisticated predictive intelligence to individuals living with diabetes on insulin and non-insulin therapy, offering advanced tools to manage their condition and reduce hypoglycemia occurrence. The LGP feature, one of the three predictive features, which provides customizable low glucose (60-100 mg/dL) predictive notifications up to 30 minutes in advance, aims to deliver targeted and less disruptive warnings compared to classical glucose threshold alarms^21,22 thereby specifically addressing the challenges of alarm fatigue and psychological burden among people with diabetes. The LGP feature previously demonstrated high performance, with a sensitivity of 95.2% and a specificity of 98.9% in CGM datasets from clinical trials and real-world individuals with T1D and T2D on MDI and insulin pump therapy. ²¹

This study validates the performance metrics of the LGP feature. Our primary objective is to confirm the high accuracy, sensitivity, and efficacy of this algorithm against capillary blood glucose measurements, establishing its clinical reliability as an effective, user centric tool for preventive hypoglycemia management in the MDI population.

Methods

Study Design and Subjects

This study is a retrospective analysis of pooled data from three prospective, non-randomized, open-label, single-arm clinical trials conducted in 2023. The trials collectively included a total of 92 participants, of whom 83 were diagnosed with type 1 diabetes (T1D) and 9 with type 2 diabetes (T2D). Participants wore multiple CGM sensors over a two-week observation period. All contributing studies adhered to ethical guidelines, and only participants who had provided informed consent were included in this analysis (Table 1). The CGM sensors in the individual studies were blinded, hence no CGM-based alerts or real-time glucose values were provided to participants. This allows an unbiased analysis of low glucose events and the performance of the predictive algorithm.

Table 1.

Demographic and Clinical Characteristics of Study Participants.

Characteristic	Unit/statistic	Dataset 1 (N = 10)	Dataset 2 (N = 34)	Dataset 3 (N = 48)
Age (years)	Mean ± SD	51.6 ± 18.4	41.4 ± 12.5	44.4 ± 15.3
	Median [Min–Max]	53.5 [28.0-81.0]	40.0 [22.0-64.0]	42.5 [20.0-71.0]
Sex (n [%])	Male	4 (40%)	19 (56%)	28 (58%)
	Female	6 (60%)	15 (44%)	20 (42%)
Height (cm)	Mean ± SD	171.4 ± 7.3	173.8 ± 9.1	Not applicable (N/A)
	Median [Min–Max]	171.5 [161.5-185.5]	173.0 [168.0-180.0]	N/A
Weight (kg)	Mean ± SD	78.4 ± 10.4	79.4 ± 14.7	N/A
	Median [Min–Max]	75.0 [68.0-93.0]	76.0 [53.0-108.0]	N/A
BMI (kg/m2)	Mean ± SD	26.8 ± 3.8	26.3 ± 4.5	27.6 ± 5.7
	Median [Min–Max]	25.9 [22.7-35.7]	25.5 [20.3-40.6]	26.3 [20.1-43.6]
HbA1c (%)	Mean ± SD	N/A	6.8 ± 0.7	7.0 ± 0.8
	Median [Min–Max]	N/A	6.8 [5.8-8.4]	6.9 [5.2-8.6]
Diabetes type (n [%])	Type 1	9 (90%)	34 (100%)	40 (83%)
	Type 2	1 (10%)	0 (0%)	8 (17%)
Duration of diabetes (years)	Mean ± SD	29.1 ± 16.4	21.6 ± 12.9	19.3 ± 13.1
	Median [Min–Max]	25.0 [10.0-58.0]	21.5 [1.0-46.0]	18.5 [1.0-53.0]
Current therapy scheme (n [%])	MDI	4 (40%)	11 (32%)	28 (58%)
	CSII	3 (30%)	23 (68%)	14 (29%)
	AID	0 (0%)	0 (0%)	6 (13%)
	Oral medications	1 (10%)	0 (0%)	0 (0%)
	Basal insulin	4 (40%)	0 (0%)	0 (0%)
Familiarity with CGM (n [%])	Yes	9 (90%)	31 (91%)	N/A
Familiarity with FGM (n [%])	Yes	5 (50%)	29 (85%)	N/A

Characteristics of the three clinical datasets retrospectively analyzed for the LGP performance evaluation. Data are presented as Mean ± Standard Deviation (SD), Median [Minimum–Maximum], or count (n) followed by the percentage (%). The total number of participants in each dataset is denoted by N. Abbreviations: BMI, body mass index; HbA1c, hemoglobin A1c; MDI, multiple daily injections; CSII, continuous subcutaneous insulin infusion; AID, automated insulin delivery; CGM, continuous glucose monitoring; FGM, flash glucose monitoring.

Low Glucose Predict Functionality

The Low Glucose Predict functionality, integrated within the Predict App, continuously predicts the risk of a low glucose event occurring within the subsequent 30 minutes. The Predict App has obtained CE label under Medical Device Regulation class 2a and complies with medical software device regulations (eg, IEC 62304) and data protection regulations (eg, GDPR). ²¹ When the algorithm predicts a high risk of hypoglycemia, a push notification is delivered to the user. This notification aims to empower people with diabetes to take proactive measures, such as consuming a snack, to prevent critical hypoglycemic situations and reduce the time spent below their customized glucose threshold, which can be selected from a range of 60 to 100 mg/dL.

In more detail, the LGP functionality employs a binary classification model to assess the probability of a low glucose event within the 30-minute prediction horizon. This binary classifier is constructed using the XGBoost algorithm, a gradient-boosting tree-based approach. ²³ The model utilizes five input features derived from past CGM readings and, if available, from recorded carbohydrate information. The objective of the Low Glucose Predict system is to provide a timely warning to the user when there is a high probability of a low glucose event, defined individually for each user based on their personal hypoglycemic threshold. A “notification freeze” period of 30 minutes is implemented ensuring no predictions are made for the subsequent 30 minutes.

Data Acquisition and Processing

The algorithm predicts low glucose events using CGM data and carbohydrate intake as input features, standardized into five-minute intervals, with CGM values averaged per interval. Meal events, from which carbohydrate intake is derived, are detected using the methodology of Colmegna et al. ²⁴

Reference Measurements

Reference measurements in this study were based on two types of glucose measurements: capillary BG samples and CGM values. Capillary BG samples were used for the first time in this system’s prediction performance evaluation as a ground truth reference. On select days during the clinical study, BG measurements were collected every 10 to 30 minutes over an approximately eight-hour period while glucose excursions were induced in a supervised environment. On regular days, there were no guidelines on the frequency of recording BG values. Notably, the capillary BG measurement frequency was lower than the five-minute sampling interval of CGM data. A hypoglycemic event was defined as either a minimum of three consecutive CGM values or a minimum of one BG value falling below the defined glucose threshold.

Performance Evaluation

The LGP’s performance was evaluated using a pooled dataset with equal recording-day samples from three datasets. Recording days were balanced by including all from the smallest dataset and randomly sampling the same number from others. Performance was evaluated using CGM and capillary BG measurements as ground truth.

Sensitivity, specificity, and receiver operating characteristic area under the curve (ROC-AUC) were calculated as performance metrics. Sensitivity, defined as the proportion of actual low glucose events that are correctly predicted, focuses on at least one correct prediction within a 30-minute window. This way, each low glucose event is accounted for exactly once and the resulting metric is interpreted as the model’s ability to predict low glucose events any time up to 30 minutes ahead. Specificity is defined as the proportion of actual non-hypoglycemic events correctly identified as such, meaning no notification is triggered. Performance was evaluated across a range of glucose thresholds spanning 60 to 100 mg/dL, in 5 mg/dL increments, reflecting potential individual personalized thresholds. Each combination of a triggered prediction and a selected threshold was considered an individual sample for evaluation. To account for near-hypoglycemic events, performance metrics were recalculated by including cases where glucose levels (minimum glucose value within the next 30 minutes) were within 10 mg/dL above the defined low glucose threshold, reclassifying these instances from false positives to true positives. These “close-call” events offer additional actionable insights for diabetes management, aligning with a user centric approach, as subjects may find such notifications helpful. ²²

The previous metrics analyzed the LGP model performance through a binary classification of low glucose events, but there is an equally important temporal aspect to consider. This time dimension is captured by lead time, which is defined as the duration from the initial risk prediction (earliest true positive) to the actual event occurrence (5-30 minutes range). A longer lead time is interpreted as more favorable for facilitating a timely and effective response. To further characterize the perspective of people with diabetes using LGP, the analysis was augmented by the daily average number of expected notifications and their success rates. This analysis utilized only CGM data for reference. Due to the low sampling frequency of BG measurements in the dataset, metrics evaluating the timeliness and correctness of notifications are deemed unreliable because the sparse data points cannot guarantee the low glucose threshold was not crossed during the entire 30-minute prediction window.

Results

Data

The final dataset comprised 2709 recording days from 85 people with diabetes and 195 individual sensors, following exclusion based on secondary use consent (Table 2). Data from the three contributing clinical datasets were pooled using an equal recording day weighting to ensure balanced representation across sources. The Predict app applies rules for triggering the LGP, and only predictions conforming to these criteria were included in the analysis. Additionally, predictions were excluded from the respective analysis if sufficient (CGM or capillary BG) data were not available within the subsequent 30 minutes, as ground truth determination was not possible in these instances. In total, the evaluation included 78 931 LGP predictions across all personalized thresholds in the pooled dataset, resulting in the identification of 901 low glucose events, using BG measurements as a reference. For the evaluation based on CGM as a reference, 1 279 262 predictions across all thresholds were utilized, including 17 093 low glucose events.

Table 2.

Data Characteristics.

Metric	Unit/statistic	Dataset 1 (N = 10)	Dataset 2 (N = 30)	Dataset 3 (N = 45)
Number of sensors	Total n	27	72	96
	Mean ± SD by user	2.7 ± 1.3	2.4 ± 0.7	2.1 ± 0.7
Recording days	Total n	373	968	1368
	Mean ± SD by sensor	13.8 ± 2.7	13.4 ± 2.8	14.3 ± 2.1
TBR (%)	Mean ± SD by recording day	5.2 ± 9.3%	10.4 ± 13.7%	7.8 ± 10.6%
TAR (%)	Mean ± SD by recording day	30.0 ± 22.6%	23.6 ± 20.4%	24.8 ± 21.5%
TIR (%)	Mean ± SD by recording day	64.8 ± 21.4%	66.0 ± 19.4%	67.4 ± 20.2%

Characteristics detailing recording data and baseline glycemic status for the three clinical datasets used in the pooled analysis. Data are presented as the total count (n) or as Mean ± Standard Deviation (SD). The total number of people with diabetes included in each dataset is denoted by N. Abbreviations: TIR, time in range (70-180 mg/dL); TAR, time above range (>180 mg/dL); TBR, time below range (<70 mg/dL); SD, standard deviation.

Performance Evaluation

Specificity, sensitivity, and ROC-AUC

Using capillary BG measurements as the reference, the analysis identified 901 low glucose events and 1431 events when close-calls were included across all nine thresholds. For predictions excluding close-calls, the system achieved a sensitivity of 87.13%, specificity of 97.43%, and a ROC-AUC of 0.9787. Including close-call predictions improved sensitivity to 91.89% and specificity to 98.09%, with an elevated ROC-AUC of 0.9850 (Figure 1A, Table S1 in the Supplementary Material).

Figure 1.

Receiver operating characteristic (ROC) curves for (A) BG and (B) CGM measurements, shown with (black dashed lines) and without (blue solid lines) close-call predictions. Sensitivity is plotted against 1 − Specificity, with corresponding area under the curve (AUC) values indicated in the legend.

When CGM values were used as the reference, a total of 17 093 low glucose events were identified across the range of personalized thresholds when close-call predictions were excluded, increasing to 33 017 events when close-calls were included. The system demonstrated a sensitivity of 94.40%, specificity of 98.25%, and a ROC-AUC of 0.9892. Including close-calls further improved sensitivity to 97.10% and specificity to 99.50%, with an increased ROC-AUC of 0.9965 (Figure 1B, Table S1 in the Supplementary Material).

Prediction performance by varying low glucose thresholds

The relationship between personalized glycemic thresholds (60-100 mg/dL) and prediction performance remained consistent across all four evaluation scenarios (BG and CGM references, excl./incl. close-calls). Specifically, a higher threshold consistently led to an increase in sensitivity coupled with a decrease in specificity (Figure 2). This demonstrates the classical trade-off between sensitivity and specificity, highlighting that the choice of threshold depends on the user’s preference for prioritizing higher sensitivity at the expense of specificity or vice versa. The total event count used for evaluation was inherently dependent on the threshold, with a threshold of 60 mg/dL identifying significantly fewer events compared to 100 mg/dL.

Figure 2.

Low glucose thresholds on prediction performance: Specificity (red, left y-axis) and sensitivity (blue, left y-axis) are plotted as functions of personalized hypoglycemia thresholds (60-100 mg/dL) for both (A) blood glucose (BG) and (B) continuous glucose monitoring (CGM) data. The corresponding number of low glucose events is displayed as bar plots on the right y-axis. The figure is organized into four subplots: BG (top left), CGM (top right), BG with close-calls (+ 10 mg/dL) included (a, bottom), and CGM with close-calls included (b, bottom).

Lead Time

Using only CGM recordings as the reference, the analysis revealed a lead time of 14.71 ± 8.30 minutes (mean ± standard deviation) for 16 136 true notifications. Including close-calls, the lead time improved to 17.03 ± 9.45 minutes for a total of 32 060 true positive predictions (Table S1 in the Supplementary Material). Notably, the lead time increased with higher low glucose thresholds, from 12.85 ± 8.82 minutes (close-call: 13.90 ± 7.86) at a threshold of 60 mg/dL to 15.95 ± 8.35 minutes (Close-call: 18.81 ± 9.83) at a threshold of 100 mg/dL (see Figure 3, Tables S4, S5 in the Supplementary Material). The average daily notification rate was 3.1, with a true notification rate of 1.31/day (including close-calls: 2.60/day). This rate also increased with higher low glucose thresholds, starting at 0.49 true notifications/day (including close-calls: 1.05) at a threshold of 60 mg/dL and rising to 2.10 true notifications/day (including close-calls: 4.21) at a threshold of 100 mg/dL (Figure 3).

Figure 3.

Lead time across low glucose thresholds. Top: Lead time (minutes) for true positive predictions (notifications) using CGM reference data across different low glucose thresholds (x-axis), displayed for predictions without close-calls (blue) and with close-calls (gray). Bottom: Number of averaged daily true positive predictions (notifications) by low glucose thresholds (60-100 mg/dL) (white: average daily total notifications, blue: true events notifications, gray: true notifications incl. close-call included).

Discussion

Predictive capability is the decisive technological leap that moves CGM from a simple monitoring tool to an indispensable component of proactive, preventative diabetes therapy. The Low Glucose Predict feature within the Predict App addresses a current, acknowledged clinical need of people using CGM technology for predictive hypoglycemia notifications by providing notifications of impending low glucose events with variable glucose thresholds up to 30 minutes ahead of time. ²⁵ This functionality allows a personalization of glucose thresholds, which people with diabetes can set between 60 and 100 mg/dL.^21,22 Our study successfully evaluated the technical performance and clinical efficacy of this feature, demonstrating that LGP reliably detects impending low glucose events, which holds substantial potential to improve proactive diabetes management.

A significant strength and truly novel aspect of this research is its methodological rigor: it represents the first time such a system’s predictive capabilities have been evaluated using capillary BG samples as the primary ground truth reference. Furthermore, the study results are based on a pooled dataset from three clinical trials, encompassing data from 85 people with diabetes, 2709 recording days, and a significant number of recorded hypoglycemic events—901 for BG and 17 093 for CGM. This comprehensive dataset enhances the robustness and generalizability of the findings.

The high performance of the LGP feature is evident across all metrics. When capillary BG was used as the reference, the mean sensitivity reached 87.13% (excluding close-calls) and 91.89% (including close-calls), demonstrating that the system correctly identifies the vast majority of all occurring hypos before they commence. This is of key interest, as sensitivity, defined as the proportion of actual hypo events that are correctly predicted, is critical due to the potentially severe nature of hypoglycemic events. The high performance assessed by using capillary blood glucose measurements as a reference validates that the predictive algorithm provides highly accurate data when using the Predict App in a real-life setting as a companion to the CGM sensor. Furthermore, the system exhibited high specificity (the proportion of actual non-hypoglycemic events correctly identified) at 97.43% (excluding close-calls) and 98.09% (including close-calls) against the BG reference. When referenced to CGM, performance was even higher, with sensitivity at 94.40% (excluding close-calls) and 97.10% (including close-calls), which is in the same range as previously reported in other datasets. ²¹

The high degree of personalization offered by the adjustable glucose thresholds (60-100 mg/dL), allows individuals to tailor the LGP feature to their specific needs and treatment goals. In particular, lower thresholds offer higher specificity, while higher thresholds provide greater sensitivity with longer lead time. By delivering timely and actionable notifications, LGP holds the potential to reduce disruptive alarms and mitigate alarm fatigue, thereby enhancing the overall user experience and potentially improving adherence to CGM usage. ²⁶

A key clinical advantage demonstrated is the actionable lead time of 14.71 ± 8.30 minutes (referenced to CGM) provided by LGP. Since fear of hypoglycemia and diabetes distress are significant psychosocial burdens for individuals with diabetes, the LGP feature’s ability to provide early warnings can empower people with diabetes. This empowerment may contribute to a perceived reduction in these burdens, consistent with previous findings, which identified glucose predictions as a valuable tool for alleviating diabetes-related distress. Previous research indicates that people with diabetes appreciate the opportunity to avoid unfavorable glucose excursions in a timely manner, rather than having to respond to traditional glucose threshold alarms. ²

Despite these strengths, this study has certain limitations. It focuses on the technical performance of the prediction algorithm and does not directly assess clinical outcomes such as reductions in time in hypoglycemia, or direct person-reported outcomes (PROs) related to LGP use in a real-world setting. While the algorithms process logged carbohydrate intake, LGP does not explicitly gather contextual information regarding other events such as physical activity, alcohol consumption, or illness, all of which are recognized as factors influencing glucose management. ²¹ Consequently, the performance of the predictive models may deviate from optimal outcomes under certain circumstances. Additionally, the predictive models do not take into account future user actions that may affect glucose levels, such as anticipating the consumption of a meal in an hour. A further limitation stemming from the methodology is that the sparse sampling frequency of capillary BG measurements, which were only taken irregularly and were not available for a full 24-hour period, precluded the reliable calculation of key time-based metrics (mean lead time and average daily notification rate) using BG as the primary reference.

Study participants used their personal glucose monitoring devices in the studies, including CGM, leading to a potential bias as participant behavior and glucose course might have been impacted by the visibility of continuous glucose data and hence do not fully reflect real-world usage. In the future, prospective studies are crucial to confirm the clinical impact of LGP on glycemic outcomes (eg, improved Time Below Range and Time In Range) and person-reported outcomes (eg, reduction in fear of hypoglycemia and diabetes distress) in routine clinical use. ²²

Conclusions

Low glucose predictions directly address the urgent clinical gap of people living with diabetes, to generate awareness of upcoming critical situations and to prevent hypoglycemia. The LGP feature within the Accu-Chek SmartGuide Predict app provides a key solution to this central challenge and achieved high performance with high sensitivity and specificity, validating its efficacy against the stringent reference standard of capillary blood measurements. This level of accuracy is critical as it allows people with diabetes to rely on timely low glucose predictions for improved and proactive glycemic management.