Evaluation of an Algorithm for Retrospective Hypoglycemia Detection Using Professional Continuous Glucose Monitoring Data

Abstract

Background:

People with type 1 diabetes (T1D) are unable to produce insulin and thus rely on exogenous supply to lower their blood glucose. Studies have shown that intensive insulin therapy reduces the risk of late-diabetic complications by lowering average blood glucose. However, the therapy leads to increased incidence of hypoglycemia. Although inaccurate, professional continuous glucose monitoring (PCGM) can be used to identify hypoglycemic events, which can be useful for adjusting glucose-regulating factors. New pattern classification approaches based on identifying hypoglycemic events through retrospective analysis of PCGM data have shown promising results. The aim of this study was to evaluate a new pattern classification approach by comparing the performance with a newly developed PCGM calibration algorithm.

Methods:

Ten male subjects with T1D were recruited and monitored with PCGM and self-monitoring blood glucose during insulin-induced hypoglycemia. A total of 19 hypoglycemic events occurred during the sessions.

Results:

The pattern classification algorithm detected 19/19 hypoglycemic events with 1 false positive, while the PCGM with the new calibration algorithm detected 17/19 events with 2 false positives.

Conclusions:

We can conclude that even after the introduction of new calibration algorithms, the pattern classification approach is still a valuable addition for improving retrospective hypoglycemia detection using PCGM.

For people with type 1 diabetes, tight glycemic control, accomplished by intensive insulin therapy, reduces the risk of developing late-diabetic complications.^1,2 The consequence of intensive insulin therapy is a 3-fold increase in the number of hypoglycemic events. These adverse effects are fatal in severe cases, and thus are of great concern, leading to potential lack of compliance with intensive insulin therapy.^3,4 Self-monitoring of blood glucose (SMBG) has become the standard approach of glucose monitoring. However, SMBG provides only a snapshot of the glycemic control throughout the day with typically 3-4 preprandial measurements. Consequently, SMBG misses postprandial, overnight, and other glucose variabilities, making it vulnerable as a method for identifying episodes of hypoglycemia. Professional continuous glucose monitoring (PCGM) technologies, on the other hand, measure interstitial glucose (IG) every 1-5 minutes and gives a much better insight into the patient’s glucose profile. PCGM is prescribed to the patient so that he or she can monitor IG typically over a short course of 3-5 days. With such a high temporal glucose resolution, episodes of hypoglycemia could be identified by the patient in consultation with a clinician so as to develop a management strategy for adjusting glucose-regulating factors, such as insulin, carbohydrate intake, and exercise to avoid hypoglycemia.

PCGM technology utilizes a sensor with a glucose-oxidase-enzyme-containing material placed in the subcutaneous tissue of the abdomen, thigh, or upper arm. Glucose diffuses from the blood to the subcutaneous tissue, passes through the sensor, and reacts with the glucose-oxidase-enzyme producing an electrical signal. The signal is recorded and interpreted as glucose proportional to the level of IG glucose, and is shown on a display to the patient. The passive diffusion causes a physiological delay of 5-10 minutes in the sensor glucose level compared to the actual blood glucose (BG) level.⁵ The physiological delay, combined with a delay caused by signal processing filter routines in PCGM devices, makes estimation of calibration parameters and offset currents very difficult. The latter, especially, causes overestimation of sensor glucose at low BG levels, which is in large part the reason why PCGM measurements can be ineffective in identifying hypoglycemia.

To optimize hypoglycemia detection using PCGM, 2 main approaches have been investigated: (1) calibration of the raw PCGM signal by applying models of the BG-IG dynamics and (2) applying intelligent modeling approaches on top of the already processed PCGM signal. For calibration approaches, the BG-IG dynamics have been linearly modeled in several attempts,^6-10 but the true underlying dynamics are complex and inaccurate when simplified linearly. A recent algorithm by Jensen et al¹¹ based on pattern classification shows very promising results in the optimization of PCGM identification of hypoglycemia. Due to the nature of the pattern classification model, the approach does not consider the complex BG-IG dynamics. The performance of the Jensen et al algorithm was compared with the performance of the calibration algorithm of the PCGM Guardian RT^® (MiniMed Inc) from 2009. Since then new promising calibration algorithms have been developed. Mahmoudi et al¹² proposed a new calibration algorithm that when compared to the PCGM Guardian RT^® algorithm from 2009, reduced the median absolute relative PCGM-SMBG deviation in all glucose ranges.

The aim of this study was to evaluate the pattern classification algorithm proposed by Jensen et al by comparing the hypoglycemia identification with the new calibration algorithm proposed by Mahmoudi et al.

Methods

Subjects

The study population consisted of 10 male adults with type 1 diabetes (T1D) recruited from Steno Diabetes Center, Denmark (Table 1). Each subject was studied in 2 repeated experimental sessions. In each session, hypoglycemia was induced 2 hours after the start of the session by injecting a bolus of insulin Aspart (NovoRapid, Novo Nordisk A/S, Denmark). The injection was given by an experienced diabetologist, determining the size by assessing BG at injection time and the normal daily insulin dose. When subjects’ plasma glucose (PG) reached 45 mg/dl, they were given oral juice to recover. During the sessions each subject’s PG was monitored by drawing capillary blood samples every 10 minutes or more frequently, in the period from insulin injection, during PG nadir and to a PG rise above 70 mg/dl; otherwise approximately every 30-60 minutes. The samples were analyzed with a HemoCue Glucose 201⁺ glucose analyzer (HemoCue^®, Ängleholm, Sweden). Furthermore, subjects wore a PCGM device (Guardian RT^®, Minimed Inc, USA), which was calibrated by a nurse as indicated with “METER BG NOW” by the device. Written informed consent was obtained from all subjects, and the study protocol was approved by the Danish Regional Ethics Committee.

Table 1.

Demographic Characteristics of the Subjects.

Parameter	Value (N = 10)
Age	44.4 ± 14.6 years
Body mass index	23.8 ± 1.4 kg/m2
Duration of diabetes	18.2 ± 13.8 years
Daily insulin dose	40 ± 10.7 U
Impaired awarenessa	30% (3/10)
Mean glucoseb	178 ± 32 mg/dl
Mean amplitude of glycemic excursionc	50 ± 18 mg/dl

Values are mean (± SD) or percent.

^aAssessed by a standardized questionnaire.¹⁸

^bCalculated from professional continuous glucose monitoring (PCGM) data (72 hours).

^cExcursions are found by identifying where the second order derivative equals 0 of PCGM data (72 hours) smoothed with a quadratic locally weighted scatterplot smoothing (n = 50).

Data Processing

The complete data set consisted of 20 sessions with PCGM, PG readings and insulin information. Three data sets were excluded due to PCGM dropouts of more than 15 minutes. The few data dropouts of less than 15 minutes were reconstructed by spline interpolating the PCGM data. A hypoglycemic event was defined as at least 1 PG reading below 70 mg/dl.^13-16 A following period lasting a minimum of 30 minutes with no PG reading below 70 mg/dl was defined as the end of the event. For comparison with other former and future studies, a Clarke error grid analysis is presented in the results section.

Algorithm Comparison

Both algorithms were tested on the same data set, and their performances in identifying hypoglycemia were compared. The algorithm by Jensen et al works on the originally processed and calibrated output from the PCGM. Features are extracted from the PCGM signal and insulin level, and are used for training with a support vector machine model. In contrast, the algorithm by Mahmoudi et al provides calibration and, therefore, works on the raw interstitial signal (ISIG) from the PCGM. It comprises 3 parts: first, correction of the calibration parameters set for low correlation coefficient between BG measurements and their paired ISIG values and also for low relative standard deviation of the BG values; second, conversion of ISIG values to sensor glucose readings via a linear function, coefficients being estimated using robust regression with a bisquare weight function; and third, addition of an adaptive offset to enhance the accuracy of the calibration algorithm in hypoglycemia. Figure 1 gives an overview of the comparison. More thorough descriptions of the algorithms are published elsewhere.^11,12,17

Figure 1.

Overview of how the 2 algorithms are compared. The algorithm by Jensen et al takes in insulin data and the originally processed and calibrated professional continuous glucose monitoring (PCGM) data. The algorithm by Mahmoudi et al takes in the raw interstitial signal (ISIG) from the PCGM and processes and recalibrates it. The 2 outputs from the algorithms are then compared.

The performances of the algorithms’ hypoglycemia identification are presented as sample-based (ie, based on the 5-minute readings) and event-based (ie, based on the number of hypoglycemic events) sensitivity and specificity. The output of the algorithm by Jensen et al is a decision as to whether each PCGM reading is above or below 70 mg/dl, despite the actual reading. The output of the algorithm by Mahmoudi et al is a new improved calibrated PCGM signal. With these outputs and PG readings assessed as the “truth” it was possible to derive the number of sample-based true positives (TP), false negatives (FN), true negatives (TN), and false positives (FP) as depicted in Table 2, and from them calculate the sample-based sensitivity and specificity. The practical usefulness of the algorithms was assessed by event-based measures, where an event is hypoglycemia based on the aforementioned definition. The definitions of event-based TP, FN, and FP are shown in Table 2. It was not possible to define event-based TN, because it is impossible to interpret time without hypoglycemia as event-free. Therefore, event-based specificity could not be derived, but a surrogate, the number of FPs in the total sampling period, is presented. In addition to the presentation of the performances of the 2 algorithms, the original performance of the PCGM was calculated and is presented. This calculation follows the procedure of the algorithm by Mahmoudi et al.

Table 2.

Definition of the Sample-Based and Event-Based Performance Measures of Each Algorithm.

Measure		Algorithm by Jensen et al	Algorithm by Mahmoudi et al
Sample based	TP	PCGM reading classified hypoglycemic; PG ≤ 70 mg/dl	PCGM reading is hypoglycemic; PG ≤ 70 mg/dl
	TN	PCGM reading classified nonhypoglycemic; PG > 70 mg/dl	PCGM reading is nonhypoglycemic; PG > 70 mg/dl
	FN	PCGM reading classified nonhypoglycemic; PG ≤ 70 mg/dl	PCGM reading is nonhypoglycemic; PG ≤ 70 mg/dl
	FP	PCGM reading classified hypoglycemic; PG > 70 mg/dl	PCGM reading is hypoglycemic; PG > 70 mg/dl
Event based	TP	Min 4 consecutive PCGM readings classified hypoglycemic; at least 1 confirmed by PG	At least 1 PCGM reading is hypoglycemic during the event
	TN	N/A	N/A
	FN	Hypoglycemic event with no PCGM readings classified hypoglycemic	No hypoglycemic PCGM readings during the event.
	FP	Min 4 consecutive PCGM readings classified hypoglycemic; none confirmed by PG	At least 1 PCGM reading is hypoglycemic; not confirmed by PG

FN, false negative; FP, false positive; PCGM, professional continuous glucose monitoring; PG, plasma glucose; TN, true negative; TP, true positive.

Results

In the 17 data sets, the 10 subjects experienced a total of 19 hypoglycemic events. Figure 2 illustrates a Clarke error grid analysis. Characteristics of these events can be seen in Table 3. In Table 4 the performances of the 2 algorithms are presented together with the original performance of the PCGM. Figures 3a to 3c illustrate how the calibration algorithm by Mahmoudi et al optimizes the PCGM signal. Furthermore, the classification results by the algorithm of Jensen et al are illustrated.

Figure 2.

The Clarke error grid analysis illustrates the relation of the reference of plasma glucose to the predicted glucose of continuous glucose monitoring (CGM). Of the pairs, 60% fall in zone A, 32% in zone B, 0% in zone C, 8% in zone D, and 0% in zone E.

Table 3.

Characteristics of the Hypoglycemic Events.

Parameter	Value
Hypoglycemic events	18
Plasma glucose at insulin injection	185 ( ± 56; 114, 290) mg/dl
Time to hypoglycemiaa	89 ( ± 44; –29, 156) min
Time in hypoglycemiab	47 ( ± 13; 25, 75) min
Peak rate of declination in plasma glucose	–1.92 ( ± 1.06; –3.64, –0.07)  mg/dl/min

Values are mean (± SD; min, max) or number.

^aFrom insulin injection to plasma glucose crossing 70 mg/dl.

^bFrom plasma glucose dropping below 70 mg/dl to rising above 70 mg/dl.

Table 4.

Results of the Hypoglycemia Identification With the Pattern Classification Algorithm in Comparison With the New PCGM Calibration Algorithm.

	Sample-Based			Event-Based
Type	TP, FN, TN, FP	Sensitivity (%)	Specificity (%)	Sensitivity (%)	FP/total sampling time
Algorithm by Jensen et al11	127, 35, 1094, 48	78	96	19/19 = 100	1 FP/4.5 days
Algorithm by Mahmoudi et al12	129, 39, 1076, 60	77	95	17/19 = 89	2 FP/4.5 days
Orig. PCGM	52, 116, 1110, 26	31	98	12/19 = 63	0 FP/4.5 days

FN, false negative; FP, false positive; PCGM, professional continuous glucose monitoring; TN, true negative; TP, true positive.

Figure 3.

Output from the 2 algorithms and the professional continuous glucose monitoring (PCGM) in 3 different sessions. (a) The algorithm by Mahmoudi et al identifies the hypoglycemic event but stays falsely below 70 mg/dl, corresponding to 1 false positive (FP). (b) The hypoglycemic event is identified by neither the original PCGM nor the algorithm by Mahmoudi et al. However, it is identified by the algorithm by Jensen et al. (c) Both algorithms identify the 2 hypoglycemic events, while the original PCGM does not identify the first one. (d) and (e) Both algorithms identify hypoglycemia, while the PCGM does reach the hypoglycemic range.

Discussion

This work evaluated the use of a pattern classification algorithm by Jensen et al¹¹ compared to a new PCGM calibration algorithm developed by Mahmoudi et al.¹² We compared the algorithms’ abilities to identify hypoglycemia in PCGM data from people with T1D. The performance measures were sample-based sensitivity and specificity and event-based sensitivity and number of FPs in the total sampling period. The Clarke error grid analysis shows that 92% of the CGM-PG pairs fall in zones A and B, deemed clinically acceptable. However, 8% fall in zone D, which is assessed as “dangerous failure to detect and treat.”¹⁹ Especially the pairs falling in the left zone D underpin the issue of the CGM: overestimated CGM values compared to PG during hypoglycemia.

Both algorithms improved the sample-based sensitivity compared to the original PCGM by 46-47%, while the sample-based specificity was decreased by 2-3%. This led to a practical improvement of 5-7 more hypoglycemic events identified with only 1-2 more FPs. The algorithm by Jensen et al improved the sample-based sensitivity and specificity by 1% compared to the algorithm by Mahmoudi et al. This small improvement, however, led to 2 additional hypoglycemic events identified (event-based sensitivity of 100% vs 89%), with only 1 FP compared to 2 FPs for the algorithm by Mahmoudi et al. The pattern classification algorithm was trained to optimize event-based sensitivity, while lowering the number of event-based FPs, and there exists, therefore, no one-to-one relationship between sample-based and event-based measures. For example, in Figure 3b the pattern classification algorithm only identifies 4 PCGM readings as hypoglycemic during the hypoglycemia. This is enough to define the episode as a hypoglycemic event, despite the large number of sample-based FNs reducing the sample-based sensitivity.

In the experimental setup, hypoglycemia was insulin-induced and patients were given oral juice to recover when reaching 45 mg/dl. Because of this, the data set consists of rapid drops in BG before a hypoglycemic event, and a relatively short duration of the event (see Table 3). Due to the inevitable physiological delay it is difficult even for the recalibrated PCGM by Mahmoudi et al to identify these events, which explains the better event-based sensitivity by the algorithm of Jensen et al. Spontaneous hypoglycemic events may not exhibit these high rates of declination, and would thus be easier to detect for the recalibrated PCGM. However, iatrogenic hypoglycemic events resembling the insulin-induced events in this study are realistic occurrences for people with T1D, which makes them important to identify to be able to adjust insulin.²⁰

The number of FPs was increased from 0 to 1 for the Jensen et al algorithm and from 0 to 2 for the Mahmoudi et al algorithm compared with the original PCGM. It is very important that this number is as close to 0 as possible, because a FP could lead to incorrect treatment decisions, for example, lowering insulin injection, which could result in hyperglycemia. Considering the limited number of FPs of this study, it is difficult to make any conclusions on the relative performance of the algorithms on FPs, and both algorithms need to be validated on a larger number of subjects’ data to assess whether this adverse effect is significant.

Recent studies have presented various approaches to optimize PCGM accuracy. El Youssef et al²¹ tried to improve accuracy by correcting for background current. The sample-based sensitivity and specificity in hypoglycemia identification were approximately 55% and 96%, respectively, more than 20% less than the algorithm by Jensen et al. However, it should be noted that El Youssef et al state that their corrections can work real-time, which in this case bias the comparison. In another study, by Nuryani et al,²² using pattern classification on electrocardiographic data, the sample-based sensitivity and specificity of the best hypoglycemia identification were 71% and 81%, respectively, again less than the algorithm by Jensen et al.

In the present study, the PCGM was the Guardian RT^® from Medtronic. This CGM works real-time and is less sensitive to rapid changes in glucose compared to retrospective versions, such as, the iPro^® PCGM from Medtronic. The result is a lower hypoglycemia sensitivity and specificity for the Guardian RT^® than the iPro^® device, and the comparisons with the retrospective algorithms are thus biased. A smaller improvement, when using the algorithms on, for example, the iPro^® device may be anticipated. On the other hand, the algorithms’ absolute performance should increase with increasing performance of the PCGM.

These results suggest that despite the development of new calibration algorithms, the pattern classification algorithm developed by Jensen et al remains a promising tool to implement in current PCGM to optimize the identification of hypoglycemic events. It should be kept in mind, though, that the algorithm proposed by Jensen et al in the current configuration works only for hypoglycemia identification and the more sophisticated calibration algorithms like the one by Mahmoudi et al enable identification of all possible glucose excursions.

Conclusion

The pattern classification algorithm by Jensen et al¹¹ provides a unique approach to optimize the identification of hypoglycemic events in PCGM data. Several new calibration algorithms have been developed, and the algorithm by Mahmoudi et al¹² shows very good results. Even with this algorithm calibrating PCGM, the pattern classification algorithm still shows better results both in the number of identified hypoglycemic events and the number of false positives. Implementation of this algorithm in current PCGM devices could lead to better identification of hypoglycemic events, which would enable the clinician together with the patient to adjust insulin and carbohydrate intake and thereby avoid these excursions. However, the algorithm still needs to be validated on a larger number of subjects, with data including spontaneous hypoglycemic events.