Hypoglycemia Alarm Enhancement Using Data Fusion

Abstract

Background

The acceptance of closed-loop blood glucose (BG) control using continuous glucose monitoring systems (CGMS) is likely to improve with enhanced performance of their integral hypoglycemia alarms. This article presents an in silico analysis (based on clinical data) of a modeled CGMS alarm system with trained thresholds on type 1 diabetes mellitus (T1DM) patients that is augmented by sensor fusion from a prototype hypoglycemia alarm system (HypoMon^®). This prototype alarm system is based on largely independent autonomic nervous system (ANS) response features.

Methods

Alarm performance was modeled using overnight BG profiles recorded previously on 98 T1DM volunteers. These data included the corresponding ANS response features detected by HypoMon (AiMedics Pty. Ltd.) systems. CGMS data and alarms were simulated by applying a probabilistic model to these overnight BG profiles. The probabilistic model developed used a mean response delay of 7.1 minutes, measurement error offsets on each sample of ± standard deviation (SD) = 4.5 mg/dl (0.25 mmol/liter), and vertical shifts (calibration offsets) of ± SD = 19.8 mg/dl (1.1 mmol/liter). Modeling produced 90 to 100 simulated measurements per patient. Alarm systems for all analyses were optimized on a training set of 46 patients and evaluated on the test set of 56 patients. The split between the sets was based on enrollment dates. Optimization was based on detection accuracy but not time to detection for these analyses. The contribution of this form of data fusion to hypoglycemia alarm performance was evaluated by comparing the performance of the trained CGMS and fused data algorithms on the test set under the same evaluation conditions.

Results

The simulated addition of HypoMon data produced an improvement in CGMS hypoglycemia alarm performance of 10% at equal specificity. Sensitivity improved from 87% (CGMS as stand-alone measurement) to 97% for the enhanced alarm system. Specificity was maintained constant at 85%. Positive predictive values on the test set improved from 61 to 66% with negative predictive values improving from 96 to 99%. These enhancements were stable within sensitivity analyses. Sensitivity analyses also suggested larger performance increases at lower CGMS alarm performance levels.

Conclusion

Autonomic nervous system response features provide complementary information suitable for fusion with CGMS data to enhance nocturnal hypoglycemia alarms.

Introduction

Landmark studies have demonstrated the efficacy of tight glucose control in the prevention of long-term complications of diabetes.^1,2 Despite this, a high proportion of people suffering from type 1 diabetes mellitus (T1DM), do not achieve recommended glycemic targets. For these people, the near term fear of undetected hypoglycemia is a key barrier to achieving tight glucose control in practice. Advances in the development of continuous glucose monitoring systems (CGMS) have offered a major potential to improve diabetes care, but as yet have not proved accurate enough to provide a reliable hypoglycemia alarm. Furthermore, the implementation of improvements to insulin delivery such as closed-loop systems is limited by safety concerns such as the possible consequence of closed-loop systems continuing to infuse insulin under hypoglycemic conditions. All forms of insulin therapy, including closed-loop blood glucose (BG) control systems, could benefit from integrated hypoglycemia alarms that can function as key safety devices.³ In analogous applications where available detection data may contain significant noise, such as alarms for arrhythmia, the problem of key safety alarms has been addressed by data fusion techniques.⁴ Data fusion refers to the combination of data from multiple sensors in order to gather more reliable and accurate information than can be achieved using a single data source. Data fusion is most effective when distinct and complementary sources of alarm confirmation are available. This article evaluates potential hypoglycemia alarm enhancement through the fusion of simulated CGMS data and alarms with autonomic nervous system response (ANS) features as detected by HypoMon^® systems.

Methods

Data for this simulation were derived from overnight clinical studies on HypoMon systems (AiMedics Pty. Ltd., Sydney, Australia) conducted between August 2007 and November 2008. These studies were part of a larger HypoMon development program.⁵ The first of the two protocols for this component of the clinical study program of the HypoMon enabled the collection of data for the integrated alarm algorithms; the second enabled a review of real-time performance of the HypoMon on an independent test set of patients.⁶ These protocols were approved by the local ethics committee, and subjects (or, in the case of minors, their parents or guardians) gave informed written consent.

Subjects

Ninety-eight adolescents and young adults with T1DM were enrolled in these studies at Princess Margaret Hospital, Perth, Australia (59 males). The mean age of participants was 16.3 ± 2.0 years (range 12.1–20.9 years), body mass index was 24.3 ± 3.5 kg/m² (range 17.2– 34.0 kg/m²), duration of diabetes was 7.1 ± 4.3 years (range 0.3–17.5 years), and hemoglobin A1c was 8.6 ± 1.5% (range 6.2–14.0%).

Of the 98 subjects monitored overnight, 36 developed a BG level <68.4 mg/dl (3.8 mmol/liter), 25 of these with BG levels <57.6 mg/dl (3.2 mmol/liter) and 19 with BG levels <54.0 mg/dl (3.0 mmol/liter).

Protocols

The protocols for both study phases were essentially identical. Volunteers in both studies were asked to continue with their normal diabetes management, having taken their usual insulin with dinner. Participant exclusions included previously enrolled, severe hypoglycemic episode in the previous 3 months, and use of any medication that would affect the autonomic system (e.g., β blocker). During these overnight studies, BG levels were monitored at 15- to 30-minute intervals via approximately 0.2-ml samples from an intravenous cannula. The venous samples were read on two laboratory glucose analyzers. The use of two analyzers enabled verification of readings and periodic quality checks. The frequency of blood glucose sampling was increased to every 15 minutes if the BG level fell below 90 mg/dl (5.0 mmol/liter). Subjects were given carbohydrates when the BG level fell below 45 mg/dl (2.5 mmol/liter). Volunteers wore a HypoMon sensor belt and transmitter overnight. Transmitted ANS responses were monitored at 1-minute intervals by a bedside HypoMon receiver system.

The HypoMon

The HypoMon (AiMedics Pty. Ltd.) has been developed to address the problem of overnight monitoring for hypoglycemia. Conceptually, the system is based on identifying specific patterns of physiological responses to hypoglycemia in order to enable an appropriate alarm sequence. The system consists of a chest belt that non- invasively measures physiological parameters extracted from electrocardiogram and skin impedance measurements. A radio frequency transmitter attached to the chest belt transmits collected data to the receiver to complete the system. The receiver can be positioned at the bedside or in an adjacent room. It incorporates interpretation algorithms to recognize hypoglycemia signatures within transforms of the monitored physiological parameters. In this study, selected features within a spectrum of ANS responses used by the HypoMon alarms were fused with simulated CGMS data.

Modeling

Simulation of CGMS data involved applying probabilistic modeling techniques.⁷ The modeling process was implemented using the following three stages.

Stage 1. The actual measured BG profile [Yellow Springs Instruments (YSI)] measurements) for each patient served as the basis of the model. These overnight BG profiles were sampled at 10- to 30-minute intervals, resulting in an average of 19 BG samples per patient per night, with a standard deviation (SD) of ± 3 samples. Each patient's overnight BG profile was linearly interpolated onto a uniform time grid of 5-minute intervals. This allowed for simulation of CGMS data as if it were obtained at 5-minute intervals for the whole night (around 90–100 samples per patient per night) as described in stage 2.

Stage 2. Overnight CGMS data traces were produced by adding simulated CGMS errors to the actual interpolated overnight YSI BG profiles of patients as determined in stage 1. The addition of measurement errors transformed the actual BG profile for each patient to a probable CGMS trace.

Simulated measurement errors included three different components relating to actual known errors of CGMS devices. Such errors have been reported by Wentholt and colleagues,⁸ who described a temporal delay error of CGMS data from the actual BG profile by mean = 7.1 minutes ± SD = 5.5 minutes, offset error of each CGMS sample by ± SD = 4.5 mg/dl, and vertical shift (calibration error) of CGMS profile by ± SD = 19.8 mg/dl.

To simulate CGMS traces using our actual measured patient BG profile, each error type was added in a probabilistic way. This was achieved by the addition of random values to the whole BG profile as well as each BG point. These random values were chosen from Gaussian probability density functions with means and variances given by the actual reported CGMS errors given earlier. Because they exhibit some degree of correlation, the assumption that the offset error is random is not generally applicable to offsets of CGMS data points.⁹ In our analysis, however, this approximation is justified by its relatively low contribution to CGMS performance degradation, compared to the effect of vertical shift error (calibration offset error).

Stage 3. In stage 2, all patients' BG profiles were essentially converted to simulated CGMS traces. For input into our algorithms for hypoglycemia alarms, stage 2 was iterated 100 times for each patient. This meant 100 probable CGMS traces for each patient were used as input to the algorithm. In this way, the average performance of the detection algorithm [sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV)] could be determined in a probabilistic way. We believe that this probabilistic method of modeling gives the most robust analyses of simulated CGMS data. An example of an overnight BG profile from a single subject with 10 modeled CGMS traces is given in Figure 1.

Figure 1.

Example of probabilistic simulation of CGMS data using an overnight BG level profile of a patient experiencing nocturnal hypoglycemia. The interpolated YSI profile is shown in red, with 10 probable CGMS traces (cyan). CGMS traces were simulated by adding three error components to YSI data. Errors were realistic of CGMS accuracy as described by Wentholt and colleagues⁸: vertical shift (calibration bias) of BG level profile by ± SD = 19.8 mg/dl (1.1 mmol/liter), temporal delay of CGMS trace from BG level profile by a mean time of 7.1 minutes ± SD = 5.5 minutes, and measurement error offsets of BG level data points by ± SD = 4.5 mg/dl (0.25 mmol/liter).

Analysis

The 98 patient cohort used in this study was split on the basis of entry into the HypoMon clinical development and evaluation (validation study) programs. The training subset was formed from the first 46 patients, which was the same subset used to develop the HypoMon algorithms for the subsequent in-house validation study program that enrolled 52 patients. The rationale for continuing to split data in this way for this study was that the 52 patient evaluation set was not used for any level of algorithm/threshold training.

The CGMS and fused data alarm algorithms were optimized on the training data subset. Within this subset, 18 patients experienced a hypoglycemia event as defined for training purposes. Prior training experience on earlier patient sets showed that a single threshold of <63 mg/dl (3.5 mmol/liter) produced more stable algorithm performance with training sets of this size. Parameters of CGMS data processing and ANS feature extraction, including alarm thresholds, were optimized over many iterations, with the goal of achieving maximum possible performance in terms of sensitivity and specificity but not time to alarm. Time-to-alarm optimization would require access to proprietary BG trend analysis methods not currently available. At each iteration, the performance vs threshold was produced by calculating an average performance over 25 separate runs of CGMS signal simulation and the optimum threshold was selected. During algorithm training, performance was calculated without using an error band, aiming to achieve a dense clustering of predictions near the targeted BG value of 63 mg/dl (3.5 mmol/liter).

Fusion of the HypoMon ANS response and simulated CGMS data was implemented using logic “OR” conditions on threshold transforms of both ANS and CGMS data (yes/no of crossing relevant threshold). Triggering of an alarm using this “OR” function was only permitted to operate (“AND” function) at simulated CGMS values below 86.4 mg/dl (4.8 mmol/liter). Optimization training for both CGMS alarms and fused data alarms used the training subset of 46 subjects. Performance evaluations were conducted on the independent test set.

In order to structure our interpretation to reflect practical hypoglycemia alarm usage rather than the more usual multiple correlated data point analyses used for CGMS, analyses for all HypoMon studies were based on the first alarm (either true or false) each night. Test data set analyses for both CGMS and fused data allowed a common error band for all analyses of between 68.4 mg/dl (3.8 mmol/liter) and 54.0 mg/dl (3.0 mmol/liter) and a window for the true alarm period that extended from 40 minutes before the hypoglycemia event to 40 minutes after the event. The metrics for establishing true positives and negatives and false positives and negatives for all test set evaluations were as follow: if an alarm was associated with at least one BG value below 68.4 mg/dl and it occurred within the allowed time window, it was a true positive; all other alarms were false positives. False negatives occurred if any BG values below 54 mg/dl occurred during the night without a true positive alarm. True negatives occurred if no other condition was satisfied. Fusion performance enhancements were analyzed further through a series of sensitivity analyses, which tested CGMS simulation assumptions and error band/true alarm time window settings.

Results

Examples of simulated CGMS hypoglycemia alarm performance based on in silico modeling are shown in Figure 2 (training set) and Figure 3 (test set). Plots show the sensitivity and specificity on the data sets as a function of alarm threshold for 25 runs of simulation. Table 1 provides a comparison between simulated CGMS alarm performance on the test data subset and three sets of published data from commercial systems, as well as test set results for the HypoMon. The performance of the HypoMon shown in Table 1 was achieved by means of an algorithm designed specifically to operate as part of a stand-alone alarm system producing a single alarm per hypoglycemic episode. This algorithm is significantly different from the fused system algorithm evaluated in this article and is shown for reference purposes only.

Figure 2.

Simulated CGMS alarm performance on training data showing the sensitivity and specificity of hypoglycemic alarms as a function of threshold of CGMS reading. Even though the error band was not applied during algorithm training, it was applied in this evaluation in order to allow comparison with the test performance in Figure 3. At the optimal threshold of 72 mg/dl determined during algorithm training, training performance with the error band applied was 85% in terms of sensitivity and 81% in terms of specificity.

Figure 3.

Simulated CGMS alarm performance on test data showing the sensitivity and specificity of hypoglycemic alarms as a function of threshold of CGMS reading. At the optimal threshold of 72 mg/dl (4.0 mmol/liter) determined during algorithm training, the test performance was 87% in terms of sensitivity and 85% in terms of specificity.

Table 1.

Comparison of Simulated and Published CGMS Hypoglycemia Alarm Performance

	Data source a	Estimated sensitivity (%)	Estimated Specificity (%)
1	Optimized CGMS alarm simulation	87	85
2	Guardian RT PMA number P980022/S011; Food and Drug Administration (FDA) approval: July 18, 2005	49	57
3	DexCom™ STS™ CGMS PMA number P050012; FDA approval: March 24, 2006	57	76
4	FreeStyle Navigator® CGMS PMA number P050020; FDA approval: March 12, 2008	79	60
5	HypoMon® optimized as a stand-alone alarm system Tested on a validation data subset of 52 subjects.	73	68

^aSources 2, 3, and 4 correspond to References 10–12. Performance estimates are relative to YSI readings of venous samples and were derived from multiple correlated data points. Other comparable studies^13,14 suggest similar performance levels. The performance of the HypoMon system⁶ as a stand-alone alarm is also shown for reference.

The simulated CGMS alarm algorithm optimized on the available training data set produced a mean sensitivity of 87% at 85% specificity on the test set (see Figure 3). The PPV was 61% and NPV was 96% for the CGMS alarms alone. Comparable fused data alarm system performance on the test data set exhibited 97% sensitivity at 85% specificity (a 10% sensitivity enhancement). Fused data performance showed an increase of PPV to 66% and NPV to 99%. A graphic representation of fused data performance is shown in Figure 4.

Figure 4.

Alarm performance of fused data algorithm on test data as a function of CGMS threshold. The addition of ANS response features to CGMS data increases the sensitivity from 87 to 97%, which significantly reduces the number of missed hypoglycemic episodes from 13 to 3%.

Sensitivity analyses confirmed the relative stability of these results. The primary sensitivity analysis assessed the impact of CGMS error assumptions on alarm performance and potential fusion enhancements. The sensitivity of enhancements attributable to our data fusion model as a function of assumed CGMS errors was evaluated for vertical shift errors (calibration errors) between 27 mg/dl (1.5 mmol/liter) and 12.6 mg/dl (0.7 mmol/liter). At the largest assumed vertical shift error, retesting showed larger data fusion improvements at lower CGMS performance. For example, sensitivity improved by 13% and specificity improved by 2% (95% sensitivity and 83% specificity) using fused data when added to a CGMS (stand-alone) that was achieving 82% sensitivity and 81% specificity on test data. Data fusion benefits persisted even at very high CGMS performance levels, where simulated errors were set to very low levels (12.6 mg/dl vertical shift error). In this simulation, CGMS stand-alone performance on the test set was 95% sensitivity and 84% specificity. Data fusion showed a 4.5% sensitivity and 3% specificity improvement on top of these high-performing CGMS alarms. Data fusion performance enhancements did not appear to be affected by changes to the allowed error band. Changing the allowed error band from 61.2 ± 7.2 to 63 ± 5.4 mg/dl showed similar data fusion improvements at lower CGMS performance. Using the smaller error band, data fusion produced 11% sensitivity and 1% specificity improvements when the CGMS performance alone was 74% sensitivity and 85% specificity. Fusion enhancements were also sensitivity tested over a range of allowed true alarm time windows. Simulated fusion enhancements were larger at lower CGMS performance when no increase was allowed in the 69.4 mg/dl (3.8 mmol/liter) crossing time window. Allowed true positive window expansions from a hypoglycemia onset of –10 to +60 and –10 to +90 minutes showed larger and similar fusion enhancements, respectively.

Discussion

The general question as to the validity of the developed CGMS model was addressed by comparing its performance on the test data set with public premarket approval (PMA) submission data on the hypoglycemia alarm performance of three commercial CGMS (Table 1). Despite the higher performance of the simulated CGMS hypoglycemia alarm reported in this article, the reported performance is still consistent with actual reported values, given that errors other than calibration offsets and drift were ignored in the simulation. While precise performance comparisons in Table 1 are not possible due to variations in study design and analysis methods, we believe that the similarity in results supports the validity of our model.

The in silico analyses described in this article were designed to evaluate a potential method for the enhancement of hypoglycemia alarms. Within the limited data set evaluated (52 T1DM patients), the fusion of CGMS data with ANS data as detected by the HypoMon produced potentially useful enhancements. Sensitivity studies on these enhance- ments suggested that higher benefits from ANS data fusion accrue at lower CGMS alarm performance levels, providing confidence in the enhancement achieved. These analyses did not address fusion with commercial CGMS alarm structures and would underestimate the impact of temporal dynamics due to interstitial glucose kinetics. Ideally, these factors would be addressed through further clinical studies in conjunction with commercially available CGMS that could provide real-time trend and alarm data for the fusion process. The average timing of alarms with respect to hypoglycemia onset [crossing threshold of 68.4 mg/dl (3.8 mmol/liter)] in this in silico analysis was 1.1 hours. The timing of alarms was in no way optimized in this analysis. Further studies with real CGMS data would enable a fusion optimization process that could efficiently address time to alarm features.

Key limitations of this study are that data evaluated are only related to overnight use and that the evaluation data set may not be fully representative of all sufferers of T1DM. Further studies will need to explore if reductions in systemic glucose levels induce usable ANS responses in broader population groups. Such studies should include neuropathy and hypoglycemia unawareness assessments as additional study parameters. An expansion into studies with repeated patient use over multiple nights would also provide useful insights.

Conclusion

Simulation modeling suggests that the addition of autonomic nervous system response features to continuous glucose monitoring system data has the potential to provide robust improvements to hypoglycemia alarm performance overnight. Results of this preliminary analysis show a reduction in the number of missed overnight hypoglycemic events from 13 to 3% at the same specificity. The positive predictive value of the simulated CGMS alarm system improved from 61 to 66% and the negative predictive value improved from 96 to 99% through the addition of independent data from the HypoMon.

Additional real-time clinical studies of CGMS will be required to substantiate the apparent benefits of this form of data fusion.

Abbreviations

ANS

autonomic nervous system

BG

blood glucose

CGMS

continuous glucose monitoring system

NPV

negative predictive value

PMA

premarket approval

PPV

positive predictive value

SD

standard deviation

T1DM

type 1 diabetes mellitus

YSI

Yellow Springs Instruments