Clinical Evaluation of a Noninvasive Alarm System for Nocturnal Hypoglycemia

Abstract

Background

The aim of this study was to evaluate the performance of a prototype noninvasive alarm system (HypoMon^®) for the detection of nocturnal hypoglycemia. A prospective cohort study evaluated an alarm system that included a sensor belt, a radio frequency transmitter for chest belt signals, and a receiver. The receiver incorporated integrated “real-time” algorithms designed to recognize hypoglycemia “signatures” in the physiological parameters monitored by the sensor belt.

Methods

Fifty-two children and young adults with type 1 diabetes mellitus (T1DM) participated in this blinded, prospective, in-clinic, overnight study. Participants had a mean age of 16 years (standard deviation 2.1, range 12–20 years) and were asked to follow their normal meal and insulin routines for the day of the study. Participants had physiological parameters monitored overnight by a single HypoMon system. Their BG levels were also monitored overnight at regular intervals via an intravenous cannula and read on two independent Yellow Springs Instruments analyzers. Hypoglycemia was not induced by any manipulations of diabetes management, rather the subjects were monitored overnight for “natural” occurrences of hypoglycemia. Performance analyses included comparing HypoMon system alarm times with allowed time windows associated with each hypoglycemic event.

Results

The primary recognition algorithm in the prototype alarm system performed at a level consistent with expectations based on prior user surveys. The HypoMon system correctly recognized 8 out of the 11 naturally occurring overnight hypoglycemic events and falsely alarmed on 13 out of the remaining 41 normal nights [sensitivity 73% (8/11), specificity 68% (28/41), positive predictive value 38%,negative predictive value 90%].

Conclusion

The prototype HypoMon shows potential as an adjunct method for noninvasive overnight monitoring for hypoglycemia events in young people with T1DM.

Introduction

Hypoglycemia impacts all patients with type 1 diabetes (T1DM), impairing quality of life and limiting attempts to achieve accepted targets for glycemic control.¹–³ Nocturnal hypoglycemia causes significant anxiety and morbidity, particularly for families of children with T1DM. A number of reports have demonstrated a high prevalence of prolonged, nocturnal hypoglycemia—up to a 35–47% chance on any given night—in children and adolescents with T1DM.⁴–⁸ Hypoglycemia can cause nocturnal seizures⁹ and, although rare, is a cause of death in the pediatric age group, usually at night.¹⁰,¹¹

The HypoMon^® (AiMedics Pty. Ltd., Sydney, Australia) has been developed to address the problem of overnight monitoring for hypoglycemia. Conceptually, the system is based on identifying specific patterns of physiological responses to hypoglycemia in order to enable an appropriate alarm sequence. It consists of a chest belt, which noninvasively monitors physiological parameters extracted from measurements of electrocardiogram and skin impedance (as detected by constant current skin surface electrodes¹²), and a radio frequency transmitter attached to the chest belt, which transmits collected data to the receiver to complete the system. The receiver can be positioned at the bedside or in an adjacent room. It incorporates interpretation algorithms to recognize hypoglycemia signatures within transforms of the monitored physiological parameters.

The algorithms used in this study were trained on an independent data set produced from a separate subject group with similar demographics. Extracted physiological features used by the detection algorithms were processed using proprietary filtering methods. These algorithms were structured around logical “OR” combinations, with time window constraints of individual threshold crossings of a subset of the filtered physiological features. Other filtered features within the algorithms acted as logic gates, that is, “AND” functions to the “OR” feature combinations. The algorithms were frozen and integrated into the HypoMon detection system prior to the study start. Algorithmic recognition of a hypoglycemic signature generates an alarm cascade, which can include text messages to selected mobile phones.

The study described in this article was designed to evaluate a prototype HypoMon system. The intended use of this system is as an adjunct to blood glucose (BG) monitoring methods with its alarm set to wake the user once during the night if a hypoglycemic event occurs. Conventional performance analysis of alarm systems reflects usage and is linked directly to the application of upper and lower alarm trigger points “allowed error band” with respect to measured reference values, in this case BG. This analysis requirement is a function of the inevitable discrepancies between the measurement system and reference values and is aimed at controlling alarm system toggling due to manual or automatic intervention. The error bands used were set prior to study start and were established through consultation with clinicians, anticipated autonomic nervous system (ANS) response thresholds from prior studies, and with reference to International Organization for Standardization (ISO) 15197. The chosen band for this study was 61.2 ± 7.2 mg/dl (3.4 ± 0.4 mmol/liter). Secondary real-time detection algorithms were also embedded into the evaluated systems in order to evaluate the potential for sensitivity and specificity adjustments to accommodate individual requirements. Only the primary algorithm results are reported in this article.

Methods

Study Subjects

Eligible participants were selected from the diabetes database held by the Endocrinology and Diabetes Service of Princess Margaret Hospital for Children, Perth, Australia (PMH). This secure database is a permanent register of all children and adolescents who are seen by the PMH clinical team. People suffering from type 1 insulin-dependent diabetes were selected according to the following criteria:

• Age range: 12–20 years

• Not previously enrolled in the study

• Not fulfilling any of exclusion criteria, which included a history of a severe hypoglycemic episode in the previous 3 months clinically assessed as having severely reduced hypoglycemia awareness, advanced diabetes complications, or a recent illness event or any medication use that would affect the autonomic system (e.g., β blocker).

Selected participants or their parents were mailed information sheets and were subsequently phoned by the recruiting nurse who confirmed eligibility and ensured that they understood the study requirements. Once the participant/parent confirmed willingness to participate in the study, an appointment was made for the study day. Written informed consent was obtained on the study day by the attending clinical research nurse. The sponsors (AiMedics) did not participate in the recruitment or enrollment process other than through preparation of the protocol and other procedural documents. The protocol was approved by the local ethics committee; participants and, in the case of minors, their guardians, gave informed written consent.

Study Protocol

Each participant was asked to follow their normal diet, exercise/activity, and insulin routines for the day of the study. Upon arrival at the hospital, data were collected from the participant, including demographic data and a finger-prick (capillary) BG level. The participant was then cannulated by the clinical research nurse for venous blood sampling throughout the night. Participants were supervised by the nursing staff (a ratio of participants to nurses of at most 3:2). Once the patient was cannulated either via antecubital or superficial dorsal digital veins, they were assigned a study number, and the research engineers and/or the nursing staff fitted the HypoMon system to the participants.

Participants had their physiological parameters monitored overnight by a single HypoMon belt and transmitter system. If the BG level of the participant (as measured by the initial finger-prick reading or by early venous measurements) was at a level at which they would normally treat before going to sleep, correction (oral glucose or subcutaneous insulin) was administered, consistent with their normal home diabetes management. Management decisions, such as study termination and BG correction needs and method, were made by nursing staff experienced in diabetes management or by the patients themselves.

When the participant was settled and ready for sleep, the hypoglycemia detection algorithms were commenced. Algorithm start times defined the study start for each participant. Time-stamped and encrypted results for each detection algorithm were then automatically written to “read only” files in the monitoring system hardware. These real-time HypoMon alarms were hidden (encrypted) until the final BG profiles for each subject were documented. Venous blood was collected by the nursing staff throughout the night at approximately 15-minute intervals or, if the BG fell below 90 mg/dl (5 mmol/liter), measurements were made at 5- to 10-minute intervals. Blood samples were tested onsite within minutes of specimen collection using two independent Yellow Springs Instruments (YSI) analyzers. Both YSI analyzers underwent standard quality control checks of calibration and linearity at the beginning of each study night. The analyzer that had the best correspondence to the target values of the quality control solutions at high, medium, and low BG level ranges was then selected by study start as the source of BG values until the next calibration. A second calibration and device selection was carried out at least once during the night or whenever a significant divergence between the two YSI device results occurred. Quality control checks were again performed at the end of the study night.

Once hypoglycemia detection algorithms were started, no treatment for abnormal BG readings was given unless the participant became symptomatic of hypoglycemia, BG as measured by the YSI analyzer fell below 50.4 mg/dl (2.8 mmol/liter), or YSI readings remained below 63 mg/dl (3.5 mmol/liter) for over 2 hours. These intervention parameters were established through discussions with the local ethics committee and the clinical team. Treatment decisions were made by the diabetes specialist nursing staff managing the study. A pediatric endocrinologist was on call for support if the nursing staff required advice or assistance. No treatment decisions were based on the output of the hypoglycemia detection algorithms as all participating staff was blinded to HypoMon outputs.

The studies generally concluded at 6:00 a.m. in order to allow for hospital routines. The monitoring devices and peripheral venous cannula were removed, and the participants were given breakfast before they left. A final finger-prick BG level measurement was obtained to monitor safe discharge levels prior to the participants leaving.

Study Analysis

All reference (YSI) BG readings were rounded to one decimal place and interpolated at 1-minute intervals by cubic spline methods.¹³ This interpolation determines the BG level between actual YSI measurements by assuming a smooth transition in the levels during the time between measurements. In order to allow for appropriate interpretation of the alarm system performance of the HypoMon, an error band was allowed for hypoglycemia detection. This error band and associated detection time window (true alarm window) was set prior to the study start and is similar, in effect, from an alarm system perspective to error bands used in other comparable analyses.¹⁴,¹⁵ The lower extremity of the error band (54 mg/dl) was chosen, taking into account prior in-house studies on the thresholds for ANS responses and the minimum BG value that could be allowed in the context of application of the HypoMon as an adjunct device.⁹ The upper limit was influenced both by the upper level of the relevant ISO 15197 band (76 mg/dl) and by market survey feedback, which suggested that a lower threshold (68 mg/dl) was more useful in an alarm context to people with diabetes.

The “truth table” (Table 1) defines the categorization of alarm results and has implications for results when the glucose nadir occurs within the error band. Such events lead to alarms being categorized as true positive and nonalarms being categorized as true negative in a manner similar to the interpretation of clinically benign errors within the Clark error grid, e.g., zone A. Hypoglycemic events for this study are true positives or false negatives as defined by this preestablished “truth table.” In this study, such events can only occur after the study start.

Table 1.

HypoMon^® Alarm System Response “Truth Table” and BG Thresholds^a

	Interpolated BG readingsb
	>68 mg/dl	≤68 mg/dl ≥54 mg/dl	<54 mg/dl
HypoMon alarm	False positive	True positive	True positive
No HypoMon response	True negative	True negative	False negative

^aThis table defines how HypoMon alarms (or lack thereof) were categorized for each participant night experiencing BG profiles with values in various ranges.

^bResults on intermittent venous blood samples (YSI analyzer).

HypoMon results were reported as the first alarm of each algorithm. This reflected prior algorithm training, which was optimized to wake the patient only once during the night. A single result was thus produced for each participant based on whether the HypoMon produced an alarm. If the alarm was associated with a hypoglycemic event and occurred within the allowed detection window (during or 40 minutes either side of the hypoglycemic event), the alarm was classified as true positive. All other alarms were classified as false positives. The classification of nonalarm states is also shown in the “truth table.” Data analysis used means and standard deviations for descriptive statistics. Two-sided confidence intervals (CI) were calculated “for the single proportion” in the reported performance statistics using methods described by Newcombe.¹⁶

General analysis criteria for performance of the HypoMon in this study were in part based on the results of two unpublished surveys conducted on the intended market segment. These surveys were part of a broader HypoMon development program and were conducted by an independent market research organization (Taverner Research) during 2007 and 2008. A specific goal of both surveys was to explore target consumer reactions to differing hypoglycemia alarm performance levels.

Results

Of the 56 subjects recruited, 3 were excluded due to equipment failures (data transmission) and 1 due to protocol exclusion criteria [below 45 mg/dl (2.5 mmol/liter) BG level on arrival]. A summary of the participant demographics in this study is shown in Table 2, and study demographics are summarized in Table 3.

Table 2.

Summary of Patient Demographics

Number of participants studied	Total:	52
Gender	Male:	28	Female:	24
Insulin delivery	Multiple daily injections:	33	Pump:	19

Parameter (units)	Mean value	Standard deviation	Range
Age (years)	16.1	2.1	12.1–20.3
Duration of diagnosis (years)	7.3	4.6	0.3–17.5
Hemoglobin A1c	8.9%	1.7%	6.3–14.0%
Body mass index	23.9	3.4	17.2–34.0

Table 3.

Summary of Study Demographics

Parameter (units)	Mean value	Standard deviation	Range
Starting BG level (venous reading) mg/dl (mmol/liter)	167.4 (9.3)	88.2 (4.9)	50.4–370.8 (2.8–20.6)
Minimum BG level mg/dl (mmol/liter)	91.8 (5.1)	50.4 (2.8)	36–235.8 (2.0–13.1)
Time between first YSI reading and study start (hh:mm)a	2:06	0:42	1:06–4:24
Study start time (hh:mm)	23:09	0:33	21:58–00:40
Time between study start and hypoglycemia (hh:mm)	2:34	1:23	0:34–4:48
Time of “hypo” onset 68.4 mg/dl (3.8 mmol/liter) (hh:mm)	1:14	1:23	22:53–04:03
Duration of hypoglycemia ≤68.4 mg/dl (3.8 mmol/liter) (hh:mm)	1:42	1:06	0:30–4:18

^aHours and minutes.

Defined hypoglycemic events occurred on 11 of the 52 participant nights. This represents a “natural” freefall prevalence of 21%.

Analysis of each participant night involved overlaying the time-stamped HypoMon alarm, if any, on the YSI-determined BG profiles. This produced a single true positive, true negative, false positive, or false negative result for each participant. Figures 1 to 4 show plots of four participant nights exemplifying true positive, false negative, true negative, and false positive results, respectively. Plots display the YSI overnight measurements for each patient and the interpolated BG profile, which is used as a reference for assessing the HypoMon system alarm performance. The two-level signal (shown in purple) in each graph represents the HypoMon signal (binary output either alarm or no alarm) and alarm timing. The error band region with respect to reference YSI BG is demarcated by two red dotted lines, one at the upper level of 68.4 mg/dl and the other at the lower level of 54 mg/dl. For the example true positive and false negative results (Figures 1 and 2), where patient BG drops below 68.4 mg/dl, the allowed time window for alarms (true alarm period) is highlighted in green.

Figure 1.

Example result for a participant night showing a true positive alarm. This alarm was raised when the patient’s BG level was within the “error band.” MDI, multiple daily injections; BMI, body mass index.

Figure 4.

Example of a participant night where a false alarm is raised when the patient’s BG level is in the euglycemic range. BMI, body mass index.

Figure 2.

Example of a participant night showing a case of false negative, as the HypoMon^® did not elicit a response, despite the BG level falling to less than 54 mg/dl. MDI, multiple daily injections; BMI, body mass index.

Figure 3.

Example of a participant night where patient’s BG level remains euglycemic and HypoMon^® does not produce an alarm (true negative). BMI, body mass index.

Interpretation of performance of the HypoMon included reference to the two prior market surveys. The finding of these unpublished surveys relevant to performance was that a comfortable nighttime alarm may achieve market acceptance at sensitivities of 70% with specificities of 65%. The assumed prevalence of nighttime hypoglycemia in these surveys was 28%, which translates into an “acceptable” positive predictive value (PPV) of 44% at a negative predictive value (NPV) of 85%.

The recognition algorithms in the prototype alarm system achieved a sensitivity and specificity level that was consistent with these market requirements at equal prevalence (PPV 47% and NPV 87%). The HypoMon system correctly recognized 8 out of the 11 [sensitivity = 73% (8/11), 95% CI: 39 to 93%] naturally occurring overnight hypoglycemic events and falsely alarmed on 13 out of the remaining 41 normal nights [specificity = 68% (28/41), 95% CI: 51 to 81%]. Positive and negative predictive values at nocturnal hypoglycemia prevalence rates in this study (21%) were PPV 38% and NPV 90%. In order to produce an alarm on 8 out of the 11 naturally occurring defined overnight hypoglycemic events, the HypoMon alarmed on 21 out of the total 52 participant nights (40%).

Poststudy in silico sensitivity analyses on the error band used for this study showed that reducing the band to 59.4 ± 5.4 mg/dl (study band: 61.2 ± 7.2 mg/dl) had no impact on performance. Raising the lower limit of the error band, however, degraded both sensitivity and specificity.

Discussion

Nocturnal hypoglycemia presents a significant barrier to good glycemic control and is associated with convulsions and rarely death. This study was designed to evaluate the performance of prototype HypoMon systems within a population of children and young adults with type 1 insulin-dependent diabetes mellitus.

Results show that the systems tested may have application as nocturnal hypoglycemia monitors for this group. These results are comparable to published values for continuous glucose monitoring systems under similar conditions.¹⁴,¹⁷ Although autonomic responses to hypoglycemia are reduced during sleep,¹⁸,¹⁹ data from this study suggest that reductions in systemic glucose levels do induce responses that can be detected with sensitive analyses.

The question of the adequacy of performance of the HypoMon is not fully answered in this single study on prototypes. Perceived adequacy of the performance of new technologies is a function of interacting factors. Key factors in such performance assessments are analyses of the ease with which the new technology can be incorporated into daily diabetes management and predictive values (PPV and NPV) of the technology. The PPV of HypoMon in this evaluation is 38%, which is lower than conventionally considered acceptable; this, however, needs to be interpreted in light of the impact of severe hypoglycemia, the fear it produces, and the ease of use of the system. Indeed, surveys (unpublished) have suggested this to be the case. This, however, will only be confirmed in practice. Predictive values need to be interpreted cautiously in that they have a direct mathematical relationship to the prevalence of the detected condition within the evaluated population.²⁰ Mathematical corrections, however, enable direct comparisons of predictive values of the same device at differing prevalence values. The predictive values found in this study were thus adjusted mathematically to the prevalence assumed in AiMedics commissioned market surveys to enable direct comparison. Results of these unpublished surveys suggested that comfortable nocturnal alarm systems would achieve acceptance at a PPV of 44% with a corresponding NPV of 85% when the prevalence was 28%. The PPV result of 38% by the HypoMon in this study when adjusted to this assumed prevalence produced similar values (PPV of 47% at NPV of 87% at 28% prevalence), suggesting an adequate performance. The mathematical relationship between prevalence and predictive values may not translate directly into user perceptions. It is possible that user perceptions will be increasingly influenced by NPV at higher prevalence values. The threshold for this transition and the associated interaction of PPV and NPV with respect to user perceptions for such alarm systems are unclear. This issue and other factors influencing the acceptability of nighttime alarm systems will require further study.

Personal control of sensitivity and specificity settings was considered by our market survey group to be of value in allowing for individual preferences and on nights following strenuous exercise or insulin regimen changes. The potential for such control was evaluated through three secondary algorithms (not reported in this article). A review of the performance of these three algorithms indicates that additional algorithm development is required.

This study focused on adolescents and young adults and was not powered to discriminate between age groups and individuals with varying diabetes durations. An evaluation of the application of HypoMon monitors in older population groups will thus require further studies. Such further studies should include neuropathy assessments and an assessment of hypoglycemia awareness as additional study parameters. The performance of this system in the hypoglycemia unaware is especially relevant, as this group is at particular risk of severe hypoglycemic events. An analysis of systematic performance variations between population subsets could also provide useful insights. Such analyses will require a larger study population with repeated participant use over multiple nights.

In silico analyses of the sensitivity of HypoMon study performance to the error band limits showed an imbalance on the selected limit values through the relative insensitivity to the upper limit of the alarm error band and the sensitivity to the lower limit of the band. This result suggests that investigation of the potential for optimization is warranted.

The recorded nocturnal hypoglycemia prevalence rate of 21% in this study sits below midrange relative to comparable freefall studies (range 10 to 55 %). This may reflect optimal corrections prior to the study start due to the availability of diabetes educator nurses for each participant or age distribution differences. The shorter average nocturnal hypoglycemia event duration of 102 minutes in this study relative to other comparable studies (192 minutes in Northam and colleagues³) reflects protocol requirements limiting absolute hypoglycemia event durations.

Conclusion

This first examination of HypoMon performance in a natural overnight setting strongly suggests that there is a place for a physiologic-based monitor as a supplement to existing glucose measurement techniques. Nocturnal hypoglycemia alarm sensitivity and specificity in this study were similar to published reports of continuous glucose monitoring systems under similar conditions. Future studies of the HypoMon should investigate optimization of performance and device utility, as well as the applicability of this physiological signature detection methodology across all subgroups of insulin-dependent diabetes.

Abbreviations

ANS

autonomic nervous system

BG

blood glucose

CI

confidence intervals

ISO

International Organization for Standardization

NPV

negative predictive value

PMH

Princess Margaret Hospital for Children

PPV

positive predictive value

T1DM

type 1 diabetes mellitus

YSI

Yellow Springs Instruments