Impact of Retrospective Calibration Algorithms on Hypoglycemia Detection in Newborn Infants Using Continuous Glucose Monitoring

Abstract

Background

Neonatal hypoglycemia is common and may cause serious brain injury. Diagnosis is by blood glucose (BG) measurements, often taken several hours apart. Continuous glucose monitoring (CGM) could improve hypoglycemia detection, while reducing the number of BG measurements. Calibration algorithms convert sensor signals into CGM output. Thus, these algorithms directly affect measures used to quantify hypoglycemia. This study was designed to quantify the effects of recalibration and filtering of CGM data on measures of hypoglycemia (BG <2.6 mmol/L) in neonates.

Subjects and Methods

CGM data from 50 infants were recalibrated using an algorithm that explicitly recognized the high-accuracy BG measurements available in this study. CGM data were analyzed as (1) original CGM output, (2) recalibrated CGM output, (3) recalibrated CGM output with postcalibration median filtering, and (4) recalibrated CGM output with precalibration median filtering. Hypoglycemia was classified by number of episodes, duration, severity, and hypoglycemic index.

Results

Recalibration increased the number of hypoglycemic events (from 161 to 193), hypoglycemia duration (from 2.2% to 2.6%), and hypoglycemic index (from 4.9 to 7.1 μmol/L). Median filtering postrecalibration reduced hypoglycemic events from 193 to 131, with little change in duration (from 2.6% to 2.5%) and hypoglycemic index (from 7.1 to 6.9 μmol/L). Median filtering prerecalibration resulted in 146 hypoglycemic events, a total duration of hypoglycemia of 2.6%, and a hypoglycemic index of 6.8 μmol/L.

Conclusions

Hypoglycemia metrics, especially counting events, are heavily dependent on CGM calibration BG error, and the calibration algorithm. CGM devices tended to read high at lower levels, so when high accuracy calibration measurements are available it may be more appropriate to recalibrate the data.

Background

Neonatal hypoglycemia is a common condition that may cause seizures and permanent brain injury in newborns.¹ There remains significant controversy regarding the definition of hypoglycemia and, consequently, the effect it can have on the child's later development.^2,3 Diagnosis is typically by blood glucose (BG) measurements. However, BG measurements are often taken several hours apart, and hypoglycemic events between BG measurements can go undetected.⁴ Continuous glucose monitoring (CGM) devices provide a continuous estimate of BG concentration and have the potential to improve the detection and diagnosis of hypoglycemia.

The first CGM device to be approved by the Food and Drug Administration, in 1999, was the Medtronic MiniMed (Northridge, CA) CGMS^® Continuous Glucose Monitoring System.⁵ The CGMS consists of a small pager-like monitor that receives information from a sensor inserted just beneath the skin.⁶ The sensor is coated with a glucose oxidase membrane, which, as glucose in the interstitial fluid is oxidized, produces a small electrical current that is proportional to the glucose concentration. The monitor stores a value every 5 min (288 per day).

Calibration BG measurements, normally obtained using a finger-stick glucometer, are required to convert electrical current into meaningful CGM output. Point-of-care testing devices are reported to have errors in the range of 2–10%^7–10 and often perform worse in intensive care unit patients because of varying levels of hematocrit, medication, and other factors.^11–13 After initial calibration, it is recommended that CGM devices be calibrated at least four times daily.¹⁴ A 10-min time delay is incorporated into the calibration process to account for the transport of glucose from the blood to the interstitial fluid.¹⁵

The retrospective calibration algorithm used by the CGMS Systems Solutions software uses linear regression.⁵ Linear regression with multiple calibration BG measurements may contribute to the CGMS reporting high during hypoglycemia and reporting low during hyperglycemia.⁶ However, it may also balance the impact of errors in finger-stick glucose meters. Thus, important clinical observations such as excursions from normal BG levels may be directly affected by the calibration algorithm used and the quality of calibration BG measurements.

Several previous studies have investigated CGM calibration schemes, both retrospective and real-time, in adults and children.^16–18 However, CGM devices have also been used to evaluate aspects of glucose metabolism in at-risk newborn infants.^4,19–22 In the case of Harris et al.,⁴ laboratory determinations of BG concentrations were available for the dataset and are assumed to be a “gold-standard” assessment. Alternative calibration algorithms can be applied to the CGM readings, using the high accuracy BG measurements, and compared with the factory-calibrated CGM readings. This study explores and quantifies the impact of calibration and nonlinear filtering on metrics of hypoglycemia in neonates using CGM devices.

Subjects and Methods

Subjects

This study used CGM data from 50 babies at risk of hypoglycemia, admitted to the Waikato Hospital Newborn Intensive Care Unit, Hamilton, New Zealand. The cohort contained 26 boys and 24 girls with a median gestational age of 34 weeks and a median birth weight of 2,172 g. The primary risk factors used to identify infants likely to become hypoglycemic include having a mother with diabetes, prematurity, and being small or large for gestational age.

CGM

All patients had interstitial glucose monitoring using the CGMS System Gold™ (Medtronic Minimed). Monitoring began on admission to the Newborn Intensive Care Unit and finished after 7 days or when the baby was no longer considered at risk of hypoglycemia. During the monitoring period nurses recorded all BG concentrations, feeding, and medication for the management of hypoglycemia. Nurses remained blinded to CGM measurements, and all calibration BG measurements were entered per the manufacturer's recommendations. Data were downloaded to a PC using CGMS System Solutions software version 3.0C, which calibrated the CGM readings retrospectively.

Calibration measurements

Blood samples were taken by nursing staff via heel-pricking at 1 h of age and then every 2–4 h (before feeds) for 12 h. In babies receiving intravenous dextrose, BG was measured every 4 h for 12 h and then less frequently as clinically indicated. The median (interquartile range) interval between calibration BG measurements was 4.8 (3.5–6.4) h.

All BG calibration measurements were made using a blood gas analyzer (model ABL800Flex; Radiometer, Copenhagen, Denmark). This device has a reading range of 0.0–60.0 mmol/L and a coefficient of variance of 1.4–2.2%.^23,24 Furthermore, a study by Watkinson et al.²⁴ showed that a device from the same family, using the same glucose electrode, had a coefficient of variation of 2.1% in intensive care unit patients and that performance was not affected by hematocrit, pH, or arterial pressure of O₂. Because of the location of the blood gas analyzer, a short time delay (estimated <15 min maximum) was possible between taking the blood sample and obtaining the glucose concentration.

Calibration algorithms

The factory calibration algorithm used by the CGMS is based on linear regression.^6,25 The linear regression algorithm is aimed primarily at ambulatory individuals with type 1 diabetes who use the CGM device to help manage BG levels. This population typically uses a finger-stick glucometer for calibration that analyzes capillary BG and typically has up to 10% error.^7–10 Hence, the use of linear regression implicitly balances calibration BG errors and CGMS errors, and CGM outputs do not necessarily exactly correspond to BG measurements.

The calibration measurements in this study were determined using a gold standard for BG measurement. Thus, data were recalibrated to make better use of the accurate calibration measurements by forcing the CGM output through all BG measurements. It should be noted there are many ways that the data could be recalibrated and that the algorithm used in this study represents just one example based as directly as possible on the current method.^5,14

The factory CGM BG estimation is determined using Eq. 1:

(1)

where BG_CGM is the BG level estimate by the CGM device (in mmol/L), the Slope is the calibration parameter found using linear regression [in (mmol/L)/nA], the Valid I_SIG is the electrical current detected by the monitor from the sensor (in nA), and the Offset is the calibration parameter that is used if the sensitivity ratio is below a threshold.

To recalibrate, Eq. 1 is rearranged to Eq. 2:

(2)

where BG_Cal is the BG level for calibration (BG analyzer measurement) (in mmol/L) and Reqd slope is the slope that forces BG_CGM through calibration measurements [in (mmol/L)/nA].

The recalibration algorithm forces the output CGM trace to pass through the calibration BG measurements, while preserving the raw sensor current (Valid I_SIG) and Offset parameter. At each calibration measurement a value of slope (Reqd slope) is calculated using Eq. 2. Linear interpolation of Reqd slope gives the new, continuous slope function, which is inserted into Eq. 1 with the unmodified Valid I_SIG and Offset parameters to give the recalibrated CGM. This recalibration provides a comparator to assess the impact of calibration on outcome CGM traces.

Median filtering

Median filters are used to remove unwanted and potentially unphysiological high-frequency noise from the CGM signal. They have proven to be a simple and effective method of removing this noise and smoothing CGM traces.^26,27 A retrospective composite median filter was used in this study because it allows faster and slower glucose dynamics to be captured more effectively. The filter averages a 3 point median and a 5 point median, both centered about the time point of interest. The filter was implemented both prior to recalibration (on the Valid I_SIG) and postcalibration on the CGM output.

Analysis

Part 1

Four analyses of the CGMS data from the 50 babies were performed in this study: (1) original CGM output, (2) recalibrated CGM output, (3) recalibrated and median filtered CGM output, and (4) filtered Valid I_SIG and then recalibrated CGM output. Each of the recalibrated variations, with and without filtering, is compared with the original CGM output to see the effect of recalibrating/filtering on clinical measures of hypoglycemia. Hypoglycemia was defined as one or more consecutive CGM measurement(s) below 2.6 mmol/L, surrounded by CGM measurements ≥2.6 mmol/L. The metrics used to quantify hypoglycemia were:

• Number: Number of independent hypoglycemic events

• Duration: Percentage of CGM recordings below 2.6 mmol/L

• Severity: Lowest measurement of hypoglycemic event

• Hypoglycemic index: Similar in concept to the hyperglycemic index presented in Vogelzang et al.,²⁸ defined as the area between the 2.6 mmol/L threshold and the CGM trace (for CGM traces <2.6 mmol/L) summed over the entire length of stay, normalized by the length of data record. Note that the units used in this study are μmol/L, not mmol/L as in Vogelzang et al.²⁸

Part 2

A subset of 43 infants with at least 3 days of CGM data was analyzed for hypoglycemic index on a day-by-day basis, for recalibration and filtering. Seven patients were excluded from the analysis because of having less than 3 days of CGM data. For each day, using the original CGM output, patients were ranked by hypoglycemic index. The ranks were preserved for the three recalibrated and filtered CGM analyses to highlight changes in hypoglycemic index for individual patients. For example, a patient with a high hypoglycemic index using the default manufacturer calibration may show low hypoglycemic index when recalibrated. A further integral index metric was used to represent the total hypoglycemic index across the 43 patients on that day, quantifying this result to a single value for easy comparison.

Results

Part 1

Figure 1 shows the distributions of CGM errors (CGM minus BG) at the time of calibrations. Subplot 1 shows the distribution for all CGM–BG data (1,074 pairs). Subplot 2 shows the distribution of errors where either the CGM or BG measurement is ≤3 mmol/L (145 pairs). Subplot 3 shows the distribution of errors where either the CGM or BG measurement is >7 mmol/L (62 pairs). In all three subplots the dashed vertical line represents the median, and the solid vertical lines represent the interquartile range.

FIG. 1.

Distribution of errors between continuous glucose monitoring (CGM) and blood glucose (BG) measurements with median (dashed vertical line) and interquartile range (solid vertical lines), for different glucose levels. Color images available online at www.liebertonline.com/dia

Calibration BG measurements detected 53 hypoglycemic events during the study, of which 16 were at a time when the CGM was also hypoglycemic. For the other 37 hypoglycemic calibration BG measurements, the CGM was >2.6 mmol/L.

Figure 2 shows a section of CGM trace comparing original (dashed line), recalibrated (solid line), recalibrated and filtered (dotted line), and filtered valid I_SIG and then recalibrated variations (dash-dot line). In this example, overall trends in calibration parameters and CGM output are preserved. However, it is clear in Figure 2 that the metrics of hypoglycemia will vary for each method of signal processing and for each patient. Figure 3 shows the CGM trace in this study that had the largest change in hypoglycemia metrics after recalibration. The original CGM trace (dashed line) suffered from particularly bad factory calibration for the first 12 h of monitoring, during which time there was a significant period of hypoglycemia. However, all four calibration BG measurements in the first 12 h were >2.6 mmol/L, and consequently the recalibrated CGM trace (solid line) had no hypoglycemia.

FIG. 2.

Comparison of a section of continuous glucose monitoring (CGM) trace containing hypoglycemia for the original CGM, recalibrated CGM, recalibrated and filtered CGM, and filtered valid electrical current detected by the monitor from the sensor (I_SIG) and then recalibrated CGM. Hypoglycemia is defined as one or more consecutive CGM measurement(s) <2.6 mmol/L, surrounded by one or more CGM measurement(s) >2.6 mmol/L. Note that recalibrating increases the number of hypoglycemic events from one to four; then filtering reduces it back to one in this example. Color images available online at www.liebertonline.com/dia

FIG. 3.

The continuous glucose monitoring (CGM) trace that had the largest change in hypoglycemia metrics after recalibration. The original CGM trace (dashed line) contains a long period of hypoglycemia in the first 12 h of monitoring. However, all four calibration blood glucose (BG) measurements in this period were >2.6 mmol/L, and consequently the recalibrated CGM trace (solid line) had no hypoglycemia. Color images available online at www.liebertonline.com/dia

The top section of Table 1 compares the number, duration, and severity of hypoglycemia events, as well as the hypoglycemic index for each variation of the CGM calibration. The results are presented for the overall cohort (50 patients) and per patient to show any potential skewed results from individual patients. The bottom section of Table 1 shows the effect of recalibration and/or filtering on the hypoglycemic state of patients over the monitoring period. For example, the top left cell in the bottom section of Table 1 (“24”) indicates that 24 patients who had hypoglycemia in the original CGM data still had hypoglycemia in the recalibrated CGM data. Conversely, the second row in column 1 (“1”) indicates that one patient who had hypoglycemia in the original CGM data had no hypoglycemia in the recalibrated CGM data.

Table 1.

Effect of Recalibration and Filtering on Recorded Continuous Glucose Monitoring of Hypoglycemia for the Entire Cohort and per Patient

	Original CGM	Recalibrated CGM	Recalibrated and filtered CGM	Filtered valid ISIGthen recalibrated CGM
Overall cohort results
Number of hypoglycemic events	161	193	131	146
Duration (% of CGM record<2.6 mmol/L)	2.2	2.6	2.5	2.6
Hypoglycemic index (μmol/L)	4.9	7.1	6.9	6.8
Hypoglycemia events
Between 2.4 and 2.6 mmol/L	87	87	51	61
Between 2.2 and 2.4 mmol/L	35	40	35	34
Between 2.0 and 2.2 mmol/L	18	38	23	30
<2.0 mmol/L	21	28	22	21
Number of patients with no hypoglycemia	25	19	21	19
Per-patient results
Number of hypoglycemic events	1 (0–3)	1 (0–4)	1 (0–3)	1 (0–3)
Duration (% of data hypoglycemic)	0.1 (0.0–1.0)	0.5 (0.0–2.4)	0.5 (0.0–2.3)	0.6 (0.0–2.4)
Hypoglycemic index (μmol/L)	2.1 (0.7–6.8)	0.8 (0.0–7.1)	0.5 (0.0–6.0)	3.9 (0.7–9.5)
Hypoglycemic state (no. of patients)
Original hypoglycemia→hypoglycemia	—	24	22	23
Original hypoglycemia→no hypoglycemia	—	1	3	2
Originally no hypoglycemia→hypoglycemia	—	7	7	8
Originally no hypoglycemia→no hypoglycemia	—	18	18	17

Data are median (interquartile range) values where applicable.

CGM, continuous glucose monitoring; I_SIG, electrical current detected by the monitor from the sensor.

Part 2

Figure 4 shows the effect of recalibration and filtering on the hypoglycemic index of individual patients on a day-by-day basis. The top row of Figure 4 shows the hypoglycemic index for each patient over the first 24 h of monitoring, the second row shows the second 24-h period, and the third row shows the third 24-h period. From left to right in each row, the change in hypoglycemic index for each patient due to calibration and/or filtering can be seen (the patient order is preserved across these plots). For example, the infant with most hypoglycemia by original CGM calibration on Day 1 (shown in Fig. 3) had no hypoglycemia with any of the recalibration methods. Finally, the integral index is also indicated on each panel of Figure 4.

FIG. 4.

Comparison of ranked hypoglycemic (Hypo) index for 43 patients (3 days) for original continuous glucose monitoring (CGM), recalibrated CGM, recalibrated and Filtered CGM, and filtered electrical current detected by the monitor from the sensor (I_SIG) and then recalibrated. The curved solid line repeats the ranked distribution as determined by the original CGM data. The integral index captures the overall area of each panel for a single comparator value. Color images available online at www.liebertonline.com/dia

Discussion

The aim of this study was to investigate how recalibration and nonlinear filtering of CGM data affect metrics of hypoglycemia in at-risk preterm babies. This knowledge is important for accurately relating hypoglycemia to long-term outcomes.

Part 1

The top section of Table 1 shows all metrics of hypoglycemia increased after recalibration compared with the original CGM results, which can potentially be explained by skew in the distribution of BG versus CGM readings at low BG concentrations. Figure 1 shows the distribution of errors between the CGM–BG paired measurements. The dataset contains 1,074 paired BG–CGM measurements, of which 51% have a BG measurement higher than the CGM and 49% have a BG measurement lower than the CGM, and this is overall relatively centered, as expected from the regression aspect of the calibration algorithm.

More important is that the second plot in Figure 1 shows a definite positive shift in the median when only considering low glycemic levels. Of the 145 pairs containing either a CGM or BG measurement <3 mmol/L, 63% have a BG measurement lower than the CGM. These lower measurements pull the CGM trace down to the more accurate BG analyzer value when recalibrating and cause the hypoglycemia metrics to increase. Additionally, the bottom plot in Figure 1 shows the opposite is also true for high BG concentrations. At least for our dataset, CGM readings have a greater tendency to be lower than their BG counterparts when the concentrations are >7 mmol/L.

When comparing recalibrated CGM data with recalibrated and filtered CGM data, the large reduction in the number of hypoglycemic events in Table 1 (from 193 to 131) with little change in hypoglycemic duration (from 2.6% to 2.5%) can be explained with reference to Figure 2. Two different phenomena occur that reduce the number of events from four to one in this exemplar case. First, at 3.8 days the peak in the CGM trace is trimmed by the filter (filter after recalibration), stopping it from crossing the normoglycemic threshold and reducing the number of hypoglycemic events. The opposite can also occur, where a hypoglycemic event observed in the recalibrated CGM trace is removed by filtering (in this case a trough would be trimmed).

The second phenomenon is seen at approximately 3.88–3.9 days, where high-frequency fluctuations in CGM measurements are smoothed by the filter. Smoothing these fluctuations around the threshold is likely to be the major influence on the reduced number of hypoglycemic events observed. The variations in the number of hypoglycemic events observed over the four analyses suggests that this metric alone (number of events) may not be reliable when classifying hypoglycemia.

The bottom section of Table 1 shows the number of patients who gained, lost, or stayed with or without hypoglycemia when CGM recordings were recalibrated and filtered, compared with the original CGM data. Of the 25 babies who had hypoglycemia in the original dataset, 22–24 had hypoglycemia in the modified datasets, and 17–18 out of 25 babies who had no hypoglycemia in the original dataset still had no hypoglycemia. These results suggest that over the duration of monitoring, the CGM should be consistent approximately 80% of the time about which patients had experienced hypoglycemia, independent of calibration method.

Part 2

Figure 4 shows three important trends. First, the hypoglycemic index, a measure of duration and severity, for a patient on any given day can change significantly after recalibration. For example, the highest ranked patient on Day 1 using the original CGM output had a hypoglycemic index of approximately 105 μmol/L, which was reduced to 0 μmol/L after recalibration. Conversely, several patients have no hypoglycemic events on Day 1 (original CGM), but after recalibration all have a hypoglycemic index greater than 0 μmol/L. For Days 2 and 3 there is far more agreement in which patients had hypoglycemia, although in some cases the index still changes with recalibration. This outcome suggests that after Day 1, regardless of calibration method, patients who experienced hypoglycemia could be identified with a higher level of confidence.

The next trend shown by Figure 4 is that hypoglycemia is most prevalent during Day 1. Comparing the plots in rows 1 and 2 of Figure 4 shows the decrease in integral index from Day 1 to Day 2 is in the range of 45–54%, suggesting hypoglycemia is more prevalent in the overall cohort on Day 1, by any calibration method. It is interesting that further decreases were not observed from Day 2 to Day 3 in either the original CGM data or the recalibrated/filtered CGM data of Figure 4. This result reinforces the importance of capturing the first 24 h of CGM data and thus of proper placement and initial device calibration.

These first two trends suggest Day 1 is most affected by the calibration scheme. This could be due to two reasons. First, the error between calibration BG measurements and CGM measurements tends to be larger on Day 1, so recalibrating has more of an effect. Second, these infants are at risk by definition, and hypoglycemia can be more prevalent on this first day of life.^29,30 Neither effect can be ruled out in this analysis.

The third trend is an increase in integral index with recalibration, regardless of the day, which can potentially be explained by the previously mentioned tendency of the CGM device to read high low glucose levels. Conversely, there is a reduction in integral index with the addition of filtering, which is likely explained by the “rounding” or clipping of troughs, in this analysis.

Limitations

The main limitation of this study is the limited number of BG measurements available. Because of the pain and discomfort of blood sampling in neonates (typically by heel prick), it is unethical to measure BG more frequently than approximately every 4 h. Ideally, a reference measurement would be sampled for nearly every CGM measurement, in addition to the approximately four calibration measurements per day. The reference measurements would allow the impact of calibration on a “true” level of hypoglycemia to be assessed more thoroughly and conclusively.

The limited number of BG measurements also restricts the recalibration and filtering strategies. For example, the estimated physiological delay between BG and interstitial glucose was left constant at 10 min (the manufacturer's default value). Studies in the literature report delays typically in the range of 5–15 min.^27,31–34 However, without sufficient reference BG measurements the delay cannot be determined in this study for this group of patients. Finally, the filter implemented in this study is effective but represents one of a wide variety of available filters. Other more advanced filtering options are available, but without several reference BG measurements it is not possible to determine the optimal filtering strategy.

Conclusions

The aim of this study was to investigate how recalibrating and filtering CGM data affect metrics of hypoglycemia in preterm infants. The results suggest that conventional hypoglycemia metrics are heavily dependent on both the error in calibration BG measurements and the calibration algorithm used. All metrics of hypoglycemia for our cohort increased after recalibration, confirming that the original, unmodified CGM output tended to report high at lower levels. If highly accurate calibration measurements are available it may be more appropriate to recalibrate the data, especially when trying to accurately classify hypoglycemia or other specific extreme events.

More importantly and generally, calibration BG measurement error and thus calibration algorithms play a significant role in quantifying hypoglycemia using CGM data. In particular, metrics such as number of hypoglycemic events are particularly sensitive to recalibration effects. Although this conclusion may be expected, its potential impact is quantified here, in this case for at-risk neonates for whom hypoglycemia may carry long-term negative consequences.

Acknowledgments

The authors would like to acknowledge members of the CHYLD Study Group for collecting and compiling data for this study and the CHYLD Study Steering Group for oversight and approval of the manuscript. Members of the CHYLD Study Group are as follows: Jane Alsweiler, Judith Ansell, Nicola Anstice, Jo Arthur, Coila Bevan, Kate Bluett, Ellen Campbell, Arijit Chakraborty, Geoff Chase, Tineke Crawford, Karen Frost, Greg Gamble, Jane Harding, Deborah Harris, Rob Jacobs, Kelly Jones, Aaron Le Compte, Sapphire Martin, Gill Matheson, Grace McKnight, Christina McQuoid, Janine Paynter, Jenny Rogers, Matthew Signal, Heather Stewart, Ben Thompson, Anna Timmings, Trecia Wouldes, Rebecca Young, and Sandy Yu. This work was supported financially by the University of Canterbury's Department of Mechanical Engineering and the Health Research Council of New Zealand. The project described was supported in part by grant RO1HD069622 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

Author Disclosure Statement

No competing financial interests exist.