Simple Post-Processing of Continuous Glucose Monitoring Measurements Improves Endpoints in Clinical Trials

Abstract

Background:

Continuous glucose monitoring (CGM) is a powerful tool to be considered both in clinical practice and clinical trials. However, CGM has been criticized for being inaccurate for many reasons including a physiological delay. This study sought to investigate the current delay issue and propose a simple post-processing procedure.

Method:

More than a million hours of the Dexcom G4 CGM from 472 subjects investigated in a state-of-the-art clinical trial were analyzed by time shifting the CGM measurements and comparing them to plasma glucose (PG) measurements. The resultant CGM measurements were then assessed in relation to real-world clinical research endpoints.

Results:

A CGM time shift of −9 minutes was optimal and reduced mean absolute relative difference (MARD) statistically significantly with 1.0% point. The MARD reduction resulted in better clinical research endpoints of hypoglycemia and postprandial glucose increments.

Conclusions:

The delay in CGM is still an issue. The delay in this study was identified to be 9 minutes compared to PG. With a simple post-processing approach of time shifting the CGM measurements with −9 minutes, it was possible to obtain a statistically significantly lower MARD and subsequently obtain clinical research endpoints of improved validity.

Continuous glucose monitoring (CGM) has for many years been a promising and utilized tool for both people with diabetes mellitus (DM) and health care physicians responsible for the diabetes treatment.^1-3 Simultaneously, a recurrent topic has been the accuracy of the devices.^4,5 The preferred site for glucose sensing continuous to be the subcutaneous interstitial fluid, but changes in this glucose concentration will always be delayed with respect to changes in plasma glucose (PG) causing a so-called physiological delay.^6,7 Also, a device-related filter routine delay is typically observed, and the sensors based on glucose oxidase often have a nonspecific offset current.^8,9 The resulting inaccuracy has led to criticism and questioning of the fundamental assumption that interstitial fluid glucose and PG can be made identical.¹⁰ On the other hand, many experts and studies have concluded that CGM devices can be deemed acceptable, and that the inaccuracy was related to older devices.^4,5,11 In the beginning of the third millennium, mean absolute relative difference (MARD) of CGM devices was reported to be around 20% with delays exceeding 10-15 minutes,^8,12,13 but with the advent of new algorithms for updated noise reduction and modified calibration, devices have improved a lot with MARD going toward 10% and delays of less than 10 minutes.^6,7,14 These improvements have paved the way for a more positive attitude toward CGM devices.⁴

In February 2017, an international panel of experts in CGM technology at the Advanced Technologies & Treatments for Diabetes Congress reached consensus about the use of CGM.² Not only was the conclusion that the advanced CGM metrics should be considered as parameters that complement HbA_1c in clinical practice, they should also be recognized by governing bodies as valuable and meaningful in clinical trials of new drugs and devices for diabetes treatment. Moreover, hypoglycemia was mentioned as an important endpoint in clinical trials. To correctly detect episodes of hypoglycemia, the CGM accuracy is very important, and especially during conditions with fast declining PG resulting in iatrogenic hypoglycemia, low detection rates have been observed.^15,16 This is unfortunate when hypoglycemia endpoints have high impact in clinical trials. Another important research endpoint in clinical trials investigating fast-acting insulins is the postprandial glucose levels.¹⁷ The research endpoints will typically be calculated based on the first 0.5-1 hour and then a delay of around 10 minutes can be particular important. Device-related inaccuracies, such as, the filter routine delay and offset current might be possible to reduce, but the fundamental physiological delay, when using the interstitial fluid measuring site, cannot be removed.

This study sought to investigate (1) the delay issue of Dexcom G4 CGM measurements compared to PG in a clinical trial investigating the ultra-fast acting insulin Fiasp® and (2) how simple post-processing can be utilized to improve the associated clinical research endpoints.

Methods

Data Material

Data from a recent clinical trial conducted by Novo Nordisk A/S, to investigate the efficacy and safety of continuous subcutaneous insulin infusion of Fiasp® compared to NovoRapid® in 472 people with type 1 diabetes (T1D), were available for analysis. During three periods of 14 days, the subjects wore the CGM device Dexcom® G4 Platinum (DG4P, Dexcom, San Diego, CA, USA). They were instructed to calibrate the device according to manufacturer’s instructions. During two of the periods, the subjects underwent a standardized liquid meal test where six blood samples (–2, 30, 60, 120, 180 and 240 minutes after start) were drawn an analyzed by a central laboratory. Furthermore, they were instructed to perform 4 self-measured blood glucose (SMBG) measurements per day throughout the conduct of the trial and so-called 7-7-9 SMBG profiles (three consecutive days where SMBG was performed 7, 7, and 9 times, respectively) three times during the trial. Information about treatment was not used in this study.

Time Shifting CGM Measurements

Due to the physiological delay between interstitial glucose and PG, a negative time shift of the CGM measurements was proposed as an ultra-simple and quick solution to optimize the clinical research endpoints derived from the CGM measurements. The CGM measurements was time shifted for all subjects from −30 to 10 minutes by each minute to find the optimal time shift. The CGM measurements were then compared to the PG laboratory values and to SMBG measurements by the mean absolute difference (MAD).

Statistical Analyses

Subject characteristics are presented with means and standard deviations or percentages. Amount of glucose readings are presented as durations or counts. Number of hypoglycemic episodes are presented as the number of PG readings below or equal to 70 mg/dL. Symptomatic PG unconfirmed hypoglycemic episodes are not included in the number.

To find the most optimal time shift of the CGM measurements, MAD was calculated as the absolute difference between each CGM and PG measurement and between each CGM and SMBG measurement, which was then averaged by subject. The CGM measurements were interpolated linearly to get pairs of CGM and PG measurements and pairs of CGM and SMBG measurements. An interpolation was only carried out if the distance between the two adjacent CGM measurements were 5 minutes or less. The presented MAD was then the grand mean of the MADs calculated per subject. The procedure was done similarly for MARD with the addition that the absolute CGM-PG difference was divided by the PG value and the absolute CGM-SMBG difference was divided by SMBG value. To test the difference between MARD before and after time shifting, a paired t-test was performed.

Two typical clinical research endpoints are presented before and after time shifting the CGM measurements: (1) number of hypoglycemic episodes and (2) postprandial glucose increments (after meal tests).¹⁷ For hypoglycemia, the sensitivity and specificity of the CGM device’ ability to detect Level 1 (PG ≤ 70 mg/dL) and Level 2 (PG < 54 mg/dL) hypoglycemic episodes¹⁸ are presented. MAD between CGM and PG of postprandial glucose increments following the meal test is presented at the four postprandial timepoints for blood samples, 30, 60, 120, and 240 minutes.

The descriptive and inferential analyses were conducted in R version 2.15.2. The significance level was set at a P value of less than .05 for two-sided testing.

Results

Subject characteristics and glucose data are shown in Table 1. Slightly more males were enrolled and the subjects had an average age of 44 and were on average preobese as defined by WHO.¹⁹

Table 1.

Subject Characteristics at Randomization and Amount of Glucose Data Available.

Variable
n	472
Age, mean ± SD	44 ± 15
Sex (%)
Female	57
Male	43
Body mass index at baseline (kg/m2), mean ± SD	26 ± 4
HbA1c at baseline (%), mean ± SD	7.5 ± 0.5
Diabetes duration, mean ± SD	24 ± 12
CGM duration (hours)	1 005 425
Number of PG measurementsa	13 059
Number of SMBG measurements	38 182
Number of hypoglycemic episodesb	561

^aBlood samples analyzed by a central laboratory. ^bCount of PG measurements below or equal to 70 mg/dL.

In Figure 1, MAD between CGM and PG/SMBG measurements is shown as a function of time shift of CGM measurements. As can be observed there is a nadir at −9/−10 minutes, and the drop in MAD is most pronounced for SMBG.

Figure 1.

Mean absolute difference between CGM and PG and CGM and SMBG as a function time shifted CGM measurements. The nadir is at −9 minutes for PG and −10 minutes for SMBG.

Based on the above investigation of the time shift of CGM leading to lowest MAD, MARD for CGM and PG is shown in Table 2 where CGM_shift denotes CGM measurements shifted −9 minutes and -10 minutes. As is evident from the table, shifting the CGM measurements leads to a statistically significant 1.0%-points lower MARD with PG as reference and 1.8%-point lower MARD with SMBG as reference.

Table 2.

Change in Mean Absolute Relative Difference (MARD) After Shift of the CGM Measurements.

	CGM	CGMshift	P *
MARDa (%), mean ± SD	13.7 ± 7.2	12.7 ± 7.5	<.0001
MARDb (%), mean ± SD	13.1 ± 5.5	11.3 ± 5.8	<.0001

The Reference to CGM is PG and SMBG, Respectively. ^aPG is reference and CGM is shifted −9 minutes. ^bSMBG is reference and CGM is shifted −10 minutes. *Test was applied on MARDs per subject.

Clinical research endpoints for CGM and CGM_shift with PG as reference are shown in Table 3. For all endpoints, time shifting the CGM measurements leads to an improvement. The specificity of Level 2 hypoglycemia is unchanged to first decimal point though. The improvement is in terms of a higher sensitivity and specificity in the hypoglycemic endpoints. The sensitivity of Level 2 hypoglycemia is increased with more than 11%-points. For the postprandial glucose endpoints, the improvement is in terms of reductions in MAD. The improvement is most pronounced 30 minutes after the meal intake.

Table 3.

Changes in Clinical Research Endpoints After −9 minutes Shift of the CGM Measurements.

Research endpoint	CGM	CGMshift	Changeb
Level 1 hypoglycemia (PG ≤ 70 mg/dL)
Sensitivity	35.7%	41.6%	5.9% c
Specificity	97.5%	98.1%	0.6% c
Level 2 hypoglycemia (PG < 54 mg/dL)
Sensitivity	44.8%	56.2%	11.4% c
Specificity	99.4%	99.4%	+0.0% c
Postprandial glucose (mg/dL)
30 min MADa	27.9	23.0	–4.9
60 min MADa	30.8	27.6	–3.2
120 min MADa	32.3	29.2	–3.1
180 min MADa	26.7	23.8	–2.9

The Reference to CGM is the PG Measurements from Blood Samples. ^aMean absolute difference between CGM and PG postprandial glucose increments. Notice that the assumption is that the PG value and timing is the truth. ^bAll changes marked with bold are improvements. ^C%-point change.

Discussion

From Figure 1 it is evident that a delay of CGM measurements of 9-10 minutes compared to PG still exist for a newer CGM device used in a state-of-the-art clinical trial. From a direct measurement using a microdialysis catheter in the abdominal subcutaneous space, the physiological delay is, according to Basu et al,²⁰ 7-8 minutes in fasting patients with T1D. With the 9-10 minutes delay found in this study, the filter-routine delay is therefore no more than 1-3 minutes in this study of the Dexcom G4 CGM. The 9-10 minutes delay is in line with a mean delay of 9.5 minutes of the raw signal from a CGM sensor compared to SMBG reported by Schmelzeisen-Redeker et al.⁶ In another study by Sinha et al,⁷ the delay of a DG4P sensor was found to be 5.6 minutes. However, the population was healthy subjects, and due to the rapid changes in PG seen in people with diabetes, it is anticipated that a larger delay will be seen in people with diabetes compared to people without. In a study by Kuroda et al,²¹ the delay during a hyperglycemic clamp was investigated. They showed that the delay of CGM measurements to reach maximum glucose value was 27 minutes longer compared to blood glucose measurements, which indicates that during rapid glucose changes, the delay of CGM becomes larger, and at some point, the delay causes CGM to completely fail to resemble blood glucose. The bad performance during rapid changes is confirmed by Pleus et al.¹⁶ This property is especially important to keep in mind for clinical trials investigating bolus insulins where rapid blood glucose changes are often seen. The delay of −9 minutes found in this study will not be the same for other devices, which mean that different delays should be identified and used for post-processing of CGM measurements from other devices.

The observed MARD of approximately 14% with PG as reference in this study is similar to studies investigating MARD of the DG4P.^14,16 However, during rapid PG changes (<−3 mg/dL/min and >3 mg/dL/min) Pleus et al¹⁶ observed an increase of MARD to 25%, and the performance of the CGM sensor in this study is thus deemed good. Nevertheless, it was possible to obtain a 1.0%-point decrease in MARD, simply by shifting the CGM measurements −9 minutes post hoc. The effect of this time shift on the clinical research endpoints is significant with a more than 11%-point increase in the CGM detection of Level 2 hypoglycemia. The time shift did not only improve research endpoints on glucose excursions, but also improved postprandial glucose estimation for all blood sample time points. Better results might have been obtained with a personalized time shift of the CGM measurements as proposed by Schmelzeisen-Redeker et al,⁶ but to keep simplicity as focal point, we choose to suggest a general time shift approach.

A limitation of this study is that the CGM measurements are being post-processed to mimic blood glucose values. One could argue that interstitial glucose values themselves could have a stronger association to clinical outcomes, for example, late-diabetic complications. However, more evidence about such associations is needed. Another limitation is that measures of exercise were not obtained in the Onset® 5 trial. Exercise and movement have been shown to affect the performance of the CGM devices.²² However, since the participants wore the CGM devices at home, it is expected that the devices have been exposed to exercise in different everyday situations.

We acknowledge CGM as a tool for assessing glucose variability and for detection of glucose excursions where SMBG is inappropriate, for example, to detect nocturnal hypoglycemia. However, with the current low detection rate of hypoglycemia (sensitivity<50%, assuming that PG is the truth, value- and time-wise) and poor estimates of postprandial glucose increments, we recommend to use and interpret these research endpoints in clinical trials with care.

Conclusion

The delay in CGM is still an issue in a state-of-the-art clinical trial investigating an ultra-fast-acting bolus insulin in 472 subjects with T1D. The delay in this study was identified to be 9 minutes compared to PG. With a simple post-processing approach of time shifting the CGM measurements with −9 minutes, it was possible to obtain a statistically significantly lower MARD and subsequently obtain clinical research endpoints of improved validity.