Assessment of Glucose Control Metrics by Discriminant Ratio

Abstract

Objective: Increasing use of continuous glucose monitoring (CGM) data has created an array of glucose metrics for glucose variability, temporal patterns, and times in ranges. However, a gold standard metric has not been defined. We assess the performance of multiple glucose metrics to determine their ability to detect intra- and interperson variability to determine a set of recommended metrics.

Methods: The Juvenile Diabetes Research Foundation data set, a randomized controlled study of CGM and self-monitored blood glucose conducted in children and adults with type 1 diabetes (T1D), was used. To determine the ability of the evaluated glycemic metrics to discriminate between different subjects and attenuate the effect of within-subject variation, the discriminant ratio was calculated and compared for each metric. Then, the findings were confirmed using data from two other recent randomized clinical trials.

Results: Mean absolute glucose (MAG) has the highest discriminant ratio value (2.98 [95% confidence interval {CI} 1.64–3.67]). In addition, low blood glucose index and index of glycemic control performed well (1.93 [95% CI 1.15–3.44] and 1.92 [95% CI 1.27–2.93], respectively). For percentage times in glucose target ranges, the optimal discriminator was percentage time in glucose target 70–180 mg/dL.

Conclusions: MAG is the optimal index to differentiate glucose variability in people with T1D, and may be a complementary therapeutic monitoring tool in addition to glycated hemoglobin and a measure of hypoglycemia. Percentage time in glucose target 70–180 mg/dL is the optimal percentage time in range to report.

Introduction

Glucose variability describes within-day and between-day fluctuations in glucose concentration. It is elevated in people with type 1 diabetes (T1D) compared with people with normal glucose tolerance.¹ In vitro, animal, and human studies support a potential role for both hypo- and hyperglycemic excursions in promoting mitochondrial oxidative stress, secondary endothelial damage, accelerated atherosclerosis, impaired quality of life, and even mortality.^2–5 Increased glycemic variability (GV) may be associated with higher levels of markers of oxidative stress in type 2 diabetes⁶ and has been associated with increased mortality in sepsis⁷ and critical illness.⁸ Increased GV has also been associated with a higher incidence of severe hypoglycemia in people with T1D.¹

Currently available continuous glucose monitoring (CGM) devices provide glucose data every 5 min, and from these data, multiple indices of GV and other metrics of quality of glycemic control can be calculated. Commonly used GV metrics are standard deviation (SD), coefficient of variation (%CV), mean amplitude of glucose excursion (MAGE), mean of daily differences (MODD), average daily risk range (ADRR), glycemic variability percentage (GVP), mean absolute glucose (MAG), and continuous overlapping net glycemic action over an n-hour period (CONGAn). Quality of glycemic indices are: glycemic risk assessment diabetes equation (GRADE), index of glycemic control (IGC), high blood glucose index (HBGI), low blood glucose index (LBGI), personal glycemic state (PGS), and percentage time under, within and above specified glucose ranges.

Glycated hemoglobin (HbA1c) remains the reference marker for assessing glycemic control and predicting the risk of development of long-term complications, but has limitations, and targets may be individualized. To complement HbA1c, and in some cases replace it, there is consensus that CGM data should be considered to support glycemic management.⁹ In particular, the use of LBGI and HBGI metrics for glycemic control analysis along with different percentage times in range metrics has been advocated. For assessment of GV, the %CV has been proposed as the primary metric and SD as the secondary.⁹ However, the consensus assessed traditional glycemic metrics only—MAGE, %CV, and SD—and a wider range of metrics have been presented in the literature.

The advantages, limitations, and interrelationships among the GV metrics have been described previously.^10–14 However, a gold standard measurement has not been identified, limiting efforts to demonstrate a relationship between variability and clinically relevant micro- and macrovascular diabetes complications, and leading to heterogeneous study designs and outcomes.

Discriminant ratios assess the ability of a test to distinguish individual variation within a population and have been used previously to compare insulin sensitivity measures by calculating the ratio of the underlying between-subject standard deviation (SD_B) to the within-subject standard deviation (SD_w).^15,16

We have applied discriminant ratios to commonly used measures of glucose variability and quality of glycemic control to demonstrate which metric is the most effective at distinguishing between-subject glucose variability differences in a large population with T1D using CGM data. We have additionally evaluated the correlations between the considered metrics to describe interrelationships and to identify similar features within them.

Research Design and Methods

Data

Data from the Juvenile Diabetes Research Foundation (JDRF) CGM study were used. The JDRF data set is freely accessible and was obtained from the Jaeb Center for Health Research.¹⁷ The study was a 26-week randomized parallel group study evaluating the impact of CGM on glucose control in children and adults with T1D. The rules to standardize the analyzed data were 24 weeks of complete CGM data were used for each participant and incomplete day recordings due to sensor changes were allowed. Data were processed each 5 min and the lack of data in a period larger than 2 h was considered as a gap.

Finally, to confirm the results obtained with the JDRF data set, data from two more randomized clinical trials REPLACE-BG (N = 119)¹⁸ and I HART CGM (N = 40)¹⁹ were used.

Assessed metrics

The GV measures that were evaluated are average absolute rate of change (AARC),²⁰ CONGA₁,²¹ CV, GVP,²² J-index,²³ lability index (LI),²⁴ MAG,²⁵ MAGE,²⁶ MODD,²⁷ ADRR,²⁸ and SD.

Likewise, the assessed quality of glycemic control indices are GRADE,²⁹ %GRADE_Hypo,²⁹ %GRADE_Hyper,²⁹ glucose risk index (GRI),²⁸ HBGI,²⁸ LBGI,²⁸ M-value,³⁰ PGS,²² IGC,¹¹ the percentage of time between several ranges (%T 50–140, %T 70–180), the percentage of time <54 mg/dL (%T < 54) and 70 mg/dL (%T < 70), and the percentage of time >140 mg/dL (%T > 140) and 180 mg/dL (%T > 180). They were classified as metrics for hypoglycemia (LBGI, %GRADE_Hypo, %T < 54, %T < 70); metrics for overall glycemic control (GRI, M-value, IGC, GRADE, PGS, %T 50–140, %T 70–180); and metrics for hyperglycemia (HBGI, %GRADE_Hyper, %T > 140, %T > 180). All of these metrics are described in Supplementary Table S1. It is important to note that the M-value, PGS, and IGC were calculated using the default values of their parameters, that is, M-value₁₀₀, ICG₁.

All metrics were calculated for each complete participant's data set considering temporal windows of 12 days, previously described as the minimum duration of sensor data from which GV can be consistently assessed.³¹

Discriminant ratio

The discriminant ratio was used to compare the different metrics using the same glucose recordings to determine the ability of each metric to discriminate between different subjects (i.e., intersubject variability) and attenuate the effect of within-subject variation. The discriminant ratio methodology was chosen for this study since it compares different metrics measuring the same underlying physiological variable, determining the ability of a test to discriminate between subjects, and enabling comparison of discriminatory power between tests.^15,16 Importantly, the discriminant ratio is independent of the measurement error obtained within.

Discriminant ratios were calculated as the ratio of the unbiased between-subject standard deviation (SD_u) to SD_w according to the following equations³²:

$$DR=S{D}_u∕S{D}_w,$$ (1)

$$S{D}_u=\sqrt{{SD}_B^2-{SD}_w^2∕k)},$$ (2)

where SD_B is the between-subject standard deviation of the metric values and k is the number of replicate measurements performed in each subject (i.e., the number of temporal windows of 12 days considered). More details can be found in Appendix A1.

In addition to the discriminant ratio comparisons across the several glycemic metrics, the evaluation of the effect of the number of considered temporal windows k on the discriminant ratio results was also carried out. All data processing and calculations were done in Matlab 2017a.

Data analysis

Results were statistically evaluated with nonparametric methods because of the non-normal distribution of data. Kruskal–Wallis test and then a post hoc analysis corrected by Bonferroni for multiple comparisons were carried out to identify the differences between the discriminant ratios of the evaluated metrics. To strengthen the findings, the discriminant ratio results obtained with the JDRF data set were compared with the results obtained with the REPLACE-BG/I HART CGM data set by means of the Mann–Whitney U-test. A P-value of <0.05 was considered to indicate statistical significance. Data analysis was performed in SPSS.

To assess the effect of k value consideration, a statistical comparison was carried out between the results obtained with different number of windows (k = 3, 4, …, 24). The discriminant ratio was considered constant from the k value whose results were statistically different from the immediately lower k results and not significantly different from the results of the subsequent higher k values. Correlation between metrics was assessed by Spearman's correlation coefficient due to the non-normality of the data.

Results

In total, 179 participants were included in the main analysis (55.3% males; age: 24.12 ± 14.60 years; HbA1c: 7.44% ± 0.88%; T1D duration: 14.07 ± 12.39 years).

Discriminant ratio values for each of the evaluated metrics are given in Tables 1 and 2. For the evaluated GV metrics, the MAG has the highest discriminant ratio value and is statistically significantly greater than the other metrics (P < 0.001) except for the GVP (P = 0.430), indicating that both metrics may be the most unbiased and most effective at distinguishing between variability differences across individuals. However, the LI and J-index also showed no significant differences from GVP (P = 0.989 and P = 0.977, respectively).

Table 1.

Discriminant Ratio of Glucose Variability Metrics

Metrics	DR
MAG	2.98 (1.64–3.67)
GVP	2.20 (1.39–3.01)
LI	2.11 (1.20–3.10)
AARC	1.95 (1.41–2.64)
J-index	1.85 (1.34–2.59)
ADRR	1.81 (1.40–2.28)
CONGA1	1.73 (1.37–2.18)
SD	1.62 (1.21–1.99)
MAGE	1.56 (1.22–1.93)
MODD	1.47 (1.11–1.98)
CV	1.33 (1.08–1.63)

AARC, average absolute rate of change; ADRR, average daily risk range; CONGA₁, continuous overlapping net glycemic action over an 1-hour period; CV, coefficient of variation; DR, discriminant ratio; GVP, glycemic variability percentage; LI, lability index; MAG, mean absolute glucose; MAGE, mean amplitude of glucose excursion; MODD, mean of daily differences; SD, standard deviation.

Table 2.

Discriminant Ratio of Glucose Control Quality Indices

Metrics for hypoglycemia		Metrics for overall glycemic control		Metrics for hyperglycemia
Metrics	DR	Metrics	DR	Metrics	DR
%T < 50 mg/dL	2.33 (1.06–6.35)	M-value	2.00 (1.30–2.94)	HBGI	1.83 (1.34–2.72)
%T < 54 mg/dL	2.15 (1.08; 5.28)	IGC	1.92 (1.27–2.93)	%T > 180 mg/dL	1.59 (1.27–2.09)
LBGI	1.93 (1.15–3.44)	GRI	1.87 (1.32–2.72)	%T > 160 mg/dL	1.56 (1.32–1.97)
%GRADEhypo	1.79 (1.04–4.46)	PGS	1.71 (1.44–2.02)	%T > 140 mg/dL	1.54 (1.29–1.85)
%T < 70 mg/dL	1.51 (1.07–2.77)	GRADE	1.67 (1.30–2.27)	%GRADEhyper	1.50 (1.24–1.86)
		%T 70–180 mg/dL	1.63 (1.31–2.02)
		%T 70–160 mg/dL	1.63 (1.31–2.02)
		%T 54–180 mg/dL	1.61 (1.32–2.09)
		%T 70–140 mg/dL	1.58 (1.36–1.85)
		%T 54–140 mg/dL	1.61 (1.32–2.09)

GRADE, glycemic risk assessment diabetes equation; GRI, glucose risk index; HBGI, high blood glucose index; IGC, index of glycemic control; LBGI, low blood glucose index; PGS, personal glycemic state.

In Figure 1, the mean of discriminant ratios versus the number of considered windows is represented to assess the effect of parameter k and the vulnerability of the metrics. MAG converges faster to a stable value at k = 8 compared with the slightly slower convergence of GVP at k = 9 (P < 0.001) and M-value₁₀₀ and J-index at k = 13 (P = 0.012 and P = 0.019, respectively). That means that MAG is less vulnerable to the effect of interday variability and is a better discriminator.

FIG. 1.

Discriminant ratio versus temporal length consideration (number of weeks) used to obtain the calculations. Only the metrics that have the highest DR are represented. ADRR, average daily risk range; DR, discriminant ratio; GRI, glucose risk index; GVP, glycemic variability percentage; HBGI, high blood glucose index; IGC, index of glycemic control; LBGI, low blood glucose index; LI, lability index; MAG, mean absolute glucose.

These findings were supported by the second analysis carried out with the REPLACE-BG and IHART CGM data. In this analysis, 155 participants were included (52.20% males; age 44.78 ± 14.37 years; HbA1c 7.16% ± 0.73%; T1D duration 24.48 ± 11.84 years). Figure 2 shows the discriminant ratio values obtained for both studied populations, and numerical values are presented in Supplementary Table S2. Results obtained for each metric are similar to those derived with the JDRF data set. The discriminant ratio values are slightly higher in the analysis carried out with REPLACE-BG/IHART CGM population but these differences are not statistically significant.

FIG. 2.

Median of discriminant ratio corresponding to the evaluated glucose variability metrics (top) and glucose control indices (bottom). The glucose control indices are divided into metrics for hypoglycemia, metrics for overall glycemic control, and metrics for hyperglycemia. AARC, average absolute rate of change; CONGA, continuous overlapping net glycemic action; CV, coefficient of variation; GRADE, glycemic risk assessment diabetes equation; JDRF, Juvenile Diabetes Research Foundation; MAGE, mean amplitude of glucose excursion; MODD, mean of daily differences; PGS, personal glycemic state; SD, standard deviation.

For glucose control indices, LBGI has the highest value for discriminant ratio of the metrics for hypoglycemia. It is not statistically different from %GRADE_Hypo (P = 0.477); however, considering k screening, a greater robustness of LBGI is demonstrated as it is constant from k = 9.

In addition, percentage time <50 and 54 mg/dL showed the highest discriminant ratio values of the metrics for percentage time hypoglycemia assessment; however, the values of both metrics were concentrated around similar values describing a left skewed distribution. They are, therefore, relatively less affected by variability and it may not be appropriate to compare percentages of time to %GRADE_Hypo and LBGI.

The discriminant ratio values for LBGI and %GRADE_Hypo were not statistically different between the two analyzed databases (P = 0.662 and P = 0.182 for LBGI and %GRADE_Hypo, respectively).

M-value₁₀₀ and IGC₁ are the metrics for overall glycemic control with the highest values of discriminant ratio. Both are not significantly different from each other (P = 0.143). However, IGC₁ is constant from k = 8, whereas M-value₁₀₀ was constant from k = 12. The discriminant ratio of GRI is not statistically different from IGC₁ (P = 0.709); however, GRI is not stable at k = 13. Therefore, IGC₁ may be preferable to M-value₁₀₀ and GRI as it requires less data to become stable.

Finally, HBGI is the metrics for hyperglycemia that shows higher discriminant ratio values and is constant from k = 13.

Several percentage times in ranges indices were evaluated separately. Based on these results, the optimal ones are the commonly used percentage of time between 70 and 180 mg/dL followed by the range defined by 70–160 mg/dL without a statistically significant difference between both ranges (P = 0.197).

The interrelationships between the metrics were assessed and the values are given in Supplementary Table S3. For different evaluated thresholds (th), the metrics that were strongly correlated with percentage times in ranges, %T(<th), and %T(>th) are summarized in Table 3.

Table 3.

Spearman's Correlation Coefficient for Metrics Correlated with Times in Ranges, Percentage of Times in Hypoglycemia (BG < th), and Percentage of Times in Hyperglycemia (BG > th)

	% time in ranges
	(54–140)	(54–180)	(70–140)	(70–180)
M-value	−0.63	−0.83	−0.70	−0.88
J-index	−0.85	−0.92	−0.84	−0.87
HBGI	−0.87	−0.95	−0.86	−0.90
GRI	−0.77	−0.93	−0.82	−0.95
GRADE	−0.93	−0.95	−0.93	−0.91
IGC	−0.47	−0.68	−0.59	−0.79
PGS	−0.72	−0.87	−0.80	−0.92
%GRADEeu	0.55	0.70	0.59	0.73
	% time below ranges
	<50	<54	<70
LBGI	0.77	0.82	0.96
%GRADEhypo	0.73	0.76	0.74
	% time above ranges
	>140	>160	>180
J-index	0.84	0.90	0.94
HBGI	0.87	0.93	0.97
GRI	0.70	0.80	0.86
GRADE	0.89	0.93	0.93
PGS	0.64	0.74	0.79
%GRADEhyper	0.59	0.61	0.60

Results showed a high correlation between the percentage time in ranges and M-value₁₀₀, J-index, GRI, GRADE, PGS, and IGC₁. Nevertheless, the relationship with IGC₁ is weaker with the narrowest range of time (i.e., %T 50–140 mg/dL). The same phenomenon occurred with PGS, but less markedly since it also depends on other parameters. Therefore, the customization of range in both IGC₁ and PGS (penalty scores) is the determinant that defines the relative influence of hypo- and hyperglycemia in the outcomes.

The Spearman coefficient (r_s) is greater for the range 70–180 mg/dL and this reinforces its potential as the optimal definition of time in range in line with the discriminant ratio analysis.

For percentages time in hypoglycemia, LBGI and %GRADE_Hypo were the glycemic control metrics with the greatest hypoglycemia correlation.

For percentage times in hyperglycemia, J-index, HBGI, and GRADE were the most highly correlated. A threshold of 180 mg/dL to define hyperglycemia presented the best coefficients, again reinforcing this as the ideal threshold for hyperglycemia.

For variability metrics, MAG was correlated with LI (r_s = 0.87); CONGA with LI (r_s = 0.89); and GPV with AARC (r_s = 0.9). MAGE, MODD, and ADRR were not correlated with percentages time in ranges, suggesting that they are able to describe glucose variability independently of the glucose distribution.

Discussion

The results reported here describe the ability of different GV metrics and glucose control quality indices to discriminate between individuals. MAG, a GV metric that is the measure of glucose rate of change over time, has the highest discriminant ratio, whereas LBGI and IGC₁ have the higher discriminant ratio values for measures of glucose control quality. These results differ from a previous study³³ where it was suggested that MAG may not be an ideal reference for assessment of GV based on the correlation between MAG and other GV metrics as MAGE, SD, interquartile range, and CONGA. However, that study did not differentiate between variability metrics and quality of control indicators and compared MAG (a variability metric) with control quality indicators. That work also concluded that MAG represents a valid GV metric if closely spaced sensor glucose measurements (i.e., CGM) are used. This is supported by the results of an analysis of the normative values for MAG derived from CGM data in subjects without diabetes.³⁴

LBGI has been supported by an analysis of glucose variability metrics in children using principal components analysis,³⁵ despite the underestimation of the low-risk range of LBGI seen when CGM measurements are used.³⁶ This underestimate of hypoglycemia risk with CGM, compared with blood glucose, could be due to fewer values in the range, the sampling differences, the plasma–interstitium glucose transport, or the intrinsic characteristics and artifacts of CGM technology.

For percentage time in glucose target range, the range of 70–180 mg/dL is the most effective at discriminating between individuals. To specifically assess percentage time in hypoglycemia, the International Hypoglycemia Study Group-defined threshold of 54 mg/dL³⁷ appears to discriminate effectively when percentage time below threshold is assessed. These results strengthen support for the standardization of the times in ranges metrics as 70–180, <54, and >180 mg/dL.

In the intercorrelation analysis, although percentage of time in ranges has the theoretical limitation that it assigns the same penalty score to all glucose values, calculations suggested that simple indices such as times in ranges may be as informative as the “risk index” criteria, as previously proposed.³⁸ However, the percentage times in ranges showed higher vulnerability to intra- and interpatient variability than the risk indices with lower values of discriminant ratio.

This study suggests that the best metrics—based on the discriminant ratio values—should be considered depending on researchers' hypotheses (GV or glycemic control quality), rather than a blanket approach to assess all available metrics.

The limitations of this study include the exclusion of glycemic profiles that include episodes of severe hypoglycemia. This may result in a bias in the analysis of the percentage of time <50 and 54 mg/dL due to relatively fewer samples in these ranges. An analysis with data from studies that include people with even higher risk of hypoglycemia could reinforce the results of both these metrics, but the sensitivity analysis includes participants in the I HART CGM study that recruited people with impaired awareness of hypoglycemia or a recent history of severe hypoglycemia.

Another limitation would be the reproducibility of results using other distinct data sets. Although we tried to ensure that the considered data set represents a wide group of people with T1D, the discriminant ratio results may be different if other studies including diverse participants were considered.

Different approaches to determine a gold standard for measuring GV have been explored.³⁹ A classifier system based on different machine learning algorithms trained with physician expertise, which rates GV on a scale of low, borderline, high, or extremely high, has been proposed. This system gives an overall idea of the glucose management, including only the MAGE and SD as variability metrics along with other typical metrics of temporal signal processing. Others have assessed the predictive value of glucose variability metrics in predicting hypoglycemia, and concluded that, of the variability metrics, only CV correlated with hypoglycemia, and that LBGI had the strongest relationship with exposure to hypoglycemia.⁴⁰ Future study may include expanding similar approaches with all available metrics (and considering MAG, IGC₁, and LBGI, which have higher discriminant ratio values), and glucose outcomes.

Conclusion

The increasing use of CGM enables calculation of glucose metrics such as mean glucose, glucose management index, and percentage time in range. These are accessible and easily understood but may not be sensitive to variability in glucose; therefore, considering other nonconventional metrics, which have been evaluated in this study, may draw a better picture of the management of the disease. There is potential benefit from standardizing metrics to assess variability, widening the range of applicability of the available metrics, and to measure the impact of an intervention, especially in clinical research.

We propose that MAG, as the most discriminant metric, is used as the glucose variability metric for people with T1D. MAG may be used alongside HbA1c and a measure of hypoglycemia to assess the impact of any intervention on glycemic control. The optimal metric for time in hypoglycemia is time spent <54 mg/dL, in line with the International hypoglycemia study group recommendation, and the most robust metric for measuring percentage time in glucose target range is 70–180 mg/dL, in line with the consensus statement.⁴¹

Appendix A1

Appendix A1

Glucose temporal signal is divided into k windows or replicas in each subject; the variability metrics are calculated for each window in each patient. Then the following matrices are defined for each of the evaluated glycemic variability metrics and glucose control quality indices:

$$X=\left[\begin{array}{ccc}{x}_{11}& \dots & x_{1k}\\ ⋮& \ddots & ⋮\\ x_{n1}& \dots & x_{nk}\\ \end{array}\right],$$ (3)

$$X_{mean}=\left[{\sum }_{j=1}^k\frac{x_{1j}}{k}{\sum }_{j=1}^k\frac{x_{2j}}{k}\dots {\sum }_{j=1}^k\frac{x_{nj}}{k}\right],$$ (4)

where $X\in R^{nxk}$, being the i-th row the i-th subject, and the j-th column the j-th window. X is the metrics considered and $X_{mean}$is the mean of the metric obtained for each subject considering all windows.

Particularly, from mentioned matrices, the within-subject standard deviation (SD_w) and between-subject standard deviation (SD_B) are calculated for each metrics as follows:

$$S{D}_{w_i}=\sqrt{{\sum }_{j=1}^k\frac{{\left(x_{ij}-x_{mea{n}_i}\right)}^2}{k}},$$ (5)

Therefore, Equations (1) and (2) are rewritten as follows:

$$D{R}_i=S{D}_{u_i}∕S{D}_{w_i},$$ (7)

$$S{D}_{u_i}=\sqrt{}{SD}_B^2-{SD}_{w_i}^2∕k).$$ (8)

The absolute value of discriminant ratio is the median of discriminant ratios across the subjects for each metrics.

Author Disclosure Statement

M.R. has received honoraria for advisory board participation from Dexcom and Roche Diabetes. N.O. has received honoraria for speaking and advisory board participation from Abbott Diabetes, Dexcom, Medtronic Diabetes, and Roche Diabetes.

Funding Information

This research was supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Imperial College London. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.

Supplementary Material

Supplementary Table S1

Supplementary Table S2

Supplementary Table S3