How to compare algorithms for automated insulin delivery using different sensors?

Recent caution has been advised against comparing automatic insulin delivery (AID) systems that use different sensors. ¹ , ² While the same two AID systems (Tandem Control IQ (CoIQ) and Minimed 780 G (MM780G)) have shown clear differences in time in range (TIR) outcomes, studies report no clinically relevant difference in HbA1c ³ , ⁴ or only minimal changes in HbA1c and TIR. ⁵ , ⁶ A simplified overview of these four studies is presented in Table 1. Of note, all TIR values are numerically higher in persons using MM780G (from 2.4% to 7.0% points), but all HbA1c values are minimally lower in persons using CoIQ (about 0.3% points). Tree studies ³ , ⁴ , ⁶ provide information on sensor mean glucose or glucose monitoring indicator (GMI) ⁷ over a 2‐week period and HbA1c, which can be translated into estimated p‐glucose from the ADAG equation (A1c‐derived average glucose). ⁸ Although this indirect approach contains several pitfalls, the result may indicate differences in sensor performances. Specifically, the Dexcom sensor connected to CoIQ appears to measure glucose levels higher than the Guardian sensor connected to the MM780G. This observation aligns with a recent small‐scale study in which persons used both sensors simultaneously, reporting a slight positive bias for the Dexcom and a negative bias for the Guardian sensor when compared to self‐monitored blood glucose. ⁹

TABLE 1.

An overview of four studies comparing HbA1c and CGM data follow‐up values for two different AID systems.

	CoIQ	MM780G	p‐value
Henry, 3 Retrospective observational study Follow‐up time: 90 days
Number of participants	64	167
HbA1c %	6.96	7.05	NS
TIR % (CGM sampling minimum 2 weeks)	67.0	73.77	<0.001
Glucose monitoring indicator (GMI) (%)	7.14	6.89	<0.01
Mean sensor glucose from GMI (mmol/L)	8.9 a	8.3 a	<0.01
Mean P‐glucose estimated from HbA1c (mmol/L)	8.5 b	8.6 b	NS
Piona, 4 Cross‐sectional study
Number of participants	284	329
HbA1c %	6.7	6.9	<0.001
TIR % (CGM sampling 2 weeks)	68.2	75.2	<0.001
Mean sensor glucose (mmol/L)	8.9	8.2	<0.001
Mean P‐glucose estimated from HbA1c (mmol/L)	8.1 b	8.4 b	<0.001
Singh, 5 Retrospective observational study Follow‐up time: median 1.7 years
Number of participants	92	50
HbA1c %	6.7 c	6.8 c	NA
TIR % (CGM sampling up to 4 weeks, minimum 5 days)	69.2 c	74.5 c	NA
Beato‐Vibora, 6 Prospective observational study Follow‐up time: 3 months
Number of participants	75	75
HbA1c %	6.56	6.88	NS
TIR % (CGM sampling 2 weeks)	74.99	77.38	NS
Mean sensor glucose (mmol/L)	8.2	7.9	NS
P‐glucose estimated from HbA1c (mmol/L)	7.8	8.3	NS

^a Calculated from: GMI (%) = 0.02392 × mean sensor glucose (mg/dL) + 3.3.

^b Calculated from Glucose (mmol/L) = 1.59 × HbA1c (%) – 2.59.

^c Calculated from original data of baseline and change. Changes from baseline were compared but not the follow‐up values.

[Correction added on 28 February 2025, after first online publication: The study by Beato‐Vibora⁶ has been corrected to ‘Prospective observational study’ in this version.]

This discrepancy between sensor‐driven and sensor‐independent glycaemic metrics indicates the influence of varying study populations and sensor performance across brands. However, does this necessarily imply that the insulin delivery algorithm and continuous glucose monitoring (CGM) data hold no significance?

Consider a scenario in which an insulin pump using algorithm 1 with sensor A is compared with an insulin pump using algorithm 2 with sensor B. Suppose users of algorithm 1 achieve a TIR of 68% and a sensor‐based mean glucose of 9.0 mmol/L, while a comparable group of users of algorithm 2 achieves a TIR of 75% and a mean glucose of 8.5 mmol/L, yet with a similar HbA1c. In such cases, TIR and sensor mean glucose metrics may be misleading from a clinical perspective, as sensor A likely measures glucose levels higher than sensor B. If algorithm 1 receives glucose input from sensor B, this is a new product with unknown results requiring new validation.

The goal of developing algorithms for insulin delivery is to enhance safety, minimize the risk of hypoglycaemia, ensure patient adherence and increase TIR, guided by input from the specific sensor. In this context, algorithm 2 is more effective than algorithm 1. A 7 percentage point difference in TIR could translate into a meaningful difference in the risk of diabetic complications. ¹⁰ The apparent ‘rescue’ of algorithm 1 due to differences in sensor performance fails to capture the fundamental intellectual and clinical objectives underlying algorithm design.

Sensor performance has traditionally been reported as the mean absolute relative difference (MARD) between a sensor and a reference glucose concentration from a blood sample. There is a growing understanding that a similar and low MARD value does not guarantee the absence of any clinically relevant differences between sensors. ² , ¹¹ One of the drawbacks of MARD is that this metric does not provide information about the direction of any deviation from the reference value. ¹²

To reliably evaluate the clinical performance of different AID systems based solely on CGM data, an internationally accepted standardization of glucose sensors is essential, with adherence mandated for manufacturers. ¹³ , ¹⁴ , ¹⁵ However, this standardization process is complicated by the lack of a definitive reference value for interstitial glucose and the inherent difficulties in estimating plasma glucose from interstitial glucose under varying physiological conditions. ¹³

While we fully agree that sensor standardization is essential, and HbA1c is still needed for clinical assessment, we believe that insulin delivery algorithms per se can still be evaluated effectively with sensor‐based data despite differences in sensor performance. In other words, the insulin delivery algorithm, as a data engineering product, can be evaluated in its own right from the sensor data that served as input during its development and optimisation process.

CONFLICT OF INTEREST STATEMENT

KWH has received grants for an investigator‐initiated study from Abbott Diabetes Care and Novo Nordisk.

PEER REVIEW

The peer review history for this article is available at https://www.webofscience.com/api/gateway/wos/peer‐review/10.1111/dom.16234.

Hansen KW. How to compare algorithms for automated insulin delivery using different sensors? Diabetes Obes Metab. 2025;27(5):2319‐2321. doi: 10.1111/dom.16234

DATA AVAILABILITY STATEMENT

Data sharing is not applicable to this article as no new data were created or analyzed in this study.