First Accuracy and User-Experience Evaluation of New Continuous Glucose Monitoring System for Hypoglycemia Due to Hyperinsulinism

Abstract

Introduction:

Patients with congenital hyperinsulinism (HI) require constant glucose monitoring to detect and treat recurrent and severe hypoglycemia. Historically, this has been achieved with intermittent self-monitoring blood glucose (SMBG), but patients are increasingly using continuous glucose monitoring (CGM). Given the rapidity of CGM device development, and increasing calls for CGM use from HI families, it is vital that new devices are evaluated early.

Methods:

We provided two months of supplies for the new Dexcom G7 CGM device to 10 patients with HI who had recently finished using the Dexcom G6. Self-monitoring blood glucose was performed concurrently with paired readings providing accuracy calculations. Patients and families completed questionnaires about device use at the end of the two-month study period.

Results:

Compared to the G6, the G7 showed a significant reduction in mean absolute relative difference (25%-18%, P < .001) and in the over-read error (Bland Altman +1.96 SD; 3.54 mmol/L to 2.95 mmol/L). This resulted in an improvement in hypoglycemia detection from 42% to 62% (P < .001). Families reported an overall preference for the G7 but highlighted concerns about high sensor failure rates.

Discussion:

The reduction in mean absolute relative difference and over-read error and the improvement in hypoglycemia detection implies that the G7 is a safer and more useful device in the management of hypoglycemia for patients with HI. Accuracy, while improved from previous devices, remains suboptimal with 40% of hypoglycemia episodes not detected.

Introduction

Congenital hyperinsulinism (HI) is a rare disorder of glucose metabolism with an incidence varying by country but established as at least 1:28 000 in the United Kingdom. ¹ The cause is variable but, most commonly, patients’ disease is caused by mutations in the adenosine triphosphate (ATP)-sensitive K channel encoded by ABCC8/KCNJ11, resulting in the uncoupling of glucose levels from insulin secretion. Several mutations in other genes such as GLUD1 and GCK have also been identified adding to genetic heterogeneity. Hyperinsulinism may also be due to perinatal stress with associated transient HI. ² Regardless of the cause, patients with HI experience recurrent and severe hypoglycemia secondary to dysregulated and excessive insulin secretion and suffer brain injury with incidence ranging from 15% ³ to almost 50%. ⁴

Medical management of HI is complex, and treatment with medications is often insufficient to completely prevent ongoing hypoglycemia. ⁵ Patients and their families thus rely on the timely detection and treatment of hypoglycemia through self-testing of glucose and administration of carbohydrate. The current standard of care remains intermittent, self-monitoring of blood glucose (SMBG) with fingerprick capillary blood samples. ⁶ However, this method risks missing hypoglycemia between tests and provides no trend information to allow for prediction of upcoming hypoglycemia. As such, continuous glucose monitoring (CGM) is emerging as a promising solution for glucose monitoring in HI.^7,8

Despite 20 years of CGM use by people living with diabetes, CGM provision has been much slower for people living with HI. The reasons for this are multifactorial but are centered around a manufacturer focus on high prevalence disease such as Type 1 Diabetes. Over the last five to six years, CGM has been increasingly used within the HI community, and results have been variable. Accuracy has remained much lower than for those with diabetes from small, early studies ⁹ to more recent, larger-scale studies^10,11 with hypoglycemia sensitivity rates below 50%. While studies have not been able to demonstrate clear evidence of hypoglycemia reduction through simple provision of CGM, ¹² there has been success with using CGM as a phenotyping tool in HI^13,14 and then using this new knowledge to inform algorithmic support to support behavior change. ¹⁵

Regardless of the current lack of firm evidence for hypoglycemia reduction through use of CGM in HI, devices are largely popular ¹⁶ and there has been increasing demand for CGM provision from patient advocacy groups.^17,18 With advancing CGM technologies and availability of a new generation of devices (eg, Dexcom G6 followed by Dexcom G7), HI patients and families using CGM now face a wider choice but lack any information about comparative performance and long-term user insight.

Aims

This study intended to provide an early evaluation of the G7 device in patients with HI to provide initial data to inform patient and clinician choice.

The aims of the study were:

1. Undertake an accuracy analysis of the G7 CGM device.

   1. Provide direct comparison of individual patient accuracy between the G6 and G7.

2. Evaluate patient and family perceptions of the G7 device when used for an extended period, in comparison with the G6 device.

Methods

Patients were identified from a database of a separate national study, providing G6 devices to patients with HI. Patients, with a diagnosis of HI, who had been using a G6 device for at least two months and had been contributing data for accuracy analyses, were eligible for inclusion. Ethical approval was obtained from the University of Manchester and the Health Research Authority of the National Health Service (REC reference 07/H1010/88) to transfer patients at the time of previous study discontinuation. G7 CGM devices and two months’ supplies were provided to participants, free of charge. Spare supplies were available for patients experiencing device failure. Patients and their families were asked to perform at least four SMBG checks per day using a Contour Next One glucometer and also at any time that CGM reported hypoglycemia (glucose < 3.5mmol/L).

Patients provided CGM and SMBG data uploads through proprietary software (Dexcom Clarity) every 10 days and returned devices at the end of the two-month study period. Online questionnaires were designed to ascertain patient and family opinions on the G7 device and how it compared to the G6. Questions were based on findings from previous qualitative studies^16,19 and piloted with the research team and representatives from the Children’s Hyperinsulinism Charity. At the end of the study, patients and families were sent a link to the questionnaire which they completed anonymously.

Additional data from G6 devices were downloaded from Clarity for the same patients enrolled in this study. Thus both G6 and G7 CGM groups contained the same participants, using the G6 followed by the G7 device. Data were processed using bespoke code in Python 3.1.1; this was written by the research team with the express purpose of performing this accuracy analysis. For both groups (G6 and G7), CGM and SMBG values within five minutes of one another were paired in a comprehensive process described elsewhere ¹⁰ and evaluated for routine measures of accuracy as well as plotted on the Hypoglycemia Error Grid (HEG) ¹⁰ to provide a clinical context to errors. Bland Altman analysis was calculated to provide a visual representation of the degree of agreement between the two measures as well as the directional trend of any error. Hypoglycemia detection was defined as any CGM value <3.5mmol/L within a 15-minute window either side of an SMBG <3.5mmol/L. Mean absolute relative difference (MARD) was calculated for both groups using all pooled data and then compared. Hypoglycemia sensitivity (when SMBG value <3.5 mmol/L, nearest paired CGM also <3.5 mmol/L) and hypoglycemia detection (when SMBG <3.5 mmol/L, at least one CGM value within a 30-minute window also <3.5 mmol/L) were calculated for individual patients and then compared between G6 and G7 groups.

Results

Twenty patients with HI using G6 were identified from the study database. Twelve patients and families were eligible for inclusion and participated. One patient disengaged with the study and was lost to follow-up; therefore, 11 patients contributed CGM data to the study. One patient’s SMBG data were not useable for accuracy analyses as their glucometer had been used interchangeably to check glucose levels in the patient and a sibling. Paired CGM versus SMBG values were thus available for accuracy analyses from 10 patients who had used both the G6 (in the previous study) and the G7 (in this study). All 12 families (including the one lost to follow-up) completed the online questionnaire.

Demographics

Mean (range) age of participants was 3 years 10 months (5 months - 8 years 8 months) with prior CGM (G6) duration being between 4 and 12 months. Ten (83%) patients were female, one patient was Black African, six were White British and five were Asian. All patients had a confirmed diagnosis of HI and were receiving disease-modifying therapy with diazoxide or somatostatin analogue. Six of 12 patients had an inactivating pathogenic variant in ABCC8 or KCNJ11 and the remainder were negative on 13-gene HI panel testing.

Patients contributed a mean 55 days of G7 CGM data and spent a mean (range) of 3.3% (0.2%-10.5%) time hypoglycaemic (<3.5 mmol/L), experiencing a mean of 1.0 hypoglycaemic events per day, lasting mean (SD) 43 (46) minutes; hypoglycemia was consistent with other U.K. studies of patients with HI. ¹⁴

CGM Device Accuracy

Over the two-month study period using the G7, patients performed a mean 4.7 SMBG measurements per day, resulting in 2834 SMBG values, 2124 of which were paired with a CGM value within the specified five-minute window. Comparisons were made with the 7452 paired CGM versus SMBG values available for the same patients when they were using the same glucometer but a G6 device in a previous study.

Bland Altman Analysis (Figure 1) shows the reduction in the mean error of the G6 (+0.46 mmol/L) down to an almost neutral mean error (+0.05 mmol/L) in the G7. The +1.96 SD upper limit is also significantly lower for the G7 compared to the G6 (2.97 vs 3.54 mmol/L). There was a small increase in the −1.96 SD lower limit in the G7 compared to the G6 (−2.86 vs −2.63 mmol/L). Combined, these results demonstrate less tendency of the G7 receiver to over-read (provide a higher value) compared to the G6, as well as a reduction in the magnitude of this over-read error.

Figure 1.

Bland Altman plots for both devices (G6 left, G7 right) versus SMBG values from the contour next one.

The tighter upper limit and reduced mean of the G7 can be appreciated from these graphs.

The G7 showed improved performance over the G6 in hypoglycemia detection with rates of 62% versus 42% (P < .001 on chi-square) as well as improvements in hypoglycemia sensitivity and MARD (Table 1). When paired values are plotted on the Hypoglycemia Error Grid, the G7 demonstrates significantly lower clinical risk in its errors (Figure 2, Table 1).

Table 1.

Accuracy Measures for the G6 and G7 CGM Devices and Percentage of Values Within Each Area of the HEG.

	G6, %	G7, %	P value
MARD	25.7	18.9	<.001
Hypo sensitivity	32.2	46.2	.002
Hypo detection	42.2	62.2	<.001
Hypo specificity	88.1	90.3	.009
HEG A	86	92	<.001
HEG B	5.6	4.4
HEG C	8.2	3.6
HEG D	0.2	0.0

Abbreviations: HEG, hypoglycemia error grid (Figure 2); MARD, mean absolute relative difference; hypo, hypoglycemia (<3.5mmol/L)

P value of < .05 is considered significant. P values are calculated by chi-square tests.

Figure 2.

Hypoglycemia error grid plotted with SMBG values paired with G6 CGM values (left) and G7 CGM values (right).

Zone A = no risk, Zone B = slight risk, Zone C = moderate risk, Zone D = severe risk. It can be appreciated that a significantly higher number of values lie within Zone A on the G7 plot. There are also significantly fewer missed hypoglycemias with the G7.

Patient and Family Experience of G7 Versus G6

All 12 families responded to the questionnaire asking for opinions on the G7 versus the G6 and responses were varied. Overall, the G7 system was preferred, with six respondents reporting that sensor changes were simpler and less painful with the G7 (four preferred the G6 and two reported no difference). The G7 receiver device was also preferred over the G6 (six vs two families [four no preference]) due to the smaller size and improved battery life. When asked about accuracy, responses were mixed, but overall, the G7 was perceived to be more accurate with median score of 4 on Likert scale (Figure 3).

Figure 3.

Perception of CGM device accuracy by families. While opinions varied, overall families felt the G7 to be more accurate.

However, there was an overwhelming report of high sensor failure rate with the G7 device with eight out of 12 families reporting that the G7 was significantly worse than the G6 in this regard. Additional supplies had to be sent to these eight families and, despite this, failure rates were so high that in many cases, two months of data were not possible. Many families reported either complete failure of the G7 sensors to start-up or that they failed after three to four days and had to be changed. This rate of device failure was not predicted and, therefore, prospective data on the failure rate was not collected.

When families were asked which device they preferred overall, the G7 was marginally more popular than the G6 (seven vs four [one no preference]). Reasons for choosing the G7 included better accuracy, less painful sensor insertions, quicker warm up time, and better receiver battery. For those that preferred the G6, this was due to the high sensor failure rate of the G7.

Discussion

This study provides the first evaluation of both the accuracy and user experience of the G7 for patients with HI, shortly after device release. While G6 and G7 data were not collected simultaneously, patients were identical; thus, release of this comparative data with an older generation of CGM device provides up to date evidence upon which the HI community can base decisions around provision for clinical care and monitoring within therapeutic trials.

Overall analytic and clinical accuracy of the G7 was greater than that of G6 in this patient group but remains significantly below values reported for children ²⁰ and adults ²¹ living with diabetes. ²¹ Mean absolute relative difference was improved with the G7 but, more importantly, hypoglycemia sensitivity and detection rates showed a significant improvement. The reduction in mean error and +1.96 SD value on Bland Altman analysis indicates a reduction in the magnitude of over-read errors and explains the improved hypoglycemia detection rate of the G7. This was achieved without any decrease in specificity of hypoglycemia detection. Analysis with HEG showed an important improvement in the clinical accuracy of the G7 over the G6, with a significant reduction in missed hypoglycemia values. The combined accuracy results strongly suggest that the G7 offers an improved safety profile over the G6 hypoglycemia detection, intrinsic to the management of HI.

Patient experience of the G7 device was largely positive and based on the perception of improved accuracy and simpler and less painful sensor insertions. However, patients and families experienced significant problems with sensor failure. Early release devices, such as those used in this study, used an algorithm that has since been updated, and it is possible that the updated algorithm can reduce sensor failure and improve accuracy. Anecdotal reports from families support this suggestion that the supply of devices later in the study, from a different batch, lasted longer and failed less; however, this remains unproven.

A limitation of this study is that all participants were already using CGM at the time of recruitment and thus were likely to have positive bias in their opinion of the technology. As other studies ¹² have indicated, a more comprehensive evaluation may report negative bias to balance out the positive recruitment bias in this relatively small patient group with HI.

Overall, the G7 demonstrates an improvement over the G6 and is likely to be preferred by most patients and families with HI as well as those using CGM for performance evaluation within therapeutic trials. However, point accuracy and missed hypoglycemia remain suboptimal for use of CGM as a standalone hypoglycemia tool. This study highlights that, despite performance improvements, CGM does not replace SMBG and use is likely to be helpful only for those receiving training on appropriate use or algorithmic, interpretative support. Nevertheless, the improvement in the newer generation of CGM provides an optimistic outlook for progressive point accuracy to consider future use of CGM as an integral part of the clinical management of hypoglycemia due to HI and within therapeutic trials. Further studies, incorporating more patients, updated algorithms and blinded CGM groups would contribute significantly to the knowledge base of CGM in HI and should form a priority for upcoming research.