Comparing the Validity of Three Methods for Identifying Impaired Awareness of Hypoglycemia in Predicting Severe Hypoglycemia in Adults with Type 1 Diabetes Mellitus: The PR-IAH Study

Abstract

Objective

Impaired awareness of hypoglycemia (IAH) contributes to severe hypoglycemia (SH) in adults with type 1 diabetes mellitus (T1DM). This study compared the validity of the Gold, Clarke, and Pedersen-Bjergaard methods for predicting SH in Japanese adults with T1DM.

Methods

IAH was assessed at baseline using three methods, and a prospective cohort study was conducted in adults with T1DM. Multivariate Cox proportional hazards regression models adjusted for covariates were used to compare the three methods for predicting SH, and diagnostic validity was calculated.

Patients

We enrolled 286 participants (mean age: 50.5±14.6 years, men: 36.7%, diabetes duration: 17.6±11.1 years, mean HbA1c level: 7.7±0.9%).

Results

The prevalence of IAH identified using the Gold, Clarke, and Pedersen-Bjergaard methods was 12.2%, 19.2%, and 30.1%, respectively. The Clarke method showed the strongest association with SH [adjusted hazard ratio (aHR), 8.27; 95% confidence interval (CI), 3.43-20.0, p＜0.001], whereas the Gold and Pedersen-Bjergaard methods had associations of aHR 1.90 (95% CI: 0.67-5.36, p=0.227) and aHR 2.65 (95% CI: 1.16-6.03; p=0.020), respectively. The Clarke method demonstrated 65.2% sensitivity, 84.8% specificity, a 27.3% positive predictive value, a 96.5% negative predictive value, a positive likelihood ratio of 4.29, a negative likelihood ratio of 0.41, and an overall diagnostic validity of 83.2%.

Conclusion

Among the three methods, the Clarke method demonstrated the highest validity for predicting SH development. This information may assist physicians in assessing IAH in clinical practice.

Introduction

To achieve optimal glycemic control, individuals with type 1 diabetes mellitus (T1DM) require intensive insulin regimens, which often lead to insulin-induced hypoglycemia (1). Hypoglycemia has a profound impact on patients and their families, causing distress, medication nonadherence, and disruptions to daily life and work. It contributes to costly emergency department visits, hospitalizations, morbidity, and mortality (2-4). Recurrent episodes can lead to the loss of characteristic warning symptoms and diminished counterregulatory hormone response, a condition known as impaired awareness of hypoglycemia (IAH) (5,6).

Although advancements in continuous glucose monitoring (CGM) and continuous subcutaneous insulin infusion (CSII) have helped mitigate hypoglycemia in individuals with T1DM, they remain a significant burden despite widespread adoption of these technologies (7,8). In addition, IAH remains prevalent and is associated with an increased risk of severe hypoglycemia even among CGM users (9). Although evaluating IAH in clinical practice is challenging, the Gold, Clarke, and Pedersen-Bjergaard methods are often used for the assessment (10-12). However, which of these methods is the most effective in predicting severe hypoglycemia (SH) remains unclear. The primary objective of the PR-IAH study was to examine the impact of IAH on SH development.

In the present analysis, we compared the predictive validity of these three methods for SH in adults with T1DM.

Materials and Methods

This observational study was approved by the National Hospital Organization (NHO) Central Research Ethics Committee (R2-0117002) (13) and conducted in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology guidelines.

Participants and settings

Between February 2020 and October 2021, we enrolled adults with IAH from NHO collaboration centers in Japan. Seven institutions participated in this study: the NHO Mie Hospital, Kyoto Medical Center, Osaka National Hospital, Himeji Medical Center, Hyogo-Chuo National Hospital, Okayama Medical Center, and Kokura Medical Center.

The inclusion criteria were T1DM (14), diabetes duration ≥1 year, age ≥20 years old, and attendance at a collaborating center. The exclusion criteria were non-insulin therapy, anti-dementia drug use, and inappropriate use, as judged by the research director or coordinators. Furthermore, participants who did not respond to the Gold, Clarke, or Pedersen-Bjergaard methods were excluded.

Diabetic complications and diabetes treatment

Diabetic retinopathy, nephropathy, and peripheral neuropathy (DPN) were treated by certified diabetologists according to treatment guidelines for diabetes in 2018-2019. Diabetic retinopathy was assessed by an ophthalmologist using retinal photography. Retinopathy was classified as absent, simple, pre-proliferative, or proliferative. Nephropathy was classified into stages 1-5 based on the estimated glomerular filtration rate, presence of albuminuria, or hemodialysis stage (15). DPN was considered present following the criteria after the diagnosis of diabetes, and polyneuropathy was excluded, excluding diabetic polyneuropathy, which was determined to be positive in the presence of at least two of the following criteria: 1) subjective symptoms (numbness, pain, or dysesthesia in the bilateral lower extremities); 2) decreased or absent bilateral Achilles tendon reflexes; and 3) diminished bilateral vibratory sensation in the malleolus medialis (＜10 s using a tuning fork at 128 Hz) (16). Clinical data, including the age, body mass index (BMI), duration of diabetes, HbA1c levels, diabetes treatment (such as CSII), and monitoring methods (such as CGM), were extracted from medical records.

IAH and SH

IAH was assessed using the Gold, Clarke, and Pedersen-Bjergaard questionnaire. The Gold method used a single question (“Do you know when your hypoglycemias are commencing?”) rated on a seven-point Likert scale (from 1=“always aware” to 7=“never aware”), with a score of ≥4 indicating IAH. The Clarke method comprises eight questions that assess the glycemic threshold at which individuals detect hypoglycemia symptoms and examine their exposure to moderate and severe hypoglycemia episodes, with a score of ≥4 suggesting IAH. The Pedersen-Bjergaard method uses a single question (“Can you feel when your blood sugar is low?”) with possible answers of “always,” “usually,” “occasionally,” or “never.” Responses of “occasionally” or “never” indicated IAH. SH was defined as an event requiring assistance from another person to actively administer glucose or glucagon or to take corrective action (17), with confirmation through medical records.

Sample size

A minimum sample size of 200 participants (40 participants with IAH and 160 without IAH) was required to achieve a significance level of 5% and a power of 0.8, assuming an estimated IAH prevalence of 20% and a medium effect size of 0.5.

Data analyses

Qualitative variables were analyzed using Fisher's exact test, whereas quantitative variables were compared using either Student's t-test or one-way analysis of variance with Tukey's post hoc test for multiple comparisons. To assess the association between binary and continuous variables, the point-biserial correlation coefficient was calculated using the ltm package in R. The phi correlation coefficient (Φ), a measure of the association between two binary variables, was used to assess the correlation between methods. A Kaplan-Meier survival analysis model was created to investigate the association between IAH, based on each method, and the development of SH, with the log-rank test used to assess differences between groups. Univariate and multivariate Cox proportional hazards models were used to analyze the association between the variables of interest and SH. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), diagnostic validity, and positive and negative likelihood ratios (LRs) for each method. The LR is the ratio of the probability of the test result among patients who truly have the disease to the probability of the same test result among those who do not have the disease. In particular, the LR is the ratio of sensitivity to (1 - specificity). Therefore, it is independent of disease prevalence. The magnitude of the LR reflects the certainty of a positive diagnosis. Generally, an LR value of 1 indicates that the test result is equally likely in patients with and without the disease, while ＞1 suggests that the test result is more likely in patients with the disease, and ＜1 indicates that the test result is more likely in patients without the disease (18). A receiver operating characteristic (ROC) analysis was employed to determine the optimal cutoff values and area under the curve (AUC) for each score in predicting the development of SH within two years. An AUC ≥0.9 was considered excellent, 0.8 to ＜0.9 very good, 0.7 to ＜0.8 good, 0.6 to ＜0.7 sufficient, 0.5 to ＜0.6 poor, and ＜0.5 as not useful for diagnostic purposes.

Statistical significance was defined as p＜0.05. Patients with missing data were excluded from the analysis. The analysis was performed using the R software program, version 4.1.2.

Results

Participants and IAH

Of the 341 patients screened for eligibility, 55 were excluded: 27 due to a diagnosis of type 2 diabetes mellitus, 27 for missing questionnaire data, and 1 for other reasons, resulting in 286 patients included in the study (Fig. 1). The study included 286 Japanese adults with T1DM (mean age: 50.5±14.6 years old, men: 36.7%, diabetes duration: 17.6±11.1 years, mean HbA1c level: 7.7±0.9%). The prevalence of IAH in this cohort, identified using the Gold, Clarke, and Pedersen-Bjergaard methods at baseline, was 12.2%, 19.2%, and 30.1%, respectively. Our results suggested that the IAH classification may be discordant among the three methods (Fig. 2). The Φ was 0.5 for the Gold and Clarke methods, 0.6 for the Gold and Pedersen-Bjergaard methods, and 0.7 for the Clarke and Pedersen-Bjergaard methods. The overall dropout rate was 5.9%.

Figure 1.

STROBE flow diagram: PR-IAH study.

Figure 2.

Venn diagram showing detailed agreement among the three methods (n=286).

Clinical characteristics at baseline

The SH group had a longer duration of diabetes and a higher prevalence of diabetic neuropathy than did the non-SH group. In addition, the SH group showed a higher prevalence of IAH if identified using the Clarke and Pedersen-Bjergaard methods, but not the Gold method. At baseline, participants with IAH, as classified by the Gold, Clarke, and Pedersen-Bjergaard methods, showed no significant differences in the age, sex distribution, diabetes duration, BMI, HbA1c levels, or prevalence of diabetic complications, including retinopathy, nephropathy, and neuropathy (Table 1). Similarly, the proportions of CGM (54.3%, 56.4%, and 57.0%; p=0.979) and CSII use (22.9%, 23.6%, and 36.0%; p=0.215) did not differ significantly among the 3 groups.

Table 1.

Baseline Characteristics of All Participants or Individuals with IAH Defined by the Gold, Clarke, and Pedersen-Bjergaard Criteria.

Variables	All (n=286)	Gold method (n=35)	Clarke method (n=55)	Pedersen-Bjergaard method (n=86)	p value
Age, years	50.5±14.6	52.9±15.7	52.8±15.1	49.6±15.4	0.371
Male sex, %	36.7	40.0	36.4	30.2	0.552
Diabetes duration, years	17.6±11.1	19.1±12.4	17.6±11.8	16.7±11.4	0.605
BMI, kg/m2	23.4±3.7	23.3±3.3	22.9±4.4	23.1±3.6	0.928
HbA1c, %	7.7±0.9	7.6±1.1	7.5±1.0	7.5±0.9	0.921
Diabetic Complication, %
Retinopathy	23.1	31.0	20.4	24.1	0.563
Nephropathy	18.2	20.6	14.8	18.8	0.727
Neuropathy, %	14.4	28.6	26.5	13.2	0.077
Treatment
CSII	36.7	22.9	23.6	36.0	0.215
SAP	21.7	17.1	18.2	26.7	0.397
TDD/BW	0.6±0.2	0.7±0.2	0.6±0.2	0.6±0.2	0.256
Anti-hypertensive drug	22.7	22.9	20.0	22.1	0.942
Cholesterol-lowering drug	26.6	34.3	25.5	22.1	0.379
Monitoring
CGM usage	56.3	54.3	56.4	57.0	0.979
isCGM usage	32.9	31.4	36.4	26.7	0.450
rtCGM usage	23.4	22.9	20.0	30.2	0.390

Mean±standard deviation or percent (%).

BMI: body mass index, BW: body weight, CSII: continuous subcutaneous insulin infusion, isCGM: intermittently scanned continuous glucose monitoring, rtCGM: real-time continuous glucose monitoring, SAP: sensor-augmented pump, TDD: total daily dose

Kaplan-Meier and hazard ratios (HRs)

The 1- and 2-year survival rates according to the Gold method were 0.987 [95% confidence interval (CI): 0.961-0.996] and 0.937 (95% CI: 0.895-0.962), respectively, in the non-IAH group and 0.968 (95% CI: 0.792-0.995) and 0.903 (95% CI: 0.729-0.968), respectively, in the IAH group (log-rank=2.8, p=0.100). The 1- and 2-year survival rates according to the Clarke method were 0.995 (95% CI: 0.968-0.999) and 0.971 (95% CI: 0.936-0.987), respectively, in the non-IAH group and 0.939 (95% CI: 0.822-0.980) and 0.766 (95% CI: 0.616-0.864), respectively, in the IAH group (log-rank=36.1, p＜0.001). The 1- and 2-year survival rates according to the Pedersen-Bjergaard method were 0.995 (95% CI: 0.962-0.999) and 0.960 (95% CI: 0.918-0.981), respectively, in the non-IAH group and 0.962 (95% CI: 0.887-0.988) and 0.871 (95% CI: 0.772-0.928), respectively, in the IAH group (log-rank=5.3, p=0.020) (Fig. 3).

Figure 3.

Kaplan-Meier curve according the three methods.

Diabetes duration (HR; 1.05, 95% CI: 1.02-1.09, p=0.005), diabetic neuropathy (HR: 3.22, 95% CI: 1.35-7.68, p=0.008), and IAH identified by the Clarke (HR: 8.85, 95% CI: 3.75-20.91, P＜0.001) and Pedersen-Bjergaard methods (HR: 2.54, 95% CI: 1.12-5.75, p=0.026) were associated with SH, whereas IAH identified by the Gold method was not (HR: 2.29, 95% CI: 0.84-6.20, p=0.105). After adjusting for diabetes duration and diabetic neuropathy, IAH identified by the Clarke (HR: 8.27, 95% CI: 3.43-20.0, p=0.001) and Pedersen-Bjergaard methods (HR: 2.65, 95% CI: 1.16-6.03, p=0.020) remained associated with SH, whereas IAH identified by the Gold method did not (Table 2).

Table 2.

Predictors of Severe Hypoglycemia, Univariable and Multivariable Hazard Ratio.

Variables	Univariable HR (95%CI)		Multivariable HR (95%CI)
	Hazard ratio (95% CI)		Gold method	Clarke method	Pedersen-Bjergaard method
Age, 10 years	1.22 (0.92-1.63)
Male sex	0.89 (0.34-2.10)
Diabetes duration, 10 years	1.05 (1.02-1.09)*		1.04 (0.997-1.08)	1.04 (1.00-1.08)*	1.04 (0.997-1.08)
BMI, kg/m2	0.98 (0.88-1.09)
HbA1c, %	0.85 (0.53-1.37)
Diabetic Complication
Retinopathy	2.38 (0.55-10.29)
Nephropathy	2.15 (0.87-5.29)
Neuropathy	3.22 (1.35-7.68)*		2.13 (0.82-5.55)	1.45 (0.55-3.81)	2.46 (0.96-6.32)
IAH
Gold	2.29 (0.84-6.20)		1.90 (0.67-5.36)
Clarke	8.85 (3.75-20.91)*			8.27 (3.43-20)*
Pedersen	2.54 (1.12-5.75)*				2.65 (1.16-6.03)*
Treatment
CSII	0.45 (0.17-1.23)
SAP	0.54 (0.16-1.81)
TDD/BW	0.52 (0.07-3.67)
Anti-hypertensive drug	1.57 (0.64-3.82)
Cholesterol-lowering drug	2.13 (0.92-4.92)
Monitoring
isCGM usage	0.65 (0.24-1.77)
rtCGM usage	0.70 (0.24-2.07)

Hazard ratio (95% confidence interval). *p<0.05.

BMI: body mass index, BW: body weight, CI: confidence interval, CSII: continuous subcutaneous insulin infusion, isCGM: intermittently scanned continuous glucose monitoring, rtCGM: real-time continuous glucose monitoring, SAP: sensor-augmented pump, TDD: total daily dose

Validity of three methods for predicting SH

The Gold method demonstrated 21.7% sensitivity, 88.6% specificity, a 14.3% PPV, a 92.8% NPV, a 1.91 positive likelihood ratio (LR+), a 0.88 negative likelihood ratio (LR-), and 83.2% overall diagnostic validity. The Pedersen-Bjergaard method demonstrated 52.2% sensitivity, 71.9% specificity, a 14.0% PPV, a 96.5% NPV, a 4.29 LR+, a 0.41 LR-, and 83.2% overall diagnostic validity.

The correlation between the development of SH and the severity of IAH, as identified by the Gold, Clarke, and Pedersen-Bjergaard methods, was r=0.14 (p=0.024), 0.38 (p＜0.001), and 0.115 (p=0.060), respectively.

The Clarke score demonstrated the highest predictive validity for SH, with an AUC of 0.771 (95% CI: 0.683-0.859). The optimal cutoff value was 4 points, yielding a sensitivity of 55.6% and a specificity of 86.3%. In contrast, the Gold and Pedersen scores showed poor predictive performance, with AUCs of 0.596 (95% CI: 0.498-0.695) and 0.575 (95% CI: 0.484-0.666), respectively. The optimal cutoff value for both scores was 2 points, with the Gold score achieving a sensitivity of 69.4% and specificity of 44.6% and the Pedersen score a sensitivity of 41.7% and specificity of 71.2%.

Discussion

This is the first study to compare the validity of the Gold, Clarke, and Pedersen-Bjergaard methods in predicting SH in a cohort study of Japanese adults with T1DM. Our findings revealed discrepancies in IAH classification among the three methods, with the Clarke method showing higher validity in predicting SH, whereas the Gold and Pedersen-Bjergaard methods were less effective.

In the present study, the prevalence of IAH identified by the Gold, Clarke, and Pedersen-Bjergaard methods was 12.2%, 19.2%, and 30.1%, respectively, with the Gold method showing the lowest proportion and the Pedersen-Bjergaard method showing the highest. A meta-analysis of 62 studies from 21 countries, including 39,180 participants (19,304 with T1DM and 14,650 with type 2 diabetes mellitus), reported a pooled prevalence of IAH of 26.2% (95% CI: 22.9-29.9%), 23.2% (95% CI: 18.4-29.3%), and 58.5% (95% CI: 53.0-64.6%) using the Gold, Clarke, and Pedersen-Bjergaard methods, respectively (19).

However, only a few studies have directly compared the three methods. A study by Geddes et al. reported IAH prevalence rates of 24%, 26%, and 62.5% using the Gold, Clarke, and Pedersen-Bjergaard methods, respectively (20). Similarly, Ghandi et al. found a prevalence of 19%, 18%, and 61% using the same methods (21). In both studies, the Pedersen-Bjergaard method identified a significantly higher prevalence of IAH than the Gold and Clarke methods, whereas the prevalence estimates from the Gold and Clarke methods were relatively similar.

In the present study, the prevalence of IAH was the highest using the Gold method, followed by the Clarke and Pedersen-Bjergaard methods. Compared with previous studies, the finding that the Pedersen-Bjergaard method had the highest prevalence is consistent; however, the reason for the lower proportion observed with the Gold method remains unclear. Japanese individuals may have difficulty in making judgments using the seven-item method, which requires further investigation in future studies.

In our study, DPN was associated with an increased risk of SH. Cross-sectional and observational studies have indicated that DPN is a neuropathy, and gastroparesis is associated with SH (22-26). However, Olsen et al. reported that IAH was not associated with autonomic DPN in adults with T1DM (27). Conversely, Flatt et al. reported that peripheral neuropathy was more prevalent in patients with SH than in those without SH at a 24-month follow-up in the HypoCOMPASS study (39% vs. 4.7%, respectively) (28). However, the discrepancy between these results is unknown. The diagnostic criteria, ethnicity, and differences in the population with diabetes may explain this finding.

The use of the CSII or CGM was not associated with SH in the present study. However, CGM usage resulted in greater detection of hypoglycemia, although the clinical significance of this finding remains unclear. The Hypo-METRICS study, which included 276 individuals with T1DM, investigated the frequency and duration of sensor-detected hypoglycemia and its correlation with person-reported hypoglycemia using blinded CGM over 10 weeks. The results showed that approximately 50% of the CGM-detected hypoglycemic episodes were asymptomatic, even at levels ＜54 mg/dL, whereas many reported symptomatic episodes occurred at levels ＞70 mg/dL. In clinical practice, these two measures cannot be used interchangeably and should be documented and addressed (29).

Limitations

The strengths of this study included its observational design and diagnostic validity. According to McGee's rule for simplifying the use of LRs (18), an increase in LR+ from 2 to 4 increases the probability by approximately 10%, whereas a decrease in LR- from 0.5 to 0.4 increases the probability by 15-20%. Based on the LR, the Clarke method was found to be superior to the Gold and Pedersen-Bjergaard methods.

However, several limitations associated with the present study warrant mention. We did not assess hormonal and symptomatic responses to hypoglycemia (30), hypoglycemia subtypes (31), or the hypo-AQ tool. Although the Gold, Clarke, and Pedersen-Bjergaard methods are useful, they do not meet the U.S. Food and Drug Administration (FDA) standards, which require validation and reproducibility of data. Therefore, Dr. Speight developed the Hypo-AQ tool that meets the FDA standards. In addition, Pedersen-Bjergaard et al. updated their method to assess hypoglycemia awareness in individuals with T1DM (32). Further research is required to confirm these results. Finally, this study excluded individuals with insulin-treated type 2 diabetes mellitus, limiting its generalizability.

Overall, this prospective cohort study found that among the three methods, the Clarke method was the most accurate in predicting SH, whereas the Gold method performed poorly. The Gold method comprised a single question, but participants may have struggled to respond using a seven-point scale. In contrast, the Pedersen-Bjergaard method offers four response options, making it easier to use; however, it tends to overestimate SH. From the perspective of IAH severity, the Clarke method demonstrated the highest validity for predicting SH. In contrast, the Gold method showed low specificity, indicating that while a positive result suggests a possible risk of SH, there remains a considerable likelihood of false positives, warranting careful interpretation. Conversely, the Pedersen method exhibited low sensitivity, implying a higher risk of false negatives. Thus, caution is required when interpreting negative results. This information may assist physicians in evaluating IAH in clinical practice.

Informed consent or substitute was obtained from all patients for inclusion in the study.

The authors state that they have no Conflict of Interest (COI).

Funding Statement

This study was funded by the National Hospital Organization Clinical Research (NHO) [grant number: H31-NHO (Endocrinology and Nephrology)-01].