Validity of diabetes self-reports in the Women's Health Initiative: comparison with medication inventories and fasting glucose measurements

Abstract

Objective

Although diabetes is conveniently assessed by self-report, few validation studies have been performed. Therefore, we studied whether self-report of prevalent and incident diabetes in Women's Health Initiative (WHI) participants was concordant with other diagnostic evidence of diabetes.

Study Design and Setting

A total of 161 808 postmenopausal women aged 50–79 were enrolled at 40 clinical centers in the U.S. in 1993–1998 and followed prospectively. At baseline, prevalent medication treated diabetes was defined as a self-report of physician diagnosis and treatment with insulin or oral antidiabetic drugs. During followup, incident treated diabetes was defined as a self-report of a new physician diagnosis of diabetes treated with insulin or oral drugs. Diabetes self-reports were compared with medication inventories and fasting glucose levels at baseline and during follow-up.

Results

At baseline, self-reported treated diabetes was concordant with the medication inventory in 79% of clinical trial, and 77% of observational study participants. Self-reported incident treated diabetes was concordant with the medication inventory in 78% between baseline and Year 1 in the clinical trials, in 62% between Year 1 and Year 3 in the clinical trials, and in 72% between baseline and Year 3 in the observational study. Over similar periods, 99.9% of those who did not report treated diabetes had no oral antidiabetic drugs or insulin in the medication inventory. At baseline, about 3% not reporting diabetes had fasting glucose >126 mg/dl, and 88% of these subjects subsequently reported treated diabetes during 6.9 years of follow-up.

Limitations

Incident self-reported diabetes treated by lifestyle alone was not determined in WHI. Medication inventories may have been incomplete and fasting glucose may have been lowered by treatment; therefore, concordance with self-reported treatment or fasting glucose ≥ 126 may have been underestimated.

Conclusion

In the WHI, self-reported prevalent and incident diabetes was consistent with medication inventories, and a high proportion of those with undiagnosed diabetes subsequently reported diabetes treatment. Self-reports of ‘treated diabetes’ are sufficiently accurate to allow use in epidemiologic studies.

Introduction

The United States is experiencing an epidemic of type 2 diabetes, a major medical problem associated with significant morbidity, mortality, and cost [1–3]. The prevalence of diabetes rises sharply with each decade of age in post-menopausal women, and surpasses that in men after the 7th decade of life [4,5]. Thus, epidemiologic studies of diabetes in older women are of particular interest.

The Women's Health Initiative (WHI) enrolled over 161 000 postmenopausal women aged 50 to 79 years in its clinical trials and observational study [6,7]. Given its large size and extensive data characterizing the participants, WHI represents an unparalleled resource for further investigation of the effects of dietary, hormonal, and other influences on diabetes in older women. In WHI, oral glucose tolerance tests were not performed, and fasting glucose was measured only in a relatively small random sample of participants. Therefore, diabetes prevalence and incidence throughout the study has been assessed primarily on the basis of self-report by the participants.

A number of population-based studies conducted in the U.S and Europe have addressed the validity of self-reported diabetes. Confirmations of self-reported diabetes by comparison to medical records have ranged widely from 64% to 98%. [8–12] Differences in confirmation rates may be due to the completeness of obtaining medical records, the question used to elicit self-report, and the characteristics of the study participants that may have contributed to awareness of diabetes. In this article, we compare WHI participant self-report of ‘treated diabetes’ both at baseline and during follow-up with data from medication inventories and fasting glucose measurements in order to determine whether self-report is a valid measure of prevalent and incident diabetes.

Methods

The WHI recruited postmenopausal women aged 50–79 at 40 clinical centers in the U.S in 1993–1998. The WHI enrolled 68 132 participants in two randomized controlled trials, the Dietary Modification Trial (DMT) and the Hormone Trial (HT). In the HT, 10 739 post-hysterectomy women were randomly assigned to receive either placebo or conjugated equine estrogen 0.625 mg/day (E-alone), and 16 608 women with a uterus at baseline were randomly assigned to receive either placebo or conjugated equine estrogen 0.625 mg/day plus medroxyprogesterone acetate 2.5 mg/day (E+P). Another 93 676 women were enrolled into an observational study (OS). The design of the WHI and baseline characteristics of the WHI participants have been described. [13–15] The protocol and consent forms were approved by the institutional review boards for all participating institutions.

At baseline, participants were asked if a physician had ever told them that they had ‘sugar diabetes or high blood sugar’ when they were not pregnant. Follow-up questions asked about age at diagnosis, hospitalization for diabetic coma, dietary treatment, history of treatment with oral medications, and past and current treatment with insulin shots. Prevalent diabetes at baseline was defined as a self-report of ever having received a physician diagnosis of sugar diabetes when not pregnant. Prevalent medication treated diabetes (‘treated diabetes’) was defined as the subset of participants with prevalent diabetes, who reported ever taking insulin shots or pills.

Women who reported diabetes onset at age 21 or less, and who were currently taking insulin were considered to have probable type 1 diabetes and were excluded from the DMT, as were women who reported a history of hospitalization for diabetic coma [7]. There were no such exclusions from the HT or OS, so a small proportion of diabetes present at baseline may have been type 1.

At each semi-annual or annual contact, all participants were asked, ‘Since the date given on the front of this form, has a doctor prescribed any of the following pills or treatments?’ Choices included ‘pills for diabetes’ and ‘insulin shots for diabetes’. Thus, only incident treated diabetes was ascertained, and this was defined as a self-report of a new physician diagnosis of diabetes treated with oral drugs or insulin. During WHI follow-up, participants were not asked about new-onset diabetes treated with lifestyle measures alone. We analyzed incident treated diabetes reported as of the termination dates for the HTs (July, 2002 for the E+P trial and February 29, 2004 for the E-alone trial), and through December, 2004 for the DMT and OS.

Participants in the clinical trials were instructed to bring all current medications in their original containers, both prescription and nonprescription, to the baseline WHI clinic visit and again after 1 and 3 years of follow-up. The product or generic name of the medications on the label was entered into the study database and matched to the corresponding item in a pharmacy database: the Master Drug Data Base (MDDB: Medi-Span, Indianapolis, IN). A similar procedure was followed for the OS, except that the medication inventories occurred only at baseline and Year 3.

All women enrolled in the clinical trials had blood drawn after an overnight 12 h fast at baseline. Glucose was measured in a randomly selected sample of 5884 specimens (8.6% for the HTs, 4.5% for the Diet Modification Trial and 1% for the OS) was analyzed for glucose in the central laboratory using the hexokinase method on the Hitachi 747 (Boehringer Mannheim Diagnostics, Indianapolis, Indiana). The random sampling procedure was stratified by age, clinical center, hysterectomy status, and ethnicity. Minority women were oversampled to make up 50% of the sample. In addition, as part of an ancillary study to the OS, stored baseline plasma samples were analyzed for glucose on 1584 cases of incident treated diabetes and 2198 controls who did not report incident diabetes. [16]

We prepared descriptive tables of baseline characteristics and self-reported diabetes prevalence and incidence. Annual incidence of self-reported treated diabetes was calculated by dividing the number of incident cases by the total person-years of follow-up of the entire sample. We compared prevalent and incident diabetes self-reports to the medication inventories, reporting the proportion of women with self-reported diabetes who had a diabetes medication in the inventory. We also computed the proportion of women, without self-report diabetes who had no diabetes medication in the inventory.

For clinical trial participants, we examined the proportion of participants with a diabetes medication in the relevant inventory who reported prevalent treated diabetes at baseline, incident treated diabetes in the interval between baseline and the Year 1 visit, and incident treated diabetes in the interval between the Year 1 and Year 3 visit. Since there was no medication inventory in the observational study at Year 1, incident self-reports in the interval between baseline and Year 3 were examined. Self-reports of treated diabetes were compared with medication inventory in WHI as a whole, by study component, and by randomized treatment. We tested differences between the intervention and control arms of the clinical trials by chi-square tests. We examined the distribution of glucose values measured at baseline among randomly sampled women with and without self-reported prevalent diabetes. We also examined the distribution of glucose values measured at baseline among women with and without self-reported incident treated diabetes. The latter analyses were done combining the randomly sampled participants and those selected for the case-control ancillary study of incident diabetes. [16]

Results

Description of cohort, diabetes prevalence and diabetes incidence

The characteristics at baseline of the cohort by key selected variables are summarized in Table 1. The mean age was ∼63 years and about 18% of the women were from ethnic minority groups. Ethnicity was similar in the sub-cohorts except for the E-alone trial, which had a lower proportion of women of white non-Hispanic origin. Women in the OS had the highest level of education, were the leanest, had the lowest intake of calories and fat, and had the highest level of physical activity. About 30–40% of the women in each study component had hypertension and a family history of diabetes, but the highest proportion was found in women in the E-alone trial, who also had the highest mean body mass index.

Table 1.

Characteristics of each sub-cohort by key selected variables

Variable	E+P	E-alone	DMT	OS
	N = 16608	N = 10739	N = 48835	N = 93676
Age at baseline (mean, yrs)	63.27	63.59	62.26	63.61
Race/ethnicity (%)
American Indian	0.34	0.70	0.42	0.45
Asian/Pacific Islander	2.19	1.53	2.27	2.85
Black	6.77	15.06	10.78	8.15
Hispanic	5.35	6.10	3.80	3.87
White	83.97	75.26	81.41	83.28
Unknown	1.40	1.36	1.32	1.40
College graduate (%)	34.86	23.90	38.10	41.98
BMI (mean, kg/m2)	28.48	30.08	29.12	27.26
Hypertension (%)	29.96	40.34	35.48	33.47
High cholesterol requiring pills (%)	12.70	15.20	11.98	15.04
Current smoking (%)	10.47	10.48	6.73	6.27
Dietary intake (Mean)
Total energy intake (kcal/d)	1673.24	1652.72	1789.36	1572.46
Energy from fat (%)	33.24	34.22	37.77	30.17
Physical activity (mean, METs/wk)	11.51	9.67	10.08	13.69
Family history of diabetes (%)	33.51	39.39	35.08	33.08

E+P (Estrogen plus Progestin Trial), E-alone (Estrogen-alone Trial), DMT (Diet Modification Trial), OS (Observational Study), BMI (body mass index), MET (metabolic equivalent task).

In each study component, the prevalence of self-reported diabetes at baseline was ∼6%, with the exception of the E-alone trial, in which it was 9.5% (Table 2). Medication treated diabetes at baseline represented ∼75% of all self-reported cases of diabetes at baseline in each study component. The E-alone study also had the highest incidence of self-reported, medication treated diabetes during follow-up (annual incidence of 1.14% and a cumulative incidence of 8.8%), while the lowest incidence was seen in the OS (annual incidence 0.64% and cumulative incidence 4.6%) and E+P trial (annual incidence 0.50% and cumulative incidence 3.8%). In the DMT, the annualized incidence of medication-treated diabetes was 0.89% and cumulative incidence was 6.9%. The differences in prevalence and incidence of self-reported diabetes were similar to the differing rates of diabetes risk factors such as obesity, hypertension, and dyslipidemia, which were most common among the hysterectomized women enrolled in the E-alone trial, intermediate in the DMT, and lowest in the OS and E+P Trial (Table 1).

Table 2.

Baseline prevalence and follow-up incidence of self-reported diabetes

Self-reported diabetes	E+P N = 16 608		E-alone N = 10 739		DMT N = 48835		OS N = 93676
Mean follow-up	5.6 years		6.8 years		7.8 years		6.9 years
	N	%	N	%	N	%	N	%
Baseline prevalence (all diabetes)
No	15641	94.2	9712	90.5	45887	94.0	88263	94.3
Yes	959	5.8	1021	9.5	2948	6.0	5318	5.7
Baseline prevalence
(medication-treated diabetes only)
No	15864	95.6	9907	92.3	46628	95.5	89654	95.8
Yes	734	4.4	821	7.7	2202	4.5	3902	4.2
Cumulative incidence
(medication-treated diabetes only)
No	15041	96.2	8860	91.2	42708	93.1	84201	95.4
Yes	600	3.8	852	8.8	3179	6.9	4062	4.6
Annual incidence		0.50		1.16		0.89		0.64
(rate per 100 person-years)

E+P (Estrogen plus Progestin Trial), E-alone (Estrogen-alone Trial), DMT (Diet Modification Trial), OS (Observational Study).

Comparison of diabetes self-reports with medication inventory

In the clinical trials, 79% of the participants at baseline who self-reported treatment with either insulin or oral medication for diabetes had a diabetes medication in the baseline medication inventory. The proportion was similar in each study component and between the intervention and control groups (Figure 1). The corresponding proportion was 77% in the observational study. A very high proportion (>99.5%) of women without self-reported treated diabetes had no diabetes medication in the baseline inventory (data not shown).

Figure 1.

Comparison of self-reports of treated diabetes with medication inventory, by study component and randomized treatment. Bars represent the proportion of participants with a diabetes medication in the inventory of those who reported prevalent treated diabetes at baseline, incident treated diabetes in the interval between baseline and the Year 1 visit in the clinical trials, incident treated diabetes in the interval between the Year 1 and Year 3 visit in the clinical trials, and incident treated diabetes in the interval between baseline and Year 3 visit in the observational study E+P (Estrogen plus Progestin trial), E-alone (Estrogen-alone trial), DMT (Diet Modification trial), OS (Observational Study), BL (baseline), Yr1 (Year 1), Yr3 (Year 3)

In the clinical trials, 78% of the participants who self-reported incident treated diabetes between baseline and Year 1 had a diabetes medication in the Year 1 medication inventory. This concordance fell to 62% between Year 1 and Year 3 in the clinical trials. The concordance was similar in each of the clinical trial components, and there were no statistically significant differences between clinical trial intervention and control groups, with the exception of the E+P trial, where the concordance in the active arm was only 45% and in the placebo arm was 61% (p = 0.02). In the observational study, 72% of the participants who self-reported incident treated diabetes between baseline and Year 3 had a diabetes medication in the Year 3 medication inventory. In all study components, very few women without self-reported incident treated diabetes had a diabetes medication in the medication inventory (>99.9%, data not shown).

Comparison of diabetes self-reports with fasting glucose levels

Of the 525 women who reported a diagnosis of diabetes at baseline and had a baseline fasting glucose measurement, 74% had fasting glucose values ≥126 mg/dL (79% in both the E-alone and E+P trial, 73% in the DMT and 61% in the OS.) Nearly all of the remaining women (18%) had fasting glucose values 100–125mg/dl.

Of the 5884 randomly sampled women with measured fasting glucose levels (6% of clinical trial participants and 1% of OS participants) who did not report a physician diagnosis of diabetes at baseline, 170 (3.4%) had diabetes using a criterion of a single fasting glucose level ≥126 md/dL (Table 3). Mirroring the prevalence of self-reported diabetes, probable undiagnosed diabetes was highest in the E-Alone trial (4.6%) compared to the other study components (∼3%). Using the pre-1997 criterion of fasting glucose >140 mg/dL, which was the diagnostic standard during most of the WHI recruitment, approximately halved the proportion of fasting hyperglycemia that was not self-reported as diabetes for each component of the study. Another 1466 (24.9%) of the women would have been classified as having impaired fasting glucose using contemporary criteria.

Table 3.

Distribution of fasting glucose at baseline in WHI participants without self-reported diabetes at baseline, by study component

Baseline fasting glucose	E+P tria		E-alone trial		DM trial		Observational study
	N	%	N	%	N	%	N	%
<100mg/Dl	951	70.0	683	65.6	1790	71.5	796	81.2
100–125 mg/dL	366	26.9	310	29.8	634	25.3	156	15.9
126–1 39 mg/dL	23	1.7	23	2.2	46	1.8	13	1.3
≥140 mg/dL	19	1.4	25	2.4	34	1.4	15	1.5

Of participants without self-reported diabetes at baseline who had fasting glucose <100 mg/dL, only 6.3% subsequently reported incident treated diabetes (Table 4). As baseline fasting glucose increased, the likelihood of self-reported newly treated diabetes increased. Nearly all (88.1%) women with fasting glucose >126 mg/dl at baseline subsequently reported treated diabetes during an average of 6.9 years of follow-up.

Table 4.

Fasting glucose at baseline in WHI participants without self-reported diabetes at baseline by self-reported treated diabetes during 6.9 years of follow-up

	Self-reported incident treated diabetes
	No		Yes
Baseline fasting glucose	N	%	N	%
< 100 mg/dL	5245	93.7	355	6.3
100–125 mg/dL	1420	66.8	705	33.2
126–1 39 mg/dL	66	23.1	220	76.9
≥140 mg/dL	21	4.7	426	95.3

Discussion

These data show that the diabetes self-reports of the postmenopausal women in the WHI study had good concordance with the medication inventory and fasting glucose values. Baseline self-reports of diabetes medication use were supported by additional evidence of diabetes medication in the inventory for approximately ∼80% and by fasting glucose levels >126 mg/dl in 75% of women. Concordance of self-reported incident treated diabetes during the first year of the study with a diabetes medication in the Year 1 medication inventory was similar to that of prevalent diabetes at baseline (∼80%), although concordance was lower for self-reported incident treated diabetes after Year 1. Exceedingly few WHI participants had diabetes medications in the inventory when they did not report diabetes treatment. At baseline about 3% had undiagnosed diabetes based on a single fasting glucose level > 126 mg/dl.

Our data on diabetes self-reports in WHI compares favorably to that reported in previous population-based studies. In the Iowa Women's Health Study, participants aged 55–69 were asked, ‘Have you ever been told by a physician that you had diabetes mellitus (sugar diabetes)?’ The validity of the responses was evaluated by requesting confirmation from the three most recent private physicians of 44 randomly selected women who reported diabetes mellitus; there was physician confirmation in only 28 cases (64%). [8] In a case-control study of cataract risk factors in Boston, male and female subjects were asked if diabetes had ever been diagnosed by a physician, and to report medications that they took regularly. [9] Of 148 self-reported cases of diabetes, 124 (84%) were confirmed by the primary physician, including 28 of 36 (78%) reports of regular insulin use and 56 of 75 (75%) reports of oral hypoglycemic medication. The authors of this article acknowledged that physician disagreement with self-report might reflect incomplete records of their patients' medical histories. In a population-based study of Norwegian adults from Nord-Trøndelag county conducted in the mid-1980's, 163 of 169 (96%) reports of diagnosed diabetes were verified by review of general practitioner medical records. [10] In this study, 95% of self-reports of insulin use and 100% of self-reports of oral hypoglycemic use were confirmed. The concordance of self-reports and medical records for the diagnosis of diabetes were considerably higher than those obtained in a comparable Norwegian survey (66%) performed in Finnmark county a decade earlier [11]. The discrepancy was attributed to improvement in community and physician diabetes awareness, as well as socioeconomic and linguistic differences between the study populations. In the Longitudinal Aging Study Amsterdam of men and women aged 55 and older, 92% of self-reports of diabetes on a participant interview were confirmed on a general practitioner questionnaire [12].

The Nurses’ Health Study, which enrolled only female registered nurses, represents a highly educated and health-conscious study population. Participants who self-reported a diagnosis of diabetes mellitus were mailed a supplementary questionnaire regarding diagnosis date, symptoms, diagnostic test results and treatment. Based on the responses, cases of gestational diabetes and type 1 diabetes were excluded. Cases of definite type 2 diabetes were defined as 1) at least one classical symptom plus a fasting plasma glucose concentration of > 140 mg/dL or a random plasma glucose concentration of > 200 mg/dL or 2) at least two elevated plasma glucose concentrations of > 200 mg/dL in the absence of symptoms. The medical records of a random sample of participants classified with definite type 2 diabetes by the supplementary questionnaire were reviewed, and 61 of 62 (98%) were confirmed [17]. A similarly high confirmation rate of self-reported diabetes (97%) was found in the men enrolled in the Health Professional Follow-up Study [18].

The proportion of WHI participants with undiagnosed diabetes at baseline was lower than that observed in comparably aged European women in the DECODE study based on glucose tolerance testing (3.2% in women aged 50–59, 6.8% in women aged 60–69 and 11.8% in women aged 70–79) [4]. In the Third National Health and Nutrition Examination Surveys (NHANES III) from 1988 to 1994, among women aged 50 and older, the prevalence of undiagnosed diabetes based on oral glucose tolerance testing was about 5–6% [5]. The lower proportion of undiagnosed diabetes compared to these two studies is likely explained by the greater sensitivity of glucose tolerance testing compared to fasting glucose measurement alone. Nevertheless, WHI follow-up data suggests that nearly all cases of undiagnosed diabetes at baseline were later detected as cases of incident treated diabetes.

We have considered the potential reasons for the lower concordance of self-reported incident diabetes with the Year 3 medication inventory compared to the Year 1 medication inventory in clinical trial participants. Although participants were reminded of the medication inventory, it seems likely that the 2-year interval between inventories may have lowered adherence with the Year 3 inventory. It seems less likely that the validity of diabetes self-report deteriorated as the study progressed, although this possibility cannot be excluded.

We conducted several analyses to explore the lower concordance of incident treated diabetes self-report with the medication inventory in the active treatment arm in the E+P trial. Although, the study was double-blinded, treatment allocation was difficult to conceal due to vaginal bleeding [19]. We considered the possibility that women in the active and placebo arms had different likelihood of making visits to their personal physicians and undergoing diagnostic tests that revealed diabetes. However, this was not the case: during follow-up women in both E+P arms made an average of 7.1 visits to their physicians for general check-up, and women in both E-alone arms made an average of 7.8 visits. We also considered the possibility that differential stopping of study drug, which was higher among women in the active arm of the E+P trial [19], could have led to lower adherence with the drug inventory. However, the results were similar when we excluded women who had become nonadherent with study pills. Thus, the reasons for the finding are unclear, and it could be due to chance.

This study has a number of limitations that could have resulted in underestimation of the true valditiy of self-reported diabetes. Since the baseline questionnaire regarding diabetes treatment did not include a specific question about current oral diabetes drug use, some women who had previously been treated with oral medication might no longer have been receiving any medication at the time of the baseline medication inventory. The participants may not have brought all medications for the inventory, or medications that were prescribed at the time of the semi-annual follow-up questionnaires may have been discontinued at the time of the Year 1 or Year 3 medication inventory. Fasting glucose values are problematic for detection of treated diabetes since the treatment will lower the glucose level. Since the majority of the diabetic women reporting a diabetes diagnosis were treated with medication, some undoubtedly had good glycemic control with fasting glucose <126 mg/dl at baseline.

We conclude that in the WHI self-reported prevalent and incident diabetes was consistent with medication inventories, and a high proportion of those with undiagnosed diabetes subsequently reported diabetes treatment. A self-report of ‘treated diabetes’ is sufficiently accurate to allow use in epidemiologic studies.