Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Aug 1.
Published in final edited form as: Menopause. 2014 Aug;21(8):861–868. doi: 10.1097/GME.0000000000000189

Validity of Diabetes Self-Reports in the Women's Health Initiative

Jody M Jackson 1, Terese A DeFor 2, A Lauren Crain 3, Tessa J Kerby 4, Lori S Strayer 5, Cora E Lewis 6, Evelyn P Whitlock 7, Selvi B Williams 8, Mara Z Vitolins 9, Rebecca J Rodabough 10, Joseph C Larson 11, Elizabeth B Habermann 12, Karen L Margolis 13
PMCID: PMC4278641  NIHMSID: NIHMS542344  PMID: 24496083

Abstract

Objective

To determine positive and negative predictive values of self-reported diabetes during the Women's Health Initiative (WHI) clinical trials.

Methods

All WHI trial participants from four field centers who self-reported diabetes at baseline or during follow up, as well as a random sample who did not self-report diabetes, were identified. Women were surveyed regarding diagnosis and treatment. Medical records were obtained and reviewed for documented treatment with anti-diabetic medications or physician diagnosis of diabetes supported by laboratory measurements of glucose.

Results

We identified 1,275 eligible participants; 732 consented and provided survey data. Medical records were obtained for 715 (207 prevalent diabetes, 325 incident diabetes, and 183 no diabetes). Records confirmed 91.8% (95% CI 87.0%-95.0%) of self-reported prevalent diabetes and 82.2% (95% CI 77.5%-86.1%) incident diabetes. Of those who never self-reported diabetes, there was no medical record or laboratory evidence for diabetes in 94.5% (95% CI 89.9%-97.2%). Women with higher BMI were more likely to accurately self-report incident diabetes. In a subgroup of participants enrolled in fee-for-service Medicare, a claims algorithm correctly classified nearly all diabetes cases and non-cases.

Conclusions

Among WHI clinical trial participants, there was a high positive predictive value of self-reported prevalent (91.8%) and incident diabetes (82.2%) and a high negative predictive value (94.5%) when diabetes was not reported. For participants enrolled in fee-for-service Medicare, a claims algorithm also had a high positive and negative predictive value.

Keywords: Diabetes mellitus, type 2, medical record, self report, validation studies, questionnaires, reproducibility of results

INTRODUCTION

The United States is experiencing an epidemic of type 2 diabetes, a major medical problem associated with significant morbidity, mortality, and cost.[1-3] An estimated 24 million adults, or about 8% of the population, are considered to have diabetes (both diagnosed and undiagnosed).[4] The prevalence of diabetes rises sharply with each decade of age in postmenopausal women and surpasses that in men after the seventh decade of life[5, 6], suggesting that the risk of diabetes not only increases with age but increases more sharply with age for women than for men. In fact, among women participating in two nationally representative surveys, the National Health and Nutrition Examination Survey 1999-2004 and the National Health Interview Survey 2005, about 40% to 45% of participants with diagnosed diabetes were 45 years old and older.[7] Consequently, it is critical to understand the risk factors for and the health consequences of diabetes as women age.

The Women's Health Initiative (WHI) is a data source that has tremendous untapped potential for exploring the health implications of diabetes in older women. From 1993 to 1998, the WHI enrolled 68,133 postmenopausal women aged 50 to 79 years at baseline in its randomized controlled clinical trials and 93,676 women in an observational study.[8, 9] The trials included the Dietary Modification Trial, in which 48,836 eligible women were assigned to either a diet intervention to consume a low-fat diet high in fruits, vegetables, and grains or continue to consume their usual diet. In the Hormone Trials, placebo was compared with either conjugated equine estrogen alone or conjugated equine estrogen plus medroxyprogesterone acetate in 27,347 women, depending on hysterectomy status. The cohort has been followed annually through 2010, with an additional extension study providing 5 years of supplementary observational follow up through 2015.

The primary outcomes of the trials were breast cancer, colorectal cancer, coronary heart disease, and hip fracture. Self-reports of these outcomes were confirmed by reviews of medical records by trained physician adjudicators. Incident self-reported diabetes was also assessed as one of the secondary outcomes of the clinical trials but was not independently confirmed by review of medical records. While fasting blood specimens were collected and stored for all WHI participants, the WHI design called for analysis of only a 6% random sample of the clinical trial participants’ samples for fasting glucose and other laboratory measures. Therefore, very few of the trial participants who self-reported diabetes had measurements for fasting blood glucose recorded in the WHI database.

In previous population-based studies, participant self-reports of a diagnosis of diabetes or use of medications for diabetes were confirmed by review of medical records or queries to physicians 64% to 98.5% of the time.[10-24] The wide range of positive predictive value of diabetes self-reports is coupled with a paucity of data on the negative predictive value of not reporting diabetes during longitudinal follow-up studies.[18-20, 24] Confirming that participant self-report is a reliable method of classifying diabetes status during the WHI follow up would greatly enhance the value of the WHI data as a resource for further investigation of the effects of dietary, hormonal, and other influences on diabetes in older women, as well as analyses with diabetes as the exposure of interest.

The purpose of this study was to examine the agreement of self-reports of diabetes in the WHI clinical trials and with medical record review, estimating both the positive and negative predictive value of self-report. Secondary aims were to compare the confirmation rates of self-reported diabetes between the clinical trials and their intervention and control arms and to assess whether participant characteristics (eg, age, race/ethnicity, education, BMI) influenced the accuracy of self-reported diabetes.

METHODS

Participants

Participants in this study were recruited from among the clinical trial enrollees from four WHI field centers (Minneapolis, MN; Winston-Salem, NC; Birmingham, AL; and Portland, OR). Prevalent self-reported cases were defined as participants who reported a diabetes diagnosis by a physician (medication-treated and untreated) at baseline. Incident cases were defined as all nondiabetic participants at baseline who, during follow up (including the main WHI trial or its extension), reported being newly prescribed insulin or oral hypoglycemic medication. A random sample of participants without self-reported diabetes (No Diabetes Group) was also identified from the roughly 6% of the overall WHI clinical trials cohort who had serial blood samples analyzed for fasting glucose at baseline and years 1, 3, or 6. Eligible participants were alive and had not been lost to follow up or withdrawn consent to participate in WHI as of the last quarter of 2007.

All eligible women with prevalent or incident self-reported diabetes and the random sample without self-reported diabetes were mailed a request to consent to the study, sign a release of information to obtain medical records, and complete a questionnaire seeking verification of their self-report, any symptoms at diagnosis, treatment history, and contact information for health care providers. A second questionnaire was sent to women who did not respond to the initial questionnaire. All remaining nonresponders were contacted by telephone. Of the 1,275 women invited to participate, 715 were enrolled. Medical records were obtained, and the agreement between their WHI self-report of diabetes and medical record review is reported in this paper.

Confirmation of Diabetes Status by Medical Record Review

Outpatient and inpatient medical records related to diabetes diagnosis, treatment, and laboratory testing were requested for consenting women and reviewed by a trained physician adjudicator (KM, CL, SW, DB) at each site. Medical records of all self-reported incident and prevalent diabetes cases were requested and reviewed. The records of any participant from the No Diabetes Group who had only one fasting glucose value available, any fasting glucose value of >100 mg/dL, or who reported diabetes on the confirmation questionnaire were also requested and reviewed. If all of two or more fasting glucose values were <100 mg/dL and the participant reported no diabetes on the confirmation questionnaire, she was assumed without further review not to have diabetes.

The medical record review form included notations of diabetes on the problem list or as a diagnosis, the use of medications for diabetes (oral medications and/or insulin), glycosylated hemoglobin levels, fasting and casual plasma glucose values, and results of any oral glucose tolerance tests. Based on the medical records reviewed, the adjudicators recorded their summary impression about whether the participant had diabetes and listed the supporting diagnostic criteria on which their decision was based. Diabetes was defined at the study's inception in accordance with the American Diabetes Association definition[25] as one or more of the following found in the medical record: 1) Fasting plasma glucose >126 mg/dL, 2) Symptoms of diabetes (polyuria, polydipsia, and unexplained weight loss) plus casual plasma glucose concentration >200 mg/dL, 3) 2-hour postload glucose >200 mg/dL during an oral glucose tolerance test using a glucose load containing the equivalent of 75 g anhydrous glucose dissolved in water, and 4) Treatment with insulin or an oral hypoglycemic agent.

All physician adjudicators participated in a training session in which 12 sample charts were reviewed and discussed. Furthermore, shortly after medical record abstraction was under way, quality assurance checks were conducted; a second adjudicator (KM) independently reviewed 12 randomly selected records (5 incident, 5 prevalent, and 2 nondiabetes cases) from each site and verified the summary impression of diabetes vs. no diabetes, as supported by the diagnostic elements listed above. Agreement between adjudicators regarding diabetes status was 100%. In addition to the quality assurance sample, the medical records of any discordant cases (WHI self-report of diabetes did not match summary impression from medical record review) were reviewed by a second physician adjudicator (KM), who made a final determination of diabetes status. The WHI cohort was also linked to Medicare data from 1991 to 2007. Of women aged 65 and older with a valid Social Security number, 196 were successfully matched to the Medicare enrollment file.

Statistical Analysis

Agreement Between WHI Self-reports of Diabetes and Medical Record Review

Self-reports of diabetes confirmed by medical record review were considered true positives, while self-reports of diabetes without medical record confirmation were considered false positives. Self-reports of no diabetes confirmed by either two fasting glucose levels <100 mg/dL from WHI laboratory records or information from medical record review were categorized as true negatives, while self-reports of no diabetes but with evidence of diabetes in the medical record were categorized as false negatives. The positive predictive value of self-reported incident diabetes was computed as (true positives)/(true positives + false positives) and negative predictive value as (true negatives)/(true negatives + false negatives). Ninety-five percent confidence intervals for the underlying positive and negative predictive values were based on the normal distribution approximations to the corresponding exact binomial probability distributions.[26]

Agreement Between Medical Record Review and Medicare Claims Data

We also examined the agreement of diabetes status as assessed by medical record review with that of Medicare claims data that had been linked to the WHI cohort. This analysis was performed in a subset of women who were continuously enrolled in Part A and Part B fee-for-service Medicare for at least 2 years before their last date of WHI contact. We used a validated algorithm for identifying diabetes from Medicare claims data that required at least two physician claims for ambulatory care on different days or one inpatient claim (ICD-99 code 250.xx).[27]

Agreement Between WHI Self-reports of Diabetes and Medical Record Review by Trial and Treatment Assignment

We examined whether the random treatment assignment (intervention, control) or the trial in which women were enrolled (estrogen, estrogen + medroxyprogesterone [E+P] diet) was a significant predictor of agreement between self-reported diabetes status and medical record review. A saturated logistic regression model was estimated in which agreement was predicted from self-reported diabetes groups (no diabetes – reference, incident diabetes, prevalent diabetes), trial (diet – reference, estrogen, E+P), random treatment assignment (control – reference, intervention), and all two-way interactions among these variables. To reduce the possibility of overfitting the model to the data, nonsignificant interaction terms were removed individually from the saturated model until all that remained in the reduced logistic model were the three main effects (ie, self-reported diabetes group, trial, randomly assigned treatment group) and significant interaction terms. Simple effects tests were calculated to help interpret significant interactions.

Agreement Between WHI Self-reports of Diabetes and Medical Record Review by Personal Characteristics

Finally, we sought to identify the characteristics of study participants other than self-reported diabetes, trial, and treatment assignment, which were associated with the probability of accurately self-reporting diabetes status. This logistic regression model included all of the parameters retained in the previous analysis as well as characteristics that, when added individually to the model, significantly predicted agreement. The characteristics considered for inclusion were age and BMI (both centered on the mean value), Hispanic ethnicity or non-White race (non-Hispanic white reference), educational attainment (less than high school, high school, post high school, college degree or higher reference), annual household income (less than $10,000, $10,000-$19,999, $20,000-$34,999, $35,000-$49,999, $50,000-$74,999, $75,000+ reference) and whether the woman participated in the observational post-trial follow-up component of WHI (non-participant reference). Only characteristics that significantly predicted agreement were retained to reduce the likelihood that overfitting the data would produce unstable parameter estimates. Two-way interactions between remaining demographic characteristics and self-reported diabetes status, trial, and treatment assignment were also assessed and significant interaction terms retained.

Assessing Enrollment Bias

Preliminary analyses were conducted to identify any systematic differences between the 715 women in the analytic group and the overall sample of 1,275 and whether these differences would bias the primary analyses. A propensity model predicted the likelihood of enrollment from features of the women's previous WHI participation (ie, WHI trial, extension study enrollment), personal characteristics (ie, self-reported diabetes, age, race/ethnicity, education, income, BMI) and any significant (P<0.10) twoway interactions between covariates.

RESULTS

Participants

Of the overall sample of 1,275 women from the four WHI field centers invited to participate, 732 (57%) women consented and were enrolled in the study. Among the 543 who were not enrolled, reasons for non-participation included: unable to be contacted (41%), refused participation (58%), and inability to participate (eg, dementia) (<1%). Medical records were obtained for 715 (98%) of enrolled participants and constituted the analytic sample. Of these, 183 (26%) self-reported no diabetes in WHI, 207 (29%) reported prevalent diabetes, and 325 (45%) incident diabetes. Table 1 provides the demographic characteristics of the overall sample and analytic sample. A total of 196 (111 with confirmed prevalent or incident diabetes and 85 without confirmed diabetes) of the 715 analytic participants met criteria to be included in the Medicare claims data analysis.

Women who had participated in the extension study and who were in the E+P trial and self-reported prevalent diabetes were more likely to enroll. Women who were non-white, lacked a high school education, were in both the diet and the hormone trial, and had self-reported incident diabetes were less likely to enroll. The propensity model significantly predicted enrollment, χ2(35) = 217.31, P<0.001, had acceptable discrimination, c=0.72, and was a good fit to the data, Hosmer-Lemeshow χ2(8) = 3.46, P=0.90. Next, a logistic regression model was estimated to assess whether likelihood of enrollment predicted agreement of self-reported diabetes and medical record review among enrollees. A significant prediction would provide evidence that the observed systematic differences between study enrollees and non-enrollees were also related to agreement and therefore biased the study results. However, there was not a significant relationship between likelihood of enrollment and agreement (model χ2(1) = 0.008, P=0.93, c = 0.48, Hosmer-Lemeshow χ2(8) = 11.50, P=0.18), suggesting that there was no evidence of nonresponse bias, although unmeasured factors related to both enrollment and agreement may nonetheless introduce nonresponse bias.[28]

Agreement Between WHI Self-reports of Diabetes and Medical Record Review

The positive predictive value of self-reporting prevalent diabetes was 91.8% and was 82.2% for incident diabetes, while the negative predictive value of self-reporting no diabetes was 94.5% (Table 2). Agreement between self-reported diabetes and medical record review was significantly lower among the incident diabetes group than the prevalent diabetes and no diabetes groups, which were not different from each other.

Among the 10 false negatives, medical record review provided ample evidence of diabetes in most (70%) cases. Two cases had limited evidence in the medical record (ie, some inconsistency in progress notes or problem list about impaired fasting glucose or pre-diabetes versus diabetes) but enough to meet study criteria for having diabetes. In one case, the participant had medical records indicating lifestyle-managed diabetes, but she accurately answered the WHI survey question that asked only about medication-treated diabetes. About one-third of the false-positive cases were due to questionnaire scanning errors (eg, scanner picked up an erased response), and roughly one-third were for unknown reasons. For one-quarter of the false positives, medical records indicated equivocal evidence of diabetes or untreated prediabetic states. Less than 10% of the false-positive cases appeared to be due to a transient or acute episode of diabetes precipitated by an event such as myocardial infarction or steroid use.

Agreement Between Medical Record Review and Medicare Claims Data

Based on the adjudicated determination of whether diabetes was (n=111) or was not present (n=85), the Medicare claims algorithm performed very well in correctly classifying diabetes: 98% (108/110) of women meeting the claims algorithm criteria had confirmed diabetes by medical record review, and 97% (83/86) of women who did not meet the claims algorithm criteria did not have diabetes by medical record review.

Differences in Agreement of WHI Self-reports of Diabetes by WHI Trial and Treatment Assignment

There were some statistically significant differences in agreement between self-reported diabetes and medical record review overall across the WHI trials (P<0.001). Figure 1 represents the predicted probability of accurately reporting diabetes status by diabetes group and trial. The model that tested these differences had adequate discrimination, c=0.67 and was a good fit to the data, Hosmer-Lemeshow P=0.82. Agreement between self-report and medical record review was not different for women in the E+P trial relative to those in the diet trial (P=0.10), and the difference between E+P and diet was similar across self-reported diabetes groups (interaction P=0.33). A different pattern emerged when comparing women in the estrogen and diet trials (interaction P=0.04). Although women in the estrogen trial with prevalent and incident diabetes had somewhat higher agreement than those in the diet trial, these differences were not statistically significant (estrogen prevalent 96.0%, diet prevalent 90.5%, P=0.25; estrogen incident 87.8%, diet incident 81.0%, P=0.19). However, those who reported no diabetes and in the estrogen trial were less likely to agree than those in the diet trial (estrogen no diabetes 88.5%, diet no diabetes 98.7%, P=0.04). There were no significant differences between the treatment arms for any of the trials (P=0.97).

Figure 1.

Figure 1

Open in a new tab

Predicted probability of accurate self-reporting of diabetes status by Women's Health Initiative trial and treatment assignment

Differences in Agreement by Personal Characteristics

The final analysis considered personal characteristics in addition to self-reported diabetes group, trial, and treatment group. This model significantly predicted agreement between self-report and medical record review (χ2(12) = 55.41, P<0.001) and demonstrated good discrimination (c=0.75) but was not a good fit to the data (Hosmer-Lemeshow χ2(8) = 15.21, P=0.06), possibly due to overfitting. The pattern of effects in Figure 1 did not change when personal characteristics were added.

BMI independently predicted agreement between self-report and medical record evidence of diabetes, and it interacted with self-reported diabetes group (P<0.001). The BMI by diabetes group interaction (Figure 2) showed that, while a higher BMI tended to be related to a higher probability of agreement, this relationship was strongest among those with incident diabetes (1.7% increase in agreement for every unit increase in BMI), weaker among those with prevalent diabetes, and nonexistent or even slightly negative among those with no evidence of diabetes.

Figure 2.

Figure 2

Open in a new tab

Predicted probability of accurate self-reporting of diabetes status by group and body mass index

DISCUSSION

In this examination of the agreement between women's self-reported diabetes status during WHI and medical record evidence of diabetes, we found a high positive predictive value of both prevalent and incident self-report of diabetes and a high negative predictive value for non-report of diabetes. When a WHI participant reported that she had diabetes on enrollment in WHI, her self-report was confirmed in 92% of cases by medical record review. When a woman reported that she was later diagnosed with and started on treatment for diabetes, this was confirmed in 82% of cases. Most (two-thirds) of the false-positive cases were related to technical errors in questionnaire completion, with diagnostic uncertainty and transient hyperglycemia accounting for the remainder. When a woman in WHI reported that she did not have diabetes, either on enrollment or over the course of the trials, her self-report was confirmed in 94.5% of cases. This study is one of the few longitudinal examinations of diabetes self-report, having sought to determine agreement for both prevalent and incident diabetes. There was little variation among the trials and none among the treatment arms. The only personal characteristic that influenced the accuracy of self-report was BMI; women with a higher BMI were more likely to correctly report having new-onset diabetes. Finally, for the subset of women who were covered by fee-for-service Medicare, a previously validated claims algorithm had a very high agreement with diabetes status confirmed by medical record review.

Numerous studies have addressed the reliability of self-report of diabetes. Examinations of the agreement between self-reports with either medical record or physician validation have found concordance values of 64% to 98.5%.[10-22, 29] Assessments of agreement of self-report of diabetes compared with biomedical information (eg, glycosylated hemoglobin, fasting serum glucose) indicate agreements of 95% to 96.3%.[23, 24] Authors of these studies consistently conclude that self-report of diabetes is quite accurate, particularly compared with that of other chronic conditions.

Several of these examinations have yielded high concordance values similar to those reported in this paper. Skinner et al. assessed the agreement between patient report and medical records on five chronic conditions, including diabetes.[21] The patient report was obtained from a self-administered screening questionnaire conducted as part of the Veteran's Health Study. Participants were asked, “Has a doctor ever told you that you have diabetes or high blood sugar?” as well as about current treatment with medication or diet. The authors reported a positive predictive value for diabetes of 90.8% among the study sample of 402 men. Okura and colleagues also report the agreement between medical record evidence and self-reported diabetes and other chronic disorders.[19] In this examination, randomly selected Mayo Clinic patients 45 and older residing in Southern Minnesota were asked, “Has a medical provider ever told you that you had any of following conditions?” Responses from the 1,950 participants were validated by searching the comprehensive medical record maintained by Mayo Clinic for a diagnosis of diabetes. Authors found a positive predictive value for diabetes of 94.3%.

Tisnado et al. examined the concordance between the medical record and self-reports of chronic conditions, including diabetes, in 1,270 patients from 39 West Coast area health care organizations.[22] Self-report was obtained from a participant survey designed to measure disease based on four components of the care process: diagnoses, clinical services received, counseling and referrals, and medication use. The medical record was then reviewed and data collected regarding diabetes in these four dimensions. The authors found an overall agreement of 92% between participant self-report and the medical record. A substantial body of literature also describes the concordance of diabetes self-report with claims data. [27, 30-34] Using a well-established methodology of two or more outpatient or one hospital claim [27] we found excellent agreement between adjudicated self-reported diabetes status and Medicare claims data.

The high concordance of values found in these studies may be explained by the thoroughness or precision of the protocols used to obtain medical record evidence of diabetes. Authors of these examinations either reviewed multiple sources in the medical record, including discharge summaries, and problem and medication lists, or they sought an explicit physician's diagnosis to confirm self-report of diabetes. In fact, Tisnado and colleagues found greater agreement in their examination when medication use was used as source of information in the medical record compared with that of clinical services, counseling, referrals, and even diagnoses.[22]

For other examinations, it may be the rigor with which confirmation of diabetes was sought, as well as the sample sizes, that helps explain the variations in concordance. In the Iowa Women's Health Study, participants aged 55 to 69 were asked, “Have you ever been told by a physician that you had diabetes mellitus (sugar diabetes)?” Self-report was examined by requesting physician confirmation of 44 randomly selected participants who reported diabetes. Agreement was found among 28 cases (64%).[10] Male and female participants of a case-control study of cataract risk factors in Boston were asked if diabetes had ever been diagnosed by a physician and to report medications that they took regularly.[11] Of self-reported cases, 84% (124 of 148) were confirmed; whereas 78% of insulin use reports (28 of 36) and 75% (56 of 75) reports of oral hypoglycemic medication use were confirmed. These studies could also be viewed to have imprecise estimates, because they were quite small. In contrast, another smaller study (n=62) from the Nurse's Health Study (NHS) cohort found a higher agreement (98%) between self-report of both incident and prevalent diabetes and medical records review. The NHS represents only female registered nurses, a highly educated and health-conscious study population, and thus may not be representative of the general population.[29]

However, several large, international, population-based validation studies show some range of agreement as well. In a study of Norwegian adults from Nord-Trøndelag County, 163 of 169 (96%) reports of diabetes were verified by review of general practitioner medical records.[12] Furthermore, 95% of self-reports of insulin use and 100% of self-reports of oral hypoglycemic use were confirmed. Another Norwegian survey performed in Finnmark County in the 1970s found a 66% concordance between self-reports and medical records for the diagnosis of diabetes.[13] Huerta et al. examined the accuracy of self-reported diabetes, hypertension, and hyperlipidemia using data from the DINO study, a population-based study of prevalence of these conditions among the overall adult (>20 years) residents of Murcia in Southern Spain [24]. Self-report, elicited by asking, “Have you ever been told that you are diabetic or have high blood sugar?” and about family history of diabetes, was compared with a fasting serum glucose sample obtained from each participant. The authors found a positive predictive value of 95% for diabetes. In the Longitudinal Aging Study conducted in Amsterdam of men and women 55 and older, 92% of self-reports of diabetes on a participant interview were confirmed on a general practitioner questionnaire.[14]

Finally, the differences in PPV from these previous studies may also be due to the characteristics of the study participants, which may contribute to awareness of diabetes, as proposed above in the case of the NHS. The characteristics found to be associated with lower accuracy of diabetes self-report include family history of diabetes[24], lower education, advanced age[20], and decreased cognitive function [23]. Several examinations have found that agreement between self-report of diabetes compared with medical records or biomedical data decreased as the number of comorbid conditions increased[18, 20, 21]. Our findings indicated that, of the participant characteristics examined (eg, age, race, education, income, BMI), the only additional factor found to influence the accuracy of a woman's self-report was BMI. Self-reported incident diabetes was more likely to be confirmed by medical record review in women with higher BMI. This finding is in contrast with the results of previous examinations of the factors associated with undiagnosed diabetes, which indicate that individuals who did not know they had diabetes had higher BMI [35-37]. This difference with the current study may be related to the greater health consciousness of women enrolled in WHI; they may have been more aware of the risk of diabetes conferred by overweight than study participants in other settings.

CONCLUSIONS

Strengths and Limitations

This large validation study of self-reported diabetes compared with medical record review was a very rigorous examination that used American Diabetes Association criteria as the basis for medical record evidence of diabetes. While this analysis did not include all women enrolled in the WHI, our study sample involved all WHI participants from four geographically diverse field centers. These centers are a good representation of the overall WHI enrollees with regard to racial, ethnic, and socioeconomic diversity. Furthermore, there was no evidence that the relatively low rate of enrollment (57%) produced bias in the analyses presented here. We were also able to obtain medical records on most enrolled participants and therefore included them in our analytic sample. One of the key strengths of this examination is that few previous longitudinal studies comparing the accuracy of self-report of diabetes with medical record review have reported the false-negative rate (ie, percentage of participants who do not self-report diabetes but who would be found to have diabetes if external sources of confirmation were sought). Our examination, which found a false-negative rate of 5.5%, helps fill this gap.

The findings of this study, however, should be viewed in light of some limitations. As stated previously, we did not seek to validate self-report of all WHI participants, only enrollees in the clinical trial of WHI at four, albeit geographically and ethnically diverse, sites. Women who enrolled in WHI generally tended to be well-educated and motivated by health awareness compared with the general public, which may limit the generalizability of our findings to the broader population. Despite this limitation, the WHI participants in our examination were more diverse than some of the unique participant populations involved in previous studies (eg, NHS). Furthermore, very little variation was noted by race/ethnicity or education. Because of the small number of non-white participants and those with less than high school education, we may have had limited power to examine these variables as predictors of accuracy of self-report.

Implications for Practice and/or Policy

The WHI clinical trials offer a wealth of data on patient outcomes, including cancer, cardiovascular disease, and hip fracture, which were confirmed via medical record review. The body of literature shows wide variations on the validity of self-reported diabetes in diverse populations but, in general, diabetes is one of the self-reported outcomes with the highest validity. The results of this study indicate that both WHI information on self-reported diabetes status and Medicare claims are reliable proxies for medical record review and should result in little misclassification.

Supplementary Material

Table 1
Table 2

Acknowledgements

The WHI program is funded by the National Heart, Lung, and Blood Institute, National Institutes of Health, U.S. Department of Health and Human Services, through contracts N01WH22110, 24152, 32100-2, 32105-6, 32108-9, 32111-13, 32115, 32118-32119, 32122, 42107-26, 42129-32, and 44221. This analysis was funded by the National Institute of Diabetes and Digestive and Kidney Diseases, R21 DK074646 (Margolis, PI)

APPENDIX

SHORT LIST OF WHI INVESTIGATORS

Program Office

(National Heart, Lung, and Blood Institute, Bethesda, Maryland) Jacques Rossouw, Shari Ludlam, Joan McGowan, Leslie Ford, and Nancy Geller

Clinical Coordinating Center: Clinical Coordinating Center

(Fred Hutchinson Cancer Research Center, Seattle, WA) Garnet Anderson, Ross Prentice, Andrea LaCroix, and Charles Kooperberg

Investigators and Academic Centers

(Brigham and Women's Hospital, Harvard Medical School, Boston, MA) JoAnn E. Manson; (MedStar Health Research Institute/Howard University, Washington, DC) Barbara V. Howard; (Stanford Prevention Research Center, Stanford, CA) Marcia L. Stefanick; (The Ohio State University, Columbus, OH) Rebecca Jackson; (University of Arizona, Tucson/Phoenix, AZ) Cynthia A. Thomson; (University at Buffalo, Buffalo, NY) Jean Wactawski-Wende; (University of Florida, Gainesville/Jacksonville, FL) Marian Limacher; (University of Iowa, Iowa City/Davenport, IA) Robert Wallace; (University of Pittsburgh, Pittsburgh, PA) Lewis Kuller; (Wake Forest University School of Medicine, Winston-Salem, NC) Sally Shumaker

Women's Health Initiative Memory Study

(Wake Forest University School of Medicine, Winston-Salem, NC) Sally Shumaker

Footnotes

Conflict of Interest and Source of Funding: Cora Lewis is PI of a study at the University of Alabama at Birmingham (UAB). UAB receives research funding from Novo Nordisk, but she does not receive any money directly. All other authors have no conflicts of interest to declare. National Institute of Diabetes and Digestive and Kidney Diseases, R21 DK074646 (Margolis, PI)

Contributor Information

Jody M. Jackson, Health Partners Institute for Education and Research PO Box 1524 MS 21111R Minneapolis, MN 55440-1524.

Terese A. DeFor, Health Partners Institute for Education and Research PO Box 1524 MS 21111R Minneapolis, MN 55440-1524 Phone: 952-967-7304 Fax: 952-967-5022 terese.a.defor@healthpartners.com.

A. Lauren Crain, Health Partners Institute for Education and Research PO Box 1524 MS 21111R Minneapolis, MN 55440-1524 Phone: 952-967-5354 Fax: 952-967-5022 lauren.a.crain@healthpartners.com.

Tessa J. Kerby, Health Partners Health Improvement & Care Innovation PO Box 1524 MS21101D Minneapolis, MN 55440-1524 Phone: 952-883-5016 Fax: 952-883-5899 Tessa.j.kerby@healthpartners.com.

Lori S. Strayer, University of Minnesota School of Public Health Epidemiology Room 300 WBOB 7525A 1300 S 2nd St Minneapolis, MN 55454 Phone: 612-626-8885 Fax: 612-624-0315 strayer@umn.edu.

Cora E. Lewis, University of Alabama at Birmingham Department of Medicine MT 614 Phone: 205-934-6383 Fax: bethlew@uab.edu.

Evelyn P. Whitlock, Kaiser Permanente Center for Health Research-Northwest 3800 N. Interstate Avenue Portland, OR 97227-1098 Phone: 503.335.2400 Fax: Evelyn.whitlock@kpchr.org.

Selvi B. Williams, Kaiser Permanente Center for Health Research-Northwest 3800 N. Interstate Avenue Portland, OR 97227-1098 Phone: 503.335.2400 Fax: sw1014@comcast.net.

Mara Z. Vitolins, Wake Forest School of Medicine Medical Center Boulevard Winston-Salem, NC 27157 Phone: 336-716-2886 mvitolin@wakehealth.edu.

Rebecca J. Rodabough, Fred Hutchinson Cancer Research Center M3-A410 Phone:206-667-6997 Fax:206-667-4142 rrodabou@whi.org.

Joseph C. Larson, Fred Hutchinson Cancer Research Center M3-A410 Phone:206-667-6571 Fax:206-667-4142 jlarson@whi.org.

Elizabeth B. Habermann, Department of Surgery Masonic Cancer Center, University of Minnesota Mayo Mail Code 806 420 Delaware Street SE Minneapolis, MN 55455 Phone: 612-626-2563 Fax: ebh@umn.edu.

Karen L. Margolis, HealthPartners Institute for Education and Research PO Box 1524 MS 21111R Minneapolis, MN 55440-1524 Phone: 952-967-7301 Fax: 952-967-5022 karen.l.margolis@healthpartners.com.

References

  • 1.American Diabetes Association Economic costs of diabetes in the United States in 2007. Diabetes Care. 2008;31:596–615. doi: 10.2337/dc08-9017. [DOI] [PubMed] [Google Scholar]
  • 2.Cowie CC, Rust KF, Ford ES, Eberhardt MS, Byrd-Holt DD, Li CY, Williams DE, Gregg EW, Bainbridge KE, Sayday SH, Geiss LS. Full accounting of diabetes and prediabetes in the U.S. population in 1988-1994 and 2005-2006. Diabetes Care. 2009;32:287–94. doi: 10.2337/dc08-1296. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Cowie CC, Rust KF, Byrd-Holt DD, Gregg EW, Ford ES, Geiss LS, Bainbridge K, Frandkin J. Prevalence of diabetes and high risk for diabetes using A1C criteria in the U.S. population in 1988-2006. Diabetes Care. 2010;33:562–8. doi: 10.2337/dc09-1524. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Centers for Disease Control and Prevention . National diabetes fact sheet:national estimates and general information on diabetes and prediabetes in the United States, 2011. U.S. Department of Health and Human Services Centers for Disease Control and Prevention; Atlanta, GA: 2011. [Google Scholar]
  • 5.Harris MI, Flegal KM, Cowie CC, Eberhardt MS, Goldstein DE, Little RR, Wiedmeyer HM, Byrd-Holt DD. Prevalence of diabetes, impaired fasting glucose, and impaired glucose tolerance in US adults. The Third National Health and Nutrition Examination Survey, 1988-1994. Diabetes Care. 1998 Apr;21(4):518–24. doi: 10.2337/diacare.21.4.518. [DOI] [PubMed] [Google Scholar]
  • 6.DECODE Study Group Age- and sex-specific prevalences of diabetes and impaired glucose regulation in 13 European cohorts. Diabetes Care. 2003;26:61–9. doi: 10.2337/diacare.26.1.61. [DOI] [PubMed] [Google Scholar]
  • 7.Owens MD, Beckles GL, Ho KK, Gorrell P, Brady J, Kaftarian JS. Women with diagnosed diabetes across the life stages: underuse of recommended preventive care services. J Womens Health (Larchmt) 2008 Nov;17(9):1415–23. doi: 10.1089/jwh.2008.1125. [DOI] [PubMed] [Google Scholar]
  • 8.Anderson GL, Manson J, Wallace R, Lund B, Hall D, Davis S, Shumaker S, Wang CY, Stein E, Prentice RL. Implementation of the Women's Health Initiative study design. Ann Epidemiol. 2003 Oct;13(9 Suppl):S5–17. doi: 10.1016/s1047-2797(03)00043-7. [DOI] [PubMed] [Google Scholar]
  • 9.Hays J, Hunt JR, Hubbell FA, Anderson GL, Limacher M, Allen C, Rossouw JE. The Women's Health Initiative recruitment methods and results. Ann Epidemiol. 2003 Oct;13(9 Suppl):S18–77. doi: 10.1016/s1047-2797(03)00042-5. [DOI] [PubMed] [Google Scholar]
  • 10.Kaye SA, Folsom AR, Sprafka JM, Prineas RJ, Wallace RB. Increased incidence of diabetes mellitus in relation to abdominal adiposity in older women. J Clin Epidemiol. 1991;44(3):329–34. doi: 10.1016/0895-4356(91)90044-a. [DOI] [PubMed] [Google Scholar]
  • 11.Kehoe R, Wu SY, Leske MC, Chylack LT., Jr Comparing self-reported and physician-reported medical history. Am J Epidemiol. 1994 Apr 15;139(8):813–8. doi: 10.1093/oxfordjournals.aje.a117078. [DOI] [PubMed] [Google Scholar]
  • 12.Midthjell K, Holmen J, Bjorndal A, Lund-Larsen G. Is questionnaire information valid in the study of a chronic disease such as diabetes? The Nord-Trondelag diabetes study. J Epidemiol Community Health. 1992 Oct;46(5):537–42. doi: 10.1136/jech.46.5.537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Tretli S, Lund-Larsen PG, Foss OP. Reliability of questionnaire information on cardiovascular disease and diabetes: cardiovascular disease study in Finnmark county. J Epidemiol Community Health. 1982 Dec;36(4):269–73. doi: 10.1136/jech.36.4.269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Kriegsman DM, Penninx BW, van Eijk JT, Boeke AJ, Deeg DJ. Self-reports and general practitioner information on the presence of chronic diseases in community dwelling elderly. A study on the accuracy of patients' self-reports and on determinants of inaccuracy. J Clin Epidemiol. 1996 Dec;49(12):1407–17. doi: 10.1016/s0895-4356(96)00274-0. [DOI] [PubMed] [Google Scholar]
  • 15.Barber J, Muller S, Whitehurst T, Hay E. Measuring morbidity: self-report or health care records? Fam Pract. 2010 Feb;27(1):25–30. doi: 10.1093/fampra/cmp098. [DOI] [PubMed] [Google Scholar]
  • 16.El Fakiri F, Bruijnzeels MA, Hoes AW. No evidence for marked ethnic differences in accuracy of self-reported diabetes, hypertension, and hypercholesterolemia. J Clin Epidemiol. 2007 Dec;60(12):1271–9. doi: 10.1016/j.jclinepi.2007.02.014. [DOI] [PubMed] [Google Scholar]
  • 17.Leikauf J, Federman AD. Comparisons of self-reported and chart-identified chronic diseases in inner-city seniors. J Am Geriatr Soc. 2009 Jul;57(7):1219–25. doi: 10.1111/j.1532-5415.2009.02313.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Merkin SS, Cavanaugh K, Longenecker JC, Fink NE, Levey AS, Powe NR. Agreement of self-reported comorbid conditions with medical and physician reports varied by disease among end-stage renal disease patients. J Clin Epidemiol. 2007 Jun;60(6):634–42. doi: 10.1016/j.jclinepi.2006.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol. 2004 Oct;57(10):1096–103. doi: 10.1016/j.jclinepi.2004.04.005. [DOI] [PubMed] [Google Scholar]
  • 20.Simpson CF, Boyd CM, Carlson MC, Griswold ME, Guralnik JM, Fried LP. Agreement between self-report of disease diagnoses and medical record validation in disabled older women: factors that modify agreement. J Am Geriatr Soc. 2004 Jan;52(1):123–7. doi: 10.1111/j.1532-5415.2004.52021.x. [DOI] [PubMed] [Google Scholar]
  • 21.Skinner KM, Miller DR, Lincoln E, Lee A, Kazis LE. Concordance between respondent self-reports and medical records for chronic conditions: experience from the Veterans Health Study. J Ambul Care Manage. 2005 Apr-Jun;28(2):102–10. doi: 10.1097/00004479-200504000-00002. [DOI] [PubMed] [Google Scholar]
  • 22.Tisnado DM, Adams JL, Liu H, Damberg CL, Chen WP, Hu FA, Carlisle DM, Mangione CM, Kahn KL. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006 Feb;44(2):132–40. doi: 10.1097/01.mlr.0000196952.15921.bf. [DOI] [PubMed] [Google Scholar]
  • 23.Goldman N, Lin IF, Weinstein M, Lin YH. Evaluating the quality of self-reports of hypertension and diabetes. J Clin Epidemiol. 2003 Feb;56(2):148–54. doi: 10.1016/s0895-4356(02)00580-2. [DOI] [PubMed] [Google Scholar]
  • 24.Huerta JM, Tormo MJ, Egea-Caparros JM, Ortola-Devesa JB, Navarro C. Accuracy of self-reported diabetes, hypertension and hyperlipidemia in the adult Spanish population. DINO study findings. Rev Esp Cardiol. 2009 Feb;62(2):143–52. doi: 10.1016/s1885-5857(09)71532-4. [DOI] [PubMed] [Google Scholar]
  • 25.American Diabetes Association Report of the Expert Committee on the Diagnosis and Classification of Diabetes Mellitus. Diabetes Care. 1997 Jul;20:1183–97. doi: 10.2337/diacare.20.7.1183. [DOI] [PubMed] [Google Scholar]
  • 26.Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. Wiley-Interscience; New York: 1981. [Google Scholar]
  • 27.Hebert PL, Geiss LS, Tierney EF, Engelgau MM, Yawn BP, McBean AM. Identifying persons with diabetes using Medicare claims data. Am J Med Qual. 1999 Nov-Dec;14(6):270–7. doi: 10.1177/106286069901400607. [DOI] [PubMed] [Google Scholar]
  • 28.Little R, Vartivarian S. Does Weighting for Nonresponse Increase the Variance of Survey Means? Survey Methodology. 2005;3:161–8. [Google Scholar]
  • 29.Manson JE, Rimm EB, Stampfer MJ, Colditz GA, Willett WC, Krolewski AS, Rosner B, Hennekens CH, Speizer FE. Physical activity and incidence of non-insulin-dependent diabetes mellitus in women. Lancet. 1991 Sep 28;338(8770):774–8. doi: 10.1016/0140-6736(91)90664-b. [DOI] [PubMed] [Google Scholar]
  • 30.Fowles JB, Fowler EJ, Craft C. Validation of claims diagnoses and self-reported conditions compared with medical records for selected chronic diseases. The Journal of ambulatory care management. 1998 Jan;21(1):24–34. doi: 10.1097/00004479-199801000-00004. [DOI] [PubMed] [Google Scholar]
  • 31.Ngo DL, Marshall LM, Howard RN, Woodward JA, Southwick K, Hedberg K. Agreement between self-reported information and medical claims data on diagnosed diabetes in Oregon's Medicaid population. J Public Health Manag Pract. 2003 Nov-Dec;9(6):542–4. doi: 10.1097/00124784-200311000-00016. [DOI] [PubMed] [Google Scholar]
  • 32.Ching-Lin C, Lai MS, Syu CY, Chang SC, Tseng FY. Accuracy of diabetes diagnosis in health insurance claims data in Taiwan. J Formos Med Assoc. 2005 Mar;104(3):157–63. [PubMed] [Google Scholar]
  • 33.Gorina Y, Kramarow EA. Identifying chronic conditions in Medicare claims data: evaluating the Chronic Condition Data Warehouse algorithm. Health Serv Res. 2011 Oct;46(5):1610–27. doi: 10.1111/j.1475-6773.2011.01277.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Robinson JR, Young TK, Roos LL, Gelskey DE. Estimating the burden of disease. Comparing administrative data and self-reports. Medical Care. 1997;35(9):932–47. doi: 10.1097/00005650-199709000-00006. [DOI] [PubMed] [Google Scholar]
  • 35.Franse LV, Di Bari M, Shorr RI, Resnick HE, van Eijk JT, Bauer DC, Newman AB, Pahor M. Type 2 diabetes in older well-functioning people: who is undiagnosed? Data from the Health, Aging, and Body Composition study. Diabetes Care. 2001 Dec;24(12):2065–70. doi: 10.2337/diacare.24.12.2065. [DOI] [PubMed] [Google Scholar]
  • 36.Rathmann W, Haastert B, Icks A, Lowel H, Meisinger C, Holle R, Giani G. High prevalence of undiagnosed diabetes mellitus in Southern Germany: target populations for efficient screening. The KORA survey 2000. Diabetologia. 2003 Feb;46(2):182–9. doi: 10.1007/s00125-002-1025-0. [DOI] [PubMed] [Google Scholar]
  • 37.Wilder RP, Majumdar SR, Klarenbach SW, Jacobs P. Socio-economic status and undiagnosed diabetes. Diabetes Res Clin Pract. 2005 Oct;70(1):26–30. doi: 10.1016/j.diabres.2005.02.008. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Table 1
NIHMS542344-supplement-Table_1.doc (38.5KB, doc)
Table 2
NIHMS542344-supplement-Table_2.doc (38KB, doc)

RESOURCES