Sensitivity and Specificity of Death Certificates for Diabetes: As Good as it Gets?

Abstract

Objective

Diabetes is the sixth leading cause of death in U.S adults, which may be an underestimate due to under-reporting on the death certificate. This study examines death certificate sensitivity and specificity, and the factors related to better-reporting for diabetes in a community-based sample.

Research Design and Methods

Death certificates were obtained for 3209 decedents who were enrolled in the Rancho Bernardo cohort in 1972–74 and followed through 2003. Diabetes status was reassessed at periodic clinic visits and annual mailed surveys during an average follow-up of 15.2 ± 7.6 years. Diabetes reported anywhere on death certificates was abstracted. Sensitivity and specificity calculations among diabetic participants were stratified by age, sex, year, place, and cause of death, and diabetic medication use.

Results

Among 1641 men and 1568 women, 378 decedents had a history of diabetes; 168 of whom had diabetes listed anywhere on their death certificates. The sensitivity and specificity were 34.7% and 98.1%. Diabetes reporting on death certificates did not improve over time or vary significantly by age and sex, but sensitivity for diabetes reporting was better for recent (1992–2003) cardiovascular disease deaths compared to any other causes of death (48.9% vs. 28.6% respectively, p<0.05).

Conclusions

Although diabetes reporting on death certificates did not improve over time, sensitivity was better for diabetes in context of cardiovascular disease deaths, likely reflecting the increasing recognition that diabetes is a major cardiovascular risk factor.

The National Center for Health Statistics rates diabetes as the sixth major cause of death in the United States (1,2). However, this may be an underestimate. The American Diabetes Association reported that diabetes is listed anywhere on the death certificate less than half the time, and less than 20% of the time as the underlying cause of death (3). Several factors may contribute to this under-reporting: immediate cause of death masking contributing diseases (4,5), differences between physicians and others in reporting direct and contributing causes on death certificates (6–9), inaccuracy or variability in criteria for diagnosing diabetes, lack of training in completing death certificates (10,11), and variations in the perception of importance of contributing diseases to the final cause of death (7,12). Since mortality data are frequently utilized to estimate the burden or cost of disease and needed policy changes, it is important to examine factors related to validity of diabetes reporting on death certificates.

The few previous longitudinal studies of diabetes reporting on death certificates were conducted in foreign countries making comparisons with US data difficult (13–16), based on data collected prior to 1994 (17,18), or only from diabetic populations (4). In these studies the correct reporting of diabetes on death certificates ranged from 35–65%. To our knowledge, the only studies examining whether diabetes reporting on death certificates has improved in the last decade have been restricted to diabetic patients (19,20), where known diabetes status may bias toward better reporting on death certificates. Three national studies examining the reporting of diabetes on death certificates using interviews from the decedents’ kin (21,22) or medical history abstraction of diabetes (23), lacked direct access to decedents and were vulnerable to misclassification of diabetes status. Two other US studies relied upon last known physician of the decedent to ascertain diabetes status (4,24), potentially missing earlier diagnoses, or biasing diabetes reporting by over-sampling those with higher co-morbidity or disease severity.

The present study examines the changes and determinants of accurate reporting as assessed by the sensitivity and specificity of diabetes on death certificates in a population-based sample whose vital status was ascertained over 32 years.

Research Design and Methods

Participants

Between 1972 and 1974 all adults residing in the middle-class community of Rancho Bernardo, California were invited to participate in a study of cardiovascular risk factors. A total of 6339 men and women, representing 82% of all adult residents, were enrolled. Participants were followed since then with yearly mailed questionnaires and periodic clinic visits (25). In 1984–87, all participants aged 40 and older were invited to participate in a follow-up visit focused on diabetes; 85% of the eligible men and 78% of the women participated (26). Follow-up clinic visits in 1992–96 and 1997–99 also focused on diabetes. Questionnaires requesting information including diagnosis of diabetes were mailed in 1982, 1988, 1993, 1996, and 1998. Cumulative follow-up information from clinical visits and mailed surveys was available prior to death for more than 99% of all decedents (<1 year since last follow-up at death).

Vital status was known for 99% of the original cohort through the end of 2003. Death certificates were obtained for 3209 decedents who died before the end of 2003. This study was approved by the Institutional Review Board of the University of California, San Diego; written informed consent was obtained at all clinic visits.

Procedures

At each clinic visit and on mailed questionnaires, participants were asked whether they had ever been told by a physician they had diabetes, and whether they had any history of diabetes medication use. During each clinic visit, current medication use, including medications for diabetes, was verified by examination of pills and prescriptions brought to the clinic for that purpose. Information concerning diabetes from any and all visits and/or annual mailers was used to ascertain lifetime diagnosis of diabetes. Use of diabetes medication reported at any visit or mailer was used as a marker for more severe diabetes, which might be more likely to be reported on a death certificate.

Death certificates were coded by a nosologist using International Statistical Classification of Disease, version 9 (ICD9) criteria. Individuals were identified as having diabetes if the disease was listed anywhere on the death certificate. All sections of the death certificate (underlying cause of death, contributory causes of death, contributory diseases or conditions, and consequences that led up to the underlying cause of death) were examined for mention of diabetes (ICD code 250) and related conditions (e.g. hypoglycemia (251.1, 251.2), neuropathy in diabetes (357.2), diabetic retinopathy (362.0), diabetic cataract (366.41), or abnormal glucose in diabetes (648.8).

Variables potentially affecting the accuracy of reporting were abstracted from death certificates, and from data collected at clinic visits and mailed questionnaires. Dates of birth and baseline clinic visit, and sex were recorded. Date and place of death (home, hospital, or other), and underlying cause of death (diabetes (ICD9: 250), cardiovascular disease (ICD9: 390–459), malignancy (ICD9: 179–208), external cause (ICD: 905–9), or other causes of death) were obtained from death certificates.

Statistical Analysis

Before and after stratification by diabetes status, means and standard deviations were calculated for age variables; rates were calculated for categorical variables. Differences between decedents with and without known diabetes were examined using analysis of variance (ANOVA) and independent t-tests. Sensitivity and specificity were calculated for diabetes listed as underlying cause of death, and separately, for diabetes listed anywhere on the death certificate. Calculations of sensitivity and specificity were also stratified by sex, age at death in quartiles (<76, 77–82, 83–88, and ≥89 years), cause of death (cardiovascular disease (CVD) versus all other causes), place of death (home, hospital, or other), decade of death starting from 1972, and whether diagnosis of diabetes included use of medication for diabetes. Differences in sensitivity and specificity were examined with z-tests for binomial proportions.

Univariate and multivariate logistic regression analysis of data from 378 men and women with diabetes were used to assess potential variables that contributed to diabetes being reported or not reported on the death certificate. This included 322 men and women who reported a diagnosis of diabetes at a clinic visit or on a mailed questionnaire, plus 56 men and women who had not reported a diagnosis of diabetes during follow-up, but who had diabetes listed anywhere on the death certificate. Variables significantly contributing to univariate logistic regression models (p<0.10) were used to construct multivariate models predicting diabetes listed on death certificates by age, cause of death, sex, place of death, and/or use of diabetes medication. Potential effect modification of sex or year of death with cause, place, or age at death was examined by creating and testing interaction terms in the multivariate models. Data were examined using age and year of death as continuous and categorical variables. Analyses were performed with SAS, version 8.1 statistical software (SAS Institute, Cary, NC); all statistical tests were two-tailed.

Results

This study examined 3209 death certificates (>99%) from Rancho Bernardo Study participants who died between 1972 and 2003. As shown in Table 1, mean age at enrollment was 64.3 years (8.9); and the mean age at death was 81.9 years (9.6). A total of 322 decedents (10%) had known diabetes based on physician diagnosis or diabetes-specific medication use; of these 139 reported no use of diabetic medication. Approximately equal numbers of men and women (51.1% and 48.9% respectively, p>0.10) had died. Major causes of death were cardiovascular disease and cancer (41.3% and 23.1%, respectively); two-thirds (66.9%) died in a hospital. Also shown in Table 1, there was a greater percentage of men among those with diabetes than those without diabetes (60.6% vs. 50.1%, p<0.05). Those with known diabetes were on average slightly younger at enrollment than those without known diabetes (64.2 vs. 65.4 years old respectively, p<0.05) and had higher cancer rates (23.8% vs. 17.4%, p<0.05).

Table 1.

Descriptive characteristics of decedents with and without known diabetes Rancho Bernardo Study; 1972–2003.

Characteristics	All Decedents (n=3209)	Reported Diabetes (n=322)	No Known Diabetes (n=2887)
	Mean (Standard Deviation)
Age (years)
At Death	81.9 (9.6)	81.4 (8.8)	81.9 (9.7)
At Enrollment	64.3 (8.9)	64.2 (8.3)	65.4 (9.0)*
	% (n)
Sex
Men	51.1 (1641)	60.6 (195)	50.1 (1446)*
Women	48.9 (1568)	39.4 (127)	49.9 (1441)*
Cause of Death
Cancer	23.1 (742)	17.4 (56)	23.8 (686)*
CVD	41.3 (1324)	44.1 (142)	41.0 (1182)
External Cause	3.1 (98)	2.8 (9)	3.1 (89)
Other	31.8 (1022)	29.5 (95)	32.1 (927)
Diabetes	0.7 (23)	6.2 (20)	0.1 (3)
Place of Death
Hospital	66.9 (2136)	68.8 (221)	66.7 (1915)
Home	21.6 (689)	19.3 (62)	21.9 (627)
Other	11.5 (366)	11.8 (38)	11.4 (328)
Year of Death
1972–1981	22.3 (714)	20.2 (65)	22.5 (649)
1982–1991	41.3 (1324)	39.8 (128)	41.4 (1196)
1992–2003	36.5 (1171)	40.1 (129)	36.1 (1042)

^*Significant differences (p<0.05) comparing diabetic versus non-diabetic decedents

Of the 322 individuals with known diabetes, diabetes was listed as primary cause of death for only 20 (6%); an additional 92 (29%) had diabetes listed elsewhere on their death certificates (sensitivity). From those individuals with no known diabetes history or medications, 56 individuals had diabetes listed on their death certificates (specificity). As shown in Table 2, the overall sensitivity and specificity of reporting diabetes anywhere on the death certificate was 34.7% and 98.1%, respectively. The sensitivity and specificity for reporting diabetes as the underlying cause of death on the death certificate was 6.2% and 99.8%, respectively.

Table 2.

Comparisons of sensitivity and specificity of death certificates in reporting diabetes after stratification by characteristics using z-test of binomial proportions: the Rancho Bernardo Study, 1972–2003.^*

Characteristic	Sensitivity	Specificity	N
Overall Sensitivity
Anywhere on Death Certificate	34.7	98.1	3209
Underlying Cause of Death	6.2	99.8	3209
Age at Death (in quartiles)
≤76 years old*	36.7	98.6	863
77–82 years old	31.6	97.8	719
83–88 years old	40.2	97.9	856
89+ years old	29.2	97.9	771
Sex
Men	31.8	98.0	1641
Women*	39.4	98.1	1568
Cause of Death
CVD	39.4†	97.6	1324
All Other Causes*	22.5	98.5	1862
Place of Death
Hospital	35.8†	98.2	2136
Home	37.1	98.6	689
Other*	26.3	96.3	366
Year of Death
1972–1981*	36.9	98.0	714
1982–1991	33.6†	98.6	1324
1992–2003	34.9†	97.5	1171
Diagnostic Method
Doctor’s Diagnosis Only*	16.6	95.3	139
Doctor’s Diagnosis & Medication Use	48.6†	97.4	183

^*Significance examined using z-test of binomial proportions (reference categories were <76 years old, other places of death and year of deaths from 1972–1981)

^†p<0.05

Table 2 also shows the sensitivity and specificity of diabetes reporting on the death certificate after stratifying by age, sex, cause of death, place, and year of death, and diabetes medication use. CVD deaths had greater sensitivity compared to death by all other causes (39.4% vs. 22.5%, p<0.05). Diabetes sensitivity was better for those who died in hospitals or at home vs. other locales, but still poor (35.8% and 37.1% vs. 26.3%, p<0.05). Additionally, sensitivity for diabetes reported on the death certificate was significantly better among diabetic adults using medication for diabetes as compared to those not using medication (48.6% vs. 16.6%, p<0.05). Comparisons by decade of death showed greater sensitivity for deaths in 1972–1981 vs. more recent deaths (p<0.05), although no discernible trends were apparent when death was categorized by decade (data not shown). There were no other significant differences in sensitivity or specificity by sex or age.

The Figure shows the sensitivity of diabetes reporting for those who died of CVD compared to all other causes of death after stratification by decade of death. As shown, sensitivity for reporting diabetes in 1992–2003 for CVD -related deaths was significantly better than the sensitivity for all other causes of death (48.9% vs. 26.8%, p<0.05).

Figure 1. Sensitivity of Recording of Diabetes on Death Certificates by Cause of Death Stratified by Decade of Death: the Rancho Bernardo Study, 1972–2003.

*Significant difference (p-value<0.05) in diabetes sensitivity between those dying from CVD versus All Other Causes of Death from 1992–2003

1972–1981, 42.9% CVD, n=714; 1982–1991, 42.5% CVD, n=1324; 1992–2003, 38.9% CVD, n=1171

Table 3 presents the association of age, sex, and other characteristics with likelihood of diabetes being reported on death certificates among adults with known diabetes or those with diabetes listed anywhere on the death certificate. Those who used diabetes medications were nearly five times (odds ratio 4.78 95% confidence interval= 2.80, 8.14) more likely to have diabetes listed on the death certificate than diabetic decedents with no reported diabetes medication use. No other characteristic significantly improved reporting of diabetes on death certificates. Multivariate models examining the combined effects of diagnostic method, sex, and cause, place, age, and year of death does not materially change the observed univariate associations. Interactions of sex, age, decade or cause of death, or follow-up time with diabetes status, and age and decade of death with cause of death, did not significantly affect likelihood of reporting diabetes (p>0.05; data not shown).

Table 3.

Associations^* of characteristics with likelihood of diabetes being reported on death certificates for those with known diabetes^† or diabetes listed anywhere on the death certificate^§; Rancho Bernardo Study, 1972–2003.

Characteristic	Univariate OR (95% CI)	Multivariate Log Reg OR (95%CI)
Age (yrs)	1.00 (0.98,1.02)	1.01 (0.98,1.03)
Sex
Women v. Men	1.47 (0.96,2.21)	1.71 (1.03,2.83)
Cause of Death
CVD vs. Other	1.44 (0.96,2.17)	1.62 (0.98,2.67)
Place of Death
Hospital vs. Other	1.04 (0.50,2.16)	1.52 (0.56,4.16)
Home vs. Other	1.02 (0.56,1.88)	1.08 (0.45,2.62)
Year of Death (yrs)	1.00 (0.98,1.03)	1.01 (0.97,1.05)
Diagnostic Method
Diagnosis & Medication vs. Only Diagnosis	4.78 (2.80,8.14)	8.75 (5.45,14.06)

^*Results of univariate and multivariate logistic regression reporting odds ratio (OR) and 95% confidence intervals (95%CI)

^†n=322

^§n=56

Conclusions

Previous mortality studies indicate that diabetes is under-reported as a direct cause of death, and often not mentioned anywhere on death certificates (4,5,19,21,22,27–31). The present study shows that among those with known diabetes, overall sensitivity was low (34.7%) but specificity of diabetes reporting was good (98.1%). Diabetes was listed as a direct or contributing cause of death on only 6.2% of death certificates for adults who had known diabetes; specificity was excellent, 99.8%. Compared to non-diabetic adults, the proportion of CVD deaths was higher in diabetic participants, and diabetes reporting was better among those dying of CVD. Additionally, sensitivity for diabetes reporting was higher among hospital deaths and for participants using diabetes-specific medications. Sensitivity did not vary significantly by year or age of death. Greater sensitivity for diabetes reporting was noted when CVD was listed as the cause of death, but only for death occurring between 1992–2003. This may reflect increased awareness of the important association of diabetes with CVD leading to increased reporting of diabetes as a contributing cause of death among patients with fatal CVD, or this may reflect the documented contribution of diabetes towards CVD incidence or mortality. Further, good medical practice now requires that patients hospitalized with CVD be evaluated for diabetes.

Other US studies have also reported poor death certificate sensitivity for diabetes (4,17,18,20,23,32), and document the resultant bias when mortality data are used to estimate the burden of diabetes in the population. The present study, which is the only longitudinal study to include data from the last decade, shows no improvement of overall diabetes reporting on death certificates. All other US population based studies were based on data collected prior to 1992 or small samples (n<550), and had sensitivity ranging from 35%–54% (17,20–23,27).

In the present study, there was no significant sex difference in sensitivity or specificity. In contrast two other studies reported somewhat better sensitivity in men (23,28). Better sensitivity was also reported in studies with smaller (n<550) sample sizes (4,14,16,19,20,24,27,28,31) and limited to patients with diabetes (4,14,16,19,31). Studies outside the US tended to report higher sensitivity (51%–70%), but differences in diabetes diagnostic criteria and procedures for completion of death certificates make comparisons difficult (13,14,16,28,31,33,34).

This study also assessed whether sensitivity improved between 1972 and 2003, with the increasing awareness of the association between CVD and diabetes. Given the significant contribution of diabetes to both the incidence of CVD and mortality (14,16,32,35,36), it is reasonable to expect improved reporting of diabetes for CVD versus all cause mortality in the last decade, evidenced by the observed improvement in diabetes reporting among those dying of CVD in the third decade of our study. However, the lack of statistical significance argues for a conservative interpretation of this observation; consistent with other studies, this study also shows that the overall sensitivity of diabetes reporting did not improve over time (22).

Several factors may decrease diabetes reporting on death certificates (12). Because diabetes is often not the direct cause of death but a contributing or underlying cause, it may not be mentioned on the death certificate. Differences between physicians with regard to ranking diseases leading up to death may account for some of the variability in diabetes reporting on death certificates (6–8,37,38). Lack of accurate medical data on decedents may lead to under-reporting of diabetes. Differences in diagnostic criteria for diabetes may account for additional variability in diabetes reporting. Although decedents may have had glucose tolerance tests suggesting hyperinsulinemia or hyperglycemia, physicians may have been reluctant to diagnose patients with diabetes if the condition was controlled without medication.

Plasma glucose measures, available for most participants, were not included since the objective was to examine the validity of death certificates in identifying known diabetes status, which is a conservative measure of the underestimation by mortality data. Single episodes of hyperglycemia observed in a research clinic (although reported to participants) may not parallel participant or physician awareness of diabetes status or be confirmed by repeat testing. The presumed best scenario for diabetes on the death certificate would occur if both the patient and doctor knew that the patient had diabetes. Analyses based on glucose levels identified sensitivity of 25% based on WHO guidelines (data not shown).

Several strengths and limitations of this study were considered. This longitudinal study had access to participant data from enrollment until death, and therefore, had direct, participant-provided ascertainment of diabetes status. The high specificity rates (>95%) reflect the strength of this study with multiple points of data ascertainment. Several other death certificate studies based the diabetes diagnosis on proxy reports from spouses or relatives (19,21,22). Information on diabetes status collected from the surrogates is expected to increase misclassification; medication use served as a confirmation of diabetes diagnosis. Participants in the Rancho Bernardo Study are almost entirely white, relatively well educated and middle to upper middle class, with good access to medical care. Although results from this study may not generalize to other populations, they are similar to those from other US-based studies (17,18,21,22). Thus, self-reported physician diagnosis of diabetes was likely reliable.

In conclusion, the overall sensitivity of diabetes on death certificates in this study was only 35%, with no evidence of overall temporal improvement in diabetes reporting on the death certificate. Only within the last decade did reporting improve for those whose death was attributed to CVD. Medication use among diabetic decedents was the only other significant determinant of better diabetes reporting, likely reflecting greater severity of diabetes and greater awareness of the physician and the patient. The persistent under-reporting of diabetes on death certificates underestimates the burden of disease and the influence of diabetes on death rates. Future studies should focus on examining the effect of duration of disease, different diagnostic criteria, and more diverse populations on diabetes reporting on death certificates. Improved education on the completion of death certificates is recommended.