Agreement on Cause of Death Between Proxies, Death Certificates, and Clinician Adjudicators in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) Study

Abstract

Death certificates may lack accuracy and misclassify the cause of death. The validity of proxy-reported cause of death is not well established. The authors examined death records on 336 participants in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) Study, a national cohort study of 30,239 community-dwelling US adults (2003–2010). Trained experts used study data, medical records, death certificates, and proxy reports to adjudicate causes of death. The authors computed agreement on cause of death from the death certificate, proxy, and adjudication, as well as sensitivity and specificity for certain diseases. Adjudicated cause of death had a higher rate of agreement with proxy reports (73%; Cohen's kappa (κ) statistic = 0.69) than with death certificates (61%; κ = 0.54). The agreement between proxy reports and adjudicators was better than agreement with death certificates for all disease-specific causes of death. Using the adjudicator assessments as the “gold standard,” for disease-specific causes of death, proxy reports had similar or higher specificity and higher sensitivity (sensitivity = 50%–89%) than death certificates (sensitivity = 31%–81%). Proxy reports may be more concordant with adjudicated causes of death than with the causes of death listed on death certificates. In many settings, proxy reports may represent a better strategy for determining cause of death than reliance on death certificates.

Ascertaining the cause of death is essential for classifying disease-specific mortality in epidemiologic and clinical studies. Death certificates often serve as the source of this information, but causes of death on death certificates are often fraught with misclassification of the underlying cause of death, which may potentially lead to bias in reporting cause-specific mortality (1). This potential bias has contributed to calls for more structured training of medical students on completing death certificates (2). Furthermore, obtaining death certificates is time-consuming and resource-intensive. Death certificates can be acquired from the deceased's family or local health departments, but most often data from death certificates are purchased from the Centers for Disease Control and Prevention (CDC) through the National Death Index (3). Obtaining cause-of-death data can be slow; for instance, there is a 2-year lag between a participant's year of death and when the death records become available through the National Death Index, with additional time required for an application to be processed (3). In addition, the CDC charges approximately $2,450 to search the National Death Index for 1,000 records over a 10-year period (4).

To address these concerns, many investigators determine cause of death using a team of expert clinician adjudicators. These adjudication committees often use clinical records, death certificates, and internationally recognized criteria to classify the cause of death (5). Although classifying cause of death by means of an adjudication committee is the gold standard, the amount of time and resources required to assemble materials and clinicians for adjudication on a regular basis may be beyond the scope of some studies. Bangdiwala et al. (6) estimated these costs in 1989 to be approximately $54.68 per death (excluding labor). Moreover, in large studies with multiple outcomes, convening separate disease-specific adjudication committees, considering the limited number of clinicians who are trained in outcome-specific adjudication criteria, leads to significant delays in adjudicating some events (7). This delay, in turn, hinders the analysis of study outcomes and dissemination of important research findings. Our experience indicates that, although deceased participants’ next of kin or friends (referred to henceforth as “proxies”) may not always wish to provide copies of death certificates, they are frequently willing to provide information about the health-related circumstances surrounding the death. Very little information is available about the validity of proxy-provided causes of death or the agreement between proxy-provided cause of death, cause of death from the death certificate, and cause of death as adjudicated by a team of clinician experts.

The Reasons for Geographic and Racial Differences in Stroke (REGARDS) Study is a national prospective cohort study that is investigating causes of death from stroke, myocardial infarction, sepsis, and other outcomes among 30,239 community-dwelling adults aged 45 years or older. Expert clinicians are adjudicating the cause of death using death certificates, medical records from recent hospitalizations, and interviews with proxies. We examined the agreement between causes of death assessed by proxy, death certificates, and clinician adjudication among deaths reported in the REGARDS Study.

MATERIALS AND METHODS

Study population

The REGARDS Study involves a cohort of persons recruited from across the United States to study the geographic and racial variation in stroke mortality. Details on recruitment and data collection are available elsewhere (8). Briefly, from 2003 to 2007, using commercially available lists, community-dwelling persons aged 45 years or older were contacted by telephone and mail. Demographic recruitment goals were to enroll 50% African Americans and 50% women. The regional recruitment goal was to enroll 50% of the participants from the “Stroke Buckle” (the coastal plains of North and South Carolina and parts of Georgia) and the “Stroke Belt” (the remainder of North and South Carolina and Georgia, as well as Tennessee, Alabama, Mississippi, Louisiana, and Arkansas) and 50% from the rest of the continental United States.

After obtaining verbal and written informed consent, baseline data were collected via telephone interviews and in-home visits. Follow-up is being conducted with participants using telephone interviews every 6 months. This analysis included all deceased participants for whom the study investigators had: 1) obtained the death certificate (which included cause of death), 2) conducted a proxy interview regarding the participant's cause of death, and 3) adjudicated the cause of death between January 2003 and January 2009. The study was approved by the University of Alabama at Birmingham's institutional review board.

Identifying death events

Deaths were recorded through active surveillance and were detected in 3 ways. Deaths were most commonly detected when REGARDS interviewers called for the 6-month follow-up and were informed of the death by the participant's family or friend. Deaths were also identified through a search of Web-based restricted-access search engines (e.g., Lexis-Nexis) and the Social Security Death Index for participants lost to follow-up. Searches were done by name and Social Security number. Finally, some deaths were identified when the next of kin or proxy called the REGARDS Study's toll-free number and reported the death.

Cause of death from proxy interviews

Upon receiving a report of the death, a trained interviewer conducted a telephone interview with the participant's proxy (the next of kin, family member, or close friend). The interviewer used a semistructured interview to determine the events leading up to the participant's death and the cause of death. Data were collected regarding the person's relationship to the participant, the participant's date of death, and the details of any hospitalizations prior to the participant's death. The proxy's responses were recorded verbatim. Because the proxies were not REGARDS Study participants, no data were collected on them other than their relationship to the participant.

Cause of death from death certificates

Death certificates were obtained from the participant's next of kin or state department of health. The cause-of-death section of death certificates contains 2 parts (9). The first part reports the chain of events leading directly to death. The immediate cause of death is noted on the first line, followed by the underlying cause(s) of death (the disease(s) or injury that initiated the chain of morbid events that led directly and inevitably to death). The second part reports all other significant conditions contributing to death but not resulting in the immediate cause of death. As recommended by the CDC, we used the last entry for the underlying cause of death to assign the death certificate cause of death (10). For example, if coronary artery disease is listed as the underlying cause of death and acute myocardial infarction is listed as the immediate cause of death, the CDC recommends using coronary artery disease as the official cause of death.

Cause of death from clinician adjudicators

The adjudicators were clinicians (general internists, cardiologists, and physician assistants) who underwent specific training for adjudicating myocardial infarctions and deaths. The approach to determination of cause of death was modeled on other epidemiologic studies such as the Atherosclerosis Risk in Communities Study (11) and the Women's Health Initiative (12). Death cases were reviewed independently by 2 adjudicators, and disagreements were decided by committee. The prespecified minimum level of acceptable agreement on cause of death was 80%; if an adjudicator fell below this level, retraining was initiated. Adjudicators used baseline participant clinical characteristics, proxy interviews, death certificates, and, if available, medical records from hospitalizations occurring within 30 days of the participant's death to determine the cause of death.

Using all available data, adjudicators were asked to determine the main underlying cause of death, defined as the 1 disease or injury that initiated the events resulting in death. Response options included: 1) definite, probable, or possible myocardial infarction, per American Heart Association case definitions (13); 2) stroke, per the World Health Organization case definition (8); 3) sudden cardiac death (death occurring within 1 hour of onset of symptoms and suggestive of an arrhythmic event); 4) congestive heart failure; 5) other cardiac cause (e.g., myocarditis); 6) not cardiac but other cardiovascular cause (e.g., ruptured aortic aneurysm); 7) cancer; 8) accident/injury/suicide/homicide; 9) other noncardiac, nonstroke cause designated by the adjudicator (e.g., cirrhosis); and 10) unclassifiable. If definitive biochemical, radiologic, or pathologic evidence was lacking or if the underlying cause of death occurred months prior to death, the adjudicators used their clinical judgment based on comorbid conditions, hospital presentation, proxy reports, and the death certificate to determine cause of death.

Statistical analyses

Continuous variables were summarized as mean values with standard deviations, and categorical variables were summarized as percentages. Causes of death that were derived from proxy interviews, death certificates, and adjudication were categorized broadly into heart-related conditions, stroke, cancer, infection, lung-related conditions, and other causes. Cross-tabulations were used to derive the percentage of agreement between proxy-reported, death certificate-reported, and adjudicated causes of death. Cohen's kappa (κ) statistic was used to assess the degree of agreement between the different methods of ascertaining cause of death. In addition, agreements were assessed individually for causes of death due to heart-related conditions, stroke, cancer, infection, injury, and other conditions. Using the GLM procedure in SAS, we conducted a test of homogeneity of variance of the agreement (κ) across causes of death.

We also conducted analyses to determine whether or not the type of illness predicted agreement between 1) proxy-reported and adjudicated cause of death and 2) death certificate and adjudicated cause of death. Bivariate odds ratios were derived for each of the predictor variables, with agreement between sources of cause of death serving as the dependent variable. Using a backward selection process, multivariable adjusted odds ratios were estimated by logistic regression. Examined predictor variables included participant age, education, health insurance status, marital status, race, and rurality of residence. Predictor variables with a P value of 0.05 or less were retained in the final model. All analyses were performed using SAS, version 9.1.3 (SAS Institute Inc., Cary, North Carolina).

RESULTS

At the time of this analysis (January 2009), 1,883 REGARDS participants were deceased. Participants with missing data were excluded from this analysis, for the following reasons: 497 (26.4%) did not have a death certificate or proxy interview, 51 (2.7%) did not have a proxy interview, 734 (39.0%) did not have a death certificate, 16 (0.8%) did not have a cause of death included on the death certificate, and 249 (13.2%) had deaths that had not yet been adjudicated. A total of 336 participants (17.8% of deceased REGARDS participants) were included in this analysis.

Participant characteristics are shown in Table 1. Compared with the analysis cohort, the excluded group had more women (39% vs. 29%) and more African Americans (48% vs. 24%). In this analysis, the average age of subjects at enrollment was 72 years (standard deviation, 9). Approximately two-thirds were male, married, and earned at least $20,000 per annum. A quarter of participants were African Americans. Most (82%) had at least a high school education. Almost all (96%) were insured, and three-quarters had access to a clinic or a physician providing medical care. In 90% of cases, the proxy who provided the cause-of-death information was a first-degree relative.

Table 1.

Characteristics of 336 Deceased REGARDS Participants With Available Death Certificates, Proxy Interviews, and Recent Hospitalizations, 2003–2010^a

Characteristic	No.	%
Female gender	97	29
African-American race	81	24
Insuredb	324	96
Married	235	70
Less than high school education	59	18
Annual income
<$20,000	65	19
$20,000–$34,999	114	34
$35,000–$74,999	72	21
≥$75,000	32	10
Refused to answer question	53	16
Regionc
Stroke Buckle	68	20
Stroke Belt	151	45
Rest of the United States	117	35
Ruralityd
Rural (<25% urban)	84	25
Mixed (25%–75% urban)	41	12
Urban (>75% urban)	211	63
Proxy was a first-degree relative	303	90
Access to clinic or physician providing medical care	239	71

Abbreviation: REGARDS, Reasons for Geographic and Racial Differences in Stroke.

^aThe mean age of the study participants was 72 years (standard deviation, 9).

^bHaving any type of health insurance.

^cThe Stroke Buckle is the coastal plains of North and South Carolina and parts of Georgia; the Stroke Belt is the remainder of North and South Carolina and Georgia, as well as Tennessee, Alabama, Mississippi, Louisiana, and Arkansas.

^dRurality of county was based on criteria set by the US Census Bureau.

Table 2 presents the relationship of the proxies to the REGARDS Study participants. Slightly over half were participants’ spouses, and nearly 30% were participants’ children.

Table 2.

Relationship of Proxies to the 336 Deceased Participants in the REGARDS Cohort Study, 2003–2010

Relationship of Proxy to Participant	No.	%
Spouse	190	56.6
Child	94	27.9
Sibling	16	4.8
Othera	36	10.7

Abbreviation: REGARDS, Reasons for Geographic and Racial Differences in Stroke.

^aIncluded in-laws (n = 18), friends (n = 3), cousins (n = 4), grandsons (n = 3), nephews (n = 2), stepchildren (n = 2), companions/partners (n = 2), a Baptist foundation (n = 1), and a widow's sister (n = 1).

Table 3 presents the rate of agreement between adjudicator-, proxy-, and death certificate-reported causes of death. Percent agreement and Cohen's kappa statistic were highest between adjudicators and proxies, followed by adjudicators and death certificates.

Table 3.

Agreement Between Adjudicator-, Proxy-, and Death Certificate-Reported Causes of Death (n = 336) in the REGARDS Cohort Study, 2003–2010

Comparison	Agreement Frequencya	Agreement Rate, %b	κc (SE)	95% CI
Adjudicator–proxy	246	73	0.69 (0.03)	0.63, 0.75
Adjudicator–death certificate	206	61	0.54 (0.04)	0.46, 0.61
Proxy–death certificate	196	58	0.49 (0.04)	0.41, 0.56
Adjudicator–proxy–death certificate	170	51

Abbreviations: CI, confidence interval; REGARDS, Reasons for Geographic and Racial Differences in Stroke; SE, standard error.

^aNumber of times there was agreement between the methods of classifying cause of death.

^bAgreement frequency/336 × 100.

^cCohen's kappa (κ) statistic for observer agreement.

Table 4 and Table 5 show the sensitivity and specificity of proxy reports and death certificates, respectively, versus adjudication. The sensitivity of the proxy reports was higher than that of death certificates for all causes of death. Specificity was similar for proxies and death certificates. The agreement, determined by Cohen's kappa statistic, between proxy reports and adjudicator-determined cause of death was substantial for stroke (κ = 0.73), heart-related diseases (κ = 0.65), and respiratory diseases (κ = 0.74). When the cause of death was obtained from the death certificate (compared with the adjudicated cause of death), the kappa statistic was more moderate for stroke (κ = 0.60), heart-related diseases (κ = 0.57), and respiratory diseases (κ = 0.52). The kappa statistic was quite high for reported cause of death for cancer in both proxy reports (κ = 0.87) and death certificates (κ = 0.77). Tests for homogeneity in kappa across causes of death resulted in an F statistic of 6.03 (P < 0.0001) for agreement between proxy reports and adjudicated causes of death and an F statistic of 8.97 (P < 0.0001) for agreement between death certificates and adjudicated causes of death.

Table 4.

Accuracy of Proxy-Reported Causes of Death as Compared With Adjudication in 336 Deceased Participants From the REGARDS Cohort Study, 2003–2010

Cause of Death	No.	%	Sensitivity, %	Specificity, %	PPV, %	NPV, %	Agreement Rate, %a	κb,c (SE)	95% CI
Stroke	33	10	82	96	71	98	95	0.73 (0.06)	0.61, 0.85
Heart-related	102	30	68	94	83	87	86	0.65 (0.05)	0.56, 0.74
Cancer	83	25	89	97	91	96	95	0.87 (0.03)	0.81, 0.93
Respiratory	28	8	79	97	73	98	96	0.74 (0.07)	0.60, 0.87
Infection	36	11	50	97	67	94	92	0.53 (0.08)	0.37, 0.68
Injury	15	4	67	98	67	98	97	0.65 (0.10)	0.45, 0.85
Other	39	12

Abbreviations: CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; REGARDS, Reasons for Geographic and Racial Differences in Stroke; SE, standard error.

^aAgreement frequency/336 × 100.

^bCohen's kappa (κ) statistic for observer agreement.

^cTest for homogeneity in kappa across causes of death: F statistic = 6.03, P < 0.0001.

Table 5.

Accuracy of Death Certificate Causes of Death as Compared With Adjudication in 336 Deceased Participants From the REGARDS Cohort Study, 2003–2010

Cause of Death	No.	%	Sensitivity, %	Specificity, %	PPV, %	NPV, %	Agreement Rate, %a	κb,c (SE)	95% CI
Stroke	33	10	52	99	81	95	94	0.60 (0.08)	0.44, 0.76
Heart-related	102	30	61	93	78	84	83	0.57 (0.05)	0.47, 0.67
Cancer	83	25	81	95	84	94	91	0.77 (0.04)	0.68, 0.85
Respiratory	28	8	61	95	53	96	92	0.52 (0.08)	0.36, 0.68
Infection	36	11	31	96	50	92	89	0.32 (0.08)	0.16, 0.49
Injury	15	4	60	100	90	98	98	0.71 (0.10)	0.51, 0.91
Other	39	12

Abbreviations: CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; REGARDS, Reasons for Geographic and Racial Differences in Stroke; SE, standard error.

^aAgreement frequency/336 × 100.

^bCohen's kappa (κ) statistic for observer agreement.

^cTest for homogeneity in kappa across causes of death: F statistic = 8.97, P < 0.0001.

Table 6 and Table 7 present results from the multivariable regression analyses of agreement between sources of cause of death for selected diseases. In both proxy–adjudicator agreement and death certificate–adjudicator agreement, the odds ratios for agreement were significantly higher for cancer, stroke, and respiratory diseases than for all other conditions. Proxy–adjudicator agreement also had significantly higher odds ratios for heart-related disease and infection relative to all other conditions.

Table 6.

Crude and Adjusted Odds Ratios for Agreement Between Proxy-Reported and Adjudicated Causes of Death for Selected Diseases in the REGARDS Cohort Study, 2003–2010

Cause of Deatha	Crude OR	Adjustedb
		OR	95% CI
Cancer	14.64	16.19	5.99, 43.79
Heart-related	6.82	6.27	2.77, 14.20
Stroke	3.40	3.04	1.21, 7.61
Infection	2.77	3.24	1.17, 8.99
Injury	2.77	1.65	0.45, 6.03
Respiratory	3.81	4.67	1.68, 13.01

Abbreviations: CI, confidence interval; OR, odds ratio; REGARDS, Reasons for Geographic and Racial Differences in Stroke.

^aProxy and adjudication agreement for the selected disease as compared with all other diseases.

^bAdjusted for age, education, health insurance status, marital status, race, and rurality of residence.

Table 7.

Crude and Adjusted Odds Ratios for Agreement Between Death Certificate-Reported and Adjudicated Causes of Death for Selected Diseases in the REGARDS Cohorts Study, 2003–2010

Cause of Deatha	Crude OR	Adjustedb
		OR	95% CI
Cancer	4.22	5.08	2.25, 11.47
Heart-related	1.74	1.92	0.95, 3.85
Stroke	6.39	6.64	1.37, 32.27
Infection	1.18	1.47	0.52, 4.16
Injury	1.57	1.06	0.23, 4.95
Respiratory	2.40	3.20	1.15, 8.91

Abbreviations: CI, confidence interval; OR, odds ratio; REGARDS, Reasons for Geographic and Racial Differences in Stroke.

^aDeath certificate and adjudication agreement for the selected disease as compared with all other diseases.

^bAdjusted for age, education, health insurance status, marital status, race, and rurality of residence.

DISCUSSION

In this population-based national sample, we found that agreement between adjudicator-determined cause of death and proxy-reported cause of death was higher than that for adjudicator-determined and death certificate-reported cause of death. Furthermore, proxy reports, compared with death certificates, had a higher sensitivity, specificity, and positive predictive value for most diseases when adjudicated cause of death was used as the gold standard. The sensitivity of proxy reports in detecting deaths due to infection, injury, and heart-related diseases was fairly low but still better than that of death certificates.

Our findings suggest that prospective studies may use proxy interviews as a reasonable alternative to death certificates in determining cause of death—at least for the causes studied here. Given the frequency with which deaths from coronary heart disease and sudden cardiac death are misclassified or overestimated (11, 14) and evidence that indicates physician bias in documenting underlying causes of death on death certificates (15), our findings suggest that proxy reports may be a better source of cause of death than death certificates. These results are similar to findings from studies that indicate the need for caution when utilizing death certificates for case identification (11, 14).

Our findings are interesting given that many large cohort studies rely on death certificates in spite of the mixed results obtained when using them (1, 11, 16). Studies in extant literature have mostly identified moderate agreement between adjudicator- and death certificate-reported causes of death; for example, the kappa statistic for observer agreement obtained in the current study (κ = 0.54) was similar to that obtained by Bangdiwala et al. (κ = 0.58) (1, 6, 17).

The consistency with which the proxy reports agreed with adjudicated causes of death implies that proxy reports may be used by investigators to classify death endpoints before the death certificate data become available through the National Death Index. This has significant implications for more timely availability of research results to be applied to clinical practice and policy formulation. Moreover, because researchers, public health practitioners, and policy-makers often use death statistics to assess the impact of diseases on survival and to guide decisions on the allocation of resources, using proxy reports could provide a reasonable alternative to the use of expensive adjudication committees to determine causes of death.

In 2 studies that have assessed the utility of reports from interviewing proxies, investigators have advocated the need to supplement death certificate reports with interviews of proxies and physicians (11, 18). Coady et al. (11) found that death certificates tended to overestimate deaths from coronary heart disease by as much as 20%, while Wang et al. (18) found that using verbal autopsies was helpful in reducing the proportion of deaths reported with ill-defined causes. Studies that have used proxies to obtain information on patients have focused mainly on patients’ symptoms or quality of life in defined clinical conditions (19–21). These studies found significant accuracy and validity in the information provided by proxies when compared with physician-documented reports and patients’ self-reports. This validity was increased when the questions asked were well-defined and specific (19, 22). After-death interviews of proxies also gave accurate information about the prevalence of malignant neoplasms, cerebrovascular diseases, and chronic diseases (23). It may be reasonable to assume that the validity of causes of death reported by the proxies in REGARDS may be similar to what the latter studies have already demonstrated.

From the foregoing paragraphs, it is clear that adjudication committees have the full range of information necessary to arrive at the cause of death, which includes information from death certificates and proxy reports. Proxy reports may also have information from death certificates and firsthand experience of the circumstances surrounding the death of the relative or friend. Death certificates, on the other hand, have comparatively less information on which to base the cause of death.

This study had potential limitations. First, proxies have different educational and sociocultural backgrounds, as well as different degrees of involvement with the participant. Hence, the information obtained from a proxy is subject to varying levels of accuracy and must be assessed with caution. Additionally, in some instances, proxies report the cause of death directly from the official death certificate, so those data may be inaccurate for the reasons already discussed. Similarly, adjudication committees have information from death certificates and proxy reports, which inherently leads to correlation in the cause-of-death classification between the 3 methods. Our relatively small sample size was another limitation. We only had enough cases to examine causes of death when they were grouped into large categories (e.g., heart-related deaths varied from coronary artery disease to ruptured abdominal aortic aneurysm), and we could not examine some causes at all (e.g., kidney and liver disease). Finally, the sample included only 18% of deceased REGARDS participants, because for some deaths study staff had not yet obtained either the death certificate or a proxy interview or both or because records were awaiting adjudication. The participants included in this analysis were not representative of the entire REGARDS cohort, which is 41% African-American and 53% female, or representative of all deceased REGARDS participants, who are 44% African-American and 38% female. The differences in the racial and gender composition of the included sample were due to there being more missing death certificates among African Americans, especially African-American women. This is probably due to the fact that the primary source for death certificates was next of kin. African Americans are less likely to participate in research (24), which may contribute to more African-American next of kin not providing death certificates. In addition, the proxies of female participants were less often spouses, and more distantly related proxies may be less likely to provide death certificates. The REGARDS staff is continuing to gather death certificates and medical records, so in the future we will be able to reexamine the validity of proxy-reported and death certificate-reported causes of death.

In conclusion, this study corroborates earlier studies that found that death certificates are not always accurate sources of cause of death. When compared with death certificates, the reports provided by proxies had a higher degree of agreement with expert clinician-adjudicated cause of death. These findings were particularly robust when the cause of death was cancer, stroke, or respiratory disease, but the sensitivity of proxy reports in detecting heart-related, respiratory, and infection-related deaths was lower. Although the numbers in each disease category were relatively modest, these findings provide evidence on the reliability of exit interviews with relatives and friends for determining death-related study endpoints. This may help save human and financial resources that are frequently deployed towards obtaining death certificates and permit investigators who cannot afford the expense of adjudication to obtain reasonably reliable information on causes of death.

Glossary

Abbreviations

CDC

Centers for Disease Control and Prevention

REGARDS

Reasons for Geographic and Racial Differences in Stroke