Abstract
Objectives
Inaccuracies in cause-of-death information in death certificates can reduce the validity of national death statistics and result in poor targeting of resources to reduce morbidity and mortality in people with HIV. Our objective was to measure the sensitivity, specificity, and agreement between multiple causes of deaths from death certificates obtained from the National Death Index (NDI) and causes determined by expert physician review.
Methods
Physician specialists determined the cause of death using information collected from the medical records of 50 randomly selected HIV-infected people who died in San Francisco from July 1, 2016, through May 31, 2017. Using expert review as the gold standard, we measured sensitivity, specificity, and agreement.
Results
The NDI had a sensitivity of 53.9% and a specificity of 66.7% for HIV deaths. The NDI had a moderate sensitivity for non–AIDS-related infectious diseases and non–AIDS-related cancers (70.6% and 75.0%, respectively) and high specificity for these causes (100.0% and 94.7%, respectively). The NDI had low sensitivity and high specificity for substance abuse (27.3% and 100.0%, respectively), heart disease (58.3% and 86.8%, respectively), hepatitis B/C (33.3% and 97.7%, respectively), and mental illness (50.0% and 97.8%, respectively). The measure of agreement between expert review and the NDI was lowest for HIV (κ = 0.20); moderate for heart disease (κ = 0.45) and hepatitis B/C (κ = 0.40); high for non–AIDS-related infectious diseases (κ = 0.76) and non–AIDS-related cancers (κ = 0.72); and low for all other causes of death (κ < 0.35).
Conclusions
Our findings support education and training of health care providers to improve the accuracy of cause-of-death information on death certificates.
Keywords: validation, HIV, mortality statistics, death certificate bias, sensitivity, specificity
People living with HIV who are taking effective HIV antiretroviral therapy are living longer and more productive lives than they did earlier in the epidemic. Nevertheless, the life expectancy of people with HIV, even in recent years, is shorter than that of people without HIV.1,2 To monitor the HIV epidemic and appropriate resources, documenting deaths in people with HIV is necessary to calculate both HIV prevalence and survival. In addition, ascertaining trends in the causes of death in people with HIV is important to identify and target interventions to prevent premature mortality and to assess the effect of improved screening and clinical management of diseases.3 Ascertaining trends is increasingly relevant because continued advances in treatment for HIV have resulted in improved survival and an increase in the proportion of deaths in people with HIV due to non–HIV-related causes.4-6
Information from death certificates regarding the cause of death may be inaccurate, as shown in studies that compared information on death certificates with information from autopsies,7 expert review and adjudication,8-10 hospital discharge records,11,12 and proxy informants such as next of kin.10 Inaccurate reporting of causes on death certificates may arise from several factors. These factors include lack of knowledge about how to properly complete the death certificate, incomplete medical information at the time of death because the physician may not have had access to the decedent’s complete medical history, a wish to protect patient or family confidentiality (eg, not wanting the record to show HIV)13 or to prevent a possible loss of life insurance (eg, not wanting the record to show suicide), or physician bias.14
Local, state, and national mortality data are obtained from standardized death certificates. In part I of the standard US death certificate (Figure), the person completing the death certificate should indicate the conditions or events that directly led to death, beginning with the immediate cause.15 The immediate cause of death is followed, sequentially, by the conditions or events that led to death, with the last entry indicating the underlying cause (ie, the condition that initiated the chain of events that resulted in death). In part II of the death certificate, any significant conditions that contributed to death but did not result in the underlying cause are documented.
Figure.
Cause and manner of death information (part I and part II) from a US death certificate. Source: Centers for Disease Control and Prevention.15
Information from death certificates is first reported to local vital statistics and then to state vital statistics, which then forwards the data to the National Death Index (NDI). Information on causes of death nationally is obtained from the NDI.16 For standardization of the underlying cause of death, the NDI applies a computer algorithm to systematically select the underlying cause of death from all causes and significant conditions listed on the death certificate. Information from the NDI includes the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes17 for both the underlying cause of death as well as all causes of death and conditions that contributed to death, all information obtained from the death certificate.
As part of routine HIV surveillance, states and local jurisdictions monitor deaths in people reported with HIV through computer matches with state vital statistics, social security death files, and the NDI. Mortality data are forwarded to the Centers for Disease Control and Prevention for national statistics. The causes of death are obtained from the NDI. Local and state vital statistics may also report people with HIV who had not previously been reported to local and state HIV registries.
The San Francisco Department of Public Health monitors the frequency of and trends in both underlying and all other causes of death (also called multiple causes of death). Examination of the underlying cause of death is valuable because it indicates the condition that caused the chain of events that resulted in death, it is the only cause of death available for the general population, and it is used in national statistics and for other important measures, such as standardized mortality ratios. Analysis of all causes of death and not just the underlying cause is important because it provides comprehensive information on the health status of the decedent and potential opportunities to reduce preventable causes of death.18 Earlier in the HIV epidemic, several studies demonstrated underreporting of HIV as a cause of death, but recent assessments are lacking.13,19 Given the underreporting and misclassification biases previously found for HIV and associated with death certificates in general, we compared causes of death using NDI data with causes determined by expert review among people reported with HIV in San Francisco.
Methods
We designed this exploratory study to inform the use of mortality data as a means for targeting HIV prevention efforts in San Francisco. We selected a random sample of 50 deaths that occurred in San Francisco among people reported to the San Francisco Department of Public Health as having HIV (25 deaths that occurred from July 1 through December 31, 2016, and 25 deaths that occurred from January 1 through May 31, 2017). During this period, 121 deaths occurred among people with HIV in San Francisco.
Our analysis included personally identifiable data because such data are required by law to be reported to the local health authority. The institutional review board of the University of California, San Francisco determined that because our study was an evaluation of the health department’s use of its HIV mortality data, institutional review board oversight was not required.
To determine causes of death by expert review, we adapted the Coding of Cause of Death in HIV (CoDe) project protocol,20,21 which uses standardized methods to assign the underlying cause of death for HIV cohorts and clinical trials. We modified the CoDe data collection form to include both recent (within the 12 months before death) and historic use of substances, specific types of substances, and both recent and any history of specific types of mental illness. We obtained information on demographic characteristics, HIV transmission risk, CD4 count, and viral load values from the HIV case registry. Trained HIV surveillance staff members abstracted data for the 3 years before death from all HIV care facilities in San Francisco, where the decedent was known to have received care and where surveillance staff members have access to medical records. This data abstraction period was based on routine surveillance, in which surveillance staff members review the medical records of people reported with HIV at the time of HIV diagnosis and every 18-24 months thereafter, with a final review at death. Surveillance staff members have found that relevant data from these reviews are typically available for 3 years before the initial medical record review and before all subsequent medical record reviews. Although medical record review was limited to the 3 years before death, we collected information on conditions that were noted on the problem list or anywhere in the medical record during the 3-year abstraction period, regardless of when they occurred. Five HIV physician specialists conducted all reviews. Two physician specialists reviewed each case report form independently and determined the immediate, contributing, and underlying causes of death.
Given the small sample size of 50 deaths, we limited our analysis to deaths from HIV, non–AIDS-related infectious diseases, non–AIDS-related cancers, substance abuse, heart disease, hepatitis B and hepatitis C (hepatitis B/C), and mental and behavioral disorders (including suicide). We selected these non–HIV-related causes because of their relative frequency and appropriateness for HIV prevention interventions. Discrepancies between reviewers for these selected causes of death, regardless of whether they were listed as the underlying or any other cause or condition, underwent a third review by 1 of the 3 expert reviewers who had not previously reviewed the record. In contrast to CoDe methods, we identified final causes of death as those on which at least 2 reviewers agreed.
Analysis
We compared the characteristics of sampled cases and unsampled cases by using the χ2 test for differences in proportions and considered P values <.05 to be significant. To assess agreement between causes of death determined by expert review and causes listed in the NDI, we first identified the ICD-10-CM codes that corresponded to the CoDe codes. Next, we compared the CoDe ICD-10-CM codes with the NDI ICD-10-CM codes. To ensure that disagreements between the expert reviews and NDI were not the result of coding errors made during expert review, we reviewed any discrepant records by using the text description of the ICD-10-CM code and recoded causes if we determined that the conditions were the same. For example, the NDI codes pneumonia as a respiratory illness, whereas the expert reviewers coded pneumonia as an infectious disease; in our analysis, we considered these causes to be the same.
To assess the agreement, sensitivity, and specificity22 of the causes of death as determined by the NDI with those determined by the expert reviewers, we compared all causes of death from the death certificate and the expert reviewers. We did not limit the analysis to only the underlying cause because the expert reviewers had not been trained in nosology or NDI methods for selecting the underlying cause. We assessed agreement on the causes of death between the expert review and the NDI with the κ statistic.22 We calculated the sensitivity and specificity of the NDI, using the causes of death by the expert reviews as the gold standard. We analyzed the most frequent causes of death using all causes. We conducted all analyses using SAS version 9.4 (SAS Institute Inc).
Results
The characteristics of the 50 sampled decedents were similar to those of the 71 unsampled decedents (Table 1). Most were aged 50-69 at the time of death (60.0% sampled; 66.2% not sampled), non-Hispanic White (66.0% sampled; 57.8% not sampled), male (90.0% sampled; 97.2% not sampled), and men who had acquired HIV through sex with other men (92.0% sampled; 95.9% not sampled).
Table 1.
Demographic and risk characteristics of sampled and unsampled decedents (N = 121) reported with HIV infection, in a study of the sensitivity and specificity of the National Death Index for multiple causes of death in people with HIV, San Francisco, 2016-2017
| Characteristic | Sampled, no. (%) (n = 50) | Not sampled, no. (%) (n = 71) | P valuea |
|---|---|---|---|
| Age at death, y | .97 | ||
| <40 | 3 (6.0) | 4 (5.6) | |
| 40-49 | 8 (16.0) | 9 (12.7) | |
| 50-59 | 16 (32.0) | 25 (35.2) | |
| 60-69 | 14 (28.0) | 22 (31.0) | |
| ≥70 | 9 (18.0) | 11 (15.5) | |
| Sex at birth | .10 | ||
| Male | 45 (90.0) | 69 (97.2) | |
| Female | 5 (10.0) | 2 (2.8) | |
| Race/ethnicity | .13 | ||
| Non-Hispanic White | 33 (66.0) | 41 (57.8) | |
| Non-Hispanic Black/African American | 8 (16.0) | 15 (21.1) | |
| Latino/Latina | 4 (8.0) | 13 (18.3) | |
| Otherb | 5 (10.0) | 2 (2.8) | |
| Transmission risk | .51 | ||
| MSM | 25 (50.0) | 42 (59.2) | |
| Non-MSM who injected drugs | 8 (16.0) | 7 (9.9) | |
| MSM who injected drugs | 13 (26.0) | 19 (26.8) | |
| Heterosexual men and women | 3 (6.0) | 1 (1.4) | |
| No reported risk | 1 (2.0) | 2 (2.8) |
Abbreviation: MSM, men who have sex with men.
aDetermined by χ2 test; P < .05 considered significant.
bIncludes respondents who identified as multiple races; includes Hispanic and non-Hispanic.
The most frequent multiple causes of death as determined by expert review were HIV (52.0%), substance abuse (44.0%), non–AIDS-related infectious diseases (34.0%), non–AIDS-related cancers (24.0%), and heart disease (24.0%) (Table 2). The NDI also reported HIV as the most frequent multiple cause of death (44.0%). Similar to the expert reviewer’s determination of causes of death, the NDI frequently reported non–AIDS-related infectious diseases (24.0%), heart disease (24.0%), and non–AIDS-related cancers (22.0%).
Table 2.
Multiple causes of deatha as determined by expert review and the National Death Index among decedents (n = 50) reported with HIV infection, San Francisco, 2016-2017
| Cause of death | Expert review, no. (%) | National Death Index, no. (%) |
|---|---|---|
| HIV | 26 (52.0) | 22 (44.0) |
| Substance abuse | 22 (44.0) | 6 (12.0) |
| Non–AIDS-related infectious diseases | 17 (34.0) | 12 (24.0) |
| Non–AIDS-related cancers | 12 (24.0) | 11 (22.0) |
| Heart disease | 12 (24.0) | 12 (24.0) |
| Renal/urinary tract disease | 8 (16.0) | 8 (16.0) |
| Hepatitis B/C | 6 (12.0) | 3 (6.0) |
| Mental illness including suicide | 6 (12.0) | 3 (6.0) |
| Chronic obstructive pulmonary disease | 4 (8.0) | 3 (6.0) |
| Accident/injury | 3 (6.0) | 5 (10.0) |
| Diabetes | 3 (6.0) | 1 (2.0) |
| Central nervous system disorder | 3 (6.0) | 2 (4.0) |
| Respiratory disease | 3 (6.0) | 5 (10.0) |
aCauses include the immediate, contributing, underlying, and other significant conditions present at the time of death.
Comparing multiple causes of death, we found that the NDI had low sensitivity and specificity for HIV deaths (53.9% and 66.7%, respectively; Table 3). Sensitivities of the NDI for non–AIDS-related infectious diseases and non–AIDS-related cancers were moderate (70.6% and 75.0%, respectively), but the specificities for both causes were high (>95.0%). The sensitivity of the NDI for substance abuse was low (27.3%), but the specificity was 100.0%. The sensitivity of the NDI for other conditions was low, whereas the specificities were high. The measure of agreement between expert review and the NDI was good for non–AIDS-related infectious diseases (κ = 0.76) and non–AIDS-related cancers (κ = 0.72), moderate for heart disease (κ = 0.45) and mental illness (κ = 0.54), fair for substance abuse (κ = 0.30) and hepatitis B/C (κ = 0.40), and poor for HIV.
Table 3.
Sensitivity and specificity of the National Death Index for causes of deatha using expert physician review as the gold standard among decedents (n = 50) reported with HIV infection, San Francisco, 2016-2017
| Cause of death | Sensitivity, % | Specificity, % | κ statistic |
|---|---|---|---|
| HIV/AIDS | 53.9 | 66.7 | 0.20 |
| Non–AIDS-related infectious diseases | 70.6 | 100.0 | 0.76 |
| Non–AIDS-related cancers | 75.0 | 94.7 | 0.72 |
| Substance abuse | 27.3 | 100.0 | 0.30 |
| Heart disease | 58.3 | 86.8 | 0.45 |
| Hepatitis B/C | 33.3 | 97.7 | 0.40 |
| Mental illness | 50.0 | 97.8 | 0.54 |
aCauses include the immediate, contributing, underlying, and other significant conditions present at the time of death.
Discussion
Our findings suggest substantial misclassification of causes of death among people with HIV. Differences in populations, settings, and methods make direct comparison with other studies difficult, but our findings are consistent with other evaluations of death certificate inaccuracies.13,19,23 Both the sensitivity and specificity of the NDI for HIV were low, indicating that compared with expert reviews, the NDI incorrectly included HIV as a cause of death (type I error) and incorrectly missed HIV as 1 of multiple causes of death (type II error).
The incorrect inclusion of HIV as a cause of death most likely reflects the inclusion of HIV in part II of the death certificate (ie, the section where other significant conditions that contributed to death but did not contribute to the underlying cause of death are listed). For example, in 2 of the deaths due to injuries (1 from a traffic accident and the other from a burn), HIV was included as a cause of death in the NDI but not in the expert review. The contribution of HIV to these deaths is difficult to understand. It may be that including HIV in these situations reflects a misunderstanding by the person completing the death certificate about what constitutes a significant contributing condition.
The incorrect exclusion of HIV as a cause of death in the NDI may be due to incomplete clinical information, lack of HIV expertise, or bias on the part of the person completing the death certificate. We found 2 deaths from metastatic anal cancer. The expert reviewers, but not the NDI, included HIV as a contributing cause of death, which is consistent with the literature that shows immune suppression as a risk factor for anal cancer.24 In 1 case, the medical record clearly listed non-Hodgkin lymphoma, an AIDS-defining cancer, but the cause of death listed in the NDI was a secondary malignancy of the retroperitoneum and peritoneum, suggesting either a lack of information on AIDS-defining cancers or a deliberate omission on the part of the person who completed the death certificate.
When the NDI failed to list HIV as a cause of death, the other causes of death listed were similar to those listed by the expert reviewers. These other causes included several non–AIDS-related cancers, sepsis, renal disease, and cardiomyopathy. The absence of HIV as a cause of death in these situations may reflect differing opinions on the role that HIV plays in these conditions or the absence of information on the decedent’s HIV status by the person who completed the death certificate.
The NDI had low sensitivity for substance abuse–related deaths. The expert reviewers, physicians who care for people with HIV, were keenly aware of the role that substance abuse plays in this population. As such, they may have been more inclined than the person who completed the death certificate, who might not have been an HIV expert, to include substance abuse as a contributing cause of death. In addition, the data collection form used by the expert reviewers included substance use data abstracted from the 3 years before death, which may have given them more information on substance use than was readily available to the person completing the death certificate. Although substance abuse is not always a contributing cause of death, a substantial proportion of the sampled decedents had a history of substance abuse, including noninjection drug use (58%), injection drug use (36%), and excessive alcohol use (42%).25 In contrast to the NDI’s low sensitivity for substance abuse–related deaths, the NDI had a high specificity for these deaths. This high specificity was most likely due to the proportion of these deaths that were from drug overdose as determined by toxicology screening.
The sensitivity of death certificates for mental illness was low, whereas the specificity was high. Our definition of mental illness included suicide and other mental disorders. Often an underlying mental illness such as depression leads to suicide; the expert reviewers had detailed information on mental health status, which may not have been available to people who completed the death certificates. Classification of a suicide death requires evidence of intent, which would lead to high specificity. In a supplemental analysis, we calculated the sensitivity and specificity of the NDI for suicide alone and found that both were 100%. We also found that expert reviewers indicated depression as the underlying cause of death that led to a drug overdose, whereas the NDI did not include information on mental illness as a cause of death in this situation. Although the number of deaths attributed to mental illness was small, the inclusion of a 3-year review period increased the likelihood of finding evidence of a mental disorder. In addition, expert reviewers knew that people living with HIV have high rates of mental disorders, including depression, that can contribute to substance abuse and drug overdose.26,27
Limitations
Our study had several limitations. First, the study was small and was designed to improve the use of NDI data as part of routine HIV surveillance in San Francisco. As such, the findings may not have high precision or be representative of populations in other jurisdictions. Second, we abstracted information only from facilities where surveillance staff members had direct access to medical records. Although this approach could have resulted in selection bias, we think that this bias was unlikely because all 50 of the sampled decedents and 70 of the 71 unsampled decedents received care at one of the facilities. Third, we may have found such a large degree of disagreement between the causes of death identified by the expert reviewers and the causes of death identified in the NDI because of the large number of physicians (n = 36) who completed the death certificates; the greater number of physicians who complete death certificates, the greater the variability we might expect. We have confidence in the causes of death identified by the expert reviewers because of their expertise and our requirement that a third independent review be conducted when the 2 initial reviewers disagreed on the causes of death (which happened in 23 of the 50 sampled decedents).
Finally, we did not measure the sensitivity and specificity of the underlying cause of death as a primary outcome because the expert reviewers were not trained nosologists. However, in a supplemental analysis, we compared the sensitivity and specificity of the NDI for the underlying cause of death with the multiple causes documented by the expert reviewers. In this comparison, the specificity of the NDI was higher (range, 97.4%-100.0%) and the sensitivity was lower for HIV (42.3%), non–AIDS-related infectious diseases (5.9%), non–AIDS-related cancers (50.0%), heart disease (25.0%), and mental illness (33.3%). Specificity and sensitivity did not change for substance abuse or hepatitis B/C.
Conclusions
Despite these limitations, our study provides valuable information for use in HIV surveillance and care for people with HIV. Our findings suggest that the number of HIV-related deaths reported in the NDI may be inaccurate, and this inaccuracy may adversely affect the targeting of resources and clinical practices to prevent premature mortality. There is a critical need to improve the quality of data from death certificates. Education and training of physicians through various modalities, such as interactive workshops, videos, seminars, and written material, have been shown to improve death certification, with interactive interventions producing the greatest benefit.28,29 Training of medical students, residents, and practicing physicians on the proper completion of part I and part II of the death certificate may improve the accuracy of the NDI for both the underlying and multiple causes of death. We also recommend that when the person completing the death certificate does not have access to the full medical record or is not thoroughly familiar with the decedent’s medical history, including medical examiner cases, the death certificate be reviewed by the primary care physician and amended as needed. The additional time required to thoroughly investigate deaths and properly document causes of death does not conflict with rapid reporting of deaths, because deaths may be reported while the causes of death are under investigation. Proper completion of a death certificate is time-consuming; as such, reimbursement from health insurance should also be considered as a way to improve the quality of cause-of-death information.
Our results indicate a need to improve the accuracy of death certificate information on causes of death in people with HIV. Accuracy is important, because such data are used nationwide to evaluate public health efforts to reduce HIV-related mortality. We also recommend larger and more extensive studies to measure and classify errors in death certification.
Acknowledgments
The authors thank Maree Kay Parisi for project management, Sharon Pipkin for data management, Diane Jones for input on the data collection form, and the surveillance staff of the San Francisco Department of Public Health and Signy Toquinto for data abstraction.
Footnotes
Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded through the Centers for Disease Control and Prevention cooperative agreement number NU62PS924536-01-00 to the San Francisco Department of Public Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the article.
ORCID iDs
Sandra Schwarcz, MD, MPH https://orcid.org/0000-0001-7611-4387
Nancy A. Hessol, MSPH https://orcid.org/0000-0003-3374-8384
Ling Chin Hsu, MPH https://orcid.org/0000-0002-4894-3492
References
- 1.Antiretroviral Therapy Cohort Collaboration . Life expectancy of individuals on combination antiretroviral therapy in high-income countries: a collaborative analysis of 14 cohort studies. Lancet. 2008;372(9635):293-299. 10.1016/S0140-6736(08)61113-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Marcus JL., Chao CR., Leyden WAet al. Narrowing the gap in life expectancy between HIV-infected and HIV-uninfected individuals with access to care. J Acquir Immune Defic Syndr. 2016;73(1):39-46. 10.1097/QAI.0000000000001014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.National Vital Statistics System . Results and publications for mortality data. Accessed December 16, 2018. https://www.cdc.gov/nchs/nvss/mortality_products.htm
- 4.Farahani M., Mulinder H., Farahani A., Marlink R. Prevalence and distribution of non-AIDS causes of death among HIV-infected individuals receiving antiretroviral therapy: a systematic review and meta-analysis. Int J STD AIDS. 2017;28(7):636-650. 10.1177/0956462416632428 [DOI] [PubMed] [Google Scholar]
- 5.Goehringer F., Bonnet F., Salmon Det al. Causes of death in HIV-infected individuals with immunovirologic success in a national prospective survey. AIDS Res Hum Retroviruses. 2017;33(2):187-193. 10.1089/aid.2016.0222 [DOI] [PubMed] [Google Scholar]
- 6.Cima M., Parker RD., Ahmed Yet al. Cause of death in HIV-infected patients in South Carolina (2005-2013). Int J STD AIDS. 2016;27(1):25-32. 10.1177/0956462415571970 [DOI] [PubMed] [Google Scholar]
- 7.Smith Sehdev AE., Hutchins GM. Problems with proper completion and accuracy of the cause-of-death statement. Arch Intern Med. 2001;161(2):277-284. 10.1001/archinte.161.2.277 [DOI] [PubMed] [Google Scholar]
- 8.Pritt BS., Hardin NJ., Richmond JA., Shapiro SL. Death certification errors at an academic institution. Arch Pathol Lab Med. 2005;129(11):1476-1479. 10.1043/1543-2165(2005)129[1476:DCEAAA]2.0.CO;2 [DOI] [PubMed] [Google Scholar]
- 9.McGivern L., Shulman L., Carney JK., Shapiro S., Bundock E. Death certification errors and the effect on mortality statistics. Public Health Rep. 2017;132(6):669-675. 10.1177/0033354917736514 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Halanych JH., Shuaib F., Parmar Get al. Agreement on cause of death between proxies, death certificates, and clinician adjudicators in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study. Am J Epidemiol. 2011;173(11):1319-1326. 10.1093/aje/kwr033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Johansson LA., Westerling R. Comparing hospital discharge records with death certificates: can the differences be explained? J Epidemiol Community Health. 2002;56(4):301-308. 10.1136/jech.56.4.301 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Govindan S., Shapiro L., Langa KM., Iwashyna TJ. Death certificates underestimate infections as proximal causes of death in the U.S. PLoS One. 2014;9(5):e97714. 10.1371/journal.pone.0097714 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Hessol NA., Buchbinder SP., Colbert Det al. Impact of HIV infection on mortality and accuracy of AIDS reporting on death certificates. Am J Public Health. 1992;82(4):561-564. 10.2105/AJPH.82.4.561 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Sterling TD., Rosenbaum WL., Weinkam JJ. Bias in the attribution of lung cancer as cause of death and its possible consequences for calculating smoking-related risks. Epidemiology. 1992;3(1):11-16. 10.1097/00001648-199201000-00004 [DOI] [PubMed] [Google Scholar]
- 15.Centers for Disease Control and Prevention . US Standard Certificate of Death. Accessed December 4, 2018. https://www.cdc.gov/nchs/data/dvs/DEATH11-03final-acc.pdf
- 16.National Center for Health Statistics . National Death Index user’s guide; appendix B. 2013. Accessed December 16, 2018. https://www.cdc.gov/nchs/data/ndi/NDI_Users_Guide.pdf
- 17.Centers for Disease Control and Prevention . International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). Accessed December 4, 2018. ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/ICD10CM/2017
- 18.Crews DE., Stamler J., Dyer A. Conditions other than underlying cause of death listed on death certificates provide additional useful information for epidemiologic research. Epidemiology. 1991;2(4):271-275. 10.1097/00001648-199107000-00006 [DOI] [PubMed] [Google Scholar]
- 19.Barchielli A., Buiatti E., Galanti C., Giovannetti L., Acciai S., Lazzeri V. Completeness of AIDS reporting and quality of AIDS death certification in Tuscany (Italy): a linkage study between surveillance system of cases and death certificates. Eur J Epidemiol. 1995;11(5):513-517. 10.1007/BF01719302 [DOI] [PubMed] [Google Scholar]
- 20.Copenhagen HIV Programme . Protocol: coding of causes of death in HIV: CoDe. Version 2.3. Updated August 2013. Accessed August 1, 2017. https://www.chip.dk/Portals/0/files/Code%20Protocol%202.3.pdf
- 21.Kowalska JD., Friis-Møller N., Kirk Oet al. The Coding Causes of Death in HIV (CoDe) project: initial results and evaluation of methodology. Epidemiology. 2011;22(4):516-523. 10.1097/EDE.0b013e31821b5332 [DOI] [PubMed] [Google Scholar]
- 22.Newman T., Kohn M. Evidence-Based Diagnosis: An Introduction to Clinical Epidemiology. Cambridge University Press; 2020. [Google Scholar]
- 23.Dowdy DW., Geng EH., Christopoulos KAet al. Mortality among antiretroviral-eligible patients in an urban public clinic. J Acquir Immune Defic Syndr. 2011;57(4):297-300. 10.1097/QAI.0b013e31822233aa [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Grulich AE., van Leeuwen MT., Falster MO., Vajdic CM. Incidence of cancers in people with HIV/AIDS compared with immunosuppressed transplant recipients: a meta-analysis. Lancet. 2007;370(9581):59-67. 10.1016/S0140-6736(07)61050-2 [DOI] [PubMed] [Google Scholar]
- 25.San Francisco Department of Public Health . HIV epidemiology annual report 2017. Published 2018. Accessed November 10, 2018. https://www.sfdph.org/dph/comupg/oprograms/HIVepiSec/HIVepiSecReports.asp
- 26.Bing EG., Burnam MA., Longshore Det al. Psychiatric disorders and drug use among human immunodeficiency virus–infected adults in the United States. Arch Gen Psychiatry. 2001;58(8):721-728. 10.1001/archpsyc.58.8.721 [DOI] [PubMed] [Google Scholar]
- 27.Foley M., Schwab-Reese LM. Associations of state-level rates of depression and fatal opioid overdose in the United States, 2011-2015. Soc Psychiatry Psychiatr Epidemiol. 2019;54(1):131-134. 10.1007/s00127-018-1594-y [DOI] [PubMed] [Google Scholar]
- 28.Villar J., Pérez-Méndez L. Evaluating an educational intervention to improve the accuracy of death certification among trainees from various specialties. BMC Health Serv Res. 2007;7:183. 10.1186/1472-6963-7-183 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Aung E., Rao C., Walker S. Teaching cause-of-death certification: lessons from international experience. Postgrad Med J. 2010;86(1013):143-152. 10.1136/pgmj.2009.089821 [DOI] [PubMed] [Google Scholar]