Comparison of Diagnoses of Amyotrophic Lateral Sclerosis by Use of Death Certificates and Hospital Discharge Data in the Danish Population

Abstract

Background

Because ALS is rare, large-scale studies are difficult. Hospital and death certificate data are valuable tools, but understanding how well they capture cases is needed.

Methods

We identified 3650 incident cases in the Danish National Patient Register (NPR) between 1982–2009, using ICD-8 (before 1994) or ICD-10 codes. Death certificates were obtained from the Danish Register of Causes of Death. We obtained medical records for 173 of the cases identified in the NPR and classified these according to El Escorial criteria. We compared ALS identification from death certificates to hospital discharges, and both to medical records.

Results

The sensitivity for use of death certificates was 84.2% (95%CI:82.9–85.5%), and was significantly higher for women, subjects younger than 77 years and when coded with ICD-8. Using only the underlying cause of death resulted in significantly lower sensitivity. The estimated overall positive predictive value (PPV) was 82.0% (95%CI:80.0–83.8%). Sensitivity and PPV were similar when comparison was with medical records.

Conclusion

We found that use of hospital discharges and death certificates is highly reliable and, therefore, a valuable tool for ALS epidemiologic studies. The possible effects on findings of slight differences by age, gender, and ICD coding should be considered.

Introduction

Amyotrophic lateral sclerosis (ALS) is a relatively rare disease characterized by progressive lower and upper motor neuron degeneration [1], with a reported incidence rate between 1.5 and 2.5 per 100,000 persons per year [2]. ALS is usually rapidly fatal with a median survival of approximately 24–36 months [3, 4]. The etiology of ALS is poorly understood and large epidemiologic studies can play an important role in studying the pattern of, and risk factors for, ALS [5]. Large scale cohort studies can provide some of the strongest epidemiologic data, but the relative rarity of ALS makes such studies difficult. Using hospital discharge or death certificate data from large administrative datasets can facilitate such studies. Such approaches, however, rely on an understanding of how well such administrative data capture true ALS cases. Several studies have explored these issues, but none have examined these questions in the context of complete population registry systems. Further, none have compared mortality to hospital data, an analysis that has specifically been called for [6]. Such national registry systems are more commonly being used to examine ALS epidemiology and provide a tremendous opportunity for such studies [7–10]. Denmark is a country that for almost half a century has accumulated data on all citizens, including a unique personal identifier that makes it possible to link information from several nationwide registries, creating a powerful and dynamic database and thus allowing for a very large cohort study that would not be possible otherwise [11]. We used this registry system to examine the relation between hospital discharge data, death registry data, and medical records.

Methods

Data Collection

Data were obtained through 2009 from the Danish Registers system.

Registry ALS Case Ascertainment

Incident ALS cases were identified via the Danish National Patient Register (NPR). The NPR was established in 1977, since when it has included nationwide administrative and clinical records of all somatic inpatient hospital visits. Since 1995 it has also included outpatient visits [12]. ALS deaths were identified via the Danish Register of Causes of Death (RCD) that includes completely computerized individual data of all deaths among Danish residents dying in Denmark since 1970. Classification of causes of death is done in accordance with the World Health Organization (WHO). All records cover the underlying and up to four contributory causes of death [13].

ALS was defined as an ICD-8 code of 348.0 (before 1994) or ICD-10 code of G12.2 (1994 and later). For incident ALS, the date of the first relevant code was assigned as the diagnosis date and we restricted our analyses to cases identified through the NPR between the years 1982–2009, to avoid inclusion of prevalent cases. We also restricted our analysis to subjects that were 20 years old or older when they were diagnosed with ALS.

Medical Records

We randomly selected 401 ALS cases identified through the NPR within strata of ICD code (8 vs. 10), age at diagnosis (<55, 55–74, 75+), and sex for medical record retrieval. Of the 12 main hospitals treating ALS in Denmark, from which we requested medical records, 9 provided us with the requested information. Of the total 401 requested medical records, we received 197 (49.1%), 190 of which were complete (96.4%). The proportion of received to requested records by hospital, among the 9 hospitals that sent us data, ranged between 8–100%, with 6 hospitals having sent ≥70% (173 records). Some Danish hospitals only retain medical records for a limited time following the death of a patient, typically five years, and this was the primary reason we did not receive all requested medical records.

The records were abstracted and reviewed by a physiotherapeutic expert in movement disorder (LH), under the supervision of the study neurologist (OG). Detailed information was extracted from the medical charts and each case was assigned to one of the following diagnostic categories, according to the revised El Escorial Criteria: clinically definite ALS, clinically probable ALS, clinically probable – laboratory supported ALS and clinically possible ALS. We also separated clinically suspected ALS from no ALS as two separate categories.

Control Selection

We used controls to determine the specificity of comparing death certificates to hospital discharge data. For each case identified through NPR, we randomly selected 100 controls, individually matched on sex and birth year. Controls were free of ALS at the date of the case’s first ALS discharge code. If a control became a case, they were only included as cases in our analysis.

Controls were selected through the Danish Civil Registration System (CRS), which includes administrative records on all persons alive and living in Denmark since 1968 [14]. CRS also includes historical records on all individuals that disappeared, immigrated or emigrated.

Socioeconomic Status (SES)

In our analysis, SES was coded as the highest level in occupational history, obtained either by the subject or their spouse. Occupational history was assessed according to job titles, which were categorized into 6 groups according to the Danish Institute of Social Sciences [15]. The highest status is group 1 (corporate managers and subjects in academia), followed by group 2 (proprietors, managers of small businesses and teachers), group 3 (technicians and nurses), group 4 (skilled workers), and group 5 (unskilled workers). A separate group was included for subjects with unknown status.

Data Analysis

First, we determined the number and percentage of ALS deaths that did not appear as ALS cases in the NPR. All subjects were identified by name and their unique Central Person Registry number, including information on birthday and sex.

We estimated the sensitivity and positive predictive value (PPV) of death certificates compared to hospital discharge diagnoses. Sensitivity in this analysis, therefore, is the proportion of hospital discharge-identified cases that had an ALS-related death certificate. The PPV corresponds to the proportion of ALS-related death certificates that had been identified as cases through hospital records. Moreover, we estimated the specificity of death certificates using the controls, i.e. the proportion of the controls that were never hospitalized for ALS that also did not have an ALS-related death certificate. Since we used the selected controls for the estimation of the specificity instead of the entire Danish population meeting our inclusion criteria, and given that ALS is a rare disease, the reported specificity would likely be slightly underestimated.

For this analysis, only subjects that had died and had a non-missing cause of death (underlying or contributory) on their death certificate were included. We repeated our analyses for inpatient vs outpatient cases, by sex, SES, age category and ICD coding (8 vs 10). Age categories were assigned according to the quintiles of the age distribution at death of the cases. Significant differences across groups were assessed by Chi-squared tests for equality of proportions. We repeated analyses both including any ALS diagnosis on the death certificates as the cause of death (underlying or contributory) and only including the underlying cause of death. We present the results from the analysis including both underlying and contributory ALS-related causes of death, unless otherwise specified.

All 95% confidence intervals were calculated using binomial exact probabilities. All statistical analyses were conducted using the R Statistical Software, version 3.0.3 (Foundation for Statistical Computing, Vienna, Austria).

Sensitivity Analysis

As a sensitivity analysis, to assess the robustness of our results, we repeated analyses restricting to cases with at least two inpatient hospitalizations, thus reducing potential hospital discharge misclassification.

Medical Records

We also estimated the sensitivity and PPV of the death certificates compared to medical records. As our main analysis, we included only records obtained from hospitals that provided us with at least 70% of the requested records. We repeated analyses by sex, age at diagnosis (<55, 55–74, 75+) and ICD-8 or ICD-10 codes. As a sensitivity analysis we repeated our analyses including all available complete records.

Furthermore, we also compared the hospital discharge data to the detailed medical records obtained during hospitalization and on which the hospital register is based. Given that medical records were selected from the hospital-identified cases, the sensitivity could not be estimated. We, therefore, only estimated the PPV.

Results

Between 1982 and 2009, 3650 incident ALS cases were identified in the hospital discharge data meeting our inclusion criteria. Among these, the mean age at diagnosis was 65.4 years (SD: 11.6), and 1696 (45.6%) were female. We identified 420 ALS deaths through death certificates that did not have an ALS diagnosis in the hospital discharge data. The average age at death of these 420 subjects was 73.3 years (SD: 10.9) and 198 (47.1%) were female.

At the time of our analysis, among the 3650 identified incident cases, 449 (12.3%) were still alive, 5 had emigrated and 1 had disappeared. We obtained the death certificates of the 3195 remaining cases that had died, 197 of which were missing the cause of death on their death certificates. Of these, 179 subjects died in 2010 and 2011; the RCD has a delay of up to two years due to coding, and hence the causes of death were not known at the time of our analyses. The RCD might also not have the cause of death of subjects that died abroad, which could explain the missing causes of death for the remaining 18 subjects. A total of 2998 hospital-identified cases, therefore, were included in further analyses. Table 1 presents the underlying causes of death for those hospital-identified cases that had no mention of ALS on the death certificates.

Table 1.

Underlying causes of death for cases with no underlying or contributory ALS-related death certificates.

	N	%
Diabetes	6	0.9
Dementia/Alzheimer	8	1.19
Infectious Disease	9	1.34
Accidental Fall	13	1.94
Pneumonia/Influenza	19	2.84
COPD	26	3.88
Cerebrovascular	34	5.07
Ischemic Heart Disease	53	7.91
CVD, other	19	2.84
Cancer	70	10.45
Other	216	32.24
Missing Diagnoses	197	29.4

When comparing death certificates against hospital discharge diagnoses, and looking at both underlying and contributory causes of death, we estimated an overall sensitivity for death certificates of 84.2% (95%CI: 82.9–85.5%, Table 2). The sensitivity was significantly lower when we only included the underlying cause of death (80.1%; 95%CI: 78.6–81.5%). The overall PPV was 86.5% (95%CI: 85.2–87.7%) for underlying or contributory ALS cause of death and 88.3% (95%CI: 87.0–89.5%) for underlying ALS cause of death only. The overall estimated specificity was 99.89% (95%CI: 99.88–99.90%).

Table 2.

Comparison of hospital discharges and death certificates, among hospital-identified cases that had died at the time of analysis.

Death	Certificates		Hospital Discharges
			ALS	No ALS
		ALS	2,525	394
		No ALS	473	364,710

Death Certificate Sensitivity = 84.2%, PPV = 86.5%, Specificity=99.9%

The sensitivity for female cases (86.7%; 95%CI: 84.8–88.4%) was significantly higher than that of male cases (82.0%; 95%CI: 80.0–83.8%). Among the 2806 who had been admitted as inpatient cases at least once, the sensitivity was 87.3% (95%CI: 86.0–88.5%). The sensitivity for the 192 cases that were never inpatient (only outpatient) was significantly lower (39.1%; 95%CI: 32.1–46.3%).

The estimated sensitivity for those whose first ALS hospitalizations was before 1994 (ICD-8) was 88.2% (95%CI: 86.1–90.1%) and it was significantly higher than the sensitivity for later years (ICD-10, 82.2%; 95%CI: 80.4–83.8%, Figure 1). The same pattern was observed for both sexes, with higher sensitivity for the female than the male cases.

Figure 1.

Sensitivities before and after 1994, comparing hospital discharges to death certificates; overall and by gender. In 1994 Denmark adopted the ICD-10.

We observed no significant differences by SES (p-value = 0.14). Further, no significant differences in sensitivity by quintile of age were apparent up to the third quintile, but the sensitivity was lower for the fourth quintile and significantly lower for the fifth quintile (older than 77.6 years; Figure 2). The same pattern was also observed by sex, with higher sensitivity for the female vs. male cases. The difference in the sensitivities across sexes increased with increasing age.

Figure 2.

Sensitivities by quintile of age at death, comparing hospital discharges to death certificates; overall and by gender.

Sensitivity Analysis

To assess potential misclassification in the hospital diagnoses, we restricted our cases to those with at least two inpatient hospitalizations. Of these 2446 cases (67% of the total number of cases included in the main analyses), 161 (6.58%) were still alive at the time of analyses, 1 had emigrated and 2284 had died, 121 of which had missing cause of death on their death certificates. Therefore, 2163 cases were included in further analyses.

In this subset, the overall sensitivity was significantly higher than in the main analysis (90.8%; 95% CI: 89.5–92.0%), while the PPV was 83.3% (95% CI: 81.7–84.8%). As in the main analysis, the sensitivity for the male cases (89.4%; 95% CI: 87.4–91.1%) was lower than for females (92.2%; 95% CI: 90.5–93.8%), with both of these sensitivities significantly higher than the corresponding ones in the main analysis.

Medical Records

Of the 173 hospital-discharge identified cases whose medical records we examined, 134 satisfied the El Escorial Criteria, 26 had clinically suspect ALS and 13 had no ALS (Table 3). Of the 134 that satisfied the El Escorial Criteria, 78.4% (95%CI: 70.4–85.0%) also had an ALS death certificate (sensitivity), while 83.3% (75.7–89.4%) of the subjects with ALS in the death certificate also satisfied the El Escorial Criteria (PPV) (Table 4). Both sensitivity and PPV were higher among female (88.7%; 95%CI: 78.1–95.3% for both) than male subjects (sensitivity = 69.4%; 95%CI: 57.5–79.8% and PPV = 78.1%; 95%CI: 66.0–87.5%).

Table 3.

Percent of identified cases by El Escorial Criteria categories.

El Escorial Criteria	All available records			Records from hospitals that provided ≥ 70%
	N	(%)	% Female	N	(%)	% Female
Clinically Definite	131	68.9	46.6	121	69.9	47.1
Clinically Probable	5	2.6	40.0	4	2.3	50.0
Clinically Probable - Lab Supported	7	3.7	42.9	7	4.0	42.9
Clinically Possible	2	1.1	0.0	2	1.2	0.0
Clinically Suspected	26	13.7	42.3	26	15.0	42.3
No ALS	19	10.0	42.1	13	7.5	53.8

Table 4.

Comparison of medical records and death certificates, among hospital-identified cases that had died at the time of analysis, for which we obtained medical records, a) excluding and b) including clinically suspect ALS subjects as true ALS cases.

a)
Death	Certificates		Hospital Discharges
			ALS	No ALS
		ALS	105	21
		No ALS	29	18

b)
Death	Certificates		Hospital Discharges
			ALS	No ALS
		ALS	125	1
		No ALS	35	12

Sensitivity = 78.4%, PPV = 83.3%,

Specificity = 46.2%

Sensitivity = 78.1%, PPV = 99.2%,

Specificity = 92.3%

Death certificate sensitivities and PPVs varied by age, with both the highest sensitivity and PPV observed among subjects 55–74 years old (sensitivity = 81.4%; 95%CI: 71.6–89.0%, PPV = 85.4%; 95%CI: 75.8–92.2%). The lowest sensitivity was among subjects younger than 55 years (71.4%; 95%CI: 47.8–88.7%), while the lowest PPV among subjects older than 75 years (72.0%; 95%CI: 50.6–87.9%).

Of the medical records obtained, 33 diagnoses occurred before 1994 (ICD-8) and 140 after that (ICD-10). We observed higher sensitivity during the first period (before 1994: 96.2%; 95%CI: 80.4–99.9% vs. later: 74.1%; 95%CI: 64.8–82.0%). The opposite pattern was found for the estimated PPVs, with higher values during the later period (before 1994: 78.1%; 95%CI: 60.0–90.7% vs. later: 85.1%; 95%CI: 76.3–91.6%).

When we included the 26 cases with clinically suspected ALS, the estimated overall death certificate sensitivity remained similar (78.1%; 95%CI: 70.9–84.3%), while the overall PPV was significantly higher (99.2%; 95%CI: 95.7–100.0%).

Our results did not materially change when we included records from all hospitals, regardless of the proportion of requested records received. For instance, the overall sensitivity when all records were considered was 76.6% (95%CI: 68.8–83.2%) and the overall PPV was 84.1% (95%CI: 76.7–89.9%).

When we compared the hospital discharge data to the medical records, the estimated PPV for hospital discharge was 77.5% (95%CI: 70.5–83.5%) when only those fully meeting El Escorial criteria on medical record review were considered confirmed cases. No differences were observed across sexes or ICD codes. The PPVs, however, varied by age (77.8% (95%CI: 57.7–91.4%) among subjects <55 years, 81.9% (95%CI: 73.2–88.7%) among subjects between 55–74 years and 67.6% (95%CI: 50.2–82.0%) between those older than 75 years). When we included as confirmed cases subjects with clinically suspect ALS on medical record review, the estimated PPV for hospital discharge data was 92.5% (95%CI: 87.5–95.9%).

Discussion

Overall, we observed very good agreement for the ALS diagnosis between death certificates and hospital discharge, a comparison that has not been previously conducted, indicating that information from death certificates is also highly valuable for large-scale epidemiologic studies, especially when discharge data are not available. We also compared the death certificates to medical records for 173 subjects. The estimated sensitivity in our study is within the range of previously published reports (74–94%) [16–18]. When comparing the hospital discharge diagnoses to medical records the estimated PPV was also high and within the range of previously published values [16, 19–21].

There is great advantage in the use of administrative data or population-based registries for ALS epidemiology. Use of hospital discharges has been validated in several studies, always yielding high estimated sensitivities (78.9% to 93%) [16, 19–21]. Death certificates have also been validated for use in epidemiologic studies, also with high reported sensitivity and PPV estimates [17, 18].

For this study, we used data from the Danish Registers system to compare death certificate with hospital discharge ALS diagnoses. Our results indicate very good agreement between the two, as indicated by the high sensitivity and PPV estimates. It should be noted, that we do not believe that the hospital discharge data should be considered as the “gold standard”. To address this limitation, we conducted a sensitivity analysis including only those subjects with at least two inpatient hospitalizations. In this analysis, the estimated death certificate sensitivity was indeed higher than in the main analysis (91% vs. 84%).

To assess the accuracy of the hospital discharges further, we compared discharge data to medical records for 173 subjects. The estimated PPV from our study is comparable, albeit slightly higher, with those of other published studies [16, 19–21]. For instance, Pisa et al. [19] estimated a PPV of 67%, vs. 77.5% in our study, and when they included the clinically suspected ALS cases the PPV increased to 87% compared to our 92.5%. Sensitivity could not be estimated as medical records could only be selected for cases identified through hospital discharges. It should be noted that we were only able to obtain slightly less than 50% of the requested medical records, although we have no reason to suspect that this was related to their accuracy.

An interesting finding from our study is that we observed significant differences in estimated sensitivities both by sex and age. In particular, estimated sensitivity dropped with increasing age, but more so for men than women (Figure 2). This finding indicates differential outcome misclassification by sex and age that could lead to detection of effect modification by sex or age, when death certificate data are used in large epidemiologic studies. The impact of the different sensitivity estimates by sex and age and the implications for detection of spurious effect modifications should be further explored. Furthermore, given the very low estimated sensitivity for the older ages, we recommend sensitivity analyses that restrict case identification to subjects younger than 75 years old, to alleviate outcome misclassification.

We also observed significant differences in the estimated sensitivity when we stratified by ICD-8 and ICD-10, indicating that the ALS codes are not completely compatible across classifications. Specifically, higher sensitivities were observed for ICD-8 codes, under which a specific code for ALS exists, than ICD-10 (only MND). Given our data and the analyses we conducted, we could not exclude the possibility that these differences could also be due to a calendar year effect.

In conclusion, we compared death certificate vs. hospital discharge ALS diagnoses, using data from the Danish Registers system. We found the two data sources to be in very good agreement, both capturing ALS cases very well. Our findings provide evidence that use of hospital discharge or death certificate data for ALS ascertainment is a valuable tool in large epidemiologic studies, and we believe this to likely be generalizable to many other settings. In epidemiological studies where identification of incident ALS cases may be difficult or less reliable than with a National Registry like that in Denmark, death certificates may be a valuable tool. Our findings suggest that use of cases identified through death certificates that are older than 75 years old should be conducted with caution, as these could lead to increased outcome misclassification.