Positive predictive value of cardiac examination, procedure and surgery codes in the Danish National Patient Registry: a population-based validation study

Abstract

Objective

Danish medical registries are widely used for cardiovascular research, but little is known about the data quality of cardiac interventions. We computed positive predictive values (PPVs) of codes for cardiac examinations, procedures and surgeries registered in the Danish National Patient Registry during 2010–2012.

Design

Population-based validation study.

Setting

We randomly sampled patients from 1 university hospital and 2 regional hospitals in the Central Denmark Region.

Participants

1239 patients undergoing different cardiac interventions.

Main outcome measure

PPVs with medical record review as reference standard.

Results

A total of 1233 medical records (99% of the total sample) were available for review. PPVs ranged from 83% to 100%. For examinations, the PPV was overall 98%, reflecting PPVs of 97% for echocardiography, 97% for right heart catheterisation and 100% for coronary angiogram. For procedures, the PPV was 98% overall, with PPVs of 98% for thrombolysis, 92% for cardioversion, 100% for radiofrequency ablation, 98% for percutaneous coronary intervention, and 100% for both cardiac pacemakers and implantable cardiac defibrillators. For cardiac surgery, the overall PPVs was 99%, encompassing PPVs of 100% for mitral valve surgery, 99% for aortic valve surgery, 98% for coronary artery bypass graft surgery, and 100% for heart transplantation. The accuracy of coding was consistent within age, sex, and calendar year categories, and the agreement between independent reviewers was high (99%).

Conclusions

Cardiac examinations, procedures and surgeries have high PPVs in the Danish National Patient Registry.

Strengths and limitations of this study.

• This is the first study to examine the positive predictive value of the most commonly performed cardiac examinations, procedures and surgeries recorded in the Danish National Patient Registry.

• Medical charts information served as the reference for the validation. The agreement between independent reviewers was high (99%).

• Our study was restricted to one out of five Danish regions; however, owing to a highly homogeneous Danish healthcare system, our results are likely generalisable to other Danish regions.

Introduction

The mortality rate among patients with cardiovascular disease has declined over the past two decades.¹ However, it remains the leading cause of death worldwide. Danish medical registries are widely recognised as being among the best population-based data sources in the world,² owing to the capability for individual-level data linkage across registries and the possibility for long-term follow-up.² As the key register, the Danish National Patient Registry (DNPR) has provided registration of all hospital admissions in Denmark since 1977.³ It has been extensively used to identify cardiovascular diagnoses, but to a lesser extent used to ascertain information on cardiac examinations, procedures and surgeries, partly due to lack of knowledge about the validity of these variables.⁴ The data validity is important for several reasons. First, misclassification threatens study findings. Second, validation studies permit researchers to quantify the extent of misclassification and evaluate its impact on the study results. Finally, reporting findings from validation studies may motivate physicians to improve coding accuracy.⁵ We therefore examined the positive predictive value (PPV) of codes for cardiac examinations, procedures and surgeries in the DNPR.

Methods

Setting and design

This study was conducted in the Central Denmark Region with a source population of 1.2 million residents. The study period was from 1 January 2010 to 31 December 2012.⁶ All Danish residents have unfettered access to healthcare services, including all types of cardiac examinations, procedures and surgeries.³ The DNPR has maintained information on all non-psychiatric hospital admissions since 1977.³ Cardiac surgeries have been recorded in the DNPR according to the Nordic Medico-statistical Committee's Classification of Surgical Procedures (NOMESCO) since 1996.³ Denmark is divided into five regions, which each are representative of the Danish population with respect to demographic, and socioeconomic characteristics as well as healthcare usage and medication use.⁷ Each region typically has one major university hospital (including a high volume cardiac intervention centre) and several smaller regional hospitals (performing some, but not all cardiac interventions). Registration of cardiac procedures in the DNPRs is performed by the treating physician. Each hospital is required by law to submit their data to the DNPR at least monthly.³ Data from the DNPR are used by Danish researchers, but collaboration with foreign researchers is common.

Study population

We used the DNPR to randomly sample patients from different types of hospitals in the Central Denmark Region. Specifically, we sampled from the region's university hospital (Aarhus University Hospital) and from two larger regional hospitals (Regional Hospital of Randers and Regional Hospital of Herning). Given the homogeneity of the Danish healthcare system, we considered these hospitals representative for hospitals of similar size in other Danish regions.⁷ Patients were sampled from the departments of cardiology, internal medicine, acute medicine, neurology and cardiothoracic surgery. We identified patients who underwent cardiac examinations, which included echocardiography, right heart catheterisation and coronary angiogram. We also identified patients who underwent the following procedures: thrombolysis, cardioversion, radiofrequency ablation (used for cardiac diseases), percutaneous coronary intervention (PCI), and implantation of a cardiac pacemaker or implantable cardiac defibrillator (ICD). We identified patients undergoing cardiac surgery, including mitral valve and aortic valve surgery, coronary artery bypass graft surgery and heart transplantation. We sampled 100 patients for each code (or if fewer were available, the highest number obtainable). For echocardiography and PCI, the sample of 100 patients was attained by sampling 50 patients each for transthoracic echocardiography and transoesophageal echocardiography and 50 patients each for unspecified PCI and PCI with stent implantation. All patients with a given code in the study period were identified and assigned a random number between 0 and 1, and we then selected the 100 lowest numbers. All codes used in the study are given in online supplementary table S1.

Medical record review

We considered the information in the medical record review as the gold standard. One physician (KA) reviewed all medical records. This investigator identified the relevant part of the medical record (ie, the description of the examination, procedure or surgery), and judged if the corresponding record in the DNPR was correct. In cases of doubt, secondary independent reviews (by JS and TM) were planned to reach consensus. As no doubts were raised in any cases, second or third reviews were not performed. However, to investigate whether the assessment of the data extraction could be considered independent, we performed a sensitivity analysis of 100 randomly selected patients, whose medical records were also adjudicated by the two other reviewers (JS and TM). We subsequently calculated the proportion of cases that could be confirmed by these second reviewers.

When validating the indications for ICD implantation, we defined primary prevention and secondary indication according to the definitions given in online supplementary table S2.

All data were entered into Epidata V.3.1 (EpiData Association, Odense, Denmark, http://www.epidata.dk) using a medical chart extraction form (online supplementary table S3).

Statistical analysis

We calculated the PPVs with 95% CIs according to the Wilson Score method.⁸ We computed PPVs separately for subgroups of echocardiography (transthoracic vs transoesophageal echocardiography) and PCI (unspecified PCI vs PCI with stent implantation). For ICDs, we disaggregated the sample into patients receiving ICDs for primary versus secondary prophylaxis.

Analyses were stratified by age group (<60, 60–80, and >80 years), sex and calendar year (2010, 2011 and 2012). The patients were sampled using SAS, V.9.2 (SAS Institute, Cary, North Carolina, USA), while the analyses were performed using Microsoft Excel 2010 and Stata statistical software, V.13 (StataCorp LP). In accordance with Danish law, no approval from the Ethics Committee was required.

Results

We identified 1239 patients from the DNPR during 2010–2012. Of these, medical records were available for 1233 (99%) patients. Except for heart transplantation, 100 patients were sampled for each of the codes, while 50 patients were sampled for each prespecified subgroup (figure 1). Majority of patients were sampled from Aarhus University Hospital (89% for examinations, 90% for procedures and 100% for surgery). For cardiac examinations, the overall PPV was 98% (95% CI 96% to 99%), reflecting a PPV of 97% for echocardiography, 97% for right heart catheterisation and 100% for coronary angiogram (figure 1). The overall PPV for cardiac procedures was 98% (95% CI 97% to 99%). Individual PPVs were 98% for thrombolysis, 92% for cardioversion, 100% for radiofrequency ablation, 98% for PCIs, and 100% both for cardiac pacemakers and ICDs. The PPV was 100% for secondary ICDs, but was somewhat lower for primary ICDs (83%). The overall PPV was 99% (95% CI 97% to 100%) for cardiac surgery; individual PPVs were 100% for mitral valve surgery, 99% for aortic valve surgery, 98% for coronary artery bypass graft surgery and 100% for heart transplantation. Analyses stratified by age, sex and calendar year closely agreed with our main findings (tables 1 and 2 and online supplementary table S4). Finally, in the sample of 100 randomly selected patients, the decisions made by the primary reviewer could be confirmed by the second and third reviewer in 99% of the cases.

Figure 1.

PPV of codes for cardiac examinations, procedures and surgeries in the Danish National Patient Registry, 2010–2012. *Number of correct codes/total number of medical record reviews. CABG, coronary artery bypass graft surgery; ICD, implantable cardiac defibrillator; PCI, percutaneous coronary intervention; PPV, positive predictive value; RFA, radiofrequency ablation; RHC, right heart catheterisation; TEE, transoesophageal echocardiography; TTE, transthoracic echocardiography.

Table 1.

Positive predictive value of cardiac examinations, procedures and surgeries in the Danish National Patient Registry, by age group

	<60 years		60–80 years		>80 years
	Number of patients	Positive predictive value (95% CI)	Number of patients	Positive predictive value (95% CI)	Number of patients	Positive predictive value (95% CI)
Examination
TTE	23	100 (86 to 100)	21	95 (77 to 99)	5	100 (57 to 100)
TEE	19	100 (83 to 100)	22	91 (72 to 97)	8	100 (68 to 100)
RHC	58	95 (86 to 98)	37	100 (91 to 100)	5	100 (57 to 100)
Coronary angiogram	27	100 (88 to 100)	55	100 (93 to 100)	18	100 (82 to 100)
Procedure
Thrombolysis	23	100 (86 to 100)	52	96 (87 to 99)	21	100 (85 to 100)
Cardioversion	28	96 (82 to 99)	66	91 (82 to 96)	83	83 (44 to 97)
RFA	58	100 (94 to 100)	38	100 (91 to 100)	4	100 (51 to 100)
Unspecified PCI	16	100 (81 to 100)	24	100 (86 to 100)	10	100 (72 to 100)
PCI with stent implantation	17	88 (66 to 97)	29	100 (88 to 100)	4	100 (51 to 100)
Cardiac pacemaker	10	100 (72 to 100)	46	100 (92 to 100)	44	100 (92 to 100)
ICD	33	100 (90 to 100)	63	100 (95 to 100)	4	100 (51 to 100)
Surgery
Mitral valve surgery	45	100 (92 to 100)	48	100 (93 to 100)	7	100 (65 to 100)
Aortic valve surgery	16	94 (72 to 99)	53	100 (93 to 100)	31	100 (89 to 100)
CABG surgery	19	95 (75 to 99)	70	99 (92 to 100)	11	100 (74 to 100)
Heart transplantation	33	100 (90 to 100)	6	100 (61 to 100)	N/A	N/A

CABG, coronary artery bypass graft surgery; ICD, implantable cardiac defibrillator; PCI, percutaneous coronary intervention; RFA, radiofrequency ablation; RHC, right heart catheterisation; TEE, transoesophageal echocardiography; TTE, transthoracic echocardiography.

Table 2.

Positive predictive value of cardiac examinations, procedures and surgeries in the Danish National Patient Registry, by gender

	Men		Women
	Number of patients	Positive predictive value (95% CI)	Number of patients	Positive predictive value (95% CI)
Examination
TTE	22	95 (78 to 99)	27	100 (88 to 100)
TEE	34	97 (85 to 99)	15	93 (70 to 99)
RHC	54	94 (85 to 98)	46	100 (92 to 100)
Coronary angiogram	62	100 (94 to 100)	38	100 (91 to 100)
Procedure
Thrombolysis	57	96 (88 to 99)	39	100 (91 to 100)
Cardioversion	60	93 (84 to 97)	40	90 (77 to 96)
RFA	61	100 (94 to 100)	39	100 (91 to 100)
Unspecified PCI	31	100 (89 to 100)	19	100 (83 to 100)
PCI with stent implantation	44	95 (85 to 99)	6	100 (61 to 100)
Cardiac pacemaker	60	100 (94 to 100)	40	100 (91 to 100)
ICD	79	100 (95 to 100)	21	100 (85 to 100)
Surgery
Mitral valve surgery	63	100 (94 to 100)	37	100 (91 to 100)
Aortic valve surgery	59	100 (94 to 100)	41	98 (87 to 100)
CABG surgery	71	97 (90 to 99)	29	100 (88 to 100)
Heart transplantation	23	100 (86 to 100)	16	100 (81 to 100)

CABG, coronary artery bypass graft surgery; ICD, implantable cardiac defibrillator; PCI, percutaneous coronary intervention; RFA, radiofrequency ablation; RHC, right heart catheterisation; TEE, transoesophageal echocardiography; TTE, transthoracic echocardiography.

Discussion

This study showed that cardiac examinations, procedures and surgeries were coded with high accuracy in the DNPR for all sex and age groups during 2010–2012.

Our study provides the first validation of codes for the most frequently performed cardiac examinations, procedures and surgeries in the DNPR. The only previous study to examine the validity of cardiac examinations in the DNPR focused on 282 patients, on whom cardiac CT angiography was performed between 2008 and 2012.⁹ Using medical records as reference, this study found a PPV of 100% (95% CI 99% to 100%) for this examination.⁹ The accuracy of the codes for non-cardiac surgery in the DNPR have previously been found to vary substantially.³ It seems high for gastrointestinal surgery (PPV=99% for appendectomy and 100% for cholestectomia),³ but lower for orthopaedic surgery procedures (PPV=69%).³ For different types of gynaecological surgery, the PPVs varied considerably (55% to 99%).³

In healthcare systems outside Denmark, the accuracy of codes for cardiac examinations, procedures and surgeries remains largely unknown. However, a survey from the Canadian Institute for Health Information demonstrated that codes for cardiac procedures had high PPVs compared with a prospective clinical registry.¹⁰ In line with our findings, the study reported PPVs of 96% for PCI, 98% for coronary artery bypass graft surgery, 97% for valve surgery and 95% for cardiac catheterisation.¹⁰

The Diagnosis-Related Group (DRG) system, established in 2002, ensures that public hospitals receive payment for procedures and surgeries.¹¹ These economic incentives increase the likelihood of accurate coding. Although not examined in this study, they, along with the nationwide coverage, also increase the completeness of registration. Private hospitals and clinics remain a potential source of under-reporting, although registration of procedures at these institutions is mandatory and urged by the Danish Health and Medicines Authority.³ It remains for future studies to estimate other measures of data quality such as sensitivity and specificity.

The DNPR offers a variety of potential uses in research,³ given its routine, longitudinal registration of health history and the possibility of individual-level linkage across different registries. Assessing data quality for epidemiological research (sensitivity vs specificity), it is always necessary to consider it in the context of individual study design. A high PPV is particular important when identifying cohorts for prognosis studies or in subanalyses restricted to patients undergoing specific cardiac interventions. In addition to supporting studies of trends and prognosis of cardiac diseases,¹ the DNPR offers the opportunity to study trends in cardiac examinations such as echocardiography,¹² cardiac procedures such as ICD implantation¹³ and surgeries. Finally, the DNPR may be used to study prognostic factors, as well as procedure outcomes (eg, revascularisation) that are useful in defining composite outcomes. Still, the DNPR lacks detailed information on other variables, including examination results (eg, left ventricular ejection fraction) and procedure and surgery details (eg, types of cardiac stent).

Our study has potential limitations. It was restricted to one out of five Danish regions. However, the homogeneity of the Danish healthcare system makes it likely that our results also apply to other Danish regions.⁷ Our results may not necessarily be applicable to other countries, other healthcare systems or earlier study periods.³ Still, we find it less likely that the validity of the codes have varied substantially since the introduction of the DRG system in 2002 in Denmark. Most patients in our study were sampled from the university hospital because a majority of cardiac procedures are performed in that setting. We were therefore unable to stratify our results by regional versus university hospital. In the Central Denmark Region, right heart catheterisation, radiofrequency ablation, PCI, ICD implantation and all types of cardiac surgery are performed at the university hospital only, while other cardiac procedures and examinations are performed both at the university and regional hospitals. For the procedures examined in subgroup analyses (echocardiography and PCI), it should be noted that the patient subpopulations were not randomly sampled. Still, PPVs were consistently high within all subgroups.

Conclusion

We found consistently high PPVs for cardiac examinations, procedures and surgeries in the DNPR during 2010–2012, confirming the potential of these variables for cardiovascular research.