Estimating the number of coronary artery bypass graft and percutaneous coronary intervention procedures in Canada: A comparison of cardiac registry and Canadian Institute for Health Information data sources

Abstract

BACKGROUND:

Provincial cardiac registries and the Canadian Institute for Health Information (CIHI) pan-Canadian administrative databases are invaluable tools for understanding Canadian cardiovascular health and health care. Both sources are used to enumerate cardiovascular procedures performed in Canada.

OBJECTIVE:

To examine the level of agreement between provincial cardiac registry data and CIHI data regarding procedural counts for coronary artery bypass grafts (CABGs) and percutaneous coronary interventions (PCIs).

METHODS:

CIHI staff obtained CABG and PCI counts from seven provinces that, in 2004, performed these procedures and had a cardiac registry (ie, British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Nova Scotia, and Newfoundland and Labrador). Structured mail questionnaires, and e-mail and telephone follow-ups elicited information from a designated registry respondent. The CIHI derived its counts of CABG and PCI procedures by applying the geographical boundaries, procedural definitions and analytical case criteria used by the cardiac registries to CIHI inpatient and day procedure databases. Steps were taken to reduce double-counting procedures when combining results from the two CIHI databases. Two measures were calculated: the absolute difference between registry and CIHI estimates, and the per cent agreement between estimates from the two sources.

RESULTS:

All seven cardiac registries identified as eligible for the study participated. Agreement was high between the two sources for CABG (98.8%). For PCI, the level of agreement was high (97.9%) when CIHI sources were supplemented with day procedure data from Alberta.

CONCLUSIONS:

The high level of agreement between cardiac registry and CIHI administrative data should increase confidence in estimates of CABG and PCI counts derived from these sources.

Increased investments are needed in the knowledge infrastructure to better inform practitioners as they care for patients afflicted with heart disease. This recommendation is one of many outlined in The Canadian Heart Health Strategy and Action Plan – a comprehensive and far-reaching report issued in early 2009 by a broad coalition of health groups (1). Widespread adoption of electronic health records, further development of integrated, population-based databases, and targeted research efforts will all contribute to realizing the goals of the strategy’s action plan. Until these efforts are implemented, available data sources can be mined to answer some of the outstanding questions that are being posed by clinicians, consumers and policy makers regarding the status of cardiovascular health and health care in Canada.

Canadian cardiac specialists have been at the forefront of the use of clinical and administrative data to gain insights into population health, the effectiveness of interventions and how to better deliver care (2–10). These efforts have been aided by initiatives to systematically collect information on cardiac revascularization procedures such as coronary artery bypass grafts (CABGs) and percutaneous coronary interventions (PCIs). In many places where these specialized procedures are available, clinicians have organized provincial or regional cardiac registries to capture important clinical and administrative data. With similar aims to improve our knowledge of Canadian’s health and their health care, the Canadian Institute for Health Information (CIHI) has assembled pan-Canadian databases on both inpatient and day procedure care to inform clinicians, consumers, health administrators and policy makers as they make decisions about health care delivery across Canada. These CIHI data are also increasingly being used by health services researchers to study patient health outcomes and quality of health care. The CIHI publishes several indicators of the effectiveness of cardiovascular care (eg, 30-day acute myocardial infarction in-hospital mortality rate and acute myocardial infarction readmission rate) by province and regional health authority (11), and has analyzed the relationship between cardiac revascularization and mortality rates (11,12).

While provincial cardiac registries are invaluable to practitioners and planners within provinces, it is often useful to compare across provinces to identify variations that may be studied to identify best practices or opportunities to improve care. Such pan-Canadian analyses of care are difficult to conduct because of differences in the content and structure of provincial cardiac registries. The standardized data provided by the CIHI permit cross-province comparisons, but lack much of the more detailed clinical data collected within cardiac registries. While designed for different purposes, it is useful to compare the two sources of information.

The present study examines how comparable CIHI data and cardiac registry data are for two common interventions: CABG and PCI. The number of procedures captured within CIHI’s databases is compared with the number identified by provincial cardiac registries. Investigators were motivated to make this comparison because a high level of agreement between the two sources would increase confidence in estimates derived from both of them. The present study is consistent with the CIHI’s ongoing efforts to improve data quality. The CIHI data quality program and activities are documented on the CIHI website.

METHODS

Cardiac registry sources

The present study was limited to provinces that, in 2004, had facilities in which CABG and PCI procedures were performed, and procedural data that were reported to cardiac registries. Prince Edward Island, Yukon, the Northwest Territories and Nunavut do not have facilities that perform CABG or PCI. Quebec and New Brunswick were excluded from the study because these provinces lacked a cardiac registry in 2004. Of the seven registries included in the study, all but one was comprehensive, covering all CABG and PCI procedures performed in the province. In Saskatchewan, the registry was limited to procedures performed in a hospital located in the Regina Qu’Appelle Regional Health Authority – one of the two regional health authorities in the province performing revascularization procedures (Appendix 1).

To solicit study participation, a letter of invitation was sent to senior staff of the provincial cardiac registries. The study coordinator contacted potential respondents by telephone to further explain the study aims, describe its methods, respond to any questions and identify a suitable respondent within each registry. All seven registries agreed to participate and a structured questionnaire was sent to the designated contact within each registry.

Respondents were asked to provide the following: CABG and PCI counts; details on procedural definitions (eg, how multivessel procedures are counted) and the analytical criteria used to identify cases (eg, age, residence, and inclusion or exclusion of outpatient procedures); and the names of facilities reporting to the cardiac registry. Details of analytical criteria used by each provincial registry are provided in Appendix 1. Returned questionnaires were reviewed and respondents were recontacted to complete any unanswered questions or amend responses that were not clear.

Administrative data sources

CABG counts were obtained from inpatient data. PCI counts were obtained from inpatient and day procedure data.

Inpatient data:

Inpatient hospitalization statistics were obtained from CIHI’s Discharge Abstract Database (DAD). This database captures administrative, clinical and demographic information on inpatient and day surgery discharges from acute care hospitals in Canada outside Quebec.

Day procedure data:

Information on day procedures was obtained from the DAD, and CIHI’s National Ambulatory Care Reporting System. The National Ambulatory Care Reporting System data holding captures administrative, clinical and demographic information on ambulatory care events in Ontario, and selected Nova Scotia facilities. At the time of the study, Alberta was not submitting day procedure data to the CIHI. A special tabulation from the Alberta ambulatory care data set, provided by Alberta Health and Wellness, was used to complete the day procedure PCI data.

Data analysis

Data analysis was conducted in two stages. First, provincial CABG and PCI counts were tabulated from CIHI databases using local registry procedure definitions and analytical criteria as described in questionnaire responses. Second, the level of agreement between CIHI and cardiac registry CABG and PCI counts was calculated by province and for the seven-province total.

Tabulation of CABG and the PCI counts from CIHI’s databases:

For each province, two CIHI research analysts worked independently to develop SAS (SAS Institute Inc, USA) computer programs that applied cardiac registry procedure definitions and analytical criteria to CIHI’s databases. The following Canadian Classification of Health Interventions (version 2003) codes were used to identify procedures in CIHI databases: 1.IJ.50^^ or 1.IJ.57.GQ^^ (PCI); 1.IJ.76^^ (CABG). Any disagreements in results were reconciled with input from study team members.

Because both inpatient and day procedure data sets were used, there was a potential risk of double-counting PCI procedures. Three steps were taken to minimize this risk. First, all inpatient PCIs with an ‘out-of-hospital’ flag were removed from the data set (ie, the records for the procedures performed in a facility other than the reporting hospital). Second, any procedures reported by a facility without on-site PCI services (ie, nonperforming facilities as identified on questionnaires) were removed from the counts. Finally, if a patient was reported as having both a day procedure PCI and an inpatient PCI on the same day, only one of these procedures was included in the total counts.

Calculation of the level of agreement between cardiac registries and CIHI’s databases:

The following measures were calculated for each province and for the pooled provincial counts:

$$\text{Absolute}\hspace{0.17em}\text{difference}=|\text{CIHI's}\hspace{0.17em}\text{count}-\text{cardiac}\hspace{0.17em}\text{registry's}\hspace{0.17em}\text{count}|$$

$$\text{Per}\hspace{0.17em}\text{cent}\hspace{0.17em}\text{agreement}=100\%-(\text{absolute}\hspace{0.17em}\text{difference}/\text{cardiac}\hspace{0.17em}\text{registry's}\hspace{0.17em}\text{count})\times 100.$$

Agreement over 95% was considered to be high (13,14).

RESULTS

The counts, absolute difference in counts, and per cent agreement between counts from the provincial cardiac registries and CIHI databases are shown in Tables 1 and 2 for CABG and PCI, respectively.

TABLE 1.

Agreement between the coronary artery bypass graft (CABG) counts from the Canadian Institute for Health Information (CIHI)’s databases and provincial cardiac registries, 2004^*

Province	CABG count, n			Agreement§, %
	CIHI database†	Cardiac registry	Absolute difference‡
British Columbia	2115	2087	28	98.7
Alberta	1812	1807	5	99.7
Saskatchewan¶	535	539	4	99.3
Manitoba	898	850	48	94.4
Ontario	9213	9114	99	98.9
Nova Scotia	885	872	13	98.5
Newfoundland and Labrador	612	609	3	99.5
Overall	16,070	15,878	192	98.8

^*Fiscal year (ie, April 1, 2004, to March 31, 2005) for all provinces;

^†Discharge Abstract Database;

^‡Absolute difference = |CIHI count – cardiac registry’s count|;

^§Per cent agreement = 100% – (absolute difference / cardiac registry’s count) × 100;

^¶Limited to Regina Qu’Appelle Regional Health Authority

TABLE 2.

Agreement between the percutaneous coronary intervention (PCI) counts from the Canadian Institute for Health Information (CIHI)’s databases and cardiac registries, 2004^*

Province	PCI count, n			Agreement§, %
	CIHI database†	Cardiac registry	Absolute difference‡
British Columbia	6747	6897	150	97.8
Alberta (CIHI data only)	3319	5083	1764	65.3
Alberta (CIHI and AHW¶ data)	5078	5083	5	99.9
Saskatchewan**	700	686	14	98.0
Manitoba	1297	1285	12	99.1
Ontario	17,340	17,881	541	97.0
Nova Scotia	1632	1651	19	98.8
Newfoundland and Labrador	614	635	21	96.7
Overall
Without AHW data	31,649	34,118	2469	92.8
With AHW data	33,408	34,118	710	97.9

^*Fiscal year (ie, April 1, 2004, to March 31, 2005) for all provinces;

^†Discharge Abstract Database and National Ambulatory Care Reporting System;

^‡Absolute difference = |CIHI count – cardiac registry’s count|;

^§Per cent agreement = 100% – (absolute difference / cardiac registry’s count) × 100;

^¶Alberta Health and Wellness (AHW) Ambulatory Care Database: same day surgery PCI data obtained at special request;

^**Limited to Regina Qu’Appelle Regional Health Authority

Agreement between CIHI and registry counts for CABG

For CABG, the agreement between the CIHI and registry overall pooled counts was 98.8%. The absolute difference in counts between the two data sources was 192 procedures. The level of agreement between CIHI and registry sources ranged from 94.4% in Manitoba to 99.7% in Alberta. Therefore, according to the study criteria, agreement between the two sources of data was high for all but Manitoba, where the level of agreement was below the cut-off point of 95% by only 0.6%. With the exception of Saskatchewan, CIHI counts were higher than registry counts in all provinces (Table 1).

Agreement between CIHI and registry counts for PCI

For PCI, the agreement between the CIHI and registry overall pooled counts was 97.9%. The absolute difference in counts between the data sources was 710 procedures. These estimates are based on the use of day procedure PCI data from the Alberta Health and Wellness Ambulatory Care Database along with CIHI data. Therefore, according to the study criteria, agreement between the two sources of data was high for all provinces. CIHI PCI counts were slightly lower than registry counts in all provinces except Saskatchewan and Manitoba (Table 2).

DISCUSSION

Data from clinical registries and administrative databases have increasingly been used to study the use of health services and outcomes of care. The use of these data for research purposes is based on the assumption that the information is reasonably accurate. However, errors could occur in the process of creating administrative databases or clinical registries as a result of unclear coding directives, coders’ noncompliance to these directives, hospital policies that unintentionally negatively impact the quality of the data, the quality and completeness of the chart documentation, and unintentional human error introduced during the abstracting and coding process (15).

Validation studies are important for understanding and improving data quality. Several recent studies (15–20) assessed the validity of administrative databases in Canada. Research findings suggest that cardiovascular procedures are accurately recorded in hospital discharge records. For example, a study (16) of the coding accuracy of Ontario hospitals participating in the Ontario Case Costing Initiative from 2002 to 2004 found the concordance (measured by Kappa index) between the CIHI DAD records and reabstracted medical records for CABG and PCI to be 1.00 and 0.98, respectively. A study (17) that compared hospital discharge records with corresponding medical records in four hospitals in Alberta for 2003 reported a sensitivity of 94% and a positive predictive value of 90% for PCI procedures. Another study (15) that compared coding accuracy of PCI procedures in the DAD for fiscal year 2005 reported a sensitivity of 89% and a positive predictive value of 87%. However, we could not identify published studies comparing cardiac clinical registries and administrative databases in Canada.

The high level of agreement observed between the counts for CABG and PCI from provincial cardiac registries and CIHI’s databases (supplemented, in the case of Alberta, with provincial day procedure data) provides some measure of confidence that each source is complete.

Limitations

An important limitation of the present study is that it is not a true validation study. Individual records from provincial cardiac registries were not linked to records held within CIHI’s databases. The high level of agreement in counts between the two sources does not necessarily indicate that information on CABG and PCI are accurate. Both sources could be incorrect if there were errors of overestimation and underestimation that cancelled each other out. For example, if CIHI systematically over-counted procedures among the elderly, but under-counted PCIs among younger adults and, conversely, if the cardiac registries over-counted procedures among younger adults, but under-counted procedures among the elderly, there could be a high level of overall agreement between the two sources, with both sources being inaccurate. However, findings from the provincial studies showing hospital discharge data precisely reflecting medical records and physician reports of cardiac procedures (16,17) suggest that this is not a likely threat to the study’s findings. A second limitation of the present study is that it includes only one year of data. Temporal variation in reporting is not captured.

Despite these limitations, the study’s findings support the use of registry and CIHI data to produce regional, provincial and national estimates of cardiovascular procedures for clinicians, consumers, health decision makers and health services researchers.

APPENDIX 1.

Analytical criteria used by the provincial cardiac registries

Province	Cardiac registry	Analytical criteria
		Age	All patients undergoing procedure in the hospitals reporting to the registry included	Used procedure as an encounter	Reporting time frame	Cancelled procedures included	Procedures abandoned after onset included	Day surgery PCIs included	CABG counts included
British Columbia	British Columbia Cardiac Registries	All ages	Yes	Yes	Procedure date	No	No	Yes	Isolated CABG only
Alberta	APPROACH	20 years and older	Yes	Yes	Procedure date	No	No	Yes	Any CABG
Saskatchewan	APPROACH partner site*	All ages	Yes	Yes	Procedure date	No	Yes	Yes	Any CABG
Manitoba	Cardiac Sciences Program of Winnipeg Regional Health Authority	18 years and older	Yes	Yes	Procedure date	No	Yes	Yes	Any CABG
Ontario	Cardiac Care Network of Ontario	All ages	PCI – Yes; CABG – only patients with a valid OHIP number	Yes	Procedure date	No	No	Yes	Isolated CABG only
Nova Scotia	Maritime Heart Centre Cardiac Surgery Registry (CABG data)	All ages	Yes	Yes	Discharge date	No	Yes	N/A	Any CABG
	APPROACH partner site (PCI data)	All ages	Yes	Yes	Procedure date	No	Yes	Yes	N/A
Newfoundland and Labrador	Cardiac Care Program of St John’s Health Services Corporation	All ages	Yes	Yes	Procedure date	No	No	Yes	Any CABG

The table reflects analytical criteria used to produce counts for this study. These criteria might be different from those used by the registries to produce their reports.

^*Saskatchewan registry coverage limited to patients treated in hospitals located in the Regina Qu’Appelle Regional Health Authority. APPROACH Alberta Provincial Project for Outcomes Assessment Coronary Heart Disease; CABG Coronary artery bypass graft; N/A Not applicable; OHIP Ontario Health Insurance Plan; PCI Percutaneous coronary intervention