Abstract
Objective To determine the association of non-invasive cardiac stress testing before elective intermediate to high risk non-cardiac surgery with survival and hospital stay.
Design Population based retrospective cohort study.
Setting Acute care hospitals in Ontario, Canada, between 1 April 1994 and 31 March 2004.
Participants Patients aged 40 years or older who underwent specific elective intermediate to high risk non-cardiac surgical procedures.
Interventions Non-invasive cardiac stress testing performed within six months before surgery.
Main outcome measures Postoperative one year survival and length of stay in hospital.
Results Of the 271 082 patients in the entire cohort, 23 991 (8.9%) underwent stress testing. After propensity score methods were used to reduce important differences between patients who did or did not undergo preoperative stress testing and assemble a matched cohort (n=46 120), testing was associated with improved one year survival (hazard ratio (HR) 0.92, 95% CI 0.86 to 0.99; P=0.03) and reduced mean hospital stay (difference −0.24 days, 95% CI −0.07 to −0.43; P<0.001). In an analysis of subgroups defined by Revised Cardiac Risk Index (RCRI) class, testing was associated with harm in low risk patients (RCRI 0 points: HR 1.35, 95% CI 1.05 to 1.74), but with benefit in patients who were at intermediate risk (RCRI 1-2 points: 0.92, 95% CI 0.85 to 0.99) or high risk (RCRI 3-6 points: 0.80, 95% CI 0.67 to 0.97).
Conclusions Preoperative non-invasive cardiac stress testing is associated with improved one year survival and length of hospital stay in patients undergoing elective intermediate to high risk non-cardiac surgery. These benefits principally apply to patients with risk factors for perioperative cardiac complications.
Introduction
Every year, approximately 900 000 adults worldwide experience major cardiac complications after surgery,1 which are associated with increased mortality2 and hospital stay.3 Preoperative non-invasive cardiac stress testing may help prevent these complications. This strategy detects underlying ischaemic heart disease and identifies individuals who might benefit from preoperative interventions, aggressive intraoperative haemodynamic management, closer postoperative surveillance, or avoiding surgery. Additionally, the results of stress testing can guide the use of perioperative β blockade, which has greatest benefit in patients with inducible ischaemia.4 Given these benefits, the American College of Cardiology and American Heart Association guidelines recommend preoperative non-invasive stress testing,5 but only in individuals with clinical risk factors for cardiac complications.
Conversely, some authors have discouraged the use of preoperative stress testing6 7 8 because it may delay surgery and has not been shown to improve postoperative outcomes. Instead, they advocate routine perioperative β blockade.6 8 This recommendation was supported by the Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echo II (DECREASE II) randomised trial, which evaluated routine stress testing in intermediate risk patients receiving perioperative β blockade.8 The study found no significant effect of preoperative stress testing on cardiac complications. None the less, the estimated effect had a wide 95% confidence interval (odds ratio 0.28 to 2.91) that did not exclude either substantial benefit or harm. Additionally, the Perioperative Ischemic Evaluation (POISE) trial has raised concerns about the safety of perioperative β blockade.9
Given the potential benefits of preoperative stress testing but the lack of proved impact on outcomes, we undertook a population based cohort study of non-invasive cardiac stress testing in Ontario, Canada. Our objective was to determine whether stress testing before major elective non-cardiac surgery was associated with improved one year survival and reduced hospital stay.
Methods
We used linked population based administrative healthcare databases in Ontario, Canada, to undertake a retrospective cohort study. The databases used were the Canadian Institute for Health Information (CIHI) discharge abstract database, which describes all hospital admissions; the Ontario Health Insurance Plan (OHIP) database, which describes physician billing for inpatient and outpatient services; the Registered Persons Database (RPDB), which describes demographics and vital statistics; the Corporate Providers Database (CPDB), which describes physicians’ specialties; the Ontario Drug Benefit (ODB) database, which describes prescription medications dispensed to all individuals aged older than 65 years; and the 2001 Canadian census. Although these databases lack information on physiological and laboratory measures (for example, blood pressure and haemoglobin concentration), they have been validated for many other outcomes, exposures, and comorbidities.10 11 12 13 14 15 During the study period, Ontario was Canada’s most populous province, with approximately 12 million residents who have access to physician and hospital services through a universal healthcare programme.
Cohort
Using previously described methods,16 we retrospectively identified all Ontario residents aged 40 years or older who underwent one of the following specific types of elective surgery during fiscal years 1994 to 2003 (1 April 1994 to 31 March 2004): abdominal aortic aneurysm repair; carotid endarterectomy; peripheral vascular bypass; total hip replacement; total knee replacement; large bowel surgery; liver resection; Whipple procedure; pneumonectomy; pulmonary lobectomy; gastrectomy; oesophagectomy; nephrectomy; or cystectomy. These procedures were selected because they are intermediate to high risk for perioperative cardiac complications,5 applicable to either sex, and described in research studies that used the CIHI database17 18 19 20 21: procedure codes in the CIHI database have excellent accuracy.15
The principal exposure was preoperative non-invasive stress testing, as defined by a physician billing for an outpatient stress test within 180 days before surgery. These tests included graded exercise treadmill testing, nuclear perfusion imaging (exercise or dipyridamole myocardial perfusion imaging using sestamibi or thallium as a radiotracer, with or without single photon emission computed tomography), and stress (exercise or dobutamine) echocardiography. We used OHIP fee codes and methods adapted from previous research.22 The 180 day window was chosen so that instances where stress testing led to preoperative coronary revascularisation could be included—it accounted for a 90 day period between stress testing and revascularisation23 and another 90 day window between revascularisation and surgery.5
The outcomes of interest were mortality (one year after surgery) and hospital stay. Mortality was determined using the CIHI database (in hospital deaths) and the RPDB (out of hospital deaths). We used the CIHI database to measure hospital stay.
Demographic information was obtained from the RPDB. We used validated administrative data algorithms to identify cases of diabetes and hypertension.11 14 The OHIP database was used to identify any patient who required dialysis before the index surgery. We used previously described methods to identify in the CIHI database the following other comorbidities from hospital admissions within two years preceding surgery on the basis of International Classification of Diseases codes (9th revision, clinical modification; or 10th revision): ischaemic heart disease; congestive heart failure; cerebrovascular disease; pulmonary disease; chronic renal insufficiency; malignancy; liver disease; and dementia.24 25 When identifying comorbidities, only information from acute care hospital admissions before the index surgery was used to determine the presence of ischaemic heart disease, congestive heart failure, cerebrovascular disease, pulmonary disease, renal insufficiency, and liver disease. Our primary aim was to ensure that postoperative complications were not misclassified as pre-existing comorbid diseases.26 This approach also ensured that the relevant comorbidity information was almost certainly derived from data available before the preoperative cardiac stress test.
The OHIP database and the CPDB were used to identify outpatient consultations (anaesthesiology, internal medicine, and cardiology) within 60 days before surgery, echocardiography within 180 days before surgery,22 epidural anaesthesia or analgesia (hereafter referred to as anaesthesia), and intraoperative invasive monitoring. Procedure codes in the OHIP database are very accurate.15 We imputed patients’ incomes on the basis of their neighbourhood median income in the 2001 Canadian census.
To understand how non-invasive stress testing might influence outcomes, we used the OHIP database to identify related processes of care before surgery—namely echocardiography, coronary angiography, percutaneous coronary intervention, and coronary artery bypass graft surgery—as well as processes of care after surgery—namely admission to a monitored bed (critical care unit or step-down unit) and mechanical ventilation. Additionally, the ODB database was used to identify outpatient prescriptions for β blockers or statins within 100 days before surgery. We defined new users of these medications as those who had prescriptions within 100 days before surgery but no prescriptions during the period from 180 days to one year before surgery.
Analyses
A two tailed P value of less than 0.05 was used to define statistical significance, and all estimates were calculated with 95% confidence intervals. Bivariate tests were initially used to compare the characteristics of patients who did or did not undergo preoperative stress testing (t test, Mann-Whitney U test, χ2 test, Fisher’s exact test).
We used propensity score methods to adjust for systematic differences in measured baseline characteristics between the two groups of patients in the study.27 28 A non-parsimonious multivariable logistic regression model was developed to estimate a propensity score for preoperative stress testing. Clinical significance guided the initial choice of covariates: age; sex; year of surgery; surgery; income; hospital type (teaching, low volume non-teaching, moderate volume non-teaching, or high volume non-teaching); comorbid disease; specialist consultations (anaesthesiology, general internal medicine, or cardiology); preoperative echocardiography; epidural anaesthesia; and intraoperative invasive monitoring. Previously described methods were used to categorise non-teaching hospitals into tertiles.29 We considered the following comorbid conditions that were present in at least 1% of the cohort: ischaemic heart disease; congestive heart failure; cerebrovascular disease; hypertension; diabetes; pulmonary disease; renal disease; and malignancy.
We then used a structured iterative approach to refine this model, with the goal of achieving a balance of covariates in the matched pairs.30 31 Covariate balance was measured using the standardised difference, where an absolute standardised difference above 10% is suggested to represent meaningful imbalance.31 32 We then matched patients who underwent testing to those who did not on the basis of a calliper width of 0.2 standard deviations of the log odds of the propensity score. This method involved sampling without replacement and has been shown to remove 98% of bias from measured covariates.33 Within the matched pairs, we used a stratified Cox proportional hazards model to compare one year survival.31 Given that the interaction term between the exposure (stress testing) and time was not statistically significant, we were satisfied that the model met the proportional hazards assumption. Other continuous and dichotomous outcomes were compared using statistical methods appropriate for paired data.31
Subgroup analyses were performed on the basis of ischaemic heart disease, surgical procedure (vascular, orthopaedic, intraperitoneal, or intrathoracic), and Revised Cardiac Risk Index (RCRI) class (0 points, 1-2 points, or 3 or more points).34 The RCRI is composed of six equally weighted clinical risk factors: ischaemic heart disease; congestive heart failure; cerebrovascular disease; diabetes; renal insufficiency; and high risk surgery (intra-abdominal, intrathoracic, or suprainguinal vascular procedures). For the subgroup analyses, we repeated the same propensity score matching process while simultaneously forcing an exact match on the subgroup characteristics. One year survival was then compared within the subgroup specific matched pairs. We used an interaction term in the Cox model to test for any subgroup effects.35 An additional subgroup analysis was performed among patients older than 66 years to describe preoperative β blocker and statin use. We used this particular subgroup because data on outpatient prescriptions are available only for individuals aged above 65 years in Ontario.
We conducted several sensitivity analyses to test the robustness of our results. Firstly, we assessed the influence of an alternative matching method on our results. The original propensity score was modified to include an estimate of unmeasured disease burden: the number of acute care hospital admissions within two years before surgery. Secondly, we used the CIHI database to determine the association of preoperative testing with an outcome where no differences would be expected, namely surgical site infections. Previous research suggests that administrative data can identify surgical site infections with reasonable accuracy.36 The purpose of this “tracer” analysis was to assess for unmeasured residual confounding. Given that surgical site infections are associated with increased patient risk37 but should be unaffected by stress testing, we hypothesised that testing would not be associated with increased rates of these complications.
Results
The study cohort consisted of 271 082 patients, of whom 23 991 (8.9%) underwent non-invasive stress testing within 180 days before surgery (tables 1 and 2). Some patients underwent more than one test; hence, a total of 25 877 stress tests were performed. The median time between testing and surgery was 48 days (interquartile range 16 to 101 days). A total of 7795 (30%) of these tests were ordered by cardiologists, 7234 (28%) by internists, 6281 (24%) by family physicians, 3594 (14%) by surgeons, 477 (1.8%) by anaesthesiologists, and 496 (1.9%) by other specialists. Of the patients who underwent preoperative testing, 914 (3.8%) underwent coronary angiography, 149 (0.6%) underwent percutaneous coronary intervention, and 134 (0.6%) underwent coronary artery bypass graft surgery between the dates of stress testing and surgery.
Table 1.
Stress testing (n=23 991) | No testing (n=247 091) | P value | |
---|---|---|---|
Demographics | |||
Female sex | 9662 (40.3) | 129 645 (52.5) | <0.001 |
Age (years; mean (SD)) | 69.1 (9.2) | 68.3 (10.3) | <0.001 |
Socioeconomic status | |||
Annual income (Canadian dollars; mean (SD)) | 24 846 (5105) | 24 693 (5067) | <0.001 |
Comorbid disease | |||
Ischaemic heart disease | 5566 (23.2) | 20 996 (8.5) | <0.001 |
Congestive heart failure | 1022 (4.3) | 6005 (2.4) | <0.001 |
Cerebrovascular disease | 1684 (7.0) | 10 206 (4.1) | <0.001 |
Hypertension | 14 887 (62.1) | 130 950 (53.0) | <0.001 |
Diabetes | 5881 (24.5) | 43 442 (17.6) | <0.001 |
Pulmonary disease | 1582 (6.6) | 12 425 (5.0) | <0.001 |
Dialysis or renal disease | 481 (2.0) | 2907 (1.2) | <0.001 |
Malignancy | 2265 (9.4%) | 21 647 (8.8) | 0.004 |
Specialist consultation* | |||
Anaesthesiology | 12 967 (54.0) | 91 749 (37.1) | <0.001 |
General internal medicine | 5758 (24.0) | 57 416 (23.2) | 0.008 |
Cardiology | 5828 (24.3) | 14 709 (6.0) | <0.001 |
Preoperative cardiac procedure† | |||
Echocardiogram | 10 855 (45.2) | 21 113 (8.5) | <0.001 |
Coronary angiogram | 1256 (5.2) | 1463 (0.6) | <0.001 |
Percutaneous coronary intervention | 314 (1.3) | 297 (0.1) | <0.001 |
Aorto-coronary bypass surgery | 302 (1.3) | 525 (0.2) | <0.001 |
*Within 60 days before surgery.
†Within 180 days before surgery.
Table 2.
Stress testing (n=23 991) | No testing (n=247 091) | P value | |
---|---|---|---|
Hospital type | |||
Teaching | 7759 (32.3) | 84 786 (34.3) | <0.001 |
High volume non-teaching | 5951 (24.8) | 51 582 (20.9) | |
Moderate volume non-teaching | 5747 (24.0) | 54 418 (22.0) | |
Low volume non-teaching | 4534 (18.9) | 56 305 (22.8) | |
Procedure | |||
Abdominal aortic aneurysm repair | 2651 (11.0) | 5280 (2.1) | <0.001 |
Carotid endarterectomy | 2001 (8.3) | 10 044 (4.1) | |
Peripheral vascular bypass | 2579 (10.7) | 13 467 (5.5) | |
Total hip replacement | 4020 (16.8) | 60 671 (24.6) | |
Total knee replacement | 5694 (23.7) | 80 771 (32.7) | |
Large bowel surgery | 3363 (14.0) | 50 497 (20.4) | |
Liver resection | 273 (1.1) | 1661 (0.7) | |
Whipple procedure | 76 (0.3) | 1030 (0.4) | |
Pneumonectomy or lobectomy | 1560 (6.5) | 8709 (3.5) | |
Gastrectomy or oesophagectomy | 623 (2.6) | 5004 (2.0) | |
Nephrectomy | 957 (4.0) | 7900 (3.2) | |
Cystectomy | 194 (0.8) | 2057 (0.8) | |
Intraoperative care | |||
Epidural anaesthesia | 7433 (31.0) | 49 593 (20.1) | <0.001 |
Arterial line | 11 160 (46.5) | 70 148 (28.3) | <0.001 |
Central venous line | 3288 (13.7) | 18 883 (7.6) | <0.001 |
Pulmonary artery catheter | 2270 (9.5) | 6743 (2.7) | <0.001 |
Patients who underwent preoperative testing and those who did not differed with regard to all measured characteristics (tables 1 and 2). Patients who underwent testing were typically men who had surgery at a high volume or moderate volume non-teaching hospital and had an increased burden of comorbid disease. They were also more likely to be evaluated by a specialist before surgery, undergo preoperative cardiac procedures, and require intraoperative care such as epidural anaesthesia or intraoperative invasive monitoring.
Of the patients who underwent stress testing, 23 060 (96%) were successfully matched to a similar patient who did not. The covariate balance between the two arms was improved considerably by propensity score matching (tables 3 and 4): the mean standardised difference between the two groups decreased from 14.9% (range 0.3 to 90.9) to 0.48% (0.02 to 1.5). Of the matched patients who underwent testing, 914 (4.0%) underwent coronary angiography, 136 (0.6%) underwent percutaneous coronary intervention, and 119 (0.5%) underwent coronary artery bypass graft surgery between the dates of stress testing and surgery.
Table 3.
Stress testing (n=23 060) | No stress testing (n=23 060) | Absolute standardised difference | ||
---|---|---|---|---|
Before matching | After matching | |||
Demographics | ||||
Female sex | 13 572 (58.9) | 13 623 (59.1) | 24.6% | 0.4% |
Age (years; mean (SD)) | 69.2 (9.2) | 69.2 (9.2) | 7.6% | <0.1% |
Socioeconomic status | ||||
Annual income (Canadian dollars; mean (SD)) | 24 837 (5104) | 24 810 (5155) | 3.0% | 0.5% |
Comorbid disease | ||||
Ischaemic heart disease | 5272 (22.9) | 5343 (23.2) | 41.1% | 0.7% |
Congestive heart failure | 1011 (4.4) | 1018 (4.4) | 10.2% | 0.1% |
Cerebrovascular disease | 1638 (7.1) | 1661 (7.2) | 12.6% | 0.4% |
Hypertension | 14 355 (62.3) | 14 381 (62.4) | 18.4% | 0.2% |
Diabetes | 5655 (24.5) | 5572 (24.2) | 17.1% | 0.8% |
Pulmonary disease | 1528 (6.6) | 1562 (6.8) | 6.7% | 0.6% |
Dialysis or renal disease | 463 (2.0) | 458 (2.0) | 6.6% | 0.2% |
Malignancy | 2185 (9.5) | 2210 (9.6) | 2.4% | 0.4% |
Specialist consultation* | ||||
Anaesthesiology | 12 299 (53.3) | 12 279 (53.2) | 34.4% | 0.2% |
General internal medicine | 5670 (24.6) | 5804 (25.2) | 1.8% | 1.3% |
Cardiology | 5084 (22.0) | 4941 (21.4) | 53.0% | 1.5% |
Preoperative cardiac procedure† | ||||
Echocardiogram | 9948 (43.1) | 9909 (43.0) | 90.9% | 0.3% |
*Within 60 days before surgery.
†Within 180 days before surgery.
Table 4.
Stress testing (n=23 060) | No stress testing (n=23 060) | Absolute standardised difference | ||
---|---|---|---|---|
Before matching | After matching | |||
Hospital type | ||||
Teaching | 7647 (33.2) | 7645 (33.2) | 4.2% | <0.1% |
High volume non-teaching | 5618 (24.4) | 5639 (24.5) | 9.4% | 0.2% |
Moderate volume non-teaching | 5432 (23.6) | 5336 (23.1) | 4.6% | 0.9% |
Low volume non-teaching | 4363 (18.9) | 4440 (19.3) | 9.6% | 0.8% |
Procedure | ||||
Abdominal aortic aneurysm repair | 2247 (9.7) | 2262 (9.8) | 36.5% | 0.2% |
Carotid endarterectomy | 1954 (8.5) | 1914 (8.3) | 17.8% | 0.6% |
Peripheral vascular bypass | 2384 (10.3) | 2393 (10.4) | 19.5% | 0.1% |
Total hip replacement | 4011 (17.4) | 4043 (17.5) | 19.4% | 0.4% |
Total knee replacement | 5632 (24.4) | 5682 (24.6) | 20.0% | 0.5% |
Large bowel surgery | 3276 (14.2) | 3282 (14.2) | 17.1% | <0.1% |
Liver resection | 272 (1.2) | 294 (1.3) | 4.9% | 0.9% |
Whipple procedure | 68 (0.3) | 57 (0.2) | 1.7% | 0.9% |
Pneumonectomy or lobectomy | 1460 (6.3) | 1431 (6.2) | 13.7% | 0.5% |
Gastrectomy or oesophagectomy | 609 (2.6) | 589 (2.6) | 3.8% | 0.5% |
Nephrectomy | 917 (4.0) | 975 (4.2) | 4.3% | 1.3% |
Cystectomy | 180 (0.8) | 188 (0.8) | 0.3% | 0.4% |
Intraoperative care | ||||
Epidural anaesthesia | 6895 (29.9) | 6842 (29.7) | 25.2% | 0.5% |
Arterial line | 10 497 (45.5) | 10 577 (45.9) | 38.1% | 0.7% |
Central venous line | 3000 (13.0) | 3009 (13.0) | 19.7% | 0.1% |
Pulmonary artery catheter | 2037 (8.8) | 2070 (9.0) | 28.4% | 0.5% |
Within the matched cohort, one year survival was higher among patients who had undergone preoperative testing than in those who had not (hazard ratio (HR) 0.92, 95% CI 0.86 to 0.99, P=0.03; fig 1). This corresponded to a number needed to treat38 to prevent mortality at one year of 221 (95% CI 111 to 16 067). In hospital mortality and hospital stay were also reduced among patients who underwent stress testing (relative risk (RR) 0.85, 95% CI 0.73 to 0.98; P=0.03 and 8.72 days v 8.96 days, difference −0.24 days, 95% CI −0.07 to −0.43; P<0.001, respectively; table 5). Patients who had undergone stress testing were more likely to be admitted to a monitored bed after surgery than were those who had not undergone testing (RR 1.09, 95% CI 1.06 to 1.12; P<0.001; table 5). Conversely, rates of postoperative mechanical ventilation were similar in the two groups (RR 1.02, 95% CI 0.98 to 1.08; P=0.25).
Table 5.
Stress testing (n=23 060) | No stress testing (n=23 060) | Difference (95% CI) | |
---|---|---|---|
Preoperative cardiac procedure* | |||
Coronary angiogram | 1139 (4.9) | 549 (2.4) | RR 2.08 (1.88 to 2.92) |
Percutaneous coronary intervention | 288 (1.2) | 132 (0.6) | RR 2.18 (1.78 to 2.68) |
Aorto-coronary bypass surgery | 267 (1.2) | 213 (0.9) | RR 1.25 (1.05 to 1.50) |
Postoperative care† | |||
Admission to a monitored bed | 6175 (26.8) | 5682 (24.6) | RR 1.09 (1.06 to 1.12) |
Mechanical ventilation | 2672 (11.6) | 2596 (11.3) | RR 1.02 (0.98 to 1.08) |
Outcome | |||
One year mortality | 1622 (7.0) | 1738 (7.5) | HR 0.92 (0.86 to 0.99) |
In-hospital postoperative death | 310 (1.3) | 366 (1.6) | RR 0.85 (0.73 to 0.98) |
Surgical site infection | 1815 (7.9) | 1807 (7.8) | RR 1.00 (0.94 to 1.07) |
Abbreviations: HR, hazard ratio; RR, relative risk.
*Within 180 days before surgery.
†Within 5 days after surgery.
In sensitivity analyses, the association of stress testing with improved one year survival was unaffected when the number of previous acute care hospital admissions within two years before the index surgery was added to the original propensity score (HR 0.92, 95% CI 0.86 to 0.99; P=0.02). Additionally, we found no statistically significant association between stress testing and surgical site infections (RR 1.00, 95% CI 0.94 to 1.07; P=0.89; table 5).
The subgroup of individuals older than 66 years consisted of 15 475 patients who underwent stress testing and 15 475 who did not. In this subgroup, 5626 (36%) of those who underwent preoperative stress testing and 3998 (26%) of those who did not were receiving β blockers (RR 1.41, 95% CI 1.36 to 1.45; P<0.001). Additionally, 1895 (12%) patients who had undergone testing were new β blocker users, compared with 827 (5.3%) who had not undergone testing (RR 2.23, 95% CI 2.12 to 2.48; P<0.001). More patients in the stress testing group were receiving statins (5000 (32%)) than were those who had not been tested (3679 (24%); RR 1.36, 95% CI 1.31 to 1.41; P<0.001). A total of 974 (6.3%) patients who had undergone testing were new statin users, compared with 539 (3.5%) of patients who had not (RR 1.81, 95% CI 1.63 to 2.00; P<0.001).
The association of stress testing with mortality was unchanged when the analyses were repeated in subgroups defined by procedure type (P=0.28 for interaction; fig 2). Conversely, the effects of testing on mortality varied with RCRI class (P=0.005) and, to a degree, ischaemic heart disease (P=0.08). Preoperative stress testing was associated with harm in low risk patients (RCRI 0 points: HR 1.35, 95% CI 1.05 to 1.74); however, it was associated with improved survival in intermediate risk patients (RCRI 1-2 points: HR 0.92, 95% CI 0.85 to 0.99) and high risk patients (RCRI 3-6 points: HR 0.80, 95% CI 0.67 to 0.97; fig 2). These differences corresponded to a number needed to treat to prevent mortality at one year of 156 for intermediate risk patients (95% CI 79 to 6127) and 38 for high risk patients (95% CI 21 to 315). Conversely, the number needed to harm in low risk patients was 179 (95% CI 97 to 1090). The relative rates of intervention use (that is, preoperative medications and cardiac procedures) were higher in low risk patients who underwent stress testing than in intermediate or high risk patients who were tested (web table A). The characteristics of the subgroups defined by RCRI class are presented in web table B.
Discussion
In this population based retrospective cohort study, non-invasive cardiac stress testing before elective intermediate to high risk non-cardiac surgery was associated with improved one year survival and reduced hospital stay. These benefits largely applied to patients who were at high risk for cardiac complications on the basis of three or more clinical risk factors. In contrast, stress testing was associated with only minor benefits for intermediate risk patients (1-2 risk factors) and with harm in low risk individuals.
Our results suggest that preoperative stress testing should be reserved for patients with clinical risk factors for cardiac complications. In high risk patients with three or more clinical risk factors, the use of preoperative stress testing was supported by an associated moderate improvement in one year survival and favourable number needed to treat. Conversely, in intermediate risk patients with one or two clinical risk factors, testing was associated with a small, albeit statistically significant, benefit. Given the small magnitude of this effect and the associated relatively large number needed to treat, routine preoperative stress testing is not justified in intermediate risk patients. Our results do, however, support the safety and potential benefits of selective testing in intermediate risk patients. Future research should therefore determine whether stress testing provides additional prognostic information in specific subgroups of intermediate risk patients; for example, in subgroups defined by risk factors that were not captured by administrative data ( such as poor functional status) or were suggested by our subgroup analyses (such as previous history of ischaemic heart disease). Our study does not support the use of preoperative stress testing in low risk patients; furthermore, the results suggest that such testing is associated with harm.
Our results are largely consistent with the position of the American College of Cardiology and American Heart Association guidelines, which emphasise stress testing specifically in individuals who are undergoing intermediate to high risk surgery and have one or more clinical risk factors (for example, ischaemic heart disease, congestive heart failure, cerebrovascular disease, diabetes, or renal insufficiency).5 These guidelines also suggest, however, that testing be restricted to individuals who concurrently have poor or unknown functional capacity. As described above, our data sources did not capture information on exercise tolerance; consequently, further research is needed to determine whether the prognostic significance of preoperative stress testing in intermediate to high risk surgical patients varies with their functional capacity.
The beneficial effects of stress testing on mortality risk in intermediate to high risk patients are likely to be the result of a reduction in perioperative cardiac complications. Prevention of such complications would be expected to improve both length of hospital stay3 and one year survival.2 By comparison, testing was not associated with any difference in the risk of surgical site infections.
Several plausible mechanisms might, in combination, explain a reduction in cardiac complications among individuals who underwent stress testing. Firstly, the results of preoperative stress testing can help guide perioperative drug therapy with β blockers or statins. β blockade across a broad spectrum of surgical patients may cause harm,9 but it has been associated with strong benefits in patients with multiple clinical risk factors,39 especially in those with inducible ischaemia.4 Thus, preoperative stress testing may help identify subgroups of patients that would benefit from β blocker therapy. Similarly, evidence of ischaemic heart disease on preoperative testing may indicate that a patient requires statin therapy, which is also associated with improved postoperative outcomes.40
Secondly, clinicians could use preoperative testing to determine which patients warrant closer postoperative surveillance. This hypothesis is supported by the higher rate of postoperative admission to monitored beds among patients who had undergone stress testing before surgery. Thirdly, stress testing can identify patients with high risk ischaemic heart disease who meet usual indications for revascularisation5 or who would benefit from avoiding surgery. Finally, clinicians might use the results of preoperative testing to determine which patients require more aggressive clinical care, such as stringent haemodynamic management.
In contrast to these benefits for intermediate to high risk patients, we found that stress testing was associated with harm in low risk patients. The increased mortality observed might be explained by the use of unnecessary, and potentially deleterious, interventions such as β blockade.9 39 It is noteworthy that the frequency of new β blocker use in low risk patients was 2.8 times higher in those who were tested than in those who were not (web table A).
Comparison with other studies
Our study warrants comparison with the DECREASE II trial,8 in which 770 patients with one or two clinical risk factors were randomly allocated to either stress testing or no testing before major vascular surgery. Although the rates of cardiac death or myocardial infarction at 30 days after surgery did not significantly differ between the two strategies, the few outcome events (n=16) and wide 95% confidence interval (odds ratio 0.28 to 2.91) suggest that the trial was underpowered to detect a plausible treatment effect. Indeed, this confidence interval is still consistent with our finding of a small benefit for intermediate risk patients.
Additionally, the design of the DECREASE II trial might have minimised its ability to detect differences between the study arms. All participants received perioperative β blockade and similar postoperative surveillance, thereby diminishing the influence of testing on clinical care. Stress testing altered subsequent management only in that patients with extensive ischaemia were considered for preoperative revascularisation, which has not been shown to improve outcomes.41 42
Strengths and limitations of study
Our study has several strengths. Previous studies evaluated the diagnostic accuracy of preoperative stress testing,43 44 whereas our study focused on the clinically relevant question of whether it influences outcomes. Additionally, the large sample size enabled us to detect small treatment effects that would have been deemed non-significant in smaller studies. The population based sample we used describes the effects of preoperative stress testing in “real world” clinical care, as opposed to in a protocol driven randomised trial. Thus, our findings can be generalised to other healthcare systems reasonably similar to that in Ontario, Canada. Finally, our study included only patients scheduled for elective intermediate to high risk surgery. Urgent or emergency procedures are unlikely to be delayed to facilitate preoperative stress testing, whereas patients undergoing low risk ambulatory surgery have a very low risk of major complications45 and are unlikely to benefit from preoperative stress testing. Our study, therefore, focused on individuals who had reasonable opportunities to undergo, and potentially benefit from, preoperative stress testing.
Our study also has several limitations. Firstly, we could not compare outcomes from different stress tests (for example, exercise treadmill testing and nuclear perfusion). The American College of Cardiology and American Heart Association guidelines recommend exercise electrocardiography testing as the test of choice, with the use of other modalities on the basis of factors such as resting electrocardiogram abnormalities, patient physique or build, and function tolerance.5 Given that these factors are not recorded in administrative databases, yet might be prognostically important, comparison on the basis of testing modality would be biased by residual confounding.
Secondly, our study was observational in design; hence, our results demonstrate an association between preoperative testing and survival, but do not prove causation. None the less, randomised trials of preoperative stress testing also have limitations. A trial of stress testing in intermediate risk patients would probably not be feasible. On the basis of the one year mortality rate of approximately 8.9% among intermediate risk patients in our study, roughly 31 100 participants would be required to detect a 10% relative risk reduction in a randomised trial (two tailed α of 0.05 and 80% power). Conversely, a randomised trial in high risk participants would also be large, but potentially feasible. On the basis of the 15.5% one year mortality rate among high risk participants in our study, approximately 4000 high risk individuals would be required to detect a 20% relative risk reduction (two tailed α of 0.05 and 80% power).
Thirdly, our data sources could not account for individuals who underwent preoperative coronary revascularisation on the basis of high risk findings on preoperative stress testing but subsequently died before their planned non-cardiac surgeries. These deaths before surgery are unlikely to affect our results significantly. In the stress testing arm of the cohort matched by propensity scores, 335 (1.1%) individuals subsequently underwent either percutaneous coronary intervention or coronary artery bypass graft surgery before the planned non-cardiac procedures. If the mortality rate associated with preoperative coronary revascularisation is assumed to be 2%,41 42 46 seven individuals may have undergone stress testing and revascularisation but died before their planned non-cardiac surgeries. These individuals would have been missing from our matched cohort, which included only patients who actually underwent non-cardiac surgery. None the less, even if these seven missing deaths were included in our analyses, the proportion of patients dead at one year in the stress testing arm of the matched cohort would have changed only from 7.03% to 7.06%. The corresponding difference in absolute risk between the stress testing and no testing arms would also have changed negligibly, from 0.50% to 0.47%.
Finally, the administrative healthcare databases used as data sources in our study have inherent limitations. The data in such databases do not adequately capture many postoperative complications,47 causes of death, detailed clinical information, and some processes of care (for example, inpatient medications). Such information may have helped to better describe how testing might alter outcomes. Future research on preoperative stress testing should therefore include more detailed measurements of such variables (for example, stress test results and in-hospital medications). Additionally, these administrative data were not originally collected for research purposes and may therefore be susceptible to random miscoding. However, the data are very accurate with respect to describing patient demographics,15 hospital stay,12 15 mortality,12 15 procedures,12 15 and physician services.48 Given that information on medical comorbidities can be of variable accuracy, we used definitions with generally high specificity and moderate to good sensitivity.10 11 12 14 We further improved the sensitivity of the information by using hospital admissions information from the two years preceding surgery.13 Despite these limitations, administrative data also offer important advantages of very large sample sizes, systematic data collection, and population based information free from referral bias.48
Conclusions
Non-invasive cardiac stress testing before elective intermediate to high risk non-cardiac surgery is associated with improved one year survival and reduced hospital stay. These benefits principally apply to patients at high risk for cardiac complications on the basis of clinical risk factors. Our results are generally supportive of the current American College of Cardiology and American Heart Association guidelines for preoperative cardiac evaluation.
What is already known on this topic
Non-invasive cardiac stress testing can help risk stratify surgical patients for perioperative cardiac complications and thereby better inform clinical care
Current consensus based guidelines recommend non-invasive stress testing before surgery, but only in individuals with clinical risk factors for cardiac complications
The effect of preoperative stress testing on postoperative clinical outcomes is unclear
What this study adds
Preoperative stress testing was associated with reduced one year mortality, hospital stay, and hospital mortality
These benefits largely applied to patients who were at high risk for cardiac complications on the basis of three or more clinical risk factors
In contrast, stress testing was associated with only minor benefits for intermediate risk patients (one or two risk factors) and with harm in low risk individuals
Funding: DNW is supported by a Clinician-Scientist Award from the Canadian Institutes of Health Research and a Merit Award from the Department of Anesthesia at the University of Toronto. WSB is the R Fraser Elliot Chair of Cardiac Anesthesia at the University Health Network. PCA is supported by a Career Investigator Award from the Heart and Stroke Foundation of Ontario. This study was supported in part by the Institute for Clinical Evaluative Sciences, which is itself supported in part by the Ontario Ministry of Health and Long-Term Care. The study sponsor had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The opinions, results, and conclusions are those of the authors, and no endorsement by the Ontario Ministry of Health and Long-Term Care or the Institute for Clinical Evaluative Sciences is intended or should be inferred.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare (1) No financial support for the submitted work from anyone other than their employer; (2) No financial relationships with commercial entities that might have an interest in the submitted work; (3) No spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; (4) No non-financial interests that may be relevant to the submitted work.
Ethical approval: Research ethics approval was obtained from Sunnybrook Health Sciences Centre.
Data sharing: No additional data available.
Cite this as: BMJ 2010;340:b5526
Web Extra. Extra material supplied by the author
References
- 1.Devereaux PJ, Goldman L, Cook DJ, Gilbert K, Leslie K, Guyatt GH. Perioperative cardiac events in patients undergoing noncardiac surgery: a review of the magnitude of the problem, the pathophysiology of the events and methods to estimate and communicate risk. CMAJ 2005;173:627-34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Landesberg G, Shatz V, Akopnik I, Wolf YG, Mayer M, Berlatzky Y, et al. Association of cardiac troponin, CK-MB, and postoperative myocardial ischemia with long-term survival after major vascular surgery. J Am Coll Cardiol 2003;42:1547-54. [DOI] [PubMed] [Google Scholar]
- 3.Fleischmann KE, Goldman L, Young B, Lee TH. Association between cardiac and noncardiac complications in patients undergoing noncardiac surgery: outcomes and effects on length of stay. Am J Med 2003;115:515-20. [DOI] [PubMed] [Google Scholar]
- 4.Poldermans D, Boersma E, Bax JJ, Thomson IR, van de Ven LLM, Blankensteijn JD, et al. The effect of bisoprolol on perioperative mortality and myocardial infarction in high-risk patients undergoing vascular surgery. N Engl J Med 1999;341:1789-94. [DOI] [PubMed] [Google Scholar]
- 5.Fleisher LA, Beckman JA, Brown KA, Calkins H, Chaikof E, Fleischmann KE, et al. ACC/AHA 2007 guidelines on perioperative cardiovascular evaluation and care for noncardiac surgery: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines on Perioperative Cardiovascular Evaluation for Noncardiac Surgery): developed in collaboration with the American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Rhythm Society, Society of Cardiovascular Anesthesiologists, Society for Cardiovascular Angiography and Interventions, Society for Vascular Medicine and Biology, and Society for Vascular Surgery. Circulation 2007;116:e418-99. [DOI] [PubMed] [Google Scholar]
- 6.Grayburn PA, Hillis LD. Cardiac events in patients undergoing noncardiac surgery: shifting the paradigm from noninvasive risk stratification to therapy. Ann Intern Med 2003;138:506-11. [DOI] [PubMed] [Google Scholar]
- 7.Monahan TS, Shrikhande GV, Pomposelli FB, Skillman JJ, Campbell DR, Scovell SD, et al. Preoperative cardiac evaluation does not improve or predict perioperative or late survival in asymptomatic diabetic patients undergoing elective infrainguinal arterial reconstruction. J Vasc Surg 2005;41:38-45. [DOI] [PubMed] [Google Scholar]
- 8.Poldermans D, Bax JJ, Schouten O, Neskovic AN, Paelinck B, Rocci G, et al. Should major vascular surgery be delayed because of preoperative cardiac testing in intermediate-risk patients receiving beta-blocker therapy with tight heart rate control? J Am Coll Cardiol 2006;48:964-9. [DOI] [PubMed] [Google Scholar]
- 9.Devereaux PJ, Yang H, Yusuf S, Guyatt G, Leslie K, Villar JC, et al. Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. Lancet 2008;371:1839-47. [DOI] [PubMed] [Google Scholar]
- 10.Austin PC, Daly PA, Tu JV. A multicenter study of the coding accuracy of hospital discharge administrative data for patients admitted to cardiac care units in Ontario. Am Heart J 2002;144:290-6. [DOI] [PubMed] [Google Scholar]
- 11.Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care 2002;25:512-6. [DOI] [PubMed] [Google Scholar]
- 12.Juurlink D, Preya C, Croxford R, Chong A, Austin P, Tu J, et al. Canadian Institute for Health Information Discharge Abstract Database: A validation study. ICES investigative report. Institute for Clinical Evaluative Sciences, 2006. http://www.ices.on.ca/file/CIHI_DAD_Reabstractors_study.pdf.
- 13.Lee DS, Donovan L, Austin PC, Gong Y, Liu PP, Rouleau JL, et al. Comparison of coding of heart failure and comorbidities in administrative and clinical data for use in outcomes research. Med Care 2005;43:182-8. [DOI] [PubMed] [Google Scholar]
- 14.Tu K, Campbell NRC, Chen ZL, Cauch-Dudek KJ, McAlister FA. Accuracy of administrative databases in identifying patients with hypertension. Open Med 2007;1:E3-E5. [PMC free article] [PubMed] [Google Scholar]
- 15.Williams JI, Young W. Appendix: A summary of studies on the quality of health careadministrative databases in Canada. In: Goel V, Williams JI, Anderson GM, Blackstein-Hirsch P, Fooks C, Naylor CD, eds. Patterns of health care in Ontario. The ICES practice atlas. Canadian Medical Association, 1996. http://www.ices.on.ca/file/Practice2-appendix.pdf.
- 16.Wijeysundera DN, Austin PC, Beattie WS, Hux JE, Laupacis A. A population-based study of anesthesia consultation before major non-cardic surgery. Arch Intern Med 2009;169:595-602. [DOI] [PubMed] [Google Scholar]
- 17.Technical supplement: Health care in Canada 2005. Canadian Institute for Health Information, 2005. http://secure.cihi.ca/cihiweb/products/HCIC_Tech_Report_2005_e.pdf.
- 18.Basinski ASH. Methods appendix—Chapter 7: Procedures for abdominal aortic aneurysm and peripheral vascular disease. In: Naylor CD, Slaughter PM, eds. Cardiovascular health andsServices in Ontario: An ICES atlas. Institute for Clinical Evaluative Sciences, 1999. http://www.ices.on.ca/file/TechMeth_Appendix3.pdf.
- 19.Bourne RB, DeBoer D, Hawker G, Kreder H, Mahomed N, Paterson JM, et al. Chapter 5: Total hip and knee replacement. In: Tu JV, Pinfold SP, McColgan P, Laupacis A, eds. Access to health service in Ontario: ICES atlas. Institute for Clinical Evaluative Sciences, 2005. http://www.ices.on.ca/file/Chp5_v5.pdf.
- 20.Gentleman JF, Vayda E, Parsons GF, Walsh MN. Surgical rates in subprovincial areas across Canada: rankings of 39 procedures in order of variation. Can J Surg 1996;39:361-7. [PMC free article] [PubMed] [Google Scholar]
- 21.Hayter CR, Paszat LF, Groome PA, Schulze K, Mackillop WJ. The management and outcome of bladder carcinoma in Ontario, 1982-1994. Cancer 2000;89:142-51. [PubMed] [Google Scholar]
- 22.Alter DA, Stukel TA, Newman A. Proliferation of cardiac technology in Canada: a challenge to the sustainability of Medicare. Circulation 2006;113:380-7. [DOI] [PubMed] [Google Scholar]
- 23.Lin GA, Dudley RA, Lucas FL, Malenka DJ, Vittinghoff E, Redberg RF. Frequency of stress testing to document ischemia prior to elective percutaneous coronary intervention. JAMA 2008;300:1765-73. [DOI] [PubMed] [Google Scholar]
- 24.Bondy SJ, Jaglal S, Slaughter P. Methods appendix—Chapter 3: Area variation in heart disease mortality rates. In: Naylor CD, Slaughter PM, eds. Cardiovascular health and services in Ontario: An ICES atlas. Institute for Clinical Evaluative Sciences, 1999. http://www.ices.on.ca/file/TechMeth_Appendix3.pdf.
- 25.Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005;43:1130-9. [DOI] [PubMed] [Google Scholar]
- 26.Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data. J Clin Epidemiol 2004;57:366-72. [DOI] [PubMed] [Google Scholar]
- 27.Joffe MM, Rosenbaum PR. Invited commentary: propensity scores. Am J Epidemiol 1999;150:327-33. [DOI] [PubMed] [Google Scholar]
- 28.Rubin DB. The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat Med 2007;26:20-36. [DOI] [PubMed] [Google Scholar]
- 29.Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, et al. Hospital volume and surgical mortality in the United States. N Engl J Med 2002;346:1128-37. [DOI] [PubMed] [Google Scholar]
- 30.Austin PC, Grootendorst P, Anderson GM. A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med 2007;26:734-53. [DOI] [PubMed] [Google Scholar]
- 31.Austin PC. Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: A systematic review and suggestions for improvement. J Thorac Cardiovasc Surg 2007;134:1128-35. [DOI] [PubMed] [Google Scholar]
- 32.Austin PC, Mamdani MM. A comparison of propensity score methods: a case-study estimating the effectiveness of post-AMI statin use. Stat Med 2006;25:2084-106. [DOI] [PubMed] [Google Scholar]
- 33.Rosenbaum PR, Rubin DB. Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am Stat 1985;39:33-8. [Google Scholar]
- 34.Lee TH, Marcantonio ER, Mangione CM, Thomas EJ, Polanczyk CA, Cook EF, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation 1999;100:1043-9. [DOI] [PubMed] [Google Scholar]
- 35.Rothwell PM. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet 2005;365:176-86. [DOI] [PubMed] [Google Scholar]
- 36.Yokoe DS, Noskin GA, Cunnigham SM, Zuccotti G, Plaskett T, Fraser VJ, et al. Enhanced identification of postoperative infections among inpatients. Emerging Infect Dis 2004;10:1924-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Daneman N, Simor AE, Redelmeier DA. Validation of a modified version of the national nosocomial infections surveillance system risk index for health services research. Infect Control Hosp Epidemiol 2009;30:563-9. [DOI] [PubMed] [Google Scholar]
- 38.Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988;318:1728-33. [DOI] [PubMed] [Google Scholar]
- 39.Lindenauer PK, Pekow P, Wang K, Mamidi DK, Gutierrez B, Benjamin EM. Perioperative beta-blocker therapy and mortality after major noncardiac surgery. N Engl J Med 2005;353:349-61. [DOI] [PubMed] [Google Scholar]
- 40.Kapoor AS, Kanji H, Buckingham J, Devereaux PJ, McAlister FA. Strength of evidence for perioperative use of statins to reduce cardiovascular risk: systematic review of controlled studies. BMJ 2006;333:1149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.McFalls EO, Ward HB, Moritz TE, Goldman S, Krupski WC, Littooy F, et al. Coronary-artery revascularization before elective major vascular surgery. N Engl J Med 2004;351:2795-804. [DOI] [PubMed] [Google Scholar]
- 42.Poldermans D, Schouten O, Vidakovic R, Bax JJ, Thomson IR, Hoeks SE, et al. A clinical randomized trial to evaluate the safety of a noninvasive approach in high-risk patients undergoing major vascular surgery: the DECREASE-V pilot study. J Am Coll Cardiol 2007;49:1763-9. [DOI] [PubMed] [Google Scholar]
- 43.Beattie WS, Abdelnaem E, Wijeysundera DN, Buckley DN. A meta-analytic comparison of preoperative stress echocardiography and nuclear scintigraphy imaging. Anesth Analg 2006;102:8-16. [DOI] [PubMed] [Google Scholar]
- 44.Kertai MD, Boersma E, Bax JJ, Heijenbrok-Kal MH, Hunink MGM, L’talien GJ, et al. A meta-analysis comparing the prognostic accuracy of six diagnostic tests for predicting perioperative cardiac risk in patients undergoing major vascular surgery. Heart 2003;89:1327-34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Warner MA, Shields SE, Chute CG. Major morbidity and mortality within 1 month of ambulatory surgery and anesthesia. JAMA 1993;270:1437-41. [DOI] [PubMed] [Google Scholar]
- 46.Monaco M, Stassano P, Di Tommaso L, Pepino P, Giordano A, Pinna GB, et al. Systematic strategy of prophylactic coronary angiography improves long-term outcome after major vascular surgery in medium- to high-risk patients: a prospective, randomized study. J Am Coll Cardiol 2009;54:989-96. [DOI] [PubMed] [Google Scholar]
- 47.Romano PS, Schembri ME, Rainwater JA. Can administrative data be used to ascertain clinically significant postoperative complications? Am J Med Qual 2002;17:145-54. [DOI] [PubMed] [Google Scholar]
- 48.Scales DC, Guan J, Martin CM, Redelmeier DA. Administrative data accurately identified intensive care unit admissions in Ontario. J Clin Epidemiol 2006;59:802-7. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.