Abstract
Background
Concern regarding overutilization of cardiac imaging has led to the development of appropriate use criteria (AUC). Myocardial perfusion imaging (MPI) is one of the most commonly used cardiac imaging modalities worldwide. Despite multiple iterations of AUC, there is currently no evidence regarding their real‐world impact on population‐based utilization rates of MPI. Our goal was to assess the impact of the AUC on rates of MPI in Ontario, Canada. We hypothesized that publication of the AUC would be associated with a significant reduction in MPI rates.
Methods and Results
We conducted a retrospective cohort study of the adult population of Ontario from January 1, 2000, to December 31, 2015. Age‐ and sex‐standardized rates were compared from 4 different periods intersected by 3 published iterations of the AUC. Overall, 3 072 611 MPI scans were performed in Ontario during our study period. The mean monthly rate increased from 14.1/10 000 in the period from January 2000 to October 2005 to 18.2/10 000 between November 2005 and June 2009. After this point in time, there was a reduction in rates, falling to a mean monthly rate of 17.1/10 000 between March 2014 and December 2015. Time series analysis revealed that publication of the 2009 AUC was associated with a significant reduction in MPI rates (P<0.001). This translated into ≈88 849 fewer MPI scans at a cost savings of ≈72 million Canadian dollars.
Conclusions
Our results reflect a potential real‐world impact of the 2009 MPI AUC by demonstrating evidence of a significant effect on population‐based rates of MPI.
Keywords: appropriateness criteria, health services research, nuclear medicine, population studies
Subject Categories: Imaging, Nuclear Cardiology and PET, Health Services, Ethics and Policy, Quality and Outcomes
Clinical Perspective
What Is New?
Concern regarding overutilization of cardiac imaging has led to the development of AUC.
Myocardial perfusion imaging (MPI) is a commonly utilized cardiac imaging modality.
Our time series analysis of the adult population of Ontario, Canada (≈10.7 million population) revealed that publication of the 2009 AUC was associated with a significant reduction in MPI rates.
This translated into ≈88 849 fewer MPI scans performed at a cost savings of ≈72 million Canadian dollars.
What Are the Clinical Implications?
Our results reflect a potential real‐world impact of the 2009 MPI AUC by demonstrating evidence of a significant effect on population‐based rates of MPI.
Our study is the first to report such an impact of the AUC on population‐based utilization rates of cardiac imaging tests.
We hope that studies such as this will provide the impetus for further consideration of AUC in future funding and health policy decisions in Ontario and beyond.
Introduction
Rapid growth in healthcare costs has provided the impetus for understanding utilization patterns of diagnostic tests. Specifically, the costs of medical imaging have risen sharply in recent years.1, 2, 3, 4, 5 Myocardial perfusion imaging (MPI) with single photon emission computed tomography is a commonly utilized cardiac imaging modality. In fact, in 2002, ≈9.3 million such tests were performed in the Unites States.6 Concern about the potential overutilization of noninvasive cardiac imaging has led to the development of health policy initiatives such as appropriate use criteria (AUC) publications geared to curb utilization.7, 8, 9, 10, 11 AUC have been published on MPI exclusively on 2 occasions (2005 and 2009) and as part of a multimodality AUC publication in 2014.12, 13, 14 Despite the multiple iterations of the AUC, the impact of these publications on physician behavior as reflected by population‐based rates of MPI utilization is unknown. The goal of our study was to describe the utilization of MPI in Ontario over the past 15 years and to assess the impact of publication of the AUC on population‐based rates of MPI utilization. We hypothesized that AUC publication would be associated with a reduction in MPI utilization.
Methods
Design
We conducted a retrospective cohort study of the population of Ontario aged ≥20 years from January 1, 2000, until December 31, 2015.
Data Sources
Information regarding receipt of MPIs and graded exercise stress tests (GXTs) was obtained through medical claims data from the Ontario Health Insurance Plan (OHIP) Physicians Claim Database. Given that MPIs may be conducted over one or more consecutive days, we applied a 2‐day window on either side of the date of an MPI claim to avoid duplicate counting. The diagnostic codes used to identify the MPI studies and our methods to avoid duplicate counting have been validated previously and used in prior studies.15, 16, 17 The Registered Persons Database was used to obtain patient demographic information, including age and sex. Patients’ postal codes of residence were also obtained from the Registered Persons Database. Postal codes were linked to the Census Area Profile using the Postal Code Conversion File to obtain median neighborhood income in a manner described previously.15, 16, 17 Hospitalizations and receipt of invasive coronary angiography were determined using the Canadian Institute for Health Information Discharge Abstract Database (CIHI‐DAD). The CIHI‐DAD is an administrative database containing summary data of discharge abstracts of every inpatient hospital admission in Ontario.18, 19 Ontario has universal insurance for all residents of the province and a single‐payer system. Thus, these databases record all of the health service use for the current study.
Outcome
The primary outcome was the age‐ and sex‐standardized monthly rate of MPI scans per 10 000 adults in Ontario, Canada. To provide context, we also calculated the age‐ and sex‐standardized rates of GXT and invasive angiography as secondary outcome measures.
Interventions
There were 3 interventions representing the publication of the 3 iterations of the AUC: (1) October 2005, (2) June 2009, and (3) February 2014.
Costs
Case costs were estimated using the OHIP schedule of benefits and included both professional and technical fees that were billed to OHIP for each MPI scan.20, 21 The total fee billed to OHIP per MPI scan was ≈811 Canadian dollars.
Statistical Analysis
Population‐based rates of MPI, GXT, and invasive angiography were calculated for each year by dividing the number of diagnostic tests by the corresponding adult population of Ontario and were then estimated per 10 000 individuals. Age‐ and sex‐standardized rates were obtained through direct standardization using the 2006 Ontario population as the standard population, as described previously.15, 16 Characteristics of patients receiving MPI scans were compared over time using the χ2 test for categorical variables and ANOVA for continuous variables.
An autoregressive integrated moving average (ARIMA) model was used to compare mean monthly utilization rates before and after publication of the guidelines (ie, the interventions). The impact of these interventions were assessed after accounting for seasonality (if present), background trends, and autocorrelation. Linear spline functions with knots at the dates of publications of the AUC were incorporated into this model.22, 23 The ARIMA model accounted for existing temporal trends in the use of MPI. Existing temporal trends in use of MPI could be due to temporal changes in patient characteristics, changes in the utilization of other noninvasive diagnostic tests, and changes in pharmacological therapy for coronary artery disease. Accounting for existing temporal trends permits one to isolate the effect of publication of the AUC from existing temporal trends.
To graphically represent the overall trend and any change in the rates before and after the intervention, we estimated a smoothing line using robust locally weighted regression (LOESS).24, 25 Reductions in numbers of MPI scans as a potential result of publication of the AUC were estimated by multiplying the change in monthly rate between the periods of any statistically significant intervention by the yearly population of Ontario aged ≥20 years. That number was then multiplied by 12 to obtain an annual estimate. In years when the statistically significant intervention occurred, we accounted for only those months that occurred after publication. Specifically, for the June 2009 intervention, the annual number was divided by 2 to account for the ≈6 months that followed AUC publication in that year. Potential cost savings were then determined by multiplying the fees billed to OHIP per scan by the number of scans potentially reduced by the intervention. Analyses were performed with SAS version 9.3. This study was approved by the research ethics board at Sunnybrook Health Sciences Center.
Results
Derivation of the Study Population
Figure 1 displays the derivation of the study population. There were 3 257 454 OHIP claims for MPI studies between the calendar years 2000 and 2015. Of these, 154 072 were excluded due to multiple billings within 2 days of the initial claim. A further 30 771 claims for patients aged <20 years were also excluded. The final sample consisted of 3 072 611 MPI scans performed in Ontario during our study period.
Figure 1.
Derivation of the study cohort. MPI indicates myocardial perfusion imaging; OHIP, Ontario Health Insurance Plan.
Characteristics of Patients Undergoing MPI
Table 1 summarizes the characteristics of patients undergoing MPI and compares these characteristics in the 4 periods straddling the 3 interventions. Over time, MPI scans were performed on older patients, on a greater proportion of female patients, and on increasingly sicker patents, as measured by their hospital admission rate within 30 days of the test (overall P value <0.001 for all comparisons).
Table 1.
Characteristics of Patients Undergoing MPI in Ontario, Canada, 2000–2015
| January 1, 2000–October 18, 2005 | October 19, 2005–June 9, 2009 | June 10, 2009–February 4, 2014 | February 4, 2014–December 31, 2015 | Total | P Value | |
|---|---|---|---|---|---|---|
| n | 827 594 | 749 564 | 1 051 647 | 443 806 | 3 072 611 | |
| Age (mean±SD) | 60.1±13.2 | 61.4±13.4 | 62.9±13.5 | 64.2±13.4 | 62.0±13.5 | <0.001 |
| Female sex | 395 092 (47.7%) | 363 449 (48.5%) | 526 269 (50.0%) | 221 461 (49.9%) | 1 506 271 (49.0%) | <0.001 |
| Income quintile | <0.001 | |||||
| 1 (lowest) | 161 219 (19.5%) | 148 974 (19.9%) | 205 170 (19.5%) | 85 575 (19.3%) | 600 938 (19.6%) | |
| 2 | 168 052 (20.3%) | 153 797 (20.5%) | 214 605 (20.4%) | 89 562 (20.2%) | 626 016 (20.4%) | |
| 3 | 163 050 (19.7%) | 147 027 (19.6%) | 210 107 (20.0%) | 89 318 (20.1%) | 609 502 (19.8%) | |
| 4 | 161 046 (19.5%) | 147 933 (19.7%) | 214 391 (20.4%) | 91 793 (20.7%) | 615 163 (20.0%) | |
| 5 (highest) | 171 503 (20.7%) | 149 353 (19.9%) | 203 194 (19.3%) | 85 431 (19.2%) | 609 481 (19.8%) | |
| Self‐referral | 65 514 (7.9%) | 50 532 (6.7%) | 73 224 (7.0%) | 35 560 (8.0%) | 224 830 (7.3%) | <0.001 |
| Diabetes mellitus | 141 485 (17.1%) | 165 320 (22.1%) | 265 894 (25.3%) | 120 138 (27.1%) | 692 837 (22.5%) | <0.001 |
| Hypertension | 310 907 (37.6%) | 295 978 (39.5%) | 403 867 (38.4%) | 163 680 (36.9%) | 1 174 432 (38.2%) | <0.001 |
| Hospital admission within 30 d | 34 644 (4.2%) | 35 200 (4.7%) | 53 376 (5.1%) | 23 926 (5.4%) | 147 146 (4.8%) | <0.001 |
Age‐ and Sex‐Standardized Rates of MPI Between 2000 and 2015
MPI rates increased steadily from ≈2000 to 2009 (see Figure 2). The mean age‐ and sex‐standardized monthly rate between January 2000 and October 2005 was 14.1/10 000, and this increased to 18.2/10 000 in the November 2005 to June 2009 period. After this point in time, there was a reduction in rates, falling to a mean monthly rate of 17.1/10 000 in the time period from March 2014 to December 2015.
Figure 2.
Mean monthly age‐ and sex‐standardized rates of myocardial perfusion imaging (MPI) in Ontario; 2000–2015. The red vertical lines indicate the interventions (ie, publication of the appropriate use criteria). The blue dotted line indicates the LOESS smoothing curve.
Age ‐and Sex‐Standardized Rates of GXT Between 2000 and 2015
Mean monthly age‐ and sex‐standardized GXT rates increased from 26.3/10 000 in the January 2000 to October 2005 period to 29.2/10 000 in the November 2005 to June 2009 period. After this point in time, there was a reduction in rates to 28.3/10 000 in the time period from March 2014 to December 2015 (see Figure 3A).
Figure 3.
A, Mean monthly age‐ and sex‐standardized rates of graded exercise stress tests (GXTs) in Ontario, 2000–2015. The red vertical lines indicate the interventions (ie, publication of the appropriate use criteria [AUC]). The blue dotted line indicates the LOESS smoothing curve. B, Mean monthly age‐ and sex‐standardized rates of invasive angiography in Ontario, 2000–2015. The red vertical lines indicate the interventions (ie, publication of the AUC). The blue dotted line indicates the LOESS smoothing curve.
Age‐ and Sex‐Standardized Rates of Invasive Angiography Between 2000 and 2015
Mean monthly age and sex‐standardized rates of invasive angiography increased from 4.8/10 000 in the January 2000 to October 2005 period to 5.4/10 000 in the June 2009 to February 2014 period. After this point in time, there was a slight decline to 5.1/10 000 in the time period from March 2014 to December 2015 (see Figure 3B).
The Impact of AUC Publication on Rates of MPI Utilization
In the time series analysis, we found that the second intervention (publication of the 2009 AUC) was associated with a significant reduction in MPI rates after accounting for the background trend, seasonality, and autocorrelation (P<0.001). In contrast, publication of the first (P=0.69) and third (P=0.76) AUC interventions were not associated with a significant reduction in the rate of MPI utilization. Furthermore, we found that the second intervention was also associated with a significant reduction in GXT rates (P<0.001); however, it was not associated with a significant reduction or increase in invasive angiography rates (P=0.22).
Estimation of the Reduction in MPI Scans and Potential Associated Cost Savings Resulting From Publication of the 2009 AUC
The reduced utilization rate translated into ≈88 849 fewer MPI scans performed after publication of the 2009 AUC. In addition, total cost reductions were estimated at 72 056 539 Canadian dollars during that time frame (see Table 2).
Table 2.
Potential Numbers of MPI Scans Reduced and Associated Estimated Cost Savings as a Result of Publication of the 2009 AUC
| 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | Total | |
|---|---|---|---|---|---|---|---|---|
| Population of Ontario aged ≥20 y | 9 884 042 | 10 026 236 | 10 160 346 | 10 322 805 | 10 480 945 | 10 623 748 | 10 753 497 | |
| Number of MPIs potentially reduced per y as a result of publication of the 2009 AUC | 6 523a | 13 235 | 13 412 | 13 626 | 13 835 | 14 023 | 14 195 | 88 849 |
| Potential cost savings per y (Canadian dollars) | 5 290 153 | 10 733 585 | 10 877 132 | 11 050 686 | 11 220 185 | 11 372 653 | 11 512 145 | 72 056 539 |
AUC indicates appropriate use criteria; MPI, myocardial perfusion imaging.
Reflects estimates for 6 months after publication of the AUC (July–December 2009).
Discussion
Over time, MPI scans were performed on patients who were older, were more likely to be female, and were sicker overall, as measured by an increasing frequency of admission to hospital within 30 days of the test. Age‐ and sex‐standardized utilization rates of MPI increased in Ontario between 2000 and ≈2009. After 2009, rates began to decrease. Publication of the 2009 AUC was associated with a significant decrease in the population‐based rates of MPI in Ontario. This translated into ≈88 849 fewer MPI scans at a cost savings of ≈72 million Canadian dollars.
There is limited available evidence on the impact of AUC on utilization and physician behavior. A recent population‐based study reported that the publication of AUC for invasive coronary angiography was associated with a decreasing trend in the utilization of diagnostic angiography for the diagnosis of coronary artery disease.26 There have been mixed reports with regard to the efficacy of MPI‐based AUC. After publication of the 2005 MPI AUC guidelines, Mehta et al reported that although the majority of studies performed at a single large academic center were appropriate, a large proportion of ordered MPI studies were categorized as uncertain, inappropriate, or no category.27 Similarly, an American quality improvement project, focused on feedback, physician education, and remeasurement, did not reduce the rate of inappropriate stress MPI studies in a single academic medical center.28 A recent metaregression analysis involving 59 studies and 103 567 tests published from 2000 to 2012 reported no significant difference in the rate of appropriate MPI scans before and after publication of the AUC.29 In contrast, Gibbons et al reported a significant decrease in the proportion of inappropriate MPI scans performed in a single academic center after publications of the 2009 AUC.30 Furthermore, a recent budget impact analysis showed that application of the 2009 AUC could result in savings of 18.6% in a single tertiary care hospital in Brazil.31 To our knowledge, no studies have assessed the impact of the AUC on utilization rates of noninvasive cardiac diagnostic testing (including MPI) at the population level. This is a relevant metric to examine the potential impact of the AUC, given that publication of the AUC was driven, in part, by the need to control the rapid expansion of imaging utilization.
Our results provide evidence to suggest a significant effect of the 2009 AUC publication on population‐based MPI rates in Ontario, Canada. In contrast, publication of the 2005 AUC was not associated with a significant change in rates. This is consistent with previous work documenting that adherence to AUC among clinicians may gradually increase over time.32, 33 We speculate that although the 2005 AUC may have introduced the concept of appropriateness to most clinicians, the 2009 iteration may have cemented the importance of these criteria, ultimately culminating in the effect that we observed. In addition, there was no significant impact of the 2014 multimodality criteria on MPI rates. In the observed data, rates continued to decline after publication of these guidelines; however, after accounting for trend and seasonality, publication of the criteria themselves were not associated with a significant change in the rate of change of MPI rates. A potential explanation may lie in the fact that these were multimodality AUC not addressing MPI exclusively. Thus, the effect on MPI may not have been as profound as modality‐specific AUC. Furthermore, our results indicate that it is unlikely that MPI scans were simply replaced by another test. Up until 2011, billing information was only available for GXT and MPI in Ontario. Our analysis did not show a significant increase in rates of GXT coinciding with a reduction in MPI rates. Moreover, previous published work by our group reported that there was no increase in total noninvasive testing rates (including GXT, MPI, stress echocardiography, and computed tomography angiography) between 2011 and 2014.16
Our findings of a significant impact of the 2009 AUC are important. We are the first group to examine the relationship between AUC and rates of MPI utilization in the population. As such, our results may be meaningful from a health policy perspective by providing evidence of the real‐world impact of the AUC. Currently, AUC are not factored into funding decisions by the OHIP. Despite this, there is growing awareness of the need to limit unnecessary testing in Ontario and in Canada. We hope that studies such as this will provide the impetus for further consideration of AUC in future funding and health policy decisions in Ontario and beyond.
Limitations
Our study needs to be interpreted in the context of its limitations. First, we did not have clinical information regarding the appropriateness of tests. As such, we cannot comment on changes in inappropriate MPI utilization. Second, this is a population‐based study and our observations were at the population and not the individual level; therefore, our results are ecological in nature. Third, we did not specifically adjust for potential confounders such as changing patient demographics and risk factor profiles; however, our ARIMA model incorporated such factors into the background trends and tests for an impact on an intervention (in our case, publication of the AUC) above and beyond these background trends. Fourth, billing codes for newer noninvasive diagnostic tests, namely, stress echocardiography and computed tomography angiography, only became available in Ontario in 2011. Consequently, we were unable to track utilization of these tests during the time of the observed significant reduction in MPI rates to determine if MPIs were being replaced by either of these tests. With that said, a previously published study from our group reported that between 2011 and 2014 (ie, when data for all 4 tests were available), the vast majority of tests performed in Ontario were still GXTs and MPIs, accounting for ≈80% to 85% of all noninvasive diagnostic tests performed. Furthermore, these data indicated that there was a slight reduction in the overall testing rate during that time16; therefore, it is unlikely that the reduction in MPI was caused by replacement with other testing modalities. Fifth, our study did not assess the impact of test‐specific AUC on the rates of other important noninvasive diagnostic tests such as GXT, stress echocardiography, and computed tomography angiography. Future work targeting each of these modalities is needed to assess the impact of test‐specific AUC publication on the rates of utilization of each individual testing modality. Last, we report results from Ontario, Canada. These results may not be representative of other jurisdictions.
Conclusions
Publication of the 2009 AUC was associated with a significant reduction in the MPI utilization rates in Ontario after accounting for background trend and seasonality. This reduced rate translated into ≈88 849 fewer MPI scans performed at a cost savings of ≈72 million Canadian dollars. These population‐based results illustrate a potential real‐world impact of the MPI AUC.
Sources of Funding
This article was funded by operating funds from Schulich Heart Program and the Sunnybrook Research Institute. This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long‐Term Care (MOHLTC). The funding organizations did not have any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.
Disclosures
None.
(J Am Heart Assoc. 2017;6:e005961 DOI: 10.1161/JAHA.117.005961.)28584072
References
- 1. Douglas PS, Taylor A, Bild D, Bonow R, Greenland P, Lauer M, Peacock F, Udelson J. Outcomes research in cardiovascular imaging: report of a workshop sponsored by the National Heart, Lung, and Blood Institute. Circ Cardiovasc Imaging. 2009;2:339–48. [DOI] [PubMed] [Google Scholar]
- 2. Douglas PS, Taylor A, Bild D, Bonow R, Greenland P, Lauer M, Peacock F, Udelson J. Outcomes research in cardiovascular imaging: report of a workshop sponsored by the National Heart, Lung, and Blood Institute. JACC Cardiovasc Imaging. 2009;2:897–907. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Hendel RC. Utilization management of cardiovascular imaging pre‐certification and appropriateness. JACC Cardiovasc Imaging. 2008;1:241–8. [DOI] [PubMed] [Google Scholar]
- 4. Iglehart JK. The new era of medical imaging–progress and pitfalls. New Engl J Med. 2006;354:2822–8. [DOI] [PubMed] [Google Scholar]
- 5. Iglehart JK. Health insurers and medical‐imaging policy–a work in progress. New Engl J Med. 2009;360:1030–7. [DOI] [PubMed] [Google Scholar]
- 6. Levin DC, Parker L, Intenzo CM, Sunshine JH. Recent rapid increase in utilization of radionuclide myocardial perfusion imaging and related procedures: 1996‐1998 practice patterns. Radiology. 2002;222:144–8. [DOI] [PubMed] [Google Scholar]
- 7. Levin DC, Rao VM, Parker L, Frangos AJ, Sunshine JH. Recent trends in utilization rates of noncardiac thoracic imaging: an example of how imaging growth might be controlled. J Am Coll Radiol. 2007;4:886–9. [DOI] [PubMed] [Google Scholar]
- 8. Levin DC, Rao VM, Parker L, Frangos AJ, Sunshine JH. Recent trends in utilization rates of abdominal imaging: the relative roles of radiologists and nonradiologist physicians. J Am Coll Radiol. 2008;5:744–7. [DOI] [PubMed] [Google Scholar]
- 9. Lucas FL, De Lorenzo MA, Siewers AE, Wennberg DE. Temporal trends in the utilization of diagnostic testing and treatments for cardiovascular disease in the United States, 1993–2001. Circulation. 2006;113:374–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Ladapo JA, Blecker S, Douglas PS. Physician decision making and trends in the use of cardiac stress testing in the United States: an analysis of repeated cross‐sectional data. Ann Intern Med. 2014;161:482–90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Blecker S, Bhatia RS, You JJ, Lee DS, Alter DA, Wang JT, Wong HJ, Tu JV. Temporal trends in the utilization of echocardiography in Ontario, 2001 to 2009. JACC Cardiovasc Imaging. 2013;6:515–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Brindis RG, Douglas PS, Hendel RC, Peterson ED, Wolk MJ, Allen JM, Patel MR, Raskin IE, Hendel RC, Bateman TM, Cerqueira MD, Gibbons RJ, Gillam LD, Gillespie JA, Hendel RC, Iskandrian AE, Jerome SD, Krumholz HM, Messer JV, Spertus JA, Stowers SA. ACCF/ASNC appropriateness criteria for single‐photon emission computed tomography myocardial perfusion imaging (SPECT MPI): a report of the American College of Cardiology Foundation Quality Strategic Directions Committee Appropriateness Criteria Working Group and the American Society of Nuclear Cardiology endorsed by the American Heart Association. J Am Coll Cardiol. 2005;46:1587–605. [DOI] [PubMed] [Google Scholar]
- 13. Hendel RC, Berman DS, Di Carli MF, Heidenreich PA, Henkin RE, Pellikka PA, Pohost GM, Williams KA. ACCF/ASNC/ACR/AHA/ASE/SCCT/SCMR/SNM 2009 Appropriate Use Criteria for Cardiac Radionuclide Imaging: a Report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, the American Society of Nuclear Cardiology, the American College of Radiology, the American Heart Association, the American Society of Echocardiography, the Society of Cardiovascular Computed Tomography, the Society for Cardiovascular Magnetic Resonance, and the Society of Nuclear Medicine. J Am Coll Cardiol. 2009;53:2201–29. [DOI] [PubMed] [Google Scholar]
- 14. Wolk MJ, Bailey SR, Doherty JU, Douglas PS, Hendel RC, Kramer CM, Min JK, Patel MR, Rosenbaum L, Shaw LJ, Stainback RF, Allen JM. ACCF/AHA/ASE/ASNC/HFSA/HRS/SCAI/SCCT/SCMR/STS 2013 multimodality appropriate use criteria for the detection and risk assessment of stable ischemic heart disease: a report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, American Heart Association, American Society of Echocardiography, American Society of Nuclear Cardiology, Heart Failure Society of America, Heart Rhythm Society, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, and Society of Thoracic Surgeons. J Am Coll Cardiol. 2014;63:380–406. [DOI] [PubMed] [Google Scholar]
- 15. Roifman I, Rezai MR, Wijeysundera HC, Chow BJ, Wright GA, Tu JV. Utilization of cardiac computed tomography angiography and outpatient invasive coronary angiography in Ontario, Canada. J Cardiovasc Comput Tomogr. 2015;9:567–571. [DOI] [PubMed] [Google Scholar]
- 16. Roifman I, Wijeysundera HC, Austin PC, Maclagan LC, Rezai MR, Wright GA, Tu JV. Temporal trends in the utilization of noninvasive diagnostic tests for coronary artery disease in ontario between 2008 and 2014: a Population‐Based Study. Can J Cardiol. 2016:33;279–282. [DOI] [PubMed] [Google Scholar]
- 17. Alter DA, Stukel TA, Newman A. Proliferation of cardiac technology in Canada: a challenge to the sustainability of Medicare. Circulation. 2006;113:380–7. [DOI] [PubMed] [Google Scholar]
- 18. Juurlink D PC, Croxford R, Chong A, Austin P, Tu J, Laupacis A. Canadian Institute for Health Information Discharge Abstract Database: A Validation Study. ICES Investigative Report. 2006.
- 19. Joseph KS, Fahey J. Validation of perinatal data in the Discharge Abstract Database of the Canadian Institute for Health Information. Chronic Dis Can. 2009;29:96–100. [PubMed] [Google Scholar]
- 20. Ontario Ministry of Health and Long‐Term Care . Schedule of Benefits for Physician Services under the Health Insurance Act. 2014.
- 21. Ontario TRC‐eoFN‐iCITfDCADi . Toronto Health Economics and Technology Assessment Collaborative. 2010.
- 22. Gupta AK, Udrea A. Beyond linear methods of data analysis: time series analysis and its applications in renal research. Nephron Physiol. 2013;124:14–27. [DOI] [PubMed] [Google Scholar]
- 23. Jackevicius CA, Tu JV, Demers V, Melo M, Cox J, Rinfret S, Kalavrouziotis D, Johansen H, Behlouli H, Newman A, Pilote L. Cardiovascular outcomes after a change in prescription policy for clopidogrel. New Engl J Med. 2008;359:1802–10. [DOI] [PubMed] [Google Scholar]
- 24. Pintus E, Sorbolini S, Albera A, Gaspa G, Dimauro C, Steri R, Marras G, Macciotta NP. Use of locally weighted scatterplot smoothing (LOWESS) regression to study selection signatures in Piedmontese and Italian Brown cattle breeds. Anim Genet. 2014;45:1–11. [DOI] [PubMed] [Google Scholar]
- 25. Cleveland W. LOWESS: a program for smoothing scatterplots by robust locally weighted regression. Am Stat. 1981;35:54. [Google Scholar]
- 26. Arbel Y, Qiu F, Bennell MC, Austin PC, Roifman I, Rezai MR, Tu JV, Ko DT, Wijeysundera HC. Association between publication of appropriate use criteria and the temporal trends in diagnostic angiography in stable coronary artery disease: a population‐based study. Am Heart J. 2016;175:153–9. [DOI] [PubMed] [Google Scholar]
- 27. Mehta R, Ward RP, Chandra S, Agarwal R, Williams KA. American College of Cardiology F and American Society of Nuclear C. Evaluation of the American College of Cardiology Foundation/American Society of Nuclear Cardiology appropriateness criteria for SPECT myocardial perfusion imaging. J Nucl Cardiol. 2008;15:337–44. [DOI] [PubMed] [Google Scholar]
- 28. Gibbons RJ, Askew JW, Hodge D, Kaping B, Carryer DJ, Miller T. Appropriate use criteria for stress single‐photon emission computed tomography sestamibi studies: a quality improvement project. Circulation. 2011;123:499–503. [DOI] [PubMed] [Google Scholar]
- 29. Fonseca R, Negishi K, Otahal P, Marwick TH. Temporal changes in appropriateness of cardiac imaging. J Am Coll Cardiol. 2015;65:763–73. [DOI] [PubMed] [Google Scholar]
- 30. Gibbons RJ, Askew JW, Hodge D, Miller TD. Temporal trends in compliance with appropriateness criteria for stress single‐photon emission computed tomography sestamibi studies in an academic medical center. Am Heart J. 2010;159:484–9. [DOI] [PubMed] [Google Scholar]
- 31. Dos Santos MA, Santos MS, Tura BR, Felix R, Brito AS, De Lorenzo A. Budget impact of applying appropriateness criteria for myocardial perfusion scintigraphy: the perspective of a developing country. J Nucl Cardiol. 2016;23:1160–5. [DOI] [PubMed] [Google Scholar]
- 32. Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, West P, Wright D, Wright J. What's the evidence that NICE guidance has been implemented? Results from a national evaluation using time series analysis, audit of patients’ notes, and interviews. BMJ. 2004;329:999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Altman CA. Appropriate use criteria: vital new link in the chain of quality in pediatric echocardiography. J Am Coll Cardiol. 2015;66:1141–3. [DOI] [PubMed] [Google Scholar]