Abstract
BACKGROUND:
Although accurate risk prediction is essential in guiding treatment decisions in primary prevention of atherosclerotic cardiovascular disease, the accuracy of the Framingham Risk Score (recommended by a Canadian guideline) and the Pooled Cohort Equations (recommended by US guidelines) has not been assessed in a large contemporary Canadian population. Our primary objective was to assess the calibration and discrimination of the Framingham Risk Score and Pooled Cohort Equations in Ontario, Canada.
METHODS:
We conducted an observational study involving Ontario residents aged 40 to 79 years, without a history of atherosclerotic cardiovascular disease, who underwent cholesterol testing and blood pressure measurement from Jan. 1, 2010, to Dec. 31, 2014. We compared predicted event rates generated by the Framingham Risk Score and the Pooled Cohort Equations with observed event rates at 5 years using linkages from validated administrative databases.
RESULTS:
Our study cohort included 84 617 individuals (mean age 56.3 yr, 56.9% female). Over a maximum follow-up period of 5 years, we observed 2162 (2.6%) events according to the outcome definition of the Framingham Risk Score, and 1224 (1.4%) events according to the outcome definition of the Pooled Cohort Equations. The predicted event rate of 5.78% by the Framingham Risk Score and 3.51% by the Pooled Cohort Equations at 5 years overestimated observed event rates by 101% and 115%, respectively. The degree of overestimation differed by age and ethnicity. The C statistics for the Framingham Risk Score (0.74) and Pooled Cohort Equations (0.73) were similar.
INTERPRETATION:
The Framingham Risk Score and Pooled Cohort Equations significantly overpredicted the actual risks of atherosclerotic cardiovascular disease events in a large population from Ontario. Our finding suggests the need for further refinement of cardiovascular disease risk prediction scores to suit the characteristics of a multiethnic Canadian population.
Accurate individual risk estimation is essential in guiding informed treatment decisions in primary prevention, because treatment has greater benefit in those at increased risk of developing the disease.1,2 Primary prevention guidelines in cardiology around the world have endorsed the use of risk prediction scores to estimate the risk of atherosclerotic cardiovascular disease as an initial step in assessing the need for preventive treatment.2–6 The Canadian Cardiovascular Society (CCS) guideline recommends using the Framingham Risk Score to assess risk of atherosclerotic cardiovascular disease.2 The American College of Cardiology/American Heart Association (ACC/AHA) practice guidelines currently endorse the use of the Pooled Cohort Equations over the Framingham Risk Score for risk estimations.1,4
Since the Framingham cohort was assembled more than 3 decades ago, there have been substantial declines in the risk of cardiovascular events.7 Furthermore, these risk scores predict cardiovascular events but do not consider the potential competing impact of noncardiovascular deaths during follow-up. The accuracy of commonly used risk scores has also not been evaluated in a large contemporary Canadian cohort and has not been measured using newer statistical techniques. Our main objective was to assess the calibration and discrimination of the Framingham Risk Score and Pooled Cohort Equations in Ontario, Canada. Second, we aimed to assess the validity of these risk scores in prespecified subgroups to gain additional perspectives.
Methods
Study design and data sources
We performed an observational study of Ontario adults to evaluate the calibration and discrimination of the Framingham Risk Score and Pooled Cohort Equations using the Electronic Medical Record Administrative Data Linked Database (EMRALD) to identify the study cohort.8 The database is housed at ICES and includes electronic medical records of more than 350 primary care physicians caring for more than 500 000 individuals in Ontario, Canada. This data set was linked to various administrative data sets that included the following: the Ontario Registered Persons Database, a registry of Ontario residents with coverage under the Ontario Health Insurance Plan; multiple databases created using information from the Canadian Institute for Health Information (CIHI) Discharge Abstract Database to capture comorbid conditions and hospital admissions; the Ontario Cancer Registry to capture prior diagnosis of cancer; the Registrar General of Ontario Database to determine cause of death; the Ontario Laboratories Information System to determine laboratory results; and the Canadian Immigration and Citizenship Permanent Resident database, which includes the ethnicity of landed immigrants of Ontario, used with the application of a surname algorithm to determine ethnicity of individuals in the study.9 These data sets were linked using unique encoded identifiers and analyzed at ICES. Additional information about these databases can be found elsewhere.10,11
Study cohort
Individuals aged 40–79 years who visited their primary care physician from Jan. 1, 2010, to Dec. 31, 2014, and had blood pressure and cholesterol measurements taken within a year of each other were considered for inclusion. This age restriction was chosen to be consistent with the recently developed Pooled Cohort Equations.1 We excluded patients with prior hospital admissions for myocardial infarction, cerebrovascular disease, heart failure, peripheral vascular disease and coronary revascularization from 1988 to the date individuals entered the cohort.10 Nonresidents of Ontario and those without a valid Ontario Health Card were excluded. We also excluded individuals if their primary care physicians had been enrolled in EMRALD for less than 1 year to ensure we had complete capture of the electronic medical records on eligible individuals.
Outcomes
To assess the accuracy of the Framingham Risk Score, we used the Framingham outcome definition that included a composite of circulatory death, and hospital admission for myocardial infarction, angina, cerebrovascular events (ischemic stroke, hemorrhagic stroke and transient ischemic attack), heart failure and peripheral artery disease (also including abdominal aortic aneurysm and carotid endarterectomy or stent).12,13 The Pooled Cohort Equations outcome definition included a composite of death due to coronary heart disease and stroke, and hospital admission for nonfatal myocardial infarction or stroke.14 The original weighting of the risk factors for the Framingham Risk Score and Pooled Cohort Equations are shown in Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.190848/-/DC1), and the codes used to define outcomes are shown in Appendix 1, Supplemental Table 1. Follow-up continued until either the study end date of Dec. 31, 2016, until individuals had 5 years of follow-up, until death from a cause unrelated to cardiovascular disease outcome or until an end point had occurred.
Statistical analysis
For both the Framingham Risk Score and Pooled Cohort Equations, we assessed calibration, which refers to the agreement between observed and predicted risk, and discrimination, which refers to how well the model differentiates between different risk levels.15 We compared 5-year predicted and observed rates of atherosclerotic cardiovascular disease using the equations from the Framingham Risk Score and Pooled Cohort Equations, and the baseline survival function at 5 years.16 The observed rate was calculated using the Kalbfleisch–Prentice estimator of cumulative incidence that accounts for the competing risk of non-outcome death (defined as deaths due to noncirculatory causes for validating the Framingham Risk Score and deaths from causes other than myocardial infarction or stroke for validating the Pooled Cohort Equations), which has been shown to generate estimates that are closer to the true survival estimate in a simulation study.17 We generated 95% confidence intervals (CIs) of observed risk using a Wald-based approach with Aalen variance estimator. The relative percentage of difference in the predicted to observed risk was defined as discordance (discordance = ( predicted rate – observed rate)/observed rate). Calibration was also assessed qualitatively by plotting the predicted versus observed risks across deciles of predicted risk. We assessed discrimination by calculating the C statistic using the R package (timeROC), which took into account competing risks.
A series of additional analyses was performed. First, we evaluated the accuracy of the risk scores in prespecified subgroups (age and sex) to evaluate whether there were discrepancies in the discordance ratios. Second, we compared the 5-year predicted and observed rates of atherosclerotic cardiovascular disease in different risk categories. The Canadian guidelines, using the Framingham Risk Score, categorized 10-year risk as less than 10%, 10%–20% and greater than 20%.18 The US guidelines, using the Pooled Cohort Equations, categorized 10-year risk as less than 5%, 5%–7.5%, 7.5%–10% and greater than 10%.14 Third, because smoking status was missing for about 15% of included individuals, we performed an additional multiple imputation analysis using 10 imputed data sets. Fourth, we repeated our analyses in a “statin-eligible” cohort according to the 2016 CCS guidelines.2 This cohort included individuals aged 40–75 years, who did not have diabetes, whose low-density lipoprotein (LDL) cholesterol was less than 5 mmol/L, who did not have chronic kidney disease and who were not taking statins. We also excluded those with substantial comorbidities (i.e., history of cancer and receiving hemodialysis) because statins may not be indicated in these subgroups. Analyses were conducted at ICES using SAS version 9.4 and R version 3.1.2.
Ethics approval
This project was approved by the research ethics board at Sunnybrook Health Sciences Centre, Toronto.
Results
A total of 133 892 Ontario residents of all ages who had blood pressure measurement and cholesterol assessment within 1 year of each other from 2010 to 2014 were captured in the EMRALD cohort (Figure 1). After we applied the inclusion and exclusion criteria, our cohort included 84 617 individuals in the general cohort.
Figure 1:
Creation flowchart. A total of 84 617 individuals were included in our study cohort after inclusion and exclusion criteria were applied to the Electronic Medical Record Administrative Data Linked Database (EMRALD) cohort. Note: CABG = coronary artery bypass grafting, HF = heart failure, MI = myocardial infarction, PAD = peripheral artery disease, PCI = percutaneous coronary intervention.
The mean age of the general cohort was 56.3 years, and 56.9% were women. The participants’ mean total cholesterol level was 5.1 mmol/L, mean LDL cholesterol was 3.1 mmol/L, mean systolic blood pressure was 127 mm Hg, 13.4% had diabetes and 16.9% were current smokers (Table 1). Of the participants, 2.2% were of Chinese ethnicity, 1.6% were of South Asian ethnicity and 1.1% were black.
Table 1:
Baseline characteristics of Ontario residents who had blood pressure measurement and cholesterol assessment within 1 year of each other from 2010 to 2014
| Characteristic | No. (%) of participants* | ||
|---|---|---|---|
| Overall n = 84 617 |
Men n = 36 433 |
Women n = 48 184 |
|
| Age, yr, mean ± SD | 56.3 ± 10.1 | 56.1 ± 10.0 | 56.3 ± 10.1 |
| Income quintile | |||
| 1 (lowest) | 12 777 (15.1) | 5639 (15.5) | 7138 (14.8) |
| 2 | 15 076 (17.8) | 6401 (17.6) | 8675 (18.0) |
| 3 | 16 151 (19.1) | 7010 (19.2) | 9141 (19.0) |
| 4 | 18 192 (21.5) | 7900 (21.7) | 10 292 (21.4) |
| 5 (highest) | 22 050 (26.1) | 9302 (25.5) | 12 748 (26.5) |
| Rural residency | 17 766 (21.0) | 8081 (22.2) | 9685 (20.1) |
| Ethnicity | |||
| Chinese | 1886 (2.2) | 746 (2.0) | 1140 (2.4) |
| South Asian | 1343 (1.6) | 652 (1.8) | 691 (1.4) |
| Black | 914 (1.1) | 398 (1.1) | 516 (1.1) |
| Lipid measurements, mmol/L, mean ± SD | |||
| Total cholesterol | 5.1 ± 1.1 | 5.0 ± 1.1 | 5.3 ± 1.1 |
| HDL cholesterol | 1.4 ± 0.4 | 1.3 ± 0.4 | 1.6 ± 0.4 |
| LDL cholesterol | 3.1 ± 0.9 | 3.0 ± 0.9 | 3.1 ± 0.9 |
| Diabetes mellitus | 11 311 (13.4) | 5657 (15.5) | 5654 (11.7) |
| Body mass index, mean ± SD | 28.9 ± 6.5 | 29.4 ± 5.8 | 28.6 ± 7.0 |
| Systolic blood pressure, mm Hg, mean ± SD | 127.4 ± 17.3 | 129.6 ± 16.7 | 125.8 ± 17.6 |
| Smoking | |||
| Nonsmoker | 45 979 (54.3) | 17 211 (47.2) | 28 768 (59.7) |
| Smoker | 14 271 (16.9) | 7186 (19.7) | 7085 (14.7) |
| Former smoker | 24 367 (28.8) | 12 036 (33.0) | 12 331 (25.6) |
| Medications | |||
| Antihypertensive | 20 690 (24.5) | 9486 (26.0) | 11 204 (23.3) |
| Statin | 12 069 (14.3) | 6453 (17.7) | 5616 (11.7) |
Note: HDL = high-density lipoprotein, LDL = low-density lipoprotein, SD = standard deviation.
Unless stated otherwise.
Predicted and observed events for Framingham Risk Score and Pooled Cohort Equations
During the follow-up period (maximum of 5 yr), we observed 2162 atherosclerotic cardiovascular disease events according to the Framingham Risk Score outcome definition. The observed 5-year incidence rate accounting for the competing risk of noncirculatory death was 2.88% (95% CI 2.76%–3.00%) (Table 2). The mean 5-year predicted risk as estimated by the Framingham Risk Score was 5.78%, leading to an absolute difference of 2.9% and a relative discordance of 101%. The Framingham Risk Score overestimated risk by 110% in men and 85% in women. The C statistic of the Framingham Risk Score was 0.74 overall, 0.69 in men and 0.74 in women.
Table 2:
Five-year predicted and observed events of the Framingham Risk Score and Pooled Cohort Equations
| Variable | Predicted, % (95% CI) | Observed, % (95% CI) | Absolute difference, % | Discordance, %* | C statistic |
|---|---|---|---|---|---|
| Overall | |||||
| Framingham Risk Score | 5.78 (5.74–5.82) | 2.88 (2.76–3.00) | 2.9 | 101 | 0.74 |
| Pooled Cohort Equations | 3.51 (3.48–3.54) | 1.63 (1.53–1.72) | 1.9 | 115 | 0.73 |
| Men | |||||
| Framingham Risk Score | 8.57 (8.50–8.65) | 4.09 (3.87–4.31) | 4.5 | 110 | 0.69 |
| Pooled Cohort Equations | 5.33 (5.28–5.39) | 2.43 (2.26–2.60) | 2.9 | 119 | 0.67 |
| Women | |||||
| Framingham Risk Score | 3.67 (3.64–3.70) | 1.98 (1.84–2.11) | 1.7 | 85 | 0.74 |
| Pooled Cohort Equations | 2.13 (2.10–2.15) | 1.03 (0.93–1.12) | 1.1 | 107 | 0.72 |
Note: CI = confidence interval.
Discordance is calculated as (predicted event rate – observed event rate)/observed event rate × 100.
During the follow-up period, we observed 1224 cardiovascular disease events according to the Pooled Cohort Equations outcome definition. The 5-year observed Pooled Cohort Equations event rate was 1.63% (95% CI 1.53%–1.72%), and 1443 events were observed. The mean predicted overall event rate as estimated by the Pooled Cohort Equations was 3.51%, resulting in a relative overestimation of 115% in all individuals. The C statistic of the Pooled Cohort Equations was 0.73 overall, 0.67 in men and 0.72 in women (Table 2). There was a lack of calibration for the Framingham Risk Score and Pooled Cohort Equations in the overall study cohort (Figure 2).
Figure 2:
Predicted and observed 5-year risk for Framingham Risk Score and Pooled Cohort Equations in the overall cohort (40–79 yr), by decile of risk. Each dot shows predicted versus observed rate in each decline of risk. The solid line shows when predicted risk is equal to observed risk.
Calibration of the Framingham Risk Score and Pooled Cohort Equations according to strata
The plots of overall calibration in subgroups of age, sex, ethnicity and risk thresholds are shown in Figure 3. For age strata, the Framingham Risk Score had a discordance of 183% in the youngest age group (40–45 yr) and a smaller discordance of 44% in the older age group (75–79 yr). In contrast, the Pooled Cohort Equations had smaller discordance in younger age groups (51% in the group aged 40–45 yr) and a much larger discordance in older individuals (232% in the group aged 75–80 yr). Discordance of the 2 risk scores was similar for men and women.
Figure 3:
Predicted and observed 5-year risk for the Framingham Risk Score and Pooled Cohort Equations by (A) age category, (B) sex, (C) ethnicity and (D) risk threshold recommended by practice guidelines at 10 years.
Validation of risk scores after smoking imputation
Similar to the original analysis in which individuals with missing smoking status were excluded, both the Framingham Risk Score and the Pooled Cohort Equations significantly overestimated the actual risk of events after smoking data were imputed (Appendix 1, Supplemental Table 4). The Framingham Risk Score overestimated risk by 104% in all individuals, and the Pooled Cohort Equations overestimated risk by 118%.
Characteristics and validation of risk scores among statin-eligible individuals
Appendix 1, Supplemental Figure 1 details the construction of the cohort of individuals eligible for statins according to CCS guidelines, and Appendix 1, Supplemental Table 2 shows the characteristics of the 59 126 individuals. The statin-eligible cohort was younger than the overall cohort, with a mean age of 53.8 years (v. 56.3 yr); 58.6% were women. The participants’ mean total cholesterol level was 5.25 mmol/L, mean LDL cholesterol was 3.1 mmol/L, mean systolic blood pressure was 126 mm Hg and 17.3% were smokers. They were less likely than individuals in the overall cohort to be taking antihypertensive medications (15% v. 24.6%).
A summary of the calibration and discrimination characteristics in this cohort is shown in Appendix 1, Supplemental Table 3. In general, the Framingham Risk Score had greater overestimation of risk than the Pooled Cohort Equations among patients who could be considered eligible for statin treatment. For example, the Framingham Risk Score overestimated risk by 129% in all individuals, whereas the Pooled Cohort Equations overestimated risk by 83%. C statistics were 0.75 for the Framingham Risk Score and 0.74 for the Pooled Cohort Equations in this cohort.
Interpretation
Using a large contemporary cohort of individuals in an Ontario primary care setting, we found that both the Framingham Risk Score and the Pooled Cohort Equations predicted risks of atherosclerotic cardiovascular disease that were more than double the actual observed risk. Overestimation was seen in the overall study cohort, as well as in subgroups of age, sex and eligibility for statins. The Framingham Risk Score overestimated risk more in younger patients, whereas the Pooled Cohort Equations overestimated it more in older patients. Given the importance of accurate risk estimation in primary prevention of atherosclerotic cardiovascular disease, our findings suggest the need for additional validation studies across Canada. Evidence of consistent overprediction would strongly support redevelopment of prediction risk scores for atherosclerotic cardiovascular disease to improve treatment decisions.
The Framingham Risk Score was initially developed using data on individuals residing in Framingham, Massachusetts, collected in the 1970s.12 Because these individuals may not be representative of a contemporary US population, the ACC/AHA developed the Pooled Cohort Equations by assembling more contemporary, diverse cohorts from the US that enabled accurate risk estimations of atherosclerotic cardiovascular disease among black Americans.14 Other jurisdictions have developed risk scores that are better calibrated to their populations. Currently, the European Society of Cardiology endorses the Systematic Coronary Risk Evaluation (SCORE),5 the United Kingdom endorses QRISK3 and New Zealand has recently created a new score called PREDICT.6
Our results showing overestimation of the Framingham Risk Score and Pooled Cohort Equations are consistent with those of several studies that examined validation of these scores using US cohorts.19–23 For example, using the Multi-Ethnic Study of Atherosclerosis (MESA) cohort, DeFilippis found that the Framingham Coronary Heart Disease prediction overestimated actual risk by 51% and the Pooled Cohort Equations overestimated risk by 78%.21 To the best of our knowledge, only 1 study has evaluated the performance of the original Framingham Risk Score using data on Canadian participants.24 Grover and colleagues used data from the Lipid Research Clinics Follow-up Cohort, which included data on 1173 Canadian participants, and found good calibration and discrimination of the Framingham Risk Score. However, these findings are likely not applicable to contemporary practice given that the participants were enrolled from 1971 to 1976.24
We observed that the Framingham Risk Score substantially overestimated risk among individuals in younger age groups, whereas the Pooled Cohort Equations substantially overestimated risk among individuals in older age groups. This discrepancy may be attributable to the Pooled Cohort Equations risk estimates using an age-squared term in the prediction model.14 None of the existing risk scores included ethnic minorities such as Chinese or South Asians, which are the most common ethnic minority groups in Canada.
There may be several reasons why existing risk scores overestimated the actual risks of cardiovascular events in our cohort.20,25–27 First, individuals in our study were recruited in a contemporary period in which atherosclerotic cardiovascular disease events occur less frequently than when those scores were derived. In the Framingham Heart Study, a coronary heart disease outcome developed in 11.4% of the participants at 12 years.12 In the pooled cohort, a Pooled Cohort Equations outcome developed in 5.3% of white participants, and an outcome developed in 6.1% of black participants at 10 years.14 In contrast, we observed much lower event rates at 5 years in that a Framingham Risk Score outcome developed in only 2.6% of our study cohort and a Pooled Cohort Equations outcome developed in 1.4%. Researchers who recalibrated the performance of 4 commonly used risk scores also found that the discrepancy was in part explained by the year the cohorts were assembled.27 Second, individuals likely had earlier diagnosis and improved control of cardiovascular risk factors. Third, our Ontario cohort benefited from universal coverage of hospital and physician services, which is available to most residents. Finally, neither the Framingham Risk Score nor the Pooled Cohort Equations accounted for competing risks of noncardiac death, which could have produced more biased estimates.
Limitations
Several potential limitations of our study merit discussion. First, it is possible that patients whose data are included in EMRALD may be different from the general Ontario population. In previous research, we have shown that they were representative of the Ontario patients who had a primary care physician, but we have not shown that they are representative of patients without a primary care physician or those residing in other Canadian provinces.8 Accordingly, additional studies using cohorts across Canada would be important to ensure the robustness of our findings. Second, we were able to estimate cardiovascular risk in a contemporary cohort and we validated the Framingham Risk Score and Pooled Cohort Equations at 5 years, but we did not have 10-year follow-up because our cohort included individuals who had cholesterol testing and blood pressure measurement from 2010 to 2014. Additional follow-up of this cohort is needed to ensure our findings can be extended over a longer duration. Third, although Canadian family physicians commonly use automated blood pressure measurement,28 we did not have information on the way blood pressure measurements were obtained in our study cohort. Blood pressure readings in clinical practice are usually higher than in the research setting, and this could have contributed to an overestimation of risk.29 Fourth, the CCS recommends a modification of the Framingham Risk Score that doubles the risk if there is a family history of premature atherosclerotic cardiovascular disease.2 Because we did not have information on family history, we were unable to duplicate this CCS recommendation completely. However, the use of family history information would lead to further overestimation of the already inflated risk.
Conclusion
Our findings suggest overestimations of atherosclerotic cardiovascular disease risk with use of the Framingham Risk Score and Pooled Cohort Equations in an Ontario population without prior history of cardiovascular disease. Further efforts are needed to refine the currently recommended prediction risk scores for atherosclerotic cardiovascular disease to suit the characteristics of a diverse Canadian population.
Acknowledgement
The authors are indebted to Dr. Jack V. Tu (deceased May 30, 2018), who contributed to the idea, early work of this study and funding from the Foundation Grant (FDN 143313) from CIHR.
Footnotes
Competing interests: George Thanassoulis reports personal fees from Amgen, personal fees from Sanofi and Regeneron, grants and personal fees from Ionis, grants and personal fees from Servier, and personal fees from HLS Therapeutics, outside the submitted work. Jacob Udell has received consulting or speaker honoraria from Amgen, AstraZeneca, Boehringer Ingelheim, Janssen, Merck, Novartis and Sanofi, and research grants to his institution from AstraZeneca, Novartis and Sanofi. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Dennis Ko and Todd Anderson made substantial contributions to the conception and design of the work. Atul Sivaswamy, Maneesh Sud, Gynter Kotrri, Paymon Azizi and Peter Austin contributed to the analysis and interpretation of data. Maria Koh contributed to the acquisition, analysis and interpretation of data. Douglas Lee, Idan Roifman, George Thanassoulis, Karen Tu, Jacob Udell and Harindra Wijeysundera contributed to the interpretation of data. Dennis Ko and Todd Anderson drafted the work. Atul Sivaswamy, Maneesh Sud, Gynter Kotrri, Paymon Azizi, Maria Koh, Peter Austin, Douglas Lee, Idan Roifman, George Thanassoulis, Karen Tu, Jacob Udell and Harindra Wijeysundera revised the work critically for important intellectual content. All of the authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: Early work on this study was funded by a Foundation Grant (FDN 143313) from the Canadian Institutes of Health Research (CIHR). Additional funding was provided by Foundation Grants (FDN 154333). Dennis Ko, Peter Austin and Douglas Lee are supported by a Mid-Career Award from the Heart and Stroke Foundation of Canada. Douglas Lee is supported by the Ted Rogers Chair in Heart Function Outcomes. Jacob Udell is supported in part by a National New Investigator/Ontario Clinician Scientist Award from the Heart and Stroke Foundation of Canada and an Early Researcher Award from the Government of Ontario. Karen Tu is supported by a Research Scholar Award from the Department of Family and Community Medicine, University of Toronto. Harindra Wijeysundera is supported by a Clinician Scientist Award from Heart and Stroke Foundation of Canada. Maneesh Sud is funded through a CIHR Post-Doctoral Fellowship, the Eliot Phillipson Clinician–Scientist Program at the University of Toronto and a SANSAR (South Asian Network Supporting Awareness and Research)–Burgundy Young Investigator Award. George Thanassoulis is supported by a Clinician Scholar Award from the Fonds de recherche Québec — Santé.
Data sharing: The data set from this study is held securely in coded form at ICES. While data-sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS. The full data set creation plan and underlying analytic code are available from the authors on request, understanding that the computer programs may rely on coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.
Disclaimer: This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information (CIHI). The analyses, conclusions, opinions and statements expressed herein are those of the authors, and not necessarily those of CIHI.
References
- 1.Stone NJ, Robinson JG, Lichtenstein AH, et al. 2013 ACC/AHA Guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults. J Am Coll Cardiol 2014;63:2889–934. [DOI] [PubMed] [Google Scholar]
- 2.Anderson TJ, Gregoire J, Pearson GJ, et al. 2016 Canadian Cardiovascular Society Guidelines for the Management of Dyslipidemia for the Prevention of Cardiovascular Disease in the adult. Can J Cardiol 2016;32:1263–82. [DOI] [PubMed] [Google Scholar]
- 3.Collins GS, Altman DG. Predicting the 10-year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2. BMJ 2012;344:e4181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Grundy SM, Stone NJ, Bailey AL, et al. 2018. Guideline on the management of blood cholesterol. J Am Coll Cardiol Nov. 2018; 10.1016/j.jacc.2018.11.003. [DOI] [Google Scholar]
- 5.Catapano AL, Graham I, De Backer G, et al. 2016 ESC/EAS guidelines for the management of dyslipidaemias. Eur Heart J 2016;37:2999–3058. [DOI] [PubMed] [Google Scholar]
- 6.Pylypchuk R, Wells S, Kerr A, et al. Cardiovascular disease risk prediction equations in 400 000 primary care patients in New Zealand: a derivation and validation study. Lancet 2018;391:1897–907. [DOI] [PubMed] [Google Scholar]
- 7.Tu JV, Khan AM, Ng K, et al. Recent temporal changes in atherosclerotic cardiovascular diseases in Ontario: Clinical and Health Systems Impact. Can J Cardiol 2017;33:378–84. [DOI] [PubMed] [Google Scholar]
- 8.Tu K, Widdifield J, Young J, et al. Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective. BMC Med Inform Decis Mak 2015;15:67. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shah BR, Chiu M, Amin S, et al. Surname lists to identify South Asian and Chinese ethnicity from secondary data in Ontario, Canada: a validation study. BMC Med Res Methodol 2010;10:42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Tu JV, Chu A, Donovan LR, et al. The Cardiovascular Health in Ambulatory Care Research Team (CANHEART): using big data to measure and improve cardiovascular health and healthcare services. Circ Cardiovasc Qual Outcomes 2015; 8:204–12. [DOI] [PubMed] [Google Scholar]
- 11.Tu JV, Chu A, Rezai MR, et al. The incidence of major cardiovascular events in immigrants to Ontario, Canada: the CANHEART Immigrant Study. Circulation 2015;132:1549–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wilson PWF, D’Agostino RB, Levy D, et al. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837–47. [DOI] [PubMed] [Google Scholar]
- 13.D’Agostino RB, Sr, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 2008;117: 743–53. [DOI] [PubMed] [Google Scholar]
- 14.Goff DC, Jr, Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;129:S49–73. [DOI] [PubMed] [Google Scholar]
- 15.Austin PC, Steyerberg EW. Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med 2014; 33: 517–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Muntner P, Colantonio LD, Cushman M, et al. Validation of the atherosclerotic cardiovascular disease Pooled Cohort Risk Equations. JAMA 2014;311:1406–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Xia F, Ning J, Huang X. Empirical comparison of the Breslow Estimator and the Kalbfleisch Prentice Estimator for Survival Functions. J Biom Biostat 2018;9. pii 392. 10.4172/2155-6180.1000392. [Epub 2018 Feb.]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Anderson TJ, Mancini GB, Genest J, Jr, et al. The new dyslipidemia guidelines: what is the debate? Can J Cardiol 2015;31:605–12. [DOI] [PubMed] [Google Scholar]
- 19.Ridker PM, Cook NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet 2013;382:1762–5. [DOI] [PubMed] [Google Scholar]
- 20.Cook NR, Ridker PM. Further insight into the cardiovascular risk calculator: the roles of statins, revascularizations, and underascertainment in the Women’s Health Study. JAMA Intern Med 2014;174:1964–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.DeFilippis AP, Young R, Carrubba CJ, et al. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Ann Intern Med 2015;162:266–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jung KJ, Jang Y, Oh DJ, et al. The ACC/AHA 2013 Pooled Cohort Equations compared to a Korean Risk Prediction Model for atherosclerotic cardiovascular disease. Atherosclerosis 2015;242:367–75. [DOI] [PubMed] [Google Scholar]
- 23.Rana JS, Tabada GH, Solomon MD, et al. Accuracy of the Atherosclerotic Cardiovascular Risk Equation in a large contemporary, multiethnic population. J Am Coll Cardiol 2016;67:2118–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Grover SA, Hemmelgarn B, Joseph L, et al. The role of global risk assessment in hypertension therapy. Can J Cardiol 2006;22:606–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.DeFilippis AP, Young R, McEvoy JW, et al. Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort. Eur Heart J 2017;38:598–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Mora S, Wenger NK, Cook NR, et al. Evaluation of the Pooled Cohort Risk Equations for Cardiovascular Risk Prediction in a multiethnic cohort from the Women’s Health Initiative. JAMA Intern Med 2018;178:1231–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Pennells L, Kaptoge S, Wood A, et al. Equalization of four cardiovascular risk algorithms after systematic recalibration: individual-participant meta-analysis of 86 prospective studies. Eur Heart J 2019;40:621–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Kaczorowski J, Myers MG, Gelfer M, et al. How do family physicians measure blood pressure in routine clinical practice? National survey of Canadian family physicians. Can Fam Physician 2017;63:e193–9. [PMC free article] [PubMed] [Google Scholar]
- 29.Roerecke M, Kaczorowski J, Myers MG. Comparing automated office blood pressure readings with other methods of blood pressure measurement for identifying patients with possible hypertension: a systematic review and meta-analysis. JAMA Intern Med 2019;179:351–62. [DOI] [PMC free article] [PubMed] [Google Scholar]