Abstract
Background
Systematic COronary Risk Evaluation 2 (SCORE2) and SCORE2-Older Persons (OP) models have been proposed as new cardiovascular risk evaluation tools.
Objectives
This study evaluated the performance of SCORE/SCORE-OP and SCORE2/SCORE2-OP in the East Asian population by using population-based cohort data from the National Health Insurance Service (NHIS) Health Screening Cohort of Korea.
Methods
A total of 324,384 NHIS examinees from 2004 to 2005 were divided into 5 age groups: 40-49 years, 50-59 years, 60-69 years,70-79 years, and more than 80 years. The examinees had their predicted cardiovascular disease risks calculated by using SCORE, SCORE2, SCORE-OP, and SCORE2-OP models. The low-risk model was applied on the basis of the cohort’s observed event rates. The observed and predicted cardiovascular risks were compared.
Results
A total of 324,384 subjects were included (mean age 51.4 ± 7.3 years; women, 37.9% for the SCORE/SCORE2 group and mean age 73.0 ± 2.8 years; women, 47.5% for the SCORE/SCORE2-OP group). Over a median follow-up of 9 years, cardiovascular events occurred in 15.0% and 28.9% in SCORE/SCORE2 and SCORE/SCORE2-OP groups, respectively. The SCORE/SCORE-OP model underestimated cardiovascular disease risk in young men (aged 40-49 years) and women (aged 40-59 years) and overestimated it in older age groups. In contrast, SCORE2/SCORE2-OP invariably overestimated the risk in all age groups and sexes. SCORE2/SCORE2-OP showed no improvement in Harrell’s concordance index (C-index) compared with SCORE/SCORE-OP. Calibration plots favored SCORE2 over SCORE but not SCORE2-OP over SCORE-OP.
Conclusions
Both SCORE2/SCORE2-OP and SCORE/SCORE-OP overestimated cardiovascular disease risk with low performance. SCORE2/SCORE2-OP showed slight improvement over older versions, but modifications are necessary for the East Asian population.
Key Words: Asian population, cardiovascular disease, Systematic COronary Risk Evaluation, validity
Central Illustration
Cardiovascular disease is a leading cause of mortality and accounts for more than 30% of deaths worldwide.1 Several cardiovascular risk prediction models have been proposed to assess and treat high-risk populations, including the Framingham,2 pooled cohort equation,3 and SCORE (Systematic COronary Risk Evaluation) models.4
The SCORE and SCORE-Older Persons (OP) models were proposed for European populations because of concerns regarding the applicability of the Framingham risk estimation model, which was derived from U.S. data.4,5 The SCORE model has been widely used to calculate the 10-year risk of fatal cardiovascular disease, with recalibration depending on the region.6, 7, 8 However, accumulating data on the limitations of the SCORE model, such as underestimation of the total cardiovascular disease burden,9, 10, 11 suggested the need for the development of a revised version, SCORE2. Recently, the SCORE2 and SCORE2-OP models were developed to estimate the risk of combined 10-year outcome of fatal and nonfatal cardiovascular disease events in persons without previous cardiovascular disease or diabetes who are aged 40 to 69 years and more than 70 years, respectively.12,13
In East Asia, there are no established cardiovascular risk prediction models; therefore, risk engines from Western countries, including SCORE, are widely used. However, validation of the SCORE and SCORE2 models in Asian populations has been limited. The only validations done on Asian populations so far are studies by Selvarajah et al9 and Ahn et al.14 The study by Selvarajah et al9 showed the problem of underestimation of the SCORE model in women and was limited to the Malaysian population. The study by Ahn et al14 showed SCORE risk in association with carotid atherosclerosis and peripheral arterial disease that was limited to patients with type 2 diabetes mellitus in Korea. Thus, validation of the SCORE and newly developed SCORE2 in general Asian populations remains an unmet need. This study aimed to evaluate the performances of SCORE, SCORE-OP, SCORE2, and SCORE2-OP in an East Asian (Korean) population.
Methods
Data source
The National Health Insurance Program in Korea covers approximately 50 million people residing in Korea. It is a mandatory medical care system operated by the National Health Insurance Service (NHIS). From the NHIS database, this study used the NHIS Health Screening Cohort (NHIS-HEALS) for the data source. The NHIS-HEALS cohort database was established by the NHIS in 2015 to provide information on noncommunicable diseases and health risk factors. For the NHIS-HEALS, the sample cohort was randomly selected from participants aged 40 to 79 years who underwent health screening between 2002 and 2003 and were followed up until 2013.15 The data used in this study consisted of an updated version that was followed up until 2015. The cohort included 514,866 participants, 10% of those who undertook health screening between 2002 and 2003. The cohort was followed up annually or biennially depending on the type of information. The cohort’s death information and health care use were checked annually, and health screening information was checked biennially. The NHIS-HEALS cohort includes data not only on socioeconomic and demographic factors but also on target health problems such as hypertension and diabetes mellitus, risk factors such as cigarette smoking and alcohol consumption, medical and family history, and health care use.15 Further specific information on NHIS-HEALS is described elsewhere.15
The participants’ smoking status was assessed using a questionnaire on smoking status, packs per day, and duration. Blood pressure and other laboratory results, including total cholesterol and high-density lipoprotein cholesterol (HDL-C), were collected from the health screening database. The laboratory values, including cholesterol levels, were collected after overnight fasting. Mortality data in the NHIS-HEALS were provided by Statistics Korea, a government agency. The data included information on date and cause of death encoded in the International Classification of Diseases-10th Revision (ICD-10) codes–based Korean Standard Classification of Diseases and Causes of Death.
This study was exempted from Institutional Review Board review (IRB No. 2107-081-1234) because it used anonymized data. Informed consent could not be acquired because the database was deidentified when provided from the NHIS. Approval for use of the released database by the NHIS was received in 2020 and 2021.
Study design
All participants were followed up from the start date between 2004 and 2005 until the end of the study in 2015, the date of the first cardiovascular disease outcome, death, or emigration, whichever came first. The most recently reported World Health Organization age- and sex-standardized overall cardiovascular disease mortality rate was 74 per 100,000 people in 2016,16 and a more recently published paper by Lee et al,17 using the Korean nationwide database, reported the rate to be 122.7 per 100,000 people in 2018. However, because our study’s participants were followed up until 2015, a low-risk region (<100 cardiovascular disease deaths per 100,000) was tested to calculate the predicted cardiovascular disease risk and evaluate the performance of SCORE2/SCORE2-OP and SCORE/SCORE-OP in the Korean population.
The coefficient for a low-risk region was used to calculate the predicted risk in the SCORE and SCORE2 models and compare the risks with the actual observed risk. The observed cardiovascular disease risk, consisting of both fatal and nonfatal cardiovascular disease, was used to compare the SCORE and SCORE2 models. This comparison was performed in consideration of a real-world busy clinical setting where precise differentiation on fatality of cardiovascular disease is not always applicable.
Study group
The study group included participants aged more than 40 years, without a previous history of cardiovascular disease, from the NHIS-HEALS. The occurrence of prevalent ischemic heart disease (ICD-10 codes I20-25) or cerebrovascular disease (ICD-10 codes I60-69) before the baseline year (2004) was counted as evidence of previous cardiovascular disease history. All the values except for HDL-C were collected in 2004. The data on HDL-C were collected in 2009 because HDL-C was not included in National Health Examination until 2009. Subjects were divided into 5 age groups (40-49 years, 50-59 years, 60-69 years, 70-79 years, and more than 80 years). For subjects aged <70 years, SCORE and SCORE2 models were used to calculate the predicted cardiovascular disease risk. SCORE-OP and SCORE2-OP were used to calculate the predicted cardiovascular disease risk of subjects aged more than 70 years.
Outcome
The primary outcome was a composite of cardiovascular disease events, including cardiovascular mortality, nonfatal ischemic heart disease (ICD-10 codes I21-I23), and nonfatal stroke (ICD-10 codes I60-69), during the 10-year period.12 Cardiovascular mortality was defined as death resulting from coronary heart disease, heart failure, stroke, or sudden death as proposed by the original SCORE model.4 The ICD-10 codes of fatal cardiovascular events included hypertensive disease (I10-16), ischemic heart disease (I20-25), arrhythmias or heart failure (I46-52), cerebrovascular disease (I60-69), atherosclerosis or abdominal aortic aneurysms (I70-73), and sudden death within 24 hours of symptom onset (R96.0-96.1). Events encoded under unspecified myocarditis (I51.4), subarachnoid haemorrhage (I60), subdural haemorrhage (I62), cerebral aneurysm (I67.1), and moyamoya disease (I67.5) were excluded from the endpoint. The detailed definitions of outcomes are described in Supplemental Table 1. Deaths caused by noncardiovascular disease were treated as competing events.12 Rehospitalization was not considered for all these outcomes.
Statistical analysis
Baseline characteristics are described as numbers (percentage) for categorical variables and mean ± SD (quartile 1-quartile 3 [Q1-Q3]) for continuous variables. Median (Q1-Q3) was used for non-normally distributed continuous variables.
An extended description of the application of SCORE/SCORE-OP and SCORE2/SCORE2-OP models is presented in the Supplemental Methods and Supplemental Table 2. For discrimination accuracy to evaluate the performance of SCORE/SCORE-OP and SCORE2/SCORE2-OP at a specific prediction time point, Harrell’s concordance index (C-index) was used. For calibration, the same method with the original SCORE2 model was used.12
To assess the calibration performance of the predictive model, we used the predtools R package version 0.0.3 R Foundation) to construct a calibration plot. The calibration plot is a graphic representation that compares the predicted probabilities generated by the model with the observed outcomes. This plot allows for a visual evaluation of the agreement between the predicted and observed probabilities across a range of predicted values. In addition, we presented the plots together for each group SCORE/SCORE2 or SCORE-OP/SCORE2-OP to compare the calibration performance of the 2 models.
All statistical analyses were performed using SAS software version 9.4 (SAS Institute, Inc) and R software version 4.3.0, with a significance level of 5%.
Results
Baseline characteristics
A total of 324,384 participants without a history of cardiovascular disease were enrolled between 2004 and 2005 (Figure 1). The mean age at baseline was 51.4 ± 7.3 years for the SCORE/SCORE2 group and 73.0 ± 2.8 years for the SCORE/SCORE2-OP group (Table 1). The proportion of women was 37.9% in SCORE/SCORE2 group and 47.5% in SCORE/SCORE2-OP group. The median follow-up duration in both groups was 9 years. During the follow-up period, 15.0% of patients in the SCORE/SCORE2 group and 28.9% of patients in the SCORE/SCORE2-OP group experienced cardiovascular events. The SCORE/SCORE2-OP group showed a higher prevalence of diabetes mellitus and had fewer current smokers. There were no significant differences in body mass index (BMI), total cholesterol, and HDL-C levels between the 2 groups.
Figure 1.
Subject Selection
The study selection flow is depicted. From the National Health Insurance Service Health Screening Cohort (NHIS-HEALS), persons who had prevalent cardiovascular disease (CVD) were excluded, and the final population was divided into age-specific groups: 40-49 years, 50-59 years, 60-69 years, 70-79 years, and more than 80 years. Those aged between 40 and 69 years were tested with the Systematic COronary Risk Evaluation (SCORE) and SCORE2 model. Those aged over 70 years old were tested with the Systematic COronary Risk Evaluation-Older Persons (SCORE-OP) and SCORE2-OP model. ∗Subjects aged 40 to 69 years; †subjects aged more than 70 years. NHIS = National Health Insurance Service.
Table 1.
Baseline Characteristics
| SCORE/SCORE2 Validation Groupa (n = 312,381) | SCORE-OP/SCORE2-OP Validation Groupb (n = 12,003) | |
|---|---|---|
| Age, y | 51.4 ± 7.3 | 73.0 ± 2.8 |
| BMI, kg/m2 | 23.9 ± 2.8 | 23.3 ± 3.1 |
| Total cholesterol, mg/dL | 5.1 ± 0.9 | 5.1 ± 1.0 |
| HDL-C, mg/dL | 1.4 ± 0.8 | 1.4 ± 0.8 |
| SBP, mm Hg | 125.5 ± 16.4 | 133.2 ± 18.6 |
| Follow-up, yc | 9.0 (9.0-9.0) | 9.0 (9.0-9.0) |
| Women | 37.9 | 47.5 |
| Diabetes mellitus | 9.4 | 15.1 |
| Current smoker | 22.6 | 12.3 |
Values are mean ± SD, median (Q1-Q3), or %.
BMI = body mass index; HDL-C = high density lipoprotein cholesterol; Q = quartile; SBP = systolic blood pressure; SCORE = Systematic COronary Risk Evaluation; SCORE-OP = Systematic COronary Risk Evaluation-Older Persons.
Subjects aged 40-69 years.
Subjects aged more than 70 years.
Follow-up duration ranged from 7.0 to 12.0 years for SCORE/SCORE2 and from 5.0 to 11.0 years for SCORE-OP/SCORE2-OP.
Observed and predicted 10-year cardiovascular disease risk using the SCORE/SCORE2 model
The observed cardiovascular disease risk and predicted risk using the SCORE and SCORE2 models are shown in Figure 2 and Supplemental Table 3. In men, the SCORE model underestimated cardiovascular disease risk in the young age group (40-49 years) and overestimated it when the age was more than 50 years. In women, the SCORE model underestimated cardiovascular disease risk in those aged 40 to 59 years but overestimated it among those aged 60 to 69 years. The SCORE-OP models overestimated cardiovascular disease risk both in men and women.
Figure 2.
Comparison of Observed and Predicted 10-Year Risks
The calculated predicted 10-year risk using SCORE, SCORE2, SCORE-OP, and SCORE2-OP was compared with the observed risk. The newly developed SCORE2/SCORE2-OP models and the previous SCORE/SCORE-OP models both tended to overestimate cardiovascular disease risk. There was an underestimation in persons aged <60 years in the SCORE model. Abbreviations as in Figure 1.
In contrast, SCORE2 and SCORE2-OP invariably overestimated the risk in all age groups of both sexes. Moreover, the degree of overestimation was greater than with SCORE/SCORE-OP.
The calibration plots for 10-year risk in different subpopulations are depicted in Figures 3A to 3D. In the calibration plots, SCORE2 appeared to overestimate the risk, whereas SCORE appeared to underestimate it. Additionally, SCORE2 demonstrated slightly better predictability compared with SCORE. However, both SCORE-OP and SCORE2-OP consistently overestimated the risk in all subpopulations.
Figure 3.
Calibration Plots
In the calibration plots for 10-year risk, SCORE showed an underestimation compared with an overestimation in SCORE2. However, the predictability of SCORE-OP and SCORE2-OP did not differ significantly. Calibration plots of (A) 40- to 69-year-old men, (B) 40- to 69-year-old women, (C) 70+-year-old men, and (D) 70+-year-old women. Abbreviations as in Figure 1.
Harrell’s C-index evaluating adequacy of the SCORE/SCORE2 model
Harrell’s C-index of each model is presented in Table 2. The C-index was similar for both SCORE2 and SCORE, but it was higher for SCORE (men, 0.607; women, 0.626) than for SCORE2 (men, 0.604; women, 0.625). However, SCORE2-OP and SCORE-OP models showed a similar but slightly higher C-index for SCORE2-OP (SCORE2-OP: men, 0.521; women, 0.508; SCORE-OP: men, 0.518; women, 0.507). When the C-index was evaluated according to age and sex, the trend was consistent in both men and women (Supplemental Table 2).
Table 2.
Observed and Predicted 10-Year Cardiovascular Disease Risk in the Korean Population
| n | Predicted 10-Year CVD Risk, % | Observed 10-Year CVD Risk, % | O/P Ratio | Harrell’s C-Index (95% CI) | |
|---|---|---|---|---|---|
| SCORE2 | |||||
| Men | 193,934 | 4.31 | 1.94 | 0.45 | 0.604 (0.601-0.608) |
| Women | 118,447 | 2.39 | 0.90 | 0.38 | 0.625 (0.620-0.629) |
| SCORE2-OP | |||||
| Men | 6,301 | 12.50 | 3.68 | 0.29 | 0.521 (0.507-0.534) |
| Women | 5,702 | 10.41 | 2.06 | 0.23 | 0.508 (0.493-0.523) |
| SCORE | |||||
| Men | 193,934 | 1.70 | 1.94 | 1.14 | 0.607 (0.604-0.611) |
| Women | 118,447 | 0.68 | 0.90 | 1.32 | 0.626 (0.622-0.630) |
| SCORE-OP | |||||
| Men | 6,301 | 18.53 | 3.68 | 0.26 | 0.518 (0.505-0.532) |
| Women | 5,702 | 8.83 | 2.06 | 0.23 | 0.507 (0.492-0.521) |
C-index = concordance index; CVD = cardiovascular disease; O = observed; P = predicted; other abbreviations as in Table 1.
Discussion
In this study, we compared the observed cardiovascular disease rate and predicted risk from the SCORE/SCORE-OP and SCORE2/SCORE2-OP models in an East Asian nationwide population cohort of Korea. The SCORE model overestimated the risk in both sexes more than the age of 60 years, and the SCORE2 model consistently overestimated the risk in both middle-aged and older adults. The SCORE-OP and SCORE2-OP models overestimated the risk in both sexes (Central Illustration). The C-index were not high enough even in the most predictive group (female 40-69 years old); therefore, these models are not effective in predicting the cardiovascular disease risk of Koreans. This is the first study to validate the newly developed SCORE2 models in an East Asian population.
Central Illustration.
Systematic COronary Risk Evaluation and Systematic COronary Risk Evaluation 2 Models in the Korean Population
This study validated the predictability of Systematic COronary Risk Evaluation (SCORE), Systematic COronary Risk Evaluation-Older Persons (SCORE-OP), SCORE2, and SCORE2-OP models using 324,384 persons from the National Health Insurance Health Screening Cohort between 2004 and 2015. The coefficient for a low-risk region was used to calculate the predicted risk in the SCORE and SCORE2 models and was compared with the risks from the actual observed risk. Both fatal and nonfatal cardiovascular disease (CVD) was used to count the observed cardiovascular disease risk in both SCORE and SCORE2 models. Harrell’s concordance index (C-index) was used for discrimination accuracy to evaluate the performance of SCORE/SCORE-OP and SCORE2/SCORE2-OP at a specific prediction time point, and calibration performance was also compared. In the Korean population, the newly developed SCORE2/SCORE2-OP models and the previous SCORE/SCORE-OP models both tended to overestimate cardiovascular disease risk, although the newer version showed a slight improvement compared with the older one.
The previously developed SCORE model has been validated in various European countries.18, 19, 20, 21, 22, 23 In Asian countries, only 1 study was done on validation, and it used a Malaysian population.9 Not only validation, but also recalibration of the SCORE model has been performed for better adaptation in different countries.8,24,25 However, the SCORE model has not been properly validated in the Korean population except for the study done on patients with type 2 diabetes mellitus in predicting carotid atherosclerosis and peripheral artery disease.14 This study is the first to address the predictive performance of cardiovascular disease risk by using the SCORE and SCORE2 models in the Korean population.
Although the SCORE model seemed acceptable in terms of overall performance, it showed a slight discrepancy in overestimation in some countries19,20 and underestimation in others.9,21 In this study, application of the SCORE model to the Korean population showed underestimation in those <60 years of age and overestimation in those more than 60 years of age. The underestimation of the SCORE model in participants <60 years of age may be attributable to the current study design. This study counted both fatal and nonfatal events for the observed risks even for the SCORE model, when it was originally designed to predict only fatal cardiovascular disease risk and not nonfatal cardiovascular disease risk.4 Thus, the actual observed risk of only fatal cardiovascular disease events may be lower than the observed cardiovascular disease risk used in the present study. In addition, although our study excluded those persons with prevalent cardiovascular disease, there is a possibility that those who did not have a current diagnosis of cardiovascular diseases but were at high risk and receiving preventive treatment may have been included in the study population. These reasons may have contributed to underestimating the cardiovascular disease risk by the SCORE model in participants <60 years of age.
Considering the limitation mentioned earlier, the clinical implication of underestimation in the SCORE model among participants aged <60 years is not high. Thus, it seems reasonable to construe that both SCORE and SCORE2 invariably overestimated cardiovascular disease risk. This overestimation may have occurred because of the decreasing prevalence of cardiovascular disease risk factors among Koreans. The study population used in this study was based on the years 2004 and 2005. In Korea, the prevalence of hypertension was decreased in both men (32.5% to 31.5%) and women (26.9% to 24.3%) from 1998 to 2012 and remained stable until 2021.26 Smoking rates decreased only in men. In contrast, the prevalence rate of obesity,27 diabetes mellitus,28 and hypercholesterolemia29 was significantly increased during the same period. However, the management indicators, such as the awareness rate, treatment rate, and control rate of cardiovascular disease risk factors, have improved continuously and have therefore overtaken the increase in incidence.27,30,31 For example, the awareness, treatment, and control rate of hypertension and hypercholesterolemia increased by 2- to 3-fold in the 1998-2018 Korea National Health and Nutrition Examination Survey,31 which could attenuate the impact of risk factors on the incidence and mortality of cardiovascular disease. In Korea, cardiovascular disease mortality was as high as 1,659 per 100,000 persons in 1983 and decreased to 109.2 in 2009.17 This steadily decreasing trend of cardiovascular disease risk factors in the Korean population may be one of the causes of overestimation.
Although both SCORE and SCORE2 models showed overestimation, there were differences in the degree of overestimation between the SCORE/SCORE2 and SCORE-OP/SCORE2-OP models. When comparing the SCORE and SCORE2 models, the degree of overestimation was worse in SCORE2. However, in the case of SCORE-OP and SCORE2-OP, the overestimating degree was lower in SCORE2-OP than that in the SCORE-OP model except for women aged 70 to 79 years. This discrepancy may reflect the slight difference in the cardiovascular disease risk prevalence trend among young and older adults. Regarding hypertension, although the overall age-standardized prevalence has decreased over 20 years, the prevalence has increased among those more than age 70 years.32 Similar to hypertension, the prevalence of diabetes showed an increasing trend only among people aged more than 70 years.33 The crude incidence of myocardial infarction also increased as a result of an aging population, but the age-adjusted incidence remained unchanged.34 These modest differences may have affected the different performance of SCORE2 and SCORE2-OP models.
One of the interesting results of this study is that SCORE2 did not improve applicability of the SCORE model in the Korean population. Although the calibration plot showed better predictability when using SCORE2 compared with SCORE, the performance of SCORE-OP and SCORE2-OP did not differ. The cause of this discrepancy may be attributable to the following reasons. This study used the observed risk of both fatal and nonfatal risk, even for the SCORE model. This approach may have resulted in a relatively higher C-index of SCORE than the actual. Another possible explanation is the difference in the proportion of risk factors attributable to cardiovascular disease. Although high blood pressure remains the leading risk factor for cardiovascular disease–related death globally, the relative importance of high BMI and high total cholesterol is lower in the Korean population compared with that in European countries.35 South Korea showed the second lowest combined effect of high blood pressure, serum cholesterol, blood glucose, and BMI on cardiovascular disease mortality.35 Because high cholesterol and BMI were associated with death from ischemic heart disease and hemorrhagic stroke, respectively,35 a lower impact of these factors would have resulted in overestimation in the Korean population. These differences in significance of each risk factor would have affected not only the mortality but also the morbidity of cardiovascular disease and nonfatal cardiovascular disease events, thus leading to nonsuperiority of the SCORE2 compared with the SCORE model.
Although Korea can be considered a low-risk country, considering the annual cardiovascular disease mortality reported by the World Health Organization, the use of a low-risk model still showed a divergence between the predicted and observed values. Such overestimation among low-risk countries has also been shown in Austria.36 Considering the limitations within the current SCORE2/SCORE2-OP model, we attempted to propose a new coefficient for the Korean population. Unfortunately, we did not achieve a satisfactory result by adjusting the coefficient alone. This finding underscores the imperative need for further modification or the development of an entirely novel model tailored specifically to the Asian population.
Considering the unique risk factor profile within the Asian population, previous studies not only recalibrated Western models but also developed novel models tailored specifically to Asian demographics.37, 38, 39 These recently established models include the Prediction for ASCVD Risk in China (China-PAR) and the Japan Arteriosclerosis Longitudinal Study (JALS) score.37,38 The China-PAR model, designed as a sex-specific equation, exhibited C-statistics of 0.794 for men and 0.811 for women.37 Similarly, the JALS score demonstrated an area under the curve of 0.828 for the model without atrial fibrillation and 0.832 for the model with atrial fibrillation.38 Given the reasonably sound predictive performance of both risk scoring systems in the Asian population, further studies are warranted to facilitate a direct comparison with the SCORE2 models.
Study limitations
First, even though we aimed to establish a rigorous baseline, given the nature of our data tracked until 2015, the median follow-up duration was 9.0 years, with a Q1-Q3 of 9.0-9.0 years, which was shorter than the 10 years proposed in SCORE and SCORE2 models. This issue could potentially influence the generalizability of the results. A subsequent comparative study with a 10-year follow-up duration is needed. Second, the number of participants more than 80 years of age was relatively small compared with other age groups. Thus, further evaluation in a larger population is needed. Third, the SCORE-OP was originally developed for those aged up to 80 years. The application of SCORE-OP in participants more than 80 years of age, as in this study, could be an extrapolation of the scoring system without firm evidence. Fourth, because HDL-C has been included in the National Health Examination since 2009, the gap between that and other baseline characteristics, which were mostly collected between 2004 and 2005, may have affected the results. Fifth, observed cardiovascular risk consisting both fatal and nonfatal cardiovascular disease risk was also used for comparison in the SCORE model. Given that the SCORE model was intended to estimate only fatal cardiovascular disease risk, the actual observed risk of only fatal cardiovascular disease events would be lower than the observed cardiovascular disease risk assessed in this study. Thus, a head-to-head comparison of the C-index in SCORE and SCORE2 models should be performed with caution. Sixth, although the authors of the original SCORE2 article imputed missing data, our study, in contrast, excluded missing data and analyzed the available data. This may have caused a bias and reduction of generalizability of the results. Finally, cardiovascular disease mortality steadily increased up to 123 per 100,000 persons from 2010 to 2018 in Korea. A continuous increase in cardiovascular disease mortality appears to be unavoidable in Korea with an aging population.17 Therefore, we cannot assume that overestimation of cardiovascular disease risk will continue in the future.
Conclusions
The newly developed SCORE2/SCORE2-OP models and the previous SCORE/SCORE-OP models both tended to overestimate cardiovascular disease risk, although the newer version showed a slight improvement compared with the older models. It is evident that a further modification or a new model tailored to the East Asian population is necessary.
Perspectives.
COMPETENCY IN PATIENT CARE AND PROCEDURAL SKILLS: The proposed SCORE2 models serve to assess the 10-year risk of both fatal and nonfatal cardiovascular disease, thereby representing an update from the previous SCORE models, which predicted only the 10-year risk of fatal cardiovascular disease.
TRANSLATIONAL OUTLOOK: The validation of SCORE models and SCORE2 models in the East Asian population revealed that both the previous and new versions tended to overestimate cardiovascular disease risk, although the new SCORE2 model showed a slight improvement over the previous model.
Funding Support and Author Disclosures
This study was supported by the Korean Society of Cardiovascular Disease Prevention (grant number, 2021-01). Dr Williams has received research grants from Omron, Japan, and Vascular Dynamics, Inc; and has received lecture honoraria from Daiichi-Sankyo, Pfizer, Boehringer Ingelheim, and Servier. Dr Lee has received institutional grant support from the Korean Society of Cardiovascular Disease Prevention for this work. No fees were received personally by any author. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
Acknowledgment
The authors would like to thank the National Health Insurance Service for approval of the data use.
Footnotes
The authors attest they are in compliance with human studies committees and animal welfare regulations of the authors’ institutions and Food and Drug Administration guidelines, including patient consent where appropriate. For more information, visit the Author Center.
Appendix
For an expanded Methods section and for supplemental tables and references, please see the online version of this paper.
Appendix
References
- 1.World Health Organization Global health estimates: leading causes of death. https://www.who.int/data/gho/data/themes/mortality-and-global-health-estimates/ghe-leading-causes-of-death
- 2.Anderson K.M., Odell P.M., Wilson P.W., Kannel W.B. Cardiovascular disease risk profiles. Am Heart J. 1991;121(1 Pt 2):293–298. doi: 10.1016/0002-8703(91)90861-b. [DOI] [PubMed] [Google Scholar]
- 3.Goff D.C., Jr., Lloyd-Jones D.M., Bennett G., et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25 part B):2935–2959. doi: 10.1016/j.jacc.2013.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Conroy R.M., Pyörälä K., Fitzgerald A.P., et al. Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003;24(11):987–1003. doi: 10.1016/s0195-668x(03)00114-3. [DOI] [PubMed] [Google Scholar]
- 5.Cooney M.T., Selmer R., Lindman A., et al. Cardiovascular risk estimation in older persons: SCORE O.P. Eur J Prev Cardiol. 2016;23(10):1093–1103. doi: 10.1177/2047487315588390. [DOI] [PubMed] [Google Scholar]
- 6.Goh L.G., Welborn T.A., Dhaliwal S.S. Independent external validation of cardiovascular disease mortality in women utilising Framingham and SCORE risk models: a mortality follow-up study. BMC Womens Health. 2014;14:118. doi: 10.1186/1472-6874-14-118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Chen L., Tonkin A.M., Moon L., et al. Recalibration and validation of the SCORE risk chart in the Australian population: the AusSCORE chart. Eur J Cardiovasc Prev Rehabil. 2009;16(5):562–570. doi: 10.1097/HJR.0b013e32832cd9cb. [DOI] [PubMed] [Google Scholar]
- 8.Hense H.W., Koesters E., Wellmann J., Meisinger C., Völzke H., Keil U. Evaluation of a recalibrated Systematic Coronary Risk Evaluation cardiovascular risk chart: results from Systematic Coronary Risk Evaluation Germany. Eur J Cardiovasc Prev Rehabil. 2008;15(4):409–415. doi: 10.1097/HJR.0b013e3282feec66. [DOI] [PubMed] [Google Scholar]
- 9.Selvarajah S., Kaur G., Haniff J., et al. Comparison of the Framingham risk score, SCORE and WHO/ISH cardiovascular risk prediction models in an Asian population. Int J Cardioy. 2014;176(1):211–218. doi: 10.1016/j.ijcard.2014.07.066. [DOI] [PubMed] [Google Scholar]
- 10.Szyndler A., Kucharska W.A., Dubiela D.B.A.E., et al. SCORE model underestimates cardiovascular risk in hypertensive patients: results of the Polish Hypertension Registry. Blood Press. 2011;20(6):342–347. doi: 10.3109/08037051.2011.587267. [DOI] [PubMed] [Google Scholar]
- 11.Jdanov D.A., Deev A.D., Jasilionis D., Shalnova S.A., Shkolnikova M.A., Shkolnikov V.M. Recalibration of the SCORE risk chart for the Russian population. Eur J Epidemiol. 2014;29(9):621–628. doi: 10.1007/s10654-014-9947-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.SCORE2 working group and ESC Cardiovascular risk collaboration. SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe. Eur Heart J. 2021;42(25):2439–2454. doi: 10.1093/eurheartj/ehab309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.SCORE2-OP working group and ESC Cardiovascular risk collaboration. SCORE2-OP risk prediction algorithms: estimating incident cardiovascular event risk in older persons in four geographical risk regions. Eur Heart J. 2021;42(25):2455–2467. doi: 10.1093/eurheartj/ehab312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Ahn H.R., Shin M.H., Yun W.J., et al. Comparison of the Framingham risk score, UKPDS risk engine, and SCORE for predicting carotid atherosclerosis and peripheral arterial disease in Korean type 2 diabetic patients. Korean J Fam Med. 2011;32(3):189–196. doi: 10.4082/kjfm.2011.32.3.189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Seong S.C., Kim Y.Y., Park S.K., et al. Cohort profile: the National Health Insurance Service-National Health Screening Cohort (NHIS-HEALS) in Korea. BMJ Open. 2017;7(9) doi: 10.1136/bmjopen-2017-016640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Health at a Glance: Asia/Pacific 2020. OECD; 2020. Organisation for Economic Co-operation and Development (OECD), World Health Organization. [Google Scholar]
- 17.Lee H.H., Cho S.M.J., Lee H., et al. Korea heart disease fact sheet 2020: analysis of nationwide data. Korean Circ J. 2021;51(6):495–503. doi: 10.4070/kcj.2021.0097. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Aspelund T., Thorgeirsson G., Sigurdsson G., Gudnason V. Estimation of 10-year risk of fatal cardiovascular disease and coronary heart disease in Iceland with results comparable with those of the Systematic Coronary Risk Evaluation project. Eur J Cardiovasc Prev Rehabil. 2007;14(6):761–768. doi: 10.1097/HJR.0b013e32825fea6d. [DOI] [PubMed] [Google Scholar]
- 19.Lindman A.S., Veierød M.B., Pedersen J.I., Tverdal A., Njølstad I., Selmer R. The ability of the SCORE high-risk model to predict 10-year cardiovascular disease mortality in Norway. Eur J Cardiovasc Prev Rehabil. 2007;14(4):501–507. doi: 10.1097/HJR.0b013e328011490a. [DOI] [PubMed] [Google Scholar]
- 20.van Dis I., Kromhout D., Geleijnse J.M., Boer J.M., Verschuren W.M. Evaluation of cardiovascular risk predicted by different SCORE equations: the Netherlands as an example. Eur J Cardiovasc Prev Rehabil. 2010;17(2):244–249. doi: 10.1097/HJR.0b013e328337cca2. [DOI] [PubMed] [Google Scholar]
- 21.Vikhireva O., Pajak A., Broda G., et al. SCORE performance in Central and Eastern Europe and former Soviet Union: MONICA and HAPIEE results. Eur Heart J. 2014;35(9):571–577. doi: 10.1093/eurheartj/eht189. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Jørstad H.T., Colkesen E.B., Minneboo M., et al. The Systematic COronary Risk Evaluation (SCORE) in a large UK population: 10-year follow-up in the EPIC-Norfolk prospective population study. Eur J Prev Cardiol. 2015;22(1):119–126. doi: 10.1177/2047487313503609. [DOI] [PubMed] [Google Scholar]
- 23.Piotrowski W., Waśkiewicz A., Cicha-Mikołajczyk A. Global cardiovascular mortality risk in the adult Polish population: prospective assessment of the cohorts studied in multicentre national WOBASZ and WOBASZ Senior studies. Kardiol Pol. 2016;74(3):262–273. doi: 10.5603/KP.a2015.0175. [DOI] [PubMed] [Google Scholar]
- 24.Panagiotakos D.B., Georgousopoulou E.N., Fitzgerald A.P., Pitsavos C., Stefanadis C. Validation of the HellenicSCORE (A Calibration of the ESC SCORE Project) regarding 10-year risk of fatal cardiovascular disease in Greece. Hellenic J Cardiol. 2015;56(4):302–308. [PubMed] [Google Scholar]
- 25.Karjalainen T., Adiels M., Björck L., et al. An evaluation of the performance of SCORE Sweden 2015 in estimating cardiovascular risk: The Northern Sweden MONICA Study 1999-2014. Eur J Prev Cardiol. 2017;24(1):103–110. doi: 10.1177/2047487316673142. [DOI] [PubMed] [Google Scholar]
- 26.Kim H.C., Lee H., Lee H.H., et al. Korea hypertension fact sheet 2021: analysis of nationwide population-based data with special focus on hypertension in women. Clin Hypertens. 2022;28(1):1. doi: 10.1186/s40885-021-00188-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Kim H.J., Kim Y., Cho Y., Jun B., Oh K.W. Trends in the prevalence of major cardiovascular disease risk factors among Korean adults: results from the Korea National Health and Nutrition Examination Survey, 1998-2012. Int J Cardiol. 2014;174(1):64–72. doi: 10.1016/j.ijcard.2014.03.163. [DOI] [PubMed] [Google Scholar]
- 28.Hong Y.H., Chung I.H., Han K., Chung S. Prevalence of type 2 diabetes mellitus among Korean children, adolescents, and adults younger than 30 years: changes from 2002 to 2016. Diabetes Metab J. 2022;46(2):297–306. doi: 10.4093/dmj.2021.0038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cho S.M.J., Lee H., Lee H.H., et al. Dyslipidemia fact sheets in Korea 2020: an analysis of nationwide population-based data. J Lipid Atheroscler. 2021;10(2):202–209. doi: 10.12997/jla.2021.10.2.202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Roh E., Ko S.H., Kwon H.S., et al. Prevalence and management of dyslipidemia in Korea: Korea National Health and Nutrition Examination Survey during 1998 to 2010. Diabetes Metab J. 2013;37(6):433–449. doi: 10.4093/dmj.2013.37.6.433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kim Y., Nho S.J., Woo G., et al. Trends in the prevalence and management of major metabolic risk factors for chronic disease over 20 years: findings from the 1998-2018 Korea National Health and Nutrition Examination Survey. Epidemiol Health. 2021;43 doi: 10.4178/epih.e2021028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Kang S.H., Kim S.H., Cho J.H., et al. Prevalence, awareness, treatment, and control of hypertension in Korea. Sci Rep. 2019;9(1) doi: 10.1038/s41598-019-46965-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Shin J.Y. Trends in the prevalence and management of diabetes in Korea: 2007-2017. Epidemiol Health. 2019;41 doi: 10.4178/epih.e2019029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Kim H.C. Epidemiology of cardiovascular disease and its risk factors in Korea. Glob Health Med. 2021;3(3):134–141. doi: 10.35772/ghm.2021.01008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Global Burden of Metabolic Risk Factors for Chronic Diseases Collaboration Cardiovascular disease, chronic kidney disease, and diabetes mortality burden of cardiometabolic risk factors from 1980 to 2010: a comparative risk assessment. Lancet Diabetes Endocrinol. 2014;2(8):634–647. doi: 10.1016/s2213-8587(14)70102-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Ulmer H., Kollerits B., Kelleher C., Diem G., Concin H. Predictive accuracy of the SCORE risk function for cardiovascular disease in clinical practice: a prospective evaluation of 44 649 Austrian men and women. Eur J Cardiovasc Prev Rehabil. 2005;12(5):433–441. doi: 10.1097/01.hjr.0000174791.47059.80. [DOI] [PubMed] [Google Scholar]
- 37.Yang X., Li J., Hu D., et al. Predicting the 10-year risks of atherosclerotic cardiovascular disease in chinese population: the China-PAR Project (Prediction for ASCVD Risk in China) Circulation. 2016;134(19):1430–1440. doi: 10.1161/circulationaha.116.022367. [DOI] [PubMed] [Google Scholar]
- 38.Harada A., Ueshima H., Kinoshita Y., et al. Absolute risk score for stroke, myocardial infarction, and all cardiovascular disease: Japan Arteriosclerosis Longitudinal Study. Hypertens Res. 2019;42(4):567–579. doi: 10.1038/s41440-019-0220-z. [DOI] [PubMed] [Google Scholar]
- 39.Barzi F., Patel A., Gu D., et al. Cardiovascular risk prediction tools for populations in Asia. J Epidemiol Community Health. 2007;61(2):115–121. doi: 10.1136/jech.2005.044842. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.





