Recalibration of the SCORE risk chart for the Russian population

Abstract

Persisting high levels of cardiovascular mortality in Russia present a specific case among developed countries. Application of cardiovascular risk prediction models holds great potential for primary prevention in this country. Using a unique set of cohort follow-up data from Moscow and Saint Petersburg, this study aims to test and recalibrate the Systematic Coronary Risk Evaluation (SCORE) methods for predicting CVD mortality risks in the general population. The study is based on pooled epidemiological cohort data covering the period 1975–2001. The algorithms from the SCORE project were used for the calibration of the SCORE equation for the Moscow and St. Petersburg populations (SCORE-MoSP). Age-specific 10-year cumulative cardiovascular mortality rates were estimated according to the original SCORE-High and SCORE-Low equations and compared to the estimates based on the recalibrated SCORE-MoSP model and observed CVD mortality rates. Ten-year risk prediction charts for CVD mortality were derived and compared using conventional SCORE-High and recalibrated SCORE-MoSP methods. The original SCORE-High model tends to substantially under-estimate 10-year cardiovascular mortality risk for females. The SCORE-MoSP model provided better results which were closer to the observed rates. For males, both the SCORE-High and SCORE-MoSP provided similar estimates which tend to under-estimate CVD mortality risk at younger ages. These differences are also reflected in the risk prediction charts. Using non-calibrated scoring models for Russia may lead to substantial underestimation of cardiovascular mortality risk in some groups of individuals. Although the SCORE-MoSP provide better results for females, more complex scoring methods involving a wider range of risk factors are needed.

Introduction

It has long been recognised that assessment of risk of cardiovascular diseases based on a single risk factor can be misleading and often lead to wrong strategies of medical interventions and treatments [1]. Since the 1950s, various multivariate risk assessment (risk scoring) methods allowing assessment of a total risk of cardiovascular diseases for an individual within a fixed time period in future have been widely applied for different populations [2].

The most widely used risk scoring methods are the Framingham Score based on the Framingham cohort study in the USA and the Systematic Coronary Risk Evaluation (SCORE) study based on pooled data from various European countries [3–5]. However, it has been shown that risk scoring methods are not universal and in many cases recalibrations are needed in order to take into account specifics of different populations or even sub-populations [2, 6]. For example, the SCORE equations accounting for differences in the baseline survival for cardiovascular system diseases by introducing separate equations for low- and high-risk populations also cannot provide universal solutions. Using the SCORE-high equation (designed for countries with high CVD mortality) leads to overestimation of cardiovascular mortality risk in some “high” risk countries such as the Netherlands or Germany and underestimation of this risk in some “very high” CVD mortality countries such as the former USSR countries [7–10]. These findings have important implications because clinical decisions based on these widely used scoring methods may lead to wrong decisions and sub-optimal treatment strategies.

In terms of long-term changes and levels of cardiovascular mortality, Russia presents a specific case among developed countries. Unlike Western countries, in Russia cardiovascular mortality have been stagnating at very high levels since the mid-1960s [11–14]. Very high levels of cardiovascular mortality observed during the 1970s and 1980s persisted up today (Shkolnikov et al. 2013).

Experience of other countries show that the burden cardiovascular diseases can be substantially reduced by introducing modern treatment technologies and preventive measures. Correctly identifying people with an elevated risk of cardiovascular disease at early stages and choosing appropriate cost-effective treatment would be one of the key steps of appropriate preventive strategies for Russia. The application of cardiovascular risk prediction models holds a great potential in this case.

Some prior studies have applied risk algorithms for construction of aggregated risk scales on the basis of the Russian data [15, 16]. However, the knowledge about the quality of performance of CVD risk assessment algorithms in Russia is still scarce and contradictory. A recent study on Central and Eastern Europe, including analysis of data from the MONICA and HAPIEE surveys conducted in Novosibirsk city (Russia), shows that using the SCORE-high algorithm leads to underestimation of 10 year CVD mortality risk in the MONICA-Novosibirsk sample, whereas in the HAPIEE-Novosibirsk sample it provides accurate estimates [10]. Our study advances the prior research in four important respects. First, it uses pooled data from seven epidemiological cohorts from Moscow and Saint Petersburg representing the largest dataset on CVD risk factors ever collected in Russia. Second, the study applies a canonic SCORE algorithm using exactly the same estimation techniques. Third, we systematically evaluate performance of the general SCORE algorithm by comparing the observed and predicted (using original and recalibrated SCORE algorithms) CVD mortality rates. Fourth, two sets of CVD mortality prediction charts are produced, allowing depiction of the differences between the results derived using the original (SCORE-High) and recalibrated SCORE algorithms. Understanding the patterns and nature of these differences is one of the key steps in developing more comprehensive prediction models which reproduce the Russian reality more accurately.

Data and methods

Study population

Over the past three decades, Russia has taken part in several major international projects (including those coordinated by the WHO) examining risk factors of cardiovascular diseases. Most of the data used in this study stem from those internationally recognized studies and represent the core of available evidence and epidemiologic data ever collected in this country.

This study uses unique data pooled from seven epidemiological cohorts of the residents of Moscow and St. Petersburg who were followed between the 1970s and the 2000s. These cohorts tracked by the State Research Center for Preventive Medicine (SRCPM) in Moscow include two cohorts of the Lipid Research Clinics Project (LRC1 and LRC2), four cohorts of the WHO MONICA project (Moscow) (MONICA 1, MONICA 2, MONICA 3, and MONICA 4), and one cohort from the “Shigan” (Moscow) study [17–19]. The first cohort (LRC1) was set up in 1975–1977 as a part of the USA-USSR Lipid Research Clinics Project and included 7,815 men from two typical residential districts in Moscow and St. Petersburg. The members of this cohort randomly drawn from voters’ registration lists were medically tested and interviewed in 1975–1977 according to the North American LRC Prevalence Study Protocol (US-USSR Steering Committee, 1977). The LRC2 cohort (N = 4,772) was recruited in 1979–1982 and included both males and females from Moscow and only females from St. Petersburg. Four waves of the WHO MONICA surveys (set up in 1983–1986 (N = 3,973), 1988–1989 (N = 1,251), 1992–1995 (N = 1,103), and 2000–2001 (N = 810)) were conducted by the WHO collaboration center in Moscow (SRCPM) following the WHO MONICA protocols (WHO 1999). Finally, the “Shigan” epidemiological survey (N = 1,171) was recruited in 1979–1980 and was conducted using similar international guidelines (Shigan 1993).

The pooled cohort data were divided into the 10 year follow-up (observation) periods for the first six cohorts (LRC1, LRC2, MONICA 1-MONICA 3, and “Shigan”). The most recent period based on the MONICA 4 data was slightly shorter (2000–2001—December 31, 2009). Person years of exposure were defined as the total lifetime lived by each individual from the point of entrance to one of the seven cohorts until the date of death, migration out of the city or censoring after 10-years of follow-up or on the 31st of December 2009. The structure of the dataset using pooled cohort data with respect to calendar time and age is shown in the Fig. 1.

Fig. 1.

Regions of observation for seven Russian cohorts on the Lexis diagram

The cohort data used in this study are representative in terms of mortality and population composition by age and socio-demographic status of the two largest Russian cities, Moscow and St. Petersburg [20]. However, they are not representative for Russia as a whole. In particular, the educational structure of the study population is notably shifted upwards. Therefore, statistical estimations of risk charts took into account age- and education-specific weights based on the corresponding population structure of Moscow taken from the 1989 census.

Due to the exclusion of females in the largest (LRC1) cohort and a very small female share (16 %) in the third largest (MONICA 1) cohort, the overall percentage of females is only 29 % (Table 1). Following the SCORE algorithm, the study population was restricted to those aged 45–64. The major characteristics of these cohorts are presented in the Table 1.

Table 1.

Characteristics of epidemiologic cohort studies included in the pooled dataset for Moscow and St. Petersburg

Study	Study population	Years recruited	Response rate (%)	Participants at baseline (women %)	Age range	Major ECG abnormalities (%)	Participants aged 45–64 (women %) at baseline
LRC1	Moscow/St Petersburg	1975–1977	77.8	7,815 (0.0)	36–62	274 (3.5)	5,801 (0.0)
LRC2	Moscow/St Petersburg	1979–1982	72.1	4,772 (74.1)	20–72	276 (5.8)	1,864 (76.4)
“Shigan” study	Moscow	1989–1991	71.9	1,171 (37.2)	20–82	73 (6.2)	482 (32.6)
MONICA 1	Moscow	1983–1986	73.0	3,973 (16.4)	19–71	164 (4.1)	1,862 (23.9)
MONICA 2	Moscow	1988–1989	72.1	1,251 (48.3)	30–69	64 (5.1)	807 (47.0)
MONICA 3	Moscow	1992–1985	64.3	1,103 (49.1)	20–76	52 (4.7)	645 (52.4)
MONICA 4	Moscow	2000–2001	64.2	810 (47.5)	22–71	67 (8.3)	514 (52.9)
Total	Moscow/St Petersburg	1975–2001	73.5	20,895 (29.4)	19–82	970 (4.6)	13,291 (22.7)

Subjects at risk of heart attack (as defined by the presence of major abnormalities on resting ECG such as arrhythmia, QQS, and ST-T abnormalities) were excluded. Thus, the final sample includes 689 cardiovascular deaths, and 103.7 thousands person years of exposure to risk. Cardiovascular mortality was defined using the following ICD8 codes: 401 through 419 and 427 through 443. These codes generally correspond to the ICD9 codes used in the original SCORE study [4]. Unlike in the original SCORE algorithm, it was not possible here to exclude the following causes of death coded in the four digits (codes are provided in the ICD9): 426.7, 429.0, 430.0, 432.1, 437.3, 437.4, and 437.5. However, these causes of death refer only to very small numbers of deaths and thus cannot influence the results.

The procedures of collection of data on conventional risk factors were the same in all aforementioned epidemiological surveys. All the explanatory variables were measured at the beginning of the follow-up and were categorized on the basis of experience of prior analyses of the LRC data and relevant international studies in cardiovascular epidemiology [4, 21, 22].

Statistical methods

The guidelines and algorithm from the SCORE project were taken as the basis for the calibration of the SCORE equation for the Moscow and St. Petersburg population (SCORE-MoSP) and for the calculations of predicted risks according to the original SCORE-Low and SCORE-High equations. For estimation of risks based on the original SCORE-Low and SCORE-High equations, the baseline survival coefficients and b-coefficients showing effects of smoking, total cholesterol and systolic blood pressure from the SCORE project were used [4]. For the SCORE-MoSP, the corresponding coefficients were estimated from the pooled cohort dataset for Moscow and St. Petersburg as described in the previous section. The exact steps of calculation of 10-year mortality risk estimates for the SCORE-MoSP are described in Appendix 1 (Supplementary Appendix). Following the SCORE guidelines, we used the Weibull proportional hazards model stratified by sex with adjustment for the epidemiological cohort. As a duration variable, age rather than time under observation was used. This reduces the bias of the estimated coefficients and does not produce any distortions [23]. All results were cross-checked using classical Cox proportional hazard model which returned almost identical results.

There is one important difference between the original SCORE and SCORE-MoSP equations: baseline survival coefficients based on the adapted SCORE-MoSP equation were calculated only for all cardiovascular system diseases; not distinguishing between coronary heart diseases and non-coronary cardiovascular diseases. This modification was introduced because evidence from data quality studies in Russia suggests that, although cardiovascular diseases as a group are reported correctly, there is a notable risk of misclassification of more detailed causes inside this group [24]. Prior validation studies have shown that there was an over-registration of deaths due to cerebrovascular diseases and coronary atherosclerosis, and an under-registration of deaths due to hypertension and myocardial infarction which cannot affect estimates for the entire aggregated CVD category [23–26].

Results

The total population included in the pooled cohort data for Moscow and St. Petersburg (SCORE-MoSP) was 13,291 persons, 77 % male and 23 % female. The major bio-demographic characteristics of the total study population are presented in Table 2. With exception of the prevalence of smoking and total amount of CVD deaths, the selected characteristics were fairly similar for both males and females. At the same time, almost half of the men were current smokers, whereas among females this percentage was 5 times smaller and was less than 10 %. During the period of observation (1975–2009), 1567 persons died; of these 672 due to cardiovascular diseases.

Table 2.

Bio-demographic characteristics of pooled cohort data for males and females aged 45–64 years at baseline without major ECG abnormalities in Moscow and St. Petersburg, 1975–2001

Characteristic	Males (N = 8377)	Females (N = 2,742)
Mean age (years)	52.0 (4.9)	54.3 (5.7)
Serum total cholesterol   (mean, mmol/l)	5.75 (1.08)	6.01 (1.19)
Systolic blood pressure   (mean, mmHg)	139 (23)	146 (28)
Current smokers (%)	49.4	9.8
CVD deaths (n)	561	91

Standard deviations of mean values are given in brackets. Mean of serum total cholesterol, mean of systolic blood pressure, and percentage of smokers are adjusted using age and education weights for Moscow from the 1989 census

Figure 2 shows the estimated 10-year cumulative CVD mortality risks by age calculated using three different risk scoring equations (original SCORE-High, original SCORE-Low, and SCORE-MoSP) and corresponding observed rates calculated from the original pooled data from the Moscow and St. Petersburg studies. It is immediately obvious that the original SCORE-Low equation leads to substantial underestimation CVD mortality risk for both males and females. It is more important to assess performance of the original SCORE-High equation which is designed for populations with high CVD mortality levels. However, it is evident that using the SCORE-High equation for females leads to notable under-estimation of mortality risk across all age groups. The differences between the observed and estimated CVD mortality ranged from a low of 1.4 times for the oldest age groups to a high of 2.2 and 2.3 times for the youngest age groups (45–49 and 50–54 years). For males, the SCORE-High estimates were also much lower than the observed rates, but the under-estimation was much less pronounced, especially at the oldest ages (Fig. 2).

Fig. 2.

Observed and estimated 10-year cumulative cardiovascular mortality rates by 5-year age groups in Moscow-St. Petersburg (MoSP) adult population

Figure 2 shows that for females, the SCORE-MoSP estimates show a much closer agreement to the observed mortality rates than do those based on the SCORE-High model. The only exception concerns the age group 50–54 years showing mortality rates that were about one-third higher than those in the observed data. For males, the SCORE-MoSP estimates were quite similar to those based on the SCORE-High model. Although the SCORE-MoSP showed slightly closer mortality rates to those observed in the original data, both models tended to underestimate male CVD mortality risks, especially in the youngest age groups (aged 45–54) by about 40–45 %.

The estimated 10-year cumulative cardiovascular mortality rates and CVD risk factor estimates obtained from the pooled MoSP cohorts were used to construct risk prediction charts for the Moscow-St. Petersburg population using the SCORE-MoSP model. These charts were compared to those derived using the SCORE-High equation (Figs. 3 and 4). As for females, Fig. 3 points to a notable underestimation of 10-year mortality risk in the SCORE-High model, especially for older females who smoke having the highest blood pressure and high cholesterol levels. The predicted risk for this group of females was under-estimated by 3–4 % points. The risk based on the SCORE-High also showed lower values of risk for female smokers and non-smokers in the highest blood pressure category.

Fig. 3.

The Moscow-St. Petersburg ten-year risk risk prediction chart for female cardiovascular mortality (SCORE-MoSP and SCORE-High)

Fig. 4.

The Moscow-St. Petersburg 10-year risk prediction chart for male cardiovascular mortality (SCORE-MoSP and SCORE-High)

Figure 4 suggests a slightly better agreement between the two risk prediction charts for males. There are some distinctive features in the pattern of the observed differences between the two prediction charts. First, the SCORE-MoSP chart provides higher predicted risks for both male smokers and non-smokers with a high level of blood pressure, while the SCORE-High overestimates CVD mortality risk for males with normal blood pressure. The underestimation of CVD mortality risk estimated using the SCORE-High algorithm is especially notable at ages 50–59. Second, the SCORE-High model over estimates the relative role of cholesterol for 10-year cumulative cardiovascular mortality among smokers above the age 60 (Fig. 4) because the underestimation of risk is less for males with high level of cholesterol.

Discussion

Data pooled from seven well-known epidemiological cohorts from Moscow and St. Petersburg were used for recalibration of the SCORE method in order to predict future CVD mortality risk in the general populations of these two cities more accurately. The Moscow-St. Petersburg SCORE (SCORE-MoSP) study used the same SCORE algorithm with the exception that baseline survival coefficients were calculated for all cardiovascular system diseases combined while the original SCORE risk is calculated as a sum of risks of coronary and non-coronary cardiovascular diseases. Previous data quality studies of Russia support this decision, suggesting that cardiovascular diseases as a group are classified correctly, whereas there is a notable risk of misclassification of more detailed causes, such as over-diagnosis of coronary heart diseases [24].

The main goal of this study was to provide a recalibrated risk score chart for Russia which maintains the core features of the original SCORE model and accounts for specifics of CVD mortality and its risk factors in Russia. Despite of limitations of the original SCORE model, we replicated the same approach using more comprehensive data for Russia. The only difference between the original SCORE and SCORE-MoSP models concerns the estimation of the baseline survival coefficients for risk factors. For the SCORE-MoSP we estimated such coefficients for all cardiovascular system diseases combined, whereas the original SCORE model requires distinguishing between coronary heart diseases and non-coronary cardiovascular diseases. We found that such modification does not make significant difference and association between risk factors and cardiovascular mortality is similar in both models (Table 3).

Table 3.

Estimated coefficients for selected risk factors based on SCORE-MoSP and original SCORE algorithms

Risk factor	Coefficientsa
	SCORE-MoSPb	SCORE CHD	SCORE non-CHD CVD
Cholesterol   (mmol/L)	0.1653	0.24	0.02
Systolic BP	0.0222	0.018	0.022
Current smoker	0.6958	0.71	0.63

^aCoefficients for original SCORE model (SCORE CHD and SCORE Non-CHD CVD) are taken from [4] and given with the original precision level

^bIncludes both CHD and non-CHD CVD

The study has several limitations. First, the data used are not representative for all of Russia and thus far there are no epidemiologic data which can be seen as representative for this enormously diverse country. Although CVD mortality in Moscow and St. Petersburg is lower than the country’s average, this gap is small compared with the huge difference between Russia and most other developed countries. Second, these data do not include marginal population groups, the institutionalized population, and heavy drinkers, all of which may show peculiar patterns of cardiovascular mortality and its risk factors. Third, the predicted CVD mortality estimates at young ages deviate from the observed data. This can be at least partly related to the lack of statistical power of the data due to too few CVD deaths at these ages and/or unaccounted effects of other unobserved factors such as alcohol. However, even with the aforementioned limitations, these pooled cohort data for the two largest cities are the most reliable data providing comprehensive epidemiological information about CVD risk factors in Russia and have a greater potential to reflect a real situation in Russia than the risk estimates based on data from substantially lower CVD mortality settings.

We evaluated the performance of both the original SCORE and recalibrated SCORE-MoSP equations by comparing estimated CVD mortality rates to those based on the observed data. The study found that the original SCORE-High model tends to substantially under-estimate 10-year cardiovascular mortality risk for females. Using the SCORE-MoSP model we obtained better results which were much closer to the observed rates. As for males, both the original SCORE-High and SCORE-MoSP provided similar estimates which tend to under-estimate CVD mortality risk at younger ages. The aforementioned differences are also reflected in the risk prediction charts. The fact that the original SCORE-High largely underestimates mortality at all ages for women but not for men is mostly related to the underestimation of the baseline mortality risk in women. This implies that the death hazard among females with no risk factors is higher than expected. This is especially important in respect to smoking since non-smokers constitute ninety percent of females and only less than fifty percent among males. The differences between original SCORE and recalibrated SCORE-MoSP models may also be explained by the fact that original SCORE model mostly relied on data from other countries showing lower CVD mortality. The only data for Russia used for this model was the first LRC cohort of males, whereas the corresponding data for Russian females were not available. However, it is not likely that the small Russian LRC cohort may have influenced SCORE estimates in a significant way.

It is also necessary to take into account that changes and fluctuations in CVD mortality in Russia since the beginning of the 1970s were not uniform across different age groups and concerned mostly younger adult ages [27, 28]. Subsequently, we found the fit of the SCORE models was worse for some younger age groups. Despite important changes and variations in CVD mortality since 1970-s (especially among males), associations between risk factors and CVD mortality remained the same even after controlling for period effects (results not shown). Peculiarities of CVD risk factors in Russia including excessive alcohol consumption among adult males and lower smoking rates among females contradicts to the major assumption about the same risk factors for males and females used in the standard SCORE model. This could be another reason of the differences between the observed and predicted estimates.

Thus, the results for males showing underestimation of CVD mortality risk using conventional SCORE-High and recalibrated SCORE-MoSP equations can be explained by specific CVD risk factors in Russia [27–31]. The findings also warn that conventional SCORE-High equations cannot adequately predict the elevated CVD mortality risk in Russia even if national or sub-national mortality data are used. This is a consequence of exclusion from equations of other unaccounted factors which may play important role in the prediction of CVD mortality. One such unobserved factor can be alcohol consumption (not included in the SCORE), which is often considered to be one of the major factors of excess and huge fluctuations in adult mortality in general and very high CVD mortality in Russia [27–29]. Another example of peculiarities of CVD risk factors in Russia concerns specific relationship between cholesterol levels and coronary heart disease risk. The Russian LRC studies found that differently from other developed countries showing increasing CHD risk with increasing levels of cholesterol, this relationship among adult males in Russia was non-linear but rather U or J shape [21]. Finally, there is some evidence that psychosocial factors such as stress, effort-reward imbalance, and a lack of social support, which were unaccounted for in the original and recalibrated SCORE algorithms, may also explain a substantial part of elevated risk of cardiovascular diseases in Russia [30, 31].

Conclusion

Our study supports concerns about universal application of scoring methods, especially for populations with specific and/or changing CVD mortality patterns. Whereas many validation and recalibration studies based on Western countries show over-estimation of CVD mortality risk in the SCORE-High model, our study suggests that in the case of two Russian cities the individual risks would be substantially under-estimated, especially for females. Using non-calibrated scoring models for Russia and other former USSR countries may lead to under-treatment of some patient groups which deserve interventions at early stages of the disease. Although the SCORE-MoSP provide better opportunities for the prediction of cardiovascular mortality risk (especially for females), more complex scoring methods involving a wider range of risk factors are needed.