Abstract
Background
It is recommended in cardiovascular prevention guidelines that treatment should be based on overall cardiovascular risk. The arriba instrument has been widely used for this purpose in Germany. The aim of this study is to validate risk prediction by arriba with the aid of morbidity and mortality data from the population-based Study of Health in Pomerania.
Methods
In a longitudinal analysis, the arriba instrument was used to calculate the 10-year overall cardiovascular risk at baseline for subjects who had not sustained any prior cardiovascular event. Cardiovascular event rates were determined from follow-up data, and discrimination and calibration measures for the risk determination algorithm were calculated.
Results
Data from 1973 subjects (mean age 51 ± 13 years, 48% men) were included in the analysis. After a median follow-up of 10.9 years, cardiovascular events had occurred in 196 subjects, or 9.8%. The ratio of predicted to observed event rate was 0.8 (95% confidence interval: [0.5; 1.1]), 1.3 [1.0; 1.8], and 1.1 [0.8; 1.4] for subjects at low, intermediate, and high cardiovascular risk, respectively. Arriba underestimated cardiovascular event rates in women and overestimated them in persons aged 30–44 and 45–59. The area under curve was 0.84 [95% CI 0.81; 0.86].
Conclusion
The discrimination scores of the arriba instrument resemble those of SCORE-Germany and PROCAM, but a better adjustment to the target population would be desirable. The results support the recommendation of the German Guideline for Cardiovascular Risk Counseling in General Practice for the use of the arriba instrument. An unresolved problem is the failure to consider intervention effects, resulting in an overall mild overestimation of risk.
Determining a person’s cardiovascular risk and counseling them accordingly are crucial components of cardiovascular primary prevention. The guidelines on cardiovascular prevention from the German College of General Practitioners and Family Physicians (DEGAM) as well as the German cardiac Society (DGK) and the European Society of Cardiology (ESC) recommend assessing the overall cardiovascular risk on the basis of multivariate functions (1, 2). In Germany the risk prediction instruments in use are SCORE2 (previously SCORE-Germany, Systematic Coronary Risk Evaluation), PROCAM (Prospective Cardiovascular Münster), and arriba (Aufgabe gemeinsam definieren, Risiko subjektiv, Risiko objektiv, Information über Präventionsmöglichkeiten, Bewertung der Möglichkeiten, Absprache über weiteres Vorgehen [define the task jointly, subjective risk, objective risk, information about prevention options, assessing the options, agreeing on how to proceed]) (3– 13). They predict the 10-year overall risk for a cardiovascular event (myocardial infarction, stroke) in persons who had not experienced a prior cardiovascular event.
To predict the risk, information on individual cardiovascular risk factors is collected and the overall risk is calculated from those (table 1) (3, 14, 15). The SCORE-2-, PROCAM, and arriba instruments predict the 10-year overall risk for fatal and non-fatal cardiovascular events. The arriba instrument was developed from the Framingham risk score (4, 16). It was adapted for European data by adjusting using a calibration factor on the basis of the British Regional Heart Study (17). Additionally the instrument was extended for predicting stroke and for glycemic control on the basis of HbA1c in patients with diabetes (4). The mentioned risk prediction instruments are available free of charge online (table 1). The arriba instrument is often used for the purpose of cardiovascular risk counseling in primary care/general practice because of the option to depict the effects of treatment options.
Table 1. Risk predictors for the risk prediction instruments arriba, PROCAM, and SCORE2.
| arriba*1 | PROCAM*1 | Score2*1 | |
| Risk predictors | |||
| Age | + | + | + |
| Sex | + | + | + |
| Smoking status | + | + | + |
| Positive family history | + | + | − |
| Takes antihypertensive drugs | + | − | − |
| Systolic blood pressure | + | + | + |
| Total cholesterol | + | − | + |
| LDL cholesterol | − | + | (+)2 |
| Triglycerides | − | + | − |
| HDL cholesterol | + | + | + |
| Diabetes | + | + | − |
| HbA1c | + | − | − |
| Age category for use of the prediction instrument (years) | 20–80 | 20–75 | 40–69 (70–89)*3 |
| Area under the curve (AUC) | − | 0.78–0.82 | 0.67–0.81 |
*1 Online version at:
arriba: arriba-hausarzt.de/zugang-arriba/arriba-für-hausärzte
PROCAM: www.assmann-stiftung.de/procam-tests/
Score2: www.escardio.org/Education/Practice-Tools/CVD-prevention-toolbox/SCORE-Risk-Charts
*2 Non-HDL cholesterol
*3 SCORE2-OP for persons aged 70 or older
HbA 1c, glycated hemoglobin ((hemoglobin A1c); HDL, high density lipoprotein; LDL, low density lipoprotein
This study aims to validate the arriba instrument regarding the agreement between predicted and observed cardiovascular 10-year event rate (calibration) and to identify persons with cardiovascular events (discrimination) on the basis of morbidity and mortality data of the population-based Study of Health in Pomerania (SHIP) (18).
Methods
Basis of our data
The analysis is based on data from the population-based cohort study SHIP (source population 214 057 subjects from the north east of Germany in 1996) (18). We used data from the baseline examination (study period 1997–2002, 4308 participants, response rate at baseline 68.8%) and the first and second follow-up examination (SHIP-1: 3300 participants, SHIP-2: 2333 participants, response rate 71%). We used self-reported data on diagnoses of hypertension and diabetes, myocardial infarction, stroke, medication intake, laboratory parameters, results from blood pressure and somatometric measurements, data from the medication review, and data from the mortality follow-up (18).
Inclusion criteria
We included in our analysis all subjects without a prior self-reported cardiovascular event (myocardial infarction, stroke) aged 30–80 years who participated in the baseline examination and both follow-ups, or who died from a cardiovascular event during the study period of 10.9 years to the second follow-up (median follow-up period). We included only subjects with complete data in the calculation of the overall cardiovascular risk with the arriba instrument (figure 1).
Figure 1.
Composition of the study population
Evaluation
Our study was a longitudinal analysis. For all included subjects we used the arriba instrument to calculate cardiovascular 10-year risk based on cardiovascular risk factors (predicted cardiovascular event rate) (for definitions, see the eBox and eTable 1). Non-fatal events were defined on the basis of self-reported myocardial infarction and stroke from the data of both follow-up examinations (response rate of the second follow-up to baseline 54%). Fatal cardiovascular events were defined on the basis of the mortality follow-up if a main cause of death had been coded to the diagnostic groups ICD-GM I10-I15, I20-I25, I26-I28, I30-I52, I60-I69, I70-I79 (etable 2).
eBOX. Definition of cardiovascular events.
-
Non-fatal
-
Self reported myocardial infarction at the time of the first follow up examination, based on the question “In the past 5 years, have you had a myocardial infarction that was diagnosed by a doctor?”
or
-
Self reported myocardial infarction at the time of the second follow up examination, based on the question “Since the latest SHIP examination, since << month/year latest examination >>, a myocardial infarction that was diagnosed by a doctor?”
or
-
Self reported stroke at the time of the first follow up examination, based on the question “In the past 5 years, have you had a stroke that was diagnosed by a doctor?”
or
Self reported stroke at the time of the second follow up examination, based on the question “Since the latest SHIP examination, since << month/year latest examination >>, a stroke that was diagnosed by a doctor?
-
-
Fatal
Death within the median follow-up period from baseline to the second follow-up (corresponding to 3 974 days) with main cause of death coded to the following diagnoses: ICD-GM I10-I15, I20-I25, I26-I28, I30-I52, I60-I69, I70-I79, or R96
eTable 1. Overview of measurements and definitions of laboratory-chemical and clinical variables.
| Lab parameter / risk factor | Time of examination SHIP-0 |
| Total cholesterol serum concentration (mg/dL) | Photometric measurement in non-fasting serum sample by Hitachi 704 |
| HDL cholesterol serum concentration (mg/dL) | Precipitation with phosphotungstic acid/MgCl2 and subsequent cholesterol measurement in serum sample by EPOS 5060 |
| LDL cholesterol serum concentration (mg/dL) | Measurement using the Friedewald formula from values of total serum cholesterol-, HDL-, and triglyceride concentrations |
| Triglyceride serum concentration (mg/dL) | Enzymatic color test in serum sample by Hitachi 717 |
| HbA1c (%) | Photometric measurement in EDTA whole blood sample by Diamat (Biorad) |
| Systolic blood pressure | Mean value of the second and third blood pressure measurement (systolic pressure) (e1) |
| Treatment with antihypertensive drugs | Intake of medication based on the medication review with the AT codes C02A*,C03C*, C03E*, C09BA*,C07A*,C08C*, C09AA*, C09CA* |
| Diabetes mellitus | Self reported diabetes mellitus, in answer to the question: “Are you diabetic?” and “has this been diagnosed or confirmed by a doctor?” (e2) |
| Cigarette smoking | Self reported cigarette smoking, in answer to the question: “Do you currently smoke cigarettes?”; never-smokers and former smokers defined as current non-smokers on the basis of the question: “Have you ever smoked cigarettes?” (e3) |
| Familial predisposition for cardiovascular disorders | For all subjects, the absence of a cardiovascular familial predisposition was defined, as for the data collection, only myocardial infarctions were considered and the age at the time of the event was not recorded. |
HbA1c, hemoglobin A1c; HDL, high density lipoprotein; LDL, low density lipoprotein
eTable 2. Numbers of coded main causes of death in subjects with a fatal cardiovascular event (n = 116).
| ICD10-GM code | Frequency |
| I10 | 2 |
| I10.0 | 1 |
| I11.9 | 1 |
| I20.0 | 1 |
| I21.1 | 1 |
| I21.9 | 25 |
| I24.9 | 1 |
| I25.0 | 1 |
| I25.1 | 1 |
| I25.5 | 1 |
| I25.8 | 2 |
| I25.9 | 15 |
| I26.9 | 4 |
| I35.0 | 3 |
| I42.6 | 1 |
| I48 | 1 |
| I49.9 | 1 |
| I50.1 | 4 |
| I50.9 | 9 |
| I51.9 | 1 |
| I61.5 | 1 |
| I61.9 | 3 |
| I62.0 | 1 |
| I63.2 | 2 |
| I63.8 | 1 |
| I63.9 | 3 |
| I64 | 15 |
| I67.2 | 1 |
| I67.8 | 1 |
| I67.9 | 1 |
| I69.3 | 1 |
| I69.4 | 1 |
| I70.2 | 2 |
| I70.9 | 3 |
| I71.9 | 1 |
| I73.9 | 1 |
| I74.0 | 1 |
| I80.3 | 1 |
Study participants who met the inclusion criteria at the time of the baseline examination formed the baseline cohort. Missing values for the endpoints under consideration were the result of study drop-out or “lost to follow-up” or concurrent events (death from non-cardiovascular causes). Study participants without missing endpoint values formed the complete case cohort. Assuming “missing at random” we used a logistic regression model to model the probability of participation by study participants depending on relevant variables of the baseline examination (sex, age, cigarette smoking, total and HDL cholesterol concentrations, systolic blood pressure, intake of antihypertensive medication, body mass index, presence of diabetes mellitus, number of medications taken in the preceding seven days, number of doctors’ visits within the preceding four weeks). The cases in the complete case cohort were weighted by using inverse probability weights for participation in the follow-up examinations. Applying these weights means that persons with a low probability of participating are given a greater weight so as to reduce the influence of bias as a result of selective participation.
Calibration (agreement between predicted and observed events) and discrimination (differentiating between persons who developed a cardiovascular event and those who did not) were assessed as criteria for the validity of the arriba instrument. We reported the ratio between predicted and observed cardiovascular events (predicted over observed ratio, P/O ratio) as the parameter for calibration for the total cohort and defined subgroups by age, sex, and predicted risk (low [<10%], moderate [10–20%], and high [≥20%]. A P/O ratio >1 indicates an overestimate and a P/O ratio <1 an underestimate of the cardiovascular risk. For the central analysis we calculated event rates and P/O ratios on the basis of the weighted cases of the complete case cohort. Additionally we report unweighted event rates for the baseline and complete case cohorts. We used receiver operating characteristic (ROC) curves and AUC (area under the curve) values to determine discrimination. No established methods to consider inverse probability weights are known for this analysis, therefore, unweighted data from the complete case cohort was used. We used SAS 9.3 (SAS Institute Inc., Cary, NC, USA) to analyze our data.
Research ethics and data protection
SHIP received ethics approval and consent from the participants, as well as a framework concept for data protection and storage that was agreed with the state data protection commissioner (18).
Results
At baseline, n=3446 subjects met the inclusion criteria (baseline cohort). The complete case cohort consisted of the n=1973 subjects who had participated in both follow-up examinations and those who had died from a cardiovascular event by the time of the second follow-up (figure 1). Table 2 shows the characteristics of both cohorts. The mean age in the complete case cohort was 51 (standard deviation 13) years; 48% were men. Of the subjects, 69% (1356/1973) had a low cardiovascular risk, 30% (388/1973) a moderate risk, and 12% (229/1973) a high risk (table 3). By the second follow-up, 80 subjects (41.%, 80/1973) in the complete case cohort had experienced a non-fatal cardiovascular event and 116 subjects (5.9%, 119/1973) a fatal events.
Table 2. Characteristics of the study population (data from baseline examination).
| Baseline cohort (n = 3446) | Complete case cohort (n = 1973) | |||||
| Number | Mean (SD)/proportion % | [95% CI] | Number | Mean (SD)/proportion % | [95% CI] | |
| Age (years) | 3446 | 52.9 (13.8) | [52.5; 53.4] | 1973 | 50.6 (12.5) | [50.1; 51.2] |
| Male sex (%) | 1656 | 48.1 (1 656/3 446) | [46.4; 49.7] | 945 | 47.9 (945/1 973) | [45.7; 50.1] |
| Arterial hypertension (%) *1 | 2259 | 65.5 (2 259/3 446) | [63.9; 67.1] | 1206 | 61.1 (1 206/1 973) | [58.9; 63.3] |
| Normotension | 515 | 22.8 (515/2 259) | [21.1; 24.6] | 292 | 24.2 (292/1 206) | [21.8; 26.7] |
| Grade 1 | 110 | 49.1 (1 110/2 259) | [47.1; 51.2] | 580 | 48.1 (580/1 206) | [45.2; 50.9] |
| Grade 2 | 448 | 19.8 (448/2 259) | [18.2; 21.5] | 243 | 20.2 (243/1 206) | [17.9; 22.5] |
| Grade 3 | 186 | 8.2 (186/2 259) | [7.1; 9.4] | 91 | 7.6 (91/1 206) | [6.1; 9.2] |
| Antihypertensive drug according to medication review *2 | 989 | 43.8 (989/2 259) | [41.7; 45.9] | 475 | 39.4 (475/1 206) | [36.6; 42.2] |
| Diabetes mellitus known from ‧patient history(%) | 284 | 8.2 (284/3 446) | [7.3; 9.2] | 123 | 6.2 (123/1 973) | [5.2; 77.4] |
| HbA1c target range ≤ 7.5 % | 185 | 65.1 (185/284) | [59.3; 70.7] | 78 | 63.4 (78/123) | [54.2; 771.9] |
| Drug treatment | 221 | 77.8 (221/284) | [72.5; 82.5] | 98 | 79.7 (98/123) | [71.5; 886.4] |
| Cigarette smoking *3 | 982 | 28.5 (982/3 446) | [27.0; 30.0] | 490 | 24.8 (490/1 973) | [22.9; 226.8] |
| Total cholesterol (mmol/L) | 3446 | 5.9 (1.2) | [5.9; 6.0] | 1973 | 5.19 (1.2) | [5.8; 5.9] |
| Total cholesterol (mg/dL) | 3446 | 228.1 (44.9) | [226.6; 229.6] | 1973 | 226.7 (45.1) | [224.7; 228.6] |
| HDL (mmol/L) | 3446 | 1.5 (0.5) | [1.4; 1.5] | 1973 | 1.5 (0.4) | [1.4; 1.5] |
| HDL (mg/dL) | 3446 | 56.1 (16.1) | [55.5; 56.6] | 1973 | 56.4 (15.9) | [55.7; 57.1] |
| Body mass index (kg/m2) *3 | 3440 | 27.7 (4.7) | [27.5; 27.8] | 1971 | 27.3 (4.4) | [27.1; 27.5] |
*1 Baseline cohort: n = 12 subjects missing (12/3 446, 0.4 %); complete case cohort: n = 6 subjects missing (6/1 973, 0.3 %);
*2 Baseline cohort: n = 8 subjects missing (8/2 258, 0.3 %); complete case cohort: n = 4 missing (4/1 206);
*3 Baseline cohort: n = 6 subjects missing (6/3 446, 0.2 %); complete case cohort: n = 2 subjects missing (2/1 973, 0.1 %)
95% CI, 95% confidence interval; HbA 1c., hemoglobin A 1c; HDL, high density lipoprotein; SD, standard deviation
Table 3. Predicted and observed 10 year event rates for the arriba instrument.
| n | Predicted event rate *1 (weighted) (%) [95% CI] | Observed event rate *1 (weighted) (%) [95% CI] | P/O ratio [95% CI *2] | |
| Total cohort | 1 973 | 9.79 [9.33; 10.25] | 9.30 [8.02; 10.59] | 1.05 [0.88; 1.25] |
| Subgroups by risk | ||||
| arriba risk < 10 % | 1 356 | 3.21 [3.06; 3.37] | 4.16 [3.10; 5.23] | 0.77 [0.54; 1.08] |
| arriba risk 10 % bis < 20 % | 388 | 14.55 [14.28; 14.83] | 10.87 [7.76; 13.99] | 1.34 [1.02; 1.77] |
| arriba risk ≥ 20 % | 229 | 29.22 [28.15; 30.30] | 27.33 [21.51; 33.14] | 1.07 [0.82; 1.44] |
| Subgroups by sex | ||||
| Men | 945 | 14.23 [13.54; 14.93] | 11.18 [9.17; 13.20] | 1.27 [1.03; 1.58] |
| Women | 1 028 | 5.64 [5.13; 6.1] | 7.55 [5.93; 9.17] | 0.75 [0.55; 1.03] |
| Subgroups by age | ||||
| 30–44 years | 690 | 3.56 [3.12; 4.0] | 2.50 [1.33; 3.67] | 1.42 [0.93; 2.32] |
| 45–59 years | 775 | 7.97 [7.41; 8.54] | 5.62 [4.00; 7.25] | 1.42 [1.08; 1.89] |
| 60–80 years | 508 | 17.61 [16.62; 18.60] | 19.56 [16.10; 23.02] | 0.90 [0.72; 1.14] |
*1 Predicted and observed event rates were calculated respectively on the basis of weighted cases of the complete case cohort.
*2 Confidence intervals for P/O ratios were calculated by bootstrapping (n = 10 000 bootstrap samples. Intervals were calculated by the bias-corrected and accelerated method)
95% CI, 95% confidence interval; P/O ratio, predicted-over-observed ratio: ratio of predicted events/observed events
The unweighted predicted event rate in the baseline cohort was 9.80% and in the complete case cohort, 8.27% (etable 3). Differences were also seen for the subgroups; these are indications that weighting for missing endpoints was important. The weighted predicted event rate in the complete case cohort was near identical to that in the baseline cohort (9.79 versus 9.80). The results were also similar for the individual subgroups, which can be seen as an indication that weighting adjusts for missing values.
eTable 3. Predicted and observed 10-year event rates for the arriba instrument.
|
Predicted event rate
baseline cohort n = 3446 % [95% CI] |
Predicted event rate
complete case cohort n = 1973 % [95% CI] (unweighted) *1 |
Predicted event rate
complete case cohort n = 1973 % [95% CI] (weighted) *2 |
Observed event rate
complete case cohort n = 1973 %[95% CI] (unweighted) *1 |
Observed event rate
complete case cohort n = 1973 % [95% CI] (weighted) *2 |
||||||
| Total | 9.80 | [9.46; 10.14] | 8.27 | [7.85; 8.70] | 9.79 | [9.33; 10.25] | 9.93 | [8.61; 11.26] | 9.30 | [8.02; 10.59] |
| Subgroups by risk | ||||||||||
| arriba risk < 10 % | 3.25 | [3.13; 3.38] | 3.02 | [2.87; 3.17] | 3.21 | [3.06; 3.37] | 3.61 | [2.62; 4.61] | 4.16 | [3.10; 5.23] |
| arriba risk 10 % bis < 20 % | 14.60 | [14.39; 14.80] | 14.45 | [14.17; 14.72] | 14.55 | [14.28; 14.83] | 14.69 | [11.15; 18.23] | 10.87 | [7.76; 13.99] |
| arriba risk ≥ 20 % | 28.48 | [27.83; 29.13] | 28.92 | [27.87; 29.98] | 29.22 | [28.15; 30.30] | 39.30 | [32.93; 45.67] | 27.33 | [21.51; 33.14] |
| Subgroups by sex | ||||||||||
| Men | 14.28 | [13.77; 14.79] | 12.67 | [12.01; 13.33] | 14.23 | [13.54; 14.93] | 12.59 | [10.47; 14.71] | 11.18 | [9.17; 13.20] |
| Women | 5.66 | [5.30; 6.01] | 4.24 | [3.84; 4.63] | 5.64 | [5.13; 6.15] | 7.49 | [5.88; 9.10] | 7.55 | [5.93; 9.17] |
| Subgroups by age | ||||||||||
| 30–44 years | 3.64 | [3.29; 4.00] | 3.23 | [2.80; 3.65] | 3.56 | [3.12; 4.0] | 2.60 | [1.42; 3.80] | 2.50 | [1.33; 3.67] |
| 45–59 years | 8.27 | [7.77; 8.77] | 7.51 | [6.97; 8.05] | 7.97 | [7.41; 8.54] | 5.94 | [4.27; 7.60] | 5.62 | [4.00; 7.25] |
| 60–80 years | 16.91 | [16.33; 17.49] | 16.29 | [15.34; 17.24] | 17.61 | [16.62; 18.60] | 25.98 | [22.16; 29.81] | 19.56 | [16.10; 23.02] |
*1 Cases were not weighted. No adjustment took place.
*2 Cases were weighted by inverse probability weights.
95% CI. 95% confidence interval
The P/O ratio based on the weighted cases of the complete case cohort for the total cohort was 1.05 (table 3). For subjects with a low, moderate, and high cardiovascular risk, the P/O ratio was 0.8% each (95% confidence interval [0.5; 1.1]), 1.3 [1.0; 1.8], and 1.1 [0/8; 1.4]. The P/O ratio in women and in the group aged 60–80 years was below 1, and in men and in the age groups 30–44 and 45–59 years in some cases notably above 1 (table 3). The AUC for discrimination of persons with a cardiovascular event was 0.84 [0.81; 0.86] (figure 2).
Figure 2.
ROC (receiver operating characteristic) curve with AUC (area under curve) value; an AUC value near 1 corresponds to a good discrimination between persons with/without a cardiovascular event on the basis of the arriba instrument; a flat AUC curve (low AUC value) close to the diagonal would correspond to a random discrimination between persons with/without a cardiovascular event in the time period of 10 years on the basis of the risk prediction of the arriba instrument.
Youden Index maximum 0.51 for arriba threshold value ≥ 0.093 corresponding to a sensitivity of 0.8 and a specificity of 0.72 (Youden Index = sensitivity + specificity – 1.
The calculation was done on the basis of the unweighted cases of the complete case analysis (n = 1 973).
Discussion
Summary of the main findings
Overall, the arriba instrument shows good discrimination for persons with a cardiovascular event on the basis of the AUC value. The event risk in persons with a low cardiovascular risk is slightly underestimated by the arriba instrument and the event rate especially in persons with a moderate risk is slightly overestimated.
Prognostic quality
Analyses with a high number of observations and a sufficiently high number of events are the gold standard for validating risk prediction instruments (19, 20). For the primary care setting in Germany, no comprehensive data exists, hence population-based cohort studies provide an alternative option for external validation.
As far as we are aware, thus far only the DETECT study has reported findings for the calibration or discrimination of the arriba instrument (21). In contrast to our analysis, this study reported a risk overestimate even in the risk range <10% and a notably higher overestimate in the risk range >20%. The study had a median follow-up period of only four years, and an older paper version (point score) of arriba was used to calculate the risk. The validity of the paper version is considerably limited, since included predictors are categorized. Patients with diabetes were excluded, whereas the arriba instrument explicitly allows for a risk calculation for diabetes patients. Because of the methodological limitations of the study, its importance regarding the validity of arriba is limited. The same authors reported indications of a risk overestimate for persons with a moderate or high risk for two investigated versions of PROCAM (21). The limitations mentioned above partly apply here too.
Data on the external validation of SCORE-2 for Germany on the basis of different cohorts have been published recently and show an AUC between 0.68 and 0.78 (5). German data on calibration have been published for the SCORE instrument, but not for SCORE2 (data from SHIP Study, study period 1997–2001 and KORA Study ([KORA, Kooperative Gesundheitsforschung in der Region Augsburg—cooperative health services research in the Augsburg region]; study period 1999–2001, age standardized state-related mortality data) [9–11]. Depending on age group, the calculated P/O ratios for SHIP were 10.5–9.28 for men and 0.26–3.15 for women. The results show an underestimate for women aged 40–54 years, and an overestimate for women aged 55–64 years as well as for men aged 45–65. The overestimate was greater the older the age group and was greater for the SHIP than for the KORA study populations. Because of divergent categories and methods, these data are not directly comparable with ours. It is not known whether SCORE2 can be assumed to have similar characteristics.
Three review articles compared a selection of cardiovascular risk prediction instruments in the time period to 2016 regarding measures of discrimination (14, 15, 22). For SCORE, AUC values (externally validated) of 0.65–0.87 and P/O ratios of 0.28–1.50 were reported in different international populations; for the PROCAM instrument the AUC values were 0.61 and 0.64. The AUC value for the arriba instrument of 0.84 [0.81; 0.86) in our study is comparable with the values published for SCORE and SCORE2 and higher than that for the PROCAM instrument.
A limitation regarding the quality of the prediction for risk prediction instruments is the need for continuous recalibration because of the decrease in cardiovascular events over time and because of the changes in medication treatments and their effect on cardiovascular risk factors and events (14, 23, 24). These challenges have so far not been resolved for any of the published instruments (25).
Suitability for clinical use
In addition to validity, the selection of appropriate endpoints and risk factors, the practical applicability and the evidence of patient-relevant effects are criteria for the good clinical applicability of a risk prediction instrument (14).
Of the risk prediction instruments in use in Germany, the endpoints of the arriba, PROCAM, and SCORE2 instruments (cardiovascular morbidity or mortality) enable the implementation of the regulations of the Federal Joint Committee regarding the prescription of lipid lowering drugs from an overall cardiovascular 10-year risk of ≥20 % (1, 5, 12, 26). The arriba instrument considers—in contrast to SCORE (SCORE2) diabetes and HbA1c as risk predictors; the differentiation of the cardiovascular risk in persons with type 2 diabetes was, however, improved in SCORE2 (1, 2, 27).
With regard to applicability in routine clinical practice, an intervention study showed that arriba Herz risk counseling can be integrated in the Disease Management Program for coronary heart disease, and an observational study showed that less than 10% of doctors applying the model reported an unacceptably longer consultation time if they administered the computer based arriba instrument (28, 29).
An effect on endpoints such as overall mortality or cardiovascular mortality has thus far not been confirmed for any of the cardiovascular risk prediction instruments (30). Collins et al showed in a meta-analysis a slight reduction of systolic blood pressure, total and LDL cholesterol, and a slight increase in abstinence from smoking. All effects were categorized as uncertain because of unsatisfactory study quality and short follow-up periods (3). For the arriba instrument, an intervention study showed an effect of physician counseling on the prescription rate of antiplatelet drugs in the follow-up consultations, but not on prescription rates of statins or antihypertensive medications (31). A single consultation with the arriba instrument did not lead to change in smoking behavior in another intervention study (28).
The principle of joint decision making of doctors and patients, however, is in contrast to the expectation that using risk prediction instruments results in better risk constellations and a corresponding reduction in disease events. Those decisions of patients that do not result in the optimization of cardiovascular risk factors also have to be accepted.
Regarding patient relevant effects, a randomized controlled trial for the paper-based version of the arriba instrument showed greater satisfaction for patients in the intervention group as well as a subjectively experiences better participation in decision making during the consultation (32). An observational study also found high satisfaction among patients for the consultation using the electronic version; 81% of patients reported that they had implemented the decision made as a result of the consultation (33).
Study results have shown that 15–20% of patients overestimate their own cardiovascular risk and 13–40% underestimate it (34). In contrast to the SCORE and PROCAM instruments, the absolute cardiovascular risk can be depicted by age, so that individual categorization is possible.
For the SCORE and PROCAM instruments, we are not aware of any studies investigating their applicability in clinical practice using German data.
Strengths and limitations
To our knowledge, our analysis was the first to study the predictive validity of the arriba instrument over an observation period of 10 years. We used data from a population-based cohort study with a quality assured comprehensive data collection of cardiovascular risk factors. We used inverse probability weights to reduce the effect of bias as a result of selective participation. This was feasible for the parameter calibration but not for discrimination. Confounding as a result of selective participation is possible. Overall, the validity of the results is limited because of the limited number of available observations and cardiovascular events (19). Difference between the time period of the risk prediction (=10 years) and individual inconsistent follow-up periods, as well as a missing date for the timing of the cardiovascular event can lead to overestimates or underestimates. Lacking quality in documenting fatal cardiovascular events from the coded main cause of death are known from other studies (false positive results documented in up to 30% of coded cases) (35, 36). In our study, no precise main cause of death was given for 25 persons (4%, 25/567) of all those deceased in the observation period, with the result that assuming a missing cardiovascular event in these persons may have prompted an underestimate in the event rate. Family history as a cardiovascular risk factor was defined as negative for all included persons since the age of the relatives at onset of the cardiovascular disorder had not been recorded. In this context, the overall cardiovascular risk may be subject to an overestimate or underestimate. No untreated cohort was analyzed in our study. If in patient groups at higher risk the observed risk is lower than the predicted risk, this may be the result of an overestimate by the score or the result of effective prevention (37).
Conclusions
Guidelines for cardiovascular prevention consistently recommend predicting the overall cardiovascular risk. Our analyses show that the validity of the arriba instrument is comparable to other risk prediction instruments used in Germany. The analyses support the recommendation of the German guideline for risk counseling for cardiovascular prevention in primary care in favor of the arriba instrument. Guideline recommendations for cardiovascular risk prediction instruments should—in addition to risk prediction—also consider aspects of clinical applicability.
Acknowledgments
Translated from the original German by Birte Twisselmann, PhD.
Footnotes
Conflict of interest statement Professor Chenot received financial funding from the Zentralinstitut für die kassenärztliche Versorgung [the central institute for healthcare delivered by statutory health insurance physicians] for a research project on the subject of this article, which he initiated.
Dr Donner-Bantzhoff holds share in Novartis and Alcon. He is involved in developing arriba, joint managing director or the charitable Society for patient Centered Communication, and chairman of the arriba cooperative.
Dr Haasenritter is a member of the guideline committee of DEGAM, joint author and coordinator of the DEGAM guideline on “general practitioner advice on cardiovascular risk prevention.”
Dr Angelow, Prof Donner-Banzhoff, and Prof Chenot were coauthors on the DEGAM guideline.
The remaining authors declare that no conflict of interest exists.
References
- 1.Deutsche Gesellschaft für Allgemeinmedizin und Familienmedizin. www.degam.de/degam-leitlinien-379.html (last accessed on 8 December 2021) [Google Scholar]
- 2.Deutsche Gesellschaft für Kardiologie. leitlinien.dgk.org/2022/pocket-leitlinie-praevention-von-herz-kreislauf-erkrankungen-version-2021/; leitlinien.dgk.org/2021/2021-esc-guidelines-on-cardiovascular-disease-prevention-in-clinical-practice/) (last accessed on 13 April 2022) [Google Scholar]
- 3.arriba. Kardiovaskuläre Prävention. arriba-hausarzt.de/module/kardiovaskuläre-prävention (last accessed on 8 December 2021) [Google Scholar]
- 4.Donner-Banzhoff N, Keller H, Sadowski E-M, et al. Verlag Hans Huber. Bern: 2007. Individualisierte Risikoprognose für die Herz-Kreislauf-Prävention: ein Werkstattbericht In: Kunz R, Ollenschläger G, Raspe H, Jonitz G (eds.): Lehrbuch Evidenzbasierte Medizin in Klinik und Praxis; pp. 295–304. [Google Scholar]
- 5.SCORE2 working group and ESC cardiovascular risk collaboration, SCORE2 risk prediction algorithms. new models to estimate 10-year risk of cardiovascular disease in Europe. Eur Heart J. 2021;42:2439–2454. doi: 10.1093/eurheartj/ehab309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.SCORE2-OP working group and ESC cardiovascular risk collaboration, SCORE2-OP risk prediction algorithms. estimating incident cardiovascular event risk in older persons in four geographical risk regions. Eur Heart J. 2021;42:2455–2467. doi: 10.1093/eurheartj/ehab312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.European Society of Cardiology ESC) Heart Score. www.heartscore.org/en_GB/ (last accessed on 2 June 2022) [Google Scholar]
- 8.Keil U, Fitzgerald AP, Gohlke H, Wellmann J, Hense HW. Risikoabschätzung tödlicher Herz-Kreislauf-Erkrankungen: die neuen SCORE-Deutschland-Tabellen für die Primärprävention. Dtsch Arztebl. 2005;102 A:1808–1812. [Google Scholar]
- 9.Hense HW, Koesters E, Wellmann J, Meisinger C, Völzke H, Keil U. Evaluation of a recalibrated systematic coronary risk evaluation cardiovascular risk chart: results from systematic coronary risk evaluation Germany. Eur J Cardiovasc Prev Rehabil. 2008;15:409–415. doi: 10.1097/HJR.0b013e3282feec66. [DOI] [PubMed] [Google Scholar]
- 10.Rücker V, Keil U, Fitzgerald AP, et al. Predicting 10-year risk of fatal cardiovascular disease in Germany: an update based on the SCORE-Deutschland risk charts. PLoS One. 2016;11 doi: 10.1371/journal.pone.0162188. e0162188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Holle R, Happich M, Löwel H, Wichmann HE. MONICA/KORA study group KORA—a research platform for population based health research. Gesundheitswesen. 2005;67(Suppl 1):S19–S25. doi: 10.1055/s-2005-858235. [DOI] [PubMed] [Google Scholar]
- 12.Assmann-Stiftung für Prävention 2018. Procam Gesundheitstests. https://www.assmann-stiftung.de/procam-tests/ (last accessed on 8 December 2021) [Google Scholar]
- 13.Assmann G, Schulte H, Cullen P, Seedorf U. Assessing risk of myocardial infarction and stroke: new data from the Prospective Cardiovascular Münster (PROCAM) study. Eur J Clin Invest. 2007;37:925–932. doi: 10.1111/j.1365-2362.2007.01888.x. [DOI] [PubMed] [Google Scholar]
- 14.Gorenoi V, Hagen A. Overview of risk-estimation tools for primary prevention of cardiovascular diseases in European populations. Cent Eur J Public Health. 2015;23:91–99. doi: 10.21101/cejph.a4004. [DOI] [PubMed] [Google Scholar]
- 15.Cooney MT, Dudina A, D’Agostino R, Graham IM. Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation. 2010;122:300–310. doi: 10.1161/CIRCULATIONAHA.109.852756. [DOI] [PubMed] [Google Scholar]
- 16.Sullivan LM, Massaro JM, D’Agostino RB Sr. Presentation of multivariate data for clinical use: the framingham study risk score functions. Stat Med. 2004;23:1631–1660. doi: 10.1002/sim.1742. [DOI] [PubMed] [Google Scholar]
- 17.Brindle P, Emberson J, Lampe F, et al. Predictive accuracy of the framingham coronary risk score in British men: prospective cohort study. BMJ. 2003;327 doi: 10.1136/bmj.327.7426.1267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Völzke H, Alte D, Schmidt CO, et al. Cohort profile: the study of health in Pomerania. Int J Epidemiol. 2011;40:294–307. doi: 10.1093/ije/dyp394. [DOI] [PubMed] [Google Scholar]
- 19.Van Calster B, Nieboer D, Vergouwe Y, De Cock B, Pencina MJ, Steyerberg EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167–176. doi: 10.1016/j.jclinepi.2015.12.005. [DOI] [PubMed] [Google Scholar]
- 20.Collins GS, Altman DG. An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ. 2010;340 doi: 10.1136/bmj.c2442. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Grammer TB, Dressel A, Gergei I, et al. Cardiovascular risk algorithms in primary care: results from the DETECT study. Sci Rep. 2019;9 doi: 10.1038/s41598-018-37092-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Damen JAAG, Hooft L, Schuit E, et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016;353 doi: 10.1136/bmj.i2416. i2416. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Liew SM, Doust J, Glasziou P. Cardiovascular risk scores do not account for the effect of treatment: a review. Heart. 2011;97:689–697. doi: 10.1136/hrt.2010.220442. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Mehta S, Jackson R, Wells S, Harrison J, Exeter DJ, Kerr AJ. Cardiovascular medication changes over 5 years in a national data linkage study: implications for risk prediction models. Clin Epidemiol. 2018;10:133–141. doi: 10.2147/CLEP.S138100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Rospleszcz S, Thorand B, de Las Heras Gala T, et al. Temporal trends in cardiovascular risk factors and performance of the framingham risk score and the pooled cohort equations. J Epidemiol Community Health. 2019;73:19–25. doi: 10.1136/jech-2018-211102. [DOI] [PubMed] [Google Scholar]
- 26.Gemeinsamer Bundesausschuss (GBA) Anlage III - Übersicht über Verordnungseinschränkungen und -ausschlüsse in der Arzneimittelversorgung durch die Arzneimittel-Richtlinie und aufgrund anderer Vorschriften (§ 34 Absatz 1 Satz 6 und Absatz 3 SGB V) www.g-ba.de/downloads/83-691-714/AM-RL-III-Verordnungseinschraenkung_2022-01-18.pdf (last accessed on 2 June 2022) 2016 [Google Scholar]
- 27.Bulugahapitiya U, Siyambalapitiya S, Sithole J, et al. Is diabetes a coronary risk equivalent? Systematic review and meta-analysis. Diabet med. 2009;26:142–148. doi: 10.1111/j.1464-5491.2008.02640.x. [DOI] [PubMed] [Google Scholar]
- 28.Bleckwenn M, Münster S, Weckbecker K. Motivation of high-risk patients to stop smoking in the DMPM. Gesundheitswesen. 2015;77:885–887. doi: 10.1055/s-0034-1387778. [DOI] [PubMed] [Google Scholar]
- 29.Hirsch O, Keller H, Krones T, Donner-Banzhoff N. Arriba-lib: evaluation of an electronic library of decision aids in primary care physicians. BMC Med Inform Decis Mak. 2012;12 doi: 10.1186/1472-6947-12-48. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Collins DR, Tompson AC, Onakpoya IJ, Roberts N, Ward AM, Heneghan CJ. Global cardiovascular risk assessment in the primary prevention of cardiovascular disease in adults: systematic review of systematic reviews. BMJ Open. 2017;7 doi: 10.1136/bmjopen-2016-013650. e013650. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Keller H, Krones T, Becker A, et al. Arriba: effects of an educational intervention on prescribing behaviour in prevention of CVD in general practice. Eur J Prev Cardiol. 2012;19:322–329. doi: 10.1177/1741826711404502. [DOI] [PubMed] [Google Scholar]
- 32.Krones T, Keller H, Sönnichsen A, et al. Absolute cardiovascular disease risk and shared decision making in primary care: a randomized controlled trial. Ann Fam Med. 2008;6:218–227. doi: 10.1370/afm.854. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Hirsch O, Keller H, Krones T, Donner-Banzhoff N. Acceptance of shared decision making with reference to an electronic library of decision aids (arriba-lib) and its association to decision making in patients: an evaluation study. Implement Sci. 2011;6 doi: 10.1186/1748-5908-6-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.van der Weijden T, Bos LB, Koelewijn-van Loon MS. Primary care patients’ recognition of their own risk for cardiovascular disease: implications for risk communication in practice. Curr Opin Cardiol. 2008;23:471–476. doi: 10.1097/HCO.0b013e32830b35f6. [DOI] [PubMed] [Google Scholar]
- 35.Modelmog D, Rahlenbeck S, Trichopoulos D. Accuracy of death certificates: a population-based, complete-coverage, one-year autopsy study in East Germany. Cancer Causes Control. 1992;3:541–546. doi: 10.1007/BF00052751. [DOI] [PubMed] [Google Scholar]
- 36.Pagidipati NJ, Gaziano TA. Estimating deaths from cardiovascular disease: a review of global methodologies of mortality measurement. Circulation. 2013;127:749–756. doi: 10.1161/CIRCULATIONAHA.112.128413. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Pajouheshnia R, Peelen LM, Moons KGM, Reitsma JB, Groenwold RHH. Accounting for treatment use when validating a prognostic model: a simulation study. BMC Med Res Methodol. 2017;17 doi: 10.1186/s12874-017-0375-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- E1.European Guidelines for Cardiovascular Disease Prevention. Eur Heart J. 2007;28:2375–2414. doi: 10.1093/eurheartj/ehm316. [DOI] [PubMed] [Google Scholar]
- E2.The Expert Committee on the diagnosis and classification of diabetes mellitus Follow-up report on the diagnosis of diabetes mellitus. Diabetes Care. 2003;26:3160–3167. doi: 10.2337/diacare.26.11.3160. [DOI] [PubMed] [Google Scholar]
- E3.Latza U, Hoffmann W, Terschüren C, et al. Erhebung, Quantifizierung und Analyse der Rauchexposition in epidemiologischen Studien. Berlin: Robert Koch-Institut; 2007 [Google Scholar]