Accuracy and impact of risk assessment in the primary prevention of cardiovascular disease: a systematic review

Abstract

Objective

To determine the accuracy of assessing cardiovascular disease (CVD) risk in the primary prevention of CVD and its impact on clinical outcomes.

Design

Systematic review.

Data sources

Published studies retrieved from Medline and other databases. Reference lists of identified articles were inspected for further relevant articles.

Selection of studies

Any study that compared the predicted risk of coronary heart disease (CHD) or CVD, with observed 10‐year risk based on the widely recommended Framingham methods (review A). Randomised controlled trials examining the effect on clinical outcomes of a healthcare professional assigning a cardiovascular risk score to people predominantly without CVD (review B).

Review methods

Data were extracted on the ratio of the predicted to the observed 10‐year risk of CVD and CHD (review A), and on cardiovascular or coronary fatal or non‐fatal events, risk factor levels, absolute cardiovascular or coronary risk, prescription of risk‐reducing drugs and changes in health‐related behaviour (review B).

Results

27 studies with data from 71 727 participants on predicted and observed risk for either CHD or CVD were identified. For CHD, the predicted to observed ratios ranged from an underprediction of 0.43 (95% CI 0.27 to 0.67) in a high‐risk population to an overprediction of 2.87 (95% CI 1.91 to 4.31) in a lower‐risk population. In review B, four randomised controlled trials confined to people with hypertension or diabetes found no strong evidence that a cardiovascular risk assessment performed by a clinician improves health outcomes.

Conclusion

The performance of the Framingham risk scores varies considerably between populations and evidence supporting the use of cardiovascular risk scores for primary prevention is scarce.

Assessing a person's cardiovascular risk has become the accepted way of targeting preventive treatment at patients who are asymptomatic but at high risk of cardiovascular disease (CVD). Multivariate risk functions derived in several cohort studies and randomised trials form the basis of predictive functions and risk scores.¹,²,³,⁴ Many, especially those derived from the Framingham Heart Study, have been adapted for use in primary care as simplified charts, tables, computer programs and web‐based tools, and are routinely recommended in policy documents and guidelines.⁵,⁶,⁷,⁸ Depending on their absolute risk, asymptomatic people may be offered blood pressure and cholesterol‐lowering treatment and aspirin, in addition to advice about relevant health behaviours. Such interventions may be life long and are associated with risks as well as benefits.

Cardiovascular risk scores, like clinical prediction rules, help clinicians prioritise treatment and should be subject to evaluation before implementation. The predictive performance of the risk score needs to be examined in different populations, and then its clinical impact must be assessed by means of a randomised controlled trial.⁹ For the risk‐scoring approach to be a viable strategy for primary prevention, it should favourably influence people's risk of disease or risk factors or, in the absence of such information, increase prescription of effective preventive treatments to appropriate patients.

The objectives of this study were to systematically review: (1) the external validity—that is, the extent to which predicted risk assessments accurately reflected observed risk—of widely recommended Framingham risk scores in different populations; and (2) the randomised controlled trials that have evaluated the effectiveness of risk‐scoring methods for improving CVD‐related outcomes.

METHODS

For the two systematic reviews, a common literature base was identified and a search strategy was designed to find all studies of the external validity and clinical impact of cardiovascular risk scores. The scope of the review was to determine how well any of the relevant prediction models or scores perform in terms of observed event rates compared with predicted event rates in different settings and populations. For evaluations, only randomised trials were considered sufficiently robust to determine the unbiased and unconfounded effects of risk factor scoring on clinical outcomes.

Search strategy

Table 1 details the terms used to search Medline. Appropriate adaptations of search syntax were made when searching other databases. The Cochrane controlled trials register (CENTRAL), Medline, Embase, CINAHL, PsycINFO, ISI Proceedings and ZETOC were searched. Searches covered from database inception to September 2004. Reference lists of articles were searched to identify additional relevant reports and key journals were hand searched. No language restrictions were applied and articles were translated when necessary. No restrictions were applied to the years of publication. Articles were incorporated into a Reference Manager database (Thomson ResearchSoft, Carlsbad, California, USA).

Table 1 Medline search terms and strategy.

#1 chd risk assessment$	#37 risk calculation$	#73 erica risk score$
#2 cvd risk assessment$	#38 risk calculator$	#74 framingham scor$
#3 heart disease risk assessment$	#39 risk factor$ calculator$	#75 dundee scor$
#4 coronary disease risk assessment$	#40 risk factor$ calculation$	#76 brhs scor$
#5 cardiovascular disease risk assessment$	#41 risk engine$	#77 British Regional Heart study risk scor$
#6 cardiovascular risk assessment$	#42 risk equation$	#78 brhs risk scor$
#7 cv risk assessment$	#43 risk table$	#79 dundee risk scor$
#8 cardiovascular disease$ risk assessment$	#44 risk threshold$	#80 framingham guideline$
#9 coronary risk assessment$	#45 risk disc?	#81 framingham risk?
#10 coronary risk scor$	#46 risk disk?	#82 new zealand table$
#11 heart disease risk scor$	#47 risk scoring method?	#83 ncep guideline?
#12 chd risk scor$	#48 scoring scheme?	#84 smac guideline?
#13 cardiovascular risk scor$	#49 risk scoring system?	#85 copenhagen risk?
#14 cardiovascular disease$ risk scor$	#50 risk prediction?	#86 or/57–85
#15 cvd risk scor$	#51 predictive instrument?	#87 56 or 86
#16 cv risk scor$	#52 project$ risk?	#88 exp decision support techniques/
#17 or/1–16	#53 cdss	#89 Diagnosis, Computer‐Assisted/
#18 cardiovascular diseases/	#54 or/28–53	#90 Decision Support Systems, Clinical/
#19 coronary disease/	#55 27 and 54	#91 algorithms/
#20 cardiovascular disease$	#56 17 or 55	#92 algorithm?
#21 heart disease$	#57 new zealand chart$	#93 algorythm?
#22 coronary disease$	#58 sheffield table$	#94 decision support?
#23 cardiovascular risk?	#59 procam	#95 predictive model?
#24 coronary risk?	#60 General Rule to Enable Atheroma Treatment	#96 treatment decision?
#25 exp hypertension/	#61 dundee guideline$	#97 scoring method$
#26 exp hyperlipidemia/	#62 shaper scor$	#98 (prediction$ adj3 method$)
#27 or/18–26	#63 (brhs adj3 score$)	#99 or/88–98
#28 risk function	#64 (brhs adj3 risk$)	#100 Risk Factors/
#29 Risk Assessment/mt [Methods]	#65 copenhagen risk	#101 exp Risk Assessment/
#30 risk functions	#66 precard	#102 (risk? adj1 assess$)
#31 risk equation$	#67 (framingham adj1 (function or functions))	#103 risk factor?
#32 risk chart?	#68 (framingham adj2 risk)	#104 or/100–103
#33 (risk adj3 tool$)	#69 framingham equation	#105 27 and 99 and 104
#34 risk assessment function?	#70 framingham model$	#106 87 or 105
#35 risk assessor	#71 (busselton adj2 risk$)
#36 risk appraisal$	#72 (busselton adj2 score$)

Abstract screening, data extraction and inclusion criteria

Titles and abstracts were initially screened by two reviewers (ADB, PB), and potentially relevant articles were acquired and independently read by the reviewers who also extracted and checked relevant data. Authors of studies with insufficient information were contacted.

External validity—review A

When the external validity of the Framingham risk score was examined, information was extracted on the patient characteristics of the test dataset, as well as risk factors included in the risk score, the disease outcomes, prediction period and statistical methods.

We reviewed studies that evaluated the calibration by means of the risk of coronary heart disease (CHD) or CVD predicted by Framingham risk scores compared with the risk observed in the test population. A model is perfectly calibrated if the predicted risk of a person or a group of people is the same as the observed risk. The predicted and observed risks for all the studies were calculated with the number of events as the numerator and the number of participants as the denominator. For easy comparison between studies of different follow‐up periods, the observed risk for each study was presented as a 10‐year risk.

Effectiveness—review B

For the second review, any published randomised controlled trial that assessed the effectiveness of a healthcare professional using a cardiovascular risk score to aid primary prevention was considered. Control patients were required to have received usual care as provided by a general practitioner or healthcare professional with appropriate treatment and lifestyle recommendations based on current practice. The participants of studies were not subject to any age, sex or nationality exclusion criteria, but were required to be predominantly free from symptomatic CVD (less than 20% of the population studied with clinically established CVD). Patients with diabetes, raised risk factors or given preventive treatment were eligible. Studies were required to provide data on at least one of the following outcomes: cardiovascular or coronary fatal or non‐fatal events, risk factor levels, absolute cardiovascular or coronary risk, the prescription of risk‐reducing drugs and changes in health‐related behaviour such as smoking. Information on the methodological quality of the trials including the method of randomisation, concealment of allocation, baseline group comparisons and blind outcome assessment was collected. Disagreements were resolved by discussion and, if necessary, in consultation with members of the project advisory panel.

RESULTS

We identified a total of 3439 articles, of which 996 were considered potentially relevant to cardiovascular risk assessment and were acquired for assessment (fig 1). We found 52 studies examining the external validity of four Framingham risk scores¹,³,¹⁰,¹¹ in 112 different population groups, of which 34 provided data on predicted and observed risk for combined fatal and non‐fatal CHD or CVD outcomes. The more recent Framingham methods based on these outcomes described by Anderson et al¹ and Wilson et al³ form the basis of widely recommended charts, tables and computer programs. These were subject to validation in 27 population groups and are reported here. Seven studies investigating the validity of two older risk scores not used in clinical practice were excluded,¹⁰,¹¹ as were studies reporting only fatal outcomes.

Figure 1 Flow of studies through the review. RCT, randomised controlled trial.

A further 26 studies that examined the issue of effectiveness of risk‐scoring methods were found, of which four were randomised controlled trials.

External validity—review A

Table 2¹²,¹³,¹⁴,¹⁵,¹⁶,¹⁷,¹⁸,¹⁹,²⁰,²¹,²²,²³,²⁴,²⁵,²⁶,²⁷,²⁸ shows the characteristics of patient groups with predicted to observed ratios based on the Framingham Anderson and Wilson methods. The populations were derived from cohort studies, randomised controlled trials or health checks, or were studies of specific patient groups. Populations varied in age range and sex, date of recruitment and outcomes studied. The groups studied were representative samples of men and women, and people with diabetes, raised cholesterol, treated hypertension, no CHD determined by angiography and a family history of CVD. The start of baseline data collection in the studies ranged from 1961 to 1996. Outcomes were combined fatal and non‐fatal CHD or combined fatal and non‐fatal CVD.

Table 2 Characteristics of studies examining the external validity of the Framingham Anderson and Wilson risk scores.

Location, reference, date	Name of study	Specific inclusions	Men	Age (years)	Start date of recruitment	Follow up (years)	Outcomes	Study type
Framingham Anderson
Europe, Bastuji‐Garin et al17 2002	INSIGHT	All patients taking HT drugs	45%	55–74	1994	3.7	CHD, CVD	RCT hypertension intervention and control groups
Augsburg, Germany, men, Hense et al13 2003	WHO MONICA Augsburg	Men	100%	35–64	1984	10.5	CHD	Community
Augsburg, Germany, women, Hense et al13 2003	WHO MONICA Augsburg	Women	0%	35–64	1984	10.5	CHD	Community
Munster, Germany, men, Hense et al13 2003	PROCAM	Men	100%	35–64	1979	11.6	CHD	Occupational
Munster, Germany, women, Hense et al13 2003	PROCAM	Women	0%	35–64	1979	11.1	CHD	Occupational
N Europe/USA, Kjeldsen et al18 2000	LIFE	Essential HT and ECG‐LVH	46% (overall in study)	55–80	1995	1	CVD	RCT hypertension intervention and control groups
New Zealand, men, Milne et al19 2003	Fletcher‐Challenge–University of Auckland Heart and Health Study	Men	100%	35–74	1992	5	CVD	Community and occupational
New Zealand, women, Milne et al19 2003	Fletcher‐Challenge–University of Auckland Heart and Health Study	Women	0%	35–74	1992	5	CVD	Community and occupational
Scotland, WOSCOPS control, WOSCOPS Group20 1998	WOSCOPS	Moderate hypercholesterolaemia (control)	100%	45–64	1989	4.4	CVD	RCT lipid‐lowering control group
Scotland, WOSCOPS pravastatin, WOSCOPS Group20 1998	WOSCOPS	Moderate hypercholesterolaemia (pravastatin)	100%	45–64	1989	4.4	CVD	RCT lipid‐lowering intervention group
UK, Brindle et al21 2003	BRHS		100%	40–59	1978	10	CHD	Community
UK, Caerphilly & Speedwell 1, Brindle et al22 2004	Caerphilly and Speedwell Studies	1979–1983	100%	45–63	1979	10	CHD	Community
UK, Caerphilly & Speedwell 2, Brindle et al22 2004	Caerphilly and Speedwell Studies	1982–1988	100%	47–67	1982	10	CHD	Community
UK, diabetic men, McEwan et al14 2004	Cardiff Diabetes Database	Men with new diabetes diagnosis	100%	30–74	1996	4	CHD	Patients with diabetes
UK, diabetic women, McEwan et al14 2004	Cardiff Diabetes Database	Women with new diabetes diagnosis	0%	30–74	1996	4	CHD	Patients with diabetes
UK, diabetics, Guzder et al23 2002	Poole	New diabetes diagnosis	57%	30–74	1996	4.2	CVD	Patients with diabetes
UK, Whickham, Ramachandran et al16 2000	Whickham Study		44%	30–75	1972	20	CHD	Community
USA, Orford et al24 2002	Normative Aging Study, Boston	Health check volunteers	100%	30–74	1961	10	CHD	Health check volunteers
Framingham Wilson
Australia, men, Simons et al25 2003	Dubbo Study, Australia	Men	100%	60–79	1988	10	CVD	Community
Australia, women, Simons et al25 2003	Dubbo Study, Australia	Women	0%	60–79	1988	10	CVD	Community
France, Empana et al26 2003	PRIME, France		100%	50–59	1991	5	CHD	Occupational
Germany, angiography, Schulz et al27 2003		Patients with CHD negative coronary angiography	100%	Mean 53	1992	10	MI	Angiography patients
Northern Ireland, Empana et al26 2003	PRIME, Belfast		100%	50–59	1991	5	CHD	Community and occupational
USA, Johns Hopkins, siblings 1, Blumenthal et al12 2003	Johns Hopkins Sibling Study	Family history of premature CHD	64%	Mean 46		10	CHD	CVD family history
USA, Johns Hopkins, siblings 2, Becker et al15 2001	Johns Hopkins Sibling Study	Family history of premature CHD	52%	30–59	1983	10	CHD	CVD family history
USA, Los Angeles, Greenland et al28 2004	South Bay Heart Watch, Los Angeles	High risk	90%	>45	1990	7	CHD	High risk

BRHS, British Regional Heart Study; CHD, coronary heart disease; CVD, cardiovascular disease; HT, hypertension; INSIGHT, Intervention as a Goal in Hypertension Treatment; LIFE, Losartan Intervention For Endpoint reduction in hypertension study; LVH, left ventricular hypertrophy; PRIME, Prospective Epidemiological Study of Myocardial Infarction; PROCAM, Prospective Cardiovascular Munster; RCT, randomised controlled trial; WHO MONICA, World Health Organization Multinational Monitoring of Trends and Determinants in Cardiovascular Disease; WOSCOPS, West of Scotland Coronary Prevention Study.

Figure 2 shows predicted to observed ratios in populations ordered by level of observed risk of fatal and non‐fatal CHD and CVD. No summary estimate is presented due to the considerable heterogeneity between the studies as indicated by the large χ² and I² scores. For CHD, the predicted to observed ratios ranged from an underprediction of 0.43 in a study of people with a family history of CHD¹² to an overprediction of 2.87 in women from Munster.¹³ Underprediction was observed in studies of higher‐risk patients, specifically patients with diabetes¹⁴ and a family history of premature CHD¹²,¹⁵ and in a higher‐risk UK primary care population.¹⁶ For CVD, there was a similar trend of increasing underprediction with increasing risk of the population (the INSIGHT (Intervention as a Goal in Hypertension Treatment) trial excepted), although the range from maximum overprediction to maximum underprediction was less than that for the CHD outcome. This reflects the smaller number of studies available and the narrower range of background 10‐year risk between them. The INSIGHT trial compared the effectiveness of two different hypertension treatment regimens and was an exception to this trend, probably because all the participants received blood pressure‐lowering drugs and many were also taking concurrent cholesterol‐lowering drugs—variables not included in the Anderson equation.¹⁷

Figure 2 Studies examining the predicted to observed ratio of Framingham Anderson and Wilson risk scores, ordered by the observed 10‐year risk (%) in the test populations. BRHS, British Regional Heart Study; CHD, coronary heart disease; CI, confidence interval; CVD, cardiovascular disease; INSIGHT, Intervention as a Goal in Hypertension Treatment; LIFE, Losartan Intervention For Endpoint reduction in hypertension study; PRIME, Prospective Epidemiological Study of Myocardial Infarction; RR, relative risk; WOSCOPS, West of Scotland Coronary Prevention Study.

Effectiveness—review B

Table 3 shows the study characteristics of the four randomised controlled trials. Three of the studies included patients with a predefined diagnosis of hypertension,²⁹,³⁰,³¹ and the other comprised exclusively patients with diabetes.³² Two of them used computerised clinical decision support systems,²⁹,³⁰ and the others informed doctors of the patient's risk either directly³¹ or by recording it prominently in the medical notes.³² The risk scores used were based on the Framingham Anderson 1991 “all CVD events” equation¹,²⁹,³¹,³² or the Westlund Score derived in a Norwegian population.³⁰ Outcomes related to absolute risk, treatment, referral and changes in risk factor levels.

Table 3 Controlled trials examining the effectiveness of using a risk score to aid primary prevention of CVD.

Study and participants	Intervention	Results*			Study quality
		Outcome†	Intervention (95% CI/SD)	Control (95% CI/SD)
Hall et al32: Hospital outpatients with type 2 diabetes, age 35–75, 52% men	CVD risk score documented at front of patient notes	All patients	n = 162	n = 161	Alternate allocation of participants
		Diabetes treatment	42% (34 to 50)	36% ( 29 to 45)	Doctors were unaware of allocation
		Hypertension treatment	16% (10 to 22)	10% ( 5 to 16)	Length of follow up, losses to follow up unreported
		Lipid‐lowering treatment	12% (7 to 17)	9% (4 to 14)
		Referral to dietician	10% (6 to 15)	13% (7 to 19)	No power calculation
		High‐risk patients (>20% 5‐year risk)	n = 86	n = 82
		Diabetes treatment	44% (35 to 54)	35% (24 to 47)
		Hypertension treatment	23% (15 to 31)	10% (3 to 17)
		Lipid‐lowering treatment	20% (12 to 27)	9% (2 to 15)
		Referral to dietician	10% (5 to 16)	7% (1 to 17)
Montgomery et al29: General practice patients with hypertension, age 60–80, 46% men	1. Computer‐based CDSS and risk chart		n = 202	n = 130	Cluster randomisation of general practices
		Change in mean 5‐year risk ⩾10%	0.65 (0.39)	0.77 (0.37)	Participants not blinded to study group
		Change in SBP (mm Hg)	−0.04 (1.4)	0.25 (1.7)‡	Losses to follow up were 10% at 6 months and 14% at 12 months
		Change in DBP (mm Hg)	0.36 (0.74)	−1.64 (1.03)‡
		OR§ for taking 2 classes of drugs v 0 or 1	0.5 (0.2 to 0.9)	0.5 (0.2 to 1.0)	Power calculation using intracluster correlation coefficient from a published study
		OR for taking ⩾3 classes of drugs v 0 or 1	0.3 (0.1 to 0.6)	0.3 (0.2 to 0.7)
	2. Chart only, interventions by GP or practice nurse. 6 and 12 month follow up		n = 199
		Change in mean 5‐year risk ⩾10%	−0.48 (0.35)
		Change in SBP (mm Hg)	2.66 (1.4)
		Change in DBP (mm Hg)	−1.1 (0.78)
		OR§ for taking 2 classes of drugs v 0 or 1	1.0
		OR for taking ⩾3 classes of drugs v 0 or 1	1.0
Hanon et al31” Patients with hypertension, age 19–74, 54% men	10‐year CVD risk communicated to GP as <15%, 15–20%, 20–30%, >30%. Controls had CVD risk estimated by physician. 8 week follow up		n = 556	n = 712
		Framingham Anderson 1991 10‐year CVD risk	26 (12)	25 (12)	Individual randomisation
					Physicians not blind to intervention
		Change in SBP (mm Hg)	−27	−26	No information on assessor blinding
		Change in DBP (mm Hg)	−15	−16	17% lost to follow up
		Patients with BP <140/90 mm Hg	64%	62%	No power calculation
		Patients taking 2 hypertension drugs rather than 1	41%	46%	No confidence intervals reported
Hetlevik et al30: General practice patients with hypertension, age range not reported, 42% men	Computer‐based CDSS incorporating Westlund MI risk score	Change in SBP (mm Hg)	−2.3 (n = 816)	0.8 (n = 1023)	Cluster randomisation of health centres
		Change in DBP (mm Hg)	−1.8 (n = 816)	−1.2 (n = 1023)	56/213 (26%) of invited GPs were randomly assigned
		Change in serum TC (mmol/l)	0.04 (n = 581)	−0.13 (n = 768)	Unblinded
					Uptake of computerised CDSS was only 12% in intervention group
					Some risk score variables were missing for >91% of participant
					No power calculation
					No confidence intervals reported

*Results for Montgomery et al unadjusted for baseline blood pressure (BP) and practice computer system.

†Change in numbers of patients receiving treatment or being referred to dietician.

‡Mantel–Haenszel p<0.02.

§Odds ratio (OR) <1 indicates less likely to take >1 class of drug.

CDSS, clinical decision support system; CVD, cardiovascular disease; DBP, diastolic blood pressure; GP, general practitioner; MI, myocardial infarction; SBP, systolic blood pressure; TC, total cholesterol.

Hall et al³² recruited 167 men and 156 women with type 2 diabetes attending a hospital outpatients' clinic in Dundee, Scotland, and allocated 162 of them to an intervention group and 161 to a control group. The intervention group had the cardiovascular risk score documented on the front of the notes and the control group did not. The authors found that overall the intervention and control groups did not differ in change of diabetes treatment, change in hypertension drugs, change in lipid‐lowering drugs or referral to a dietician. However, they noted that within a high‐risk subgroup of patients (> 20% five‐year risk) those in the intervention group were more likely to be prescribed blood pressure‐lowering (23% v 10%) and lipid‐lowering drugs (20% v 9%) than in the control group (p  =  0.01 for both comparisons).

Montgomery et al²⁹ used a cluster randomised controlled trial design with 614 patients from 27 general practices in Avon, England. Patients were randomly allocated to a computerised clinical decision support system plus cardiovascular risk chart; cardiovascular risk chart alone; or usual care. The authors found no differences between the computerised clinical decision support system plus chart group and the usual care group, but the chart‐only group had significantly lower systolic blood pressure and was more likely to be prescribed cardiovascular drugs than the control group. Information on adherence to the intervention by the doctors and nurses was not supplied.

Hanon et al³¹ randomly assigned 1526 patients with hypertension from 953 general practitioners in France to two groups, where one group of general practitioners were told the patients' calculated risk and the other group were not. They found no difference between the two groups in the final blood pressure, 10‐year CVD risk or proportions prescribed two hypertension drugs compared with monotherapy.

Hetlevik et al³⁰ offered a computerised clinical decision support system to 17 Norwegian health centres in the intervention group, and the general practitioners in the control group practised usual care. They found no clinically significant difference in blood pressure or total cholesterol between the two groups at the end of 21 months' follow up. Despite the doctors having an average of 1.5 h of training on the clinical decision support system, it had been used in the treatment of only 12% of the patients in the intervention group.

Trial quality

As few trials were found, none that met the inclusion criteria were excluded because of their study quality. The information reported was limited, making formal comparison with set quality criteria difficult (table 3).

DISCUSSION

This systematic review has shown that the accuracy of the Framingham risk scores cannot be assumed and that it relates to the background risk of the population to which they are being applied. We have also found no strong evidence supporting the assumption that cardiovascular risk assessment performed by a clinician improves health outcomes. Screening of the population with Framingham‐based risk‐scoring methods continues to be recommended in current guidelines in the UK and elsewhere.⁶,⁷,³³,³⁴ The lack of evidence supporting the effectiveness of risk scores and the variable accuracy of the screening methods is of concern.

We found only four randomised controlled trials that had investigated the effectiveness of cardiovascular risk‐scoring methods, in contrast to the volume of information about the accuracy of risk prediction with 52 studies examining the external validity of the Framingham risk scores. In particular, no studies included people without hypertension or diabetes—the patients who often require a cardiovascular risk assessment to determine need for drug treatments. The two studies that used computerised clinical decision support systems showed very poor uptake by the doctors in one trial³⁰ and a negative effect when added to a risk chart in the other,²⁹ suggesting that including clinicians in the design of decision aids may improve their use.

Strengths and limitations

This review used a sensitive search strategy with no language restrictions, and it was performed according to standard Cochrane review methods. Comparison of the studies assessing the external validity of the Framingham scores was difficult. There is no standard format for applying them or assessing their quality, and each of the studies had slightly different inclusion criteria, methods of case ascertainment and end point definitions. Broad CHD and CVD end points including fatal and non‐fatal outcomes had to be used due to the variable definitions. Had it been possible, it would have been preferable to separate the harder CHD outcomes, such as non‐fatal myocardial infarction and coronary death, from outcomes that include angina pectoris. We have not examined the ability of the Framingham scores to accurately identify high‐ and low‐risk patients (discrimination) and concentrated only on calibration in different populations. We recognise that the discriminatory ability of a model is an important property; however, it is the calibration that varies most between populations and it is more amenable to adjustment.³⁵

Other studies

To our knowledge, this is the only study that has reviewed the international literature on the effectiveness of calculating a risk score. One existing review on the validity of Framingham prediction rules included only three studies with data on predicted to observed ratios.³⁶ Our results are consistent with a study by the Diverse Populations Collaborative Group, which examined the accuracy of a single Framingham proportional hazards predictive function in 16 observational studies.³⁷ Unlike the models used in studies in our review, their model had not been used as a risk score in clinical practice. Nevertheless, like us, they concluded that their model tended to overpredict absolute risk in populations with low observed CHD mortality and to underpredict risk in populations with high CHD mortality.

Implications

The findings of this review suggest that true cardiovascular risk in low‐risk populations is likely to be overestimated, perhaps leading to unnecessary treatment of many patients. Conversely, in high‐risk populations, true cardiovascular risk is likely to be underestimated, potentially resulting in these high‐risk people not reaching a treatment threshold and being denied appropriate drug treatment. For example, in a deprived Scottish population Framingham predicted CVD mortality risk tended to increase with increasing socioeconomic deprivation. However, this significantly underestimated the observed gradient of increasing risk across socioeconomic groups.³⁸ This inaccuracy between populations is relevant to other cardiovascular risk‐scoring methods based on similar combinations of risk factors. Including a variable representing social deprivation may improve the performance of risk prediction models. Recalibrating the prediction models to adjust for the background risk of different geographical regions⁴ and ethnic groups³⁵ is an alternative solution.

No matter how well calibrated a risk score may be, its primary purpose is to improve the management of those patients it identifies as being at high cardiovascular risk. This involves understanding how a clinician and patient interact once cardiovascular risk has been assessed. While absolute cardiovascular risk assessment remains the recommended method of targeting primary prevention, considerable work is needed to make it a practical and effective clinical tool.

Abbreviations

CHD - coronary heart disease

CVD - cardiovascular disease

INSIGHT - Intervention as a Goal in Hypertension Treatment