The Framingham Heart Study’s Impact on Global Risk Assessment

Abstract

Cardiovascular Disease (CVD) is the leading cause of mortality, responsible for about 30 percent of deaths worldwide. Globally, 80 percent of total CVD deaths occur in developing countries. In recent years age-adjusted CVD death has been cut in half in developed countries. Much of the decline is due to reductions in risk factors which the Framingham Heart Study helped to identify. The Framingham Heart Study also helped to classify those at highest risk by creating multivariate risk scores. As a result other investigators have created various risk prediction scores for their countries. These scores have been the foundation for guidelines and prevention strategies in developed countries. However, most scores requiring blood tests may be difficult to implement in developing countries where limited resources for screening exist. New studies and risk scores inspired by the Framingham Heart Study need to simplify risk scoring in developing countries so affordable prevention strategies can be implemented.

INTRODUCTION

At the close of World War II in 1945, US President Franklin Delano Roosevelt met with Winston Churchill and Josef Stalin at the Black Sea Resort of Yalta. The leaders of the major allied countries, the United States, the United Kingdom and the Soviet Union, were meeting to discuss the plans for post war Europe in the likely event of an allied victory over the axis nations. At the time of the meeting in Yalta, President Roosevelt’s blood pressure was 260/150 mm Hg. According to some historians Roosevelt was too weak and tired at the meeting to assert a stronger stand regarding the protection of Eastern European countries after the war (1,2). He died 2 months later from a massive stroke. One historian wonders if Roosevelt’s health had been better (3), whether the Cold War, which shaped European and American foreign policy for the next 50 years and fueled nuclear proliferation at enormous financial cost, would have been as severe and so consuming of human and financial resources.

One of the preeminent cardiologists at the time, Dr. Paul Dudley White of the Massachusetts General Hospital and Harvard Medical School, as well as the physicians treating the President, believed that hypertension was a compensatory response to atherosclerosis and “should not be tampered with”(4). It would take a later observational cohort such as the Framingham Heart Study to better understand the causal relationship between hypertension and other risk factors and coronary heart disease (CHD). The Framingham study was the first of its kind to investigate cardiovascular risk factors in a well-constructed and followed cohort. It originally consisted of 5,209 men and women who were free of CVD between the ages of 30 and 62 recruited beginning in 1948 from Framingham, Massachusetts (5).

One of the first breakthroughs in our understanding of CHD risk came with the early analysis of the cohort, 12 years into its existence. The 1961 paper by the Framingham Study’s Dr. William Kannel and colleagues published about the “Factors of Risk” enabled physicians and scientists to be more certain that blood pressure and other risk factors “precede the development of overt coronary heart disease and are associated with increased risk of its development” (6). It was from this time forward that we began to better understand the risk factors for coronary heart disease and stroke, and thus were able to develop both population-based and personal interventions targeting these risk factors.

The Framingham Study further reported risk equations linking common risk factors to both coronary heart disease, stroke, and overall fatal and non-fatal cardiovascular disease (CVD) (7,8). These publications led to improved screening techniques and serve as the backbone of many clinical guidelines throughout the world. This paper reviews the evolution of the Framingham Risk Score and its impact on guidelines, as well as other risk scores used throughout the world by guideline committees and individual clinicians in the care of patients with CVD. Further we will summarize some major CVD risk scores that have been validated in large cohort populations motivated by the Framingham experience and present the variables that are used in their construction. We will then compare the main scores using data available for receiver operator characteristic (ROC) curves, as well as internal and external validation studies. We also discuss limitations of various risk assessment tools in both developed and developing countries. Finally, we present a comparison checklist for policymakers to use in evaluating the utility of different cardiovascular disease risk scores in low-resource settings.

HISTORY OF FRAMINGHAM AND THE ABSOLUTE RISK SCORE

After identifying individual risk factors for coronary heart disease, the Framingham Heart Study took the next step in risk assessment by enhancing methods to asses one’s overall risk based on the multiple risk factors. Absolute global risk is the actual risk of developing a disease in question by the defined population(9). Absolute global risk is calculated by using several key risk factors identified by carefully constructed prospective longitudinal studies. The absolute risk of cardiovascular disease is strongly influenced by the combination of risk factors present, particularly a history of cardiovascular disease, age, gender, diabetes, smoking, blood pressure, and blood lipid concentrations(10). Risk scores use risk function equations based on multivariate risk models derived from large population databases. Risk scores individualize the absolute risk by calculating a score based on multiple continuous or dichotomous risk factors. The use of continuous variable estimation is based on the notion that many of the most important risk factors produce a graded increase in risk per variable sub-category.

In 1967 Truett et al published a multivariate analysis on the Framingham Cohort(11). In this analysis, seven risk factors were used to create a risk function in men and women aged 30–62: age, cholesterol, systolic blood pressure, relative weight (adjusted for sex and height), hemoglobin, cigarette smoking, and ECG evidence of left ventricular hypertrophy (LVH). No formal validation was done at the time and the for the most part the tool was not adopted clinically. Over time hemoglobin was dropped from the risk factor equations and glucose intolerance was added. A book of risk tables was published by the American Heart Association (AHA) (12) and the original equations could be used but at that time they were still not in widespread use.

The next major advance came in 1976. Kannel et al confirmed (13) the continuous nature of the risk factors, showing that those with a systolic blood pressure below 140 mm Hg still had significant risk of a CVD event. This analysis was important for two reasons. First, it showed clinicians that a patient with moderate levels of multiple risk factors could have an overall risk of CVD that was greater than a patient with an isolated high level of one risk factor. Further, the Framingham investigators showed that the risk factors had differential effects on various CVD outcomes. For example, the risk associated with blood pressure was much greater for stroke and congestive heart failure than it was for CHD and claudication. Moreover, whereas the risk associated with cholesterol was more highly associated with CHD and claudication than with stroke, there was a lesser magnitude of difference in the beta-coefficients between the two diseases. A pocket guide on CVD risk calculation for clinicians was produced in 1982(14).

By the 1990’s major improvements were made with the addition of HDL cholesterol and the extended duration allowed for risk assessment in an older population up to the age of 74. A point scoring system(7) was added to ease the use for clinicians who wanted to calculate risk in individual patients. In 1998 LVH was dropped from the risk scoring mechanism due in part to its correlation with blood pressure(15). Further changes included the impact of blood pressure treatment on risk assessment and ultimately tools for the computing risk scores on the web were added to increase the ease of use among clinicians. These additions made it possible for clinicians to come up with a probability of a patient developing coronary heart disease without having to use a calculator. Most recently a tool was developed for CVD for use in primary care. (16)

IMPACT ON GUIDELINES

After the 1961 paper on “Factors of risk”(6), national expert panels developed guidelines for risk management(17). Initially, separate guidelines were developed for each individual risk factor and treatment was recommended when the risk factor reached a threshold above a specified level(10). One limitation, however, wass that for any given level of a risk factor there was a broad range of overall risk for CVD depending on the level of other known risk factors. During the 1970’s numerous efforts were made using multivariate regression techniques to create models for predicting an absolute CVD risk using the associated individual risk factors. Absolute risk scores carry an additional advantage in that they have been adapted into easily used score calculators that are available on paper and increasingly on the internet(18). These efforts were the predecessors to cumulative risk tables and scores. However, up until now risk scores have been developed and validated almost exclusively in high-income countries. Clinical guideline recommendations on assessing risk and preventing cardiovascular disease have undergone significant change over the past two decades. More recent clinical guidelines and studies focus on global absolute risk as an important key to assessing individual risk(19–24).

These efforts playeda significant role in halving the age-adjusted death rate from over four hundred to two hundred per hundred thousand population in the United States between 1960 to 2010. Cardiovascular disease is currently the largest cause of morbidity and mortality in the world(25). While there have been improvements in age-adjusted CVD mortality rates in the developed world, CVD represents a large and growing cause of death and disability in the developing world. Given its enormous burden and current expensive treatment, finding low-cost prevention strategies is a top priority. The origin of identifying those at highest risk lies with the development of the Framingham Risk Score, and has been a cornerstone to personalized prevention strategies ever since..

OTHER RISK SCORES

Based on the success of the Framingham Heart Study, other scores were created from various cohorts throughout the U.S. and other European countries. We include seven other scores for review in this paper: PROCAM, SCORE, CUORE, ASSIGN, QRISK, Reynolds, and the NHANES Follow-Up Study Cohort (NHEFS) non laboratory-based score. We believe these risk tools to be the most robust and best validated among those currently published. Overall, however, a dearth of research exists regarding the validity and generalizability of these scores outside their regions of origin with the exception of the Framingham score.

First we present the characteristics of the study populations that formed these cohorts, including duration of follow-up. We also present the risk factor variables used in the construction of these risk assessment tools. Then we looked at endpoints used for predicting and evaluating risk in each cohort. Next we examined the performance of each risk tool using both internal and external validity measures. One way to examine a screening tool is to use sensitivity and specificity measures. Another way to assess the validity of a screening tool such as a risk score is by plotting the sensitivity of the tool versus (1-specificity) for each threshold of a patients risk as being positive. This plot is called the receiver operator characteristic (ROC) curve. The area under the curve (AUC) is often referred to as the “c-statistic.” Values close to 1.0 are considered a perfect predictive model and values close to 0.5 are no better than chance. While no specific standard exists many screening tools in use have a c-statistic of 0.70 or greater. We present estimates of internal and external validity using ROC curves. Internal validity can be thought of as the consistency and soundness of a study’s findings that cannot be attributed to confounding. External validity can be conceptualized as the generalizability of results to other populations outside of the original study population. Both internal and external validity results can be presented as ROC curves and sensitivity/specificity measures. Finally we present a comparison sheet for evaluating the various risk assessment tools in developing world settings. Specifically, we analyze each score’s utility with respect to ability to rank risk appropriately, validation across different population and geographic settings, and ease of administration.

Cohort derivation and characteristics

Table 1 identifies each score’s baseline derivation, population characteristics, risk assessment variables, and main outcomes measured. The Prospective Cardiovascular Munster (PROCAM) study followed 5389 men aged 35–65 to determine the number of acute coronary events over 10 years(26). Recruitment for the study began in 1979 and ended in 1985. The study cohort excluded men who had a history of MI, stroke, angina, or EKG changes consistent with prior ischemia. The authors identified age, LDL, smoking, HDL, systolic blood pressure (BP), family history of myocardial infarction, diabetes, and triglycerides as the most important variables in their proportional hazards model. The PROCAM score uses the same risk factors as Framingham plus triglycerides and a family history of heart disease to predict the 10 year risk of a first coronary heart event (cardiac arrest and fatal and nonfatal MI) in these men. A follow-up cohort including women is being studied currently. No minorities or individuals greater than 65 years of age were included. The endpoints were limited to coronary disease outcomes, including fatal and non-fatal MI and sudden death.

TABLE 1.

Cohort Description

Cohort	Population	Location	Time of subject entry	Number of subjects	Age	Gender	Variables	End Points	Time of predicted risk
Framingham	General population (without pre- existing CVD)	Framingham MA (USA)	1968–75	5573	30–74	♂; ♀	Age ; SBP; DBP; TC; HDL; DM; Sm; RxBP	AMI Coronary Death(SD and NSD) AP; CVA	10 years
PROCAM	Company employees	Munster Germany	1979	4407♂	40–65	♂;	Age SBP; HDL; LDL; DM; Sm; AP; FHx	AMI (Non-fatal and Fatal); SD	10 years
SCORE	Multiple country general population cohorts	Finland Russia Norway UK Denmark Sweden Belgium Germany Italy France Spain	1967–1991	117098♂ 88080♀	19–80	♂; ♀	SBP; TC; TC/HDL; Sm; Age [ age used as temporal variable] [ separate scores for high/low risk countries]	CV mortality from: CHD; SD; CHF; PVD	10 years
CUORE	11 cohorts of men	Italy	1982–96	6865♂	35–69	♂;	Age; SBP; TC; Sm; DM;	Fatal and nonfatal major: -coronary events -CVA RvTx (Coronary and Carotid); SD	10 years
ASSIGN	Random community sample	Scotland	1983–87	6450♂ 6757♀	30–74	♂;♀	Age; Sex; TC; HDL; SBP; DM; Sm; FHx; Area-based deprivation index	Fatal and nonfatal CV events	10 years
QRISK	General Practice Database	United Kingdom	1993–2008	1.28 million (QRISK1) 2.29 million (QRISK2)	35–74	♂;♀	Age; Sex; TC/HDL; SBP; Sm; DM; FHx; RxBP; BMI; Area-based Deprivation Index; Ethnicity (QRISK2 only); Chronic disease history (QRISK2 only)	Fatal and nonfatal CV events	10 years
Reynolds	Women’s Health Study (WHS) Participants (women over the age of 45 without cardiovascular disease or cancer) Physicians Health Study (PHS) II participants	United States	1992 – 2004 (WHS) 1995–2008 (PHS)	24558♀ 10724♂	≥ 45 > 50	♂;♀	Age; Sm; SBP; TC; HDL; hsCRP; FHx (< 60); HgbA1c (not men);	AMI; CVA (ischemic); RvTx (Coronary); CV Death	10 years
NHEFS	NHANES I participants without cardiovascular disease	United States	1971 – 1975 (followed through 1992)	3349W 2837♂	25 – 74	♂; ♀	Age; SBP; Sm; DM; RxBP; BMI	CV death; AMI; CVA; RvTx (Coronary); CHF; CABG; PTCA )	5 years

Abbreviations:

CV – Cardiovascular

CVD – Cardiovascular Disease

SBP – Systolic Blood Pressure

DBP – Diastolic Blood Pressure

HDL – High Density Lipoprotein

LDL – Low Density Lipoprotein

DM – Diabetes Mellitus

Sm – Smoking

AP – Angina Pectoris

FHx – Family History of Myocardial Infarction

RxBP – Treatment for Hypertension

AMI – Acute Myocardial Infarction

SD – Sudden Death

NSD – Non-Sudden Death

CVA – Cardiovascular Accident

CHF - Congestive Heart Failure

PVD - Peripheral Vascular Disease

RvTx - Revascularization Treatment

hsCRP – high sensitivity C-Reactive Protein

BMI – Body Mass Index

CABG - Coronary Artery Bypass Grafting

PTCA- Percutaneous Transluminal Coronary Angioplasty

HgbA1c – Hemoglobin A1c

The SCORE (Systematic COronary Risk Evaluation) project pooled data sets from 12 European countries in order to develop a risk scoring system applicable to European populations(27). The cohorts used for the SCORE started in the 1960s with the majority recruited in the 1970s and early 1980s. A total of 205,178 people (88,080 women and 117,098 men) were followed for 2.7 million person-years. Ten-year risk of fatal cardiovascular death (as opposed to events measured in other scores) was calculated using a Weibull model that used age as a measure of exposure time to risk as opposed to a separate risk factor. Separate equations were calculated for coronary and non-coronary cardiovascular disease death, as well as high and low risk areas in Europe. The variables incorporated into this model included age, gender, total cholesterol, HDL, systolic blood pressure, and smoking. Endpoints include cardiovascular mortality from coronary heart disease, heart failure, sudden death, peripheral vascular disease.

The aim of the CUORE study was to derive 10 year coronary risk prediction equation for low-risk Italian men, and then compare the accuracy of this equation with PROCAM and Framingham. Previous work recognized that the Framingham equations overestimated risk in Italian populations(9). The CUORE study was a prospective fixed-cohort study of 6865 men from 11 cohorts in north and south-center Italy studied between 1982 and 1996 without CVD at baseline(28). The CUORE equation includes age, total cholesterol, systolic blood pressure, HDL, diabetes, hypertension drug treatment, smoking, and family history of CVD. Endpoints include fatal and nonfatal major coronary and cerebrovascular events, as well as revascularization and sudden death.

The ASSIGN score is derived from a prospective study in Scotland of a random community sample of 6540 men and 6757 women between the ages of 30 and 74(29). The investigators in the Scottish Heart Health Extended Cohort (SHHEC) collected baseline data between 1984 and 1987 and calculate 10 year risk of CVD events based on age, sex, smoking (measured as number of cigarettes), total cholesterol, HDL cholesterol, systolic blood pressure, diabetes, geographical deprivation index, and family history of coronary heart disease. The inclusion of social deprivation scores derived by area of residence, as well as family history, were novel features of the ASSIGN risk score.

The QRISK scores (QRISK1 and QRISK2) were formulated off of the QRESEARCH database in the United Kingdom. This database included health records of patients seen in UK general practice settings with baseline data collected between 1993 and 2008(30). A total of 1.28 million patients were included in the first QRISK1 score, which measured age, sex, total cholesterol to HDL cholesterol ratio, systolic blood pressure, smoking, diabetes, an area-based deprivation index, family history of coronary heart disease, BMI, and use of antihypertensive treatment(30). QRISK2 included 2.29 million subjects and also added ethnicity and chronic disease history variables to the score(31). Notably, the QRESEARCH database of UK patient contained substantial missing data so imputation was necessary to derive the score. The QRISK scores include terms to adjust for the interactions between age and other variables in the model.

The Reynolds Risk Score(32) was developed within the Women’s Health Study cohort. Study participants were derived from a nationwide cohort of US women 45 years and older free of cardiovascular disease and cancer at study entry initiated in September 1992. The score was developed to assess the impact of a novel biomarker on more traditional risk factors. Risk factors included those in the FCRS as well as high-sensitivity C-reactive protein (hsCRP), glycated hemoglobin A_1C and parental history of myocardial infarction before the age of 60. Endpoints included were incident myocardial infarction, ischemic stroke, coronary revascularization, and cardiovascular deaths. A Reynolds Risk Score for men was also developed for non diabetic males in the Physicians Health Study II Cohort (33) with the hemoglobin A_1C removed from the risk equation.

The NHEFS Non Laboratory-based Score was developed in the prospective cohort study of all NHANES I participants who were between 25–74 years-old (N=14,407) at the time they were first examined between 1971 and 1975(34), and followed through 1992. The NHEFS non laboratory-based risk score uses the same risk factors as Framingham except cholesterol is replaced by body-mass index (BMI) and was designed initially to be used in settings with limited resources.(35) The study population included participants with complete information on these surveys who did not report a history of CVD (history of myocardial infarction (MI), heart failure, stroke, or angina) or cancer, yielding an analysis dataset of 6186.

RISK ASSESSMENT TOOL VALIDATION

Table 2 identifies internal and external validation studies of the above risk assessment tools and ROC curves. A number of studies have been carried out across predominantly European populations in an attempt to apply risk scores to additional populations. The Framingham equation is the most frequently used, exhaustively studied, and oldest risk score available. In the 1980s, the Framingham equation was validated in another US population and later updated with the addition of age and HDL cholesterol. (36)

TABLE 2.

Tool Validation Using Receiver Operating Characteristic (ROC) Curves

Tool	ROC – Internal Validation	ROC – External Validation
		Cohort Study	AUC
Framingham	D’Agostino (2001): 0.79 M/0.83 W	MONICA Augsberg Study (Hense et al 2003)	0.78 M/0.88 F
		PROCAM cohort (Assman et al 2002):	0.78
		Glostrup Study (Thomsen et al 2002)	0.75
		CUORE (Ferrario et al 2005):	0.72
		China CMRS Cohort (Liu et al 2004)	0.71 M/0.74 W
		6 diverse validation cohorts in the US and Puerto Rico (D’Agostino et al 2001):
		ARIC	0.75 M/0.83 W
		HHP	0.72 M
		PR:	0.69 M
		SHS:	0.69 M/0.75 W
		CHS:	0.63 M/0.66 W
		PHS:	0.63M
		PRIME study (Empana et al 2003):
		Northern Ireland	0.66
		France	0.68
		Second Northwick Park Heart Study (UK) – (Cooper et al 2005)	0.62
		New Zealand Risk Tables (Milne et al 2003)	0.74 M/0.77W
PROCAM	PROCAM Cox model: 0.83	CUORE	0.74
	PROCAM score: 0.82	PRIME study - (Empana et al 2003):
		Northern Ireland	0.61
		France	0.64
		Second Northwick Park Heart Study (UK) – (Cooper et al 2005):	0.63
SCORE	High risk using total cholesterol: 0.70 – 0.81	Austria – (Ulmer et al 2005):	0.76 M/0.78 W
	High risk using total/HDL: 0.71–0.80	UK- (Stephens et al 2004):
	Low risk using total cholesterol: 0.71–0.84	Coronary heart disease:	0.77
	Low risk using total/HDL: 0.75–0.82	Cardiovascular disease:	0.74
CUORE	CUORE model: 0.74	None	None
ASSIGN	Woodward (2007): 0.73M/0.77W	None	None
QRISK	QRISK2 Hippisley-Cox (2008): 0.79M/0.82W	UK – THIN Database (Hippisley-Cox et al 2008):	0.76 M/0.79 W
Reynolds	Simplified (without diabetes): 0.81W/0.70M	None	None
NHHEFS	Non-laboratory model: 0.83 Laboratory model: 0.83	None	None

The Framingham coronary heart disease risk score (FCRS) is the score that has been best validated across different populations. A recent analysis showed that the score performed well in at least 6 populations across the US.(37) When looked at in populations other than the Framingham population, recent evidence indicates that the c-statistic or the area under the ROC curve for the FCRS varies between 0.63 to 0.83 in different populations.(37) Framingham scores were found to overestimate risk of initial CAD events in Native Americans, Japanese-American, Hispanic(37), European(38,39), and Chinese populations(40). In an analysis of a large Chinese cohort, the Framingham score was recalibrated using local CAD rates and cohort average levels of risk factors.(40) This recalibration resulted in a more accurate estimation of individual risk. Published and updated data using the FRCS score uses recalibration techniques to better estimate risk among minority populations in the United States.

The Sheffield risk tables used in the UK are derived from the Framingham population. For people without a history of CVD the table presented in Haq et al. (1995) shows each combination of age, sex, tobacco smoking status, hypertension, history of diabetes, and left ventricular hypertrophy on electrocardiogram, for which there is a total cholesterol level that confers a 1.5% or greater risk of having a fatal coronary event within one year(41). These are the patients who are predicted to most benefit from statin treatment. The Sheffield tables were modified by Wallis et al. (2000) to show thresholds of risk (30% or 15%) of a fatal CVD event over 10 years and use the total/HDL cholesterol ratio rather than the total serum cholesterol alone(42). This updated table was tested with a population of 1000 people examined from the 1995 Scottish Health Survey. The performance characteristics of the modified Sheffield tables were compared with the “gold standard” risk calculated from the Framingham algorithms to produce sensitivity and specificity calculations for this population at different risk levels(42). The modified tables had a sensitivity of 97% and specificity of 95% for people who had a coronary risk of >15% over 10 years. The sensitivity was 82% and specificity was 99% for those people with a risk of >30% over 10 years. Those people with a risk of >20% who also had mild hypertension had a sensitivity of 88% and specificity of 90%(42).

The New Zealand risk charts are also based on the Framingham Heart Study algorithm of Anderson et al(43). The risk charts incorporate the following risk factors: age, sex, systolic blood pressure, total cholesterol, HDL cholesterol, history of diabetes, and tobacco smoking status. The benefit of drug treatment is described as the number of CVD events prevented per 100 treated individuals for 5 years. ROC curve analysis showed that the AUC for the Framingham original risk equation and the New Zealand risk charts were nearly identical in a validation cohort in New Zealand(44).

Based on evidence that Framingham scores may overestimate risk in European populations, PROCAM researchers developed a scoring system to predict individual risk, with ROC statistics of approximately 0.83(26). In comparison, the Framingham model ROC statistic in this population was 0.78. Like Framingham, the PROCAM study has potentially limited external validity due to its geographical setting and population homogeneity. More importantly, the initial cohort only included men. Haq et al. (1999) showed that the Sheffield, modified Sheffield, and New Zealand tables had sensitivities ranging from 90% to 97% and specificity ranging from 43% to 63% when compared with PROCAM model in a cohort of 216 people in the UK(45).

In the SCORE study, the AUC was calculated between 0.70 and 0.84(27). In high-risk countries for which ROC curves were applied, the values ranged from 0.70 – 0.81. Low-risk countries had ROC curves that ranged from 0.71–0.84. One external validation study was done in Austria using a large cohort of more than 44,000 people found that the AUC of the SCORE was between 0.76 and 0.78(46).

The AUC for the CUORE equation was calculated as 0.75(28). In comparison, in this population the revised Framingham equations had an area under the curve of 0.72 and the PROCAM was measured at 0.74. For the ASSIGN scores, AUC values were 0.73 and 0.77 for men and women respectively. No external validation scores exist for the CUORE nor the ASSIGN scores.

The QRISK2 score with ethnicity and chronic disease variables had an AUC of 0.79 and 0.82 for men and women respectively. One external validation study was available. The THIN Database in the UK showed an AUC of 0.76 and 0.79 for men and women respectively using the QRISK2 score(31).

The Reynolds Risk Score had an AUC score of 0.81 for women with excellent calibration and an AUC score of 0.71 for cardiovascular events in men. No external validation studies of the Reynold’s Risk Score have been published so far. Framingham investigators found no additional discrimination using novel biomarkers(47). However, there appears to be some benefit for the use hsCRP and family history to reclassify men and women (32,33), particularly in the intermediate risk groups in the Reynolds Risk Score. However, the cost-effectiveness of such approaches has not been explored.

The NHEFs non-laboratory Risk Score had comparable AUCs to the other risk scores of 0.78 and 0.83 for men and women respectively.(48) The AUCs for men and women were the same for the “Framingham” lab-based score with no loss of reclassification at various risk levels. No external validation study of this risk score has been conducted. The advantage of a non-laboratory based method of risk prediction is that it can be applied in one clinic visit with minimum equipment needed - tape measure, scale, and sphygmomanometer or automated blood pressure machine. A risk prediction value can be ascertained and a treatment decision can be made within the same 5–10 minute visit without the cost or the time needed to wait for laboratory results.

USE IN DEVELOPING COUNTRIES

Few of the tools presented are ideal for policymakers in the developing world (Table 3). All eight of the tools appear to rank risk appropriately. Framingham, SCORE, CUORE, ASSIGN, NHEFS non laboratory based, Reynolds, and QRISK are the scores that are based on total cardiovascular outcomes, not just coronary heart disease. This is important as the burden of CVD in the many parts of the developing world is different than in high income countries. In addition, only Framingham, SCORE, ASSIGN, NHEFS, Reynolds and QRISK are inclusive of both men and women. Only Framingham, SCORE, and PROCAM have been evaluated across multiple settings. In the few comparative studies available (none for CUORE nor ASSIGN), they appear to all overestimate risk in settings beyond their original cohorts. Few of the scores include measures of social inequality, and only Framingham has been evaluated in a developing country (China). Only the NHEFS risk score can be easily used in resource poor settings where laboratories are not easily accessible or affordable.

TABLE 3.

Tool Comparisons for Policymaking

Tool Characteristic	Tool
	FCRS	PROCAM	SCORE	CUORE	ASSIGN	QRISK	Reynolds	NHEFS
Ranks risk appropriately	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Inclusive of men and women	Yes	No	Yes	No	Yes	Yes	Yes	Yes
Validated across multiple settings	Yes	Yes	Yes	No	No	No	No	No
Overestimates risk in other settings	Yes	Yes	Yes	Unclear	Unclear	Unclear	Unclear	Unclear
Validated in a developing world setting	Yes (China)	No	No	No	No	No	No	No
Based on total cardiovascular outcomes, not just coronary heart disease	No	No	Yes	Yes	Yes	Yes	Yes	Yes
Includes measures of social inequality	No	No	No	No	Yes	Yes (QRISK2)	No	No
Easy to administer in a low-resource setting (i.e. does not need blood work or electrocardiograms)	No	No	No	No	No	No	No	Yes

DISCUSSION

The Framingham Heart Study has had a major impact on the evolution of risk prediction scores. In turn the risk scores have had a moderate influence on CVD guidelines. Ironically, the use of the Framingham risk equations is less frequently used in clinical practice guidelines in the United States than outside of the country. This is to some extent related to the initial concentration on individual risk factors in the United States. The JNC VII guidelines(49) still use the blood pressure level as the primary determinant for initiating treatment despite a recommendation of assessing overall risk. The National Cholesterol Education Program Expert Panel in its guidelines(20) recommend checking a CHD risk based on Framingham, but the treatment decision is still dependent on cholesterol level which appears redundant given the fact that cholesterol is part of the risk equation. In contrast the World Health Organization, the British and European Hypertension Societies and the European Society of Cardiology and New Zealand (22,23,50–52) have adopted a measurement of overall risk as a baseline for treatment decisions.

The quest to find the most internally and externally valid measurement tool certainly has been led by the Framingham Heart Study but perfect risk prediction has so far proved elusive. A number of high-quality measurement tools based on large well-defined cohorts with long-term follow-up are presented in this review. However, their utility in worldwide clinical practice is limited by a number of important factors, including poor agreement between scores, gender specificity, intrinsic score characteristics, and lack of widespread clinical validation. Moreover, most are based in part on risk factors that require laboratory measurement, making their application in settings without widespread laboratory availability problematic.

Lenz and Mühlhauser (2004) reviewed 12 cardiovascular risk assessment tools (both charts and scores) as well as their respective validation studies(53). Cooney et al. (2009) updated this work to include newer risk scores as well as a detailed discussion of calibration and score limitations(54). These studies examined prognostic accuracy and generalizability across populations and similarly concluded that external validation of all of these instruments is poor and agreement between instruments is only moderate. They recommended individual country recalibration using regional data before implementation.

One major critique of the Framingham risk tables and scores centers on their use of high-risk populations. The Framingham study consisted of mainly Caucasian individuals who had high levels of hypercholesterolemia, intake of saturated fat, smoking, and other important CVD risk factors. Moreover, at the time of entry into this study in the late 1960’s and early 1970’s, most patients were not treated for their CVD risk factors. According to critics of the study, this “high-risk” study population does not mirror other population group risk levels. For example, US populations that have higher risk factor treatment levels and European countries where a “Mediterranean” diet is more prevalent have significantly different levels of current risk. Lenz and Mühlhauser (2004) concluded that Framingham-based instruments (upon which the majority of risk tools are based) overestimate cardiovascular risk for some European populations by at least 30%(53). Thus one advantage of other studies such as SCORE centers on their stratification of high income European countries into low and high-risk groups. However, this intrinsic advantage of SCORE is somewhat dampened by the heterogeneity of data collection across the different cohorts used to construct it. Moreover, some data suggests that SCORE may overestimate risk even in high risk settings such as Germany(55). Current formulations of the Reynolds risk score are targeted exclusively toward low or intermediate risk populations (e.g. those without diabetes or previous CVD). Incorporating measures of area-level social deprivation such as in ASSIGN and QRISK are also important steps forward.

With the notable exception of QRISK2, most risk scores also fail to take ethnic diversity within study populations into account. The Framingham cohort consisted mainly of white men and women in Massachusetts. Subsequent work validated the risk estimates in black Americans, but found large discrepancies in risk estimates among other ethnic minorities. Of note however, recalibration techniques markedly improved validity in these populations. Furthermore, Framingham equations have tended to perform relatively poorly in large European cohorts (56) but with mixed results in Chinese cohorts (40,57). It is possible that the mixed results in China are due to the Framingham focus on coronary endpoints instead of total endpoints, which would include stroke - the most prevalent form of CVD in China(57). Interestingly, while Framingham consistently overestimated risk, it has been shown to accurately rank individuals according to absolute risk(58). Framingham did, however, perform better than SCORE in predicting risk among South Asians living in the UK, likely because it included HDL and diabetes as CVD risk factors(59).

Current assessment of absolute cardiovascular risk requires population screening of blood lipids, blood pressure, and various other risk factors included in the risk scores. While ascertainment of some risk factors such as smoking status and family history are low-cost, screening entire populations for biochemical, electrocardiographic, or even more technologically sensitive measure such as electrocardiographic abnormalities can quickly become expensive on a population level. We present in Table 3 a comparison of different scores from a policymaking perspective. The scores are compared with respect to their inclusiveness, ability to rank and overestimate risk, measurement of total CVD endpoints, and potential for application in low resource settings. As the burden of cardiovascular disease quickly shifts to the developing world and low-resource areas, an urgent need exists to find CVD risk scores that require the minimum of resource expenditures and health care personnel to measure. To this end, scores like NHEFS that are based on easily measured demographic and physical parameters (obesity indices and blood pressure) without the inclusion of laboratory variables have the potential to greatly enhance targeted CVD prevention efforts in low resource settings.

CONCLUSION

The Framingham Heart Study has helped to elucidate major risk factors for CVD. Furthermore, through its various multivariate risk scores it has helped to identify those at highest risk for developing CVD. These screening tools have been replicated in other developed country cohorts and contribute to the screening and management of those at high risk of CVD. These screening strategies have contributed to a reduction in CVD deaths throughout the developed world. Further evaluation is required to determine the best methods for identifying high risk individuals in developing countries where eighty percent of the global CVD burden exists.

ABREVIATIONS

ASSIGN

Assessing cardiovascular risk using the Scottish Intercollegiate Guidelines Network

AUC

Area under the curve

BMI

Body-Mass Index

BP

Blood Pressure

CAD

Coronary Artery Disease

CHD

Coronary Heart Disease

CRP

C-Reactive Protein

CVD

Cardiovascular Disease

EKG

Electrocardiogram

FCRS

Framingham Coronary Risk Score

JNC VII

Joint National Committee 7

LVH

Left ventricular hypertrophy

NHANES

National Health and Nutrition Examination Survey

NHEFS

NHANES Follow-Up Study Cohort

PROCAM

Prospective Cardiovascular Munster

ROC

Receiver operator characteristic

SCORE

Systematic COronary Risk Evaluation

SHHEC

Scottish Heart Health Extended Cohort

THIN

The Health Improvement Network