Validating the Framingham Hypertension Risk Score: A 4‐year follow‐up from the Brazilian Longitudinal Study of the Adult Health (ELSA‐Brasil)

Abstract

The Framingham Heart Study published an equation that permits to estimate the 4‐year incidence of hypertension among adults. In Brazil, only the Brazilian Longitudinal Study of Adult Health (ELSA‐Brasil) of 15 105 men and women aged 35‐74 years enrolled in 2008‐2010 has data that can validate the Framingham Risk Score for Hypertension and create a new equation according to the Brazilian population. We examined the predictive performance of the Framingham Risk Score for Hypertension in the ELSA‐Brasil using as an outcome variable, the 4‐year incidence of hypertension. We split randomly the 8027 participants who participated in the second visit (2012‐2014) and without hypertension at baseline in derivation data set (n = 4825; 60%) and a validation data set (n = 3202 participants; 40%). The area under the curve for Framingham Risk Score for Hypertension and ELSA‐Brasil Risk Score was relatively similar. Hosmer‐Lemeshow chi‐squared statistic applied for the Framingham Risk Score was 3.78 (P‐value = .876) and for our model was 8.22 (P‐value = .41), disclosing good discrimination and calibration for both models. Even with these classification intervals, our model presents more underestimation of the risk, classifying 15% of the participants with new onset of hypertension in low risk vs 9% of the Framingham model and less overestimation of the risk, classifying 17% of the participants without hypertension as high risk vs 24% of the Framingham model. We concluded that the Framingham Risk Score for Hypertension has an acceptable performance when applied in the ELSA‐Brasil population with good discrimination and calibration.

1. INTRODUCTION

The worldwide burden of high blood pressure is related to both, the years of life lost and years living with disability. ¹ , ² Despite the relative decline of levels of blood pressure and prevalence of hypertension, ³ hypertension is the most relevant risk factor for all cause of deaths among adults in lower‐middle‐income countries like Brazil. ⁴

As a result of the high prevalence and the negative impact on cardiovascular health adverse outcomes, the prevention of high blood pressure has been one of the cornerstones of contemporary public health policies. ⁵ The population strategies to reduce high blood pressure prevalence imply policies to reduce the intake of salt, control of the intake of alcoholic beverages, and actions to avoid weight gain. The effectiveness of those actions can lessen hypertension‐related aging and decrease the levels of blood pressure at the population level. At an individual level, beyond the traditional orientation to lifestyle change, a hypertension risk prediction score to predict at short‐time among middle‐aged adults who will present a future diagnosis of high blood pressure can be of paramount importance in primary care clinical practice. ⁶

Longitudinal studies in upper‐income countries as the Framingham Heart Study, ⁷ the Whitehall study, ⁸ and the Multi‐Ethnic Study of Atherosclerosis (MESA) ⁹ have been addressing the prediction of hypertension among populations composed with a majority of people of European ancestry. A review of risk models to predict hypertension emphasized that “most existing hypertension risk models are still at the early stages of the development and evaluation process, and only two of them have been tested in populations different from those used to develop the models.” ¹⁰ Consequently, there is a piece of scarce information related to the validity of those equations, mainly the Framingham Risk Score, for populations racially admixed living in a lower‐middle‐income country.

The Brazilian Longitudinal Study of Adult Health ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ that is a cohort of middle‐aged men and women living in urban areas of Brazil is a unique possibility to verify the validity of Framingham Risk Score for Hypertension among people with different racial and cultural characteristics compared to the original cohort studies.

2. METHODS

2.1. Population

The Brazilian Longitudinal Study of Adult Health (ELSA‐Brasil) addresses the incidence of cardiovascular diseases and significant associated risk factors as high blood pressure. The design and the baseline findings can be found elsewhere. ¹¹ , ¹² , ¹³ , ¹⁴ Briefly, 15 105 civil servants aged 35‐74 years from six cities in Brazil (Belo Horizonte, Porto Alegre, Rio de Janeiro, Salvador, Sao Paulo, and Vitoria) were enrolled between August 2008 and December 2010 for baseline examination. The second visit (2012‐2014) had 14 014 participants. All six participating centers approved the ELSA‐Brasil protocol, and all participants granted signed informed consent.

2.2. Blood pressure assessment and hypertension definition

Blood pressure was measured at the right arm after 5 minutes of rest, with the participant sitting in a quiet, temperature‐controlled room (20‐24°C) using a validated device (Omron HEM 705). Three measurements were taken at intervals of 1 minute each. The mean of the second and third measurements was used as casual blood pressure. Quality control of measurements was provided by centralized training and data collection under strict supervision in all centers. Test‐retest for blood pressure measurements were performed under similar conditions rapidly after the original measurements. ¹⁴ The correlation between intra‐class coefficients was 0.88 (95% CI 0.82; 0.91) for arterial systolic pressure. Data from 230 participants revealed an agreement in the diagnosis between casually measuring blood pressure and 24‐hour ambulatory blood pressure. ¹⁵

In each visit, the criteria for hypertension diagnosis were the same. Hypertension was defined based on three criteria: systolic blood pressure ≥140 mm Hg or diastolic blood pressure ≥90 mm  Hg or under antihypertensive medicines. To be considered an antihypertensive medication, the participant should report at least one medication of the antihypertensive categories listed and answer yes to the question, “Is there any pressure medication you have taken in the last 2 weeks?”. Those who answered yes to the question, “Has any doctor ever told you that you have hypertension?” ¹⁵ , ¹⁶ , ¹⁷

All participants were asked about the use of continuous medications for up to 2 weeks before the study as part of the questionnaire and were asked to take their medication prescription to the research center on the day of the visit. Anthropometric measurements were assessed with the participant standing, dressed in a standard lightweight uniform for the study, without shoes, and fasting for 8‐12 hours. The weight was adjusted to the nearest 0.1 kg with a calibrated scale, and the height measured with a vertical stadiometer adjusted to the nearest 0.1 cm. Waist circumference was measured with a measuring tape and adjusted to the nearest 0.1 cm from the center between the lower ribs. Body mass index (BMI) was calculated with weight in kilograms (kg) divided by the height in m². ¹⁴ Physical activity variables were defined using the leisure activity section of the International Physical Activity Questionnaire (IPAQ), long form. ¹⁸ Alcohol consumption was calculated after the determination of the amount of beer, wine, and spirits transformed in grams of alcohol per week. Excessive alcohol intake was defined as ≥210 g alcohol per week for men or ≥140 g alcohol per week for women. ¹³

2.3. Study design

To analyze the incidence of hypertension in the second wave, we excluded participants with hypertension at baseline, who reported cardiovascular diseases (myocardial infarction, stroke, and heart failure), and participants with diabetes or a serum creatinine >2 mg/dL. To compare with the results of the Framingham Hypertension Score, we restricted our analysis to people younger than 70 years old.

We followed the Transparent Reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD statement) to conduct the research. ¹⁹ First of all, after the exclusion criteria, we analyzed the characteristics of the remaining population (8027 participants) and variables related to the incidence of hypertension (Figure 1). The bivariate comparison of demographic and clinical data between groups (with and without hypertension at the second wave) was performed with the chi‐square test for categorical variables and the unpaired t test for continuous variables (Table 1). We calculated the Framingham Risk Score for Hypertension at the second visit, using the beta coefficients derived from the Framingham Heart Study as described previously. ⁷

FIGURE 1.

Description of the sampling for the determination of 4‐y incident hypertension among participants of the Brazilian Longitudinal Study of Health (ELSA‐Brasil)

TABLE 1.

General characteristics of participants of the Brazilian Longitudinal Study of Adult Health (ELSA‐Brasil) according to the diagnosis of hypertension at baseline

Variables	Hypertension at baseline		P‐value
	No (n = 9689)	Yes (n = 5402)
Age (y) a	49. 9 ± 8.5	56.0 ± 8.8	<.001
Sex (%)
Women	57.5	48.9	<.001
Race (%)
White	55.9	45.5	<.001
Brown	27.6	29.3
Black	12.9	21.8
Asian	2.6	2.4
Indigenous	1.0	1.1
Educational level (%)
Elementary	9.4	18.7	<.001
High school	33.4	37.0
College	57.3	44.3
Leisure‐time physical activity (%)
Low	42.1	44.6	<.001
Moderate	24.4	25.1
Vigorous	33.5	30.3
Smoking (%)
Never	59.2	52.9	<.001
Past	27.2	35.0
Current	13.7	12.1
Parental history of hypertension (%)
None	40.7	34.8	<.001
One parent	46.9	45.5
Both	12.3	19.7
Excessive alcohol intake (%)	12.8	20.3	<.001
Weight (kg) a	71.6 ± 13.9	78.5 ± 16.1	<.001
BMI (kg/m2)
<25	44.6	23.1	<.001
25‐29.9	39.0	42.4
≥30	16.4	34.5
Waist circumference (cm) a	88.2 ± 11.7	96.6 ± 12.8	<.001
Neck circumference (cm) a	35.9 ± 3.6	37.8 ± 3.9	<.001
Systolic blood pressure (mm Hg) a	114.2 ± 11.4	133.9 ± 19.0	<.001
Diastolic blood pressure (mm Hg) a	72.5 ± 8.1	82.9 ± 96.6	<.001

^aMean ± standard deviation.

The cutoff points for predictive tests of the Framingham Risk Score for Hypertension results (discrimination analysis) were obtained by the receiver operating characteristic curve (ROC) and the area under the curve ROC. The predicted: observed risk ratio calculations and calibration were indicated by the Hosmer‐Lemeshow goodness‐of‐fit test. ²⁰ , ²¹ , ²²

After, we randomly split the sample into two groups, 60% for a “derivation” data set (n = 4825) and 40% for a “validation” data set (n = 3202). We created a “new” Framingham Risk Score for Hypertension using the same variables, from derivation data set using the multiple logistic regression model. A stepwise selection algorithm (backward) was then applied with different significance levels to entry (0.05) and stay (0.10). A new beta coefficients of this model (adjusted model from the Framingham Risk Score for Hypertension) were applied in the validation data set to assess discrimination and calibration (Table 3). Another comparison model, using variables that could be easily assessed during the clinical practice using ELSA‐Brasil, was created from the derivation data set and was compared vs the adjusted Framingham Risk Score for Hypertension model in the validation data set. After this, a net classification improvement was calculated comparing the Framingham Risk Score for Hypertension and new predictive model using the ELSA‐Brasil variables that were significantly associated with the 4‐year incidence of hypertension.

TABLE 3.

Multiple logistic regression model for 4‐y incident hypertension using exposures variables at the baseline of the Brazilian Longitudinal Study of Adult Health (ELSA‐Brasil)

Variable	N	Coefficient	Odds ratio	95% confidence interval
Age (y)	4672	0.17	1.18	1.03‐1.35
Sex
Female	2803	Ref
Male	1869	0.38	1.46	1.05‐2.03
Educational level
Elementary	373	0.25	1.29	0.94‐1.80
High school	2758	0.22	1.25	1.01‐1.53
College	1541	Ref
Parental history of hypertension
None	1819	Ref
One	2268	0.33	1.38	1.12‐1.71
Both	585	0.44	1.56	1.14‐2.12
Leisure‐time physical activity
None/moderate	660	Ref
Vigorous	494	−0.53	0.59	0.41‐0.86
Body mass index (kg/m2)
<25	2152	Ref
25‐29.9	1814	0.31	1.36	1.07‐1.72
≥30	706	0.48	1.62	1.16‐2.25
Neck circumference (cm)	4672	0.04	1.04	1.00‐1.10
Smoking
Never	2888	Ref
Former/current	1784	0.21	1.24	1.02‐1.50
Systolic blood pressure (mm Hg)	4672	0.07	1.07	1.06‐1.09
Diastolic blood pressure (mm Hg)	4672	0.16	1.17	1.07‐1.28

The significance level was fixed at 5% for all tests. Statistical analyses were performed using IBM SPSS Statistics version 24.0 (IBM Corporation) and R software version 3.5 (R Foundation for Statistical Computing).

3. RESULTS

From the 15 105 participants enrolled at baseline (2008‐2010), 35.7% had a diagnosis of hypertension. Table 1 shows the characteristics of the baseline participants according to the diagnosis of hypertension. Persons with hypertension were more likely men and older compared with people without hypertension. They self‐reported to be brown or black and had a lesser formal education. Besides, they were more obese (general and regional) and practiced fewer hours of physical activity. Moreover, they were more frequently former smokers, and they had a higher frequency of excessive intake of alcoholic beverages. Finally, they have a higher proportion of parents with a medical diagnosis of high blood pressure.

After a 4‐year follow‐up, 8027 individuals were eligible for our analyses. From these participants, 1088 (13.5%) developed new‐onset hypertension at the second visit (2012‐2014) (Figure 1).

Table 2 discloses the beta coefficients from the Framingham Risk Score for Hypertension for the ELSA‐Brasil derivation data set. A significant association was found for age, parental history of hypertension, body mass index, smoking habit, and both systolic and diastolic blood pressure. We confirmed that the variables used in the Framingham Risk Score for Hypertension are predicting the 4‐year incidence of high blood pressure in the ELSA‐Brasil in the same age‐strata range. However, the measure of association is different from beta coefficients for sex, body mass index, and smoking, and relatively similar coefficients for age, parental history of hypertension, and the previous values of systolic and diastolic blood pressure.

TABLE 2.

Multiple logistic regression model following the Framingham Risk Score for 4‐y incident hypertension applied among the participants of the Brazilian Longitudinal Study of Adult Health (ELSA‐Brasil)

Variable	N	Coefficient	Odds ratio	95% confidence interval
Age (y)	4823	0.16	1.18	1.03‐1.34
Sex
Male	1928	Ref
Female	2895	0.13	1.14	0.93‐1.39
Parental history of hypertension
None	1867	Ref
One	2350	0.32	1.38	1.12‐1.70
Both	606	0.48	1.61	1.20‐2.18
Smoking
Never	4192	Ref
Former/current	631	0.18	1.20	0.92‐1.58
Body mass index (kg/m2)
<25	2220	Ref
25‐29.9	1878	0.44	1.56	1.26‐1.93
≥30	725	0.70	2.02	1.56‐2.61
Systolic blood pressure (mm Hg)	4823	0.07	1.07	1.06‐1.09
Diastolic blood pressure (mm Hg)	4823	0.16	1.17	1.07‐1.28
Intercept		−24.55	<0.0001

In Table 3, applying logistic regression model for all variables associated with incident hypertension, we were able first to confirm a positive association for age, male sex, parental history of hypertension, to be overweighed or obese, smoking habit, and previous values of systolic and diastolic pressure; and second to add three independent variables beyond the Framingham Risk Score for Hypertension. People with less formal education, with higher neck circumference, and who did not have a vigorous leisure‐time physical activity had a significant odd to be with high blood pressure after 4 years of follow‐up. Self‐reported race, abdominal adiposity, alcohol beverage, and intake of salty foods did not change the association with new‐onset hypertension materially.

Figure 2 displays the receiver operating characteristic curves applied for the ELSA‐Brasil validation data set applying the Framingham Risk Score (Figure 2A) and our predictive equation (Figure 2B). The area under the curve (and the 95% confidence intervals) was relatively similar. Hosmer‐Lemeshow chi‐squared statistic applied for the Framingham Risk Score was 3.78; P‐value = .876 and for our model was 8.22; P‐value = .41, disclosing good discrimination and calibration for both models, Framingham Risk Score for Hypertension and ELSA‐Brasil.

FIGURE 2.

Receiver operating characteristic curves applied for the ELSA‐Brasil validation data set applying the Framingham Risk Score (A) and the ELSA‐Brasil predictive equation (B)

In terms of reclassification of the risk, Table 4 discloses the differences between the models to predict the new cases of hypertension, according to the best intervals of risk classification obtained. For the Framingham Risk Score, the better prediction was seen with classifying as low risk (<5%), moderate risk (≥5% and <20%), and high risk (≥20%), different from the original study. For achieving the best Net Classification Index (−0.18%) comparing with the Framingham model, ELSA‐Brasil model best classification intervals were considered low risk (<10%), moderate risk (≥10% and <20%), and high risk (≥20%). Even with these classification intervals, our model presents more underestimation of the risk, classifying 15% of the participants with new onset of hypertension in low risk vs 9% of the Framingham model and less overestimation of the risk, classifying 17% of the participants without hypertension as high risk vs 24% of the Framingham model.

TABLE 4.

Reclassification of the predicted risk of 4‐y incident hypertension based on the Framingham Heart Study vs ELSA‐Brasil Hypertension Score in the validation cohort (3140 observations)

Status at follow‐up examination	Risk (FRSNTH) (%)	ELSA_BRASIL Hypertension Score			Reclassified		Net correctly reclassified (%)
		<10% (Low)	10%‐20% (Moderate)	>20% (High)	Increased risk	Decreased risk
Hypertension (n = 434)	<5	41	1	0	2	78	−17.51
	5‐20	26	35	1
	>20	0	52	278
Non‐hypertension (n = 2706)	<5	1466	13	0	18	487	17.33
	5‐20	275	275	5
	>20	10	202	460
Net reclassification improvement (NRI)							−0.18

4. DISCUSSION

We were able to identify that the Framingham Risk Score for Hypertension had fair discrimination to predict 4‐year new‐onset hypertension among the participants of ELSA‐Brasil. Moreover, our predictive equation added new variables as formal education, neck circumference, and leisure‐time physical activity as determinants of incident hypertension. The magnitude of sex was higher in our model, but the importance of the body mass index was higher in the Framingham model. Risks Scores for Hypertension have been performed in another sample, with results of the receiver operating characteristic (ROC) curve (AUC) varying from 0.71 to 0.80 (a popular metric). ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ However, to evaluate new markers or risk factors for predictive models, we know that they may have more significant independent associations with the outcome to result in a meaningfully bigger receiver operating characteristic (ROC) curve (AUC). Many authors suggest new approaches beyond the receiver operating characteristic (ROC) curve (AUC) as the net reclassification improvement or NRI and the integrated discrimination improvement (IDI). ²²

In our study, we calculated the net reclassification indexes that both models were practically similar. The Framingham, Risk Score model, predicted 90.3% of the people with hypertension vs 84.5% in the ELSA‐Brasil model. On the other hand, the Framingham model classified as a low risk of 54.6% of participants without incident hypertension vs 64.7% in low risk applying the ELSA‐Brasil model.

Framingham Risk Score super estimates the risk for hypertension (24%), and our model underestimates in general (15%). As described previously, prediction models have limitations. ²¹ , ²² The quality of pieces of information of the dependent and independent variables can differ among cohorts. During the application of the regression analyses, the Nagelkerke's R‐squared did not reach values higher than 0.30 so, even with these results, the contribution response of the presented variables as risk factors only explain that the dependent variable, that is, the new onset of hypertension is no more than 30%. This fact could be added to these limitations. Another aspect is the possibility of overestimation or underestimation of the risks that could imply errors of interpretation, treatment, and prevention. In terms of diagnostics risk factors as hypertension, the importance of the risk scores depends on the impact on the risk factor in specific populations to permit preventive actions, early treatment, reducing the costs, and improving the quality of life after primary care providers identify the patients with a high risk for the development of the determinant disease.

Although the Framingham Risk Score for Hypertension model is very well established, the ELSA‐BRASIL risk score model added new variables that are suitable for all primary care systems, and it also has variables related to our population in a contemporary sample. We consider that the three new conditions predicting hypertension from ELSA‐Brasil as formal education, leisure‐time physical activity, and neck circumference are feasible to be determined at the primary care setting. The level of formal education is a piece of information collected in the enrollment of any family health program. Physical activity is asked in any contact with community health workers, nurses, and physicians. The measurement of the neck circumference is a not complicated procedure using the same metric tape for the determination of waist circumference performed in a regular office visit. As in Brazil, hypertension is the leading risk factor for mortality and disability, and this study was an opportunity to understand the effect of some risk factors and the possibility to create a comparison score trying to use simple variables that could be reached in a single medical visit.

The limitations of our study are the same as the others addressing prediction models for hypertension, as the inherent variability of blood pressure; however, we consider this as nondifferential misclassification. The blood pressure measurements were clinic‐based, whereas recent evidence suggests that the 24‐hour blood pressure recording should be a better assessment of blood pressure. On the other hand, the loss of participants between the visits was not materially significant, and the data related to the use of antihypertensives for both visits were very accurate.

Further analysis, adding data from the third visit (2017‐2019), will add more information to develop an equation to predict hypertension for the Brazilian population.

5. CONCLUSIONS

We conclude that the Framingham Risk Score Near‐Term Incidence of Hypertension has an excellent performance in the ELSA‐Brasil sample, with good discrimination and calibration. This tool can be useful in Brazilian examples as the comparison model, ELSA‐Brasil Hypertension Score to predict new of the disease.

CONFLICT OF INTEREST

We have no conflict of interest to declare.

AUTHOR CONTRIBUTION

DHS and PAL contributed to the conceptualization of the manuscript and conducted the data management and analyses, and wrote the first draft. VC contributed to the statistical analysis. IMB is the PI of the study and contributed to interpretation and writing. All authors reviewed and approved the manuscript.

Syllos DH, Calsavara VF, Bensenor IM, Lotufo PA. Validating the Framingham Hypertension Risk Score: A 4‐year follow‐up from the Brazilian Longitudinal Study of the Adult Health (ELSA‐Brasil). J Clin Hypertens. 2020;22:850–856. 10.1111/jch.13855