A Prediction Model for the Peripheral Arterial Disease Using NHANES Data

Abstract

We aim to build models for peripheral arterial disease (PAD) risk prediction and seek to validate these models in 2 different surveys in the US general population.

Model building survey was based on the National Health and Nutrition Examination Surveys (NHANES, 1999–2002). Potential predicting variables included race, gender, age, smoking status, total cholesterol (TC), body mass index, high-density lipoprotein (HDL), ratio of TC to HDL, diabetes status, HbA1c, hypertension status, and pulse pressure. The PAD was diagnosed as ankle brachial index <0.9. We used multiple logistic regression method for the prediction model construction. The final predictive variables were chosen based on the likelihood ratio test. Model internal validation was done by the bootstrap method. The NHANES 2003–2004 survey was used for model external validation.

Age, race, sex, pulse pressure, the ratio of TC to HDL, and smoking status were selected in the final prediction model. The odds ratio (OR) and 95% confidence interval (CI) for age with 10 years increase was 2.00 (1.72, 2.33), whereas that of pulse pressure for 10 mm Hg increase was 1.19 (1.10, 1.28). The OR of PAD was 1.11 (95% CI: 1.02, 1.21) for 1 unit increase in the TC to HDL ratio and was 1.61 (95% CI: 1.40, 1.85) for people who were currently smoking compared with those who were not. The respective area under receiver operating characteristics (AUC) of the final model from the training survey and validation survey were 0.82 (0.82, 0.83) and 0.76 (0.72, 0.79) indicating good model calibrations.

Our model, to some extent, has a moderate usefulness for PAD risk prediction in the general US population.

INTRODUCTION

Peripheral arterial disease (PAD), an obstructive atherosclerotic disease in the lower extremities, was repeatedly found to be associated with cardiovascular diseases,¹ stroke,² mortality,³ or decreased quality of life⁴ in different populations. The prevalence of PAD has been of increasing concern around the world for the last decades. It was reported that a substantial proportion of people in the general population had PAD.^5–7 According to a recent study, however, only around one-tenth of patients with PAD had clinical intermittent claudication symptoms and as high as 75% of these patients could be without symptomatic PAD.⁸ The National Institute for Health and Clinical Excellence (NICE) recommended that PAD patients should be provided with a detailed clinical examination that included assessing the most modifiable risk factors, such as smoking status, glycemic status, abnormal lipid, obesity, and physical activity.⁹ In practice, however, epidemiological studies about PAD are much fewer compared with those of other type of cardiovascular diseases. Thus, there is an urgent need for PAD surveillance, to investigate the etiology, and to develop possible preventive and treatment strategies.

To assess the prevalence of PAD, thorough examinations using ankle brachial index (ABI) or combing clinical symptoms are the typical methods for most of the present clinical research and epidemiological studies. However, because these methods are sometimes expensive, time-consuming, and resource and labor demanding, many epidemiological surveys do not have these equipment or measurements, which might be a hurdle to further examine the risk factors of PAD or its predictive performance for other clinical outcomes. Consequently, alternative cost-effective, reliable, and valid instruments for PAD prevalence surveillance at the national or regional level, such as scales or other routine examinations in epidemiological studies are on the agenda of the scientific community.

In the present study, we aimed to (1) develop models for PAD risk prediction using a combination of self-reported questions and routine metabolic biomarkers from National Health and Nutrition Examination Surveys (NHANES, 1999–2002); (2) to externally validate these models in NHANES (2003–2004).

METHODS

Study Population

As a nationally representative survey of the US population, NHANES was conducted by the National Center for Health Statistics of the Centers for Disease Control and Prevention. NHANES has been collecting data from personal interviews and physical examinations since 1999. The model training survey is based on 4 years of the continuous NHANES (1999–2002), and the model validation survey is based on NHANES (2003–2004).

All study participants in this study were examined at homes where detailed information on demographic characteristics, smoking and drinking status as well as other risk factors were collected. A physical examination at a mobile examination center was also offered to those who were able to participate. The ABI assessment and the lower extremity disease examination were limited to those who were 40 years and over. Participants who had a bilateral amputation or were obese (>400 pounds) were excluded from the examination. According to a predetermined operation protocol, trained health staff performed the ABI examination in a separate room at the mobile center. Written informed consent was obtained from all study participants. Ethical approval was waived because publicly available data was used. Details of study design were previously described elsewhere.^6,10–12

PAD Ascertainment

The ABI value was calculated for each participant. With the participants in the supine position, trained health staff used an 8.1-MHz Doppler probe to perform the examination following a standard operation protocol. The ABI was calculated by dividing the ankle mean systolic blood pressure by brachial mean systolic blood pressure in the same side. The presence of PAD was defined as an ABI < 0.9 in either side.

Covariates

Pulse pressure was calculated as the systolic blood pressure minus the diastolic blood pressure. Hypertension was defined as SBP ≥ 140 mm Hg, DBP ≥ 90 mm Hg, or current medication for hypertension and diabetes mellitus was defined as HbA1c ≥ 6.5% or current medication for diabetes. The TC/HDL ratio was calculated by TC divided by HDL. Smoking status was coded as if participants smoked at least 100 cigarettes in life.

Statistical Analysis

All estimates were weighted, with the sample weights accounting for the unequal selection probability of the complex NHANES sampling and the oversampling of selected population subgroups. Basic characteristics of the 2 study samples were shown as mean (± standard deviation) and numbers (proportions). Multivariable logistic regression models were applied to develop the prediction models in the model training survey. We first searched the potential variables used in current risk prediction models from the published literature. Then we applied likelihood ratio tests to determine which variables being included in the models. For this model selection process, we first included all the potential variables in the models, and then sequentially removed those which were not significant according to the likelihood ratio test in the reduced models compared with the full model. The final model was listed below: 

Area under receiver operating characteristics curve (C-statistics), and goodness-of-fit test in a multiple logistic regression¹³ were used as the metrics to assess the model's performance including discrimination and calibration, respectively. This method has been used in other prediction models previously.¹⁴ We also performed bootstrap analysis to evaluate the internal model performance. Replication on 200 different samples drawn with replacement was performed for the bootstrap method. C-statistics from the original study minus optimism was used to calculate the optimisms-corrected C-statistics.¹⁵ The optimal cut-off value for predicted probabilities of PAD, distinguishing PAD cases from healthy participants, was chosen by Youden's index. We also validated the final model in another NHANES survey. All statistical analyses were performed using Stata/MP 13.0 and P < 0.05 was regarded as statistical significance.

RESULTS

Basic characteristics of study participants are shown in Table 1. In model training survey, the average age was 59.6 years with men comprising 51.1% of the study population, whereas in the model validation survey, the average age was 61.0 years with men comprising 51.7%. The prevalence of PAD was 4.7% in the model training survey and 5.6% in model validation survey.

TABLE 1.

Basic Characteristics of Study Participants in Each Cohort

The final model included age, sex, race, pulse pressure, TC/HDL ratio, and smoking status as predicting variables (Table 2). Figure 1 presented the ROC curve for the training and validation surveys. The C-statistics and 95% confidence interval (CI) were 0.82(0.82, 0.83) in the training survey (Table 3). Optimism from Bootstrap internal validation was 0.0015 and optimism-corrected C-statisitcs was comparable to the previous uncorrected one. Hosmer–Lemeshow goodness of fit test revealed P value of 0.56 indicating very good model calibration in the training survey. In terms of model external validation, the C-statistics (95% CI) was 0.76 (0.72, 0.79) as shown in Table 3.

TABLE 2.

Model Coefficients Based on Training Cohort

FIGURE 1.

Model performance for peripheral arterial disease prediction in NHANES. The black line indicates model performance in model training survey, and the red line represents model performance in model validation survey. NHANES = National Health and Nutrition Examination Surveys.

TABLE 3.

Model Performance for Peripheral Arterial Disease

DISCUSSION

In the present study, we developed a risk prediction model for PAD in a training survey and externally validated it in another survey. We found that the models based on self-reported questionnaires and routine clinical metabolic biomarkers had a good discrimination capacity with C-statistics of 0.82 and 0.76 in the training and validation survey, respectively. These results were suggestive that the model performance was moderately good and it might be useful in PAD surveillance and epidemiological studies.

Until now, several groups have proposed PAD prevalence estimation algorithms, of which the predictors in the models varied among different studies.^16–20 For example, the Netherland PREVALENT score¹⁶ model included age, smoking behavior, hypertension, coronary heart disease, and cerebrovascular disease (CVD) in the final models using stepwise logistic regression models. Likewise, the Spain REASON risk score included age, sex, smoking status, pulse pressure, and diabetes in the final model, which yielded AUC of 0.76 in both the training and validation samples.¹⁷ Later, the same group compared the performance of 2 prediction scores and found REASON had better performance in Spain populations.¹⁸ Another group developed a PAD score in a US survey²¹ and selected age, sex, race/ethnicity, smoking status, BMI, hypertension, heart failure, CVD, coronary artery disease, and diabetes in the final models with a moderate discrimination performance (C-statistics:0.61–0.64 in the training and validation samples).

An approach for predicting the risk of PAD in communities is multivariable modeling using self-reported questionnaires combining some routine examination measurements. In our study, we developed a model in the training survey considering several potential variables including age, sex, race, BMI, pulse pressure, fasting glucose, TC, HDL, TC/HDL ratio, smoking, diabetes, and hypertension. Most of the variables were regarded as risk factors of PAD. Consistent with previous research results, age, sex, smoking, and pulse pressure were selected in the final model. Nevertheless, we considered more possible variables to be selected for model development.

In addition to common lipid biomarkers (TC and HDL), we considered TC/HDL ratio as one of the candidate predicting variables because several studies had found a strong association between TC/HDL ratio and PAD independent of other conventional risk factors.^22,23 In parallel with previous findings, TC/HDL showed a strong association with PAD and was superior to other lipid biomarkers,²² and it was selected in the final models via likelihood ratio test. Some more variables were also taken into account of model training in the present study. However, they did not show significant improvement for model performance and were not included in the final model. Previous studies have found there were different PAD risk among different sex and race groups,^24,25 and smoking had a greater impact on developing PAD.²⁶ Consistent with these findings, our final models also included these parameters. In addition to that, several studies had also found pulse pressure was predictive of PAD independent of traditional risk factors,^7,27 we considered it as a potential predictive parameter that showed similar effect size in the final models compared with previous findings.

Our results demonstrated that the prevalence prediction model we developed might be a promising PAD surveillance instrument for the community-based population. In the first place, internal validation analysis was conducted to avoid over-fitting of the prediction models. Bootstrap methods were used to estimate the over-optimism, and the optimism-corrected C-statistics were still good. Next, external validation analysis was performed in a different population. Because the source populations of the training survey and validation survey might be similar, the model performed well in the validation sample. The results of the external validation analyses ascertained the performance of our prediction model.

We do acknowledge that our study has some limitations. The definition of PAD was solely based on ABI. Further information regarding intermittent claudication would definitely reduce information bias. Secondly, although the model we developed was validated in an external survey, the survey was based on similar participants in USA. Thus, additional community-based surveys used for validation might provide robust evidence for the model performance.

In conclusion, we developed a PAD risk prediction model based on self-reported questions, demographic characteristics, and routine metabolic biomarkers that moderately predicted the population PAD risk. This model was validated in an external sample in a different time period and showed a moderate discriminatory power. Our findings may be helpful to assess PAD surveillance and to track susceptible populations. In the future, the proposed model should be validated in other community-based surveys to evaluate their external performance.