Abstract
BACKGROUND
The aim of this study is to present an objective method based on support vector machines (SVMs) and gravitational search algorithm (GSA) which is initially utilized for recognition the pattern among risk factors and hypertension (HTN) to stratify and analysis HTN’s risk factors in an Iranian urban population.
METHODS
This community-based and cross-sectional research has been designed based on the probabilistic sample of residents of Isfahan, Iran, aged 19 years or over from 2001 to 2007. One of the household members was randomly selected from different age groups. Selected individuals were invited to a predefined health center to be educated on how to collect 24-hour urine sample as well as learning about topographic parameters and blood pressure measurement. The data from both the estimated and measured blood pressure [for both systolic blood pressure (SBP) and diastolic blood pressure (DBP)] demonstrated that optimized SVMs have a highest estimation potential.
RESULTS
This result was particularly more evident when SVMs performance is evaluated with regression and generalized linear modeling (GLM) as common methods. Blood pressure risk factors impact analysis shows that age has the highest impact level on SBP while it falls second on the impact level ranking on DBP. The results also showed that body mass index (BMI) falls first on the impact level ranking on DBP while have a lower impact on SBP.
CONCLUSION
Our analysis suggests that salt intake could efficiently influence both DBP and SBP with greater impact level on SBP. Therefore, controlling salt intake may lead to not only control of HTN but also its prevention.
Keywords: Support Vector Machines, Gravitational Search Algorithm, High Blood Pressure
Introduction
Hypertension (HTN) is one the most prevalent risk factors of cardiovascular disease (CVD) in Iran.1 Worldwide, approximately 12.8% of total deaths are estimated to be due to HTN.2 It is well-documented that some risk factors such as age, sex, obesity, and dietary habits strongly affect the occurrence of HTN.3 HTN is demonstrated to be a multifactorial disease. Therefore, numerous lifestyle changes could decrease and control high blood pressure by modifying the risk factors.4 Many studies have shown different factors such as aging, genetics, obesity, non-healthy diet, and bad lifestyle all correlates with high blood pressure.5 Therefore, both genetics and environmental factors could affect the increase in blood pressure.6 The impact degree of genetic factors on high blood pressure reported to be between 0.5 and 0.6, while the environmental parameters include bad lifestyle, unhealthy diet, obesity, and aging impact reported approximated between 0.4 and 0.5.7 Genetic factors are complicated and impossible for us to modify. Therefore, we should focus on modifiable risk factors to select a good strategy in HTN control.8 Lifestyle is one of the modifiable factors that if changed to a healthy lifestyle could prevent or reduce the incidence of high blood pressure.9
Novel soft computing methods are utilized in realizing tractability, robustness, and to present a solution with an acceptance of vagueness, uncertainty, partial truth, and approximation.10 Support vector machine (SVMs) are a novel machine learning technique formerly used for pattern recognition problem based on structural risk minimization.11 Essentially, SVMs have a theoretical relation with artificial neural network (ANN). An SVM model using sigmoid kernel function is equal to a two-layer perceptron neural network. Using a kernel function, the SVMs are different training techniques for polynomial, radial basis function, and multilayer perceptron classifiers in which the weights of the network are set up by answering a quadratic programing problem with linear constraints, rather than by solving a non-convex, unconstrained minimization problem as in standard ANN training.11 Finding optimal values for parameters of SVMs is a significant step in the SVMs analysis which has a great influence on its modeling ability and on its accuracy. Using metaheuristics can be helpful in determining the suitable value of SVM parameters for the best estimation and approximation performance.12 Gravitational search algorithm (GSA) is one of the recognized metaheuristics that can find out a high-quality solution to an optimization problem.13,14 In comparison with some other research techniques, GSA has established higher performance in handling various non-linear functions.
We investigated the potential use of the optimized SVMs for analyzing the impact levels of risk factors on developing high blood pressure. Showing some examples, we evaluated the capabilities of optimized SVMs and compare it with two common statistical methods, generalized linear modeling (GLM) and multiple linear regression (MLR), to model the associations between age, urinary volume, serum creatinine, waist circumference (WC), salt intake, body mass index (BMI), and systolic blood pressure (SBP) and diastolic blood pressure (DBP). The proposed approach is evaluated and used to stratify the impact level of the above risk factors on SBP and DBP in the Iranian urban population.
Materials and Methods
The salt intake study is a cross-sectional study which has been developed based on a probabilistic sample of Isfahan, Iran, residents aged 19 years or more from 2001 to 2007. Sampling strategy, survey, data entry, and analysis have been previously described.13 Study subjects were selected through stratified random sampling based on age and sex. Inclusion and exclusion criteria of study participants explained elsewhere.14 One of the household members was randomly selected to train on the purpose of the research and invited to participate in the study. An oral consent was obtained from each participant. The project received approval from the Research and Ethics Committees of Isfahan University of Medical Sciences.
Primary evaluation and the baseline data collection were carried out by a trained nurse and through phone interviews. A questionnaire was used to collect demographic and anthropometric information. The subjects were invited to an information session to be briefed on urine sample collection and blood pressure measurement. We used 24-hour urinary sodium excretion as an indicator of sodium intake for the accuracy. The participants instructed to collect urine samples for 24-hour (7 AM to 7 AM of the following day). The Sodium (mmol/l) was measured by commercial kits using flame spectrophotome. The blood pressure was measured by a trained nurse using a bench-mounted mercury column sphygmomanometer. The blood pressure was measured with the individual seated and rested for a minimum of 5 minutes after emptying the bladder.
Descriptive statistics of the investigational data include mean, minimum, maximum, standard deviation (SD), and skewness were all calculated by SPSS software (version 18, SPSS Inc., Chicago, IL, USA). Scatterplot matrices, demonstrating associations between the data (i.e., age, urinary volume, serum creatinine, WC, salt intake, BMI, SBP, and DBP), were acquired by VisuLab (ETH University, Zurich, Switzerland). The subsequent equation used to normalize every input data to a range of 0.1-0.9:
Furthermore, the data set was separated into three subsets for training, testing, and verification. The training subset was randomly selected from 60% of the main data set. The remaining data set (40% of the data) were utilized uniformly and separated into two parts as the testing and validation subsets.
Optimized SVM
The SVM implemented the principle of structural risk minimization by constructing an optimal separating hyperplane for more mathematical details; please refer to Vapnik.15 The parameters of SVM have a most important impact to the model’s correctness, learning effectiveness, and generalization capability.12 Since using classical optimization techniques for parameters optimization is time-consuming and needs human knowledge, in this paper, the GSA is utilized to optimize the parameters above for SVM model. It uses the law of gravity and mass interactions and has been applied in different areas.16,17
Results
Table 1 demonstrates the small variation of observed data (variables utilized in the study). It also shows a normal distribution of the data. The scatterplot matrices show correlations between the input variables (i.e., BMI, WC, Salt intake, age, urinary volume, and serum creatinine) and model outputs, SBP, and DBP. The results show although the associations between input variables and the model outputs exist. However, the trends and patterns appear to be comparatively complex. While based on clinical studies mentioned variables influence both SBP and DBP, it is not understood how this process occur and also there is no study about stratifying their impact levels on blood pressure. It seems that the studied variables possibly either directly or indirectly influence SBP and DBP in the Iranian urban population. Therefore, the proposed approach is useful when looking for a correlation of such data to estimations SBP and DBP. Table 2 shows some fitting values of different models. To calculate the performance of recommended techniques, the measured and estimated values that were utilized are as follows; mean square error (MSE), model efficiency factor (MEF), correlation coefficient (r), and error percentage (ERROR %). The MSE, MEF, and ERROR % statistics are described as follow:
Table 1.
Parameter | Descriptive statistics |
|||
---|---|---|---|---|
Mean ± SD | Minimum | Maximum | Skewness | |
SBP (mmHg) | 104.00 ± 11.88 | 80.00 | 145.00 | 0.819 |
DBP (mmHg) | 69.18 ± 8.81 | 50.00 | 97.50 | 0.346 |
BMI (m2kg) | 25.47 ± 4.39 | 15.76 | 45.64 | 0.512 |
WC (cm) | 82.59 ± 12.07 | 54.50 | 172.00 | 0.698 |
Salt intake | 139.07 ± 3.18 | 132.00 | 147.00 | 0.079 |
Age (year) | 37.29 ± 12.59 | 19.00 | 81.00 | 0.725 |
Urinary volume (ml) | 1094.09 ± 437.63 | 200.00 | 2875.00 | 1.006 |
Serum creatinine (mg/dl) | 0.98 ± 0.21 | 0.60 | 2.40 | 1.490 |
SBP: Systolic blood pressure; DBP: Diastolic blood pressure; BMI: Body mass index; WC: Waist circumference; SD: Standard deviation
Table 2.
Blood pressure type | Model type | Evaluation criterion |
|||
---|---|---|---|---|---|
MEF | MSE | r | ERROR (%) | ||
SBP | MLR | 0.298080 | 0.007748 | 0.559 | 14.1445 |
SBP | SVM-GSA | 0.962003 | 0.000419 | 0.992 | 3.1356 |
SBP | GLM | 0.484303 | 0.005692 | 0.748 | 12.1238 |
DBP | MLR | 0.200616 | 0.007370 | 0.452 | 13.8368 |
DBP | SVM-GSA | 0.931873 | 0.000628 | 0.989 | 4.0151 |
DBP | GLM | 0.372892 | 0.005782 | 0.658 | 12.2555 |
MLR: Multiple linear regression model; SVM: Support vector machine model; GSA: Gravitational search algorithm; MEF: Model efficiency factor; MSE: Mean square error; GLM: Generalized linear modeling; SBP: Systolic blood pressure; DBP: Diastolic blood pressure
Where yk denotes the estimated value, yk is the measured value, yk is the mean of measured values, and n is the total number of observations.
The MLR model had lowest estimation effectiveness compared to the optimized SVM. Using MLR with input data resulted in the lower correlation coefficient between the measured and estimated SBP and DBP values (Figure 1). The MSE value for the developed MLR model for SBP was 0.00774 (Table 2), while the MEF and ERROR % values were 0.298 and 14.14, respectively. The MSE, MEF, and ERROR % values for the developed MLR model using DBP data set were 0.0073, 0.2006, and 13.83, respectively (Table 2). In accordance with the evaluation results, it seems that regression model was to some level poor in estimating SBP and DBP in this study.
The MSE, MEF, and ERROR % values for the developed GLM model using SBP data set were 0.0056, 0.4843, and 12.123, respectively (Table 2). Furthermore, the coefficient of correlation between the measured and estimated DBP values for the constructed GLM model was 0.65802. MSE, MEF, and ERROR % value for DBP were 0.0057, 0.3728, and 12.25, respectively (Table 2). These consequences propose a better performance of GLM method for predicting DBP and SBP in comparison to MLR technique. Nevertheless, it seems that this method also fails to be reliable for estimating SBP and DBP.
The acquired evaluation criteria and correlation coefficient values among the estimated and measured values for both SBP and DBP demonstrated that optimized SVM has the highest estimation potential (Figure 2), particularly, when its performance is evaluated with regression and GLM methods. The MSE, MEF, and ERROR % values for the developed optimized SVMs model using SBP as a target were 0.0004194, 0.962003012, and 3.135, respectively while for the DBP were 0.0006281625, 0.931873605, and 4.0151, respectively (Table 2).
Evaluating the acquired consequences from MLR, GLM, and optimized SVMs models showed that optimized SVMs method yields better results compared with GLM and MLR methods. This may be caused by the ability of optimized SVMs in recognizing complex correlations because of their distributed and parallel computing nature. The cause of the estimators on the SBP and DBP may not be linear in nature, and this might be a reason the linear models may not be reliable for estimation of DBP and SBP in the Iranian urban population. The optimized SVMs model could result in more acceptable estimation of blood pressure parameters due to the flexibility and ability to model nonlinear correlations. Furthermore, the better performances of the developed soft computing models were probably because of their higher degree of robustness tolerance than the classical statistical models. Therefore, it seems that in the case of estimating blood pressure and risk factors stratification which MLR and GLM fail to be reliable, optimized SVMs might be chosen as a preferred option. After choosing a superior model based on performance evaluation procedure, we used sensitivity analysis and reported its output as an impact level (Figure 3).
Discussion
In this study, the novel method for exploratory risk factor stratification was performed to determine the impact level of each risk factor on DBP and SBP. Based on our findings, the first important risk factor is age. It has the highest impact level on SBP while falls second on the impact level ranking on DBP (Figure 3). This is in agreement with intersalt study result that showed an increase in SBP with age.18,19 Another study by Janus et al. showed an increase in SBP with age in 7730 individuals randomly chosen in Hong Kong.20
The results also showed that basal metabolic rate (BMR) falls first on the impact level ranking on DBP while have a lower impact on SBP, While the evaluation of WC indicated that it falls on the second and fifth impact level ranking on SBP and DBP, respectively. In previous studies, BMI and waist-to-hip ratio used to be frequently employed as indicators of overweight and obesity as well as body fat distribution pattern. In some researches from Western countries, BMI has been constantly shown to be associated with high blood pressure. Some cross-sectional reports documented such an association among overweight and hypertensive.21-23
Some hypothesis proposed that overweight is related with elevated cardiac output, increased blood volume, and increased peripheral vascular resistance.24 Weight circumference demonstrated to be a superior indicator of abdominal fat compared to waist-to-hip ratio. The use of WC when evaluating abdominal fat is newly suggested by the National Institutes of Health Guidelines on the Identification, Evaluation, and Treatment of Overweight and Obesity. In our population, WC is recognized as a risk factor with a more impact on SBP compared to BMI itself. As shown in scatterplot matrices, the association between BMI and WC is very significant. The body fat distribution is an important contributor to the association of obesity and high SBP. This result is in agreement with the results obtained from the Olivetti Heart study.25 Our results also demonstrate the association between WC and high blood pressure among Iranian urban population.
This present study is the first community-based research on salt intake impact level stratification in the urban region of Iran. We attempted to achieve a population-view to choose proper management strategy and reducing the high blood pressure. We estimated blood pressure and salt intake in Iranians based on the new formulation presented by Tanaka et al., for determining daily sodium intake from urinary sodium secretion (estimated 24-hour sodium intake (mEq/day) = 21.98 × Na0.392).24,25 Derived from intersalt results26 association between blood pressure and salt intake was weak in some countries. Even within a single country, the results from studies on the correlation between salt intake and blood pressure could vary.27 Our analysis suggests that salt intake could efficiently influence both DBP and SBP with greater impact level on SBP. Therefore, controlling salt intake may lead to not only control of HTN but also its prevention. Our results also in agreement with the Dietary Approaches to Stop Hypertension (DASH) study.28 Which demonstrated that SBP considerably decreased both in the control-diet group and DASH-diet group when the sodium intake level decreased (from high to intermediate and then to low level).29
Based on our results, serum creatinine has a higher degree of impact level on DBP than salt while it has fifth impact level on SBP. This study also considered the potential relation between blood pressure and renal function which is demonstrated by serum creatinine in the Iranian urban population. Creatinine clearance has been frequently employed as a factor to show renal function in health. Since this process needs 24-hour urine gathering, which is not easy to achieve from individuals involved in large population-based researches, increasing in serum creatinine level has commonly been utilized as a measure of renal morbidity. Our analyzes advise that serum creatinine could powerfully influences both DBP and SBP with superior impact level on SBP which is in agreement with results from the Bogalusa Heart Study on white community.30-33
Acknowledgments
None.
Footnotes
Conflicts of Interest
Authors have no conflict of interests.
REFERENCES
- 1.Khosravi AR, Kiani Mehr G, Kelishadi R, Shirani S, Gharipour M, Tavassoli A, et al. The impact of a 6-year comprehensive community trial on the awareness, treatment and control rates of hypertension in Iran: experiences from the Isfahan healthy heart program. BMC Cardiovascular Disorders. 2010;10:61. doi: 10.1186/1471-2261-10-61. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Cené CW, Cooper LA. Death toll from uncontrolled blood pressure in ethnic populations: universal access and quality improvement may not be enough. Annals of Family Medicine. 2008;6(6):486–9. doi: 10.1370/afm.922. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Shirani S, Gharipour M, Khosravi A, Kelishadi R, Habibi HR, Abdalvand A, et al. Gender differences in the prevalence of hypertension in a representative sample of Iranian population: the Isfahan Healthy Heart Program. Acta Biomed. 2011;82(3):223–9. [PubMed] [Google Scholar]
- 4.Dunn FG, Pringle SD. Hypertension and coronary artery disease. Can the chain be broken? Hypertension. 1991;18(3 Suppl):I126–I132. doi: 10.1161/01.hyp.18.3_suppl.i126. [DOI] [PubMed] [Google Scholar]
- 5.Lathrop GM. Genetic approaches to common diseases. Curr Opin Biotechnol. 1993;4(6):678–83. doi: 10.1016/0958-1669(93)90049-3. [DOI] [PubMed] [Google Scholar]
- 6.Sing CF, Boerwinkle E, Turner ST. Genetics of primary hypertension. Clin Exp Hypertens A. 1986;8(4-5):623–51. doi: 10.3109/10641968609046580. [DOI] [PubMed] [Google Scholar]
- 7.Chien KL, Yang CY, Lee YT. Major gene effects in systolic and diastolic blood pressure in families receiving a health examination in Taiwan. J Hypertens. 2003;21(1):73–9. doi: 10.1097/00004872-200301000-00016. [DOI] [PubMed] [Google Scholar]
- 8.Frisancho AR, Farrow S, Friedenzohn I, Johnson T, Kapp B, Miranda C, et al. Role of genetic and environmental factors in the increased blood pressures of Bolivian blacks. Am J Hum Biol. 1999;11(4):489–98. doi: 10.1002/(SICI)1520-6300(1999)11:4<489::AID-AJHB8>3.0.CO;2-S. [DOI] [PubMed] [Google Scholar]
- 9.Ibrahim MM, Damasceno A. Hypertension in developing countries. Lancet. 2012;380(9841):611–9. doi: 10.1016/S0140-6736(12)60861-7. [DOI] [PubMed] [Google Scholar]
- 10.Huang Y, Lan Y, Thomson SJ, Fang A, Hoffmann WC, Lacey RE. Development of soft computing and applications in agricultural and biological engineering. Computers and Electronics in Agriculture. 2010;71(2):107–27. [Google Scholar]
- 11.Li H, Liang Y, Xu Q. Support vector machines and its applications in chemistry. Chemometrics and Intelligent Laboratory Systems. 2009;95(2):188–98. [Google Scholar]
- 12.Zhang X, Guo Y. Optimization of SVM parameters based on PSO algorithm.; Proceedings of the 5th International Conference on Natural; 2009 Aug 14; Tianjin, China. 2009. pp. 536–9. [Google Scholar]
- 13.Sarrafzadegan N, Baghaei A, Sadri G, Kelishadi R, Malekafzali H, Boshtam M, et al. Isfahan healthy heart program: Evaluation of comprehensive, community-based interventions for non-communicable disease prevention. Prevention and Control. 2006;2(2):73–84. [Google Scholar]
- 14.Sarrafzadegan N, Talaei M, Sadeghi M, Kelishadi R, Oveisgharan S, Mohammadifard N, et al. The Isfahan cohort study: rationale, methods and main findings. J Hum Hypertens. 2011;25(9):545–53. doi: 10.1038/jhh.2010.99. [DOI] [PubMed] [Google Scholar]
- 15.Vapnik VN. The nature of statistical learning theory. 2nd. Berlin, Germany: Springer; 1995. [Google Scholar]
- 16.Rashedi E, Nezamabadi-pour H, Saryazdi S. Filter modeling using gravitational search algorithm. Engineering Applications of Artificial Intelligence. 2011;24(1):117–22. [Google Scholar]
- 17.Rashedi E, Nezamabadi-pour H, Saryazdi S. GSA: A Gravitational Search Algorithm. Information Sciences. 2009;179(13):2232–48. [Google Scholar]
- 18.National Center for Biotechnology Information. Intersalt: an international study of electrolyte excretion and blood pressure. Results for 24 hour urinary sodium and potassium excretion. Intersalt Cooperative Research Group. BMJ: British Medical Journal. 1988;297(6644):319–26. doi: 10.1136/bmj.297.6644.319. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Smith WC, Crombie IK, Tavendale RT, Gulland SK. Urinary electrolyte excretion, alcohol consumption, and blood pressure in the Scottish heart health study. BMJ. 1988;297:329. doi: 10.1136/bmj.297.6644.329. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Janus ED, Wat NMS, Lam KSL, Cockram CS, Siu STS, Liu LJ, et al. The prevalence of diabetes, association with cardiovascular risk factors and implications of diagnostic criteria (ADA 1997 and WHO 1998) in a 1996 community-based population study in Hong Kong Chinese. Diabetic Medicine. 2000;17(10):741–5. doi: 10.1046/j.1464-5491.2000.00376.x. [DOI] [PubMed] [Google Scholar]
- 21.Stamler R, Stamler J, Riedlinger WF, Algera G, Roberts RH. Weight and blood pressure. Findings in hypertension screening of 1 million Americans. JAMA. 1978;240(15):1607–10. doi: 10.1001/jama.240.15.1607. [DOI] [PubMed] [Google Scholar]
- 22.Hirschler V, Aranda C, Calcagno ML, Maccalini G, Jadzinsky M. Can waist circumference identify children with the metabolic syndrome? Arch Pediatr Adolesc Med. 2005;159(8):740–4. doi: 10.1001/archpedi.159.8.740. [DOI] [PubMed] [Google Scholar]
- 23.Moreno LA, Pineda I, Rodriguez G, Fleta J, Sarria A, Bueno M. Waist circumference for the screening of the metabolic syndrome in children. Acta Paediatr. 2002;91(12):1307–12. doi: 10.1080/08035250216112. [DOI] [PubMed] [Google Scholar]
- 24.Alexander JK. Obesity and circulation. Mod Concepts Cardiovasc Dis. 1963;32:799–803. [PubMed] [Google Scholar]
- 25.Siani A, Cappuccio FP, Barba G, Trevisan M, Farinaro E, Iacone R, et al. The relationship of waist circumference to blood pressure: the Olivetti heart study. Am J Hypertens. 2002;15(9):780–6. doi: 10.1016/s0895-7061(02)02976-x. [DOI] [PubMed] [Google Scholar]
- 26.Beard TC, Blizzard L, O'Brien DJ, Dwyer T. Association between blood pressure and dietary factors in the dietary and nutritional survey of British adults. Arch Intern Med. 1997;157(2):234–8. [PubMed] [Google Scholar]
- 27.Tanaka T, Okamura T, Miura K, Kadowaki T, Ueshima H, Nakagawa H, et al. A simple method to estimate populational 24-h urinary sodium and potassium excretion using a casual urine specimen. J Hum Hypertens. 2002;16(2):97–103. doi: 10.1038/sj.jhh.1001307. [DOI] [PubMed] [Google Scholar]
- 28.Johnson D, Prud'homme D, Despres JP, Nadeau A, Tremblay A, Bouchard C. Relation of abdominal obesity to hyperinsulinemia and high blood pressure in men. Int J Obes Relat Metab Disord. 1992;16(11):881–90. [PubMed] [Google Scholar]
- 29.Okosun IS, Prewitt TE, Cooper RS. Abdominal obesity in the United States: prevalence and attributable risk of hypertension. J Hum Hypertens. 1999;13(7):425–30. doi: 10.1038/sj.jhh.1000862. [DOI] [PubMed] [Google Scholar]
- 30.Elliott P, Stamler J, Nichols R, Dyer AR, Stamler R, Kesteloot H, et al. Intersalt revisited: further analyses of 24 hour sodium excretion and blood pressure within and across populations. Intersalt Cooperative Research Group. BMJ. 1996;312(7041):1249–53. doi: 10.1136/bmj.312.7041.1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ho SY, Lam TH, Janus ED. The Hong Kong Cardiovascular Risk Factor Prevalence Study steering committee. Waist to stature ratio is more strongly associated with cardiovascular risk factors than other simple anthropometric indices. Ann Epidemiol. 2003;13(10):638–91. doi: 10.1016/s1047-2797(03)00067-x. [DOI] [PubMed] [Google Scholar]
- 32.Youssef AA, Srinivasan SR, Elkasabany A, Cruickshank JK, Berenson GS. Temporal relation between blood pressure and serum creatinine in young adults from a biracial community: the Bogalusa Heart Study. Am J Hypertens. 2000;13(7):770–5. doi: 10.1016/s0895-7061(00)00233-8. [DOI] [PubMed] [Google Scholar]
- 33.Olden JD, Joy MK, Death RG. An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling. 2004;178(3GÇô4):389–97. [Google Scholar]