Validity of continuous metabolic syndrome score for predicting metabolic syndrome; a systematic review and meta-analysis

Abstract

Background

Nowadays, use of continuous metabolic syndrome (cMetS) score has been suggested to improve recognition of metabolic syndrome (MetS). The aim of this study was to evaluate the validity of cMetS scores for predicting MetS.

Methods

We searched the electronic databases included MEDLINE/PubMed, Embase, ISI Web of Science, and Scopus from 1 January 1980 to 30 September 2020. Observational studies on participants with different cMetS scores were included in this meta-analysis. The sensitivity, specificity, positive likelihood ratio (LR+), negative likelihood ratio (LR) and diagnostic odds ratio (DOR) with 95% CI were calculated.

Results

Ten studies involving a total of 25,073 participants were included. All studies had cross-sectional design. The pooled sensitivity and specificity of cMetS scores for predicting MetS were 0.90 (95% CI: 0.83 to 0.95) and 0.86 (95% CI: 0.83 to 0.89), respectively. Moreover, cMetS scores had the pooled LR+ of 6.5 (95% CI: 5.0 to 8.6), and a pooled (LR-) of 0.11 (95% CI: 0.063 to 0.21). The pooled DOR of cMetS scores to predict MetS were 57 (95% CI: 26 to 127).

Conclusions

The high sensitivity and specificity of cMetS scores indicates that it has a high accuracy to predict the risk of MetS. Furthermore, the cMetS scores has a good ability to rule out healthy people.

Study registration

This study was registered as PROSPERO CRD42020157273.

Background

Metabolic syndrome (MetS) encompasses a set of cardiometabolic risk factors and is defined by coexistence of abdominal obesity, elevated levels of blood pressure (BP), high concentration of serum triglycerides (TG) and blood glucose and reduced high density lipoprotein cholesterol (HDL-C) [1]. MetS prevalence is becoming a worldwide challenge due to the global rise in being overweight and obesity [2, 3]. According to the previous studies, it is estimated that 11.9–37.1% of the Asians, 11.6–26.3% of the Europeans, and 18.8–43.3% of the South Americans suffer from MetS [4–6]. Nevertheless, this categorical definition of MetS has some limitations to identify abnormalities in the early stages. Thus, a continuous metabolic syndrome (cMetS) score has been proposed for overcoming such limitations. Compared with traditional MetS criteria, cMetS is statistically more sensitive and less error-prone and also has greater statistical power [7, 8]. In other words, cMetS scores are more reliable to predict the risk of MetS and cardiovascular outcomes [9–11]. Hence, using cMetS instead of a binary definition for epidemiological analyses has been suggested [12, 13]. However, in clinical practice, the dichotomous definition of MetS score remains valuable [14]. Additionally, there is no universal and uniform definition of MetS in children or adolescence, and in most studies, the adults modified definition of MetS have been used [15, 16]. Thus, it is also challenging to define MetS in the children and adolescents [7, 17]

A number of analysis have been used to derive a continuous score for MetS some of which include Z-score, factor analysis, principal components analysis (PCA), and confirmatory factor analysis (CFA) [5, 6, 17–23]. Still, as the cMetS score is sample-specific and also has not yet been evaluated for different population as its main limitations, the validity of this score should be evaluated. Consequently, the purpose of the present study is to evaluate the efficacy and validity of this score in predicting the risk of MetS in different population all over the world. This is the first study that has systematically reviewed all published studies which investigates the validity of the cMets score in predicting MetS.

Methods

Literature search

We searched the electronic databases included MEDLINE/PubMed, Embase, ISI Web of Science, and SCOPUS from January 1, 1980 to August 31, 2020 without any language limitations. All included reference lists of studies were checked for additional relevant studies. Moreover, we searched the scientific conferences of both Annual Scientific Meeting of the Obesity Society and Annual Meeting of the American College of Sports Medicine (ACSM) and PLoS ONE journal. The adjusted search terms for each databases were “continuous metabolic syndrome score” or “Metabolic Syndrome Score” or “CMets” or “Metabolic Risk Score”. Two authors independently screened the titles and abstracts and then the full-texts of documents to exclude irrelevant studies. Disagreements between the two authors were resolved by consensus, with involvement of a third author if needed. For missing information or documents which we could not access to the full-text, the authors were contacted for full-text or more information.

Inclusion and exclusion criteria

Two authors independently selected the eligible studies according to the following inclusion criteria: [1] Population: we included participants with any age groups [2] Type of study: any type of observational studies including cross-sectional, case-control and cohort [3] Reference standard: five diagnostic criteria of the MetS such as National Cholesterol Education Program (ATPIII) or International Diabetes Federation (IDF), World Health Organization (WHO), American Association of Clinical Endocrinologists (AACE) and revised National Cholesterol Education Program (NCEP-R) must be used as reference test (4) Index test: cMetS scores approaches of Z-score, confirmatory factor analysis (CFA) and principal components analysis (PCA). We excluded case reports, reviews, abstracts and conference papers with insufficient data. Also, relevant studies without sufficient data were excluded.

Data extraction

We used a standardized data extraction form to obtain study characteristics (first author, publication year, country, study design, sample size), participants’ characteristics (e.g., age, gender, study population), type of index test, calculation methods of index test, type of reference test, optimal threshold of index test, diagnostic statistics (e.g., specificity, sensitivity, true positive, false positive, true negative and false negative). Two reviewers independently extracted the data, compared data extraction forms for each study and resolved all disagreements.

Quality assessment

Quality assessment of included studies was performed using the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool independently by two authors. Disagreements were resolved by discussion or by a third reviewer. The QUADAS tool consists of 4 key domains that discuss patient selection, index test, gold standard, and flow of patients via the study and timing of the index tests and gold standard. All domains are assessed in terms of the risk of bias, and the first 3 domains are also assessed in terms of applicability. All items answered as “yes,” “no,” or “unclear” and “yes” indicates low risk of bias [24].

Statistical analysis

The meta-analysis was carried out using the STATA V.14.0 software using bivariate random-effects model approach. The pooled estimates of area under receiver operating characteristics (AUROC) curve, specificity, sensitivity, positive likelihood ratio (LR+), negative likelihood ratio (LR-) and diagnostic odds ratio (DOR), also their 95% confidence intervals were calculated using the true positive, false positive, true negative and false negative data extracted in the included studies. The study results of sensitivities, specificities, LR+ and LR- were presented graphically in coupled forest plots. We used a summary receiver operating characteristics (SROC) curve to show pooled sensitivities and specificities. We evaluated the heterogeneity using the I²-statistic of the pooled DOR. A p value >0.10, represented lack of heterogeneity. The sources of heterogeneity were explored by subgroup analysis. The publication bias was assessed using Deeks’ funnel plot and a P value<0.05 was indicated as a considerable publication bias.

Results

Literature research

A total of 1113 relevant records were identified initially from the mentioned databases and gray literatures. After deletion of duplicates, 517 records remained. Then, 437 records were excluded by titles and abstracts screening. Of these, 80 records were reviewed in full texts. Sixty five articles were excluded for not being relevant to our review or not having valid data. Finally, 10 articles were included in the meta-analysis [6, 12, 18, 21, 25–30]. The flow diagram shows the screening and selection process in detail (Fig. 1).

Fig. 1.

The PRISMA flowchart for selection of primary study

Study characteristics

The characteristics of the 10 included studies with 25,073 participants are detailed in Table 1. All studies had cross-sectional design. The articles were published from 2010 to 2019 with age range between 7 to 94 years. Study population in the eight studies were children or adolescents, while two studies recruited adult population. All of the studies enrolled both female and male almost in the same ratio. In eight studies, the cMetS score was derived using Z-score approach. Whereas, two others were applied principal components analysis (PCA) or regression models to predict potential risk of MetS. The majority of studies (9 studies) performed the diagnosis of MetS according to the criteria of National Cholesterol Education Program (ATPIII); however, only one study used International Diabetes Federation (IDF) criteria as reference test. Almost in all studies ROC curve analysis was used to calculate cMetS score’ optimal cutoff as predictor of MetS. The optimal cutoffs varied between −1.1 and 3.7 in the studies.

Table 1.

Study characteristics of the 10 included studies

Study	Country	Study design	Population	Sample size	Age (range)	Index test (type of Cmets)	Index test calculation	Diagnostic test of Mets	Optimal cut off points	Sensitivity(%) overall	Specificity (%) overall
Sawant 2019	Slovakia	Cross-Sectional	Overweight and obese children	T:104 M:50 F:54	7–14	cMetS	Z-score	IDF*	−0.68	91%	70%
Reuter 2019	Brazil Slovakia	Cross-Sectional	Children	T:1739 M:754 F:985	10–17	cMetS	Z-score	ATPIIIa	3.42	100%	92%
Khoshhali 2019	Iran	Cross-Sectional	Healthy children and adolescents	T: 3843 M:2012 F:1831	7–18	cMetS First PCA siMS score	Z-score PCA Z-score	ATP III	2.41 0.65 2.64	87% 86% 92%	86% 79% 78%
Heshmat 2017	Iran	Cross-Sectional	Healthy children and adolescents	T:3843	7–18	CmetS	Z-score	ATPIII	1.76	93%	82%
Wiley 2016	Australia	Cross-Sectional	Healthy adults	T: 2125 M: 907 F:1218	18–94	MetSSS	PCA	ATPIII	3.3	51%	83%
Hosseini 2014	Iran	Cross-Sectional	Healthy adults	T: 8313	≥19	CMetSS	Regression	ATPIII	−1.15	89%	87%
Shafiee 2013	Iran	Cross-Sectional	Healthy adolescents	T: 3254 M:1679 F:1575	10–18	CmetS	Z-score	ATPIII	2.93	92%	91%
Pandit 2011	India	Cross-Sectional	Children and adolescents	T: 236 M:121 F:115	6–17	CmetS	Z-score	ATPIII	−0.8	80%	94%
Okosun 2010	USA	Cross-Sectional	Healthy adolescents	T: 1239 M:674 F:562	12–19	CmetS	Z-score	ATPIII	1.94	83%	83%
Eisemann 2010	USA	Cross-Sectional	Children	T: 378 M:194 F:184	7–9	CmetS	Z-score	ATPIII	3.72	99%	93%

*IDF: International Diabetes Federation

^aATPIII: Adult Treatment Panel III

Pooled results

Overall, the combined values of the sensitivities and specificities were 0.90 (95% CI: 0.83 to 0.95) and 0.86 (95% CI: 0.83 to 0.89), respectively (Fig. 2). Also, cMetS scores had a pooled (LR+) of 6.5 (95% CI: 5.0 to 8.6), and a pooled (LR-) of 0.11 (95% CI: 0.063 to 0.21). The forest plots are presented in Fig. 3. The pooled estimates of DOR of cMetS scores to predict MetS were 57 (95% CI: 26 to 127) (Fig. 4). The SROC curve shows an overall prediction region and 95% CI in Fig. 5. The AUC was 0.93 (95% CI: 0.90 to 0.95).

Fig. 2.

The pooled of sensitivities and specificities

Fig. 3.

The pooled of negative and positive likelihood ratio

Fig. 4.

The pooled of diagnostic odds ratio

Fig. 5.

Summary receiver operating characteristics curve

Subgroup analysis and sensitivity analysis

We performed subgroup analysis based on different cMetS scores, population subsets and also quality of included studies. Due to lack of diversity between diagnostic criteria of MetS (reference tests), subgroup analysis was not carried out. The heterogeneity (I²) was slightly lower in studies with children and adolescence participants and also in studies that used Z-score approach to calculate cMetS scores. Funnel plots were presented in Appendix 1. The DOR was higher in studies with low risk of bias than the high risk of bias studies. The results are shown in Table 2. Sensitivity analysis using one-out removing method was done in order to explore the studies affecting the results. The funnel plot showed no study with significant impact. Even though we tried to find the potential sources of heterogeneity in the meta-analysis by subgroup analysis, but due to the lack of sufficient number of studies this could not be done.

Table 2.

Subgroup meta-analysis of included studies for association of CmetS with MetS

Subgroups		OR (95% CI)	No of study	Heterogeneity X2	P value	I-squared%
Population	Adult Children and adolescence	17.25 (1.57–188.92) 56.40(33.80–94.11)	2 8	327.59 37.17	<0.001 <0.001	99.7% 75.8%
Method of cMetS scores calculation	Z-score PCA Regression	69.64(37.11–130.68) 10.85(2.35–50.11) 57.39(48.43–68.01)	8 2 2	29.13 28.99 0.72	<0.001 <0.001 0.39	76.0% 96.6% 0.0%
Quality assessment	Low risk of bias High risk of bias	57.89(37.89–88.45) 19.82(4.52–97.74)	7 3	33.00 40.30	<0.001 <0.001	75.8% 95.0%
All studies		51.99(22.63–119.44)	10	405.26	<0.001	97.3%

Quality assessment and publication bias

The results of quality assessment of included studies are presented in Table 3. Overall, the risk of bias in most of the studies was low. Patient selection domain was recognized as potential bias source in three studies. Deeks plot asymmetry test was performed to assess publication bias. The funnel plot displayed inconsiderable publication bias (P = 0.54) (Appendix 2).

Table 3.

The quality assessment of included studies based on the QUADAS-2 tool

STUDY	RISK OF BIAS				APPLICABILITY CONCERNS			Quality score
	PATIENT SELECTION	INDEX TEST	REFERENCE STANDARD	FLOW AND TIMING	PATIENT SELECTION	INDEX TEST	REFERENCE STANDARD
Sawant et al., 2019	L	L	L	L	L	L	L	Low risk of bias
Reuter et al., 2019	L	L	L	L	L	L	L	Low risk of bias
Khoshhali et al., 2019	L	L	L	L	L	L	L	Low risk of bias
Shafiee et al., 2013	L	L	L	L	L	L	L	Low risk of bias
Heshmat et al., 2017	L	L	L	L	L	L	L	Low risk of bias
Eisemann et al., 2010	L	L	L	L	L	L	L	Low risk of bias
Pandit et al., 2011	U	L	L	L	U	L	L	At risk of bias
Okosun et al., 2010	U	L	L	L	U	L	L	At risk of bias
Wiley et al., 2016	H	L	L	L	L	L	L	At risk of bias
Hosseini et al., 2014	L	L	L	L	L	L	L	Low risk of bias

L: low; H: high; U: unclear

Discussion

The present systematic review and meta-analysis demonstrates that cMetS score can be a valuable test with moderate to high accuracy in predicting the risk of MetS. The overall AUC ROC curve was 0.93 in this meta-analysis which was acceptable (> 0.9) [31] and DOR was 57 with 95% CI: 26 to 127, so cMetS score was approximately highly accurate, sensitive and specific for predicting risk of MetS. Various studies on the utility of cMetS have documented that the score predicts cMetS accurately [6, 12, 32]. Therefore, the results of our study are in line with the literature.

Despite the considerable heterogeneity within the included studies, the present meta-analysis provided good evidence for the use of derived cMetS score cutoff values and pooled AUC. The high sensitivity and moderate to high specificity of cMetS score can help predict patients who are at a high risk or low risk of developing MetS, respectively. This could boost confidence in initial exit strategies aimed at saving the healthcare system’s vast costs. It can also help to more accurately identify people at risk of MetS or those with the syndrome. However, MetS is not limited to adults. There is a growing body of evidence about its presence from early childhood [33]. On the other hand, studies have demonstrated significant relationship between childhood MetS, defined by cMetS, and adult cardiovascular risk [11, 34]. Therefore, the predictive ability of cMetS is especially important in the field of chronic diseases and diseases related to obesity in all age groups.

In the present meta-analysis as mentioned above, the amount of both positive and negative LR is significantly considerable (pooled (LR+) of 6.5 (95% CI: 5.0 to 8.6), and a pooled (LR-) of 0.11 (95% CI: 0.063 to 0.21)). This indicates that cMetS score has a substantial ability to separate healthy people and ruling in people at risk of or with MetS. However, comparing the positive and negative LR showed that the power of ruling out healthy people is significantly higher than identifying people at risk or affected. Although, there are some disadvantages to using LRs. As is well known, LRs express odds results whereas clinicians are more used to dealing with probabilities as in the case of predictive values therefore LRs are not often quoted in the medical literature.

The findings of this study should be interpreted with considerable limitations in mind. The major limitation of our study was the presence of significant heterogeneity which may be partly explained by differences in study population, setting, country, methods for calculating cMetS score and differences in the reference standard used for predicting MetS. However, we used a random effects model when pooling study data in all of these analyses, to provide a more conservative estimate of accuracy. Also, to reduce heterogeneity we performed subgroup analysis based on test calculation method and subset population. Although a decreasing trend was seen in studies on children and adolescents compared to adults, unfortunately heterogeneity remained significant in both groups. The same was true for the different methods of calculating the cMetS.

Furthermore, it should also be noted that there was no uniform or specific matched cut-off value between included studies, which could generate heterogeneity. Thus, we performed meta-regression to find a significant relationship between the cut-offs and the accuracy of the cMetS. The results did not show a significant relationship. Although, all of the included studies were observational (cross-sectional) and we only included studies that utilized ROC analysis. So small number of studies were included in this meta-analysis. Since this score is novel in the world, therefore more high-quality studies need to be carried out in the future.

Strengths of this study include a comprehensive search strategy, including a recursive search of the reference of all eligible articles, and searching of gray literature such as conference proceedings to discover any potential studies published that may not have been found in the original search. It should also be borne in mind that most of the included studies were of high quality.

Conclusion

Based on our knowledge this is the first comprehensive systematic review and meta-analysis to explore the utility and validity of cMetS score in predicting the risk of MetS as compared with reference standard in children, adolescents and adults.

In conclusion, the high sensitivity and moderate to high specificity of cMetS score demonstrated that the score can be a moderately accurate and a cost-effective test to predict risk of MetS. The score is becoming widely used in obesity epidemiological research and additional studies with high sample size and relevant design are needed for using this score in clinical practice. Moreover, multicenter studies are needed to be conducted in order to confirm results existing in the current literature.

Appendix 1

Fig. 6.

Graph quality assessment subgroup

Fig. 7.

Graph population subgroup

Fig. 8.

Graph cMetS subsets

Appendix 2

Fig. 9.

Graph population subgroup

Authors’ contributions

MK, MH, SM, HSE, SD, SKS, AMG, MQ, AK design and data gathering, MQ, SD, MEA, SKS, AK design and revision, MQ, AK data analysis. All authors read and approved the final manuscript.

Funding

This study was funded and designed by Alborz University of Medical Sciences.

Declarations

Ethics approval and consent to participate

The Research and Ethics council of Alborz University of Medical Sciences approved the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.