Hypothyroidism and related diseases: a methodological quality assessment of meta-analysis

Abstract

Objectives

There is an increasing prevalence of hypothyroidism and there is a growing body of meta-analyses (MAs) on the association between hypothyroidism and other diseases. However, the methodological quality of the MAs significantly varies. Thus, this study aimed to evaluate and summarise data on the methodological quality of MAs on the associations between hypothyroidism and other diseases using the Assessment of Multiple Systematic Reviews (AMSTAR) scale, providing suggestions for clinical decision-making processes.

Design

To assess the methodological quality of MAs using the AMSTAR scale.

Data sources

A systematic literature search was performed in PubMed, EMBASE, the Cochrane Library, web of science and Chinese Biomedicine Literature Database.

Eligibility criteria

We included MAs that had assessed the association between hypothyroidism and other diseases in humans and that had full texts regardless of the publication status. No restriction applied on language or date.

Data extraction and synthesis

Two reviewers independently screened the titles and abstracts of all searched literature to acquire potentially eligible publications. The full texts of possible eligible publications were downloaded and assessed. Inconsistent comments were resolved through discussions with a third reviewer.

Results

52 studies were included. The average AMSTAR score of the included articles was 8.6 (range: 5–10), and those of English and Chinese MAs were 8.8 and 7.0, respectively. A total of 52 MAs were evaluated, and 19 (36.5%) and 33 (63.5%) of these MAs were of moderate and high quality, respectively. None of the MAs were of low quality. Only two MAs had an a priori design. Items 3, 5 and 9 had the highest compliance (50/52, 96.2%), and aside from item 1, items 7 and 8 had the lowest compliance (33/52,63.5%). According to the results of these MAs, hypothyroidism was significantly associated with cardiovascular diseases, metabolic diseases, neuropsychiatric disorders, breast cancer and pregnancy outcome.

Conclusions

The methodological quality of the included MAs on the association between hypothyroidism and other diseases was moderate to high. MAs with high qualities confirmed that hypothyroidism was significantly associated with cardiovascular diseases, metabolic syndrome, preterm birth and neonatal outcomes. Consideration of scientific quality when formulating conclusions should be made explicit and more attention should be paid to improving the methodological quality of MAs, and increasing their applicability for clinical decision-making.

Strengths and limitations of this study.

• This study first assessed the methodological quality of meta-analyses (MAs) on the association between hypothyroidism and other diseases.

• We used the Assessment of Multiple Systematic Reviews (AMSTAR) scale to assess the methodological quality of MAs.

• The included MAs were randomly selected without restriction and followed rigorous inclusion and exclusion criteria.

• One limitation of this study is that the AMSTAR appraisal process was difficult to implement when the reporting quality was poor.

• We tried our best to make a conversion between OR and relative ratio by extracting raw data from MAs, but some data were not obtained.

Introduction

Hypothyroidism is defined as an increase in serum thyroid-stimulating hormone levels, with decreased (overt hypothyroidism [OH]) or normal (subclinical hypothyroidism [SCH]) serum thyroid hormone levels.¹ Epidemiological data have indicated that the prevalence of hypothyroidism in the general population ranged from 4.6% to 23.5%,^{2 3} and is predominant in elderly individuals and women.⁴ OH is associated with weight gain, dyslipidaemia and hypertension, which are confounders of atherosclerosis and subsequently causes coronary heart disease (CHD).⁵ Prospective cohort studies have shown that SCH can increase the risk of heart failure (HF),⁶CHD events and mortality.⁷ Recently, Canpolat et al ⁸ have revealed that impaired gastric motility and resultant upper gastrointestinal symptoms were observed in individuals with SCH, and thyroid hormone replacement was beneficial in improving symptoms and changes in clinical indicators. In addition, studies on the effects of SCH on cognitive impairment,^{9 10} metabolic syndrome (MetS)^{11 12} and fractures^{13 14} have shown conflicting results.

Meta-analyses (MAs) have been becoming a popular and powerful type of evidence, and they have several advantages, including overcoming the limitation of small sample sizes and pooling individual study results to generate a single best estimate.¹⁵ High-quality MAs can provide guideline developers better evidence to formulate guidelines, show the advantages and disadvantages of an intervention and guide health professionals, clinicians and stakeholders with the best interventions for targeted patients.¹⁶ The Cochrane Collaboration is an international network of healthcare professionals who prepare and regularly update systematic reviews (SRs), and Cochrane reviews were generally of high quality.¹⁵ In 2016, 73 reviews, which were completed by 11 Cochrane Review Groups, were used in 14 of 18 (78%) WHO guidelines.¹⁷ By contrast, in MAs, a poor methodology can negatively affect decision-making processes.¹⁵ A great number of MAs that evaluate the role of hypothyroidism in the development and progression of other diseases have been published. However, the methodological quality of these MAs had not been evaluated, and the conclusions of these MAs were debatable.

Assessment of Multiple Systematic Reviews (AMSTAR), which was developed based on previous tools, empirical evidence and expert consensus and can be applied to SRs/MAs of both randomised controlled and non-randomised studies, is the most recent, reliable and valid tool for evaluating MAs.^{18 19} Gagnier et al have assessed the methodological quality of SRs and MAs in the top five orthopaedic journals using AMSTAR. The included articles did not conform to the accepted standards of quality. Moreover, the validity of the published SRs is questionable, and their contribution to clinical decision-making is suboptimal.¹⁹ Remschmidt et al have used the AMSTAR to investigate the methodological quality of the SRs and MAs on influenza vaccination and identify influencing factors.²⁰ Tian et al compared the methodological quality of SRs from China and the USA using the AMSTAR scale and concluded that the overall methodological quality of the SRs from China was similar to that from the USA.²¹

Thus, this study aimed to evaluate and summarise data on the methodological quality of MAs on the associations between hypothyroidism and other diseases using the AMSTAR scale, providing suggestions for clinical decision-making processes.

Materials and methods

Data sources and study selection

We performed a systematic literature search from inception to 1 April 2018 in PubMed, Embase, the Cochrane Library, Web of Science and Chinese Biomedical Literature Database to identify MAs on the associations between hypothyroidism and other diseases. The combinations of the following keywords were used in the search strategy: ‘thyroid’, ‘hypothyroidism’, ‘subclinical hypothyroidism’ and ‘meta-analysis’. Free-text words and MeSH terms were entered depending on the characteristics of the database. Search strategy is shown in online supplementary appendix 1. To minimise the risk of missing relevant literature, reference lists from retrieved articles were hand screened for additional applicable studies. We did not apply any restriction to language or date.

Inclusion and exclusion criteria

We included MAs that fulfilled the following criteria: (1) MAs that assessed the association between hypothyroidism and other diseases in humans (2) and that had full texts regardless of publication status, (3) studies in abstract form or meeting reports after the authors were contacted and full texts were obtained within a month, (4) recent studies with the same topic from the same author and (5) duplicate copies of MAs in both Chinese and English, including English articles.

Study selection

Two reviewers (FS and YQ) independently screened the titles and abstracts of all searched literature to acquire potentially eligible publications. The full texts of possible eligible publications were downloaded and assessed. Inconsistent comments were resolved through discussions with a third reviewer (LT).

Assessment of methodological quality of the included MAs

Two authors (FS and YQ) independently assessed the methodological quality of the included MAs. The AMSTAR scale, which uses an 11-item questionnaire, was used in this study to assess the methodological quality of MAs because it is the most recent, reliable and valid tool.²² The items in the AMSTAR scale are the following: an a priori design, duplicate study selection and data extraction, comprehensive literature search, use of the status of publication as an inclusion criteria, a list of included and excluded studies, characteristics of included studies, documented assessment of the scientific quality of the included studies, appropriate use of the scientific quality in forming conclusions, appropriate use of methods to combine the findings of studies, assessment of the likelihood of publication bias and documentation of potential conflicts of interest. The items may be answered with a ‘yes’, ‘no’, ‘can’t answer’ or ‘not applicable’. One point was provided when the answer was ‘Yes’; otherwise, no score was provided. The AMSTAR quality score was the summation of the number of ‘yes’. According to the number of criteria met, the included articles were ranked into three levels: ‘high’ (range: 9–11), ‘moderate’ (range: 5–8) and ‘low’ (range: 0–4).

Agreement of assessment

To obtain more reliable data on the methodological quality of MAs, the agreement of the methodological quality assessment results between the two independent reviewers (FS and YQ) was investigated through a plot test. In this process, 10 MAs were selected from the included studies. The two reviewers independently assessed the methodological quality of these MAs using the AMSTAR scale. We calculated the agreement proportion and Cohen’s kappa (k) value for each of the 11 AMSTAR items. A k value between 0.81 and 1.00 indicated a good agreement.²³ During the first assessment, the agreement for items 7 and 8 between the two reviewers were poor (0.67 and 0.70, respectively), and good agreement was obtained after a discussion between the reviewers.

Patient involvement

Given its methodological focus, we did not evaluate patient-related outcomes. Therefore, we also chose not to involve patients’ input in its design. However, the aim of this study is to indirectly benefit the welfare of patients by promoting the development of high-quality MAs.

Results

Study identification

Initially, we yielded 3107 potentially relevant articles by searching the five electronic databases and other sources. After identifying duplications and screening the titles and abstracts, 3039 articles were excluded. We attempted to find the full texts of the remaining 68 articles for intensive reading. Of these, 12 articles that met the criteria were published as abstracts. Moreover, we aimed to obtain the full texts by contacting the authors. However, the authors did not respond within a month. Thus, these articles were excluded. Two Chinese articles^{24 25} were excluded from publishing in both Chinese and English languages. In addition, two MAs^{26 27} were excluded from the old version. Thus, we only included the recently updated ones.²⁸ Finally, 52 eligible MAs were included in our analysis. The flowchart of the review selection process is presented in figure 1.

Figure 1.

Flow chart of the review search and identification.

Characteristics of the included MAs

Among the 52 included eligible MAs, 46^28–73 (46/52, 88.5%) were published in English, and the rest^74–79 (6/52, 11.5%) were published in Chinese. The 52 included MAs contained 685 studies, with an average of 13.2 (range: 3–39) studies per article. The publication years of the included articles ranged from 2008 to 2018, and 82.7% of the articles were published in the last 5 years. The characteristics of the 52 MAs are shown in online supplementary appendix 2.

Ten39 40 43 49 57 58 65 68–70 MAs were about the associations between OH and other diseases, and 3528–36 38 42 44–47 50–56 59–64 66 67 75–79 MAs have shown the associations between SCH and other diseases. Seven37 41 48 71–74MAs have investigated the effects of both OH and SCH on other diseases. Except for the general population, pregnant women with hypothyroidism were also included in the target groups of the included MAs.35 37 40 41 48 51 61 64 67 71 74 The characteristics of these MAs are shown in table 1.

Table 1.

Characteristics of MAs

Characteristics of MAs	No of MAs
Target groups
General population with OH or SCH	40
Pregnant women with OH or SCH	12
Associated diseases
Cardiovascular events	7
Preterm birth	6
Fracture	5
Carotid intima-media thickness	4
Neonatal outcomes	4
Cognitive function	3
Intelligence development of the offspring	3
Blood pressure	3
Metabolic syndrome	2
Serum lipid level	2
Breast cancer	2
Chronic kidney disease	2
Glaucoma	2
Diabetes complications	2
Non-alcoholic fatty liver disease	2
Intrauterine growth restriction	2
Left ventricular diastolic function	1
Stroke	1
Plasma homocysteine status	1
Carpal tunnel syndrome	1
Obstructive sleep apnoea	1
Gestational diabetes	1
Metabolic and hormonal profile	1
Miscarriage	1
Language
English	46
Chinese	6
Publication year
2008–2010	3
2011–2013	6
2014–2016	27
2017–2018	16
Impact factor
<3.5	24
≥3.5	20

OH, overt hypothyroidism; MAs, meta-analyses; SCH, subclinical hypothyroidism

Methodological quality of the included MAs

The average AMSTAR score of the included articles was 8.6 (range: 5–10), and those of English and Chinese MAs were 8.8 and 7.0, respectively. A total of 52 MAs were evaluated, and 19 (36.5%) and 33 (63.5%) of these MAs were of moderate and high quality. None of the MAs was of low quality. The numbers and percentages of each response (‘yes’, ‘no’, ‘can’t answer’ and ‘not applicable’) among the MAs for each AMSTAR item are shown in table 2. Two MAs had an a priori design. Items 3, 5 and 9 had the highest compliance (50/52, 96.2%), and aside from item 1, items on whether the scientific quality of the included studies was assessed and documented and whether the scientific quality of the included studies has been used appropriately in formulating conclusions had the lowest compliance 35/52 (67.3%). For item 7, the Newcastle–Ottawa Scale, the most commonly used tool for observational studies, was used.

Table 2.

Methodological quality of the included MAs

Items	Y, n (%)	N, n (%)	CA, n (%)	NA, n (%)
1. Was an ‘a priori’ design provided?	2 (3.8)	50 (96.2)	0 (0)	0 (0)
2. Was there duplicate study selection and data extraction?	44 (84.6)	1 (1.9)	7 (13.5)	0 (0)
3. Was a comprehensive literature search performed?	51 (98.1)	1 (1.9)	0 (0)	0 (0)
4. Was the status of publication used as an inclusion criterion?	44 (84.6)	8 (15.4)	0 (0)	0 (0)
5. Was a list of studies provided?	51 (98.1)	1 (1.9)	0 (0)	0 (0)
6. Were the characteristics of the included studies provided?	50 (96.2)	2 (3.8)	0 (0)	0 (0)
7. Was the scientific quality of the included studies assessed and documented?	38 (73.1)	14 (26.9)	0 (0)	0 (0)
8. Was the scientific quality of the included studies used appropriately in formulating conclusions?	35 (67.3)	17 (32.7)	0 (0)	0 (0)
9. Were the methods used to combine the findings of studies appropriate?	50 (96.2)	2 (3.8)	0 (0)	0 (0)
10. Was the likelihood of publication bias assessed?	44 (84.6)	8 (15.4)	0 (0)	0 (0)
11. Was the conflict of interest stated?	40 (76.9)	12 (23.1)	0 (0)	0 (0)

Y, yes; N, no; CA, cannot answer; NA, not applicable.

Association between hypothyroidism and other diseases

Conclusions and the number of MAs, case–control (CC)/cohort (CO)/cross-sectional (CS) studies and patients, and average AMSTAR score about the association between hypothyroidism and other diseases are shown in tables 3 and 4.

Table 3.

Conclusions, the number of MAs, average CC/CO/CS studies and patients and average AMSTAR score about the association between OH and other diseases

Conclusions	No of included MAs	Average CC/CO/CS studies included	Average patients included	Average AMSTAR score
Plasma homocysteine levels were found to be significantly higher in patients with OH58	1	2/3/0	586	10
There is a significant association between OH and tunnel syndrome49	1	3/1/0	71 133	7
OH is significantly associated with increased all-cause mortality in patients with HF43	1	0/10/0	19 354	10
OH is significantly associated with increased cardiac death and/or hospitalisation in patients with HF43	1	0/10/0	21 858	10
OH is significantly associated with a risk factor for gestational diabetes37	1	0/3/0	225 427	7
OH is significantly associated with the severity of obstructive sleep apnea57	1	0/8/4	1615	7
OH is significantly associated with breast cancer39	1	1/2/3	6175	6
Maternal OH is significantly associated with the occurrence of preterm birth40 41 48 74	4	0.25/7.5/0.25	1 152 475	8.75
Mothers with hypothyroidism during pregnancy have a significant increased tendency to give birth to children with higher birth weight40	1	0/9/0	1 627 521	8
Mothers with hypothyroidism during pregnancy have a significant increased tendency to give birth to children with lower birth weight40	1	0/5/0	23 879	8
Maternal OH shows a significant trend of reduced risk of large for gestational age40	1	0/3/0	1 612 705	8
No evidence shows a significant relationship between maternal OH and small for gestational age40	1	0/4/0	1 613 846	8
Hypothyroidism has an increased risk of developing glaucoma68	1	2/5/4	381 695	9
There seems to be an association between hypothyroidism and glaucoma69	1	4/2/7	173 763	9
Hypothyroidism is a risk factor for CHD and cardiac mortality70	1	0/13/0	615 596	9
No significant association between NAFLD and OH72	1	1/1/4	27 070	10
OH is at higher risk for NAFLD than euthyroid subjects73	1	4/2/4	42 143	10
Hypothyroidism is not related to the risk for breast cancer65	1	12/0/0	24 571	9

AMSTAR, Assessment of Multiple Systematic Reviews; CC/CO/CS; case–control/cohort/cross-sectional; CHD, coronary heart disease; HF, heart failure; MAs, meta-analyses; NAFLD, non-alcoholic fatty liver disease; OH, overt hypothyroidism.

Table 4.

Conclusions, the number of MAs, average CC/CO/CS studies and patients and average AMSTAR score about the association between SCH and other diseases

Conclusions	No of included MAs	Average CC/CO/CS studies included	Average patients included	Average AMSTAR score
Plasma homocysteine levels were not found to be significantly higher in patients with SCH58	1	3/5/0	926	10.00
SCH is not significantly associated with fractures30 52 54	3	0/8/0	128 667	9.7
SCH was associated with increased risk of any location of fractures, hip fractures and forearm fractures60	1	0/13/0	62 490	10
No evidence which could prove a definite association between SCH and the risk of fracture79	1	0/6/0	289 575	7
Serum TC, LDL-C and TG levels were significantly increased in patients with SCH compared with euthyroidism individuals. No significant difference was observed for serum HDL-C36 42	2	5/0/7	22 767	9.00
SCH is associated with a significant decrease in fasting plasma glucose36	1	3/0/1	3507	9.00
SCH is not significantly associated with BMI36	1	7/0/1	3971	9.00
SCH was associated with a significant increase in SBP31 36 56	3	1.7/2/7	23 485	8.00
SCH was associated with a significant increase in DBP31	1	0/0/6	17 323	8.00
SCH is not significantly associated with increased DBP36 56	2	1.7/1.7/5	25 810	8.00
SCH is associated with a significant increase in C-IMT36 59 66 75	4	6.75/3.5/0.5	2420	8.75
SCH has a significant association with arterial wall thickening and stiffening and endothelial dysfunction and increased risk of cardiovascular events59	1	27/0/0	1931	9
SCH is significantly associated with an increased risk for CHD50 63	2	0/6.5/1	8528	7.5
SCH is not significantly associated with an increased risk for CHD28 44 77		0/9.3/0	18 525	7.30
SCH is significantly associated with an increased risk for cardiovascular mortality50 63	2	0/3.5/0.5	6525	7.5
SCH is not significantly associated with an increased risk for cardiovascular mortality28 44	2	0/10/0	33 444	8.00
SCH is not significantly associated with an increased risk for all-cause mortality28 44 50	3	0/6.3/0.3	24 853	7.00
SCH is significantly associated with MetS as defined by the IDF Criteria55	1	0/0/2	7258	10.00
SCH is not significantly associated with MetS as defined by the NCEP-ATP III Criteria34 55	2	2/0/5	24 717	10.00
SCH is not significantly associated with MetS as defined by the Chinese Criteria55	1	0/0/1	1399	10.00
SCH is not significantly associated with MetS as defined by the Japanese Criteria55	1	0/0/2	10 350	10.00
SCH is not significantly associated with cognitive impairment29 45 47	3	0/8.3/4.3	16 833	9.33
SCH patients had significantly worse parameters of left ventricular diastolic function than euthyroid subjects aged <60 years33	1	0/0/14	675	7.00
SCH is significantly associated with a risk factor for gestational diabetes37	1	0/6/0	63 567	7
SCH can significantly increase the risk of diabetic retinopathy in T2DM patients38 53	2	0/8.5/0.5	4101	9.5
SCH can significantly increase the risk of diabetic nephropathy in T2DM patients38 62	2	6/0/1.5	2653	8.5
SCH can significantly increase the risk of diabetic peripheral neuropathy in T2DM patients38	1	3/0/0	1710	10
SCH can significantly increase the risk of peripheral arterial disease in T2DM patients38	1	4/0/0	801	10
SCH is not significantly associated with coronary heart disease in T2DM patients38	1	7/0/0	1896	10
SCH is a significant risk factor of chronic kidney disease in T2DM patients78	1	4/0/2	38 284	6
No significant correlation was found between SCH and stroke32	1	0/5/0	10 118	10
SCH does not influence the hormonal profile of women with polycystic ovary syndrome. But it results in mild metabolic abnormalities in a short-term setting66	1	0/12/0	2341	10
Maternal SCH is not significantly associated with the occurrence of preterm birth48	1	0/10/0	48 684	8
Maternal SCH significantly increases the risk of preterm birth41 59 64 71	4	0/14.1/0	110 951	9.3
Maternal SCH is significantly associated with the risk for intrauterine growth restriction51 64	2	0/5/0	12 558	8.5
Maternal SCH has a significant adverse affect on the intelligence of offspring35 46 76	3	1/0/37	303 360	8.3
SCH patients have a higher prevalence of miscarriage67	1	0/3/0	6036	9
Children of women with SCH were found have a significant lower mean motor scores than those of euthyroidism35	1	0/1/0	160	10
No significant association was found between NAFLD and SCH72	1	0/1/4	26 454	10

AMSTAR, Assessment of Multiple Systematic Reviews; BMI, body mass index; CC/CO/CS, case–control/cohort/cross-sectional; CHD, coronary heart disease; C-IMT, carotid intima-media thickness; DBP, diastolic blood pressure; HDL-C, high-density lipoprotein cholesterol; IDF, International Diabetes Federation; LDL-C, low-density lipoprotein cholesterol; MAs, meta-analyses; MetS, metabolic syndrome; NAFLD, non-alcoholic fatty liver disease; NCEP-ATP III, National Cholesterol Education Programme’s Adult Treatment Panel III; SBP, systolic blood pressure; SCH, subclinical hypothyroidism; TC, total cholesterol; TG, total triglyceride; T2DM, type 2 diabetes mellitus.

According to the AMSTAR score and the number of patients included in these MAs, the association between hypothyroidism and other diseases was summarised by showing the relative ratio (RR)/OR/HR/weighted mean difference/standardised mean difference (tables 5 and 6). Regarding the association between SCH and other diseases, diabetic peripheral neuropathy had the highest RR value (2.27) and OR value (1.87). The association between OH and breast cancer had the highest OR (1.79). However, this value is not considered statistically significant.

Table 5.

Association between SCH/OH and other diseases (showed by RR/HR/OR)

Author and year	Diseases	Sample size	No of included CC/CO/CS studies	RR/HR	OR	AMSTAR score
Subclinical hypothyroidism
Yan et al (2016)54	Fractures	314 146	0/5/0	1.20 (0.70, 2.04)		9
Han et al (2015)38	DN	4761	7/0/3	1.35 (1.26, 1.44)*	1.74 (1.34, 2.28)	10
	DR	4572	9/0/1	1.08 (0.99, 1.18)*	1.42 (1.21, 1.67)
	PAD	801	4/0/0	1.33 (1.10, 1.60)*	1.85 (1.35, 2.54)
	CHD	1896	7/0/0	1.29 (1.09, 1.52)*	1.59 (0.92, 2.76)
	DPN	1710	3/0/0	2.27 (1.98, 2.60)*	1.87 (1.06, 3.28)
Rodondi et al (2010)28	CHD	25 977	0/7/0	1.26 (1.16, 1.38)*	1.33 (1.19, 1.49)*	9
	CHD mortality	54 301	0/10/0	1.63 (1.42, 1.87)*	1.67 (1.45, 1.94)*
	Total mortality	55 287	0/11/0	1.57 (1.48, 1.67)*	1.78 (1.64, 1.92)*
Ning et al (2015)43	All-cause mortality of heart failure patients	19 354	0/10/0	1.44 (1.29, 1.61)		10
	Cardiac death and/or hospitalisation of heart failure patients	21 858	0/10/0	1.37 (1.22, 1.55)
Rieben et al (2016)47	Dementia	7401	0/6/0	1.14 (0.84, 1.55)	1.08 (0.78, 1.51)*	10
Eftekharzadeh et al (2016)34	Metabolic syndrome	34 517	1/1/7		1.13 (0.95, 1.34)	10
Chaker et al (2014)32	Stroke	10 118	0/5/0	1.08 (0.87, 1.34)†		10
Zhou et al (2016)78	CKD in diabetes patients	38 284	4/0/2		1.81 (1.43, 2.29)	6
Hou et al (2016)40	Low birth weight	23 879	0/5/0	1.31 (1.00, 1.72)	1.11 (0.84, 1.48)*	8
	SGA	1 613 846	0/4/0	1.02 (0.87, 1.19)
	LGA	1 612 705	0/3/0	1.17 (0.99, 1.38)	0.99 (0.84, 1.17)*
Li et al (2016)41	Preterm birth	455 716	0/14/1	1.17 (1.07, 1.28)*	1.25 (1.04, 1.51)	10
Tong et al (2016)51	IGR	16 157	0/7/0	2.05 (1.43, 2.94)*	1.54 (1.06, 2.25)	8
Gong et al (2016)37	Gestational diabetes	225 427	0/3/0	1.89 (1.70, 2.11)*	1.558 (1.292, 1.877)	7
Thompson et al (2018)61	Intelligence development of the offspring	909 176	0/37/0		2.14 (1.20, 3.83)	10
Zhou et al (2017)62	Diabetic nephropathy	545	5/0/0		1.8 (1.38, 2.35)	7
Sun et al (2017)63	CHD5	4979	0/10/0	1.17 (0.91, 1.52)		10
Liu et al (2018)64	Fetal growth restriction	8958	0/3/0	2.4 (1.56, 3.7)		9
Zhang et al (2017)67	Premature delivery miscarriage	6036	0/3/0	1.45 (1.07, 1.96)		9
Nasirkandy et al (2017)71	Preterm birth	68 465	0/17/0	1.36 (1.09, 1.68)		9
Xu et al (2017)79	Fracture	289 575	0/6/0	1.22 (0.61, 2.47）		7
Overt hypothyroidism
Shiri (2014)49	Carpal tunnel syndrome	71 133	3/1/0		1.44 (1.27, 1.63)	7
Gong et al (2016)37	Gestational diabetes	63 567	0/6/0	1.57 (1.28, 1.93)*	1.62 (1.30, 2.01)*	7
Fang et al (2017)65	Breast cancer	24 571	12/0/0		0.83 (0.64, 1.08)	9
Wang et al (2017)68	Glaucoma	381 695	2/5/4		1.64 (1.27, 2.13)	9
Ning et al (2017)70	Cardiovascular	615 596	0/13/0	1.13 (1.01, 1.26)		9
Nasirkandy et al (2017)71	Preterm birth	2 472 896	0/10/0	1.3 (1.05, 1.61)		9
He et al (2017)73	NAFLD	42 143	4/2/4		1.52 (1.24, 1.87)	10

*The RR or OR calculated according to the sample size.

**†HR.

AMSTAR, Assessment of Multiple Systematic Reviews; CC /CO /CS, case–control/cohort/cross-sectional; CHD, coronary heart disease; CKD, chronic kidney diseases; DPN, diabetic peripheral neuropathy; DN, diabetic nephropathy; DR, diabetic retinopathy; HR, hazard ratio; IGR, intrauterine growth restriction; LGA, large for gestational age; NAFLD, non- alcoholic fatty liver disease; PAD, peripheral arterial disease; RR, relative ratio; SGA, small for gestational age.

Table 6.

Association between SCH/OH and other diseases (showed by WMD/SMD)

Author and year	Diseases	Sample size	No of included CC/CO/CS studies	WMD/SMD (95% CI)	AMSTAR score
Subclinical hypothyroidism
Ye et al (2014)56	SBP	50 147	0/6/14	1.47 mm Hg (0.54 to 2.39 mm Hg), p=0.002	8
	DBP	48 636	0/5/14	0.44 mm Hg (–0.15 to 1.02 mm Hg), p=0.142
Liu et al (2014)42	TC level	40 546	3/0/13	12.17 mg/dL (7.79 to 16.54 mg/dL), p<0.001	10
	LDL-C level	39 131	3/0/12	7.01 mg/dL (3.95 to 10.06 mg/dL), p<0.001
	HDL-C level	40 559	3/0/13	−0.50 mg/dL (−1.90 to 0.89 mg/dL), p=0.481
	TG level	40 420	3/0/11	13.19 mg/dL (4.92 to 21.46 mg/dL), p<0.001
Zhou et al (2014)58	Plasma homocysteine status	926	0/8/0	0.07 (−0.10 to 0.24), p=0.425*	10
Wang et al (2013)75	C-IMT	4551	0/0/9	0.05 mm (0.02 to 0.07 mm), p=0.000	9
Rieben et al (2016)47	MMSE	16 805	0/11/0	ES, 0.01 points difference from baseline (−0.10 to 0.12)	10
Aziz et al (2017)66	C-IMT	543	9 clinical trials	0.44 mm Hg (0.14 to 0.74), p=0.004	10
Overt hypothyroidism
Zhang et al (2016)57	AHI	1389	0/3/2	0.41 (0.21 to 0.61), p<0.001*	5
	Desat time	1414	0/4/2	0.32 (0.12 to 0.51), p<0.01*
	ESS	1383	0/3/2	2.12 (0.89 to 3.35), p<0.001*
	RDI	204	0/3/0	−2.17 (−11.80 to 7.46), p=0.66*
	Sleeping efficiency	444	0/3/0	−1.60 (−5.57 to 2.37), p=0.43*
	Time of sleep with oxygen desaturation under 90%	1416	0/4/2	0.32 (0.12 to 0.51), p=0.001
Zhou et al (2014)58	Plasma homocysteine status	586	2/3/0	0.67 (0.40 to 0.94), p=0.000*	10

*SMD.

AHI, Apnoea–Hypopnoea Index; AMSTAR, Assessment of Multiple Systematic Reviews; CCs/COs/CSs, case–control/cohort/cross-sectional studies; C-IMT, carotid intima-media thickness; DBP, diastolic blood pressure; ESS, Epworth Sleepiness Scale; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; MMSE, Mini Mental State Examination; OH, overt hypothyroidism; RDI, Respiratory Disturbance Index; SBP, systolic blood pressure; SCH, subclinical hypothyroidism; SMD, standardised mean difference; TC, total cholesterol; TG, total triglyceride; WMD, weighted mean difference; ES, Expected Shortfall

Discussion

In view of the growing expansion of MAs that are related to the association between hypothyroidism and other diseases, the methodological quality of these articles was investigated. To the best of our knowledge, this study first used the AMSTAR tool to evaluate the quality of MAs in this field. The AMSTAR tool is the recommended scale for assessing design progress and minimising bias in research.

The methodological quality of the included MAs was moderate to high. Various flaws regarding the information in the included articles were found, and the main problems were the following.

First, to increase scientific credibility and improve research standards, a standard protocol for MAs must be designed. A protocol can make the research process prospective, strict and transparent. In this study, only two MAs^{61 72} provided an a priori study design. Whether the authors had been influenced by the published articles during a certain process in preparing the MAs is challenging to assess. Therefore, more attention should be provided in drafting the protocol.

Second, to avoid inappropriate inclusion or exclusion of articles and minimise selection bias, the AMSTAR scale recommends that at least two independent reviewers should obtain results and data. In this study, eight (15.4%) studies did not follow this recommendation, which may increase selection bias and decrease the quality of the included MAs.

Moreover, the AMSTAR scale suggested that a comprehensive search strategy with a systematic search plan and wide retrieval range should be developed. The former refers to keywords and retrieval types, and the latter indicates that at least two databases should be searched. In addition, item 3 emphasised that supplemental searching is also significantly important. Similarly, to ensure a comprehensive search and decrease selection bias, the fourth AMSTAR scale recommends that the status of publication should not be restricted, which include grey articles and published language. The authors are all willing to obtain positive outcomes, and journals published articles with positive outcomes. However, articles with negative outcomes are not usually published or only published in informal journals. The inclusion of grey articles is significant in decreasing selection bias. For item 3, one⁴⁴ article was not in accordance to this item. However, for item 4, 15.4% (8/52)28 37 45 48 51 57 76 77 of the studies did not consider using published literature, which might result in selection bias.

The AMSTAR scale also suggests that a list of both the included and excluded articles should be provided. The included and excluded articles list can explicitly show the qualities of the selected articles. However, due to restricted page layouts, apart from the Cochrane MAs, most articles only listed the included articles. In this study, 51 MAs included an articles list, and one⁴⁹ article had neither included nor excluded a list of articles.

The sixth item focuses on whether the characteristics of the included studies were provided. In this study, most (96.1%) of the included articles described subject, gender, age and other characteristics of included articles.

Items 7 and 8 recommend the assessment of the scientific quality of the original articles and consideration of the scientific quality when formulating conclusions, which should be based on scientific and cautious research results to provide an objective and reasonable suggestion for clinical professionals and stakeholders. Nineteen (36.5%)31 33 36 37 39 44 50 53 56 57 59 61 62 70–72 77–79 MAs did not assess the scientific quality of the included studies, which might decrease the credibility of the conclusions.

Thus, publication bias should be assessed in every MA according to the Cochrane handbook for the SRs of interventions,⁸⁰ which can be performed by using a certain statistical test method or chart.⁸¹ Among the 52 included articles, seven29 40 50 54 57 62 65 failed to assess publication bias, and all of which were Chinese articles. Another area of concern is the lack of reports on the conflict of interest among the authors of the MAs. Readers should know whether the results of the MAs are influenced by any funders. In our results, 23.1% (12/52)33 50 62 67–69 74–79 of the authors did not declare any conflict of interest.

The conclusions of the included articles were summarised, and the number of randomised controlled trials/CC/CO/CS studies and patients included in each MA and the average AMSTAR score were analysed.

For the association between OH and other diseases, one MA³⁹ has confirmed that breast cancer was significantly associated with OH. This MA included one, two and three CC, CO and CS studies, respectively. The AMSTAR score was moderate. However, another MA⁶⁵ has shown that OH was not considered a risk factor of breast cancer, and such study included 24 571 participants and 12 CC study. Moreover, its AMSTAR score was high. Thus, the credibility of the conclusion is relatively high. One article⁴³ has included 10 CO studies and shown that OH was significantly associated with increased cardiac death, hospitalisation and all-cause mortality in patients with HF. The large sample size and high AMSTAR score increased its credibility. Based on the effect of OH on gestational diabetes, one MA³⁷ has included 225 427 patients in three CO studies and shown that OH was a risk factor of gestational diabetes. The AMSTAR score was moderate.

Six MAs40 41 48 64 71 74 have shown that pregnant women with OH were at high risk for preterm birth. The average number of patients was more than one million, and the average AMSTAR score was 8.8 (range: 8–10). By contrast, one MA⁴⁰ has confirmed that maternal OH might be associated with high and low birth weight and it might reduce the risk of large for gestational age. The AMSTAR score of this article was 8. These data can provide greater power regarding the credibility of the conclusion.

Additionally, OH was significantly associated with stroke,³² left ventricular diastolic function,³³ carpal tunnel syndrome,⁴⁹ obstructive sleep apnea⁵⁷and plasma homocysteine;⁵⁸ the AMSTAR scores of MAs were from moderate to high quality. More high-quality evidence is needed to support these results.

For the association between SCH and other diseases, four MAs36 59 66 75 reported that SCH was associated with a significant increase in carotid intima-media thickness. These MAs have an average of 2420 patients, and an average AMSTAR score of 8.75. Larger study containing more patients will make the result more credible. All three MAs^{30 52 54} have shown that the lack of association between SCH and fractures. A large number of patients and a high AMSTAR score increased the credibility of the conclusion. In addition, one MA⁶⁰ has confirmed that SCH was associated with increased risk for fractures; therefore, more high-quality evidence is needed.

Two^{36 42} high-quality MAs have shown that serum total cholesterol, low-density lipoprotein cholesterol and triglyceride levels were significantly higher in patients with SCH than those with euthyroidism. However, no significant difference was observed in serum high-density lipoprotein cholesterol levels. SCH was associated with a significant decrease in fasting plasma glucose levels but not significantly associated with Body Mass Index.³⁶ The AMSTAR score of this article was high. However, a small sample size decreased its credibility.

The included MAs had different conclusions regarding the role of SCH in blood pressure. One MA³¹has shown that SCH was associated with a significant increase in diastolic blood pressure. However, the other two^{36 56} had a different conclusion. The former included 6 CS studies with 17 323 patients, and the two other MAs had an average of 1.7 CC studies, 1.7 CO studies and 5 CS studies with an average of 25 810 patients, and the average AMSTAR score for both MAs was 8.0. All three MAs have shown that SCH was associated with an increased systolic blood pressure.

Six MAs28 44 50 63 70 77 have assessed the effects of SCH on CHD and cardiovascular mortality. Among these, three^{28 44 77} have evaluated the association between SCH and CHD and confirmed that SCH was not significantly associated with an increased risk for CHD, whereas the other two studies^{50 63} had a conflicting result. Two^{50 70} MAs have shown that SCH was associated with an increased cardiovascular mortality rate from cardiovascular disease, and another two^{28 44} have revealed that SCH was not significantly associated with an increased mortality rate from cardiovascular disease. In addition, three studies^{28 44 50} have concluded that SCH is not significantly associated with an increased risk for all-cause mortality. All the studies had moderate AMSTAR scores. Thus, the association between them should be further validated.

Two MAs^{34 35} have confirmed that SCH was not associated with MetS, and the AMSTAR scores were high. Three MAs^{29 45 47} have investigated the effect of SCH on cognitive function and shown that SCH was not significantly associated with cognitive impairment. A large number of studies and patients and a high AMSTAR score increased the credibility of this conclusion. In addition, two MAs^{38 53} have reported that SCH significantly increased the risk of complications from diabetes.

One MA⁴⁸ has shown that maternal SCH is not significantly related with the occurrence of preterm birth, whereas the other four41 64 71 74 with more patients had a conflicting result, which showed that SCH increased the risk of preterm birth. Two MAs^{51 64} have shown that maternal SCH is significantly associated with the risk for intrauterine growth restriction, and these MAs had a moderate AMSTAR score. Three MAs^{35 64 76} have shown that maternal SCH also had a significant adverse effect on the intelligence of an offspring. However, the sample size of these MAs was small.

Two MAs^{68 69} have indicated that individuals with OH have an increased risk of developing glaucoma. These MAs included 9 CC studies, 4 CO studies and 11 CC studies, and had an average AMSTAR score of 9. One MA⁷² has shown no significant association between non-alcoholic fatty liver disease as well as SCH and OH. However, another study⁷³ has revealed that both individuals with SCH and OH are at higher risk for non-alcoholic fatty liver disease than those with euthyroid; therefore, more high-quality evidence is required.

In summary, when evaluating the association between hypothyroidism and other diseases, the hypothyroidism activity index, follow-up duration, baseline demographic data and clinical characteristics should also be considered in evaluating the role of hypothyroidism in other diseases.

Conclusions

The methodological quality of the included MAs on the association between hypothyroidism and other diseases was moderate to high. MAs with high qualities confirmed that hypothyroidism was significantly associated with cardiovascular diseases, MetS, preterm birth and neonatal outcomes. Consideration of scientific quality when formulating conclusions should be made explicit and more attention should be paid to improving the methodological quality of MAs, increasing their applicability for clinical decision-making.

Strengths and limitations

To the best of our knowledge, this study first assessed the methodological quality of MAs on the association between hypothyroidism and other diseases. The included MAs were randomly selected without restriction and followed rigorous inclusion and exclusion criteria. Our study has several limitations. First, the AMSTAR appraisal process was difficult to implement when the reporting quality was poor. This could be attributed to space restrictions in some journals. Second, we tried our best to make a conversion between OR and RR by extracting raw data from MAs, but some data were not obtained.