Abstract
Methodological quality of meta‐analyses on hypertension treatments can affect treatment decision‐making. The authors conducted a cross‐sectional study to investigate the methodological quality of meta‐analyses on hypertension treatments. One hundred and fifty‐eight meta‐analyses were identified. Overall, methodological quality was unsatisfactory in the following aspects: comprehensive reporting of financial support (1.9%), provision of included and excluded lists of studies (22.8%), inclusion of grey literature (27.2%), and inclusion of protocols (32.9%). The 126 non‐Cochrane meta‐analyses had poor performance on almost all the methodological items. Non‐Cochrane meta‐analyses focused on nonpharmacologic treatments were more likely to consider scientific quality of included studies when making conclusions. The 32 Cochrane meta‐analyses generally had good methodological quality except for comprehensive reporting of the sources of support. These results highlight the need for cautious interpretation of these meta‐analyses, especially among physicians and policy makers when guidelines are formulated. Future meta‐analyses should pay attention to improving these methodological aspects.
Hypertension is one of the major chronic noncommunicable disease (NCD) burdens globally. It is a risk factor for many other NCDs, including stroke, coronary heart disease, congestive heart failure, and renal diseases, contributing to substantial mortality and morbidity in many health systems.1 Hypertension is categorized as primary and secondary according to etiology, with the former accounting for around 90% to 95% of all incident hypertension.2 The global prevalence of primary hypertension is generally high, with an estimated figure of 40% among adults aged 25 years and older.3 It has been estimated that persons with hypertension around the world increased from 600 million in 1980 to 1 billion in 2008.3 Approximately 7.5 million deaths, which account for 12.8% of overall annual mortality, were attributed to hypertension in 2010.4
Identifying, disseminating, and implementing effective interventions for hypertension is a major global health challenge. Rigorous meta‐analyses (MAs) of randomized controlled trials (RCTs) are one the best sources of evidence for the efficacy and effectiveness of various hypertension treatments.5 However, methodological shortcomings of MAs may influence the trustworthiness of synthesis findings and may subsequently mislead decision‐making.5 Previous appraisals of MAs in other healthcare disciplines have shown varying extents of methodological limitations.6, 7, 8, 9, 10, 11, 12, 13, 14 The rigor of MAs on hypertension treatments has not yet been assessed in a comprehensive manner. As results of MAs are key to decision‐making in hypertension clinical guidelines such as the Joint National Committee report,15 the British Hypertension Society guidelines for hypertension management,16 and the joint guideline from the European Society of Hypertension and European Society of Cardiology,17 it is crucial that the rigor of MAs must attain a high standard. There is a need to assess the quality of existing MAs in the field, such that results will shed light on how future MAs should improve in terms of scientific rigor.
In view of this research gap, we conducted the present cross‐sectional study with the objectives of: (1) describing bibliographic characteristics of MAs on hypertension treatments, (2) evaluating the methodological rigor of MAs using a validated tool, and (3) examining the association between bibliographic characteristics and the methodological rigor of MAs.
Methods
Eligibility Criteria
To be eligible, the MA was required to have: (1) synthesized primary studies that included primary hypertension patients with hypertension diagnosed by healthcare professionals (we sampled primary hypertension patients as they account for the majority of the hypertension population)2; (2) focused on the treatment effects of primary hypertension, with no restriction on types of treatment—both pharmacologic and nonpharmacologic treatments were considered eligible; and (3) performed quantitative syntheses of results from primary studies. MAs of primary studies on the treatments of pulmonary hypertension, secondary hypertension, narrative reviews, and network MA were excluded. MAs on preventative interventions of hypertension or treatments for complications of hypertension (eg, stroke) were excluded as well. For those MAs that were updated, the most up‐to‐date version was selected for appraisal.
Sampling of MA
We searched for eligible MAs in the Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE). These two databases were selected as CDSR covered all the Cochrane systematic reviews while DARE collected non‐Cochrane systematic reviews.18 Our search strategy included the keywords of hypertens* or (blood pressure). There were no restrictions on publication date and language. Two reviewers independently evaluated the eligibility of the retrieved citations. Citations were first screened based on titles and abstracts. The full texts of potentially eligible articles were then retrieved and further assessed for eligibility.
Assessing the Methodological Quality of Systematic Reviews (AMSTAR) is a validated tool for assessing the methodological quality of MAs.5 Validation studies have shown that AMSTAR is a reliable tool with good agreement, construct validity, and feasibility.19, 20, 21 In this study, AMSTAR5 was used to evaluate the methodological quality of included MAs. AMSTAR is an 11‐item instrument, with each item scored as “yes,” “no,” “not applicable,” or “can't answer” during the critical appraisal process. Detailed operational guidelines for applying the AMSTAR tool is shown in Appendix S1. Basic characteristics of each included MA were also extracted.
Two reviewers independently assessed the eligibility of literature searched from the databases, extracted information on basic characteristics, and conducted the critical appraisal for each included MA. We planned to resolve discrepancies through discussion or consultation with a third reviewer, but it was deemed unnecessary as there were no major disagreements between the two reviewers.
Statistical Analysis
Data on bibliographical characteristics and methodological quality were described as frequencies with percentages, or medians with range, as appropriate. Among non‐Cochrane MAs, multivariable regression analyses were used to identify potential covariates that are associated with results of each AMSTAR item. Regression analyses were not performed among Cochrane MAs as they were found to be of uniformly high methodological quality.
Multivariable logistic regressions were used in AMSTAR items 1 and 5 to 11 on non‐Cochrane MAs, as they were assessed with a binary option of “yes” or “no.” Adjusted odds ratios (AORs) with 95% confidence intervals (CIs) were calculated. For AMSTAR items 2 to 4, multinominal logistic regressions were used as they allowed answers of “yes,” “no,” or “can't answer.” For these analyses, the adjusted relative risk ratios (RRRs) with 95% CIs were evaluated. Forward selection procedures were used to improve model fitness. Results on covariates that did not show any statistical significance will be provided on request. Regression assumptions of continuous variables (journal impact factor and number of authors) were checked by examining histograms and by confirming normal distribution of the residuals. Model fitness was assessed by: (1) Hosmer‐Lemeshow test for multivariable logistic regression, and (2) likelihood ratio test for multinominal logistic regression. All statistical tests were two‐sided, and a P value <.05 was considered statistically significant, except for Hosmer‐Lemeshow test, among which a P value >.1 indicating good model‐fitting. All statistical analyses were performed using SPSS version 21.0 (IBM, Armonk, NY).
Results
Search and Selection of MAs
We retrieved 727 citations from the databases. After exclusion of irrelevant citations and publications that did not fulfill eligibility criteria, a total of 158 MAs were included (Appendix S2). The Figure shows a flow chart of the screening and selection procedure, with reasons for exclusion.
Figure 1.
Sampling of meta‐analyses on hypertension treatments: flow chart. The literature search yielded 727 citations. Six duplicates or protocols were excluded, leaving 721 records for screening. Four hundred forty‐nine citations were excluded based on title and abstract. The remaining 272 records went through full‐text assessment. Fourteen narrative reviews, 43 systematic reviews without any meta‐analysis, and 57 meta‐analyses assessing the effect of blood pressure on health population were excluded during full‐text assessment. Finally, 158 meta‐analyses were included in this cross‐sectional study.
Basic Characteristics of Included MAs
Table 1 summarizes the basic characteristics of 158 included MAs on primary hypertension treatments. One fifth (20.3%) were Cochrane MAs. The median year of publication was 2010. Only 16 (10.1%) were an update of a previous MA, with the proportion of updated MAs being higher among Cochrane MAs (37.5%) than that of non‐Cochrane MAs (3.2%). The majority of MAs were conducted by teams with a corresponding author from North America (34.8%), Europe (34.2%), and Asia (28.5%). The review team size had a median of four, and only 10.1% did not receive funding support. A total of 101 (63.9%) investigated the effects of pharmacologic treatments. Fifty‐five (34.8%) did not report information on treatment harms. The included MAs searched a median number of three databases; however, only 19.6% searched non‐English databases. Furthermore, 23.4% of MAs were restricted to English publications only.
Table 1.
Basic Characteristics of 158 Included MAs on Hypertension Treatments
| Basic Characteristics | Results |
|---|---|
| Cochrane review | 32 (20.3) |
| Non‐Cochrane review | 126 (79.7) |
| An update of a previous MA | 16 (10.1) |
| An update of a previous MA among Cochrane review | 12 (37.5) |
| An update of a previous MA among non‐Cochrane review | 4 (3.2) |
| Median year of publication (range) | 2010 (1994–2014) |
| Median impact factor of the journal for which the MA was published (range) | 4.4 (0.0–30.8) |
| Median number of review authors (range) | 4 (1–16) |
| Location of corresponding author | |
| North America | 55 (34.8) |
| Europe | 54 (34.2) |
| Asia | 45 (28.5) |
| Other regions | 4 (2.5) |
| Funding location of the MA | |
| North America | 38 (24.1) |
| Europe | 33 (20.9) |
| Asia | 29 (18.4) |
| Other regions | 7 (4.4) |
| Not reported | 35 (22.1) |
| No funding support | 16 (10.1) |
| Number of MAs that searched international databases | 157 (99.4) |
| Median number of international databases searched (range) | 3 (1–64) |
| Number of MAs that searched non‐English databases | 31 (19.6) |
| Reporting of coverage year of search | |
| Yes, reported both starting and ending years | 92 (58.2) |
| Partially, only reported starting years | 30 (19.0) |
| Not mentioned | 36 (22.8) |
| Search terms reported for one or more electronic databases | |
| Topics/free text/keywords/MeSH | 85 (53.8) |
| Full Boolean | 54 (34.2) |
| No search terms provided | 19 (12.0) |
| Eligibility criteria based on language of publication | |
| Included English publications only | 37 (23.4) |
| English and languages other than English | 69 (43.7) |
| Language criteria not reported | 52 (32.9) |
| Median number of included studies in each MA (range) | 4 (2–16) |
| Number of MAs that reported harm of the intervention | 103 (65.2) |
| Number of MAs that included a PRISMA‐like flow diagram | 94 (59.5) |
Abbreviations: MA, meta‐analysis; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta‐analyses; RCT, randomized controlled trial. Values are expressed as number (percentage) unless otherwise indicated.
Methodological Quality
Overall, the methodological quality of included MAs was low, especially in the following criteria: (1) not reporting sources of support for both the MA and included primary studies (98.1%), (2) not reporting lists for both included and excluded primary studies (77.2%), (3) explicit exclusion of grey literature or describing no related information (72.8%), and (4) not providing protocol of the MAs (67.1%) (Table 2). Since Cochrane MAs are expected to have better methodological quality as a result of tight editorial policies, we reported the findings from Cochrane and non‐Cochrane MAs separately (Table 2).
Table 2.
Methodological Quality of the Included 158 MAs on Hypertension Treatments
| Individual AMSTAR Items | Yes (%) | No (%) | Can't Answer (%) | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Total (158) | Cochrane MA (32) | Non‐Cochrane MA (126) | Total (158) | Cochrane MA (32) | Non‐Cochrane MA (126) | Total (158) | Cochrane MA (32) | Non‐Cochrane MA (126) | |
| Item 1 | 52 (32.9) | 32 (100.0) | 20 (15.9) | 106 (67.1) | 0 (0.0) | 106 (84.1) | NA | NA | NA |
| Item 2 | 98 (62.0) | 31 (96.9) | 67 (53.2) | 33 (20.9) | 1 (3.1) | 32 (25.4) | 27 (17.1) | 0 (0.0) | 27 (21.4) |
| Item 3 | 95 (60.1) | 31 (96.9) | 64 (50.8) | 61 (38.6) | 1 (3.1) | 60 (47.6) | 2 (1.3) | 0 (0.0) | 2 (1.6) |
| Item 4 | 43 (27.2) | 17 (53.1) | 26 (20.6) | 20 (12.7) | 0 (0.0) | 20 (15.9) | 95 (60.1) | 15 (46.9) | 80 (63.5) |
| Item 5 | 36 (22.8) | 32 (100.0) | 4 (3.2) | 122 (77.2) | 0 (0.0) | 122 (96.8) | NA | NA | NA |
| Item 6 | 112 (70.9) | 32 (100.0) | 80 (63.5) | 46 (29.1) | 0 (0.0) | 46 (36.5) | NA | NA | NA |
| Item 7 | 109 (69.0) | 32 (100.0) | 77 (61.1) | 49 (31.0) | 0 (0.0) | 49 (38.9) | NA | NA | NA |
| Item 8 | 64 (40.5) | 18 (56.2) | 46 (36.5) | 94 (59.5) | 14 (43.8) | 80 (63.5) | NA | NA | NA |
| Item 9 | 84 (53.2) | 27 (84.4) | 57 (45.2) | 74 (46.8) | 5 (15.6) | 69 (54.8) | NA | NA | NA |
| Item 10 | 70 (44.3) | 16 (50.0) | 54 (48.8) | 88 (55.7) | 16 (50.0) | 72 (51.2) | NA | NA | NA |
| Item 11 | 3 (1.9) | 0 (0.0) | 3 (2.4) | 155 (98.1) | 32 (100.0) | 123 (97.6) | NA | NA | NA |
Abbreviations: AMSTAR, Assessing the Methodological Quality of Systematic Reviews; MAs, meta‐analyses; NA, not applicable. Individual AMSTAR items: 1: Was the protocol provided? 2: Was there duplicate study selection and data extraction? 3: Was a comprehensive literature search performed? 4: Was the status of publication used as an inclusion criterion (ie, grey literature would be included, if located)? 5: Was a list of studies (included and excluded) provided? 6: Were the characteristics of the included studies provided? 7: Was the scientific quality of the included studies assessed and documented? 8: Was the scientific quality of the included studies used appropriately in formulating conclusions? 9: Were the methods used to combine the findings of studies appropriate? 10: Was the likelihood of publication bias assessed? 11: Were the sources of support for both the systematic review and the included primary studies reported?
The 32 Cochrane MAs had good performance in the majority of the AMSTAR items (Table 2), except for reporting sources of support for both the MAs and included primary studies (0.0%), appropriately considering the scientific quality of included study during conclusion (56.2%) and evaluating the possibility of publication bias (50.0%). In addition, 46.9% of the Cochrane MAs did not report information on whether grey literature was included.
On the other hand, the 126 non‐Cochrane MAs generally had poor methodological quality (Table 2). Only 2.4% provided sources of support for both MA and included primary studies, 3.2% provided a list of included and excluded studies, 15.9% provided a protocol, and 20.6% used the status of publication as an inclusion criterion. Even for the three AMSTAR items that had relatively better performance, only 63.5% provided characteristics of the included primary studies, 61.1% assessed and documented the scientific quality of included studies, and 53.2% performed duplicated literature selection and data extraction.
Association Between Bibliographical Characteristics and Methodological Quality: Regression Analyses on Non‐Cochrane MAs
In regression analyses (Table 3) investigating the relationship between bibliographical characteristics and methodological quality among non‐Cochrane MAs, the following associations were found to be statistically significant. Updated MAs were associated with provision of the protocol (AOR, 20.82; 95% CI, 1.70–255.42) and MAs with more authors were less likely to assess the likelihood of publication bias (AOR, 0.69; 95% CI, 0.55–0.87). Recently published MAs were more likely to provide characteristics of the included studies (AOR, 1.12; 95% CI, 1.01–1.24), to perform duplicate study selection and data extraction (adjusted RRR, 1.23; 95% CI, 1.08–1.40), to use appropriate methods to combine the findings (AOR, 1.15; 95% CI, 1.04–1.27), and to assess the likelihood of publication bias (AOR, 1.28; 95% CI, 1.13–1.45).
Table 3.
Association Between Basic Characteristics and Methodological Quality of Non‐Cochrane MAs on Hypertension Treatments: Regression Analyses
| AMSTAR Item (Dependent Variable) | Predictors | Adjusted Results (95% CI) | P Value |
|---|---|---|---|
| 1. Was the protocol provided?a | Updated MA | OR: 20.82 (1.70–255.42) | .018 |
| 2. Was there duplicate study selection and data extraction?b | Nonpharmacologic treatmentc | RRR: 0.07 (0.01–0.36) | .002 |
| Published more recently | RRR: 1.23 (1.08–1.40) | .002 | |
| MA with funding support | RRR: 12.17 (1.64–90.63) | .015 | |
| 6. Were the characteristics of the included studies provided?a | Nonpharmacologic treatmentc | OR: 0.06 (0.02–0.22) | <.001 |
| Published more recently | OR: 1.12 (1.01–1.24) | .045 | |
| 7. Was the scientific quality of the included studies assessed and documented?a | Corresponding author from Europed | OR: 0.22 (0.07–0.74) | .015 |
| 8. Was the scientific quality of the included studies used appropriately in formulating conclusions?a | Corresponding author from Europed | OR: 0.18 (0.05–0.61) | .006 |
| Nonpharmacologic treatmentc | OR: 5.45 (1.75–16.96) | .003 | |
| 9. Were the methods used to combine the findings of studies appropriate?a | Published more recently | OR: 1.15 (1.04–1.27) | .005 |
| 10. Was the likelihood of publication bias assessed?a | More review authors | OR: 0.69 (0.55–0.87) | .002 |
| Published more recently | OR: 1.28 (1.13–1.45) | <.001 |
Abbreviations: AMSTAR, Assessing the Methodological Quality of Systematic Reviews; CI, confidence interval; MA, meta‐analysis; OR, odds ratio; RRR, relative risk ratio; SR, systematic review. a P value of Hosmer and Lemeshow tests were >.1, indicating good model fit for the logistic regression analyses. b P value of likelihood ratio test was <.05, indicating good model fit for the multinominal regression analysis. cPharmacologic treatment was used as a reference. dCorresponding author from Asia was used as a reference.
MAs on studies of nonpharmacologic treatment performed less well in conducting duplicate study selection and data extraction (adjusted RRR, 0.07; 95% CI, 0.01–0.36) and in providing both included and excluded list of studies (AOR, 0.06; 95% CI, 0.02–0.22). On the other hand, these MAs did better in considering scientific quality of included studies when formulating conclusions (AOR, 5.45; 95% CI, 1.75–16.96). When compared with teams with corresponding authors from Asia, MAs with European corresponding authors were less likely to assess the scientific quality of included studies (AOR, 0.22; 95% CI, 0.07–0.74) and less likely to properly consider the appraisal results when conclusions were drawn (AOR, 0.18; 95% CI, 0.05–0.61). MAs with funding resources were associated with better performance in conducting duplicate study selection and data extraction (adjusted RRR, 12.17; 95% CI, 1.64–90.63). Good model fits were observed in all regression analyses.
Discussion
This study summarized the basic characteristics and assessed the methodological quality of all MAs on hypertension treatments indexed in CDSR and DARE. A total of 158 MAs were included, with one fifth being Cochrane MAs. The majority of the MAs were financially supported (45.0%) and conducted by research teams from Europe and North America (69.0%). Methodologically, Cochrane MAs performed well in half of the AMSTAR items (1–3, 5–7), while there is an urgent need of improvement among non‐Cochrane MAs in most AMSTAR items.
Only a very small number of non‐Cochrane MAs provided a list of all included and excluded studies and reported funding support for all primary studies as well as the MAs itself. Indeed, none of the Cochrane MAs comprehensively described sources of support. Without this information, readers cannot judge whether exclusion of certain primary studies could introduce bias or whether financial interests might have influenced the conclusion. Such underreporting may be attributed to word limits of many journals,22 especially for older MAs, but with the availability of online space, this should not be regarded as an excuse for not supplying these materials. Protocols are unavailable in 84.1% of the non‐Cochrane reviews. This lack of transparency makes detection of reporting bias difficult at the MA level, and may lead to wasteful duplication of reviews. Authors of MAs are encouraged to prospectively register their protocol in PROSPERO.23
Another threat to the trustworthiness of results is the lack of considerations on publication bias and scientific quality of primary studies when conclusions were drawn, which is commonly observed in both Cochrane and non‐Cochrane MAs. This risk of overoptimism on treatment effect is compounded by uncertainties on whether included MAs would include grey literature (60.1%) and not searching non‐English literature (80.4%), as both publication sources are known to be associated with negative results.24, 25, 26 Thus, future MAs are suggested to consider the likelihood of publication bias so as to inform readers about the potential possibility of overestimating on hypertension treatment effects.27 Appropriate strategies should be adopted to reduce the impact of publication bias when it is detected.28 Finally, since only 10.1% were an update of a previous MA, conclusions may not have considered the latest findings.29 This problem is particularly acute among non‐Cochrane MAs, of which only 3.2% were updates. This phenomenon may be explained by a lack of incentive among authors, as well as insufficient guidance on when and how to perform an update.30 Authors of MAs and journal editors are suggested to put efforts in keeping their MAs up‐to‐date, while addressing previous methodological shortcomings.31
It is observed that MAs on pharmacologic treatments performed better in conducting duplicate study selection and data extraction, and in reporting the characteristics of included studies when compared with MAs on nonpharmacologic treatment. However, MAs on pharmacologic treatments did worse in appropriately considering the scientific quality of included studies to formulate conclusions. As a higher proportion of MAs on pharmacologic treatments might have received commercial support, the research team may have more resources in performing labor‐intensive tasks, while putting less emphasis on considering critical appraisal results when drawing conclusions. These observations echo previous studies reporting links between industry sponsorship and favorable results supporting the treatment.32, 33
Methodological quality of MAs on the treatments of primary hypertension showed better performance in the majority of the AMSTAR items than that of MAs in orthodontics34 and nursing,35 except for providing protocol and stating sources of supports. On the other hand, our included MAs had worse performance in the majority of AMSTAR items than MAs on Chinese herbal medicine13 and chronic obstructive pulmonary disease.14
In the present study, we did not calculate the total AMSTAR score for each systematic review, as there is no recommendation criterion on calculating the overall AMSTAR score.5 Furthermore, it is difficult to justify the weight of each item if scores are given to different items.36 We believe that simply calculating the total score cannot reflect the true methodological quality of a systematic review, until a standardized weighting system for each item is established. Thus, we chose to report results of each AMSTAR item comprehensively so as to inform researchers on the methodological aspects they should pay more attention to in the future.
Conclusions
The methodological quality of MAs on hypertension treatment is unsatisfactory, although Cochrane MAs showed a higher degree of scientific rigor. Our findings suggest that users of evidence presented in hypertension MAs need to be cautious of the possible impact of treatment effects attributable to reporting bias, publication bias, industry influence, and lack of considerations on scientific quality of primary studies when conclusions are drawn. While the need for improvement is imminent, we have also observed rigor in performing duplicate study selection and data extraction, providing characteristics of the included studies and using appropriate statistical methods to combine the results among recently published MAs. This may imply awareness of the importance of rigor among systematic reviewers in the field. Editors and authors are recommended to consult the Cochrane Handbook for Systematic Reviews of Interventions36 for methodological guidance and to follow the PRISMA statement37 when writing up or evaluating quality of reporting.
Disclosure
The authors declare no conflict of interest.
Supporting information
Appendix S1. Detailed operational guide for applying the AMSTAR tool.
Appendix S2. Reference list of the 158 included meta‐analyses.
Acknowledgment
This research received no specific financial support.
J Clin Hypertens (Greenwich). 2017;19:137–142. 10.1111/jch.12889. © 2016 Wiley Periodicals, Inc.
References
- 1. Messerli FH, Williams B, Ritz E. Essential hypertension. Lancet. 2007;370:591–603. [DOI] [PubMed] [Google Scholar]
- 2. Carretero OA, Oparil S. Essential hypertension: part I: definition and Etiology. Circulation. 2000;101:329–335. [DOI] [PubMed] [Google Scholar]
- 3. Danaei G, Finucane MM, Lin JK, et al. National, regional, and global trends in systolic blood pressure since 1980: systematic analysis of health examination surveys and epidemiological studies with 786 country‐years and 5.4 million participants. Lancet. 2011;377:568–577. [DOI] [PubMed] [Google Scholar]
- 4. World Health Organization . Global Status Report on Noncommunicable Diseases 2010. Geneva: World Health Organization; 2011. [Google Scholar]
- 5. Shea B, Grimshaw J, Wells G, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kitsiou S, Paré G, Jaana M. Systematic reviews and meta‐analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. J Med Internet Res. 2013;15:e150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Momeni A, Lee GK, Talley JR. The quality of systematic reviews in hand surgery: an analysis using AMSTAR. Plast Reconstr Surg. 2013;131:831–837. [DOI] [PubMed] [Google Scholar]
- 8. Johnson BT, MacDonald HV, Bruneau ML, et al. Methodological quality of meta‐analyses on the blood pressure response to exercise: a review. J Hypertens. 2014;32:706–723. [DOI] [PubMed] [Google Scholar]
- 9. Remschmidt C, Wichmann O, Harder T. Methodological quality of systematic reviews on influenza vaccination. Vaccine. 2014;32:1678–1684. [DOI] [PubMed] [Google Scholar]
- 10. Sheikh L, Johnston S, Thangaratinam S, et al. A review of the methodological features of systematic reviews in maternal medicine. BMC Med. 2007;5:10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Martel G, Duhaime S, Barkun JS, et al. The quality of research synthesis in surgery: the case of laparoscopic surgery for colorectal cancer. Syst Rev. 2012;1:14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Fleming PS, Seehra J, Polychronopoulou A, et al. Cochrane and non‐Cochrane systematic reviews in leading orthodontic journals: a quality paradigm? Eur J Orthod. 2013;35:244–248. [DOI] [PubMed] [Google Scholar]
- 13. Chung VCH, Ho RST, Wu X, et al. Are meta‐analyses of Chinese herbal medicine trials trustworthy and clinically applicable? A cross‐sectional study. J Ethnopharmacol. 2015;162:47–54. [DOI] [PubMed] [Google Scholar]
- 14. Ho RS, Wu X, Yuan J, et al. Methodological quality of meta‐analyses on treatments for chronic obstructive pulmonary disease: a cross‐sectional study using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool. NPJ Prim Care Respir Med. 2015;25:14102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Chobanian AV, Bakris GL, Black HR, et al. The seventh report of the joint national committee on prevention, detection, evaluation, and treatment of high blood pressure: the jnc 7 report. JAMA. 2003;289:2560–2571. [DOI] [PubMed] [Google Scholar]
- 16. Williams B, Poulter NR, Brown MJ, et al. British Hypertension Society guidelines for hypertension management 2004 (BHS‐IV): summary. BMJ. 2004;328:634–640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Mancia G, Fagard R, Narkiewicz K, et al. 2013 ESH/ESC guidelines for the management of arterial hypertension. The Task Force for the management of arterial hypertension of the European Society of Hypertension (ESH) and of the European Society of Cardiology (ESC). Eur Heart J. 2013;34:2159–2219. [DOI] [PubMed] [Google Scholar]
- 18. Cochrane Community (beta) . Database of Abstracts of Reviews of Effects (DARE), 2015.
- 19. Shea BJ, Bouter LM, Peterson J, et al. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One. 2007;2:e1350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–1020. [DOI] [PubMed] [Google Scholar]
- 21. Pieper D, Buechter RB, Li L, et al. Systematic review found AMSTAR, but not R(evised)‐AMSTAR, to have good measurement properties. J Clin Epidemiol. 2015;68:574–583. [DOI] [PubMed] [Google Scholar]
- 22. Deeks JJ. Word limits best explain failings of industry supported meta‐analyses. BMJ. 2006;333:1021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Booth A, Clarke M, Dooley G, et al. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst Rev. 2012;1:2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Easterbrook PJ, Gopalan R, Berlin JA, et al. Publication bias in clinical research. Lancet. 1991;337:867–872. [DOI] [PubMed] [Google Scholar]
- 25. Hopewell S, Loudon K, Clarke MJ, et al. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev. 2009;(1):MR000006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Egger M, Zellweger‐Zähner T, Schneider M, et al. Language bias in randomised controlled trials published in English and German. Lancet. 1997;350:326–329. [DOI] [PubMed] [Google Scholar]
- 27. Dwan K, Altman DG, Arnaiz JA, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One. 2008;3:e3081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mueller K, Meerpohl J, Briel M, et al. Detecting, quantifying and adjusting for publication bias in meta‐analyses: protocol of a systematic review on methods. Syst Rev. 2013;2:60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Moher D, Tsertsvadze A. Systematic reviews: when is an update an update? Lancet. 2006;367:881–883. [DOI] [PubMed] [Google Scholar]
- 30. Moher D, Tsertsvadze A, Tricco AC, et al. A systematic review identified few methods and strategies describing when and how to update systematic reviews. J Clin Epidemiol. 2007;60:1095.e1091–1095.e1011. [DOI] [PubMed] [Google Scholar]
- 31. Shea B, Boers M, Grimshaw J, et al. Does updating improve the methodological and reporting quality of systematic reviews? BMC Med Res Methodol. 2006;6:27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Lexchin J, Bero LA, Djulbegovic B, et al. Pharmaceutical industry sponsorship and research outcome and quality: systematic review, Vol. 326, 2003. [DOI] [PMC free article] [PubMed]
- 33. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta‐analyses and other meta‐analyses of the same drugs: systematic review. BMJ. 2006;333:782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Evaluation of methodology and quality characteristics of systematic reviews in orthodontics. Orthod Craniofac Res. 2011;14:116–137. [DOI] [PubMed] [Google Scholar]
- 35. Seo H‐J, Kim KU. Quality assessment of systematic reviews or meta‐analyses of nursing interventions conducted by Korean reviewers. BMC Med Res Methodol. 2012;12:129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration 2011. http://handbook.cochrane.org/. [Google Scholar]
- 37. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta‐analyses: the PRISMA statement. BMJ. 2009;339:b2535. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix S1. Detailed operational guide for applying the AMSTAR tool.
Appendix S2. Reference list of the 158 included meta‐analyses.