Abstract
Aims
The aim of this review was to evaluate the reporting and methodological quality of systematic reviews and meta‐analyses (SRs/MAs) on nursing interventions in patients with chronic obstructive pulmonary disease (COPD) and to determine potential factors that predict high quality.
Design
The review is a quantitative systematic review.
Data Sources
PubMed, Embase and the Cochrane Database of Systematic Reviews.
Review Methods
A comprehensive literature search was conducted in three databases for SRs/MAs published up to 6 May 2020. The PRISMA statement and AMSTAR checklist were used to evaluate the reporting and methodological quality.
Results
A total of 130 articles published between 1996–2020 from 69 journals were included in this review. Multivariate regression analyses demonstrated that the following factors were related to the higher reporting quality of included articles: having a protocol or registration and being published on the Cochrane Database of Systematic Reviews. Systematic reviews including meta‐analyses, number of authors >5, number of pages and having protocol or registration were related to higher methodological quality. A strong linear correlation (r = 0.860) was detected between the scores of PRISMA and AMSTAR.
Conclusion
A significant number of systematic reviews and meta‐analyses on nursing interventions in patients with COPD show suboptimal reporting and poor methodology quality. The use of PRISMA and AMSTAR guidelines in conducting, reading, reviewing and editing systematic reviews and meta‐analyses is recommended to improve the quality of systematic reviews and meta‐analyses.
Impact
The findings of this review can provide references for health workers and health policy makers to evaluate and apply evidence‐based knowledge. Additionally, such high‐quality systematic reviews/meta‐analyses can guide medical and health practice.
Keywords: chronic obstructive pulmonary disease, nursing interventions, quality assessment, systematic review/meta‐analyses
1. INTRODUCTION
Evidence‐based medicine (EBM) has spread with amazing speed in the past twenty years (Eddy, 2005). With the rise of EBM, systematic reviews/meta‐analyses (SRs/MAs) are a soaring trend in various specialties and subspecialties in medicine and are generally considered the highest source of evidence (Manchikanti et al., 2009; Sim et al., 2001). In spite of the superiority of SRs and MAs, some performed different quality because of inappropriate design, conducting or reporting. Thus, it is crucial for high quality of reporting and methodology to make sure trustworthy, transparent and accurate explanation of evidence.
The treatment goal of chronic disease patients is not only to prevent the disease deterioration, but also to alleviate the physical or mental pain of patients and improve their quality of life. Clinically, the use of drugs to eliminate diseases and long‐term bed rest cannot meet the needs of patients. Nursing intervention can shorten the length of hospital stay and improve their negative emotions such as anxiety or depression. It is often delivered, coordinated or led by nurses. Nurses play an important role in improving living quality for people with chronic disease. SRs and MAs can provide valuable and credible evidence of nursing interventions. And it is significant to help nurses, clinicians, research workers and policy makers to develop recommendations, guidelines and clinical decision‐making.
Chronic obstructive pulmonary disease is a multi‐factorial progressive chronic lung disease which causes enormous physical and social burdens (McCarthy et al., 2015). Globally, COPD is a main cause of morbidity (Gendron et al., 2018). Worldwide, it is estimated that 210 million people suffer from COPD and COPD will be the third leading cause of death by the year 2030 (Word health Organization, 2008). Treatment therapies for COPD include pharmacological and non‐pharmacological interventions (Blackstock & Webster, 2007). Regarding the latter, nursing interventions like psychological therapies, smoking cessation, supplemental oxygen, pulmonary rehabilitation and palliative care are considered important.
2. BACKGROUND
To fully realize the application value of SRs/MAs and ensure the validity of evidence, corresponding checklists and guidelines of assessment of quality have emerged (Moher et al., 1999; Shea et al., 2009). The former Quality Of Reporting Of Meta‐analysis (QUOROM) guideline (Liberati et al., 2009) have developed into Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA), which focused on advancing reporting quality. Assessment of Multiple Systematic Reviews (AMSTAR) is the most common tool for methodology assessment (Shea et al., 2007). Some studies have assessed the quality of SRs/MAs in various medical domains using PRISMA and AMSTAR tools and have stated clearly that SRs/MAs have varying defects of weaknesses and that quality needs to be further improved (Cullis et al., 2017; H. Liu et al., 2019; Sun et al., 2019).
So far, there have been 130 SRs/MAs nursing interventions for COPD patients. However, no studies have been conducted to evaluate the reporting and methodological quality of these SRs/MAs. We hope to provide references for readers, authors, reviewers and journal editors and to further explore the relevant factors to improve the reporting and methodological quality of SRs/MAs in this field.
3. THE REVIEW
3.1. Aims
The aim of this review was to evaluate the reporting and methodological quality of systematic reviews and meta‐analyses on nursing interventions in patients with chronic obstructive pulmonary disease (COPD) and to determine potential factors that predict high quality.
3.2. Design
The review is a quantitative systematic review. It assessed the reporting and methodological quality of SRs/MAs on nursing interventions for COPD patients using the PRISMA statement and AMSTAR checklist.
3.3. Inclusion and exclusion criteria
Systematic reviews and meta‐analyses that met the following criteria were included: (a) articles being identified as a SR or MA; (b) articles that evaluated the effect of nursing intervention (nursing interventions like psychological therapies, smoking cessation, supplemental oxygen, pulmonary rehabilitation and palliative care) on outcomes for COPD patients; and (c) articles published in English. The following were excluded (a) articles being identified as review protocol, scoping review, traditional literature review, abstracts, conference proceedings, letters to editors and evidence‐based commentaries; (b) articles of non‐nursing interventions for COPD patients, such as pharmacological therapies, diagnosis and prognosis; and (c) articles not available in full text.
3.4. Search methods
A comprehensive literature search was conducted in PubMed, Embase and the Cochrane Database of Systematic Reviews for SRs/MAs published up to 6 May 2020. Searches were limited to reports published in English. The search strategy included the use of the following search items in the title/abstract related to: (“Chronic obstructive pulmonary disease” or “COPD”) and (“systematic review” or “meta‐analyses” or “overview” or “systematic literature review” or “meta‐analysis” or “review” or “synthesize review” or “integrated review” or “comprehensive review”) and (“nursing” or “nursing intervention” or “nurse” or “care”). Apart from articles on Google Scholar and Baidu Scholar, we also screened the reference list that was manually reviewed from selected articles to identify additional relevant articles that met the inclusion criteria. The search strategies of three database are presented in [Link], [Link], [Link].
3.5. Search outcomes
The initial search identified 4,494 articles and a manual search identified 11 additional articles for potential inclusion. Finally, after removing duplicate articles, reviewing titles and abstracts and reviewing the full text and 130 SRs/MAs were included for the assessment and analysis in this review. Details of the search screening process are presented in Figure 1. For the screening process, two investigators independently screened the titles and abstracts of all the retrieved articles after removing duplication. Full‐text articles were then reviewed and retrieved independently by two investigators to select potentially eligible articles.
FIGURE 1.
Flow chart of article screening and selection process
3.6. Risk of bias assessment
The PRISMA statement was used to evaluate the reporting quality, which has a list of 27 items. For each of the items, yes = 1, partial yes = 0.5, no or cannot answer = 0 and not applicable = 1 (i.e., items 16, 21 and 23 only applied to MAs). Therefore, every study had an overall PRISMA score ranging from 0–27, with a higher score reflecting better reporting quality.
The AMSTAR checklist was used to evaluate the methodological quality, which has a list of 11 items. For each of the items, yes = 1, partial yes = 0.5, no or cannot answer = 0 and not applicable = 1 (i.e., items 9 only applied to MAs). Therefore, every study had an overall AMSTAR score ranged from 0–11, with a higher reflecting better methodological quality. Each article was separately and independently assessed by two investigators. Kappa values for the inter‐observer agreement between the two investigators were calculated.
3.7. Data abstraction
All data were independently extracted by two investigators. Unobtainable data were not considered, and discrepancies were resolved by two investigators. The data extraction form included the following details: year of publication, country of first author, author origin, number of authors, first author's affiliation, name and type of the journal, number of affiliations, type of article, whether a randomized control trial was identified, number of times cited, followed PRISMA guideline, number of included studies, impact factor (IF), funding support, international collaborative authorship, number of pages, protocol or registration, journal source of SCI (Science Citation Index) and details of being published in Cochrane Database of Systematic Review.
3.8. Data synthesis
Data entries and statistical analyses were conducted by SPSS 23.0 and were two‐sided. Data on each item were presented as counts and percentages. Frequency (percentage) are used for categorical variables and descriptive statistics are presented as mean (SD) for continuous variables. χ 2 test or Fisher's exact test was used to evaluate the differences in categorical data. The independent Student's t‐test was used to assess the differences in continuous data and a one‐way analysis of variance (ANOVA) for multifactor variables using the SNK post hoc test.
The PRISMA score and AMSTAR score were both divided into the superior and inferior quality groups by a cut‐off value of the 75th percentile of the respective ranges (Xia et al., 2017). Comparisons were made between manuscripts published before (1996–2008) and after (2009–2020) the introduction of the PRISMA statement and before (1996–2006) and after (2007–2020) the introduction of the AMSTAR checklist. Univariate and multivariate logistic regression analyses were used to explore potential factors of article qualities. Factors found to be statistically significant (p < .1) were then entered into the multivariate logistic regression analysis. The relationship between reporting and methodological quality of SRs/MAs on nursing interventions in patients with COPD was assessed using Spearman's rank correlation coefficient (r S). The significance level was set at 0.05.
4. RESULTS
4.1. General characteristics of included SRs/MAs
The 130 SRs/MAs were selected from 69 journals and published between 1996–2020. The largest number of SRs/MAs came from Europe (N = 67, 51.5%), with the United Kingdom accounting for the largest proportion of SRs/MAs (N = 27, 50.3%). A large portion (N = 119, 91.8%) of the publishing journals belong to SCI. Almost half of included articles (N = 67, 56.9%) conduct meta‐analyses. In addition, only 17 (13.1%) SRs/MAs were published in the Cochrane Database of Systematic Reviews. The main characteristics of the included articles are summarized in Table 1. The overall kappa statistic for PRISMA and AMSTAR results was 0.756 and 0.824 (p < .05), respectively, which indicates a good level of agreement between scorers. All disagreements were resolved by consensus between the two investigators.
TABLE 1.
The main characteristics of included articles
| Characteristic | N (%) | PRISMA | AMSTAR | ||||
|---|---|---|---|---|---|---|---|
| (SD) | F/t | p | (SD) | F/t | p | ||
| Type of article | |||||||
| Systematic reviews only | 56 (43.1) | 19.857 (3.640) | −5.216 | <.001 | 6.518 (1.689) | −6.902 | <.001 |
| Systematic reviews only | 74 (56.9) | 22.878 (2.704) | 8.466 (1.518) | ||||
| Origin region of first author | |||||||
| Systematic reviews only | 15 (11.5) | 22.100 (3.001) | 0.754 | .557 | 7.867 (1.827) | 0.125 | .973 |
| Systematic reviews only | 67 (51.5) | 21.724 (3.503) | 7.642 (1.798) | ||||
| Systematic reviews only | 21 (16.2) | 20.762 (3.659) | 7.476 (1.997) | ||||
| Systematic reviews only | 26 (20.0) | 21.712 (3.542) | 7.596 (2.045) | ||||
| Systematic reviews only | 1 (0.8) | 17.500 | 7.000 | ||||
| Number of author | |||||||
| Systematic reviews only | 64 (49.2) | 20.719 (3.852) | −2.839 | .005 | 7.164 (2.026) | −2.872 | .005 |
| Systematic reviews only | 66 (50.8) | 21.570 (3.385) | 8.076 (1.572) | ||||
| International collaborative authorship | |||||||
| Systematic reviews only | 102 (78.5) | 21.426 (3.463) | −0.943 | .347 | 7.544 (1.817) | −0.969 | .335 |
| Systematic reviews only | 28 (21.5) | 22.125 (3.506) | 7.929 (2.013) | ||||
| Number of author's affiliation | |||||||
| Systematic reviews only | 58 (44.6) | 20.819 (3.890) | −2.271 | .025 | 7.017 (1.919) | −3.498 | .001 |
| Systematic reviews only | 72 (55.4) | 22.188 (2.982) | 8.118 (1.667) | ||||
| Affiliation of first author | |||||||
| Systematic reviews only | 26 (20.0) | 20.942 (4.126) | 0.591 | .555 | 7.519 (1.962) | 0.054 | .947 |
| Systematic reviews only | 83 (63.8) | 21.681 (3.416) | 7.651 (1.821) | ||||
| Systematic reviews only | 21 (16.2) | 21.952 (2.801) | 7.667 (1.971) | ||||
| Journal type of published article | |||||||
| Systematic reviews only | 110 (81.6) | 21.682 (3.411) | 0.807 | .421 | 7.731 (1.785) | 1.516 | .132 |
| Systematic reviews only | 20 (15.4) | 21.000 (3.825) | 7.050 (2.188) | ||||
| Number of included studies | |||||||
| Systematic reviews only | 42 (32.3) | 21.750 (3.454) | 0.392 | .696 | 7.845 (1.806) | 0.924 | .357 |
| Systematic reviews only | 88 (67.7) | 21.494 (3.495) | 7.523 (1.886) | ||||
| RCT identified | |||||||
| Systematic reviews only | 68 (52.3) | 20.559 (3.894) | −3.738 | <.001 | 7.000 (1.947) | −4.288 | <.001 |
| Systematic reviews only | 62 (47.7) | 22.694 (2.529) | 8.135 (1.494) | ||||
| Followed PRISMA guideline | |||||||
| Systematic reviews only | 88 (67.7) | 20.938 (3.749) | −3.69 | <.001 | 7.386 (2.022) | −2.485 | .014 |
| Systematic reviews only | 42 (32.3) | 22.917 (2.314) | 8.131 (1.348) | ||||
| Protocol and Registration | |||||||
| Systematic reviews only | 90 (69.2) | 20.350 (3.370) | −9.100 | <.001 | 6.928 (1.650) | −7.769 | <.001 |
| Systematic reviews only | 40 (30.8) | 24.338 (1.623) | 9.200 (1.250) | ||||
| Journal source of SCI | |||||||
| Systematic reviews only | 11 (8.5) | 18.136 (4.190) | −3.592 | <.001 | 5.909 (2.200) | −3.324 | 0.001 |
| Systematic reviews only | 119 (91.5) | 21.895 (3.236) | 7.786 (1.752) | ||||
| IF | |||||||
| Systematic reviews only | 59 (49.6) | 20.805 (3.281) | −3.850 | <.001 | 7.051 (1.508) | −4.972 | <.001 |
| Systematic reviews only | 60 (50.4) | 22.967 (2.830) | 8.508 (1.684) | ||||
| Number of times cited | |||||||
| Systematic reviews only | 64 (49.2) | 21.461 (3.629) | −0.374 | .709 | 7.391 (1.842) | −1.432 | .154 |
| Systematic reviews only | 66 (51.8) | 21.689 (3.334) | 7.856 (1.862) | ||||
| Published on Cochrane database of systematic reviews | |||||||
| Systematic reviews only | 113 (86.9) | 20.996 (3.330) | −10.794 | <.001 | 7.725 (1.650) | −12.946 | <.001 |
| Systematic reviews only | 17 (13.1) | 25.441 (1.102) | 10.235 (0.710) | ||||
| Manuscript length (no. of pages) | |||||||
| Systematic reviews only | 56 (43.1) | 19.884 (3.802) | −5.054 | <.001 | 6.661 (1.684) | −5.760 | <.001 |
| Systematic reviews only | 74 (56.9) | 22.858 (2.556) | 8.358 (1.648) | ||||
| Funding support | |||||||
| Systematic reviews only | 70 (53.8) | 20.986 (3.960) | −2.189 | .030 | 7.314 (2.052) | −2.143 | .034 |
| Systematic reviews only | 60 (46.2) | 22.267 (2.644) | 7.992 (1.545) | ||||
Significant results are shown in bold. Reduced denominators indicate missing data. Percentage may not total 100 because of rounding.
4.2. Reporting quality (PRISMA)
The overall mean PRISMA checklist score of all the included SRs/MAs was 21.577 (3.471) . Item 3 (rational, 99.2%), item 21 (synthesis of results, 99.2%), item 24 (summary of evidence, 99.2%), item 4 (objective, 98.5%) and item 7 (information sources, 97.7%) showed the highest adherence. The top five items with poor adherence were item 22 (risk of bias across studies, 20.0% ), item 15 (risk of bias across studies, 26.9% ), item 5 (protocol or registration, 30.8%), item 14 (synthesis of results, 66.2%) and item 8 (search, 67.7%). The compliance of the articles with each item including before (1996–2008) and after (2009–2020) introduction of the PRISMA statement showed wide variance (Figure 2).
FIGURE 2.
The results of the PRISMA assessment
4.3. Methodological quality (AMSTAR)
The overall mean AMSTAR checklist score of all the included SRs/MAs was 7.627 (1.860). Item 6 (characteristics of included studies provided, 94.6%), item 9 (appropriate methods used to combine studies, 93.8%) and item 11 (conflict of interest stated, 84.6%) showed highest adherence. The top three items with poor adherence were item 5 (list of studies provided, 15.4%), item 10 (likelihood of publication bias assessed, 19.2%) and item 4 (status of publication used as inclusion criteria, 21.5%). The compliance of the articles with each item before (1996–2006) and after (2007–2020) introduction of the AMSTAR checklist showed wide variance (Figure 3).
FIGURE 3.
The results of the AMSTAR assessment
4.4. Univariate and multivariate logistic analyses
Based on the highest quartile of the PRISMA results, we divided all the included SRs/MAs into the superior (N = 46, 35.4%) and inferior (N = 84, 64.6%) reporting quality groups. A univariate logistic regression analysis demonstrated that the following factors were associated with the superior reporting quality of the included SRs/MAs: SRs including MAs (OR = 2.1, [95% CI = 1.4–3.1]), RCT (OR = 2.3, [95% CI = 1.1–4.8]), number of affiliations (continues, OR = 1.3, [95% CI = 1.1–1.5]), number of pages (continues, OR = 1.1, [95% CI = 0.9–1.2]), number of pages > 12 (OR = 5.2, [95% CI = 2.2–12.2]), impact factor (continues, OR = 1.3, [95% CI = 1.1–1.5]), following PRISMA guidelines (OR = 2.2, [95% CI = 1.1–4.6]), having protocol or registration (OR = 17.2, [95% CI = 6.8–43.5]) and being published in the Cochrane Database of Systematic Reviews (OR = 44.3, [95% CI = 5.6–348.4]). Multivariate regression analyses demonstrated that the following factors were associated with the higher reporting quality of included SRs/MAs: having a protocol or registration (Adjusted OR = 7.6, [95% CI = 12.7–21.3]) and being published in Cochrane Database of Systematic Reviews (Adjusted OR = 9.1, [95% CI = 1.1–82.4]) (Table 2).
TABLE 2.
Univariate and multivariate logistic regression analyses of predictive factors associated with superior reporting quality (PRISMA)
| Variables | Univariate | Multivariate | ||
|---|---|---|---|---|
| OR (95% CI) | p | Adjusted OR (95% CI) | p | |
| Systematic reviews including meta‐analyses (ref. Systematic reviews only) | 2.1 (1.4, 3.1) | <.001 | ||
| International collaborative authorship (ref.no) | 1.5 (0.6, 3.5) | .352 | ||
| University (ref. Hospital) | 1.3 (0.4, 4.2) | .716 | ||
| Institute (ref. Hospital) | 1.1 (0.4, 3.0) | .890 | ||
| Specialty journal (ref. General) | 1.0 (0.4, 2.7) | .969 | ||
| RCT (ref.no) | 2.3 (1.1, 4.8) | .027 | ||
| Number of authors (continues) | 1.2 (0.9, 1.3) | .188 | ||
| Number of authors > 5 (ref. 1 ~ 5) | 0.5 (0.3, 1.1) | .090 | ||
| Number of affiliations (continues) | 1.3 (1.0, 1.5) | .027 | ||
| Number of affiliations > 3 (ref. 1 ~ 3) | 1.9 (0.9, 4.0) | .097 | ||
| Number of included studies (continues) | 0.9 (0.8, 1.0) | .772 | ||
| Number of included studies ≥ 10 (ref. < 10) | 0.7 (0.3, 1.5) | .402 | ||
| Number of pages (continues) | 1.1 (0.9, 1.2) | .001 | ||
| Number of pages > 12 (ref. 1 ~ 12) | 5.2 (2.2, 12.2) | <.001 | ||
| Impact factor (continues) | 1.3 (1.1, 1.5) | .007 | ||
| Number of times cited (continues) | 0.9 (0.8, 1.1) | .363 | ||
| Followed PRISMA guideline (ref.no) | 2.2 (1.1, 4.6) | .046 | ||
| Protocol or Registration (ref.no) | 17.2 (6.8, 43.5) | <.001 | 7.6 (2.7, 21.3) | <.001 |
| Journal source of SCI (ref.no) | 6.1 (0.8, 49.2) | .090 | ||
| Update previous review (ref.no) | 2.8 (0.8, 9.5) | .091 | ||
| Published on Cochrane Database of Systematic Reviews (ref.no) | 44.3 (5.6, 348.4) | <.001 | 9.1 (1.1, 82.4) | .049 |
| Funding support (ref.no) | 1.1 (0.5, 2.3) | .777 | ||
Significant results are shown in bold.
Based on the highest quartile of the AMSTAR results, we divided all the included SRs/MAs into the superior (N = 36, 27.7%) and inferior (N = 94, 72.3%) methodological quality groups. A univariate logistic regression analysis demonstrated that the following factors were associated with the superior methodological quality of the included SRs/MAs: SRs including MAs (OR = 3.1, [95% CI = 1.8–5.5]), RCT (OR = 2.9, [95% CI = 1.3–6.6]), number of authors (continues, OR = 1.4, [95% CI = 1.1–1.7]), number of authors > 5 (OR = 2.9, [95% CI = 1.3–6.7]), number of affiliations (continues, OR = 1.4, [95% CI = 1.1–1.7]), number of affiliations > 3 (OR = 3.3, [95% CI = 1.4–7.7]), number of pages (continues, OR = 1.1, [95% CI = 0.9–1.2]), number of pages > 12 (OR = 14.2, [95% CI = 4.1–49.6]), impact factor (continues, OR = 1.3, [95% CI = 1.1–1.5]), having protocol or registration (OR = 14.9, [95% CI = 5.9–37.4]) and updating a previous review (OR = 10.1, [95% CI = 2.6–40.0]). Multivariate regression analyses demonstrated that the following factors were associated with the higher methodological quality of included SRs/MAs: SRs including MAs (Adjusted OR = 1.9, [95% CI = 1.0–2.3]), number of authors > 5 (OR = 4.5, [95% CI = 1.2–17.4]), number of pages (continues, Adjusted OR = 1.1, [95% CI = 0.9–1.2]) and having protocol or registration ( Adjusted OR = 3.7, [95% CI = 1.2–11.8]) (Table 3).
TABLE 3.
Univariate and multivariate logistic regression analyses of predictive factors associated with superior methodological quality (AMSTAR)
| Variables | Univariate | Multivariate | ||
|---|---|---|---|---|
| OR (95% CI) | p | Adjusted OR (95% CI) | p | |
| Systematic reviews including meta‐analyses (ref. Systematic reviews only) | 3.1 (1.8, 5.5) | <.001 | 1.9 (1.1, 2.3) | .049 |
| International collaborative authorship (ref.no) | 2.0 (0.8, 4.8) | .126 | ||
| University (ref. Hospital) | 0.7 (0.2, 2.6) | .633 | ||
| Institute (ref. Hospital) | 0.7 (0.3, 2.0) | .534 | ||
| Specialty journal (ref. General) | 0.9 (0.4, 2.6) | .969 | ||
| RCT (ref.no) | 2.9 (1.3, 6.6) | .009 | ||
| Number of authors (continues) | 1.4 (1.3, 1.7) | .002 | ||
| Number of authors > 5 (ref. 1 ~ 5) | 2.9 (1.3, 6.7) | .010 | 4.5 (1.2, 17.4) | .028 |
| Number of affiliations (continues) | 1.4 (1.1, 1.7) | .003 | ||
| Number of affiliations > 3 (ref. 1 ~ 3) | 3.3 (1.4, 7.7) | .007 | ||
| Number of included studies (continues) | 1.0 (0.1, 1.1) | .832 | ||
| Number of included studies ≥ 10 (ref. < 10) | 0.9 (0.4, 2.1) | .877 | ||
| Number of pages (continues) | 1.1 (0.9, 1.2) | <.001 | 1.1 (0.9, 1.2) | .010 |
| Number of pages > 12 (ref. 1 ~ 12) | 14.2 (4.1, 49.6) | <.001 | ||
| Impact factor (continues) | 1.3 (1.1, 1.5) | .003 | ||
| Number of times cited (continues) | 0.2 (0.1, 1.0) | .125 | ||
| Followed PRISMA guideline (ref.no) | 1.1 (0.5, 2.4) | .877 | ||
| Protocol or Registration (ref.no) | 14.9 (5.9, 37.4) | <.001 | 3.7 (1.2, 11.8) | .028 |
| Update previous review (ref.no) | 10.1 (2.6,40.0) | .001 | ||
| Funding support (ref.no) | 1.4 (0.7, 3.1) | .350 | ||
Significant results are shown in bold.
4.5. The relationship between reporting and methodology quality
As shown in Figure 4, a strong linear correlation (r = .860) was detected between the scores of PRISMA and AMSTAR.
FIGURE 4.
Relationship between total PRISMA score and total AMSTAR score
5. DISCUSSION
In consideration of the continuous development of EBM in the field of nursing, especially for globally prevalent diseases such as COPD, which have long relied on nursing intervention to maintain a good prognosis and improve symptom management, quality of life, self‐management, the quality of SRs/MAs has received increased attention by nurses and healthcare workers in this field and patients and their caregivers. This review is the first to assess the epidemiological characteristics and the reporting and methodological quality of 130 SRs/MAs on nursing interventions for COPD patients using the PRISMA statement and AMSTAR checklist. The results indicated that the quality of the reporting [21.577 (3.471)] and methodologies [7.627 (1.860)] were suboptimal, although they are significantly higher than those of articles (Nagendrababu et al., 2018; Shi et al., 2014; Wasiak et al., 2017). The compliance rates of some items with the PRISMA statement and the AMSTAR checklist were low, which was the same as that of Liu et al. (2017) and Cullis et al. (2017).
Regarding reporting quality, the compliance rates of three items (item 22, item 15, item 5) were lower than 50.0%. Based on our data, only 30.8% of the SRs/MAs showed protocol and registration information (item 5). The protocol is the basis for the design, implementation, reporting and evaluation of SRs/MAs(Gao et al., 2020), which pre‐specifies the objectives and methods(Liberati et al., 2009). It is important to verify protocol and registration before doing a review as the protocol increases the accuracy and credibility of research and supports transparency and improvements during the review process. The registration resources such as PROSPERO ("PROSPERO—International prospective register of systematic reviews") and Systematic Reviews ("Systematic Reviews Journal") can allow authors to easily obtain information regarding protocol and registration. Therefore, registration with an clearly designed protocol is suggested in future. Furthermore, many common defects were also found in these SRs/MAs: risk of bias across studies, mostly publication bias (PRISMA items 15, 22, AMSTAR item 10). For publication bias, it is probable that authors and journal editors are willing to report significant results rather than negative results (Furukawa et al., 2007; Kirkham et al., 2010; Polychronopoulou et al., 2010). If there is publication bias, the treatment effects and risk factors strength may be overstated, leading to mistakes in clinical treatment and health decisions (Sun et al., 2018). Hence, authors can determine whether their results are influenced by publication biases according to drawing funnel plots or performing relevant statistical tests (Polychronopoulou et al., 2010).
For the methodological quality, the compliance rates of three items (item 5, item 10, item 4) were lower than 50.0%. Only about 20.8% of SRs/MAs pay attention to publication status (AMSTAR item 4). This means that grey literature is easily ignored in the article search process. The unpublished data are not included in a SR/MA, which may overestimate predictions of intervention effectiveness to some extent and lead to poor methodological quality (McAuley et al., 2000). In theory, researchers should try to search for articles on multiple databases and different languages in a comprehensive and systematic manner and include grey literature and literature in other languages (Phan et al., 2015). This avoids publication and language bias. Owing to limitations of space, exclusion lists of SRs/MAs usually do not appear in published papers. Cochrane systematic reviews generally present lists of inclusions and exclusions of literature in articles (AMSTAR item 5). However, only about 15.4% of SRs/MAs offered a list of excluded studies and justified the exclusions.
A multivariate logistic analysis was conducted to explore factors associated with potential predictors of high reporting and methodological quality. It was hypothesized that having a protocol and registration would improve the overall quality of including SRs/MAs (reporting quality: OR = 7.583, methodological quality: OR = 3.690). As required by the PRISMA Statement item 5, the need for registration and protocol availability is obviously significant to promote transparency of methodology. Previous studies have showed the SRs/MAs published in the Cochrane Database of Systematic Reviews generally have higher reporting quality than SRs/MAs published in general medical journals (Cullis et al., 2017; Delaney et al., 2007; Wasiak et al., 2017). Our finding was also similar. Compared with the traditional descriptive SRs, those including meta‐analyses were of higher methodological quality (OR = 1.889), which can affect the validity and reliability of evidence. However, it must be emphasized that meta‐analysis is not always appropriate or effective because of clinical or statistical heterogeneity. Therefore, the lack of a meta‐analysis should not mean that the methodological quality of systematic reviews is not good. According to Li's (Li et al., 2015) findings, SRs/MAs with more than five authors showed higher methodological quality (OR = 4.517). This reflects that SRs/MAs requires teamwork and are produced by researchers in different fields, such as among staff, statistical researchers, which can help improve their quality. Manuscript length was also associated with higher methodological quality (OR = 1.088). To some extent, greater length indicated fullness and comprehensiveness of content.
The results also emphasized a positive relationship between PRISMA scores and AMSTAR scores. This may show an opportunity to improve study quality by integrating the principles of PRISMA and AMSTAR into the planning of systematic reviews.
Some limitations are showed in this review. First, the sample size included in this review is still not large enough. We have only searched 3 databases and that CINAHL was not systematically searched. Although we have manually searched relevant references, there are still some articles that are not obtained in full text. Second, only SRs/MAs published in English were included and the results may not apply to SRs/MAs published in other languages (Ge et al., 2018).
The main strengths of this study include the focused search and selection of SRs/MAs, comprehensive assessment and planned logistic regression analyses. For selection of SRs/MAs, we not only included SRs but also MAs because our study has to be more comprehensive and objective. In addition, some of the assessment items only applied to meta‐analyses, such as PRISMA statement (items 14, 16, 23) and AMSTAR checklist (item 9). To minimize bias against systematic review, we identified these items as not applicable and considered the item to be fulfilled when the total number of completed items was compiled. Thus, they have a more consistent score criterion. Also, our results would have more credibility and our conclusions would have a more specific implication.
6. CONCLUSION
Despite the continued increase of SRs/MAs on nursing interventions for COPD patients, a significant number of existing SRs/MAs contain suboptimal reporting and methodology quality, with some room for improvement. Journals should consider having specific “a priori” criteria based on checklists such as those provided by PRISMA and AMSTAR before publication of manuscripts to ensure the highest possible reporting and methodological quality of these reviews. We recommend that authors, readers, reviewers and editors consider and adherence to PRISMA and AMSTAR checklists when conducting, reading, reviewing and editing SRs/MAs in the future.
7. ETHICS APPROVAL AND CONSENT TO PARTICIPATE
Not applicable. This review does not involve human participants, human dataor human tissue.
CONFLICT OF INTEREST
The authors declare that they have no conflict of interest.
AUTHOR CONTRIBUTION
Review concepts: S.X., W.D; review design: S.X., L.B.; Manuscript writing S.X., W.D.; Manuscript editing: S.X., L.B, W.D.; Data extraction: S.X., W. M, L.H.; Data elaboration and interpretation: S.X., W.M., L.H.; Statistical analysis: S.X., W.D.; Manuscript revision and approval of submission in its present form: S.X., WD., L.B., W.M..
Supporting information
Figure S1‐S3
Appendix S1
Appendix S2
Sun X, Wang D, Wang M, Li H, Liu B. The reporting and methodological quality of systematic reviews and meta‐analyses of nursing interventions for chronic obstructive pulmonary disease ‐ A systematic review. Nurs Open. 2021;8:1489–1500. 10.1002/nop2.767
Funding information
This study was supported by Shandong Medical and Health Technology Development Project (2018WS006) and Zibo key research and development plan (2018kj060040).
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the author (XS), upon reasonable request.
REFERENCES
- Blackstock, F. , & Webster, K. (2007). Disease‐specific health education for COPD: A systematic review of changes in health outcomes. Health Educution Research, 22(5), 703–717. 10.1093/her/cyl150 [DOI] [PubMed] [Google Scholar]
- Cullis, P. S. , Gudlaugsdottir, K. , & Andrews, J. (2017). A systematic review of the quality of conduct and reporting of systematic reviews and meta‐analyses in paediatric surgery. PLoS One, 12(4), e0175213. 10.1371/journal.pone.0175213 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Delaney, A. , Bagshaw, S. M. , Ferland, A. , Laupland, K. , Manns, B. , & Doig, C. (2007). The quality of reports of critical care meta‐analyses in the Cochrane Database of Systematic Reviews: An independent appraisal. Critical Care Medicine, 35(2), 589–594. 10.1097/01.ccm.0000253394.15628.fd [DOI] [PubMed] [Google Scholar]
- Eddy, D. M. (2005). Evidence‐based medicine: A unified approach. Health Affairs, 24(1), 9–17. 10.1377/hlthaff.24.1.9 [DOI] [PubMed] [Google Scholar]
- Furukawa, T. A. , Watanabe, N. , Omori, I. M. , Montori, V. M. , & Guyatt, G. H. (2007). Association between unreported outcomes and effect size estimates in Cochrane meta‐analyses. JAMA, 297(5), 468–470. 10.1001/jama.297.5.468-b [DOI] [PubMed] [Google Scholar]
- Gao, Y. A. , Cai, Y. , Yang, K. , Liu, M. , Shi, S. , Chen, J. I. , Sun, Y. , Song, F. , Zhang, J. , & Tian, J. (2020). Methodological and reporting quality in non‐Cochrane systematic review updates could be improved: A comparative study. Journal of Clinal Epidemiology, 119, 36–46. 10.1016/j.jclinepi.2019.11.012 [DOI] [PubMed] [Google Scholar]
- Ge, L. , Tian, J.‐H. , Li, Y.‐N. , Pan, J.‐X. , Li, G. E. , Wei, D. , Xing, X. , Pan, B. , Chen, Y.‐L. , Song, F.‐J. , & Yang, K.‐H. (2018). Association between prospective registration and overall reporting and methodological quality of systematic reviews: A meta‐epidemiological study. Journal of Clinical Epidemiology, 93, 45–55. 10.1016/j.jclinepi.2017.10.012 [DOI] [PubMed] [Google Scholar]
- Gendron, L. M. , Nyberg, A. , Saey, D. , Maltais, F. , & Lacasse, Y. (2018). Active mind‐body movement therapies as an adjunct to or in comparison with pulmonary rehabilitation for people with chronic obstructive pulmonary disease. Thorax, 10, Cd012290. 10.1002/14651858.CD012290.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kirkham, J. J. , Dwan, K. M. , Altman, D. G. , Gamble, C. , Dodd, S. , Smyth, R. , & Williamson, P. R. (2010). The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ, 340, c365. 10.1136/bmj.c365 [DOI] [PubMed] [Google Scholar]
- Li, T. , Wei, N. , Fu, C. , Fu, S. , Yang, L. , Wang, Z. , & Cheng, J. (2015). Methodological and reporting quality of systematic reviews/meta‐analyses of transurethral procedure for benign prostatic hyperplasia. Chinese Journal ofEvidence‐based Medicine, 10, 113–119. [Google Scholar]
- Liberati, A. , Altman, D. G. , Tetzlaff, J. , Mulrow, C. , Gotzsche, P. C. , Ioannidis, J. P. A. , Clarke, M. , Devereaux, P. J. , Kleijnen, J. , & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta‐analyses of studies that evaluate healthcare interventions: Explanation and elaboration. BMJ, 339, b2700. 10.1136/bmj.b2700 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu, H. , Zhou, X. , Yu, G. , & Sun, X. (2019). The effects of the PRISMA statement to improve the conduct and reporting of systematic reviews and meta‐analyses of nursing interventions for patients with heart failure. International Journal of Nursing Practice, 25(3), e12729. 10.1111/ijn.12729 [DOI] [PubMed] [Google Scholar]
- Liu, P. , Qiu, Y. , Qian, Y. , Chen, X. , Wang, Y. , Cui, J. , & Zhai, X. (2017). Quality of meta‐analyses in major leading gastroenterology and hepatology journals: A systematic review. Journal of Gastroenterology and Hepatology, 32(1), 39–44. 10.1111/jgh.13591 [DOI] [PubMed] [Google Scholar]
- Manchikanti, L. , Datta, S. , Smith, H. S. , & Hirsch, J. A. (2009). Evidence‐based medicine, systematic reviews and guidelines in interventional pain management: Part 6. Systematic reviews and meta‐analyses of observational studies. Pain Physician, 12(5), 819–850. [PubMed] [Google Scholar]
- McAuley, L. , Pham, B. , Tugwell, P. , & Moher, D. (2000). Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta‐analyses? Lancet, 356(9237), 1228–1231. 10.1016/s0140-6736(00)02786-0 [DOI] [PubMed] [Google Scholar]
- McCarthy, B. , Casey, D. , Devane, D. , Murphy, K. , Murphy, E. , & Lacasse, Y. (2015). Pulmonary rehabilitation for chronic obstructive pulmonary disease. Cochrane Database of Systematic Reviews, 23(2), CD003793. 10.1002/14651858.CD003793 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Moher, D. , Cook, D. J. , Eastwood, S. , Olkin, I. , Rennie, D. , & Stroup, D. F. (1999). Improving the quality of reports of meta‐analyses of randomised controlled trials: The QUOROM statement. Quality of Reporting of Meta‐analyses. Lancet, 354(9193), 1896–1900. 10.1016/s0140-6736(99)04149-5 [DOI] [PubMed] [Google Scholar]
- Nagendrababu, V. , Pulikkotil, S. J. , Sultan, O. S. , Jayaraman, J. , & Peters, O. A. (2018). Methodological and reporting quality of systematic reviews and meta‐analyses in endodontics. Journal of Endodontics, 44(6), 903–913. 10.1016/j.joen.2018.02.013 [DOI] [PubMed] [Google Scholar]
- Phan, K. , Tian, D. H. , Cao, C. , Black, D. , & Yan, T. D. (2015). Systematic review and meta‐analysis: Techniques and a guide for the academic surgeon. Annals of Cardiothoracic Surgery, 4(2), 112–122. 10.3978/j.issn.2225-319X.2015.02.04 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Polychronopoulou, A. , Pandis, N. , & Eliades, T. (2010). Assessment of publication bias in dental specialty journals. Journal of Evidence‐Based Dental Practice, 10(4), 207–211. 10.1016/j.jebdp.2010.09.014 [DOI] [PubMed] [Google Scholar]
- PROSPERO – International prospective register of systematic reviews . Retrieved from http://www.systematicreviewsjournal.com/ [DOI] [PMC free article] [PubMed]
- Shea, B. J. , Grimshaw, J. M. , Wells, G. A. , Boers, M. , Andersson, N. , Hamel, C. , Porter, A. C. , Tugwell, P. , Moher, D. , & Bouter, L. M. (2007). Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7, 10. 10.1186/1471-2288-7-10 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shea, B. J. , Hamel, C. , Wells, G. A. , Bouter, L. M. , Kristjansson, E. , Grimshaw, J. , Henry, D. A. , & Boers, M. (2009). AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology, 62(10), 1013–1020. 10.1016/j.jclinepi.2008.10.009 [DOI] [PubMed] [Google Scholar]
- Shi, C. , Zhu, L. , Wang, X. , Qin, C. , Xu, Q. , & Tian, J. (2014). Epidemiology, methodological and reporting characteristics of systematic reviews of nursing interventions published in China. International Journal of Nursing Practice, 20(6), 689–700. 10.1111/ijn.12255 [DOI] [PubMed] [Google Scholar]
- Sim, I. , Gorman, P. , Greenes, R. A. , Haynes, R. B. , Kaplan, B. , Lehmann, H. , & Tang, P. C. (2001). Clinical decision support systems for the practice of evidence‐based medicine. Journal of American Medical Informatics Association, 8(6), 527–534. 10.1136/jamia.2001.0080527 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sun, X. , Zhou, X. , Yu, Y. , & Liu, H. (2018). Exploring reporting quality of systematic reviews and Meta‐analyses on nursing interventions in patients with Alzheimer's disease before and after PRISMA introduction. BMC Medical Research Methodol, 18(1), 154. 10.1186/s12874-018-0622-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sun, X. , Zhou, X. , Zhang, Y. , & Liu, H. (2019). Reporting and methodological quality of systematic reviews and meta‐analyses of nursing interventions in patients with Alzheimer's disease: General implications of the findings. Journal of Nursing Scholarship, 51(3), 308–316. 10.1111/jnu.12462 [DOI] [PubMed] [Google Scholar]
- Systematic Reviews Journal . Retrieved from http://www.systematicreviewsjournal.com/
- Wasiak, J. , Tyack, Z. , Ware, R. , Goodwin, N. , & Faggion, C. M. Jr (2017). Poor methodological quality and reporting standards of systematic reviews in burn care management. International Wound Journal, 14(5), 754–763. 10.1111/iwj.12692 [DOI] [PMC free article] [PubMed] [Google Scholar]
- World Health Organization, W. H. (2008). World Health Statistics. Retrieved from https://www.who.int/respiratory/copd/causes/en/
- Xia, L. , Xu, J. , & Guzzo, T. J. (2017). Reporting and methodological quality of meta‐analyses in urological literature. PeerJ, 5, e3129. 10.7717/peerj.3129 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Figure S1‐S3
Appendix S1
Appendix S2
Data Availability Statement
The data that support the findings of this study are available from the author (XS), upon reasonable request.