Skip to main content
PMC home page
Medicine logoLink to Medicine
. 2024 Sep 27;103(39):e39933. doi: 10.1097/MD.0000000000039933

Reporting quality of meta-analyses in acupuncture: Investigating adherence to the PRISMA statement

Chu Qin a, Huan Ma a, Oscar Onayi Mandizadza a, Xiujuan Xu b, Conghua Ji a,*
PMCID: PMC11441946  PMID: 39331860

Abstract

Although Systematic Reviews and Meta-Analyses (PRISMA) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Acupuncture (PRISMA-A) checklists had been in use for several years, compliance rate was still not optimistic. We investigated the quality of reporting for meta-analyses of acupuncture published in PubMed. We compared the compliance rate for the quality of reporting following the publication of both the PRISMA and PRISMA-A recommendations. We searched PubMed for articles published between January 1st, 2020 and December 31st, 2022, after Endnote X9 document management software and manual screening, 180 meta-analyses of acupuncture were selected as samples. The PRISMA, and PRISMA-A checklists were used to evaluate the quality of the literature. Data were collected using a standard form. Pearson χ2 test and/or Fisher exact test were used to assess differences in reporting among groups. Logistic regression is used to calculate OR and its 95% CI. The total reported compliance rate of all items in the PRISMA list was 61.3%, and the reported compliance rate of the items with a compliance rate of <50% accounted for 35.71% of the total items. The total reported coincidence rate of all items in the PRISMA-A was 56.9%, and the reported coincidence rate of the items with a reported coincidence rate of <50% accounted for 31.25% of all the items. The compliance rate of the published research to PRISMA or PRISMA-A has no statistical difference between the Journal Citation Reports partition (Quarter1–Quarter2) and Journal Citation Reports partition (Quarter3–Qurater4) (P > .05). Regardless of the level of journals published, have obvious deficiencies in the details of the study, the reference basis for the design of the study, the analysis method, the degree of strictness, the scientific nature, and other aspects. We must strengthen education on the standardization of research reports.

Keywords: acupuncture, meta-analysis, PRISMA, PRISMA-A

1. Introduction

Randomized controlled trials (RCTs) are the basis of evidence-based medicine, the most reliable epidemiological study design,[1] and the gold standard for evaluating the efficacy of interventions.[2] Meta-analysis is the main secondary research method, and RCT-based meta-analysis is at the tip of the evidence-based medicine pyramid, especially with high-quality RCT research.[3] Acupuncture and moxibustion originated in China more than 2000 years ago and are integral part of traditional Chinese medicine.[4] To improve the quality of these reports and avoid systematic reporting errors, the Systematic Reviews and Meta-Analyses (PRISMA) statement, released in 2009, is internationally used as unified standards for high-quality meta-analyses.[5]

A detailed introduction of acupuncture was published in well-known journals many years ago.[6,7] It is the most popular sensory stimulation therapy. Thin needles are inserted on specific anatomical points[8] to promote healing. In recent years, its role in the diagnosis and treatment of diseases has been widely recognized, and many RCTs and meta-analyses are published yearly.[9–11] However, due to different acupuncture and moxibustion methods and differences in the implementation of the standards in different research projects, the quality of published RCTs articles is uneven, which further affects the quality of meta-analysis from these RCTs. The expanded version of the PRISMA developed in 2019, known as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Acupuncture (PRISMA-A) check list, are formulated to complete and standardize intervention reports in acupuncture meta-analyses.[12] The list of acupuncture control items is expected to improve the quality of Meta-analyses reports.[13,14]

While the PRISMA checklist has been utilized for over a decade, compliance rates still fall short of ideal.[15] The PRISMA-A checklist, introduced and published in recent years, awaits determination of its compliance rates. This article aims to reveal potential problems and put forward suggestions for improvement, so as to further standardize the report and to improve the overall quality level of acupuncture meta-analysis.

2. Methods

2.1. Research design

This is a cross-sectional study based on sampling survey that analyzes the gap between the actual situation of published literature and the standard requirements. Our research systematically compared the reporting of retrieved literature with the PRISMA and PRISMA-A guidelines, assigning adherence scores accordingly, to investigate the compliance rate of literature reporting. PubMed (https://pubmed.ncbi.nlm.nih.gov) is a free search engine maintained by the U.S. National Library of Medicine, primarily accessing the MEDLINE database, which covers a vast amount of literature in the life sciences and biomedical fields. It is widely regarded as one of the most authoritative and comprehensive biomedical literature databases.[16] Given its representativeness, we selected PubMed as the database for sampling in this study. Papers on acupuncture meta-analysis published in recent 3 years that can be retrieved from PubMed are used as research samples for analysis, and the sample size meets the statistical requirements. We searched PubMed for articles published between January 1st, 2020 and December 31st, 2022. Using the advanced search tool on PubMed, the title/abstract category was selected, and then “acupuncture” was entered for retrieval. We selected the filters, “ Meta-Analysis,” and manually checked the confirmed research, comments, unpublished works or other relevant reference materials provided by experts for any missing studies. There was no restriction on language.

2.2. Inclusion and exclusion criteria

In order to comprehensively assess the reporting quality of existing meta-analyses, the following criteria are incorporated: (1) research type: published meta-analysis of acupuncture and moxibustion therapy. All the studies adopted clinical randomized controlled trials, (2) intervention measures: acupuncture or acupuncture combined with other therapies, (3) control group: other treatment measures except acupuncture (including pseudo acupuncture, nontreatment point acupuncture, drug therapy, and others). (4) Language: English. Articles were excluded under the following circumstances: (1) meta-analyses where acupuncture is just one of the treatment methods, (2) duplicated studies or studies with limited data, (3) animal experiments, (4) review or only systematic evaluation, (5) meta-analysis protocol.

2.3. Literature screening

We compiled the search results and deleted duplicates using EndNote X9.1 (Clarivate Analytics, Philadelphia, PA). Two reviewers(Chu Qin, Huan Ma) independently screened the retrieved articles by title and abstract, and then read the full text of articles that passed the inclusion and exclusion criteria. Any differences were resolved through discussion or by a third reviewer (Conghua Ji).

2.4. Data arrangement

Data extraction was performed independently by 2 researchers (Chu Qin, Huan Ma) using a custom data extraction table created using Microsoft Excel 2016. The extracted contents included the first author, journal type, disease type [classified according to the 11th Revision of the International Classification of Diseases (ICD-11)], Journal Citation Reports (JCR) classification [Journals in the top 25% (including 25%) based on Impact Factor are classified as Quarter1 (Q1); those in the 25% to 50% range as Quarter 2 (Q2); those in the 50–75% range as Quarter 3 (Q3); and those below the 75th percentile as Quarter 4 (Q4)].[17] PRISMA statement entries, and PRISMA-A list entries. The 2 researchers used the PRISMA 2020 checklist and PRISMA-A 2020 checklist to evaluate the quality of each included meta-analysis report. The PRISMA 2020 checklist includes 7 parts: title, abstract, introduction, method, results, discussion and other information, and is divided into 27 items and 15 secondary items; the PRISMA-A 2020 checklist statement consists of 7 parts: the title, abstract, introduction, method, results, discussion, and funds, which are divided into 27 items, 5 secondary items. According to whether each list item is reported in the article, the 2 researchers made a judgment of “satisfied” or “unsatisfied” item by item. After the evaluation was completed, the 2 researchers cross-checked the results. In case of any differences, a third party(Conghua Ji) was called upon for judgment.

2.5. Statistical methods

A pilot survey was conducted before the formal investigation and study. The sample size estimate of this study was determined by the average coincidence rate of the pilot survey literature. Estimation formula based on sample size of all included acupuncture Meta-analyses is estimated to be 0.5 based on the maximum sample size, and the power of statistical efficiency is set at 80% or higher to identify the level of double-sided significance of 5%. Considering the dropout rate of 10%, the inclusive sample size was more than 97. We used the IBM SPSS Statistics 26 (Armonk, NY) data editor to perform statistical analysis on the collected data. The conformity rate of each item was expressed as a percentage, using χ2 test compared the data of different JCR divisions and inter-country groupings. If the theoretical frequency was too small, Fisher exact probability method was applied, where P < .05 was accepted as statistically significant. The impact of the compliance of different items on the JCR partition was evaluated using the logistic dichotomy regression model, and 95% confidence interval of the OR value was calculated.

3. Results

3.1. Sample selection and flow

The meta-analysis of acupuncture obtained 391 articles after preliminary retrieval. After the removal of duplicates, 368 articles remained, and then 188 documents were excluded after reading the title and abstract and conducting a full-text screening. In the end, 180 articles were included, and the sample size met the expected statistical requirements (Fig. 1).

Figure 1.

Figure 1.

Open in a new tab

Flow chart of literature screening for meta-analysis of acupuncture.

3.2. Study description of the meta-analysis characteristics

Among all the included acupuncture meta-analysis, 19 articles (10.56%) were published in JCR area 1, 60 articles (33.33%) in JCR area 2, 81 articles (45.00%) in JCR area 3, 20 articles (11.11%) in JCR area 4, and 7 articles (5.22%) (Fig. 2A). According to the countries that published the articles, the research proportion of acupuncture clinical trials (147, 81.67%) in China is much higher than any other country (33, 18.33%) (Fig. 2B). There are 18 (69.23%) types of diseases (ICD-11) involved, among which there are many studies on symptoms and signs and clinical and laboratory abnormalities that cannot be classified elsewhere (43, 23.89%), neurological diseases (26, 14.44%), musculoskeletal or connective tissue diseases (21, 11.67%) and mental, behavioral or neurodevelopmental disorders (17, 9.44%) (Fig. 2C).

Figure 2.

Figure 2.

Open in a new tab

Study description of the meta-analysis characteristics.

3.3. Characteristics and variation in PRISMA list

It can be seen from Table 1 that in the included acupuncture meta-analysis, the overall compliance rate for all items on the PRISMA checklist was 61.3%. Among these, 35.71% of the items had a compliance rate of <50%. Specifically, the compliance rate for the revised items related to registration and agreement was only 0.6%. The research results show that the compliance rate of the published research to PRISMA 2020 has no statistical difference between the JCR partition (Q1–Q2) and JCR partition (Q3–Q4) (P > .05) (Table 2). However, we found that the research of JCR partition (Q1–Q2) complies more to the standard report of individual research results; on the contrary, the research of JCR partition (Q3–Q4) complies more with the normative report on the heterogeneity in the results (Table 2).

Table 1.

The compliance rate of PRISMA 2020 checklist.

Section/topic Item no Satisfied Not satisfied
Title
Title 1 173(96.1%) 7(3.9%)
Abstract
Abstract 2 35(19.4%) 145(80.6%)
Introduction
Rationale 3 175(97.2%) 5(2.8%)
Objectives 4 36(20.0%) 144(80.0%)
Methods
Eligibility criteria 5 85(47.2%) 95(52.8%)
Information sources 6 33(18.3%) 147(81.7%)
Search strategy 7 121(67.2%) 59(32.8%)
Selection process 8 140(77.8%) 40(22.2%)
Data collection process 9 155(86.1%) 25(13.9%)
Data items 10a 159(88.3%) 21(11.7%)
10b 43(23.9%) 137(76.1%)
Study risk of bias assessment 11 114(63.3%) 66(36.7%)
Effect measures 12 125(69.4) 55 (30.6%)
Synthesis methods 13a 170(94.4%) 10(5.6%)
13b 43(23.9%) 137(76.1%)
13c 64(35.6%) 116(64.4%)
13d 157(87.2%) 23(12.8%)
13e 78(43.3%) 102(56.7%)
13f 87(48.3%) 93(51.7%)
Reporting bias assessment 14 21 (11.7%) 159(88.3%)
Certainty assessment 15 18(10.0%) 162(90.0%)
Results
Study selection 16a 180(100.0%) 0(0.0%)
16b 162(90.0%) 18(10.0%)
Study characteristics 17 174(96.7%) 6(3.3%)
Risk of bias in studies 18 159(88.3%) 21(11.7%)
Results of individual studies 19 126(70.0%) 54(30.0%)
Results of syntheses 20a 173(96.1%) 7(3.9%)
20b 162(90.0%) 18(10.0%)
20c 128(71.1%) 52(28.9%)
20d 69(38.3%) 111(61.7%)
Reporting biases 21 104 (57.8%) 76(42.2%)
Certainty of evidence 22 56(31.1%) 124(68.9%)
Discussion
Discussion 23a 178(98.9%) 2(1.1%)
23b 153(85.0%) 27(15.0%)
23c 98(54.4%) 81(45.0%)
23d 163(90.6%) 17(9.4%)
Other information
Registration and protocol 24a 99(55.0%) 81(45.0%)
24b 28(15.6%) 152(84.4%)
24c 1(0.6%) 179(99.4%)
Support 25 140(77.8%) 40(22.2%)
Competing interests 26 158(87.8%) 22(12.2%)
Availability of data, code and other materials 27 93(51.7%) 87(48.3%)
Total 4636(61.3%) 2923(38.7%)
Open in a new tab

Table 2.

Comparison of compliance between JCR partitions of PRISMA.

Item
No		 WOS level: zone 1–2 (n = 79) WOS level: zone 3–4 (n = 101) x2 P OR (95% CI)
Satisfied Satisfied rate Satisfied Satisfied rate
1 75 94.9% 98 97.0% 0.110 .740 1.742 (0.378–8.021)
2 19 24.1% 16 15.8% 1.907 .167 0.594(0.283–1.249)
3 76 96.2% 99 98.0% 0.078 .780 1.954(0.318–11.987)
4 11 13.9% 25 24.8% 3.249 .071 2.033(0.931–4.440)
5 34 43.0% 51 50.5% 0.989 .320 1.350(0.747–2.440)
6 17 21.5% 16 15.8% 0.954 .329 0.687(0.322–1.464)
7 53 67.1% 68 67.3% 0.001 .973 1.011(0.540–1.892)
8 62 78.5% 78 77.2% 0.040 .841 0.930(0.457–1.891)
9 68 86.1% 87 86.1% <0.001 .990 1.005(0.429–2.354)
10a 72 91.1% 87 86.1% 1.076 .300 0.604(0.231–1.577)
10b 56 70.9% 66 65.3% 0.623 .430 0.774(0.410–1.462)
11 53 67.1% 61 60.4% 0.855 .355 0.748(0.404–1.385)
12 50 63.3% 75 74.3% 2.512 .113 1.673(0.883–3.170)
13a 76 96.2% 94 93.1% 0.340 .560 0.530(0.133–2.119)
13b 23 29.1% 20 19.8% 2.114 .146 0.601(0.302–1.198)
13c 28 35.4% 36 35.6% 0.001 .978 1.009(0.545–1.866)
13d 66 83.5% 91 90.1% 1.709 .191 1.792(0.741–4.335)
13e 32 40.5% 46 45.5% 0.458 .498 1.228(0.677–2.229)
13f 37 46.8% 50 49.5% 0.126 .722 1.113(0.617~2.006(
14 12 15.2% 9 8.9% 1.696 .193 0.546(0.281–1.370)
15 10 12.7% 8 7.9% 1.105 .293 0.594(0.223–1.582)
16a 79 100.0% 101 100.0% – – –
16b 71 89.9% 91 90.1% 0.003 .960 1.025(0.385–2.732)
17 77 97.5% 97 96.0% 0.012 .911 0.630(0.112–3.530)
18 69 87.3% 90 89.1% 0.134 .714 1.186(0.476–2.952)
19 62 78.5% 64 63.4% 4.822 .028 0.474(0.242–0.929)
20a 76 96.2% 97 96.0% <0.001 1.000 0.957(0.208–4.406)
20b 68 86.1% 94 93.1% 2.409 .121 2.172(0.801–5.891)
20c 50 63.3% 78 77.2% 4.191 .041 1.967(1.024–3.777)
20d 29 36.7% 40 39.6% 0.157 .692 1.131(0.616–2.074)
21 43 54.4% 61 60.4% 0.647 .421 1.277(0.704–2.317)
22 26 32.9% 30 29.7% 0.213 .644 0.861(0.457–1.624)
23a 79 100.0% 99 89.0% – .505 –
23b 70 88.6% 83 82.2% 1.437 .231 0.593(0.251–1.402)
23c 48 60.8% 50 49.5% 2.264 .132 0.633(0.349–1.150)
23d 70 88.6% 93 92.1% 0.625 .429 1.495(0.549–4.069)
24a 48 60.8% 51 50.5% 1.887 .170 0.659(0.363–1.196)
24b 16 20.3% 12 11.9% 2.365 .124 0.531(0.235–1.199)
24c 1 1.3% 0 0.0% – .439 –
25 63 79.7% 77 76.2% 0.316 .574 0.815(0.399–1.665)
26 69 87.3% 89 88.1% 0.025 .874 1.075(0.439–2.634)
27 46 58.2% 47 46.5% 2.427 .119 0.624(0.345–1.131)
Open in a new tab

3.4. Characteristics and variation in PRISMA-A list

It can be seen from Table 3 that in the included acupuncture meta-analysis, the total reported coincidence rate of all items in the PRISMA-A was 56.9%, and the reported coincidence rate of the items with a reported coincidence rate of <50% accounted for 31.25% of all the items, among which the reported coincidence rate of the items describing the study reported the characteristics related to “De-qi” was only 3.3%. The compliance rate of the published research to PRISMA-A 2020 has no statistical difference between JCR (Q1–Q2) and JCR (Q3–Q4), but we found that the research of JCR (Q1–Q2) complies more to the normative report of individual research results and the full report of limitations; However, the research of JCR partition (Q3–Q4) can better comply with the standardized report of the comprehensive results. This confirms that the lack of quality control on research reports is a common phenomenon, and we need to strengthen our knowledge on quality control of research reports (Table 4).

Table 3.

The compliance rate of PRISMA-A 2020 checklist.

Section/topic Item no Satisfied Not satisfied
Title
Title 1 1(100.0%) 0(0.0%)
Abstract
Structured summary 2 35(19.4%) 145(80.6%)
Introduction
Rationale 3 175(97.2%) 5(2.8%)
Objectives 4 36(20.0%) 144(80.0%)
Methods
Protocol and registration 5 99(55.0%) 81(45.0%)
Eligibility criteria 6 85(47.2%) 95(52.8%)
6a1 46(25.6%) 134(74.4%)
6a2 31(17.2%) 149(82.8%)
6b 111(61.7%) 69(38.3%)
6c 154(85.6%) 26(14.4%)
Information sources 7 33(18.3%) 147(81.7%)
Search 8 121(67.2%) 59(32.8%)
Study selection 9 140(77.8%) 40(22.2%)
Data collection
Process		 10 155(86.1%) 25(13.9%)
Data items 11 159(88.3%) 21(11.7%)
Risk of bias in
individual studies		 12 114(63.3%) 66(36.7%)
Summary measures 13 125(69.4) 55 (30.6%)
Synthesis of results 14 85(47.2%) 95(52.8%)
Risk of bias across
Studies		 15 21(11.7%) 159(88.3%)
Additional analyses 16 107(59.4%) 73(40.6%)
Results
Study selection 17 180(100.0%) 0(0.0%)
Study characteristics 18 174(96.7%) 6(3.3%)
18a 6(3.3%) 174(96.7%)
Risk of bias within
studies		 19 159(88.3%) 21(1.7%)
Results of individual
studies		 20 126(70.0%) 54(30.0%)
Synthesis of results 21 123(68.3%) 57(31.7%)
Risk of bias across
studies		 22 104 (57.8%) 76(42.2%)
Additional analysis 23 134(74.4%) 46(25.6%)
Discussion
Summary of evidence 24 40(22.2%) 140(77.8%)
Limitations 25 87(48.3%) 93(51.7%)
Conclusions 26 174(96.7%) 6(3.3%)
Funding
Funding 27 140(77.8%) 40(22.2%)
Total 3280(56.9%) 2301(43.1%)
Open in a new tab

Table 4.

Comparison of compliance between JCR partitions of PRISMA-A.

Item no WOS level: zone 1–2 (n = 79) WOS level: zone 3–4 (n = 101) x2 P OR (95% CI)
Satisfied Satisfied rate Satisfied Satisfied rate
1 75 94.9% 98 97.0% 0.110 .740 1.742(0.378–8.021)
2 19 24.1% 16 15.8% 1.907 .167 0.594(0.283–1.249)
3 76 96.2% 99 98.0% 0.078 .780 1.954(0.318–11.987)
4 11 13.9% 25 24.8% 3.249 .071 2.033(0.931–4.440)
5 48 60.8% 51 50.5% 1.887 .170 0.659(0.363–1.196)
6 34 43.0% 51 50.5% 0.989 .320 1.350(0.747–2.440)
6a1 23 29.1% 23 22.8% 0.937 .333 0.718(0.367–1.406)
6a2 13 16.5% 18 17.8% 0.058 .810 1.101(0.503–2.410)
6b 53 67.1% 58 57.4% 1.751 .186 0.662(0.358–1.221)
6c 68 86.1% 86 85.1% 0.031 .861 0.927(0.400–2.149)
7 17 21.5% 16 15.8% 0.954 .329 0.687(0.322–1.464)
8 53 67.1% 68 67.3% 0.001 .973 1.011(0.540–1.892)
9 62 78.5% 78 77.2% 0.040 .841 0.930(0.457–1.891)
10 68 86.1% 87 86.1% <0.001 .990 1.005(0.429–2.354)
11 72 91.1% 87 86.1% 1.076 .300 0.604(0.231–1.577)
12 53 67.1% 61 60.4% 0.855 .355 0.748(0.404–1.385)
13 50 63.3% 75 74.3% 2.512 .113 1.673(0.883–3.170)
14 37 46.8% 48 47.5% 0.008 .927 1.028(0.570–1.854)
15 12 15.2% 9 8.9% 1.696 .193 0.546(0.281–1.370)
16 47 59.5% 60 59.4% <0.001 .991 0.996(0.547–1.815)
17 79 100.0% 101 100.0% – – –
18 77 97.5% 97 96.0% 0.012 .911 1.588(0.283–8.898)
18a 5 6.3% 1 1.0% 2.440 .118 0.148(0.017–1.294)
19 69 87.3% 90 89.1% 0.134 .714 1.186(0.476–2.952)
20 62 78.5% 64 63.4% 4.822 .028 0.474(0.242–0.929)
21 47 59.5% 76 75.2% 5.084 .024 2.070(1.094–3.914)
22 43 54.4% 61 60.4% 0.647 .421 1.277(0.704–2.317)
23 60 75.9% 74 73.3% 0.168 .682 0.868(0.440–1.711)
24 15 19.0% 25 24.8% 0.852 .356 1.404(0.682–2.887)
25 45 57.0% 42 41.6% 4.198 .040 0.538(0.296–0.976)
26 77 97.5% 97 96.0% 0.012 .911 0.630(0.112–3.530)
27 63 79.7% 77 76.2% 0.316 .574 0.815(0.399–1.665)
Open in a new tab

4. Discussion

The release of the PRISMA statement and PRISMA-A statement is to improve the integrity and transparency of acupuncture meta-analysis and promote researchers to truly present the study design and implementation, thus increasing the credibility of the results and promoting the use and dissemination of evidence. Although PRISMA and PRISMA-A checklists had been in use for over 10 years, compliance rate was still low (61.3% and 56.9%).

4.1. Problems in acupuncture meta-analysis report based on PRISMA statement

  • (1) Title

Title is the essence of an article,[18] and an article should have an appropriate title that conveys the main idea.[19] Authors should use informative titles to provide quick access to key information, as some practitioners with busy schedules may prefer to see the conclusions of the review in the title.[20] Irregularities in meta-analyses titles still exist, 7 studies (3.9%) were either not having the word “meta-analysis” in the title[21] or not reflecting the key information of the study.[22]

  • (2) Abstract

It is important to have a clear, transparent and sufficiently detailed abstract as readers often screen the abstract to decide whether to read the full text or not.[23] Structured abstracts provide readers with a series of headings pertaining to the purpose, conduct, findings, and conclusions of the systematic review being reported.[24] They give readers more complete information and facilitate finding information more easily than unstructured abstracts.[25–27] In our survey 145 papers (80.6%) failed to meet all the requirements, most of which lacked reports on sponsorship and registration,[28,29] or used traditional description methods.[30]

  • (3) Introduction

Readers desire to understand the rationale behind the study and it is contribution to already existing knowledge.[20] Therefore, sufficient background and purpose of the research is crucial. In our survey 97.2 % articles sufficiently reported the background, indicating that most researchers knew the research background and knowledge status in advance before conducting the research. However, in terms of research purpose reports, only 36 (20.0 %) studies evaluating the effects of interventions fully described the research content according to the PICO framework and this should arouse the author’s attention.

  • (4) Method

Strict inclusion and exclusion criteria can ensure that the research objects clearly represent the population.[31] For acupuncture trials, the most significant challenge often lies in the design of control groups.[32] Nevertheless, effective reporting is equally crucial for well-designed studies. However, only 85(47.2 %) of the studies adhered to the PICOS framework and linked the groups involved to the comparisons specified in the goals. It is important to strive for objectivity and to avoid errors in research selection.[20] Authors should report openly and transparently how the study was conducted,[33] to guarantee the credibility of the article. More than 77.8% of the included studies reported the screening, data collection process, and results accurately. However, only 43 (23.9%) gave a complete report of the data items, most of the studies did not report the collected results.[11]Since meta-analysis is a data consolidation method of quantitative systematic review, the quality of its conclusions depends not only on the strict operation process of meta-analysis, but also on the quality of the research literature and its control of bias.[34] The likelihood that the treatment effects reported in systematic evaluations are close to the truth depends on the validity of the included studies, as certain methodological characteristics may be related to effect sizes.[35] However, in our survey only 21 (11.7%) of studies standardized reporting bias and only 18 (10.0%) standardized reporting certainty. In terms of statistical analysis, meta-analysis is a statistical method that combines the results of several studies on the same topic into a single numerical estimate.[36] The statistical effect value and analysis measures are the necessary conditions for statistical analysis. The results of this study show deficiencies in the reporting: missing data or data conversion processing (23.9%), table graphics (35.6%), sensitivity analysis (43.3%), and heterogeneity measurement (48.3%), which should arouse the attention of the authors.

  • (5) Results

The result segment of the article is the core of medical articles, showing the outcome of the research, and the basis of argument and practical application. In our survey, only few studies reported the results on sensitivity analysis (38.3%) and certainty of evidence (31.3%) according to the standard. Most of the studies omit these reports because the design of the study itself does not support this study. However, we encourage researchers to report it realistically in the results section.

  • (6) Discussion

In the discussion section of the article, authors should provide a brief and balanced summary of the nature and findings of the review.[20] Reporting of study limitations should also be comprehensive. Poor or incomplete reporting of study design, patient populations, and interventions may hinder the interpretation and synthesis of included studies.[37] Researchers should provide objective statements and discussions of study limitations. Most of the included meta-analyses reported adequately on the general interpretation of the results (98.9%), limitations in the evidence (85.0%), and implications for future research (90.6%). There is a need to strengthen the reporting of procedural limitations of the study (54.4%).

  • (7) Other information

For trial registration, the prospective registration of systematic review and meta-analysis can improve transparency, reduce potential bias, and help to avoid accidental review duplication.[38] The results of this survey show that 55.0% of the included studies provided registration numbers, yet only 15.6% of the studies have provided the source or link of the review agreement, and only 0.6% of the studies reported amendments.

4.2. Problems in acupuncture meta-analysis report based on PRISMA-A statement

For RCT research, lack of clear diagnostic criteria will affect the treatment and may adversely affect the interpretation of results in clinical trials.[34] For the RCT studies included in meta-analysis, it is important to set clear diagnostic criteria. 25.6% of the studies described the diagnostic criteria in western medical terms,[39] and only 17.2% of the studies reported the diagnostic criteria in traditional terms or traditional Chinese medicine terms,[40] which reflected that the researchers were not rigorous in the design of the meta-analysis inclusion study scheme.

The needling sensation of “De-qi” during acupuncture is the key factor affecting the therapeutic effect of acupuncture.[41,42] Although 96.7% of the meta-analyses described the characteristics of the included studies, only 6 (3.3%) of the studies reported the details of their “De-qi.”[43] Yun, J. M. et al[44] did an excellent job in this regard, clearly describing in their meta-analysis study whether each RCT utilized “De-qi” techniques, providing readers with a comprehensive understanding of the design methods of these RCTs. However, there has yet to be a meta-analysis specifically quantifying the sensation of “De-qi.” Currently, many RCT studies have already quantified patients’ “De-qi sensations using the “De-qi composite,”[45] indicating that quantification of “De-qi” is feasible. For acupuncture, “De-qi” is an important experience that connect doctors and patients, but most authors of the study of acupuncture meta-analysis pay less attention to this, which calls for our attention. Furthermore, we advocate for new initiatives to quantitatively measure the sensation of “De-qi” in order to provide a more comprehensive report. Strength of the evidence is an important statistical tool, and not all evidence has the same strength.[46] Therefore, reporting the strength of evidence makes the research results more convincing. In the general interpretation of the results, only 22.2% of the research fully explained the results and described the strength of the evidence.[47] This reflects a limited understanding of the importance of this statistical tool.

4.3. Suggestions on improving the quality of acupuncture clinical research report

Meta-analysis stands as the pinnacle of clinical evidence, affording clinicians a lucid insight into the most efficacious clinical decision-making pathways available at present.[48] Yet, when such top-tier evidence lacks credibility, it can markedly sway clinical judgment. The standardized documentation of meta-analyses through the adherence to PRISMA and PRISMA-A guidelines empowers clinicians and researchers alike to replicate studies, validate findings, thereby bolstering research integrity and dependability. Through meticulous adherence to these guidelines in reporting, clinicians can discern which patients are primed to derive benefits from acupuncture interventions, aiding in refining diagnostic workflows and elevating diagnostic precision and efficiency. By juxtaposing the impacts of diverse acupuncture treatment modalities, meta-analyses furnish clinicians with evidence-based backing to ascertain optimal treatment regimens.[49] These fosters tailored interventions, amplifying patient treatment efficacy, and contentment. Furthermore, by methodically aggregating and scrutinizing findings across multiple studies, high-caliber meta-analyses furnish holistic evidence on the efficacy of acupuncture remedies. These evidential underpinning aids clinicians in prognosticating patient outcomes post-acupuncture, facilitating the formulation of more efficacious treatment blueprints.[50] The bedrock for enabling these functionalities lies in the meticulous reporting and comprehensive data encapsulated within meta-analyses. Furthermore, if all authors of the acupuncture meta-analysis adhere to all relevant elements of the PRISMA and PRISMA-A guidelines, it facilitates a clear clarification of the most extensive evidence, enabling more precise further synthesis, ensuring consistency of all key elements to form comprehensive overviews. Through clear reporting, feedback can also be provided to guideline developers, leading to the formulation of new guidelines and enhancing the comprehensiveness of the referenced information in guidelines.

With the increasing attention paid to the quality of meta-analyses reports in China and abroad, the integrity and transparency of acupuncture clinical research reports have improved to a certain extent. We should continue to standardize the report of acupuncture meta by adhering to the PRISMA and PRISMA-A statements. When writing an article for submission, we should strictly follow the standard list to write and report, so as to make the report detailed, true, and objective. The journal should provide guidelines for submission of acupuncture meta-analysis, and request the author to submit the list of research reports (including page numbers) together with the text of the article, so as to facilitate the editors and reviewers to review the integrity of the research report. Only those who meet the requirements (such as the report rate reaching 80%) can enter the formal review process.

4.4. Limitations

We used subjective qualitative evaluation of the items, that is, “yes” or “no,” which is rather not objective and comprehensive. The number of articles included in this study was limited to a period from 2020 to 2022. As a result, the quality of documents before 2020 was not evaluated. The source of the literature included in this study is only the PubMed database, which may lead to insufficient representation and reduce the credibility of this study. As this study is a cross-sectional analysis, all statistical results are descriptive and involve group comparison analyses. No further complex statistical analyses were conducted. In future research, additional statistical analyses such as trend analysis or multivariate analysis can be carried out to provide further guidance for acupuncture meta-analysis reporting.

5. Conclusion

To sum up, reporting the results of acupuncture meta-analysis in a standard form will help more researchers and clinical workers understand the complete scheme of the trial, and improve the external and internal authenticity and the operation level of the clinical trial. Based on the acupuncture meta-analyses reviewed, we found significant deficiencies in study details, reference basis for study design, analytical methods, rigor, and scientific validity in reports published in China and other countries over the past 3 years, regardless of the journal ranking (JCR quartile). This has contributed to acupuncture meta-analysis failing to reach the internationally recognized research level, thereby affecting the promotion of the value of acupuncture clinical research, and further failing to meet the clinical efficacy level. We hope researchers will try their best to standardize the design, implementation and reporting of studies in accordance with the standards of acupuncture meta-analyses, so as to improve the scope and recognition of acupuncture. We must strengthen education on the standardization of research reports.

Acknowledgments

We thank all the participants and clinical researchers involved in the publications cited in this study. All authors listed meet the authorship criteria according to the latest guidelines of the International Committee of Medical Journal Editors, and that all authors are in agreement with the manuscript.

Author contributions

Conceptualization: Xiujuan Xu, Conghua Ji.

Data curation: Chu Qin, Huan Ma, Oscar Onayi Mandizadza.

Formal analysis: Chu Qin, Huan Ma, Oscar Onayi Mandizadza.

Funding acquisition: Conghua Ji.

Investigation: Chu Qin, Huan Ma, Oscar Onayi Mandizadza.

Methodology: Xiujuan Xu, Conghua Ji.

Project administration: Conghua Ji.

Resources: Chu Qin.

Software: Chu Qin, Huan Ma.

Supervision: Huan Ma.

Validation: Huan Ma, Xiujuan Xu.

Visualization: Xiujuan Xu.

Writing – original draft: Chu Qin, Huan Ma, Oscar Onayi Mandizadza.

Writing – review & editing: Xiujuan Xu, Conghua Ji.

Abbreviations:

JCR
Journal Citation Reports
PRISMA
Systematic Reviews and Meta-Analyses
PRISMA-A
Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Acupuncture
Q1\2\3\4
quarter 1\2\3\4
RCTs
randomized controlled trials

CQ and HM contributed equally to this work.

This work was supported by the Education and Teaching Reform Project of Zhejiang Chinese Medicine University (No. ZB23005) and the Department of Science and Technology of Zhejiang Province (No. 2023C25012), the People’s Republic of China.

The authors have no conflicts of interest to disclose.

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

How to cite this article: Qin C, Ma H, Mandizadza OO, Xu X, Ji C. Reporting quality of meta-analyses in acupuncture: Investigating adherence to the PRISMA statement. Medicine 2024;103:39(e39933).

References

  • [1].Soni PD, Hartman HE, Dess RT, et al. Comparison of population-based observational studies with randomized trials in oncology. J Clin Oncol. 2019;37:1209–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [2].Spieth PM, Kubasch AS, Penzlin AI, Illigens BM, Barlinn K, Siepmann T. Randomized controlled trials – a matter of design. Neuropsychiatr Dis Treat. 2016;12:1341–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [3].Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21:125–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [4].Maeda Y, Kim H, Kettner N, et al. Rewiring the primary somatosensory cortex in carpal tunnel syndrome with acupuncture. Brain. 2017;140:914–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [5].Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [6].Baldry P. Complementary medicine. The practice of acupuncture needs tighter safeguards. Bmj. 1993;307:326. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [7].Eisenberg DM, Kessler RC, Foster C, Norlock FE, Calkins DR, Delbanco TL. Unconventional medicine in the United States. Prevalence, costs, and patterns of use. N Engl J Med. 1993;328:246–52. [DOI] [PubMed] [Google Scholar]
  • [8].Garland SN, Xie SX, DuHamel K, et al. Acupuncture versus cognitive behavioral therapy for insomnia in cancer survivors: a randomized clinical trial. J Natl Cancer Inst. 2019;111:1323–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [9].Xu S, Yu L, Luo X, et al. Manual acupuncture versus sham acupuncture and usual care for prophylaxis of episodic migraine without aura: multicentre, randomised clinical trial. Bmj. 2020;368:m697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [10].Tu JF, Yang J-W, Shi G-X, et al. Efficacy of intensive acupuncture versus sham acupuncture in knee osteoarthritis: a randomized controlled trial. Arthritis Rheumatol. 2021;73:448–58. [DOI] [PubMed] [Google Scholar]
  • [11].He Y, Guo X, May BH, et al. Clinical evidence for association of acupuncture and acupressure with improved cancer pain: a systematic review and meta-analysis. JAMA Oncol. 2020;6:271–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [12].Wang X, Chen Y, Liu Y, et al. Reporting items for systematic reviews and meta-analyses of acupuncture: the PRISMA for acupuncture checklist. BMC Complement Altern Med. 2019;19:208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Ma B, Chen Z-M, Xu J-K, et al. Do the CONSORT and STRICTA checklists improve the reporting quality of acupuncture and moxibustion randomized controlled trials published in Chinese Journals? A systematic review and analysis of trends. PLoS One. 2016;11:e0147244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [14].Oliveira NL, Botton CE, De Nardi AT, Umpierre D. Methodological quality and reporting standards in systematic reviews with meta-analysis of physical activity studies: a report from the Strengthening the Evidence in Exercise Sciences Initiative (SEES Initiative). Syst Rev. 2021;10:304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [15].Sarkis-Onofre R, Catalá-López F, Aromataris E, Lockwood C. How to properly use the PRISMA Statement. Syst Rev. 2021;10:117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [16].Fiorini N, Canese K, Starchenko G, et al. Best match: new relevance search for PubMed. PLoS Biol. 2018;16:e2005343. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [17].Oviedo-García MA. Journal citation reports and the definition of a predatory journal: the case of the Multidisciplinary Digital Publishing Institute (MDPI). Res Eval. 2021;30:405–19a. [Google Scholar]
  • [18].Wu B, Wang L, Lv S-X, Zeng Y-R. Forecasting oil consumption with attention-based IndRNN optimized by adaptive differential evolution. Appl Intell (Dordr). 2023;53:5473–96. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19].Yang Y, Hou M, Gong X, Guo R, Feng XL, Tian R. Quality assessment of hypertension treatment-related information on WeChat: cross-sectional study. J Med Internet Res. 2022;24:e38567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [20].Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [21].Chen Q, Wang Q, Ding S, et al. Problems lowering the study quality in traditional medicine, introspection from an example of meta-analysis of acupuncture. BMC Complement Med Ther. 2020;20:41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [22].Sarmiento-Hernández I, Pérez-Marín MLÁ, Nunez-Nagy S, Pecos-Martín D, Gallego-Izquierdo T, Sosa-Reina MD. Effectiveness of invasive techniques in patients with fibromyalgia: systematic review and meta-analysis. Pain Med. 2020;21:3499–511. [DOI] [PubMed] [Google Scholar]
  • [23].Hassoun-Kheir N, Stabholz Y, Kreft J-U, et al. EMBRACE-WATERS statement: recommendations for reporting of studies on antimicrobial resistance in wastewater and related aquatic environments. One Health. 2021;13:100339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [24].Mulrow CD, Thacker SB, Pugh JA. A proposal for more informative abstracts of review articles. Ann Intern Med. 1988;108:613–5. [DOI] [PubMed] [Google Scholar]
  • [25].Froom P, Froom J. Deficiencies in structured medical abstracts. J Clin Epidemiol. 1993;46:591–4. [DOI] [PubMed] [Google Scholar]
  • [26].Hartley J. Clarifying the abstracts of systematic literature reviews. Bull Med Libr Assoc. 2000;88:332–7. [PMC free article] [PubMed] [Google Scholar]
  • [27].Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. A survey of three medical journals. N Engl J Med. 1987;317:426–32. [DOI] [PubMed] [Google Scholar]
  • [28].von Trott P, Oei SL, Ramsenthaler C. Acupuncture for breathlessness in advanced diseases: a systematic review and meta-analysis. J Pain Symptom Manage. 2020;59:327–38.e3. [DOI] [PubMed] [Google Scholar]
  • [29].Luo S, Long Y, Xiao W, et al. Risk of bias assessments and reporting quality of systematic reviews and randomized controlled trials examining acupuncture for depression: an overview and meta-epidemiology study. J Evid Based Med. 2020;13:25–33. [DOI] [PubMed] [Google Scholar]
  • [30].Farag AM, Malacarne A, Pagni SE, Maloney GE. The effectiveness of acupuncture in the management of persistent regional myofascial head and neck pain: a systematic review and meta-analysis. Complement Ther Med. 2020;49:102297. [DOI] [PubMed] [Google Scholar]
  • [31].Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited. Ann Intern Med. 1990;113:69–76. [DOI] [PubMed] [Google Scholar]
  • [32].Fei Y-T, Cao H-J, Xia R-Y, et al. Methodological challenges in design and conduct of randomised controlled trials in acupuncture. BMJ. 2022;376:e064345. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [33].Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Stat Med. 2002;21:1635–40. [DOI] [PubMed] [Google Scholar]
  • [34].Fergusson D, Glass KC, Hutton B, Shapiro S. Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding. Clin Trials. 2005;2:218–29; discussion 229. [DOI] [PubMed] [Google Scholar]
  • [35].Pildal J, Hróbjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36:847–57. [DOI] [PubMed] [Google Scholar]
  • [36].Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327:248–54. [DOI] [PubMed] [Google Scholar]
  • [37].Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of treatment in trials and reviews? Bmj. 2008;336:1472–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [38].Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. CMAJ. 1994;150:1611–5. [PMC free article] [PubMed] [Google Scholar]
  • [39].Jing R, Feng K. Efficacy of intradermal acupuncture for insomnia: a meta-analysis. Sleep Med. 2021;85:66–74. [DOI] [PubMed] [Google Scholar]
  • [40].Hullender Rubin LE, Smith CA, Schnyer RN, Tahir P, Pasch LA. Effect of acupuncture on IVF-related anxiety: a systematic review and meta-analysis. Reprod Biomed Online. 2022;45:69–80. [DOI] [PubMed] [Google Scholar]
  • [41].Li MK, Li Y-J, Zhang G-F, et al. Acupuncture for ischemic stroke: cerebellar activation may be a central mechanism following Deqi. Neural Regen Res. 2015;10:1997–2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [42].Hu NJ, Lin C, Li J, et al. [Remarks on the relationship between deqi and effect of acupuncture]. Zhongguo Zhen Jiu. 2014;34:413–6. [PubMed] [Google Scholar]
  • [43].Zhao FY, Fu Q-Q, Kennedy GA, et al. Can acupuncture improve objective sleep indices in patients with primary insomnia? A systematic review and meta-analysis. Sleep Med. 2021;80:244–59. [DOI] [PubMed] [Google Scholar]
  • [44].Yun JM, Lee S-H, Cho J-H, Kim K-W, Ha I-H. The effects of acupuncture on occipital neuralgia: a systematic review and meta-analysis. BMC Complement Med Ther. 2020;20:171. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [45].Hui KK, Nixon EE, Vangel MG, et al. Characterization of the “deqi” response in acupuncture. BMC Complement Altern Med. 2007;7:33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [46].Bath FJ, Owen VE, Bath PM. Quality of full and final publications reporting acute stroke trials: a systematic review. Stroke. 1998;29:2203–10. [DOI] [PubMed] [Google Scholar]
  • [47].Shih YW, Su JY, Kung YS, et al. Effectiveness of acupuncture in relieving chemotherapy-induced leukopenia in patients with breast cancer: a systematic review with a meta-analysis and trial sequential analysis. Integr Cancer Ther. 2021;20:15347354211063884. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [48].Kum E, Patel M, Diab N, et al. Efficacy and tolerability of gefapixant for treatment of refractory or unexplained chronic cough: a systematic review and dose-response meta-analysis. JAMA. 2023;330:1359–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [49].Qin X, Coyle ME, Yang L, et al. Acupuncture for recurrent urinary tract infection in women: a systematic review and meta-analysis. BJOG. 2020;127:1459–68. [DOI] [PubMed] [Google Scholar]
  • [50].Liu Y, Fan J, Zhang X, et al. Transcutaneous electrical acupoint stimulation reduces postoperative patients’ length of stay and hospitalization costs: a systematic review and meta-analysis. Int J Surg. 2024;110:5124–35. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Medicine are provided here courtesy of Wolters Kluwer Health

RESOURCES