Abstract
Objective
To evaluate the reporting quality of randomized controlled trials (RCT) of acupuncture for knee osteoarthritis and explore factors associated with the reporting.
Study Design and Setting
Eight databases were searched from inception to August 2024 to assess the quality of acupuncture for knee osteoarthritis RCTs based on the CONSORT, the STRICTA, and the CONSORT-Outcomes. We performed regression analyses on pre-specified study characteristics to explore factors associated with reporting quality.
Results
One hundred and seventy-four RCTs were evaluated by 69 items from 3 checklists. Seventeen of 37 items on the CONSORT were under-reported (reported in less than 20% of RCTs), and the weakest reported item was why the trial ended or was stopped (0%). Four of 17 items on the STRICTA were under-reported, and the weakest reported item was the number of needle insertions per subject per session (9.2%). Eight of 17 items on the CONSORT-Outcomes were under-reported, and the weakest reported item was identifying any outcomes that were not pre-specified in a trial registry or trial protocol (0.6%). RCT locations include countries other than China, published in English, or funded were more likely to have better reporting.
Conclusion
RCTs of acupuncture for knee osteoarthritis need to focus more on reporting details of acupuncture interventions, the reporting of protocol amendment, and the complete reporting of outcome-related content. Journals should encourage authors to adhere strictly to reporting guidelines, which is necessary to improve the quality of reporting, which is very important for Chinese journals.
Keywords: randomized controlled trials, acupuncture, knee osteoarthritis, CONSORT, STRICTA, outcomes
Introduction
Clinical research can provide a decision-making basis for clinical practice, health policy, and further research, which determines that it needs to provide credible evidence.1–3 Randomized controlled trials (RCT) of low reporting quality may not only lead to invalid data or distorted results but also be unethical and harm the safety and rights of patients.3 Higher-quality reports may improve RCT interpretation and minimize biased conclusions.4 The CONSORT statement was developed to improve the reporting of RCTs.5,6 Most of the CONSORT statement applies equally to all trial designs, but there are a few additional issues to address for each design.7 Therefore, other reporting guidelines have been published to extend the CONSORT checklist. A total of 37 extension reporting guidelines for the CONSORT statement have been published, which can be found in Enhancing the QUAlity and Transparency Of health Research (EQUATOR). Among these extension reporting guidelines, the STRICTA guidelines for RCTs with the acupuncture intervention,8,9 and the CONSORT-Outcomes for outcomes.10 Adhering to appropriate reporting guidelines is crucial for enhancing the quality of an article.
Acupuncture is a commonly used traditional intervention for treating knee osteoarthritis in both China and developed countries.11–14 The total number of publications of RCTs of acupuncture for knee osteoarthritis continually increased these years.15 However, there is little information regarding the adherence of RCTs of acupuncture for knee osteoarthritis to the reporting guidelines. Additionally, in conducting a meta-analysis to assess the efficacy and safety of acupuncture for treating patients with knee osteoarthritis, we encountered difficulties in extracting information, which forced us to exclude articles with insufficient available information, resulting in a significant reduction in the studies ultimately included in the analyses.16,17 This may be related to the poor quality of reporting. Despite one study in 2018 that evaluated the reporting quality of RCTs of acupuncture for knee osteoarthritis,18 it is unclear whether there has been an improvement in the RCTs published after 2018. In addition, the study included a variety of acupuncture treatments and did not evaluate the CONSORT-Outcomes. Therefore, for more comprehensive reporting in future RCTs of manual and electroacupuncture for knee osteoarthritis, the primary objective of this study was to assess the overall reporting quality of RCTs of manual and electroacupuncture for knee osteoarthritis according to the CONSORT statement, CONSORT-Outcomes, and the STRICTA guidelines. Additionally, our study further explored factors associated with reporting quality.
Methods
Study Strategy
We searched eight databases from inception to August 2024, including PubMed, Embase, Cochrane Library databases, Web of Science, China, National Knowledge Infrastructure, Chinese Biomedical Literature Database, VIP Database for Chinese Technical Periodicals, and Wan Fang. The detailed search strategy for each database is shown in Additional file 1. The WHO International Clinical Trials Registry Platform the National Institutes of Health Clinical Registry ClinicalTrials.gov and the Chinese Clinical Registry were searched as a supplement.
Eligibility Criteria
We only included RCTs, which should enroll knee osteoarthritis patients, whose diagnostic criteria follow the American College of Rheumatology clinical criteria, National Institute for Health and Clinical Excellence guidelines, or any other accepted guidelines.19,20 The eligible interventions include manual acupuncture and electroacupuncture. It was deemed acceptable to utilize sham acupuncture, analgesics, usual care, or waiting lists in the control groups.
Data Extraction
We identified and removed the duplicate records (Liu CY, Wang Y) before the abstract screening. To ensure that all included RCTs met our criteria, Liu CY, Wang Y, Zhou H, and Duan YS first screened the abstract independently and then the full texts. A third reviewer (Wang LQ) helped to solve disagreements.
Data Collection
As shown in Additional file 2, from every qualified RCT, we gathered data on study characteristics, including the first author’s name, the year and language of publication, the journal name, the kind of financing (funded or unfunded), the location where the study was implemented, and the total number of centers. We counted RCTs publications before and after 2010 and 2022, respectively, as The CONSORT and STRICTA checklist was revised in 2010 and the CONSORT-Outcomes checklist was drafted in 2022.
Before the examination, two impartial reviewers underwent methodical training to guarantee they fully comprehended each issue. Depending on whether the author reports or not, they choose to answer “yes” or “no”. Items in the CONSORT-Outcomes scale that were deemed “not applicable” throughout the review process were those that specifically stated they were not relevant to all studies. The third reviewer addressed and ultimately verified any discrepancies. Lastly, descriptive statistics were employed to characterize the degree to which research followed the CONSORT, the STRICTA, and the CONSORT-Outcomes. These data included frequencies and percentages. The data was descriptively analyzed using Excel 2010.
Data Analysis
Overall Reporting Quality Rating
The full quality evaluation of the included RCTs was conducted using the overall quality score. For the calculation of the overall quality score, the item was awarded one point if it was reported or zero if it was not. Some items do not apply to all RCTs and need to be assessed based on applicability. These RCTs do not require reporting of inapplicable items. Therefore, RCTs that do not apply to this item are awarded 1 point when calculating the quality score. The maximum possible score for the overall quality score was 69 (the standard CONSORT checklist, STRICTA, and CONSORT-Outcomes were 37, 17, and 17 points). We also calculated scores for each of the evaluation scales.
Only a small proportion (less than 20%) of the RCTs reporting on the project were deemed under-reported. If over 80% of the RCTs were reported on the project, it was considered to be good-reported.18
Factors Associated with Reporting Quality
We identified five potentially relevant factors, including language (English vs Chinese), number of centers (single vs multicenter), financial support (yes vs no), research location (China vs other countries), and published time.
The study used the overall quality score as a dependent variable to investigate factors related to reporting quality. Only significant variables (p < 0.10) from the univariate analysis were included in the multivariate regression model (p < 0.05).21 Similarly, we investigated the factors that impact the quality of reporting by using the scores of each scale as the dependent variable separately.
Results
Study Searching and Selection
A total of 16347 publications about acupuncture for knee osteoarthritis were found, based on the search strategy. 7483 were not included before screening. A total of 174 relevant RCTs were ultimately included after 8249 papers were excluded after reading the title and abstract and 441 papers were removed after reading the entire text. The search and selection procedure for the RCT is depicted in Figure 1.
Figure 1.
Flow diagram of the study selection process.
Characteristics of Included Trials
These 174 RCTs were published between 1905 and 2024; of them, 141 (81.0%) were single center trials, 81 (46.6%) were funded, 148 (85.1%) were conducted in China, and 137 (78.7%) were published in Chinese (Table 1).
Table 1.
General Characteristics of Included RCTs
| Features of included RCTs | Total (n=174) |
Chinese (n=137) |
English (n=37) |
|---|---|---|---|
| Publication year No. (%) | |||
| Before 2010 (including 2010) | 32 (18.4) | 13 (9.5) | 19 (51.4) |
| After 2010 | 142 (81.6) | 124 (90.5) | 18 (48.6) |
| Before 2022 (including 2022) | 70 (40.2) | 125 (91.2) | 35 (94.6) |
| After 2022 | 104 (59.8) | 12 (8.8) | 2 (5.4) |
| Funding availability No. (%) | |||
| Funded | 81 (46.6) | 53 (38.7) | 28 (75.7) |
| Not Funded | 93 (53.4) | 84 (61.3) | 9 (24.3) |
| Location No. (%) | |||
| China | 148 (85.1) | 137 (100.0) | 11 (29.7) |
| Other countries | 26 (14.9) | 0 (0) | 26 (70.3) |
| Number of centers No. (%) | |||
| Single center | 141 (81.0) | 121 (88.3) | 20 (54.1) |
| Multicenter | 33 (19.0) | 16 (11.7) | 17 (45.9) |
Overall Reporting Quality According to CONSORT Statement
Eight of the CONSORT checklist’s 37 items had positive reports, accounting for more than 80% of the studies that were reported, including structured summary (96.6%), scientific background and explanation of rationale (90.8%), eligibility criteria for participants (83.9%), settings and locations where the data were collected (93.1%), sufficient details of each intervention (94.8%), defined pre-specified primary and secondary outcome measures (88.5%), statistical methods used to compare groups for primary and secondary outcomes (94.3%) and each group, number of participants (92.0%).
Seventeen projects were under-reported (less than 20% of the RCTs reported). Poorly reported items were: identification as a randomised trial in the title (19.0%), description of trial design including allocation ratio (16.7%), important changes made to the method after the trial commencement and reasons (2.9%), any changes to trial outcomes after the beginning of the trial and reasons (1.1%), how sample size was determined (15.5%), explanation of any interim analyses and stopping guidelines when applicable (1.7%), mechanism used to implement the random allocation sequence, describing any steps taken to conceal the sequence until interventions were assigned (17.2%), who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions (10.3%), who was blinded after assignment to interventions and how if done (18.4%), description of the similarity of interventions if relevant (18.4%), methods for additional analyses (6.9%), for binary outcomes presentation of both absolute and relative effect sizes is recommended (1.7%), results of any other analyses performed (6.3%), generalisability of the trial findings (14.9%), registration number and name of trial registry (9.2%), and where the full trial protocol can be accessed (5.2%). No trial has fully reported on why the trial ended or was stopped. The details are shown in Table 2.
Table 2.
Compliance of Reporting to the Standard CONSORT Checklist
| Item | Description | No. (%) of trials clearly reported | ||
|---|---|---|---|---|
| Total (n=174) |
Chinese (n=137) |
English (n=37) |
||
| 1a | Identification as a randomised trial in the title | 33 (19.0) | 5 (3.6) | 28 (75.7) |
| 1b | Structured summary of trial design, methods, results, and conclusions | 168 (96.6) | 135 (98.5) | 33 (89.2) |
| 2a | Scientific background and explanation of rationale | 158 (90.8) | 121 (88.3) | 37 (100.0) |
| 2b | Specific objectives or hypotheses | 138 (79.3) | 103 (75.2) | 35 (94.5) |
| 3a | Description of trial design including allocation ratio | 29 (16.7) | 14 (10.2) | 15 (40.5) |
| 3b | Important changes to methods after trial commencement, with reasons | 5 (2.9) | 0 (0) | 5 (13.5) |
| 4a | Eligibility criteria for participants | 146 (83.9) | 110 (80.3) | 36 (97.3) |
| 4b | Settings and locations where the data were collected | 162 (93.1) | 131 (95.6) | 31 (83.8) |
| 5 | The interventions for each group with sufficient details to allow replication | 165 (94.8) | 129 (94.2) | 36 (97.3) |
| 6a | Completely defined pre-specified primary and secondary outcome measures | 154 (88.5) | 118 (86.1) | 36 (97.3) |
| 6b | Any changes to trial outcomes after the trial commenced, with reasons | 2 (1.1) | 1 (0.7) | 1 (2.7) |
| 7a | How sample size was determined | 27 (15.5) | 3 (2.2) | 24 (64.9) |
| 7b | When applicable, explanation of any interim analyses and stopping guidelines | 3 (1.7) | 0 (0) | 3 (8.1) |
| 8a | Method used to generate the random allocation sequence | 87 (50.0) | 63 (46.0) | 24 (64.9) |
| 8b | Type of randomisation; details of any restriction | 66 (37.9) | 39 (28.5) | 27 (73.0) |
| 9 | Mechanism used to implement the random allocation sequence, describing any steps taken to conceal the sequence until interventions were assigned | 30 (17.2) | 6 (4.4) | 24 (64.9) |
| 10 | Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions | 18 (10.3) | 1 (0.7) | 17 (45.9) |
| 11a | If done, who was blinded after assignment to interventions and how | 32 (18.4) | 7 (5.1) | 25 (67.6) |
| 11b | If relevant, description of the similarity of interventions | 32 (18.4) | 9 (6.6) | 23 (62.2) |
| 12a | Statistical methods used to compare groups for primary and secondary outcomes | 164 (94.3) | 129 (94.2) | 35 (94.6) |
| 12b | Methods for additional analyses | 12 (6.9) | 0 (0) | 12 (32.4) |
| 13a | For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome | 121 (69.5) | 90 (65.7) | 31 (83.8) |
| 13b | For each group, losses and exclusions after randomisation, together with reasons | 46 (26.4) | 21 (15.3) | 25 (67.6) |
| 14a | Dates defining the periods of recruitment and follow-up | 139 (79.9) | 116 (84.7) | 23 (62.2) |
| 14b | Why the trial ended or was stopped | 0 (0) | 0 (0) | 0 (0) |
| 15 | A table showing baseline demographic and clinical characteristics for each group | 104 (59.8) | 75 (54.7) | 29 (78.4) |
| 16 | For each group, number of participants | 160 (92.0) | 127 (92.7) | 33 (89.2) |
| 17a | For each primary and secondary outcome, results for each group, and the estimated effect size and its precision | 125 (71.8) | 93 (67.9) | 32 (86.5) |
| 17b | For binary outcomes, presentation of both absolute and relative effect sizes is recommended | 3 (1.7) | 1 (0.7) | 2 (5.4) |
| 18 | Results of any other analyses performed | 11 (6.3) | 0 (0) | 11 (29.7) |
| 19 | All important harms or unintended effects in each group | 44 (25.3) | 23 (16.8) | 21 (56.8) |
| 20 | Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses | 41 (23.6) | 15 (10.9) | 26 (70.3) |
| 21 | Generalisability of the trial findings | 26 (14.9) | 17 (12.4) | 9 (24.3) |
| 22 | Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence | 104 (59.8) | 69 (50.4) | 35 (94.6) |
| 23 | Registration number and name of trial registry | 16 (9.2) | 3 (2.2) | 13 (35.1) |
| 24 | Where the full trial protocol can be accessed, if available | 9 (5.2) | 2 (1.5) | 7 (18.9) |
| 25 | Sources of funding and other support, role of funders | 57 (32.8) | 33 (24.1) | 24 (64.9) |
Overall Reporting Quality According to STRICTA Statement
As shown in Table 3, six items were good-reported, including the extent to which treatment was varied (88.5%), needle stimulation (94.3%), needle retention time (92.5%), number of treatment sessions (87.9%), frequency and duration of treatment sessions (95.4%), and precise description of the control or comparator (97.1%). At the same time, four items were under-reported, including the number of needle insertions per subject per session (9.2%), setting and context of treatment (12.1%), description of participating acupuncturists (16.7%), and rationale for the control or comparator in the context of the research question with sources (17.8%).
Table 3.
Compliance of Reporting to the STRICTA Checklist
| Item | Description | No. (%) of trials clearly reported | ||
|---|---|---|---|---|
| Total (n=174) |
Chinese (n=137) |
English (n=37) | ||
| 1a | Style of acupuncture | 55 (31.6) | 36 (26.2) | 19 (51.4) |
| 1b | Reasoning for treatment provided | 80 (46.0) | 49 (35.8) | 31 (83.8) |
| 1c | Extent to which treatment was varied | 154 (88.5) | 121 (88.3) | 33 (89.2) |
| 2a | Number of needle insertions per subject per session | 16 (9.2) | 4 (2.9) | 12 (32.4) |
| 2b | Names of points used | 65 (37.4) | 45 (32.8) | 20 (54.1) |
| 2c | Depth of insertion | 81 (46.6) | 55 (40.1) | 26 (70.3) |
| 2d | Response sought | 121 (69.5) | 95 (69.3) | 26 (70.3) |
| 2e | Needle stimulation | 164 (94.3) | 131 (95.6) | 33 (89.2) |
| 2f | Needle retention time | 161 (92.5) | 128 (93.4) | 33 (89.2) |
| 2g | Needle type | 66 (37.9) | 51 (37.2) | 15 (40.5) |
| 3a | Number of treatment sessions | 153 (87.9) | 120 (87.6) | 33 (89.2) |
| 3b | Frequency and duration of treatment sessions | 166 (95.4) | 132 (96.4) | 34 (91.9) |
| 4a | Details of other interventions administered to the acupuncture group | 62 (35.6) | 56 (40.9) | 6 (16.2) |
| 4b | Setting and context of treatment | 21 (12.1) | 1 (0.7) | 20 (54.1) |
| 5 | Description of participating acupuncturists | 29 (16.7) | 5 (3.6) | 24 (64.9) |
| 6a | Rationale for the control or comparator in the context of the research question with sources | 31 (17.8) | 18 (13.1) | 13 (35.1) |
| 6b | Precise description of the control or comparator | 169 (97.1) | 136 (99.3) | 33 (89.2) |
Overall Reporting Quality According to CONSORT-Outcomes Statement
The quality of the 174 included studies was evaluated based on the CONSORT-Outcomes statement (Table 4). Among the 17 items, only 4 were good-reported, while 8 were under-reported. The poorly reported items including outcomes that were not prespecified (0.6%), personnel allocation and qualification or training for outcome assessment (2.9%), processes used to promote outcome data quality during data collection and after data collection (8.0%), define and justify the target difference between treatment groups (9.8%), methods used to account for multiplicity of the primary and secondary outcomes (10.9%), methods used to assess patterns of missingness (9.2%), outcome analysis population relating to nonadherence of the trial protocol (9.2%), explain why they were performed if any analyses were not prespecified (16.7%).
Table 4.
Compliance of Reporting to the CONSORT-Outcomes Checklist
| Item | Description | No. of trials clearly reported/ trials applicable to item (%) | ||
|---|---|---|---|---|
| Total | Chinese | English | ||
| 6a.1 | Provide a rationale for the selection of the domain for the trial’s primary outcome | 172/174 (98.9) | 135/137 (98.5) | 37/37 (100.0) |
| 6a.2 | Describe the specific measurement variable, analysis metric, method of aggregation, and the time point for each outcome | 137/174 (78.7) | 104/137 (75.9) | 33/37 (89.2) |
| 6a.3* | If the analysis metric for the primary outcome represents within-participant change, define and justify the minimal important change in individuals | 6/25 (24.0) | 2/19 (10.5) | 4/6 (66.7) |
| 6a.4* | If the outcome data were continuous, but were analyzed as categorical, specify the cutoff values used | 38/39 (97.4) | 34/35 (97.1) | 4/4 (100.0) |
| 6a.5* | If outcome assessments were performed at several time points after randomization, state the time points used for the analysis | 93/95 (97.9) | 66/67 (98.5) | 27/28 (96.4) |
| 6a.6* | If a composite outcome was used, define all individual components of the composite outcome | 32/32 (100.0) | 21/21 (100.0) | 11/11 (100.0) |
| 6a.7 | Identify any outcomes that were not prespecified in a trial registry or trial protocol | 1/174 (0.6) | 0/137 (0) | 1/37 (2.7) |
| 6a.8 | Provide a description of the study instruments used to assess the outcome | 86/174 (49.4) | 64/137 (46.7) | 22/37 (59.5) |
| 6a.9 | Describe who assessed the outcome and any qualifications | 5/174 (2.9) | 3/137 (2.2) | 2/37 (5.4) |
| 6a.10 | Describe any processes used to promote outcome data quality during data collection and after data collection | 14/174 (8.0) | 0/137 (0) | 14/37 (37.8) |
| 7a.1 | Define and justify the target difference between treatment groups | 17/174 (9.8) | 3/137 (2.2) | 14/37 (37.8) |
| 12a.1 | Describe any methods used to account for multiplicity in the analysis | 19/174 (10.9) | 1/137 (0.7) | 18/37 (48.6) |
| 12a.2 | State and justify any criteria for excluding any outcome data from the analysis and reporting | 40/174 (23.0) | 19/137 (13.9) | 21/37 (56.8) |
| 12a.3 | Describe the methods used to assess patterns of missingness, and describe the methods used to handle missing outcome items or entire assessments | 16/174 (9.2) | 1/137 (0.7) | 15/37 (40.5) |
| 12a.4 | Provide a definition of the outcome analysis population relating to nonadherence of the trial protocol | 16/174 (9.2) | 10/137 (7.3) | 6/37 (16.2) |
| 17a.1 | Include the results for all prespecified outcome analyses | 94/174 (54.0) | 70/137 (51.1) | 24/37 (64.9) |
| 18.1* | If there were any analyses that were not prespecified, explain why they were performed | 1/6 (16.7) | 0/0 (0) | 1/6 (16.7) |
Note: *This item is not applicable to all trails. For these items, the percentage of trials in which item was clearly reported was calculated by the number of positive trials and the total number of applicable trials.
Factors Associated with the Reporting Quality
The results of the univariate analyses showed that language, funding, and location were factors affecting both the overall quality of reporting of RCTs and the quality of reporting of CONSORT, STRICTA, and CONSORT-Outcomes (p < 0.10). There were differences in the quality of RCTs before and after the publication of the CONSORT and CONSORT -Outcomes. The results of the univariate analyses are detailed in Additional file 3.
As shown in Table 5, RCTs published in the English language (β coefficient −5.507, 95% CI −8.566 to −2.448), study locations include countries other than China (β coefficient 7.696, 95% CI 4.566 to 10.827) and funded (β coefficient −5.957, 95% CI −7.466 to −4.134) were found to be statistically associated with an elevated overall quality score, which is indicative of superior overall reporting quality. The CONSORT, the STRICTA, and the CONSORT-Outcomes scores were analyzed with the same results as the overall quality score. Moreover, studies conducted after 2010 (β coefficient 4.544, 95% CI 2.723 to 6.365) related to better quality of the CONSORT.
Table 5.
Results of Multivariable Regression Analyses of Factors Predicting Reporting Quality
| Variables | β Coefficient (95% CI) | p |
|---|---|---|
| Factors associated with overall reporting quality | ||
| Language of publication | ||
| English vs Chinese | −5.507 (−8.566 to −2.448) | <0.001 |
| Funding | ||
| Funded vs unfunded | −5.957 (−7.466 to −4.134) | <0.001 |
| Location | ||
| China vs other countries | 7.696 (4.566 to 10.827) | <0.001 |
| Factors associated with the standard CONSORT | ||
| Language of publication | ||
| English vs Chinese | −2.780 (−5.146 to −0.414) | 0.022 |
| Funding | ||
| Funded vs unfunded | −4.185 (−5.500 to −2.870) | <0.001 |
| Location | ||
| China vs other countries | 8.912 (6.526 to 11.298) | <0.001 |
| Publication year | ||
| Before 2010 (including 2010) vs after 2010 | 4.544 (2.723 to 6.365) | <0.001 |
| Factors associated with the standard STRICTA | ||
| Language of publication | ||
| English vs Chinese | −2.362 (−3.568 to −1.157) | <0.001 |
| Funding | ||
| Funded vs unfunded | −0.764 (−1.449 to −0.080) | 0.029 |
| Location | ||
| China vs other countries | −1.028 (−2.261 to 0.205) | 0.102 |
| Factors associated with the standard CONSORT-Outcomes | ||
| Language of publication | ||
| English vs Chinese | −1.511 (−2.264 to −0.758) | <0.001 |
| Funding | ||
| Funded vs unfunded | −0.725 (−1.139to −0.311) | 0.001 |
| Location | ||
| China vs other countries | 1.020 (0.243 to 1.797) | 0.010 |
| Publication year | ||
| Before 2022 (including 2022) vs after 2022 | −0.713 (−1.478 to 0.052) | 0.068 |
Discussion
This study employed a systematic approach to evaluate the reporting quality of RCTs of acupuncture for knee osteoarthritis and to identify associated factors. The results showed that overall, the quality of reporting is poor, but some items were reported well. Of a total of 37 items in the CONSORT statement, 8 were good-reported. These items include: “structured abstracts”, “background and rationale”, “eligibility criteria for participants”, “settings and locations”, “interventions details”, “complete outcomes measures”, “statistical methods for outcomes”, and “numbers analyzed”. The STRICTA statement’s overall reporting quality was the best (6 items good-reported among 17 items in total). The reporting quality of the CONSORT-Outcomes was poor (8 items under-reported among 17 items in total). This could be due to the recent publication of the CONSORT-Outcomes (published in 2022), but it also suggests that the CONSORT-Outcomes is currently receiving insufficient attention.
In contrast with the study published in 2018,18 the reporting quality has improved. For the under-reported items in the CONSORT standard, some reporting quality showed significant improvement. These items include: “completely defined pre-specified outcomes”, “type of randomisation”, “participant flow”, and “limitations”. While 15 items were still poor (less than 20% of the RCTs reported in both our study and the study published in 2018). In STRICTA, items “acupuncture rationale”, and “details of other interventions” reporting quality improved, but three items were still under-reported. The enhancement of some items noted that the publication of guidelines for reporting and studies evaluating the quality of reports contributed to the improvement in the quality of reporting. Although there has been an improvement in the quality of reporting outcome-related items, we still encountered issues with the lack of detailed reporting of outcome-related information during our meta-information extraction process. Therefore, a thorough evaluation of outcomes is necessary to address the current reporting problems and enhance the quality of reporting. Regrettably, previous studies did not offer a distinct assessment of the outcome reporting quality. Therefore, with the addition of the new RCTs, our study conducted a new evaluation based on this topic, considering only the more clinically used manual acupuncture and electroacupuncture interventions and adding a separate evaluation for outcome.
Mere eight items were found to have been good-reported (over 80% of the RCTs were reported on the project) on the CONSORT statement, which comprises a total of 36 items. The results of our study indicated a lack of reporting about amendment of the protocol in RCTs (including items 3b, 6b, and 18). Clarifying amendments to the protocol of RCTs can reduce selection and confounding bias in the research process, improving the quality and authenticity of RCTs.22–24 Items about protocol and registration were poorly reported (including items 23 and 24). Published or registered completed research protocols are crucial as they pre-specify the methods of the RCT, helping to reduce the possibility of undeclared post hoc changes to the trial methods and selective outcome reporting.22,25–29 The low quality of reporting for methodologically relevant items is not conducive to determining the credibility of the RCTs, restricting the use for meta-research (including items 7a, 7b, 9, 10, 11a, 11b), especially concealment of allocation and appropriate blinding, which are crucial for avoiding bias and subsequent distortion of the estimation of the effect.30 Moreover, the key details of sample size determination should be adequately described to inform readers whether the trial has sufficient subjects to detect statistically significant differences between intervention and control groups.31
According to the STRICTA, the overall reporting quality is good, while the quality of reporting on the details of acupuncture and control interventions was low (items 2a, 4b, 6a), which could lead to studies that are difficult to replicate, which can affect their clinical reproducibility. Practitioner qualifications (item 5) were severely lacking. Given the complexity of needling as an intervention, it is crucial to provide a comprehensive report of all intervention details, which is important for the determination of suitability or future clinical implementation of that technique.18,21 The low reporting rate of these items may be attributed to limitations in article length or the opinions of the reviewing experts. However, the page length was found to be associated with reporting quality.32,33 For acupuncture RCTs, journals may consider appropriately increasing the word count requirement or requesting authors to include content related to acupuncture details in an appendix to make the research details available to a wider range of readers.
Poor reporting of outcomes across RCTs can be identified based on the results of the CONSORT-Outcomes, 8 of the 17 items were under-reported. It is of great importance that all outcomes of a trial are fully reported, as this allows for the replication of results, the synthesis of knowledge, and the prevention of the selective non-reporting of results.10 Consistent with CONSORT standard findings in our study, CONSORT-Outcomes findings persistently have problems reporting on protocol amendment and description of the method used to determine the sample size (items 6a.7, 7a.1, and 18.1). It is crucial to include any outcomes that were not initially declared in trial reports to prevent any potential perception of reporting bias.34,35 For the determination of sample size, different from the CONSORT statement, the objective of the CONSORT-Outcomes is to encourage authors to report the specific difference that was used to inform the trial’s sample size calculation.10 Given the potential for differing pragmatic or clinical factors to inform the selected target difference and the existence of numerous alternative options, it is essential to provide a rationale for the chosen target difference.10,36–38 Meanwhile, descriptions of who assessed the outcome with their qualifications or trial-specific training and the processes employed to ensure the quality of outcome data (items 6a.9 and 6a.10) are important in the improvement of reporting quality, providing transparency, and facilitating appraisal of the quality of the trial’s data. The description of statistical methods used to compare groups for primary and secondary outcomes was not specific enough (items 12a.1, 12a.3, 12a.4), including the techniques employed to accommodate multiplicity in the analysis or interpretation of the primary and secondary outcomes.
Our regression analyses showed that the study location is China and Chinese-published RCTs had lower reporting quality. This aligns with previous studies on reporting quality in different areas.39–44 This is related to the fact that English journals are more likely to ask authors to follow the reporting quality scale during the submission process.45 Suggesting Chinese journals need to focus on the use of reporting quality scales. Moreover, to provide more detailed information regarding the intervention, Chinese journals can allow for the addition of articles. Furthermore, in alignment with prior findings, clinical trials that were funded demonstrated superior study design and higher reporting quality.18,46
Our study also has limitations. Firstly, our study was not comprehensive enough as it only included RCTs where the intervention was commonly used in clinical acupuncture (manual acupuncture or electroacupuncture). Secondly, we only included RCTs published in Chinese and English. The publication of articles in other languages is neglected, which may affect the comprehensiveness of the evaluation of the quality of the reports. Thirdly, we think that different entries are of equal importance when calculating the total score, however, this may not be the case. Fourthly, as both STRICTA and CONSORT-Outcomes are extension scales to the CONSORT statement, there is some overlap in the items. However, given the distinct criteria for each item, we have evaluated every item of the three scales. Additionally, when we calculated the overall quality score, 1 point was awarded for inapplicable items in some RCTs, which is somewhat optimistic and may impact the results. It is worthwhile to explore the method of inapplicable items in future studies.
Conclusions
Evaluation of previous publications from database inception to August 2024 showed that RCTs of acupuncture for knee osteoarthritis need to focus more on the reporting of details of acupuncture interventions, the reporting of protocol amendment, and the complete reporting of outcome-related content. To improve the reporting quality, journals, especially Chinese journals, should encourage strict adherence to the reporting guidelines.
Acknowledgments
This research was supported by the National Natural Science Foundation of China (grant number 82004223). The funder had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Data Sharing Statement
If you need the original data, please contact Li-Qiong Wang and Cun-Zhi Liu, email: wangliqiongwork@163.com, and lcz623780@126.com.
Author Contributions
All authors read and approved the final manuscript. All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis, and interpretation, or in all these areas; took part in drafting, revising, or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.
Disclosure
The authors report no conflicts of interest in this work.
The abstract of this paper was presented at the SAR/RCMI PolyU International Research Conference as a poster presentation with interim findings. The poster’s abstract was published in ‘Poster Abstracts’ in the Journal of Integrative Medicine: Abstracts for SAR/RCMI PolyU International Research Conference, Journal of Integrative Medicine, Volume 22, Issue 3, 2024, Pages I-LXXVI, ISSN 2095-4964, https://doi.org/10.1016/S2095-4964(24)00328-5.
References
- 1.Ioannidis JPA. Why Most Clinical Research Is Not Useful. PLoS Med. 2016;13(6):e1002049. doi: 10.1371/journal.pmed.1002049 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.von Niederhäusern B, Guyatt GH, Briel M, Pauli-Magnus C. Academic response to improving value and reducing waste: a comprehensive framework for INcreasing QUality In patient-oriented academic clinical REsearch (INQUIRE). PLoS Med. 2018;15(6):e1002580. doi: 10.1371/journal.pmed.1002580 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Zhang N, Tu JF, Lin Y, et al. Overall Reporting Descriptions of Acupuncture for Chronic Pain in Randomized Controlled Trials in English Journals. J Pain Res. 2021;Volume 13:2369–2379. doi: 10.2147/JPR.S258739 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Moher D, Jones A, Lepage L, the CONSORT Group F. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA. 2001;285(15):1992–1995. doi: 10.1001/jama.285.15.1992 [DOI] [PubMed] [Google Scholar]
- 5.Moher D, Schulz KF, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285(15):1987–1991. doi: 10.1001/jama.285.15.1987 [DOI] [PubMed] [Google Scholar]
- 6.Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. doi: 10.1136/bmj.c332 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. doi: 10.1136/bmj.c869 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.MacPherson H, White A, Cummings M, Jobst K, Rose K, Niemtzow R. Standards for reporting interventions in controlled trials of acupuncture: the STRICTA recommendations. Complement Ther Med. 2001;9(4):246–249. doi: 10.1054/ctim.2001.0488 [DOI] [PubMed] [Google Scholar]
- 9.MacPherson H, Altman DG, Hammerschlag R, et al. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. PLoS Med. 2010;7(6):e1000261. doi: 10.1371/journal.pmed.1000261 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Butcher NJ, Monsour A, Mew EJ, et al. Guidelines for Reporting Outcomes in Trial Reports: the CONSORT-Outcomes 2022 Extension. JAMA. 2022;328(22):2252–2264. doi: 10.1001/jama.2022.21022 [DOI] [PubMed] [Google Scholar]
- 11.He J, Du L, Liu G, et al. Quality assessment of reporting of randomization, allocation concealment, and blinding in traditional Chinese medicine RCTs: a review of 3159 RCTs identified from 260 systematic reviews. Trials. 2011;12:122. doi: 10.1186/1745-6215-12-122 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Magden ER, Haller RL, Thiele EJ, Buchl SJ, Lambeth SP, Schapiro SJ. Acupuncture as an adjunct therapy for osteoarthritis in chimpanzees (Pan troglodytes). J Am Assoc Lab Anim Sci. 2013;52(4):475–480. [PMC free article] [PubMed] [Google Scholar]
- 13.Lin LL, Tu JF, Wang LQ, et al. Acupuncture of different treatment frequencies in knee osteoarthritis: a pilot randomised controlled trial. Pain. 2020;161(11):2532–2538. doi: 10.1097/j.pain.0000000000001940 [DOI] [PubMed] [Google Scholar]
- 14.Tu JF, Yang JW, Shi GX, et al. Efficacy of Intensive Acupuncture Versus Sham Acupuncture in Knee Osteoarthritis: a Randomized Controlled Trial. Arthritis Rheumatol. 2021;73(3):448–458. doi: 10.1002/art.41584 [DOI] [PubMed] [Google Scholar]
- 15.Li R, Sun J, Hu H, et al. Research Trends of Acupuncture Therapy on Knee Osteoarthritis from 2010 to 2019: a Bibliometric Analysis. J Pain Res. 2020;13:1901–1913. doi: 10.2147/JPR.S258739 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Liu CY, Duan YS, Zhou H, et al. Clinical effect and contributing factors of acupuncture for knee osteoarthritis: a systematic review and pairwise and exploratory network meta-analysis; 2024.
- 17.Liu CY, Tu JF, Lee MS, et al. Is acupuncture effective for knee osteoarthritis? A protocol for a systematic review and meta-analysis. BMJ Open. 2022;12(1):e052270. doi: 10.1136/bmjopen-2021-052270 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jia P, Tang L, Yu J, Liu J, Kang D, Sun X. The quality of reporting in randomized controlled trials of acupuncture for knee osteoarthritis: a cross-sectional survey. PLoS One. 2018;13(4):e0195652. doi: 10.1371/journal.pone.0195652 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Hochberg MC, Altman RD, Brandt KD, et al. Guidelines for the medical management of osteoarthritis. Part II. Osteoarthritis of the knee. American Coll Rheum Arthritis Rheum. 1995;38(11):1541–1546. doi: 10.1002/art.1780381104 [DOI] [PubMed] [Google Scholar]
- 20.KELLGREN JH, LAWRENCE JS. Radiological assessment of osteo-arthrosis. Ann Rheum Dis. 1957;16(4):494–502. doi: 10.1136/ard.16.4.494 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Liu K, Zeng J, Pei W, et al. Assessing the reporting quality in randomized controlled trials of acupuncture for postherpetic neuralgia using the CONSORT statement and STRICTA guidelines. J Pain Res. 2019;12:2359–2370. doi: 10.2147/JPR.S210471 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Grant AM, Altman DG, Babiker AB, et al. Issues in data monitoring and interim analysis of trials. Health Technol Assess. 2005;9(7):1–238. doi: 10.3310/hta9070 [DOI] [PubMed] [Google Scholar]
- 23.Pildal J, Chan AW, Hróbjartsson A, Forfang E, Altman DG, Gøtzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study. BMJ. 2005;330(7499):1049. doi: 10.1136/bmj.38414.422650.8F [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chan AW, Hróbjartsson A, Jørgensen KJ, Gøtzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ. 2008;337:a2299. doi: 10.1136/bmj.a2299 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Dargie HJ. Effect of carvedilol on outcome after myocardial infarction in patients with left-ventricular dysfunction: the CAPRICORN randomised trial. Lancet. 2001;357(9266):1385–1390. [DOI] [PubMed] [Google Scholar]
- 26.Tukey JW. Some thoughts on clinical trials, especially problems of multiplicity. Science. 1977;198(4318):679–684. doi: 10.1126/science.333584 [DOI] [PubMed] [Google Scholar]
- 27.Hahn S, Williamson PR, Hutton JL, Garner P, Flynn EV. Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies. Stat Med. 2000;19(24):3325–3336. doi: [DOI] [PubMed] [Google Scholar]
- 28.Williamson PR, Gamble C. Identification and impact of outcome selection bias in meta-analysis. Stat Med. 2005;24(10):1547–1561. doi: 10.1002/sim.2025 [DOI] [PubMed] [Google Scholar]
- 29.Tramèr MR, Reynolds DJ, Moore RA, McQuay HJ. Impact of covert duplicate publication on meta-analysis: a case study. BMJ. 1997;315(7109):635–640. doi: 10.1136/bmj.315.7109.635 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–412. doi: 10.1001/jama.1995.03520290060030 [DOI] [PubMed] [Google Scholar]
- 31.Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005;365(9467):1348–1353. doi: 10.1016/S0140-6736(05)61034-3 [DOI] [PubMed] [Google Scholar]
- 32.Kim KH, Kang JW, Lee MS, Lee JD. Assessment of the quality of reporting for treatment components in Cochrane reviews of acupuncture. BMJ Open. 2014;4(1):e004136. doi: 10.1136/bmjopen-2013-004136 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Liu XT, Zhang X, Wen S, Peng L, Hong Q, Kang D. Impact of the Consolidated Standards of Reporting Trials (CONSORT) checklist on reporting of randomized clinical trials in traditional Chinese medicine. J Evid Based Med. 2015;8(4):192–208. doi: 10.1111/jebm.12173 [DOI] [PubMed] [Google Scholar]
- 34.Goldacre B, Drysdale H, Dale A, et al. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20(1):118. doi: 10.1186/s13063-019-3173-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13:282. doi: 10.1186/s12916-015-0520-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Cook JA, Hislop J, Adewuyi TE, et al. Assessing methods to specify the target difference for a randomised controlled trial: DELTA (Difference ELicitation in TriAls) review. Health Technol Assess. 2014;18(28):v–vi,1–175. doi: 10.3310/hta18280 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Woaye-Hune P, Hardouin JB, Lehur PA, Meurette G, Vanier A. Practical issues encountered while determining Minimal Clinically Important Difference in Patient-Reported Outcomes. Health Qual Life Out. 2020;18(1):156. doi: 10.1186/s12955-020-01398-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Mouelhi Y, Jouve E, Castelli C, Gentile S. How is the minimal clinically important difference established in health-related quality of life instruments? Review of anchors and methods. Health Qual Life Out. 2020;18(1):136. doi: 10.1186/s12955-020-01344-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Kim KH, Kang JW, Lee MS, Lee JD. Assessment of the quality of reporting in randomised controlled trials of acupuncture in the Korean literature using the CONSORT statement and STRICTA guidelines. BMJ Open. 2014;4(7):e005068. doi: 10.1136/bmjopen-2014-005068 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Ma B, Chen ZM, Xu JK, et al. Do the CONSORT and STRICTA Checklists Improve the Reporting Quality of Acupuncture and Moxibustion Randomized Controlled Trials Published in Chinese Journals? System Rev AnalysTren PLoS One. 2016;11(1):e0147244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Zhang Y, Zhang RM, Chang J. Quality assessment of the report of randomized controlled trials on treatment of liver carcinoma with traditional Chinese medicine. Zhongguo Zhong Xi Yi Jie He Za Zhi. 2008;28(7):588–590. [PubMed] [Google Scholar]
- 42.Adetugbo K, Williams H. How well are randomized controlled trials reported in the dermatology literature. Arch Dermatol. 2000;136(3):381–385. doi: 10.1001/archderm.136.3.381 [DOI] [PubMed] [Google Scholar]
- 43.Adie S, Harris IA, Naylor JM, Mittal R. CONSORT compliance in surgical randomized trials: are we there yet? A systematic review. Ann Surg. 2013;258(6):872–878. doi: 10.1097/SLA.0b013e31829664b9 [DOI] [PubMed] [Google Scholar]
- 44.Halpern SH, Darani R, Douglas MJ, Wight W, Yee J. Compliance with the CONSORT checklist in obstetric anaesthesia randomised controlled trials. Int J Obstet Anesth. 2004;13(4):207–214. doi: 10.1016/j.ijoa.2004.03.009 [DOI] [PubMed] [Google Scholar]
- 45.Hopewell S, Altman DG, Moher D, Schulz KF. Endorsement of the CONSORT Statement by high impact factor medical journals: a survey of journal editors and journal ‘Instructions to Authors’. Trials. 2008;9:20. doi: 10.1186/1745-6215-9-20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Al-Badriyeh D, Alameri M, Al-Okka R. Cost-effectiveness research in cancer therapy: a systematic review of literature trends, methods and the influence of funding. BMJ Open. 2017;7(1):e012648. doi: 10.1136/bmjopen-2016-012648 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
If you need the original data, please contact Li-Qiong Wang and Cun-Zhi Liu, email: wangliqiongwork@163.com, and lcz623780@126.com.