A comprehensive quality analysis of randomized controlled clinical trials of Asian ginseng and American ginseng based on the CONSORT guideline

Abstract

Ginseng is an international herb that has been used for thousands of years. Two species most commonly applied and investigated in the ginseng family are Asian ginseng and American ginseng. The number of randomized controlled clinical trials (RCTs) has conspicuously increased, driven by the rapid development of ginseng. However, the reporting of RCT items of ginseng is deficient because of different trial designs and reporting formats, which is a challenge for researchers who are looking for the data with high quality and reliability. Thus, this study focused on providing an extensive analysis of these two species and examined the quality of the RCTs, based on the Consolidated Standards of Reporting Trials (CONSORT) guideline. Ninety-one RCTs conducted from 1980 to 2019 that were related to Asian ginseng and American ginseng used singly met our inclusion criteria. We found that the reporting quality of the two species has improved during the past 40 years. Publication date and sample size were significantly associated with the reporting quality. Rigorous RCTs designed for the species of ginseng are warranted, which can shed light on product research and development of ginseng in the future.

Graphical abstract

1. Introduction

Ginseng is an international herb from the Araliaceae family. It has a long history of use as a traditional medicine in Asian countries, and is one of the most popular medicinal herbs. The ginseng family has 11 known species. However, two species are most commonly applied and investigated in RCTs: Panax ginseng Meyer (PG) (Asian ginseng) and Panax quinquefolius L. (PQ) (American ginseng) [[1], [2], [3], [4]]. PG is a perennial herb that grows in the mountains of East Asia, and is called Rénshēn in Chinese, Insam in Korean and Ninjin in Japanese. PQ is indigenous to eastern North America, although it is also cultivated in China [5].

In the hierarchy of research designs, randomized controlled trials has the highest grade of evidence [6]. It may offer the most reliable information to evidence-based medicine [7]. In the clinical research field, the “randomized controlled clinical trial” is an alternative term used [8]. The reporting of RCTs, including constructed framework and well-written form, can assist researchers in assessing validity and applicability in a comprehensive and rapid manner [9]. However, clinical trials related to Traditional Chinese Medicine have had deficiencies in reporting RCT items [10]. The poor reporting of items in the full text of clinical trials can lead to inaccurate interpretation, incomplete data and potential bias [11].

With the aim of lessening problems caused by inaccurate or inadequate reporting of RCTs, the CONSORT statement was developed by the CONSORT Group. In terms of reporting randomized trials, the CONSORT statement is often treated as an evidence-based, minimum set of advice. It can help investigators design RCTs in a standard and transparent pattern, and aid them in reporting evaluation and explication critically. The current version is the CONSORT 2010 Statement. The CONSORT Statement comprises six sections with a 37-item checklist (http://www.consort-statement.org/) [12].

Since the CONSORT publication, RCTs examining the adherence to the recommendations for Asian ginseng and American ginseng have been conducted. A few investigations on quality assessment have been published in the past few decades [[13], [14], [15]]. Despite the increased research on the two species of ginseng, the overall RCT quality analysis of Asian ginseng and American ginseng has not been evaluated. No research on this subject is available. In consideration of these findings, we aimed to provide an extensive analysis of overall Asian ginseng and American ginseng RCTs reported globally and to examine the quality between them, which can track past and current evidence, as well as identify and highlight the potential future viewpoint of Asian ginseng and American ginseng RCT research and development.

2. Materials and methods

2.1. Data collection

Based on the tutorial of PubMed advanced search, we searched the data using the following strategy: “Ginseng” OR “Panax ginseng” OR “Panax quinquefolius” OR “Asian ginseng” OR “American ginseng” in the full fields. We screened the data, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2009 Flow Diagram. The inclusion criteria were: (1) derived from the “Clinical Trial” type, (2) published before 2019, (3) written in English, (4) provided the full-text, (5) involved human clinical trials. The clinical trials were then assessed for eligibility: (1) Asian ginseng and American ginseng, (2) not a combination of different ginseng species, (3) not a combination of ginseng and other herbs, (4) not a ginseng-derived compound. Finally, completed RCTs with oral administration were included. To summarize, the RCTs that did not meet these criteria were excluded. A detailed flow chart was shown in the Fig. 1. In addition, the studies did not have any criteria regarding age, sex, or ethnicity.

Fig. 1.

The flow-chart summary of the search process.

2.2. Data extraction

All included RCTs were searched by two independent authors (WJ and XZ), based on the aforementioned search strategy. The RCTs were manually sought and had been conducted from 1980 to 2019, 40 years in total. According to the predefined criteria of the CONSORT 2010 checklist, a quality analysis form was designed to offer the extracting details in Table 1. The quality analysis contains six domains: title and abstract, introduction, methods, results, discussion, and other information. The six domains consisted of 37 items in total. Two authors (WJ and XZ) independently extracted the data, evaluated the quality, and calculated the score of the included RCTs. Any disagreements and uncertainties were discussed and resolved between the two authors. If necessary, a third author (SP) acted as judge. The final extraction data were presented in Supplementary Table 1.

Table 1.

The Extraction Criteria According to the CONSORT 2010 Checklist

Section/Topic	Item No.	Checklist item
Title and abstract
	1	Identification as a randomized trial in the title
	2	Structured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts)
Introduction
Background and objectives	3	Scientific background and explanation of rationale
	4	Specific objectives or hypotheses
Methods
Trial design	5	Description of trial design (such as parallel, factorial) including allocation ratio
	6	Important changes to methods after trial commencement (such as eligibility criteria), with reasons
Participants	7	Eligibility criteria for participants
	8	Settings and locations where the data were collected
Interventions	9	The interventions for each group with sufficient details to allow replication, including how and when they were actually administered
Outcomes	10	Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed
	11	Any changes to trial outcomes after the trial commenced, with reasons
Sample size	12	How sample size was determined
	13	When applicable, explanation of any interim analyses and stopping guidelines
Randomization:
Sequence generation	14	Method used to generate the random allocation sequence
	15	Type of randomization; details of any restriction (such as blocking and block size)
Allocation concealment mechanism	16	Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned
Implementation	17	Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions
Blinding	18	If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how
	19	If relevant, description of the similarity of interventions
Statistical methods	20	Statistical methods used to compare groups for primary and secondary outcomes
	21	Methods for additional analyses, such as subgroup analyses and adjusted analyses
Results
Participant flow (a diagram is strongly recommended)	22	For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analyzed for the primary outcome
	23	For each group, losses and exclusions after randomization, together with reasons
Recruitment	24	Dates defining the periods of recruitment and follow-up
	25	Why the trial ended or was stopped
Baseline data	26	A table showing baseline demographic and clinical characteristics for each group
Numbers analysed	27	For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups
Outcomes and estimation	28	For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval)
	29	For binary outcomes, presentation of both absolute and relative effect sizes is recommended
Ancillary analyses	30	Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory
Harms	31	All important harms or unintended effects in each group (for specific guidance see CONSORT for harms)
Discussion
Limitations	32	Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses
Generalizability	33	Generalizability (external validity, applicability) of the trial findings
Interpretation	34	Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence
Other information
Registration	35	Registration number and name of trial registry
Protocol	36	Where the full trial protocol can be accessed, if available
Funding	37	Sources of funding and other support (such as supply of drugs), role of funders

2.3. Data evaluation

Ninety-one RCTs, which included 72 RCTs for Asian ginseng and 19 RCTs for American ginseng, were analyzed, based on the time distribution, country landscape, functional classification, preparation category, and product use. For the RCT quality score, an item was scored as “1” if it was fully reported, which was represented by “yes” (Y); it was scored as “0” if it was not reported or was inadequately reported, which was represented by “no” (N). Each RCT statistically had 37 items but four of them were not available (Item 6, 11, 13 and 25) in our study. Thus, every RCT quality score was calculated, based on the summarizing the individual Y score divided by 33. Other RCT information such as publication date, sample size, and trial length was simultaneously evaluated as potential factors of quality assessment between the two species.

2.4. Data analysis

Descriptive statistical analyses were conducted using Excel 2019 (Microsoft, Redmond, WA, USA), Prism 9.0 (GraphPad Software, San Diego, CA, USA), and SPSS 25.0 (IBM, Armonk, NY, USA). The mean and standard deviation (SD) quality score values were calculated for the two species to identify the characteristics associated with the quality score. Three factors (i.e., publication date, sample size, and trial length) were entered into the model to evaluate their relationship and significance, using the Pearson correlation and two-tailed significance.

3. Results

3.1. Time distribution and country landscape

As shown in Fig. 1, 91 RCTs (i.e., 72 RCTs related to Asian ginseng and 19 RCTs related to American ginseng) were analyzed from 1980 to 2019 in the time distribution. The average annual RCT number for Asian ginseng was approximately 2 and the RCT number for American ginseng was close to 0.5. As Fig. 2A showed, the first RCT of Asian ginseng was published in 1986 and that of American ginseng, in 2001. Each year, the RCT number for Asian ginseng was much more than that for American ginseng, except in 2005, when it was equal. Country landscape is a summary of the number of countries where an RCT was conducted and provides a whole picture of the global distribution of the two different ginseng species, as shown in Fig. 2C. Obviously, Asian ginseng was concentrated in Asia, especially in South Korea with 38 RCTs. By contrast, American ginseng was most concentrated in Canada with 8 RCTs. At the same time, the Asian ginseng was more widely distributed than American ginseng.

Fig. 2.

The current tendency for time distribution, country landscape, functional classification, preparation category, and product use in Asian ginseng and American ginseng RCTs. A. The annual and cumulative number of Asian ginseng and American ginseng RCTs from 1980 to 2019. The blue column represents the annual RCT number of Asian ginseng. The pink column represents the annual RCT number of American ginseng. The dark blue line represents the cumulative RCT number of Asian ginseng. The dark pink line represents the cumulative RCT number of American ginseng. B. The analysis of function classification for Asian ginseng and American ginseng RCTs. The blue column represents the total RCT number of Asian ginseng in each function classification. The pink column represents the total RCT number of American ginseng in each function classification. C. The analysis of country landscape for Asian ginseng and American ginseng RCTs. The blue column represents the total RCT number for Asian ginseng in different countries. The pink column represents the total RCT number for American ginseng in different countries. D. The analysis of preparation category for Asian ginseng and American ginseng RCTs. The column chart shows 8 preparation categories with their total RCT numbers, including capsule, powder, tablet, tea, drink, cream, and other solid or liquid, which are represented by different colors. E. The analysis of product use for Asian ginseng and American ginseng RCTs. The orange column represents the total RCT number of marketed products. The indigo column represents the total RCT number of non-marketed products.

3.2. Functional classification, preparation category, and product use

With regard to the functional classification in Fig. 2B, the functions of the two species of ginseng were primarily classified as “cognitive & behavior”, “glucose metabolism”, “cardiovascular function”, “cancer”, “sexual function”, “inflammation & immune”, and “organ symptoms”. Most functions were in the cognitive & behavior functional classification. The preparation category of the two species were also nearly the same. The capsule preparation accounted for most preparations of Asian ginseng (80%) and American ginseng (89%). Other preparation categories were also clearly presented in Fig. 2D. Fig. 2E showed the analysis of product use for Asian ginseng (with using rate 26%) and American ginseng (with using rate at 21%).

3.3. Quality score analysis

The quality analysis involved six domains with 37 items: title and abstract, introduction, methods, results, discussion, and other information. Fig. 3 showed the quality analysis for each item between Asian ginseng and American ginseng RCTs. It presented the percentage of Y for one item in all RCTs related to Asian ginseng and American ginseng. For both species, items 2, 3, 4, 5, 7, 20, 22, 23, 26, 27, 31, 33, and 37 were higher than 50%. For Asian ginseng, the items 8, and 32 were relatively high at 64% and 57%. For American ginseng, only item 34 was higher than 50% at 53%. Fig. 4 showed the six domains quality analysis of Asian ginseng (Fig. 4A) and American ginseng (Fig. 4B). The introduction, results, and discussion for Asian ginseng were more than 50%, at 99%, 53%, and 60%, respectively. By contrast, the title and abstract, introduction, and discussion for American ginseng were more than 50%, at 53%, 100%, and 56%, respectively.

Fig. 3.

The column diagram shows the percentage of “yes” (Y) in one item for all RCTs related to Asian ginseng and American ginseng. The blue column represents the Asian ginseng. The pink column represents the American ginseng. The black line indicates that the percentage is 50%.

Fig. 4.

The quality analysis of the six domains: title and abstract, introduction, methods, results, discussion, and other information. The length of the internal radius represents the percentage of “reported” and the external radius represents the percentage of “not reported”. The indigo issue represents “title and abstract”. The yellow issue represents “introduction”. The orange issue represents “methods”. The green issue represents “results”, The blue issue represents “discussion”. The pink issue represents “other information”. A. The Asian ginseng pie chart. B. The American ginseng pie chart.

Fig. 5A presented the results of the RCT quality score for the two species of ginseng. The RCT quality score for Asian ginseng was a minimum of 0.1818, maximum of 0.7879, mean of 0.4802, range of 0.6061, and SD of 0.1347. The quality score for American ginseng was a minimum of 0.1818, maximum of 0.7273, mean of 0.4577, range of 0.5455, and SD of 0.1498. The quality score showed that 95% CI was −0.09315 to 0.04819, and the P-value was 0.5155 but without significance. Fig. 5B, C and D showed that the three factors (i.e., publication date, sample size, and trial length) were included in the model to check their relationship with the quality score, using the Pearson correlation and two-tailed significance analysis. The results were in Table 2.

Fig. 5.

The RCT quality score for Asian ginseng and American ginseng with three factors: publication date, sample size and trial length. A. The all RCTs quality score column diagram for Asian ginseng and American ginseng (presented as the mean ± the standard deviation (SD)). The blue scatter points represent the RCT quality scores for Asian ginseng. The pink scatter points represent the RCT quality scores for American ginseng. B. The all RCT quality score column diagram for Asian ginseng and American ginseng with regard to publication date (Mean ± SD). The blue scatter points represent the RCT quality scores of Asian ginseng. The pink scatter points represent the RCT quality scores for American ginseng. C. The all RCT quality score column diagram for Asian ginseng and American ginseng with regard to sample size (Mean ± SD). The blue scatter points represent the RCT quality scores of Asian ginseng. The pink scatter points represent the RCT quality scores of American ginseng. D. The all RCT quality score scatter diagram for both Asian ginseng and American ginseng with regard to trial length (Mean ± SD). The blue scatter points represent the RCT quality scores of Asian ginseng. The pink scatter points represent the RCT quality scores of American ginseng.

Table 2.

Overall Quality Score by Characteristics Between Asian Ginseng and American Ginseng

Factor	Specie	Pearson correlation	Sig. (2-tailed)
Publication year	Asian ginseng	0.531∗∗	0.000
	American ginseng	0.158	0.517
	Asian and American ginseng	0.466∗∗	0.000
Sample size	Asian ginseng	0.112	0.349
	American ginseng	0.675∗∗	0.002
	Asian and American ginseng	0.263∗	0.012
Trial length	Asian ginseng	−0.068	0.571
	American ginseng	0.146	0.551
	Asian and American ginseng	0.050	0.638

∗ Correlation is significant at the 0.05 level (2-tailed).

∗∗Correlation is significant at the 0.01 level (2-tailed).

4. Discussion

The aim of this study was to provide an extensive analysis of these two species and examining the quality of the RCTs that have been conducted on American ginseng and Asian ginseng. Asian ginseng and American ginseng had a similar tendency, which can be preliminarily divided into three stages, the emerging stage with a muted growth, the boom stage with a rapid increase, and the down stage with a relevant decrease. Asian ginseng was more popular among the researchers. With regard to popularity among countries, South Korea was highest in the global country landscape for Asian ginseng, which was unanimous with the findings of a study conducted in 2010 [4]. Canada had an important role in American ginseng research, which is probably related to the native origin [16]. In general, Asia and North America were active research areas. The global network of Asian ginseng and American ginseng manifested distinct regional distribution.

With regard to the functional classification, the two species had similar functional fields. Both species had a rich source of bioactive phytochemicals such as ginsenosides and polysaccharides, which were the major biologically active ingredients in ginseng [17,18]. The two species have often been extensively used in various food products, alternative nutraceuticals, and dietary supplements, such as the G115 capsule [19], Ginsana capsule [20], and Cheonggukjang powder [21] for Asian ginseng, and the Cereboost™ capsule [22], HT1001™ capsule [23], and CNT 2000 capsule [24] for American ginseng with fewer preparation categories in RCTs. Based on different marketed product using rates, we conjectured the possibility that different medication customs in different regions such as Asian would accept a decoction well; therefore, researchers may choose an original ginseng plant with a lower product using rate to conduct an RCT.

The quality analysis of six domains, which comprised 37 items, revealed unique characteristics between the two species. The two species showed a quite high level for the introduction between item 3 and item 4. However, the overall adherence to the CONSORT guideline in the other domains was relatively poor, especially from item 1 to item 2 for the title and abstract part. Most RCT researches did not use “randomized” in the title for Asian ginseng [[25], [26], [27]] and American ginseng [[28], [29], [30]]. In another aspect, the word restriction and the reporting format in the abstract were universally considered key influencing factors for the item 2 quality score.

A barrier may exist in using the CONSORT guideline in an all-around manner. Numerous journals restricted the abstract word count and the reporting format when researchers intended to submit their research for publication [31]. Two research study may be examples of RCTs lacking a structured abstract containing a background, aim, methods, results, and conclusion sections for Asian ginseng [32] and American ginseng [33]. The structured format can influence reporting quality, which was closely linked with other research results [34]. Thus, we strongly recommended that the title and abstract should be as accurate, comprehensive and structured as possible within the restrictions of the guidelines of a journal. The collaboration between authors and journals should be improved to avoid rigid requirements that limit the quality level.

The quality score of Asian ginseng was higher than that of American ginseng in our research findings. RCT quality scores were compared with regard to three potential factors. In our model, the RCT quality score was significantly associated with the publication date for Asian ginseng and with the sample size for American ginseng, which was similar to the results of a systematic review conducted in 2008 [35]. However, this finding was the opposite for trial length, based on a review conducted in 2009 [36]. The item was probably scored using different rules. In some studies, the quality score was regarded as 1 score, when all elements were reported [37,38], but in other studies may be regarded as partial compliance [39] or unclear [40], which should be accounted for in the analysis. Furthermore, authors' awareness of the CONSORT statement also may lead to different results [41].

The reporting RCT quality of the two species of ginseng improved overall during the past 40 years. The CONSORT statement has been updated many times. It has become increasingly comprehensive, accurate and clear, which may have influenced the quality assessment.

A focus on another important aspect, how the sample size was determined, revealed that the sample size was poorly reported at less than 30% for both Asian ginseng and American ginseng. Other researchers similarly found that this item had poor quality [[42], [43], [44]]. In general, the two factors of publication date and sample size may be significant and powerful factors in reported RCT quality, which should be well researched in future RCTs. Rigorous RCTs for ginseng species also seem warranted in the future.

The current study introduced quality analysis, based on CONSORT guideline, for two species of ginseng. Considering the current systematic analysis findings, we are confident that our research was a comprehensive and integral summary of all available RCT evidence-based data.

However, our study also had several limitations. First, we could not provide a whole picture for all types of ginseng species, but only for the two ginseng species commonly used in researches. Thus, our findings do not represent all ginseng species situation. Second, the samples of the two species in our research were different and relatively small. Therefore, some investigators' opinions may have had viewer bias. Finally, all studies we used were published in English so that non-English publications were not analyzed.

5. Conclusions

The extensive analysis of the two ginseng species revealed that each had their own characteristics. The reporting RCT quality of the two species of ginseng improved during the past 40 years. The analysis revealed that publication date and sample size were significantly associated with reporting RCT quality, which should be investigated and developed further in future research. The reporting format will be improved in terms of accuracy, comprehensiveness and structure. Rigorous RCTs designed for the species of ginseng seems warranted in the future, which could shed light on the product research and development of potential future fields of both Asian ginseng and American ginseng.

Declaration of competing interest

The authors have declared no conflict of interest.

Abbreviation

RCT(s)

Randomized controlled clinical trial(s)

CONSORT

Consolidated Standards of Reporting Trials

PG

Panax ginseng Meyer

PQ

Panax quinquefolius L.

SD

Standard deviation

Y

Yes

N

No

Compliance with ethics requirements

This research does not contain any studies with human or animal subjects.

Appendix A. Supplementary data

The following are the Supplementary data to this article: