Analyzing the number of articles with network meta-analyses using chord diagrams and temporal heatmaps over the past 10 years: Bibliometric analysis

Background:

Network meta-analyses (NMAs) are statistical techniques used to synthesize data from multiple studies and compare the effectiveness of different interventions for a particular disease or condition. They have gained popularity in recent years as a tool for evidence-based decision making in healthcare. Whether publications in NMAs have an increasing trend is still unclear. This study aimed to investigate the trends in the number of NMA articles over the past 10 years when compared to non-NMA articles.

Methods:

The study utilized data from the Web of Science database, specifically searching for articles containing the term “meta-analysis” published between 2013 and 2022. The analysis examined the annual number of articles, as well as the countries, institutions, departments, and authors associated with the articles and the journals in which they were published. Ten different visualization techniques, including line charts, choropleth maps, chord diagrams, circle packing charts, forest plots, temporal heatmaps, impact beam plots, pyramid plots, 4-quadrant radar plots, and scatter plots, were employed to support the hypothesis that the number of NMA-related articles has increased (or declined) over the past decade when compared to non-NMA articles.

Results:

Our findings indicate that there was no difference in mean citations or publication trends between NMA and non-NMA; the United States, McMaster University (Canada), medical schools, Dan Jackson from the United Kingdom, and the Journal of Medicine (Baltimore) were among the leading entities; NMA ranked highest on the coword analysis, followed by heterogeneity, quality, and protocol, with weighted centrality degrees of 32.51, 30.84, 29.43, and 24.26, respectively; and the number of NMA-related articles had increased prior to 2020 but experienced a decline in the past 3 years, potentially due to being overshadowed by the intense academic focus on COVID-19.

Conclusion:

It is evident that the number of NMA articles increased rapidly between 2013 and 2019 before leveling off in the years following. For researchers, policymakers, and healthcare professionals who are interested in evidence-based decision making, the visualizations used in this study may be useful.

Highlights

1. Visual displays to verify the hypothesis that the number of NMA-related articles has increased (or declined) over the past decade when compared to non-NMA articles, which has never been done previously.

2. This study has resulted in the development of article-theme classification that helps conduct this study (e.g., extracting NMA and non-NMA articles by keywords plus in Web of Science core collections).

3. The results show that the number of NMA-related articles increased prior to 2020 but experienced a decline in the past 3 years, potentially due to being overshadowed by the intense academic focus on COVID-19.

1. Introduction

Meta-analysis (MA) is a statistical technique that combines the results of multiple studies to estimate the overall effect of a particular intervention or exposure.^[1–3] It allows for the synthesis of data from individual studies to provide a more accurate estimate of the true effect size.

There are over 5989 MA articles indexed in PubMed^[4] with a search string of meta-analysis [MeSH Major Topic] since 1987,^[4] with a stable trend over the past decade (i.e., counts: 303, 301, 350, 366, 354, 407, 504, 447, 235, and 76, which could be medical subject headings (MeSH terms) that have not been completely assigned to articles in 2023 due to at least a 3-month delay^[5–8] as of March 1, 2023).

1.1. Network meta-analysis has been rapidly developed

Randomized controlled trials typically compare new drugs with either a placebo or a standard available drug.^[9] However, they do not allow for a comparison of all available interventions in the same study. As a result, it is often difficult to conduct head-to-head comparisons between competing interventions. In such cases, network meta-analysis (NMA) is recommended because it allows for multiple pairwise comparisons across several interventions, both through direct and indirect comparisons.^[10–15] Hence, NMA can provide summary estimates of relative treatment effects for various treatment comparisons. When direct evidence for treatment comparisons is lacking or conducting a new randomized controlled trial that includes all competing treatments is not feasible, NMA is a cost-effective option for clinical decision-making.^[16]

By March 1, 2023, 583 NMA articles were indexed in PubMed,^[17] with a stable trend over the last 10 years (i.e., 1, 6, 64, 87, 146, 161, 109, 69, and 37 in 2022).

Thus, we conceived the first research question concerning the comparison of trends between articles that employed NMA and those that did not (i.e., non-NMA for short in this study).

1.2. Barriers and challenges encountered by authors

In bibliometrics, articles are often classified by humans,^[18–20] which is both tedious and time consuming. Cowords were subjected to social network analysis (SNA) to identify article themes through cluster analysis.^[21,22] CiteSpace,^[23] HistCite,^[24] and VOSviewer^[25] are commonly used to assign cluster names based on the most frequently observed keywords^[26–28] rather than the chief keywords derived from SNA, which is required in this study.

This has led to the second research question regarding how to classify themes in MA articles. Otherwise, it would be impossible to compare trends between articles that used NMA and those that did not.

Additional barriers and challenges are encountered by authors, including assigning article themes to countries of origin, comparing NMA and non-NMA articles, and conducting a comprehensive bibliometric analysis of MA articles’ characteristics. Although previous studies have illustrated theme extraction via coword analysis,^[18,29] comparison of citation achievements in bibliographic and meta-analysis studies,^[30] and use of the descriptive, diagnostic, predictive, and prescriptive analytics model (DDPP),^[31,32] there has been no analysis of the trend comparison between NMA and non-NMA articles with visualizations (such as chord diagrams and temporal heatmaps [THM]) during the past 10 years.

1.3. Citation prediction and article worth reading required in bibliometrics

Searching PubMed for titles containing the phrase “100 top-cited” revealed 433 publications.^[33] It is common for bibliographical studies to refer to 3 types of information: descriptive statistics (DS), significant topics or article types with research domains (RD), and research achievements in entities (RA)^[19] (i.e., descriptive and diagnostic analytics in the DDPP model^[31,32]). Furthermore, citation prediction^[34–37] has been employed in bibliometrics to predict article citations based on weighted scores of article keywords, and THM has been applied to display articles worth reading.^[31,32,38] In this manner, the DDPP model can be used to conduct a comprehensive bibliometric analysis of MA articles.

1.4. Citation cartels should be cautious in author collaborations

In 1999, Franck published an essay discussing citation cartels, which are collaborations between editors and journals for mutual benefit.^[39] One method that editors employ to boost their journal’s impact factor is through inter-journal citations.^[40] More recently, citation cartels have also been observed between editors and authors, where members cite each other’s papers for mutual gain, regardless of familiarity.^[41]

While modern semantic web tools can assist in detecting online citation cartels,^[41] confirming their existence in the real world requires in-depth analysis.^[42–44] A temporary indication of a citation cartel may be a self-citation rate exceeding 20%, 15%, or 10%.^[42–44]

The relationship between productivity and impact in scientific production has been a long-standing debate.^[45] Evidence suggests that academics are hesitant to simultaneously alter their productivity and the prestige level of the journals they publish in over consecutive career years.^[45] However, it remains uncertain whether citation cartels exist within articles. Journal editors and reviewers are aware of the phenomenon of citation cartels,^[40] and caution should be exercised when evaluating author research assessment based on article citations.^[46]

1.5. Visualizations are vital in bibliometrics

As previously noted, the DDPP model^[31,32] provides guidelines for physicians and researchers to identify key features that distinguish their field or discipline by means of DS, RD, RA, citation prediction, and articles worth reading. However, this model overlooks 3 important perspectives that require unique visualizations to effectively highlight relevant entities on a graph.^[47,48] For instance, a 4-quadrant plot^[49] offers a more comprehensive approach than a 1-quadrant plot.^[50] Additionally, an overall score, such as the category, journal, authorship and L-index (CJAL) score,^[49,51] can be used to highlight leading entities. Finally, a circular packing chart (CPC)^[52] can be used to condense cluster displays in a circular layout, where each circle represents a node and its size corresponds to a specific variable, such as the number of publications or citations denoted by the weighted centrality degree in SNA.

Several breakthroughs are necessary to improve the bibliographical study when comparing trends between NMA and non-NMA articles, including categorizing articles thematically, assigning article themes to countries of origin, and utilizing the DDPP model with visualizations.

1.6. Study aims

This study aims to investigate the trends in the number of NMA articles over the past 10 years when compared to non-NMA articles by applying the DDPP model with visualizations.

2. Methods

2.1. Data source

A total of 2870 articles were indexed in PubMed in search of meta-analysis [MeSH Major Topic] between 2013 and 2022, and 2645 articles were matched in Web of Science (WoS) core collections by PMID (i.e., unique identification number of articles in PubMed). Two forms of articles were analyzed: 2645 and 100 top-cited articles in the top 20 themes (T100); see Supplemental Digital Content 1, http://links.lww.com/MD/J161.

As this study did not involve the examination or treatment of patients or review of patient records, it was exempt from review and approval by our research ethics committee.

2.2. Four parts of the DDPP model

2.2.1. Descriptive analytics in 2645 articles.

We compared the average number of citations per publication, also known as the mean citations, between articles indexed in PubMed and WoS. This was accomplished using both a forest plot^[30] and a line chart.^[53,54] Additionally, a 4-quadrant plot^[49] was utilized to identify the top entities based on the CJAL score^[49,51] denoted by Equations 1 to 3, which includes country, institute, department, and author.

$$CJA score=\underset{i=1}{\sum ^n}{C}_i\times J_i\times A_i,$$ (1)

$$CJAL score=\underset{i=1}{\sum ^n}{C}_i\times J_i\times A_i\times L-inde{x}_i,$$ (2)

$$\text{L-index}=\text{round}\left(\text{log}\left(\frac{Citation}{A_n\times Age}+1\right),0\right),>=1$$ (3)

The CJAL score is computed by the CJA score^[47] and the L-index,^[55] including 3 major factors: the category (C; e.g., review, original article, case report, etc.), the journal “quality” (J; e.g., impact factor, JIF, or ranking of the journal), and the authorship order (A). By multiplying each of these 3 aspects as well as the L-index^[50] (Equation 3), the CJAL score is calculated.

In this study, only co-first authors (denoted RP and FP to compute the Y-index^[50]) were involved to compute the CJAL score (i.e., it is the reason why articles in WoS were applied because not such information about corresponding authors is provided in PubMed). The journal impact factors were based on SCI/SSCI journal rankings^[56] for SCI/SSCI-indexed papers in the Journal Citation Report in WoS.

The absolute advantage coefficient (AAC)^[57] was applied to determine the dominance strength for the top member beyond the next two in a variable (i.e., country, journal, or category) by the effect size based on the CJAL score, with criteria of <0.5, between 0.5 and 0.7, and not less than 0.7 as the small, medium, and large effect sizes, respectively.^[58]

To visualize the distribution of article counts and CJAL scores across different countries, 2 choropleth maps^[59] were used to compare US states and Chinese provinces with other countries/regions. Finally, we used a pyramid plot to compare the number of publications and mean citations among the top 20 journals.

2.2.2. Diagnostic analytics in 2645 articles.

Themes for 2645 MA articles were classified by SNA and Equation 4.^[18]

$$\text{Theme}=At[\underset{0\le x\le 1}{\max }\underset{i=1}{\sum ^L}\underset{j=1}{\sum ^n}\underset{k\in F}{(\text{m}<-\text{m}+1)}]$$ (4)

L represents the number of keywords in article i. n corresponds to the number of keywords denoted by keyword k that belong to the subject category defined by SNA (i.e., the keywords that occur in the same cluster). Through Equation 4, the theme is redirected to the maximal number of keywords (m) involved in the cluster.

In the next step, themes were assigned to country-based author collaboration networks using Equation 5.^[29]

$$Them{e}_{rj}=\underset{r in j}{\max }(\underset{n=1}{\sum ^N}\underset{l=1,a\in D,a\in r}{\sum ^L}\underset{j=1, t\in c}{\sum ^J}ter{m}_{rj}(count<-count+1/L)),$$ (5)

L represents the number of terms (e.g., names of countries in this study) in an article. To record the summed counts, a contingency table with clusters in row (r) and themes in column (j) was constructed. Through Equation 4,^[18] the term was matched with the cluster number corresponding to the theme defined in an article (e.g., the article belongs to a theme). Using Equation 4,^[29] the total weighted scores were summed, and a maximum likelihood selection was made.

A mapping of the themes represented by the primary Keywords Plus in WoS into clusters was performed for 2645 articles and the top 20 countries. These were accomplished using CPC^[52] and chord diagrams,^{[18,43,60,61]} respectively, for visualization.

A line chart was used to compare the number of articles related to NMA versus non-NMA over the last decade, followed by a THM to visualize the trends of the top 10 themes. and chord diagrams.

2.2.3. Predictive analytics in T100 articles.

Citation weights were calculated for Keywords Plus in WoS core collections. Based on the weighted mean citations, a scatter plot with 95% control lines was drawn to predict article citations.^[34–37,58]

To determine the predictive power between weighted keywords and original article citations (e.g., denoted by × and y, respectively), the correlation coefficient (CC), defined as the degree of relation between 2 variables, was referred to in Equation 6.^[62] The t value based on CC was calculated using the formula (=cc^[34–37,58]).

$$CC=\frac{\text{n}({\displaystyle \sum x}y-({\displaystyle \sum x})({\displaystyle \sum y})}{[{\displaystyle \sum x^2}-\left({\displaystyle \sum x}{)}^2\right][{\displaystyle \sum y^2}-\left({\displaystyle \sum y}{)}^2\right]},$$ (6)

where n = Number of values or elements, ∑x = Sum of 1st values list, ∑y = Sum of 2nd values list, ∑xy = Sum of the product of 1st and 2nd values, ∑x² = Sum of squares of 1st values, ∑y² = Sum of squares of 2nd values.

2.2.4. Prescriptive analytics in T100 articles.

Using THM,^[31,32,38] we extracted articles worth reading from T100 articles, which were determined based on their high citations and increasing growth rates over the past 4 years (from 2019 to 2022).

Based on normalized citations for each article, the T100^[63] was represented on the dot plot (namely, the impact beam plot, impact beam plot^[64]) using citation percentiles (i.e., using the MSExcel function percentrank()).

2.3. Creating dashboards on Google Maps

By using MedCalc statistical software, version 9.5.0.0 (MedCalc, New York, NY), a prediction equation was developed. We set the significance level at Type I error (0.05).

Graphs were drawn using author-made modules in Excel (Microsoft Corporation). We created HTML pages that were used to display visualizations on Google Maps. The relevant CJAL scores for each member in each entity can be displayed on a 4-quadrant radar plot. Supplementary Digital Content 2, http://links.lww.com/MD/J162 contains the method used to draw the visualizations for this study. The study flowchart is shown in Figure 1.

Figure 1.

Study flowchart based on the DDPP model (including descriptive statistics, diagnostic analytics, predictive analytics, and prescription for articles that are worth reading).

3. Results

3.1. Descriptive analytics

The top panel of Figure 2 shows that there is no significant difference (P > .05) in the mean citations of articles indexed in PubMed and WoS from 2013 to 2022. However, the bottom panel of Figure 2 indicates a considerable difference in the mean citations of the 2 bibliometric databases in 2015. Nonetheless, this difference did not exist due to larger standard deviations in the 2 groups in 2015.

Figure 2.

Comparison of publication total counts of network meta-analysis and mean citations between articles in WoS and PubMed. WoS = Web of Science.

The United States, McMaster University (Canada), medical schools, Dan Jackson from the United Kingdom, and the Journal of Medicine (Baltimore) were among the leading members in their respective variables (Fig. 3). There was a weak AAC effect (=0.37) in countries and medium AAC effects in institutes (=0.52), departments (=0.54), and authors (=0.51).

Figure 3.

Characteristics of articles related to meta-analysis shown on the 4-quadrant radar plot (note. AAC denotes the dominance strength over the next two elements for Top 1 element: ≥0.7 means large effect, ≥0.5 is medium effect, and <0.5 indicates small effect). AAC = absolute advantage coefficient.

To visualize the distribution of article counts and CJAL scores across different countries, 2 choropleth maps^[59] show that the top three are the UK, Canada, and Germany in comparison to US states, Chinese provinces, and other countries/regions, as shown in Figure 4.

Figure 4.

The article counts and CJAL scores for MA articles between 2013 and 2022, as distributed geographically, will be compared between US states and Chinese provinces with other countries/regions, based on two scenarios of article counts and CJAL scores across, respectively, on the left and right panels. CJAL = category, journal, authorship and L-index, MA = meta-analysis.

Among journals, the Journal of Medicine (Baltimore) holds the top rank in terms of publications (245), while Research Synthesis Methods has the highest mean citations (8.9).

3.2. Diagnostic analytics

The CPC^[52] shows that NMA ranked highest on the coword analysis via Equation 4, followed by heterogeneity, quality, and protocol, with weighted centrality degrees of 32.51, 30.84, 29.43, and 24.26, respectively, as shown in the top panel of Figure 6. In the CPC, only the top 20 themes (defined by the primary keywords in clusters) are displayed.

Figure 6.

Theme classification for 2645 MA articles using circle packing charts and chord diagrams (note. the extraction of themes (top) and assignment of themes to articles (bottom)). MA = meta-analysis.

Themes are assigned to the top 20 countries based on Equation 5, as shown in the bottom panel of Figure 6. An example would be that in China, a significant portion follows the protocol indicated by the white curve, whereas in the US and the UK, heterogeneity is observed and indicated by the red curves linked together.

The number of NMA-related articles increased prior to 2020 but experienced a decline in the past 3 years, with CC = 0.91 against non-NMA = 0.92 in years from 2017 to 2020 (|t|>2.0, P < .05) and CC = −0.88 against non-NMA = −0.97 in years from 2019 to 2022 (|t|>2.0, P < .05), as shown in Figure 7.

Figure 7.

The number of NMA articles has not shown a significant increasing trend in the past 10 years in comparison to non-NMA articles, but with more published NMA articles in recent 3 yr. NMA = network meta-analysis.

All of the top 10 themes, except SIZE, show a decline in count over years (e.g., the column of Type in Figure 8), where BS is burst strength and Growth is the correlation coefficient against years.

Figure 8.

Trend analysis for the top 10 themes based on chief words in clusters (note. BS = burst strength and Growth = correlation coefficient against years; counts in cells are the number of articles belonging to the theme; the burst points start at the beginning of red font).

Trend analysis for the top 10 themes based on chief words in clusters (note. BS = burst strength^[65–67] and Growth = correlation coefficient against years; the burst points start at the beginning of red font).

3.3. Predictive analytics

There was a significant correlation between the number of article citations and the number of weighted keywords (F = 5643.3865, P < .0001), as shown in Figure 9. The prediction linear equation is expressed as y = −27.5034 + 1.1503 × weights(x) of keywords. All 100 articles were located within the 1-dimensional zone in the scatter plot (CC = 0.99, df = 98, t = 75.12, P < .0001), indicating that weighted keywords (=mean citations) are useful for predicting citations in articles.

Figure 9.

Predictive analytics of citations predicted by weighted scores of keywords based on mean citations using the scatter plot with 95% control lines.

3.4. Prescriptive analytics

In the top panel of Figure 10, we extracted articles worth reading from T100 articles, which were determined based on increasing growth rates over the past 4 years (from 2019 to 2022) and their high citations. Of these 20 articles, eight articles were associated with NMA. Three worth-reading articles are summarized in the Discussion section.

Figure 10.

Articles worth reading regarding using IBP (bottom) and THM (top) to display (articles appear once the dot of interest is clicked on the IBP after the QR-code is scanned). IBP = impact beam plot, THM = temporal heatmap.

In the bottom panel of Figure 10, the T100 articles are shown on a dot plot, where red dots represent the theme of non-NMA articles. We encourage readers to scan the QR code, click on the dot of interest, and read the abstract of the article on the PubMed website. For instance, when the ultimate rightmost dot in 2013 is clicked, the highly cited articles^[10] (entitled Graphical tools for network meta-analysis in STATA) appear immediately, with 1255 article citations in WoS.

3.5. Online dashboards shown on Google Maps

All the QR codes in the graphs are linked to the dashboards.^[68–74] Readers are suggested to examine the displayed dashboards on Google Maps.

4. Discussion

4.1. Principal findings

According to our findings, there was no significant difference in citation averages or publication trends between NMA and non-NMA. The leading entities included the United States, McMaster University in Canada, medical schools, Dan Jackson from the United Kingdom, and the Journal of Medicine in Baltimore. NMA ranked highest on the coword analysis, with heterogeneity, quality, and protocol following closely behind, with weighted centrality degrees of 32.51, 30.84, 29.43, and 24.26, respectively. The number of NMA-related articles increased before 2020 but experienced a decline in the last 3 years, which could be attributed to the intense academic focus on COVID-19 overshadowing other research.

Accordingly, the study aims to determine whether the trends in the number of NMA articles over the past 10 years are similar to those of non-NMA articles.

To enhance the bibliographical analysis of trends between NMA and non-NMA articles, various breakthroughs have been made, such as categorizing articles thematically, assigning article themes to countries of origin, and utilizing the DDPP model with visualizations.

4.2. Trends in NMA articles versus non-NMA articles

According to Figure 7, the ratio of NMA to non-NMA articles is approximately one-tenth. Before 2020, there was an increase in the number of articles related to NMA, but in the past 3 years, there has been a decline, possibly because of the significant academic attention on COVID-19. This trend can also be observed in non-NMA articles. Previous studies have not reported the observation of this trend in non-NMA articles.

Two bibliometric databases, such as WoS and PubMed, should have different citation counts for identical articles. However, our analysis in Figure 2 shows that there is no significant difference (P > .05) in the average citations of articles indexed in PubMed and WoS from 2013 to 2022. However, upon closer examination of the bottom panel of Figure 2, we observe a significant difference in the average citations of the 2 databases in 2015. This difference, however, was not statistically significant due to larger standard deviations in the 2 groups in 2015.

Leading entities included the US, McMaster University in Canada, medical schools, and Dan Jackson from the UK when referred to Figure 3, indicating that a 4-quadrant plot^[49] offers a more comprehensive approach than a 1-quadrant plot.^[50]

The CJA score is not only distinctive but also useful. Numerous medical schools in Taiwan require a minimum FAP score, which is assessed using the CJA score.^[56] This score takes into account 3 factors: article category, journal, and authorship, rather than relying on bibliometric metrics such as the h-/g-/x-/Y-/hT-/hx-index^{[50,55,75–81]} that are commonly used in research and practice. Since there is no citation factor included in computing the CJA score,^[56] the CJAL score is thus calculated by incorporating the L-index^[55] through Equations 1 to 3.

NMA ranked highest in the coword analysis, followed by heterogeneity, quality, and protocol, with weighted centrality degrees of 32.51, 30.84, 29.43, and 24.26, respectively. Applying the AAC^[57] results in a dominance strength of 0.50 for NMA, which has a medium effect. This suggests that the AAC is both viable and useful, as evidenced by the formula [(32.51/30.84)/(30.84/29.43)]/(1+[(32.51/30.84)/(30.84/29.43)]).

Visualizations were provided to enhance the bibliographical analysis of trends between NMA and non-NMA articles. Barriers and challenges encountered by the authors addressed in Section 1.2 have been overcome by categorizing articles thematically, assigning article themes to countries of origin, and utilizing the DDPP model. The chord diagram (Fig. 6) provides us with a clear view of the relationship between two or more entities, and the DDPP model^[31,32] provides a comprehensive analysis of bibliometrics.

4.3. Here are worth reading articles

Based on the THM in Figure 10, 3 NMA articles worth reading are summarized below:

The article^[10] entitled graphical tools for network meta-analysis in STATA was cited 1255 times in WoS, was authored by Chaimani (Greece) et al, and was published in PLOS One (2013). The authors simplified the methodology for individuals without a statistical background and provided a collection of STATA routines that can be effortlessly utilized to illustrate the supporting data, assess the suppositions, implement the NMA model, and understand its outcomes.

The article^[82] entitled conceptual and technical challenges in network meta-analysis was cited 586 times in WoS, was authored by Cipriani (Italy) et al, and was published in Ann Intern Med (2013). The author delves into the possibilities and constraints of NMA and provides recommendations for addressing issues such as heterogeneity, inconsistency, and potential biases in existing data. The goal is to enhance physicians’ understanding of the difficulties involved in interpreting research findings.

The article^[83] entitled GRADE approach to rate the certainty from a network meta-analysis was cited 60 times in WoS (but with increasing citation rate = 0.98) and was authored by Brignardello-Petersen (Canada) et al and published in J Clin Epidemiol (2019). When NMA estimates are coherent, they should have increased precision (narrower confidence or credible intervals compared with relying on direct estimates alone), but in many cases, the confidence intervals are markedly widening. The assumption of common between-study heterogeneity across the network seems to be responsible for this.

4.4. Implications and possible changes outlined in this study

There are several notable features of this study. First, the CJAL scores are superior to biometric metrics such as the h-/g-/x-/Y-/hT-/hx-index,^{[50,55,75–81]} as they consider more aspects of article quality and quantity.

Second, Figure 6 employs the CPC and chord diagram to emphasize important entities, which were found to be effective in bibliometrics.

Third, the study utilizes a 4-quadrant radar plot,^[49] which offers readers a quick visual representation of 4 perspectives on article entities. This is particularly useful when research accomplishments are measured by the CJAL score rather than the Y-index, as is the case in traditional studies.^[50]

In addition, the scatter chart is capable of predicting article citations based on keyword weights. This may prove useful for future bibliometric analyses, and it is not limited to DS, RD, and RA.

The SNA-based theme classification in this study is objective and unique compared to previous studies that utilized manual methods.^[19] The findings suggest that the classification method is valid and worth recommending to future researchers, especially when used in conjunction with the chord diagram to illustrate the relationship between themes and clusters. The R codes for creating the chord diagram are provided in Supplemental Digital Content 2, http://links.lww.com/MD/J162.

4.5. Limitations and suggestions

To further advance the research, several issues need to be addressed. First, it is important to note that the software used for creating CPC and chord diagrams is not exclusive and can be easily replaced with other software packages. A simple MS Excel model regarding NMA is provided in Supplemental Digital Content 3, http://links.lww.com/MD/J163 for readers who are able to practice the NMA in R.

Second, this study uses Google Maps to display dashboards, which requires a paid project key, making it inaccessible for free. Therefore, replicating the usage may prove challenging for other authors, particularly within a short period of time.

Third, significant computation is needed to calculate the CJAL score for this study, and in the future, dedicated software will be necessary for its development.

Fourth, when searching bibliometrics, it is important to note that articles extracted from PubMed using different search strings such as meta-analysis [MeSH Major Topic], meta-analysis [Title], and meta-analysis [All Fields] may yield different results. Future studies should exercise caution when selecting a search string to ensure consistency in bibliometric analysis.

Fifth, this study employs 10 different visualization techniques to support its hypothesis. However, for future studies, it is recommended that more appropriate visual displays be used for a better understanding of study trends(e.g., the scatter with 95% control lines^[84] in Figure 5).

Figure 5.

Comparison of publications and mean citations among the top 20 journals in 2645 MA-related articles (note. the Journal of Medicine (Baltimore) ranks top 1 among journals). MA = meta-analysis.

Sixth, the phenomenon of citation cartels is well known among journal editors and reviewers.^[39,45] However, this study does not delve deeply into this topic. Previous research^[40] developed a module to calculate the self-citation rate of specific journals, which offers additional understanding of citation cartels and warrants ongoing investigation.

Seventh, the visual displays in this study were created using custom modules developed by the authors and implemented in the R programming language, resulting in a significantly different appearance compared to the visual displays generated by commercially available bibliometric software, such as CiteSpace,^[85] commonly used in the literature (e.g., the one on NMA^[86]). However, there is a need for further improvements in visual aesthetics of the images in future studies to enhance their overall appeal.

Last, it is worth noting that although articles were extracted mainly from PubMed and matched with PMID to WoS, articles retrieved from other databases such as Google Scholar and Scopus may have different results, particularly with regard to different article citations. Therefore, future studies are necessary to compare the difference in mean citations based on identical articles between bibliometric databases.

5. Conclusion

By analyzing NMA and non-MA articles to understand their characteristics, a significant achievement was made, which involved 3 key steps: thematically categorizing articles, linking article themes to their countries of origin, and utilizing the DDPP model with visualizations. In future studies, instead of focusing on MA, the 4-quadrant radar plot combined with the CJAL score should be applied to the 100 top-cited articles to gain more insights.

Acknowledgments

We thank Enago (www.enago.tw) for the English language review of this manuscript.

Author contributions

Conceptualization: Yu-Erh Liang, Sam Yu-Chieh Ho.

Investigation: Willy Chou.

Methodology: Tsair-Wei Chien.