A leading author of meta-analysis does not have a dominant contribution to research based on the CJAL score: Bibliometric analysis

Background:

There have been nearly 200 thousand meta-analysis articles indexed by web of science (WoS) since 2013. To date, a bibliometric analysis of leading authors of meta-analyses that contribute to the field has not been conducted. Analyzing trend patterns in article citations and comparing individual research achievements (IRAs) are required following the extraction of meta-analysis articles. Using trend analysis, this study aims to verify the hypotheses that; The leading author has a dominant research achievement and; Recent articles that deserve worth reading can be identified.

Methods:

In the WoS collection, we identified the top 20 authors with the most articles related to meta-analysis. Using coword analysis, 2882 articles were collected to cluster author collaborations and identify the top 3 authors with the highest weighted centrality degrees. Based on the CJAL (category, journal raking by impact factor, authorship, and L-index on article citation) score and absolute advantage coefficient (AAC), we compared the IRAs and identified the author who dominated the field significantly beyond the next 2 authors. In WoS collection, coword analysis was used to highlight the characteristics of research domains for the top authors contributing to meta-analyses. The selection of articles that deserve reading is based on a temporal heatmap.

Results:

The top 2 authors were Young–Ho Lee (South Korea), Patompong Ungprasert (U.S.), and Brendon Stubbs (US) with CJAL scores of 240.71, 230.99, and 240.71, respectively. Based on the weak dominance coefficient (AAC = 0.49 < 0.50), it is evident that the leading meta-analysis author does not possess a significant dominant position over the next 2 leading authors in IRAs. Coword analysis was used to illustrate the characteristics of the 3 authors research domains. The 3 articles worth reading were selected based on a trend analysis of the last 4 years using the temporal heatmap.

Conclusion:

A coword analysis of meta-analysis studies identified 3 leading authors. There was no evidence that 1 author possessed a dominant position due to the lower AAC (=0.49 < 0.50) for the leading author. As we have demonstrated in this study, the CJAL score and the AAC can be applied to many bibliographical studies in the future.

Key points:

• •A coword analysis was applied to highlight the leading authors and their research domains, which is a novel and modern approach in bibliometrics.
• •It was determined whether an author possessed a dominant position over the next two leading authors on the basis of the CJAL score and the absolute advantage coefficient (AAC), a crucial aspect of this study.
• •The study recommends that future relevant studies consider the CJAL score and AAC as indicators to verify a leading author who has made a significant contribution to a field of research in the academy.

1. Introduction

Meta-analyses are statistical analyses that combine the results of multiple scientific studies.^[1] When there are multiple scientific studies that address the same question and each study reports measurements that are expected to contain some degree of error, meta-analyses can be conducted. Based on how this error is perceived, we can use statistical approaches to derive the pooled estimate that is closest to the unknown common truth.^[2] Evidence-based medicine considers meta-analytic results to be the most trustworthy source of evidence.^[3,4] The number of publications related to meta-analysis has increased significantly over the past decade.^[5] Research should be conducted on the authors who have made the greatest contribution to the field of meta-analysis.

1.1. Evaluating research achievements using metrics

Studies of astronomy in the 17th century are believed to be the origins of meta-analysis,^[6] while a paper published in the British Medical Journal by the statistician Karl Pearson in 1904^[7] consolidated data from multiple studies of typhoid vaccination, marking the first time the results of multiple clinical studies were aggregated using meta-analysis.^[8,9]

In most cases, a meta-analysis is preceded by a systematic review, which allows for the identification and critical evaluation of all relevant evidence (thereby minimizing the likelihood of bias in summary estimates).^[2] A few meta-analyses took into account article citations and trends when selecting or reporting studies, without mentioning the authors contributions to the field.

Authors contributions are commonly evaluated by author research achievements (RAs) using a variety of metrics (e.g., the author impact factor, the number of citations to the top or 10th most cited publication, and the number of publications with at least 10 citations).^[10] To evaluate the RAs of authors, some bibliometric indices (e.g., the h-/g-/x-/Y-/hT-/L-/category, journal impact factor, and authorship (CJA)-/CJAL (category, journal raking by impact factor, authorship, and L-index on article citation)-index^[11–18]) considered both the number of citations and publications. However, it is important to note that these indices suffer from a number of limitations, including ignoring journal impact factors (JIFs), article types and not taking into account authorship in their respective fields.

1.2. Authors with > 100 articles versus >100 citations

Over the past 10 years, there has been a significant increase in the number of top-cited articles^[19] (i.e., [18, 12, 11, 24, 29, 37, 57, 80, 91, 114] by count between 2013 and 2022). In contrast to those research articles examining author contributions based on top-cited articles only,^[20,21] no studies have found that; The leading authors contributing to a discipline or research topic should be compared and; Articles worth reading in this field should be identified based on their citation trends.

Higher RAs are generally associated with more articles written by an author, regardless of the metric used for computing these indices (e.g., h-/g-/x-/Y-/hT-/L-/CJA-/CJAL-index^[11–18]). According to the Kano model for the x-index,^[22–24] publications are more closely associated with these indices than citations. In this regard, it is necessary to screen out articles based on their author publications first and then to evaluate their citations later.

1.3. A single meta-analysis author dominates the field (1st question)

In these indices (e.g., h-/g-/x-/hT-/L-index^[11–15]), authorship, journal prestige, and document type are not taken into account. A Y-index^[14] is calculated based only on publications by first and corresponding authors. There are no citations involved in the CJA score.^[17] As such, the CJAL score^[18] considers 4 factors contributing to RAs, including; The category (e.g., review, original article, case report, etc); The journal “quality” (J; e.g., impact factor, JIF, or ranking of the journal); The authorship order denoted by A), and; The article citation using the L-index.^[16] Similarly, the CJAL has the disadvantage of not taking into account the extent of dominance (hegemony). RA can be evaluated using the absolute advantage coefficient (AAC)^[24] in contrast to those with comparative advantage (RCA) (or Balassa Index),^[25] which measures a country’s relative advantage or disadvantage as a result of trade flows in a certain class of goods or services. Thus, it is necessary to verify that the leading author has a dominant RA in a meta-analysis using the AAC.^[24]

The first research question is to identify the leading meta-analysis author who has a dominant RA in comparison to his or her counterparts.

1.4. Citation cartels should be cautious in author collaborations

In 1999, Franck published an essay describing citation cartels as collaborations between editors and journals for mutual benefit.^[26] One way editors can increase their journal’s impact factor (IF) is through inter-journal citations.^[27] More recently, citation cartels have also been observed between editors and authors.^[28] These cartels involve members citing each other’s papers for mutual gain, regardless of familiarity.^[27]

Modern semantic web tools can aid in detecting citation cartels online,^[28] but it is difficult to confirm their existence in the real world without detailed analysis.^[29–31] A self-citation rate exceeding 20%, 15%, or 10% may be a temporary indication of a citation cartel.^[29–31]

The relationship between productivity and impact of scientific production is a long-standing debate in science.^[32] Evidence suggests that academics are hesitant to make concurrent changes to their productivity and journal prestige levels over consecutive career years.^[32] However, it remains unclear whether citation cartels exist within articles. Journal editors and reviewers are aware of the phenomenon of citation cartels.^[27] It should be cautious when evaluating author RA based on article citations.^[33]

1.5. Articles worth reading required for readers (2nd question)

In bibliometric analysis,^[3,4] documents can be grouped by topic (e.g., meta-analysis) for a specific feature (e.g., citations and publications). This method can be used by researchers to gain a better understanding of the landscape of a particular research topic and to determine the direction for future research.^[34–36] It has been essential to bibliometrics to be able to detect trends in articles, such as the top 11 references with the highest bursts of citations^[24] and a temporal bubble graph (TBG) using the CiteSpace package.^[37] Cite Space’s TBG has 2 drawbacks: Articles are cited references rather than active articles that are relevant to the current research, and; No articles that deserve to be read are included in the TBG based on the citation trend. A challenge lies in selecting articles for reading based on recent citation trends rather than total citations that are too old to be useful to readers.

The second research question focuses on identifying articles worth reading based on meta-analysis articles.

1.6. Study aims

In this study, we aim to inspect the 2 hypotheses that; The leading bibliometric author has a dominant RA and; The temporal heatmap can select articles that deserve to be read based on trend analysis.

2. Methods

2.1. Data sources

We searched the keywords (TS = meta-analysis) in the web of science core collection (WoSCC) based on the top 20 authors who published the most articles related to meta-analysis. A total of 2882 articles were downloaded and analyzed; see Supplemental Digital Content 1, http://links.lww.com/MD/I788.

As this study did not involve the examination or treatment of patients or review of patient records, it was exempt from review and approval by our research ethics committee.

2.2. Two major indicators used in this study

1.2.2. The CJAL score

The CJAL score^[18] comprises 4 components, including document type, journal impact based on SCI/SSCI-indexed papers,^[17] authorship, and the L-index^[16] based on article citations via Eqs. 1 to 3. The criteria and thresholds are displayed in Figure 1.

Figure 1.

Seven article entities with the top 5 elements shown on the Sankey diagram (note. AACs are shown below for explaining the dominance strength of top 1 beyond the next 2 in publications). AAC = absolute advantage coefficient.

$${\displaystyle \begin{array}{l}{\displaystyle CJAscore={\displaystyle \underset{i=1}{{\displaystyle \sum ^n}}}{C}_i\times J_i\times A_i}\end{array}}$$ (1)

$${\displaystyle \begin{array}{l}{\displaystyle CJALscore={\displaystyle \underset{i=1}{{\displaystyle \sum ^n}}}{C}_i\times J_i\times A_i\times L-inde{x}_i}\end{array}}$$ (2)

L-index = round (log ($\frac{Citation}{A_n\times Age}$ +1),0), >=1 (3)

2.2.2. The AAC

The AAC is determined by the 3 consecutive numbers of values (e.g., RA in this study in descending order denoted by A1, A2, and A3 in Eqs. 4–6).^[38]

AAC = (R₁₂/R₂₃)/(1 + (R₁₂/R₂₃)), (4)

R₁₂ = A1/A2, (5)

R₂₃ = A2/A3, (6)

The AAC ranged from 0 to 1.0, representing the strength of dominance for the top 1 when compared to the next 2 counterparts in RAs. Through the computation of AAC, the dominance strength in RA can be measured and judged by the effect size, with criteria of < 0.5, between 0.5 and 0.7, and not <0.7 as the small, medium, and large effect sizes, respectively.^[24]

2.3. Three approaches applied to this study

1.2.3. Descriptive analytics of 2882 articles

Visualizations were involved in this section, including the Sankey diagram^[24,39] and 4-quadrant radar plots.^[18] In the former, edges (i.e., connections between adjacent members) were used to depict the relationship between the top 5 article entities in publications, and in the latter, the CJAL scores were presented for entities such as countries of origin, research institutes, departments, and authors. Moreover, the 4 attributes of journals, publication years, subject categories, and research areas in publications, IFs (i.e., mean citations), and AACs would also be displayed on a 4-quadratn radar plot.

2.2.3. To select three leading bibliometric authors

Using social network analysis (SNA)^[40,41] with the Pajek package (in Koeln; Pajek Man in Osoje [Ossiach, Austria]),^[42] coword analysis was conducted on author collaborations and keywords. In this study, the higher the weighted centrality degree, the more articles exist in each entity (e.g., author and keyword). In an article, each author (or keyword) has an equal weight (=1/L, where L is the number of elements in the article).^[43–45]

Therefore, authors with a large number of coauthors would have a higher weighted article when compared to authors with a smaller number of coauthors in their bylines.^[43–45] SNA will be used to select the top 3 authors.

The radar plot was used to compare AACs on seven aspects, including article IFs, counts, CJAL scores, and the elements of CJAL (i.e., categories, journal rankings, authorships, and L-index mentioned in Section 2.2.1). The dominance strengths in RA using the AACs were compared for the top 3 productive authors of the meta-analysis by the effect size, with criteria of < 0.5, between 0.5 and 0.7, and not <0.7 as the small, medium, and large effect sizes, respectively.^[24]

3.2.3. Worth reading articles selected by HTM

The top-cited articles in the 3 productive meta-analysis authors were selected to compute their growth trend (denoted by the correlation coefficient between citation counts and a series of numbers from 1–4 in types denoted by increasing, ready to rise, slowdown, and declining). The trends of the worth reading articles in meta-analysis based on the leading authors were displayed and highlighted using temporal heat map (THM). The growth types refer to previous studies.^[38,46]

Moreover, 20 top-cited articles in leading authors were displayed on a dot plot (namely, the impact beam plot^[47]). The article characteristics for the 3 leading authors in the meta-analysis were clustered using coword analysis of keywords in web of science (WoS).

2.4. Creating dashboards on google maps

All graphs were drawn by author-made modules in Excel (Microsoft Corp). We created pages of HTML used for Google Maps. The method of how to draw the THM is deposited with a PDF file and an MP4 video in Supplemental Digital Content 2, http://links.lww.com/MD/I789 and 3, http://links.lww.com/MD/I790.

3. Results

3.1. Descriptive analytics of 2882 articles

The most productive entities include the year 2019, the US, Mayo Clin (US), and Korea University in first authors, the US in corresponding authors, the journal of Plos One, Medicine, General & Internal in subject categories, and General & Internal Medicine in research areas, as shown in Figure 1.

Higher CJAL scores were found in the US, Mayo Clinic (US), Medicine School, and the author Young-Ho Lee from South Korea (Fig. 2). The US and Dr Lee are the only 2 leading counterparts with dominant strength (>0.70).

Figure 2.

Four attributes of article entities in comparison of AACs using the 4-quadrant radar plot (Note: FP = publications of first author, RP = publication of corresponding author; the coordinates are based on Y-index). AAC = absolute advantage coefficient.

Nature Genet, in 2013, Genetics & Heredity, had higher CJAL scores in the subject categories and research areas (Fig. 3). According to this analysis, only the year 2013 exhibits a weak dominant strength (=0.25 < 0.70) when compared to other years.

Figure 3.

Four attributes of article entities in comparison of AACs using the 4-quadrant radar plot (note. citations and publications are on the 2 axes to represent the location in radar plot; the bubbles are sized by the mean citations). AAC = absolute advantage coefficient.

3.2. To select three leading bibliometric authors

The top 3 authors were Young-Ho Lee (South Korea), Patompong Ungprasert (U.S.), and Brendon Stubbs (US) with CJAL scores of 240.71, 230.99, and 240.71, respectively, using author collaboration analysis (i.e., coword analysis or SNA)^[36,37] (Fig. 4).

Figure 4.

Using SNA to select the top 3 leading authors in meta-analysis articles (Note: bubbles are sized by the weighted counts in publications; the bigger, the more publications were attributable to the entity). SNA = social network analysis.

Comparisons of seven AACs for 3 leading authors were made to determine a solely dominant contribution to meta-analysis in Figure 5. A weak dominance coefficient (AAC = 0.49 < 0.50) for the CJAL scores was found. It is evident that the leading meta-analysis author does not possess a significant dominant position over the next 2 leading authors in individual research achievements.

Figure 5.

Comparisons of seven AACs for 3 leading authors to determine a solely dominant contribution to meta-analysis (Note: the research achievements are based on the CJAL score, including article category, journal ranking by impact factor, authorship, and article citations being taken into account). AAC = absolute advantage coefficient.

3.3. Worth reading articles selected by HTM

THM presents the top 10 articles each published by the 3 leading authors of meta-analyses (Fig. 6). The selection of articles worthy of reading is based on citation trend with type 1, which indicates that their article citations have increased over the last 4 years since 2019. As shown in Figure 6, not all articles with more citations are deemed to be worthy of reading since their citations have not continued to increase over the past few years. Each author’s top article will be abstracted in the Discussion section.

Figure 6.

Selecting articles worth reading using the THM (Note: BS = burst strength; growth = correlation coefficient between years and citations over the last 4 years; PMID = PubMed unique identity number).

The top 20 articles each from 3 leading authors were displayed on a dot plot (namely, the impact beam plot^[47]) and colored by author, as shown in Figure 7. Readers are invited to scan the QR-code in Figures and click on the dot of interest to see the article abstract on PubMed.

Figure 7.

Top 20 articles in leading authors are displayed on a dot plot and colored by author (note. each dot can be linked to PubMed if the QR-code is scanned and the dot of interest is clicked).

Figure 8 illustrates the characteristics of the 3 authors research domains based on coword analysis. Dr Lee is primarily interested in clinical trials, gene polymorphisms, and systemic lupus erythematosus. The research of Dr Ungprasert focuses on risk, bias, and physical activity. The focus of Dr Stubbs research is on prevalence, inflammation, and mortality.

Figure 8.

Article characteristics identified by keyword Plus in WoS using coword analysis (Note: through the network chart, we can capture the major research domain for each researcher easily and apparently). WoS = web of science.

3.4. Online dashboards shown on google maps

All the QR codes in Figures are linked to the dashboards.^[48–52] Readers are suggested to examine the displayed dashboards on Google Maps.

4. Discussion

4.1. Principal findings

We observed that the top 3 authors were Young–Ho Lee (South Korea), Patompong Ungprasert (U.S.), and Brendon Stubbs (US) with CJAL scores of 240.71, 230.99, and 240.71, respectively. Based on the weak dominance coefficient (AAC = 0.49 < 0.50), it is evident that the leading meta-analysis author does not possess a significant dominant position over the next 2 leading authors in individual research achievements. The 3 articles worth reading were selected based on a trend analysis of the last 4 years using the temporal heatmap.

Accordingly, the 2 hypotheses that; The leading bibliometric author has not a dominant RA and; The THM can be used to select articles that deserve worth reading have been confirmed.

4.2. Additional information

The leading country in the meta-analysis was the US, with an extremely higher AAC (=0.93) (Fig. 2), in contrast to the leading county of China, with a weak AAC (=0.21) in bibliographical studies.^[5] The Mayo Clinic (US) in meta-analysis has a strong CJAL (=1076.70) higher than the leading research institute of Huazhong Univ Sci & Technol (China), with CJAL = 17.68 in the bibliometric field.^[5] However, the leading department in the meta-analysis and bibliometric fields is identical to the Medicine School,^[5] indicating that most papers in meta-analysis and bibliometric articles are authored by researchers working in the medicine department (or medicine school because the abbreviation of Med is applied in WoSCC).

Although the TBG was applied to select articles wording reading with frequently cited green bars to denote burst spots in the Cite Space package,^[37] the THM used in this study has many advantages over the TBG, such as additional information on maximum counts and citation trends in recent years contained in the THM. The articles worthy of reading can thus be highlighted and selected for readers to read.

Meta-analysis is a statistical analysis that combines the findings of several scientific studies. There were no articles that provided readers with more information about article citation trends. It is essential to highlight the impact and significance of studies in meta-analyses when comparing the findings of several scientific studies, so THM can be used to supplement the selection of articles worth reading related to meta-analyses in the future.

The combination of existing knowledge in new ways is seen as a prerequisite for important research that will inspire further investigation. A novelty index is usually calculated by looking at the number of unusual combinations of references cited in a paper,^[53] which is aligned with our criteria for selecting articles that are worth reading in light of recent increases in citations, as we did in this study.

This study has the advantage of providing a comprehensive indicator of CJAL,^[18] which comprises 4 components: document type, journal impact factor, authorship, and article citations. CJAL is more comprehensive than other metrics (including the h-/g-/x-/Y-/hT-/L-/CJA-index^[11–17]), but its computation is somewhat complex, which can be overcome due to advances in computer techniques and programming. Furthermore, the AAC^[38] was applied to evaluate the hegemony in the field, a topic rarely discussed in the literature. As a result, this study does not support the hypothesis that the leading meta-analysis author has a dominant RA.

THM was applied to citation trends of articles (Fig. 7), as opposed to the traditional TBG applied solely to keywords or referenced articles in Cite Space^[37] (rather than active articles). As a consequence, the second hypothesis that articles worth reading can be screened out by the THM has been confirmed.

4.3. Most worth reading articles

The article (PMID = 26811368) cited 144 times was authored by Dr Lee (South Korea) et al^[54] and published in 2016, with increasing citations of (21, 23, 31, 38) in the last 4 years. The authors addressed 15 reports including 26,101 patients with systemic lupus erythematosus with 4640 deaths met the inclusion criteria. The all-cause standardized mortality ratios were significantly increased 2.6-fold in patients with systemic lupus erythematosus compared to the general population. The risk of mortality was significantly increased for mortality due to renal disease, cardiovascular disease, and infection but not for mortality due to cancer.

The article (PMID = 24424839) cited 64 times was authored by Dr Ungprasert (US) et al^[55] and published in 2014, with citations of (4, 14, 30, 16) in the last 4 years. The authors performed a meta-analysis to assess the risk of venous thromboembolism in patients with rheumatoid arthritis compared with non-rheumatoid arthritis (RA) participants. The risk was significantly increased in every study design.

The article (PMID = 28122130)^[56] cited 619 times was authored by Dr Stubbs (US) et al and published in 2017, with citations of (108, 145, 141, 130) in the last 4 years. The authors addressed that levels of several cytokines were elevated in patients with major depressive disorder compared to healthy controls, but levels of several other cytokines were not significantly altered.

4.4. Implications and changes

There are 3 features with the potential to make possible changes to bibliographical studies in the future:

1. The THM is unique and modern compared to the traditional TBG shown in Cite Space,^[37] without taking into account both the maximum counts and the growth trends provided to readers. It is possible to select articles that deserve to be read as we did in this study. Furthermore, the recent 5-year article citations are displayed in WoSCC’s reference reports. Researchers will be able to sort articles worth reading if the THM is applied to WoSCC.

2. According to the CJAL score and the AAC, an author who possessed a dominant position over the next 2 leading authors can be compared. These 2 factors are crucial to the success of this study. Scopus and WoS annually announce the top 2% of authors with the most publications in each discipline. It is unclear whether the leading researcher has a dominant position over other researchers. As a viable and applicable indicator, the AAC could be used by Scopus and WoS to determine the level of output hegemony in the future.

3. Several bibliographical studies used visualization to explore only 1 aspect of research (e.g., Citation analysis; Theme exploration; Topic analysis; Reference citations; Research; Research achievement; Author collaboration, and; Trend and hot spot in terms and articles^{[22,23,38,45,57,58]}). It was not found that such research was conducted to condense information in a concise 4-quadrant radar plot (e.g., Figs. 2 and 3) in contrast to those providing many tables and graphs (such as the 1^[59] with 15 tables and 27 figures in an article) to explore the knowledge of interest and noninterest to readers.

4.5. Limitations and suggestions

A number of issues need to be examined in further research. The first concern is that the data and leading authors were retrieved from WoSCC. It is possible that publications in other major citation databases (e.g., Scopus, PubMed, Google Scholar, etc) have been overlooked, and the number of citations has been underestimated.

Second, the dashboards in Figures are displayed on Google Maps. It is not free to use Google Maps due to the use of an application programming interface (API) that requires a paid project key. In the absence of such an application programming interface, the dashboard limitations are not publicly accessible.

Third, when measuring the RAs using the CJAL score, we assume that the first and corresponding authors have equal contributions to articles. If the authors are not placed in either the first or corresponding position, the CJAL score and AAC may differ from the results: the leading author may not have a dominant RA in the field of bibliometrics (i.e., only the first and corresponding authors are considered in the CJAL score).

Forth, calculating the CJAL score requires considerable computational effort. Developing this technology on computer programming will require dedicated software in the future. As a result of the JIF and the journal ranking in Journal Citation Reports on WoS, the CJAL is suitable only for articles that were indexed in WoS.

Fifth, the phenomenon of citation cartels is well known among journal editors and reviewers.^[26,32] However, this study does not delve deeply into this topic. Previous research^[27] developed a module to calculate the self-citation rate of specific journals, which offers additional understanding of citation cartels and warrants ongoing investigation.

Finally, as a result of the time effect, there may be biases in study findings resulting from article citations, which may impact CJAL scores in some way. Since we analyzed articles published as of the end of 2022, some recently published but important literature may have received fewer citations and may even have been omitted.

5. Conclusion

It was achieved by confirming the 2 hypotheses that; The leading bibliometric author has not a dominant RA and; The THM can select articles that deserve to be read based on trend analysis. The study approaches used in this study, such as the CJAL score, the AAC, and the THM, can be replicated in other fields or on other topics in the future and are not restricted to meta-analysis alone as we did in this study.

Acknowledgments

We thank Enago (www.enago.tw) for the English language review of this manuscript.

Author contributions

Conceptualization: Julie Chi Chow, Sam Yu-Chieh Ho, Tsair-Wei Chien.

Methodology: Tsair-Wei Chien.

Investigation: Willy Chou.