Skip to main content
NCBI home page
Journal of Public Health Research logoLink to Journal of Public Health Research
. 2023 Feb 21;12(1):22799036221149840. doi: 10.1177/22799036221149840

A new system of authorship best assessment

Luca Saba 1,✉, Michele Porcu 1, Gianluca De Rubeis 2, Antonella Balestrieri 1, Alessandra Serra 1, Mauro Giovanni Carta 1
PMCID: PMC9947697  PMID: 36846303

Abstract

Purpose:

The standard bibliometric indexes (“m-quotient “H-,” “H2-,” “g-,” “a-,” “m-,” and “r-” index) do not considered the research’ position in the author list of the paper. We proposed a new methodology, System of Authorship Best Assessment (SABA), to characterize the scientific output based on authors’ position.

Material and Methods:

Four classes S1A, S1B, S2A, and S2B include only papers where the researcher is in first, first/last, first/second/last, and first/second/second-last/last position respectively were used for the calculation of H-index and number of citations The system was tested with Noble prize winners controlled with researchers matched for H-index. The different in percentage between standard bibliometric index and S2B was calculated and compared.

Results:

The percentage differences in Noble prize winners between S2B-H-index versus Global H-index and number of citations is very lower comparing with control group (median 4.15% [adjusted 95% CI, 2.54–5.30] vs 9.00 [adjusted 95% CI, 7.16–11.84], p < 0.001; average difference 8.7% vs 20.3%). All different in percentage between standard bibliometric index and S2B except two (H2- and m-index) were significantly lower among Noble prize compared with control group.

Conclusion:

The SABA methodology better weight the research impact by showing that for excellent profiles the S2B is similar to global values whereas for other researchers there is a significant difference.

Keywords: Abstracting and indexing, bibliometrics

Introduction

The identification of correct metric system to objectively assess the impact and visibility in literature of a researcher represents a critical need in academic and non-academic perspectives. These are related to the academic and professional progression, to the transnationality, to the commercial effects, to the diffusion of the products and to the relapses that his research has induced in terms of knowledge as well as to the probability to obtain funds and research grants.1

In the past years several methodologies have been suggested in order to quantify the value of the researcher in particular the total number of citations, the Hirsh-index.2,3 These systems have several strengths in their application, and these are accepted as good systems to quantify the impact and visibility in literature of a researcher. However, this strength (the impact of the research papers of a researcher) could be at the same time the weakness of these systems because it is not the production of the researcher (in terms of papers written or leaded) to be computed but the papers he/she authored or co-authored.4

This fact is linked to a new phenomenon: the increase in the number of authors included in a research paper as showed in Nature by Greene.5 Some authors speculated that this increase in the number of authors in a research paper could be explained by the new level of complexity the research where “fewer and fewer people know enough to work and write alone”5,6 whereas other authors hypothesize that the increased number of authors included for a paper is also linked to bibliometric needs and/or honorary authorship7,8 even if there are the International Committee of Medical Journal Editors (ICMJE) criteria that address what the rules that allow a researcher to be consider author or not.9 More recently other indexes such as m-quotient, g index, H2 index, an index, m index and r index10 were introduced to compensate some drawbacks of H-index and number of citations (the details of the indexes and their advantages over H-index are displayed in Table 1).

Table 1.

Various indexes in literature.10

Indexes description Advantages
M-quotient H-index/year last publication-year first publication Compensate based length of career
g index The highest number g of papers that together received g2 or more citations Much more weight for high citation paper
H2 index Highest natural number such that his h(2) most cited papers received each at least [h(2)]2 citations Reduced precision problem
a index Average citation of Hirsch core Evaluate the most productive core
m index Median citation of Hirsch core Evaluate the central tendency of the most productive core
r index The square root of the sum of citations in the Hirsch core Evaluate the citations intensity
Open in a new tab

Moreover, another debatable point is that search and database engine of authors/papers such as Scopus or Google scholar, widely used for bibliometric analysis, include in their output quantification analysis not only the “authors” but also “contributors”: this means that it is possible to see papers with 30 authors and 200 contributors and the system of analysis compute in the same way the authors and the contributors (!) with consequent impact to citations and H-index. Interestingly for multi-authors papers some colleagues, such as Rennie et al.,11 suggested that in our era of multi-author articles, the concept of authorship should be replaced by that of contributorship that is quite different from the authorship. It is noteworthy, that those type of papers, with several contributors, are usually highly cited12 with the paradoxical effect that for some authors most of the research citations is generated by papers where they are not authors but, simply, contributors.

Therefore, we hypothesize that the H-index does not measure the impact of an author correctly because it does not take into account the actual intellectual property. A potential solution could be generating new metric system, that should not substitute the other already used, but could be useful to derive further information necessary to have a better insight of the researcher impact and not the that of the papers where is co-author/contributor and in this scenario the position of the author in the list plays a fundamental role.

The first author is usually the researcher who has made the most significant intellectual contribution to the paper, in terms of designing, acquiring and analyzing data from experiments, and writing the manuscript. The importance of the first author is reflected in the common practice of referring to a paper by the first author’s name for example, “Jones et al. report that. . .” The last author is commonly the senior (lead-group author) who has supervised the research whereas second and second last positions usually represent are attributed to the second most important contributor and second senior contributor. For this reason, we introduce a new criterion that “weighs” the order in the list of authors, because we believe that this balances the effective intellectual property better, and to check its effectiveness we evaluate if, unlike the H-index, it can discriminate the Nobel winners from a group of authors matched for discipline and H-index to the Nobel winners themselves.

The purpose of this paper is to present a new methodology, System of Authorship Best Assessment (SABA), that weight the impact of the position name of the authors by checking this system on two homogeneous (on the basis of traditional H-index) cohorts of high-level researchers but different according to the accomplishment of result of excellence (Nobel Prize). Furthermore, the SABA methodology was applied to other bibliometric indexes.

Method

The SABA methodology was tested into two different ways. Firstly, the four classes listed below were compared among Nobel prize winner and control group for H-index and total number of citations. Secondly, the difference in percentage between global and S2B in the other bibliometric indexes (m-quotient, a-index, m-index, H2 index, g-index and r-index) were confronted between Nobel prize winner and control group.

The SABA methodology considers the position of the authors, and the following groups were considered:

  • S1A = included in the analysis only papers with author in first position

  • S1B = included in the analysis only papers with author in first/last position

  • S2A = included in the analysis only papers with author in first/second/last position

  • S2B = included in the analysis only papers with author in first/second/second-last/last position

  • SnA = included in the analysis only papers with author in first/second//n/second-last/last position

  • SnB = included in the analysis only papers with author in first/second//n/n-last/second-last/last position

The system could be applied to the all the metric systems used:

  • • Hirsch index (H-Index)

  • • Total number of citations (Nc, tot).

  • • Impact factor (IF)

  • • Total number of papers (Np)

Study’s population

In order to test the effect of this metric systems, the System of Authorship Best Assessment was applied to a group of high-level researchers in biomedical field by testing the effects to tow of the most used parameters: H-index and citations number. Two homogeneous (on the basis of traditional H-index) cohort of high-level researchers but different according to the accomplishment of result of excellence (Nobel Prize) were selected. It is assumed that the Nobel Prize is a criterion for the impact of scientific production, or if one has won the Nobel Prize is an irrefutable element of the impact of its production and quality.

In the first phase of the analysis, the winners of the Nobel Prize in Physiology or Medicine from 1997 to 2017 for a total of 50 researchers were therefore included and another group matched for similar H index, possible, age, gender, and topic of research, were matched with the Nobel Prize winners. The global number of researchers assessed was 100.

Four classes analysis

In the second phase a Scopus database analysis was performed and the CSV files with the output was exported for each researcher. Therefore, for each one, all papers were classified according to five categories:

  • S1A: papers with researcher in first position

  • S1B: papers with researcher in first/last position

  • S2A: papers with researcher in first/second/last position

  • S2B: papers with researcher in first/second/second-last/last position

  • Global: papers with researcher in other positions.

Accordingly, the H factor was calculated for S1A, S1B, S2A, S2B, and Global categories (global category included all the papers and therefore represents the current H-index factor as indicated by Hirsch3). The absolute difference in H-index between the S2B and Global as well as the percentage differences was calculated. Moreover, the percentage difference was grouped in four classes (<5%, 5%–10%, 10%, 15%, and >15%).

Bibliometric indexes tests

For each bibliometric indexes including m-quotient, a-index, m-index, H2 index, g-index and r-index (Table 1) the percentage different between global and S2B calculation were calculated in Nobel prize winner and in control group. Subsequently, the differences were compared between the two groups

Outcomes

The primary outcome was to assess the effect of the research position in the author list of the paper on H-index and number of citations between Noble prize winner and control group depending on the four classes (S1A, S1B, S2A, S2B) and Global

The secondary outcome was to compare the percentages difference between Global and S2B class of all bibliometric indexes among Nobel prize winner and control group.

Statistical analysis

The normality of each continuous variable group was tested using Kolmogorov-Smirnov Z test and because the normality was rejected nonparametric tests were applied. Mann-Whitney test was used for comparing all bibliometric indexes between global and S2B class of all bibliometric indexes among Nobel prize winner and control group. A p value <0.05 was regarded to indicate statistical significance and all correlation values were calculated using a two-tailed significance level. R software (www.r-project.org) was employed for statistical analyses.

Results

The summary of H-index and citation analysis according to the System of Authorship Best Assessment for Nobel Winners and control group are given in the Tables 2 and 3 respectively.

Table 2.

Summary table for H factor analysis in Nobel prize winner groups and controls calculated for S1A, S1B, S2A, S2B, and Global categories including absolute difference in H-index between the S2B and Global and percentage difference. In the last column “difference class” the percentage difference between the S2B and Global H are grouped into four classes (<5%, 5%–10%, 10%, 15%, and >15%). The name of the Scientists of the control group are blinded for privacy but are at disposal previous authorization and upon specific request.

Researcher H-index S1A S1B S2A S2B Gap H-index/S-index % gap H N Difference class (%)
Nobel prize winners Jeffrey C. Hall 80 22 61 63 76 4 5.0 <5
Michael Rosbash 96 18 80 81 93 3 3.1 <5
Michael W. Young 54 11 44 45 53 1 1.9 <5
Yoshinori Ohsumi 91 10 68 74 87 4 4.4 <5
Tu Youyou 9 4 9 9 9 0 0.0 <5
Satoshi Omura 77 35 58 60 70 7 9.1 5−10
Campbell WC 25 18 22 23 23 2 8.0 5−10
O’Keefe JM 56 18 50 52 55 1 1.8 <5
Moser MB 56 9 31 36 54 2 3.6 <5%
Moser EI 63 17 48 52 61 2 3.2 <5
Sudhof T 158 32 122 124 145 13 8.2 5−10
Schekman RW 94 17 85 89 92 2 2.1 <5
Rothman JE 105 30 93 94 105 0 0.0 <5
Yamanaka S 87 18 61 61 75 12 13.8 10−15
Gurdon JB 74 43 73 73 74 0 0.0 <5
Steinman RM 148 47 116 124 144 4 2.7 <5
Hoffmann JA 91 16 51 53 79 12 13.2 10−15
Beutler BA 102 43 78 80 92 10 9.8 5−10
Edwards RG 56 35 45 47 48 8 14.3 10−15
Szostak JW 83 14 77 80 82 1 1.2 <5
Greider CW 66 18 56 58 65 1 1.5 <5
Blackburn EH 87 27 74 76 83 4 4.6 <5%
Montagnier L 68 17 40 43 59 9 13.2 10−15
Barré-Sinoussi F 66 9 18 22 39 27 40.9 >15
zur Hausen H 80 46 73 73 79 1 1.3 <5
Smithies O 91 29 66 67 82 9 9.9 5−10
Evans MJ 52 12 30 35 44 8 15.4 >15
Capecchi MR 81 19 70 71 80 1 1.2 <5
Mello CG 53 9 35 38 47 6 11.3 10−15
Fire AZ 70 13 49 56 66 4 5.7 5−10
Warren JR 11 2 3 7 9 2 18.2 >15
Marshall BJ 45 26 37 42 42 3 6.7 5−10
Buck LB 35 12 33 34 34 1 2.9 <5
Axel R 92 13 68 71 88 4 4.3 <5
Mansfield P 41 29 36 38 39 2 4.9 <5
Lauterbur P 42 20 39 39 42 0 0.0 <5
Sulston JE 51 16 31 35 39 12 23.5 >15
Horvitz R 114 13 94 94 107 7 6.1 5−10
Brenner S 80 24 57 63 73 7 8.8 5−10
Nurse PM 94 22 81 83 92 2 2.1 <5
Hunt T 67 20 49 53 64 3 4.5 <5
Hartwell LH 69 30 63 67 68 1 1.4 <5
Kandel ER 148 35 118 125 142 6 4.1 <5
Greengard P 163 20 131 133 155 8 4.9 <5
Carlsson A 87 52 75 83 85 2 2.3 <5
Blobel G 116 23 103 108 115 1 0.9 <5
Murad F 90 28 82 83 87 3 3.3 <5
Ignarro LJ 98 52 80 85 93 5 5.1 5−10
Furchgott RF 40 29 37 40 40 0 0.0 <5
Prusiner SB 144 40 116 119 138 6 4.2 <5
Control group 95 24 62 66 85 10 10.5 10−15
Blinded Name at disposal upon request 96 26 79 83 93 3 3.1 <5
62 24 42 47 58 4 6.5 5−10
103 33 97 98 102 1 1.0 <5
59 11 38 43 52 7 11.9 10−15
76 19 70 71 76 0 0.0 <5
44 11 35 38 42 2 4.5 <5
87 41 78 80 86 1 1.1 <5
83 27 63 72 77 6 7.2 5−10
68 36 60 61 65 3 4.4 <5
140 38 124 128 135 5 3.6 <5
116 11 80 84 104 12 10.3 10−15
126 22 89 103 119 7 5.6 5−10
100 22 69 71 86 14 14.0 10−15
79 10 48 51 68 11 13.9 10−15
131 39 86 92 120 11 8.4 5−10
92 13 71 73 79 13 14.1 10−15
103 29 68 59 86 17 16.5 >15
79 29 57 69 75 4 5.1 10−15
89 25 59 68 81 8 9.0 5−10
64 14 60 61 64 0 0.0 <5
82 25 67 71 77 5 6.1 5−10
76 22 46 52 59 17 22.4 >15
67 22 52 56 60 7 10.4 10−15
89 22 61 62 73 16 18.0 >15
107 19 60 70 79 28 26.2 >15
66 22 41 49 61 5 7.6 5−10
83 5 40 42 62 21 25.3 >15
71 15 56 59 65 6 8.5 5−10
140 14 70 76 121 19 13.6 10−15
41 11 21 24 28 13 31.7 >15
53 25 42 44 48 5 9.4 5−10
88 20 73 74 79 9 10.2 10−15
73 38 62 71 71 2 2.7 <5
53 12 34 41 42 11 20.8 >15
69 16 47 50 60 9 13.0 10−15
65 36 55 58 65 0 0.0 <5
150 38 111 114 138 12 8.0 5−10
94 17 54 60 70 24 25.5 >15
105 31 84 86 96 9 8.6 5−10
93 17 46 66 81 12 12.9 10−15
70 25 54 56 65 5 7.1 5−10
140 50 90 101 114 26 18.6 >15
171 92 155 157 164 7 4.1 <5
89 15 67 67 81 8 9.0 5−10
141 23 70 72 103 38 27.0 >15
93 31 71 75 82 11 11.8 10−15
115 27 88 94 109 6 5.2 5−10
67 44 62 64 66 1 1.5 <5
105 36 68 71 87 18 17.1 >15
Open in a new tab

Table 3.

Summary table for citation analysis in Nobel prize winner groups and controls calculated for S1A, S1B, S2A, S2B, and Global categories including absolute difference in H index between the S2B and Global and percentage difference. In the last column “difference class” the percentage difference between the S2B and total number of citations are grouped into four classes (<5%, 5%–10%, 10%, 15%, and >15%). The name of the Scientists of the control group are blinded for privacy but are at disposal previous authorization and upon specific request.

Researcher Citations S1A S1B S2A S2B Gap of citations % gap cit N Difference class (%)
Nobel prize winners Jeffrey C. Hall 19,308 2135 11,296 12,724 17,968 1340 6.9 5−10
Michael Rosbash 27,910 1091 19,324 20,204 26,326 1584 5.7 5−10
Michael W. Young 11,320 1332 8753 8944 10,687 633 5.6 5−10
Yoshinori Ohsumi 42,567 1811 23,473 25,634 36,907 5660 13.3 10−15
Tu Youyou 564 434 503 505 555 9 1.6 <5
Satoshi Omura 30,376 5011 18,811 20,658 26,617 3759 12.4 10−15
Campbell WC 3225 1789 2618 2687 2834 391 12.1 10−15
O’Keefe JM 21,059 7710 19,843 20,301 20,855 204 1.0 <5
Moser MB 16,327 2188 7569 8483 16,087 240 1.5 <5
Moser EI 18,440 2403 12,663 13,967 17,766 674 3.7 <5
Sudhof T 79,495 10,535 51,753 53,697 72,432 7063 8.9 5−10
Schekman RW 27,350 1921 22,673 24,034 26,559 791 2.9 <5
Rothman JE 40,827 7946 34,981 35,197 40,022 805 2.0 <5
Yamanaka S 54,197 3842 43,211 43,276 48,364 5833 10.8 10−15
Gurdon JB 16,130 7312 15,184 15,473 16,095 35 0.2 <5
Steinman RM 92,661 20,808 63,101 69,180 86,911 5750 6.2 5−10
Hoffmann JA 28,639 4643 16,541 16,915 25,107 3532 12.3 10−15
Beutler BA 54,943 16,929 36,461 41,918 46,613 8330 15.2 >15
Edwards RG 11,914 4862 8764 9410 9647 2267 19.0 >15
Szostak JW 30,584 3634 27,044 29,065 30,250 334 1.1 <5
Greider CW 30,587 6452 19,343 19,888 28,321 2266 7.4 5−10
Blackburn EH 33,387 9223 26,239 28,505 30,933 2454 7.4 5−10
Montagnier L 22,561 1918 13,644 14,115 19,876 2685 11.9 10−15
Barré-Sinoussi F 17,859 4462 5837 7149 10,492 7367 41.3 >15
zur Hausen H 32,877 14,446 28,606 29,488 32,322 555 1.7 <5
Smithies O 46,847 4287 32,746 33,025 40,618 6229 13.3 10−15
Evans MJ 16,995 5982 10,522 12,008 14,268 2727 16.0 >15
Capecchi MR 25,686 3760 20,054 20,246 24,032 1654 6.4 5−10
Mello CG 23,663 4251 20,409 20,883 22,700 963 4.1 <5
Fire AZ 30,672 11,297 22,684 24,285 28,954 1718 5.6 5−10
Warren JR 5765 70 3583 4910 5080 685 11.9 10−15
Marshall BJ 12,752 929 10,939 11,559 11,984 768 6.0 5−10
Buck LB 12,218 4144 11,545 11,781 11,830 388 3.2 <5
Axel R 33,771 1260 24,950 25,942 31,195 2576 7.6 5−10
Mansfield P 6721 4061 5667 5930 6267 454 6.8 5−10
Lauterbur P 7558 3149 6701 6920 7511 47 0.6 <5
Sulston JE 34,048 7065 9148 10,473 11,563 22,485 66.0 >15
Horvitz R 55,864 2191 32,647 33,328 51,537 4327 7.7 5−10
Brenner S 30,727 12,838 21,811 23,250 27,468 3259 10.6 10−15
Nurse PM 31,444 6009 26,568 27,172 30,827 617 2.0 <5
Hunt T 16,137 1794 8375 9337 14,832 1305 8.1 5−10
Hartwell LH 23,290 12,601 21,824 22,276 22,722 568 2.4 <5
Kandel ER 72,653 8660 46,934 50,915 67,280 5373 7.4 5−10
Greengard P 92,951 4921 55,412 58,536 83,333 9618 10.3 10−15
Carlsson A 29,637 13,538 20,839 25,671 27,306 2331 7.9 5−10
Blobel G 44,365 6981 33,766 38,551 43,861 504 1.1 <5
Murad F 29,551 3731 25,693 2136 28,453 1098 3.7 <5
Ignarro LJ 38,417 18,797 29,990 32,569 36,746 1671 4.3 <5
Furchgott RF 20,109 16,708 19,059 20,089 20,109 0 0.0 <5
Prusiner SB 73,607 18,342 52,261 53,597 66,966 6641 9.0 5−10
Control group 38,256 3396 19,435 21,575 33,957 4299 11.2 10−15
Blinded Name at disposal upon request 32,926 6045 22,241 23,793 28,901 4025 12.2 10−15
13,771 1821 6087 7790 11,800 1971 14.3 10−15
51,089 13,901 45,824 46,812 48,936 2153 4.2 <5
11,608 508 4232 5198 8461 3147 27.1 >15
20,377 1941 15,011 15,728 19,602 775 3.8 <5
6551 706 3644 4072 5078 1473 22.5 >15
25,586 10,895 22,291 23,469 25,450 136 0.5 <5
26,424 5246 14,837 18,177 22,351 4073 15.4 >15
16,576 7364 12,840 13,741 15,459 1117 6.7 5−10
59,641 7244 44,743 48,422 56,923 2718 4.6 <5
42,617 1585 22,782 24,125 35,515 7102 16.7 >15
57,480 9274 36,992 44,676 54,276 3204 5.6 5−10
45,260 3644 22,684 23,933 34,304 10,956 24.2 >15
27,363 2389 12,769 13,222 22,356 5007 18.3 >15
64,930 17,342 37,272 40,443 56,369 8561 13.2 10−15
31,923 1613 15,801 16,308 21,478 10,445 32.7 >15
45,894 7228 22,002 24,983 35,122 10,772 23.5 >15
17,836 3429 9918 12,425 14,837 2999 16.8 >15
39,273 5811 17,774 20,828 26,794 12,479 31.8 >15
14,607 2230 13,792 14,013 14,326 281 1.9 <5
26,090 5263 17,131 19,835 22,689 3401 13.0 10−15
22,609 2441 7207 9240 11,696 10,913 48.3 >15
16,737 2519 9290 11,481 13,100 3637 21.7 >15
23,677 1963 12,027 12,450 16,469 7208 30.4 >15
68,940 17,039 26,986 39,097 45,272 23,668 34.3 >15
16,847 1486 5768 8613 14,351 2496 14.8 10−15
22,935 834 7507 8208 13,711 9224 40.2 >15
29,299 4918 24,005 25,306 26,926 2373 8.1 5−10
101,068 6721 42,879 52,209 80,370 20,698 20.5 >15
8621 412 1594 1858 2510 6111 70.9 >15
11,169 3897 6860 7730 8472 2697 24.1 >15
24,896 5545 17,165 17,622 19,738 5158 20.7 >15
22,581 5795 16,272 19,909 21,398 1183 5.2 5−10
11,782 940 4779 7263 7954 3828 32.5 >15
16,832 930 6878 7513 13,305 3527 21.0 >15
16,483 7754 12,743 13,639 15,828 655 4.0 <5
99,709 19,199 59,798 61,277 86,759 12,950 13.0 10−15
52,727 5726 20,005 22,916 29,124 23,603 44.8 >15
34,488 6815 23,057 23,457 30,238 4250 12.3 10−15
32,925 3631 14,332 19,290 26,828 6097 18.5 >15
19,152 4445 12,176 13,064 15,792 3360 17.5 >15
82,428 10,506 27,571 38,537 48,820 33,608 40.8 >15
102,390 31,009 76,547 79,048 89,393 12,997 12.7 10−15
29,125 4691 16,183 16,462 24,155 4970 17.1 >15
90,016 2651 16,227 18,450 35,509 54,507 60.6 >15
30,385 5654 18,656 20,457 23,993 6392 21.0 >15
49,239 5236 32,563 34,885 43,286 5953 12.1 10−15
16,456 6404 13,546 14,698 15,837 619 3.8 <5
38,077 5311 16,135 1395 29,081 8996 23.6 >15
Open in a new tab

From the data analysis it is extremely clear that at the class S2B, the H values of the Noble winners of the are extremely close to the global H index with a mean % difference of 6.54% and 62% of the cases with a variation <5%, in 20% of cases a variation between 5% and 10%, in 10% of cases with a variation between 10% and 15% and only in 8% of cases with a variation >15%; with only 8% of the analyzed researchers with differences >15% between H-index with S2B correction and Global H index. In the control group the percentage differences between H measured with S2B correction and Global H-index showed a statistically significant difference (Wilcoxon analysis showed a p value = 0.0008) with 20% of cases with difference with a difference >15% and an average difference of 10.7%.

The same approach was applied by analyzing the effects to the number of citations and the results are summarized in the Table 3. In this case the mean % difference between those obtained in S2B and the total number is 9.13% with 36% of the cases with a variation <5%, in 34% of cases a variation between 5% and 10%, in 20% of cases with a variation between 10% and 15% and only in 10% of cases with a variation >15%; therefore, only 10% of Nobel prize winners have differences >15% between the number of citations with S2B correction and the total number of citations. In the control group the percentage differences between global number of citations and values obtained with S2B show a statistically significant difference (Wilcoxon analysis showed a p value = 0.0001) where 56% of control group researchers have difference is >15%; average difference 20.3%).

In the Table 4 the percentages of differences are showed in order to have an easy view of the differences in H-index and Citations for Nobel and non-Nobel group whereas in the Figure 1 the boxplot is given.

Table 4.

The percentages of differences are showed in order to have an easy view of the differences in H-index and citations for Nobel and non-Nobel group.

% difference H-index Nobel % difference citations Nobel % difference H-index control % difference citation cit control
5.0 6.9 10.5 11.2
3.1 5.7 3.1 12.2
1.9 5.6 6.5 14.3
4.4 13.3 1.0 4.2
0.0 1.6 11.9 27.1
9.1 12.4 0.0 3.8
8.0 12.1 4.5 22.5
1.8 1.0 1.1 0.5
3.6 1.5 7.2 15.4
3.2 3.7 4.4 6.7
8.2 8.9 3.6 4.6
2.1 2.9 10.3 16.7
0.0 2.0 5.6 5.6
13.8 10.8 14.0 24.2
0.0 0.2 13.9 18.3
2.7 6.2 8.4 13.2
13.2 12.3 14.1 32.7
9.8 15.2 16.5 23.5
14.3 19.0 5.1 16.8
1.2 1.1 9.0 31.8
1.5 7.4 0.0 1.9
4.6 7.4 6.1 13.0
13.2 11.9 22.4 48.3
40.9 41.3 10.4 21.7
1.3 1.7 18.0 30.4
9.9 13.3 26.2 34.3
15.4 16.0 7.6 14.8
1.2 6.4 25.3 40.2
11.3 4.1 8.5 8.1
5.7 5.6 13.6 20.5
18.2 11.9 31.7 70.9
6.7 6.0 9.4 24.1
2.9 3.2 10.2 20.7
4.3 7.6 2.7 5.2
4.9 6.8 20.8 32.5
0.0 0.6 13.0 21.0
23.5 66.0 0.0 4.0
6.1 7.7 8.0 13.0
8.8 10.6 25.5 44.8
2.1 2.0 8.6 12.3
4.5 8.1 12.9 18.5
1.4 2.4 7.1 17.5
4.1 7.4 18.6 40.8
4.9 10.3 4.1 12.7
2.3 7.9 9.0 17.1
0.9 1.1 27.0 60.6
3.3 3.7 11.8 21.0
5.1 4.3 5.2 12.1
0.0 0.0 1.5 3.8
4.2 9.0 17.1 23.6
Open in a new tab

Figure 1.

Figure 1.

Open in a new tab

Box-plot representing the percentage differences in Nobel prize winner groups and controls for H-index and citations.

In order to better understand the impact of the different categories to the H-index and total number of citations a bar-percentage graph was created (Figure 2) where the percentages of the contributions of H and total number of citations derived respectively from each class (S1A, S1B, S2A, S2B, Global) are computed for Nobel and non-Nobel group.

Figure 2.

Figure 2.

Open in a new tab

Percentage effect to H index (panel a) and citations (panel b) of the S1A-S1B-S2A-S2B and Global positions in author list in Noble prize winners and controls. The name of the Scientists of the control group are blinded for privacy but are at disposal previous authorization and upon specific request.

Concerning the other bibliometric indexes all but two (H2- and m-indexes) percentages differences resulted significantly lower in Nobel prize winner comparing with control group. In particular, the median differences for a-index was 3.5% (adjusted 95% CI, 1.6–5.5), for g-index was 3.1% (adjusted 95% CI, 0.8−5.0), for H index was 4.2% (adjusted 95% CI, 1.7%–6.7%), for H2 index was 0.0 (95% CI,−4.5−0.0), for m-quotient was 4.2% (adjusted 95% CI, 1.5–6.7), for m-index 0.5 (adjusted 95% CI, −1.5–2.8) and for r-index was 4.2% (adjusted 95% CI, 2.0–6.0). The complete results are displayed in Table 5.

Table 5.

Analysis of the variations among different metric systems.

Noble prize (n [CI95%]) Control group (n [CI95%]) Median difference p
m-quotient 3.30 [2.22–5.78] 8.80 [6.92–11.02] 4.20 [1.50–6.70] 0.0025
g index 3.85 [3.00–5.64] 7.10 [6.10–9.92] 3.15 [0.80–5.00] 0.0096
H2 index 5.30 [2.53–7.09] 1.85 [0.00–5.30] 0.00 [4.50–0.00] 0.16
a index 1.35 [0.18–2.37] 4.50 [3.10–6.22] 3.50 [1.60–5.50] 0.0003
m index 3.60 [2.20–5.31] 4.30 [2.60–6.04] 0.50 [−1.50–2.80] 0.58
r index 2.80 [1.40–3.48] 6.75 [5.20–9.18] 4.20 [2.50–6.00] <0.0001
H index 4.15 [2.54–5.23] 9.00 [7.16–11.84] 4.20 [1.70–6.70] 0.0014
Open in a new tab

Discussion

The purpose of this paper was to present a new methodology, the System of Authorship Best Assessment, that weight the impact of the position name of the authors as system to better characterize the scientific output of a researcher compared to other methods currently used. The need of such type of system relies on the fact that the increase in the number of authors included in a research paper5 together with the automated method of calculation of scientific output performed by tools such as Google scholar or Scopus, makes complex to distinguish the real impact of a researchers. The Nobel prize winner have less reduction of all bibliometric indexes respect to the control group between Global and S2B class.

The first phase was to test the difference of our model compared to traditional system (H-index and citations) in excellent researchers. The first question was: how we can identify quite “objectively” an excellent researcher? We decided to define such type of researcher as objectively excellent if he/she was awarded with the Nobel Prize. And we found that in this group of 50 people the System of Authorship Best Assessment shows results similar to the conventional H-index and global citation values: it is clear the impact derived from the papers not included in first/second/second-last/last position for the Nobel Prizes winners usually do not play a significant role whereas the most of the results came out from the first/last position and become almost complete by including also the papers in second and second-last position.

In the second phase we wanted to test if the results we found were generalizable or if on other groups of researchers, the system showed difference and quite surprisingly we found that in another cohort of excellent researcher (from bibliometric point-of-view) the system showed a statistically significant difference compared to the Nobel Prize winner groups by lowering the H-index and citations of the controls.

By applying our model seems to be possible to obtain a screenshot of the impact of a researcher by deleting the influences of papers where the author has not preeminent or significant position. Some authors, such as Kovacs,13 suggested to consider the “weight” of the contributions of each author in a paper and the model proposed in the current paper tries to easily optimize this concept with the target to differentiate the global output of a researcher from his/her original capacity/contributions. The use of position in the authors list is not a novel idea for weighting the contribution of a single authors on H-index14,15 or in g-index,16 however, to out knowledge this is the first paper that use Nobel prize winner as standard of reference for excellent research.

The effect of S2B methodology is confirmed also by the other bibliometric indexes. In fact, in all but two indexes the percentages reduction between global and S2B was significantly lower in Nobel winner group comparing with control group. The H-index3 is a well-established bibliometric parameter for researcher evaluation. However, it is not free from drawbacks, in particular it is sensible to field of research, it is sensitive to scientific age, it does not taken into account the context of the citation and auto citations.3 For example, two of the most important physics of all time, Paul Dirac and Richard Feynman had only H-index, calculated by Scholar, of 62 and 58. Some other bibliometric indexes were proposed in literature.10 The present analysis confirmed that Nobel prize winner have more consistent research compared with control group with a more preeminent role in their articles. Interestingly, H2 and m-index have an inverse trend compared to the other ones. However, it could be reasonable that Noble prize winners have more citations in less articles (the articles that support the Nobel prize) respect to the control group in which the citations and the articles have a less skewed distribution. In this set, H2 and m-index could have a paradoxical effect. A finishing touch of bibliometric indexes seems to be important nowadays, in fact, Koltun and Hafner17 demonstrated that the correlation of H-index to physics scientific award decline from 0.39 in 2010 to 0.00 in 2019 mainly due to hyper-authorship. In this set S2B may a reliable tool to overcome this trend.

It is evident that also this new system has limitations because it is possible that some academic biases could occur in the authorship position and because some complex research needs several people working on it, but it is unquestionable that the value of a first position is different from the indeterminate position. Moreover, in this analysis a perfect match for H-index, gender, age, and topic of research was not possible between Nobel and non-Nobel winners. Another limitation is that in some cases some of the control researchers shared publications with the Noble prize winners by generating a bias into the model.

This new System of Authorship Best Assessment could help to better understand the research output and could be useful to compare, in an unbiased way, different researchers in the scientific achievement by further expanding the knowledge derived from the simple H-index.

In conclusion two key results could be derived from this analysis

  1. The H-index and number of citations calculated with the S2B (first/second/second-last/last position) correction for high-level researcher is similar to the global H-index and global number of citations

  2. The percentage difference between H-index and citations calculated with the S2B correction and global H-index and total number of citations is very small for high level researcher (Nobel prize winners) and this evidence was confirmed by the performance of the other bibliometric indexes.

It is hoped that metric database systems (such as Google scholar, Scopus, ISI web, ResearchGate et al) incorporate these parameters in the researcher output quantification options and that further studies are being performed to test this model.

Footnotes

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.

Significancy for public health: Bibliometric index is a critical need for academic and non-academic aspects. A new method is necessary for correctly evaluate research. h-index is limited by several drawback. The System of Authorship Best Assessment may help for better assess the researcher literature impact.

References

  • 1. Edwards MA, Roy S. Academic Research in the 21st century: maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environ Eng Sci 2017; 34(1): 51–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Choi M, Fuller CD, Thomas Cr., Jr. Estimation of citation-based scholarly activity among radiation oncology faculty at domestic residency-training institutions: 1996-2007. Int J Radiat Oncol Biol Phys 2009; 74(1): 172–178. [DOI] [PubMed] [Google Scholar]
  • 3. Hirsch JE. An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 2005; 102(46): 16569–16572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Bornmann L, Daniel H. The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep 2009; 10(1): 2–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Greene M. The demise of the lone author. Nature 2007; 450(7173): 1165–1165. [DOI] [PubMed] [Google Scholar]
  • 6. Adams J. The rise of research networks. Nature 2012; 490(7420): 335–336. [DOI] [PubMed] [Google Scholar]
  • 7. Kovacs J. Honorary authorship and symbolic violence. Med Health Care Philos 2017; 20(1): 51–59. [DOI] [PubMed] [Google Scholar]
  • 8. Tang BL. Letter to the editor: on plurality and authorship in science. Account Res 2018; 25(4): 254–258. [DOI] [PubMed] [Google Scholar]
  • 9. ICMJE. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals, https://www.icmje.org/recommendations/ (2022). [PubMed]
  • 10. Bornmann L, Mutz R, Daniel H-D. Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. J Am Soc Inf Sci Technol 2008; 59: 830–837. [Google Scholar]
  • 11. Rennie D, Yank V, Emanuel L. When authorship fails. A proposal to make contributors accountable. JAMA 1997; 278(7): 579–585. [DOI] [PubMed] [Google Scholar]
  • 12. Elango B. Does co-authored articles receive more citations? Int J Inf Stud Lib 2017; 2: 1–4. [Google Scholar]
  • 13. Kovacs J. Honorary authorship epidemic in scholarly publications? How the current use of citation-based evaluative metrics make (pseudo)honorary authors from honest contributors of every multi-author article. Med Ethics J 2013; 39(8): 509–512. [DOI] [PubMed] [Google Scholar]
  • 14. Zhang C-T. A proposal for calculating weighted citations based on author rank. EMBO Rep 2009; 10(5): 416–417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Sekercioglu CH. Quantifying coauthor contributions. Science 2008; 322(5900): 371. [DOI] [PubMed] [Google Scholar]
  • 16. Liu XZ, Fang H. Fairly sharing the credit of multi-authored papers and its application in the modification of h-index and g-index. Scientometrics 2012; 91(1): 37–49. [Google Scholar]
  • 17. Koltun V, Hafner D. The h-index is no longer an effective correlate of scientific reputation. PLoS One 2021; 16(6): e0253397. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Journal of Public Health Research are provided here courtesy of SAGE Publications

RESOURCES