Quality Indicators of Scientific Research

Quality indicators are needed for separating proverbial “chaff from grain”. This is true for scientific research as well which has become a high input enterprise in terms of finance and human resources. Scientific research does not end with successful culmination of experiments. One of the requirements for earning credit for scientific research is publishing the outcome of experiments [1]. Over the last more than 5 decades there has been mushrooming growth of the journals which provided an easy option for researchers to publish their work, even if unsound. The requirement of objective criteria for faculty hiring as well as performance evaluation made it necessary in our country as well to resort to quality indicators for scientific research. Three such indicators are:

Journal Impact Factor

An historical perspective of journal impact factors (JIF) has been given by Eugene Garfield [2, 3]. Currently, JIFs are provided every year by Thomson Reuter [4]. JIF is the ratio of citations in the current year to articles published in the journal in the previous 2 years divided by the number of the articles published in the same 2 years. Obviously, the core factor in JIF is the citations of the articles published in a journal. This implies that the journal attracts manuscripts from the authors whose work is of interest to wider scientific fraternity in the area. Citations, in fact, have a cascade effect. Consciousness of JIF by virtue of individual instinct and motivation and/or institutional mandate has the effect of raising the quality bar of scientific research.

While JIF developed by Thomson Reuter is internationally acceptable, the National Academy of Agricultural Sciences, India (NAAS) has developed the NAAS impact factor (http://naasindia.org/documents/jrnlist.pdf) on the scale of 0–10. The NAAS impact factor does provide a consideration to a large number of scientific society journals or small publications of regional or institutional level in our country but apparently is unfair to high impact and most coveted journals like Nature, Science and a number of Annual Reviews published by Annual Reviews (http://www.annualreviews.org/).

Citations

The citation of a publication, especially, other than self citation represents peer recognition and is accepted as one of the most important indicators of quality. Non-citation would imply that either work is so abstruse that peers could not understand and appreciate, which is rare, or of so low quality that peers did not consider worth citing. An easy way to know one’s citations is using Google Scholar (http://scholar.google.co.in/), entering the name of one of the authors followed by one key word of the title of the publication or the name of journal or a key word of the title of publication (e.g. sharma op biochem pharmacol). Journal impact factor combined with citations would provide a convenient and effective way of performance appraisal of research work. The following permutations and combinations may also be considered for evaluation of the total research of a scientist, division or institution for recognition, awards, and academy fellowships:

1. Cumulative JIF (Sum of the JIF of all the publications)

2. Cumulative citations (Sum of the citations of all the publications)

3. where N is the total number of publications, JIF_i represent the Thomson Reuter impact factor of the journal which contains the ith publication and Cn_i denotes the total number of citations of the ith publication.

The equation at (c) factors in the quality of the publication in terms of impact factor as well as visibility by virtue of citations by the peers. RI, thus, would give a comprehensive assessment, though less rigorous, of the impact of research publications. Table 1 provides a sample format for calculating the three parameters (a, b, c) stated above.

Table 1.

Research Impact Analysis

Name of the researcher: Designation:
Publication number, i	Name of journal	Authors	JIFi (Thomson Reuter)a	Citations, Cni	JIFi(1 + Cni)
1 to N

N is the total number of publications of the researcher, JIF journal impact factor, Cn citations of the publication

^aNAAS impact factor can be used till the Thomson Reuter Impact factors are adopted

h-Index

h-Index is the most rigorous quality indicator of scientific research. This index was devised by J. E. Hirsch who called it h index [5, 6]. A high value of h indicates a high quality of research. A scientist has an index h if h of his/her Np papers (total publications) have at least h citations each, and the other (Np–h) papers have no more than h citations each. For example, if a scientist is rated to have h = 20, it means that 20 of his papers (out of, say, total of 50, Np is 50) were cited at least 20 times each. The remaining 30 were cited less than 20 times each [6]. Obviously, high citations for most of the publications would provide high h-index. Similarly, high citations of publications in a journal raise the JIF of the journal. Some typical values of h-index for different scientific positions in the USA are 10–12 for associate professor and nearly 18 for full professor. Fellowship in the American Physical Society might occur typically for h-index of 15–20. Membership in the US National Academy of Sciences may typically require an h-index of nearly 45 and higher except in exceptional circumstances [6]. h-Index can be obtained from “Find Your h-Index” on Web of Science [7] or calculated from the citations list in Google Scholar.

I am not sure if h-index or citation scores, in any form, are used in awarding fellowships of different academies or awards. In fact, the sanctity of academy fellowships or awards would be in objective criteria in place in order to minimize lobbying, networking, subjectivity and sometimes the culture of patronizing. “Mediocrity breeds mediocrity”. Change is painful but is inevitable in order to usher in the culture of innovation and science for technology. It is important that young researchers and faculty get sensitized to the quality indicators early in their scientific careers. Well begun is always half done. Scientific research has become globalized and is highly competitive. In order to bridge the gap in our scientific output vis-à-vis China, the USA and other nations ahead of us, we must fully integrate our institutions and human resource into international parameters in faculty hiring, doing and managing science, evaluating scientific research and conferring recognition in the form of academy fellowships and prestigious awards.