Implications of Web of Science journal impact factor for scientific output evaluation in 16 institutions and investigators’ opinion

Abstract

Journal based metrics is known not to be ideal for the measurement of the quality of individual researcher’s scientific output. In the current report 16 contributors from Hong Kong SAR, India, Korea, Taiwan, Russia, Germany, Japan, Turkey, Belgium, France, Italy, UK, The Netherlands, Malaysia, and USA are invited. The following six questions were asked: (I) is Web of Sciences journal impact factor (IF) and Institute for Scientific Information (ISI) citation the main academic output performance evaluation tool in your institution? and your country? (II) How does Google citation count in your institution? and your country? (III) If paper is published in a non-SCI journal but it is included in PubMed and searchable by Google scholar, how it is valued when compared with a paper published in a journal with an IF? (IV) Do you value to publish a piece of your work in a non-SCI journal as much as a paper published in a journal with an IF? (V) What is your personal view on the metric measurement of scientific output? (VI) Overall, do you think Web of Sciences journal IF is beneficial, or actually it is doing more harm? The results show that IF and ISI citation is heavily affecting the academic life in most of the institutions. Google citation and evaluation, while is being used and convenient and speedy, has not gain wide ‘official’ recognition as a tool for scientific output evaluation.

Web of Science (Web of Knowledge) is an online subscription-based scientific citation indexing service maintained by Thomson Reuters that provides citation search, which gives access to multiple databases that reference cross-disciplinary research, and allows for in-depth exploration of specialized sub-fields. Eugene Garfield, the “father of citation indexing of academic literature”, launched the Science Citation Index (SCI), which in turn led to the Web of Science (1). The multidisciplinary coverage of the Web of Science encompasses over scholarly books, journals and conference proceedings. The selection is made on the basis of impact evaluations and comprises multiple academic disciplines, including sciences, social sciences, arts, and humanities. However, Web of Science does not index all journals, and its coverage in some fields is less complete than in others. Web of Science’s usage increased along with the easier access to electronic information.

The journal impact factor (IF), as calculated by Thomson Reuters, was originally created as a tool to help librarians identify journals to purchase, not as a measure of the scientific quality of research in an article. As with other scientific approaches, scientometrics and bibliometrics have their limitations. Recently, criticisms were voiced pointing toward certain deficiencies of the journal IF calculation process, based on Thomson Reuters Web of Science, such as: journal citation distributions usually are highly skewed towards established journals; journal IF properties are field-specific and can be easily manipulated by editors, or even by changing the editorial policies; this makes the entire process essentially nontransparent (2-4). The 2012 San Francisco Declaration on Research Assessment recommended that journal-based metrics should not be used as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions (4).

The aim of the current report was to see how this recommendation is being implemented in different institutions. Contributors from 16 institutions from Hong Kong SAR, India, Korea, Taiwan, Russia, Germany, Japan, Turkey, Belgium, France, Italy, UK, The Netherlands, Malaysia, and USA were invited. The following six questions were asked:

1. Is Web of Sciences journal IF and ISI citation the main academic output performance evaluation tool in your institution? and your country ? (abbreviated as Q1: IF and ISI citation as the evaluation tool).

2. How does Google citation count in your institution? and your country? (abbreviated as Q2: value of Google citation).

3. If paper is published in a non-SCI journal but it is included in PubMed and searchable by Google scholar, how it is valued when compared with a paper published in a journal with an IF? (abbreviated as Q3: Paper in PubMed vs. paper in Web of Science).

4. Do you value to publish a piece of your work in a non-SCI journal as much as a paper published in a journal with an IF? (abbreviated as Q4: Own paper in SCI Journal vs. non-SCI Journal).

5. What is your personal view on the metric measurement of scientific output? (abbreviated as Q5: metric measurement of scientific output).

6. Overall, do you think Web of Sciences journal IF is beneficial, or actually it is doing more harm? (abbreviated as Q6: Impact factor beneficial?).

This report presents the current status of approach towards metric scientific output measurement in 16 institutions, as well as each contributor’s personal views. Our intention in initiating this paper was not to take part in the debate which a lot of discussions have been presented already (5-8). This report also try to avoid draw any conclusion or make any recommendation. Except the first author who initiated and coordinated this effort, the order of the contributors appeared in this report is alphabetical. The results show that IF and ISI citation is heavily affecting the academic life in most of the institutions. Google citation and evaluation, while is being used and convenient and speedy, has not gain wide ‘official’ recognition as a tool for scientific output evaluation. The main limitation of this report is that only one contributor was invited from each institution, their view therefore cannot be confirmed as the consensus representing the official positions of the respective institutions or regions.

(I) Yì-Xiáng Wáng, Department of Imaging and Interventional Radiology, Faculty of Medicine, The Chinese University of Hong Kong, Hong Kong SAR

Q1: IF and ISI citation as the evaluation tool. A: Web of Sciences journal IF and ISI citation are the core output performance evaluation tools in my institution and also Hong Kong universities, besides the competitive grant obtained. For scientific research output evaluation, non-SCI paper counts very little.

Q2: Value of Google citation. A: Google citation is not counted for formal evaluation.

Q3: Paper in PubMed vs. paper in Web of Science. A: In our university all non-SCI papers are grouped together with conference abstracts and count very little.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: For myself as I am tenured already, and I publish a few SCI papers every year anyway. So I do not mind that some of my papers, even good papers, are published in non-SCI journals, as long as they are included in PubMed and searchable. However, my graduate students and post-doctoral fellows are keen to publish in journals with good IF.

Q5: Metric measurement of scientific output. A: Overall, with the current practice in Hong Kong it is probably not a good thing. Sometimes it seems it is not important what we publish, but it is important where, i.e., which journal, we publish. This puts lots of stress on junior faculty members. Some researchers are inevitably attracted to areas which are hot and likely be able to publish in journals with high IF. Even some faculties are investing in these areas though these areas may not be where their traditional strength lies. On the other hand, important subjects as engineering and mathematics tend to have low IF journals. Review papers also tend to generate lots of citations unfairly. For per paper evaluation I think Google citation is a better tool, since if one my work is cited in a non-popular journal (for example in a non-English journal), or in a locally circulating book, this citation may also be valuable.

Q6: Impact factor beneficial? A: There are pros and cons, but till now probably the harms weigh more than the benefits. IF and citation lure investigators to pursue hot areas which may not align with their own strength, for example engineering faculty doing biomaterial research. This can be a dangerous trend and lead to many low quality output. A more comprehensive approach with some conversion factors which allow cross-discipline comparison may be very valuable, and other complementary measurement methods should be further developed. Another problem is by gaining an IF becomes a justification of some low quality journals to seek disproportional commercial profit (9,10).

(II) Dr. Richa Arora, Department of Radiology and Imageology, Nizam’s Institute of Medical Sciences, Hyderabad, India

Q1: IF and ISI citation as the evaluation tool. A: The academic performance of an individual is assessed with multiple factors and Web of Sciences journal IF and ISI citation is one of those factors to judge that in institutes here in India. Publications in indexed journals are the minimum requirement for appointment as faculty as well as for promotion; however, institutes of academic excellence and national importance also take into account the IF of journals.

Q2: Value of Google citation. A: Google citation comes lower in ranking in comparison to PubMed and journal IF in forming an opinion about an individual and it is not taken as a standard criteria for recruitment and annual assessment till now.

Q3: Paper in PubMed vs. paper in Web of Science. A: Papers published in journals with good IF carry more weightage than PubMed indexed journals with lower IF.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: I don’t value my papers solely on the basis of journal’s IF, instead, the liking is based on the type of paper, its topic, the efforts for completing it and its results.

Q5: Metric measurement of scientific output. A: I believe that scientific output or contribution cannot be solely measured based on the number of citations or IF of the journal, as it has several pitfalls. Original research papers get fewer citations as compared to review articles, as the latter can be used as a substitute for earlier literature. Moreover, number of references and type of speciality further influences this. Additionally, a paper published in a journal with low ranking may get more citations and vice-versa. Lastly, freely available articles get more citations as compared to paid articles.

Q6: Impact factor beneficial? A: Web of Sciences journal IF is one of the important tools for evaluating and ranking the journals and to assess academic potential of doctors, provided it should be used judiciously taking into consideration other factors as well.

(III) Yongdoo Choi, Molecular Imaging and Therapy Branch, National Cancer Center, Goyang-si, Gyeonggi-do, Korea

Q1: IF and ISI citation as the evaluation tool. A: Yes. My institution cares about the journal IF of the articles published by each researchers. Journal IF is one of the important factors in the evaluation of the research proposals of scientists. I mean if some researchers want to get bigger research funds from Korean government, they need to have published papers with high IF.

Q2: Value of Google citation. A: Until now, Google citation count is not considered as a standard tool for evaluation of academic performance of scientists.

Q3: Paper in PubMed vs. paper in Web of Science. A: Papers indexed in SCI and SCIE are much more important than the papers which is included in PubMed and searchable by google scholar.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Yes, I do because I have several papers published in non-SCI journals. That papers contain really good data and show scientific advances. However, one thing that disappoints me, I could not get any benefit (such as incentive) after publishing these non-SCI papers.

Q5: Metric measurement of scientific output. A: It should be the citation number for the published papers whatever it is from ISI citation or Google citation.

Q6: Impact factor beneficial? A: Most of the times journal IF could be a good tool for evaluating someone’s achievement in their research fields more objectively. However, it seems that these days most scientists only care about number of papers published in the journals with IF, instead of developing their unique research fields. Manuscript with special interest instead of broad interest could not be published in the journal with high IF. That would be a kind of disaster for scientists.

(IV) Hsiao-Wen Chung, Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan

Q1: IF and ISI citation as the evaluation tool. A: They are both “one of the” performance evaluation tools in my institution and in my country.

Q2: Value of Google citation. A: So far, Google citation counts in neither. Unlikely to be counted in the near future either.

Q3: Paper in PubMed vs. paper in Web of Science. A: Practically speaking, poorly.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Equally, to me because I am relatively established in my academic career. Poorly, to my students who are still struggling in finding their jobs or for promotion. Even for the same paper.

Q5: Metric measurement of scientific output. A: Its presence is as ridiculous as its absence. Having a numerical evaluator is prone to “number pursuing” with priority higher than the scientific contents. Having no numerical evaluator risks paper generating without going through rigorous peer review.

Q6: Impact factor beneficial? A: Both. I personally feel that this ridiculous IF is actually more beneficial to the scientific community than the harms it brings. The reason is because, after all, “paper generating” (in the absence of IF for evaluation) can be easily done by individual with truth hidden, whereas “number pursuing” (in the presence of IF for evaluation) often entails an entire team. And when an entire team is involved in a scandal, the chance that things can be revealed to the public increases. But I have to say that the benefits and harms brought about by IF are both substantial.

(V) Dr. Vyacheslav I Egorov, Head of the Department of Surgical Oncology, Moscow City Hospital No. 5, Sechenov First Moscow State Medical University, Stromynka Street 7, Moscow 107076, Russia

Q1: IF and ISI citation as the evaluation tool. A: Yes, it has become so since several years ago, and it is currently used (along with SCOPUS database) by the governmental research funds and institutions as a criterion to evaluate the quality of the project and/or individual researcher outcome.

Q2: Value of Google citation. A: Not used, at least so far. Some researchers, which have no access to Web of Science and/or SCOPUS, use it to evaluate publications.

Q3: Paper in PubMed vs. paper in Web of Science. A: Google scholar is mainly used to search for relevant papers, but not to evaluate them.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: I would certainly try to publish in a SCI journal, since it will then be counted to evaluate my scientific activity.

Q5: Metric measurement of scientific output. A: I think it works, but of course it cannot be used as the only criterion of a person’s scientific qualification/activity.

Q6: Impact factor beneficial? A: Certainly beneficial, and since it has been established, the community- and the administration- have got clear criterion how to evaluate a researcher and which can be checked by anybody. It is now impossible for ‘honoured’ persons to position themselves as honoured scientists, once they do not have valuable publications.

(VI) Dr. Jens Frahm, Biomedizinische NMR Forschungs GmbH am Max-Planck-Institut für biophysikalische Chemie, Göttingen, Germany

Web of Science is of course known and used, but personally I must admit that the Google Scholar service turns out to be easier, faster, and more comprehensive in finding all my papers. So, since about a year I am using Google Scholar “my citations” and put the corresponding link (and Hirsch index) into my CV.

(VII) Dr. Hiroyuki Kudo, Division of Information Engineering, Faculty of Engineering, Information and Systems, University of Tsukuba, Tsukuba, Japan

Q1: IF and ISI citation as the evaluation tool. A: In Japan, it is still difficult to evaluate achievements of individual researcher by using ISI (or even Google scholar). The reason is that these factors do not include domestic (Japanese) papers. But, of course, the situation is gradually changing to the direction of using these factors and the internationalization. I guess that these comments are true in most of research institutes and universities in Japan. I have a feeling that we should have other factor which includes the contribution in domestic communities.

Q2: Value of Google citation. A: Same as the above comments concerning the ISI.

Q3: Paper in PubMed vs. paper in Web of Science. A: Same as the above comments concerning the ISI. But, the situation is gradually changing to give more weight to searchable papers by PubMed and Google scholar.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: I believe that quality of each paper should be evaluated by the content (not by the name of journal which publishes each paper). So, I personally do not give more credit to a low quality paper even it is published in top journals like IEEE Trans Med Imaging.

Q5: Metric measurement of scientific output. A: The quality and the evaluation of the work itself (NOT the number of papers, NOT the number of papers published in top journals).

Q6: Impact factor beneficial? A: In my case, the number of citations to my published papers is relatively good when compared to other Japanese researchers working in the same field. So, I am always using it to get funding, grant, award, and grade up in my university. However, as I mentioned above, I am NOT thinking that the citations by ISI and the number of top journal papers necessarily reflect the true contribution of each researcher to individual field. So, I guess that it is sometimes very dangerous to use these factors to evaluate each researcher.

(VIII) Mr Suleyman Kuyumcu, Department of Medical Biology, Faculty of Medicine, Sifa University, Turkey

There is a governmental center for all universities. Either government or foundation universities should follow the rules of this center for academic issues. There are few issues that institutes can follow their own rules. This center is called as YOK (Council of Higher Education) and I will answer the questions generally regarding to YOK rules. But I will also add exceptions for some institutes and my university, i.e., Sifa University.

Q1: IF and ISI citation as the evaluation tool. A: YOK: The answer is No and Yes. There is a Turkish national evaluation tool classifying journals into A, B, and C categories. IF is not considered for academic evaluation. In addition these categories also applied to the prizes for publishing paper, but some institutes are giving prizes to papers published in journals with higher rank IFs. My institute (Sifa University) is the same.

Q2: Value of Google citation. A: YOK: Google citation count is not considered in any aspect. Only Web of Science citation report is valued for citation.

Q3: Paper in PubMed vs. paper in Web of Science. A: YOK: IF is never a tool for evaluation. If a paper published in indexed (especially SCI or SSCI) international journals, it is more than valued than unindexed journal paper and national journal paper. Google scholar is never considered. And if a paper is included in PubMed, then it is more valued than unincluded ones, but not in a formal way.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Over all, Turkish scientists do not value a non-SCI paper as much as a paper published in a journal with IF. Some Turkish researchers believe that Turkish journals should be supported and in some cases having a paper published in a Turkish journal, whether being a SCI journal or non-SCI journal, is an obligation for academic evaluation.

Q5: Metric measurement of scientific output. A: I believe that metric measurement should be used as a tool for scientific output however statistics sometimes does not show the reality. A low quality paper can be published in high IF journals and have over-credit for that. And high quality paper can be published in low IF or non-SCI journals and have lower credit than it should deserve.

Q6: Impact factor beneficial? A: I think it is the best available tool currently and we should use it. Especially for the journal evaluation it is really important. However papers should be evaluated with another method. There should be some kind of per paper statistical evaluation tool and thereby every paper should have its own score. Only citation number is not enough since if an author is not popular then his/her paper will be cited less and vice versa.

(IX) Dr. Sophie Laurent, Department of General, Organic and Biomedical Chemistry, Université de Mons, Mons, Belgium

Q1: IF and ISI citation as the evaluation tool. A: Our institution uses Scopus from Elsevier. Other Belgian universities have access to Web of Sciences but the subscription is very expensive.

Q2: Value of Google citation. A: Google citation or Scopus is accepted for activity reports or CV.

Q3: Paper in PubMed vs. paper in Web of Science. A: There were several years IF was very important and we have to indicate it for each publication in our CV but it is not asked now.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Generally, papers published in high IF or with a lot of citations like all “Journals of American Society” are highlighted in the CV.

(X) Dr. Romaric Loffroy, Department of Vascular, Oncologic and Interventional Radiology, Le2i UMR CNRS 6306, Bocage Teaching Hospital, University of Dijon School of Medicine, Dijon Cedex, France

All universities are governed by a center called the National Council of Universities (NCU). There is one NCU per specialty. All universities must follow the strict rules of this center for academic purpose.

Q1: IF and ISI citation as the evaluation tool. A: The answer is mixed. In France in general as in my institution in particular, the academic performance evaluation tool is mainly based on a classification of scientific journals into three categories: A (high-ranking), B (medium-ranking) and C (low-ranking). IF is not directly considered for academic evaluation but A-ranking journals usually have a high IF and vice versa, depending of the specialty. Indeed, some journals may have a low IF despite being considered as the referent journal in the specialty. On the other hand, Web of Science journal IF and ISI citation are often used by our National Council of Universities (one per specialty) for individual evaluation and academic promotion.

Q2: Value of Google citation. A: Google citation does not count neither in my institution nor in France. Only Web of Science citation may count.

Q3: Paper in PubMed vs. paper in Web of Science. A: Google scholar is not considered to evaluate relevant papers but increasingly to search them. However, if a paper is included in PubMed, then it is preferable. Furthermore, publication in a journal with an IF is more valued than without ones.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Publication in a non-SCI journal is not evaluated as much as a paper published in a journal with IF. In France, it’s always preferable to publish papers in journals with IF.

Q5: Metric measurement of scientific output. A: The metric measurement of scientific output is obviously mandatory since it’s the only way to get objective evaluation of the scientific activity of researchers across the disciplines. However, numerical evaluation alone has some limitations. Indeed, it is not always comparable between the different specialties. Firstly, some disciplines have more high-ranking journals as compared as others. Secondly, citation may be different according to the type of article. Lastly, publication in a journal with high IF is not always the guarantee of high quality paper, even if it should be the rule, and vice versa.

Q6: Impact factor beneficial? A: I would say that the perfect method for evaluation of scientific activity of researchers does not exist. Anyone would have limitations. However, Web of Science is probably the “less worst”. I think it remains the best available tool for ranking the journals and to assess the scientific activity of researchers since it provides a quantitative and objective indicator of activity. However, it should not be used solely. Taking into consideration other relevant factors seems to be very important and urgent.

(XI) Dr. Simone Maurea, Dipartimento di Scienze Biomediche Avanzate, Università degli Studi di Napoli Federico II (UNINA), Istituto di Biostrutture e Bioimmagini-Consiglio Nazionale delle Ricerche (IBB-CNR); Fondazione SDN (IRCCS), Napoli, Italy

Q1: IF and ISI citation as the evaluation tool. A: Yes.

Q2: Value of Google citation. A: As a comparative data base for ISI Web of Science.

Q3: Paper in PubMed vs. paper in Web of Science. A: It is better a paper with IF.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Not absolutely, but it is better a paper published in a journal with IF.

Q5: Metric measurement of scientific output. A: Not really in agreement with my opinion, since it is an automatic method.

Q6: Impact factor beneficial? A: Web of Science is a method and as it is, it may be approximative. Since the significance of IF evaluation is, for example, not absolutely related with the major or minor role of an author in a research group; thus, it is considered only the scientific value of a paper, but not the individual value of the single author.

(XII) Dr. Sameh K Morcos, Department of Diagnostic Imaging, the University of Sheffield, Sheffield, UK

Q1: IF and ISI citation as the evaluation tool. A: Yes.

Q2: Value of Google citation. A: Not sure.

Q3: Paper in PubMed vs. paper in Web of Science. A: I judge the quality of the paper regardless the status of the journal. Some important articles are published in obscure journals.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: In academic life unfortunately impact factor remains an important tool in assessing quality of publications.

Q5: Metric measurement of scientific output. A: Has limitations and not necessary offers accurate assessment of scientific output.

Q6: Impact factor beneficial? A: Can be misleading. Research Gate assessment is another forum for giving feedback on scientific publications.

(XIII) Dr. Yicheng Ni, Theragnostic Laboratory, Department of Imaging and Pathology, and Radiology Section, University Hospital, Campus Gasthuisberg, KU Leuven, Belgium

Q1: IF and ISI citation as the evaluation tool. A: In general, yes.

Q2: Value of Google citation. A: It does not count officially, but still used by individuals for the sake of convenience.

Q3: Paper in PubMed vs. paper in Web of Science. A: The former is regarded less important than the latter.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Yes, especially when a new finding has not been recognized by the mainstream, but of course not as much as in an IF journal.

Q5: Metric measurement of scientific output. A: The metric measurement of scientific output would make sense only when are other crucial factors also taken into account, especially the input. For instance, to what extent the output (publications) matches with the amount of founding or investment.

(XIV) Dr. Edwin Oei, Department of Radiology, Erasmus University Medical Center Rotterdam, Rotterdam, The Netherlands

Q1: IF and ISI citation as the evaluation tool. A: Both at my institution (Erasmus MC Rotterdam) and nationwide (The Netherlands) the journal IF and ISI citation score are important indicators of scientific output (probably the most important). For “academic performance” in general, there are other important factors as well, such as external funding (grants), number of PhD dissertations, etc.

Q2: Value of Google citation. A: I believe it is not widely used in The Netherlands, at least not (at all) at my institution.

Q3: Papers in PubMed vs. papers in Web of Science. A: At my institution, publication in a journal with an IF is only valued more than a publication without an IF if that IF is in the top 10 or 25 percentile of the scientific area (see also below). The reason for this is that the departments get some funding from the university based on the number of top-end (IF) publications. However, for grant proposals, a track record with (any) high IF is highly relevant, as applicants are sometimes requires to mention an average impact for their publications in the field.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: Generally speaking, yes. However, I acknowledge that non-SCI journals may contain equally interesting papers.

Q5: Metric measurement of scientific output. A: The IF provides some “objective” way of comparing a proxy of importance between journals. Although there are many disadvantages of IFs, possible confounders, validity issues etc., I believe it will be hard to fight against its use, and, in fact, other possible metrics or alternatives may be equally problematic. One specific problem that I have identified for the field of radiology is the fact that the impact factor cannot be easily compared across disciplines. This is problematic for radiology since our traditional journals usually have a lower impact factor than those of our clinical colleagues. Therefore, if we apply for a cross-disciplinary grant (such as the Netherlands Organisation for Scientific Research (NWO), the Dutch NIH equivalent), for which we need to calculate the average IF of previous publications but only in our discipline, we might easily end up with lower scores than clinical specialists. When we as radiologists choose to publish in a clinical (non-radiological) journal (which increases IF but decreases the visibility/exposure in the own field which is probably more important), this will not always count as it will be “outside our own discipline”.

Q6: Impact factor beneficial? A: It may do some harm (see point 5 above) because there are many confounders and validity issues. However, for some of us, it may also be beneficial and provide a quantitative indicator of success.

(XV) Dr. Akmal Sabarudin, School of Diagnostic and Applied Health Sciences, Faculty of Health Sciences, Universiti Kebangsaan Malaysia, Kuala Lumpur, Malaysia

Q1: IF and ISI citation as the evaluation tool. A: Yes, in University Kebangsaan Malaysia, Web of Sciences journal IF and ISI citation is commonly used to quantitatively measure the academic research performance of each individual. It is a reliable and effective tool to measure the research performance as it has filtered out publications that are published in the journal with low impact or without IF.

Q2: Value of Google citation. A: Google citation provides a platform for authors to keep track on their article citations. In addition, it also creates graph citations over time and calculates the h-index and i10-index for the authors. However, this Google citation is not widely used in Malaysia specifically in Universiti Kebangsaan Malaysia since it contains all articles published online regardless of their academic credential. We are commonly rely on SciVal (www.experts.scival.com) to measure authors research performance in terms of index and publications as it only counts the scientific papers published in the highly impact journal.

Q3: Paper in PubMed vs. paper in Web of Science. A: As we know, not all papers published which are included in PubMed and available by Google scholar have IF. However, all papers published in academic journals with IF are searchable by Google scholar and also included in the PubMed database. Thus, creates a vast difference in terms of value and quality. The readers expect that the articles published with IF are timely and correlate with current problems or global issue which can improve the technology development and social policies. Those high value papers are also produced with best quality research materials, well equipped facilities and the target populations also associated with the relevant current issues. However, it does not mean that articles published without IF have no quality or failed to advance understanding in useful ways or because they contain important errors but sometimes we could find good studies published in the journal without IF. Again, it depends on the perception of the readers how would they value the article.

Q4: Own paper in SCI Journal vs. non-SCI Journal. A: I value my paper equally at high standard regardless of where it has been published either in the non-SCI journal or in the journal with IF. Both papers (non-SCI and SCI) have been through the same processes inclusive of the experimental phase, writing process until it gets accepted to be published in a journal. Therefore, I do not see any reasons why should we double standard those papers.

Q5: Metric measurement of scientific output. A: In my opinion, it is important to measure the scientific output of scientific research in that entire fields inclusive of laboratories, departments, centers and institutions in order to show the achievement or performance for respective projects. In fact, it benefits certain agencies for example funding organizations in order to make funding decision on the particular project based on the evaluation of the overall research project performance.

Q6: Impact factor beneficial? A: Yes, I think the quantitative measurement of research output tool provided by Web of Sciences journal IF to some extend benefits the world of scientific research. It is useful to some agencies in order to evaluate the research performance before a crucial decision making can be made. However, the disadvantage of having Web of Sciences IF is providing a limit or creating a boundary for the readers to highly select the article that they consider has high value and good quality.

(XVI) Dr. Xin Yu, Department of Biomedical Engineering, Case Western Reserve University, Cleveland, USA

Q1: IF and ISI citation as the evaluation tool. A: Maybe, not sure.

Q2: Value of Google citation. A: It’s been used increasingly, although slowly. We do not need to list it in our annual evaluation. But some people put it on their CV when looking for jobs. Others link their website to google scholar.

Q3: Papers in PubMed vs. papers in Web of Science. A: Publication in a journal with an IF is more preferable.

Q4: Own papers in SCI Journal vs. non-SCI Journal. A: Publication in a journal with an IF is more preferable.

Q5: Metric measurement of scientific output. A: IF can be misleading as it might be dependent on the research areas. Some areas that are trendy and attract a lot of researchers are more likely to generate high impact works. Some sort of normalization might correct for this.

Q6: Impact factor beneficial? A: I think it’s neither beneficial nor harmful. Just another metric.