A comment on “Scientometrics in a changing research landscape”

Comment on: L Bornmann & L Leydesdorff (December 2014)

In their Science & Society article ‘Scientometrics in a changing research landscape’ ¹ Lutz Bornmann and Loet Leydesdorff state that ‘the new UK Research Excellence Framework (REF) puts more emphasis on bibliometric data and less on peer review than did its predecessor’.

Having overseen the UK Research Excellence Framework 2014 (http://www.ref.ac.uk/) for the life sciences as Main Panel A (MPA), we would like to confirm to the community that expert peer review was the principle means of assessing the quality of outputs. Citation data were used, where available and appropriate, as an indicator of academic significance to inform the assessment of output quality but not as a primary tool in the assessment. All evaluations were made under the well-defined criteria outlined in the published REF guidance: Assessment framework and guidance on submissions (http://www.ref.ac.uk/pubs/2011-02/) and the Panel criteria and working methods (http://www.ref.ac.uk/pubs/2012-01/).

For each research, active individual submitted in a unit of assessment, up to 4 outputs (nearly always peer-reviewed original research) were submitted. We wish to emphasize that each output was then subjected to peer review being read by at least two experts within the sub-panel. Main Panel A was clear that citation data could be considered only as a positive indicator of academic significance, recognizing both its limitations and variation across disciplines and units of assessment. Thus, the use of citation played a minimal, but on occasions, useful role. The use of journal impact factors was specifically proscribed.

In conclusion, we wish to stress that for the research profiles determined for REF 2014, to be made public on December 18, 2014, peer review remained the core of the assessment.