Assessing the Quality of Primary Care Research in the United Kingdom: The 2008 Research Assessment Exercise

Universities in the United Kingdom receive separate funding from the government for their teaching and research activities. For the past couple of decades, the research grant of each institution has been determined by the results of national peer-based Research Assessment Exercises (RAE) conducted in 1986, 1989, 1992, 1996, 2001, and 2008 (http://www.rae.ac.uk/). The amount of money received depends on both research quality (with higher quality work attracting disproportionately more money than lower quality activity) and total number of people evaluated.

For the 2008 RAE, universities were invited to submit information about research active staff to 1 of 67 discipline-specific subpanels comprised of experts in the field or users of research. Universities were free to decide how many staff and to which subpanel they wished to submit. Details were supplied about 4 research outputs (predominantly academic papers, books, or monographs) produced between 2001 and 2007 by each fulltime staff member, with reductions allowed for circumstances which could have adversely affected an individual’s contribution to the submission, such as family or domestic matters, illness or disability, engagement in major long-term projects, or early career status. Information was also supplied about the number of research students, studentships, postgraduate degrees awarded, and external research income obtained over the same time period; and statements provided about the research environment and indicators of esteem for those submitted. Using predefined criteria, the subpanels assessed all of the information to produce, for each university submitting to their discipline, separate quality profiles for: research outputs, research environment, and indicators of esteem. The profiles indicated the proportion of research (in 5% aliquots) that was deemed, in terms of originality, significance and rigor, to be 4^* (world-leading), 3^* (internationally excellent), 2^* (recognized internationally), 1^* (recognized nationally) or unclassified (standard below recognized nationally or not meeting the research definition). The 3 components were then combined, using a weighting of 75% for research outputs, 20% for research environment, and 5% for esteem, to produce an overall quality profile.

Subpanel 8 (Primary Care and Other Community-based Subjects) assessed applied, theoretical, or methodological research that was focused on, in, or for primary care. There were 11 subpanel members,^* with additional advice and calibration provided by an international advisor, van Weel from Nijmegen, the Netherlands. A benchmarking exercise was conducted before the subpanel assessed a total of 670 outputs submitted by 14 universities, relating to 170 (151.3 full-time equivalent) staff (median 13 per institution, range 5–19). Each output was assessed by at least 2 members of the subpanel, to reach a consensus score.

The outputs covered an impressive range of disciplines and subject areas, using a range of quantitative and qualitative methods. They were published in a variety of general medical and specialist media, and included many examples of excellent clinical trials (including multi-center investigations involving many institutions), qualitative studies, epidemiologic studies, and systematic reviews. A considerable proportion of the work had direct clinical relevance and impact. In many instances, the work capitalized on primary care’s central position within the National Health Service (NHS) in the United Kingdom, and the registered lists of practices, to provide a population perspective to clinical/individual patient care research. This enabled the researchers to assess the societal as well as individual relevance, impact, and costs of epidemiologic, diagnostic, and therapeutic advances. The best submissions reflected multi-disciplinary and multi-professional research, appropriately employing both qualitative and quantitative techniques to answer questions of obvious importance to primary health care. Highly rated institutions invariably displayed good collaborations with their local primary care community and with experts from other disciplines (both clinical and non-clinical). They submitted focused work with a strong theoretical and/or methodological base, and had coherent plans for the future. More than one-half of the outputs were considered to be internationally excellent or world-leading, an assessment compatible with UK primary care research frequently being among the best internationally, especially with respect to innovation and leadership.

A number of universities provided evidence of strong, sustained commitment to building research capacity, for example, by the inclusion of early career researchers, support of a large number of fellowships or postgraduate students, or other career development initiatives. Some institutions had also developed strong infrastructure for their research, such as networks of research active practices or centralized clinical databases. This enabled them to conduct high quality research in an increasingly challenging research environment. Grant income was very good.

Subpanel members who had participated in the previous RAE felt that there had been a discernible increase in: the amount of methodological and multi-disciplinary research being undertaken, the strength of infrastructure underpinning research activities, the overall quality of outputs submitted, and the impact of research on the delivery of health care. Some submissions were larger than in the previous RAE, suggesting sustained investment in primary care by some universities. There were, however, several notable exceptions. It was also noted that many universities with academic departments of primary care did not make a submission to subpanel 8. This may have been because primary care researchers in those institutions are integrated in other research groupings (and so were submitted to another subpanel), or because primary care research in these universities is at a less advanced stage of development. Continued investment in the sector will be essential if advances are to be sustained, and weaker departments are to flourish.