Table 2.
Metrics for Assessing and Communicating the Value of Biomedical Research, by Potential Data Sources
| Bibliometric analysis | Analysis of research administrative documents | Analysis of wider institutional records | Internal policy documents | Funder records | Records of external academic bodies | Educational outcomes and workforce tracking data | Quality improvement data | Public health data | Electronic health records data | StarMetrics data | Patent database | Review of key policy documents | Clippings service | Social media analysis | Feedback forms from events and courses | PI survey | Departmental secretary survey | Wider staff survey | Alumni survey | Survey or interviews with companies | Survey or interviews with research participants | Survey or interviews with community partners | PI interviews | Staff interviews | External researcher site visits or peer review | Representative case studies | High impact case studies | Labor market analysis | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Research impacts | 1 | Number of journal articles published | y | ||||||||||||||||||||||||||||
| 2 | Number of citations | y | |||||||||||||||||||||||||||||
| 3 | Number of research output downloads | y | |||||||||||||||||||||||||||||
| 4 | Mentions in social media | y | |||||||||||||||||||||||||||||
| 5 | Number and size of grant awards | y | y | ||||||||||||||||||||||||||||
| 6 | Number and size of awards from major funders | y | y | y | |||||||||||||||||||||||||||
| 7 | Number of different research funders supporting research | y | y | y | |||||||||||||||||||||||||||
| 8 | Success rate of applications | y | |||||||||||||||||||||||||||||
| 9 | Catalogue of infrastructure | y | y | ||||||||||||||||||||||||||||
| 10 | Use of infrastructure by other researchers | y | y | ||||||||||||||||||||||||||||
| Measures of prestige | 11 | Number of editorships of high profile journals | y | y | |||||||||||||||||||||||||||
| 12 | Number of staff on relevant boards and committees | y | y | ||||||||||||||||||||||||||||
| 13 | Number of academy members | y | |||||||||||||||||||||||||||||
| 14 | Number and type of prizes | y | y | ||||||||||||||||||||||||||||
| 15 | Number of (international) speaker invitations/conference invitations | y | |||||||||||||||||||||||||||||
| 16 | Number of media mentions | y | y | ||||||||||||||||||||||||||||
| 17 | Number of applications per open post | y | |||||||||||||||||||||||||||||
| 18 | Percentage of out-of-state and international applications per research job/PhD post | y | |||||||||||||||||||||||||||||
| 19 | Track record of new hires | y | y | y | |||||||||||||||||||||||||||
| 20 | Undergraduate applications | y | |||||||||||||||||||||||||||||
| 21 | Grade Point Average of incoming students | y | |||||||||||||||||||||||||||||
| Teaching and career development impacts | 22 | Grade Point Average of graduates | y | y | |||||||||||||||||||||||||||
| 23 | Longitudinal data on career progression of students | y | y | ||||||||||||||||||||||||||||
| 24 | Number of PhD graduates | y | y | ||||||||||||||||||||||||||||
| 25 | Completion rate of PhD graduates | y | y | ||||||||||||||||||||||||||||
| 26 | Number of publications per PhD graduate | y | y | ||||||||||||||||||||||||||||
| 27 | 5/10/15-year career outcomes for PhD graduates | y | y | ||||||||||||||||||||||||||||
| 28 | K to R conversion rate | y | y | ||||||||||||||||||||||||||||
| 29 | Career outcomes for researchers | y | y | ||||||||||||||||||||||||||||
| 30 | Subject coverage of the professional development programme | y | |||||||||||||||||||||||||||||
| 31 | Uptake of the professional development programme | y | |||||||||||||||||||||||||||||
| 32 | Feedback on the professional development programme | y | |||||||||||||||||||||||||||||
| 33 | Improved educational attainment/reduced drop out rate | y | |||||||||||||||||||||||||||||
| Research and institutional processes | 34 | Start-up time for research projects | y | ||||||||||||||||||||||||||||
| 35 | Start-up time for clinical trials | y | |||||||||||||||||||||||||||||
| 36 | Average time from funding to publication | y | |||||||||||||||||||||||||||||
| 37 | Proportion of funds spent on administration | y | |||||||||||||||||||||||||||||
| 38 | Support staff to researcher ratio | y | |||||||||||||||||||||||||||||
| 39 | Prompt payment of community partners | y | y | ||||||||||||||||||||||||||||
| 40 | How hiring decisions are made | y | y | ||||||||||||||||||||||||||||
| 41 | How decisions are made to apply for grants | y | y | ||||||||||||||||||||||||||||
| 42 | How publications decisions are made | y | y | ||||||||||||||||||||||||||||
| 43 | Proportion of publications that are open access | y | y | ||||||||||||||||||||||||||||
| 44 | Proportion of trials where protocol and findings are published | y | y | ||||||||||||||||||||||||||||
| 45 | Description of institution's policy on health equity in research | y | |||||||||||||||||||||||||||||
| 46 | Proportion of projects which consider health equity in their design and conduct | y | y | y | |||||||||||||||||||||||||||
| Networks and dissemination | 47 | Number of collaborations on grant applications and projects | y | y | y | ||||||||||||||||||||||||||
| 48 | Levels of co-authorship | y | |||||||||||||||||||||||||||||
| 49 | Bilbiometric networks | y | |||||||||||||||||||||||||||||
| 50 | Total number of different collaborators across all projects | y | y | y | |||||||||||||||||||||||||||
| 51 | Description of range of collaborations | y | y | ||||||||||||||||||||||||||||
| 52 | Number of research projects engaging community partners | y | y | ||||||||||||||||||||||||||||
| 53 | Number of research projects engaging community partners for the entire duration of the project | y | y | ||||||||||||||||||||||||||||
| 54 | Number of articles co-authored with community partner | y | |||||||||||||||||||||||||||||
| 55 | Existence of specifically tailored material for different community groups | y | y | y | |||||||||||||||||||||||||||
| 56 | Size of communications office | y | |||||||||||||||||||||||||||||
| 57 | Number of staff engaged in outreach | y | y | ||||||||||||||||||||||||||||
| 58 | Number of people attending outreach events and their perceptions | y | y | ||||||||||||||||||||||||||||
| 59 | Level of participation in clinical trials | y | |||||||||||||||||||||||||||||
| 60 | Number of projects with an industry partner | y | y | ||||||||||||||||||||||||||||
| 61 | Industrial research funding for PhD/secondment positions in industry and PhD scholarships | y | y | ||||||||||||||||||||||||||||
| 62 | Number of policy secondments | y | y | ||||||||||||||||||||||||||||
| Policy impacts | 63 | Number of invitations from policy makers | y | ||||||||||||||||||||||||||||
| 64 | Number of citations on clinical guidelines | y | y | ||||||||||||||||||||||||||||
| 65 | Number of citations in policy documents | y | y | ||||||||||||||||||||||||||||
| Health impacts | 66 | Improved health of patients | y | y | y | ||||||||||||||||||||||||||
| 67 | Improved quality of care metrics | y | y | y | |||||||||||||||||||||||||||
| 68 | Number of lives touched | y | y | ||||||||||||||||||||||||||||
| 69 | Narrowing of health/health-care disparities | y | y | ||||||||||||||||||||||||||||
| 70 | Improved awareness of preventative measures in the community | y | |||||||||||||||||||||||||||||
| 71 | Number of treatment developed in house | y | |||||||||||||||||||||||||||||
| 72 | Number of new treatments available (adopted from elsewhere) | y | y | ||||||||||||||||||||||||||||
| 73 | Percentage, number, and range of types of clinicians on research projects | y | y | y | |||||||||||||||||||||||||||
| 74 | Number of uses of research infrastructure in clinical practice | y | y | ||||||||||||||||||||||||||||
| Economic impacts and commercialisation | 75 | Level of (local) spending | y | y | |||||||||||||||||||||||||||
| 76 | Amount of direct employment | y | y | ||||||||||||||||||||||||||||
| 77 | Number and type of new offices (including subsidiaries) in the area | y | y | ||||||||||||||||||||||||||||
| 78 | Size of tech transfer office | y | |||||||||||||||||||||||||||||
| 79 | Existence of Intellectual Property policy | y | |||||||||||||||||||||||||||||
| 80 | Number of patent applications | y | y | y | y | ||||||||||||||||||||||||||
| 81 | Number of patents awarded | y | y | y | |||||||||||||||||||||||||||
| 82 | Number of patent citations | y | |||||||||||||||||||||||||||||
| 83 | Number of licensing agreements and licensing revenue | y | |||||||||||||||||||||||||||||
| 84 | List/examples of know-how taken up by industry | y | y | ||||||||||||||||||||||||||||
| 85 | Number of private sector innovations/products/devices brought to market | y | |||||||||||||||||||||||||||||
| 86 | Number of spin outs | y | y | ||||||||||||||||||||||||||||
| 87 | VC money invested in startups | y | |||||||||||||||||||||||||||||
| 88 | Number and size of consultancy agreements | y | |||||||||||||||||||||||||||||
| 89 | Contract funding from industry | y | |||||||||||||||||||||||||||||
| 90 | Number of and list of new treatments | y | y | ||||||||||||||||||||||||||||
| 91 | Fraction of indirect costs covered | y | |||||||||||||||||||||||||||||
| 92 | Cost/benefit calculations | y | y | y | y | y | y | ||||||||||||||||||||||||
| Broader metrics | 93 | Perceptions of equity, quality, access | y | y | |||||||||||||||||||||||||||
| 94 | Perceptions of staff | y | y | y | y | ||||||||||||||||||||||||||
| 95 | Perceptions of community partners | y | |||||||||||||||||||||||||||||
| 96 | Perceptions of external experts | y | |||||||||||||||||||||||||||||
| 97 | Perceptions of people participating in research | y | |||||||||||||||||||||||||||||
| 98 | Attitudes of participants towards science and towards research | y | |||||||||||||||||||||||||||||
| 99 | Narratives of success | y | |||||||||||||||||||||||||||||
| 100 | Narratives of performance | y | |||||||||||||||||||||||||||||
| NOTE: This table does not represent an exhaustive list, but rather highlights the potential data sources that stand out as most appropriate for each metric. | |||||||||||||||||||||||||||||||