Establishing local priorities for a health research agenda

Abstract

Aim/Background

To describe the two‐stage prioritization process being used by the UK National Institute for Health Research's Collaboration for Leadership in Applied Health Research and Care for the South‐West Peninsula (or PenCLAHRC) – a joint health service and university partnership and reflect on implications for the wider context of priority setting in health‐care research.

Method

PenCLAHRC's process establishes the priorities of Stakeholders including service users across a regional health system for locally relevant health services research and implementation. Health research questions are collected from clinicians, academics and service users in Devon and Cornwall (UK) using a web‐based question formulation tool. There is a two‐stage prioritization process which uses explicit criteria and a wide Stakeholder group, including service users to identify important research questions relevant to the south‐west peninsula locality.

Results

To date, a wide variety of health research topics have been prioritized by the PenCLAHRC Stakeholders. The research agenda reflects the interests of academics, clinicians and service users in the local area. Potential challenges to implementation of the process include time constraints, variable quality of questions (including the language of research) and initiating and maintaining engagement in the process. Shared prioritization of local health research needs can be achieved between Stakeholders from a wide range of perspectives.

Conclusions

The processes developed have been successful and, with minor changes, will continue to be used during subsequent rounds of prioritization. Engagement of Stakeholders in establishing a research agenda encourages the most relevant health questions to be asked and may improve implementation of research findings and take up by service users.

Context

PenCLAHRC (the National Institute for Health Research's Collaboration for Leadership in Applied Health Research and Care for the South‐West Peninsula (NIHR CLAHRC SWP)) was established as a pilot on 1 October 2008 alongside eight other collaborations across the UK.¹, 1 CLAHRCs are partnerships between higher education institutions and National Health Service (NHS) organizations which aim to improve patient outcomes through the conduct and application of applied health research. They were established by England's National Institute for Health Research (NIHR) in response to Sir David Cooksey's Review of Health Research2 and the report of the Chief Medical Officer's High Level Group on Clinical Effectiveness.3 The former highlighted the ‘second gap in translation’ between research outputs and clinical practice. The latter suggested that the NHS should better utilize the capacity of higher education to support initiatives to enhance clinical effectiveness by developing new models of partnership to encourage relevant research and greater openness to change.4 The vision for CLAHRCs is to make the NHS more sensitive to the findings of research and make academia more responsive to the needs of the health service.

PenCLAHRC's overall objectives are driven by the principles of Evidence‐Based Practice (EBP).5 These are to support the identification of research questions that address clinical concerns; to support and undertake research that tests treatments, diagnostics and new ways of working in specific clinical areas; and where effective interventions are identified, to support research into how NHS staff can incorporate them into routine practice. By doing this, PenCLAHRC hopes to enable patients across the local community and beyond to receive a better standard of health care (http://clahrc-peninsula.nihr.ac.uk/).6

PenCLAHRC's distinctive features (in comparison with other CLAHRCs) are its belief in a more dynamic model of research prioritization and conduct (rather than linear – see Fig. 1), its engagement with services users (Boddy K, et al. 7) and its capacity building to enhance the involvement of all Stakeholders across the research cycle. By involving a wide group of Stakeholders (including service users) at all stages of the research process, from the inception of research ideas through to delivery of outputs, a portfolio of clinically relevant, locally tractable and patient‐informed projects can be established. Further, as the process itself helps to identify existing evidence needs and barriers to implementation, it is believed that the research findings are more likely to be taken up effectively. Widespread and genuine involvement of the local health community in the process may generate a significant cultural change in attitudes to research. PenCLAHRC sees such engagement as crucial to secure the ultimate aim of a shift in the way applied health research is conducted and valued such that it is readily sustainable beyond the original five years of the pilot. The extent to which the Peninsula and other CLAHRCs meet their objectives is the subject of research originally sponsored by the NHS Service Delivery and Organization programme8 which has now been subsumed into the Health Services and Delivery Research programme (HS&DR http://www.netscc.ac.uk/hsdr/).

Figure 1.

The role of PenCLAHRC in a dynamic approach to research and research prioritization.

Research prioritization is necessary when the resources required to answer the questions raised outweigh the resources that are available.9 Prioritization can also be used to identify research questions with the potential for greater impact on patient care and outcomes, particularly in the context of localized priority setting. In this article, we describe the development of the PenCLAHRC prioritization process and how it has changed over the five rounds completed to date and reflect on the implications it may have for a wider (national) context of priority setting in health‐care research. Other authors have reported on processes of priority setting in health care on a national scale; 9, 10 this article reflects how processes used on a national level can also be used at the local level.

Question generation

Question identification began during the development of the bid to establish the CLAHRC with a simple letter emailed to NHS Stakeholder organizations seeking questions regarding everyday practice. Over 100 responses were obtained via email of varying detail and structure. In many cases, respondents described an issue, but no specific question was identifiable; effective prioritization of questions in these cases was not possible. It became apparent that a structured format was necessary to capture clinical uncertainties in the form of a question with sufficient supporting information to facilitate prioritization. The following strategies were developed to aid this process:

• Web‐based tool

• Locality Leads

• Workshops

The web tool was incorporated into the PenCLAHRC website (http://clahrc-peninsula.nihr.ac.uk/submit-question.php)6 to help question originators develop a clear and focused question. The web tool provides two structures for questions: a PICO (Population, Intervention, Comparator, Outcomes) structure, which is appropriate for questions about therapies or diagnostic tools, and a structure for questions concerned with the implementation of change in the NHS related to practices for which good evidence already exists (e.g. prevention of venous thromboembolism).12

Locality Leads (part‐time secondees to PenCLAHRC at consultant level, drawn from acute and primary care trusts) and the Public and Patient Involvement (PPI) Lead, encourage the submission of research questions to PenCLAHRC through workshops and meetings with clinicians and service users. Some of these workshops are specific Evidence‐Based Practice (EBP) training events designed to help clinicians and service users understand a common language of research in order to be able to develop their own research questions in the future. Since January 2010, over 2900 people (including 277 service users) have attended these meetings and workshops across Devon and Cornwall.

Prioritization stage 1 – Executive Group prioritization

The Executive Group consists of Peninsula CLAHRC's director and deputies as well as a number of other academic and clinical positions (see Table 1).

Table 1.

Members of the executive group

Role	Number	Locality or theme
Locality lead (clinical)	2	Cornwall
	2	Plymouth (with south and west Devon)
	2	Exeter (with north and east Devon)
Theme lead (academic)	1	Environment and Human Health
	1	Mental Health and Neurology
	1	Public and Patient Involvement
	1	Primary care and Improvement
	1	Ageing and Development and Implementation

The purpose of the prioritization meeting is to refine the scope of the question and to prioritize the submitted questions using criteria which aim to highlight the most locally relevant and feasible questions:

• Size of the health problem which may be addressed by the research.

• Local health‐care resource impact of the problem.

• Potential for health gain from change contingent on carrying out research.

• Feasibility of answering the question.

• Appropriateness of the local environment to address the question (e.g. local demography).

These criteria reflect the common categories of prioritization criteria identified in a review conducted by Noorani and colleagues13 which looked at practical approaches to priority setting in health care. To date (during five rounds over 2 years), over 290 questions have been considered at Stage 1 of prioritization. Members receive a pack detailing all the questions in advance of the meeting to ensure that the discussion during the meeting is informed and the meeting productive. All topics are introduced by the Chair (one of the PenCLAHRC deputy directors) who summarizes the question and states where particular adherence (or otherwise) to PenCLAHRC priorities is met. This instigates further discussion around local health priorities and prevalence of health problems locally which requires input from each of the 10–15 group members. The meetings last approximately four hours. Discussion continues until all members of the group feel they have sufficient information to rank the questions. A proportion of the highest ranking questions are selected to be taken forward to the next stage of prioritization. The number is limited, by the agreed timescale and staff capacity, to no more than 12 topics for each Stakeholder meeting. A wide range of questions have been submitted, a selection of which can be seen in Appendix A. Fig. 2 details the total number of questions to have come through the PenCLAHRC process from January 2009 to June 2012 (five rounds of prioritization) (Fig. 2).

Figure 2.

Total number of questions through the prioritization process (August 2009–June 2012).

Prioritization stage 2 – Stakeholder group prioritization

PenCLAHRC is comprised of the following major Stakeholders:

Higher education institutions

• University of Exeter

• Plymouth University

Strategic Health Authority

• NHS South West

Primary Care NHS Trusts

• NHS Cornwall and Isles of Scilly

• NHS Devon

• NHS Plymouth

NHS Foundation Trusts

• South Devon Healthcare NHS Foundation Trust

• Royal Devon & Exeter Healthcare Foundation Trust

• Cornwall Partnership NHS Foundation Trust

• South Western Ambulance Service NHS Foundation Trust

Acute NHS Trusts

• Northern Devon Healthcare NHS Trust

• Plymouth Hospitals NHS Trust

• Royal Cornwall Hospitals NHS Trust

Mental Health Trusts

• Devon Partnership NHS Trust

Primary Care Trusts

• Torbay and Southern Devon Health and Care NHS Trust

Each of the Stakeholder organizations and service users are invited to contribute to the prioritization process at a face‐to‐face meeting held at a central location in the region and led by the same Chair. The aim is to discuss each topic in more detail, supported by additional information; focus questions in more depth using the expertise of the Stakeholder group; and generate a ranked list of research ideas for further development within PenCLAHRC.

In preparation for this meeting, the evidence synthesis team (which consists of three researchers and two information specialists) spend 2–3 days preparing each Priority Briefing for the topics prioritized from Stage 1 to supplement and extend the information initially provided by the question submitter. In each case, the question originator is consulted to clarify and develop the question. A brief literature review is performed to identify pertinent guidelines, published and on‐going research, information to assist in assessing the size and impact of the issue, relevance to local health objectives and the potential benefits of resolution of uncertainty. Priority Briefings are not designed to be systematic reviews of the literature.

Service users are also represented at the Stakeholder meeting by two volunteers from PenCLAHRC's patient involvement group (PenPIG). The group meets prior to the event to discuss the Priority Briefings. PenPIG is a diverse group of service users consisting of approximately 14 people who discuss each question and agree on their votes for the meeting. The service user representatives are able to alter their votes if appropriate following the discussion at the Stakeholder meeting. This process is unique to this particular group of Stakeholders (although other Stakeholders may consult colleagues informally). PenPIG also provides useful feedback on the accessibility of the Priority Briefings in terms of language and structure.

The Stakeholder meetings have been well attended with representatives from at least 13 of the potential 16 partner organizations in the collaboration plus two patient representatives, 15–20 voting participants in total (with dedicated support from PenCLAHRC staff) at each meeting. Each question is introduced by a member of the group (with no specialist knowledge of the topic area and is specifically chosen by the Chair in order to reduce any bias that prior knowledge might bring to the introduction of the question). No attempt is made to reach a consensus on the priority of each topic as the meeting progresses. However, the focus of questions may be altered by consensus if necessary. This meeting lasts about five hours. At the end of the discussion, the Stakeholders vote for the topics they believe most worthy of further PenCLAHRC involvement according to the prioritization criteria. The voting process is further described below.

Prioritization method

The voting method used by PenCLAHRC in the Executive and Stakeholder prioritization meetings is based on Rank Order Voting. Participants are each allocated a quota of individual votes (10 in the Executive meeting and six in the Stakeholder meeting), of these votes half are worth one point and half are worth two points. All Stakeholders' votes hold an equal weighting. Voting is anonymous. This system allows for individual priority setting (ranking) to be achieved without the group having to reach an overall consensus on each topic. Rank Order Voting was considered to be the best format on which to base prioritization for the PenCLAHRC process as it enables a number of topics to be selected in a fair and practical manner. The reasons why this particular method was chosen and how it compares to other research prioritization processes are discussed further in the reflections section.

Questions and topics

Since the first round of PenCLAHRC's prioritization process in 2009, questions have been submitted from a range of health areas. Figure 3 shows what percentage of the questions submitted to the process and what percentage of questions supported by PenCLAHRC (over the first four rounds of prioritization) fall into each International Statistical Classification of Disease and Related Health Problems (ICD 10) with a separate category for Service Delivery questions in the final column (SD). From this data, we can see that the range of questions supported by PenCLAHRC is narrower than the range of questions submitted to the process. This might be expected due to the small proportion of questions that PenCLAHRC are able to support. For example, the proportion of questions submitted in the ICD10 V classification (Mental and Behavioural Disorders) is fairly balanced with the proportion that is represented in the questions supported by PenCLAHRC. The imbalances work in both directions and may reflect the local relevance and priority of those health areas, perceived by the PenCLAHRC Stakeholders.

Figure 3.

Proportion of questions submitted to – and supported by‐ PenCLAHRC by ICD 10 classification. ICD 10 Classification Key, I – Infectious and parasitic diseases, II – Neoplasms, III – Diseases of the blood and blood forming organs and certain disorders of immune mechanism, IV – Endocrine, nutritional and metabolic diseases, V – Mental and Behavioural disorders, VI – Diseases of the nervous system, VII – Diseases of the eye and adnexa, VIII – Diseases of the ear and mastoid process, IX – Diseases of the circulatory system, X – Diseases of the respiratory system, XI ‐ Diseases of the digestive system, XII – Diseases of the skin and subcutaneous tissue, XIII – Diseases of the musculoskeletal system and connective tissue, XIV – Diseases of the genitourinary system, XV – Pregnancy, childbirth and the puerperium, XVI – Certain conditions originating in the perinatal period, XVII – Congenital malformations and chromosomal abnormalities, XVIII – Symptoms, signs and abnormal clinical laboratory findings, not elsewhere classified, XIX – Injury, poisoning and certain other consequences of external causes, XX – External causes of morbidity and mortality, XXI – Factors influencing health status and contact with health services, SD – Service design/organizational topics.

The types of questions submitted to – and supported by PenCLAHRC also varied. We identified four questions types: development – questions that require the development of an intervention or tool to be tested, intervention – questions about the effectiveness or cost‐effectiveness of an intervention that has already been developed, implementation questions regarding barriers and facilitators to interventions being implemented when there is already clear evidence on effectiveness, and modelling – questions regarding the arrangements of NHS services which can be addressed using operational research methods (for which PenCLAHRC has established a small research team in collaboration with the NHS, http://clahrc-peninsula.nihr.ac.uk/penchord.php). Figure 4 shows the percentage of the types of questions submitted to PenCLAHRC, and Fig. 5 shows the percentage of the types of questions supported by PenCLAHRC. The pie charts show that the proportion of development, intervention and modelling questions submitted to and supported by PenCLAHRC are relatively balanced.

Figure 4.

Proportion of questions submitted to PenCLAHRC by question type – and supported by‐ PenCLAHRC by question type.

Figure 5.

Proportion of questions supported by PenCLAHRC by question type.

Table 2 lists some examples of the topics that have been prioritized and how they have been taken forward by PenCLAHRC.

Table 2.

Example projects resulting from Stakeholder prioritization

Topic	Project/Action
How can we get the most benefit from thrombolysis for acute ischaemic stroke in the south‐west peninsula	A modelling study to examine different configurations of stroke services to inform service changes.
Does the Health Compass programme promote healthy diet and physical activity in people with high cardiovascular risk	A protocol will be developed for funding for a multicentre cluster randomized controlled trial to examine the effect of a locally developed intervention
Is the Action for Rehabilitation from Neurological Injury Training technique effective in increasing functionality, improving activity and improving autonomy in people who are between 1 and 5 years after their stroke	A protocol will be developed for a trial of long‐term rehabilitation
Would a parent delivered programme of training for paediatric ward staff improve the effectiveness of communication with disabled children and their parents when they are admitted to hospital	An intervention will be developed and evaluated with service users
What school level interventions would improve social and emotional behaviour among primary school children (specifically two programmes)	A systematic review will be conducted on teacher‐oriented interventions to improve social and emotional well being in primary school children
Would a mental health/developmental paediatric assessment of primary school children who are excluded from school lead to evidence‐based management of underlying problems and hence improve the likelihood of a positive long‐term outcome?	A systematic review will be conducted to establish the prevalence of mental health problems in school‐aged children and how children with problems are dealt with within the school environment
Does a simple pre‐clinic form improve patient satisfaction following a diabetes appointment?	An intervention will be developed and evaluated with service users
How can patient‐initiated clinics (PIC) be implemented for rheumatoid arthritis given the demonstration of the acceptability, effectiveness and cost‐effectiveness of such service organization?	A systematic review will be conducted on the acceptability, effectiveness and cost‐effectiveness of PICs across disease areas (including rheumatoid arthritis
What are the benefits and costs of providing peer support to parents of children with disability?	A systematic review and evaluation of a pilot scheme on the topic will be conducted
What is the value of respite care for unpaid carers of people with dementia? (If there is some value, what forms of respite care are most beneficial?)	This will be developed into a full research proposal and funding will be sought
In some situations, are telephone consultations a safe and effective method of providing routine follow‐up in secondary care?	A systematic review on this topic will be conducted

Reflections

PenCLAHRC is just one example of how mixed groups of Stakeholders can be encouraged to develop relevant health‐care priorities. A review by Stewart and colleagues14 found many reports of engagement of service users and/or clinicians in developing health‐related research questions. The James Lind Alliance (JLA) also facilitates collaboration between patients, carers and clinicians through priority setting partnerships. The JLA, much like PenCLAHRC, aims to initiate health‐care research which has more relevance and benefit to the service users and clinicians.15 However, JLA uses different methods of prioritization, concentrating on one health area at a time and specifically excluding academics from the initial prioritization processes aims for national coverage of clinicians and service users. Engaging with service users in priority setting is in line with the emphasis being placed on the role for service users in the on‐going identification, development and implementation of health research (http://www.involve.org.uk/).

It is interesting to note that although those carrying out the prioritization in PenCLAHRC came from a wide range of backgrounds, there appears to be considerable convergence of priorities across groups, particularly for the highest ranked questions which were supported by most constituents. This can also be seen in other Stakeholder group settings16 and provides support for partnership working. However, topics (particularly implementation questions) that do not have the support of the relevant NHS Trust cannot be expected to go far without a successful business case.

From the outset, a number of challenges were recognized that might influence the success of the PenCLAHRC structure and aims:

• Quality of questions – acquiring the resources and skills across the collaboration to develop the language of research and to develop specific research questions as opposed to general identification of issues.

• Time constraints – the timescale for delivery expected by Stakeholders, particularly given the need to establish a process and resource it with appropriate staff.

• Engagement – potential for bias within the process towards known contacts within the local health community, leading to disenfranchisement within the collaboration. The wide range of Stakeholders to be included, in terms of different organizations, professional groups and geographic areas. The process itself raising expectations amongst the clinical community which could not be met.

As PenCLAHRC has developed, there has been growing clarity about these challenges. Feedback on the prioritization process (gathered by an evaluation form presented at the Executive and Stakeholder meetings – Appendix B) has informed a number of thoughts and decisions which are discussed below.

Quality of questions

Concern was raised regarding the varying detail that was provided in question submissions for the Executive stage of prioritization. It was felt that this variation may provoke bias in the selection of questions during the rapid prioritization of topics by the Executive Group as those with more information may be favoured, disregarding the question itself. In response to this risk, the development of the web tool and the delivery of EBP workshops have helped to standardize the detail provided at the question generation stage. However, questions with little initial information in their submission have been prioritized in the past.

Interestingly, use of the web tool highlighted difficulties in usefully defining the difference between the question types described earlier; development, intervention and implementation (language of research). There are instances of questions being categorized as intervention questions when in reality they cover implementation. This may be resolved through further development of the EBP workshops and will improve over time as the language of EBP develops and becomes more widespread. These workshops will also improve the skills and resources available to produce more focussed research questions.

Another potential criticism of the PenCLAHRC process is that topics lost at the first stage of prioritization do not benefit from the information gathering that happens for the Priority Briefing. This means there is a possibility that topics discarded at the first stage of prioritization might have been preferred over those that progressed if they had been subject to more lengthy and careful information support. To reduce the potential for this to happen, question originators are advised to contact their Locality Lead for support and to submit the question in alliance with them. Furthermore, originators of discarded questions are sent feedback letters to encourage them to develop their question further or submit their question to other research organizations that may also be interested in the topic, for example, NIHR Health Technology Assessment (HTA) programme, the Database of Uncertainties about the Effects of Treatments (DUETS), and the local NHS Research Design Service (RDS).

Time constraints

Two concerns regarding the meeting organization were the time taken during the Executive Group meeting to discuss each topic and the use of video conferencing. Whilst many of the group reported the amount of discussion and information available to be adequate, there were concerns that tiredness was affecting concentration and therefore influencing the voting on topics discussed later in the meeting. To date, there has been no evidence that this has been a problem. The initial use of video conferencing during the Executive meeting was awkward, and both prioritization meetings (Executive and Stakeholder) are now held face‐to‐face. This also seems to help improve engagement in the meetings from all Stakeholders.

Engagement

Geographically, the areas of Torbay and North Devon have been under‐represented in the questions submitted to the PenCLAHRC process to date. This highlights the problem of bias towards known contacts in the local health community. Difficulties in engaging with these areas may also be attributed to the travelling distances and time involved and to a lack of personal contacts. This is also reflected in the clinical specialities that have been covered. Question generation activity is recorded and analysed on a regular basis to identify gaps in engagement.

It has been a particular challenge to engage clinicians and academics alike in the initial PenCLAHRC process once they have realized that no direct funding is available. However, disengagement at the Executive and Stakeholder meetings (resulting in a lack of or inappropriate representation of Stakeholder groups) was rare. This occurred once, and the importance of engagement from all Stakeholders was emphasized such that it did not happen again. PenCLAHRC is able to provide support for research projects irrespective of whether the question originator has the time, resources or knowledge to lead on that project. If a project is supported by PenCLAHRC, a team of relevant researchers, academics, clinicians and service users is brought together by a designated project facilitator to take the project forward (including applying for external funding).

Voting method

The voting method used by PenCLAHRC in the Executive and Stakeholder prioritization meetings is based on Rank Order Voting. This was considered to be the best format on which to base prioritization for the PenCLAHRC process as voting is speedy, accurate and anonymous following a shared, informative discussion. Several other voting systems/methods could have been available to prioritize research suggestions, such as Nominal Group Process,17 the Hanlon Process of Prioritization,18 Majority Rule19 or Rank Order Voting (developed by Borda and referred to as the Borda Count).19 Reasons for these methods were not taken up are described below.

Nominal Group Process provides a structured group discussion whereby input from all participants is balanced and makes the best use of experience and expertise without the risk of dominant individuals skewing the process or results. Ideas can be generated, problems solved and prioritizing decisions proposed in a face‐to‐face non‐threatening environment.18 The Hanlon Process of Prioritization encourages members of the group to work together to ensure all participants have consistency in language and understanding. This is to help when discussing the variety of issues/programmes on the prioritization agenda.18 Majority Rule works by taking into account the runner‐up places of projects. Participants vote independently for their first, second and third project preference. For example, if we look at the results of a vote in Table 3, in the Plurality Rule method, the project who received the most votes as the most preferable project will win (i.e. candidate A), all other information/votes are discarded. In the Majority Rule process, however, the winner is the project that would beat each opponent in a pairwise comparison. So from Table 3, we can see that 64% of the voters prefer project B to project A and 66% prefer B to C. Hence, project B would win and A would be second with 66% preferring A to C and 36% preferring A to B, and C would come in last with only 34% preferring C to A or B.18 These methods of prioritization were not chosen due to time restraints and the need to rank the topics rather than to seek one clear ‘winner’.

Table 3.

Example of a candidate vote

Candidate/vote	30%	36%	34%
1	B	A	C
2	A	B	B
3	C	C	A

A number of other research organizations use similar methods of prioritization (and identification). Although now undergoing some change, the NIHR HTA programme consults with Stakeholders in the NHS and external organizations, encourages submissions from researchers and scans the medical literature to identify research questions. In the process up to 2013 HTA prepared briefings for those questions that made it through the first round of prioritization, where each question is considered in relation to the research programme criteria. The briefings were then considered by expert panels in the second round of prioritization. The Medical Research Council (MRC)20 consults with a wide range of Stakeholders including Government departments, the public sector, industry, charities and the general public. The NIHR SDO programme (now HS&DR http://www.netscc.ac.uk/hsdr/) engages a wide range of Stakeholders and several levels of prioritization to identify relevant areas of research. Other research funders in the United States such as the National Institute of Health (NIH) and the Centre for Disease Control and Prevention (CDC) also use a collaborative reviewing format, although the inclusion of service users is unclear and voting methods vary.

The future

PenCLAHRC has developed into a team of over 50 people, with wide ranging skills including statisticians, decision modellers, systematic reviewers, information scientists and other methodologists. Around 25 research projects have been established through the prioritization process and are either underway or the subject of major funding bids. So far two large funding bids have been successful total grants have amounted in over £3 million for PenCLAHRC projects.

PenCLAHRC will continue to run cycles of question generation and prioritization, incorporating appropriate changes and improvements according to internal evaluation. In addition, PenCLAHRC has developed more intensive EBP courses to help clinicians and service users in Devon and Cornwall improve their understanding of research, development of research questions and translation of research into practice. Such courses will empower attendees to construct and submit quality research questions to answer service and practice‐based queries. This will help build capacity for implementation and implementation research as advocated in the Chief Medical Officer's High Level Group on Clinical Effectiveness3 as part of a more sustainable approach to research.4 It is believed that engagement of Stakeholders in establishing a research agenda increases relevance and may enhance the implementation of research findings.20

Source of funding

This systematic review was funded by the National Institute of Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) for the South West. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Conflicts of interest

Authors Whear, Thompson‐Coon, Boddy, Papworth and Stein have continuing professional involvement in PenCLAHRC and are funded through PenCLAHRC.

Appendix A.

Examples of submitted questions

Question

Does Vitamin D supplementation in pregnancy increase Vitamin D levels in newborns?

Is the prospect of bio‐identical bio‐mimetic oestrogen (E2) and progesterone hormone restoration therapy safe, effective and quality of life giving for women as they age?

Can a high‐impact intergenerational volunteering programme in an English setting provide the same range of health and social benefits the US programme has delivered?

In the absence of language assessment for young bilingual toddlers born in the UK, can we design a diagnostic tool to help health professionals (SLTs and Health visitors) identify children who are at risk for language impairment to implement an intervention plan?

Does laparoscopy improve diagnosis and treatment decision making in suspected ovarian cancer?

Can taking part in outdoor walking groups, within natural surroundings, have a positive affect on those with mild‐moderate depression and anxiety disorders?

Can effective opportunistic screening of skin cancer, particularly malignant melanoma, be carried out by non‐medically trained personnel?

How can microalbuminuria screening in patients with diabetes be dictated by clinical need rather than financial inducements of the Quality Outcomes Framework (QOF)?

Can screening intervals for diabetic digital retinal photography be individualized, to enhance screening performance and cost‐effectiveness, on the basis of individual clinical risk?

What is the prevalence of Hepatitis C in IV drug users in Cornwall? Can we implement a Hepatitis C screening programme that will address the current health inequalities for these service users?

Would the use of phosphodiesterase‐5 inhibitors (PDE5i) in the treatment of erectile dysfunction (ED) following rectal surgery?

Would an enhanced support service, consisting of antenatal contact and additional postnatal visits/contact from a midwifery support worker increase breast‐feeding initiation and prevalence rates in a deprived area of Cornwall?

In pregnancies complicated by diabetes can antenatal harvesting of colostrum be used in the treatment of neonatal hypoglycaemia and improve breast‐feeding initiation and continuation rates?

How could we implement a structured training package delivered by a multidisciplinary team to people with Type 2 diabetes within 3 months of being identified as high risk of foot ulcers, with the aim of reducing the presentations of foot complications?

Can mortality and morbidity be reduced in hospitals during heat waves through a system of distributed alerts and guidance?

Would an educational leaflet provided alongside a prescription for: Angiotensin 2 receptor blockers, Angiotensin‐converting enzyme inhibiters or diuretics, explaining the importance of suspending these medications when you are unwell, for example, with a diarrhoeal illness, reduce hospitalization?

Appendix B.