Can research development bursaries for patient and public involvement have a positive impact on grant applications? A UK‐based, small‐scale service evaluation

Abstract

Background

Increasingly, research grant awarding bodies are regarding involvement at all stages of research, including prior to funding, as good practice. However, it is unclear how researchers should pay for this. Therefore, a pre‐funding bursary scheme was designed. Up to £500 could be requested for involvement to develop a grant application for which user involvement is a key requisite for the funding body concerned. As the bursary scheme had run for 2 years, an evaluation was conducted to ascertain whether the scheme was effective for incorporating early involvement and in developing the grant proposal.

Results

Twelve applications were made of which all were funded. The mean amount requested was £432.91; with the mean amount awarded £308.72. The involvement activities conducted all used qualitative methodology. Feedback regarding the bursaries was positive: enabling refinement of the research question and design; developing dialogue between the service users and the researchers; and helping with team building, with service users sometimes becoming co‐applicants or members of the steering groups. The bursaries provided a learning opportunity – about involvement for the researchers and about research for the service users. The ultimate aim of the scheme was to enhance the research grant. Regarding this, the involvement paid for by the bursary meant that applicants could complete the involvement sections with in‐depth information and clarity.

Conclusion

For a relatively small financial outlay, appropriate involvement was made possible at an important part of the research process which is usually neglected due to lack of funding. Recommendations for implementation made.

Background

The concept of patient and public involvement (PPI) has been evolving since the World Health Organization first mentioned it in 1978.1 Since then, PPI has become integral in health‐care research in many countries including in the UK.2 PPI is defined as a partnership between researchers, whether they are clinicians or academics, and those affected by the research.3 The rationale for PPI draws on the argument that it has beneficial effects for the quality of research and promotes more reliable, relevant research.4 As the National Institute for Health Research states, ‘Involving patients and members of the public in research can lead to better research, clearer outcomes and faster uptake of new evidence’.5 Whilst the Chief Medical Officer and Director General of Research and Development in the Department of Health, Professor Dame Sally Davies, asserts that ‘Increasingly, research that involves patients and the public in the design and evaluation stages is carrying greater weight and greater significance….’ (pg 4).6 Therefore, appropriate PPI is believed to increase the quality and acceptability of the research.7, 8, 9 Although these ethoi sound plausible, there is a problem with a lack of supporting evidence that validate them.6, 10, 11, 12 Therefore, calls for PPI to be robustly evaluated and measured for impact are increasingly being made. e.g^. 6, 13, 14, 15, 16

Although there are hundreds of examples of the impact of PPI being discussed in various academic and clinical papers, only a few papers have attempted to define it. A useful definition was put forward by Staley6 in which she referred to impact as meaning the difference involvement makes upon the research, whether positive or negative, and regardless of the stage of the research or from whose perspective: researcher's, public's, patient's or community's. In her structured literature review, Staley explored the different types of impact that were noted. She concluded that it was ‘…difficult to assess the impact of involvement or to predict where involvement would have the greatest impact’ and that ‘more work is needed to clarify the added value of involvement in different research contexts’ (pg 9).6 Although involvement processes and methods continue to evolve, this lack of clarity regarding impact remains. Later in 2011, Brett and team13 conducted a systematic review to explore the issues around measuring the impact service user involvement has upon research. It found an inconsistency in how impact was understood or measured and concluded that ‘There is a clear need to develop a much more consistent and robust base by enhancing the quality of reporting to enable impact to be fully identified and evaluated’ (pg 114). However with the emergence of guidelines which attempts to standardize the reporting of involvement, so that impact is more transparent, for example the Guidance for Reporting Involvement of Patients and Public (GRIPP) checklist,15 a clearer understanding of what type of PPI works, for whom, why this is and in what circumstances, will surely emerge.

Despite the still debatable impact of PPI however, here in the UK there are focused policies and systems instigated by the Department of Health designed to actively encourage the involvement of people within research. Postal surveys and interviews with UK funders of health research carried out in 2004 asked whether commissioners promoted PPI, and if so, why and how they did so. Several funders agreed that involving people was a matter of policy and that involvement was important as people are sources of information that commissioners need to know about, such as how it feels to live with a condition, whether research questions are relevant to them, or whether they felt that the proposed methods were suitable for participants, therefore resulting in more appropriate and empirical studies. Funders said that they also assessed the PPI strategy in the grant applications that they received, such as whether the proposed involvement, including the number of service users and their role descriptions, any training or mentorship offered and budgets for involvement, seemed appropriate. The study recommended that funders should be clear on what their expectations for PPI are within their research criteria so that applicants are aware of these views.17

Increasingly, research grant awarding bodies such as the National Institute for Health Research (NIHR) in the UK are regarding involvement at all stages of research including prior to award application as good practice.5 However, although it is regarded as good practice, as it is prior to award, it is unclear how researchers should pay for this involvement, as it is recommended that as a minimum, all out of pocket expenses should be paid for thereby acknowledging the value of the patient/public's contribution. Payment also equalizes the equity of power and reduces the barriers preventing people to become involved.3

To ensure appropriate involvement in research, the NIHR funds two units, INVOLVE which promotes PPI in general and The Research Design Services (RDSs) who have the broader remit of supporting researchers in their quest to obtain a research grant, therefore work with researchers early on in the process, often pre‐protocol, where involvement is imperative for good research design. There are ten RDSs covering England each one having a designated person for PPI. Based upon the fact that many grant awarding bodies approve of involvement prior to application, many RDSs offer small awards to support service user engagement where their involvement will contribute to the development of a high‐quality funding application, and so, hypothetically, enhancing the probability of success. These awards pay for any legitimate expense associated with facilitating involvement in the research bid development such as follows:

1. Service user's time in preparing for, and attending, meetings (please note some RDS's may not pay for service user's time).

2. Out of pocket expenses

3. Hiring a venue in which to hold a PPI event such as a focus group

4. Providing refreshments

5. Associated childcare (or other caring responsibilities) incurred

Method

The RDS for the East Midlands (RDS‐EM) where the authors are based has run its PPI in Research Development Award (PPIRDA) since 2010, so a small‐scale evaluation exploring the impact of these awards was conducted within the service.

Up to six bursaries can be awarded per financial year (April–March), and are funded on a first‐come, first‐serve basis. Up to £500 per application can be requested and only one PPIRDA per project is allowed. A breakdown of the budget is required on the application form, and items approved may include payment to the service users for their time and out of pocket expenses, hiring a venue and catering. Applications are reviewed internally by two RDS staff members. For the application to be successful, the criteria used for approval is that (i) the involvement will develop the ultimate grant application and that user involvement is a key requisite for the funding body concerned; (ii) the bursary pays for good practice regarding involvement; (iii) evaluation of the proposed involvement will be obtained; and (iv) involvement will continue throughout the ultimate project if research funding is secured.

Feedback was sought from all researchers via email who had completed the involvement and who had invoiced for the final amount of the bursary money. Upon being granted one of the bursaries, the applicants are given a third of the money so that preparation for the involvement can take place. To gain the final amount, invoices, together with receipts, need to be sent.

Participants were informed that an article would be drafted regarding the efficacy of the awards. Although they were assured anonymity in any dissemination of the results, as they were returning their feedback by email, it was not anonymous and therefore is a limitation of this evaluation. Ethical approval was not required as this was not a research study but a small‐scale evaluation of a service.

Participants were asked whether they felt that the award they received:

1. Developed or changed their research proposal (either positively or negatively)

2. Developed the research team (either as a group or individually)

3. Developed any of the service users they worked with

They were also asked whether they had gathered any feedback from the service users regarding their involvement and whether any of the service users had remained being involved post‐bursary. As the bursary was designed to support the development of a larger, research grant application, they were also asked if this grant application had been submitted, and in what way, if any, the bursary had supported this. There was also space for any other comments.

Due to a semi‐structured survey being used to collect the feedback, and because specific answers needed to be obtained, that is, whether the PPRIDA was achieving its aims or not, a basic framework approach18 to analyse the data was used. Using this approach also ensured that the results would be valuable and informative to policy makers and research organizations.

Results

A breakdown of all of the applications and their outcomes for the two years the scheme has been running can be seen in Table 1. As can be seen, all applications were funded. This is due to the applicants being clients of the RDS and so have received support on the application prior to submission. The discrepancy between the amount requested and the amount awarded is due to either an item being in their budget which the bursary does not cover, for example transcription, or a cheaper alternative being suggested to them, such as the offer of a free room in which to conduct their involvement activity, rather than a room which has a rental fee.

Table 1.

PPIRDA applications and outcomes

Financial year	Number of applications	Mean amount requested	Number funded	Mean amount awarded	Number withdrew
2010–2011	5	£465.82	5	£403.15	0
2011–2012	7	£400.00	7	£214.29	1

Feedback was given by 5 bursary recipients. The involvement activities in all bursaries used qualitative methods including focus groups, interviews and workshops. The questions that were asked during the involvement activities centred around the proposed research question, design and the priority of outcomes, as well as the proposed recruitment strategy. With regard to involvement helping to hone the research design, there was unanimous agreement that the PPIRDA helped with this process:

The research team found the contribution of maternity service users extremely valuable in helping to shape the research ideas. (A1)

It was also deemed useful in ascertaining the research priorities of service users:

The [PPIRDA] developed our research proposal by pointing out research areas that were underrepresented. (A2)

The bursary was reported to be extremely useful in developing dialogue between the service users and the researchers, and from the groups of service users involved in the PPIRDA work, recruitment for the prospective, ultimate research project was secured by all bursary recipients, such as service users being willing to become members of the steering group, or offering to be a co‐applicant, and so the PPIRDA also helped with the research team building:

[The PPIRDA] helped form the research team with integrated service users for continuous involvement if successful in securing a grant…. (A2)

The lead for the PPI group and some the other members are keen to be further involved if the project gets funded and be part of the steering group. (A3)

The award was especially useful to create a permanent PPI panel willing to participate in research. (A4)

There was also positive feedback from the service users who were involved as a result of the bursary. The researchers who gained feedback from them reported that the service users enjoyed the experience, and found it informative:

In general they all enjoyed the day and learnt more about the research we are already conducting in this research unit. (A2)

They valued the opportunity to contribute to what they considered to be an important research agenda and felt that their views and experiences were taken account of. (A3)

There was also a capacity‐building element apparent from the feedback, in that the bursary enabled the service users to learn about research, and access mentorship and training, a process which was appreciated by them:

[The service user] wants it to be known that his prior involvement with the group was an important trigger in taking up his postgraduate studies. (A4)

All of [the service users] were willing to further participate in various tasks and receive training to improve their research skills. (A2)

The bursary also enabled the research team to learn about PPI, including how to secure involvement, how to work with service users and what management strategies needs to be in place, such as mentorship, peer support, expense procedures, etc., and this ensured appropriate and enduring involvement, and allowed researchers to learn about this process prior to the ultimate grant being awarded, therefore allowing for a seamless transition into the main body of research work:

The [PPIRDA] was useful to strengthen the PPI strategy of the research unit and to clarify the role of the PPI manager proposed for the next awarding period. (A2)

[the PPIRDA brought] awareness to the whole research unit about useful and effective ways to involve patients in research for providing a different perspective, or to assist in the development of policies and information materials that have to meet patients' needs. (A4)

The definitive aim of the award is to enable involvement which will enhance the odds of gaining the ultimate research grant. As can be seen in Table 2, all of the 5 bursaries for which we received feedback supported the ultimate grant application. Three of these knew the outcome with 2 being successful and one not. Feedback from researchers regarding this definitive aim was positive:

Table 2.

PPIRDA and data regarding the ultimate grant application

Number of awards paid in total: 2010–2012	Ultimate grant applications submitted	Ultimate grant application successful/unsuccessful	Application outcome outstanding
5	5	2:1	2

I could complete the PPI sections on the [grant] application form comprehensively and with much more information and clarity. (A3)

The PPI group offered, and sent, a support letter towards the grant application. (A3)

Further support for the efficacy of the PPIRDA was evident from the dissemination regarding the involvement process itself that many of the researchers conducted. As PPI is a relatively new requirement in health services research, models of best practice and evaluation are often reported:

The group was chosen to exhibit a poster presentation at last year's INVOLVE conference in Nottingham [UK], which generated interest from other [NHS] Trusts around the country on how our group was formed and maintained. A paper has been submitted for publication on the group's experience on working with the [NHS] Trust. (A4)

Conclusion

Evaluation is on‐going regarding the efficacy of these awards. However, feedback obtained suggests that for a relatively small financial outlay, good public relations and dialogue have been achieved between the RDS and our researchers. The awards have also helped capacity build in terms of educating researchers regarding involvement, service users regarding the research process and to a wider audience via dissemination strategies regarding the involvement itself. Although only 3 of the applicants knew the outcome of the ultimate grant, two of three were successful. Although due to this small number, no firm conclusions about the bursaries' efficacy can be made, it indicates promise, as usually the ratio for a competitive national grant is more like a 1 in 5 success rate.19 Fundamentally, these awards have ensured that the ultimate research design is democratic, ethical and one which promotes service user empowerment.

It is a useful award which supports work with service users. This, in turn, helps in developing the team, the research proposal and links with local and relevant service user groups. (A3)

Recommendations for implementation

• 1Although this service evaluation is UK based, this model of early involvement bursaries could be adopted internationally.
• 2Applicants should be registered with your organization to ensure appropriate involvement throughout the award period and the ultimate research grant.
• 3
  The application procedure should include the following:

  1. Provision of a realistic budget for the involvement.
  2. An appropriate method employed for collecting service user's opinions.
  3. An outline of a support structure for the service users, such as a named mentor, or training provision.
  4. A pathway outlined for continued involvement in the ultimate research project.
  5. A method for evaluating the involvement including feedback from the service users.
• 4Feedback regarding the efficacy of the bursary should be collected from recipients upon conclusion of the work outlined in the application form.
• 5Models of good practice and impact arising from the bursaries should be disseminated to promote further good practice.

Source of funding

Unfunded.

Conflict of Interests

None to declare.