Abstract
Objectives
Standardised reporting of patient and public involvement (PPI) in research studies is needed to facilitate learning about how to achieve effective PPI. The aim of this evaluation was to explore the impact of PPI in a large UK study, the Life After Prostate Cancer Diagnosis (LAPCD) study, and to explore the facilitators and challenges experienced.
Design
Mixed-methods study using an online survey and semistructured interviews. Survey and topic guide were informed by systematic review evidence of the impact of PPI and by realist evaluation. Descriptive analysis of survey data and thematic analysis of interview data were conducted. Results are reported using the GRIPP2 (Guidance for Reporting Involvement of Patients and the Public, Version 2) reporting guidelines.
Setting
LAPCD study, a UK-wide patient-reported outcome study.
Participants
User Advisory Group (UAG) members (n=9) and researchers (n=29) from the LAPCD study.
Results
Impact was greatest on improving survey design and topic guides for interviews, enhancing clarity of patient-facing materials, informing best practices around data collection and ensuring steering group meetings were grounded in what is important to the patient. Further impacts included ensuring patient-focused dissemination of study findings at conference presentations and in lay summaries.
Facilitating context factors included clear aims, time to contribute, confidence to contribute, and feeling valued and supported by researchers and other UAG members. Facilitating mechanisms included embedding the UAG within the study as a separate workstream, allocating time and resources to the UAG reflecting the value of input, and putting in place clear communication channels. Hindering factors included time commitment, geographical distance, and lack of standardised feedback mechanisms.
Conclusion
Including PPI as an integral component of the LAPCD study and providing the right context and mechanisms for involving the UAG helped maximise the programme’s effectiveness and impact.
Keywords: Adult oncology, Urological tumours, QUALITATIVE RESEARCH, STATISTICS & RESEARCH METHODS
Strengths and limitations of this study.
This paper provides an example of reporting patient and public involvement (PPI) using the Equator Guidelines for Reporting of PPI (Guidance for Reporting Involvement of Patients and the Public, Version 2; GRIPP2).
The survey and topic guide have been informed by evidence on the impact of PPI and by realist evaluation.
The paper provides the views and experiences of both patient representatives and a varied sample of researchers involved in the Life After Prostate Cancer Diagnosis (LAPCD) study.
A convenience sample of both patient representatives and researchers was used, so results may not be generalisable.
The survey was limited to only those who were involved in this study, and therefore small numbers are reported.
Background
Patient and public involvement (PPI) has the potential to increase the quality and relevance of healthcare research. Systematic reviews of the impacts of PPI on healthcare research have been published.1–3 However, a lack of in-depth and accurate reporting of PPI has been recognised as a limitation in reaching evidence-based guidance on the most appropriate methods to use for successful involvement. Guidelines for the reporting of patient and public involvement in research (Guidance for Reporting Involvement of Patients and the Public, Version 2; GRIPP2) have been developed to help standardise the reporting of PPI and advance the evidence base.4
Frameworks and models have attempted to identify factors that influence impact. The Research with Patient and Public Involvement: a Realistic evaluation (RAPPORT) study used realist evaluation drawing on Normalisation Process Theory to understand how far PPI was embedded within healthcare research in six areas: diabetes mellitus, arthritis, cystic fibrosis, dementia, public health and learning disabilities.5 6 They reported a context-mechanism-outcome model and suggested that six salient actions are required for effective PPI: a clear purpose, role and structure for PPI; ensuring diversity; whole research team engagement with PPI; mutual understanding and trust between the researchers and lay representatives; ensuring opportunities for PPI throughout the research process; and reflecting on, appraising and evaluating PPI within a research study. More recently, the Public Involvement Impact Assessment Framework (PiiAF) has been developed.7 The main elements that influence public involvement in research and the impact this involvement can have are identified in PiiAF: the approaches (way in which members of the public are involved in the study), the values (values associated with public involvement by members of the research team), the focus of the research and the study design and practical issues including human and material resources.
This paper reports an evaluation of the impact of PPI in the Life After Prostate Cancer Diagnosis (LAPCD) study, a large UK-wide study of men living with and beyond prostate cancer8 using the GRIPP2 guidelines.4 The LAPCD study aimed to explore the impact of prostate cancer on men’s health and well-being, using a self-completion survey (n=35 823) and in-depth telephone interviews (n=119), to inform future service delivery and policy development. The LAPCD study had six workstreams and adopted the novel approach of dedicating one workstream to PPI (figure 1).
Figure 1.
Workstreams (WS) in the Life After Prostate Cancer Diagnosis (LAPCD) study.
A user advisory group (UAG) was established to lead this workstream and was integrated into the research programme from the outset. The UAG, including the Chair, comprised seven men from different parts of the UK who had experienced different stages of prostate cancer and experienced different treatments, and two representatives from Prostate Cancer UK. Each UAG meeting was attended by two researchers. Each workstream lead worked with the Chair of the UAG to develop a plan of how the UAG group would be involved. The Chair then discussed this plan with the UAG group before confirming the programme of work. The level and nature of the UAG involvement was different for each workstream. For example, it was easier to plan involvement in developing patient-facing materials or in aiding the qualitative analysis, but more difficult within the statistical analysis of the survey data or health economic data. In formulating their mode of operation, members of the UAG drew from earlier research findings from patient and public views on the impact of PPI in health research and followed the methods of Crocker et al.9 The study set out six different types of impactful contributors for a user advisory group including the ‘expert in lived experience’, the ‘creative outsider’, the ‘free challenger’, the ‘bridger’, the ‘motivator’ and the ‘passive presence’, and reported the importance of PPI contributors should be involved as equal partners.
The primary purpose of PPI in the LAPCD study was to ensure that the research was conducted and disseminated in ways useful to patients and the public and to ensure that the purpose and aims of the research were clearly understood by the patients, so that participation was facilitated. The UAG members sought to add value to the LAPCD research by offering a perspective that drew on their lived experience, both as a patient with cancer and as a patient advocate and volunteer support worker. The definition of PPI used in LAPCD was ‘research being carried out ‘with’ or ‘by’ members of the public, rather than ‘to’, ‘about’ or ‘for’ them’.10
The evaluation of PPI aimed to assess the ‘value added’ or impact of the UAG on LAPCD study and to explore the facilitating and hindering factors experienced.
Methods
Sample
The sample for this retrospective evaluation included all members of the UAG (described earlier) and the research team. The research team included clinical and non-clinical health service researchers of all grades from immediate postgraduate to senior team leaders, disciplines covered included statisticians, health economists, social scientists, qualitative researchers and clinicians with medical and surgical backgrounds.
Realist evaluation ‘theory of change’
This evaluation was informed by realist evaluation.6 Realist evaluation seeks to find the contextual conditions that make interventions effective therefore developing lessons about how they produce outcomes to inform policy decisions. Tilley outlined three investigative areas that need to be addressed when evaluating the impact of an intervention: what is the mechanism or process needed to produce the outcome, what is the context or environmental factors needed to produce particular outcomes and the outcome pattern, that is, what are the practical effects produced by causal mechanisms being triggered in a given context?6 This informed the development of a ‘theory of change’ model (figure 2).
Figure 2.
Theory of change model.
Design
Based on previous systematic literature reviews describing the impact of PPI on research1 2 11 and informed by realist evaluation6 (see figure 1), an online survey was developed in collaboration with the UAG to explore both the UAG and researchers’ views on the impact that the UAG had on the LAPCD study. To enable a more in-depth evaluation of PPI, two authors (ZD and JB) developed an interview topic guide alongside the UAG. Semistructured telephone interviews were conducted with both researchers and UAG members following this topic guide. The surveys were piloted, and the interview topic guides were reviewed by three academics and three patient representatives with prostate cancer. Minor changes to the wording of the documents were made because of this process.
A link to the online survey was emailed to all researchers (n=29) and UAG members (n=9 including Prostate Cancer UK members. All responses were anonymous. The survey included questions on their definition of PPI, level of user contribution to different parts of the study, how user involvement was supported, what hindered user involvement and personal benefits to both the users and researchers. Two open questions were asked to both the users and the researchers, the first to provide three examples of how the UAG added value to the study, and the second how the method of user involvement could be modified to gain an even greater impact. The questionnaire took approximately 15 min to complete. Survey results were reported using descriptive statistics.
Semistructured telephone interviews were conducted with all participants who agreed to be interviewed. Interviews were conducted by ZD, an experienced qualitative researcher. Participants were asked to reflect on the contexts, environment, processes and mechanisms that influenced PPI impact, both positive (facilitating) and negative (hindering). They were asked to describe how they perceived the impacts of PPI on the study. The interviews were transcribed and inductive and deductive analyses were conducted following Braun and Clarkes (2006) six-step approach to thematic analysis.12 This started with the familiarisation with the data by reading and re-reading the transcripts, followed by the generation of initial codes. Five initial transcripts were independently coded and then discussed within the research team and UAG, before coding the rest of the transcripts. After coding, ZD searched for themes by examining the codes and how some of them clearly fitted together into a theme. The themes with relevant quotes were then sent to the UAG for discussion. The UAG either agreed that the quotes fitted the theme that ZD had developed, suggested that specific quotes would fit better with one of the other themes ZD had developed or decided that certain quotes did not fit with any of the existing themes, suggesting a new theme. ZD and the UAG had several discussions and reflections on the themes, then a final decision about the themes was decided on by ZD and the authors. Finally, we reviewed the themes and defined the themes before writing up a first draft of the analysis. Results are reported according to the GRIPP2 reporting guidelines.4
Ethics approval
This study involves human participants and was approved by Oxford Brookes University Research Ethics Committee (ref Brett 22019). All participants provided informed consent.
Consent for the survey was completed on the initial page, and participants had to complete this consent process to gain access to the survey. Survey responses were anonymous. Consent for the interviews was recorded verbally just before the telephone interviews. This was recoded separately to the interview recording.
Patient and public involvement
Patients with prostate cancer and Prostate Cancer UK were involved in the design of the study, development of all participant facing materials used, development of survey and topic guide, analysis of qualitative data and write up of paper.
Results
The survey was completed by 79% (n=23/29) of the researchers and 100% (n=9/9) of the service users. Interviews were conducted with 7 UAG members, 2 representatives from Prostate Cancer UK and 14 researchers. Results are ordered by facilitating and hindering context factors of PPI; facilitating and hindering mechanisms of PPI and impacts of PPI on research, researchers and service users. Selected, illustrative quotes from the interviews are reported in tables 1–3 under the same headings as the Results section.
Table 1.
Facilitating and hindering contexts: illustrative quotes from interviews with researchers and service users
| Context (environmental) factors | Researchers (R) and service users (U) |
| Facilitating context factors | |
| Diverse patient representatives with good leadership | “It’s very important to have the right sort of Chair and the right group of people because … one of the things that I found with this group is that individually we’ve got on and that does make a lot of difference. We all come from different backgrounds, different parts of the country and we all seem to gel as a group” (U4) “[The Chair] was absolutely a central point and … it’s important to have that - somebody who is able to facilitate and involve others, not just himself, from a wider group” (R7) “We were very diverse in our way with different issues and different problems and different perspectives but maybe we were lucky but we operated brilliantly as a group…The dynamics were very good. We had a good team and I think that is absolutely critical.” (U2) “So it’s like any other research team you are putting together…[the UAG] needs the right background and experience, it helps if they’ve got communication skills and then obviously to come from a professional background that helps but it also hinders in a way because you’re not going to get the full experience of people who are not professionals and of course that’s a large proportion of the population” (R7) |
| Feeling confident to get involved | “I think we all had a certain background – we were used to big meetings and didn’t find it intimidating” (U3) |
| Feeling included in communications and discussions | “Right from day one I felt I was involved and included and I attended the first meeting where everyone attended at [city] and I received a very warm welcome. People came and introduced themselves to me and I just felt welcomed and valued” (U5) “One [way of approaching the UAG] was at the formal meetings. Then in between times if we had specific things … I would contact [the Chair], then he would disseminate it round to the other men … That was helpful … he was the central point of contact and that worked very well … it’s less confusing if you have a point of contact” (R8) |
| Hindering context factors | |
| Time taken to be involved | “There’s a darn sight more work in this that I thought, that’s the one thing - I became a lot more involved than I ever thought I would” (U2) “If I had a document to go through I could take two/three days. Whatever it took to go through it” (U4) “there was a huge commitment over and above, you know, the resources, you couldn’t have recompensed people for what they were doing – (R3) “I think one of the difficulties is that all the other members of the research team are full time researchers and working on the project and it’s quite difficult [with the UAG] - you feel quite conscious that you are taking up someone else’s time when they could be doing something else” (R1) |
| Lack of knowledge of some areas of research | “Obviously, they deal with the stats, they deal with the technicalities, they deal with that stuff but I thought sometimes, you know, hold on a second, this is about improving the life of men after prostate cancer” (U2) |
Table 2.
Facilitating and hindering mechanisms: illustrative quotes from interviews with researchers and service users
| Mechanisms (processes) | |
| Facilitating mechanisms | |
| Embedding the UAG from the start | “They were embedded from the start so it just kind of became second nature almost. So in terms of every time we have one of our monthly investigator meetings, there is always a section on the agenda for an update from the user advisory group and every time if we’re writing a paper we’ve always involved [the Chair] as a co-author because he’s a co-investigator on the study. So it’s just sort of always been there. (R4) “We weren’t just bolstered on we were embedded and that was great but we weren’t embedded early enough to have a full influence or attempt to make an influence [on the protocol] to make suggestions and so on and so forth that might have been quite useful.” (U1) |
| Clear aims and guidelines | “We put together Terms of Reference for ourselves and discussed that …[and], we developed a good practice guide to managing things online and how we were going to follow things up and so on and so forth. Yes, there was quite a lot of grounding and stuff and how we are going to work on this etc and I think that was time well spent actually” (U1) “I suppose it’s being clear at the outset if you take for example a qualitative strand how involved are the users going to be … mapped that out a little bit more clearly” (R6) “I think the general consensus on our part of the users group was that we felt it was quite important what we had contributed, in as much as it gave them a baseline to work from so they weren’t just making up questions that they thought were important rather than questions from people who had been involved … unless you’ve got input from people who’ve actually been through the process, you can sometimes end up missing some of the points” (U3) |
| Need for training | “I think maybe the staff, certainly in our stream, could have had a one day workshop on user involvement … I don’t think my team at least knew how to deal with the UAG” (R5) |
| Overcoming different ways of working | “it’s sometimes hard to marry up how the users think about it and how we as researchers think about it”(R4) “So I know that there were sometimes some very strong views from [the UAG] … but I think we did reach common ground in the end and it’s all generally very healthy to have those discussions (R12) “I did an economics degree, OK it was years ago. But I just couldn’t follow it [statistical discussion] and a lot of other people–you look across the table and their eyes would be glazed over.” (U1) |
| Time and resources | “… so there were honorarium which I’m grateful for so we weren’t employees or anything like that but there was a gesture or token whatever you’d like to call it to say you are giving up your time and your thinking and so on and so forth so we are going to be rewarded for it so that was good” (U1) “It’s important that it’s properly resourced otherwise you get to an absolutely unacceptable balance where you have professionals who are being paid … then you have users who are working very hard on something sometimes making a difference who are not” (R11) |
| Communication channels and feedback mechanisms | “I’ve had emails from some of the other members [of the UAG] but mainly with the Chair. He’s really really good and really responsive and always gives detailed thoughtful comments on the work so yes, it is really helpful” (R2) “One [way of approaching the UAG] was at the formal meetings. Then in between times if we had specific things … I would contact [the Chair], then he would disseminate it round to the other men … That was helpful … he was the central point of contact and that worked very well.… it’s less confusing if you have a point of contact” (R8) “We’d thank them but we probably didn’t give enough as to how we would incorporate what they had done” (R5) “I think feedback was pretty good but I think it could have been better. I think it could have been built in as kind of a requirement in a sense […] Give us a bit of feedback and maybe how we could do it differently next time etc …” (U1) |
UAG, User Advisory Group.
Table 3.
Impacts on research, researchers and service users: illustrative quotes from interviews with researchers and service users
| Quotes | |
| Impacts on research | “There is no doubt we added a completely different dimension to the review. I mean they’re brilliant academics obviously, and in their own field they’re absolutely brilliant but I think we brought them down to ground sometimes … I think we brought them down by saying ‘don’t forget what this is about’. (U2) “They’ve been involved in so many different aspects of it, in terms of giving feedback on results, and I know they have done a lot of stuff on the qualitative work streams […] identifying themes and going through comments. I think, just making sure that what we’re producing is actually relevant to men is the main thing.” (R4) “They have been involved in lots of parts … putting the questionnaires together … topic guides for the interviews … general feedback on what was coming out [from the transcripts]. They’ve had input on papers, meetings, presentations, all that sort of thing. And I think what they’ve been really good at is driving on the dissemination side of things and making sure the findings make a difference. They’ve been very active on that and made it clear that that’s an expectation from them” (R6) “All I can say is if [the UAG] hadn’t been here the study would have been … much thinner, less thoughtful exploration and I think they have added a dimension to it, added a richness to it” (R8) |
| Impacts of service users | “It was very well run. I was very impressed. I have nothing to compare it with but I thought it was extremely well run, well organised, well thought out and beneficial in terms of producing the result that it was intended to. I learnt a lot from it which I will take back to my workplace” (U5) “Yes, I would be quite happy to do it again, I enjoyed it and I would do it without expecting anything for it” (U5) “I think definitely it’s something I’ll try and bring in more in future studies I think […] I think for big studies, definitely. Big studies with lots of different kind of aspects of the things we are looking at” (R2) “It was good speaking to people. For example, the user advisory group - each one of us had prostate cancer and it was good talking to professionals and it was actually quite strange because after about three or 4 months everybody forgot that we had prostate cancer which was absolutely brilliant. So that in itself was good. I found it fairly cathartic the whole thing”. (U2) |
| Impacts on researchers | “You are always at risk of a certain type of tokenism with patient engagement activities and on this occasion there wasn’t any. It was a very real and productive way of adding value to the project as a whole I thought. It made it more ‘real’ to all of us” (R11) “Just meeting different people, different perspectives - yeah I think it’s great actually” (R6) |
Impacts on the study and on the researchers and UAG are also summarised in figure 2, a ‘theory of change’ model developed by HB. The results are summarised in figure 3.
Figure 3.
Summary of results. UAG, User Advisory Group.
Facilitating and hindering context factors
The primary facilitating context factors reported by the UAG included feeling they had a clear role (a lot 100%, n=9) and an agreed set of aims and principles (a lot 78%, n=7), having enough time to contribute (a lot 100%, n=9) and feeling confident in contributing (a lot 75%, n=6). They felt valued (100%, n=9) and supported by the researchers (a lot 89% n=8, somewhat 11% n=1), and supported by the other UAG members (100%, n=9). All the users felt included in the study communications and discussions (100%, n=9). Minor hindering context factors included geographical distance between the UAG members and the research team (mainly based in Belfast, Leeds, Oxford and Southampton) and therefore travelling inconvenience and lack of knowledge to contribute to certain areas of the research.
The facilitating context factors from the researchers’ perspective were involving the service users early enough (definitely 86%, n=18) and fully enough (definitely 70%, n=16), clear aims of PPI in the activity (definitely 57%, n=13), feeling well supported by research colleagues in the PPI activities (definitely 75%, n=17) and having good relationships with the UAG members (definitely 68%, n=15). Minor hindering factors included not having enough time to fully involve the UAG, not having sufficient knowledge of how to involve them, and ‘worries about taking up the time of UAG members’. Researchers felt they included the UAG in communications about the study (definitely 45%, n=10).
In the interviews, both the UAG members and researchers felt PPI in the LAPCD study was well structured and benefited from strong leadership, and a committed group of members, who were keen to be proactively involved in the study. A positive group’s dynamic and diverse range of experiences and perspectives was also beneficial to how the UAG was able to operate and contribute to various aspects of the study. This included members’ own experiences of cancer but also experiences gained from their involvement in support groups for other men with prostate cancer. The involvement of a representative from the Black and Minority Ethnic (BME) community was noted as a particular strength, although it was also acknowledged that this benefit could have been used better by greater efforts to tap into the wider network of this BME member to gain a greater understanding of the issues of this population.
The previous experience, professional backgrounds and prior knowledge of UAG members were seen as an important facilitator for their individual involvement including familiarity with the various tasks involved and their general confidence within the UAG during meetings and conferences with the academics. However, this was also seen as a potential limitation, with a need for more cultural and socioeconomic diversity. It was acknowledged that recruitment of such a diverse group is difficult and that while hearing the representative voice is important, lack of confidence and skills to contribute may be a barrier to involvement. Consultation with a wider patient representative group was suggested as a possible solution.
While previous experience of PPI among the researchers was variable, PPI was generally viewed as a valuable component of research studies, and researchers were open to the involvement and potential impact of the UAG on the LAPCD study.
Researchers acknowledged that involving the UAG could take extra time and that occasionally external deadlines hindered engagement. Researchers voiced concerns over taking up too much of the UAG member’s time, recognising that they had volunteered to be a part of the study team even though they were offered a small honorarium for their time. Members of the UAG reported that being involved in the project required a significant time commitment, but they were willing to take on this commitment and took their role seriously. The availability of an honorarium was an important signal to the UAG that their commitment was valued, although the UAG members did not expect payment for their time and were just happy for their out-of-pocket expenses to be paid. Clear and open communication of key concepts and tasks were seen as a key process for positive involvement. Some workstreams, such as the health economic workstream, were more difficult to understand and sometimes it was difficult for the UAG member to understand the jargon.
Having the UAG Chair as the main point of contact with researchers functioned well. Communication with the UAG appeared to be regular and integrated into the existing communication channels set up for the project.
Facilitating and hindering mechanisms
Various mechanisms that helped foster and support the integration and engagement of the UAG within the study were identified, including embedding the UAG within the study as a separate workstream package, involving the UAG in study meetings, developing clear documentation such as a term of references at an early stage, allocating time and resources to the UAG, and putting in place clear communication channels and feedback mechanisms.
The interview data revealed that both the researchers and the UAG members agreed that embedding the UAG into the study through a dedicated workstream for PPI was a particular strength of the approach adopted. A key element of this was involving the UAG in regular study team meetings, which both facilitated involvement and helped to build relationships. By integrating PPI into the study in this way, the UAG were seen and treated by many researchers as equal contributors to the research process. Moreover, the involvement of a group of patient representatives rather than just one or two allowed for consistency and stability in PPI throughout the life of the study.
Social activities, such as going for dinner or drinks outside of more formal research activities, fostered relationships between the research teams and members of the UAG and were seen as important to facilitating engagement within the project. The development of clear documentation such as their terms of reference, both internally within the UAG and with regard to the UAG’s involvement in the research tasks, was seen as an important facilitator of effective PPI in the study. However, the provision of researcher training on how to best engage patients and the public was seen as a possible area of improvement within the approach. The UAG members reported training in certain research areas may have been useful to enable greater involvement in some areas of the research study, such as a basic understanding of health economics.
UAG members commented that the priorities of the researchers sometimes lost their patient focus, and it was the role of the UAG to bring this back. Researchers recognised that sometimes there was a mismatch between their focus and the focus of the UAG. However, the UAG were seen as on an equal playing ground to researchers and they worked together to overcome any difference of opinion. Furthermore, members of the academic research team recognised that their feedback mechanisms to UAG members could have been better. At times, their understanding of the impact of their contributions appeared to be implicit, as opposed to being the result of specific, formal feedback sessions. When direct feedback, whether formal or informal, was given, this appeared to be highly valued by UAG members.
Impact on research, researchers and service users
Both the UAG and the researchers reported the greatest impacts of PPI were on the improved survey design and topic guide, enhanced clarity of patient-facing materials, input into the likely stress and burden on the participants in the ethics application, ensuring the steering group meetings were grounded in what was important to patients and assisting with the dissemination of results of the study through papers and at conferences. The least impact of the PPI group was on data analysis methods, recruitment of participants and dissemination of results to participating hospital trusts.
In the survey, UAG members reported personal benefits from the PPI that included a ‘sense of helping others’ (a lot=100, n=8), enjoying the camaraderie (a lot=100, n=8), gaining confidence (a lot=75%, n=6; somewhat=25%, n=2) and sharing experiential knowledge to inform better services for men with prostate cancer (a lot=86%, n=6; somewhat=14%, n=1).
Researchers reported personal benefits including having a greater understanding and insight into what it is like to have prostate cancer as service users shared valuable experiential knowledge (definitely=86%, n=19; somewhat=14%, n=3), gaining a rapport with the UAG (definitely 81% n=18, somewhat 18% n=4), gaining knowledge of how to engage service users in research (definitely=67%, n=14; somewhat=33%, n=7), benefiting from the additional support UAG members provided in the research process (definitely=71%, n=15; somewhat=29%, n=6) and making the study feel ‘real’ to them (definitely=57%, n=12; somewhat=43%, n=9).
The interview data revealed that overall, the approach used to involve PPI in the study (ie, embedded UAG) was seen positively by both members of the UAG and researchers. Members of the UAG and researchers believed that the contributions of the UAG had a real impact on the project, including individual pieces of work (eg, survey development, qualitative analysis) as well as the project as a whole (eg, making sure that the findings of the study had a real impact on men with prostate cancer).
The positive experience that UAG members reported motivated them to be involved in future projects, while the positive experience reported by the researchers encouraged them to consider using a similar approach to PPI for future studies. Both researchers and members of the UAG saw the project and process as valuable, identifying personal and wider benefits to the project.
Discussion
Contextual factors that contributed to the beneficial impact of PPI on the LAPCD study included clear aims and roles, time to contribute, confidence in contributing and strong PPI leadership, feeling valued and supported by the researchers and inclusion in study communications. Mechanisms that contributed to beneficial impact included incorporating PPI into the study from the start with a planned programme of work dedicated to user involvement activities embedded in each workstream; the collaborative nature of PPI; having resources available to allow the integration of the UAG into the study; and regular attendance of the UAG members at study research days, teleconferences and social events outside of formal research activities to build relationships.
The LAPCD study embedded PPI into the study with a collaborative approach between the PPI workstream and all other workstreams of the study. The importance of systematic partnership working across all settings has been reported in other studies.13–15 Wilson et al reported that a ‘fully intertwined’ partnership approach alongside enabling contexts including resources, research host and organisation of PPI leads to a greater positive impact.16 Building these reciprocal relationships early is also important to develop shared goals for PPI at an early stage that fits the needs of the study.14 15
Feeling valued and supported by the researchers and feeling included in communication about the study are important drivers of impact improvement and motivation to stay involved in research and highlights the importance of the researchers’ attitude to the success of PPI.9 One study concluded that the most important contextual factors that influence the outcome of involvement are the researchers themselves and the skills, assumptions, values and priorities they start with.17 While training is available to prepare patient advisors for their new role as advisor, reviewer or collaborative partner, it is clear that the training of researchers is equally important.18–21 Researchers in this study reported a positive attitude to PPI but admitted knowledge of how to include them could have been improved. Development of training and awareness of existing exemplar training for researchers is needed.19
Hindering contextual factors included the geographical distance between UAG members and lack of knowledge to contribute in certain areas of the research programme. Mechanisms that hindered the impact of PPI included time limitations and adjustment to different ways of working between researchers and UAG members. These factors have been reported in previous evidence on the impact of PPI.2 Many of the challenges of PPI occur because of colliding worlds, where priorities, motivations and ways of working differ causing conflict and power struggles between researchers and service users.19 It is therefore vital that clear aims and roles are identified at the start of the study.2 22
Reported impacts of PPI on the LAPCD study were evident throughout the LAPCD study. The reported impacts on improvements in patient-facing materials, the design of research tools such as questionnaires, interview schedules and questions for focus groups and recruitment have previously been reported.2 23 Studies have also reported the impact of PPI on analysis of data.2 23 24 This can check the validity of study conclusions, correct misinterpretation of data, identify themes that would have otherwise been missed, identify which findings would be most relevant to patients or the public and improve the way in which results have been described in reports.20 PPI contribution to write up of papers, presentation of results at conferences and other dissemination activities has been reported to increase the likelihood of people acting on the findings.2 23
This study also reports the personal impact that PPI has on the patient advisors and the researchers. The UAG members reported a ‘sense of helping others’, enjoying the camaraderie, gaining confidence and sharing experiential knowledge to inform better services for men with prostate cancer. These personal impacts have previously been identified.11 Other studies have reported the notion of the ‘good citizen’, with PPI in research being a natural extension of their wider civic interests, and how involvement in research helps patient advisors to make sense of living with or recovering from disease and therefore offering space for the reconfiguration of self and identity.25 Researchers in the LAPCD study reported having a greater understanding and insight into what it is like to have prostate cancer as service users shared valuable experiential knowledge. This experiential knowledge has been referred to as ‘knowledge in context’.17
Evaluating the impact of PPI on a research study is complex, and several authors have explored frameworks that illustrate the factors that influence the difference that PPI has on a research study.5 7 This study found that similar context factors and mechanisms reported in the PiiAF influenced the value added from PPI in the LAPCD study.7 The ‘theory of change’ (figure 2) model used in this evaluation was informed by realist evaluation.6 This model identifies how the context, environment, processes and mechanisms influence the impact that PPI has on a research study. An adapted version of this model was used to present the results of this evaluation in figure 3 above.
Despite geographical diversity, there could have been more diversity in terms of socioeconomic status, cultural diversity and education in the UAG for this study. It could be argued that there is a need for both those who are confident enough to be a part of the research environment and happy to attend meetings, and those who are less confident in this environment but their contribution could be valuable in representing a wider population to help shape more representative research studies and healthcare services. A wider UAG group or ‘community of interest’ group could be established that the central UAG group tap into when needed.
Strengths and limitations
This is one of the first papers to directly trace the impact of PPI, and the strength of this impact within all aspects of the research cycle. The GRIPP2 guidelines for the reporting of PPI in healthcare research have been populated to provide an example (online supplemental appendix 1). This study was a retrospective evaluation of the LAPCD study and therefore may have been affected by recall bias. While the number of qualitative interviews was sufficient to gain saturation of themes, the sample size was low due to the limited size of the research team and UAG.
bmjopen-2022-060861supp001.pdf (258KB, pdf)
Summary/conclusions
The LAPCD study introduced a novel approach by integrating PPI into the study as a separate workstream that contributed to each of the other five workstreams. This enabled the UAG to be involved early in the study and to contribute to every stage of the LAPCD study. It provided facilitating contexts (clear aims and roles of PPI, equitability with the research team, strong relationships between the UAG and research team and perceived confidence and support around PPI) and facilitating mechanisms (planned time and resources for PPI available from the start, development of documentation for engagement: terms of reference, clear communication channels arranged, involvement of UAG in all team meetings and social activities to foster strong relationships). Beneficial impacts on the study were reported by both researchers and UAG members. Personal benefits were reported by UAG members and researchers which may have fostered commitment and influenced future attitudes to PPI. This paper provides an example of reporting of PPI using the GRIPP2 guidelines to contribute to standardised reporting of PPI in research.
Supplementary Material
Acknowledgments
The work was funded by the Prostate Cancer UK and Movember Foundation. We would like to thank all of the service users and researchers who gave up their time to take part.
Footnotes
Twitter: @BrettDennis
Contributors: JB and HB: project inception, management, questionnaire design, development of qualitative topic guides, analysis, write up. ZD: development of topic guide, interviews, analysis of qualitative data, write up. FM: development and distribution of questionnaire, analysis, write up. JK and DC: questionnaire design, qualitative topic guides, analysis, write up. EW and PW: input into development of questionnaire and topic guide, analysis, write up. AG and AWG: principal investigators to LAPCD study, commented on questionnaire and topic guide, input to interpreted findings, write up. JB is responsible for the overall content as the guarantor. All contributed to this manuscript and approved the final draft.
Funding: This study was funded by the Movember Foundation, in partnership with the Prostate Cancer UK (HO-LAPCD-14-001).
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were involved in the design, conduct, reporting or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Data availability statement
Data are available on reasonable request.
Ethics approval
This study involves human participants and was approved by Oxford Brookes University Ethics Committee (ValueAddedLapcd 2018). Participants gave informed consent to participate in the study before taking part.
References
- 1.Evans D, Coad J, Cottrell K. Public involvement in research: assessing impact through a realist evaluation. Health Serv Del Res 2014. 10.3310/hsdr02360 [DOI] [PubMed] [Google Scholar]
- 2.Brett J, Staniszewska S, Mockford C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect 2014;17:637–50. 10.1111/j.1369-7625.2012.00795.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Staley K, Buckland SA, Hayes H, et al. 'The missing links': understanding how context and mechanism influence the impact of public involvement in research. Health Expect 2014;17:755–64. 10.1111/hex.12017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ 2017;358:j3453. 10.1136/bmj.j3453 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wilson P, Mathie E, Keenan J. Research with patient and public invOlvement: a realist evaluation – the RAPPORT study. Southampton (UK): NIHR Journals Library, 2015. [PubMed] [Google Scholar]
- 6.Pawson R, Tilley N. An introduction to scientific realist evaluation. Sage Publications, Inc, 1997. [Google Scholar]
- 7.Popay & Colins, PiiAF The Public Involvement Impact Assessment Framework Guidance Popay, J and Collins, M (editors) with the PiiAF Study Group January, 2014. Available: http://piiaf.org.uk/documents/piiaf-guidance-jan14.pdf [Accessed Jan 2020].
- 8.Downing A, Wright P, Hounsome L, et al. Quality of life in men living with advanced and localised prostate cancer in the UK: a population-based study. Lancet Oncol 2019;20:436–47. 10.1016/S1470-2045(18)30780-0 [DOI] [PubMed] [Google Scholar]
- 9.Crocker JC, Boylan A-M, Bostock J, et al. Is it worth it? patient and public views on the impact of their involvement in health research and its assessment: a UK-based qualitative interview study. Health Expect 2017;20:519–28. 10.1111/hex.12479 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.NIHR Briefing notes for researchers - public involvement in NHS, health and social care research. Available: https://www.nihr.ac.uk/[Accessed Aug 2021].
- 11.Brett J, Staniszewska S, Mockford C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient 2014;7:387–95. 10.1007/s40271-014-0065-0 [DOI] [PubMed] [Google Scholar]
- 12.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]
- 13.Evans D, Bird E, Gibson A, et al. Extent, quality and impact of patient and public involvement in antimicrobial drug development research: a systematic review. Health Expect 2018;21:75–81. 10.1111/hex.12587 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect 2015;18:1151–66. 10.1111/hex.12090 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Dudley L, Gamble C, Preston J, et al. What difference does patient and public involvement make and what are its pathways to impact? qualitative study of patients and researchers from a cohort of randomised clinical trials. PLoS One 2015;10:e0128817. 10.1371/journal.pone.0128817 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Wilson P, Mathie E, Keenan J. Research with patient and public invOlvement: a realist evaluation – the RAPPORT study. Health Serv Deliv Res 2015. 10.3310/hsdr03380 [DOI] [PubMed] [Google Scholar]
- 17.Staley K. 'Is it worth doing?' measuring the impact of patient and public involvement in research. Res Involv Engagem 2015;1:6. 10.1186/s40900-015-0008-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Gibson A, Boddy K, Maguire K, et al. Exploring the impact of providing evidence-based medicine training to service users. Res Involv Engagem 2015;1:10. 10.1186/s40900-015-0010-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jinks C, Carter P, Rhodes C, et al. Sustaining patient and public involvement in research: a case study of a research centre. J Care Serv Manag 2013;7:146–54. 10.1179/1750168715Y.0000000003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.de Wit MPT, Elberse JE, Broerse JEW, et al. Do not forget the professional--the value of the FIRST model for guiding the structural involvement of patients in rheumatology research. Health Expect 2015;18:489–503. 10.1111/hex.12048 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Haywood K, Brett J, Salek S, et al. Patient and public engagement in health-related quality of life and patient-reported outcomes research: what is important and why should we care? findings from the first ISOQOL patient engagement symposium. Qual Life Res 2015;24:1069–76. 10.1007/s11136-014-0796-3 [DOI] [PubMed] [Google Scholar]
- 22.Faulkner A. Capturing the experiences of those involved in the true project. A story of colliding worlds. Eastleigh: Involve, 2004. [Google Scholar]
- 23.Staley K. What impact does patient and public involvement have on health and social care research? A literature review. Eastleigh, United Kingdom: INVOLVE, 2009. [Google Scholar]
- 24.Stocker R, Brittain K, Spilsbury K, et al. Patient and public involvement in care home research: Reflections on the how and why of involving patient and public involvement partners in qualitative data analysis and interpretation. Health Expect 2021;24:1349–56. 10.1111/hex.13269 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Thompson J, Bissell P, Cooper CL, et al. Exploring the impact of patient and public involvement in a cancer research setting. Qual Health Res 2014;24:46–54. 10.1177/1049732313514482 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2022-060861supp001.pdf (258KB, pdf)
Data Availability Statement
Data are available on reasonable request.