Facilitating public and patient involvement in basic and preclinical health research

Abstract

Involving patients in research broadens a researcher’s field of influence and may generate novel ideas. Preclinical research is integral to the progression of innovative healthcare. These are not patient-facing disciplines and implementing meaningful public and patient involvement (PPI) can be a challenge. A discussion forum and thematic analysis identified key challenges of implementing public and patient involvement for preclinical researchers. In response we developed a “PPI Ready” planning canvas. For contemporaneous evaluation of public and patient involvement, a psychometric questionnaire and an open source tool for its evaluation were developed. The questionnaire measures information, procedural and quality assessment. Combined with the open source evaluation tool, researchers are notified if public and patient involvement is unsatisfactory in any of these areas. The tool is easy to use and adapts a psychometric test into a format familiar to preclinical scientists. Designed to be used iteratively across a research project, it provides a simple reporting grade to document satisfaction trend over the research lifecycle.

Introduction

Involving patients or the interested public in research broadens a researcher’s field of influence, generating novel ideas, challenges and discussions. Basic, translational and preclinical research (hereto preclinical) is integral to the progression of innovative healthcare, indeed the majority of National Institute for Health (NIH) funds in the USA are focused on preclinical research[1]. Preclinical research is not a traditionally patient-facing discipline and implementing meaningful public and patient involvement (PPI) can be a serious challenge in the absence of well-defined support structures.

Increasingly in healthcare, patients and the interested public are sought as partners in study design and governance. This trend is growing due to an increasing requirement by national, international and charitable funding bodies to include PPI as a condition of funding[2–6]. We use the INVOLVE definition of PPI, whereby PPI in research is defined as research carried out with or by patients and those who have experience of a condition, rather than for, to, or about them[7]. PPI has multiple demonstrated positive impacts on research including the potential to reduce waste in the research landscape[4,8,9]. Furthermore, if a project or researcher is funded by a public body there is a duty to demonstrate accountability for the expenditure of supporting funding. Thus, incorporating PPI should be beneficial to both researcher and research outputs. There tends to be a recognition within the biomedical community that PPI can be beneficial to all stakeholders[10]. Awareness of PPI is certainly increasing, however, the incorporation of meaningful PPI as standard research practice is progressing slower than expected[11,12].

PPI is collaborative by its very nature and its impact on key stakeholders should be assessed in considering PPI success. Preclinical researchers are not patient-facing by nature and as such, some of the basic aspects of implementing meaningful PPI can be challenging without the relevant support structures. Understanding the perceived challenges and barriers as viewed by preclinical researchers is a first step in the development of appropriate resources and guidance documents to facilitate valuable collaboration and involvement between the public and the largest recipient cohort of publicly funded health research[13]. Here we describe an analysis of the views of preclinical researchers into the challenges regarding PPI. In response, we have developed open source tools to help individual researchers prepare themselves to successfully incorporate PPI into their research, and have also developed a PPI assessment survey and analysis tool for iterative and contemporaneous assessment which facilitates refinement and improvement of PPI activities in response to feedback from the patients or public involved.

Materials and methods

Ethics statement

Ethical approval was granted by UCD Human Research Ethics Committee. Survey responses were collected anonymously with informed consent of the participant for non-commercial use of the data provided.

Public and patient involvement statement

Patients were engaged through The Patient Voice in Arthritis Research PPI group. People with experience of living with any rheumatic disease from any area of Ireland were invited to apply to be involved in this study. Communication was remote, via email and phone. The patient insight partner (PIP) group of 12 were sent a review guide and a structured template for their review (Supporting Methods B in S1 File). PIPs were asked specifically about: language accessibility, relevance, usefulness, necessity of the questionnaire, missing aspects, the scale used, and the likelihood of intended use, questionnaire length, overall views and alternative assessment methods. The questionnaire was adjusted in response to PIP feedback. Telephone follow-up was used to obtain views on questionnaire refinement. PIPs were also asked to share the pilot survey within their relevant networks. The results of the study will be shared with PIPs via email and through the patient/researcher co-produced newsletter of the UCD Centre for Arthritis Research, News Rheum.

Researcher view on public and patient involvement

Researchers

An institutional wide notice and school-wide email was sent to researchers in the School of Medicine, The Conway Institute and the associated research faculties. Basic, biomedical, biomolecular, translational and preclinical researchers were invited to attend a discussion forum focusing on what they perceive to be barriers or challenges to patient involvement in research. Researchers from all experience levels (undergraduate to PI), a range of disease areas and research disciplines were in attendance (researcher characteristics, Table 1).

Table 1. PPI discussion forum researcher characteristics.

Research Experience	n (%)	Primary Disease Area	n (%)	Primary Research Area	n (%)
Undergrad	1 [6%]	Rheumatic Diseases	4 [25%]	Statistical	2 [13%]
Research Assistant	1 [6%]	Diabetes	2 [13%]	Molecular	7 [44%]
Project Manager	1 [6%]	Prostate Cancer	4 [25%]	Basic	6 [38%]
Postdoc	7 [44%]	Breast Cancer	2 [13%]	Psycho-social	1 [6%]
Research Fellow	3 [19%]	Cancer (Other)	2 [13%]
Principal Investigator	3 [19%]	Multi	2 [13%]

Discussion forum format

A semi-structured model was used as per Braun and Clarke[14]. A facilitator’s guide was prepared based on a review of the literature[15–28]. Discussion forum structure: an introduction to PPI, what it is, what it is not, how it applies to research, and consequences for funding, ethics and policy[29]. Clarification of key terms was also printed and posted on the walls for the entirety of the discussion forum for reference to all who attended. The attendees were divided into three discussion groups, each with a facilitator and a note-taker. An option for written contribution was also provided for those who did not wish to voice their opinions. The discussion forum lasted 2 hours and was held in an innovation meeting room, designed to facilitate community-brainstorming within the research institution.

Basic, translational and preclinical health researchers within the UCD College of Health and Agricultural Sciences were invited to a discussion forum to express their views on implementing PPI. All attendees consented to the non-commercial use of the data provided. A facilitated semi-structured discussion forum structure was used, consisting of three groups, each with a note-taker and a facilitator. There was also a roaming central facilitator. Notes were combined and a qualitative textual analysis was performed.

Planning canvas development

In response to the preclinical researcher views on implementing PPI, we proposed that reflecting on the main theoretical challenges for implementing PPI, which stem from the uncertain boundaries of the concept, in advance of starting a research project would facilitate downstream success for PPI. A common tool in to help business rethink their business strategy in a fast-evolving landscape is the Business Model Canvas (BMC). The BMC is used to enhance strategic thinking about business innovation[30]. We used the theory and design concept of the BMC informed by researcher views to develop the PPI Ready: Researcher Planning Canvas. The PPI Ready Canvas is designed to facilitate the researcher’s preparedness for PPI, rather than for planning an individual PPI activity.

Questionnaire development

A review of the health, public engagement, and marketing research literature was conducted[17,31–44]. A long questionnaire of 15 questions was developed in response to the researcher discussion forum (Supporting Methods C in S1 File). Three key processes for assessment were information (n = 4 questions), procedural fairness (n = 4) and quality (n = 7). The global assessment question “Overall, how satisfied/dissatisfied are you with your involvement in this project” was included as question 16 for convergent validity.

Face validity and questionnaire accessibility

A voluntary, community recruited panel of PIPs (n = 12) reviewed the questionnaire for face validity and language accessibility. The PIP review document can be found in supporting information (Supporting Methods B in S1 File). Questions were simplified and refined in response and changes discussed with PIPs. An 11 point satisfaction scale with 3 anchor points (at 0, 5 and 10) was used for all questions[45].

Survey pilot

A survey containing the 15 public involvement (PI) questions, one global assessment of satisfaction (GAS) question and the well characterized 10-question general self-efficacy (GSE) scale[46] was piloted on a cohort of 63 adults (Supporting Methods C in S1 File). All respondents self-reported as having attended a meeting or event(s) that gave them the opportunity to discuss or express their views about health research or to share their experience with researchers. Three did not consent to data storage and use; therefore responses were excluded from analysis. Of the 60 respondents, 72% (n = 43) were patients, 8% (n = 5) were carers, 17% (n = 10) were family members and 3% (n = 2) were other members of the public. All surveys were complete and there were no missing entries.

Data analysis

Data was analysed in IBM SPSS v24. Factor analysis with correlation matrix was used to identify co-linearity and redundancy within the 15 PI questionnaire. As expected, there was a high degree of co-linearity and variables were removed based on a correlation greater than 0.8. To inform refinement, PIP insight, views, and reported relevance was considered. Eight questions remained after refinement (8QPI). Factor analysis was repeated on the 8QPI data, which met all requirements for determinant (>0.0001), KMO (>0.8) and Bartlett’s test for sampling adequacy (p<0.05). Cronbach’s alpha with a cut-off of 0.7 was used for internal consistency reliability testing. Discriminant validity was analysed via factor analysis and component analysis and correlation between 8QPI and GSE. Convergent validity was established via linear regression between the average 8QPI and GAS question, using a cut-off value of 1-(2SE) (two standard errors).

Modelling the 8QPI into a quality control framework

The validated PI questionnaire includes two questions for information assessment, two questions for assessment of procedural fairness, four questions for quality assessment and the GAS question. We developed a simple flagging system for each of the three key assessment categories that are measured on a per-individual basis. Furthermore, there is also an overall assessment grade that provides a global measure of PIP satisfaction with an individual PPI scheme.

The excel analysis template can be found in supporting information (Supporting Methods A in S1 File and S1 Excel Template). PAS results are entered directly or via a linked file. For each question, responses of 0–3 issue an ‘Immediate Attention Required’ flag; 4–6 issued a ‘Some Attention Required’ flag and responses of 7–10 issued a ‘No Attention Required’ flag[47]. Content validity is automatically tested by comparing the GAS response to the overall mean PAS (Q1-8) response. If the GAS is within the mean +/- 2 standard deviations (as determined in the PAS pilot), a ‘PASS’ flag is generated, otherwise a ‘FAIL’ flag is produced and the data must be interpreted with caution.

An output summary table is generated on tab 2. Each category (communication, procedural and quality) receives a score based on the mean responses for all PIPs. An overall score based on mean response for all questions (Q1-8) and associated PPI Grade is generated for simple PPI satisfaction reporting. Based on the risk matrix concept, overall scores of 0–3 are ‘High Risk’ for PIP dissatisfaction/PPI failure, scores of 4–6 represent a ‘Moderate Risk’ and a score of 7–10 are ‘Acceptable’[48].

Results and discussion

Determining preclinical researcher views on PPI

Attendees were also encouraged to note their opinions and views if they did not wish to vocalise them or if a point occurred to them after the discussion had moved on. Discussions focused on key challenges and facilitators for implementation of PPI. The challenges were broken into three concepts: barriers, worries and concerns. Barriers refer to institutional barriers that prevent or discourage active PPI (for example: ethics, funding, administrative supports). Worries refer to personal challenges a researcher faces in implementing PPI (for example: social anxiety, imposter syndrome). Concerns refer to challenges about the impact on research (for example: loss of research focus, public discomfort with research methods (such as animal research).

Notes were collated and a textual and thematic analysis was performed (Fig 1A). The major barriers identified were ethical challenges, engagement of patients and the interested public, funding to carry out PPI and a perceived lack of relevant guiding documents. Key concerns identified were time commitment, dilution of research objectives, sacrificing long-term value for short-term gains, and public judgement of ethically approved research (particularly animal models and experimentation). Personal worries that hindered PPI included lack of communication skills, fear of public disengagement, burden of expectation and social anxieties.

Fig 1. Key challenges hindering PPI and a tool for researchers to develop strategies to overcome these challenges.

(A) Thematic analysis of a researcher discussion forum with preclinical researchers identified key challenges in the implementation and adoption of PPI into research practice. (B) A visual design tool based on the business model canvas, which we’ve called PPI Ready: Researcher Planning Canvas was developed to aid the individual researcher to explore their personal barriers to adopting PPI and map a strategy to overcome or circumvent them.

Communication is a key factor in researcher hesitancy to adopt PPI

A common theme across all challenges was communication. Communication barriers included a lack of predefined routes or mechanisms to source PIPs, and a lack of guidance documents regarding the legal and ethical responsibilities for local, national and international involvement of patients in research.

“One of the biggest barriers is engaging patients and getting them to come in […] we need to be able to sell PPI to the public.”

[Principal Investigator, Diabetes Research]

Communication worries included discomfort or reluctance to talk about personal experiences with patients; fear of upsetting a patient; anxiety about how to handle disagreements with a PIP.

“Worried that a mistake I make will upset a patient”

[Postdoctoral Researcher, Cancer Biology].

Communication concerns included being able to explain and justify the need for preclinical and basic research, which has a longer term and less direct research impact. There was concern around public representatives bringing personal judgement upon research methods.

“Patients maybe against some aspects of research such as animal testing. We need […] to convey the importance of animal testing when it comes to the progression of research”

[Principal Investigator, Neuroscience]

There was concern that the time spent justifying research that has already been through rigorous ethics application to public partners with alternative agendas could slow the research process and be counter-productive.

Communicating with vulnerable groups is not in the preclinical researchers’ toolkit

Preclinical research training may include scientific communications, presentation skills and even media training. Seldom, however, is training provided for communicating with vulnerable groups such as patient cohorts. The worry of appropriately communicating with vulnerable groups was particularly prominent with less experienced researchers and in researchers investigating fatal and life-limiting illnesses.

“The unfortunate reality of PPI is that some patients may have life-threatening diseases. What happens if a patient dies? As a researcher I don’t know what to do in that situation and I don’t know how to handle it”

[Postdoctoral Researcher, Pathobiology].

More generally, there was a recurrent worry of upsetting the public, fear of “saying the wrong thing” or causing upset or public disengagement. The perceived lack of guidance, combined with the bespoke nature of PPI resulted in many researchers feeling the burden of publicly representing their research and their institute. Whereas they may be comfortable presenting a fully peer-reviewed “sanctioned” piece of research output, there was a personal and professional vulnerability associated with sharing early stage and on-going research with the public. Researchers felt they would be guarded and could not discuss preliminary research fully if a public member was present for fear of said research being misconstrued or offering false hope.

Time consumption is a major concern hindering PPI implementation

The overwhelming largest concern about implementing PPI was the time required for the engagement, recruitment, procedural, and on-going management of public involvement. This may be a function of the relatively new strategy change towards PPI in Ireland.

“I simply don’t have enough time to implement PPI. I can’t delegate any work in relation to PPI because I don’t think enough people know what it is and what to do”

[Principal Investigator, Diabetes].

In particular, researchers who had no experience in PPI had the strongest sense that the time consuming nature of PPI outweighed its benefit to preclinical research.

“Time spent engaging with patients is time spent away from doing research”

[Postdoctoral Researcher].

Again, this could be influenced by the lack of experience in engaging patients and the perceived lack of guidance and clarity around PPI[49] and dissociation between preclinical research and policy setting[50–52].

Development of a tool to facilitate an individual researcher to be become PPI ready

PPI is bespoke; there is no standard formula for a good PPI initiative. However, one can be as prepared as possible to implement a bespoke PPI initiative. By reflecting on the main theoretical challenges for implementing PPI in advance of starting a research project, an individual researcher can facilitate the downstream success of their PPI initiative. An individual researcher’s theoretical challenges will stem from the uncertain boundaries of the concept of PPI. Therefore, we have adapted a standard business tool, the business model canvas (BMC), which is designed to enhance strategic thinking for business innovation and inform the exploitation of emerging opportunities in changing environments[30,53]. The PPI Ready Researcher Planning Canvas (Fig 1B) is designed to enhance strategic thinking about PPI and enable a researcher to prepare themselves for PPI. The PPI Ready Canvas is a visual plan with elements describing a researcher’s challenges, proposed solutions, the solutions’ value proposition, research impact, financial and time costs. It is designed to assist researchers in aligning their activities by illustrating potential benefits and trade-offs.

The PPI Ready Researcher Planning Canvas elements allow researchers to consider what the challenges are that are hindering their implementation of PPI. It is broken down into Barriers: Institutional policies, procedures or situations that may hinder PPI; Worries: Personal hurdles that cause anxiety or unease about PPI; Concerns: Perceived impediments to research value or practice. There is corresponding space to consider the response to the challenges. In order to consider the cost/benefit analysis of implementing PPI, there is a module of Value: what additional benefit will PPI bring to your research; and Impact: What effect on impact will inclusion of PPI bring? Consider cultural, economic, health, political, social and knowledge impacts. To ground the proposed solutions or responses, there are modules to consider both the timeframe required to prepare for PPI and also the associated costs and funding required.

A mechanism for iterative PPI assessment to allow incremental revisions and PPI development

Formalised PPI is a new concept in preclinical research. All initiatives, especially paradigm shifts, have associated risk. The earlier in the project lifecycle you can avoid a risk, the more accurate you can make your design. Many risks are not discovered until they have been to be integrated into a system and it is not feasible to identify all risks. Therefore, just like any area of the research cycle and design process, PPI should be assessed, evaluated and improved iteratively in response to experience and lessons learned. This may be tricky considering PPI by its very nature is not prescriptive. The form that PPI takes will be bespoke depending upon the needs of the research. However, the underlying concepts for valuable and meaningful PPI remain constant and can be measured and assessed.

We have developed a PPI satisfaction assessment survey to be used for iterative and contemporaneous PPI assessment throughout the life cycle of a research project (Table 2). The survey should be completed by PIPs throughout their role in the lifecycle of the research project. This iterative approach has a number of benefits including: mitigated risks earlier because PPI elements can be integrated progressively; accommodation of changing requirements and tactics as experience is gained and lessons learned; facilitated refinement and improvement of PPI resulting in a more robust and meaningful initiative; facilitated knowledge exchange as institutions can learn from this approach and improve their process; and enhance reusability of PPI initiatives.

Table 2. Validated PPI assessment survey (PAS).

		In response to the following questions, please indicate your level of satisfaction. 0 = very dissatisfied, 5 = neither satisfied nor dissatisfied, 10 = very satisfied.
Value Cate-gory	Q.	How satisfied are you:
I	1	With the facilities provided?
I	2	That your needs were considered in the planning of the project meeting(s)?
P	3	With your level of contribution in relation to what you expected?
P	4	With your comfort in voicing your thoughts and opinions?
Q	5	With your understanding of the research project and its aims?
Q	6	That you understand the roles of other team members?
Q	7	With the communication and feedback tools that you have been asked to use in this project?
Q	8	That you know who to contact if an issue arises or, if you have concerns about the project or your involvement?
GAS	9	Overall, how satisfied/dissatisfied are you with your involvement in this project?

I = Informational; P = Procedural; Q = Quality; GAS = Global Assessment of Satisfaction

PPI assessment survey development

A review of the literature and qualitative review by PIPs (n = 12, Table 3) lead to the development of 15 PPI assessment questions. Based on feedback from our PIPs and previous evaluation of scale types for health related research, we used an 11-point satisfaction scale with 3 anchor points (0 = very dissatisfied, 5 = neither dissatisfied nor satisfied, 10 = very satisfied)[45,54]. The global assessment of satisfaction (GAS) question was added to the questionnaire as a measure of convergent validity. The 16 question PPI evaluation (PI) questionnaire and the well characterized 10-question generalized self-efficacy scale (GSE)[46] were piloted on a cohort of 62 patients or interested public who self-reported as having attended a meeting or event that gave them the opportunity to discuss or express their views about health research or to share their experience with researchers. 60/62 provided the appropriate consent for analysis. The analysis pathway is outlined in Fig 2. Dimension reduction by oblique Factor analysis indicated multi-colinearity within the PI questionnaire (determinant 8.279x10^-11). Sampling was adequate (Kaiser-Meyer-Olkin (KMO, 0.885)) and there was homogeneity of variance (Barlett’s Test df 105, p<0.001). Analysis of the correlation matrix (Table A in S1 File) leads to the removal of highly collinear questions (Q3, 4, 7, 9, 11, 12 and 14). Factor analysis was rerun on the refined 8-question PI questionnaire (8QPI). Determinant value of 0.001 indicates an absence of collinearity. Sampling was adequate (KMO 0.886) and there is homogeneity of variance (Bartlett’s df 28, p<0.001).

Table 3. PIP characteristics.

Sex	n (%)	Public/Patient	n (%)	Rheumatic Disease	n (%)
Male	3 [25%]	Patient	11 [92%]	Osteo-arthritis	3 [25%]
Female	9 [75%]	Carer	1 [8%]	Rheumatoid Arthritis	4 [33%]
				Psoriatic Arthritis	2 [17%]
				Juvenile Idiopathic Arthritis	1 [8%]
				Not specified	2 [17%]

Fig 2. Development of the PPI assessment survey (PAS).

Questions were adapted from the literature and reviewed by a PIP cohort of n = 12 for face validity, relevance, jargon use, comprehension and language and format accessibility. Question wording and scale were refined in response and reassessed by a subset of the PIP cohort. The refined questionnaire and the general self-efficacy scale (GSE) were piloted on a cohort of patients and public that had attended a research involvement or engagement event. N = 60 responses passed the consent check. Oblique factor analysis identified co-linearity between PPI questions which lead to question exclusion based on quantitative analysis and informed by qualitative input for PIPs. The refined questionnaire passed content validity analysis, including, internal validity, discriminant and convergent validity analysis.

Internal consistency was measured for 8QPI questionnaire via reliability analysis and produced a Cronbach’s alpha of 0.932, indicating high reliability/internal consistency of the variables in the scale. Discriminant validity was measured between the 8QPI and GSE scales. Factor analysis identified 3 components: GSE variables, 8QPI variable 1 and 2, and 8QPI variable 3–8 (Table C in S1 File). Correlation between components ranged from -0.132–0.396 (Fig 2). The variance extracted (range 0.56–0.68) is greater than the correlation squared (range 0.02–0.12) and thus discriminant validity is established.

Convergent validity was measured via the correlation between the per-case mean 8QPI response and the GAS question (Overall, how satisfied/dissatisfied are you with your involvement in this project?), which was included in the pilot survey. Linear regression demonstrated a positive correlation (R 0.914, RSq 0.835 and Adjusted RSq 0.832, Fig 2).

An open-source simple analysis and reporting tool for the PPI assessment survey

Time consumption was highlighted as a major barrier to implementing PPI in the preclinical sciences. Therefore, development of a PPI assessment survey (PAS) would be redundant unless it was quick and easy to use. The tool is simple by design and at the suggestion of preclinical researchers who did not want to require any further training in order to be able to use it. An excel spreadsheet was selected as this programme is widely used and available and researchers were comfortable with it. We have therefore developed a series of simple excel templates with pre-formulated inputs for the analysis of the survey (Fig 3). Furthermore, there are linked templates that can be sent to the PIP which allows the researcher’s file to be updated and analysed as the PIP fills in their survey response in a separate excel document.

Fig 3. PAS assessment tool.

A preformatted excel file with embedded analysis formula is available for use with the PAS. (A) Tab 1 of the excel file is the data input tab. The PIP details are entered in (1). The date is entered in (2). The responses to the PAS survey are entered in (3) and the flagging system is automatically generated. The PAS scores can be entered manually or can be updated via a linked excel file sent directly to the PIP(s). (4) The GSE score should be within two standard errors of the mean PPI scores (Q1-8). If not, a FAIL flag is generated and caution is advised in the interpretation of results. (B) Tab 2 is the output summary. It summarises the scores for each value category across all PIPs for each given time point and also generates an overall score and grade. (C) The scores can be plotted over time to visualise the PPI satisfaction trend. As illustrated, the overall score is relatively flat, but plotting of each value score illustrates that the informational assessment required attention and improved upon implementation of changes (after the second quarter) to increase clarity in communications.

The analysis is multi-dimensional and uses a flagging system, which preclinical researchers would be familiar with from standard quality control measures used by laboratory equipment. The analysis tool uses a flagging system for each question answered on an individual PIP level. The banding for our flag system was based upon a previously published large analysis of different response scales[47]. If a PIP scored a question ≥7 no flag is generated. If a question is scored between 4–6 a yellow warning flag “some attention required” is generated. A score on any individual question of ≤3 generates a red warning flag “Immediate attention required”. These flags alert the researcher to risks associated with their PPI initiatives and allows them to put control measures in place. The iterative use of the PAS over time will facilitate trend analysis to determine the success of the control measures.

The GAS question is included in the PAS (question 9). For validation of the survey during use, the analysis template has an inbuilt content validation (Fig 3). If the GAS response is within the range of the Mean Q1-8 +/- two standard errors, the PAS is consistent in its measurement of satisfaction and a “PASS” flag is produced. If the GAS is outside this range a “FAIL” flag is produce and caution should be used in interpreting the results as there may be an underlying construct skewing the PAS response.

There is also an overall PAS assessment grade produced based on the mean PIP response across all questions. This can be used as an easy reporting measure for annual grant reports and for assessing a PPI initiative across the lifecycle of the project. The overall grade uses categorisation based upon standard risk assessment, a concept familiar to most preclinical researchers[55]. An overall grade ≥7 is acceptable, 4–6 highlights a moderate risk to PPI success, and a grade <4 highlights a substantial risk to PPI success. The overall global grade is a useful tool for the measurement and reporting of PAS. However, if used in isolation there is a risk of masking potential subscale concerns. The PAS assesses three value levels: informational (quality and clarity of the information and communication provided, n = 2 questions), procedural fairness (consideration of the PIP needs, n = 2 questions), and quality (how valued and valuable a PIP perceives the PPI initiative, n = 4 questions). The analysis tool generates a per-category summary, which is the mean response from PIPs within each value category. This allows easy identification of improvements required between value categories and allows the researcher to pinpoint the area requiring refinement in order to improve their PAS grade (Fig 3B).

Limitations of this study

The PAS is designed to facilitate assessment of PPI for preclinical researchers who tend to have little to no experience in qualitative, semi-quantitative or psychometric research. It is designed specifically for those with limited experience or access to established PPI resources. It addresses the key concepts underlying successful PPI collaborations between patients/public and researchers and ensures they are being addressed and assessed throughout the lifecycle of the project. The use of the open source analysis tool takes away the need for specialist training to use the PAS. The PAS can tell a researcher if there is a problem with PPI and in which category (information/communication, procedural fairness or quality); however, it does not identify the issue. The use of the flagging system on an individual PIP basis, allows for the rapid identification of whether it is an issue perceived by a single member of a PIP cohort, or if multiple PIPs have identified the same issue. This allows for simple direct follow-up with the PIP(s) to establish the cause of the issue without in depth qualitative analysis.

The circumvention of the qualitative analysis of other assessment tools, such as feedback forms which allow free text input, reduces the length of time and requirement for specialist knowledge and training for PPI assessment. It also allows the satisfaction trend to be easily tracked across time and in response to improvement measures. This can be beneficial but is also a limitation as there is a loss to the extent of information that can be assessed. A quantitative method cannot provide in-depth information on the motivations behind the levels of satisfaction nor provide details of PIP experiences with PPI[56]. The use of the PAS with informal PIP feedback in response to issues identified in PAS analysis provides a realist and feasible solution given the researcher challenges described above. However, for the most robust development and progression of PPI as a research and impact tool, a mixed methods approach using the PAS and qualitative in-depth exploratory PPI narrative research and analysis could enable new topics and insights to emerge, which could also be confirmatory for the quantitative findings.

Concluding remarks

Preclinical researchers are increasingly required to wear multiple hats as part of the standard expectations of a research career position. Established roles include study design, conduct and experimental analysis but they are also now expected to be professional teachers, presenters, public speakers, writers, reviewers, editors, managers and mentors. There is increasing pressure to commercialise research in-house, act as innovators and become social and political influencers[57,58]. It is therefore unsurprising that time commitment, communication and lack of guidance were considered major challenges to the implementation of PPI in preclinical research. Understanding that researchers face personal challenges, anxieties and worries, as well as institution barriers and research concerns is a first step in encouraging meaningful PPI collaborations. Addressing the challenges faced by preclinical researchers will greatly help the development of resources and guidance documents to facilitate PPI to one of the largest recipient cohorts of publicly funded health research[1,59].

The research describe here will aid preclinical researchers in the formulation of PPI. Adequate preparation in advance of a research project with public involvement should improve the success of that involvement. All new concepts have challenges and risk and involving the public in research is no different. Challenges are inevitable, however assessing public involvement iteratively through the use of the PAS described here within allows for those challenges to be addressed and managed contemporaneously. In conclusion, we have developed a tool specifically for the needs of the preclinical researchers to facilitate the implementation, assessment and improvement of PPI throughout the lifecycle of a research project.

Supporting information

S1 File. Supporting information.

Supporting methods and tables.

(PDF)

S1 Excel Template. PAS analysis template.

(XLSX)

Data Availability

All relevant data are within the manuscript and its Supporting Information files.

Funding Statement

ED received funding from the Irish Research Council (GOIPD/2017/1310) URL: http://research.ie/. JM and ED received funding from the Health Research Board (SS-2018-094) URL: https://www.hrb.ie/. The funders had no role in the study design, data collection or analysis, decision to publish or preparation of the manuscript.