Reporting of patient involvement: a mixed-methods analysis of current practice in health research publications using a targeted search strategy

Abstract

Objectives

To evaluate the extent and quality of patient involvement reporting in examples of current practice in health research.

Design

Mixed-methods study. We used a targeted search strategy across three cohorts to identify health research publications that reported patient involvement: original research articles published in 2019 in the British Medical Journal (BMJ), articles listed in the Patient-Centered Outcomes Research Institute (PCORI) database (2019), and articles citing the GRIPP2 (Guidance for Reporting Involvement of Patients and Public) reporting checklist for patient involvement or a critical appraisal guideline for user involvement. Publications were coded according to three coding schemes: ‘phase of involvement’, the GRIPP2-Short Form (GRIPP2-SF) reporting checklist and the critical appraisal guideline.

Outcome measures

The phase of the study in which patients were actively involved. For the BMJ sample, the proportion of publications that reported patient involvement. The quality of reporting based on the GRIPP2-SF reporting guideline. The quality of patient involvement based on the critical appraisal guideline. Quantitative and qualitative results are reported.

Results

We included 86 publications that reported patient involvement. Patients were most frequently involved in study design (90% of publications, n=77), followed by study conduct (71%, n=61) and dissemination (42%, n=36). Reporting of patient involvement was often incomplete, for example, only 40% of publications (n=34) reported the aim of patient involvement. While the methods (57%, n=49) and results (59%, n=51) of involvement were reported more frequently, reporting was often unspecific and the influence of patients’ input remained vague. Therefore, a systematic assessment of the quality and impact of patient involvement according to the critical appraisal guideline was not feasible across samples.

Conclusions

As patient involvement is increasingly seen as an integral part of the research process and requested by funding bodies, it is essential that researchers receive specific guidance on how to report patient involvement activities. Complete reporting builds the foundation for assessing the quality of patient involvement and its impact on research.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• A targeted search strategy was used to identify examples of patient involvement reporting in a variety of publication types and study designs in health research.

• A mixed-methods approach allowed for an analysis of both the completeness and quality of patient involvement reporting.

• In this study, we coded statements reporting on patient involvement in 86 health research publications that may be adapted for further use.

• Qualitative studies were excluded from the analysis: in many cases, it was difficult to distinguish between patients’ contributions as research partners or as subjects of the qualitative research.

• Reporting of patient involvement was insufficiently detailed to allow for a systematic assessment of the quality of patient involvement.

Introduction

Patients’ viewpoints should be included in clinical research as they are the most affected by it.¹ Different approaches can be used to make the outcomes of clinical research more relevant to patients. One option is to actively involve patients or patient representatives in study design, study conduct and dissemination. Different terms are used to describe this active involvement, for example, ‘patient and public involvement’ (PPI) or ‘patient engagement’. Patient involvement in health research varies widely and can be categorised, for example, according to the level or continuity of involvement, involvement in different phases of the research or the methods applied for involvement.^{1 2} Standards and principles for patient involvement focus mainly on the management of the relationship between patients and researchers.³ These principles are important for a good collaboration, but only reflect one quality aspect of patient involvement or its impact on outcomes of clinical research. Given the efforts from patients, there is also an ethical imperative to reflect about their input in the publication, and the results and the impact of patient involvement should be evaluated and published.

Quality in clinical research is assessed with critical appraisal tools, such as the widely used risk of bias tool for randomised controlled trials (RCTs).^{4 5} A critical appraisal tool to assess the quality of patient involvement was developed in 2010.⁶ High-quality reporting is needed to allow for critical appraisal and quality assessment. A reporting guideline GRIPP (Guidance for Reporting Involvement of Patients and Public) was developed in 2011⁷ and updated in 2017 to GRIPP2.^{8 9} GRIPP2 comes in two different formats: a long form (LF) for studies with patient involvement as primary focus and a short form (SF) for studies with patient involvement as secondary or tertiary focus, such as, for example, clinical studies being informed by an active involvement of patients.

In 2014, the British Medical Journal (BMJ) group endorsed a policy which made it a requirement to report on PPI in BMJ journals and recommends GRIPP2 as a reporting standard.¹⁰ Price et al¹¹ compared reporting of PPI before and after the introduction of the BMJ policy. They found that while 86% of research articles included a PPI statement about 1 year after the introduction of the policy, only 11% actually reported PPI activities. Funding organisations are also likely to play an important role in improving the quality and reporting of patient involvement, especially as they increasingly require patient involvement in clinical research.¹²

The objective of this study was to analyse the extent and quality of patient involvement reporting and the quality of patient involvement in examples of current practice in health research. We are aware of several studies that investigated the rate and quality of reporting¹³ or critically appraised patient involvement in specific domains of clinical research.^{12 14–16} Overall, these previous assessments identified very few publications that reported patient involvement and/or engagement and reporting quality was suboptimal (eg, 0.4% of the sample described the active involvement of patients in orthopaedic research¹⁶). Jones et al¹⁵ also included studies which had patient involvement as primary focus, such as prioritising research topics. Our scope was different: we focused on studies that actively involved patients to inform the study methodology (including dissemination) but did not have patient involvement as primary focus. We did not limit our analysis to a specific research area or experimental design (eg, RCTs), but included three purposively selected cohorts of publications in which we expected reporting of patient involvement. We considered patients as people affected by the disease or topic, their family members or representatives of those affected.

Methods

A protocol detailing the methods of this study was pre-registered in the Open Science Framework (28 April 2020) and is openly available¹⁷ (see also online supplemental material 1).

Samples

In order to identify publications that report on patient involvement, we used a targeted search strategy in the following three samples, restricted to English-language publications:

1. Publications in the journal the BMJ, which requires reporting on patient involvement in research articles. We performed a Web of Science search (Web of Science Core Collection, 4 March 2020) to obtain all publications published in 2019 in the BMJ (document types: ‘Article’ or ‘Review’).

2. Publications listed in the Patient-Centered Outcomes Research Institute (PCORI) database.¹⁸ PCORI is a US-based organisation funding patient-centred research, which continuously screens Medline via PubMed, relevant journals and PCORI staff recommendations for publications on patient engagement in health research. We filtered for topic: example of engagement in health research; stakeholder involvement: patients; year: 2019.

3. Publications indexed in Dimensions¹⁹ citing one of the two GRIPP2 publications^{8 9} or the critical appraisal publication.⁶

Details of the search strategies can be found in online supplemental material 2. All included publications across these samples were checked for additional links or references, which described patient involvement in more detail. If additional relevant documents were found (eg, online supplemental materials, previously published protocols), they were included in the sample.

Inclusion and exclusion criteria

The inclusion and exclusion criteria outlined in the protocol were refined during the screening process to accommodate for the wide variety of studies included in the sample. As a result, the following inclusion and exclusion criteria were applied across all samples (see flow chart in online supplemental figure 1).

Study type

• Quantitative studies (RCTs, observational studies, etc) were included; qualitative research studies were excluded. We excluded qualitative research publications, as in many cases it was difficult to distinguish between the active involvement of patients in the study and their involvement as subjects of the qualitative research.

• Systematic reviews and scoping reviews were included; narrative reviews were excluded.

• Mixed-method studies and those which used qualitative and quantitative methods were included if the methods were mainly quantitative.

• Protocols were excluded. If an original publication in our sample cited a protocol which provided more detailed information on patient involvement, this protocol was included as an additional document.

• Studies in which a tool was developed and tested were included. Studies in which tools/interventions/outcomes were developed but not applied were excluded.

• Comments, editorials, guidelines, consensus papers and other publications, which did not aim to answer a research question, were excluded.

Patient involvement

• Publications were included if a patient involvement activity was described for at least one phase of the study in the main text (only in the acknowledgements was not considered as sufficient), that is, patients or patient representatives were actively involved in designing or running the study, were engaged as co-researchers, supported the dissemination of results or had an advisory function; participating in a study as a ‘subject’ or ‘participant’ was not considered as sufficient to qualify as patient involvement.

• Publications were included if patient involvement was used to inform the study, but it was not the primary focus of the study. The authors’ decision whether to complete the SF or LF of the GRIPP2 reporting checklist was used as an indicator of the focus of the study (if applicable).

• Publications were included if patient involvement activities had already been conducted (not only planned). The only exception was for patient involvement in dissemination activities. Given that dissemination activities often take place after a study is published, studies with planned patient involvement in dissemination activities were included.

All identified studies were screened by at least two members of the team. Discrepancies were discussed in the team until a consensus was reached.

Coding

We coded all publications and additional documents using three coding schemes (online supplemental tables 1–3):

1. Phase of involvement: included publications had to report patient involvement in at least one of the three study phases: study design (subcodes ‘research question’ and ‘outcome measures’), study conduct or dissemination (subcode ‘coauthoring the manuscript’).

2. GRIPP2-SF⁸ to assess the reporting of patient involvement.

3. Critical appraisal tool⁶ to assess the quality of patient involvement.

We used an inclusive and pragmatic approach in the coding. For example, we accepted a statement such as ‘patients were included to inform the study design’ as sufficient to describe the aim of patient involvement according to GRIPP2. A mere description of tasks was considered as sufficient to code ‘have the researchers discussed the nature of tasks’ according to the critical appraisal tool. We also coded acknowledgement and contribution statements if these mentioned phases or activities of patient involvement. With the aim of assessing the quality and impact of patient involvement based on included publications, we coded statements that addressed criteria in the critical appraisal tool. However, the varying amount of detail reported across publications and samples did not allow for a systematic appraisal of the quality of patient involvement.

Two raters coded all included publications according to the coding schemes. Discrepancies were discussed until a consensus was reached. If this was not possible, a third person assessed the respective passage and the team decided by majority vote.

Analysis

Coded segments were exported from MAXQDA²⁰ and analysed further in Microsoft Excel. Codes from additional documents were merged with that of the original study. We quantified how many publications in each sample reported one of the codes at least once. For the BMJ sample, we additionally report the frequency of patient involvement across all research articles that were published in 2019, given the journal’s requirement to report whether PPI has taken place.¹⁰ Of the n=200 search results for the BMJ sample, n=155 were articles that reported (original) research (see online supplemental figure 1). These had to be identified to compare our results with Price et al.¹¹

Additionally, we conducted a qualitative content analysis based on the extracted GRIPP2 codes. Similar codes within the same GRIPP2 category were grouped into overarching themes. All coded segments were also reviewed for illustrative examples.

Patient and public involvement

Patients or the public were not involved in the planning or conduct of this meta-research study. The analyses were not restricted to studies on specific diseases or patient populations; therefore, it would not have been adequate to include a specific patient group since this research is not specifically relevant for them. The main target audience includes researchers and other stakeholders in health research (eg, journal editors, funders). The results have been discussed in workshops with health researchers and patients and/or patient representatives and other stakeholder (eg, funders) to raise awareness of this topic and to describe the progress of integrating patient involvement in health research.

Results

Inclusion and exclusion

A total of 86 research publications were included in the analysis after applying our inclusion and exclusion criteria (see online supplemental figure 1). From the BMJ sample, 32 of 155 research articles (21%) were included because they reported PPI activities and qualified as quantitative study. We included a further 41 publications from the PCORI sample and 13 from the Citation sample (12 citing GRIPP2 and 1 citing the critical appraisal tool). Most frequently applied exclusion criteria were ‘no patient involvement’ and ‘no research publication’. We included 35 additional documents, which provided further information on patient involvement described in the publications.²¹

Phase of involvement

Patients were most frequently involved in the study design (90% of included publications, n=77), followed by study conduct (71%, n=61) and dissemination (42%, n=36) (table 1 and ²¹). In 17% (n=15) of the publications, patients were involved in formulating the research question and in 31% (n=27) in defining outcome measures.

Table 1.

Phase of involvement

Phase of involvement	BMJ (N=32)	PCORI (N=41)	Citation (N=13)	Total (N=86)
Study design	75% (n=24)	100% (n=41)	92% (n=12)	90% (n=77)
Research question	16% (n=5)	15% (n=6)	31% (n=4)	17% (n=15)
Outcome measures	22% (n=7)	41% (n=17)	23% (n=3)	31% (n=27)
Study conduct	56% (n=18)	76% (n=31)	92% (n=12)	71% (n=61)
Dissemination	41% (n=13)	39% (n=16)	54% (n=7)	42% (n=36)
Coauthoring the manuscript	6% (n=2)	24% (n=10)	38% (n=5)	20% (n=17)

BMJ, British Medical Journal; PCORI, Patient-Centered Outcomes Research Institute.

GRIPP2 (short form)

Between 13% (n=4, BMJ sample) and 77% (n=10, Citation sample) of the publications reported the aim of patient involvement (table 2 and ²¹). The predominant code for aim identified in our sample was ensuring that patients’ perspectives were taken into account. Coded segments ranged from vague statements (table 2, Example (E) 1) to more elaborated accounts (table 2, E2). Other examples included support with recruitment or the dissemination of the results and ensuring the accessibility or acceptability of the study.

Table 2.

Reporting of patient involvement according to GRIPP2-SF: quantitative and qualitative results

GRIPP2-SF	BMJ (N=32)	PCORI (N=41)	Citation (N=13)	Total (N=86)	Examples (E) from the publications
Aim Report the aim of PPI in the study	13% (n=4)	49% (n=20)	77% (n=10)	40% (n=34)	E1: vague, unspecific statement ‘He was recruited to provide a patient’s perspective.’30 (Citation) E2: elaborated, informative statement ‘To ensure an improved completion rate in the full study, we hired a patient partner (B.G.S.) to approach bedside nurses, enrol participants and assist family caregivers with the completion of the questionnaires. We expect this will increase the completion rate because this family caregiver has the lived experience, perseverance and communication skills that are necessary to make the consent process less overwhelming.’31 (Citation)
Methods Provide a clear description of the methods used for PPI in the study	28% (n=9)	68% (n=28)	92% (n=12)	57% (n=49)	E3: broad statement lacking details ‘Patient discussion groups were used as a means of involving patients in setting the research question and for determining the outcome measures.’ and ‘The outcome measures were identified after discussion groups in our unit, with patients who had previously undergone knee arthroplasty surgery.’32 (BMJ) E4: detailed, informative statement ‘During the pre-award period, the researchers developed a stakeholder management plan (‘roadmap’) outlining where stakeholders could influence the study. (…) Creating a stakeholder roadmap allowed us to target defined actions to engage stakeholders quickly, even before the study begun. Our stakeholder engagement plan focused on 4 key sets of activities: (1) study planning (including study design, intervention design and procedures, outcomes measurement, (materials); (2) hospital/patient recruitment and retention; (3) study implementation; and (4) translation, including interpreting study findings and disseminating results back to participating communities and the public.’33 (PCORI)
Study results Report the results of PPI in the study, including both positive and negative outcomes	28% (n=9)	78% (n=32)	77% (n=10)	59% (n=51)	E5: broad statement lacking details ‘Their input helped to refine the research question and to improve the protocol considerably.’34 (BMJ) E6: informative statement ‘The group stated that participant-facing material (participant information leaflet and invitation brochure) were not inclusive enough for all ‘father figures’ (e.g. stepfathers). The material in question was changed to be more inclusive. To boost recruitment to the trial, the PPI group suggested organisations that might be accessed by our target population and that might be interested in supporting the study (e.g. youth clubs, after-school clubs and sports centres). Therefore, we approached and successfully recruited fathers from after-school and martial arts clubs and Scouts groups for the cultural adaptation phase. When asked about dissemination of study results, PPI representatives thought that it was important to report the findings back to the local authorities that funded the HDHK [Healthy Dads, Healthy Kids] programmes and to include the use of the local authorities’ social media channels for dissemination. The group encouraged the research team to be fully open about the challenges faced in delivering the HDHK programme and the research study.’35 (Citation)
Discussion and conclusions Comment on the extent to which PPI influenced the study overall. Describe positive and negative effects	16% (n=5)	54% (n=22)	69% (n=9)	42% (n=36)	E7: statement reporting on influence on usability ‘We enhanced the usability of our findings by engaging key stakeholders (i.e. clinicians, parents and researchers) as part of our integrated knowledge translation activities (‥). Their guidance ensured that the review findings were both clinically meaningful and family-centred.’36 (Citation) E8: statement providing details on the influence on the primary outcome ‘Patient prioritization of meaningful physician discussion over reducing anxiety and depression, therefore, resulted in a substantial redesign of both the ACP [advance care planning] video and its evaluation trial. The team negotiated with its funder to change the primary outcome of the video’s evaluation from anxiety and depression to having a meaningful discussion. Meaningfulness of the discussion was operationalized by the measure ‘patient-centered nature of the patient–surgeon conversation’ in the resulting video evaluation trial, procedures for which have been previously described. This change in outcome also required that language be added to the video to encourage open communication between the patient and surgeon. In the practice-oriented context of the current study, using both engagement and research approaches in endpoint selection provided an innovative means of identifying and prioritizing endpoints.’37 (PCORI)
Reflections/critical perspective Comment critically on the study, reflecting on the things that went well and those that did not, so others can learn from this experience	3% (n=1)	20% (n=8)	77% (n=10)	22% (n=19)	E9: statement reflecting on organizational issues ‘Originally, we intended to conduct these sessions in a group format, due to difficulties with PPI partners’ schedule commitments, one-to-one sessions were conducted.’ and ‘It is extremely important that researchers plan PPI at the grant proposal stage and estimate the costs appropriately. If these costs are not correctly estimated during the initial stages of developing research proposals, they may cause a financial burden on PPI partners.’38 (Citation) E10: statement reflecting on representativeness ‘One limitation will be engaging patients and family members who are deeply committed to the project and who consequently may not provide a representative range of patient-family perspectives. However, their insights will be vital to identifying key aspects of patient/family-centered decision aids and they will directly inform the next stage of our research—cognitive interviews with patients and their families about advance care planning aids.’39 (PCORI)

BMJ, British Medical Journal; GRIPP2-SF, Guidance for Reporting Involvement of Patients and Public-Short Form; PCORI, Patient-Centered Outcomes Research Institute; PPI, patient and public involvement.

More than half of the publications (57%, n=49) provided some information about the methods used for patient involvement in the study. However, these accounts were often not very detailed. The most predominant code was consultation or giving feedback, indicated by describing the group involved: patient representative, advisory group, patient group, adviser to the steering committee, patient engagement group or lay representative. In many cases, even very basic information such as the number of involved patients, the frequency of meetings or explanations on how discussions took place was lacking (table 2, E3). Some publications reported on approaches they used for the consultation, such as working on an online platform or group meetings. Others used additional methods such as focus and discussion groups or interviews with patients to get further input on specific questions. A detailed, informative example was the development of a ‘roadmap’ prior to the study (table 2, E4), which served to identify how stakeholders could influence the study.

Of all publications, 59% (n=51) reported the results of patient involvement in the study. Examples of the reported outcomes included making materials easily understandable, supporting with or sharing ideas on recruitment, raising awareness about the study and identifying patient-centred outcomes. The level of reporting varied from broad statements with only few or no concrete examples (table 2, E5) to more detailed information on the outcomes of the patient involvement and its influence on the study (table 2, E6; see also Minneci et al²² for detailed information in a supplement).

Of all publications, 42% (n=36) provided information on the influence of patient involvement on the study (‘discussion and conclusions’). Examples include its influence on the intervention, recruitment, retention, usability of study findings (table 2, E7) and outcomes (table 2, E8).

A relatively small number of publications (22%, n=19) reported reflections and critical perspectives on patient involvement. Some reflections related to the research context and how its structure and settings may not always be welcoming for patient involvement (table 2, E9). Others discussed a possible lack of representativeness or diversity in the sample of PPI contributors (table 2, E10).

Critical appraisal tool

The critical appraisal tool⁶ focuses on the quality and impact of user involvement in research (table 3 and ²¹). Specific appraisal criteria were rarely reported, such as discussing the level of involvement (7%, n=6), considering whether findings were disseminated appropriately to recipients (7%, n=6) or conducting a formal evaluation (6%, n=5). More general appraisal criteria were reported more frequently, such as the nature of tasks patients were asked to perform (45%, n=39), how findings were disseminated (not requiring an active part of patients) (47%, n=40) or a general evaluation of the added value of involving patients in the research process (48%, n=41).

Table 3.

Reporting of patient involvement according to Wright et al⁶ critical appraisal tool: quantitative results

Question	Consider the following	BMJ (N=32)	PCORI (N=41)	Citation (N=13)	Total (N=86)
Planning and project design		38% (n=12)	54% (n=22)	69% (n=9)	50% (n=43)
1. Is the rationale for involving users clearly demonstrated?	(a) Have the researchers explained the rationale for user involvement? (rationale)	3% (n=1)	34% (n=14)	46% (n=6)	24% (n=21)
2. Is the level of user involvement appropriate?	(a) Have the researchers explained and justified the level of user involvement (level of involvement)	—	7% (n=3)	23% (n=3)	7% (n=6)
	(b) Have the researchers discussed the nature of tasks users were asked to perform (eg, identifying the research question, selecting the research method, commenting on information sheets, data collection, data analysis, dissemination)? (nature of tasks)	34% (n=11)	46% (n=19)	69% (n=9)	45% (n=39)
Recruitment and training		6% (n=2)	39% (n=16)	77% (n=10)	33% (n=28)
3. Is the recruitment strategy appropriate?	(a) Have the researchers explained how users have been identified? (identification)	6% (n=2)	29% (n=12)	62% (n=8)	26% (n=22)
	(b) Have attempts been made to involve a wide cross-section of interests where appropriate (eg, ethnic minorities, age, gender)? (diversity)	—	10% (n=4)	8% (n=1)	6% (n=5)
	(c) Have the researchers discussed the credentials of the users involved? (eg, Do the researchers discuss why the users involved are appropriate to meeting the aims of the involvement activity?) (credentials)	3% (n=1)	15% (n=6)	23% (n=3)	12% (n=10)
4. Is the nature of training appropriate?	(a) Have the researchers discussed the nature of the training provided? (nature of training)	—	5% (n=2)	38% (n=5)	8% (n=7)
	(b) Is the nature and extent of the training justified by the researchers? (eg, Do the researchers discuss how the training meets the needs of the users during the course of the study?) (justification of the training)	—	—	31% (n=4)	5% (n=4)
	(c) Has an account been given of user involvement training for professional researchers, where necessary? (user involvement training for researchers)	—	2% (n=1)	—	1% (n=1)
Data collection and analysis		3% (n=1)	10% (n=4)	31% (n=4)	10% (n=9)
5. Has sufficient attention been given to the ethical considerations of user involvement and how these were managed?	(a) Do the researchers discuss ethical issues relating to the involvement of users in research (eg, fatigue, the emotional demands of data collection)? (ethical issues)	—	2% (n=1)	15% (n=2)	3% (n=3)
	(b) Are there any discussions about the management of ethical issues (eg, provision of adequate information about research tasks, peer supervision)? (management of ethical issues)	3% (n=1)	2% (n=1)	8% (n=1)	3% (n=3)
6. Has sufficient attention been given to the methodological considerations of user involvement and how these were managed?	(a) Have the researchers discussed methodological issues relating to user involvement in research (eg, potential impact on the quality of the data)? (methodological issues)	—	10% (n=4)	8% (n=1)	6% (n=5)
	(b) Do the researchers discuss how methodological issues are managed (eg, how differences in interpretations of qualitative data are negotiated?) (management of methodological issues)	—	5% (n=2)	15% (n=2)	5% (n=4)
Dissemination		84% (n=27)	46% (n=19)	69% (n=9)	64% (n=55)
7. Have there been any attempts to involve users in the dissemination of findings?	(a) Have users been involved in the writing of the … funding application? (writing of the funding application)	9% (n=3)	10% (n=4)	38% (n=5)	14% (n=12)
	… of the publication? (writing of the publication)	6% (n=2)	27% (n=11)	46% (n=6)	22% (n=19)
	(b) Have the researchers described how the findings have been disseminated to participants and service users? (description of dissemination)	81% (n=26)	27% (n=11)	23% (n=3)	47% (n=40)
	(c) Are findings disseminated appropriately where necessary (eg, translation of findings into different languages, provision of interim findings to participants in receipt of palliative care)? (accessible dissemination)	9% (n=3)	5% (n=2)	8% (n=1)	7% (n=6)
Evaluation and impact assessment		19% (n=6)	59% (n=24)	85% (n=11)	48% (n=41)
8. Has the ‘added-value’ of user involvement been clearly demonstrated?	(a) Do the researchers discuss what difference involving users in the design and conduct of the research has made to the research process? (ie, Have the researchers considered whether the study and findings would look any different if users were not involved?) (difference made to the research process)	19% (n=6)	59% (n=24)	85% (n=11)	48% (n=41)
	(b) Do the researchers support the claims for the benefits of user involvement with examples from the research project? (examples of assessment of the benefits)	13% (n=4)	51% (n=21)	62% (n=8)	38% (n=33)
9. Have there been any attempts to evaluate the user involvement component of the research?	(a) Have the researchers discussed the evaluation of the impact of user involvement on the research project (eg, impact on the length of the study, the financial cost of involvement activities, cost–benefit analyses)? (evaluation of the made impact)	—	7% (n=3)	15% (n=2)	6% (n=5)
	(b) Do the researchers support claims about the impact of user involvement with examples from the evaluation? (examples of evaluation)	—	2% (n=1)	8% (n=1)	2% (n=2)

BMJ, British Medical Journal; PCORI, Patient-Centered Outcomes Research Institute.

Further appraisal criteria that were addressed in very few publications were the nature of training of patients (8%, n=7) and researchers (1%, n=1), as well as ethical (3%, n=3) or methodological (6%, n=5) considerations and how these were managed (3%, n=3 and 5%, n=4, respectively).

Discussion

We analysed a sample of 86 publications in health research that reported on patient involvement. While many publications provided information on general aspects relating to patient involvement, even very basic details were often lacking. For example, nearly all publications reported generally that patient involvement took place during study design. However, more specific information about whether this involvement in study design included defining the research question or prioritising outcome measures was reported to a much lower extent. Similarly, 40% and 57% of publications reported on GRIPP2 aims and methods, respectively, but the reporting was often suboptimal and statements rather vague. Despite authors alluding to many aspects of patient involvement included in the GRIPP2-SF and critical appraisal guidelines, we identified a need to improve completeness and details of reporting. This corroborates findings from previous studies on reporting of patient involvement and/or engagement.^{12 13 16}

Due to this incomplete reporting, coding according to the GRIPP2-SF categories was a challenge: the sparseness of reporting in many of the publications meant that these categories were relatively broad and overlapping. For example, a statement such as ‘patients helped with the identification of meaningful outcomes’ could describe the method (ie, focusing on the process) or the results (ie, identifying outcomes) of patient involvement. While the GRIPP2-SF reporting checklist is certainly useful to guide reporting in studies not having patient involvement as primary focus, our findings suggest that complementary measures could further bolster its impact on the quality and consistency of the patient involvement evidence base. Such measures could, for example, include broader requirements to include a statement on patient involvement in publications, more specific guidance for authors and peer reviewers, and standardised formats without word count restrictions to support more complete and consistent reporting. High-quality reporting is the basis for assessing the quality of patient involvement.

We observed considerable differences between our three samples regarding reporting, with both the Citation and PCORI sample providing more information on patient involvement than the BMJ sample. In these samples, patients were also more often coauthors of the manuscripts, reflecting their active roles in the whole research process. This is not surprising given the expected emphasis on patient involvement in the former samples compared with providing information in a mandatory section.

For the BMJ sample, we assessed the percentage of publications that reported patient involvement in the mandatory PPI section. Of all research articles published in 2019 (n=155), 21% reported patient involvement activities. In the sample of Price et al,¹¹ which included research articles published between June 2015 and May 2016 in the BMJ, patient involvement was reported in only 11% of the articles. Thus, the proportion of research articles reporting on patient involvement doubled in only a few years, demonstrating the impact of this journal policy to enhance visibility and to raise awareness for patient involvement. While this trend is encouraging, descriptions of patient involvement in the BMJ sample were generally very short and did not elaborate on the results of patient involvement or provide a thorough description of the process. For example, several publications reported on the inclusion of patient discussion groups without describing the composition, specific tasks or influence of this group.

In contrast, in the PCORI and Citation samples, we often found very detailed descriptions of patient involvement, including the nature of performed tasks, concrete examples of its influence on the study and critical reflections (see Weschke et al²¹ and table 2). In many cases, these descriptions were provided in additional documents. This suggests that the word count limit imposed by journals likely contributes to the limited detail in patient involvement reporting. Additional documents or structured tables for reporting of patient involvement may be helpful. However, this approach may come with the risk that patient involvement is seen as an add-on rather than as an integral part of the conducted research.

Strengths and limitations

One of the strengths of our study was the use of a targeted search strategy to identify examples of patient involvement reporting in current practice across a variety of publication types and study designs in health research. Moreover, coding according to three distinct schemes allowed us to capture different aspects of relevance, including the phase of patient involvement, the use of and adherence to reporting guidelines (GRIPP2-SF), and the quality and impact of patient involvement (critical appraisal tool). All statements reporting patient involvement analysed in this study are openly available²¹ for further use. For example, coded statements may inform the development of automated tools to detect reporting of patient involvement in publications.

We could not systematically assess the quality of patient involvement according to the critical appraisal criteria as originally planned. Quality assessment highly depends on reporting completeness and detail, which was inconsistent across publications. Such an analysis in our diverse sample might have favoured long versus short reports, or participatory health research approaches versus PPI activities informing a clinical trial.

Initially, we did not plan to exclude publications applying qualitative research methods. However, we did not find a clear definition to differentiate between the active involvement of stakeholders and their involvement as participants in qualitative research, for example, in focus groups or interview studies.^23–26 In some cases, both were reported in the same publication.²³ This particular challenge has previously been noted in the context of assessing reporting of PPI.¹¹ Despite attempts to delineate these approaches by differentiating between producing data to answer research questions versus informing decision-making processes of a research project,²⁷ this is not common practice yet. More generally, excluded qualitative research studies that reported patient involvement often had patient involvement as primary focus. An analysis of these studies was beyond our scope. While the targeted search used in this study facilitated the identification of patient involvement reporting in practice, it was challenging to develop appropriate inclusion and exclusion criteria and apply them consistently across the wide variety of included studies. In some cases, a different categorisation may have been possible. Moreover, since reporting of patient involvement is sometimes limited to very brief statements within the main text of a publication, it is possible that some publications reporting patient involvement were missed. Finally, we used GRIPP2 as a reporting guideline to assess the completeness of reporting of the included publications. The use of reporting guidelines without modification to serve as evaluation tools has been questioned by Logullo et al²⁸ as their purpose is to guide writing. However, the authors of GRIPP2 explicitly stated that it can also be used for planning patient involvement or for quality assurance.⁸

Conclusion

Despite important developments in the last years, patient involvement is still not a well-established approach in clinical or health research.^{14 29} Therefore, we would encourage journals to request an obligatory patient involvement statement from their authors, and to give guidance on detailed reporting in a structured table or additional document. We would also encourage journals and funding organisations to support the reporting of patient involvement by requiring the use of GRIPP2-SF as a reporting tool. Finally, we encourage researchers to include sufficient detail on patient involvement in their study to allow others to derive and apply lessons learnt in their own studies.

We expect that patient involvement will become more important in the next years to increase the relevance of research, in line with increasing demand from funders, publishers and society. Broader implementation of policies and more specific guidance are needed to leverage the impact of existing reporting guidelines, and thereby improve the quality of the patient involvement evidence base. Complete reporting builds the foundation of assessing the quality and appropriateness of patient involvement and is essential towards increasing its impact on research.

Data availability statement

Data are available in a public, open access repository. The data generated in this study are openly available under the Creative Commons Attribution 4.0 International license in Zenodo (https://doi.org/10.5281/zenodo.6245750).

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study does not involve animal subjects, human participants or patient data; thus, ethics approval was not required.