Achieving effective patient and public involvement in international clinical trials in neurology

Abstract

There is a growing need for patient and public involvement (PPI) to inform the way that research is developed and performed. International randomized controlled trials are particularly likely to benefit from PPI, but guidance is lacking on how or when it should be incorporated. In this article, we describe the PPI process that occurred during the design and initiation of an international treatment clinical trial in MS. PPI was incorporated using a structured approach, aiming to minimize bias and achieve equivalence in study design, implementation, and interpretation. Methods included PPI representation within the study research team, and the use of focus groups, analyzed using thematic framework analysis. We report the outcomes of PPI and make recommendations on its use in other neurology clinical trials. By sharing our model for PPI, we aim to maximize effectiveness of future public involvement and to allow its effect to be better evaluated.

Patient and public involvement (PPI) in research refers to patients, carers, and/or public stakeholders working in partnership with researchers, often providing insights into what it is like to live with a particular health condition. PPI is aimed at improving the quality, relevance, appropriateness, transparency, and implementation of research and developing better relationships between researchers and the public.^1,2 PPI refers to members of the public meaningfully contributing to the process of research rather than being recruited as participants. PPI in research is recommended by national^3,4 and international organizations and can be a prerequisite for securing research funding⁵ or publication.⁶ PPI can improve the quality of clinical trials and ensure outcomes are of direct relevance to patients,^7–9 but consensus about what constitutes effective PPI during the design and conduct of international randomized clinical trials (iRCTs) is lacking.¹⁰ There are many regulatory and practical challenges in designing and conducting iRCTs, including the potential effects of cultural contextual differences between countries in prioritizing research, framing research questions, design, conduct, and analysis. Sharing models of effective PPI in iRCTs will help to maximize study potential and assist the incorporation of PPI in future iRCTs.

In this article, we present an example of PPI used during development of an iRCT, DELIVER-MS: Determining the Effectiveness of earLy Intensive Versus Escalation approaches for the Treatment of Relapsing-remitting Multiple Sclerosis (DELIVER-MS), conducted in the United Kingdom and United States.

The DELIVER-MS experience

DELIVER-MS (NCT03535298) is a pragmatic iRCT, which will recruit 800 people with early RRMS from 24 sites (12 United States and 12 United Kingdom). Participants will enroll into either a randomized (n = 400) or observational (n = 400) cohort for 36 months. The overall goal of the study is to compare an early highly effective vs an escalation approach to MS disease-modifying therapy (DMT) using a primary end point of brain volume loss, with long-term extension to examine clinical disability. PPI methodology has been incorporated at several stages of trial development (figure 1).

Figure 1. Flowchart summarizing patient and public involvement during the development of the application and the study design of the DELIVER-MS international clinical trial.

Methods of PPI in DELIVER-MS

Pre-award: contributions to priority setting

PPI shaped the original idea of a prospective, comparative effectiveness study of treatment algorithms in MS. In 2012, the UK MS Society and James Lind Alliance, a nonprofit organization hosted within the UK National Institute for Health Research, brought patients with MS, caregivers, and health professionals together in a priority-setting exercise.¹¹ Their top 10 research questions, included: “Does early treatment with aggressive DMT improve the prognosis for people with MS?” In 2017, a funding application for a prospective trial, developed by a UK-US collaboration of researchers, people affected by MS, and health care professionals, was awarded funding by the Patient-Centered Research Outcomes Institute (PCORI). PCORI is an independent nonprofit, federally funded organization in the United States that relies on input from key stakeholders, including patients, to set its research priorities and aims to generate evidence to allow patients to make informed health decisions.

Patient and public focus groups: contributions to the research protocol

We organized 4 pre-award and 2 post-award focus groups (4 United Kingdom and 2 United States) to explore the feasibility of the study and explore issues related to protocol design. PPI contributors were recruited from MS clinics by word of mouth and also from a panel of volunteers from an existing PPI network. Each focus group involved 5–11 contributors, seeking to include a range of participants who were heterogeneous with respect to disease status and time since diagnosis (table 1 and table e-1, links.lww.com/CPJ/A138).

Table 1.

Disease status of focus group participants

Focus groups lasted less than 3 hours, including regular scheduled breaks. Expert facilitators (A.M., D.M.M., and C.B.), including a person with MS (PwMS) who was also a coinvestigator (C.B.) chaired 5 focus groups; a coinvestigator (E.C.T.) chaired the other one. The use of expert facilitators, rather than principal investigators (PIs), was aimed at avoiding any unwanted influence of PIs on participants' views. Focus groups started with an introductory session that explained key concepts, objectives, and ground rules, followed by a more open discussion on study processes or design.

Pre-award focus groups were informal and explored study feasibility and design. Researchers took notes and responses to particular questions were surveyed. Two researchers (A.M. and C.B.) analyzed the data using a coding framework to identify key barriers and facilitators to study participation. Outcomes from the pre-award focus groups included the following: (1) a considerable proportion of individuals would decline randomization, prompting the development of a parallel observational cohort, and (2) the consensus that brain volume loss was considered relevant to PwMS.

After receiving funding from PCORI, 2 additional focus groups (1 United Kingdom and 1 United States) were completed using a semistructured guide (table e-2, links.lww.com/CPJ/A138) to obtain perspectives on several predefined topics. Approval from an ethical standards committee to conduct a post-award focus group was received in the United States. Questions were tailored for country contexts and included views on study acceptability and design, participant procedures, recruitment, and retention strategies. Conversations were audio-recorded and transcribed verbatim, and data were analyzed using a thematic approach structured around stages of the trial and patient information needs. Themes (table 2 and table e-3, links.lww.com/CPJ/A138) included advice on the likely mindset of potential participants at the time of first contact, what information to provide, and potential barriers and facilitators to randomization. These data allowed researchers to understand how contextual differences in each country (e.g., different funding models) might affect participant experience.

Table 2.

Summary of themes emerging from analysis of the post-award focus groups for DELIVER-MS

Patient Advisory Committee: contribution to study governance

To ensure that PPI played a role in the overall governance of the study, an international Patient Advisory Committee (PAC) was established to provide direct communication between stakeholders and the trial Steering Committee. The PAC ensures stakeholders' inform ongoing decisions, particularly with regard to addressing the needs and priorities of PwMS, while ensuring that appropriate information is provided to participants and the public. Membership of the PAC includes the DELIVER-MS PIs and project managers, US and UK PwMS (some with scientific expertise in MS), caregivers, and representatives of the UK and US MS Societies, UK National Health Service, and insurance providers (United States). Members initially attended a training session, aiming to ensure that stakeholders appreciated the intricacies of the study, and that investigators understood the perspectives of stakeholders. Two PwMS (1 United Kingdom and 1 United States) represent the PAC during Steering Committee meetings. The PAC meets via quarterly teleconference and is provided with frequent email updates about study progress to maintain continuity of engagement.

The PAC influenced study development in several ways. Members of the PAC recommended that a treating neurologist, rather than a research coordinator, should first approach prospective participants to discuss the study in view of the proximity to a recent diagnosis of MS and the potential for questions that extend beyond the study itself. The PAC also raised the issue of information governance when patient videos were posted on the DELIVER-MS web site (see Participant Video section). The PAC provide feedback on study materials, articles, news articles, website content, and advise on the optimum content and frequency of study updates.

Stakeholders engagement plan

At the recommendation of PCORI, researchers developed a Stakeholder Engagement Plan, a core study document aimed at ensuring meaningful involvement with public stakeholders. Key components of the Engagement Plan were (1) frequent contact of the study staff with the PAC and (2) reciprocal training sessions between the PAC and study staff. The PPI engagement process is monitored by the Steering Committee and by PCORI, including an audit trail of engagement with stakeholders to allow review(s) of their participation and the quality of their experiences.

Participant video: contribution to trial participant information

Focus groups recommended that study information be provided in several formats, including a video, to suit the literacy skills and preferences of prospective participants. A patient contributor was interviewed to seek views and experiences related to the study rationale and the pros and cons of research participation. The transcript was used to develop a patient script, and focus group recommendations were used to shape an investigator script. Ethics panels including lay members (United Kingdom and United States) scrutinized the script to ensure that it was accessible, informative, and nonpromotional. PwMS appearing in the video consented for content to be posted on the study web site, understanding the potential risks for online sharing of content.

Public dissemination concerning information about the trial

An impact plan was put in place to ensure the widest possible societal influence of study outcomes. Dissemination will be conducted throughout the study via scientific publications, media announcements, social media activity, and the study web site (deliver-ms.com), which was created using PCORI recommendations and input from PwMS. The web site has portals for the general public/potential participants and DELIVER-MS investigators. Content includes information on the study rationale and design, trial updates, and publications.

End-of-study results webinars presenting the key efficacy and safety outcomes will be broadcasted to study investigators, study participants, and PAC members. Recordings of the webinars will be made available to the general public on the study web site, contextualized for health systems in the different countries. The main study results will be submitted for presentation at key annual neurology conferences and for publication in a high-impact medical journal. Public dissemination of study outcomes will be monitored using output metrics relating to published articles, news items, and Twitter activity.

Discussion and recommendations

The DELIVER-MS study has demonstrated the direct and indirect value of a structured context-sensitive PPI strategy in international clinical trials. PPI should be an essential component of any clinical trial, using a systematic and planned approach from the outset. The geography of clinical trial participation is changing rapidly, including the movement of pharmaceutical trials into the developing world. PPI is likely to be particularly important in the planning and implementation of iRCTs by addressing the cultural realities of patients, health systems, and research sites across countries to ensure that the autonomy of individuals takes precedence over the demands of science or the interests of society.¹² In this article, we provide an example of how PPI was used during the development of an iRCT in MS. Here, we reflect on where our methods have built on existing evidence for good practice and also where uncertainties remain.

We used several key strategies to maximize the effectiveness of PPI (figure 2). Crucial features included early development of a strong PPI group, regular meetings and updates, and a culture of investigators interacting with PPI partners to address specific study issues in a targeted and responsive way.

Figure 2. Recommendations on the process of PPI during the development of an international clinical trial.

PPI = patient and public involvement.

Timing of PPI

Early use of PPI during study development is associated with greater influence on study design.¹³ Reasons may include greater potential to influence study design before it is immutable, fostering favorable relationships with PPI contributors by providing ownership from the outset, and the application of PPI to address discrete objectives rather than as tokenism during the preparation of funding applications.¹³ Ideally, pre-award PPI for an iRCT should occur in each country in parallel.

Recruitment of PPI contributors

Consensus is lacking about what experience or credentials are required for effective participation. It may be necessary to strike a balance between engaging regular PPI contributors with skills and experience in research, who arguably may be less able to reflect a broader patient perspective,¹⁴ vs recruitment of new PPI contributors, bringing fresh viewpoints. Our own PPI contributors were recruited by either word of mouth or a panel of existing PPI volunteers. We recognize that this introduced recruitment bias toward contributors experienced in research or who had an existing rapport with the investigators. Our recommendation would be to recruit groups with similar size and composition in each country, including some people experienced in PPI and others who are not.

Recruitment of a truly diverse international group of patients and public members is challenging, particularly avoiding domination of views from particular individuals or countries.^6,15 For instance, difficulty engaging marginalized populations, including those with financial and social constraints, represents a potential barrier to full inclusion,¹⁵ but public members from various educational backgrounds, with variable skills and knowledge, have been shown to be equally influential during group discussions about health care.¹⁶

PPI models of participation in research studies

Approaches to PPI vary in terms of (1) the people involved, e.g., single people/formation of groups, (2) the degree of involvement, e.g., consultation/collaboration/lay leadership of a study, (3) the forum for discussion, e.g., face-to-face/written consultation, (4) the degree of engagement of researchers, e.g., inviting/responding to PPI, and (5) the methodology used in analyses, e.g., opinion poll/thematic analysis. The nature of PPI may affect outcomes.¹⁶ Certain forms of involvement such as responsive models (e.g., focus group) or managerial roles appear to be more influential than public oversight of a study.¹³ Incorporating several different models of PPI into the study development process is likely desirable.¹³ Focus group discussion provides the opportunity for members to draw from each other's experiences and to gain confidence by sharing a collective voice.¹⁶ We found that involving facilitators who had expertise in PPI and context-sensitive qualitative analysis allowed for more structured discussions, providing practical conclusions. Importantly, by listening and dealing with difficult personal stories without allowing individual cases to dominate the conversation, trained facilitators were able to handle instances where discussing MS caused patients/caregivers to become emotional.

PPI preparation and planning

Preparation for PPI includes practical aspects such as venue accessibility, transport, reimbursement, and the provision of training and support. Structured training has been shown to enhance the legitimacy, credibility, and power of public representatives to influence health care decision making.¹⁶ PPI contributors accessed from the MS societies are supported by a PPI officer and have access to resources such as the European Patient Academy on Therapeutic Innovation Toolbox, providing educational tools to support research and development.¹⁷ Each of our focus groups began with a presentation outlining the scientific background in lay terms, using language and content that was appropriate to public representatives residing in that country. This was followed by information on the format and objectives of the engagement activity and some ground rules to encourage a relaxed and open discussion. In preparation for PAC meetings, bidirectional training was completed to ensure that both professional and public representatives were apprised of each other's needs and perspectives.

Barriers and costs relating to PPI

The financial costs of PPI must be incorporated into funding applications and study design; guidance exists to aid such budgeting.^15,18 We budgeted approximately $75,000 for PPI during DELIVER-MS (<1.0% overall study budget). However, these costs do not include PPI work done before the award of the grant and reflect the activities planned, aligned with the scale and scope of our study, and may not be directly translatable to other studies.

Funding pre-award PPI can be challenging because of short time frames and the lack of funding.¹⁴ Capitalizing on established local PPI structures with sustainable funding streams, partnering with public interest groups or applying for funds from local charities can overcome some barriers. As a minimum, budgeting for travel expenses, refreshments, and a small inconvenience payment for PPI contributors is recommended. In the United Kingdom, some PPI contributors refuse payment because it can jeopardize their social security payments; the use of gift vouchers may avoid this situation. Receipt of payment is a complex area likely to be affected by national social security regulations, which need to be taken into consideration in iRCT.

Time costs must also be anticipated, and the short application window for some funding schemes may be a barrier to engaging in detailed pre-award PPI. Researchers with an active research portfolio should consider the value of developing their own local permanent PPI networks, or forming links with existing networks, to facilitate brief PPI interventions.

Long-term sustainability of PPI throughout a study

Maintaining active engagement of PPI contributors before, during, and after a study requires planning. Lay representation on the trial Steering Committee is one option; we found that the addition of a parallel PAC promoted more active engagement of PPI partners in discussion. Maintaining effective communication between PPI partners residing in different countries, using means such as emails or text groups, regular tele-/video-conferencing, or web forums, can facilitate durable and meaningful engagement throughout the study.

Evaluating PPI strategies

PPI requires time and resources and therefore warrants scrutiny and evaluation.¹³ Evaluating the outcomes of PPI is also valuable for several reasons. First, planning evaluation requires that tangible aims of PPI be specified at the outset. Evaluation then allows researchers to determine whether their aims were achieved. Second, it is important for PPI contributors to understand the value of their input. We found that reflecting collectively on the value of PPI interventions, celebrating successes, and providing formal feedback for PPI partners were all important in maintaining engagement. Third, evaluating and reporting on the effects of PPI is useful to guide future practice; allowing other researchers to know what works, for whom, in what context, and why.

Identifying ways to quantify the effects that PPI exert on the way research is performed remains challenging; no agreed robust ways of capturing or measuring the effects exist, so the majority of reports are descriptive. The key PPI outcomes are likely to be the effects on study protocol, research process, and the PPI contributors themselves (e.g., personal reward or building skills).¹⁹ However, PPI may also potentially have an effect on researchers, research participants, communities, funders, publishers, and policy makers.¹⁹ Until recently, there was also no universally accepted framework for reporting or evaluating the effects of PPI. However, the EQUATOR method for developing reporting guidelines has now been used to produce Guidance for Reporting Involvement of Patients and the Public, which is in its second revision (GRIPP2).² The long and short GRIPP2 checklists should enhance the quality and consistency of PPI reporting, but can also be used as a template to design studies to formally evaluate the effects of PPI.²⁰ Both applications will serve to enhance the robustness of the evidence base for PPI.

In the increasingly global framework of modern clinical trials, cultural and contextual differences between countries affect the relevance, design, conduct, analysis, and influence of research. These same cultural and contextual differences also pose challenges for designing effective PPI, which must be well planned, well resourced, and timely to be used to the greatest possible effect.

Appendix. Authors

Study funding

This work was supported by the Patient-Centered Outcomes Research Institute (PCORI), MS-1610-37047 to D.O. and N.E., Multiple Sclerosis Society, UK, and National Multiple Sclerosis Society, USA.

Disclosure

E.C. Tallantyre reports honoraria and support to attend educational meetings from Merck and Novartis, support to attend educational meetings from Biogen, and salary as a UK MS Registry fellow from Biogen, outside the submitted work. N. Evangelou reports funding and support from the MS society, NIHR, MRC, Biogen, Novartis, Roche, and Teva, outside the submitted work. C. Bale reports no conflicts of interest. B.Z. Chaudhry reports personal fees for consulting and speaking from Genentech and Sanofi Genzyme, outside the submitted work. E.H. Gray, N. LaRocca, and S. Pavitt report no disclosures. D.M. Miller shares rights to intellectual property underlying the Multiple Sclerosis Performance Test, currently licensed to Qr8 Health and Biogen. S.M. Planchon reports funding from the Guthy Jackson Charitable Foundation, outside the submitted work. D. Ontaneda reports funding from the NIH, NMSS, Race to Erase, Genentech, Genzyme, and Novartis and consulting fees from Biogen Idec, Genentech, Genzyme, and Merck, outside the submitted work. A. Manzano reports funding from the MS Society (UK), the NIHR, and the Wellcome Trust, outside the submitted work. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.