Patient participation as an integral part of patient-reported outcomes development ensures the representation of the patient voice: a case study from the field of rheumatology

Abstract

Introduction

Patient-reported outcomes (PROs) are important instruments to evaluate healthcare interventions, both in clinical practice and clinical research.

Objective

To describe how representation of the perspective of people with psoriatic arthritis was obtained through active participation on different levels in the development of PROs.

Methods

This case study focuses on the methods of involving patients in the elaboration and validation of the Psoriatic Arthritis Impact of Disease (PsAID) score. We used the concept of the participation ladder and the European League Against Rheumatism (EULAR) recommendations for the involvement of patient representatives in scientific projects to analyse the variety of ways patients participated in this process.

Results

Two patient experts were part of the steering group. 12 patient research partners, coming from 12 different European countries, participated in identifying domains, formulating items for the questionnaire and determining the number of items, the recall period and the questionnaire format. They also helped with the translation of the items into different European languages. Then, 139 patients took part in ranking and prioritising the domains for importance; 65 patients were involved in cognitive debriefing interviews; 499 new patients were recruited for the validation study. Challenges of patient participation in PRO development, such as the representation of patients, are discussed.

Conclusions

Making patient participation an integral part of the PRO development and validation process is an important requisite for outcome research. The variety of patient contributions at different phases in this case study resulted in an instrument with high face validity.

Key messages.

What is already known about this subject?

• Active participation of patients is an essential requirement for developing and validating patient-reported outcome (PROs) measures.

What does this study add?

• Patient participation on multiple levels and in different phases of PRO development is feasible and enhances the representation of the patients’ perspective.

How might this impact on clinical practice?

• Practical examples of patient involvement in an international project are given and will help project leaders to implement patient involvement in comparable projects.

Introduction

Patients are an essential stakeholder in outcome research, in particular when developing patient-reported outcomes (PROs).^{1 2} The potential benefits are measurement instruments that reflect outcomes that are important for patients and that have greater face validity than instruments that have been devised by physicians and often focus on disease symptoms only.³ The use of such PROs is expected to be more acceptable for patients and better meet the challenges of their daily life because they have been developed and tested with patients who have knowledge, perspectives and experiences that are unique.^{4 5}

Despite these potential benefits, patient participation in PRO development is still uncommon.⁶ Many PROs have been devised by health professionals and researchers with little or no participation of patients.⁷ An often used model to describe levels of involvement is the participation ladder (figure 1). This model distinguishes between information, consultation, advise, collaboration and control.⁸ When patients were involved in PRO development, this happened mostly in the passive role as study participant or respondent, a form of patient involvement representing a lower level on the participation ladder, called ‘consultation’.⁹

Figure 1.

Levels of participation in the Psoriatic Arthritis Impact of Disease (PsAID) development process. Reproduced from: Teunissen, T. Values and criteria of people with a chronic illness or disability. Strengthening the voice of their representatives in the health debate and the decision making process [dissertation]. Amsterdam: VU University; 2014.

Reasons for the lack of patient involvement are the scarcity of well-described case studies of how patients can participate in this process and what their contribution should be as well as concerns regarding the representation of the involved patients.^{10 11} A few years ago the European League Against Rheumatism (EULAR) decided to develop patient-derived PROs: the Rheumatoid Arthritis Impact of Disease (RAID) score followed by the Psoriatic Arthritis Impact of Disease (PsAID) score.^12–14 The purpose of these new patient-derived quality of life instruments was to fully capture disease impact from the patient's perspective, and both instruments are recognised by the rheumatology community as examples of participatory research.¹⁵ In these projects, patient research partners (PRPs) were involved, that is, persons with the relevant disease who operated as active research team members on an equal basis with professional researchers, adding the benefit of their experiential knowledge to the research projects.¹⁶ However, the involvement of patients in the development process, its advantages and drawbacks, and its impact, were only briefly alluded to in the main publications.^{13 14}

The aims of this article are to describe and reflect on the role, contributions and representation of patient participants in the PsAID research process and to present the lessons learned.

Case study

Context and general presentation of the PsAID process

Psoriatic arthritis (PsA) is a chronic and heterogeneous disease with a great impact on the individual in terms of reduced quality of life, functional impairment and often loss of work capacity.¹⁷ Traditionally, the assessment of PsA includes few PROs^{18 19} and the scores used were almost all devised by physicians.⁷

In 2011, EULAR initiated the development of the PsAID score.¹⁴ The objective was to develop an outcome measure to assess disease impact on quality of life of people with PsA in clinical trials and clinical practice. The project was led by a steering group comprising, among others, two senior rheumatologists and a patient researcher (coauthors of this paper). In this project the steering group was determined to incorporate the patient's perspective, and followed EULAR recommendations for patient involvement (table 1).¹⁶

Table 1.

Application of the EULAR recommendations¹⁶ in the development of the PsAID

	EULAR recommendations	Application in PsAID
1	Participation of PRPs is strongly recommended for clinical research projects and for the development of recommendations and guidelines, and should be considered for all other research projects	Patient participation has been an essential feature of the project to ensure that the PRO truly reflects the perspective of patients with psoriatic arthritis
2	Participation of PRPs should be considered in all phases of the project to provide experiential knowledge, with the aim of improving the relevance, quality and validity of the research process	Patients with psoriatic arthritis were involved in every phase of the development and validation process. The contribution varied depending on their patient role
3	A minimum of two PRPs should be involved in each project	PRPs were proportionally represented, their number was similar to that of the physician-researchers
4	Identification of potential PRPs should be supported by a clear description of expected contributions	Potential candidates received a comprehensive invitation, explaining in detail what was expected from them
5	The selection process of PRPs should take into account communication skills, motivation and constructive assertiveness in a team setting	PRPs were selected by their treating physicians based on a set of criteria, among which were communication skills, English reading and speaking and ability to travel
6	The principal investigator must facilitate and encourage the contribution of PRPs, and consider their specific needs	Support was provided by premeeting update sessions for patients. Identification of domains took place in a homogeneous patient meeting to guarantee a safe environment. During team meetings patients were encouraged to take part in the discussions and were regularly asked for their opinion
7	The principal investigator must ensure that PRPs receive information and training appropriate to their roles	The steering group provided regular updates and produced patient information packs before every team meeting, including a glossary of terms
8	The contribution of PRPs to projects should be appropriately recognised, including coauthorship when eligible	All PRPs whose participation maintained throughout the project became coauthor of the scientific manuscript

EULAR, European League Against Rheumatism; PRO, patient-reported outcome; PRPs, patient research partners; PsAID, Psoriatic Arthritis Impact of Disease.

The project was carried out by a research team incorporating the steering group members, 12 national principal investigators (who were all treating rheumatologists), one dermatologist, one International Classification of Functioning, Disability and Health expert, one health professional and 12 PRPs.

The project was run in several phases including a first physical meeting to determine the domains of interest, a priority exercise, a preliminary elaboration of the instrument, translations and cognitive interviews, a large validation study and finally a second physical meeting to make the final decisions (figure 2).¹⁴ During this last meeting an informal evaluation took place of the process and results of collaboration between PRPs and researchers in this project.

Figure 2.

The Psoriatic Arthritis Impact of Disease (PsAID) elaboration and validation process.

Design of patient involvement

Patients in different numbers contributed to subsequent phases of the elaboration and validation process, carrying out a variety of tasks and on different levels according to the participation ladder (figure 1).

Control

One patient expert, member of the steering group, diagnosed with PsA and a qualitative researcher (MPTdW), took part in all discussions and decision-making at every stage of the project. Although his primary focus lay on preserving the patient's perspective in the project, the entire research team felt responsible for ensuring that the patient participation was meaningful throughout the study.

Collaboration

Twelve PRPs representing 12 European countries were recruited and selected through the clinics of the participating physician-researchers. It was decided from the start of the project that the number of PRPs would be proportional to that of the physician-researchers.

The PRPs were involved in different phases and performed different roles, some similar to that of the researchers. During the first team meeting patients and physicians were split into two homogeneous break-out groups. The patient group was led by the patient researcher and a nurse/qualitative researcher. The patients were invited to discuss the domains that are important when assessing the impact of their PsA in a focus group-type exercise. While the patient group explored these life-impact domains, the physicians reviewed the pros and cons of available measurement instruments in PsA. In the plenary session of the first team meeting, patients assisted in finding the right wording of the items, the recall period and the questionnaire format, such as a Numeric Rating Scale versus a Likert scale.²⁰ At a later stage they were involved in the translation of items into different European languages, following a strict procedure of forward and backward translation.¹⁴

During the second team meeting, the PRPs had an active role in analysing and interpreting the final outcomes of the validation study.

Advice

The content validity of the instrument was increased through cognitive interviews with 65 patients. Cognitive interviewing is a formal research methodology, where participants are prompted to “think aloud” as they complete the PRO, with interviews taped and rigorously analysed for understanding, retrieval of information, judgement and response options.²¹

Consultation

Ranking and prioritisation of the original 16 life-impact domains, identified by the PRPs during the first team meeting, was carried out by 139 patients, from all the participating countries. During the validation process, 499 other patients were invited to fill in several questionnaires and to undergo a clinical examination (of whom 474 had full data which was analysed) (figure 2). The study participants were again recruited through the 12 participating researchers.

Methodological notions for organising patient participation

Recruitment

Patients were recruited in different roles (table 1). Two patient representatives in the steering group were leading members of the EULAR Standing Committee of People with Arthritis/Rheumatism in Europe (PARE) and an Italian Patient Association. The PRPs were identified and approached by the participating national principal investigators. Criteria were a diagnosis of PsA, being able to read and speak English, and the ability to speak up in a mixed group of patients and professionals.

Finally, patients involved in the ranking and prioritisation exercise, the cognitive interviews and the validation study were randomly selected through health professionals of the 12 academic centres.

Education and support

Apart from full coverage of all travel and accommodation expenses, PRPs received tailor-made support, before, during and after team meetings. The support was geared towards enabling the PRPs to understand the team discussions, feel confident to speak up and to get involved in real dialogue with physicians, and to stay motivated. Having both groups separated during the morning of the first team meeting, helped the patients to speak up freely without feeling limited by the presence of their treating physician or feeling intimidated by other physicians.

PRPs were invited to introduction sessions the evening preceding the two team meetings and received regular project updates and newsletters. Before the second team meeting, PRPs received an 8-page guide including a comprehensive explanation of the validation process, the project time frame and a glossary of terms in lay language.

Finally, the national principal investigators provided individual support to their own PRP when necessary.

Acknowledgement

PRPs who participated in the meetings and reviewed the draft manuscript, are coauthors of the PsAID publication.¹⁴

Reflections on patient involvement

From the start, the steering group wanted to develop a patient-derived PRO score with full involvement of people from the target group. It was evident that only patients could identify life-impact domains that are most important for them. It was less clear whether patients should be involved in other research phases, in particular in the validation and the data analysis. Here the authors report their personal experiences and the challenges of working with PRPs.

Patient participation in identifying domains

The active participation of PRPs during the first meeting was key in the elaboration of this new PRO instrument. We believe that the fact that the patient group discussion in the morning was moderated by a patient representative and a nurse researcher, established a safe atmosphere where participants did not feel restricted to express opinions that they would not discuss in the presence of their own physicians. Many participants indicated that domains such as shame, embarrassment, social isolation and sexual inhibition are often not discussed during consultations, despite the fact that their impact is substantial.¹⁹ Getting the opportunity to discuss these domains in a homogeneous group of fellow patients, enhanced the generation of in-depth knowledge of the impact of PsA on daily life.

Moderating the discussions between physicians and PRPs in the afternoon was challenging. Discussions about overlap, similarities and differences between patient-identified domains and the need to reduce the number of relevant domains to a feasible number for validation revealed the differences in prioritisation between both groups. The PRPs were reluctant to combine domains or to leave out domains that they perceived and experienced as important. For instance, PRPs were strongly in favour of keeping ‘coping’ as an important domain to measure the impact of PsA on daily life although some physicians argued that ‘coping’ does not represent disease impact per se. PRPs experienced the absence of the ability to cope with the disease as an important indicator for disease activity and therefore perceived it to be an appropriate feature of disease impact.

The discussion in the plenary session with the PRPs and the researchers emphasised the importance of having PRPs with a constructive attitude, understanding the research context, and an openness to listen to the arguments of the researchers. From the researchers it was expected that they sincerely listened to the stories of patients without imposing their own ideas. In the end consensus was obtained on 16 domains, including coping, that were taken further in the development process. The PRPs helped subsequently in determining the exact wording of the 16 items, advised about the preferred length of the recall period and the choice of instrument: a numeric rating scale.

Although the 12 PRPs came from 12 different countries, it might be argued whether the participation of this number in one meeting is sufficient to generate a valid principal list of domains. Other qualitative methods of generating domains from a patient's perspective can be applied, such as multiple focus groups, individual interviews or narrative analysis.^22–24

Patient participation in data analysis

During the second meeting, the PRPs were actively involved in the analysis and interpretation of the statistical data of the validation study. They were instrumental in determining the number of items. There was a debate about the items embarrassment and/or shame, social participation and depression. Some physicians emphasised the fact that these items did not add significant value to the overall score of the instrument in terms of psychometric properties.²⁵ Other participants, including many patients, wanted to keep these items because they are important for a minority of patients and should not be lost in the clinical encounter between patient and physician.

Finally a vote was made; as many physician-researchers as well as most of the patients were in favour of keeping more domains in the score, a longer version (with 12 domains) was kept; but as the three additional items only weakly contributed to the pooled result and there was no significant difference in performance of the 9-item or 12-item version, a 9-item version was recommended for clinical trials. This compromise achieved a high level of agreement.¹⁴

Representation

One of the most expressed concerns of researchers towards patient participation is the question to what extent PRPs can represent the entire patient group.²⁶ This case study is presented as a best practice because it shows that it is possible to achieve ‘representation of the patients’ voice’ by organising multiple forms of patient participation. First, the patient members of the steering group were selected for their ability to transcend their personal experience. Their main tasks were to ensure that the steering group considered the patient's perspective in every phase of the research process and that PRPs were adequately supported to participate in the project. Other studies have shown that enabling PRPs is crucial to achieve meaningful participation of patients.^{26 27}

As PsA is a heterogeneous disease with various disease manifestations and morbidity, and because characteristics such as gender, age, cultural backgrounds and socioeconomic strata are important to consider,²⁸ proportional representation of patients in the research team was decided on. Despite the relatively high number of PRPs, it is important to acknowledge that this number can never be fully representative of the full patient's perspective. Similar to clinical research, it was a challenge to select PRPs representing different cultural backgrounds, socioeconomic strata and levels of education.²⁹ However, representativeness in numbers was obtained through the prioritisation and weighting exercise (n=139), the cognitive debriefing interviews (n=65) and the validation study (n=499).

Recruitment

In the present study, the PRPs were selected by their physicians. We feel physicians are best suited to make the judgment which patients are eligible for the patient role to collaborate with professionals in a scientific project. However, it may also be helpful for the physician to consult other health professionals in the clinical care team during this selection process. Another advantage of recruiting PRPs through the clinics rather than patient organisations, is the absence of a “hidden agenda” or of overt patient advocacy.³⁰ Although patient advocacy is helpful and even key for some issues, it may be difficult in the context of a research project.

Equality of perspectives

Patient participation is based on the assumption that the experiential knowledge of patients is complementary to the evidence-based knowledge of physicians.⁹ It was a challenge during the meetings to make sure that all participants felt equally facilitated to contribute to the discussions and to avoid the dominance of strongly opinionated participants. It is known from the literature that PRPs can be influenced by existing hierarchical relationships.³¹ It is therefore essential that moderators are able to deal with power imbalances and have the skills to create optimal conditions for open dialogues.³² In practice this problem occurs between patients and physicians as well as within both groups.³³ In our opinion, power imbalances are a universal feature in all relationships and something every moderator needs to account for. There is overwhelming evidence that the role of the principal investigator is key to enable PRPs to provide a meaningful contribution to research.^{27 34 8} In this case study, the enthusiasm of the steering group and the collaboration with qualitative and clinical researchers were important elements of success.

During an informal evaluation at the end of the second meeting the PRPs confirmed that they believed that they had contributed something that the physicians could not provide. They felt well supported in the process and satisfied about their role.

Discussion

There is growing agreement that active participation of patients in the development of PROs is a moral imperative, as well as beneficial for the entire research process.^{35 36} Involvement of PRPs enriches the discussions and brings in new items that are relevant for the validity of the instrument.^{37 38} We believe that our case study has shown that it is possible to obtain a valid representation of the patient's voice through multiple forms of participation in different phases and on different levels. The variety of patient contributions at different phases profoundly modified the final version of the questionnaire. The process was mutually enriching for PRPs and physicians and resulted in an instrument with high face validity.

Although patient involvement is very much a trend right now, in many outcome studies the patient participation is either extremely limited, or still tokenism.^{1 7 10} Furthermore, practical examples of patient involvement in outcome research are rare.⁶ In the field of rheumatology, one best practice has been published where different patient roles and contributions were described in detail in the process of developing a PRO for fatigue in people with rheumatoid arthritis.³⁹ Patients were involved in pilot interviews by discussing “measurement properties of wording, time-frame and descriptors” and articulating different meanings of words like cope and manage. Their involvement improved the process and the resultant score.

An internal evaluation has shown that the EULAR recommendations for the involvement of patient representatives in scientific projects¹⁶ have been useful for task force leaders developing management recommendations as well as for PRPs involved in these task forces.⁴⁰ In our case study, when incorporating the patients’ perspective and applying these EULAR recommendations, we identified facilitators of which some are similar to those found in an evaluation study of structural involvement of patients in another international organisation, Outcome Measures in Rheumatology (OMERACT)³⁰ and in other studies.²⁶ We have summarised these facilitators in box 1. Most importantly, we perceive patient participation as a process that requires deliberate and constant consideration by the principal investigator and explicit efforts to enable patients to contribute in a meaningful and constructive manner. No universal format exists that fits all research contexts. Therefore participation should be planned and developed for each project individually. Nevertheless, there are some lessons learned that can be transferred to other research contexts. The key role of the leadership is already mentioned. Part of that role is to guarantee that financial resources are available to allow for appropriate compensation of PRPs and time to organise their involvement. In outcome research participation is not an add-on but an integral part of the research process, including careful recruitment, adequate support and a fair acknowledgement of PRPs. A structural approach guarantees sustainable partnerships and increases opportunities for mutual learning and empowerment.

Box 1. Key conditions for effective involvement of patients in patient-reported outcome development.

• Recognition of participation as a process; participation requires an ongoing, direct dialogue between patients and researchers; make patient participation an integral part of the project

• Participation should be tailor made, no concept exists that fits all

• Proportional representation of patients during team meetings as equal collaborators

• Participation requires an additional effort in terms of time, energy and resources

• The role of the principal investigator is key in providing support to enable patients to contribute

• Management of expectations is crucial

• Achieving representation of the patient's perspective requires multiple forms of participation in different phases and on different levels. Apart from having PRPs in the steering group, the input from PRPs may be broadened by adding surveys, interviews, focus group meetings, or Delphi exercises to the research design

• A structural approach guarantees sustainable partnerships between professionals and patients

• Willingness for mutual learning

Our case study has a few limitations. Although we followed the EULAR recommendations for patient involvement,¹⁶ a more formal evaluation among PRPs as well as the participating physicians would be recommendable. Opportunities for involvement in the dissemination and implementation of the PsAID were not explored. Also the number of PRPs for the identification of domains important to patients may be arbitrary. An alternative approach of individual interviews and/or focus groups might be considered in future studies. This would allow to ensure data saturation. Finally, the call for more diversity in the group of PRPs remains a challenge. Further studies will show if the lessons learned in the PsAID process, are applicable to other settings.