A Systematic Review of Approaches for Engaging Patients for Research on Rare Diseases

ABSTRACT

BACKGROUND

Patients with rare diseases have limited access to useful information to guide treatment decisions. Engagement of patients and other stakeholders in clinical research may help to ensure that research efforts in rare diseases address relevant clinical questions and patient-centered health outcomes. Rare disease organizations may provide an effective means to facilitate patient engagement in research. However, the effectiveness of patient-engagement approaches, particularly for the study of rare diseases, has not been well studied.

OBJECTIVES

To synthesize evidence about engagement of patients and other stakeholders in research on rare diseases, including the role of rare disease organizations in facilitating patient-centered research.

METHODS/RESEARCH DESIGN

A systematic review and gray literature search were guided by a technical expert panel composed of patient representatives, clinicians, and researchers. English-language studies that engaged patients or other stakeholders in research on rare diseases or evaluated engagement were included. Studies were assessed on how well key research questions were answered, based on the level of detail describing engagement activities and whether outcomes from engagement were assessed.

RESULTS

Thirty-five studies were included, although many reported minimal information on engagement. Patients and other stakeholders were most commonly engaged to identify patient-centered research agendas, to select which study outcomes were important to patients, to provide input on study design, and to identify strategies for increasing enrollment in trials. Rare disease organizations mainly helped provide access to patients and communicated research opportunities and findings. They also helped promote collaborative networks and provided financial support for research infrastructures. Although authors reported benefits of engagement and identified changes to their research processes, no empirical assessments of engagement practices and their effectiveness were found.

CONCLUSIONS

Researchers studying rare diseases can obtain patient input regarding which research questions and health outcomes to study; however, the most effective approaches to engagement have not been well defined.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-014-2895-9) contains supplementary material, which is available to authorized users.

INTRODUCTION

Patients with rare diseases and their caregivers often have limited evidence-based information to guide decisions about management and symptom relief.¹ Patients are increasingly interested in and calling for a more proactive role as partners in clinical research.^2–4 Engaging patients and other relevant healthcare stakeholders (e.g., patients, caregivers such as parents of children with rare diseases, advocacy organizations, and clinicians)⁵ in the planning and conduct of research (as distinct from increased patient involvement in clinical care⁶) is a promising approach for addressing evidence gaps for the management of rare diseases. Engagement may promote research that evaluates health outcomes that are both relevant to patients with rare diseases⁷ and useful for decision making.^8–12 Engagement is feasible for many populations and research phases,¹³ and can involve patients as consultants, as collaborators, or as leaders.^13,14 Further, given their interest in advancing research, rare disease organizations¹⁵ could help facilitate patient engagement in clinical research beyond efforts by the research community.

We undertook a systematic literature review to assess the extent and nature of involvement of patients with rare diseases, their caregivers, and relevant organizations in research initiatives. Five key questions guided the review:

1. For what purposes (goals and stages of the research process) have patients and other relevant health care stakeholders been engaged in research on rare diseases?

2. In what ways have they been engaged? How were patients and other stakeholders identified and recruited for engagement? What approaches were used to obtain their input?

3. How does engagement affect the design, conduct, dissemination, and relevance of the research to patients with rare diseases?

4. What is the role of patient organizations in bringing researchers and patients together?

5. What are the challenges for engagement in research on rare diseases?

METHODS

Stakeholder Involvement

The review was guided by a multi-stakeholder technical expert panel (TEP) composed of patient representatives, clinicians, and researchers recruited through the informal professional networks of the authors. The TEP provided input on the key research questions, the definition of rare disease, the search strategy, key gray literature references, interpretation of study findings, and recommendations for future research.

Electronic Database Search

Eligibility Criteria

We included original studies of any design published in English, in which patients, caregivers, or other stakeholders participated in planning or conducting biomedical or health services research related to rare diseases. Studies available only in abstract form were excluded due to insufficient information. Studies were considered to be related to rare diseases if the focus was a condition considered to be rare according to lists maintained by the National Institutes of Health¹⁶ or the National Organization of Rare Disorders,¹⁵ or if the authors stated that their research focused on a rare disease. Research engagement was defined by active involvement in the planning or conduct of research (e.g., providing input on a research agenda, contributing to study design, identifying study comparators or outcomes, monitoring study progress, interpreting results, or disseminating findings).¹⁷ Enrolling patients only as passive participants (subjects) or actively involving them in clinical care were not considered research engagement. Non-original or summarized literature was used to identify additional studies.

Search strategy

The overall search strategy was developed by a reference librarian and methodologists with expertise in conducting systematic reviews to capture both clinical studies on rare disease(s) that engaged patients or other stakeholders and studies that evaluated engagement. This review covers the origin of the biomedical data bases through September 2013, searching: PubMed/Ovid MEDLINE, Ovid EMBASE; Ovid PsycInfo, Ovid Cochrane (Sys Rev, Methods, HTA), EBSCO, CINAHL, SCOPUS (social sciences content), Web of Science (multidisciplinary scientific content), and Business Search Premier, Academic Search Premier and Google Scholar. To identify studies relevant to research engagement, medical subject heading (MeSH) terms and text words were selected based on common indexing practices. We included, among others, terms related to patients and other stakeholders (e.g., “patient*,” consumer*, stakeholder*, etc.), engagement (e.g., “participa*,” “collaborat*,”, “partner*,” etc.) (See online Appendix for full search strategy). These engagement research search terms were compiled and tested repeatedly to enhance search sensitivity and identify all potentially relevant publications. Due to the lack of indexed search terms for rare diseases, as well as the large number and heterogeneity of rare diseases, the relevant literature on research engagement was hand-screened by the authors for relevance. For studies published prior to 2011, we capitalized on a previous systematic review by three of the investigators (HM, ZW, TE) concerning research engagement broadly, although not specific to rare diseases,⁹ that utilized the same database search strategy with respect to engagement in research. The reference list for this original review was screened to identify studies related to rare diseases. We also screened reference lists from eligible studies, used the PubMed ‘related articles’ feature to identify eligible studies, reviewed conference proceedings, and used SciSearch for publications that cited potentially eligible studies.

Gray Literature Search

To identify relevant resources not included in bibliographic databases, we used search engines Scirus and Sciverse (which contain scientific journal content, scientists’ homepages, courseware, preprint server material, patents, and institutional repository and Website information), Google, and Bing, in addition to other websites recommended by the TEP.

Study Selection

We organized initial references in citation files using Endnote software and removed duplicates. Titles and abstracts were screened independently by two reviewers against eligibility criteria using the DistillerSR software (Evidence Partners Incorporated, Ontario, Canada). All studies judged relevant by either reviewer were included for full text review. We conducted full text screening of potentially eligible studies in duplicate using a similar procedure. If both reviewers judged a study to be relevant, it was included. Disagreements in full text screening were reconciled through arbitration by a third reviewer (MH).

Data Extraction

Data were extracted from studies with a standardized form developed using DistillerSR. Data were extracted by one investigator and reviewed for accuracy by a second. The data extraction form was tested on a sample of included studies (n = 5) to assure consistent data extraction. We extracted the following data: study description, stage(s) of research in which there was engagement, purpose of research engagement, methods for identifying and recruiting patients/stakeholders for research engagement and for obtaining their input, roles of rare disease organizations, perceived challenges to engagement, and any reported outcomes of patient engagement.

Analysis

The included studies were assessed by two reviewers (with disagreements arbitrated by a third reviewer) regarding the extent to which information answering the key research questions was provided. Studies were classified based on whether or not engagement activities were described with sufficient detail and clarity for others to replicate the approach (‘sufficiently descriptive’ vs. ‘minimally descriptive,’ respectively). Studies were also classified based on whether or not there was a direct method for measuring and examining the effect of the engagement processes on the design and conduct of the study (evaluative vs. non-evaluative studies). Findings were synthesized by key research question.

RESULTS

The systematic review and gray literature search identified 35 relevant studies (Figure 1). All studies were observational; none employed experimental or quasi-experimental designs. Eleven studies used qualitative or survey methods to obtain input from patients,^4,7,18–26 and 17 studies were narrative reports describing one group’s experiences with engagement.^27–43 The remaining seven studies were descriptions of specific initiatives reported on websites^44–50 (Table 1 and Table 2). The studies were mostly conducted in North America and Europe, and most focused on a specific rare disease rather than rare diseases in general.^{7,18–21,23–31,35–38,40–42,47–50} Thirty studies reported on engagement conducted by the author(s),^{7,18–21,23–33,35–42,45–50} three described engagement conducted by others based on surveys or key informant interviews,^4,22,43 and two broadly described contributions of patient organizations.^34,44 Nineteen reported on engagement of patients,^{7,18,20,21,23,25–27,32,33,35,38,39,43,45,47–50} 18 reported on engaging patient organizations,^{4,19,22,27–33,35,37–39,41–43,46} 13 reported engaging parents or other caregivers,^{19,23,24,26,32,35,36,38–40,43,47,50} and five reported engaging clinicians.^{20,38,39,47,49} Seven studies were classified as sufficiently descriptive^{7,18,19,23–26} (see Box 1 for example of a sufficiently descriptive study). No studies were classified as evaluative, because none formally assessed outcomes related to engagement.

1. For what purposes (goals, stages of the research process) have patients and other stakeholders been engaged in research on rare diseases?

Figure 1.

Study screening and selection.

Table 1.

Characteristics of Included Studies

	Included Studies (N = 35)
	N	(%)
Study type
Narrative report	17	(49 %)
Website describing a specific initiative	7	(20 %)
Qualitative study	6	(17 %)
Survey study	5	(14 %)
Study location
United States (US)	15	(43 %)
Europe	14	(40 %)
Other	4	(11 %)
Australia	2	(6 %)
Rare disease focus
Specific condition*	25	(71 %)
General	10	(29 %)
Stakeholders engaged†
Patients	19	(54 %)
Rare disease organizations	17	(49 %)
Parents or family members	13	(37 %)
Clinicians	5	(14 %)
Description of engagement activities
Minimally descriptive	28	(80 %)
Sufficiently descriptive for others to replicate	7	(20 %)
Evaluation of the effects of engagement ††
Non-evaluative studies	35	(100 %)
Evaluative studies	0	(0 %)

*studies focused on: achalasia, neuromuscular disorders (n = 3), girls with abnormally tall stature, pulmonary arterial hypertension, cystic fibrosis, juvenile neuronal ceroid lipofuscinosis, phenylketonuria (PKU), cerebral palsy, vitiligo, teenagers and young adults with cancer, Paget disease, lupus, Li-Fraumeni Syndrome, pseudoxanthoma elasticum, fibromyalgia, tuberculosis, Parkinson’s disease, spinal muscular atrophy, pachyonychia, urea cycle disorders, degenerative ataxias, von Hippel-Lindau, and hemophilia

† Categories are not mutually exclusive; two studies^{2, 44} described the role of rare-disease organizations more broadly

†† Whether or not there was a direct method for measuring and examining the effect of the processes of engagement on the study design or impact of the study findings

Table 2.

Detailed Characteristics of Individual Studies

Study	Study type	Rare disease	Engaged stakeholder(s)	Stage of research for which engagement is described	Purpose of engagement
Bedgood, 200727	Narrative report	Achalasia	Patients, disease-specific patient organization (ASG)	Execution Stage: Study recruitment	To access a larger patient population for developing and conducting a survey
Boon, 201028	Narrative report	Rare neuromuscular diseases	Disease-specific patient organization (Vereniging Spierziekten Nederland [VSN])	Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures	To stimulate research into neuromuscular diseases
Bruinsma, 201129	Narrative report	Girls with abnormally tall stature	Disease-specific patient organization (Tall Girls Inc)	Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures; Execution Stage: Study recruitment	To help determine project goals and provide input on study questions
Carroll, 201218	Qualitative study	Pulmonary arterial hypertension (PAH)	Patients	Execution Stage: Study design & procedures	To examine the motivations for and concerns regarding participation in randomized clinical trials of therapies in order to facilitate enrollment in future trials
Cystic Fibrosis Foundation, Therapeutics Development Network47	Website describing a specific initiative	Cystic fibrosis	Patients, families, clinicians	Execution Stage: Study design & procedures, Study recruitment	NR
de Blieck, 201330	Narrative report	Juvenile neuronal ceroid lipofuscinosis (JNCL).	Disease-specific patient organization (Batten Disease Support and Research Association [BDSRA])	Preparatory Stage: Research funding; Execution Stage: Study recruitment	To provide financial support and assist with recruitment (for a disease registry)
DeWard, 201331	Narrative report	Phenylketonuria (PKU)	Patient organizations	Execution Stage: Study recruitment	To improve recruitment and retention in clinical trials.
Edwards, 201119	Qualitative study	Cerebral palsy	Disease-specific organization, Parents	Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures	To identify patient priorities and produce an RCT which was feasible, acceptable, based on parents’ experiences, and responsive to their needs and expectations
Eleftheriadou, 201220	Survey study	Vitiligo	Patients, clinicians	Preparatory Stage: Agenda setting	To identify disease core set of outcomes to be studied in trials
EMA (European Medicines Committees), Fifth report on the interaction with patients' and consumers' organizations (2011)33	Narrative report	General	Patients, patient organizations	Preparatory Stage: Research funding; Execution Stage: Study design & procedures; Translational Stage: Dissemination	To provide input on procedures concerning rare disease designations
EMA (European Medicines Committee), The role of patients as members of the EMA Human Scientific Committees32	Narrative report	General	Patients, parents/ Family/Friends, patient organizations	Preparatory Stage: Agenda setting; Translational Stage: Dissemination; Translational Stage: Evaluation	To advise and support regulators on the feasibility of planned investigations (e.g., for pediatric investigation, rare disease designation, risk management plan, etc.), and to guarantee that scientific opinions address patient needs and that there is a rational and adequate use of incentives (e.g., in rare disease designation) for the benefit of patients
EUPATI, EUPATI (European Patients’ Academy on Therapeutic Innovation)44	Website describing a specific initiative	General	NA	Execution Stage: Study recruitment; Translational Stage: Dissemination	NA (To address training issues and significantly improve the availability of both patient-centric information for the public as well as educated patient experts who have the capacity and capability to contribute to medical research & development)
The European Organisation for Rare Diseases (EURORDIS), Activity Report 2012 & Work plan 201334	Narrative report	General	NA	ERUORDIS encourages engagement at these stages: Preparatory Stage: Agenda setting; Preparatory Stage: Research funding, Execution Stage: Data collection; Translational Stage: Dissemination	NA
Fern, 201321	Qualitative study	Teenagers and young adults with cancer (including rare cancers)	Patients	Execution Stage: Study design & procedures; Execution Stage: Study recruitment; Execution Stage: Data collection; Translational Stage: Evaluation	To ensure that a future survey (developed through the current qualitative study) will be a true reflection of young people’s experience and that the methods of administration will be acceptable to young people in order to enhance study enrollment
Landy, 20124	Survey study	Rare genetic disorders	Non-specific patient organizations	Preparatory Stage: Research funding; agenda setting; Execution stage: study design & procedures; recruitment, data collection; Translational stage: dissemination	NR
Langston, 200535	Narrative report	Paget Disease	Disease-specific organization, Patients, caregivers	Preparatory stage: agenda setting; Execution stage: study design & procedures; Execution stage: study recruitment; Translational stage: dissemination*	To aid the design, conduct, and delivery of a clinical trial
The Lupus Research Institute, A patient’s voice in lupus research48	Website describing a specific initiative	Lupus	Patients	Preparatory Stage: Research funding	To bring the unique patient perspective to the forefront of proposal review
Mai, 201236	Narrative report	Li-Fraumeni Syndrome	Parents/Family	Preparatory Stage: Agenda setting	To bring together researchers, clinicians, and individuals from patient families to review the state of science, identify research priorities, and establish a multi-center research collaboration
Marshall, 200437	Narrative report	Pseudoxanthoma elasticum (PXE)	Disease-specific patient organization (PXE International)	Execution stage: data collection; Translation stage: dissemination	NR
Mavris, 201222	Cross-sectional survey	General	Non-disease-specific patient organization (EURORDIS), other patient organizations	Preparatory Stage: Research funding; Execution stage: study design & procedures; recruitment; data collection	NA (describes findings of a survey intended to document the role of patient organizations in rare disease research)
McCormack, 201338	Narrative report	Neuromuscular disorders	Patients, parents, patient organizations, clinicians, legal academics, sociologists, and ethicists.	Execution stage: study design & procedures	To provide a “strategic steer” on ethical issues related to the study of neuromuscular disorders
Mease, 20077	Survey study	Fibromyalgia	Patients	Preparatory Stage: Agenda Setting	To develop consensus regarding the outcomes to be assessed in clinical trials and longitudinal studies
Molster, 201239	Narrative report	General	Patients, parents, patient support groups, clinicians, funders, industry, policy makers, regulators	Preparatory Stage: Agenda setting	To get patients' input relevant to the development of a national plan for rare diseases, including the national research agenda
Nierse, 201023	Qualitative study	Non-curable neuromuscular diseases	Patients, parents	Preparatory Stage: Agenda setting	To identify patient priorities for research on neuromuscular disorders and achieve a shared research agenda among patients, clinicians and researchers
Ntshanga, 201040	Narrative report	Tuberculosis	Community members, community leaders, religious leaders, traditional healers, school principals, community health workers	Preparatory Stage: Agenda setting; Translational Stage: Dissemination	To conduct research that matches local needs and circumstances, and to enhance community participation through open communication with researchers
Orphanet, Researchers' wish-list for rare disease research in Europe45	Website describing a specific initiative	General	Patients	Preparatory Stage: Agenda setting	NR
The Parkinson Pipeline Project49	Website describing a specific initiative	Parkinson’s Disease	Patients, clinicians treating Parkinson’s Disease patients	Preparatory Stage: Agenda setting; Translational Stage: Dissemination; Translational Stage: Evaluation	To provide the patient perspective in the treatment development process
Rare disease matters, Patient-initiated research foundations46	Website describing specific initiatives	General	Patient organizations	Preparatory Stage: Research funding	NR
Rothwell, 201324	Qualitative study	Spinal muscular atrophy	Parents	Execution Stage: Study design & procedures	To assess parents' attitudes and opinions toward a pilot newborn screen study for spinal muscular atrophy
Schwartz, 201341	Narrative report	Pachyonychia	Disease-specific patient organization (PC Project)	Preparatory Stage: Agenda setting; Preparatory Stage: Research funding; Execution Stage: Study recruitment; Translational Stage: Dissemination	To help connect researchers with patients to increase enrollment in studies
Seminara, 201042	Narrative report	Urea Cycle Disorders	Non-disease-specific patient organization (Coalition of Patient Advocacy Groups), Disease-specific group (National Urea Cycle Disorders Foundation)	Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures; Execution Stage: Study recruitment; Execution Stage: data collection; Translational Stage: Dissemination	NR
Serrano-Aguilar, 200925	Survey study	Degenerative ataxias; majority (67.9 %) suffered Friedreich’s ataxia	Patients	Preparatory Stage: Agenda setting	To incorporate needs and expectations of patients in the design of this systematic review
von Hippel-Lindau Alliance (VHLA)50	Website describing specific initiatives	von Hippel-Lindau	Patients, families	Preparatory Stage: Agenda Setting; Execution Stage: Study design & procedures, recruitment, data collection	To create a patient-driven registry that is unlike databases in which limited information is entered by clinicians; it contains a complete “picture” of each patient with insights and observations only a patient can give
Workman, Engaging patients in information sharing and data collection: the role of patient-powered registries and research networks prepared for AHRQ43	Narrative Report	General	Patients, families, and patient organizations	Preparatory Stage: Agenda setting; Execution Stage: Study design & procedures; Execution Stage: Study recruitment; Execution Stage: Data collection; Translational Stage: Dissemination	NA
Wu, 201226	Qualitative study	Hemophilia	Patients, Parents	Execution Stage: Study design & procedures	To explore the need for potential new items of the Canadian Hemophilia Outcomes-Kids Life Assessment Tool (CHO-KLAT)

NR = Not reported, such that information was relevant to the included study, but was not described in the publication/website; NA = Not applicable, such that the information was not relevant for the specific type of study included. *Engagement in dissemination is planned, but the project has not reached this stage as of publication

Text Box 1.

Illustrative Example of a Study that Was Sufficiently Descriptive with Respect to Research Engagement

The purposes of engagement included identifying patient-centered research topics or agendas,^{19,23,29,36,39} identifying outcomes important to patients or developing measures relevant to patients’ needs,^{7,19,20,23,25,26} increasing recruitment or enrollment through development of patient-centered study designs,^{4,18,21,24,27,30,31,41} and incorporating the patient perspective into study design^{19,25,35,39,40} (Table 2, Table 3). Patients and other stakeholders were most commonly engaged in the preparatory stage (n = 19 for agenda setting)^{4,7,19,20,23,25,28,29,32,35,36,39–43,45,49,50} and study execution (n = 15 for study design and procedures,^{4,18,19,21,24,26,28,29,33,35,38,42,47,50,51} n = 12 for recruitment,^{4,21,27,29–31,35,41–43,47,50} and n = 6 for data collection^{4,21,37,42,43,50}). Eleven studies reported engagement for research translation, with a greater focus on dissemination (n = 10)^{4,32,33,35,37,40–43,49} than on evaluation of findings (n = 3)^21,32,49 or implementation (n = 0). Nineteen studies^{4,19,21,22,28–30,32–35,37,40–44,49,50} described engagement at multiple stages, particularly those reporting on the role of patient organizations.^{33,35,41–43}

• Key Question 2:

  In what ways have patients and other stakeholders been engaged in research on rare diseases?

Table 3.

Engagement Summary: Stage, Purpose, Method, and Role of Rare Disease Organizations

	Studies Reporting (Total = 35)*
Stage of research in which engagement occurred
Preparatory	22
Execution	19
Translational	10
Purpose of engagement
Identifying patient-centered research topics or agendas	5
Selecting outcomes and measures	6
Increasing recruitment, enrollment, or retention	7
Incorporating patient perspective into study design	5
How patients and other stakeholders were engaged
Providing input through workshops, focus groups, or Delphi	11
methods
Developing study interview guides, study materials, websites	4
Including patients on governing or advisory committees	5
Reviewing research findings	2
Sharing study findings	6
Role of rare disease organizations
Providing access to or recruiting patients for engagement and as study subjects	9
Communicating research opportunities or findings	5
Providing input on study design or procedures	6
Initiating clinical trials or other studies of rare diseases	4
Funding of infrastructure (e.g., registries) or training grants	3
Initiating collaborative research networks	3
Providing training opportunities	2

* Categories are not mutually exclusive; no studies were classified as “evaluative”

Identifying Patients and Other Stakeholders for Engagement in Research

Eight studies reported information on how patients and other stakeholders were identified for engagement^{18,19,23–25,27,35,48} (Table 4 [online]). All of these reported convenience sampling without an explicit selection process. Recruitment occurred through patient organizations,^19,23,27,35 clinics,¹⁸ self-referral on the internet,⁴⁸ or a recruiting agency.²⁴ No studies engaging patient organizations described how collaborations between research teams and these organizations were initiated.

Approaches for Patient and Other Stakeholder Input

Eleven studies referred to engagement activities based on consultation with patients and stakeholders. Obtaining input through workshops, focus groups, interviews, or Delphi methods was common.^{7,18–20,23–26,36,39,45} One study obtained patient feedback on informational materials that would be presented to study subjects.¹⁹ Thirteen studies described more collaborative involvement in at least one stage of the research process; for example, including patients on governing or advisory committees,^{32,33,38,42,48} in developing study interview guides, study materials and websites;^21,35,52 in collecting data (conducting interviews),^4,21 in reviewing research findings,^21,28 and in disseminating findings.^{4,33,37,40–42} Stakeholder-driven approaches were reported; included here are examples of patients or advocacy organizations initiating new studies^{4,29,37,41,42,46} and controlling the research process by setting the research agenda, having responsibility for data collection, and disseminating findings (e.g., Patient-Powered Registries⁴³). Conversely, in four studies, patient organizations were engaged only for providing access to their members.^27,30,31,46 Four studies reported that patients or other stakeholders were prepared for engagement in research, ranging from an informational video²⁴ to in-person workshops;^21,40,48 none reported training researchers in engagement methods.

• Key Question 3:

  How does engagement affect the design, conduct, dissemination, and relevance of the research?

Four studies clearly articulated how they used input from patient engagement to inform their work, including identifying research topics for future study,²³ the study design to use,¹⁹ the outcomes to assess,^7,19 and the domains to include on a patient-reported outcome measure²⁶ (Table 3, Table 5 [online]). Five studies used either qualitative or survey methods to obtain information directly from patients about the outcomes that were important to them.^{7,19,23,25,26} These reports permitted rankings of symptoms by importance^7,25 and also clarified how patients describe symptoms.^7,19,23 For example, due to patient input, these studies included measures of pain, stiffness, sleep, motor function, quality of life, and the economic impact of disease.

No studies empirically evaluated the effect of engagement on the design, conduct, dissemination, or relevance of the research. However, many studies reported perceived impact of engagement that was not measured and could not be confirmed. Some perceived benefits relative to other studies that did not use engagement were practical, such as improved access to patients or improved recruitment methods,^{4,19,29,30,35,42,43} improved study retention,¹⁹ more in-depth responses in interviews (when using patients to conduct the interviews),²¹ less time to complete the project,⁴ and incorporation of a broader range of outcome measures to inform future risk-benefit assessments for investigational treatments.⁴⁹ Other perceived benefits were less tangible, such as increased relevance of the project,^4,29,35 broader dissemination of findings,³⁵ improved acceptance of research findings among end-users,²⁹ greater mutual respect between scientists and patients,³² and increased transparency of and public confidence in the research.^32,33 Only one study articulated the benefits of engagement for the engaged partners, including increased awareness of the disease studied and greater access to health information.³⁵

• Key Question 4:

  What is the role of rare disease organizations in bringing researchers and patients together?

Nineteen studies reported on engagement of patient organizations.^{19,22,27–33,35,37–39,41–43,46} However, no studies formally evaluated the role of patient organizations in connecting researchers and patients. Studies frequently acknowledged the role of rare disease organizations in providing access to or recruiting patients both for engagement and as study subjects,^{19,22,29–31,35,41} and in enhancing communication about research participation opportunities and research findings (e.g., through social media, websites, conferences, and newsletters).^31,35,36,41 In some cases, representatives from these organizations were directly involved in the research process, serving on advisory and regulatory committees^28,42 or providing input on study protocols,^19,22,35 for example. Some studies reported that patient organizations initiated clinical trials or other studies of rare diseases.^28,29,41

Several studies also highlighted broader support provided by rare disease organizations. For example, these groups initiated the organization of collaborative research networks,^22,28,41 provided financial support for research infrastructure such as registries,^22,30,43,46 supported training activities,^22,34,44 and developed policy statements regarding engaging patients in research.³⁴

• Key Question 5:

  What are the challenges for engagement in research on rare diseases?

The studies included in this review provide insights into the process of research engagement. First, four of the studies stated that engagement requires substantial time and resources from both researchers and those who are engaged as partners.^26,29,32,35 One author notes that scientific groups face challenges in finding time to obtain and use patient input,³² and another author notes that patient organizations have limited resources to devote to research engagement versus other priority activities.³⁵ Second, two studies stated that engaged partners must overcome barriers to engagement such as logistical issues (e.g., access to clinics) and health problems that may prevent participation.^18,31 Third, several authors expressed concerns about possible bias among patients and patient representatives chosen for engagement due to higher motivation, high educational levels, methods of recruitment (e.g., via internet or annual meetings) or possible conflicts of interest (e.g., relationships between patient organizations and industry groups).^{23,25,27,30,33,43} Other challenges identified include lack of researcher and patient experience in or training for engagement,^25,32 an absence of policies facilitating engagement,²⁵ the potential for both researchers and patient partners to undervalue engagement,²⁵ ethical concerns (e.g., patient rights⁴³), and the possibility of uncoordinated efforts by competing patient-led groups.⁴³

DISCUSSION

Given the diversity of conditions and limited research resources associated with rare diseases, engagement of patients and other stakeholders in research has the potential to facilitate the study of relevant clinical questions and patient-centered health outcomes for rare disease populations. In fact, these issues were among the most commonly reported reasons for engaging patients or their caregivers. In contrast, the studies generally did not address the role of engagement for dissemination of research findings, even though engagement of patients and patient organizations in sharing research results may increase the reach of findings, which is particularly important in the context of treating rare diseases.

Most studies reported a consultative approach to engagement (i.e., unidirectional communication from patients to researchers with unknown influence on the research). We expect that collaborative approaches (i.e., bidirectional communication, shared responsibility for decision-making) may be more effective for advancing the study of rare disease, given the diversity of patient experiences. Although other researchers have reported similar expectations based on their experiences with research engagement in other settings (not specific to rare diseases), definitive evidence to support this is not yet available.^9,53 Very few studies reported training researchers, patients, or other stakeholders for engagement. Other literature supports training as increasing the impact of engagement; while some training opportunities are available for patients⁴³ and researchers,⁵⁴ these are not specific to research on rare diseases.

This systematic review suggests that rare disease organizations play multiple roles in facilitating engagement of patients and other stakeholders in research. Nearly half of the identified studies involved patient organizations; these organizations met critical needs, including facilitating access to patients who may be too geographically dispersed or difficult to identify for participation in clinical trials. Patient organizations also provided broad support for patient-centered research in rare diseases, including wide communication about research opportunities and findings, financial support, formation of collaborative research networks, and creation of policies regarding engagement.

Many studies reported the effects of engagement, but none measured these effects in ways that permit estimation of the strength of the impact of the engagement efforts. The purported benefits included both practical effects (e.g., incorporation of a broader range of outcome measures, increased study enrollment) and intangible effects (e.g., increased relevance of the research, greater public trust). However, consistent with gaps in the broader literature on engagement overall,⁵¹ no studies evaluated the effect of engagement on the inclusion of patient-centered outcomes, the impact of the research findings, or their relevance for clinical decision-making. This lack of empirical evidence precludes drawing conclusions about the impact of engagement in research on rare diseases. This gap in the literature can be addressed in future studies that directly measure the impact of engagement on health outcomes.

This review identified additional evidence gaps, including the lack of standard reporting guidelines for information about engagement of patients and other stakeholders.¹³ In addition, very little is known about the characteristics of those engaged, the approaches for collaboration with these individuals or groups, or the impact of these characteristics on the research process or outcomes. Connecting researchers with engaged partners is a challenge. Although researchers can learn from successful engagement approaches for the study of more common conditions (e.g., HIV),^55,56 more information about how to identify and recruit patient and stakeholder partners would allow for both shared learning and greater transparency regarding possible biases.

Several initiatives may help to fill the evidence gaps. First, the Patient-Centered Outcomes Research Institute (PCORI)⁵⁷ funds studies that address methods for improving engagement, for evaluating the impact of engagement on research outcomes, and for assuring that study questions and outcomes are meaningful to patients.⁵⁸ Evidence regarding best practices for research engagement will also be generated by the Agency for Healthcare Research and Quality’s Community Forum project.⁵⁹ Two initiatives will support infrastructure for patient-centered approaches to rare disease research, including PCORI’s National Patient-Centered Clinical Research Network⁶⁰ and the National Institutes of Health’s Global Rare Diseases Patient Registry Data Repository.⁹ PCORI’s Advisory Panel on Rare Diseases may also provide guidance regarding promising approaches for fostering collaborative approaches to research and involving relevant patient organizations.⁶¹.

This review was limited by the small number of peer-reviewed studies that reported on research engagement in the study of rare diseases and the lack of detail about engagement. The findings from this review could be supplemented by interviews and/or surveys with a range of rare disease organizations to ensure comprehensive capture of research engagement activities. Our assessment of the effects of engagement relied heavily on the perceptions of the researchers who conducted the included studies, which have not been empirically validated. Moreover, this review could not fully capture the wide variation in research resources and infrastructure or the engagement capacity of patient groups across the spectrum of rare diseases. The studies included in this review provide limited insight into whether the successful approaches are generalizable to other rare disease organizations. Future research should examine differences in approaches to, and the impacts of, engagement for diverse types of stakeholders (e.g., children with rare diseases compared to their parents) and assess the authenticity (i.e., the extent to which patient and stakeholder input was actually incorporated into decision making) of approaches to engagement, as the impacts of engagement may depend on the quality and genuineness (i.e., the sincerity) of engagement interactions.

This systematic review found that patient engagement for guiding research on rare diseases tended to use a limited number of multi-purpose approaches. The published studies provide examples of approaches that could be adapted and further evaluated by other organizations, and also highlight the particularly important role of patient organizations in fostering engagement in research on rare diseases. While no insurmountable barriers were identified in this review, there were meaningful challenges. Building an evidence base that documents how different approaches to patient engagement facilitate or hinder research will inform guidance on how best to make clinical research on rare diseases more useful.