Enhancing Neurofibromatosis Clinical Trial Outcome Measures Through Patient Engagement

Abstract

Objective

As part of an evaluation of the Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) International Collaboration patient representative program, we surveyed REiNS members to (1) identify facilitators and barriers to involving patient representatives and (2) understand whether and how involving patient representatives affected recommendations for clinical trial outcomes.

Methods

We administered an anonymous online survey to all REiNS members. Facilitators and barriers to patient representative involvement were solicited using a modified free listing technique; responses were inductively grouped into higher-order categories and ranked based on saliency score (Smith s). Open-ended questions assessed patient representative expectations for engagement, perceived benefits/costs of patient engagement, and patient representative contributions; responses were analyzed using conventional content analysis.

Results

A total of 63/172 (37%) members responded, including 18/30 (60%) patient representatives. Providing sufficient opportunities to meaningfully engage in research tasks and cultivating a respectful, inclusive atmosphere were key facilitators to patient representatives' satisfaction and ability to make an impact. Respondents perceived that patient representatives directly (through their input on research tasks) and indirectly (through effects on other stakeholders' knowledge and communication style) improved the organization's research, leading to selection of more meaningful, relevant, and feasible clinical trial outcome measures. Ongoing challenges to patient engagement include difficulty scheduling meetings and concerns about the level of scientific knowledge patient representatives needed to effectively engage.

Conclusions

Involving patient representatives in REiNS improved perceived quality of neurofibromatosis clinical trial outcome measures. Negotiating sufficient opportunities to engage, fostering an inclusive atmosphere, and navigating time pressures are key to effective patient engagement.

The selection of meaningful, valid outcome measures for clinical trials is particularly important in rare neurologic diseases. Most rare diseases have no approved treatments and a limited number of patients available to participate in clinical trials.¹ Selecting appropriate outcomes measures and standardizing their use across trials is needed to effectively and efficiently provide information about the benefits and harms of new therapies. Engaging patients and other stakeholders in the design, conduct, and dissemination of outcomes research is one avenue to improve clinical trials for rare diseases.^2,3 Patient engagement has been shown to help identify new research domains, select more patient-centered outcomes and instruments, improve participant accrual rates, and shorten regulatory approval.^4-6

The Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) International Collaboration has played a key role in the creation and dissemination of outcome measures for clinical trials of neurofibromatosis type 1 (NF1), neurofibromatosis type 2 (NF2), and schwannomatosis.^7-9 Recognizing the potential benefits of patient engagement on this work, REiNS invited patients with neurofibromatosis (NF) and caregivers to join the organization in 2017. In 2019, we administered an online survey of REiNS members to evaluate this patient representative program. We assessed (1) facilitators and barriers to effectively involving patient representatives in the REiNS collaboration, (2) patient representatives' motivations and expectations for engagement, and (3) outcomes of the patient representative program to date. In doing so, we hope to demonstrate the potential benefits of engaging patients and their caregivers in clinical trial design and provide guidance to other research groups pursuing patient engagement.

Methods

Structure and Activities of the REiNS International Collaboration

REiNS is a volunteer organization open to all NF clinicians and researchers. It has 8 working groups focused on imaging, functional outcomes, visual outcomes, patient-reported outcomes, neurocognitive outcomes, disease biomarkers, cutaneous neurofibromas, and patient representation. Working group activities include reviewing published literature on existing clinical trial outcome measures; administering surveys to obtain NF patient and caregiver input on outcome domains; overseeing experimental studies to validate new outcome measures; and writing consensus recommendations for NF clinical trial design. Previously recommended outcome measures include patient-reported questionnaires assessing pain and physical functioning; clinical measures of vision, hearing, and pulmonary function; and guidelines for radiologic assessment of tumor volume.

Working groups are led by topic-specific experts who direct the agenda and practices of their group. The majority of working group activities are conducted via email and teleconferences, which vary in frequency across groups. Coordination of working group efforts is facilitated by a leadership council and an executive steering committee. REiNS also holds open in-person meetings twice a year. At these meetings, working groups present preliminary findings for collaborative discussion and feedback from the larger NF research community. Meetings include representatives from regulatory, industry, and patient advocacy groups to incorporate wider stakeholder perspectives into REiNS consensus recommendations.

Description of the REiNS Patient Representative Program

Patient representatives were solicited via an open application process in fall 2017 advertised by REiNS members and NF advocacy organizations. Because some patients with NF may be too young or too sick to participate themselves, parents and other caregivers were also considered as patient representatives. Thirty individuals submitted complete applications, representing all 3 forms of NF (17 NF1, 11 NF2, and 2 schwannomatosis). Seventeen representatives (57%) were patients and 13 (43%) were a parent or caregiver. The majority of patient representatives were female (70%), White (94%), non-Hispanic (90%), had a college or graduate degree (90%), and were from the United States (90%). Two-thirds of representatives were employees of or volunteers with 9 NF advocacy organizations.

Patient representatives formally began work with REiNS in December 2017. Patient representatives were assigned to working groups based on their interests and a desire to distribute patient representatives equally among all working groups. All patient representatives were also invited to join the newly formed working group on patient representation, which was created to generate policies and procedures to facilitate patient representative engagement throughout the organization. Patient representative participation in REiNS continues in this manner to date.

Patient Representative Program Evaluation

We conducted a formative evaluation of the REiNS patient representative program. Formative evaluations are designed to provide feedback about programs as they are implemented to identify unforeseen obstacles and potential remedies.¹⁰ As the first step in our evaluation, we administered an anonymous online survey of all REiNS members. The survey was designed and analyzed by a volunteer subgroup of REiNS researchers, clinicians, and patient representatives.

Survey Design

Two versions of the survey were created: one for patient representatives and one for all other stakeholders (defined as clinicians, researchers, and representatives from nonprofit patient advocacy organizations). All participants were asked to identify their working groups, complete free listing exercises on facilitators and barriers to patient representative engagement (i.e., things that helped or made it harder to participate in or contribute to REiNS), and describe a specific contribution of patient representatives to REiNS. REiNS patient representatives also were asked to complete a free listing exercise on reasons they joined the organization and to reflect on how the experience had or had not fulfilled their expectations. Other stakeholders also were asked to assess the cost/benefit ratio of patient engagement (using a 5-point Likert scale and an open-ended explanation). The survey was administered using REDCap, a secure online data collection platform.¹¹

Participant Recruitment

As our primary goal was to solicit broad input on the patient representative program, all recent REiNS members were eligible to participate in the survey, regardless of their frequency of participation in REiNS activities (which varies widely). Eligible participants were identified using publicly available data on the organization's website, including an overall membership list updated in November 2017 and working group membership lists updated in spring/summer 2019.⁹ This process identified 184 potential participants; 172 participants (including all 30 patient representatives) had active email addresses and were included in the sample. These participants were affiliated with more than 70 institutions/organizations across at least 8 countries (the United States, United Kingdom, Australia, France, Italy, Germany, Austria, and Argentina). An invitation to complete the survey was emailed to all eligible participants by the chair of the REiNS leadership council in September 2019, with a follow-up reminder email 2 weeks later.

Data Analysis: Free Listing

In free listing, a qualitative method used in anthropology to elicit cultural domains, respondents are asked to list as many answers to a question as they can think of.^12-14 This method allows researchers to solicit the full range of examples of a specified category and compare group members' perceptions of that category.^12,15 In our study, we adapted the free listing technique to elicit participants' perceptions of motivations, facilitators, and barriers to patient representative engagement. Participant responses were inductively coded into larger categories by 2 analysts (a patient representative and a researcher), who then collaboratively discussed code definitions and coding application until reaching consensus on all items.

Free lists were analyzed using Anthropic 4.983/X software. Given the unique position of patient representatives to comment on the program and the small sample size of their cohort compared to other stakeholders, we analyzed patient representatives' free lists separately from those of other stakeholders to highlight their input. For both cohorts, we calculated the saliency scores (Smith s) of coded responses to each question independently. This statistic accounts for the frequency with which a code is mentioned across respondents' free lists and the rank order of codes within each respondent's free list.¹⁵ Smith s ranges from 0 (not salient; no group members endorse concept) to 1 (highly salient; all group members endorse concept as first item on free list). To achieve accurate weighting of ranks during analysis, only the first instance of each code was retained within each respondent's free list.

Data Analysis: Content Analysis

Conventional content analysis, a qualitative research method used to categorize and synthesize text, was used to analyze open-ended survey questions.¹⁶ Inductive codes were generated and iteratively refined for each open-ended question separately by one analyst (a researcher). Coding examples were then reviewed with a second analyst (a patient representative) to ensure patient representative responses were interpreted accurately and enhance credibility of the analysis.¹⁷ Codes for each open-ended question were then aggregated into larger themes that summarized patterns in the responses to each question. Evidence supporting each theme is presented in quotations, which are identified by participant ID number and the code PR for patient representatives and OS for other stakeholders. Potentially identifying information has been redacted and replaced with descriptions in square brackets.

Standard Protocol Approvals, Registrations, and Patient Consents

The Partners Human Research Committee approved all study procedures. Participants read a short fact sheet about the study and indicated their consent to participate before completing the survey.

Data Availability

This was an anonymous survey, but some respondents included potentially identifying details in their responses. As such, the full source data for this study cannot be publicly released, in order to maintain participant confidentiality.

Results

Sixty-three of 172 REiNS members responded to our survey (37% response rate), with representation from every REiNS working group. Respondents included 18/30 patient representatives (60% response rate) and 45/142 other stakeholders (32% response rate).

Patient Engagement Facilitators and Barriers

Patient representatives and other stakeholders largely endorsed the presence of similar types of facilitators (table 1) and barriers (table 2) to patient representative engagement. For this reason, a single set of codes and coding definitions was applied across responses from both cohorts. Facilitators and barriers reflected patient representative characteristics (e.g., patient representatives' prior scientific knowledge or family support), characteristics of the engagement process (e.g., sense of inclusion and feeling valued), and structural supports (e.g., availability of training, funding, and accessibility services). Of note, the same topic was sometimes raised as both a facilitator and a barrier by different respondents or in reference to different working groups. In these cases, barriers were given parallel code names (marked with ^a in table 2) to highlight the conceptual overlap with related facilitators.

Table 1.

Facilitators to Patient Representative Involvement

Table 2.

Barriers to Patient Representative Involvement

Top engagement facilitators identified by patient representatives were creating an inclusive atmosphere (s = 0.260) and lived experience (as patients, caregivers, or NF organization volunteers), which gave them insight into the real-world issues of patients with NF (s = 0.229). Top facilitators identified by other stakeholders were providing sufficient opportunities to engage in meetings and research tasks (s = 0.389) and making sure that patient representatives felt valued for their contributions (s = 0.198). Patient representatives and other stakeholders both identified time availability as a major barrier to engagement (s = 0.382 and s = 0.212, respectively). Difficulties included inconvenient timing of meetings due to differences in members' time zones and preferred meeting times. Respondents also reported it could be difficult for patient representatives to arrange to attend meetings if the meetings were held at inconsistent times month to month or were set with limited advanced notice. This concept was related to another major barrier reported by patient representatives: limited bandwidth to engage in REiNS activities due to other commitments to their family (including caring for a child with NF) and the NF community (including volunteer work).

Comparing facilitators and barriers revealed additional issues that were salient to patient representatives and other stakeholders but differed in how they were expressed. Patient representatives' prior scientific knowledge and experience was highly salient for both cohorts. However, patient representatives focused on the presence of scientific knowledge as a strength (#3 facilitator); other stakeholders focused on how lack of scientific knowledge was a challenge, particularly regarding participation in technical or jargon-filled conversations (#1 barrier). Similarly, while other stakeholders lauded the multiple opportunities to engage via teleconferences, in-person meetings, and email exchanges (#1 facilitator), patient representatives highlighted uneven access to engagement opportunities across working groups and a desire for even greater involvement in clinical trial protocols (#2 barrier).

Patient Representatives' Motivations and Expectations for Engagement

Patient Representatives' Reasons for Engaging

Patient representatives endorsed 4 main reasons for joining REiNS. Patient representatives desired to advocate on behalf of the needs and interests of patients with NF (advocacy, s = 0.407); be involved in and help advance NF research (research, s = 0.378); ensure patients with NF, their family members, and advocacy organizations have a voice in the research process (representation, s = 0.374); and learn more about NF or NF research (education, s = 0.313). The prominence of these 4 factors in the scree plot of saliency scores (figure 1), with a visual elbow down to less frequent responses, suggests a group consensus that these motivations were of primary importance.

Figure 1. Patient Representatives' Reasons for Engaging.

Scree plot of coded free listing responses describing why patient representatives joined the Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) research collaboration. The y-axis is calculated saliency score (Smith s) for each code and the x-axis is codes presented in ranked order by saliency score.

Fulfillment of Patient Representatives' Expectations

Patient representatives were asked to describe in what way their experience in REiNS had or had not fulfilled their expectations (table 3). Responses revealed substantial variability in patient representatives' experiences, largely based on the primary working group to which they were assigned (theme 1). This variability was evident in the emotional tone of comments (roughly equally mixed among positive, negative, and neutral/mixed experiences) and was explicitly referenced by some respondents. For example, as one patient representative noted, “That feeling of working as a team is a very important one and definitely feels stronger in certain groups, while other groups give off more a vibe of exclusivity where more meetings are held without patient reps.” (PR42)

Table 3.

Content Analysis of Open-Ended Questions

Whether patient representatives' expectations were fulfilled was largely determined by 3 additive factors, which built upon one another to enhance participant satisfaction (theme 2) (figure 2). These factors were (1) having sufficient, meaningful opportunities to participate in the working group; (2) sense of inclusion within the working group (i.e., whether one feels like a respected member of the team whose input is desired and valued); and (3) whether the patient representative felt he or she was making a difference to the working group and, by extension, to NF clinical trials and patients with NF at large. Another patient representative highlighted the importance of these factors in achieving a fulfilling experience:

All of the researchers/clinicians I have dealt with have been respectful of my opinions, as have the other patient reps I have talked to and met with. I have been able to add to the conversations about what is important in my various groups and many of my suggestions have been taken into account. The brainstorming that has gone on in meetings is both stimulating and invigorating and. . . [seeing] how the group as a whole wants to increase patient input makes me feel personally that NF research is geared towards the patient as a whole, and that we are not just a body with a specific disorder, but people whose ideas and needs are respected in a humanistic way. The fact that we are listened to as an equal member of the team and are not just for show is very important. (PR71)

Figure 2. Factors Affecting Whether Patient Representatives' Expectations for Engagement Were Fulfilled.

Graphic depicting the 3 major contributors to fulfilling patient representatives' expectations for their role (as derived from content analysis of survey responses) and the resulting effect of achieving these goals on patient representatives' satisfaction. Having sufficient opportunities to engage was the foundation for building an inclusive climate, which facilitated patient representatives' ability to have an effect on research.

Patient representatives also discussed the consequences of unfilled expectations (theme 3). There was some tolerance of less fulfilling experiences due to the expectation that REiNS was still developing the patient engagement program, expressed by one representative as “I believe REiNS is totally in a startup phase and so therefore is evolving and I have not had a lot of expectations just for that very reason” [PR59]. However, other patient representatives with less satisfying experiences noted feeling disappointed and frustrated, which could lead to reduced levels of participation or drop-out from the organization. For example, one patient representative reported:

This was unfortunately a highly underwhelming experience. I expected to have more guidance in the beginning. I also expected better leadership from the start . . . with how disorganized my section seemed to be, it made me not want to participate and ultimately withdraw. (PR45)

Effect of Patient Engagement

In response to the statement “The benefits of patient representatives in REiNS outweigh the effort/cost,” 47% of other stakeholders strongly agreed, 40% agreed, 7% were neutral, 2% disagreed, and 2% strongly disagreed. In accompanying free-text comments, respondents largely endorsed that engaging patient representatives was vital to the organization's work. Respondents highlighted the ways that patient representatives' unique perspective affected REiNS processes and research outputs (theme 4). For example, some respondents reflected that patient representatives had insights into patients’ needs and challenges that clinicians and researchers might miss. As one researcher explained:

Patient representatives provide a unique perspective in conversations of choosing outcome measures. They are able to report from their own experiences what is important as an actual patient or parent of a patient who is living with NF. It is critical that outcomes measure what we believe is important clinically, but also reflect the needs and reality of patients with the disorder. (OS61)

Some respondents also described difficulties with and potential limitations of patient engagement (theme 5). Respondents noted that effective patient engagement takes additional time and resources, and some doubted that patient representatives could help with more technical research tasks (such as meta-analyses). One clinician noted communication barriers that may slow down meetings:

It takes time and effort to listen to nonprofessionals as sometimes they do not understand the issues, they take a long time to express a medical concern, some of their remarks are not to the point—but as the whole point is to find a better way to serve the patients, we need to understand what they worry about most and what their issues are.” (OS30)

Another clinician-researcher commented on patient representatives' potentially limited ability to participate in all committee tasks, and suggested changing the group's engagement strategies to address these limitations:

Patient representatives provide a valuable perspective that enhances research. I think the role is likely limited, because much of the day to day work is likely difficult for them to weigh in on effectively, but I think it would help to have each committee define specific ways for patient representatives to be involved (e.g., quarterly phone calls specific to that purpose, meeting at the beginning of a new measure/strategy review to determine patient priorities, etc.). (OS62)

As both quotes demonstrate, even respondents who noted potential difficulties with patient engagement still believed that efforts to engage patients were warranted.

Respondents' comments reflected 2 pathways by which patient representatives' input affected REiNS research: direct performance of research tasks and indirect effect on other stakeholders (figure 3). Patient representatives directly assisted with research tasks (for example, by giving feedback on study design or analysis) (theme 6). For example, one working group developed a survey of patients with NF to help identify important outcome domains, clarify goals for defining treatment response, and reveal potential barriers to clinical trial participation. One patient representative described contributions to the project as follows:

I was involved from the beginning by helping design the questionnaire, was able to give input on dissemination, worked on assessing/analyzing the results and ultimately helped disseminate the information . . . I was asked to deliver the results of a survey to the REiNS group. I subsequently delivered the same presentation at a regional symposium near my home.” (PR04)

Figure 3. Patient Representative Contributions to and Effect on Clinical Trial Outcomes.

Graphic depicting how and why patient representatives were perceived to have improved Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) recommendations for neurofibromatosis clinical trial outcomes, as derived from content analysis of evaluation survey responses.

A clinician-researcher in the same working group expanded on the effect these activities had on the group's research:

Patient reps have been critical for the [REiNS working group] surveys. Questions have been added/removed, words/phrases were modified, and formats reconfigured based on their feedback. Also, the interpretation of the survey data is different from the patient rep perspective. The presentation of the [REiNS topic] survey data by a patient rep was powerful, because it reinforced the concerns and needs of the patient population. The results would not have seemed as honest or heartfelt coming from a researcher. (OS20)

Not all patient representatives described such extensive involvement, but these 2 respondents' experiences highlight the broad continuum of activities to which patient representatives contributed, from study design and analysis to dissemination of results internally to REiNS members and externally to patients with NF and their families.

Patient representatives also indirectly improved research by affecting how other stakeholders thought and behaved (theme 7). For example, one researcher described how having patient representatives on the team improved communication:

I think that [patient representative] participation directly changes the way clinicians/researchers work together. You are more focused on things that closely influence the patients' quality of life and . . . discussing in lay language makes sure that everyone, including different cultures/languages and centers, understand what is being said.” (OS35)

Another clinician-researcher discussed the way patient representatives' comments could lead to a more holistic consideration of patients' needs:

Patient representatives give us clarity that the objectives we are aiming to meet are relevant to the population(s) we serve. Patient representatives provide a wealth of information and highlight areas of need that cut across disciplines, making us as researchers and clinicians think about the bigger picture.” (OS58)

Respondents commented on how they perceived the patient representative program had a positive effect on REiNS recommendations for clinical trial outcomes measures, response end points, and other trial design considerations (theme 8). Patient representatives contributed by “highlighting the most important topics for the given patient population.” (OS18) Respondents perceived that this type of input helped REiNS assess the domains that were most relevant to patients’ daily functioning and overall quality of life. Patient representatives also contributed by giving helpful feedback “regarding the burden of questionnaires expected for parents to fill out during clinical trials” (OS28) and “the feasibility of the use of outcomes for future trial design” (OS58). Respondents also believed that patient representatives helped select patient-reported outcome measures and cognitive tests that patients would understand and thus more accurately respond to. For example, respondents noted that patient representatives could give “feedback about which measures seemed more relevant to [their] condition and were easier to understand” (OS19) and identify when patients “might have difficulty with certain items.” (OS47) Overall, respondents believed these collective benefits of patient engagement would allow NF clinical trials that eventually use REiNS recommended end points to provide more meaningful results to patients and clinicians about the value of different treatments. As one researcher stated, “Including persons who actually experience the consequences of disease and disorder is a crucial component to determining what 'works' and what doesn't because personal judgments of functioning and perception are critical components of successful or failed treatment.” (OS37)

Discussion

We evaluated the REiNS patient representative program to understand the preliminary effect of patient engagement on NF clinical trials research and factors that facilitate this effect. Our data show that providing sufficient opportunities to meaningfully engage in research tasks and fostering an inclusive atmosphere in which patient representative contributions are used and appreciated were key to (1) providing fulfilling engagement experiences and (2) maximizing the effect of patient engagement on research outcomes. Despite some challenges (including the additional time and resources required for patient engagement), the patient representative program was widely perceived by respondents to be critical to improving REiNS recommendations. Respondents perceived that patient representative involvement helped REiNS assess outcome domains most relevant to patients’ functioning and quality of life and select outcome measures that would be easier to implement in clinical trials. These findings add to a growing body of literature suggesting that patient engagement leads to more relevant and actionable research findings.^18-20

Patient engagement in research has been recognized as a practical, ethical, and political imperative^21-24 and is increasingly required by health research funders worldwide.^25-27 As a result, more research collaboratives are likely to engage patient representatives and should proactively address key challenges to effective patient engagement. We observed significant difficulty scheduling meetings (which may amplify existing scheduling challenges inherent to an international collaboration functioning across many time zones). Setting clear expectations in advance about when meetings will be held, having rotating meeting times, and flexibility from clinicians/researchers to meet outside standard business hours may mitigate this barrier. Clinicians and researchers also shared concerns that patient representatives' lack of scientific knowledge or difficulty participating in jargon-filled conversations may limit their contributions. Similar concerns have been noted in other collaborations,²⁸ suggesting that teams should explicitly consider what kind of input from patient representatives would be helpful for their projects. Using jargon-free, plain language during meetings may improve communication across diverse teams, facilitating not only patient representatives' comprehension and participation, but also that of clinicians and researchers coming from multiple specialties and countries.

Based on published literature and our evaluation experience, we also recommend that other research collaboratives starting patient engagement programs think about program evaluation early.^29-31 Setting a timeline with defined targets for patient representative involvement can help focus program planning. Early evaluation ensures that barriers can be addressed as the program is rolled out, minimizing participant dissatisfaction (which may lead to disengagement) and maximizing patient representatives' ability to contribute to the research. Patient representatives and other stakeholders may identify different key facilitators and barriers to program roll-out, highlighting the need to include all relevant parties in the evaluation process. Co-designed and consensus-based evaluations may take more time to implement but may be more effective at highlighting different parties' blind spots, leading to the development of more comprehensive program improvement strategies.

Program improvement is a dynamic process requiring ongoing monitoring of program effectiveness as potential improvements are tested and adapted. A focus on practical, efficient methods of evaluation (such as brief surveys) may assist in this effort.^32,33 Translating evaluation data into actionable steps for improvement is key to maximizing patient representative impact. In the REiNS collaboration, this is being achieved by a multiphase process. Evaluation findings were presented at the in-person REiNS meeting in December 2019, and attendees participated in structured small-group discussions to brainstorm potential program improvements. After getting additional feedback from the REiNS patient representation working group, the evaluation team translated suggestions into tangible action items and prioritized them for implementation. One priority was increasing the number of and diversity within our patient representative cohort; as of February 2021, 51 more patient representatives joined REiNS, increasing diversity in race, education level, and country of residence in our cohort. A formal assessment of this improvement process and its success in increasing patient representative satisfaction and impact is planned for future publication.

Our study has some limitations. The overall response rate for our survey was only 37%. Nonrespondents may have different views on the patient engagement program and as such, these qualitative data may not represent all barriers, facilitators, or effects of patient engagement in REiNS or their precise quantitative frequency/ranks. We administered a brief survey for feasibility, but this limited the amount of information we could collect; in-depth interviews may be helpful to confirm our explanations of patient representative satisfaction and contributions. Finally, our assessment of the effect of patient engagement is limited to REiNS members' perceptions of how recommended clinical trial outcomes measures will perform. Future research should assess the actual performance of recommended measures as they are incorporated into NF clinical trials.

Our results demonstrate that engaging patient representatives in clinical trial outcomes research is feasible and was perceived to improve trial design recommendations. In the future, we hope to improve our impact on clinical trials by involving a more diverse group of patient representatives, reflective of the full range of people with NF, and expanding opportunities for patient representative across all working groups. Further exploration of the differences in facilitators and barriers to effective engagement noted by patient representatives and other stakeholders, including how best to navigate conflicting perspectives, is warranted. Further research is also needed to understand the best methods to engage patients with rare diseases in research³⁴ and to develop rigorous evaluation tools to support this process.^35,36

Glossary

NF

neurofibromatosis

NF1

neurofibromatosis type 1

NF2

neurofibromatosis type 2

REiNS

Response Evaluation in Neurofibromatosis and Schwannomatosis

Appendix. Authors

Study Funding

Writing of this manuscript was supported by the Department of Veterans Affairs Office of Academic Affiliations Advanced Fellowship Program in Health Services Research, the Center for Healthcare Organization and Implementation Research (CHOIR), and Edith Nourse Rogers Memorial Veterans Hospital, and the Intramural Research Program of the NIH. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government.

Disclosure

The authors report no financial disclosures relevant to this manuscript. Go to Neurology.org/N for full disclosures.