Patient-reported outcomes in neurofibromatosis and schwannomatosis clinical trials

Abstract

Objectives:

Neurofibromatosis (NF) is a genetic disease with multiple clinical manifestations that can significantly impact quality of life (QOL). Clinical trials should include patient-reported outcomes (PROs) as endpoints to assess treatment effects on various aspects of QOL, but there is no consensus on the selection and use of such measures in NF. This article describes the PRO Working Group of the Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) Collaboration, its main goals, methods for identifying appropriate PRO measures for NF clinical trials, and recommendations for assessing pain intensity.

Methods:

The REiNS PRO group selected core endpoint domains important to assess in NF. The members developed criteria to rate PRO measures, including patient characteristics, psychometric properties, and feasibility, and utilized a systematic process to evaluate PROs for NF clinical trials. Within the subdomain of pain intensity, the group reviewed the Numerical Rating Scale-11 (NRS-11), the Visual Analogue Scale, and the Faces Pain Scale-Revised using this process.

Results:

Based on the review criteria, each of these pain intensity scales is brief, reliable, valid, and widely used. However, the NRS-11 was given the highest rating for use in NF clinical trials due to recommendations from pain experts and other consensus groups, its extensive use in research, strong psychometric data including sensitivity to change, and excellent feasibility in ages ≥8 years.

Conclusions:

The systematic review criteria and process are effective for identifying appropriate PRO measures and provide information utilized by the REiNS Collaboration to achieve consensus regarding PROs in NF clinical trials.

Neurofibromatosis (NF) is an umbrella term for 3 different neurogenetic diseases: neurofibromatosis type 1 (NF1), neurofibromatosis type 2 (NF2), and schwannomatosis, which share some features¹ and predispose patients to multiple nerve sheath tumors.² These diseases each have their own distinct clinical manifestations, including chronic pain, large tumors, bone abnormalities, skin disorders, hearing problems, and learning disabilities,^2–4 all of which can negatively affect quality of life (QOL). Clinical trials of new treatments for NF manifestations are critical to reduce the morbidity of these diseases and improve QOL. Outcome measures that can be used as response endpoints are important for assessing the impact of treatments on clinical manifestations and everyday functioning. This article will focus on patient-reported outcomes (PROs) and the process of achieving international consensus regarding their use in NF clinical trials.

The term “PRO” was suggested by the US Food and Drug Administration (FDA) to include “any report of the status of a patient's health condition that comes directly from the patient.”^5(p2) PROs are based on the patient's subjective experience, or, if necessary, from others on their behalf (e.g., parent proxy for young children). PROs assess different domains, such as general or disease-specific QOL or condition-specific symptoms (table 1).⁶

Table 1.

Different types of patient-reported outcomes (PROs)

PROs are beneficial on several different levels.⁷ In research, PROs are valuable because they add a unique source of information that is not addressed by primary medical outcomes of a clinical trial.⁸ PROs provide data about the positive or negative effects of a treatment or intervention, such as a reduction in symptoms (e.g., pain) or the development of toxicities (e.g., nausea). PROs also are useful for identifying patients' clinical needs,⁹ assessing population health, and determining public policy.⁷ The inclusion of PROs as trial endpoints is supported by the US FDA and European regulatory agencies, which led to the development of an international group to harmonize criteria regarding the use of PROs across countries¹⁰ and an FDA document providing guidance for utilizing PROs in drug approval and labeling claims.⁵

Clinical trials for the treatment of tumor manifestations of NF, such as plexiform neurofibromas,³ have only started in the past decade. Due to the location, size, and invasive nature of NF-related tumors, complete surgical resection is often difficult and has limited success, demonstrating the need for additional treatment modalities.¹¹ Furthermore, most of these tumors are benign and slow-growing, indicating that endpoints other than tumor shrinkage or survival, such as those assessing clinical and functional changes, are essential. In particular, PROs are useful in trials for conditions that are disabling and chronic like NF, where instead of a cure, symptom reduction and improved functioning and QOL currently are the main treatment goals.¹² Finally, the FDA supports the use of PROs in NF clinical trials, especially for assessing changes in symptoms such as pain (personal communication, S. Plotkin and B. Widemann, March 12, 2012). Thus, PROs are important endpoints in clinical trials for NF-related manifestations.

Despite this need, there are challenges to including PROs in trials for individuals with NF. Specifically, few PRO measures have been developed or validated for use with the NF population, newly developed disease-specific scales do not yet assess children, and some general QOL measures do not target all the domains that need to be assessed in NF, such as cognitive function. Typically, measures designed for other chronic illnesses have been utilized in NF studies to date. However, PRO measures need to be reliable and valid within the specific populations under study.⁵ Furthermore, trials to evaluate treatments for NF manifestations may include a wide age range of participants, from children through adults. Very few PRO measures assess individuals throughout the lifespan, and methodologic problems arise with using separate measures for different age groups. In addition, limited child self-report forms exist despite consensus about the importance of assessing PROs in children.¹³ Finally, the inclusion of PROs in clinical trials is complicated by the additional burden on patients and staff to complete these assessments, the perception by some that PROs are a “less important” outcome measure, lack of familiarity with these types of measures and data, and limited resources to support PRO studies. For these reasons, achieving consensus regarding the use of PROs in NF clinical trials is difficult.

To address the multiple challenges of using PRO measures as endpoints in NF clinical trials, the PRO Working Group was formed as part of the Response Evaluation in Neurofibromatosis and Schwannomatosis (REiNS) International Collaboration. This article describes the REiNS PRO group, its main goals, and the development of its systematic process for review of PRO measures, using the pain intensity subdomain as an example. This information will illustrate the group's rigorous efforts to identify appropriate PRO measures and methodologies for use in NF clinical trials and support the current and future consensus recommendations offered by the REiNS Collaboration.

METHODS

The REiNS Collaboration was formed to address the need for appropriate, standard, and consensus endpoints in clinical trials for individuals with NF (see Plotkin et al., this supplement). The PRO Working Group is one of several REiNS subcommittees charged with identifying outcome measures to use as NF trial endpoints. Currently the PRO group has 12 active participants consisting of professionals from various disciplines who work with individuals with NF, including psychologists, physicians, a nurse practitioner, a genetic counselor, a clinical research coordinator, and a patient advocate from around the United States and the United Kingdom.

Goals of the REiNS PRO group.

The initial phone and Web conference of the PRO group was in August 2010. The members agreed that our main goals were to 1) identify core endpoint domains relevant to NF clinical trials, 2) select a pool of PRO measures assessing these domains, 3) develop a systematic and scientifically sound process for reviewing these measures, and 4) provide methodologic guidelines regarding the use of PROs in NF clinical trials.

Core PRO endpoint domains.

The group researched, discussed, and generated 4 core endpoint domains important to assess in the NF population as part of a clinical trial: 1) pain, 2) functional disability, 3) disease-specific QOL, and 4) general QOL. Some of these domains are comprised of various subdomains. For example, the assessment of pain includes measures of pain intensity, pain interference, and pain behavior.

Development of the PRO rating form.

Since there are numerous PRO measures assessing these various endpoint domains, the group established a systematic method for reviewing, rating, and recommending measures for use as NF clinical trial endpoints. The group generated a list of criteria that are important to consider when choosing outcome measures. For guidance, members reviewed publications describing criteria that have been used by other groups¹⁴ and the FDA.⁵ The group leader also talked to members of the Childhood Oncology Group who are involved in similar tasks (P. Hinds, personal communication, November 8, 2010) so that our procedures would be consistent with the methodologies used by other PRO working groups.

Based on this information, the group generated a rating form to identify PRO measures for NF clinical trials, named the PRO-RATE (Patient-Reported Outcomes Rating and Acceptance Tool for Endpoints). The criteria and information considered when rating an outcome measure on a scale of 0 to 3 are listed in table 2. The systematic process, outlined in table 3, involves nominating measures, reviewing selected measures using the structured rating criteria, and reaching a consensus regarding each measure's suitability for use in NF clinical trials. It is important to note that this is a dynamic process by which the group may re-review measures and update guidelines based on newly developed scales and additional published data.

Table 2.

PRO-RATE rating and scoring criteria

Table 3.

Full review and rating process of PRO measures

RESULTS

To date, the PRO group has completed 18 reviews of outcome measures in the pain and functional disability domains. To provide an example of the systematic process developed for rating PRO measures, this article presents the results of our group's critical review of outcome measures in the pain intensity subdomain.

Based on information gathered about various pain intensity measures through member nominations and literature reviews, including consensus articles from other pain working groups,^15,16 we chose to conduct full reviews of the Numerical Rating Scale-11 (NRS-11),¹⁷ the Visual Analogue Scale (VAS),¹⁸ and the Faces Pain Scale-Revised (FPS-R; figure).¹⁹ As an example of our rating system, the NRS-11 rating is discussed in detail and the final group ratings of the 3 measures are compared in table 4.

Table 4.

PRO-RATE final group ratings for pain intensity measures

Figure. Three measures of pain intensity.

(A) Numerical Rating Scale-11 (NRS-11). (B) Visual Analogue Scale (VAS). (C) Faces Pain Scale-Revised (FPS-R). This Faces Pain Scale-Revised has been reproduced with permission of the International Association for the Study of Pain® (IASP). The figure may not be reproduced for any other purpose without permission. Copyright of the FPS-R is held by the IASP ©2001 (www.iasp-pain.org/FPS-R). Hicks CL, von Baeyer CL, Spafford P, van Korlaar I, Goodenough B. The Faces Pain Scale – Revised: Toward a common metric in pediatric pain measurement. Pain 2001; 93:173-183.

NRS-11.

The NRS-11 is a 1-item measure consisting of a horizontal line with numbers from 0 to 10 spaced equidistant along the line to rate pain intensity.¹⁷ Respondents are asked to circle the 1 number that best represents their pain. The wording on the anchors varies, but typically 0 represents no pain and 10 represents very much pain or the worst pain the patient can imagine. The time frame can vary as well, so respondents may be asked to rate their current pain or pain during the past week.

Patient characteristics.

The NRS-11 can be administered reliably to individuals ages 8 years and older,^20,21 although recent studies have shown support for use of the measure with children as young as 6 years.²² It has been used with a variety of patient populations, including those with acute postoperative pain²³ and chronic pain from cancer,^5,24 fibromyalgia,²⁵ and complex regional pain syndrome.²⁶ In addition, the NRS-11 is reliable and valid with the elderly and with individuals with cognitive impairment.^27,28 It also is used clinically in many outpatient and inpatient medical settings. However, no normative data exist for the NF population.

Use in published studies.

The NRS-11 has been used extensively in published studies, sometimes as a stand-alone measure²⁴ and sometimes embedded within a comprehensive pain scale such as the Brief Pain Inventory (BPI).²⁹ The NRS-11 has been utilized as an outcome measure in clinical trials, and adult and pediatric pain experts have recommended it as a measure of pain intensity in research for individuals ages 8 years and older.^15,21,30 This item of the BPI also meets the PRO recommendations put forth by the FDA.³¹ However, the NRS-11 has not yet been used in published studies with individuals with NF.

Domains assessed/item content.

The NRS-11 measures the construct of pain intensity only, which is an important construct to consider in individuals with chronic pain, including those with NF. The FPS-R ratings may be influenced somewhat by affective responses to the facial expressions.

Scores available.

The score obtained on the NRS-11 is a single integer number between 0 and 10. Some debate exists about whether scores should be interpreted as interval or ratio data, since the difference between each integer may or may not be equal.³² Researchers also have pointed out that the meaning associated with a particular number may vary between patients, (i.e., one person's rating of 10 may mean something different than another person's 10).²¹ While this is an important consideration for cross-sectional studies, it is not an issue when assessing pain longitudinally within patients in clinical trials.

Psychometric data.

The NRS-11 shows excellent test-retest reliability in children³³ and adults,³⁴ including individuals who are illiterate.³⁵ Correlations with other pain intensity measures, such as the FPS-R²⁰ and VAS,²¹ support the construct validity of the NRS-11. Additionally, the tool shows good sensitivity to change over time in both pediatric and adult studies^33,34 and may be more responsive than the VAS and FPS-R.³⁶ The FPS-R has only 6 response choices, which may be limited further by some of the more painful facial expressions that are less appropriate for chronic pain populations. Concerns about reliability and validity have been noted with respect to the need for standardized instructions³⁷ and clinically meaningful change in children,³⁸ particularly in those younger than 8 years or those who may have NF-related learning or attention deficits. Additional preadministration screening may be required to ensure they understand the quantitative numbering of the NRS-11.

Feasibility.

The NRS-11 is a free, publicly available measure that takes less than 1 minute to administer and score. It is well accepted in children through adults, is easily integrated into clinic settings, and can be administered verbally, including over the telephone.³³ Some studies have found better compliance from patients using the NRS-11 compared to the VAS.³⁷ Also, the VAS line length may become distorted by faxing or printing, possibly affecting ratings, and scoring requires the extra step of measuring with a ruler. While pediatric studies suggest that younger children prefer the FPS-R,²⁰ more adolescents and adults seem to prefer the NRS-11.^20,36 Furthermore, the NRS-11 has been validated in many languages and cultures and is easily translatable.^22,35

Overall impressions.

The NRS-11 is a reliable and valid measure of pain intensity for ages 8 years and older that has been utilized in numerous studies, including as a primary outcome measure, and has been recommended for clinical trials by consensus groups and pain experts.^15,21,30 Further, its feasibility and ease of use make the NRS-11 a good option for NF clinical trials. Additional research is needed to evaluate its use in young children and to assess the effect of different scale anchors and administration instructions.³⁸

Among the measures of pain intensity reviewed by the PRO group, the NRS-11 received the highest overall rating (table 4). All 3 scales are reliable and valid tools and could be used to assess pain intensity in various populations. However, our group's current consensus is that the NRS-11 is the most appropriate scale of pain intensity for use as a primary outcome measure in clinical trials for NF in ages ≥8 years. Depending on the study, it also may be important to assess other subdomains such as pain interference and pain behavior due to the multifaceted nature of chronic pain.¹⁶

DISCUSSION

Within the REiNS Collaboration, the PRO group is leading the effort to systematically examine PROs and issue guidance regarding the core endpoint domains, the criteria that PRO measures should meet for use as an endpoint, and the selection of measures appropriate for NF clinical trials. Our multidisciplinary group determined that the most important PRO endpoint domains for NF clinical trials are pain, functional ability, disease-specific QOL, and general QOL. Further, our group developed a systematic process for reviewing and rating PRO measures. The information generated by our working group is utilized by the REiNS International Collaboration to achieve consensus recommendations on the use of PROs in NF clinical trials, which will be disseminated to the NF research community.

Our extensive review of pain intensity measures has led us to suggest that the NRS-11 be used to assess this endpoint domain in NF trials. According to the PRO-RATE criteria, it is a well-researched scale for ages 8 years and older that is reliable, valid, simple, and feasible. There are advantages and disadvantages to using any measure, but the group agreed that the pros of the NRS-11 outweigh the cons, particularly for multicenter NF clinical trials enrolling a wide age range of patients. Other pain intensity scales the group reviewed may be acceptable for specific studies, for example, if the age ranges are limited to young children or if interval data are preferred for testing a certain hypothesis. Additional measures may need to be administered along with the NRS-11, such as a pain interference scale or a body diagram to assess pain location. It is important to note that any measure by itself does not guarantee a reliable and valid PRO assessment. Sound study design, appropriate administration, and proper analysis and interpretation are as crucial as the measure selected and should be carefully addressed in NF clinical trials to ensure appropriate PRO data.

Despite our best efforts, there are limitations to our system and recommendations. We are a small unfunded group with time constraints from other professional responsibilities. Although the group thoroughly searched the literature, the members may not have identified every PRO measure that might be applicable to NF clinical trials or every paper about each measure. Even when identified measures are rated highly, they still may need to be validated in the NF population or have other limitations. Outcomes of our work should be considered as guidelines for establishing REiNS consensus recommendations to improve the comparability of clinical trials.

A final goal of the PRO subcommittee is to provide expertise and generate recommendations regarding the methodology of using PROs in NF clinical trials. To obtain valid data, it is critical to develop the PRO objectives early in the protocol design, select the most appropriate PRO measures that fit the study population and objectives, train staff in PRO administration, and use appropriate data analysis procedures. Currently, some members are working with other REiNS groups as well as the NF Consortium to provide expertise regarding PRO methodology. For example, several PRO group members worked with the visual outcomes subcommittee to help select a scale for measuring vision-related QOL (see Fisher et al., this supplement), other REiNS groups adapted the PRO-RATE form for reviewing outcome measures in their domains, and PROs have been included in several NF Consortium clinical trials.

The PRO group's future plans include reviewing and rating measures in the remaining core endpoint domains. The members also await further information on new scales that were previously reviewed before setting final guidelines. Subsequently, the group will focus on PRO measures for children younger than 8 years and explore possible electronic assessment approaches. Members also plan to conduct validation studies with NF samples. Finally, to disseminate the REiNS PRO consensus recommendations to the research community, the group aims to publish a series of papers and post the final recommendations on the REiNS Web site www.reinscollaboration.org.

CONCLUSIONS

In patients with a chronic medical condition like NF, endpoints assessing clinical effects are useful for trials aimed at reducing tumor size or improving other disease complications. Thus, reliable, valid, and feasible PRO measures that are appropriate for use with individuals with NF are sorely needed. The systematic review process developed by the REiNS PRO group has proven to be effective. The group will continue to identify the most appropriate PRO measures for individuals of all ages with NF who may be enrolled in future clinical trials using this process, develop sound methodologies for use of these important endpoints, and disseminate consensus recommendations to the NF research community.

GLOSSARY

BPI

Brief Pain Inventory

FDA

US Food and Drug Administration

FPS-R

Faces Pain Scale-Revised

NF

neurofibromatosis

NRS-11

Numerical Rating Scale-11

PRO

patient-reported outcome

PRO-RATE

Patient-Reported Outcomes Rating and Acceptance Tool for Endpoints

QOL

quality of life

REiNS

Response Evaluation in Neurofibromatosis and Schwannomatosis

VAS

Visual Analogue Scale

AUTHOR CONTRIBUTIONS

Pamela Wolters: design and conceptualization of the study, collection and interpretation of the data, drafting and revising the manuscript. Staci Martin: design and conceptualization of the study, collection and interpretation of the data, drafting and revising the manuscript. Vanessa Merker: study concept, collection and interpretation of the data, drafting and revising the manuscript. Kathy Gardner: study concept, collection and interpretation of the data, revising the manuscript. Cynthia Hingtgen: study concept, collection and interpretation of the data, revising the manuscript. Jim Tonsgard: study concept, collection and interpretation of the data, revising the manuscript. Elizabeth Schorry: study concept, collection and interpretation of the data, revising the manuscript. Andrea Baldwin: study concept, collection and interpretation of the data, revising the manuscript.

STUDY FUNDING

This research is supported by the Intramural Research Program of the NIH, National Cancer Institute.

DISCLOSURE

P. Wolters received research support from the Childhood Brain Tumor Foundation and holds stock in Bristol-Meyers-Squibb, General Electric, and Zimmer Holdings, Inc. S. Martin received funding for a trip from the Children's Tumor Foundation. V. Merker reports no disclosures. K. Gardner has research funding from a Department of Veterans Affairs grant, #B5043R. C. Hingtgen has received research funding from the NIH, the Department of Defense, and the Children's Tumor Foundation in the past 2 years. She has no current grant support. She holds stock in Merck. She serves or has served as a subcontracted examiner on multiple sclerosis drug trials sponsored by Actelion, Acordia, Genentech, Genzyme, Lilly, Novartis, Sanofi-Aventis, and Teva. J. Tonsgard receives funding from a grant from Midwest NF and from a grant from the Department of Defense Research Command for the Consortium for Neurofibromatosis. E. Schorry receives funding from the Department of Defense as a site PI for the NF Consortium. A. Baldwin reports no disclosures. Go to Neurology.org for full disclosures.