Association between patient-reported outcomes and objective disease indices in people with NF2

Abstract

Objective

The association between patient-reported outcomes and currently used clinical trial endpoints, including total vestibular schwannoma (VS) volume and word recognition score (WRS) in neurofibromatosis type 2 (NF2), is not known.

Methods

A prospective observational study enrolling adult patients with NF2 was conducted at a single specialty center. Measures included: NF2 impact on quality of life (NFTI-QOL), short form (SF)-36; total VS volume, WRS; provider- and patient-reported disease severity (ProvSev, PatSev) measured with an institutionally derived multi-item (e.g., symptom burden, age-of-onset, and fatality-risk) and single-item Likert (mild, moderate, severe) scale.

Results

Fifty-one patients were enrolled between June 2014 and August 2017. Mean age was 42.1 ± 18.2 years and 37.3% patients were male. Mean WRS was 74.4 ± 37.3%; mean total VS volume was 4.2 ± 5.2 cc. Additional lesions were common including meningioma (79.2%) and spinal ependymoma (39.6%). Mean NFTI-QOL score was 7.6 ± 4.9 and correlated with responses on the SF-36. NFTI-QOL also correlated well with PatSev (r = 0.63, p < 0.001) and both multi- and single-item ProvSev (r = 0.62, p < 0.001; r = 0.52, p < 0.001, respectively). A weak correlation was observed between NFTI-QOL and WRS (r = −0.34, p = 0.0156). There was no correlation with VS volume (r = 0.23, p = 0.15).

Conclusions

The NFTI-QOL correlated well with multiple measures of disease severity but not commonly accepted endpoints for NF2 clinical trials including total VS volume in this US cohort of patients with NF2. This suggests that the NFTI-QOL captures components of the patient experience not sufficiently represented by objective measures of disease and underscores the important and complementary role of patient-focused measures in therapeutic outcome assessment in people with NF2.

Neurofibromatosis type 2 (NF2) is an autosomal-dominant tumor suppressor disorder characterized by multiple benign nervous system tumors. The hallmark of NF2 is the development of bilateral vestibular schwannomas (VSs) that usually present with hearing loss, tinnitus, imbalance, or a combination of the three.¹ Patients also characteristically have intracranial and spinal meningiomas, spinal ependymomas, other cranial nerve schwannomas, and peripheral nerve schwannomas.^1,2 Although the majority of the tumors in NF2 are histologically benign, they commonly result in profound morbidity including sensorineural hearing loss, imbalance, vision loss, facial weakness, bowel or bladder dysfunction, and gait instability, all of which can negatively affect quality of life (QOL). QOL is both an overarching term as well as a specific concept defined by the World Health Organization QOL Assessment Initiative as “individuals' perception of their position in life in the context of the culture and value systems in which they live and in relation to their goals, expectations, standards and concerns.”³

The traditional therapies for NF2-associated tumors are surgery and radiation therapy; however, these therapies are limited in managing disease-associated morbidity as they themselves can result in neurologic impairment and do not prevent future tumor development.⁴ Recent advancements in the biochemical understanding of NF2 have led to ongoing (Clinicaltrials.gov Identifier NCT02129647, NCT02831257, NCT03095248) and published^5–9 trials of targeted drug therapy for NF2. For the majority of these trials, reduction in VS size or improvements in hearing function are the primary endpoints.

Patient-reported outcome (PRO) measures are equally important endpoints and are gaining importance in regulatory assessments of new therapies.¹⁰ PRO measures can assess symptoms of disease known only to the patient and can provide unique information about the effect of treatment on QOL.¹¹ PRO measures are particularly useful endpoints in chronic debilitating conditions such as NF2 in which improvement of function and QOL are primary goals of treatment.¹² PRO measures can be either general or disease-specific. In patients with NF2, general PRO measures have been used in clinical trials and clinical care for decades (e.g., SF-36, EuroQOL) but general PRO measures may lack specificity for the disease and sensitivity for a therapeutic response.

A disease-specific PRO measure, the NF2 impact on quality of life (NFTI-QOL), was developed and validated in a population of patients with NF2 in the United Kingdom in 2013.¹³ In the original validation studies, NFTI-QOL scores correlated well with SF-36 scores and clinician-rated disease severity in patients with NF2. Although this important work was critical in bringing a needed NF2-specific PRO measure forward, to date, the NFTI-QOL has not been evaluated in a population outside of the United Kingdom. Cultural differences can mediate PRO assessments in different patient populations¹⁴; therefore, there is a need to assess the performance of the NFTI-QOL outside the United Kingdom. A recent review of disease-specific PRO measures found that the NFTI-QOL instrument had good construct and content validity but did not have sufficient data on reproducibility or responsiveness.¹⁵ Further, there are no existing data about the relationship between NFTI-QOL scores and commonly used endpoints in clinical trials such as word recognition score (WRS) and VS volume.^16,17 This is particularly needed now as ongoing clinical trials in the United States have incorporated the NFTI-QOL as an exploratory endpoint to assess response to change in the setting of therapeutics (NCT02129647, NCT02831257, NCT01767792). We hypothesized that the NFTI-QOL instrument would perform similarly in a US population as it did in the United Kingdom and that NFTI-QOL scores would correlate with objective measures of disease in NF2. The aims of this study were to 1) assess the performance of the NFTI-QOL in a population of US patients with NF2 and 2) assess the relationship between NFTI-QOL and conventional measures of disease including VS volume, WRS by audiometry, and provider-reported disease severity.

Methods

Study design and population

A single institution prospective observational study was performed at the Johns Hopkins University School of Medicine Comprehensive Neurofibromatosis Center. New and follow-up adult patients with a diagnosis of NF2 were invited to participate in the study. Patients were enrolled from June 2014 to August 2017. All patients 18 years and older who met criteria for NF2¹⁸ were eligible for the study. Patients under 18 years of age, those unwilling to participate, and patients who could not read or speak English were excluded from the study. During this observational study, patients received standard medical and surgical care as directed by their treating physicians.

Standard protocol approvals, registrations, and patient consents

The study was approved by the local IRB (IRB# 00027466). Written informed consent was obtained from all patients in the study.

Data collection

Patients completed the disease-specific PRO measure NFTI-QOL, the RAND short form-36 (RAND SF-36), and the single-item patient severity score (PatSev) (appendix e-1, links.lww.com/CPJ/A91). All of the questionnaires were completed prior to the patient-provider encounter to standardize the timing of assessment and not bias results by visit outcome.

The NFTI-QOL is an 8-item questionnaire, assessing various disease-specific domains including balance, hearing, facial weakness, vision, mobility/walking, role/outlook in life, pain, and anxiety/depression.¹³ The maximum score for each item is 3 (four-point scale 0–3, with 3 being the most impaired) and maximum total score is 24. Higher NFTI-QOL scores indicate worse outcome.

The SF-36 is a 36-item generic instrument that evaluates 8 aspects of perceived health: physical functioning, role functioning/physical, role functioning/emotional, energy/fatigue, emotional well-being, social functioning, pain, and general health. Each item in the SF-36 questionnaire is scored individually on a scale of 0–100, and items in the same scale are averaged together to create the scale score. For each respondent, only those items that are answered are averaged together to create the scale score, and items left blank are not taken into account. Higher values indicate better outcomes.

The PatSev score is an institutionally generated single-item scale. Patients were instructed to rate their disease severity by selecting mild, moderate, or severe, and this was scored on a scale of 0–2.

After the patient encounter, providers were asked to complete a physician-reported disease severity form (ProvSev) for each patient (appendix e-1, links.lww.com/CPJ/A91). The institutionally generated ProvSev included a multi-item assessment that consisted of 3 domains: symptom severity (e.g., vision, hearing, swallowing, vocal quality, balance, bowel/bladder, gait, communication; 1 point given for each function lost; score range 0–8), age at onset (<10 years old, 10–18 years old, 18–40 years old, >40 years old; score range 1–4), and fatality risk (estimated risk that patient will suffer a fatal complication within 6 months; score range 1–3). The overall score range for multi-item ProvSev was 2–15, with higher scores indicating greater disease severity. From this multi-item assessment, an overall score of mild, moderate, or severe (scored 0–2) was generated based on the sub scores for each of the 3 domains evaluated. Multiple providers evaluated 29 of 51 patients during the same patient encounter. For these patients, the ProvSev score is the average of all providers' evaluations.

Data collected for each enrolled patient at the time of their clinic visit included demographic (age, sex) and clinical data (age at diagnosis, diagnostic criteria, prior surgery, radiation or systemic therapy, active systemic therapy at time of evaluation).

MR imaging and audiometry performed within 6 months of completion of the questionnaire were analyzed for this study. We only included MR imaging and audiometry data within 6 months of study enrollment as we aimed to capture tumor volume and objective hearing function at or near the time of evaluation for NFTI-QOL. All imaging and audiometric assessments were performed as the patient's standard clinical care and were not study procedures. Brain and spine MRI studies were reviewed for the presence of spinal or intracranial meningiomas, spinal ependymomas, and spinal or intracranial schwannomas. Three-dimensional volumetric measurements of VS were calculated using the Vitrea2 semiautomated segmentation software on MRI brain and internal auditory canal scans.¹⁹ Total tumor volume was calculated by adding the left and right VS volumes. WRSs on audiometry were calculated as a percent correct using a standard 50-word list as part of a formal audiometry assessment.²⁰ Left and right WRSs were calculated separately. For patients who have two hearing ears, the better of the two WRS was used for analysis. For patients who only had one hearing ear, WRS is reported for that ear.

Statistical analysis

Demographic data, clinical characteristics, prior treatment characteristics, presence of other intracranial or spinal tumors, NFTI-QOL score, and SF-36 score were summarized using descriptive statistics. Spearman's rank correlation coefficients were used to explore correlations between disease-specific PROs (NFTI-QOL) and ProvSev, PatSev, total VS volume, WRS, and SF-36 scores; 95% confidence intervals and p-values were derived from Fisher's Z transformation. Wilcoxon rank sum tests were used to compare differences in the mean NFTI-QOL score by the presence or absence of intracranial meningioma or spinal ependymoma. A Bonferroni correction was applied to account for multiple comparisons; p-values < 0.00625 (= 0.05/8 tests) were considered significant for associations between NFTI-QOL scores and SF-36 domains. All analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC).

Data availability

Anonymized data will be shared by request from any qualified investigator.

Results

Fifty-three patients consented to participate in the study; however, two did not complete the NFTI-QOL and were excluded from the analysis. Thus, data from 51 patients were analyzed. Demographics of the study population are summarized in table 1. The mean age was 42.1 ± 18.2 years (range 19.9–82.9 years) and 37.3% were male. The majority of patients (96.1%) were diagnosed with NF2 based on the presence of bilateral VS; one patient was diagnosed by presence of a unilateral VS and multiple (>2) meningiomas. The mean age at diagnosis was 29.3 ± 17.3 years (range 9–75 years; median age of diagnosis: 26 years; median age at study entry: 38 years). Prior treatments included surgery (31, 60.8%), radiation therapy (9, 17.7%), and systemic therapy (13, 26.5%). At the time of enrollment, 9 (17.7%) were undergoing active systemic therapy. No patients were receiving active radiation therapy. Forty-eight patients had complete neuroaxis imaging available for review. In these patients, 38 of 48 patients (79.2%) had a documented intracranial or spinal meningioma, 31 of 48 (64.6%) had spinal schwannoma, and 19 of 48 (39.6%) had spinal ependymomas.

Table 1.

Characteristics of the study population

The NFTI-QOL scores are summarized in table 2. The mean NFTI-QOL score was 7.6 ± 4.9. In general, there was a moderate correlation between higher NFTI-QOL scores (worse disease-specific PROs) and lower SF-36 scores (worse health-related PROs) in all domains (table 3). After considering Bonferroni correction, the moderate correlation between higher NFTI-QOL and lower SF-36 scores remained statistically significant for all SF-36 domains except for the domains of role functioning/emotional and emotional well-being.

Table 2.

NFTI-QOL scores in study population

Table 3.

Description of SF-36 and association between SF-36 and NFTI-QOL

The mean provider-reported disease severity score was 6.3 ± 2.2 out of 15 on the multi-item scale. The median score on the single-item provider-reported severity scale was 1, corresponding to moderate provider-reported severity (3 item scale with scores 0/1/2, corresponding to mild/moderate/severe). A correlation was observed between the total NFTI-QOL score and the single-item ProvSev and the multi-item ProvSev score (r = 0.52, p < 0.001; r = 0.62, p < 0.001, respectively; table 4). The total NFTI-QOL score correlated moderately with PatSev (r = 0.63, p < 0.001, table 4). A moderate correlation was observed between PatSev and both the multi-item ProvSev score (r = 0.45; p = 0.001) and the single-item ProvSev score (r = 0.41; p = 0.003).

Table 4.

Association between NFTI-QOL and patient- and provider-reported disease severity (PatSev, ProvSev)

Forty-one of the 51 patients had VS amenable to volumetric assessment within 6 months of enrollment. The mean combined VS volume was 4.2 ± 5.2 cc for the 41 patients with evaluable lesions on MRI (table 1). Fifty of 51 patients had WRS within 6 months of enrollment. The mean WRS was 74.4% ± 37.3% (table 1). The PatSev score did not correlate with total VS volume (r = 0.20, p = 0.23), but it did have a moderate negative association with WRS (r = −0.44, p = 0.0018). NFTI-QOL scores were correlated with best audiometry WRS (r = −0.34, p = 0.0156, table 5) but not with VS volume (r = 0.23, p = 0.15, table 5).

Table 5.

Association between NFTI-QOL and VS volume and WRS

Poorer NFTI-QOL scores were associated with the presence of meningiomas. The mean NFTI-QOL score in patients with a meningioma was 8.6 ± 4.9 while the mean NFTI-QOL score in patients without a meningioma was 5.0 ± 3.9 (p = 0.043). There was no association between the mean NFTI-QOL score and the presence of spinal ependymomas (mean NFTI-QOL if ependymomas present = 7.6 ± 3.9, mean NFTI-QOL if ependymomas absent = 7.9 ± 5.5; p = 0.93).

Discussion

Although the need for PROs to assess both treatment toxicity and efficacy is well recognized for NF2,^10–12 there have only recently been efforts to create, validate, and measure the sensitivity to change of such tools in clinical trials or clinical practice.^13,21,22 Indeed, the current outcome measures for evaluating therapies in people with NF2 are objective radiographic response (reduction in VS volume) and hearing response (improvement in WRS).^23,24 Newer clinical trials for people with NF2 (NCT02129647, NCT02831257, and NCT03095248) include the NFTI-QOL as a secondary or exploratory outcome measure. However, there is limited information about the performance of this measure outside of the United Kingdom or the relationship between the NFTI-QOL and standard clinical trial endpoints in people with NF2.

Our results show that NFTI-QOL scores in a US population closely matched scores previously reported in a UK population of patients with NF2. Specifically, our mean NFTI-QOL score was 7.6 (SD 4.9), which is similar to the published norm in the original cohort study (mean 9.4; SD 5.5)¹³ and in the larger longitudinal cohort (mean 9.2, SD 5.4).²²

Also similar to the prior studies of the NFTI-QOL in the United Kingdom, we found a moderate correlation between poorer NFTI-QOL and worse provider-reported disease severity. Hornigold et al.¹³ and Ferner et al.²² both found a moderate correlation (r = 0.512, p < 0.001 and r = 0.41, p < 0.001, respectively) between NFTI-QOL score and clinician-reported disease severity. Although the present study used a different set of criterion to determine clinician-reported disease severity, we found very similar correlations between multi-item (r = 0.55, p < 0.001) and single-item (r = 0.45, p = 0.005) clinician-reported severity and the NFTI-QOL score. Similarly, there was a correlation between patient-reported disease severity (mild, moderate, severe) and the NFTI-QOL score (table 4). Correlation of both the patient-reported disease severity and provider-reported severity with NFTI-QOL scores both in the present and prior studies support the conclusion that the NFTI-QOL measure is valid in people with NF2.

We also validated the NFTI-QOL against the SF-36 in this US specialty center population and found that total NFTI-QOL score correlated with all the domains of the SF-36 with the exception of the emotional role functioning and emotional well-being domains for which the correlation was not significant after using Bonferroni correction. This is in contrast to Hornigold et al.,¹³ who found the NFTI-QOL score to correlate with all domains of the SF-36. Plausible reasons for this observed difference in the emotional role functioning and emotional well-being domains might be that our sample size was not large enough for the correlation to reach statistical significance or our patient population has different baseline emotional function than the original patient population in the United Kingdom. Another possibility is that the NFTI-QOL does not adequately capture the emotional domains of PROs, as the majority of the items of the NFTI-QOL address perceptions of physical impairment.

Importantly, this study evaluated the relationship between the NFTI-QOL score and the endpoints used most commonly in clinical trials for people with NF2: VS size (as assessed by tumor volume) and WRS. We found no correlation between NFTI-QOL scores and tumor volume. These results parallel studies of generic PRO measures (PROMs) in patients with NF2 showing a lack of correlation between generic PROMs and VS volume²⁵ and other studies which show that VS size correlates poorly with severity of symptoms and functional measures of hearing.²⁶ The relationship between NFTI-QOL and hearing function has not been assessed previously. Indeed, the existing literature is conflicting about the association between hearing loss and other measures of PROs in NF2. Cosetti et al.²⁷ found disease-specific PROs measured by their 61-item questionnaire to be associated with hearing loss but Merker et al.²¹ found no association between generic PROMs and hearing loss in people with NF2. Our results show a weak correlation between lower WRS and poorer NFTI-QOL scores. Our findings are somewhat expected given the phenomenon of the “disability paradox,” in which patients' perception of impairments does not correlate with objective clinical parameters.²⁸ Within this context, our results support the use of NFTI-QOL in patients with NF2–both in clinical practice and clinical trials–to provide additional information that objective measures do not provide.

There are some important limitations of this study. We were unable to perform volumetric analysis of the VS for 10 of the enrolled patients due to tumors not amenable to volumetric assessment or an absence of brain MRIs within 6 months of enrollment. Therefore, the associations between tumor volume and other variables such as the NFTI-QOL scores and PatSev should be interpreted cautiously. Our study excluded patients who could not read or speak English. Thus, our results are not generalizable to non-English speaking patients. As with any observational study, patients who declined to participate in our study may have been different than those who agreed to participate, and this may have skewed our results. In addition, all patients were enrolled from a multidisciplinary tertiary care center, which may have affected the underlying disease severity of our enrolled patient population. However, patients presenting to a tertiary center likely have disease severity and demographic factors similar to those of patients enrolled in clinical trials where the NFTI-QOL may be used as an outcome measure.

Despite these limitations, some important conclusions can be drawn. Our study found that the NFTI-QOL instrument measures disease-specific PROs in a US population of patients with NF2, and that NFTI-QOL scores correlate with several measures of disease, including patient- and provider-reported overall disease severity and objectively measured hearing function. However, NFTI-QOL scores did not correlate with the commonly accepted clinical trial endpoint of total VS volume. This suggests that the NFTI-QOL captures a component of the patient experience that is not adequately represented by objective measures of disease in NF2. Future studies should evaluate the performance of the NFTI-QOL in assessing self-perceived disease progression longitudinally or with therapeutic intervention. Our results confirm many of the results seen in the prior UK studies supporting the reproducibility and validity of the NFTI-QOL instrument in people with NF2. Further, the results suggest that at baseline, NFTI-QOL scores have weak or no correlation with other functional measures of disease in NF2, and hence, NFTI-QOL score changes should be considered on their own merit in the setting of clinical interventions.

Appendix. Authors

Study funding

No targeted funding reported.

Disclosure

A. Shukla, F-C. Hsu, B. Slobogean, S. Langmead, Y. Lu, and J.O. Blakeley report no disclosures. R. Strowd serves as the Deputy Section Editor of the Resident and Fellow Section of Neurology. Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.