Opportunities and challenges of incorporating clinical outcome assessments in brain tumor clinical trials

Abstract

Regulatory agencies have progressively emphasized the importance of assessing broader aspects of patient well-being to better define therapeutic gain. As a result, clinical outcome assessments (COAs) are increasingly used to evaluate the impact, both positive and negative, of cancer treatments and in some instances have played a major factor in the regulatory approval of drugs. Challenges remain, however, in the routine incorporation of these measures in cancer clinical trials, particularly in brain tumor studies. Factors unique to brain tumor patients such as cognitive decline and language dysfunction may hamper their successful implementation. Study designs often relegated these outcome measures to exploratory endpoints, further compromising data completion. New strategies are needed to maximize the complementary information that COAs could add to clinical trials alongside more traditional measures such as progression-free and overall survival. The routine incorporation of COAs as either primary or secondary objectives with attention to minimizing missing data should define a novel clinical trial design. We provide a review of the approaches, challenges, and opportunities for incorporating COAs into brain tumor clinical research, providing a perspective for integrating these measures into clinical trials.

Neuro-oncology is a continually evolving field, although it remains burdened by overall poor patient outcomes. Advances in treatment can be defined as either “living longer” (prolongation of survival), or “living better” (maintaining or improving functional status and quality of life [QoL]). Given the difficulties in finding treatments that prolong survival or deciding between treatments that equally prolong survival, there is an increasing emphasis on incorporating methods to evaluate the QoL and functional status into clinical trials. The World Health Organization defined the concept of QoL as “A state of complete physical, mental, and social well-being not merely the absence of disease.”¹ The WHO treatise continues with “It follows that the measurement of health and the effects of health care must include, not only an indication of changes in the frequency and severity of diseases but also an estimation of well-being and this can be assessed by measuring the improvement in the quality of life related to health care.” In the 1990s, attempts moved from assessing the strict concept of physical health as functional status to encompass wider health-related aspects.² The International Society for Quality of Life Research (ISOQoL) defines health-related quality of life (HRQoL) as “the functional effect of a medical condition and/or its consequent therapy upon a patient and is thus subjective and multidimensional, encompassing physical and occupational function, psychological state, social interaction and somatic sensation.”³ To better measure the impact of diseases and treatments on a patient’s well-being, an increasing number of outcomes under the umbrella of clinical outcome assessment measures (COAs) have been developed. They assess the broader concept of QoL including specific domains such as symptoms burden. COAs can address both efficacy and toxicity of treatments, and many measures have been developed and validated in cancer patients. The FDA and European Medical Agency have advocated their value in deciding the effectiveness of a treatment and therefore in the cost/benefit balance that lead a drug to be approved. While there is overall consensus among patients and the regulatory community on the importance and benefits of using COAs, challenges remain in incorporating these measures into clinical trials so the results augment traditional outcome measures such as progression-free (PFS) and overall survival (OS). The aim of this article is to highlight the challenges of incorporating COAs into brain tumor clinical trials, to identify opportunities, and to set the direction for the next generation of clinical trial designs.

The Evolving Field of Clinical Outcome Assessments

A COA is a clinical assessment that can be used as a clinical trial outcome. The assessment is made by a person, either a health care professional or a nonprofessional, on a symptom, a physical or mental health state or on an effect that can be related to a treatment. It represents an aspect of the patient’s health status and informs how the patient functions through a rating or a score. Based on the measurement properties of COAs (eg, a recording of the patient’s performance on a specific task or a subjective evaluation on a meaningful aspect of the patient’s health), and by whose judgment the measure can be influenced, they are further divided into 4 main subtypes: patient-reported outcome (PRO), clinician-reported outcome (ClinRO), observer-reported outcome (ObsRO), and performance outcome (PerfO). Each category has unique attributes that make it more responsive depending on the circumstances of specific research settings (Table 1). PRO scales can assess a variety of areas of the patient’s life and experience. The common background is that they put into the foreground the patient’s view, capturing symptoms that cannot otherwise be directly measured, such as pain, and areas that are difficult to assess, such as QoL.

Table 1.

Definition and Main Characteristics of Clinical Outcome Assessment (COA) Subcategories

ClinO	Definition	Any clinical judgment or interpretation of the observable signs, behaviors, or other physical indicators considered to be related to a disease or condition
	Strength	Correlation with objective outcomes
	Weakness	Assessment could have a wide interrater variability Moderate level of agreement between the clinician and the patient Does not stratify the heterogeneity of a brain tumor group
	Current use	Eligibility, tolerability to treatment, overall effect of treatment
PerfO	Definition	Any assessment based on a series of tasks performed by a patient according to standardized instructions and administered by a health care professional
	Strength	May show changes prior to neuroimaging features of progression
	Weakness	Demanding for the patients and for investigators (resources allocation) To detect changes, it often requires monitoring over a medium-long period of time
	Current use	Treatment efficacy and toxicity, net clinical benefit
PRO	Definition	Any report of status that comes directly from the patient, without interpretation of the patient’s response by a clinician, caregiver or any observer
	Strength	Patient-centered Capture unique areas otherwise difficult to quantify, such as pain and QoL Can be collected via different modes of administration (paper, interactive voice response, web-based system)
	Weakness	Influenced by life events, mechanisms of internal adaptation, mood, cognition to date
	Current use	Treatment efficacy and toxicity, net clinical benefit
ObsO	Definition	Any assessment based on an observation made by someone other than the patient or a clinician
	Strength	Helpful when the patient cannot provide an assessment Assessment of day-to-day functioning
	Weakness	Not always concordant with patient’s view of physical and mental symptoms Shows greater impact on what is more distressful for the caregiver (ie, cognitive decline)
	Current use	In specific circumstances

Abbreviations: ClinO, clinician-reported outcome; ObsO, observer-reported outcome, PerfO, performance outcome; PRO, patient-reported outcome; QoL, quality of life.

A recent evolution of PROs has been the incorporation of measures to evaluate treatment tolerability and toxicity. The reporting of adverse events in clinical trials represents a key factor to ensure patient safety and to detect the toxicity of new drugs. Patient-reported outcome measure reporting symptomatic toxicity (PRO-CTCAE)⁴ has been developed to complement the Common Terminology Criteria for Adverse Events (CTCAE), which is the standard recording of adverse events by clinicians. Other important scales complementing PROs have been originated by incorporating the proxy’s viewpoint, either a health care professional or informal caregiver (ObsRO). Caregivers have been shown to provide essential insight through their daily observation of the patient, and their assessment is felt to be of particular importance in a subgroup of patients who would otherwise struggle to provide information. For example, they apply to very young or very elderly patients, vulnerable and critically ill patients, patients with cognitive impairment or those in emotional distress.

Another subgroup of COAs is represented by PerfOs, which in neuro-oncology find their main application in the assessment of neurocognitive functions. Neurocognitive impairment is often seen in brain tumor patients, and it is a strong determinant of an individual’s QoL, greatly affecting patients and their relatives. Cognitive deficits affect the patient’s ability to perform activities of daily living (ADL) and contribute to the decline of functional independence. A change in cognitive function may have severe repercussions on the ability of a patient to continue a job and career, to hold a family role including child-care responsibilities, to recognize unsafe behaviors, and to manage finances. Cognitive dysfunction is a key factor in the planning of patient care and rehabilitation, correlates with poor prognosis,⁵ and represents a sensitive predictor of tumor recurrence.⁶ Depending on the purpose of the assessment, neurocognitive scales can measure the global functioning of the brain or domain-specific performances.

ClinROs are the most widely used COAs in neuro-oncology. A broad assessment of physical functionality defines the “performance status” (PS) of the patient through a linear scale from death to fully active. The judgment is made by the clinician, who then uses the index to determine the patient’s fitness to receive treatment. Brain tumor clinical trials and clinical practice both heavily rely on KPS⁷ and Eastern Cooperative Oncology Group Performance Status (ECOG PS)⁸ for treatment decisions. KPS and ECOG PS measure the patients’ ability to perform daily activities and the amount of assistance required to do so.

Setting the Stage for a Change in Direction: Challenges and Responses

Over the last few decades COA scales (Table 2), including PROs, have been developed and implemented in clinical trials to capture different aspects of patient well-being. The application of COAs is contributing to tailored-treatment decisions based on a risk-stratification of patients across several areas of medicine⁹ and across ages.¹⁰ There is growing evidence that demonstrates the clinical benefit of utilizing PRO measures and QoL measures in routine symptom monitoring.¹¹ Their advantage is the ability to quantify aspects that may elude the clinician or may be underreported by patients fearing treatment cessation. In clinical trials, the view of physicians and patients on symptoms scoring could be complementary, thus improving accuracy on data collection of adverse events and better informing on OS results.¹² Treatment decisions are strictly related to the physician’s assessment of the patient’s functional status, which in turns correlates with survival.¹³ Nonetheless, the physician may not adequately evaluate the patients’ psychological burden, which influences their functional status. There are examples of using PROs to evaluate drugs or drug regimens by highlighting their impact on patient well-being, particularly in areas not otherwise evaluated. For example, the use of PROs in women with breast cancer receiving taxane-based adjunctive chemotherapy have highlighted the need for monitoring mental health symptoms,¹⁴ defining a population at higher risk for negative psychological consequences.

Table 2.

Commonly Used Clinical Outcome Assessment (COA) Scales in Brain Tumor Patients

Category of COA	Scale	Number of Items	Time to Complete	Categories/Domains Tested
Patient-reported outcome (PRO)	EORTC BN20	30 items general cancer-related and 20 items rated on a 4-point scale (2-item 7-point scale)	5-10 minutes	C30: Functional status (5), symptoms scale (3), overall QoL and health status, (6) individual items BN 20: Future uncertainty, visual disorder, motor dysfunction, and communication deficit
	FACT-Br	27 items general cancer-related and 23-item scale brain-tumor specific	5-10 minutes	Well-being items: physical (7), social/family (7) emotional (6), functional (7)
	MDASI-BT	21 symptom items and 7 interference items 11-point rating scale	4 minutes	Symptoms commonly associated with cancer therapies and neurocognitive symptoms common in brain tumor
	SQLI	5 items; range = 0 (best QoL) to 2 (worst QoL)	1 minute	Activity, daily life, health perceptions, social support, behavior
Clinician-reported outcome (ClinRO)	NANO	8 domains	4 minutes	Neurology signs
	ECOG	1 item: grade 0 (fully active) to 5 (dead)	1 minute	Overall health status
	KPS	1 item; range = 100% (normal) to 0% (dead)	1 minute	Overall health status
Performance outcome (PerfO)	MMSE	11 items	11-15 minutes	Cognitive impairment
	COWA, WAIS-Digit Span Test, Trail Making Test A and B, HVLT-R, Stroop test	Variable number (few items/ each)	≃ 5-30 minutes/each	Domain-specific tests respectively: verbal fluency, attention/short-term memory, processing speed/mental flexibility, memory, executive functions

Abbreviations: COWA, Controlled Oral Word Association; EOCG, Eastern Oncology Cooperative Group; EORTC BN, European Organization for Research and Treatment of Cancer Brain Tumor Module; FACT-Br, Functional Assessment of Cancer Therapy-Brain; HVTL-R, Hopkins Verbal Learning Test-Revised; MDASI, MD Anderson Symptom Inventory; MMSE, Mini-Mental State Examination; NANO, Neurologic Assessment in Neuro-Oncology; QoL, quality of life; SQLI, Spitzer Quality of Life Index; WAIS, Wechsler Adult Intelligence Scale;.

The field of oncology has produced several examples of COAs that greatly contributed both to study interpretation as well as treatment approval and registration. Comparable examples are limited for brain tumor clinical trials (Table 3). PROs captured the increase in symptoms burden in patients with glioblastoma receiving dose-dense temozolomide after concurrent chemo-radiation when compared to conventional adjuvant dosing.¹⁵ Some studies prioritized the neurocognitive outcome as the primary endpoint, for example, when investigating the neuroprotective effect of memantine and hippocampus-sparing in whole-brain radiotherapy for the treatment of brain metastasis.^16,17 As COAs have been proven to be a valuable integral component of clinical trials, there are strong indications that their systematic introduction can bring benefits at various levels. This change in clinical trial practice is not exempt from challenges, and much experience has to be gained to refine the methodology on how to incorporate the combination of COAs into the study design and analysis. A number of interest groups and coalitions have been convened to develop uniform approaches to ensure the quality, appropriateness, and validity of the measures. The Jumpstarting Brain Tumor Drug Development Coalition represents 1 example of these. It set the direction for brain tumor clinical trials by highlighting which measures may provide a better assessment of the “net clinical benefit” of a therapy (ie, a composite assessment of a treatment effect on the patient’s life) and how to better optimize the design of clinical trials incorporating COAs.¹⁸ The Response Assessment in Neuro-Oncology-Patient-Reported Outcome (RANO-PRO) is another multidisciplinary international group that has recently focused its effort on guiding the choice of PRO scales in brain tumor research.¹⁹

Table 3.

Incorporating Clinical Outcome Assessment (COA) Measures in Clinical Trials: Impact of Patient-Reported Outcome (PRO) Measures on Study Findings and Implications (Examples)

Study Authors, YearRef	Type of Cancer	Drug/Treatment/Intervention	PROs Endpoint	Impact on Objective Measure	Impact on COAs and Broader Aspects	Overall Effect	Conclusions/Implications
Tannock et al, 199620	Prostate cancer	Adjuvant mitoxantrone with prednisolone	PE	–	Improve in pain	+	FDA approval for advanced hormone-refractory prostate cancer
LATITUDE trial (Fizazi et al, 2017)22	Prostate cancer	Abiraterone acetate	EE	Increased OS	Improvement in HRQoL	+	FDA approval for high-risk castration- sensitive prostate cancer
Verstovsek et al, 201222	Myelo-fibrosis	Ruxolitinib	SE	Reduction of spleen volume	Reduction in symptoms burden	+	Approval of ruxolitinib for treatment of myelofibrosis
Thornton et al, 200814	Breast cancer	Taxanes in adjuvant chemotherapy regimen	PE	–	Worse mental health outcome and double emotional recovery time	–	Monitoring depression in patients receiving taxanes
RTOG 0525 substudy (Armstrong et al, 2013)15	Brain tumor (GBM)	Dose-dense temozolomide treatment associated with radiotherapy	EE	No change in survival	Greater symptom burden	–	Standard dose of temozolomide
RTOG 0933 (Gondi et al, 2014)17	Brain metastases	HA-WBRT	SEa	–	Memory preservation and improved QoL	+	Evidence supporting HA-WBRT
EORTC-NCIC trial (Stupp et al, 2009)23	Brain tumor (GBM)	Adjuvant temozolomide with radiotherapy	SE	Increased survival	No change in QoL	+	Change in standard of care

Abbreviations: EE, exploratory endpoint; EORTC-NCIC, European Organisation for Research and Treatment of Cancer-National Cancer Institute of Canada; GBM, glioblastoma; HA-WBRT, hippocampal-sparing whole-brain radiotherapy; HRQoL, health-related quality of life; PE, primary endpoint; QoL, quality of life; Ref, reference; SE, secondary endpoint; WBRT, whole-brain radiotherapy.

^aWith performance outcome (PerfO: neurocognitive testing) as PE.

Despite the increasing interest, there is no consensus on the optimal approach to the statistical analysis of COA data.²⁴ It is possible that, even utilizing similar COAs and PROs, the statistical analysis approach may determine discordant results. One of the most recently debated example consists of 2 glioblastoma studies, Avastin in Glioblastoma (AVAglio)^25,26 and Radiation Therapy Oncology Group (RTOG) 0825.²⁷ Both trials studied the effect of adjunct bevacizumab on a standard chemo-radiation regimen and showed similar results on traditional measures. Both studies incorporated the same instruments to measure the patients’ QoL but their findings were discordant. AVAglio had limited neurocognitive data whereas RTOG 0825 had robust measures of symptoms burden and neurocognitive function to complement the QoL data. In addition to the discordant collection of COAs, the 2 protocols used a different study design and statistical analysis approach. Possible explanations for the discordant results may be the difference in the selection criteria of the 2 trials such as the extent of surgical resection, the neuroimaging criteria to determine disease progression, and the clinical benefit domains tested. Furthermore, the analytical methods of the 2 studies as well as their respective survey response rates were different. The AVAglio analysis was performed at the time of either QoL deterioration or tumor progression, while the RTOG 0825 study analyzed longitudinal changes in patients without tumor progression. Thus the above studies exposed the dilemma of the standardization of clinical trials methodology and the analysis of PRO data. The Setting International Standards in Analyzing Patient-Reported Outcomes and Quality of Life Endpoints Data Consortium has been formed to bridge gaps on statistical methods. The group is still developing its recommendations.

Characteristics and Challenges of the Constructs Measured by Clinical Outcome Assessments

The first step to address how to incorporate COAs is to analyze the constructs they ought to measure, thus providing the net clinical benefit²⁸ that can be concordant or discordant with the effects of a treatment measured by objective outcomes (eg, PFS and OS). In patients with brain tumors, the functional endpoints of a treatment are often represented by the triad of QoL, symptoms burden, and neurocognitive outcome. The evaluation of these domains requires investigators to be aware of the intrinsic characteristics of each that partly dictate the method of data collection as well as the schedule and timing of assessments that may expose them to possible analysis and interpretation bias.

Quality of Life

In public debates the concept of QoL is intuitive, and it is often defined as subjective well-being. The concept is central to the human condition, and of utmost importance in incurable diseases, but its quantification as a clinical trial outcome presents several challenges. QoL is a moving target, a multidimensional condition that changes accordingly to life events and hence vulnerable to shifting in accordance with patient expectations, anxiety, and worries. QoL encompasses areas that are indirectly affected by disease, such as employment, child care and family life, household responsibilities, and relationships. The individual perception of QoL can change over time because of a variety of factors as highlighted in Table 4. QoL measures are captured by PRO scales, and researchers must be aware of those challenges to frame the results into a context that allows their correct interpretation. Combining measures that specifically inform on those areas, such as the role of the patient in society, should balance fluctuation in scoring due to life events. Carefully considering the timing of the assessment may help avoid times when emotional distress has higher chances of affecting the patient’s judgment and consequently affecting the scoring. It is important to take into account the changes in internal standards that occur with age as they do modify the conceptualization of QoL. Elderly patients may have views on QoL that are different from younger patients; for example, they may place a higher value on the preservation of independence. This aspect should be carefully considered when choosing the questionnaire specific to the population under study. If the study design involves a wide age range, either a stratification by age group or an analysis that takes into account a possible variability in the QoL results should be considered. Furthermore, neurocognitive impairment does have an impact on PROs, including QoL questionnaires. It is important to recognize whether cognitive decline is expected to occur within the clinical trial time frame. The longitudinal assessment of QoL is of particular relevance to provide information that could potentially predict the patient’s survival probabilities along the disease journey, thus dynamically guiding treatment decisions. Compliance can become a major issue, causing missing data over time. Coupling longitudinal assessment of QoL outcomes with traditional time-to-event outcomes, for example survival, requires adopting appropriate statistical analysis approaches, as discussed later.

Table 4.

Factors Likely to Affect Patient-Reported Outcomes (PROs)

Factors Affecting Assessment	Duration of Working Effect	Construct Affected	Implicated Mechanism of Action	Possible Measures to Balance Bias
Response shift	Over time	Quality of life	Internal dynamics of adaptation to disease (recalibration/reprioritization/ reconceptualization)	Model to account for that effect (then-test)
Significant life events	Fluctuating in waves	Quality of life Symptoms burden	Social and familial dynamics	Correlation with life changes
Timing of assessments	Transient	Quality of life Symptoms burden	Anxiety and worries	Avoiding questionnaire completion in proximity of known emotional times
Location of tumor	Over time	Quality of life Symptoms burden	Judgment and mood may be impaired in frontal tumors	Subgroup analysis
Cognitive decline	Over time	Quality of life Symptoms burden	An interplay of factors may hamper assessment	Subgroup analysis

Symptoms Burden

Symptoms burden describes the presence of symptoms and suffering that significantly affects patients, family, and carers. Assessing the symptoms burden is widely used in research and clinical practice whether the intent is to cure or to palliate. The increased involvement of patients in their own care, known as a patient-centered approach, has led to the identification of a core set of symptoms that primarily affects the daily activities and QoL of brain tumor patients.²⁹ The incorporation of the patient’s view provides an opportunity to examine the impact of a treatment through the measuring of symptoms not otherwise easily quantified, such as nausea and pain. It adds a new dimension to the treatment evaluation, increasing the chances to detect minimal, yet significant, differences between treatment regimens that can in turn influence drug licensing and treatment guidelines. The core set of symptoms has been defined with the symptoms reported by brain tumor patients as more severe or frequent. In particular headache/pain, seizures, difficulties in concentrating and memory, alteration in mood and personality, and difficulty walking. Evaluating the same core set of symptoms could facilitate comparisons across trials, and PRO scales like the MD Anderson Symptom Inventory for Brain Tumor (MDASI-BT) and the MDASI for Spine Tumors (MDASI-SP) have been increasingly used to evaluate symptoms burden and daily-life interference in brain tumor patients.³⁰ Novel therapies may require the addition of specific symptoms to the core set given their different mechanisms of action and possible toxicity. For example, a clinical trial on a new immunotherapy drug should incorporate additional validated items wherever possible to cover adverse effects of the specific treatment that would not otherwise be targeted. Investigators may take advantage of widely available technology, such as smart phones, to corroborate reported symptoms. Mobile applications or electronic devices can monitor sleep-wake cycle, mobility, and overall activity level, thereby adding an extra dimension to the symptoms evaluation.

Neurocognitive Outcome

Brain tumor patients often show a decline in cognitive functions as a result of the disease and/or the treatment, with more frequent deficits in processing speed, memory, attention, and executive functions.³¹ It is likely that the respective contribution of tumor-related factors (such as edema and increased intracranial pressure, tumor-related epilepsy, and emotional and mood status) and treatment-related factors (such as surgical intervention, chemo-radiation therapy, medications, fatigue, and sleep cycle disruption) to cognitive outcome varies over time. For example, edema temporarily affects the cognitive processes and improves over time when appropriately addressed. On the other hand, the incidence and severity of radiation-induced cognitive impairment increases over time. With improvements in survival, cognitive deficits become more obviously recognized, but unfortunately these late effects have not been well studied in adults with long-term survival. There are no biomarkers defining patients at higher risk of neurocognitive impairment, and factors such as tumor localization, population age, and to some extent tumor aggressiveness (slow growing vs fast growing) are only modestly helpful in predicting the likelihood and the severity of neurocognitive deficits.

The impact of a treatment on neurocognitive outcome, both during the early phases and as a late complication of the disease, remains an area of concern, with increasing awareness of the synergistic role of treatments (chemotherapy and radiotherapy) on cognition. Being able to minimize the contributing factors and prevent additional damage to the neurocognitive function would strongly affect the quality of the remaining life and change the care of brain tumor patients. The incorporation of neurocognitive assessments into clinical trials may provide a better estimate of the relative effects of new drugs and a platform for treatment comparisons. Batteries assessing neuropsychological profiles in brain tumor patients are widely utilized. Depending on the purpose of the assessment, they can measure the global functioning of the brain or domain-specific performances.

Despite the importance of assessing the neurocognitive function in clinical trials, there are several challenges. Implementing the neurocognitive test battery requires extensive resource utilization. The assessment demands a trained assessor and the allocation of appropriate time for the test and its evaluation. Patients may be reluctant to undergo cognitive function testing, particularly over time and when there is disease progression, likely because of increased symptoms burden and decreased motivation. There is no consensus on the criteria to apply to neurocognitive evaluation in clinical trials and there are no guidelines on how to determine the best compromise between extensive time- and resource-consuming assessments vs selected components of neurocognitive testing. Traditionally, neuropsychological tests in clinical practice are used to aid the diagnostic process by detecting a pattern of domain-specific deficits that defines an underlying disease. In clinical trials, however, it may not be necessary for such a detailed evaluation. It can be argued that in the context of a clinical trial, the neurocognitive evaluation may be optimally used to measure changes over time that are important for patient functionality. Therefore, ecological validity paradigms³² and technology-based tests provide an efficient³³ and cost-effective resource. Neuropsychological assessments with a higher level of ecological validity (eg, tests that assess the cognitive performances on tasks resembling everyday activities) aim to provide results that predict a patient’s behavior in real life and have shown to correlate with ADL. Alongside this, virtual-reality tools³⁴ may provide an effective way to assess cognitive functions that are essential in routine daily activities. Computerized cognitive assessments can be built to simulate everyday life tasks and activities with virtual-reality effects, such as shopping in a supermarket and traveling on public transportation. Patients and families may feel it beneficial to gain a practical insight into the impact of a treatment, and researchers can more effectively document the impact of a drug or a rehabilitation intervention as a change in everyday performance. In addition, these assessments can be utilized longitudinally at several endpoints, thereby increasing data collection and improving patient compliance without significantly affecting resources and feasibility.

Considerations of Clinical Trials Design for the Integration of Clinical Outcome Assessments

There are several key points in the incorporation of COAs into clinical trial protocol and design that require careful consideration. We list these points and provide some additional insights, defining the benefits and risks that investigators may face.

Eligibility

Brain tumor clinical trials are becoming more complex because of a variety of factors, including advances in molecular and genetic tumor profiling that predict a pattern of clinical course, response to treatment, and outcome. Patient selection becomes increasingly important to define the “optimal” treatment for a group of patients and to correctly interpret the efficacy of novel treatments. The comparison between treatments often relies on modest differences and unfortunately, the efficacy of new drugs or regimens rarely significantly affects OS. The clinical benefits are more often demonstrated as a prolongation of PFS, a reduction in steroid requirement, or maintenance of neurocognitive function, improvement or stabilization of symptoms burden, and/or QoL. For many years the selection of patients deemed suitable for cancer treatments has been based on the assessment of individual health and functional status using a simple-to-use standard measure. Eligibility for clinical trials significantly relied on the patient’s performance status, such as KPS or the Eastern Cooperative Group Performance Score (ECOG PS). These scales were developed more than 50 years ago and aimed to assess eligibility to undergo systemic treatments. They have proven to be valuable tools both in clinical practice and in research, correlating with objective outcome measures and tolerability to treatment. Still, they showed a degree of variability particularly with interobserver concordance^35,36 and may fall short in light of the recognized complexity of brain tumor treatment evaluations. Despite the well-recognized correlation with prognosis, PS does not take into consideration the patient’s view of his or her functional status and QoL, nor patient emotional well-being or the neurocognitive function.³⁷ In addition, the PS scales do not take into account the very nature of brain tumors: A single lesion in an eloquent area can significantly affect KPS/ECOG score but still does not represent the overall health and fitness-for-treatment of the patient.

Correctly assessing and scoring a patient’s functionality has important repercussions on patient selection, recruitment, and eventually findings’ interpretation; even when the primary measure is OS. Combining scales that assess the functionality of patients on broad aspects of their life and health may increase the accuracy of the assessment. Incorporating a combination of COAs into the eligibility criteria should be considered when the primary aim of the trial is the evaluation of the therapeutic effect (Phase III clinical trials). In this context, dichotomization of a quantitative variable should be approached with caution (eg, grouping patients into arbitrarily defined classes, such as high cognitive impairment and low cognitive impairment) as in most circumstances this results in a decrease in measurement reliability, loss of representation of individual differences, and increased complexity in statistical analysis and interpretation.³⁸ COAs could be used as a stratification factor in the randomization process to produce homogeneous cohorts also for those variables. A more accurate eligibility that relies on a complete assessment of patient functionality may both improve the balance of patient randomization and also better reflect the impact of a therapy on real-world patients, thus increasing the generalizability of the results.

Clinical Trial Framework

Clinical trials should start from a highly relevant and specific research question. To produce high-quality and informative data, researchers should first choose the most appropriate outcome measures to detect the response to the treatment under investigation. The measuring tool must be relevant and specific to not only the patient population under study but also to the time within the disease trajectory of the study. To enhance interpretation, the measures should be designed to appropriately detect minimal clinically important differences within the research time frame. The considerations for choosing the most appropriate combination of COA measures that are likely to set the stage for future neuro-oncology clinical trials are provided in Figure 1. The disease at the early stages is better evaluated by a set of measures that is different from what would be used far down the disease trajectory, and patients with longer survival are exposed to the risk of late effects that little concern patients who have rapidly progressing brain tumors.

Fig. 1.

Choosing COAs: Practical Aspects for Clinical Trials.

COAs indicates clinical outcome assessments; OS, overall survival; PFS, progression-free survival; NCF, neurocognitive function; QoL, quality of life.

The choice of the most appropriate COA is also based on the phase of the clinical trial and the duration of the study. Phase I clinical trials usually have a shorter duration and determine safety and dosage. Measures such as PRO-CTCAE may provide a more complete evaluation of the tolerability of the drug. Although not widely implemented, incorporation of PROs in Phase I trials may help determine the best measures and statistical modeling for subsequent Phase II and Phase III trials (Figure 2). Phase II clinical trials, which are designed to evaluate efficacy of a treatment, may be enhanced by monitoring the treatment-related effects using instruments to measure changes in symptoms burden and neurocognitive function, rather than more traditional measures of QoL. In contrast, Phase III clinical trials are designed to definitively test the efficacy of a treatment with a fully statistically powered, randomized trial, and as such the choice of the instruments should focus on the ability to determine whether the experimental treatment improves outcome compared with the standard therapy. Therefore, depending on the study design, COAs can serve as primary outcomes measures or secondary measures to enhance the interpretation of the primary efficacy measure.

Fig. 2.

Practical Guide to Incorporate COAs into Clinical Trials.

The choice of outcomes and measuring tools in each phase of a clinical trial can influence the next study and help refining the research hypothesis. Each phase can be based on the previous results whether the net clinical benefit and overall treatment evaluation is promising (ie, providing positive results) or has produced discordant results that have to be further explored.

COAs indicates clinical outcome assessments; CTCAE, Common Terminology Criteria for Adverse Events; EO, exploratory outcome; OS; overall survival; PerfO, performance outcome; PFS, progression-free survival; PO, primary outcome; PRO, patient-reported outcome; PRO-CTCAE, patient-reported outcome Common Terminology Criteria for Adverse Events; QoL, quality of life; SO, secondary outcome; ± discordant results (ie, tools have provided a conflicting result, eg, increased PFS and worsening of cognitive functioning).

Clinical Trial Objectives: Clinical Outcome Assessment Incorporation

Several critical questions and issues arise when considering the incorporation of these measures into clinical trials. Traditionally, PRO measures were used only as exploratory or secondary endpoints to further strengthen relevant findings, leading to a change in the standard-of-care treatment. More recently it has been advocated that COAs and PROs should be incorporated as relevant outcomes complementing traditional endpoints such as OS and PFS. It may be appropriate to have the COA as the primary outcome measure if the goal of the trial is to evaluate a regimen designed to reduce toxicity or restore function. Furthermore, innovative statistical designs are being considered to allow composite measures of efficacy that combine traditional outcomes with COAs. This has the great advantage of elevating the perceived importance of the COAs, thereby improving compliance, a common issue when COAs are implemented as a secondary or lower-level endpoint.

Minimizing Known Clinical Outcome Assessment Bias

When using PROs, cognitive decline may affect the ability to self-report. The increasingly recognized utility of proxy reporting by caregivers could provide a valid substitute in these circumstances. A balance between the length of the instrument and the potential burden for the patient must be considered to address the loss of compliance over time that can be further improved with thoughtful planning of the timing and frequency of testing. Furthermore, nonredundant questionnaires are helpful in maintaining patients’ involvement. In summary, instruments that are short, easy to understand, and specific to the disease are most likely to maintain compliance. Additionally, incorporating the COA component as either the primary or a secondary objective increases the mandatory oversight and warrants allocation of dedicated personnel for the administration and collection of the data, and the education of the patient about the importance of the assessments. Additional measures can be instituted such as setting previsit reminders for patients for questionnaire completion via a variety of routes, such as email, text, phone or letters. Contacting patients who fail to submit their assessment by the predetermined deadline could further improve compliance.

The study design should also carefully consider the assessment schedule, particularly for PRO measures. Avoiding proximity to emotional times, such as the delivery of MRI results and the status of the disease, may reduce assessment bias due to the patient’s emotional turmoil. To minimize errors in missing questions, handling or storing data, investigators should limit data collection to a maximum of 30 minutes and take advantage of electronic technologies (tablet, weblink) that, for example, provide a format with courteous error messages for skipped questions.

Statistical Considerations and Interpretation of Results

Incorporating COAs into clinical trials raises the complexity of the analysis (Figure 3), thus requiring appropriate statistical methods to match the research questions. As explained in the previous section, the designation of primary endpoint places the highest level of priority mandating adequate resources for sites to complete the measurements and oversight of compliance. Secondary endpoint status does afford the opportunity to have a formal statistical analysis plan incorporated into the primary protocol document, providing a prospectively planned roadmap that enhances the impact of the outcome results. Studies lacking funding to provide adequate resources both for staff at the treatment sites and for the monitoring group face the discovery of compliance and missing data issues late in the trial or even after the study has completed accrual.

Fig. 3.

Clinical Outcome Assessment Incorporation into Clinical Trials: Schematic Aspects of Analysis and Interpretation

One of the other important considerations for incorporating COAs into clinical trials is the handling of missing data. Invariably, patients fail to complete the panel of tests but there are methods that help determine the impact of the missing data. Using pattern mixture models and performing sensitivity analyses are common techniques. Collecting auxiliary information has the potential for correcting bias when the missing process is nonrandom. A joint modeling algorithm analyzing combined HRQoL data and survival outcomes has been applied by a Belgium-Dutch group.³⁹ The group analyzed the information collected in the long-term study European Organisation for Research and Treatment of Cancer 26951⁴⁰ to evaluate the association between QoL and survival. One of the advantages of this model is its validity under a missing not-at-random assumption. In addition, the joint model of longitudinal HRQoL data and time-to-event outcomes analysis showed that survival and QoL information can be combined to provide a more accurate estimate of the treatment effect.

Novel statistical designs that incorporate both traditional and COA measures have been proposed, but there are no established guidelines to help interpret results when 1 measure is positive and the other is negative. As an example, how should a clinical trial be interpreted that demonstrates an improvement in PFS for the experimental treatment compared to the standard of care, but with significant worsening of PRO measures? While the same individual measures can be compared across clinical trials, it may prove more challenging to compare overall composite measures (net clinical benefit) as they rely on the interpretation of the value of each different component. If composite scoring is used, these must be specified in advance and taken into account in the statistical analysis plan. Otherwise, post-hoc interpretation methods may put different weight on statistically and clinically significant results, thus reaching different conclusions. As mentioned above, although not routinely performed, individual measures could also be integrated into a single composite score. An example of this mathematical approach to compare treatments is provided by the quality-adjusted effect size,⁴¹ which weighs the combination of outcomes, for example, survival and toxicity, based on the value assigned to their differences—eg, assigning equal weight on survival and toxicity vs weighing one 3 times over the other. Although this method could provide a way to interpret the findings of a clinical trial, further experience will be needed to better justify this approach in research and clinical practice.

Clinical Practice Implications

The ultimate goal of clinical trials is to influence and change the standard of care. Although it is not clear when COAs will be considered an essential component both of clinical trial design and patient care, the clinical community has been moving toward patient-centered care with tailored approaches to treatment that incorporate the patient’s perspective and preferences. Although initial suitability to chemo- and radiotherapy is often based on KPS/ECOG PS, decision making about the optimal treatment for an individual patient is a complex process that takes into account many variables beyond the PS and encompasses the patient’s mood, priorities, environment, and preferences. Nonetheless, the current method lacks standardization and clinicians are reluctant to interpret the information collected through COA measures, in particular longitudinal PROs.⁴²

Despite the well-recognized potential to improve patient care, at present there is not widespread use of these measures in the clinical community. In the future, specialized centers may be better placed to start the process of incorporating COAs into standard practice. Health systems across the world are working on integrating electronic medical records with an electronic database collection that will allow the integration of PROs in routine clinical practice. Long-sighted and pioneering institutions should consider allocating resources in dedicated structures such as a symptom-monitoring team and an implementation science team. As experience has to be learned in the field of COAs, initial guidance should come from dedicated coalition groups such as the ISOQoL.⁴³ The specific characteristics and needs of the brain tumor population warrant the continued development of international collaborative efforts to generate a global consensus on instrument selection, implementation, and interpretation.

Conclusion

The neuro-oncology community has become increasingly interested in various COAs as valuable tools complementing standard objective measures to better evaluate treatment effects. Despite the well-recognized contributions of COAs to clinical trials, many challenges exist. This overview highlights the importance, opportunities, and practical challenges of incorporating and analyzing COAs and PROs in clinical trials, focusing specifically on brain tumor studies. Embedding outcome measures that are patient-centered is complex. Challenges include but are not limited to the development of protocol-specific hypotheses that lead to the choice of the instruments to use, the recognition of resource limitations, the minimization of patient burden, the incorporation of measures into the clinical designs, and the appropriate statistical approach. Increased utilization of COAs in brain tumor clinical trials will provide the necessary experience and information to optimize both the trial design and the statistical method, further enhancing the importance of these measures in the interpretation of study results, in facilitating the choice of the optimal treatment for individual patients, and in aiding regulatory decisions.

Funding

The authors Dr. E. Molinari, Dr. T. R. Mendoza, Dr. M. R. Gilbert, certify that no funding has been received for the conduct of this work and/or preparation of this manuscript.

Conflict of interest statement. None declared.