Measuring everyday functioning in patients with brain tumor: The long rows yet to hoe

Oort et al are in the process of developing a 32-item instrumental activities of daily living (IADL) questionnaire (EORTC IADL-BN 32) that they hope will be a reliable and valid instrument for measuring brain tumor patients’ everyday functioning both in clinical practice and in clinical trials.¹ It has been shown in other neurologic populations (ie, predominantly those with dementia) that IADL ratings are associated with tests of neurocognitive function (NCF) and lack of concordance between patients and proxies increases as patient NCF declines. In this issue of Neuro-Oncology Practice, Oort et al² specifically examined the concordance between patient and proxy IADL ratings on the EORTC IADL-BN 32 in a heterogeneous primary and metastatic brain tumor population. They additionally explored the role of NCF impairment in the concordance between patient and proxy IADL ratings.

Oort et al are to be applauded for including patients and proxies that had frequent contact with each other, employing robust standardized NCF tests, using rigorous NCF impairment criteria, and for measuring self-reported NCF concerns. Contrary to their hypotheses, there were no statistically significant discrepancies between patient and proxy IADL ratings. When compared to their proxies, there were non-significant trends in which patients with intact NCF rated their IADL functioning worse and patients with impaired NCF rated their IADL functioning better. On the basis of non-significant trends, Oort et al recommend obtaining both patient and proxy IADL ratings when patients are NCF impaired “to gain a better picture of the patients IADL functioning.” Beyond the fact that statistically significant differences were not observed, it is important to consider the complexities of IADL instrument development, IADL selection and measurement considerations, and the evidence required before integrating these recommendations into clinical practice or research.

Establishing reliability- and validity-related evidence for an instrument is an iterative process, specific to different types of reliability (eg, interrater, test-retest) and validity (eg, content, criterion, construct, convergent, face), and may be specific to situations of use and/or populations of interest. Oort et al previously described their approach to develop face and content validity-related evidence¹ for a new IADL measure for patients with glioma. This resulted in a list of 32 activities described as important for patients with glioma. However, arguably one of the most important forms of validity-related evidence that needs to be established is construct validity, to include demonstrating that ratings of IADLs, by either the patient or their proxy, accurately reflect the patient’s capacity and manifest functioning. It is important to distinguish between what IADLs a patient can do under ideal circumstances (ie, capacity) vs what they are actually doing independently in their daily lives (ie, manifest functioning). All IADLs are observable behaviors amenable to behavioral sampling such that it is reasonable to suggest that the process of establishing construct validity of a patient- or proxy-reported IADL measure should include showing good correspondence with a performance-based IADL test in a standardized environment (as recommended by the World Health Organization³) or a directly observed behavioral sample of the IADL being assessed. The clinical usefulness of the EORTC IADL-BN 32 may hinge on its associations with performance-based everyday functioning and real-world, objectively determined household and health functioning.

A large body of evidence demonstrates that both patients and their proxies have biases that shape their IADL ratings. In addition to the possible role of decreased awareness and insight on the part of the patient that may contribute to errors in rating of IADL capacity, both mood disturbance and burden levels lead to more negative appraisal of abilities by both patients and proxy informants.^4,5 Clinically, we are acutely aware of the substantial stress our patients and their care providers experience throughout the cancer experience. Research has confirmed the prevalent, persistent, and disruptive nature of the burden experienced by brain tumor patient caregivers.^6–8 Thus, it is an oversimplification to say that patient and caregiver proxy subjective reports are uniformly preferred, unadulterated, and/or more reflective of the actual IADL functioning of a patient. We agree that gathering data from multiple sources can be helpful in triangulating the ground truth on such matters but it is critical to consider the context, experiences, mood, and personality features of patients and caregiver proxies when trying to link ratings of IADL to actual IADL capacity and other objective indicators of everyday functioning.

Two other important sources of information about IADL functioning are data from objective performance-based IADL tests and unbiased direct observations. Demonstrating convergent and concurrent validity between these objective IADL measurement approaches and subjective patient/proxy IADL ratings is an important and necessary step before utilizing a subjective surrogate measure of IADL capacity for clinical and research decision making. Recent research has examined evolving types of IADL with modern performance-based IADL assays (eg, internet navigation and online banking, virtual reality) and technologically enabled approaches for direct observation (eg, smart homes, wearable sensors), which have shown feasibility and acceptability of these approaches, increased sensitivity to detection of IADL problems, and greater ability to identify specific cognitive processes that lead to difficulties in separable IADL.^9–12

This study also raises important psychometric issues that are relevant to assessing the statistical and clinical relevance of discrepancies between related methods of ascertainment. It is challenging to fully capture meaningful disparities in scores derived from two measures that are both conceptually (ie, they measure the same construct) and practically (ie, they share item structure and content) related. In this instance, there were no significant mean differences between patient- and proxy-rated IADLs and the level of agreement was actually quite good. However, whether scores from these measures are genuinely variant across different raters and/or neurocognitive status will require future work that uses more advanced psychometric approaches, such as measurement invariance.¹³ Clearly, further validation and psychometric work are needed before the EORTC IADL-BN 32 is ready to be implemented in routine clinical practice or as a clinical trial outcome.