Feasibility and utility of a tablet-based digital neurocognitive assessment following radiosurgery for brain metastases

Abstract

Neurocognitive decline (NCD) is a significant concern for patients with brain metastases, yet traditional paper-based assessments are often burdensome. This prospective cohort study evaluated the feasibility and outcomes of a tablet-based digital neurocognitive battery in patients with prior stereotactic radiosurgery (SRS). Data were analyzed from an institutional prospective registry (NCT05504681) including 41 adults with prior SRS. Patients were stratified into a high-risk group (undergoing repeat SRS for progression) and a low-risk group (surveillance only). Neurocognitive function was assessed using the “Cognition” tablet-based platform. NCD was defined as significant deterioration in at least 1 test at 3 months using the Reliable Change Index. The median age was 66 years, and 46% of assessments were conducted in Spanish. Learning and memory were the most frequently affected domains (32%). At 3 months, NCD rates were significantly higher in the high-risk cohort (76%) compared to the low-risk cohort (40%; p = 0.019). On multivariable analysis, risk group status was the only independent predictor of NCD (p = 0.030). Patient feedback indicated high satisfaction and usability. Digital neurocognitive assessment is a feasible and sensitive tool for monitoring patients with brain metastases. The platform effectively differentiated between patients undergoing repeat SRS and those under surveillance, underscoring the importance of routine cognitive monitoring.

Introduction

Brain metastases have debilitating effects on patients with cancer and are commonly associated with baseline neurocognitive impairment¹. Moreover, standard therapeutic approaches for brain metastases including surgery, radiotherapy, and systemic agents, may themselves further contribute to neurocognitive decline (NCD)^2–4. With improved survival due to advances in systemic therapies and cancer care, the detrimental effects of treatments on neurocognitive function (NCF), patient-reported outcomes, and quality of life have become more prominently recognized in the neuro-oncology community^5–8. Mitigating these effects begins with the early recognition of NCD using neurocognitive assessments (NCAs). According to the International Cognition and Cancer Task Force (ICCTF), patients with cancer should undergo ongoing cognitive monitoring over time, with particular attention to areas such as memory, executive skills, and verbal fluency^9,10. However, adherence to NCAs has been limited even in prospective randomized trials, largely due to high dropout rates, as these evaluations are often perceived as burdensome, a concern that is entirely understandable given the progressive decline in patients’ performance status over time^11,12. The use of traditional pen-and-paper assessments may further complicate the process, as they require substantial time and effort^8,13. Although the widespread adoption of digital technologies in healthcare has enabled the practical implementation of NCAs in routine clinical practice,^14,15 reports on the integration of digital NCA tools specifically within neuro-oncology practice remain remarkably scarce. Recent pilot studies have begun to explore this frontier; for instance, Leemans et al. performed a validation study of a tablet-based app to evaluate its test-retest reliability and comparability against traditional pen-and-paper tests in a CNS-healthy cohort¹⁴. Overall, performance of the digital tool showed moderate comparability with a learning effect for attention and processing speed. Butenschoen et al. demonstrated the baseline feasibility of a tablet-based app compared to pen-and-paper tests in neurosurgical patients and healthy subjects¹⁵. While yielding equivalent performance outcomes, the digital approach significantly reduced assessment time. These early reports are largely limited by cross-sectional designs, narrow demographic inclusion, a strict focus on CNS healthy or newly diagnosed, treatment-naive populations, and more importantly a learning effect in assessment of this method, which is well-known downside for digital neurocognitive tests. Consequently, there is a distinct lack of longitudinal data with a reliable assessment metric evaluating how these digital tools perform in complex, real-world clinical scenarios, particularly in the setting of radiation therapy, especially in patients undergoing salvage therapy for intracranial progression. Our group recently established the initial clinical feasibility, multi-language adaptability, and patient acceptability of this tablet-based app using a more accurate assessment formula, which is called “Reliable Change Index Iverson Modification (RCI_Iverson)”,¹⁸ in a patient cohort with brain metastases, demonstrating its ability to detect gross cognitive differences between whole-brain radiotherapy (WBRT) and initial SRS¹⁶. Building upon this, the present study extends the application of this digital tool to a more specific patient population and evaluation of the longitudinal monitoring of patients with a history of prior SRS using a robust evaluation metric (RCI_iverson). Specifically, this investigation advances our prior work by evaluating whether the digital battery is sensitive enough to detect the incremental cognitive burden of repeat SRS for disease progression compared to the baseline cognitive trajectory of patients on active surveillance without subsequent disease relapse. Therefore, we aimed to assess the feasibility and early outcomes of a tablet-based digital neurocognitive battery in patients with prior radiosurgery undergoing repeat treatment, and to compare their performance with that of patients under surveillance.

Methods

Patients

This analysis consisted of patient data from a single-center prospective registry study (NCT05504681), which included adults (≥ 18 years) treated with SRS (stereotactic radiosurgery) or FSRS (fractionated SRS) for brain metastases between December 2021 and March 2024. The Miami Cancer Institute (Baptist Health South Florida) Institutional Review Board (IRB) approved the study protocol (2020-KOT-004) in March 2021. All participants and/or their legal guardians provided written informed consent prior to enrollment into the study. Participants were eligible for this analysis if they had a Karnofsky Performance Status (KPS) of 60 or higher, an estimated survival of at least six months, and had provided informed consent. Patients were excluded if they were younger than 18, pregnant, or incarcerated or they previously received whole brain radiotherapy (WBRT). For the purposes of this study, we included consecutive cases of brain metastases treated with primary SRS/FSRS using the Gamma Knife^® platform (Elekta AB, Stockholm, Sweden). Neurocognitive outcomes were compared between two subgroups: a high-risk group who underwent further SRS/FSRS for distant intracranial progression, and a low-risk group who required no additional brain-directed radiotherapy.

Demographic and clinical data, including sex, race, age, educational background, tumor histology, KPS, systemic therapies received, and treatment details, were retrieved from the institutional electronic health records (Cerner) at the Miami Cancer Institute (Baptist Health South Florida). Radiotherapy-related parameters were obtained via the Leksell GammaPlan^® system and the ARIA oncology platform (Varian Medical Systems, Palo Alto, CA). The study was conducted in accordance with appropriate guidelines/regulations.

Digitized neurocognitive test battery and analysis

Neurocognitive evaluation was conducted using “Cognition,” an innovative tablet-based assessment tool developed by Brainlab AG (Munich, Germany). Hardware (tablets) and software access for the application were provided by the developer as part of the study funding grant; therefore, no additional institutional licensing was required for administration. Designed for multilingual use, the platform offers six language options, English, German, French, Dutch, Spanish, and Brazilian Portuguese, allowing for broad international applicability. The platform’s test-retest reliability and comparability to gold-standard traditional pen-and-paper assessments have been previously validated in healthy and neurosurgical cohorts, and our group recently showed its clinical feasibility specifically in patients with brain metastases^14–16. The battery encompasses a range of cognitive tasks assessing memory, attention, and executive function. These include verbal recall, verbal revision, verbal recognition, words that start with, numbers ordering, numbers and letters ordering, and symbols matching. Most subtests are scored such that higher values represent better performance. However, in the sequencing tasks, shorter completion times indicate superior cognitive function, as scoring is based on task duration in seconds.

First, the test-retest reliability coefficient (r) was calculated using the formula proposed by Hinton-Bayre et al., based on previously published normative retest data from CNS-healthy individuals^14,17. The standard error of measurement at each time point (SEM₁ and SEM₂) was calculated using the standard deviation of test scores at Time 1 and Time 2 (SD₁, SD₂) and the test-retest reliability coefficient (r), according to the formula: SEM = SD × √(1 − r). These values were then used to compute the standard error of difference (SED), using the following equation: SED = √[(SEM₁)² + (SEM₂)²] or equivalently, SED = √[((SD₁ × √(1 − r))² + ((SD₂ × √(1 − r))²]. Next, a Reliable Change Index (RCI) formula controlling for practice effects was calculated using the Iverson method. This approach incorporated individual test scores (T) and control group means (M) at time 1 and time 2, along with previously derived SEDiverson for each test. The formula used was [(T2 - T1) - (M2 - M1)]/SEDiverson.²¹ A prespecified threshold of ± 1.645 was used as the RCI cutoff, consistent with its widespread acceptance in literature¹⁸. A RCI exceeding this threshold (reflecting a 90% confidence level) was considered indicative of a meaningful change on the specific cognitive test and NCD was subsequently defined as a significant deterioration in performance on at least 1 out of the 7 cognitive tests, in accordance with the previously reported criteria¹⁹.

Procedure

Neurocognitive assessments were conducted in a quiet, distraction-free environment, with testing supervised by trained research personnel. To ensure confidentiality, each participant’s personal identifiers were removed and replaced with a unique study ID assigned at the time of enrollment. Prior to initiating the formal assessment, participants received verbal instructions and completed a brief practice session to ensure familiarity with the testing interface. The full assessment typically required 15 to 20 min to complete, depending on the individual’s pace and performance. All de-identified data were securely transferred to an external cloud-based platform, with compiled datasets retrieved through monthly reports for further analysis.

Statistical analysis

To address missing data, a complete case analysis approach was utilized; therefore, any patients with incomplete data for the primary variables of interest were not included in this analysis. Statistical comparisons between the high-risk and low-risk cohorts were carried out using non-parametric and categorical tests, depending on variable type. Specifically, the Chi-square test was applied for categorical comparisons, while the Mann–Whitney U test was used to assess differences in continuous variables. Results for categorical data are expressed as counts and percentages, whereas continuous variables are reported as medians with interquartile ranges (IQRs).

All participants completed baseline neurocognitive testing via a tablet-based assessment platform. Follow-up assessments were conducted longitudinally, beginning around the time of first imaging follow-up at 8 weeks with subsequent imaging timing adapted as necessary based on patient specific factors but typically at 2–3 month intervals. Follow-up scores were analyzed relative to baseline to evaluate changes in cognitive function.

Within the SRS/FSRS-treated population, variables demonstrating a potential association in univariable analysis (p < 0.1) were subsequently included in a multivariable model to identify independent predictors. All statistical procedures were executed using SPSS software (version 27.0; IBM Corp., Armonk, NY), with two-tailed significance testing and a predefined alpha threshold of p < 0.05.

Results

Forty-one WBRT-naive patients who had baseline NCAs between December 2021 and June 2024 were included in this analysis. All patients had at least one course of previous SRS/FSRS (IQR: 1–1) with only four patients having > 1 (three patients had two courses; one patient had five). At the time of baseline NCA, 21 high-risk patients were treated with repeat SRS/FSRS for new lesions while 20 low-risk patients did not receive any type of radiotherapy until the 3-month NCA. The median age was 66 years (interquartile range [IQR]: 59–76), 66% were female, 56% had at KPS of ≥ 90, and 51% had a high school/equivalent or less level of education, 46% were evaluated in Spanish, and 93% had subsequent systemic therapy, 20% underwent previous craniotomy, and 81% had lung primary, without any significant differences between the cohorts (p > 0.05). However, time from previous SRS/FSRS and time from initial brain metastases diagnosis to baseline NCA was significantly longer in the high-risk group compared to low-risk group, 5.4 months (IQR: 2.8–8.7 months) and 6.8 (IQR: 4.5–13.7) vs. 2.4 months (IQR: 2.1–3.3 months) and 3.4 (IQR: 2.8–4.0.8.0), respectively, p < 0.001 and p = 0.001 (Table 1).

Table 1.

Baseline demographic, clinical, and treatment characteristics of patients in the low-risk and high-risk groups.

Characteristics	Low risk group n = 20	High risk group n = 21	p value
	n (%)/median (IQR)
Age (y)	64.5 (58.5–74.2)	70 (59.0–77.0)	0.506
Gender			0.152
Female	11 (55.0%)	16 (76.2%)
Race/Ethnicity			0.819
Hispanic/Latino	14 (70.0%)	14 (66.7%)
Other	6 (30.0%)	7 (33.3%)
Education			0.437
HS equivalent or less	9 (45.0%)	12 (57.1%)
Higher than HS	11 (55.0%)	9 (42.9%)
Language			0.647
Spanish	10 (50.0%)	9 (42.9%)
English/other	10 (50.0%)	12 (57.1%)
KPS	90 (80–90)	90 (80–90)	0.838
Time from initial diagnosis to baseline NCA (mo)	9.7 (4.1–21.5)	16.6 (9.9–34.5)	0.081
Time from initial brain metastases diagnosis to baseline NCA (mo)	3.4 (2.8–4.0.8.0)	6.8 (4.5–13.7)	0.001
Time from previous SRS/FSRS (mo)	2.4 (2.1–3.3)	5.4 (2.8–8.7)	< 0.001
Previous craniotomy, yes	3 (15.0%)	5 (23.8%)	0.477
Subsequent systemic therapy, yes	19 (95.0%)	19 (90.0%)	1.000
Lung primary	17 (85.0%)	16 (76.2%)	0.697
Radiosurgery details at the time of baseline NCA
Number of lesions treated	N/A	2 (1–5.5.5)	N/A
Median CITV of lesions treated per session (cc)	N/A	0.37 (0.07–2.45)	N/A
Median SRS/FSRS dose	N/A	24 Gy (24–24)	N/A
Median number of fractions	N/A	1 (1–1)	N/A

SRS, stereotactic radiosurgery; FSRS, fractionated SRS; IQR: Interquartile range; HS, High school; KPS, Karnofsky performance status; y, year; mo, month; CITV, cumulative intracranial tumor volume; cc, centimeter cube; NCA, neurocognitive assessment; N/A, not available.

For the entire cohort, learning and memory was the most affected domain (13/41, 32%), followed by attention and information processing speed (8/41, 20%), executive function (7/41, 17%), visual-motor speed (3/41, 7%), and verbal fluency (3/41, 7%). The decline rates were also stratified by treatment groups (Fig.1). At the 3-month NCA evaluation, NCD rates on ≥ 1 neurocognitive test were significantly higher for the high-risk cohort (16/21, 76%) compared to the low-risk cohort (8/20, 40%, p = 0.019). In univariable analysis, risk group (low-risk vs. high-risk, p = 0.022) and education status (higher than high school vs. high school or less, p = 0.090) were associated with NCD. In multivariable analysis, only risk group remained significantly associated with NCD (p = 0.030).

Fig. 1.

Neurocognitive decline rates by cognitive domain. Distribution of neurocognitive decline across individual domains.

Most participants reported participating in the neurocognitive program improved discussions with their clinicians (69.2%), helped them feel more in control of their own care (74.4%), and 94.9% recommended the system to other similar patients (Fig. 2). Differences in raw neurocognitive scores between baseline and 3-month follow-up are presented in Fig. 3a–c.

Fig. 2.

Patient-reported experience with the digital neurocognitive assessment tool. Responses were captured using a 5-point Likert scale and presented as percentage distributions.

Fig. 3.

The figures display the difference in neurocognitive scores from baseline to 3-month follow-up for (a) Verbal Recall, (b) Verbal Revision, and (c) Verbal Recognition.

Discussion

In this study, our novel tablet-based digital neurocognitive battery effectively distinguished patients at high risk for cognitive decline, namely those undergoing repeat stereotactic radiosurgery (SRS/FSRS), from those at lower risk who did not receive additional radiotherapy. Notably, at the three-month follow-up, the high-risk group exhibited significantly higher rates of NCD (76%) compared to the low-risk group (40%; p = 0.019). These findings underscore the practical utility of digital neurocognitive assessment tools in identifying NCD across patient populations with varying levels of neurocognitive risk.

Interestingly, a substantial proportion of patients within the low-risk cohort, who did not undergo further radiosurgery, also experienced cognitive deterioration. Several mechanisms could underlie this observation. First, given the sensitivity of this multi-domain NCA, it is possible that subtle NCD was detected using this test more often than is typically observed in clinical practice. Second, it is plausible that detrimental effects on neurocognition from previous radiosurgical treatment could continue to manifest over time even in the absence of repeat CNS interventions, though some recovery may be observed with long-term follow-up.¹⁹ Third, the NCD can be multi-factorial, and systemic therapies, especially those with intracranial penetration can independently contribute to cognitive impairment^4,20. Overall, our findings are consistent with earlier research reporting NCD following radiosurgery. Specifically, multiple randomized controlled trials using traditional paper-based cognitive assessments have shown that 22% to 63% of patients undergoing SRS/FSRS exhibit cognitive deterioration within 3 to 6 months after treatment^21–24. Additionally, the domain most frequently affected was learning and memory, with a decline observed in approximately 32% of cases, an observation that also aligns with the previously published literature²⁵.

The novel aspect of our work lies in the utilization of a digital, tablet-based platform, which addresses the practical barriers frequently encountered with traditional pen-and-paper assessments^11,12. Adoption of digital platforms in healthcare has rapidly advanced, offering streamlined solutions to everyday clinical operations across disciplines^26,27. In neuro-oncology, a major hurdle remains: practical and scalable assessment of cognitive function in routine care^13,28. Digital neurocognitive assessment systems now provide clinicians with integrated, real-time evaluations of well-established cognitive domains in a variety of languages¹⁴. In our study, 46% of assessments were conducted in Spanish, a key advantage considering the significant proportion of individuals in the United States who identify as Hispanic or speak Spanish as a native language²⁹. The digital platform demonstrated strong usability and acceptability, with high completion rates and favorable patient feedback indicating reduced perceived burden. Remarkably, these outcomes were observed in an older patient cohort, with a median age of 66 years.

Nonetheless, several limitations of our study should be acknowledged. First, the relatively small sample size restricts the ability to make definitive inferences regarding causal relationships. Additionally, heterogeneity in patient characteristics and systemic treatments represent significant confounding factors that were not statistically corrected for in this analysis. Because these findings are based on a relatively small convenience sample with short-term follow-up and uncorrected confounders, the specific neurocognitive decline rates reported here should be considered strictly exploratory; they must not be used to guide clinical decision-making or alter current therapeutic discussions. Rather, the ultimate goal and primary value of this study lie in demonstrating the operational viability of the digital software within a busy clinical workflow, identifying the logistical challenges of implementation, and highlighting the overwhelmingly positive patient experience and acceptability of the platform. Future research would benefit from a larger, multicenter design with better control of systemic therapy to more clearly isolate the cognitive effects of radiosurgery on NCD. Another important consideration is that digital neurocognitive assessments utilize entirely different stimuli and user interfaces compared to traditional paper-based methods, making direct comparisons challenging. Additionally, digital neurocognitive testing is subject to inherent psychometric biases, prompting the development of various evaluation methods to mitigate these effects³⁰. The most prominent of these biases is the practice or learning effect, where patients may artificially improve on subsequent assessments simply due to test familiarity and repeated exposure, rather than true cognitive stability¹⁴. To explicitly mitigate this specific bias, we applied the Iverson modification of the Reliable Change Index (RCI). This approach adjusts for expected practice effects by incorporating test-retest data from normative control populations to calculate a baseline of expected improvement. However, we acknowledge that while this modification of the RCI is a robust statistical correction, practice effects may not be completely eliminated, and subtle residual learning effects could still influence individual performance trajectories. Furthermore, because this study represents a specific subgroup analysis of a broader prospective neurocognitive registry, no formal a priori sample size calculation was performed. The cohort represents a convenience sample of all eligible patients treated during the study period; consequently, the relatively small sample size limits our statistical power, and the findings should be interpreted as exploratory and hypothesis-generating.

Conclusion

In summary, our study demonstrates that digital neurocognitive assessments provide a feasible, efficient, and sensitive approach for monitoring cognitive outcomes in differentiating patients undergoing repeat radiosurgery and those who do not. These findings underscore the importance of routine cognitive monitoring in neuro-oncology practice, emphasizing that cognitive decline may occur even in patients who are not actively undergoing further CNS directed radiotherapy. Thus, integrating digital neurocognitive assessments into clinical practice holds promise for better managing cognitive health and patient-reported outcomes in this vulnerable patient population.

Author contributions

Authorship statement. Conception/Design: EYA, RK; Data Collection/Analysis/Interpretation: EYA; Statistical Analysis: EYA; Drafting of the Manuscript: EYA, RK; Manuscript Editing/Critical Revision of the Manuscript: All Authors.

Funding

Funded by Brainlab Inc.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due being part of an investigator initiated prospective study, but may be available from the corresponding author on reasonable request (RK).

Declarations

Competing interests

EYA, JD, TCR: None, YO: Consulting fees from Istari Oncology, PharPoint Research; Participation on a Data Safety Monitoring Board or Advisory Board in Actuate, GammaTile, Novocure; other financial or non-financial interests in Blue Earth Diagnostics, BMS, Cantex Pharmaceuticals, Chimerix, CNS Pharmaceuticals, Exelixis, Karyopharm, MimiVax LLC, VBI Vaccines. RHP, DJW, YCL, RT: None, ANG: Honoraria from ViewRay, Inc., Elekta AB, IBA; Ownership interest of Atlantic Health Solutions (uPlan), LLC., MWM: Stock shareholder in Light Helmets, Deinde Medical; honoraria from Insightec; consulting fees from Stryker, Diende, Light Helmets, Insight tech, Zap surgical., MDH: Grant support from Florida Department of Health, Grant 22L01. Proton Therapy Cooperative Group of North America (PTCOG-NA) Treasurer (unpaid); Proton Therapy Cooperative Group (PTCOG) Pediatric Subcommittee Co-Chair (unpaid); Proton Collaborative Group (PCG) Executive Council Representative for Miami Cancer Institute (unpaid). , MPM: Consulting Fees from Telix, Zap, Xoft, Kazia Therapeutics, Novocure; Mevion Technological Advisory Board; NRG Brain Tumor Committee Chair; Leadership (unpaid) Board of Directors, Xcision; Stock in Chimerix., RK: Honoraria from Elekta AB, Accuray Inc., Novocure Inc., ViewRay Inc., Elsevier Inc., Brainlab, Peerview Institute for Medical Education, GT Medical Technologies, and Ion Beam Applications; consulting fees from Kazia Therapeutics, Elekta AB, ViewRay Inc., Castle Biosciences, Novocure Inc.; institutional research funding from Medtronic Inc., Blue Earth Diagnostics Ltd., Novocure Inc., GT Medical Technologies, AstraZeneca, Exelixis, ViewRay Inc., Brainlab, Cantex Pharmaceuticals, Kazia Therapeutics and Ion Beam Applications; support for travel or meeting attendance by Elekta AB, Accuray Inc., Novocure Inc., Peerview Institute for Medical Education, Brainlab, ViewRay Inc.; and participation on an advisory board for Viewray Medical Advisory Board, GT Medical Technologies Data Safety Monitoring Board, Insightec Ltd, Plus Therapeutics, Inc.

Institutional review board statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of the Miami Cancer Institute, Baptist Health South Florida (protocol code 2020-KOT-004; March 9, 2021). All participants provided written informed consent prior to enrollment.