Assessing cancer-related cognitive function in the context of everyday life using ecological mobile cognitive testing: A protocol for a prospective quantitative study

Abstract

Objective

Millions of cancer survivors are at risk for cancer-related cognitive impairment (CRCI), yet accurate and accessible assessments of cognitive functioning remain limited. Ecological mobile cognitive testing (EMCT) could offer a solution. This paper presents the protocol for a study that aims to (1) establish the reliability and validity of EMCT to assess CRCI in breast cancer survivors, and (2) prospectively evaluate within-person processes (and interactions) among context, mood, and behavior that explain cognitive variability, everyday functioning, and quality of life of cancer survivors.

Methods

Participants will include breast cancer survivors (>21 years old) who are within 5 years of completing chemotherapy treatment. Participants will complete two virtual visits (baseline, follow-up) 2 months apart to assess self-reported cognitive symptoms and cognitive performance, sociodemographic characteristics, clinical history, everyday functioning, and quality of life. Between virtual visits, EMCT will be used to sample cognitive functioning every other day (28 times total). We will use linear mixed-effect regressions and single-level multiple regression models to analyze the data.

Results

We anticipate a minimum of 124 breast cancer survivors enrolling and completing data collection. Study results will be published in peer-reviewed scientific journals.

Conclusions

Our findings will have broad implications for assessing CRCI in an ecologically valid and person-centered way using EMCT. We aim to provide this protocol to aid researchers who would like to apply this approach to their studies.

Introduction

Up to 75% of cancer survivors have cancer-related cognitive impairments (CRCI), which typically manifest as post-treatment difficulties with attention, executive function, memory, and processing speed.^1–6 CRCI can have devastating effects on the daily functioning of survivors^7,8 and are associated with reduced quality of life (QOL), poor social and occupational function, and decreased survival.^9,10 Evidence suggests that CRCI are subtle, diffuse, and vary considerably from person to person and across different settings. ¹¹ Long-term trajectories of CRCI are variable—some will improve over time, others remain unchanged, and some progressively worsen or evolve into more severe cognitive disorders.^5,12–14 Despite the millions of cancer survivors at risk for CRCI, ¹⁵ accurate and accessible assessments of cognitive functioning remain limited (e.g. high cost, low access, high demand).

In cancer patients and survivors, CRCI are typically measured using standardized neuropsychological tests ¹⁶ and patient-reported outcomes (PROs) of cognitive function. ¹⁷ Specific cognitive tests are recommended for the quantification and identification of CRCI. ¹⁶ However, standardized neuropsychological measures were originally developed for other, typically more severely neurologically compromised populations (e.g. post-stroke, dementia, traumatic brain injury) and thus have been criticized for having limited sensitivity, specificity, and reliability for CRCI.^18–21 Standardized neuropsychological tests also likely lack ecological validity.^22,23 They are typically administered in lab or research settings and rarely correlate with (or predict) functioning on everyday tasks, occupational outcomes, QOL, or well-being in cancer patients.^24–27 On the other hand, cognitive PROs used to assess CRCI often do; however, correlate with everyday cognitive functioning and QOL,^{2,9,19,24,26,28–33} and several were developed for use specifically in cancer populations.^34,35 One limitation of cognitive PROs is a reliance on retrospective recall, which can bias data. They have also been criticized for not correlating with standardized cognitive tests.^22,24,26

Real life context is a critical component of cognitive functioning not directly addressed by traditional assessments (i.e. standardized cognitive tests, cognitive PROs). ³⁶ In real world environments, one's cognitive performance is highly influenced by many state-dependent factors (e.g. mood, energy) and contexts (e.g. a cognitively demanding activity, distractions) that individually, or in combination, interfere with cognitive functioning. Research findings, including our own, have emphasized the importance of these day-to-day variations in cancer survivors, documenting survivors’ critiques that retrospective cognitive outcome measures may not capture their cognitive fluctuation.^9,37 To date, use of ecologically valid cognitive assessments is scarce in the field of CRCI, which limits our understanding of the nature, trajectory, and impact of this clinical problem.

Ecological momentary assessments (EMAs) allow researchers to directly capture within-person variation in, and interactions among, within-person behavioral and cognitive processes, ³⁸ and are most often delivered via mobile technologies such as smartphones. Mobile cognitive testing can be coupled with EMA to objectively measure cognitive performance in people's natural environments—what we term ecological momentary cognitive testing (EMCT). Use of EMCT in research studies is on the rise, and feasibility, validity, and reliability of EMCT has been established in several cognitively vulnerable populations, including people with serious mental illness, mood disorders, and mild cognitive impairment.^39–43 However, EMCT studies in the field of CRCI are only recently gaining traction.

Small et al. ⁴⁴ examined associations between fatigue and depressed mood and cognitive variability in 47 breast cancer survivors (BCS) over 14 days using EMCT. They reported reliable and valid within- and between-person EMCT measures and found predictive relationships between fatigue and cognitive performance on days when fatigue was worse. They also reported that over half of the variability in cognitive performance was attributed to within-person variations rather than between-person variations. ⁴⁴ This group also reported that survivors were 3 times more likely to experience memory lapses associated with negative affect compared to a control group. ⁴⁵ This work provides preliminary evidence that within-person variability outweighs between person variability for cognitive measures in CRCI, and that negative affect and worse fatigue are predictive of cognitive performance in BCS. However, questions remain regarding how well these EMCTs map onto more traditional measures for CRCI (standardized cognitive testing, PROs), and what within-person processes (and interactions) among context, mood, and behavior explain cognitive variability in survivors with CRCI.

In this study, our primary aim to establish the reliability and validity of EMCT measures in BCS, including their relations to performance-based measures of executive functioning, attention, processing speed, and memory, as well as self-reported EMAs of psychological and cognitive symptoms. Our secondary aim is to quantify longitudinal relationships among contextual factors (i.e. daily activities and self-reported cognitive function) and EMCT performance within BCS. Finally, our exploratory aim is to determine how within-person variation in EMCT predicts everyday functioning and QOL of BCS across time.

Methods

Design

This will be a remote, prospective observational study of BCS within 5 years of completing chemotherapy treatment. We will collect baseline and follow-up data using traditional surveys delivered via REDCap (Vanderbuilt, TN) and remote cognitive assessments delivered via BrainCheck (BrainCheck, Inc.) 2 months apart. Baseline data collection will take approximately 60 minutes to complete (45 minutes of surveys, 15 minutes of cognitive tests) and follow-up data collection will take approximately 45 minutes to complete (30 minutes of surveys, 15 minutes of cognitive tests). Procedures will be explained to participants verbally upon enrollment, and written instructions for completing the REDCap survey and BrainCheck batteries will be provided via email. BrainCheck batteries include video instructions prior to starting testing. Video conference appointments will be offered to assist participants with data collection as needed. As of June 14, 2023, we have enrolled 14 participants, and all have seamlessly completed baseline data collection; none have requested or required assistance.

We will administer an EMCT protocol for 56 days (8 weeks), every other day in between baseline and follow-up data collection (28 assessments total) via participants’ smartphones using the NeuroUX platform (https://www.getneuroux.com/). Daily assessments take approximately 10 minutes to complete. Study start-up activities began in March 2023 and recruitment and data collection began in May of 2023. We anticipate data collection will be completed by September 2024, and final data analyses to be completed by January 2025. See Figure 1 for protocol data collection timeline for each participant.

Figure 1.

Protocol data collection timeline. Abbreviations: D: day; Hx: history; PROs: patient-reported outcome measures. Baseline data collection will take approximately 1 hour; follow-up data will take approximately 45 minutes; EMCT every other day will take approximately 10 minutes/day. Total time for each participant in the protocol is 8 weeks.

Ethics and Institutional Review Board

All study related procedures have been reviewed and are overseen by the University of Texas at Austin Institutional Review Board (STUDY00002393) and were determined to be exempt, thus written consent was not deemed necessary. The potential risks associated with this study entail psychological risks (fatigue related to cognitive testing and data collection burden) and privacy and/or confidentiality risks. We do not expect that these risks will be greater than those experienced in meeting the cognitive, physical, and emotional demands of everyday life for cancer survivors. The primary benefit to participants is that they will help gain new knowledge for the clinical and practical care of BCS. While not a benefit, participants will receive a small incentive (up to $78) for participating in the study ($25 for baseline data, $25 for follow-up data, $1/day of EMCT protocol) and reports of their scores on EMCT gamified cognitive tests after completing their protocols.

Study population inclusion and exclusion criteria

We will recruit and enroll 124 female BCS, aged 21 years and older, within 5 years of finishing chemotherapy who are willing and physically/cognitively able to participate in data collection (Karnofsky Performance score > 70; Mini Moca Score > 12).^46,47 CRCI onset is common after chemotherapy completion, with trajectories of improvement/decline emerging during this period.^24,48 We aim to capitalize on this within- and between-person variability in cognitive functioning to achieve the aims of this study. BCS currently on hormonal therapies or HER-2 targeted therapies will be included. Participants will be excluded for any prior history of systemic cancer treatment (other than breast cancer diagnosis and treatment) as cancer recurrence and multiple cancer diagnoses requiring systemic treatment could confound cognitive functioning. Those with a healthcare provider diagnosed serious psychiatric or neurological conditions (e.g. dementia, active substance abuse, unmanaged mental health diagnoses), current metastases to the brain, or those who are pregnant will also be excluded as these diagnoses are known to impact cognitive functioning. It is estimated that 85% of the U.S. population has a smartphone ⁴⁹ and participants will be asked to use their personal smartphones for this study; however, if participants do not own a smartphone or do not wish to use their personal phone, we will provide them with one for the duration of the study. We will recruit participants both locally in Central Texas and Southern California, and nationally (via Social Media platforms), since all data collection will be conducted remotely. We plan to enroll a demographically representative sample of BCS (see Table 1 for Planned Study Enrollment Table based on race and ethnicity, and sex).

Table 1.

Planned study enrollment table.

Racial categories	Ethnic categories
	Not Hispanic/Latina		Hispanic/Latina		Total
	Female	Male	Female	Male
American Indian/Alaska Native	2	—	—	—	2
Asian	12	—	—	—	12
Native Hawaiian/Pacific Islander	2	—	—	—	2
Black or African American	13	—	3	—	16
White	63	—	26	—	89
More than One Race	2	—	1	—	3
Total	94	0	30	0	124

Instruments

The corresponding author can be contacted to request a complete packet and references of all study measures that are not proprietary.

Sociodemographic and clinical variables

A questionnaire will be used to collect sociodemographic characteristics (e.g. age, education, race, ethnicity, marital status, children/dependents, income, employment), health history (e.g. comorbidities, menstrual history, current medications) and cancer history (e.g. breast cancer type/stage, cancer treatment details, end date of chemotherapy) at baseline.

Cognitive function: At baseline and follow-up we will administer the Functional Assessment of Cancer Treatment Cognitive Function version 3 (FACT-Cog) to assess cognitive symptoms in the previous 7 days in four domains—perceived impairments, perceived abilities, QOL, and comments from others. ⁵⁰ The FACT-Cog Perceived Cognitive Impairment subscale will be used in data analyses and is an often used measure to capture cognitive symptoms in this population. We will administer a computerized battery of standardized neuropsychological tests (BrainCheck) to assess cognitive performance, including: the Flanker Test for visual executive attention, the Trail Making Test for executive function, the Digit Symbol Substitution Test for processing speed, the Stroop Test for response inhibition, and the recall test (list learning) for verbal memory. ⁵¹ BrainCheck administration includes sending participants detailed instructions for accessing and using the platform, including an introductory video, and study staff being available to assist via video conference if needed. BrainCheck has demonstrated high sensitivity and reliability ⁵² for mild cognitive impairment ⁵³ and is a novel tool for efficient and remote cognitive testing. We will use standardized domain-specific scores in the analyses.

Mood, anxiety, fatigue: At baseline and follow-up, we will administer the Patient-Reported Outcomes Measurement Information System (PROMIS) scales—Emotional Distress—Anxiety Short Form 8a (“PROMIS Anxiety”), Emotional Distress—Depression Short Form 8a (“PROMIS Depression”), and PROMIS Fatigue Short Form 8a (“PROMIS Fatigue”). ⁵⁴ These instruments are psychometrically sound and have demonstrated strong reliability in our previous studies with BCS (reliabilities 0.93–0.96).^54,55 Raw total scores will be used in the analyses.

Everyday functioning and quality of life. At baseline and follow-up, we will administer the social difficulties inventory (SDI) to comprehensively evaluate social/occupational functioning in the previous week. ⁵⁶ We will thus operationalize “everyday functioning” as social/occupational functioning and social/occupational satisfaction. The SDI was recommended in a systematic review of instruments used to measure social impact in cancer patients. ⁵⁷ This 21-item instrument, developed for oncology populations, demonstrates validity and reliability. ⁵⁸ The PROMIS Satisfaction with Participation in Social Roles—Short Form 8a (“PROMIS Social”) will also be administered to capture satisfaction with one's ability to participate in their social/occupational roles. This instrument has demonstrated measurement invariance, internal reliability, validity, and no ceiling effects in BCS. ⁵⁹ The FACT-General (G) will be used to measure domains of QOL and well-being (physical; social; family; emotional; functional) in the previous week. ⁶⁰ The FACT-G was chosen based on its widespread use in oncology research^61,62 including CRCI specific research. ⁶³ Total scores will be used in the analyses.

EMCT protocol

EMCT, including EMA surveys and gamified mobile cognitive tests, will be administered via NeuroUX and texted to participants via a web link every other day between baseline and follow-up data collection (total of 28 times across the 56-day protocol). EMCT administration time will occur between participant's average waking and bedtime, and vary across morning, midday, and evening throughout the study period. The session times are personalized for each participant based on their availability and preferred times. The research team inputs the earliest and latest session times for each participant in the Investigator dashboard. A random time is used within that range for each of the 28 sessions for that participant. We send one reminder notification if a session is not completed within an hour of the initial notification and the sessions will expire 2 hours after the notification is sent if not completed. The session times are thus varied pseudorandomly for each participant based on the time ranges inputed by the research team. The time intervals between sessions for each participant also vary since the sessions are administered on alternative days. NeuroUX EMCT has demonstrated reliability and validity in middle-aged and older adults, with and without cognitive impairment.^39,42,43

Each EMCT session will query current daily activity using a list of items that are categorized as cognitively demanding activities, passive leisure activities, instrumental activities of daily living, physical activity, social activities, and other activities as previously described. ⁶⁴ Each session will also collect single-item Likert-type ratings for “depressed/ sadness,” “anxiety,” “fatigue,” and “cognitive functioning” that assess how bad/good each symptom is “right now.” Response options will range from “Not at all” (0) to “Extremely” (7), consistent with our previous EMCT studies. These questions were adapted from an EMA study with BCS. ⁴⁴ After daily activity and symptoms are assessed, 4 mobile cognitive tests will be administered, each approximately 2-minute long, in the domains of attention, executive functioning, processing speed, and verbal/spatial memory. These domains were chosen as they are the most common domains impacted by CRCI. ¹

Two different tests will be administered for each cognitive domain and balanced throughout the protocol (14 times each). For executive attention, the N-Back (using a 2-back design, 12 trials each test) and the CopyKat tasks will be used. The N-Back is a common task that requires the participant to remember if a certain number was displayed 2 trials back. The CopyKat is similar to the popular electronic game Simon, where the participants are presented with a 2 × 2 matrix of colored tiles in a fixed position. The tiles briefly light up in a random order, and participants are asked to replicate the pattern by pressing on the colored tiles in the correct order. The time to complete is variable, but 3 trials take approximately 2 to 3 minutes. For executive functioning, the Color Trick and the Hand Swype tasks will be used. The color trick asks the participant to match the color of the word with its meaning, with 15 trials each test. The Hand Swype test asks the participant to swipe in the direction of a hand symbol or in the direction that matches the way the symbols are moving across the screen. Hand Swype is a time-based task. The test will continue for 1 minutes, no matter if responses are correct or incorrect.

For processing speed, the matching pair and quick tap tasks will be used. For matching pair, the participant is asked to quickly identify the matching pair of tiles out of 6 or more tiles. Matching pair is a time-based task and runs for 90 seconds. The grid size increases based on correct responses and the max grid size is 4 × 4 tiles. For quick tap, the participant is asked to wait and tap the symbol when it is displayed. Twelve trials are administered each test. For memory, the variable difficulty list memory test (VLMT) and Memory Matrix task will be used. For the VLMT, participants are provided with a list of 12–18 random words and given 30 seconds to memorize the list. Then they are asked yes/no questions to determine if words were on the list or not, immediately following the memorization time. For memory matrix, patterns are quickly displayed to the participant, then they are asked to indicate the pattern that was displayed by touching the tiles that were in the pattern. This test gets progressively harder if responses are correct: the first trial starts with a 2 × 2 grid size and the grid size can go up to 7 × 7 tiles until the participant gives three incorrect responses.

Data analysis plan

Data preprocessing

We will use valid and complete EMCT data to first calculate day-specific and aggregate mobile cognitive test indices for longitudinal and cross-sectional analyses, respectively. Day-specific mobile cognitive test indices capture a participant's mobile cognitive test performance on a given day and include their total score and/or median reaction times across trials for each task. Aggregate mobile cognitive test indices are calculated by collapsing day-specific indices over the entire length of the study period to measure participants’ average and day-to-day variability in task performances over time. Domain-specific cognitive composite scores will be calculated (i.e. executive attention, executive functioning, processing speed, memory) for both day-specific and aggregate mobile cognitive test indices.

Aim 1: To establish the reliability and validity of EMCT administration in BCS

We expect that EMCTs will demonstrate good levels of reliability and convergent validity. We will use generalizability theory to evaluate the reliability of between-person individual differences and within-person change for each symptom assessment and mobile cognitive task. ⁶⁵ We will then compute average within-person ratings for each EMA-reported symptom (depressive symptoms, anxiety, fatigue, and self-rated cognitive functioning) across the 28 days. Convergent validity for the EMCT symptom assessments will be evaluated by correlating the average within-person EMA-reported symptom severities with the baseline and follow-up PROMIS depressive, PROMIS anxiety, PROMIS fatigue, and FACT-Cog scores. Moreover, convergent validity for the aggregate mobile cognitive test indices will be evaluated by correlating them with the baseline and follow-up BrainCheck test scores for Flanker (attention), trail making test B and Stroop test (executive functioning), digit symbol substitution (processing speed), and immediate recall scores (memory).

Aim 2: To determine longitudinal relationships among contextual factors and EMCTs across time

We expect that self-rated cognitive functioning will predict same day mobile cognitive test performance (Hypothesis 2a) and that type of daily activity (cognitively demanding versus not) will predict both the self-rating for cognitive functioning and mobile cognitive test performance on the same day (Hypothesis 2b).

To test Hypothesis 2a, we will use linear mixed-effect regressions to model the within-person concurrent relationship between EMCT self-rated cognitive functioning and same-day domain-specific cognitive test performance. We will covary for participants’ average self-rating for cognitive functioning throughout the study duration, as this appropriately differentiates within-person versus between-person effects. All within-person variables will be person-mean centered, and all between-person variables will be grand-mean centered. Participant-specific random intercepts will be modeled. Age, education, menopausal status, and time since chemotherapy will be considered as between person covariates if they are associated with average mobile cognitive test performance at p < .10.

To test Hypothesis 2b, we will use linear mixed-effect regressions to model the within-person concurrent relationships between: (1) type of daily activity and same-day self-rated cognitive functioning, and (2) type of daily activity and same-day MCT performance. We will covary for the proportion of surveys on which participants endorsed doing a cognitively demanding activity to appropriately differentiate within-person versus between-person effects. Participant-specific random intercepts will be modeled. Age, education, menopausal status, phone type (iOS; Android), and time since chemotherapy will be considered as between person covariates if they are associated with average self-rating for cognitive functioning or average mobile cognitive test performance at p < .10.

Aim 3: To determine how within-person variability in EMCT predicts everyday functioning and quality of life of BCS across time

We expect that within-person variability in EMCT variables will predict cognitive PROs (Hypothesis 3a), social functioning (Hypothesis 3b), and QOL (Hypothesis 3c) at follow-up. We will calculate the mean square of successive differences (MSSDs) as our measure of within-person variability for each EMCT variable (symptoms of depression, anxiety, fatigue, cognitive dysfunction, and domain composite scores for mobile cognitive test performance). Then three separate multiple linear regressions will examine the relationship between MSSDs for EMCT variables (entered simultaneously in the models as independent variables) and follow-up PROs (FACT-Cog perceived cognitive impairment subscale [Hypothesis 3a], SDI [Hypothesis 3b], FACT-G [Hypothesis 3c]) covarying for age, education, time since chemotherapy, and menopausal status.

Sample size determination

Monte Carlo simulations were used to conduct power analyses for the longitudinal hypotheses specified in Aim 2, which use linear mixed-effects regression. ⁶⁶ The “SIMR” package in R (version 4.0.5) was used to run 1000 simulations with a two-sided alpha of .0125 (Bonferroni adjusted for 4 cognitive domain scores), 28 data points per participant, and a medium effect of f²= 0.15, which yielded a sample size of 109 at 80% power. The medium effect size was based on our previous study that found longitudinal relationships between self-report and cognitive performance using EMCT ⁶⁷ and another cross-sectional study reporting relationships between subjective and objective cognitive outcomes in BCS with CRCI. ⁶⁸ We will oversample to account for a 13% attrition rate (N = 124). The attrition rate was informed by another prospective study we conducted with BCS (16-week protocol). ³⁷ For Aim 3, a power analysis was conducted using an estimated a medium effect (f²= 0.2), a Bonferroni adjusted α (.05/3 = .017), 13 tested predictors in a linear regression model, and a sample size of 124, which yielded 80% power.

Discussion

Principal findings

This study will provide essential psychometric data for using EMCT to assess cognitive functioning of BCS in the context of everyday life that can be used for future observational or interventional CRCI research. Neuropsychological testing for domain-specific cognitive functioning remains the most common assessment tool for CRCI measurement. ⁶⁹ In recent years, efforts have been made to translate these traditional paper and pencil tests into digital formats for remote administration (e.g. Cogsuite, ⁷⁰ Cogstate, ⁷¹ NIH Toolbox, ⁷² BrainCheck ⁷³ ). These platforms can ease burdens related to in-person data collection for both participants and researchers and may be more sensitive to CRCI detection than paper and pencil testing.^70,74 However, they are typically administered in place of traditional neuropsychological testing and under similar circumstances (i.e. in quiet/non-distracting environments, with administrator oversite via phone or videoconference) and at similar frequencies (i.e. weeks to months), limiting ecological validity and ability to capture frequent variations in cognitive functioning. Findings from this study will also provide new insights on assessing both the within- and between-person day-to-day variations in cognitive functioning of BCS in an ecologically valid way (using EMCT). Traditional measures of CRCI (standardized cognitive tests; PROs) are likely not administered frequently enough, or in ecologically valid environments, to capture the dynamic changes and variability of cognitive function and may be a contributing factor to missed or under diagnosed CRCI in cancer survivors.

To be inclusive, we will sample BCS from a large age range, limiting internal validity but strengthening external validity. We will control for age in our analyses and collect other age confounders (e.g. menopausal status; medications; comorbidities) to control for these effects if needed. We acknowledge the possibility of recruitment difficulties. To facilitate study enrollment/ retention, we will collect all data remotely on participants’ own schedules (except EMCTs which must be completed within 2 hours of receiving the text). We have also decreased study demands by sending assessments only one time per day rather than multiple times per day. We will use a commercially available platform for EMCT administration, open-source statistical software for analyses (R studio) and include STROBE checklists ³⁶ (recommended for disseminating EMA data) when disseminating study findings.

This study will provide evidence for context-specific factors associated with worse or better cognitive functioning, and for using within-person cognitive variations to predict everyday functioning and QOL of survivors. This approach is aligned with precision health methodologies (i.e. person-centered) and could lead to better screening and diagnosis of CRCI. Findings could also be used to develop personalized intervention for survivors with CRCI, ultimately reducing the disabling impact of CRCI and improving survivors’ QOL. Finally, we hope that this protocol will provide a resource for other investigators who aim to apply a similar approach to their studies.