Establishing the Ecological Validity of Cognitive Patient Reported Outcome Measures in Breast Cancer Survivors: A Prospective Cohort Study

ABSTRACT

Objective

The aim of this study was to determine which cognitive patient reported outcome measure (PROM) best represents self‐reported cognitive functioning in real‐world environments among breast cancer survivors (BCS) as measured by ecological momentary assessments (EMA), and to compare their ability to predict future everyday functioning.

Methods

One‐hundred twenty‐four BCS (ages 24–88) completed self‐report measures of cognitive functioning (FACT‐Cog PCI, PROMIS Cog, CFQ, EORTC‐CF) and everyday functioning (SDI, FACT‐G Functional Well‐being subscale) at baseline (Time 1) and at 9 weeks follow up (Time 2). Between assessments, EMA protocols (including one item to assess cancer‐related cognitive symptoms) were administered every other day for 8 weeks. Person‐specific means and standard deviations were calculated for EMA data. Hierarchical linear regression models were used to model cognitive PROM predictors of person‐specific averages and variability in EMAs, SDI, and FACT‐G functional well‐being, and model parameters (R ², AIC, BIC, semi‐partial R) were compared.

Results

Follow‐up FACT‐Cog PCI most accurately predicted both the average (ΔR ² = 0.213, p < 0.001) and variability (ΔR ² = 0.071, p < 0.001) in EMA CRCI symptoms. For future functional outcomes, the PROMIS Cog and FACT‐Cog PCI at baseline demonstrated similar predictive power for Time 2 Functional Well‐being (ΔR ² = 0.210, p < 0.001). Additionally, baseline FACT‐Cog PCI was the strongest predictor of social dysfunction (SDI; ΔR ² = 0.205, p < 0.001).

Conclusions

These findings support the ecological validity of cognitive PROMs in BCS and indicate that both FACT‐Cog PCI and PROMIS Cog effectively capture real‐world cognitive symptoms and predict future everyday functioning, including social function and well‐being.

1. Background

Cancer‐related cognitive impairment (CRCI) is one of the most distressing and burdensome effects of breast cancer and its treatment [1, 2]. When cognitive patient reported outcome measures (PROMs) are used, it is estimated that 21%–83% (mean 44%) of breast cancer survivors (BCS) report CRCI after adjuvant treatments end [3], that can last for years [1, 2]. BCS most often report cognitive difficulties in the domains of attention, executive functioning, memory, and processing speed [1, 2] that negatively affect their ability to engage in social and occupational roles [4, 5], self‐regulatory abilities [6], overall quality of life [7], and contribute to work‐related disability [8].

CRCI is most often measured with standardized neuropsychological tests [9], also known as objective cognitive assessments, and self‐report measures, also referred to as PROMs [10]. Poor performance on neuropsychological tests is identified when compared to normative values, or when there is an observed decline in cognitive performance over repeated assessments. It is generally accepted that cognitive tests and PROMs are complementary measures of CRCI— supported by evidence that correlations between these two are typically weak or absent [11, 12]. Furthermore, cognitive PROMs consistently correlate with psychological distress and evidence supports that these symptoms co‐occur for many survivors suggesting shared underlying neurobiological mechanisms [11, 13]. Perceived cognitive difficulties, along with pain, fatigue, anxiety, and depression, have been identified as a core set of PROMs to monitor in cancer survivor research since these symptoms are consistently present in individuals after cancer treatment [14]. The practical utility of PROMs allows for convenient, efficient, and low cost CRCI assessments, which is especially important in clinical settings, where brief and accessible tools are preferred.

There are numerous valid and reliable cognitive PROMs available to assess CRCI [10]. However, this variety has led to inconsistency in how cognitive PROM endpoints are used and reported across CRCI studies [15], which limits the ability to make cross‐study comparisons. Cognitive PROMs also reflect survivors' experience of their cognitive functioning in daily life contexts and correlate with their quality of life [1, 16], yet the ecological validity of these measures is underreported. Ecological validity, in the context of cognitive PROMs, refers to the extent to which the assessed symptoms generalize to real world settings and circumstances experienced in everyday life. It also encompasses how well these symptoms predict future levels of functioning, a concept often described as verticality, and their ability to forecast neurocognitive outcomes, known as generalization [17].

Cognitive PROMs assess the extent to which specific cognitive problems are experienced and the impact of cognitive problems on everyday life (typically in the previous week, month, or 6‐month period), and aim to capture real‐world impact of cognitive dysfunction. A fundamental concern for patients, providers, and family members struggling with CRCI is whether they will be able to function independently and perform their daily activities at the same level as before cancer [18]. Most research to date has focused on CRCI prevalence, mechanisms, and treatments, but less is known about the specific impact on survivor's daily functioning and everyday lives beyond what has been described in qualitative studies [19]. Recently, perceived cognitive functioning has been found to predict return to work for BCS [20]; however, the predictive ability of cognitive PROMs for social or more general functional status is not well understood. A deeper understanding of the ecological validity of cognitive PROMs relating to everyday functioning is required to facilitate the development of best practices for the conduct of meaningful CRCI assessment.

Ecological momentary assessments (EMAs) offer a method to enhance ecological validity and real‐world assessment and are increasingly being used to address limitations of traditional clinical measures, such as recall bias [21]. EMAs are a type of experience sampling that allow researchers to directly capture frequent variation in behavioral and cognitive processes [22] and improve evaluation of ecological validity by repeatedly administering assessments in real‐world settings. In the real‐world, one's cognitive performance is influenced by many state‐dependent factors (e.g., mood, symptom burden) and situational factors (e.g., environment, contextual demands), and previous studies have emphasized the importance of these day‐to‐day cognitive variations in breast cancer survivors [23, 24].

In this study, we aimed to improve our understanding of the ecological validity of cognitive PROMs among BCS by determining which PROMs most accurately reflect real‐world cognitive function (using EMAs) and predict everyday functioning. We specifically address the above gaps in knowledge and directly compared four of the most commonly used cognitive PROMs in CRCI research [10]— the Functional Assessment of Cancer Therapy ‐ Cognitive Function (FACT‐Cog), Patient‐Reported Outcome Measurement Information System Cognitive Function (PROMIS Cog), Cognitive Failures Questionnaire (CFQ), and the European Organization for Research and Treatment of Cancer‐Cognitive Function subscale (EORTC‐CF). The specific aims of this study were to (1) determine which cognitive PROM best represents real‐world cancer‐related cognitive symptoms, and (2) describe which cognitive PROM best predicts everyday functioning (social dysfunction and functional well‐being). Establishing the ecological validity of common cognitive PROMs could move the field forward and facilitate development of best practices in CRCI assessment.

2. Methods

A prospective intensive longitudinal design was used. The full study protocol has been previously described and included objective and subjective assessments of CRCI using traditional research measurement approaches and EMA [25]. All study‐related procedures were conducted in accordance with the Declaration of Helsinki and approved by the University of Texas at Austin Institutional Review Board (STUDY00002393). Participants provided written, informed consent.

2.1. Sample

We enrolled women who were at least 21 years old, lived in the U.S., had been diagnosed with and completed adjuvant treatment (within the previous 6 years) for breast cancer (stage 0–III). To be enrolled in the study, women were also required to have the physical (i.e., confirmed ability to use a computer and smartphone; Karnofsky Performance Scale ≥ 70) and cognitive (MiniMoca Telephone screening with orientation to place question omitted, score ≥ 11) [26] ability to participate, be English‐language proficient, and have access to a personal smartphone. Those who might be pregnant, had major sensory deficits (e.g., deafness or blindness) that would interfere with data collection, had prior history of cancer with systemic treatment (other than breast cancer), or had neurological or cognitive comorbidities (e.g., dementia, brain injury/trauma, neurodegenerative diseases) were excluded. We recruited nationally through breast cancer social networks and the UCLA Clinical and Translational Science Institute cancer registry.

2.2. Data Collection Procedures

All study procedures were conducted remotely from the University of Texas at Austin from June 2022 to December 2024. Those interested in participating contacted the team and scheduled telephone enrollment appointments, and if eligible, completed informed consent. Data were collected using Research Electronic Data Capture (REDCap) tools hosted at the University of Texas at Austin. For each participant, baseline data (Time 1) included a survey (demographic and clinical characteristics, cognitive PROMs, and PROMs of everyday functioning) and a remote cognitive test battery (test data not included in this report). After baseline data collection, the EMA protocol started and was administered once/day, every other day for 8 weeks (28 assessments total). All data collection procedures, including the EMA protocol and software, were reviewed and explained to participants during their enrollment appointments. An interval‐based design, with random sampling times on session days, was used for the EMA protocol based on qualitative findings that CRCI in breast cancer survivors can vary day‐to‐day, week‐to‐week [24], and similar EMA protocols (once daily for 4 weeks) that have been successfully delivered to assess CRCI variability in BCS [27, 28]. To balance participant burden with study objectives we opted to administer EMAs every other day, rather than daily, for the 8‐week study protocol.

Links to complete the EMA sessions were texted to participants on their personal smartphones at varying times each day across the protocol (i.e., morning, afternoon or evening) on a rotating basis based on participants' average wake and sleep times, to randomly sample CRCI symptoms at different times of the day and in varying contexts. The EMA links directed participants to a browser interface for session completion. Participants had 6 h to complete the EMA session with up to two reminders sent if assessments were incomplete. They could opt out of completing the session but did not have the ability to suspend or delay responses. The same items were administered each session, and no branching logic was used. Follow‐up data collection (Time 2) occurred after the EMA sessions were completed and included surveys of cognitive, everyday functioning, and quality of life PROMs administered via REDCap (See Figure 1 for overview of Data Collection Procedures).

FIGURE 1.

Overview of data collection procedures for each participant. CFQ, Cognitive Failures Questionnaire; CRCI, cancer‐related cognitive impairments; EMA, ecological momentary assessment; EORTC‐CF, the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Cognitive Function subscale; FACT‐Cog PCI, Functional Assessment of Cancer Treatment Cognitive Function Subscale; FACT‐G FWB, Functional Assessment of Cancer Treatment General Functional Well‐being subscale; PROMIS Cog, Patient‐Reported Outcome Measurement Information System‐Cognitive Function Short Form 8a; SDI, Social Difficulties Inventory.

2.3. Measures

2.3.1. Demographic and Clinical Variables

A study specific self‐report questionnaire was used to collect demographics (age, education, race, ethnicity, marital status, children/dependents, income, employment), health history (co‐morbidities, weight, menopausal status, current medications) and cancer history (breast cancer type, cancer treatment details, end date of chemotherapy, current medications).

2.3.2. Cognitive PROMs (Assessed at Times 1 and 2)

We administered the most commonly used cognitive PROMs for CRCI [10] — FACT‐Cog, CFQ, EORTC‐CF, and PROMIS Cog. The FACT‐Cog assesses cognitive symptoms in the previous 7 days in with 37‐items across 4 domains—perceived cognitive impairments (PCI), perceived abilities, quality of life, and comments from others, and has consistently demonstrated excellent convergent and divergent validity [29]. The FACT Cog PCI 20 item subscale was used in this study and demonstrated excellent internal reliability (Cronbach's alpha = 0.96). Lower scores indicate worse PCI. The CFQ asks about cognitive symptoms that occurred in the past 6 months across 25 items and has subscales for forgetfulness, distractibility, and false triggering, and has exhibited adequate criterion and construct validity [30, 31]. The CFQ total score was used in these analyses, with higher scores indicating higher cognitive failures (Cronbach's alpha = 0.95). The EORTC‐CF is a subscale of the EORTC QLQ‐C30, and includes 2 items to assess frequency of difficulty concentrating and difficulty remembering things in the past week that have demonstrated clinical criterion and construct validity [32]. The EORTC‐CF scoring manual was followed to score total scores on a 0–100 scale with lower scores indicating worse cognitive functioning (Cronbach's alpha = 0.76). The PROMIS Cog (8‐items) assesses cognitive symptoms in the past week [33] and has been validated in BCS [34]. Raw scores were used in the analyses, with lower scores indicating worse cognitive functioning (Cronbach's alpha = 0.93).

2.3.3. Everyday Functioning PROMs (Assessed at Times 1 and 2)

Functioning in everyday life was operationalized as social dysfunction and functional well‐being. The 21‐item Social Difficulties Inventory (SDI) was used to evaluate social dysfunction, which was developed for and validated in oncology populations and assesses social difficulties in the previous month [35]. In these analyses, we used the SDI‐16 total score that can range from 0 to 44 (Cronbach's alpha = 0.92), with scores > 10 indicating social dysfunction [35]. The FACT‐ General Functional Wellbeing subscale (FWB) was used to measure functional well‐being. The FACT‐ General, which includes 27 items assess the domains of physical, social/family, emotional, and functional well‐being, was chosen based on its widespread use in oncology research [36] and evidence of excellent validity [37]. We selected the FWB subscale, rather than the other three subscales, to increase conceptual precision of the estimated models. Cronbach's alpha for the FWB was 0.84.

2.3.4. EMA Data

EMA surveys were administered via the NeuroUX platform (https://www.getneuroux.com/). Full details of the EMA protocol, including gamified mobile cognitive tests and symptom assessments, have been previously described [25] and the EMA items can be found in the Supporting Information (See Supporting Information S1: Table 1). Mobile cognitive test data were not used in this analysis because objective and subjective measures of CRCI rarely correlate [11, 12] (as discussed above), and previous findings indicate that the specific mobile cognitive tests administered do not strongly correlate with cognitive PROMs in women with metastatic breast cancer [27]. In the present study, four of the EMA items were used—(1) cancer‐related cognitive symptoms (i.e., “I have cancer‐related cognitive or brain symptoms”), (2) anxiety/worry, (3) depressed/sadness; and (4) fatigue/tired. These same EMA items have demonstrated strong temporal reliability (i.e., ICC's > 0.95) and concurrent and divergent validity in BCS [27, 38]. The EMA questions were prefaced with the prompt, “The next questions are about the way you feel at this moment” and participants could select any number from 0 (not at all) to 7 (extremely). Person‐specific means and standard deviations (i.e., a metric of within‐person variability) were calculated for EMA questions across the full protocol (28 sessions) and used in the analyses.

2.3.5. Statistical Methods

All PROMs and aggregated EMA data were checked for normality and outliers and descriptive statistics were summarized.

To address the first aim, four separate hierarchical linear regression models were constructed with aggregated person‐specific mean scores of cancer‐related cognitive symptoms (measured with EMA, across all 28 administrations) as the dependent variable, and each follow‐up (Time 2) cognitive PROM as an independent variable. All four models included the same demographic/clinical covariates in Step 1, age, years since treatment, and person‐specific mean of anxiety ratings from EMA protocol. The latter was included to account for the variance explained by anxiety in relation to CRCI symptoms during EMA sessions, as these two are strongly correlated, in general, in BCS [11, 13] and in this sample (See Supporting Information S1: Figure 1). Average fatigue and mood also significantly correlated with CRCI, but we chose anxiety as the covariate as it was the strongest correlation and to avoid multicollinearity (these three symptoms were highly correlated, See Supporting Information S1: Figure 1). These steps were repeated using within‐person variability (person‐specific standard deviations) in CRCI symptoms across the protocol as the dependent variable. All four models included the same demographic/clinical covariates in Step 1, age, years since treatment, and person‐specific variability in mood ratings (i.e., depressed/sadness symptoms) from EMA protocol (as variability in this symptom correlated strongest with variability in within‐person variability in CRCI symptoms, See Supporting Information S1: Figure 1).

To address the second aim, Pearson's correlations (with FDR corrected p values) were examined among selected demographic and clinical factors (age, years of education, and time since treatment completion), baseline cognitive PROMs, and functional PROMs (FWB; SDI). Next, eight hierarchical linear regression models were constructed, four with FWB as the dependent variable, and four with SDI as the dependent variable. The same demographic/clinical covariates were used in Step 1 for all models (determined based on those that were significantly associated with FWB or SDI at baseline).

Prior to modeling, we assessed assumptions of homoscedasticity (i.e., Breusch‐Pagan, White tests), independence and normality of residuals (Breusch‐Godfrey; Jarque tests), multicollinearity (VIF), and undue influence of outliers (i.e., Cooks; dfbetas). If assumptions were violated, robust regression estimates were calculated to account for the violations [39]. Model fit parameters (ΔR ², AIC, BIC, semi‐partial R ²) were compared for each model, and those with the highest explained variance and lowest prediction error estimates (AIC and BIC) were considered the best performing. To prevent Type I error, we used Bonferroni adjustments for each set of hierarchical models (for each of the dependent variables).

Overall, missing data were low in this study. For PROM data collected at Time 1 and 2, case‐mean substitution was used for item level PROM missing data for items with < 10% missing data (all items met this criteria) and for participants with < 10% missing data on any one scale/subscale. Four participants who enrolled and completed the EMA protocol did not complete Time 2 collection, resulting in 4% missing data rate for the dependent variables in Aim 2. Demographic and clinical variables did not differ between participants with missing data at baseline and follow‐up (See Supporting Information S1: Table 2); thus, Time 1 PROM data were carried forward for these participants to include their data in the analyses; sensitivity analyses with and without these participants were conducted to examine the stability of the findings. Overall, there was a 15% rate of missing EMA data across all participants and sessions (adherence rates: minimum: 11%; maximum: 100%, median: 93%). Since cognitive EMA data were aggregated across all 28 sessions for each participant, missing EMA data were left as missing, and scores were calculated based on the number of completed responses. All analyses were conducted in R (version 2024.12.0 + 467) or JASP (Version 0.18.3). For the robust regressions, we used the heteroscedasticity‐consistent covariance matrix version 3 (“HC3”), which is recommended for samples < 251 [40]. A priori power analyses indicated 104 participants were needed to achieve 80% power (See Supporting Information S1: Table 3 for details).

3. Results

3.1. Descriptives

The sample ranged in age from 24 to 88 years of age, with approximately 12% identifying as Hispanic, and 28% identifying as a member of a racial minority group. Clinically, most of the sample had a history of stage II‐III breast cancer and were approximately 2 years post adjuvant treatment completion (See Table 1). The average person‐specific mean CRCI symptoms ranged from 0 to 7 (mean 1.64; SD = 1.59) and varied on average 0.71 SD across the protocol. See Table 2 for descriptive statistics for all Time 1 PROMs and aggregated EMAs. Sensitivity analyses were conducted for all models tested for Aims 1 and 2 after excluding participants with missing data (n = 119) and yielded comparable results in terms of estimates and significance testing (See Supporting Information S1: Tables 4–5); thus, findings from the full sample are reported and interpreted.

TABLE 1.

Demographic and clinical characteristics of the sample (N = 124).

Demographic characteristic	Mean (SD); minimum to maximum	Frequency (percentage)
Age in years	51.4 (11.9); 24 to 88	—
Hispanic ethnicity		15 (12.1%)
Racial minority		35 (28.2%)
Black/African American (n = 9)
Asian (n = 13)
American Indian/Native Hawaiian/Pacific Islander (n = 3)
Other race (n = 10)
Partnered (married or living with significant other)		87 (65.3%)
Have dependents (children, parents)		63 (50.8%)
Employed (part time or full time)		84 (67.7%)
Years of education	17.1 (2.8); 4 to 27

Clinical characteristic	Mean (SD); minimum to maximum	Frequency (percentage)
Years since treatment ended	2.2 (1.6); 0.01 to 5.8
History of stage 0–I breast cancer		54 (43.5%)
History of stage II–III breast cancer		65 (52.4%)
Post‐menopausal		79 (63.7%)

TABLE 2.

Time 1 (baseline) PROMs and aggregated cognitive EMA descriptive statistics (N = 124).

Measure (possible range)	Mean (SD)	SE	Minimum, maximum	Median	Skewness	Kurtosis
FACT‐Cog PCI (0–80)	53.41 (18.05)	1.62	4, 79	56.50	−0.56	−0.47
PROMIS Cog 8a (0–40)	28.84 (8.36)	0.75	9, 40	30	−0.45	−0.66
CFQ (0–100)	34.76 (17.86)	1.60	2,87	33	0.56	−0.23
EORTC‐CF (0–100)	70.43 (23.46)	2.11	0, 100	66.67	−0.83	0.78
FACT‐G FWB (0–28)	19.65 (5.49)	0.49	3, 28	20	−0.59	0.03
SDI‐16 (0–44)	9.63 (8.4)	0.75	0, 36	7	1.03	0.32
EMA person specific mean CRCI symptoms (0–7)	1.64 (1.59)	0.14	0, 7	1.19	0.8	0.03
EMA within person variability (SD) in CRCI symptoms	0.71 (0.54)	0.05	0, 2.08	0.75	0.24	−0.8

Note: For CFQ higher scores indicate more cognitive failures; for EORTC‐CF and PROMIS Cog lower scores indicating worse cognitive functioning; for FACT Cog‐ PCI lower scores indicate worse cognitive impairment; for FACT‐G FWB higher scores indicate higher well‐being; for SDI 16, higher scores indicate worse dysfunction.

Abbreviations: CFQ, Cognitive Failures Questionnaire; CRCI, cancer‐related cognitive impairments; EMA, ecological momentary assessment; EORTC‐CF, the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Cognitive Function; FACT‐Cog PCI, Functional Assessment of Cancer Treatment Cognitive Function Subscale; FACT‐G FWB, Functional Assessment of Cancer Treatment General Functional Well‐being subscale; PROMIS Cog, Patient‐Reported Outcome Measurement Information System‐Cognitive Function Short Form 8a; SD, standard deviation; SE, standard error.

3.2. Cognitive PROM Models of CRCI Symptoms Measured With EMA (Aim 1)

Age, time since treatment, and average anxiety symptoms (measured with EMA) were included as Step 1 in all models of average CRCI symptoms and significantly explained 35.4% of the variance (adjusted R ² = 0.345, F (3, 119) = 21.73, p < 0.0001), and average anxiety symptoms was a significant predictor (t = 5.02, p < 0.0001). Regression diagnostics for average CRCI symptom models indicated violations of assumptions, therefore robust regression models were used for the hierarchical multiple regression models (Table 3). Compared to the other cognitive PROM models, the FACT‐Cog PCI model demonstrated the best fit for the EMA CRCI symptoms (ΔR ² = 0.213, p < 0.001, AIC = 372.26, BIC = 388.41, semi‐partial R ² = 0.570) followed by the PROMIS Cog (ΔR ² = 0.206, p < 0.001, AIC = 374.08, BIC = 390.23, semi‐partial R ² = 0.559).

TABLE 3.

Hierarchical multiple regression models with the cognitive PRO measures at Time 2 as the independent variables and ecological momentary assessments (across 28 days) as dependent variables (N = 124).

		Estimate	SE	t	a R 2	Model F statistic (df)	ΔR 2	AIC	BIC	Semi‐partial R 2
Person‐specific average in cancer‐related cognitive symptoms
Step 1	Age	−0.01	0.01	−1.36	0.354	21.73 (3, 119)***	—	—	—	—
	Time since treatment	−0.10	0.09	−1.17	—	—	—	—	—	—
	Average anxiety (EMA)	0.67	0.13	5.02***	—	—	—	—	—	—
Step 2
Model A	FACT‐Cog PCI	−0.05	0.01	−6.15 ***	0.567	38.55 (4, 118) ***	0.213 ***	372.26	388.41	0.570
Model B	PROMIS Cog	−0.10	0.02	−6.19***	0.560	37.55 (4, 118)***	0.206***	374.08	390.23	0.559
Model C	CFQ	0.03	0.01	3.51***	0.431	22.30 (4, 118)***	0.077***	405.82	421.97	0.402
Model D	EORTC‐CF	−0.02	0.01	−3.15***	0.451	24.26 (4, 118)***	0.097***	401.26	417.41	0.454
Within‐person variability in cancer‐related cognitive symptoms
Step 1	Age	−0.00	0.00	−0.58	0.24	13.5 (3, 119)***	—	—	—	—
	Time since treatment	−0.06	0.03	−2.28	—	—	—	—	—	—
	Variability in mood (EMA)	0.40	0.07	5.58***	—	—	—	—	—	—
Step 2
Model E	FACT‐Cog PCI	−0.01	0.00	−3.20 *	0.303	14.23 (4, 118) ***	0.071 **	158.70	174.85	0.588
Model F	PROMIS Cog	−0.02	0.01	−2.78	0.290	13.48 (4, 118)***	0.059*	160.84	176.99	0.551
Model G	CFQ	0.01	0.00	2.67	0.255	11.42 (4, 118)***	0.025	166.87	183.02	0.276
Model H	EORTC‐CF	−0.00	0.00	−1.13	0.241	10.70 (4, 118)***	0.012	169.06	185.21	0.353

Note: Parameter estimates (SE, t, and p values) for person‐specific models were estimated with robust regression because assumptions were violated. The dependent variable for Models A‐D was person‐specific average in cancer‐related cognitive symptoms (measured with EMA), and the dependent variable for Models E‐H was within‐person variability (SD) in cancer‐related cognitive symptoms (measured with EMA). The same covariates were used in step 1 for models A‐D (age, years since treatment, person‐specific mean of anxiety ratings from EMA protocol), and the same covariates were used in step 1 for models E‐H (age, years since treatment, person‐specific variability in mood ratings from EMA protocol). For Models A and E the independent variable was the Functional Assessment of Cancer Treatment Cognitive Function (FACT‐Cog PCI) Subscale in Step 2; for Models B & F it was the Patient‐Reported Outcome Measurement Information System‐Cognitive Function Short Form 8a (PROMIS Cog) Short form 8a total score in Step 2; for Models C & G it was the Cognitive Failures Questionnaire (CFQ) in Step 2; for Models D & H it was the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Cognitive Function (EORTC‐CF) in Step 2. Bonferroni corrected p value = 0.05/4 (0.0125). Bolded rows indicate the model(s) with the best fit.

^a Adjusted R ².

^* p < 0.01.

^** p < 0.001.

^*** p < 0.0001.

Age, time since treatment, and average mood symptoms (measured with EMA) were included as Step 1 in all models of variability in CRCI symptom models and significantly explained 24% of the variance (adjusted R ² = 0.24, F (3, 119) = 13.5, p < 0.0001), and average mood symptoms was a significant predictor (t = 5.58, p < 0.0001). Regression diagnostics for variability in CRCI symptom models indicated that regression assumptions were met, so standard models were used and reported in Table 3. The FACT‐Cog PCI model demonstrated the best fit for variability in EMA CRCI symptoms (ΔR ² = 0.071, p < 0.001, AIC = 158.70, BIC = 174.85, semi‐partial R ² = 0.588) followed by the PROMIS Cog (ΔR ² = 0.059, p < 0.01, AIC = 160.84, BIC = 176.99, semi‐partial R ² = 0.551).

3.3. Time 1 Cognitive PROM Predictors of Everyday Functioning at Time 2 (Aim 2)

Time 1 correlations revealed a similar pattern of correlations among the cognitive PROMs, FWB, and SDI (See Supporting Information S1: Figure 2). Pearson's r correlations among cognitive PROMs and FWB ranged from |0.48| to |0.54| (p's < 0.001) and among cognitive PROMs and SDI from |0.53| to |0.63| (p's < 0.001). Based on correlation findings, age and time since treatment were selected as covariates for hierarchical models of Time 2 FWB and SDI (p's < 0.05).

For FWB at Time 2, regression diagnostics indicated that the assumptions were met, therefore standard hierarchical models were used (See Table 4). Age and time since treatment, were included as Step 1 in all models FWB and significantly explained 7.3% of the variance (adjusted R ² = 0.073, F (2, 121) = 4.74, p < 0.01), and time since treatment was a significant predictor (t = 2.04, p < 0.01). PROMIS Cog (ΔR ² = 0.210, p < 0.001, AIC = 727.06, BIC = 740.65, semi‐partial R = 0.835) and FACT‐Cog PCI (ΔR ² = 0.206, p < 0.001, AIC = 727.79, BIC = 741.39, semi‐partial R = 0.839) models of FWB demonstrated similar fit, better than both CFQ and EORTC‐CF.

TABLE 4.

Hierarchical multiple regression models for well‐being and social dysfunction at Time 2 as dependent variables and the cognitive PROMs at baseline as the independent variables (N = 124).

		Estimate	SE	t	a R 2	Model F statistic (df)	ΔR 2	AIC	BIC	Semi‐partial R 2
Functional well‐being
Step 1	Age	0.06	0.04	1.60	0.073	4.74 (2, 121)*	—	—	—	—
	Time since treatment	0.60	0.30	2.04*	—	—	—	—	—	—
Step 2
Model I	FACT‐Cog PCI	0.14	0.02	5.85 ***	0.278	15.42 (3, 120) ***	0.206 ***	727.79	741.39	0.839
Model J	PROMIS Cog	0.30	0.05	5.92 ***	0.283	15.75 (3, 120) ***	0.210 ***	727.06	740.65	0.835
Model K	CFQ	−0.13	0.02	−5.38***	0.253	13.52 (3, 120)***	0.180***	732.11	745.70	0.801
Model L	EORTC‐CF	0.10	0.02	5.42***	0.255	13.69 (3, 120)***	0.182***	731.72	745.31	0.810
Social dysfunction
Step 1	Age	−0.18	0.05	−3.45***	0.125	8.68 (2121)***	—	—	—	—
	Time since treatment	−0.64	0.43	−1.51	—	—	—	—	—	—
Step 2
Model M	FACT‐Cog PCI	−0.20	0.03	−6.01 ***	0.330	19.72 (3, 120) ***	0.205 ***	815.12	828.71	0.749
Model N	PROMIS Cog	−0.42	0.07	−5.93***	0.314	18.34 (3, 120)***	0.189***	818.02	831.61	0.736
Model O	CFQ	0.20	0.04	5.39***	0.309	17.91 (3, 120)***	0.184***	818.94	832.53	0.729
Model P	EORTC‐CF	−0.14	0.03	−5.11***	0.288	16.19 (3, 120)***	0.163***	822.67	836.26	0.678

Note: Parameter estimates (SE, t, and p values) for social dysfunction models were estimated with robust regression because assumptions were violated. The dependent variable for Models I‐J was Functional Well‐being and the dependent variable for Models M‐P was Social Dysfunction. The same covariates were used in step 1 for models I‐P (age; years since treatment). For Models I & M the independent variable was the Functional Assessment of Cancer Treatment Cognitive Function (FACT‐Cog PCI) Subscale in Step 2; for Models J & N it was the Patient‐Reported Outcome Measurement Information System‐Cognitive Function Short Form 8a (PROMIS Cog) Short form 8a total score in Step 2; for Models K & O it was the Cognitive Failures Questionnaire (CFQ) in Step 2; and for Models L & P it was the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Cognitive Function (EORTC‐CF) in Step 2. Bonferroni corrected p value = 0.05/4 (0.0125). Bolded rows indicate the model(s) with the best fit.

^a Adjusted R ².

^* p < 0.01.

**p < 0.001.

^*** p < 0.0001.

For SDI at Time 2, regression diagnostics indicated assumptions violations, therefore robust regression models were used for the hierarchical multiple regression models (Table 4). Age and time since treatment, were included as Step 1 in all models SDI and significantly explained 12.5% of the variance (adjusted R ² = 0.125, F (2, 121) = 8.68, p < 0.0001), and age was a significant predictor (t = −3.45, p < 0.0001). Compared to the other cognitive PROM models, the FACT‐Cog PCI model demonstrated the best fit for SDI (ΔR ² = 0.205, p < 0.001, AIC = 815.12, BIC = 828.71, semi‐partial R = 0.749).

4. Discussion

In this study we directly compared four commonly used cognitive PROMs in CRCI research in a sample of BCS to determine which best reflected CRCI symptoms in real‐world settings measured by EMA, and which PROMs best predicted future everyday functioning. Our findings indicated that the FACT‐Cog PCI subscale best reflected individual level average and variability in cognitive symptoms measured across 2 months using daily EMAs and predicted future social functioning. These findings support the wide use of the FACT‐Cog in measuring cognitive symptoms of CRCI [41]. However, we also note that the PROMIS Cog was only marginally inferior to the FACT‐Cog in terms of representing CRCI symptoms using daily EMA, as well as predicting future social functioning. Therefore, our study also supports that the shorter PROMIS Cog scale presents a valid and reliable alternative to using the FACT‐Cog PCI in certain settings.

These findings provide new evidence related to the ecological validity of the FACT‐Cog PCI, PROMIS Cog, CFQ, and EORTC‐CF, specifically related to the representativeness (i.e., how the scores generalize to ratings in real world settings measured through EMA) and the generalization (i.e., the degree to which the scores predict future levels of functioning) of these measures in BCS. More broadly, these study results extend previous findings that cognitive PROMs are predictive of overall quality of life [42], psychological and physical well‐being [16], and work‐related outcomes [20] among BCS by providing new evidence that poorer patient‐reported cognitive functioning is a significant predictor of worse social functioning and lower functional well‐being.

There are currently no gold standards for using cognitive PROMs in CRCI research and clinical practice. The FACT‐Cog is the most widely used and validated in this population, and our findings support this measure, particularly the PCI subscale, as the most informative and ecologically valid. In addition, the PROMIS Cog has been recommended to be used, at a minimum, in all CRCI research as it is psychometrically robust, relatively short (8‐items), freely accessible, and 7 of the 8 items map directly to the FACT‐Cog PCI [10]. Our findings indicate that the PROMIS Cog was the “next best” or almost just as strong (i.e., for the functional well‐being models) across the analyses compared to the FACT‐Cog. The FACT‐Cog offers some advantages including more variance and larger range, and inclusion of CRCI‐specific items—such as difficulty concentrating, trouble finding the right word, and slow thinking—since it was purposefully developed to assess cognitive functioning in cancer survivors [29]. Our study supports its continued use as the best option for a PROM of CRCI. In addition, the PROMIS Cog offers a briefer alternative while sacrificing minimal concurrent validity as indicated by our prior work [34], further supported by this study that found only minimal comparative weakness in terms of ecological validity. As such, for studies targeting CRCI as a main outcome, the FACT‐Cog remains the ongoing preferred measure, and the PROMIS Cog continues to emerge a robust alternative as a secondary outcome for studies or for settings with limited resources.

We found that the CFQ and EORTC‐CF also predicted real‐world cognitive functioning and future everyday functioning, although to a lesser degree than the FACT‐Cog and PROMIS Cog (i.e., less variance explained, smaller fit indices). The EORTC‐CF (a subscale of the Full EORTC Quality of Life core questionnaire) was developed specifically for cancer patients [32]; however, it only includes 2 items to assess memory and concentration and fails to capture other important aspects of CRCI like processing speed and executive functioning [2, 11]. The CFQ was developed for adults, not cancer patients specifically, to assess cognitive failures in the prior 6 months [30]. The explained variance of the CFQ model might have been higher if the EMA protocols captured the same time frame as the instrument (e.g., over 6 months). Additionally, although the PROMIS Cog was not designed specifically for use in cancer patients, its developers used items from the full FACT‐Cog scale to create the item bank used in this measure (79). Considering that 7 of the 8 items of the PROMIS Cog are included in the FACT Cog PCI, it is not surprising that the model indices of PROMIS Cog were so similar to those of the FACT‐Cog PCI.

4.1. Strengths

This study has several notable strengths. To our knowledge it is the first to simultaneously compare four cognitive PROMs in the same sample of BCS and integrate cognitive EMA data and social and functional outcomes to “anchor” ecological validity assessments. Prior studies have frequently been limited to only cognitive PROMs as anchors, limiting generalizability and clinical translation and presuming ecological validity. Since our study was specifically aiming to evaluate the relative ecological validity of the most commonly used measures, we approached the utility of each without presumptive value to fairly assess. This is the only study to examine the relative utility of different commonly used PROMs by leveraging EMA data, which has been highlighted by the field as a particularly relevant assessment approach [41]. Our sample size was relatively large (n > 100) with decent diversity in terms of race, ethnicity, and breast cancer stage, enhancing external validity.

4.2. Limitations

We sampled BCS from a large age range (24–88) and range in time since treatment ended (from 2 weeks to almost 6 years), which limits the internal validity of the design; however, we covaried for age and time since treatment in all models to account for these effects. The study sample was limited to BCS only, limiting generalizability of these findings beyond breast cancer. The cognitive PROMs in this study are commonly used in other cancer populations with CRCI [10], therefore future studies should establish the ecological validity of these measures in other cancer populations. We tested multiple statistical models in the data analyses, inflating type I error; however, we used a conservative Bonferroni p value adjustment. Only one global EMA item was used to measure CRCI symptoms, which may not have completely captured cognitive variability across the study in this sample. Also, the EMA item assessing real‐time CRCI symptoms was not domain‐specific and future work may benefit from testing cognitive symptoms within specific domains (e.g., memory, processing speed, executive function). Future studies could employ cognitive interviewing, in addition to EMA, to evaluate how participants interpret and respond to cognitive PROM items in real‐world contexts, thereby further supporting their ecological validity.

We used aggregated person‐specific averages and variability in CRCI symptoms across the 28 EMA sessions in our models, which limits our ability to assess time‐based patterns, temporal dynamics, or partitioning of with‐person and between‐person variance in CRCI symptoms. Future studies could utilize multilevel modeling to examine these temporal dynamics in EMA‐measured CRCI symptoms, potentially providing insight into how changes over time predict clinically significant cancer‐related cognitive changes. Finally, the different time period for the recall of cognitive symptoms in the CFQ (previous 6 months) compared to the other 3 instruments (recall in previous week) limits the direct comparisons of these measures, as there is potentially more recall bias for the CFQ compared to the other measures.

4.3. Clinical Implications

These findings add to efforts that have been made to enhance the clinical utility of the FACT‐Cog and the PROMIS Cog, including proposed clinically important differences [43], minimal change values [44], and cut‐off scores [34, 45], providing new evidence on their ecologically validity for BCS post adjuvant treatment, which is fundamentally important to the clinical management of CRCI. The NIH PROMIS measures initiative has facilitated the integration of PROMIS scales into electronic health records and clinical processes, so the PROMIS Cog may be a more practical cognitive PRO to incorporate into clinical oncology settings for patient surveillance and monitoring.

4.4. Conclusion

These findings provide new evidence on the ecological validity of commonly used cognitive PROMs, specifically the FACT‐Cog PCI and PROMIS Cog, for assessing CRCI in BCS that can inform the development of future recommendations for assessing CRCI in research and/or practice settings. These findings are also hypothesis generating for future studies evaluating contextual, situational, or individual factors—such as psychological resilience, social support, or being well rested—that influence cognitive symptoms in everyday settings and may contribute to variability in cognitive symptoms experienced by BCS in real‐life.

Author Contributions

A.M.H. was the PI on this study and R.C.M., K.M.V.D. helped contribute to protocol design and funding acquisition. A.M.H., K.M.V.D., and R.C.M. are responsible for the manuscript's conception and design. A.M.H., O.Y.F.‐R., and E.W.P. conducted the data analyses. A.M.H. and O.Y.F.‐R. contributed to the first draft of this manuscript. A.M.H., K.M.V.D., R.C.M., O.Y.F.‐R., D.H., and E.W.P. contributed to revisions of the manuscript. All authors approved the final version.

Funding

This research was supported by funding from the National Institutes of Health (R21NR020497; A.M.H., K.M.V.D., R.C.M.). K.M.V.D. is supported by grants from the NIH: K08CA241337 and R35CA283926. O.Y.F.‐R. is a MASCC Equity Fellow.

Ethics Statement

This study was performed in line with the principles of the Declaration of Helsinki. The University of Texas at Austin Institutional Review Board reviewed and monitored all study related procedures (STUDY00002393).

Consent

Participants provided informed written consent.

Conflicts of Interest

RCM has equity interest in KeyWise AI Inc. and has equity interest, is a consultant and receives compensation from NeuroUX. The terms of this arrangement have been reviewed and approved by UC San Diego in accordance with its conflict‐of‐interest policies. AMH was a consultant for Prodeo Inc. The terms of this arrangement have been reviewed and approved by University of Texas at Austin in accordance with its conflict‐of‐interest policies.

Supporting information

Supporting Information S1

Henneghan, Ashley M. , Franco‐Rocha Oscar Y., Van Dyk Kathleen M., Paolillo Emily W., Haywood Darren, and Moore Raeanne C.. 2025. “Establishing the Ecological Validity of Cognitive Patient Reported Outcome Measures in Breast Cancer Survivors: A Prospective Cohort Study,” Psycho‐Oncology: e70324. 10.1002/pon.70324.

Data Availability Statement

De‐identified data will be made available upon reasonable requests to the corresponding author and with data use agreements in place.