The Factor Structure of Items Assessing Subjective Memory: Between-Persons and Within-Persons Across Time

Abstract

Introduction:

Current understanding of the psychometric properties of items intended to assess the multidimensional construct of subjective memory (SM) is limited, as longitudinal studies of aging commonly use single items or brief sets of items to assess SM. Investigating how SM items cluster within individuals over time would increase understanding of how combining these items impacts their utility as an early indicator of cognitive change in the aging trajectory. To address this need, the current paper examined the factor structure of a brief set of SM items in an existing longitudinal study focused on cognitive aging at both the within-person and between-person levels.

Methods.

Data were drawn from the Einstein Aging Study, a longitudinal cohort study of aging (N=1,239, Mage=77.51, SD=5.03; 69.50% White; 24.27% Black; 6.23% Other). Community-dwelling older adults from an urban area of New York City were interviewed annually. At each wave, participants responded to six items intended to assess SM. Items assessed participants’ perceived memory decline as well as current memory ability. Multilevel exploratory factor analyses examined which factor solution best fit the data at between-person and within-person levels.

Results:

Factor structure of the SM items varied at the two levels. At the within-person level, two factors emerged, whereas at the between-person level, a single factor best represented the SM items. Items assessing perceived declines in memory functioning tended to have similar trajectories, while items assessing current memory ability were less related to change over time.

Conclusion:

Items appeared to assess two different dimensions of SM when examining within-person changes in SM across time, however the item structure suggested no other items covaried systematically within-persons over time. In contrast to the conceptualization of SM as a multidimensional construct, our findings suggest that when measuring SM between individuals, SM items tend to capture a single dimension underlying subjective memory. This may be due to the long retrospection period of items assessing perceived memory ability. A single item assessing perceived memory decline in older adults without evidence of objective cognitive impairment may be sufficient to monitor memory change in clinical or research settings.

Introduction

The construct of subjective cognition is identified as self-perceptions of cognitive functioning gathered via self-report [1]. The most common domain of subjective cognition assessed in longitudinal studies of older adults is subjective memory (SM) [2,3]. Specifically these studies are interested in impairments in SM: the perception that memory functioning is poor or declining despite testing normally on objective, clinical assessments of memory performance [1]. SM impairment is common among older adults, and its reporting tends to increase with age [3]. Indeed, 25% to over 50% of older adults report SM impairments [4–6]. Empirical research has linked SM impairments to a number of adverse outcomes, including psychological symptoms (e.g., depression and anxiety) as well as non-normative cognitive decline and Alzheimer’s disease (AD) [1,7]. However, there are inconsistencies in these linkages that limit our understanding of SM and how it contributes to negative outcomes, particularly longitudinally. One source of inconsistency is in the approach to the assessment of SM.

SM is a multidimensional construct that includes subdomains such as current perceived memory functioning, perceived changes in functioning (typically declines), concerns about changes in functioning, and perceived control of changes in functioning [8–12]. Comprehensive measures of SM, such as the Memory Functioning Questionnaire (MFQ) [8] and the Multifactorial Memory Questionnaire (MMQ) [12] include items that capture these different domains to create a thorough understanding of an older adult’s perceptions of their memory. Factor analyses of these scales support the underlying complexity of SM with items loading onto factors such as capacity versus change [8] and frequency of forgetting versus retrospective functioning [13]. In contrast, other items cluster into those asking about the seriousness of specific memory problems [8]. However, the length of these multifaceted assessments decreases their use in epidemiological, longitudinal studies of aging that have to balance assessment of a wide range of constructs with assessment length [14]. For example, in their review of measures across 19 international studies of aging, Rabin and colleagues identified eight of the 19 studies as having five or fewer items assessing perceptions of cognitive functioning, and most of these items were specific to memory functioning [2]. Additionally, several large projects focused on broad changes in function that occur with aging by assessing SM with single items not intended as part of a larger scale focused on the construct (i.e., Health and Retirement Study [15], the National Health and Aging Trends Study [16], and the Midlife in the United States study [17]).

Substantial diversity exists in assessment strategies for SM across longitudinal studies of aging and cognitive functioning. Importantly, in their review of several studies examining preclinical AD, Rabin and colleagues found an estimated 75% of studies designed unique measures of SM not adopted by other studies [2]. The identification of SM impairment as a potential precursor to early AD [18] coupled with the wide variety of items and measures available [8–11] presents challenges for researchers interested in identifying changes in SM within an individual across time. Perceptions of current memory functioning may be based on many different aspects of personal experience including evaluation of current performance, comparison of current to past performance, or comparison to peers, not all of which can be assessed in brief assessments of SM. However, evidence suggests these types of distinctions are critically important to the longitudinal utility of SM impairments in predicting objective cognitive change [19]. For example, some studies show that impairments in SM accompanied by worry about memory is a stronger predictor of AD than perceived impairments alone [5,20], while others have found that SM impairment that influences daily activities is associated with higher AD risk [21,22]. Given the potential importance of SM as an early indicator of cognitive change and the amount of empirical evidence generated by longitudinal studies that employ brief SM assessments [23], a better understanding of the psychometric performance of these items, particularly longitudinally, would aid in the interpretation of changes in SM within persons across time. Much of the importance of SM is due to how impairments may develop and increase within individuals over time (e.g., more frequent or severe problems) [24–26] and these expected changes make it critical to examine whether and how items reflect the underlying construct of SM as an individual ages.

In response to the need to refine conceptualization and operationalization of reports on cognitive functioning in general, and memory specifically, the Subjective Cognitive Decline Initiative (SCD-I) proposed two general domains of subjective cognition (i.e., perceived cognitive functioning), which also applies to subjective memory: 1) ability (i.e., ratings or responses relevant to current cognitive performance) including severity, frequency, and impact of cognitive problems; or, 2) change (i.e., reports of change/decline in cognitive performance over time) including temporal or atemporal questions, referring to whether a specific time frame of reference is provided. Somewhat similar recommendations have been made for clinical contexts as well. Hill et al. [27] proposed four characteristics to guide the assessment of subjective cognition clinically: 1) specificity, 2) applicability, 3) time window, and 4) framing. However, whether these different conceptualizations of subjective cognition that encompass SM are reflected in the underlying structure of participants’ responses to SM items across time remains unclear. Given the variety SM items currently in use in existing, ongoing longitudinal studies of aging, a critical next step is to understand how these items cluster within individuals across time [2].

The current study used data from the Einstein Aging Study (EAS) to examine the factor structure of SM items frequently used in longitudinal, epidemiological studies of cognitive aging. Specifically, we explored the factor structure of SM items at the within-person level and whether this within-person factor structure was consistent with the structure at the between-person level.

Materials and Methods

Settings and Participants

Data were drawn from the Einstein Aging Study (EAS) – a longitudinal cohort study examining cognitive aging and dementia among community dwelling older adults in an urban, multi-ethnic area of New York City that began in 1993. The study protocol was approved by the Albert Einstein College of Medicine Institutional Review Board (See Katz et al. [28] for full study details). Participants completed written, informed consent at their initial visit and in-person comprehensive medical and neuropsychological examination annually. The current study included up to 11 waves of data (1993 – 2004) per participant, collected annually via self-administered questionnaires. On average, participants completed three waves of data (M = 3.24, SD = 2.48; range: 1 – 11). The sample included 1,239 community dwelling older adults above 69 years of age. To be included in the current study, participants had to complete self-reports of memory functioning and have no clinical diagnosis of MCI, Alzheimer’s disease, or other dementia at any point throughout the study period.

Measures

Subjective memory.

Items intended to assess SM were selected at the study onset in 1993 and were part of a health self-assessment questionnaire. Items selected for the parent study were intended to complement an extensive battery of neuropsychological tests and a clinical neurological exam [29]. Although this questionnaire included a number of items, the current study includes only those that were administered to the entire sample at each wave.

At each wave, participants responded to six questions about memory that tapped into participants’ perceived memory decline over the last one and ten years (Compared with one year ago, do you have trouble remembering things more often, less often, or about the same?; Compared with ten years ago, do you have trouble remembering things more often, less often, or about the same?), perceived frequency of memory problems (People find that they sometimes have more trouble remembering things as they get older. In the past year, how often did you have trouble remembering things?), perceived memory function compared to 30 years ago (If your memory was 100% when you were thirty years old, what percent would you say it is now?), frequency of forgetting important things (Comparing important and unimportant things, what percent of the time do you forget things that are important to you?), and perceived problems due to changes in memory (Has your memory change caused you any serious health problems [e.g., Forgot to turn off the stove, got lost, misplaced valuables])? Response options for all items were re-coded such that higher numbers represented poorer perceived memory functioning (see Table 1). Additionally, perceived decline items were recoded to dichotomous variables (0 = less often/about the same; 1 = more often) due to the low frequency (2.7% – 4.4%) of participants responding that they forgot things less often compared to a year or ten years ago.

Table 1.

Subjective Memory Items and Their Response Options

Items	Original Response Options	Re-coded Response Options	ICCs
Compared with one year ago, do you have trouble remembering things more often, less often or about the same?	1= More often; 2= Less often; 3= About the same	0 = Less often/about the same; 1 = More often	.51
Compared with ten years ago, do you have trouble remembering things more often, less often or about the same?	1= More often; 2= Less often; 3= About the same	0 = Less often/about the same; 1 = More often	.59
In the past year, how often did you have trouble remembering things?	1 = Frequently; 2= Sometimes; 3= Rarely; 4= Never	1= Never; 2= Rarely; 3= Sometimes; 4= Frequently	.51
If your memory was 100% when you were thirty years old, what percent would you say it is now?	*Participants reported percentage	(100 – response)/10	.53
Comparing important and unimportant things, what percent of the time do you forget things that are important to you?	*Participants reported percentage	Response/10	.45
Has your memory change caused you any serious health problems (e.g., Forgot to turn off the stove, got lost, misplaced valuables)?	0 = No; 1 = Yes	0 = No; 1 = Yes	.36

Statistical Analysis

First, the intraclass correlation (ICC) was calculated for each of the six items to determine the percentage of total variability that could be attributed to between-person differences. Next, multilevel confirmatory factor analyses (using MPLUS v. 8, [30]) were employed to determine whether a one or two-factor solution best fits the data at a between-person and within-person level. A total of four models with different factor combinations were examined: 1) a model with one factor at both the between-person and within-person levels, 2) a model with two factors at the within-person level and one factor at between-person level, 3) a model with one factor at the within-person level and two factors at between-person level, and 4) a model with two factors at the within-person and between-person levels. Model fits were examined to establish the optimal number of factors at each level. The fit indices of a good (acceptable) fitting model reflect a Comparative Fit Index (CFI) >.95 (between .90 and .95), Standardized Root Mean Square Residual (SRMR) < .05, and Root Mean Square Error of Approximation (RMSEA) < .06 [31]. RMSEAs reflect the fit per degree of freedom and therefore are sensitive to the model’s parsimony. Additionally, a good-fitting model reflects a nonsignificant χ² statistic. However, chi-square statistics are a measure of absolute fit, are impacted by sample size, and often are significant even with good fitting models [31].

Results

Participant Characteristics

At baseline, the mean age of participants was 77.51 (± 5.03) years (age range: 70 – 104), 63.17% of the participants were female, 69.50% were White (24.27% Black; 6.23% Other), and 39.45% were married. On average, participants had 13.56 years of education (range: 3.53 – 23 years); 38.78% had a family income above $30,000 (i.e., lived more than two times above poverty level), 38.60% had a family income between $15,000 and $30,000 (i.e., lived at poverty level to up to two times above poverty level), and 22.61% had family income below $15,000 (i.e., lived below poverty level). The majority of participants (90%) reported being retired.

Substantive Results

ICCs showed that approximately 36 – 59% of the variability in each of the SM items was due to between-person differences. An examination of model fit indices suggested two models (models 2 and 4, see Table 2) that best explained the factor structure of SM items at a between- and within-person level. Model 2, with two within-person level factors and one between-person level factor, had overall good fit indices (χ² (130) = 74.713, p < .001; RMSEA = 0.036; CFI = 0.957; TLI = 0.900). SRMR for the within-person level model was good (0.024) and acceptable for the between-person level model (0.079). At the within-person level, items 1 (Compared with one year ago, do you have trouble remembering things more often, less often or about the same?), 2 (Compared with ten years ago, do you have trouble remembering things more often, less often or about the same?), 3 (In the past year, how often did you have trouble remembering things?), and 6 (Has your memory change caused you any serious health problems?) loaded significantly on factor 1; however, factor loadings for items 3 and 6 were low and below the cutoff value (i.e., 0.225 and 0.136, respectively). None of the items loaded significantly on factor 2. At a between-person level, all items loaded significantly on one factor with factor loadings ranging from 0.345 – 0.850 (see Table 3).

Table 2.

Model Fits for Multilevel Exploratory Factor Analysis Models Examining Subjective Memory

	χ2	df	p	RMSEA	CFI	TLI	SRMR (within)	SRMR (between)
Model 1: 1 factor at the between level and 1 factor at the within level	253.999	18	< .001	0.06	0.834	0.724	0.076	0.079
Model 2: 2 factors at the within level and 1 factor at the between level	74.713	13	<.001	0.036	0.957	0.900	0.024	0.079
Model 3: 1 factor at the within level and 2 factors at the between level	205.763	13	<.001	0.063	0.865	0.688	0.076	0.024
Model 4: 2 factors at the within level and 2 factors at the between level	26.492	8	<.001	0.025	0.987	0.951	0.024	0.024

Table 3.

Factor Loadings for Multilevel Confirmatory Factor Analysis

	Model 2			Model 4
	Within Factor 1	Within Factor 2	Between Factor	Within Factor 1	Within Factor 2	Between Factor 1	Between Factor 2
1.Compared with one year ago, do you have trouble remembering things more often, less often or about the same?	0.926*	−0.008	0.735*	0.926*	−0.008	0.800*	0.024
2.Compared with ten years ago, do you have trouble remembering things more often, less often or about the same?	0.707*	0.093	0.808*	0.707*	0.093	1.008*	−0.080
3.In the past year, how often did you have trouble remembering things?	0.225*	0.135	0.850*	0.225*	0.135	0.678*	0.283*
4.If your memory was 100% when you were thirty years old, what percent would you say it is now?	0.000	1.134	0.793*	0.000	1.134	0.429*	0.525*
5.Comparing important and unimportant things, what percent of the time do you forget things that are important to you?	−0.006	.128	0.525*	−0.006	.128	−0.004	0.762*
6.Has your memory change caused you any serious health problems (e.g., Forgot to turn off the stove, got lost, misplaced valuables)?	0.136*	−0.021	0.345*	0.136*	−0.021	0.108	0.332*

^*p < .05

Model 4, with two within-person level factors and two between-person level factors, had excellent fit (χ² (8) = 26.492, p < .001; RMSEA = 0.025; CFI = 0.987; TLI = 0.951). SRMR for within- and between-person level models was 0.024. As in model 2, at the within-person level, items 1, 2, 3, and 6 loaded significantly on factor 1 and none of the six items loaded significantly on factor 2. At the between-person level, items 1, 2, and 3 loaded significantly on factor 1. Item 3 also loaded significantly on factor 2, but the loading value was low and below the cutoff value. Items 5 (Comparing important and unimportant things, what percent of the time do you forget things that are important to you?) and 6 (Has your memory change caused you any serious health problems?) loaded significantly on factor 2. Additionally, item 4 (If your memory was 100% when you were thirty years old, what percent would you say it is now?) had equivalent loadings on both factors (see Table 3)¹.

Discussion

The current study examined the factor structure of a set of items from an existing longitudinal study of cognitive aging intended to assess subjective memory (SM). We found that the factor structures differed whether we were considering differences between persons or changes within individuals over time. At the within-person level, two factors emerged and fit the data best. The first factor was characterized by items asking an individual to compare their current memory functioning to previous time periods (one and ten years, respectively). However, the second factor was dominated by a single item (i.e., percent change in memory from age 30). Several other items failed to load substantially on either within-person factor. In the between-person solution, a single factor appeared to best represent the SM items in the current study. Although including two factors at the between-person level provided improved model fit, the interpretation of the two factors was not conceptually distinguishable. A single factor at this level was the more parsimonious interpretation.

The within-person factor structure examined how items traveled together over time and two factors provided the best fit. Consistent with the SCD-I proposed dimensions underlying subjective cognitive reports broadly (i.e., ability and change), one of these factors represented perceived change in memory functioning over two different time periods [1,2]. This is also consistent with existing multidimensional measures of SM [8,10,12]. For example, the MFQ includes a subscale of items specific to perceived differences in memory over time [8]. The more complex change item (percent change from age 30) loaded on a separate factor. However, this is a unique item that could require more complex computational effort from participants, thereby tapping into constructs other than SM specifically (e.g., mathematical ability) [32,33]. At the within-person level, three items did not load on either factor. The content of these items appears to be more related to an individual’s perceived memory ability, one of the two dimensions proposed by the SCD-I. There was sufficient within-person variability in these items; however, they did not systematically vary together over time, suggesting that the variability in these items may be linked to changes in other constructs (e.g., depressive symptoms) [34].

For the between-person factor structure of the SM items in the current study, five of the six items loaded on a single factor. This is not consistent with the definition of SM as a multidimensional construct [10,12]. Although the content of the items in the current study conceptually relates to separable subdomains of SM (specifically change and ability), these specific items appear to capture a single dimension to distinguish how one older adult’s SM differs from another. This suggests that more refined items such as those used in multidimensional assessments of SM (e.g., MFQ [8]) are needed to appropriately assess the underlying facets of SM (e.g., perceived functioning, concerns about functioning) at the between-person level. One potential explanation for the clustering of items at the between-person level is the long retrospection period associated with the items with content related to ability. Empirical work examining the role of expanded time frames for reporting indicates that items with longer time frames are more quickly responded to, suggesting a reliance on a person’s beliefs about themselves rather than an accurate representation of their experiences [35–37].

To our knowledge, this is the first attempt to examine how responses to SM items change together over time using multilevel factor analysis. This is in large part due to the use of a single or a set of independent items to assess SM in studies with sufficient longitudinal data to conduct a psychometric evaluation of items [2,15,16]. However, this type of examination can provide important information about the behavior of items over time. The current analysis suggests similar trajectories for items assessing perceived declines in memory functioning. This would imply that a researcher could potentially select one of these items rather than including both in their questionnaire battery. In contrast, items assessing ability appear to be less related to change over time and are therefore likely related to other time-varying constructs [34,38]. More work is needed to examine other time-varying constructs influencing these items and whether different items are differentially impacted (e.g., depressive symptoms). In addition to informing future research, our findings have potential clinical implications. Our results suggest that questions regarding perceived memory decline, regardless of the time frame of retrospection, assess the same experience in an individual over time. For clinicians, who often have only brief visits with patients and are considering many factors in their assessments, a single question about memory change at each annual visit could be important for monitoring purposes. SCD-I recommendations identified “self-experienced persistent decline in cognitive capacity” as a key criterion for identifying preclinical Alzheimer’s disease (AD), and perceived decline in memory as opposed to other cognitive domains as particularly important for identifying heightened AD risk. As evidence accumulates regarding the predictive utility of SM, assessments of perceived memory decline in clinical settings may be warranted and a single assessment question may be sufficient in older adults with no evidence of cognitive impairment. However, additional evidence beyond this single study would strengthen this as a clinical tool.

Our findings have important limitations to consider. First, the current study was limited to the six SM items included longitudinally in a secondary dataset, EAS. Although these items represent some of the most common types that are currently in use in longitudinal cohort studies [2], small changes to wording across different datasets could potentially affect how items are interpreted by older adults, which may result in a change in factor structure. This limits the generalizability of our results. Next, the response scale of some items was limited. As a result, participants had fewer options to choose from which may have resulted in less variability in these items. Additionally, the interpretation of some of the SM items included in this study may vary across participants. While older adults may similarly interpret the simpler items, interpretation of the more complex items (e.g., percent change from age 30) may differ by participants’ ability to remember their memory performance at the age of 30 [32,33]. Responses to this item may also differ by individuals’ ability to calculate and provide a response in a percentage format [39]. Also, responses to the following item (Comparing important and unimportant things, what percent of the time do you forget things that are important to you?) may vary by older adults’ interpretations of important vs. unimportant things. A qualitative study by Hill and colleagues [39] showed that older adults described the question about comparing important and unimportant things as vague and instructions as complicated. The inability of these two complex items to load on any factor suggests that researchers should be careful in utilizing these questions to measure SM. Last, the sample in the current study was primarily a White sample, and therefore, the interpretation of the items may differ for participants who do not identify as White. Given the smaller sample size of participants that identified as Black or “other”, we could not test for measurement invariance by race, i.e., if the factor structure was same across participants from different racial groups. Further, the factor structure identified may be specific to White participants given that they made up the majority of the sample in this dataset. Future research conducting factor analyses should test for measurement invariance based on demographic and other individual characteristics.

Conclusion

Reports of SM in cognitively intact older adults have important implications for health and well-being throughout the aging trajectory. However, our understanding of when and how SM may influence outcomes such as depression or future cognitive performance is limited by inconsistencies in findings across studies. Variation in the items used to measure SM is a potential contributor to these differences [2], and results of the current study can be used to inform future consideration of such influences. Our findings suggest that items assessing memory decline tend to capture the same experience within an individual over time, regardless of the time frame of reference. In contrast, when assessing SM differences across persons, the construct appeared more unitary. In line with the recommendations of SCD-I [1], including items assessing perceived memory decline in research with older adults is likely to capture a construct that differentiates individual differences as well as fluctuations within individuals.