Evaluation of PROMIS Cognitive Function Scores and Correlates in a Clinical Sample of Older Adults

Abstract

In this study we assessed the utility of self-reported cognitive function using two PROMIS^® Cognitive Function (PROMIS-CF) items in an observational clinical sample of patients aged 65 and older (n = 16,249) at a large health system. We evaluated the association of PROMIS-CF scores with clinical characteristics and Montreal Cognitive Assessment (MoCA) scores, and we used logistic regression to examine predictors of 1-year decline in PROMIS-CF scores among patients with available data. PROMIS-CF scores were associated with clinical characteristics as hypothesized, with lower (more impaired) scores for patients with cognitive impairment (CI) diagnoses, multiple comorbidities, and those taking cognitive enhancing or interfering medications. PROMIS-CF scores were also positively associated with MoCA scores. Predictors of 1-year decline in PROMIS-CF scores included CI diagnoses, use of cognitive enhancing medications, higher depression scores, and lower social role function. Our findings suggest potential utility of PROMIS-CF items in a brief patient-administered screening tool for CI.

Introduction

An estimated 12% to 18% of individuals aged 60 and older have mild cognitive impairment (MCI) (Alzheimer’s Association, 2022). Considered an intermediate stage between normal aging and dementia, MCI is characterized by objective cognitive decline without notable interference in daily functioning (Sabbagh et al., 2020). Individuals with MCI are more likely than those without MCI to develop dementia, particularly Alzheimer’s dementia (Alzheimer’s Association, 2022). A systematic review found that approximately one-third of individuals with MCI develop Alzheimer’s dementia within 5 years (Ward et al., 2013). Despite the presence of early warning signs, MCI and dementia are often unrecognized by primary care clinicians until impairment becomes severe (Alzheimer’s Association, 2022).

Though current evidence is insufficient to evaluate the benefits versus harms of universal cognitive screening for older adults in primary care (Patnode et al., 2020), assessment of cognition is a required component of the Medicare Annual Wellness Visit. Early diagnosis of Alzheimer’s disease and related dementias through detection of cognitive impairment (CI) may facilitate timely intervention, recruitment of patients into clinical trials early in the disease trajectory, and provide important information to support care planning conversations while patients are still able to communicate their wishes. Cognitive screening may also enable identification of reversible causes of CI. However, a major limitation of existing cognitive screening tools is the time and burden to administer during short primary care visits (Sabbagh et al., 2020). A brief, reliable, and validated cognitive assessment that can be patient-administered may be more likely to be adopted and sustained in practice. The use of a patient-reported outcome measure can alert the clinician to subtle changes in cognition that may imply a need for further assessment and follow-up. In this study, we assessed the utility of self-reported cognitive function using two Patient Reported Outcome Measurement Information System^® Cognitive Function (PROMIS-CF) items in an observational clinical sample of patients aged 65 and older. Our aim was to evaluate the association of PROMIS-CF scores with clinical characteristics and other measures of cognitive function.

Methods

Data

We used electronic medical record (EMR) data from a large health system in western Pennsylvania that began routine collection of two PROMIS-CF items in outpatient specialty clinics, primarily neurology clinics. We extracted EMR data for 16,249 patients aged 65 and older with one or more PROMIS-CF scores collected between December 2017 and March 2020. We obtained data on patient demographics (age, sex, and race/ethnicity) and clinical characteristics (ICD-10 diagnoses, medications, other cognitive assessments, and other PROMIS assessments). The study was approved with a waiver of consent by the lead author’s institutional review board.

Variables

Diagnoses

For each patient, we identified all ICD-10 codes from clinical encounters occurring within 6 months prior to the first PROMIS-CF assessment. We measured comorbidity burden using the Charlson Comorbidity Index (Charlson et al., 1987). To identify patients with neurologic diagnoses associated with cognitive impairment, we used the Clinical Classifications Software Refined (CCS-R) to group patients’ ICD-10 diagnoses into clinically meaningful diagnostic groups, including neurologic disorders. Two physicians, a general internist and a neurologist, reviewed all CCS-R categories under neurologic disorders and classified them as not associated with CI, possibly associated with CI, or definitely associated with CI (Supplemental Table S-1). For cases in which the CCS-R category was too broad to classify, the individual diagnoses within that category were classified using the same criteria (Supplemental Table S-2). We used this system to classify each patient as having no CI, possible CI, or definite CI based on their ICD-10 diagnoses.

Medications

For each patient, we identified all medication orders written within 6 months prior to the first PROMIS-CF assessment. We identified classes of medications that may cause cognitive impairment, informed by the American Geriatrics Society Beers Criteria (2019 American Geriatrics Society Beers Criteria^® Update Expert Panel, 2019). We categorized patients as taking cognitive interfering medications if they had one or more orders for medications in any of the following classes: benzodiazepines, Z-drugs, narcotics, anticholinergics, steroids, and H2 receptor antagonists. We also identified classes of medications prescribed to treat symptoms of dementia and categorized patients as taking cognitive enhancing medications if they had one or more orders for acetylcholinesterase inhibitors or NMDA receptor antagonists.

Cognitive assessments

PROMIS-CF

We derived the PROMIS-CF score from two items: (1) My memory has been as good as usual and (2) I have been able to focus my attention, which ask patients to self-report on their cognitive abilities with the following response scale: 1 = Not at all, 2 = A little bit, 3 = Somewhat; 4 = Quite a bit, and 5 = Very much. These items assess patient-perceived cognitive function in the domains of memory and concentration, respectively. Higher scores indicate better function. Patterns of scores for the two items combined were transformed into the T-score metric (mean = 50, SD = 10) using PROMIS item parameters (Supplemental Table S-3). These items came from the PROMIS v2.0 Cognitive Function Abilities item bank, a collection of items developed to assess cognitive function perceived by individuals (Lai et al., 2014). All items have established reliability and validity in medical outpatient populations, including associations with cognitive performance outcomes (Howland et al., 2017).

Montreal Cognitive Assessment (MoCA)

Among patients with PROMIS-CF scores, we identified the subset of patients who also had a clinician-administered MoCA within 30 days of the PROMIS-CF (n = 656). The MoCA is widely used to screen for MCI and may be administered when clinicians suspect the existence of MCI (Nasreddine et al., 2005). Scores range from 0 to 30. Carson et al. (2018) recommend a cutoff score of 23 for detection of MCI.

PROMIS assessments

Patients completed PROMIS short form assessments for depression, anxiety, fatigue, pain interference, sleep disturbance, physical function, and social role function. Tailored two-item short forms from each PROMIS domain item bank were selected for their psychometric properties and clinical relevance for use in specialty clinics throughout the health system. All short forms use item response theory-based scoring.

Statistical Analysis

We used analysis of variance and t-tests to examine the association of PROMIS-CF scores with CI diagnoses, comorbidity burden, and cognitive enhancing and interfering medications. We hypothesized that PROMIS-CF scores are associated with clinical characteristics, with lower (more impaired) scores among patients with a definite CI diagnosis, those with multiple comorbidities, and those taking cognitive enhancing or interfering medications.

We evaluated correlations between PROMIS-CF scores and the MoCA. We hypothesized that: (1) patients with a recorded MoCA score have lower PROMIS-CF scores than those who did not complete a MoCA; (2) the proportion of patients with a low MoCA score (below 23) increases as PROMIS-CF scores decrease; and (3) MoCA and PROMIS-CF scores are positively correlated. We also evaluated correlations between PROMIS-CF scores and other PROMIS short form assessments.

Some patients had multiple PROMIS-CF scores collected during the study period. Among patients with available data who had baseline scores within normal limits (n = 4,133), we used adjusted logistic regression to evaluate clinical characteristics associated with 1-year decline in PROMIS-CF scores. “Within normal limits” was defined as a PROMIS-CF score within 0.5 standard deviations of the mean or higher (i.e., a score of 45 or higher; Rothrock et al., 2019). One-year decline was defined as a decrease in PROMIS-CF scores from within normal limits at baseline to below normal limits (i.e., a score of 44 or lower) the following year. Models were adjusted for patient demographics (age group, sex, and race/ethnicity). Independent variables included CI diagnoses, number of comorbidities, use of cognitive enhancing and interfering medications, and other PROMIS assessments.

Results

The patient sample was 54% female and predominantly non-Hispanic white, ranging in age from 65 to 101 years (Supplemental Table S-4). The distribution of PROMIS-CF scores for all patients ranged from 29.7 to 62.2 with a mean of 49.9 and a median of 50.7. The distribution of PROMIS-CF scores for patients with a recorded MoCA score had the same range but a slightly lower mean (47.4) and median (47.6). PROMIS-CF scores were associated with clinical characteristics as hypothesized, with lower (more impaired) scores for those with a definite CI diagnosis, those with multiple comorbidities, and those taking cognitive enhancing or interfering medications (Table 1).

Table 1.

PROMIS Cognitive Function Scores According to Clinical Group.

	N	Mean (SD)	Test
Cognitive impairment diagnosis			F = 464.68
			p < .0001
No CI diagnosis	7,655	51.5 (8.2)
Possible CI diagnosis	4,811	49.9 (8.3)
Definite CI diagnosis	3,783	46.5 (8.2)
Number of Charlson comorbidities			F = 24.05
			p < .0001
None	8,250	50.2 (8.4)
One	3,884	50.0 (8.5)
Two or more	4,115	49.1 (8.5)
Cognitive enhancing medications			t = 29.33
			p < .0001
No	14,431	50.5 (8.3)
Yes	1,818	44.5 (7.7)
Cognitive interfering medications			t = 6.43
			p < .0001
No	6,747	50.4 (8.5)
Yes	9,502	49.5 (8.4)

Among patients with a PROMIS-CF score, 656 had a recorded MoCA score within 30 days of the PROMIS-CF, with 97% of MoCAs administered on the same day. The percentage of patients with a low MoCA score (below 23) increased with decreasing PROMIS-CF scores, ranging from 51.2% for patients with a PROMIS-CF score of 60 or greater to 79.1% for those with a PROMIS-CF score under 40 (Supplemental Table S-5). As hypothesized, PROMIS-CF scores were positively correlated with MoCA scores (Pearson’s r = .27, p < .0001). PROMIS-CF scores were also correlated with scores on other PROMIS assessments. PROMIS-CF scores were significantly negatively correlated with depression (r = -.39; p < .0001), anxiety (r = −.37; p < .0001), fatigue (r = −.33; p < .0001), pain interference (r = −.19; p < .0001), and sleep disturbance (r = −.23; p < .0001) and positively correlated with physical function (r = .26; p < .0001) and social role function (r = .35; p < .0001; Supplemental Table S-6).

Among the patients with longitudinal data who had baseline PROMIS-CF scores within normal limits (n = 4,133), 18% scored below normal limits 1 year later. Controlling for patient demographics, characteristics associated with 1-year decline in PROMIS-CF scores included CI diagnoses (OR = 1.47), use of cognitive enhancing medications (OR = 2.57), higher depression scores (OR = 1.04), and lower social role function (OR = 0.98; Supplemental Table 7).

Discussion

This study provides evidence that PROMIS-CF items are useful in a brief, self-administered CI screening tool. PROMIS-CF scores in a large clinical sample of older adults were associated with clinical characteristics and MoCA performance as hypothesized, and scores changed meaningfully over time. Correlations between PROMIS-CF and MoCA scores were moderate, which is consistent with previous findings (Howland et al., 2017) and may reflect differences between subjective and objective measures of cognition. Though self-reported measures are subject to variation in individual definitions of cognitive function, longitudinal analyses remove the influence of these baseline differences (Howland et al., 2017). The association of a decline in PROMIS-CF scores with clinical characteristics such as CI diagnoses and use of cognitive enhancing medications provides preliminary evidence to suggest the PROMIS-CF may be sensitive to objective cognitive decline.

In the absence of an efficient, inexpensive and highly reliable test for CI, clinicians need a range of diagnostic tools to increase early detection, from initial brief screening tools like the PROMIS-CF to intensive neuropsychological evaluation. Given the time and resource constraints in primary care, development of an extremely brief, patient-reported cognitive assessment offers advantages over performance-based measures of cognition such as the MoCA that take more time to administer and introduce potential for administrator error. Patient-reported measures of cognitive function can provide information to understand patients’ cognitive behaviors in the real world, in addition to offering flexibility in timing and mode of administration (Howland et al., 2017; Lai et al., 2014). Due to their standardization and ease of interpretation, documentation of PROMIS-CF scores in the EMR on an annual basis may allow clinicians to track subtle changes in cognition over time. However, self-reported cognitive assessments also have potential drawbacks, as some evidence suggests a lack of correlation between subjective and objective measures of cognition (Hess et al., 2020; Lau et al., 2021). Further work is needed to understand the clinical utility of self-reported cognitive assessments, which may be useful as a prescreening or to augment performance-based measures of cognition.

Our findings should be interpreted in the context of some limitations. Our data are limited to a single health system and were collected predominantly in neurology clinics, therefore generalizability to other patient populations may be limited. We did not have extensive patient demographic data such as education level or literacy. Among patients with cognitive complaints, those with higher education levels may be more likely to endorse responses indicating cognitive dysfunction compared to those with lower education levels, or vice versa (Fieo et al., 2016). Further studies should be conducted to evaluate the impacts of literacy and education level on PROMIS-CF scores. In addition, patients in our sample completed only two items from the PROMIS Cognitive Function Abilities item bank. However, our findings should theoretically extend to other items in the bank, and scores based on more items may be more robust. Further validation in an older adult primary care population is necessary to evaluate the utility of the PROMIS-CF for cognitive screening. Prospective studies can help to better understand the impact of universal cognitive screening on clinician behaviors such as testing, diagnosis, and referrals, as well as downstream effects on patient outcomes.