Informant report of practical judgment ability in a clinical sample of older adults with subjective cognitive decline, mild cognitive impairment, and dementia

Abstract

Loss of judgment is a consequence of the dementing process, as executive functions that permit complex, goal-directed behavior progressively fail. Unfortunately, there are a limited number of validated clinical measures to assess judgment and no informant rating scales—even though neuropsychologists overwhelmingly report gathering data from significant others when assessing patients’ judgment ability. We developed an informant-report measure to complement the increasingly used objective Test of Practical Judgment (TOP-J). Based on review of the literature and existing informant report measures, we generated items that enable informants to report compromised judgment in areas tapped by the TOP-J (safety, medical, financial, social-ethical). After item refinement and piloting on informants of patients form a memory disorders clinic, we added the 15-item Likert-type measure to neuropsychological batteries of older adult patients at two clinics in the Midwestern U.S. The sample included 189 patient and informant dyads (mean age = 79.0, mean years of education = 13.8, %female = 67.7) with: subjective cognitive decline, mild cognitive impairment, Alzheimer’s disease, vascular dementia, and behavioral variant frontotemporal dementia. We found psychometric support, including evidence for convergent, discriminant, and criterion-related validity, and internal consistency. Importantly, the TOP-J-Informant discriminated between diagnostic groups in the expected direction. Overall, the TOP-J-Informant is brief (<5 minutes), easy to administer, and well-tolerated by informants. We discuss its potential utility to reveal areas of concern related to poor judgment when administered in the context of a neuropsychological evaluation or clinic visit.

Introduction

Informant report of older adults’ cognitive and adaptive abilities are a crucial component of neurocognitive evaluations. In contrast to objective neuropsychological performance, which may represent a time-limited, “snapshot” of cognitive functioning obtained within a controlled clinical or laboratory setting, informant report by someone who knows an individual well provides valuable information about changes in relevant, ecologically salient capacities (Galvin, 2018). Informant screens are relatively brief and inexpensive, and successfully discriminate between healthy aging, subjective cognitive decline (SCD), mild cognitive impairment (MCI; Ryu et al., 2019), and degree of dementia severity (Neri et al., 2001; Rueda et al., 2015). In SCD, conceptualized as a possible pre-MCI condition (Jessen et al., 2014), self-reported cognitive concerns tend to precede (Caselli et al., 2014) and are elevated relative to those reported by informants (Mulligan et al., 2016; Ryu et al., 2019). Informant and self-report of cognitive concerns become more aligned in MCI, suggesting that self-awareness is decreased at this stage while presentation of problems as observed by informants is increased (Edmonds et al., 2018; Rabin et al., 2017; Ryu et al., 2019). As neurodegeneration progresses and self-assessment abilities diminish, informant report of cognition may become increasingly more accurate relative to self-report—i.e., in late MCI (Edmonds et al., 2014; Rabin et al., 2017; Rueda et al., 2015) and dementia (Rueda et al., 2015).

In preclinical dementia stages, informant report also can be more predictive than self-report of incident cognitive decline (Nicholas et al., 2017; Numbers et al., 2020; Risacher et al., 2013), MCI (Caselli et al., 2014), and dementia (Edmonds et al., 2018; Numbers et al., 2020; Rabin et al., 2012; Risacher et al., 2013). Additionally, relative to self-assessment, informant screens may be more strongly correlated with objective cognitive scores (Edmonds et al., 2018; Farias et al., 2005; Fyock & Hampstead, 2015; Rueda et al., 2015; Slavin et al., 2010). Informant reported cognitive or functional issues are sensitive to positive cerebrospinal fluid Alzheimer’s disease (AD) biomarkers (Edmonds et al., 2014; Rueda et al., 2015), global brain atrophy (Rueda et al., 2015), larger ventricular volume (Rueda et al., 2015), hippocampal (Fyock & Hampstead, 2015; Rueda et al. 2015) and amygdala (Fyock & Hampstead, 2015) atrophy, and diminished whole brain functional connectivity in individuals without objective cognitive impairment (Dong et al., 2018). Taken together, informant report plays an important role in dementia evaluations, serving both as an early marker of functional decline and insidious neural changes, as well as a crucial measure of cognition and function in later disease stages when insight is diminished.

Informant perception of an older adult’s cognitive or adaptive daily functioning may be sensitive to and influenced by executive function abilities. Declines in executive functioning are common in normal aging (Fjell et al., 2017; Kirova et al., 2015; Lezak et al., 2012; Oosterman et al., 2010; Singh-Manoux et al., 2012; Wecker et al., 2005), present in early neurodegenerative processes (Ho & Nation., 2018; Kirova et al., 2015; Seo at al., 2016; Sudo et al., 2017), and frequently impaired in more advanced dementia (Duke & Kaszniak, 2000; Gansler et al., 2016; Guarino et al., 2019; Ramanan et al., 2017; Voss & Bullock, 2004). There is evidence that informant reports of functional problems are closely associated with impairments in executive functioning (Mulligan et al., 2016; Rueda et al., 2015), and that informant reports of executive functioning may portend clinical progression in nondemented older adults (Rabin et al., 2010; Rabin et al., 2012).

An important aspect of executive functioning, practical judgment ability, is highly relevant to real world adaptive capacities in older adulthood, such as avoiding potentially unsafe situations or scams, making sound financial and medical decisions, or engaging in socially appropriate behavior (Quinn et al., 2018; Rabin et al., 2007). Making sound judgments requires the ability to appraise information relevant to a novel situation and formulate conclusions based on thoughtful consideration (Rabin et al., 2007). Practical judgment may be compromised even in older adults with intact neuropsychological functioning or in preclinical disease stages (Han et al., 2016; Stewart et al., 2018). Such individuals may exercise poor judgment in daily life and/or become susceptible to problematic reasoning and decision making (Denburg et al., 2007; Löckenhoff, 2018; Peters et al., 2000).

Identifying older adults at risk for exercising poor judgment is crucial to preventing possible exploitation and abuse (Gatz et al., 2016). Additionally, information derived from judgment evaluations informs diagnosis and provides an objective understanding of safety and functional competence, including the ability to live independently (American Psychiatric Association [APA], 2013; Kim et al., 2002; Quinn et al., 2018). These data may be helpful to family members and loved ones who must prepare for possible changes in an individual’s functional and decision making capacities (Hanks et al., 1999). Because traditional, performance-based measures of executive functions are often limited in ecological validity, clinicians and researchers commonly rely upon self- and informant rating scales (Isquith et al., 2013; Meltzer et al., 2017) to provide both unique and corroborative information about executive functions as they pertain to everyday experiences (Isquith et al., 2013; McAuley et al., 2010; Toplak et al., 2012).

Lapses in judgment and decline in pragmatic executive functioning may be readily apparent to informants who know an individual well. However, there are only a few informant report measures that focus on executive functioning in adults (with some support for their use in older adult populations): the Frontal Systems Behavior Scale (FrSBe; Stout et al., 2003) and Behavioral Rating Inventory of Executive Function-Adult version (BRIEF-A; Roth et al., 2005). Other subjective cognitive measures include items related to executive functioning, such as the: Cambridge Behavioural Inventory (CBI-revised; Wear et al., 2008), Measurement of Everyday Cognition (ECog; Farias et al., 2008), Subjective Cognitive Decline Questionnaire (SCD-Q; Rami et al., 2014); Cognitive Change Index (CCI; Rattanabannakit et al., 2016); Quick Dementia Rating System (QDRS; Galvin, 2015), and AD8 (Galvin et al., 2005). While these measures may contain items that relate to judgment ability, to our knowledge, there are no informant measures that comprehensively assess everyday judgment and related skills. Given that intact judgment is central for functional independence and safety in older adulthood, and informant reporting increasingly corresponds with objective cognitive functioning as the disease progresses (Edmonds et al., 2018), an informant measure of practical judgment would provide clinically valuable information.

The current study introduces an informant-rating measure that taps into everyday judgment problems commonly faced by older adults. The test was developed to accompany the previously validated, objective Test of Practical Judgment (TOP-J, Rabin et al., 2007; Rabin et al., 2009), which assesses judgment related to safety, medical, social/ethical, and financial issues. In the current study we describe the process of developing the Test of Practical Judgment Informant Form (i.e., TOP-J-Informant), provide initial psychometric support, and illustrate how the measure performs in patient groups across the dementia continuum.

Methods

Measure Development and Study Procedures

To create the TOP-J-Informant, an initial group of 25 Likert scale and 5 open-ended items was generated by authors (LR, CQ), after reviewing published cognitive self- and informant report questionnaires and the neuropsychological literature on judgment in older adult populations (identified through searching PsycInfo and PubMed databases). We generated items that: (1) tapped important aspects of practical judgment and related constructs such as problem solving, planning, and decision making; and (2) were similar in content to items on the objective TOP-J. We attempted to create item stems that would be easy to understand, with simple, clear, and unambiguous wording. Items initially consisted of a stem and seven possible response options, ranging from 1 = above average ability to 7 = severe difficulty.

We presented the initial version of the measure to two doctoral level neuropsychologists (including author NP) and one neuropsychology doctoral student, who reviewed the content and wording of items. In addition, we tested the items for comprehension on a small group of informant volunteers (n = 8) of patients assessed at a geriatric clinic housed in a private hospital in the Midwestern U.S. These informants were family members who had accompanied patients to their neuropsychological assessment and agreed to provide informal feedback about a newly developed measure. Based on these responses, we discarded items deemed to be potentially confusing, irrelevant, or redundant. We then retained 12 Likert scale and five open-ended items. The form was then administered to an additional 66 informants at the geriatric clinic (as part of clinical care). Response patterns were reviewed for further refinement. We also requested feedback about item content and the overall length of the measure from the supervising neuropsychologist, bachelor level neuropsychology technicians, and post-doctoral fellows.

During this development period, we undertook item revision, added new items, and modified scoring criteria. The resulting measure contained 15 Likert scale items that ranged from 0 = normal ability/almost never a problem; 1 = mild difficulty/sometimes a problem; 2 = moderate difficulty/often a problem to 3 = severe difficulty/almost always a problem. The total range for the quantitative portion of the measure was 0–45, identical to range of the 15-item version of the TOP-J, with higher scores indicating more severe judgment difficulties or problems. We also added descriptive items to the beginning of the measure that asked informants to report their relation to the patient/participant (spouse, child, friend, caregiver, other), approximate number of years they have known the patient/participant, and approximate hours per week spent with the patient/participant. We retained one open-ended question that may be useful in clinical settings—i.e., “Please describe any other recent situations in which you felt the participant/patient’s judgment was compromised.” The measure is available upon request to authors L. Rabin (lrabin@brooklyn.cuny.edu) and/or C. Quinn (crystalgquinn@gmail.com).

We obtained IRB approval for retrospective data analysis for the current study. As appropriate for clinical care, the TOP-J-Informant was included in test batteries at the geriatric clinic (described above) and at a neuropsychology clinic affiliated with an academic/university medical center in the Midwestern U.S. To be included in retrospective data analyses, participants were diagnosed with SCD, MCI, or dementia, were 60 years of age or older, and fluent in English. Informants included family members, friends and caregivers of patients who had accompanied the patient to the testing session. Informants were excluded if they were not able to answer questions about patients’ judgment ability (e.g., were not cognitively intact themselves based on clinical judgment). At both clinical sites, informants completed the TOP-J-Informant while the patient was undergoing neuropsychological assessment.

Participant Classification

All participants completed a comprehensive clinical assessment that included a neuropsychological test battery. Diagnoses for patients seen as the geriatric clinic were established through case review by a multidisciplinary care team (geriatricians, geriatric neuropsychologists, social workers, geriatric nurse). Diagnoses for patients seen in the neuropsychological clinic were established by the neuropsychologist in conjunction with the referring neurologist. Clinical participant groups included SCD, MCI, vascular dementia (VaD), AD (including AD and mixed AD/VAD—i.e., AD+VaD), behavioral variant frontotemporal dementia (bvFTD) with (i.e., bvFTD+) and without (i.e., bvFTD) a comorbid process such as VaD, parkinsonism, and primary progressive aphasia.

Generally consistent with DSM-5 criteria for major neurocognitive disorder (APA, 2013), patients were classified as having dementia, if: 1) there was substantial progressive cognitive impairment (performing at least 2 standard deviations below the age- and education appropriate normative mean on at least 2 cognitive tests across 2 or more cognitive domains); 2) the patient or informant reported changes in cognitive function; 3) there was evidence of functional decline based on patient and informant report such that the individual requires assistance with daily life activities; and 4) cognitive change was not better accounted for by the effects of a substance or medication.

More specifically, individuals were classified as VaD if they had: 1) evidence of focal neurological signs; 2) evidence of cerebrovascular disease on imaging; and 3) and a subcortical pattern of cognitive impairment (Roman, 1993). Conversely, individuals were classified as having mixed dementia if they met the two first criteria consistent with VaD, but exhibited a pattern of cognitive impairment similar to AD. In order to be classified as dementia due to AD, and consistent with the classic, amnestic presentation (McKhann et al., 2011), memory functions needed to be one of the domains of cognitive impairment. Behavioral variant FTD was diagnosed when individuals classified as having dementia presented initially with changes in personality, social comportment, and cognition, as well as with a predominant pattern of executive dysfunction on neuropsychological testing (Rascovsky et al., 2011).

Patients were classified as MCI (generally consistent with criteria set forth by Petersen, 2004) if: 1) cognitive performance was below the expected range based on all available information from the clinical assessment (neuropsychological test performance, interview with the patient and informant, clinical judgment of multidisciplinary team). For the current study, we required scores on at least two cognitive tests from the clinical battery to be 1.5 SD below the age- and education appropriate normative mean (possible cognitive domains included memory, executive function, language, attention, and visuospatial ability); 2) there was a decline in cognitive performance from baseline as reported by the individual or informant, or observed change on longitudinal testing (if previous test scores are available); and 3) the individual performed daily life activities independently, though there may be detectable but mild functional impact on complex, instrumental activities (IADL), either self- or informant-reported.

The syndromal staging of cognitive impairment, independent of biomarker profiles, accounts for a subset of cognitively unimpaired individuals who report subjective cognitive decline (Jack et al., 2018). We included these individuals as a separate SCD group, based on the increasing recognition that SCD is associated with AD biomarkers and clinical progression (Jessen et al., 2014; Molinuevo et al., 2017; Rabin et al., 2017). Participants were classified as SCD if: 1) there was self-reported persisting cognitive decline from a previous level of cognitive function and unrelated to an acute event (based on the clinical interview); 2) the individual had normal age-, gender-, and education-adjusted performance on standardized cognitive tests (based on tests used to classify MCI or dementia in the clinical assessment); and 3) the individual was able to independently complete ADLs based on information from the clinical interview.

Additional Study Measures

Because data were retrospective and gathered across two different clinical settings with varying referral questions, neuropsychological test batteries varied across participants. All participants received the TOP-J and had informants who completed the TOP-J-Informant. Tests available for most participants and included in our analyses were: Repeatable Battery for the Assessment of Neuropsychological Status (Randolph, 1998), Letter Fluency and Category Fluency from the Delis-Kaplan Executive Function System (D-KEFS; Delis et al., 2001), Rey–Osterrieth Complex Figure Test (ROCF; Rey, 1941).

We gathered basic demographic information (age, gender, race/ethnicity, years of education) from patients; these data were not available for informants. In addition to the TOP-J-Informant, informants completed the Brief Informant Form of Neurobehavioral Symptomatology (BINS; Paré et al., 2020), which consists of 24 Likert items related to cognitive change over the past 2 years, ranging from 0–3 points per item (0–27 in total). Response options include: 0 = never/no change; 1 = occasional/mild change; 2 = often/noticeable change; 3 = very often/severe problem/much worse. An additional 14 open-ended BINS questions inquire about personality, behavior changes, and basic and instrumental ADLs.

Data Analyses

A series of analyses were conducted, including: (1) descriptive statistics for the patient groups on relevant study variables including both quantitative and qualitative TOP-J-Informant items; (2) bivariate Pearson’s correlation between age and TOP-J-Informant scores to understand the strength of association between this demographic variable and TOP-J-Informant performance; (3) Spearman’s correlation between patient’s years of education and TOP-J Informant scores to understand the association between education and TOP-J-Informant performance; (4) bivariate Pearson’s correlations between TOP-J-Informant scores and a) length of informant-patient relationship and b) average weekly time spent with patient; (5) Exploratory Factor Analysis (EFA) to reveal the factor structure of the TOP-J-Informant; (6) bivariate Pearson’s correlations between the TOP-J-Informant scores and BINS, TOP-J 9-item, TOP-J 15-item total scores, and relevant neuropsychological measures for convergent validity evidence; (7) bivariate Pearson’s correlations between the TOP-J-Informant scores and unrelated measures for divergent validity evidence; (8) analysis of variance (ANOVA) between diagnostic group on TOP-J-Informant scores for criterion validity evidence; (9) ANOVA between diagnostic group on age; (10) Kruskal-Wallis H test between diagnostic group on patient’s years of education; (11) Fisher’s exact test between sex and diagnostic group; and (12) alpha coefficient to determine internal consistency/item homogeneity (Cronbach, 1951).

Parametric assumptions were assessed prior to all analyses (Field, 2013; Laerd Statistics, 2015). Square-root transformation was applied to the following variables for parametric analyses: TOP-J-Informant scores due to moderate positive skew, TOP-J 9 and 15-item total scores due to moderate negative skew, length of informant-patient relationship in years due to moderate negative skew, RBANS story recall due to moderate positive skew. Logarithmic transformation was applied to weekly time in hours spent with patient due to strong positive skew. The nonparametric Spearman’s correlation was used to assess the association between patient’s years of education and TOP-J-Informant due to extremely positively skewed education scores that were not corrected by transformation. Similarly, the nonparametric Kruskal-Wallis H test was conducted to determine if there were differences in patient’s years of education between diagnostic group.

Results

Descriptive Statistics and Demographic Effects

The sample included 189 patient and informant dyads, and was predominantly Caucasian/White, with one African-American/Black patient, consistent with the demographic composition of the surrounding region. Table 1 presents relevant descriptive statistics of the informant-patient relationship, including length of relationship, average weekly time spent with patient, and relation to patient. Pearson’s correlation between patient age and TOP-J-Informant scores revealed a statistically significant, weak association, r(187) = .182, p = .012. Spearman’s correlation between patients’ years of education and TOP-J-Informant scores revealed a statistically significant, weak association, r_s(187) = −.146, p = .045. The length of informant-patient relationship in years, r(171) = .034, p = .657, and average weekly time spent with patient in hours, r(128) = −.001, p = .994, were not associated with TOP-J-Informant scores. Fisher’s exact test revealed a nonsignificant difference in sex between diagnostic group, p = .108. Kruskal-Wallis H test revealed a nonsignificant difference in education between diagnostic group, H(4) = 6.174, p = .187.

Table 1.

Informant Data (N = 189)

Number of years of patient relationship		52.31 (13.51) Median = 55.5
Hours per week spent with (or speaking with) the patient		55.45 (67.07) Median = 20.00
Relation to patient (%)	Child	48.6
	Spouse	41.8
	Other	9.6

Validity and Reliability Evidence

Internal structure evidence.

EFA was conducted using a Principal Axis Factoring (PAF). Based on the results of the initial analysis, the scree plot showed a clear one-factor solution. Although two eigenvalues were identified as greater than 1 (i.e., 8.54 and 1.11), not all items were clearly loading to specific factors. Moreover, referring to eigenvalues above 1 has been criticized as tending to overestimate the number of factors (Furr & Bacharach, 2014). Thus, we conducted a second analysis with one fixed factor. Results (Table 2) indicated that one factor was appropriate because all of the items had high loadings on that factor. The scree plot (Figure 1) again supported a one-factor solution.

Table 2.

Factor Loadings with One Factor Extracted

Item	Loading
1	.765
2	.675
3	.677
4	.678
5	.703
6	.697
7	.740
8	.728
9	.799
10	.642
11	.754
12	.756
13	.831
14	.749
15	.791

Figure 1.

Scree Plot with One Factor Extracted

Convergent and discriminant validity evidence.

Correlations were conducted across the entire sample (collapsed across diagnostic group). Results revealed a statistically significant strong correlation between the TOP-J-Informant and another informant measure of general cognition (BINS), r(184) = .747, p = < .001. Statistically significant correlations emerged between the TOP-J-Informant scores and the objective TOP-J 9-item, r(187) = −.231, p = .001, and TOP-J 15-item scores, r(187) = −.233, p = .001, with small effect sizes. We examined associations between the TOP-J-Informant and neuropsychological tests of executive functioning with which the TOP-J-Informant theoretically should correlate with a small-moderate effect size. Statistically significant correlations emerged between TOP-J-Informant scores and D-KEFS Category Fluency Switching scores, r(184) = −.296, p < .001, and RBANS coding scores, r(168) = −.214, p = .005, with small effect sizes. In support of discriminant validity, TOP-J-Informant scores were not significantly associated with measures of memory, including list recognition, r(169) = −.125, p = .104, and story recall, r(172) = −.066, p = .387.

Criterion validity evidence.

There was a statistically significant difference in TOP-J-Informant score between diagnostic group with a medium effect size, F(4, 184) = 7.598, p < .001, ω² = .12. Bonferroni post hoc analysis (Table 3) revealed that TOP-J-Informant scores for each dementia group was statistically significantly higher than the SCD and MCI groups. That is, TOP-J-Informant scores for the bvFTD/bvFTD+ group were statistically significantly higher than for the SCD (p = .001) and MCI (p < .001) groups. TOP-J-Informant scores for the VaD group were statistically significantly higher than for the SCD (p = .027) and MCI (p = .026) groups. TOP-J-Informant scores for the AD/AD+VaD group were significantly higher than for the SCD (p = .045) and MCI (p = .012) groups.

Table 3.

Patient Demographic and Informant Score Data

	Diagnostic Group					Group Differences
	SCD (n = 17)	MCI (n = 48)	AD/AD+VaD (n = 88)	VaD (n = 14)	bvFTD/bvFTD+(n = 22)
TOP-J-Informant	10.59 (9.88)	11.02 (8.85)	17.27 (11.52)	20.36 (9.68)	22.82 (11.69)	p < .001 bvFTD/bvFTD+ > SCD***, MCI**** VaD > SCD*, MCI* AD/AD+ > SCD*, MCI**
Age	76.24 (7.01)	77.38 (7.27)	80.98 (5.46)	79.71 (5.74)	76.05 (7.52)	p = .001 AD/AD+ > MCI*, bvFTD/bvFTD+*
Education	14.18 (2.07)	14.00 (2.58)	13.73 (2.33)	12.50 (1.16)	13.77 (2.11)	NS (Kruskal-Wallis H test)
Gender (M, F)	5, 12	18, 30	21, 67	6, 8	11, 11	NS (Fisher’s exact test)

Note. Data are mean (SD) except for gender. SCD = subjective cognitive decline; MCI = mild cognitive impairment; AD = Alzheimer’s disease; VaD = vascular dementia; bvFTD = behavioral variant frontotemporal dementia. ANOVA analyses for TOP-J-Informant score used the Bonferroni correction for post hoc comparisons.

^*p<.05;

^**p=.01;

^***p=.001;

^****p<.001

Reliability evidence.

The alpha coefficient was determined to be .95.

Additional Descriptive Results

Across the entire sample (collapsed across diagnostic group), item 10 “has trouble making up his/her mind” received the highest score (M = 1.49, SD = 0.99), followed by item 5 “manages medical matters” (M = 1.45, SD = 1.10) and item 2 “comes up with various ways to solve a problem” (M = 1.32, SD = 0.92). These three items were the highest rated in each diagnostic group of varying order, with the exception of the SCD group in which item 8 (“handles sensitive social situations”) was slightly higher than item 2. The item with the lowest score was item 7 “is ethically responsible,” and this was true in the entire sample (M = 0.46, SD = 0.89) and within each diagnostic group. See Table 4 for ranking of the three highest scored items by diagnostic group.

Table 4.

Ranking of the three items with highest scores by diagnostic group

Diagnostic Group	Item	Mean
SCD	10. Has trouble making up his/her mind	1.06
	8. Handles sensitive social situations	1.00
	5. Manages medical matters	0.94
MCI	10. Has trouble making up his/her mind	1.33
	5. Manages medical matters	1.10
	2. Comes up with various ways to solve a problem*	0.98
	3. Carries out a plan*	0.98
AD / AD+VaD	5. Manages medical matters	1.52
	10. Has trouble making up his/her mind	1.51
	3. Carries out a plan	1.44
bvFTD / bvFTD+	5. Manages medical matters*	1.91
	10. Has trouble making up his/her mind*	1.91
	2. Comes up with various ways to solve a problem**	1.86
	1. Uses good judgment**	1.73
VaD	5. Manages medical matters**	2.07
	10. Has trouble making up his/her mind	1.79
	15. Handles emergencies	1.64

^*Items are tied in endorsement ranking;

^**Item was endorsed in all patients within diagnostic group

In terms of the qualitative/open-ended item, 31% of respondents chose to provide a response (or multiple responses), and responses generally coincided with the domains assessed by the TOP-J-Informant (Table 5 presents sample responses). Some of the responses did not reflect problems in judgment and related skills—instead they related to problems with memory, language, IADLs, or medical/psychological issues.

Table 5.

Sample responses to open-ended question about recent examples of compromised judgment

Domain	Response
Safety	Allows people into his house without knowing them, late at night on occasion Left dog in hot car (with bad outcome)
Medical	Rearranges pills that have been set out, changes dosage/time Covers up medical issues in front children, refuses their offers of assistance
Financial	Sent $10k to craigslist scam Withdraws large amounts of cash without checking bills/balance
Social	Allows young children to watch inappropriate movies, does not understand why an issue Threw temper tantrum when asked not to cut the grass, lashed out at spouse

Discussion

Practical judgment, under the umbrella domain of executive functioning, is an ecologically relevant ability that underlies functioning and safety in daily life, and a cognitive domain almost always assessed by neuropsychologists during dementia evaluations (Rabin et al., 2008). Informant reports of cognitive and adaptive functioning skills are increasingly recognized as crucial markers of diminished decision making and functional capacity (Edmonds et al., 2018; Farias et al., 2005; Fyock et al., 2015; Rueda et al., 2015; Slavin et al., 2010) as well as risk for incident cognitive decline (Caselli et al., 2014; Edmonds et al., 2018; Nicholas et al., 2017; Numbers et al., 2020; Rabin et al., 2012; Risacher et al., 2013), especially when assessing executive functioning related abilities (Rabin et al. 2010; Rabin et al., 2012). Moreover, older adults deal with a multitude of complex life matters that have important consequences (e.g., estate, savings, and retirement planning, assistant living/nursing home placements, medical problems and associated costs, role shifts following the death of a spouse). When one considers possible executive dysfunction, in conjunction with fraudulent and nefarious intentions aimed at older adults, the importance of assessing judgment in older adults cannot be understated. Unfortunately, no informant report measures of judgment with evidence of validity and reliability are currently used, likely reducing the ability of clinicians and researchers to identify older adults at risk for abuse, exploitation, and compromised decision making in essential domains. To address this gap, we present the TOP-J-Informant, a rating measure that taps into everyday judgment problems commonly faced by older adults. Our results demonstrated preliminary psychometric evidence including strong reliability, strong association with another informant report measure of general cognition, and the ability to distinguish between dementia and nondementia groups.

Our approach to development of the TOP-J-Informant sought to complement the objective TOP-J, by including items related to safety, financial, social/ethical, and medical domains. We reviewed the literature related to clinical assessment of judgment and related constructs (e.g. planning, problem solving decision making), consulted on item development with neuropsychologist colleagues, and piloted the TOP-J-Informant to assess for comprehension, accessibility, and response patterns for refinement of item and scoring criteria. The resulting measure, comprised of 15 Likert scale items, ranging from 0 = normal ability/almost never a problem to 3 = severe difficulty/almost always a problem, surveys issues regularly faced by older adults as observed by informants. We also include one open-ended item (for use in clinical settings), inquiring about recent examples of compromised judgment. With promising utility in both clinical and research settings, the TOP-J-Informant is brief (< 5 minutes), simple to administer, and well-tolerated by informants.

As patient age increased and at lower levels of education, informant report of problems with judgment increased, although only to a minimal extent. In addition, there was no association between the TOP-J-Informant scores and length of informant-patient relationship and time spent with the patient. However, these results should be interpreted with caution because the restriction of range on these characteristics may have attenuated the association with TOP-J-Informant scores. Specifically, our informants on average had longstanding, multi-decade relationships with the patients and spent many hours with the patient each week. Future investigations should include a more diverse range of informant-patient relationship strength, as individuals without close relationships or who experience social isolation may be at increased risk for cognitive and functional decline (Andrew & Rockwood, 2010).

The TOP-J-Informant demonstrated a single-factor internal structure, which is also found in the original TOP-J (Rabin et al., 2007). We found statistically significant evidence for convergent validity as demonstrated by a strong association with another informant report of general cognitive abilities (i.e., BINS), and with weaker yet statistically significant associations with the 9- and 15-item TOP-J versions, and with other objective measures of executive functioning. Divergent validity evidence emerged as weak nonsignificant associations with objective memory measures. Further, as evidence of criterion validity, the TOP-J-Informant was able to distinguish between dementia and nondementia groups (i.e., average AD/AD+VaD, VaD, bvFTD/bvFTD+ scores were each significantly higher than SCD and MCI). An unsurprising pattern was revealed, with the lowest level of judgment problems reported for the SCD group, followed by MCI, AD/AD+VaD, VaD, and bvFTD/bvFTD+, consistent with overall levels of objective cognitive impairment and specific deficits in executive functions in these patient groups (Elderkin-Thompson et al., 2004; Karantzoulis & Galvin, 2011). This pattern is also consistent with previous research suggesting that informant report of cognitive and adaptive functioning problems is lowest in preclinical stages (Mulligan et al., 2016; Ryu et al., 2019) and increases as dementia ensues (Edmonds et al., 2014; Edmonds et al., 2018; Rabin et al, 2017; Rueda et al., 2015; Ryu et al., 2019). With regard to reliability evidence, the alpha coefficient of the TOP-J-Informant was .95, indicating strong internal consistency.

Although the TOP-J-Informant is a unitarily structured measure that generates a total score, representing overall problems with practical judgment abilities, particular attention to specific item endorsement by the informant may yield clinically useful data. Therefore, we also examined the item-by-item pattern of responding in the entire sample and by diagnostic group. Item 10 “has trouble making up his/her mind”, item 5 “manages medical matters”, and item 2 “comes up with various ways to solve a problem” were among the items endorsed as most problematic by informants, regardless of clinical diagnosis. Endorsement of these highly rated items by informants might cue the clinician that, in the proper clinical context, early changes in judgment may be occurring.

In addition, among those with SCD, item 8 “handles sensitive social situations” was highly endorsed (more so than item 2), which may indicate that subtle changes in social realms occur early in the disease process but become less concerning to informants as other, more problematic deficits in judgment emerge later in the disease course. In bvFTD/bvFTD+, item 1 “uses good judgment” was highly rated as problematic, consistent with the frontal dysexecutive clinical presentation of these individuals. In addition, all patients with bvFTD/bvFTD+ were identified as having at least mild difficulty on items 1 “uses good judgment” and 2 “comes up with various ways to solve a problem.” Together, these findings may reflect overarching issues that these individuals experience with judgment and with divergent thinking and problem solving, often reflected by prominent issues with perseverative behavior or cognitive rigidity. Finally, across the entire sample and within each diagnostic group the least endorsed was item 7 “is ethically responsible.” While further research would be required to determine the underlying reason(s) for this low endorsement, possible explanations include that it was a less well understood item or that loved ones were underreporting problems with this sensitive issue that touches upon personal values and ethics.

Overall, given its brevity and simplicity, the TOP-J-Informant is well suited to be used as a quick screen, which may be administered during a neuropsychological evaluation or before a clinic visit. Items endorsed (or additional problems noted for the open-ended question) may serve as a guide for clinical interview and facilitate detection of possible areas of concern related to judgment abilities. Such a use of the TOP-J-Informant may yield essential information to help safeguard older adults at risk for cognitive and functional decline, exploitation, and dangerous decision making.

Our study is not without limitations. The available sample size was relatively small, particularly within specific diagnostic groups, and largely homogenous across sociodemographic variables. Because this was a clinical sample, we did not have the opportunity to include healthy control participants without subjective or objective cognitive deficits. Future research should investigate the TOP-J-Informant in larger and more demographically diverse samples, including cognitively healthy older adults to establish normative data. We present cross-sectional data; future studies should investigate the ability of the TOP-J-Informant to predict cognitive and functional decline longitudinally. Additionally, previous work has indicated that informant report may be influenced by characteristics such as mood/affect (Jorm et al., 1994), personality (Best et al., 2019; Sutin et al., 2019), sociocultural factors (Hackett, et al., 2020) of both the informant and the patient. Exploring the manner in which such variables may be related to TOP-J-Informant scores was limited by the homogeneity of the sample and beyond the scope of the current study, but should be addressed in future work. Finally, in future research we would hope to provide additional validity evidence for the TOP-J-Informant by evaluating its association with specific neuroanatomical correlates (e.g., prefrontal brain regions of patients).