Evaluating Residual Cognition in Advanced Cognitive Impairment: The Residual Cognition Assessment

Abstract

Background

In nursing homes, most of the patients with dementia are affected by severe cognitive disorder. Care interventions follow an accurate and recurring multidimensional assessment, including cognitive status. There is still a need to develop new performance-based scales for moderate-to-advanced dementia.

Objectives

The development of the Residual Cognition Assessment (RCA) responds to the need to create new scales for global cognitive screening in advanced dementia, with some peculiar features: performance based, brief (<5 m), available without specific training, and suitable for nonverbal patients with minimal distress.

Methods

Two raters have administered the RCA and the Severe Impairment Battery-short version (SIB-S) to 84 participants with MMSE = 0. After 2–3 weeks, the same sample has been retested. The RCA has been also administered to 40 participants with MMSE 1–10 for a comparison.

Results

The RCA has exhibited excellent values for test-retest reliability (intraclass correlation [ICC] = 0.956) as well as for inter-rater reliability (ICC = 0.997). The concurrent validity analyzes have shown strong correlations between the RCA and the SIB-S with ρ = 0.807 (p < 0.01), and the RCA and the Clinical Dementia Rating (CDR) with ρ = −0.663 (p < 0.01). Moderate correlation has been found between the RCA and the Functional Assessment Staging Scale with ρ = −0.435 (p < 0.01). The instrument has showed high internal reliability, too (total: α = 0.899). The RCA has low floor effect (2%) with respect to the SIB-S (58%) but shows ceiling effect in the MMSE 1–10 sample (50%). The ROC curve analyses demonstrate that the RCA is acceptably able to discriminate between subjects with CDR 4/5 with an AUC of 0.92. Exploratory factor analysis shows 3 factors, defined as three major degrees of cognitive performance in advanced dementia, indeed hierarchically structured in three possible levels of decline.

Conclusions

The RCA has showed excellent validity and reliability as well as good sensitivity to identify advanced cognitive impairment in dementia, without floor effect. The RCA seems complementary to the MMSE, so advisable when the latter reaches 0. Administration and scoring are simple, and only few minutes are required to assess the patient. The RCA can discriminate at least 3 different major stages in advanced dementias: severe, profound, and late.

Introduction

Though unique epidemiological data are not available, people with advanced dementia represent 28% of the over 85 population [1], reaching 31% in nursing homes [2]. It seems therefore necessary to consistently measure their needs in order to define the best intervention and care strategies following an accurate and recurring multidimensional assessment (organic, cognitive, functional, psychological, and social) [3]. Ferris et al. [4] recommend to separate the measure of cognitive function from motor and behavioral ones, also in advanced dementia.

One of the hardest and most fascinating challenges in neuropsychology is evaluating residual cognition in advanced cognitive impairment. Neuropsychology usually employs diagnostic instruments in the general population to collect the first signs of impairment, like the Mini Mental State Examination [5], the Mini-ACE [6], the MoCA [7], and the RUDAS [8]. All of them show good psychometric properties, even though they diverge in sensibility and specificity in different settings [9, 10, 11]. The floor effect [12, 13] is their main limit, leading to the risk of homogenizing different conditions and neglecting residual and marginal cognitive abilities. To cope with this limit, since the ‘90s, several cognitive screening batteries have been specifically designed to evaluate moderate to severe cognitive decline.

The Hierarchic Dementia Scale (HDS) [14] is based upon a model of hierarchical organization of mind: cognitive functions hierarchically organize themselves during childhood and symmetrically de-organize in dementia. However, the authors observed a large variability between individual decline patterns, which is later confirmed by a low intercorrelation in some subscales [15]. The same approach was adopted in the Modified Ordinal Scales of Psychological Development (M-OSPD) [16], which applies a reversed and modified scale for childhood.

From a different point of view, the Severe Impairment Battery (SIB) [17] comes from an oriented to single-performance assessment. An author's aim was to test nonverbal and automatic responses, elicited by gestual clues based upon previous social skills (over-knowledge), well established in crystallized intelligence [18, 19]; for the same reason, the battery is proposed as an ecological and fluent interview, rather than a classical test presentation.

Previous literature has often emphasized the value of performance-based methods in advanced dementia [18, 19, 20]. Volicer et al. [21] and Doody et al. [22] found that the SIB has too many verbal items and testing equipment, plus a 30-min assessment time that exceeds attention span in advanced dementia. Furthermore, the SIB shows certain floor effect. The battery was revised in 2005, introducing a shorter version, that is the SIB-short version (SIB-S) [23], which reduces testing time but still requires verbal responses, specific equipment, and examiner training [18, 20, 22]. The following version, the SIB-8 [24], is further reduced, with a 3-min testing time but still many verbal responses.

The Test for Severe Impairment (TSI) [25] examines 7 cognitive domains. This test suffers floor effect in advanced dementia [20, 26] and requires equipment not readily available in typical psychometric settings [18].

The Bedford Alzheimer Nursing Severity Scale (BANS-S) [21], as well as the Severe Impairment Rating Scale (SIRS) [27], shows greater sensitivity in advanced dementia. However, cognitive assessment is limited to a few domains, and mixed with functional and motor behaviors [20]. Furthermore, the BANS-S requires information about patients' behaviors outside the psychometric setting, which is not always available [21].

Another multidimensional battery is the Severe Cognitive Impairment Profile (SCIP) [28], developed for a longer and exhaustive assessment [22]. The Baylord Profound Mental State Examination (BPMSE) [22] has a behavioral and a cognitive subscale; the second one contains 4 domains, mostly assessed by verbal items.

The Severe Mini-Mental State Examination (SMMSE) [18], based on the MMSE, is designed to assess cognitive domains in moderate-to-severe cognitive decline, using items that require verbal skills [19] and that are culture-specific [20]. The scoring, going from 0 to 30 following the most common MMSE, was considered by the authors a facilitation for clinical practice. The Severe Cognitive Impairment Rating Scale (SCIRS) [29] assesses moderate-to-severe cognitive decline, even though it shows floor effect in advanced dementia [20], and some lacks in psychometric reliability [19].

The Clinical Evaluation of Moderate-to-Severe Dementia (KUD; Swedish acronym) [19] evaluates advanced decline using 15 items that require fair verbal understanding. The Cognitive Test for Severe Dementia (CTSD) [20] explores 7 cognitive domains, showing low floor effect, but requires verbal responses and specific equipment.

Therefore, the following features are advisable in a battery for fast and global cognition screening in moderate to advanced cognitive decline: (1) repeatable assignments, able to overcome even the verbal comprehension through gestural clues; (2) performance-based items, evaluating simple and automatic − especially nonverbal − reactions (over-knowledge); (3) items suitable for any condition, even for bedridden patients; (4) assessment of as many cognitive domains as possible, including social skills, separated from mood, or motor aspects; (5) ecological administration, as a smooth and motivating interview; (6) testing time within 10 min (including time to establish contact), ideally around 5 min; (7) no specific equipment, limited to available instruments in typical clinical settings; (8) no specific examiner training, simple scoring, preferably with the already known 0–30 range.

To date, none of the abovementioned batteries owns all of these features (Table 1). Specifically, quoting Boller et al. [30], “they are appropriate for the assessment of individuals at the ‘severe’ stage of dementia, but there is a stage between severe and ‘vegetative’ which we might call ‘profound’ dementia during which measurable levels of cognitive functions still occur. Appropriate tools to assess functioning at this level are not yet available” [30]. Accordingly, there is still a need to develop new performance-based scales for moderate-to-advanced dementia [19, 23].

Table 1.

Comparative table between different scales for the screening of severe cognitive decline

Scale	Testing time'	N Item	EFA	Concurrent validity	Internal reliability	Test-retest reliability	Inter-rater reliability
HDS (Cole et al. [14])	26	200		r = 0.72 BS r = 0.74 CS r = −0.71 CDR	α = 0.97	0.84	0.89
M-OSPD (Sclan et al. [16])	30	55		ρ = −0.77 FAST	α = 0.95	ICC = 0.99
SIB (Saxton et al. [17])	30	51		r = 0.74 MMSE		r = 0.85	r = 1
SIB-S (Saxton et al. [23])	15	26		r = 0.97 SIB ρ = −0.61 ADL
SIB-8 (Schmitt et al. [24])	3	8		r = 0.90 SIB	α = 0.80
TSI (Albert et al. [25])	10	24	3	r = 0.83 MMSE	α = 0.91	r = 0.96
BANS-S (Volicer et al. [21])		7		ρ = 0.56 ADL ρ =0.65 TSI r = −0.31 MMSE	α = 0.64		r = 0.84
SIRS (Rabins et al. [27])		11		r = 0.87 GCS r = 0.77 MMSE	α = 0.76	r = 0.97	r = 0.99
SCIP (Peavy et al. [28])	30	160		r = 0.91 DRS r = 0.84 MMSE r = 0.93 SIB		r = 0.96	r = 0.99
BPMSE (Doody et al. [22])	5	25		r = 0.76 MMSE r = −0.68 CDR r = −0.58 IADL	α = 0.92	r = 0.95	r = 0.97
SMMSE (Harrell et al. [18])		10		r = 0.61 MMSE		r = 0.75	r = 0.99
SCIRS (Choe et al. [29])	4	11	2	r = 0.96 SMMSE r = 0.86 MMSE r = −0.83 CDR	α = 0.93	r = 0.90	r = 0.99
KUD (Ericsson et al. [15])	20	15	2	r = 0.80 MMSE	α = 0.93	r = 0.92
CTSD (Tanaka et al. [20])	9	13		ρ = 0.94 MMSE ρ = 0.94 SCIRS	α = 0.90	ICC = 0.97	ICC = 0.96

BS, Blessed Scale; CS, Crichton Scale; ADL, Activities of Daily Living; GCS, Glasgow Coma Scale; DRS, Dementia Rating Scale; IADL, Instrumental Activities of Daily Living; CTSD, Cognitive Test for Severe Dementia; KUD, Clinical Evaluation of Moderate-to-Severe Dementia; SCIRS, Severe Cognitive Impairment Rating Scale; SMMSE, Severe Mini-Mental State Examination; BPMSE, Baylord Profound Mental State Examination; SCIP, Severe Cognitive Impairment Profile; SIRS, Severe Impairment Rating Scale; BANS-S, Bedford Alzheimer Nursing Severity Scale; TSI, Test for Severe Impairment; SIB-S, Severe Impairment Battery-short version; SIB, Severe Impairment Battery; M-OSPD, Modified Ordinal Scales of Psychological Development; HDS, Hierarchic Dementia Scale; EFA, exploratory factor analysis; MSSE, Mini-Mental State Examination; CDR, Clinical Dementia Rating; FAST, Functional Assessment Staging.

Materials and Methods

Instruments

The Residual Cognition Assessment

The Residual Cognition Assessment (RCA) has been developed by a panel of 4 experts, neuropsychologists and geriatricians, operating in Fondazione Carisma's Alzheimer Nursing Home. Starting from a bibliographic review, 14 fast screening cognitive tests for advanced decline have been identified (cf. Introduction). All their items have been extracted and readjusted on the SIB's scoring model [23], which has already been widely validated [31], to stimulate 3 hierarchical response levels (0, −1, and −2). Standardizing items' scoring has been considered useful in order to simplify the RCA's usage and comparison, and to homogenize each domain's relevance. The use of negative scores has a dual function. On the one hand, it avoids confusion with other tests widely used in the same clinical settings; on the other hand, it prevents miscalculation because the RCA provides some conditional administration steps too.

Later, each specialist has been asked to modify these items or create new ones, according to the same scoring criteria, in order to measure cognitive domains in advanced dementia through simplest residual verbal and nonverbal person reactions [17, 19]. Experts have also collected suggestions from other colleagues (physicians, nurses, psychologists, and educators).

Once a single list of potential items has been developed, a Delphi panel [32] has occurred. In the first round, experts have indicated a preferred items order (A1), favoring a minor hypothetical floor effect, according to their observations in ecological settings and to the available publication data.

In the next round, experts have separately expressed another preference order (A2), according to a new criterion that considers items with simple answer and social-functional utility at the same time, as already suggested by Saxton et al. [17]. A summative list (B), adding up A1 and A2, favors items with both simple and functionally useful answers.

In the final selection, list B has been shared among the panel to progressively identify 1 cognitive domain for each item, starting from the first one and discarding from time to time items belonging to the same domain, up to the fifteenth different domains. Indeed, by considering the desirable objectives to test as many cognitive domains as possible [20], the administration time [20, 23] was reduced and a 0–30 scoring range [18]was maintained; a single item for each domain has been established as the best compromise, with a maximum of 15 items. Searching for a fast and smooth assessment [23], items with shared or linked tasks have been preferred among top list B positions, arranged in an ecological order to reflect an everyday interview. Finally, 2 items have been subordinated to the previous one, in order to avoid any possible discomfort to frail patients and to speed up the assessment.

The prototype version of the RCA has been administered to a sample of 20 subjects in the presence of the whole panel, to test items' eligibility and feasibility. Minor revisions have been agreed before reaching the final version.

RCA − Item Composition

The test consists of 15 items. Each performance can be executed correctly (0 pt), partially (−1 pt), or wrongly (−2 pt), according to the specific criteria for each item, not interpretable, written below the relative assignment, for a total test range from point 0 (severe dementia) to −30 (late dementia). Further indications are provided in the RCA protocol (online suppl. Material; for all online suppl. material, see www.karger.com/doi/10.1159/000520322).

Item 1: Phasic alertness. The test begins by greeting the person and taking care to approach them from the side in order to avoid a casual eye contact. It is a typical task in different cognitive batteries (SIB, M-OSPD, BANS-S, SIRS, SCIP, KUD, and CTSD) that introduces the test in an ecological way. The ability of direct attention to the task is necessary for administering all the items after the third that require an active patient's response. For this reason, a negative response (−2 pt) leads to atonic alertness investigation with item 2, which requires fewer resources [33].

Item 2: Tonic alertness. Only if the previous task is failed (−2 pt), the person undergoes an irritating stimulus as the pressure on a fingernail. This item can be found also in the Disability Rating Scale [34]. It consists of a minimum generalized activation, different from sleep or coma, even if not oriented to the task, also referred to in literature as sustained attention or vigilance [33], a residual activity in the most advanced or terminal dementia. Further cognitive surveys appear to be impractical and needless in the absence of tonic alertness. Thus, if the item is scored −2, the test ends here and each of the remaining items is automatically scored −2 pt. By evaluating phasic alertness as first (item 1), any discomfort related to item 2 can be avoided when not necessary. If item 1 response is positive (0) or partial (−1), tonic alertness is assumed to be preserved. So, in this case, item 2 is skipped and scored 0 pt; therefore, the assessment proceeds with item 3 and then with the next.

Item 3: Personal orientation. Another typical cognitive request is asking for the person's name (SIB, SCIRS, SMMSE, SIRS, SCIP, BPMSE, KUD, and CTSD). The aim is to inquire basic aspects of autobiographical memory, usually preserved in advanced dementia [21, 22]. In addition, at the beginning, a personal question prompts interest and curiosity by encouraging a less anxious relation, the ideal setting for the next requests. Finally, the ability to communicate their name is relevant for social interactions with unknown people.

Item 4: Verbal production. Although a valid response to the previous item is itself a verbal act, sometimes also a simpler destructured verbal voluntary answer is possible, even in the most compromised patient who do not pronounce their name. Vocalization is a useful ability for drawing attention. By proposing a vocal stimulus, on the basis of the TSI model, a verbal production can be induced by repetition of a singson; an automatism separated from the content comprehension.

Item 5: Comprehension. Another crucial ability to everyday life is understanding others' requests. Asking the persons to open their mouth is considered the simplest command also in the HDS, and here it is reproduced both through verbal code and imitation. The assignment is typical of a classic medical examination setting and functionally useful for the person feeding.

Item 6: Selective attention. The first item already assesses phasic alertness. The next level up is to be able to identify, track, and maintain the focus toward a moving stimulus like a hand that moves in the field of view. It is another core skill for an active environmental interaction. Visual tracking tasks are also included in the M-OSPD.

Item 7: Inhibitory control. The ability to inhibit automatic inappropriate responses can be observed on a basic level in the oral tactile reflex (cf. HDS), or, in a more advanced frontal degeneration, in the sucking reflex [35].

Item 8: Executive initiative. A trivial behavior of environmental interaction in the face of a confusing stimulus can be a sign of a residual frontal ability in contrast to apathy, like managing a sheet placed on a patient's hand without apparent reason. A similar task of object interaction-inspection can be found in the M-OSPD and HDS.

Item 9: Reading. Although reading skill declines as all the other cognitive functions in decay progression, it seems to remain more preserved than others like episodic memory [36].

Item 10: Ideational praxis. Ideal praxis handling of a common and simple object can be evaluated by offering to the person a pen to sign, an item already present in the TSI and here simplified.

Item 11: Writing. In line with the previous item, graphic execution of signature is assessed, an item already present in the TSI, SIRS, SMMSE, CTSD, and BPMSE, here simplified. The ability to sign, sometimes residual in advanced dementia [27], has a high functional utility for personal advanced activities though not always associated with full informed consent in advanced dementia: it could be just considered as an automatism without legal validity. Therefore, caution in the patient's real decisional ability assessment is recommended.

Item 12: Short-term memory. Memory is often already compromised from the early stages of dementia [18]. Its assessment in advanced decay has to be reduced to essential aspects as Piaget's object permanence (already evaluated in the M-OSPD). It can be detected by a recognition task like a pen cap hidden in the examiner's hand, an item already present in the TSI and here further simplified.

Item 13: Positive affective competence. Processing an affective stimulus in an appropriate way is an important social skill in significant relationships. The cognitive coding of an affective stimulus is a fairly preserved ability in advanced dementia, whose assessment should be based on mimic and nonverbal reactions [37]. The task proposes a stimulus with a clear positive value, the examiner's smiling face, reinforced by a nonverbal cue as a caress.

Item 14: Negative affective reactivity. In the most advanced dementia, where positive reactions cannot be observed, emotive reactions of fear or anger seem to be almost possibile [38]. Considering the negative value (and the discomfort that it can cause), a scary stimulus, as an unexpected hand gesture in the face of the person, is proposed only where positive elicitation of previous item 13 has been unsuccessful (−2 pt). Differently, item 14 is omitted and scored as 0 pt.

Item 15: Social automatism. Performing some automatic interpersonal gestures has a useful socio-cultural value. Shaking hands is an item already present in the TSI and the HDS. Placed at the end of the test, it allows the examiner to close the assessment in an ecological way. Except item 3 and partially item 4, all other items are designed to obtain possible correct responses in patients with severe comprehension and verbal production impairment. Indeed, verbal assignments, even if present, underline gestural clues used to elicit in the person well-established, nonverbal, and contextual behavioral procedures (over-knowledge).

Multidimensional Assessment

The MMSE [5], which is the most common battery for fast and global screening of cognition in elderly people, has been used to select the sample. It consists of 5 sections (spatial-temporal orientation, memory, attention and calculation, recall, and language and praxia) and performance-based items, with a 0 (severe impairment) to 30 (unimpaired) global scoring.

To test the RCA's concurrent validity, the SIB-S [23] has been chosen among the abovementioned assessment tools for cognitive impairment in moderate to advanced dementia. It consists of 9 sections (social interaction, memory, orientation, language, attention, praxia, visuo-spatial abilities, constructional apraxia, and name orientation) and performance-based items, with a 0 (severe impairment) to 50 (moderate impairment) global scoring [31]. To evaluate the RCA's discriminatory properties, among the several instruments to stage advanced dementia, the Functional Assessment Staging (FAST) [39] and the Clinical Dementia Rating (CDR) scale [40, 41] have been selected in this study; both are based on observations of functional behaviors.

The multidimensional geriatric assessment (MGA) of advanced dementia patients is based on cognitive frailty construct [3] that overtakes the strictly cognitive evaluation and finds strong intercorrelations with clinical, organic, psychosocial, and functional markers. Validated biological markers involved in the study are the body mass index (BMI) [42], skin integrity assessed with the Braden Scale [43], and pain assessed with the NOPPAIN [44]. With regard to functional autonomies, residual mobility has been evaluated by the Performance Oriented Mobility Assessment (POMA) scale [45], while care dependence has been assessed with the Barthel Index [46]. Neuropsychiatric symptoms have been detected with the Neuropsychiatric Inventory (NPI) [47, 48]. The Cumulative Illness Rating Scale quantifies the global and clinical organic condition through illness severity and comorbidity indexes [49].

Previous findings have already reported significant correlations between cognitive decline and some of the abovementioned scales [50, 51]. Finally, to complete the assessment, individual participation in social activities and nonpharmacological therapies has been collected in the study. These activities have been selected for each person by the nursing home staff according to the MGA, in association with qualitative evaluation of the person's psychosocial needs.

Participants

All participants were recruited in Fondazione Carisma ONLUS nursing home in Bergamo (Lombardy, Italy), following these inclusion criteria: (1) diagnosis of probable dementia according to the DSM-5 criteria, (2) lack of serious hypovisus or bilateral upper extremity paresis, (3) detection of advanced cognitive impairment in the last 6 months (MMSE = 0). Patients with impossible or incomplete MMSE administration have been excluded. An additional group of participants with moderate-to-severe impairment (MMSE 1–10) has been included in the study, (4) therapy with antidementia drugs stable for at least 2 weeks before enrollment and unchanged until T1, and (5) general stability of clinical conditions up to T1, according to the continuous medical evaluation.

Methods

For each participant with MMSE = 0, data from the MGA scales (cf. Instruments) have been collected at the first assessment (T0), including the possible frequency of nonpharmacological/social activities. The MGA, previously carried out according to clinical routine apart from this study, has been conducted by geriatricians, nurses, educators, and psychologists. The RCA and the SIB-S have been then administered in close but distinct sessions (1–2 days) so as not to tire the person. During the RCA administration, 2 neuropsychologists have separately corrected the test in order to assess inter-rater reliability. After 2–3 weeks (T1), the RCA has been readministered to evaluate test-retest reliability. Furthermore, the RCA has been detected for a group of 40 participants with moderate-severe impairment (MMSE 1–10), sampled according to uniform distribution for each MMSE whole point between 1 and 10 (4 participants for each point), with the aim of comparing the 2 scales.

Analysis

Comparison with MMSE (MMSE 1–10)

Once the correlation (Spearman ρ) between MMSE 1–10 and respective RCA values has been assessed, the distribution of the RCA scores for the moderate-severe impairment group (MMSE 1–10) has been compared with the uniform distribution of the related MMSE scores between 1 and 10, evaluating significant differences (Kolmogorov-Smirnov test). Furthermore, for the RCA distribution, symmetry and percentage of total scores = 0 (the maximum achievable score for RCA) have been calculated in order to estimate a possible ceiling effect for subjects with MMSE >0. All subsequent analyzes have been carried out exclusively with the MMSE = 0 sample, which is reference population for the actual intended use of the RCA.

RCA Validity Parameters

Concurrent validity has been estimated with the correlation (Spearman ρ) between the RCA and cognitive (SIB-S) and functional (FAST and CDR) scales already validated for advanced dementia staging. Internal reliability has been estimated through the Cronbach alpha value. In addition, the item-total correlations (Spearman ρ) have been calculated. Test-retest reliability has been evaluated for the RCA values collected at T0 and T1, by calculating the intraclass correlation (ICC3, 1) 2-way mixed, absolute agreement, single measurement [52], both for overall value and for each single item.

Inter-rater reliability has been evaluated for the 2-rater RCA scores by calculating the ICC (ICC2, 1) 2-way random, absolute agreement, single rater [52], both for global value and for each single item. In addition, the RCA average total administration time (from initial to final greetings) has been calculated.

Comparison with the SIB-S

The distribution of the RCA scores in the sample with MMSE = 0 has been compared (Kolmogorov-Smirnov test) to the respective SIB-S. Symmetry and percentages of minimum scores have been calculated to assess the presence of floor effect.

Independent Factors Effect

The possible interfering effect of sex, age, and education on the RCA has been obtained by testing for significative differences between RCA means, grouped from time to time according to the aforementioned categorical variables.

RCA Sensitivity and Specificity

The sample has been divided between positive/negative according to the CDR scores to estimate the RCA sensitivity and specificity in staging advanced cognitive impairment, final RCA purpose. In fact, the European Medicines Agency proposed the CDR among tools of proven reliability as a reference in the global staging of dementia [53]. Therefore, the ROC and AUC have been calculated for the RCA values.

Exploratory Factor Analysis

Once basic assumptions are checked, an exploratory factor analysis (EFA) has been conducted through principal axis factory to reveal any latent traits.

RCA and Multidimensional Assessment

In order to evaluate the relationship between advanced cognitive impairment and organic, motor and affective-behavioral aspects, correlations (Spearman ρ) between the RCA values and relative scores obtained in the BMI, BRADEN, NOPPAIN, NPI, POMA, and Barthel scales have been calculated. As an exploratory attempt, the point-biserial correlations between the RCA and dichotomous participation (0: absent/1: present) in nonpharmacological therapies/social activities have been evaluated.

Results

Participants

One hundred twenty-four participants, of which forty with MMSE >0, have taken part in the study (50 males and 74 females). The mean age of the sample was 82.3 years (SD 7.40; range 59–97). Mean formal education was 8.1 years (SD 4.30; range 0–18). Fifty-three participants had an Alzheimer dementia diagnosis, 27 vascular dementia, 14 mixed dementia, 5 frontotemporal dementia, 4 Parkinson disease dementia, 3 Lewy body dementia, 3 alcoholic dementia, 2 residual psychosis dementia, 1 Huntington's disease, 1 primary progressive aphasia, and 11 unspecified dementia.

Comparison with the MMSE (MMSE 1–10)

The correlation between the MMSE 1–10 values and the respective RCA scores is significant (0.504; p < 0.001); RCA median scores grow with the MMSE scores increasing (Fig. 1). However, the distributions of the 2 tests for the moderate-severe impairment sample (MMSE 1–10) are significantly different (D = 4.243; p < 0.001). While the distribution of the MMSE scores is sampled to be uniform, there is a negative asymmetry (−1.153) in the respective RCA scores (−5, 0), with a percentage of RCA = 0 of 50%, 95% CI (33, 67) (Fig. 2).

Fig. 1.

Median RCA scores for each MMSE score between 1 and 10. RCA, Residual Cognition Assessment; MMSE, Mini-Mental State Examination.

Fig. 2.

RCA distribution scores in the moderate-severe impairment sample (MMSE 1–10). RCA, Residual Cognition Assessment; MMSE, Mini-Mental State Examination.

RCA Validity Parameters

Concurrent validity analyses show strong correlations between the RCA and the SIB-S (0.807, p < 0.01), the RCA and the CDR (−0.663; p < 0.01), as well as a moderate correlation between the RCA and the FAST (−0.435; p < 0.01) (Table 2). A high internal reliability for the RCA scores has been found (α = 0.899). The α value does not increase over 0.90 excluding any item. Item-total correlations and descriptive statistics of single items are presented in Table 3. Test-retest reliability between T0 and T1 is high, with an ICC3, 1 value for the RCA totals of 0.956 (0.932, −0.972).

Table 2.

Spearman's rho correlation coefficients between the RCA and the validated scales for staging cognitive-functional impairment

	SIB-S	CDR	FAST
RCA	0.807**	−0.663**	−0.435**

RCA, Residual Cognition Assessment; SIB-S, Severe Impairment Battery-short version; CDR, Clinical Dementia Rating; FAST, Functional Assessment Staging.

^**p < 0.01 (2 tails).

Table 3.

Mean, standard deviation, and item-total correlation for each RCA item

RCA	Mean ± SD	Item-total P
Phasic alertness	−0.42±0.7	0.180
Tonic alertness	−0.11±0.4	0.407
Personal orientation	−1.44±0.9	0.574
Verbal production	−1.42±0.8	0.639
Comprehension	−1.35±0.8	0.557
Selective attention	−1.55±0.6	0.514
Inhibitory control	−0.96±1	0.497
Executive initiative	−0.95±1	0.761
Reading	−1.68±0.7	0.457
Ideational praxia	−1.59±0.6	0.658
Writing	−1.82±0.5	0.343
Short-term memory	−1.64±0.7	0.538
Positive affective competence	−0.73±0.9	0.684
Negative affective competence	−0.31±0.7	0.611
Social automatism	−1.14±0.9	0.765

RCA, Residual Cognition Assessment.

Inter-rater reliability is high too, with an ICC2, 1 value for the RCA totals of 0.997 (0.996, −0.998). The overall RCA administration average time is 278 s (±85). The assessment has been conducted without any relevant notes for all the participants with MMSE = 0.

Comparison with the SIB-S

The distributions of the RCA and the respective SIB-S scores differ significantly (D = 6.48; p < 0.01) (Fig. 3). The RCA score distribution (−30, −3) is approximately standard (asymmetry: −0.56; kurtosis: −1.081) with the minimum score (−30) obtained in only 2% of the sample. The corresponding distribution of the SIB-S scores (0, 32) is instead compressed (asymmetry: 3.22; kurtosis: 14.12) toward the minimum value (0), which is present in 58% of the sample. Moreover, 19% of these SIB-S scores is the result of partial administrations, due to excessive attentional fatigue or patient's opposition, an event that never occurred with the RCA.

Fig. 3.

SIB-S and RCA scores distributions in the advanced impairment sample (MMSE = 0). RCA, Residual Cognition Assessment; MMSE, Mini-Mental State Examination; SIB-S, Severe Impairment Battery-short version.

Independent Factors Effect

Since the sample residuals are not normally distributed at the Kolmogorov-Smirnov test (D = 0.155; p < 0.01), the RCA means for each independent variable are compared with the Kruskal-Wallis test. There are no significant differences for the variables' age (D = 1.01; p > 0.05), sex (D = 1.177; p > 0.05), or education (D = 0.902; p > 0.05).

RCA Sensitivity and Specificity

Almost the whole sample has obtained a CDR score of 4 or 5. Only 2 participants had CDR = 3, excluded from this analysis. The sample has been then divided between “positive” to stage 5 (terminal) of more advanced global impairment and “negative,” classified in stage 4 (profound).

Analyzing the respective RCA scores, the ROC curve demonstrates acceptable discrimination between subjects with profound and terminal dementia, with an AUC of 0.92 (Fig. 4). Using the Youden index, an optimal threshold of −16 is determined for the RCA.

Fig. 4.

ROC for RCA scores based on CDR classification. CDR, Clinical Dementia Rating; RCA, Residual Cognition Assessment; ROC, receiver operating characteristic.

Exploratory Factor Analysis

The correlation matrix determinant is 0.0002. The KMO measure (0.856) reveals sampling adequacy. Bartlett's test of sphericity is significant (679.7; p < 0.001).

The EFA, conducted through the principal axis factory with an initial oblique rotation (oblimin criterion), finds a non-negligible maximum correlation between the components of r = 0.447. Since this rotation does not uniquely identify a simple structure, a promax oblique rotation is applied. The scree plot visual inspection, in line with the Mineigen criterion and the parallel analyzes, indicates the presence of 3 main factors, which explains a total variance of 64%. The components of the rotated matrix are presented in Table 4.

Table 4.

Matrix of rotated components for the RCA

Items	Factors
	1	2	3
Phasic alertness	0.937	−0.007	0.003
Tonic alertness	0.758	−0.152	−0.072
Personal orientation	−0.110	−0.098	0.956
Verbal production	0.020	−0.026	0.815
Comprehension	0.034	−0.075	0.739
Selective attention	0.089	0.508	0.022
Inhibitory control	0.396	0.119	0.097
Executive initiative	0.229	0.295	0.405
Reading	−0.099	0.620	0.033
Ideational praxis	0.011	0.724	0.069
Writing	−0.025	0.848	−0.396
Short-term memory	−0.149	0.666	0.156
Positive affective competence	0.677	0.170	0.015
Negative affective reactivity	0.920	−0.091	−0.028
Social automatism	0.086	0.649	0.212

Extraction method: Factorization of principal axis. Rotation method: Promax with Kaiser normalization. RCA, Residual Cognition Assessment.

The RCA and the Multidimensional Assessment

Significant correlations are found between the RCA and the Barthel Index (0.663; p < 0.01), the BMI (0.470; p < 0. 01), the POMA (0.638; p < 0.01), the NPI (0.627; p < 0.01), and the CIRS scales, both for the comorbidity index (−0.227; p < 0.05) and severity (−0.23; p < 0.05). The RCA correlations with the BRADEN (−0.083) and the NOPPAIN (0.032) are not significant (Table 5). Concerning nonpharmacological therapies and social activities, modest significant correlations emerge between the RCA and empathic listening (0.367; p < 0.01), individual intervention (−0.354; p < 0.01), walks (0.372; p < 0.01), light exercise (0.313; p < 0.01), pet therapy (−0.241; p < 0.05), aromatic stimulation (−0.391; p < 0.01), cognitive stimulation (0.261; p < 0.01), and multisensory stimulation (−0.325; p < 0.01) (Table 6).

Table 5.

Spearman's Rho correlation coefficients between the RCA and the functional/clinical scales of geriatric multidimensional assessment

	RCA
RCA	1**
BARTHEL	0.663**
BRADEN	−0.083
BMI	0.470**
NOPPAIN	0.032
POMA	0.638**
NPI	0.627**
CIRS − comorbidity	−0.227*
CIRS − severity	−0.230*

RCA, Residual Cognition Assessment; NPI, Neuropsychiatric Inventory; POMA, Performance Oriented Mobility Assessment; BMI, body mass index.

^**p < 0.01.

^*p < 0.05 (2 tails).

Table 6.

Pearson's r correlation coefficient between the RCA and the participation in nonpharmacological therapies/social activities

	RCA
	R	median
RCA	1
Empathic listening	0.367**	−16
Music therapy	−0.211
Garden therapy	0
Spiritual activity	0.199
Parties	0.114
Coffe and happy hour	0.114
Self-care	0.196
Maternage facilitation	−0.03
Validation group	−0.078
Individual intervention	−0.354**	−23
Expressive workshop	−0.071
Walk	0.372**	−10
Reminiscence	0
Light exercise	0.313**	−5
Pet therapy	−0.241 *	−23
Aromatic stimulation	−0.391 **	−28
Cognitive stimulation	0.261 *	−12
Cooking	0
Multisensory stimulation	−0.325**	−20
Doll therapy	0.007
Virtual train	−0.086
Use of mediators	−0.097

RCA median scores are indicated for significant correlations. RCA, Residual Cognition Assessment.

^**p < 0.01.

^*p < 0.05 (2 tails).

Discussion/Conclusion

The significant correlation between MMSE 1–10 values and respective RCAs allows to hypothesize a generic construct (“cognition”) consistently measured by both tests. However, a uniform MMSE values distribution matches an asymmetrical RCA scores distribution, with a progressive ceiling effect. In fact, 78% of the RCA = 0 (max score) of this group is located in the most performing range (MMSE 5–10). The data suggest a low RCA sensitivity to mild-moderate impairment, so its use is not recommended for people with MMSE >0, preferring traditional cognitive screening tools like the MMSE, the MOCA, or similar in such cases. In the sample with MMSE = 0, on the other hand, the RCA shows excellent psychometric properties.

The concurrent validity with the already validated cognitive (SIB-S) and global (CDR and FAST) monitoring tools for advanced impairment is high. The RCA however demonstrates greater sensitivity than the SIB-S to more advanced cognitive impairment, avoiding the floor effect found in the latter (2% vs. 58%). A limitation of this work, common to many studies with people with advanced dementia, is the impossibility of administering a large cognitive test battery to assess convergences and differences, due to the extreme frailty and limited attention span of the target population. Internal, test-retest, and inter-rater reliability are also considered high [54].

The average time of overall RCA administration (including the initial and final greeting) is contained below the 5 min threshold, in order to avoid excessive fatigue of the person. The RCA does not seem to be affected by the effect of age, sex, or education, in line with previous studies in the same target population [20, 26].

The EFA suggests the presence of 3 main factors, which seem to group 3 different levels of cognitive performance. A series of induced, passive reactions, mostly primitive, affective, or imitative reflexes (1. phasic alertness, 2. tonic alertness, 7. inhibitory control, 13. positive affective competence, and 14. negative affective reactivity), have been found at the most basic level. More complex responses are observed, at an intermediate level, beyond simple imitation, which seem to imply a minimum self-perception and active sensory-motor processing (3. personal prientation, 4. verbal production, 5. comprehension, and 8. executive initiative). At the most performing level, simple instrumental or social responses remain, linked to the classic higher cortical functions (6. selective attention, 9. reading, 10. ideational praxis, 11. writing, 12. short-term memory, and 15. social automatism).

Faced with this clustering, it could be hypothesized as a progressive hierarchical loss of cognitive efficiency from the most performing to the most basic level during dementia progression. In this case, the RCA would seem to discriminate at least 3 different major steps over the advanced impairment, one more than the two (4-profund and 5-terminal) identified by the CDR for the same population. In fact, a RCA cutoff of −16, which demonstrates excellent sensitivity and specificity in classifying between the CDR 4/5, is positioned only at half depth of the RCA scale (0, −30).

Alternatively, we could stratify the sample with the RCA factors into 3 theoretical levels, defined by the maximum partial RCA scores of each factor (10, 8, and 12). For each of these levels, we have added actual median scores of the items included; the emerging partials indicate a progressive cognitive loss according to a gradient consistent with an effective hierarchy of the 3 factors, which are therefore proposed as 3 major levels of advanced impairment: a “severe” impairment (RCA 0, −12), a “profound” impairment (RCA −13, −20), and a “late” impairment (RCA −21, −30) (Table 7).

Table 7.

Sample stratified in 3 RCA score levels according to the factors identified by EFA

Item	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	Partials	Loss, %	Cumulative, %
RCA severe (0–12)
Severe						−1			−2	−1	−2	−1			0	−7/12	−58*
Profound			0	−1	−1			0								−1.5/8	−19	−
Late	0	0					0						0	0		0/10	0	−
RCA profound (−13 to −20)
Severe						−2			−2	−2	−2	−2			−1	−11/12	−33	−92
Profound			−2	−2	−2			0								−6/8	−56*	−75
Late	0	0					0						0	0		0/10	0	0
RCA late (−21 to −30)
Severe						−2			−2	−2	−2	−2			−2	−12/12	−8	−100
Profound			−2	−2	−2			−2								−8/8	−25	−100
Late	−1	0					−2						−2	−1		−6/10	−60*	−60a

For each level, the median items scores are given and added up in partials according to the 3 factors. The percentage of cognitive performance loss is identified for each factor on each level. Cumulative loss for each factor as levels advance is calculated. RCA, Residual Cognition Assessment; EFA, exploratory factor analysis.

^*The factor with the worst loss on each level is the one consistently expected for that level.

^aAt the most advanced level, while scores on the items of intermediate factors consistently reach the floor, there is a residual performance margin for items of the factor with simpler reactions (late stage), to underline the absence of a floor effect.

While this evidence provides some support to the theories of hierarchical “involution” of dementia [14], the presence of numerous outliers among the item scores included in each level suggests a wide variability of the individual impairment trajectories of the various cognitive functions, even in the most advanced stages. Therefore, it is preferable to maintain a performance-based assessment approach, in line with the previous literature (cf. Introduction).

The RCA findings have been shown to be consistent with the other MGA tools. In particular, the worsening of advanced cognitive impairment, measured with the RCA, correlates with a loss of functional autonomy (Barthel Index), a reduction in BMI, a loss of mobility and balance (POMA), a reduction in behavioral disturbances (NPI), and an increase in comorbidity and general clinical severity (CIRS). On the other hand, we have not found a significant relationship with the risk of pressure injury (BRADEN) or the presence of pain (NOPPAIN).

Regarding the assessment of skin integrity through the BRADEN, Capon et al. [55] identified moderate cognitive impairment as a risk factor; however, no previous evidence of a specific correlation between skin integrity and progression of advanced cognitive impairment has been found, although possible. Concerning pain, some studies have shown an inverse relationship between the presence of pain and advanced cognitive impairment, but the relationship between the two remains controversial due to the objective difficulty of measuring pain in this particular population, which could lead to underestimation [56].

An exploratory analysis shows significant correlations between the RCA and some nonpharmacological therapies and social activities, typically carried out with persons in advanced dementia. For example, walking, motor awakening, and mild cognitive stimulation appear to characterize the less advanced stage of severe impairment, when higher level cognitive and functional resources remain. Sensory stimulation and empathic listening seem to be more common in an intermediate phase of profound impairment. Finally, tailor-made individual social interventions, pet therapy, and aromatic stimulation seem to better adapt to the most advanced stage of late impairment. Other widely used activities in people with dementia have a more transversal presence, independent of the cognitive level. In any case, further studies in this field are needed for evaluations of effective clinical utility.

The main limitation of this work is the lack of longitudinal monitoring of the RCA detections: initially foreseen, it was necessarily canceled because of the numerous and tough losses due to the COVID-19 pandemic. Further studies in this direction, like validations involving other contexts, are needed to better understand RCA long-term psychometric properties.

To date, the RCA represents a validated tool capable of satisfying the main requirements identified to assess people with advanced cognitive impairment. Based on simple and gestural assignments aimed at evoking mostly nonverbal automatic responses, even in the bedridden person, it adopts a performance-based scoring, requires only a pen, evaluates different cognitive domains during a short fluid interaction, and is as ecological as possible. Administration and scoring are simple and objective; the attached protocol indicates both the assignments, which can be repeated several times, and the instructions for scoring.

Detecting, studying, and monitoring advanced cognitive impairment in dementia have great clinical utility. It is essential for understanding and satisfying the needs of the patients, communicating effectively with caregivers, and adopting timely and personalized therapeutic strategies.

Statement of Ethics

None of the participants to the study have previously expressed a preference to an eventual participation in scientific research trials. Because all the involved people showed moderate to advanced dementia, with high vulnerability in the ability to understand, reason, and take decisions, double-informed consent has been given to the participants and relevant legal representatives [57]. For each participant in the study, due to common inability to sign, informed consent has been discussed with the person verbally, adjusting to individual residual understanding. At the same time, written consent has been requested from the respective legal representative, according to Alzheimer Europe's ethical guidelines. ASST Papa Giovanni XXIII's Ethics Committee approved the informed consent and the experimental procedure in November 2019 (Protocol ID: CaRiSMA002).

Conflict of Interest Statement

The authors have no conflicts of interest to declare.

Funding Sources

The authors have no dedicated funding sources to declare.

Author Contributions

A. Soli, G. Belotti, and F. Lazzarini have contributed to the conception and design of the present work. A. Soli, G. Savoldelli, S. Zonca, and A. Rota have contributed to the acquisition of data, while A. Soli and A. Rota have contributed to the analysis and interpretation of data. All the authors have revised the work for intellectual content and approved the final version to be published.

Data Availability Statement

The data that support the findings of this study are available for consultation within the limits of European GDPR 679/16 at Carisma Foundation archives, writing to info@fondazionecarisma.it. Further inquiries can be directed to the corresponding author.

Supplementary Material

Supplementary data