Clinical Utility of the Cognitive Test for Severe Dementia: Factor Analysis, Minimal Detectable Change, and Longitudinal Changes

Abstract

Aims

This study sought to conduct additional analyses of the Cognitive Test for Severe Dementia (CTSD) using the COSMIN checklist to ensure the development of adequate outcome measures.

Methods

The following analyses were conducted: (1) factor analyses were used to evaluate construct validity; (2) the standard error of measurement (SEM) and minimal detectable change (MDC) were assessed to evaluate reliability and interpretability; and (3) longitudinal change was assessed to evaluate responsiveness.

Results

The CTSD factor analysis yielded 2 factors, each of which was dichotomized by the passage rate of each item. We calculated a SEM and MDC of 1.6 and 3.6 points, respectively. Finally, the mean CTSD score showed a significant reduction (p < 0.001) over time, and the change in score exceeded the SEM and MDC.

Conclusion

Our results indicate that the CTSD could provide a reliable outcome measure in severe dementia.

Introduction

For dementia patients, it is important to assess cognitive functions to evaluate the ability to conduct daily activities [1, 2]. Although there are many neuropsychological measures available to assess cognitive functions, patients with severe dementia have often been found to be untestable when using traditional measurements such as the Mini-Mental State Examination (MMSE) [3], the Hasegawa Dementia Scale-Revised (HDS-R) [4], or the Alzheimer Disease Assessment Scale (ADAS) [5]. These measures were designed to assess patients in the mild-to-moderate stages of dementia, and have been found to be deficient in detecting more severe cognitive decline, tending to show floor effects in patients with profound dementia [2, 6].

Herrmann et al. [6] indicated that severe dementia outcome measures differ from those for mild or moderate dementia, and therefore recommend different stage-specific goals. Furthermore, Vellas et al. [2] proposed that one purpose of evaluation of severe dementia is to record disease progression even in the profound stages [2]. To overcome these difficulties, a new cognitive test was developed to measure cognition in patients with severe and profound dementia, the Cognitive Test for Severe Dementia (CTSD). Subsequently, the clinical utility of the CTSD was evaluated, showing generally good validity and reliability [7]. However, in recent years, the methodological quality of studies aiming to develop outcome measures has been questioned [8].

The COSMIN checklist is a standardized tool for assessing measurement properties and quality of methodological studies [9, 10]. The checklist has 9 items, as follows: internal consistency, test-retest and interrater reliability, measurement error, content validity, criterion validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity, responsiveness, and interpretability. Each item assesses the quality of one measurement property, while overall the checklist provides valuable information about study design and statistical methods. The COSMIN checklist was initially designed for health-related patient-reported outcomes [9, 10]. However, it has since been applied to studies seeking to develop cognitive function tests and ADL scales to achieve better clinical utility [8].

We could not have reported based on the COSMIN check list in a previous study [7]. Therefore, in this paper, we report on additional analyses to confirm the viability of the CTSD as a more reliable outcome measurement in clinical and research settings. The analyses conducted were as follows: (1) a factor analysis was conducted to evaluate construct validity; (2) a test-retest reliability analysis was performed evaluating reliability and interpretability with the standard error of measurement (SEM) and minimal detectable change (MDC); and (3) an analysis of longitudinal changes was conducted to evaluate responsiveness.

Methods

Setting and Participants

From April 2014 to March 2016, we conducted a unicenter, observational study at three timepoints across 12 months (baseline, 6 months, and 12 months), at a 270-bed rural recuperation hospital in Hyogo prefecture, Japan. Almost all patients in this hospital have dementia of varying severities and types, with around half having severe or profound dementia.

All participants underwent a standard clinical interview, physical and neurological examinations, and a CT scan. Dementia was assessed with the Diagnostic and Statistical Manual of Mental Disorders V (DSM-V) and classified into the following subtypes: Alzheimer's (AD), vascular dementia (VaD), or Lewy body dementia. Additionally, some others were diagnosed, including normal pressure hydrocephalus and chronic subdural hematoma.

Patients with the following conditions were excluded: (1) severely impaired consciousness (e.g., coma from head injury or a kinetic mutism); (2) severe aphasia and motor function disabilities because of a stroke; (3) blindness and deafness; (4) had taken antipsychotic drugs in the last week before test administration; and (5) attending a doctor's assessment to participate would adversely influence the patient's condition.

Cognitive Function Assessments

We assessed all participants with the MMSE and CTSD over a 1-week period. Dementia severity was assessed with the Clinical Dementia Rating (CDR) [11, 12, 13] by 5 occupational therapists familiar with the participants' abilities.

The CTSD has a total possible score of 30 and comprises 13 items across 7 domains: memory, orientation, language, visuospatial, praxis, frontal function, and social interaction. Unlike the MMSE, the CTSD has not been found to show floor effects when assessing severe and profound dementia [7].

Procedures

Each participant was evaluated three times. The first evaluation (t0) was conducted in the first week after informed consent was received, and two further follow-up evaluations were taken at 6 (t1) and 12 (t2) months. The tests were implemented by raters who were not responsible for caring for the patients. Within a week, 50 randomly selected participants were assessed with the CTSD again by the same rater to calculate intraclass correlation (ICC) for test-retest reliability (SEM and MDC for the CTSD using ICC values).

Data Analysis

We conducted factor analysis, and assessed test-retest reliability, and longitudinal changes, selecting a CDR ≥3 and an MMSE < 10.

1. Factorial validity. In our previous study, we did not report on the factorial validity of the CTSD. However, as it is important to evaluate test factors when interpreting results, an exploratory factor analysis was performed using principal factor analysis, and the loadings were estimated with promax rotation to determine the CTSD factor structure. An eigenvalue > 1 and a loading factor > 0.3 [14] were considered. The analyses were conducted with SPSS version 23 (IBM, Armonk, NY, USA).

2. SEM and MDC. We previously reported ICCs for relative reliability of the CTSD, but were unable to examine the absolute reliability. Therefore, in this study, we examined absolute reliability with the SEM and MDC. The SEM was calculated with the standard deviation (SD) formula:

   $$\times \sqrt{1-ICC,}$$

   where ICC represents the test-retest reliability coefficient. An acceptable SEM score was defined as less than half the baseline SD [8]. The MDC, which is the threshold for determining clinical changes beyond measurement error, was calculated with the following formula:

   $$\text{SEM}\times 1.96\times \sqrt{2}\left[15\right].$$

   Terwee et al. [15] suggested that 50 samples were needed to examine test-retest reliability to increase reproducibility. Therefore, we examined the test-retest reliability of the ICC with 50 samples, as only 21 samples had been examined in our previous report. Subsequently, we reexamined the MDC of the CTSD ratios in participants who showed changes after 6 and 12 months.

3. Longitudinal changes in item and total scores. In our previous report, total score changes from baseline to 6 months were assessed for 36 participants [7]. However, as this was conducted with an insufficient sample size to examine longitudinal changes, we were unable to examine each CTSD item and factor 12 months later.

In this study, longitudinal changes in CTSD and MMSE scores from baseline to 6 and 12 months were measured to identify the percentage of participants whose CTSD scores improved or worsened. Changes in the CTSD total score and scores for each item and factor were examined at 6 and 12 months with the Mann-Whitney U test with the significance level set at p < 0.01. Effect sizes (ES = mean difference/SD_baseline) were calculated to compare the CTSD score changes between baseline and 6 and 12 months, where values of 0.20–0.49 were seen as small changes, values of 0.50–0.79 were seen as moderate changes, and values 0.80 were seen as large changes [16].

Ethical Considerations

A family member of each participant provided written informed consent. This study was approved by the Ethics Committee of Osaka Prefecture University (2012-OT06).

Results

Participants and Score Distribution

Table 1 presents the participant demographic characteristics (male, n = 77; female, n = 155; mean age = 84.4 ± 9.7 years). Participants were classified into the following dementia classes: AD (n = 119), VaD (n = 87), Lewy body dementia (n = 11), and other (n = 15). The distribution of participants across dementia severity categories was as follows: CDR1 = 25, CDR2 = 46, and CDR3 = 161. Almost all participants with severe dementia scored less than 10 on the MMSE, but had widely dispersed CTSD scores (0–29).

Table 1.

Demographic characteristics of participants

	CDR1	CDR2	CDR ≥3	Total
Sex (male, female)	25 (6, 19)	46 (18, 28)	161 (53, 108)	232 (77, 155)
Subtype (AD:VaD:LBD:other)	11:13:0:1	27:15:1:3	85:59:10:11	119:87:11:15
Age, years	84.95±6.02	88.53±8.62	87.23±8.12	86.70±7.56
CTSD	29.75±0.61	26.73±4.33	14.90±8.60	19.79±9.01
MMSE	19.14±3.67	12.54±3.53	4.01±3.50	7.52±6.21
HDS-R	18.22±4.66	11.74±4.79	2.96±2.91	6.28±6.23

AD, Alzheimer's disease; VaD, vascular dementia; LBD, Lewy body dementia; CTSD, Cognitive Test for Severe Dementia; MMSE, Mini-Mental State Examinations; HDS-R, Hasegawa Dementia Scale – Revised. Values for age, CTSD, MMSE, and HDS-R are expressed as mean ± SD.

CTSD Factor Analysis

The CTSD factor analysis yielded 2 factors that accounted for 52.0% of the variability (Table 2). These 2 factors in the CTSD were dichotomized through an assessment of the passage rate for each item. Factor 1 accounted for 46.6% of the total variance and included 9 items: greetings, orientation to name, date of birth, repetition, naming objects, following simple directions, clock reading, using items, and naming colors. This factor had items with a high passage rate. These CTSD items were judged as having low difficulty because they were relatively easy to answer; therefore, this factor was named the automatic response factor. Factor 2 accounted for 5.4% of the total variance and included 4 items: vegetable generation, immediate memory, square copying, and writing their name. This factor had items with a low passage rate. These CTSD items were judged as being of high difficulty because they were not easy to answer; this factor was named the attentional control factor.

Table 2.

Factor analysis of the Cognitive Test for Severe Dementia (principal factor analysis with promax rotation)

Items	Factor 1 (automatic response factor)	Factor 2 (attentional control factor)	Passage rate
Giving one's name	0.843	–0.095	55
Greeting	0.833	–0.279	84.5
Naming objects	0.746	0.153	44.9
Following simple directions	0.713	0.147	55.1
Repetition	0.709	0.149	44.9
Using items	0.566	0.286	32.2
Date of birth	0.550	0.011	22.9
Naming colors	0.521	0.163	65.3
Clock reading	0.377	0.248	44.9
Writing one's own name	–0.238	0.926	16.1
Naming vegetables	0.103	0.611	10.1
Copying a square	–0.101	0.601	24.6
Immediate memory	0.273	0.477	8.4
Explained variance, %	46.621	5.396

Figure 1 compares the scores for factor 1 and factor 2 for each dementia severity level. We observed that the automatic response factor was relevant until the profound stage, when it began to decline (MMSE < 4). However, the “attentional control factor” began to decline in the severe stage (3 <MMSE < 10). Indeed, the total CTSD score was almost the same as the total score for factor 1 in the profound stage.

Fig. 1.

Scores for the automatic response factor and the attentional control factor for each dementia severity. The automatic response began to decline from the severe to the profound stage. The attentional control factor declined as severity increased and reached nearly zero at CDR > 3. CTSD, Cognitive Test for Severe Dementia; MMSE, Mini-Mental State Examination; CDR, Clinical Dementia Rating.

SEM and MDC Assessment

Test-retest reliability was calculated to examine the SEM and MDC. We randomly selected 50 participants with severe dementia, none of whom had zero points on the CTSD, to be assessed with the CTSD twice within 2 weeks by the same examiner. As shown in Table 3, the ICC on all items and the total score was found to be significant at 0.635–0.969 (p < 0.001).

Table 3.

Interrater and test-retest reliability of the Cognitive Test for Severe Dementia (n = 50)

	Item	Test-retest reliability, ICC1,2
1	Greeting (0–3)	0.840***
2	Orientation to one's name (0–3)	0.943***
3	Date of birth (0–2)	0.876***
4	Repetition (0–3)	0.945***
5	Naming objects (0–2)	0.911***
6	Immediate memory (0–3)	0.708***
7	Clock reading (0–3)	0.742***
8	Using items (0–4)	0.918***
9	Naming color (0–1)	0.635***
10	Vegetable generation (0–3)	0.866***
11	Follow simple direction (0–2)	0.855***
12	Copy square (0–1)	0.883***
13	Writing name (0–2)	0.752***
Total		0.969***

ICC, intraclass correlation coefficient.

^***p < 0.001.

The SEM for the CTSD using the ICC was 1.6, with an acceptable SEM value defined as less than half the baseline SD [8]. Given that the SD for the CTSD was 9.17 at baseline, the SEM for the CTSD was found to be sufficiently acceptable.

The MDC was used as a benchmark for the outcome scale to determine whether the observed change was clinically important (i.e., whether or not it exceeded the measurement error). The MDC for the CTSD was 3.6. Then, change of score in CTSD greater than 4 would be interpreted as a clinically important change in each subject.

Longitudinal Changes in Item and Total Scores

Outcome measures need to show responsiveness or sensitivity to longitudinal changes [15]. Therefore, we retested 81 participants with a CDR ≥3 after 6 months, and 40 participants with a CDR ≥3 were retested after 12 months. Eighty participants at 6 months and 121 participants at 12 months had been discharged, moved, or had died.

Figure 2 shows longitudinal changes in the CTSD and MMSE at 6 and 12 months. The mean CTSD score at 6 months was 11.78 ± 8.80, representing a reduction of 1.77 points compared with baseline (13.55 ± 9.17). The MMSE score at 6 months was 3.38 ± 3.91, representing a reduction of 0.26 points from baseline (3.64 ± 3.78). The mean CTSD score decreased significantly (p < 0.001) with the change exceeding the SEM. However, this was not found for the MMSE (Table 4; Fig. 2).

Fig. 2.

Longitudinal changes in the Cognitive Test for Severe Dementia (CTSD) and Mini-Mental State Examination (MMSE). At 6 months, CTSD scores had reduced significantly (p < 0.001), whereas MMSE scores had not. At 12 months, both CTSD and MMSE scores had reduced significantly (p < 0.001 and p < 0.01).

Table 4.

Change in the Mini-Mental State Examination (MMSE) and the Cognitive Test for Severe Dementia (CTSD) scores from baseline to 6 and 12 months

	6 months (n = 81)			12 months (n = 40)
	score (T1)	score (T2)	p value (T1 vs. T2)	score (T1)	score (T3)	p value (T1 vs. T3)
MMSE Orientation
Year	0.01±0.11	0.01±0.11	1.000	0.00±0.00	0.00±0.00	1.000
Season	0.04±0.19	0.06±0.24	0.414	0.03±0.16	0.03±0.16	1.000
Date	0.00±0.00	0.00±0.00	1.000	0.00±0.00	0.00±0.00	1.000
Day	0.00±0.00	0.01±0.11	0.317	0.00±0.00	0.00±0.00	1.000
Month	0.04±0.19	0.05±0.22	0.705	0.04±0.19	0.00±0.00	1.000
State	0.00±0.00	0.00±0.00	1.000	0.00±0.00	0.08±0.27	1.000
Country	0.25±0.43	0.15±0.36	0.11*	0.023±0.42	0.03±0.16	0.034*
Town	0.01±0.11	0.01±0.11	1.00	0.00±0.00	0.00±0.00	1.000
Hospital	0.01±0.11	0.04±0.19	0.317	0.03±0.16	0.00±0.00	0.317
Floor	0.25±0.43	0.03±0.16	1.000	0.03±0.16	0.05±0.22	0.564
Registration	1.25±1.28	1.23±1.29	0.953	1.15±1.23	0.93±1.33	0.102
Calculation	0.09±0.42	0.06±0.24	0.414	0.08±0.27	0.03±0.16	0.157
Recall	0.09±0.42	0.07±0.04	0.666	0.03±0.16	0.00±0.00	0.317
Language
Naming objects	1.01±0.89	0.80±0.87	0.002**	0.97±0.87	0.63±0.78	0.005**
Sentence repetition	0.09±0.42	0.04±0.19	0.157	0.05±0.22	0.03±0.16	0.317
Following three-stage commands	0.59±0.96	0.65±0.92	0.529	0.53±0.84	0.28±0.60	0.064
Read and obey	0.15±0.36	0.15±0.36	1.000	0.03±0.16	0.08±0.27	0.564
Writing a sentence	0.00±0.00	0.00±0.00	1.000	0.00±0.00	0.00±0.00	1.000
Copy double pentagon	0.01±0.11	0.00±0.00	0.317	0.00±0.00	0.00±0.00	0.317
Total	3.64±3.78	3.38±3.91	0.311	3.21±3.31	2.12±3.03	0.003**
CTSD
Greeting	2.70±0.78	2.40±1.11	0.016*	2.67±0.83	2.05±1.26	0.001**
Orientation to one's name	1.93±1.31	1.59±1.34	0.007**	1.87±1.36	1.30±1.36	0.008**
Date of birth	0.60±0.79	0.60±0.79	0.168	0.48±0.75	0.40±0.74	0.416
Repetition	1.40±1.31	1.41±1.31	0.973	1.26±1.30	0.85±0.36	0.049*
Naming objects	1.05±0.90	0.91±0.90	0.047*	1.03±0.91	0.70±0.88	0.021*
Immediate memory	0.70±1.02	0.54±0.50	0.184	0.50±0.82	0.15±0.36	0.004**
Clock reading	0.37±0.49	0.40±0.56	0.819	0.33±0.47	0.23±0.42	0.206
Using items	1.88±1.80	0.54±0.50	0.003**	1.78±1.80	0.87±1.45	0.001**
Naming color	0.63±0.64	0.54±0.50	0.216	0.53±0.84	0.43±0.74	0.134
Vegetable generation	0.62±0.83	0.37±0.68	0.000***	0.50±0.82	0.15±0.48	0.021*
Following simple direction	1.04±0.93	0.85±0.90	0.045*	0.93±0.92	0.73±0.85	0.151
Copy square	0.24±0.45	0.17±0.38	0.251	0.15±0.36	0.10±0.30	0.157
Writing name	0.44±0.77	0.54±0.50	0.132	0.38±0.77	0.15±0.36	0.037*
Automatic response factor	11.20±7.11	9.84±7.08	0.002**	10.43±6.86	7.03±6.66	0.000***
Attentional control factor	2.33±2.36	1.73±2.17	0.002**	1.95±2.38	1.13±1.63	0.008**
Total	13.55±9.17	11.78±8.80	0.000***	12.37±8.90	8.15±7.90	0.000***

Wilcoxon signed rank test:

^***p < 0.001,

^**p < 0.01,

^*p < 0.05.

The mean score for the CTSD at 12 months was 8.15 ± 7.90, representing a reduction of 4.22 points compared with baseline (12.37 ± 8.90). The MMSE score at 12 months was 2.12 ± 3.03, representing a reduction of 1.09 points from baseline (3.21 ± 3.31). The mean score for the CTSD decreased significantly (p < 0.001), with the change exceeding the SEM and MDC. However, there was also a significant reduction in MMSE (p < 0.01) (Table 4; Fig. 2). Feeney et al. [17] suggested a SEM for the MMSE of 1.0 and for the MDC of 3.0 for people with mild to moderate dementia. In this study, the MMSE score at 12 months decreased significantly (p < 0.01) compared with baseline, but did not exceed the MDC which is shown by Feeney et al. [17].

The participants assessed at 12 months (n = 40) were divided into three groups based on MDC scores. Four participants diagnosed with VaD had improved scores, exceeding the MDC, 17 participants showed no score changes or had changes within the MDC range, and 19 participants had reduced scores that exceeded the MDC (Fig. 3). However, there were no improvements in MMSE scores when the MDC exceeded 3 points. In summary, 26 participants had no score changes or had changes within the MDC range, and 16 participants had reduced scores that exceeded the MDC.

Fig. 3.

Amount of change in Cognitive Test for Severe Dementia (CTSD) scores for each participant. Four participants diagnosed with vascular dementia had improved scores over the minimal detectable change (MDC) at 12 months.

The mean variation in CTSD scores at 6 and 12 months from baseline were 3.22 ± 3.68 (range 0–15) and 5.27 ± 4.77 (range 0–19), and the mean variations in MMSE scores were 1.17 ± 1.86 (range 0–7) and 1.48 ± 1.69 (range 0–9). Comparing these statistically, we observed that the variation in total CTSD scores was significantly larger than that in MMSE scores (Mann-Whitney U test; p < 0.001).

A small ES was observed for the CTSD at 6 months (ES = −0.42), and a moderate ES was observed at 12 months (ES = −0.68). However, the ES for the MMSE at 6 months was negligible (ES = −0.11), with only a small effect observed at 12 months (ES = −0.48).

Table 4 shows the longitudinal changes for each CTSD item and factor. The date of birth, clock reading, color naming, and copy square items did not significantly change at 12 months (Wilcoxon rank signed test; p < 0.05). However, there were significant changes in the total score for the automatic response factor and attentional control factor (Wilcoxon rank signed test; p < 0.01). Therefore, the CTSD showed greater sensitivity than the MMSE for severe and profound dementia.

Discussion

In this paper, we conducted additional factor analyses to determine the MDC and longitudinal changes in CTSD scores, with the results demonstrating that the CTSD was a more reliable outcome measurement. Here, we will discuss the clinical utility of the CTSD by comparing it with various existing cognitive function tests for severe dementia.

Validity of CTSD Factors

We found 2 CTSD factors, which we designated as the automatic response factor and attentional control factor. Therefore, the CTSD could be clearly separated into 2 factors by the passage rate for each item and was effective at assessing cognitive decline in the severe and profound dementia phases. The automatic response factor had items with a high passage rate; therefore, a reduction would be expected in the later phases of severe and profound dementia. We observed that date of birth and using an item were lost at an early phase of the profound stage. However, the greeting item tended to remain. These items did not require much attention. In contrast, the “attentional control factor” had items with a low passage rate; therefore, a reduction would be expected in the early phase of the severe stage. Registration ability began to decline in the early mild AD stage. The passage rate for immediate memory was the lowest in the CTSD, indicating that registration ability was lost in the earliest phase of the severe stage.

The SCIRS, which has 11 items, also has 2 factors. One factor has items such as recognizing names and naming colors similar to those in the automatic response factor of the CTSD. Choe et al. [18] found that this factor began to decline in the severe to profound stage. The second SCIRS factor consisted of items such as recalling the examiner's name and place orientation similar to the attentional control factors of the CTSD, with this factor also being found to decline in the mild or moderate stage. Therefore, the CTSD and SCIRS show a similar factorial constitution since each factor is determined by the difficulty level of each item.

Ferris et al. [19] reported that the Severe Impairment Battery (SIB) had 40 items organized across 5 factors, while Saxton et al. [20] reported that the SIB had 8 factors. Therefore, the number of factors in the SIB has varied in previous research. Ferris et al. [19] proposed that one factor in the SIB had similar items included in different cognitive domains. For example, the color naming item was included in the language domain, color matching was included in the visuospatial perception domain, and color discrimination was included in the memory domain, even though these items were all included within the same “visual” factor. Therefore, because the SIB has many items and the factors do not separate by passage rate, it may be difficult to determine the degree of deterioration. Saxton et al. [21] developed the SIB to validate treatment efficacy for severe dementia. However, even though the SIB can assess detailed cognitive functions, it may not be appropriate for understanding stage changes.

Our results suggest that the CTSD has merit as it can easily assess the change process from the severe to the profound stage through its assessment of how each factor changes. Therefore, the CTSD may facilitate further investigations into treatment efficacy in patients with severe and profound dementia.

Interpretability of the CTSD Based on the MDC Score

Jaeschke et al. [22] proposed a definition for a minimally clinically important difference (MCID) in which it was stated that MCID scores reflected meaningful changes arising from clinical interventions. Several methods are currently used to obtain the MCID, which has been referred to as MDC or minimally important change (MIC). The MDC is generally calculated by using a distribution-based approach based on the statistical characteristics of the sample [23], and the MIC is calculated by using anchor-based approaches that utilize an external criterion to operationalize important changes [15]. These two benchmarks for the interpretation of change scores are particularly beneficial when seeking to understand the effects of various interventions or when assessing longitudinal changes in the disease process. However, there have been few studies which examined the utility of cognitive tests for severe dementia using SEM and MDC as indices of reliability and interpretability. Therefore, we conducted this study attempting to develop a new outcome measure according to the COSMIN checklist [9].

Terwee et al. [15] recommended that to determine test-retest reliability, an ICC value of 0.70 should be the minimum standard in a minimum sample size of 50 participants. In this study, the ICC was 0.969, with a sample size of 50.

Ideally, it is necessary to examine simultaneously both of MDC and MIC to determine whether a patient has improved, declined, or remained the same. MIC is calculated based on both subjective and objective data, and MDC is calculated only based on objective data. In our study, we used only MDC since subjective data could not be derived due to the patients' severe dementia.

Feeney et al. [17] reported that the SEM for the MMSE and Montreal Cognitive Assessment were 1.0 and 1.5, and the MDC were 3.0 and 4.0 based on the result of their study. Therefore, the SEM (1.6) and the MDC (3.6) for the CTSD were similar to the values observed for existing cognitive tests of mild to moderate dementia.

Sensitivity of the CTSD to Detect Longitudinal Change

The CTSD scores were found to have changed significantly at 6 and 12 months compared with baseline, whereas the MMSE showed no changes at 6 months. This result indicated that the CTSD showed greater sensitivity to change than MMSE. Previously, Na [24] reported that the Baylor Profound Mental State Examination (BPMSE)-cog, a cognitive assessment tool for profound dementia, showed changes of around 2.4 points (SD 3.3) 12 months after the MMSE reached 0. The CTSD changed by 2.1 points (SD 1.6). Therefore, the CTSD score change after 12 months was matchable to that of the BPMSE, indicating that the CTSD has good sensitivity to interval changes for severe and profound dementia even after the MMSE reaches 0.

To the best of our knowledge, there have been few cognitive tests for severe dementia which examined longitudinal changes without SIB. To adequately assess remaining cognitive functions in patients with dementia, it is necessary to examine the change in each item. Some CTSD items changed significantly between baseline and 6 or 12 months, but almost all the MMSE items showed floor effects and did not change significantly except for the naming objects and country orientation items. As 8 of 13 item scores of the CTSD changed significantly, this seemed to indicate that the CTSD has greater sensitivity than the MMSE when assessing longitudinal change.

While the MMSE could not assess whether participants with severe dementia worsen or improve in detail, the CTSD was able to provide more detailed item-by-item assessments of the participants' cognitive functions, and identify the specific cognitive domain which is improved or deteriorated. Interestingly, all 4 participants, who showed improvements by exceeding the MDC in the CTSD, were diagnosed with VaD. While it is difficult to improve cognitive functions after AD or other degenerative diseases have progressed to the severe and profound stages, our results suggest that those diagnosed with VaD could potentially improve their cognitive functions even when they have severe dementia.

The SIB has frequently been used in clinical trials as one outcome measure [25] and to identify the transition from moderate to severe dementia [26]. The CTSD would facilitate further investigations into treatment efficacy for patients with severe and profound dementia in contrast to the SIB which was originally developed to assess changes in moderate to severe dementia.

Limitations

In the examination of interpretability according to the COSMIN checklist, the MIC score derived from subjective data is generally used. We used, however, the MDC score derived from objective data in this study since the participants had severe dementia.

Conclusion

Here, we report on additional statistical analyses conducted with the COSMIN checklist to demonstrate the high clinical utility of the CTSD. From the positive results achieved in this analysis, we plan to use the CTSD in future studies to examine treatment efficacy for severe and profound dementia.

Disclosure Statement

The authors declare no conflicts of interest.