Optimizing cCOG, a Web‐based tool, to detect dementia with Lewy Bodies

Abstract

Introduction

Distinguishing dementia with Lewy bodies (DLB) from Alzheimer's disease (AD) is challenging due to overlapping presentations. We adapted a Web‐based test tool, cCOG, by adding a visuospatial task and a brief clinical survey and assessed its ability to differentiate between DLB and AD.

Methods

We included 110 patients (n = 30 DLB, n = 32 AD dementia, and n = 48 controls with subjective cognitive decline (SCD)). Full cCOG comprises six cognitive subtasks and a survey addressing self‐reported DLB core and autonomic features. First, we compared cCOG cognitive tasks to traditional neuropsychological tasks for all diagnostic groups and clinical questions to validated assessments of clinical features in DLB only. Then, we studied the performance of cCOG cognitive tasks and clinical questions, separately and combined, in differentiating diagnostic groups.

Results

cCOG cognitive tasks and clinical survey had moderate to strong correlations to standard neuropsychological testing (.61≤ r ^s ≤ .77) and to validated assessments of clinical features (.41≤ r ^s ≤ .65), except for fluctuations and REM‐sleep behavior disorder (RBD) (r ^s = .32 and r ^s = .10). Full cCOG, including both cognitive tasks and brief survey had a diagnostic accuracy (acc) of 0.82 [95% CI 0.73–0.89], with good discrimination of DLB versus AD (acc 0.87 [0.76–0.95]) and DLB versus controls (acc 0.94 [0.86–0.98]).

Conclusion

We illustrated that cCOG aids in distinguishing DLB and AD patients by using remote assessment of cognition and clinical features. Our findings pave the way to a funneled, harmonized diagnostic process among memory clinics and, eventually, a more timely and accurate diagnosis of DLB and AD.

1. INTRODUCTION

Standardized cognitive and clinical assessment is essential for the differential diagnosis of dementia. While pen and paper neuropsychological testing is still the norm, these traditional tests are time‐consuming and require trained neuropsychologists to assess and interpret the results. Web‐based testing provides several advantages, such as enhanced precision of measurements and standardizing measurements between hospitals. ¹ In addition, they create the possibility of self‐administration at home, a feature that, due to the recent covid‐19 pandemic, has been shown of great value and, as a result, led to accelerated development of such tools. ² Our recent study showed the diagnostic value of cCOG, a Web‐based cognitive screening tool. ³ cCOG had comparable accuracy with traditional neuropsychological testing and could detect mild cognitive impairment (MCI) and dementia from cognitively healthy controls. However, the ability to distinguish dementia subtypes was not addressed in this study and remained to be elucidated.

Dementia with Lewy bodies (DLB) is one of the main neurodegenerative diseases causing dementia, with an estimated prevalence of up to 24 percent of all dementia cases. ⁴ DLB is characterized by a complex presentation of cognitive and clinical features and can be diagnosed when there is cognitive impairment accompanied by two or more clinical core features, specifically visual hallucinations, parkinsonism, rapid eye movement sleep behavior disorder (RBD), and fluctuating cognition. ⁵ In addition to core features, DLB is associated with several supportive clinical features, such as autonomic disturbances causing constipation, urinary incontinence, and orthostatic hypotension. Standard cognitive screeners usually focus on detecting memory impairment and lack sensitivity to detect DLB‐specific impairment, such as impaired visuospatial functioning. ⁶ Still, even with elaborate neuropsychological testing across multiple cognitive domains, the sensitivity to discriminate DLB from AD remains limited. ⁷ This is likely because cognitive tests do not consider non‐cognitive clinical symptomatology, while this plays a critical role in DLB. ⁶ , ⁸ As a result, diagnosing DLB and discriminating DLB from Alzheimer's disease (AD) remains challenging. ⁶

RESEARCH IN CONTEXT

1. Systematic review: Web‐based cognitive testing holds the potential to aid in the differential diagnosis of dementia. An accurate diagnosis of the underlying neurodegenerative disorder is essential for adequate disease management. It is of particular importance to distinguish dementia with Lewy bodies (DLB) from Alzheimer's disease (AD). The ability to distinguish dementia subtypes with Web‐based testing is not yet elucidated.

2. Interpretation: This study indicates that adding a survey addressing non‐cognitive symptomatology and a visuospatial test to a set of Web‐based cognitive tests (cCOG) led to the improvement of distinguishing DLB from AD and controls in an apparently simple machine learning approach.

3. Future direction: This study paves the way to a funneled, harmonized diagnostic process among memory clinics and, eventually, a more timely and accurate diagnosis of DLB and AD.

We aimed to improve the differential diagnostic value of cCOG for DLB versus AD and controls. To address the challenges mentioned above, we modified the initial cCOG protocol by including a visuospatial test. Additionally, to capture the non‐cognitive symptoms of DLB, we added a survey to assess clinical DLB‐specific symptoms. This modified cCOG protocol was tested in patients with DLB, AD, and controls. We compared cCOG results to standard neuropsychological testing and validated clinical questionnaires and subsequently determined the performance of the full cCOG te, combining cognitive tasks and clinical questions using an artificial intelligence‐based approach in discriminating diagnostic groups.

2. METHODS

2.1. Participants

We included 110 participants, 62 patients on the dementia spectrum with a diagnosis of DLB (n = 30) or AD (n = 32), and 48 controls with subjective cognitive decline (SCD) from the Amsterdam Dementia Cohort, ⁹ and the embedded DEvELOP, ¹⁰ and SCIENCe studies. ¹¹ A diagnosis of AD or DLB was made according to the criteria for the underlying neurodegenerative disorder. ⁵ , ¹² All diagnoses were made in a multidisciplinary consensus meeting. In both groups, we included patients on the dementia spectrum, with either mild cognitive impairment (MCI) ¹³ , ¹⁴ or dementia. Participants were diagnosed with SCD when criteria for MCI or dementia were not met. Inclusion criteria were (1) having a standard neuropsychological assessment available of <6 months and (2) an Mini‐Mental State Examination (MMSE) score of >16. Demographics and clinical characteristics are described in Table 1. An ethical review board approved the study protocols of the ADC and the embedded DEvELOP and SCIENCe studies. All participants provided written informed consent for their clinical data to be used for research purposes.

TABLE 1.

Baseline characteristics

	Total N = 110	DLB N = 30	AD N = 32	Controls N = 48	p‐value
Age, years	66 ± 7	70 ± 6	66 ± 6	65 ± 8	0.021a
Sex, female (%)	49 (45%)	8 (27%)	14 (44%)	27 (56%)	0.038
Education, years	13 ± 3	11 ± 3	13 ± 3	13 ± 3	0.015a,c
Time since diagnosis, years	3 ± 3	2 ± 2	2 ± 2	4 ± 4	0.007a,b
CDR ≥1 (%)	38 (35%)	22 (73%)	16 (50%)	0 (0%)	<0.001a,b
CSF p‐tau/Aβ42ratio abnormal	45 (52%)	6 (40%)	32 (100%)	7 (18%)	<0.001b,c
Traditional neuropsychological tests
MMSE	26 ± 4	23 ± 4	24 ± 4	29 ± 2	<0.001a,b
VAT‐A	9 ± 4	8 ± 3	6 ± 3	12 ± 1	<0.001a,b,c
AVLT, immediate recall	34 ± 14	25 ± 8	24 ± 8	46 ± 11	<0.001a,b
AVLT, delayed recall	6 ± 5	4 ± 5	2 ± 2	9 ± 3	<0.001a,b,c
TMT‐A (s)	61 ± 54	104 ± 74	64 ± 52	36 ± 14	<0.001a,b,c
TMT‐B/TMT‐A ratio	3 ± 1	5 ± 2	3 ± 1	2 ± 1	<0.001a,b,c
Stroop‐I (naming, s)	54 ± 24	76 ± 31	56 ± 24	43 ± 12	<0.001a,b,c
Stroop‐II (color, s)	74 ± 30	102 ± 43	78 ± 20	58 ± 15	<0.001 a,b,c
Stroop‐III (color/naming, se)	131 ± 65	168 ± 74	167 ± 73	94 ± 26	<0.001a,b
Fragmented letters	17 ± 4	15 ± 4	19 ± 2	20 ± 1	<0.001a,c

Note Data represent mean ± SD, n (%) or median [interquartile range].

^a‐c Post hoc pairwise comparisons indicate group differences after false discovery rate correction: ^a p<0.05 DLB‐controls, ^b p<0.05 AD‐controls, ^c p <0.05 DLB‐AD.

Abbreviations: AD, Alzheimer's disease; AVLT, Auditory Verbal Learning Test; CSF, cerebrospinal fluid; DLB, dementia with Lewy bodies; MMSE, Mini‐Mental State Examination; TMT, Trail Making Test; VAT, Visual Association Test.

2.2. Clinical assessment

Participants received a standardized workup, including medical history, neurological examination, and neuropsychological assessment at their most recent visit to the memory clinic. Cerebrospinal fluid (CSF) was obtained via lumbar puncture at the first visit to the memory clinic and was available for n = 91 patients. AD patients with CSF available had a p‐tau/Aβ42 ratio indicative of AD pathology, compared to 40% of DLB patients and 18% of controls. For n = 23 DLB patients, dopamine transporter imaging (¹²³FP‐CIT(DAT)‐SPECT) was performed at the clinician's discretion to support diagnosis. In 82% (19/23), diagnosis was supported by the DAT‐SPECT scan.

2.2.1. Traditional neuropsychological assessment

Traditional neuropsychological testing consisted of MMSE ¹⁵ to assess global cognition. Memory was assessed using the Visual Association Test (VAT) ¹⁶ and the immediate and delayed recall of the Dutch auditory verbal learning test (AVLT). ¹⁷ Attention and processing speed were measured using the Trail Making Test – A (TMT‐A) and Stroop I and II. ¹⁸ , ¹⁹ Executive functioning was assessed using the ratio between TMT‐B and TMT‐A and Stroop III and Stroop II. ²⁰ To assess visuospatial functioning, we used the fragmented letter test, a subtest from the Visual Object and Space Perception Battery (VOSP). ²¹ , ²² The median time between cCOG administration and the last neuropsychological assessment was 55 days (interquartile range [IQR] 0‐155). As expected, controls scored higher on all traditional neuropsychological tasks than patients with AD or DLB. DLB patients performed worse on TMT‐A, TMT‐B/TMT‐A ratio, and fragmented letters compared to AD, while AVLT delayed recall and VAT‐A were lower for AD than DLB patients.

2.2.2. DLB‐specific questionnaires

In the DEvELOP cohort, validated questionnaires were used to assess DLB‐specific symptoms. ¹⁰ Motor problems were assessed by a trained medical doctor using the Unified Parkinson's disease Rating Scale (UPDRS) – part III (motor). ²³ We rated parkinsonism as present when the UPDRS indicated that bradykinesia was present (≥1) with additional rigidity (≥1) and/or resting tremor (≥1). Parkinsonism was present in 23(79%) DLB patients. The other core features were assessed using caregiver‐rated clinical questionnaires. The presence of hallucinations was assessed with the neuropsychiatric inventory (NPI) – hallucinations subscale and was dichotomized as being present or not (≥1). The NPI was indicative of hallucinations in 16(55%) DLB patients. ²⁴ We evaluated the presence of cognitive fluctuations with the Mayo Fluctuations Questionnaire (MFQ, cutoff ≥3). ²⁵ MFQ was indicative of fluctuations in 11(38%) patients. The presence of RBD symptoms was assessed using the Mayo Sleep Questionnaire (MSQ, ≥1). ²⁶ Caregivers reported RBD on the MSQ in 22(75%) of DLB patients. The suggestive features of autonomic dysfunction (constipation, urinary problems, and orthostatic hypotension) were assessed with the Non‐Motor Symptoms Scale (NMSS) (questions 21, 22, and 1) ²⁷ or with the SCOPA‐AUT (questions 5, 8, and 14). Constipation was present in 9(30%), urinary problems in 15(50%) and orthostatic hypotension in 15(50%) DLB patients. ²⁸

2.3. cCOG

cCOG is a Web‐based test that can be executed on all devices (e.g. personal computer (PC), laptop, tablet) with access to Internet Explorer. In total, 81% of participants completed cCOG on a PC or laptop. Patients were invited to complete cCOG at home or during a memory clinic visit. Table 2 displays cCOG cognitive tasks and clinical survey. The cognitive task protocol of cCOG entails six subtasks, of which five were in line with the previous study ³ . To enhance the ability of cCOG to detect DLB, we expanded cCOG with a visuospatial test (fragmented letters ²² , ²⁹ ). In addition, to better capture the non‐cognitive symptomatology, we added a brief online survey consisting of seven questions to assess the presence of DLB‐specific core features and suggestive autonomic symptoms. Questions were based on the DLB‐specific questionnaires mentioned above and related to all core symptoms (motor problems, sleep problems, hallucinations, fluctuations) and suggestive autonomic dysfunction (constipation, urinary incontinence, and dizziness due to standing). For comparison with validated assessments, clinical questions were dichotomized as present or not.

TABLE 2.

cCOG cognitive tasks and clinical questions

cCOG cognitive tasks	Task	Quantification
Task 1: Episodic memory test: learning task	The user is asked to remember 12 words shown one by one. Memory encoding is supported by a simultaneously presented visual image of the target word, i.e. the word “CAR” is presented with a picture of a car. After word/picture combinations have been presented, the subject is asked to type as many words as she/he can remember. The same list is shown three times, followed by the immediate recalls.	Total word count of correct words in immediate trials (maximum = 36)
Task 2: Reaction time task	Stimuli are letters shown on the screen indicating the direction (right and left) to which the user should react by pressing the arrow button as quickly as possible. In Task 2, the user should hit the arrow button on the right “→” whenever “R” is displayed. In Task 3, both “R” and “L” letters are displayed, and the user should hit the right arrow “→” for “R” and the left arrow “←” for “L”.	Standard deviation (SD) of the average reaction time in seconds, mean of delays and coefficient of variation (SD/mean) (responses <50ms and >3000ms were filtered out);
Task 3: Modified trail making test – A	The user is asked to select the numbers from 1 to 24 in the ascending order as quickly as possible. Numbers from 1 to 24 located on the squares are shown in random locations on the screen.	Time to completion (seconds) and number of completed steps (maximum = 24);
Task 4: Modified trail making test – B	The user must again click the numbers in order, however, this time each number from 1 to 24 is presented both in a circle and a square. Altogether 48 stimuli are shown on the screen and the user is asked to select numbers in ascending order but every first time a circle and every second a square in a sequence (1 inside circle, 2 inside square, 3 inside circle, etc.).	Time to completion (seconds) and number of completed steps (maximum = 24);
Task 5: Episodic memory test: recall task	The user is asked to recall and type the words from Task 1.	Total word count in delayed recall, range (maximum = 12)
Task 6: Fragmented letters	The user is shown incomplete letters and asked to type the corresponding letter	Total letter count (maximum = 20)
cCOG clinical questions	cCOG question	Quantification
Question 1: Motor problems	During 30 days, have you experienced any stiffness of the limbs or torso, slowness of movement, and/or tremors (i.e., shaking of the hands or limbs)?	No Yes
Question 2: Visual hallucinations	Has there ever been an occurrence in which you have seen something that someone with you has not? For example, seeing an object, shadow, person, animal, or other thing which was not really there?	No, never I have experienced this, but less than once a week I have experienced this at least once a week
Question 3: RBD	Has anybody told you that you act out in your dreams while you are asleep, such as waving your arms around, kicking your legs about, or shouting?	No Yes If yes: since how long?
Question 4: Fluctuations	Do you notice differences in levels of alertness and confusion during the day or from day to day?	No Yes If yes: how often? bullet‐Daily/Monthly/Weekly If yes: how long? bullet‐>1hr / 5 mins – 1hr / <5 mins / Seconds
Question 5: Constipation	Have you experienced any constipation within the last month?	Never Often Sometimes
Question 6: Urinary retention	Have you experienced problems with urinary retention within the last month?	Never Often Sometimes
Question 7: Orthostatic hypotension	Have you experienced becoming lightheaded or dizzy, or no longer able to think properly when standing up within the last month?	Never Often Sometimes

Patients were instructed to complete the cognitive tasks without help and answer the clinical questions with help from a caregiver. In total, 94 participants completed the test at home and 14 in the hospital. A total of 26 participants received help during the test. Of these participants, n = 20 reported being helped with understanding instructions (14 DLB, 6 AD), 2 with typing (1 DLB, 1 AD), and 4 with doing tasks (3 DLB, 1 AD).

Subsequently, all scores on cCOG cognitive tasks and survey outcomes (not dichotomized) were used as input for the Disease State Index classifier (DSI). ³⁰ The DSI is a supervised machine learning method that processes heterogeneous patient data to derive numeric index values between zero and one, indicating a patient's disease status. The DSI is computed by comparing measurement values of an individual patient with previously diagnosed participants with and without a disease. Due to its design of treating all variables independently, the DSI classifier can be fitted and evaluated using data with missing values. Therefore, individuals with missing data on one or more tasks were also included in the analysis. ⁷ , ³⁰ The DSI method is described in detail in the supplementary appendix. As an additional step compared to our earlier studies utilizing DSI, DSI values were converted to calibrated probabilities using Platt scaling ³¹ and extended to a multiclass scenario using one‐vs‐all comparisons. ³² For each diagnostic group (DLB, AD, controls), three DSI classifiers were computed: (1) full cCOG (cognitive tasks and clinical survey), (2) cognitive tasks only, and (3) clinical survey only. Leave‐one‐out cross‐validation was used for computing the results. Additionally, to compare full cCOG to traditional screening, we computed the same classification results using MMSE as an input.

2.4. Data analyses

Analyses were performed in SPSS version 22.0 and R version 4.0.3. Analysis of variance (ANOVA) and chi‐squared tasks were used to assess differences in demographics, neuropsychological test scores, individual and total cCOG scores. Post hoc pairwise comparisons were corrected using the false discovery rate (FDR) method for multiple comparisons. The area under the curve (AUC) for differentiating DLB from CN and DLB versus AD was calculated for each separate cCOG cognitive task. Correlations between cCOG cognitive tasks and clinical cognitive test results were computed using Spearman's correlation coefficient. The correlations are rated as: 0–0.39 weak, 0.40–0.59 moderate, 0.60–1.0 strong. ³³ Sub‐analyses were done in the DLB group to examine whether answers to the DLB‐specific questionnaires correlated to each participant's available clinical assessment data.

To study the performance of cCOG, we calculated accuracy by dividing the number of correctly classified patients by all. First, we studied the overall accuracies of (1) full cCOG, including cognitive tasks and clinical survey, (2) cCOG cognitive tasks only, and (3) cCOG clinical survey only. The accuracy of MMSE was calculated the same way as a comparison. Then, we calculated accuracy, sensitivity, and specificity to study the performance of the classifiers in diagnosing DLB, AD, and controls and differentiating DLB from AD.

3. RESULTS

3.1. cCOG results

Table 3 shows cCOG cognitive tasks and clinical survey results for all diagnostic groups. The mean completion time of cCOG was 24 ± 7 min (range 12–58 min) and was longer for both dementia groups (DLB: 29 ± 8, AD: 27 ± 5) compared to controls (18 ± 4) (p < 0.001). Completion rates were the lowest for the modified TMT‐B (77%). For TMT‐A and TMT‐B tasks, DLB patients had lower completion rates than AD and controls. DLB patients scored lower on modified TMT‐A, TMT‐B, and fragmented letters than AD and controls. Both dementia groups had lower scores on Word list immediate and delayed recall and the reaction time task compared to controls, but there were no differences between DLB and AD.

TABLE 3.

Results on cCOG cognitive tasks and clinical survey for all diagnostic groups

	n	Total N = 110	DLB N = 30	AD N = 32	Controls N = 48	P‐values
cCOG cognitive tasks
Word list ‐ immediate recall	110	17 ± 9	13 ± 6	12 ± 7	23 ± 8	<0.001a,b
Reaction time task (ms)	87	882 ± 327	1099 ± 338	991 ± 355	696 ± 166	<0.001a,b
Modified TMT‐A (s)	101*	78 ± 62	134 ± 75	82 ± 62	48 ± 26	<0.001a,b,c
Modified TMT‐B (s)	85*	237 ± 161	382 ± 251	292 ± 113	161 ± 89	<0.001a,b,c
Word list – delayed recall	110	6 ± 4	4 ± 3	3 ± 3	8 ± 3	<0.001a,b
Modified fragmented letters	103	17 ± 4	14 ± 5	18 ± 1	19 ± 2	<0.001a,c
cCOG clinical survey
Motor problems, present (%)	110	37 (34%)	24 (80%)	2 (6%)	11 (23%)	<0.001a,c
Hallucinations, present (%)	110	25 (23%)	19 (63%)	4 (12%)	2 (4%)	<0.001a,c
Fluctuations, present (%)	110	48 (44%)	22 (73%)	13 (41%)	13 (27%)	<0.001a,c
RBD, present (%)	110	29 (26%)	20 (67%)	4 (12%)	5 (10%)	<0.001a,c
Dizziness, present (%)	110	68 (62%)	21 (70%)	14 (44%)	33 (69%)	0.044
Urinary problems, present (%)	110	50 (45%)	20 (67%)	9 (28%)	21 (44%)	0.009c
Constipation, present (%)	110	34 (31%)	15 (50%)	6 (19%)	13 (27%)	0.022
Completion time of full cCOG test (min)	110	24.0 ± 7.3	28.8 ± 8.0	27.3 ± 5.4	18.7 ± 3.8	<0.001a,b

Note Note Data represent mean ± SD or n(%).

^a‐c Post hoc pairwise comparisons indicate group differences after false discovery rate correction: ^a p<0.05 DLB‐controls, ^b p<0.05 AD‐controls, ^c p<0.05 DLB‐AD.

*DLB had lower completion rates compared to AD and controls.

Abbreviations: AD, Alzheimer's disease; DLB, dementia with Lewy bodies; RBD, rapid eye movement sleep behavior disorder; TMT, Trail Making Test.

As expected, DLB patients reported more core clinical features in cCOG clinical survey (hallucinations, motor problems, fluctuations, RBD) than AD and controls (Table 3). The suggestive clinical features of constipation and dizziness did not differ between groups with post hoc comparisons. DLB patients reported urinary problems more often than AD patients but did not differ from controls.

3.2. Correlation of cCOG with traditional neuropsychological testing and clinical assessment

The highest correlations were found between cCOG memory tasks (tasks 1 &5) and VAT/AVLT learning MMSE (r ^s = 0.61‐0.75) and between cCOG modified TMT‐A (task 3) and TMT‐A, Stroop I‐II, and TMT‐B (r ^s = 0.64‐0.71). cCOG modified TMT‐B (task 4), and TMT‐B strongly correlated (r ^s = 0.65). cCOG fragmented letters (task 6) had a moderate correlation with VOSP fragmented letters (r ^s = 0.58) (Figure S1).

For DLB patients only, we calculated the correlation between cCOG clinical survey and its equivalents as assessed by validated questionnaires (Table S1). The cCOG constipation question strongly correlated with NMSS and SCOPA‐AUT (r ^s = .65). Moderate correlations were found for motor problems (r ^s = .58 with UPDRS), hallucinations (r ^s = .58 with NPI‐hallucinations), urinary problems (r ^s = .41 with NMSS/SCOPA‐AUT), and dizziness (r ^s = .50 with NMSS/SCOPA‐AUT). The correlations between cCOG fluctuations and RBD questions and their validated equivalent were weak (r ^s = .32 and r ^s = .10, respectively). Overall, the self‐reported questions had higher sensitivity than specificity, indicating a higher tendency to self‐assess symptoms positively.

3.3. Classification performance

Figure 1 shows, for each diagnostic group (DLB, AD, CN), the probability that the diagnosis computed by the DSI‐classifier is the actual diagnosis. The figure shows that a patient with a DSI‐classifier of DLB has a mean probability of 0.70 ± 0.29 for having DLB, 0 20 ± 0 22 for AD, and 0 07 ± 0 16 for CN. The actual predicted diagnosis was defined as the diagnosis with the highest probability value, leading to the corresponding confusion matrix displayed in Table 4.

FIGURE 1.

Classification performance of full cCOG (probabilities) for each diagnostic group. Distributions of cCOG DSI probability scores for different diagnostic groups (true diagnosis on x‐axis), based on data from full cCOG (cognitive tasks and clinical survey). The predicted diagnosis was determined based on the highest DSI probability score per patient.

TABLE 4.

Confusion matrix of cCOG cognitive tasks and clinical survey for DLB, AD, and controls

	Predicted diagnosis
	DLB	AD	Controls
Reference diagnosis
DLB	25	3	2
AD	4	23	5
Controls	2	4	42

Note In the confusion matrix each row represents the clinical diagnosis and each column the diagnosis suggested by the classifier; the cells show the number of patients in each category.

Abbreviations: AD, Alzheimer's disease; DLB, dementia with Lewy bodies.

Table 5 shows an overview of the classification performance of cCOG (full/cognitive tests only/survey only) and MMSE for comparison for the different diagnostic groups (DLB vs. CN, DLB vs. AD, and AD vs. CN). The overall accuracy of full cCOG (all correct diagnoses/total diagnoses) was 0.82[0.73‐0.89]. For cCOG cognitive tasks, the overall accuracy was 0.70[0.61–0.78]. This accuracy was comparable to the overall accuracy of the MMSE (0.69[0.60–0.78]). When evaluating cognitive tasks individually, cCOG fragmented letters had the highest AUC (0.80[95% CI 0.68–0.92]) in differentiating DLB from AD. Both cCOG TMTs had the highest accuracy for detecting DLB from controls (TMT‐A: 0.89[0.83–0.95], 0.81[0.72‐0.92]). The immediate recall and reaction time task had the lowest values for detecting DLB versus AD (Table S2). For the clinical survey, overall accuracy was 0.60[0.50–0.69], with an accuracy of 0.87[0.74‐0.95] in differentiating DLB from AD. The combination of cCOG clinical survey and cognitive tasks led to an accuracy of 0.87 in distinguishing DLB from AD (85% sensitivity, 89% specificity).

TABLE 5.

Classification performance for all diagnostic groups using different subsets of cCOG

	DLB vs. controls	DLB vs. AD	AD vs. controls
cCOG full (cognitive tasks + clinical survey)
Accuracy [95% CI]	0.94 [0.86‐0.98]	0.87 [0.76‐0.95]	0.88 [0.78‐0.94]
Sensitivity	0.95 [0.85, 0.99]	0.85 [0.66‐0.96]	0.91 [0.79‐0.98]
Specificity	0.93 [0.76, 0.99]	0.89 [0.72‐0.98]	0.82 [0.63‐0.94]
cCOG cognitive tasks
Accuracy [95% CI]	0.92 [0.83‐0.97]	0.68 [0.54‐0.80]	0.83 [0.73‐0.92]
Sensitivity	0.98 [0.87‐1.00]	0.59 [0.39‐0.78]	0.87 [0.74‐0.95]
Specificity	0.83 [0.63, 0.95]	0.77 [0.56‐0.91]	0.76 [0.53‐0.92]
cCOG clinical survey
Accuracy [95% CI]	0.82 [0.70‐0.90]	0.87 [0.74‐0.95]	0.61 [0.49‐0.73]
Sensitivity	0.81 [0.64‐0.93]	0.81 [0.58‐0.95]	0.62 [0.46‐0.76]
Specificity	0.82 [0.63‐0.94]	0.92 [0.74‐0.99]	0.61 [0.41‐0.78]
MMSE for comparison
Accuracy [95% CI]	0.83 [0.69‐0.78]	0.66 [0.52‐0.78]	0.61 [0.49‐0.74]
Sensitivity	0.85 [0.65‐0.96]	0.69 [0.49‐0.85]	0.62 [0.46‐0.76]
Specificity	0.80 [0.56‐0.94]	0.62 [0.41‐0.81]	0.62 [0.38‐0.82]

Abbreviations: AD, Alzheimer's disease; DLB, dementia with Lewy bodies; MMSE, Mini‐Mental State Examination.

4. DISCUSSION

We demonstrated that adding a survey addressing non‐cognitive symptoms to a Web‐based cognitive tool (cCOG) improved the differential diagnosis of DLB. By using an apparently simple machine learning approach, cCOG classifies patients on the spectrum of DLB or AD versus cognitively normal with high accuracy. We found moderate‐to‐strong associations between cCOG cognitive tasks and standard neuropsychological tasks and between cCOG clinical survey and validated assessments for DLB‐specific symptoms. These findings suggest that cCOG can support the now often challenging differential diagnostic process.

Standard cognitive screeners such as the Mini‐Mental State Examination (MMSE) lack sensitivity in capturing cognitive impairment in DLB. ⁶ Correspondingly, our data shows that the MMSE cannot distinguish AD from DLB accurately. DLB‐specific symptoms are difficult to capture with cognitive tests, and none of the standard screening tools consider non‐cognitive symptomatology. Possibly, Web‐based cognitive tasks can aid. However, the ability of such tools to discriminate between dementia subtypes remained to be elucidated, and non‐cognitive symptomatology is not yet captured. ¹ , ³ We addressed this issue with a simple and straightforward solution by not only adding a visuospatial cognitive test to the cognitive test protocol but also a brief clinical survey to assess DLB‐specific symptoms by self‐report. We subsequently showed that combining cognitive test results with survey results using a machine learning approach provides a simple and effective optimization of the cCOG protocol.

When focusing on the specific test elements, most cCOG cognitive tasks strongly correlated to standard neuropsychological tasks, following the findings of our previous study. ³ The test profile of DLB patients was mainly in line with the known cognitive profile of DLB patients, with lower performance on tasks addressing attention and executive functions (TMT‐A and TMT‐B) compared to AD. ³⁴ Furthermore, we added the fragmented letter test to assess visuospatial functioning. We found that this test could differentiate between groups with good accuracy (AUC 0.80[0.68‐0.92]. Notably, performance on cCOG memory tasks did not differ between DLB and AD patients. Generally, this finding is in line with recent literature on the ability of neuropsychological testing to differentiate DLB from AD. ⁶ , ³⁵ One explanation for this finding in our study could be that memory impairment in DLB patients was secondary to attentional impairment. It might also be due to the visual function of our memory test (reading a word with a picture) and DLB patients performing worse on visual memory than verbal memory. ³⁶ An alternative explanation could be that a substantial proportion of DLB patients had concomitant AD pathology, which has been related to more severe memory impairment. ³⁷ Altogether, our findings indicate that cCOG has reasonable construct validity in assessing cognitive impairment in DLB.

Using the survey in isolation led to an overall accuracy of 0.60 and an accuracy of 0.87 for distinguishing DLB from AD, with high sensitivity (81%) and specificity (92%). Using cognitive tests only led to an overall accuracy of 0.70. For distinguishing DLB from AD, the accuracy of cCOG cognitive tasks was 0.68, comparable to MMSE (0.66). Combining the clinical survey with the cognitive tests increased the overall accuracy (0.82[0.73‐0.89]) and the accuracy of distinguishing DLB from CN and AD from CN. Thus, the questions added value to detecting DLB patients but should not be used isolated since some questions lack specificity. Most questions corresponded well to the standard clinical assessments. Only the RBD and fluctuation questions showed weak correlations between standardized measures. A potential explanation is that RBD and fluctuations are difficult to capture with self‐report questions since patients are usually unaware of or lack insight into these symptoms. While we encouraged patients to fill in the clinical questions with a caregiver, we do not have insight into whether patients actually did. The standardized measures (MFQ and MSQ) are rated during an interview with caregivers only, who might have a more accurate view of the presence of these symptoms. Nonetheless, other symptoms corresponded with reasonable values, thereby obtaining important clinical information concisely and systematically.

Patient numbers presenting at the memory clinic are expected to rise in the near future, especially when disease‐modifying drugs will become available. ³⁸ As a result, there is a need for efficient and straightforward tools to harmonize the diagnostic approach and ensure accurate and timely diagnosis for all patients. ³⁹ As such, cCOG might be used as a screening tool to guide a funneled patient journey, for instance, prior to the first visit to the memory clinic, to inform triage and determine whether additional diagnostic testing at the memory clinic is warranted for this specific patient. Not all patients with cognitive complaints might need extensive assessment in a memory clinic, and some can be reassured prior to visiting a memory clinic. ⁴⁰ Clinicians have shown a positive attitude toward using tools for this purpose, ⁴¹ and this approach might lead to more patient‐centered care. cCOG could be further optimized by extending the clinical survey with questions that assess neuropsychiatric symptoms. This would not only improve the detection of DLB but could also be relevant for other types of dementia, such as frontotemporal dementia. In this study, we also included patients with MCI due to DLB, but numbers were limited (27% had CDR<1). With prominent non‐cognitive symptomatology present in the prodromal stages of DLB, ⁴² , ⁴³ cCOG might be particularly important. However, the sensitivity of cCOG to detect these earliest disease phases remains to be elucidated.

4.1. Strengths and limitations

The unique strength of our study is the combination of assessing cognitive functioning with non‐cognitive symptoms in a machine‐learning approach. It is of great importance that self‐report works sufficiently, and then this approach also offers opportunities for other types of dementia. Another strength of our study is our well‐defined diagnostic groups. All DLB patients were diagnosed with probable DLB according to the most recent DLB criteria. ⁵ Furthermore, several other Web‐based screening tools are available, ⁴⁴ but to our knowledge, none addresses differentiating between dementia subtypes, and none is available for diagnosing DLB. Previous studies using the DSI‐classifier showed that diagnosing DLB is challenging, even when incorporating elaborate neuropsychological tasks and ancillary investigations, such as MRI. ⁷ Web‐based tools provide the opportunity for cost‐effective testing and could aid in times of social distancing due to coronavirus disease 2019 (COVID‐19) when hospital visits have to be limited. Furthermore, digital testing provides the ability to enhance data collection, making it possible to gather more indirect data, such as measuring variations in attentional performance during the test. Also, clinicians, patients, and caregivers have shown positive attitudes regarding using such tools as long as they are complementary and do not replace current care. ⁴¹

Among the potential limitations is that our sample was relatively young and all participants were relatively highly educated. Therefore, generalizability to other patient groups was not addressed. It is conceivable that older, less‐educated patients experience more difficulties with computerized settings. Therefore, additional validation of cCOG in community settings, including less educated patients, is needed. Furthermore, in the DSI models, we did not correct age differences, a potential limitation of our study.

Some factors could have influenced test accuracy. First, most patients have conducted cCOG at home. In total, a quarter of the participants reported having received any help during the test, of whom 18 performed the test at home and were all DLB or AD patients. This might indicate that performing an online cognitive test is challenging for patients with advanced cognitive impairment. Furthermore, helping could indicate that the application has issues considering usability, and follow‐up research has to be done on user experiences (e.g., using qualitative methods) to notice what is happening when people perform this test at home. Merely four patients indicated to have been helped with doing tasks. However, we cannot exclude the possibility that more patients have received help or deviated from instructions, and we would not know whether patients would perform better or worse with help. Even though this potential limitation, we showed that full cCOG performs well in differential diagnostics.

5. CONCLUSION

Our study shows that using a simple and straightforward (machine‐learning) approach to combine sensitive cognitive tests and a brief clinical survey in a Web‐based test tool improves the differential diagnostics of DLB versus AD and controls.

CONFLICT OF INTEREST

H.R.M. performs contract research for Combinostics; all funding is paid to her institution. W.F. performs contract research for Biogen. Research programs of W.F. have been funded by ZonMW, NWO, EU‐FP7, EU‐JPND, Alzheimer Nederland, CardioVascular Onderzoek Nederland, Health∼Holland, Topsector Life Sciences & Health, stichting Dioraphte, Gieskes‐Strijbis fonds, stichting Equilibrio, Pasman stichting, stichting Alzheimer & Neuropsychiatrie Foundation, Philips, Biogen MA Inc, Novartis‐NL, Life‐MI, AVID, Roche BV, Fujifilm, Combinostics. W.F. holds the Pasman chair. W.F. is a recipient of ABOARD, which is a public‐private partnership receiving funding from ZonMW (#73305095007) and Health∼Holland, Topsector Life Sciences & Health (PPP‐allowance; #LSHM20106). W.F. has performed contract research for Biogen MA Inc, and Boehringer Ingelheim. W.F. has been an invited speaker at Boehringer Ingelheim, Biogen MA Inc, Danone, Eisai, WebMD Neurology (Medscape), Springer Healthcare. W.F. is consultant to Oxford Health Policy Forum CIC, Roche, and Biogen MA Inc. W.F. participated in advisory boards of Biogen MA Inc and Roche. All funding is paid to her institution. W.F. was associate editor of Alzheimer, Research & Therapy in 2020/2021. W.F. is associate editor at Brain. Jyrki Lötjönen reports that Combinostics owns the following IPR related to the article: 1. J. Koikkalainen and J. Lötjönen. A method for inferring the state of a system, US 7,840,510 B2. 2. J. Lötjönen, J. Koikkalainen, and J. Mattila. State Inference in a heterogeneous system, US 10,372,786 B2. Lötjönen is shareholder in Combinostics. All other authors report no conflicts of interest. Author disclosures are available in the supporting information.

Supporting information

SUPPORTING INFORMATION

Gils AM, Beek M, Unnik AAJM, et al. Optimizing cCOG, a Web‐based tool, to detect dementia with Lewy Bodies. Alzheimer's Dement. 2022;14:e12379. 10.1002/dad2.12379