Integrated Cognitive Assessment: Speed and Accuracy of Visual Processing as a Reliable Proxy to Cognitive Performance

Abstract

Various mental disorders are accompanied by some degree of cognitive impairment. Particularly in neurodegenerative disorders, cognitive impairment is the phenotypical hallmark of the disease. Effective, accurate and timely cognitive assessment is key to early diagnosis of this family of mental disorders. Current standard-of-care techniques for cognitive assessment are primarily paper-based, and need to be administered by a healthcare professional; they are additionally language and education-dependent and typically suffer from a learning bias. These tests are thus not ideal for large-scale pro-active cognitive screening and disease progression monitoring. We developed the Integrated Cognitive Assessment (referred to as CGN_ICA), a 5-minute computerized cognitive assessment tool based on a rapid visual categorization task, in which a series of carefully selected natural images of varied difficulty are presented to participants. Overall 448 participants, across a wide age-range with different levels of education took the CGN_ICA test. We compared participants’ CGN_ICA test results with a variety of standard pen-and-paper tests, such as Symbol Digit Modalities Test (SDMT) and Montreal Cognitive Assessment (MoCA), that are routinely used to assess cognitive performance. CGN_ICA had excellent test-retest reliability, showed convergent validity with the standard-of-care cognitive tests used here, and demonstrated to be suitable for micro-monitoring of cognitive performance.

Introduction

Brain disorders can cause deficiency in cognitive performance. In particular, in neurodegenerative disorders, cognitive impairment is the phenotypical hallmark of the disease. Neurodegenerative disorders, including Dementia and Alzheimer’s disease, continue to represent a major economic, social and healthcare burden¹. These diseases remain underdiagnosed or are diagnosed too late²; resulting in less favorable health outcomes. Current routinely used approaches to cognitive assessment, such as the Mini Mental State Examination (MMSE)³, Montreal Cognitive Assessment (MoCA)⁴, and Addenbrooke’s Cognitive Examination (ACE)⁵ are primarily paper-based, language and education-dependent and need to be administered by a healthcare professional (e.g. physician). These tests are therefore not ideal tools for wide pro-active screening of cognitive impairment, which can be crucial to earlier diagnosis.

Several studies have emphasized the importance of early diagnosis^2,6–9 and its role in driving better treatment and improvement of cognition and quality of life¹⁰. Therefore, developing new tools for effective, accurate and timely cognitive assessment is key to tackling this family of brain disorders.

Growing attention has been drawn to changes in the visual system in connection with dementia and cognitive impairment^11–16. Previous studies have linked visual function abnormalities with Alzheimer’s Disease and other types of cognitive impairment^17–19. All parts of the visual system may be affected in Alzheimer’s disease, including the optic nerve, retina, lateral geniculate nucleus (LGN) and the visual cortex¹⁹. Therefore, visual dysfunction can predict cognitive deficits in Alzheimer’s Disease^19,20. The human motor cortex^21,22, and the oculomotor^23,24 are also shown to be affected in Alzheimer’s Disease.

We therefore developed a rapid visual categorization test that measures subject’s accuracy and response reaction times, engaging both visual and motor cortices as well as oculomotor function. Categorization accuracies and reaction times are then summarized to assess participants’ cognitive performance. The proposed integrated cognitive assessment (CGN_ICA) test is designed to target cognitive domains and brain areas that are affected in the initial stages of cognitive disorders such as dementia, ideally before the onset of memory symptoms. Thus, as opposed to solely focusing on working memory, the test engages the retina, the visual cortex and the motor cortex, all of them are shown to be affected pre-dementia or in early stages of the disease^21,25–31. The CGN_ICA’s focus on speed and accuracy of processing visual information^32–35 is in line with latest evidence suggesting that simultaneous object perception deficits are related to reduced visual processing speed in amnestic mild cognitive impairment³⁶. Additionally, the proposed test is self-administered and is intrinsically independent of language and culture, thus making it ideal for large-scale pro-active cognitive screening and cognitive monitoring.

This study aims to assess CGN_ICA’s convergent validity with the routinely used standard pen-and-paper cognitive tests, its test-retest reliability, and whether the proposed test is suitable for micro-monitoring of cognitive performance.

Material and Methods

CGN_ICA test description

The CGN_ICA test is a rapid visual categorization task with backward masking^33,37,38. One hundred natural images (50 animal and 50 non-animal) with various levels of difficulty were presented to the participants. Each image was presented for 100 ms followed by a 20 millisecond inter-stimulus interval (ISI), followed by a dynamic noisy mask (for 250 ms), followed by subject’s categorization into animal vs non-animal (Fig. 1). When using iPad, the categorization was done by tapping on the left or right side of the screen; when using RasPi, subjects indicated their responses by pressing either of the two pre-assigned keys on a keyboard (‘F’ vs. ‘J’). Images were presented at the center of the screen at 7 degree visual angle. For more information about rapid visual categorization tasks refer to Mirzaei et al.³³.

Figure 1.

The CGN_ICA test pipeline. One hundred natural images (50 animal and 50 non-animal) with various levels of difficulty are presented to the participants. Each image is presented for 100 ms followed by 20 ms inter-stimulus interval (ISI), followed by a dynamic noisy mask (for 250 ms), followed by subject’s categorization into animal vs. non-animal. Few sample images are shown for demonstration purposes.

The CGN_ICA test starts with a different set of 10 test images (5 animal, 5 non-animal) to familiarize participants with the task. These images are later removed from further analysis. If participants perform above chance (>50%) on these 10 images, they will continue to the main task. If they perform at chance level (or below), the test instructions will be presented again, and a new set of 10 introductory images will follow. If they perform above chance in this second attempt, they will progress to the main task. If they perform below chance for the second time the test will be aborted. Only in experiment 2, three participants, out of 61, were aborted from the study due to this reason, thus 58 subjects remaining in experiment 2 that are shown in Table 1.

Table 1.

Summary of all the experiments.

Experiment	Number of Participants	Age mean years, SD [min, max]	Education mean years, SD [min, max]	Gender (#female)	Cognitive Tests	ICA Platform
1	212	74,10, [46, 98]	9, 6, [0, 23]	110 (51%)	MoCA, CGN_ICA	Raspberry Pi (RaPi)
2	58	62, 6, [54, 79]	14, 5, [3, 24]	33 (56%)	MoCA, ACE-R, CGN_ICA	iPad
3	166	37,10, [19, 65]	14, 3, [1, 20]	125 (75%)	SDMT, BVMT-R, CVLT-II, CGN_ICA	iPad
3′ (re-test)	44	38, 12 [18, 64]	14, 2 [8, 20]	29 (66%)	CGN_ICA, SDMT	iPad
4	12	29, 3, [20, 36]	19, 4, [15, 24]	5 (41%)	CGN_ICA	Web

For each experiment, the table shows number of participants, their demographics (age, education and gender), and the cognitive tests they have taken in each experiment. A total number of 448 volunteers took part in these experiments. Experiments 1 and 2 were to establish CGN_ICA correlation with standard-of-care cognitive assessment tools for MCI and dementia screening in older adults (i.e. MoCA and ACE-R). The third experiment covers younger individuals (19 to 65 years-old) who have taken CGN_ICA along with three other standard cognitive tests, particularly suitable for this age-range. Experiment 3′ is a test-retest: 44 volunteers who participated in the third experiment were called back after five weeks (+−15 days) to take the CGN_ICA and SDMT test for the second time. This was to assess CGN_ICA test-retest reliability (r = 0.96, p < 10⁻⁷)]). Experiment 4 is the learning-effect experiment in which 12 young university students took the CGN_ICA test every other day for two weeks to see whether CGN_ICA is free from learning bias and suitable for micro-monitoring of cognitive performance.

Scientific rationale behind the CGN_ICA test

The CGN_ICA test takes advantage of millions of years of human evolution – the human brain’s strong reaction to animal stimuli^39–42. Human observers are very good at recognising whether briefly flashed novel images contain the image of an animal, and previous work has shown that the underlying visual processing can be performed quickly^38,43. The strongest categorical division represented in the human higher level visual cortex (known as inferior temporal cortex) appears to be that between animates and inanimates. Several studies have shown this in human and non-human primates^{38–40,44,45}. Studies also show that on average it takes about 100 ms to 120 ms for humans to differentiate animate from inanimate stimuli^46–48. Following this rationale, in the CGN_ICA test, the images are presented for 100 ms followed by a short inter-stimulus-interval (ISI), then followed by a dynamic mask. Shorter periods of ISI can make the animal detection task more difficult and longer periods reduce the potential use for testing purposes as it may not allow for detecting less severe cognitive impairments. The dynamic mask is used to remove (or at least reduce) the effect of recurrent processes in the brain^49–53. This makes the task more challenging by reducing the ongoing recurrent neural activity that could boost subject’s performance. This leaves less room for the resilient brain to compensate for the subtle ongoing neurodegeneration in early stages of the disease.

Participants

As shown in Table 1, we conducted four different experiments; in total, 448 volunteers took part in this study. The study was conducted according to the Declaration of Helsinki and approved by the local ethics committee at Royan Institute. Informed consent was obtained from all participants.

Participants’ inclusion criteria were individuals above age 18, with normal or corrected-to-normal vision, without severe upper limb arthropathy or motor problems that could prevent them from completing the tests independently. For each participant, information about age, education and gender was also collected.

Stimulus set

We used a set of 100 grayscale natural images, half of them contained an animal. The images varied in their level of difficulty. In some images the head or body of the animal is clearly visible to the participants, which makes it easier to detect. In other images the animals are further away or otherwise presented in cluttered environments, making them more difficult to detect. Few sample images are shown in Fig. 1. Grayscale images were used to remove the possibility of some typical color blindness affecting participants’ results. Furthermore, color images can facilitate animal detection solely based on color, without fully processing the shape of the stimulus. This could have made the task easier and less suitable for detecting less severe cognitive dysfunctions.

To construct the mask, a white noise image was filtered at four different spatial scales, and the resulting images were thresholded to generate high contrast binary patterns. For each spatial scale, four new images were generated by rotating and mirroring the original image. This leaves us with a pool of 16 images. The noisy mask used in the CGN_ICA test was a sequence of 8 images, chosen randomly from the pool, with each of the spatial scales to appear twice in the dynamic mask.

Reference pen-and-paper cognitive tests

Montreal Cognitive Assessment (MoCA)

MoCA⁴ is a widely used screening tool for detecting cognitive impairment, typically in older adults. The MoCA test is a one-page 30-point test administered in approximately 10 minutes.

Mini-Mental State Examination (MMSE)

The MMSE³ test is a 30-point questionnaire that is used in clinical and research settings to measure cognitive impairment. It is commonly used to screen for dementia in older adults; and takes about 10 minutes to administer.

Addenbrooke’s Cognitive Examination -Revised (ACE-R)

The ACE^54,55 was originally developed at Cambridge Memory Clinic as an extension to the MMSE. ACE-R is a revised version of ACE that includes MMSE score as one its sub-scores. The ACE-R⁵ assesses five cognitive domains: attention, memory, verbal fluency, language and visuospatial abilities. On average, the test takes about 20 minutes to administer and score.

Symbol Digit Modalities Test (SDMT)

The SDMT is designed to assess speed of information processing, and takes about 5 minutes to administer⁵⁶. A series of nine symbols are presented at the top of a standard sheet of paper, each paired with a single digit. The rest of the page contains symbols with an empty box next to them, in which participants are asked to write down the digit associated with this symbol as quickly as possible. The outcome score is the number of correct matches over a time span of 90 seconds.

California Verbal Learning Test -2^nd edition (CVLT-II)

The CVLT-II test^57,58 begins with the examiner reading a list of 16 words. Participants listen to the list and then report as many of the items as they can recall. After that, the entire list is read again followed by a second attempt at recall. Altogether, there are five learning trials. The final score, which is out of 80, is the summation of all the correct recalls. As in the brief international cognitive assessment for multiple sclerosis (BICAMS) battery⁵⁹, we only used the learning trials of the CVLT-II, which takes about 10 minutes to administer.

Brief Visual Memory Test–Revised (BVMT-R)

The BVMT-R test assesses visuo-spatial memory^60,61. In this test, six abstract shapes are presented to the participant for 10 seconds. The display is removed from view and patients are asked to draw the stimuli via pencil on paper manual responses. There are three learning trials, and the primary outcome measure is the total number of points earned over the three learning trials. The test takes about 5 minutes to administer.

Experiments

We conducted four different experiments, as summarized in Table 1. The first three experiments were designed to measure the CGN_ICA correlation with a wide range of routinely used reference cognitive tests. The goal was to investigate whether the speed and accuracy of visual processing in a rapid visual categorization task is correlated with subject’s cognitive performance.

In the first and second experiments, we aimed to test CGN_ICA’s ability in assessing cognitive performance in older adults. Therefore, we used MoCA and/or ACE-R as reference cognitive tests, both of which are routinely used to screen for mild cognitive impairment (MCI) and dementia in older adults. In the first experiment, 212 volunteers participated; the CGN_ICA test was delivered via a Raspberry Pi (RaPi) platform, which is a small single-board computer, attached to a keyboard and a LCD monitor; and MoCA was used as the reference cognitive test. For the second experiment, we had 58 participants; the CGN_ICA was delivered via iPad, and both MoCA and ACE-R were used as reference tests in this experiment.

The third experiment had SDMT, BVMT-R and CVLT-II as the reference cognitive tests, measuring speed of information processing, visuo-spatial memory and verbal learning, respectively. These three tests together form the BICAMS battery, which requires about 15 to 20 minutes to administer, and is primarily used to detect cognitive dysfunction in younger adults who may suffer from multiple sclerosis (MS). 166 participants took part in this experiment. Forty-four of them were selected for a re-test as part of a second visit to assess CGN_ICA test-retest reliability. Participants for the re-test session were selected at random, while keeping the age-range, level of education, and gender ratio relatively similar to the set of participants in the first session. The CGN_ICA was delivered via an iPad platform.

All the pen-and-paper cognitive tests were administered by a healthcare professional. The administration order for CGN_ICA vs. reference cognitive tests was at random.

Finally, experiment 4 was designed to study whether the CGN_ICA test had a learning bias if taken multiple times in short intervals. Learning bias is defined as the ability to improve your test score by learning the test simply because of several exposures to the test. 12 young volunteers participated in this study. For convenience, the CGN_ICA was delivered remotely via a web platform. Participants took the CGN_ICA test every other day for two weeks.

Accuracy, speed, and CGN_ICA summary score calculations

Preprocessing

We used boxplot to remove outlier reaction times, before computing the CGN_ICA score. Boxplot is a non-parametric method for describing groups of numerical data through their quartiles; and allows for detection of outliers in the data. Following the boxplot approach, reaction times greater than q3 + w * (q3 − q1) or less than q1 − w * (q3 − q1) are considered outliers. q1 is the lower quartile, and q3 is the upper quartile of the reaction times. Where “w” is a ‘whisker’; w = 1.5.

Accuracy is simply defined as the number of correct categorisations divided by the total number of images, multiplied by a 100.

$$\mathrm{Accuracy}=\frac{numberofcorrectcategorisations}{totalnumberofimages}\times 100$$ 1

Speed is defined based on participant’s response reaction times in trials they responded correctly.

$$Speed=\min \left[100,100\times e^{\frac{-meancorrectRT}{1025}+0.341}\right]$$ 2

RT: reaction time

e: Euler’s number ~2.7182……

Speed is inversely related with participants’ reaction times; the higher the speed, the lower the reaction time. The reason for defining the above formula for speed, instead of using the raw reaction times, was to have a more intuitive and standardized score to report to the clinicians, scaled between 0 to 100.

The CGN_ICA summary score is a simple combination of accuracy and speed, defined as follows:

$$\mathrm{CGN}\_\mathrm{ICA}\mathrm{Score}=\left(\frac{\mathrm{Speed}}{100}\times \frac{Accuracy}{100}\right)\times 100$$ 3

Statistical analysis

Within the manuscript, convergent validity, and test-retest reliability for the CGN_ICA test is shown with Pearson’s Correlation. P-values for Pearson’s correlation are based on a Student’s t distribution. Calculations are done using MathWorks’ statistics and machine learning toolbox (https://www.mathworks.com/help/stats/index.html).

To measure dependency of the cognitive tests with level of education, we used explained variance, defined as the square of Pearson’s Correlation between participants’ cognitive score and their level of education (i.e. number of years). Here the statistical significance was obtained by a permutation test (10,000 permutations of participants). To formally assess statistical independence, we used a non-parametric independence test, proposed by Gretton and Gyorfi⁶², based on 10,000 bootstrap resampling of participants.

Finally, we used a single factor analysis of variance (ANOVA) to compare average CGN_ICA scores for participants who had taken the CGN_ICA test every other day for two weeks. The goal was to see if the mean CGN_ICA scores are significantly different at any given day.

Results

Convergent validity with the standard-of-care cognitive tests

A key requirement for a clinically useful cognitive assessment test is to establish validity and a correlation with an existing recognized neuropsychological test that is routinely used in clinical practice. Here in three different experiments (see Table 1, experiments 1 to 3), we show that the CGN_ICA test is significantly correlated with six standard neuropsychological tests (Fig. 2 and Table 2).

Figure 2.

The CGN_ICA test score is significantly correlated with a wide range of standard cognitive tests. Participants have taken the CGN_ICA test along with one or more standard cognitive tests (see Table 1). Each scatter plot shows the ICA score (y axis) vs. one of the standard cognitive tests (x axis). Each blue dot indicates an individual; the lines are results of linear regression, fitting a linear line to the data in each plot. For each plot, number of participants who have taken the tests and platform on which the CGN_ICA is taken are written on top of the scatter plot. ‘r’ and ‘p’ on top-right corner of each plot show the Pearson correlation between the two candidate tests, and the p-value of the correlation, respectively.

Table 2.

Correlation of the CGN_ICA test scores with various domains of cognition.

Cognitive Test	Correlation with			Cognitive Domain
	CGN_ICA	Accuracy	Speed
SDMT	0.80, p < 10−7	0.41, p < 10−7	0.71, p < 10−7	speed of processing
CVLT-II	0.66, p < 10−7	0.40, p < 10−6	0.56, p < 10−14	verbal learning
BVMT-R	0.54, p < 10−6	0.40, p < 10−8	0.43, p < 10−7	visual memory
MoCA (1) -RaPi	0.46, p < 10−11	0.48, p < 10−12	0.19, p < 0.01	General
MoCA (2) -iPad	0.55, p < 10−4	0.56, p < 10−5	0.19, ns	General
ACE-R total score	0.60, p < 10−6	0.52, p < 10−4	0.28, p < 0.05	total score of 5 domains
ACE|Attention	0.27, p < 0.05	0.25, ns	0.11, ns	attention
ACE|Memory	0.47, p < 10−3	0.37, p < 0.01	0.26, p < 0.05	verbal memory
ACE|Fluency	0.45, p < 10−3	0.25 ns	0.35, p < 0.01	Fluency
ACE|Language	0.53, p < 10−4	0.62, p < 10−6	0.12, ns	Language
ACE|Visuospatial	0.42, p < 10−3	0.44, p < 10−3	0.12, ns	visuospatial
MMSE	0.33, p < 0.01	0.36, p < 0.01	0.08, ns	General

This table shows correlation of the CGN_ICA test with a wide variety of standard-of-care tests for cognitive assessment, each measuring various domains of cognition. The two columns for accuracy and speed indicate the contribution of each of these two components separately toward the CGN_ICA score correlation with the reference cognitive tests. The correlations are Pearson correlations, p-values are written in front of each correlation; ns means not-significant (p > 0.05). MoCA(1): correlation between MoCA and RaPi implementaiont of the ICA test (experiment 1). MoCA(2): correlations between MoCA and the iPad implementation of the CGN_ICA test (experiment 2).

Given the variability in subject’s demographics, such as age, gender, and level of education, a statistically significant correlation typically above 0.4 (sometimes > 0.3) with reference cognitive tests is considered within the acceptable range for convergent validity (i.e. construct validity)^63–65. To give few examples, convergent validity for ACE-R is shown with a correlation of −0.32 with clinical dementia scale⁵; for CogState (a computerized cognitive battery), convergent validity is shown by correlations that vary between 0.11 and 0.53 with reference pen-and-paper cognitive tests^64,66,67. Similarly, cerebral spline fluid (CSF) and blood biomarkers have correlations in the range of 0.4 to 0.6^68,69 with standard cognitive tests, such as MoCA.

We show that the CGN_ICA score is significantly correlated with MoCA, tested on two different hardware platforms (RaPi and iPad). CGN_ICA correlation with MoCA varies from 0.46 to 0.55 (Fig. 2D and E) that is within the range for determining construct validity.

The CGN_ICA test had a slightly higher correlation with ACE-R (r = 0.60, p < 10⁻⁶), compared to MoCA. ACE-R provides a more comprehensive assessment of cognitive abilities and takes a longer time to administer and score (~20 minutes). It is comprised of five subsections, assessing attention, memory, fluency, language, and visuospatial abilities. The CGN_ICA correlation with ACE-R (Fig. 2F) and its different sub-sections are shown in Table 2. Subject’s MMSE score can also be extracted from the ACE-R test (see Table 2). MoCA and ACE-R are typically used to screen for MCI and dementia in older adults.

MMSE is shown to be less sensitive in detecting cognitive impairment^4,5 compared to MoCA or ACE-R. Therefore, a smaller correlation with MMSE (r = 0.33), and a higher correlation with MoCA and ACE-R (r = 0.55 and 0.60) is of interest.

In addition, we compared CGN_ICA against another set of tests, including SDMT, BVMT-R and CVLT-II (Fig. 2A–C, and Table 2) that are more often used in younger individuals to assess cognitive performance. For example, all these three tests are included as part of larger battery of tests that assess cognitive impairment in individuals with MS, such as the ‘minimal assessment of cognitive function in MS’ (MACFIMS) and the ‘brief international cognitive assessment for MS’ (BICAMS).

ICA had the highest correlation with SDMT (r = 0.80, p < 10⁻⁷), which is a pen-and-paper test mostly measuring the speed of information processing. CVLT-II, measuring verbal learning, and BVMT-R, measuring visual memory, had correlations of 0.66 and 0.54 with the CGN_ICA score, respectively. SDMT is shown to be more sensitive in detecting cognitive impairment in patients with MS^59,70, compared to CVLT-II and BVMT-R, therefore, CGN_ICA’s higher correlation with SDMT (compared to CVLT-II and BVMT-R) is of interest.

It is worth noting that a correlation of one is not desirable between the CGN_ICA test and any of these cognitive tests, as none of these standard tests are considered the ground truth (or gold standard) in detecting cognitive impairments.

The majority of cognitive tests (Table 2) were more correlated with the accuracy component of the CGN_ICA test, except for SDMT and CVLT-II, both of which have got a significantly higher correlation with speed compared to that of accuracy (p < 0.001; bootstrap resampling of subjects).

Each reference cognitive test used in this study (shown in Table 2) measured different domains of cognition. The CGN_ICA score had significant correlations with all of these tests, suggesting that it can be effectively used as one integrated test to provide insights about different cognitive domains (e.g. speed of processing, memory, verbal learning, attention, and fluency).

CGN_ICA shows excellent test-retest reliability

One of the most critical psychometric criteria for the applicability of a test is its reliability. That is the deviation observed when using the same instrument multiple times under similar circumstances.

To assess the reliability of the CGN_ICA test, a subgroup of 44 participants from experiment 3 (see Table 1) took the CGN_ICA test for the second time after about five weeks (+−15 days). Test-retest reliability was measured by computing the Pearson correlation between the two CGN_ICA scores [Fig. 3; Pearson’s r = 0.96 (p < 10⁻⁷)]. R values for test-retest correlation are considered adequate if > 0.70 and good if > 0.80 (Anastasi, 1988).

Figure 3.

CGN_ICA score, test-retest scatter plot. Each blue dot shows the ICA score for an individual taken on two different days. The blue line indicates the linear curve fitted to the test-retest data. [Pearson’s r = 0.96 (*p < 10⁻⁷)].

How much of the CGN_ICA score is explained by education?

People with higher levels of education tend to score better in the standard pen-and-paper tests, compared to their age-matched group that fall into the same cognitive category. This makes ‘the level of education’ a confounding factor for cognitive assessment.

We were interested to see how much of the CGN_ICA score is explained by education in comparison to other cognitive tests. To this end, we computed the Pearson’s correlation between participants’ test scores and their level of education (in years). Explained variance is defined as the square of this correlation, and indicates how of much of the variance of these test scores can be explained by education (Fig. 4).

Figure 4.

Dependency of standard-of-care cognitive tests on education. Bars indicate how much of the scores reported in each test are explained by education [explained variance = (Pearson’s r)²]. Stars show statistical significance, indicating that a significant variance of the test score is explained by education. Statistical significance is obtained by permutation test (10,000 permutations). Error bars are standard errors of the mean (SEM) obtained by 10,000 bootstrap resampling of subjects. P-values smaller than 0.01 (after Bonferroni correction for multiple comparison) are considered significant. ‘ns’ means not significant. The results for CGN_ICA (RaPi platform) are based on 212 subjects (Experiment 1 in Table 1); results for CGN_ICA (iPad platform) are based on the combined data from Experiments 2 and 3 (224 participants in total) all of whom took the CGN_ICA on an iPad. The results for MoCA are based on the combined data from Experiments 1 and 2, in both of which participants took MoCA (270 participants in total). ACE-R and MMSE results are based on the data from Experiment 2. Results for SDMT, BVMT-R and CVLT-II are based on the data from Experiment 3.

We find that a significant variance of all the standard cognitive assessment tests is explained by education, whereas the CGN_ICA test does not show a significant relationship with education (Fig. 4). In Fig. 4, we separately reported the CGN_ICA test results for the RaspberryPi platform (Experiment 1: 212 participants) and the iPad platform (experiments 2 and 3: 58 + 166 = 224 participants).

Furthermore, we formally tested whether the CGN_ICA score is independent of education using a non-parametric test of independence⁶². In experiment 1 (i.e. CGN_ICA taken on RaPi), the statistical test of independence was positive, showing that CGN_ICA score is independent of education (based on 10,000 bootstrap resampling of subjects).

The CGN_ICA test has no learning bias

One problem with many existing cognitive tests is that they have a learning bias, meaning that subject’s cognitive performance is improved by repeated exposure to the test as a result of learning the task, without any change in their cognitive ability. A learning bias reduces the reliability of a test if repeatedly used, for example when monitoring performance over time. An ideal test for early diagnosis of cognitive disorders and monitoring cognitive performance would show no ‘learning bias’.

The currently available pen-and-paper tests, such as MoCA, MMSE and Addenbrooke’s Cognitive Examination (ACE), are not appropriate for micro-monitoring of cognitive performance because if identical questions are repeated, healthy participants and those with mild impairment can easily learn the test and improve upon their previous scores – as a result of learning rather than any improvement in their cognitive performance.

To investigate whether CGN_ICA might be appropriate for such micro-monitoring, we recruited 12 young individuals with high capacity for learning [University students, aged 20 to 36], and asked them to take the test every other day for two weeks (8 days in total). The CGN_ICA was delivered remotely via a web platform.

The test data indicate that even in subjects with a high capacity to learn, no learning bias was detected (Fig. 5). The CGN_ICA score does not increase monotonically, and comparing the mean of the CGN_ICA scores across these days, no significant difference was observed (ANOVA, F(7) = 0.62; P-value = 0.73).

Figure 5.

No significant effect of learning in repeated exposure to the CGN_ICA test. We find no learning bias when the test is taken multiple times. 12 healthy participants (age range = [20, 36]) took the CGN_ICA test every other day for over two weeks (ANOVA, F(7) = 0.62; P-value = 0.73). From these 12 participants, 7 of them completed all the sessions (8 days); and the rest did the test for at least the first three days.

Discussion

Early diagnosis is the mainstay of focus in scientific research^71,72. There is currently no available cognitive screening tool that can detect early phenotypical changes prior to the emergence of memory problems and other symptoms of dementia. The vast majority of cognitive tests rely on the patients’ capacity to read and write while more educated individuals can often “second-guess” them. All of these standard tests require a clinician or a health-care professional to administer them, thus adding a considerable cost to the procedure.

We demonstrated that the combination of speed and accuracy of visual processing in a rapid visual categorization task can be used as a reliable measure to assess individual’s cognitive performance. The proposed visual test has significant advantages over the conventional cognitive tests because of its efficient administration, shorter duration, automatic scoring, language and education independency, potential for medical record or research database integration, and the capacity for micro monitoring of cognitive performance given the absence of a “learning bias”. Thus, we suggest CGN_ICA as a practical tool for routine screening of cognitive performance.

Potential use of CGN_ICA for early detection of dementia

Because of the high compensatory potential of the brain, symptoms of chronic neurodegenerative diseases, such as Alzheimer’s (AD), Parkinson (PD), Huntington (HD) diseases, vascular and frontotemporal (FTD) dementias occur 10–20 years after the beginning of the pathology⁹. Late stages of these disorders are characterized by massive neuronal death that is irreversible. Therefore, any late therapeutic treatment in the course of the disease will most likely fail to positively affect the disease progression in any meaningful way. This is illustrated by recent failures of anti-AD therapies in late stage clinical trials^2,73. Thus further emphasizing the importance of the development of screening tests capable of detecting such diseases in their early asymptomatic stage.

ICA aims at early detection of cognitive dysfunction by targeting brain functionalities that are affected in the initial stages of the neurodegenerative disorders (e.g. dementia), specifically before the onset of memory symptoms. Given the decade-long lag between tissue damage and memory deficits in dementia, the CGN_ICA instead examines the visuo-motor pathway. Studies in the past 20 years reveal that all parts of the visual system may be affected in Alzheimer’s Disease, including the optic nerve, retina, lateral geniculate nucleus (LGN) and the visual cortex¹⁹. Particularly, in early stages of the disease, brain areas associated with the visuo-motor pathway are affected, beginning with the retina^26–28,30, the visual cortex^25,29,30 and the motor cortex^21,31, so together these represent more effective areas to look for the impact of early stage neurodegeneration as opposed to solely focusing on memory. The CGN_ICA focuses on cognitive functions such as speed and accuracy of processing visual information which have been shown to engage a large volume of cortex, while being a predictor of people’s cognitive performance^33–35; thus, monitoring the performance and functionality of these areas altogether can be a reliable early indicator of the disease onset.

Suitability for remote and frequent cognitive assessment

Remote monitoring or home-based online assessments is beneficial for patients, clinicians and researchers. Home-based assessment allows for a more comfortable setting for patients with a low stress environment. In addition, researchers and clinicians will have a time-efficient and convenient assessment instrument, which enables a valid and reliable evaluation of individuals’ cognitive performance. Furthermore, online assessment allows the researcher to collect data from a large number of participants in a short time period.

Given that the CGN_ICA test is self-administered and that it does not suffer from a learning bias, it can be used remotely and frequently to track changes in individuals’ cognitive performance over time. This makes the test even more useful for early diagnosis, by allowing the test to be used longitudinally, in a design wherein individuals are compared against their own baseline.

Conclusion and Future Directions

The CGN_ICA is designed to be an extremely easy to use, versatile and practical measurement tool for studies into dementias and other conditions that have an element of cognitive function, as it allows simple, sensitive and repeatable data collection of an overall score of a subject’s cognitive ability. The CGN_ICA platform is being further developed to employ artificial intelligence (AI) to improve its predictive power, utilizing patterns of participants’ response reaction times. The AI platform will allow for accurate classification of participants into cognitively healthy or cognitively impaired by comparing their CGN_ICA test profile with a large dataset of many individuals with validated clinical status which the AI platform has “learned” from. The AI engine will have the ability to improve its accuracy over time by learning from new data points that are incorporated into its training datasets.

Author Contributions

S.K.R. wrote the manuscript. C.K. gave comments on the manuscript. S.K.R., S.H., M.S., H.M., M.K., S.M.N., and E.S. helped with the data acquisitions. S.K.R., S.H., S.M.N. and C.K. helped with devising the protocol for the experiments. S.K.R., M.S., H.M., M.K. analyzed the data.

Data Availability Statement

The data generated during this study are included in this published article. Commercially insensitive raw data can be made available upon reasonable request from the corresponding author.

Competing Interests

Dr. Khaligh-Razavi, Dr. Habibi, and Dr. Kalafatis serve as directors at Cognetivity ltd. Other authors declared no competing interests.