ABSTRACT
Background
The Three‐Objects‐Three‐Places (3O3P) test is a 5‐min screen for episodic memory impairment due to Alzheimer's disease, known for its briefness and easy administration, culture‐ and language‐free nature, and the absence of specific equipment. However, no studies have validated its potential in memory clinic cohorts. The aim of this study was to test its convergent, discriminant, and known‐group validities and to define thresholds for its clinical use.
Methods
We included 2062 cognitively unimpaired (CU), mild cognitive impairment (MCI) and dementia patients from the Geneva Memory Center cohort who underwent the 3O3P test in the context of clinical practice. Convergent and discriminant validities were assessed using an exploratory factor analysis. The known‐group validity was assessed in CU vs. MCI and dementia using the area under the curve (AUC). 3O3P test scores vs. amyloid and tau positivity, neurodegeneration, and cognition (ATNC) were assessed using the Kruskal‐Wallis test. The 3O3P test cut‐offs were calculated using sensitivity, specificity, PPV, NPV, and accuracy.
Results
Mean age was 72 years (SD = 11), 60% were female, mean education was 13 years (SD = 4), and mean MMSE was 25 (SD = 5). The 3O3P and Delayed Total Recall tests loaded strongly on the “memory” factor and weakly on “non‐memory” factors. The 3O3P test can discriminate CU vs. MCI (AUC = 0.71) and dementia (AUC = 0.92). Higher 3O3P scores were associated with lower prevalence of ATNC (p < 0.001). A 3O3P value of 7 can detect MCI and dementia patients.
Conclusions
The 3O3P test has demonstrated good convergent, discriminant, and known‐group validity in a large memory clinic population.
Keywords: Alzheimer's disease, biomarkers, dementia, episodic memory, mild cognitive impairment
1. Introduction
Deficits in episodic memory, a form of explicit memory, are an early clinical manifestation of Alzheimer's disease (AD) [1].
Several screening tests have been developed and proposed as part of memory clinic workups to evaluate global cognition. Among them, the Mini Mental State Examination (MMSE) [2] or the Montreal Cognitive Assessment (MoCA) [3] can be mentioned, both of which are widely used in memory clinic settings. Studies have shown that the MMSE may be better suited than the MoCA for assessing AD as its progresses, due to most aspects of cognition being affected as the disease worsens [4]. On the other hand, the MoCA has demonstrated similar diagnostic validity in assessing for AD, but it excels in identifying patients with Mild Cognitive Impairment (MCI) [5]. Many other tests exist, such as the Clock Drawing Test [6], Mini‐Cog [7], or 6‐CIT [8], but their respective effectiveness remains unclear [2, 9]. Moreover, several of these tests have been criticized, primarily due to their poor ecological validity [10].
The Three‐Objects‐Three‐Places (3O3P) test has been developed as a quick screen for episodic memory impairment due to AD, and it requires a conscious retrieval of information acquired in a specific time and place. The examiner shows the patient three objects and hides them in three separate places. It is inspired by the Rivermead Behavioral Memory Test (RBMT) designed to serve as a link between laboratory‐based memory measures and assessments derived from questionnaires and observations. The goal of the RBMT is to offer analogs of everyday memory situations that may pose challenges for specific patients with brain injuries [11]. The RBMT includes 12 tasks, including “remembering a hidden belonging” where a personal object is requested from the subject and placed in a specified location. The subject must remember the correct location at the end of the complete 12‐item test. The present version [12] is different in the number of items and in their fixed nature (a pencil, a key, and a coin) and hidden placement (the pencil behind a telephone 1 m away, the key on a cupboard 2 m away, and the coin under a book 1.2 m away from the patient). The patient is later asked what the objects and hiding places were and which object was hidden where, resulting in a score on a scale of 1–9.
The 3O3P administration is characterized by its efficiency, short time of administration, easy utilization without the need for acquisition of score sheets or equipment, and easy to score. At the same time, it has a strong ecological nature, as forgetting where one has placed personal objects is one of the most frequent episodic memory complaints [12, 13].
The study of Prestia et al. assessed the validity of the 3O3P test as a screening tool for AD and reported high homogeneity, interrater reliability, good convergent and discriminant validity, as well as high sensitivity for identifying AD across different age groups. However, this study analyzed a small cohort (controls N = 65, MCI N = 114, dementia N = 83) of research patients, without any characterization of disease biomarkers [12].
The aim of this study was to investigate the convergent, discriminant, and known‐group validity of the 3O3P test results in a larger (N = 2062) and deeply phenotyped memory clinic cohort. Additionally, our research aimed to explore the associations between 3O3P scores and AD biomarkers, such as amyloid, tau, neurodegeneration, and cognitive status. Finally, we sought to establish specific 3O3P cut‐off scores by diagnoses and age groups and provide clinical guidelines on how to use this test effectively in a clinical setting.
2. Methods
2.1. Participants
We included Cognitively Unimpaired (CU), Mild Cognitive Impairment (MCI), and demented patients who were consecutively assessed at the Geneva Memory Center as part of routine clinical evaluations from January 2014 to June 2023. All patients underwent a comprehensive diagnostic workup, including clinical and neuropsychological assessment, as previously outlined by Ribaldi et al. [14] The Geneva Memory Center's workup begins with an initial visit, during which we collect demographic information, medical and family histories, and assess depression, anxiety, functional impairment, and cognitive complaints. This visit includes a neurological examination and a brief cognitive screening, which comprises the Mini‐Mental State Examination (MMSE), the Three‐Objects‐Three‐Places (3O3P) test, and the clock drawing test. Following this initial assessment, patients undergo a comprehensive neuropsychological evaluation that includes a variety of tests, such as the Free and Cued Selective Reminding Test (FCSRT) and the writing language test from the GREMOTS battery (for more details see Ribaldi et al., 2021) [14]. Based on the results of these assessments, patients are classified into cognitive stages: CU, MCI, and dementia, according to established clinical diagnostic criteria outlined in previous work [14]. Additionally, for those clinically indicated or participating in specific research protocols, further investigations are conducted, including MRI scans (N = 1401), amyloid PET scans (N = 390), and tau PET scans (N = 191). In this study, cognitive impairment (C) was also used as a dichotomous variable using MMSE with a cutoff of 26.
2.2. Biomarkers Collection
In the present study, we employed dichotomous (positive vs. negative) measures of AD biomarkers assessed through conventional techniques. Detailed information on acquisition protocols can be found in our previous publication [14].
For structural imaging, 3T MRI scans were acquired. We utilized the average medial temporal lobe atrophy scale (MTA) visual score after its assessment on both sides. MTA positivity (N‐positive/negative) was defined using age‐specific cutoffs that offer the best discrimination between individuals with AD dementia and control subjects [15].
Amyloid‐PET images were obtained using 18F‐florbetapir or 18F‐flutemetamol tracers, and tau‐PET images were acquired using 18F‐flortaucipir, following standard imaging protocols and reconstructions. These protocols were previously described [16, 17] in detail and briefly summarized here: (i) 18F‐florbetapir images were acquired 50 min after injection of 200 MBq during 15 min; (ii) 18F‐flutemetamol images were acquired 90 min after injection of 150 MBq during 20 min; and (iii) 18F‐flortaucipir images were acquired 75 min after injection of 180 MBq during 30 min. All acquisitions were performed using Siemens Biograph and Biograph Vision scanners, and the images were reconstructed using a 3D OSEM iterative reconstruction method, corrected for randoms, dead time, normalization, scatter, attenuation, and sensitivity.
Amyloid‐PET positivity (A‐positive/negative) was assessed by an expert nuclear medicine physician (VG) using established visual assessment procedures and standard operating protocols approved by the European Medicines Agency [18]. Tau distribution in each patient was determined by the same expert in nuclear medicine, who visually analyzed images in agreement with recently published recommendations. These recommendations describe regions with increased 18F‐flortaucipir uptake corresponding to pathologically defined Braak stages: medial temporal regions for Braak stages I–III, other temporal regions for Braak stage IV, parietal and frontal lobes for Braak stage V, and sensorimotor areas and visual primary cortex for Braak stage VI. To define tau‐PET positivity (T‐positive/negative), we considered visually scored Braak stages 0–III as tau negative and Braak stages IV–VI as tau positive [19].
2.3. Statistical Analysis
We calculated means and standard deviations (SD) for continuous variables (including demographics and neuropsychological test scores), while we reported numbers and percentages for biomarker positivity. To assess differences in demographics, clinical characteristics, cognitive performance, and biomarker features, we conducted a Kruskal–Wallis test for continuous variables and a chi‐squared test for categorical variables.
2.3.1. Convergent and Discriminant Validity
To evaluate the convergent and discriminant validity of the 3O3P test, we conducted an exploratory factor analysis, specifically using a principal component analysis extraction model with Varimax and Kaiser normalization rotation method. The results are presented in terms of saturation values, which indicate the extent to which a specific variable is associated with an identified factor in the factorial analysis. Additionally, loadings are reported to represent the correlations between the observed variables and the factors extracted during the analysis. This analysis included the following tests: delayed total recall (DTR) of the FCSRT, clock drawing test, language writing test from the GREMOTS battery, and the 3O3P test. These selected tests allowed us to explore the associations between the 3O3P test and other memory, as well as non‐memory factors.
2.3.2. Diagnostic and Known Group Validity
The diagnostic validity of the 3O3P, MMSE, and DTR was examined using the Kruskal–Wallis test and effect size (Cohen's d) among diagnostic groups (CU, MCI, and dementia). Cohen's d is interpreted following the guidelines proposed by Cohen (1988) as follows: a small effect size corresponds to d = 0.2, a medium effect size to d = 0.5, and a large effect size to d = 0.8 [20]. We also calculated the area under the curve (AUC) for each neuropsychological test for CU vs. MCI and CU vs. dementia. DeLong's tests were performed to compare the AUC scores of the 3O3P vs. MMSE and 3O3P vs. DTR.
Known group validity was assessed in relation to AD markers positivity, including A/T/N/C. We analyzed the differences in 3O3P test scores by the aforementioned markers positivity using the Kruskal–Wallis test.
2.3.3. Determination of Cut‐Off Scores
We first tested which variables had an impact on the differences in the 3O3P test score among diagnoses. To do so, we ran a linear model with 3O3P as the dependent variable and diagnosis, age, sex, education, and the interaction between diagnosis and age/sex/education as independent variables. To identify appropriate cut‐offs for the 3O3P test to differentiate between diagnostic groups (CU vs. MCI and CU vs. dementia) we calculated sensitivity, specificity, positive‐predictive values (PPV), negative‐predictive values (NPV), accuracy, and AUC. The most suitable cut‐off scores to screen cognitively impaired patients were selected based on these metrics.
3. Results
A total of 2062 participants were included in this study, comprising 555 CU, 948 MCI, and 559 demented patients, as summarized in Table 1. The average age across the cohort was 72 years (SD = 11), with 60% being female. Participants had an average of 13 years of education (SD = 4), and the mean MMSE was 25 (SD = 5). Significant differences were observed among the groups in the expected direction. Notably, patients with dementia were significantly older (78 ± 8) compared to both the CU group (65 ± 11) and the MCI group (72 ± 11; p < 0.001). Furthermore, the CU group exhibited a significantly higher level of education (15 years ±4) than both the MCI group (13 years ±4) and the dementia one (11 years ±4; p < 0.001). Cognitive performance decreased with increasing disease severity (better cognition in CU, followed by MCI, and finally dementia) in the following tests: MMSE, 3O3P, DTR—FCSRT, Clock drawing test, and Language—writing (p < 0.001). For the subset of participants whose biomarkers were tested, individuals with dementia had a significantly higher prevalence of A/T/N biomarkers, with 81% being A+, 68% T+, and 73% N+. In comparison, those with MCI had lower prevalence rates (A+ in 62%, T+ in 48%, N+ in 41%), while CU individuals had the lowest prevalence rates (A+ in 19%, T+ in 2%, N+ in 23%; p < 0.001).
TABLE 1.
Descriptive features of study participants from the Geneva Memory Center cohort (N = 2062).
| Variables | Unimpaired, n = 555 | MCI, n = 948 | Dementia, n = 559 | p |
|---|---|---|---|---|
| Demographics | ||||
| Age | 65 ± 11 | 72 ± 11 | 78 ± 8 | < 0.001 |
| Sex, female | 380 (66%) | 528 (56%) | 329 (58%) | 0.005 |
| Education | 15 ± 4 | 13 ± 4 | 11 ± 4 | < 0.001 |
| Neuropsychological Test | ||||
| Mini Mental State Examination | 28.6 ± 1.3 | 25.7 ± 3.0 | 19.9 ± 4.9 | < 0.001 |
| Three‐Objects‐Three‐Places | 8.8 ± 0.8 | 7.3 ± 2.2 | 4.2 ± 2.8 | < 0.001 |
| FCSRT—Delayed Total Recall | 15.8 ± 0.6 | 13.5 ± 3.0 | 10.5 ± 4.0 | < 0.001 |
| Clock Drawing Test | 9.3 ± 1.1 | 8.1 ± 2.1 | 5.5 ± 2.9 | < 0.001 |
| Language—Writing | 11.2 ± 1.4 | 10.1 ± 2.1 | 9.1 ± 2.7 | < 0.001 |
| Biomarkers, positivity | ||||
| A (PET visual read) | 20/106 (19%) | 130/209 (62%) | 61/75 (81%) | < 0.001 |
| T (PET visual read) | 1/53 (2%) | 53/110 (48%) | 19/28 (68%) | < 0.001 |
| N (MTA mean on MRI) | 80/345 (23%) | 274/669 (41%) | 284/387 (73%) | < 0.001 |
| C (MMSE ≤ 26) | 12 (2%) | 381 (40%) | 502 (90%) | < 0.001 |
Abbreviations: A, Amyloid; C, Cognition; FCSRT, Free and Cued Selective Reminding Test; MMSE, Mini Mental State Examination; MRI, Magnetic Resonance Imaging; MTA, Medial Temporal Lobe Atrophy scale; N, Neurodegeneration; PET, Positron Emission Tomography; T, Tau.
3.1. Convergent and Discriminant Validity
The principal component analysis revealed that a three‐factor solution provided the best fit, accounting for 88% of the variance. The first factor was labeled “memory,” with strong standardized loadings of 0.86 and 0.87 for the 3O3P and DTR tests, respectively. In contrast, the loadings for the Clock drawing test and Language‐writing were 0.07 and 0.19, respectively, on the “memory” factor. The second factor, “praxis,” exhibited a robust loading of 0.98 for the clock drawing test, highlighting its association with this factor. Finally, the third factor, designated as “Frontal‐executive” demonstrated a substantial loading of 0.99 for the Language‐writing test (see Table 2).
TABLE 2.
Factor loadings of the Three‐Objects‐Three‐Places test (3O3P), memory, verbal, and nonverbal tests.
| Tests | Factors | ||
|---|---|---|---|
| Memory | Praxis | Frontal‐Executive | |
| Memory | |||
| FCSRT—Delayed Total Recall | 0.87 | 0.11 | < 0.10 |
| Verbal | |||
| Language—Writing | < 0.10 | 0.06 | 0.99 |
| Non‐verbal | |||
| Clock Drawing Test | 0.19 | 0.98 | < 0.10 |
| 3O3P | 0.86 | 0.17 | < 0.10 |
Note: The first three factors account for 38%, 25%, and 25% of the total variance, respectively. Extraction method: principal component analysis. Rotation method: Varimax with Kaiser Normalization.
3.2. Diagnostic and Known Group Validity
Figure 1 illustrates the distribution of 3O3P, MMSE, and DTR scores across different diagnostic stages. The median scores were highest for the CU group, followed by the MCI group, whereas the lowest scores were found in the group composed of patients diagnosed with dementia, for each of the three tests. Cohen's d demonstrated substantial differences (large effect size), ranging from 0.83 to 2.21 for 3O3P, 1.13 to 2.40 for MMSE, and 1.00 to 2.89 for DTR across these diagnostic comparisons.
FIGURE 1.
Head‐to‐head comparison of scores on Three‐Objects‐Three‐Places, Mini Mental State Examination, and Delayed Total Recall—Free and cued selective reminding test, by cognitive stage. Boxplots denote the median and interquartile ranges of test scores. Asterisks represent statistically significant post hoc comparisons (****p < 0.0001). Cohen's d is reported below the significant comparisons. CU, Cognitively Unimpaired; MCI, Mild Cognitive Impairment.
Figure 2 displays moderate to very good AUC values for the three tests, indicating their ability to discriminate CU from MCI and dementia. As expected, AUC values were higher when comparing CU to MCI or dementia. However, the DeLong's test indicated that both MMSE (CU vs. MCI AUC = 0.83; CU vs. dementia AUC = 0.98) and DTR (CU vs. MCI AUC = 0.79; CU vs. dementia AUC = 0.94) exhibited superior discriminative power between diagnostic groups compared to 3O3P (CU vs. MCI AUC = 0.71; CU vs. dementia AUC = 0.92, p < 0.001).
FIGURE 2.
Known‐group validity of the Three‐Objects‐Three‐Places test (3O3P), Mini Mental State Examination (MMSE), and Delayed Total Recall—FCSRT. The figure shows the ROCs for each test in the comparisons between CU vs MCI (A) and CU vs dementia (B). The table below the curves reports the AUCs and 95% confidence interval of each test and p‐values of the comparisons between different tests using DeLong's test. AUC, area under the ROC Curve; CU, Cognitively Unimpaired; FCSRT, Free and cued selective reminding test; MCI, Mild Cognitive Impairment; ROC, receiver operating characteristic curve.
In the comparisons between 3O3P and A/T/N/C biomarkers, 3O3P demonstrated moderate to very good effect sizes. Notably, the highest effect size was observed for the discrimination between C+ versus C− (d = 1.47), followed by T+ vs. T− (d = 1.27), A+ vs. A− (d = 1.04), and finally N+ vs. N− (d = 0.65, Figure 3).
FIGURE 3.
Three‐Objects‐Three‐Places (3O3P) scores by A, T, C, and N status. Boxplots denote the median and interquartile ranges of the 3O3P scores by A, T, C, and N status. A and T positivity was assessed on visual rating by an expert nuclear medicine physician. N has been assessed with the Medial Temporal Lobe Atrophy (MTA) scale and dichotomized using cut‐offs corrected by age [15]. Cognitive impairment was dichotomized using an MMSE cut‐off ≥ 26 to identify C negative patients. Asterisks represent statistically significant post hoc comparisons (****p < 0.0001). Cohen's d is reported below the significant comparisons. A, Amyloid; C, Cognition; N, Neurodegeneration; T, Tau.
3.3. Determination of Cut‐Off Scores
We observed a significant association only in the interaction between diagnosis and age with 3O3P test scores. Therefore, to distinguish between CU and MCI or CU and dementia, we employed varying cut‐off scores based on age ranges (Table 3). The optimal cut‐off value was 8 for the age range of 40–65 years, which decreased to 7 for the age range of 66–80 years, and further decreased to 5 for ages > 81. The overall optimal cut‐off score for the entire population was determined to be 7. These cut‐offs were selected to ensure high specificity (≥ 0.85), making them valuable for identifying individuals with amnestic cognitive impairment. The accuracy and AUC values were higher for discriminating between CU and dementia than for discriminating between CU and MCI. Specifically, the 3O3P test exhibited higher accuracy in discriminating CU versus MCI within the 66–80 years age group (accuracy 0.62, AUC = 0.73), while very good accuracy and AUC values were observed for all age groups in the comparison between CU and dementia (accuracy range 0.87–0.94, AUC range = 0.85–0.91).
TABLE 3.
Metrics of diagnostic accuracy and age‐specific thresholds of the Three‐Objects‐Three‐Places (3O3P) test score.
| Age range (years) | Cut‐off | Sensitivity | Specificity | PPV | NPV | Accuracy | AUC |
|---|---|---|---|---|---|---|---|
| Cognitively unimpaired versus Mild Cognitive Impairment | |||||||
| 40–50 (CU N = 60 vs. MCI N = 51) | ≤ 5 | 0.04 | 1.00 | 1.00 | 0.55 | 0.56 | 0.55 |
| ≤ 6 | 0.10 | 1.00 | 1.00 | 0.57 | 0.59 | ||
| ≤ 7 | 0.12 | 0.97 | 0.75 | 0.56 | 0.58 | ||
| ≤ 8 | 0.14 | 0.95 | 0.70 | 0.56 | 0.58 | ||
| ≤ 9 | 1.00 | 0.00 | 0.46 | NA | 0.46 | ||
| 51–65 (215 vs. 189) | ≤ 5 | 0.07 | 1.00 | 1.00 | 0.55 | 0.57 | 0.63 |
| ≤ 6 | 0.13 | 1.00 | 1.00 | 0.57 | 0.59 | ||
| ≤ 7 | 0.24 | 0.96 | 0.84 | 0.59 | 0.62 | ||
| ≤ 8 | 0.30 | 0.95 | 0.84 | 0.61 | 0.65 | ||
| ≤ 9 | 1.00 | 0.00 | 0.47 | NA | 0.47 | ||
| 66–80 (247 vs. 517) | ≤ 5 | 0.23 | 1.00 | 0.99 | 0.38 | 0.48 | 0.73 |
| ≤ 6 | 0.33 | 0.99 | 0.98 | 0.41 | 0.54 | ||
| ≤ 7 | 0.47 | 0.93 | 0.93 | 0.46 | 0.62 | ||
| ≤ 8 | 0.52 | 0.90 | 0.91 | 0.47 | 0.64 | ||
| ≤ 9 | 1.00 | 0.00 | 0.68 | NA | 0.68 | ||
| > 81 (33 vs. 191) | ≤ 5 | 0.35 | 0.88 | 0.94 | 0.19 | 0.42 | 0.65 |
| ≤ 6 | 0.46 | 0.82 | 0.94 | 0.21 | 0.51 | ||
| ≤ 7 | 0.61 | 0.58 | 0.89 | 0.20 | 0.60 | ||
| ≤ 8 | 0.68 | 0.55 | 0.90 | 0.23 | 0.66 | ||
| ≤ 9 | 1.00 | 0.00 | 0.85 | NA | 0.85 | ||
| Whole sample (555 vs. 948) | ≤ 5 | 0.21 | 0.99 | 0.98 | 0.42 | 0.50 | 0.71 |
| ≤ 6 | 0.30 | 0.98 | 0.97 | 0.45 | 0.55 | ||
| ≤ 7 | 0.43 | 0.92 | 0.91 | 0.49 | 0.61 | ||
| ≤ 8 | 0.49 | 0.90 | 0.90 | 0.51 | 0.64 | ||
| ≤ 9 | 1.00 | 0.00 | 0.63 | NA | 0.63 | ||
| Cognitively unimpaired versus dementia | |||||||
| 40–50 (CU N = 60 vs. dementia N = 4) | ≤ 5 | 0.25 | 1.00 | 1.00 | 0.95 | 0.95 | 0.86 |
| ≤ 6 | 0.50 | 1.00 | 1.00 | 0.97 | 0.97 | ||
| ≤ 7 | 0.50 | 0.97 | 0.50 | 0.97 | 0.94 | ||
| ≤ 8 | 0.75 | 0.95 | 0.50 | 0.98 | 0.94 | ||
| ≤ 9 | 1.00 | 0.00 | 0.06 | NA | 0.06 | ||
| 51–65 (215 vs. 37) | ≤ 5 | 0.70 | 1.00 | 1.00 | 0.95 | 0.96 | 0.91 |
| ≤ 6 | 0.78 | 1.00 | 1.00 | 0.96 | 0.97 | ||
| ≤ 7 | 0.84 | 0.96 | 0.78 | 0.97 | 0.94 | ||
| ≤ 8 | 0.84 | 0.95 | 0.74 | 0.97 | 0.93 | ||
| ≤ 9 | 1.00 | 0.00 | 0.15 | NA | 0.15 | ||
| 66–80 (247 vs. 283) | ≤ 5 | 0.64 | 1.00 | 0.99 | 0.71 | 0.81 | 0.90 |
| ≤ 6 | 0.73 | 0.99 | 0.99 | 0.76 | 0.85 | ||
| ≤ 7 | 0.82 | 0.93 | 0.93 | 0.82 | 0.87 | ||
| ≤ 8 | 0.83 | 0.90 | 0.90 | 0.83 | 0.86 | ||
| ≤ 9 | 1.00 | 0.00 | 0.53 | NA | 0.53 | ||
| > 81 (33 vs. 235) | ≤ 5 | 0.70 | 0.88 | 0.98 | 0.29 | 0.72 | 0.85 |
| ≤ 6 | 0.80 | 0.82 | 0.97 | 0.37 | 0.81 | ||
| ≤ 7 | 0.88 | 0.58 | 0.94 | 0.40 | 0.84 | ||
| ≤ 8 | 0.90 | 0.55 | 0.93 | 0.43 | 0.85 | ||
| ≤ 9 | 1.00 | 0.00 | 0.88 | NA | 0.88 | ||
| Whole sample (555 vs. 559) | ≤ 5 | 0.67 | 0.99 | 0.99 | 0.75 | 0.83 | 0.91 |
| ≤ 6 | 0.77 | 0.98 | 0.98 | 0.81 | 0.87 | ||
| ≤ 7 | 0.84 | 0.92 | 0.92 | 0.85 | 0.88 | ||
| ≤ 8 | 0.86 | 0.90 | 0.90 | 0.87 | 0.88 | ||
| ≤ 9 | 1.00 | 0.00 | 0.50 | NA | 0.50 | ||
Note: Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for the 3O3P test are reported in cognitively unimpaired (CU, N = 555) versus mild cognitive impairment (MCI, N = 948) and CU versus dementia (N = 559). Values in boldface are those recommended for clinical use based on the most favorable diagnostic accuracy metrics.
4. Discussion
The present study highlights the robust convergent, discriminant, and group validity of the 3O3P test when applied within a large memory clinic cohort. Although the MMSE and DTR tests may have superior discriminant abilities among diagnostic groups, the 3O3P test features several practical advantages. Its brief administration time, cultural and language neutrality, and minimal equipment requirements make it a valuable tool for identifying episodic memory impairment associated with AD in memory clinic settings.
The present study demonstrates that the 3O3P test assesses memory functions that are strongly correlated with those evaluated by the DTR of the free and cued selective reminding test. Furthermore, the 3O3P test exhibits similar diagnostic accuracy to the MMSE and DTR in discriminating dementia patients from CU individuals, as evidenced by the large effect size for all three tests (3O3P d = 2.21; MMSE d = 2.40; DTR d = 2.89). These effect sizes are slightly higher than those reported in the literature for the MMSE (d = 1.59) and other screening tests like the Montreal Cognitive Assessment (d = 1.80) in the comparison between dementia and CU [21]. Similar results were observed in the comparisons between MCI and CU [22]. Importantly, the AUCs for MMSE and DTR were closely aligned with previously reported values (e.g., AUC = 0.96 for MMSE in dementia vs. CU [22]) [23], indicating the generalizability of our results. Therefore, the 3O3P test demonstrates good diagnostic accuracy, particularly in differentiating CU from dementia, and especially within the age range of 51–80 years.
Our findings reveal that the 3O3P test exhibits high specificity when using a cutoff of 7. This suggests its potential as a diagnostic tool for identifying patients with cognitive impairment and positive AD biomarkers. In a landscape where disease‐modifying treatments are becoming increasingly relevant, a rapid cognitive screening tool such as the 3O3P test could find valuable applications in memory clinics or primary care settings. Indeed, the 3O3P test provides clinicians with an easy and quick tool to identify episodic memory problems. With its high specificity, an impaired 3O3P test should prompt further cognitive and biomarker evaluations to confirm AD pathology. While it does not replace comprehensive neuropsychological testing, it serves as a valuable preliminary screening tool for clinical decision‐making. For instance, the 3O3P test can be combined with a global cognition assessment, such as the MMSE, and potentially complemented with biomarker analyses, including CSF, PET, or, in the future, blood‐based biomarkers, to enhance diagnostic confidence.
One key distinction between the 3O3P test and traditional episodic memory tests, such as word list recall, lies in its ecological nature. The 3O3P test assesses standard memory functions such as encoding, memorization, and recall while maintaining relevance to real‐world scenarios. Moreover, given the simplicity and ecological relevance of the 3O3P test, future studies should explore its potential for digital administration. This approach could facilitate its use in remote e‐health contexts, allowing remote cognitive screening for older adults or patients with limited access to in‐person assessments. Digital platforms could incorporate 3D simulations to replicate the physical nature of the test, enhancing the ecological validity of remote assessments. Further work should investigate the test's validity when delivered remotely.
4.1. Limitations and Strengths
The primary limitation of this study lies in its retrospective design, which might limit the generalizability of the results. This design may have introduced selection bias, as not all participants underwent biomarker assessment, potentially influencing the interpretation of the relationship between 3O3P scores and biomarker positivity. Indeed, biomarker data were primarily collected from individuals with clinical indications for further testing or as part of specific research studies with varying inclusion and exclusion criteria, making this subgroup potentially unrepresentative of the entire cohort. Additionally, while our study has a substantial overall sample size, the subgroups within the age range of 40–50 years and those with biomarkers, especially tau, were relatively small. As a result, certain subgroup analyses, such as discriminating between AD dementia and non‐AD dementia patients, were not feasible within the scope of this study. Another noteworthy limitation is the absence of parallel forms for the 3O3P test. This lack of alternative forms limits the utility of the test to cross‐sectional screening assessment, while it is not suitable to monitor disease progression. Nonetheless, this study possesses several strengths. The robustness of our findings is underpinned by the large sample size, allowing for meaningful statistical analyses and generalizability of results to other memory clinic settings. Furthermore, the lack of influence from sex and education on 3O3P scores supports the broad applicability of this test across diverse populations. Finally, the inclusion of A/T/N biomarkers assessed through established clinical gold standard measures enhances the validity and clinical relevance of our study findings.
5. Conclusion
The 3O3P test emerges as a valid, efficient, and ecologically relevant screening tool for identifying early episodic memory deficits associated with AD. Its simplicity and cultural neutrality make it adaptable to various healthcare settings, including primary care and memory clinics. The high specificity of the 3O3P test underscores its utility in prompting further diagnostic evaluations, including comprehensive cognitive and biomarker assessments. Additionally, its brief administration time and minimal equipment requirements enhance its accessibility, particularly in resource‐limited environments. Future research should explore the longitudinal utility of the 3O3P test in monitoring cognitive trajectories and its potential integration into digital platforms for remote cognitive assessments. Digital adaptations, such as mobile health applications or virtual reality simulations, could further extend its strength, providing innovative approaches to early detection and intervention. These advancements hold promise for improving the accessibility and effectiveness of cognitive health services across diverse populations.
Author Contributions
Federica Ribaldi: conceptualization, methodology, data curation, investigation, validation, formal analysis, project administration, writing – review and editing, writing – original draft. Sophie Krug: writing – original draft, data curation, formal analysis, writing – review and editing. Daniele Altomare: conceptualization, methodology, writing – original draft, writing – review and editing. Valentina Garibotto: conceptualization, data curation, investigation, writing – review and editing. Max Scheffler: investigation, data curation, writing – review and editing. Augusto J. Mendes: data curation, formal analysis, writing – review and editing. Aurelien Lathuiliere: methodology, writing – review and editing, supervision. Frederic Assal: data curation, investigation, writing – review and editing. Aldara Vazquez Fernandez: writing – review and editing, data curation. Stefano F. Cappa: methodology, supervision, writing – review and editing. Christian Chicherio: conceptualization, methodology, data curation, writing – review and editing. Giovanni B. Frisoni: conceptualization, methodology, data curation, investigation, writing – review and editing, supervision.
Consent
All participants signed an informed consent form prior to enrollment in the study. The Geneva Ethics Committee approved the study (IDs of the ethics approvals: PB_2016‐01346 and 2020_00403).
Conflicts of Interest
V.G. received research support and speaker fees through her institution from GE Healthcare, Siemens Healthineers, Novo Nordisk, and Janssen.
Acknowledgments
The Clinical Research Center at Geneva University Hospital and Faculty of Medicine provides valuable support for regulatory submissions and data management, and the Biobank at Geneva University Hospital for biofluid processing and storage. The authors thank Avid Radiopharmaceuticals Inc. for providing the 18F‐Flortaucipir tracer without being involved in the data analysis or interpretation. We thank Prof. Orazio Zanetti for supporting the use of the Three‐Objects‐Three‐Places test in clinical practice.
Funding: This study was supported by the Centre de la mémoire is funded by the following private donors under the supervision of the Private Foundation of Geneva University Hospitals: A.P.R.A. – Association Suisse pour la Recherche sur la Maladie d'Alzheimer, Genève; Fondation Segré, Genève; Race Against Dementia Foundation, London, UK; Fondation Child Care, Genève; Fondation Edmond J. Safra, Genève; Fondation Minkoff, Genève; Fondazione Agusta, Lugano; McCall Macbain Foundation, Canada; Nicole et René Keller, Genève; Fondation AETAS, Genève. Competitive research projects have been funded by: H2020 (projects n. 667375), Innovative Medicines Initiative (IMI contract n. 115736 and 115952), IMI2, Swiss National Science Foundation (projects n. 320030_182772 and n. 320030_169876), and Velux Stiftung. The authors thank Avid Radiopharmaceuticals Inc. for providing the 18F‐Flortaucipir tracer without being involved in the data analysis or interpretation. FR is supported by the Swiss National Science Foundation. VG was supported by the Swiss National Science Foundation (projects 320030_169876, 320030_185028 and IZSEZ0_188355), by the Velux Stiftung, by the Schmidheiny Foundation, and by the Aetas Foundation.
Federica Ribaldi and Sophie Krug contributed equally to this work.
Data Availability Statement
Data included in this paper will be made available upon reasonable request to the Principal Investigator (Prof. Giovanni B Frisoni). Federica Ribaldi and Giovanni B. Frisoni have full responsibility for the data, the analyses and interpretation, and the conduct of the research; that they have full access to all of the data; and that they have the right to publish any and all data.
References
- 1. Gallagher M. and Koh M. T., “Episodic Memory on the Path to Alzheimer's Disease,” Current Opinion in Neurobiology 21, no. 6 (2011): 929–934, 10.1016/j.conb.2011.10.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Pozueta A., Rodríguez‐Rodríguez E., Vazquez‐Higuera J. L., et al., “Detection of Early Alzheimer's Disease in MCI Patients by the Combination of MMSE and an Episodic Memory Test,” BMC Neurology 11 (2011): 11, 10.1186/1471-2377-11-78. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Freitas S., Simões M. R., Alves L., and Santana I., “Montreal Cognitive Assessment: Validation Study for Mild Cognitive Impairment and Alzheimer Disease,” Alzheimer Disease and Associated Disorders 37, no. 1 (2013): 27–43, https://journals.lww.com/alzheimerjournal/fulltext/2013/01000/montreal_cognitive_assessment__validation_study.6.aspx. [DOI] [PubMed] [Google Scholar]
- 4. Greene J. D. W., Baddeley A. D., and Hodges J. R., “Analysis of the Episodic Memory Deficit in Early Alzheimer's Disease: Evidence From the Doors and People Test,” Neuropsychologia 34, no. 6 (1996): 537–551, 10.1016/0028-3932(95)00151-4. [DOI] [PubMed] [Google Scholar]
- 5. Pinto T. C. C., Machado L., Bulgacov T. M., et al., “Is the Montreal Cognitive Assessment (MoCA) Screening Superior to the Mini‐Mental State Examination (MMSE) in the Detection of Mild Cognitive Impairment (MCI) and Alzheimer's Disease (AD) in the Elderly?,” International Psychogeriatrics 31, no. 4 (2019): 491–504, 10.1017/S1041610218001370. [DOI] [PubMed] [Google Scholar]
- 6. Brodaty H. and Moore C. M., “The Clock Drawing Test for Dementia of the Alzheimer's Type: A Comparison of Three Scoring Methods in a Memory Disorders Clinic,” International Journal of Geriatric Psychiatry 12, no. 6 (1997): 619–627. [PubMed] [Google Scholar]
- 7. Borson S., Scanlan J., Brush M., Vitaliano P., and Dokmak A., “The Mini‐Cog: A Cognitive “Vital Signs” Measure for Dementia Screening in Multi‐Lingual Elderly,” International Journal of Geriatric Psychiatry 15, no. 11 (2000): 1021–1027, . [DOI] [PubMed] [Google Scholar]
- 8. Brooke P. and Bullock R., “Validation of a 6 Item Cognitive Impairment Test With a View to Primary Care Usage,” International Journal of Geriatric Psychiatry 14, no. 11 (1999): 936–940. [PubMed] [Google Scholar]
- 9. de Jager C. A., Budge M. M., and Clarke R., “Utility of TICS‐M for the Assessment of Cognitive Function in Older Adults,” International Journal of Geriatric Psychiatry 18, no. 4 (2003): 318–324, 10.1002/gps.830. [DOI] [PubMed] [Google Scholar]
- 10. Hodges J., “The Handbook of Memory Disorders, 2nd Edition,” Journal of Neurology, Neurosurgery, and Psychiatry 75, no. 1 (2004): 173. [Google Scholar]
- 11. Wilson B., Cockburn J., Baddeley A. D., and Hiorns R., “The Development and Validation of a Test Battery for Detecting and Monitoring Everyday Memory Problems,” Journal of Clinical and Experimental Neuropsychology 11, no. 6 (1989): 855–870, 10.1080/01688638908400940. [DOI] [PubMed] [Google Scholar]
- 12. Prestia A., Rossi R., Geroldi C., et al., “Validation Study of the Three‐Objects‐Three‐Places Test: A Screening Test for Alzheimer's Disease,” Experimental Aging Research 32, no. 4 (2006): 395–410, 10.1080/03610730600875767. [DOI] [PubMed] [Google Scholar]
- 13. Tenney Y. J., “Ageing and the Misplacing of Objects,” British Journal of Developmental Psychology 2, no. 1 (1984): 43–50, 10.1111/j.2044-835X.1984.tb00533.x. [DOI] [Google Scholar]
- 14. Ribaldi F., Chicherio C., Altomare D., et al., “Brain Connectivity and Metacognition in Persons With Subjective Cognitive Decline (COSCODE): Rationale and Study Design,” Alzheimer's Research & Therapy 13 (2021): 105, 10.1186/s13195-021-00846-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Rhodius‐Meester H. F. M., Benedictus M. R., Wattjes M. P., et al., “MRI Visual Ratings of Brain Atrophy and White Matter Hyperintensities Across the Spectrum of Cognitive Decline Are Differently Affected by Age and Diagnosis,” Frontiers in Aging Neuroscience 9 (2017): 117, 10.3389/fnagi.2017.00117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Boccalini C., Ribaldi F., Hristovska I., et al., “The Impact of Tau Deposition and Hypometabolism on Cognitive Impairment and Longitudinal Cognitive Decline,” Alzheimers Dement 20, no. 1 (2024): 221–233, 10.1002/alz.13355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Peretti D. E., Ribaldi F., Scheffler M., Chicherio C., Frisoni G. B., and Garibotto V., “Prognostic Value of Imaging‐Based ATN Profiles in a Memory Clinic Cohort,” European Journal of Nuclear Medicine and Molecular Imaging 50 (2023): 3313–3323, 10.1007/s00259-023-06311-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Guedj E., Varrone A., Boellaard R., et al., “EANM Procedure Guidelines for Brain PET Imaging Using [(18)F]FDG, Version 3,” European Journal of Nuclear Medicine and Molecular Imaging 49, no. 2 (2022): 632–651, 10.1007/s00259-021-05603-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Altomare D., Caprioglio C., Assal F., et al., “Diagnostic Value of Amyloid‐PET and Tau‐PET: A Head‐To‐Head Comparison,” European Journal of Nuclear Medicine and Molecular Imaging 48, no. 7 (2021): 2200–2211, 10.1007/s00259-021-05246-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Cohen J., “Statistical Power Analysis for the Behavioral Sciences,” Technometrics 31 (1989): 499–500. [Google Scholar]
- 21. Larner A. J., “Effect Size (Cohen's d) of Cognitive Screening Instruments Examined in Pragmatic Diagnostic Accuracy Studies,” Dementia and Geriatric Cognitive Disorders Extra 4, no. 2 (2014): 236–241, 10.1159/000363735. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Larner A. J., Cognitive Screening Instruments: A Practical Approach, 2nd ed., ed. Larner A. J. (Springer Cham, 2017), 10.1097/00005053-198602000-00002. [DOI] [Google Scholar]
- 23. Cerami C., Dubois B., Boccardi M., Monsch A. U., Demonet J. F., and Cappa S. F., “Clinical Validity of Delayed Recall Tests as a Gateway Biomarker for Alzheimer's Disease in the Context of a Structured 5‐Phase Development Framework,” Neurobiology of Aging 52 (2017): 153–166, 10.1016/j.neurobiolaging.2016.03.034. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data included in this paper will be made available upon reasonable request to the Principal Investigator (Prof. Giovanni B Frisoni). Federica Ribaldi and Giovanni B. Frisoni have full responsibility for the data, the analyses and interpretation, and the conduct of the research; that they have full access to all of the data; and that they have the right to publish any and all data.