Abstract
Aims
The present study set out to evaluate the psychometric properties of the Cognitive Telephone Screening Instrument (COGTEL) in 2 different samples of older adults.
Methods
We assessed COGTEL in 116 older adults, with retest after 7 days to evaluate the test-retest reliability. Moreover, we assessed COGTEL in 868 older adults to evaluate convergent validity to the Mini-Mental State Examination (MMSE).
Results
Test-retest reliability of the COGTEL total score was good at 0.85 (p < 0.001). Latent variable analyses revealed that COGTEL and MMSE correlated by 0.93 (p < 0.001), indicating convergent validity of the COGTEL.
Conclusion
The present analyses suggest COGTEL as a brief, reliable, and valid instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies, with the advantage of covering more cognitive domains than traditional screening tools such as the MMSE, as well as differentiating between individual performance levels, in healthy older adults.
Keywords: Assessment of cognitive functioning, Interindividual differences, Older adults
Introduction
It is a major target of many epidemiological and aging studies to better understand the pathways through which interindividual differences in vulnerability to cognitive decline emerge, which has relevance for the health and well-being of older adults [1, 2]. However, study protocols in most studies often only allow the application of a brief measure to screen for cognitive functioning. That is why only the Mini-Mental State Examination (MMSE [3]) has been included in many epidemiological and aging studies [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]. However, focusing only on the MMSE may present a limitation for studies addressing healthy aging in cognitive functioning, since within the healthy range of cognitive functioning the MMSE cannot differentiate well between individual performance levels, leading to ceiling effects, which makes it difficult to evaluate interindividual differences in cognitive functioning [15, 16].
The Cognitive Telephone Screening Instrument (COGTEL) is a test battery that may be a useful alternative to the MMSE as it allows the detailed assessment of performance in 6 cognitive domains (prospective, short-term, long-term, and working memory, verbal fluency, and inductive reasoning – using tests adopted from well-established neuropsychological instruments such as the Wechsler scales), including an additional total score that is indicative of overall cognitive functioning [17]. Importantly, the COGTEL can be widely applied since it differentiates between individuals on the full performance range of adult cognitive functioning. It is not limited to cognitive impairments but is also indicative of interindividual differences in cognitive functioning within the healthy performance range [17, 18]. Moreover, the COGTEL is flexible in its administration since it can be applied in a face-to-face session as well as over the phone, and takes only about 10–15 min. With these advantages, the COGTEL has been optimized to be a detailed measure of interindividual differences in cognitive functioning that can be included in epidemiological and aging studies [2, 17, 18, 19].
In terms of psychometric properties, the validity of phone-based versus face-to-face application of the COGTEL subtests has been demonstrated [17]. However, to date, an evaluation of the reliability and validity of the COGTEL to external measures of global cognitive functioning (such as the MMSE) is missing. This would be important in order to investigate the suitability of the COGTEL as an instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies. Therefore, the present study set out to evaluate the psychometric properties of the COGTEL in terms of its test-retest reliability and convergent validity to the MMSE in 2 different samples of older adults.
Methods
Participants
We analyzed data from 2 different (nonoverlapping) samples of older adults. First, we evaluated the test-retest reliability of the COGTEL in a sample of 116 community-dwelling older adults from Apuí, Fonte Boa, Manaus, and Tonantis, Brazil (45 men and 71 women). The mean age was 68.3 years (SD = 7.1, range: 60–88). The interval between test and retest was 7 days. Second, we evaluated the convergent validity of the COGTEL to the MMSE in a sample of 868 community-dwelling older adults from the same cities (361 men and 507 women). The mean age was 70.1 years (SD = 6.8, range: 57–92). In both samples, the participants were volunteers recruited via advertisements distributed through newspapers, local radio, churches, and senior centers. As we explicitly aimed to take the full range of interindividual differences available in the cognitive status of participants into account, no inclusion/exclusion criteria based on MMSE scores were applied. All participants gave informed consent. The present study was conducted in accordance with the Declaration of Helsinki and was approved by the local ethics commission.
Materials
Cognitive Telephone Screening Instrument
We used the COGTEL to assess cognitive functioning. The COGTEL consists of 6 subtests covering prospective memory (0 or 1 point), verbal short- and long-term memory (0–8 points each), working memory (0–12 points), verbal fluency (0 to unlimited; as many words as the participant can name within 1 min), and inductive reasoning (0–8 points; see Kliegel et al. [17] for a more detailed description). The scores of the 6 subtests can be analyzed individually or combined into a weighted total score (7.2 × prospective memory + 1.0 × verbal short-term memory + 0.9 × verbal long-term memory + 0.8 × working memory + 0.2 × verbal fluency + 1.7 × inductive reasoning score) [17].
Mini-Mental State Examination
We used the MMSE [3] as an indicator of the amount of cognitive impairment. The MMSE consists of 5 subsections covering orientation (0–10 points), immediate and delayed free recall (0–3 points each), counting backwards (0–5 points), and language (0–9 points). A total score can be derived by adding the 5 subsection scores.
Procedure
Participants were individually tested in face-to-face sessions by study personnel specifically trained in the application of the COGTEL and the MMSE.
Statistical Analyses
We evaluated the reliability and validity of the COGTEL as follows. First, we inspected the test-retest reliability of the COGTEL total score (as well as that of the 6 COGTEL subtests separately) in the sample of 116 older adults. Second, we evaluated the convergent validity by inspecting the relation of the COGTEL total score to the MMSE total score in the sample of 868 older adults. Third, using latent variable analysis, we investigated this relation in the latter sample in more detail. Specifically, we modeled a latent COGTEL variable based on the 6 COGTEL subtests as indicators and a latent MMSE variable based on the 5 MMSE subsections as indicators and investigated the correlation between the latent COGTEL and the latent MMSE variable.
Results
Descriptive Statistics
Means and standard deviations in the sample of 116 participants for the 6 COGTEL subtests and the COGTEL total score are displayed in Table 1. Means and standard deviations in the sample of 868 participants for the 6 COGTEL subtests and the COGTEL total score, as well as for the 5 MMSE subsections and the MMSE total score, are displayed in Table 2.
Table 1.
Descriptive statistics and test-retest reliability in the sample of 116 older adults
| Variable |
Test |
Retest |
Reliability | ||
|---|---|---|---|---|---|
| mean | SD | mean | SD | r | |
| COGTEL total score | 13.56 | 7.51 | 15.28 | 8.95 | 0.85*** |
| Prospective memory | 0.25 | 0.43 | 0.26 | 0.44 | 0.75*** |
| Short-term memory | 2.91 | 1.64 | 3.68 | 2.10 | 0.71*** |
| Long-term memory | 3.32 | 2.08 | 3.80 | 2.43 | 0.77*** |
| Working memory | 2.48 | 1.94 | 2.67 | 2.06 | 0.81*** |
| Verbal fluency | 11.95 | 8.24 | 13.09 | 8.64 | 0.90*** |
| Reasoning | 0.87 | 1.23 | 0.91 | 1.23 | 0.60*** |
Means and standard deviations as well as test-retest reliability for Cognitive Telephone Screening Instrument (COGTEL) total score and the 6 COGTEL subtests (raw scores) in the sample of 116 older adults.
p < 0.001.
Table 2.
Descriptive statistics in the sample of 868 older adults
| Variable | Mean | SD |
|---|---|---|
| COGTEL | ||
| Prospective memory | 0.28 | 0.45 |
| Short-term memory | 3.74 | 2.03 |
| Long-term memory | 3.97 | 2.36 |
| Working memory | 3.15 | 2.61 |
| Verbal fluency | 12.71 | 7.92 |
| Reasoning | 1.87 | 1.82 |
| COGTEL total score | 17.59 | 9.59 |
| MMSE | ||
| Orientation | 8.27 | 1.90 |
| Immediate recall | 2.94 | 0.29 |
| Delayed recall | 2.10 | 1.03 |
| Counting backwards | 2.86 | 2.16 |
| Language | 7.45 | 1.23 |
| MMSE total score | 23.62 | 4.58 |
Means and standard deviations for the 6 Cognitive Telephone Screening Instrument (COGTEL) subtests (raw scores) and COGTEL total score as well as for the 5 Mini-Mental State Examination (MMSE) subsections (raw scores) and MMSE total score in the sample of 868 older adults.
Test-Retest Reliability of the COGTEL Instrument
We observed good test-retest reliability for the COGTEL total score. For the 6 COGTEL subtests, the test-retest reliability ranged from acceptable to good (see Table 1).
Convergent Validity of the COGTEL Instrument
There was a substantial correlation of the COGTEL total score with the MMSE total score (r = 0.65, p < 0.001). To evaluate this relation in more detail, we investigated the correlation between a latent COGTEL and a latent MMSE variable. First, we applied a model in which the latent COGTEL variable and the latent MMSE variable were allowed to correlate freely (see Fig. 1). This model provided a good account of the data (χ2 = 22.14, df = 24, p = 0.571, comparative fit index >0.99, incremental fit index >0.99, root mean square error of approximation <0.001, standardized root mean square residual = 0.02). In this model, the latent COGTEL and MMSE variables were significantly correlated (r = 0.93, p < 0.001).
Fig. 1.
Correlation between the latent Cognitive Telephone Screening Instrument (COGTEL) and the latent Mini-Mental State Examination (MMSE) variables. All coefficients are standardized. *** p < 0.001.
In a next step, we contrasted this model with 2 alternative models and statistically tested the change in model fit (Δχ2 and its significance). First, we applied an alternative model in which the correlation between the latent COGTEL and MMSE variables was constrained to be equal to zero, therefore representing 2 completely independent variables. Compared to the initial model, this alternative model provided a significantly worse model fit (Δχ2 = 601.71, Δdf = 1, p < 0.001), indicating that in the initial model the latent COGTEL and MMSE variables shared a substantial part of their interindividual variance.
Second, we applied an alternative model in which the correlation between the latent COGTEL and MMSE variables was constrained to be equal to 1, therefore sharing all of their interindividual variance. Compared to the initial model, this alternative model provided a significantly worse model fit (Δχ2 = 4.06, Δdf = 1, p = 0.044), indicating that in the initial model there was a substantial part of interindividual variance that the latent COGTEL and MMSE variables did not have in common.
Discussion
The present study set out to evaluate the reliability and validity of the COGTEL in 2 different samples of older adults. First, the test-retest reliability was of good size for the COGTEL total score and ranged from acceptable to good size for the 6 COGTEL subtests. Thus, the observed COGTEL test-retest reliability was comparable to that of the adopted Wechsler scales (ranging from 0.38 to 0.87, e.g., as reported for young and middle-aged adults [20, 21]). The test-retest reliability of the COGTEL total score was also within the range of the test-retest reliability of the MMSE total score (commonly between 0.80 and 0.95 [22]). This indicates that the COGTEL total score and the COGTEL subtests can be reliably used for the assessment of cognitive functioning.
Second, we observed a substantial correlation of the COGTEL total score with the MMSE total score, indicating convergent validity of the COGTEL instrument. Specifically, both instruments shared about half of their interindividual variance, with the rest being residual variance. The latter suggests that, compared to the MMSE, the COGTEL may provide additional information on interindividual differences in cognitive functioning.
Third, we evaluated this issue in latent variable analyses in more detail. Model comparisons indicated that COGTEL and MMSE share a substantial part of interindividual variance (again indicating convergent validity of the COGTEL instrument at a latent level), but that there is a substantial part of interindividual variance that COGTEL and MMSE do not have in common. The latter may be due to the fact that the COGTEL allows a more fine-grained differentiation between individuals within the healthy range of functioning in a broad set of cognitive domains [17, 18]. This may be because the COGTEL subtests place a much greater demand on memory and executive control [17], for which a large performance range can be captured in healthy aging using COGTEL [2, 18, 19]. For example, the COGTEL requires a greater amount of information that needs to be retained in short- and long-term memory (recall of 8 word pairs, of which 4 are semantically unrelated, in COGTEL vs. recall of 3 words in MMSE), it places a greater load on working memory (recall 12 series of up to 7 digits in reverse order in COGTEL vs. count 5 times backwards in MMSE), and it involves a much larger demand on executive control in the applied language tests (generating words according to prespecified rules in COGTEL vs. basic evaluation of understanding, reading, and writing in MMSE) [17, 18]. Besides that, the COGTEL assesses 2 abilities that are not covered in the MMSE, namely, prospective memory (remembering to say one's year of birth later when encountering the verbal fluency test) and inductive reasoning (completing 8 number sequences according to a mathematical rule that the participant needs to detect him-/herself), which place a great demand on executive control [17, 23, 24]. This further underlines that the COGTEL can add valuable information regarding interindividual differences in cognitive functioning, which is the key outcome variable in many epidemiological and aging studies with the aim of better understanding the pathways through which interindividual differences in vulnerability to cognitive decline emerge. Thereby, particularly in studies addressing healthy aging in cognitive functioning, using the COGTEL could overcome the limitation of using only the MMSE, which cannot differentiate well between individual performance levels in healthy aging and leads to ceiling effects [15, 16].
Thus, given that the COGTEL differentiates between individuals on the full performance range of functioning in a broad set of cognitive domains [17, 18], and given its quick and flexible assessment (in a face-to-face session or alternatively over the phone), we propose the COGTEL as a brief, reliable, and valid instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies.
Disclosure Statement
The authors declare that there is no conflict of interest.
Acknowledgments
The authors are grateful to Duarte L. Freitas and Jefferson Jurema for their help in setting up the study as well as to Maria A. Tinôco, Floramara T. Machado, Angenay P. Odim, Bárbara R. Muniz, and Alex B. Lima for technical assistance in the data collection and management. We are especially grateful to the older people for their participation and interest. A. Ihle and M. Kliegel belong to the Swiss National Center of Competences in Research LIVES – Overcoming Vulnerability: Life Course Perspectives – which is financed by the Swiss National Science Foundation (grant No. 51NF40-160590). The authors are grateful to the Swiss National Science Foundation for its financial assistance. We are also grateful for the support from the City Hall of Apuí – Secretary of Health, Amazonas, Brazil.
References
- 1.Lawton MP, Moss M, Hoffman C, Grant R, Have TT, Kleban MH. Health, valuation of life, and wish to live. Gerontologist. 1999;32:406–416. doi: 10.1093/geront/39.4.406. [DOI] [PubMed] [Google Scholar]
- 2.Perna L, Mons U, Rujescu D, Kliegel M, Brenner H. Apolipoprotein E e4 and cognitive function: a modifiable association? Results from two independent cohort studies. Dement Geriatr Cogn Disord. 2016;32:35–45. doi: 10.1159/000440697. [DOI] [PubMed] [Google Scholar]
- 3.Folstein MF, Folstein SE, McHugh PR. Mini-Mental State – A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;32:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- 4.Yoon JA, Kim DY, Sohn MK, Lee J, Lee SG, Lee YS, Han EY, Joo MC, Oh GJ, Han J, Park M, Park KP, Noh KH, Chang WH, Shin YI, Kim YH. Factors associated with improvement or decline in cognitive function after an ischemic stroke in Korea: the Korean stroke cohort for functioning and rehabilitation (KOSCO) study. BMC Neurol. 2017;17:9. doi: 10.1186/s12883-016-0780-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Liu J, Huang YL, Chen GJ, Liu XX, Wang ZJ, Cao YB, Li HT, Song L, Li CH, Zhao HL, Chen SH, Wang YM, Zhang RY, Wang AX, Wu SL. Cumulative systolic blood pressure exposure in relation to cognitive function in middle-aged and elderly adults: a prospective, population-based study. Medicine (Baltimore) 2016;95:e5514. doi: 10.1097/MD.0000000000005514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Matchar DB, Chei CL, Yin ZX, Koh V, Chakraborty B, Shi XM, Zeng Y. Vitamin D levels and the risk of cognitive decline in Chinese elderly people: the Chinese Longitudinal Healthy Longevity Survey. J Gerontol A Biol Sci Med Sci. 2016;32:1363–1368. doi: 10.1093/gerona/glw128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Nishiguchi S, Yamada M, Shirooka H, Nozaki Y, Fukutani N, Tashiro Y, Hirata H, Yamaguchi M, Tasaka S, Matsushita T, Matsubara K, Tsuboyama T, Aoyama T. Sarcopenia as a risk factor for cognitive deterioration in community-dwelling older adults: a 1-year prospective study. J Am Med Dir Assoc. 2016;17:372.e5–8. doi: 10.1016/j.jamda.2015.12.096. [DOI] [PubMed] [Google Scholar]
- 8.Kawasaki M, Arai Y, Takayama M, Hirata T, Takayama M, Abe Y, Niimura H, Mimura M, Takebayashi T, Hirose N. Carotid atherosclerosis, cytomegalovirus infection, and cognitive decline in the very old: a community-based prospective cohort study. Age. 2016;38:29. doi: 10.1007/s11357-016-9890-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hong T, Mitchell P, Burlutsky G, Liew G, Wang JJ. Visual impairment, hearing loss and cognitive function in an older population: longitudinal findings from the Blue Mountains Eye Study. PLoS One. 2016;11:e0147646. doi: 10.1371/journal.pone.0147646. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Moreira A, Diogenes MJ, de Mendonca A, Lunet N, Barros H. Chocolate consumption is associated with a lower risk of cognitive decline. J Alzheimers Dis. 2016;32:85–93. doi: 10.3233/JAD-160142. [DOI] [PubMed] [Google Scholar]
- 11.Veronese N, Stubbs B, Trevisan C, Bolzetta F, de Rui M, Solmi M, Sartori L, Musacchio E, Zambon S, Perissinotto E, Crepaldi G, Manzato E, Sergi G. What physical performance measures predict incident cognitive decline among intact older adults? A 4.4-year follow-up study. Exp Gerontol. 2016;32:110–118. doi: 10.1016/j.exger.2016.05.008. [DOI] [PubMed] [Google Scholar]
- 12.Auyeung TW, Lee JSW, Kwok T, Woo J. Physical frailty predicts future cognitive decline – a four-year prospective study in 2,737 cognitively normal older adults. J Nutr Health Aging. 2011;32:690–694. doi: 10.1007/s12603-011-0110-9. [DOI] [PubMed] [Google Scholar]
- 13.Takata Y, Ansai T, Soh I, Kimura Y, Yoshitake Y, Sonoki K, Awano S, Kagiyama S, Yoshida A, Nakamichi I, Hamasaki T, Torisu T, Toyoshima K, Takehara T. Physical fitness and cognitive function in an 85-year-old community-dwelling population. Gerontology. 2008;32:354–360. doi: 10.1159/000129757. [DOI] [PubMed] [Google Scholar]
- 14.Raji MA, Kuo YF, Al Snih S, Markides KS, Peek MK, Ottenbacher KJ. Cognitive status, muscle strength, and subsequent disability in older Mexican Americans. J Am Geriatr Soc. 2005;32:1462–1468. doi: 10.1111/j.1532-5415.2005.53457.x. [DOI] [PubMed] [Google Scholar]
- 15.Franco-Marina F, Garcia-Gonzalez JJ, Wagner-Echeagaray F, Gallo J, Ugalde O, Sanchez-Garcia S, Espinel-Bermudez C, Juarez-Cedillo T, Rodriguez MAV, Garcia-Pena C. The Mini-Mental State Examination revisited: ceiling and floor effects after score adjustment for educational level in an aging Mexican population. Int Psychogeriatr. 2010;32:72–81. doi: 10.1017/S1041610209990822. [DOI] [PubMed] [Google Scholar]
- 16.Lopez M, Charter R, Mostafavi B, Nibut L, Smith W. Psychometric properties of the Folstein Mini-Mental State Examination. Assessment. 2005;32:137–144. doi: 10.1177/1073191105275412. [DOI] [PubMed] [Google Scholar]
- 17.Kliegel M, Martin M, Jäger T. Development and validation of the Cognitive Telephone Screening Instrument (COGTEL) for the assessment of cognitive function across adulthood. J Psychol. 2007;32:147–170. doi: 10.3200/JRLP.141.2.147-172. [DOI] [PubMed] [Google Scholar]
- 18.Breitling LP, Wolf M, Müller H, Raum E, Kliegel M, Brenner H. Large-scale application of a telephone-based test of cognitive functioning in older adults. Dement Geriatr Cogn Disord. 2010;32:309–316. doi: 10.1159/000319896. [DOI] [PubMed] [Google Scholar]
- 19.Perna L, Mons U, Kliegel M, Brenner H. Serum 25-hydroxyvitamin D and cognitive decline: a longitudinal study among non-demented older adults. Dement Geriatr Cogn Disord. 2014;32:254–263. doi: 10.1159/000362870. [DOI] [PubMed] [Google Scholar]
- 20.Karakas S, Kafadar H, Eski R. Test-retest reliability of the Turkish standardization of Wechsler memory scale-revised. Turk Psikoloji Dergisi. 1996;32:46–55. [Google Scholar]
- 21.Lo AHY, Humphreys M, Byrne GJ, Pachana NA. Test-retest reliability and practice effects of the Wechsler Memory Scale-III. J Neuropsychol. 2012;32:212–231. doi: 10.1111/j.1748-6653.2011.02023.x. [DOI] [PubMed] [Google Scholar]
- 22.Tombaugh TN, McIntyre NJ. The Mini-Mental State Examination: a comprehensive review. J Am Geriatr Soc. 1992;32:922–935. doi: 10.1111/j.1532-5415.1992.tb01992.x. [DOI] [PubMed] [Google Scholar]
- 23.Kliegel M, Ballhausen N, Hering A, Ihle A, Schnitzspahn KM, Zuber S. Prospective memory in older adults: where we are now and what is next. Gerontology. 2016;32:459–466. doi: 10.1159/000443698. [DOI] [PubMed] [Google Scholar]
- 24.Reverberi C, Lavaroni A, Gigli GL, Skrap M, Shallice T. Specific impairments of rule induction in different frontal lobe subgroups. Neuropsychologia. 2005;32:460–472. doi: 10.1016/j.neuropsychologia.2004.06.008. [DOI] [PubMed] [Google Scholar]