Differences in the measurement of cognition for the assessment of dementia across geographic contexts: Recommendations for cross-national research

Abstract

Introduction:

Most cognitive assessments have been developed in high-income countries but are used in diverse contexts. Differences in culture and context may affect the performance of cognitive items.

Methods:

We used the Harmonized Cognitive Assessment Protocol (HCAP) surveys in the United States, Mexico, India, England, and South Africa (combined N = 11,364) to quantify associations across countries between cognitive items and cognitive impairment status using age- and sex-adjusted logistic regression.

Results:

Associations were stronger in the United States (median odds ratio [OR] across items = 0.17) and England (median OR = 0.19), compared to South Africa (median OR = 0.23), India (median OR = 0.29), and Mexico (median OR = 0.28). Items assessing memory (e.g., delayed recall tasks) had the most consistent associations of the largest magnitudes across contexts.

Discussion:

Transporting cognitive items among countries and cultures warrants caution. Our results can guide the design of future instruments by identifying items that performed well either in individual contexts or across the range of contexts considered.

1 |. BACKGROUND

It is expected that 71% of individuals living with dementia will reside in low- and middle-income countries by 2050.¹ However, most dementia research conducted to date has taken place in high-income countries.² Research in diverse geographic settings can inform our understanding of the distribution of disease burden, raise awareness of dementia in contexts where this may be lacking, and can guide policy decisions, resource allocation, and public health planning efforts. Cross-national research can also identify differences in the effects of modifiable risk factors, informing targeted prevention efforts. Furthermore, comparisons across countries with wider ranges of risk-factor profiles and larger variation in the causes and consequences of dementia may lead to new findings on modifiable risk factors or disease progression.

Recently, there has been increased attention on cross-national research focused on dementia and cognitive aging, spearheaded by large coordinated efforts such as the 10/66 Studies or the Harmonized Cognitive Assessment Protocol (HCAP) surveys.^3,4 The HCAP surveys represent one of the largest efforts to date to conduct comparable population-representative studies on dementia and cognitive aging across geographic contexts.⁴

Despite these efforts to conduct research in diverse geographical contexts, there is little available evidence to guide the design and implementation of cross-national studies on dementia. For example, the HCAP surveys leveraged evidence from the Aging, Demographics and Memory Study and the Religious Orders Study and Memory and Aging Project, two US-based cohorts, to guide selection of survey questions (items) on cognition for inclusion in the cross-national HCAP battery.^4–6 Other dementia studies in diverse settings, including in central Africa, Brazil, and China, have based item selection on expert opinion or prior work in other low-income settings without context-specific validation studies or other quantitative evidence.^7–9

However, demographic and cultural factors, such as language of test administration, sex/gender (sex), urbanicity, and race/ethnicity can impact performance on cognitive test items, holding underlying cognitive ability constant.^10–13 Many of these factors vary across geographies. Therefore, it is necessary to closely consider the utility of survey items selected for cross-national research; standardization of instruments may not be enough for valid and comparable measurement.

This study aims to provide concrete guidance for dementia measurement in future cross-national efforts through the evaluation of items on cognitive functioning for use in measuring and classifying dementia using the HCAP surveys. We will quantify differences and similarities across countries in associations between cognitive impairment and items on cognitive functioning to evaluate the utility of items for future research.

2 |. METHODS

2.1 |. Methods overview

The analytic plan had two main components: (1) classification of cognitive impairment and (2) evaluation of associations between cognitive impairment and items on cognition (Figure 1). Step 1 was required because HCAP studies did not include clinical evaluations for formal dementia diagnoses. Therefore, we used an actuarial neuropsychological norms approach to define impairment; this approach conceptualizes impairment as a discrepancy between cognitive performance and demographically adjusted norms.¹⁴

FIGURE 1.

Flow chart illustrating the analytic process used throughout the study. We first used confirmatory factor analysis (CFA) to estimate cognitive domain scores in each sample (Step 1). We then used the demographically corrected norms to define cognitive impairment in each sample using an actuarial neuropsychological approach (Steps 2–4). This process of estimating cognitive domain scores and classifying cognitive impairment was repeated 65 times (the total number of items), leaving out data on each item of interest in turn to prevent circularity in inferences from these analyses (Step 5). The leave one out impairment status for each item was then used in logistic regression analyses to assess the associations between cognitive items and cognitive impairment (Step 6). Boxes with rounded edges illustrate data or estimates, whereas boxes with hard edges illustrate analytical steps. Numbers included in the boxes show the order of steps

2.2 |. Harmonized Cognitive Assessment Protocol (HCAP) Surveys

The HCAP series aimed to assess cognitive aging and dementia cross-nationally in sub-samples from the larger Health and Retirement Study International Partner Surveys (HRS IPS). The HRS IPS surveys used multistage probability sampling to generate nationally representative (with the exception of South Africa) samples of adults in private households.^15–18 The South African HRS IPS is instead representative of the rural sub-district of Agincourt.¹⁹ HCAP sub-samples in the United States and Mexico randomly sampled eligible participants, whereas the other studies oversampled those with low levels of cognition. We used data from the baseline HCAP wave in the United States,⁴ Mexico,²⁰ India,²¹ England,²² and South Africa.¹⁹ Informed consent was obtained for all participants. Sample sizes ranged from 4096 in India to 606 in South Africa (combined N = 11,364). We excluded individuals with missing data on covariates (age, sex, education, race/ethnicity in the United States) or high levels of missingness in cognitive testing (greater than 50% missingness leading to poor reliability of scores in all cognitive domains), resulting in a final sample size of 11,250 (excluded N = 62 [US], 18 [England], 46 [South Africa], 1 [India], 56 [Mexico]) (details in the Appendix A).

2.3 |. Cognitive measures

Table 1 describes the full list of cognitive items and compares their inclusion across studies. Although collaborative efforts sought to ensure the highest possible concordance, some adaptations were necessary to accommodate different languages, cultures, and levels of numeracy and literacy.⁴ Items on memory had the highest overlap among studies, followed by items on orientation and language. Items on executive functioning had the least overlap. Assessments of visuospatial functioning were brief, but included at least one item in all studies.

TABLE 1.

Cognitive items administered by cognitive domain in each of the United States, England, South Africa, India, and Mexico Harmonized Cognitive Assessment Protocol (HCAP) samples

Cognitive Item	United States	England	South Africa	India	Mexico
Orientation
Day of the week	X	X	X	X	X
Day of month	X	X		X	X
Month	X	X		X	X
Season	X	X	X	X
Year	X	X		X	X
What time is it?					X
Where are we?					X
What city are we in?	X	X	X	X
What county are we in?	X	X
What province are we in?	X		X	X	X
What country are we in?		X			X
What floor are we on?	X			X
What street are we on?		X
What building are we in?		X		X
What address are we at?	X			X
Memory
CERAD immediate sum of three trials	X	X	X	X	X
CERADword list delay	X	X	X	X	X
CERAD recognition	X	X	X	X	X
Logical memory immediate	X	X	X	X	X
Logical memory delay	X	X	X	X	X
Logical memory recognition	X	X	X	X
Brave man delay	X	X		X	X
Brave man immediate	X	X		X	X
CERAD constructional praxis delay	X	X	X	X	X
MMSE 3-word immediate	X	X		X	X
MMSE 3-word delay	X	X		X	X
Executive functioning
Symbol/letter cancellation	X	X	X	X	X
Symbol digit modalities test	X	X	X		X
Serial threes					X
Serial sevens		X		X	X
MMSE spelling backwards	X
Backward day naming			X	X
Backward counting	X	X	X
Backward counting from 20					X
HRS number series	X	X
Digit span forward				X
Digit span backward				X
Trails A time	X	X	X
Trails B time	X	X	X
Ravens progressive matrices	X	X	X	X
Go-no-go				X	X
Token test				X
Similarities				X	X
Problem solving				X
Language
Animal fluency	X	X	X	X	X
TICS name cactus	X	X
TICS name coconut				X
TICS name scissors	X	X	X	X	X
TICS name prime minister	X	X	X	X
TICS name deputy president			X
CSI-D name elbow	X	X	X	X	X
CSI-D do with a hammer	X	X	X	X	X
CSI-D following instructions	X	X	X	X	X
CSI-D where is the local market?	X	X	X	X	X
Define bridge					X
MMSE naming (watch)	X	X		X
MMSE naming (pencil)	X	X		X	X
MMSE naming (shoe)					X
MMSE write/say a sentence	X	X		X	X
MMSE read and follow command	X	X		X	X
MMSE repetition of phrase	X	X		X	X
MMSE following instructions three step (paper)	X	X		X	X
Visuospatial functioning
CERAD constructional praxis (copy four figures)	X	X	X	X	X
MMSE polygons (copy one figure)	X			X	X
Clock drawing				X

Abbreviations: CERAD, Consortium to Establish a Registry for Alzheimer’s Disease; CSI-D, Community Screening Instrument for Dementia; HRS, Health and Retirement Study; MMSE, Mini-Mental State Examination; TICS, Telephone Interview for Cognitive Status.

2.4 |. Sociodemographic and health questions

We considered sociodemographic factors in HCAP studies based on cultural relevance and data availability. In the United States, we considered race and ethnicity. In India and Mexico we considered rurality, and in India and South Africa we also used literacy status. In all countries we dichotomized educational attainment based on the distribution in each study. To evaluate depressive symptomology, we considered all items administered from the Center for Epidemiologic Studies–Depression scale in each study (Appendix; Table S1).²³ Details on the definitions of these variables are in Appendix A. Finally, we used information on informant-reported stroke, Alzheimer’s disease, and memory problems from all HCAP studies with the exception of Mexico due to a lack of data availability. We additionally considered self-reported stroke and heart attack from the prior HRS IPS wave in all studies.

2.5 |. Step 1: Classification of cognitive impairment

We used an actuarial neuropsychological norms approach to classify cognitive impairment. This approach has three steps: (1) quantify cognitive functioning by cognitive domain; (2) define a normative sample of individuals unlikely to develop cognitive impairment; and (3) within basic demographic categories, compare cognitive scores between the normative sample and individual participants to define impairment. Previous work used similar methodology within the Mexico HCAP sample.²⁴ This process was completed independently within each HCAP study.

To quantify cognitive functioning by domain in each study, we used confirmatory factor analysis (CFA) models.²⁵ We estimated models for orientation, executive functioning, memory, and language. We were unable to estimate visuospatial functioning, as two studies included only one item assessing visuospatial functioning.

We used information on functional limitations and self-reported health to define a cognitively robust group in each study. Using multivariable linear regression, we estimated normative cognitive scores within demographic categories from data on participants from the cognitively robust group. Norms were estimated separately in each country. Demographic categories included an individual’s age, sex, and educational attainment (dichotomized). We further stratified norms by race and ethnicity in the United States, rurality in India and Mexico, and literacy in India and South Africa due to the relevance of these additional characteristics in each setting. To compare cognitive scores from the normative sample to scores from participants in the broader study samples within demographic categories, we calculated residual scores, which represent the difference between an individual’s cognitive performance and their expected cognitive performance based on the demographic characteristics. Individuals were defined as impaired if they had a residual score of less than 1.5 SD from demographically corrected norms in any cognitive domain.¹³ Individuals with missing scores on all cognitive domains were excluded (N = 36 across all studies). Details on CFA models (Table S2), reliability of cognitive scores (Figure S1), definitions of the cognitively robust group, and the calculation of residual scores are in Appendix A.

2.6 |. Step 2: Description of data and evaluation of associations between cognitive impairment and items on cognition and functional limitations

We characterized HCAP samples using descriptive statistics. We then assessed patterns of missing data and quantified variability of responses to binary cognitive items (the proportion answering items correctly).

For our primary analysis, we used weighted multiple logistic regression (details on survey weights in Appendix A) controlling for age and sex to quantify associations between cognitive impairment and each item on cognition. To ensure that effect sizes were comparable between binary and continuous items, we divided all non-binary items by two times the item’s SD.²⁶ Because each item on cognitive functioning also contributed to the classification of cognitive impairment, we used an iterative approach to avoid circularity. Specifically, to estimate the association between each cognitive item and cognitive impairment we re-calculated the classification of cognitive impairment (including re-estimating CFA models and re-calculating demographically adjusted norms) leaving out data on the item of interest. Although this procedure does lead to 65 different sets of classifications (one for each item of interest), differences between classification sets were minimal (details in Appendix A; Figure S2). When there were fewer than five individuals in any given item response category and impairment status combination, we did not estimate odds ratios due to model instability (details in Appendix A; Figure S4). To make direct comparisons of the effect sizes between different HCAP studies, we subtracted parameters on the log scale. We assumed additive variance for normally distributed parameter estimates to calculate the variance of differences. To summarize effect sizes either across items or countries, we used the median as a measure of central tendency to prevent outliers from having undue influence. We used histograms of estimated odds ratios to inspect differences in the distribution of associations across countries.

2.7 |. Sensitivity analyses

The United States and England studies included individuals 65 years and older; younger participants were included in other countries. Therefore, we conducted a sensitivity analysis where we subset data to individuals 65 and older across all studies to ensure that observed differences were not due to differences in age distributions of studies.

To test the sensitivity of results to the use of the neuropsychological norms approach for classification, we repeated primary analyses using latent class analysis as an alternative strategy for classification²⁷ (details in Appendix A).

3 |. RESULTS

3.1 |. Descriptive statistics

The mean age was higher in the United States (75.8, SD = 7.5) and England (75.9, SD = 7.1) in comparison to South Africa (69.2, SD = 11.1), India (69.0, SD = 7.6), and Mexico (68.1, SD = 9.0) (Table 2). Educational attainment was highest in the United States (28.7% with post-secondary education), and in England (13.0% with post-secondary education). In comparison, in South Africa, India, and Mexico, most participants had either no education or primary education only.

TABLE 2.

Characteristics of the United States, England, South Africa, India, and Mexico Harmonized Cognitive Assessment Protocol (HCAP) samples

	United States	England	South Africa	India	Mexico
Number of participants, N	3329	1255	560	4095	2011
Years of data collection	2016–2017	2018	2016–2017	2017–2019	2015
Age, mean (SD)	75.8 (7.5)	75.9 (7.1)	69.2 (11.1)	69.0 (7.6)	68.1 (9.0)
Percent female (N)	60.5% (2014)	54.9% (689)	56.2% (315)	53.9% (2207)	59.3% (1193)
No education – primary education, % (N)	18.2% (607)	33.1% (416)	92.7% (519)	75.3% (3085)	72.9% (1467)
Some secondary - completed secondary education, % (N)	53.0% (1766)	53.9% (676)	5.4% (30)	20.6% (845)	20.8% (419)
Post-secondary education, % (N)	28.7% (956)	13.0% (163)	2.0% (11)	4.0% (165)	6.2% (125)
White race, % (N)	78.9% (2627)
Black race, % (N)	16.0% (533)
Other race, % (N)	5.1% (169)
Percent Hispanic (N)	10.8% (360)
Percent Rural (N)				62.0% (2539)	28.3% (569)
Percent illiterate (N)			58.6% (328)	56.6% (2319)

3.2 |. Missingness for items on cognition

Missingness was less than 10% for almost all items in the United States and England, with the exceptions of the HRS Number Series in the United States and the Trail-Making Test Part B in England (Appendix A; Figure S3). Higher levels of missingness were observed in a larger number of items in Mexico (4 items), India (12 items), and South Africa (9 items). In South Africa and India, items on executive functioning had the highest levels of missingness, with 68% missingness on the Trail-Making Test Part B and 54% missingness on the Symbol Digit Modalities Test in South Africa and 44% missingness on the Serial 7’s test in India.

3.3 |. Associations for items on cognition

High performance (good or correct scores) on all cognitive items was associated negatively with cognitive impairment across all locations. However, there was substantial heterogeneity in the strength of the associations observed (Figure 2).

FIGURE 2.

Associations between each cognitive test item and cognitive impairment by domain for each Harmonized Cognitive Assessment Protocol Studies (HCAP) conducted in the United States (N = 3329), England (N = 1255), South Africa (N = 560), India (N = 4095), and Mexico (N = 2011) from logistic regression models, controlling for age and sex. Odds ratios are displayed for significant associations. For example, the number 0.14 in the top left-hand corner indicates that in the United States, those who answered the question on the day of the week correctly had an odds of cognitive impairment that was 0.14 times the odds of cognitive impairment for those who did not answer this question correctly. Gray boxes represent instances where an item was not administered or an odds ratio was suppressed due to small cells. Color scale shows differences in associations on the log odds scale

3.3.1 |. Memory

Across all settings, some of the items with the most consistently large associations with cognitive impairment tested memory performance, including the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) immediate sum of three trials (median odds ratio [OR] = 0.09; range = 0.07–0.17), the CERAD word list delay (median OR = 0.12; range = 0.09–0.20), and the logical memory delayed task (median OR = 0.13; range = 0.13–0.16).

3.3.2 |. Language

A number of items assessing language had low variability (most individuals answered correctly), suggesting that these items may only help in classifying a small number of individuals (Appendix A; Figure S5). In addition, several items, including the following instructions, do with a hammer, and naming the prime minister/president items, showed notable variation in estimated associations between countries. For example, the do with a hammer item from the Community Screening Instrument–Dementia (CSID) battery had a substantially stronger association with cognition in Mexico (OR = 0.14; 95% confidence interval [CI] 0.08–0.25) as compared to the United States (OR = 0.53; 0.41–0.70) or India (OR = 0.44; 0.38–0.51). Of the language items administered, the animal fluency task showed the most consistently strong relationship with cognitive impairment across each HCAP study (median OR = 0.19; range = 0.14–0.32).

3.3.3 |. Executive functioning

Of the items measuring executive functioning, only letter or symbol cancellation was administered across all HCAP studies, and it showed a fairly strong and consistent association with cognitive impairment in all locations (median OR = 0.17; range = 0.11–0.49); associations were weakest in England (OR = 0.49; 0.37–0.65) and South Africa (OR = 0.38; 0.22–0.63). Although the Symbol Digit Modalities Test was not administered in India, it also showed robust associations with cognitive impairment across the remaining countries (median OR = 0.18; range = 0.10–0.28). A number of items were administered in only one or two studies. The Token test and Problem solving test were administered in India only, but showed the strongest associations with cognitive functioning (Token test OR = 0.16, 0.13–0.19; Problem solving test OR = 0.15, 0.12–0.18) of executive functioning items administered in the India HCAP study.

3.3.4 |. Orientation

Similarly to items on language, the majority of orientation items had low variability (most individuals answered correctly), indicating that these items may help in classifying only a small proportion of individuals (Appendix A; Figure S6). Due to differences in the administration of orientation items across studies as well as low numbers of incorrect responses, which led to model instability and suppressed estimates, there were no orientation items with associations for all studies (Appendix A; Figure S5). Across the four samples evaluated (variability was too low in England to estimate an odds ratio), the item assessing the day of the week had the strongest and most consistent associations with cognitive impairment (median OR = 0.19; range = 0.14–0.29).

3.4 |. Overall patterns and sensitivity analyses

Across all items, associations between cognitive impairment and survey items were stronger in the United States (median OR, inter-quartile range [IQR] = 0.17, 0.13–0.32) and England (median OR, IQR = 0.19, 0.13–0.25), as compared to South Africa (median OR, IQR = 0.23, 0.18–0.35), India (median OR, IQR = 0.29, 0.22–0.33), and Mexico (median OR, IQR = 0.28, 0.15–0.35), although associations were meaningfully strong in all studies. These differences can additionally be visualized in terms of shifts in the distributions of estimated ORs between countries (Figure 3).

FIGURE 3.

Distributions of estimated odds ratios describing the association between items on cognition and cognitive impairment across Harmonized Cognitive Assessment Protocol Studies (HCAP) conducted in the United States (N = 3329), England (N = 1255), South Africa (N = 560), India (N = 4095), and Mexico (N = 2011) from logistic regression models, controlling for age and sex. Odds ratios farther from 1 represent stronger associations with cognitive impairment. There is a larger left tail in the distributions for the United States and England, and to a smaller extent, Mexico, indicating the presence of some items with stronger associations in these countries as compared to other settings

Subsetting to individuals over 65 had minimal effects on comparisons (Appendix A; Figure S8). Results from latent class analysis also broadly replicated the pattern of findings from primary analyses (Appendix A; Figures S6–S7).

4 |. DISCUSSION

This study evaluated patterns in associations between cognitive impairment and items assessing cognition across countries. We found substantial variability across HCAP studies, although the magnitude of variation was different across items. The observed heterogeneity suggests that the performance of items for classification purposes is not consistent across settings. In general, associations between cognitive impairment and items on cognition were strongest in the United States and England, as compared to South Africa, India, and Mexico. Many items in the HCAP battery were developed in high-income settings.^28–31 Associations between responses to these items and cognitive impairment may be somewhat weaker, to varying degrees, in other contexts.

Despite overall patterns, some cognitive items showed strong to moderate associations with cognitive impairment across all studies and should be recommended for use in future cross-national research. In particular, a number of memory items (CERAD immediate and delayed recall, and logical memory delayed recall) as well as the animal fluency task and the orientation item on naming the day of the week, showed consistently strong associations with cognitive impairment in each study. Other items performed well in specific settings, such as the item on naming a hammer, which had a stronger association with cognitive impairment in Mexico compared to other contexts. Such items should be considered for use in settings they perform well in, but may not be optimal candidates for cross-national comparisons.

Differences in item performance may be due to differences in the cultural contexts and educational attainment of participants in different HCAP studies. Prior work on the Hindi version of the Mini-Mental State Examination for use in Ballabgarh, India found that participants did not keep track of years and were often not attuned to geographic location beyond the boundaries of their village, which affected performance of items on orientation to time and place.³² In this study, we also found weaker associations between items on orientation and cognitive impairment in India.

Prior work on cognition in Cree-speaking natives in Canada found that items involving calculations or numeracy requirements were challenging to implement due to low levels of educational attainment.³³ The Mexico and India HCAP studies did not administer many of the executive functioning/attention items included in the United States and England studies due to concerns about education and numeracy. The South Africa HCAP study did administer items requiring numeracy, but we found high levels of missingness and weak associations with cognitive impairment in some of these items, including the Trail-Making Test Parts A & B and the Backwards counting test. Based on these convergent findings, we would not recommend the use of executive functioning tests with strong numeracy requirements for cross-national research. The symbol or letter cancellation task does not have such requirements and had strong to moderate associations with cognitive impairment across studies, indicating that this item may be a better choice for cross-national research. In addition, the two executive functioning items added to the India HCAP survey to assess executive functioning performed well compared to other executive functioning items in this setting. Future work should explore whether these items perform well in other low numeracy settings and in cross-national research.

This study leveraged large population-representative samples; minimal sample exclusions and use of sampling weights help ensure that findings are relevant to broader populations. However, the size and scale of the HCAP studies made the administration of gold-standard clinician-based diagnoses of dementia cost-prohibitive.³⁴ Instead, we used a neuropsychological norms approach to classifying cognitive impairment and assumed that normative samples across countries represented comparably healthy groups. The neuropsychological norms approach has been shown to result in fewer false positives compared to conventional criteria for mild cognitive impairment, and is highly correlated with Alzheimer’s disease biomarkers.^14,35,36 We also conducted a sensitivity analysis using latent class analysis as an alternative classification method, and found that overall patterns remained consistent. Our classification of cognitive impairment likely captured more mild forms of impairment as compared to a dementia diagnosis and we did not require deficits in functional limitations. Despite differences, the measurement of cognitive impairment is critical to the measurement of dementia, therefore conclusions regarding the measurement of cognitive impairment will apply to the measurement of dementia as well. In our primary analyses, we were unable to incorporate uncertainty from the estimation of cognitive impairment in logistic regression models; instead we treated impairment status as fixed, in line with other studies relying on algorithmic classifications.^37,38 Classification uncertainty was taken into consideration in secondary analyses using latent class analysis, which yielded similar inferences.

In addition, due to data availability constraints, our analysis is limited to data from five countries, two of which were similar high-income contexts (United States and England). However, our results can provide important insights into the measurement of dementia in the specific contexts examined, and broader patterns highlight differences between the measurement in high-income contexts (United States and England) compared to other settings (Mexico, South Africa, India). Future research should seek to incorporate new HCAP data from additional countries, as these are released.

We focused on one metric of item quality, the association between cognitive impairment and specific items, but other information will likely impact item selection in future studies. Considerations surrounding the magnitude of missing data, the variability of binary items, and comprehensive content coverage across cognitive domains will also be important. Furthermore, although this study evaluated the utility of items on cognition for classification purposes, other uses exist (e.g., for the evaluation of specific cognitive subdomains), which may be considered.

The diversity of studies using different methods for decisions on the selection and inclusion of survey items has led to extreme heterogeneity across the literature, with a recent systematic review finding over 230 different diagnostic procedures used in 237 studies for the assessment of dementia prevalence or incidence.² This variation highlights the lack of consensus on the best way to measure dementia across settings. Our results highlight the challenges in conducting cross-national research, which likely contribute to the observed heterogeneity in measurement within the field.

Despite these challenges, we identified items on cognition that had strong associations with cognitive impairment, either across settings or in individual HCAP studies. Items that performed consistently well across settings may be useful in future cross-national research, and can potentially be leveraged to allow post hoc statistical harmonization efforts using item response theory methods.³⁹ Items that had strong associations with cognitive impairment in specific locations should be considered for use in those locations to improve measurement quality in a given study. Some differences in assessments of cognition can and should persist in cross-national research due to differences in culture and context. However, our results can guide the selection of a common set of items for use in cross-national research and can help standardize assessments across new epidemiologic studies on cognitive aging and dementia.

HIGHLIGHTS.

• Little quantitative evidence exists to guide the design of cognitive assessments in cross-national studies.

• The performance of cognitive items for the measurement of dementia varied across countries.

• Items with lower variation across countries (e.g., delayed word recall) should be used in future cross-national assessments.

• Across countries, there was variability in the performance of language assessments, with the exception of the animal naming task.

• Results can be used to design future cross-national or location-specific cognitive assessments.

RESEARCH IN CONTEXT.

1. Systematic review:

Based on a literature review, prior research on the measurement of dementia in cross-national contexts has focused on the validation of specific scales in a limited number of settings. Previous work has not systematically assessed the performance of cognitive items across a range of countries.

2. Interpretation:

This study illustrated that cognitive items developed in high-income settings had stronger associations with cognitive impairment in contexts more similar to those in which they were developed. Items on memory performed best across the range of contexts, whereas many items on language showed wide variability in performance between settings. Items with numeracy requirements did not perform well in contexts with low educational attainment.

3. Future directions:

Future studies including additional cognitive items and a wider range of geographic locations can further enrich the evidence. Results from this study and future similar studies can inform the design of dementia assessments in global and cross-national research.