Skip to main content
BMJ Open logoLink to BMJ Open
. 2023 Nov 1;13(11):e073027. doi: 10.1136/bmjopen-2023-073027

Protocol for validating an algorithm to identify neurocognitive disorders in Canadian Longitudinal Study on Aging participants: an observational study

Alexandra J Mayhew 1,2,3, David Hogan 4,5,6, Parminder Raina 1,2,3, Christina Wolfson 7,8,9, Andrew P Costa 1,10, Aaron Jones 1,10, Susan Kirkland 11, Megan O'Connell 12, Vanessa Taler 13,14, Eric E Smith 15, Teresa Liu-Ambrose 16,17,18, Jinhui Ma 1, Mary Thompson 19, Changbao Wu 19, Howard Chertkow 20,21, Lauren E Griffith 1,2,3,; CLSA Memory Study Working Group
PMCID: PMC10626860  PMID: 37914306

Abstract

Introduction

In population-based research, disease ascertainment algorithms can be as accurate as, and less costly than, performing supplementary clinical examinations on selected participants to confirm a diagnosis of a neurocognitive disorder (NCD), but they require cohort-specific validation. To optimise the use of the Canadian Longitudinal Study on Aging (CLSA) to understand the epidemiology and burden of NCDs, the CLSA Memory Study will validate an NCD ascertainment algorithm to identify CLSA participants with these disorders using routinely acquired study data.

Methods and analysis

Up to 600 CLSA participants with equal numbers of those likely to have no NCD, mild NCD or major NCD based on prior self-reported physician diagnosis of a memory problem or dementia, medication consumption (ie, cholinesterase inhibitors, memantine) and/or self-reported function will be recruited during the follow-up 3 CLSA evaluations (started August 2021). Participants will undergo an assessment by a study clinician who will also review an informant interview and make a preliminary determination of the presence or absence of an NCD. The clinical assessment and available CLSA data will be reviewed by a Central Review Panel who will make a final categorisation of participants as having (1) no NCD, (2) mild NCD or, (3) major NCD (according to fifth version of the Diagnostic and Statistical Manual of Mental Disorders criteria). These will be used as our gold standard diagnosis to determine if the NCD ascertainment algorithm accurately identifies CLSA participants with an NCD. Weighted Kappa statistics will be the primary measure of agreement. Sensitivity, specificity, the C-statistic and the phi coefficient will also be estimated.

Ethics and dissemination

Ethics approval has been received from the institutional research ethics boards for each CLSA Data Collection Site (Université de Sherbrooke, Hamilton Integrated Research Ethics Board, Dalhousie University, Nova Scotia Health Research Ethics Board, University of Manitoba, McGill University, McGill University Health Centre Research Institute, Memorial University of Newfoundland, University of Victoria, Élisabeth Bruyère Research Institute of Ottawa, University of British Columbia, Island Health (Formerly the Vancouver Island Health Authority, Simon Fraser University, Calgary Conjoint Health Research Ethics Board).

The results of this work will be disseminated to public health professionals, researchers, health professionals, administrators and policy-makers through journal publications, conference presentations, publicly available reports and presentations to stakeholder groups.

Keywords: Dementia, Aging, PUBLIC HEALTH, EPIDEMIOLOGY


STRENGTHS AND LIMITATIONS OF THIS STUDY.

  • Validation of a neurocognitive disorder case ascertainment algorithm for the Canadian Longitudinal Study on Aging (CLSA) will allow use of this longitudinal and comprehensive database of this large population-based study to explore risk factors, early manifestations, aetiology and trajectory of these disorders.

  • Two particular challenges being faced in ascertaining the presence of a neurocognitive disorder are the lack of an informant and the use of cognitive measures that were not selected to diagnose a neurocognitive disorder. Lessons learnt in overcoming these obstacles will be of use for other longitudinal studies with similar limitations.

  • The results of the blinded clinician assessments and the additional information collected from their identified informant will allow us to refine and improve the accuracy of our case ascertainment algorithm.

  • If validated, the neurocognitive disorder case ascertainment algorithm developed for the CLSA is validated cannot be used by other population-based studies that differ in the data being collected on participants.

Introduction

A key challenge in population-based studies in ageing is to accurately identify individuals who have neurocognitive disorders (NCDs). A common approach is to use a two-stage evaluation based on participants’ estimated risk of an NCD. High risk participants and a random sample of those at lower risk undergo a clinical assessment specifically designed to identify NCDs. This approach adds complexity and costs to the study while being burdensome for participants. Relying on self-reports is likely insensitive. The Canadian Study of Health and Ageing, which used a two-stage evaluation to ascertain the presence of dementia, found that nearly two-thirds (64%) of participants identified with prevalent dementia in the study had never seen a physician for a memory problem.1 This was particularly common among those with mild functional impairment. While administrative data can also be used to estimate the burden of physician diagnosed and documented NCDs, the proportion with undocumented mild and major NCD is significant.2

The estimated population-based burden of diagnosed and undiagnosed dementia in Canada is based on data collected two decades ago in the CSHA1 that does not reflect updated criteria for the diagnosis of mild (mild cognitive impairment, MCI) and major (dementia) NCD as described in the fifth version of the Diagnostic and Statistical Manual of Mental Disorders (DSM)-5.3 Moreover, the increased awareness of NCDs over time may have led to earlier and more comprehensive identification and diagnosis.4 Previous analyses focused on major NCD, but mild NCD, which is viewed as a precursor to major NCD in many cases, has attracted increasing research interest. Approximately 50% of people with milder degrees of impaired cognition in later life progress to dementia within 5 years.5 Mild NCD is believed by many to be more likely to respond to disease-modifying interventions, making those with this condition a prime target group for their use.6–8

Contemporary estimates of the burden of mild and major NCD including in individuals that have not received a diagnosis is important to the understanding of the epidemiology of these disorders, their risk and protective factors, associated health outcomes, informing health and social care planning, and possibly leading to improved, proactive care of those living with or at risk for these conditions.

The accuracy of self-reported diagnoses for identifying chronic diseases is dependent on the condition, what is considered the gold-standard diagnosis, as well as the population studied.9–12 To improve the identification of individuals with chronic conditions in observational population-based studies, researchers often create disease ascertainment algorithms. These algorithms include multiple data items such as self-reported diagnosis, disease-specific questionnaires, performance measures and medication data to classify participants into those with and without diseases.13 Population-based studies have used algorithms to classify individuals as having an NCD or not. The Health and Retirement Study found that their algorithms correctly identified 87%–94% of participants on dementia status.14 The Personality and Total Health Through Life Project found that their algorithm had very good performance for identifying major NCD (area under the curve (AUC) of 0.95) and good performance (AUC of 0.76) for identifying mild NCD.15

Although the application of algorithms to population-based data has the potential to be cost-effective and meet the need for a standardised and comprehensive identification of cases, because of variability in the studied populations and the data collected on them cohort-specific validation is required.16 To validate an NCD algorithm, an assessment conducted by a clinician with training to diagnose NCDs is typically used as the gold standard. Ideally, this assessment should include a participant interview, cognitive testing, physical examination and an interview with an informant who knows the participant well enough to answer questions about their cognition, function and behaviour. Informant ratings have been found to reveal greater loss of everyday functional ability and cognitive competency than self-reports and are more strongly associated with objective measures of cognitive performance compared with how an individual rates their abilities.17

The Canadian Longitudinal Study on Aging (CLSA) is a large (51 338 participants aged 45–85 years at enrolment) national, longitudinal research platform that includes participants from all 10 Canadian provinces.18 It is being used to address a wide variety of aging-related research challenges including NCD. Disease ascertainment algorithms are already being used in the CLSA for several conditions (eg, type II diabetes mellitus, parkinsonism, chronic obstructive airway disease, osteoarthritis, coronary artery disease).13

To better understand the epidemiology and burden of diagnosed and undiagnosed mild and major NCD in CLSA participants (and by extrapolation the Canadian population), the CLSA Memory Study will be conducted to validate a disease ascertainment algorithm for NCD.

Methods and analysis

Study design and participant eligibility

The CLSA Memory Study will recruit participants from the CLSA. The CLSA is composed of two complementary cohorts that may be studied separately or together (figure 1): (1) Tracking cohort of 21 241 participants randomly selected from within all 10 provinces who are interviewed by telephone and (2) Comprehensive cohort of 30 097 participants randomly selected from within 25–50 km of 11 data collection sites (DCSs) across the country who are first interviewed at home and then visit their local DCS for a more in-depth assessment that includes additional interviews, physical measures and blood and urine samples. Participants are evaluated every 3 years and will be followed for 20 years (until 2033) unless they withdraw, are lost to follow-up or die.

Figure 1.

Figure 1

CLSA study design: the CLSA Memory Study will recruit comprehensive cohort and tracking cohort participants who are currently undergoing their follow-up three assessment (started August 2021) for the CLSA. CLSA, Canadian Longitudinal Study on Aging.CATI, Computer Assisted Telephone Interview. CAPI, Computer Assisted In-Person Home Interview

Consenting CLSA Memory Study participants will be asked to undergo a clinical assessment at a local DCS. For this reason, we will include participants from the Comprehensive cohort as well as Tracking Cohort participants who live within 25–50 km of a DCS. CLSA participants unable to visit their local DCS, complete the clinical assessment for any reason (eg, aphasia, hearing loss) or cannot identify an informant will be excluded from participation.

Patient and public involvement

Participants and the public were not involved in our research design.

Participant selection and recruitment

Participant selection

Prior to being contacted for the CLSA Memory Study, potential participants will be categorised on their presumed cognitive status according to DSM-5 criteria: (1) no NCD, (2) mild NCD and (3) major NCD. The categorisation will be based on data collected during the CLSA baseline (from 2011 to 2015), follow-up 1 (conducted from 2015 to 2018) and follow-up 2 visits (conducted from 2018 to 2021). This preliminary categorisation for participant selection is not the algorithm this project aims to validate.

Participants are presumed to have a mild NCD if they have a self-reported physician diagnosis of a memory problem, can both take medicine and manage money without help and have not lost their driver’s license or have restrictions on their license other than wearing eyeglasses. Additionally, participants who demonstrated cognitive problems in scheduling or during CLSA DCS visits that were documented by staff will be presumed to have a mild NCD.

Participants are presumed to have a major NCD if they meet one or more of the following criteria:

  1. Use of prescription medications for the treatment of a major NCD (specifically donepezil, galantamine, rivastigmine, memantine).

  2. Self-reported physician diagnosis of dementia or Alzheimer’s disease.

  3. Self-reported physician diagnosis of a memory problem and at least one of the following functional limitations.

    1. Requires assistance taking medication.

    2. Requires assistance managing money.

    3. Among those who formerly drove, no longer having a driver’s licence or having a driver’s license with restrictions other than eyeglasses.

Participants who do not meet the criteria for presumed mild or major NCD will be presumed not to have an NCD.

Approximately equal numbers from each of the three categories will be recruited, though final recruitment goals will be based on NCD status as determined through the Memory Study (see the Statistical methods section). Participants presumed to have mild or major NCD will first be selected. For one-third of the participants presumed to have major NCD and for two-thirds of the participants presumed to have mild NCD, a person of the same age (using participants’ age category as of 1 June 2022 (54–63, 64–73, 74–83, 84+ years) and sex presumed to have no cognitive impairment will be chosen at random.

Participant recruitment

Participants will be recruited into the CLSA Memory Study during CLSA follow-up 3 (started August 2021). Recruitment for the CLSA Memory Study started on 25 August 2022 and all data collection will be completed by 31 March 2024. Tracking cohort participants and comprehensive cohort participants who have completed their CLSA follow-up 3 interview will be emailed/mailed the participant information package (online supplemental appendix 1). Comprehensive cohort participants who have not yet completed their main CLSA follow-up 3 interview will be given the participant information package during their follow-up 3 in-home interview.

Supplementary data

bmjopen-2023-073027supp001.pdf (295.3KB, pdf)

After the participant has received an information package, the local CLSA DCS will contact the participant by phone to determine their interest in the study. Interested participants will complete a short questionnaire to determine if they understand the purpose of the study and what participant entails. Potential participants who, as judged by the interviewer, do not understand the details of the study will be ineligible. There are no additional eligibility criteria for participants selected for this sub-study beyond the general requirements for participation in the CLSA.18 Eligible participants will provide informed consent, identify and provide contact information for an informant and schedule their clinical assessment (online supplemental appendix 2). If a participant is unable or unwilling to identify an informant, they will not be able to participate in the study.

Supplementary data

bmjopen-2023-073027supp002.pdf (322.2KB, pdf)

Informant recruitment

Each participant will be asked to identify a family member or friend that knows them well enough to respond to questions about their cognitive health, ability to complete daily tasks and behaviour. Potential informants will be provided with a copy of the family member or friend information package (online supplemental appendix 3). The local DCS will contact the identified potential informant by phone prior to the participant’s clinical assessment to discuss the study, obtain consent from the informant and schedule a time to complete the informant interview via phone (online supplemental appendix 4). If the identified informant does not wish to take part in the study, the participant will be contacted and asked to identify an alternative informant.

Supplementary data

bmjopen-2023-073027supp003.pdf (346.7KB, pdf)

Supplementary data

bmjopen-2023-073027supp004.pdf (146.5KB, pdf)

Measurements

The CLSA Memory Study includes a clinical assessment of the study participant and a phone interview with the informant which will take place between September 2022 and March 2024. This information will be used to provide a provisional study diagnosis of (1) no evidence of cognitive impairment, (2) mild NCD (MCI) or (3) major NCD (dementia) based on DSM-5 criteria, which will be used as the reference standard for which the algorithm will be compared.

Clinical assessment

The clinical assessments will be conducted by a study clinician (medical specialist or senior trainee in geriatric medicine, geriatric psychiatry, neurology or psychiatry; internist with training and experience in cognitive assessment; neuropsychologist) who will undergo local and/or virtual training in the performance of the standardised assessment and completion of all required forms. The clinical assessment (online supplemental appendix 5) requires approximately 1 hour with the participant. It consists of a standardised history and physical examination designed to categorise the participant as having no evidence of an NCD, mild NCD or major NCD. The study clinician will not have access to CLSA data on the participant other than name, age, sex, gender identity, education, employment status and occupation and will be blinded to the participant’s presumed cognitive status. The clinical assessment has not been designed to determine the likely underlying cause of the NCD, risk of progression or specific care needs of the participant. The components of the assessment are as follows:

Supplementary data

bmjopen-2023-073027supp005.pdf (594.4KB, pdf)

  1. Participant interview

    1. Sociodemographic information (age, sex, gender identity, education, occupation, employment status).

    2. History of cognitive decline.

    3. Medical history including medical conditions, a review of medications focusing on those with cognitive effects, use of tobacco, cannabis and alcohol, and a family history of dementia.

    4. Basic activities of daily living measured using the Older Americans Resource and Services Programme (OARS) scale.19

    5. Instrumental activities of daily living measured using the OARS scale19 with additional questions regarding transportation (ie, driving).

    6. Behavioural symptoms including depression measured using the Patient Health Questionnaire-2,20 anxiety, psychotic symptoms and changes in personality.

  2. Cognitive testing

    1. The Montreal Cognitive Assessment (MoCA)21 will be used as a general measure of cognition. The MoCA is a brief instrument that has been shown to be a valid screening test for mild (MCI) and major NCD (dementia)22 with validated versions and normative data for both English and Quebec-French23 populations. The MoCA-BLIND version will be used for participants with visual impairments that would prevent them from completing the MoCA.24 An optional section of the MoCA called the Memory Impairment Score (MIS) will be used to assess uncued and cued (category and multiple-choice options) recall of the memory items. The use of the MoCA total and MoCA-MIS scores with all the other information being collected on participants will be used to help identify participants with mild and major NCD.25

  3. Physical examination

    1. Alertness.

    2. Hearing.

    3. Focal/lateralising neurological findings.

    4. Extrapyramidal signs.

    5. Balance and gait assessment including transfers, gait and the Romberg test.

Informant interview

The informant interview will be conducted by CLSA DCS staff using a standardised protocol. All CLSA DCSs have highly trained data collection teams. The informant interview (online supplemental appendix 6) includes several overlapping items to those directly asked of participant. Interview questions will focus on the participant’s cognitive, functional and mood/behavioural history. The components of the interview are as follows:

Supplementary data

bmjopen-2023-073027supp006.pdf (530.8KB, pdf)

  1. Cognitive changes measured using the eight-item informant interview to differentiate ageing and dementia (AD8 Dementia Screening Interview).26 The AD8 asks about changes in memory, orientation, judgement and function that might indicate a dementing illness.

  2. Medical history including medical conditions, use of tobacco, cannabis and alcohol, and a family history of dementia.

  3. Basic activities of daily living measured using the OARS scale.19

  4. Instrumental activities of daily living measured using the OARS scale19 with additional questions regarding transportation.

  5. Presence of current mood and psychiatric symptoms using the Mild Behavioural Impairment Checklist (MBI-C).27 The MBI-C was designed to measure neuropsychiatric symptoms that precede or coincide with the diagnosis of MCI. The instrument measures the domains of (1) decreased motivation; (2) emotional dysregulation; (3) loss of impulse control; (4) social inappropriateness and (5) abnormal perception or thought content.

Participant categorisation based on clinical assessment and informant interview

Study clinician

Based on the clinical assessment and the informant interview, the study clinician will make a provisional clinical determination of: (1) no evidence of cognitive impairment, (2) mild NCD (MCI) or (3) major NCD (dementia) based on DSM-5 criteria.3 All participants who complete the medical assessment will have a provisional clinical determination.

Study physicians will not provide participants with their provisional diagnosis, as to make a clinical diagnosis of mild or major NCD with confidence would require a more in-depth evaluation including review of prior health records, laboratory and/or imaging investigations as well as possible follow-up visits that our study clinicians are unable to provide. The study clinician will verbally tell the participant if there is a potential concern regarding their memory (the term memory will be used to describe any cognitive concern when communicating with the participant) or if they do not have any concerns based on the assessment and informant interview just conducted. The study clinician will tailor the conversation based on the participant’s level of understanding and their own degree of concern. Each participant will then be provided with a letter indicating if the clinician identified a potential problem with the participant’s memory (online supplemental appendix 7) or no evidence of a potential problem with the participant’s memory (online supplemental appendix 8), as well as the participant’s total score on the MoCA and details about the CLSA Memory Study. Participants identified by the clinician as having potential concerns about their memory will be encouraged to speak with their family physician and share the information provided verbally and in writing. If the participant does not have a family physician, the study clinician will provide the participant with local resources that the participant may use for follow-up care.

Supplementary data

bmjopen-2023-073027supp007.pdf (296.4KB, pdf)

Supplementary data

bmjopen-2023-073027supp008.pdf (296.6KB, pdf)

Central review panel

A central review panel including medical specialists (eg, geriatric medicine, geriatric psychiatry, neurology or psychiatry with training and experience in cognitive assessment) and neuropsychologists will review the clinical assessment, informant interview and available CLSA data such as performance on the neurocognitive battery conducted at baseline through to the follow-up 2 CLSA assessment (which the examining physician will not have seen). Based on the review of these data, the panel will make a final study categorisation. This will be compared with the one made by the study clinician, and, if different, an explanation for reaching a differing determination will be documented and provided to the examining clinician. The central review panel will help ensure that the study is implemented in a standardised manner across all sites by the participating clinicians. Any concerns will be brought to the attention of the involved clinician and the CLSA Memory Study investigators.

Pilot study and adaptation of recruitment criteria

Prior to the full implementation of the CLSA Memory Study, pilot testing will be conducted on a sample of 10 participants at two DCS sites (Hamilton and Calgary) to (1) identify any issues needing correction and (2) develop implementation advice for all DCS sites. These participants will be included in the final sample with their data retained as study data.

CLSA Memory Study investigators and staff will monitor the number of recruited participants by presumed NCD status, study clinician NCD determinations and central review panel categorisations at a group level. This monitoring will allow the detection of unbalanced recruitment and the opportunity to adapt the recruitment strategy during the study to ensure we end up with approximately equal number of participants in each NCD diagnostic category based on the central review panel categorisations. For example, if the number of participants determined by the study clinician and/or central review panel to have major NCD is lower than expected, we will start to oversample from the group of participants presumed to have a major NCD to compensate.

CLSA NCD ascertainment algorithm

Development of the CLSA NCD ascertainment algorithm

The CLSA NCD ascertainment algorithm was informed by a systematic review of methods used to identify cases of mild and major NCD in population-based studies (https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=78874). Identified potential items for inclusion were categorised based on DSM-5 criteria and then mapped onto available CLSA data (online supplemental appendix 9). Conditional use (eg, only include functional data provided by participants who achieve a certain cognitive threshold on the MoCA) and alternative weighting of select items that might improve on the accuracy of the algorithm will be explored in the study.

Supplementary data

bmjopen-2023-073027supp009.pdf (219.5KB, pdf)

Participant categorisation based on CLSA ascertainment algorithm

The initial validation of the CLSA ascertainment algorithm will include CLSA data from baseline, follow-up 1 and follow-up 2 assessments. The follow-up 2 interview data were collected three or more years before the Memory Study was initiated and may not accurately reflect the current cognitive status for all participants (eg, for those with new onset NCDs). Therefore, final validation of the algorithm will occur when the follow-up 3 assessment data, which were collected at the time of the CLSA Memory Study, are available to the central review panel in 2024. The algorithm will only include previously collected CLSA data and will not include information that was collected as part of the CLSA Memory Study (eg, informant interview, clinical assessment).

Broadly, the ascertainment algorithm will determine NCD status as: (1) no evidence of cognitive impairment, (2) mild NCD (MCI) or (3) major NCD (dementia) hierarchically using the criteria identified in online supplemental appendix 9; first identifying participants meeting the DSM-5 criteria for major NCD, then of the remaining participants, identifying those that meet the DSM-5 criteria for mild NCD. The algorithm will then classify participants as either having no evidence of cognitive impairment, or indeterminant for participants with missing data that prevents the algorithm from making a final determination. A version of the algorithm using an imputed dataset which considers other waves of data collection and missing data patterns will also be developed. The imputed algorithm will not have an indeterminant category.

Statistical analyses and sample size determination

Kappa using Cicchetti-Allison weights and the percent of agreement between the reference standard and the CLSA NCD algorithm will be calculated to assess the reliability of the CLSA algorithm. Sensitivity, specificity and C statistics for the CLSA NCD algorithm for each outcome category (major NCD, mild NCD or no evidence of cognitive impairment) will be estimated using logistic regression.28 Analyses will be completed overall and stratified by sex and age group (age 45–65 years old and 65+) using SAS (version 9.4). We will conduct the analyses using the version of the algorithm with the indeterminate category for participants with missing data as well as using the version of the algorithm with imputed data.

We have calculated the minimum sample size required based on different combinations of kappa values and precision (distance between the lower and upper 95% confidence limits) (table 1) using the ‘kappaSize’ Package in R with three outcome categories. This package assumed unweighted kappa to provide a conservative sample size estimate. Our final sample size will range between approximately 200 participants assuming an expected kappa of 0.7 and precision of 0.2, and 600 participants assuming an expected kappa of 0.7 and a precision of 0.1. Our aspiration is to recruit as close to 600 participants as possible, but this will be dependent on sufficient funding. We currently have funding confirmed for 320.

Table 1.

Minimum sample size for 95% CI width (0.05, 0.1, 0.15 and 0.2) by kappa

Kappa Precision (the distance between the lower and upper 95% confidence limits) Minimum required total sample size
0.7 0.05 2348
0.10 619
0.15 289
0.20 170
0.8 0.05 1764
0.10 481
0.15 231
0.20 139

Ethics and dissemination

Ethics approval for this project was provided by the Research Ethics Board responsible for each participating site (online supplemental appendix 10).

Supplementary data

bmjopen-2023-073027supp010.pdf (173.9KB, pdf)

Our knowledge translation plan includes sharing the results of the project with researchers and health professionals through journal publications and conference presentations. The CLSA will host a webinar on the Memory Study that will be open to researchers, health professionals, public health workers, as well as participants with an interest in NCD research. We will work with other partners to present our results to key groups. The CLSA will develop and disseminate a report that describes the results of the project and implications for health system stakeholders likely to use the results (eg, health professionals, administrators, policy-makers). The report and presentations will be tailored to specific stakeholder groups including those responsible for provincial and national dementia strategies (eg, Ministerial Advisory Board on Dementia), health professional organisations (eg, Canadian Geriatrics Society) and health charities (eg, Alzheimer’s Society of Canada). The report will also be available on the CLSA website. The CLSA website and social media platforms will be used to disseminate a summary of the project to participants. It is anticipated that the targets of tailored knowledge translation activities will use the results in various ways including: additional research on risk and protective factors for NCDs; development and implementation of best practices for early intervention and treatment for people with mild and major NCD; and, improving public health surveillance systems that develop population estimates for dementia in Canada that can be used to inform current and future government investment in prevention and care.

Discussion

There are some limitations with the use of CLSA data for developing an NCD ascertainment algorithm. First, CLSA interview data do not include an informant interview on most participants. In clinical settings, informant reports are an important component of the diagnosis of NCDs, as individuals with an NCD may be unaware of their own functional status and behavioural changes.29 Although the CLSA asks participants over the age of 70 years to identify a proxy, proxy interviews have only been conducted on a small number of participants and under specific conditions. Informant data, therefore, cannot be used to inform the algorithm. Another limitation is that the CLSA neurocognitive battery was not developed to diagnose NCDs.30 Rather, the battery items were selected to be applicable to a wide age range without ceiling or floor effects in order to capture decline over time. The neurocognitive battery items reflect the domains of executive function and memory, but not complex attention, language, perceptual motor or social cognition.

There are also several strengths of the CLSA dataset for developing an NCD ascertainment algorithm. The breadth of routinely collected CLSA data (eg, balance and gait performance measures, trajectory of changes in cognitive test performance) and the high percentage of participants (~88%) that have provided permission to the CLSA data to be linked to healthcare administrative databases provides an opportunity to explore the creation of an expanded and superior NCD ascertainment algorithm. Having a relatively large (up to 600) group of participants who have gone through a gold standard assessment for NCDs will make this effort possible.

Conclusion

If the results of the CLSA Memory Study suggest that the proposed NCD ascertainment algorithm is a valid method of identifying NCD cases, it will be applied to all CLSA participants. This will enhance the CLSA dataset for NCD research and provide important insights regarding the risk and protective factors of NCD and associated health outcomes. Linkage to healthcare administrative databases will allow the CLSA to estimate the burden of mild and major NCD in Canada. Together, these sources of data will help inform health and social care planning for individuals with NCD.

Supplementary Material

Reviewer comments
Author's manuscript

Acknowledgments

This research was made possible using the data/biospecimens collected by the Canadian Longitudinal Study on Aging (CLSA). Funding for the Canadian Longitudinal Study on Aging (CLSA) is provided by the Government of Canada through the Canadian Institutes of Health Research (CIHR) under grant reference: LSA 9447 and the Canada Foundation for Innovation, as well as the following provinces, Newfoundland, Nova Scotia, Quebec, Ontario, Manitoba, Alberta, and British Columbia. The CLSA provided data to the authors to conduct this study through an internal methods project application. The CLSA is led by Drs. Parminder Raina, Christina Wolfson, and Susan Kirkland. The opinions expressed in this manuscript are the authors’ own and do not reflect the views of the CLSA. This research was also made possible by CIHR Team Grant (New Directions in Dementia Research: Big Data) funding provided to the Canadian Consortium on Neurodegeneration in Aging (CCNA) for the Broad and Deep Analyses in Neurodegeneration (BRAIN) project. The Scientific Director of the CCNA is Dr. Howard Chertkow.

Footnotes

Twitter: @aaronjonesstats, @LaurenGriff1

Collaborators: The following are members of the CLSA Memory Study Working Group: Andrew Costa, Benoit Cossette, Lauren E. Griffith, David B. Hogan, Aaron Jones, Susan Kirkland, Teresa Liu-Ambrose, Jinhui Ma, Alexandra J. Mayhew, Jacqueline McMillan, Verena Menec, Gerry Mugford, Megan E. O’Connell, Theone Paterson, Christopher Patterson, Parminder Raina, Eric E. Smith, Vanessa Taler, Mary Thompson, Andrew Wister, Christina Wolfson, & Changbao Wu.

Contributors: LEG, PR, APC and DH led the conceptualisation of the study methodology. AM made contributions to the design of the study methods and wrote the first draft of the manuscript. CW, AJ, SK, MO'C, VT, EES, TL-A, JM, MT, CW and HC made contributions to the design of the study methods and supported the manuscript. All authors critically revised the manuscript and approved manuscript, approved the final version and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Funding: Funding for this project is provided by the Public Health Agency of Canada under project number 2021-HQ-000076 and the CIHR (funding reference number BDO 148341).

Disclaimer: The opinions expressed in this manuscript are the authors’ own and do not reflect the views of the CCNA.

Competing interests: ES has been paid consulting fees by Eli Lilly and Alnylam. No other authors have conflicts of interest to report.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Contributor Information

CLSA Memory Study Working Group:

Andrew Costa, Benoit Cossette, Lauren E Griffith, David B Hogan, Aaron Jones, Susan Kirkland, Teresa Liu-Ambrose, Jinhui Ma, Alexandra J Mayhew, Jacqueline McMillan, Verena Menec, Gerry Mugford, Megan E O’Connell, Theone Paterson, Christopher Patterson, Parminder Raina, Eric E Smith, Vanessa Taler, Mary Thompson, Andrew Wister, Christina Wolfson, and Changbao Wu

Ethics statements

Patient consent for publication

Not applicable.

References

  • 1. Sternberg SA, Wolfson C, Baumgarten M. Undetected dementia in community-dwelling older people: the Canadian study of health and aging. J Am Geriatr Soc 2000;48:1430–4. 10.1111/j.1532-5415.2000.tb02633.x [DOI] [PubMed] [Google Scholar]
  • 2. Lang L, Clifford A, Wei L, et al. Prevalence and determinants of undetected dementia in the community: a systematic literature review and a meta-analysis. BMJ Open 2017;7:e011146. 10.1136/bmjopen-2016-011146 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders, 5th ed. American Psychiatric Association Publishing, 2013. 10.1176/appi.books.9780890425596 [DOI] [Google Scholar]
  • 4. Tang W, Kannaley K, Friedman DB, et al. Concern about developing Alzheimer's disease or dementia and intention to be screened: an analysis of national survey data. Arch Gerontol Geriatr 2017;71:43–9. 10.1016/j.archger.2017.02.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Tuokko H, Frerichs RJ. Cognitive impairment with no dementia (CIND): longitudinal studies, the findings, and the issues. Clin Neuropsychol 2000;14:504–25. 10.1076/clin.14.4.504.7200 [DOI] [PubMed] [Google Scholar]
  • 6. U.S. Department of Health and Human Services Food and Drug Administration . Early Alzheimer’s disease: developing drugs for treatment-guidance for industry. 2018. Available: https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm
  • 7. van Dyck CH, Swanson CJ, Aisen P, et al. Lecanemab in early Alzheimer’s disease. N Engl J Med 2023;388:9–21. 10.1056/NEJMoa2212948 [DOI] [PubMed] [Google Scholar]
  • 8. Getsios D, Blume S, Ishak KJ, et al. An economic evaluation of early assessment for Alzheimer’s disease in the United Kingdom. Alzheimer’s & Dementia 2012;8:22–30. 10.1016/j.jalz.2010.07.001 [DOI] [PubMed] [Google Scholar]
  • 9. Stoye G, Zaranko B. IFS working paper W20/13 how accurate are self-reported diagnoses? Comparing self-reported health events in the English longitudinal study of ageing with administrative hospital records; 2020.
  • 10. Smith B, Chu LK, Smith TC, et al. Challenges of self-reported medical conditions and electronic medical records among members of a large military cohort. BMC Med Res Methodol 2008;8:37. 10.1186/1471-2288-8-37 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Okura Y, Urban LH, Mahoney DW, et al. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol 2004;57:1096–103. 10.1016/j.jclinepi.2004.04.005 [DOI] [PubMed] [Google Scholar]
  • 12. Payette Y, Moura CS de, Boileau C, et al. Is there an agreement between self-reported medical diagnosis in the cartagene cohort and the Québec administrative health databases? IJPDS 2019;5. 10.23889/ijpds.v5i1.1155 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Oremus M, Postuma R, Griffith L, et al. Validating chronic disease ascertainment algorithms for use in the Canadian longitudinal study on aging. Can J Aging 2013;32:232–9. 10.1017/S0714980813000275 [DOI] [PubMed] [Google Scholar]
  • 14. Gianattasio KZ, Wu Q, Glymour MM, et al. Comparison of methods for algorithmic classification of dementia status in the health and retirement study. Epidemiology 2019;30:291–302. 10.1097/EDE.0000000000000945 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Eramudugolla R, Mortby ME, Sachdev P, et al. Evaluation of a research diagnostic algorithm for DSM-5 neurocognitive disorders in a population-based cohort of older adults. Alzheimers Res Ther 2017;9:15. 10.1186/s13195-017-0246-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. Wilkinson T, Ly A, Schnier C, et al. Identifying dementia cases with routinely collected health data: a systematic review. Alzheimers Dement 2018;14:1038–51. 10.1016/j.jalz.2018.02.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17. Rueda AD, Lau KM, Saito N, et al. Self-rated and informant-rated everyday function in comparison to objective markers of Alzheimer’s disease. Alzheimers Dement 2015;11:1080–9. 10.1016/j.jalz.2014.09.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Raina PS, Wolfson C, Kirkland SA, et al. Cohort profile: the Canadian longitudinal study on aging (CLSA). Can J Aging 2009;28:221–9. 10.1017/S0714980809990055 [DOI] [PubMed] [Google Scholar]
  • 19. Fillenbaum GG, Smyer MA. The development, validity, and reliability of the OARS multidimensional functional assessment questionnaire. J Gerontol 1981;36:428–34. 10.1093/geronj/36.4.428 [DOI] [PubMed] [Google Scholar]
  • 20. Kroenke K, Spitzer RL, Williams JBW. The patient health questionnaire-2 validity of a two-item depression screener. Med Care 2003;41:1284–92. 10.1097/01.MLR.0000093487.78664.3C [DOI] [PubMed] [Google Scholar]
  • 21. Nasreddine ZS, Phillips NA, Bédirian V, et al. The Montreal cognitive assessment, Moca: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695–9. 10.1111/j.1532-5415.2005.53221.x [DOI] [PubMed] [Google Scholar]
  • 22. Tsoi KKF, Chan JYC, Hirai HW, et al. Cognitive tests to detect dementia a systematic review and meta-analysis. JAMA Intern Med 2015;175:1450–8. 10.1001/jamainternmed.2015.2152 [DOI] [PubMed] [Google Scholar]
  • 23. Larouche E, Tremblay M-P, Potvin O, et al. Normative data for the Montreal cognitive assessment in middle-aged and elderly Quebec-French people. Arch Clin Neuropsychol 2016;31:819–26. 10.1093/arclin/acw076 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Wittich W, Phillips N, Nasreddine ZS, et al. Sensitivity and specificity of the Montreal cognitive assessment modified for individuals who are visually impaired. J Vis Impair Blind 2010;104:360–8. 10.1177/0145482X1010400606 [DOI] [Google Scholar]
  • 25. Julayanont P, Brousseau M, Chertkow H, et al. Montreal cognitive assessment memory index score (Moca-MIS) as a predictor of conversion from mild cognitive impairment to Alzheimer’s disease. J Am Geriatr Soc 2014;62:679–84. 10.1111/jgs.12742 [DOI] [PubMed] [Google Scholar]
  • 26. Galvin JE, Roe CM, Powlishta KK, et al. The AD8: a brief informant interview to detect dementia. Neurology 2005;65:559–64. 10.1212/01.wnl.0000172958.95282.2a [DOI] [PubMed] [Google Scholar]
  • 27. Ismail Z, Agüera-Ortiz L, Brodaty H, et al. The mild behavioral impairment checklist (MBI-C): a rating scale for neuropsychiatric symptoms in pre-dementia populations. J Alzheimers Dis 2017;56:929–38. 10.3233/JAD-160979 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Cook RJ, Farewell VT. Conditional inference for subject specific and marginal agreement. two families of agreement measures. Can J Statistics 1995;23:333–44. 10.2307/3315378 [DOI] [Google Scholar]
  • 29. Schinka JA. Use of informants to identify mild cognitive impairment in older adults. Curr Psychiatry Rep 2010;12:4–12. 10.1007/s11920-009-0079-9 [DOI] [PubMed] [Google Scholar]
  • 30. Tuokko H, Griffith LE, Simard M, et al. Cognitive measures in the Canadian longitudinal study on aging. Clin Neuropsychol 2017;31:233–50. 10.1080/13854046.2016.1254279 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2023-073027supp001.pdf (295.3KB, pdf)

Supplementary data

bmjopen-2023-073027supp002.pdf (322.2KB, pdf)

Supplementary data

bmjopen-2023-073027supp003.pdf (346.7KB, pdf)

Supplementary data

bmjopen-2023-073027supp004.pdf (146.5KB, pdf)

Supplementary data

bmjopen-2023-073027supp005.pdf (594.4KB, pdf)

Supplementary data

bmjopen-2023-073027supp006.pdf (530.8KB, pdf)

Supplementary data

bmjopen-2023-073027supp007.pdf (296.4KB, pdf)

Supplementary data

bmjopen-2023-073027supp008.pdf (296.6KB, pdf)

Supplementary data

bmjopen-2023-073027supp009.pdf (219.5KB, pdf)

Supplementary data

bmjopen-2023-073027supp010.pdf (173.9KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES