Addenbrooke’s Cognitive Examination III (ACE‐III) and mini‐ACE for the detection of dementia and mild cognitive impairment

Abstract

This is a protocol for a Cochrane Review (Diagnostic test accuracy). The objectives are as follows:

To assess the diagnostic test accuracy of the Addenbrooke’s Cognitive Examination‐III (ACE‐III) and the mini‐ACE, for the screening of all‐cause dementia, dementia subtypes (Alzheimer’s disease, vascular dementia, frontotemporal dementia, Lewy body dementia), and mild cognitive impairment, across all healthcare settings at all prespecified thresholds.

Background

Dementia is an emerging public health concern; 46 million people currently live with dementia worldwide (Alzheimer's Society 2016). As the population ages, this figure is only expected to rise further, and thus sensitive screening tests are becoming increasingly important to distinguish healthy older adults from those with early cognitive impairment (Alzheimer's Society 2016; Prince 2015). Early identification of people with dementia is important to facilitate the early introduction of current available therapies, and to instigate important holistic patient and carer support through the provision of allied health professional and support services (Aminzadeh 2007; de Vugt 2013). Therefore, sensitive screening tests are required to support early referral for specialist assessment and management. The Addenbrooke’s Cognitive Examination‐III (ACE‐III), and its shorter counterpart, the mini‐ACE, are two such cognitive screening tests that are widely available for use across a variety of healthcare settings (Hsieh 2013; Hsieh 2015). The ACE‐III and mini‐ACE have reported good sensitivity and specificity in the literature (Hsieh 2013; Hsieh 2015), but, to date, have not been included in systematic reviews or meta‐analyses. In this review we will evaluate the validity of the mini‐ACE and ACE‐III to screen for dementia and mild cognitive impairment across all healthcare settings.

Target condition being diagnosed

Dementia currently affects 850 000 people in the UK alone, and this is projected to rise by 40% over the next decade as the population ages (Alzheimer's Society 2016). Dementia is characterised by a progressive loss of memory or cognitive function, resulting in impaired ability to perform activities of daily living (Creavin 2016; Davis 2015). The most typical presentation of dementia is that of progressive memory loss. However, dementia can present in a multitude of ways, from language deficits to loss of executive functioning (Robinson 2015). Dementia is an over‐arching term that encompasses several forms, including Alzheimer’s disease, vascular dementia, frontotemporal dementia and Lewy body dementia (Robinson 2015). As knowledge and understanding has evolved, it has become increasingly difficult to distinguish between these dementia subtypes, as there is considerable clinical and pathological overlap between them (Attems 2014; Mandal 2006). Alzheimer’s disease is the most common dementia subtype, accounting for 62% of all cases (Alzheimer's Society 2016). Alzheimer’s disease is notably characterised by the development of amyloid plaques, tau deposits, and neurofibrillatory tangles, resulting in a progressive deterioration in cognitive function (Takahashi 2017). Vascular dementia is the second most common form, comprising 17% of all dementia cases (Alzheimer's Society 2016). It is associated with vascular risk factors and events (i.e. transient ischaemic attack, acute stroke), resulting in chronic small vessel disease and leading to sustained cerebral hypoperfusion and thus cognitive impairment (Dichgans 2017). Deterioration in cognitive function would characteristically result in a step‐wise decline in cognition, although a slow progression similar to that seen with Alzheimer’s disease is also seen in vascular dementia secondary to small vessel disease, rather than discrete vascular events (Dichgans 2017). Ten per cent of dementia is mixed between subtypes, and the remainder is comprised of rarer forms: frontotemporal (2%), Parkinson’s disease (2%), and Lewy body dementia (4%) (Alzheimer's Society 2016). It is important to distinguish between these dementia subtypes as this can affect both the approach to diagnosis and treatment. Furthermore, identifying and stratifying the subtypes of dementia allows therapies to be tailored on an individual and personalised basis. Acetylcholinesterase inhibitors and N‐methyl‐D‐aspartate (NMDA) receptor antagonists are now established therapies for the treatment of mild to moderate Alzheimer’s disease (NICE 2011). The evidence base for the use of acetylcholinesterase inhibitors in vascular dementia is considerably smaller, however, the use of donepezil and rivastigmine are supported in a number of Cochrane Reviews (Birks 2013; Malouf 2004).

Mild cognitive impairment (MCI) is an emerging concept, where patients exhibit subjective and objective evidence of cognitive decline, but importantly, their functional status is maintained (Petersen 2004). Up to 60% of people with MCI will develop dementia by ten years (Korolev 2016; Petersen 2004). However, it is unclear why 40% of people with MCI do not progress to dementia (Korolev 2016; Petersen 2004). Therefore, tools that can identify and distinguish MCI, and predict those that are likely to develop dementia in the future, are becoming increasingly important for patients, clinicians, and researchers (Petersen 2004).

Index test(s)

Despite the emergence of a number of novel biomarkers, the detection and diagnosis of dementia remains to be achieved by thorough clinical assessment, and exclusion of important, potential reversible causes of cognitive decline (Health Quality Ontario 2014; Panegyres 2016; Robinson 2015). Cognitive assessment tools are a key component of this process, and allow physicians to identify not only the presence of cognitive impairment, but the severity, and key cognitive domains affected (Panegyres 2016; Velayudhan 2014). Radiological and biochemical investigations are adjunctive in the assessment of dementia, and are primarily used to exclude important structural and reversible causes of cognitive decline, i.e. tumours, hydrocephalus, and subdural haematoma (Harper 2014; Health Quality Ontario 2014; Panegyres 2016). Pathological changes (such as hippocampal atrophy and small vessel disease) are identified on brain imaging, but formal cognitive testing remains the primary tool for the identification and diagnosis of dementia and specific cognitive deficits (Harper 2014; Health Quality Ontario 2014; NICE 2018; Panegyres 2016; Robinson 2015). There are now several validated cognitive assessments tools available for screening, diagnosis and monitoring of cognitive disorders (Velayudhan 2014); thus, standard assessment practice is currently highly variable across the UK (Care Quality Commission 2014; Walker 2017). Choice of cognitive assessment tool is dependent on clinician and area, which introduces significant variations in dementia assessment practices nationally (Care Quality Commission 2014; Walker 2017), and worldwide due to lack of standardisation of tests across languages, literacy levels, and cultures (Kalaria 2010). Furthermore, there is a lack of consistent international guidance on the assessment and management of dementia, which has the potential to introduce further geographical disparities in care (Ngo 2015). Certainly, concerns have been raised regarding the widespread use of common assessment tools, particularly for the assessment of mild cognitive impairment, where the sensitivity is low (Nasreddine 2005). Therefore, clarity is urgently required on the most appropriate and valid cognitive assessment tool for the early identification and monitoring of cognitive disorders.

Cognitive impairment is frequently not identified in routine assessments in primary care; cognitive decline is not recognised in up to 76% of patients (Chodosh 2004; Ganguli 2004; Lin 2013; Valcour 2000). The majority of these patients will be diagnosed in the later stages of disease (Lin 2013). Early identification of dementia can often be the gateway to accessing crucial support and care services available to patients and their carers (Aminzadeh 2007; de Vugt 2013).

The Addenbrooke’s Cognitive Examination (ACE) was originally designed as a brief, bedside cognitive screen that was specifically developed to incorporate tests of memory, visuospatial, and executive function, with the ability to detect early dementia and differentiate Alzheimer’s disease from frontotemporal and Parkinson’s dementia (Larner 2014; Mioshi 2006; Noone 2015; Velayudhan 2014). A number of limitations were identified with the ACE, and this was updated to improve sensitivity, ease of administration, and to facilitate translation and cross‐cultural use as the Addenbrooke’s Cognitive Examination Revised (ACE‐R) (Mioshi 2006). The ACE‐R demonstrated significantly better sensitivity and specificity than the ACE (Larner 2014; Mioshi 2006), but further weaknesses were identified, including ceiling effects to several questions, confounding to verbal repetition by poor hearing, and difficulty translating for cross‐cultural use (Hsieh 2013; Velayudhan 2014). The ACE‐III was developed to address these limitations (Hsieh 2013). The ACE‐III has subsequently been translated into a number of languages, including Portuguese, Spanish, and Egyptian Arabic (Mirza 2017).

The ACE‐III is a brief, bedside, cognitive screening test that takes approximately 15 to 20 minutes to deliver; it encompasses five major cognitive domains: attention, memory, language, visuospatial function, and verbal fluency (Hsieh 2013; Noone 2015; Velayudhan 2014). It is composed of 21 cognitive tasks and has a total score of 100, where the common cut‐offs for dementia and MCI are considered at scores lower than 82 and 88, respectively (Hsieh 2013; Velayudhan 2014). Studies have demonstrated good sensitivity (93% to 100%) and specificity (96% to 100%) at these cut‐offs, but pooled estimates are lacking (Noone 2015; Velayudhan 2014). The mini‐ACE was derived as a shorter version of the ACE‐III, which takes under five minutes to perform, but maintains good sensitivity (61%, 85%), and specificity (100%, 87%), at established thresholds of 21 and 25 respectively (Hsieh 2015). Furthermore, the mini‐ACE can be used to distinguish between Alzheimer's disease and other forms of dementia (i.e. frontotemporal dementia, primary progressive aphasia, and corticobasal syndrome) (Hsieh 2015). The mini‐ACE is a 30‐point scale covering four cognitive domains: orientation, memory, language and visuospatial function. It can be used in a variety of clinical settings and is easily translated (Hsieh 2015). The mini‐ACE is designed to be used as a brief screening tool to facilitate referral for formal neuropsychological testing and cognitive assessment (Hsieh 2015).

Clinical pathway

Patients presenting with cognitive decline are encountered in a variety of healthcare settings, including general practice, inpatient settings, outreach, and community services (Creavin 2016; Davis 2015; Robinson 2015). National screening for dementia is not currently recommended for all people aged over 65 (NICE 2018). However, the Government’s Commissioning for Quality and Innovation (CQUIN) has recently expressed support for targeted screening of at‐risk groups in accident and emergency departments and general practice (Alzheimer's Research UK 2017). This identifies patients presenting in these settings that are more likely to be at risk of dementia, and prompts further questioning and investigation (Alzheimer's Research UK 2017). Cognitive assessment tools are becoming increasingly important as part of this targeted screening approach in identifying who should be referred for further specialist assessment.

Patients with dementia typically present with a progressive history of declining cognitive function over a period of months to years, which eventually results in loss of daily function for that individual (Creavin 2016; Davis 2015; Robinson 2015). Current guidance from the National Institute for Health and Care Excellence (NICE) advocates early referral to a specialist memory service when a diagnosis of dementia is suspected (NICE 2018). Brief cognitive assessments, specifically designed for community and general practice, are available to assist community practitioners in deciding where referral may be appropriate (NICE 2018; Velayudhan 2014). A diagnosis of dementia should only be made following a comprehensive, specialist assessment (NICE 2018). Therefore, all patients with a suspected diagnosis of dementia should undergo formal cognitive testing at the initial specialist assessment, and this should include measures of: attention and orientation, short‐ and long‐term memory, praxis, language and executive function (NICE 2018). Cognitive assessment should be undertaken alongside a full history, collateral history, mental state examination, physical examination, medication review, laboratory investigations, and brain imaging (NICE 2018; Robinson 2015). A diagnosis of dementia requires deficits in at least two cognitive domains, with an impact on the patient’s ability to carry out activities of daily living (Robinson 2015).

Patients with MCI typically present with cognitive decline or change in memory, and can be identified in primary, secondary, and community care settings. The key factor which distinguishes MCI from dementia is the absence of functional impact on day‐to‐day living (Petersen 2004). In order to confirm a diagnosis of MCI, patients must have both subjective and objective cognitive decline, in addition to remaining functionally independent (Petersen 2004). Cognitive assessment tools form an integral component in identifying any objective cognitive deficits. It is important to distinguish MCI from dementia, as it has clinically relevant consequences for therapeutic management. Where patients with mild dementia would be eligible for initiation of acetylcholinesterase inhibitors, there is currently no evidence to support their use in the treatment of MCI (NICE 2018). Ensuring the correct identification and diagnosis of individuals is a crucial step in the clinical pathway for these patients.

Alternative test(s)

There are numerous cognitive assessment tools available for the screening and diagnosis of dementia, and these have been assessed in a number of previous reviews (Tsoi 2015; Velayudhan 2014). The Mini Mental State Examination (MMSE) is amongst one of the more widely used tests, but its use has been limited in recent years due to availability, and concerns about lack of sensitivity (Tsoi 2015). The findings of a recent Cochrane Review do not support the use of the MMSE to identify patients with MCI who could develop dementia (Arevalo‐Rodriguez 2015), but Creavin and colleagues stated it can be used to support the diagnosis of dementia in primary care (Creavin 2016). The Montreal Cognitive Assessment (MoCA) has recently been evaluated in a Cochrane Review for the diagnosis of dementia (Davis 2015). The MoCA was able to correctly identify dementia in 94% of cases, across all settings, but was limited by a high rate of false positive diagnoses (Davis 2015). Furthermore, the evidence supporting the use of MoCA was only in secondary care settings, which limits the generalisability of these findings to primary care (Davis 2015). The Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) is questionnaire based on informant responses to support a diagnosis of dementia (Harrison 2016). The IQCODE has good sensitivity, but was found to lack sufficient specificity for diagnosing dementia across several healthcare settings (Harrison 2016). A full list detailing the currently available Cochrane diagnostic test accuracy (DTA) reviews for neuropsychological assessments in dementia is available in Table 1.

Table 1.

Cochrane reviews of DTA studies for neuropsychological assessment tools in dementia

Cognitive Test	Available	Community	Primary	Secondary
Mini‐Cog	Y	x	x	Protocol
IQCODE	Y	x	x	x
AD‐8	Protocol	x	x	x
MMSE	Y	x	x	x
MoCA	Y	x		x

IQCODE = Informant Questionnaire on Cognitive Decline in the Elderly; MMSE = Mini‐Mental State Examination; MoCA = Montreal Cognitive Assessment

Rationale

A diagnosis of dementia still carries much stigma and fear in modern society (Aminzadeh 2007; de Vugt 2013). Despite increasing research, sensitive diagnostic tests and curative treatments remain elusive. Given the absence of an available cure, the consequences of a dementia diagnosis are profound and have an enormous impact on the patient, their family, and support network (Aminzadeh 2007; Davis 2015; de Vugt 2013). A high level of test sensitivity is important to minimise the rate of false negative diagnoses, which can prevent or delay access to available treatments and support services, and potentially worsen the dementia state and carer strain, and evoke loss of confidence in care services (de Vugt 2013). Similarly, a high test specificity will minimise the number of false positive diagnoses. A false positive diagnosis of dementia could cause serious psychological harm, and lead to unnecessary further investigations and treatments for a patient and their carers (de Vugt 2013). If clinical practitioners had access to a screening test with high sensitivity and specificity, it would reduce the negative consequences outlined above, and facilitate the timely delivery of support and available treatments (de Vugt 2013).

In summary, there have been a number of reviews of the ACE and ACE‐R (Crawford 2012; Larner 2014; Tsoi 2015), but no comprehensive review of later versions of the ACE (ACE‐III and mini‐ACE) has been carried out to date. Therefore, a Cochrane Review is required to assess the validity of the ACE‐III and mini‐ACE across all the available evidence, cut‐off scores, settings in which the tools have been validated, and the quality of the evidence to date. In particular, the mini‐ACE and ACE‐III have shown promising results in a number of studies, and so may prove more sensitive and specific tests for the early detection of cognitive disorders, with the ability to distinguish between dementia subtypes (Hsieh 2013; Hsieh 2015). Correct, early identification and stratification of patients with dementia can result in better clinical outcomes, through the early initiation of available therapeutics and support services for patients and carers (Creavin 2016; Davis 2015; de Vugt 2013).

Objectives

To assess the diagnostic test accuracy of the Addenbrooke’s Cognitive Examination‐III (ACE‐III) and the mini‐ACE, for the screening of all‐cause dementia, dementia subtypes (Alzheimer’s disease, vascular dementia, frontotemporal dementia, Lewy body dementia), and mild cognitive impairment, across all healthcare settings at all prespecified thresholds.

Secondary objectives

1) To identify the quality and quantity of the research evidence on the diagnostic test accuracy of the ACE‐III and mini‐ACE for the assessment of all‐cause dementia, dementia subtypes (Alzheimer’s disease, vascular dementia, frontotemporal dementia, Lewy body dementia), and mild cognitive impairment, across all healthcare settings at all reported thresholds.

2) To identify sources of heterogeneity (age, sex, education, severity or stage of the target condition, operator characteristic of the index test and reference standard) in the included studies.

3) To identify gaps in the evidence where further research is required.

Methods

Criteria for considering studies for this review

Types of studies

We will consider cross‐sectional studies for inclusion in this review, where the index test is administered alongside expert confirmation for reference. Comparative studies between dementia subtypes (i.e. Alzheimer’s disease and frontotemporal dementia), or comparing the index tests with an alternative (i.e. the Mini Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA)), will also be considered for inclusion if an appropriate reference standard is present, but only data on the ACE‐III and mini‐ACE will be included. We anticipate the majority of studies will follow a cross‐sectional design.

We will not include case control studies in this review due to the high risk of bias in these studies. Nested case control studies, where cases and controls are selected from the cohort population, will also be considered for inclusion. Delayed verification or longitudinal studies will not be included in this review.

Studies with small number of cases (less than 10), will not be considered for inclusion due to their associated high risk of bias.

Participants

Patients presenting with cognitive decline, undergoing cognitive testing in primary or secondary care, will be suitable for inclusion. Studies which included participants with a comorbidity associated with cognitive impairment (motor neurone disease (MND), multiple sclerosis (MS), Parkinson’s disease, brain injury/tumour/infection), but not a primary dementia diagnosis, will be excluded if these participants comprised more than 20% of the study population. In addition, studies which included participants with known substance abuse or medication use known to affect cognition will be excluded if these participants comprised more than 20% of the study population. Studies with mixed populations will be excluded from this review, if they comprised more than 20% of the study population.

Index tests

The ACE‐III and mini‐ACE only will be considered for screening accuracy. There are other versions, such as the Addenbrooke’s Cognitive Examination and the Addenbrooke’s Cognitive Examination Revised (ACE‐R), however the ACE‐III and mini‐ACE have superseded these older, previous versions and thus represent the most up‐to‐date versions of the tool. Threshold scores of 82 and 88 for the ACE‐III (Velayudhan 2014), and 21 and 25 for the mini‐ACE (Hsieh 2015), have been reported consistently in the literature, and are currently used conventionally in clinical practice. Therefore this review will investigate the summary sensitivity and specificity values at these predefined thresholds. The ACE‐III and mini‐ACE have been translated into several languages, and all studies assessing the screening accuracy of translated versions of the ACE‐III and mini‐ACE will be considered for inclusion. The ACE‐III and mini‐ACE tools are available at http://dementia.ie/images/uploads/site‐images/ACE‐III_Administration_(UK).pdf and https://s3‐eu‐west‐1.amazonaws.com/pstorage‐karger‐594308543098/6990263/450784_sm1.pdf, respectively.

Target conditions

The target conditions to be detected by the ACE‐III or mini‐ACE are as follows: all‐cause dementia (undifferentiated), specific dementia subtypes (Alzheimer’s disease, vascular dementia, frontotemporal dementia, Lewy body dementia), and mild cognitive impairment (MCI). All‐cause dementia has been included as a target condition, as it is anticipated that some studies will not have differentiated between dementia subtypes. In addition, the ACE‐III and mini‐ACE are being evaluated as screening tests, therefore, understanding the ability of the test to identify undifferentiated cognitive impairment for onward specialist referral for subtype and classification would be of relevance to primary care practitioners.

Reference standards

At present, there is no gold‐standard test for the confirmation of MCI, dementia, or subtype. In current practice, dementia and MCI are confirmed by an appropriately qualified clinical specialist or expert (i.e. neurologist or psychiatrist), using internationally developed and validated criteria. The reference standard for this review is a clinical confirmation of dementia or MCI using disease‐specific reference standards developed by a consensus group or accredited body, as follows.

• Undifferentiated dementia: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition and Fifth Edition (DSM‐IV and DSM‐5) (American Psychiatric Association 2000; American Psychiatric Association 2013), International Classification of Diseases 10th Revision and 11th Revision (ICD‐10 and ICD‐11) (World Health Organization 2010; World Health Organization 2018).

• Alzheimer’s disease: NINCDS/ADRDA (McKhann 1984), ICD‐10 and ICD‐11 (World Health Organization 2010; World Health Organization 2018), DSM‐IV and DSM‐5 (American Psychiatric Association 2000; American Psychiatric Association 2013), National Institute on Aging and the Alzheimer's Association (NIA/AA) (McKhann 2011).

• Vascular dementia: NINDS‐AIREN (Roman 1993), DSM‐IV and DSM‐5 (American Psychiatric Association 2000; American Psychiatric Association 2013), ICD‐10 and ICD‐11 (World Health Organization 2010; World Health Organization 2018).

• Frontotemporal dementia: Lund‐Manchester criteria (Lund 1994), NINDS (Rascovsky 2011).

• Lewy body dementia: International consensus criteria (McKeith 2006)

• MCI: NIA/AA (McKhann 2011), DSM‐IV and DSM‐5 (American Psychiatric Association 2000; American Psychiatric Association 2013), Mayo (Petersen 2013), Petersen (Petersen 2004).

• Post‐stroke dementia (DSM‐IV and DSM‐5 (American Psychiatric Association 2000; American Psychiatric Association 2013), ICD‐10 and ICD‐11 (World Health Organization 2010; World Health Organization 2018).

The presence of the disease will need to be confirmed using one of these recognised criteria by an appropriately qualified specialist, expert, or consensus group in order for a study to be included in this review. Radiological and biochemical investigations are often used alongside clinical assessment to confirm dementia or MCI, however, studies which rely on imaging and biochemical investigations alone, without clinical assessment, will be excluded from this review.

Studies using a histopathological diagnosis of dementia as a reference standard will not be suitable for inclusion as this is a post‐mortem diagnosis.

Search methods for identification of studies

Search methods will be carried out in accordance with Chapter 7 of the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. The search strategy will be determined in conjunction with the Information Specialist at the Cochrane Dementia and Cognitive Improvement Group (CD‐CIG).

Electronic searches

We will search MEDLINE (OvidSP), Embase (OvidSP), BIO‐SIS (Ovid), Science Citation Index (ISI Web of Knowledge), PsycINFO (Ovid), LILACS (Bireme), ALOIS (specialised register of the CD‐CIG), and the Cochrane Register of Diagnostic Test Accuracy Studies, using a structured search strategy appropriate for each database. Controlled vocabulary, such as MeSH terms and EMTREE, will be used where appropriate. The search will not be restricted by sampling frame, setting, or language. Translation services will be used as required. A single researcher, with extensive experience in systematic reviewing, will perform subsequent searches. The MEDLINE search strategy can be seen in Appendix 1.

Searching other resources

We will review the reference lists of all included studies. We will also search the following additional databases.

• Database of Abstracts of Reviews of Effects (DARE): www.york.ac.uk/inst/crd/crddatabases.html (updated to 2015).

• Health Technology Assessment (HTA) Database: www.cochrane.org/about‐us/evidence‐based‐health‐care/webliography/books/hta, via the Cochrane Library.

• Aggressive Research Intelligence Facility (ARIF): www.arif.bham.ac.uk (updated to 2018).

We will use the 'related articles' feature of PubMed to search for additional studies. Citation databases, such as Science Citation Index and Scopus, will be searched using key studies to identify any additional relevant studies. We will search grey literature, including conference proceedings, theses, and PhD abstracts. We will not perform handsearching, in accordance with the generic protocol (Davis 2013). Research groups involved in previously published or ongoing research on the ACE‐III or mini‐ACE will be contacted to identify any relevant, unpublished data.

Data collection and analysis

Selection of studies

The eligibility criteria are as follows.

Inclusion criteria:

• community, primary and secondary care services;

• patients presenting with cognitive decline;

• cross‐sectional, comparative, or nested case‐control studies;

• studies utilising the ACE‐III or mini‐ACE as the index test;

• presence of a referenced standard as specified above.

Exclusion criteria:

• patients with a diagnosis of dementia at presentation;

• patients with comorbidity associated with cognitive impairment, motor neurone disease (MND), multiple sclerosis (MS), Parkinson’s disease, brain injury, tumour, infection;

• patients with presence of substance abuse, or medication use known to affect cognition;

• case‐control studies, longitudinal or delayed‐verification studies;

• small sample size (less than 10 participants);

• studies utilising older versions of the tool (ACE, ACE‐R);

• absence of a reference standard as specified above.

Two review authors (LCB, APB) will independently screen eligible articles based on title and abstract. After this, full papers will be reviewed independently by at least two authors (LCB, APB, VJH) for inclusion in the review. Disagreements will be resolved by discussion, and if they remain unresolved despite this, will be referred to an arbitrator within the study team (VJH, TGR). Where disagreements are not resolved, the default position will be to include the study in the review. The study selection process will be detailed in a PRISMA flow diagram.

Data extraction and management

A study‐specific proforma will be developed, and data will be extracted on the following: study characteristics (setting, type, number of participants, diagnostic criteria, language, index test), demographics of the participants (age, gender, diagnosis, comorbidities), study quality assessment, and heterogeneity. The proforma will be piloted on ten diagnostic studies, before use in the review. The data that will be collected with the study proforma are detailed in Appendix 2.

Data will be extracted independently by two review authors (LCB, APB). Test accuracy data will be dichotomised if required, and cross‐tabulated in two‐by‐two tables of index test results (positive or negative) against the target condition (positive or negative). Disagreements between authors on data extraction will be resolved initially by discussion, and if this not possible referred to a third author (VJH, TGR) for arbitration. The results will be extracted directly into tables in Review Manager version 5.3 software (Review Manager 2014).

For each included study, we will outline the flow of the participant (i.e. number recruited, included and assessed), in a flow diagram.

Assessment of methodological quality

Methodological quality will be assessed independently by two authors (LCB, APB), using the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS‐2) (Whiting 2011). The tool consists of four domains: patient selection, index tests, reference standard, and patient flow. Each domain will be assessed in terms of risk of bias, and the first three domains will be considered in terms of applicability. The QUADAS‐2 tool will be piloted on the first five studies included in the review. Where there is poor agreement between the two review authors, the tool will be revised and re‐piloted. Disagreements between authors on study quality will be resolved initially by discussion, and if this not possible referred to a third author (VJH, TGR) for arbitration. Studies will be graded as being at high, medium or low risk of bias, with a narrative summary presented for each study. The QUADAS‐2 tool is available in Appendix 3, and the anchoring statements in Appendix 4. The use of the reference standard and index tests are not completely independent of one another, and this introduces a risk of incorporation bias. Included studies will be assessed for the presence of incorporation bias.

The STARdem tool (Noel‐Storr 2014) has been recently developed to report the quality of study reporting in dementia. In addition to reporting methodological quality, this review will also report on the quality of study reporting using this checklist (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4115600/table/T3/?report=objectonly).

Statistical analysis and data synthesis

The target condition comprises three categories: 1) undifferentiated (all‐cause) dementia, 2) specific dementia subtypes (Alzheimer’s disease, vascular dementia, frontotemporal dementia, Lewy body dementia), and 3) MCI. The index test comprises two categories, ACE‐III, or the mini‐ACE. The setting also comprises three categories: community, primary, and secondary care. A separate meta‐analysis will be required for each index test, in each study setting, and for each of the above target conditions. Given that the ACE‐III and mini‐ACE have only been developed recently, it is possible that there will be insufficient study numbers for meta‐analysis across these conditions and settings. If there are insufficient studies for meta‐analysis, a descriptive summary of the numerical results will be provided.

For all included studies (cross‐sectional), we will extract data in binary two‐by‐two tables (binary test results cross‐classified with the binary reference standard) and this will be used to calculate sensitivities and specificities, with 95% confidence intervals. Individual study results will be presented graphically by plotting estimates of sensitivities and specificities in a forest plot and receiver operating characteristic (ROC) curves. We will perform analyses using Review Manager 5 software for Windows. As outlined above, predefined thresholds of 82 and 88 for the ACE‐III (Velayudhan 2014), and 21 and 25 for the min‐ACE (Hsieh 2015) will be used to calculate summary sensitivity and specificity values at each threshold. Each study included in this review can contribute to one or more thresholds, and studies which do not report any of these thresholds will be excluded from this review. Graphical presentations will be undertaken for all predefined thresholds reported in the included studies.

Summary analysis will be undertaken using a bivariate random‐effects approach, based on pairs of sensitivity and specificity to calculate pooled estimates of: sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and 95% confidence intervals. We will not undertake univariate analyses if there are insufficient studies to calculate summary statistics, due to a risk of under‐estimating test accuracy (Deeks 2001). In this instance, results will be presented for each study in tables and forest plots. Summary analysis will be performed using MetaDTA software (MetaDTA 2018). The MetaDTA app is evolving to meet the needs of end‐users. The current beta version has new functionality and further modifications are planned. However, it is not guaranteed that all the analytical options required for this review will be incorporated by the time we come to analysis. We propose that we use MetaDTA in the first instance, to explore the data. If needed, we will use macros developed for use in SAS software for more sophisticated analyses.

Investigations of heterogeneity

As outlined above, it is unlikely that there will be sufficient studies for each of the categories of target condition, index test, and setting for meta‐analysis. Therefore, there are unlikely to be sufficient data for heterogeneity analyses. In line with previous Cochrane DTA reviews of neuropsychological tests, we anticipate there will be a number of sources of heterogeneity in the studies identified for review (Creavin 2016; Davis 2013; Davis 2015; Harrison 2016). We have identified key factors to be explored in a prespecified heterogeneity analysis, and these are outlined below.

Case mix

The case mix of the populations included in the studies could enter significant heterogeneity in terms of age, dementia diagnosis, specific populations versus unselected populations, and the severity or stage of the dementia diagnosis. The test properties are likely to differ in younger compared to older populations. In studies where less than 20% of the population is under 65 years of age, they are not likely to be representative of this population, and sensitivity analyses will be carried out to determine the effect of removing these studies on the summary results. Finally, the majority of studies will enrol adults from an unselected population. However, where studies enrol a specific or limited population, a sensitivity analysis will be conducted to identify the effect of removing these studies on summary results.

Reference standard criteria

An important source of heterogeneity, and a key component of methodological quality, is the process by which the cases of dementia or MCI are confirmed and subclassified. Data will be collected on this process, including which reference standard or criteria were used, whether it was by consensus meeting, individual assessment, or algorithm, and whether imaging or biochemical investigations were included. The quality of this process will be assessed at study level using the QUADAS‐2 tool.

Technical features of the index tests

Several thresholds have been reported in the literature for both the ACE‐III and mini‐ACE, however, the two most consistent levels which are currently used in clinical practice have been selected for analysis. Analyses will be conducted for all of the predefined thresholds for each test.

Heterogeneity will be investigated informally in the first instance through visual examination of forest plots of sensitivities and specificities and through visual examination of the ROC plot of the raw data. If sufficient data are present, formal investigation of the sources of heterogeneity will be explored through subgroup analyses, and regression analyses with up to ten studies per covariate. Assistance for subgroup and regression analyses will be provided by a statistician (CPN).

Sensitivity analyses

Sensitivity analyses will be undertaken to determine the effects of excluding low‐quality studies on analyses. Primary analysis will include all studies, and sensitivity analyses will exclude studies felt to be at high risk of bias using the QUADAS‐2 tool, and studies identified as less appropriate for inclusion (i.e. where there was unresolved disagreement between authors). ROC curves, forest plots, and summary statistics will be re‐run with the exclusion of studies at high risk of bias, and compared to the original analyses. Sensitvity analysis will be performed using MetaDTA software (MetaDTA 2018).

Assessment of reporting bias

We will not explore reporting bias in this review, as current quantitative methods for exploring reporting bias are not well established for studies of DTA. Specifically, funnel plots of the diagnostic odds ratio versus the standard error of this estimate will not be considered.

Appendices

Appendix 1. Search strategy

The search strategy uses two concepts: index test/s and populations of interest. The search was devised and then tested on a set of known studies. All known studies were identified by the search.

1. Addenbrooke* Cognitive Exam*.ti,ab. 2. ACE.ti,ab. 3. ACE‐r.ti,ab. 4. Mini‐Addenbrooke* Cognitive Exam*.ti,ab. 5. mini‐ACE.ti,ab. 6. ACE‐III.ti,ab. 7. or/1‐6 8. ((cognit$ or memory or cerebr$ or mental$) adj3 (declin$ or impair$ or los$ or deteriorat$ or degenerat$ or complain$ or disturb$ or disorder$)).ti,ab. 9. (forgetful$ or confused or confusion).ti,ab. 10. MCI.ti,ab. 11. AMCI.ti,ab. 12. ARCD.ti,ab. 13. SMC.ti,ab. 14. CIND.ti,ab. 15. BSF.ti,ab. 16. AAMI.ti,ab. 17. MD.ti,ab. 18. LCD.ti,ab. 19. QD.ti,ab. 20. AACD.ti,ab. 21. MNCD.ti,ab. 22. MCD.ti,ab. 23. ("N‐MCI" or "A‐MCI" or "M‐MCI").ti,ab. 24. minor neurocognitive disorder.ti,ab. 25. Cognitive Dysfunction/ 26. Cognition Disorders/ 27. or/8‐26 28. exp DEMENTIA/ 29. major cognitive disorder.ti,ab. 30. alzheimer*.ti,ab. 31. dement*.ti,ab. 32. ((lewy adj2 bod*) or LBD or DLB).ti,ab. 33. (FTLD or frontotemp*).ti,ab. 34. or/28‐33 35. 27 or 34 36. 7 and 35

Appendix 2. Study data to be included in the data collection proforma

1. Bibliographic details of primary paper: author, title of study, year, and journal.

2. Details of index test: method of ACE‐III and mini‐ACE administration, including who administered and interpreted the test, and their training. Thresholds used to define positive and negative tests.

3. Reference standard: reference standard used. Method of reference standard administration, including who administered the test and their training.

4. Study population: number of subjects. Age. Gender. Other characteristics (e.g. ApoE status). Settings: community, primary care, secondary care outpatients, and secondary care inpatients and residential care. Participant recruitment. Sampling procedures. Time between index test and reference standard. Proportion of people in sample with dementia. Subtype and stage of dementia if available. MCI definition used (if applicable). Duration of follow‐up in delayed verification studies. Attrition and missing data.

Appendix 3. QUADAS‐2 tool

DOMAIN	PARTICIPANT SELECTION	INDEX TEST	REFERENCE STANDARD	FLOW AND TIMING
Description	Describe methods of participant selection: describe included participants (prior testing, presentation, intended use of index test and setting):	Describe the index test and how it was conducted and interpreted	Describe the reference standard and how it was conducted and interpreted.	Describe any participants who did not receive the index test(s) and/or reference standard or who were excluded from the 2x2 table (refer to flow diagram): describe the time interval and any interventions between index test(s) and reference standard:
Signalling questions (yes/no/unclear)	Was a consecutive or random sample of participants enrolled?	Were the index test results interpreted without knowledge of the results of the reference standard?	Is the reference standard likely to correctly classify the target condition?	Was there an appropriate interval between index test(s) and reference standard?
	Was a case‐control design avoided?	If a threshold was used, was it prespecified?	Were the reference standard results interpreted without knowledge of the results of the index test?	Did all participants receive a reference standard?
	Did the study avoid inappropriate exclusions?			Did all participants receive the same reference standard?
				Were all participants included in the analysis?
Risk of bias: (high/low/ unclear)	Could the selection of participants have introduced bias?	Could the conduct or interpretation of the index test have introduced bias?	Could the reference standard, its conduct, or its interpretation have introduced bias?	Could the participant flow have introduced bias?
Concerns regarding applicability: (high/low/ unclear)	Are there concerns that the included participants do not match the review question?	Are there concerns that the index test, its conduct, or interpretation differ from the review question?	Are there concerns that the target condition as defined by the reference standard does not match the review question?

Appendix 4. QUADAS‐2 anchoring statements

We have adapted the core anchoring statements provided for use with the QUADAS‐2 tool. The original anchoring statements were determined from a two day multi‐disciplinary group meeting, designed for use with the QUADAS‐2 tool to support decisions concerning methodological quality for studies included in systematic reviews. Some of the original anchoring statements are less applicable to DTA reviews of neuropsychological assessments (ref MMSE review, etc.). Thus, two authors (LCB, APB) adapted the original anchoring statements specifically for this review, and these revised statements were reviewed by the co‐authors. The tool and anchoring statements will be piloted against the first five studies included in this review and if there is poor inter‐rater agreement of study methodological quality, the statements will be revised and re‐piloted until good agreement between raters is achieved.

Domain 1: participant selection

Was a consecutive or random sample of participants enrolled?

The method of sampling should be stated or described. Non‐random sampling, sampling based on volunteers, or selecting participants from a clinic or research population is more likely to introduce a high risk of bias and should be classified as such, whereas consecutive or random sampling are least likely to introduce bias, and should be classified as low risk.

Weighting: high risk

Was a case control design avoided?

Case control designs are associated with a high risk of bias and should be excluded from this review. However, nested case control studies (where the study population is drawn from a larger pool of patients from an interventional or cohort study) are associated with a lower risk of bias, and are considered for inclusion in this review. Nested‐case control studies should be classified as a high risk of bias, and any study which increases or decreases the proportion of patients with the target condition (i.e. enrichment from secondary care settings) should be classified as high risk of bias.

Weighting: high risk

Did the study avoid inappropriate exclusions?

Studies which do not explicitly detail exclusion criteria will be classified as unclear risk of bias, but study authors will be contacted for this information. Studies which clearly detail all exclusions, and are felt to be appropriate by review authors will be classified as low risk of bias. Exclusion criteria must be justified for studies which exclude difficult to diagnose groups. It is anticipated that there will be common exclusion criteria (e.g. substance misuse, other degenerative disease) for included studies, which are listed in the protocol. Community studies with extensive exclusion criteria should be classified at high risk of bias. Post‐hoc exclusions will be classified as high risk of bias.

Weighting: high risk

Domain 2: index test

Could the conduct or interpretation of the ACE‐III/mini‐ACE have introduced bias?

Studies will be considered low risk where the investigators conducting the ACE‐III/mini‐ACE were blinded to the participant’s diagnosis or were independent from the study and without knowledge of the reference standard. Studies which explicitly state this do not require further information on the blinding or independence of the process and will be classified as low risk of bias. Studies will be classified as low risk of bias if the ACE‐III or mini‐ACE were conducted prior to the reference standard.

Weighting: high risk

Were the ACE‐III/mini‐ACE thresholds pre‐specified?

A study will be classified as high risk of bias where the authors set the optimal cut off point post‐hoc using their own study data. Studies that do not use defined thresholds, and use an alternative methods of analysis will be classified as not applicable.

Weighting: high risk

Were sufficient data on ACE‐III or mini‐ACE application given for the test to be repeated in an independent study?

For studies to be classified at low risk of bias, information on the method of administration (i.e. appropriately qualified/trained), and the language of assessment should be provided. If a translated version of the ACE‐III or mini‐ACE is used, details of the scale and on the validation process will be needed to be classified at low risk of bias.

Weighting: low risk

Domain 3: reference standard

Is the reference standard likely to correctly classify the target condition?

Studies using reference standards listed in the protocol or a recognised/validated reference standard will be considered at low risk of bias. Studies using a reference standard not recognised by the authors or the Cochrane Dementia and Cognitive Improvement Group, will be classified at high risk of bias.

Weighting: high risk

Were the reference standard results interpreted without knowledge of the results of the ACE‐III/mini‐ACE?

For a study to be classified as low risk of bias, the investigators would need to have interpreted the reference standard results independently to those of the ACE‐III or mini‐ACE. Studies which explicitly state this do not require further information on the blinding or independence of the process and will be classified as low risk of bias. If the ACE‐III or mini‐ACE were used as part of the clinical dementia/MCI assessment as reference standard, this will be considered to be at high risk of bias.

Weighting: high risk

Were sufficient information on the method of dementia/MCI assessment given for the assessment to be repeated in an independent study?

The method of dementia assessment will need to be described to be considered at low risk of bias. Information should be provided on: the training and expertise of the assessor, whether it was by individual, consensus, or algorithm, and the use of neuropsychological, laboratory and neuroimaging assessments.

Weighting: high risk if not described

Domain 4: patient flow and timing

Was there an appropriate interval between the ACE‐III or mini‐ACE and the reference standard?

Ideally, the reference standard and ACE‐III or mini‐ACE would be completed on the same day or visit, to minimise changes or fluctuations in cognition over time. However, dementia is slowly progressive and an irreversible condition so delay is unlikely to introduce significant bias. However, patients with MCI can revert to normal cognition, progress, or remain stable over time. Therefore, a time delay could affect the measured cognitive status of these individuals, however the duration over which this might occur is not known. We have therefore set an arbitrary cut off of one month for studies assessing MCI. Longitudinal and delayed verification studies are excluded from this review.

Weighting: low risk

Did all subjects receive the same reference standard?

Where the clinical assessment or reference standard differs between participants in a study, this will be classified at high risk of bias. Participants who score test positive on the ACE‐III or mini‐ACE who are subject to further testing above other participants will be classified at high risk of bias.

Weighting: high risk

Were all participants included in the final analysis?

Attrition will vary with study design, but drop‐out rates and missing data should be reported and accounted for. Where attrition is higher than expected (greater than 20% of study cohort), these studies will be classified at high risk of bias.

Weighting: high risk

Applicability

Were those included representative of the general population?

The included participants should match the intended population as described in the review protocol. The setting of the included study will need to be taken into account, and the prevalence of the disease within that setting. Included participants should be presenting with cognitive decline, but the disease status should not be known at the time of administering the ACE‐III or mini‐ACE. Studies will be classified as low applicability where they included a highly selected population, or sub‐group.

Was the ACE‐III or mini‐ACE performed consistently and in a manner similar to its use in clinical practice?

Variation in the length, structure, language, and/or administration of the ACE‐III or mini‐ACE not in line with the original description of the ACE‐III or mini‐ACE may affect the applicability. Included studies will be judged against the original description of the ACE‐III or mini‐ACE.

Was the clinical diagnosis of dementia or MCI (reference standard) made in a manner similar to current clinical practice?

Although studies may have utilised a validated reference standard for the diagnosis of dementia or MCI, there is a risk that the reference standard may over‐ or under‐diagnose the proportion of participants with the disease. If there are concerns that the reference standard diagnosed a smaller or larger than anticipated proportion of participants given the specified clinical population, this would be rated as poor applicability.

Contributions of authors

LCB developed the draft and final versions of the protocol.

TGR, VJH, AB, RBP, TJQ, and CPN all reviewed and contributed to the draft and final versions of the protocol.

Sources of support

Internal sources

• No sources of support supplied

External sources

• Dunhill Medical Trust, Other.

  Funding of LCB as a clinical research fellow.

• NIHR, UK.

  This protocol was supported by the National Institute for Health Research (NIHR), via Cochrane Infrastructure funding to the Cochrane Dementia and Cognitive Improvement group. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Systematic Reviews Programme, NIHR, National Health Service or the Department of Health

Declarations of interest

Lucy C Beishon: none known. Angus P Batterham: none known. Terry J Quinn: none known. Ronney B Panerai: none known. Christopher P Nelson: none known. Thompson Robinson: none known. Victoria J Haunton: none known.

New