Improving clinical cognitive testing

Abstract

Objective:

To evaluate the evidence basis of single-domain cognitive tests frequently used by behavioral neurologists in an effort to improve the quality of clinical cognitive assessment.

Methods:

Behavioral Neurology Section members of the American Academy of Neurology were surveyed about how they conduct clinical cognitive testing, with a particular focus on the Neurobehavioral Status Exam (NBSE). In contrast to general screening cognitive tests, an NBSE consists of tests of individual cognitive domains (e.g., memory or language) that provide a more comprehensive diagnostic assessment. Workgroups for each of 5 cognitive domains (attention, executive function, memory, language, and spatial cognition) conducted evidence-based reviews of frequently used tests. Reviews focused on suitability for office-based clinical practice, including test administration time, accessibility of normative data, disease populations studied, and availability in the public domain.

Results:

Demographic and clinical practice data were obtained from 200 respondents who reported using a wide range of cognitive tests. Based on survey data and ancillary information, between 5 and 15 tests in each cognitive domain were reviewed. Within each domain, several tests are highlighted as being well-suited for an NBSE.

Conclusions:

We identified frequently used single-domain cognitive tests that are suitable for an NBSE to help make informed choices about clinical cognitive assessment. Some frequently used tests have limited normative data or have not been well-studied in common neurologic disorders. Utilizing standardized cognitive tests, particularly those with normative data based on the individual's age and educational level, can enhance the rigor and utility of clinical cognitive assessment.

Cognitive testing is essential for evaluating memory and other cognitive complaints associated with neurologic disorders. General cognitive tests, such as the Montreal Cognitive Assessment (MoCA),¹ Addenbrooke's Cognitive Examination–Revised,² or Mini-Mental State Examination (MMSE),³ are useful for screening but have limited diagnostic specificity. By contrast, a Neurobehavioral Status Exam (NBSE)⁴ is a collection of cognitive tests, chosen by a neurologist, intended to provide a more in-depth assessment of multiple cognitive domains, usually in conjunction with a neurologic evaluation, and sometimes as a separate assessment procedure (Current Procedural Terminology [CPT] 96116). If neuropsychology services are unavailable, the NBSE is an essential component of diagnostic assessment. If neuropsychology testing services are available, the NBSE may be still be useful as an adjunct or to help in determining whether more in-depth testing is needed, particularly for evaluating personality and psychological traits. In general, the NBSE is used to identify a patient's pattern of relative cognitive strengths and weaknesses, inform differential diagnosis, and guide therapeutic management. However, there are no guidelines for selecting tests for NBSE testing and virtually no information about how the NBSE is deployed in clinical practice. An NBSE differs from a formal neuropsychological assessment in its brevity, integration with other findings of a broader neurologic evaluation, and absence of formal intelligence testing or detailed assessment of personality or psychological attributes.

Many neurologists do not consider normative values for tests, and neurologists often use cognitive tests without ample normative data or that have not been studied in the conditions they are using them to assess. The NBSE Workgroup of the American Academy of Neurology (AAN) was established to review office-based cognitive testing in clinical practice as a reference point for quality improvement. Toward this end, behavioral neurologists were surveyed about their use of cognitive testing in clinical practice. A large sample of frequently used tests was reviewed in a standardized way, emphasizing desirable features for office-based testing: brief administration time (total NBSE duration of 30–60 minutes or less), diseases studied (diagnostic utility), existence of normative values (normal, abnormal, borderline), and availability in the public domain (accessibility and low cost). While not an exhaustive, systematic review, characterizing how the NBSE is used in clinical practice helped identify a compendium of tests most suitable for domain-specific, normative, office-based testing to improve the quality and utility of cognitive testing.

METHODS

NBSE practice survey.

The executive council of the Behavioral Neurology Section (BNS) of the AAN formed a workgroup to evaluate the NBSE. The workgroup identified 5 main cognitive domains by consensus and assigned leaders for each: attention (D.G.), executive function (K.D.), language (J.H.), memory (D.K.), and spatial cognition (A.M.B.). Between 5 and 15 tests within each cognitive domain were chosen by each subgroup to include in the survey (table 1). Tests that assess multiple domains (e.g., Clock-Drawing Test) were assigned to a single domain for review purposes.

Table 1.

Tests reviewed by cognitive domain

A web-based survey using Google Documents was e-mailed to all 713 members of the AAN-BNS in March 2012, inquiring about the frequency of use of these selected tests and any additional tests that members administer. Respondents rated each test as “often use,” “occasionally use,” or “never use.” The survey also asked about time devoted to testing, patient populations evaluated, and whether global cognitive screens (e.g., MMSE, Addenbrooke's Cognitive Examination–Revised), neuropsychiatric scales (e.g., Neuropsychiatric Inventory),⁵ and functional rating scales (e.g., Functional Activities Questionnaire)⁶ are also administered. A more detailed account of the survey data will be conveyed in a separate report.

Evidenced-based review.

Tests that survey respondents indicated were either “often” or “occasionally” used were selected for review. Each cognitive domain subgroup used PubMed, Ovid, and textbooks to review the literature on the included tests. Each test review followed a similar format: (1) historical background and test description; (2) specific cognitive functions assessed; (3) copyright status (restricted or public domain); (4) administration time; (5) normative data across the adult lifespan; (6) patient populations studied; (7) advantages/limitations of the test; and (8) key references.

RESULTS

Survey.

Two hundred of the 713 AAN-BNS members (28%) responded to the survey (table 2). Most respondents (87%) were board-certified in Neurology and 22% had United Council for Neurologic Subspecialties (UCNS) certification in the Behavior Neurology & Neuropsychiatry subspecialty. The majority of respondents (79%) were adult neurologists and most (72%) were US-based.

Table 2.

Demographics of the 200 survey respondents (members of the AAN-BNS)

Review.

Data compiled for each test were used to create 37 summary tables, which are included in appendix e-1 on the Neurology® Web site at Neurology.org. Table 3 provides an example of a summary table, in this case for the Trail Making Test (TMT). These tables cover administration time, variations of the test used, a focused review of validity, normative data, some limited psychometric properties, and a consensus summary of the test's benefits and shortcomings. The data are intended to provide a concise reference guide to the clinical neurologist, in contrast to a comprehensive review of test characteristics and psychometric properties contained in neuropsychology textbooks.^7,8 Table 4 highlights some of the reviewed tests and summarizes whether each has properties that the workgroup believes may be important for test selection in clinical practice (e.g., available normative values; a brief administration time; availability in the public domain).

Table 3.

Example of a summary table from the evidenced-based review

Table 4.

Summary of results: Selected cognitive tests

Attention domain tests.

Tests of attention can be broadly divided into 2 groups: “basic” or “complex” attention. Basic attention is a prerequisite for all other mental functions and reflects the ability to sustain focus in order to perform a simple task. Complex attention reflects the ability to control, shift, and divide attentional focus, allowing for the manipulation of information and execution of multiple steps to accomplish a goal. Among tests of attention, all but the 3 continuous performance tests and some versions of the Stroop Test are not copyright-restricted (i.e., in the public domain).

Digit Span Forward⁹ and TMT, Part A (TMT-A)¹⁰ are frequently used tests of basic attention. The former assesses maintenance aspects of working memory and TMT-A indexes processing speed. These tests are brief, have robust normative values across the adult lifespan, and have been broadly studied in patients with Alzheimer disease (AD) and other cognitive disorders.^11,12 The Corsi Blocks Test¹³ involves imitating sequential tapping patterns on blocks arrayed on a board. The longest sequence (i.e., number of blocks) tapped correctly is a measure of spatially mediated attentional capacity. The test is easy to administer and has been widely studied,¹⁴ but scoring schemes are not well standardized.¹⁵

TMT, Part B (TMT-B), Months-of-the-Year-Backward, and the Stroop Test are complex attention tasks that are accessible in the public domain and have age-adjusted normative values across the adult lifespan.^16–18 TMT-B is a timed test in which patients are asked to alternately connect circled numbers and letters that are irregularly distributed across a page. It requires planning, sequencing, set-shifting, and response-inhibition. Months-of-the-Year-Backward is a Sequential Operations Series test in which a patient is asked to recite an overlearned sequence in reverse order. It can serve as a brief assessment of complex attention and working memory, and may also be useful for tracking change if administered as a timed test.

Executive domain tests.

Tests of executive function range from assessing set-shifting and inhibitory control, to evaluating higher-order conceptual skills and abstract reasoning. Among tests reviewed, only some versions of the Proverbs Test¹⁹ and the Similarities subtest of the Wechsler Adult Intelligence Scale²⁰ were copyright-restricted. The antisaccade task, which examines the ability to inhibit automatic motor behavior by exerting control over the strong tendency to look toward a visual stimulus, takes 3 minutes or less, has normative values throughout the lifespan,²¹ and has been validated in tracking AD progression.²² It has also been studied in different subtypes of mild cognitive impairment,²³ progressive supranuclear palsy, schizophrenia,²⁴ and frontotemporal lobar degeneration (FTLD).²⁵

The Frontal Assessment Battery (FAB) includes 6 tasks that range from motor control programs to higher-order concept formation and interpretation.²⁶ It has robust age- and educational-based normative data, is not copyright-restricted, and has been used in FTLD, AD, and Parkinson disease.²⁷ Higher-order executive functions, including abstract thinking and concept formation, can also be assessed with the Similarities subtest of the Wechsler Adult Intelligence Scale–Fourth Edition²⁸ and the Proverbs Test of the Delis-Kaplan Executive Function System.¹⁹ These tests may be useful as an adjunct to the FAB.

Language domain tests.

Language tests were reviewed covering naming, comprehension, reading, writing, speech production, verbal fluency, and semantic-lexical retrieval. Of the 5 tests reviewed, only the Semantic/Category Fluency test is in the public domain.

Verbal fluency is often assessed by the ability to generate words based on semantic category (semantic fluency) or first letter (phonemic/lexical fluency). The semantic fluency test reviewed is part of several language batteries, including the Boston Diagnostic Aphasia Examination²⁹ and the Western Aphasia Battery–Revised (WAB-R).³⁰ Patients are asked to list as many exemplars of a specified category (e.g., animals, fruits) as possible in 1 minute. Normative values across the adult lifespan exist for several common categories.^31,32 This test has been studied in longitudinal cohorts of aging and patients with AD at various stages of severity.^33,34 Impaired semantic processing, as indexed by reduced semantic fluency, is a common early sign of underlying AD pathology.³⁵ The Controlled Oral Word Association Test,^36,37 a phonemic fluency task in which patients generate words from a letter cue (e.g., “F”), also has age- and education-adjusted norms across the adult lifespan³⁸ and has been studied in multiple conditions.^33,34,39,40 The WAB-R is a 45- to 60-minute battery that examines the major aspects of language (e.g., comprehension, writing, fluency) and has ample normative data and validity in detecting and tracking language impairment in many disorders.^e1,e2 Although the length of the WAB-R makes it impractical for brief testing, there is a “Bedside WAB-R” tool that screens for major aspects of language dysfunction, takes only 15 to 20 minutes to administer, and has separate normative values.^e3

The Boston Naming Test (BNT)–15-Item Short Form^e4 is one of several abbreviated versions of the BNT,^e5 a visual confrontation naming test. The BNT has been investigated for use in neurodegenerative diseases (e.g., AD, FTLD), vascular dementia, multiple sclerosis, and epilepsy and may be useful for the longitudinal tracking of aphasia due to multiple etiologies.^e6 Performance on many BNT-15 versions, composed of different subsets of the original BNT stimuli,^e7 is highly correlated with the full 60-item BNT.^e8 In addition to the existence of normative data for older adults (50s to 90s),^e8 this feature makes the BNT-15 useful for brief evaluation like the NBSE.

Memory domain tests.

All tests reviewed were verbally based auditory memory tests. Several tests (e.g., Rey-Osterrieth Complex Figures Test [ROCFT])^e9 discussed in the attention and spatial cognition sections have visual-based memory components. Among the 7 tests reviewed, only the Consortium to Establish a Registry in Alzheimer's Disease (CERAD) Word-List Memory Test^e10 and the Rey Auditory Verbal Learning Test^e9 are in the public domain. The CERAD is a 10-word list of unrelated words that includes 3 learning trials, followed by a delayed recall and forced-choice recognition trial. The test has ample normative values for ages 50 to 90, is validated in several longitudinal AD trials, and has a relatively brief administration time.^e8–e10 The Hopkins Verbal Learning Test–Revised (HVLT-R)^e11 is another relatively brief 12-word list that contains 4 words from each of 3 categories (e.g., gems). This “semantic clustering” provides an opportunity to use associative learning as a strategy for encoding and retrieving words. The HVLT-R has been studied in a number of conditions, including vascular dementia^e12 and traumatic brain injury (TBI).^e13

Another approach to assessing verbal memory is the recall of prose passages. Learning and storing information that is part of a colorful narrative text is often less demanding than memorizing a list of unrelated words. Assessing both can provide multiple perspectives on the severity of memory deficits. The Logical Memory I and II subtests of the Wechsler Memory Scale^e12 have robust normative values for ages 16 to 90^e12 and, together, take about 10 minutes to complete, with a suggested 20-minute delay. Performance on Logical Memory correlates with outcomes in several neuropsychiatric settings, including recovery from TBI,^e13 social functioning in schizophrenia,^e14 and the ability to comprehend and reason about treatment decisions in mild to moderate AD.^e15

Various 3-word and 5-word recall tests were not reviewed in this study. There are limited normative data for these tests and recall performance varies widely depending on the words used and whether patients are instructed to remember the words. The recall of 0 or 1 of 3 words presented may be a “red flag” for memory impairment in adults aged 65 to 90 years, as suggested by the Mini-Cog screening test.^e16 However, up to 19% of older adults who recall only 1 word on a 3-word test perform normally on more detailed memory testing, suggesting that these simple tests may not accurately reflect a patient's memory function.^e16,e17

Spatial cognition domain tests.

Nine tests in this domain were reviewed, including tests of visuoperceptual processing; visuoconstructive ability (which depends on organization and planning); and spatial bias processing associated with spatial inattention, neglect syndromes, or simultanagnosia. Most of the spatial cognition tests reviewed are in the public domain, except the Behavioral Inattention Test (BIT)^e18 and the Benton Judgment of Line Orientation (JLO).^e19

Among visuoconstructive tests, the Clock-Drawing Test is well-suited for an NBSE. It is brief, has abundant normative data^e20,e21 (although based on several different scoring criteria), and is validated in stroke, AD, and a range of psychiatric disorders.^e22–e24 It has been used for decades to both detect and serially grade disease severity in degenerative dementias.^e25–e27 The Cube Copying Test, also a part of the Short Test of Mental Status^e28 and the MoCA, involves copying a 3-dimensional cube. Although typically scored as correct/incorrect, several quantitative scoring methods are available.^e29,e30 The JLO^e19 is useful for identifying visuoperceptual deficits in judging spatial relationships. There are several short forms of the JLO that utilize a subset of the stimuli and significantly reduce administration time but are also copyright-protected.^e31 Performance on short forms of the JLO generally correlates well with full JLO scores.^e31 Normative values for both the JLO^e32 and some frequently used short forms^e33 are available, particularly for ages 50 to 90.

Visual cancellation tests^e34 assess visual scanning strategies and spatial bias and may be a useful component of an NBSE. Typically, patients are asked to mark each occurrence of target stimuli that are scattered across a page containing distracter items. Many versions of this brief test are in the public domain, and some have normative values across the adult lifespan.^e35 Line bisection tests are easy to administer and also provide a straightforward assessment of spatial bias. Both cancellation and line bisection tests have been well studied and validated in patients with hemispheric lesions due to stroke or tumors.

The BIT^e18 and ROCFT^e9 examine spatial cognitive processes, but may be challenging to include in an office-based NBSE. The BIT is lengthy, has limited normative values across the adult lifespan, and is copyright-protected. The ROCFT complements verbal memory tests by assessing the acquisition and storage of visual (nonverbal) information, and has abundant normative values.^e36 Drawbacks include lengthy administration time and potential confounding by organizational and planning deficits.

DISCUSSION

Cognitive testing by behavioral neurologists often entails a mixture of relatively brief cognitive tests, such as 3-word memory recall or a motor Go/No Go test, and abbreviated versions of neuropsychological assessments or subtests of test batteries. Whereas the former may be informative in some clinical contexts, tests that lack standardized administration and scoring guidelines have less utility for an NBSE. The tests reviewed varied considerably in terms of available normative data and information on their use in specific patient populations (e.g., AD, TBI). Some tests (e.g., cancellation, Luria Hand-Sequencing) do not have available normative values to use for office-based reference and scoring.

Relevance to clinical practice: Toward evidence-based NBSE.

The availability of age- and education-appropriate normative values may be the most important test characteristic to help accurately interpret a patient's performance. Given the wide age range of adults seeking neurology evaluations, tests with normative values across the adult lifespan are often most helpful. Many normative values are published and some relevant citations for each test are in appendix e-1. Although test choice should ideally be based on psychometric validity for the symptoms being assessed, both the cost and limited access of copyrighted tests may be nontrivial barriers for many practices. Tests used in an NBSE, especially those that are subtests of neuropsychological batteries, generally require formal training in test administration, scoring, and interpretation, with provision for expert guidance and feedback during the learning phase.

As a reference guide for the NBSE, this review is intended to help neurologists better assess major cognitive domains with tests that are most appropriate for evaluating hypotheses about a given patient. The pattern of impairments can suggest which neural networks may be disrupted. Along with the available history, neurologic examination, and laboratory or imaging data, this can help to identify the likely underlying pathologic process. For example, a profile of deficits in memory storage, semantic fluency, and complex attention suggests disruption of temporolimbic structures (e.g., hippocampus), temporal neocortex, and frontal networks. In a 75-year-old patient with progressive deficits but no functional decline, this pattern suggests a diagnosis of mild cognitive impairment, amnestic subtype, which often reflects underlying AD pathology.

Our review suggests some incongruence between a few widely used tests and those that we suggest may be most optimal for an NBSE. For example, the Luria Hand-Sequencing Test was reported as either often used or occasionally used by more than two-thirds of respondents. Although this test is brief and does not require test materials, it has limited published norms across the adult lifespan. The Luria test is a component of several batteries (e.g., the FAB) for which norms are available. Although a patient's inability to learn a Luria sequence or the tendency to exhibit sustained, perseverative responses would clearly represent impaired performance, judging whether more subtle sequencing errors is abnormal for age is much more challenging in the absence of norms. Also, various 3- or 5-word memory recall tests were reported as used often by more than half of respondents. Although brief and often available in the public domain, these frequently used bedside tests have limited normative data and have been less studied in dementia populations than the word-list memory tests reviewed here.

Neuropsychological evaluation.

An NBSE conducted by neurologists may bring to light issues that would benefit from a comprehensive neuropsychological assessment. Such evaluations are much more extensive (2–6 hours) than the typical 30- to 60-minute NBSE and provide a more detailed, quantitative assessment of cognitive function, particularly regarding premorbid estimates of cognitive abilities. For example, neuropsychological assessment may corroborate or challenge results from an NBSE and help track disease progression over time, provide feedback to families about changes in the patient's status, and allow for the planning of cognitive rehabilitation programs. Neuropsychological evaluation may also be helpful in patients who show borderline impairment on an NBSE, are at either end of the educational spectrum, have prominent psychiatric comorbidities, or who may be involved in medicolegal action. In our experience, collaboration between neurologists and neuropsychologists often facilitates optimal diagnostic and treatment plans for patients. It is essential that clinicians be cognizant of practice effects if both an NBSE and neuropsychological evaluation are completed in close succession.

Limitations.

We reviewed a small number of the hundreds of cognitive tests that have been developed. Although our survey data helped identify the most frequently used tests for review, our findings do not preclude other available tests from being appropriate for an NBSE. This may be particularly true for tests that were not initially, or are not widely, used in English. We did not review the tests “written-in” by respondents as being used in addition to the surveyed tests. Some of the tests evaluated have limited normative data across certain patient populations and do not account for medication-related or practice effects. In addition, the literature on cognitive testing is extensive, and important findings for the reviewed tests may have been inadvertently omitted.

In an effort to maintain brevity, we only summarily reviewed the psychometric properties of individual tests, and we urge clinicians to refer to comprehensive neuropsychology texts as needed for additional information about individual tests. We also did not specifically address general cognitive tests such as the MMSE or MoCA or how to integrate them into an NBSE, both issues which are beyond the scope of this report. Our survey sample rate was relatively low (200/713, 28%) and only 44 of 200 respondents (22%) were UCNS certified in Behavioral Neurology & Neuropsychiatry. For the current report, the survey data were primarily used as a guide, along with the authors' consensus, for determining which tests would be reviewed. This study provides a glimpse into the current practice of formal and informal cognitive testing among behavioral neurologists and provides an evidence-based review and a test selection resource to help neurologists make informed choices about testing. A large-scale and broader survey of other subspecialty neurologists who perform cognitive assessment in various settings would complement this study. We also envision future studies that incorporate general screening tests and longitudinal follow-up. This review of single-domain cognitive tests serves as a starting point for quality improvement in clinical cognitive assessment.

GLOSSARY

AAN

American Academy of Neurology

AD

Alzheimer disease

BIT

Behavioral Inattention Test

BNS

Behavioral Neurology Section

BNT

Boston Naming Test

CERAD

Consortium to Establish a Registry in Alzheimer's Disease

FAB

Frontal Assessment Battery

FTLD

frontotemporal lobar degeneration

HVLT-R

Hopkins Verbal Learning Test–Revised

JLO

Judgment of Line Orientation

MMSE

Mini-Mental State Examination

MoCA

Montreal Cognitive Assessment

NBSE

Neurobehavioral Status Exam

ROCFT

Rey-Osterrieth Complex Figures Test

TBI

traumatic brain injury

TMT

Trail Making Test

UCNS

United Council for Neurologic Subspecialties

WAB-R

Western Aphasia Battery–Revised

AUTHOR CONTRIBUTIONS

Dr. Daffner led the AAN Behavioral Neurology Section Workgroup on the NBSE, directed one of the subgroups, and helped to draft the original manuscript, summarize/interpret the findings, and establish the final version. Dr. Gale was responsible for drafting the original manuscript and coordinating revisions of both the manuscript and the primary data supplement. Drs. Daffner and Gale take overall responsibility for the content of the manuscript. Dr. Kaufer was responsible for design and analysis of the survey, led one of the subgroups, and edited the final draft of the manuscript. Dr. D'Esposito helped with study concept and design. Drs. Barrett, Gitelman, and Hart each directed one of the subgroups, completing an evidence-based literature review and contributing to the content and revisions of the manuscript and data supplement. Drs. Boeve, Chatterjee, Coslett, Finney, Lerner, Meador, and Voeller, and Ms. Pietras helped to complete an evidence-based literature review and contributed to revision of the manuscript and data supplement.

STUDY FUNDING

This evidence-based review was completed on behalf of the Behavioral Neurology Section (BNS) of the American Academy of Neurology. No author received honoraria or financial support to develop this document.

DISCLOSURE

The authors report no disclosures relevant to the manuscript. Go to Neurology.org for full disclosures.