Clinical applications of neuropsychological assessment

Abstract

Neuropsychological assessment is a performance-based method to assess cognitive functioning. This method is used to examine the cognitive consequences of brain damage, brain disease, and severe mental illness. There are several specific uses of neuropsychological assessment, including collection of diagnostic information, differential diagnostic information, assessment of treatment response, and prediction of functional potential and functional recovery. We anticipate that clinical neuropsychological assessment will continue to be used, even in the face of advances in imaging technology, because it is already well known that the presence of significant brain changes can be associated with nearly normal cognitive functioning, while individuals with no lesions detectable on imaging can have substantial cognitive and functional limitations.

Introduction

Neuropsychological assessment is the normatively informed application of performance-based assessments of various cognitive skills. Typically, neuropsychological assessment is performed with a battery approach, which involves tests of a variety of cognitive ability areas, with more than one test per ability area. These ability areas include skills such as memory, attention, processing speed, reasoning, judgment, and problem-solving, spatial, and language functions. These assessments are commonly performed in conjunction with assessments designed to examine lifelong academic and cognitive achievement and potential,¹ for a variety of reasons described below. The assessment battery can be standardized or targeted to the individual participant in the assessment. Assessment data may be collected either directly by a psychologist or by a trained examiner, who performs and scores assessments and delivers them to the neuropsychologist. While neuropsychological assessments were originally targeted at individuals who had experienced brain injuries in wartime,² the populations for whom neuropsychological assessments are useful spans the whole range of neuropsychiatric conditions.³

Neuropsychological tests are intrinsically performance-based. They are structured to require individuals to exercise their skills in the presence of an examiner/observer. Self-reports of functioning, as well as observations of behavior while performing testing, are critically important pieces of information, as described below. Self-reports of functioning are often affected by the presence of neuropsychiatric conditions,⁴ and do not have the same value as performance under standard conditions, which is compared with normative standards. A critical concept in neuropsychological assessment is normative comparison.⁵ This involves taking the performance of an individual at the time they are tested and comparing that performance to reference groups of the same age, sex, race, and educational attainment. All of these demographic factors impact performance on the tests in a neuropsychological assessment battery, and interpreting the test performance of people, regardless of the illness or injury that they have experienced, is based on comparisons with individuals who are similar to them. These normative comparisons allow for determination whether an individual is performing as would be expected, given their lifetime levels of achievements and their educational attainment, or if their performance is poorer than expected. Performance that is poorer than expectations can be quantified and interpreted accordingly.

Definition of a meaningful cognitive deficit

Neuropsychological assessment provides both general and specific information about current levels of cognitive performance. An average or composite score across multiple ability areas provides an overall index of how well a person functions cognitively at the current time. As noted below, these global scores are the most reliable results of a neuropsychological assessment. These global scores are the indices most commonly used to predict real-world functional milestones and to make judgments about functioning in conditions where multiple ability domains are affected (eg, serious mental illness or traumatic brain injury).⁶

However, it is also important to be able to make judgments about specific differential deficits across ability areas. For instance, an individual who experiences a focal stroke or brain injury may have limited cognitive deficits, with most abilities unchanged. Thus, when making a judgment about the presence of a single cognitive deficit such as amnesia or a broader condition such as dementia it is critical to be able to identify exactly what a “differential deficit” would be. This judgment process is complicated by the fact that healthy individuals with no evidence of, or risk factors for, neuropsychiatric conditions show some variability across their abilities.⁷ As a result, it is important to consider several different factors when identifying normal variation between ability areas from neuropsychological deficits.

There are several factors that impact on within-individual variation across cognitive ability areas. These include the reliability of the measures, the normative standards for the measures, and the level of performance of the individual. Tests with less reliability produce more variable scores at both single assessment and retest. The discrepancies between ability areas that can be interpreted as truly different from each other also depend on whether the normative standards for the tests were developed in a single sample (ie, co-normed) or separately.⁸ For example, meaningful differences between individual subtests on intelligence tests such as the Wechsler Adult intelligence scales⁹ are smaller than differences between tests that were developed completely separately from each other, because of their co-norming on a single sample. Likewise, normative comprehensive standards for extended neuropsychological assessment batteries have also been developed with the same purposes in mind.¹⁰ Finally, extremes in performance, both higher and lower, lead to greater apparent discrepancies between ability areas. This is because that, at the tails of the distribution, smaller absolute score differences lead to larger normative differences.

In terms of interpretation of meaningful differences between abilities in neuropsychiatric conditions, a widely accepted rule of for a clinically meaningful difference between two ability areas is about one -half of a standard deviation.¹¹ This translates into about 7 IQ points and this level of difference has been shown to be detectable by observers. Specific, multiple studies have suggested that untrained observers can detect differences in functioning that occur over time that reach this threshold. As a result, treatment studies for cognitive impairments would not need to induce treatment effects smaller than this, because they might not be detectable.

It should be noted that the changes seen in many neuropsychiatric conditions are much more substantial than this 0.5 SD threshold. As a clear example, data regarding immediate memory changes, particularly rapid forgetting, at the outset of Alzheimer's disease (AD) are considerably more substantial than 0.5 SD. Data examining differences in performance across ability areas at the time of diagnosis has suggested memory performance about 3.0 SD below that of demographically similar healthy controls.¹² Further, differential deficits between abilities at the time of diagnosis are also substantial. In that same, very large-scale study, memory performance was about 2.0 SD below that of confrontation naming at the time of diagnosis.¹³ Although subtle differences can be detected by observers as described above, many of the differences between abilities in neuropsychiatric conditions are not subtle.

Conditions where neuropsychological assessment provides useful information

Situations where an illness or injury has the potential to adversely impact on cognitive functioning is one where neuropsychological assessment is indicated. These situations include illnesses or injuries that directly impact on cognition (Degenerative dementias or traumatic brain injuries) or where the treatment for the illness impacts on cognitive functioning (chemotherapy for breast cancer). Finally, as neuropsychiatric conditions are complex, many of them have the potential to induce changes in mood or motivational states that can have secondary impacts on cognitive functioning. As these secondary impacts can cause cognitive changes that are as just as real as those caused by a brain injury, part of a comprehensive contemporary neuropsychological assessment requires an assessment of other factors that may be contributing to impaired cognitive functioning.

Information obtained from neuropsychological assessment

There are several different uses for neuropsychological assessments. These include assessment for the purpose of diagnosis, differential diagnosis, prediction of functional potential, measuring treatment response, and clinical correlation with imaging findings. Some of these uses are related to each other and some are impossible in certain circumstances, because neuropsychological assessments do not provide information helpful for these tasks. These uses are presented in Table I.

TABLE 1. Uses of neuropsychological assessment.

• Diagnostic information for detection of dementias or other traumatic conditions
• Differential diagnosis of dementias vs less complex conditions
• Measurement of functional potential
• Course of degenerative conditions
• Measurement of recovery of functioning
• Measurement of treatment response

Diagnosis

Some conditions are defined by the presence of cognitive impairment. A prototypical example is dementia as defined by the DSM-TV-TR.¹⁴ Dementia requires the presence of functional deficits and cognitive impairments. These impairments must be in two domains: memory, and one other cognitive deficit. In contrast to dementia, amnesia, also defined in DSM-TV-TR, requires only the presence of memory deficits for its diagnosis.

For these conditions, therefore, neuropsychological assessment would serve to provide diagnostic information, because the presence of specific or multiple cognitive deficits, including memory, would provide information for a diagnosis. Similarly there are other conditions, such as postconcussion syndrome where the presence of cognitive impairments of various types is required as a part of the diagnosis. Further, mental retardation requires the presence of a certain level of current intellectual functioning that can only be obtained psychometrically.

The way the DSM-TV-TR is structured, however, there is no diagnosis that is confirmed simply as a function of the data obtained in a neuropsychological assessment. In the case of dementia, for instance, there are multiple additional criteria that must be met as well, and many of these pieces of information are obtained from other sources. These include history (eg, prior better levels of functioning), assessment of current adaptive deficits, and identification of a potential cause of the condition. As a result, neuropsychological assessments are only part of the diagnostic process.

Due to the way the DSM-TV-TR is set up, neuropsychological assessment does not provide information relevant to the diagnosis of most conditions where cognitive impairments are present. For example, many serious mental illnesses are marked by the presence of substantial cognitive impairments. Schizophrenia,¹⁵ bipolar disorder,¹⁶ and major depression¹⁷ have substantial cognitive deficits as a common feature of their presentation, even in patients with current minimal levels of symptoms. Since these impairments are not part of the diagnostic criteria, neuropsychological assessment does not provide diagnostically relevant information. As noted below, however, there is considerable information that can be obtained from neuropsychological assessments in these conditions, particularly in functional and prognostic domains.

Differential diagnosis

There are some conditions where neuropsychological assessment can be important for differential diagnosis. As noted above, dementia requires memory deficits in the presence of other cognitive impairments, while amnesia is diagnosed by the presence of only deficits in memory. Detection of multiple cognitive impairments would therefore rule out the presence of amnesia and argue for a diagnosis of dementia in this case.

Differential diagnosis is much more challenging for most conditions, however. For example, studies attempting to differentiate between dementing conditions of different etiologies, such as vascular dementia as compared with AD, have found little evidence of differential diagnostic utility from neuropsychological assessment.¹⁸ In fact, a fascinating book by Zakzanis et al¹⁹ that broadly approached this topic has suggested that for many conditions there is very little differential diagnostic information contained in a neuropsychological assessment that even allows for differentiation between healthy populations and patients with a variety of neuropsychiatric conditions. Their meta-analysis includes all of the research published on neuropsychological test differences between healthy controls and several neuropsychiatric target populations during the years 1980-1997. As a result, there is a wealth of detail on how much information each of these neuropsychological tests provides for test-based differential diagnosis of the target populations compared with healthy comparison subjects.

It is important in this area to consider the differences between differential diagnosis and statistically significant differences in performance across different conditions. An effect size of .6 SD in the difference of two means, by convention a large effect and easy to detect in samples as small as 20 individuals per group, is associated with 62% overlap between the two samples. In order to be able to tell with 90% certainty that an individual's test score is consistent with a psychiatric or neurological diagnosis and not part of the lower end of the distribution of healthy, an average difference of about 2.5 SD between populations is required.

Many statistically significant differences between samples would fare poorly as candidates for differential diagnosis. For example, people with schizophrenia routinely have more significant cognitive deficits than people with bipolar disorder, regardless of the mood state of the bipolar patients.²⁰ However, since bipolar patients themselves are more impaired in their cognitive performance than healthy people, there is substantial overlap in the distributions of cognitive performance between people with schizophrenia and bipolar disorder and minimal differential diagnostic information available. In contrast to the differences between people with AD and healthy populations on delayed recall memory, there is little discrimination between bipolar and schizophrenia populations. The distributions of patients with severe mental illness and healthy people have substantial overlap. As can be seen in Figure 1, there is considerable overlap in the distributions of scores on neuropsychological assessments of people with schizophrenia and healthy people, even if the means of the distributions are two full standard deviations apart. The r-BANS²¹ is an abbreviated neuropsychological assessment that examines multiple ability domains in a repeatable format. It is scaled like an IQ test, with a mean of 100 and standard deviation of 15 in healthy populations. As can be seen in Figure 1,²² people with schizophrenia have a mean level of performance that is 2.0 SD below that of healthy people (70 vs 100). However, half of the healthy population is performing within 2 SD of the mean of people with schizophrenia, and 35% of the people with schizophrenia perform within 2.0 SD of the mean of the healthy population. While a score of 115 would be much more rare for someone with schizophrenia than a healthy individual, a score of 85 would be at the 67th percentile for someone with schizophrenia and at the 17th for the healthy population; both of these are clearly within not outlying scores.

Figure 1. Normative data compared with a schizophrenia sample on the RBANS neuropsychological test. RBANS, Repeatable Battery for Assessments of Neuropsychological Status.

An additional intriguing result of the Zakzanis et al analyses is that many of the tests that are often described as capturing fundamental characteristics of illnesses such as schizophrenia fare relatively poorly when evaluated with differential diagnostic standards. For instance, the Wisconsin Card Sorting test/²³ a multidimensional test of executive functioning, is associated with 40% overlap between the performance of patients and healthy controls. In schizophrenia, in fact, the top five discriminators, all associated with 20% or less overlap, are in the domains of verbal and visuospatial memory. In the domain of chronic multiple sclerosis only 1 test is associated with less than 25% overlap between healthy individuals and MS patients, while many of the tests are associated with about 50% overlap between MS patients and healthy controls. These tests would provide essentially no data useful for differential diagnosis. There are some areas where there a number of excellent differential diagnostic candidates. In the domain of AD there are 15 different tests, all of memory, that are associated with less than 5% overlap between healthy controls and AD samples. Similarly, the difference between schizophrenia patients and AD patients on delayed recall memory was found to be similar to differences between healthy controls and AD patients.

Assessment of functional potential and the course of degenerative conditions

One of the more robust correlations in research in mental health is the association between cognitive performance and achievements in everyday functioning. This relationship has been appreciated for over 30 years and has been replicated across multiple neuropsychiatric conditions. Table II shows multiple examples of exactly this type of relationship. There are also several additional important points about these findings. These findings tend to be most robust for global aspects of cognitive performance, as indexed by performance on composite measures. In fact, in one recent study in severe mental illness the predictive power of a composite score for correlation with functional deficits was 2 to 3 times as great as any individual neuropsychological measure.³⁰ Similarly, functional deficits in AD are more severe and debilitating after the illness has progressed, and there are multiple cognitive processes affected. Although it is quite possible to have functional deficits originating from a single residual cognitive deficit, on average more wide-ranging cognitive deficits, even if moderate in nature, leader to broader functional deficits. There will always be individual cases where a single, apparently delineated, cognitive deficit leads to gross impairment in functioning.

TABLE II. Neuropsychiatric conditions where cognitive functioning predicts everyday functioning.

• Reduced cognitive impairment post TBI predicts greater potential for functional recovery25
• Progression of cognitive impairment leads to functional decline in Alzheimer's disease26
• Cognitive impairments predict everyday functional deficits in people with schizophrenia27
• Cognitive impairments in schizophrenia and bipolar disorders have nearly identical relationships with everyday functioning28
• Cognitive impairments in Parkinson's disease are associated with functional deficits consistent with dementia29

The most important clinical implication of what we know about cognition and functioning is this: when individuals affected by a neuropsychiatric condition are found to have current cognitive abilities congruent with pre-illness functioning they are least likely to have functional deficits. This is particularly true in conditions such as HIV neuropathology³¹ or traumatic brain injury (TBI)³² where changes can occur in the context of unimpaired previous functioning. Multiple studies of TBI have also have shown that recovery of cognitive functioning predicts recovery of everyday functioning much more efficiently than measures of the “severity” of the injury and some studies of TBI have had some success in the identification of the most efficient predictors of recovery of functioning. They tend to be from the domains of executive functioning and processing speed, but some studies also suggest that memory measures may be important (see ref 33, p 12).

It has proven difficult to establish absolute standards for how much impairment in cognitive functioning will definitely lead to functional changes. In addition, the search for specific cognitive to functional relationships has also proven challenging in conditions other than TBI. The group average data do suggest some general guidance, but clinical prediction will require analyses of specific cases. What is clear, however, is that neuropsychological assessment is an excellent tool for the prediction of recovery.

Assessment of changes in cognition in progressive degenerative conditions requires a different approach than required for the initial diagnosis of dementia or the assessment of improvement following TBI. If delayed recall performance is at a level that is close to 0 at the time that dementia is detected, this ability will not be a feature of the illness with the potential to change over time. In fact, research comparing individuals with AD at different levels of illness duration (and progressive course) have suggested that there is a pattern of progression in the worsening of cognitive impairments, with delayed recall nearly completely absent at the time of diagnosis, with other changes occurring in close temporal proximity, including reductions in rate of learning, executive functioning, and processing speed. Later on in the course, changes in longterm memory such as confrontation naming are detected and spatial and perceptual deficits become more severe.^12-13 These changes are not necessarily uniform or predictable for individual cases and many individuals will manifest impairments in one ability area that are more severe than expected by their current stage of illness. What is clear from research, however, is that in individuals with AD and considerable cognitive impairments, functional performance tends to worsen quite markedly.

Measurement of recovery of functioning and treatment response

There is major interest in treatment of cognitive deficits in degenerative conditions, attention-deficit disorder, and severe mental illness. These approaches have ranged from in person and computerized cognitive remediation efforts to multiple pharmacological interventions. It makes sense that the same measures of cognitive functioning used to identify functionally relevant deficits across different neuropsychiatric conditions would be used to measure treatment outcomes. This approach has been used in multiple different studies, although there are some issues that require attention in interpreting the results of the studies. These include changes in performance that are due to random variation and practice effects and the fact that certain cognitive measures are more vulnerable to these effects than others, limiting their utility as outcome measures. One of the things that will render neuropsychological assessment consistently important is the new development of rehabilitation therapies. Development and marketing of computerized cognitive remediation interventions has not always been accompanied by the systematic assessment of their efficacy and long-term usefulness. It seems likely the performance on structured neuropsychological measures will continue to be the gold standard for selection of patients for these interventions and evaluation of their efficacy.

One of the strategies that has been developed to understand “real” cognitive improvements vs psychometric artifacts is the “reliable change index (RCI)” method.³⁴ The RCI adjusts for expected practice effects and unreliability of measures in order to develop an index of change on an individual basis that would be definitely non-random. Essentially, a statistic is calculated that takes test scores at two different times and examines the difference between them, establishing a range of scores that could be attributed to practice effects or unreliability of measures. Differences that exceed this range are then considered to be reliable. Thus, measures with greater test-retest reliability and smaller practice effects in healthy controls would be better candidates for detection of small amounts of change that would still be clinically meaningful. Previous results in severe mental illness have suggested that changes in typically administered cognitive assessment batteries would need to be in the vicinity of 1.0^35-36 standard deviations on the part of an individual patient to be nonrandom, suggesting that quite substantial improvements may be required with current instrumentation.

Reduction, or at least the clear recognition, of practice effects is an important goal, because large practice effects in treatment studies on the part of the patients in the inactive treatment group can make it impossible to detect change in the treatment group.³⁷ Certain measures are particularly vulnerable to such effects, and some of them may actually change in their characteristics upon repeated administration. Episodic memory tests are particularly vulnerable to practice effects, because of the possibility of learning of the content. However, it is critical to have alternate forms of such measures be closely equivalent, because if the alternate forms are different in their difficulty, an apparently improvement effect can be spuriously detected. Problem-solving tests are quite vulnerable to changes with retesting, because if there is only one problem, like in the widely used Wisconsin Card Sorting Test, once it is solved the test is no longer a problem-solving test. As a result, systematic efforts to develop problem-solving tests with similarly problems (like mazes) but with alternative stimuli have been conducted.

One of the major issues in using neuropsychological assessment as a sole outcome measure to measure either spontaneous recovery or treatment response is the lack of definitive information as to how much change is required to be important. In a sense, this is the converse of how much worsening due to illness or injury is significant, because both are equally hard to define without additional reference points. For an adequately powered randomized trial, separation of active treatment from inactive treatment is certainly one standard; one that will be applied by regulatory agencies. Another perspective is the empirically derived standard described above a ½ standard deviation improvement as having clinical meaning. A third strategy, which is optimal in certain circumstances where it can be applied, is that of using concurrent assessment of functional outcomes. As improvement in functioning is the goal of treatment of cognition, whenever possible improvements in functioning occur, accompanying cognitive improvements should be measured.

For instance, in a study of cognitive remediation in schizophrenia published a few years ago, the level of improvement in neuropsychological test performance on the part of patients was less than 0.5 SD compared with the inactive treatment group.³⁸ However, the patients who received cognitive remediation were able to work much more effectively and earned more than 10 times as much money in the ensuing 3-year follow-up period compared with patients randomized to the inactive treatment.³⁹ Thus, the cognitive improvement seen must have been adequate for some patients, in order for them to achieve such substantial functional gains.

The above study is different from many other studies because of its duration and because of the fact that patients who entered were all receiving a psychosocial intervention: supported employment. Such concurrent interventions have been shown to be a prerequisite for functional gains in cognitive remediation studies in severe mental illness.⁴⁰ In studies where treatments are offered for briefer periods, such as pharmacological efficacy studies, or in cases where patients are not receiving concurrent psychosocial interventions, such outcome measures would not be practical. A suggested approach has been to use performance-based measures of functional capacity,⁴¹ which have shown considerable validity in terms of prediction of everyday outcomes and sensitivity to functional decline in very elderly patients with severe mental illness. These measures, because they capture ability and not everyday outcomes, do not require environmental opportunities to perform skills and have been shown to be sensitive to the effects of short-term behavioral interventions.

Clinical correlation

Among the exciting developments in medical technology has been the advent of high-resolution structural and functioning imaging of the brain. These techniques allow for highly precise examination of lesions associated with TBI and stroke, They also can identify potentially dangerous vascular abnormalities which may be repaired before catastrophic ruptures. Also possible is the visualization of previous “silent” ischemic changes, strokes, and other potential lesions. With the advent of ligands that can label amyloid;⁴² it will also likely be the case that many individuals will be informed that they have substantial potential to experience degenerative changes. A major question that arises after detection of any such a brain change is whether there is any functional importance of these changes. Given the consistent findings that cortical degenerative changes are often found at postmortem in individuals who had no observational evidence of deteriorated cognitive functioning during life,⁴³ there will be considerable need to perform cognitive assessments following such scans. Similarly, serial neuropsychological assessment will likely provide better (and cheaper) information about changes in cognitive functioning than repeated scans.

Conclusions

Neuropsychological assessment has multiple clinical applications, ranging from collecting diagnostic information for dementia to predicting functionality and recovery from TBI. These assessments are not likely to be replaced by technology, because of the issues, reviewed immediately above, regarding the lack of clear prediction of cognition and functioning from cortical changes in late life. Neuropsychological testing does not provide differential diagnostic information for neuropsychiatric disorders, but it provides information that cannot be obtained anywhere else on abilities, motivation, and potential for future outcomes. There are likely to be new advances in assessment technology, but not assessment philosophy, over time, These improvements may include validly deliverable remote assessments and increased ease of administration of assessment tools. At this time, neuropsychological assessment has many uses and adds critical information to psychological, neurological, and neuroimaging assessments.