Neuropsychological Assessment: Past and Future

Abstract

Neuropsychological assessment tools are the staple of our field. The development of standardized metrics sensitive to brain-behavior relationships has shaped the neuropsychological questions we can ask, our understanding of discrete brain functions, and has informed the detection and treatment of neurological disorders. We identify key turning points and innovations in neuropsychological assessment over the past 40–50 years that highlight how the tools used in common practice today came to be. Also selected for emphasis are several exciting lines of research and novel approaches that are underway to further probe and characterize brain functions to enhance diagnostic and treatment outcomes. We providea brief historical review of different clinical neuropsychological assessment approaches (Lurian, Flexible and Fixed Batteries, Boston Process Approach) and critical developments that have influenced their interpretation (normative standards, cultural considerations, longitudinal change, common metric batteries, and translational assessment constructs). Lastly, we discuss growing trends in assessment including technological advances, efforts to integrate neuropsychology across disciplines (e.g., primary care), and changes in neuropsychological assessment infrastructure. Neuropsychological assessment has undergone massive growth in the past several decades. Nonetheless, there remain many unanswered questions and future challenges to better support measurement tools and translate assessment findings into meaningful recommendations and treatments. As technology and our understanding of brain function advance, efforts to support infrastructure for bothtraditional and novel assessment approaches and integration of complementary brain assessment tools from other disciplines will be integral to inform brain health treatments and promote the growth of our field.

INTRODUCTION

The goal of this review is to highlight key developments and turning points in neuropsychological assessment over the past several decades. We will touch on a breadth of assessment-related topics to provide an overview of the evolution of neuropsychological evaluation, leading to consideration of current practices and areas for future research. One limitation is that the content here is selective and admittedly draws heavily from practices based in the United States; while a majority of published neuropsychological tests and test norms have been developed within the United States and primarily for English-speaking populations, such cultures actually contain a disproportionately small fraction of the world’s population. Still, the accomplishments and progress of neuropsychological assessment to date are remarkable, and we hope to inspire continued growth through this review. By its nature, neuropsychology is a distinctly transdisciplinary service. Neuropsychologists are in the unique position to be able to interpret brain measurement across modalities in a clinically meaningful way (e.g., neuroimaging, proteomics, genetics, technology-based service delivery). Leveraging the systematic and foundational approaches of our field, neuropsychologists are encouraged to continue to expand their repertoire and impact by creatively integrating brain assessment techniques across biological, cognitive science, and technology-based platforms as the field continues to evolve and define itself.

CLINICAL NEUROPSYCHOLOGICAL ASSESSMENT

Although the roles of neuropsychologists have evolved over time, the primary purposes for clinical neuropsychological assessment have remained fairly constant, to: (1) detect neurological dysfunction and guide differential diagnosis,(2) characterize changes in cognitive strengths and weaknesses over time, and (3) guide recommendations regarding everyday life and treatment planning. With the advent of increasingly sensitive and multimodal neurologic biomarker data, neuropsychological assessment shifted from its original role in “finding the lesion” to in-depth characterization of the patterns arising from disruptions in brain-behavior relationships. Although perhaps taken for granted given the body of gold-standard measures available at this stage in the field, advancement of cognitive assessment tools that are (1) sensitive to brain–behavior relationships of interest, (2) developed alongside other brain measurement tools, and (3) accessible and feasible across diverse settings continue to be needed areas of study. Ultimately, the neuropsychological questions we ask are only limited by the tools with which we have to probe the brain.

NEUROPSYCHOLOGICAL ASSESSMENT APPROACHES: A BRIEF MODERN HISTORY

Lurian Approach

Alexander Luria’s seminal work following World War II in Russia represents an important early time-point in neuropsychological assessment. Luria’s innovative ideas that the brain underlies the ability to carry out goal-directed behaviors and is shaped by environmental and cultural contexts (e.g., language) led to his systematic characterization of functional brain systems (Luria, 1966). Although largely qualitative, Luria’s primary goal was to describe the cerebral bases that support corresponding functional systems, emphasizing the importance of understanding the multiple components that may comprise even simple neurobehavioral functions (Luria, 1966). Luria stimulated appreciation of brain specialization and moved away from the one-size-fits-all diagnosis of brain disease; however, his techniques were highly flexible and nonstandardized, making it difficult to reliably reproduce across examiners and patients.

Recognizing this problem, his student, Anne-Lise Christensen, published a more structured version of the Lurian approach, combining both qualitative and quantitative aspects of his battery (Draper, 1976). Ultimately, Charles Golden further standardized and combined the works of both Luria and Christensen in his development of the Luria-Nebraska Neuropsychological Battery (LNNB; Golden, Purisch, & Hammeke, 1979), scaling down almost 2000 original measurement items to 269 items covering 14 scales (e.g., motor, rhythm, memory, intelligence; Purisch, 2001).

In the United States, other neuropsychological assessment standardizations were also developing. In the 1950s, Arthur Benton was among the first thought leaders to criticize neurology for the lack of validated tools to measure common neurological syndromes (e.g., aphasia, agnosia). Through his pursuit of systematic neurobehavioral assessment, Benton developed several individual measures that are still widely used today (e.g., Benton Visual Rention Test) and shaped the Iowa-Benton school of neuropsychology (Tranel, 2009). Additionally, through these test developments, Benton began raising awareness of the apparent import of demographic factors (e.g., age and education) on test performances.

Flexible Battery

One assessment style that naturally extended from clinicians’ early instincts to describe observed behaviors was the flexible integration of standardized measures. In the pure flexible battery approach, neuropsychologists administer only those measures directly related to the patient’s presenting symptoms (e.g., multiple memory measures to characterize memory symptoms). Consequently, the administered battery inherently changes with a patient’s referral question, while the examiner pursues the aim of being both as efficient and sensitive as possible to the presenting problem (Benton, 1994; Lezak, 1976).

Fixed Battery Approach

Concurrently, an entirely distinct assessment approach was developing that used a common, relatively comprehensive set of measures administered to all patients regardless of presenting symptoms, neuromedical history, or clinically apparent syndrome. Although lengthier and requiring greater burden and expense, fixed batteries maintain a constant testing condition, allowing for direct comparison of measures across disparate neurological disorders. Given their highly standard nature, research regarding fixed batteries is more easily facilitated, has garnered more comprehensive normative data, better understanding of their psychometric properties, and a broader empirical base from which to draw interpretations.

Two major leaders in the history of neuropsychological assessment pioneered the fixed battery approach, Ward Halstead and Ralph Reitan, his student (Reitan & Davidson, 1974). As a physiological psychologist in the mid-twentieth century, Halstead espoused a philosophy of neuropsychological assessment that was akin to a series of scientific experiments. He believed that there was no science in the individual event, and emphasized the need to develop exact, systematic procedures with sufficient comparison cases to interpret an individual test score (Reitan, 1994). Reitan similarly held strong beliefs that empiricism should be central in neuropsychological assessment (Grant & Heaton, 2015).

Building off of Halstead’s early work, Halstead and Reitan closely observed individual patients in clinic and at home, helping to shape and refine measures that captured the observed brain–behavior relationships that could be generalized more broadly. Backed by more than two decades of blind interpretation of patient results and studies of groups with diverse neurologic disorders, Reitan ultimately published the Halstead-Reitan Battery (HRB) in 1985. The goal of the HRB was to be a systematic, quantitative means to measure the presence, location, extent, and nature of neurological disease equitably across clinical syndromes (Reitan, 1985). With its replicable procedures and highly quantitative results, the HRB intended to shift the practice of neuropsychology from an “art to a science” (Reitan, 1985) and indeed has been one of the most researched assessment batteries in neuropsychology (Kreutzer, DeLuca, Caplan, 2011).

As part of the trend toward highly quantitative assessment in the 1970s, some early approaches attempted to develop computerized algorithms for neuropsychological interpretation: the Key Approach by Russell, Neuringer, and Goldstein (1970), Finkelstein’s BRAIN (1977), and Adams’ ability-based algorithm (Adams, 1975). An overarching goal of these automated programs was to leverage the predictive power of computers to more accurately localize, categorize, and potentially infer etiology of brain damage such that “a technician or clerk, without knowledge of neurology, neuropsychology, or psychometrics” could use these tools (Deuel, 1971, p. 95). Although forward-thinking, the accuracy of such automated systems was inconsistent. The programs reliably identified presence of brain injury, but inferences regarding localization and etiology were imprecise and led to the conclusion that contextual information(i.e., clinical history) was critical to neuropsychological interpretation (Adams, Kvale, & Keegan, 1984).

Boston Process Approach

Another important turning point in neuropsychological practices was the development of what is now referred to as the “Boston Process Approach” led by Edith Kaplan (Kaplan, 1988). Although the Boston approach can be used with either flexible or fixed batteries, it is most commonly associated with and perhaps conceptually suitable to more flexible evaluations. The Boston approach places emphasis on how the patient comes to an answer (e.g., types of errors committed) rather than reliance on a single objective score. In this approach, testing the limits of a patient’s cognitive abilities to elicit behaviors that may not traditionally present during standardized testing is emphasized.

Through the influence of Kaplan’s training, Dean Delis advanced the standardization of the Boston approach, along with Kaplan and Joel Kramer, into what have come to be some of the most widely used neuropsychological assessment tools today. The California Verbal Learning Test (CVLT, Delis, Kramer, Kaplan, & Ober, 1987; and CVLT-second edition, Delis, Kramer, Kaplan, & Ober, 2000) and the Wechsler Adult Intelligence Scale-Revised as a Neuropsychological Instrument (Kaplan, Fein, Morris, Delis, 1991), followed by the Delis Kaplan Executive Functions System (DKEFS; Delis, Kaplan, Kramer, 2001) were borne out of a need to provide statistical parameters quantifying cognitive strategies that deviated from expectations (e.g., CVLT primacy effects; DKEFS error analysis).

Flexible Evaluation Approach

To avoid misconceptions, the pure flexible battery differs from the flexible evaluation approach that is used by over three-quarters of contemporary clinical neuropsychologists (Larrabee, 2008). Current common practice represents aspects of both flexible and fixed approaches, such that a fairly standardized, fixed set of measures is given to most patients with some flexibility to add or subtract measures given the specific referral question (Bigler, 2007).

NORMATIVE STANDARDS

As these fundamental assessment tools developed, there also grew increasing recognition that nonneurological, premorbid factors were significantly impacting test scores. The concept that cultural environment shapes brain development dates back at least to Luria (Luria, 1966). Mounting evidence in the 1970–80s then quantified that >40% of the variance in many test scores was accounted for by a single demographic factor (e.g., age, education), while even greater amounts of normal variance could be adjusted for when combing demographic factors together (Heaton, Grant, Matthews, 1991). Yet, development of a systematic way to adjust for such pre-morbid factors lagged.

While initial normative efforts focused on the effects of age alone (e.g., Wechsler measures), Robert Heaton, one of Reitan’s students, recognized this gap and pioneered the development of normative standards adjusting for multiple background demographic factors. Publication of the “Heaton Norms” for the expanded HRB in 1991 signaled the most comprehensive standards at the time, adjusting for effects of age, sex, and education (Heaton et al., 1991). His subsequent revision in 2004 incorporated the effects of race, integrating the increasing literature demonstrating its influence on test performance (Heaton, Miller, Taylor, & Grant, 2004).

His advocacy to apply race as a proxy for important socio-cultural factors that impact brain development (e.g., education quality, socioeconomic status, medical care access, testing acculturation) represents a pivotal point in assessment approaches that at times is still received as controversial today (see detailed discussion by Manly, 2008). Heaton was not alone in addressing the need: in parallel, the Mayo Clinic, led by Robert Ivnik and Glenn Smith, was carrying out the Mayo Older Americans Normative Studies (MOANS), which aimed to improve the utility of commonly used neuropsychological tools for older adults (Ivnik et al., 1992a, 1992b). The MOANS project began publishing normative standards for various measures in 1992, and continued to update these standards through the early 2000s, including adjustments for racial minorities and careful consideration of what is considered neurologically normal in the elderly (e.g., Harris, Ivnik, & Smith, 2002; Lucas et al., 2005; Machulda et al., 2008; Smith, Wong, Ivnik, & Malec, 1997).

Other novel normative advancements were the development of co-normed batteries (e.g., WAIS-IV, WMS-IV, CVLT-II) and “robust norms” (Zhu & Tulsky, 2000). Co-normed batteries allow for test score interpretation on a common metric that is adjusted for the same demographic factors from the same normative cohort. Thus, they provide an equitable standard against which performances on all tests can be compared and interpreted. Robust normative standards simply refer to the use of as neurologically normal individuals as possible to provide standards of comparison. This may be particularly relevant in adult standards in which underlying neurodegenerative pathology may accumulate decades before clinical manifestation (Holtzer et al., 2008). Inclusion of preclinical but asymptomatic individuals in a normative cohort may increase representativeness, but it also increases variability and decreases the sensitivity to acquired injury or disease. Nevertheless, there are many practical difficulties that limit feasibility in identifying highly neurologically normal individuals for inclusion in large-scale normative studies (e.g., expense in accessing brain scans and comprehensive family histories).

Despite major growth in normative development in the past two decades, many other interesting and potentially important questions about how to operationalize “normal” brain functioning remain unanswered. For example, do age effects on cognition differ across presumably “normal” groups based upon educational backgrounds, cognitive reserve, access to quality healthcare, nutrition, lifestyle factors, or environmental risks? Do education effects differ on the basis of how resourced the person’s educational system has been, how education was valued in the home, and what other stresses in the environment may have interfered with educational pursuits?

For example, Jennifer Manly’s innovative work parsing out contributing factors that account for racial differences on neuropsychological testing demonstrates that quality of education may be a critical underlying differentiator in these testing disparities (Manly et al., 2002). Other questions include, will gender effects on test performance differ based upon the society’s traditional gender roles, gender-based opportunities and expectations? Also, if such effects of background factors change over time, how will this inform the need for updated normative standards, or perhaps even generation-based norms that reflect different effects of age, education, gender, and race/ethnicity? Answers to these questions now, and how they may change across time with shifting cultural values (e.g., gender/minority equality programs), may importantly impact our expectations and interpretation of neuropsychological test performances in the future.

CULTURAL AND INTERNATIONAL CONSIDERATIONS

Relatedly, during the past several decades, there has been a dramatic shift in the demographics of the United States, especially including individuals who speak languages other than English and who were born, grew up, and were educated (in part or totally) in cultures outside of the United States. Converging evidence demonstrates that test performance is significantly influenced by the values, customs, experiences, and cognitive styles that differ from the majority culture on which tests were originally developed and standardized (e.g., Arnold, Montgomery, Castaneda, & Longoria, 1994). Although their overlap and individual contributions are less clear, a host of culture-specific factors are undoubtedly associated with test scores across populations even within a single country (e.g., United States): acculturation, language, education quality and literacy, poverty and low socioeconomic status, familiarity with the evaluation process, and communication style (Fuiji, 2017).

As a rule of thumb, the larger the discrepancy between the individual being assessed and the majority culture in which the measure was developed, the higher the possibility that the test score may not reflect the construct it is posited to measure. Even when a measure is developed in the patient’s native language and for examinees with the same racial/ethnic background, variability within a culture may still significantly influence test scores. For example, a recent publication demonstrated significant effects of frequency of everyday Spanish language use and location of education and birth among healthy U.S. Hispanic adults, even on a battery developed and normed in a national Hispanic sample (Flores et al., 2017).

These issues are further underscored and potentially amplified when neuropsychological measures developed in American culture are applied in non-U.S. contexts. The importance of the basic human ability being assessed may differ across cultures and be dependent on the values and everyday requirements of a given culture. For example, in the majority U.S. culture, speeded information processing is valued and educational experiences frequently reinforce that faster performances indicate better results. However, in Hispanic culture, speed and quality are oftentimes contradictory goals, such that slow and careful processing is thought to lead to the best results (Ardila, Rosselli, Matute, & Guajardo, 2005).

Some authors have suggested that what are needed in developing countries are tests that are more relevant to the backgrounds of those populations. However, it is unknown whether changes in the tests themselves would make them more valid in other settings, or whether all that is needed are different normative standards. Nonetheless, it may be that some tests simply are not useful across settings whereas others are, and we do not yet know what factors may drive these differences. Similarly, especially in the large and diverse developing world, it is unclear whether normative standards in one country or setting can be generalizable to other settings and, if they can, what common factors account for that generalizability. These are all questions to which neuropsychology researchers still need to seek answers. The need for valid neuropsychological practice and research is worldwide and it is simply not feasible to develop population-specific test norms, or new tests for every population on earth.

Currently, measures developed in the United States and Europe are commonly applied in other cultural contexts, although there are several notable efforts for culture-specific test and normative developments. As one example, Egypt has been at the forefront developing standardized Arabic evaluations, beginning as early as the 1920s (led by El-Kabbani) and extending into commonly used batteries developed in other cultures (e.g., Halstead-Retain and Luria-Nebraska; Al-Joudi, 2015). A recent review by Fasfous, Al-Joudi, Puente, and Perez-Garcia (2017) identified that, although 117 individual neuropsychological measures were used in studies examining Arabic populations, only 53 were normed in the culture and evidenced appropriate cultural adaptation and validation per standard guidelines. Notably, almost all of the identified measures were tools initially developed in the United States and Europe: Verbal Fluency, Wechsler Memory and Intelligence batteries, Trail Making Test, Wisconsin Card Sorting Test, Ravens Matrices, and Bayley Scale of Infant Development-second edition were among the most extensively used cognitive measures in contemporary Arabic countries (Fasfous et al., 2017).

Other efforts to develop normative standards outside of American and European cultures suggest comparability of the construct measured when normative standardizations and underlying issues of cultural validity are appropriately addressed (e.g., Zambia, Hestad et al., 2016, Kabula et al., 2017; China, Gupta et al., 2014, Heaton et al., 2008, Shi et al., 2015; Brazil, de Almeida, et al., 2013; India, Ghate et al., 2015, Kamat et al., 2012, 2017, Malda, van de Vijver, Srinivasan, Transler, & Sukumar, 2010; Czech Republic, Bezdicek et al., 2012, 2014; South Africa, Nell, 1999; Nell, Myers, Colvin, & Rees, 1994; Cameroon, Ruffieux, et al., 2010; South Korea, Ko, Rosen, Simpson, & Brown, 2014), although there are some exceptions (e.g., lexical fluency and block design in Cameroon, Ruffieux, et al., 2010).

A common theme in reports of these cross-cultural test adaptations is the nonequivalence of the testing experience from Western to non-Western environments (e.g., Cassitto, Camerino, Hanninen, & Anger, 1990; Valciukas, Levin, Nicholson, & Selikoff, 1986). A host of factors, including repetition of test instructions, in-depth orientation to the testing environment, performance motivation, and profound effects of vocation and socioeconomic status have been identified as potentially contributing to test performance (Weinstein, Fucetola, & Mollica, 2001). As the world becomes increasingly interconnected and the need to assess and treat culturally different individuals grows, our field and its journals have an ethical responsibility to increase their priorities for such cross-cultural research endeavors.

SCIENTIFIC ADVANCEMENTS IN NEUROPSYCHOLOGICAL ASSESSMENT

Techniques for Defining Longitudinal Change

Neuropsychological batteries were originally constructed with the goal of identifying brain dysfunction, but a prominent emerging role of the neuropsychologist is to monitor syndrome progression or recovery via repeated evaluations. This is particularly true in the rehabilitation setting. As such, quantifying what constitutes significant change on a test battery is another relatively recent advance in our empirical assessment approach. Among the first and still widely used methods is the Reliable Change Index (RCI; Jacobson & Truax, 1991). The RCI was developed by Jacobson and Truax from the personality literature to establish whether the difference between an initial and follow-up test score exceeds what can be attributed to chance variation versus a clinically significant change. The RCI is based upon the reliability of the individual measure, which can be used to estimate the standard error of the difference between scores on the test in question.

Several years later, Chelune, Naugle, Luders, Selak, and Awad (1993) developed change modeling that additionally adjusted for practice effects, and McSweeney, Naugle, Chelune, and Luders (1993) proposed a linear multivariate regression approach adjusting for demographic effects and baseline performances to additionally control for regression to the mean. In 1996, similar regression models developed by Sawrie et al. and Hermann et al. included test–retest intervals (Standardized Regression Based norms) to quantify cognitive changes following epilepsy surgeries (Hermann et al., 1996; Sawrie, Chelune, Naugle, & Luders, 1996). These methods along with more complex multiple regression-based modeling including baseline scores, other potentially contributing factors (e.g., retest interval, demographics, nonlinear effects), and their interactions were then directly compared by Temkin, Heaton, and colleagues in neurologically normal and clinical groups (Heaton et al., 2001; Temkin, Heaton, Grant, & Dikmen, 1999).

Of interest, the authors found that the simple RCI was the least accurate method while the other approaches appeared comparably accurate. However, regardless of approach, baseline performance on individual tests had the largest impact on change classification, especially in clinical cohorts, and should be accounted for when developing change modeling (see Cysique et al., 2011 for further support and importance of “neuropsychological competence” predicting longitudinal cognitive change).

These studies laid the groundwork for the Serial Assessment report from the Wechsler Advanced Clinical Solutions software and the development of other calculators and published regression equations aiding in our quantification of cognitive “impairment” and change scores. For example, Crawford and colleagues leveraged existing datasets to develop enriched regression-based predictions of cognitive functioning for the individual case, which may also be applied longitudinally (Crawford & Garthwaite, 2007; Crawford, Garthwaite, Denham, & Chelune, 2012). Such programs that use the multitude of existing datasets to answer ongoing neuropsychological questions are innovative and will help optimize the growth of our field.

Common Metric Assessment Batteries

In an effort to leverage data and combine scientific efforts, development of standardized methods to evaluate cognition comparably across studies has been a major recent goal of the U.S. National Institutes of Health (NIH). Beyond answering innumerable scientific questions with greater power, such tools are particularly well-positioned for multisite epidemio-logical studies and clinical trial monitoring. With these goals in mind, several novel common metric batteries were developed that are population-specific (Measurement and Treatment Research to Improve Cognition in Schizophrenia, MATRICS; Nuechterlein et al., 2008), construct-specific (Executive Abilities: Measure and Instruments for Neurobehavioral Evaluation and Research, EXAMINER; Kramer et al., 2014), and address broader, lifespan neurobehavioral assessment questions (NIH Toolbox for Assessment of Neurological and Behavioral Function; Gershon et al., 2013).

While these batteries represent massive accomplishments in the standardized assessment of neurologic function, and potential turning points in our resulting future understanding of brain function, they still maintain the common pitfalls of our traditional set of tools. Namely, although large infrastructure and broad expertise were dedicated to the theoretical and empirical development of these measures, efforts toward gathering sufficient normative information across populations, languages, and especially minority cultures, is ongoing and difficult, and generalizability of results with each battery remains to be demonstrated.

Novel and Translational Constructs

Although the traditional set of cognitive domains are an important foundation in the neuropsychological toolset (e.g., episodic memory, language, visuospatial skills), several domains drawing on the cognitive psychology and neurosciences literatures are highly complementary and warrant mention. These constructs have significantly furthered our capture of brain–behavior relationships and their assessments have demonstrated strong psychometric properties. Application of such measures depends entirely on the investigator or clinician’s aims (e.g., if aiming to measure interpersonal function, social cognition is highly appropriate), although we encourage readers to consider such constructs for inclusion in their neuropsychological toolkit. We note only several exemplars here and encourage reading across related fields for further inspiration (e.g., Carter & Barch, 2007; Merikle, Smilek, & Eastwoord, 2001).

Action Fluency

Although not a novel cognitive construct, action fluency (ability to generate verbs, for example, “things that people do”) was developed as a counterpart to the traditional lexical and semantic fluency tasks. Action fluency taps into a more selective frontal-striatal network compared with the more temporal networks involved in noun generation (Piatt, Fields, Paolo, & Troster, 1999). Clinical lesion studies demonstrate a double-dissociation such that patients with frontal versus temporal lobe injury demonstrate disproportionate difficul-ties generating verbs than nouns, and vice versa (Damasio & Tranel, 1993). Similarly, more recent work in Parkinson’s and HIV diseases (diseases with known selective involvement of frontal-striatal networks) show disproportionate disparities on verb compared to lexical or semantic generation tasks that is importantly predictive of daily functioning outcomes (Piatt, Fields, Paolo, Koller, & Troster, 1999; Woods, Carey, Troster, Grant, & Centre, 2005; Woods, Scott, et al., 2005; Woods et al., 2006).

Prospective Memory

Prospective memory (PM) is the ability to remember an intention in pursuit of a future goal. PM necessitates both executive (e.g., planning, monitoring, set-shifting) and episodic memory (e.g., recalled intention) abilities, drawing on frontal-temporal systems and their related networks (Ellis & Kvavilashvili, 2000). Although initial studies date back to Elizabeth Loftus in 1971, the past decade has seen a surge of renewed interest in the neuropsychological study of PM (Raskin, 2004). Prospective remembering declines across a range of neurologic conditions (e.g., HIV, Carey et al., 2006; schizophrenia, Twamley, Woods, Dawson, Narvaez, & Jeste, 2007; Parkinson’s disease, Kliegel, Altgassen, Hering, & Rose, 2011; even normal aging, Einstein & Mcdaniel, 1990) and is consistently associated with declines in everyday functioning, even more so than traditional episodic memory measures (Einstein & Mcdaniel, 1990; Kliegel et al., 2011; Woods et al., 2007, 2008, 2009; Zogg, Woods, Sauceda, Wiebe, & Simoni, 2012).

Social Cognition

Social cognition was originally identified through the social psychology literature and is a multifaceted construct referring to the ability to perceive, interpret, and generate responses to the intentions and behaviors of others (see McDonald, 2017, for a comprehensive review; McDonald, Flanagan, & Rollins, 2011; McDonald, Flanagan, Rollins, & Kinch, 2003). Examples of social cognition tasks include identifying emotions, theory of mind (ability to understand and attribute others’ mental states), and knowledge of social pragmatics (Green, Olivier, Crawley, Penn, & Silverstein, 2005).

Although its early and most in-depth neuropsychological study is largely in the context of schizophrenia-related adaptive functioning (e.g., NIMH MATRICS), there has been an explosion of studies in the last several years recognizing the primary social cognition impairment across neurologic conditions (e.g., behavioral variant of frontotemporal dementia, Shany-Ur & Rankin, 2011; Autism spectrum disorder, Pinkham, Hopfinger, Pelphrey, Piven, & Penn, 2008; Alzheimer’s disease, Bediou et al., 2009; substance use disorders, Homer et al., 2008). Importantly, social cognitive measures are more strongly associated with community functioning outcomes than traditionally assessed constructs (Pijnenborg et al., 2009).

Incorporating social cognition measures into the standard neuropsychological toolbox may, therefore, afford highly relevant information for guiding daily functioning recommendations. Several examples of well-validated measures that neuropsychologists may consider include the NIMH MATRICS Social Cognition subdomain (Green et al., 2004), Implicit Association Test (Greenwald, McGhee, & Schwartz, 1998), Ekman and Friesen Battery of Emotion Processing (Ekman & Friesen, 1976), and Benton Facial Recognition Test (Benton, Hamsher, Varney, Spreen, 1983).

Everyday Functioning

A common complaint regarding neuropsychological evaluations is their apparent lack of relevance to the real-life problems that the patient may be experiencing. Expansion of the standardized neuropsychological toolset to include assessments of everyday abilities that are also sensitive to neurologic decline will enrich the ecological validity of evaluations. The Behavioral Assessment of the Dysexecutive Syndrome (BADS) was among the first batteries developed with this in mind (Wilson, Alderman, Burgess, Emslie, and Evans, 1996). For example, one of the BADS subtests, the Six Elements Test, is a widely used measure of cognitive multitasking that elicits planning and strategic thinking in a relatively unstructured context.

Other direct measures of daily living skills have grown in popularity in research settings, although their application in the clinic appears less common. A review by Moore, Palmer, Patterson, and Jeste (2007) identified 31 published performance-based measures of various functional skills, ranging from medication management and cooking to dressing and safety. We also encourage review of other comprehensive texts, such as self-report everyday measures as reviewed in Robyn Tate’s A compendium of Test, Scales, and Questionnaires (2010) and The Neuropsychology of Everyday Functioning edited by Marcotte and Grant (2009), which provides in-depth performance-based measurement approaches of daily functioning across diseases (e.g., driving simulators).

Additionally, given the ubiquity of the Internet, tools assessing the ability to navigate the World Wide Web are highly relevant, easily accessible, and gaining momentum. For example, Woods and colleagues (2016) recently examined the validity of the Simulated Market Task (S-MarT) and Web-based Evaluation of Banking Skills (WEBS) as examiner-controlled Web sites of common household (i.e., shopping and banking) and health-related (i.e., pharmacy refills and healthcare communications) abilities. These tasks differentiated between HIV + individuals with and without mild cognitive impairment independent of previous Internet experience, and were moderately related to performances on standard neuropsychological measures (Woods et al., 2016; Woods, et al., in press).

Although the reliability, internal consistency, and concurrent validity of these real-world measures are consistently high, surprisingly few studies have directly examined their predictive validities. Given that this is one of their primary goals, future work supporting the ability of objective functional measures to relate to real world status is needed, although admittedly difficult to achieve (e.g., operationalization of real world status is inherently complex). In an effort to not simply become another neuropsychological test, performance-based instrumental activities of daily living measures must also balance ecological validity (e.g., less structured environment for task completion) and ability to reliably administer the task. Empirical translations of neuropsychological assessments that more precisely predict and guide patient recommendations for the real world continue to be an area of ongoing growth for the field.

TRENDS AND FUTURE DIRECTIONS

Technological Advancements: Computerized Neuropsychological Assessment

As technology becomes ever more accessible, the availability of computer- and tablet-based neuropsychological assessments has rapidly risen over the past decade. Recent U.S. NIH assessment initiatives investing in the development of computerized tools (e.g., NIH EXAMINER, Kramer et al., 2014; NIH Toolbox, Gershon et al., 2013) mirror and further highlight this trend. Although we are only able to briefly touch upon computerized assessment techniques here, we refer readers to the 2012 American Academy of Clinical Neuropsychology and National Academy of Neuropsychology position paper discussing relevant issues ranging from ethics and privacy to psychometrics, device marketing, and automated reporting services (Bauer et al., 2012).

Of historical relevance, the MicroCog Assessment of Cognitive Functioning in 1993 represents one of the first computerized cognitive assessment batteries commercially available via Pearson (see review, Elwood, 2001). Relatedly, the Automated Neuropsychological Assessment Metrics^R (ANAM) is unique in that it was initially developed by the U.S. Department of Defense and became commercially available through Vista Sciences at the University of Oklahoma (Reeves, Winter, Bleiberg, & Kane, 2007). There are now decades of clinical and laboratory data generated on the ANAM^R, including >300 peer-reviewed articles, detailing its development and use (for comprehensive review, see the McCaffrey (2007) special issue of Archives of Clinical Neuropsychology).

While several other relatively older computerized batteries (e.g., Cambridge Neuropsychological Test Automated Battery, Sahakian & Owen, 1992; Cutler et al., 1993; Lenehan, Summers, Saunders, Summers, & Vickers, 2016; and Cog-State, Fratti, Bowden, & Cook, 2016) have developed a large user-base and substantial validation data, novel assessment platforms continue to be developed. Online, tablet, and smart phone-based test batteries, wearable devices, as well as virtual reality-based cognitive assessment represent only some of the variety of emerging modalities (Brouliette et al., 2013; Parsey & Schmitter-Edgecombe, 2013; Parsons, 2015). Using these platforms, one innovative application is frequent, at-home serial cognitive assessments over multiple days or weeks. Such sequential ambulatory testing approaches have demonstrated strong reliability, construct validity with gold-standard measures, and enhanced ecologic validity (Sliwinski et al., 2016). Combining both traditional static, comprehensive testing with ambulatory, brief tests may represent a complementary brain measurement approach that is more adept at detecting clinical changes at earlier stages of disease or injury, akin to other medical fields (e.g., cardiology onetime stress test vs. real-time Holter monitor).

Computers can do many things well and automatically, supporting their integration into neuropsychological practices (e.g., follow standardized procedures, time responses, and quickly and accurately score results). These assessment techniques have the potential to provide highly standardized and ecologically valid windows into real-time cognitive and behavioral changes, and could greatly improve accessibility (and costs) for underserved people who may have difficulty traveling to laboratory or healthcare settings. Yet, integration of such technologies is complex, not simply plug-and-play, and there are still major hurdles that must be addressed to ensure competent and valid application of online assessment (e.g., Internet connectivity, or examinee difficul-ties with comprehension of instructions, inconsistent effort, or distractions). A major consideration is that technology-based assessment tools inherently rely on continually changing technologies; while integration of the latest technology is appealing, any alterations to a validated measure will change its psychometric properties potentially rendering the new version as invalid or at the very least, highly difficult to interpret without a costly process of re-norming. If technology changes are unavoidable after test norming, careful piloting and documentation of any effects on test results are highly recommended and should occur before adopting or deploying the new technology. Technology will undoubtedly play a major role in the future of neuropsychological assessment by enhancing outreach, and potentially standardization and facilitation of diagnosis and early treatment. The development and implementation of such initiatives necessitates involvement of the neuropsychologist with our highly trained psychometric skillsets at the forefront.

Integration into Primary Care and Brain Health Assessment Initiatives

Although our understanding of the (oftentimes modifiable) factors associated with brain health has grown immensely in the past two decades, the systematic dissemination of this information to the public (including other medical fields) has lagged significantly behind. Emerging initiatives are underway to implement screening tools and visualize neuro-behavioral data in real-world practices outside of the neuropsychologist’s office. For example, University of California, San Francisco and Quest Diagnostics™ have partnered to develop the Dementia Care Pathway, a suite of technology-based tools to support navigation, brain health assessment (tablet-based cognitive screener, TabCAT; Possin et al., manuscript submitted for publication), automated analysis, and multimedia educational materials to nonspecialist providers (Rankin et al., manuscript submitted for publication).

Primary care is at the crux of health prevention for other body systems and is ideally positioned for inclusion of the brain. With the development of sensitive, standardized, easy-to-administer and portable screening devices, objective monitoring of brain status will be possible. Continued efforts to translate the important advances our field into real-world diagnostics and treatments are major areas for anticipated growth.

Neuropsychological Assessment Infrastructure

Another significant, under-addressed need in neuropsychology is normative infrastructure. Despite our progress, too often, we do not know how unusual or abnormal a test score or pattern of scores is for a specific group or individual. Of course, all tests must be interpreted in relation to “normal expectations” for the examinee, yet many tests do not have available norms that are appropriate for many types of examinees. Large-scale normative studies would of course help, but these are expensive and funding agencies typically are not supportive of grant funding for such work.

When a governmental funding agency (e.g., U.S. NIH) does fund norming of a nonproprietary set of cognitive tests (e.g., NIH Toolbox), this typically is a one-time effort and the norms may become out of date due to changes in a national census composition. By contrast, proprietary tests may get revised fairly often due to financial motives, and whatever may have been learned about associations between those tests and others (with their different norms) may be rendered tenuous or invalid. This issue is particularly evident in technology-based assessment development, as noted earlier. This of course is a moving target, but our field and its journals should prioritize work in this area, even if it does not have the cachet of a new, experimental approach to assessing a particular disease or brain system. At the very least, we recommend that new studies that use healthy control groups consider and report whether the groups’ results are in the expected range on available norms.

CONCLUSIONS

Neuropsychology has deservedly prided itself as being more empirically based than other domains of professional psychology. But there is still much to learn. Many types of neuropsychological test interpretations are still performed by clinical judgment unaided by empirical guidelines, and many of the questions without empirical guidelines are actually empirical questions that have just not been asked and answered, to date. The evolution from patient observation to highly standardized measures largely relying on only a pencil and paper to obtain a reliable and objective window into brain functioning is resourceful and remarkable, particularly given the relative nascence of our field. As neuropsychology continues to grow, assessment advancements will center on increased ability to detect brain changes at the earliest possible points, likely in conjunction with multimodal markers of neurological function (e.g., neuroimaging and biofluid markers), and dissemination of our understanding of brain–behavior relationships to promote public brain health more broadly.