Teleneuropsychology for vascular cognitive impairment: Which tools do we have?

Highlights

• •Teleneuropsychology guidelines recommend reproducing a standard in-person assessment.
• •Consistency of cognitive tests adaptations to videoconference needs to be elucidated.
• •Cognitive tests recommended in VCI have partial suitability for teleneuropsychology.
• •Global efficiency, verbal memory and language are assessable by videoconference.
• •Remote evaluation of attention and executive functions is a major limitation in VCI.

Abstract

The halt of clinical activities imposed during the COVID-19 pandemic forced clinicians to find alternative strategies to provide continuity of care and services, and led to a renewed interest in use of teleneuropsychology (TNP) to remotely assess patients. Recent TNP guidelines recommend maximizing the reproduction of standard in-person assessment, particularly through videoconferences. However, consistency of the adaptations of usual cognitive tests to videoconference needs further elucidation.

This review aims at critical reviewing which cognitive tests could be recommended for a remote evaluation of patients with vascular cognitive impairment (VCI) among those widely recognized as reference standards.

Current evidence supports the use of global cognitive efficiency (MMSE and MoCA), verbal memory (Revised Hopkins Verbal Learning Test), and language tests (phonemic and semantic verbal fluencies, Boston Naming Test), while there is a lack of strong validity support for measures of visuospatial functions (Rey-Osterreith Complex Figure), and executive functioning and processing speed (Trail making Test, and Digit symbol or Symbol digit tests). This represents a major limitation in the evaluation of VCI because its cognitive profile in often characterized by attention and executive deficits.

At present, a videoconference TNP visit appears useful for a brief evaluation of global cognitive efficiency, and to ‘triage’ patients towards a second level in person evaluation. In future, hybrid models of TNP based on data collected across multiple modalities, incorporating both adaptation of usual cognitive tools and new computerized tools in the supervised videoconference setting, are likely to become the best option for a comprehensive remote cognitive assessment.

1. Introduction

Teleneuropsychology (TNP) is part of the telehealth services, and is based on the application of audiovisual technologies to support remote meetings with patients to deliver neuropsychological assessments and interventions [1], [2], [3].

During the healthcare systems’ crisis caused by the coronavirus disease 2019 (COVID-19) pandemic, preventive measures, such as physical distancing constraints, caused an unprecedented halt of several clinical activities based on usual face-to-face interpersonal contact. Thus, remote assessment of cognition rapidly became an alternative strategy to provide continuity of care and services. TNP includes both supervised and unsupervised approaches: telephone or videoconferencing administration's conditions are based on examiner's interaction and supervision in real time, while automated computer testing does not require supervision, and can be administered either real time or not (Fig. 1). Recent TNP guidelines that have been proposed in response to the COVID-19 crisis recommend using telehealth methods aimed at maximizing the simulation of standard in-person practice; in this sense, remote cognitive evaluation by means of videoconferencing is the approach that better fits with these needs [1], [2], [3], [4]. TNP services include two main delivery modalities: in-clinic and in-home [5]. In the first scenario the examiner and the patient are in the same or different clinic but in separate rooms, and usually an assistant is on site to support the patient (assisted-TNP). In the second scenario, the patient is at home and the examiner can be either in the clinic or at home (direct-to-home-TNP).

Fig. 1.

Characteristics of telephone, videoconference, or computerized administration conditions in teleneuropsychology.

Although the undeniable exponential acceleration recently caused by the COVID-19 pandemic, the research interest in the development of methods to conduct remote cognitive assessments by means of videoconferencing had started before [6,7]. This area of research was solicited by some possible advantages of videoconference-based TNP, i.e., the potential to expand the range of neuropsychological facilities in underserviced areas, increase access to neuropsychology for those patients with mobility restrictions, and reduce the costs. From the economical point of view, cost-effectiveness of TNP is increasing over time together with the reduction of costs related to technological devices and infrastructures. In 2017, Brearly and colleagues conducted a systematic review and meta-analysis to assess the consistency of cognitive tests scores acquired during a videoconference administration [6]. Marra and colleagues published a critical update based on the methodological framework offered by Brearly and colleagues [8]. Recently, other reviews have been published on the reliability and diagnostic accuracy of videoconference TNP [9,10]. Overall, findings from these reviews provide encouraging evidence in favor of adequate psychometric and clinimetric properties of videoconference cognitive assessments for older adults. However, there is some variability in reliability and validity of videoconference-mediated neuropsychological testing. Moreover, specific test characteristics, e.g., response modalities and timing dependency, contribute to the heterogeneity in previous findings. Specifically, cognitive tests that rely on verbal responses from participants were found consistent for the videoconference administration across studies, while few and contrasting data are available for tests requiring interaction with physical objects [6], [7], [8], [9]. Of note, most of the research was conducted by means of assisted-TNP where appropriate hardware and technician support was available. The direct-to-home-TNP setting, where the patient is located at home, is a different scenario which has considerably less control over extraneous variables in both environment and technology. The direct-to-home-TNP approach has received increasing interest only during the last few years, as a direct consequence of the COVID-19 pandemic crisis.

Some of the issues related to TNP may be disease-specific. In 2006, the National Institute for Neurological Disorders and Stroke and the Canadian Stroke Network (NINDS-CSN) convened a consensus conference on the harmonization standards for vascular cognitive impairment (VCI) and proposed three neuropsychological assessment protocols (60-min, 30-min and 5-min) for early identification and diagnosis of VCI (Table 1) [11]. The cerebrovascular diseases’ research community welcomed these protocols and several efforts have been done to adapt and validate them for clinical use in different languages and cultures. The proposed cognitive tools nowadays represent a widely accepted reference standard in the field of VCI [[12], [13], [14], [15], [16], [17], [18], [19]]. The aim of this narrative critical review is to identify which cognitive tests could be used for a remote evaluation of VCI patients among those included in the NINDS-CSN neuropsychological protocols. Literature review will be primarily focused on studies on VCI patients. The present review will also highlight some criticisms in progressing from the assisted-TNP setting to the in-home TNP one, that has the potential to maximize the advantages of a remote cognitive assessment.

Table 1.

Cognitive tests included in the NINDS-CSN protocols for vascular cognitive impairment and suitability for videoconference administration.

		NINDS-CSN VCI protocols
Cognitive domain	Test	60-min.	30-min.	5-min.	Suitability for videoconference administration
Global cognitive efficiency	MMSE	√	√		***
	MoCA			√	***
Memory	Revised Hopkins Verbal Learning Test	√	√		***
	California Verbal Learning Test (2nd ed)	√	√		*
Attention and executive functions	Digit Symbol-Coding (WAIS)	√	√		**
	Trail making Test	√			*
Language	Semantic fluency (animal naming)	√	√		**
	Phonemic fluency (COWAT)	√	√		***
	Boston Naming Test (15 item)	√			**
Visuospatial functions	Rey-Osterreith Complex Figure copy	√			*

NINDS-CSN: Neurological Disorders and Stroke and the Canadian Stroke Network; COWAT: Controlled Oral Word Association Test, WAIS: Wechsler Adult Intelligence Scale.

Level of evidence for suitability in videoconference administration: * Little evidence; ** Moderate evidence; *** Good evidence.

1.1. Global cognitive efficiency tests

Within the NINDS-CSN VCI neuropsychological batteries, MMSE was considered a supplemental measure in both the 60-min and 30-min protocols, while the 5-min protocol consists of a selection of subtests from the MoCA: 5-word immediate and delayed memory, 1-letter phonemic fluency, 6-item orientation (Table 1) [11].

Studies on MMSE showed that it can be considered a promising TNP measure for screening cognitive status across different clinical populations, and that it also has good reliability and utility as a tool for longitudinal assessment [[6], [7], [8], [9], [10],20]. However, the performance at the videoconference version of the MMSE is influenced by disease severity, and it may become insufficiently valid for patients with severe cognitive impairment, e.g., a MMSE score<17 [8,16].

Despite less frequently studied than the MMSE, videoconference administration of the MoCA showed good validity compared to its face-to-face administration in different populations [8], [9], [10]. Only one study compared face-to-face MoCA assessment to videoconference administration in community-based survivors of stroke [21]. The findings showed no significant difference in total MoCA scores across administration's conditions, and, thus, no systematic bias influencing the performance in a particular setting, and a substantial equality in terms of their perceived difficulty.

Adaptations for the videoconference administration differed between MMSE and MoCA. MMSE adaptations were heterogeneous, and included the full standard version, the telephone version, and variations that excluded the written items (i.e., interlocking pentagons and writing task).

MoCA adaptations for videoconference were minimal across studies, and an official audiovisual full version of the test (standard versions 8.1, 8.2 and 8.3) to be administered via videoconference has been released, and it is available through the website (www.mocatest.org) in the following languages: English, French, Italian, Spanish, and Portuguese.

Among the adaptations for the videoconference administration of global cognitive efficiency tests, the choice of the use of visual and/or motor items and the variations of the orientation questions are probably the most relevant. Visual items that need verbal responses (e.g., naming tasks based on visual stimuli) can be administered via screen sharing. Motor items poses the problem of how to record the responses. In case of simple drawings copy or writing tasks, the more common strategy is to ask participants to use a paper and pencil response form, and then to show their drawings/texts to the camera for scoring purposes. In case of more complex items, such as the short trail-making test version included in the MoCA, it is preferable to adapt the procedure and to require an oral answer. Orientation questions pose two main problems: 1) to impede the use of orientation cues (e.g., calendars, notes, pc screen) available in the environment, 2) to evaluate the spatial orientation according to patients’ location. The first problem can be overcome by asking patients to close their eyes before administering time orientation questions. The second issue varies as a function of the setting: when the patient is in the clinic (assisted-TNP), spatial orientation items can be administered in their standard version, but when the patients is at home (direct-to-home-TNP) some adjustment is required. A frequent solution is to ask patients their address information, but this may represent a facilitation as the home setting is usually very familiar. The authors of the MoCA decided for a different approach: before beginning the assessment, the examiner tells which structure or institution and which city he/she is in, and this information is asked back to the patient as spatial orientation questions at the end of the test. This strategy allows the administration of the same items independently from the in-clinic or in-home setting, but could introduce some memory load that could both increase difficulty and reduce validity.

1.2. Second level cognitive tests

To the best of our knowledge, only one assisted-TNP study by Chapman and colleagues compared the in-person and videoconference-based administration of an adaptation of the NINDS-CSN VCI protocols to community-based survivors of stroke [22]. Stroke patients did not perform systematically better/worse in the videoconference administration condition compared to the standard one, and they broadly accepted a videoconference-based TNP assessment.

Overall, evidence on the validity of the videoconference administration of second level cognitive tests is strictly dependent on input (stimuli presentation) and output (patients’ response) modalities and timing constraints [6], [7], [8], [9]. Presentation and response modalities may be verbal, visual, or physical. Timing constraints pertain to synchronicity, i.e., the extent to which administration might be negatively affected by disruptions in information transmission. Administration of tests that permit instructions’ repetitions, need errors’ correction in real time, and/or are based on execution time could be hampered by interferences and latencies due to technological problems, mainly related to internet connection stability.

Verbally mediated tasks are widely recognized as those less affected by videoconference administration [6], [7], [8], [9], [10]. Looking at the tests in the NINDS-CSN VCI protocols, both the Revised Hopkins Verbal Learning Test (HVLT-R) and verbal fluencies had strong support for validity in TNP assessments. Unfortunately, the Revised Hopkins Verbal Learning Test was one of the two measures that was associated with a worse performance in the videoconference condition in the study by Chapman and colleagues on stroke patients, and the authors cautioned against its use for a remote assessment [22]. Taking into account other memory list learning tests, some encouraging preliminary evidence come from the few studies that evaluated the California Verbal Learning Test and the Rey Auditory Verbal Learning Test, but further efforts are needed in this direction, particularly on cerebrovascular populations.

Letter fluency received higher support for TNP validity than category fluency, and Marra and colleagues argued that this might depend on the single-trial nature of the most used version of the latter test (e.g., animals) which may be more susceptible to performance variability in comparison to the three-trial nature of letter fluency [8]. Remaining within the language domain, despite some conflicting results that may be due to inconsistencies across the alternate forms of the test, there appears to be good support also for the validity of the Boston Naming Test (BNT) [6], [7], [8], [9], [10]. Overall, picture naming tests require minimal adaptations for the videoconference administration, as visual stimuli can be easily presented by means of the screen sharing function.

Among available TNP studies, the most traditional measures of executive functioning (e.g., set-shifting and mental flexibility, verbal inhibitory control, abstract problem-solving) were not well-represented, and therefore evidence of validity of these tools for videoconference assessments is very limited [6], [7], [8], [9]. Synchronicity is one on the main challenges in adapting standard attention and executive function tests to the videoconference-based administration. The Trail Making Test and the Digit Symbol-Coding subtest of the WAIS are included in the NINDS-CSN VCI protocols, as measures of attention, executive function, and processing speed [11]. Most adaptations of the NINDS-CSN VCI protocols, including the assisted-TNP study by Chapman and colleagues, substituted the WAIS-III (Wechsler Adult Intelligence Scale – Third Edition) Digit Symbol Coding subtest with the oral Symbol Digit Modalities Test (SDMT) [23,24]. Both TMT and SDMT are administered under timed conditions and successful performance depends on rapid and efficient processing. The motor component is thus one of the determinants of the performance on these tests, and oral versions of the TMT and SDMT have been developed to accurately assess patients with physical impairments [24], [25], [26], [27]. The oral SDMT uses the same test paradigm and form, but requires the examinee to provide verbal responses within a 90 s time limit. The instructions for administration of the oral TMT are similar to the written version, but error correction needs some refinement. According to the correction instructions from Mrazik and colleagues, in case of an error on Part A, the examiner reorients the patient to the last correct number by saying: ‘You last said [specific number], please continue from there’, while in case of an error on Part B, the examiner refers to the last correct pair by saying: ‘You said [specific number] [specific letter], continue from there’ [28]. Taking into account studies on standard in-person cognitive assessment, evidences on the psychometric properties and clinical utility were excellent for the oral SDMT, adequate for the oral TMT-B, but limited for the oral TMT-A [26,27].

Very few studies applied the oral versions of TMT and SDMT to the videoconference administration. Encouraging evidence is coming from studies on patients with multiple sclerosis: the videoconference assessment by means of the oral SDMT seems to provide valid and reliable measures of cognitive function [29], [30], [31]. Only one validity study used the oral TMT in a mixed sample that included Alzheimer's disease patients, mild cognitive impairment patients, and healthy controls [32]. Despite a significant difference in completion time between conditions (with a better performance in person), the authors sustained that this small difference was not clinically meaningful, and concluded that there is some support for the validity of oral TMT in TNP assessments.

The study by Chapman and colleagues on stroke patients included the standard version of the TMT, the oral version of the SDMT, and the Stroop Test as additional measure [22]. Results showed comparable performances between the in-person and videoconference-based conditions for the TMT and SDMT, while the Stroop Interference scores were superior (worse performance) in the videoconference condition. However, the study design was based on assisted-TNP: a research assistant was present to physically manage test stimuli and modifications to standardized procedures were limited to the use of envelopes containing stimuli or paper-and-pencil standard response forms.

Finally, for the evaluation of visuo-spatial functions, the NINDS-CSN VCI consensus implements the Rey-Osterreith Complex Figure (ROCF) copy. Some preliminary encouraging evidence on the validity of a videoconference administration of the ROCF came from a couple of assisted-TNP studies, including the study by Chapman and colleagues [22,33]. Overall, reliability of the visuospatial/constructional tasks in videoconference was fair across studies, but these tests were scarcely adopted. In terms of feasibility, the adaptations of those tests for the remote administration could be minimal: screen sharing allows presentation of visual stimuli, and patient's written responses can be held up to the camera and documented via screen capture.

2. Discussion

At present, a videoconference TNP evaluation based on the cognitive tests included in the NINDS-CSN protocols would be only partially feasible and reliable. The main limitation pertains the possibility to examine the entire cognitive profile, including all core domains. Current evidence could support the use of global cognitive efficiency, memory, and language tests, while there is a lack of strong validity support for measures of visuospatial functions, executive functioning and processing speed. The latter is a main problem in VCI, as its cognitive profile is often predominantly characterized by executive deficits, such as slowed information processing, impairments in the ability to shift or select across tasks, and deficits in the ability to hold and manipulate information. Moreover, the authors of the NINDS-CSN VCI neuropsychological protocols clearly stated that timed executive function tests may be especially sensitive in VCI patients, but these tests are the most problematic in videoconference due to administration/response modifications and technical problems related to synchronicity [11]. From the technological point of view, the administration of timed tests requires a higher quality in terms of speed and stability of the internet connection and audio-video resources. Uncontrolled latency effects negatively impact execution time and real time error corrections, thus reducing reliability and validity.

In remote cognitive evaluation, the need to adapt administration modalities is the main issue in terms of reliability and validity. Furthermore, specific normative data are required to interpret tests’ results in this condition, but few efforts have been made in this direction [34], [35], [36]. Current recommendations are to rely on normative data obtained using the standard assessments for those cognitive tests proven to be consistent across conditions, but caution is needed in this regard. In these cases, the report should clearly state that administration was nonstandard, and that this condition is likely to impact on reliability of the clinical judgment and decision, mainly when patients score close to cutoffs, as in mild cognitive impairment.

Beyond administration and evaluation modalities adaptation, further critical issues related to remote cognitive assessment include: technology, setting and environment, patients’ characteristics, and ethical and normative issues (Fig. 2). An extensive discussion of TNP challenges is beyond the focus of this review, but some issues are particularly relevant moving from assisted-TNP to in-home TNP assessments. The barriers to adopting and implementing in-home TNP are frequently technology-specific: older adults may lack easy access to technological devices or high-speed internet connections (availability), as well as familiarity with digital device (digital illiteracy). In terms of feasibility, functionality of the device, audio and video quality, and internet connection speed and stability are of upmost importance, mainly for timed synchronous tests. In terms of familiarity, it is not clear whether and how digital illiteracy may influence cognitive performance during a videoconference TNP evaluation, particularly in older adults. Considering the need of further studies aimed at the establishment of specific normative data, a measure of computer proficiency could be tested as a potential influencing factor, and then used to calculate ad hoc corrections norms if relevant.

Fig. 2.

Current critical issues related to remote cognitive assessment.

In in-home TNP, environmental control is very reduced: patients can be influenced by distractions, aids, and/or unrequired assistance from third party, and both the communication circle and the relational dimension (eye contact, behavioral interaction/observation) could be very problematic. Very few data are available on patients’ characteristics that allow or impede a TNP evaluation. The only available recommendation come from a Latin America Working Group for TNP in: 1) patients with visual or auditory deficits, acute confusional states, or severe communication difficulties should not be evaluated using TNP assessments (Class B Recommendation), 2) TNP assessments are not recommended for patients with severe dementia. (Class F Recommendation) [37]. Furthermore, across clinical populations, the nature and the degree of the cognitive symptoms (e.g., the presence of behavioral dysregulation) are likely to be additional barriers that need to be fully elucidated. Finally, ethical and normative problems include issues related to informed consent, digital security, and privacy. An appropriate informed consent detailing modifications of standard procedures and potential limitations to diagnostic conclusions and recommendations has to be obtained, at least verbally. To ensure protection to copyrights related to tests’ materials, current recommendation suggest to obtain documented agreement from the examinee that the session will not be recorded, reproduced or published, and that copies of the materials will not be made, and the same recommendation is released also for the examiner. According to our opinion, this last point could be debatable. The agreement to record the audio track can be requested to patients within the informed consent form, and it could represent an effective strategy to balance from one side the examiner's need to stay concentrated on the patients during the visit and manage environmental and/or technological difficulties, to the other side to ensure an adequate scoring, even using a verbatim approach.

2.1. Present and future of teleneuropsychology: towards hybrid models for clinical practice

Currently, the paucity of data for the in-home TNP and the several challenges that we are facing reduce our possibilities to conduct a second level cognitive assessment by means of videoconference. Looking both at present options and future potentialities, hybrid models of TNP incorporating both traditional and technology-based modalities are the most promising approaches, and should be implemented and studied (Fig. 3) [38,39].

Fig. 3.

Hybrid teleneuropsychology models: actual and future potentialities.

At present, a videoconference visit allows an evaluation of global cognitive efficiency (MMSE, MoCA) that could be useful to ‘triage’ patients and decide for the need of a second level cognitive evaluation to be conducted in person. This multi-level approach would enable timely delivery of screening assessments and more accurate triage decisions.

For the future, hybrid models of TNP based on data collected across multiple modalities are likely to be the best option to use TNP for a comprehensive second level cognitive assessment. In this direction, it will be necessary to work to extend the kit of cognitive tools available for videoconference administration. From one side, efforts could be made to adapt and validate standard cognitive tests to the videoconference modality, and to obtain specific normative data. From another side, despite recent TNP guidelines recommended duplicating the standard in-clinic assessment because of the need to promptly response to the COVID-19 crisis, telehealth should not be confined to this approach. TNP can and should give a boost to the development of new cognitive tests starting from paradigms of the standard in-person. In line with this, the development of new computerized cognitive tools is likely to be necessary for the evaluation of some cognitive abilities. Attention and executive functions tests that require motor input/output modalities are the best candidates. As an example, in 2006 the authors of the NINDS-CSN VCI protocols already recommended the implementation of Simple and Choice Reaction Time tasks for the evaluation of attention, and this paradigm is among the most suitable for the computerized adaptation and administration.

Furthermore, digital technology gives the opportunity of an in-depth analysis of patients’ performances on cognitive tests. Computerized testing could allow the identification of additional cognitive parameters whose role as potential early cognitive markers need to be further explored [40]. Time-based parameters, e.g. the latency necessary to generate responses, could be used to measure the construct of temporal organization that modulates executive control, and thus executive functions. Interesting evidence on how the construct of temporal organization regulates the dysexecutive behavior in patients with cognitive impairment came from studies based on the Kaplan's Process Approach. Lamar and colleagues found a striking negative slope of decline over time in performances both at the Boston Revision of the Wechsler Memory Scale Mental Control subtest and at the letter fluency test in patients with Vascular Dementia (VaD) compared to Alzheimer's Disease (AD) ones [41]. In a recent study, Emrani and colleagues administered an iPad-version of a 5-span Backward Digit Span Test to memory clinic patients, and found distinct patterns of responses’ latencies that differentiate MCI from non-MCI patients. The authors concluded that latencies obtained from tests assessing mental re-ordering may constitute useful and early neurocognitive biomarkers [42]. Finally, Davoudi and colleagues administered a digital Clock Drawing Test to patients with AD, VaD, and healthy elderly controls and found that the groups could be discriminated based on kinematic (i.e. measures of pen pressure) and time-based parameters [43]. The combination of digital technology with a qualitative approach to cognitive evaluation is a unique opportunity to identify and objectively measure new behavioral patterns [44]. Moreover, features related to temporal organization and strategies used to complete cognitive tests are strongly related to executive control, a domain well-known to be compromised in VCI.

The present narrative review has intrinsic methodological limitations that depend on the lack of a systematic review approach. During the last 3 years, the renewed interest in TNP in response to the COVID-19 crisis has led to the publication of systematic reviews on validity, reliability and diagnostic accuracy of videoconference TNP in the elderly. On the other hand, our literature review on videoconference TNP studies on VCI patients revealed a dramatic paucity of data in this population, and only one study by Chapman and colleagues compared the in-person and videoconference-based administration of the MoCA and the NINDS-CSN VCI protocols to stroke survivors. We thus decided to collect all available evidence taking primarily in consideration the neuropsychological protocols that the experts in the field of VCI consider their reference standard with the idea to give a practical summary that could encourage further research in the field of TNP in VCI.

As a consequence of the acceleration given by the COVID-19 pandemic to this research field, we expect that further evidence on consistency and clinical models of TNP will be available soon, and we wish for more specific data on the in-home setting. Despite all the above mentioned limitations on the use of videoconference TNP, this approach has important advantages that extend beyond the pandemic constraints and mitigation strategies. One of the main opportunities offered by TNP is the provision of neuropsychological services in rural settings. In rural communities, difficulties related to inadequate resources, distances and costs, and isolation are of upmost relevance and limited patients’ access to clinical services, resulting in under-diagnosis and suboptimal management of neurocognitive disorders, as well as the lack of chances to receive interventions [45,46]. TNP represents an opportunity for equity and to allow underrepresented population to access health care services, or even get involved in research.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.