Saving cognitive outcome data in Alzheimer's disease clinical trials during the COVID‐19 pandemic: Commentary on the virtual administration of the ADAS‐Cog

Abstract

Elderly participants in Alzheimer's disease (AD) clinical trials are at high risk of morbidity and mortality with interpersonal exposure to COVID‐19, a situation that is likely to continue for the foreseeable future. Yet, in‐person neuropsychological assessments remain the mainstay primary outcomes for clinical trials seeking prevention and cure for AD. The Alzheimer's Disease Assessment Scale‐Cognitive (ADAS‐Cog) is among the most commonly used cognitive assessment in AD clinical trials, and though currently lacking specific guidelines for virtual administrations, it can be used remotely with appropriate modifications and considerations. Here we propose a novel method of virtual administration of the ADAS‐Cog, which considers workarounds for technological and human limitations imposed when the participant is no longer sitting across from the test administrator.

1. INTRODUCTION

The Alzheimer's Disease Assessment Scale‐Cognitive (ADAS‐Cog) ¹ , ² is the most commonly administered cognitive assessment in clinical trials of Alzheimer's disease (AD) and is frequently used as a primary outcome measure of treatment effectiveness in drug trials. ³ It consists of 11 sections and tests broad domains of cognitive functioning. The test was developed for in‐person administration in a standardized office setting. However, the COVID‐19 pandemic has necessitated a shift toward non‐standardized methods of administration, as the interpersonal exposures of travel and clinical research settings may present significant risk.

There are currently no standardized guidelines for approaching a virtual administration of the ADAS‐Cog. One systematic review of virtual administration of neuropsychological assessments supported the feasibility and utility of remote administration, while highlighting increased variability in the oldest adult populations (over age 75). ⁴ Other research further supports virtual testing in both cognitively healthy and impaired populations. ⁵ , ⁶ Specifically regarding ADAS‐Cog, two groups have validated the use of international versions of the test in a virtual setting in older adults with and without cognitive impairment. ⁷ , ⁸ These findings are encouraging; however, in practice, we find numerous barriers to implementing the traditional ADAS‐Cog, warranting protocol modification. Here we discuss adjustments and solutions set forth by our research group toward generating a standardized virtual ADAS‐Cog for use in ongoing AD clinical trials, with the goal of promoting continuity in AD research in the current era of improved digital technology and public health crisis.

2. PRE‐VISIT PLANNING

Prior to the modification of any clinical trial's study protocol, investigators must ensure that they are familiar with the guidance on conduct of clinical trials during COVID‐19 set forth by the U.S. Food and Drug Administration. ⁹

A successful encounter starts with enlistment and preparation of a participant's study partner (SP). We schedule a call with participants’ SP ahead of the remote interaction to prepare expectations and enlist assistance for the pending cognitive testing. We reinforce the need for testing integrity and instruct SPs to avoid aiding the participant during any portion of the examination. All appropriate testing materials are mailed to SPs with a reminder of the proprietary nature of the materials. SPs are also asked to help prepare the testing environment by ensuring the home exam room is private, well lit, quiet, and provides a sitting area with a clear writing surface. We also ask SPs to remove items that may be used to help answer questions related to orientation (i.e. hide clocks and calendars on walls or computer screens).

A "technology check" is also performed to ensure an adequate interface for assessment. Participants need a desktop or laptop computer with video, audio, and microphone capabilities, and are instructed on how to operate the video conference platform used during the virtual visit. We prefer computers over tablets or smartphones, to eliminate additional barriers to test administration such as screen size and movement. Device volume should be raised to an appropriate level and glasses and/or hearing aids should be available during the visit.

It is important to note that access to adequate technology to enable a virtual study visit may not be possible for some research participants, such as those in remote areas with limited internet access or those with lower socioeconomic status. While every effort should be made to include every individual enrolled in the trial, the issue of lack of access to technology is, for some, an unfortunate and unavoidable obstacle common to any virtual approach.

SPs are provided with a copy of necessary testing stimuli (see Table 1) via mail ahead of the scheduled visit and are instructed not to reveal materials to the participant until the appropriate time. SPs are also asked to gather specific materials from their own items to assist with the Commands and Ideational Praxis subsections (see Table 1).

TABLE 1.

Virtual administration guide by subsection

Subsection	Adaptability to virtual administration	Considerations for administration
Word Recall	Yes, without significant deviation	Word cards should be in full view of the camera.
Commands	Yes, without significant deviation	SP should gather subtest materials (pencil, blank index card or small sheet of paper, wrist watch) ahead of time and present/arrange the materials at the appropriate time; participant's upper body should be in full view of the camera to ensure appropriate scoring.
Constructional Praxis	Yes, with deviation in administration	Participant's camera should be pointed toward their writing surface/hands; SP should assist with presenting individual figures at the appropriate time and hold them up in view of the camera for scoring, if necessary.
Delayed Word recall	Yes, without significant deviation	Unaltered.
Naming 1. Objects 2. Fingers	1. Yes, without significant deviation 2. Yes, with deviation in administration AND test instructions	1. Stimuli should be presented in the center of the camera's field of view. 2. Instructions should be altered (see supporting information for script); test administrator will point to their own fingers while demonstrating the rostral side of their hand in full view of the camera.
Ideational Praxis	Yes, with deviation in administration	SP should gather subtest materials (blank envelope, blank sheet of paper) ahead of time and present the materials when propted by the test administrator; participant's camera should be pointed toward their writing surface/hands.
Orientation	Yes, with deviation in administration AND scoring	Orientation to place/location should be altered so the participant is asked to identify their location more specifically than simply “home”; they may respond with the room of the house in which the testing is taking place OR their specific home address OR the type of home (ie, single‐family ranch style, specific name of condominium); see supporting information for details.
Word Recognition	Yes, without significant deviation	Word cards should be in full view of the camera.
Remembering Word Recognition Instructions	Yes, without significant deviation	Unaltered.
Comprehension of Spoken Language	Yes, without significant deviation	Unaltered; instances in which comprehension was disrupted due to audio latencies should not be considered.
Word‐Finding Difficulty	Yes, without significant deviation	Unaltered.
Spoken Language Ability	Yes, without significant deviation	Unaltered. Paucity of language should be considered in the context of the participant's comfort level with the virtual environment. Paraphasias and other nuanced language impairments may be missed due to audio disruption. SPs may be helpful in clarifying whether language abilities exhibited during the visit are typical of the participant's everyday speech.
Concentration/Distractibility	Yes, without significant deviation	Consider the testing environment; some level of distractibility should be considered normal in situations in which disruptions in the environment are uncontrollable (i.e., traffic noise, neighborhood dogs barking).
Maze	No	Virtual administration is not possible without major deviations in subtest instructions, subtest practice, and administration.
Number Cancellation	No	Virtual administration is not possible without major deviations in subtest instructions, subtest practice, and administration.

Critically, the visit's procedures should be well planned and rehearsed by the test administration staff to ensure seamless flow of testing during the virtual visit. The test administrator should be provided the full ADAS‐Cog testing kit and asked to prepare Object Naming stimuli in order of their presentation before each administration. A supervising neuropsychologist should also prepare a virtual administration script for administrators to follow during the session. See supporting information for a Manual of Procedures that includes such a script and other important elements of virtual ADAS‐Cog administration.

3. ADMINISTRATION

3.1. Considerations

Although virtual administration of ADAS‐Cog is feasible, the testing session is likely to take more time and may be more taxing on the participant, SP, and test administrator. It may be more difficult to establish rapport, or redirect a distracted participant over a remote device, especially if a participant is more impaired or less comfortable with technology. As is the case in any non‐standard setting, there may be distractions due to uncontrollable environmental interruptions, such as telephones, children, and pets.

Conversations and comprehension of test instructions may be made more difficult by audio and/or video disturbances. Due to restrictions with how many times a set of instructions can be repeated on certain subtests, a test administrator must decide whether a request to repeat instructions is due to technological glitches or lack of comprehension. SPs may be asked to clarify whether a technology disturbance may have occurred.

If a participant does not comprehend test instructions, or if a participant struggles to complete a task, there is a risk that SPs may unintentionally interfere by providing aid; SPs may repeat words or instructions or accidentally redirect a participant's attention (e.g., pointing). The SP's presence itself may also influence participant performance. Participants may feel self‐conscious of declining cognitive abilities and perform differently than they would in a private testing environment, or they may be more tempted to give up rather than risk providing an incorrect response or demonstrate forgetfulness.

Another important consideration of the virtual environment is the camera's restricted field of view. When a participant is writing or following instructions for certain subtests, the camera must be angled toward the table so the administrator can observe performance. However, by removing a participant's face from the field of view, test administrators are deprived of valuable behavioral information such as facial expressions indicating confusion or inattention. Should the necessity of virtual neuropsychological assessment continue in the future, this limitation could be attenuated with multiple or expanded view cameras allowing simultaneous participant and tabletop views; however, such would require additional technologies be available or provided to participants.

3.2. Subtest administration

The majority of the ADAS‐Cog can be completed with only minor deviations from normal procedures and small technological adjustments. One major deviation, however, was our decision to omit Mazes and Number Cancellation from virtual administration because they require the presence of a testing administrator to appropriately explain instructions, and to provide both feedback and error correction during practice and completion of the subtests.

Another significant alteration was made in the procedure for the Finger Naming subtest to adjust for the fact that the testing administrator would not be able to point to the participant's fingers. Instead, administrators hold up their own hand in view of the camera and ask participants to name the finger he or she is pointing to (see Table 1).

4. SCORING AND INTERPRETATION

For subtests requiring verbal responses, scoring procedures remain unchanged; scoring is more challenging for subtests requiring a participant to produce written responses. For tasks with a drawing component, once a participant completes their response, the administrator must ask that the response sheet be held in view of the camera so the item can be scored; a screen capture may be helpful for later review and adjudication. After the visit, SPs should mail back the participant's response pages to ensure appropriate possession of original source materials and to double‐check scoring.

Due to myriad opportunities for interruptions and other confounding factors interfering with the testing session, it is important that the administrator keep detailed notes about the testing process, commenting on the participant's ability to cope with virtual testing and note any other factors that may have impacted performance.

Score interpretation is another topic for consideration. Scores that are poorer than anticipated, especially compared to previous in‐person testing, may be due to factors unrelated to disease progression, such as difficulties with virtual testing or disruptions in daily routine. For example, orientation to time and date may be more difficult for participants who have been quarantined for months, with little day‐to‐day variation and fewer scheduled activities marking time. Conversely, scoring orientation to place will likely be inflated due to the overlearned familiarity of the unchanging home environment compared to the novel space of an in‐office testing environment.

While we await a rigorous assessment of validity and reliability of virtual administration of ADAS‐Cog, a growing body of evidence suggests that such neuropsychological testing completed in person and in a virtual setting are statistically comparable. ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ¹⁰ Nevertheless, it would be prudent for investigators to take advantage of this opportunity and consider using any cognitive outcome data collected during virtual study visits in future studies to formally assess the validity and reliability of such data in a virtual platform. Until there is validation, however, researchers must approach longitudinal analyses with caution when comparing time points with differing administration procedures.

5. CONCLUSIONS

Global circumstances demand new means of assessments in clinical trials, balancing safety with valid continuity of research. By switching from a highly controlled face‐to‐face assessment in AD to virtual assessment in remote settings, the field confronts uncertainties of major outcomes based on missing or noisy data. On the other hand, we may find that home assessment with steadily improving technologies will improve research participation, decrease the burden (or risk) of travel, and broaden access of older adults to the research with which we hope to cure AD. Here, we present a straightforward means of assessing cognition using a slightly modified ADAS‐Cog, the current gold standard for cognitive assessment in AD clinical trials. We see this as both a functional alternative to the traditional approach, as well as a starting point for continued discussion and improvement by the broader neuropsychological and AD clinical trials community.

CONFLICTS OF INTEREST

The authors have no conflicts of interest to disclose.

Supporting information

Supporting Information

Schwab NA, DesRuisseaux LA, Weinberg MS, Arnold SE. Saving cognitive outcome data in Alzheimer's disease clinical trials during the COVID‐19 pandemic: commentary on the virtual administration of the ADAS‐Cog. Alzheimer's Dement. 2020;6:e12081 10.1002/trc2.12081