Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Dec 27.
Published in final edited form as: Clin Neuropsychol. 2020 Jul 16;34(7-8):1314–1334. doi: 10.1080/13854046.2020.1767214

InterOrganizational Practice Committee Recommendations/Guidance for Teleneuropsychology (TeleNP) in Response to the COVID-19 Pandemic

Robert M Bilder 1, Karen S Postal 2,*, Mark Barisa 3, Darrin M Aase 4, C Munro Cullum 5, Stephen R Gillaspy 6, Lana Harder 7, Geoffrey Kanter 8, Margaret Lanca 9, David M Lechuga 10, Jennifer M Morgan 11, Randi Most 12, Antonio E Puente 13, Christine M Salinas 14, Jonathan Woodhouse 15
PMCID: PMC7767580  NIHMSID: NIHMS1654610  PMID: 32673163

Abstract

Objective:

The Inter Organizational Practice Committee (IOPC) convened a workgroup to provide rapid guidance about teleneuropsychology (TeleNP) in response to the COVID-19 pandemic.

Method:

A collaborative panel of experts from major professional organizations developed provisional guidance for neuropsychological practice during the pandemic. The stakeholders included the American Academy of Clinical Neuropsychology/American Board of Clinical Neuropsychology, the National Academy of Neuropsychology, Division 40 of the American Psychological Association, the American Board of Professional Neuropsychology, and the American Psychological Association Services, Inc. (APAS). The group reviewed literature, collated federal, regional and state regulations and information from insurers, and surveyed practitioners to identify best practices.

Results:

Literature indicates that TeleNP may offer reliable and valid assessments, but clinicians need to consider limitations, develop new informed consent procedures, report modifications of standard procedures, and state limitations to diagnostic conclusions and recommendations. Specific limitations affect TeleNP assessments of older adults, younger children, individuals with limited access to technology, and individuals with other individual, cultural, and/or linguistic differences. TeleNP may be contraindicated or infeasible given specific patient characteristics, circumstances, and referral questions. Considerations for billing TeleNP services are offered with reservations that clinicians must verify procedures independently. Guidance about technical issues and “tips” for TeleNP procedures are provided.

Conclusion:

This document provides provisional guidance with links to resources and established guidelines for telepsychology. Specific recommendations extend these practices to TeleNP. These recommendations may be revised as circumstances evolve, with updates posted continuously at IOPC.online.

Goals of the TeleNP Recommendations and Guidance

An advocacy team was established by the Inter Organizational Practice Committee (IOPC)1 to help provide rapid recommendations and guidance about the use of teleNP to the community of clinical neuropsychologists in response to the global novel coronavirus (COVID-19) pandemic. TeleNP is defined here as the application of audiovisual technologies to enable remote clinical encounters with patients to conduct neuropsychological (NP) assessments. These guidance recommendations aim to consider how TeleNP currently aligns with existing practice standards and guidelines. It must first be acknowledged that, so far, the evidence base is limited (e.g. Mara et al, 2020; Brearly et al., 2017; Miller and Barr 2017). The available evidence supports concurrent validity, including robust within-person, across-modality correlations for a variety of tests, but typically under controlled conditions of a remote Tele-NP clinic rather than a less controlled patient home environment, and for many others data are lacking. So far the evidence supporting application of TeleNP in pediatric populations is limited to pilot investigations, and there is currently sparse evidence validating use of TeleNP in under-represented minority or limited English proficient populations.

The current crisis has caused an unprecedented disruption of the usual face-to-face interpersonal contact that characterizes the conventional NP exam. These recommendations necessarily extrapolate from existing evidence, in ways that are believed to be justified in some but not all clinical circumstances by the current crisis. It is hoped that this provisional guidance can serve as a framework for future formal TeleNP guidelines for use in general practice in the future. Given the rapid evolution of impacts associated with the COVID-19 pandemic and the active response of the clinical neuropsychology community to address these challenges, we expect to update these recommendations and guidance when appropriate. The IOPC advocacy website, IOPC.online (https://iopc.online), is continuously updated with regards to research, reimbursement, training, and other relevant resources for TeleNP practice.

These recommendations are an effort to offer help to those providing neuropsychological assessment service under physical distancing constraints during the COVID-19 pandemic. They are not meant to supplant typical practices and guidelines under normal circumstances. That is, when it becomes safe and feasible to resume in-person services, these recommendations should not override existing practice standards. Further, no recommendation provided here should be followed if it contradicts federal, state, or local laws overseeing the practice of psychologists providing assessment services. It is assumed that all psychologists will adhere to respective federal and state rules and regulations, the American Psychological Association’s Ethical Guidelines (APA, 2017), and the Standards for Educational and Psychological Testing (AERA 2014). The difference between accommodation and modification of tests should be understood, considered and, if applicable, reported.

This guidance is intended to allow some continuity of care and services provided during this unprecedented time in cases where clinically appropriate. There are many circumstances where TeleNP will not be feasible and/ or is contraindicated given the complexities of patient characteristics, circumstances, and referral questions. Nothing in this document should be interpreted as a requirement to conduct assessment via Tele-NP. Telemedicine practices in psychology and neuropsychology are complex, and the evidence base for equivalence of cognitive and other interactional measures in a remote, online format compared to a traditional, face-to-face format is extremely nascent. This guidance represents the best, current, available knowledge and opinions of IOPC member organizations. IOPC.online serves as a repository of continuously updated research and resources for Tele-NP practice.

Challenges Posed by the Current COVID-19 Pandemic

January 15th, 2020 marked the first case of COVID-19 in the US. At the writing of these recommendations there were more than one million cases in the US spanning all 50 states, and the count continues to rise. This pandemic has forced the healthcare system and the regulatory ecosystem in which we operate to adapt and evolve at a breathtaking pace. As Governors seek to flatten the curve and slow the pace of community transmission, restrictions in many states have been put into place including the prohibition of all non-essential business, and many states have mandated that businesses utilize telecommuting procedures to the maximum extent possible. This situation has thrust neuropsychologists into the challenging position of considering TeleNP as a practice option quickly, without the luxury of time to fully adapt their procedures to this new medium of patient interaction.

Kruse et. al. (2018) observed that the top barriers to adopting and implementing telehealth are frequently technology-specific. TeleNP poses different challenges across the lifespan: older adults may lack familiarity with online platforms, while children and adolescents may be more attuned to navigating multiple platforms simultaneously. Those from a lower socioeconomic background may lack easy access to technological devices or high speed internet connections. Patients with lower educational levels may struggle to understand how to use the technology. Implementation of TeleNP with an interpreter for limited English proficient patient populations may exacerbate the typical challenges that exist with interpreted in-person assessment. There are situations in which the special challenges TeleNP presents may preclude testing altogether (e.g. children with significant developmental delay, cognitive impairment, and behavioral dysregulation). Furthermore, across clinical populations, the nature of the cognitive symptoms for which patients are seeking neuropsychological evaluation may be barriers to navigating remote technologies.

Beyond Performance-Based Assessment: Breadth and Value of TeleNP

Neuropsychologists possess a wide range of competencies and knowledge that go beyond traditional face to face testing procedures (e.g. Braun et al, 2011). As neuropsychologists consider providing services via telehealth, it is important to recognize the breadth of clinical services we can offer in an episode of neuropsychological care during the COVID-19 pandemic. It is particularly important to recognize that testing is only one component in the broader neuropsychological assessment process. While testing may be limited via telehealth, the other components of the assessment process including the differential diagnostic interview, collateral interviews, and medical records review continue to be readily available. In addition to Tele-NP testing, practitioners can continue to offer TeleNP intake assessments resulting in actionable treatment plans as well as resources to support patients and families. Neuropsychologists can additionally offer consultations for individuals previously seen, and provide support to patients and families who are struggling to cope with the current situation in the context of their developmental or acquired cognitive, emotional or behavioral limitations. Such consultations might include assisting families who are homeschooling children with learning and developmental disabilities, assisting caregivers of patients with dementia and other memory disorders where social distancing instructions are quickly forgotten, and traditional psychotherapy as training allows.

Gaining Competence in Telepsychology and Teleneuropsychology

Guidelines

At this time, there are no formal guidelines for the practice of TeleNP. This will be a longer-term goal for our field. However, guidelines and best practices have been established for telepsychology and these should be considered foundational in the practice of TeleNP. These include the American Psychology Association (APA) guidelines for the practice of telepsychology https://www.apa.org/practice/guidelines/telepsychology, the Association of State and Provincial Psychology Boards (ASPPB) Telepsychology Task Force Principles and Standards https://cdn.ymaws.com/www.asppb.net/resource/resmgr/PSYPACT_Docs/ASPPB_TELEPSYCH_PRINCIPLES.pdf, and the American Psychiatric Association and American Telemedicine Association Best Practices in Videoconferencing-Based Telemental Health https://www.psychiatry.org/psychiatrists/practice/telepsychiatry/blog/apa-and-ata-release-new-telemental-health-guide.

Training Resources

Webinars for developing competence in telepsychology are currently available and should be considered foundational training for developing competence in TeleNP. Through primary telepsychology training, practitioners will develop competencies in establishing telehealth platforms, regulatory issues such as HIPAA compliance, ethical issues including basic informed consent and safety procedures for vulnerable patients, addressing technical issues with patients, and social pragmatics strategies to communicate empathy over a video screen.

Webinars have also more recently become available for developing additional competence specific to TeleNP including understanding issues with standardization, available norms, and technical information/equipment. Webinars addressing risk management for both telepsychology and TeleNP practice are also available and are particularly relevant given the speed with which many practitioners are attempting to shift from in person to telehealth practice models.

In the future, competency in TeleNP will likely require that technology and assessment strategies via telehealth platforms be woven into all levels of education and practice, including professionals beyond their postdoctoral training. Specific training will be necessary to fully implement TeleNP in special populations such as pediatric as well as culturally and linguistically diverse groups.

Licensure Issues

Prior to the COVID-19 pandemic, most states restricted the practice of psychology to psychologists licensed in the state and made no exceptions for telehealth sessions. That is, a psychologist would need to be licensed both in the state where they are physically located when conducting a telehealth session and in the state in which their patient is physically located at the time of the interaction. If either the psychologist or the patient temporarily relocated to another state, the practitioner could be practicing without a license when engaging on telehealth platforms. In the context of the pandemic, some states have temporarily lifted the restrictions. Psychologists may now practice via telehealth or in person across many state lines. The American Psychological Association has created a 50 state grid indicating which states currently have temporary licensing measures during the pandemic. https://www.apaservices.org/practice/clinic/covid-19-telehealth-state-summary.pdf. However, practitioners should also visit the websites of boards governing psychology in any state in which they are not licensed to clarify the rules for temporary or telehealth practice. Particularly as pandemic surges pass, states may return to more restrictive licensing patterns. Additionally, the Association of State and Provincial Psychology Boards has been developing the Psychology Interjurisdictional Compact (PSYPACT) to reduce regulatory barriers and increase access to mental healthcare (https://psypact.org). This is an interstate compact structured so that psychologists licensed in one of a handful of states who have signed on to the compact have the opportunity to apply for reciprocal ability to practice in the other participating states. Neuropsychologists who are currently licensed in a PSYPACT state may wish to contact ASPPB for additional information.

Reimbursement

Ethics Code standard 6.4, “As early as is feasible in a professional or scientific relationship, psychologists and recipients of psychological services reach an agreement specifying compensation and billing arrangements,” (American Psychological Association, 2017), requires neuropsychologists who utilize third party insurance reimbursement to understand the often highly fluid rules that apply to reimbursement in the context of the pandemic.

The IOPC has been leveraging its established network of connections to vigorously advocate for expanded coverage of TeleNP during the current COVID-19 crisis with both public and private insurers. IOPC.online is continuously updated as coverage for Tele-NP expands. It is the responsibility of the individual clinician to determine if an insurer they are billing covers TeleNP. The following sections regarding Medicare, Medicaid, and private insurers reflect information that is available at the time of this document. This information does not guarantee payment from insurance carriers.

Medicare

Prior to the pandemic, Medicare reimbursed CPT© code 96116 (neurobehavioral status) via telehealth. Following effective APA advocacy with the Centers for Medicare and Medicaid Services (CMS), all other neuropsychological testing codes have been temporarily approved for Medicare telehealth reimbursement. The exception is CPT© code 96121, the add on code used to signify additional time spent engaging in a neurobehavioral status exam. Currently, this is not on the list of temporarily approved codes. APA believes this is a typo/error and is engaged with CMS to rectify the situation. Knowing this oddity of CPT© code billing is important as many private insurers and state Medicaid carriers have pandemic coverage policies that specify they are following CMS COVID-19 pandemic policy. Details regarding place of service codes and modifiers, as well as reimbursement updates are available on IOPC.online.

Private Insurers and State Medicaid Carriers

On March 27, 2020, Congress passed the Coronavirus Aid, Relief, and Economic Security Act (H.R.748). The CARES Act signaled strong support for telehealth and recognition that expanding access to telehealth is critical to defeating COVID-19. Additionally, many state Governors have signed executive orders mandating Medicaid carriers and private insurers in their states cover telehealth. The American Psychological Association has created a 50 state grid specifying which states have such executive mandates. https://www.apaservices.org/practice/clinic/covid-19-telehealth-state-summary.pdf.

Many private insurers and state Medicaid providers have temporarily authorized neuropsychology CPT© codes to be reimbursed via telehealth. It is important to note that Medicare HMO products are state or regionally based and do not have to follow Medicare coverage decisions. Many Medicare HMOs are currently more restrictive in their temporary coverage, allowing limited or no Tele-NP reimbursement.. The IOPC.online website includes a state by state catalogue of reimbursement for Tele-NP services. https://iopc.online/state-by-state-teleneuropsychology-resources. As advocacy efforts successfully expand the temporary coverage, the site is continuously updated.

Informed Consent

Recommendations for informed consent in telepsychology practice are available from multiple national organizations including the American Psychological Association (https://www.apa.org/practice/programs/dmhi/research-information/informed-consent-checklist). These should be considered foundational for TeleNP informed consent practices. TeleNP requires additional consent regarding issues outlined below.

  • It should be explained clearly and in language the patient or their representatives understand, that:
    • Standard test administration will be modified, and this may affect results in ways that are so far unknown. This will reduce confidence in the diagnostic conclusions and may impact recommendations for treatment.
    • Involvement of a third-party in the TeleNP session (caregiver, guardian, parent, facilitator) may add additional concerns about the impact of observation on performance.
    • Error may be compounded when TeleNP procedures are used with people who come from culturally and linguistically diverse populations, require an interpreter during TeleNP, or have limited experience/comfort with the technology being employed.
    • There will be a loss of some qualitative data usually obtained during an in-person exam, and this loss may reduce the richness of the clinical data and further limit conclusions and recommendations.
    • TeleNP may pose additional risks to privacy and confidentiality.
    • TeleNP test results may not be acceptable by organizations for administrative purposes (e.g. seeking accommodations or documenting a diagnosis).
  • TeleNP consent forms should always be available in the language of your patient.

  • Refer to the Special Populations section below for further consideration of factors affecting individuals’ ability to participate meaningfully in the TeleNP exam.

Interviewing and Feedback in Teleneuropsychology

Conducting interviews and feedback through TeleNP, like conducting psychotherapy using telehealth methods, should aim to maximize simulation of standard, in-person practice. Use of the “Gallery” view may help engage, and enable the clinician to assess the reactions of multiple participants during interviewing and feedback processes. When private interviews or feedback conversations with specific family members or caregivers are clinically indicated, some participants can be asked to temporarily sign off or leave the room.

Screen-sharing features may help illustrate to patients and caregivers specific test results during feedback sessions, or share with the group visualizations in the form of charts or other helpful graphical explanatory tools.

In the era of physical distancing, key stakeholders in a patient’s family normally included in an interview or feedback session may be in a different location. For example in a dementia evaluation, there may be adult children from several households that need to be sent video conferencing instructions in advance of the session. On a positive note, family stakeholders living out of state that might not have been able to be present for an in-person interview or feedback session due to travel limitations now have the opportunity to participate in the care of their loved one.

The typical social communication feedback loop that provides practitioners and patients with rich data from body language, facial expression, and tone of voice is often muted over telehealth platforms. Anxious and or cognitively limited patients may experience this feedback as even more difficult to access. It is important to talk about this with patients and families at the beginning of interviews and feedback sessions and discuss ways that participants might address the issue.

Development of clinical rapport over telehealth platforms may vary by generation, with adolescents and children feeling the greatest level of comfort with the technology. Older adults, individuals from low SES populations with less experience with technology, and individuals for whom the technology navigation tools may be in a second or third language, may require considerable emotional reassurance as well as technical assistance as they interact with a virtual doctors’ office. Normalizing the difficulty with new technology is particularly important as we meet patients for the first time who otherwise might feel they are being judged as ‘cognitively incapable’ due to their difficulty with the telehealth technology. It is important that ‘pre-interview’ technology checks and run throughs be scheduled with support staff if possible, or in solo private practice substantial extra time for technology adjustments be built into the initial interview session.

Special considerations may be needed with unique patient populations such as those from linguistically and culturally diverse groups. There may be potential higher levels of distrust associated with compilation of personal information/data. There may also be amplified concerns about how private information may be used to identify an examinee who needs to protect their personal or family’s privacy. It is essential to consider legal vs. illegal resident status in a climate where there may be increased risk of adverse legal actions (e.g., deportation). Interviewing should be conducted by a linguistically and culturally competent neuropsychologist consistent with APA Standards for Educational and Psychological Testing.

Reporting Results of TeleNP Assessment Limitations

Reports of neuropsychological assessments based on TeleNP should include clear statements about the limitations posed by non-standard administration and the potential impact this might have on diagnostic conclusions and treatment recommendations. For example: “Due to circumstances that prevent in-person clinical visits, this assessment was conducted using telehealth methods (including remote audiovisual presentation of test instructions and test stimuli, and remote observation of performance via audiovisual technologies). The standard administration of these procedures involves in-person, face-to-face methods. The impact of applying non-standard administration methods has been evaluated only in part by scientific research. While every effort was made to simulate standard assessment practices, the diagnostic conclusions and recommendations for treatment provided in this report are being advanced with these reservations.”

The same documentation of testing limitations extends to the use of interpreters, patient’s educational background, computer literacy, internet connectivity, and other factors that may interact with and further limit the comparability of TeleNP with standard assessment practice.

Reports of neuropsychological assessments based on TeleNP should include specific descriptions of the platforms used and how the tests were adapted or modified, including specific administration modifications.

Telehealth and Teleneuropsychology Platforms

Healthcare disparities

Telehealth delivery exposes deep societal disparities in patient’s access to computers and internet connectivity. It is important for neuropsychologists to examine assumptions that most patients share our privileged access to technology that is foundational for conducting the testing portion of a Tele-NP assessment. Many patients have no access to computers but might have a smartphone. Others will not have a smartphone or reliable internet access.

Choosing a Platform

Under normal circumstances, telehealth platforms must be HIPAA-compliant, have an established Business Associate Agreement (BAA), and follow any additional legal and regulatory requirements that are relevant in your state and institution. During the Covid-19 pandemic, HIPPA regulations have been relaxed in many circumstances, but we recommend still using a HIPAA-compliant platform if possible. Practitioners should check with their state board of psychology, and institutional compliance, legal affairs, and/or risk management offices to ensure they are in compliance.

EMR Based Platforms

There are platforms “built in” to many existing institutional electronic medical records (EMRs), and some platforms use third-party or free-standing platforms. The methods built into existing EMRs have the general advantage of assuring compliance with health system regulations. Given that large health systems were mandated to deploy EMRs in 2014 as part of the Patient Protection and Affordable Care Act, these are widespread. Major platforms include EPIC, Cerner, and the VA system (which includes some proprietary systems, but also has contracted with Cerner for some of its hospitals).

Implementation of EPIC systems varies by site, but certain features are likely available at all EPIC sites. For example, UCLA’s EPIC system uses “MyChart” to enable two-way audiovisual communication with patients in a HIPAA-compliant secure portal. Consent processes, billing, and smart-phrases to document telehealth procedures have been developed although these do not automatically propagate to all EPIC sites. As another example, Cerner, a health care innovation and communication technology company, has a “Virtual Health” platform that supports remote access and patient care in rural areas. They have a site dedicated to COVID-19 response (https://www.cerner.com/pages/covid-19). There is a Patient Observer function that is recommended for remote interactions using telehealth audiovisual technology.

Third-Party Telecommunications Platforms

Third-party telecommunication platforms may be the only option for many practitioners who are not part of a large health system and/or do not have access to one of the large EMR platforms. These platforms may have features superior to those provided by the large EMR platforms, but clinicians must be vigilant to be sure the platform they choose is compliant with both state and federal regulations.

There are a number of telehealth platforms currently in use, such as Zoom, Doxy.Me, VSee, Theranest, and SimplePractice. Zoom is among the most widely used current platforms for teleconferencing, and in some health systems it has been approved for clinical interactions and is considered HIPAA-compliant (this may apply only for the Professional version; the free version of Zoom does not include a Business Associate Agreement (BAA) and only allows unlimited time with one-on-one use. Zoom’s free multi-user conference mode is limited to 40 minutes). Zoom enables multiple participants so it works better for patient-trainee-supervisor interactions than some other platforms. Zoom includes a number of features that are helpful in conducting TeleNP assessments (and these ideas may be adapted to other platforms). A disadvantage with Zoom and some other telehealth platforms is that these require the patient to download an app. Some others (e.g., Doxy.Me) have an advantage in that patients can simply check in by clicking on a Virtual Waiting Room button that is embedded onto a website, requiring no downloads. Some telehealth platforms do not have user interfaces or capabilities to switch to different language modes.

Practitioners should choose a telehealth platform with a user interface in the language of one’s patients. It is important to become familiar with language features in advance of any TeleNP sessions.

Technical Specifications

Practitioners should carefully consider technical specifications prior to conducting TeleNP exams. A comprehensive guide to video platforms and technological standards is available through the National Telehealth Technology Assessment Resource Center at: http://telehealthtechnology.org/toolkit/clinicians-guide-to-video-platforms/ and standards for audio/video can be found here: http://telehealthtechnology.org/toolkit/desktop-video-applications-standards/.

  • Bandwidth assessment: All transmissions, and particularly video transmissions, are heavily impacted by bandwidth issues. It is important to test your clinician-side internet speed, connections.
    • Two-way live video services through consumer devices should have a bandwidth of at least 384 Kbps in both downlink and uplink directions. Higher bandwidth speeds may be needed for specialty services (ATA, 2014).
    • The FCC recommends internet access at varying speeds depending on the practice setting. A list of minimum speeds is provided here: https://www.healthit.gov/faq/what-recommended-bandwidth-different-types-health-care-providers). Bandwidth considerations should include: number of users, user locations, real-time transactions, hardware, and storage technology.
    • For full functionality in healthcare applications, the FCC recommends 2 Mbps for SD videoconferencing and 10 Mbps for HD videoconferencing (FCC, 2010).
    • The FCC provides on their website mapping of broadband availability nationwide: https://www.fcc.gov/health/maps.
    • Both patients and providers should pre-test the connection before starting the session to ensure the link is of sufficient quality for the interaction (ATA, 2014)
    • Whenever possible, each party should use the most reliable connection method to access the internet (ATA, 2014).
    • The platform should be able to adapt to changing bandwidth environments without losing the connection (ATA, 2014).
  • Managing patient-side connectivity: It is recommended that patients use a private wifi or hard-wired connection when engaging in TeleNP at home rather than working on a public wifi.

  • Equipment specifications:
    • To the extent possible, both the professional and patient site should utilize high quality video cameras, audio devices, and related data capture/transmission equipment appropriate for the visit (ATA, 2014).
    • Devices should have up-to-date security software per manufacturer’s recommendations as well as device management software (ATA, 2014).
    • All audiovisual data transmission should occur through the use of encryption (at least on the side of the neuropsychologist) that meets recognized standards (ATA, 2014).
    • Professionals should be familiar with all devices and software that they are utilizing in providing care over distances, and have taken any required specialty training, prior to providing TeleNP (ATA, 2014).
    • The National Telehealth Technology Assessment Resource Center has a resource page including information about innovative technologies as well as technical assistance for selecting appropriate technologies at: http://telehealthtechnology.org/
    • Display options: consider both your experience and that of your patient. Note that Pearson recommends a display size of at least 9.75” diagonal on the patient side.

Strategies For Conducting a Teleneuropsychology Episode of Care

Clear, well thought out strategies for conducting psychotherapy via telehealth have been published and should be considered foundational in setting up a virtual neuropsychology office visit. The APA Telepsychology Checklist (https://www.apa.org/practice/programs/dmhi/research-information/telepsychological-services-checklist) provides a solid starting point. Here are key points to consider in ushering patients through a TeleNP session:

Prior to Sessions

  • Screen patients to make sure TeleNP is appropriate given their clinical and cognitive status. This is particularly important in patient populations referred for neuropsychological assessment with sensory, cognitive and behavioral limitations that interact directly with the utility of TeleNP. It is important to recognize that TeleNP may not be appropriate for many patients.

  • As TeleNP may enable more access to a linguistically and culturally competent neuropsychologist within a state or across state lines it is strongly recommended that a referral is made to one of these providers before interpreters are utilized given the known limitations inherent in interpreted exams. Many states have relaxed licensure requirements for telehealth during the pandemic. Prior to referring, make sure the provider is licensed (or license requirements have been waived) in both the state where the patient will be and the state where the provider will be during the Tele-NP session.

  • Anticipate variable levels of access to appropriate equipment, wireless service (e.g., data/minutes limitations), and software. Providers should be cautious about assuming patients have access. If patients do not have access, assist the patient (and facilitator if relevant) in identifying a suitable device for the evaluation including borrowing a device and/or leveraging hospital/community resources to increase access. In the context of social distancing during the pandemic, borrowing devices may be impossible. If the patient is expected to view stimuli projected from a webcam, we strongly recommend against use of smartphones as compared to computer screens.

  • Clearly define the need for a private, quiet, distraction free space on the patient end to conduct the session. Be cautious about assuming patients have access to such a space. This may require negotiating with facilitators to agree to turn off household TVs, mute cell phones, remove pets from a room, take siblings or other family members out of the house for a walk (as allowed with stay at home orders/ quarantines) or other arrangements.

  • Determine if your exam will require an on-site (with patient) facilitator, and clearly define the role of that facilitator in advance of the session. This may require an in depth conversation with the facilitator regarding boundaries between facilitating interacting with the TeleNP platform vs. ‘hinting’ or ‘helping’ to improve performance. Make sure you obtain appropriate consents and have a plan to manage their interactions with the patient and the technology during the assessment.

  • Conduct a pre-TeleNP session to share information about the structure of the upcoming session(s), begin the informed consent process (see above for discussion of DocHub or DocuSign), review billing policies, provide links to intake forms, collateral release and contact forms, and arrange back-up plans for communication of the TeleNP session is disrupted for technical reasons.

Beginning a Session

  • At the start of any virtual visit, disable recording on the telehealth platform as recording poses challenges to test security, and is specifically prohibited by some vendors. Zoom has an option in “Settings” to disable recording options.

  • Confirm identity of the patient, review the accuracy of call-back numbers, discuss privacy issues and prohibitions against recording, and turn off other apps/notifications.

  • For pediatric populations, begin and end sessions with the parent/guardian in the room. Remind the parent/guardian that they need to remain in the house, particularly if the patient is a minor or requires onsite adult supervision. Obtain the best phone number to reach the parent/guardian at the beginning of the session in the event you need to make contact during the video session. Make sure the parent/guardian also has your best contact number.

  • Assist patient and/or facilitator to scan the room for potentially distracting stimuli. Headphones connected to the videoconferencing device may assist in eliminating distractions. Ask patients to “hide self view” on the screen, so that they are not distracted by seeing their own face during testing.

  • Make use of the “Breakout Rooms” feature so that patients can be in a “waiting room” while you prepare stimuli for presentation, or enabling trainees to have discussion separately with their supervisors.

  • Consenting patients and facilitators on the limitations of TeleNP (see consent section above) is critical. Even if consent forms have been signed in advance during a pre-session with office staff, limitations of TeleNP should be revisited in depth at the beginning of the TeleNP session and again during the feedback session.

  • Ensure the patient has all needed materials to participate in assessment, if applicable. If materials have been provided, instruct the patient and/or facilitators not to open or view materials until instructed to do so during the session. Consider including a self-addressed envelope with pre-paid postage so that materials may be returned easily and promptly.

During the Testing Process

  • Track and document the following throughout:
    • Technological problems such as disconnection, video and/or audio outage, lag in video, etc.
    • Environmental interruptions and distractions including sounds, family members or pets walking in, etc.
    • Specific patient characteristics that make it difficult to engage with the TeleNP testing experience (e.g., sensory, motor, language etc.)
  • Utilize the “Share Screen” feature so that you can present higher quality images of test stimulus materials, compared to for example holding stimulus materials up to the camera. Many test companies are making stimuli available in digital form for this purpose.

  • The logistics of TeleNP test stimuli presentation may be facilitated by the use of specific equipment (i.e., use of wall mount monitor stands to hold tablet or laptop instead of holding booklets on a clipboard).

  • Show the patient or facilitator in how to use the Shared Screen Remote Control transfer so that you can give control to the examinee when required for tests that are usually administered on a computer in the clinic. Note that examiners must be alert and return control to the examiner’s computer as soon as the examinee has completed the test.

  • Managing the patient-side work product: You will need to create methods to have the patient or facilitators “help” you perform certain examiner functions. For example, usually the Examiner will take patient’s drawings away after the patient has produced them so that they are not visible (e.g., Visual Reproductions or Rey-Osterrieth Complex Figure Test). Patients can at the beginning of the session set up a folder or envelope into which you can observe them placing these products immediately upon completion. At the beginning of the testing session, clearly explain these procedures to the patient/ facilitator.

  • Under no circumstances should you leave your computer unattended while an examinee has control over your computer. This could both incur HIPAA violations and/or pose a security risk including access to your personal or business files or data. Examiners should check with their local compliance experts to assure security and privacy guidelines are being followed.Assist the patient in arranging the camera in an optimal position to maximize viewing of the patient in order to observe the patient’s work and make behavioral observations. This is likely to be limited compared to in-person assessment.

  • Multi-screen options: if feasible, use a multi-screen option on the clinician side to facilitate visualization and separation of patient-facing and clinician-facing content. This helps with optimal efficiency in test administration, which is critical in certain populations such as pediatrics due to distractibility.

  • At the end of the session, ask the patient to call parent/caregiver back to the room to conclude the session, if applicable. Call the parent/guardian by phone if needed.

IOPC.online has additional strategies for managing technical issues of TeleNP testing sessions.

Test Selection

The Standards for Educational and Psychological Testing (AERA et al., 2014) specifically covers test construction, evaluation, documentation, fairness in testing, and testing applications. All of these apply to TeleNP as they do to in-person assessment. The APA Multicultural Guidelines (American Psychological Association, 2017) likewise apply to TeleNP as they do to standard assessment. For example, specific consideration of an examinee’s primary language as well as other important cultural factors such as level of education, acculturation, country of origin, ad socioeconomic status is important when considering test selection for TeleNP as these factors already pose challenges in traditional, face-to-face testing that is predominantly standardized with English-language samples.

Literature Review

The current literature on TeleNP is compiled on the IOPC website (see https://iopc.squarespace.com/teleneuropsychology-research). A useful table originally compiled by a VA workgroup in 2018 further shows specific tests that have been studied in TeleNP dementia evaluations, including: The Boston Naming Test, Brief Visual Memory Test - Revised; California Verbal Learning Test - Second Edition; Clock Drawing Test; Delis-Kaplan Executive Function System (Proverbs Test); Digit Span; Hopkins Verbal Learning Test-Revised; Independent Living Scales (Health and Safety Subtest); Mattis Dementia Rating Scale (Memory I subtest); Modified Rey-Osterrieth Complex Figure Test (Copy, Recall and recognition); Oral Trail Making Test, Parts A and B; Repeatable Battery for the Assessment of Neuropsychological Status (Forms A and B); Rey-Osterrieth Complex Figure Test (Copy, 3” Delay); Trail Making Test, parts A and B; Test of Practical Judgement; Verbal Fluency (Semantic, Phonemic Fluency); Wechsler Memory Scale - Fourth Edition (Logical Memory I, II; Adult and Older Forms). It is important to note that most of the research was conducted via remote TeleNP clinic where appropriately calibrated hardware and technician support was available rather than a home TeleNP setting which has considerably less control over extraneous variables in both environment and technology.

Familiar Tests Adapted to TeleNP

The IOPC website summary highlights that multiple familiar tests have been used successfully in a TeleNP format. The Brearly et al (2017) systematic review and meta-analysis summarized 12 studies over an age range of 34 to 88 years. The overall difference between in-clinic and TeleNP administration was small (Hedges g = −.03), an effect size that was not statistically significant and given that this reflects a difference of 1/33rd standard deviation (SD), it would not be considered clinically significant. But it should be recognized this is a summary statistic across multiple studies and methods. Given that clinical neuropsychology involves interpreting individual test results and their patterns, the results should be seen as encouraging but not adequate to generalize to the practice of TeleNP broadly. Instead, the findings may be seen as encouraging, highlighting certain moderating factors that need to be considered, and pointing to certain kinds of tests that are likely to be more easily used in TeleNP than others. Age and internet connection speed were key moderators, with results being less consistent in patients older than age 75 and on slower connections. The TeleNP scores for untimed tasks and those allowing for repetition were within 1/10th standard deviation (SD) of in-clinic scores. Results on verbal tests including digit span, verbal fluency, and verbal learning and memory test scores were particularly close to in-clinic findings. Boston Naming Test scores were 1/10th SD below in-clinic scores. Tests involving a motor component were considered too heterogeneous to interpret.

Web-Based and Computerized Testing Platforms

Web-based and computerized testing platforms have been considered as an alternative to administering conventional paper and pencil tests using telehealth methods. Fortunately, we possess specific guidelines for computerized neuropsychological assessment devices (please refer to Computerized Neuropsychological Assessment Devices: Joint Position Paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology (Bauer et al., 2012)).

Conceptually, it would seem reasonable to consider computerized or web-based assessments to assist with remote testing, when our patients are interacting via their own computer. Unfortunately, most of the currently deployed computerized and web-based tests have not undergone the kinds of normative and validation studies that have been used for conventional in-person assessments. Moreover, some companies conducted normative or validation studies using in-laboratory versions of the computer tests rather than remote administration, so these studies are not clearly relevant to their web-based versions. With downloadable tests (those that run on local systems), it may be difficult or not feasible to have patients installing this software on their own computers, and evaluating the security risks of this practice may be daunting. The web-based platforms may appear to resolve some of these issues and vendors of web-based products may provide reassurance that their programs are HIPAA compliant, but the potential risks to privacy and security ultimately fall on the clinician. The existing web-based systems also face the challenge of not necessarily incorporating full evaluation of patient-side system characteristics, and given the findings of the Brearly et al (2017) meta-analysis and other findings showing possible clinically significant discrepancies between scores on a fixed battery depending only on the computer software version (e.g., Roberson et al., 2018).

The Disruptive Technology Initiative of the American Academy of Clinical Neuropsychology recently surveyed leading vendors of applications for TeleNP assessment, and presented results of this survey at the 2019 Annual Meeting in Chicago, IL (Barr et.al, 2019). The slides from this presentation are available on IOPC.online (https://iopc.online/remote-neuropsychological-assessment-models). Overall, the findings from this survey revealed some promising results but generally less persuasive evidence of robust normative standards and validation data to support their immediate adoption for clinical NP in the United States. Some vendors have received “FDA Approval” for their products but it should be recognized that the standards for FDA clearance of devices is not the same as it is for pharmaceutical products, and the device clearance process does not require rigorous clinical validation of the devices (the focus is more on “comparability” to other products, and safety). The following products were identified, and for most there was not sufficient evidence available to provide a recommendation that they be used to replace conventional neuropsychological assessment, based on the criteria outlined by Bauer et al (2012): Amsterdam Cognition Scan, BrainCheck, CANTAB Mobile and CANTAB insight, CNS Vital Signs, CogniFit, Cognivue, Cogstate, Digital MOCA, Food for the Brain, Lumosity NeuroCognitiv Performance Test, NeuroTrax, Philips’ IntelliSpace Cognition, TabCat, TestMyBrain. It currently remains up to clinicians to consider carefully the relevant evidence and make determinations about whether any of these tools may be useful in clinical exams. Unfortunately, there is little regulation of the advertising and promotion of these products, so clinicians should be wary of claims made on vendor websites and investigate the demonstrated validity of the product themselves. The caveats noted above, regarding the capacity to observe the patient during the examination, remain important to consider before using these products to draw clinical conclusions.

Addressing/Acknowledging Threats to Test Validity with TeleNP

It is important to recognize that TeleNP methods pose limitations on our capacity to observe and document behavior during the administration of any given test in the same way that we can in person. These limitations may be exacerbated with culturally/ linguistically diverse patients.

In response to the current crisis, some test publishers have issues permissions or statements of “no objection” to use their materials in non-standard administration formats, but these continue to place responsibility on the clinician to assure the validity of assessment and integrity of the test materials. For example, a WPS Publishing statement on teleassessment indicates: “All of our current individually-administered assessments (“performance tests”) were standardized using in-person administration. For these tests telehealth methods would be considered an adaptation of the standardized administration and should be taken into consideration when reporting and interpreting the results of a remote administration. The Pearson letter further states: “Before test administration, the qualified professional must obtain documented agreement from the examinee that the session will not be recorded, reproduced or published, and that copies of the materials will not be made. Further, the qualified professional may not utilize recording capabilities to record live test administrations.” (WPS, 2020).

Modification of Familiar Tests for Telehealth Platforms

There is not sufficient evidence to provide clear guidance about how to modify most specific tests for TeleNP, although some TeleNP research articles describe how specific tests were administered in research studies that obtained comparable results to in-person assessment (e.g. Barcellos et al., 2018, Galusha-Glasscock et al., 2016). The general guidance is to simulate in-person administration as closely as possible.

Because there are insufficient data to suggest any systematic modifications of norms used to interpret TeleNP test results differently, current recommendations are to rely on normative and validity data obtained using the standard assessments, with clear documentation in the report, including a note in any test-score summary sheet that lists reference scores, percentiles, or other interpretive comments, that administration was non-standard and that the non-standard administration is likely to result in measurement error. Active studies regarding reliability, validity, and normative considerations are warranted for future regular use of TeleNP practice.

Test Vendor Resources

Major test vendors are actively in the process of facilitating access to their test materials and the appropriate application of TeleNP during the COVID-19 pandemic. A listing of tests available from MHS, PAR, Pearson, and WPS is available on IOPC.online https://iopc.online/remote-neuropsychological-assessment-models.

Managing In-Person Exams When Necessary and Feasible When There is Concern About COVID-19 Exposure

While many health systems and individual practitioners have stopped or severely limited their conduct of in-person neuropsychological examinations, there may be exceptions when assessments must occur in a face-to-face fashion. Additionally, as stay at home orders and recommendations to postpone non-emergent medical procedures are lifted, practitioners will need to consider how to proceed with in-person evaluations as appropriate where containment and mitigation strategies may persist.

It is important for private practitioners to consider risk management issues prior to deciding to proceed with in-person assessments if they are practicing in a state that has not yet instituted strict work from home guidelines, particularly if directing employees and psychometrists to do so. Following and documenting appropriate screening for patients, as well as staff, (see below) is crucial to protect patients, staff, and your practice from the effects of exposure. Neuropsychologists who are also employers should seek legal guidance regarding specific documentation of screening and other activities prior to allowing staff to have contact with patients.

In some health systems, neuropsychologists continue to be available and are providing services involving direct patient contact if necessary for emergency procedures or in other exigent circumstances. It is imperative that neuropsychologists maintain strict social distancing, exposure management, and disinfection practices as patients are being seen. Suggested practices include:

  • Screening for illness, contacts with people infected by COVID-19, and/or travel within the prior 14 days to any area impacted by coronavirus (this now includes all 50 states of the United States, so questions about travel may not help screening at this point; checking with CDC and your own state and county officials may be important to get updated guidance).

  • Personal protective equipment (masks, gloves) should be worn by both the examiner and the patient, and all current safety guidelines from the CDC and your practice location should be followed. If personal protective equipment is not available, the neuropsychologist should weigh the risks and benefits before deciding whether to evaluate using only behavioral precautions.

  • Methods to increase distance and manage patient-clinician contacts:
    • Unstandardized administration modifications should be considered to maintain appropriate social distancing guidelines. Arrange testing room chairs, tables to maximize distance. Conduct as much of the exam as possible more than 6 feet from the participant
    • Choose tests that can be used at a distance
    • Use non-manipulatives when possible
    • Clean the room and materials in advance of the appointment
    • Use screen-based stimuli (that can be more easily cleaned)
    • iPad separated testing (e.g., Q-interactive)
    • Placing used (i.e., exposed) materials to a safe location in the room for disinfection later
  • Disinfecting materials (e.g., cleansers, laminating materials to make these easier to use): Pearson Assessments has tips for keeping materials clean here: https://www.pearsonassessments.com/content/dam/school/global/clinical/us/assets/telepractice/disinfecting-test-materials.pdf. These include some practical tips about how to administer tests without patients touching materials, ensuring patients do not have latex allergies prior to offering gloves, and keeping resealable plastic bags on hand to store “dirty” materials that must be disinfected prior to reuse.

Footnotes

1

The Inter Organizational Practice Committee (IOPC) is a committee of the practice chairs of the American Academy of Clinical Neuropsychology/ American Board of Clinical Neuropsychology, the National Academy of Neuropsychology, Division 40 of the American Psychological Association, the American Board of Professional Neuropsychology, and the American Psychological Association Services, Inc. (APAS) tasked with coordinating advocacy efforts and improving the practice climate for Neuropsychology.

References

  1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (U.S.). (2014). Standards for educational and psychological testing. Washington, DC: AERA [Google Scholar]
  2. American Psychological Association. (2017). Ethical principles of psychologists and code of conduct (2002, amended effective June 1, 2010, and January 1, 2017). https://www.apa.org/ethics/code/
  3. American Psychological Association (2017). Multicultural Guidelines: An Ecological Approach to Context, Identity, and Intersectionality. [PDF]. Retrieved from: http://www.apa.org/about/policy/multicultural-guidelines.pdf [DOI] [PubMed]
  4. American Telemedicine Association. (2014). Core operational guidelines for telehealth services involving provider-patient interactions. Washington, DC: Author. [Google Scholar]
  5. Barcellos LF, Bellesis KH, Shen L, Shao X, Chinn T, Frndak S, … & Benedict RH. (2018). Remote assessment of verbal memory in MS patients using the California Verbal Learning Test. Multiple Sclerosis Journal, 24 (3), 354–357. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Barr W, Bilder R, Miller J, and Millman T (June, 2019). Disruptive Technologies in Neuropsychological Assessment: What is Our Role in New Models of Care? Workshop conducted at the conference of the American Academy of Clinical Neuropsychology. Chicago, Il. [Google Scholar]
  7. Barton C, Morris R, Rothlind J, & Yaffe K (2011). Video-telemedicine in a memory disorders clinic: evaluation and management of rural elders with cognitive impairment. Telemedicine and e-Health, 17 (10), 789–793. [DOI] [PubMed] [Google Scholar]
  8. Bauer RM, Iverson GL, Cernich AN, et al. (2012). Computerized Neuropsychological Assessment Devices: Joint Position Paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology, Archives of Clinical Neuropsychology, Volume 27, Issue 3, May 2012, Pages 362–373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Braun M, Tupper D, Kaufman P, McCrea M, Postal K, Westerveld M, Wills K, and Derr T (2011) Neuropsychological assessment: A valuable tool in the diagnosis and management of neurologic, neurodevelopmental, medical, and psychiatric disorders. Journal of Cognitive and Behavioral Neurology 24(3) 107–114. [DOI] [PubMed] [Google Scholar]
  10. Brearly TW, Shura RD, Martindale SL, Lazowski RA, Luxton DD, Shenal BV, & Rowland JA (2017). Neuropsychological test administration by videoconference: a systematic review and meta-analysis. Neuropsychology review, 27 (2), 174–186. [DOI] [PubMed] [Google Scholar]
  11. Cullum C, Weiner M, Gehrmann H, & Hynan L (2006). Feasibility of telecognitive assessment in dementia. Assessment, 13 (4), 385–390. [DOI] [PubMed] [Google Scholar]
  12. Cullum C, Hynan L, Grosch M, Parikh M, & Weiner M (2014). Teleneuropsychology: evidence for video teleconference-based neuropsychological assessment. Journal of the International Neuropsychological Society, 20 (10), 1028–1033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. de Jager CA, Budge MM, & Clarke R (2003). Utility of TICS-M for the assessment of cognitive function in older adults. International journal of geriatric psychiatry, 18 (4), 318–324. [DOI] [PubMed] [Google Scholar]
  14. DeYoung N, & Shenal BV (2019). The reliability of the Montreal Cognitive Assessment using telehealth in a rural setting with veterans. Journal of telemedicine and telecare, 25 (4), 197–203. [DOI] [PubMed] [Google Scholar]
  15. Federal Communications Commission (2010). Health Care Broadband in America: Early Analysis and a Path Forward. [PDF file]. Retrieved from https://transition.fcc.gov/national-broadband-plan/health-care-broadband-in-america-paper.pdf
  16. Galusha-Glasscock JM, Horton DK, Weiner MF, & Cullum CM (2015). Video teleconference administration of the Repeatable Battery for the assessment of neuropsychological status. Archives of Clinical Neuropsychology. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Grosch MC, Gottlieb MC, & Cullum CM (2011). Initial practice recommendations for teleneuropsychology. The Clinical Neuropsychologist, 25 (7), 1119–1133. [DOI] [PubMed] [Google Scholar]
  18. Grosch MC, Weiner MF, Hynan LS, Shore J, & Cullum CM (2015). Video teleconference-based neurocognitive screening in geropsychiatry. Psychiatry Research, 225 (3), 734–735. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4410696/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gurnani AS, John SE, & Gavett BE (2015). Regression-based norms for a bi-factor model for scoring the Brief Test of Adult Cognition by Telephone (BTACT). Archives of Clinical Neuropsychology, 30 (3), 280–291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Harrell KM, Wilkins SS, Connor MK, & Chodosh J (2014). Telemedicine and the evaluation of cognitive impairment: the additive value of neuropsychological assessment. Journal of the American Medical Directors Association, 15(8), 600–606. [DOI] [PubMed] [Google Scholar]
  21. Hildebrand R, Chow H, Williams C, Nelson M, & Wass P (2004). Feasibility of neuropsychological testing of older adults via videoconference: implications for assessing the capacity for independent living. Journal of Telemedicine and Telecare, 10 (3), 130–134. [DOI] [PubMed] [Google Scholar]
  22. Jacobsen SE, Sprenger T, Andersson S, & Krogstad JM (2003). Neuropsychological assessment and telemedicine: a preliminary study examining the reliability of neuropsychology services performed via telecommunication. Journal of the International Neuropsychological Society, 9 (3), 472–478. [DOI] [PubMed] [Google Scholar]
  23. Kirkwood KT, Peck DF, & Bennie L (2000). The consistency of neuropsychological assessments performed via telecommunication and face to face. Journal of Telemedicine and Telecare, 6 (3), 147–151. [DOI] [PubMed] [Google Scholar]
  24. Kruse CS, Karem P, Shifflett K, Vegi L, Ravi K, & Brooks M (2018). Evaluating barriers to adopting telemedicine worldwide: A systematic review. J Telemed Telecare, 24 (1):4–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lachman ME, Agrigoroaei S, Tun PA, & Weaver SL (2014). Monitoring cognitive functioning: Psychometric properties of the Brief Test of Adult Cognition by Telephone. Assessment, 21 (4), 404–417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Loh PK, Ramesh P, Maher S, Saligari J, Flicker L, & Goldswain P (2004). Can patients with dementia be assessed at a distance? The use of Telehealth and standardised assessments. Internal Medicine Journal, 34 (5), 239–242. [DOI] [PubMed] [Google Scholar]
  27. Loh PK, Donaldson M, Flicker L, Maher S, & Goldswain P (2007). Development of a telemedicine protocol for the diagnosis of Alzheimer’s disease. Journal of Telemedicine and Telecare, 13 (2), 90–94. [DOI] [PubMed] [Google Scholar]
  28. Marra D, Hamlet K, Bauer R, and Bowers D (2020). Validity of Teleneuropsychology for Older Adults in Response to COVID-19: A Systematic Review. Manuscript in review. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Miller JB, & Barr WB (2017). The technology crisis in neuropsychology. Archives of Clinical Neuropsychology, 32 (5), 541–554. [DOI] [PubMed] [Google Scholar]
  30. Montani C, Billaud N, Tyrrell J, Fluchaire I, Malterre C, Lauvernay N, … & Franco A (1997). Psychological impact of a remote psychometric consultation with hospitalized elderly people. Journal of Telemedicine and Telecare, 3 (3), 140–145. [DOI] [PubMed] [Google Scholar]
  31. Parikh M, Grosch MC, Graham LL, Hynan LS, Weiner M, Shore JH, & Cullum CM (2013). Consumer acceptability of brief videoconference-based neuropsychological assessment in older individuals with and without cognitive impairment. The Clinical Neuropsychologist, 27 (5), 808–817. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Roberson B, Arrieux J, Russell K, & Cole W (2018). Differences in Reaction Time Latency Error on the ANAM4 Across Three Computer Platforms. Archives of Physical Medicine and Rehabilitation, 99(11), p.160. [Google Scholar]
  33. Tun PA, & Lachman ME (2006). Telephone assessment of cognitive function in adulthood: the Brief Test of Adult Cognition by Telephone. Age and Ageing, 35 (6), 629–632. [DOI] [PubMed] [Google Scholar]
  34. Unverzagt FW, Monahan PO, Moser LR, Zhao Q, Carpenter JS, Sledge GW, & Champion VL (2007). The Indiana University telephone-based assessment of neuropsychological status: a new method for large scale neuropsychological assessment. Journal of the International Neuropsychological Society, 13 (5), 799–806. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Vestal L, Smith-Olinde L, Hicks G, Hutton T, & Hart J (2006). Efficacy of language assessment in Alzheimer’s disease: comparing in-person examination and telemedicine. Clinical Interventions in Aging, 1 (4), 467–471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Wadsworth HE, Galusha-Glasscock JM, Womack KB, Quiceno M, Weiner MF, Hynan LS, et al. (2016). Remote neuropsychological assessment in rural American Indians with and without cognitive impairment. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. WPS Publishing (2020). Statement on Tele-Assessment. https://content.wpspublish.com/Submitted-Content/pdfs/WPS%20Tele-assessment%20Statement.pdf

RESOURCES