Skip to main content
NCBI home page
Oxford University Press - PMC COVID-19 Collection logoLink to Oxford University Press - PMC COVID-19 Collection
. 2020 Nov 5:acaa100. doi: 10.1093/arclin/acaa100

Teleneuropsychology for Monolingual and Bilingual Spanish-Speaking Adults in the Time of COVID-19: Rationale, Professional Considerations, and Resources

Franchesca Arias 1,2,3,✉, Diomaris E Safi 4,5, Michelle Miranda 6, Carmen I Carrión 7, Ana L Diaz Santos 8, Victoria Armendariz 9, Irene E Jose 10, Kevin D Vuong 11,12, Paola Suarez 13,14, Adriana M Strutt 15,16,#; The STAR Consortium#
PMCID: PMC7665473  PMID: 33150414

Abstract

Objective

Neuropsychological assessments with monolingual Spanish and bilingual Spanish/English-speaking adults present unique challenges. Barriers include, but are not limited to, the paucity of test norms, uncertainty about the equivalence of translated neuropsychological tests, and limited proficiency in the provision of culturally competent services. Similar issues generalize to telephone- and video-based administration of neuropsychological tests or teleneuropsychology (TeleNP) with Hispanics/Latinos (as), and few studies have examined its feasibility and validity in this group. The sudden onset of the COVID-19 pandemic prompted neuropsychologists to identify alternative ways to provide equitable care. Clinicians providing TeleNP to this population during (and after) the pandemic must consider safety, professional factors, and systemic barriers to accessing and benefitting from virtual modalities.

Method

This clinical process manuscript describes how cross-cultural neuropsychologists across five U.S. academic institutions serving Hispanics/Latinos (as) developed TeleNP models of care during the pandemic.

Results

Workflows, test batteries, and resources for TeleNP assessment with monolingual and bilingual Spanish-speaking patients are included. Factors guiding model development and informing decisions to incorporate virtual administration of neuropsychological tests into their practice are also discussed.

Conclusions

Provision of TeleNP is a promising modality. Additional research in this area is warranted with focus on cultural and contextual factors that support or limit the use of TeleNP with this community.

Keywords: COVID-19, Cross-cultural Minority, Teleneuropsychology, Spanish-speaking Patients

Introduction

Neuropsychological assessments with monolingual and bilingual Spanish-speaking patients present unique challenges. Fundamental barriers revolve around the paucity of appropriate tests and corresponding norms that capture the heterogeneity of the cultural and educational backgrounds that characterize this population in the USA (Ardila, 2003; Judd et al., 2009). Moreover, the construct validity and equivalence of many cognitive tests have not been thoroughly examined (Pontón, 2001; Rivera Mindt, Byrd, Saez, & Manly, 2010), and most standard instruments demonstrate marginal specificity when used with linguistically and culturally diverse patients (Heaton, Taylor, & Manly, 2003). Additional challenges relate to proficiencies necessary to provide culturally competent neuropsychological services (Brickman, Cabo, & Manly, 2006; Rivera Mindt et al., 2008), and it has been noted that most clinical neuropsychologists feel poorly prepared to work with this population (Echemendia, Harris, Congett, Diaz, & Puente, 1997).

Clinicians working to provide neuropsychological services for Hispanic/Latino(a) adults during the Severe Acute Respiratory Syndrome Coronavirus 2 pandemic (COVID-19) face additional challenges. Evidence documenting the feasibility of teleneuropsychology (TeleNP) in this population is scant. In a USA-based study, Vahia and colleagues (2015) administered a neuropsychological battery including Mini-Mental State Examination, Hopkins Verbal Learning Test—Revised, Digit Span subtest of the Escala de Inteligencia de Wechsler para Adultos (EIWA-III), Letter and Category Fluency, Clock Drawing, Brief Visuospatial Memory Test—Revised, and Pontón-Satz Boston Naming Test to Spanish-speaking adults over the age of 65. Overall, scores from the in-person evaluation did not differ significantly from those obtained via video TeleNP (though their sample was small—n = 22). Outside of the USA, Franco-Martin, Bernardo-Ramos, & Soto-Perez (2012) administered the Screen for Cognitive Impairment in Psychiatry (SCIP-S) to 30 Spanish-speaking adult patients in-person and via videoconferencing and found that remote neuropsychological assessments yielded similar results.

More generally, research on TeleNP supports the feasibility of testing via videoconference (Cullum, Hynan, Grosch, Parikh, & Weiner, 2014; Hildebrand, Chow, Williams, Nelson, & Wass, 2004; Jacobsen, Sprenger, Andersson, & Krogstad, 2003; Kirkwood, Peck, & Bennie, 2000) and telephone (Brandt, Spencer, & Folstein, 1988; Castanho et al., 2014; de Jager, Budge, & Clarke, 2003; Duff, Tometich, & Dennett, 2015; Lachman, Agrigoroaei, Tun, & Weaver, 2014; Mitsis et al., 2010). In non-Hispanic white persons, video TeleNP has been shown to effectively evaluate global cognitive functioning and detect cognitive impairment (Brearly et al., 2017; Harrell, Wilkins, Connor, & Chodosh, 2014; Loh, Donaldson, Flicker, Maher, & Goldswain, 2007; Wadsworth et al., 2018). Although less systematically examined, telephone-based TeleNP has been useful in detecting cognitive impairment and monitoring cognition over time (Bunker et al., 2017; Gurnani, John, & Gavett, 2015; Manly et al., 2011).

Shelter-in-place orders at the outset of the pandemic prevented (or strongly discouraged) in-person medical visits. Even when nonessential outpatient medical services are allowed, Hispanic/Latino(a) patients may encounter additional risks. Ethno-racially underrepresented groups have higher rates of infection and death from COVID-19 than their white counterparts (Tai, Shah, Doubeni, Sia, & Wieland, 2020). Further, older Hispanic/Latino(a) adults have higher age-specific death rates from COVID-19 relative to their non-Hispanic white peers (Sáenz & Garcia, 2020), partly due to social factors (Clark, Fredericks, Woc-Colburn, Bottazzi, & Weatherhead, 2020; Macias Gil et al., 2020) including poverty (Fontenot, Semega, & Kollar, 2018), overcrowding, low paying positions, working environments with inability to social distance, non or underinsured (Berchick, Hood, & Barnett, 2018; Cabassa, Zayas, & Hansen, 2006; Hooper, Nápoles, & Pérez-Stable, 2020), and reliance on public transit (Blumenberg & Evans, 2010; Smart, 2015). Even if TeleNP services are offered, Hispanic/Latino(a) persons lag behind their non-Hispanic white counterparts in terms of access to high-speed Internet (Brown, López, & Lopez, 2016) and to personal computers (Lopez, Gonzalez-Barrera, & Patten, 2013). Further, adults who identify as Hispanic/Latino(a) persons are more likely to decline telehealth due to privacy concerns (George, Hamilton, & Baker, 2012).

Given the factors mentioned earlier, Neuropsychology with adult Spanish and Spanish–English-speaking adults demands an approach that supports patient safety, mitigates health disparities, promotes equity (Baquet, Carter-Pokras, & Bergen-Seltzer, 2004), and upholds professional standards. With this in mind, and inspired by the “socially responsible neuropsychology” model (Suarez, Casas, Lechuga, & Cagigas, 2016), the Spanish TeleNP Assessment and Research (STAR) Consortium (2020), a grassroots movement that emerged in response to the rapidly changing provision of care under COVID-19, was developed to connect and support neuropsychologists serving Hispanic/Latino(a) patients. In this manuscript, we set out to: 1) describe ethical and professional factors influencing how culturally competent care was provided by Spanish-speaking neuropsychologists in five U.S. academic institutions during the COVID-19 pandemic from March to May 2020; and 2) present models of practice and share resources. Workflows (Figs. 1 to 5), testing batteries (Table 3), and a sample assessment for TeleNP candidacy screener (Fig. 6) are provided (resources such as consent/assent forms may be available upon request). Our goal is to offer a clinical process blueprint that can be of assistance to clinicians working with monolingual and bilingual Spanish-speaking persons in the USA to develop TeleNP protocols tailored to the needs of their communities during or beyond the COVID-19 pandemic.

Fig. 1.

Fig. 1

Open in a new tab

University of California at Los Angeles (UCLA)’s Spanish TeleNP workflow. Spanish Teleneuropsychology (TeleNP) workflow at the University of California at Los Angeles (UCLA)’s Hispanic Neuropsychiatric Center Excellence (HCNE).

Fig. 5.

Fig. 5

Open in a new tab

University of Utah’s Spanish TeleNP workflow. Spanish Teleneuropsycology (TeleNP) workflow in the Department of Neurology at the University of Utah.

Table 3.

Spanish TeleNP instruments used at each site

Domains/Instruments UCLA BIDMC Yale BCM Utah
Informant questionnaires
Activities of Daily Living Questionnaire—Spanish version (ADLQ-SV)1 X X X X*
Neuropsychiatric Inventory Questionnaire (NPI-Q)2 X X*
WHO Disability Assessment Schedule 2.0 (WHODAS 2.0) in Spanish3 X
Patient questionnaires
Generalized Anxiety Disorder 7-Item Scale (GAD-7)4 X X*
Symptom Checklist 90-R, Spanish version (SCL-90-R)5 X
Geriatric Depression Scale (Short Form) (GDS-SF)6 X
Geriatric Anxiety Inventory (GAI)7 X
Beck Anxiety Inventory (BAI)8 X X X
Geriatric Depression Inventory (GDS)9 X
Beck Depression Inventory X
The Short Acculturation Scale for Hispanics (SASH)10 X X
Patient Health Questionnaire (PHQ-9)11 X X*
Pittsburgh Sleep Quality Index-Spanish (PSQI) X
Effort
Rey-15 Item12 X X^
Test of Memory Malingering (TOMM)13 X X^
Wechsler Adult Intelligence Scale, Fourth Edition (WAIS-IV)14 Wechsler Adult Intelligence Scale, Third Edition (EIWA-III) Reliable Digit Span15 X X X
Screeners
Montreal Cognitive Assessment (MoCA)/Montreal Cognitive Assessment (MoCA) Blind16 X X X X* X
Telephone Interview for Cognitive Status modified (STICS-m) Spanish17 X*
Premorbid/Academic Performance
Domain/Test UCLA BIDMC Yale BCM Utah
Batería IV/Bateria III Woodcock-Muñoz – Letter-Word Identification18,19 X X
Word Accentuation Test-Chicago (WAT-C)20 X*
Intelligence
WAIS-IV/EIWA-III Information14,15 X X*
WAIS-IV/EIWA-III Vocabulary14,15 X
Attention/Concentration
Batería Neuropsicológica en Espanol (BNE) Digit Span21 X
BNE Stroop Test Color/Reading21 X
The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Digit Span22,23 X
Verbal Series Attention Test (VSAT) X*
WAIS-IV/EIWA-III Digit Span14,15 X X X*
Language proficiency
Bilingual Verbal Ability Tests (BVAT)24 X
Woodcock-Johnson, Fourth Edition (WJ-IV) Tests of Oral Language25 X
Language
Semantic Fluency- Animals & Fruits26–31 X X X X*
Boston Naming Test (BNT)32,32 X X
BNE – Letter Fluency PMR/Control Oral Word Association Test (COWA)21 X X X X*
RBANS Semantic Fluency22,23 X
RBANS Naming22,23 X
Multilingual Aphasia Examination, Third Edition (MAE) – PTM34 X
Intellectual functioning
Raven Progressive Matrices (RPM)33 X
Executive functioning
Independent Living Scales (ILS): Health and Safety34 X
Neuropsi Attention & Memory (A&M) Category Formation26–28 X X
Domain/Test UCLA BIDMC Yale BCM Utah
Neuropsi A&M Motor sequencing 26–28 X
Stroop Color Word Test (SCWT) Inhibition21 X
WAIS-IV/EIWA-III Similarities14,15 X X X* X
Wisconsin Card Sorting Test (WCST)-12821 X
Verbal learning and memory
BNE—SVLT/Neuropsi A&M List Learning21 X
BNE—Stories/Neuropsi A&M Stories21 X
Hopkins Verbal Learning Test (HVLT)36 X X X^*
Neuropsi A&M List & Story26–28 X X
Neuropsi Breve List & Story37 X
RBANS Story Memory22,23 X X
RBANS List Learning22,23 X
Wechsler Memory Scale, Fourth Edition (WMS-IV) Logical Memory I, II, & Recognition38 X
WHO-UCLA Auditory Verbal Learning Test (AVLT)33 X
Brief Visual Memory Test-Revised (BVMT-R)39,40 X X
Neuropsychological Assessment Battery (NAB) Memory Module- Shape learning41 X
Neuropsi A&M Face Memory 27,28 X
Neuropsi Rey Complex Figure Test (RCFT) Copy/Semicomplex Copy37 X
RBANS Figure Recall22,22 X
Visuospatial
Clock Command & Copy X
The Verbal Clock Test42 X X
Hooper Visual Organization (HVOT) Test43 X
Neuropsi RCFT Copy/Semicomplex Copy37 X
WAIS-IV Visual Puzzles14 X^
RBANS Figure Copy22,23
RBANS Line Orientation22,23 X
Open in a new tab

Note. UCLA = University of California in Los Angeles; Protocols were administered via video TeleNP. BIDMC = Beth Israel Deaconess Medical Center; TeleNP = Teleneuropsychology; Protocols were administered via video or telephone-based TeleNP. Yale = Yale School of Medicine; WHO UCLA = World Health Organization University of California, Los Angeles; Protocols were administered via video TeleNP. BCM = Baylor College of Medicine; * Telephone assessment measures; ^ Optional. Utah = University of Utah; Protocols were administered via video TeleNP; UCLA alternated between WAIS-IV and EIWA-III.

1  Gleichgerrcht, Camino, Roca, Torralva, & Manes, 2009; 2  Boada, Cejudo, Tárraga, López, & Kaufer, 2002; 3  Üstün, et al., 2010; 4  Garcia-Campayo, et al., 2010; 5  Derogatis, 2002; 6  Fernández-San Martín, et al., 2002; 7  Márquez-González, Losada, Fernández-Fernández, & Pachana, 2012; 8  Magan, Sanz, & Garcia-Vera, 2008; 9  Lucas-Carrasco, 2012; 10  Ellison, Jandorf, & Duhamel, 2011; 11  Kroenke, Spitzer, & Williams, 2001; 12  Boone, Salazar, Lu, Warner-Chacon, & Razani, 2002; 13  Tombaugh, 1996; 14  Wechsler, Meng, Martínez, & Zhu, 2014; 15  Pons et al., 2008; 16  Nasreddine, et al., 2005; 17  Muñoz-García, et al., 2020; 18  Muñoz-Sandoval, Woodcock, McGrew, Mather, & Ardoino, 2009; 19  Woodcock et al., 2019; 20  Krueger, Lam, & Wilson, 2006; 21  Artiola i Fortuny, Hermosillo, Heaton, & Pardee, 1999; 22  Hall et al., 2018; 23  Muntal, et al., 2017; 24  Munoz-Sandoval, Cummins, Alvaredo, & Ruef, 1998; 25  Schrank & Wendling, 2018; 26  Ostrosky, Gomez, Matute, Rosselli, Ardila, & Pineda, 2012; 27  Ostrosky-Solis et al., 2007; 28  Ostrosky-Sollis et al., 2003; 29  Olabarrieta-Landa et al., 2015a; 30  Olabarrienta-Landa et al., 2015b; 31  Benito-Cuadrado, Esteba-Castillo, Böhm, Cejudo-Bolívar, & Peña-Casanova, 2002; 32  Rivera, et al., 2019; 33  Pontón et al., 1996; 34  Rey & Benton, 1992; 35  Loeb, 1996; 36  Arango-Lasprilla et al., 2015; 37  Ostrosky-Solis, Ardila, & Rosselli, 1999; 38  Wechsler, 2008; 39  Cherner et al., 2007; 40  Benedict, Schretlen, Groninger, Dobraski, & Shpritz, 1996; 41  White & Stern, 2003; 42  Cercy, 2012; 43  Boyd, 1981

Fig. 6.

Fig. 6

Open in a new tab

BCM Cerebro’s assessment for TeleNP candidacy.

Methods

Patient Classification

In an attempt to correctly classify the patients who engaged in the initial TeleNP delivery of services, the term Hispanic/Latino(a) was agreed upon by the authors across sites. La Real Academia Española (2001) defines, “Latino” as persons from Latin American or Hispanic origin who reside in the USA. To be consistent with the literature cited in this manuscript (United States Census Bureau, 2019a, 2019b, 2019c, 2019d, 2019e, 2019f) and to promote inclusivity, the term Hispanic/Latino(a) is used throughout this manuscript. Although we recognize that our choice does not capture nonbinary gender identities, Hispanic/Latino(a) is most consistent with our patients’ self-reports and recent national data on rates of familiarity with, and use of, the Latinx term (Noe-Bustamante, Mora, & Lopez, 2020).

Settings

Eligibility

Clinicians from across the country, who were providing services to Hispanic/Latino(a) patients, and who expressed interest in TeleNP via national listservs and public forum were approached. Providers from sites that were willing to share resources and attend weekly meetings were invited through personal and/or mutual contacts to join the STAR Consortium. Of the 26 sites approached, 14 joined the STAR Consortium and five of these sites scheduled to engage in-home TeleNP visits with adult Spanish-speaking patients by April 15, 2020. These five sites presented the Taquitos de Sesos April 2020 webinar (STAR Consortium, 2020) and expanded their presentation for this manuscript. Only one site (Cerebro, Baylor College of Medicine) provided both video conference and telephone virtual visits, whereas all other sites provided only video conference visits immediately following the shelter-in-place orders. Table 1 details demographic characteristics of Hispanic/Latino(a) persons residing in the counties represented in this paper (United States Census Bureau, 2019a, 2019b, 2019c, 2019d, 2019e). For comparison, information about patients evaluated at each site prior to COVID-19 is also included in Table 2. These numbers document shifts in clinical trends that emerged in response to the COVID-19 pandemic from March to May 2020. Our sample sizes are low and reflect the nuanced process involved in identifying patients appropriate for TeleNP modality. Since the development of this manuscript, sites have continued practicing through TeleNP, via in-person while incorporating protective equipment or through hybrid models and have reached close to pre-COVID-19 capacity. Sites included the following:

Table 1.

Demographic characteristics of monolingual and bilingual Spanish-speaking persons residing in select U.S. counties

Population County
Los Angeles County (UCLA) Suffolk County (BIDMC) New Haven County (Yale) Harris County (BCM) Salt Lake County (Utah)
Total 10,039,107 803,907 854,757 4,713,325 1,160,437
Hispanic persons, % 49 23 18.6 43 19
Spanish speakers, % 38 19 25 35 14
Median household income $64,251 $64,582 $41,142 $60,146 $71,230
Living in poverty, % 14 18 26 17 9
Open in a new tab

Note. UCLA = University of California in Los Angeles; BIDMC = Beth Israel Deaconess Medical Center; Yale = Yale School of Medicine; BCM = Baylor College of Medicine; Utah = University of Utah

Table 2.

Basic characteristics of patients seen across five academic institutions in the USA

Patient characteristics
Site and timeframe Patients seen in-person UCLA January 2019–January 2020 BIDMC January 2020–March 2020 Yale January 2020–March 2020 BCM January 2019–January 2020 Utah January 2020–May 2020
Total 183 11 6 58 10
Age, mean (SD) 49.3 (±25.6) 62.5 (±16.9) 61.17 (±16.29) 68.05 (±13.47) 62 (±14.9)
Education in years, range 0–20 2–14 8–20 0–20 0–18
Women, % 74 57 17 52 44
Monolingual Spanish speaking, % 26 81 67 100 87
Site and timeframe Patients seen via TeleNP UCLA March 2020–May 2020 BIDMC April 2020–May 2020 Yale April 2020–May 2020 BCM March 2020–May 2020 Utah April 2020–May 2020
Total 19 4 1 6 3
Number of referrals N/A 13 6 13 15
Age, mean (SD) 40.7 (±23.6) 60.8 (±12.2) 70 70.5 (±11.47) 41 (±7.5)
Education in years, range 0–16 8–16 0 2–20 9–12
Women, % 58 100 0 100 33
Monolingual Spanish speaking, % 29 75 100 80 100
Open in a new tab

Note. UCLA = University of California in Los Angeles; BIDMC = Beth Israel Deaconess Medical Center; Yale = Yale School of Medicine; BCM = Baylor College of Medicine; Utah = University of Utah; SD = Standard deviation; TeleNP = Teleneuropsychology.

UCLA Hispanic Neuropsychiatric Center of Excellence (HNCE), Los Angeles, CA

The bilingual/bicultural staff includes four neuropsychologists, four fellows, three social workers, four practicum students, and three administrative staff. HNCE is able to serve socioeconomically diverse patients, receiving referrals from UCLA specialties, non-UCLA providers, and the LA County Department of Mental Health.

Beth Israel Deaconess Medical Center (BIDMC), Boston, MA

The Clinical Neuropsychology team includes one bilingual provider. There are also six monolingual, English-speaking providers: two fellows, three practicum students, and three psychometrists. BIDMC provides outpatient testing to Spanish speakers via the bilingual provider, and to linguistically diverse patients via interpreting services. Referrals are accepted from Neurology, other BIDMC specialty clinics, and community organizations.

Yale School of Medicine, New Haven, CT

The Neuropsychology Division comprises one bilingual neuropsychologist, four other neuropsychologists, and a bilingual referral specialist for scheduling. The clinic receives referrals from within the Yale system as well as community organizations.

Baylor College of Medicine (BCM) Cerebro, Houston, TX

Part of the Neuropsychology section in the Department of Neurology, BCM Cerebro is the only site led by a bilingual/bicultural board-certified neuropsychologist. Other staff includes a neuropsychology fellow and a psychometrist (all bilingual/bicultural). Although a scheduling service is available, workflow for TeleNP services was managed by faculty, fellows, and psychometrists. BCM Cerebro offers outpatient neuropsychological services in Spanish (and other languages via interpreting services). Referrals are accepted from within the BCM system as well as community clinics and medical centers, nationally and internationally.

University of Utah, Salt Lake City, UT

The Division of Neuropsychology includes one bilingual neuropsychologist and seven monolingual English-speaking staff: three neuropsychologists, two of whom are board-certified, three psychometrists, and one scheduler. Services are offered to the Hispanic/Latino(a) population by the bilingual provider (or via interpreting services for other languages). Referrals are accepted from within the university system or from in-state community organizations.

Relevant Considerations

Due to different regional approaches to the pandemic, as well as unique clinical needs and models of practice, we present each site’s approach to telephone or video-based TeleNP separately. Nevertheless, similar ethical and professional factors informed each site’s decisions and protocols.

Ethical

Ethical and professional considerations involved in the practice of TeleNP with linguistically and culturally diverse groups extend beyond those relevant in traditional face-to-face assessments. In each site, clinical decisions were informed by a comprehensive assessment of the risk associated with delaying the neuropsychological evaluation, the need for continuity of care, and a patient’s individual characteristics and medical history (Inter Organizational Practice Committee (IOPC) 2020a; Principal A, Beneficence and Non-Maleficence; Ethical Code of the (American Psychological Association (APA) 2002, 2016). These decisions were discussed at the institution level as well as between institutions, via weekly STAR Consortium meetings.

In terms of Principle B (Fidelity and Responsibility; APA, 2002), TeleNP protocols included instruments that were available via open access. Additionally, copyrighted instruments available through the site were utilized in each assessment following COVID-19 and general guidelines developed by test publishers. Test security was also discussed prior to all telephone and video-based TeleNP visits, and patients were discouraged from recording the evaluation or writing details shared during the session. When intake forms or scales were mailed to patients, they were provided with stamped envelopes and were asked to return all materials used during the session.

Principle C (Integrity; APA, 2002) guided our approach to obtaining assent and/or consent to proceed with a TeleNP visit. Patients learned about the nature of the evaluation and were presented with available alternatives during scheduling, which were conducted via telephone across all sites. Assent/consent was again discussed/obtained prior to the beginning of their testing session. Appointments were scheduled days in advance via telephone to give patients time to weigh their options and decide to proceed with telephone or video-based TeleNP.

The decision to incorporate TeleNP into practice was driven by Principle D (Justice) of the APA Ethics Code (APA, 2002) and our commitment to ensure that Hispanic/Latino(a) patients continued to benefit from neuropsychological services provided in a culturally informed and safe manner. Prior to COVID-19, licensed providers leading TeleNP efforts had several years of experience conducting face-to-face assessments with Hispanic/Latino(a) patients and many were directly involved in training emerging Spanish-speaking clinicians (CC, AMS, DS, and PS). Thus, all providers are well-versed in conducting culturally responsive evaluations. Overall, this group was uniquely equipped to understand the cultural and socioeconomic challenges of this patient population.

Professional

Authors attended numerous webinars, reviewed available resources, and consulted with IOPC workgroup members prior to developing their TeleNP programs. Public health updates and ethical recommendations were carefully examined when developing risk-stratification models (APA, 2013; Grosch, Gottlieb, & Cullum, 2011; Pitts & Naumenko, 2016).

Financial factors were also considered. At the time of this writing (April to August 2020), most insurance companies were reimbursing for TeleNP at rates similar to those offered for traditional neuropsychological services. Each site developed TeleNP programs keeping in mind the risks of delaying neuropsychological evaluations, provider and patient safety, as well as the potential for COVID-19 surges and the need to apply these modalities in the future (IOPC, 2020b). Models were also developed keeping in mind that telemedicine has expanded services to patients facing barriers to in-person evaluations across multiple specialties such as telestroke (Demaerschalk, 2010), telecardiology (Backman, Bendel, & Rakhit, 2010), and teleradiology (Bashshur, Krupinski, Thrall, & Bashshur, 2016).

Process

Stepwise triaging was implemented to assess the appropriateness of TeleNP for each patient. Initially, providers (and trainees when available) reviewed patients’ medical history and identified preexisting conditions that may preclude them from benefiting from TeleNP (e.g., sensory limitations, lack of access to a space where confidential information can be discussed). Next, we assessed patients’ attitudes about engaging in an evaluation via telephone or video, familiarity with relevant technology, and access to internet, video/telephone capabilities, and distraction-free environments. Finally, the risk/benefit ratio associated with each option for providing care (TeleNP, modified in-person evaluation, no evaluation) was carefully considered in order to ensure the patient’s safety, continuity of care, and validity of the evaluation. For example, for patients with sensory limitations, an in-person assessment was deemed most appropriate. In those cases, psychoeducation was provided to families and the patients were placed on a priority list. These results were communicated to patients and all protocols incorporated a detailed consent/assent process including discussion regarding limitations associated with TeleNP and alternatives to TeleNP (e.g., cancellation, postponement, complete interview only, or engage in a brief cognitive screener). Alternatives available to patients varied by site.

Efforts were made to involve informants in all remote TeleNP assessments, but success varied depending on the availability of an informant, a patient’s desire to involve an informant in the visit, and a patient’s ability to successfully navigate technology. Sites implemented abbreviated protocols that incorporated cognitive screeners, mood scales, and instruments to assess attention/concentration, language, learning and memory, and executive functioning, as processing speed and visuoconstructive skills proved difficult to assess. Consistent with the patient’s needs, evaluations were either performed in one session, per TeleNP literature (Cullum et al., 2014; Harrell et al., 2014), or extended across two visits: consent and clinical interview (visit 1) and cognitive assessment (visit 2) to diminish fatigue. Implementing a cultural perspective, the patient’s resources/access and medical and sociodemographic factors informed the decision for single versus multiple visits.

Statistical Analysis

Because this is a clinical process paper designed to illustrate models developed by different providers across the USA, descriptive statistics were used to characterize the patient population evaluated at each site. Given the nature of this manuscript, and in light of small sample sizes (n < 15) per site, no inferential statistics were calculated.

Results

Prior to the COVID-19 pandemic, sites provided in-person neuropsychological assessments to Hispanic/Latino(a) patients. To illustrate, Table 2 provides basic characteristics of the patients evaluated (pre-pandemic and early-pandemic) at each site. This is a clinical process paper that characterizes the implementation of TeleNP models across five U.S. institutions between March and May 2020. States and institutions differed in their approach to COVID-19, as different regions were differentially affected by COVID-19-related morbidity and mortality. Although our sample sizes are small, our data illustrate shifts in clinical practice with monolingual and bilingual Spanish-speaking patients that occurred in the early stages of the pandemic.

Spanish TeleNP at UCLA HNCE

Neuropsychology faculty met to weigh TeleNP versus deferring visits. They leaned toward proceeding with TeleNP evaluations, noting that not doing so—particularly for pre-neurosurgical evaluations where neuropsychology is part of standard of care—would jeopardize continuity of care and potentially alter postsurgical prognosis. In light of an anti-TeleNP position statement of the International League Against Epilepsy (2020), the neuropsychology team approached the ethics committee of the Ronald Reagan UCLA Medical Center, which deemed ethical the incorporation of TeleNP into all presurgical evaluations (i.e., epilepsy, deep brain stimulation, and patients with brain tumors). Table 3 lists the UCLA protocol, and Fig. 1 illustrates the UCLA TeleNP workflow. UCLA expeditiously provided infrastructure (including an HIPAA-compliant video platform). Daily/weekly updates on reimbursement and documentation were held in the weeks following TeleNP implementation.

Spanish TeleNP at BIDMC

BIDMC’s Spanish TeleNP protocol—developed in collaboration with internal/external colleagues—assesses global cognition and various cognitive domains (Table 3). Most tests in BIDMC’s Spanish TeleNP protocol can be administered via video or telephone. The most common referral reason was memory loss (75%). Figure 2 illustrates the workflow. At BIDMC, barriers to the implementation of TeleNP for monolingual and bilingual Spanish-speaking patients included rapid changes in state/local regulations. Clinician effort was also a concern, and the bilingual provider was responsible for identifying a suitable battery, developing/refining screening forms to introduce the service, and confirming appropriateness of referral and patient’s access to, and level of comfort with, technology. Overall, significant gains were made in a short period of time, and BIDMC piloted numerous HIPAA-compliant platforms, purchased equipment, and provided updates on pandemic-related developments to support a rapid transition to TeleNP. Clinical neuropsychologists met weekly to discuss cases and challenges.

Fig. 2.

Fig. 2

Open in a new tab

Beth Israel Deaconess Medical Center (BIDMC)’s Spanish TeleNP workflow. Spanish Teleneuropsychology (TeleNP) workflow at the Cognitive Neurology Unit (CNU) at Beth Israel Deaconess Medical Center (BIDMC).

Spanish TeleNP at Yale

Neuropsychologists in the Yale Department of Neurology developed a process to determine when TeleNP would be appropriate (screening questions to assess access to and familiarity with digital devices). Virtual evaluations were scheduled if TeleNP could address the referral question or if delays in services affected continuity of care. Given the paucity of norms available for individuals of Caribbean descent (an overrepresented group in the bilingual service line), test selection was based on whether age-appropriate norms were available for a demographically representative normative sample (Table 3). Because not all patients may have access to computer/webcam/internet, the Yale protocol is suitable for video or telephone administration. Figure 3 illustrates the workflow. Barriers to the implementation of TeleNP at Yale revolved around clinician effort and state-level regulations that delayed the implementation of remote models. Yale Medicine provided HIPAA-complaint platforms, regular training for these platforms, and updates regarding billing and documentation changes.

Fig. 3.

Fig. 3

Open in a new tab

Yale School of Medicine, Division of Neuropsychology’s Spanish TeleNP workflow. Spanish Teleneuropsycology (TeleNP) workflow in the Neuropsycology Section of Neurology at the Yale School of Medicine.

Spanish TeleNP at BCM Cerebro

The BCM neuropsychology faculty held meetings to develop a workflow ensuring continuity of care (Fig. 4). The BCM Cerebro Spanish TeleNP battery is a modified version of the in-clinic flexible battery administered to Spanish-speaking adults referred for memory loss (Table 3). Two batteries were created (telephone and video). Although planning for remote service provision, BCM provided institution-wide guidelines, departmental/section meetings, assistance with state regulations and billing, as well as an HIPAA-compliant secure platform. In addition, regular consults and assistance from the departmental systems administrator were conducted to address issues regarding digital devices, webcams, and platform security.

Fig. 4.

Fig. 4

Open in a new tab

Baylor College of Medicine (BCM)’s Spanish TeleNP workflow. Spanish Teleneuropsychology (TeleNP) workflow at Baylor College of Medicine (BCM).

Spanish TeleNP at the University of Utah

Neuropsychologists at the University of Utah worked collaboratively to delicately examine ethical and professional considerations related to TeleNP, as some clinicians had reservations about the validity of in-home evaluations. Ultimately, a TeleNP screening service that follows APA/IOPC recommendations (APA, 2013; IOPC, 2020b) appeared to be an ethical alternative to not providing or delaying service. Figure 5 illustrates the workflow. The Spanish battery (Table 3) was designed to maintain uniformity with other providers in the Division of Neuropsychology, in an effort to offer comparable services across providers. In addition, the battery incorporated tests that were not administered during in-person evaluations (to avoid practice effects), as patients may be asked to return to complete an in-person assessment upon re-opening of the clinic. For the RBANS, different norms were used depending on the characteristics of the patient (norms from Spain for patients from South American countries; norms from Mexico/USA for patients from Mexico, Central America, and the Caribbean; Hall et al., 2018; Muntal, et al., 2017).

The University of Utah School of Medicine provided an HIPAA-compliant video conference platform immediately after outpatient clinics were closed to in-person services. The Neurology Department provided clinician training and financial assistance to support future TeleNP within the clinic.

The small sample in this manuscript is a reflection of the multi-system changes required to develop and implement a TeleNP service in the uncertainty of a global pandemic. Clinical experiences presented are included to elucidate the limitations and challenges faced by these providers, their patients and institutions. Workflows reflect not only the decision-making process of each institution, but also highlight the work that was needed to begin TeleNP (e.g., scheduling, determining candidacy for TeleNP, consents, electronic questionnaires, etc.).

Discussion

All five sites discussed were motivated by a desire to bridge (rather than widen) pre-pandemic gaps in access to services for monolingual and bilingual Spanish-speaking adult patients. Although a standard workflow and core battery was considered by the STAR Consortium for future research and dissemination of a structured clinical protocol, each site included in this manuscript independently implemented a systematic approach to establishing a TeleNP service. This occurred secondary to a myriad of factors including differences across institutional telemedicine guidelines, billing requirements/limitations, institutional resources (e.g., webcams, laptops, secure platform), patients’ medical/neuropsychological needs, and patients’ access to digital resources (e.g., devices, internet, platform familiarity). Although tailoring of the Tele-NP approach may be considered a limitation, it is consistent with the overarching objective to provide readers with a clinical process blueprint that can be customized to meet the needs of the community and abide by institutional standards.

Institutional resources available to providers are particularly important to incorporating TeleNP into practice. Across all sites, information about steps to billing, rates of reimbursement, and codes appropriate for virtual modalities came directly from the respective institutions. Providers at BIDMC and the University of Utah gained temporary licenses to surrounding states, which allowed them to provide services to patients residing in selected regions of the country. UCLA HNCE was able to establish a laptop/tablet/hotspot loaning program for patients, which approximately 5–10% of their patients since March have taken advantage of. Further, UCLA HNCE and BCM Cerebro have teams of bilingual providers/staff at different training levels available to assist patients prior to the visit. Notably, providers who do not work within well-established bilingual/bicultural programs have less access to resources (e.g., psychometrists, trainees) and generally evaluate a lower volume of patients in their practices even prior to COVID-19. Thus, they may take longer to implement changes, such as adopting a culturally informed TeleNP model, and differences in numbers across sites reflect this process. At Yale, BIDMC, and Utah, bilingual practitioners were responsible for troubleshooting technological challenges with patients, requiring additional practitioner time and reducing the number of patients that could be evaluated.

During the COVID-19 pandemic, efforts to create TeleNP protocols appropriate for monolingual and bilingual Spanish-speaking patients accelerated. Prior work in this area focused on expanding access to neuropsychological services to patients living in rural areas who did not have the means to travel to in-person evaluations (Franco-Martin, Bernardo-Ramos, & Soto-Perez, 2012). The advent of COVID-19 and associated shelter-in-place orders presented with the unique challenge of providing neuropsychological services, which is part of the medical planning protocol across different illness and associated treatments. To avoid delaying care during the pandemic, it has become imperative to have well-thought-out TeleNP protocols and workflows.

Ethical, practical, and professional consideration is relevant to TeleNP practice. Additional factors come into play when providing TeleNP to linguistically and socioeconomically diverse groups, as there are preexisting limitations that also influence the validity of our instrument in these groups (Rivera Mindt et al., 2010). Similar to in-person evaluations, providers interested in administering TeleNP to diverse patients must interpret cognitive data in the context of cultural factors and rely heavily on collateral information, behavioral observations, and other qualitative data to make diagnostic decisions. Neuropsychologists who specialize in cross-cultural work have extensive experience in this area (Puente, Perez-Garcia, Lopez, Hidalgo-Ruzzante, & Fasfous, 2013; Uzzell, Pontón, & Ardila, 2013). They are uniquely equipped for this challenge as the professional experience relevant to in-person administration of neuropsychological tests with diverse groups is particularly useful in TeleNP modalities.

This study was not designed or powered to conduct analyses on the benefits and challenges of TeleNP with monolingual and bilingual Spanish-speaking patients. Anecdotal reports from patients, caregivers, and colleagues suggest that TeleNP helped to inform immigration procedures, guided clinical decision-making in presurgical and transplant cases, provided services to patients with limited transportation, and informed recommendations for patients where safety was a concern. Barriers to TeleNP included lack of resources to the technology necessary to support this modality, limited availability of space in the home to conduct confidential evaluations, and low computer literacy, which extended assessment times and reduced patient volumes. Additional challenges to TeleNP were professional in nature. Providers were unable to use certain instruments due to copyright policies or barriers to administration.

Despite the challenges highlighted throughout this paper, TeleNP is a promising modality that is likely here to stay post-COVID-19. For working families, TeleNP offers a unique opportunity to incorporate the patient and all relevant caregivers for clinical interviews, testing, feedback sessions (all sites) and therapy (Utah) within the comfort of their home and/or community. Also, TeleNP provides an opportunity to increase access to monolingual Spanish and bilingual patients who sometimes have to travel a long distance to receive language-congruent assessments. However, aspects of TeleNP pose unique barriers/challenges for the Hispanic/Latino(a) population who do not always have the required equipment (i.e., computer, smart phone, internet services, etc.), space (i.e., a quiet space where confidential information can be discussed), or technical knowledge necessary to conduct these evaluations. Continued investment in providers’ training needs, permanent reimbursement guidelines, and technical requirements are needed to support the effective use of TeleNP.

Limitations

Concerns regarding TeleNP data quality existed prior to the pandemic (Cullum et al., 2014). Overestimates of cognitive capability can occur if patients are receiving assistance from someone in the room, recording prompts/questions, or accessing online resources. Among other things, underestimates can occur if slow internet bandwidth affects processing speed tasks, low screen resolution affects appropriate color differentiation, unreliable audio quality leads to missed information, and interruptions. Our paper described factors considered by five USA-based sites when developing TeleNP programs for linguistically diverse patients. Future studies should expand on patient satisfaction with TeleNP as well as barriers to this modality with the Hispanic/Latino(a) community. At all sites, providers carefully discussed ground rules with patients/caregivers to increase cooperation. Following a clinical interview, the provider at the University of Utah determined that the patient required a mental health intervention prior to proceeding with an assessment. At BIDMC, a cognitive screening was discontinued due to poor internet connectivity.

Other limitations included a lack of studies establishing acceptable diagnostic accuracy guided by appropriate norms via TeleNP for Hispanic/Latino(a) patients, inability to conduct widely used tests virtually (e.g., Block Designs, Tower Test, Symbol Search), and an absence of preexisting/permanent TeleNP reimbursement protocols. Reduced access to technology and unfamiliarity with virtual platforms may also serve as barriers. In this study, cross-cultural neuropsychologists providing services across five U.S. institutions developed and implemented culturally informed TeleNP protocols. Our findings document shifts in clinical trends in the early months of the COVID-19 pandemic, March to May 2020. Future studies should expand on these findings to determine whether certain patient-specific characteristic render some patients more likely to benefit from TeleNP. Additionally, future work should systematically examine whether clinician characteristics, resources available at a site, and the presence of an informant/caregiver propel the implementation of TeleNP services.

Conclusions

Telemedicine has improved the provision of healthcare in underserved areas through services such as telestroke (Demaerschalk, 2010), telecardiology (Backman et al., 2010), teledermatology (Lamel et al., 2012), telepediatrics (Lopez-Magallon et al., 2015), teleradiology (Bashshur et al., 2016), and telepsychiatry (Chakrabarti, 2015). For many, the impetus to incorporate TeleNP was born out of the necessity to continue to serve patients in the context of the COVID-19 pandemic. However, TeleNP has the potential to expand access to testing services for patients who are homebound, immunocompromised, have limited access to transportation, and/or live in remote areas. This is particularly true for linguistically diverse patients who may find themselves crossing state lines to receive language-concordant assessments. Neuropsychologists with experience providing culturally informed services to linguistically diverse patients are uniquely equipped to develop and implement TeleNP protocols for this population because they have experience integrating clinical observations and cultural factors to inform clinical care. Future studies should systematically document the validity and feasibility of TeleNP and determine whether Hispanic/Latino(a) individuals are likely to benefit from this method of service delivery. TeleNP is a promising modality for the assessment of monolingual and bilingual Spanish-speaking adult patients. Additional research in this area is warranted. Efforts to standardize TeleNP practices may have long-term implications in the field.

Declarations of Interest

None. All authors have no competing interests to declare.

Acknowledgments

We thank our respective institutions and departments for resources provided, our colleagues who participated alongside in the process assisting us in providing continuity of care to our underserved Hispanic/Latino(a) community, and our patients for their flexibility and trust in the process.

Contributor Information

Franchesca Arias, Hinda & Arthur Marcus Institute for Aging Research, The Aging Brain Center, Hebrew SeniorLife, Boston, MA 02131, USA; Department of Cognitive Neurology, Beth Israel Deaconess Medical Center, Boston, MA 02215, USA; Harvard Medical School, Boston, MA 02115, USA.

Diomaris E Safi, Department of Psychiatry and Behavioral Sciences, UCLA, Los Angeles, CA 90095, USA; UCLA Hispanic Neuropsychiatric Center of Excellence, Los Angeles, CA 90095, USA.

Michelle Miranda, Department of Cognitive Neurology, University of Utah, Salt Lake City, UT 84112, USA.

Carmen I Carrión, Department of Neurology, Yale School of Medicine, New Haven, CT 06519, USA.

Ana L Diaz Santos, Baylor College of Medicine Cerebro, Houston, TX 77030, USA.

Victoria Armendariz, Baylor College of Medicine Cerebro, Houston, TX 77030, USA.

Irene E Jose, Baylor College of Medicine Cerebro, Houston, TX 77030, USA.

Kevin D Vuong, Baylor College of Medicine Cerebro, Houston, TX 77030, USA; Department of Neurology, Baylor College of Medicine, Houston, TX 77030, USA.

Paola Suarez, Department of Psychiatry and Behavioral Sciences, UCLA, Los Angeles, CA 90095, USA; UCLA Hispanic Neuropsychiatric Center of Excellence, Los Angeles, CA 90095, USA.

Adriana M Strutt, Baylor College of Medicine Cerebro, Houston, TX 77030, USA; Department of Neurology, Baylor College of Medicine, Houston, TX 77030, USA.

References

  1. American Psychological Association (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57(12), 1060–1073. [PubMed] [Google Scholar]
  2. American Psychological Association (2013). Guidelines for the practice of telepsychology. American Psychologist, 68(9), 791–800. [DOI] [PubMed] [Google Scholar]
  3. American Psychological Association (2016). Revision of ethical standard 3.04 of the “ethical principles of psychologists and code of conduct” (2002, as amended 2010). American Psychologist, 71(9), 900. [DOI] [PubMed] [Google Scholar]
  4. Arango-Lasprilla J. C., Rivera D., Garza M. T., Saracho C. P., Rodríguez W., Rodríguez-Agudelo Y.  et al. (2015). Hopkins verbal learning test-revised: Normative data for the Latin American Spanish-speaking adult population. Neurorehabilitation, 37(4), 699–718. [DOI] [PubMed] [Google Scholar]
  5. Ardila A. (2003). Culture in our brains: Cross-cultural differences in the brain-behavior relationships In Toomela A. (Ed.), Cultural guidance in the development of the human mind (, pp. 63–86). Westport, CT: Ablex Publishing. [Google Scholar]
  6. Artiola i Fortuny L., Hermosillo D., Heaton R. K., & Pardee R. E. (1999). Manual de normas y procedimientos para la Batería Neuropsicológica en Español. Tucson, AZ: Neuropsychology Press. [Google Scholar]
  7. Backman W., Bendel D., & Rakhit R. (2010). The telecardiology revolution: Improving the management of cardiac disease in primary care. Journal of the Royal Society of Medicine, 103(111), 442–446. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Baquet C. R., Carter-Pokras O., & Bengen-Seltzer B. (2004). Healthcare disparities and models for change. American Journal of Managed Care, 10, SP5–SP11. [PubMed] [Google Scholar]
  9. Bashshur R. L., Krupinski E. A., Thrall J. H., & Bashshur N. (2016). The empirical foundations of teleradiology and related applications: A review of the evidence. Telemedicine and e-Health, 22(11), 868–898. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Benedict R. H. B., Schretlen D., Groninger L., Dobraski M., & Shpritz B. (1996). Revision of the brief Visuospatial memory test: Studies of normal performance, reliability, and validity. Psychological Assessment, 8(2), 145–153. [Google Scholar]
  11. Benito-Cuadrado M. M., Esteba-Castillo S., Böhm P., Cejudo-Bolívar J., & Peña-Casanova J. (2002). Semantic verbal fluency of animals: A normative and predictive study in a Spanish population. Journal of Clinical and Experimental Neuropsychology, 24(8), 1117–1122. [DOI] [PubMed] [Google Scholar]
  12. Berchick E. R., Hood E., & Barnett J. C. (2018). Health insurance coverage in the United States: 2017 Current Population Reports (U.S. Census Bureau Publication No. P60-264) Washington, DC: U.S. Government Printing Office. [Google Scholar]
  13. Blumenberg E., & Evans A. E. (2010). Planning for demographic diversity: The case of immigrants and public transit. Journal of Public Transportation, 13(2), 23–45. [Google Scholar]
  14. Boada M., Cejudo J. C., Tàrraga L., López O. L., & Kaufer D. (2002). Neuropsychiatric iventory questionnaire (NPI-Q): Validación española de una forma abreviada del neuropsychiatric inventory NPI. Neurologia, 17(6), 317–323. [PubMed] [Google Scholar]
  15. Boone K. B., Salazar X., Lu P., Warner-Chacon K., & Razani J. (2002). The Rey 15-item recognition trial: A technique to enhance sensitivity of the Rey 15-item memorization test. Journal of Clinical and Experimental Neuropsychology, 24(5), 561–573. [DOI] [PubMed] [Google Scholar]
  16. Boyd J. L. (1981). A validity study of the Hooper visual organization test. Journal of Consulting and Clinical Psychology, 49(1), 15. [DOI] [PubMed] [Google Scholar]
  17. Brandt J., Spencer M., & Folstein M. (1988). The telephone interview for cognitive status. Neuropsychiatry, Neuropsychology, and Behavioral Neurology, 1(2), 111–117. [Google Scholar]
  18. Brearly T., Shura R., Martindale S., Lazowski R., Luxton D., Shenal B.  et al. (2017). Neuropsychological test administration by videoconference: A systematic review and meta-analysis. Neuropsychology Review, 27(2), 174–186. [DOI] [PubMed] [Google Scholar]
  19. Brickman A. M., Cabo R., & Manly J. J. (2006). Ethical issues in cross-cultural neuropsychology. Applied Neuropsychology, 13(2), 91–100. [DOI] [PubMed] [Google Scholar]
  20. Brown A., López G., & Lopez M. G. (2016). Broadband subscription among Latinos little changed since 2010. Retrieved from  https://www.pewresearch.org/hispanic/2016/07/20/2-broadband-subscriptions-among-latinos-little-changed-since-2010/. Accessed May 13, 2020.
  21. Bunker L., Hshieh T. T., Wong B., Schmitt E. M., Travison T., Yee J.  et al. (2017). The SAGES telephone neuropsychological battery: Correlation with in-person measures. International Journal of Geriatric Psychiatry, 32(9), 991–999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Cabassa L. J., Zayas L. H., & Hansen M. C. (2006). Latino adults’ access to mental health care: A review of epidemiological studies. Administration and Policy in Mental Health and Mental Health Services Research, 33(3), 316–330. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Castanho T. C., Amorim L., Zihl J., Palha J. A., Sousa N., & Santos N. C. (2014). Telephone-based screening tools for mild cognitive impairment and dementia in aging studies: A review of validated instruments. Frontiers in Aging Neuroscience, 6, 16. doi: 10.3389/fnagi.2014.00016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Cercy S. P. (2012). The verbal clock test: Preliminary validation of a brief, vision-and motor-free measure of executive function in a clinical sample. The Clinical Neuropsychologist, 26(8), 1312–1341. [DOI] [PubMed] [Google Scholar]
  25. Chakrabarti S. (2015). Usefulness of telepsychiatry: A critical evaluation of videoconferencing-based approaches. World Journal of Psychiatry, 5(3), 286–304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Cherner M., Suarez P., Lazzaretto D., Artiola i., Fortuny L., Rivera Mindt M.  et al. (2007). Demographically-corrected norms for the brief Visuospatial memory test-revised and Hopkins verbal learning test-revised in monolingual Spanish speakers from the U.S.-Mexico border region. Archives of Clinical Neuropsychology, 22(3), 343–353. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Clark E., Fredricks K., Woc-Colburn L., Bottazzi M. E., & Weatherhead J. (2020). Disproportionate impact of the COVID-19 pandemic on immigrant communities in the United States. PLOS Neglected Tropical Diseases, 14(7), e0008484.Advance online publication. doi: 10.1371/journal.pntd.0008484. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Cullum C. M., Hynan L., Grosch M., Parikh M., & Weiner M. (2014). Teleneuropsychology: Evidence for video teleconference-based neuropsychological assessment. Assessment, 20(10), 1028–1033. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Jager C. A., Budge M. M., & Clarke R. (2003). Utility of TICS-M for the assessment of cognitive function in older adults. International Journal of Geriatric Psychiatry, 18(4), 318–324. [DOI] [PubMed] [Google Scholar]
  30. Demaerschalk B. M. (2010). Telestrokologists: Treating stroke patients here, there, and everywhere with telemedicine. Seminars in Neurology, 30(5), 477–491. [DOI] [PubMed] [Google Scholar]
  31. Derogatis L. R. (2002). Cuestionario de 90 Síntomas (SCL-90-R). Manual. Madrid: TEA Ediciones, S.A. [Google Scholar]
  32. Duff K., Tometich D., & Dennett K. (2015). The modified telephone interview for cognitive status is more predictive of memory abilities than the mini-mental state examination. Journal of Geriatric Psychiatry and Neurology, 28(3), 193–197. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Echemendia R. J., Harris J. G., Congett S. M., Diaz M. L., & Puente A. E. (1997). Neuropsychological training and practices with Hispanics: A national survey. The Clinical Neuropsychologist, 11(3), 229–243. [Google Scholar]
  34. Ellison J., Jandorf L., & Duhamel K. (2011). Assessment of the short acculturation scale for Hispanics (SASH) among low-income, immigrant Hispanics. Journal of Cancer Education, 26(3), 478–483. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Fernández-San Martín M. I., Andrade C., Molina J., Muñoz P. E., Carretero B., Rodríguez M.  et al. (2002). Validation of the Spanish version of the geriatric depression scale (GDS) in primary care. International Journal of Geriatric Psychiatry, 17(3), 279–287. [DOI] [PubMed] [Google Scholar]
  36. Franco-Martin M. A., Bernardo-Ramos M., & Soto-Perez F. (2012). Cyber-neuropsychology: Application of new technologies in neuropsychological evaluation. Actas Espanolas de Psychiatria, 40(6), 308–314. [PubMed] [Google Scholar]
  37. Fontenot K., Semega J., & Kollar M. (2018). Income and Poverty in the United States: 2017  (U.S. Census Bureau Publication No. P60-263) Washington, DC: U.S. Government Printing Office; Retrieved from:  https://www.census.gov/library/publications/2018/demo/p60-263.html [Google Scholar]
  38. García-Campayo J., Zamorano E., Ruiz M. A., Pardo A., Pérez-Páramo M., López-Gómez V.  et al. (2010). Cultural adaptation into Spanish of the generalized anxiety Disorder-7 (GAD-7) scale as a screening tool. Health and Quality of Life Outcomes, 8, 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. George S., Hamilton A., & Baker R. S. (2012, 2012Article 715194). How do low-income urban African Americans and Latinos feel about telemedicine? A diffusion of innovation analysis. International Journal of Telemedicine and Applications. doi: 10.1155/2012/715194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Gleichgerrcht E., Camino J., Roca M., Torralva T., & Manes F. (2009). Assessment of functional impairment in dementia with the Spanish version of the activities of daily living questionnaire. Dementia and Geriatric Cognitive Disorders, 28(4), 380–388. [DOI] [PubMed] [Google Scholar]
  41. Grosch M., Gottlieb M., & Cullum C. M. (2011). Initial practice recommendations for teleneuropsychology. The Clinical Neuropsychologist, 25(7), 1119–1133. [DOI] [PubMed] [Google Scholar]
  42. Gurnani A. S., John S. E., & Gavett B. E. (2015). Regression-based norms for a bi-factor model for scoring the brief test of adult cognition by telephone (BTACT). Archives of Clinical Neuropsychology, 30(3), 280–291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Hall J. R., Balldin V. H., Gamboa A., Edwards M. L., Johnson L. A., & O'Bryant S. E. (2018). Texas Mexican-American adult normative studies: Normative data for the repeatable battery for the assessment of neuropsychological status (RBANS). Developmental Neuropsychology, 43(1), 27–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Harrell K., Wilkins S., Connor M., & Chodosh J. (2014). Telemedicine and the evaluation of cognitive impairment: The additive value of neuropsychological assessment. Journal of the American Medical Directors Association, 15(8), 600–606. [DOI] [PubMed] [Google Scholar]
  45. Heaton R. K., Taylor M. J., & Manly J. (2003). Demographic effects and use of demographically corrected norms with the WAIS-III and WMS-III In Tulsky D. S.  et al. (Eds.), Clinical interpretation of the WAIS-III and WMS-III (, pp. 181–210). San Diego, CA: Academic Press. [Google Scholar]
  46. Hildebrand R., Chow H., Williams C., Nelson M., & Wass P. (2004). Feasibility of neuropsychological testing of older adults via videoconference: Implications for assessing the capacity for independent living. Journal of Telemedicine and Telecare, 10(3), 130–134. [DOI] [PubMed] [Google Scholar]
  47. Hooper M. W., Nápoles A. M., & Pérez-Stable E. J. (2020). COVID-19 and racial/ethnic disparities. Journal of the American Medical Association, 323(24), 2466–2467.32391864 [Google Scholar]
  48. International League Against Epilepsy (2020). Neuropsychological assessments for epilepsy surgery during COVID-19 restrictions. Retrieved from  https://www.ilae.org/files/dmfile/Neuropsychological-Assessments-for-Epilepsy-Surgery-.pdf. Accessed June 25, 2020.
  49. Inter Organizational Practice Committee (2020a). Recommendations/guidance for teleneuropsychology (TeleNP) in response to the COVID-19 pandemic. Retrieved from  https://www.vapsych.org/assets/docs/COVID19/Provisional%20%20Recommendations-Guidance%20for%20Teleneuropsychology-COVID-19-4.pdf. Accessed May 21, 2020.
  50. Inter Organizational Practice Committee (2020b). Guidance/recommendation for models of care during the novel coronavirus pandemic. Retrieved from:  https://static1.squarespace.com/static/50a3e393e4b07025e1a4f0d0/t/5ed7d6c58ec40f3dce143b40/1591203525610/IOPC+Models+of+Care+During+COVID-19+Pandemic.pdf. Accessed September 15, 2020.
  51. Jacobsen S., Sprenger T., Andersson S., & Krogstad J. (2003). Neuropsychological assessment and telemedicine: A preliminary study examining the reliability of neuropsychology services performed via telecommunication. Journal of the International Neuropsychological Society, 9(3), 472–478. [DOI] [PubMed] [Google Scholar]
  52. Judd T., Capetillo D., Carrión-Baralt J., Mármol L. M., Miguel-Montes L. S., Navarrete M. G.  et al. (2009). Professional considerations for improving the neuropsychological evaluation of Hispanics: A National Academy of neuropsychology education paper. Archives of Clinical Neuropsychology, 24(2), 127–135. [DOI] [PubMed] [Google Scholar]
  53. Kirkwood K., Peck D., & Bennie L. (2000). The consistency of neuropsychological assessments performed via telecommunication and face to face. Journal of Telemedicine and Telecare, 6(3), 147–151. [DOI] [PubMed] [Google Scholar]
  54. Kroenke K., Spitzer R. L., & Williams J. B. (2001). The PHQ-9: Validity of a brief depression severity measure. General Internal Medicine, 16(9), 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Krueger K. R., Lam C. S., & Wilson R. S. (2006). The word accentuation test - Chicago. Journal of Clinical and Experimental Neuropsychology, 28(7), 1201–1207. [DOI] [PubMed] [Google Scholar]
  56. Española L. R. A. (2001). Diccionario de la lengua española [Dictionary of the Spanish Language] (22nd ed.). Madrid, Spain: Author. Accessed September 15, 2020. [Google Scholar]
  57. Lachman M., Agrigoroaei S., Tun P., & Weaver S. (2014). Monitoring cognitive functioning: Psychometric properties of the brief test of adult cognition by telephone (BTACT). Assessment, 21(4), 404–417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Lamel S. A., Haldeman K. M., Ely H., Kovaril C. L., Pak H., & Armstrong A. W. (2012). Application of mobile teledermatology for skin cancer screening. Journal of the American Academy of Dermatology, 67(4), 576–581. [DOI] [PubMed] [Google Scholar]
  59. Loeb P. A. (1996). ILS: Independent Living Scales Manual. San Antonio, TX: Psychological Corp. & Harcourt Brace Jovanovich. [Google Scholar]
  60. Loh P. K., Donaldson M., Flicker L., Maher S., & Goldswain P. (2007). Development of a telemedicine protocol for the diagnosis of Alzheimer's disease. Journal of Telemedicine and Telecare, 13(2), 90–94. [DOI] [PubMed] [Google Scholar]
  61. Lopez M. H., Gonzalez-Barrera A. & Patten E. (2013). Closing the digital divide: Latinos and technology adoption. Retrieved from:  https://www.pewresearch.org/hispanic/2013/03/07/closing-the-digital-divide-latinos-and-technology-adoption/.. Accessed May 21, 2020.
  62. Lopez-Magallón A. J., Otero A. V., Welchering N., Bermon A., Castillo V., Duran A.  et al. (2015). Patient outcomes of an international telepediatric cardiac critical care program. Telemedicine and eHealth, 21(8), 601–610. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Lucas-Carrasco R. (2012). Spanish version of the geriatric depression scale: Reliability and validity in persons with mild-moderate dementia. International Psychogeriatrics, 24(8), 1284–1290. [DOI] [PubMed] [Google Scholar]
  64. Macias Gil R., Marcelin J. R., Zuniga-Blanco B., Marquez C., Mathew T., & Piggott D. A. (2020). COVID-19 pandemic: Disparate health impact on the Hispanic/Latinx population in the United States. Journal of Infectious Diseases Article jiaa474. Advance online publication. doi: 10.1093/infdis/jiaa474. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Magán I., Sanz J., & García-Vera M. P. (2008). Psychometric properties of a Spanish version of the Beck anxiety inventory (BAI) in the general population. The Spanish Journal of Psychology, 11(2), 626–640. [PubMed] [Google Scholar]
  66. Manly J. J., Schupf N., Stern Y., Brickman A. M., Tang M., & Mayeux R. (2011). Telephone-based identification of mild cognitive impairment and dementia in a multicultural cohort. Archives of Neurology, 68(5), 607–614. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Márquez-González M., Losada A., Fernández-Fernández V., & Pachana N. A. (2012). Psychometric properties of the Spanish version of the geriatric anxiety inventory. International Psychogeriatrics, 24(1), 137–144. [DOI] [PubMed] [Google Scholar]
  68. Mitsis E., Jacobs D., Luo X., Andrews H., Andrews K., & Sano M. (2010). Evaluating cognition in an elderly cohort via telephone assessment. International Journal of Geriatric Psychiatry, 25(5), 531–539. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Muñoz-García M., Cervantes S., Razquin C., Guillén-Grima F., Toledo J. B., Martínez-González M. Á.  et al. (2020). Validation study of a Spanish version of the modified telephone interview for cognitive status (STICS-m). Gaceta Sanitaria, 33(5), 415–420. [DOI] [PubMed] [Google Scholar]
  70. Muñoz-Sandoval A. F., Cummins J., Alvarado C. G., & Ruef M. L. (1998). Bilingual Verbal Ability Tests. Itasca, IL: Riverside Publishing. [Google Scholar]
  71. Muñoz-Sandoval A. F., Woodcock R. W., McGrew K. S., Mather N., & Ardoino G. (2009). Batería III Woodcock-Muñoz. Ciencias Psicológicas, 3(2), 245–246. [Google Scholar]
  72. Muntal S., Doval E., Badenes D., Casas-Hernanz L., Cerulla N., Calzado N.  et al. (2017). Nuevos datos normativos de la versión española de la repeatable battery for the assessment of neuropsychological status (RBANS) forma a. Neurologia, 35(5), 303–310. [DOI] [PubMed] [Google Scholar]
  73. Nasreddine Z. S., Phillips N. A., Bédirian V., Charbonneau S., Whitehead V., Collin I.  et al. (2005). The Montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4), 695–699. [DOI] [PubMed] [Google Scholar]
  74. Noe-Bustamante L., Mora L., & Lopez H. (2020). About one-in-four U.S. Hispanics have heard of Latinx, but just 3% use it. Retrieved from  https://www.pewresearch.org/hispanic/2020/08/11/about-one-in-four-u-s-hispanics-have-heard-of-latinx-but-just-3-use-it/. Accessed May 21, 2020.
  75. Olabarrieta-Landa L., Rivera D., Galarza-Del-Angel J., Garza M. T., Saracho C. P., Rodríguez W.  et al. (2015a). Verbal fluency tests: Normative data for the Latin American Spanish-speaking adult population. Neurorehabilitation, 37(4), 515–561. [DOI] [PubMed] [Google Scholar]
  76. Olabarrieta-Landa L., Rivera D., Morlett-Paredes A., Jaimes-Bautista A., Garza M. T., Galarza-del-Angel J.  et al. (2015b). Standard form of the Boston naming test: Normative data for the Latin American Spanish-speaking adult population. Neurorehabilitation, 37(4), 501–513. [DOI] [PubMed] [Google Scholar]
  77. Ostrosky F., Gomez E., Matute E., Rosselli M., Ardila A., & Pineda D. (2012). Neuropsi: Atención y Memoria, segunda edicion. Mexico, D.F: Manual Moderno S.A. de C.V. [Google Scholar]
  78. Ostrosky-Solis F., Ardila A., & Rosselli M. (1999). NEUROPSI: A brief neuropsychological test battery in Spanish with norms by age and educational level. Journal of the International Neuropsychological Society, 5(5), 413–433. [DOI] [PubMed] [Google Scholar]
  79. Ostrosky-Solís F., Gómez M. E., Villaseñor E. M., Roselli M., Ardila A., & Pineda D. (2003). Neuropsi: atención y memoria: 6 a 85 años. Mexico City: American Book Store. [Google Scholar]
  80. Ostrosky-Solís F., Gómez-Pérez M. E., Matute E., Rosselli M., Ardila A., & Pineda D. (2007). Neuropsi attention and memory: A neuropsychological test battery in Spanish with norms by age and educational level. Applied Neuropsychology, 14(3), 156–170. [DOI] [PubMed] [Google Scholar]
  81. Phillips N. A., Chertkow H., Pichora-Fuller K., & Wittich W. (2020). Special issues on using the Montreal cognitive assessment for telemedicine assessment during COVID-19. Journal of the American Geriatrics Society, 68(5), 942–944. [DOI] [PubMed] [Google Scholar]
  82. Pitts R., & Naumenko O. (2016). The 2014 standards for educational and psychological testing: What teachers initially need to know. Working Papers in Education, 2(1), 1–6. [Google Scholar]
  83. Pons J. I., Matías-Carrelo L., Rodríguez M., Rodríguez J. M., Herrans L. L., Jiménez M. E.  et al. (2008). Estudios de validez de la Escala de Inteligencia Wechsler Para Adultos Versión III, Puerto Rico (EIWA-III). Revista Puertorriqueña de Psicología, 19, 75–111. [Google Scholar]
  84. Pontón M. O., Satz P., Herrera L., Ortiz F., Urrutia C. P., Young R.  et al. (1996). Normative data stratified by age and education for the neuropsychological screening battery for Hispanics (NeSBHis): Initial report. Journal of the International Neuropsychological Society, 2(2), 96–104. [DOI] [PubMed] [Google Scholar]
  85. Pontón M. O. (2001). Research and assessment issues with Hispanic populations In Pontón M. O., & Leon-Carrion J. (Eds.), Neuropsychology and the Hispanic patient: A clinical handbook (, pp. 39–58). New York, NY: Psychology Press. [Google Scholar]
  86. Puente A. E., Perez-Garcia M., Lopez R. V., Hidalgo-Ruzzante N. A., & Fasfous A. F. (2013). Neuropsychological assessment of culturally and educationally dissimilar individuals In Paniagua F. A., & Yamada A. M. (Eds.), Handbook of multicultural mental health: Assessment and treatment of diverse populations (2nd ed., pp. 225–241). New York: Elsevier Academic Press. [Google Scholar]
  87. Rabin L. A., Brodale D. L., Elbulok-Charcape M. M., & Barr W. B. (2019). Challenges in the neuropsychological assessment of ethnic minorities In Pedraza O. (Ed.), Clinical cultural neuroscience: An integrative approach to cross-cultural neuropsychology (, pp. 55–80). New York, NY: Oxford University Press. [Google Scholar]
  88. Rey G. J., & Benton A. L. (1992). Examen de Afasia Multilingüe. Iowa City, IA: AJA Associates. [Google Scholar]
  89. Rivera D., Olabarrieta-Landa L., Van Der Elst W., Gonzalez I., Rodriguez-Agudelo Y., Aguayo Arelis A.  et al. (2019). Normative data for verbal fluency in healthy Latin American adults: Letter M and fruits and occupations categories. Neuropsychology, 33(3), 287–300. [DOI] [PubMed] [Google Scholar]
  90. Rivera Mindt M., Arentoft A., Kubo Germano K., D'Aquila E., Scheiner D., Pizzirusso M.  et al. (2008). Neuropsychological, cognitive, and theoretical considerations for evaluation of bilingual individuals. Neuropsychology Review, 18(3), 255–268. [DOI] [PMC free article] [PubMed] [Google Scholar]
  91. Rivera Mindt M., Byrd D., Saez P., & Manly J. (2010). Increasing culturally-competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24(3), 429–453. [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Sáenz R., & Garcia M. A. (2020). The disproportionate impact of COVID-19 on older Latino mortality: The rapidly diminishing Latino paradox. The Journals of Gerontology: Series B, Psychological Sciences and Social Sciences Article gbaa158. Advance online publication. doi: 10.1093/geronb/gbaa158. [DOI] [PMC free article] [PubMed] [Google Scholar]
  93. Schrank F. A., & Wendling B. J. (2018). The Woodcock–Johnson IV: Tests of cognitive abilities, tests of oral language, tests of achievement In Flanagan D. P., & McDonough E. M. (Eds.), Contemporary Intellectual Assessment: Theories, Tests, and Issues (4th ed., pp. 383–451). New York, NY: The Guilford Press. [Google Scholar]
  94. Smart M. J. (2015). A nationwide look at the immigrant neighborhood effect on travel mode choice. Transportation, 42(1), 189–209. [Google Scholar]
  95. Spanish TeleNP Assessment & Research (STAR) Consortium (2020). TeleNP for Spanish speakers in the time of COVID-19: Benefits, challenges, and practical considerations. Retrieved from:  https://drive.google.com/open?id=1grcno-Du9iQyk4KmfTwliBKFdYHQpg2W. Accessed May 21, 2020.
  96. Suarez P., Casas R., Lechuga D., & Cagigas X. (2016). Socially responsible neuropsychology in action: Another opportunity for California to lead the way. The California Psychologist, 49(4), 16–18. [Google Scholar]
  97. Tai D. B. G., Shah A., Doubeni C. A., Sia I. G., & Wieland M. L. (2020). The disproportionate impact of COVID-19 on racial and ethnic minorities in the United States. Clinical Infectious Diseases Article ciaa815. Advance online publication. doi: 10.1093/cid/ciaa815. [DOI] [PMC free article] [PubMed] [Google Scholar]
  98. Tombaugh T. N. (1996). Test of Memory and Malingering (TOMM) Manual. Tonawanda, NY: Multi Health Systems. [Google Scholar]
  99. U.S. Census Bureau (2019a). Quick facts: Los Angeles County, California, Retrieved from:  https://www.census.gov/quickfacts/fact/table/losangelescountycalifornia/PST045219#PST045219. Accessed May 21, 2020.
  100. U.S. Census Bureau (2019b). Quick facts: Suffolk County, Massachusetts, Retrieved from:  https://www.census.gov/quickfacts/suffolkcountymassachusetts. Accessed May 21, 2020.
  101. U.S. Census Bureau (2019c). Quick facts: New Haven City, Connecticut, Retrieved from:  https://www.census.gov/quickfacts/fact/table/newhavencityconnecticut/RHI725218#RHI725218. Accessed May 21, 2020.
  102. U.S. Census Bureau (2019d). Quick facts: Houston City, Texas, Retrieved from:  https://www.census.gov/quickfacts/houstoncitytexas. Accessed May 21, 2020.
  103. U.S. Census Bureau (2019e). Quick facts: Salt Lake County, Utah, Retrieved from:  https://www.census.gov/quickfacts/fact/table/saltlakecountyutah/PST045219. Accessed May 21, 2020.
  104. U.S. Census Bureau (2019f). Quick facts: United States. Retrieved from:  https://www.census.gov/quickfacts/fact/table/US/RHI725218. Accessed May 21, 2020.
  105. Üstün T. B., Chatterji S., Kostanjsek N., Rehm J., Kennedy C., Epping-Jordan J.  et al. (2010). Developing the World Health Organization disability assessment schedule 2.0. Bulletin of the World Health Organization, 88, 815–823. [DOI] [PMC free article] [PubMed] [Google Scholar]
  106. Uzzell B. P., Pontón M., & Ardila A. (Eds.) (2013). International handbook of cross-cultural neuropsychology. Hove, East Sussex, UK: Psychology Press. [Google Scholar]
  107. Vahia I., Ng B., Camacho A., Cardenas V., Cherner M., Depp C.  et al. (2015). Telepsychiatry for neurocognitive testing in older rural Latino adults. The American Journal of Geriatric Psychiatry, 23(7), 666–670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  108. Wadsworth H. E., Dhima K., Womack K. B., Hart J. Jr., Weiner M. F., Hynan L. S.  et al. (2018). Validity of teleneuropsychological assessment in older patients with cognitive disorders. Archives of Clinical Neuropsychology, 33(8), 1040–1045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  109. Wechsler D. (2008). Escala de Memoria de Wechsler-IV: Manual de corrección y aplicación. New York, NY: Pearson. [Google Scholar]
  110. Wechsler D., Meng Y., Martínez I., & Zhu J. (2014). WAIS-IV: Escala Wechsler de Inteligencia para Adultos-IV. Mexico City, Mexico: Manual Moderno. [Google Scholar]
  111. White T., & Stern R. A. (2003). NAB, Neuropsychological Assessment Battery: Psychometric and technical manual. Lutz, FL: Psychological Assessment Resources. [Google Scholar]
  112. Woodcock R. W., Alvarado C. G., Schrank F. A., McGrew K. S., Mather N., & Muñoz-Sandoval A. F. (2019). Batería IV Woodcock-Muñoz. Itasca, IL: Houghton Mifflin Harcourt. [Google Scholar]

Articles from Archives of Clinical Neuropsychology are provided here courtesy of Oxford University Press

RESOURCES