Remote Neuropsychological Assessment in Rural American Indians with and without Cognitive Impairment

Abstract

Objective:

To determine the feasibility and reliability of a brief battery of standard neuropsychological tests administered via video teleconference (VTC) to a sample of rural American Indians compared with traditional face-to-face administration.

Methods:

The sample consisted of 84 participants from the Choctaw Nation in Oklahoma, including 53 females and 31 males [M age = 64.89 (SD = 9.73), M education = 12.58 (SD = 2.35)]. Of these, 29 had a diagnosis of mild cognitive impairment or dementia, and 55 were cognitively normal. Tests included the MMSE, Clock Drawing, Digit Span Forward and Backward, Oral Trails, Hopkins Verbal Learning Test-Revised, Letter and Category Fluency, and a short form Boston Naming Test. Alternative forms of tests were administered in counterbalanced fashion in both face-to-face and VTC conditions. Intraclass correlation coefficients (ICCs) were used to compare test scores between test conditions across the entire sample.

Results:

All ICCs were significant (p< .0001) and ranged from 0.65 (Clock Drawing) to 0.93 (Boston Naming Test), with a mean ICC of 0.82.

Conclusion:

Results add to the expanding literature supporting the feasibility and reliability of remote videoconference-based neuropsychological test administration and extend findings to American Indians.

Introduction

Telemedicine has grown substantially over the past decade, expanding to include a variety of healthcare services including telepsychiatry, telepsychology, and teleneuropsychology. The feasibility of video teleconferencing (VTC) methods of administering neuropsychological tests and assessing cognitive functioning in Caucasian populations is now well documented (Cullum, Hynan, Grosch, Parikh, & Weiner, 2014; Jacobsen, Sprenger, Andersson, & Krogstad, 2002), specifically among those with dementia (Cullum, Weiner, Gehrmann, & Hynan, 2006; Hildebrand, Chow, Williams, Nelson, & Wass, 2004; Loh et al., 2004; Poon, Hui, Dai, Kwok, & Woo, 2005). Additionally, VTC interactions have been shown to have good acceptance by patients and providers (Grubaugh, Cain, Elhai, Patrick, & Frueh, 2008; Hilty, Nesbitt, Kunneth, Crus, & Hales, 2007; Myers & Turvey, 2012; Parikh et al., 2013; Shore, 2013). While VTC has been found to be feasible and reliable, virtually all studies to date have included predominantly Caucasian populations, so additional research is needed to explore the utility and benefit of VTC for rural and ethnic minority groups.

American Indians often face barriers to healthcare access and are less likely than Caucasians to utilize available healthcare in their area (Zuckerman, Haley, Roubideauex, & Lillie-Blanton, 2004). Much of the American Indian population lives in rural areas with more limited access to specialty healthcare services (Norris, Vines, & Hoeffel, 2012). A 2004 survey of rural primary care providers identified that limited access to consultants, community support, and educational resources were all barriers to diagnosing and treating dementia in rural areas (Teel, 2004). As such, telehealth technology offers a potential solution to bring specialty healthcare services to underserved communities.

In terms of those with dementia, due to the increased number of people who are living to advanced age, it has been predicted that dementia will increase six-fold in the United States between the years 2010 and 2050 (Hebert, Weuve, Scherr, & Evans, 2013). Among the elderly, the American Indian and Alaska Native population is increasing twice as fast as the American population as a whole (Norris et al., 2012). In addition to more American Indians living to an advanced age, this population has higher rates of conditions (e.g., diabetes, heart disease) that can adversely affect cognitive functioning (Center for Disease Control, 2003; Denny, Holtzman, & Cobb, 2003; Evans & Kantrowitz, 2002; Harris, Albaugh, Goldman, & Enoch, 2003; Jervis et al., 2010; Langlois et al., 2003; Patterson Silver Wolf [Adelv unegv Waya], Duran, Dulmus, & Manning, 2014; Spicer et al., 2003; Steele, Cardinez, Richardson, Tom-Orme, & Shaw, 2008). Whereas the literature suggests that Alzheimer disease (AD) may be less common in some American Indian tribes than in Caucasians (Henderson et al., 2002; Hendrie et al., 1993) and may have a later age of onset (Rosenberg et al., 1996), relatively little is known about cognitive assessment in this population (Jervis et al., 2010).

Whereas VTC is becoming an established methodology for remote neuropsychological assessment (Cullum et al., 2014), VTC's feasibility and reliability for remote neuropsychological assessment has never specifically been examined in American Indians. Previous research has found that consultations and evaluations can be provided remotely to patients with known or suspected cognitive impairment using VTC equipment, and the availability of VTC in rural American Indian clinics may reduce barriers limiting access to specialized healthcare which many contribute to the earlier detection and more accurate diagnosis of dementia in this population.

Methods

Participants

Eighty-four participants were recruited through the University of Texas Southwestern Medical Center, Alzheimer's Disease Center satellite clinic in Talihina, Oklahoma, which recruits participants with cognitive disorders, healthy controls, and provides services for the Choctaw Nation Health Services Authority. The sample consisted of 31 (37%) males and 53 (63%) females between 46 and 88 years of age (M = 64.89, SD = 9.73). Subjects had a mean of 12.58 (SD = 2.35) total years of education (range = 6–18, with 12 = high school, 16 = college, and 18 = masters degree). Of this sample, 55 (65%) were cognitively normal controls and 29 (35%) were diagnosed with mild cognitive impairment (MCI) or dementia. Diagnoses were made prior to participation in the study via multidisciplinary consensus according to standard criteria (MCI: Petersen, 2004; AD: McKhann et al., 2011). Of the 84 participants, 85.7% identified as Choctaw (n = 72), 7.1% as Cherokee (n = 6), 3.6% as Creek (n = 3), 2.4% as Chickasaw (n = 2), and 1.2% as mixed (n = 1). All participants were fluent in English. Standard tests and alternative forms were administered to each participant in a counterbalanced fashion in both face-to-face and VTC conditions, which were also counterbalanced. All but two subjects were administered the second testing condition within the same day at least 20 min apart, and two were retested 7 and 14 days later.

Materials

For the face-to-face condition, trained research assistants administered the tests to participants in Talahina. Tests administered in each condition included a battery of measures commonly used in dementia evaluations, including tests of global cognitive function (MMSE; Folstein, Folstein, & McHugh, 1975), verbal memory (Hopkins Verbal Learning Test-Revised; Benedict, Schretlen, Groninger, & Brandt, 1998), letter fluency (FAS), category fluency (Animals), confrontation naming [Boston Naming Test (BNT)—15 item; Mack, Freed, Williams, & Henderson, 1992], attention (Digit Span Forward and Backward), Clock Drawing Test, and Oral Trails A and B (Ricker & Axelrod, 1994). We utilized a Polycom iPower 680 series videoconferencing system for the VTC condition connected via the Internet as previously described (Cullum et al., 2014). Examiners viewed participants on a 26 in LCD monitor with picture-in-picture display and mobile camera, so the examiner could view the participant and test materials. Subjects viewed the examiner on an identical 26 in LCD monitor.

Procedure

This research was approved by the Institutional Review Boards of the University of Texas Southwestern Medical Center and the Choctaw Nation, and each participant provided written informed consent prior to participating. Subjects' vision and hearing were determined to be adequate for the testing conditions, and the volume on the monitor was adjusted to allow for participants to respond to conversation-level questions prior to testing in the VTC session. Individuals were administered the neuropsychological tests in the face-to-face and VTC conditions, with the order of test condition and test order decided a priori and counterbalanced across consecutive subjects. In the VTC condition, participants were seated 30″ away from the LCD monitor by a local staff member and were subsequently introduced to the remote examiner on screen. Staff were available on site to assist with VTC equipment; however, no staff were present in the VTC testing room during testing, as we found in our previous work that participants acclimated well and no special assistance was required even among individuals with cognitive impairment. Test order was fixed and tests were administered according to standard instructions. The only modification to procedures necessary for the VTC condition involved the scoring of the participants' drawings. In this case, the subjects were asked to hold up their drawing to the camera in order for the examiner to score it immediately. Test form (standard or alternative) was alternated for each consecutive participant with the exception of the MMSE, wherein items were the same except for the use of an alternative three-word recall list that was developed based on words of similar syllables, frequencies, and familiarity. Intraclass correlation (ICC) coefficients and paired-samples t-tests were used to compare scores between testing conditions. For the purpose of demonstrating feasibility and reliability of VTC-based testing in as large a sample as possible, all subjects were combined for analysis. Statistical significance was set at a 0.004, calculated using a Bonferoni correction, to conservatively adjust for the number of comparisons and reduce the chance of a type II error.

Results

Test scores were highly similar between assessment conditions, with MMSE scores ranging from 11 to 30 in the face-to face condition and 12 to 30 in the VTC conditions, and all ICCs were significant (p < .001) (Table 1). Paired samples t-test revealed no significant differences between conditions on MMSE, Clock Drawing, Digit Span Backward, Oral Trails B, HVLT-R scores, or Verbal and Category Fluency. Small but statistically significant differences were seen on Digit Span Forward, Oral Trails A, and BNT total (Table 2).

Table 1.

Intraclass correlations between face-to-face and video teleconference conditions

Test	n	ICC	p-value
MMSE total	83	0.92	<.001
Clock total	82	0.65	<.001
Digits forward	84	0.72	<.001
Digits backward	84	0.69	<.001
HVLT-R total learned	84	0.88	<.001
HVLT-R delayed Recall	84	0.90	<.001
HVLT-R percent retention	84	0.84	<.001
Letter fluency total	83	0.93	<.001
Category fluency total	84	0.74	<.001
Oral Trails A time	82	0.83	<.001
Oral Trails B time	81	0.79	<.001
Boston Naming Test—short form total	84	0.93	<.001

Note:ICC = Intraclass Correlations; MMSE = Mini-Mental State Examination; HVLT-R = Hopkins Verbal Learn Test-Revised.

Table 2.

Mean differences between face-to-face and video teleconference conditions

Test	FTF, mean (SD)	VTC, mean (SD)	t	p-value
MMSE total	27.7 (2.4)	27.5 (2.7)	1.40	.166
Clock total	5.8 (0.6)	5.6 (0.9)	1.68	.096
Digits forward	5.9 (1.4)	5.5 (1.3)	2.98	.004*
Digits backwards	4.3 (1.1)	4.2 (1.1)	0.31	.760
HVLT-R total learned	21.8 (6.3)	22.8 (6.7)	−2.11	.038
HVLT-R delayed recall	7.0 (4.0)	7.6 (3.5)	−2.25	.027
HVLT-R % retention	70.0 (35.1)	77.7 (29.9)	−2.92	.005
Letter fluency total	31.9 (12.8)	32.0 (13.0)	−0.10	.920
Category fluency total	16.5 (4.8)	15.4 (4.8)	2.27	.026
Oral Trails A time	8.9 (2.4)	11.1 (3.0)	−9.60	<.001*
Oral Trails B time	76.0 (90.5)	78.8 (77.2)	−0.35	.726
Boston Naming Test	12.9 (2.2)	12.5 (2.6)	3.21	.002*

Notes: *< = .004, α-level was set at 0.004 to adjust for number of comparisons and reduce the chance of type II error.

FTF = face-to-face; VTC = video teleconference; MMSE = Mini-Mental State Examination; HVLT-R = Hopkins Verbal Learn Test-Revised.

Discussion

The use of remote telehealth-based diagnostic evaluations, including neuropsychological assessment, is growing and appears to offer a reasonable alternative to standard face-to-face examinations. The current results extend these findings and suggest that VTC is a valid means of remotely administering a variety of neuropsychological tests to similar rural American Indian populations, as test scores in both conditions were highly consistent. All ICCs were highly significant and averaged 0.82, demonstrating good agreement between test conditions. Although paired t-tests revealed statistically significant differences on several tests (Digit Span Forward, Oral Trails A, and BNT), these differences were very small (i.e., within three points) and not clinically meaningful, given that they fall within typical test-retest ranges for similar measures in general (Dikmen, Heaton, Grant, & Temkin, 1999).

Previous research has supported the consistency between VTC and face-to-face administration of neuropsychological tests in predominately Caucasian populations. Jacobsen et al. (2002) found that test condition (i.e., VTC vs. face-to-face) did not appear to impact the reliability of the measures, although performance on attention and memory tasks was higher in the VTC condition. However, other studies have demonstrated good consistency in neuropsychological test performance across test conditions and cognitive domains, while including cognitively intact and impaired elderly populations (Cullum et al., 2014; Grosch, Weiner, Hynan, Shore, & Cullum, 2015).

Whereas these findings add further support for the feasibility and reliability of remote VTC-based neuropsychological assessment, cost of the technology has been raised as a potential barrier in some settings. For example, Schopp, Johnstone, and Merrell (2000) noted that the cost to patients of telehealth-based neuropsychology services was ∼20% lower than the cost of the patient traveling to the neuropsychologist for service when distance was an issue. In recent years, the cost of providing VTC services has declined and the technology has become more widely available.

Limitations

The highest concentration of individuals with American Indian heritage reside in Oklahoma (Norris et al., 2012); however, American Indian communities are incredibly diverse, with 566 federally recognized tribes representing a large array of cultures and languages (U.S. Department of the Interior: Indian Affairs, 2016). Therefore, the characteristics of this sample likely differ from other American Indian communities. For instance, the American Indian populations in Talihina, Oklahoma, are community-dwelling as opposed to reservation-dwelling, and are well acculturated. All participants spoke English fluently and had a mean education of 12.58 years. Finally, there was limited diversity within this sample, as the participants were predominantly of Choctaw heritage. Conversely, while there is variability within American Indian communities, this study adds to the growing body of literature that telehealth and specifically teleneuropsychology assessment can be provided effectively, with good reliability in this population, addressing important disparities in healthcare access.

Other limitations include a relatively short interval between the testing conditions, which may lead to inflated correlations. However, test-retest results in the present sample are comparable with general test-retest values for each of the tests examined. Additionally, with our inclusion of older subjects with or without dementia, these results may not be generalizable to individuals with other neurological or neuropsychiatric diagnoses or to individuals with more severe cognitive impairment. Finally, the participants in this study were research volunteers and therefore may not be a true representation of individuals who could be served by VTC evaluations in the future.

Conclusion

While only 1.7% of residents of the United States identify as American Indian and/or Alaska Native, this population is growing among aging Americans (Norris et al., 2012). Therefore, it is important to provide alternative assessment and treatment modalities to increase diagnostic and treatment services for individuals with known or suspected dementia within the American Indian population. VTC is a feasible way to conduct brief neuropsychological examinations in order to increase access to care.

Funding

This work was funded in part by National Institute on Aging Grant R01 AG027776 and the Alzheimer's Disease Center Grant P30 AG012300.

Conflict of Interest

None declared.