Evaluating the Feasibility of Videophone-Assisted Neuropsychological Testing for Intensive Care Unit Survivors: A Pilot Study

Abstract

Background:

Most hospitals lack neuropsychologists, and this lack has hampered the conduct of large-scale, multicenter clinical trials to evaluate the effect of interventions on long-term cognition in patients in intensive care units (ICUs).

Objective:

To evaluate the feasibility of videophone-assisted neuropsychological testing using an inexpensive high-definition web camera and a laptop.

Methods:

This prospective, single-center observational study, conducted at a tertiary care academic hospital, included ICU survivors aged 18 years or older. Participants were seated in a quiet room with a proctor who provided neuropsychological testing forms and addressed technical difficulties. The neuropsychological rater was in a room 100 yd (90 m) from the participant. Skype was used for videoconferencing via a wireless connection. After the testing session was completed, participants completed surveys.

Results:

In April 2017, 10 ICU survivors (median age, 63 years; range, 51–73 years) were enrolled. All indicated that “Videophone-assisted neuropsychological testing is reasonable to use in research studies.” When asked “What made the videophone-assisted cognitive testing difficult?” 1 participant (10%) reported occasionally becoming frustrated with the testing because the wireless internet speed was slower than usual and reduced the resolution of visual stimuli. Three participants (30%) reported difficulty with the line orientation task because the lines were “shaky” and the images were “hard to see.”

Conclusion:

Videophone-assisted neuropsychological testing is feasible for evaluating cognition in multicenter studies of ICU patients. Feedback provided will be used to refine this telemedicine approach to neuropsychological testing.

Introduction

Approximately a quarter of intensive care unit (ICU) survivors will develop long-term cognitive impairment, with a severity similar to that of mild Alzheimer disease, after a critical illness.¹ Interventions designed to preserve long-term cognition are lacking. The gold standard for assessing cognition is comprehensive face-to-face neuropsychological testing. Unfortunately, many hospitals do not have access to neuropsychological experts, limiting the feasibility of large-scale multi-center ICU clinical trials with cognitive outcomes. This barrier has hampered the development and evaluation of ICU interventions to preserve long-term cognition.

Videophone-assisted neuropsychological testing may overcome this barrier. Using this method, neuropsychological raters at a central coordinating center conduct cognitive assessments remotely through videophone (e.g., Skype) to patients located at distant sites. A meta-analysis of 12 studies observed that neurocognitive testing conducted by this method were correlated with face-to-face evaluations.² The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) is a comprehensive neuropsychological battery that has been used to measure global cognition ICU survivors.¹ Administering the RBANS via videophone has been shown to be feasible, reliable, and highly correlated with face-to-face evaluations.³ However, this study used expensive videoconference systems (MRSP $12,999), making their method cost-prohibitive for multi-center studies. Therefore, we conducted this pilot study to evaluate the feasibility of conducting videophone-assisted RBANS testing using an inexpensive high-definition web camera and a laptop computer.

Methods

This was a prospective single-center observational study conducted at a tertiary care academic hospital and approved by the local institutional review board. This study sample included 10 ICU survivors ≥ 18 years of age who were previously enrolled in clinical studies by our research group. Participants were excluded when hearing, visual, language impairments, or inability to follow commands precluded ability to complete neuropsychological testing.

In April 2017, subjects completed neuropsychological testing via videophone, which assessed global cognition using the RBANS.⁴ Normal was considered to be 100 with scores of 85 and 70 considered to be suggestive of mild cognitive impairment (MCI) and mild dementia, respectively.⁵

During assessments, the subject was seated in a quiet room with a proctor who provided neuropsychological testing forms to the subject and addressed technical difficulties. The room included a high-definition (1920×1080 pixels) web camera ($50) with zoom capability mounted on a tripod ($20, Figure 1). The web camera was angled towards the subject’s table, and the zoom was adjusted so that testing forms were visible to the neuropsychological rater through the videophone (Figure 2). The subject’s room included a laptop computer ($1000) with a 15.6” diagonal screen so that the subject could interact with the neuropsychological rater and see the visual stimuli for cognitive tests. External speakers ($20) were used for improved audio quality over the laptop’s internal speakers.

Figure 1.

The setup of the videophone-assisted neuropsychological testing on the subject’s side. A high definition (1920×1080) web camera is mounted on a tripod and is pointed towards the table. The web camera is connected to a laptop computer with external speakers. The neuropsychological rater is communicating with the subject using videoconferencing software such as Skype.

Figure 2.

The web camera on the subject’s end should be pointed and adjusted so that the case report form can be fully visualized by the neuropsychological rater.

A neuropsychological rater (EC), who had experience in administering the RBANS from a previous study,¹ was located in another room 100 yards away from the subject. A high-definition web camera was pointed to the rater’s face and connected to a desktop computer with a 21” diagonal high-definition (1920×1080 pixels) computer monitor. As dictated by each neuropsychological test, the rater held visual stimuli in front of the web camera for subjects to see. After the testing session ended, participants completed a survey to rate their overall experience with the videophone-assisted neuropsychological testing and provide feedback on how it could be improved (survey questions are listed in the Table).

Table.

Subject survey about their experience with the videophone-assisted neuropsychological examination.

Survey Questions	Subject responses n (%)
1) On a scale from 0 (the worst) to 10 (the best), rate your overall experience with the videophone-assisted neuropsychological testing.	0 – 0 (0%) 1 to 6 – 0 (0%) 7 – 1 (10%) 8 – 5 (50%) 9 – 2 (20%) 10 – 2 (20%)
2) The videophone-assisted neuropsychological testing was reasonable to use in research studies.	Strongly agree – 9 (90%) Somewhat agree – 1 (10%) Neutral – 0 (0%) Somewhat disagree – 0 (0%) Strong disagree – 0 (0%)
3) The videophone-assisted neuropsychological testing was a frustrating experience.	Strongly agree – 0 (0%) Somewhat agree – 1 (10%) Neutral – 0 (0%) Somewhat disagree – 2 (20%) Strong disagree – 7 (70%)
4) What did you like about the videophone-assisted cognitive testing?	“It was novel. The portability is a plus. It’s a great idea. You can reach more people…” “Pretty easy” “It was plain and comfortable. There was no pressure. She was easy to understand.” “Convenience. Good for long distant assessments.” “Potentially mobile, potential for neuropsychological assessment to come to the patient.” “Easy to do, pretty straightforward.” “I can see the tester. Easier to do the digit span test when you can see somebody.” “I liked it better than the telephone neuropsychological testing.” “Easy to follow, it could be done at home.” “Very convenient. I can see why you would use it out in the field. It makes it easy for the patient.” “With the videophone tests over the computer, there was no feelings or challenge versus one on one.”
5) What made the videophone-assisted cognitive testing difficult?	“There were some autofocusing issues.” “The line orientation questions were shaky and made it harder. Gave me vertigo. Forward and backwards with the reading part was also difficult.” “The line orientation task was difficult.” “It was not difficult at all”. “Image distortion and breaking up due to poor wireless connection.” “The process over the videophone was not difficult.”
6) Is there anything we could do to improve the videophone-assisted testing process?	“None.” “No, it’s perfect.” “Just trying to keep thing more still.” “[Proctor] should not provide feedback. He/she should be a blank slate and should not provide input how the patient is doing.” “Maybe have some of the numbers in the flip charts be larger, more readable, but I could read them.” “None.” “The numbers were hard to see on the line orientation task. The picture could be larger.” “Ensure [internet] connection reliability.” “It would be good to have more rapport with the tester. Personal touch at the beginning versus a robot critical feeling of someone about to judge me.”

Results

We enrolled 10 ICU survivors with a median (range) age of 63 (51, 73) years. Four (40%) were female, 1 (10%) were nonwhite race, 6 (60%) graduated from college or graduate school, 10 (100%) resided at home, and 1 (10%) was wheelchair dependent. The mean (SD) RBANS was 89.4 (11.6); 4 0% of participants had an RBANS score suggestive of mild cognitive impairment, and no participants had an RBANS score suggestive of dementia.

The Table summarizes the subjects’ survey responses. When asked to rate their overall experience with the videophone-assisted neuropsychological testing from 0 (worst) to 10 (best), the subject’s median (range) score was 8 (7,10). All subjects agreed that “Videophone-assisted neuropsychological testing was reasonable to use in research studies,” and 9 (90%) disagreed with the statement that “the videophone-assisted neuropsychological testing was a frustrating experience.” One (10%) subject occasionally felt frustrated with videophone-assisted neuropsychological testing, because the wireless internet speed was slower than usual and decreased the resolution of visual stimuli. The participant denied that these technical difficulties significantly interfered with the completion of the cognitive tasks.

Various general themes emerged from participant responses to the question, “What did you like about the videophone-assisted cognitive testing?”: it was easy, convenient, and a potential approach to allow participants to complete testing in their homes. When asked, “What made the videophone-assisted cognitive testing difficult?” 3 participants reported difficulty with the line orientation task because the lines were “shaky” and the images were “hard to see.” In response to the question, “Is there anything we could do to improve the videophone-assisted testing process?” participants suggested keeping the visual stimuli steadier on the screen and making the pictures and text larger.

Discussion

Previous studies have found videophone-assisted neuropsychological testing using expensive videoconferencing systems to be highly correlated with face-to-face testing.^3,6,7 Because traditional videoconferencing equipment may be cost-prohibitive for large multi-center trials, we tested the feasibility of using an inexpensive high-definition web camera. We observed that this method was feasible. All subjects reported a positive experience with videophone-assisted neuropsychological testing and felt that this method was reasonable to use for research. More importantly, they felt that this approach to neuropsychological testing could be potentially be conducted in subjects’ homes. This could potentially improve follow-up especially for ICU survivors who develop substantial disability.⁸

Several subjects endorsed difficultly completing the line orientation task due to an unsteady image when the rater held up the visual stimuli. Consequently, we have modified our videophone-assisted neuropsychological methods, so that we will present scanned visual stimuli directly on the subject’s computer screen using the screen share function.. Due to participant feedback of poor video quality secondary to internet connectivity, we now recommend an ethernet connection to the internet to maximize internet bandwidth.

This pilot study has several limitations, including small sample size and possible under detection of problems with videophone-assisted testing. Our cohort had a mean RBANS that was higher than previously reported (89 versus 80),¹ were functionally intact, and prior experience in completing neuropsychological testing (phone and face-to-face); these factors may have inflated the acceptability or tolerability of our videophone approach. Approximately 40% had RBANS scores below the threshold for MCI and may not have been able to adequately express their concerns or difficulties with this method of testing.

In conclusion, videophone-assisted neuropsychological testing using an inexpensive web camera is feasible and well-accepted by subjects. This telemedicine-based approach may provide an inexpensive and feasible method of conducting more accessible, comprehensive neuropsychological testing across geographically diverse centers for ICU clinical trials.