Acquisition of Visual Perception in Blind Adults Using the BrainPort Artificial Vision Device

Intensive low vision rehabilitation resulted in some functional improvement in a sample of blind adults using the BrainPort artificial vision device.

Abstract

OBJECTIVE. We sought to determine whether intensive low vision rehabilitation would confer any functional improvement in a sample of blind adults using the BrainPort artificial vision device.

METHOD. Eighteen adults ages 28–69 yr (n = 10 men and n = 8 women) who had light perception only or worse vision bilaterally spent up to 6 hr per day for 1 wk undergoing structured rehabilitation interventions. The functional outcomes of object identification and word recognition were tested at baseline and after rehabilitation training.

RESULTS. At baseline, participants were unable to complete the two functional assessments. After participation in the 1-wk training protocol, participants were able to use the BrainPort device to complete the two tasks with moderate success.

CONCLUSION. Without training, participants were not able to perform above chance level using the BrainPort device. As artificial vision technologies become available, occupational therapy practitioners can play a key role in clients’ success or failure in using these devices.

Providing a limited sense of sight to people who are blind is no longer a science fiction fantasy but is now a possible reality with the use of artificial vision devices (AVDs). Retinal implant technology and sensory substitution devices have been shown to provide people who are blind with the ability to interpret their environment (Alteheld, Roessler, & Walter, 2007; Benav et al., 2010; Chader, Weiland, & Humayun, 2009; Chebat, Schneider, Kupers, & Ptito, 2011; Hub, Hartter, Kombrink, & Ertl, 2008; Johnson & Higgins, 2006; Merabet et al., 2009; Mokwa et al., 2008; Ptito, Moesgaard, Gjedde, & Kupers, 2005; Renier & De Volder, 2010; Sachs & Gabel, 2004; Striem-Amit, Guendelman, & Amedi, 2012; Zrenner et al., 2011). Most recently, Dorn et al. (2013) reported that 54% of blind participants who had the Argus II retinal prosthesis (Second Sight Medical Products, Sylmar, CA) implanted were successful in performing a motion detection task. Other strategies for restoring vision, such as visual cortical implants and ocular regeneration, are being pursued at the basic research level (Crapo et al., 2012; Dobelle, 2000; Fernández et al., 2005; Koizumi, Okumara, & Kinoshita, 2012; Ueda, Mizuno, & Araki, 2012).

The visual percepts conferred by existing AVDs do not resemble normal vision, however, because the sensations they provide are devoid of color, depth, and detailed form. In the case of sensory substitution technologies, the stimulus evokes visual qualia in most participants, but the response is decidedly nonvisual (Cattaneo, Vecchi, Monegato, Pece, Cornoldi, 2007; Ortiz et al., 2011). Nevertheless, the states of ultra-low vision provided by AVDs do allow for some functional improvements in object detection and mobility tasks (Benav et al., 2010; Chebat et al., 2011; Nau, Bach, & Fisher, 2013; Nau, Pintar, Fisher, Jeong, & Jeong, 2014; Zrenner et al., 2011).

Obtaining functional visual percepts with current artificial vision technologies is not intuitive. The need for guided practice over a protracted period must be understood by patients and clinicians alike. Basic visual skills, obvious and intuitive for those with sight, must be relearned or newly learned with AVDs. Skills that need to be developed include a sense of egocentric localization, shape discrimination, size constancy, object distance, and motor coordination with the body, including hands and feet. Our experience indicates that patients must work one on one with a visual rehabilitation specialist or occupational therapy practitioner over a prolonged period to master these fundamental skills. More complex tasks, such as navigation through cluttered visual environments, mandate additional rehabilitation training to teach effective strategies. Other blind skills (i.e., skills used by people with visual impairments), such as learning to use a white cane or reading Braille, also take considerable time and effort before proficiency is attained.

The need for extensive rehabilitation is not unique to the visual system (Kao, Shumsky, Knudsen, Murray, & Moxon, 2011; Martinez, Brezun, Zennou-Azogui, Baril, & Xerri, 2009; Schlaug, Marchina, & Norton, 2009). For example, hand transplant patients undergo rehabilitation for 6–8 hr/day, 6–7 days/wk for at least 1 yr to learn how to use their new hands. Without this intense rehabilitation schedule, patients are not able to fully integrate the transplanted limb (Eickhoff et al., 2008; Schneeberger et al., 2006, 2007). Without a clinical rehabilitation structure to support the dissemination and use of artificial vision technologies, rates of abandonment and unfair perceptions of device failure are likely to be high.

We have worked with nearly 100 blind people over the past 4 yr with an AVD known as the BrainPort vision device (Wicab Inc., Madison, WI). This sensory substitution device uses a camera and the tongue as a paired substitute for the eye (Figure 1). Visual scene information is captured by a mini–video camera that is mounted on a pair of sunglasses. The video signal is channeled to a processor that spatially averages the image, converting the grayscale pixels to a “grayscale” voltage on the dorsal surface of the tongue through a 400-pixel intraoral device (IOD). The tongue is an ideal location for representing tactile information because the tongue has a two-point discrimination ability that rivals the fingertips (Wicab Inc., 2008), and the presence of an electrolytic solution (saliva) ensures good electrical contact. Through this system, people who are blind or have very low vision are able to receive additional information about their environment and, after training with the device, are able to process this information to a functional end.

Figure 1.

Illustration of BrainPort vision device hardware and functions.

Because the field of artificial vision is so new, published rehabilitation protocols are essentially nonexistent for people whose vision has been restored by surgical means (retinal implant chips), medical intervention (gene therapy), or use of sensory substitution devices (tactile and auditory). This article outlines the accumulation of our experiences with >2,000 person–hours of training for people who are blind on the use of a sensory substitution device for vision. This protocol is intended to provide an initial infrastructure for low vision professionals and researchers who will be working with artificial vision technologies.

Method

Research Design

We used a prospective, single-center, unmasked, within-subjects repeated-measures design in which we assessed each participant’s functional vision using a set of functional assessments including object identification and word recognition. Testing was performed at baseline and again after approximately 15–20 hr of training with the BrainPort vision device. The participants were allowed to take the device home for a period of 1 yr and were followed at 3-mo intervals. During the at-home period, participants were required to log 300 min of device use per month to continue in the study. Participants were contacted monthly by study staff via telephone to reinforce skills and encourage practice. All research protocols were approved by the University of Pittsburgh institutional review board, and all participants provided informed consent before beginning any study activities.

During the training intervention, experimental use of the BrainPort device was strictly controlled. All participants used the device within the laboratory environment and underwent the same training and testing protocol administered by the same research staff. All participants were trained by the same two research assistants (authors Pintar and Fisher) working in the sensory substitution laboratory. Each assistant had more than 40 hours of hands-on training with the device before training participants.

Participants

Participants included 18 visually impaired adults (8 with congenital and 10 with acquired blindness) with documented visual acuity of light perception only or worse bilaterally. Any residual vision that would disqualify participants was verified during screening with the Freiburg Acuity Test, a computerized visual acuity test, according to standard published methods (Bach, 1996; Schulze-Bonsel, Feltgen, Burau, Hansen, & Bach, 2009; Wilke et al., 2008). Participants for this study were excluded from participation if they had any impairment that would prevent them from understanding how to use the BrainPort (e.g., traumatic brain injury). Participants were recruited from the sensory substitution laboratory research registry, which included people who contacted our laboratory to participate in artificial vision research studies. Although we have worked with approximately 100 people using the same training protocol, only 18 were included in the data presented in this article (Friberg, Nau, Pintar, Fisher, & Chen, 2011; Nau et al., 2013; Nau, Fisher, Pintar, & Engleka, 2011; Nau et al., 2014).

Instruments

The field of artificial vision is still in its infancy; therefore, a consensus has yet to be reached on which end points to use in the evaluation of visual percepts provided by these devices (National Eye Institute, 2011). We administered two functional outcomes measures for this study. The first measure involved object identification. Four objects were placed on a tabletop with a solid black cover to provide a contrasting background. Objects were placed in a horizontal line approximately 50 cm in front of the participant and were spaced about 10 cm from each other. Objects included a softball, a coffee mug with a handle, a plastic banana, and a yellow highlighter marker. The order of object placement was predetermined and randomized with a total of 20 discrete trials. Participants were given 2 min to reach out and touch the target object. If the participant did not respond within 2 min, the trial was scored as incorrect.

The second measure was a word identification test. A three- to five-letter word was presented on a 17-in. computer screen using 95-point Century Gothic font with a black screen background and contrasting white words. The resulting letters were 3.5 cm tall. Participants were given 3 min to identify the word. If they were unable to respond within 3 min, the trial was scored as incorrect. The 10 words used were bus, dog, cup, moon, ring, farm, tree, dress, bread, and plant. Words were presented in random order for a total of 10 trials.

Intervention

The interveners for this study included an optometrist (author Nau), a research assistant with a master’s degree in occupational therapy (author Pintar), and a research assistant with a bachelor’s degree. Our rehabilitation program was developed within the sensory substitution laboratory in cooperation with internal and external occupational therapists and is based on a set of core skills used as building blocks that, once attained, can be incorporated into occupational task performance. All the research interveners in this study were intimately involved in developing the training program over the course of 3 yr. Improvements to the protocol were made over time in response to participant progress and feedback. We standardized the skills to be learned within discrete learning pods accompanied by corresponding worksheets that document progression or need for continued training. Fidelity of the interveners was ensured by randomly assigning the training among each of the researchers. Analysis of data showed no difference in performance according to which intervener did the instruction. Participants for all research studies underwent at least two 3-hr supervised training sessions twice per day for 3 days, which could be spaced over a span of 2 wk according to participant preference.

Description of the Training Program

Level 1: Familiarization With BrainPort Components.

Participants were introduced to the features of the BrainPort vision device. We demonstrated maintenance tasks such as changing batteries, caring for the device, and troubleshooting. We also discussed the limitations of the technology, such as lack of color, lack of three-dimensional depth perception, and the fact that with the BrainPort vision device, the stimulus is primarily a tactile percept.

Level 2: Basic Shapes.

During early stages of training, removal of visual clutter was critical. We created a high-contrast, blank environment by seating the participant at a table covered in a piece of black felt in front of a background wall similarly covered. The initial training task was to have the participant hold a white foam bar against the black background and experience how the sensation on the tongue display changed as they altered the orientation and distance of the bar. This procedure provided vital biofeedback. Next, participants found and identified basic white two-dimensional shapes on a black vertical surface. The participant searched for characteristics of the image felt on the tongue, such as sharp corners and straight edges.

Level 3: Identification of High-Contrast Symbols.

Participants were introduced to perceiving, recognizing, and categorizing more complex shapes and symbols such as letters and numbers. Single letters and numbers were first presented with strategies for adjusting the zoom or field of view to identify the salient characteristics of the symbol. Once the participant was able to identify individual letters, multiple letters were combined to form words. When needed, two dimensional shapes were supplemented with tactile cues, such as molded plastic raised letters, puffy paint, or Wikki Stix (Omnicor, Inc., Phoenix, AZ). Participants were also introduced to commonly encountered symbols, such as exit and restroom signs, to illustrate the concept of categorizing similarly shaped symbols into groups.

Level 4: Scene Topography and Functional Reach.

Level 4 tasks moved from the two-dimensional world to more complex three-dimensional tasks, allowing participants to explore the effects of perspective, shadows, and contrast in addition to object location. Initially, training occurred in the high-contrast environment; later, lower contrast situations were created by removing the black felt. Participants were encouraged to touch the objects and to reanalyze the stimulation pattern to build the link between haptic knowledge and the newly developing tongue tactile knowledge (Figure 2). In this way, tongue-to-hand coordination took place analogous to hand-to-eye coordination. We had to retrain the concepts of egocentric location and position; for people with congenital blindness, this skill must be newly taught because most have not previously interacted in this “visual” manner. Participants were then taught two-dimensional depth perception strategies, including object size and the relative location of various objects on the IOD (i.e., higher or lower) and comparisons to known reference points such as the hand or foot. These important but sometimes difficult skills are essential for object retrieval and obstacle avoidance. At this juncture, most participants were able to stack blocks, build simple structures, and locate and retrieve 2-cm objects and place them in a cup.

Figure 2.

Pairing haptic information from the hands with information from the device and early ambulation spatial relations training task. Use of a simplified background decreases visual clutter and increases object contrast, both important for early training.

Level 5: Preambulation Techniques and Safety.

Before participants could ambulate, we addressed safety awareness by teaching scanning techniques to reduce risk of falls. Participants were asked to scan the room and locate objects outside the initial field of view. Another activity was identifying which one chair out of three was occupied. Participants were asked to explore and identify architectural features (e.g., door, window, hallway, stairs) in the safety of the lab environment before progressing to more complex activities such as object avoidance and navigation.

Level 6: Early Navigation.

We introduced recognition of landmark information by presenting spatial relationships between oneself and an object of regard while ambulating (Figure 2). Participants located a stationary object and walked toward it. As they approached the object, they were told to notice that the object occupied a larger relative space on the IOD. When they felt they were close, participants were told to reach for the object, thus obtaining further biofeedback regarding depth perception.

Level 7: Navigation.

Mobile scenarios were presented while incorporating head scanning and tracing. The concept of shore lining was illustrated by walking along a hallway with walls that contrast highly with the floors. Participants kept the juncture of the two structures at a fixed point on their tongue while walking. The feedback from the device became the surrogate shore line, thereby guiding the participant during navigation. We then moved on to navigating a hallway with turns and locating structures such as doorways and windows.

Level 8: Advanced Navigation.

Participants were now ready to detect optic flow resulting from self-motion and extrinsic motions experienced when other people or objects move. Level 8 emphasized environmental conditions and variables to provide useful feedback for outdoor navigation. Orientation and mobility instructors worked directly with the interveners to develop and implement this phase of training. Participants were taught how to discern between sidewalks and roads and to delineate crosswalks. They were also introduced to commonly encountered objects such as road signs, parking meters, and parked cars. Participants were asked to identify the presence of obstacles along their path of travel and changes in landscape such as curbs.

Level 9: Advanced and Personalized Skills.

Participants enrolled in studies that required long-term use of the device now progressed with an individually tailored program. Participants and lab staff worked together to determine which types of activities were important for a given participant and attempted to focus on acquisition and retention of the required skill set. Use in the home setting further advanced competency and strengthened neural connections as participants engaged in habitual tasks in familiar environments. For example, during initial community ambulation, participants were encouraged to use the BrainPort in conjunction with a cane or dog guide. Thus, information regarding the environment provided by the BrainPort was analogous to information provided by the participant’s typical assistive device.

Data Collection and Analysis

The two research assistants who provided the training also administered all outcomes tests. Data were stored and analyzed using IBM SPSS Statistics (Version 20; IBM Corp., Armonk, NY).

Results

The 18 participants ranged in age from 28 to 69; 10 were men and 8 were women. Information about the onset, duration, and etiology of blindness and comorbid conditions is included in Table 1. The number of participants decreased by attrition to 17 at 3 mo, 14 at 6 mo, and 13 at 9 mo; 13 participants completed the study at 12 mo. Participants who prematurely exited the study did so for the following reasons: disinterest (2), personal illness (1), family emergency (1), and a possible allergic reaction (1). After completing the training protocol, all participants demonstrated improvement in both the object and word identification assessments. At baseline, participants were unable to perform any of the tasks because they were unable to see any of the targets. At the end of 1 wk of training, participants were able to correctly identify and touch the target object (from among 4 objects) an average of 15 of 20 trials (range = 5–19 correct). Participants were able to read an average of 1.5 of 10 words (range = 0–10 words correct). Percentages of correct responses at all six time points are shown in Figures 3A and B.

Table 1.

Participant Demographic Characteristics

Gender	Age, Yr	Age at Onset of Blindness, Yr	Duration of Blindness, Yr	Etiology	Comorbid Conditions
Male	46	45	0.5	Shrapnel	Hypertension, hyperlipidemia
Male	39	36	3	Optic nerve atrophy	Retinal detachment, minor head trauma
Female	65	Birth	65	Congenital	Hypertension, hyperlipidemia
Male	60	51	8.5	Trauma, motor vehicle accident	Diabetes, hypertension
Male	28	24	3.5	Diabetic retinopathy	Diabetes, retinal detachment
Female	54	4	50	Keratoglobus	Osteoporosis, otosclerosis, hearing loss
Female	54	R age 50, L age 4	4	Keratoglobus, trauma	Diabetes, otosclerosis, meningioma, hypothyroidism, hyperlipidemia
Female	41	Birth	41	Glaucoma, cataract	None
Male	28	Birth	28	Retinopathy of prematurity	Amputation of left leg
Male	61	Birth	61	Retinopathy of prematurity	Hypertension, hyperlipidemia, prostate cancer
Female	69	Birth	69	Retinopathy of prematurity	Diabetes, arthritis, hypertension, hypolipidema, colitis
Female	54	Birth	54	Retinopathy of prematurity	Hypertension, hyperlipidemia
Male	48	28	20	Retinitis pigmentosis	Hypothyroidism, hypertension
Female	58	Birth	58	Retinopathy of prematurity	Herniated disc, headaches
Male	68	55	13	Diabetic retinopathy	Diabetes, hypertension, chronic kidney disease, hypercholesterolemia, peripheral vascular disease, hyperthyroidism
Male	64	28	36	Glaucoma, retinitis pigmentosa, cataract	Hypertension, hyperlipidemia, Stage 2 chronic kidney disease
Male	55	Birth	55	Keratoconus, retinal scar	Hypertension
Female	44	R age 39, L age 18	5	Retinopathy of prematurity, glaucoma	Appendicitis, vertigo

Figure 3.

Box plots illustrating percentages correct in trials. (A) Object identification trials. (B) Word identification trials.

Note. Black lines indicate median scores.

Discussion

Several studies have shown that rehabilitation can augment the brain’s inherent plasticity (Eickhoff et al., 2008; Kao et al., 2011; Martinez et al., 2009; Merabet & Pascual-Leone, 2010; Schlaug et al., 2009; Schneeberger et al., 2006, 2007). We are only beginning to understand how this finding applies to the use of AVDs in the field of artificial vision. New vision technology and efforts at ocular regeneration offer the eventual promise of meaningful improvement in the lives of people who are blind. However, this burgeoning field risks failure unless appropriate rehabilitative components are developed and multidisciplinary involvement occurs. Our results show that without instruction, the blind participants performed no better than chance when using the BrainPort vision device. However, after even a short amount of directed training in core skill acquisition, performance on the majority of outcomes significantly improved. Moreover, the scores for object and word recognition continued to improve after the participants practiced using the device at home. Notably, scores for the more advanced task of word recognition did not improve until the 3-mo visit. This delayed effect and low score outliers were likely a result of two factors: (1) the extra practice needed to identify the complex shapes of letters and (2) the fact that many participants had limited exposure to letters and words outside the context of Braille. Participants with relatively higher performance scores had previous experience with other forms of tactile sensory substitution such as the Optacon device (Telesensory Systems, Inc., Palo Alto, CA).

Limitations and Future Research

Our study has several limitations. We recruited participants from a registry of people who had expressed an interest in artificial vision, and a prerequisite to enrollment in our study was previous blind skill training. Therefore, our results may not be applicable to the blind community at large or to blind people who have yet to acquire blind skills. Additionally, generalizability of the object and word results presented in this article is limited because of the small sample size.

Because long-term rehabilitation at a tertiary care facility presents significant logistic, geographic, and financial barriers, we purposefully designed our initial on-site training program to be completed within 2 wk. Subsequent training over the course of 6 mo to 1 yr is probably necessary to develop advanced skills. Future directions for this research will include deploying longer term rehabilitation programs and including formal orientation and mobility training.

Implications for Occupational Therapy Practice

Low vision rehabilitation specialists must be prepared to teach concepts that people normally acquire in infancy. Users of the BrainPort have to learn or relearn a sense of egocentric direction, “vision”-to-hand coordination, motion detection, visual mapping, and interpretation of stimuli. The role of the occupational therapist is to teach “vision” skills, which can be difficult if the patient has preconceived notions about what seeing is “supposed to be.” For example, adult patients with a lifetime of visual experiences can emotionally reject a surrogate visual perception that does not restore vision to their predisease state. Other patient issues, such as depression or inflated perceptions about what current technology is able to provide, can negatively affect progress. Learning to “see” again takes patience and considerable ongoing mental effort from both the patient and the practitioner. Significant effort is required on the part of blind patients to successfully use current means of restoring vision. Patients must have realistic expectations about the amount of time and energy required to acquire the skills needed to make ultra-low vision useful.

The results of this study have the following implications for occupational therapy practice:

• Sensory substitution devices are a novel and potentially viable method for enabling a sense of the environment for people with blindness.

• Use of these devices is not intuitive and requires structured and detailed training over the course of several months.

• The occupational therapy community must work with developers of artificial vision technologies to optimize training programs.