Applications of Augmented Vision Head-Mounted Systems in Vision Rehabilitation

Abstract

Vision loss typically affects either the wide peripheral vision (important for mobility), or central vision (important for seeing details). Traditional optical visual aids usually recover the lost visual function, but at a high cost for the remaining visual function. We have developed a novel concept of vision-multiplexing using augmented vision head-mounted display systems to address vision loss. Two applications are discussed in this paper. In the first, minified edge images from a head-mounted video camera are presented on a see-through display providing visual field expansion for people with peripheral vision loss, while still enabling the full resolution of the residual central vision to be maintained. The concept has been applied in daytime and nighttime devices. A series of studies suggested that the system could help with visual search, obstacle avoidance, and nighttime mobility. Subjects were positive in their ratings of device cosmetics and ergonomics. The second application is for people with central vision loss. Using an on-axis aligned camera and display system, central visibility is enhanced with 1:1 scale edge images, while still enabling the wide field of the unimpaired peripheral vision to be maintained. The registration error of the system was found to be low in laboratory testing.

1 Introduction

With normal vision we enjoy the benefits of a wide field of view, primarily used for navigation and orientation, together with the high-resolution capabilities of the fovea that enable discrimination of fine details. Visual impairments due to diseases or injuries typically affect only one of these two aspects, either restricting the wide peripheral visual field (VF) in conditions such as retinitis pigmentosa (RP) and glaucoma, or damaging the central high-resolution vision in conditions such as age-related macular degeneration (AMD). When peripheral VF loss is severe (leaving useful VFs less than 20° in diameter, i.e. tunnel vision), a patient's mobility can be greatly affected due to reduced ability to spot obstacles and difficulties in navigation. Social interactions of patients may be affected by failing to note or respond to people appropriately. In addition, RP and other similar retinal diseases often cause night blindness due to early loss of the rod cells that function at low light levels. Damage to central vision causes reduced visual resolution and reduced contrast sensitivity, which result in difficulties in performing tasks that require the ability to discern fine detail or small differences in contrast.

Traditional vision rehabilitation devices are usually designed to recover, at least partially, the lost visual function, but at a high cost for the remaining visual function. For example, magnification increases resolution but inherently limits the field of view. Similarly, minifying devices increase the field of view, but cause a loss of resolution in addition to generating spatial distortions, and may restrict scanning eye movements. A design approach for vision-rehabilitation devices called “vision multiplexing” attempts to avoid or reduce these limitations by combining the wide field-of-view and the high-resolution capabilities in devices in ways that permit these functionalities to be both separable and useful [1]. We have suggested the use of an augmented vision system based on a see-through head-mounted display (HMD) that implements spatial multiplexing via superposition [1].

For patients with tunnel vision, an augmented-vision system using a miniature video camera with a wider field of view than that of the see-through HMD has been developed. The camera's images are processed at video rates to provide cartoon-like edge images of the scene, and are shown in the HMD (see Figure 1). The minified edge images enable the patients to see and detect potential obstacles and locate other objects that, without the minification, would fall outside of their residual VF. Once an object is detected in the HMD, patients can view it directly though the see-through display with the full resolution and color sensitivity of their natural vision. Since the edges occupy only a small portion of the display, they do not limit the clarity of the see-through view. We have developed and started to test such systems for patients with tunnel vision for use in daytime and nighttime conditions.

Figure 1.

Simulation of a street-crossing scene as it might appear to a patient with tunnel vision using an augmented-vision visual field expander. The faded area in the wide-scene picture (upper panel) represents the peripheral field that patients with tunnel vision typically cannot see when looking straight ahead. The picture in the lower panel provides a magnified representation of the display area and the view seen through the display. The natural (see-through) view is observed in full resolution, while the superimposed minified edge images provide a wide field of view enabling detection of all the pedestrians (not just the lady in the see-through view), which could be potential collisions not visible without the display.

For patients with central vision impairments, a camera-HMD aligned system has been developed based on our finding that superposition of edge images [2] on an original TV image is preferred by patients with resolution and/or contrast sensitivity losses [3]. In this system, the visibility of real-world objects are enhanced by high contrast edges, however, the edges have to be precisely aligned with the natural see-through view. As illustrated in Figure 2, for people with reduced high-resolution vision, HMD devices that provide magnification [4] can enable them to see more detail in a distant object (Figure 2b), but the restricted VF caused by the magnification may result in failure to detect potential obstacles in the periphery (e.g., the chair in the corridor). On the other hand, the on-axis aligned camera and HMD system allows users to maintain their full VF, while at the same time enhancing visibility (Figure 2c).

Figure 2.

Simulation of the enhancement for loss of high-resolution vision using an augmented vision system with an on-axis aligned camera and head-mounted display. (a) People with resolution loss will have difficulty discerning details at the end of the corridor. (b) Magnification assists in seeing details of the distant object, but restricts the field of vision, e.g., the chair is not present in the magnified view. (c) Superimposed edge images enhance visibility in the central area, while the full field of view is maintained.

2 Hardware

2.1 The augmented vision HMD visual field expander

In an initial evaluation, we compared 4 off-the-shelf HMD systems and 2 different cameras [5]. The HMDs were the Glasstron PLM-50 (Sony Corp., Tokyo, Japan), Virtual Stereo I-O i-glasses HMD (I-O Display Systems, Sacramento, CA), PC Eye-Trek (Olympus Optical Co. Ltd. Shinjuku-ku, Tokyo, Japan), Integrated EyeGlass (MicroOptical Corp., Westwood, MA), and two of MicroOptical's ClipOn systems: the QVGA CO-1 and VGA CO-3. Following the initial evaluation, we contracted with MicroOptical to modify their integrated EyeGlasses design for further assessments. Early prototypes were used to demonstrate the VF expansion and vision multiplexing potential of the system, and to assess usability with 2 RP patients [5]. The patients, who had severely restricted VFs, commented that the augmented-vision concept could be useful for navigation, obstacle avoidance, and hazard prevention. They did not find a small field-of-view display to be restrictive, provided that they could see through it (multiplexing) and it had an open design allowing viewing outside the display area (clearance) [6]; and they preferred a relatively high level of minification and minimal user control (automated system). A number of designs and experimental iterations served to refine the carrier lens size and the shape of the frame, including facilities for adjustments to ensure the frame fits securely and comfortably on people with various facial dimensions. The current generation of HMD weighs approximately 110g and provides a field of view of 16°(H) by 12°(V) (Figure 3), which represents a substantial improvement in cosmetics and ergonomics (reduced weight and better fit). In addition, it has a vertically-expanded exit pupil such that normal variation in device position while walking (e.g., the HMD sliding down the nose) can be accommodated without significant detriment to the display image.

Figure 3.

The current generation of the augmented vision HMD visual field expander for patients with tunnel vision. The wide-angle image captured by the video camera is processed by the controller to provide edge contour images of the scene. The edge images are displayed on the see-through display providing an expanded view. Once an object is detected via the minified cartoon, it can be examined with full resolution and color through the transparent display.

2.1.1 Cameras

The 2 cameras used in the early study [5] were the Mitsubishi M64283FP CMOS Artificial Retina (Mitsubishi Electric Research Laboratories, Cambridge) with 128×128 in-chip image processing, including edge enhancement that provided horizontal angular fields of 58° and 78° with 2 lenses, and a color 640×480 adaptation of the ViCam® USB PC Digital Camera by Vista Imaging Inc. (San Carlos, CA) that provided 59°, 72°, and 97° horizontal fields with 3 lenses. In a further evaluation, we used a Marshall V3214 lipstick CCD camera that provided horizontal fields of 52° and 90° with 2 lenses. After the manufacturer discontinued that product, it was replaced with a miniature CMOS camera (Supercircuits PC206) that provided a field of view of 60°. Because the sensitivity of that camera was insufficient for the night vision application, it was replaced with the KT&C KPC-S500 camera equipped with Sony's Ex-View CCD chipset, which provides a field of view of 80°. This camera is currently used for both the day and nighttime application of the system and provides 5× field expansion.

2.1.2 Edge detection

In the early evaluation [5], edge detection was performed using software provided with the cameras. Convolution with a simple 4-pixel neighbor gradient filter performed the enhancement which was followed by thresholding to obtain a binary edge image. The update rate of the systems was compromised in these early designs (5fps). These were replaced first with a dedicated edge detection system developed for us by DigiVision (San Diego, CA) and more recently by an integrated controller developed by MicroOptical. The DigiVision FPGA-based system provided edge video images at a rate of 30 frames/sec with only a 73μs delay (a little more than one scan line of the NTSC signal). Such a small delay was not a problem for our HMD field expander, as there is no registration requirement. The DigiVision edge detector, powered by a 1.5kg sealed lead acid battery, was used in the visual search study described in section 3.1.

In the latest system (Figure 3), a small controller box (the size of a deck of cards) includes the edge detection functionality, and also drives the camera and display. This controller weighs only 175g including a 2-hour Li-ion battery. The edge detection was implemented on the FPGA (100K gates) of the control box of the display system.

2.2 On-axis HMD image enhancement system

Unlike in the HMD system for patients with tunnel vision, accurate registration is critical in the visibility enhancement application for patients with central vision impairments (resolution or contrast sensitivity loss). A miniature video camera (Panasonic GP-KS462) with the same field of view (15°) as the display was aligned with the viewer's eye in the system (Figure 4a). Camera optics and display optics share the same path in the carrier lens, but in different polarization directions [7]. 1:1 scaled edge images derived using the same Digivision processor as mentioned above were precisely superimposed over the patients' natural view to create an appearance illustrated in Figure 2c.

Figure 4.

a) The visibility enhancement system with optical see-through HMD and integrated on-axis camera b) A picture captured through the view window of the on-axis HMD, which demonstrates the consistent registration performance across a wide range of object distances. Five identical white bar targets were placed on five black cardboards at 3, 4, 5, 6, and 7 feet from the HMD system. The registration was purposely shifted down to provide a simultaneously clear view of the real objects and their virtual edge images. Five identical gauges depicted next to the white bars show that the registration was consistent from 3 to 7 feet. Note, however, that the edges are dimmer than the white target in this implementation.

The registration error of the on-axis HMD system was found to be consistent across a wide depth range (Figure 4b). The registration difference between targets at 3 and 7 feet was only 0.07° [7]. We found the variability in the position of the HMD relative to the eye to be the major cause of this registration error, while the on-axis alignment of the HMD itself could be very accurate. For example, 2mm deviation of eye position relative to the HMD caused 0.06° registration difference, which can easily occur in daily wearing, but may not be a problem for people with central vision loss. Although the registration error of the on-axis HMD system is low, it is not yet ready for evaluations by patients with central vision loss. The edge images are not of sufficient quality, as the on-axis optics reduces the light reaching the camera, and the miniature camera used in the registration evaluation had limited light sensitivity.

3 Laboratory and outdoor evaluations of HMD visual field expander

A series of studies were conducted to test the value of the augmented vision HMD visual field expander for a variety of tasks and conditions. Laboratory studies allowed us to assess device prototypes, identify limitations, acquire early users' feedback, and thereafter improve design and configuration. These studies were interleaved with successive prototype development. For example, pilot trials with the device indicated that patients with severe tunnel vision might have difficulty locating a real target, even though they could see a contour image of the target in the HMD. We realized that with residual VFs much smaller than the display, patients could not determine where within the transparent display they were looking, and therefore they had difficulty in registering the minified view to the real-world view. A center mark was added to the display as a registration aid to help users locate targets in the real world. In a simple calibration procedure, the camera is adjusted so that the center mark, a real target seen through the display, and the minified target contour are aligned. In real use, when a target contour is noted in the display, moving the head to align the center mark with the target contour image will bring the real target into the see-through view. We also noted in early mobility (walking) studies that, although the image visibility was satisfactory in indoor environments and outdoors on cloudy days, it was not adequate outdoors on sunny days. Therefore the prototype of the daytime device was redesigned with a tinted carrier lens and an improved beam splitter.

3.1 Visual search

Within a controlled environment, visual search tasks resemble some of the visual demands of daily life (e.g., navigation, scanning the environment, and finding objects of interest). Kuyk et al. [8, 9] found that scanning ability in a visual search task was one of the main predictors of mobility of visually impaired adults. We conducted a search study similar to Kuyk's. Subjects with severe tunnel vision searched for targets with and without the augmented vision system, and with auditory cues indicating target directions [10]. The search targets were displayed in random positions outside of subjects' residual VFs. Therefore, the targets could not be detected with natural vision when looking at a fixation point at the center of the screen (Figure 5).

Figure 5.

The visual search task on a picture background. Targets were presented outside subjects' residual visual fields. The faded area of the image represents the blind area of the subject's field. Auditory direction cues were provided by buzzers around the screen. When using the HMD system, minified edge images could provide cues for both the direction and eccentricity of targets (See Figure 1).

Each of the targets was composed of a random low-contrast letter within a black frame (triangle, square, or circle selected randomly), 3° or 5° in size. Only the target frame could be detected and recognized in the minified contour view. Subjects had to look through the display to view the low contrast letter targets foveally in order to identify the letter. Two experiments were conducted using large and small search areas (90°x74° and 66°x52°, respectively) on a blank gray background. Subjects were allowed to move their eyes and heads freely during the search. Head and eye positions were recorded and used to compute the gaze positions, directness of search path, angular speed of gaze; and to determine the search time to find the target. The directness score for a whole search path was the average cos(θ) weighted by the length of the current gaze shift, where θ was the angle between the current sample gaze shift and the target direction. A perfect path would have a directness of 1, regardless of gaze speed. Normally-sighted subjects performing the same task had directness scores of about 0.95.

For the larger search area used in the first experiment, all three subjects found the targets significantly faster using either auditory cues (39-58% less time, p≤0.003) or contour (edge-image) cues (28-74% less time, p≤0.024) than without a cue (Figure 6a). Auditory and contour cues also helped significantly improve their directness by 94∼126% (p≤0.015, Figure 6b). For the smaller search area used in the second experiment, the contour cues significantly improved the directness score by 62% (Figure 6d, p=0.014), but the search time performances with the HMD were mixed (Figure 6c). Among the 9 subjects, 6 subjects with VFs ≥10° (horizontal diameter) found targets more quickly with the device, but the other 3 with smaller VFs (<10°) were slower. We believe this was associated with lack of device familiarity (subjects had only about an hour of training before the study) as gaze speed was quite slow when using the HMD (about half the speed without the device). Regression analyses suggested that, if the gaze speed increases, the minimum VF required to gain a benefit from the device would be less than 10° [10]. Since the directness of the search improved for all subjects, any increase in gaze speed would then result in reduced search times, even for patients with smaller VFs. In fact, the analyses suggest that, in relative terms, patients with smaller VFs would benefit more, but of course their absolute performances would remain lower than those of patients with wider residual VFs.

Figure 6.

Visual search by subjects with tunnel vision on a simple blank gray background without cues, with contour cues from the HMD 5× field expander, and with auditory cues. (a) Search time and (b) directness of the 3 subjects in the large area (90°×74°) search. Auditory cues and contour (edge-image) cues significantly reduced search time and improved directness. (c) Search time of the 9 subjects grouped by VF in the small area (66°×52°) search. Subjects with larger VF could search faster with the HMD. (d) Directness of the 9 subjects in the small area search. Directness with either auditory or contour cues was better than without cues [10].

In the experiments reported above, targets were presented on a simple blank gray background, which is not representative of the real world. In a third experiment, targets were presented on a 66°×52° street-picture background (the picture changed on each trial). Figure 7 shows the search time performance of 9 subjects. As in the first and second experiments, auditory cues were very helpful and significantly reduced search times. However, when using the HMD only subject #5 performed faster with than without it, while subject #3 was unable to perform the searches using the HMD.

Figure 7.

Search times of 9 subjects with tunnel vision on a complex street picture background without cues, with contour cues from the HMD 5× field expander, and with auditory cues. Only subject #5 could find targets faster with the HMD. Subject #3 could not perform the task with the HMD.

These search results on the picture background appear disappointing, suggesting that the HMD might not helpful in a more complex visual environment. However, results from our outdoor and indoor mobility evaluations (section 3.3) indicate that people with RP are able to use the contour image to assist search in complex environments. Based on subjects' comments after the visual search experiment, we determined that a critical cause of the disappointing results was probably the LCD leakage in the device we used. In order to see the edge images presented by the HMD, the brightness of the display had to be set much higher for the picture than the blank background. This resulted in severe leakage, i.e. the non-edge pixels formed a white background blocking (or significantly reducing) the see-through view. This created such different “see-through” views for the two eyes that subject #3 could not switch attention between the background and the edge image presentation. The successful subject #5 reported a strategy he developed himself; he first moved his head to align the see-through window to a relatively dark area of the picture background so that he could easily find targets in the minified edge images, and then located the real targets guided by the contour cues. Other studies in our lab suggest that without leakage, users should be able to attend to either the background or the superimposed contour images, and easily switch back and forth [11]. Emitting display technologies, such as the OLED display or a scanning laser display, do not suffer from such leakage. We plan to test their use to confirm that search will be performed faster over the image background with than without the augmented-vision HMD field expander.

3.2 Collision detection

Once a person with tunnel vision sees an obstacle, s/he can judge potential collisions, so that timely maneuvers to avoid a collision can be taken. When a patient is wearing the augmented vision device, it might be preferable if the risk of collision could be judged directly from the minified edge images, as that would speed up the response. However, the minified edge images may impede the user's ability to make such judgments accurately and confidently. One concern is that with minification patients might feel that they are going to collide with everything seen in the display. This could cause too many unnecessary collision-avoidance maneuvers, or even rejection of the device. To evaluate the ability to make collision judgments with the minified edge images, we conducted a study in a virtual environment (a walking simulator) [12].

Ten normally sighted subjects stood 77cm from a wide (94°×79°) rear-projection screen that displayed a photo-realistic video model of a shopping-mall corridor. The movie scene was updated as if a subject was walking along a preset zig-zag path at a speed of 1.5m/s. Each trial consisted of walking one straight segment of the path while a stationary human-sized obstacle appeared at 5m and stayed on for one second. The obstacles were placed at varying distances from the trajectory of the path segment (path offset). Subjects reported verbally whether they would make any contact with the obstacle if they continued on the same trajectory without an avoidance maneuver. The subjects were instructed to make a forced choice. Response values at different path offsets were fit to a Gaussian cumulative density function. The mean of the Gaussian represents the perceived safe passing distance (PSPD), calculated for left and right sides separately, and the summation of PSPD for both sides represents the size of a space (collision envelope), within which any obstacle would be perceived by one to cause collision. The standard deviation of the fitted Gaussian represents the decision certainty.

With only a few minutes practice, each subject performed the task with and without the augmented vision device. The see-through views of both eyes were blocked so that subjects could only see the images in the display with one eye. In the without-device condition, the eye that would not be fit with the display in the with-device condition was patched.

It was found the collision envelope increased by only 16% (13cm, p=0.048, see Figure 8). The change was mainly for obstacles on the camera side (p=0.004), but not for the display side (p=0.890). The difference between the two sides is likely to be a result of the parallax due to the position difference between the display and the camera. In addition, there seemed to be a trend for the augmented device to reduce judgment certainty, but that effect was not statistically significant (p=0.089). It is encouraging that despite the very small and low resolution images seen in the HMD, subjects' judgments of potential collisions did not change much compared to the natural viewing condition. This suggests that with training users should be able to use the minified images directly for obstacle avoidance, in addition to judging by direct observation of real obstacles.

Figure 8.

Collision envelope size (sum of perceived safe passing distance for both sides) with the HMD 5× visual field expander increased only 16% compared to the without device condition. Data are from 10 normally-sighted subjects. Error bars are standard error of mean.

3.3 Night vision system evaluation

Two generations of the prototype night vision device have been evaluated by patients with RP and night blindness. The first study [13] evaluated an early prototype (LV-3), in which the edge-image mode had not been implemented, and therefore the device displayed only gray-scale images. Visual function measurements (acuity, letter contrast sensitivity, and VFs) and indoor mobility assessments (walking through a high-density obstacle course) were conducted without a device, with the LV-3, and with a commercially-available comparison device (Multi-Vision), which is an opaque HMD showing gray-scale images in 1:1 scale. Subjects had only brief training in how to use the devices. The assessment was conducted at two light levels, 16 and 2 lux that represent well-lit and poorly-lit streets in the Boston downtown area. As expected from the 4× minification of the early prototype, the LV-3 substantially expanded patients' VFs; the average expansion was 287% at 2 lux. Patients preferred its better comfort, fit and lighter weight in comparison to the Multi-Vision device (Figure 9a). However, they found it difficult to make use of the gray-scale images superimposed over the see-through view, and walking speed was slower and mobility errors were greater with the LV-3 than without a device. The study suggested that insufficient light sensitivity of the prototype camera limited LV-3 performance.

Figure 9.

Median ratings for performance of the night vision HMD field expander (prototypes LV-3 and NV-3) and the commercially-available comparison device (M-V) during indoor and outdoor assessments. (a) In the first study, the early LV-3 prototype was rated highly for comfort and weight, and M-V was rated highly for ease of use and image quality. (b) In the second extended wear pilot study, the improved NV-3 prototype was rated highly for prolonged wearing, and rated as fairly good in well-lit and poorly-lit areas; by comparison M-V was rated very helpful in totally dark areas. Error bars represent interquartile range. Rating scores 1= Very Poor; 2= Poor; 3= Fair; 4=Good; 5= Very Good.

In response to the feedback received and device limitations noted in the first study, the prototype underwent further development and an improved version (NV-3) was evaluated in a small-scale, extended wear pilot study. The improved prototype incorporated a full implementation of the minified edge-image mode, a software-adjustable center (registration) mark, and the KT&C KPC-S500 camera with better sensitivity at low light levels. Four RP patients participated in the second study, each taking the NV-3 and the comparison Multi-Vision device home for a minimum of 2 weeks in counterbalanced order. They were encouraged to use the devices in a variety of situations (indoors and outdoors; quiet and busy streets) and lighting levels (almost total darkness to bright street lighting). They completed a diary of their experiences with each device, recording their comments and noting the locations in which the device was evaluated.

Diary entries, questionnaire responses, and informal observations of mobility during training walks with the device on a specially designed outdoor course [14] confirmed that patients were able to see the minified edge images superimposed over their natural view, and that with practice they could identify objects from the image and to a limited extent could use the image for orientation and to guide mobility. Subjects rated the improved NV-3 as very good for prolonged wearing, and thought that it was reasonably helpful for mobility in well-lit and poorly-lit areas (Figure 9b). By comparison the MultiVision device was rated less highly for prolonged wearing (it was more uncomfortable to wear), but was considered very helpful for mobility in totally dark areas (Figure 9b). For those patients with very poor night vision, the see-through feature of the HMD did not help much in very dark areas. We also noted that, even with the better camera, the edge images were degraded at low light levels because the edge detection algorithm is sensitive to image noise, while the gray-scale images were still useable.

All patients demonstrated VF expansion with the NV-3 at low light levels (2 lux) in the clinic. At 2 lux without a device, patients' horizontal VF extents shrank on average to 58% of the VF size under normal room lighting (Figure 10a & b). The Multi-Vision device restored VFs to baseline levels (normal room lighting). By comparison, the NV-3 substantially increased the VF extent by 313% (Figure 10a & b). This VF expansion was less than that which would theoretically be predicted from the 5x minification provided by the device, mainly due to the barrel distortion of the micro camera's lens.

Figure 10.

Vision measurements without a device, with the improved NV-3 prototype of the night vision HMD field expander, and with the commercially-available comparison device (M-V). a) The extent of the visual field (VF) of one RP patient under full room illumination, and at 2 lux with and without the NV-3. b) Mean horizontal VF extents for 4 patients with tunnel vision and night blindness. The NV-3 substantially increased the VF extent at low light levels (2 lux), whereas the M-V device only restored the VF to baseline (normal room lighting, without device) levels. c) Mean contrast sensitivity scores for letter targets. The NV-3 had no effect on contrast sensitivity at low light levels (2 lux) whereas the comparison M-V device restored sensitivity to baseline levels. Error bars are standard deviations.

At low light levels, in addition to the peripheral visual field restriction, patients with night blindness suffer severe loss of resolution and contrast sensitivity (much more than normally-sighted people). At 2 lux, the patients had a reduction of 0.6 log units in their contrast sensitivity (Figure 10c). While the MultiVision device restored sensitivity to the levels achieved under normal room lighting, the NV-3 device had no effect on this aspect of visual function (Figure 10c). These results were expected. With the MultiVision (M-V) device, the contrast sensitivity test was viewed as a gray-scale intensified image, but with the NV-3 the test was viewed with natural low-light vision in the see-through view. The restoration of central vision sensitivity with the MultiVision device explains why it was found to be so helpful in totally dark areas (patients reported that it was like seeing in daylight).

The diary entries revealed that patients might not have had sufficient time to integrate the use of the multiplexed, minified edge image view with their habitual mobility behaviors (total wearing times reported by each subject across the 2-week period of home use were: 3, 6, 2, 2 hrs, respectively). Although all the patients reported walking independently at night, they had already modified their lifestyle to severely limit outdoor night mobility. At the time the study was conducted, three of the patients habitually walked less than 40 minutes after dark in a typical week. Wearing times at night were also limited by bad weather in winter in the Boston area, and long day light hours in summer. Evaluations of night vision devices might be better conducted in geographical areas where people with night blindness frequently walk outdoors after dark.

4 Discussion

We have described two applications of our augmented vision systems: one for people with peripheral vision loss and one for people with central vision loss. The latter is at an early stage in development, while the former has undergone many prototype iterations and improvements and has reached the point where it was possible for patients to take the device home in a pilot extended wear trial. Although the registration error of the on-axis aligned camera and HMD image enhancement system for people with central vision loss is low, evaluations of the system by patients have not been undertaken as the edge images are not yet sufficiently bright. We plan to conduct further evaluation once we find suitable micro displays.

In initial laboratory-based evaluations, the augmented vision HMD visual field expander appears to enable more efficient visual search, which with more training may provide increased speed of search for tunnel vision patients even with very narrow VFs. The ability to properly judge collisions using the minified edge image carries the promise that users will be able to respond directly to that image, initiating early avoidance maneuvers without being confused. Both functions should increase the safety and comfort of patients' mobility. Preliminary findings from the evaluations of the nighttime version of the HMD visual field expander provide evidence that patients with night blindness can use the device within a real world environment, and derive benefit from the VF expansion provided by the minified edge-images.

Active eye-movement scanning is a very important method by which patients can compensate for their VF loss. It is much more natural, easier, and more effective to move the eyes than to move the head. Despite restricted VFs, eye movements of patients with tunnel vision are not largely confined. In walking, only the horizontal eye position dispersion was found to be smaller than that of normally sighted people (9° vs 14°) [15], but the saccade sizes of patients with tunnel vision are similar to those of normally sighted people [16].

Therefore, a visual aid for tunnel vision should not restrict eye movements. Optical see-through HMD systems with an open design have advantages over opaque HMD systems for daytime use, as patients can move their eyes freely when wearing them. However, because patients with night blindness can not benefit from the see-through view on badly-lit streets under night conditions [13], an opaque HMD presenting a gray-scale image may have advantages over an optical see-through HMD, provided the opaque HMD system has a sufficiently wide field of view to minimally limit eye movements (30°). The multiplexing concept can still be implemented in opaque HMD systems, if VF expansion multiplexing is desired. The minified edge images of the wide field can be superimposed on gray-scale intensified night vision images shown in the display, creating a video-augmented view in place of the optical-augmented view of our basic system.

With our expanded understanding of patients' needs and continuous improvements in HMD technology, we continue our efforts to create novel devices that can effectively help people with vision impairment cope better with their disabilities.