Head-Mounted Display Technology for Low Vision Rehabilitation and Vision Enhancement

Abstract

Purpose

To describe the various types of head-mounted display technology, their optical and human factors considerations, and their potential for use in low vision rehabilitation and vision enhancement.

Design

Expert perspective.

Methods

An overview of head-mounted display technology by an interdisciplinary team of experts drawing on key literature in the field.

Results

Head-mounted display technologies can be classified based on their display type and optical design. See-through displays such as retinal projection devices have the greatest potential for use as low vision aids. Devices vary by their relationship to the user’s eyes, field of view, illumination, resolution, color, stereopsis, effect on head motion and user interface. These optical and human factors considerations are important when selecting head-mounted displays for specific applications and patient groups.

Conclusions

Head-mounted display technologies may offer advantages over conventional low vision aids. Future research should compare head-mounted displays to commonly prescribed low vision aids in order to compare their effectiveness in addressing the impairments and rehabilitation goals of diverse patient populations.

Introduction

Low vision is defined as chronic, uncorrectable visual impairment that impacts an individual’s daily life.¹ Unlike individuals with blindness, those with low vision have remaining sight that can be used for planning or executing tasks. The number of adults in the United States who are blind or have low vision is expected to reach 5.5 million by 2020.² Low vision rehabilitation (LVR) aims to improve patients’ functional abilities through the use of low vision aids (LVAs), orientation and mobility training, occupational therapy and/or educational interventions. Notably, in addition to improving functioning, LVR has also been shown to improve health-related quality of life and psychosocial well-being.³

Many commonly employed LVAs and rehabilitation strategies have been in use for years, yet little work has been done to study their efficacy.³ Moving the field forward depends on three important strategies. First, rigorous well-designed studies must be conducted to assess the effectiveness of new and existing treatments in low vision; second, innovative rehabilitation strategies and technologies should be investigated and compared to existing treatments; and third, LVAs and rehabilitation approaches should be tailored to the unique needs of patient populations.

While classically LVAs have consisted of magnifiers, spectacle mounted optical aids, and devices to enlarge images, increase illumination, improve contrast, and reduce glare, in recent years several publications have described the use of head-mounted display technology (HMD) to enhance vision.^4–9 HMD was initially targeted to military applications but lower costs have made it accessible to industrial and entertainment applications. Additionally, this group of technologies has the potential to improve patients’ vision through the coupling of image processing and wearable visual display systems. HMD has become more prevalent in recent years but the first head-mounted LVA, the Low Vision Enhancement System, was in fact developed over 20 years ago¹⁰ and improved patients’ contrast sensitivity, acuity and illumination;⁸ however, this device was heavy and functioned too slowly for routine use.⁸ Modern electronic HMDs have been shown to improve constricted peripheral fields,^4,6,9 night vision⁶ and visual acuity,⁷ though their effectiveness compared to standard LVAs has not been adequately studied.⁸ Here, we present an overview of the shortcomings of currently available LVAs; describe the various types of HMD that currently exist; review the optical and human factors considerations relevant to the use of HMD in low vision; and suggest a lexicon to standardize future discussions and evaluations of HMD technologies in vision research and clinical practice.

Methods

An approved waiver was obtained from the University of Michigan Institutional Review Board (Ann Arbor, MI, USA) since this work did not involve human subjects. The study conformed to all local laws and adhered to the tenets of the Declaration of Helsinki. This perspective is based on key literature in the field and on the experiences and observations of the multidisciplinary group of authors.

Current Low Vision Aids

Among patients presenting for LVR, approximately 25% have considerable peripheral field loss (PFL), while the remaining 75% have predominantly central vision loss (CVL).¹¹ As such, it is not surprising that the majority of research and clinical innovation in LVR has been directed toward patients with CVL.³ Commonly prescribed LVAs for patients with CVL include over-correction with reading (plus) lenses, handheld or stand magnifiers and closed-circuit television systems (CCTV).¹² The use of such devices was shown to improve self-reported patient functioning in a multi-center randomized controlled trial (RCT).¹³ Additionally, educational interventions may play a role in improving adaptation to CVL, problem solving skills, and patient functioning.¹⁴ One study examined the effectiveness of older types of HMD for patients with CVL. The Jordy and the Flipperport HMDs, which had a range of magnification and illumination options, provided significantly better distance visual acuity than standard LVAs but did not result in better near acuity or contrast sensitivity.⁸ Compared to modern devices, these older forms of HMD were relatively bulky and employed large cameras that connected to the display and limited their portability. Currently, several ongoing RCTs are focused on rehabilitation of CVL and seek to demonstrate the effectiveness of competing LVR strategies¹⁵ and of newer forms of HMD¹⁶ for patients with CVL.

Less research has been done to evaluate the effectiveness of LVAs for patients with low vision due to PFL.³ Nonetheless, PFL can have an effect on the functional status of many meaningful activities of daily living including community mobility, reading, and driving.¹⁷ The goal of expanding a patient’s field of vision, however, has challenged clinicians and researchers for a half century. Early attempts at field expansion used glasses to mount prisms and shift images to extend the field of view.¹⁸ Peli built on this early work to develop glasses with high power prism segments mounted above and below central fixation.¹⁹ Gazing up or down into the prisms resulted in awareness of an additional 30 degrees of visual field and these glasses were effective for patients with hemianopia, though less than half of patients continued using them after 6 months. Another commercially available option includes the button prism, also known as Visual Field Awareness System or Gottlieb lens.²⁰ A small round prism is bonded into a hole a patient’s spectacles and when gazing in the prism patients become aware of a larger field. However, this LVA also causes shifting of an image and not true expansion of the visual field. Prism glasses have not gained widespread acceptance and are difficult to use partly due to image jump caused by the prism. Apfelbaum and Peli have noted that an ideal LVA for PFL would result in field expansion in primary gaze without obstructing central vision.²¹

Physical visual field expansion can be achieved with reversed Galilean telescopes and amorphous lenses. A reversed telescope minifies an image to increase the amount of visual information being presented to a given area of healthy retina. Mehr and Quillman found that reversed telescopes increased the visual field area from 5 to 14 degrees in a patient with retinitis pigmentosa (RP).²² Using a commercially available HMD, our team reported the case of a patient with RP secondary to Usher’s syndrome, who demonstrated a significant increase in planimetric perimetry area from 203 degrees² to 1804 degrees² representing an 8.9 fold increase in visual field area.⁴ In contrast, amorphous lenses compress an image in the horizontal plane in order to present more peripherally located information. Szlyk et al found that they resulted in improved scanning, tracking and peripheral detection in patients with PFL.²³ However, like prismatic LVAs, reverse telescopes and amorphous lenses may be poorly tolerated by some patients.²¹

Occupational therapists have developed visual skills training to assist patients with compensating for peripheral visual field loss.²⁴ After mapping the visual field, patients are asked to shift their head and eyes to attend to the area of field loss while participating in activities. Occupational therapists may also assess patients’ home environments and provide interventions to enhance lighting and contrast, improve navigation around the home. Similar environmental-based interventions may be used to promote safe driving. Additional interventions such as orientation and mobility training are available to patients with severe PFL.

For individuals with low vision there is a great need to develop LVAs that overcome some of the shortcomings of existing devices, and this is especially true for those with PFL. Building on the functions of existing LVAs such as magnification, minification, image shift and illumination, future devices may offer the ability to dynamically alter the magnitude of these functions and to personalize functionality to meet the changing needs of individual patients. Though considerably more research is needed to understand the role of HMD in low vision and vision enhancement, HMD represents a promising class of technologies for a growing patient population who have the need for vision enhancement.

Head Mounted Display Technologies

HMD are a type of electronic visual aid that attach to the user’s head and present information directly to the user’s eyes. This is in contrast to heads-up displays, which project information onto a stationary surface in the subject’s line of sight. There are several methods for classifying HMD technology,^25,26 based on i) how visual information is conveyed; ii) usability; and iii) optical design. These classifications are summarized below and important terms are defined in Table 1.

Table 1.

A Lexicon for Head-Mounted Display Technology in Low Vision Rehabilitation and Vision Enhancement.

Term	Definition	Examples	Advantages for Vision Applications
Electronic visual aids	Any device that provides a digital image to improve visual performance	HMD, closed-circuit television	To provide any digital alteration that improves visual performance such as magnification to improve visual acuity or minification to expand visual fields
Heads-up display	See-through display projected in user’s line of sight that does not move with user; not a form of HMD	Automotive and aviation displays, industrial applications	Limited applications
Augmented reality	Presentation of information to the visual system that does not otherwise exist in the user’s environment	Contour video images	Presentation of visual cues missing from a user’s field of vision; information on potential obstacles in user’s path
Head-mounted display	An electronic visual aid that is worn on the head, often like a pair of glasses or goggles	See various types below	Same functions as electronic visual aids but with improved ergonomics and ease of use since worn on the user’s head
Types of HMD:
Virtual reality	HMD that covers the eyes, occupying the entire visual field	eSight, Oculus Rift	Enhancement of central vision or night vision through image processing
Near-eye display	HMD that projects a see-through image in front of the eye	Epson Moverio, Microsoft Hololens	Expansion of perceived visual field
Retinal projection	HMD that directly project a see-through image onto the user’s retina	Fujitsu Laser Headset, Google Glass	Expansion of perceived visual field
Optical Design
Non-pupil forming display	Display mounted in front of a user’s eyes amplifies image using simple lenses	Epson Moverio, Microsoft Hololens	Easier to design and fabricate than pupil forming displays
Pupil forming display	Use complex sets of lenses to that the image source (camera) can be moved away from the eyes	Fujitsu Laser Headset, Google Glass	Improved ergonomics for retinal projection devices compared to non-pupil forming displays

HMD: head-mounted display

HMD can be classified into three categories based on how visual information is conveyed. First, monocular displays present images to one eye only. These are the simplest to implement and this type of system has been used as a LVA.^4,5 Second, biocular displays show the same image to both eyes. This solution has been employed in military applications to improve peripheral detection²⁶ and in low vision,²⁷ however the design requirements are more complex. Third, binocular displays also have strict design requirements, show independent images to each eye and more reliably result in stereo images.

HMD can be classified based on usability into virtual reality (VR), also known as immersive reality, devices and see-through displays. Immersive or virtual displays eliminate the direct path between the user’s eyes and their real environment. VR devices have been traditionally used for entertainment applications, in which users are immersed in a simulated world. VR devices typically employ binocular displays that simulate a three-dimensional environment. The main advantage of immersive displays is their relatively large field of view (>110 degrees). VR devices have been used in LVR^16,27 to present a modified or enhanced image of the real environment that is easier to discern by the patient. An ongoing RCT will test the effectiveness of eSight, a VR device, on health-related quality of life and functioning in patients with predominantly CVL.¹⁶

See-through displays superimpose images directly on top of the user’s field of view, but without occluding it; they can be monocular, biocular or binocular. The most common use of see-through displays is augmented reality (AR) applications, in which information is presented to the user that does not otherwise exist in the environment. Commonly, see-through AR devices are used in industrial applications, for example providing a worker with real-time assembly instructions. There are two types of see-through devices, near-eye displays and retinal projections. Near-eye displays project a see-through display just in front of the eye, while retinal projections use a complex set of lenses to project an image onto the user’s retina. See-through displays have the advantage of allowing the user to retain his or her habitual vision while benefitting from the added information of the display. Additionally, if a see-through display malfunctions or depletes its power source, a user is able to function with his or her normal vision. The main drawback to these devices is their relatively small effective field of view (<30 degrees). Several groups of investigators have reported on the use of see-through displays to expand constricted peripheral vision^4,6,9 and to improve central contrast.⁵

Finally, HMD can be classified based on their optical design as pupil and non-pupil forming systems. In a non-pupil forming design a display is mounted in front of a user’s eyes and the images are amplified using simple lenses. Near-eye displays are an example of a non-pupil forming system. These devices are relatively easy to design and fabricate, but have the disadvantage of adding extra weight over the eyes. In contrast, pupil forming systems are more complex and use additional sets of lenses to move the image source (e.g. the projector) away from the eyes. Retinal projections are an example of a pupil forming display. The image source in these systems can be located in a more convenient and ergonomically acceptable location for better balancing of the headset.

Optics and Human Factors Considerations in Head Mounted Display Technology

Given the large number and types of HMD it is beyond the scope of this article to discuss the optical considerations of all HMDs, however Cakmakci has provided a comprehensive review of optical issues pertinent to HMD.²⁵ Since HMD is designed to work in conjunction with the human eye, we describe the visual system parameters that are universally important and should be accounted for in the development and evaluation of HMD,²⁸ including: (a) eye dimensions and geometry; (b) field of view; (c) luminance; (d) resolution; (e) color; (f) stereopsis; and (h) head motion. Of note, most optical values for the human eye, upon which design of HMD are based, are derived from normal healthy eyes and will likely vary for individuals with visual impairment.

a. Eye Dimensions and Geometry

A critical consideration in HMD design is the size and location of the pupil. The size of the normal pupil varies from about 2 mm to 8 mm depending on luminance. Smaller pupils give rise to a larger depth of field and the eye is near diffraction limited for a pupil size of 2 mm. Additionally, the pupil constantly moves with the eye and has a maximum rotational angle of about ± 15 degrees in the horizontal plane and +30 to −35 degrees in the vertical plane. Finally, there are differences in the interpupillary distance and the temple to eye distance between individuals. Each of these features of will impact the eye box of the HMD, which is the distance a user’s eye can move in both dimensions without “loosing” the display. For example, a smaller pupil will result in a smaller eye box and this should be considered in the design and choice of HMD for specific applications.

b. Field of View

The field of view (FOV) of a healthy eye is roughly 140 degrees in the horizontal plane and 110 degrees in the vertical plane; if the gaze is directed forward, binocular FOV is roughly 190 degrees. When using HMD, FOV is directly related to content, location of the display, physical constraints of the device, and the type of HMD (e.g. immersive or see-through display). Display dimensions and where the FOV is positioned in the user’s available visual space can induce asthenopia. Several useful technologies exist that may be employed in the design of HMDs to address some of these issues. First, scanners or switchable optics can be used to enlarge FOV in real-time without sacrificing resolution. Second, optical combiner technologies can relocate the entire FOV, or fracture the available FOV into different locations, also in real-time. Use of these features should be tailored to the visual needs and goals of patients being treated.

c. Illumination

The eye is not uniformly sensitive to all combinations of contrast and spatial frequencies.²⁹ Contrast sensitivity is affected by ocular pathology and by glare, intraocular scatter, and absorption by intraocular media such as cataract. A modest binocular reduction in contrast sensitivity may occur with HMD by eliminating the benefits of binocular summation. When multiple images are presented to the eye, the maximum luminance difference between images should be 25% and ideally this difference should be less than 10%. The spectral sensitivity curve for both photopic and scotopic vision should also be considered. As the luminance increases, the pupil size is reduced and the depth of field increases. Therefore, both the luminance level and target resolution should be accounted for when designing HMD in order to attain a specific depth of field. Depth of field should be set appropriately so that the user can properly perceive both the computer generated imagery and real objects lying at a similar depth.

d. Resolution

The resolving power of a healthy fovea is about 1 arc minute based on the Rayleigh criterion. If the image on an HMD display is of low resolution it may cause fatigue and/or asthenopia. However, if the user moves at a speed of approximately 2–4 degrees/second, resolution can decrease by a factor of V without negative consequences. Additionally, as a result of peripheral aberrations and neural factors (e.g., multiplexing of photoreceptors to ganglion cells or larger spatial summation areas), there is a decrease in resolution in the periphery even in healthy eyes and this results in a trade-off between resolution and FOV.³⁰

e. Color

No available display is able to exhibit the entire range of colors that the human eye can perceive under photopic conditions.³¹ Additionally, when FOV extends beyond 50 degrees the eye detects little or no color and therefore information is reduced when HMD projects information to these areas. Since see-through displays add to the user’s visual information rather than replacing it, they may have less of an adverse effect on color perception compared to IR displays and this may be important for patients with visual impairment who can have already reduced color vision.

f. Stereopsis

Depth perception is an important factor the design and use of HMDs. If the two eyes receive different stimulation this will result in binocular rivalry in which the eye detecting the higher display luminance will prevail. With binocular vision three parameters play a major role in stereopsis: occlusion, convergence and accommodation, and vergence eye movements.³²

Occlusion is the most important depth cue for objects beyond about 2–3 meters and for moving objects. Convergence should match accommodation to within ± 0.5 diopters and a high or low convergence to accommodation ratio can result in asthenopia, diplopia or excessive image blur. However, multi-focal displays can be used to resolve a covergence-accomodation conflict. Finally, vergence movements are essential for binocularly viewing objects closer than about 1 meter and play an important role in depth perception. Depending upon the specific use of HMD and the specific visual impairment(s) of its users, emphasis could be placed on one or more of these depth cues when designing and choosing HMD for various applications in low vision.

g. Head Motion

Head movement is used for tracking a moving object and, combined with eye movement, results in the vestibular ocular reflex. Normal habitual head movements are about ± 45 degrees and the rotational velocities of the eye may be as great as 500 degrees per second.³³ If the HMD is heavy, it can slow head rotations and even decrease range of motion. Tracking and targeting of images can also be affected by heavy HMD since it may result in a lag between image display and initiation of head movements. A lag of less than 16 msec between image display and initiation of head movement is recommended to overcome these issues.

(h) User Interface

An additional factor to be considered in the design of HMD is the user interface.³⁴ These can be grouped into: voice input, trackpad input, eye/head sensing, gaze tracking, hand sensing, and other miscellaneous technologies. Since patients with visual impairment have unique needs compared to typical recreational or industrial HMD users, qualitative work should be undertaken to better understand what user interfaces are preferred for controlling features such as turning the device on or off, changing the location or size of the display, and activating ancillary features of the HMD’s computer processing unit.

The Future of HMD in Low Vision

HMD technology represents a new approach to address visual impairment. Application of HMD technologies is an opportunity to extend the benefits of LVAs and LVR to patients who have unique functional needs that may not be adequately addressed by existing treatment options. For example, one-quarter of patients seeking low vision services have primarily peripheral vision loss (PVL) or combined peripheral and central vision loss,¹¹ which is not adequately addressed by existing LVAs and may result in a unique set of impairments¹⁷ including decreased mobility and an increased risk of falls.³⁵

Since existing LVAs for patients with PVL may reduce visual acuity or cause image jump, patients have reportedly rejected these devices.²¹ However, newer HMDs, such as near-eye displays and retinal projections, overcome these barriers by allowing patients to view a minified image in the periphery while maintaining their habitual central vision. Consequently, HMD may represent a breakthrough for individuals with PVL. Future work should examine the impact of HMD using outcome measures designed to assess the functional impairments and rehabilitation goals of patients with PVL and other forms of vision impairment that are not adequately addressed by existing LVAs and rehabilitation strategies.

The applications of HMD in eye care extend beyond the rehabilitation of PVL and may include addressing central vision loss, loss of contrast sensitivity, nyctalopia, and spatial neglect, among others. As commercially available technologies improve their displays, software capabilities, human factors considerations, and cosmetic appearance, the use of HMD among patients with visual impairment is likely to become more commonplace. A common lexicon and an understanding of the various types of HMD will permit clinicians and vision researchers to describe and evaluate these technologies and to keep pace with this rapidly evolving field.