The Assessment of Visual Function and Functional Vision

Abstract

The complete assessment of vision-related abilities should consider visual function (the performance of components of the visual system) and functional vision (visual task-related ability). Assessment methods are highly dependent upon individual characteristics (e.g. the presence and type of visual impairment). Typical visual function tests assess factors such as visual acuity, contrast sensitivity, color, depth, and motion perception. These properties each represent an aspect of visual function and may impact an individual’s level of functional vision. The goal of any functional vision assessment should be to measure the visual task-related ability under real-world scenarios. Recent technological advancements such as virtual reality (VR) can provide new opportunities to improve traditional vision assessments by providing novel objective and ecologically valid measurements of performance, and allowing for the investigation of their neural basis. In this review, visual function and functional vision evaluation approaches are discussed in the context of traditional and novel acquisition methods.

Introduction

The complete assessment of an individual’s vision-related abilities requires the consideration and characterization of both visual function and functional vision, particularly in the case of visual impairment¹. Visual function describes how well the eyes and basic visual system can detect a target stimulus. By varying a single parameter at a time (for example, the size of the target), testing is typically carried out in a repeated fashion under controlled testing conditions until a threshold of performance is obtained. In contrast, functional vision refers to how well an individual performs while interacting with the visual environment. That is to say, how their vision is used in everyday activities. Characterizing functional vision involves the assessment of multiple and varying parameters captured under complex, real-life conditions. In this instance, how well an individual is able to sustain performance is a crucial factor¹. The concepts of visual function and functional vision are certainly linked and assessing one can often provide useful information regarding the other. For example, if an individual shows evidence of impaired visual functioning (e.g. reduced acuity), one may predict potential impairments with certain visual tasks (e.g. reading) and possible strategies to help remediate the situation (such as magnification or large print). Observing a patient’s functional visual behaviors in the environment (e.g. trouble descending a flight of stairs) can signal which test of visual function should be carried out in the formal clinical setting (e.g. contrast sensitivity, visual field perimetry), or help identify modifications needed to ensure accurate assessment.

It is important to recognize however, that there are situations where an individual’s visual function may be assessed as “within normal range”, yet their functional vision is still clearly impaired. This disconnect is often evident in the setting of evaluating visual performance in pediatric and adolescent populations with early developmental brain damage, particularly in the case of brain based visual impairment such as cerebral (or cortical) visual impairment (CVI)². If comprehensive assessments and careful consideration of both visual function and functional vision are not carried out, individuals with visual cognitive and higher order perceptual impairments may not be able to secure needed services if determinations are based on visual function measures alone such as visual acuity³. Furthermore, the necessity to accurately characterize visual function and functional vision performance is crucial in order to generate an appropriate management plan that best suits the needs and developmental goals of an individual⁴.

In this review, we present a summary of behavioral tests employed for assessing visual function and functional vision, including traditional techniques as well as innovative approaches incorporating virtual reality (VR) environments and eye tracking methodologies. The potential utility of VR technology combined with modern brain imaging techniques for the purposes of identifying the neural correlates related to visual performance will also be discussed.

Behavioral Assessment of Visual Function

The visual system is comprised of multiple interdependent structures and pathways that are highly functionally specialized and follow a general hierarchical organization^5,6. This specialization begins as early as the retina, where light sensitive photoreceptor and ganglion cells selectively respond to different spatial and temporal properties of light. Connecting the eyes (via the optic nerves) to subcortical structures (e.g. the lateral geniculate nucleus), visual information passes to the visual cortex, where multiple specialized brain areas preferentially process different features of the captured image. The functioning of these structures and pathways may be selectively affected across the lifespan of an individual as a result of different developmental trajectories, degeneration, and vulnerabilities^7–9. The complex organization of the visual system means that in order to characterize visual performance in a comprehensive manner, multiple tests must be carried out to assess the functioning of different visual structures and pathways. Practical testing limitations related to instrumentation and the demands placed on a participant, coupled with challenges associated with testing unique clinical populations, has led to an evolution and refinement of the assessment methods employed. In the first section of this review, we consider behavioral methods and test designs that have been developed to measure what we consider pillars of visual function. These visual function outcomes include: visual acuity, contrast sensitivity, color, depth, and motion. Note that visual fields (measured by formal perimetry testing) are also critically important in the assessment of visual function. However, a discussion of visual field function is beyond the scope of this review and the reader is directed to a number of excellent reviews on this topic for further discussion^10,11.

1. Visual Acuity

Visual acuity estimates the level of finest detail that can be detected or identified, and remains a very important measure of visual function in both clinical evaluation and research. The classical method of measuring acuity in subjects who are able to report what they perceive is with acuity charts (such as the familiar Snellen acuity chart; see figure 1 left) composed of high contrast black targets (i.e. optotypes such as letters) presented on a white background. By convention, acuity is reported in units relative to a visually healthy observer’s performance at 6 meters (approximately 20 feet). At this distance, normal visual acuity is reported as 6/6 (or 20/20) indicating the subject can resolve optotypes that subtend 5 arcmin on the viewer’s retina with lines or gaps that subtend 1 arcmin. This translates to a minimum angle of resolution (MAR) of 1.0, the logarithm of which (logMAR) is 0.0. All of these reporting standards are used interchangeably. According to the World Health Organization¹², moderate visual impairment (or moderate low vision) is defined as best corrected visual acuity in the best seeing eye worse than 6/18, severe visual impairment is acuity worse than 6/60, and profound visual impairment (or blindness) is defined with presenting visual acuity worse than 3/60¹³.

Figure 1.

Example charts and approaches used in measuring visual acuity. (Left) Classic Snellen letter chart. (Right) Early Treatment of Diabetic Retinopathy Study (ETDRS) chart. Note how this chart uses 5 letters per line with decreasing size and with equal logarithmic spacing of the rows and letters.

Many types of letter acuity charts have been designed for the purposes of testing visual acuity (for review see¹⁴). The Early Treatment of Diabetic Retinopathy Study (ETDRS) chart¹⁵ has emerged as the method of choice for visual acuity testing (see figure 1 right). This chart has a log-scaled layout with 5 letters per line as the apposed to the standard layout of the Snellen chart. Typically, a subset of 10 letters from the English alphabet are used and approximately matched for difficulty¹⁶, to discourage guessing and improve test precision¹⁷. For participants who do not know the English alphabet, a subset of letters, numbers, symbols of differing orientation (e.g. Landolt Cs or Tumbling Es), or pictograms¹⁸ may be employed, allowing the subject to name the symbols or match the target to a template. For subjects who cannot perform matching or identification tasks (e.g. infants), responses may be collected from eye movement responses, such as Preferential Looking¹⁹, pupillary responses, optokinetic nystagmus reflex, or electroencephalography based recordings such as a visual evoked potential (VEP)²⁰. It is worth noting that these alternative tests may however, speak more about detection rather than resolution ability with regard to acuity. Nonetheless, they can all be important tools for the early detection of visual acuity impairments in infants.

A range of heuristics have been developed for determining scoring and termination criteria for letter charts which can lead to different acuity specification²¹. Nevertheless, there is fairly good agreement in acuity estimates with different methods, and many studies have shown that best-corrected visual acuity increases monotonically from birth up to around 10 years of age (for review see²²) and decreases monotonically after around the age of 60 (for review see²³).

2. Contrast Sensitivity

While visual acuity measures the smallest target size that can be identified, testing is generally carried out using full black targets presented on a white high contrast background. However, the natural environment is composed of objects at multiple sizes and intensities. Differences in image intensity are quantified by contrast (typically the difference between the lightest and darkest features in an image divided by the mean intensity), and differences in size are quantified by spatial frequency (the reciprocal of the retinal distance between light or dark image regions in degrees of visual angle) (see figure 2). Contrast sensitivity is the reciprocal of the smallest luminance difference required for target identification, and is highly dependent on spatial frequency²⁴. This relationship between spatial frequency and contrast sensitivity is called the Contrast Sensitivity Function (CSF). It is important to note that contrast sensitivity has been shown to be a better measure than visual acuity in terms of predicting performance on activities of daily living²⁵ and the detection of real objects²⁶.

Figure 2:

Example Contrast Sensitivity Function. Contrast increases from top to bottom and spatial frequency increases from left to right most observers perceive an inverted U shape that separates visible from non-visible letters, illustrated by the dashed line. Adapted from²⁴).

Contrast sensitivity is impaired in many common clinical conditions even when visual acuity is found to be within normal limits. The assessment of contrast sensitivity is therefore an important complement to visual acuity²⁷. Contrast sensitivity can be assessed for a single target size with charts composed of letters of decreasing contrast (e.g.^28,29) with similar scoring principles used for acuity charts. Alternatively, contrast sensitivity can also be measured with sine wave grating patches presented at several spatial frequencies (e.g. Vistech and Vectorvision charts). From a clinical standpoint, it is important to realize that a given type of visual impairment can selectively affect the visibility of only a specific range of spatial frequencies. For example, the detection of high spatial frequencies is impaired in uncorrected refractive error³⁰ and amblyopia³¹. Medium spatial frequencies are selectively impaired in Parkinson’s Disease³². Finally, cataracts have been shown to selectively impair the detection of low spatial frequencies³³. This means that the comprehensive assessment of contrast sensitivity requires measurement at several spatial frequencies in order to detect any potential deficits related to both detection and resolution ability (for review see³⁴).

3. Color

Both acuity and contrast sensitivity measure the limits of perception for black, white, and greyscale stimuli. However, color is also an important signal in the natural environment and can help facilitate object recognition³⁵. Human color vision is referred to as trichromatic, meaning it depends on three classes of cone photoreceptors in the retina. These color sensitive cones have overlapping spectral sensitivities peaking at long (L), medium (M), and short (S) wavelengths corresponding to red, green, and blue cones respectively, based on the color appearance of light at each wavelength. Color selective responses have been reported in many cortical areas, including primary visual cortex (V1) and area V2, but responses in area V4 have been shown to be correlated with color appearance (for review see³⁶). If genes coding for photosensitive pigments are defective, this can lead to the impaired perception of color referred to as anomalous trichromacy (protanomaly, deuteranomaly or tritanomaly depending if the L, M, or S cone pigments are affected, respectively). Detailed color vision assessment can be carried out using an anomaloscope³⁷, or color chip arrangement³⁸ tasks. However, these tests are typically very time consuming and demanding for the participant. Alternative screening tests (e.g. Ishihara) employ pseudoisochromatic stimuli (chromatic targets presented in luminance noise patterns that mask non-chromatic cues to the target’s identity³⁹). These tests have good agreement for screening if a color perception anomaly is present, but limited agreement on quantifying the severity of impairment⁴⁰. For individuals that cannot provide verbal responses (such as infants), preferential looking methods, color detection⁴¹, and color preference⁴² approaches can be used. In general, the ability to discriminate colors is present in the first few months after birth, and matures to adult levels over the first few years (for review see⁴³).

4. Depth

Acuity, contrast sensitivity, and color tests all measure the limits of perception of stimuli presented in two dimensions. However, the relative position of objects in depth is also a critical cue for interacting with the environment. Depth information is available from monocular and binocular cues. Monocular cues may be exogenous (e.g. occlusion, shadow, size, texture; which are approximately assessed by acuity and contrast sensitivity), or endogenous (e.g. accommodation; which can be estimated from pupillary responses, see⁴⁴). The interocular separation of human eyes and vergence eye movements that direct the fovea of each eye to an object moving closer or farther away, generate two important endogenous binocular depth cues. The kinesthetic responses of the extraocular muscles produce convergence cues and differences in the retinal positions of objects imaged in each eye produce stereoscopic disparity cues. Binocular depth perception therefore critically depends on eye movement control, which develops into adolescence⁴⁵. Since stereoscopic disparity requires a comparison between responses from both eyes, depth disparity processing begins in the visual cortex. Visually responsive neurons that are selective for disparity are found in all visual areas of the brain⁴⁶, although some have argued for specialization of area V3 of primate brains⁴⁷.

Sensitivity to binocular disparity is generally assessed with random dot stereogram tests that present slightly differing images to each eye (for review see⁴⁸). The observer’s task is to report an embedded pattern that is defined only by binocular disparity, and cannot be detected monocularly. As with acuity testing, binocular disparity is decreased over several stereogram targets so as to find the smallest disparity the observer is able to detect; termed stereoacuity. Stereoacuity increases with age from around 100 arcsec at age 3, and reaches adult levels of around 40 arcsec at around age 7⁴⁹.

5. Motion

All the aforementioned assessments measure the limits of perception for static images. However, there is continuous motion of the images captured on the retina that is generated by relative movements between the observer and surrounding objects. Safe interaction and mobility in the environment critically depend on our ability to detect motion. Selectivity for temporal modulation (e.g. a flickering stimulus), begins at the level of the retina⁵⁰. However, direction selectivity first appears in area V1⁵¹. Direction selective visual neurons project to the middle temporal gyrus (area MT/V5); which is specialized for motion processing over large areas of visual space⁵².

As with acuity, contrast sensitivity, color and depth, motion assessment may be measured with perceptual report methods, or with observational methods such as preferential looking and pupillometry. An alternative observational method that is particularly suited to motion perception exploits optokinetic nystagmus (OKN) responses to moving stimuli (for review see⁵³). OKN is an involuntary oculomotor reflex response to moving stimuli while maintaining a steady gaze, composed of slow pursuit eye movements interspersed with fast saccadic eye movements that produces a “sawtooth” pattern of eye positions over time. This reflex is very difficult to suppress and the patterns are elicited only if the stimulus is processed by the visual system. The OKN response can be used to measure visual acuity based on electrooculogram⁵⁴ and contrast sensitivity⁵⁵. The OKN is present in the first few months (for review see⁵⁶).

Sensitivity to motion has been measured with a broad range of moving images, including drifting sine wave gratings and preferential looking methods. It has been shown that infants as young as 1 month can detect moving gratings⁵⁷. However, in a sine grating stimulus, the direction of motion is the same across all areas. Therefore, to study the integration of motion signals, global motion coherence patterns are generally used⁵⁸. Motion coherence stimuli are composed of a number of independently moving dots. A proportion of the dots move in a single direction, while the remainder move in random directions (noise) (see figure 3 for example). The observer’s task is to report the overall direction of motion signal (e.g. left or right). This task requires the observer to combine estimates of multiple dot directions across space. Since observing the motion of individual dots are uninformative, these patterns are therefore thought to test the function of higher order stages of motion processing and integration. Reduced motion sensitivity could lead to real-world deficits of visual information integration, impacting daily life tasks (e.g., navigation)⁵⁹. Testing results can be used to generate a psychometric function (see section below), which can be constructed from the proportion of correct responses at a range of signal levels to determine an individual’s motion coherence threshold. Sensitivity to motion coherence is present in infants as early as 24 weeks⁶⁰ and continues to develop through to adolescence⁵⁹.

Figure 3.

Illustration of a random dot kinetogram (RDK). In this pattern, two different types of dot motion are used. Signal dots move coherently in a given direction while noise dots move randomly.

Psychometric Functions and Thresholds

A cornerstone of behavioral assessment concerns the relationship between stimulus intensity and the subject’s perceptual report. At high signal levels (e.g. large or high contrast letters), an observer can easily perceive a target and correctly answer questions about it (e.g. its identity, location, or orientation). At low signal levels, the observer may not perceive anything and would have to guess if forced to give a response. Note that guesses will sometimes be correct, and at a level equal to the reciprocal of the number of choices. For this reason, a large number of choices (e.g. the letters of the English alphabet, where guessing rate is 1/26) are more efficient than a small number (e.g. left or right position, where guessing rate is 1/2). These tasks are called “N Alternative Forced Choice” or AFC, where N indicates the number of possible responses (e.g. 26 AFC for letters and 2 AFC for left/right preferential looking). A psychometric function describes the profile of response data and a threshold can be estimated at a criterion performance level that is usually halfway between guessing rate and 100% correct. Given that thresholds vary across individuals and over time, the range spanning guessing to 100% correct is not known before the test begins. The problems of selecting the optimal signal range and the number of trials to present at each signal level have been addressed by adaptive computer algorithms that change the test stimulus each trial, based on the observer’s responses to previous stimuli (for review see⁶¹). These methods have reduced the number of trials required to estimate a threshold to around 30⁶². This threshold is extremely informative because small differences in visual function (e.g. between groups of subjects or following disease progression or therapy treatment) may change performance near threshold without affecting performance at high signal levels (which are still easy) or low signal levels (which still elicit a guess).

Behavioral Assessment of Functional Vision

As mentioned in the previous section, behavioral methods to assess important measures of visual function can be very useful, and many are used routinely in clinical practice. However, it is important to note that assessing visual function does not definitively inform us as to how an individual uses their vision in everyday activities. Furthermore, clinical tests of visual function may fail in detecting the full array of possible visual perceptual deficits, and characterize the broad heterogeneity of performance levels in individuals with visual impairment (either ocular- or brain-based). The potential reasons for this include first, assessments of visual function are typically performed in the clinical setting under optimal testing conditions (e.g. good lighting and minimal environmental clutter). Second, traditional assessments (e.g. acuity and color) do not typically characterize the complex higher-order visual processes involved with the analysis of a dynamic and complex visual scene. Finally, visual performance can vary based on task demands and environmental complexity³. These issues are not meant to question the value nor the validity of traditional visual function assessments, but rather highlight that importance of developing more adaptive means of testing in order to fully characterize the visual performance of an individual^4,63–65. Moreover, while certain tests may be useful for assessing and characterizing vision in individuals with ocular based conditions, they may be less informative in the case of brain based visual impairment such as in CVI. This is particularly true when considering that many individuals with CVI may show normal or near-normal performance on assessments of acuity, color, and contrast, yet these same individuals will often describe a variety of visual perceptual deficits such as difficulties with visuo-spatial/motion processing, environmental complexity/crowding, and sustaining attention during tasks with high visual demands^64,66–68. Furthermore, because individuals with CVI may present with co-occurring delays in motor, cognitive, and/or other sensory functions, traditional methods of visual testing remain challenging to administer and may not necessarily be appropriate for every child.

The Need for New Techniques

The impact of varying visual task demands and environmental parameters may lead to variable performance levels. For example, differences in visual processing performance between ocular and cerebral based visual impairment have been previously well characterized^69–71. A review of pertinent literature in the case of CVI includes characteristic behaviors such as preference for dynamic versus static objects, interactions between sensory modalities (e.g. turning away when reaching), and variability of visual attention depending upon the familiarity or complexity of the surrounding environment and/or stimuli. Additionally, deficits in visuospatial processing can lead to reduced functional performance^66,67. The presence of behavioral and etiological differences between CVI and ocular-based visual impairments means that the testing parameters themselves may influence performance and, consequently, the conclusions drawn from their results. As mentioned previously, although traditional clinical assessments may adequately characterize an individual’s visual function (e.g. acuity and contrast sensitivity), it may fail to capture deficits in functional vision that manifest in real-world settings and situations. Therefore, there is an urgent need to develop novel methods of testing that approach more realistic scenarios (referred to as “ecological validity”) that will complement traditional clinical assessments. To this end, virtual reality (VR) may serve as an ideal platform for characterizing visual performance in an ecologically valid manner. The technologies required for VR (such high power computers, rendering of high quality graphics, and eye-tracking systems) have become increasingly cost efficient, readily available, and can now be more easily incorporated for the purposes of testing visual performance⁷². Thus, there is the opportunity to leverage the benefits of VR as a useful tool for the assessment of functional vision in individuals with visual impairment such as CVI.

VR has long since been implemented in clinical and behavioral neuroscience applications and research⁷³. Indeed, VR has been used to not only assess and test attention, memory, and decision making, but also as a clinical rehabilitation tool for stroke recovery and cognitive rehabilitation in dementia, Attention Deficit Hyperactivity Disorder (ADHD), and Post Traumatic Stress Disorder (PTSD)^73–75. VR-based approaches have also been used for training and rehabilitation in various pediatric populations, including individuals with ADHD, autism, and cerebral palsy (See⁷⁶ for review). There are numerous benefits to using VR as a tool for training and performance assessment, including task flexibility, reproducibility, experimental control, objective data capture, participant engagement and motivation, participant safety, and the ability to mimic real-world scenarios with a high level of ecological validity^75,77–79. With these potential benefits in mind, we have attempted to leverage the benefits of VR to develop ecologically valid testing platforms to characterize and objectively assess visual perceptual difficulties reported in individuals with visual impairments with a particular focus on CVI.

Virtual Reality Paradigms

A focus group study (consisting of parents as well as teachers and clinicians working with children with CVI) was carried out to identify key aspects of functional vision assessment that should be incorporated in the design of our VR tasks. This revealed that individuals with CVI often demonstrate difficulties associated with static and dynamic visual search such as identification of familiar objects or persons in a complex visual environment. The two most commonly described scenarios identified in the focus group study were: challenges with finding a favorite toy in a toy box, and locating a specific person in a crowd. Based on this information, two corresponding VR-based simulations were designed and developed. Furthermore, design features were incorporated that enabled the manipulation of important factors of visual environmental complexity (e.g. the number of distractor elements present and level of environmental clutter). Following development, further focus group evaluation confirmed the realism, engagement, and relevance of the final VR-based simulations were rated as very high while prompting additional manipulations of interest that were to be incorporated in subsequent design iterations^80,81.

Two VR scenarios were developed to evaluate the ability to perform static and dynamic visual search. Respectively; these were referred to as the Virtual Toy Box and the Virtual Hallway. While the general premise behind both simulations is to find a specific target (i.e. a particular toy or person) among surrounding distractor elements, each employs a unique task paradigm. Considering the Virtual Hallway first, this task is presented in a dynamic and continuous manner where the participant chooses the person they wish to search for (in this case, the target is a principal of a fictitious school), and then views a video-like scenario of a crowded hallway where the number of people walking and presence of a target person vary smoothly over time (see⁸⁰ for a complete explanation of the methodological details). The Virtual Toy Box is presented in a static trial-by-trial fashion where participants choose the toy they want to search for (i.e. the participant chooses between a yellow duck, an orange basketball, or a blue truck as their target of interest). During the task, a participant has 4 seconds to locate that toy before all the objects are covered and shuffled (see⁸¹ for a complete explanation of the methodological details). Figure 4 displays a photo of the experimental set up (A) and screenshots from both VR tasks (B and C). An eye tracking system is employed that continuously monitors and collects eye gaze (search pattern) data during the task. Participants are instructed to visually search for and fixate on the chosen target once it’s found. Importantly, because responses are based solely on eye-tracking data, the ability to assess performance on the task is not dependent on verbal responses.

Figure 4.

Photo and screenshots from VR Simulations. (A) photo of participant viewing the Virtual Hallway task. The eye-tracking unit (mounted to monitor display) is highlighted. (B) screenshot of the Virtual Toy Box task. In this example, the participant must find the target toy (blue truck, circle) amongst the high number of surrounding distractor elements. (C) screenshot of the Virtual Hallway. In this task, the participant must find the target (principal, circle) walking amongst a high number of distractors (other individuals).

Additional elements have also been built in to assess real-world interaction with the scenarios. For instance, hand tracking can be incorporated into the Virtual Toy Box to mimic the experience of reaching for the toy as well as visually searching for it. Additionally, the perspective in the Virtual Hallway can be presented in a manner in order to simulate walking down the hallway while searching for the target. Compensatory and adaptive strategies can also be tested using the virtual simulations. For example, the clothes that the principal wears in the Virtual Hallway can be changed to colors of high contrast compared to the rest of the hallway scene to test the effect of changing target saliency on visual search performance. Similarly, cues to help target identification (e.g. motion such as spinning) can be added to the target toy in the Virtual Toy Box to help draw and focus attention. Overall, these VR-based approaches create a highly flexible, immersive, and realistic platform for testing and collecting objective performance data, as well as monitoring performance over time.

Eye tracking data is acquired at a sampling rate of 90 Hz, ensuring that the precise position of the participant’s eyes on the screen with respect to the target is logged throughout the task. A heat map based on the raw coordinate gaze data is then generated to visualize the overall distribution pattern of eye movements⁸². The color within the heat map represents the level of gaze data density across spatial regions of the screen, with each color corresponding to a ratio of point density^80,81. Using this scheme, warm colors indicate that a participant spent a large portion of time looking in a given area, and cool colors indicate areas where a participant spent less time. Representative data for the Virtual Hallway task obtained from a neuro-typically developed control, an individual with ocular visual impairment (OVI; ocular albinism), and an individual with CVI are shown and compared in Figure 5. For the control subject, the overall distribution of the heat map pattern appears tight around the target principal in the center of the image (note that the data is centered across runs)⁸⁰. In contrast, the individual with OVI shows a much larger spread in the overall pattern, while it is somewhat intermediate in the individual with CVI. We can also assess how performance changes by increasing the number of surrounding distractor elements (i.e. other individuals walking in the scene). Note that with increasing distractors, the control and OVI subjects show similar distributions of eye movements compared to their respective low distractor condition performance. This suggests that for both control and OVI participants, increasing the number of distractor elements did not greatly influence the overall distribution pattern of visual search. Interestingly, in the CVI participant, the overall extent of the pattern is markedly increased with the presence of increased distractor elements. This increase in the overall distribution of the heat map pattern in the individual with CVI can be interpreted as a broader search strategy employed in order to find the target in the presence of multiple distractors. A similar overall pattern of performance is observed in the case of the Virtual Toy Box shown in figure 6. Again, a robust clustering of data around the target toy (centered data) is observed in both low and high distractor conditions for the control individual. The OVI participant reflects an overall baseline increase in eye gaze scatter across both the low and high distractor number conditions. Similar to what was observed in the Virtual Hallway, the distribution of the heat map pattern in the individual with CVI is in between that of the control and OVI participant for the low distractor condition, but increases greatly for the high distractor condition. Again, this is suggestive that in the individual with CVI, a broader search pattern was employed to perform a thorough visual search when the environment requires higher visual demand. These observations are in agreement with previous clinical reports that individuals with CVI tend to exhibit decreased visual performance in the setting of increased visual environmental complexity^69,70.

Figure 5.

Heat map displays of eye search patterns for the Virtual Hallway. Data from a control, OVI, and CVI participant are shown for both the low and high number of distractor conditions. Note how the search pattern in the CVI participant is markedly more diffuse in response to the high distractor condition. Adapted from⁸⁰.

Figure 6.

Heat map displays of eye search patterns for the Virtual Toy Box. Data from a control, OVI, and CVI participant are shown for both the low and high number of unique distractor conditions. Note that the search pattern of the CVI participant shows greater spread in response to the high unique distractor condition. Adapted from⁸¹.

Additional elements of eye tracking behavior can also be examined to provide a more complete analysis of visual search performance. Gaze error (i.e., the distance between the target and participant’s eyes on the screen) can be measured to quantify fixation/pursuit eye movements during the search task^83,84. It is also possible to calculate two reaction time metrics for each trial. A “hit” is measured as the first moment the participant’s eye gaze reaches the target on the screen and “fixate” is the first time that the participant’s eyes arrives and fixates onto the target^80,81. Finally, because the position of the participant’s eyes on the screen is constantly recorded, it is possible to quantify the frequency and duration that the participant looks away from the screen for each trial. This provides an index measure of test reliability. Each of these metrics can then be analyzed across different condition levels and manipulation factors.

In summary, VR simulations like the Virtual Toy Box and Hallway have been designed and developed to help assess functional vision performance in settings that approach real world situations. The use of eye-tracking data capture ensures an objective and accurate measure of visual performance, even in individuals who cannot verbalize their responses. Given the flexibility and engaging nature of these VR based evironments, these scenarios can be readily modified on an individual basis for montoring performance as a result of a training program, or probing the effect of potential environmental modifications and compenstary strategies. Finally, these same VR based paltforms can be coupled with brain imaging techniques such as functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) to investigate the neural corraletes associated with visual task performance. Preliminary results employing these same VR based approaches will be discussed in the following section.

Virtual Reality and Functional Imaging

Combining behavioral assessments and task-based functional neuroimaging can be useful for characterizing brain processes related to visual perception such as those associated with navigation^85,86. In combination with VR, multiple types of neuroimaging techniques can be implemented, each with their own comparative strengths and weakness. Functional MRI (fMRI) provides good spatial resolution information (on the order of millimeters) and is particularly well-suited for identifying areas of localized brain activity associated with performing a behavioral task. On the other hand, the temporal resolution of fMRI is relatively poor and is on the order of seconds⁸⁷, making fMRI less optimal for characterizing the timing of brain responses to a behavioral task. In contrast to fMRI, electroencephalography (EEG) affords high temporal resolution (on the order of milliseconds) and is thus useful in characterizing the temporal profile of task related brain activity. However, EEG has comparatively poor spatial resolution (on the order of centimeters) as compared to fMRI⁸⁸, and is therefore less optimal for the purposes of identifying a specific area of localized activity. Leveraging the assets of a given neuroimaging technique, combined with an appropriately designed behavioral task paradigm, defines and strengthens the interpretation of results that can be obtained.

In order to determine the patterns of cortical activation associated with performing dynamic search task, the Virtual Hallway was adapted as part of an fMRI paradigm. In a pilot study, patterns of activation (i.e., blood oxygenation level-dependent (BOLD) signal) were obtained in a neuro-typically developed control and individual with CVI with a particular focus on characterizing activity within visual cortical areas known to be implicated with complex motion, spatial processing, and object recognition^89–91. In the control subject (Figure 7 left panel), visual search of the dynamic visual scene was associated with robust activation within a confluence of occipital visual cortical areas as well as within the intraparietal sulcus (IPS); the latter area playing an important role in perceptual-motor coordination (e.g., directing eye movements) and visual attention^92–94. In contrast, the overall pattern of occipital activation observed in the individual with CVI was not as robust, including in the IPS (Figure 7 right panel). These differences in the overall magnitude and extent of activation (as characterized by fMRI) within cortical areas known to be implicated in visual spatial processing may be related to observed deficits in visual search performance associated with CVI.

Figure 7.

fMRI activation patterns resulting from viewing the Virtual Hallway task in a control participant (left) and individual with CVI (right) (left hemisphere and lateral view is shown). Robust activation is seen within a confluence of occipital visual areas (arrow) and intra parietal sulcus (IPS) in the control. Overall occipital area (arrow) and IPS activation is less robust for the individual with CVI as compared to the control.

A second neuroimaging study was designed using EEG combined with the Virtual Toy Box environment to further assess variations in the timing of visual cortical responses during static visual search. Electrodes placed on the surface of the scalp record the electrical activity of underling cortical neurons as the participant views the toy presented at varying locations on the screen. Cortical activation patterns are measured using event-related potentials (ERPs; see⁸⁷ for review), which represent an average response signal from repeated trials of the task and at different time points during the presentation of the stimulus. In the control participant, (figure 8, top row) cortical activation centered around the occipital and parietal areas is strongest around the 350 to 375 ms temporal window after the visual stimulus onset and remains sustained through 400 ms. In comparison, the temporal pattern of cortical responses in the individual with CVI shows an overall reduction in signal magnitude (figure 8, bottom row) appearing within the same temporal window (350 to 375 ms), as observed in the control subject. At 400 ms after the visual stimulus onset, note how the occipital signal does not appear as robustly sustained as it is in the control participant. Further quantification of the signal fluctuations can be done by analyzing the magnitude and timing of ERP peaks as well as further distinguishing the various frequency bands of each channel. As with the results obtained from the fMRI experiment, observed differences in patterns of cortical activity may help uncover the neurophysiological basis of visual perceptual deficits associated with CVI as well as other visual impairment conditions.

Figure 8.

Scalp map plots of 20 channel EEG data obtained from viewing the Virtual Toy Box task in a control and individual with CVI (posterior view, right side). Scalp maps are displayed in 25 ms intervals from 300 ms to 400 ms. The occipital-parietal signal observed in the control (top) appears robust and peaks between 350 and 375 ms. The individual with CVI (bottom) shows an overall reduction in signal with the peak being at a similar time to the control. Note further how the occipital signal does not appear as robustly sustained as it is in the control participant (at 400 ms).

Concluding Remarks

In this review, we discussed how a comprehensive evaluation of an individual’s overall visual abilities includes assessments of both visual function and functional vision performance. Established techniques commonly used in the clinical setting and novel VR-based approaches each have their own strengths in characterizing an individual’s visual abilities and deficits. Incorporating functional neuroimaging methodologies such as with the VR-based environments described here can help provide further insight into the underlying neural correlates and neurophysiological mechanisms associated with impairments in visual processing, such as in the case of CVI. The combination of all these techniques can provide a holistic understanding regarding an individual’s visual abilities.