Predicting 60–4 Visual Field Tests Using 3D Facial Reconstruction

Abstract

Background

Despite, the potential clinical utility of 60–4 visual fields, they are not frequently used in clinical practice partly due to the purported impact of facial contour on field defects. The purpose of this study was to design and test an artificial intelligence-driven platform to predict facial structure-dependent visual field defects on 60–4 visual field tests.

Methods

Subjects with no ocular pathology were included. Participants were subject to optical coherence tomography, 60–4 Swedish interactive thresholding algorithm visual field tests, and photography. The predicted visual field was compared to observed 60–4 visual field results in subjects. Average and point-specific sensitivity, specificity, precision, negative predictive value, accuracy, and F1-scores were primary outcome measures.

Results

30 healthy were enrolled. Three-dimensional facial reconstruction using a convolution neural network was able to predict facial contour-dependent 60–4 visual field defects in 30 subjects without ocular pathology. Overall model accuracy was 97±3% and 96±3% and the F1-score, dependent on precision and sensitivity, was 58±19% and 55±15% for the right eye and left eye, respectively. Spatial-dependent model performance was observed with increased sensitivity and precision within the far inferior nasal field reflected by an average F1-score of 76±20% and 70±29% for the right eye and left eye, respectively.

Conclusions

This pilot study reports the development of a CNN-enhanced platform capable of predicting 60–4 visual field defects in healthy controls based on facial contour. Further study with this platform may enhance understanding of the influence of facial contour on 60–4 visual field testing.

PRECIS

The effect of facial contour on 60–4 visual field defects has not been elucidated. In this study, a convolution neural network-augmented platform allowed for prediction of 60–4 field defects due to facial contour.

Introduction

Advances in life expectancy have in part provoked a disproportionate increase in the aging population [1]. This demographic redistribution has shifted the global disease burden towards chronic, noncommunicable diseases, which now impose significant mortality and morbidity burden [1, 2]. In this context, the burden of vision impairment is rising [2, 3].

Visual field testing allows for functional assessment of vision [4–10]. The visual field can be assessed with 10–2, 24–2, 30–2, and 60–4 testing patterns, which vary in the degree of deviation from the central axis measured and the number of testing points considered. Notably, central vision can be evaluated with 10–2, 24–2, and 30–2 field patterns; however, peripheral vision beyond 30 degrees of the central visual field axis is measured with a 60–4 visual field [4, 11–13]. Defects in both the central visual and peripheral visual fields can hinder quality of life [14, 15]. Appropriate utilization of 60–4 visual field testing is limited by wide variability and unclear appropriate thresholds in healthy control subjects, due to differences in point sensitivity and possibly the influence of facial contour [16–18]. Central visual field testing is employed more commonly for tracking glaucoma progression [16]. Yet, visual field testing beyond the central 30 degrees from the fixation point holds value for other ocular pathologies including drug toxicity and may provide a refined understanding of visual field limitations in glaucoma patients [16, 17]. Case in point, in early stages of glaucoma, central and peripheral visual field loss are not correlated as peripheral defects may manifest in the absence of central field defects. In fact, 11–17% of patients with glaucoma may have peripheral visual field defects in the absence of central visual field defects [17]. Thus, utilization of 60–4 visual fields may enhance structural loss and functional deficit concordance and provide important information for ocular pathologies [19]. Additionally, aging-associated visual field decline predominantly affects the far peripheral visual field [11]. Moreover, central, and peripheral visual defects have independent clinical value, with peripheral defects increasing fall risk and alterations in balance [4, 11, 12, 20–22].

However, there remains uncertainty whether facial contour can impact peripheral visual field results when utilizing a 60–4 testing pattern. It has been suggested that the impact of facial structure on 60–4 field defects may complicate identification of pathological peripheral field defects [17, 18]. Appropriate use of 60–4 visual field tests would require distinguishing facial contour-dependent defects from pathology-related defects. As, artificial intelligence modalities are increasingly utilized in visual field related applications [23–27], our study aims to develop a method to predict facial contour dependent far peripheral visual field defects in normal controls using a convolution neural network (CNN)-engineered platform. Additionally, we aim to compare predicted 60–4 facial contour-dependent visual field defects to obtained defects in control participants.

Methods

Enrollment

This study adhered to the tenets of the Declaration of Helsinki and was approved by Mayo Clinic’s Institutional Review Board. Written informed consent was obtained from all participants. Normal subjects without ocular pathology were prospectively and consecutively enrolled at a single tertiary referral center. Optical coherence tomography in conjunction with clinical exam was used to confirm the absence of ocular pathology for recruitment of healthy controls.

Optical Coherence Tomography

Cirrus OCT using a 200 × 200 optic disc cube protocol was used for measuring peripapillary retinal nerve fiber layer thickness.

Visual Field Testing

Visual fields were carried out on a Humphrey Field Analyzer II (Carl Zeiss Meditec, Inc., Dublin, CA, USA) using a 60–4 Swedish interactive thresholding algorithm (SITA) standard test. To ensure reliable functional assessment, subjects were asked to repeat the visual field test if false negatives or false positives rates were >10%.

2D Photos

2D facial images were taken from 1 meter away under standard fluorescent lighting using a D100 Nikon with 28–105 zoom lens. Zoom was set to 105 with program mode exposure and flash turned on. Photos were saved as high-resolution JPEG images (ISO =400).

3D Reconstruction

2D pictures were used for 3D reconstruction of an individual’s face. In this method, 2D images are resized to 256*256 pixels. From the 2D image, X and Y coordinates are placed into a UV position map, a 2D image representation of the 3D positions. It is necessary to encode a third dimension for depth. A pretrained convolution neural network (CNN) is used to correspond each RGB value to a depth [28]. This allows conversion of 2D images into a single 3D representation of facial contour. Thus, each point on the UV map can be expressed as position (u_i,v_i) = (x_i, y_i, z_i). U_i and V_i represent the X and Y coordinates for any given point, denoted as i, in the 2D UV position map. The RGB values provide depth information (Figure 1A). This 3D representation is used to calculate an angle of intercept between the visual axis and the face following visualization in Blender (version 2.93.4), a 3D graphics software.

Figure 1.

A. A 2D image was projected onto a UV map with X_i,Y_i coordinates for each pixel. RGB values were used by a convolution neural network for predicting Z_i, which represents the predicted depth. B. Angle θ corresponding to the angle of intersection between the vector from a point on the face to the pupil and the visual axis was calculated using a unit vector parallel to the z axis. C. The angle θ was calculated for all points circumferential to the visual axis. D. All points where θ<60° were recorded. E. Angle α, the angle between the vector connecting a point on the face to the pupil and unit vector parallel to the x axis was calculated for each point where θ<60°. F. Angle θ and α were used to map the predicted field defect onto a visual field chart.

Calculating the Angle Theta

Blender (version 2.93.4) [29], an open-source 3D graphics software able to implicitly run python scripts, was used for visualization of the 3D reconstructed facial images from the UV position map; thus, providing a facial reconstruction model with 3D coordinates for each point originally represented in the UV map. From the reconstruction model, the angle between the visual axis and all circumferential points on the 3D facial model were calculated using the trigonometric unit circle; this angle was termed theta (Figure 1B and C). Specifically, the point corresponding to the visual axis was chosen manually and assumed to be the center of the pupil. Since the angles formed from the intersection of a transverse line with two parallel lines are equal and the visual axis is parallel to the z-axis (depth coordinate in UV position map), the unit vector (0i+0j+1k) parallel to the z-axis could be considered the visual axis (Figure 1B).

For each vector intersecting points on the reconstructed facial contour and the visual axis, the angle theta was calculated using a python script and the following equation (Figure 1C): $\cos (\text{θ})=\frac{a\cdot b}{\parallel \text{a}\parallel \cdot \parallel \text{b}\parallel }$. where a is the unit vector parallel to the visual axis, b is the vector connecting a point on the pupil (p₁) to any point (p_i) on the face, and θ is the angle of intersection between them. Thus, the vector b will be the difference of p_i (x_i, y_i, z_i) located anywhere on the face, and p₁ (x₁,y₁,z₁), located on the pupil, b = (x_i− x₁, y_i− y₁, z_i− z₁), p_i−p₁. Substituting the 3D coordinates into the equation in place of the vectors yields the equation:

$$\cos \theta =\frac{\left(0\cdot \left(x_i-x_1\right)\right)+\left(0\cdot \left(y_i-y_1\right)\right)+\left(1\cdot \left(z_i-z_1\right)\right)}{\sqrt{({\left(x_i-x_1\right)}^2+{\left(y_i-y_1\right)}^2+{\left(z_i-z_1\right)}^2}\cdot \sqrt{{(0)}^2+{(0)}^2+{(1)}^2)}}.$$

The python script was used to find all points in which θ <60 (Figure 1D and E), as a θ <60 degrees was predicted to correspond to a visual field defect in the 60–4 field pattern.

Predicting 60–4 Visual Field Defects

For points (p_i) where θ<60, a second angle alpha (α) was calculated to allow for mapping of the predicting visual defect onto the visual field (Figure 1C). α corresponds to the angle between vector b and unit vector parallel to the x axis (c). Angle θ and α are exported together into a comma-separated values (CSV) file which is then used for plotting the points onto a visual field chart (Figure 1F).

Compensating for Facial Contour-dependent Defects

A python program is written to use the bitwise-and function of OpenCV [30], an image processing tool, for superimposing the predicted visual field defects due to facial contour (opacity=0.5) on the obtained visual field chart (Figure 2A).

Figure 2.

A. Visual field defects detected using a 60–4 Swedish interactive thresholding algorithm and predicted visual field defects are shown for a subset of healthy participants. B. The total number of field defects observed at each location are reported. C. The prevalence of a field defect at each location is reported. OD=right eye, OS=left eye.

Threshold Calculation

Single threshold values for each eye were calculated as the sum of threshold values at each point.

Intrapoint Variability

Points measured twice by the 60–4 visual field test were assessed for intrapoint variability. Previous studies have shown that areas with reduced retinal sensitivity exhibit elevated variability [31, 32].

Performance Metrics

The CNN-augmented platform presented here provides a binary output for each point assessed by the 60–4 visual field test. Thus, performance was assessed using binary classification parameters derived from the confusion matrix [33]. Given variability in peripheral threshold tests, standardized normal values are not readily available for 60–4 visual field tests [16]. Considering the association between reduced field sensitivity and increased variability [32], true positives (TP) were defined as points located in the predicted field defect with >3 dB intrapoint variability or <10 dB threshold sensitivity. True negatives (TN) were defined as points outside the predicted field defect with intrapoint variability ≤3 dB and ≥10 dB threshold sensitivity. False positives (FP) were defined as points with intrapoint variability ≤3 dB in the predicted field defect. Points with >3 dB variability or <10 dB threshold sensitivity outside the predicted visual field defect were defined as false negatives (FN). Average performance metrics were obtained by averaging each metric across the right (OD) and left (OS) eye of the participants. For average performance metrics, the standard deviation was provided unless otherwise noted.

To investigate the spatial dependence of the platform, performance metrics for each point assessed by the 60–4 visual field were calculated by summation of TPs, TNs, FPs, and FNs at each tested point and visualized using the ggplot2 package in R (v4.2.0).

Sensitivity (recall) was defined as: $\frac{TP}{TP+FN}$

Specificity was defined as: $\frac{TN}{TN+FP}$

Precision was defined as: $\frac{TP}{TP+FP}$

Negative predictive value (NPV) was defined as: $\frac{TN}{TN+FN}$

Model accuracy was defined as: $\frac{TP+TN}{TP+TN+FP+FN}\ast 100\text{\%}$.

The F-1 score, reflective of the harmonic mean of the precision and recall, was defined as: $\frac{2\ast precision\ast recall}{precision+recall}$

Results

Patient Characteristics

This pilot study enrolled a total of 30 healthy subjects (9 males, 21 females) aged 24 to 58 years old (mean: 39 years). The duration of the 60–4 Humphrey visual field tests ranged from 6:11 to 11:52 (mean: 8:45). Total threshold values ranged from 1420–2048 dB (mean, 1719 dB). Retinal nerve fiber layer (RFNL) thickness ranged from 71–115 μM with an average of 95.6 μM for all eyes. Demographic data is provided in Table 1.

Table 1.

Patient Characteristics

Gender	N(%)
Male	9 (30%)
Female	21 (70%)
Age (years)	Mean (range)
	39 (24,58)
Eye	N
Right (OD)	30
Left (OS)	30
Race	N(%)
White	14(47%)
Asian	15(50%)
African American	1(3%)
Visual Acuity (logMAR)	Mean
OD	0.003
OS	−0.020
OU	−0.008
RNFL Thickness (μm)	Mean
OD	96.8
OS	94.5
OU	95.6
Visual Field Testing Time	Mean (range)
	8:45 (6:11, 11:52)
Threshold (dB)	Mean
OD	1724
OS	1713
OU	1719

dB=decibels, OD=right, OS=left, RNFL= Retinal nerve fiber layer

Platform Application

A CNN algorithm used 2D images to create a UV position map that was then visualized as a 3D facial reconstruction (Figure 1A). From the 3D facial reconstruction, a python script was used to identify all points where θ, the angle of intersection between the vector from a point on the face to the pupil and the visual axis, was <60°. These points were then mapped onto the visual field chart as predicted visual field defects (Figure 1F). Visual field test results using a 60–4 SITA were recorded (Figure 2A). Predicted field defects were then superimposed on observed visual field defects (Figure 2A).

Assessment of Platform Performance

Many defects were observed in the nasal field of both eyes (Figure 2B) with a particularly high probability of a field defect in the inferior far nasal field beyond 42 degrees (Figure 2C). The average accuracy in predicting the presence or absence of a field defect in the 60–4 visual field was 97±3% and 96±3% for the right eye (OD) and left eye (OS), respectively. Average sensitivity was 51±29% and 47±26% and average precision was 61±30% and 58±27% for the OD and OS, respectively. Thus, the average F1-score, dependent on precision and sensitivity, was 58±19% and 55±15% On the other hand, average specificity was 97±3% and 96±3% and the average negative predictive value was 95±4% and 94±5% for the OD and OS, respectively.

A strong spatial dependence was observed for sensitivity and precision in both eyes with high sensitivity and precision in the far inferior nasal visual field, primarily beyond 42 degrees from the central axis. (Figure 3A and B). In general, the specificity and the negative predictive value (NPV) of the platform was consistent across points assessed by the 60–4 visual field test (Figure 3C and D). Overall prediction accuracy was relatively consistent across all tested points; however, the F1-score exhibited a strong spatial dependence paralleling the distribution observed with model sensitivity and precision (Figure 3E and F).

Figure 3.

Performance metrics for each point on the 60–4 visual field are shown. A. Sensitivity was much higher in the far inferior nasal field. B. Precision exhibited spatial dependent variability. C. Specificity was high throughout the visual field. D. The negative predictive value was also high across most points assessed by the 60–4 visual field. E. Location-specific accuracy paralleled the performance of specificity and negative predictive value. F. The location specific F-1 score, dependent on sensitivity and precision, paralleled the distribution observed with sensitivity and precision. Regions designated not applicable (N/A) were not predicted as defects by the platform, precluding the calculation of sensitivity or precision. Red arrows highlight locations in the far inferior nasal field with augmented sensitivity and precision.

Almost 40% of all observed defects were in the inferior nasal field, at least 42 degrees from the central axis (Figure 3A, red arrows). The artificial intelligence augmented platform exhibited distinct performance in this region. The average sensitivity of the platform within this region was 98±4% and 86±19% and the average precision was 66±26% and 63±34% for the OD and OS, respectively. The average specificity at this region was lower at 51±19% and 48±36% and the average negative predictive value was 90±20% and 78±34% for the OD and OS, respectively. The average F1-score was 76±20% and 70±29% and average accuracy was 77±20% and 68±15% for the OD and OS, respectively.

Discussion

While 60–4 visual field testing patterns may be useful for detecting early peripheral field loss in various ocular diseases, clinical adoption has been hindered by variable testing results partly due to variation in retinal sensitivity and possibly from obstruction by facial features [16]. Our goal was to illustrate the feasibility of using facial structure to predict visual field defects, develop a method to accurately identify visual field defects related to facial contour, and demonstrate the potential to correct the visual field for these facial contour-induced visual field defects. Ultimately this method could be used to separate visual field defects related to facial contour from defects related to ocular pathology.

Accurately assessing peripheral (30–60°) visual field loss has broad clinical implications given the superior association of peripheral visual field with self-reported mobility function [21] and diverse etiologies of peripheral visual defects including retinitis pigmentosa, diabetic retinopathy, optic neuropathies due to injuries, or toxicity from medications. Notably, medications including corticosteroids, antibiotics, antineoplastic and antiarrhythmics have been shown to exhibit ocular toxicity, highlighting the broad importance of proper assessment of peripheral visual defects across medical specialties [16, 34–37]. In consideration of the preferential localization of age-associated visual deterioration in the peripheral visual field, appropriate assessment of the peripheral visual field can have important quality of life consequences as the population ages [1, 11].

We here developed an augmented platform to predict 60–4 field defects using 2D images of the face collected using a standard camera. Using this CNN-enhanced predictive platform, it was possible to predict visual field defects due to facial contour in 30 healthy controls. Considering the high variability of threshold values with the 60–4 visual field test, relying on standard threshold values was not possible [16]. Rather the association between threshold variability and reduced sensitivity, allowed identification of likely field defects [31, 32]. Overall, the model exhibited high accuracy due to its ability to accurately predict areas without field defects as confirmed by the overall high specificity and negative predictive value of the model. The overall F1-score which is the harmonic mean of the positive predictive value and sensitivity, both dependent on correct detection of field defects, was much lower. The cases where the platform was not able to accurately identify field defects could be due to an inaccuracy of the platform or due to patient specific differences in retinal sensitivity in the peripheral visual field [16], as this platform is designed to only to detect facial contour dependent defects. Thus, many of the observed defects throughout the 60–4 visual field may be facial-contour independent resulting in an apparent reduced global sensitivity. Thus, the scarcity of information regarding the peripheral visual field poses a limitation for this study but also highlights the strength of this pilot study.

In line with previous studies, a large proportion of defects detected in the 60–4 visual field were within the inferior nasal field [11, 16]. Within, this region the test exhibited particularly high sensitivity and precision and thus a higher F1-score. However, the platform had a lower specificity and reduced negative predictive value within this region. Consideration of the spatial dependent performance of this platform has important pragmatic implications. For instance, if the platform predicts the absence of facial dependent defects outside the inferior nasal field and they are present on 60–4 visual field testing, this could be suggestive of a pathological field defect such as certain drug toxicities which often manifest outside the inferior nasal field [37]. Conversely, if the platform predicts a defect in the outer inferior nasal field, and one is detected with the 60–4 visual field test, this is likely a facial contour dependent. This is of particular relevance to the aging population, as a large proportion of age-associated field defects were located within this region [11].

Development of a predictive platform for 60–4 visual field testing must be paired with acknowledgement of other process which may the influence the presence of field defects within the periphery including retinal sensitivity and pathological processes including drug toxicity. The platform herein discussed provides a foundational step towards the development of clinically applicable tools. Firstly, while the influence, of facial contour on field defects in 60–4 visual field testing have previously been assumed, the high sensitivity of this platform in the far inferior nasal field suggests the presence of facial-dependent defects at least within this region of the visual field. Secondly, this platform serves as a proof of concept to inform future advances in improving the clinical applicability of the 60–4 visual field test, which if properly employed may have important effects on quality of life.

Conclusion

This pilot study reported development of a CNN-enhanced platform for predicting 60–4 visual field defects in healthy controls based on facial contour.

KEY MESSAGES.

What is already Known

The influence of facial contour on observed 60–4 visual field defects has only been suggested.

What this study adds

The present study reports on a convolution neural network-augmented platform capable of predicting facial contour-related defects in 60–4 visual field testing. Thus, this study suggests the presence of facial contour-dependent defects on 60–4 visual field testing and reports on a proof-of-concept strategy capable of predicting 60–4 field defects using 3D facial reconstruction.

How this study might affect research, practice, or policy

Identifying facial contour-dependent defects in 60–4 visual field testing can help identify pathological field defects associated with various ocular pathologies including drug toxicity. Additionally, a deeper understanding of 60–4 visual field testing may help expand the clinical applicability of 60–4 visual field testing.