Superior Visual Field Testing Using Virtual Reality with and without Eye-Tracking for Functional Upper Eyelid Surgery Evaluation: A Pilot Study

Abstract

Purpose:

To assess sensitivity and specificity of superior visual field tests administered on virtual reality (VR) with eye-tracking (VR-ET) and without eye-tracking (VR⁰) for fulfillment of insurance coverage criteria for functional upper eyelid surgery as compared to standard automated perimetry (SAP).

Methods:

This prospective cross-sectional study included 78 eyes from 41 patients with ptosis, brow ptosis, and dermatochalasis undergoing functional upper eyelid surgery evaluation. Participants underwent serial superior visual field tests using SAP and VR⁰ or VR-ET in randomized order. Fulfillment of insurance coverage criteria for blepharoplasty was defined as a 30% increase in the grid seen from the untaped to the taped state. The main outcome measure was sensitivity and specificity of VR⁰, VR-ET, and overall VR in meeting insurance coverage criteria as compared to SAP.

Results:

VR had a sensitivity of 84.1% and specificity of 67.6%, with no significant difference between VR⁰ and VR-ET. SAP agreed on insurance coverage criteria fulfillment with VR⁰ in 28 (71.8%) eyes and with VR-ET in 32 (82.1%) eyes. Insurance coverage criteria fulfillment rates varied significantly by diagnosis on SAP (p=0.012) but not VR (p=0.059).

Conclusions:

VR may be an alternative to SAP for functional upper eyelid surgery evaluation. Future studies are needed to determine differences in patient satisfaction, testing and waiting time, and test-retest reliability between VR and SAP.

Précis

A virtual reality perimeter, with or without eye-tracking capabilities, was found to be an acceptable alternative to standard automated perimetry for testing superior visual field obstruction during evaluation for functional upper eyelid surgery.

Introduction

Visual field testing documents objective evidence of superior visual field impairment to fulfill insurance coverage criteria for functional upper eyelid blepharoplasty, blepharoptosis, and brow ptosis surgeries. Medicare standards mandate documentation of visual field exams demonstrating loss of the superior field that improves by at least 30% when eyelid surgery is simulated with tape.¹ A survey of members of the American Society of Ophthalmic Plastic and Reconstructive Surgery demonstrated that the most commonly used preoperative visual field testing method was the Superior-64 program on the Humphrey Field Analyzer (HFA), a type of standard automated perimetry (SAP).² However, using SAP in a busy multi-subspecialty clinic could create a bottleneck, slowing clinic flow and prolonging patient wait times.³ Additionally, SAP machines can be expensive and require a technician and dedicated room to administer the test, consuming resources and personnel that could be utilized elsewhere in the clinic.⁴ More affordable and less technician-dependent alternatives compared to traditional equipment can help alleviate challenges faced by some clinics facing time and budgetary constraints.

Virtual reality (VR) visual field testing is a novel and nascent technology whose utility in assessing visual fields is still being delineated. Previously, patients have demonstrated a preference for and acceptance of VR over standard methods.^5,6 However, the validity of superior visual fields obtained through VR has yet to be established. Validation of reliability must occur if VR is to gain widespread acceptance and to ensure that patient care and access to insurance coverage are not compromised. Eye-tracking capabilities within VR devices can ensure continued fixation and, by extension, increase test reliability when blind spot testing is unavailable. Comparison of VR visual fields with and without eye-tracking for superior visual field testing has not been previously considered. Such an assessment can enable providers to make informed judgments on which devices to utilize in evaluating patients for upper eyelid surgery.

This pilot study aimed to assess a new technology for efficacy in detecting superior visual field deficits by measuring sensitivity and specificity of superior visual field tests administered on VR with or without eye-tracking in fulfilling insurance coverage criteria for functional upper eyelid surgery against SAP. We also evaluated rates of insurance coverage criteria fulfillment on SAP and VR by diagnosis.

Methods

This study was approved by the Institutional Review Board at the University of Miami, complied with the Health Insurance Portability and Accountability Act, and adhered to the tenets of the Declaration of Helsinki. Written informed consent was obtained from all participants.

Study enrollment occurred between October 2021 and May 2022. Consecutive patients of any age undergoing functional upper eyelid surgery evaluation with a diagnosis of ptosis, brow ptosis, or dermatochalasis at a tertiary care center were approached to participate in the study. Ptosis was defined as MRD1 less than or equal to 2 mm; brow ptosis was defined as any part of the brow at or below the level of the superior orbital rim; and dermatochalasis was defined as redundant eyelid skin hooding over the eyelid margin. Both eyes were enrolled if the patient was affected bilaterally. Exclusion criteria included the presence of other ocular pathology that could affect the visual field, including, but not limited to, orbital masses, glaucoma, retinal detachment, thyroid eye disease, history of stroke, or brain tumor.

In this study, 78 eyes from 41 patients were enrolled. Patients underwent superior visual field testing on SAP (Zeiss Humphrey Field Analyzer 3, Dublin, CA, USA) and a commercially available VR device (Virtual Vision Health, Miami, FL, USA). The VR headset was loaned to the investigators by the device company. Patients were randomized using computer-generated random numbers as to which of the two testing modalities they would complete first. Superior-64 testing on SAP was conducted by a trained technician who provided verbal instructions before initiating the exam. The exam was completed once with the eyelids in their natural state and once with the eyelids taped up by the technician to simulate the effects of surgery. In dermatochalasis patients, the excess skin fold was taped to the brow. In brow ptosis patients, the brow was suspended by tape to the forehead. In ptosis patients, skin just above the lid margin was taped and used to lift the upper eyelid margin so that it rested at the superior limbus. For each exam, the right eye was tested before the left.

A comparable superior visual field test was administered on VR by study personnel. Instructions were provided verbally by study personnel and via verbal prompts in the VR device as part of the pre-test tutorial. Participants enrolled before January 1, 2022, were tested using a VR device without eye-tracking capabilities (VR⁰), while participants enrolled after January 1, 2022, were tested using a VR device with eye-tracking enabled (VR-ET, Figure 1). Thus, patients underwent one of four possible testing orders: SAP-then-VR⁰, VR⁰-then-SAP, SAP-then-VR-ET, or VR-ET-then-SAP. VR⁰ displays stimuli in the superior visual field and instructs patients to indicate with a handheld button when stimuli are seen while focusing on an inferior fixation point. VR-ET employs the same concept while using infrared cameras to track pupil movement (Figure 2). When the eye-tracker registers the movement of the pupil off the fixation point, the test is paused, and the VR device provides an auditory reminder to the patient to maintain fixation on the target. Patients were instructed to wear corrective lenses within the device if applicable. As with the SAP testing protocol, participants using the VR testing methods initially completed exams with their eyelids in their natural state, followed by another exam with the eyelid or eyebrow pathology corrected with tape. Eyelid and brow taping for dermatochalasis, brow ptosis, and ptosis were done in a similar fashion to the SAP testing. Again, the right eye was tested before the left.

Figure 1:

Virtual reality device with eye-tracking used for superior visual field testing

Figure 2:

Sample superior visual field report using virtual reality with eye-tracking

The percentage of “seen” grid points out of 64 total points was recorded for each exam. The difference between the portion of the grid seen with and without taping was calculated for each eye. Success in qualifying for insurance coverage criteria was defined as greater than or equal to 30% improvement from the untaped to the taped exam.¹ The main outcome measure assessed was sensitivity and specificity of VR⁰, VR-ET, and VR as a whole, as compared to SAP. Agreement between VR and SAP was also assessed, where agreement was defined as meeting criteria on both modalities or failing to meet criteria on both modalities. Medical charts were reviewed to collect patient age, gender, diagnosis, and medical history.

For patients where both eyes were enrolled, each eye was treated as a separate observation for analysis, as has been done in other studies investigating superior visual field testing methods.^7,8 Statistical analysis was completed using SPSS Statistics v27 (IBM, Armonk, New York). Rates of insurance coverage criteria fulfillment and agreement between VR and SAP by diagnosis, order of test administration, and eye laterality were assessed using a chi-squared test. A comparison of ages between groups was accomplished using an independent samples t-test.

Results

SAP and VR were administered to 78 eyes (VR⁰: 39 eyes, VR-ET: 39 eyes), characteristics of which are displayed in Table 1. All participants were able to complete the SAP and VR tests in full. 40 eyes (51.3%) were randomized to complete VR before SAP, and the remaining underwent SAP followed by VR. Rates of criteria fulfillment and sensitivities and specificities of VR⁰, VR-ET, and overall VR are shown in Table 2. The overlapping 95% confidence intervals of VR⁰ and VR-ET sensitivities and VR⁰ and VR-ET specificities demonstrate the sensitivity and specificity of VR⁰ and VR-ET are not significantly different. SAP showed agreement in fulfilling insurance coverage criteria with VR⁰ in 28 (71.8%) eyes and VR-ET in 32 (82.1%) eyes. Overall, VR and SAP showed agreement in 60 (76.9%) eyes.

Table 1:

Characteristics of eyes tested using VR without and with eye-tracking

Parameter	VR without eye-tracking	VR with eye-tracking
OD	19 (48.7%)	20 (51.3%)
OS	20 (51.3%)	19 (48.7%)
Male	9 (23.1%)	8 (20.5%)
Female	30 (76.9%)	31 (79.5%)
Age (mean±SD)	65.59±10.48	58.64±13.28
Ptosis only diagnosis	11(28.2%)	10 (25.6%
Brow ptosis only diagnosis	0 (0.0%)	0 (0.0%)
Dermatochalasis only diagnosis	14 (35.9%)	12 (30.8%)
Ptosis and brow ptosis diagnosis	2 (5.1%)	2 (5.1%)
Ptosis and dermatochalasis diagnosis	8 (20.5%)	3 (7.7%)
Brow ptosis and dermatochalasis diagnosis	0 (0.0%)	1 (2.6%)
Ptosis and brow ptosis and dermatochalasis diagnosis	0 (0.0%)	1 (2.6%)

^aVR: virtual reality

^bData presented as n (%) where applicable

Table 2:

Rates of Insurance Criteria Fulfillment on VR and SAP

Criteria met on VR		Criteria met on SAP		Sensitivity, 95% CI	Specificity, 95% CI
		Yes	No
VR without eye-tracking	Yes	14 (35.9%)	8 (20.5%)	82.4%, (56.6%, 96.2%)	63.6%, (40.7%, 82.8%)
	No	3 (7.7%)	14 (35.9%)
VR with eye-tracking	Yes	23 (59.0%)	3 (7.7%)	85.2%, (66.3% 95.8%)	75.0%, (42.8%, 94.5%)
	No	4 (10.30%)	9 (23.10%)
Overall VR	Yes	37 (47.4%)	11 (14.1%)	84.1%, (69.9%, 93.4%)	67.6%, (49.5%, 82.6%)
	No	7 (9.0%)	23 (29.5%)

^aVR: virtual reality; SAP: standard automated perimeter, CI: confidence interval

The distribution of insurance coverage criteria fulfillment on VR and SAP by diagnosis of ptosis or dermatochalasis is displayed in Table 3. Distribution of insurance coverage criteria fulfillment rates differed significantly by diagnosis for visual field tests administered on SAP but not for tests administered on VR, and rates of agreement between VR and SAP did not differ by diagnosis. For both VR and SAP, participants diagnosed with dermatochalasis had a higher rate of meeting criteria than participants with ptosis. There was no statistically significant difference in age between eyes demonstrating agreement and eyes failing to show agreement for insurance coverage fulfillment for VR⁰ or VR-ET (p=0.752 and p=0.093, respectively). There was also no significant difference in agreement between VR and SAP based on test administration order or eye laterality for VR⁰ and VR-ET (p>0.05 for all).

Table 3:

Distribution of Insurance Criteria Fulfillment by Diagnosis

Diagnosis	n	Criteria met on VR0 or VR-ET		Criteria met on SAP		Agreement
		n (%)	pb	n (%)	pb	n (%)	pb
Ptosis	25	11 (44.0%)	0.059	8 (32.0%)	0.012	20 (80.0%)	0.906
Dermatochalasis	41	30 (73.2%)		28 (68.3%)		31 (75.6%)
Ptosis & Dermatochalasis	12	7 (58.3%)		8 (66.7%)		9 (75.0%)
Total	78	48 (61.5%)		44 (56.4%)		60 (76.9%)

^aVR⁰: virtual reality without eye-tracking; VR-ET: virtual reality with eye-tracking; SAP: standard automated perimeter

^bChi-Square test of independence, significance at the 0.05 level

^cParticipants with brow ptosis were included in this table but were not listed separately because brow ptosis always occurred in conjunction with ptosis or dermatochalasis in our cohort.

Of the 11 eyes that disagreed with insurance coverage criteria fulfillment between SAP and VR⁰, 8 (72.7%) met the criteria on VR⁰ but not SAP (Table 2). Of the 7 eyes with disagreement in insurance coverage criteria fulfillment between SAP and VR-ET, 3 (42.9%) met the requirements on VR-ET but not SAP. Rates of eyes meeting criteria on VR but not SAP were not significantly different between VR⁰ and VR-ET (p=0.205). There was no significant association between the order of test administration and which test failed to meet insurance coverage criteria for both VR⁰ and VR-ET (p=0.782 and p=0.659, respectively).

Discussion

The findings of our pilot study demonstrated VR had an overall sensitivity of 84.1% and specificity of 67.6% in detecting improvement of superior visual fields after taping in patients with ptosis, brow ptosis, and dermatochalasis, without significant differences between VR⁰ and VR-ET. Prior literature has identified useful diagnostic tests as ones in which sum of sensitivity and specificity is greater than 1.5, a threshold that was met by VR in this study.⁹ A high sensitivity suggests VR consistently identified upper eyelid pathology meeting insurance coverage criteria when compared to SAP. The relatively low specificity of VR indicates that it may have overstated superior visual field impairment in patients with upper eyelid pathology that was not visually significant. It is important to note that SAP is not considered a gold standard for superior visual field testing but was used as a comparison because it is the most commonly used method.²

A previous study demonstrated the ability of visual field tests administered on VR and SAP to meet insurance coverage criteria in 4 ptotic eyes. However, the authors chose to use a 30–2 testing program on SAP.⁵ We are unaware of previous reports comparing Superior-64 visual field testing on SAP to a comparable test on VR. We also believe this to be the first report comparing VR devices with and without eye-tracking in a visual field testing capacity. Eye-tracking capabilities allow for active surveillance of fixation maintenance and can offer real-time reminders to the participant when fixation is lost, potentially leading to more reliable test results. However, precise reliability indices such as fixation losses, false positives, and false negatives were not available for tests administered on VR because the fixation point is positioned inferiorly to maximize the superior visual field testing area, thereby preventing detection and testing of the patient’s blind spot. Eye-tracking capabilities should theoretically increase the sensitivity of VR, as continued fixation would ensure that participants are not seeing stimuli outside their true field of view and that upper eyelid pathologies would be adequately identified. A lack of significant difference in sensitivity between VR with and without eye tracking suggests VR-ET does not perform significantly better than VR⁰ in identifying ptosis, brow ptosis, and dermatochalasis.

Rates of insurance coverage criteria fulfillment on SAP differed by diagnosis in our cohort. Participants diagnosed with dermatochalasis had a significantly higher rate of meeting insurance coverage criteria on SAP than those with ptosis. Though the association between eyelid abnormality diagnosis and superior visual field testing results has not been previously explored, it is possible that dermatochalasis led to higher rates of insurance coverage criteria fulfillment due to ease of taping excess skin as opposed to ptotic eyelids, to effectuate a substantial improvement in the superior visual field. While rates of insurance coverage criteria fulfillment varied similarly by diagnosis on VR as on SAP, the difference was not statistically significant. It is possible diagnosis was not as influential in determining the ability to qualify for insurance coverage on VR. However, our sample size may have been smaller than necessary to reveal the relationship entirely. Future efforts may be focused on determining underlying causes of differential insurance coverage criteria fulfillment rates between diagnoses for both SAP and VR and investigating alternative methods to ensure appropriate assessment of superior visual fields in patients with ptosis.

Rates of insurance coverage criteria fulfillment were not seen to differ between VR and SAP by age, order of test administration, or laterality of eye being tested, suggesting visual field testing using VR can be used on a wide age range of patients and patients without any previous visual field test-taking experience. For eyes with disagreement between VR⁰ or VR-ET and SAP, there was no association between the order of test administration and which device failed to meet insurance coverage criteria, indicating that test fatigue or learning curve did not play a significant role in the results.

Our study was limited by a small sample size – something that may be addressed in future, larger-scale studies. Patients also had an inherent bias during the test taking process, wanting to score poorly on the untaped test or well on the taped test to obtain insurance authorization for surgery. In an effort to remove patient bias, further studies could assess VR versus SAP using volunteers without upper eyelid pathology while simulating the optical effects of ptosis with opaque contact lenses.¹⁰ While participants were randomized to order of VR and SAP test administration, use of VR⁰ or VR-ET was determined by date of recruitment, which may have introduced bias into the study. Additionally, the superior visual field test administered on VR was comparable to the Superior-64 test on SAP, but with an inferior fixation point. Prior studies have found a worsening of ptosis on downgaze in patients with acquired ptosis.^11,12 Our study examined the improvement in visual fields with taping, which has an unknown effect on the degree of ptosis on downgaze. Future studies may benefit from quantifying the comparability of superior visual field testing in the primary versus downgaze position. Additionally, there was variability in personnel taping eyelids during VR and SAP visual field exams. While all VR exams were administered by study personnel, SAP exams were administered by hospital-employed technicians. Differences in administrators for each test benefit masking personnel from the results of the previous test. Still, inconsistencies in taping techniques can lead to variability in taped visual field exam results. Future studies may benefit from obtaining photographs of the eyes before and after taping the eyelids for VR and SAP to compare consistency in taping.

An additional limitation is that the statistical power of this pilot study could not be determined before data collection and analysis because no other studies exist evaluating superior visual fields on a VR device. A larger sample size or different methodology may have resulted in a significant difference between VR⁰ and VR-ET agreement rates with SAP. Future studies may benefit from using our findings to determine statistical power. Both eyes of bilaterally affected patients were used in analyses, which may have created a bias because multiple observations were collected from a single patient but was done in keeping with other studies investigating superior visual field testing methods.^7,8 The presence of severe upper eyelid abnormalities covering the pupil in some patients tested on VR-ET may have hindered pupil detection and tracking such that the eye-tracking feature was effectively disabled. In addition, data were collected from a single institution, so extrapolation to other geographic regions and populations may be limited. Lastly, patients were enrolled from different oculoplastic attending clinics, which could lead to variability in the documentation of the various diagnoses of ptosis, dermatochalasis, and brow ptosis. For instance, some attendings could choose to document mild brow ptosis while others might leave it off the diagnosis list.

The results of this pilot study demonstrate that VR may be a useful alternative to SAP for evaluating superior visual fields before functional upper eyelid surgery. Because there was no significant difference in sensitivities and specificities between VR with and without eye-tracking, our study demonstrates no reason to support the recommendation of using only eye-tracking-enabled VR devices. A cost analysis of VR and SAP was thought to be outside the scope of this study and was not performed. Further studies would need to be conducted to determine if using VR for superior visual field testing presents advantages in test-retest reliability, completion time, waiting time, or patient satisfaction.