Artificial intelligence improves accuracy, efficiency, and reliability of a handheld infrared eccentric autorefractor for adult refractometry

Abstract

AIM

To evaluate the accuracy, efficiency, and reliability of a handheld infrared eccentric autorefractor (hICA) with artificial intelligence (AI) by comparing its refraction measurements to those recorded using hICA and a clinical table-mounted automatic refractor (TAR).

METHODS

A cross-sectional study using three optometers, including hICA with or without AI and TAR, for refractometry of adults (aged 19-49 years old) with no signs of ocular disease or trauma in the absence of cycloplegia. Right and left eye refraction data were recorded, including the spherical equivalent (SE), diopter of spherical power (DS), diopter of cylindrical power (DC) decomposed into vectors J0 and J45, and measurement times. To avoid analytical difficulties associated with the interdependence of observations between eyes from the same individual, the Generalized Estimation Equation was used to compare the SE, DS, J0 and J45 measurements, and the times thereof, among the different groups. The intraclass correlation coefficient (ICC) and Spearman's rank correlation coefficient were used to evaluate correlations among the measurements recorded by the three different instruments. Bland-Altman were used to analyze the precision of the equipment by the agreement.

RESULTS

A total of 70 patients (140 eyes; mean age: 31.37y; range: 19-49y) were assessed using refractometry. In a brightly lit environment, there was no significant difference between the mean SE recorded using TAR and that recorded using hICA with AI or without AI (both P>0.05). In an intense-light environment, hICA equipped with AI showed a better detection rate than without AI. Light intensity had a greater effect on dioptric measurements recorded using hICA without AI (P<0.001) than on those recorded using the one equipped with AI (P<0.05). Measurement times varied significantly between the different light intensities and instruments (P<0.05).

CONCLUSION

For the normal human eyes, AI may improve the accuracy, efficiency, and reliability of measurements recorded using hICA in various light environments.

INTRODUCTION

Ametropia is now a serious public health concern worldwide. Globally, it was estimated that there were 312 million cases of myopia in 2015^[1]. Nearly 5 billion people will be affected by 2050^[2]. A higher incidence of myopia means more pathological myopia patients. Refractive error has become one of the leading causes of visual impairment and preventable blindness among children and young adults.

Based on the above, regular and large-scale vision screening should be implemented as soon as possible. Accurate, affordable, and portable measuring equipment is needed to screen large populations. Retinoscopy, table-mounted autorefractors (TAR), and handheld automatic refractors are often used for vision screening. Retinoscopy, which estimates refractive power by measuring the divergence of reflected light, requires experienced and skilled optometrists. TAR is widely used and technological innovations have improved their precision. However, measuring visual acuity in subjects who are older or very young, or in those that have a disability may be more challenging; consequently, portable handheld autorefractors are also frequently used to measure visual acuity^[3]–^[7].

Handheld automatic refractors are convenient to use, and many studies have compared their accuracy and efficiency with traditional clinical optometry methods^[8]–^[10]. Results have shown that measurements of astigmatism, myopia, and anisometropia recorded using these handheld autorefractors are consistent with those recorded using cycloplegic retinoscopy^[11]–^[13]. However, these refractors are associated with small errors and may be affected by external factors^[14]. The measurement of refractive error using a handheld infrared eccentric autorefractor (hICA) is based on light tracing, which may be affected by changes in light intensity, humidity, movement caused by hand-shake, focusing blur, or eye deformation. Deep learning, as a neoteric form of artificial intelligence (AI), could improve the stability and robustness of these procedures by enhancing the representativeness of data in the form of text, images, or sound. In this study, AI was applied to increase the accuracy of hICA measurements obtained during vision screening.

This research investigated whether AI improved the clinical utility of hICA by comparing the values of diopter measurement and time control, and provides insight that could aid the development of accurate and efficient autorefractors.

SUBJECTS AND METHODS

Ethical Approval

The study adhered to the tenets of the Declaration of Helsinki. The study protocol was approved by the Ethics Committee of Shanghai Ninth People's Hospital, affiliated with Shanghai Jiao Tong University School of Medicine (Shanghai, China; SH9H-2020-T22-2). The study objectives and procedures were explained to all subjects in advance, and written informed consent was obtained.

Subjects

Subjects with small pupils (bilateral pupil diameter <2 mm in indoor light) and ocular diseases were excluded from the study. In total, 70 healthy adult volunteers participated. Subjects with a visual acuity <20/20 with correction in one eye were not eligible to participate. Data on age, date of birth, sex, spectacle use, and ophthalmological findings were collected.

Instruments

Three instruments were tested in this study, namely an automatic refractor (AR-1; Nidek, Gamagori, Japan) and two automatic vision screeners: the VS100 Spot Vision Screener (Welch Allyn, Skaneateles Falls, NY, USA) and the V100 Vision Screener (MediWorks, Shanghai, China)^[15]–^[16]. The appropriate rights to reproduce or mentioned of the V100 Vision Screener has been obtained from Shanghai MediWorks Precision Instruments Company Limited. All three instruments were calibrated before testing.

Image Processing and Development of the Deep Learning Algorithm

The AI binocular measurement method described here is based on deep learning.

U-net segmentation network

The U-net segmentation network described by Ronneberger et al^[17] in 2015 is widely used for medical image segmentation. U-net were used to segment the pupil area from red/green/blue (RGB) images. The image resolution was 320×240, and probability maps were generated by convolution, skip connection, and deconvolution operations. The pupil area was considered to correspond to the probability map that exceeded the probability threshold (Figure 1). The U-net neural network enhances information, decreases the loss thereof, and greatly improves the accuracy of medical images. As shown in Figure 2, the network framework includes an encoder, decoder, and skip connection. The encoder extracts image features, such as shallow layers and fine granular structures. The decoder restores the features, including shallow- and deep-channel features, and converts image information from low to high resolution. The decoding module can express deep- and coarse-grained features. Next, the ROI is located using probability maps. The skip connection links the encoder and the decoder, reduces information loss during the feature extraction process, and ensures accurate positioning and segmentation.

Figure 1. Flowchart for generating probability maps using the MediWorks V100 Vison Screener with AI.

Figure 2. U-net segmentation network architecture.

U-net model training and pupillary region capture

A total of 20 000 human eye images were collected and separated into a training set and a verification set (ratio of 4:1). Data augmentation was applied, including rotation, translation, scaling, grey-level stretching, and randomisation. Then the images were normalised by subtraction and accommodating variation. The “loss cross-entropy function” was dichotomous, with “0” representing the background and “1” representing the pupil. The “U-net training weight” was used as the initial weight before fine-tuning the training dataset. Stochastic gradient-descent with an optimised iteration method was applied for 60 rounds. The initial learning rate of 0.01 decreased 10-fold after 20 rounds, and then again after 40 rounds. Finally, the training weight with the minimum difference between the training and verification set data loss was selected for network reasoning. The U-net network inference procedure generated probability maps with thresholds. Areas with a probability >0.8 were designated as pupillary regions; the remaining areas were considered background. Next, a binary mask for the pupillary region was obtained and used to extract the pupillary ROI from the original image. Then the infrared eccentricity algorithm was used to obtain diopter values.

Refractive Error Screening Procedure

All 70 subjects were randomised into two groups (A and B), each including 35 subjects, using a randomised block design implemented in the R environment (R Development Core, Team, Vienna, Austria). Refractive measurements were obtained in the following sequence as showed in Figure 3.

Figure 3. Flowchart of the refractive error screening procedure.

In the first and second steps, two professional optometrists simultaneously obtained the measurements for each subject; each optometrist used a different vision screener. Then the optometrists swapped the vision screeners before the next round of measurements. Therefore, all subjects were evaluated using both vision screeners, and by both optometrists, under bright and intense light conditions. Measurements that took more than 20s were considered failures. The Welch Allyn VS100 and MediWorks V100 devices were positioned approximately 1 m from the face of each participant to obtain the measurements.

To evaluate the efficiency of each vision screener, measurement times were recorded for each subject by two timekeepers with two stopwatches of the same type (from the point at which the binocular image appeared on the screen until the results were outputted).

Parameters for Refractive Error Measurements

Measurements recorded using the TAR were used as the reference standard. The diopter of spherical power (DS) and cylindrical power (DC) were decomposed into vertical/horizontal component (J0=-(DC/2)×cos(2A), A means axis) and oblique component [J45=-(DC/2)×sin(2A)] of refractive, and spherical equivalent (SE; the DS plus half of the negative DC) were used to evaluate the accuracy of both of the handheld infrared eccentric autorefractors used in this study.

Statistical Analysis

The data collected during the project were processed using Excel software (Microsoft Corp., Redmond, WA, USA). Next, the data were reviewed for errors and analysed using SPSS software (ver. 24.0; IBM Corp., Armonk, NY, USA). The normality of the distribution of the optometry data was assessed using the Shapiro-Wilk test. For qualitative data, frequencies and proportions were calculated. Descriptive statistics were generated for the quantitative data, as medians and interquartile ranges (IQRs), because these data were not normally distributed. To avoid analytical difficulties associated with the interdependence of observations between eyes from the same individual, a generalised equation was used to compare the SE, DS, and DC measurements, and the times thereof, among the different groups. The intraclass correlation coefficient (ICC) and Spearman's rank correlation coefficient were used to evaluate correlations among the measurements recorded by the three instruments. Bland-Altman were used to analyze the precision of the equipment by the agreement. The tests were two-sided, and a P-value <0.05 was considered statistically significant.

RESULTS

Demographic Data

In total, 140 eyes of 70 participants were assessed. The sociodemographic characteristics of the participants are shown in Table 1.

Table 1. Participant characteristics.

Characteristics	n=70 (%)
Sex
Male	41 (58.57)
Female	29 (41.43)
Age groups (y)
19-29	26 (37.14)
30-39	36 (51.43)
40-49	8 (11.43)
Mean±SD	31.37±6.49
Wears spectacles
Yes	31 (44.29)
No	39 (55.71)

Accuracy of the Handheld Infrared Eccentric Autorefractors with/without AI Compared to the Table-Mounted Refractor

Results in a brightly lit environment

In a brightly lit environment (161.2 lx), the median (IQR) SE values measured using the MediWorks V100, Welch Allyn VS100, and Nidek AR-1 instruments were -1.250 (2.47) D, -1.187 (2.973) D, and -1.678 (3.094) D, respectively. There were no significant differences in the estimated marginal mean SE, DS, and DC values (J0 and J45) obtained using the Welch Allyn VS100 and Nidek AR-1 (P>0.05). The estimated marginal mean SE, DS, and DC (J0, J45) values obtained using the three instruments are presented in Table 2.

Table 2. Estimated marginal values obtained using three instruments in a brightly lit environment.

Instruments	SE	DS	DC
			J0	J45
MediWorks V100	-2.292±0.303	-1.950±0.299	-0.0002±0.023	-0.101±0.038
Welch Allyn VS100	-2.006±0.289	-1.701±0.295	0.011±0.028	0.010±0.028
Nidek AR-1	-2.144±0.279	-1.856±0.277	0.045±0.029	-0.023±0.043
P	0.190a, 0.103b	0.399a, 0.077b	0. 498a, 0.656b	0.106a, 0.577b

SE: Spherical equivalent; DS: Spherical power; DC: Cylindrical power; D: Diopters. ^aComparison of the values obtained using the MediWorks V100 and Nidek AR-1; ^bComparison of the values obtained using the Welch Allyn VS100 and Nidek AR-1.

mean±SD, D

In a brightly lit environment (161.2 lx), the ICC for the SE between the MediWorks V100 and Nidek AR-1 instruments was 0.925 (P<0.001), and that between the Welch Allyn VS100 and Nidek AR-1 was 0.955 (P<0.001). There was a statistically significant correlation in the SE, DS and DC measurements recorded using both vision screeners and the TAR (P<0.05).

Results in an intense-light environment

In total, 98 eyes of 49/70 (70%) participants were successfully evaluated using the Welch Allyn VS100. Therefore, the SE measurements of these 49 subjects were analysed. In an intense-light environment (1043 lx), the medians (IQR) SE values measured using the MediWorks V100, Welch Allyn VS100, and Nidek AR-1 instruments were -1.303 (2.89) D, -1.522 (3.164) D, and -2.030 (3.124) D, respectively. Similar to the results obtained in the brightly lit environment, the DC values significantly differed between MediWorks V100 and Nidek AR-1 (P<0.05). There were statistically significant differences in the SE and DS values obtained using the Welch Allyn VS100 and Nidek AR-1 instruments (P<0.05). The estimated marginal mean SE, DS, J0 and J45 values obtained using the three instruments are presented in Table 3.

Table 3. Estimated marginal values obtained using three instruments in two environments.

Parameters	Environments	MediWorks V100	Welch Allyn VS100	Nidek AR-1	b P	c P
SE (D)	BL	-2.537±0.384	-2.322±0.369	-2.485±0.346
	IL	-2.361±0.363	-2.077±0.333	-2.476±0.344	0.235	0.000
a P		0.022	0.000	0.752
DS (D)	BL	-2.227±0.375	-2.060±0.373	-2.244±0.332
	IL	-2.034±0.347	-1.749±0.331	-2.236±0.332	0.033	0.000
a P		0.017	0.000	0.763
J0	BL	0.018±0.027	0.015±0.026	0.010±0.029
	IL	0.030±0.041	0.005±0.028	0.050±0.020	0.659	0.214
a P		0.841	0.754	0.291
J45	BL	-0.059±0.034	-0.001±0.022	0.009±0.024
	IL	-0.022±0.030	-0.014±0.034	0.002±0.020	0.505	0.720
a P		0.459	0.797	0.844

SE: Spherical equivalent; DS: Spherical power; DC: Cylindrical power; D: Diopters. BL: Bright light (161.2 lx); IL: Intense light (1043 lx). ^aComparison of the values obtained in bright- and intense-light environments; ^bComparison of the values obtained using the MediWorks V100 and Nidek AR-1 in intense-light environment; ^cComparison of the values obtained using the Welch Allyn VS100 and Nidek AR-1 in intense-light environment.

In an intense-light environment (1043 lx), the ICC for the SE between the MediWorks V100 and Nidek AR-1 instruments was 0.956 (P<0.001), and that between the Welch Allyn VS100 and Nidek AR-1 instruments was 0.973 (P<0.001). The ICC and Bland-Altman analyses indicated a high degree of consistency and repeatability for the SE and DS measurements obtained using the two vision screeners and the TAR.

Effects of Light Intensity on Measurements

Light intensity had a significant effect on the dioptric measurements recorded using both handheld screeners (P<0.05), whereas it had little effect on the TAR measurements (P>0.05; Table 3).

Efficiency of the Handheld Infrared Eccentric Autorefractor with/without AI

Detection rates

Of the two hICAs, the instrument equipped with AI (MediWorks V100) showed the better detection rate (100% vs 70% in an intense-light environment).

Measurement time for three instruments in different light environments

As shown in Table 4, the estimated marginal mean length of time necessary to record measurements in both the brightly lit (P=0.008) and an intense-light (P=0.002) environments was shorter when using the MediWorks V100 than when using the Welch Allyn VS100. Lower light intensity decreased the time necessary for both screeners to complete the dioptric measurements in both environments.

Table 4. Measurement times for three instruments.

Instruments	Environment		P
	BL	IL
MediWorks V100	3.760±0.236	4.398±0.249	0.011
Welch Allyn VS100	5.417±0.327	7.251±0.615	0.008
Nidek AR-1	16.927±0.431	17.610±0.563	0.304

BL: Bright light (161.2 lx); IL: Intense light (1043 lx); s: Seconds.

mean±SD, s

DISCUSSION

Recent studies have evaluated the performance of deep learning-based algorithms for diagnosing ophthalmic diseases via image analyses^[18]–^[20]. This study describes a theoretical and experimental approach to vision screening using AI technology. In this cross-sectional study, the mean dioptric measurement values and times were compared between two hICAs, the MediWorks V100 and Welch Allyn VS100 instruments, and a TAR, Nidek AR-1.

The results indicated that AI could play an important role in challenging vision screening environments. In a brightly lit environment, the SE and DS measurements obtained using the hICA without AI were less negative than those obtained using TAR (P>0.05). Similar findings have been reported in previous studies^[8],^[21]–^[24]. In the intense-light environment in this study, the SE and DS values obtained using the hICA equipped with AI were more similar to the reference standard values. One explanation for the higher detection rate (100% vs 70%) and more rapid measurements observed using the vision screener with AI under intense light (P<0.05) is that AI overcomes some of the disadvantages associated with traditional image processing and enhances the sensitivity and robustness of the instrument through more precise detection and recognition in complex environments. There was statistically significant agreement in the SE and DS measurements obtained using the hICA and the TAR, which suggests that these vision screeners may be suitable for large-scale clinical screening and evaluation of patients who cannot be assessed using conventional refractometry.

Photo-screening technology is increasingly being used for optical screening due to its numerous advantages, such as high-speed binocular measurements, minimal training requirements, and a compact and lightweight instrument design. This is the first study to combine AI and photo-screening technology to assess the accuracy and efficiency of these instruments when used in healthy adults. One recent study applied deep learning for myopia screening of children and achieved high screening accuracy using deep convolution neural networks, thus demonstrating the potential benefits of AI for vision screening^[22]. Deep learning was proven to be effective for estimating refractive error in clinical practice. AI may be applied to improve routine, large-scale screening for myopia.

This study did have some limitations. In particular, although auto-refractometry is now established as a reliable tool for measuring refractive error and visual acuity, the manual refraction after cycloplegia remains as the gold standard but was not used in the study. Previous studies have shown that different autorefractors produce significantly different SE measurements, using both objective and subjective refraction^[8],^[23]–^[24]. Thus, further studies are needed to compare the accuracy of automatic refractors equipped with AI and subjective refractors, with and without cycloplegia. Handheld automatic refractors are particularly suitable for assessing vision in infants, preschool children, older subjects with mobility difficulties, and those at risk for amblyopia or severe refractive defects. Further studies are needed to better understand the typical values in various populations. DC measurements recorded using handheld automatic refractors equipped with AI were not particularly accurate. However, this inaccuracy was eliminated after decomposing DC into J0 and J45 and analyzing separately. There are still several uncertain factors in the study. First, binocular accommodation varies significantly among individuals. Second, hICA and TAR are based on different principles. Measurement distances, algorithms, and calibration criteria may vary significantly between the two instruments. Third, the results in this study may have been affected by various other factors such as measurement distance, light, humidity, eye movements, and a small sample size; these factors could explain why the DC measurement results differed from those recorded in previous studies.

In conclusion, this study tested the effectiveness of an AI-enabled hICA for clinical vision screening and found that the AI technology improved the accuracy and speed of measurements in complex environments for normal human eyes without diseases. Future research efforts should be directed toward large-scale screening and early detection/prevention of myopia.