Eye-tracking-aided digital system for strabismus diagnosis

Abstract

Strabismus is one of the most common vision disorders in preschool children. It can cause amblyopia and even permanent vision loss. In addition to a vision problem, strabismus brings to both children and adults serious negative impacts in their daily life, education, employment etc. Timely diagnosis of strabismus is thus crucial. However, traditional diagnosis methods conducted by ophthalmologists rely significantly on their experiences, making the diagnosis results subjective. It is also inconvenient for those methods being used for strabismus examination in large communities such as schools. In light of that, in this Letter, the authors develop an objective, digital and automatic system based on eye-tracking technique for diagnosing strabismus. The system exploits eye-tracking technique to acquire a person's eye gaze data while he or she is looking at some targets. A group of features are proposed to characterise the gaze data. The person's strabismus condition can be diagnosed according to the features. A strabismus gaze dataset is built using the system. Experimental results on the dataset demonstrate the effectiveness of the proposed system for strabismus diagnosis.

1. Introduction

Strabismus, termed squint and heterotropia as well, is that the two eyes do not point to the same direction. It is a common ophthalmic disorder with a prevalence of around 4% of the adult population [1]. If strabismus is not well treated, it would result in amblyopia or weak three-dimensional (3D) perception [2]. In addition to a vision problem, strabismus has been shown to have adverse psychosocial consequences in both children and adults [3]. For many young patients, their strabismus problems could be well treated or significantly alleviated if diagnosis and treatment are taken early. Therefore, timely diagnosis of strabismus is essential.

Traditional strabismus diagnosis methods include cover test, Hirschberg test, Maddox rod etc. These methods are conducted manually by the ophthalmologists according to their experiences. The diagnosis correctness is remarkably influenced by the ophthalmologist's professional qualities. Furthermore, these methods are manual, which makes it inconvenient to use for large communities (e.g. schools). In view of that, we propose and develop a digital strabismus diagnosis system based on eye-tracking technique in this Letter. The system is non-invasive and objective, and the diagnosis is generated automatically without any ophthalmologist's instruction. This makes it easy to carry out strabismus examination in large communities. For example, we can place the system in a primary school and the students can take their examinations at any time.

In the proposed system, we first show the subject nine target points ordered in different positions on a screen, and record the subject's gaze data. Then, a group of features are proposed to characterise the gaze data. A diagnosis result is finally made by analysing the features. We build a gaze dataset using our eye-tracking system for performance evaluation. The effectiveness of our method is demonstrated by comparing the diagnosis result with professional ophthalmologist's diagnosis result.

The rest of this Letter is organised as follows. We first review Letters on the eye-tracking methodology for solving various problems, especially disease detection and analysis, and then describe the proposed system and elaborate the features that are used to characterise gaze data. After that, we introduce our dataset and report experimental results. A conclusion is drawn at the end of this Letter.

2. Related work

Approaches have been developed to detect eye movements since the last century. For example, Jolson et al. [4] proposed an apparatus for evaluating the alignment of both eyes. Knapp et al. [5] developed a method and apparatus for determining the position of each eye of a human with respect to a fixed reference point or with respect to each other by using electro-oculogram signals produced by eye movement, so as to measure strabismus. Compared to modern eye trackers, however, the accuracy of this apparatus is pretty low. In [6], Schaeffel developed an automated procedure to measure kappa and the Hirschberg ratio for immediate use in a video gaze tracker. Yang et al. [7] used a 3D strabismus photo analyser to estimate binocular alignment using photographs, and found that the 3D Strabismus Photo Analyser is a simple and reliable tool for measuring ocular deviation.

Thanks to its rapid development, the eye-tracking technique has been successfully applied to solving various problems, e.g. object recognition [8], attention modelling [9], image quality assessment [10], and biometric identification [11], as well as disease investigation [12, 13].

There are some attempts that leverage eye-tracking methodology for strabismus examination [14–18]. In the last century, people have attempted to build automatic eye-tracking systems for measuring strabismic deviation based on television cameras [14, 15]. In the past several years, people continued to refine strabismus examination by developing more advanced eye-tracking methods. For example, Pulido [16] used Tobii eye tracker to acquire gaze data, and calculated the deviation of gaze data for ophthalmic diagnostics including strabismus. However, Pulido proposed a method prototype only. The author had no real strabismic gaze data to demonstrate the prototype's performance. Model and Eizenman [17] proposed an automatic method based on eye-tracking for performing the Hirschberg test, a classical method to measure binocular ocular misalignment. To precisely locate the pupil centres and corneal reflexes, they built a remote binocular gaze-tracking system using two video cameras and three infrared light sources. However, the performance was studied with five healthy infants only. Thus, the method's effectiveness for strabismus examination had not been tested. In Model's doctoral dissertation, he proposed two personal calibration procedures that do not require active user participation for estimation of gaze point and measurement of ocular alignment for adults and infants [18], but his methods were not been tested with sufficient strabismic data. Bakker et al. [19] developed a gaze direction measurement instrument to estimate strabismus angle. The instrument allows for unrestrained head movement by means of a triple camera vision system that simultaneously estimates the head rotation and the eye pose. Only three subjects participated in the experiment.

In recent year, virtual reality technique is also applied to eye health. For instance, Blaha and Gupta [20] designed a virtual reality game to help people with amblyopia restore vision, but they focused on amblyopia training rather than diagnosis.

3. Methodology

Fig. 1 shows the framework of the proposed system, in which the diagnosing procedure for a subject is carried out as follows:

1. The subject sits at a fixed near distance from a laptop with an eye tracker mounted below the laptop's screen.

2. The subject fixates on a number of calibration points to perform calibration.

3. The subject fixates on a number of target points for eye gaze data collection.

4. A group of features are extracted from all the gaze data.

5. A diagnosis result is made according to the features.

The above procedures will be detailed in the subsequent sections.

Fig. 1.

Framework of the proposed strabismus diagnosis system

3.1. Gaze data acquisition

3.1.1. Eye-tracking system

We use eye tracker Tobii X2-60 to build our system. Its sampling rate is 60 Hz and tracking accuracy is 0.4°. To make the system portable, we mount the eye tracker below the monitor of a laptop, as shown in Fig. 1. The laptop is Lenovo Thinkpad T540p with a 1920 × 1080 screen resolution. We employ Tobii MATLAB SDK to design our experiments for gaze data acquisition. The coordinate system of the screen is defined as follows. The upper-left corner of the screen is set as the origin, the coordinate value of which is (0, 0), with horizontal denoting x-coordinate and vertical denoting y-coordinate. The coordinate value of the lower-right corner is (1, 1), and the values of the upper-right corner and lower-left corner are (1, 0) and (0, 1), respectively. Both the x-value and y-value of any position on the screen are between 0 and 1, as shown in Fig. 2.

Fig. 2.

Nine-point gaze data acquisition interface. The white arrows indicate the display order

3.1.2. Calibration

The purpose of calibration is to teach the eye-tracking system the characteristics of the subject, such that the eye tracker can well detect the subject's eye movements in a subsequent test. In the calibration process, we adjust the subject's chair and the eye-tracking screen to make sure that the subject is at a fixed distance (50 cm in our experiments) from the screen and his or her eye level is horizontally pointing to the screen centre, i.e. point (0.5, 0.5). We choose a distance of 50 cm as it is an optimal distance for eye tracker Tobii X2-60 to track the subject's eye movements. We use nine-point calibration scheme. If the calibration result indicates that the fixation accuracy of either eye is acceptable (average accuracy higher than a predefined value, e.g. 0.05 in this Letter), we can then start the subsequent test. Otherwise, we should recalibrate.

3.1.3. Data acquisition

A classical way for the ophthalmologist to examine strabismus is a nine-point method. That is, the ophthalmologist instructs the patient to fixate on nine points at a certain distance in front, and then observes the patient's eye movements. The nine-point method can comprehensively examine one's eye movements with rotations at different angles. We adopt the same method to design our gaze data acquisition interface. The coordinate values of the nine points that we used are the same for the nine calibration points. The points are displayed one by one orderly. Fig. 2 shows the nine-point gaze data acquisition interface. The white arrows point out the display order. A black background of the interface helps the subject to concentrate on the target points. Data acquisition programme runs as follows. Each time one target point is displayed. Meanwhile, the subject's gaze points are recorded by eye tracker. If the number of effective gaze pairs acquired exceeds 100, current target point is removed and the next target point would be displayed. A gaze pair is defined as two gaze points of the two eyes captured by eye tracker at one sampling moment. ‘Effective’ here means that at least one gaze point of a gaze pair locates close enough to the target point. That is, the distance between the target point and either gaze point of a gaze pair must be smaller than a threshold (0.05 in this Letter) predefined empirically. For those who suffer serious strabismus, it is sometimes hard to capture effective gaze points at some target points, e.g. points at the four corners of the screen, because these points need strabismic subjects to rotate eyeballs to their extreme. To handle this case, we let eye tracker record gaze points at each point for at most 10 s. The next target point would be displayed after 10 s no matter whether or not the system has collected 100 pairs of gaze points. The sampling rate of our eye tracker is 60 Hz. It takes only 2 s to collect 100 gaze pairs for normal people. Thus, 10 s are long enough to capture gaze data for each point.

3.2. Feature extraction for diagnosis

The next step after gaze data acquisition is feature extraction from all the gaze points. On the basis of the features, we should be able to tell whether the subject has strabismus or not, which eye is strabismic, and what type of strabismus it is. The features for answering these questions are constructed and described as follows.

3.2.1. Strabismus occurrence feature

Strabismus occurrence feature is defined to answer the question of whether or not the subject has strabismus. To realise that we propose to exploit two features: fixation deviation and fixation contrast. Fixation deviation is defined as the Euclidean distance between the centre of a gaze pair and the target point. We can also calculate the Euclidean distance between left or right gaze point and the target point. Fixation contrast is then defined as the difference between the Euclidean distances of left gaze point and right gaze point to the target point. Fig. 3 shows an example of a normal gaze point pair (blue) and a strabismic gaze point pair (red) together with their target point (black). (x_nl, y_nl), (x_nr, y_nr), and (x_nc, y_nc) denote left gaze point, right gaze point, and the centre of a normal gaze pair, and (x_sl, y_sl), (x_sr, y_sr), and (x_sc, y_sc) are corresponding points for a strabismic gaze pair. (x_t, y_t) is the target point. d_nl, d_nr, d_n, d_sl, d_sr, and d_s are distances between the target point and corresponding points. The significant difference between a normal gaze pair and a strabismic gaze pair is that the strabismic pair has always a point close to the target point and the other one relatively far from the target point, while the two points of the normal pair locate at a similar distance from the target point. As a result, the strabismic pair usually has a fixation deviation d_s larger than that (d_n) of the normal pair, and a fixation contrast |d_sl−d_sr| larger than that (|d_nl−d_nr|) of the normal pair. Therefore, we can use fixation deviation and fixation contrast to classify strabismic gaze data and normal gaze data accordingly.

Fig. 3.

Example of normal (blue) and strabismic (red) gaze pairs, and their target point (black)

Mathematically, let (x_l, y_l) and (x_r, y_r) denote the left gaze point and right gaze point for a gaze pair, and (x_t, y_t) denote the target point. We can then have the formulation of fixation deviation defined as below:

$$f_{\mathrm{d}}=\sqrt{{\left(\frac{\left(x_{\mathrm{l}}+x_{\mathrm{r}}\right)}{2}-x_{\mathrm{t}}\right)}^2+{\left(\frac{\left(y_{\mathrm{l}}+y_{\mathrm{r}}\right)}{2}-y_{\mathrm{t}}\right)}^2},$$ (1)

and the formulation of fixation contrast defined as below:

$$f_{\mathrm{c}}=\left|d_{\mathrm{l}}-d_{\mathrm{r}}\right|,$$ (2)

where

$$d_{\mathrm{l}}=\sqrt{{\left(x_{\mathrm{l}}-x_{\mathrm{t}}\right)}^2+{\left(y_{\mathrm{l}}-y_{\mathrm{t}}\right)}^2}$$ (3)

and

$$d_{\mathrm{r}}=\sqrt{{\left(x_{\mathrm{r}}-x_{\mathrm{t}}\right)}^2+{\left(y_{\mathrm{r}}-y_{\mathrm{t}}\right)}^2}$$ (4)

We calculate f_d and f_c for all the gaze pairs of nine target points. A simple way to obtain a representative value for all the gaze data is to average f_d and f_c over all the gaze pairs. However, the average result would be dominated by some extreme gaze points, which cannot represent well the subject's normal fixation state. To alleviate the effect of extreme points, we take into account a statistical measure of percentile for the calculations of f_d and f_c. A percentile indicates the value below which a given percentage of observations fall. For example, the 60th percentile of f_d is the value below which 60% of gaze pairs’ f_d may be found. It can characterise the major gaze pairs’ f_d. We tested percentiles for both f_d and f_c from 70 to 95 and found that the diagnosis results do not vary much. In this Letter, for a convenient comparison, we present the widely used 85th percentile only. We denote the percentiles of f_d and f_c over all the gaze data of nine target points as f_dp and f_cp.

For some strabismic people, their eyes can fixate precisely at some directions, whereas for some other directions the two eyes do not align well. This case happens a lot in particular to incomitant strabismus, the strabismic eye of which cannot move in some specific directions. That is why the ophthalmologist needs to use the nine-point method to examine the patient's eye movements. Taking this case into account, we add two features called maximum fixation deviation f_dm and maximum fixation contrast f_cm. We first calculate 85th percentile of f_d and f_c for each target point. Then, max fixation deviation f_dm is defined as the maximum value of 85th percentile of f_d over the nine target points, and max fixation contrast f_cm is defined as the maximum value of 85th percentile of f_c over the nine target points. f_dm and f_cm can be formulated as follows:

$$f_{\mathrm{dm}}=max(f_{\mathrm{dp}}^{(i)})i\in (1,\dots ,9)$$ (5)

and

$$f_{\mathrm{cm}}=max(f_{\mathrm{cp}}^{(i)})i\in (1,\dots ,9)$$ (6)

where $f_{\mathrm{dp}}^{(i)}$ and $f_{\mathrm{cp}}^{(i)}$ represent, respectively, the 85th percentiles of f_d and f_c for the ith target point.

We can have strabismus occurrence feature f_so by combining features f_dp, f_cp, f_dm, and f_cm as follows:

$$f_{\mathrm{so}}=k_1{f}_{\mathrm{dp}}+k_2{f}_{\mathrm{cp}}+k_3{f}_{\mathrm{dm}}+k_4{f}_{\mathrm{cm}}-\alpha ,$$ (7)

where k₁, k₂, k₃, and k₄ are coefficients used to weigh the four features. They can be determined according to practical requirements. In this Letter, k₁ = k₂ = k₃ = k₄ = 1. Constant α is used to bias the output, such that f_so > 0 represents strabismus and f_so < 0 represents normal. Constant α may be calculated by different algorithms. The algorithm we adopt is to compute f_dp + f_cp + f_dm + f_cm for both normal subjects and strabismic subjects. Moreover, then the largest value of normal subjects and the smallest value of strabismic subjects are selected. Their average value is assigned to α. We calculate α based on the data attained. This mechanism is reasonable, as many medical and health indicators are decided by analysing a large number of healthy and unhealthy people's data. With strabismus occurrence feature f_so defined in (7), the decision function is finally defined as

$$F_{\mathrm{so}}(f_{\mathrm{so}})=\left\{\begin{array}{cc}1& f_{\mathrm{so}}>0\\ 0& f_{\mathrm{so}}\le 0\end{array}\right.$$ (8)

where F_so = 1 indicates the subject has strabismus and F_so = 0 indicates the subject is normal.

3.2.2. Strabismic eye feature

After the subject is diagnosed to have strabismus with strabismus occurrence feature, the next step is to work out which eye is strabismic. The feature used to solve the problem is called strabismic eye feature, denoted by f_se. Its definition is as follows:

$$f_{\mathrm{se}}=\frac{1}{N}\sum _{i=1}^N\left(d_{\mathrm{l}}^{(i)}-d_{\mathrm{r}}^{(i)}\right),$$ (9)

where $d_{\mathrm{l}}^{(i)}$ and $d_{\mathrm{r}}^{(i)}$ are the Euclidean distances between the target point and the left gaze point and right gaze point of the ith gaze pair, respectively. The formulas of $d_{\mathrm{l}}^{(i)}$ and $d_{\mathrm{r}}^{(i)}$ are the same in (3) and (4). N is the total number of gaze pairs acquired for the subject. The decision function of feature f_se is

$$F_{\mathrm{se}}(f_{\mathrm{se}})=\left\{\begin{array}{cc}1& f_{\mathrm{se}}>0\\ 0& f_{\mathrm{se}}\le 0\end{array}\right.$$ (10)

Function F_se = 1 means that the right eye fixates better than the left eye, and thus the left eye is a strabismic eye, and F_se = 0 indicates the right eye is a strabismic eye.

3.2.3. Strabismus direction feature

To determine which one of the four types (horizontal exotropia and esotropia and vertical hypertropia and hypotropia) the strabismus is, we should identify its direction (horizontal or vertical) first. The angle between the line connecting the two gaze points of a gaze pair and the horizontal line is utilised to estimate the strabismus direction. We call the angle strabismus direction feature, which is defined as follows:

$$f_{\mathrm{sd}}=\arctan \left(\frac{\left|y_{\mathrm{l}}-y_{\mathrm{r}}\right|}{\left|x_{\mathrm{l}}-x_{\mathrm{r}}\right|}\right).$$ (11)

Fig. 4 depicts the strabismus direction feature f_sd of a gaze pair. f_sd lies in the range [0°, 90°]. An intuitive method to determine the strabismus direction is to binarise f_sd with threshold 45°. f_sd > 45° signifies that the two eyes tend to fixate vertically, and it is vertical strabismus accordingly. Otherwise, it is assigned to horizontal strabismus. However, a slight (e.g. below 5°) vertical strabismus may not affect one's vision, and it is usually ignored in diagnosis. Therefore, most of the strabismus is diagnosed as exotropia or esotropia by ophthalmologists, even though some degrees of hypertropia or hypotropia may occur simultaneously. Only obvious vertical strabismus is assigned to hypertropia or hypotropia. Taking this situation into account, in our method, we evenly divide the range of f_sd into three intervals: [0°, 30°], [30°, 60°], and [60°, 90°], in which the strabismus is classified into horizontal, horizontal prior, and vertical. Horizontal prior means that the subject suffers both horizontal and vertical strabismus, but we still assign him or her to horizontal strabismus with priority in a binary decision. The decision function can be consequently formulated as

$$F_{\mathrm{sd}}(f_{\mathrm{sd}})=\left\{\begin{array}{cc}1& 0^{\circ }\le f_{\mathrm{sd}}\le {30}^{\circ }\\ 0& {30}^{\circ }\le f_{\mathrm{sd}}\le {60}^{\circ }\\ -1& {60}^{\circ }\le f_{\mathrm{sd}}\le {90}^{\circ }\end{array}\right.$$ (12)

where F_sd = 1 denotes horizontal strabismus, F_sd = 0 denotes horizontal-prior strabismus, and F_sd = −1 denotes vertical strabismus. The decision result here is three-fold rather than two-fold. One big advantage of a three-fold decision is that it can provide more detailed information to ophthalmologists in treating strabismus, e.g. strabismus surgery. For example, if the decision result is F_sd = 0, the ophthalmologist knows that the patient has both horizontal and vertical strabismuses. In real diagnosis, f_sd will be also used as an important indicator of strabismus evaluation.

Fig. 4.

Strabismus direction feature f_sd

3.2.4. Horizontal strabismus feature

The horizontal strabismus feature used to distinguish exotropia and esotropia is defined as

$$f_{\mathrm{hs}}=\frac{1}{N}\sum _{i=1}^N\left(x_{\mathrm{r}}^{(i)}-x_{\mathrm{l}}^{(i)}\right)-\beta ,$$ (13)

where $x_{\mathrm{l}}^{(i)}$ and $x_{\mathrm{r}}^{(i)}$ are the x values of the left and right gaze points of the ith gaze pair. N is the total number of gaze pairs. Bias β is calculated by averaging ($x_{\mathrm{r}}^{(i)}-x_{\mathrm{l}}^{(i)}$) over all the normal subjects. The decision function is

$$F_{\mathrm{hs}}(f_{\mathrm{hs}})=\left\{\begin{array}{cc}1& f_{\mathrm{hs}}>0\\ 0& f_{\mathrm{hs}}\le 0\end{array}\right.$$ (14)

where F_hs = 1 is exotropia and F_hs = 0 is esotropia. The magnitude of f_hs is a measurement of severity. A smaller magnitude of f_hs indicates a smaller severity of strabismus.

3.2.5. Vertical strabismus feature

The vertical strabismus feature used to classify hypertropia and hypotropia is defined as

$$f_{\mathrm{vs}}=\frac{1}{N}\sum _{i=1}^N\left(y_{\mathrm{r}}^{(i)}-y_{\mathrm{l}}^{(i)}\right),$$ (15)

where $y_{\mathrm{l}}^{(i)}$ and $y_{\mathrm{r}}^{(i)}$ are the y values of the left and right gaze points of the ith gaze pair. N is again the total number of gaze pairs. f_vs has a similar form with f_hs, except for bias β. Bias β is used to eliminate the horizontal distance produced by pupil distance. f_vs tells the relative position of the two eyes horizontally. However, if we want to determine hypertropia or hypotropia, we need to find out which eye is a strabismic eye. In other words, we have to take into account both f_vs and f_se in order to make the decision. Mathematically, the decision function of vertical strabismus is defined as follows:

$$F_{\mathrm{vs}}(f_{\mathrm{vs}},f_{\mathrm{se}})=\left\{\begin{array}{cc}1& f_{\mathrm{vs}}\times f_{\mathrm{se}}>0\\ 0& f_{\mathrm{vs}}\times f_{\mathrm{se}}\le 0\end{array}\right.$$ (16)

where F_vs = 1 is hypertropia and F_vs = 0 is hypotropia. Similar to f_hs, the magnitude of f_vs measures the severity of vertical strabismus, with a smaller value denoting a smaller severity.

3.2.6. Diagnostic procedure

Fig. 5 depicts the whole diagnostic procedure using the proposed features extracted from the gaze data of a subject. First of all, strabismus occurrence feature f_so is used to estimate whether strabismus occurs or not. If strabismus occurs, strabismic eye feature f_se is exploited to figure out which eye is a strabismic eye. After that, strabismus direction feature f_sd is employed to determine the direction of strabismus, i.e. horizontal or vertical. If the result is horizontal, horizontal strabismus feature f_hs is used to work out the strabismus type, i.e. exotropia or esotropia. Otherwise, vertical strabismus feature f_vs is used to work out the vertical strabismus type, i.e. exotropia or esotropia. Our system can also provide relative severity of strabismus. The severity of strabismus can be measured by horizontal strabismus feature f_hs and vertical strabismus feature f_vs. A larger value of these features indicates a more severe strabismus. In particular, f_hs and f_vs can be used to compare the severity of the same types of strabismuses.

Fig. 5.

Diagnostic procedure using the proposed features

4. Experiments

After ethics approval from Human Research Ethics Committee of Chu Hai College of Higher Education and informed consent, we collected gaze data from 15 normal subjects and 10 strabismic subjects. The age range of normal subjects is from 7 to 40, whereas the age range of strabismic subjects is from 3 to 63. The mean ages are 30 and 40 for normal subjects and strabismic subjects, respectively. Both male and female persons are involved. The numbers of females are 9 and 6 for normal subjects and strabismic subjects, respectively. We denote 15 normal subjects as N₁–N₁₅ and 10 strabismic subjects as S₁–S₁₀. The strabismic subjects have been diagnosed by professional ophthalmologists, and the results are used as ground truths. The ground truths of ten strabismic subjects are tabulated in Table 1. Table 1 reports strabismic eye, strabismus direction, type, and degree, such that we can have a clear comparison between the results generated by our system and the ground truths. The severity of strabismus for ten strabismic subjects can be roughly ranked as S₁ > S₈ > S₄ > S₂ > S₅ = S₇ > S₃ = S₉ > S₆ = S₁₀.

Table 1.

Ground truths of ten strabismic subjects

Subject	Strabismic eye	Direction	Type	Degree
S1	left	vertical	hypertropia	18
S2	left	horizontal	esotropia	12
S3	right	horizontal	exotropia	6
S4	right	horizontal	exotropia	14
S5	right	horizontal	esotropia	8
S6	left	horizontal	esotropia	4
S7	right	horizontal	esotropia	8
S8	right	horizontal	esotropia	16
S9	left	vertical	hypertropia	6
S10	left	horizontal	exotropia	4

4.1. Experimental results

Strabismus occurrence feature f_so is shown in Fig. 6. Bias α = 0.428. All the strabismic subjects have positive values, and all the normal subjects have negative values. Evidently, the strabismus occurrence feature f_so defined in (7) separates the normal data and strabismic data well. This indicates that feature f_so can effectively characterise the difference between gaze data of normal subjects and strabismic subjects. With a simple calculation, we can know that the mean values of f_so are −0.2 and 0.67 for normal subjects and strabismic subjects, and the standard deviations are 0.06 and 0.37 for normal subjects and strabismic subjects, respectively. This signifies that the intra-difference of normal subjects is much smaller than that of strabismic subjects, and the values of f_so are pretty diverse for strabismic subjects. To evaluate the difference between normal data and strabismic data statistically, we perform a t-test for the two groups of data. The p-value is 4.7 × 10⁻⁹, much smaller than 5% significant level, which means that the f_so values of normal subjects and strabismic subjects are significantly different. This indicates that feature f_so is an effective indicator to separate normal subjects and strabismic subjects.

Fig. 6.

Strabismus occurrence feature f_so of 15 normal subjects and 10 strabismic subjects

After strabismus is confirmed, strabismic eye feature f_se is adopted to find out the strabismic eyes for ten strabismic subjects. Fig. 7 depicts the f_se values for ten strabismic subjects. Feature f_se > 0 indicates left eye is a strabismic eye. Therefore, we can see that for subjects S₁, S₂, S₆, S₉, and S₁₀, their left eyes are strabismic eyes, and for subjects S₃, S₄, S₅, S₇, and S₈, their right eyes are strabismic eyes. The results are consistent with the ground truths as shown in Table 1. We perform a t-test for the two groups (left strabismic eye and right strabismic eye). The p-value is 0.0088, demonstrating that the feature f_se of two groups are significantly different. Note that the magnitude of f_se signifies the fixation difference between the left eye and right eye. A smaller |f_se| stands for more similar fixation accuracies that the two eyes have. To further investigate that we show |f_se| for 15 normal subjects in Fig. 8. One important finding is that all the normal subjects have a small magnitude of f_se. The largest one is not larger than 0.025. This is because those normal subjects possess good fixation capabilities for both eyes.

Fig. 7.

Strabismic eye feature f_se of ten strabismic subjects

Fig. 8.

Magnitude |f_se| of strabismic eye feature f_se of 15 normal subjects

The next step is to find out strabismus direction, e.g. horizontal or vertical, using feature f_sd. Fig. 9 depicts the values of f_sd for ten strabismic subjects. The three intervals of f_sd are differentiated by three colours in the sector. We can estimate that the strabismus directions of S₁ and S₉ are vertical, S₈ is horizontal, S₂, S₃, S₄, S₅, S₆, S₇, and S₁₀ are horizontal prior, which could be finally classified to horizontal. In real diagnosis, the value of f_sd will be outputted to the doctors for reference. The p-value of t-test for the vertical and horizontal (including horizontal prior) data is 0.008, which demonstrates the significant difference between vertical and horizontal data, and the effectiveness of feature f_sd in representing strabismus direction.

Fig. 9.

Strabismus direction feature f_sd of ten strabismic subjects

We can then use horizontal strabismus feature f_hs or vertical strabismus feature f_vs to work out strabismus type. Let us investigate S₂, S₃, S₄, S₅, S₆, S₇, S₈, and S₁₀ that have been classified to horizontal strabismus first. Fig. 10 shows their f_hs. Obviously, S₃, S₄, and S₁₀ are exotropia, and S₂, S₅, S₆, S₇, and S₈ are esotropia. The results are consistent with Table 1. Their p-value is 0.08, larger than 5% significant level. In other words, the difference between f_hs value of exotropia and that of esotropia is not significant enough. The magnitude of f_hs measures strabismus severity. A larger magnitude of f_hs represents a more serious strabismus. The subjects’ strabismus severity can be thus ranked as S₈ > S₄ > S₂ > S₅ > S₇ > S₃ > S₁₀ > S₆. As shown in Table 1, the ground truth order is S₈ > S₄ > S₂ > S₅ = S₇ > S₃ > S₁₀ = S₆. The difference between the order generated by our system and the ground truth is that S₅ and S₇ and S₁₀ and S₆ are diagnosed by ophthalmologists to have the same degrees, while our system diagnoses that S₅ is more severe than S₇ and S₁₀ is severe than S₆.

Fig. 10.

Horizontal strabismus feature f_hs of eight subjects that have horizontal strabismuses

Fig. 11 shows vertical strabismus feature f_vs of S₁ and S₉. Note that vertical strabismus type is determined by the product of f_se and f_vs in (16). It can be calculated from Figs. 7 and 11 that f_se × f_vs > 0 for both S₁ and S₉. According to (16), S₁ and S₉ are hypertropias. Moreover, the magnitude of f_vs signifies that the severity ranking is S₁ > S₉. This is consistent with the ground truth.

Fig. 11.

Vertical strabismus feature f_vs of two subjects that have vertical strabismuses

Overall, the experimental results demonstrate that our method is able to diagnose out all the normal and strabismic subjects using eye-tracking data. The results are consistent with ophthalmologist's diagnosis reports.

5. Conclusion

In this Letter, we propose and develop an intelligent digital system based on eye-tracking technique for objective and automatic strabismus diagnosis. An eye-tracking system is first developed to acquire people's gaze data. A group of features are then proposed to characterise the gaze data. On the basis of these features, the proposed system is able to tell whether or not the subject has strabismus, which eye is strabismic, what type of strabismus it is, and the severity of strabismus. We build a gaze dataset that contains both normal and strabismic gaze data for experiments. The results generated by our system are consistent with those diagnosed by ophthalmologists. All experimental results have demonstrated the effectiveness of our method. More importantly, the proposed system is objective, non-invasive, and automatic. It could be a potential powerful alternative for strabismus diagnosis.

Compared to other automatic strabismus diagnosis systems, one major advantage of our system is its portability and convenience. We can easily bring the system to different places to conduct strabismus examination for people. The setup is very simple; meanwhile, high accuracy can be still achieved for diagnosing strabismus occurrence and types. The limitation of our system is that it cannot yet precisely measure the strabismus angle as the cover test with a prism, though the relative severity is quite accurate. In the future work, we will develop more effective features for precise strabismus degree evaluation.

7. Conflicts of interest

None declared.