Applicability of the Smart Vision Screening Instrument among Chinese Primary School Students

Abstract

Background:

A smart vision screening instrument was applied for screening low eyesight in primary school students in Wuhan, China. We aimed to compare the differences in test results between this instrument and lamp-box visual acuity charts, assess the validity of the screening results, and perform a preliminary comparison of the cost inputs of the two approaches.

Methods:

In this cross-sectional study conducted in Wuhan, China in 2017, vision tests were performed on the same day among enrolled primary school students by using the two approaches. The t-test and kappa test were performed to compare the differences, and the indicators of validity were calculated and receiver operating characteristic (ROC) curves were drawn. Existing cost-input data were collected and the budget was analyzed.

Results:

In total, 1001 schoolchildren were included, and the prevalence of low eyesight was 21.18% (95% CI: 18.71–23.87%). The test results of the two approaches were not statistically different (t=1.929, P>0.05) and showed moderate consistency (kappa=0.519, P<0.001). Sensitivity and specificity of the instrument were 84.90% (95% CI: 79.21–89.30%) and 91.63% (95% CI: 89.42–91.64%), respectively; positive predictive value was 73.17% (95% CI: 67.10–78.51%); and negative predictive value was 95.76% (95% CI: 94.00–97.04%). Area under the ROC curve was 0.883 (95% CI: 0.853–0.913) and significantly differed from 0.5 (P<0.001). The budget when using the instrument decreased 48.07% compared to that when using lamp-box visual acuity charts.

Conclusion:

The test result of the instrument is reliable, and using it to conduct screening is cost-saving. Therefore, it might be popularized for vision monitoring in schoolchildren.

Introduction

There are four levels of visual function according to the International Classification of Diseases-10 (Update and Revision 2006), including normal vision, moderate visual impairment (VI), severe VI, and blindness (1). Moderate VI and severe VI are grouped under the term “low vision”. Low vision taken together with blindness represents all VI. The preventable costs of VI were as high as 80% of the total global burden in 2010 (2). VI is a serious public health problem that threatens human health.

According to data from the WHO, 285 million (4.25%) people are estimated to be visually impaired worldwide, 39 million (0.58%) are blind, and 246 million (3.65%) have low vision (3). An estimated 19 million children under 15 yr old are visually impaired: 1.4 million are blind and 17.6 million have low vision (4). Recent estimates suggest that untreated and uncorrected refractive errors (UREs) are the top causes of moderate to severe VI, accounting for 53% of cases, and the second leading cause of blindness, accounting for 21% of cases. Of these visually impaired children, 12 million have VIs caused by refractive errors (4). Refractive errors have a high prevalence in many parts of the world (5–9). As one of the results of UREs, myopia has become increasingly prevalent in modern life, especially in Asian countries (10, 11).

Therefore, early detection and control of refractive errors is of considerable importance. Many countries have dedicated vision screening programs to identify vision problems at a very young age in children. These programs in different countries, such as Great Britain and the United States, are conducted by different personnel like teachers, nurses, or simply trained health care workers using various measurement techniques (12).

In China, the Chinese Ministry of Education requires at least two visual function examinations a year for primary school students. While the diagnosis of refractive errors requires a specific instrument used by a professional optometrist, it is too expensive to adopt this approach for visual monitoring of all the students. Currently, the generally adopted monitoring method is the use of lamp-box visual acuity charts based on the standard logarithmic visual acuity chart (13), which was issued by the national standardization administration of National Health and Family Planning Commission of the People’s Republic of China to identify students who have low eyesight, as these students were more likely to have refractive errors. The students found to have low eyesight undergo further optometry and other eye examinations.

However, owing to the large number of students to be tested, the manpower and capital costs of using lamp-box visual acuity charts still remain high. In addition, the test results of all students need to be uniformly input to build a database, and this leads to a longer feedback cycle. Considering these deficiencies in the current system, the Wuhan Youth Low-Eyesight Protection & Treatment Center, a professional technical institution, adopted the smart vision screening instrument to replace lamp-box visual acuity charts for initial vision screening to identify students with low eyesight.

We aimed to address the following: 1) comparison of differences in the test results of the two screening approaches; 2) the validity of the screening results of the smart vision screening instrument to identify students with low eyesight; and 3) the preliminary comparison of the cost inputs of the two screening approaches.

Materials and Methods

This cross-sectional study was conducted to assess the test results and validity of the smart vision screening instrument. The target population of the study was 7- to 12-yr-old primary school students of Wuhan, China. Data for cost input analysis were obtained from the statistics collected by monitoring 191 primary schools in 2016, and the budget of using the smart vision screening instrument was estimated to perform a preliminary comparison.

Sample Size Calculation

Factors related to the sample size for evaluating the validity of the screening test were as follows: sensitivity and specificity, level of significance test α, and allowable error δ. If the value of both sensitivity and specificity were close to 50%, the sample size could be calculated according to the following formula:

$$n={\left[\frac{z_{\alpha }}{\delta }\right]}^2(1-p)p,$$

Where z_α is the value of z while the cumulative probability was α/2 in the normal distribution; δ represents the allowable error; and p is the estimation value of sensitivity or specificity of the screening test. The sample size of the case group was usually estimated using sensitivity while that of the control group was estimated using specificity.

The estimated values of sensitivity and specificity of the smart vision screening instrument were 80% and 90%, respectively. The distribution of the sample rate was skewed, and hence, arc sine transformation was needed to obtain the square root of the rate. The formula used for calculating the sample size was as follows:

$$n={\left[\frac{57.3\times z_{\alpha }}{\mathit{si}{\mathit{n}}^{-1}\left(\delta /\sqrt{p(1-p)}\right)}\right]}^2,$$

α=0.05, δ=0.10.

The calculation result of the case group was 60, and the prevalence of low eyesight among 7- to 12-yr-old schoolchildren in the city of Wuhan was 34.41% in 2016. Therefore, each group required at least 175 children. Considering the variance of prevalence, response rate, and other possible influencing factors, 450 students were planned to be included in each group. According to the characteristics of the learning burden that may have an impact on vision, the selected students were divided into three groups: 7- to 8-yrold students in the low grades who accept initial education, 9- to 10-yr-old students in the middle grades, and 11- to 12-yr-old students in the high grades with a greater learning burden.

Sampling Method

A multistage sampling method was applied to obtain samples. First, five schools were randomly selected, including Long March Primary School, Triangle Lake Elementary School, Mountain Eagle Primary School, Yucai Second Primary School, and Yucai Experimental Primary School. Second, an equal number of students in each grade of the elementary schools were randomly selected after making necessary arrangements with the local Bureau of Education. Consent forms were signed by guardians of the selected students who participated in the tests.

Smart Vision Screening Instrument

The smart vision screening instrument is an electronic product for visual screening based on the standard logarithmic visual acuity charts developed from the Snellen-E chart. Details of this instrument and a simplified testing procedure are presented in Fig. 1. Before conducting visual acuity testing, the school physician imported an MS Excel spreadsheet with basic information of all the students in a class to the operating system to build electronic visual archives that can be permanently stored. Students completed their own test according to voice instructions of the instrument. After each test, the instrument automatically saved the test result to the personal electronic visual archive. After completing the tests for a class, the school physician uploaded all the results to a unified database.

Fig. 1:

Overview of the Elements and Simplified Testing Procedure of the Smart Vision Screening Instrument (A) The display mainly includes a screen and a keyboard. The basic function of the screen is displaying an E glyph optotype according to the standardized testing process and other settings, such as confirming the personal information of the tested student. The keyboard is used to operate the detection system for network connection, data import and export, etc. (B) The operating section connected by an extended universal serial bus is for the tested students to choose the direction of the E glyph optotype according to their own judgment. They can use the green button to confirm their identity and the red button when they fail to identify the direction. (C) Schematic of the testing process. (D) Tips for the eyes to be tested. (E) The display shows the precise E glyph optotype and can automatically convert to the next target when the time for judgment is over

Procedure

Before the examination day, two rooms were selected by study optometrists to ensure standardized detection conditions. One room was for the test with lamp-box visual acuity charts by a skilled operator according to the technical standard for physical examination of students issued by the national standardization administration of National Health and Family Planning Commission of the People’s Republic of China (14), and the other was for the test using the smart vision screening instrument. On the examination day, after an initial interview, the students entered the first examination room and were tested for uncorrected eyesight acuity. Following a rest of few minutes for alleviating the eye strain caused by the first test, the students entered the second examination room to undergo the smart vision screening instrument test. The order in which the students took both the examinations was random. The results of the two tests were recorded separately and then merged into a database.

Definitions

The objective of screening was to initially identify students with low eyesight. The cutoff point for uncorrected eyesight acuity was set at 1.0. Visual acuity worse than 1.0 in at least one eye was considered to indicate low eyesight.

Statistical Analysis

Paired t-test and kappa test were conducted to compare the differences between the two screening approaches. Uncorrected eyesight acuity test results of lamp-box visual acuity charts were regarded as the gold standard for identifying students with low eyesight and to assess the validity of the smart vision screening instrument. The sensitivity, specificity, positive predictive value, negative predictive value, and likelihood ratio (positive and negative) were calculated. All results are reported with their 95% confidence interval (CI). Receiver operating characteristic (ROC) curves were used to compare the area under the curves in the different groups.

Ethical approval

The Ethics Committee of the School of Medicine, Wuhan University, approved the study protocol (Approval number: 2016103050005), which was conducted in accordance with the tenets of the Helsinki Declaration. All participants’ guardians signed written informed consent forms, which clarified that the children/parents were under no obligation to participate. The basic information of the schoolchildren, including age, school, and grade, were recorded and kept confidential. Necessary eye care services were provided, and children requiring further diagnostic assessment or treatment were provided with an explanation and referred to a hospital/clinic.

Results

In total, 1350 students from five primary schools in the city of Wuhan were selected through multistage sampling, and 1001 of them participated in the study (74.15% response rate). The number of students in each group and the prevalence of low eyesight are shown in Table 1.

Table 1:

Prevalence of Low Eyesight and Differences of Results of Two Screening Approaches

Age(yr)	Participants	Low eye-sight	Prevalence%	χ2	Mean of differences of two test results	t	Kappa
7∼8	295(29.47%)	41	13.90 (10.26–	23.153 ***	0.015(0.006–0.023 )	3.466 *	0.449 ***
9∼10	428(42.76%)	87	20.33 (16.68–24.52)		0.001(−0.004–0.007)	0.541 **	0.557 ***
11∼12	278(27.77%)	84	30.22 (24.95–36.04)		0.003(−0.012–0.005)	0.814 **	0.513 ***
Total	1001(100%)	212	21.18 (18.71–23.87)	-	0.004(−0.001–0.008)	1.929 **	0.519 ***

Note: yr: year-old, χ²: Chi-square test result, t: paired t-test results, Kappa: simple kappa coefficient.

^*P<0.01

^**P >0.05

^***P <0.001

The total prevalence was 21.18% (95% CI: 18.71–23.87%), and prevalence was significantly different among the three age groups (χ²=23.153, P<.0001). Table 1 also displays the differences in the test results of the two approaches in the different age groups. The overall mean of differences was 0.004 (95% CI: −0.001–0.008), which was not statistically different from 0 (t=1.929, P>0.05), and the difference in students aged 7–8 yr old was significant. The results of the two approaches showed moderate consistency (kappa=0.519, P<0.001).

Table 2 presents the results of the validity indicators in the different age groups. The sensitivity and specificity rates of uncorrected vision testing by using the smart vision screening instrument were 84.90% (95% CI: 79.21–89.30%) and 91.63% (95% CI: 91.64–89.42%), respectively, and the positive and negative predictive values were 73.17% (95% CI: 67.10–78.51%) and 95.76% (95% CI: 94.00–97.04%), respectively.

Table 2:

Validity of Smart Vision Screening Instrument According to Age

Age(yr)	Sensitivity%	Specificity%	Positive predictive value%	Negative predictive value%	+LR(C) *	+LR(W) *	−LR(C) *	−LR(W) *	AUG
7∼8	78.05 (61.87–88.89)	90.16 (85.65–93.41)	56.14 (42.4–69.02)	96.22 (92.70–98.14)	7.93 (5.28–11.90)	1.28 (0.88–1.86)	0.24 (0.14–0.43)	0.04 (0.02–0.07)	0.841 ** (0.764–0.918)
9∼10	85.06 (75.44–91.49)	92.38 (88.89–94.86)	74.00 (64.10–82.02)	96.04 (93.15–97.78)	11.16 (7.6–16.31)	2.85 (2.00–4.04)	0.16 (0.10–0.27)	0.04 (0.02–0.07)	0.887 ** (0.841–0.934)
11∼12	88.10 (78.75–93.83)	92.27 (87.33–95.46)	83.15 (73.40–89.95)	94.71 (90.21–97.29)	11.39 (6.96–18.64)	4.93 (3.08–7.90)	0.13 (0.07–0.23)	0.06 (0.03–0.10)	0.902 ** (0.856–0.947)
Total	84.90 (79.21–89.30)	91.63 (89.42–91.64)	73.17 (67.10–78.51)	95.76 (94.00–97.04)	10.15 (8.00–12.88)	2.73 (2.19–3.40)	0.16 (0.12–0.23)	0.04 (0.03–0.06)	0.883 ** (0.853–0.913)

Note: yr: year-old, AUG: the area under ROC curve.

^*LR: Likelihood Ratios [C] = conventional [W] = weighted by prevalence

^**P<0.001

The positive likelihood ratios were 10.15 (95% CI: 8.00–12.88) conventional and 2.73 (95% CI: 2.19–3.40) weighted by prevalence, and the negative likelihood ratios were 0.16 (95% CI: 0.12–0.23) conventional and 0.04 (95% CI: 0.03–0.06) weighted by prevalence. Both the sensitivity and positive predictive value increased with age; however, the specificity and negative predictive value were not very different between the three groups. Fig. 2 and Table 2 illustrate the ROC curve charts for uncorrected vision testing by using the smart vision screening instrument versus lamp-box visual acuity charts and the areas under the ROC curves of the different groups. The overall result showed that the area under the ROC curve was 0.883 (95% CI: 0.853–0.913), which was significantly different from 0.5 (P<.0001). The areas under the ROC curves of the different groups increased with age and were significantly different from 0.5 (P<.0001).

Fig. 2:

ROC Curves of Three Age Groups: Use Uncorrected Visual Screening Results by Smart Vision Screening Instrument

Table 3 reveals the cost of uncorrected visual screening using lamp-box visual acuity charts and the estimation of costs of the smart vision screening instrument for conducting one screening in 191 primary schools in Wuhan. The former method included four phases: organization, field investigation and statistics, establishment of electronic archives, and collection of information feedback, and its total cost was 1,910,380 RMB. The cost of the new method, including field investigation of the detection environment, equipment, operating system development, upgrade, and maintenance, was estimated to be 992,080 RMB, thereby presenting a 48.07% decrease compared to the former method.

Table 3:

Validity of Smart Vision Screening Instrument According to Age

Age (yr)	Sensitivity%	Specificity%	Positive predictive value%	Negative predictive value%	+LR (C) *	+LR (W) *	−LR (C) *	−LR (W) *	AUG
7∼8	78.05 (61.87–88.89)	90.16 (85.65–93.41)	56.14 (42.4–69.02)	96.22 (92.70–98.14)	7.93 (5.28–11.90)	1.28 (0.88–1.86)	0.24 (0.14–0.43)	0.04 (0.02–0.07)	0.841 ** (0.764–0.918)
9∼10	85.06 (75.44–91.49)	92.38 (88.89–94.86)	74.00 (64.10–82.02)	96.04 (93.15–97.78)	11.16 (7.6–16.31)	2.85 (2.00–4.04)	0.16 (0.10–0.27)	0.04 (0.02–0.07)	0.887 ** (0.841–0.934)
11∼12	88.10 (78.75–93.83)	92.27 (87.33–95.46)	83.15 (73.40–89.95)	94.71 (90.21–97.29)	11.39 (6.96–18.64)	4.93 (3.08–7.90)	0.13 (0.07–0.23)	0.06 (0.03–0.10)	0.902 ** (0.856–0.947)
Total	84.90 (79.21–89.30)	91.63 (89.42–91.64)	73.17 (67.10–78.51)	95.76 (94.00–97.04)	10.15 (8.00–12.88)	2.73 (2.19–3.40)	0.16 (0.12–0.23)	0.04 (0.03–0.06)	0.883 ** (0.853–0.913)

Note: yr: year-old, AUG: the area under ROC curve.

^*LR: Likelihood Ratios [C] = conventional [W] = weighted by prevalence

^**P<0.001

Discussion

Vision health is significant to children’s learning, living (15–18), and intellectual development (19). Once vision is impaired, the course of poor vision is longer in children than in adults, and uncorrected VI will have a great impact on their physical and mental health, education, and future employment (2), while simultaneously being a great economic burden on the family and society (20–24). Eighty one percent of VI could be prevented or cured if diagnosed and treated early, 1.4 million children could benefit from vision rehabilitation, and 102 billion US $ could be saved by using appropriate eye care services (25). Hence, timely screening, detection, and intervention are key to preventing VI and helping avoid long-term complications that impact the quality of life (12). Considering refractive error is one of the main causes of VI, early detection and correction of refractive error is of great importance.

The smart vision screening instrument was used to continuously monitor the visual function of all the schoolchildren to identify students with low eyesight, so as to provide more detailed eye examination for this group of children. However, its validity has not been verified, and this study adopted repeated measures to verify the consistency of its screening results with the standard diagnosis results. Moreover, the budget of screening was also analyzed and compared with that of screening using lamp-box visual acuity charts.

The principal finding of this study was the validity of the smart vision screening instrument. The test results of the instrument were not statistically different from the standard test results, and the two test results showed moderate consistency. The sensitivity of screening was 84.90% (95% CI: 79.21–89.30%), which was close to that determined by community eye-health workers and teachers in the Indian state of Andhra Pradesh (26), and the sensitivity was higher than that determined by teachers in Ludhiana (27), teachers in Udaipur City, Western India (28), and the national programs for vision screening of school-children in Iran (27, 29), whereas it was lower than that reported for an instrument based on a computer program (30). The specificity of the smart vision screening instrument is 91.63% (95% CI: 89.42–91.64%), which is similar to that reported in some studies conducted in India (26–28, 31). The area under the ROC curve of the overall diagnosis result is 0.883 (95% CI: 0.853–0.913).

Indicators of different age groups show that the results are more credible in the high age group, and it may be related to better compliance among students in higher grades. This may suggest that the results can be more reliable if guidance is provided to the students before the test. As a means of initial visual inspection, the new screening instrument can identify students with low eyesight, thereby enabling them to seek further eye examinations.

The budget analysis of this instrument revealed it helped save cost. Compared to the traditional screening method, the new instrument enables a paperless process, requiring no professional testers to complete each test and no data recorders to input the test results and build a database. Only a single teacher or school physician is necessary to maintain the test order and to ensure all the students complete their tests. Thus, this new instrument can also save a lot of manpower.

The prevalence of VI worldwide has decreased since the early estimates were obtained in the 1990s. However, it is estimated that the number of people with VI could increase owing to population growth and ageing (4).

The geographical distribution of individuals with VI is uneven worldwide. About 90% of the visually impaired live in low-income settings, and a higher level of socioeconomic development and more health investment correspond to a lower prevalence of VI (32). Therefore, VI remains a major health and social issue in a vast country that has a large population and where providing access to health care and education remains a challenge (33). Considering the validity, cost-saving feature, and technical advantages, the smart vision screening instrument may be suitable for providing initial eye care services to school children in developing countries or countries with large populations.

Limitations

The test result, i.e., low eyesight, is only a judgment of the performance of visual function, and not a clinical diagnosis that reflects the exact health status of the tested eyes. This study only discussed the feasibility of the instrument to replace the current screening approach. The health status of the eyes of the subjects was undiagnosed, and hence, we cannot analyze its validity of screening for refractive errors. In addition, in the cost comparison analysis, we had access to a comprehensive record of the cost of screening using lamp-box visual acuity charts. However, owing to the limited scope of the new instrument we tested, only a rough budget of using the instrument could be estimated on the basis of a limited range of vision screening work; therefore, there may be some unforeseen differences between the budgets and real costs.

Conclusion

The test result of the smart vision screening instrument was acceptable, and it could save the cost inputs compared to that of lamp-box visual acuity charts. Therefore, this screening instrument might be popularized for initial vision screening in schoolchildren.

Ethical considerations

Ethical issues (Including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, redundancy, etc.) have been completely observed by the authors.