A new computer-based pediatric vision-screening test

Abstract

Background

We developed and validated the Jaeb Visual Acuity Screener (JVAS), a computerized visual acuity-based screening program for children that employs a rapid, age-specific, standardized algorithm for vision screening in the medical home that is available for download at no cost.

Methods

A total of 175 children aged 3 to <8 (median, 6) years were screened with the JVAS before undergoing a complete eye examination (gold standard). The JVAS presented 2 large single surround optotypes (20/100 and 20/80) and then 5 optotypes at a predetermined, age-specific normal threshold. Failure on the gold standard examination was determined using recently published referral criteria and published visual acuity norms for age. We evaluated the sensitivity and specificity of the JVAS for detecting reduced visual acuity, amblyopia, and amblyopia risk factors. JVAS pass/fail paradigms evaluated were inability to identify 3 of 4, 3 of 5, and 4 of 5 age-appropriate optotype presentations.

Results

Screening testability for the JVAS was high, at 100%. Sensitivity of the JVAS ranged from 88% to 91%, and specificity from 73% to 86%, with positive predictive value ranging from 66% to 79% and negative predictive value from 92% to 93% (ranges reflect different pass/fail paradigms).

Conclusions

The new JVAS provides an effective and practical method for screening 3- to 7-year-olds using any Windows-based computer. Providing the JVAS free-of-charge to pediatricians and school systems would standardize currently fragmented visual acuity-based screening practices.

Amblyopia, strabismus, and refractive error are the most prevalent eye disorders in young children.^1-3 Nevertheless, such conditions are often largely asymptomatic and therefore may remain undetected unless testing is performed to identify them. Early childhood screening for amblyopia and/or amblyopia risk factors has been recommended^4,5 because early identification of amblyopia and amblyopia risk factors may result in more effective treatment.^6,7 Effective vision screening in young children should identify those most likely to benefit from early intervention.⁵

There are many different approaches to screening children for vision problems.⁸ Effective screening approaches are those that can be applied to the general population, can correctly identify a high proportion of those with disease (high sensitivity), and identifying a high proportion of healthy individuals as normal (high specificity). Currently, the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) vision screening guidelines recommend age-appropriate vision assessment for children aged 3 years and older.⁹ In addition, the recently updated AAPOS Pediatric Vision Screening guidelines recommend “direct detection of visual impairment using acuity-based screening . . .” as the gold standard for preschool and older children.⁹ To improve our ability to achieve early detection of reduced visual acuity due to amblyopia or refractive error, it is desirable to have a valid, easy-to-use, standardized visual acuity–based screening test available at no charge, which could be used to rapidly identify subnormal visual acuity in pediatrician offices and in schools. With this goal in mind we developed the Jaeb Visual Acuity Screener (JVAS), a computerized visual acuity–based screening application for young children. This study provides pilot data on the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the JVAS to detect the presence of reduced visual acuity, amblyopia, and amblyopia risk factors.

Subjects and Methods

Approval was obtained from the Mayo Clinic Institutional Review Board, and all procedures and data collection were conducted in a manner compliant with the US Health Insurance Portability and Accountability Act of 1996. Informed consent was obtained from the child's parent or legal guardian, and all research procedures adhered to the tenets of the Declaration of Helsinki.

Subjects

We recruited children presenting to Pediatric Optometry, Mayo Clinic for a full eye examination, including cycloplegic refraction. To better represent a screening population we did not enroll children with a history of spectacle use in the previous 6 months, because spectacle wear may improve uncorrected visual acuity. We also did not enroll children with a history of amblyopia treatment, or those who had undergone previous ocular surgery. Patients presenting with strabismus that was evident (by history) to the parent or guardian were also not recruited since these children would normally be referred for formal eye examinations without the use or need for vision screening.

Jaeb Visual Acuity Screener (JVAS) Program

The Jaeb Visual Acuity Screener (JVAS) is a computerized screening program that employs an optotype testing algorithm to test at published age-referenced normal visual acuity thresholds. The program is designed to identify children with subnormal visual acuity in a rapid and reproducible manner, and can run on any Windows-based computer (Windows XP Service Pack 2 or newer). Because the software relies on a set algorithm to generate the appropriate testing optotype, subjective tester bias is minimized. The JVAS is available for download from the Jaeb Center for Health Research at no charge (www.pedig.net/JVAS.aspx, accessed January 14, 2015). For 3- to <7-year-old children, single surround HOTV optotypes are used and for children 7 years and older, single-surround Early Treatment of Diabetic Retinopathy Study (ETDRS) optotypes are used.

JVAS Testing Procedure

The subject completed vision screening using the JVAS before the start of their standard eye examination, and JVAS testing was performed by a trained ophthalmic technician. The patient was positioned 5 feet from the computer display. A testing distance of 5 feet was chosen with a view to enabling easy administration of the test from a computer screen in a pediatric office or community setting. The child's left eye was patched and the screener then selected the age of the patient as either: 3-5 years, 6 years, or ≥7 years, and clicked “Start screening.” The JVAS then sequentially displayed 2 large single surround letters (20/100 and 20/80). The patient responded by either matching the presented optotype to a matching card (used if needed for subjects 3 to <7 years old) or by verbally identifying the presented letter. The screener selected the appropriate “correct” or “incorrect” radio button and continued testing by selecting the “next letter” button. After presentation of the 20/80 optotype, the subsequent optotype size was the normal threshold for the child's age. Normal visual acuity thresholds, agreed by consensus at the time the JVAS was first conceived, were set as the “pass” threshold for different age groups: 3- to 5-year-olds were required to pass 20/40 (using HOTV), 6-year-olds were required to pass 20/32 (using HOTV) and 7-year-olds were required to pass 20/25 using ETDRS letters. A total of up to 5 letters were presented at the age-appropriate normal visual acuity threshold.

The right eye was then patched and the screening process repeated for the left eye. After screening was completed, the JVAS display showed either “pass” or “fail” for each eye for each of the three pass/fail criteria: 3 of 4 letters correct, 3 of 5 letters correct, and 4 of 5 letters correct. The time elapsed for screening both eyes from first letter to last letter presentation was also displayed on the screen.

Complete Eye Examination

After JVAS testing was performed, the pediatric optometrist performed a complete eye examination as per usual clinical practice. In order to minimize potential bias, the pediatric optometrist did not review the JVAS screening results until the eye examination was completed. The eye examination included measurement of visual acuity using the Amblyopia Treatment Study HOTV (ATS-HOTV) visual acuity protocol^10,11 for <7-year-olds and the electronic ETDRS visual acuity protocol for 7-year-olds.¹² Ocular alignment testing was performed at distance and near fixation using simultaneous prism cover test and prism and alternating cover test; ocular motility was assessed and pupils were examined. Cyclopentolate 1% was administered to enable cycloplegic refraction and slit lamp examination of anterior and posterior segments through dilated pupils. If gold-standard visual acuity testing was failed, repeat visual acuity testing (using ATS-HOTV or electronic ETDRS) was performed for the failed eye(s) following cycloplegic refraction, with the appropriate refractive correction in place.

Gold Standard Failure Criteria

The complete eye examination and cycloplegic refraction were considered the gold standard for analysis purposes. Failure on the gold standard examination was based on updated American Association for Pediatric Ophthalmology and Strabismus Vision Screening Guidelines (Table 1),⁴ with visual acuity failure based on the most recent normative pediatric visual acuity data as published by Pan and colleagues¹³ for children aged 30-72 months and Drover and colleagues ¹⁴ for children aged ≥72 months (Table 1). In this way, our study evaluated the ability of the JVAS to detect reduced visual acuity, amblyopia, and amblyopia risk factors.

Table 1.

Gold standard American Association for Pediatric Ophthalmology and Strabismus eye examination failure criteria,⁴ with visual acuity failure criteria based on published normative data.^13,14

Age	Refractive error—thresholds for failure on gold standard exam
	Astigmatism	Hyperopia (any meridian)	Anisometropia	Myopia (any meridian)
31-48 months	>2.0 D	>4.0 D	>2.0 D	> −3.0 D
>48 months	>1.5 D	>3.5 D	>1.5 D	> −1.5 D
	Visual acuity—thresholds for failure on gold standard exam
30-47 months (2.5 to <4 years)13	Worse than 20/50
48-59 months (4 to <5 years)13	Worse than 20/40
60-83 months (5 to <7 years)13, 14	Worse than 20/32
≥84 months (7+ years)14	Worse than 20/25
Any age	Interocular difference of ≥3 lines
	Alignment
Any age	Manifest strabismus of >8 PD in primary position
	Media
Any age	Media opacity >1 mm

D, diopter; PD, prism diopter.

Determining Cause of Gold Standard Failure

For gold standard examination visual acuity failures, retesting of visual acuity (following cycloplegic refraction) with refractive correction in place enabled further classification regarding the likely cause of reduced visual acuity, including determining the presence of amblyopia. The cause of gold standard failure was assigned to one of seven categories (Table 2). Cause of failure was classified hierarchically as follows:

1. Uncorrected refractive error alone: visual acuity improved to within normal for age with refractive correction in place; no strabismus or media opacity.

2. Media opacity: media opacity diagnosed and visual acuity remained subnormal for age with any refractive correction in place, with no strabismus, and refractive error less than previously published normal visual acuity thresholds.¹⁵

3. Unilateral amblyopia: uniocular visual acuity remained subnormal for age with refractive correction in place, interocular difference ≥3 lines, and refractive error or strabismus present. Previously published normal visual acuity thresholds were used to establish the presence of amblyogenic refractive error¹⁵ (Table 2), that is, ≥1.00 D spherical equivalent anisohyperopia or ≥3.00 D spherical equivalent anisomyopia, or ≥1.50 D anisoastigmatism.¹⁵

4. Bilateral amblyopia: visual acuity subnormal for age with refractive correction in place and refractive error in both eyes met specific previously published criteria¹⁶ (Table 2), that is, ≥+4.00 D spherical equivalent, or ≥ +2.00 D cylinder, or ≥ −6.00 D spherical equivalent.¹⁶

5. Refractive error alone: refractive error failed by gold standard thresholds (Table 1), but uncorrected visual acuity was normal for age.

6. Unexplained visual acuity deficit: visual acuity remained subnormal for age with refractive correction in place and refractive error less than previously published normal thresholds,¹⁵ with no strabismus or media opacity present.

7. Ocular misalignment: manifest strabismus >8^Δ in primary position (gold standard examination failed for no other reason) (Table 2).

Table 2.

Breakdown of the causes of gold standard eye examination failure (N = 65)^a

Classification	Description	Number (%)
Reduced VA due to uncorrected refractive error	VA improved to normal for age with refractive correction in place; no strabismus or media opacity	N = 31 (48)
Media opacity	VA remained subnormal for age with refractive correction in place and refractive error (applying published criteria15) or strabismus not present, but media opacity diagnosed	N = 0 (0)
Unilateral amblyopia	VA remained subnormal for age with refractive correction in place, with interocular VA difference of ≥3 lines4 and refractive error and/or manifest strabismus present.	N = 7 (11)
	Amblyogenic refractive error defined according to previously published criteria15:
	• ≥1.00 D SE anisohyperopia
	• ≥3.00 D SE anisomyopia
	• ≥1.50 D anisoastigmatism
Bilateral amblyopia	VA remained subnormal for age in both eyes with refractive correction in place and refractive error in both eyes was either:	N = 8 (12)
	• ≥ +4.00 D SE
	• ≥ +2.00 D cylinder
	• ≥ −6.00 D SE16
Refractive error alone (normal VA)	Refractive error failed by gold standard thresholds (Table 1), but uncorrected VA was normal for age.	N = 8 (12)
Unexplained reduced VA	VA remained subnormal for age with refractive correction in place and refractive error (applying published criteria15); manifest strabismus or media opacity not present	N = 10 (15)
Ocular misalignment	Manifest strabismus >8 PD in primary position	N = 1 (2)

D, diopters; PD, prism diopters; SE, spherical equivalent; VA, visual acuity.

^aWhere there was more than one reason for failing the gold standard eye examination, cause of failure was classified hierarchically as follows: (1) uncorrected refractive error; (2) media opacity; (3) unilateral amblyopia; (4) bilateral amblyopia; (5) refractive error alone; (6) Unexplained reduced visual acuity. Patients were classified in the manifest strabismus category if a tropia >8 PD was present, but VA was normal and refractive error did not meet gold standard failure criteria.

Analysis

Subjects able to complete visual acuity testing using the JVAS and visual acuity testing in the gold standard examination were included for analysis. JVAS screening results were compared to the gold standard results and sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. Three different JVAS failure criteria were evaluated: (1) failure to identify at least 3 of any 4 normal threshold letters in either eye (JVAS 3 of 4); (2) failure to identify at least 3 of any 5 normal threshold letters in either eye (JVAS 3 of 5); (3) failure to identify at least 4 of any 5 normal threshold letters in either eye (JVAS 4 of 5).

As a secondary analysis, we also calculated the sensitivity, specificity, PPV, and NPV of the JVAS for detecting visual acuity <20/30, ≥3 lines interocular visual acuity difference, and manifest strabismus as suggested by the AAPOS Vision Screening Committee reporting guidelines for nonrefractive screening instruments.⁴

Results

A total of 186 subjects were enrolled; of these, 11 (6%) did not complete gold standard visual acuity testing and were therefore excluded, leaving 175 subjects for analysis (86 females [49%]; 134 white [77%]). No subjects were excluded due to inability to perform the JVAS. Median age of subjects was 6 years (range, 3 to 7 years). Twelve patients (7%) were 3 years old; 33 (19%), 4 years old; 29 (17%), 5 years old; 69 (39%), 6 years old; and 32 (18%), 7 years old. The mean elapsed screening time on the JVAS was 84 ± 43 seconds (range, 23-357 seconds; 63-208 seconds in 3-year-olds, 40-357 seconds in 4-year-olds, 31-167 seconds in 5-year-olds, 23-186 seconds in 6-year-olds, and 42-211 seconds in 7-year-olds).

Gold Standard Failure

Of the 175 children, 65 (37%) failed the gold standard examination. Overall, reasons for failing the gold standard were reduced visual acuity (n = 56 [86%]), hyperopia (n = 23 [35%]), astigmatism (n = 15 [23%]), anisometropia (n = 7 [11%]), myopia (n = 6 [9%]), and strabismus (n = 3 [5%]), with some patients failing the gold standard for more than one reason. For the 56 children failing the gold standard examination for reduced visual acuity, the primary cause assigned hierarchically was uncorrected refractive error in 31 of 56 (55%), unilateral amblyopia in 7 (13%), bilateral amblyopia in 8 (14%), and for 10 (18%) the cause of reduced visual acuity was unexplained (Table 2).

JVAS Sensitivity and Specificity

For overall gold standard failure, JVAS sensitivity was 89% by the 3 of 4 failure criteria, 88% by the 3 of 5 criteria and 91% by the 4 of 5 criteria (Table 3). JVAS specificity was 75% by the 3 of 4 failure criteria, 86% by the 3 of 5 criteria, and 73% by the 4 of 5 criteria (Table 3). The PPV ranged from 66% to 79% and NPV was 92% to 93% (Table 3).

Table 3.

Sensitivity, specificity, positive predictive value and negative predictive value of the JVAS screening test overall for detecting gold-standard eye examination failures

Screening test	Sensitivity	Specificity	PPV	NPV
JVAS 3 of 4	89%	75%	67%	92%
JVAS 3 of 5	88%	86%	79%	92%
JVAS 4 of 5	91%	73%	66%	93%

JVAS, Jaeb Visual Acuity Screener; NPV, negative predictive value; PPV, positive predictive value.

We also found excellent sensitivity and specificity of the JVAS (Table 4), with an alternative analysis using AAPOS Vision Screening Committee reporting guidelines for nonrefractive screening instruments, which suggests reporting sensitivity and specificity to detect visual acuity <20/30, ≥3 lines interocular visual acuity difference, and manifest strabismus.⁴

Table 4.

Sensitivity, specificity, positive predictive value, and negative predictive value of the JVAS screening test for detecting visual acuity <20/30, ≥3 lines interocular visual acuity difference, and manifest strabismus as suggested by the AAPOS Vision Screening Committee reporting guidelines for non-refractive screening instruments.⁴

Screening test	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value
JVAS 3 of 4	93%	73%	63%	96%
JVAS 3 of 5	90%	83%	72%	94%
JVAS 4 of 5	93%	70%	61%	95%

JVAS, Jaeb Visual Acuity Screener; NPV, negative predictive value; PPV, positive predictive value.

Discussion

In this study we evaluated the effectiveness of the newly developed JVAS computerized screening test for the detection of gold standard eye examination failures. We found the JVAS had high sensitivity and specificity for the detection of gold standard failures, the majority of whom failed the gold standard for reduced visual acuity.

High sensitivity and specificity are desirable in a vision screening test to enable correct identification of those children with the target condition(s) (ie, high sensitivity) as well as correct identification of those who do not have the target condition (ie, specificity). We found both high sensitivity (88% to 91%, depending on the number of presentations [Table 3]) and high specificity (73% to 86% [Table 3]) for the detection of gold standard failures. Compared with Vision in Preschoolers (VIP) data, we found comparable or higher sensitivity using the JVAS. With JVAS specificity at 86% (to match as closely as possible the 90% specificity value used in VIP studies^17,18), we found sensitivity of 88% for detecting gold standard fails, which compares favorably with VIP overall sensitivity of 61% using single Lea symbols¹⁷ and 37% to 76% using linear Lea symbols,^17,18 (range reflects differences between testers). Using HOTV optotypes, overall sensitivity found by the VIP group was 54%,¹⁸ with which the JVAS again compares favorably. In a study of visual acuity screening using a Lea symbol chart,¹⁹ the PPV was found to be 66.7%, similar to our PPV result of 66% to 79% (Table 3). Although JVAS sensitivity appears higher than sensitivities found for visual acuity tests in the VIP studies, it is difficult to compare directly results when there are differences between studies regarding primary care versus enriched populations, ages of subjects, screening pass/fail thresholds, and the gold standard criteria used.

High testability is another important characteristic of a good screening test, and for visual acuity testing this seems best achieved by using simplified testing algorithms. The JVAS was designed to present 2 large optotypes (20/100 and 20/80), followed by up to 5 optotypes at the child's age-normal threshold. Using this testing paradigm we achieved 100% testability. The VIP study group employed a similar approach using handheld Lea symbols or HOTV optotype cards, presenting 10/100 sized optotypes first, followed by 2 levels of smaller-sized symbols or letters at age-specific normal thresholds.^18,20 Using this approach, testability was high at 98.6% to 99.4%.^18,20 In a VIP study using the Lea symbol wheel, a 5/50 symbol was presented first, followed by 2 levels of smaller-sized symbols at age-specific normal thresholds¹⁷ and testability was again high at 99.4%.¹⁷

The simplified testing algorithm used for the JVAS also enables the test to be successfully completed in a short period of time. The mean JVAS testing time in our cohort of 3-to 7-year-old children was 84 ± 43 seconds, making it very practicable in a screening setting. In addition, the JVAS presentation of optotypes is fully automated with predetermined optotype size and pass-fail criteria, requiring minimal training. Furthermore, the JVAS is available for download at no cost to the end user (www.pedig.net/JVAS.aspx).

The proportion of false positive referrals from a screening evaluation is an important measure of the effectiveness of the screening method or device. A high proportion of false positive referrals (screening test failed but gold standard examination passed), reduces the positive predictive value of the screening test. The number of false positive screening failures using the JVAS was low, ranging from 15 of 175 (9%) with the 3 of 5 paradigm, to 30 (17%), with the 4 of 5 paradigm. Evaluating the distribution of false positives by age, there was no apparent preponderance in the younger age groups for any of the testing paradigms, with the proportion of 3-year-old false positives ranging from 23%, with the 4 of 5 paradigm, to 33%, with the 3 of 5 paradigm. Nevertheless, 3-year-olds made up only 7% of the total study cohort and it is possible that an association of more false positives with younger age was missed due to low numbers of younger children.

The version of the JVAS employed for this present study used a 20/40-sized optotype as the pass threshold for 3-year-olds. However, based on current normative visual acuity data,¹³ and as recommended in the current AAPOS screening guidelines,⁹ the JVAS has been updated to feature a 20/50-sized optotype as the pass threshold for 3-year-olds. Similarly, current normative visual acuity data^13,14 indicate a pass threshold of 20/32 for 5-year-olds, whereas the version of the JVAS used in this study utilized 20/40 as the pass threshold for this age group. Bringing JVAS screening thresholds into line with gold standard failure criteria (Table 1) by lowering the JVAS pass threshold for 3-year-olds to 20/50 and raising the pass threshold for 5-year-olds to 20/32, should improve specificity and sensitivity of the JVAS, respectively.

In the present study, introducing the JVAS as a visual acuity-based screening tool, we evaluated the performance of three different testing paradigms: a pass determined using 3 of 4 optotype presentations, 4 of 5 presentations, and 3 of 5 presentations. Sensitivity was lowest with 3 of 5 presentations (at 88%) and highest with 4 of 5 presentations (at 91%). Nevertheless, specificity was highest with 3 of 5 presentations (86%) and lowest with 4 of 5 presentations (73%). Based on the pilot data reported in this present study, we recommend that the child correctly identify 3 of up to 5 presentations in each eye to achieve a good balance between sensitivity and specificity.

There are some limitations to our study. Because we studied an enriched cohort, our estimates of PPV and NPV are not representative of those that would be derived in the general population and therefore care must be taken when interpreting these values. Since reliable screening of young children is important for early detection of disease, it would be helpful to study a larger number of 3-year-olds to evaluate the effectiveness of JVAS screening in this age group. In addition, the normal visual acuity threshold used for JVAS failure in 3-year-olds was more stringent than the normal visual acuity threshold for failure on the gold standard examination, and reevaluation of JVAS sensitivity and specificity using revised JVAS failure criteria will be informative. A further possible limitation of our method is the 5-foot working distance, which may introduce greater variability than longer working distances. As our method is disseminated, it will be important to stress the necessity of controlling the working distance. Nevertheless, the 5-foot working distance allows generalizability to almost any location.

One reason JVAS specificity and sensitivity was high in this present study is that the majority of our patients failed the gold standard examination due to reduced visual acuity and the JVAS is a visual acuity test. The JVAS may have lower specificity for the detection of high refractive error alone, or ocular misalignment alone. Nevertheless, the detection of reduced visual acuity is the primary purpose of vision screening in childhood.

The JVAS is a useful screening tool for detecting reduced visual acuity in children, providing a simple, rapid, standardized method for pediatric vision screening in the medical home. The JVAS is available free-of-charge as a download from www.pedig.net/JVAS.aspx. Broad adoption of this tool would result in a more standardized approach to pediatric vision screening in diverse medical and community office settings. In comparison to traditional vision testing methodologies, this software-based tool provides the advantage of running on any windows-based PC in a pediatrician's examination room—avoiding testing in distracting office hallways. It provides the advantage of requiring less subjective input by the tester in determining age-appropriate optotype size and type. Finally widespread adoption of such a uniform and standardized testing modality, that is reproducible in any provider's office, would generate a large dataset, which could be subsequently used for public health assessment and quality reporting. We encourage elementary school nurses, pediatricians, and other professionals who work with children aged 3-7 years to download and use the JVAS for their vision screening needs.

Clinical Implications.

The new Jaeb Visual Acuity Screener (JVAS) is a computerized vision screening program that employs optotype testing and published age-referenced normal visual acuity thresholds in a computer-controlled testing algorithm. In this pilot study of 3- to 7-year old children the JVAS showed high sensitivity and specificity for the detection of reduced visual acuity, amblyopia, and amblyopia risk factors, with high testability (100%) and short testing time (average, 1.5 minutes). The JVAS uses a Windows-based computer program, making it easily implementable into pediatrician offices or the school environment. The JVAS therefore provides an effective and practical method for vision screening and is freely available to download (www.pedig.net/JVAS.aspx). Implementation of the JVAS in pediatrician offices and in school systems would standardize currently fragmented screening practices.