Instrument Referral Criteria for PlusoptiX, SPOT and 2WIN Targeting 2021 AAPOS Guidelines

Abstract

Background

The American Association for Pediatric Ophthalmology and Strabismus (AAPOS) issued a 2021 update of Uniform Validation Guidelines for instrument-based pediatric vision screeners. With each update it is important for each manufacturer to update the Instrument Referral Criteria (IRC) programed into their devices in order to optimize sensitivity and specificity to detect AAPOS criteria.

Methods

De-identified data comparing photoscreening with simultaneous confirmatory examinations constituted separate cohorts for the development of IRC via receiver operating characteristic (ROC) curves. The refractions of the devices were also compared.

Results

This study defines three sets of refractive IRC (Sensitive, Medium, and Specific) for three leading infrared photoscreening devices, PlusoptiX A12, Welch-Allyn SPOT, and Adaptica 2WIN for children < 4 and ≥ 4 years in order to better target the 2021 AAPOS guidelines. The cohorts were similar but the SPOT group (n=755, mean age 9) was older with more astigmatism and the 2WIN (n=1362, mean age 7) was younger with more hyperopia and anisometropia compared to the cohort for PlusoptiX A12 (n=616, mean age 8). The age-based, medium magnitude IRC for anisometropia, hyperopia, astigmatism and myopia for SPOT were: <4y: 1.5, 1.75, 3.25, 3.5 and ≥4y: 1.5, 1.75, 2.25, 2.0; for PlusoptiX: <4y: 1.75, 3.0, 3.5, 3.5 and ≥4y: 1.75, 3.0, 2.5, 2.5; and for 2WIN: <4y: 1.75, 2.5, 3.5, 3.5 and ≥4y: 1.5, 2.0, 2.5, 2.0. The mean ABCD ellipsoid spectacle matches differed; SPOT: 1.8±1.3 (better) versus PlusoptiX: 1.9±1.6 and 2WIN: 2.2±1.4 (p<0.001).

Conclusion

The 2021 AAPOS exam guidelines foster early specificity before age 4 and sensitivity after age 4. These evidence-based IRC for current SPOT, PlusoptiX, and 2WIN photoscreeners should allow device manufacturers the data necessary to adjust their device IRC to maximize specificity, sensitivity or a medium between the two. This paper provides practical suggestions for better validation. Improved early screening combined with thorough treatment should reduce life-long vision impairment due to amblyopia.

Video Abstract

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

https://youtu.be/cPqIFE1_Eqo

Introduction

Amblyopia is one of the most common causes of vision impairment in young individuals. Amblyopia detection fits World Health Organization guidelines for health screening in developed countries since it can detect the condition early enough for therapy to favorably alter the natural course.¹ “Conventional screening” can be performed as early as preschool utilizing monocular visual acuity to identify defective brain learning of vision.² Since the mid 1990s, objective measures and specifically photoscreeners enabled earlier identification of amblyopia risk factors (ARFs) which lead to amblyopia.^2,3

The Vision Screening Committee of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) has defined target levels of amblyopia risk factors to unify research and development of objective vision screening instruments. In 2003, consensus levels of spherical and/or cylindrical ARFs were published for preschool children and specifically those aged under or over 42 months.⁴ In 2013, due to a perception that predictive value was too low from the 2003 report, an aged-based update with more specific refractive ARF for children less than 30 months of age was introduced transitioning to more sensitive 2003-level ARFs after 48 months of age.⁵ Unfortunately, the 2013 report lacked knowledge of community ARF prevalence. Independently and concomitantly, the MEPEDS and BPEDS epidemiological studies were discovering community prevalence of ARFs.^6–8 As a result, high prevalence astigmatism and hyperopia (10% and 8%) cases were over-referred by the 2013 guidelines, and less prevalent (1.5%) anisometropia cases were relatively under-referred. As a result, combining knowledge of community ARF prevalence and information on relative amblyopiagenic potential of ARFs (Figure 1), and emphasizing school-performance levels of moderate symmetric myopia and astigmatism, an update from 2003 and 2013 in the AAPOS uniform Amblyopia Risk Factor and Visually Significant Refractive Error guidelines was presented in 2021.⁹

Figure 1.

Refractive Amblyopia Risk Factors (ARFs) “diamond graph” AAPOS diopter cut-offs with prevalence scale under each orange axis with thickness approximating each ARF’s severity. Upper graph: 2003 AAPOS preschool guidelines indicated by red diamond. 2013 age-stratified diamond levels indicated by green (toddlers), purple (preschool) and red (Kindergarten). The lower diamonds show 2021 AAPOS ARF preK cut-offs in blue and school-aged in red attempting to reduce false positives and referral rate, while striving to detect the more severe ARFs early for which the patient cannot easily compensate with accommodation. The prevalence of risk factors is related to the area of each diamond.

Given target uniform amblyopia risk factor levels of certain age ranges, instrument makers and researchers can set customized “instrument referral criteria” (IRC) for individual and combined ARFs. Some IRC are more specific and emphasize passing patients who lack ARFs, while other IRC are more sensitive and emphasize referring more patients who have ARFs. An identical instrument will perform differently when different IRC are used.¹⁰ The natural inverse relationship between sensitivity and specificity for a given device defined by applying different instrument referral criteria is aptly demonstrated by the receiver operating characteristric (ROC) curve.^11,12

The Alaska Blind Child Discovery (ABCD) and KinderSee charitable outreaches have provided instrument-based pediatric vision screening to patients in underserved areas of America and also to other tropical third-world countries.^13,14 The de-identified data from these patients constitute the patient database for this study.

Methods

This comparative evaluation of screening tests study was approved by institutional review boards from facilities associated with our private practices: Providence Alaska Medical Center and Penn Medicine Lancaster General Health. The study complies with the Health Insurance Portability and Accountability Act (HIPAA) and the Declaration of Helsinki. Parents and/or guardians were advised of the purpose of the study and provided consent in their native language and children of sufficient age provided assent.

Three cohorts of patients were studied representing the three leading infrared autorefracting photoscreeners (Figure 2). There were a small number of patients screened with more than one device; 444 with PlusoptiX and SPOT, 51 with PlusoptiX and 2WIN, and 4 with SPOT and 2WIN and another 46 were screened with each device. De-identified data from each of the three separate cohorts of patients are available from https://www.abcd-vision.org/references/IRC%20AAPOS-2021%202%20P%20S.pdf.

Figure 2.

The three leading infrared multiradial photoscreening autorefractors; Adaptica 2WIN (left), PlusoptiX A12 (center) and the Welch Allyn SPOT (right).

Patients were undergoing vision screening and/or comprehensive pediatric eye examinations. As a part of the initial examination before cycloplegia, both eyes were simultaneously screened with a hand-held, automated infrared, multiradial eccentric-flash photorefractor. Then a comprehensive examination was completed including age-appropriate visual acuity, binocularity and strabismus assessment, anterior and posterior segment exam. Expert retinoscopy was performed with accommodation relaxed by age-appropriate means; younger children had cycloplegia with cyclopentolate 1% at least 30 minutes before retinoscopy while older children had fogging and/or SBARS¹⁵ with refraction refined at the phoropter. The refractive components from the examination (not the screening) for each cohort and the total are given in Table 1.

Table 1.

Characteristics of the Three Patient Cohorts for Each of Three Autorefractors; Adaptica 2WIN, Plusoptix A12 and Welch Allyn SPOT. Q1 and Q3 are Interquartile Ranges. K-W is Kruskall-Wallis Expressed as Chi-Squared with 2 Degrees of Freedom. S.D. Is Standard Deviation the Devices Compared with Analysis of Variance (ANOVA) Represented by F Statistic. The Ages, Then the Results from Refined Retinoscopy for Cylinder, Sphere and Anisometropia Defined by Either Spherical Equivalent or Meridional Definition. Finally the Visual Acuity-Compared Refractions are Demonstrated by the Alaska Blind Child Discovery (ABCD) Composite Ellipsoid Metric.

		Count	<4 Years	Median	Q1	Q3	K-W	Mean	S.D.	ANOVA
	Age			7	5	11	X2(2)=86	8	4	F(2,2713)=43
Age	2WIN	1335	331	7	4	10	p<0.001	7	4	p<0.001
	PlusoptiX	628	93	7	5	11		8	4
	SPOT	755	30	8	6	12		9	4
Exam	Cylinder (maximal)						X2(2)=33			F(2,2715)=22
	2WIN			0.75	0.3	2	p<0.001	1.27	1.34	p<0.001
	PlusoptiX			0.75	0.3	2		1.21	1.27
	SPOT			1	0.3	2.75		1.65	1.65
	Sphere (right eye Spherical Equivalent)						X2(2)=0.8			F(2,2715)=2
	2WIN			0.5	−0.5	2	p=0.69	0.75	2.57	p=0.11
	PlusoptiX			0.5	0	1.13		0.6	1.44
	SPOT			0.63	0	1.25		0.57	1.43
	Myopia (meridional)						X2(2)=3.6			F(2,2715)=3
	2WIN			0	−1.3	1	p=0.16	−0.11	2.7	p=0.5
	PlusoptiX			0	−1	0.75		−0.15	1.68
	SPOT			0	−1.3	0.5		−0.36	1.76
	Hyperopia (meridional)						X2(2)=6.3			F(2,2715)=4
	2WIN			1.05	0	3	p=0.04	1.54	2.64	p=0.02
	PlusoptiX			1	0.5	1.75		1.24	1.7
	SPOT			1	0.5	2		1.44	1.69
	Anisometropia (meridional)						X2(2)=11			F(2,2715)=2
	2WIN			0.25	0	0.75	p=0.004	0.65	0.98	p=0.11
	PlusoptiX			0.25	0	0.75		0.55	0.97
	SPOT			0.25	0	0.75		0.63	1.1
Compare	ABCD Ellipsoid Composite						X2(2)=82			F(2,2730)=22
	2WIN			1.92	1.3	2.83	p<0.001	2.19	1.42	p<0.001
	PlusoptiX			1.48	0.9	2.32		1.89	1.57
	SPOT			1.46	1	2.16		1.78	1.34

Some patients were screened with the Welch Allyn SPOT (Skaneateles Falls, New York; software versions Pediavision 1.1.51 and 2.0.16). Instrument referral criteria were determined by the manufacturer as one single, difficult-to-modify, age-based set. The device was used by experienced staff and volunteers following manufacturer’s guidelines. This black device has some soft glowing fixation lights and sounds to attract fixation by the child. The performance and fixation lights of the three devices can be viewed at https://vimeo.com/pdimd/photoscreeners.

Some patients were screened with the PlusoptiX A12 hand-held autorefractor (Nuremberg, Germany) using software versions 6.0. to 7.1.5.0. Plusoptix makes user-selectable IRC simple; typically the instrument referral criteria number 4 of 5 was selected which provides increased specificity. This device has a large picture of a “smiley face” with some colored lights behind and warble sounds to attract fixation.

Other patients were screened with the Adaptica 2WIN photorefractor (Padova, Italy, software version 5.0). The infrared strabismus wand and “CR” function¹⁶ and the Kaleidos luminance-control floor stand^13,17 were not used. The active, bright twinkling colored lights and music were selected to engage fixation of the children except for a few autistic children who preferred the sound turned off.

From each device, the estimate of sphere, cylinder and axis for both eyes was extracted. If misalignment seemed to limit refractive estimation, a single eye feature was used to estimate the clearer, fixing eye. Occasionally out-of-range refractive estimates were encountered with devices either unable to yield a refractive result, or returning “HYP” for excessive hyperopia or “MYP” for excessive myopia.

From the best retinoscopy-directed refraction, sphero-cylinder data were transformed to determine if that patient met or exceeded 2021 AAPOS refractive amblyopia risk factors (Table 1). Whether written in plus-cylinder format or minus-cylinder format, there are several ways hyperopia, myopia and especially anisometropia in the presence of substantial astigmatism can be determined (Figure 3). Consistent with AAPOS 2021 guidelines, we used the most myopic end of the spherocylinder vector for myopic refractions, the most hyperopic meridional value for hyperopic refractions and the difference in the most myopic meridian for anisometropic determination. Separate levels of refractive amblyopia risk factors were made for myopia and astigmatism separated by age less than 4 years or greater than or equal to 4 years.

Figure 3.

Alternative definitions of anisometropia for sphero-cylinder refractions. An advantage of spherical equivalent is identical value whether plus or minus cylinder notation. The values (A) through (F) all could be utilized to calculate “anisometropia” for a sphero-cylinder refraction. Spherical equivalent anisometropia is the absolute value of (B – E); the meridional anisometropia from the more-minus is the absolute value of (A – D).

Determination of IRC

Individual ROC curves for each device and for each subcategory of refractive error was generated over a range exceeding expected levels. The ROC curves were generated from the derived refractions and not from the internal camera red reflex crescent dimensions. From each ROC curve then, the point with maximal accuracy was determined and the medium ideal instrument referral criterion, with the point to the left as specific option and point to the right as the sensitive option. Then the ideal IRC for hyperopia, astigmatism, myopia and anisometropia were collected to generate an overall refractive ROC from which ideal medium IRC could be determined for each device. Specific and Sensitive alternatives, still maintaining a high accuracy, were generated by combining their refractive subcomponent contributions.

To better assess the different cohorts, the autorefractor results were compared to the refined retinoscopy using conventional J0 and J45 cylinder vector transformations¹⁸ and spherical equivalent. In addition, each device was compared with the corresponding refined retinoscopy by the ABCD Composite ellipsoid simple continuous and graded metric;¹⁹ a perfect spectacle match scores 0.0 while mild spectacle A-Grade mis-match resulting in a 1 line acuity blur scores 1.0, a 3-line B-Grade blur scores 2.0, and a 6-line C-grade blur scores 3.0.

Sample Size and Power Estimation

Given the Ellipsoid refraction metric of 1.8 and standard deviation of 1.4, for an alpha of 0.05 and beta of 0.8, the sample size needed to determine a 0.3 difference between cohorts would be 342. For a mean astigmatism of 1.3 and standard deviation of 1.3, to detect a 0.25 difference would take a sample size of 425.

Results

Three different cohorts completed synchronous autorefraction and refined refractions. There were 616 children with PlusoptiX A12, 755 with SPOT, and 1362 with Adaptica 2WIN. For most, but not all patients, an autorefractor estimation for both eyes was obtained, but occasionally only one eye could be estimated so anisometropia was not calculated in these situations (3 for SPOT, 15 with PlusoptiX, and 86 with 2WIN).

Cohort Comparison

The mainly separate cohorts of patients taken by each of three autorefractors were compared with respect to age and refractive data (Figure 4). There were a small number of patients screened with more than one device; 444 with PlusoptiX and SPOT, 51 with PlusoptiX and 2WIN and 4 with SPOT and 2WIN, and another 46 were screened with each of the three devices. In general, the cohorts were fairly well matched. However, the cohorts for each device differed by age, and by some aspects of their refractions during their refined retinoscopy examinations (Table 1). Patients in the SPOT cohort were older and had more astigmatism. 2WIN patients were younger and had more hyperopia and anisometropia. The prevalence of exam AAPOS guideline levels and the influence on referral rate by each IRC for each device and age group are given in Table 2.

Figure 4.

Cohort exam details and ages. Compared non-parametric and means for age and refractive components for PlusoptiX, SPOT and 2WIN photorefractors.

Table 2.

Cohorts for the Three Autorefractors and Two Age Groups with Compared Prevalence of Disease and the Referral Rates Influenced by Choice of Specific, Sensitive or Medium Instrument Referral Criteria (IRC). For Each Device and Age, The Refractive Subcomponents Hyperopia, Anisometropia, Cylinder and Myopia the Medium Cut-off is Given with Sensitive and Specific Option Cut-Offs in Parenthesis.

	AAPOS 2021 ARF				Prevalence	Cohort	IRC Medium (Sensitive, Specific)				Referral Rate
AGE	Hyperopia	Anisometropia	Cylinder	Myopia			Hyperopia	Anisometropia	Cylinder	Myopia	Sensitive	Medium	Specific
<4	>4	>1.25	>3	<-3	22%	Plusoptix	3(2.5,3.5)	1.75(1.5,1.75)	3.5(3.25,3.75)	−3.5(−3.25,-3.5)	30%	24%	21%
<4	>4	>1.25	>3	<-3	30%	SPOT	1.75(1.5,2)	1.5(1.25,1.75)	3.25(3,3.5)	−3.5(−3.25,-3.75)	31%	28%	27%
<4	>4	>1.25	>3	<-3	36%	2WIN	2.5(2,2.75)	1.75(1.5,2)	3.5(3.25,3.75)	−3.5(−3.25,-3.75)	41%	33%	28%
≥4	>4	>1.25	>1.75	<-2	34%	Plusoptix	3(2.5,3.5)	1.75(1.25,1.75)	2.5(2.25,2.75)	−2.5(−2,-2.5)	41%	35%	32%
≥4	>4	>1.25	>1.75	<-2	43%	SPOT	1.75(1.5,2)	1.5(1.25,1.75)	2.25(2,2.5)	−2(−1.75,-2.5)	41%	37%	34%
≥4	>4	>1.25	>1.75	<-2	54%	2WIN	2(1.5,2.5)	1.5(1.25,1.75)	2.5(2.25,2.75)	−2(−1.75,-2.5)	51%	44%	39%

Figure 5 shows the compared receiver operating characteristic (ROC) curves for the PlusoptiX, the SPOT and the 2WIN photorefractors against the 2021 AAPOS refractive amblyopia risk factors in patients less than 4 years, and also 4 and over. Also added are the whole cohorts against the younger age-range criterion due to low numbers less than age 4 years and a comparison of 2003 AAPOS Uniform Guidelines with 2WIN because its cohort had the highest number. Table 3 and Figure 6 (the medium levels) show the actual cut-offs for instrument referral criteria of each refractive component for preschool (<4 years) and older (≥4 years) patients for each autorefractor device. Figure 7 compares the younger and older 2021 age groups for each separate autorefractor device cohort adding regions near the northwest corner of the ROC curve with “excellent”, “good”, and “fair” validation. For these high prevalence cohorts, the shift from sensitive to specific instrument referral criteria options produced adesired reduction in referral rate, as shown in Table 2.

Figure 5.

ROC curves for three photoscreeners. Infrared multi-radial computerized autorefractors Plusoptix A12, Welch Allyn SPOT and Adaptica 2WIN performance screening for 2021 AAPOS Uniform Guidelines. Arrows point to selected “medium” instrument referral criteria refractive sub-components. In the right column, the autorefractor with the largest number of subjects (2WIN) has 2003 ROC compared with that from the 2021 AAPOS Uniform Guidelines for Amblyopia Risk Factors (Anisometropia, hyperopia and high astigmatism) and also Visually Significant Refractive Errors (moderate symmetric astigmatism and myopia).

Table 3.

Instrument Referral Criteria from 3 Infrared Photoscreeners Targeting AAPOS 2021 Uniform Guidelines. SphEq is spherical equivalent in diopters.

2021 AAPOS			Hyperopia	Anisometropia	Cylinder	Myopia
Instrument	Age	Aim	Sph.Eq	Meridional		Meridional
PlusoptiX A12	<4y	specific	≥3.5	≥1.75	≥3.75	≤-3.5
		medium	>3.0	≥1.75	≥3.5	≤-3.5
		sensitive	>2.5	≥1.5	≥3.25	≤-3.25
	≥4y	specific	≥3.5	≥1.75	≥2.75	≤-2.5
		medium	>3.0	≥1.75	≥2.5	≤-2.5
		sensitive	>2.5	≥1.5	≥2.25	≤-2.0
SPOT	<4y	specific	≥2.0	≥1.75	≥3.5	≤-2.5
		medium	≥1.75	≥1.5	≥3.25	≤-2.0
		sensitive	≥1.5	≥1.25	≥3.75	≤-1.75
	≥4y	specific	≥2.0	≥1.75	≥2.5	≤-3.75
		medium	≥1.75	≥1.5	≥2.25	≤-3.5
		sensitive	≥1.5	≥1.25	≥2.0	≤-3.25
2WIN	<4y	specific	≥2.75	≥2.0	≥3.75	≤-3.75
		medium	≥2.5	≥1.75	≥3.5	≤-3.5
		sensitive	≥2.0	≥1.5	≥3.25	≤-3.25
	≥4y	specific	≥2.5	≥1.75	≥2.75	≤-2.5
		medium	≥2.0	≥1.5	≥2.5	≤-2.0
		sensitive	≥1.5	≥1.25	≥2.25	≤-1.75

Figure 6.

Medium Instrument Referral Criteria targeting their AAPOS 2021 refractive Amblyopia Risk Factors for younger and older than 4 years of age.

Figure 7.

Receiver operating characteristic (ROC) curves for separate cohorts using three autorefractive devices targeting AAPOS 2021 Uniform Guidelines Amblyopia Risk Factors and Visually Significant Refractive Errors. Regions representing “excellent”, “good”, and “fair” accuracy are delineated by blue, green, and yellow “northwest’ regions on the ROC curve.

Refractive Comparison

The degree of match with respect to spherocylinder refractive estimation of each device compared to the refined retinoscopy are listed in Table 3 and shown in Figure 8. The ABCD composite ellipsoid match significantly differed from refractive components sphere, cylinder and anisometropia (p<0.001) and also for the ellipsoid (p<0.001) by non-parametric Kruskal Wallis and ANOVA tests (Table 4 and Figure 9).

Figure 8.

Refractive comparison of three photorefractors. Plusoptix A12, the Welch Allyn SPOT and the Adaptica 2WIN infrared autorefractors are compared with cycloplegic examination in children analyzed by Bland Altmann analysis of Spherical Equivalent (M), J0 and J45 vector transformation and also the ABCD Ellipsoid unitary variable shown in Box and Whisker Plot and Column demonstrating GRADE match.

Table 4.

Compared Ellipsoid Spectacle Match of 2WIN, PlusoptiX and SPOT

	Composite Value	GRADE	p
	Kruskal–Wallis	Chi-Square
Ellipsoid	82	80	p<0.001
Sphere	39	74	p<0.001
Astigmatism	55	71	p<0.001
Anisometropia	28	27	p<0.001

Figure 9.

Ellipsoid comparison of refractive component from three infrared autorefractive devices compared to refined retinoscopy. Left shows ellipsoid composite metric (0 = perfect spectacle match, 1.0 is 1 line blue, 2.0 is 3 line defocus nd 3.0 corresponds to a 6 line defocus).

A Excel template that can assist researchers to determine sensitivity, specificity, and a receiver operating characteristic (ROC) curve specific for 2021 AAPOS uniform guidelines can be downloaded (free) from the ABCD Website (https://www.abcd-vision.org/issues/validation.html).

Discussion

For the three leading infrared devices, an evidence-based comparison of concomitant dry photorefraction with refined retinoscopy produced age-based instrument referral criteria, the referral rate of which closely matches the exam prevalence of the 2021 AAPOS Uniform Guidlelines for Instrument-based screening targeting amblyopia risk factors and visually significant refractive errors (Table 5). A large number of patients constituted each screening device validation cohort, however there were still less patients under the age of 4 years, so the 2021 AAPOS cut-off reliability of instrument referral criteria is greater for the older children than those of preschool age.

Table 5.

2021 AAPOS Instrument-Based Vision Screening Guidelines for Amblyopia, Amblyopia Risk Factors, and Visually Significant Refractive Errors

2021 AAPOS Exam Guidelines	Age < 4 Years	Age ≥ 4 Years
“Amblyopia”	Best Va 20/40 or worse or 2-line inter-eye difference
Amblyopia Risk Factors (ARFs)
Deprivation (ie, cataract)	>1 mm axial
Strabismus	>8 PD manifest
Anisometropia	>1.25 D
Hyperopia	> 4.0 D
Cylinder	> 3.0 D
Visually Significant Refractive Errors (VSREs)
Cylinder		>1.75 D
Myopia	< −3.0D	< −2.0 D

Cohort comparison: the cohort screened with SPOT had more hispanic ethnicity which has been shown to have increased astigmatism consistent with our findings. The increased anisometropia patients screened with 2WIN represented an interpretation challenge in part because the 2WIN employed an accommodation stimulating, bright flashing fixation light that resulted in under-estimation of hyperopia compared to cycloplegic examination. The patients screened with Plusoptix had more inconclusives due to refractive error outside the interpretive range of the device; on the other hand PlusoptiX was very precise in estimation of astigmatism magnitude and axis.

This manuscript covers the three primary infrared, multi-radial photorefractors PlusoptiX, SPOT, and 2WIN. It does not specifically address nuances of the related SW-800.²⁰ This document does not cover the visible-light iScreen or GoCheck Kids^21,22 that report ranges of amblyopia risk factors, but not in complete sphero-cylinder format. Also not covered by this report are the Safir-method, monocular autorefractor Retinomax^23,24 and the birefringent scanner Rebion blinq.²⁵ Vision screening devices that target aspects other than pure refraction can still be well validated with the 2021 AAPOS guidelines emphasizing subcomponents, such as manifest strabismus and amblyopia for the “Rebion blinq”.

With cooperative, fixing patients who have wide-open eyelids and a correct low-luminance environment, each of these three infrared autorefractors promptly (2–3 seconds) gives a refraction estimate with pupil size, and estimate of ocular alignment. However, in a distracting environment with stray infrared light, with narrow eyelids with eyelashes partly covering their pupils the instrument refractive estimate can be substantially delayed or even not possible ending with an “inconclusive” screen. In patients with high refractive error, the devices, particularly the PlusoptiX will delay or not yield a refractive estimate sometimes delivering “HYP” or “MYO” for perceived, but not quantitated high hyperopia or myopia respectively. These eccentric photoscreeners utilize a crescent of bright light at the edge of the otherwise pupillary red-reflex (leukocoria with ICD10 H44.533) to determine refractive error in the corresponding axis, so small pupils, near plano refraction at the 1-meter focusing distance, or out-of-range excessive refractive error can render refractive estimate unreliable or impossible. To the infrared photoscreener, a +10.00 patient and a −10.00 patient with medium sized pupils can have nearly identical uniform pupil coloration. It is important to note that our ROC curve analysis is generated from each instrument’s sphero-cylinder notation after processing of the multi-radial red reflex crescent, and not from a re-analysis of the raw data from red reflex dimensions.

Different Cohorts

From a large meta-analysis, Plusoptix and SPOT had very similar diagnostic accuracy.²⁶ If instrument referral criteria can be set specifically, then PPV from photorefraction is favorable to pediatrician, parent and pediatric ophthalmologist alike, but age-based serial screening methods with more sensitivity (patched monocular acuity) should follow later in the child’s first decade. In preschoolers, SPOT photoscreen detected 70% of amblyopia risk factors but visual acuity screening, with sensitivity 60% found some additional cases that SPOT missed.²⁷

A strength of this study is the wide variety of race and ethnicity. Ocular pigmentation has an impact on infrared photoscreening. Eyelid opening matters; it is more difficult to attain quick, reliable interpretations if the lids and eyelashes cover part of the pupil.

A weakness of this study is that many members of each cohort were screened with only one device with only a minority having been screened by all three devices, therefore direct comparison of the performance of each device cannot be definitively determined. In addition, not every option for fixation target was utilized; we chose the more interesting and accommodation stimulating flashing colored light fixation on the 2WIN which may have diminished the ability of the dry screening by that device to detect higher cycloplegic hyperopia. We did not exhaust every method of addressing the inconclusive refractive interpretations such as ABCD sensitivity and specificity. In addition, we did not have access to the internal instrument estimation paradigms that translate red reflex crescents into estimated sphero-cylinder refractions; the nuances of even better optimized ROC performance will be the responsibility of each manufacturer (de-identified data provided).

Another strength of this paper is the consistent use of format for hyperopia and myopia consistent with AAPOS 2021 Uniform guidelines. For a patient with asymmetric astigmatism, there can be several ways to express the magnitude of spherocylinder refractive components (Figure 3). For astigmatism written in either plus-cylinder or minus-cylinder format, the spherical equivalent is identical. Meridional quantification adds to spherical equivalent ½ of cylinder magnitude for maximal hyperopia while maximal magnitude (minimal diopter value) for myopia is spherical equivalent minus ½ the absolute value of cylinder. One definition of anisometropia could be as simple as the absolute value of the difference between right and left spherical equivalent. On the other hand, meridonal anisometropia could be several values (Figure 3); we recognize the meridional anisometropia as the absolute difference between meridional myopia right and left eyes, which is the difference in the sphere when written in plus cylinder format. Experience from interpreting thousands of MTI photoscreens with corresponding follow up exams²⁸ revealed that the hyperopic astigmatic patient typically foccuses at this point on the conoid of Sturm recruiting additional accommodative effort only when target image features call for it.

Hyperopia and accommodation are critical issues in photorefraction. The photoscreener captures the eye at its natural balance between full cycloplegic hyperopia and full or even excessive accommodation in one or another meridian. Many photoscreening validation efforts document that dry photorefractors under-estimate the degree of cycloplegic hyperopia in children.^29–32 Indeed, the prevalence of cycloplegic hyperopia far exceeds the prevalence of amblyopia due to hyperopia.³³ Certain children deal with symmetric higher (ie +4.50) hyperopia differently (Table 6); one will accommodate and show an esotropia, strabismic amblyopia in one eye, a second (like President Harry Truman) will keep the eyes straight, fail to accommodate and develop symmetric bilateral ammetropic amblyopia and the third has sufficient accommodation and fusional reserve to accommodate clearly most of the time with straight eyes developing good visual acuity in both eyes, stereopsis and often, a relative aversion to spectacles.

Table 6.

Responses of Different +4.50 D Hyperopic Children to Photorefract

Compensation?	Alignment	Pupil Size	Amblyopia?	Stereo?	Photoscreen
Accommodative Esotropia	25 PD Left ET	Small	20/20 and 20/80	No	Inconclusive- misaligned, mild hyperopia right eye
Bilateral Ammetropic	Ortho	Large	20/32 and 20/40	Yes	Refer bilateral HYPEROPIA
Fusing Accommodator	Ortho or esophoria	Small	No	yes	Pass (underestimates cycloplegic hyperopia)

The type of fixation target on the screening device influences how hyperopia will be detected.³⁴ A boring, low detail target will not stimulate natural accommodation and therefore more cycloplegic hyperopia will be uncovered by the device. On the other hand, a detailed, interesting fixation target – similar to day-to-day non-daydreaming viewing – will allow the Fusing-Accommodator to obtain nearly full accommodation and the screening device will seem to underestimate cycloplegic refraction.

Refractive Comparison

The PlusoptiX, 2WIN, and Topcon KR-8900 table top autorefractor were compared showing better correlation between sphere and cylinder than J45 vector component in 100 patients.³⁵ The SPOT outperformed Plusoptix with respect to specificity in highly astigmatic children.³⁶

Inconclusive screenings need to be addressed in vision screen validation. Inconclusives can result from various different scenarios: child did not participate or even look at the screener, child did not understand the screening process and participation, child was physically unable to perform required tasks for the screening, lack of sufficient attention (especially in autistic children), child willingly or inadvertantly provided false or incorrect information (readthe wrong optotypes or crossed eyes for photoscreener). The instrument was unable to yield a result due to battery outage, computer error, improperly followed methods, screening finding out of range for instrument, screening environment was not appropriate for screening (too bright, stray infrared light, too may distractions, wrong temperature). The 2021 AAPOS guidelines present a 3×3 validation grid rather than just the classic 2×2 grid. In addition to screen “refer” and screen “pass” a screen “I” is added for any inconclusive screening interpretation. In addition to exam “True” risk factors and exam “lack of risk factors” AAPOS 2021 adds “lost to follow up”. If the classic 2×2 grid cells can be represented by refer-positive (True positive) = A, refer-false (false negative) = B, pass-exam True (false positive) = C and pass-exam normal (true negative)= D, then the 3×3 grid letters have extra E through I.

If the photoscreening device lags and takes a long time to make a refractive estimate, it would be helpful to signify that the result is a likely “refer”. This is also true for an interpretation that is outside routine bounds for hyperopia and myopia. The angle of strabismus constituting a referral should be adjustable for the screening clinic that aims to reduce false positive referrals due to children that do not easily pay attention to the fixation target,³⁷ or increase sensitivity for those clinics who prefer to examine all those referrals.³⁸

If explicitly mentioned, it is OK to exclude inconclusives and “lost-to-follow-ups” and report just classic validation statistics; therefore report all findings. However, it is better to account for the inconclusives by mentioning if they are counted as “refer” or as “pass”. Other combinations of inconclusives and “lost-to-follow-up” have been previously published.^39–41

The 2021 AAPOS guidelines utilize meridional hyperopia and myopia. We found that meridional myopic values from the autorefractors best matched exam myopia and anisometropia, however spherical equivalent autorefractor best matched exam meridional hyperopia. This may be due to the patient’s residual ability for age-related accommodation and adoption of the circle of least confusion rather than the extreme hyperopic end of the Conoid of Sturm for many patients.

The age at screening matters; the American Academy of Pediatrics (AAP) recommends a series of age-appropriate screening methods from newborn, through the first decade of life when amblyopia is most treatable.⁴² The MEPEDS study found that monocular visional acuity results are completed by far less children before the age of 4 years.⁴³ Therefore, the 2021 Uniform Guidelines separate instrument-based screening so that specific, instrument amblyopia detection is prioritized with low false positive referrals before age 4, and sensitive, monocular visual acuity screening, or instrument-based screening for significant refractive errors are emphasized in children ready for school.

From our determinations of a sensitive and a specific alternative IRC, we noted a decrease in referral rate by 13–32% for the less than 4 year old preschool cohort devices and by 18–24% for the devices for children 4 years and older. A busy pediatric ophthalmologist who notes high false positive rate with a device with sensitive IRC can expect to decrease referrals about ¼ moving to the specific IRC by the referring screeners. Such an alternative is not “too risky” as long as continuous age-based vision screening is followed for these children.⁴⁴

For some children, the individual performing the instrument-based screening might have just a few seconds to correctly refer a developmentally delayed child for curative therapy, or miss them due to delayed, distracted or deficient elicitation of the child’s fixation and attention. An autistic child may view objects and people in the room to determine whether or not they could bring harm; once assessed as “safe”, that autistic child is unlikely to fix on the vision screening device again. As such, preparations before the actual screening can improve the chance of capturing the elusive, sufficiently long fixation with eyelids open. One classic reason for failing to objectively screen an autistic child is holding the device in their view and enter demographic data into the machine before aiming and triggering the image capture.

Conclusion

Evidence-based instrument referral criteria derived from ROC curves with specific and sensitive alternatives for preschoolers (<4y) and students (age ≥4y) empower the 2021 AAPOS Uniform guidelines to target amblyopia in children for early successful therapy and also give students spectacles at an early stage to enhance learning.

Funding Statement

There is no funding to report.

Data Sharing Statement

https://www.abcd-vision.org/references/IRC%20AAPOS-2021%202%20P%20S.pdf.

Disclosure

Dr. Arnold is President of Glacier Medical software that markets cloud-based ROP-Check software. Dr. Arnold is President of PDI Check that developed a vision screening game for Nintendo 3DS. Dr Arnold is an investigator and protocol developer for PEDIG. Dr. Arnold coordinates the Alaska Blind Child Discovery which has received discounted vision screen technology from several vendors. He is a non-paid member of advisory boards to PlusoptiX, GoCheck Kids, Adaptica, NovaSight, and iScreen. Robert Arnold also reports patent pending for autostereoscopic vision screening game to PDI check. Dr. Silbert is Founder of KinderSee a school-based eye clinic serving the children of Lancaster County PA, he serves on the Medical Advisory Board and holds stock options in GoCheck Kids. He is a consultant for Kaneka America and a speaker for Horizon Therapeutics. He is a co-founder and co-owner for remote amblyopia monitoring and treatment at Amblyopia Home. Heather Modjesky reports no conflicts of interest in this work.