Validation of an Automated ETDRS Low-Contrast Letter Acuity Test

Abstract

Significance.

Automated low-contrast letter acuity (LCLA) has several advantages: consistent luminance, reduced chance of individuals memorizing test letters, and convenient and accurate VA reporting functions. Although automated LCLA might report slightly worse acuity than Sloan LCLA chart, considering its advantages, it may be a viable alternative to Sloan LCLA chart in clinical practice and research.

Purpose.

To determine repeatability of an automated LCLA measurement and its agreement with the Sloan LCLA chart test, in normal participants and reduced-vision participants.

Methods.

Adult participants (N=49) were measured with both automated ETDRS and Sloan LCLA tests, including normal and reduced-vision groups. LCLA at two contrast levels (2.5% and 10%) was measured at 3 meters in a random sequence with both LCLA tests. To test repeatability, participants were retested one week later. Repeatability of the two tests between two visits and agreement between automated and Sloan LCLA tests were evaluated using 95% limits of agreement (LoA).

Results.

In terms of the 95% LoA, the repeatability of both tests was: automated LCLA@2.5% = ±0.26, automated LCLA@10% = ±0.22, Sloan LCLA@2.5% = ±0.23, Sloan LCLA@10% = ±0.16; agreement of the two tests was: ±0.19@2.5%, ±0.24 @10%. The automated LCLA at 2.5% and 10% levels was generally reported one-half to one logMAR line lower than Sloan LCLA (mean differences= −0.04@2.5% and −0.13@ 10%, paired T-test, P<.05).

Conclusions.

The automated LCLA test shows fairly good test-retest repeatability at both 2.5% and 10% contrast levels. The agreement between the automated and the Sloan low-contrast letter acuity tests was comparable to test-retest agreement. Although the automated LCLA test reports slightly worse acuity than the Sloan LCLA test, it could be an appropriate alternative to the Sloan LCLA test.

Low-contrast letter acuity, as a visual function test, has been identified as an important component in the profile of many types of patients.^1–4 For instance, low-contrast letter acuity is widely used as a benchmark of visual dysfunction for patients with multiple sclerosis.¹ According to an increasing body of evidence, in these patients low-contrast letter acuity is a physiologically meaningful test because decreased low-contrast letter acuity scores have been correlated with retinal thinning in OCT imaging, the MRI lesion volume, and reduced responses in multifocal electroretinography.¹ Furthermore, low-contrast letter acuity is more sensitive to some diseases than is high contrast visual acuity.⁵ Reduced low-contrast letter acuity and vision-specific quality of life are evident many years following acute optic neuritis even when high contrast visual acuity has recovered.⁵ The Sloan low-contrast letter acuity chart test (Precision Vision, LaSalle, IL) is the gold standard to measure low-contrast letter acuity, which has excellent inter-rater reliability, with high intra-class correlation across all contrast levels.⁶

With recent advances in technology, computerized tests have been used to measure visual function. If test distance, luminance and contrast of the test screen are carefully calibrated and external glare is limited, computerized equipment can generate similar results compared to gold standard charts. Several commercially available computer-based contrast sensitivity tests have become available. The Freiburg Visual Acuity and Contrast Test is widely used.⁷ Kollbaum et al. validated an iPad test of letter contrast sensitivity; however, this test did not decrease letter size to measure low-contrast letter acuity.⁸ Unfortunately, there have been few studies to validate repeatability of computerized low-contrast letter acuity tests. Implementation of a low-contrast letter acuity test on an electronic platform has several advantages: 1) specific contrast levels; 2) randomization of target presentation; 3) automatic scoring for printing, which facilitates documentation and communication, thereby saving precious clinic time.

The commercially available Automated Low-contrast Letter Acuity Test (M&S Technologies, Inc., Niles, IL) is one of the new computerized tests. To overcome the common issue of variable monitor screen brightness in computerized tests, this automated test includes self-calibration of luminance. System calibration of this automated test is set in meters at virtually any distance and is adjustable and precise to within 1 cm. The system is calibrated for both distance-to-participant and pixels-per inch so that optotypes precisely follow ANSI Z80.21–2010 (R2015) and SO 8596:2009 regarding size, spacing between optotypes, and spacing between lines. Background luminance is accurately calibrated to 85 candela/m² for standardized ANSI and ISO testing (M&S Technologies, http://www.mstech-eyes.com/products/detail/automated-etdrs-defocus-curve). Letter contrast system luminance can be automatically calibrated and ambient room conditions measured with the optional integrated luminance meter.

Sloan low-contrast letter acuity charts have a standardized format based on Early Treatment Diabetic Retinopathy Study (ETDRS) visual acuity charts, which is often the standard used in clinical trials, thus it was chosen as the gold standard low-contrast letter acuity test for comparison. The purpose of this study was to determine the repeatability of an ETDRS automated low-contrast letter acuity test on a tablet-computer system and its agreement with the gold standard chart-based test, i.e. Sloan low-contrast letter acuity chart, in normally sighted participants and reduced-vision participants.

METHODS

This research protocol and the informed consent form were approved by the institutional review board of Illinois College of Optometry, Chicago, IL. Compliance with the Health Insurance Portability and Accountability Act was maintained during this study.

Participants

We enrolled adult participants from the greater Chicago area. Demographic characteristics of the participants are shown in Table 1. Informed consent was obtained from all participants. Best-corrected visual acuity was ranged from 20/25 to 20/100 each eye in the participants.

Table 1.

Demographic characteristics of the participants (N = 49).

	Number of Participants (%)
Visual Acuity
20/25 or better	32 (65.3)
20/30 to 20/100	17 (34.7)
Gender
Female	41 (83.7)
Male	8 (16.3)
Race
Black	20 (40.8)
Hispanic	8 (16.3)
White	16 (32.7)
Asian	5 (10.2)
Age (years)
Range	22.6 – 91.1
Mean (SD)	46.7 (17.5)

Low-contrast Letter Acuity Tests

To validate the commercially available automated low-contrast letter acuity test, we used the Sloan low-contrast letter acuity chart (low-contrast Sloan letter chart, Precision Vision, LaSalle, IL) as the gold standard. Because 10% contrast is considered the level to be used as adaptation and 2.5% is more sensitive to visual deficits, we chose these two contrast levels for repeatability tests. A contrast of 100% is the level of high contrast visual acuity.

Automated Low-contrast Letter Acuity Test

The system includes a laptop computer with a high-resolution 13” display and a wireless control tablet for the examiner. (Figure 1) We used the built-in automated contrast sensitivity function system. Fixed contrast with decreasing size system was used to estimate low-contrast letter acuity at 2.5% and 10% contrast levels. The test included 10 Sloan letters. The computer screen was auto-calibrated to the luminance level of 85 cd/m² with a photometer for all tests. To determine the endpoint of automated low-contrast letter acuity, two phases were used. 1) Phase I, to determine initial threshold: An ETDRS chart (from 20/200 to 20/10) at either 10% or 2.5% contrast level was displayed on the computer screen, and participants were instructed to read the smallest line in which they could read all 5 letters. The examiner submitted the lowest visual acuity level at which participants read all letters correctly. 2) Phase II, to determine threshold: ETDRS letters at the visual acuity level that examiner submitted were displayed on the computer screen as well as the remaining smaller letters of the chart; in this phase, participants were tested with a change of 0.1 logMAR acuity and a termination rule of 5 mistakes. The purpose of this procedure was to ensure a relatively efficient (thus participants were less fatigued), and at the same time a more accurate result.

Figure 1.

Automated ETDRS low contrast letter acuity measurement viewed by the participants (A) Phase I; (B) Phase II with a blue dot) and the examiner (C) Phase I: (D) Phase II).

A blue dot (Figure 1B) was shown next to the visual acuity level one line below the submitted visual acuity size and participants were instructed to read the line next to the blue dot. The examiner submitted the correct number of letters participants read and then continued to instruct participants to read letters with decreasing size, in 0.1 logMAR steps. The test stopped when the participants were unable to read any letters correctly or when no smaller lines were available to be tested. The system automatically calculated the logMAR visual acuity using the correct letters that participants read. The endpoint of five total mistakes has been used in previous studies with visual acuity outcomes.^9–11

After completion of the measurement, the test outcomes were displayed on the computer with the following parameters: which eye, test distance, visual acuity letter score, logMAR visual acuity, and Snellen visual acuity equivalent.

Sloan Low-contrast Letter Acuity Charts

The charts measure 14 × 14 inches and were mounted on a retro-illuminated cabinet. Sloan low-contrast letter acuity charts were standardized according to the ETDRS visual acuity charts with five letters per line. Sloan low-contrast letter acuity per contrast level is given as number of letters identified correctly (maximum of 60 letters).

Procedures

All tests were administered monocularly and at 3 meters in the same room with habitual refractive correction. Two contrast levels (2.5% and 10%) of one eye from each participant were measured in a random sequence with the two low-contrast letter acuity tests. A random number generator in Excel was utilized to provide the random test sequence of the automated and Sloan measurements. Twenty-four participants were measured with the automated tests first, then the Sloan tests. The remaining 25 participants were measured with the Sloan tests followed by the automated tests. The same random sequence for each participant was used for both test-retest measurements. After 1 week (±3 days), all participants were retested with the same procedure, with the exception of four participants who did not return.

Power Calculation

Prior data indicate that the difference in the response of matched pairs is normally distributed with standard deviation 0.2. We were planning a study with 35 participants. If the true difference in the mean response of matched pairs was 0.1, we would be able to reject the null hypothesis that this response difference was zero with power of 0.82. Type I error probability associated with this test was 0.05. We collected data from a total of 49 participants for comparison of the automated and Sloan tests; for repeatability, we collected data from 45 participants.

Statistical Analysis

The repeatability between two administrations of the automated and Sloan low-contrast letter acuity tests as well as agreement between the two low-contrast letter acuity tests in both groups were evaluated using the 95% limits of agreement (LoA), which corresponds to ±1.96 SD of the differences between administrations or tests. The difference between the scores for each administration or test was calculated for each participant. The distribution of these differences was analyzed by calculating the mean, SD, and the 95% LoA. The breadth of these LoA indicates the repeatability of the test. The narrower the LoA, the more repeatable the test.¹² A paired T-test was used to compare test and retest in both groups at both 2.5% and 10% contrast levels. Bonferroni correction was applied because low-contrast letter acuity was tested at two contrast levels, thus P < .025 indicates statistical significance.

RESULTS

Forty-nine participants across a range of acuities were tested. For retest, 4 participants did not return. Table 1 shows the basic characteristics of these participants. Note that 10 participants with poor high contrast visual acuity were not able to identify 2.5% contrast for both tests.

Table 2 shows results from all participants. In terms of the 95% LoA, the repeatability of both tests were: automated low-contrast letter acuity @2.5% = ±0.26, automated low-contrast letter acuity @10% = ±0.22, Sloan low-contrast letter acuity @2.5% = ±0.23, Sloan low-contrast letter acuity @10% = ±0.16.

Table 2.

Mean (±SD) scores for the first and second administration of each low-contrast letter acuity (LCLA) test in all participants. The mean difference and 95% limits of agreement (LoA) are also shown. Paired T-test was used to compare two administrations of each test.

	Repeatability: All Participants in both groups
	2.5% Contrast Level (N=35)					10% Contrast Level (N=45)
	Test	Retest	Mean difference	T-test & P-value	95% LoA	Test	Retest	Mean difference	T-test & P-value	95% LoA
Automated	0.74 (0.19)	0.74 (0.20)	−0. 002 (0.13)	T=−0.10; P=.92	±0.26	0.51 (0.34)	0.49 (0.35)	0.02 (0.11)	T=1.23; P=.22	±0.22
Sloan	0.69 (0.15)	0.65 (0.14)	0.04 (0.12)	T=2.27; P=.03	±0.23	0.38 (0.30)	0.38 (0.30)	0.01 (0.08)	T=0.64; P=.52	±0.16

1. Repeatability of automated low-contrast letter acuity test (Fig 2. A and B): At both 2.5% and 10% contrast level, the automated low-contrast letter acuity of the retest was not statistically different from that of the first administration (Table 2).

2. Repeatability of the Sloan low-contrast letter acuity test (Fig 2. C and D): At both 2.5% and 10% contrast level, the Sloan low-contrast letter acuity of the retest was not statistically different from that of the first administration (Table 2).

3. Agreement of the automated and Sloan low-contrast letter acuity tests: Agreement between the automated low-contrast letter acuity and Sloan low-contrast letter acuity at 2.5% and 10% levels is shown in Figure 3 (A and B).

Figure 2.

Repeatability of the Sloan low-contrast letter acuity (Sloan-LCLA) test and the automated low-contrast letter acuity (automated-LCLA) test is shown in Bland-Altman plots at 2.5% and 10% contrast levels. The LCLA difference between the second and the first administration (second minus first) of each test is plotted against the mean of two tests. (A) Automated-LCLA at 2.5%. (B) Automated-LCLA at 10%. (C) Sloan-LCLA at 2.5%. (D) Sloan-LCLA at 10%. From the top to the middle and the bottom, three lines show the upper 95% LoA, bias, the lower 95% LoA, respectively.

Figure 3.

Bland-Altman plots to demonstrate agreement between Sloan low-contrast letter acuity (Sloan-LCLA) and the automated low-contrast letter acuity (automated-LCLA) at 2.5% and 10% levels (A & B). The difference between the average scores (i.e. Sloan-LCLA minus automated-LCLA) for the two tests is plotted against the mean of two tests. From the top to the middle and the bottom, three lines show the upper 95% LoA, the mean difference, the lower 95% LoA, respectively.

The mean (±SD) differences between the automated and Sloan low-contrast letter acuity at 2.5% and 10% contrast levels were −0.04 (± 0.10) logMAR and −0.13 (± 0.12) respectively with statistical significance in the all participants (paired T-test, t=−2.19, P= .005; t=−7.06, P <.001). In terms of the 95% LoA, the agreement between two low-contrast letter acuity tests was: ±0.19 at 2.5% contrast level and ±0.24 at 10% contrast level.

Specifically, Figure 3 shows more data were below zero at both contrast levels, which indicates that the automated low-contrast letter acuity reported one half to one logMAR line higher values (worse acuity) than the Sloan low-contrast letter acuity.

DISCUSSION

The automated low-contrast letter acuity test showed fairly good test-retest repeatability in adult participants across the range of acuities at both 2.5% and 10% contrast levels. In addition, the agreement between the automated and the Sloan low-contrast letter acuity tests was similar to test-retest agreement.

According to the 95% LoA range of repeatability, there was one outlier data point on both the Sloan and automated tests at 2.5% level, and it was from a 42-year-old participant with high-contrast visual acuity of 20/15; the large difference between test and retest for this participant may be due to his lack of engagement with contrast tests at the second visit. The Sloan low-contrast visual acuity score of the retest was slightly but significantly better than the first test, which might represent a potential learning effect and improved familiarity with the Sloan low-contrast letter acuity chart, but not with the automated low-contrast letter acuity system.

According to the 95% LoA, the agreement between automated and Sloan low-contrast letter acuity charts was comparable to the test-retest repeatability of the automated and Sloan low-contrast letter acuity charts, which indicates that the agreement between automated and Sloan low-contrast letter acuity charts was as good as could be expected based on test-retest repeatability. There was one outlier; this participant had 20/100+ high-contrast visual acuity. Previous studies have reported that vision tests in reduced-vision participants are likely to be less repeatable than in individuals with normal vision.^8,13,14 Compared with the Sloan test, the automated low-contrast letter acuity test in this study often reported worse acuity at both 2.5% and 10% contrast levels. One factor could be the luminance of the test. The automated low-contrast letter acuity was always calibrated at 85 cd/m². On the other hand, the built-in luminance of the Sloan test was often higher than 85 cd/m²; we found it averaged 108 to 128 cd/m². Unfortunately, we were not able to adjust luminance of the Sloan test because of the built-in system. It is unclear why the built-in luminance of the Sloan test has larger variance. Our findings are opposite to the contrast sensitivity test results of Kollbaum et al.⁸ In their study, both iPad and Freiburg computerized tests yielded better contrast sensitivity function than did the Pelli-Robson test.⁸ The difference may be due to the nature of low-contrast letter acuity tests, with different study conditions and different tests.

Comparing low-contrast letter acuity with ordinary low-contrast chart testing, LoA in our study were similar to those in previous studies. Kollbaum et al. investigated 20 normal-vision and 20 low-vision participants; their LoA in normal-vision participants were ±0.19, ±0.19, and ±0.15 for iPad, Pelli-Robson, and Freiburg respectively.⁸ They reported LoA for low-vision participants as ±0.24, ±0.23, and ±0.21 respectively for the three tests.⁸ Dougherty et al. studied 37 participants and reported LoA were ±0.20 for the Mars test and ±0.20 for the Pelli-Robson test.¹⁵ Balcer et al. measured the Sloan letter acuity at contrast levels of 100%, 5%, 1.25% and 0.6% on individuals with normal visual acuity as well as individuals with multiple sclerosis.⁶ They reported the inter-rater agreement ICC between 0.86 and 0.95 at each contrast level; however, they did not study LoA.⁶

We have previously reported that individuals with amblyopia associated with myopic anisometropia had clinically and statistically significantly reduced contrast sensitivity at the middle and higher spatial frequencies.¹⁶ Although a contrast sensitivity function test can thoroughly measure individuals’ contrast sensitivity, the length of the procedure poses an obstacle for application in routine clinical care. Meanwhile, low-contrast acuity not only detects vision loss which could be missed by high-contrast visual acuity measurement, but also measures contrast sensitivity because the decrease in letter size incorporates testing of different spatial frequencies. In addition, the automated low-contrast letter acuity is a much easier and quicker test compared to contrast sensitivity function measurement; thus automated low-contrast letter acuity may result in broader application in both clinical care and clinical trials.

Limitations

1) Participants with poorer high contrast visual acuity (20/30 to 20/100) had difficulty reading 2.5% contrast level (measurement range: 20/10 to 20/200) such that fewer participants could provide the test scores. Therefore, caution must be used in generalizing our findings to individuals whose high contrast visual acuity is worse than 20/100. 2) In some studies, low-contrast letter acuity tests were conducted binocularly (both eyes open) as this approach integrates possibly relevant binocular summation/inhibition effects¹⁷, providing a measure of overall visual function closer to the “real life” situation than does monocular testing. Our study tested low-contrast letter acuity monocularly. A future study is warranted to measure repeatability of binocular low-contrast letter acuity. 3) Age relationship with tests was not investigated in this study due to the limited number of participants in different age categories. Although this sample had no significant difference in age, the older participants often had poorer vision.

Significance

The automated low-contrast letter acuity has several advantages: consistent luminance, reduced chance of individuals memorizing test letters, and convenient and accurate visual acuity reporting functions. Although the automated low-contrast letter acuity might report a higher logMAR value (worse acuity) than the Sloan chart, considering its advantages, the automated test may be a viable alternative to the Sloan low-contrast letter acuity chart in both clinical practice and research. Low-contrast letter acuity, being potentially more sensitive than high-contrast visual acuity, is not very commonly used in clinical practice. It has been recommended that clinical trials adopt a low-contrast letter acuity test as an outcome measure.^1–4 Thus, an automated low-contrast letter acuity test may facilitate low-contrast letter acuity application.

CONCLUSIONS

The automated low-contrast letter acuity test showed fairly good test-retest repeatability in participants at both 2.5% and 10% contrast levels. In addition, the agreement between the automated and the Sloan low-contrast letter acuity tests was comparable to test-retest agreement. Although the automated low-contrast letter acuity test reported slightly worse acuity than the Sloan low-contrast letter acuity test, it could be an appropriate alternative to the Sloan low-contrast letter acuity test.

Table 3.

Mean (±SD) scores for the two kinds of low-contrast letter acuity (LCLA) tests in all participants. The mean difference and 95% limits of agreement (LoA) are also shown. Paired T-test was used to compare two tests.

	Agreement between two tests in the all participants
	2.5% Contrast Level (N=39)				10% Contrast Level (N=49)
Automated	Sloan	Mean difference	T-test & P-value	95% LoA	Automated	Sloan	Mean difference	T-test & P-value	95% LoA
0.75 (0.20)	0.71 (0.17)	−0. 04 (0.10)	T= −2.195; P=.005*	±0.19	0.51 (0.33)	0.38 (0.30)	−0.13 (0.12)	T= −7.06; P<.001*	±0.24

^*indicates statistical significance.